"DON'T MENTION ROPE IN THE HOUSE OF A HANGED MAN"

As you know, at the end of 2012 the well-known international organization Transparency Inter-national (TI) published its Corruption Perceptions Index (CPI) again. It ranks countries according to perceptions of the prevalence of this "sore" and is made on the basis of many studies. The country with the lowest rating, says TI, is the most corrupt not in the world, but only among those that got into its index. We will not dwell on the analysis of the ratings themselves as it is more interesting to study the very methodology of TI.

Since 1995, when the CPI was released for the first time, TI has consistently stressed that it is using "excellent data that do not contain distortions and have been analyzed with the most reliable methods" and is trying to achieve maximum accuracy, perfecting its techniques. CPI-2012 also contains a lot of innovation. This is at least the third radical transformation of tools in the past 17 years. Actually, this could be seen as commendable commitment to perfection, if it were not for a few significant "buts". But let's not get ahead of ourselves and see how the CPI is compiled.

Initially, TI used data from various opinion polls and expert assessments collected in three years. However, a strange tradition was almost immediately founded. The list of countries included in the CPI each year changed for some reason, which did not allow us to see how the situation looks in the country compared to other countries. In addition, TI widely probed public opinion, although it caused doubts from the very start. They could not know there that a "man on the street" can make judgments only about domestic corruption. He does not have access to information about what is happening in the higher echelons of power, in the parliament or in parties. TI began to see it clearly only in 2004. It was decided to focus the CPI only on expert assessments, because, I quote the press release, "representatives of business are much better aware of corruption in the ruling top than ordinary citizens. The majority of people have an idea only about petty corruption."

But if you think that TI has actually stopped probing public opinion, you are wrong. They have created another "device" - Global Corruption Barometer (GCB) based on population surveys and related to perceptions of and encounters with corruption at different levels. Interestingly, the CPI-2007 press release emphasizes that the Global Corruption Barometer "is aimed at assessing citizens' attitude to corruption and encounters with it in everyday life". But TI misleads us. For the same GCB-2007, for example, citizens were asked about corruption in areas where they are not competent: in parties, parliament and NGOs. It seems that TI operates on the principle "if you cannot get something, but really want it - you can get it".

Another example: In 2002, the 60-billion-dollar energy colossus Enron filed for bankruptcy in the United States. But it's not just the scale of the event. As it turned out, Arthur Andersen - the auditor of the bankrupt, which is one of the world's top five companies, instead of warning shareholders about the financial risk, advised Enron managers to hide their losses! Did it affect the US ranking of corruption? According to TI, in 2002 it was 7.7 points, and the next year it fell only to 7.5. Azerbaijan's rating dropped by the same 0.2 points and, but what substantial event happened in our country in that sense? Nothing. Why is the CPI so biased: because of flaws in methodology or servility with which it was compiled?

As TI declared without a shadow of a doubt in the mid-1990s, the CPI "is a snapshot of the situation and gives an idea of the current momentary assessments of the level of corruption, not focusing on annual changes". But what kind of snapshot can we talk about if the CPI is based on averages over a period of three years? And what are these "momentary assessments" which ignore annual changes? Agree, this is some kind of an oxymoron. (By the way, for some reason, TI has thoroughly "cleaned up" its archive: many explanations of methodology have disappeared from old press releases, leaving only tedious tables).

Sometimes, however, TI recognizes its flaws. Quote: "In the past, expert assessments for the CPI sources were often provided by businessmen from... developed countries. The point of view of experts from developing countries was represented ... to significantly lesser extent." But such confessions are an exception. The rule is to over-emphasize the alleged or actual advantages of its methodology and refuse to discuss controversial issues.

Note that TI has been criticized in the past too. Back in 2002, in the article "The methodological error or computational racism", analysts D. Gorbatov and V. Baranov expressed doubts about the quality of TI information. But, alas, with the improvement of the methodology and the increasing number of CPIs, doubts only increased. As the saying goes, "the farther into the woods, the thicker the guerrillas".

"Significant changes in the level of corruption are slow," says TI. But what is the point in the annual CPI then? And if annual changes in the CPI cannot be significant, then why, for example, did Russia, which "earned" 2.3 points in 2001, "jump" by 12 per cent the next year and gain 2.7 points? Pakistan also made a leap, moving from the level of 2.3 to 2.6 points (a difference of 11 per cent). As soon the "Orange Revolution" in Ukraine and the "Rose Revolution" in Georgia happened, their rankings improved the following year, though, TI said, the CPI reacts poorly to annual changes. But TI prefers not to talk about a measurement error.

It is known that the CPI and GCB are based not on the actual situation, but on people's notions. But a direct link between these parameters is not required. This was confirmed by Latinobarometro - a survey of 17 countries in Latin America conducted in 2000. The head of the Russian branch of TI, Y. Panfilova, said at a meeting of the so-called Anti-Corruption Network (Istanbul, 2003) that Mexicans considered corruption a "very serious" problem. But this opinion is shared by Chile, although the level of corruption is much lower there. That is to say corruption may be small and be perceived as large and vice versa. A similar pattern was registered in another continent. Afrobarometer showed that the personal experience of corruption in these countries tends to be lower than levels of its perception, and corruption is assessed differently by respondents among the most important problems. In other words, TI is aware that ethnic and cultural factors influence the understanding of corruption, but respond evasively: the CPI is "based on a set of ideas not distorted by cultural prejudice". That is to say the accuracy and reliability of research instruments remain a hidden "smoke screen" for incomprehensible explanations and excuses.

What is more, when the CPI is compiled, as is known, figures obtained from surveys using different methodologies are put together. And this, of course, reduces the error (in one case, they ask how often one has to pay bribes, in another - they assess the level of corruption and in the third case - the effectiveness of the fight against it). The subject is one, but in fact, what they measure is not the same thing, as they say, the devil is in the details. Remember the Indian parable of the elephant? Blind men who had never met this animal accidentally encounter it on the road. Everyone touches it with their hands and talks about their experiences. One blind man reached out and touched the tusk: how strong and sharp, the elephant is like a spear! The second touched the tail: it is like a rope! The third one touched the ear: the elephant is like a fan! The fourth one touched the leg: the elephant is like a tree! It is the same here. Each element is related to the whole thing, but can you make a "picture" out of them and how accurately will it reflect the reality? In other words, you can put together any figures, but what kind of error will it eventually lead to? Indeed, because of the difference in questions in the questionnaire, each country's set of indicators for the CPI may be specific, and it means that the error will be so big that it will make no sense to compare the data. So it is not only a statistical error, which cannot be small by default, but also semantic, as the subject of research is always different. In that case, what is Transparency International measuring in reality?

With the decline in the number of sources of information, TI experts say, the accuracy of the confidence interval falls, and in these cases, it is appropriate to appeal to the original data. We followed this advice and ... found among the primary sources the results of a survey conducted by the infamous Azerbaijani service Puls-R, which always selects more men than women, whereas by statistics, everything should be the other way around! That is to say the data of this service and, hence, TI, cannot be trusted. It's not pleasant to read (and believe me, to write), but that is the reality.

Let's dot the "i". Although Transparency International has highlighted the flawlessness of its data in every possible way from the very start, in practice it has modified the methodology many times. That is to say TI acknowledges, albeit retrospectively: what was presented as an ideal "yesterday" is no longer such "today". Well, TI has problems with the methodology and is perfecting it, and that's understandable. "Gentlemen always play by the rules and do not lose, they just change the rules sometimes." But then there is a natural question: how did these changes affect the accuracy of studies? But there is no answer. TI is silent. Despite my repeated requests, neither the Azerbaijani, nor Russian, Kazakh and Ukrainian offices of TI and its Berlin office agreed to discuss this painful subject. "Don't mention rope in the house of a hanged man"? This principled refusal to talk about the CPI error against the background of these facts leads to a disappointing conclusion. Transparency Inter-national, which surveys tens of thousands of people each year, produces questionable indexes built from a mishmash of data, and they cannot be trusted.

What is the conclusion from all this fuss? What are these indexes for? One gets the impression that such ratings, just like road maps, country risks and other indexes, are politically servile. Yes, we have corruption, without an effective fight against it the country cannot develop fully, and it is necessary to mobilize all forces to fight this "cancer". And ratings can render a good service here, helping to find adequate responses. But, as it turns out, Transparency International is not a good assistant here.