Scoring system

The MSI uses five fundamental objectives to assess to what extent a media system is independent, sustainable and successful. The five objectives are:

1. "Legal and social norms protect and promote free speech and access to public information.
2. Journalism meets professional standards of quality.
3. Multiple news sources provide citizens with reliable, objective news.
4. Media are well-managed enterprises, allowing editorial independence.
5. Supporting institutions function in the professional interests of independent media".^[12]

These five objectives are the most central aspects of a sustainable and independent media system. Under each of these objectives are seven to nine indicators defining how well a country achieves that objective. Each indicator is scored from 0 to 4.

• A country scouring 0 does not fulfill the indicator and there might be forces holding back the implementation of the objective.^[13]
• Score 1 signifies the country slightly reaches the characteristics of the indicator. That means the government or other forces do not actively combat its implementation, but business environment do not foster it and also government or profession do not entirely support the progress.^[14]
• Score 2 implies that the country accomplishes many aspects of the indicator, but it is too early to evaluate the development. The country could still depend for example on present political forces.^[15]
• Score 3 connotes that the country reaches most features of the indicator. Implementation of the indicator has appeared over a number of years and/or during changes in government. The score refers to likely sustainability.^[16]
• Score 4 is the best score a panelist can give. It means the country fulfils all characteristics of the indicator. Implementation has endured intact over several changes in government and/or economy, shifts in public opinion and social conventions.^[17]

The scores for the indicators are then averaged to generate a single score for each objective. Next the five objectives’ scores are averaged to produce a final score for every country. In a following step the final score is interpreted as shown below:

• 0 - 1.00 = unsustainable, anti-free: Country does not reach or just slightly reaches objectives. Government and legislation actively holding back free media development. Country has little professionalism and low media-industry activity.^[18]
• 1.01 - 2.00 = unsustainable mixed system: Country marginally approaches objectives. Country has parts of a legal system, government is against a free media system, but there is obvious improvement in free-press advocacy and increasing professionalism. To evaluate sustainability one must continue to observe the development of new media businesses.^[19]
• 2.01 - 3.00 = near sustainability: Country approaches multiple objectives concerning legal norms, professionalism and a business environment supporting independent media. Improvements have survived shifts in government and have been established in law and practice. Nevertheless, the country needs more time to guarantee that change and to maintain the increased professionalism and the still young media business environment.^[20]
• 3.01 - 4.00 = sustainable: The media in the country is assumed to be professional, free and sustainable, or approaches these objectives almost. The systems fostering free media are still extant after several governments, after an unsteady economy and after changes in civic opinion or social conventions.^[21]

Scoring process

In each country IREX compiles a panel of local experts who undertake the scoring. The recruited experts represent a broad range of local media workers (editors, reports, owners, managers, media development workers) of urban and rural populations, of the main local ethnic groups in addition to representatives from different geographic regions as well as representatives from academia, NGO’s and the legal field.^[22][23][24] All panelists receive the objectives, the indicators with descriptions and an explanation of how the scoring is done correctly. First each panel member individually completes the questionnaire and scores the indicators. In a second step the panelists get together to talk about the objectives and indicators. This discussion is written up by a panel moderator, mostly a representative of the country's media or an NGO. Then it is revised by IREX editorial staff. Panelists are allowed to change their scores during the discussion, but it is not promoted by IREX.^[25][26] In a third step the "IREX editorial staff review the panelists’ scores, and then score the country independently of the MSI panel" .

Comparing the characteristics of the indices

First of all you can see in table 1 that there are some similarities between the indices. Both IREX and Freedom House have their headquarters in America, both are nonprofit organizations, which receive financial support from the American government or from agencies like USAID or NED, which are subsidized on their part by the government.^[27] Furthermore, both indices measures are designed to evaluate characteristics of media systems from the point of view of elite evaluators. Means the survey in a certain country is conducted by experts of the media environment of the country concerned. But the two indices do not assess the same number of countries. The MSI rates 80 countries, whereas the Freedom of the Press Index rates more than twice as much countries. Another difference is that Freedom House captures all 196 countries in one annual report, which allows a comparison of all 196 countries at the same time, whereas IREX assesses the 80 countries in five studies (Africa, Asia, Europe & Eurasia and Middle East & North Africa), that are not published simultaneously and some of them not even annually. For example, the last report of Middle East & North Africa one can download is from 2009.^[28] This means it is not possible to compare the development of the media environment in the surveyed countries at the same time within one year. Another point is that Freedom House has measured Freedom of the Press since 1980, more than twenty years longer than IREX measures Media Sustainability. So it becomes evident that Freedom House has more experience in conducting these studies. Furthermore, the two indices have different underlying concepts. The underlying concept of the Freedom of the Press Index is to evaluate the status of press freedom in an individual country.^[29] IREX’s concept however is to assess how successful, independent, sustainable a media system is in an individual country and “how media systems change over time and across borders”.^[30] To evaluate their respective target they use different questionnaires and scoring systems. The scoring systems as one can see in table 1 do not have much in common. IREX uses scores from 0 to 4, in which 0 is the worst score. Freedom House uses scores from 0 to 100, in which 0 is the best score. In advance Freedom House attaches different importance to the questions by giving some questions higher scores than others. So the panelists score the questions of the Freedom of the Press Index with a different number of points. The scoring system of the MSI is constantly the same. The surveyed persons score each indicator from 0 to 4. For this reason the MSI questionnaire seems to be clearer and more transparent and the questions can be better compared to each other.

Comparing the questionnaires

At first glance the questionnaires ^[44] appear to differ because of their different content and categories, their different structure and different number of questions, which you can see in table 1. But recent literature shows that there are similarities. For example, Becker and Vlad found out that although the indices pretend to measure different concepts (sustainable media vs. press freedom) they have a high average correlation (*Pearson r .87) across the years 2001 to 2007.^[45] This is so because press freedom is clearly a part of what the MSI assesses, because a media system cannot be sustainable without having a high level of press freedom. So press freedom is definitely one condition for a sustainable media system and therefore it becomes evident that the indices examine much the same phenomenon. It also explains why only about one-third of the questions could not be assigned in table 2 or why about two thirds of the questionnaires could be assigned. To find out what questions of the two indices really match and measure the same issues, the questionnaires need to be regarded in more detail. For that reason each single question of the two indices is compared to each other and assigned to new categories in table 2. Questions concerning the same subject but of two different indices are opposed to each other in two columns. If only one index measures a certain issue or category, the corresponding place in the second column remains empty, meaning this issue is not covered by the other index. As some questions contain different aspects, sometimes only one of the aspects fits to the associated question. To make this clear the non-fitting aspect is written in italics in table 2.

The MSI as well as the Freedom of the Press Index contain questions concerning the following categories:

• Protection of free speech

The questions are about legal protection for free speech. But there are slight differences between the questions. For example, the MSI includes social protection and a distinction between state and public media. The Freedom of the Press Index contains in addition or penalties of journalists and also the distinction between freedom of expression and freedom of the press. However, they both measure protection of free speech.

• Free access to media

In this category the questions are all about free access to media and its protection. Here the MSI specifies the protection against economic threats and law restrictions. Whereas the Freedom of the Press Index just asks about it in general. The MSI also makes a distinction between local and international sources and only the Freedom of the Press Index differentiates between official and unofficial sources.

• Diversity of viewpoints

The MSI has more than one question regarding this category because it specifies the viewpoints. Private and public news sources, niche reporting and programming, local, national and international information as well as social interests and minority languages should be reflected. This differentiation cannot be found in the Freedom of the Press Index. So Freedom House includes the access to plurality viewpoints, which is no subject within the MSI.

• Access to information

Here the MSI differentiates between access to information for media, journalists and citizens, whereas the Index of Freedom House focuses more on existing laws protecting the access to information for journalists.

• Entry to media markets

MSI compares the entry to the media market to entries to other industries. The Freedom of the Press Index operates in this category on a more specific and operational level. It includes both individuals and business entities, which can establish and operate media. And it also deals with the legal regulations for establishing media.

• Crimes against media workers

The Freedom of the Press Index distinguishes between crimes committed by the state, authorities and other actors and the type of crimes such as legal intimidation, physical violence. It examines if local and foreign journalists can work freely without harassments. The MSI deals with legal prosecution of the crime but does not specify the type of crime nor who commits the crime. Both indices, however, differentiate against whom crimes are committed.

• Penalties for libeling

The question about libeling asked by the Freedom of the Press Index concerns existing penalties for libeling officials or the state. The MSI relates more to the person who is defamed and what they can do to prevent libeling (e.g. public officials are held to higher standards, must prove falsity). Furthermore, the MSI question contains the aspect of the civil law against libeling.

• Access to journalistic education

Both indices ask if everyone has the freedom to become a professional journalist. But MSI also raises the question if there are restrictions against or special laws in favour of freedom. The Freedom of the Press Index instead contains the question if professional groups have the freedom to support journalists. This is just one example for the fact that the MSI as well as the Freedom of the Press Index sometimes ask two questions in one, which makes it difficult for the panelists to score.

• Self-censorship

In this category the questions are relatively similar except that the MSI also queries the self-censorship of the editors. Another little difference is the fact that the MSI indicator is formulated negatively.

• Corruption

Only the Freedom of the Press Index differentiates between private and public payments. Overall the question about corruption is put more directly in the Freedom of the Press Index than in the MSI. The MSI asks in one question if the payment of journalists is high enough and if the quality standard of the journalists is good enough to prevent corruption. Put that way, the question concerning corruption seems less direct and aggressive in the MSI index.

• Transparency of media ownership and of media concentration

Both indices ask if media ownership and concentration allows transparency so that consumers are able to judge the content and have access to different points of view. In the Freedom of the Press Index the question to what extent the government controls the media and what impact control has on the diversity of views is put in a way that public broadcasting is subliminally criticized. IREX, however, words the question in a more positive way. The way of putting a question, the choice of words may have an influence on the valuation.

• Subvention system

Again the question of the Freedom of the Press Index is asked more directly than the MSI one. It focuses on the control-aspect someone (state or other actors) can have through subsidies or advertising. By contrast the MSI indicator deals more with a fair and law-based distribution of subsidies and advertising that promotes editorial independence. But the MSI question does not refer to subsidies and advertising from other actors than the government.

• External media bodies and evaluations

Here all questions are put positively. But they contain different external media bodies. Whereas the MSI indicators include independent broadcasting ratings and market researches and its duties, the question of the Freedom of the Press Index includes independent media regulatory bodies like the communications council.

• Distribution and production restrictions

The MSI questions in this category are more detailed. They divide production and distribution in their different branches like media equipment, newsprint, and printing facilities, kiosks, transmitters, cable, Internet, mobile. And again the questions are devised positively with words like apolitical or not restricted. In contrast the Freedom of the Press Index does not divide production and distribution in different branches and here again the question is asked in a direct way in order to get an exact answer to what it wants to measure: the restrictions.

• Political parallelism

These questions examine to what extent the state, politicians or others affect or influence the media content and diversity. Again the MSI question is put in a more positive way due to the use of positive adjectives, whereas the Freedom of the Press Index uses negatively connoted verbs like determine.

Generally speaking, the Freedom of the Press Index operates with more direct questions and on a more restrictive level. The questions mostly refer to possible media restrictions, influences or control by someone or something. As already indicated by its name, this way of asking questions shows the Freedom of the Press Index's main concern and underlying concept - that is the freedom of the press. This concept of the freedom of the press must be seen together with "the legal environment for the media, political pressures that influence reporting, and economic factors that affect access to information".^[48] So the Freedom of the Press Index also includes for example questions about independent judiciary, censorship and the impact of the overall economic stability on the media, subjects that are not covered by the MSI. The MSI, however, operates on a wider field. That means it also focuses on professional standards, quality journalism, trainees for journalists, modern technical facilities for distributing and infrastructure, NGOs supporting free media, and the existence of private media as one can see in table 2. But one should keep in mind that both indices have an American bias underlying their measures and concepts. Therefore, the indices may not fit perfectly in order to access media freedom in countries with a different cultural background.