About the SGI

FAQ

The SGI
What are the pressing reform challenges for OECD/EU countries?
The SGI project is driven by the premise that the 41 developed countries of the Organization for Economic Cooperation and Development (OECD) and European Union (EU) face a range of challenges in the 21st century. Challenges related to globalization, shifting demographics, climate change and new security risks demand actions that go beyond national boundaries. At the same time, countries face their own individual challenges, which must also be resolved. Despite the pressures of internationalization and economic and political integration, industrialized countries still face internal social and economic challenges requiring domestic policy interventions. These include structural and financial weaknesses associated with states’ social security systems, issues of social justice, shortcomings in education systems, integration problems and unsustainable environmental degradation. These are closely accompanied by worrisome secondary challenges such as growing citizen dissatisfaction with government, fueled by rising distrust of democratic institutions and the structures of advanced market economies.
 
Which time periods do the SGI cover?
The most recent Sustainable Governance Indicators assess the period beginning November 2015 and ending November 2016. Previous review periods are also available on this website and demonstrate trends. The SGI 2016 assessed a period ranging from November 2014 to November 2015. The SGI 2015 assessed a period ranging from May 2013 to November 2014. The SGI 2014 examined developments in the respective countries between May 2011 and May 2013. The SGI 2011 assessed a period ranging from May 2008 to April 2010. The period under review for the SGI 2009 ranged from January 2005 to March 2007. In order to ensure comparability between the review periods, experts were asked to exclusively focus on developments within the period under investigation. Therefore, more recent developments are not incorporated into the findings. The SGI will be updated every year now.
 
Who administers the project?
The Sustainable Governance Indicators are administered by the Bertelsmann Stiftung. Working together with partners from all areas of society, the Stiftung aims to identify social problems and challenges at an early stage and develop exemplary solutions to address them. Within the Stiftung, the SGI project is managed by Dr. Daniel Schraad-Tischler, Dr. Christian Kroll, Dr. Christof Schiller and Pia Paulini.
Innovative principles
How do the SGI stand out in international comparison?
On four points, the approach of the SGI goes beyond other international rankings:

1. The capacity for government reform, one of the pillars of the SGI, has largely been neglected in previous international rankings.

2. In the SGI, the need for reform is examined not only in terms of economic outcomes. Instead, similar weight is also given to social and environmental policies. Thus, the SGI offer a comprehensive assessment of the sustainability of the 41 OECD and EU countries.

3. The SGI relies on a combination of qualitative assessments of country experts and statistical data from official sources. This ensures that our quantitative findings are backed up by extensive qualitative analyses provided by over 100 international experts. To ensure accuracy, final scores are audited and approved by an advisory board composed of renowned scholars and practitioners.

4. No index is perfect. The SGI project strives for complete methodological transparency and welcomes feedback and suggestions. On this website we provide access to our questionnaire and country reports containing the qualitative assessments of our experts. We also provide detailed metadata for all of our quantitative indicators.
 
Why are policy performance, democratic quality and governance capacities measured?
A comprehensive assessment of sustainability must not be limited to the measurement of policy outcomes. Instead, it must also look very closely at the capacity of the responsible political actors to successfully govern. The SGI project assesses the actual capacity of OECD and EU countries to take action and implement reforms in terms of developing, agreeing on and realizing policy. It seeks to answer the critical question of whether a state is able to identify pressing problems, develop proposals for strategic solutions and thus foster sustainable policy outcomes through effective governance. In addition, the sustainability of policies will depend in many ways on the quality of a country’s democracy. Guaranteeing opportunities for democratic participation as well as respect for the rule of law and civil rights cultivate citizens’ confidence in the legitimacy of actions taken by political leaders.
 
Why are qualitative and quantitative assessments combined?
To operationalize and measure the concepts used in constructing the SGI, we rely on a combination of statistical data drawn from official sources as well as the qualitative assessments of country experts. Quantitative data are conformed to cross-national standards. However, such data often do not adequately cover the full meaning of a concept. Complex concepts can be measured best through the use of expert assessments that take the country-specific context into account and provide “thick” descriptions capturing the nuances of phenomena.

In this way, the combination of expert assessments and statistical indicators assumes that both types of observations have specific strengths and weaknesses, that they cannot fully substitute for each other and that neither of them is epistemologically superior to the other. Pairing “objective” quantitative data with highly context-sensitive, qualitative expert assessments delivers a high-resolution profile of policy outcomes, the quality of democracy and governance capacity.
Methodology
How many indicators make up the SGI?
The SGI 2017 contains 67 qualitative indicators and 69 quantitative indicators, which means that the overall assessment of the 41 sample countries entailed a total of 5,576 ratings (i.e. scores). For a complete overview of the indicators, please refer to the Survey Structure.
 
How are the indices calculated?
The Policy Performance, Democracy and Governance indices scores are derived by calculating the arithmetic means of the scores for their respective categories. For example, the Policy Performance score is derived by calculating the average of the categories “Economic Policies”, “Social Policies” and “Environmental Policies”. The individual category scores are derived by calculating the arithmetic mean of the criteria scores. For example, the “Economic Policies” score is derived by calculating the average of the criterion “Economy”, “Labor Market”, “Taxes”, “Budgets”, “Research and Innovation” and “Global Financial System”. For criterion composed exclusively of qualitative indicators, their score is the arithmetic mean of those indicators. For criterion composed of both qualitative and quantitative indicators, the scores are weighted, with fifty percent of each score coming from the arithmetic mean of the qualitative indicator(s) and fifty percent from the arithmetic mean of the quantitative indicator(s).

The three composite indices – that is, the Policy Performance, Democracy and Governance indices – provide scores and ranks for each of the 41 OECD and EU countries. The ranking is based on the score to the second decimal place. If two or more states have the same score at this level of precision, they are ranked equally.
 
How are the indices structured?
The Policy Performance, Democracy and Governance indices are structured into categories. These categories, in turn, are divided into criteria made up of a varying number of quantitative and qualitative indicators. For a complete overview of this structure, please refer to the Survey Structure.
 
How does the qualitative assessment process work?
The SGI’s expert questionnaire is designed to improve the validity and reliability of expert assessments. Assessment questions are formulated so as to elicit detailed factual evidence rather than broad – and, consequently, more subjective – appraisals. In fact, many questions ask for responses that may be cross-checked with responses to other survey questions and statistical data.

The questionnaire provides detailed explanations of each question along with four tailored response options. The detailed explanations are intended to clarify the purpose of each question, to structure the way the expert words her or his assessment and to provide a standardized framework for the production of the country reports. The experts are instructed to adapt the standardized response options to the individual context of the particular country they are evaluating and to substantiate their ratings (i.e. numerical assessment) with detailed evidence. The rating scale for each question ranges from one to 10, with one being the worst and 10 being the best. The scale is differentiated by four response options provided for each question. Although the written assessments do not allow for a direct reconstruction of the numerical ratings, they do provide an explanatory background for them.

Each OECD and EU country is examined by two leading scholars with established expertise on the respective countries. To identify subjective bias and reduce any distortion it might cause, the country experts were selected so as to represent both domestic and external views as well as the viewpoints of both political scientists and economists.

In completing the questionnaire, each country expert provides numerical ratings for all survey questions. The SGI 2017 contains 67 questions, which means that the qualitative evaluations for the 41 sample countries entailed a total of 2,747 ratings (i.e. scores). Whereas the reviewer has access to the written assessments of the first country expert, he or she cannot see the first expert’s numerical ratings, which ensures that scores are given independently.

The survey process progresses through six procedural stages:

First, one expert writes a draft country report, assessing all questionnaire questions. Second, the other country expert reviews this report, making comments and providing alternative or complementary content. Both experts are instructed to restrict their country assessment to the review period (e.g. for the SGI 2014, the review period was May 1, 2011 through May 15, 2013).

The countries examined by the SGI are subdivided among eight regional coordinators. These coordinators, who are political scientists with both comparative and area expertise, are each responsible for four to eight of the 41 sample countries, grouped according to their geographic proximity. The regional coordinators monitor the development of the written assessments according to criteria of validity and objectivity, ensuring a fair and balanced country report. In addition, in the third procedural stage, the regional coordinators give their own numerical ratings based on those provided by the two county experts.

In the fourth procedural stage, the regional coordinators review their ratings collectively to ensure that comparisons are valid across the OECD and EU countries. As part of the discussions forming this review process, each regional coordinator is required to explain, defend, and if necessary, recalibrate his ratings and assessments. To make any changes agreed to during the review process more transparent, the coordinators aim to keep a calibrated score within the range of the two country experts’ ratings.

This regional coordinator review is followed, in the fifth procedural stage, by a second round of review. An advisory board composed of renowned scholars and practitioners discusses the findings and approves the final scores.

In the sixth and final procedural stage, the country reports are subjected to a rigorous editing process, in form and content. Questions that arise are checked with the coordinators and experts.
 
To what extent can the most recent SGI results be compared to previous years?
The set of indicators in the SGI are carefully revised from time to time, following a comprehensive external and internal Evaluation. In this process, some indicators have been replaced by others. Moreover, the composition of the indices has been readjusted. While the SGI 2011, for instance, contained only two indices (a “Status” index and “Management” index), the SGI 2014 contains three indices (a “Policy Performance” index, “Democracy” index and “Governance” index). In addition, the SGI 2014 added ten further countries to the sample (Bulgaria, Croatia, Cyprus, Estonia, Israel, Latvia, Lithuania, Malta, Romania and Slovenia).

These modifications inevitably create distortions whenever the new SGI 2014 results are directly compared with the SGI 2011. For countries included in both SGI 2014 and SGI 2011, comparisons between review periods are largely possible. Most qualitative indicators remained unchanged and can be directly compared to the corresponding assessments in the SGI 2011. For quantitative indicators, distortions are much less of an issue as time series data is typically available dating back to 1995. For new qualitative indicators added to the SGI 2014 (e.g. the indicators “Popular Decision-making” and “Global Social Policy”), the score for the SGI 2014 indicator has been imputed to the corresponding SGI 2011 indicator. Similarly, for quantitative indicators added to the SGI 2014 for which no time series data is available (e.g. the “Multilateral Environment Agreements” indicator), the score for the SGI 2014 indicator has likewise been imputed to the corresponding SGI 2011 indicator.
 
I want to use SGI scores for my work. Where do I get the complete data?
The set of data generated by the SGI can be accessed in its entirety and free of charge under Downloads.
 
All OECD and EU countries are democracies. Why is the quality of a state’s democracy important?
The Democracy index looks at the quality of democracy through four lenses: electoral processes, access to information, rule of law, and civil rights and political liberties. The quality of democracy and political participation in a political system are crucial to its long-term stability and capacity to perform. Indeed, this viability depends to a large extent on the levels of trust between citizens and politics. Guaranteed opportunities for democratic participation and observation, freely accessible information, rule of law and protection of civil rights are thus essential prerequisites for the legitimacy of a political system. Moreover, democratic participation and observation are essential for concrete learning and adaptation processes as well as the capacity to change. The SGI thus regard structures that ensure a high quality of democracy as necessary in achieving sustainable policy outcomes.

Since all OECD and EU member countries are democracies, the SGI’s questions focus on the quality rather than the presence of democracy. Thus, questions are designed to address whether citizens face discrimination in the electoral process, how citizens can access public information, the degree to which the media are independent and diversified, how well states protect civil rights and whether the government and administration act predictably and in accordance with the law.
 
Which states do the SGI compare?
The latest SGI compare 41 member countries of the Organization of Economic Cooperation and Development (OECD) and the European Union (EU), using OECD and EU membership as of July 2017 as a formal criterion to select states from a wider group of developed countries committed to human rights, democratic pluralism and an open market economy.
 
What if statistical data were missing for some countries?
Missing values in official statistic data were supplemented by values from the most recently available previous year or from alternate sources used as proxies. If this was not possible, the missing value was imputed by the median of the available values for the sample countries.
Back to Top