Evidence-based Instruments

   

To what extent do government ministries regularly evaluate the effectiveness and/or efficiency of public policies and use results of evaluations for the revision of existing policies or development of new policies?

EUOECD
 
Ex post evaluations are carried out for all significant policies and are generally used for the revision of existing policies or the development of new policies.
10
UK
Evidenced-based decision-making is deeply rooted in the United Kingdom’s tradition of firm and efficient legislation, and ex-post evaluations are as vital a part of public policymaking as impact and sustainability checks. The OECD ranks the United Kingdom second among its forty members for its approach. Specialist analytical functions have recently been brought together to form the Analysis Function, further strengthening their role in policymaking and service delivery. Analytical approaches to evaluation are set out by the Magenta Book and the Green Book, supported by the Cross Government Evaluation Group coordinated by HM Treasury. More recently the Behavioral Insights Team (formerly a team within the Cabinet Office, but now an independent entity) and the What Works Network (coordinated by the Cabinet Office) promote the increased use of evaluation methods, especially randomized controlled trials.

The notion here is to make government more efficient, and so to quickly identify and overcome unnecessary regulatory hurdles. In 2011, the previous coalition government launched the “Red Tape Challenge,” encouraging citizens to report unhelpful or burdensome legislation that could be cut or reformed. However, the main focus of the United Kingdom’s commitment to ex-post evaluation is surely on facilitating business.

The Regulatory Policy Committee (RPC) assesses the quality of the government’s RIA analyses and further examines all published ex-post evaluation. If the RPC submits a recommendation to the government, it is expected to be implemented into law. Further, businesses can directly address the RPC if they disagree with or feel disadvantaged by a specific governmental regulatory assessment.
Citations:
OECD Regulatory Policy Outlook 2018
9
Finland
Consultation with experts and stakeholders is a natural phase in the Finnish law-making process. In addition, the public is invited to comment on draft proposals online. Furthermore, all proposals for changing statutes must be accompanied by an assessment of their impact across several aspects of society (e.g., the economy and environment). However, the OECD has pointed out that although ex-post evaluations are frequently carried out, Finland lacks a systematic strategy for the ex-post evaluation of regulations.
Citations:
“Säädösehdotusten vaikutusten arvionti”, Oikeusministeriö, http://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/76082/saadosehdotusten_vaikutusten_arviointi_ohjeet.pdf?sequence=1&isAllowed=y
OECD: “Better Regulation in Europe: Finland”. http://www.oecd.org/gov/regulatory-policy/45054846.htm.

OECD (2018), OECD Regulatory Policy Outlook 2018, OECD Publishing, Paris, https://doi.org/10.1787/9789264303072-en.

Opinion service webpage: lausuntopalvelu.fi

Governments Registry for Projects and Initiatives (http://valtioneuvosto.fi/hankkeet).
Switzerland
Article 170 of the constitution states that “(t)he federal parliament shall ensure that the efficacy of measures taken by the confederation is evaluated.” Ex post evaluations have been strongly developed and are, to varying degrees, standard in most policy fields. Evaluations are best established in the fields of development cooperation, public health, education and economy. Ex post evaluations are a most important source of information for the revision and development of policies when used by the administration. Administration experts draft laws and reports based on the available empirical evidence, which mostly consists of policy evaluations. However, the administration formulates drafts that are subject to pre-parliamentarian and parliamentarian policymaking processes that include many relevant actors that do not prioritize evidence. The main goal of policymaking in Switzerland is acceptance rather than evidence-based policy. However, as many evaluations focus on learning within the administration rather than executive control and new legislation, the impact of evaluations remains significant in Switzerland. The Swiss evaluation community is one of the most professional in Europe and evaluations are of good quality. The role of ex-post evaluations in Switzerland can thus be considered important.
Citations:
Fritz Sager, Thomas Widmer, Andreas Balthasar (eds.) 2017: Evaluation im politischen System der Schweiz. Entwicklung, Bedeutung und Wechselwirkungen, Zürich: NZZ Verlag
 
Ex post evaluations are carried out for most significant policies and are used for the revision of existing policies or the development of new policies.
8
Denmark
Existing regulations of RIAs do not require ex-post evaluations. Such evaluations are part of the ongoing political process. Political agreements either explicitly or implicitly settle policies for some period. Though various events and developments may require subsequent action, which would provide a chance for citizens, media outlets and policy entrepreneurs to advocate policy reforms. This happens regularly in the Danish political system and very often the government or the opposition will suggest new policies. Sometimes expert committees will be set up to analyze the issues. Most significant policies are regularly debated and policy reforms are common.
The annual budget preparation is one occasion for evaluating policies. The parliament’s Rigsrevision (auditor general) also issues an annual report, which may lead to policy reforms. In some cases, an assessment is made an explicit part of a political agreement (e.g., labor market policy).
Citations:
Vejledning om konsekvensanalyser Maj 2005 (afsnit 3.2, 7.1 og 7.2 revideret i 2018). https://modst.dk/media/19528/vejledning-om-konsekvensanalyser-maj-2005-afsnit-3-2-7-1-og-7-2-revideret-i-2018.pdf (Accessed 10 October 2018),
Jørgen Grønnegård Christensen et al, Politik og forvaltning. 4. udg. Reitzels Forlag, 2017, p.300.
Norway
The Norwegian government makes use of evaluations in most policy sectors and issue areas. Evaluations are carried out by external experts or internal ministerial review bodies. Evaluations are sometimes intended to measure the effect of reforms, although more frequently evaluations serve as a starting point for a future reform process. There is broad support for evidence-based policymaking and the results of policy evaluations tend to attract considerable attention.
Sweden
Ex-post evaluations take various forms in the Swedish system. For the past 10 to 15 years, performance measurement and management has been an integral part of public management in Sweden, as in most other countries. Also, audits conducted by the Swedish supreme audit institution (Riksrevisionen) are important evaluation instruments. There is, however, a tendency to focus more on institutions and cost efficiency (the audit approach) than on programs and impact (the evaluation approach). That said, both approaches are useful as feedback on public policy.
Citations:
Pierre, J., B. G. Peters and J. de Fine Licht (2018), “Is auditing the new evaluation? Can it be? Should it be?,” (tillsammans med B. Guy Peters och Jenny de Fine Licht), International Journal of Public Sector Management 31:726-39.
USA
The United States has extensive and highly sophisticated capabilities for evaluating the impact of public policies – within government, in a large sector of think tanks and consulting firms, and in the extensive public policy-oriented academic community. In normal times, however, U.S. government is relatively inflexible, and slow to respond to evidence of the need for change. This is a widely recognized consequence of the separation-of-powers constitutional system, which was designed to inhibit policy change. In particular, the institutional barriers to policy change enhance the ability of organized groups to resist reforms that they oppose. For example, teachers unions have exploited Democratic party support and the complexity of policymaking institutions to prevent or water down reforms targeting a strengthening of school and teacher accountability for educational results. Both political parties have tended to resist change opposed by their respective interest group or ideological constituencies.

The lack of interest in information about the consequences of policies reached a new and quite extraordinary level during the first two years of the Trump presidency. In some cases, the Republican leadership overrode congressional rules by refusing to wait for CBO analysis before voting on bills. Policy journalists have described an unprecedented indifference among senior government officials to evidence of the actual effects of policy. Effective use of evaluation information is limited to issues that are not on presidential or partisan agendas.
Citations:
guess the question is less about the responsiveness, more toward the mechanisms. Most legislature is implemented just for a couple of years and well just be extended after a positiv evaluation.
7
Canada
As with other aspects of Canada’s RIA regime, ex-post evaluation differs between laws and regulations. While laws are not subject to systematic ex-post evaluation, departments and agencies are expected to regularly review existing regulations to assess their impacts and develop refinements.

In 2016, the Treasury Board of Canada introduced a new “Policy on Results” with the objective to inform decision-making, improvements, innovation and accountability. Evaluations of programs, policies and priorities under the policy is to be a “systematic and neutral analysis of evidence related to relevance, effectiveness and efficiency of policies and programs,” and should “generally employ social science research methods.” Those evaluations can be done through the Treasury Boards’ resource alignment reviews and internally by departments themselves.

In practice, it is frequently the Office of the Auditor General of Canada (OAG) that evaluates government programs and initiatives ex post. The OAG is formally charged with so-called performance audits, which aim to provide an independent, objective and systematic assessment of whether government programs are being run with due regard for the economy, efficiency and environmental impact. The OAG has considerable discretion regarding which programs it will examine, and takes requests from parliamentary committees, members of parliament, citizens, civic groups and other parties to conduct audits in specific areas. It conducts between 25 and 30 performance audits each year and publishes the results.
Citations:
Treasury Board of Canada, Cabinet Directive on Regulation, available at https://www.canada.ca/en/treasury-board-secretariat/services/federal-regulatory-management/guidelines-tools/cabinet-directive-regulation.html

Treasury Board of Canada, Policy on Results, available at https://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=31300
Estonia
The Strategy Unit of the Government Office is responsible for the quality of policymaking, including the evaluation of policy effectiveness and development of a knowledge base for future reforms. For 2014 – 2020, EU Social Fund resources have been allocated for these activities, which has resulted in an increased number of studies of ex post and ex ante policy impact analyses. Nevertheless, the overall framework remains a work in progress, as the evaluations do not cover all significant policies and are not systematically used for the development of new policies. Since the Government Office has limited analytic capacity, the studies are produced by external national and international research teams.
France
There is no practice of systematic evaluation, except for policies or laws in which the respective constitutive act stipulates the need for an evaluation. However, over the past 25 years, the Court of Accounts, previously a legalistic type of control, has transformed its mission and adapted its methods to the evaluation of public policies from a political, social, economic and financial point of view. The reports of the court have become reference documents not only for the political authorities (government and parliament), but also for the opposition, the media and public opinion. The reports are usually characterized by the depth of the analysis and the accuracy of criticisms, and its propositions are usually well received. Both the parliament and the government rarely challenge the courts’ conclusions and recommendations, which often become the basis for new legislation. Since Sarkozy, the nominee for president of the court has been a former politician from the opposition (at the time of appointment). This pattern has strengthened the legitimacy of the court, and allowed for the adoption of more “policy” and politically oriented evaluations. This dimension is not negatively perceived, as the court is not seen as biased in its conclusions, while its pragmatic suggestions are seen as useful for the preparation of new legislation.
Germany
In general, government ministries are solely responsible for policy results and, as a consequence, evaluate the effectiveness of policies within their domain. Evaluations are often part of a ministry’s daily procedures.

Traditionally, evaluation of legislation in Germany has an ex ante character. Ex post evaluations are not yet universally undertaken. Ex-post evaluations study the causal impact of policies based on credible experimental methods, for example, through comparisons of a treatment group with a control group. Today, thorough ex post analyzes are used in the fields of labor market, education and family policies. A milestone for ex post labor market research was the introduction of a legal obligation to evaluate the impact of active labor market policies in 1998. Since then, important legislation like labor market and social security reforms (Hartz reforms), and later the introduction of minimum wages have undergone a far-reaching ex-post evaluation with high methodological standards and the involvement of independent academic expertise.

Thus, while good standards for high-quality ex post evaluation have been set for some important policy fields, the broad use of these methods across a wider field of policies is still lacking in Germany.
Citations:
Boockmann, B., Buch, C. M., Schnitzer, M. (2014): Evidenzbasierte Wirtschaftspolitik in Deutschland: Defizite und Potentiale, IAW Discussion Paper Nr. 103, April 2014.
Japan
Government ministries make considerable effort to evaluate their policies on an ex post basis. The Administrative Evaluation Bureau (AEB) conducts inspections, and each ministry carries out independent evaluations of the effects of its own policies. The AEB supports such activities, for instance by encouraging ministries to share methodologies and experiences. It also works to standardize and prioritize policy evaluations, and reviews ministry and agency evaluations.

In 2017, a new rule was introduced requiring regulatory enforcement ministries and agencies, which previously had primarily evaluated regulations themselves, to engage in ex post evaluation.

At this stage, it is difficult to judge the quality of the ministerial evaluation efforts, or whether such activities have indeed become a major source of information in the reformulation of policies and projects. Such evaluations do not play a significant role in public policy discussions.
Citations:
Council for Promotion of Regulatory Reform, Third Report by the Council for Promotion of Regulatory Reform - For New Era to Come, Provisional Translation, 4 June 2018

Ministry of Internal Affairs and Communications (Administrative Counseling Division), Japanese Ombudsman System, Tokyo, March 2018
Lithuania
Government ministries frequently evaluate the effectiveness and/or efficiency of public policies, but most evaluations are related to the use of EU funds; it is mandatory to evaluate the implementation of operational programs financed from EU structural and investment funds. For instance, 63 evaluations were performed during the programming period from 2007 to 2013. Many evaluations will be executed during the 2014 to 2020 period. The implementation of evaluation recommendations is monitored on a regular basis, but a 2013 study revealed that only about 60% of all the recommendations provided by the evaluators were executed by Lithuanian ministries and other state institutions. This average rate of implementation was attributed to insufficient institutional and personnel evaluation capacities in the public administration, which reduce the demand for evaluation, hinder quality assurance and the use of evaluation results. The recommendations of the State Audit office are likewise routinely ignored by the government.

Although Lithuanian authorities should also assess the implementation of national budget programs, no such evaluations have been completed since 2016. This is related to the ongoing restructuring of the strategic planning and budget formulation system. An action plan for improving the efficiency of the public sector envisions the establishment of an evidence-based competence center that will conduct cost-benefit analysis on strategic issues and other systemic evaluations. MOSTA, which provides evidence-based information and guidance on policy formulation and implementation, is likely to become responsible for these analyses and evaluations in several policy fields.
Mexico
Overall, Mexican policies are supposed to subjected to ex post evaluation and, at least at the national level, a comparatively strong culture of ex post evaluation has grown over the last two decades. This phenomenon is rooted in two different ideological streams. On the one hand, the traditional planning euphoria from a left-leaning, corporatist system has embraced the idea of ex post evaluation as an integral part of a well-mastered policy cycle. On the other hand, market liberal reforms and the influence of international finance organizations have introduced forms of new public management, with rigorous ex post evaluation seen as a crucial way to guarantee the efficiency and effectiveness of public policies.

Since 2012, mandatory guidelines require the use of ex post evaluations. As a consequence, Mexico has established one of the most developed evaluation policies in the field of social policy, which is executed by a constitutionally anchored evaluation agency, Coneval (National Council for the Evaluation of Social Development Policy). Created in 2014, Coneval is established as an autonomous constitutional organization with a very high level of technical and management autonomy. Its task is to coordinate and manage the ex post evaluation of national social policies, and it does so with a very high level of sophistication. It is consists of the head of the Ministry of Social Development, six well-known academics chosen by the National Social Development Commission and an Executive Secretary who is in charge of the council. In the international realm, CONEVAL has been an institutional innovation in poverty measurement and the evaluation of social public policies. So far, the independent provision of scientifically based evaluations has had a substantial impact on technically improving social policy programs in Mexico.

Coneval represents a substantial move forward in the evaluation of social policies. However, other policy fields are subjected to far less scientific ex post evaluations and, at the subnational level, much more remains to be done. However, the ongoing presence of many organizations of international development cooperation in Mexico is favorable for an evaluation friendly environment.
Netherlands
The General Audit Chamber (Algemene Rekenkamer) scrutinizes ex-post policy evaluations by ministerial departments. Since 2000, the chamber has reported its findings to parliament on the third Wednesday in May each year. In 2012, the government introduced the Regulation for Regular Evaluation Studies, which specifies research criteria for assessing policy efficiency, goal achievement, evidence-based policymaking and subsidy-based policies. Yet, time and again, the chamber has reported deficits in goal achievement and weaknesses in goal formulation, which undermine the quality of ex-post evaluation research. Other weaknesses in policy evaluation studies include the lack of citizen perspectives, inability to accurately calculate societal costs and benefits, overreliance on input from implementing organizations for evidence and lack of public access to many evaluations. In line with the general trend toward more instrumental advice, over the last couple of years, the General Audit Chamber has focused its attention on specific points in departmental agendas. Moreover, there are a wide range of additional non-obligatory evaluations produced by ministerial departments, parliament, government-sponsored knowledge institutes, the ombudsman, implementation bodies and quasi-independent non-governmental bodies. Since evaluation findings are just one factor in designing new or adjusting existing policies, it is not clear how much policy learning actually occurs. Dutch ex-post evaluators closely follow international trends of “evidence-informed” and “behavioral knowledge”-driven evaluation studies. There is a tendency to move away from a focus on single, case-specific ex-post evaluation studies to a focus on the construction of broader, more balanced departmental knowledge portfolios, in which ex-post evaluation studies are embedded as elements in a larger body of knowledge accessible to policymakers and other participants in policy subsystems. It is not yet clear to what extent such trends in evaluation research really inform evaluation practices.
Citations:
A. Knottnerus, Van casus-specifieke beleidsevaluatie naar systematische opbouw van kennis en ervaring, Beleidsonderzoek Online, May 2016

Meyken Houppermans, ‘Wat is de toegevoegde waarde van de onafhankelijk
deskundige bij beleidsdoorlichtingen?’, Beleidsonderzoek Online juni 2018,
DOI: 10.5553/BO/221335502018000005001

L. Hermans, Adaptief beleid en beleidsevaluaties, Beleidsonderzoek Online, November 2017

P. van Hoesel, Beleidsevaluatie als ritueel, Beleidsonderzoek Online, February 2017

A. Korsten, Wat ervan terecht komt. Zicht op beleidsevaluatie, April 2013 (arnokorsten.nl, accessed 31 October 2018)
6
Australia
Ex post evaluation of public policies is not a mandated or standard part of policy implementation in Australia. However,, Australia was ranked first in ex post evaluation in the OECD Government at a Glance Database. While an effective evaluation system is in place, the effect on the revision of policies has so far been limited. Periodically, policies are announced with explicit accommodation for their evaluation, but this is the exception rather than the norm. In general, evaluation is more likely to occur for policies developed at the departmental or agency level (as opposed to the top level of government) or for policies that are not politically important/sensitive.
For policies that are evaluated, the quality of evaluation is variable, and rarely is the “gold standard” of a randomized controlled trial used.
Citations:
https://de.slideshare.net/OECD-GOV/effective-ex-post-evaluation-purpose-and-challenges
Chile
Since 1997, the Ministry of Finance’s Budget Office (Dirección de Presupuestos, DIPRES) has had the power to assign specific budgets to line ministries for the contracting of external consultants to carry out ex-post evaluations of their respective government programs (Evaluación de Programas Gubernamentales, EPG). Programs or institutions to be evaluated are agreed on with the Congress annually and instructed by a ministerial decree. The evaluation results are normally made publicly and freely available.
Citations:
http://www.dipres.gob.cl/598/w3-article-111762.html

Evaluación Ex-Post: Conceptos y Metodologías (Dipres 2015)
http://www.dipres.gob.cl/598/articles-135135_doc_pdf.pdf
Czechia
Ex post evaluations are a regular part of the RIA process. Save for measures financed with EU funds, however, they have not been carried out in a systematic fashion so far. In order to address this problem, the Government Office presented in mid-2018 a comprehensive proposal for improving the quality of ex post evaluations.
Israel
In 2015, the OECD Regulatory Policy Outlook stated that Israel had made progress in improving regulatory policy across several fields, including ex-post evaluations. Though the report also determined that Israel had no general policy on ex-post evaluation, and that government offices and ministries operate without clear guidelines. Since then, Israel has invested in improving its ex-post evaluations.

In 2018, an OECD report, “The Long View: Scenarios for the World Economy to 2060,” concluded that Israel had improved many regulatory fields, including ex-post evaluations. This, to our understanding, is a continuation of the process mentioned in the OECD report from 2017, which highlighted the general improvement in regulatory processes. The report also concluded that Israel has improved its consultation processes by opening the processes up to the public and integrating them into the RIA system. While Israel still focuses on reducing the regulatory burden, it seems that it is working harder on providing a solid basis for a general government regulatory policy.
Citations:
“Israel could gain from increased competition, public spending, OECD says “, Times of Israel, 2018 (Hebrew):
https://www.timesofisrael.com/israel-could-gain-from-increased-competition-public-spending-oecd-says/

“OECD economic scenarios to 2060 illustrate the long-run benefits of structural reforms,” OECD Website,
http://www.oecd.org/economy/oecd-economic-scenarios-to-2060-illustrate-the-long-run-benefits-of-structural-reforms.htm

“OECD Regulatory Policy Outlook 2015,” OECD Website,
https://www.oecd.org/gov/regulatory-policy/Israel-web.pdf

“OECD Regulatory Policy Outlook 2018“, OECD WEBSITE, 2018
https://www.oecd-ilibrary.org/docserver/9789264303072-en.pdf?expires=1540849342&id=id&accname=ocid195467&checksum=2F91C900404B34AA4EE7659685539C79

“Government at a Glance 2017 – Israel,” OECD Website,
https://www.oecd.org/gov/gov-at-a-glance-2017-israel.pdf
Latvia
Currently ex-post evaluation is carried out for development planning documents, as prescribed in the Development Planning System Law. In addition, the “methodology for developing and evaluating the results and performance indicators for ministries and other central state institutions” provides general guidelines for ministerial reporting. However, there is currently no common approach to the evaluation of legislation post-implementation, although institutions are allowed to order research studies (including ex-post impact studies) at their own discretion.

Recognizing the need for a unified approach and clear regulation, the Cabinet of Ministers approved the State Chancellery’s concept report on ex-post evaluations in 2016. The report considered several potential approaches, before recommending that two pilots should be carried out in 2017, which would then be used to finalize a new policy in 2018. This policy is currently pending.
Citations:
1. State Chancellery (2016), Annual Report (in Latvian), Available at: https://www.mk.gov.lv/sites/default/files/page/attachments/valsts_kancelejas_2016.gada_parskats.pdf, Last assessed: 06.01.2019

2. State Chancellery (2017), Annual Report (in Latvian), Available at: https://www.mk.gov.lv/sites/default/files/page/attachments/valsts_kancelejas_gada_parskats_2017_final_6.pdf, Last assessed: 06.01.2019

3. Development Planning System Law (in Latvian), Available at: https://likumi.lv/doc.php?id=175748, Last assessed: 06.01.2019

4. Methodology for developing and evaluating the results and performance indicators for ministries and other central state institutions (regulation), Available at: https://likumi.lv/doc.php?id=200935, Last assessed: 06.01.2019

5. Cabinet of Ministers (2016), Report on Ex-post Evaluation Implementation (in Latvian), Available at: http://tap.mk.gov.lv/lv/mk/tap/?pid=40386136, Last assessed: 06.01.2019
Malta
In recent years, ex-post evaluations have been carried out for most significant policies. Various tools are used, and supported by enhanced digital processes. Improvements in ministerial coordination have also contributed to this development, along with ongoing review by the Office of the Principal Permanent Secretary. A “mystery shopper” for government departments was introduced in 2017, tasked with identifying shortcomings in service delivery so that they could be remedied accordingly. The National Audit Office performs audits to determine whether government entities have adequate systems of internal controls in place, with follow-up audits conducted to determine whether identified weaknesses have been dealt with. The 2018 follow-up audit report paints a mixed picture, with recommended improvements fully or partially implemented in some cases, and no changes made in others.
Citations:
Follow Up Reports by the National Audit Office 2018 http://nao.gov.mt/en/recent-publications
New Zealand
Despite the dominant role of ex-ante RIAs, steps are also taken to implement ex-post RIAs. However, the quality of ex-post-evaluation of public policies remains unclear. They are not mandatory and there is no established methodology for conducting them. Introducing systematic reviews of regulation could potentially help strengthen the policymaking process. There is little evidence to assume that this has changed in the review period of the SGI 2019.
Citations:
https://treasury.govt.nz/publications/legislation/regulatory-impact-assessments
Gill, Derek 2016. Rgulatory Coherence: The Case of New Zealand. ERIA Discussion Paper Series 2016-12. Wellington: University of Wellington.
OECD Regulatory Policy Outlook 2015 Country profile New Zealand. https://www.oecd.org/gov/regulatory-policy/New%20Zealand-web.pdf
Kupiec, Tomasz 2015. Regulatory Impact Analysis Practice in New Zealand in the Light of Models of Evaluation Use – Inspiration for the Polish Government. „Management and Business Administration. Central Europe” 23(2), pp. 109–128.
Slovenia
Ex post evaluations are regularly carried out for the most significant policies, but rarely for all other policies. When carried out, ex post evaluations are primarily used for the improvement of existing policies rather than for the development of new policies.
South Korea
Korea was ranked 2nd behind Australia in terms of ex post evaluation in the OECD Government at A Glance database. While there is an effective evaluation system in place, the effect on policy revision has been limited to date.
Citations:
OECD, Government at a Glance Database, http://www.oecd.org/gov/government-at-a-glance-2017-database.htm
Spain
The State Agency for Evaluation of Public Policies and the Quality of Services was created in 2006 in order to promote a culture of evaluation and to improve the quality of services in the public administration. In fact, evaluations vary substantially according to the public policy. In 2017, the Agency was downgraded, becoming the Institute for the Evaluation of Public Policies which is today attached to the Ministry of Territorial Policy and Civil Service. In theory, this Institute is in responsible for the ex post evaluation of all public policies. However, during the period under review, no data regarding the activities of the Institute was available. The 2015 law on the common administrative procedure includes a provision regarding ex post evaluation, making it compulsory to evaluate the application of rules and norms in order to ascertain whether they have met the objectives pursued, and whether costs have been justified. Most autonomous regions have developed additional, systematic ex-post evaluation programs for their public policies. However, the practical impact of these measures has to date been limited.
The Independent Authority for Fiscal Responsibility (AIReF), which was created in 2014, is responsible for the evaluation of public spending and the efficiency of public policies. The AIReF analyses key expenditure and investment items such as education, healthcare and public works. The reports are not binding, but if the administration chooses not to follow the recommendations, it must justify this decision. In 2018, the AIReF’s independence was reinforced by giving it greater access to information and improving its budget-drafting process and the way in which it monitors recommendations. It has undertaken several spending reviews and policy evaluations.
Citations:
Ley 39/2015. Royal Decree 105/2018
 
Ex post evaluations are rarely carried out for significant policies and are rarely used for the revision of existing policies or the development of new policies.
5
Italy
Italian governments and parliaments rarely conduct in depth ex-post evaluations of policies before promoting a revision of existing or the development of new policies. Any new bill is accompanied by a technical report, which typically discusses existing policies and explains the benefits of the new policy. But only rarely is this the result of an evaluation process conducted through a public and open debate. A good example is the deep revision of the pension reform of 2011 (the so-called Fornero reform, named after the then minister of welfare) proposed by the current government. This revision was an election campaign promise of the Northern League, but its implementation has not followed any serious cost-benefit analysis.
4
---
---
3
Austria
Ex-post evaluation is a rather unknown field in Austrian politics. The lack of any systematic ex-post evaluation tradition and the tendency of political actors to prioritize the next election over all other perspectives makes it highly unlikely that the present government or parliament will establish a structure of ex-post evaluations. The absence of long-term strategies, beyond traditional vague ideologies (like social justice or defending Austrian identity), prevent any reasonable systematic ex-post evaluation. Though one exception concerns electoral campaigning. Following the priority given to electoral strategies, parties in Austria reflect systematically on the reasons for any specific electoral outcome, which may be viewed as a specific version of ex-post evaluation.

Moreover, the Austrian Audit Court carries out specific ex-post evaluations from time to time.
Bulgaria
The rules for impact assessments in Bulgaria established in 2016 require an ex post evaluation of policies and their effects within five years of their implementation. So far, providing ex post assessments has not become a regular practice. The public consultations portal of the government contains ex post assessments of some laws, but the latest dates from 2012. There is no evidence of ex post assessments being used by government bodies for the evaluation, revision or termination of policies.
Croatia
The process of Europeanization and Croatia’s membership in the EU have opened the space for the use of evaluation studies in Croatian public administration at the central and local government levels. Methods and theoretical approaches to evaluation are exchanged through the Croatian Evaluation Network, which is comprised of experts interested in evaluation practice. However, ex post evaluations are still rarely carried out for significant policies and are even more rarely used by policymakers as a source of evidence and inspiration. If ex post evaluations are carried out, the success indicators tend to be too general and insufficiently precise.
Hungary
There is no formal framework for carrying out ex post evaluations in Hungary. Such evaluations are rarely carried out since the Orbán governments have been more interested in exercising political control than in the effectiveness of their measures.
Ireland
The extent of overspending in health care totaled more than €2 billion over the last four years according to the Irish Fiscal Advisory Council. Such budgetary over-runs suggests that there is little ex-post evaluation of policy in this significant budgetary area. Each year the government is presented with an over-run that is tacitly accepted and paid for out of buoyant tax revenues. However, if tax revenues fall, the ability of the government to fund such over-runs will create significant political tensions.
Poland
According to the RIA rules, ex post evaluations can be requested by the Council of Ministers. In practice, however, primary laws and subordinate regulations are only rarely evaluated. Even when such procedures are carried out, the outcome of the review is not used to improve the quality of laws.
Portugal
A recent study noted that ex post evaluation is very weakly institutionalized in Portugal. The ex post evaluations that take place are largely the result of external pressures (notably, EU requirements) and circumscribed to a limited set of policy areas – usually, those that legally require ex post evaluation.
Citations:
Ana Diogo (2018), “Hacia un sistema integrado de M&E para los objetivos de desarrollo sostenible en Portugal” [dissertation], available online at: https://www.somosiberoamerica.org/investigaciones/la-evaluacion-de-la-agenda-2030/
Romania
While the institutionalization of ex post evaluations has been announced several times, they have remained the exception rather than the rule. If such evaluations have been done, their impact on decision-making has been intransparent.
Slovakia
Ex post evaluations have not been required by law so far. If they have been carried out, they have focused mainly on administrative burdens. However, the RIA 2020 Better Regulation Strategy, as approved by cabinet in January 2018, has envisaged more systematic ex post evaluations.
 
Ex post evaluations are generally not carried out and do not play any relevant role for the revision of existing policies or the development of new policies.
2
Belgium
The typical strategy is to pick the data that justify the decisions that have been made. For instance, to justify the usefulness of a subsidy, the government may argue that a large number of people demanded the subsidy, not whether the subsidy managed to achieve its political objective. This approach has led to counterproductive decisions in the areas of, for example, education, energy conservation, subsidies for solar panels and immigration. There may be some scattered ex-post evaluations undertaken on the initiative of individual line ministries, but these evaluations have no direct impact on the revision of existing policies since they are not seriously considered by ministerial cabinets, where all strategic policy choices are initiated and arbitrated.
Cyprus
Given the deficiencies in strategic planning, ex-post evaluation has not been part of government practice. However, in the framework of commitments to reform the public administration, the government in recent years solicited the drafting of studies by the World Bank and others. These studies on the administration and line ministries aimed mainly at restructuring and reforming the administration and its procedures, while also reviewing their general policies. Specific policies in ministries are not evaluated and progress achieved so far remains limited.
Citations:
Public Administration Reform – Restructuring ministries and administration http://www.reform.gov.cy/en/public-administration-reform/restructuring-in-the-public-sector
Greece
Ex-post evaluation is not a usual part of the policy cycle in Greece. There is frequent turnover among the ministers. It is rare that a minister stays in office long enough to evaluate a measure he or she adopted. After government reshuffles and certainly after a change in government, incoming ministers rarely are interested in evaluating the policies pursued by their predecessors. Notwithstanding, there are in the public administration and central headquarters of ministries, higher-ranking civil servants who could perform such evaluations if asked to do so. The potential is there, but it is rarely used. There are, however, cases when the Greek government asked international organizations (e.g., the OECD) or thinktanks to evaluate policy impacts. The Syriza-ANEL government was rather reluctant to ask for external advice.
Iceland
No regular ex post evaluations of the effectiveness or efficiency of public policies appear to be conducted by any government ministry. However, that does not mean that no such evaluations take place.
Luxembourg
An evolution of legislation rarely takes place. One example was the Violence Protection Act, which was evaluated in 2009 (after five years). There is a lack of qualified evaluation staff in Luxembourg. There is also a lack of a qualified person for the conception of laws. A scientific evaluation of legislation, commissioned by the state, is therefore not expected in the future either.
Citations:
“Evaluation „Fünf Jahre Gewaltschutzgesetz im Grossherzogtum Luxemburg.” http://mega.public.lu/fr/publications/publications-ministere/2010/evaluation-gewaltschutzgesetz/Evaluation_Gewaltschutzgesetz.pdf. Accessed 23 Oct. 2018.
Turkey
Currently, the capacity of public policy assessment institutions in Turkey lack knowledge regarding evidence-based instruments, both theoretically and practically. RIA practice was introduced by Law No. 5018 on Public Financial Management and Control (2003), and other relevant regulations. In this context, a cost-benefit analysis is required for all public agencies. However, there are various social and political barriers to the evaluation of public policies. For example, the parliament and judiciary cannot effectively supervise and review executive actions. It is unlikely that an effective evaluation mechanism will be developed in the near future. However, there are a few academic studies that have evaluated certain public policies.
Citations:
Aydın Usta, “Yeni Kamu Yönetiminde Politikaların Değerlendirilmesi: Tipolojiler, Boyutlar ve Ölçütler, ” Sayıştay Dergiṡi̇, 94/Temmuz-Eylül 2014, 5-27.
İbrahim Arap and Veysel Erat, “Bir Kamu Politikasının Analizi: Türkiye’de Geçici Köy Koruculuğu,” Mülkiye Dergisi, 39 (4), 2015: 73-108.
1
---
---
Back to Top