Methodology
Conceptual framework, indicators, coding scales and aggregation rules of the Index.
Index of Evaluation Institutionalization in the Arab Region
Scale
2. Conceptual framework
The index conceives institutionalisation as the process by which evaluation acquires rules, organisations, routines and shared expectations that give it stability beyond individual projects and actors. Organisational institutionalism reminds us that formal structures may be adopted for their legitimacy rather than their technical efficiency, and may therefore remain decoupled from practice (Meyer & Rowan, 1977). Isomorphic pressures, whether coercive, mimetic or normative, can spread evaluation units and policies without the capabilities that make them effective (DiMaggio & Powell, 1983; Pritchett et al., 2013).
Following the Evaluation GLOBE project (Stockmann et al., 2020), institutionalisation is observed in three systems. In the political system, evaluation is anchored in laws, decrees, organisational units and the audit function, and is used by parliament and across policy sectors. In the social system, it is known, discussed, published and demanded by citizens and civil society. In the profession system, it rests on academic education, professional communication, voluntary organisations for professional evaluation (VOPEs) and codified norms. Evaluation systems in this sense are configurations of supply and demand for evaluative knowledge (Leeuw & Furubo, 2008), and their use depends on the organisational and political conditions in which findings are received (Weiss, 1998).
3. Data and method
The first edition covers 11 states: six in the Gulf, two in the Levant and three in North Africa. Each of the 16 indicators is coded on a common scale from 0 (absent) to 1 (fully institutionalised) on the basis of the documented situation in each state, as analysed in the country chapters of Ben Brik (2025). Binary indicators take the values 0 or 1; three-level indicators take 0, 0.5 or 1; five-level indicators move in steps of 0.25.
Indicators are averaged into four sub-dimensions: institutional anchoring (1A, four indicators) and evaluation use (1B, three indicators) in the political system, societal system (2A, five indicators) in the social system, and professionalisation (3A, four indicators) in the profession system. The composite index is the unweighted mean of the four sub-dimensions; the political system score is the mean of and 1B. States are grouped as consolidating (22 per cent and above), emerging (14 to 22 per cent) and nascent (below 14 per cent).
Political system
- National evaluation laws and strategies
- National decrees on evaluation
- Organisational embedding
- Evaluation role of the supreme audit institution
- Parliamentary use of evaluation
- Sectoral spread of evaluation
- Scope of evaluation practice
Social system
- Use of evaluation by civil society
- Public knowledge of evaluation
- Media and public discourse
- Public availability of evaluation reports
- Civic demand for evaluation
Profession system
- Academic education in evaluation
- Professional communication and exchange
- Organisation of the profession (VOPEs)
- Professional norms and standards
Reading the figure
The composite aggregates four sub-dimensions with equal weight. The political system contributes two sub-dimensions and seven indicators, the social system one sub-dimension and five indicators, and the profession system one sub-dimension and four indicators.
Table 1
Indicator statistics: mean, dispersion and diffusion
| Indicator | Sub-dimension | Mean | Standard deviation | States above zero |
|---|---|---|---|---|
| National evaluation laws and strategies | Institutional anchoring | 0.0 | 0.0 | 0 of 11 |
| National decrees on evaluation | Institutional anchoring | 0.0 | 0.0 | 0 of 11 |
| Organisational embedding | Institutional anchoring | 45.5 | 25.7 | 9 of 11 |
| Evaluation role of the supreme audit institution | Institutional anchoring | 0.0 | 0.0 | 0 of 11 |
| Parliamentary use of evaluation | Evaluation use | 0.0 | 0.0 | 0 of 11 |
| Sectoral spread of evaluation | Evaluation use | 50.0 | 0.0 | 11 of 11 |
| Scope of evaluation practice | Evaluation use | 27.3 | 24.9 | 6 of 11 |
| Use of evaluation by civil society | Societal system | 4.5 | 9.6 | 2 of 11 |
| Public knowledge of evaluation | Societal system | 4.5 | 9.6 | 2 of 11 |
| Media and public discourse | Societal system | 4.5 | 9.6 | 2 of 11 |
| Public availability of evaluation reports | Societal system | 4.5 | 9.6 | 2 of 11 |
| Civic demand for evaluation | Societal system | 4.5 | 9.6 | 2 of 11 |
| Academic education in evaluation | Professionalisation | 54.5 | 39.6 | 8 of 11 |
| Professional communication and exchange | Professionalisation | 29.5 | 29.8 | 7 of 11 |
| Organisation of the profession (VOPEs) | Professionalisation | 13.6 | 28.9 | 3 of 11 |
| Professional norms and standards | Professionalisation | 0.0 | 0.0 | 0 of 11 |
Note. Means and population standard deviations in percentage points across the 11 states.
Reading the table
Indicators with a mean of zero and no dispersion carry no discriminating information in this edition but identify the priorities of evaluation policy. Discriminating indicators are concentrated in the profession system.