Evaluation Capacity Assessment of the Green Climate Fund’s Direct Access Entities

The Green Climate Fund (GCF) is a relatively young climate finance institution with over 200 accredited entities (AE) and delivery partners working with developing countries in designing and implementing projects and results measurement. Within this cohort of AEs, there are direct access entities (DAEs). These DAEs operate at the subnational, national, or regional level and are emphasized in many GCF policies and strategies. The Independent Evaluation Unit (IEU) has undertaken a desk-based review of the evaluation capacities of GCF’s DAEs that aims to:

1. Scan the capacity of DAEs to conduct evaluations that will feed into the IEU’s future capacity building initiatives[1] and inform future efforts for a capacity building strategy.
2. Feed into the IEU’s ongoing evaluations regarding DAE operations.

Through a desk review of documents and websites, the review[2] inspected four broad characteristics as proxies for DAE evaluation capacity:

1. Evaluation office availability
2. Evaluation staff availability
3. Evaluation(s) conducted or commissioned by an entity[3]
4. Previous/current projects with GCF/GEF (Global Environment Facility)/AF (Adaptation Fund)/MDBs (Multilateral Development Banks)/UN agencies

Findings

As of October 2022, there were 72 DAEs. The following are the key insights drawn from the analysis.

Finding 1: Availability of an evaluation office: only 32 per cent (23) of DAEs have evaluation offices, while 53 per cent (38) do not. An evaluation office may include a unit doing policy research or similar if it has a mandate, inter alia, to undertake evaluations.

Finding 2: Availability of evaluation staff: only 25 per cent (18) have evaluation specific human resources available. However, 46 per cent (33) – the majority of the 72 DAEs – did not report relevant information regarding their evaluation staff availability (Figure 2).

Finding 3: Experience in conducting evaluations: 38 per cent (27) have completed evaluations in the past or have ongoing evaluations (Figure 3).

Finding 4: Experience working with international organizations: 69 per cent – the majority of entities –have completed projects and current project pipelines with international organizations (Figure 4).

 

However, some degree of evaluation capacity exists within the institutions experienced in undertaking projects with international organizations. The analysis of 50 entities with experience in implementing international projects provides compelling evidence regarding the availability of evaluation offices and staff. Of the 50 entities, 22 have evaluation offices, while 22 do not. Six entities did not report on the availability of such offices. Moreover, 25 entities have evaluation staff, while only seven do not. Twenty-one entities did not provide any information regarding the availability of such staff. Notably, 15 entities out of the 50 did not report or possess any evaluation capacity or experience, as shown in Table 1.

Table 1. International project implementation experience by the DAEs

Indicators	Entities with project implementation experience (GCF/GEF/MDBs/UN Agencies)
	Yes	No	NR*
Evaluation Office	22	22	6
Evaluation staff	25	7	18
Evaluation experience	16	13	21
No of the above	15
Of 72 DAEs, 50 entities indicated their experience in implementing projects with GCF/GEF/MDBs/UN agencies   *“NR” indicates not reported.

Finding 5: Public reporting of evaluation capacity: Analysis of the types of entities found that research centers, funds and ministry departments have more information regarding the availability of their evaluation capacities on their websites or in public documents. At the same time, banks do not usually report their evaluation capacities or do not have some degree of evaluation capacity. Similarly, a much higher share of non-Bank DAEs has experience implementing projects with international organizations compared to Bank DAEs.

 

This analysis seems to confirm the underlying hypothesis that, apart from multilateral development banks, most of the banks are commercial entities and hence less inclined to prioritize having evaluation capacities for reporting project results per GCF requirements.

Based on our review of official websites and publicly available documents of the entities, the authors have categorized entities according to their evaluation capacity, as follows:

• Level 1 capacity: entities with neither implementation experience with international organizations nor any evaluation capacities or have not reported on them would fall into such category. There are 19 such DAEs.
• Level 2 capacity: entities that do not have experience in undertaking evaluation but have experience in implementing projects. There are 15 DAEs in this category.
• Level 3 capacity: entities with experience conducting evaluations (as indicated in figure 3, finding 3) but do not have an evaluation office or dedicated evaluation staff or have not reported them on their websites (refer to limitations section below). There are 10 such DAEs. Of the 72 DAEs covered in this work, 27 entities (including 19 national entities) reported on their previous evaluation experience.
• Level 4 capacity: entities with previous evaluation experience and dedicated evaluation staff or office. Our desk research helped us identify 27 entities that have reported on their previous evaluation experience. Of these, 13 have dedicated staff, and 14 have evaluation offices.
• Level 5 capacity: entities that have all the three components of evaluation capacity, viz. evaluation office, dedicated evaluation staff, and previous evaluation experience. Of the 72 entities covered in this paper, 10 have all three components.

Conclusions regarding the capacity of entities and implications for evaluation capacity development of IEU.

Three overall conclusions can be drawn from the findings of this desk research.

1. Most DAEs may not necessarily have institutionalized evaluation structures to support evaluation capacities. For instance, they might undertake evaluations through existing staff members who also have evaluation responsibilities. Alternatively, they do it through programme staff, as indicated in findings 1 to 4. Thus, they may have differentiated absorption capacities for undertaking evaluation capacity development.

2. Most entities seem to have experience in implementing projects financed by international organizations. Thus, there is some experience in the procedures and expectations of international organizations, even if there is no experience in evaluations.
3. Where the institutions do not have any experience in implementing projects with international organizations, there is a distinct lack of reporting or actual lack of evaluation capacity and experience.
4. There is a discrepancy in the evaluation capacities between bank and non-bank institutions. This may be because most of the banks are private sector actors that do not follow the processes of international organizations and are driven by private sector priorities, such as viability and profits, instead of “development results”.

 

[1] Decision B.BM-2021/15 Updated Terms of Reference of the Independent Evaluation Unit.

[2] This is a desk based exercise and a more comprehensive DAE capacity needs assessment may be undertaken to understand DAEs’ evaluation capacities in depth and design appropriate capacity building programmes.

[3] This exercise is desk based and hence does not assess the quality of DAEs’ evaluation function. A more comprehensive capacity needs assessment may be undertaken in the future to better understand quality of evaluation function.

 

*Temurbek Zokirov, a former Evaluations and Capacity Assistant Consultant with the IEU, was engaged with the GCF’s Division of Country Programming when this blog was published.

 

Disclaimer: This guest blog was originally published by the author. The views expressed in this guest blog are the authors' own and do not necessarily reflect the views of the Independent Evaluation Unit of the Green Climate Fund.