BREN Project Final External Evaluation

Request for Quotations (RFQ) - Terms of Reference 

Based: Remote - travel not required (preferred in one of the project countries: Armenia, Azerbaijan, Georgia, Moldova) | Availability to work in UTC+4 timezone
Reporting to: Program Manager with oversight of the IWPR MEL Specialist
Status: Consultant contract
Call published: August 14, 2024
Assignment implementation period: November 2024-March 2025 

1. Background

The Institute for War & Peace Reporting (IWPR) is an independent not-for-profit organisation working to empower local voices on the frontline to drive positive change in areas of conflict, crisis or transition around the world. For over 30 years IWPR has been promoting reliable information and public debate to build more stable, just and inclusive societies.  

Building Resilience in Eastern Neighbourhood (BREN) project (2021-2025) is implemented by the Institute for War & Peace Reporting (IWPR) and the Global Network of Women Peacebuilders (GNWP). The project aimed at strengthening the resilience of non-state actors and women’s rights organisations to enhance their ability to deliver a transformative, inclusive and sustainable contribution to peace, stability, and security in four countries: Armenia, Azerbaijan, Georgia, and Moldova through more than 40 sub-grant organisations. BREN is supported by the UK Government’s International Security Fund (ISF).  

2. Objectives of the evaluation

The overall objective of this evaluation is to provide a final review of the Building Resilience in the Eastern Neighbourhood (BREN) project by assessing its impact according to the OECD-DAC criteria and other specific criteria. This final evaluation has three main objectives:

• To assess and analyse the achievements of the project implementation, compared to the expected results and in line with the theory of change that underpinned the design and ambitions of the project. This will involve determining the current level of achievement of the project's indicators (for project objectives and results) and comparing the final achievements with the baseline situation. As for the theory of change, the aim will be to examine the hypotheses of the causal links between actions and results and to confirm or disconfirm the explicit or implicit causal steps of the theory of change.
• Capture observable changes in practices, behaviour and attitudes adopted by individual actors in the civil society sector, by targeted stakeholders including communities, state actors, internationals bodies and any other individuals and structures that are reached directly or indirectly in the activities through direct implementation and sub-granting; highlight the relevant achievements and lessons, with a view to informing the continuation of the project and/or the development of other following projects.
• To assess objectively, quantitatively and qualitatively, the relevance, efficiency, effectiveness and impact of the project interventions by the end of the project, and the sustainability of the results achieved, following the international evaluation criteria of the OECD/DAC.
• To identify and document good practices / project successes that are recommended for replication to enable scale up/out into new initiatives.

3. Evaluation Methodology

The evaluation methodology will be proposed by the consultant in their technical proposal. The consultant will have to propose robust methodology to:

• Test the validity of the project logic (theory of change statement and result framework) and its underlying assumptions, including the analysis of results and outcomes achieved and the logic chain linking interventions to results;
• Assess the impact, relevance, efficiency, effectiveness, coherence and sustainability of the project interventions, as well as the management of the project including the coherence and coordination within project partners and complementary projects.

Baseline and progress MEL data have been collected and reported quarterly by the project staff in accordance with the donor-approved logframe with relevant indicators. They include periodic project reports, event reports, subgrantee reports with monitoring data, content production lists and outreach data, mentors’ reports etc. Additionally, midterm evaluation has been conducted, with all its findings available.  IWPR has also piloted a new self-assessment tool CSO Resilience Index, piloted results of which will be available for final evaluation period. Data produced by sub-grants includes, but is not limited to, monthly and final narrative reports with the most significant change stories of their direct beneficiaries (project participants). IWPR summarises all the achievements monthly in the project data tracker, which is designed per project outcomes and outputs.  

In addition to the available qualitative and quantitative data, the evaluator will review project documents such as Theory of Change, logical framework, proposal and more.

The consultant will combine quantitative and qualitative approaches as necessary to adequately address the evaluation objectives and answer the different evaluation questions mentioned below. Preferred qualitative evaluation methodologies (but are not limited to):

• Outcome harvesting - where the OH can’t be applied fully, the consultants will be asked to develop outcome statements for significant results achieved;
• Impact stories of change for a selected number of project beneficiaries;
• Case studies to provide an in-depth analysis of specific instances of project implementation, illustrating the processes, challenges, and impacts through detailed narratives;
• Compare and integrate findings from applied methodologies to provide a holistic view of the project’s impacts.

The consultant(s) will be asked to update a qualitative evaluation database, store relevant impacts/outcomes, lessons learned, recommendations. The database will be available to conduct qualitative database analysis, consultants are welcomed to use it.

The proposed methodology should include the following phases:

• Preparatory phase: review of all project documentation, updated analysis of the project implementation context, preparation of a work plan; outlining the evaluation questions kick-off call with staff, donor (as appropriate), propose the evaluation methodology and the development of data collection tools and a sampling methodology. As a result of this phase the consultant will have to submit a plan/inception report, outlining the approach. Upon sign of the report the consultant(s) may initiate the work.
• Fieldwork: conduct data collection, consolidate, validate, triangulate project existing data. Conduct interviews, surveys, workshops to gather data as appropriate. Recruit and train local data collectors (if needed). Note any travel requirement should be outlined in the plan and in the agreed overall budget and will be allowed only after appropriate security checks.  
• Analysis of qualitative and quantitative data: analyse existing and gathered data, triangulate it and, if contradictions appear, explore and present hypotheses as to why such contradictions might exist; preliminary findings will be presented to the project team and the evaluation committee, and the consultant will need to take into consideration the project team's feedback.

4. Deliverables

Some key deliverables must be submitted in draft form to IWPR and the evaluation committee before being finalised as indicated in the table below. All deliverables and data from the evaluation will be owned by IWPR and may be used for internal and external reporting and communications.

• Deliverable 1: Proposed plan and methodologies for the evaluation, including agreed evaluation questions and case studies development, to be approved by the project manager and MEL specialist.
• Deliverable 2: Conduct online meetings, workshops with IWPR team, field staff and other key stakeholders (beneficiaries, donor) to capture, document, triangulate results.
• Deliverable 3: Draft evaluation: submit to the evaluation committee a draft evaluation and related annexes that outline the main findings with related evidence, including recommendations and lessons learned. Visualisations and case studies are encouraged.
• Deliverable 4: Final evaluation report and summary: in Word format, to be produced in English, should be a maximum of 25-30 pages (excluding annexes) and should include summary of the evaluation findings: 2 pages max with overview of overall achievements/performance and key actionable recommendations.
• Deliverable 5: Evaluation dataset: A database (excel) with list of outcomes, significance, contribution and related categorization used for the analysis.
• Deliverable 6: Case studies Up to two case studies (2 pages max each) to support “scaling up of success”. Throughout the evaluation the consultant(s) will identify up to 2 successes (either good practices, techniques, results etc) and, upon IWPR team confirmation, the consultant will document them, providing sufficient evidence and guidance to support uptake in new programming. Each case study should be 2 pages max.
• Deliverable 7: Debriefing session and presentation (7-12 slides max) with major findings and recommendations shared with key audiences at the end of the consultancy.

5. Reporting

A proposed full evaluation structure is outlined in the Annex 1.  

The final full evaluation report should provide with following information and analysis:  

1. Overall assessment of the level of success of the project as per DAC criteria: coherence, efficiency, impact, effectiveness, relevance, sustainability.
2. Beneficiaries reach: overview of beneficiaries reached directly and indirectly: number, type, gender, country and any other relevant disaggregation and involvement in project intervention.
3. List of significant outcomes achieved, intended as observable changes in actions, relationships, policy or practice of both beneficiaries and external actors linked to the project activities, categorise them using existing performance indicators, by DAC criteria and other relevant categories (e.g. gender, type of beneficiary, date etc).
4. Outline significance and relevance of each outcome in light of implementation context at national, regional and international level.
5. Outline contribution of the project for each outcome.
6. Outline key actionable recommendations for future programming in the light of current needs (national/regional context), project goal and project successes, good practices and lessons learned.
7. Outline significant challenges and/or negative changes that have limited or affected achievement of success, analyse effectiveness of mitigation strategies applied (if any) and level of impact of those challenges. Outline lessons learned for future implementation.
8. Update progress as per project performance matrix/MEL framework (baseline, targets and year by year progress).
9. Include visualisations (graphs/tables) and quotes as appropriate to support deeper understanding of evidence and analysis.

6. Timeframe and budget

Assignment should be implemented between December 1, 2024 and March 28, 2025. The consultant(s) expression of interest should present an estimate of budget that cannot exceed 10,000 GBP. Figure 1 provides the envisioned work plan:  

7. Ethics and data protection

The assessment will be conducted in accordance with IWPR policies on non-discrimination, conflict sensitivity, data protection, safeguarding and ethics. All relevant policies will be communicated to the consultant(s) with their contract. It is expected that the consultant(s) will demonstrate in their proposed methodology how they will address non-discrimination, data protection, safeguarding and ethics, and conflict and gender sensitivity.

8. Quality assurance

Evaluation Committee: An evaluation committee will be established consisting of IWPR staff (Program Manager, MEL specialist and other relevant staff). Implementing partners and donors will be involved by the committee as appropriate at the kick-off and completion of the effort to ensure validity and utilisation. The committee will accompany the process and ensure the quality of the analysis and methodology proposed in the inception report, and its implementation after submission of the preliminary results and a draft of the final report.

Data quality: Quality data are data that are not systematically biased and do not distort representativeness or coverage. A data quality assurance plan proposed by the consultant will consider anything that might go wrong with the data collection in advance and develop a strategy to prevent these problems. The plan should be shared in the inception report. This data quality assurance plan will focus on three elements: data collection tool/survey design, field management of data, and high frequency monitoring.

9. Skills and experience required

We are looking for a consultant/team of consultants/international or national consultancy firm meeting the following requirements:

• A minimum of a Master's degree in a relevant field (political science, sociology, international relations, conflict analysis and management and similar disciplines);
• At least 8 years of professional experience in the above-mentioned fields with proven experience in conducting quantitative and qualitative research, in particular in conducting evaluations of complex, multi-stakeholder projects in the areas of peacebuilding, media, conflict and/or organisational governance and theory-based evaluations;
• Experience with quantitative data analysis software or the ability to subcontract work (if quantitative analysis method is proposed);
• Excellent writing skills in and English. Knowledge of any local languages Armenian, Azerbaijani, Georgian, Romanian will be considered an advantage;
• Excellent report writing skills;
• Knowledge of local socio-cultural, economic and political context and related international geopolitics;
• Knowledge and experience of gender and conflict issues and application of suitable evaluation methodologies;
• Excellent facilitation skills and ability to work in multi-cultural environment;  
• Proven ability to design and implement work ensuring safety, security and do no harm principles;
• Ability to work and engage within all BREN implementing countries.

10. Application process

Applicants interested in this tender may submit to IWPR the following:

1. A letter of interest outlining professional experience relevant to this evaluation;
2. Two evaluation reports from a previous project of a similar nature;
3. Up-to-date CVs of the proposed consultants for this evaluation;
4. Two (2) referee contacts of the organisations for which the consultant or firm has carried out recent evaluations for similar projects (optional);
5. Estimated budget broken down by categories including consultants' working hours fee, translation/interpretation services, and other logistical needs. Note: The total budget must not exceed 10,000 GBP.

Please note: shortlisted candidates will be expected to prepare a short evaluation proposal for this consultancy (maximum 5 pages) if selected for final consideration. This will include a technical proposal including the description of the methodology and a financial proposal (number of days and daily rate). This is not necessary for the initial application process, and will be requested only from shortlisted candidates.

We encourage questions and clarifications; however, these should be asked by September 10, 2024. 

Questions and clarifications and the EoI submission should be sent to the following emails: Tamari@iwpr.net, Ana@iwpr.net. With the subject heading BREN Final Evaluation.