Terms of

Reference

(ToR)

 

Final

Evaluation

IGAD Promoting Peace and Stability in the Horn of

Africa Region (IPPSHAR)

Region: Horn of Africa

EU Contract Number: T05-EUTF-HOA-REG-36-02

ADA Project Reference number: 6545-00/2017, 6545-01/2017,

                                                            6545-00/2018, 6545-01/2018,

                                                            6545-02/2018

Name of Partner Organisation: Austrian Development Agency (ADA)

1. Introduction/Background

Sector context

Conflict and instability remain a significant challenge to the Horn of Africa region, impacted by political problems, demographic growth, forced displacement and migration, youth unemployment, lack of access to services, competition for scarce natural resources, and harsh climatic conditions, with increasingly frequent and severe droughts, floods, locust infestations and other disasters.

Project Background

The Austrian Development Agency (ADA) is the operational unit of the Austrian Development Cooperation (ADC) and is responsible for implementing all bilateral programmes and projects in ADC’s partner countries on behalf of the Austrian government.

ADA has been delegated to manage funding allocated by the European Union for the support of the Peace and Security Division of the Intergovernmental Authority on Development (IGAD) under the EU Trust Fund for Africa (EUTF), Horn of Africa window, for an Action titled “IGAD Promoting Peace and Stability in the Horn of Africa Region (IPPSHAR)”. The project, titled “Action” by the EU, is implemented by ADA in partnership with the IGAD Peace and Security Division (PSD) and its units and specialized institutions in the period from 24 March 2018 to 23 September 2023. For this purpose, an ADA team of experts is directly embedded in the PSD in Addis Ababa. The total Action budget is 28,4 m EUR, with 25,2 m EUR provided by the European Commission, 1 million EUR by the ADC, 1.14 million EUR by the Netherlands (NL) and 1 million EUR by Sweden.

The Action implementing partner, the Intergovernmental Authority on Development (IGAD) is one of eight Regional Economic Communities (RECs) under the framework of the African Peace and Security Architecture (APSA) with the goal to develop and stimulate a regional identity, attain peace and security, and alleviate poverty through sustainable development. IGAD Units and Offices of Special Envoys are entrusted with the task of implementing the Peace and Security Strategy and the strategic priorities designated to respective structures.

IPPSHAR contributes to the achievement of selected priorities of this strategy through its support for the IGAD Peace and Security Division and the following specialized units, initiatives and offices:

• Conflict Early Warning and Response Mechanism (CEWARN),

• Mediation Support Unit (MSU),

• IGAD Security Sector Programme (ISSP),

• IGAD Centre of Excellence for Preventing and Countering Violent Extremism (ICEPCVE),

• IGAD Foreign Service Institute (IFSI),

• IGAD Red Sea, Gulf of Aden and Somalia (RSGAS) Office, which covers the IGAD Task Force on the Red Sea and Gulf of Aden and the IGAD Special Mission to Somalia (ISMS)

• Peace facilitation offices: IGAD Office of the Special Envoy for South Sudan (OSESS)

Operationally, the Action is implemented through a Project Implementation Partnership Agreement (PIPA) between ADA and IGAD and a number of sub-grant agreements that address specific parts of the overall work plan. ADA, in its role as

Managing Organisation, awards, signs, and executes grant contracts according to its EU pillar assessed rules and regulations. In particular, ADA awarded and signed grants with IGAD where IGAD is the direct beneficiary, and where IGAD provides funds to other grant beneficiaries. The ADA team based in IGAD closely monitors the implementation of all grants in regard to quality assurance as well as to fiduciary compliance through vetting and special annual programme audits and field visits.

The Action has the objective of contributing to achieving sustainable peace, security and stability for the attainment of economic integration and development in the IGAD region. Advancing Peace and Security is one of the four pillars of the IGAD Regional Strategy 2016-2020 and its follow-up Strategy 2021-2025. The underlying assumption of the action is that peace, security and economic development in the IGAD region are profoundly interlinked. Therefore, by supporting peace and security, the Action is expected to contribute to economic integration and development.

The Action has four specific objectives:

Specific Objective 1 (SO 1): To enhance the IGAD Conflict Early Warning and Response Mechanism (CEWARN) systems so there is an improvement in the quantity and quality of the information collected, in data analysis, and early response action. (This includes a grant to the IGAD CEWARN Rapid Response Fund)

Specific Objective 2 (SO 2): To enable IGAD and national governments in the region to predict, prevent, and address transnational security threats. (This also includes a grant for the IGAD Centre of Excellence for Preventing and Countering Violent Extremism)

Specific Objective 3 (SO 3): To enhance the capacity of IGAD and national governments of the region on preventive diplomacy, mediation and civilian peacebuilding. (This also includes grants for the IGAD Mediation Support Unit (MSU); IGAD Special Mission to Somalia (ISMS); the Task Force for the Red Sea, Gulf of Aden and Somalia (RESGAS); and the IGAD Foreign Service Institute (IFSI))

Specific Objective 4 (SO 4): To enhance the implementation effectiveness of IGAD’s Peace and Security Division (PSD) and Units.

The Theory of Change (ToC) guides the Action’s strategic and logical framework, linking all four objectives described above with financing for IGAD’s core functions for Peace & Security Division and Units supported by the EU Salary Grant.

The Action is implemented through a Project Implementation Partnership Agreement (PIPA) between ADA and IGAD. This is supported by a project operational manual (POM), which lays out inter alia procedures for ADA ex-ante vetting of fund authorization followed by ADA ex-post controls for programme implementation including budget execution tasks carried out by IGAD. Under IPPSHAR, ADA also awards, signs and executes sub-grant contracts per its own EU pillar assessed rules and regulations. In line with the Description of Action (DoA), the ADA Project Team also supported IGAD in developing programme documents for grant funding.

2. Purpose and Objectives

This final evaluation is commissioned by ADA towards the end of Action implementation as foreseen in the Description of the Action (DoA) of the IPPSHAR Action. First and foremost, the evaluation serves to assess performance and thus is

summative in nature serving mainly the purpose of accountability. In addition, the evaluation is expected to contribute to learning for future programming and implementation. The evaluation will build upon the information and lessons derived from the mid-term evaluation undertaken in 2021 as well as undertake to gain additional input from stakeholders to address these questions. Insights gained should lead to clear recommendations for future engagements both for IGAD and the contributing development funding partners.

The main objectives of the evaluation are:

1. Determining the extent to which the Action has achieved its stated objectives and intended results detailed by the Action’s logframe, i.e., governmental (including inter-governmental) institutions, capacity and systems have been strengthened, etc. in the given context.
2. Determining the extent to which results achieved in all four specific objectives are sustainable or likely to be sustainable
3. Assessing to what extent cross-cutting issues and principles (such as conflict sensitivity, gender equality, environmental protection, social inclusion, equal and non-discriminatory participation) were applied and progress is evidenced in the Action results.
4. Issue evidence-based recommendations for future regional peace and security engagements between IGAD and diverse partners and stakeholders.

The evaluation team will focus on substantive content-related matters as many lessons regarding administrative issues have already been documented in previous discussions and the mid-term evaluation.

3. Scope

The scope of the Summative Evaluation will cover the timeframe from the Action’s start on 23 March 2018 until start of the evaluation or end of the implementation period (23 September 2023). The evaluation will cover the five OECD-DAC evaluation criteria relevance, coherence, effectiveness, efficiency and sustainability¹ and encompass the following areas of scope.

Target Groups

The main counterpart to the Action is the Intergovernmental Authority on Development (IGAD) Peace and Security Division (PSD), its units and specialized institutions, including the IGAD Conflict and Early Warning and Response Mechanism (CEWARN), the IGAD Security Sector Programme (ISSP), the IGAD Centre of Excellence in Preventing and Countering Violent Extremism (ICEPCVE), the IGAD Mediation Support Unit (MSU), the IGAD Offices for South Sudan and Somalia (ISSO and ISMS), the IGAD Foreign Service Institute (IFSI) and the IGAD Taskforce on the Red Sea and the Gulf of Aden (RESGAS).

The Action’s primary stakeholders are the IGAD Secretariat and its member states including both national and local governments of the region.

Additional stakeholders are organisations engaged in complementary efforts to promote peace and security in the region, or that have partnerships and links with IGAD. These include the African Union Commission, other Regional Economic

• Impact will not be assessed in this evaluation as it is impossible to make definitive statements about impact at this time (as the time horizon for impact in peace and security work is longer) an assessment of impact should be undertaken in 2-5 years following the end of the project implementation.

Communities (RECs), EU institutions and in particular the EU Delegation to the African Union in Addis Ababa, but also in the IGAD Member States, and United Nations agencies. This includes development partners such as ADA, which doubles as IPPSHAR’s management partner, and members of the Peace and Security Coordination Group and signatories to the Joint Financing Agreement (JFA).

The Action builds capacity and supports the work of NGOs and CSOs that support conflict prevention and peacebuilding efforts such as women and youth groups academia and research institutions. Communities affected by conflict are the ultimate beneficiaries of the Action.

Area of Coverage

The evaluation will be undertaken primarily in Addis Ababa and Djibouti and with field visits to each of the other IGAD MSs for data collection (insofar as possible considering security situations). The evaluation should engage stakeholders at the national level (MS governments and national level civil society), but also at local and grassroots levels (participants from local youth and women’s organizations, refugee communities, borderland communities), etc. For each MS visited, please plan to visit the capital and between one and two implementation sites outside of the capital. The selection of exact regions will be discussed and decided upon during the inception phase.

Furthermore, communication and exchange of information should be established with the ADA-IPPSHAR Project Team, the ADA Coordination Office in Addis Ababa and ADA Headquarters in Vienna and the management of the IGAD Peace and Security Division. The team will assist the evaluators in identifying relevant contact persons in these organisations mentioned above.

Scope and topic

Due to the complex design of the action, the evaluation should assess the four specific objectives of the action in the context of political and contextual realities of the geographic region and its political environment, including due consideration to the COVID-19 pandemic. The evaluation should focus on substantive results of the

Action at local, national and regional levels, while administrative aspects of project implementation should only be mentioned insofar as they may have impacted the results areas or have emerged since the completion of the MTE and therefore were not adequately addressed in the MTE. Recommendations should be as practical as possible and targeted to both the funding and implementing partners with an interest in funding similar work in peace and security with IGAD or in the Horn and well as IGAD as an implementer. The grants mentioned above will be evaluated separately and evaluation reports will be provided to the evaluation team as inputs to this work.

4. Specific Evaluation Questions

Relevance

Programmatic Relevance

1. To what extent was activity planning and prioritisation guided by research and evidence base throughout project implementation?
2. To what extent was the Action responsive to changes in regional political

dynamics (security and conflict situations, political transitions, institutional and member state relationships, external interventions/projects, the COVID-19 pandemic) and how did this affect implementation?

Policy Relevance

3. To what extent were the Action’s initiatives relevant and responsive to needs and demands from MSs?
4. To what extent was the Action guided by IGAD’s policy organs (The Committee of Ambassadors, Council of Ministers and Assembly of Heads of State and Government) employed in the Action?

Coherence

5. How effectively did IGAD PSD align its work internally to implement the Peace and Security Strategy (collaboration/synergy between PSD Units in planning implementation, etc.)? What contribution, if any, has the Peace and Security Coordination Group had on implementation of the Action and what is its potential for the future?

Effectiveness

6. To what extent has the programme achieved/is it likely to achieve its expected outputs and outcomes? Which IGAD units or programmatic approaches performed particularly well over the span of the Action? Were there any un-intended effects (positive or negative)?
7. To what extent were cross-cutting issues evident in the Action implementation

(gender mainstreaming, social inclusion, environmental mainstreaming/consideration of the conflict-climate nexus)?

8. What were the major factors – including factors beyond the control of the Action – influencing the achievement or non-achievement of the expected results

Efficiency

9. To what extent have interventions been cost/time effective since the mid-term evaluation? Have any issues emerged, if so which ones and why?
10. What has facilitated and/or hampered efficiency, if anything? How well were related challenges remedied since the mid-term?

Sustainability

11. To what extent are results achieved in strengthening MS capacities (both government and civil society) sustainable or likely to be sustainable? To what extent have studies supported by the action led to policy development, and to what extent have policies drafted under IPPSHAR been adopted and/or implemented by MS?
12. What have been IGAD’s strengths and weaknesses while implementing this Action? As a REC, how is it positioned to add value to peace and security interventions in the region?
13. To what extent are the staffing, systems and institutional reform processes supported by IPPSHAR incorporated into IGAD’s formal and informal working procedures? In particular, how have the efforts of this Action affected IGAD’s performance on the EU Pillar Assessment?
14. How was the Action’s ‘phased approach’ to sustainability implemented and how effective was it?
15. What factors hindered and facilitated programmatic and IGAD’s institutional sustainability in the Action?

 

5. Approach and Methods

The evaluator(s) should propose the methodological approach to be used to carry out the Summative Evaluation. The proposed approach should sufficiently address the issues and questions outlined within this ToR, specifying the, data collection and analysis methods that is suggested to answer the evaluation questions. It should encompass a combination of both qualitative and quantitative methods clearly described in the technical offer.

The approach should include a thorough document review, assessment of programme monitoring data, key informant interviews and focus group discussions with programme stakeholders and target groups. This will include field visits to IGAD project implementation areas.

The evaluation consists of several phases:

Contract and Kick-off meeting: Contract is signed, and a discussion of the assignment takes place. First documents, including available data, are provided to the evaluation team.

Desk Review: The evaluation team studies all necessary project documents; re-constructs and analyses the intervention logic/programme theory and theory of change and its assumptions. Existing data needs to be analysed and interpreted.

Inception Phase: In the inception report the evaluators will describe the design of the evaluation and elaborate on how data will be obtained and analysed. The use of a data collection planning worksheet or a similar tool is required. Data triangulation and quality control are very important and need to be discussed in the inception report. The inception report should include an evaluation matrix, a stakeholder mapping and a workplan.

Data Collection Phase: Data needs to be gathered, analysed and interpreted. It is expected that the evaluation will include quantitative and qualitative data disaggregated by gender and social group where feasible. Interviews may take place electronically when appropriate.

Data Analysis, Presentation and Draft Report: Analysis of quantitative and qualitative data in preparation for presentation of key findings and data validation (feedback workshop) at the end of the data collection period and subsequent report writing. Submission and presentation of final draft report, and subsequent incorporation of comments from partners and contractor.

Final Report: Submission of final report

The Guidelines for Project and Programme Evaluations developed by the Austrian Development Agency need to be considered throughout the entire evaluation process (link provided in Annexes).

6. Workplan and Deliverables

The consultants will submit the following deliverables:

1. Inception report (10-15 pages without annexes) An outline of the reports’ structure needs to be agreed upon during the inception phase.
2. Feedback workshop including presentation of preliminary findings, conclusions and recommendations and parameters for data validation prior to submission of first draft report. This workshop will be led by consultants for one day in Addis Ababa; participants will include the ADA team, funding partners and IGAD Coordination Team
3. Draft evaluation report (about 25-30 pages without annexes):

• Must include the results-assessment form (part of the reporting requirement)
• Must include a draft executive summary which summarizes the evaluation’s purpose, objectives, subject and methods and gives an overview of key findings, lessons learned and recommendations (max. Three five pages)
• The findings and recommendations of the draft report and final report should be structured according to the evaluation questions

4. Final evaluation report (25-30 pages without annexes), final executive

summary (max 3 pages), similar structure as above and the results-assessment form (part of the reporting requirements)

All reports must be written in English. The quality of the reports will be judged according to the criteria listed in Annex 5 and Annex 6 of ADA’s Guideline for Programme and Project Evaluations (See annexes).

Workplan and Timeframe

The estimated duration of the contract is 17 July 2023 to 28 February 2024.

A maximum total of 100 working days (combining work between two and four team members) is currently estimated for this assignment.

 

Action		Responsible		Date
Solicitation of bids		ADA		12 May
Submission of bid (electronically)		Contractor		By 16 June
		Contract signed
Contract signed and documents provided		between ADA and		Week of 17 July
		consultant
		Meeting between
Kick-Off meeting		contractor and		Week of 24 July
		consultant
Desk Study		Consultant		31 July – 25 August
Submission of draft inception report		Consultant		25 August
Providing feedback to inception report		ADA and Partners		28 August-15 September
Inclusion of comments and submission of		Consultant		18-29 September
final inception report
Field visits, data collection, data analysis and		Consultant		2 October – 8 December
drafting
Feedback workshop		Consultant		Week of 27 November or 4
				December
Submission of draft report		Consultant		8 December

 

 

Providing feedback for draft report	ADA and Partners	11 December- 5 January
Inclusion of feedback in final draft report	Contractor	8 – 26 January
Submission of final evaluation report (hard	Consultant	Submission: 22 January
copy and electronic copy) to contractor &		Presentation: Week of 22
Presentation at the IGAD/EUD premises		January

 

7. Evaluation Management Arrangements

Evaluation Locations, Financial and Logistical Arrangements

Evaluation management takes place mainly in Addis Ababa. For field visits, the locations will be discussed and agreed with the ADA-IPPSHAR Team. The evaluator(s) will be responsible to make their own arrangements for the field visits and appointments with relevant stakeholders that will be arranged in co-operation with the ADA-IPPSHAR Team and the IGAD PSD, if and when deemed necessary by the Evaluator(s).

The evaluation management needs to respect the ethical standards and guiding principles for evaluation, including impartiality and independence.

Budget and Payments

The total budget of the evaluation will not exceed EUR 65,000 excluding VAT. The proposed budget is to be considered an upper limit that will not necessarily be reached. Financial aspects will be included in the evaluation of the best bid. The contracted sum includes the evaluator’s remuneration and all expenses for the services described above, including for example, travel, board and lodging, communication expenses and (potential) interpretation costs and will be paid in instalments linked to the deliverables being accepted.

Payment shall be affected in a lump-sum payment as per the payment schedule comprising 3 (three) instalments, based on the key deliverables. The last instalment will be paid upon delivery of the final Evaluation Report.

 

Inception Report		20% of the contract sum
Draft  Evaluation  Report,  including  a  draft	executive	40% of the contract sum
summary and the result-assessment form (RAF)
Final Evaluation Report, final executive summary and the		40% of the contract
result-assessment form (RAF)

 

Publication and Processing of personal data

The Contractor agrees that the following information will be made public as required by the General Conditions to the European Community Delegation Agreement: title and type of the contract name, address and nationality of the contractor, and amount of the contract.

During initiation and performance of contracts, ADA may process personal data² of natural persons that are collected by ADA or transferred or disclosed to ADA by prospective contractors or third parties under their instruction, e.g., personal data of employees, legal representatives, agents or other partners of the prospective contractors or such third parties.

Coordination/Responsibility

Shimljash Braha, Monitoring, Evaluation and Reporting Officer for ADA-IPPSHAR, will be the main contact person for this evaluation.

Contact details:

• Monitoring, Evaluation and Reporting Officer IPPSHAR: Shimljash Braha (Braha@ada.gv.at) and
• Carbon copy (cc:) to Team Leader IPPSHAR Project: Isabella Lederer (lederer@ada.gv.at)

8. The Evaluation Team

The evaluation team will consist of between two and four members (while the offer should indicate which tasks can be covered by which evaluator). Gender diversity in the team will be an advantage.

Key qualifications in the team should be:

• Sound experience in evaluation exercises with the European Union and/or bilateral aid agencies such as ADA
• Relevant academic degrees (master’s level) in public policy, peace and conflict or security studies or other relevant social sciences
• Sound and proven knowledge and working experience in the field of peace and security as well as relevant security structures in the Horn of Africa region. A minimum of five years’ experience and expertise in the Peace and Security field
• Team leader has conducted at least five evaluations in the last five years ideally in the relevant field
• Team member(s) has/have participated in at least three evaluations ideally in the relevant field
• Knowledge of the region with focus on topics such as public policy, peace and conflict, intergovernmental organizations
• Experience in organizational development and project cycle management
• Experience preparing and analysing a theory of change
• Experience in capacity development evaluation is an advantage
• Experience and expertise in cross-cutting issues and principles (gender equality, equal and non-discriminatory participation, environmental sustain-ability)
• Experience in social science methods including qualitative data collection, planning and conducting semi-structured interviews
• By submitting information to ADA, you, as a prospective contractor, acknowledge:

• to have taken note of ADA’s Privacy Notice https://www.entwicklung.at/en/media-centre/privacy-notice

(’ADA Privacy Notice’);

• to ensure that each direct or indirect transfer or disclosure of personal data to ADA during the initiation or performance of a contract is lawful pursuant to applicable data protection law;
• to ensure that all persons, whose personal data are transferred or disclosed to ADA, were promptly and demonstrably provided the ADA Privacy Notice; and
• that if a contract is concluded and in accordance with its terms, ADA publishes, in particular on the ADA website, information about the contract and the contracting parties.
• Excellent oral and written English skills and knowledge of local language(s) (either through a national expert or a translator)
• Sound MS Office and IT skills

The consultants must not have been previously involved in the design, implementation or monitoring of this project.

9. Specifications for the Submission of Offers

The applicant(s) is/are requested to submit a technical and financial offer (using template attached) which will be evaluated with weights assigned at 80% for the technical offer and 20% for the financial offer. The package should consist of:

A technical offer of max. 8 pages, including:

• A cover letter with expression of interest and a brief description of team composition, relevant previous experiences in evaluations, qualitative data collection and planning and conducting semi-structured interviews
• A proposed methodology, presenting the overall approach and methods suggested for the conduct of the evaluation
• A work plan, with estimated working days for each phase of the evaluation and division of tasks between the team members
• A brief Curriculum Vitae (CV, in EU standard format) of each team member, including references

A financial offer in the template attached (max. 2 pages), including:

• Expert fees incl. estimated number of working days for each team member;
• Other expenses, including travel expenses
• VAT, if applicable

Submission deadline:

Austrian Development Agency (ADA) invites eligible and qualified interested experts to submit their non-binding proposal/applications in English language indicating “Final Evaluation IPPSHAR” in the subject line via email to addis.application@ada.gv.at by Friday, 16 June 2023 by 23:59 EAT (Addis Ababa time) and 22:59 CEST (Vienna time). Tenders submitted in another form or to another e-mail account will be excluded from the procedure.

10. Annexes

1. ADA Evaluation Policy

2. ADA Guidelines for Programme and Project Evaluations3
3. Results-Assessment Form, to be filled in by the evaluation team4
4. Logical Framework and M&E framework of the project5
5. EU Standard CV Format6
6. Financial Offer Template7
7. Service Contract Template

The following documents will only be made available to the successful bidder:

• Contribution Agreement between ADA and the European Commission, in particular Annex I Description of the Action and Annex III Budget in amended version following Addenda
• Grants and mid-term evaluation reports
• Project progress reports and documents
• Reports from the monitoring missions, monitoring data and participants lists
• Research, visibility and learning products produced by the implementing partner under the Action

Download attached ToR Document in PDF Below

IPPSHAR Final Eval ToR_Final

Download attached IPPSHAR ToR Annexes Below

Annex 4.1 IPPSHAR Logical Framework

Annex 4.2 IPPSHAR M&E Framework

Annex 6 Financial Offer Template

Annex 7 Service Contract Template