Consultant

 

  

Terms of Reference (ToR)    End Project Evaluation (ECHO– 2013/14)

          

  

 

  

 

 1. About the Project:

Afghans in Pakistan assess their future options in terms of how best to reduce their vulnerability, and how best to increase their security, achieve economic stability and improve their livelihood opportunities. They consider return a possibility (18.2% according to the 2005 Census) only if peace and security is assured in their proposed destination in Afghanistan. The clear presence of economic opportunities - not only jobs, but access to land and the promise of sustained livelihoods that employment presents - makes the decision to return more likely. Furthermore, GoPs changing priorities, lack of institutional capacity and emerging protection issues of Afghan refugees in Pakistan have pushed them in more vulnerable situation. 

The displacement caused by the conflict in Afghanistan has affected Pakistan and Afghanistan for more than three decades. Despite the UN having undertaken what must be one of history largest return operation – since 2002 with the return of more than 5,6 million Afghans –figures of millions still in displacement under precautious conditions from a protection and a humanitarian needs perspective, making three out of every ten refugees on the globe an Afghan.

Voluntary repatriation as viable solution, willingness of the GoP to further accommodate Afghan refugees in Pakistan (Extension of PoR cards till End of December, 2015), humanitarian support and awareness about potential of voluntary return are some of major protection concerns adding to already marginalized conditions of Afghan refugees living in Pakistan.   

 The Afghan refugees both in rural and urban areas are facing  issues including but not limited to; access to legal assistance and legal remedies, awareness on potential for voluntary return, Extremely Vulnerable Individuals (EVIs) and individuals with special protection needs (IPAs-Individual Protection assistance), access to livelihood opportunities and lack of government institutions capacity.   

 Recognizing that the absorption capacity in Afghanistan for receiving returnees is incapable to guarantee a safe and dignified return, it is deemed that legal protection activities (as mentioned above) have to be addressed as matter of priority to assure the respect of the international refugee law. Moreover, although voluntary repatriation is considered by the GoP and UNHCR as a prioritized durable solution for the Afghan refugees, the right to a dignified life is deemed to be addressed in both, the country of origin and the country of asylum.

 DG-ECHO and Danish Refugee Council initiated a project in July, 2013 in order to improve access to legal assistance, protection and promote durable solutions for Afghan refugees in four KPK Districts – Pakistan - on the basis of humanitarian principles and human rights. The project is due to be completed  in August, 2014 and requires an external evaluation. The specific objective of the project is to reach out 21,309 refugees in seven refugee villages of Haripur, Mansehra, and Peshawar to improve current social status of Afghan Refugees and increase their awareness for sustainable voluntary return by means of undertaking a series of measures aiming at:

• Capacity building/training of the government officials and relevant IPs on refugee legislation, asylum space and alternative stay arrangements.
• People in the targeted refugee villages of Haripur and Mansehra and Peshawar urban areas have increased awareness on protection and legal assistance including voluntary repatriation procedures and other stay arrangements.
• Extremely vulnerable individuals of the targeted refugee villages receive legal assistance through IPAs and information/guidance on other legal documentation (PoR card, birth registration, PCM etc) 
• Targeted communities have increased knowledge of the situation in their place of origin through the beneficiaries of Go and See visits and make informed decision for voluntary repatriation. 

 2. Purpose of the Evaluation;

The overall purpose of the End Project evaluation is to assess the degree to which the ECHO funded project has achieved the principal objective and immediate result as outlined in the LFA. The evaluation will particularly emphasize on evaluation criteria i.e. relevance, effectiveness, efficiency, impact, and sustainably of the interventions and extract the lessons/recommendations to enhance the quality of programme.  

In order to assess the evaluation questions below, the evaluator will require emphasizing on the objectives/indicators as provided below:

2.1. Problems and Needs (Relevance);

• Are the objectives and results of the programme still valid? Were the objectives relevant to the context, has the context changed?
• Which activity showed greater relevance for the beneficiaries? Why?
• How  relevant and appropriate was the project approach and the processes adopted (Awareness for voluntary repatriation and other durable solutions, Information dissemination through IEC materials, identification of EVIs for IPAs, Go-and-See visits, legal assistance through legal clinics and field visits, Coordination/cooperation by relevant Govt line Departments etc) ?
• Were the most vulnerable targeted appropriately? Were the criteria and indicators defined in the project suitable to identify the most vulnerable population?
• Was the methodology used for the beneficiary selection relevant and were communities involved in the selection process? Were the selection criteria communicated and understood by all members within the community?
• Under which criteria the different activities implemented in the different locations were selected (for instance in some communities information dissemination in schools, in others no information dissemination but legal assistance…)Were the activities proposed per community the most appropriate ones within the different possibilities at DRC's disposal?
• Were actions in the intervention areas well enough coordinated among themselves and with other actors to prevent duplications and avoid gaps?
• Were the needs assessment, monitoring, evaluation systems and associated indicators appropriate?
• Is there any evidence of change needed in the programme design or implementation strategy so that the objectives/results of the programme could be better achieved?  

 2.2. Achievement of Purpose (Effectiveness);

• Did DRC implement the project as planned to meet the project goals and needs of the target group? (with respect to all components of the programme i.e. Information dissemination, Legal assistance etc) ?
• Were the proposed activities the most effective way to achieve the intended outcomes (for instance GSV to increase communities’ awareness, etc.)? Were there alternative approaches that would have been more cost-effective without affecting quality ?Did DRC achieve outputs with respect to the agreed timeline (‘on the track’) and had effective access to all intended beneficiaries throughout the project implementation timeframe?
• Did DRC achieve the outcome indicators – results? Can this be measured and how?
• Did the project progress according to envisaged targets/objectives? How could it be improved?
• How effectively beneficiary participation was sort and included throughout the program? Had DRC put mechanisms in place to reach effectively more indirect beneficiaries?
• How were the specific needs of women and girls taken into account in the project design and implementation? What gaps can be identified in addressing those needs?
• How effective was DRCs complaints and feedback mechanism?  Did the mechanism successfully influence management decisions? What was the follow-up given to the complaints?

 2.3. Sound Management and Value for Money (Efficiency);

• How far funding, staff, time and other resources were managed contributing to or hindering the achievement of the results. Was ‘Value for money’ achieved?
• How well did the project management arrangements work? How well did the financial and procurement systems work in ensuring financial accountability?
• Was the project sufficiently monitored by senior management to ensure a good implementation and the greatest possible impact?
• What difference is expected in the lives of those targeted (Refugees & Returnees)?
• Did DRC increase protection and legal assistance (including voluntary repatriation procedures and other stay arrangements) in target communities?  Beyond the direct beneficiaries, what impact on the wider RV population?
• Did the Go and See visits have increased knowledge of the situation in their place of origin (in comparison with other existing info sources)? How is this measured?
• Did this knowledge affect the target groups’ decision-making with regard to  protection responses and repatriation?
• How well has DRC reinforced existing protection mechanisms?
• Does the project address the intended target group and what was the actual coverage, factoring-in referral and follow up mechanisms, too?
• How does the project engage with poorest of the poor and vulnerable communities? Did DRC reach the correct/planned target group?
• What unintended consequences were caused by or contributed to by the intervention and why did these occur?
• What are the prospects for the benefits of the project to be sustained after the interventions? Did the response reduce future vulnerabilities of targeted groups?
• Was the capacity of stakeholders and duty bearers improved by the training components? Will this capacity be sustained in relevant institutions?
• How was the exit strategy defined, and how this been managed at the end?
• What could concretely be done to ensure sustainability of the action and linkages with other programs?
• Were lessons and recommendations from DRC previous interventions and evaluations incorporated? If so, how are they working/if not, why not?
• Are there any significant changes required in the implementation strategies? What are the major reasons for these?
• How has the design of the project been amended as a result of lessons learned during implementation? What challenges encountered during the implementation and what mitigation strategies were adopted?
• How these lessons can be applied in the next phase of programming? Can lessons be drawn for other similar DRC interventions in the region or globally?

2.4. Achievement of Wider Effects (Impact);

• What difference is expected in the lives of those targeted (Refugees & Returnees)?
• Did DRC increase protection and legal assistance (including voluntary repatriation procedures and other stay arrangements) in target communities?  Beyond the direct beneficiaries, what impact on the wider RV population?
• Did the Go and See visits have increased knowledge of the situation in their place of origin (in comparison with other existing info sources)? How is this measured?
• Did this knowledge affect the target groups’ decision-making with regard to  protection responses and repatriation?
• How well has DRC reinforced existing protection mechanisms?
• Does the project address the intended target group and what was the actual coverage, factoring-in referral and follow up mechanisms, too?
• How does the project engage with poorest of the poor and vulnerable communities? Did DRC reach the correct/planned target group?
• What unintended consequences were caused by or contributed to by the intervention and why did these occur?

2.5. Likely Continuation of Achieved Results (Sustainability);

• What are the prospects for the benefits of the project to be sustained after the interventions? Did the response reduce future vulnerabilities of targeted groups?
• Was the capacity of stakeholders and duty bearers improved by the training components? Will this capacity be sustained in relevant institutions?
• How was the exit strategy defined, and how this been managed at the end?
• What could concretely be done to ensure sustainability of the action and linkages with other programs?

2.6. Lessons Learned;

• Were lessons and recommendations from DRC previous interventions and evaluations incorporated? If so, how are they working/if not, why not?
• Are there any significant changes required in the implementation strategies? What are the major reasons for these?
• How has the design of the project been amended as a result of lessons learned during implementation? What challenges encountered during the implementation and what mitigation strategies were adopted?
• How these lessons can be applied in the next phase of programming? Can lessons be drawn for other similar DRC interventions in the region or globally?

2.7. Recommendations:

Recommendations for improvements based on observations during the evaluation process will be extended. Those recommendations will be used by the stakeholders to evaluate the implementation of the project (DG-ECHO, DRC Afghanistan and Pakistan, Government line departments and IPs) to enhance the effectiveness of the programme and brining lasting changes related to Afghan Refugees and returnees.

The evaluation is expected to provide key examples of Best Practices from the project/programme. This example(s) should relate to the technical area of intervention, either in terms of processes and systems, and should be potentially applicable to other contexts where DRC operates.

 

 3.Scope of the Assessment;

The geographical focus of the evaluation will be the target area the ECHO programme undertaken by DRC in Hazara districts of Pakistan i.e. Haripur, Mansehra and Peshawar. The security situation in these locations will be a consideration for the evaluation team, it is expected that access will be possible to all programme areas.

While the focus is on the ECHO supported actions the evaluation should also consider the interface with similar programmes funded by other donors.

 

4. Methodology;

The evaluation will be based on the following methodology: 

• Desk review – reviewing existing data reports and documentation. Build on information that is already available, after rapidly checking its present validity and relevance. Gather information from scratch only if particular information is lacking. Make the maximum use of existing information. 
• Joint initial review meeting – the M&E department of DRC Pak will make arrangements for all meetings. The joint review meeting will be facilitated by the lead evaluator. The purpose of the meeting is to review and discuss all cycles of programme implementation in terms of achievements, challenges and lessons learnt as above. Facilitated by the lead evaluator the meeting will review the terms of reference and proposed work plan to agree an approach and operational plan for the evaluation. 
• Primary data collection from the target communities: A Mixed Method Approach will be adopted by the lead evaluator to collect both quantitative and qualitative data for the evaluation. The evaluator with the support of field staff will collect first hand data from sampled project staff (Community Mobilizers, Field officers, PMs/PCs), stakeholders (including government officials), target beneficiaries, through interviews and Focus group discussions. Data will be analyzed and will be the basis of the final report findigs.

 5. Expected Deliverable: 

Final Report: The evaluator(s) will produce/submit a report in hard and soft form (of no more than 30 A4 pages plus annexes, in Microsoft Word) in English. The report should include:

• Basic Information (1 page)
• Executive Summary (2 - 3 page)
• Introduction/Background of the project (2 page)
• Evaluation methodology
• Findings from the evaluation will be presented under the headings  detailed under section 2 above
• Summary of recommendations/lessons (Specific, simple and doable recommendations)
• Annexes - Evaluation ToRs, Evaluation schedule, List of persons interviewed and sites visited, Documents consulted, Data collection tools and detailed analysis
• Data Tables, Graphical representation, List of districts and refugee villages visited, Questionnaires, List of people met Program, List of people met IPs, List of Government officers met. List of community people met/interviewed (will ensure maintain Gender balance) with PoR/CNIC numbers and Mobile numbers.

 Presentation of evaluation findings and recommendations: The evaluator will present the evaluation findings and the recommendation to a wider group of participants from DRC, Pakistan and DG-ECHO, Pakistan prior to finalisation of the report.

 6. Role and Responsibilities;

• Evaluator:
• Design methodology for the evaluation (in accordance with the Program components/indicators).
• Design data collection instruments
• Get feedback from the evaluation committee on the methodology and the data collection tools
• Selection of Sample size together with evaluation committee
• Guidelines IDIs & FGDs (Qualitative and Quantitative Tools)
• Orient field researchers on the tools and the methodology
• Lead data collection process
• Arrange debriefing and share the initial findings
• Produce final report and present the findings and recommendation to the wider group. 
• DRC Pakistan
• Take lead in managing the evaluation
• Arrange kick off review meetings
• Arrange a meeting to present the findings
• Develop and finalize ToRs together with DG-ECHO
• Participate in selection of evaluator
• Critically review and  share feedback on the methodology and tools developed by the evaluator.

8. Timeline:

The evaluation is expected to start from first week of September and will be completed by the end of September. Please see the table below as a guide line for the task (this will be reviewed with the consultant during contracting):

Sr. No.	Activities	Timelines
1	Preparation   and familiarization with the overall context (Review of the documents)	2 Days
2	Joint   Kick off review meetings	1 Day
3	Travel   and field visits to refugee villages in Pakistan	14 Days
4	Data   collating, cleaning, analysis and review	2 Days
5	First   draft Report	2 Days
6	Review/Presentation of findings to the   Evaluation Committee	1 Day
7	Feed   back of the committee on the first draft after review/presentation	2 Days
8	Final   Report drafting with Amendments	2 Days
9	Submission   of the Final Report
Total Days of   assignment                                                                                  26 Days

Important Note:

DRC does not assume any responsibility for the evaluation except specific support stated above. All staff assigned to the action has to follow DRC security regulations. However, DRC does not assume any responsibility for the security or safety of people assigned to the evaluation.

DRC maintains its explicit right to formally disassociate from the findings provided by the evaluation in case of disagreement or different interpretation.

Education
Must	Degree	Degree Level	Country	Description
	IR/Development Studies/Other relevant	Masters Degree	Pakistan	•Post-graduate degree in IR, Development Studies, and/or related field •A solid and diversified experience in Pakistan for Refugees and Returnees in thematic areas of Protection and having good knowledge of refugees and international human rights. •The evaluator will have a credible profile of evaluation of humanitarian and refugee programmes in line with the Sphere and other internationally recognized standards •Experience in the use of participatory methodologies and developing gender sensitive evaluation methodologies; •Working and knowledge of Pakistan and Afghanistan context will be an added advantage •Proven record of undertaking Midterm evaluations /End Project Evaluations of similar nature projects/programs of UN agencies, EU and other International Donor organizations.

Skills
Must	Title	Level	Description
	Communication & Interpersonal	Excellent	•Excellent report writing skills, •Fully conversant with the principles and working methods of project cycle management