TERMS OF REFERENCE – CONSULTANCY
TITLE: Consultant for Final Evaluation of Conflict Mitigation through Community Oriented Policing in Timor-Leste (CMCOP II) Project in Timor-Leste
In 2008, The Asia Foundation (“the Foundation”) began partnering with the Polícia Nacional de Timor-Leste (PNTL) on a pilot project to mitigate conflict through community oriented policing (COP). The Foundation received initial financial support for the pilot in 2009 from the United States Agency for International Development (USAID). In 2012 the Foundation received additional funding from USAID to launch a large-scale COP project entitled Conflict Mitigation through Community Oriented Policing in Timor- Leste (CMCOP II) in several districts of the country. The USAID funding will end in January 2016.
The Foundation was able to attract co-funding for the project from the New Zealand Ministry of Foreign Affairs and Trade (MFAT), Australia’s Department for Foreign Affairs and Trade (DFAT), and the United Kingdom’s Department for International Development (DfID). This multi-donor project was rebranded in Tetun as Hametin Kooperasaun Hamutuk Polisia ho Komunidade (HAKOHAK). USAID committed a third tranche of funding in 2014 to further expand the project’s scope and geographic reach; the project is now operational in 11 out of Timor-Leste’s 13 districts and around 30% of the country’s sukus (villages). USAID’s funding for the CMCOP II project will come to an end in January 2016. In total, USAID awarded the Foundation $5,032,285 for support of the CMCOP pilot and CMCOP Phase II project from 2009-2015.
As citizens of the newest independent nation in Asia, the people of Timor-Leste have high hopes for the PNTL as a vanguard of peace, stability, and justice. However, the PNTL is still a nascent institution with limited resources, institutional knowledge, experience, and capacity to adequately address local security needs. Community-oriented policing aims to improve the performance of police and build public trust by fostering collaborative community-police partnerships that use a problem-solving approach to respond to the security needs and expectations of local communities. One of the main program interventions under CMCOPII/HAKOHAK has been the formation and training of multi-stakeholder suku (village) level Community Police Councils (CPCs) that discuss local problems, set local safety and security agendas, and coordinate efforts to resolve, refer and prevent security issues. CPCs aim to address community problems in a proactive rather than reactive manner.
The goal of the CMCOPII project is to improve security in Timor-Leste by strengthening collaboration between citizens and the police through the achievement of four objectives:
1) Strengthen the technical capacity of the PNTL, civil society, and community leaders to implement effective community-oriented policing practices;
2) Build community-police partnerships to reduce high-risk threats to security;
2a) Increase the capacity of the PNTL to more effectively prevent and respond to high priority security concerns, particularly the security needs of women and girls in Timor-Leste.
3) Promote COP as a practical approach to policing within the PNTL, district administration, and local communities.
The theory of change for the CMCOP II project is that establishing active state-community security models at the suku level, and building those examples into institutional reforms to develop proactive safety and security approaches, will contribute to strengthened state-society relations and a more stable environment in Timor-Leste.
The Foundation is currently in the design and budgeting phase for continuation of the HAKOHAK program for an additional five-year period, which will be funded by the New Zealand Aid Programme, and possibly other donors as well. Due to the fact that USAID-funding for HAKOHAK (i.e., CMCOP II) is coming to an end, and in light of the Foundation’s plans to extend the HAKOHAK program through a final phase, the Foundation is commissioning an end-of-project evaluation of CMCOP II. The main audience for the evaluation report will be The Asia Foundation, USAID’s Timor-Leste mission, and key CMCOP II implementing partners. The results of the evaluation will be used to inform future approaches to community-oriented policing in Timor-Leste.
The key objective of this evaluation is to assess the program’s achievement of its intended goal, objectives, outcomes and outputs in accordance with the indicators agreed with USAID.
In addition, the Foundation expects the evaluation to consider the project’s:
- Relevance: the extent to which the project and activities are suited to local and national development priorities and organizational policies, including changes over time.
- Effectiveness: the extent to which the Program objectives have been achieved.
- Efficiency: the extent to which results have been delivered with current resources provided.
- Results: the positive and negative, and foreseen and unforeseen, changes to and effects produced by the activities; results include direct project outputs, short-to medium term outcomes, and possible longer-term impacts.
- Sustainability: the likely ability of the intervention to continue to deliver benefits for an extended period of time after the completion.
Scope of work
The Consultant will:
1) Conduct an initial review of CMCOP II project-related documents and reports provided by the Foundation.
2) Meet with Foundation staff to discuss and come to agreement on the overall evaluation process and key questions.
3) Prepare an evaluation workplan that articulates the process and timeline for the evaluation, including a description of the evaluation methods that will be used. The consultant should propose a clear plan for analyzing and triangulating data from various sources to generate high quality and credible evidence to answer the evaluation questions.
4) Examine the following questions (to be refined at the start of the evaluation, as necessary) regarding the CMCOP II project:
- To what extent has the project achieved its intended goal, objectives, outcomes and outputs in accordance with the indicators agreed with USAID?
- To what extent has the project strengthened the technical capacity of the PNTL, civil society, and community leaders to implement effective community-oriented policing practices? Specifically, which skills have been strengthened, and which skills need additional attention in the future?
- How strong and sustainable are the community-police partnerships that have been built through the project?
- Has the project increased the PNTL’s capacity to prevent and respond to high-priority security concerns, particularly the security needs of women and girls in Timor-Leste?
- To what extent do the PNTL, district administration, and local communities understand COP principles, view it as a practical approach to policing, and buy into the concept?
- Has the project contributed to measurable improvements in the security situation in target sukus?
- Among all project interventions, which have been the most effective and least effective in achieving the project objectives?
- To what extent has the project institutionalized sustainable systems, attitudes, and structures for COP in Timor-Leste?
- Were there any unintended or negative consequences or results of the project interventions?
- What are key lessons learned?
- What are the implications for future COP programming? How should future COP investments be focused?
5) Prepare a draft evaluation report that includes an Executive Summary, a description of the evaluation process and methodology, a detailed presentation of findings, recommendations for the Foundation to consider as it implements the final phase of the HAKOHAK program, and appropriate Annexes (e.g., list of documents reviewed, list of individuals interviewed).
6) Present the core evaluation findings at a meeting of Asia Foundation management and project staff and other appropriate stakeholders invited by the Foundation.
7) Make final revisions to the report based on feedback from the Foundation.
The Foundation envisions a mixed-method evaluation approach that will include document review, data analysis, interviews, and possibly focus groups with program participants and other parties. By using both quantitative and qualitative approaches, the Consultant will gain insight on the outputs and potential impacts of both project activities and processes. The Foundation will provide the Consultant with an initial set of program documents (e.g., CMCOP proposals, workplans, monitoring and evaluation plans, progress reports; existing relevant evaluation reports; training manuals) and a list of key informants for interviews. The Consultant is expected to expand on those materials and informants during the course of the evaluation. The bulk of the evaluation workdays are expected to be spent in Timor-Leste.
Level of effort and timeline
Up to 25 days of work between October 15 and December 15 2015, with a suggested allocation of workdays among Tasks as outline below.
|1||Initial review of documents||3|
|2||Initial and subsequent meetings with the Foundation on evaluation questions and process||1|
|3||Preparation of evaluation workplan and setting up of meetings||2|
|4||Evaluation research in-country||10|
|5||Evaluation report drafting||6|
|6||Report presentation and de-brief||1|
|7||Evaluation report edits and finalization||2|
Significant experience managing and evaluating development projects, ideally with experience in the area of community security, local justice systems, or police development. Advanced degree in relevant field. Experience conducting high-quality project or program evaluations, particularly those funded by USAID. Proven ability to conceptualize an evaluation methodology and processes and to present the findings in a clear, concise manner. Strong writing skills. Excellent cross-cultural sensitivity and communication skills. On-the-ground work experience in Timor-Leste (or strong knowledge of the country context) and Tetun language skills highly desirable.
Expression of interest
Interested candidates must to submit the requirement as follows no later than 14th October 2015 at 5PM to Dian Wahyu Saputri, via email email@example.com
- A written application
- An updated CV
- Examples of past work (research contributions)
- Budget proposal
Questions regarding this TOR may be directed to Sarah Dewhurst – Program Manager, CMCOPII, The Asia Foundation, Timor-Leste via email: firstname.lastname@example.org.