Terms of Reference
Mid Term Review (MTR) Consultant: Safe Motherhood Project
ToR Title Mid Term Review Consultant
Duration 15 days
Location Dili and field locations
Reports to Health Program Manager, CITL
Start date February 2016
CARE is an international organization working to end poverty and social injustice and bring about lasting, sustainable change in communities. CARE works in over 80 countries around the world to help overcome poverty by supporting development projects and providing emergency relief. A non-religious and non-political organization, we know that achieving gender equality, increasing inclusive governance and strengthening community resilience are critical in achieving sustainable outcomes.
In Timor Leste, CARE’s goal is to improve the wellbeing and voice of women and girls in rural disadvantaged areas as we know they are the most disadvantaged members of society. We do this through our thematic priority areas of sexual reproductive and maternal health and rights, education, women’s economic empowerment and women’s voice. CARE uses a number of approaches including the Gender Equality Framework, engaging men and boys, working in partnership, robust monitoring evaluation learning and impact and a commitment to accountability and inclusion.
SAFE MOTHERHOOD PROJECT
CARE International in Timor Leste (CITL)’s Safe Motherhood Project (SMP) focuses on Women of Reproductive Age and their families, with the goal of reducing maternal mortality and morbidity in nine sucos of Timor Leste in Ermera and Covalima municipalities. The key project outcomes are:
- Iimproved SRMH behavior and utilization of services
- Improved access, provision and quality of SRMH services
SMP is more than half way through four years of implementation and CITL has decided to undertake a Mid Term Review (MTR) of SMP. The purpose of the MTR is to review the performance of the project to date in line with the project design document and monitoring evaluation and learning plan, and to assess the project’s ability to achieve its outcomes by project completion (June 2017). It also seeks to evaluate the effectiveness and relevance of the transition from the original ‘Healthy Villages’ project after the first year of implementation to the current ‘Safe Motherhood project.’ The MTR will address what is going well and what progress has been made, what can be done better and what revisions may be required, and what lessons can be taken away to inform future programming and other relevant stakeholders. The MTR will also consider how the project can consolidate outcomes to date, and orientate strategies and activities towards sustainability of results in the post-project period.
The MTR provides the opportunity to gather input from CITL staff, local partners, and project participants on CARE’s management approaches, progress towards objectives and areas for improvement. Results of the MTR will be used by CITL to inform the Year 4 work plans, exit and sustainability strategies of the project, share any lessons learned with stakeholders in country and to inform Long Term Program level priority areas as appropriate going forward.
- Objectives/Purpose/Key Responsibilities
- Assess the overall performance and achievements of the project to date in line with the objectives and outcome statement and consider the potential for long term impact
- Review current scope of project strategies and their interrelationship, considering sustainability, relevance and effectiveness including evaluating of the change in objectives and targets from Healthy Villages to Safe Motherhood
- Assess the appropriateness and effectiveness of the gender strategies undertaken, and learn how women’s empowerment can be better supported, assessed and reported in line with CARE’s women’s empowerment framework and transformative approach to gender equality
- Assess the effectiveness, quality of different pilot models tested within SMP, such as Social Analysis and Action and the Mutual Accountability process
- Build capacity of the CITL co-lead in evaluation design, implementation and reporting
- Expected Results or Outputs
By the end of the MTR, the following deliverables are expected:
- Fully developed methodology and tools including sampling and site selection, guidance documents and details of planned analysis and use of statistical tests with justification/explanation on choices made (to be submitted and approved by CITL prior to the training)
- Training session with MTR team on use of the tools
- Debrief meeting/workshop with CITL management and MTR team to calibrate initial findings
- Evaluation report and summary (as per CITL format to be provided)
- Learning Brief (as per CITL format to be provided)
- Key Evaluation Questions
The MTR must be guided by the SMP MEL plan and the key evaluation questions therein:
- How effective and relevant were the approaches such as establishing Mother Caring groups, establishing Kitchen Gardens, conducting the Mutual Accountability process and training doctors and midwives and theories of change in this context in achieving the project’s goals?
- Is SMP adapting the project activities in response to lessons learnt through project implementation?
- What immediate impacts and changes around knowledge, attitude, skills and behaviors surrounding SRMH practices and gender relations have resulted from the project and how do these contribute to long term impact (LTP) and transformation in Gender Equality (GEF)?
- How do the results contribute to the goal and specific objectives of SMP?
- Is SMP ensuring that program inputs (e.g., materials, services) are of good quality and timely?
- Are donor resources sufficient?
- How efficient was the project in terms of budget, staffing, time line and use of Community Mobilisers and what was learnt around best practices in implementation?
- How sustainable and enduring are the outcomes of the project, especially Mothers’ Caring Groups, Kitchen Gardens and ongoing dialogue between community and service providers, likely to be?
- To what extent has SMP supported equal access by women, men and people with disability to the benefits of activities, and more broadly to resources, services and skills (e.g. by increased access directly, or by removing barriers to access)?
It is expected that the lead for the evaluation will propose the methodology and the tools to be used that are best suited to the project and context. These can then be fully developed and finalized with the co-lead and inputs from the project team. The methodology proposed for the MTR should employ qualitative and quantitative methods to collect data for understanding project performance and quality. The quantitative data should provide a snapshot of the project progress and it is recommended this is undertaken through a desk review of existing data prior to undertaking field work. The field work can then focus on qualitative data which should add richness and context to the quantitative outcomes and provide an assessment of overall engagement by beneficiaries with the project. This methodology would allow the MTR to look beyond immediate results to understand how and why things are happening (or not) in the community. The evaluation process should be participatory, incorporating a cross section of key stakeholders, including but not limited to, beneficiaries, local government institutions, SMP’s local partner, SMP’s technical partner, donors (DFAT), and CITL project staff.
The MTR will be conducted in three phases between January 2015 and March 2015, with a field work component in February 2015.
Phase 1: Desk Review and Design (3-4 days)
From early January to end of January, the consultant will conduct a desk review of relevant project documentation and existing work undertaken, such as project design, MEL and theory of change documents, annual reports and work plans and secondary data sources. This review will both inform responses to the key evaluation questions and also help to finalize the methodology and tools for the field work. The final methodology, tools, sampling (including site selection), planned statistical analysis and proposed schedule should be designed and submitted to the CITL team for feedback and approval based on their experience and knowledge of the communities and project. The proposed methodology should include details of the analysis that will be undertaken using the data collected, including the most relevant statistical tests with an explanation of the choice of tests and methodology chosen. All data and findings should be disaggregated sex, age and marginalized groups. Tools for this MTR could include: questionnaires; key informant interview guides/outlines for various stakeholders; detailed field guides for community focus group discussions; most significant change interview guide. Once the methodology and tools are finalized, training must be conducted with all staff involved on how to use the tools, data integrity and timelines.
Phase 2: Field Work (9 days)
From early February 2016 – mid February 2016, the MTR team will undertake the data collection component in the field. This team will comprise of the lead consultant, CITL Co-Lead, temporary data collectors and potentially a representative from CARE Australia. The team will split into 2 groups of 4 members to cover the two municipalities. At the end of the field work the team will reconvene in Dili for an initial analysis workshop to compare findings. The MTR team should undertake the following activities:
- Community-level consultations at the aldeia level with key stakeholders
- Surveys/FGDs/interviews with project participants
- Interviews with stakeholders in Dili as relevant
- Preliminary data cleaning and analysis
- Mini workshop on emerging findings
Phase 3: Reporting (2-3 days)
During the field work findings from the desk review should be cross-referenced and triangulated with responses in the field as part of the initial analysis and emerging findings workshop. The third phase will follow this to include any follow up discussions where required, full analysis and report writing, development of learning brief(s) and any adjustments based on feedback from CITL.
Whilst the SMP has coverage over 44 aldeias the sample frame for the study should be determined by the parameters of the baseline in order to maintain continuity. In addition to the baseline, aldeias should be chosen based on variety of factors such as population of the aldeia, distance between the capital city of the municipality, hospital and market access etc. It is recommended that some aldeias included in the baseline are revisited as part of the MTR field work. The criteria and the reasoning for the choice of the aldeias should clearly be listed within the evaluation report and should be shared during the post data collection workshops.
TRAINING AND CAPACITY BUILDING
This evaluation ToR includes a specific objective to build the capacity of CITL staff, specifically the co-lead, in evaluation design, planning, implementation and reporting. It is not expected that the co-lead will be an ‘assistant’ but an equal member of the team who will be able to input into the design process, learn from the experience of the evaluation lead and better understand the process of managing an evaluation. Furthermore the lead consultant is expected to provide specific guidance and on the job mentoring.
The MTR lead and co-lead should spend 1-2 days training the rest of the evaluation team on the implementation of the tools to be used during the data collection. Both women and men from the communities will be involved, thus it is important to consider the gender balance of the evaluation team. This training may be undertaken in Dili or the field (or a mix of both) in order to test the tools. In addition to a detailed review of the tools, training should include a review of the purpose of the MTR, gender concepts and terminology, data management protocols, and tips on building rapport, ensuring privacy, and obtaining the highest quality data through appropriate phrasing.
FLEXIBLE AND RESPONSIVE DESIGN
Evaluation is not a static process, and there is no practical way to anticipate all of the tasks necessary to produce good results. It is important that the process be allowed some room to maneuver, in order to evolve as the situation dictates. Therefore, it is proposed that the evaluation remains a flexible, dynamic process, and that CITL and the MTR team work closely together to be responsive to the workload constraints and time demands of project beneficiaries and staff and local partners. In consultation with CITL, the MTR lead should develop a review process that is flexible, acknowledging that from time to time alterations in schedules, unanticipated interviews and visits, and adjustments to the overall MTR plan will be necessary.
- Management and Reporting Arrangements
The MTR consultant will be directly managed by Grishma Bista, the Health Program Manager, who will provide all the relevant information for the consultancy and will arrange for support from other staff where required.
The following staff is also expected to participate in the input and feedback of the evaluation process at different times:
- Peter Raynes, Country Director, CARE Timor Leste
- Saad Karim, Assistant Country Director Programs
- Fanny Juneida Bernardo, Finance Manager
- Grishma Bista, Safe Motherhood Project Manager
- Samantha Fox, Program Quality Manager
- Floriano Carvalho, Impact and Partnership Officer
- Safe Motherhood project team
CITL will arrange and provide all transportation and logistical support for the duration of the MTR team’s field visits and workshops (i.e. training and analysis workshops) and any other relevant meetings in Dili. CITL is responsible for managing field logistics such as accommodation and meals for the MTR team.
- Schedule and Dates
The proposed schedule for the MTR is as follows:
|21st December – 8thJanuary||Advertisement of ToR|
|15th January||Identification of consultant|
|18th – 29th January||Phase 1: Desk review and development of full methodology, tools||3-4 days|
|1st – 12th February||Phase 2: Field work, training and workshops||2 days training
5 days field work
2 days follow up
|13th – 29th February||Phase 3: Reporting, analysis and final submission||2-3 days|
For this MTR, a lead with the following skills and qualities are required:
- Technical understanding and knowledge about Sexual Reproductive Maternal Health
- Previous experience of having conducted an evaluation including designing the methodology and creating the tools
- Contextual knowledge of Timor Leste (preferred)
- Strong research and analytical skills
- Strong project and time management skills
- Excellent ability to communicate with stakeholders including staff
- An orientation towards collaboration
- Strong writing skills
The consultant will be expected to sign and adhere to CARE’s Code of Conduct and Child Protection policy. Interested applicants should respond to this ToR with a letter outlining their interest, availability and daily rate and a recent CV to Grishma Bista, Health Program Manager (Grishma.Bista@careint.org) and Sam Fox, Program Quality Manager (Samantha.Fox@careint.org) by 5pm Sunday 14th February 2016.