Independent consultant to conduct an external evaluation of
the Judicial System Monitoring Program (JSMP)
The Asia Foundation (the Foundation) is implementing an ending violence against women program in Timor-Leste under a Grant Agreement with the Australian Department of Foreign Affairs and Trade. The program’s goal is to reduce the proportion of women who have experienced violence, and to better meet the needs of women and children affected by violence. The objective is to improve outcomes for women in prevention, support services, and access to justice by improving capacity in key ministries, reducing levels of violence, improving services, and strengthening judicial processes serving the needs of women survivors of violence.
Under the access to justice pillar, the program is providing core funding support and accompaniment to the Judicial System Monitoring Program (JSMP) to monitor the formal justice system in Timor-Leste. This activity is expected to contribute to strengthening judicial processes in Timor-Leste, serving the needs of women and girl survivors of violence. JSMP conducts regular monitoring of court hearings in all four district courts, produces case summaries, press releases and analytical reports on issues affecting the formal justice system, and conducts advocacy to improve the formal justice system. Further background information regarding JSMP, please get the complete ToR.
These terms of reference address the requirements for an external evaluation of JSMP’s program activities from January 2013 until present. Based on the findings and recommendations, the consultant will also be required to facilitate JSMP’s strategic planning for the next five years
(2016 – 2020). The evaluation will not review JSMP’s financial management systems and governance practices as these are being reviewed by an independent financial audit.
The Asia Foundation is looking for a consultant to:
1. Assess the quality of JSMP’s activities in the following areas: court monitoring & case summaries; legal research, analysis & publications; community training; advocacy and outreach; and assess the impact of JSMP’s activities against stated objectives/desired outcomes.
2. Review the efficiency of JSMP’s current organisational structure and management of its program activities.
3. Produce a detailed evaluation report with key findings and recommendations for JSMP’s future programming, over the next five year period.
4. Facilitate one workshop with all JSMP staff to introduce the evaluation findings and recommendations.
5. Facilitate strategic planning sessions with JSMP staff and assist JSMP management to develop a five year strategic plan (2016 – 2020).
1. External assessment of quality and impact
The evaluation will review the quality and impact of JSMP’s activities divided by the following program areas:
• Court monitoring & case summaries
This activity involves monitoring court hearings in the four district courts (Dili, Baucau, Suai and Oecusse) and producing short written case summaries describing the cases and decisions. Monitoring information is recorded on a monitoring form and then entered into an online database (WuFoo). Case summaries are produced in both Tetum and English, and distributed by email through JSMP’s mailing list. Significant decisions may also be publicised through press releases which are also distributed by email.
• Legal research, analysis & publications
This activity involves deeper analysis of thematic issues in the formal justice system as identified through court monitoring, such as the implementation of the Law Against Domestic Violence, sentencing in sexual assault cases, prosecution of cases of incest, and review of draft laws. Based on research and analysis of these issues, JSMP produces thematic reports (such as Incest in Timor-Leste: An unrecognised crime, 2012 and Law Against Domestic Violence: Obstacles to implementation three years on, 2013), submissions on draft laws (such as Improving the Penal Code to better protect women and children, 2015), and shorter Justice Updates.
• Community training
Community training is delivered by all units within JSMP and vary from local workshops with community leaders and members, to seminars for school students, to more formal events such as national seminars.
• Advocacy and outreach
This core component of JSMP’s program includes various activities, including distribution of press releases on significant cases, contributing to information on the justice sector through local newspapers, producing radio programs, and lobbying policy-makers on proposed reforms to the justice system. Publications such as submission on draft laws and analytical reports often form the basis of advocacy efforts.
It is expected that the evaluation will not be able to assess the quality of all activities delivered by JSMP from January 2013 to present, therefore a sample from each program area should be selected to inform a general assessment. The assessment should involve interviews with external stakeholders, such as formal justice sector actors, international donors, NGOs, government agencies and policy-makers/legislators.
In assessing the impact of JSMP’s activities, the evaluation will first define the objectives/desired outcomes of these activities, referring to both program documents (such as funding agreements with donors) and JSMP’s articulation of the broader objectives and desired outcomes (such as through interviews with JSMP and referencing JSMP’s strategic plan). Some key questions to be answered are:
a. Were JSMP’s activities delivered to a high-quality? If not, what were the factors effecting the delivery of these activities?
b. What has been the impact of the activities?
c. To what extent did the activities achieve the stated objectives/desired outcomes?
d. Were there any unexpected outcomes?
2. Review the efficiency of JSMP’s organizational structure and management
This will involve reviewing the current division of JSMP into the following units: Legal Research Unit; Women and Children’s Justice Unit; Parliamentary Watch Unit; Outreach Unit; and Management, Finance and Support Unit.
The key question to the answered is whether the current organizational structure best reflects the reality of JSMP’s day-to-day work, and whether changes should be made in order to improve the division of human resources, maximize expertise and efficiency, and respond to future priorities.
3. Evaluation report
The report will explain the key findings from the evaluation and make recommendations for JSMP’s future programming, over the next five year period. The report must contain a summary of the methodology used in conducting the evaluation, including a list of people interviewed and information analyzed. The report must be written in English.
4. Workshop with JSMP on evaluation findings and recommendations
The purpose of this workshop is to present the findings and recommendations to all JSMP staff, and provide an opportunity for discussion and feedback before finalization of the evaluation report.
5. Strategic planning sessions & Strategic Plan 2016 – 2020
The consultant will be required to facilitate one half-day strategic planning session with JSMP management staff, and one full-day strategic planning session with all staff. The strategic planning will respond to the findings and recommendations of the evaluation report.
Based on the discussions, the consultant will assist JSMP management to develop a five year strategic plan (2016 – 2020). The consultant will be required to develop the first draft of the strategic plan for approval by JSMP management.
The evaluation will use both quantitative and qualitative methods to assess JSMP’s program activities from January 2013 until present. The consultant will work closely with JSMP and the Asia Foundation to identify the key questions to guide the assessment.
At a minimum, the consultant is expected to:
• Interview key JSMP staff, particularly the Director, Deputy Director, Coordinators and international advisers.
• Interview at least 10 external stakeholders regarding their views on the quality and impact of JSMP’s activities, the efficiency of its current organizational structure and any recommendations.
• Analyze a sample of JSMP’s publications – including thematic reports, submissions, press releases and case summaries.
• Analyze data regarding the impact of community training and outreach activities (such as pre/post training questionnaires, radio program listener surveys, and written feedback from national seminars).
• Review relevant program documents (monitoring forms, funding agreements, progress reports, monitoring statistics and previous evaluation reports).
• Provide regular feedback to JSMP management and the Asia Foundation on the progress of the evaluation.
1. A workplan for the evaluation
2. Evaluation report – written in English, no more than 25 pages. The report should be written in an accessible manner, presenting information in a way that is useful to non-native English speakers. The report should outline the methodology used in conducting the evaluation, and list the people interviewed and data analyzed.
3. One workshop with JSMP to discuss the key findings and recommendations
4. One half-day strategic planning session with JSMP management and one full-day strategic planning session with all staff, focusing on implementing recommendations from the evaluation report.
5. Draft strategic plan for 2016 – 2020 – written in English, no more than 10 pages.
All documents will be provided to JSMP and the Asia Foundation in soft copy.
The Consultant will be supervised by Sophia Cason, the Organizational Strengthening Coordinator and Access to Justice Coordinator, the Asia Foundation.
Required qualifications and experience
• A university degree in a relevant discipline (law, social science, international development)
• A minimum of five years professional experience in a relevant field
• Previous experience in implementing and/or evaluating development programs in the area of justice, human rights or governance
• Demonstrated understanding of the justice sector in Timor-Leste
• Demonstrated understanding of evaluation tools, methodology and reporting
• Excellent analytical, communication and report writing skills, and
• Working level understanding of Tetum.
Core skills and attributes
• A personal commitment to promoting human rights and gender equality, and to modelling personal behaviours consistent with program focus
• Demonstrated ability to work effectively in a cross-cultural setting
• A commitment to working respectfully and collaboratively with individuals and organizations at a range of levels, and
• Strong oral and written communication skills.
Expression of interest
Expressions of interest must be submitted no later than 5pm Friday 29 May 2015 containing the following information:
• Cover letter outlining prior relevant experience and contact details of two referees
• Curriculum Vitae
Applications should be submitted to Dian Wahyu Saputri – email firstname.lastname@example.org.
Questions regarding this TOR may be directed to Sophia Cason, OIS and Access to Justice Coordinator, the Asia Foundation Timor-Leste. Email: email@example.com