Colombia: Final Evaluation Consultant

TERMS OF REFERENCE – EXTERNAL EVALUATION INDEPENDENT CONTRACTOR

A.Organizational Background

Heartland Alliance International (HAI) is the youngest and fastest growing part of Heartland Alliance for Human Needs & Human Rights, a family of organizations that has been leading anti-poverty and social justice work in Chicago for more than 125 years. HAI is comprised of nearly a dozen country offices implementing programs on a broad range of human rights issues globally, as well as the Chicago-based Marjorie Kovler Center for the Treatment of Survivors of Torture, which serves individuals from more than 50 countries. HAI has significant expertise in the fields of trauma-informed mental health care and access to justice for survivors of rights abuses. It is also an industry leader in access to high-quality and stigma-free health care. Across all of its programs, HAI promotes progressive, innovative approaches to human rights protections and gender equality.

B.Project Background

The Community-Based Treatment Services for Afro-Colombian Victims of Torture (ACOPLE) is a USAID-funded program implemented by Heartland Alliance International (HAI) together with local partners, Asociación Nacional de Afrocolombianos Desplazados (AFRODES) and the Cisalva Research Institute of the Universidad del Valle, in alliance with the US-based Johns Hopkins University (JHU). It started in October 2010 and was originally programmed to run for three years. It was approved for a cost extension (CE) in December 2013 that prolonged the implementation period to December 2015; after this period, the project was approved for another two-year cost extension through December 2017, and recently, for a final three-year cost extension through September 2021, which is the current project period.

This program aims to develop and implement evidence-based, effective, and culturally appropriate Mental Health and Psychosocial Support (MHPSS) services for victims of torture on Colombia’s Pacific Coast, expand holistic services to victims, and build the capacity of Afro-Colombian civil society organizations to address the needs of torture survivors during and after the project period. It does this through three self-reinforcing strategic objectives (SO). For the first three phases of the project, the three SOs were the following:

SO 1: The ACOPLE team provides technical assistance to the Government of Colombia to assess the prevalence and impact of torture on Afro-Colombian individuals and communities in two cities of Colombia’s Pacific Coast (Quibdó and Buenaventura)

SO 2: Provide context-specific, replicable, effective and sustainable community mental health services for 1600 survivors of torture and political violence

SO 3: Develop the long-term capacity of Afro-Colombian communities and municipal authorities to respond to the treatment needs of victims of torture

In the most recent cost-extension, SOs were altered slightly to focus on sustainability*:*

SO 1 Phase 4: National and territorial governments are better able to incorporate mental health and psychosocial wellbeing best practices into existing community rehabilitation programs.

SO 2 Phase 4:Survivors of torture and organized violence have improved mental health and psychosocial well-being.

SO 3 Phase 4: Communities affected by torture and violence are able to access continued community psychosocial rehabilitation services.

C.Evaluation Rationale

HAI seeks a Contractor to conduct a final evaluation of ACOPLE over the life of the project. The evaluation will measure the project’s effectiveness, efficiency, relevance, potential for impact, and potential for sustainability. HAI and the Contractor will collaboratively design the evaluation methodology to investigate research questions that include but are not limited to:

1. How did services provided by the CPWs and psychologists impact the mental health and psychosocial wellbeing of beneficiaries?

o Differentiate between individual and group interventions and office- and field-based services.

o Explore differences based on sociodemographic variables

o Use both qualitative and quantitative data (i.e., quantitative improvement in symptoms)

1. How effective was the project in leading to improved access to and quality of care for victims of torture/conflict both in terms of offering services and in building capacity/raising awareness in the public sector? What internal and external factors contributed to this impact?

2. How have CPWs been impacted through participating in this project?

o How effective has the CPW training been in terms of preparing individuals for providing services to participants? Which training elements are most important?

o In addition to skills and knowledge gained through formal training, how has participation in the ACOPLE project affected CPWs?

1. How effective have MHPSS trainings by project staff been at the individual and organizational/institutional level? How have individual trainees, organizations, or institutions used knowledge or skills through the life of the project?

2. What key achievements will be sustained once the project ends?

3. How inclusive/sensitive/adapted have ACOPLE services been for population subsets, including for both individuals who are and who are not accessing services? Is a key subset of the population with great need being under-served by ACOPLE?

4. Do the services offered through the project (individual, group, and specialized) meet the needs and priorities defined by project participants?

5. What has the impact of HAI’s technical and operational assistance to public entities been? How has ACOPLE’s methodology been adapted or referenced by government guidelines or programs?

The evaluation will be shared both with HAI and the donor and will help inform the implementation of HAI’s mental health programming both locally and worldwide. The external evaluation will complement HAI’s internal assessment of the project, and HAI will share the evaluation with staff, partners, and other stakeholders as necessary.

D.Evaluation Design & Methodology

The design and methodology applied to this evaluation should generate the highest quality and most credible evidence to answer the evaluation questions listed above, and should primarily focus on assessing project activities implemented under ACOPLE. Prior to the start of data collection, the Contractor will develop and present, for HAI review and approval, a written methodology that details the evaluation activities, evaluation tools, and data analysis plan for this evaluation. The written methodology should be guided by a participatory approach to evaluation, and will be comprised of a mix of quantitative and qualitative tools appropriate to the evaluation’s research questions. HAI expects the methodology to include, but not be limited to, the following activities:

• A desk review of relevant information on the status of MHPSS services for Afro-Colombian victims in Colombia, the social and political context of the target population, and strategies to measure community-based MHPSS interventions;
• A review of all relevant project documentation (in English and Spanish);
• A review of project data;
• Site visits;
• Data collection methods and tools, including a pre-selection of key informant interviews;
• Data analysis plan. Evaluation questions should be answered with information that is disaggregated by sociodemographic characteristics, location, and different types of services.

HAI will provide the evaluation team access to all relevant project documents, including but not limited to:

• The original grant agreement and subsequent modifications to the grant agreement;
• Quarterly reports, work plans and management reviews developed as part of routine project monitoring; and
• All data, including data collection tools, previously collected from routine project monitoring.

E.Evaluation Deliverables

• Evaluation Methodology & Work Plan: A written, draft methodology for the evaluation will be submitted to HAI for review and approval no later than the 7^th day following the signing of this agreement. The methodology plan should include the evaluation design, data collection plan, instruments, tools, and an operational work plan, including a timeline. The methodology plan will be discussed with HAI prior to any data collection activities. The Contractor may change the methodology but must submit any changes for approval to HAI prior to implementation.

• Evaluation Data: All raw data used in the evaluation analysis should be submitted no later than two weeks after data collection activities are completed. The Contractor will provide data in an electronic file in an easy to read format; organized and fully documented for use by those not familiar with the project or evaluation.

• Draft Evaluation Report: The Contractor will submit a draft report of the findings and recommendations from the evaluation no later than April 4, 2018. Prior approval by the Company will be required should the Contractor want to revise this deadline. The format of the evaluation report should follow the guidelines provided by HAI. The report must be submitted in English, electronically.

• Debriefing with HAI and partner staff: The Contractor will present major accomplishments and findings from the evaluation to HAI and partner staff through PowerPoint presentation after completion of field-based data collection. The debriefing will include a discussion of achievement and issues as well as any recommendation the team has for possible modifications to project approaches, results, or activities. The Contractor will consider HAI project staff comments and revise the evaluation report accordingly, as appropriate.

• Final Report: HAI will provide the Contractor with written comments on the Draft Evaluation Report no later than April 13, 2018. The Contractor will submit a Final Evaluation Report that incorporates HAI’s responses to the Draft Evaluation Report no later than April 27, 2018. The format of the evaluation report should follow the guidelines provided to the Contractor by HAI. The final version of the Final Evaluation Report will be submitted in English, electronically no later than May 4, 2018.

F.Evaluation Team Composition & Desired Qualifications

The Contractor will receive support from HAI’s project team, as well as HAI’s Monitoring & Evaluation team. The Contractor should possess the following qualifications:

• Fluency in Spanish and English preferred. Applicants who are fluent in Spanish only or English only will also be considered. Applicants who are fluent in English only must include costs for translation services by a certified translator for all data collection activities in the evaluation budget. Applicants who are fluent in Spanish only must include costs for translation of all deliverables into English by a certified translator in the evaluation budget;
• Outstanding oral and written communication skills;
• At least four years demonstrated experience in conducting evaluations and writing reports;
• Demonstrated experience evaluating MHPSS programming;

§ Demonstrated knowledge of the key challenges and actors associated with MHPSS services in Colombia;

• Demonstrated experience evaluating programs for gender inclusiveness preferred.

G.Evaluation Management

HAI will provide overall supervision to the Contractor during contract period. HAI will assist the Contractor with finalizing the evaluation methodology and work plan, and assist in identifying and providing key project documents for the Contractor to review during the contract period. HAI will provide overall logistical support and assistance to the Contractor, as needed and when possible. HAI’s project-based staff will be made available to the Contractor for consultations during the evaluation process.

The Contractor is responsible for arranging and completing all evaluation-related activities listed in the approved methodology and work plan, in coordination with HAI’s local project and logistics staff. The Contractor will advise HAI prior to each of those activities. The Contractor will be responsible for completing and submitting to the Company the evaluation products listed in Section E of this document. The Contractor will be responsible for procuring his/her own work/office space, computers, Internet access, printing, and photocopying throughout the evaluation. The Contractor will be required to make his/her own payments for such services. The Contractor will be responsible for any expenses beyond the fee provided by the Company as determined in this document.

H.Payment Schedule

Payment will be tied to deliverables. A final payment schedule will be specified in the final signed Contractor Agreement.

If the applicant is fluent in English and Spanish, the submission will be expected to be in English. If the applicant is not fluent in English, the application may be submitted in English, given that a budget for translation is included within the $20,000 budget. Incomplete submission packages and submission packages that do not adhere to the page limits stated above will not be considered. Present employees of HAI or HAI’s implementing partners need not apply. 9

I. Submission Process

To be considered for this contract, please submit the following to alblack@heartlandalliance.org:

1. A letter of interest, indicating how your skills and experience meet the qualifications listed above;

2. A résumé or CV (no more than four pages);

3. A writing sample (no more than three pages);

4. A draft budget, including consultation fees and all other evaluation-related expenses, including travel, with a maximum amount of $20,000 USD.

Do NOT submit an application through the Heartland Alliance career website. Such submissions will not be considered.