Sustainable Multi-Actor Solutions to Migration-Related Child Poverty in the Greater Mekong Sub-Region
SUSTAINABLE MULTI-ACTOR SOLUTIONS TO MIGRATION-RELATED CHILD POVERTY IN THE GREATER MEKONG SUB-REGION
TERMS OF REFERENCE
October 2011
1.0 Background information
1.1 Introduction to the evaluation
Save the Children is commissioning an external, final evaluation of the 3-year programme: sustainable multi-actor solutions to migration-related child poverty in the greater Mekong sub-region. The programme, funded by the EU, started in January 2009 and will end in December 2011. The purpose of the evaluation is to evaluate the effectiveness and efficiency of the programme, in terms of improving the protection of children. This learning will be used to inform the future of the regional programme and the strategy for this work going forward. The evidence generated about what has worked and what has not worked will also inform global learning across Save the Children’s child protection portfolio.
The Greater Mekong Sub-region (GMS) is experiencing rapid economic growth, driven mainly by China, Vietnam and Thailand. However, local economic disparities and regional economic integration reinforce migration and challenge fragile Child Protection Systems within each country and across the region. Children are part of large and increasing movements of population in the GMS. While many children receive minimal protection services in their home or place of origin, their entitlements and the likelihood of accessing them are reduced as they move: adolescent girls, lone migrant and stateless children, children left behind and returning children face particular risks of violence, abuse, exploitation and poverty. Protection policies and legislation in the 5 countries have not been designed to take into account migrant children. Comprehensive protection policies, laws and implementing services are often absent. Where they exist, national governments, Local Authorities and Non-State Actors do not have the capacity to implement them effectively. Levels of community and civil society capacity also vary. Thailand has the most developed civil society. In China, Myanmar and Vietnam responsibility for social policy belongs to the government and state institutions. Recognising that migration cannot be prevented and may even be a positive choice for children and their families, Save the Children initiated the Cross-Border Project (XBP) in 1999 to promote safer migration and to combat trafficking, exploitation and worsening poverty. This 3-year EU-funded intervention is the 4th phase of the Cross-Border Project. It builds on previous work and relationships developed with local NSAs, civil society and LAs in Cambodia, China, Myanmar, Thailand and Vietnam.
1.2 The Programme
For over a decade, Save the Children has been striving to address abuse and exploitation of migrant children in the six countries in the Mekong Region. Save the Children’s response started through a Participatory Action Research Project in 1999 along the borders of China, Myanmar, and Thailand. The report “ Small Dreams Beyond Reach” outlined the struggles of migrant Burmese, Chinese, and ethnic Thai children, and resulted to country based action programmes in Thailand, Myanmar, and China and in-country action researches in Lao PDR and Vietnam, all addressing the issue of trafficking and emphasizing on prevention and protection of victims. These gave rise to the idea of establishing a coordinated regional initiative. In Myanmar, the project started to work in selected townships in three States in Myanmar – Mon, Kayin, and Northern Shan- that were known for high rates of risky migration. Participatory researches were also started in Lao PDR, Vietnam, and in Guangxi Province in China. Initial actions for victim protection were started in Thailand and Myanmar to support identification of trafficking victims and supporting their repatriation. This period was considered as the first phase of the Regional Cross-border Project and established the potentials of the project for grassroots work with communities and children.
The second phase of the Regional Cross-border Project (XBP2) was the “Cross-border Community-based Initiatives Against Trafficking in Children in the Mekong Sub-region” that was implemented from January 2002 to March 2006 (including a 3 month no-cost extension), funded by the DfID Asia Regional Poverty Fund, covering all the six countries of the GMS. The project was focused mainly on looking at communities of origin and aimed to help reduce the incidence and impact of trafficking and abuse of the most vulnerable migrant children, working on four major outputs: communities are able to mobilise and address the causes and effects of child trafficking, policies are able to protect rights of trafficked children and address safe return, awareness and knowledge raised among children and their families and communities, and children in communities of origin are able to enjoy greater education and livelihood opportunities before and after migration.
The third phase of the Regional Cross-border Project (XBP3) was the ‘Cross-border Project Against Trafficking and Exploitation of Migrant and Vulnerable Children’ that was implemented from April 2006 to March 2009, with funding from DfID Asia Regional Poverty Fund. This phase of the project was designed to align the anti-trafficking work in the Mekong with Save the Children’s broader programme of work on children on the move. The third phase of the project developed the work to focus more centrally on improving access to, and the quality of, local and national care and protection systems. The third phase of the Cross-border Project took on important lessons from phase 2, recognising that the migration of children cannot be stopped given the economic, social, and political trends in the region. Increasingly the focus was on how to promote safe migration and child protection systems that support and protect children as they move.
The fourth phase of the Project, which is the focus of this external evaluation, has been implemented from April 2009 and will finish in December 2011. This phase of the Project titled, sustainable multi-actor solutions to migration-related child poverty in the greater Mekong sub-region, and funded by the EU, is being implemented in China, Vietnam, Myanmar, Cambodia and Thailand, and at the regional level. The project targets 16,000 children across the 5 countries. The final beneficiaries also include parents, community members, LAs and NSAs engaged in work with migrants as well as regional and global agencies or networks with particular mandates on population flows.
The overall objective of the project is: National Child Protection systems serve as real mechanisms to reduce child poverty levels
The specific objective is: Child protection systems which work for the interests of children affected by migration become embedded into government structures and policies in the GMS to locally appropriate extents
The expected results are:
1. Target vulnerable children and young people are empowered as active members of the community, capable of engaging in the protection of themselves & other children
2. Communities affected by migration take concrete actions for the protection of children
3. LAs & NSAs better able to fulfil their responsibilities for children affected by migration
4. Local child protection models inform the development of more robust, higher level (provincial, national) child protection systems
5. Project experience influences regional and global fora on migration and child protection
Key activities include:
• Organising children’s groups, including migrant children’s groups
• Peer education, awareness raising activities and other child led and participatory community activities
• Regional Youth Empowerment and Leadership trainings and dialogues
• Organising and support for community-led child protection schemes
• Organising and training of adult community volunteers on child protection
• Capacity building for project partners on child rights, child protection, participation; safe migration; trafficking victim protection; and project management
• Organising and development of child protection mechanisms/systems at district/township and provincial levels
• Support for trafficking victims on identification, reporting rescue, recovery, return, and reintegration
• Support for safer migration services
• Supporting children's access to birth registration, citizenship, and legal status
• Supporting children's access to basic services including education, recreation, health and others
• Use of database system for children served by child protection system
• Policy Analysis on budget allocation to child protection system across Mekong Countries
• Support for the creation and development of higher level child protection system
• Policy research, monitoring and analysis
• Capacity building for NSAs/Las responsible for child protection
• Support to COMMIT and to its Sub-Regional Plan of Action
• Advocacy with Association of South East Asian Nations and with the Global Forum for Migration and Development
• Advocacy with the five project GMS governments on strengthening child protection systems
• Regional research on migration of children and on the statelessness of children
• The Mekong Youth Forum
The Project is implemented in 5 countries and at the regional level: Thailand, China, Vietnam, Myanmar and Cambodia. Across these countries the project is implemented in 45 Target sites across 22 Provinces/State/Division (5 provincial level, 28 township/district level and 12 ward/commune level).
Given the depth and history of the programme, and the considerable breadth and scope of its objectives and activities, this evaluation is an opportunity to independently analyse the relevance, efficiency, effectiveness, impact and sustainability of the programme against its stated objectives; and provide strategic direction to Save the Children and other partners to guide future implementation.
1.3 Programme structure and framework
The Project is managed overall by a regional team based in Bangkok. Within in each country, the project interventions are implemented by a variety of project partners including state actors (China and Vietnam), local NGOs (Thailand and Cambodia) and Save the Children staff itself (Myanmar).
2.0 Scope, objectives and outputs of the evaluation
This final evaluation will provide evidence and learning of achievements, outcomes, challenges and lessons learned in implementing the sustainable multi-actor solutions to migration-related child poverty in the greater Mekong sub-region project. This final evaluation builds on the outcomes of evaluations of previous phases of the project, an EU monitoring report of 2010 and a mid-term review of the project that were implemented in 2010. It will also build on a recent review (summer 2011) of the project’s effectiveness in building child protection systems.
2.1 Overall objectives
1. test the programme hypothesis
2. determine whether migrant and other vulnerable children in target communities experience less exploitation and discrimination, and the contribution of the project to this outcome
3. make an overall independent assessment about the past performance of the project, paying particularly attention to the impact of the project actions against its objectives
4. identify unintended consequences of the program, both positive and negative
5. document the enabling factors and challenges that influence programme implementation
6. provide evidence based recommendations on whether to continue with the current programme and whether an amended programme hypothesis and / or structure will enable better results to be achieved
7. determine whether lessons from past evaluations (from this and previous phases of the project) have been taken on board in the current project implementation
8. identify key lessons and to propose practical recommendations for follow-up actions
The core evaluation question is:
- Does a systems approach to child protection improve children’s protection and well-being outcomes?
This evaluation question will be empirically tested through the generation of evidence on the target group of children and target communities of this project.
2.2 Scope of evaluation
The evaluation will cover to some degree all 5 project countries and the regional-level work. However the evaluation will cover in-depth only 3 of the project countries (Thailand, Cambodia and Myanmar) in line with resources available for the evaluation.
The consultants shall verify, analyse and assess in detail the issues vis-Ã -vis the five evaluation criteria endorsed by the OECD-DAC (relevance, effectiveness, efficiency, sustainability and impact), and to the EC-specific evaluation criteria (EC added value and coherence).
3.0 Methodology
The following provides a guide on the methodological process expected for this evaluation. A final methodology will be proposed by the selected evaluators and presented to the steering committee overseeing the evaluation for their endorsement. The methodology will be based on the project indicators as specified in the project’s logical framework.
3.1 Steering committee
The steering committee will hold a virtual meeting with the evaluator(s) to review the Terms of Reference and finalise the evaluation process, including the data collection and analysis process to be put forward by the evaluator(s). The steering committee will also oversee the design and implementation of ethical safeguards for the protection of the confidentiality, dignity, safeguarding, rights and welfare of participants.
3.2 Desk review
The evaluator(s) will review the existing programme documents and relevant empirical literature, to be provided by Save the Children.
3.3 Inception Report
The evaluator(s) will develop an inception report that outlines the following:
• Outline the overall approach of the evaluation, highlighting the conceptual model(s) adopted
• Risks and limitations
• Detailed methodology, including data collection tools, key data sources, sampling and analysis methods
• Work plan, including time frames and travel schedules.
This will be presented to the Steering Committee for their review and endorsement, prior to commencing the field process.
3.4 Data collection in three countries and at regional level
The evaluation must include a mix of quantitative and qualitative data collection methodologies. The evaluators should consider the use of retrospective evaluation designs since baseline data is available for a limited number of metrics only. The evaluators should also consider the feasibility of include comparison communities in the evaluation design.
3.4.1 Quantitative data
Secondary data will be collected and analysed from Save the Children’s monitoring system, case management system, government and other service providers. Primary data will be collected through the use of survey tools at community level, if deemed feasible and appropriate by the evaluator and Steering Group. Such survey tools may aim to explore: the incidence of particular child protection issues; and knowledge, awareness and behaviour amongst children and adult community members in target sites regarding child protection, in particular migration and exploitation.
3.4.2 Qualitative data
Primary data will be collected through the use of carefully designed and documented key informant interviews, small group discussions, participatory methodologies (such as body mapping, risk analysis with children) and observation. The quality of the collection, documentation and analysis of qualitative data will be paramount and will be carefully monitored by the Steering Group.
3.4.3 Sampling
The evaluators must develop and present a sampling plan for all types of primary data collection – quantitative and qualitative - in the inception report, to be agreed with the Steering Group.
3.5 Feedback
Debrief sessions in each of the programme countries visited should be held to provide initial feedback on findings and recommendations. A final de-brief will also take place with the Programme team from 19-20 December in Bangkok at the Programme Management Team meeting at the end of the fieldwork. These de-briefs will be used to guide the development of the final report.
4.0 Recommendations
The evaluation recommendations should be action-oriented, describe clear follow-up and should be owned (i.e. should clearly identify who is responsible for each action). The recommendations should cover how Save the Children could:
o build on this project, develop a new phase for the regional programme
o take the programme to scale in each country, if the programme is deemed effective
o improve delivery, efficiency and cost-effectiveness of to achieve greater child protection results
o improve the monitoring and evaluation of the programme in future phases
5.0 Deliverables
The deliverables are:
1. Desk review of relevant programme documents and data
2. Inception Report of maximum 10 pages, detailing the methodology and work plan, for the review and approval by the Steering Committee (to be based on inception report format that will be provided by Save the Children)
3. Data collection tools for this evaluation
4. Debrief sessions in each country and with Steering Group on early analysis and findings
5. A comprehensive draft evaluation report of maximum 50 pages, based on the scope outlined above and the report format to be provided by Save the Children
6. Final evaluation report of maximum 50 pages, executive summary and action plan, incorporating the inputs from the feedback
6.0 Management of the evaluation
6.1 Steering committee
A small steering committee will be established to oversee the evaluation. The members are from Save the Children and the EU donor: Edel Silan, Brian Jungwiwattanaporn, Sarah Lilley and Jane Calder.The steering committee will be chaired by Sarah Lilley.
The key roles of the steering committee are to:
1. Provide technical input into the finalisation of the terms of reference, including the methodology and schedule of the evaluation.
2. Support the evaluation by sharing resources and networks with the evaluators to facilitate appropriate research directions, to enable the appropriate distribution of the draft report and ensure the protection and participation of children
3. Provide technical input and feedback to the draft report.
6.2 Evaluation team
The evaluation team will consist of up to two external evaluators. The evaluators will be expected to work closely together and with the Steering Group to deliver the outputs required. The expectation is that the evaluators will work together during the document review, planning and delivery of inception report and review of Thailand project as the first country. Each evaluator will then go to one country - either Myanmar or Cambodia - to conduct the evaluation in these countries simultaneously. The evaluators will jointly deliver the analysis and final report.
6.3 Selection of evaluators
The consultancy will be advertised nationally and internationally and evaluators will be required to have the following competencies:
6.3.1 Technical competencies and experience
The evaluator will have the following technical experience:
1. Sound understanding of the child-rights based approach to programming
2. Sound understanding of results-based management
3. Experience in the evaluation of child migration programmes
4. Sound understanding of child rights principles, particularly those relating to child protection and child participation
5. Sound analytical skills
6. Sound understanding of gender
7. Demonstrated experience in the application of qualitative and quantitative data collection and analysis tools, techniques and approaches
8. Fluency English, written and verbal
Additional desirable criteria:
• Experience in the Sub-Mekong region
6.3.2 Logistical capacity requirements
The evaluators will have the following logistical capacity
1. Capacity to maintain and submit accurate and timely financial acquittals
2. Access to computers and appropriate office software to keep data and generate the evaluation report.
6.3.3 Submission of tender
Tenders should be submitted electronically to Sarah Lilley, s.lilley@savethechildren.org.uk with copies to Edel Silan, e.silan@savethechildren.org.th no later than 15 November 2011. Tenders should include a cover letter, outlining how the evaluator meets the technical competencies and experience, and logistical capacity requirements outlined above, and include a daily fee in Euros. Tenders will be accepted from individual evaluators (to be paired with another selected by Save the Children) or from teams of two evaluators.
6.4 Timeframe for evaluation
It is estimated that the evaluation will take around 64 days but this will be finalised with the selected evaluators at the outset of the work:
Deliverable | Location | No. of work days
Planning call with Steering Group | Distance | 0.5
Literature review | Distance | 3
Inception report and development of draft tools, including fieldwork template | Distance | 4
Travel to Bangkok, Thailand | - | 1
Evaluation planning meeting and finalisation of workplan and tools | Bangkok | 2
Data collection in Thailand and de-brief | Thailand |10
Travel to country 2 and 3 | - | 1 per country
Data collection in country 2 and 3 and debrief | TBC | 10 days per country
Final de-brief with programme team at PMT meeting | Bangkok | 2
Data analysis | Distance | 5
Draft report | Distance |10
Feedback call with Steering Group | Distance | 0.5
Finalisation of report and other deliverables | Distance | 4
Total | 64 days
TERMS OF REFERENCE
October 2011
1.0 Background information
1.1 Introduction to the evaluation
Save the Children is commissioning an external, final evaluation of the 3-year programme: sustainable multi-actor solutions to migration-related child poverty in the greater Mekong sub-region. The programme, funded by the EU, started in January 2009 and will end in December 2011. The purpose of the evaluation is to evaluate the effectiveness and efficiency of the programme, in terms of improving the protection of children. This learning will be used to inform the future of the regional programme and the strategy for this work going forward. The evidence generated about what has worked and what has not worked will also inform global learning across Save the Children’s child protection portfolio.
The Greater Mekong Sub-region (GMS) is experiencing rapid economic growth, driven mainly by China, Vietnam and Thailand. However, local economic disparities and regional economic integration reinforce migration and challenge fragile Child Protection Systems within each country and across the region. Children are part of large and increasing movements of population in the GMS. While many children receive minimal protection services in their home or place of origin, their entitlements and the likelihood of accessing them are reduced as they move: adolescent girls, lone migrant and stateless children, children left behind and returning children face particular risks of violence, abuse, exploitation and poverty. Protection policies and legislation in the 5 countries have not been designed to take into account migrant children. Comprehensive protection policies, laws and implementing services are often absent. Where they exist, national governments, Local Authorities and Non-State Actors do not have the capacity to implement them effectively. Levels of community and civil society capacity also vary. Thailand has the most developed civil society. In China, Myanmar and Vietnam responsibility for social policy belongs to the government and state institutions. Recognising that migration cannot be prevented and may even be a positive choice for children and their families, Save the Children initiated the Cross-Border Project (XBP) in 1999 to promote safer migration and to combat trafficking, exploitation and worsening poverty. This 3-year EU-funded intervention is the 4th phase of the Cross-Border Project. It builds on previous work and relationships developed with local NSAs, civil society and LAs in Cambodia, China, Myanmar, Thailand and Vietnam.
1.2 The Programme
For over a decade, Save the Children has been striving to address abuse and exploitation of migrant children in the six countries in the Mekong Region. Save the Children’s response started through a Participatory Action Research Project in 1999 along the borders of China, Myanmar, and Thailand. The report “ Small Dreams Beyond Reach” outlined the struggles of migrant Burmese, Chinese, and ethnic Thai children, and resulted to country based action programmes in Thailand, Myanmar, and China and in-country action researches in Lao PDR and Vietnam, all addressing the issue of trafficking and emphasizing on prevention and protection of victims. These gave rise to the idea of establishing a coordinated regional initiative. In Myanmar, the project started to work in selected townships in three States in Myanmar – Mon, Kayin, and Northern Shan- that were known for high rates of risky migration. Participatory researches were also started in Lao PDR, Vietnam, and in Guangxi Province in China. Initial actions for victim protection were started in Thailand and Myanmar to support identification of trafficking victims and supporting their repatriation. This period was considered as the first phase of the Regional Cross-border Project and established the potentials of the project for grassroots work with communities and children.
The second phase of the Regional Cross-border Project (XBP2) was the “Cross-border Community-based Initiatives Against Trafficking in Children in the Mekong Sub-region” that was implemented from January 2002 to March 2006 (including a 3 month no-cost extension), funded by the DfID Asia Regional Poverty Fund, covering all the six countries of the GMS. The project was focused mainly on looking at communities of origin and aimed to help reduce the incidence and impact of trafficking and abuse of the most vulnerable migrant children, working on four major outputs: communities are able to mobilise and address the causes and effects of child trafficking, policies are able to protect rights of trafficked children and address safe return, awareness and knowledge raised among children and their families and communities, and children in communities of origin are able to enjoy greater education and livelihood opportunities before and after migration.
The third phase of the Regional Cross-border Project (XBP3) was the ‘Cross-border Project Against Trafficking and Exploitation of Migrant and Vulnerable Children’ that was implemented from April 2006 to March 2009, with funding from DfID Asia Regional Poverty Fund. This phase of the project was designed to align the anti-trafficking work in the Mekong with Save the Children’s broader programme of work on children on the move. The third phase of the project developed the work to focus more centrally on improving access to, and the quality of, local and national care and protection systems. The third phase of the Cross-border Project took on important lessons from phase 2, recognising that the migration of children cannot be stopped given the economic, social, and political trends in the region. Increasingly the focus was on how to promote safe migration and child protection systems that support and protect children as they move.
The fourth phase of the Project, which is the focus of this external evaluation, has been implemented from April 2009 and will finish in December 2011. This phase of the Project titled, sustainable multi-actor solutions to migration-related child poverty in the greater Mekong sub-region, and funded by the EU, is being implemented in China, Vietnam, Myanmar, Cambodia and Thailand, and at the regional level. The project targets 16,000 children across the 5 countries. The final beneficiaries also include parents, community members, LAs and NSAs engaged in work with migrants as well as regional and global agencies or networks with particular mandates on population flows.
The overall objective of the project is: National Child Protection systems serve as real mechanisms to reduce child poverty levels
The specific objective is: Child protection systems which work for the interests of children affected by migration become embedded into government structures and policies in the GMS to locally appropriate extents
The expected results are:
1. Target vulnerable children and young people are empowered as active members of the community, capable of engaging in the protection of themselves & other children
2. Communities affected by migration take concrete actions for the protection of children
3. LAs & NSAs better able to fulfil their responsibilities for children affected by migration
4. Local child protection models inform the development of more robust, higher level (provincial, national) child protection systems
5. Project experience influences regional and global fora on migration and child protection
Key activities include:
• Organising children’s groups, including migrant children’s groups
• Peer education, awareness raising activities and other child led and participatory community activities
• Regional Youth Empowerment and Leadership trainings and dialogues
• Organising and support for community-led child protection schemes
• Organising and training of adult community volunteers on child protection
• Capacity building for project partners on child rights, child protection, participation; safe migration; trafficking victim protection; and project management
• Organising and development of child protection mechanisms/systems at district/township and provincial levels
• Support for trafficking victims on identification, reporting rescue, recovery, return, and reintegration
• Support for safer migration services
• Supporting children's access to birth registration, citizenship, and legal status
• Supporting children's access to basic services including education, recreation, health and others
• Use of database system for children served by child protection system
• Policy Analysis on budget allocation to child protection system across Mekong Countries
• Support for the creation and development of higher level child protection system
• Policy research, monitoring and analysis
• Capacity building for NSAs/Las responsible for child protection
• Support to COMMIT and to its Sub-Regional Plan of Action
• Advocacy with Association of South East Asian Nations and with the Global Forum for Migration and Development
• Advocacy with the five project GMS governments on strengthening child protection systems
• Regional research on migration of children and on the statelessness of children
• The Mekong Youth Forum
The Project is implemented in 5 countries and at the regional level: Thailand, China, Vietnam, Myanmar and Cambodia. Across these countries the project is implemented in 45 Target sites across 22 Provinces/State/Division (5 provincial level, 28 township/district level and 12 ward/commune level).
Given the depth and history of the programme, and the considerable breadth and scope of its objectives and activities, this evaluation is an opportunity to independently analyse the relevance, efficiency, effectiveness, impact and sustainability of the programme against its stated objectives; and provide strategic direction to Save the Children and other partners to guide future implementation.
1.3 Programme structure and framework
The Project is managed overall by a regional team based in Bangkok. Within in each country, the project interventions are implemented by a variety of project partners including state actors (China and Vietnam), local NGOs (Thailand and Cambodia) and Save the Children staff itself (Myanmar).
2.0 Scope, objectives and outputs of the evaluation
This final evaluation will provide evidence and learning of achievements, outcomes, challenges and lessons learned in implementing the sustainable multi-actor solutions to migration-related child poverty in the greater Mekong sub-region project. This final evaluation builds on the outcomes of evaluations of previous phases of the project, an EU monitoring report of 2010 and a mid-term review of the project that were implemented in 2010. It will also build on a recent review (summer 2011) of the project’s effectiveness in building child protection systems.
2.1 Overall objectives
1. test the programme hypothesis
2. determine whether migrant and other vulnerable children in target communities experience less exploitation and discrimination, and the contribution of the project to this outcome
3. make an overall independent assessment about the past performance of the project, paying particularly attention to the impact of the project actions against its objectives
4. identify unintended consequences of the program, both positive and negative
5. document the enabling factors and challenges that influence programme implementation
6. provide evidence based recommendations on whether to continue with the current programme and whether an amended programme hypothesis and / or structure will enable better results to be achieved
7. determine whether lessons from past evaluations (from this and previous phases of the project) have been taken on board in the current project implementation
8. identify key lessons and to propose practical recommendations for follow-up actions
The core evaluation question is:
- Does a systems approach to child protection improve children’s protection and well-being outcomes?
This evaluation question will be empirically tested through the generation of evidence on the target group of children and target communities of this project.
2.2 Scope of evaluation
The evaluation will cover to some degree all 5 project countries and the regional-level work. However the evaluation will cover in-depth only 3 of the project countries (Thailand, Cambodia and Myanmar) in line with resources available for the evaluation.
The consultants shall verify, analyse and assess in detail the issues vis-Ã -vis the five evaluation criteria endorsed by the OECD-DAC (relevance, effectiveness, efficiency, sustainability and impact), and to the EC-specific evaluation criteria (EC added value and coherence).
3.0 Methodology
The following provides a guide on the methodological process expected for this evaluation. A final methodology will be proposed by the selected evaluators and presented to the steering committee overseeing the evaluation for their endorsement. The methodology will be based on the project indicators as specified in the project’s logical framework.
3.1 Steering committee
The steering committee will hold a virtual meeting with the evaluator(s) to review the Terms of Reference and finalise the evaluation process, including the data collection and analysis process to be put forward by the evaluator(s). The steering committee will also oversee the design and implementation of ethical safeguards for the protection of the confidentiality, dignity, safeguarding, rights and welfare of participants.
3.2 Desk review
The evaluator(s) will review the existing programme documents and relevant empirical literature, to be provided by Save the Children.
3.3 Inception Report
The evaluator(s) will develop an inception report that outlines the following:
• Outline the overall approach of the evaluation, highlighting the conceptual model(s) adopted
• Risks and limitations
• Detailed methodology, including data collection tools, key data sources, sampling and analysis methods
• Work plan, including time frames and travel schedules.
This will be presented to the Steering Committee for their review and endorsement, prior to commencing the field process.
3.4 Data collection in three countries and at regional level
The evaluation must include a mix of quantitative and qualitative data collection methodologies. The evaluators should consider the use of retrospective evaluation designs since baseline data is available for a limited number of metrics only. The evaluators should also consider the feasibility of include comparison communities in the evaluation design.
3.4.1 Quantitative data
Secondary data will be collected and analysed from Save the Children’s monitoring system, case management system, government and other service providers. Primary data will be collected through the use of survey tools at community level, if deemed feasible and appropriate by the evaluator and Steering Group. Such survey tools may aim to explore: the incidence of particular child protection issues; and knowledge, awareness and behaviour amongst children and adult community members in target sites regarding child protection, in particular migration and exploitation.
3.4.2 Qualitative data
Primary data will be collected through the use of carefully designed and documented key informant interviews, small group discussions, participatory methodologies (such as body mapping, risk analysis with children) and observation. The quality of the collection, documentation and analysis of qualitative data will be paramount and will be carefully monitored by the Steering Group.
3.4.3 Sampling
The evaluators must develop and present a sampling plan for all types of primary data collection – quantitative and qualitative - in the inception report, to be agreed with the Steering Group.
3.5 Feedback
Debrief sessions in each of the programme countries visited should be held to provide initial feedback on findings and recommendations. A final de-brief will also take place with the Programme team from 19-20 December in Bangkok at the Programme Management Team meeting at the end of the fieldwork. These de-briefs will be used to guide the development of the final report.
4.0 Recommendations
The evaluation recommendations should be action-oriented, describe clear follow-up and should be owned (i.e. should clearly identify who is responsible for each action). The recommendations should cover how Save the Children could:
o build on this project, develop a new phase for the regional programme
o take the programme to scale in each country, if the programme is deemed effective
o improve delivery, efficiency and cost-effectiveness of to achieve greater child protection results
o improve the monitoring and evaluation of the programme in future phases
5.0 Deliverables
The deliverables are:
1. Desk review of relevant programme documents and data
2. Inception Report of maximum 10 pages, detailing the methodology and work plan, for the review and approval by the Steering Committee (to be based on inception report format that will be provided by Save the Children)
3. Data collection tools for this evaluation
4. Debrief sessions in each country and with Steering Group on early analysis and findings
5. A comprehensive draft evaluation report of maximum 50 pages, based on the scope outlined above and the report format to be provided by Save the Children
6. Final evaluation report of maximum 50 pages, executive summary and action plan, incorporating the inputs from the feedback
6.0 Management of the evaluation
6.1 Steering committee
A small steering committee will be established to oversee the evaluation. The members are from Save the Children and the EU donor: Edel Silan, Brian Jungwiwattanaporn, Sarah Lilley and Jane Calder.The steering committee will be chaired by Sarah Lilley.
The key roles of the steering committee are to:
1. Provide technical input into the finalisation of the terms of reference, including the methodology and schedule of the evaluation.
2. Support the evaluation by sharing resources and networks with the evaluators to facilitate appropriate research directions, to enable the appropriate distribution of the draft report and ensure the protection and participation of children
3. Provide technical input and feedback to the draft report.
6.2 Evaluation team
The evaluation team will consist of up to two external evaluators. The evaluators will be expected to work closely together and with the Steering Group to deliver the outputs required. The expectation is that the evaluators will work together during the document review, planning and delivery of inception report and review of Thailand project as the first country. Each evaluator will then go to one country - either Myanmar or Cambodia - to conduct the evaluation in these countries simultaneously. The evaluators will jointly deliver the analysis and final report.
6.3 Selection of evaluators
The consultancy will be advertised nationally and internationally and evaluators will be required to have the following competencies:
6.3.1 Technical competencies and experience
The evaluator will have the following technical experience:
1. Sound understanding of the child-rights based approach to programming
2. Sound understanding of results-based management
3. Experience in the evaluation of child migration programmes
4. Sound understanding of child rights principles, particularly those relating to child protection and child participation
5. Sound analytical skills
6. Sound understanding of gender
7. Demonstrated experience in the application of qualitative and quantitative data collection and analysis tools, techniques and approaches
8. Fluency English, written and verbal
Additional desirable criteria:
• Experience in the Sub-Mekong region
6.3.2 Logistical capacity requirements
The evaluators will have the following logistical capacity
1. Capacity to maintain and submit accurate and timely financial acquittals
2. Access to computers and appropriate office software to keep data and generate the evaluation report.
6.3.3 Submission of tender
Tenders should be submitted electronically to Sarah Lilley, s.lilley@savethechildren.org.uk with copies to Edel Silan, e.silan@savethechildren.org.th no later than 15 November 2011. Tenders should include a cover letter, outlining how the evaluator meets the technical competencies and experience, and logistical capacity requirements outlined above, and include a daily fee in Euros. Tenders will be accepted from individual evaluators (to be paired with another selected by Save the Children) or from teams of two evaluators.
6.4 Timeframe for evaluation
It is estimated that the evaluation will take around 64 days but this will be finalised with the selected evaluators at the outset of the work:
Deliverable | Location | No. of work days
Planning call with Steering Group | Distance | 0.5
Literature review | Distance | 3
Inception report and development of draft tools, including fieldwork template | Distance | 4
Travel to Bangkok, Thailand | - | 1
Evaluation planning meeting and finalisation of workplan and tools | Bangkok | 2
Data collection in Thailand and de-brief | Thailand |10
Travel to country 2 and 3 | - | 1 per country
Data collection in country 2 and 3 and debrief | TBC | 10 days per country
Final de-brief with programme team at PMT meeting | Bangkok | 2
Data analysis | Distance | 5
Draft report | Distance |10
Feedback call with Steering Group | Distance | 0.5
Finalisation of report and other deliverables | Distance | 4
Total | 64 days
Post a Comment