Request for tenders: Mid-term review of ACP MEA project
SPREP Secretariat of the Pacific Regional Environment Programme
PROE
Programme regional océanien de l'environnementRequest for Tenders
File: AP1/4/3File number
Date: 15 March, 2011
To: ACP MEA mid-term review consultant
Contact: Jill Key jillk@sprep.org
Subject: Request for tenders: Mid-term review of ACP MEA project
1. Background
1.1. The Pacific Regional Environment Programme (SPREP) is an intergovernmental organisation charged with promoting cooperation among Pacific islands countries and territories to protect and improve their environment and ensure sustainable development. 1.2. For more information, see: www.sprep.org.
1.3. The goal of the EC funded ACP MEA Project is to build capacity in African, Caribbean and Pacific ACP countries to help them implement their Multilateral Environmental Agreement (MEA) commitments . The overall objective of the programme is to promote environmental sustainability in the ACP countries thereby contributing to sustainable development and poverty reduction strategies. UNEP is the overall Coordinator and facilitator of ACP MEAs. The Pacific Hub of the ACP MEA project is hosted by SPREP. The four year programme covers the 15 Pacific ACP countries.
The specific objective is to strengthen the implementation of MEAs at the national and regional levels in the Pacific ACP countries, through the enhancement of SPREP as an environmental hub for the Pacific island countries. The Pacific Hub will in turn deliver quality capacity-building services to the Pacific island countries, such as project writing skills, negotiations training, drafting and information management and exchange.
Additionally, support for the implementation of specific MEAs is provided through four sub-components, namely Clean Development Mechanism (CDM), Combating Desertification, Sound Management of Chemicals and Pesticides management.
The outcome of the needs assessment exercises indicated that capacity development in the areas of environmental concern (biodiversity, pollution, climate change, water management issues and land degradation) was needed in:
· negotiation skills
· awareness raising
· mainstreaming of MEAs into development policies and programmes
· use of information-exchange systems
· sustainability of programmes
2. Specifications: statement of requirement
2.1. SPREP is calling for tenders from qualified and experienced project evaluation experts who can offer their services to carry out a mid-term review of the EC funded project Capacity Building related to Multilateral Environmental Agreements in African, Caribbean and Pacific Countries.2.2. The successful applicant will need to provide:
1. A detailed work plan within 1 week of commencing the consultancy.
2. An interim report detailing the findings on review of relevant documents, on lessons learned from Years 1 – 2, recommendations for Years 3 and 4 of the ACP MEA project delivery, propose a resource mobilization strategy, and propose a sustainable and replicable plan on MEAS in the Pacific region.
3. Submit and make a presentation of the findings of the consultancy for consideration, review and validation at an ad hoc team at SPREP.
4. A final report taking into account comments made on interim report. 2.3. The terms of reference for the consultancy are as follows:
Review objective:
1. Determine if the Pacific hub is fulfilling its goal for: i. The ACP secretariat and EU
ii. The 15 ACP country beneficiaries
2. Determine if the project delivery will have long-term impact
3. Determine what elements of the project are replicable for other capacity building activities for MEA implementation.
Tasks:
1. Carry out a desk review of relevant documents such as the project documents, donor agreement, annual work plans, budget documents, inception workshops reports, technical and financial progress reports, Programme Steering Committee reports, etc.
2. Through consultations with representatives of countries who have participated in the project activities, assess the outcomes of the activities carried out to date, evaluating usefulness, satisfaction and anticipated impact:
a. What is SPREP doing well?
b. Where is improvement needed?
c. What is the likelihood of long-term impact from the activities, as they have been delivered to date?
d. Will the outcomes of the activities achieve the results? Will the outcomes of the results achieve the goal?
e. Is the project delivery giving value for money?
3. Identify challenges that the project is facing in implementation of activities, and make proposals and/or recommendations to overcome these constraints in order to design and implement a sustainable MEAs Program.
4. Assess SPREP capacity to implement, monitor and evaluate ACP MEAs activities
5. Undertake consultations on the development of the strategy for resource mobilization in conjunction with SPREP MEAs implementation, seek views from partner organizations and traditional as well as potential donors on a strategy for resource mobilization, and make an assessment of major potential elements for the resource mobilization strategy. 6. Explore ways and means of accommodating MEAs in the development of new and innovative sources of national, regional and international development finance.
7. Identify potential donor funding windows for implementation of MEAs.
8. Propose a sustainable and replicable plan on MEAS in the Pacific region as well as a resource mobilization strategy (using CROP agencies structures and systems as means of identifying sustainable financing mechanisms) that SPREP will use to identify ways and entry points for approaching donors.
All proposed activities should take into account gender considerations
These tasks will be carried out by:
1. Desk review
2. Telephone interviews of selected participants of activities to date
3. Face-to-face interviews with selected participants of activities to date, in at least 3 countries
4. Meetings with SPREP staff involved in the project
5. Telephone interviews and meetings (where appropriate) with donor representatives, and other CROP agencies
3. Conditions: information for applicants
3.1. To be considered for this tender, interested suppliers must meet the following conditions:
1. Proven track record in project evaluation and reviews2. Professional experience in the Pacific islands region, in the area of project planning, environmental management, institutional management change, or similar.
3. Experience in project and programme evaluation;
4. Experience in elaboration of project plans related to environment projects and particularly MEAs;
5. Good knowledge of the policy aspects of the environmental management framework in the Pacific;
6. Demonstrated ability to analyse information and write concise, meaningful reports7. Proven ability to work independently and collaboratively and in a multidisciplinary and multicultural environment
8. Excellent organizational skills
9. Demonstrated strategic ability
10. Consistent drive for tangible results11. Fluent English.
Timeline for deliverables:
1. Detailed work planearly May 2011
2. Interim reportmid-June 2011
3. Presentation at SPREPmid-June 2011
4. Final reportend of June 2011
The remunerations of the consultant will be negotiable, but will be based on SPREP rules and the duration of the consultancy. All costs to be submitted in US$.
The consultant will be paid 30% of the total amount after approval of inception report and working plan, 40% upon the delivery of the interim report and its approval, the remaining 30% upon submission and approval of the final report
4. Submission guidelines
4.1. Tender documentation should demonstrate that the interested supplier satisfies the conditions stated above and is capable of meeting the specifications and timeframes, and provide supporting examples to address the evaluation criteria. 4.2. Tender documentation should outline the interested supplier’s complete proposal: methods, personnel (and their skill sets / curricula vitae), timeframes, outline work plan and costs.
4.3. Tenders should conform to the Key Evaluation Principles, Annex 1.
5. Evaluation criteria
5.1. SPREP will select a preferred supplier on the basis of SPREP’s evaluation of the extent to which their tender documentation demonstrates that they offer the best value for money, and that they satisfy the following criteria:· Expertise in project and programme evaluation
· Demonstrated experience in capacity development
· Comprehensive understanding of multilateral environmental agreements in the Pacific island country context
6. Deadline
6.1. The due date for submission of the tender is: Friday 8 April 2011. 6.2. Late submissions will be returned unopened to the sender.
6.3 Please send all tenders clearly marked ‘TENDER: Mid-term review of ACP MEA project’ to:
Mail: SPREP
Attention: Jill Key
PO Box 240
Apia, SAMOA
Email: jillk@sprep.org
Fax: +685 20231
Person: Submit by hand in the tenders box at SPREP reception, Vailima, Samoa.
ANNEX I
Key Evaluation principles
In attempting to evaluate any outcomes and impacts that the project may have achieved, evaluators should remember that the project’s performance should be assessed by considering the difference between the answers to two simple questions “what happened?” and “what would have happened anyway?” These questions imply that there should be consideration of the baseline conditions and trends in relation to the intended project outcomes and impacts. In addition it implies that there should be plausible evidence to attribute such outcomes and impacts to the actions of the project.
Sometimes, adequate information on baseline conditions and trends is lacking. In such cases this should be clearly highlighted by the evaluator, along with any simplifying assumptions that were taken to enable the evaluator to make informed judgements about project performance.
1. Project Evaluation Parameters
A. Attainment of objectives and planned results (progress to date):
The assessment of project results seeks to determine the extent to which the project objectives were achieved, or are expected to be achieved, and assess if the project has led to any other positive or negative consequences. While assessing a project’s outcomes the evaluation will seek to determine the extent of achievement and shortcomings in reaching the project’s objectives as stated in the project document and also indicate if there were any changes and whether those changes were approved. If the project did not establish a baseline (initial conditions), the evaluator should seek to estimate the baseline condition so that achievements and results can be properly established (or simplifying assumptions used). Since most GEF projects can be expected to achieve the anticipated outcomes by project closing, assessment of project outcomes should be a priority. Outcomes are the likely or achieved short-term and medium-term effects of an intervention’s outputs. Examples of outcomes could include but are not restricted to stronger institutional capacities, higher public awareness (when leading to changes of behaviour), and transformed policy frameworks or markets. The evaluation should assess the extent to which the project's major relevant objectives were effectively and efficiently achieved or are expected to be achieved and their relevance.
· Effectiveness: Evaluate how, and to what extent, the stated project objectives have been met, taking into account the “achievement indicators” specified in the project document and logical framework[1].· Relevance: In retrospect, were the project’s outcomes consistent with the focal areas/operational program strategies and country priorities? The evaluation should also assess the whether outcomes specified in the project document and or logical framework are actually outcomes and not outputs or inputs. Ascertain the nature and significance of the contribution of the project outcomes to the wider portfolio under GEF's Strategic Priority 3.
· Efficiency: Cost-effectiveness assesses the achievement of the environmental and developmental objectives as well as the project’s outputs in relation to the inputs, costs, and implementing time. Include an assessment of outcomes in relation to inputs, costs, and implementation times based on the following questions: Was the project cost-effective? Was the project the least cost option? Was the project implementation delayed and if it was then did that affect cost-effectiveness? The evaluation should assess the contribution of cash and in-kind co-financing to project implementation and to what extent the project leveraged additional resources.
Specifically the evaluation shall:
· Evaluate the outcomes of the project with regard to assisting governments to negotiate and implement MEAs.
B. Assessment of Sustainability of project outcomes:
Sustainability is understood as the probability of continued long-term project-derived outcomes and impacts after the project funding ends. The evaluation will identify and assess the key conditions or factors that are likely to contribute or undermine the persistence of benefits after the project ends. At mid-term, identification of any likely barriers to sustaining the intended outcomes of the project is especially important. Some of these factors might be outcomes of the project, e.g. stronger institutional capacities or better informed decision-making. Other factors will include contextual circumstances or developments that are not outcomes of the project but that are relevant to the sustainability of outcomes. The evaluation should ascertain to what extent follow-up work has been initiated and how project outcomes will be sustained and enhanced over time. In this case, sustainability will be linked to the long-term capacity of countries targeted by the project to negotiate and implement MEAs.
Four aspects of sustainability should be addressed: financial, socio-political, institutional frameworks and governance, and ecological (if applicable). The following questions provide guidance on the assessment of these aspects:
· Financial resources. To what extent are the outcomes of the project dependent on continued financial support? What is the likelihood that any required financial resources will be available to sustain the project outcomes/benefits once the current assistance ends (resources can be from multiple sources, such as the public and private sectors, income generating activities, and market trends that support the project’s objectives)? Was the project was successful in identifying and leveraging co-financing?
· Socio-political: To what extent are the outcomes of the project dependent on socio-political factors? What is the likelihood that the level of stakeholder ownership will allow for the project outcomes/benefits to be sustained? Is there sufficient public / stakeholder awareness in support of the long term objectives of the project?
· Institutional framework and governance. To what extent are the outcomes of the project dependent on issues relating to institutional frameworks and governance? What is the likelihood that institutional and technical achievements, legal frameworks, policies and governance structures and processes will allow for, the project outcomes/benefits to be sustained? While responding to these questions consider if the required systems for accountability and transparency and the required technical know-how are in place.
· Ecological. Are there any environmental risks that can undermine the future flow of project-derived environmental benefits?
As far as possible, also identify the potential longer-term impacts considering that the evaluation is taking place at mid-term and that longer term impact is expected to be seen in a few years time. Frame any recommendations to enhance future project impact in this context. Which will be the major ‘channels’ for longer term impact from the project at the national and international scales? The evaluation should formulate recommendations that outline possible approaches and necessary actions to facilitate an impact assessment study in a few years time.
C. Catalytic role
The mid-term evaluation will also describe any catalytic or replication effect of the project. What examples are there of replication and catalytic outcomes that suggest increased likelihood of sustainability? Replication approach can be defined as lessons and experiences coming out of the project that are replicated or scaled up in the design and implementation of other projects. Replication can have two aspects, replication proper (lessons and experiences are replicated in different geographic area) or scaling up (lessons and experiences are replicated within the same geographic area but funded by other sources). If no effects are identified, the evaluation will describe the catalytic or replication actions that the project carried out.
D. Achievement of outputs and activities:
· Delivered outputs: Assessment of the project’s success in producing each of the programmed outputs to date, both in quantity and quality as well as usefulness and timeliness.
· Assess to what extent the project outputs produced so far have the weight of authority / credibility, necessary to influence policy and decision-makers, particularly at the national or regional levels.
E. Assessment of processes that affected attainment of project results.
The evaluation will consider, but need not be limited to, consideration of the following issues that may have affected project implementation and attainment of project results: i. Preparation and readiness. Were the project’s objectives and components clear, practicable and feasible within its timeframe? Were capacities of the executing institutions and counterparts properly considered when the project was designed? Were lessons from other relevant projects properly incorporated in design? Were the partnership arrangements properly identified and the roles and responsibilities negotiated prior to implementation? Was availability of counterpart resources (funding, staff, and facilities), passage of enabling legislation, and adequate project management arrangements in place at project entry?
· Ascertain to what extent the project implementation mechanisms outlined in the project document have been closely followed. In particular, assess the role of the various committees established and whether the project document was clear and realistic to enable effective and efficient implementation, whether the project was executed according to the plan and how well the management was able to adapt to changes during the life of the project to enable the implementation of the project.
· Evaluate the effectiveness and efficiency and adaptability of project management and the supervision of project activities / project execution arrangements at all levels (1) policy decisions: Steering Committee; (2) day to day project management; (3) Overall coordination and facilitation by UNEP.
ii. Country ownership/Drivenness. This is the relevance of the project to national development and environmental agendas, recipient country commitment, and regional and international agreements. Examples of possible evaluative questions include: Was the project design in-line with the national sectoral and development priorities and plans? Are project outcomes contributing to national development priorities and plans? Were the relevant country representatives, from government and civil society, involved in the project? Did the recipient government maintain its financial commitment to the project? iii. Stakeholder involvement. Did the project involve the relevant stakeholders through information sharing, consultation and by seeking their participation in project’s design, implementation, and monitoring and evaluation? For example, did the project implement appropriate outreach and public awareness campaigns? Did the project consult and make use of the skills, experience and knowledge of the appropriate government entities, NGOs, community groups, private sector, local governments and academic institutions in the design, implementation and evaluation of project activities? Were perspectives of those that would be affected by decisions, those that could affect the outcomes and those that could contribute information or other resources to the process taken into account while taking decisions? Were the relevant vulnerable groups and the powerful, the supporters and the opponents, of the processes properly involved? Specifically the evaluation will:
· Assess the mechanisms put in place by the project for identification and engagement of stakeholders in each participating country and establish, in consultation with the stakeholders, whether this mechanism was successful, and identify its strengths and weaknesses.
· Assess the degree and effectiveness of collaboration/interactions between the various project partners and institutions during the course of implementation of the project.
· Assess the degree and effectiveness of any various public awareness activities that were undertaken during the course of implementation of the project.
iv. Financial planning. Did the project have the appropriate financial controls, including reporting and planning, that allowed management to make informed decisions regarding the budget and allowed for timely flow of funds. Specifically, the evaluation should:· Assess the strength and utility of financial controls, including reporting, and planning to allow the project management to make informed decisions regarding the budget and allow for a proper and timely flow of funds for the payment of satisfactory project deliverables throughout the project’s lifetime.
· Present the major findings from the financial audit if one has been conducted.
· Did promised co-financing materialize? Identify and verify the sources of co- financing as well as leveraged and associated financing.
· Assess whether the project has applied appropriate standards of due diligence in the management of funds and financial audits.
· The evaluation should also include a breakdown of final actual project costs by activities compared to budget (variances), financial management (including disbursement issues), and co- financing. v. UNEP Supervision and backstopping. Did UNEP Agency staff identify problems in a timely fashion and accurately estimate its seriousness? Did UNEP staff provide quality support and advice to the project, approved modifications in time and restructure the project when needed? Did UNEP Agencies provide the right staffing levels, continuity, skill mix, frequency of field visits? vi. Co-financing and Project Outcomes & Sustainability. If there was a difference in the level of expected co-financing and actual co-financing, then what were the reasons for this? Did the extent of materialization of co-financing affect the project’s outcomes and/or sustainability, and if it did affect outcomes and sustainability then in what ways and through what causal linkages?
vii. Delays and Project Outcomes & Sustainability. If there were delays in project implementation and completion, the evaluation will summarise the reasons for them. Did delays affect the project’s outcomes and/or sustainability, and if so in what ways and through what causal linkages?
The ratings will be presented in the form of a table with each of the categories rated separately and with brief justifications for the rating based on the findings of the main analysis. An overall rating for the project should also be given.
2. Evaluation report format and review proceduresThe report should be brief, to the point and easy to understand. It must explain; the purpose of the evaluation, exactly what was evaluated and the methods used. The report must highlight any methodological limitations, identify key concerns and present evidence-based findings, consequent conclusions, recommendations and lessons. The report should provide information on when the evaluation took place, the places visited, who was involved and be presented in a way that makes the information accessible and comprehensible. The report should include an executive summary that encapsulates the essence of the information contained in the report to facilitate dissemination and distillation of lessons.
Evidence, findings, conclusions and recommendations should be presented in a complete and balanced manner. The evaluation report shall be written in English, be of no more than 50 pages (excluding annexes), use numbered paragraphs and include:
i) An executive summary (no more than 3 pages) providing a brief overview of the main conclusions and recommendations of the evaluation;
ii) Introduction and background giving a brief overview of the evaluated project, for example, the objective and status of activities;
iii) Scope, objective and methods presenting the evaluation’s purpose, the evaluation criteria used and questions to be addressed;
iv) Project Performance and Impact providing factual evidence relevant to the questions asked by the evaluator and interpretations of such evidence. This is the main substantive section of the report and should provide a commentary on all evaluation aspects (A − F above).
v) Conclusions and rating of project implementation success giving the evaluator’s concluding assessments and ratings of the project against given evaluation criteria and standards of performance. The conclusions should provide answers to questions about whether the project is considered good or bad, and whether the results are considered positive or negative;
vi) Lessons learned presenting general conclusions from the standpoint of the design and implementation of the project, based on good practices and successes or problems and mistakes. Lessons should have the potential for wider application and use. All lessons should ‘stand alone and should:
§ Specify the context from which they are derived
§ State or imply some prescriptive action;
§ Specify the contexts in which they may be applied (if possible who when and where)
vii) Recommendations. High quality recommendations should be actionable proposals that are:
1. Implementable within the timeframe and resources available2. Commensurate with the available capacities of project team and partners
3. Specific in terms of who would do what and when
4. Contain results-based language (i.e. a measurable performance target)
5. Include a trade off analysis, when its implementation may require utilizing significant resources that would have otherwise been used for other project purposes.
viii) Annexes include Terms of Reference, list of interviewees, documents reviewed, brief summary of the expertise of the evaluator / evaluation team, etc. Dissident views or management responses to the evaluation findings may later be appended in an annex.
PROE
Programme regional océanien de l'environnementRequest for Tenders
File: AP1/4/3File number
Date: 15 March, 2011
To: ACP MEA mid-term review consultant
Contact: Jill Key jillk@sprep.org
Subject: Request for tenders: Mid-term review of ACP MEA project
1. Background
1.1. The Pacific Regional Environment Programme (SPREP) is an intergovernmental organisation charged with promoting cooperation among Pacific islands countries and territories to protect and improve their environment and ensure sustainable development. 1.2. For more information, see: www.sprep.org.
1.3. The goal of the EC funded ACP MEA Project is to build capacity in African, Caribbean and Pacific ACP countries to help them implement their Multilateral Environmental Agreement (MEA) commitments . The overall objective of the programme is to promote environmental sustainability in the ACP countries thereby contributing to sustainable development and poverty reduction strategies. UNEP is the overall Coordinator and facilitator of ACP MEAs. The Pacific Hub of the ACP MEA project is hosted by SPREP. The four year programme covers the 15 Pacific ACP countries.
The specific objective is to strengthen the implementation of MEAs at the national and regional levels in the Pacific ACP countries, through the enhancement of SPREP as an environmental hub for the Pacific island countries. The Pacific Hub will in turn deliver quality capacity-building services to the Pacific island countries, such as project writing skills, negotiations training, drafting and information management and exchange.
Additionally, support for the implementation of specific MEAs is provided through four sub-components, namely Clean Development Mechanism (CDM), Combating Desertification, Sound Management of Chemicals and Pesticides management.
The outcome of the needs assessment exercises indicated that capacity development in the areas of environmental concern (biodiversity, pollution, climate change, water management issues and land degradation) was needed in:
· negotiation skills
· awareness raising
· mainstreaming of MEAs into development policies and programmes
· use of information-exchange systems
· sustainability of programmes
2. Specifications: statement of requirement
2.1. SPREP is calling for tenders from qualified and experienced project evaluation experts who can offer their services to carry out a mid-term review of the EC funded project Capacity Building related to Multilateral Environmental Agreements in African, Caribbean and Pacific Countries.2.2. The successful applicant will need to provide:
1. A detailed work plan within 1 week of commencing the consultancy.
2. An interim report detailing the findings on review of relevant documents, on lessons learned from Years 1 – 2, recommendations for Years 3 and 4 of the ACP MEA project delivery, propose a resource mobilization strategy, and propose a sustainable and replicable plan on MEAS in the Pacific region.
3. Submit and make a presentation of the findings of the consultancy for consideration, review and validation at an ad hoc team at SPREP.
4. A final report taking into account comments made on interim report. 2.3. The terms of reference for the consultancy are as follows:
Review objective:
1. Determine if the Pacific hub is fulfilling its goal for: i. The ACP secretariat and EU
ii. The 15 ACP country beneficiaries
2. Determine if the project delivery will have long-term impact
3. Determine what elements of the project are replicable for other capacity building activities for MEA implementation.
Tasks:
1. Carry out a desk review of relevant documents such as the project documents, donor agreement, annual work plans, budget documents, inception workshops reports, technical and financial progress reports, Programme Steering Committee reports, etc.
2. Through consultations with representatives of countries who have participated in the project activities, assess the outcomes of the activities carried out to date, evaluating usefulness, satisfaction and anticipated impact:
a. What is SPREP doing well?
b. Where is improvement needed?
c. What is the likelihood of long-term impact from the activities, as they have been delivered to date?
d. Will the outcomes of the activities achieve the results? Will the outcomes of the results achieve the goal?
e. Is the project delivery giving value for money?
3. Identify challenges that the project is facing in implementation of activities, and make proposals and/or recommendations to overcome these constraints in order to design and implement a sustainable MEAs Program.
4. Assess SPREP capacity to implement, monitor and evaluate ACP MEAs activities
5. Undertake consultations on the development of the strategy for resource mobilization in conjunction with SPREP MEAs implementation, seek views from partner organizations and traditional as well as potential donors on a strategy for resource mobilization, and make an assessment of major potential elements for the resource mobilization strategy. 6. Explore ways and means of accommodating MEAs in the development of new and innovative sources of national, regional and international development finance.
7. Identify potential donor funding windows for implementation of MEAs.
8. Propose a sustainable and replicable plan on MEAS in the Pacific region as well as a resource mobilization strategy (using CROP agencies structures and systems as means of identifying sustainable financing mechanisms) that SPREP will use to identify ways and entry points for approaching donors.
All proposed activities should take into account gender considerations
These tasks will be carried out by:
1. Desk review
2. Telephone interviews of selected participants of activities to date
3. Face-to-face interviews with selected participants of activities to date, in at least 3 countries
4. Meetings with SPREP staff involved in the project
5. Telephone interviews and meetings (where appropriate) with donor representatives, and other CROP agencies
3. Conditions: information for applicants
3.1. To be considered for this tender, interested suppliers must meet the following conditions:
1. Proven track record in project evaluation and reviews2. Professional experience in the Pacific islands region, in the area of project planning, environmental management, institutional management change, or similar.
3. Experience in project and programme evaluation;
4. Experience in elaboration of project plans related to environment projects and particularly MEAs;
5. Good knowledge of the policy aspects of the environmental management framework in the Pacific;
6. Demonstrated ability to analyse information and write concise, meaningful reports7. Proven ability to work independently and collaboratively and in a multidisciplinary and multicultural environment
8. Excellent organizational skills
9. Demonstrated strategic ability
10. Consistent drive for tangible results11. Fluent English.
Timeline for deliverables:
1. Detailed work planearly May 2011
2. Interim reportmid-June 2011
3. Presentation at SPREPmid-June 2011
4. Final reportend of June 2011
The remunerations of the consultant will be negotiable, but will be based on SPREP rules and the duration of the consultancy. All costs to be submitted in US$.
The consultant will be paid 30% of the total amount after approval of inception report and working plan, 40% upon the delivery of the interim report and its approval, the remaining 30% upon submission and approval of the final report
4. Submission guidelines
4.1. Tender documentation should demonstrate that the interested supplier satisfies the conditions stated above and is capable of meeting the specifications and timeframes, and provide supporting examples to address the evaluation criteria. 4.2. Tender documentation should outline the interested supplier’s complete proposal: methods, personnel (and their skill sets / curricula vitae), timeframes, outline work plan and costs.
4.3. Tenders should conform to the Key Evaluation Principles, Annex 1.
5. Evaluation criteria
5.1. SPREP will select a preferred supplier on the basis of SPREP’s evaluation of the extent to which their tender documentation demonstrates that they offer the best value for money, and that they satisfy the following criteria:· Expertise in project and programme evaluation
· Demonstrated experience in capacity development
· Comprehensive understanding of multilateral environmental agreements in the Pacific island country context
6. Deadline
6.1. The due date for submission of the tender is: Friday 8 April 2011. 6.2. Late submissions will be returned unopened to the sender.
6.3 Please send all tenders clearly marked ‘TENDER: Mid-term review of ACP MEA project’ to:
Mail: SPREP
Attention: Jill Key
PO Box 240
Apia, SAMOA
Email: jillk@sprep.org
Fax: +685 20231
Person: Submit by hand in the tenders box at SPREP reception, Vailima, Samoa.
ANNEX I
Key Evaluation principles
In attempting to evaluate any outcomes and impacts that the project may have achieved, evaluators should remember that the project’s performance should be assessed by considering the difference between the answers to two simple questions “what happened?” and “what would have happened anyway?” These questions imply that there should be consideration of the baseline conditions and trends in relation to the intended project outcomes and impacts. In addition it implies that there should be plausible evidence to attribute such outcomes and impacts to the actions of the project.
Sometimes, adequate information on baseline conditions and trends is lacking. In such cases this should be clearly highlighted by the evaluator, along with any simplifying assumptions that were taken to enable the evaluator to make informed judgements about project performance.
1. Project Evaluation Parameters
A. Attainment of objectives and planned results (progress to date):
The assessment of project results seeks to determine the extent to which the project objectives were achieved, or are expected to be achieved, and assess if the project has led to any other positive or negative consequences. While assessing a project’s outcomes the evaluation will seek to determine the extent of achievement and shortcomings in reaching the project’s objectives as stated in the project document and also indicate if there were any changes and whether those changes were approved. If the project did not establish a baseline (initial conditions), the evaluator should seek to estimate the baseline condition so that achievements and results can be properly established (or simplifying assumptions used). Since most GEF projects can be expected to achieve the anticipated outcomes by project closing, assessment of project outcomes should be a priority. Outcomes are the likely or achieved short-term and medium-term effects of an intervention’s outputs. Examples of outcomes could include but are not restricted to stronger institutional capacities, higher public awareness (when leading to changes of behaviour), and transformed policy frameworks or markets. The evaluation should assess the extent to which the project's major relevant objectives were effectively and efficiently achieved or are expected to be achieved and their relevance.
· Effectiveness: Evaluate how, and to what extent, the stated project objectives have been met, taking into account the “achievement indicators” specified in the project document and logical framework[1].· Relevance: In retrospect, were the project’s outcomes consistent with the focal areas/operational program strategies and country priorities? The evaluation should also assess the whether outcomes specified in the project document and or logical framework are actually outcomes and not outputs or inputs. Ascertain the nature and significance of the contribution of the project outcomes to the wider portfolio under GEF's Strategic Priority 3.
· Efficiency: Cost-effectiveness assesses the achievement of the environmental and developmental objectives as well as the project’s outputs in relation to the inputs, costs, and implementing time. Include an assessment of outcomes in relation to inputs, costs, and implementation times based on the following questions: Was the project cost-effective? Was the project the least cost option? Was the project implementation delayed and if it was then did that affect cost-effectiveness? The evaluation should assess the contribution of cash and in-kind co-financing to project implementation and to what extent the project leveraged additional resources.
Specifically the evaluation shall:
· Evaluate the outcomes of the project with regard to assisting governments to negotiate and implement MEAs.
B. Assessment of Sustainability of project outcomes:
Sustainability is understood as the probability of continued long-term project-derived outcomes and impacts after the project funding ends. The evaluation will identify and assess the key conditions or factors that are likely to contribute or undermine the persistence of benefits after the project ends. At mid-term, identification of any likely barriers to sustaining the intended outcomes of the project is especially important. Some of these factors might be outcomes of the project, e.g. stronger institutional capacities or better informed decision-making. Other factors will include contextual circumstances or developments that are not outcomes of the project but that are relevant to the sustainability of outcomes. The evaluation should ascertain to what extent follow-up work has been initiated and how project outcomes will be sustained and enhanced over time. In this case, sustainability will be linked to the long-term capacity of countries targeted by the project to negotiate and implement MEAs.
Four aspects of sustainability should be addressed: financial, socio-political, institutional frameworks and governance, and ecological (if applicable). The following questions provide guidance on the assessment of these aspects:
· Financial resources. To what extent are the outcomes of the project dependent on continued financial support? What is the likelihood that any required financial resources will be available to sustain the project outcomes/benefits once the current assistance ends (resources can be from multiple sources, such as the public and private sectors, income generating activities, and market trends that support the project’s objectives)? Was the project was successful in identifying and leveraging co-financing?
· Socio-political: To what extent are the outcomes of the project dependent on socio-political factors? What is the likelihood that the level of stakeholder ownership will allow for the project outcomes/benefits to be sustained? Is there sufficient public / stakeholder awareness in support of the long term objectives of the project?
· Institutional framework and governance. To what extent are the outcomes of the project dependent on issues relating to institutional frameworks and governance? What is the likelihood that institutional and technical achievements, legal frameworks, policies and governance structures and processes will allow for, the project outcomes/benefits to be sustained? While responding to these questions consider if the required systems for accountability and transparency and the required technical know-how are in place.
· Ecological. Are there any environmental risks that can undermine the future flow of project-derived environmental benefits?
As far as possible, also identify the potential longer-term impacts considering that the evaluation is taking place at mid-term and that longer term impact is expected to be seen in a few years time. Frame any recommendations to enhance future project impact in this context. Which will be the major ‘channels’ for longer term impact from the project at the national and international scales? The evaluation should formulate recommendations that outline possible approaches and necessary actions to facilitate an impact assessment study in a few years time.
C. Catalytic role
The mid-term evaluation will also describe any catalytic or replication effect of the project. What examples are there of replication and catalytic outcomes that suggest increased likelihood of sustainability? Replication approach can be defined as lessons and experiences coming out of the project that are replicated or scaled up in the design and implementation of other projects. Replication can have two aspects, replication proper (lessons and experiences are replicated in different geographic area) or scaling up (lessons and experiences are replicated within the same geographic area but funded by other sources). If no effects are identified, the evaluation will describe the catalytic or replication actions that the project carried out.
D. Achievement of outputs and activities:
· Delivered outputs: Assessment of the project’s success in producing each of the programmed outputs to date, both in quantity and quality as well as usefulness and timeliness.
· Assess to what extent the project outputs produced so far have the weight of authority / credibility, necessary to influence policy and decision-makers, particularly at the national or regional levels.
E. Assessment of processes that affected attainment of project results.
The evaluation will consider, but need not be limited to, consideration of the following issues that may have affected project implementation and attainment of project results: i. Preparation and readiness. Were the project’s objectives and components clear, practicable and feasible within its timeframe? Were capacities of the executing institutions and counterparts properly considered when the project was designed? Were lessons from other relevant projects properly incorporated in design? Were the partnership arrangements properly identified and the roles and responsibilities negotiated prior to implementation? Was availability of counterpart resources (funding, staff, and facilities), passage of enabling legislation, and adequate project management arrangements in place at project entry?
· Ascertain to what extent the project implementation mechanisms outlined in the project document have been closely followed. In particular, assess the role of the various committees established and whether the project document was clear and realistic to enable effective and efficient implementation, whether the project was executed according to the plan and how well the management was able to adapt to changes during the life of the project to enable the implementation of the project.
· Evaluate the effectiveness and efficiency and adaptability of project management and the supervision of project activities / project execution arrangements at all levels (1) policy decisions: Steering Committee; (2) day to day project management; (3) Overall coordination and facilitation by UNEP.
ii. Country ownership/Drivenness. This is the relevance of the project to national development and environmental agendas, recipient country commitment, and regional and international agreements. Examples of possible evaluative questions include: Was the project design in-line with the national sectoral and development priorities and plans? Are project outcomes contributing to national development priorities and plans? Were the relevant country representatives, from government and civil society, involved in the project? Did the recipient government maintain its financial commitment to the project? iii. Stakeholder involvement. Did the project involve the relevant stakeholders through information sharing, consultation and by seeking their participation in project’s design, implementation, and monitoring and evaluation? For example, did the project implement appropriate outreach and public awareness campaigns? Did the project consult and make use of the skills, experience and knowledge of the appropriate government entities, NGOs, community groups, private sector, local governments and academic institutions in the design, implementation and evaluation of project activities? Were perspectives of those that would be affected by decisions, those that could affect the outcomes and those that could contribute information or other resources to the process taken into account while taking decisions? Were the relevant vulnerable groups and the powerful, the supporters and the opponents, of the processes properly involved? Specifically the evaluation will:
· Assess the mechanisms put in place by the project for identification and engagement of stakeholders in each participating country and establish, in consultation with the stakeholders, whether this mechanism was successful, and identify its strengths and weaknesses.
· Assess the degree and effectiveness of collaboration/interactions between the various project partners and institutions during the course of implementation of the project.
· Assess the degree and effectiveness of any various public awareness activities that were undertaken during the course of implementation of the project.
iv. Financial planning. Did the project have the appropriate financial controls, including reporting and planning, that allowed management to make informed decisions regarding the budget and allowed for timely flow of funds. Specifically, the evaluation should:· Assess the strength and utility of financial controls, including reporting, and planning to allow the project management to make informed decisions regarding the budget and allow for a proper and timely flow of funds for the payment of satisfactory project deliverables throughout the project’s lifetime.
· Present the major findings from the financial audit if one has been conducted.
· Did promised co-financing materialize? Identify and verify the sources of co- financing as well as leveraged and associated financing.
· Assess whether the project has applied appropriate standards of due diligence in the management of funds and financial audits.
· The evaluation should also include a breakdown of final actual project costs by activities compared to budget (variances), financial management (including disbursement issues), and co- financing. v. UNEP Supervision and backstopping. Did UNEP Agency staff identify problems in a timely fashion and accurately estimate its seriousness? Did UNEP staff provide quality support and advice to the project, approved modifications in time and restructure the project when needed? Did UNEP Agencies provide the right staffing levels, continuity, skill mix, frequency of field visits? vi. Co-financing and Project Outcomes & Sustainability. If there was a difference in the level of expected co-financing and actual co-financing, then what were the reasons for this? Did the extent of materialization of co-financing affect the project’s outcomes and/or sustainability, and if it did affect outcomes and sustainability then in what ways and through what causal linkages?
vii. Delays and Project Outcomes & Sustainability. If there were delays in project implementation and completion, the evaluation will summarise the reasons for them. Did delays affect the project’s outcomes and/or sustainability, and if so in what ways and through what causal linkages?
The ratings will be presented in the form of a table with each of the categories rated separately and with brief justifications for the rating based on the findings of the main analysis. An overall rating for the project should also be given.
2. Evaluation report format and review proceduresThe report should be brief, to the point and easy to understand. It must explain; the purpose of the evaluation, exactly what was evaluated and the methods used. The report must highlight any methodological limitations, identify key concerns and present evidence-based findings, consequent conclusions, recommendations and lessons. The report should provide information on when the evaluation took place, the places visited, who was involved and be presented in a way that makes the information accessible and comprehensible. The report should include an executive summary that encapsulates the essence of the information contained in the report to facilitate dissemination and distillation of lessons.
Evidence, findings, conclusions and recommendations should be presented in a complete and balanced manner. The evaluation report shall be written in English, be of no more than 50 pages (excluding annexes), use numbered paragraphs and include:
i) An executive summary (no more than 3 pages) providing a brief overview of the main conclusions and recommendations of the evaluation;
ii) Introduction and background giving a brief overview of the evaluated project, for example, the objective and status of activities;
iii) Scope, objective and methods presenting the evaluation’s purpose, the evaluation criteria used and questions to be addressed;
iv) Project Performance and Impact providing factual evidence relevant to the questions asked by the evaluator and interpretations of such evidence. This is the main substantive section of the report and should provide a commentary on all evaluation aspects (A − F above).
v) Conclusions and rating of project implementation success giving the evaluator’s concluding assessments and ratings of the project against given evaluation criteria and standards of performance. The conclusions should provide answers to questions about whether the project is considered good or bad, and whether the results are considered positive or negative;
vi) Lessons learned presenting general conclusions from the standpoint of the design and implementation of the project, based on good practices and successes or problems and mistakes. Lessons should have the potential for wider application and use. All lessons should ‘stand alone and should:
§ Specify the context from which they are derived
§ State or imply some prescriptive action;
§ Specify the contexts in which they may be applied (if possible who when and where)
vii) Recommendations. High quality recommendations should be actionable proposals that are:
1. Implementable within the timeframe and resources available2. Commensurate with the available capacities of project team and partners
3. Specific in terms of who would do what and when
4. Contain results-based language (i.e. a measurable performance target)
5. Include a trade off analysis, when its implementation may require utilizing significant resources that would have otherwise been used for other project purposes.
viii) Annexes include Terms of Reference, list of interviewees, documents reviewed, brief summary of the expertise of the evaluator / evaluation team, etc. Dissident views or management responses to the evaluation findings may later be appended in an annex.
Post a Comment