Header

Research Proposals on the Impact of Transparency and Accountability Interventions

Request for Research Proposals on the Impact of Transparency and Accountability Interventions

Call Background

The Transparency and Accountability Initiative (T/AI) is pleased to announce a request for submissions by research or evaluation institutions or consortia to develop a rigorous multi-year research proposal on the impact of T/A interventions. The aim is to fill critical gaps in our knowledge about T/A interventions and further our common understanding of what works, why and in what circumstances? The research will be funded by the Bill & Melinda Gates Foundation, the United Kingdom Department for International Development, the William and Flora Hewlett Foundation and potentially others, under the umbrella of the T/AI.

The transparency and accountability field is one of the fastest growing cross-sectoral movements of recent years. It brings together a wide range of organizations and efforts aimed at promoting greater openness on the part of governments, companies and other institutions so that the public can hold them to account. Transparency International, Revenue Watch Institute, International Budget
Partnership, Global Witness, the nascent aid transparency movement and many other leading civil society organizations have laid the groundwork.

There is a wide array of initiatives and interventions in the field of transparency and accountability (T/A), including scorecards, social audits, public expenditure tracking surveys and more. To date, a mix of qualitative and quantitative evidence on these efforts suggests these interventions may positively affect outcomes, such as reducing stock outs in health centers, increasing budgetary allocations for agriculture, increasing effective disbursements of health and agriculture related funding, etc1. However, research that will yield generalizable and policy significant findings focused on T/A interventions that have a high likelihood of being replicable and scalable across a wide range of contexts is still needed.

The T/AI is a donor collaborative that aims to seize this momentum and expand the impact, scale and coordination of funding and activity in the T/A field, as well as explore applications of this work in new areas. The T/AI is led by a diverse group of eight leading funders in the field that includes private foundations, a leading bilateral donor and regranting international NGOs: Ford Foundation, Hivos,
International Budget Partnership, Omidyar Network, Open Society Foundations, Revenue Watch Institute, the United Kingdom Department for International Development (DFID) and the William and
Flora Hewlett Foundation (Hewlett Foundation).

In particular, the T/AI’s Impact and Learning working group enables funders, researchers and practitioners to learn if, where, when and how efforts succeed or fail, and use this knowledge to inform more effective T/A interventions that build on the field’s past two decades of experience. T/AI supports research across contexts, sectors, and interventions as well as quick-cycle evaluations of innovations.
T/AI wishes to commission research that is rigorous, and has legitimacy and buy-in within the T/A field2.
T/AI has a high priority for research that will yield generalizable and policy significant findings focused on T/A interventions that have a high likelihood of being replicable and scalable.


Call Details

The overarching questions for the research are:

• To what extent can health, agricultural, sanitation and/or education outcomes be attributed to transparency and accountability interventions?

• How and why do transparency interventions generate change in these outcomes (e.g. through which mechanisms, under what conditions, and in what contexts)?


Outcomes of Interest:

The key types of outcomes that should be the focus of the research are those far down the causal chain, as close to actual social and economic benefits for individuals and households as possible. For example, outcome categories could consist of nutrition, mortality or morbidity, incomes, literacy or numeracy, etc. Intermediary outcomes with clear links to these ultimate outcomes (e.g. actual resources spent on health, agricultural extension, primary education, etc.) could also be of interest.


Programmatic Focus – T/A Interventions:

The research should focus on T/A interventions which are well known and widely implemented (in terms of application and documentation), and for which there is reasonable prior evidence of causal linkage to outcomes.

Research should yield generalizable and policy significant findings focused on T/A interventions that have a high likelihood of being replicable and scalable.

Examples of T/A interventions that could be the focus of the research include, but are not limited to:

a. Complaint mechanisms and citizen charters intended to bring problems to the attention of governing institutions and to establish standards for addressing complaints;

b. Citizen report cards that follow the principle of consumer satisfaction surveys in the private sector and expose poor administrative performance;

c. Community score cards that assess the performance of local public services and that assess levels of service satisfaction by users and involve community meetings where service performance is discussed;

d. Civil society budget analysis and advocacy whereby civil society organizations (CSOs) engage with the budget process (e.g. analyzing budget information, working with the media, parliamentarians, audit institutions) to influence both budget allocations and budget effectiveness;

e. Public expenditure tracking surveys monitoring leakages and gaps in the delivery of funds to the local level;

f. Public hearings and social audits to hold public officials directly accountable for local level implementation of programs by verifying monitoring results in public gatherings where all relevant stakeholders are present;

g. Participatory budgeting whereby voluntary and regular contributions to decision-making over at least part of a public budget occur through a structured series of scheduled engagements with government authorities.


Sectoral Focus:

The research should focus on one or more of the following sectors: health, agriculture, sanitation and education, and/or sub-sectors therein. These are key areas of interest for the project funders, within which applicants are invited to suggest a suitable focus and scope to generate credible findings. There is no a priori preference for proposals that cover more than one sector.


Geographic Focus:

Generating findings that are generalizable and focused on T/A interventions that can be replicated and scaled across contexts is a key objective of the funders. The assumption is that the design will involve research across multiple country contexts.

The research should focus at least 75% on countries in Sub-Saharan Africa and South Asia. The research should also focus to the extent possible, and at least 50% on low-income countries, according to the World Bank definition.


Time Frame:

The research study is expected to take 3 to 4 years, with a maximum of five years. If there are justifiable reasons for longer commitments, these will be considered.


Research Design and Methods Selection:

The goals of the research are on both causal descriptions (generating valid inferences about the outcomes attributable or not to T/A interventions) and causal explanations (generating valid inferences about the mechanisms and conditions under which outcomes occur or not).

In line with norms generally agreed upon by leading international research and evaluation associations, it is assumed that the ultimate research proposal developed by the selected research institution or consortium will be based on carefully elaborated theories linking interventions to outcomes, combination of research methods (mixed-methods), and a range of quantitative and qualitative evidence (triangulation). The research will rigorously address the full range of potential threats to the validity (construct, internal, external and statistical conclusion).

The funders have a strong preference for the research design to include an experimental/quasiexperimental component given the objective of producing statistically significant findings that attribute outcomes to T/A interventions (causal descriptions). The funders also have a strong preference for studies that involve randomized experiments to the extent feasible and appropriate to increase confidence in the attribution findings (internal validity).

Funders have a strong preference for studies that replicate the experimental component across contexts to the extent appropriate and feasible to increase the generalizability of the findings (external validity).
Funders also have a preference for – and intend to fund - studies where the T/A interventions are themselves replicated across contexts. Funders anticipate that this may imply researchers having to focus on one or a limited set of T/A intervention as opposed to several.

The research should also be designed to elaborate valid causal explanations given the objective of increasing knowledge about the causal mechanisms through and contextual conditions under which outcomes from T/A interventions occur. Rigorous application of comparative and case study methods will likely be essential to the research design to address these objectives. Ethnographic and/or participatory research methods could also be integrated into the study to the extent they contribute to further deepening and broadening the results obtained.

The research should also examine the distribution of outcomes and potential unintended (positive and negative) outcomes to the extent feasible. Applicants should suggest a range of methods that fulfill this goal. Again, rigorous application of comparative and case study methods will likely be essential to the research design to address these objectives. Ethnographic and/or participatory research methods could also be integrated into the study to the extent they contribute to further deepening and broadening the results obtained.


Research Team Composition:

An “Applicant” may constitute EITHER a research institution with staff and other relevant capacity to undertake the research study OR a consortium of researchers (from across institutions) who are applying to undertake the research study together.

An institution or consortium able to design and execute a rigorous mixed-methods research design, with experience in the T/A field, should undertake the study. The institution or consortium should include researchers with extensive experience in:

a. Experimental and statistical methods to achieve the goal of generating valid inferences on causal descriptions.

b. Comparative, case study, ethnographic and/or participatory methods to achieve the goal of generating valid inferences on causal mechanisms, as well as the goal of understanding the distribution of outcomes and unintended outcomes.

Both types of researchers should be more or less equally weighted in terms of participation in the design and implementation of the research team and study

The funders strongly encourage developing country researchers and evaluators, as well as partnerships between developed and developing country researchers and evaluators to apply.

The institution or consortium should have the necessary level of administrative capacity to manage the logistics of a complex project with multiple interventions in different regions.


Implementing Partners:

The research institution or consortium will work with one or more reputable and experienced organizations with proven track records in implementing the T/A intervention(s) across the sectors and contexts that will be the focus of the research. The TAI will help facilitate the selected research team in the final determination of T/A implementation partners.


Policy Significance:

The funders prioritize research that will yield generalizable and policy significant findings focused on T/A interventions that have a high likelihood of being replicable and scalable.

Choices about each aspect of the research design including the selection of interventions, sectors, and countries on which to focus should bear this in mind.

The involvement of the implementing partners, policy-makers and other key international, national and local stakeholders throughout the research process is essential to ensure both the high quality of the research and its policy relevance. Early engagement of potential end users is critical, and research teams should engage deeply and consistently with practitioner and policymaker counterparts.


Budget:

The budget envelope for the multi-year research study is in the range of USD 5M, subject to discussions with funders and quality of proposals and designs developed.


Value for Money:

The funders will prioritize studies that are cost effective and promise the most value for money.


Submissions Process, Requirements and Review Criteria

There is a two-stage submission process. In Stage 1, applicants will submit their qualifications in support of their ability to design and execute the proposed research based on the questions, goals and criteria elaborated in this RFP. Applicants will also submit ideas on potential designs of the research, again in relation to the questions, goals and criteria elaborated in this RFP.


The deadline for submissions for Stage 1 is 11.59 GMT January 3rd, 2012. Submissions should be sent to smathes@scalingimpact.net. A panel of experts and the funding partners will review submissions. Three applicants will be short-listed to be interviewed by January 20, 2012. One of the three short-listed applicants will be selected by January 31, 2012 to move on to Stage 2, which entails developing a full and final proposal that will be developed with donors for funding in the second quarter of 2012.

In Stage 2, the selected research team will work with the T/AI and funders to systematically design and plan the research including selection of outcome categories, T/A intervention(s), sector(s), geographic context(s), implementing partner(s), methods and data. The planning phase will also allow for a fuller understanding of the challenges, constraints and risks posed by the research.

It is envisaged that the selected research team will require funding for their time, travel, and meetings with key stakeholders, including the implementing partner(s), in order to finalize the research proposal.
The budget envelope for this Stage 2 planning phase will be approximately USD 150,000. T/AI will manage these funds. The finalized proposal will be submitted to the donors (e.g. Gates, Hewlett, DFID) for consideration of full funding by April/May 2012.

The budget envelope for the multi-year research study (Stage 3) is in the range of USD 5M, subject to discussions with funders and quality of proposals and designs developed during Stage 2.


Stage 1 Submission Requirements:

The submission for Stage 1 involves two parts. An “Applicant” may constitute EITHER a research institution with staff and other relevant capacity to undertake the research study OR a consortium of researchers (from across institutions) who are applying to undertake the research study together. If the latter, each of the below responses must indicate which individual(s) within the consortium is/are being referenced.


Part A: Applicant’s Experience and Expertise

(Responses to Part A should be no more than 3 pages)

Please provide applicable project management skills, including experience in:

Organizing and working in interdisciplinary research teams,

Working in multiple contexts,

Working with implementing partners,

Managing (including financial management of) large scale, multi-year, multi-country research studies, etc.

Please provide specific experience in the following areas:

1. T/A Interventions:

Provide applicant’s experience conducting research or evaluations on transparency and accountability interventions, and relevant related areas.

2. Sectors:

Provide applicant’s experience and expertise conducting research or evaluations in the sectors of interest: Health, Agriculture, Sanitation, and Education.

3. Geographies/Countries:

Provide applicant’s experience conducting research and evaluations across countries in Africa and South Asia.

4. Research Design and Methodologies:

Provide applicant’s experience and expertise in designing and executing experimental and statistical research;

Provide applicant’s experience and expertise in designing and executing comparative, casestudy, ethnographic or participatory methods;

Provide 3 abstracts of applicant’s relevant previous research and/or evaluation experience (as appendices – abstracts will not count towards 5-page limit for this section).
Please provide 3 references (name, title, contact information) to support the above.


Part B: Discussion of Potential Research Study (Responses to Part B should be no more than 7 pages)

Applicants are requested to provide ideas on potential designs of the research, in relation to the questions, goals and criteria elaborated in this RFP. The purpose of the discussion of the potential research study in part B is to assess candidates. The selected research team will be funded (up to $150,000) in stage 2 to design a finalized research proposal to be submitted in April/May 2012. The successful applicant will not be required to submit the exact same proposal for full funding in 2012 as that described in this section.

In this section, applicants should provide an analytic and strategic discussion related to designing and implementing the proposed research study by considering the questions and goals identified in this RFP on the following study components: T/A interventions, sectors, geographic focus, implementing partners, methods and process.

Applicants should focus their discussion of the study components on how to most effectively balance the following priorities:

Research that will yield generalizable and policy significant findings focused on T/A interventions that have a high likelihood of being replicable and scalable. The scalability and replicability of the interventions will help T/AI and funders assess how relevant and significant the study findings may be.
Research that is both scalable and cost effective:

a. Applicants should explain how they would ensure value for money in this project and briefly outline their thoughts on what can be done given the overall budget envelope of approximately USD 5M over 3-4 years. Applicants should outline brief budget scenarios, in which low, middle, and high options are provided with respect to budget and what can be achieved accordingly. Applicants should consider that the budget envelope includes funds both for the study, as well as for the replication of the T/A intervention.

b. For example, applicants could include information on fee rates, detailed overhead rates and approximate unit cost for workshops, travel etc.

c. Applicants should include basic information on the estimated cost of replicating the T/A intervention.


Discussion of Study Components:

1. T/A Interventions

Applicants should suggest which intervention or interventions should be seriously considered for the research and explain why, given the above priorities.

Given the strong preference for replication, applicants may suggest a narrowed design replicating one type of accountability intervention and one sector (health, agriculture, sanitation).

2. Sectors

Applicants should suggest which sector(s)or sub-sectors within health, agriculture, sanitation and education should be seriously considered for the research and explain why, given the above priorities.

Given the strong preference for replication, applicants may suggest that only one intervention and one sector should be selected in order to achieve this aim.

3. Geographic Focus

Applicants should suggest the geographic contexts/countries that should be seriously considered for the research and explain why, given the above priorities.

4. Implementing Partners

Applicants should suggest implementing partners that should be seriously considered for the research and explain why, given the above priorities. Applicants should detail past or current relationships between the researchers and suggested implementing partners.

Researchers will select experienced and reputable implementing partners during the Stage 2 planning phase, with the support and input of the T/AI and respective funders.

5. Potential Research Design, Methods and Data:

Applicants should discuss and explain elements of potential mixed-method designs for the research. As stated above, it is possible for the study to examine one or more T/A interventions, in one or more sectors, across one or more country contexts, etc. Applicants should discuss the pros and cons of research designs involving different potential combinations of interventions, sectors, countries, etc. in relation to the overall research questions, goals, and priorities conveyed in this RFP. In particular, this discussion should address the preference for research that will yield generalizable and policy significant findings focused on T/A interventions that have a high likelihood of being replicable and scalable.

6. Process:

Applicants are requested to briefly describe key elements of the process and what steps they would take to plan, design and implement the research in Stage 2 and beyond the planning phase during execution of the actual research.


Assessment Criteria for Stage 1 Applications

In general, Stage 1 applications will be assessed based on the strength of the applicant’s relevant expertise and experience, and their ability to address the broader research questions, goals and issues in their discussion of the potential research study. More specifically, applications will be assessed according to the following criteria:

• Qualifications of proposed research team – relevant expertise and experience (50%)

• Quality of discussion about potential research study (20%)

• Potential policy significance (research that will yield generalizable findings focused on T/A interventions that have a high likelihood of being replicable and scalable. (20%)

• Evidence of existing relationships with potential implementing organizations (10%)