Title PGMF Impact Evaluation Consultancy
Location Washington DC
Department Name Program Advancement
Pact Overview
IMPACT ASSESSMENT
For
PACT GLOBAL MICROFINANCE FUND (PGMF)
TERMS OF REFERENCE
Consultant for Design, Sampling, Instrument Development, Data Collection, Data Entry,
Analysis and Report Writing
Introduction
Background
Since 1997, Pact, with the support of UNDP, has been making small loans to the country’s rural poor to help them build their incomes and improve their lives. UNDP turned over Pact’s longstanding microfinance operations in Myanmar in 2014 to the new, government-licensed entity Pact Global Microfinance Fund (PGMF). Today, PGMF is the country’s largest microfinance institution, currently serving more than 700,000 households in more than 10,000 villages. In 2015, PGMF disbursed more than 1 million loans worth $280 million USD, with an astounding repayment rate of 99.85 %. For most of those it serves, 99% of them women, PGMF is the only access to affordable financial services available. The majority of PGMF loans are for family farming, helping increase yields and get products to market, resulting in higher household incomes.
Institutional Goal, Vision, and Mission
PGMF shares the vision of Pact: “A world where those who are poor and marginalized exercise their voice, build their own solutions and take ownership over their future.” PGMF works to help Pact achieve one of its key organizational objectives—“people with limited livelihood choices gain the resources needed to be income secure.”
In addition, PGMF has the following strategic objectives:
Increase PGMF’s ability to impact its target area of livelihoods with a particular focus on traditional microfinance work.
Access a wider range of financing (donor, debt, equity, etc.) to expand PGMF’s capability to enable those with limited livelihood choices to gain the resources needed to be income secure.
Purpose of the Evaluation
The purpose of the evaluation is to measure the effects of PGMF on a variety of social and economic outcomes, including income, housing, education, food security, and women’s empowerment. As a secondary purpose, the study will assess client satisfaction on loan and savings products as well as consumer behaviors around utilization of these products.
Evaluation Questions
The following questions will be answered by the evaluation:
To what extent have the expected outcomes been achieved as a result of the project?
Property and household assets
New self-owned businesses and improvements to existing self-owned businesses
Income
Households spending on food, health, education and other expenditures
Household food security
Levels of education among clients and household members
Poverty, as measured by the Progress out of Poverty Index
Dependency on external money lenders
Women’s roles in decision-making and women’s empowerment.
What are lending patterns among clients, and how are clients consuming loans?
What opportunities and challenges do clients face in trying to establish new business or expand existing business?
What is overall client satisfaction with PGMF products and services?
How can PGMF products and services be altered to better serve clients?
Evaluation Background and Rationale
Below is a chart of previous evaluations of Pact’s microfinance projects.
Title Year Study Design and Methodology Sample
Vulnerable Assessment Studies in Microfinance Delta; by EDA 2004 Quasi-experimental cross sectional study Clients: n=231
Non-clients: n= 79
Impact Study of Microfinance Project in Three Zones 2007 Quasi-experimental cross sectional study Clients: n=464
Non-clients: n= 272
Evaluation of the UNDP Myanmar Microfinance Project in the Delta, Dry Zone, and Shan State 2011 Quasi-experimental cross sectional study Clients: n=2,700
Non-clients: n= 900
Baseline and End line assessment in USAID funded branches (Internal Assessment) 2011-2013 Quasi-experimental cross sectional study) Clients: n=1,650
Non-clients: n= 550
Evaluation Methodology
A retrospective, longitudinal cohort comprised of PGMF clients is proposed as the evaluation design for this impact assessment. The participants selected in the 2016 study will include individuals interviewed in 2011[1] in order to capture longitudinal changes from 2011 to 2016. The sample will include villages in geographic areas where PGMF has expanded since 2011 in order for the sample to be representative of where PGMF currently operates and set the baseline measurement for possible future measurements. For the retrospective cohort component, the estimated sample size is 1,500, comprised of clients who participated in the 2011 survey. It is estimated that another sample of 1,000 ‘new’ clients from areas where PGMF has expanded to will also be needed. The sample size and approach will be subject to further consultation with the evaluation consultant.
A quantitative survey will be administered to selected PGMF clients. The questionnaire will measure socio-demographics, household property and assets, household expenditures, detailed information on businesses, women’s household decision-making power and empowerment, loan consumption behaviors, and satisfaction with PGMF products and services. The household questionnaire will be translated into Burmese and will be piloted and refined prior to the commencement of data collection. The tool will be primarily based on the questionnaire administered in 2011 to ensure comparable measures from 2011 to 2016.
Focus group discussions (FGD) will also be conducted to triangulate findings from quantitative data using purposive sampling. FGDs will be conducted by the consultant who is expected to use experienced qualitative researchers.
Survey data collection and entry will be performed by the consultant with support from PGMF MIS staff.
Data Sources, Collection Methods, Management and Analysis
The quantitative survey will be administered face-to-face by data collectors. Repeat visits will be conducted to ensure high participation rates among the randomly selected clients. The quantitative survey will be administered on mobile phones and data will be stored on secure servers. Some of the data management features the mobile application will possess include:
The platform will ensure that each user has a name and password
Validation rules to minimize errors.
The application will ensure compulsory entries and skip patterns to ensure there are no omissions and unnecessary responses
The mobile application will ensure that forms encrypted, are synchronized to server, thus forms will be irretrievable once they have been submitted.
Data sets from quantitative survey will be downloadable through an administrator’s username and password from the platform, in excel and/or CSV formats ready for analysis.
Data collectors need to be trained on a variety of relevant topics including Human Subjects protection, interviewing techniques, and data entry on mobile phones. Appropriate supervision structures and data quality checks will be put in place by the consultant to ensure high quality data are generated.
Data cleaning, tabulation and analysis of both the quantitative and qualitative data will be performed by the consultant with the help of his/her team.
Human Subjects Consideration
Approval of this protocol will be obtained prior to data collection. This evaluation involves no more than minimal risks to participants (for the survey or FGDs). Informed consent will be obtained from all participants. During the informed consent procedures potential participants will be informed of survey procedures. No incentives for participation will provided. The benefit of participation is that information collected will contribute to the broader base of knowledge. Potential participants will be informed that their participation is voluntary and they are able to refuse to answer question or withdraw from the survey at any point in time. They will also be informed that their responses will remain confidential and that their choice to participate will have no effect on the services provided by PGMF.
Report and Dissemination
The consultant will be expected to develop the full evaluation report. PGMF will be responsible for the dissemination of the evaluation findings to appropriate audiences within Myanmar and at international forums.
The findings of the evaluation will be used to inform future PGMF programming.
Data ownership
Data collected through this contract will be solely owned by PGMF. Consultants may not use this data for any purpose outside of this TOR unless permission is given explicitly in writing.
Activities and Outputs of the Consultancy
The following table describes expected activities, time period, and estimated allocation of days needed to complete the activities:
Task Time Period
Sign with consultant First of June
Consultant develop protocol, tools and dummy tables First week of June to Second week of June
Review first draft of protocol Third week of June
Second draft of Protocol Fourth week of June
Develop Mobile Application for front end data collection and entries First week of June to Third week of June
Finalize Mobile Application Last week of June
Consultant work on training material Third and Last week of June
End of Part A
Data Collection Training by consultant First week of July
Data Collection Second to last week of July
Data Cleaning Second week of July to First week of August
Preliminary Analysis Third week of July to Second week of August
First draft report First week of August to fourth week of August
Final draft Second week of September
End of Part B
Within the first two weeks of the assignment, the consultant should present a complete protocol with the following components:
Refined evaluation objectives, questionnaires and indicators
Descriptions of the data collection methods
Detailed explanation of the sampling approach and calculations
Training plan for data collection teams and data entry/analysis staff
Estimate of translation requirements
Data analysis plan
Workplan
Data collection instruments (draft and final)
The consultant should hire and ensure highly qualified interviewers and moderators for the qualitative components of this assessment.
Deliverables
Protocol including descriptions of the data collection methods, sample size calculation, sampling strategy, ethical considerations, data management procedures, and analysis plan (including dummy tables)
Final survey and FGD tools, including informed consent in both English and Burmese
Training agenda, manual, and materials for field staff
Final impact assessment survey dataset (SPSS format preferred) and syntax files showing how the analysis was conducted
Final impact assessment report
Required Competencies and Qualifications
The consultant should have extensive, demonstrable experience in field research design, complex sampling strategy, project evaluation, and participatory qualitative and quantitative methods. In addition, strong interpersonal skills across different groups and excellent analytical writing skills are necessary. The consultant must provide three references, and three samples of evaluations in which the consultant was the lead evaluator. Qualifications must include:
Advanced university degree in Development Studies, Statistics, Business/Public Administration, Economics, or other relevant field. Ph.D. in demography, rural development, regional economics are preferable.
At least ten years of relevant work experience with substantive experience and expertise in evaluation design. Knowledge of microfinance and livelihoods development interventions will be a plus.
Excellent skills on advanced quantitative analysis skills with hands on experience in SPSS, STATA, R and relevant statistical packages.
Research companies are preferred
Proposal Submissions
Interested parties are encouraged to submit their proposals by May 30th, 2016 to:
Nanette Barkey
Director of Results and Measurement
Pact
nbarkey@pactworld.org
with cc: to jsmeikle@pactworld.org and mmcdermott@pactworld.org
Any questions may be addressed to Nanette Barkey
EmoticonEmoticon