Title PGMF Impact Evaluation ConsultancyLocationWashington, DCDepartment NameProgram Advancement Pact Overview
IMPACT ASSESSMENT
For
PACT GLOBAL MICROFINANCE FUND (PGMF)
TERMS OF REFERENCE
Consultant for Design, Sampling, Instrument Development, Data Collection, Data Entry,
Analysis and Report Writing
Introduction
BackgroundSince 1997, Pact, with the support of UNDP, has been making small loans to the country’s rural poor to help them build their incomes and improve their lives. UNDP turned over Pact’s longstanding microfinance operations in Myanmar in 2014 to the new, government-licensed entity Pact Global Microfinance Fund (PGMF). Today, PGMF is the country’s largest microfinance institution, currently serving more than 700,000 households in more than 10,000 villages. In 2015, PGMF disbursed more than 1 million loans worth $280 million USD, with an astounding repayment rate of 99.85 %. For most of those it serves, 99% of them women, PGMF is the only access to affordable financial services available. The majority of PGMF loans are for family farming, helping increase yields and get products to market, resulting in higher household incomes.
Institutional Goal, Vision, and MissionPGMF shares the vision of Pact: “A world where those who are poor and marginalized exercise their voice, build their own solutions and take ownership over their future.” PGMF works to help Pact achieve one of its key organizational objectives—“people with limited livelihood choices gain the resources needed to be income secure.”
In addition, PGMF has the following strategic objectives:Increase PGMF’s ability to impact its target area of livelihoods with a particular focus on traditional microfinance work.
Access a wider range of financing (donor, debt, equity, etc.) to expand PGMF’s capability to enable those with limited livelihood choices to gain the resources needed to be income secure.
Purpose of the EvaluationThe purpose of the evaluation is to measure the effects of PGMF on a variety of social and economic outcomes, including income, housing, education, food security, and women’s empowerment. As a secondary purpose, the study will assess client satisfaction on loan and savings products as well as consumer behaviors around utilization of these products.
Evaluation QuestionsThe following questions will be answered by the evaluation:
To what extent have the expected outcomes been achieved as a result of the project?
- Property and household assets
- New self-owned businesses and improvements to existing self-owned businesses
- Income
- Households spending on food, health, education and other expenditures
- Household food security
- Levels of education among clients and household members
- Poverty, as measured by the Progress out of Poverty Index
- Dependency on external money lenders
- Women’s roles in decision-making and women’s empowerment.
What are lending patterns among clients, and how are clients consuming loans?
What opportunities and challenges do clients face in trying to establish new business or expand existing business?
What is overall client satisfaction with PGMF products and services?
How can PGMF products and services be altered to better serve clients?
Evaluation Background and RationaleBelow is a chart of previous evaluations of Pact’s microfinance projects.
TitleYearStudy Design and MethodologySample**Vulnerable Assessment Studies in Microfinance Delta; by EDA2004Quasi-experimental cross sectional studyClients: n=231
Non-clients: n= 79Impact Study of Microfinance Project in Three Zones2007Quasi-experimental cross sectional studyClients: n=464
Non-clients: n= 272Evaluation of the UNDP Myanmar Microfinance Project in the Delta, Dry Zone, and Shan State2011Quasi-experimental cross sectional studyClients: n=2,700
Non-clients: n= 900
Baseline and End line assessment in USAID funded branches (Internal Assessment)2011-2013Quasi-experimental cross sectional study)Clients: n=1,650
Non-clients: n= 550
- Evaluation MethodologyA retrospective, longitudinal cohort comprised of PGMF clients is proposed as the evaluation design for this impact assessment. The participants selected in the 2016 study will include individuals interviewed in 2011[1] in order to capture longitudinal changes from 2011 to 2016. The sample will include villages in geographic areas where PGMF has expanded since 2011 in order for the sample to be representative of where PGMF currently operates and set the baseline measurement for possible future measurements. For the retrospective cohort component, the estimated sample size is 1,500, comprised of clients who participated in the 2011 survey. It is estimated that another sample of 1,000 ‘new’ clients from areas where PGMF has expanded to will also be needed. The sample size and approach will be subject to further consultation with the evaluation consultant.
A quantitative survey will be administered to selected PGMF clients. The questionnaire will measure socio-demographics, household property and assets, household expenditures, detailed information on businesses, women’s household decision-making power and empowerment, loan consumption behaviors, and satisfaction with PGMF products and services. The household questionnaire will be translated into Burmese and will be piloted and refined prior to the commencement of data collection. The tool will be primarily based on the questionnaire administered in 2011 to ensure comparable measures from 2011 to 2016.
Focus group discussions (FGD) will also be conducted to triangulate findings from quantitative data using purposive sampling. FGDs will be conducted by the consultant who is expected to use experienced qualitative researchers.
Survey data collection and entry will be performed by the consultant with support from PGMF MIS staff.
Data Sources, Collection Methods, Management and Analysis
The quantitative survey will be administered face-to-face by data collectors. Repeat visits will be conducted to ensure high participation rates among the randomly selected clients. The quantitative survey will be administered on mobile phones and data will be stored on secure servers. Some of the data management features the mobile application will possess include:
- The platform will ensure that each user has a name and password
- Validation rules to minimize errors.
- The application will ensure compulsory entries and skip patterns to ensure there are no omissions and unnecessary responses
- The mobile application will ensure that forms encrypted, are synchronized to server, thus forms will be irretrievable once they have been submitted.
Data sets from quantitative survey will be downloadable through an administrator’s username and password from the platform, in excel and/or CSV formats ready for analysis.
Data collectors need to be trained on a variety of relevant topics including Human Subjects protection, interviewing techniques, and data entry on mobile phones. Appropriate supervision structures and data quality checks will be put in place by the consultant to ensure high quality data are generated.
Data cleaning, tabulation and analysis of both the quantitative and qualitative data will be performed by the consultant with the help of his/her team.
Human Subjects Consideration
Approval of this protocol will be obtained prior to data collection. This evaluation involves no more than minimal risks to participants (for the survey or FGDs). Informed consent will be obtained from all participants. During the informed consent procedures potential participants will be informed of survey procedures. No incentives for participation will provided. The benefit of participation is that information collected will contribute to the broader base of knowledge. Potential participants will be informed that their participation is voluntary and they are able to refuse to answer question or withdraw from the survey at any point in time. They will also be informed that their responses will remain confidential and that their choice to participate will have no effect on the services provided by PGMF.
Report and Dissemination
The consultant will be expected to develop the full evaluation report. PGMF will be responsible for the dissemination of the evaluation findings to appropriate audiences within Myanmar and at international forums.
The findings of the evaluation will be used to inform future PGMF programming.
Data ownership
Data collected through this contract will be solely owned by PGMF. Consultants may not use this data for any purpose outside of this TOR unless permission is given explicitly in writing.
- Activities and Outputs of the ConsultancyThe following table describes expected activities, time period, and estimated allocation of days needed to complete the activities:
TaskTime Period**Sign with consultantFirst of JuneConsultant develop protocol, tools and dummy tablesFirst week of June to Second week of JuneReview first draft of protocolThird week of JuneSecond draft of ProtocolFourth week of JuneDevelop Mobile Application for front end data collection and entriesFirst week of June to Third week of JuneFinalize Mobile ApplicationLast week of JuneConsultant work on training materialThird and Last week of JuneEnd of Part A Data Collection Training by consultantFirst week of JulyData CollectionSecond to last week of JulyData CleaningSecond week of July to First week of AugustPreliminary AnalysisThird week of July to Second week of AugustFirst draft reportFirst week of August to fourth week of AugustFinal draftSecond week of SeptemberEnd of Part B
Within the first two weeks of the assignment, the consultant should present a complete protocol with the following components:
- Refined evaluation objectives, questionnaires and indicators
- Descriptions of the data collection methods
- Detailed explanation of the sampling approach and calculations
- Training plan for data collection teams and data entry/analysis staff
- Estimate of translation requirements
- Data analysis plan
- Workplan
- Data collection instruments (draft and final)
The consultant should hire and ensure highly qualified interviewers and moderators for the qualitative components of this assessment.
Deliverables
- Protocol including descriptions of the data collection methods, sample size calculation, sampling strategy, ethical considerations, data management procedures, and analysis plan (including dummy tables)
- Final survey and FGD tools, including informed consent in both English and Burmese
- Training agenda, manual, and materials for field staff
- Final impact assessment survey dataset (SPSS format preferred) and syntax files showing how the analysis was conducted
Final impact assessment reportRequired Competencies and Qualifications
The consultant should have extensive, demonstrable experience in field research design, complex sampling strategy, project evaluation, and participatory qualitative and quantitative methods. In addition, strong interpersonal skills across different groups and excellent analytical writing skills are necessary. The consultant must provide three references, and three samples of evaluations in which the consultant was the lead evaluator. Qualifications must include:Advanced university degree in Development Studies, Statistics, Business/Public Administration, Economics, or other relevant field. Ph.D. in demography, rural development, regional economics are preferable.
At least ten years of relevant work experience with substantive experience and expertise in evaluation design. Knowledge of microfinance and livelihoods development interventions will be a plus.
Excellent skills on advanced quantitative analysis skills with hands on experience in SPSS, STATA, R and relevant statistical packages.
Research companies are preferredProposal Submissions
**Interested parties are encouraged to submit their proposals by May 15th, 2016 to:
U Sai Kyaw Han
Monitoring, Evaluation, Research and Development Manager
Pact Global Microfinance Fund
skyawhan@pactworld.org
with cc: to jsmeikle@pactworld.org, nbarkey@pactworld.org, and mmcdermott@pactworld.org
Any questions may be addressed to U Sai Kyaw Han**
EmoticonEmoticon