Mastering your Information System in a
Conditional Cash Transfer Program
July 2013
Authors: Farkhod Saidulloev – Director of Monitoring & Evaluation, Save the Children-US and seconded to the BOTA Foundation in Almaty, Kazakhstan.
[email protected] Larry Dershem – Design, Monitoring & Evaluation Advisor, Save the Children-US, and technical advisor to the BOTA Foundation’s Department of Monitoring & Evaluation.
[email protected] Editor: Katarzyna Jarosiewicz-Wargan – independent consultant and technical advisor for the BOTA Foundation.
[email protected]
BOTA Foundation, IREX and Save the Children The BOTA Foundation is a Kazakhstani non-governmental organization (NGO) that implements social programs and is managed by two international organizations, Save the Children and IREX. Registered in May 2008, the foundation started its activities in January 2009. BOTA’s programs focus on improvement of lives of children and youth from poor households, through three programs: Conditional Cash Transfers (CCT), Social Service Support Program and Tuition Assistance Program. More about the BOTA Foundation can be viewed at: www.bota.kz Although the management information system (MIS) was developed and adapted for all three programs undertaken by BOTA, only the CCT component is discussed in detail in this handbook. Compared to similar CCT programs around the world, BOTA is unique because it is fully implemented by an NGO, independent from, but working in close cooperation with, the government. The Kazakhstani government runs Targeted Social Assistance for households whose overall income is less than the qualifying threshold (40 % of the subsistence minimum), determined through a verified means test. In contrast, the BOTA CCT program uses a Proxy Means Testing, which is integrated into the MIS, to determine eligible beneficiaries.
2
Contents Introduction 2 1. Purpose and Elements of a Useful MIS 3 2. Design and Development of an MIS 8 3. MIS Reports 22 4. Maintenance of a MIS 33 5. Cost of a MIS 35 6. Supplementary Databases in a MIS 38 References 40 BOTA Resources 41
3
Introduction In 2009, the World Bank selected IREX and its partner Save the Children (SC) to establish the BOTA Foundation. One of BOTA’s three programs was a conditional cash transfer (CCT) program. The CCT program was designed to offer financial incentives to encourage and enable households suffering from poverty to access services that would improve their children’s development and welfare. In addition to meeting eligibility requirements under the Proxy Means Test (PMT), beneficiaries must meet specific characteristics defined by the scope of BOTA work. These include being: 1) a mother who is pregnant and/or lactating, 2) a child age 4-6 years who needs early childhood development (ECD) services, or 3) a child age 4-16 years who is disabled. At a later state, a fourth eligibility category was added: youth, aged 16 – 19, to participate in livelihood related (LLH) training programs. At the beginning IREX and SC conducted an extensive search to find literature on practical, real-world examples, such as a step-by-step process, of how to: a) setup a CCT program, b) determine a beneficiary targeting method, and c) develop a comprehensive management information system (MIS). Although ample literature on CCT programs, targeting and MIS was found, this literature tended to be directed at broader social welfare systems, focused toward government audiences, and quite technical in nature. Over the past four years of implementing the CCT program in Kazakhstan, IREX and SC have been contacted by various country offices, small programs and local civil society organizations wanting to learn more about CCT programs implemented by a NGO. The concept for a series of practical handbooks came about from these requests for an easy to understand guide on “how to do a CCT program.” Three handbooks are envisioned: 1) How To Implement a Conditional Cash Transfer Program, 2) Using Proxy Means Test (PMT) for Targeting in a CCT Program, and 3) Mastering Your Information System (MIS) in a CCT Program. The three handbooks are designed to be practical, real-world guides, written in a style that is easy to understand and to be complementary to one another. As CCT programs are increasingly implemented on smaller scales and by NGOs - Devex (www.devex.com) has observed that in 2011, there were four times as many CCT-related projects and tenders for NGOs than major aid donors as in 2007 - these handbooks will provide important information to practitioners. This handbook, Mastering Your Information System (MIS) in a CCT Program discusses the issues and processes encountered by IREX and SC in the development of its management information system within BOTA’s CCT program. The issues include determining the purpose, design, operations, reporting, maintenance, and costs. 4
1. Purpose and Elements of a Useful MIS A MIS is a vital tool to manage data and information for all CCT program aspects, from application, eligibility testing, enrollment and meeting conditionalities (also referred to as co-responsibilities), to making cash transfer payments and reporting. A good MIS also provides timely information for accurate, evidence-based decision-making, ensures accountability and transparency to donors and beneficiaries, and can report progress toward achieving program outcomes and results. The most important goal a MIS must achieve for a CCT program is to be useful! BOTA’s CCT program in Kazakhstan began small-scale, processing hundreds of applications and enrollments in the first few months. Within two years (2010-2011) it increased exponentially, eventually reaching 30,000 households. Although requirements for each CCT program will be different based on a context and need, all MIS in a CCT program share the same practical elements. Below is a check list of elements that will ensure that the MIS is useful and a corresponding list of how each element was addressed within the BOTA CCT program. Checklist Elements Needed in a Useful MIS Issue
P
Accuracy
Explanation
How BOTA’s MIS Addressed Issue
An essential element of a useful MIS is valid and reliable data. Ensuring accuracy entails having quality control procedures in place for all steps in data collection, management, storage, and reporting processes.
BOTA has integrated a number of quality controls (from simple to more complex) into its MIS to minimize errors that may occur for different reasons. Examples of simple controls include automated date of beneficiary’s application and other mandatory data fields. Examples of complex controls include verifying composition of the beneficiary’s unique identification numbers (UINs) such as taxpayer’s number and individual number, and detection of duplications of the UINs at the enrollment and later stages. Detailed guidebooks and trainings are provider to all MIS users.
5
6
P Timeliness
Outdated data are rarely useful, especially when prompt decisions must be made. A useful MIS should be capable of providing and distributing up-to-date data and information to appropriate users.
The online MIS report module is accessible to authorized users to retrieve data via Internet 24 hours a day. Relevant BOTA staff members and other key stakeholders receive data about cumulative trends and progress (application, enrollment, compliance and payment process) of the program on a systematic basis through a monthly reporting cycle. In addition, the M&E department runs special reports on a regular basis and upon request.
P Consistency
In order for data to be reliable, it must be processed and compiled consistently and uniformly. Variations in how data are collected and reported can distort information and trend analysis. Additionally, because data collection and reporting processes may change over time, the management must establish sound procedures to allow for system changes.
Regular reports generated on program progress are based on standard reporting templates. Besides reports, the MIS has integrated data screens. Both have been developed and harmonized to meet the needs and expectations of MIS users.
P Completeness
As with a puzzle, all MIS pieces must be put together to see the whole picture. Accurate and timely decisionmaking requires comprehensive data and information in a summarized form. Reports should be designed to eliminate clutter and voluminous detail, thereby avoiding “information overload.”
All data captured, entered, and maintained by BOTA feed into program indicators at different levels, from activities (outputs) to outcomes (results), to ensure evidence-based management of the program and ability to asses results achieved to date.
P Relevance
Few things hinder a program more than inappropriate and unnecessary data and information. The MIS must provide what is needed by donors, project managers and field staff to perform their jobs.
BOTA’s M&E plan is a key tool in determining information and reporting needs by different stakeholders. Data collected and means to convert the data into information are mainly guided by the program’s established M&E plan.
11 Key Facts about BOTA CCT Program MIS 1. The MIS (central database) operates online and is supported with an offline e-application on small netbook computers. These netbook computers allow for gathering data in the field without access to the Internet. They have a minimal database required to avoid duplication at enrollment. All data collected through the offline e-application are then uploaded to the central database via an Internet connection. 2. Both the MIS central database and offline application incorporate a beneficiary targeting instrument (Proxy-Means Test) administered to determine eligibility of applicants during enrollment. 3. In-house capacity was built to ensure sound management and administration of the MIS system developed by external consultants. Specialists within the M&E department possess complete information about programming codes used to develop the MIS and about its structure. Having in-house capacity allows the program to integrate any changes into the MIS without involvement of a third party. Trial versions of the MIS updates are developed and pilot tested before transferring to the actual MIS. 4. A duplicate, back-up copy of the MIS, updated daily, assures recovery if needed.
5. Two MIS specialists with interchangeable relevant skills are engaged in administration and management of the system at the central level. 6. The MIS enables disaggregation of data at sub-central levels, such as oblast (province). 7. The MIS tracks the entire CCT process, from application to exit from the program, for all applicants and beneficiaries. 8. A test version of the MIS integrates geographic information system (GIS) data that allows mapping out program activities using the free application, Google Maps. 9. The geo-data incorporated into the MIS is based on the National Classifier of the Administrative and Territorial Units - OKATO. 10. The MIS is accessible through a secure web link. 11. The MIS generates reports in Excel format for ease of data analysis and/or importing data into specialized software (e.g., SPSS) for statistical analysis. The most common reports are integrated into the system. This enables users to run reports without an involvement of the central administrators.
7
2. Design and Development of an MIS Implementing a CCT program, even on a small scale, will demand collecting and processing a large amount of data and reporting it in a consistent and reliable manner, all within a relatively short time. There are three cautions that should be heeded in the design and development of an MIS for a CCT program. Caution 1: Managing the MIS technology is not an end in itself, and should not take more time than what is required by information needs of the CCT program. BOTA’s Approach: Determine the information needs first. Once the MIS is up and running, it can process an enormous volume of data that otherwise could not be processed so quickly and efficiently without automation. BOTA’s guiding principle was “the minimum data needed for a comprehensive understanding of the program beneficiaries, activities and results”. Caution 2: The MIS should not be limited to only certain CCT activities. Instead, the MIS should integrate all CCT processes as well as higher-level program results and objectives. BOTA’s Approach: Integrate all CCT processes from the application to exit. The BOTA MIS is the primary mechanism to capture and store program data. Paperbased forms are used in very few cases, such as in the field when monitoring reports by case managers are completed and when a beneficiary must sign an agreement. To minimize further cost and logistics related to data collection, questionnaires - designed to capture knowledge and behavior change data from program beneficiaries before (at enrollment) and after (upon exit) participation in the CCT program - were integrated into the MIS. Caution 3: Designing and developing an MIS is not only a technology process, it is also a social and political process. Without buy-in and cooperation of various program staff and end-users, any technology and design can be rendered ineffective. BOTA’s Approach: Working with the director and staff of the CCT program. The M&E team assured ownership by involving actively the CCT director, staff and endusers in the development and upgrading of the MIS, as their information needs evolved over time. Once the system became operational, a MIS working group was formed to formally respond to needs and make adjustments to the existing MIS in order to accommodate increasing demands and expectations. The graph below presents seven steps in the design and development of the MIS in BOTA. The design phase had four steps: Step 1 - determining what information is needed, Step 2 - identifying database platform, Step 3 - determining organizational capacity, 8
and Step 4 - drafting a diagram of the MIS structure and flow of information. The development phase involved: Step 5 – initial development of the MIS database, Step 6 - testing the MIS database, and Step 7 - implementing the MIS database. Data and information needs revealed during implementation informed further updates and upgrades in the MIS. Although the graph presents a seemingly chronological procedure, the development of an MIS is a much more interactive and iterative process. Figure: Steps in Designing and Developing an MIS
Design Phase: Step 1 – Determine information needs.
As previously stated, the MIS is as much a social and political process as it is technological. It is critical that during the design phase all users of the MIS data and information help determine what data and information are needed. This process requires differentiating between data and information.
Information is data that has been processed in such a way that is meaningful to the person who receives it, and can be communicated clearly to others.
Data is raw material such as facts, events and transactions. Data Data is raw material refers to unprocessed information. Information is data that has been processed in such a way that is meaningful to the person such as facts, events and transactions. Data refers to who receives it, and can be communicated clearly to others. unprocessed information. The most important goal of Step 1 is for all users to identify information they want to receive from the MIS, which is different than identifying the data. Data are merely the raw “material” entered into the MIS, such as facts, events, and 9
transactions. Information is the result of processing and transforming the raw data in a way that is meaningful to the user. Two important types of information were needed in the BOTA CCT program: 1) information to determine progress toward CCT results and 2) information to allow for tracking key CCT processes for on-going program improvement. 1) Information to determine progress toward results and outcomes – the key document to identify information needs related to CCT results was the program’s Logical Framework (see a complete CCT Logical Framework in the How To Implement a Conditional Cash Transfer Program Handbook in this series). For example, one of many results in the CCT program was: “Percentage of children 4-6 years of age living in households receiving CCT who meet satisfactory attendance at early childhood development centers.” The CCT program wanted this result, and others, reported on a monthly basis. 2) Information to track key processes – the CCT program determined key processes that needed to be tracked to ensure program quality. One of many key processes was “Number and percentage of bank cards delivered to CCT beneficiaries.” The CCT program wanted this key process tracked on a monthly basis. 3) Data needed for results and key processes – in order to report on “Number and percentage of bank cards delivered to CCT beneficiaries”, the following data are needed in a timely manner: a) number of beneficiaries who were enrolled each month, b) eligible households with at least one member having a bank card, c) number of households without a bank card who have a taxpayer number needed to apply for the bank card, and d) recipients with bank cards at the end of the month. 4) Additional data – this included such data as a) unique identification numbers, b) proxy means test scores, c) tracking program volunteers who help beneficiaries, d) list of all household members, e) various characteristics of the beneficiary. 5) Sources of data – this will vary depending on the stage of the CCT program. During the recruitment and enrollment process, all data are collected from the applicant/beneficiary and stored in the MIS. If needed, the field staff can verify the data. Examples of Information, Data and Source of Data Used in BOTA’s MIS
10
Information Needed
Data Needed
Source of Data
Result – Percentage of children 4-6 years of age living in households receiving CCT who meet satisfactory attendance at early childhood development centers.
• Number of children 4-6 years of age living in household receiving CCT that attended, at least 80% of the days, the early childhood development center in the last quarter. • Total number of children 4-6 years of age living in households receiving CCT in the last quarter.
• ECD records collected and provided by CCT volunteers
Process – Number and percentage of cash transfers received on time by beneficiaries.
Process – Number and percentage of bank cards delivered to CCT beneficiaries.
• Number of households receiving CCT that had their cash transfer deposited into their account on the specified date. • Total number of households receiving CCT. • Number of qualified households/recipients with bank cards. • Total number of qualified households/recipients.
• Transfers and beneficiary followup reports • Bank records • Satisfaction survey reports
• Bank cards’ records in the MIS.
Once the program stakeholders know what data they need for successful program management, this data can be defined as the main product expected from the MIS.
Step 2 – Identifying a database platform
Choosing a MIS platform depends upon many factors, such as a) local capacity, b) costs related to software and hardware, and c) time to develop and maintain. In Kazakhstan, the MIS are usually built on dynamic relational database platforms, with the two most widely used being ORACLE and Microsoft SQL (MySQL) Server. The BOTA program held a tender for an external developer of the MIS. The selected firm had considerable experience in both ORACLE and MySQL database development. For example, it had developed, using ORACLE, the state-run Pension Fund information system, which had a history of functioning well. Even though BOTA used ORACLE, both ORACLE and MySQL are viable options for a MIS platform. ORACLE is primarily designed for enterprises with relatively large budgets and more complex data and information needs. MySQL is a free-of-charge, open source software program that is most commonly used for database-driven web sites and for non-critical application needs. Both ORACLE and MySQL have their strengths and weaknesses in supporting a CCT program. Strengths and Weakness of ORACLE and MySQL as MIS Platforms Issue
ORACLE
MySQL
Cost
Only limited edition available for free. The full edition of ORACLE database costs thousands of USD (see Section 6 on Cost of MIS).
Free database
Ease of use
Requires specialized programming skills
Requires basic programming skills 11
Security
Wider security possibilities, and can be configured for numerous security parameters, such as username, password, profile, local and external authentication.
Three security parameters to authenticate a user: 1) user name, 2) password and 3) location
Hardware demands
Considerable
Limited
Management
Centralized
Can be decentralized
Change
Allows creation of a change plan and ability to run it before implementing
Not available
Usage patterns Owned storing procedures and files
Tracks usage patterns to help plan upgrades
Not available
Available
Not available
Priority: For the BOTA CCT program, three requirements of the database were considered a priority: a) ability to store and process efficiently an increasing amount of data, b) ability to function both offline and online for multiple users, and c) ability to ensure data protection and security. Other important prerequisites included a) a good search function (e.g., capability to search by ID, location and other criteria), b) compatibility with local social databases, such as ones developed by the World Bank and UNICEF for data merging and cross-checks, c) compatibility with financial institutions’ (banks) databases for cross-validation and generating lists of financial transactions (payment lists, transfers to banks, returns, etc.) BOTA uses ORACLE Standard One version, which was selected because of price and the number of users. It has proved to be quite sufficient for the BOTA CCT program. Although other ORACLE products offer more options, the prices of these products are much higher. The free version of the Oracle did not have the necessary features to meet the basic needs of the CCT program. (See Section 6 on Cost of MIS). Step 3 – Organizational capacity After contracting an external firm, BOTA hired a MIS specialist to work closely with the contracted developers, to ensure that CCT program expectations were communicated properly and to take over the product along the development process. This ensured that BOTA’s MIS specialist understood all the codes and scripts once the contract with developers ended. 12
Step 4 – Diagram MIS database structure and information flow The visual scheme of BOTA CCT program’s operating model enabled the M&E team and MIS designers to distinguish the number of stages in the model that became fully mirrored, later on, in the structure of the MIS. The entire operating model was segregated into a number of stages: Visual Scheme of CCT Program’s Operating Model
Developing the visual scheme of the program’s operating model is the next important step in a startup of the MIS design for a CCT. The scheme allows visualizing the model that is understood by all concerned (M&E, CCT and MIS developers) to enable them to elaborate further details of the program.
Stage1: Application stage An applicant (or a representative of the household) comes to a location where the BOTA CCT enrollment team is accepting applications. The enrollment team requires each applicant to have a minimum set of documents, such a residency permit and/ or tax registration number to show legal residence, before the applicant can proceed to the next stage.
Unique Identification Number By the end of 2013, Kazakhstan will have shift to requiring citizens to possess one unique ID number. The new unique ID number will replace all other IDs currently used, such as taxpayer number, individual identification number, and social number. Before implementation of the one unique ID number in Kazakhstan, the CCT program requires applicants to present one or more of the other IDs. 13
Once the CCT team has verified that the applicant is a legal resident, the next important data element during Stage 1 is the CCT team giving a unique identification number to the applicant to distinguish him/her from other CCT applicants. For the purpose of efficiency, only one application is completed for each household, since it is assumed that all household members share income and jointly consume goods and services. Thus, the applicant is considered the household representative. If the household is qualified to participate in the CCT program, as determined by the PMT, the applicant becomes the official recipient of cash transfers. An applicant’s individual identification information plays an important role in a relational database, since it is the “key” to identify an individual and also to retrieve data about their household. In countries with a unique identification number (UIN) system, collection of different identification numbers can be avoided. Mandatory data collected at the application stage, which must come from verifiable sources, are the following: 1. Full name – to identify the person and record the applicant in the system. 2. Date of application – to trace the history of application/case. 3. National identification number(s) - again for identification purpose, any unique number from official documents -such us a national passport or an ID card - can be used. This entry is also essential for the search function of the MIS (i.e., used as a searching term or a number when there is need to retrieve data on a concrete household) 4. Full residential address or proof of local residency – is also especially important for programs that cover certain geographical areas and in countries with high internal migration (i.e., the person could be officially registered in one area but living physically in a different place). 5. Taxpayer number, if applicable and available – in many countries, local tax regulations oblige every individual engaged in any kind of financial activities to obtain a taxpayer number. Without this number, the bank will not open an account or perform any transaction to the individual. Other important data that is collected during the application stage are: 6. Age and Gender – age and gender of the applicant are used for identification purposes, and are also important data to track and report potential age and gender differences. 7. Location and mode of application - Capturing data about the location and mode of application is important to track and report on geographical distribution of the CCT. It is also necessary if the program practices different
14
application modes, such as a) application at an enrollment center, b) distant application (by phone or online) or c) paper-based application. 8. CCT targeting criteria – After collecting personal identification data and in order to move forward with the enrollment process, the applicant needs to confirm that one or more household members meet the beneficiary criteria of the CCT program. In the BOTA CCT program there are four beneficiary criteria: a) pregnant or lactating woman, b) child 4-6 years of age not attending pre-school and c) disabled children 0 – 16 years of age, d) youth, aged 16 -19. If the household has one or more members who meet these criteria, they are allowed to enroll in the CCT program. 9. Ethnicity and language – Although often a sensitive issue, it is important to collect data on ethnicity, especially in relation to primary language of the household. This data is not used to disaggregate beneficiaries by ethnicity for reporting purposes, but rather to inform the languages needed for Information, Education and Communication (IEC) materials. Having a report that summarizes the outcome of the application stage at any point in time is important to for management decision-making. The BOTA MIS provides various reports on the application process, and one of these reports, “Pre- and Post-Enrollment Efficiency” is presented in Section 4: MIS Reports. This report shows: a) number of households that have applied, b) number of households that qualified (eligible) to enroll in the CCT program, c) number of beneficiaries with taxpayer numbers, and d) number of beneficiaries who have a bank card that could be used for cash transfers.
Stage 2: Eligibility test. Upon meeting the minimum enrollment requirements at the application stage, because the BOTA CCT program is only trying to assist potential beneficiaries from poor households (i.e., not all pregnant and lactating women), the applicant now must complete the proxy means test (PMT) to determine financial eligibility. The PMT is a tool to determine the applicant’s and their household’s poverty level. The PMT is a complex econometric test that enables the prediction of consumption level of a household and is based on a number of sociodemographic and economic variables pertaining to the household in question.
See BOTA Using Proxy Means Test (PMT) for Targeting in a CCT Program Handbook in this series
1
15
If the applicant receives a “poverty score” equal to or less than the CCT program threshold, he/she is eligible for the CCT program and goes to the next stage. If the applicant is found ineligible because their PMT poverty score was above the CCT program threshold, he/she can appeal the results of the eligibility test. (A more detailed discussion of targeting and the PMT can be found in the Using Proxy Means Test (PMT) for Targeting in a CCT Program Handbook in this series.)
Proxy Means Test (PMT) BOTA’s CCT program, the applicant must provide information for the PMT on 40 different questions (variables). A regression analysis is conducted automatically and coefficients of the variables are summed to generate a poverty score which determines whether the applying household is qualified for participation in the program. Some of these variables are single measures and others are indexes of multiple data. For example: a single measure is the level of education of every adult member in the household, and an index of multiple variables is the “livestock owned”, which is a weighted variable depending on a) type of livestock owned and b) number of livestock owned.
The PMT is administered on site through the use of netbook computers and is integrated into the MIS. The use of the PMT has a number of important advantages. First, the PMT is generally Since eligibility for the CCT program is viewed as an unbiased and transpar- based on the PMT scores, all 40 questions ent process, since the automated test must be answered during the eligibility stage. prevents human error in determining eligibility of the household. Secondly, the PMT score is generated by a complex formula and calculations that have been tested to reduce errors of inclusion (households found eligible but should be ineligible) and errors of exclusion (households found ineligible but The BOTA M&E team has developed a Standard should be eligible). Operating Procedure for Verification and Appeal Process (SOP for VAP), to offer a clear guide to field staff on how to verify household poverty status when an applicant appealed their PMT score (see: SOP for VAP in BOTA Resources section at the end of this Handbook).
The BOTA CCT program also integrated fieldbased VAP into the MIS. Field staff are able to re-run the PMT during the household verification visit. This allows for cross-checking information provided at the time of enrolment, and instantly comparing the new PMT score with the score generated at enrollment. Frequently, this procedure has identified the main reason(s) for discrepancies in the initial PMT score. If a household’s new PMT score makes it eligible, field staff will explain the change in the narrative field-box provided in the MIS. Also, it is noteworthy that automation of the VAP allows for reporting to program management on a regular and systematic basis, about the efficiency and effectiveness of the targeting instrument. 16
Having a report that summarizes the outcome of the eligibility stage at any point in time is important for management’s decision-making. The BOTA MIS provides various reports on the eligibility stage and one of these reports, “Number of Households Not Eligible by PMT Model 2008 by Urban/ Rural Division” is presented in Section 4: MIS Reports. This report shows: a) number of households found to be not eligible and b) number of households found to be eligible, by urban and rural settlements. PMT –Verification and Appeal Processes The PMT tool is based on the best available national house-
hold data (Household Budget Survey in Kazakhstan, conducted regularly by the Agency of the Republic of Kazakhstan on Statistics since 2001). But the BOTA CCT program recognizes that the PMT has a number of drawbacks in accurately identifying poor households, since it does not measure household income directly but rather predicts it based on the applicants’ answers. The most common drawbacks are errors of inclusion (leakage) and exclusion (under-coverage). They can be reduced significantly if corresponding correction strategies are integrated into operation. One such strategy is using random verification of eligible households, or special verification if the CCT staff thinks any of the information provided is dubious. In addition, all households found ineligible have the opportunity to appeal their PMT results. Stage 3: Enrollment stage After the applicant is determined to be eligible via the PMT, he/she must go through the final enrollment process that includes signing an agreement (to meet conditionalities) and filling out an application for a bank card for cash transfers as a pre-condition to participate in the program. The data that needs to be captured and recorded in the MIS at this stage are the following: 1. Date of Enrollment into the CCT Program – Date of enrollment into and planned exit from the program, if it has a fixed duration for the provision of cash assistance (i.e., either for a certain set time period, or until the beneficiary status changes: a pregnant woman gives birth, a lactating mother stops breastfeeding, a pre-school age child reaches schooling age and so forth). Enrollment records are most often made on the day of application. However, if for some reason the enrolment is postponed, then the enrollment date can be different from the application date. Postponed enrollment into the CCT program has been due to reasons such as inability of the applicant to provide some of the necessary legal documents. The program may restrict processing of an application at this stage if the applicant cannot present the minimum set of legal documents. But if that happens due to plausible reasons (e.g., residency permit The BOTA CCT program allows its applicants is currently with a government to complete the enrollment process by taking the agency for application to another eligibility test (PMT) even if the applicant cannot social service program), the pro- provide legal documents during the application stage. gram may opt to complete the application process and request Even the state-run targeted social assistance (TSA) the applicant to provide required program in Kazakhstan has reported that poor documents, to be verified by households, due to their day to day problems, are not in position to obtain some of the legal documents, field staff, at a later stage. and this can prevent or delay their participation in Tracking the “status” of a beneficiary, such as “active”, “sus-
the TSA. The all-inclusive approach enables the BOTA CCT program to save the applicant’s time.
17
pended”, “ended” or “completed,” from entering to exiting the CCT program, is vital. A “suspended” status can occur due to a failure to meet a conditionality or to migration of the beneficiary outside of the targeted area. An “ended” status can occur when the beneficiary prematurely quits the CCT program. At this point in the process the program knows what members of a given household will be benefiting from cash transfers, and for which category of beneficiaries. 2. Link to Case Manager - In order to facilitate monitoring of a beneficiary participation and meeting conditionalities, it is important to ensure that the MIS links the beneficiary to his/her case manager (or anyone assigned by the program to oversee fulfillment of conditionalities) and to a service provider, such as a health clinic or an early childhood center. Since case managers and service providers are linked to a specific location, the residential address of the beneficiary captured in Stage 1: Application is the key to establish this connection. The MIS uses the residential address to connect beneficiaries to the closest service providers and to a case manager. It is very important to make this connection between the beneficiary, service providers and a case manager at this stage, before the beneficiary is expected to begin meeting the conditionalities. Linking the beneficiary to the service provider and/or case manager can be assured if the MIS stores data about them beforehand. In the BOTA CCT program, information about local service providers (health and ECD facilities) and case managers (in most provinces) was collected and stored right before the program was launched (see more details in the MIS supplementary database section below).
3. Cash Transfer Information – The enrollment stage is a good time to obtain any bank card details or mobile numbers the eligible household has (if mobile banking is possible). Although cash transfer information may be readily available from beneficiaries, it can be more efficient if the CCT program organizes a group-based bank card or a mobile SIM card system. Using a group-based system, cash transfers can be standardized for all beneficiaries, with a possibility of receiving this service at a discounted rate. However, if this is to be organized by the CCT program, then details of the new bank cards (specifically for the CCT) would not be available until each beneficiary’s application for a bank card is processed and the bank card itself is issued to the beneficiary. Acquiring SIM cards is less cumbersome and they might be available for distribution immediately. 4. Agreement Number. The BOTA MIS generates a unique number, called an Agreement Number, for all beneficiaries who sign the CCT agreement. The Agreement NumThe BOTA CCT program had a focus of integration with the mobile banking practice in Kazakhstan. This, however, was not pursued due to the lack of local legislative support on mobile banking and a potential extra cost. The CCT program decided to organize and distribute bank cards to its beneficiaries, which required BOTA to sign an agreement with commercial banks. A number of field staff involved in the enrollment of beneficiaries was trained on administering the bank card application in the field. The completed bank card applications are delivered to the banks to issue bank cards, which are then delivered to beneficiaries by the field staff. All these additional logistics are time consuming, averaging about one month before the bank card reaches the beneficiary. 18
ber is based on various codes, such as the geographical location (e.g., AZ), beneficiary category (e.g., 4) and so forth. Although the Agreement Number is yet another unique ID, it provides a way to reference a household other than by confidential government IDs, and allows maintaining a log of the beneficiary agreements that can be disaggregated in numerous ways. The first three stages constitute the initial phase of the CCT operating model that begins with the first contact with an applicant. Data collected during this initial phase are entered into MIS once and updated only when needed. Obtaining data during the next stages of the CCT operating model takes place in a more regular way. This is stipulated by the need to track such core activities of the program as fulfillment of a conditionality and to follow cash transfers. The data flow and key actors facilitating this process at Stages 4 and 5 of the CCT operating model are illustrated below (based on the BOTA’s CCT program for pregnant/lactating women – PWL).
Overview of the data flow in Stage 4 and 5 of the CCT operating model
The BOTA MIS provides various reports on the enrollment process and two of these reports, “Verification and Appeal Process” and “Male and Female Child Beneficiaries 4-6 Years of Age in Early Childhood Development Program” are presented in Section 4: MIS Reports.
19
Stage 4: Fulfillment of Conditionalities (co-responsibility) Data collection related to fulfillment of conditionalities depends on the beneficiary category and fulfillment requirements over a specific period of time. This stage involves dynamic and regular data collection. The key actors facilitating data flow in this stage are represented by “a” to “d” of the above diagram: The beneficiary (a) must use the service as part of the conditionality, The service provider (b) provides the service, records it and communicates these data to The responsible case manager (c) who collates data and hands these data over to the next level, The field team (d) enters the data into the MIS. 1. Fulfillment of Conditionalities – The principal data that should be collected during this stage is whether or not the beneficiary fulfills the program’s conditionalities to receive a cash transfer. To be reliable, the data on whether the beneficiaries are meeting conditionalities should be collected from the service providers by case managers. Collaboration between service providers and the CCT program is based on both formal and informal arrangements, and does not occur in all locations. 2. Results Monitoring – If a CCT program aims to track the results of the beneficiary’s use of a service, such as the level of anemia of pregnant women during health clinic visits, it can be captured by the MIS as well. This will enable the CCT program to track trends of anemia among pregnant beneficiaries over time, and reports on these trends can be automatically generated. 3. Supplemental Validation of Conditionality being Met – Other data that need to be collected at this stage to support validation of conditionality being met are: - Date of conditionality monitoring - Name of the CCT field team member who conducted the monitoring; - Contact information of the service provider who confirmed the use of service; - Date the service was used; and - Outcome of the conditionality monitoring (beneficiary met or did not meet conditionality). The BOTA MIS provides various reports on the fulfillment of conditionality process and two of these reports, “Number of CCT Volunteers (case managers) by Location and Number of Beneficiaries” and “Results of the Training Attendance (coresponsibility) for Pregnant & Lactating Women Beneficiaries” are presented in 20
Section 4: MISReports. Stage 5: Cash transfer In BOTA, cash is transferred to beneficiaries on a monthly basis. This depends on the fulfillment of the conditionality by the beneficiary. At this stage, the CCT program does not require much data collection and entry. But it does requires thorough attention of the program staff to all the data collected and stored in the MIS up to this stage to accomplish the final key activity of the whole program – the cash transfer. The cash transfer stage is complex because the MIS must take into consideration multiple instances (steps) of confirmation and validation before generating a payment list. Before the payment list is generated by the MIS, it must check the following at each stage: Step 1 – check if the household passed minimum enrollment requirments. Step 2 – check if the household is eligible based on its PMT score, and (if applicable) whether the household ended up being eligible after the Verification and Appeals process. Step 3 – check several prerequisites, such as a) the category/ies of beneficiaries within the household; b) an amount of the cash transfer corresponding beneficiary category; c) the status of each beneficiary within the household to verify that the conditionality applies; and Step 4 – check whether the beneficiary fulfilled conditionality (co-responsibility) and thus is entitled to receive cash transfer. If the beneficiary household passed all the control checks then it should be included in the payment list. The payment list generated by the BOTA MIS uses the following fields: Taxpayer number
As part of the legislative requirement
Name Birth date Gender UINs Amount of transfer Bank card number
Due amount to be transferred To enable transfer
# of the agreement
For identification purposes
Transaction date
For recording purposes
For identification purposes
The CCT staff responsible for generating the payment list are also required to run a number of control tests prior to generating this list. These control tests are meant to detect errors in MIS records, mainly human errors, to avoid any problems they may create in processing transfer in cash. The accuracy of Stage 5: Cash Transfer is highly dependent on the precision of data obtained in Stage 1 to 4 of the operating model. 21
(a) Payment list clearance – Even though all the checks mentioned above are run by the MIS before generating beneficiary payment lists, it must be remembered that it is a person who programs the MIS and “clicks the buttons” to run the MIS scripts, so errors are still possible. Therefore, it is highly recommended to randomly select beneficiaries to manually verify the accuracy of the payment lists. Each CCT program can determine for itself which method to use, for example conducting a random verification of every nth households from the list, or looking for any abnormal amount of transfer and cross-verifying those transfers with records in the MIS. (b) Payment list processing and transfer – In the BOTA program, this involves matching the CCT program and the commercial bank’s pay-sheets prior to and after the cash transfer; however, this is unnecessary if no intermediary financial institution is used. The key actors who are involved in completion of the cash transfer stage are “e” – CCT staff who obtain the payment list from the MIS, “f” – the finance unit in the CCT program that processes the payment transfer to the bank, “g” – the commercial bank that deposits the cash transfer into beneficiary accounts, and “h” – an ATM, a bank branch or other financial institution near the beneficiary’s residence that allows for cash withdrawal. The BOTA MIS provides various reports on the cash transfer process and one of these reports, “CCT Beneficiaries with Tax Payer Number and/or Bank Cards and by Cash Transfers in KZT” is presented in Section 4: MIS Reports.
MIS Development Phase: Step 5 – Development of the MIS database There are several factors that influence a decision whether to develop the MIS in-house or outsource development. The key following interrelated factors must be taken into account before reaching a decision: a) Human resources b) Financial resources c) Deadline for an operational MIS d) Capacity-building policy within the organization All these factors depend upon the size of the CCT program and the level of sophistication needed in an MIS. Human resources – Having a team that includes a good software programmer and a good M&E specialist is crucial for development of an efficient and effective MIS for a CCT program. 22
The skill level of the software programmer will help determine if the MIS system should be based on an off-the-shelf software (such as MS Access for programmers with limited skills) or tailor-made (for programmers with higher level skills), and if the MIS development should be done in-house (internally) or out-sourced (externally). Just as important as a good programmer is an M&E specialist who understands the CCT processes and needs, and can work with the programmer to ensure that the MIS responds to the needs of the CCT program.
The BOTA CCT program issued an Expression of Interest for an external consultancy to develop its MIS. Four software companies responded; however, one company failed to meet the minimum requirement of providing a recent audit and tax report. The remaining companies were invited to submit their proposals and price quotations on a certain date. After a thorough review of the submitted documents, BOTA’s procurement committee recommended one software company for management’s approval. The BOTA management then, after consultation with the technical advisor, endorsed the process and signed the contract with the company. The entire procurement process took about 2-3 months.
Financial resources – Apart from the purchase of hardware for the MIS, additional financial resources are needed to cover costs associated with developing the MIS, whether that is done in-house or outsourced, using off-the-shelf solutions or tailored-made. Even though CCT programs are based on the same fundamental concept, every CCT program has its own special features, dependent on program design and local country context (social policies and services), which more often than not compels each CCT program to develop its own MIS solutions. If the program lacks in-house capacity to develop software but has sufficient budget resources to outsource the development and would like to meet an early deadline for completion, then it is better to contract an external vendor to develop the software. Deadline for Operational MIS - How soon the program needs an operational MIS is another key question which influences some of the above factors. If an on-going CCT program would like an alternative MIS (to “transition” to a new MIS) then it must consider a long-term plan with enough time allocated to complete the re-design and upgrade of the current system. In most cases, in-house development is possible if the CCT program has a staff member with appropriate skills. However, if the CCT program is just starting and has additional administrative and logistical demands in order to launch, then sufficient human resources necessary to develop the MIS might not be readily available in-house. In this case, a better option is to outsource the development of the MIS. Capacity-building policy within the organization – If a CCT program would like to build its internal capacity for the long-term operation of the program, then it will require a commitment (time and budget) to securing in-house competence. The advantages and disadvantages of internally or externally developing an MIS are well described in a World Bank technical paper (Lecuit and et al., 1999). In addition to the pros and cons reflected on pg. 23 of that paper, the following strengths, weaknesses, opportunities and threats (SWOT) analysis of having the MIS developed internally or externally can be taken into consideration:
23
SWOT Analysis of Using External or In-house Developer for a CCT Program MIS
Internal Developer
External Developer
Strength
Weaknesses
Opportunities
Threats
Contributes to developing learning environment while designing the MIS
Requires administrative and logistical arrangements – recruitment and placement of the staff
Search qualified specialists through specialized resources – networks
Lack of qualified specialists
Effective to build inhouse capacity
Can be time-consuming
Contract individual consultant to support the staff member
Cost-effective (low cost)
Entails involvement of other staff to communicate the design
Formal communication strategy – set up working groups, etc.
Staff turn-over
Time saving, due to involvement of a group of people, if the firm – not an individual - is contracted.
Entails going through the procurement process – Expression of Interest, Request for Quotations/ Proposals, negotiation, etc., May require additional approvals at each stage.
Well-structured organization’s procurement policies and procedures or alternative procurement procedures
Lack of consultancies that meet all requirements of the bidding process
Consultant transfers lessons from their past experience with developing a social database in the same or similar context
It may take time before external developers understand all features of the CCT program
Ensure formal communication strategy in place for effective communication – set checkpoints, regular meeting, etc.
Irregular/unorganized communication between the designer and developer will hinder the process
Documentation (technical books) may accompany development.
Developed documentations might not be relevant after sometime as the MIS evolves
Update documentations as the MIS evolves
Updating documentation may require additional effort (time and funds)
May increase the cost
MIS Development Timeline Depending on the type of the MIS platform chosen, the capacity of programmers and the amount of resources available, time needed for the design and development of a CCT program’s MIS can vary from a few months to more than one year. Lecuit et al. (1999) estimate that the time needed to design and develop a MIS averages 6 months, launching it takes an additional 2 months, and a follow-up redesign/review of the MIS - up to 10 months. The table below presents the time it took to design and develop BOTA’s CCT program MIS, using an external developer.
24
Selecting and contracting service provider (developer)
№
Activities
1
Call for a) Expression of Interest (EoI) and b) proposals
2
Selection of the proposal and price quote
3
Contract negotiation and award Sub-total:
Time line
Minimum number of required EoI received 2 months
Minimum number of proposals for review Signed contract
2 months
- Development and finalization of the ToR with technical details 1,5 - Detailed analysis of the MIS 4 month process and requirements to the system; - Finalization and endorsement of the ToR - Trial version of the software - Development of trial version of the software - Development of the testing 2,5 method with follow-up 5 months documentation - Acceptance Test and Operational Testing Development - Post-trial adjustment of the of the MIS system, if needed - Operationalization and rollout of the software - Training of users in pilot provinces (oblasts); - Operational testing in the field; - Revision/update of the 2 months 6 system and documentation of the operational testing in the field; - Development of the method for roll out; - Acceptance testing of the system Sub-total: 6 months 8,5 Total: months Post-development fine-tuning, adjustment
Results
Final detailed ToR for contract
- Testing Method - Testing records/ documents - Acceptance report and deed of conveyance of the system for operationalization; - raining reports - Manual for roll out and introduction to the system; - Administrator’s manual - User’s manual; - Acceptance report and deed of conveyance of the system.
Ongoing 25
Step 6 – Testing the MIS database
The basic objective of the testing phase is to make sure that initial modules of the MIS are complete and that the system allows for easy and reliable data entry, storage and retrieval using the configured security credentials. It is also important to guarantee compatibility of different elements of the system, such as offline and online applications for processing and transferring data. The completeness of the modules is assured by: •
Confirming that all data fields are present to capture minimum data requirements;
•
Testing data entry to verify correctness of values in coded data fields (i.e., ensuring text entries are strings, numbers are numeric values, field sizes can handle the required number of characters, etc.); and
•
Ensuring that data can be saved (stored) and accessed using beneficiary ID and retrieved by running special queries.
BOTA’s CCT program’s MIS, prior to launch and roll-out, underwent a number of pilot tests conducted in the office and in the field. The company contracted to develop the MIS made accessible an online trial version of the MIS that was regularly reviewed by the M&E unit during the last 3 months of the development process. A trial version of the offline application, to be used in the field with the netbook computers, was also made available during this period for field testing. Throughout the test period, special attention was given to the accuracy of a complex PMT formula integrated into the MIS. This was assured by cross-verification of the PMT score derived from the MIS and the PMT score generated by an observable Excel model of the formula. The pre-launch testing of the system lasted about 3 months and the contracted firm continued providing its technical support and maintenance for 3 months after the launch.
Upgrading the MIS Every change to the MIS should be introduced only after a thorough analysis of why the change is important, how the change could affect other modules, and how much downtime of the MIS is required, to avoid any interruption with program delivery. Prior to integrating any changes to the MIS, it is recommended to pilot-test them on a “test” version of the MIS (an MIS clone) that is maintained specifically for this purpose. Once a new script or a new module is developed, it is first needs tested by the MIS specialists and other authorized users on the “test” version of the MIS for any possible errors it may cause and elements that may affect functionality of the overall MIS. Once the testing is completed with no more occurring errors and the “test” MIS functions normally, the developed script or module can be transferred to the real MIS. End users should be involved in the upgrading progress. Just as importantly, they should immediately communicate any errors they observe later on, during the real-life application in the field. This two way communication, as well as commitment of all users to improve functionality of the MIS, is vital in maintaining effectiveness and efficiency of the MIS. During almost 4 years, the CCT program’s MIS at BOTA has evolved greatly, along with the program expansion, to accommodate new emerging needs and to remain effective and efficient in managing the program for results. This was made possible because of the flexibility of the system assured by the M&E team during the development of the MIS -as well as the in-house capacity to implement changes. 26
The diagram below shows some key upgrades that have been made to the MIS since the system launch. Additional upgrades will continue in the future as well, probably in a less intensive manner, to meet new emerging needs of the program. Diagram – Upgrading and Expansion of the BOTA MIS Functions
Step 7 – Implement the MIS database The following are basic issues that need to be considered when launching a MIS in a CCT program. 1) Identify MIS actors and their responsibilities – Depending on a CCT program’s organizational framework, there are two groups of actors involved in implementation of the MIS. The first are technical specialists, or so-called central or field-based MIS administrators, whose core responsibilities are to maintain the system and provide troubleshooting. They are responsible for the administration and maintenance of the MIS at different levels and require advanced knowledge of IT techniques and complete command of the software. The MIS administrators are given different levels of access authorization depending on their roles and responsibilities and program arrangements. Some administrators might be given authorization to view and retrieve data only. With proper technical knowledge and skills, they also can carry out any upgrading or adjustments of the MIS. This group of administrators plays a key role in the launch and implementation of the MIS.
Key Facts About BOTA’s CCT Program MIS 1. Number of authorized users – 100 2. Data contained - Over 5 million single data for 30,000 applied / beneficiary HHs). 3. Data fields - Almost 500 different data fields. 4. Data screens – 20 different data screens. 5. Management decentralized - 5 pilot provinces can each manage their own part of the system, also unlimited.
27
The second group of actors will be comprised of general users: staff that have a specific task, such as data entry, running reports, or generating payment lists. Most of the end users will be field staff and their primary responsibility will be to verify the quality of data and generate reports and payment lists. This group of users does not require indepth computer skills, the only requirements are rudimentary typing and basic data entry ability. General users will need authorization to view, enter and edit data of their own entry only. 2) Ensure Information Technology is ready to implement the MIS – Information technology includes all the software and hardware needed to implement the MIS. Software and hardware compatibility is essential. Even minor incompatibility between software and hardware, or between different hardware, can cause implementation delays. 3) Develop MIS reference manuals – MIS reference manuals will guide both groups of users during the implementation of the MIS. Reference manuals should be developed to meet different needs: one should be a step-by-step, easy to follow user’s manual that presents what, how, when, why and by whom certain data fields in the MIS need to be filled. The second reference manual, a more complex and technical one, should be developed for MIS administrators and programmers to guide maintenance and upgrading of the MIS. Both reference manuals will serve as important documents to enable knowledge transfer and facilitate in-house learning. 4) Organize customized training – Training and capacity building of CCT program staff in the use of the MIS is vital. However, training must be tailored to the needs of each group mentioned above. Make sure that in addition to theoretical issues being covered, the training also includes some practical sessions with the application of the MIS in real-life situations. The MIS administrators can organize and deliver training to end users, using the user’s manual. If the program aims to build in-house capacity to oversee administration and maintenance of the MIS, it can be best assured through continuous, on–going training. Also, a formal, end-of-assignment training for MIS administrators by the external developers is a good opportunity to improve the MIS administrators’ technical knowledge and skills. 5) Follow-up and feedback – Once the MIS is launched, it is important to maintain close contact between end users and administrators for troubleshooting and enhancing the overall MIS. This can best be achieved by having a formal communication mechanism that establishes procedures and dates for review of the MIS performance. Examples of issues that need constant follow-up and feedback are: a) ability of the system to capture the required data, b) the need for adjusting existing data fields (coded value, sequence), and c) automation of data fields to minimize human error.
28
Overview of the BOTA CCT Program MIS Implementation Process
6) Informed consent – Applying for the CCT program is voluntary, however, the application process requires the collection of personal data. If the applicant is found to be eligible for the CCT program, additional personal data is collected throughout the time of their involvement in the program. Some of this data may be very sensitive. Before collecting data it is important to obtain consent from every applicant and participant. Although the collected data is mainly used for management, the program may also use it for cross-verification purposes: such as verifying employment status or cross-verifying with a national social assistance database if the applicant or beneficiary is participating in other social programs. Signed consent should be obtained from every applicant.
3. MIS Reports Implementing a MIS in a CCT program requires not only collection of data but the processing and transformation of that data into relevant and timely information which management needs for decision-making. The volume of data that needs to be collected and processed has a direct relationship with the complexity of the MIS: the more data that is required, the more complex the MIS. Increasingly, both software and hardware are making the processing of data easier and quicker. In order to avoid fatigue from too much data and information, the reporting system should be based on the M&E plan used at the design stage so the MIS can retrieve data and information specified by the M&E plan. The key advantage of this integrated relationship is that the M&E plan gives a clear direction for your reporting system - that is, what data and information to report, how often and to whom. However, it is not always possible to reflect all types of data and information needs in a single M&E plan. Thus, the reporting system should not be restricted only to the M&E Plan but should also allow for opportunities to explore other important data and information. 29
There is not a standard template to report data and information for a CCT program. The M&E staff involved in the reporting process must exercise different approaches to come up with a template that is complete, readable, easily perceived and would not generate more questions. The key principle that must be adhered to during the development of reports is: “report data and information with an outside reader in mind”. Imagine a person who does not have any idea about what the program is but who can draw conclusions and conceptualize the program from the data provided in the report.” Samples of BOTA data table reports for one province, generated out of data stored in the MIS: Example of Report for Stage 1 – Application This report shows that 7,583 households applied, 7,547 of these households were checked regarding eligibility criteria, and of these 6,687 qualified for the CCT program. Of the households that qualified, 6,674 (or 99%) had a taxpayer number and 5,035 (75%) had a bank card. Pre-post Enrollment Efficiency Province Akmola
Households applied
Households tested
Households qualified
Applicant with taxpayer number
Applicant with bank card
Households Loss % loss
7 583 0 0
7 547 36 0,5
6 687 860 11,4
6 674 13 0,2
5 035 1 639 24,6
Example of Report for Stage 2 – Eligibility Test This report, generated in 2009, shows the number of households, urban and rural, that were found to be eligible and ineligible based on the proxy means test (PMT). Number of Households Not Eligible by PMT Model 2009 by Urban/ Rural Division Type of Settlement Rural Urban
Total
30
Number of Households Not Eligible
Eligible
Total
Number
722
21 416
% within Rural
3,3
96,7
22 138 100
Number
239
2 326
2 565
% within Urban
9,3
90,7
100
Number
961
23 742
% within Total
3,9
96,1
24 703 100
Example of Report for Stage 2 – Verification Process This report shows that of 6,687 household that were eligible (met the CCT program’s criteria), 132 households (or 2%) were verified by CCT staff based on a random selection process. Of these 132 households, 16 (or 12 %) were found to be above poverty level since they under-reported (intentionally or unintentionally) their assets asked about in the PMT. Verification Process Total Verified
Province
Total Eligible HHs
Number
Akmola
6 687
132
%
Below Poverty Level
Above Poverty Level
Concealed Assets at Enrollment (PMT Variables)
Correct Assets
2,0%
116
16
16
116
Example of Report for Stage 3 – Enrollment Process This report is about households that have met the criterion of having one or more children 4 to 6 years of age and have passed the PMT. A total of 4,051 children age 4-6 were participating in the CCT program, of which 2,006 (49%) were boys and 2,045 (51%) were girls. Male and Female Child Beneficiaries 4-6 Years of Age in Early Childhood Development Program Province
Districts
Total in ECD
Males
Females
Akmola
Akkol Atbasar Burabai Bulandy Esil Kokshetau Sandyktau Shortandy Zhaksy Zharkain Zerendy Yenbekshilder
297 466 685 585 190 87 195 114 315 188 808 121 4 051 (100%)
152 219 349 294 96 45 93 50 144 97 414 53 2 006 (49.1%)
145 247 336 291 94 42 102 64 171 91 394 68 2 045 (50.9)
Total Akmola
31
Example of Report for Stage 3 – Link to Case Manager This report shows the number of volunteers per district, the number of cases (beneficiaries) they assisted (by CCT program criteria), and average number of beneficiaries per volunteer. Number of CCT Volunteers (Case Managers) by Location and Number of Beneficiaries
Early childhood development
Homebased care
Pregnant & lactating women
Youth livelihoods
Total Number of Beneficiaries
Caseload per volunteer
12
244
35
80
35
394
32,8
Atbasar Bulandy
21 18
349 401
95 72
228 159
167 101
839 733
40,0 40,7
Burabai
29
398
134
221
36
789
27,2
Esil
13
122
16
50
33
221
17,0
Kokshetau city
2
63
15
60
7
145
72,5
Sandyktau
16
146
42
77
17
282
17,6
Shortandy
9
24
4
7
3
38
4,2
Zerendy
25
494
132
221
202
1 049
42,0
Zhaksy
16
172
43
75
122
412
25,8
Zharkain
16
139
34
68
116
357
22,3
Yenbekshilder
16
85
14
46
34
179
11,2
Total for Akmola
193
2 637
636
1 292
873
5438
28,2
Akmola
Province
Total number of volunteers
Number of beneficiaries covered by volunteers by type
Districts
Akkol
Example of Report for Stage 5 – Fulfillment and Non-Fulfillment of Conditionalities This report shows results of the group of beneficiaries, pregnant and lactating women (PLW), who did not meet the conditionality of attending a total of six (6) education sessions related to maternal health, pre-natal care and benefits of health clinic visits. 32
Province
Results of the Training Attendance (co-responsibility) for Pregnant & Lactating Women Beneficiaries. 2011 Enrollment Month
Akmola
Jan 2011 Feb 2011 Mar 2011 Apr 2011 May 2011 Jun 2011 Jul 2011 Aug 2011
# of PLW ben 179 215 272 246 234 35 89 137
2010 - 2011 Enrollment
# of beneficiaries missed education class by sessions
Cumulative
S1
S2
S3
S4
S5
S6
1 721 1 936 2 208 2 454 2 688 2 723 2 812 2 949
3 3 0 2 5 0 0 0
3 0 0 3 5 0 0 0
1 3 0 0 1 1 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
Example of Report for Stage 5 – Cash Transfer This report indicates the cumulative number of households (6,674), number of beneficiaries with taxpayer numbers (8,698, which indicates several members in a household are CCT eligible), number of beneficiaries with bank cards, total number of cash transfers from the beginning of the CCT program to July 2010, cumulative amount of cash transferred to July 2010, and the average amount of cash transferred each month per household. CCT Beneficiaries with Tax Payer Number and/or Bank Cards and by Cash Transfers in KZT Province
Akmola
%
# cash transfers to July 2011
Cumulative amount transferred to July 2011
Average amount of transfer per household per month
76,9%
48 566
201 475 496
4 148
Beneficiaries with Bank Card
Households with tax payer number
Beneficiaries with tax payer number
Number
6 674
8 698
6 685
4. Maintenance of a MIS On-going maintenance is of vital importance to ensure trouble-free functioning of the MIS, especially when the data flow is intensive or if the system undergoes modifications to meet emerging needs. The maintenance of the MIS is best assured by its developers or anyone else who is familiar with the specifics of the system. Post-deploying warranty maintenance is a common practice when 33
a third party is contracted to develop the MIS. The warranty usually lasts for at least 3 months and may continue as long as the program feels there is a need and can afford it. The program may also choose to engage external specialists only as and when needed. Usually, as soon as the new MIS starts being actively used, a need arises for some adjustment or upgrading; thus, additional short-term work is needed periodically to fine-tune or upgrade the system, which increases the need for maintenance. Regardless of whether the maintenance is conducted in-house or outsourced, a systematic back-up of the MIS will be essential to recover system data and information when needed. It is highly recommended that you set up a daily automatic back-up of the system. On-going maintenance tasks include being aware of and protecting the MIS from threats, both internal and external. Presented in the table below are examples of internal and external threats that should be mitigated and possible preventative measures. Internal and External Threats to a MIS & Preventative Measures Factors
Internal threats
External threats
Description A password is unintentionally obtained by a 3rd party. This can happen if a user has a simple password or accesses the MIS from a public computer where the password might get saved in a “cookie.” Use of MIS password by CCT staff who have left the program.
Users should be warned not to use simple IDs and access the MIS from public source with caution. MIS administrator should be in the clearance list of the HR check-out form, to cancel the account.
Unauthorized disclosure of MIS password.
Corresponding Information Security and Confidentiality Policy in place.
Force-majeure situation, damaged server due to a disaster (natural or man-made).
Backup system in a different place than the main data repository.
Access of MIS information by someone other than program staff or authorized user.
Secure system for transmitting and receiving information via Internet – by setting https protocol.
Server virus attack
34
Preventive Measures
Install anti-virus software for the MIS server.
5. Cost of a MIS Investment is often a prerequisite for useful results. The amount of investment needed for a useful MIS will depend on several factors. First, there are two basic costs that should be considered: a) development costs and b) operational costs. The costs associated with each of these will depend on the platform chosen, complexity of structure, quantity of hardware needed, and local costs. The true costs may vary greatly and depend on the complexity of the MIS and whether the program plans to adapt an existing system or to develop the system from scratch. While the cost of the hardware and software is more or less the same across different places, the cost of the development may vary depending on the local software development market. It is important to research the local market to project the budget for development and operationalization of the MIS. a) Design and Development Costs Design – In the BOTA CCT program, the design phase involved one fulltime MIS specialist to communicate the design and expectations from the MIS to developers (an external company), and regular support from the management for decision making. The cost of the MIS specialist in Kazakhstan is around 800 -1,000 USD per month, and the design and development phase took approximately 8 months of almost full-time involvement, for a total of 7,200 USD. Development – The contract with the software company to develop the MIS was for 45,000 USD. The price included not only development of the MIS for the CCT program but also for two others (Social Service Support Program and Tuition Assistance Program). Nevertheless, due to the complexity of the CCT program, the vast majority of the costs (70%, or $32,000) were related to it. The software company involved five personnel in the development of the MIS. The table below shows each position needed, number of personnel required, and their responsibilities. #
Personnel
1
Project Supervisor
Units 1
2
Programmer
2
3
Analyst
1
4
Testing Engineer
1
Key Responsibility To provide management support to the project from the firm’s side To write a computer program to process information to be used for the application A contact person from the firm’s side who converted the needs and expectations to complex programming language used by the programmers Ensured that the product met daily demands of usage. 35
The team composition and number of personnel needed is very much dependent on the needs and complexity of the MIS. Expenses associated with training end users (conducted by developers), such as travel, accommodation, and time of participating field staff, are not included in this estimate. The hardware, software and overall costs of the MIS are presented in tables below. Hardware Costs for the MIS Item
Specifications
Hardware
Server (NON-TPM – Power Edge R410 Rack Chassis) with configuration CPU: Intel Xeon E5620, 4C, 2.40GHz, 12M Cache Memory: DDR3-1066MHz, 16GB, HDD: RAID Controller, 3 x 300GB, SAS 1U Rack Bezel Net book for offline application. (Acer Aspire One D255-2BQkk) with configuration CPU: Intel Atom 1.66 Hz; Memory: 1 Gb HDD: 160 Gb
Cost
~ 4,000 USD ~ 300 USD each = ~2,100 USD total ( 7 field offices x $300)
*While one server is sufficient to run the whole MIS, the number of required net-books depends on the number of field staff involved in enrollment and applying the MIS offline application. On average, one field office required at least 5 -10 net-books for the team of 10 field members.
Software Costs for the MIS Item
Specifications Oracle SE One 64 bit Software for the server Apache 64bit Oracle XE 32 bit Software for the netbook Apache 32bit TOTAL for operationalization:
Cost ~1,150 USD Freeware Freeware Freeware 5,150 USD
Overall Costs of the MIS Item M&E Specialist Netbooks Server Software for server External developer Total 36
Cost $7,200 $2,100 $4,000 $1,150 $32,000 ~$60,000
The overall cost of the MIS for BOTA CCT program in Kazakhstan was 60,000 USD, which is less than 0.1% of the total budget allocation for the CCT program. During the design of the MIS, the BOTA CCT program investigated using hand-held Personal Digital Assistants (PDAs or handheld-computers) to automate the enrollment stage. Although PDAs are increasingly used for data collection and clearly have advantages (e.g., time saving on data entry, and quality assurance), in the BOTA CCT program PDAs were not used for enrollment for the following reasons: • As mentioned in Stage 1: Application Process, the names (first and last) of all household members (on average 5 members) and their unique IDs (i.e., national ID, taxpayer number) must be obtained; it was determined that inputting all these on a small touch screen would not be as comfortable for field staff as a keyboard. • The PMT, used during the application process to determine eligibility, uses a complex formula to calculate a household’s poverty score. The programmers were uncertain how reliably the PDAs could run the PMT. • The software company contracted to develop the MIS lacked expertise to program an offline version of the MIS on the PDAs and to have the PDAs synchronized with central MIS. • It was doubtful that the PDAs could be linked to check for duplication and avoid double-enrolment. • PDA software did not have national (Kazakh) and Russian language support. • The average cost of the PDAs was about the same as the netbook computer. There was certainty that netbook computers could adequately mange the above concerns, while uncertainty remained around the role of PDAs therefore netbook computers were chosen for primary data collection in the field. b) On-going operationalization costs These costs may vary depending on the complexity and the scope of the program and its MIS, as well as the resources available for on-going operationalization. Human resources (except for program staff) that are needed to ensure on-going operationalization include: Central MIS Administrator – a central MIS administrator is needed to ensure trouble-free and secure functioning of the MIS, and to integrate any necessary modifications into the system over time. It is important that the MIS administrator possess at least minimal pro37
gramming skills to assist with upgrading. However, if the MIS requires complex modifications, it is more efficient to contract a temporary external consultant, under the supervision of the central MIS administrator who is then free to continue with his/her daily responsibilities. It is recommended that the central MIS administrator have adequate back-up support to ensure continuous trouble-free operation of the MIS, especially if the data flow and processing become demanding. In the BOTA CCT program, once the numbers of applicants/ beneficiaries went beyond 5,000 households, it became necessary to hire an assistant to the MIS administrator. Reliable connection – Since the BOTA CCT program allows for application and enrollment in the field, it requires a reliable Internet connection for uploading the application and enrollment data to the central server. To date, a broadband connection provided by the national Internet service provider (ISP) has been reliable and sufficient for the CCT program’s data transmission needs. Anti-virus program – All computers need to have the latest antivirus software installed to detect and eliminate potential virus threats that can endanger all data. The BOTA CCT program uses Kaspersky server edition, but there are other good anti-virus programs available.
6. Supplementary Databases in a MIS The BOTA CCT program has two supplemental databases: a) Service Provider and Case Manager and b) Geographical Information System. a) Service Provider and Case Manager Database – This database is necessary for the CCT program because it links the beneficiary (demand side) with the service provider (supply side) and with the CCT volunteer (case manager) who ensures fulfillment of co-responsibility (conditionality), as described in Stage 3 - Enrollment Process. In countries with relatively well organized service delivery, service providers such as health care facilities are located in communities or districts where CCT recipients live. Consequently, the service provider has an opportunity to follow its clients who are enrolled in the CCT program, and the beneficiary is made aware of the nearest service provider to meet their co-responsibility (conditionality). Identifying a service provider close to where the CCT beneficiary resides is important in order to ensure access and enable fulfillment of conditionality. The same is relevant to program volunteers - case managers who are recruited and appointed by the CCT program (more can be read about CCT program volunteers in the How To Implement a Conditional Cash Transfer Program Handbook in this series). A CCT case manager helps arrange beneficiary visits to the service provider and assists in maintaining communication between the beneficiary and BOTA’s CCT office. A sample of information reported to CCT program managers from the Service Provider and Case Manager Databases includes: Service provider: • Total number of service providers, by type of services. 38
• The caseload of service providers, by CCT beneficiaries (i.e., how many CCT beneficiaries a service provider handles). • The extent of the coverage (area and population) of service providers. • The physical address of the service provider to develop maps in the GIS supplemental database (described below). • Beneficiary satisfaction with the services received, by service provider. Case managers: • Total number of case managers. • Caseload, by case managers • Coverage data: area and population • Physical address of the case manager, necessary for developing maps. • Beneficiary feedback to the case manager Schematic of the Service Provider & Case Manager Supplemental Database in the CCT MIS
39
b) Geographical Information System (GIS) - A GIS integrates hardware, software and data in order to capture, manage, analyze, and display all forms of geographically referenced information. The use of the GIS in the CCT program allows management to map various types of data in different ways to reveal patterns and trends. The BOTA CCT program is currently developing its plans for GIS data. One of the first GIS reports will show CCT beneficiary households by location. Other reports are being discussed and planned for, such as reports by geographical location of service providers and location of households not meeting conditionality. In order to develop a GIS map of CCT beneficiary households, the following were needed: Geo-database – It is recommended to use an officially approved document (classifier, if available) that contains the place-names of all official administrative and territorial units in the program area, whether small (village, city, province) or large (country). Having a geo-database of official administrative and territorial units has a number of advantages: a) it enables an automated selection of a living place of the applicant, from a drop-down menu during the application stage and b) it allows for setting the connection to a service provider and case manager at the enrollment stage. Also, a geo-database can help to avoid human errors that may occur if locations are entered in the MIS manually. Geo-codes – A geo-code is a geographical code that identifies a point or area on earth, generally represented as latitude and longitude values. Geo-codes are needed if the CCT program plans to integrate GIS mapping into its analysis and reporting. One challenge to GIS mapping is that that geo-codes are not always available publicly, and collecting them physically through the use of a handheld GPS unit would require extra time, logistics and costs. Mapping software – There are numerous applications available such as Google Maps and ARC-GIS, both of which are pre-enabled to display geo-codes.
40
References Arnold, Catherine, Tim Conway, Matthew Greenslade. 2011. Cash Transfers: Literature Review. Policy Division, United Kingdom Department for International Development. Balden, Cesar and Maria D. Arribas-Baňos. 2008. Management Information Systems in Social Safety Net Programs: a look at accountability and control mechanisms. SP Discussion Paper No.0819, The World Bank, Washington, DC. Gallaher, Mike. 2006. A Technology White Paper on Improving the Efficiency of Social Safety Net Delivery in Low Income Countries: an introduction to available and emerging mobile technologies. Third International Conference on Conditional Cash Transfers. Istanbul, Turkey. Lecuit, Luc, John Elder, Christian Hurtado, Francois Tantrua, Kamal Siblini and MauriziaTovo. 1999. DeMIStifying MIS: Guidelines for Management Information Systems in Social Funds. World Bank Technical Paper No. 443. Washington, D.C. Siles, Rodolfo. 2004. Management Information Systems; guidelines for planning, implementing, and managing a DME project information system. CARE International. Villalobos, Veronica Silva, with Gaston Blanco and Lucy Bassett. 2010. Management Information Systems for CCTs and Social Protection Systems in Latin America: a tool for improved program management and evidence-based decision-making. Social Protection Sector, The World Bank, Latin American and Caribbean Region. World Bank. 2008. Management Information Systems in Social Safety Net Programs: a look at accountability and control mechanism. Safety Nets Primer Notes. No. 28.
BOTA Resources These documents can be downloaded at: CCT Manual, June, 2012, BOTA Foundation, http://www.bota.kz/uploads/51ff67bb300a6.doc BOTA M&E Handbook, November, 2010, BOTA Foundation http://www.bota.kz/uploads/51ff67bcec715.pdf Innovative approach to social safety net: conditional cash transfer experience 2009-2011, May, 2012, BOTA Foundation http://bota.kz/en/index.php/pages/index/87 41
42
BOTA beneficiaries 43
“BOTA” FOUNDATION 160, Dostyk ave., 050051, Almaty, Republic of Kazakhstan Tel.: + 7 (727) 264 12 69/70
[email protected] www.bota.kz
![FREE [DOWNLOAD] MASTERING YOUR ADULT ADHD: A ...](https://m.moam.info/img/260x300/free-download-mastering-your-adult-adhd-a-_6476f82e097c474d228b55db.jpg)
![[PDF] Mastering Your Adult ADHD - Google Sites](https://m.moam.info/img/260x300/pdf-mastering-your-adult-adhd-google-sites_6477c300097c474d228c6694.jpg)
