Community-Integrated Management of Childhood Illness (IMCI) Programme Evaluation

Overview

The proposed four-year randomized study will attempt to test the hypothesis that community-based child health interventions in conjunction with facility-based IMCI will improve child care practices, nutritional status and child survival. The objectives of this research are: 1. To measure the effectiveness of the community-based interventions in improving selected child care practices in the community. 2. To measure the effectiveness of the community-based interventions in improving child nutritional status and in reducing child morbidity and mortality. 3. To document the process of implementation of community-based interventions at scale to promote selected key family and community practices related to child health. 4. To undertake cost-effectiveness analysis of the interventions. Integrated Management of Childhood Illness (IMCI) is a strategy developed by the World Health Organization (WHO) and UNICEF to reduce childhood mortality and morbidity and to contribute to improved growth and development of children under-5 years of age Experience suggests that a purely facility based strategy will not reach the significant portion of the population that does not have access to or choose not to use a health facility. Links between the service providers and families at household levels is essential in order to ensure that families have the knowledge, skills and ability to provide appropriate preventative and curative care to their children. However, globally, actual evidence of effectiveness of community-based IMCI interventions implemented at scale is meager. C-IMCI in Bangladesh will be implemented by GoB in partnership with NGOs and also through active participation of different community groups, civil societies, and the private sector. Both GoB and UNICEF/Bangladesh have agreed that an evaluation of the C-IMCI implementation by GoB, as proposed here, would be very opportune and useful in providing the evidence and analysis of lessons that will guide further scale-up in the country. A cluster-randomized design will be used for this evaluation. Fourteen Upazilas where facility-based IMCI is already in place will be selected, and 7 upazillas will be randomly allocated to C-IMCI intervention and 7 to comparison. Community-based IMCI in the intervention upazillas will be implemented by GoB through the district health system while in the comparison upazillas existing services will continue, including facility-based IMCI.

Overall study design For the proposed cluster randomized community-based intervention trial, upazilas (Sub-districts) will be randomized to either intervention or comparison. The upazila is the lowest local government structure with a health management unit. The usual upazila has a population of about 200,000 to 250,000. Fourteen upazilas where facility-based IMCI is already in place will be selected for the study, and 7 upazilas will be randomized to intervention and 7 to comparison. The objective of the randomization is to achieve a balance on the basis of facility IMCI interventions, functionality of health system, demographic and socioeconomic indicators, and population size. Community-based IMCI in the intervention upazilas will be implemented through the district health system while in the comparison upazilas existing services will continue, including facility-based IMCI. Selection of Upazilas and randomization The upazilas for the study will be selected based on the following steps: Step 1: Eligibility Facility based IMCI has been implemented in 300 upazilas as of December 2008. Most of these upazilas were selected from parts of Bangladesh with high rates of under-five mortality. In addition, the IMCI expansion plan of the Government of Bangladesh emphasizes achieving complete coverage of a district and selecting upazilas where other implementing partners are present so as to facilitate eventual expansion of community IMCI interventions. Out of 300 upazilas, 100 were excluded where facility IMCI is in very early stage. Within remaining 200 upazilas, Government has decided to implement community IMCI in all upazials of 14 districts and began the implementation in 5 districts in 2008 and early 2009. Seven pairs of upazilas will be selected form the upzilas of these 6 districts for evaluation. We are using district mortality rate as upazila under-five mortality rates are not available. We assume that that the variation of the true upazila under-five mortality rates is similar to that of district rates. We also exclude upazilas with or adjoining municipal areas as these are atypical. Step 2: Pairing and re-randomization The fourteen upazilas will be paired based on demographic and socio-economic indicators: population density, literacy rate (total \& female), male and female school attendance, and work status, source of drinking water, sanitary toilet, electricity, and flow of remittance. . A composite score using Principal Component Analysis will be constructed from these demographic and socio-economic indicators. This score will be used to sort upazilas within a district. In addition geographical location (proximity, within same district), population size, and health system functionality) assessed using immunization coverage) will also be considered in pairing of upazilas within a district. The National Nutrition Program (NNP) is the largest nutrition programme being implemented by GoB. Whether or not NNP is covering a particular upazila or is planning to is an important piece of information for this study as NNP was designed to reduce the prevalence of moderate and severe underweight in young children, address micronutrient deficiencies, increase pregnancy weight gain, reduce the incidence of low birth weight, and the prevalence of iron-deficiency anaemia among adolescent girls and pregnant women. Ongoing or planned implementation of NNP or the presence of any other substantive and similar community-based programmes will also be considered before selecting a upazila. Seven (07) pairs of upazilas will be formed and one upazila from each of the 7 pairs would be randomly assigned to C-IMCI intervention arm and the other to comparison. We have increased the number of pairs for the evaluation study from six to seven based on discussions with GoB, UNICEF, and WHO. The additional pair was from Bandarban district which was selected at the request of the MNCS programme (GoB/UNICEF). Bandarban district is in the hilly South-Eastern region of Bangladesh and is home to ethnically distinct populations and GoB/UNICEF is keen on including the district in the evaluation as programme experiences in these populations are limited. Documentation and monitoring of implementation Documentation and monitoring is essential to achieve the proposal's third aim - to document the process of implementation of community-based interventions at scale to promote selected key family and community practices related to child health. This documentation and monitoring of the implementation will be conducted both qualitatively and quantitatively to assess the feasibility and acceptability of the intervention and to document how and why the intervention may or may not work. A combination of qualitative and quantitative documentation is important to triangulate the findings of the study. Implementation of the C-IMCI component of IMCI will involve a series of processes, undertaken by the government. To identify the lessons learnt from the processes so that they may be useful in other settings and for the intervention itself, documentation and monitoring of key processes are very important. Five key processes have been identified for documentation - situation analysis, partnerships with NGOs, training, implementation and community perceptions and as mentioned earlier, these processes will be documented both qualitatively and quantitatively. While routine MIS data and other questionnaires will be analysed for quantitative documentation, for qualitative documentation, periodic in-depth studies will be carried out with community members, different health care providers and other populations involved with the intervention. These processes along with the system for process documentation are described below. 1. Documenting situation analyses The process by which the Upazilla health manager conducts the situation analysis of the human resources and communication mediums available in the upazilla will be documented. This will include documentation of what method and sources are used to establish the number of government health workers (health assistants and family welfare assistants), village doctors, NGO health workers. Additionally the methods and sources of determining who in the community are considered opinion leaders, the number of mosques and imams will be documented. Especially important to this process is to document the confounding factors that may effect the inclusion or exclusion of human resources or communication mediums (e.g. village doctors may not be included because they do not wish to participate, NGOs may be excluded because of lack of necessary skills or infrastructure etc.). This will be documented prior to the intervention. 2. Documenting the development of Partnerships with NGOs Creation of cross-institutional partnerships is a major and innovative part of the proposal. For this reason the documentation of this process is vitally important for its application in other areas of Bangladesh and outside of Bangladesh. Documentation of how partnerships with NGOs are established, the roles of the government and the NGO in the partnership, modes of communication and other factors affecting the functioning of the partnership should be identified. Further to this the relationships of the partner organization with mothers, workers and community leaders need to be documented. This will be documented through out the study. 3. Documenting training The government will facilitate a training module for each component of the intervention (R1-R5). The major goal of documentation will be to record the process of the training and the quality of the training. In doing so, special focus will be given to the recruitment methods of trainers and participants, their attendance, completion of the training modules, factors affecting their participation, their knowledge retention, attitudes to training and intentions to practice what they have learnt. This will be documented at the beginning of the study. 4. Documenting implementation GoB has authority over the planning and implementation of the C-IMCI interventions. Development of the interventions has already taken place however continued revisions, planning and monitoring is expected to continue through the implementation process. The implementation process as conducted by the District and Upazilla as well as National Core Committee will be documented and monitored. The operations, planning activities and decision making processes will be observed and documented through meeting records, and interviews with the District and Upazilla as well as National Core Committee staff. Of key importance will be the documentation of decision making processes to observe how and for what reason actionable decisions are made. 5. Documenting community perceptions Prior to implementation of the intervention, documentation of attitudes of the community at baseline will be conducted. This will address the community's current attitudes towards the health system and health workers currently available, care-seeking patterns, and areas of demand (for example what services would they like to get from health workers). In addition to this, documentation of different community leaders and health care providers (including teachers, opinion leaders, religious leaders, village doctors, government health workers etc) will be held to determine their current attitudes to the health system, their perceived barriers to accessing health needs and areas of demand. The same activities will be conducted at the end of the intervention. This will particularly focus on documenting changes in the community's health care-seeking behaviour, changes in knowledge and attitudes of the community leaders and heath care providers, social participation of the community leaders and religious leaders in health care, mother's interactions with outreach workers and community leaders and the perceived benefits and disadvantages of the component approaches to improving child health. These documentations will take place periodically throughout the study. Documentation for each process will be in the form of reports providing a brief overview, rationale, objectives and description of the process. Each report will record the process and lessons learned by identifying the key phases of each process, the factors which affect these phases, how these factors relate to each phase and to each other and confounding or external factors. Methods for documentation Qualitative: The study will have an anthropologist to carry out the qualitative documentation. The following methods will be used to document the intervention implementation 1. Documenting situation analysis 1\. In-depth semi-structured interviews will be carried out with Upazilla health managers to ascertain which method and sources were used to establish what type and how many health providers are available in the upazilla, who in the community were opinion leaders and the number and mosques and imams as well as the confounding factors that may have effected the inclusion or exclusion of these human resources/communication mediums: 7 (one from each intervention upazilla) 2. Documenting the development of partnerships with NGOs 1. Focus group discussion will be conducted to understand community's knowledge, attitudes and expectations towards the partner NGOs: 14 (2 in each of the 6 7 intervention upazillas, one with women and one with men) 2. In-depth semi-structured interviews will be carried out with focal persons of both GoB and partner NGOs who will be in charge of enrolling the partnership to understand the modes of communication and other associating factors while working together: the number of respondents for this component will be determined in the field since the number of NGOs involved in the intervention areas is unknown. 2 GoB focal persons from each intervention upazilla will be selected for interview (14 persons total). 3. Documenting training 1. Focus group discussions will be carried out with the participants of the training to understand their perceptions and attitudes regarding the training and the intervention: the number of focus group discussions for this component will be based on the number of training sessions conducted. We will be doing one group discussion for each training session. 2. In-depth semi-structured interviews will be carried out with trainers to understand their perceptions regarding the training module, time allocated for the training and participant's attitudes: the number of in-depth interviews for this component will be based on the number of training sessions conducted. We will be doing one trainer interview from each training session. 4. Documenting implementation 1. In-depth semi-structured interviews will be conducted with the District and Upazilla as well as National Core Committee of DGHS members to ascertain their attitudes and understanding of decision making processes surrounding the implementation of the C-IMCI intervention. 2. Regular attendance at the District and Upazilla as well as National Core Committee meetings to observe decision making processes, structure of meetings, and the main issues discussed will be conducted. The observer will in particular be documenting the processes for planning and decision making by taking into account how issues are raised and managed. The number of observation attendances at the meetings will be based on the number of meetings held. 5. Documenting community perceptions At baseline: 1. In-depth semi-structured interviews will be carried out with community members (both married women and men) to understand their current attitudes towards the health system and health workers, care-seeking patterns for child health and their demand: 42 respondents (6 from each intervention upazilla and among them three interviews will be with married women and three will be with married men having children/grandchildren age under five years of age.) 2. In-depth semi-structured interviews with community leaders (teachers, opinion leaders, religious leaders) will be carried out to understand their current attitudes towards the health system and health workers, care-seeking patterns for child health and their demand: 21 respondents (one from each category in each intervention upazilla). 3. In-depth semi-structured interviews with health workers (village doctors, traditional healers, HA, FWA, and other Govt. and NGO health workers who are currently available in the community) will be carried out to understand their current role, their attitudes towards the health system and their perceptions: 42 respondents (one from each category in each upazilla). At end-line The same number and type of interviews will be conducted at the end of the intervention. However, this piece of work will particularly focus on changes in the community's health care-seeking behaviour, changes in knowledge and attitudes of the community leaders and health care providers. In addition to baseline and end-line, data collection may also take place with these types of respondents throughout the study to understand the ongoing changes or if there is any changes in the community over time. Quantitative: MIS data and other questionnaires will be used to document the intervention quantitatively. Evaluation and monitoring of the performance of the community based health workers To assess the adequacy of the implementation of the proposed intervention it is necessary to regularly document and monitor the performance of the community based health workers. Both utilization of the community based health workers and quality of care will be measured. Utilization of Community Based Health Workers Quantitative data will be collected from routine MIS, e.g., patient registers, referral slips, reports, drug utilization registers as well as reports from referral centres. Quality of Care of provided by Health Workers Existing generic MCE -IMCI health facility survey tools will be modified, translated and used to assess the quality of care at first level facility and at community. Information will be collected through observation of case management using a standard checklist, exit interviews with care givers, re-examination of each child with "gold standard" surveyor, interviews with health care providers and an audit of the supplies and equipment available at the facilities and with the community health workers. Quality of care and facility preparedness for antenatal checkups at GoB first level health facilities will be assessed using a standard checklist. Quality will be assessed at baseline and end line of the study. Evaluation of Effectiveness Measuring effectiveness of C-IMCI intervention implementation is the primary objective of this project. The principle outcome indicators are under-five child mortality, appropriate care seeking for childhood illness, exclusive breast feeding and nutritional status (weight for age). The secondary outcome indicators include number of antenatal care received by women during pregnancy; deliveries conducted by trained birth attendants; mothers and newborns who received postnatal care from a trained provider; newborn received drying and wrapping and bath delayed by at least three days ; newborn breastfed within 1 hour of birth, and infants 0-5 months exclusively breastfeed . These indicators will be measured on a population of children aged 0-5 years and women who have given birth in last one year and compared between C-IMCI intervention areas and comparison areas. Primary comparisons will be made on the basis of key indicators measured over the last two years of the project. However, baseline rates will be estimated to identify any differences between the two arms at project commencement and to make before and after comparisons within intervention arm. Special evaluations will be designed during the course of the intervention implementation to understand and assess C-IMCI intervention sub-components based on both MIS and independently collected data. Household surveys of selected households in each upazila will provide information on indicators such as under-five child mortality, exclusive breast feeding, newborn breastfed within 1 hour of birth, infants 0-5 months exclusively breastfed, antenatal care received by women during pregnancy; deliveries conducted by trained birth attendants, newborn dried and wrapped immediately after bath and delayed bathing, child nutritional status (weight, height/length), under-5 morbidity, care-seeking practices associated with the episodes of morbidity, practices relating to home care and case management, and compliance with health care advice. For cost-effectiveness analyses of the facility and community IMCI interventions activities the study will also track costs and make estimates of the cost-effectiveness of different IMCI intervention models. Methods of Data Collection Household Surveys Three large household surveys will be carried out in the selected study population, first at commencement, at midterm and then at the end of the project. For each household coverage survey a fresh sample of household will be selected from each upazila. The sample HHs will be drawn through a multi-stage sampling process. In each upazila, first villages will be randomly selected by probability proportional to size (PPS) sampling and in the second stage a fixed number of households will be selected randomly from the household lists of the selected villages to reach the desired number of households. Each village will be mapped and all households of the selected villages will be listed. The household lists will contain bari name and household head name of every existing household. Data collectors will identify the sampled households in each village (cluster) and will complete interviews of caretakers of all available under-5 children in the households, including mothers of newborns born in the previous 12 months. The baseline survey will be completed within first 6 months of the project and using a Demographic Health Survey-type instrument. The instrument used by the IMCI-MCE study in Matlab, Bangladesh and Bangladesh Demographic and Health Survey 2007 (BDHS) will be adapted, and used for this purpose. Information on socio-economic and demographic variables, practices and care during pregnancy, delivery and after delivery, immediate newborn care, care seeking practices for under-five children with the episodes of morbidity, practices relating to home care and case management, contact with service providers, compliance with health care advice, and user costs will be collected. Morbidity data will be based on 2-weeks recall of diarrhea, ARI and other serious illnesses. Causes of under-5 deaths, water and sanitation conditions and other possible confounders or correlates of mortality will be collected through this baseline survey so that any differences between the intervention and comparison arms at baseline can be adjusted for in the analysis. The midterm survey will provide insight on the direction and achievements of the project objectives and will be used for any modification of intervention. The end-of-project (EoP) survey (in year 4) will be completed with similar tools that used at baseline. The EoP survey will be completed in last 6 months period. A mortality survey module will be included in the End of Project survey. The data will be used to estimate baseline mortality levels. Cost-effectiveness analyses of the facility and community IMCI interventions activities: For the purpose of cost-effectiveness analyses of the facility and community IMCI intervention the study will also track costs and make estimates of the cost-effectiveness of different IMCI interventions (Facility and Community IMCI). The analyses of both costs and effects will generate 2 cost-effectiveness ratios in terms of costs per YLS (Years of Life Saved). 1. Total costs of IMCI/YLS from treating all under-five based on facility and community IMCI. 2. Incremental costs of introducing community IMCI (aggregate approach)/difference in YLS between intervention and comparison districts. This will show if the additional costs adding community IMCI into the current system justified the additional health benefits. Cost data will be collected in three facility IMCI upazilas and three facility and community IMCI upazilas from societal perspective. Programme intervention costs (in case of facility and community IMCI), which can be directly compared to changes in outcome measures in cost-effectiveness analysis, will be collected by the research team and will include payments to health workers; training courses for health workers; drugs, supplies, and all other activities in the course of the intervention; and amortized capital costs (equipment, vehicles, buildings). We will track and include capital costs, using depreciation techniques to calculate the portion of the investment cost that is appropriate to include in the cost-effectiveness calculation. Users' costs (individuals and families) will be measured through the household surveys. These costs include any direct financial payments for services, as well as transportation costs and the amount of time that individuals spend to use the services of the intervention. Time costs will be translated into financial costs based on the economic level of the household, as measured by the household survey. In addition to efforts to capture programme and users' costs we will capture Provider costs associated with project activities. Data collection will focus on the amassing of all costs associated with the provision of IMCI services at the health facility and community levels. Major direct cost categories will include costs associated with staff time treating under 5 children drugs and all other community activities. While major indirect cost categories will include building costs (e.g. percent of capital and recurrent costs, including electricity and maintenance costs); a proportion of non-medical staff salary costs (e.g. cleaning staff, security guards) etc. Survey management and Quality control The study team will recruit data collectors and their local supervisors at upazilla level. Training of the Trainors (ToT) will be arranged at Dhaka and training of the data collectors will be arranged at upazilla level. All training, data collection process, progress and quality will be closely monitored by the central team at ICDDR,B. Data quality and Data management All questionnaires and data forms will be reviewed for accuracy, consistency and completeness. Repeat interviews will be conducted in a 5% sample of the households. The Project Coordinator and investigators will make periodic field visits to observe data collection by the interviewers. All data will be entered in databases using on-line custom-designed data entry programmes with necessary range and consistency checks built in. Data will be periodically checked by running and reviewing frequency distributions and cross tabulations. Ethical Considerations No children in the study area will be deprived of existing health care services or other services scheduled to be implemented in the area during the study period. Children of the intervention area are expected to be benefited from the interventions to promote family and community practices, e.g., through the intensive education of the families and communities on improved child caring and care seeking. If the study hypothesis is proven to be correct, community IMCI could be scaled up and all the children in the study area will be benefited. Sample Size Calculation and Outcome Variable(s) Sample Size Calculation Since the upazila is the unit of randomization, sample size calculations have been carried out to determine the number of households needed for detecting expected declines in under-5 mortality/1,000 live births, assuming 7 upazilas in each group, and allowing for cluster level randomization. The formula estimates the number of clusters need for given cluster size (n). Since we know the number of clusters in each group is fixed at 6 7, we can also use the formula to estimate cluster size (n) for a given indicator. Here k is the coefficient of variation of the true (population) proportions between clusters within each group. The estimation was done iteratively based on the t-distribution, confidence levels of 95%, and 80% power, and assuming refusal rate of 20% and an expected 20% reduction in under-5 mortality. Thus we get 6 clusters required for each of the arms (intervention and comparison). 9,375households are needed for each upazila to measure 20% under-5 mortality reduction. The sample HHs will be drawn through a multi-stage sampling process. In the first stage 60 villages will be selected with probability proportional to size from each upazila and in the second stage fixed number of households will be randomly selected from those 60 villages to reach the 9,375 households. If a selected village does not include enough households then we will add households from the nearest village. A sub-sample of 930 households including 110 households with women who have given birth in last one year from each upazila will be required for the baseline, midterm and final household coverage surveys. At first stage 30 villages will be selected from each upazila by PPS sampling and in the second stage 31 households will be randomly selected from the household list. If a selected village does not include enough households then we will add households from the nearest village. The same procedure will be repeated for each survey to draw the fresh sample of 930 households. Data collectors will identify the sampled households and will interview caretakers of all available under-5 children in the household, including mothers of newborns who given birth in the previous 12 months