Categories of Evaluation Design: Analysis of Experimental Design

In this term paper, I will discuss the evaluation design which is broadly classified into three categories, experimental, non-experimental as well as quasi-experimental design. Hence I will discuss how those designs would be useful in finding out the impact of different intervention like in TASAF intervention in Kagera Region, Zanzibar Universal Pension Scheme (ZUPS), Five Years Development Plan II (2016-2021), Agriculture Support Development Plan II as well as Southern Agriculture Growth Corridor of Tanzania (SAGCOT), hence those design may vary in a feasibility, cost of the entire intervention, the degree of clarity, validity of the result as well as the degree of selection bias.

According to the International Bank for Reconstruction and Development/ The World Bank defines Impact Evaluation as a particular type of evaluation that seeks to answer cause and effect question. To evaluate the cause and effect question the impact evaluation examine outcome, hence asses what difference the intervention makes in outcomes (Gertler. et al, 2011) The Impact evaluation account for the changes that appear to an individual or a group of people as an outcome of a defined intervention within a specific context. To conduct impact evaluation has to observe the objective of a certain program, it is necessary as many of the programs focus only on outcomes and that makes it difficult on the process of measuring the impact of that program.

The quasi-experimental design is one of the categories of evaluation design, it assigns the participant to either a treatment group which includes the group of people who affected by the intervention, hence a control group also include a group of people who did not affect by the intervention (Hinton, 2015).

Non Experimental design is the categories of evaluation design whereby also known as observation designs because it encompasses a wide range of valid empirical methods. The main aim is to explore the causal relationship of an intervention on a group (Hinton, 2015). Non- experimental design accomplices the simple difference and pre and post-treatment comparison. Simple difference comparing the group that received the program with that did not. Those who did not receive the program represent a valid counterfactual of what would have happened to those who receive the program. Pre and post-treatment comparison, compare the data of a group before receiving the intervention and later observe the data of the same group after they received the intervention (Pomeranz, D. 2017)

Experimental designs randomly assign subjects to either a treatment or control group to determine the effect of a program. Those in the treatment group receive the intervention, while those in the control group do not. Random allocation to either group helps to ensure validity and increases the probability that effects are due only to the intervention itself, eliminating other variables. Effects are then statistically calculated by comparing the observed outcomes of the two groups (Hinton, 2015). An experimental evaluation may also be known as randomized evaluation, under such condition the researcher has to study the participant before the intervention. Hen in a random way may assign those participants on to either received the intervention or treatment and not receive or comparison group. Then soon after the accomplishment of the intervention the researcher may use the identified sample to make a comparison to evaluate the impact of a certain intervention.

There a framework that is useful concurrently for choosing the best methods applied in impact evaluation based on cost, clarity as well as the validity of the programs to produce the necessary information. The framework must base on firstly is the available resources and constraints which including time, timing, expertise, and existing data, as well as organizational standards and definitions for evaluation. Second is the nature of what is being evaluated, it comprises of important features of the project, program or organization being evaluated, often described by a program logic or theory of change, also considering the stage of the policy or program in its lifecycle, and whether it has aspects that are simple, complicated or complex. The third is the nature of the evaluation, includes its purpose, the key evaluation questions it is intended to answer and the requirements of key stakeholders (Rogers, P. et al 2015).

Tanzania Social Action Fund (TASAF) became effective on November 2000. It is a project of the Government of Tanzania, operating as a Government's efforts in poverty eradication. To achieve maximum efficiency, the Government has established TASAF to make sure that targeted development initiatives in remote areas of the country get requisite support in a timely and cost-effective manner (World Bank, 2000). However, apart from TASAF which is focusing on rural and peri-urban communities, the government also has instituted 'social funds' to cater to the income poor that are unable to raise credit from banks, targeting women, the unemployed youth and those in the informal sector. Also in 2006, the government initiated the Presidential Fund whereby each region received one billion (Tshs) for purpose of enabling the poor people to borrow from banks so that they can initiate business and hence improve their living standard. However, these are limited to urban and peri-urban areas mainly due to limited accessibility to rural and most people in the rural areas are not aware (informed) of most of these funds or the new credit organizations (Rutasitara, 2002b).

TASAF intervention in Kagera Region was implemented through various projects, hence each of which project owned by either an individual or group of people. The projects owned by the group of an individual was Milling project at Ibaraizibu, Tailoring and Carpentry subproject at Kassamby Village (Kamzora, 1999)

In the TASAF intervention in Kagera Region, I decide to use the Quasi-experimental design mainly the sup design of Difference in Difference. DD methods compare baseline and follow-up data for treatment and comparison groups pre- and post-intervention, as was done in the case of academy schools in England (Machin and Vernoit 2011). In this design, I will collect the data before and after the implementation of the program. Before the implementation of the program, the data which I will collect may adhere to both who will receive the intervention (treatment group) and the ones who did not receive the intervention (control group). The things to note is that during the process of collecting data the sample which I will use some of the criteria that directly control the process of choosing the sample and not to consider the randomization assignment to the group.

For instance, the TASAF in the Kagera region among the main focus is to the capacity enhancement of beneficiaries and institutions supporting targeted communities and households. So based on my criteria I will select one district where the TASAF was implemented, hence from that district I will on the ward which is most households are not far from health facilities. Then I have to collect the data before and after the implementation of the program. in the process of collecting data after the implementation of the program, I have to choose the treatment as well as the control group, hence to make the comparison between these groups and come up with the impact of the certain intervention in a particular area.

Zanzibar Universal Pension Scheme (ZUPS) is the first universal cash transfer scheme of its kind in East Africa fully funded by the Government The Revolutionary Government of Zanzibar allocated US$2.909 million in 2017/2018 for the implementation of the scheme Zanzibar residents who qualify for the pension scheme receive a monthly non-contributory pension worth TSh20, 000 (US$8.95)() The main objective of Zanzibar Universal Pension Scheme has to eradicate absolute poverty in a society.

Zanzibar Universal Pension Scheme ZUPS was introducing the age interval to consider the targeted population who will be benefited from the program. According to Helpage international 2017, A total of 251 people aged 70 years and over, from all five regions of Zanzibar were interviewed on issues ranging from living arrangements, income and expenditure, food consumption and nutrition, health and wellbeing hence the findings was respondents were 54 percent male and 46 percent female. 65 percent of respondents were aged between 70 and 79 years old, 29 percent between 80 and 89 years old and 8 percent above 90 years old.

Zanzibar Universal Pension Scheme ZUPS will undertake the impact evaluation through the use of Quasi experimentation design whereby it assigns the participant to either a treatment group (the group which receives the intervention) compare with the control group (the group which does not receive the intervention) (Honton.2015). In criteria of selection bias, I will focus only on the individuals above a given threshold, through the cutoff point of the program was the rely upon under eligibility criteria of a person's 70 years old and above. Now our sample design will be the variable aged 70 and the above, in order to make comparisons on the impact of the program, I will group our variable onto treatment group (those who are aged 70 years old and the above) and the control group (those aged below the 70 years old) that means the older one aged between 60-69 (HelpAge International, 2017). The impact evaluation to Zanzibar Universal Pension Scheme the focus will be on making the causal relationship between the control and treatment group by assessing how the intervention has impacted those who directly affected and the one who did not affect.

Save your time!
We can take care of your essay

• Proper editing and formatting
• Free revision, title page, and bibliography
• Flexible prices and money-back guarantee

Place Order

Zanzibar Universal Pension Scheme flow under the Quasi-Experimental Design within the type of Regression Discontinuity because the program has a specific cutoff point, the cutoff point help in a process of choosing the sample to distinguish the treatment group to comparison group. Regression Discontinuity Design is a methodology that allows making causal conclusions that are nearly as reliable as the randomized control trial. It can only be applied in cases where a program or policy has a specific threshold that determines who is eligible to participate. Regression Discontinuity Design uses the fact that the individuals or entities just barely above the threshold are identical to individuals just below. Under certain assumptions, it is, therefore, possible to measure the treatment effect is the difference between the outcomes of the individuals just below the threshold who are therefore not eligible and the outcomes of those just above who are therefore eligible (Imbens and Lemieux, 2008; Lee and Lemieux, 2010)

The Zanzibar Universal Pension Scheme has its cutoff point whereby it was designed to identifying the members who will be involved in the intervention. For example, they address the targeted participants who will be included in the program there must reach the age of 70 and the above and with the same criteria (HelpAge International, 2017). Then to evaluate the impact of the program my sample will be divided on to two categories which are treatment as well as comparison group, the treatment group may be the one who are elder and reach the age of 70 and the above, then the comparison group will be those who seem are elder but did not reach the age of 70, meaning those who are likely in 65 up to 69 years old. The level of bias in a sample will be high as the researcher may use his or her criteria in choosing the sample, but there some means that a researcher may apply to reduce the bias in a sample. In a degree of clarity, the main focus will be on how people understand or have the knowledge about the program information. To understand that we will non-randomly select the participant regardless of being the treatment or control group, hence chose one of the methods either interview or conducting the questionnaire (the open-ended is more usefully here), from that will get the full picture about how the entire people have understood the program.

Hence by encompassing the wide range also in conducting the impact evaluation design, I will use also the non-experimental design, within which the evaluator does not assign the treatment group or control group as the way to determine the effect of the program in a specific context. But the evaluator may observe the situation that that whole people experience before and after the program being implemented. Generally, to conduct an impact evaluation of the Zanzibar Universal Pension Scheme on my side I have to choose both two kinds of techniques that are used in making an impact evaluation. These techniques are non-experimental design or quasi-experimental design, and this may depend on the approaches under which each kind is elaborated. For instance, if the evaluator observes the program in a wide range and decides to make a causal relationship before and after the intervention he/she has to use pre and post comparison (non-experimental design), but in a circumstance when he decides to choose the Intervention group as well as comparison group direct have to use the quasi-experimental design.

The Second Five Year Development Plan (FYDP II), 2016/17 – 2020/21, has integrated frameworks of the first Five Year Development Plan (FYDP I, 2011/2012-2015/2016) The theme of FYDP II 'Nurturing Industrialization for Economic Transformation and Human Development' incorporates the main focus of the two frameworks, namely growth and transformation (FYDP I) and poverty reduction (MKUKUTA II). FYDP II outlines new interventions to enable Tanzania to industrialize in a way that will transform its economy and its society (Mof, 2014). The Second Five Year Development Plan (FYDP II) introduces four priority areas for action: (i) fostering economic growth and industrialization; (ii) fostering human development and social transformation; (iii) improving the environment for business and enterprise development; and (iv), strengthening implementation effectiveness (Mof, 2014).

In a Five Year Development Plan II, I will use a non-experimental design to evaluate the impact of the program. And I decide to use this simply because the Five Year Development Plan covers the wide range so it will be difficult on the process of selection of samples on both treatment and control group when I will use other design, however it may require a large amount of fund. For instance, the Five Year Development Plan II was aimed at nurturing Tanzania's Industrialization for Economic Transformation and Human Development has a dual focus on growth and transformation and poverty reduction (Mof, 2014). As the aim of the non-experimental design is to observe and make a causal relationship so by using that notion we can argue that the development or increase of many industries in the Tanzania country is due to the result Five Years Development Plan II. Instead of using the control group, in non-experimental design, the impact of the program is measured through pre and post comparison. The impact will be measured as the difference between the outcome of interest before and after the intervention.

In Five Year Development Plan II, before the intervention to take place under the pre and post evaluation I will select the sample in a particular area decided the program to take place, hence later after the program takes place in a particular area I had to come up and observe my sample that I had selected before, then make the comparison to understand the impact of the program. Feasibility analysis of Five Years Development Plan will assess the plan that was addressed before the implementation of the project, under such conditions Five Years Development Plan was feasible because the program proposed the various potential plan of the source of Finance, the listed and key measures designed to strengthen revenue collections from these sources. These span government revenues includes tax, non-tax revenue, domestic borrowing, innovative domestic sources, ODA, other official flows and other external resources, other new sources of finance are foundations/philanthropies, foreign market bonds, national climate fund, debt-to-health swaps, Diaspora bonds, regional economic arrangements, and South-South cooperation, also the use of PPPs and non-governmental resources like debt instruments, venture capital, FDI and other private sector financial flows (Mof, 2014), from the proposed source of income will accelerate the success of the program as it was planned .

In the Five years Development Plan II them to conduct the impact of the program we will select the unit randomly and assigned to the treatment group and the control group. As the targeted area for Five Years Development Plan II is on industrialization, human development and implementation effectiveness in attaining these targets, the Five Years Development Plan II seeks to boost annual real growth in gross domestic product, reduce the poverty rate, increase flows of foreign direct investment (FDI), boost electricity generation and support manufacturing growth and exports. It also aims to reduce infant and maternal mortality rates, enhance access to clean and safe water and raise the national human development index. Then the member will be randomly selected from both rural areas as well as in urban (Mof, 2014). For example, the treatment group will be the ones who are affected by either electricity generation.

Hence on the validity of the result will be like the maternal mortality rates to the women affected or receive the intervention compare to the one who did not receive the intervention within the same context, the comparisons will come up with the result like either there a gap between two groups.

Agricultural Sector Development Strategy is an important guiding tool for the implementation of the sectoral policies for the next ten years (2015/16 – 2024/25). It aims at operationalizing transformation of the agricultural sector into modern, commercial, highly productive, resilient, competitive in the national and international market which leads to achieving food security and poverty reduction, contributing to realization of Tanzania Development Vision 2025 (TDV) that envisages raising the general standard of living of Tanzanians to the level of a typical medium-income developing country by 2025(ASDP II, 2015).

The Agricultural Sector Development Programme - Phase 1 (ASDP 1) was launched in 2006 to lead the transformation of agriculture in Tanzania. The overall goal was to reduce poverty and enhance economic development for the vast majority of Tanzanians who depend on agriculture for their livelihood. At the onset of 2015, the government embarked on the second phase of ASDP. This time, the implementation of the program takes off in the context of climate change. This brief, therefore, looks at ASDP 2 operationalization in the context of climate change and the strategic position of smallholder farmers (ASDP II, 2015).

To conduct an impact evaluation design of Agriculture Sector development design I decide to use the design which will help me to overview the whole program similarly. One of the designs which I decide to use is Non-experimental design whereby through observation I can make the analysis that how the current situations are attributed to the certain kind of intervention as introduced in a particular context. For instance as the aims of introducing the ASDP II was to reduce poverty and enhance economic development so the Improved agricultural productivity in our country was attributed directly with the introduction of Agriculture Sector Development Program(ASDP II, 2015)

However, in the non-experimental design, the participants who decide to participate in the program are more likely to accept the changes that may be attributed to the program apart from those who are forced to participate. Hence after the implementation of ASDP II direct I have to observe the ones who are self-selected to participate in the program to evaluate the impact of the program soon after the implementation taking place. The cost of the program relied upon the financing of agricultural activities in Tanzania which continues to be through a variety of sources with the Government traditionally being the main source of funds, supplemented by Development Partners who have been supporting mostly the development budget of the sector.

The Southern Agricultural Growth Corridor of Tanzania ('SAGCOT') Initiative is a Public-Private Partnership initiated at the World Economic Forum (WEF) Africa (WEFA) Summit in Dar es Salaam in 2010. Its implementation period runs for 20 years up to 2030. Its ultimate objective is to boost agricultural productivity, improve food security, reduce poverty and ensure environmental sustainability through the commercialization of smallholder agriculture (http://sagcot.co.tz/) The SAGCOT is a public-private partnership explicitly designed to achieve higher rates of income growth and job creation through the development of competitive agribusiness value chains across the Southern Corridor. The initiative initially concentrates investments within the rail and road corridor stretching from Dar es Salaam in the east through to Morogoro, Iringa, Mbeya, and west to Sumbawanga (Project Information document, 2011)

In this SAGCOT to evaluate the impact, I decide to use the quasi-experimental design. As the intervention is designed to cover the whole nations but I apply the quasi-experimental design because there someplace which are already the program taking place and there someplace did not, and this appears because the program was first initiated by the concentration on investments within the rail and road corridor stretching from Dar es Salaam in the east through to Morogoro, Iringa, Mbeya, and west to Sumbawanga (Project Information document, 2011). Hence in quasi-experimental design, I decided also to use one of the techniques which are the difference in different methods to compare the changes in outcomes over the time between the population included in the program (the intervention group) as well as the population not included in the program (the comparison group).