Cuando el cambio importa: La identificacion de los determinantes escolares de la mejora de calificaciones en Mexico, un enfoque de valor agregado dentro de cohorts.
| Autor | Vivanco, Edgar Franco |
| Cargo | Clinical report |
Introduction
Poor educational achievement in Mexico is a perennial topic on the public policy agenda, with particular emphasis being paid over the past decade. This country consistently scores at the bottom of academic achievement among the OECD countries. In the 2009 PISA evaluation, 0.7 per cent of Mexican students scored in the highest level of mathematic skills; 0.4 per cent for language skills; and 0.2 per cent for science (OECD 2010, p. 35). These results rank Mexico at a lower level than poorer economies. There are many interpretations of this poor performance. For example, political economy perspectives remark the perverse incentives product of the decentralization process and the political muscle of the teachers union (Alvarez et al., 2007; Ornelas, 2000). Other studies focus on the impact of particular policies or external shocks on schooling (Binder, 1999; McKenzie, 2003). Contrary to those analyses, the present research uses an educational production function approach to more accurately identify school level characteristics that determine student achievement, especially those related to teachers. Specifically, this paper addresses the question: What are the school characteristics that affect student performance change in Mexican schools? Conclusions from existing research in the same vein are limited because it is constrained to a single point in time (Alvarez et al., 2007; Backhoff et al., 2007; Brodziak, 2009; Valenti et al., 2009; De Hoyos et al., 2012; De la Vega, 2010) or to regional data (Santibanez, 2006; Luschei, 2012; Rubio and Farias, 2013). The aim of this paper is to fill this gap in the literature developing a score gains value-added model using nationwide data to control for previous achievement and reduce selection bias.
In recent years, Mexico has taken important steps in the modernization of its educational system with the implementation of national reforms focused on educational quality, particularly the Alianza por la Calidad de la Educacion in 2008, and Pacto por Mexico in 2012; the latter led to the educational reform in 2013. The country has also been recognized by the OECD as a successful reformer after the increases presented in the PISA 2009 tests (OECD, 2010). Whereas Mexico is certainly in a better position compared to other Latin American countries regarding the quality and availability of information of the educational system, there are still important limitations. These restrictions come from the absence of long-term reliable administrative data. In Mexico, the national standardized test Evaluacion Nacional del Logro Academico en Centros Escolares (Enlace) first began in 2006 for basic education (primaria and secundaria) and in 2008 for high school education. The information with student identifiers is not publicly available, so it is not possible to track students through time or identify their individual characteristics. There is also limited information for teachers, as there is no reliable register at the national level and it is not possible to link teachers with specific students or classrooms. School level information is also inconsistent and difficult to verify. (1) For these reasons, measuring school or teacher effects is a difficult task.
Value-added measures are a family of models built to disentangle those effects. Value-added models (VAM) recognize that educational achievement is affected by factors outside the school's control related to students' background, characteristics, funding and student mobility, among others (Harris, 2011). VAM take into account that in order to build a causal link between teacher/school characteristics and student performance, it is crucial to understand the elements that determine change in student achievement across time. Additionally, VAM try to control for the effect of selection bias due to the non-random assignment of students to schools (Tiebout, 1956). This paper uses a simple definition of value, added as a model that is meant to approximate the contribution of the school on student performance (Braun et al., 2010). These models have been implemented in educational research with very different approaches (McCaffrey et al., 2003). However, their core assumption is that they should control for background factors and initial achievement. Although more sophisticated specifications of value-added models are interested in the difference between the predicted improvement and the actual improvement of the students, due to the characteristics of the Mexican data this research uses a score growth approach which could also be understood as a "quasi-value model", and is based on measuring the academic progress of a group of students controlling for several factors.
In order to overcome the shortcomings related to data availability, this research puts together Enlace scores from 2007 to 2010 for different cohorts. Since aggregated data at school level might be affected by confounding variables, one of the objectives of linking Enlace scores for several years is to track the same cohort from the middle point of their primary education in 3rd grade through their progression in primary school; analogously, it tracks progression of the cohort from the beginning of their secondary school in 7th grade. In addition, I complement the data with a questionnaire taken by school principals when their students take the Enlace test. These questionnaires provide a wide range of information about school conditions, teaching practices and school environment. Another source of information is the census data generated at electoral precinct level in a joint project between the Instituto Federal Electoral (IFE) and the Instituto Nacional de Estadistica y Geografia (INEGI). This information allows identifying several socio-demographic characteristics at a very local geographical level which might be influencing both student performance and school variables (IFE-INEGI, 2010).
Using a fixed effects model and an HLM model, I find that schools with larger intra-cohort gains have a stronger system for monitoring teachers' content domain and student performance. Schools with larger intra-cohort gains also have teachers that are more likely to arrive on time to their classes. After controlling for several measures of student and neighborhood socioeconomic level, the impact of basic infrastructure of the school is still high, which implies that lacking adequate facilities is an important factor explaining achievement gaps (Woessmann, 2003). Although results presented in this paper do not claim causality in determining schools with large performance growths, they can be useful for pointing out the elements that are in the scope of authorities to promote long-term gains in student attainment. This research builds on the large literature studying school effects that emanates from the so called Coleman Report (Coleman et al., 1966) and tries to disentangle the question of how important schools really are in increasing student achievement.
-
Determinants of Change in Student Achievement: A Literature Review
Academic achievement is understood as a "cumulative function of current and prior family, community and school experiences" (Rivkin et al., 2005, p. 422). Based on this principle, recent literature interested in measuring the impact of those elements on academic achievement highlights the importance of understanding the rate of learning over time using VAM, of which the primary objective is to reduce confounding and unobserved influences. Researchers and policymakers are attracted to those models because in statistical terms they are able to separate the effects of teachers and schools from non-educational factors like family background and student characteristics.
Value-added models are used for two primary purposes: first, to hold teachers and schools accountable and to reward or punish them based on their performance; and second, to identify differences across teachers and schools in order to improve education (McCaffrey et al., 2003). While there are several models that try to estimate those effects, they vary in their approach and the type of data used, and therefore differ in their results. For example, Sanders and Rivers (1996) use the Tennessee Value-Added Assessment System (TVAAS) data to study the cumulative effect of teachers in a single student cohort from grades 2nd to 5th, as well as the differential effects of teachers on students of different race and varying levels of achievement. Authors estimate teacher effects and effectiveness with a mixed-effects model with current-year score as dependent variable and prior year score as independent, plus a random teacher effect. They find that there are consequences of having ineffective teachers: students with three consecutive ineffective teachers score 52 to 54 percentile points behind students taught by more effective teachers. There are, however, some reasons to think that these results are biased since Sanders and Rivers do not consider the mixing of students into groups and other omitted student characteristics that might be related to teacher effects. Wright, Horn and Sanders (1997) also use the TVAAS to model gains in student test scores with a mixed model as a function of teacher and a set of student and classroom-level covariates. Then, they use standardized contributions of each variable to compare with teacher effects through a meta-analysis. They find strong and persistent teacher effects, and conclude that teacher effects are dominant to determine achievement gains. However, their results could be biased because their standardized measures are not a robust indicator of contribution to total variance in scores (McCaffrey et al., 2003, pp. 20-23).
Rowan et al. (2002) also support the importance of teachers using residuals for classroom level variance with data from the Prospects study. (2) Authors use data from two cohorts to specify a different set of models to test how much...
Para continuar leyendo
Solicita tu pruebaCOPYRIGHT GALE, Cengage Learning. All rights reserved.
