Professor Adely discusses how flawed quality measures often shape our understanding of education in the Middle East and what a closer look at these measures can tell us.
By Fida Adely
A common theme in regional reports about education in the Arab world is that, while the region has made remarkable strides in extending public education to its citizens in a relatively short period of time, education overall faces serious quality challenges. Typically, the empirical basis for such conclusions are limited. Absent data about what actually transpires in schools and what students are learning, youth unemployment is often held up as the best evidence of the poor quality of education—the assumption being that unemployment is a “skills mismatch” problem. In other words, if only the right skills were taught in schools there would be jobs for the taking.
What are we told about education in the Middle East?
The Road Not Traveled
In the 2008 World Bank flagship report, “The Road Not Travelled: Education Reform in the Middle East and North Africa,” the authors acknowledge educational achievements in the Middle East, but go on to argue that the region “has not capitalized fully on past investments in education, let alone developed education systems capable of meeting new challenges.” While the authors admit that “measuring the quality of education is illusive,” they attempt to do so using literacy rates, fields of study in higher education, and scores on international tests as indicators, each of which are problematic in their own ways.
Literacy rates are a poor proxy for quality since the prevalence of adult illiteracy among older adults typically reflects the later start many countries in the region had to mass-based education, and even the huge strides made by some countries are not a reflection of quality per se but rather of increased access to education.
Using fields of study in higher education—specifically the percentage of students studying science or engineering—to approximate quality is even more problematic. The authors defend this measure by arguing that “scientists and engineers are likely to contribute more to economic growth than are social scientists and students of humanities,” a problematic assertion in its own right. Also, they do not assess the quality of education in these fields, but rather focus only on the number of students enrolled.
The third quality measure used by the report is the performance of nine Middle Eastern countries on “Trends in International Math and Science Study” (TIMSS), an international comparative assessment developed by the International Association for the Evaluation of Educational Achievement. International standardized exams such as TIMSS have been critiqued for their “translatability” across countries, their use of global averages as benchmarks, and for reducing learning to a set of measurable math, science, and literacy skills.
Based on these measures, the authors conclude: “An important gap exists between what education systems currently produce and what the region needs to achieve its development objectives.” They go on to explain that this is because the school systems in the region tend to focus too much on “engineering education,” which they liken to a business production mentality. Thus, simplistically, it takes a classroom, a teacher, a textbook, and the like to educate a student. The quantity, quality, and mix of these inputs determine educational outcomes. When outcomes are not satisfactory, the engineering perspective suggests increasing the quantity of inputs, improving their quality, or changing their mix by means of more resources and better management. The authors argue that school systems in the region need to move from the engineering mentality on to the stuff of more advanced educational systems—namely incentives and public accountability, ironically the same types of policies that have been subject to lengthy debate in the United States.
Given the great diversity of educational and economic contexts in the region—including rural/urban differences and challenges such as major influxes of refugees—inputs such as safe and secure school buildings, well-trained teachers, or quality curriculum are clearly still fundamental to providing education and for improving its quality throughout the region.
Arab World Learning Barometer
The 2014 Brookings “Arab World Learning Barometer” provides a more recent example of an education-focused regional report. The
Barometer takes widely available data such as enrollment rates, “survival rates” in schools, youth unemployment rates, and the results from international standardized tests, to describe the state of education in the Arab World. While this report adds no new data about the quality of education in the region, it does employ new language to talk about quality, or the lack of it.
Using the results for 13 countries on three different international assessments (TIMSS, PIRLS, and PISA), the authors make the bold claim that 56% of children at the primary level and 48% of students at the lower secondary level are “not learning” anything. This type of discourse, which is clearly meant to be provocative, is deeply problematic and yet is emblematic of reports like this. They make broad claims about a region with 22 very different countries on the basis of limited empirical data—data, in this case, that is drawn only from the 13 countries that have participated in one or more of these three international assessments. Also, the global benchmarks that the Arab countries are being measured against are averages based on the performances of countries who have taken these tests before, countries with often vastly different educational systems and histories.
What does it do to claim that over half of the children in the region learn nothing?
Most immediately, it paints a picture of twenty-two Arab countries as “failing to develop” or as “backward.” By extension, teachers are failures too, as are parents and students. Such conclusions also produce a very limited view of what learning entails and erases the many types of learning that happen in and outside of school—learning that cannot be measured by a test conceptualized in Amsterdam and designed in Boston.
The Brookings report also points to the fact that girls consistently outperform boys yet almost immediately dismisses the significance of this reverse gender bias because women are less likely to be employed. We are told to disregard this female advantage from the outset, and yet quality education and valuable learning cannot be reduced to labor force participation rates, even though much of the policy discourse of recent decades tries to do so. Education and learning serve many other purposes in human life, such as defining the communities we want to live in, the values and habits we want to reproduce, and sometimes the ones we want to change.
The Brookings report acknowledges that rural/urban discrepancies are significant (urban students have significant advantages when it comes to enrollment, longevity in school, and performance on such standardized assessments). What the report fails to do is discuss why this may be the case or make the links to rural poverty that need to be made. Socio-economic status, irrespective of geography, is a significant issue that is hardly mentioned in the report. Finally, the report concludes with a discussion of unemployment—a factor that is not empirically linked to the discussion of quality education but is assumed to be the logical outcome of the rates of students “not learning.”
What is the purpose of such regional sweeps based upon thin, if not questionable, measures?
Clearly the agencies that produce these reports are interested in shining the spotlight on particular issues. In this sense, the reports serve as advocacy pieces or rallying cries. Whether in fact they are ever effective in changing priorities is less clear, but the discursive effects of such reports can be quite negative—contributing to an essentialization, if not denigration, of the Arab world.
Such reports typically de-historicize and draw on comparisons that seem almost meaningless when we consider the very short trajectories of formal public education in countries such Oman, Qatar, or the UAE. None of the Arab Gulf countries except for Saudi Arabia even had a ministry of education before 1970. Even Jordan, which by the weak measures available in these reports seems to do relatively well, had only a handful of public schools before 1950.
Even more problematic, they mask the resource issues at the core of educational quality issues. Throughout the region educators and what they can accomplish are shaped by low teacher salaries and increasing precariousness of teaching jobs. If we have learned anything from international and comparative education, it’s that teaching matters—and not just in the training of teachers, or in teaching methodologies, but also the status of teachers as laborers and professionals.
These reports also tend to elide or simplify the costs of war, weaponry and militarization. For example, male tertiary education rates in Palestine have decreased by nearly 5% in the last ten years, and male secondary enrollment rates by 7% (UNESCO; Brookings). While educational infrastructure has suffered from Israeli military attacks, especially in the recent wars in Gaza, something much more systematic is happening throughout the territories due to prolonged occupation (50 years) and the more than ten-year blockade of the Gaza Strip.
Major setbacks in education have also been documented in Iraq, which suffered years of sanctions, the U.S. invasion and the on-going violence that the invasion unleashed. In the 1970s and 80s, Iraq had among the strongest educational indicators (in terms of access) in the region with secondary net enrollment of 45% in 1982. While many countries in the region have continued to increase secondary enrollments over the last three decades, Iraq has yet to reach the levels of secondary enrollment it had in 1982. By 2007, secondary enrollments were only 44.7 %, a significant improvement from a low of 30% in 1999. While regional reports don’t ignore the reality of conflict, it is often treated as a somewhat unconnected issue—too political to reckon with.
What do we know about education in the Arab world beyond enrollments and graduation rates?
We know that socio-economic status (SES) is the biggest predictor of educational survival and “success,” as it is just about everywhere in the world (with SES often dovetailing with geography). Indeed SES and/or family background explains almost all of the within-country variability on test outcomes from international assessments. But beyond this basic structural reality that shapes the opportunity of children in DC as much as it does in Riyadh, or Amman, or Casablanca, what do we know? In recent years, scholars of education working in the region have made important progress toward emphasizing inequality and understanding the structures and processes that lead to different opportunities for students in the region. They have drawn our attention to the importance of school-based factors, to community solidarity in overcoming adversity, and to the effects of labor market opportunities in fields like tourism or the military for male drop-out rates. Much more work needs to be done in this vein.
What do we mean by “quality” when discussing education?
This discussion still begs the question of what we mean by “quality.” Robin Alexander argues in Essays on Pedagogy that what is measureable has come to denote quality education. For Alexander and others, quality education is about process, and process is fundamentally about pedagogy—or the practice of teaching. But rather than placing pedagogy at the center of our inquiries about quality education, Alexander argues “pedagogy has been made to fit the available measures rather than the other way around.” These measures have now become global in the form of international standardized tests. “What happens to be within the bounds of statistical computation comes to define the very nature of teaching itself.” This is clearly evinced in the overwhelming reliance on international assessments to say something about quality education in the Arab world.
Given the critical challenges we face today in supporting the learning and intellectual and emotional lives of millions of children and youth affected by war, there is also a desperate need to broaden our frameworks—our vision of learning and quality—so that these children too do not get written off as failures.
Dr. Fida Adely is Director of the Master of Arts in Arab Studies program and Associate Professor at CCAS.
The “Al Mutanabbi Street Starts Here” project was founded by Beau Beausoleil in response to the 2007 car bombing of Baghdad’s historic Al-Mutanabbi Street, the heart of the city’s intellectual community and home to numerous booksellers.