This article has been retrieved   times since April 24, 2001

   other vols.   |   abstracts   |   editors   |   board   |   submit   |   comment   |   subscribe   |   search


Education Policy Analysis Archives

Volume 9 Number 14

April 24, 2001

ISSN 1068-2341

A peer-reviewed scholarly journal
Editor: Gene V Glass, College of Education
Arizona State University

Permission is hereby granted to copy any article
if EPAA is credited and copies are not sold.

Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education.

Conceptualizing the Process of Education Reform
From An International Perspective

Benjamin Levin
Manitoba Education and Training
Winnipeg, Manitoba, Canada

A great deal of comparative work on education reform is now being done, but this work often lacks a clearly articulated conceptual frame. This paper, based on a study of change in five jurisdictions in four countries, develops a model of reform based on four interactive elements - origins, adoption, implementation, and outcomes. Within each of these elements, questions and concepts from the relevant literature are developed with the intent of building a more comprehensive approach to the analysis of reform from political, organizational and educational perspectives.

        The past twenty years has seen an enormous amount of change in education policy in the industrialized English-speaking countries. In the United States reform has been a constant since at least the publication of A Nation At Risk in 1983, though as Mazzoni (1993) points out, there was also a great deal of state level reform before 1983. Other countries, too, have made significant changes in many aspects of education policy. England and New Zealand had particularly dramatic periods of change during the 1980s and early 1990s. Australian states and Canadian provinces, like U. S. states, have seen significant policy shifts, though, again like the U. S., it is the rare reform that is actually taken up universally.
        All of this reform has been the subject of an enormous amount of discussion and analysis (Fowler, 1995). Some of the most controversial reforms, such as increased assessment, decentralized governance and parental choice of school have generated vast amounts of literature. Many case studies of particular reforms have been done, whether at the local, state or national level.
        Scholarly work comparing education reform across countries has also developed rapidly in the last few years (e.g. Beare & Boyd, 1993; Carter & O'Neil, 1995; Glatter, Woods & Bagley, 1997; Whitty, 1997; Whitty, Power & Halpin, 1998). It seems clear that such work is likely to grow as states or countries are increasingly interested in policy directions elsewhere (Levin, 1998) and as scholars consider the implications of internationalization.
This comparative work is interesting and important, but much of it lacks a clearly articulated conceptual frame and is therefore relatively a-theoretical. Studies typically do not explain why they have chosen to look at particular features of change, or how these features might be related to each other or to larger frames of reference drawn from the relevant literature. Yet comparative work is particularly challenging conceptually (e.g. Kohn, 1987; Archer, 1989) because of the myriad variables and perspectives that could
be used to explore the issues. This paper, which is part of a larger comparative project, develops a conceptual framework for studying education reform across political jurisdictions with the intent of being useful to scholars working in these areas.
        The most complete available effort to develop an overall frame of analysis of education reform is, in my view, the work of Stephen Ball (1990) on reform in England. Ball introduces a variety of frames in which to think about education reform. He points out the important differences in thinking about reform from economic, political or ideological perspectives, and describes the different emphases resulting from forms of analysis that focus more or less on structural, interactional or discursive elements. Ball also recognizes the dangers that analysis can lead to an excessive determinism. Many accounts, he suggests
fail to capture the messy realities of influence, pressure, dogma, expediency, conflict, compromise, intransigence, resistance, error, opposition and pragmatism in the policy process. It is easy to be simple, neat and superficial and to gloss over these awkward realities. It is difficult to retain messiness and complexity and still be penetrating. (p. 9)
        In another work (Bowe, Ball & Gold, 1992), the authors develop a tripartite frame for thinking about education reform. They describe education policy as having three phases that are at least partly autonomous from each other ¨C influence, text production and practice. Influence is the process of bringing policy into being, having to do with who shapes the nature of policy. Text production deals with the creation of policy as a product. Bowe, Ball and Gold point out that what gets produced as policy often deviates from at least some of the intentions of its promoters. Third, they note that education practice, while it is influenced by policy texts, also has a degree of autonomy from them. They characterize these three elements as forming a "trajectory of policy."
        Ball's work is important and has rightly been frequently cited (though a large body of work on education reform continues to appear without being linked to any explicit framework). However since it appeared there have been relatively few efforts to extend or deepen the framework. In this paper I attempt to develop an approach to thinking about and studying education reform that builds on but also (I hope) provides a fuller basis for both analytical and descriptive work on education policy.

The Research Base

        The analysis in this article grows out of a research project that looks at education reform in five different settings¡ªthe Canadian provinces of Alberta and Manitoba, the state of Minnesota, and the countries of England and New Zealand. (To be precise, England is neither a country nor a province or state, but a part of Britain. However, because education in England and Wales differs from that in Scotland and Northern Ireland in important respects, my work has been limited to the former.) In each setting I looked at a major reform process from its inception to outcomes several years later.
        The study has used a variety of data sources, including official documents (position papers, records of legislative debates, newspaper articles), secondary analysis of data from other studies, and interviews with key participants in several of the settings. The study will be fully described in a forthcoming book (reference deleted for author anonymity).

Stage Theory as a Basic Approach

        Every study of policy uses some version of a stage theory. There are many of these (a good overview can be found in Howlett and Ramesh, 1995), all of which involve some series of stages moving from the identification of a problem through the identification or adoption of particular strategies to issues of implementation and impact. The tripartite analysis of Bowe, Ball and Gold is one example. To an extent, the choice of stages is a matter of personal preference. It is vital to remember that although the division into component parts is useful for analytical purposes, in reality reform is not neatly divided in this way, nor can any set of headings adequately represent the complexities of a reform process. In political analysis, discrete categories and periods are devices of the analyst, not the experience of those directly involved.
        The four components I use here are:
  1. Origins. Where did particular reform proposals come from? How did they become part of the government agenda, when so many ideas do not? What role did various actors and interests play in the development of reform programs?
  2. Adoption. How do policies as finally adopted or made into law differ from those originally proposed? What factors lead to changes between proposals and approval? Who supported and proposed various policies, and to what effect?
  3. Implementation. A considerable body of research, in education and other policy fields, lays out clearly the difficulties of moving from policy to practice. What model of implementation, if any, did governments use to move their reforms into practice. What "policy levers" were used to support reforms? How did schools and school systems respond to reforms?
  4. Outcomes. Interest here is on the available evidence as to the effects of reforms. Any political action may have a number of results, some of which were intended by policy-makers and others which were not. Because the reforms under study are about education, the study gives particular attention to what may be known about how the reforms have affected student outcomes and learning processes in schools.
        This delineation draws more attention to the political process of reform, since it highlights the degree to which policy ideas are subject to modification not only after they are promulgated, but at every step of the process. The framework also gives explicit attention to the process of implementation, which some other analyses tend to underplay.

Developing the Framework

        Creating four major stages or elements is only the first step towards an adequate conceptual framework for a comparative policy study. It is also necessary to provide much greater specification of the main considerations within each of the four stages, so that general terms such as "adoption" or "implementation" are fully developed conceptually. Each of the elements must also be grounded in the relevant literatures. However before taking up the four main themes in this way, attention has to be given to two general themes: the overarching importance of context in comparative policy analysis, and the balance between linearity and contingency in thinking about political processes.

A first consideration: The importance of a historical and cultural perspective

        Education reform is political work, and political work can only be understood appropriately in a historical and cultural perspective. Reforms necessarily arise in particular social, economic, political and institutional contexts. The way any reform program is conceptualized, developed, defended (and attacked), and implemented will owe a great deal to previous events and practices in a given jurisdiction.
        A few examples can illustrate. The legacy of social class distinctions in Britain, including a long history of elitism in education, conditions the way all new reform proposals are seen (Whitty & Edwards, 1998). The debate over such proposals as opting out (grant-maintained schools) or parental choice occurs in a system that for most of its history has provided educational opportunity based largely on class background. In the United States, on the other hand, concerns about race relations have been at least as powerful as class issues in shaping debates about the same kinds of issues. In Canada, with a long history of separate but co-existing institutions based on language and religion, concerns about choice are much more muted. ¡°Separate but equal¡± means something quite different in Canada than it has in the U. S. The politics of language, religion and ethnicity can have powerful effects on education and are strongly conditioned by unique historical factors in each setting.
        Institutions of government and political cultures are also powerful influences on education reform with quite variable effects. Federal states will approach reform differently than unitary states. American political institutions, with their constitutional checks and balances, provide very different political dynamics than do parliamentary majority governments. Jurisdictions, whether countries, states, or provinces, can have very different political cultures and practices depending on whether they have a history of strong or weak executives, a single house or two, a two-party system or a history of multiple parties and coalitions. For example, a two party state with a long history of polarized politics will have quite different political processes than one with multiple parties and a history of seeking central ground.
        Even those developments that are seen as having international scope need to be viewed in a historical perspective. The current zeal for reform in education, for example, follows a long period of expansion of educational systems accompanied by a high level of confidence and substantial resourcing. Economic cycles have a strong impact on people's thinking about education; when economic times are good, there tends to be more optimism about education (Krahn, 1996). Increased attention to consultation in policy-making (whether real or symbolic) and to the role of parents is very likely connected to an increasingly educated populace and growing skepticism in many quarters about the degree of discretion that should be afforded to professionals. One does not have to be a Hegelian to see the influence of dialectic in these developments.
        All of this means that the analyst should, in studying reform, pay careful attention to the way that new developments emerge from past events and practices. This is a particular problem in comparative work, in that the more jurisdictions one considers, the harder it is to develop a reasonable degree of familiarity with the requisite background.

The intended and the contingent: Conceptualising reform as a whole.

        The common view of reform tends to assume that political or ideological analysis leads to a reform program that in turn leads to changes in practice leading to particular outcomes. Politics is treated as largely an intellectual process embodying a relatively straightforward means-ends rationality. One body of work embodying these assumptions operates at a high level of abstraction, concerned with such matters as the changing role of the state and the impact of globalisation as determining forces in political events (e.g. Ball, 1998; Carter & O'Neil, 1995; Taylor et al., 1997). Education reform in these treatments is often discussed as the implementation of a set of well-defined political views having to do with beliefs in the reduced role of the state or the primacy of markets over public provision. The same line of thinking can be found in many of the analyses and critiques of such recent policies as school choice (e.g. Lauder et al.; 1999).
Reform is certainly driven in some important ways by a linear, means-ends rationality. If there were no sense that an action would produce particular consequences there would be no reason to undertake the action. In politics, careful calculation of consequences is of absolutely central importance, although the consequences that are of interest include personal, partisan and symbolic outcomes as well as substantive policy consequences (Edelman, 1988).
        At the same time, it is important not to overstate the degree to which reform is driven by a straightforward rationality. Usually one finds a high level of ambiguity and contingency in every aspect of the political process. At every step, multiple and conflicting influences come to bear, purposes change or are worn down by existing structures and processes, and circumstances change in ways that require modification of plans and actions. As Ball puts it:
National policy making is inevitably a process of bricolage: a matter of borrowing and copying bits and pieces of ideas from elsewhere, drawing upon and amending locally tried and tested approaches, cannibalising theories, research, trends and fashions and not infrequently flailing around for anything at all that looks at though it might work. Most policies are ramshackle, compromise, hit and miss affairs, that are reworked, tinkered with, nuanced and inflected through complex process of influence, text production, dissemination and, ultimately, re-creation in contexts of practice. (1998, p.126)
        The literatures on organization change and on government policy- making support a model that goes beyond a simple means-ends rationality to include attention to the inevitable ambiguity and contingency of the political process. Work in political science (such as Edelman, 1988, and Stone, 1988) remind us that that politics is shaped by many considerations other than policy, including the requirements of staying in office and the vicissitudes of the moment. Symbolic and emotional issues are often at the forefront of political work. Work in politics also emphasizes the degree to which the entire process of policy development and implementation takes place in a short-term context that is constantly changing, multi- faceted, and very difficult to read but is also affected by long-term trends in government and society. Dror (1986) describes this environment as ¡°fuzzy gambling¡±, in which the rules change while the game is played and ¡°surprise dominates¡±. Dror is among writers (see also Lindblom, 1990) who also point out that even in the best of circumstances human abilities to understand problems and generate appropriate solutions are limited and often inadequate. Most of the time governments are operating under circumstances that are far from optimal in this regard due to pressures of time, lack of information, and multiple competing issues.
        Work under the heading of neo-institutionalism (March & Olsen, 1989; Wilson, 1989) shows the powerful effect that institutional structures have on the political process, constraining the available political choices and also shaping the way in which political decisions are put into effect, not necessarily consistent with the intentions of their originators. Institutions possess considerable ability to resist changes or to reduce their impact significantly. Research on policy effects (e.g. Elmore, 1995) also suggests that in many cases strategies for reform may focus on elements that cannot produce the kinds of changes that are really wanted or, to put it another way, that reforms focus on what can be done instead of on what might really make a difference.
        The task of the analyst, then, is to consider the ways in which policies are driven by a particular logic or ideology, but also the ways in which they are shaped by other factors¡ªhistorical, cultural, institutional, and political¡ªthat are far less predictable. At the macro level, long-term changes in societies and the role of the state are important. At the micro level, chance, in the form of individual personalities or unexpected events, is also an important consideration in understanding reform. Neither the importance of means-ends rationality nor the underlying contingency of life can be ignored¡ªboth must be accommodated in an adequate theoretical account.
        One way of thinking about this balance is to think of the logic of policy¡ªactions leading to particular consequences¡ªas the numerator in an equation. However the power of this numerator is affected by a denominator that contains all the contingent elements. Sometimes the numerator is larger and policy is driven by careful strategy. Other times the denominator is larger and policy is primarily the result of unforeseeable elements.

Considering the Four Stages

         With those points in mind, we turn to a fuller discussion of the four elements of the proposed framework: origins, adoption, implementation and outcomes.

The origins of reform.

        The work of Kingdon (1994) provides a valuable approach to looking at the origins of policy. Kingdon believes that political decisions emerge from the interaction of three streams: political events, problem recognition, and policy proposals. The balance of importance between political events, problem recognition and policy proposals, and the way the three streams interact will vary from setting to setting.
        Within each of these streams, additional work has been done in this study to define key elements. Kingdon's framework also needs to be adjusted to take into account the differences between parliamentary systems and the U S governance model of separation of powers.
         Political events are often under-emphasized in analyses of education policy because they are so unpredictable. The outcome of political processes, however, can be affected strongly by such factors as stages in the political cycle, the internal dynamics of the governing party, the personalities of important actors, the nature of the relationships among key players, and unanticipated events or crises.
        Problem recognition involves a variety of influences bearing on political decision makers and on each other. One important set of influences is found within government itself, including both political and bureaucratic elements. Within-government political sources of problem recognition can include individuals in key roles (such as ministers, governors, or members of a legislature), political parties and their associated bodies, legislatures, and central agencies of government (such as finance units or central policy units). Sometimes policy issues emerge not from the particular policy area but as a result of broader government agendas, such as a focus on reducing expenditure. Problem recognition also emerges from the apparatus of government as various agencies try to convince politicians of the importance of particular issues or problems, or expand their own influence and budget, or manage external pressures.
        External influences on the definition of problems are wide ranging. They can arise through various consultative processes as well as formal and informal lobbying efforts by many different interests. An area like education, which involves a great deal of money and affects a large number of people, is the subject of a great deal of lobbying from many different directions. As Lindblom pointed out many years ago (1980), because of its economic importance business plays a particularly significant role in shaping problem agendas. Research, polling and media reports are also a potential source of problem definition, though typically mediated through some kind of lobbying process.
        Policy proposals are often, but not always, connected to problem definition, so many of the same sources are important in developing or promoting policy proposals. Policy proposals have eventually to get the approval or support of political leaders. However the actual ideas can come from many different places. In fact, Kingdon suggests that policy proposals have so many different versions and sources that the search for their origins is a vain task (1994, p. 71).
        Wherever they originate, policy proposals may be promoted by individual politicians or civil servants, political parties, government agencies, lobby groups, think tanks, policy entrepreneurs, or from research of various kinds.
        The process of policy borrowing¡ªof taking ideas from one jurisdiction and applying them to another¡ªis also relevant to the discussion of both problem definition and policy proposals. Much of the comparative work cited earlier argues that current education reform proposals have moved from one country to another. However careful examination is needed to determine whether referrals to other country's policies are anything more than rhetorical. Similar policy labels¡ªsuch as "choice"¡ªmay hide very great differences in policy content. This is so across states in the U. S. (see Mintrom & Vergari, 1997) and provinces in Canada (Levin, 1998), and even more so across national boundaries (Halpin & Troyna, 1995).
        The entire process of shaping policy also occurs within a more general context of ideas and policy preferences¡ªwhat Schon (1971) called "ideas in good currency." These ideas provide a taken-for-granted backdrop to policy discussions, helping to determine the range of ideas that will even be considered, let alone adopted. Changes in dominant ideas about the role of the state, for example, have been important in recent years in many English-speaking countries (Manzer, 1994). Ideas are themselves affected by and affect changing economic and social conditions. There is a constant interaction between changes in material conditions, the way that people name or explain these, and the proposals that may be advanced to address perceived problems.
        These more fundamental ideas about policy are often labelled as "ideological." Some argue that all policy is ideological by definition, but if so than the idea loses its analytic value. Others use the term "ideological" as rather meaning the absence of common sense, and apply it to those policies with which they do not agree (e.g. Lawton, 1994). Again, it is not clear how such terminology is very helpful in any analytic sense.
         Manzer (1994) distinguishes between ideology as a public justification or rationale for reform, and ideology as an actual constituting element of reform, whether so stated publicly or not. Any combination of these possibilities could exist - that is, reform programs that are justified and constituted ideologically, programs that are justified ideologically but are in practice more pragmatic, programs that are justified in pragmatic terms but are actually ideologically constituted, and programs that are neither justified nor constituted ideologically. It could also be argued that a policy could have effects that are ideological regardless of either justification or its constitution. There is a danger here of circular argument, in which every policy can be found to be ideological on at least some criterion (for a fuller discussion see Levin, in press).
        All of this suggests that discussion of the role of ideology in education reform needs to be well grounded empirically, and that analysts need to be clear about what they mean when they make claims about ideology and policy.


         For purposes of this analysis, adoption is the process of moving from a policy proposal to an approved piece of legislation, regulation, or policy. The literature in this area in regard to education reform is sparse even though policies often change in important ways from inception to final adoption.
In the adoption process several elements collide. What began as a slogan or a concept¡ªschool choice, local management, open enrolment, provincial testing, charter schools¡ªmust be turned into a detailed scheme in the form of legislation, regulations, or policy guidelines so that it can actually be put into place in a large and complex system. Both administrative and political issues can result. Many important policy initiatives begin as ideas that are not fully developed, so turning them into something workable may involve quite a bit of debate as to what the intentions originally were and how they can best be realised. The debates can be political, in that opponents of a particular reform in and out of government may revisit their concerns as the details are worked out. At other times the issues will be administrative as the system tries to work out detailed procedures for managing large-scale changes. Although all of these processes may be intertwined and often occur simultaneously, they can usefully be considered under the headings of internal political debate, bureaucratic accommodation, and public political debate.
         Internal political debate refers to discussion within a government, among contending political factions. These debates can be motivated by substantive disagreements about policies, by arguments over the politics of action (such as whether the timing is right for a particular idea), or just as easily by rivalries and animosities between individuals or organizational units. The development of detailed plans or the requirement to approve budgets or legislative proposals can reopen internal disagreements about how strongly or how far a policy should be pursued, or which part of the bureaucracy should manage it. These debates may continue long after a policy is officially announced, and even after its implementation.
         It is not always easy to know which people are going to be key to a policy's fate. Heads of government are vital, but they are not necessarily the only essential actors. Ministers of finance and their deputies can be especially important if a new policy has financial implications ¨C as they almost always do. Heads of other units may be important if they see a policy as affecting their own programs or plans or if their co-operation is needed to move a proposal forward. In the U. S. system, individual legislators of any party can play a vital role in the adoption process, regardless of the position of the executive.
         The political process of adoption often leads to policy proposals that are vague or even contradictory. Politicians are often amateurs in the substantive policy field who may not understand the complexities of existing organization, the impact of other competing agendas, or the difficulties that inevitably arise in the attempt to move from a general idea to a specific set of procedures. Civil servants, on the other hand, as Wilson (1989) points out, are concerned to make the system work as smoothly as possible. They may have no personal commitment to a government's purposes, but they do have to think about the procedures in detail, what could go wrong, who will administer or manage the policy, how exceptions will be handled, what timelines are possible, what resources will be needed¡ªall things that may not be part of an attractive political vision. As a policy proposal moves towards implementation¡ªor, as Fitz and Halpin (1991) describe it in their study of Grant Maintained School policy in England, ¡°from a sketchy policy to a workable scheme¡±¡ªthere will inevitably be a process of limiting and narrowing, of trying to rub off the sharp corners of policy that will create the most difficulty, of trying to make new policies at least partly consistent with existing practices.
         Much of this discussion takes place in a public political arena in which conflicts over both intent and implementation are debated. The latter may include the "official" debate in parliament or a legislature as well as the debate that goes on in public, through various consultation processes, the media, and with various interest groups. Through the entire process, all sorts of proponents and opponents of reform are trying to advance their position and counter opposing arguments, so reforms are frequently accompanied by intense political disagreement. In the oppositional world of politics, even groups that rather like an idea may take a critical stance in public.
        Public political debate involves an effort to "frame" thinking (Davies, 1999), or to shape discourses (Ball, 1990) about educational issues. The parties to the debate have various devices available to them in their efforts. Governments can use such means as press releases, white papers, speaking tours, legislative committees and hearings, advisory groups and public consultation processes in efforts to mould public opinion. Opposition groups have their own set of devices, such as public rallies and the media's interest in conflict and controversy, to get their views across. Public distrust of government is itself a weapon that can be used by opponents. In Canada and the United States opponents have also sometimes used the courts to attack elements of various reform programs. A number of other devices can be used by any of the parties, such as issuing official statements and press releases, advertising, polling, research, and the citing of authorities. All sides may appeal to the views of supposedly neutral or objective third parties; research is often used in this way in political debate.
        The process of adoption is also influenced by the degree of commitment to a given program. Some reforms may be deeply important to governments, in which case a high degree of conflict may be tolerated before there is any willingness to make changes. Sometimes a government actively seeks conflict on an issue as a way of convincing its supporters of its commitment. In other cases a sponsor may only be willing to expend a small amount of political capital before retreating from a position. These decisions are themselves influenced by the kinds of political events discussed earlier, such as the timing in relation to an election, leadership rivalries, or other competing issues.
         The study of adoption is important because it shows the ways in which intentions are modified by political and administrative considerations. Since reforms often end up looking rather different than was originally intended, an analysis of the sources of these changes should be an important part of an overall analysis of reform.


         In contrast to adoption, the literature on the problems of implementation is very large and quite well developed (e.g. Fullan, 1991; McLaughlin, 1987). A whole series of obstacles¡ªsome of them generic to policy implementation and others particular to schools¡ªstand in the way of policies being put in place as intended. Although the problems of implementation are well known, governments have tended to give relatively short shrift to these issues in the policy process.
        Perspectives on implementation have become more complex over time. While the greatest amount of attention has been given to ways of making implementation more effective, another body of work (e.g. Hargreaves, 1994) takes the view that disputes over change are best thought of as political struggles, and that resistance may be well-justified. More recently there has been increasing interest in ideas about organizational learning, and the need to see change as a process of testing and refining ideas on the basis of evidence about their impact and value.
        Factors affecting implementation can be thought of (Fullan, 1991) as pertaining to the change itself, to the setting where implementation is to occur, and to the wider context. The first heading would include the clarity of the change and the degree of difficulty involved in implementation. Political direction for change is often either vague or contradictory because of the need to reconcile divergent interest. Further complexity and lack of clarity occur because education does not have a generally accepted core technology, or way of doing things, but depends greatly on the values and approaches of individual teachers or administrators. Thus almost all policy is subject to extensive interpretation.
        The second category includes the degree of understanding of the proposed change, the level of commitment to it by relevant actors, and the various resources allocated to support change. Commitment to change is shaped not only by the skills of those involved, but also by their attitude to a given reform. Attitudes in turn are shaped by educators' views of the practicality of a proposal, and also by its fit with the existing culture of schools. These two elements also reinforce each other, such that culture shapes attitudes to practicality and practicalities of teaching also shape school culture.
        The third category includes the other pressures either supporting or inhibiting implementation, such as competing demands and community support or opposition. Not only are education reforms themselves often multi-faceted and sometimes inconsistent, but they take place in a context which is itself changing. Too many reforms happening too quickly may lead to increasing cynicism and resistance in schools. The availability or lack of resources also affects the willingness of schools to adopt particular reforms.
        The concept of "policy levers" or "policy instruments" (Howlett, 1991; McDonnell & Elmore, 1987) is an important part of this analysis. Governments have a number of means they might use to promote implementation. The most important of these levers are mandates (such as legislation or regulation), inducements (money, recognition), capacity building (training, professional development, research), system changing (reorganization, governance changes), and opinion mobilization (exhortation, public pressure). However in few cases do governments appear to develop comprehensive strategies to support implementation of their policies.
         Implementation cannot be taken for granted. Many reforms end up leaving few lasting marks on the system they were designed to change. Consideration of the ways in which implementation is structured and supported is thus an important part of the overall analysis of reform.


         Reform programs are always justified on the basis of the outcomes they will yield. Most reforms, however, are justified on the basis of a number of outcomes, and these are sometimes quite different from one another. Vigorous debate already exists around the outcomes of such changes as more student assessment or increased parental choice of schools. As is always the case in education, the discussion is made more difficult because the purposes of schools are multiple, sometimes mutually contradictory, and often very difficult to assess. Moreover, reforms may yield outcomes quite different from or in addition to those intended by advocates or feared by opponents.
        A framework for considering outcomes needs to include impacts on students, impacts on schools, and impacts on the broader society. The most frequently cited reasons for education reform have to do with impact on students, with the most common outcome measure being some form of assessment of students' skill or knowledge in the various curriculum areas. However a variety of outcome measures beyond academic achievement have also been used to assess the impact of education policies. These include graduation rates, attendance rates, numbers of disciplinary problems, or rates of referral to special education.
        Students' assessments of the quality and value of their schools experience are an important, if seldom evaluated outcome indicator, if only because they say something about motivation, which is absolutely critical to all other outcomes. Life-chance indicators are also very important, since the most important purposes of schools often have to do with what happens to students after they leave the institution. These could include such outcomes as post- secondary education rates, employment outcomes, interest in lifelong learning, income, and citizenship indicators (such as propensity to volunteer, voting behavior, or criminality).
        A surprisingly large proportion of the research on reform focuses on the impacts on schools rather than students. One of the most frequently assessed aspects of reform is its impact on teachers' work and their attitudes towards their work. Outcomes related to work might include hours, time in and out of the classroom, attention to individual students, professional development activities, skill levels or teaching practices. Indicators in regard to attitude include teachers' sense of effort, satisfaction and stress, among others. Reform is also held to have had different effects on administrators than on teachers, partly because governance changes have altered the work of administrators in important ways. However the relevant measures for administrators are generally similar to those used for teachers.
        Greater involvement of parents has been a goal of most reform programs. Parents' active role in school governance and in their children's education as well as their sense of satisfaction with the school and their part in it have been measured.
        Some reforms are intended to affect school programs. Curriculum changes or graduation requirements are obvious examples. School choice is also often argued on the basis that it will lead schools to diversify and improve programs. Many reform programs have not given very much attention directly to teaching and learning practices per se, with the possible exception of efforts to extend the use of educational technologies. However improved teaching and learning practices are clearly central to the achievement of all school outcomes and so should be a key part of assessing any reform. Although changes in school organization such as devolution of authority, are usually argued as means to achieve other more important ends, they could also be considered as outcomes in themselves.
        The larger social impact of reform has also been an important subject of debate. Economic, equity and social cohesion outcomes are all potentially important. Insofar as reform has frequently been justified on economic grounds¡ªthat is, on the contribution of schooling to national economic success¡ªsocietal economic outcomes would be important indicators of the success of reforms. Such outcomes could include labor force participation rates, employment rates, earnings, and productivity growth, not only for students but more generally.
        Many critics and some proponents of reform have been concerned about the potential of reform to increase inequity in society. An important outcome measure is thus the extent to which reforms act either to reduce or to increase the gaps in outcomes in society that are due to socioeconomic status, ethnicity, gender, or other demographic factors. A related issue is the degree to which reforms serve to build or reduce an overall sense of community among people. Efforts to assess social cohesion have included measures of ethnic segregation, citizen participation, or of attitudes such as tolerance.
        Many of these outcomes are clearly very difficult to assess. The assessment is made more difficult because policies are not the only factors that produce outcomes ¨C in fact, the evidence in education is that the most important shapers of many outcomes lie outside the school system. Reforms themselves may have a variety of unintended consequences, and because many reform programs are multifaceted, interactive effects are common such that developments in one area have strong effects on other elements.
        All of this suggests that analysts need to pay careful attention to their choice of outcomes and need to justify these in relation to the particular reforms under study.


        The elements of reform sketched here are, as mentioned earlier, highly interactive rather than discrete. Considerations of implementation and outcomes may shape original intentions, just as perceptions of outcomes may modify policies and their implementation. Each of the four elements is important to understanding reform, but complex in its own right.
        Doing such analysis on a comparative basis is even more difficult. The study of education reform is complicated even in a single setting, so trying to make comparisons across settings is fraught with additional difficulties. One inevitably risks drawing comparisons without full knowledge of local circumstances, and seeing as similar what would, with closer analysis, look quite different. In these conditions researchers need to be especially careful to be clear about their presuppositions and the frameworks within which they are assembling data and deriving findings. My hope is that this paper will contribute to that process.


This article is drawn from a larger study of international education reform that has just been published as a book¡ªReforming Education From Origins to Outcomes (London/New York: Routledge-Falmer, 2001). The research was supported financially by the Social Sciences and Humanities Research Council of Canada. Particular thanks are due to Jonathan Young. Many other colleagues around the world also contributed to the study. An earlier version of this work was presented at the American Educational Research Association, New Orleans, April 2000. All views and opinions are solely those of the author.


Archer, M. (1989). Cross-national research and the analysis of educational systems. In M. Kohn (Ed.), Cross-national research in sociology (pp. 242-262). Newbury Park: SAGE.

Ball, S. (1990). Politics and policy making in education. London: Routledge

Ball, S. (1998) Big policies/small world: an introduction to international perspectives in education policy. Comparative Education, 34(2), 119-129

Beare, H. & Boyd, W.L. (eds.). (1993). Restructuring Schools. London: Falmer Press.

Bowe, R., Ball, S. & Gold, A. (1992). Reforming education and changing schools. London: Routledge.

Carter, D. & O'Neill, M. (eds.) (1995). International perspectives on educational reform and policy implementation. London: Falmer.

Davies, S. (1999). From moral duty to cultural rights: A case study of political framing in education. Sociology of Education, 72(1), 1-21.

Dror, Y. (1986). Policymaking under adversity. New York: Transaction.

Edelman, M. (1988). Constructing the political spectacle. Chicago: University of Chicago Press.

Elmore, R. (1995). Structural reform in educational practice. Educational Researcher 24(9), 23-26.

Fitz, J. & Halpin, D. (1991). From a "sketchy policy" to a "workable scheme": the DES and grant-maintained schools. International Studies in Sociology of Education, 1(2), 129-151.

Fowler, F. (1995). The international arena: The global village. In J. Scribner and D. Layton (eds.), The study of educational politics (pp. 89-102). Washington: Falmer Press.

Fullan, M. (1991). The new meaning of educational change. New York: Teachers College Press/ OISE Press.

Glatter, R., Woods, P. & Bagley, C. (eds.) (1997). Choice and diversity in schooling. London: Routledge.

Halpin, D. & Troyna, B. (1995) The politics of education policy borrowing. Comparative Education. 31(3), 303-310.

Hargreaves, A. (1994). Changing teachers, changing times. New York: Teachers College Press.

Howlett, M. (1991). Policy instruments, policy styles, and policy implementation: National approaches to theories of instrument choice. Policy Studies Journal 19(2), 1-21.

Howlett , M. & Ramesh,. M. (1995). Studying Public Policy: Policy Cycles and
Policy Subsystems
. Toronto: Oxford University Press.

Kingdon, J. (1994). Agendas, alternatives and public policies. 2nd edn. New York: HarperCollins.

Kohn, M. (1987). Cross-national research as an analytic strategy. American Sociological Review 52(4), 713-731.

Krahn, H. (1996). School-work transitions: Changing patterns and research needs. Consultation paper for the Applied Research Branch of Human Resources Development

Lawton, D. (1994). The Tory mind on education. London: Falmer.

Lauder, H., et al. (1999). Trading in futures: Why markets in education don't work. Buckingham: Open University Press.

Levin, B. (1998). An epidemic of education policy: (what) can we learn from each other? Comparative Education 34(2), 131-141.

Lindblom, C. (1980). The policy-making process.2nd ed. Englewood Cliffs, NJ: Prentice-Hall.

Lindblom, C. (1990). Inquiry and change. New Haven: Yale University Press.

Manzer, R. (1994). Public schools and political ideas. Toronto: University of Toronto Press.

March, J. & Olsen, J. (1989). Rediscovering Institutions. New York: The Free Press.

Mazzoni, T. (1993). The changing politics of state education policy making: A 20-year Minnesota perspective. Educational Evaluation and Policy Analysis 15(4), 357-379

McDonnell, L., & Elmore, R. (1987). Getting the job done: Alternative policy instruments. Educational Evaluation and Policy Analysis, 9 (2), 133-152.

McLaughlin, M. (1987). Learning from experience: Lessons from policy implementation. Educational Evaluation and Policy Analysis, 9(2), 171-178.

Mintrom, M. & Vergari, S. (1997) Political factors shaping charter school laws. Paper presented to the American Educational Research Association, Chicago, April.

Schon, D. (1971). Beyond the stable state. New York: Norton.

Stone. D. (1988). Policy paradox and political reason. New York: Harper-Collins.

Taylor, S., Rizvi, F., Lingard, R. & Henry, M. (1997). Educational policy and the politics of change. London: Falmer.

Whitty, G., Power, S. & Halpin, D. (1998). Devolution and choice in education: The school, the state and the market. Buckingham: Open University Press.

Whitty, G. (1997). Creating quasi-markets in education: A review of recent research on parental choice and school autonomy in three countries. In M. Apple, (Ed), Review of Research in Education, 22 (pp 3-48). Washington: AERA.

Whitty, G. & Edwards, T. (1998). School choice policies in England and the United States: an exploration of their origins and significance. Comparative Education, 34(2), 211-227

Wilson, J. (1989). Bureaucracy. New York: Basic.

About the Author

Benjamin Levin, Ph.D.
Deputy Minister
Manitoba Education and Training
162 Legislative Building
Winnipeg, Manitoba, R3C 0V8

Benjamin Levin is Deputy Minister of Advanced Education and Deputy Minister of Education, Training and Youth for the Province of Manitoba, in which capacity he is the chief civil servant responsible for these areas. He is filling this position on secondment from The University of Manitoba where he is professor of educational administration with particular interests in education policy, politics and economics. This is his third article for EPAA.

Copyright 2001 by the Education Policy Analysis Archives

The World Wide Web address for the Education Policy Analysis Archives is

General questions about appropriateness of topics or particular articles may be addressed to the Editor, Gene V Glass, or reach him at College of Education, Arizona State University, Tempe, AZ 85287-0211. (602-965-9644). The Commentary Editor is Casey D. Cobb:

EPAA Editorial Board

Michael W. Apple
University of Wisconsin
Greg Camilli
Rutgers University
John Covaleskie
Northern Michigan University
Alan Davis
University of Colorado, Denver
Sherman Dorn
University of South Florida
Mark E. Fetler
California Commission on Teacher Credentialing
Richard Garlikov
Thomas F. Green
Syracuse University
Alison I. Griffith
York University
Arlen Gullickson
Western Michigan University
Ernest R. House
University of Colorado
Aimee Howley
Ohio University
Craig B. Howley
Appalachia Educational Laboratory
William Hunter
University of Calgary
Daniel Kall¨®s
Umeå University
Benjamin Levin
University of Manitoba
Thomas Mauhs-Pugh
Green Mountain College
Dewayne Matthews
Western Interstate Commission for Higher Education
William McInerney
Purdue University
Mary McKeown-Moak
MGT of America (Austin, TX)
Les McLean
University of Toronto
Susan Bobbitt Nolen
University of Washington
Anne L. Pemberton
Hugh G. Petrie
SUNY Buffalo
Richard C. Richardson
New York University
Anthony G. Rud Jr.
Purdue University
Dennis Sayers
Ann Leavenworth Center
for Accelerated Learning
Jay D. Scribner
University of Texas at Austin
Michael Scriven
Robert E. Stake
University of Illinois¡ªUC
Robert Stonehill
U.S. Department of Education
David D. Williams
Brigham Young University

EPAA Spanish Language Editorial Board

Associate Editor for Spanish Language
Roberto Rodríguez Gómez
Universidad Nacional Autónoma de México

Adrián Acosta (México)
Universidad de Guadalajara
J. Félix Angulo Rasco (Spain)
Universidad de Cádiz
Teresa Bracho (México)
Centro de Investigación y Docencia Económica-CIDE
Alejandro Canales (México)
Universidad Nacional Autónoma de México
Ursula Casanova (U.S.A.)
Arizona State University
Jos¨¦ Contreras Domingo
Universitat de Barcelona
Erwin Epstein (U.S.A.)
Loyola University of Chicago
Josu? González (U.S.A.)
Arizona State University
Rollin Kent (México)
Departamento de Investigación Educativa-DIE/CINVESTAV
María Beatriz Luce (Brazil)
Universidad Federal de Rio Grande do Sul-UFRGS
Javier Mendoza Rojas (México)
Universidad Nacional Autónoma de México
Marcela Mollis (Argentina)
Universidad de Buenos Aires
Humberto Muñoz García (México)
Universidad Nacional Autónoma de México
Angel Ignacio Pérez Gómez (Spain)
Universidad de Málaga
Daniel Schugurensky (Argentina-Canad?
OISE/UT, Canada
Simon Schwartzman (Brazil)
Fundação Instituto Brasileiro e Geografia e Estatística
Jurjo Torres Santom? (Spain)
Universidad de A Coruña
Carlos Alberto Torres (U.S.A.)
University of California, Los Angeles

   other vols.   |   abstracts   |   editors   |   board   |   submit   |   comment   |   subscribe   |   search