Milbank Q. 2004 Dec; 82(4): 581–629. This article summarizes an extensive literature review addressing the question, How can we spread and sustain innovations in health service delivery and organization? It considers both content (defining and measuring the diffusion of innovation in organizations) and process (reviewing the literature
in a systematic and reproducible way). This article discusses (1) a parsimonious and evidence-based model for considering the diffusion of innovations in health service organizations, (2) clear knowledge gaps where further research should be focused, and (3) a robust and transferable methodology for systematically reviewing health service policy and management. Both the model and the method should be tested more widely in a range of contexts. Keywords: Diffusion of
innovation, systematic review, implementation This article summarizes the findings of a systematic literature review of the diffusion of service innovations. The United Kingdom Department of Health explicitly commissioned this work, which was carried out between October 2002 and December 2003, for its National Health Service's extensive modernization agenda
(UK Department of Health 2001). Our review, which supplements and extends previous overviews and meta-analyses (Damanpour 1991, 1992,
1996; Granados et al. 1997; Meyers, Sivakumar, and Nakata 1999; Rogers 1995;
Tornatsky and Klein 1982; Wejnert 2002; Wolfe 1994), focuses primarily, but not exclusively, on research studies of health care. Because of the size and scope of our review, we cannot describe all our
findings or discuss all our sources in this article. Instead, we encourage interested readers to read the complete project report (Greenhalgh et al. 2005a). We defined a systematic review as a review of the literature according to an explicit, rigorous, and transparent methodology. We defined innovation in service delivery and organization as a novel set of
behaviors, routines, and ways of working that are directed at improving health outcomes, administrative efficiency, cost effectiveness, or users’ experience and that are implemented by planned and coordinated actions. We distinguished among diffusion (passive spread), dissemination (active and planned efforts to persuade target groups to adopt an innovation), implementation (active and planned efforts to mainstream an innovation within an organization), and
sustainability (making an innovation routine until it reaches obsolescence). But we did note an ambiguity in the notion of sustainability (i.e., the longer an innovation is sustained, the less likely the organization will be open to additional innovations). The breakdown of sources that contributed to the final report is shown in
Figure 1. Because formal search techniques (e.g., entering index terms or key words in electronic databases) drew a poor yield, we relied mainly on “snowball” methods (pursuing references of references and using citation-tracking software) and sought advice on sources from experts in various fields. Our search strategy was
designed to concentrate on the service sector, particularly health care. Our original inclusion criteria were (1) studies from the health care sector; (2) those that had addressed innovation in service delivery and organization; (3) those that had looked specifically at the diffusion, dissemination, implementation, and/or routinization of these innovations; and (4) those that met our stringent criteria for methodological quality. But as the review unfolded, two things became clear: first, in
many areas, the evidence meeting all these criteria was sparse, and second, we could gain critical insights from beyond the parameters we had set. We therefore extended our criteria to a wider range of literature. In particular, we added both overview articles and “landmark” empirical studies from outside the health sector if they had important methodological or theoretical lessons for our research question. The literature on the sustainability of service innovations was, incidentally, very
sparse, and so we did not include it in this article. Summary of Sources Contributing to the Systematic Review To help explore this large and heterogeneous literature, we developed a new technique, which we called meta-narrative review. It is summarized in Box 1 and explained in detail in a separate paper (Greenhalgh et al. 2005b). A meta-narrative is the unfolding “storyline” of research in a particular scientific tradition (defined as a coherent body of theoretical knowledge and a linked set of primary studies in which successive studies are influenced by the findings of previous studies; see Kuhn 1962). We mapped the meta-narratives (i.e., we traced the historical development of concepts, theory, and methods in each research tradition) by identifying the seminal theoretical and overview papers and books and analyzing the conceptual and theoretical models proposed by recognized experts in each field. BOX 1Phases in Meta-Narrative Review
Source: Greenhalgh et al. 2005b. For each empirical study, we developed a data extraction form (available on request) to summarize the research question, study design, validity and robustness of methods, sample size and power, nature and strength of findings, and validity of conclusions. We modified published critical appraisal checklists to assess the quality of primary studies evaluating service interventions, qualitative research, mixed-methodology case studies, action research, and process. We then divided the primary studies’ findings into six broad categories: (1) the innovation itself; (2) the adoption/assimilation process; (3) communication and influence (diffusion and dissemination, including social networks, opinion leadership, champions, and change agents); (4) the inner (organizational) context, including both antecedents for innovation in general and readiness for particular innovations; (5) the outer (interorganizational) context, including the impact of environmental variables, policy incentives and mandates, and interorganizational norms and networking; and (6) the implementation process. Within each category, we identified subtopics and painted a rich picture of each by grouping together the contributions from different research traditions. Because different researchers in different traditions generally conceptualized their topic differently; used different language and metaphors for diffusion, dissemination, and implementation; asked different questions; privileged different methods; and used different criteria to judge “quality” and “success,” we used narrative, rather than statistical, synthesis techniques (Dixon-Woods et al. 2004). We highlighted the similarities and differences of the findings from different research traditions and considered the reasons for the differences. In this way, the heterogeneity of approaches and “contradictions” in findings could be turned into data and analyzed systematically. Based on the evidence from the primary studies, we developed a unifying conceptual model (Figure 3) and tested the model on four case studies (telemedicine, integrated care pathways, general practitioner fund holding, and the UK's electronic patient record), which are analyzed in detail in the full report (Greenhalgh et al. 2005a). Conceptual Model for Considering the Determinants of Diffusion, Dissemination, and Implementation of Innovations in Health Service Delivery and Organization, Based on a Systematic Review of Empirical Research Studies We graded the overall evidence supporting each of our conclusions using a modified version of the World Health Organization Health Evidence Network (WHO-HEN) criteria (Øvretveit 2003):
Key Research AreasBecause of the very large number of empirical sources identified, we have cited in this article only illustrative studies and/or overviews, and we have not given citations for statements for which we had only limited evidence. For a full list of primary sources, please see our main report (Greenhalgh et al. 2005a). We identified 13 research areas that had, largely independently of one another, provided evidence relevant to the diffusion of innovations in health service organizations (Table 1). Four of these traditions can be classified as “early diffusion research”: TABLE 1Research Traditions Relevant to Diffusion of Innovations in Health Service Organizations
These early studies produced some robust empirical findings (discussed later) on the attributes of innovations, the characteristics and behavior of adopters, and the nature and extent of interpersonal and mass media influence on adoption decisions. But the work had a number of theoretical limitations, notably the erroneous assumptions that (1) the only relevant unit of analysis is the individual innovation and/or the individual adopter; (2) an innovation is necessarily better than what has gone before and adoption is more worthy of study than is nonadoption or rejection; (3) patterns of adoption reflect fixed personality traits; and (4) the findings of diffusion research are invariably transferable to new contexts and settings. Research areas that emerged as developments—and sometimes as breakaways—from such conceptual models include
In the organization and management literature, we found the following areas that were relevant to our review:
One other relevant area in the organization and management literature is organizational psychology, in which innovativeness is seen as dependent on good leadership, sound decision making, and effective human resource management (especially the motivation, training, and support of staff). We did not explore this literature in detail, as it was the subject of several other projects funded by the UK Department of Health Service Delivery and Organization Programme (see http://www.sdo.lshtm.ac.uk/changemanagement.htm). A Model of Diffusion in Service OrganizationsFigure 3 shows the unifying conceptual model that we derived from our synthesis of theoretical and empirical findings. As noted later, the model is intended mainly as a memory aide for considering the different aspects of a complex situation and their many interactions. It should not be viewed as a prescriptive formula. The next section presents the principal empirical findings from across the different research traditions, organized broadly around the model's main components. The InnovationIndividual people adopt different innovations and then spread them at different rates to other individuals. Some innovations are never adopted at all; others are subsequently abandoned. A very extensive evidence base from sociology (including medical sociology) supports the notion of key attributes of innovations (as perceived by prospective adopters), which explain much of the variance in innovations’ adoption rates. In addition to Rogers's authoritative review (1995), the following conclusions are based on a number of more recent empirical studies of service innovations in health care (see Greenhalgh et al. 2005a for the full references): Relative AdvantageInnovations that have a clear, unambiguous advantage in either effectiveness or cost-effectiveness are more easily adopted and implemented (for strong indirect and moderate direct evidence, see Dirksen, Ament, and Go 1996; Marshall 1990; Meyer, Johnson, and Ethington 1997; and Rogers 1995). If potential users see no relative advantage in the innovation, they generally will not consider it further; in other words, relative advantage is a sine qua non for adoption (for strong direct and moderate indirect evidence, see Rogers 1995). Nevertheless, relative advantage alone does not guarantee widespread adoption (for strong direct evidence, see Denis et al. 2002; Fitzgerald et al. 2002; and Grimshaw et al. 2004). Even so-called evidence-based innovations undergo a lengthy period of negotiation among potential adopters, in which their meaning is discussed, contested, and reframed. Such discourse can increase or decrease the innovation's perceived relative advantage (for moderate direct evidence, see Ferlie et al. 2001). ComplexityInnovations that are perceived by key players as simple to use are more easily adopted (for strong direct evidence, see Denis et al. 2002; Grilli and Lomas 1994; Marshall 1990; Meyer and Goes 1988; Meyer, Johnson, and Ethington 1997; and Rogers 1995). Perceived complexity can be reduced by practical experience and demonstration (for moderate direct evidence, see Plsek 2003). If the innovation can be broken down into more manageable parts and adopted incrementally, it will be more easily adopted (for strong indirect and moderate direct evidence, see Plsek 2003; and Rogers 1995). If an innovation in an organizational setting has few response barriers that must be overcome, it will be assimilated more easily (for strong indirect and moderate direct evidence, see Rogers 1995). Interventions to reduce the number and extent of such response barriers improve the chances of successful adoption (limited evidence). ObservabilityIf the benefits of an innovation are visible to intended adopters, it will be adopted more easily (for strong direct evidence, see Denis et al. 2002; Grilli and Lomas 1994; Meyer and Goes 1988; and Øvretveit et al. 2002). Initiatives to make more visible the benefits of an innovation (e.g., through demonstrations) increase the likelihood of their assimilation (limited evidence). ReinventionIf potential adopters can adapt, refine, or otherwise modify the innovation to suit their own needs, it will be adopted more easily (for strong direct evidence, see Meyer, Johnson, and Ethington 1997; and Rogers 1995). Reinvention is especially important to those innovations that arise spontaneously as “good ideas in practice” and spread through informal, decentralized, horizontal social networks (for moderate indirect evidence, see Rogers 1995; see also “fuzzy boundaries” in this article). These “standard” attributes (which, apart from reinvention, are extensively cited) are necessary but not sufficient to explain the adoption and assimilation of complex innovations in organizations. Additional key attributes are as follows (note that for clarity we have conflated some that researchers considered separately): Fuzzy BoundariesComplex innovations in service organizations can be conceptualized as having a “hard core” (the irreducible elements of the innovation itself) and a “soft periphery” (the organizational structures and systems required for the full implementation of the innovation); the adaptiveness of the “soft periphery” is a key attribute of the innovation (for moderate direct evidence, see Denis et al. 2002). The concept of a soft periphery links with Rogers's aforementioned concept of reinvention and with “innovation-system fit” as an important feature of system readiness. RiskIf the innovation carries a high degree of uncertainty of outcome that the individual perceives as personally risky, it is less likely to be adopted (for strong direct evidence, see Meyer and Goes 1988; and Meyer, Johnson, and Ethington 1997). Because an innovation's risks and benefits are not evenly distributed in an organization, the more the balance between risks and benefits reflects the organization's power base, the more likely the innovation is to be assimilated (for moderate direct evidence, see Denis et al. 2002; and Ferlie et al. 2001). Task IssuesIf the innovation is relevant to the performance of the intended user's work and if it improves task performance, it will be adopted more easily (for moderate direct evidence, see Yetton, Sharma, and Southon 1999). Interventions to enhance task relevance improve the chances of successful adoption (limited evidence). If the innovation is feasible, workable, and easy to use, it will be adopted more easily (for strong direct evidence, see Dobbins, Cockerill, and Barnsley 2001; Foy et al. 2002; Meyer and Goes 1988; and Yetton, Sharma, and Southon 1999). Interventions to improve the feasibility and workability of innovations for key staff members and teams improve the chances of successful adoption (limited evidence). Augmentation/SupportIf a technology is supplied as an “augmented product” (e.g., with customization, training, and a help desk), it will be assimilated more easily (for strong moderate direct evidence, see Aubert and Hamel 2001). Our full report gives a number of examples of studies that failed to support the importance of even the most well-established attributes in certain settings (Greenhalgh et al. 2005a). This finding illustrates the important principle that the attributes are neither stable features of the innovation nor sure determinants of their adoption or assimilation. Rather, it is the interaction among the innovation, the intended adopter(s), and a particular context that determines the adoption rate. As Dearing and And commented:
Adoption by IndividualsPeople are not passive recipients of innovations. Rather (and to a greater or lesser extent in different persons), they seek innovations, experiment with them, evaluate them, find (or fail to find) meaning in them, develop feelings (positive or negative) about them, challenge them, worry about them, complain about them, “work around” them, gain experience with them, modify them to fit particular tasks, and try to improve or redesign them—often through dialogue with other users. This diverse list of actions and feelings highlights the complex nature of adoption as a process and contrasts markedly with the widely cited “adopter categories” (“early adopter,” “laggard”) that have been extensively misapplied as explanatory variables. There is little empirical support for these stereotypical and value-laden terms, which fail to acknowledge the adopter as an actor who interacts purposefully and creatively with a complex innovation. The seven aspects of adopters and the adoption process that we used in our overall model are based on Rogers's extensive overview of the wider literature on adoption (1995) plus additional empirical studies of health service innovations (see full report for details). General Psychological AntecedentsWe identified a large literature from cognitive and social psychology on individual traits associated with the propensity to try out and use innovations (e.g., tolerance of ambiguity, intellectual ability, motivation, values, and learning style). This evidence has been largely ignored by researchers studying the diffusion of innovations and was beyond the scope of our own study, but it is ripe for review in relation to this research question. Context-Specific Psychological AntecedentsAn intended adopter who is motivated and able (in terms of values, goals, specific skills, and so on) to use a particular innovation is more likely to adopt it (for strong direct evidence, see Ferlie et al. 2001; Gladwin, Dixon, and Wilson 2002; and Yetton, Sharma, and Southon 1999). If the innovation meets an identified need by the intended adopter, he or she is more likely to adopt it (for strong indirect evidence, see Hall and Hord 1987; and Wejnert 2002). MeaningThe meaning of the innovation for the intended adopter has a powerful influence on the adoption decision (for strong indirect and moderate direct evidence, see Dearing and And 1994; and Timmons 2001). If the meaning attached to the innovation by individual adopters matches the meaning attached by top management, service users, and other stakeholders, the innovation is more likely to be assimilated (for moderate indirect evidence, see Eveland 1986). The meaning attached to an innovation is generally not fixed but can be negotiated and reframed, for example, through discourse within the organization or across interorganizational networks (for strong direct evidence, see Ferlie et al. 2001). The success of initiatives to support such a reframing of meaning is variable and not easy to predict (limited evidence). The Adoption DecisionThe decision by an individual within an organization to adopt a particular innovation is rarely independent of other decisions. It may be contingent (dependent on a decision made by someone else in the organization), collective (the individual has a “vote” but ultimately must acquiesce to the decision of a group), or authoritative (the individual is told whether or not to adopt it) (Rogers 1995). Authoritative decisions (e.g., making adoption by individuals compulsory) may increase the chance of initial adoption by individuals but may also reduce the chance that the innovation is successfully implemented and routinized (for moderate indirect evidence, see Rogers 1995). Adoption is a process rather than an event, with different concerns being dominant at different stages. The adoption process in individuals is traditionally presented as having five stages: awareness, persuasion, decision, implementation, and confirmation (Rogers 1995). However, we found that a lesser-known model, the Concerns Based Adoption Model developed for innovation in schools, better explained the findings of empirical studies of complex service innovations in an organizational context. This model provided three components for our model: Concerns in Preadoption StageImportant prerequisites for adoption are that the intended adopters are aware of the innovation; have sufficient information about what it does and how to use it; and are clear about how the innovation would affect them personally, for example, in terms of costs (for strong indirect evidence, see Hall and Hord 1987). Concerns during Early UseSuccessful adoption is more likely if the intended adopters have continuing access to information about what the innovation does and to sufficient training and support on task issues (i.e., about fitting the innovation to daily work) (for strong indirect evidence, see Hall and Hord 1987). Concerns in Established UsersSuccessful adoption is more likely if adequate feedback is provided to the intended adopters about the consequences of adoption (for strong indirect evidence, see Hall and Hord 1987) and if the intended adopters have sufficient opportunity, autonomy, and support to adapt and refine the innovation to improve its fitness for purpose (for strong indirect evidence, see Rogers 1995). Assimilation by the SystemMost of the research on the diffusion of innovations focused on simple, product-based innovations, for which the unit of adoption is the individual, and diffusion occurs by means of simple imitation (Rogers 1995). It is important not to use this literature to overgeneralize to complex, process-based innovations in service organizations, for which the unit of adoption (at this level, more often called assimilation) is the team, department, or organization in which various changes in structures or ways of working will be required. In such circumstances, there is almost always a formal decision-making process, an evaluation phase or phases, and planned and sustained efforts at implementation. In other words, empirical work in the field of organization and management clearly shows that successful individual adoption is only one component of the assimilation of complex innovations in organizations. The evaluation of system readiness and the crucial implementation phase are considered separately later, but one overarching concept should be borne in mind about the assimilation process as a whole: The AssimilationAlthough one large, high-quality study (Meyer and Goes 1988) demonstrated an organizational parallel to the “stages” of individual adoption, comprising “knowledge-awareness,” “evaluation-choice,” and “adoption-implementation,” the remaining empirical evidence was more consistent with an organic and often rather messy model of assimilation in which the organization moved back and forth between initiation, development, and implementation, variously punctuated by shocks, setbacks, and surprises (for strong direct evidence, see Van de Ven et al. 1999). Diffusion and DisseminationThe various influences that help spread the innovation can be thought of as lying on a continuum between pure diffusion (in which the spread of innovations is unplanned, informal, decentralized, and largely horizontal or mediated by peers) and active dissemination (in which the spread of innovation is planned, formal, often centralized, and likely to occur more through vertical hierarchies; see Figure 2). Whereas mass media and other impersonal channels may create awareness of an innovation, interpersonal influence through social networks (defined as “the pattern of friendship, advice, communication and support which exists among members of a social system” [Valente 1996, 70]) is the dominant mechanism for diffusion. Again drawing on Rogers's overview (1995) as well as other empirical work (see full report for more references), we identified a number of components for our model: Network StructureThe adoption of innovations by individuals is powerfully influenced by the structure and quality of their social networks (for strong indirect and moderate direct evidence, see Fennell and Warnecke 1988; Valente 1996; and West et al. 1999). Different groups have different types of social networks. Doctors, for example, tend to operate in informal, horizontal networks, and nurses more often have formal, vertical networks (for moderate direct evidence, see West et al. 1999). Different social networks also have different uses for different types of influence; for example, horizontal networks are more effective for spreading peer influence and supporting the construction and reframing of meaning; vertical networks are more effective for cascading codified information and passing on authoritative decisions (for moderate indirect evidence and limited direct evidence, see Rogers 1995; and West et al. 1999). HomophilyThe adoption of innovations by individuals is more likely if they are homophilous—that is, have similar socioeconomic, educational, professional, and cultural backgrounds—with current users of the innovation (for strong direct evidence, see Fennell and Warnecke 1988; Fitzgerald et al. 2002; and West et al. 1999). Opinion LeadersSome persons have a particular influence on the beliefs and actions of their colleagues (for strong direct evidence, see Becker 1970; and Coleman, Katz, and Menzel 1966). Expert opinion leaders exert influence through their authority and status, and peer opinion leaders exert influence through their representativeness and credibility (for moderate direct evidence, see Fitzgerald et al. 2002; and Locock et al. 2001). Opinion leaders can have either a positive or negative influence. If a project is insufficiently appealing (e.g., in clarity of goals, organization, and resources), it will not attract the support of key opinion leaders (for strong indirect and moderate direct evidence, see Locock et al. 2001; Rogers 1995). Harnessing the Opinion Leader's InfluenceEven though the powerful impact of social influence (such as that of opinion leaders) in naturalistic settings is well established, attempts to engage such individuals in planned change efforts have often had disappointing results. In cases in which opinion leaders have been trained to influence the behavior of their peers (e.g., to persuade fellow clinicians to follow a new guideline), the impact is generally positive in direction but small in magnitude (for strong direct evidence, see Thomson O'Brien et al. 2003). The failure to identify the true opinion leaders and, in particular, the failure to distinguish between monomorphic opinion leaders (influential for a particular innovation only) and polymorphic opinion leaders (influential across a wide range of innovations) may limit the success of such intervention strategies (for strong indirect and moderate direct evidence, see Locock et al. 2001; and Rogers 1995). ChampionsThe adoption of an innovation by individuals in an organization is more likely if key individuals in their social networks are willing to support the innovation (for strong indirect and moderate direct evidence, see Backer and Rogers 1998; Markham 1998; Meyer and Goes 1988; and Schon 1963). The different champion roles for organizational innovations include (1) the organizational maverick, who gives the innovators autonomy from the organization's rules, procedures, and systems so they can establish creative solutions to existing problems; (2) the transformational leader, who harnesses support from other members of the organization; (3) the organizational buffer, who creates a loose monitoring system to ensure that innovators properly use the organization's resources while still allowing them to act creatively; and (4) the network facilitator, who develops cross-functional coalitions within the organization (for moderate indirect evidence, see Shane 1995). There is very little direct empirical evidence on how to identify, and systematically harness the energy of, organizational champions. Boundary SpannersAn organization is more likely to adopt an innovation if those people who have significant social ties both inside and outside the organization are able and willing to link the organization to the outside world in relation to this particular innovation. Such individuals play a pivotal role in capturing the ideas that will become organizational innovations (for strong indirect evidence, see Rogers 1995; for moderate direct evidence, see Kimberly and Evanisko 1981). Organizations that promote and support the development and execution of boundary-spanning roles are more likely to become aware of and assimilate innovations quickly (for moderate direct evidence, see Barnsley, Lemieux-Charles, and McKinney 1998; Ferlie et al. 2001; and Tushman 1977). Formal Dissemination ProgramsWhen a planned dissemination program is used for the innovation (e.g., led by an external change agency), it will be more effective if the program's organizers (1) take full account of potential adopters’ needs and perspectives, with particular attention to the balance of costs and benefits for them; (2) tailor different strategies to the different demographic, structural, and cultural features of different subgroups; (3) use a message with appropriate style, imagery, metaphors, and so on; (4) identify and use appropriate communication channels; and (5) incorporate rigorous evaluation and monitoring of defined goals and milestones (for strong indirect evidence, see Rogers 1995). The diverse literature on diffusion and dissemination highlighted an important area of contestation in paradigms of diffusion. Most diffusion research has addressed proactively developed innovations (e.g., technologies or products developed in formal research programs) whose main mechanism of spread is centrally driven and controlled (what we have defined as dissemination). But many innovations in service delivery and organization occur as “good ideas” in local services, which spread informally and in a largely uncontrolled way (diffusion). This tension, which has received remarkably little attention in the literature we reviewed, is illustrated in Figure 2. System Antecedents for InnovationDifferent organizations provide widely differing contexts for innovations, and some features of organizations (both structural and “cultural”) have been shown to influence the likelihood that an innovation will be successfully assimilated (i.e., adopted by all relevant individuals and incorporated into “business as usual”). Structural Determinants of InnovativenessWe identified four previous meta-analyses that included both manufacturing and service organizations (Damanpour 1991 [see Table 2], 1992, 1996; Tornatsky and Klein 1982) and 15 additional empirical studies (examined in 17 papers) from the service sector (Anderson and West 1998; Baldridge and Burnham 1975; Burns and Wholey 1993; Castle 2001; Champagne et al. 1991; Dopson et al. 2002; Dufault et al. 1995; Fitzgerald et al. 2002; Goes and Park 1997; Gosling, Westbrook, and Braithwaite 2003; Kimberly and Evanisko 1981; Meyer and Goes 1988; Newton et al. 2003; Nystrom, Ramamurthy, and Wilson 2002; Patel 1996; Rashman and Hartley 2002; Wilson, Ramamurthy, and Nystrom 1999). Their findings vary somewhat, though less so than is often claimed. They suggest that an organization will assimilate innovations more readily if it is large, mature, functionally differentiated (i.e., divided into semiautonomous departments and units), and specialized, with foci of professional knowledge; if it has slack resources to channel into new projects; and if it has decentralized decision-making structures (strong direct evidence). Size is almost certainly a proxy for other determinants, including slack resources and functional differentiation. TABLE 2Impact of Structural Determinants on Organizational Innovativeness from a Meta-Analysis of 23 Studies of Service and Manufacturing
Although these structural determinants are significantly, positively, and consistently associated with organizational innovativeness, together they account for less than 15 percent of the variation among comparable organizations. Furthermore, the relationship between structural determinants and innovativeness is moderated by and/or contingent on additional factors (e.g., the radicalness of the innovation, whether it is administrative or technical, and the stage of adoption). There is little empirical evidence to support the efficacy of interventions to change an organization's structure to make it more “innovative,” except that establishing semiautonomous multidisciplinary project teams is independently associated with successful implementation. One important weakness of the literature on structural determinants of innovativeness is the assumption that they can be treated as variables whose impact can be isolated and independently quantified. For example, the empirical studies of organizational size implicitly assume that there is a “size effect” that is worth measuring and that is to some extent generalizable. An alternative theoretical approach (House, Rousseau, and Thomas-Hunt 1995), supported by a number of recent detailed qualitative studies (Champagne et al. 1991; Ferlie et al. 2001), is that the determinants of organizational innovativeness interact in a complex, unpredictable, and nongeneralizable way with one another. There is consistent empirical evidence for two more nonstructural determinants of organizational innovativeness: Absorptive Capacity for New KnowledgeAn organization that is systematically able to identify, capture, interpret, share, reframe, and recodify new knowledge; to link it with its own existing knowledge base; and to put it to appropriate use will be better able to assimilate innovations, especially those that include technologies (for strong direct evidence, see Barnsley, Lemieux-Charles, and McKinney 1998; and Ferlie et al. 2001). Prerequisites for absorptive capacity include the organization's existing knowledge and skills base (especially its store of tacit, uncodifiable knowledge) and preexisting related technologies, a “learning organization” culture, and proactive leadership directed toward sharing knowledge (for strong direct evidence, see Barnsley, Lemieux-Charles, and McKinney 1998; Ferlie et al. 2001; and Zahra and George 2002). The knowledge that underpins the adoption, dissemination, and implementation of a complex innovation within an organization is not objective or given. Rather, it is socially constructed and frequently contested and must be continually negotiated among members of the organization or system. Strong, diverse, and organic (i.e., flexible, adaptable, and locally grown) intraorganizational networks (especially opportunities for interprofessional teamwork, and the involvement of clinicians in management networks and vice versa) help this process and facilitate the development of shared meanings and values in relation to the innovation (for moderate direct evidence, see Barnsley, Lemieux-Charles, and McKinney 1998; and Ferlie et al. 2001). An important use of knowledge in health care organizations is the application of research evidence for the efficacy of health technologies. Health professionals should ensure that they and their staff are aware of new developments (and new definitions of what is obsolete) in diagnostic tests, drugs, surgical procedures, and so on, and modify their practice accordingly. A major overview of high-quality qualitative studies of how research evidence is identified, circulated, evaluated, and used in health care organizations (Dopson et al. 2002) confirms other findings from the mainstream knowledge-utilization literature, which suggest that before it can contribute to organizational change initiatives, knowledge must be enacted and made social, entering into the stock of knowledge constructed and shared by other individuals. Knowledge depends for its circulation on interpersonal networks and will spread only if these social features are taken into account and barriers are overcome. Receptive Context for ChangeThe receptive context for change incorporates a number of organizational features that have been independently associated with its ability to embrace new ideas and face the prospect of change (Pettigrew and McKee 1992). An organization with such a receptive context will be better able to assimilate innovations. In addition to an absorptive capacity for new knowledge, the components of receptive context include strong leadership, clear strategic vision, good managerial relations, visionary staff in pivotal positions, a climate conducive to experimentation and risk taking, and effective data capture systems (for strong indirect and moderate direct evidence, see Anderson and West 1998; Barnsley, Lemieux-Charles, and McKinney 1998; Dopson et al. 2002; Gosling, Westbrook, and Braithwaite 2003; Newton et al. 2003; Nystrom, Ramamurthy, and Wilson 2002; Pettigrew and McKee 1992; and Van de Ven et al. 1999). Leadership may be especially helpful in encouraging organizational members to break out of the convergent thinking and routines that are the norm in large, well-established organizations (for strong direct evidence, see Van de Ven et al. 1999). System Readiness for InnovationAn organization may be amenable to innovation in general but not ready or willing to assimilate a particular innovation. As Figure 3 shows, formal consideration of the innovation allows the organization to move (or perhaps choose not to move) to a specific state of system readiness for that innovation. The elements of system readiness are as follows: Tension for ChangeIf staff perceive that the current situation is intolerable, a potential innovation is more likely to be assimilated successfully (for moderate direct evidence, see Gustafson et al. 2003). Innovation-System FitAn innovation that fits with the organization's existing values, norms, strategies, goals, skill mix, supporting technologies, and ways of working is more likely to be assimilated (for strong indirect and moderate direct evidence, see Gustafson et al. 2003; Rogers 1995; and the related concept of “fuzzy boundaries” in this article). Assessment of ImplicationsIf the implications of the innovation (including its subsequent effects) are fully assessed and anticipated, the innovation is more likely to be assimilated (for strong indirect and moderate direct evidence, see Gustafson et al. 2003; and Rogers 1995). Most of the following implementation issues are amenable to advance assessment and planning: Support and AdvocacyIf the supporters of the innovation outnumber and are more strategically placed than its opponents are, it is more likely to be assimilated (for strong indirect and moderate direct evidence, see Champagne et al. 1991; Gustafson et al. 2003; Rogers 1995; and also “champions,” in this article). Dedicated Time and ResourcesIf the innovation starts out with a budget and if the allocation of resources is both adequate and continuing, it is more likely to be assimilated (for strong indirect and moderate direct evidence, see Gustafson et al. 2003; and Rogers 1995). Capacity to Evaluate the InnovationIf the organization has tight systems and appropriate skills in place to monitor and evaluate the impact of the innovation (both anticipated and unanticipated), the innovation is more likely to be assimilated and sustained (for strong indirect and moderate direct evidence, see Gustafson et al. 2003; Plsek 2003; and Rogers 1995). The Outer Context: Interorganizational Networks and CollaborationAn organization's decision to adopt an innovation and its efforts to implement and sustain it depend on a number of external influences: Informal Interorganizational NetworksAn important influence on an organization's decision to adopt is whether a threshold proportion of comparable (homophilous) organizations have done so or plan to do so (for strong direct evidence, see Burns and Wholey 1993; Fennell and Warnecke 1988; Robertson and Wind 1983; and Westphal, Gulati, and Shortell 1997). A “cosmopolitan” organization (one that is externally well networked with others) is more susceptible to this influence (for strong direct evidence, see Burns and Wholey 1993; Fennell and Warnecke 1988; Robertson and Wind 1983; and Westphal, Gulati, and Shortell 1997). Interorganizational networks promote the adoption of an innovation only after this is generally perceived as “the norm.” Until that time, networks can also serve to dissuade organizations from adopting innovations that have no perceived advantages (for strong indirect and moderate direct evidence, see Abrahamson 1991; Fitzgerald et al. 2002; and Westphal, Gulati, and Shortell 1997). Integrative organizational forms (such as the UK National Health Service, Health Maintenance Organizations, and professionally led networks of health care providers), which link provider organizations through common management and governance structures and explicit shared values and goals, can help spread innovations among member organizations (for moderate direct evidence, see Meyer, Johnson, and Ethington 1997). Intentional Spread StrategiesFormal networking initiatives such as quality improvement collaboratives (Øvretveit et al. 2002) or “Beacon” schemes (Rashman and Hartley 2002), aimed at sharing ideas and knowledge construction, are sometimes but not always effective (for moderate direct evidence, see Flamm, Berwick, and Kabcenell 1998; Horbar et al. 2001; Leape et al. 2000; O'Connor et al. 1996; Øvretveit et al. 2002; Rashman and Hartley 2002; and Rogowski et al. 2001). Such initiatives are often expensive, and the gains from them are difficult to measure; evidence of their cost-effectiveness is limited. The greatest success factors of health care quality improvement collaboratives are (1) the nature of the topic chosen for improvement; (2) the capacity and motivation of participating teams, particularly their leadership and team dynamics; (3) the motivation and receptivity to change of the organizations they represent; (4) the quality of facilitation, particularly the provision of opportunities to learn from others in an informal space; and (5) the quality of support provided to teams during the implementation phase (for moderate direct evidence, see Øvretveit et al. 2002). Wider EnvironmentThe evidence for the impact of environmental variables on organizational innovativeness in the service sector is sparse and heterogeneous, with each group of researchers exploring somewhat different aspects of the “environment” or “changes in the environment.” Environmental uncertainty has either a small positive impact or no impact on innovativeness (for moderate direct evidence, see Kimberly and Evanisko 1981; and Meyer and Goes 1988), and there may be small positive effects from interorganizational competition and the higher socioeconomic status of patients/clients (limited evidence). Political DirectivesAlthough our review was not designed to tap centrally into the literature on policymaking and its impact, some empirical studies of innovation formally measured the effect of the policy context on the adoption of a particular innovation. A policy “push” occurring at the early stage of implementation of an innovation initiative can increase its chances of success, perhaps most crucially by making available a dedicated funding stream (for strong direct evidence, see Exworthy, Berney, and Powell 2003; Fitzgerald et al. 2002; Granados et al. 1997; and Hughes et al. 2002). External mandates (political “must-dos”) increase an organization's predisposition (i.e., motivation), but not its capacity, to adopt an innovation (for moderate direct evidence, see Taylor et al. 1998). Such mandates (or the fear of them) may divert activity away from innovations as organizations second-guess what they will be required to do next rather than focus on locally generated ideas and priorities (for strong indirect evidence, see Meyers, Sivakumar, and Nakata 1999; for moderate direct evidence, see Exworthy, Berney, and Powell 2003). Implementation and RoutinizationMeyers, Sivakumar, and Nakata define implementation as “the early usage activities that often follow the adoption decision” (1999, 295). The evidence regarding the implementation of innovations was particularly complex and relatively sparse, and it was difficult to disentangle it from that regarding change management and organizational development in general. Implementation depends on many of the factors already covered in relation to the initial adoption decision and the early stages of assimilation. At the organizational level, the move from considering an innovation to successfully routinizing it is generally a nonlinear process characterized by multiple shocks, setbacks, and unanticipated events (Van de Ven et al. 1999). The key components of system readiness for an innovation are highly relevant to the early stages of implementation. In addition, a number of additional elements are specifically associated with successful routinization: Organizational StructureAn adaptive and flexible organizational structure, and structures and processes that support devolved decision making in the organization (e.g., strategic decision making devolved to departments, operational decision making devolved to teams on the ground) enhance the success of implementation and the chances of routinization (for strong indirect and direct evidence, see Meyers, Sivakumar, and Nakata 1999; and Van de Ven et al. 1999). Leadership and ManagementTop management support, advocacy of the implementation process, and continued commitment to it enhance the success of implementation and routinization (for strong indirect and moderate direct evidence, see Green 1998; Gustafson et al. 2003; Meyers, Sivakumar, and Nakata 1999). If the innovation aligns with the earlier goals of both top management and middle management and if the leaders are actively involved and frequently consulted, the innovation is more likely to be routinized (for moderate direct evidence, see Gustafson et al. 2003). Human Resource IssuesSuccessful routinization of an innovation in an organization depends on the motivation, capacity, and competence of individual practitioners (for moderate direct evidence, see Gustafson et al. 2003). The early and widespread involvement of staff at all levels, perhaps through formal facilitation initiatives, enhances the success of implementation and routinization (for strong indirect evidence, see Meyers, Sivakumar, and Nakata 1999; for moderate direct evidence, see Kitson, Harney, and McCormack 1998). When job changes are few and clear, high-quality training materials are available, and timely on-the-job training is provided, successful and sustained implementation is more likely (for strong indirect and moderate direct evidence, see Green 1998; Gustafson et al. 2003; Meyers, Sivakumar, and Nakata 1999; and McCormick, Steckler, and Mcleroy 1995). Team-based training may be more effective than individual training when the learning involves implementing a complex technology (for moderate direct evidence, see Edmondson, Bohmer, and Pisano 2001). Intraorganizational CommunicationEffective communication across structural (e.g., departmental) boundaries within the organization enhances the success of implementation and the chances of routinization (for strong indirect evidence, see Meyers, Sivakumar, and Nakata 1999). A narrative approach (i.e., the purposeful construction of a shared and emergent organizational story of “what we are doing with this innovation”) can serve as a powerful cue to action (for moderate indirect evidence, see Gabriel 2000; for limited direct evidence, see Bate 2004). Interorganizational NetworksThe more complex the implementation that is needed for a particular innovation, the greater the significance of the interorganizational network will be to the implementation's success (for moderate indirect evidence, see Meyers, Sivakumar, and Nakata 1999; and Valente 1995). FeedbackAccurate and timely information about the impact of the implementation process (through efficient data collection and review systems) increases the chance of successful routinization (for strong indirect and moderate direct evidence, see Green 1998; and Grimshaw et al. 2004). Linkage among Components of the ModelThere is some empirical evidence (and also robust theoretical arguments) for building strong links among the different components of the model in Figure 3: Linkage at the Development StageAn innovation that is centrally developed (e.g., in a research center) is more likely to be widely and successfully adopted if the developers or their agents are linked with potential users at the development stage in order to capture and incorporate the users’ perspective (for strong indirect evidence, see Rogers 1995). Such linkage should aim not merely for “specification” but also for a shared and organic (developing, adaptive) understanding of the meaning and value of the innovation in use and should also work toward a shared language for describing the innovation and its impact.
Role of the Change AgencyIf a change agency is part of a dissemination program, the nature and quality of any linkage with intended adopter organizations will influence the likelihood of adoption and the success of implementation (strong indirect and moderate direct evidence). In particular, human relations should be positive and supportive; the two systems should have a common language, meanings, and value systems; they should share resources; the change agency should enable and facilitate networking and collaboration among organizations; and the consequences of innovations should be jointly evaluated. The change agency should have the capacity, commitment, technical capability, communication skills, and project management skills to assist with operational issues. This is particularly important in relation to technology-based innovations, which should be disseminated as augmented products with tools, resources, technical help, and so on (for moderate direct evidence, see Lomas 2000; and Rogers 1995). External Change AgentsChange agents employed by external agencies will be more effective if they are (1) selected for their homophily and credibility with the potential users of the innovation; (2) trained and supported to develop strong interpersonal relationships with potential users and to explore and empathize with the user's perspective; (3) encouraged to communicate the users’ needs and perspective to the developers of the innovation; and (4) able to empower the users to make independent evaluative decisions about the innovation (for strong indirect and limited direct evidence, see Rogers 1995). Discussion and Recommendations for Further ResearchThis study has attempted to combine a large and diverse literature into a unifying model of the diffusion of innovations in health care organizations. Our methods were systematic and independently verifiable. However, the literature was vast and complex, our approach was emergent and somewhat unconventional, and many subjective judgments and serendipitous discoveries were involved. A different group of researchers setting out to answer the same research question would inevitably have identified a different set of primary sources and made different judgments about their quality and relevance. Their synthesis might have produced a different unifying model. This is, arguably, an inherent characteristic of any systematic review that addresses complex interventions and seeks to unpack the nuances of their implementation in different social, organizational, or environmental contexts. In this respect, a meta-narrative review can be thought of as a particular application of a realist review, in which the reviewer's interpretive judgments are integral to the synthesis process and can never be fully rationalized or standardized (Greenhalgh et al. 2005b; Pawson et al. 2005). The findings presented here, and especially the model in Figure 3, should therefore be seen as “illuminating the problem and raising areas to consider” rather than “providing the definitive answers.” A recently published review of diffusion of innovations aimed at changing individual clinician behavior, not available when we were developing our model, was consistent with our own conclusions (Fleuren, Wiefferink, and Paulussen 2004). Our review affirmed many well-described themes in the literature, such as the useful list of innovation attributes that predict (but do not guarantee) successful adoption; the importance of social influence and the networks through which it operates; the complex and contingent nature of the adoption process; the characteristics (both “hard” and “soft”) of organizations that encourage and inhibit innovation; and the messy, stop-start, and difficult-to-research process of assimilation and routinization. We also exposed some demons in this literature, such as the lack of empirical evidence for the widely cited “adopter traits”; the focus on innovations that arise centrally and are disseminated through official channels at the expense of those that arise peripherally and spread informally; the limited generalizability of the empirical work on product-based innovation in companies to process innovation in service organizations; and the near absence of studies focusing primarily on the sustainability of complex service innovations. The components of this model do not, of course, represent a comprehensive list of the determinants of organizational innovativeness and successful assimilation. They are simply the areas on which research has been undertaken and findings have been published. Conspicuously absent from most empirical work in the service sector, for example, is the important issue of internal politics (e.g., doctor-manager power balances), which was identified in a single qualitative study as one of several critical influences (Champagne et al. 1991). In an evaluation of five projects to implement complex service innovations in primary health care, our own team found that power relations (especially between a project steering group and the main project worker) were critical to successful implementation but that they were extremely difficult to explore systematically and raised ethical issues for the research team (Hughes et al. 2002). A striking finding of this extensive review was the tiny proportion of empirical studies that acknowledged, let alone explicitly set out to study, the complexities of spreading and sustaining innovation in service organizations. Most studies concentrated on a few of the components depicted in our model and failed to take account of their different interactions and contextual and contingent features. This, of course, is an inherent limitation of any experimental or quasi-experimental research: The shifting baseline of context and the multiplicity of confounding variables must be stripped away (“controlled for”) to make the research objective (Pawson et al. 2005). But herein lies a paradox. Context and “confounders” lie at the very heart of the diffusion, dissemination, and implementation of complex innovations. They are not extraneous to the object of study; they are an integral part of it. The multiple (and often unpredictable) interactions that arise in particular contexts and settings are precisely what determine the success or failure of a dissemination initiative. Champions, for example, emerged in our review as a key determinant of organizational innovation, but no amount of empirical research will provide a simple recipe for how champions should behave that is independent of the nature of the innovation, the organizational setting, the sociopolitical context, and so on. Based on the findings of this review, on some of the methodological recommendations made by others (Green 2001; Pawson et al. 2005; Rootman et al. 2001), and on feedback from policymakers who read drafts of this review, we suggest that the next generation of research on diffusion of health service innovations should be
When carrying out this review, we were struck by the duplication of empirical studies and also by the number of studies that had been undertaken without a comprehensive review of the existing relevant research, many of which asked what appeared to be obsolete questions. Bearing in mind our general recommendation for a more “whole-systems” approach to researching organizational innovation, we next highlight specific areas in which we believe further empirical research is—and, equally important, is not—needed. InnovationsFurther research into the attributes of innovations that promote their adoption is probably not needed. Instead, research in this area should be directed at the following questions: How do innovations in health service organizations arise, and in what circumstances? What mix of what factors tends to produce “adoptable” innovations (e.g., ones that have clear advantages beyond their source organization and low implementation complexity and are readily adaptable to new contexts)? How can innovations in health service organizations be adapted to be perceived as more advantageous, more compatible with prevailing norms and values, less complex, more trialable, with more observable results, and with greater scope for local reinvention? Is there a role for a central agency, resource center, or officially sanctioned demonstration programs in this? How are innovations arising as “good ideas” in local health care systems reinvented as they are transmitted through individual and organizational networks, and how can this process be supported or enhanced? How can we identify “bad ideas” likely to spread so that we can intervene to prevent this? Adopters and AdoptionWe do not recommend further descriptive studies of patterns of adoption by individuals. We believe the main unanswered questions are the following: Why and how do people (and organizations) reject an innovation after adopting it? (In the more than 200 empirical research studies covered in our review, only one explicitly and prospectively studied discontinuance; see Riemer-Reiss 1999). What are the transferable lessons from cognitive and social psychology about the ability and tendency of individuals to adopt particular innovations in particular circumstances? For example, what can we glean from the mainstream literature about how individuals process information, make decisions, apply heuristics, and so on? A particularly fruitful area is likely to be the psychological literature on the interaction between humans and computers as it applies to the adoption and assimilation of information and communications technology (ICT) innovations in the service sector. Dissemination and Social InfluenceWe do not recommend further “intervention” trials of the use of opinion leaders to change the behavior of potential adopters. We already know from published research that opinion leadership is a complex and delicate process, and research that fails to capture these process elements is unlikely to add to what we already know. We recommend that research into dissemination address the following questions: What is the nature of interpersonal influence and opinion leadership in the range of different professional and managerial groups in the health service, especially in relation to organizational innovations? In particular, how are key players identified and influenced? What is the nature and extent of the social networks of different players in the health service (both clinical and nonclinical)? How do these networks serve as channels for social influence and the reinvention and embedding of complex service innovations? Who are the individuals who act as champions for organizational innovations in health services? What is the nature of their role, and how might it be enabled and enhanced? Who are the individuals who act as boundary spanners among health service organizations, especially in relation to complex service innovations? What is the nature of their role, and how might it be enabled and enhanced? The Organizational ContextWe do not recommend further survey-based research to identify structural determinants of innovativeness in health care organizations, since the small but significant effect of structural determinants is well established. We suggest the following questions as possible directions for further research: To what extent do “restructuring” initiatives (popular in health service organizations) improve their ability to adopt, implement, and sustain innovations? In particular, will a planned move from a traditional hierarchical structure to one based on semiautonomous teams with independent decision-making power improve innovativeness? How can we improve the absorptive capacity of service organizations for new knowledge? In particular, what is the detailed process by which ideas are captured from outside, circulated internally, adapted, reframed, implemented, and routinized in a service organization, and how might this process be systematically enhanced? How can leaders of service organizations set about achieving a receptive context for change; that is, the kind of culture and climate that supports and enables change in general? A systematic review centering on the mainstream change management literature (which we explicitly excluded from this review) is probably the most appropriate first step for this question. What is the process leading to long-term routinization (with appropriate adaptation and development) of innovations in health service delivery and organization? System Readiness for InnovationThere is relatively little systematic research on the development of system readiness (i.e., the steps that organizations can take to assess and anticipate the impact of an innovation). The following questions should be addressed: What steps must be taken by service organizations when moving toward a state of “readiness” (i.e., with all players on board and with protected time and funding), and how can this overall process be supported and enhanced? In particular, (1) How can tension for change be engendered? (2) How can innovation-system fit best be assessed? (3) How can the implications of the innovation be assessed and fed into the decision-making process? (4) What measures enhance the success of efforts to secure funding for the innovation in the resource allocation cycle? and (5) How can the organization's capacity to evaluate the impact of the innovation be enhanced? What are the characteristics of organizations that successfully avoid taking up “bad ideas”? Are they just lucky, or do they have better mechanisms for evaluating the ideas and anticipating the subsequent effects? The Outer ContextAside from questions in the fields of political science and macroeconomics, the main research questions on the environmental context are the following: What is the nature of informal interorganizational networking in different areas of activity, and how can this be enhanced through explicit knowledge management activities (such as the appointment and support of knowledge workers and boundary spanners)? What is (or could be) the role of professional organizations and informal interprofessional networks in spreading innovation among health care organizations? What is the cost-effectiveness of structured health care quality collaboratives and comparable models of quality improvement, and how can this be enhanced? To what sort of projects in what sort of contexts should resources for such interorganizational collaboratives be allocated? What are the harmful effects of an external “push” (such as a policy directive or incentive) for a particular innovation when the system is not ready? What are the characteristics of more successful external pushes promoting the assimilation and implementation of innovations by health service organizations? ImplementationOverall, we found that empirical studies of implementing and maintaining innovations in service organizations (1) had been undertaken from a pragmatic rather than an academic perspective and been presented as “gray literature” reports (which for practical reasons we did not include in this review); (2) were difficult to disentangle from the literature on change management in general; and (3) were impoverished by lack of process information. We recommend that further research focus on the following two questions: By what processes are particular innovations in health service delivery and organization implemented and sustained (or not) in particular contexts and settings, and can these processes be enhanced? This question, which was probably the most serious gap in the literature we uncovered for this review, would benefit from in-depth mixed-methodology studies aimed at building up a rich picture of process and impact. Are there any additional lessons from the mainstream change management literature (to add to the diffusion of innovations literature reviewed here) for implementing and sustaining innovations in health care organizations? ConclusionWe believe that this extensive systematic review has produced three important outputs: (1) a parsimonious and evidence-based model for considering the diffusion of innovations in health service organizations; (2) clear knowledge gaps on which further research on the diffusion of innovations in service organizations should be focused; and (3) a robust and transferable methodology for systematically reviewing complex research evidence (methodological issues, especially the transferability of this method to other systematic review topics, are discussed in more detail in Greenhalgh et al. 2005b). Research commissioners, in particular, should note our suggestions for where further research is unlikely to add substantially to the knowledge base. We encourage other research teams to test both our proposed model for the diffusion of service innovations and our proposed methodology for a systematic review of diffuse bodies of literature. AcknowledgmentsThis study was funded by the UK Department of Health. Thanks to the many people, too numerous to mention individually, who provided formative feedback on our work during this project or comments on drafts of the final report. References
Articles from The Milbank Quarterly are provided here courtesy of Milbank Memorial Fund Which of the following agricultural techniques did the Aztecs use to increase food production before contact with Spaniards?Based on your knowledge of World History and Source 1, which of the following agricultural techniques did the Aztecs use to increase food production before contact with Spaniards? use of very rich soil in "floating gardens." Increased agricultural productivity contributed to revival of some urban areas.
On which of the following did Inca builders rely?On which of the following did Inca builders rely? Cleverly constructed wheeled carts, which helped haul materials up steep mountain slopes. The skill and industry of an experienced labor force.
Which of the following was an important long term effect of the creation and expansion of the English East India Company?Which of the following was an important long-term effect of the creation and expansion of the English (later British) East India Company? British economic and political colonialism in South Asia.
|