{"id":468,"date":"2013-05-30T08:18:58","date_gmt":"2013-05-30T08:18:58","guid":{"rendered":"http:\/\/managingtoteach.org\/?p=468"},"modified":"2013-05-30T08:18:58","modified_gmt":"2013-05-30T08:18:58","slug":"sociotechnical-reflections-and-four-key-issues-for-moocs","status":"publish","type":"post","link":"http:\/\/edinfoconnections.com\/?p=468","title":{"rendered":"SocioTechnical Reflections and Four Key Issues for MOOCS"},"content":{"rendered":"<h2>Introduction<\/h2>\n<p>In a recent essay titled <a href=\"http:\/\/scenic.princeton.edu\/files\/MOOC_FLIP_thoughts.pdf\"><i>Innovating Education with MOOC\/FLIP<\/i><\/a>, Princeton University professor Mung Chiang thoughtfully discusses Massive Online Open Courses (MOOCs) and some related innovations that are presenting educators with dilemmas and bringing to the surface pressures for possible change in traditional higher educational structures. Chiang sketches some of the recent history of these rapidly developing phenomena and shares some of his experiences developing and teaching a MOOC around computer and social networks.\u00a0 The core of Chiang\u2019s essay\u2014the ten questions focusing on opportunity and challenges\u2014deals with some of the most important tensions surrounding MOOCs today.\u00a0<!--more-->  These tensions include huge initial course enrollments and low completion rates (at or under 5%), developing high-quality assessments that are practical at a large scale, emergence of \u201cnew\u201d education economics with a potential to increase academy access through free education while also needing sustainable business models, achieving both large scale and personalization, and using technologies in principal roles that are themselves often still developing and incomplete.\u00a0 He concludes with speculations on MOOC productivity with an equation that describes the enrollment, completion ratio, and graduation size.<\/p>\n<p>My reflection draws upon sociotechnical theory to help conceptualize some interrelated innovations including MOOCs and flipped classrooms.\u00a0 Is uses a frame that deconstructs these movements and places their possible impacts into a historical perspective.\u00a0 Like Chiang\u2019s piece, this reflection also focuses on both opportunities and challenges.\u00a0 It presents a way of viewing innovations like MOOCs and suggests some implications for a research and development agenda to support these kinds of innovations.\u00a0 The \u201cflipping of classrooms,\u201d something I discuss in more detail later, is less emphasized in this paper than MOOCs are. While Chiang\u2019s MOOC experience involves the rearrangement of the classroom (ex: flipping lecture and individual support), there are some important differences between MOOCs and flipping.\u00a0 Both have emerged around the same time and are topics of similar reform rhetoric.\u00a0 However, MOOCs can occur without flipping and vice versa. \u00a0They are both interrelated sociotechnically and so their co-occurrence in Chiang\u2019s essay is not accidental.<\/p>\n<p>As a field, we find ourselves often asking some of the same kinds of questions about MOOCs, flipping classrooms, and other related topics that include learning analytics and digital personalization.\u00a0 These questions, like those asked about other educational technologies include: \u201cWill they live up to their hype or will there be another fad and fashion next year to replace them?; Are they really that different from what we have today?\u201d\u00a0 \u201cHow much of a real impact can these new approaches make on substantive educational problems many have been working on for a long time?\u201d The sociotechnical frame helps with these kinds of questions as it draws upon other kinds of educational movements and looks broadly at not only a specific set of innovations, but periods of change that can involve complexes of technologies and related practice areas. The sociotechnical frame is similar to another that is common today\u2014<i>disruptive innovations<\/i>\u2014popularized by Harvard professor Clayton Christensen (Christensen, Baumann, Ruggles, &amp; Sadtler, 2006, Christensen, Horn, &amp; Johnson, 2008).\u00a0 It differs, however, in not only considering how specific technology can be a change agent, but also how a technology is situated within social and technical contexts that can enable and make possible its integration into social practice.<br \/>\nThe sociotechnical approach for many issues differs from many approaches common to a particular education sub-discipline.\u00a0 Because many technologies can impact several parts of an educational system at the same time, the discussion of some topics can draw on several different literatures.\u00a0 Rather than looking for the kinds of measures common to traditional communities, the sociotechnical lens will often focus on different kinds of effects that reflect the evolution and social impact of the technologies. The result is often a better picture of the shape of the movement of an innovation through society rather than its momentary measurable effects.<\/p>\n<h2>Sociotechnical Framing of the MOOC Phenomena<\/h2>\n<p>I agree with Chiang in his framing of the issues by using the term <i>disruptive forces<\/i>. This, I believe, is more appropriate than calling MOOCs disruptive innovations because MOOCs have yet to actually disrupt the business of traditional higher education.\u00a0 Despite their dramatic increase in popularity, their ultimate impact is today unproven.\u00a0 MOOCs are, however, putting pressure on traditional structures can be framed in the current context as forces rather than actual disrupters.\u00a0 However, MOOCs do fit the classical model of <i>diffusing innovations<\/i> (Rogers, 1995). Diffusing innovations are ones that move through societies and organizations, often rapidly, according to fairly well established patterns of early adoption followed by broader acceptance.\u00a0 Certainly MOOCs fit this kind of innovation.\u00a0 However, not all innovations that diffuse are lasting.\u00a0 Some fade quickly and some have a longer life.\u00a0 In fact, as Chiang points out in several ways, the path for MOOCs to reshape higher education is not an easy one and his questions focus us on the extent to which we should expect that MOOCs will reshape different parts of education and how quickly.<\/p>\n<p>As we consider MOOCs today, the sociotechnical frame will allow us to see them within a continuum that includes other forms of online learning and blended models (Bonk, Graham, Cross, &amp; Moore, 2006; Horn and Staker, 2011; Means, et at., 2010). Open educational resources (OERs) that include the kinds of videos combined with simple assessments used by Khan Academy; big data and learning analytics (Brynjolfsson, 2012; LaValle, Lesser, Shockley, &amp; Kruschwitz, 2010; Lohr, 2012; Pea, Childress, and Yowell, 2012); and next generation models of learning are related innovations with disruptive potential.\u00a0 \u00a0Furthermore, media and data infrastructures (NSF, 2007) that support the high-bandwidth exchange of information are also importantly related to many of these innovations.\u00a0 While one might have imagined something like MOOCs decades ago, the medium to transmit high-quality video to local devices and the devices to play them took some years to develop so infrastructures are critically related to MOOCs and their sibling innovations.<\/p>\n<p>One issue Chiang raises at the end of his paper that I agree is important and little discussed is the variation in different kinds of subject areas such as the sciences and humanities.\u00a0 It is easy to talk about education as a unitary field.\u00a0 From a distance it can seem so.\u00a0 However, in reality, education is made up of many different overlapping fields. All of these overlapping fields are sensitive to context, but the contexts frequently differ.\u00a0 If we just consider the profession of teaching and how difficult it would be to move teachers from one job in a particular grade or even kind of school to another, the variation in education becomes clear.\u00a0 This variation across subjects, grades, and social settings make diffusion and disruption complicated.\u00a0 Many of the technical challenges that Chiang discusses, including assessments and defining standards for learning, vary considerably across the curricular landscape so that sociotechnical effects will be complicated and uneven. As we consider these topics from the sociotechnical perspective, we see how care should be taken when responding to some of the expectations for radical change or even extrapolate from a handful of existence proofs of disruption in one corner of education to other parts of this vast educational enterprise.\u00a0 The experience of other innovations in education that had transformative potential is too sobering. All too often, the result has been like the personal computer: less fundamental change in practice from technologies than a longer process of assimilation and adaption into the special circumstances of education (Cuban, 2009).<\/p>\n<h2>Four Key Sociotechnical Issues for MOOCs<\/h2>\n<p>Below are four key issues that are related to MOOCs.\u00a0 Educational researchers often look immediately for some direct measure of improvement, whether better learning or greater access.\u00a0 The sociotechnical frame does not focus so much on productivity as the overall context that innovations are situated in; influenced by and influencing.\u00a0 Each of these issues then help to highlight how MOOCs can be seen in terms of some of the changes they can bring and some of the important factors that may affect their success.<\/p>\n<h3>1. MOOCs are hybrids of social activism, innovation and entrepreneurship and commerce<\/h3>\n<p>MOOCs are more than a technology to deliver content.\u00a0 They are more than new ways of assessing students to be able to scale a course and lower its cost. MOOCs transcend technical innovation.\u00a0 For the current moment at least, they represent a conflux of interests that includes social activism, innovation and entrepreneurship, and commerce. To understand MOOCs as a sociotechnical movement that has gained so much momentum in such a short amount of time we need to bring together several viewpoints that are not always considered together in contemporary educational research.<\/p>\n<p>For many, MOOCs represent a social movement to democratize education; break down barriers that have excluded underprivileged students from the best educations possible.\u00a0 It is no coincidence that MOOCs are promoted by the same funders and interests that seek to reform traditional education so that it better serves needy populations than people think it does.\u00a0 While there may be legitimate questions about how the ways these interests seek to accomplishing that mission, it is clear that for many the goal of MOOCs is to provide access to high-quality education as Stanford Professor Andrew Ng (founder of for-profit MOOC platform Coursera) expressed in April, 2013 by saying \u201cEducation should not just be for the elite. Education should be a fundamental human right\u2026Students shouldn\u2019t have to choose between tuition and groceries.\u201d\u00a0 The idea of opening up education to the masses has been one of the signature hallmarks of the MOOC movement. \u00a0It is likely one of the reasons the hype has gotten so far ahead of any reliable research.<\/p>\n<p>At the same time, the MOOC space is about more than lowering barriers.\u00a0 Those promoting MOOCs also believe in the power of innovation and entrepreneurship to help drive educational improvements.\u00a0 Along with the MOOC platforms (Coursera, EdX, udemy, etc.), there are now countless innovators developing related products to work with MOOCs.\u00a0 Many are working with venture funders, developing products, and hoping to capitalize on new opportunities.\u00a0 This movement now has a vibrant development sector is bringing a stream of new components and products to be used to support distance learning.\u00a0 Many of these new innovations have broader use so that they help create new opportunities in many educational areas in addition to MOOCs.\u00a0 Whether or not the MOOC wave continues to have the force it has today, many of the new products developed in response to the current interest in them will be available for other uses. \u00a0This innovation sector also aligns with some views of equity because many of the MOOC backers believe that the education enterprise fundamentally needs new ideas and innovation.\u00a0 This view is also not supported with research, perhaps because there have historically been very few studies of these dimensions.<\/p>\n<p>Clearly the commercial dimensions of MOOCs remain one of the most curious areas to consider. On the surface the MOOC seems to give away what universities have traditionally charged a lot of money for. They come along at a time of scarcity rather than abundance so the emergence of free education may create pressures in the higher educational landscape that has long been stable with universities traditionally providing the labor of a professoriate and their assistants to deliver the instruction. By is using technology to deliver a comparable product to many more people and by using technology to help automate the assessment of students, MOOCs can be universities using tools to cannibalize from their own foundations. However, there are other ways MOOCs can create revenue. Some MOOC providers are looking at charging for the data they collect about students.\u00a0 In these cases the providers can sell potential employers lists of students with profiles of their class performance so employers can assess not only content knowledge, but competencies such as teamwork and collaboration (Young, 2012).\u00a0 Some universities are dealing with one of the essential problems with the MOOC platform, lack of a credential, by selling a completion certificate.\u00a0 I also suspect that before long we will see MOOCs being used not so much as another way to deliver traditional university classes, but as a complementary service.\u00a0 Perhaps universities will use a MOOC to help provide a survey of a field for those considering further study, an onramp so to speak, that allows students a low cost trial at what might be a larger and more traditional commitment later on.\u00a0 This model would be similar to the \u201cfreemium\u201d software marketing where a basic version of some product is available for free, but the more advanced features require a fee.<\/p>\n<h3>2. Flipping Classes and Changes in Professional Roles\/Organizational Structures<\/h3>\n<p>We see in Chiang\u2019s experience several realignments of traditional roles and organizational structures.\u00a0 For example, his virtual office hours allow common topics to be presented through pre-packaged video and then the few exceptional questions to be handled by specific messages.\u00a0 The course Wiki can be used to help develop future materials, which then frees up instructor time for other activities.\u00a0 These uses of technologies to, as Chiang says, \u201credraw the boundaries of education\u201d are actually not specific to MOOCs or even to higher education.\u00a0 They are examples of how technology helps make malleable ways of working(Brynjolfsson &amp; Hitt, 2000; Ciborra, 1993). As sociotechnical changes occur, one of the common side effects is changes in professional roles and organizational structures. For example, personal computers and word processing did more than increase individual productivity.\u00a0 They impacted the careers of secretaries as executives and others began to write their own correspondence rather than rely on staff for that. More recently, the Internet has now through the introduction a range of communications tools made telecommuting and off-shore services a viable and healthy path in a number of fields.\u00a0 One of the most salient of these areas today is the switching of class and individual time so that lecture through recorded video becomes an individual activity and class time used for tutoring and extra help.\u00a0 This is what has been called \u201cFlipping\u201d the classroom by Sal Khan (2012) and others following him. Chiang\u2019s experience combines a MOOC with flipping, but flipping the classroom is also not specific to MOOCs, but initially attempted in the K-12 space and one of a number of blended learning models that innovators are experimenting with involving younger children (Horn &amp; Staker, 2011).<\/p>\n<p>One lesson here is that as MOOCs become established, they can be seen not just in terms of ways to teach more students with the great teachers within a traditional course structure, but as opportunities to redesign educational processes where the roles of teaching assistants and professors and others such as on-call tutoring experts, can be brought to bear as needed. As the complexes of technologies used in MOOCs mature and become more integrated they will likely be able to support even more kinds of work structures. The organizational fluidity that these technologies have brought may very well lead to reconsiderations of how certain organizational elements are designed. While Chiang and most others working with MOOCs have used them in the context of higher education, they might also be adaptable to certain K-12 and community college settings and other professional\u00a0 learning settings as well.\u00a0 If a high-school has a need to provide additional coursework in statistics, but no teacher available, a MOOC could be used as another instructional option that would be more self-directed and provide transition opportunities to secondary students. The ultimate implications of this flexibility in terms of institutional design is not yet clear as most educational organizations, including those distance universities that operate fully within a digital infrastructure, are designed according to traditional instructor-course-student constraints.<\/p>\n<p>Another possible scenario is outsourcing of MOOC functions. Based on other history with technologies (Brynjolfsson &amp; Hitt, 2000; Cibborra, 1993; Piety, 2013a), it would be likely that commercial firms will play active roles in providing MOOC elements, including media, assessments, and infrastructure.\u00a0 The outsourcing of educational support and tutoring will fit well within the MOOC model because of the use of distance technologies.<\/p>\n<h3>3. The A\/B Test and New Approaches to Research and Knowledge Creation<\/h3>\n<p>These disruptive forces are helping to open up new research and evidence possibilities.\u00a0 Technology can ease some of the traditional constraints that have been long-standing in educational research while also making more evidence collection processes possible. For example, when there are thousands of students in a class and those students are interacting with digital tools, it can be cost effective and easy to collect data in sufficient numbers to support quantitative evidence about the learning process.\u00a0 This data can be used to generate knowledge that informs the design of the program as well as for data-driven decision making within the course itself (what to reteach and to whom). For many years, educational knowledge creation\u2014<i>research<\/i>\u2014has been largely dominated by external measures and studies that attempt to understand what educational systems are producing or how things are working within them.\u00a0 Even qualitative research that includes situating the researcher within the setting is external in terms of the knowledge creation.\u00a0 In an ethnographic study the researcher will collect various forms of evidence and then afterwards will analyze, compile, and publish.\u00a0 The kinds of measures used in quantitative research\u2014assessments, surveys, outcome data\u2014are largely collected external to the event that they represent and are for the most part external to learning.<\/p>\n<p>An important factor in the determination of what settings were aligned with qualitative versus quantitative research has been the size of the participants. Studies of learning in classroom and other structured settings tended to attract more qualitative research where the small numbers of students make quantitative studies problematic, while those external to learning environments tend to be able to use larger number of students and quantitative methods, but with less detail about learning. One of the most widely discussed examples of new knowledge creation is the A\/B test.\u00a0 In A\/B testing, different approaches to some digital experience are randomly assigned to individuals and data collected about the different conditions without interrupting the natural activity of the user.\u00a0 The experience tested can be a web page sequence, design layout options, or topics in a course.<\/p>\n<p>Many of the early MOOC successes have been in subjects involving science, technology, engineering, and math (STEM) where decomposition of topics into cognitive sequences with dependencies is possible.\u00a0 A computer science course, for example, may need to teach students about different kinds of data structures and also different methods for using them. A common instructional question is which order is better.\u00a0 Understanding the structures entails understanding how to use them.\u00a0 Understanding the tools for using the structures requires some knowledge of the structures put the tools into context.\u00a0 When instruction occurs in small discrete elements, this kind of testing can be fast and extremely cost effective.\u00a0 However, it is not clear which parts of the curriculum this kind of testing can be applied to.\u00a0 As the scale of the cohesive curricular unit expands, for example with problem-based learning and with many areas of literacy and the humanities, it is not clear how well A\/B testing can be used.<\/p>\n<p>At the same time that MOOCs have been ascending in popularity, so has another topic: <i>learning analytics.<\/i> Learning analytics is an emergent field that draws on many different disciplines to help understand both micro processes of learning that occur within a specific setting and also the systemic processes as students move through different educational engagements (Piety, Behrens, &amp; Pea, 2013).\u00a0 These kinds of analytics, however, raise new questions about evidence, inferences and rigor as their technology-rich environments bring many new kinds of artifacts for analysis.\u00a0 There are clickstreams and logs of student activity, peer grading and discussion threads, as well as projects and other forms of student work that can be analyzed both within and external to classes.\u00a0 Furthermore, many of the more familiar kinds of educational evidentiary artifacts such as tests and test items can be confused with similar tasks created by less formal methods. These tests and items may look externally to be of the same kind as those developed professionally for accountability purposes, but those looks may be deceiving.\u00a0 When MOOCs and similar offerings are assembled using Open Educational Resources (OERs), these \u201cnew\u201d kinds of \u00a0assessments will be used and provide useful information, but with very different design affordances in terms of how the data from them can be compared to other evidence forms and what kinds of inferences they support (Behrens, Mislevy, Piety, and DiCerbo, 2013).<\/p>\n<h3>4. Importance of Information Architecture<\/h3>\n<p>Throughout Chiang\u2019s discussion, including the ten opportunities and challenges he poses are issues that can be described as <i>information architecture<\/i>. Information architecture is a broad and fairly recent term that can mean both the graphical displays of data, often related to visualization technologies, and the behind-the-scenes ways in which data systems (including assessment systems) are constructed (Piety and Palincsar, 2006). Information architecture can include the coding and classification schemes (Bowker and Starr, 2000) that are seen in the organization of learning standards (called by Chiang \u201contologies of knowledge\u201d) and the various groupings and structures of instructional units and materials.\u00a0 Even the \u201cMOOC supports\u201d that Chiang describes in his experience\u2014the online office hours space and the Wiki, that provide information channels for his students\u2014can be seen as information architectural components. As MOOCs increase the scale of individual offerings from the tens or few hundred students common in traditional higher-education settings to thousands and in some cases tens of thousands of students, these information architectures will play an increasing role in the structuring of the activity system. While a traditional course has a design that is loosely reflected in a professor\u2019s syllabus, that design can be flexible; the professor can alter instruction according to his or her perceptions and the dialog that emerges with the students. In a MOOC-sized event, much of that communication will be facilitated digitally and so the digital artifacts and their architecture will be important for ensuring there are comparable feedback systems and the ability for students to personalize their experience in the way that students can when their access to the instructor is more direct.\u00a0 It is not clear whether better information architectures could improve the low completion rates in many MOOCs (3-5%), but I suggest this is an area to consider for research.<\/p>\n<h2>Beyond Learning: Opportunities and Challenges for NSF, IES, and Education Schools<\/h2>\n<p>The wave of innovation that includes MOOCs is moving fast and presenting new, often unproven, ideas.\u00a0 In a few cases these innovations are here today and merit some response. There are both opportunities and challenges of MOOCs and other recent innovations for NSF, the Institute of Education Sciences, and education schools that receive much of their direction and funding from these agencies.\u00a0 There will be new research questions.\u00a0 Some of these will and should focus on the processes of learning in these settings; on productivity differences between MOOCs and more traditional structures and on how different kinds of students participate and thrive in these different circumstances. These studies will likely align with prior research into learning in general and learning using distance education. It seems that at least some of these questions could be taken up by communities in the future academic landscape. What those communities will look like and how these innovations will relate to more traditional lines of inquiry is unclear.<\/p>\n<p>The sociotechnical perspective augments, rather than replaces, other lines of inquiry bout MOOCs.\u00a0 It is still important to look at this new medium for its impact on learning and teaching.\u00a0 Different domains of education, for example STEM and humanities educators, will look at MOOCs for what they mean for specific learning goals.\u00a0 These investigations, however, are likely to not focus on some of the larger and historical dimensions of MOOCs and to explore the directions they may take in the future.\u00a0 The sociotechnical perspective helps with these questions.\u00a0 While MOOCs themselves are new, MOOCs are not the first big thing to come along with great promise for education and many questions about impact.\u00a0 Back in 1980, there was computer programming in the book <i>Mindstorms <\/i>by Papert.\u00a0 There were similar promises from the Internet, social networking, and handheld devices.\u00a0 None of these innovations reshaped education fundamentally.\u00a0 All, however, had some impact in the complex field of education.\u00a0 In a few cases they have added new capabilities that have not existed before.\u00a0 From a sociotechnical perspective, each had an impact on practice and its use in education has been shaped by practice in ways consistent with what has occurred in other sociotechnical revolutions (Bikjer, 1993; Carayannis, Gonzales, and Wetter, 2003).\u00a0 We can see in MOOCs similarities in terms of the early promise as well as some reasons that promise may be difficult to realize, many of which Chiang discusses.\u00a0 Will MOOCs impact learning processes?\u00a0 They probably will in some ways. Will they impact the knowledge base about learning?\u00a0 They probably will help in that area as well as in our understanding of how some technologies mediate in the learning process (Pea, 1993).\u00a0 However, if history is a guide, these advances will likely be modest compared to the totality of the educational enterprise.\u00a0 I believe the terrain of public education is too vast and complex for any single innovation to reshape it and that waves of related sociotechnical developments, of which MOOCs are a recent example, will be dispersed by the complexity of this educational world in the same ways that waves on a shore find new trajectories with rocks and docks and turf. The sociotechnical lens helps us to see how these waves are related to other waves and how their impact is often related and cumulative.\u00a0 In an period of heightened expectation, this lens can help to provide a balanced perspective, not only for reflection, but for action.\u00a0 By looking not only at the single hallmark innovation, but the interrelated sociotechnical progress, this view will help funders and educators to be able to anticipate some future directions this process can take so that they can do more than study, they can innovate and take advantage of the real opportunities that this movement provides.<\/p>\n<p>Another consideration that needs to be considered is the nature of evidence that can support scientific claims. Their rapid growth and change makes them important and also difficult to study as a program or intervention that should be associated with specific learning or educational outcomes.\u00a0 While, these outcomes measures are important, in many cases the innovations are changing as they are being studied and the organizations are also changing as they adapt to these new innovations so that the a clear assessment of impact may take some time to be realized. In responding to these innovations, academics and researchers may need to consider not only their impact on productivity and completion rates as Chiang suggests, but also the broader <i>systemic effects<\/i> (Piety, 2013b).\u00a0 These systemic effects can include adoption rates, relationship between MOOCs and student future matriculation, cost effectiveness of MOOC approaches, equity issues, and other changes that the MOOCs are implicated in.<\/p>\n<p>There is much about MOOCs to be learned and much about learning that could influence the design of different MOOC approaches.\u00a0 The sociotechnical lens provides a way to conceptualize the innovation broadly to see how its evolution is related to other important developments in technologies that are impacting practices that involve learning. This understanding will be helpful in shaping responses in higher education, human capital pipeline development, and K-12 learning.<\/p>\n<h1>References<\/h1>\n<p>&nbsp;<\/p>\n<p>Bijker, W., E. (1995). Of bicycles, bakelites, and bulbs: toward a theory of sociotechnical change. Cambridge, MA: MIT Press.<\/p>\n<p>Bonk, C. J., Graham, C. R., Cross, J., &amp; Moore, M. G. (2006). The handbook of blended learning: Global perspectives, local designs. Pfeiffer San Francisco, CA.<\/p>\n<p>Brynjolfsson, E. (2012)\u00a0 Erik Brynjolfsson on Big Data: A revolution in decision-making improves productivity &#8211; MIT Sloan Experts. Retrieved November 26, 2012, from http:\/\/mitsloanexperts.mit.edu\/erik-brynjolfsson-on-big-data-a-revolution-in-decision-making-improves-productivity\/<\/p>\n<p>Brynjolfsson, E., &amp; Hitt, L. (2000). Beyond Computation: Information Technology, Organizational Transformation and Business Performance. The Journal of Economic Perspectives, 14(4), 23-48.<\/p>\n<p>Carayannis, E. G., Gonzalez, E., &amp; Wetter, J. (2003). The nature and dynamics of discontinuous and disruptive innovations from a learning and knowledge management perspective. In L. V. Shavinina (Ed.), International handbook on innovation (pp.116\u2013138). Elsevier Press.<\/p>\n<p>Chiang, M. (2013) <i>Innovating Education with MOOC\/FLIP<\/i>. Working paper. Princeton University EDGE Lab. <a href=\"http:\/\/scenic.princeton.edu\/files\/MOOC_FLIP_thoughts.pdf\">http:\/\/scenic.princeton.edu\/files\/MOOC_FLIP_thoughts.pdf<\/a>.<\/p>\n<p>Christensen, C. M., Baumann, H., Ruggles, R., &amp; Sadtler, T. M. (2006). Disruptive innovation for social change. Harvard business review, 84(12), 94<\/p>\n<p>Christensen, C., Horn, M., &amp; Johnson, C. (2008). Disrupting class: How disruptive innovation will change the way the world learns. New York: McGraw-Hill.<\/p>\n<p>Ciborra, C. (1993). Teams, Markets, and Systems: Business Innovation and Information Technology. Cambridge: Cambridge University Press.<\/p>\n<p>Coursera (2013) Coursera Announces Professional Development Courses to Facilitate Lifelong Learning for Teachers. Retrieved May 21, 2013, from <a href=\"http:\/\/blog.coursera.org\/post\/49331574337\/coursera-announces-professional-development-courses-to\">http:\/\/blog.coursera.org\/post\/49331574337\/coursera-announces-professional-development-courses-to<\/a><\/p>\n<p>Cuban, L., &amp; Cuban, L. (2009). <i>Oversold and underused: Computers in the classroom.<\/i> Harvard University Press.<\/p>\n<p>DiCerbo, K. E., &amp; Behrens, J. T. (2012). Implications of the Digital Ocean on Current and Future Assessment. In R. Lissitz &amp; H. Jiao (Eds.) Computers and Their Impact on State Assessment: Recent History and Predictions for the Future. (pp. 273-306). Charlotte, NC: Information Age Publishing.<\/p>\n<p>Horn, M. and Staker, H. (2011). The Rise of K-12 Blended learning.\u00a0\u00a0 Innosight Institute. Retrieved on September, 7, 2011 (http:\/\/www.innosightinstitute.org\/innosight\/wp-content\/uploads\/2011\/05\/The-Rise-of-K-12-Blended-Learning.pdf).<\/p>\n<p>Khan, S. (2012). The One World Schoolhouse: Education Reimagined. Twelve.<\/p>\n<p>LaValle, S., Lesser, E., Shockley, R., M, H., &amp;\u00a0 Kruschwitz, N. (2010, December 21). Big Data, Analytics and the Path From Insights to Value. MIT Sloan Management Review.<\/p>\n<p>Lohr, S. (2012, February 11). Big Data&#8217;s Impact in the World. The New York Times. Retrieved from http:\/\/www.nytimes.com\/2012\/02\/12\/sunday-review\/big-datas-impact-in-the-world.html<\/p>\n<p>Means, B., Toyama, Y., Murphy, R., Bakia, M., &amp; Jones, K. (2010). Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies. U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, Evaluation of Washington, D.C.<\/p>\n<p>National Science Foundation. (2007). NSF\u2019s cyberinfrastructure vision for 21st century discovery. Retrieved from <a href=\"http:\/\/www.nsf.gov\/pubs\/2007\/nsf0728\/nsf0728_6.pdf\">http:\/\/www.nsf.gov\/pubs\/2007\/nsf0728\/nsf0728_6.pdf<\/a><\/p>\n<p>Pea, R.; Childress, S.; and Yowell, C. (2012).\u00a0 The Demand for Education Data Scientists working paper by H-Star Institute. Stanford, CA. Stanford University.<\/p>\n<p>Pea, R. D. (1993). Practices of distributed intelligence and designs for education. Distributed cognitions: Psychological and educational considerations, 47-87.<\/p>\n<p>Piety, P. Behrens, J. and Pea, R. (2013). <i>Educational Data Sciences\u00a0 and Interpretive Skills.<\/i> Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA.<\/p>\n<p>Piety, P.J. (2013a). <i>Assessing the Educational Data Movement<\/i>. Teachers College Press, New York.<\/p>\n<p>Piety, P.J. (2013b). <i>Beyond Productivity: Considering Systemic Effects alongside Outcome Measures.\u00a0 <\/i>Comments for IES\/NSF Common Standards for Research and Development Proposals.<\/p>\n<p>Rogers, E. M. (1995). <i>Diffusion of innovations<\/i> (4th ed.). New York: Free Press.<\/p>\n<p>Young, J. (2012) Providers of Free MOOC&#8217;s Now Charge Employers for Access to Student Data. Chronicle of Higher Education (December 4, 2012).<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction In a recent essay titled Innovating Education with MOOC\/FLIP, Princeton University professor Mung Chiang thoughtfully discusses Massive Online Open Courses (MOOCs) and some related innovations that are presenting educators with dilemmas and bringing to the surface pressures for possible<span class=\"ellipsis\">&hellip;<\/span><\/p>\n<div class=\"read-more\"><a href=\"http:\/\/edinfoconnections.com\/?p=468\">Read more &#8250;<\/a><\/div>\n<p><!-- end of .read-more --><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"_links":{"self":[{"href":"http:\/\/edinfoconnections.com\/index.php?rest_route=\/wp\/v2\/posts\/468"}],"collection":[{"href":"http:\/\/edinfoconnections.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/edinfoconnections.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/edinfoconnections.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/edinfoconnections.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=468"}],"version-history":[{"count":0,"href":"http:\/\/edinfoconnections.com\/index.php?rest_route=\/wp\/v2\/posts\/468\/revisions"}],"wp:attachment":[{"href":"http:\/\/edinfoconnections.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=468"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/edinfoconnections.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=468"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/edinfoconnections.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=468"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}