Syndicate content

SRI

Contextualizing lessons from the use of the Khan Academy

Michael Trucano's picture
at first glance things look the same, but upon further reflection there are some potentially important differences to consider
at first glance things look the same,
but upon further reflection there are some
potentially important differences to consider

Based on my interactions with educational policymakers, and those who advise them, it seems to be a truth (almost) universally acknowledged that there is insufficient research into models of educational technology use, the impact of such use, and related costs. This is not to say that there is *no* useful research into the use of technology in education around the world, of course. Online tools like ERIC and SSRN can help you find some useful studies; the popular press and the blogosphere increasingly reference such work (sometimes even in ways where you can actually track down the referenced studies!); there are of course a lot of academic, industry and professional journals dedicated to the subject; and a healthy amount of 'grey' literature circulates informally (including stuff commissioned by companies that is never formally published). Firms also circulate 'white papers' touting the 'impact' of their products and services, something which I tend to place into its own separate category, given the commercial and marketing imperatives that often animate such work. That said, just because a lot of 'research' is produced doesn't mean that such research is helpful to meet the practical information demands of educational policymakers, planners and educators.

Even if you *are* of the opinion that there is indeed a lot of useful, policy- and practice-relevant research out there related to the use of technologies in education, the fact remains that most of our collective knowledgebase has been constructed as a result of studying and attempting to learn from experiences in 'highly developed' (OECD) countries. While there is always danger when trying to draw generalized lessons from a research study that examines a specific context, it would seem reasonable to me that the difficulties when looking to draw lessons from experiences in Quebec that might be relevant to Kansas or Canberra pale in comparison to those when trying to extend such lessons to policymakers making decisions which will affect students and teachers in places like Quito or Kampala -- let alone rural Cambodia.

Thankfully, there are a number of promising moves afoot which hope to direct more energy and resources to investigate issues and circumstances of relevance to those exploring the use of ICTs in education in middle and low income learning environments and contexts around the world. (As such efforts kick off, and especially as related studies emerge, we would hope to feature them on the EduTech blog.) Until we start to see results from these sorts of efforts, however, the practical reality is that, in most cases, policymakers in middle and low income countries who wish to draw lessons 'from the research' in order to inform their policymaking related to potential educational technology initiatives will continue to try to contextualize results from research in higher income countries in the attempt to divine what lessons (if any) might be relevant to their own circumstances, even in places where contexts for use and typical use cases may be quite different.

---

A previous EduTech blog post, "Evaluating the Khan Academy", explores some of the lessons that have emerged as a result of research by SRI into the use of Khan Academy in a number of schools in Northern California. For those who don't know it: The Khan Academy is a widely known and much celebrated educational website which features thousands of short video tutorials on educational topics, as well as linked sets of over 100,000 practice problems and a 'personal learning dashboard'. Policymakers from a number of countries have approached the World Bank for specific advice and guidance on how they might make use of Khan Academy resources within their schools, and during the course of related conversations (some of which were catalyzed by a talk given by SRI's Robert Murphy at the World Bank earlier this year, which was discussed in a subsequent blog post) we have passed along and discussed the Research on the Use of Khan Academy in Schools published by SRI -- as well as a report that appeared a few months ago from EDC that looked at the use of Khan Academy in Chile. While individual lessons and insights drawn from an analysis of the use of Khan Academy content at sites examined in the SRI and EDC reports may or may not be more broadly generalizable to other contexts, examining the various usage models documented and explored in those studies may help raise *questions* that might be relevant to educators and policymakers who work in other contexts.

Indeed:

What are some useful questions that policymakers in middle and low income countries might ask as they attempt to contextualize insights from the use of the Khan Academy in California and Chile as part of their efforts to investigate and plan for the use of digital learning resources from the Khan Academy (and from other sources as well) within their own education systems?

Evaluating the Khan Academy

Michael Trucano's picture
this is fascinating, but wouldn't it be better online?
this is fascinating, but wouldn't it be better online?
Over the past five years, there has perhaps been no educational technology initiative that has been more celebrated around the world than the Khan Academy. Born of efforts by one man to provide tutoring help for his niece at a distance, in 2006 the Khan Academy became an NGO providing short video tutorials on YouTube for students. It is now a multi-million dollar non-profit enterprise, reaching over ten million students a month in both after-school and in-school settings around the world with a combination of offerings, including over 100,000 exercise problems, over 5,000 short videos on YouTube, and an online 'personalized learning dashboard'. Large scale efforts to translate Khan Academy into scores of languages are underway, with over 1000 learning items currently available in eleven languages (including French, Xhosa, Bangla, Turkish, Urdu, Portuguese, Arabic and Spanish). Founder Sal Khan's related TED video ("Let's use video to reinvent education") has been viewed over three million times, and the Khan Academy has been the leading example cited in support of a movement to 'flip the classroom', with video lectures viewed at home while teachers assist students doing their 'homework' in class.

As efforts to distribute low cost computing devices and connectivity to schools pick up steam in developing countries around the world, many ministries of education are systematically thinking about the large scale use of digital educational content for the first time. Given that many countries have already spent, are spending, or soon plan to spend large amounts of money on computer hardware, they are often less willing or able to consider large scale purchases of digital learning materials -- at least until they get a better handle on what works, what doesn't and what they really need. In some cases this phenomenon is consistent with one of the ten 'worst practices' in ICT use in education which have been previously discussed on the EduTech blog: "Think about educational content only after you have rolled out your hardware". Whether or not considerations of digital learning materials are happening 'too early' or 'too late', it is of course encouraging that they are now happening within many ministries of education.

As arguably the world's highest profile digital educational content offering in the world -- and free at that! -- with materials in scores of languages, it is perhaps not surprising that many ministries of education are proposing to use Khan Academy content in their schools.

The promise and potential for using materials from Khan Academy (and other groups as well) is often pretty clear. Less is known about the actual practice of using digital educational content in schools in middle and low income countries in systematic ways.
 
What do we know about how Khan Academy is actually being used in practice, and how might this knowledge be useful or relevant to educational policymakers in developing countries?

Videogames and Learning

Michael Trucano's picture
playing to learn?
playing to learn?

Not a week goes by where I don't receive an unsolicited email from a company touting the benefits of its new 'educational videogame'. Indeed, just last week I opened my inbox to find two separate emails proclaiming how two different mobile gaming apps were destined to "transform learning!!!" Now, in a lot of the cases, I must confess that I am not always sure why something is an 'educational game', and not just a 'game' (although if I am in a difficult mood, I might offer that in too many instances an 'educational game' is 'a game that really isn't much fun'). That said, there is no denying that videogames are big business around the world. So -- increasingly -- is education. Even most people who fear that potential negative effects of some (or even most) videogames on young people would, at the same time, acknowledge the promise and potential for videogames to offer enriching learning experiences. The history of the introduction of educational technologies is in many ways long on promise and potential, however, and short on actual evidence of how they impact learning in tangible and fundamental ways.

Much is made of the potential for ICTs to be used to promote more personalized learning experiences through the introduction of various types of ICT-enabled assessment systems. For me, it has long seemed like the most powerful real-time learning assessment engines have been found in videogames, where actions (or inactions) are often met with near instantaneous responses, to which the player is then challenged to respond in turn. This feedback loop -- taking an action, being presented with information as a result, having to synthesize and analyze this information and doing something as a result -- might meet some people's definition of 'learning'. A good videogame engages its users so strongly that they are willing to fail, and fail, and fail again, until they learn enough from this failure that they can proceed with the game. Even where educational software is not explicitly labeled as a 'game', designers are increasingly introducing game-like elements (badges, achievement bonuses, scoring systems) as a way to promote user (learner? player?) engagement as part of a process known as 'gamification'.

The use of videogames for educational purposes, or at least in educational contexts, is far from an OECD or U.S. phenomenon. Whether I am visiting a school computer lab after hours in central Russia, an Internet cafe filled with students in Indonesia or standing behind some schoolgirls carrying phones between classes in Tanzania, 'educational' videogames seem to be nearly everywhere. Past posts on the EduTech blog have profiled things like the use of video games on mobile phones to promote literacy in rural India and EVOKE, an online game for students across Africa which the World Bank helped sponsor a few years ago. When I speak with young software entrepreneurs in Nairobi or Accra or Manila, they often talk excitedly about the latest educational game they are developing (for markets local and distant).

Do educational games 'work'?
Are they 'effective'?
And if so: How can they be used in schools?

Questions such as these are of increasing interest to scholars. Given both their potential for learning, and how aggressively videogames are being marketed to many education systems, they should be of increasing interest to educational policymakers as well. Some recent research brings us a little closer to a time when we can answer some of them.