Syndicate content

Contextualizing lessons from the use of the Khan Academy

Michael Trucano's picture
at first glance things look the same, but upon further reflection there are some potentially important differences to consider
at first glance things look the same,
but upon further reflection there are some
potentially important differences to consider

Based on my interactions with educational policymakers, and those who advise them, it seems to be a truth (almost) universally acknowledged that there is insufficient research into models of educational technology use, the impact of such use, and related costs. This is not to say that there is *no* useful research into the use of technology in education around the world, of course. Online tools like ERIC and SSRN can help you find some useful studies; the popular press and the blogosphere increasingly reference such work (sometimes even in ways where you can actually track down the referenced studies!); there are of course a lot of academic, industry and professional journals dedicated to the subject; and a healthy amount of 'grey' literature circulates informally (including stuff commissioned by companies that is never formally published). Firms also circulate 'white papers' touting the 'impact' of their products and services, something which I tend to place into its own separate category, given the commercial and marketing imperatives that often animate such work. That said, just because a lot of 'research' is produced doesn't mean that such research is helpful to meet the practical information demands of educational policymakers, planners and educators.

Even if you *are* of the opinion that there is indeed a lot of useful, policy- and practice-relevant research out there related to the use of technologies in education, the fact remains that most of our collective knowledgebase has been constructed as a result of studying and attempting to learn from experiences in 'highly developed' (OECD) countries. While there is always danger when trying to draw generalized lessons from a research study that examines a specific context, it would seem reasonable to me that the difficulties when looking to draw lessons from experiences in Quebec that might be relevant to Kansas or Canberra pale in comparison to those when trying to extend such lessons to policymakers making decisions which will affect students and teachers in places like Quito or Kampala -- let alone rural Cambodia.

Thankfully, there are a number of promising moves afoot which hope to direct more energy and resources to investigate issues and circumstances of relevance to those exploring the use of ICTs in education in middle and low income learning environments and contexts around the world. (As such efforts kick off, and especially as related studies emerge, we would hope to feature them on the EduTech blog.) Until we start to see results from these sorts of efforts, however, the practical reality is that, in most cases, policymakers in middle and low income countries who wish to draw lessons 'from the research' in order to inform their policymaking related to potential educational technology initiatives will continue to try to contextualize results from research in higher income countries in the attempt to divine what lessons (if any) might be relevant to their own circumstances, even in places where contexts for use and typical use cases may be quite different.

---

A previous EduTech blog post, "Evaluating the Khan Academy", explores some of the lessons that have emerged as a result of research by SRI into the use of Khan Academy in a number of schools in Northern California. For those who don't know it: The Khan Academy is a widely known and much celebrated educational website which features thousands of short video tutorials on educational topics, as well as linked sets of over 100,000 practice problems and a 'personal learning dashboard'. Policymakers from a number of countries have approached the World Bank for specific advice and guidance on how they might make use of Khan Academy resources within their schools, and during the course of related conversations (some of which were catalyzed by a talk given by SRI's Robert Murphy at the World Bank earlier this year, which was discussed in a subsequent blog post) we have passed along and discussed the Research on the Use of Khan Academy in Schools published by SRI -- as well as a report that appeared a few months ago from EDC that looked at the use of Khan Academy in Chile. While individual lessons and insights drawn from an analysis of the use of Khan Academy content at sites examined in the SRI and EDC reports may or may not be more broadly generalizable to other contexts, examining the various usage models documented and explored in those studies may help raise *questions* that might be relevant to educators and policymakers who work in other contexts.

Indeed:

What are some useful questions that policymakers in middle and low income countries might ask as they attempt to contextualize insights from the use of the Khan Academy in California and Chile as part of their efforts to investigate and plan for the use of digital learning resources from the Khan Academy (and from other sources as well) within their own education systems?

Bad practices in mobile learning

Michael Trucano's picture
something doesn't seem quite right with this particular implementation ...
something doesn't seem quite right
with this particular implementation ...

The World Bank's EduTech blog explores issues related to the use of information and communication technologies (computers, laptops, tablets, the Internet, ...) to benefit education in middle and low income countries around the world. While I tend to view, with a fair degree of skepticism, many of the statistics which purport to document just how many people have visited a particular web site, it seems that the EduTech blog was recently visited by its one millionth reader. When viewing the mass of blog posts in their entirety, together with our visitor logs and other relevant data, it is quite clear that BY FAR the single most popular post remains one I did over four years ago on 'worst practice in ICT use in education'. What was relevant back in 2010 appears still to be quite relevant today.

(This isn't always the case: If memory serves, I quickly drafted and published that particular blog post because I was having trouble completing one 'Exploring the Use of Second Life in Education' -- I'm guessing that the half-life for *that* one, had it even been finalized and published, would have been pretty short!)

Recent news articles -- whether reporting that the one tablet per child project in Thailand 'has been scrapped' or the decision of the school district in Hoboken, New Jersey (USA) to 'throw away all its laptops' -- suggest that debris continues to pile up on the landscape of 'failed' attempts to use new technologies effectively in education in various ways. The Franco-Czech writer Milan Kundera has a short story called "Let the Old Dead Make Room for the Young Dead". Sometimes I feel like this title could be adapted for use in an introductory essay to a book documenting many of the unfortunate 'educational technology deployments' that have been irresistable fodder for politicians and headline writers alike (and clickbait for folks on Twitter) over the past decade.

And yet .... just because because we continue to hear variations on a sadly familiar theme, I don't know that the best response is to admit defeat, throw up our hands, throw everything away and go back to the 'good old days'. Learners would not be terribly well served if educational planners in 2014 simply decided to emulate the impulses and actions of Silesian weavers back in 1844 and smash all the machines in reaction to the spread of new technologies. Attempting to stuff this particular genie back in the bottle isn't only impractical: I would hazard a guess that it is well nigh impossible.

The recent article on the Hoboken experience labels it a 'failed experiment'. Personally, I am not sure that this label fits in this particular case. In an experiment, it seems to me that you are usually trying to learn something. This rather large purchase of technology seems to me like yet another solution in search of a problem that no one bothered to actually tried to define in any meaningful way. I suspect that, at a fundamental level, the problem wasn't (really) with the technology. In other words: It seems more like human failure to me.

A short note to the new philanthropist looking to support education and technology initiatives in the developing world

Michael Trucano's picture
I have some important decisions in front of me
I have some important decisions
in front of me

I get contacted from time to time by 'new' philanthropists looking to do something positive and productive with their wealth. Usually this is someone who has made a lot of money in the technology industry and who is now starting her or his own family foundation.

Chronologically speaking, many of these people are closer to what one would consider the age at which someone starts a career than the age at which one 'retires'. In other words: They are often rather young.

Based on my (admittedly limited) experience, many of these sorts of folks are often firm believers in the value of education (even if they themselves dropped out of formal schooling early to focus on writing code), in the transformative potential of technology (something which has profoundly and positively impacted their lives personally, offering them opportunities and riches they could never have dreamed of) and in the desire to do something globally (the person may be an immigrant her/himself, be married to an immigrant, have worked with lots of people from different countries or cultures, and/or just have done a lot of international travel).

Last year, for example, I was contacted by someone (writing on behalf of someone else) who wished to (I am slightly paraphrasing here) "explore innovative ways that technology can be harnessed to help overcome longstanding challenges in education around the world". (As for how such folks find me, they usually say: I stumbled across the EduTech blog.)

Given that I have been approached a number of times in a similar sort of way quite recently, and that I serve on a number of externally advisory boards where this sort of thing is discussed, I thought I'd share this scenario here, as well as little bit about some of the things I sometimes say in response, in case it might be of interest to anyone else:

Let's say I had the equivalent of a few million U.S. dollars
available to do something innovative at the intersection of
education and technology somewhere in the 'developing world'.

If it works out, a lot more money could potentially be used
to support activities further, and more systematically,
over a longer period of time.

If it doesn't work out -- well, that would not be great, of course,
but I am willing to take some risks.

 I want to be innovative,
and would really like to do something
that no one else is doing.

What should my new foundation do, and how should we do it?


I must confess that, whenever I am asked these sorts of questions, I find it to be a rather exciting, and perhaps even a little terrifying, scenario. (Often times such adjectives are not mutually exclusive!) When presented with a blank canvas of this sort, where and how does one start to paint?

Promising uses of technology in education in poor, rural and isolated communities around the world

Michael Trucano's picture
don’t worry: your solutions -- and possibly your salvation – have finally arrived!
don’t worry: your solution (salvation?)
has finally arrived!

One persistent challenge for educational policymakers and planners related to the potential use of informational and communication technologies (ICTs) in remote, low income communities around the world is that most products, services, usage models, expertise, and research related to ICT use in education come from high-income contexts and environments.

One consequence is that technology-enabled 'solutions' are imported and 'made to fit' into what are often much more challenging environments. When they don't work, or where they are too expensive to be replicated at any scale, this is taken as 'evidence' that ICT use in education in such places is irrelevant -- and possibly irresponsible.

That said, lessons are being learned as a result of emerging practices, both good and bad, in the use of ICTs in education in low resource, poor, rural and isolated communities in Africa, Asia, Latin America and the Pacific that may be useful to help guide the planning and implementation of educational technology initiatives in such environments. (It may even turn out that the technological innovations that emerge from such places many have a wider relevance …. but that is a topic for another discussion.)

Products like the BRCK (a connectivity device designed and prototyped in Nairobi, Kenya by many of the people behind Ushahidi to better address user needs in places where electricity and internet connections are, for lack of a better word, ‘problematic’) and MobiStation (a solar-powered 'classroom in a suitcase' which features a projector and lots of off-line educational content developed by UNICEF Uganda) remain notable exceptions to the lamentable reality that, for the most part, ‘solutions’ touted for use in schools in e.g. rural Africa, or in isolated communities in the Andes, are designed elsewhere, with little understanding of the practical day-to-day realities and contexts in which such technologies are to be used. Many people who have lived and worked in such environments are quite familiar with well-meaning but comparatively high cost efforts often informed more by the marketing imperatives embedded in many corporate social responsibility efforts than by notions of cost-effectiveness and sustainability over time or the results of user-centered design exercises.

The Development and Evolution of National Educational Technology Agencies Over Time

Michael Trucano's picture
a different sort of life cycle
a different sort of life cycle

As part of our work advising such groups over the years, we have observed that national ICT/education agencies -- the organizations found in many countries which serve as the focal groups coordinating large scale efforts to introduce, use and support new technologies in schools -- pass through a general ‘life-cycle’ over the course of their existence, with five semi-distinct stages of development.

Each stage may bring with it a new set of functional responsibilities and mandates, different staffing (including leadership) and budgeting requirements, and entail varied levels of oversight and relationships with other groups, causing organizational structures to adapt, and be adapted, over time.

This life-cycle hypothesis has been proposed as a simple tool to help people who play critical roles in the planning for large scale national educational technology initiatives to develop an understanding of how their organizations may compare with other organizations doing similar sorts of things in other parts of the world, and how they might expect that their organizations may evolve and change over time. Such an evolution can potentially have a profound impact on a variety of key decisions that policymakers may have to make related to funding, staffing, functions and coordination with a variety of key stakeholder groups over time. There is no right or wrong answer as to whether it is 'good' or 'bad' that a particular organization finds itself at one of these five identified stages. There also appears to be no hard and fast rule about how long individual institutions may stay at a particular stage in the life cycle. Some organizations may move quickly from one stage to another, others may stay in a particular stage for many years, even (potentially) decades. Most organizations observed around the world, especially those in middle and low income countries, find themselves today somewhere between the stages of 'childhood' and 'adolescence', with a heavy focus on the technology itself (buying it, rolling it out, supporting and maintaining it) and less of a focus on trying to integrate the technology into standard or transformed teaching and learning practices. While it is worth noting that this technology focus is not necessarily bad (or good) -- judgments of this sort are presumably more useful when made relative to certain specific contexts, and not in the abstract -- it is usually true that this focus is a direct consequence of the views of policy makers about how technology can and should be used in education.

Why Establish a National Educational Technology Agency?

Michael Trucano's picture
OK, you go this way, we'll go that way ... no, wait a minute, that isn't working ... maybe we need some formal organization here ...
OK, you go this way, we'll go that way ...
no, wait a minute, that isn't working ...
maybe we need some formal organization here ...

In most countries around the world, a single institution is core to the implementation of national initiatives related to the use of new technologies ('ICTs') in education. Whether we are talking about large scale rollouts of things like tablets or laptops, or educational computing efforts of the more 'traditional' variety, a single organization often serves as a focal point for many related efforts to introduce, support, maintain direct, coordinate, fund, manage and/or evaluate national efforts to utilize information and communications technologies (ICTs) in innovative -- and, if we are honest with ourselves, perhaps not so innovative -- ways in schools.

A few years ago, the World Bank, in partnership with the government of Korea, convened a meeting in Seoul to bring together the heads of many of these sorts of organizations to share experiences about what has worked, what hasn't, what people wish they had done differently, and what new challenges might lie ahead.

It turns out that this topic was of very immediate relevance in a number of countries which were considering starting up a 'national ICT/education agency', for lack of a better term, but were searching about for useful models and lessons that might help them in their efforts. We'll publish some related analytical work later this year, including a set of ten cases studies documenting efforts in this regard around the world.

As we finalize this work, and in case it might be of relevance to anyone, we thought it might be useful share some of the varied answers we are finding to a question that many countries have asked themselves in the recent past, and which many more countries are considering right now:

Why, and how, might a country decide to establish
a single organization dedicated to the use of ICTs in education?

Evaluating the Khan Academy

Michael Trucano's picture
this is fascinating, but wouldn't it be better online?
this is fascinating, but wouldn't it be better online?
Over the past five years, there has perhaps been no educational technology initiative that has been more celebrated around the world than the Khan Academy. Born of efforts by one man to provide tutoring help for his niece at a distance, in 2006 the Khan Academy became an NGO providing short video tutorials on YouTube for students. It is now a multi-million dollar non-profit enterprise, reaching over ten million students a month in both after-school and in-school settings around the world with a combination of offerings, including over 100,000 exercise problems, over 5,000 short videos on YouTube, and an online 'personalized learning dashboard'. Large scale efforts to translate Khan Academy into scores of languages are underway, with over 1000 learning items currently available in eleven languages (including French, Xhosa, Bangla, Turkish, Urdu, Portuguese, Arabic and Spanish). Founder Sal Khan's related TED video ("Let's use video to reinvent education") has been viewed over three million times, and the Khan Academy has been the leading example cited in support of a movement to 'flip the classroom', with video lectures viewed at home while teachers assist students doing their 'homework' in class.

As efforts to distribute low cost computing devices and connectivity to schools pick up steam in developing countries around the world, many ministries of education are systematically thinking about the large scale use of digital educational content for the first time. Given that many countries have already spent, are spending, or soon plan to spend large amounts of money on computer hardware, they are often less willing or able to consider large scale purchases of digital learning materials -- at least until they get a better handle on what works, what doesn't and what they really need. In some cases this phenomenon is consistent with one of the ten 'worst practices' in ICT use in education which have been previously discussed on the EduTech blog: "Think about educational content only after you have rolled out your hardware". Whether or not considerations of digital learning materials are happening 'too early' or 'too late', it is of course encouraging that they are now happening within many ministries of education.

As arguably the world's highest profile digital educational content offering in the world -- and free at that! -- with materials in scores of languages, it is perhaps not surprising that many ministries of education are proposing to use Khan Academy content in their schools.

The promise and potential for using materials from Khan Academy (and other groups as well) is often pretty clear. Less is known about the actual practice of using digital educational content in schools in middle and low income countries in systematic ways.
 
What do we know about how Khan Academy is actually being used in practice, and how might this knowledge be useful or relevant to educational policymakers in developing countries?

Sachet educational publishing in a digital and mobile world

Michael Trucano's picture
I'll take one packet of Pythagorean Theorem and one packet of verb conjugations, please
I'll take one packet of Pythagorean Theorem
and one packet of verb conjugations, please

In the course of my job at the World Bank helping ministries of education in middle and low income countries think about, analyze, plan for, implement and evaluate issues, ideas and projects at the intersection of the worlds of technology and education, I spend a fair amount of time considering issues related to the digital publishing of educational materials. The World Bank has over the years funded the purchase of lots of textbooks around the world and we maintain active dialogues with scores of education ministries, helping to provide related advice and technical assistance.

In many countries, especially poor ones where disposable income is very low and where there is not an established culture of leisure reading, the educational publishing industry is sometimes largely indistinguishable from the publishing industry as a whole, and government purchases of textbooks (and/or government directives about which textbooks families or schools should purchase, where such things are not centrally procured) have huge, often determining influence on the direction of the publishing industry in general. In order to better understand how all of this impacts educational publishing, I talk regularly with lots of 'traditional' educational publishers, big and small, both international and local. I also talk a lot with technology companies who do things that look a lot like educational publishing to me, or who provide the tools and services to enable and support related activities.

Last year I participated in two fascinating events a few weeks apart, the EdTech Industry Summit in San Francisco and a symposium convened by the International Publishers Association (IPA) at the London Book Fair. (I was lucky enough to be at the IPA symposium this year as well.) At these meetings, the agendas and items discussed were largely the same, but were often approached from quite different perspectives. For the sake of argument here -- and I admit I am greatly oversimplifying things by making this characterization -- EdTech Summit participants were mainly 'tech companies', while the London Book Fair event was mainly attended by 'traditional publishers'. (I concede that such distinctions are increasingly difficult, and less useful, to make as time goes on; in my opinion all publishers are technology companies these days, whether they self-identify as such or not and/or whether outsiders see them that way.)

At both events, a data point that was quoted quite often was that '1% of national education budgets around the world are devoted to the purchasing of textbooks and other learning materials'. While I have never been able to find this assertion supported by hard data, I have heard it expressed so many times over the years by people who work in or around the educational publishing industry that I have taken it as almost 'conventional wisdom'. Whether this figure is actually .5% or 2% or 4% (or whatever), what has struck me when in conversation with many vendors is that many traditional publishers have, in the face of the steady rise in many countries around the world of large scale purchases of laptops and tablets for students and teachers, worried that technology purchases are eating into traditional budgets for the purchases of textbooks.

As one traditional publisher put it to me at the London event in 2013, "we need to figure out how to protect this 1% so that it is not tapped to buy iPads". Contrast this with a statement made to me by an enthusiastic founder of an edtech start-up in San Francisco, who said that the goal of firms like his was to "eventually capture 20% of education budgets" by transforming the way education is delivered as a result of the use of new technologies. Whether or not such figures are accurate, they for me exemplify a difference in perspective and ambition that is consistent with many stereotypical characterizations of brash young tech entrepreneurs in their hoodies (and/or khakis) versus the tweedy old-school booksellers whose business model that I have been told on many occasions -- especially by those not that business -- was one for "dinosaurs".

(While conceding that the business models for selling books will have to change rather radically going forward, a concession to which no educational publisher I know would object in the age of e-readers and hypertext, of apps and APIs, I am fairly confident that extinction rates for edtech startups will remain much greater than that of book publishers for the foreseeable future, although in the end I wouldn't be too surprised if the most successful 'tech' firms doing business in this space end up buying up a lot of the 'publishers' -- some of whom will themselves be buying and merging with 'tech' firms along the way.)

If we accept the premise that educational publishing industry in the 'developed' countries of Europe and North America are being, and will continue to be, increasingly radically disrupted -- a contention with which I expect few people would disagree -- what might this mean for business models for educational publishing in less developed, 'poor' countries? Will the related business models from OECD contexts simply, and eventually, be transplanted to middle and low income countries? Or: Might some new business models for digital educational publishing emerge from less developed countries, based on specific local contexts and consumer demands in an increasingly digital -- and mobile -- age?

Surveying ICT use in education in Asia

Michael Trucano's picture
we're not all uniform in our use of ICTs
we're not all uniform in our use of ICTs

If you've ever been involved in discussions about current uses of technology in education -- and, given that you are currently reading a post on the World Bank's EduTech blog, it's probably safe to assume that you have -- you've probably noticed that, at some point in the back-and-forth, someone will inevitably be unable to resist talking about what's coming next. The history of technology use in education is, in part, a history of predictions about the use of technology in the future.

For the past few decades, many people around the world have almost instinctively looked toward Asia to get glimpses and insight into what the next wave of consumer technologies might look like and do, and how young people might use them. From the 'computer nerds' who frequented the Akihabara section of Tokyo in the 1980s to the young Filipinos whose affinity for SMS earned their country its designation as the 'texting capital of the world' around the turn of the century to today's designation of Indonesia as the 'social media capital of the world', the center of gravity for emerging uses of new technologies by young people has often been in the East. It is indeed no coincidence that the World Bank has co-sponsored an annual event bringing education policymakers to Seoul each fall to help discuss and plan for their country's potential uses of new technologies in schools in the future.

Of course, the stereotypically tech-savvy, mobile-phone wielding, hyper-connected youth in the big cities of East Asia, reviewing vocabulary on their smartphones while commuting on the subway or studying to the wee hours of the night on broadband connections at home, occupy one end of a very wide and diverse spectrum. Rural youth for whom the Internet is more aspiration than avocation and whose schools may not even have electricity, let alone a computer, or for whom 'computer time' means the two hours a month spent in a crowded school computer lab learning how to use a word processing program while waiting, waiting, waiting for their desperately slow Internet connection to bring up a single web page: Such young people and circumstances represent the reality of current technology use in education across Asia as well.

If we hypothesize that many future uses of technology in education might first appear in Asia, where might we want to look to get some first glimpses as what is likely to come to our own schools (wherever they may be)? If you want to know what a place might look like tomorrow, a good place to start might be by looking at what things look like there today.  With that in mind:

How and to what extent are countries across Asia currently utilizing information and communication technologies (ICTs) in their education systems?

Two recent publications from UNESCO provide much useful data and documentation to help those trying to come up with possible answers to this question.

Checking in with Portugal's big projects to support technology use in education

Michael Trucano's picture
sometimes looking inside an unfamiliar place can provide you with a new perspective on what's happening outside as well
sometimes looking inside an unfamiliar place
can provide you with a new perspective on
what's happening outside as well

As part of my job at the World Bank helping to advise governments on what works, and what doesn't, related to the use of new technologies in education around the world, especially in middle- and low-income countries, I spend a fair amount of time trying to track down information about projects -- sometimes quite large in scale and invariably described as 'innovative' in some way -- that were announced with much fanfare which received a great deal of press attention, but about which very little information is subsequently made widely available.

Most of these projects prominently featured some new type of technology gear, whether low cost laptops for students or new ways to connect people in remote places to the Internet or low-power e-reader devices. Other projects featured new software (English learning apps for phones! Free science curricula for teachers! A learning management system that enables personalized learning!). A sub-set of these projects -- the really ambitious and 'visionary' ones -- combined both hardware and software, and a variety of services to support their introduction and use.

I do this follow up for two very basic reasons:

(1) I am generally interested in learning from these sorts of projects, wherever they may be happening; and

(2) I am asked about them a lot.

These conversations generally go one of two ways:

"Whatever happened to that project in [fill in country name] -- how are things going there these days?"
"Things are proceeding [well / not so well], and a bit more slowly than originally envisioned. Here's what you need to know ..."

 or, alternatively:

"Can you give me an update on the exciting stuff that is happening with computers in schools in ___?"
"You mean the ___ project? Actually, that never actually happened."
"No, that's not true, I read that ---"
"Yes, you probably did read that. You may well have heard about it during a presentation by [insert name of vendor] as well. But I assure you: I talk regularly with [the ministry of education / companies / NGOs / researchers] there: Nothing actually happened there related to this stuff in the past, and nothing is happening there related to this stuff now. Will something happen there in the future? Undoubtedly something will ... perhaps even something as potentially 'transformative' as was promised ... although whether it happens in the way it was originally marketed or advertised: Your guess is as good as mine."

In retrospect, the rather short half-life of an unfortunate number of such aborted projects can largely be measured not by things actually implemented 'on the ground', but rather by PowerPoint presentations and press releases. (A rather charitable characterization of what happened in some such cases, but one that is not always or necessarily more accurate, might be that people were 'overly optimistic' or that someone or some group 'was simply ahead of her/their time'. Technology folks sometimes just dismiss such efforts as 'vaporware'.)

When it comes to educational technology projects, most of the press attention tends to come when new initiatives of these sorts are announced, with some momentum continuing on for awhile in the early days of a project, especially when, for example, kids get new tablets for the first time, an occasion that presents a nice, and ready-made, photo opportunity (not that such things are ever conceived of as photo opportunities, of course!). Then, often: Silence.

Projects that do get implemented, and last for awhile, tend eventually to be crowded out of the popular consciousness by the latest and greatest new (new!) thing -- and, when it comes to the use of technology in education, one thing can be certain:

There is always a next new (new!) thing.

(In addition to lots of press attention, the well-known One Laptop Per Child project was the subject of many papers and presentations from academics in the early days that were largely speculative -- e.g. here's what could happen -- and theoretical -- e.g. here's a pedagogical approach whose time has come. Only recently have we started to see more deliberative, rigorous academic work looking at actual implementation models, and what has happened as a result.)

---

For me, the most interesting part of the use of technology in education isn't the planning for it (although I spend a lot of time helping people who do that sort of thing) nor the evaluation of the impact of such use (I spend a lot of time on that stuff as well).

The most interesting part is implementation -- because it's so messy; because a fidelity to certain theoretical constructs and models often comes into rude collision with reality; because that's where you really *learn* about what works, and what doesn't, and what impact the whole enterprise may be having. How are kids, and teachers, actually using the stuff? What unexpected problems are people having -- and how are they being addressed? What is changing or happening that is interesting or surprising that wasn't part of the original plan, but which is potentially quite exciting?

One place where things have actually happened related to technology use in education, and where they continue to happen, at a rather large scale, is Portugal.

---

Back in 2012, we had a small event here at the World Bank that attempted to share some of the lessons learned from recent Portuguese experiences in introducing new technologies into the education sector (see Around the World with Portugal's eEscola Project and Magellan Initiative). The U.S.-based Consortium for School Networking (CoSN) released a report last month as a follow-up to a study visit to Portugal in late 2013. While written from a North American perspective and for a North American audience, "Reinventing Learning in Portugal: An Ecosystem Approach" provides a useful lens through which an outsider, regardless which continent she calls home, can start to take stock of some of the high level lessons from the ongoing Portuguese experience.

(Side note: I would also be quite interested to read a companion report at some point that focuses on what went wrong in Portugal, and what changed as a result; I am a big believer in the power and value of learning from failure.)

---

Countries interested in learning about the 'impact' of efforts to introduce and sustain the use of technologies to benefit education in Portugal might do well to understand the context of what has happened in Portugal, and the circumstances that may make it either unique, or a good comparator, to their own national circumstances.

A quick review of what's happened in Portugal:

Pages