Worst practice in ICT use in education
This page in:
In business and in international development circles, much is made about the potential for 'learning from best practice'. Considerations of the use of educational technologies offer no exception to this impulse. That said, 'best practice' in the education sector is often a rather elusive concept (at best! some informed observers would say it is actually dangerous). The term 'good practice' may be more useful, for in many (if not most) cases and places, learning from and adapting 'good' practices is often much more practical -- and more likely to lead to success. Given that many initiatives seem immune to learning from either 'best' or even 'good' practice in other places or contexts, it may be most practical to recommend 'lots of practice', as there appears to be a natural learning curve that accompanies large scale adoption of ICTs in the education sector in many countries -- even if this means 'repeating the mistakes' of others.
But do we really need to repeat the mistakes of others? If adopting 'best practice' is fraught with difficulties, and 'good practice' often noted but ignored, perhaps it is useful instead to look at 'worst practice'. The good news is that, in the area of ICT use in education, there appears to be a good deal of agreement about what this is!
Here's a list of some of what I consider to be the preeminent 'worst practices' related to the large scale use of ICTs in education in developing countries, based on first hand observation over the past dozen or so years. I have omitted names (please feel free to fill them in yourself). The criterion I used for selection was simple: The given worst practice was easily observable in multiple prominent initiatives, with (one fears) a high likelihood of re-occurrence, in the same or other places. In no particular order:
1. Dump hardware in schools, hope for magic to happen
This is, in many cases, the classic example of worst practice in ICT use in education. Unfortunately, it shows no sign of disappearing soon, and is the precursor in many ways to the other worst practices on this list. "If we supply it they will learn": Maybe in some cases this is true, for a very small minority of exceptional students and teachers, but this simplistic approach is often at the root of failure of many educational technology initiatives.
2. Design for OECD learning environments, implement elsewhere
With the best of intentions, and often 'assisted' by vendors, many groups (including many governments) have sought to simply transfer ICT-related models and practices from classrooms in industrialized countries to less developed education systems in other parts of the world. Sometimes this works, but unfortunately many places roll out programs and products that have at their core sets of assumptions (reliable electricity and connectivity, well-trained teachers, sufficient available time-on-task, highly literate students, space to implement student-centric pedagogies, relevant content, a variety of cultural norms, etc.) that do not correspond with local realities. The result is often (and not unsurprisingly) not very good.
3. Think about educational content only after you have rolled out your hardware
Deploying lots of computer infrastructure in schools is expensive (and complicated). So expensive, in fact, that many critical complementary investments (in training, in tech support, in content, etc.) are 'postponed' until a later date. Sometimes this is a calculated bureaucratic maneuver/risk -- the thinking is that, once the hardware is in place, the need for content will be more clear, and it will be easier to make the case for related funding at that time) -- and other times this is simply a lack of good planning. But it is a fact that, in many places, only once computers are in place and a certain level of basic ICT literacy is imparted to teachers and students is the rather basic question asked: What are we going to do with all of this stuff?
Related to this ...
4. Assume you can just import content from somewhere else
Some places recognize the need for quality educational content from the start, but assume they can simply import it from somewhere else. In addition to obvious potential cultural issues, the successful integration of content developed elsewhere into daily teaching and learning practices is inhibited by a lack of clear understanding by teachers of the relevance of such materials to the required curricula. Much effort typically needs to be expended to map this content to explicit objectives and activities in the local curricula. (And of course: Teacher training helps too!)
5. Don't monitor, don't evaluate
This should be self-evident. That said, there are only a handful of really credible, rigorous impact evaluation studies done of educational technology initiatives in developing countries. Most evaluation work focuses on (perceptions of) changes in attitudes as the result of the use of educational technologies, and the success (or lack of success) in meeting various simple metrics (number of computers installed, number of teachers trained, etc.). Such information is important, of course, but it is hardly sufficient. What is the impact of ICT use in education? If we don't evaluate potential answers to this question, rigorously and credibly, all we are left with is well-intentioned guesswork and marketing dross.
6. Make a big bet on an unproven technology (especially one based on a closed/proprietary standard) or single vendor, don't plan for how to avoid 'lock-in
Let's acknowledge that the speed of technological changes almost always outpaces the ability of educational planners to keep up. In response, some policymakers seek to get 'ahead of the curve' by placing large bets on new, largely unproven technologies in an effort to 'leapfrog' what is happening in other education systems. In other cases, education systems effectively outsource most of the capacity to manage activities in this area to a vendor or other third party. There are potentially valid reasons to pursue such courses of action in some cases, but they are inherently very risky, especially if clear plans are not made on how to 'exit' such decisions and relationships.
7. Don't think about (or acknowledge) total cost of ownership/operation issues or calculations
What does ICT use in education cost? Some people would have you believe it is only the initial cost of hardware. Businesses have long known that this is not the case, but many education policymakers seem not to have grasped (or willfully ignore) this fundamental issue. We know that "total cost of ownership or operation" (TCO) is often underestimated, sometimes grossly, when calculating costs of ICT in education initiatives in developing countries. Estimates of initial costs to purchase equipment to overall costs over time vary widely; typically they lie between 10-25% of total cost. That said, there is a dearth of reliable data, and useful tools, to help guide education decisionmakers in their assessments of the true costs of educational technology initiatives.
8. Assume away equity issues
One compelling justification for large-scale investments in the use of ICTs in education is that they can help address equity issues related to the 'digital divide'. That said, introduction of ICT in schools often exacerbate various entrenched inequities in education systems (urban-rural, rich-poor, boy-girl, linguistic and cultural divides, special needs students -- the list is long). Things can be done to mitigate such challenges, and indeed pro-equity approaches of utilizing ICTs are possible, but they don't happen without careful proactive attention to this issue.
9. Don't train your teachers (nor your school headmasters, for that matter)
If there is one clear lesson from the introduction of educational technologies in schools around the world, it is that teacher training is critical to the success of such initiatives. Outreach to teachers, through both regular technical and pedagogical support and on-going professional development, should be seen as cornerstones of any large ICT investment in schools. And: Targeted outreach to school principals is often crucial if teachers are to have the necessary freedom to take advantage of new opportunities offered through the use of ICTs.
10. ___
[I thought I would leave #10 blank as an acknowledgement that there are many additional worst practices that merit mention, but I have run out of space. Do feel free to submit your candidates below.]
For those who work in educational technology, none of these will be new. For many others new to this topic, the items on this list may appear to be so obvious that they need not even be mentioned. Even if indeed they are 'obvious', that unfortunately hasn't stopped them from occurring (and re-occurring) around the world with depressing regularity.
Other recent items related to ICT/education of potential interest from the World Bank:
1. infoDev has released a preview of the key findings [pdf] from its upcoming Survey of ICT & Education in India & South Asia.
2. You can now RSVP to [email protected] for the following two events at the World Bank in DC in May (both start at 12:30pm and are open to the public):
- 5 May: Interactive Radio Instruction: Increasing Access to Quality Early Childhood Development Programs in Developing Countries
- 13 May: Uruguay's Plan Ceibal: What happens when every student and teacher has their own free laptop
Your post is thought provoking but for me more so if I think of replacing ICT with "pencil and paper" (at least for points 1,3, 4, 5 and 9). The questions around educational design, professional development, monitoring and evaluation etc are all just as relevant no matter what level of technology (or lack of it) is involved.
Just an FYI That the link above to the event at the World Bank on 13 May open to the public has now been updated (previous link was incorrect):
Uruguay's Plan Ceibal: What happens when every student and teacher has their own free laptop
http://go.worldbank.org/1421LKAEL0
Hi Paul,
Thanks for your comment. I agree wholeheartedly that these sorts of things should be relevant no matter what 'technology' we are talking about.
If you haven't seen it, you might be interested in the cheeky 'Adventures in Pencil Integration' blog, http://pencilintegration.blogspot.com, which makes this point as well (albeit in a different way).
-Mike
fyi For those interested in the post from the World Bank Development Marketplace blog to which Homira refers above ('No toilets, but they have Bluetooth'), you can find it at http://blogs.worldbank.org/dmblog/node/745
Hi All,
This is interesting...
"We've had several high-tech ventures in Afghanistan that blissfully said they would just rely on power generators to juice up the laptops, but generators require fuel, and people living in absolute poverty have to make tough choices about resource use. The U.S. government's Leapfrog laptop initiative in Afghanistan is another really interesting example to look at."
Not quite sure which one you are meaning....
But if it's One Laptop Per Child Afghanistan - I can assure you we've learned from that list. This article is great because we can look at what can make programs succeed and what can make them fail. What we're doing is making an entire interactive curriculum that double to triples time on task (410hrs/yr in Afghanistan vs. 800hrs OECD average), provides structured feedback and access to further learning resources using a digital library that doesn't need Internet access.
We actually created the what might be the first sustainably human powered computer system (foot pedal as you go based). OLPC is also incredibly low power making it pretty cost effective to use solar etc. Generators can be interesting experiments, but maintaining them nevermind fuel is a nightmare.
As what works can be massively culturally sensitive one needs a couple phases of piloting and research. Firstly the pilot has to explore what models can work. Then the validation phase has to kick in to test the intervention designed against a conventional control case with similar financial resources.
http://www.paiwastoon.af/index.php?option=com_content&view=article&id=4…
-Mike
I agree with every word, with a single big caveat: context.
While these are all common mistakes (or, in design language "anti-patterns", each and every one of them can boast a particular set of circumstances where they are the right thing to do.
But that's an exception, which bounds the rule. On the other hand, in the case of "best practices" it is often the exceptions that are presented as rules. Even a broken watch is right twice a day, and a watch that is 5 minutes early helps you catch the bus. No solution is always good or always bad - a solution is effective per certain problems in a certain context.
I propose a change of language. Instead of best and worst practices, lets talk of design narratives and design patterns.
A design narrative tells the story of a problem and how it was solved (or attempted to be solved) in full detail, including all the contextual information, misfires, recoveries and lessons learnt. A design pattern identifies recurring triplets of problem, context and solution across narratives.
Thank you for an interesting post. You're right, these guidelines on what not to do should be common sense, but strangely enough you often come across (some of) them. In South Africa, we seem to avoid these problems, not because we have a better understanding of implementing tech in the wrong way, but because our digital divide keeps the hardware and skills out of the areas where we would be prone to make the mistakes.
Mike,
In my humble opinion, points 3 and 4 about content are the most important. My understanding of education - in Africa at least - is that most of the main players in education underestimate how hard it is to create relevant local content.
The African Virtual School is all about content, both digital and in print. Our flagship book - The African Virtual School Maths Book is available on Amazon and as a digital subscription. All we need now is to convince Institutions, Agencies and African Ministries of Education to subscribe to our service :).
I like this approach of working from the "worst practice". We go to so much trouble identifying best and good practices that are often in the eyes of the beholder or very contextual and difficult to implement.
What an interesting way to change the nature of the conversation ... well beyond ICT.
Hi ! There is mad rush for pouring technology hardware in the schools. There have be no philosophies, no strategies, no aim or objectives even. School full of technology is a fashion. The real use of technology in learning and other associated areas need to be spelt out. Otherwise, the MNCs will sell and we will dump them in schools.
Prakash Deo
Bhopal, India
Dear Mike,
Yet another very relevant and candid post from you. Many of the lessons, or 'worst' practices that you describe can also be easily transferred over to the use of ICT in the health sector, so all nine points are applicable to us too. If I had to add a number 10 to your list, it would be to consider the energy needs of the equipment necessary to keep operating, and to consider the environmental sustainability of the energy source. We've had several high-tech ventures in Afghanistan that blissfully said they would just rely on power generators to juice up the laptops, but generators require fuel, and people living in absolute poverty have to make tough choices about resource use. The U.S. government's Leapfrog laptop initiative in Afghanistan is another really interesting example to look at.
I also like your emphasis on what DOESN'T work because in being honest about our mistakes or mis-steps, we really learn. I also just read Aleem Walji's post on the need for public toilets in S.Africa versus the availability of Bluetooth technology. I must admit I didn't quite get the connection between the two, but his larger point about how technology can empower people, but not if their most basic needs are not met (i.e., safe water, sanitation, security). Keep up the good work dude.
10. Do not embed your project into your social context: educational community, family, neighbourhood, etc.
I agree with all of them, but especially with #5 and #9.
Hi Mike,
Thanks for your comments.
FYI The Jhai Foundation also has a pedal-powered computing solution, which was (is?) being used in Laos. Some more info, if you haven't seen it:
http://fonly.typepad.com/fonlyblog/2006/10/what_is_the_jha.html
http://www1.voanews.com/learningenglish/home/a-23-2008-09-27-voa1-83137…
Hi,
Actually found out about Jhai after posting our version on OLPC News. We did an experiment to find out how much power children could actually sustainably generate - 11N against gravity was OK. Converting that at 50-60% which would be the best that one could hope for one would get about 6.5W. That would be enough for the XO but not even a normal netbook...
Excellently clear and frank post Mike. But isn't it truly shocking that it just seems to go on and on? The thing that most interests me in this situation is how to comes about, and my view is that one of the most damaging aspects is the way that the purse holders on educational ICT projects, never function as true consumers.
I've worked on many where the key customer regards the project as their next step on a career ladder and doesn't invest any effort or energy in implementation, but everything in the relationships they establish during procurement. They inevitably leave their role the moment the contract is signed and there is absolutely no continuity but worse, no effective consumer/supplier relationship to ensure that what is delivered does its job.
Add to that the skilled exploitation by marketing departments of "thought leaders" and other techno-zealots, and you have the perfect recipe for an ICT project with no educational value, substance or meaning.
Excellent entry Mike. Coming from a developing country, I can say that almost everything you says in this article holds true for most ineffective ICT projects. Some of these entail each other, for example 2 and 4. What I see as a big problem is lack of long-term and comprehensive thinking in the design and planning phase. I would also second the idea of Joe Nutt about the incentives of people managing ICT educational projects. How can we design an incentive system so as to reduce corruption and to focus on long-term goals?
Hi Michael, congratulations for the excellent summary.
I definitely subscribe to all 9 "worst practices" and I have a couple of candidates for the list based on my experience on the field. Here they go 10 and 11. I'd love to hear the groups thoughts?
10. Don't think about the logistics and deployment challenges of ICT in a school system
Large ICT deployments in education, where hundreds, possibly thousands of schools are equipped with new servers and desktops/netbooks constitute a massive logistics undertaking whose importance tends to be totally minimized. If a Business was to equip some thousands of branch offices with new servers and provide individual laptops/desktops for tens or hundreds of thousands of employees, that would certainly involve massively detailed planning and coordinated teams in the field along with round-the-clock support help-desks and finally appropriate tools and systems to manage installation, activation, authorization, access management, permission control, remote activity monitoring. However, in Education, such deployments tend to be taken in a pretty much ad-hoc manner - essentially dumping hardware at the school - and the fact of the matter is that the specific context in schools makes this even more challenging for a school then to a business. The consequences can be disastrous. We all heard the horror stories of computers that got to schools but were still waiting to be unpacked after a year, where no one knows exactly where and if they came live, who's using it, etc. and where substantial portions of the equipment deployed was never properly configured and when their configuration was broken they just got put in a pile of non-working equipment for months.
11. Assume that schools will have either a resident IT person or a IT-savvy teacher that will "make things work" locally.
It is well known that the vast majority of the schools, don't have internal IT staff and their knowledge of IT is limited. On the other hand, the ICT infrastructure that goes into the schools is becoming increasingly complex (and increasingly resembles the infrastructure of a small and medium business). Including networking equipment to access the internet, security, firewalls, Wifi, email, web, content management, so on and so forth. So when the time comes to deploy this infrastructure at the school, if the expectation is that a local IT-savvy teacher will find its way through this complex set of services, then we're headed for disaster. The solution must be to provide interfaces for the Teacher's that do not require any IT expertise and are as easy-to-use and error-proof as possible and guarantee that all aspects of the Solution can be fully remotely managed online by a team of IT Experts. Otherwise, it is almost certain that local people won't be able to do the troubleshooting and management of issues that will inevitably occur and systems will soon be working deficiently or even unavailable.
There's also a couple more topics that I find specially relevant for emerging markets and were partially listed in point 2, such as higher than feasible energy consumption levels, cooling needs that require even more energy, connectivity issues that require the ability to work offline, etc... which are taken as given in the some developed countries but can be huge show-stoppers in emerging ones.
Your list is very good, and thanks for this, Mike. But, since you invited additional comments, I will add two :
Plan to adapt educational objectives to the features and possibilities of the hardware instead of the other way around. We have all seen attractive gizmos which someone thinks MUST have some educational uses and therefore imports into classrooms for vanity, market or other non-educational purposes. The first question to ask is of course "what is the problem we are trying to solve?" followed by "what technology would help solve it?".
Assume teachers are the problem and that technology can help make classrooms teacher-proof. This is more or less identical to no. 9. Not involving teachers right from the beginning is a sure way to get them less rather than more motivated. Technology can disempower teachers pretty quickly if lack of training or lack of coherence with the programmes and pedagogy they are used to makes them feel incompetent.
No point in elaborating too much on these
Congratulations to OLPC for getting 8 out of 9. Nearly perfect marks!
Addin to the list, If you don't plan for your HW and SW maintenance costs as well as ensure you have access (not neceesarily a stock) of critical spare aprts, you may end up with a pile of non working machines in less than 5 years. In emerging markets unline mature markets a machine or PC have a significanlty longer life time.
A list of ten practices is offered, and the discussion taking place in the Comments section is just as valuable as the list itself. Here are some highlights from the post:
* Content - Not thinking about content before hardware is installed and assuming that content can simply transfer throughout different contexts
* Cost - Forgetting that cost is an ongoing issue, not just the initial purchase of hardware
* Training - If teachers are expected to use ICT in their classrooms, they must receive both technical and pedagogical training.
An excellent blog which deserves wide circulation. I suggest as a candidate for No 10: Appoint a CEO who is an expert on ICT and knows little or nothing about Education.
Hi Mike,
Enjoyed reading your 9 worst practices--as well as the additional ones posted by readers. These certainly reflect the issues that we in iEARN have encountered over the past 22 years. One additional one is to assume that the introduction of ICT in the classroom and particularly the connectivity of the C (Communications) can be introduced without resulting in major changes in how teachers teach.
In addition to pedagogy and technology training, attention needs also to be paid to the methods and style of teaching--something often much harder to change than dealing with new tools and content. Becoming the facilitator of student learning, rather than the expected font of wisdom, is not an easy transition to make for an educator, yet the power of the technology puts learning directly into student hands.
Thanks for the observations and the ideas they stimulated.
Ed
In no particular order:
1. Disregard any research on how people learn
2. Assume that whatever works for your children will work for the poor
3. Adopt your professor's favorite theories on how to produce learning outcomes.
4. Summarize the theory in cute succinct messages (learning = AAA= autonomy accountability assessment)
5. Demand evidence from everyone else about alternatives, leave your own evidence-free.
6. Hire the best communication strategist to put out your messages.
7. Ensure its success by inviting only like-minded people to events that discuss it.
8. Disregard negative peer reviews and evaluations; ridicule their lessons, suggest mean-spirited ulterior motives.
9. Get donors to finance it for billions of dollars.
10. Hope for magic to happen.
Great post, and here are a couple more issues that I have seen many times:
Don't consult the users. They usually have little idea of what they want and even if they do know they won't tell you; so the best thing to do is decide what you think they need and then go ahead and give it to them. In the long run they will thank you.
Enjoy the development, and the roll out will just happen. A large number of projects never get beyond the initial funding because they haven't planned how it will be implemented (support, training, change management) from the start.
I think the 10th example in your worst practice list could be implementing a 'one-size fits all' approach. In this approach, a school or a district or Local Authority deploys a particular piece of software or hardware in the belief that it is appropriate for every pupil, student or teacher.
In this approach, the versatility and flexibility of ICT to meet individual needs is overlooked. The ability of the technology to support the range of educational approaches is ignored in favour of a 'common' approach. As a result, the quality of ICT in a school becomes poorer rather than enriching the education for all.
Thank you for this blog and all the comments. I got asked last year by a community in Lesotho to find laptops for their children. I was ignorant of most of the issues involved when I founded the nonprofit organization Laptops to Lesotho, but thanks to your comments and other similar forums, our group has been working very hard to avoid all these pitfalls. We've taken a year and a half to work closely with the local community and school officials before even bringing the computers into the community. We're working out formal contracts so everybody knows exactly what to expect and what their responsibilities are, including the students, parents, teachers, headmasters, community leaders, and our organization. The very first thing we plan to do when we do deploy the laptops is to send the teachers to training. We've lined up IT support for them, and we're working to acquire culturally appropriate lessons and to customize activities and materials to fit the local curriculum. Our biggest problem in trying to find grant-makers willing to pay for the less flashy items like support services, training, shipping, solar power, and evaluations. Until donors are willing to chip in to pay for these things, projects will continue to short-change them.
Thanks for your comments. For the most part, I don't think we are in disagreement here (even if we are saying things in different ways). One of the ideas that animate this post is that we shouldn't always be looking for 'perfection' (or at least 'best practices') as the lens through which look for models and insights to help guide our own planning and implementation behavior. 'Lots of practice' -- and the ability to learn and improve as a result! -- seems to be what separates initiatives that do well from those that don't, no matter their starting points.
In a perfect world you would be right on the money. But those in Education and EdTech exist in a far from perfect world. Education is an organic process of making lessons turn into learning, not a process of following fill-in-the-blank forms and lamenated planbooks.
EdTech closely follows that organic identity. You may beg for all the government money you want to set up a computerized learning environment but the one thing you can be certain of is that 'if' they respond, it will be too little, too late, and it will be a 'take it or leave it' load of hardware and software.
Today's schools with advanced integrated systems of learning would never have gotten off the ground if they had initially refused the offer when it came, been 'overly selective' in vendor relationships and squeamish about taking on the challenge of self-learning in networking and other EdTech subject matter. Educators make lessons work with what is available reinforcing core values regardless of whether the Sun was shining or not.
A nice column for those with the luxury of options but it forgets that the humanbeing's attitude is the key ingredient of any educational experience - computerized or not.
I agree completely. For us IT-people in healtchare i think #6 is specially relevant, at least here in Sweden. Closed standards, compatibility issues and outrageous prices of journalsystems are a big problem for our organization and have forced us to keep using software originally written for Windows 3.1. Something which causes a lot of trouble for me as an administrator.
An open universal standard for medical journals is something we must agree on to stop the ridiculous situation we are in now.
Thank you, Mike, for this great article. I can't agree more with the points you've made.
In my view, the main problem when implementing new policies concerning education is that sometimes those who design these programs are completely isolated from the reality to be faced where these plans have to be implemented. As you pointed out in the first worst practice, "Dump hardware in schools, hope for magic to happen", this happens a lot especially when the technical side doesn't take into account the importance of pedagogy or learning. At the same time, I would add that sometimes this happens because there are hidden incentives for governments, institutions or companies ($) to deploy a specific technology quickly. In any team or new EdTech implementation, I think that the most important component has to be the pedagogical view, otherwise, the developed technology is may be useless (point 3?).
I would also emphasize the worst practice of "Design for OECD learning environments, implement elsewhere". Having just completed an Ed. M. in the US, in one of the most important universities in the world and coming from South America, I was impressed how often all the statements that were made and "best practices" mentioned, only applied to the US, and because of that, it was assumed that those were the way to do things. I also wonder why "in the developed world" we usually don't want to learn from experiences from developing countries where sometimes they get incredible results with few resources and in much harder conditions.