National online testing as an indicator for edtech readiness?

A recent post on the EduTech blog explored the potential usefulness of an 'edtech readiness index', a new tool to help inform conversations and decisions currently occurring in many ministries of education around the world about how to best (most impactfully, most equitably, most strategically) roll-out and utilize new technologies across an education system.

When creating an index of most any sort, there is a natural tension regarding how many indicators or types of data to include. Chose too few, and you're likely to exclude a lot of important stuff. The more you include, however, the more complicated things get. This is especially true if there are few or no existing sources of the data you would like to utilize, and so you need to build related data collection mechanisms from scratch. (When it comes to edtech-related topics, there is very little rigorously collected, globally comparable data available in most countries. That's the stark reality.) 

If I were forced to choose one indicator that would suggest to me that an education system was equipped to utilize educational technologies at scale to (as the slogan often proclaims) 'transform teaching and learning', it would be whether or not the country implements national online testing of some sort.

Let me be clear: I am not arguing that implementing national online testing should be a goal for an education system, nor that it should be a priority, nor that it is even useful. Whether or not a tool or approach 'works' in a particular environment will depend on a lot of factors, a good number of them specific to a given operating context. I leave it to others to enter into and comment on related debates, which can quickly become ideological and polarized, in other forums. (Those interested and willing to do so have no shortage of opportunities and outlets elsewhere on the Internet to do so, whether they are looking to have their own perspectives validated, pick a fight with someone with whom they disagree, or engage in conversation that falls somewhere in between.)

Rather, I am observing that, in order for national online testing to occur, a lot of things need to be in place, and working, and that such things can provide the foundation upon which other large scale edtech projects can be built.

When I speak with big companies that sell edtech-related products and services who are exploring moving into new markets, they often tell me that they closely monitor which countries are making the move to online testing. I am not talking about the (small handful of international) companies in the 'assessments business' -- of course they track such stuff. Rather, I am talking about most everyone else. I have heard countless tech firms remark that, 'If a country is implementing online testing at a national scale, then we know they are serious about edtech. Once this particular genie is out of the bottle, it can be difficult to stuff it back in.' (Usually such a pronouncement is accompanied by a rather large smile.)

In order to institute online testing, a country needs to buy lots of hardware. It needs to improve connectivity. It needs a lot more than just this, however. Lots of countries already buy lots of hardware, spend lots on connectivity, and create and purchase digital learning resources of various types. Alongside such things, lots of money is often spent on complementary training and professional development (of teachers, of technical staff, of administrators -- and programs need to be put in place to ensure that students have a basic level of 'digital skills'). When this happens at a national scale, this often represents big business, and can potentially mean big opportunities for many businesses.

This isn't all that happens, though, for national online testing to take place:

• Enabling policies, regulations and guidelines (and sometimes even laws) need to be formulated and/or updated -- and implemented.
• National examination bodies usually need to be reformed, and the skills of their staff upgraded, so that they can successfully oversee, manage and advise on the details of implementing national online testing. Local academic and research communities need to be engaged to help with this effort.
• Lots of different stakeholder groups need to be consulted and brought into the process. Cases need to be made, for example, to parents, to teachers, to civil service employees, as well as to other stakeholder and interest groups.
• Within government itself, policymakers and technical staff need to be able to make informed decisions and to manage related budgeting and procurement processes (a scenario often complicated by the fact that they may be negotiating with large international testing companies that know a lot more about the topic than they do).
• New approaches to data security and privacy often need to be formulated and implemented. (Testing them before they are put in place is usually a good idea as well.)
• There are huge logistical challenges in pulling off national online testing -- in scheduling, in use of school facilities and infrastructure, in ensuring adequate connectivity and electricity, in providing local technical support, in preventing cheating, in identifying and correcting problems quickly, and in coordinating all of the multitude of moving parts in a way so that, at a given moment in time, everything is in sync.
• (etc. etc. etc.)

Even if national online tests are not 'high stakes' for students, where the results determine whether a student passes on to the next level of schooling or teachers or administrators are given bonuses or fired (worthy topics for investigation and debate that I'll leave aside for the purposes of this essay), such an effort tends to be high stakes politically for government. And, at least in the short term -- disruptive for most everyone.

Let's be honest (and this is the dirty little secret of many educational technology initiatives in many countries, even quite high-profile ones):

In many schools in most countries, even where education systems have spent *lots* of money on tablets, connectivity, digital textbooks, and/or training, ensuring that everything 'works' is often not a terribly high priority. Yes, it would be great if students actually used the devices that were purchased for them. It'd be great if teachers put to use in the classroom some of what they learning in their technology training classes. It'd be nice if teachers and students actually accessed and used the national education portal put up at great expense by the education ministry or the digital content or applications that a local educational authority made available. But if none of this happens, life will go on, much as it had previously, for better or for worse. No doubt many people and groups will complain (about the expense, about wasted opportunities), and their criticisms will likely be quite valid, but such complaints and criticisms may be unlikely to incentivise practical action in a way that leads to real change. This is an unfortunate but observable reality around much of the world.

But if national online testing rolls out and it doesn't work -- that's another story! Students (and teachers, and school administrators) get demoralized. Parents and community groups mobilize. Vendors may be in breach of provisions in their contracts. Education and government officials face pressure in the media, and politically. As such, there are often big incentives in place to ensure that, when it comes to national online testing, things work.

And when they don't work -- and in practice they often don't work all that well the first time this sort of thing is attempted (and often not the second or fifth time either) -- there are usually compelling incentives in place for groups to work together to fix what's wrong, at least eventually.

When things (finally) do work, the preconditions are in place for an education system to do other things (perhaps more useful and impactful things) with educational technologies at scale.

Doing something at scale with technology in which *all* students and *all* schools participate is, as they like to say in Silicon Valley, non-trivial. National testing initiatives must, by their very nature, include *all students*. Unlike national edtech efforts that aim to connect all schools to the Internet but only end up connecting 95% or so (with the excluded 5% typically including schools in some of the poorest or most geographically isolated communities), national tests need to happen everywhere, and anything less than 100% coverage is a failure. This means that whatever approach or 'solution' is conceived and implemented needs to work for everyone -- not just for the kids in the nice schools in the big cities where edtech initiatives usually are first attempted and considered to be 'successful'.

That's why, when I'm asked about what's the one indicator I look for that might suggest whether a country is prepared to utilize educational technologies at scale across its entire education system to impact teaching and learning, I usually look to see if national online testing is occurring.

If it is not: Well, it's certainly possibly that an effort to introduce digital textbooks for use by every learner will be successful, or that a project to help schools offer more 'personalized learning' experiences to all of their students will meet its objectives. By no means does the existence of national online testing in a country ensure that all of the prerequisites to do something else with technology in education at scale are in place. But I expect that a lot of them might be.

When conceiving and embarking on something that's quite ambitious, the devil is often to be found in the details. When it comes to pulling off online testing at a national scale, there are lots of details to attend to, big and small. Get them right, and many of the ingredients and tools will be in place to cook up something else. That's no guarantee that the eventual result will be healthy, or applauded, or even edible or digestible, of course. No one said this stuff is easy. But pull this sort of thing off, and it's much more likely that you'll be able to succeed at whatever you would like to appear next on your plate. 

(At minimum, you'll at least have a better sense of what you might be getting yourself into!)

 
You may also be interested in the following posts on the EduTech blog:

• The case for a new Global Edtech Readiness Index
• The introduction of large scale computer adaptive testing in Georgia
• Crowdsourcing, collaborative learning or cheating?

Note: The image used at the top of this blog post ("there are so many ingredients ... what if I could only choose one?") comes via Pixabay member fxxu and is used according to the related Pixabay license.