MOOCs in Africa

MOOCs? MOOCs!

The excitement about the promise and potential of Massive Open Online Courses is white hot in many quarters. For those who aren't familiar with the phenomenon:

A MOOC is an online course, usually at the university level, offered for free over the Internet which aims for large-scale (some courses have enrolled over 100,000 students at a time), 'open' (anyone can join) participation over the Internet. 

Daphne Koller, the co-founder of Coursera, one of the largest and best-known MOOCs (the two other 'leaders in this space are Udacity and edX) stopped by the World Bank in late February to talk about what Coursera is doing, and learning. While MOOCs have enrolled students from developing countries pretty much from the start, there have not yet been many attempts to systematically include MOOCs as part of targeted education efforts in low income countries. What might such an attempt look like?

With support from the World Bank, a new pilot initiative in Tanzania is seeking to incorporate Coursera offerings as part of a broader initiative to help equip students with market-relevant IT skills. Employers in Tanzania complain that there is a mismatch of skills in the local labor market. Many jobs go unfilled because there are deficits of people with the relevant skills in the local market. There is a growing need for IT and ICT knowledge and skills necessary meet growing demand for technically skilled workers across Tanzanian corporations. For this and other reasons, Tanzania is trying to improve the quality of its higher education system. Currently a very small number of highly capable African students go abroad to meet their related educational and training needs. At the same time, Tanzania is hoping to improve its capacity to attract high caliber students from across the region to study at Tanzanian universities. What, then, to do?

The World Bank's NESAP-ICT program has been trying to help countries with such challenges. The New Economy Skills for Africa Program - ICT was launched in 2008 to support countries in Sub-Sahara Africa (SSA) in building skills for the knowledge economy. It has focused initially on supporting the development of globally benchmarked, employable skills for the Information Technology (IT) and IT Enabled Services (ITES) industry -- sectors that can create thousands of new jobs and catalyze economic and social transformation. (Here's a related World Bank publication on The Global Opportunity in IT-Based Services: Assessing and Enhancing Country Competitiveness [pdf]).

In Tanzania, NESAP-ICT is helping to support the development of what are known as SMART Knowledge Hubs, which are hoped to help form a 'backbone' of sorts the development of education in IT and a broader set of 'new economy skills' in the country. The SMART Skills (Software, Mobile Applications, Research and Technology) project began by asking about the type of IT-related skills are being sought by the local IT sector, and about the demands from students to acquire such skills that aren't being met by existing course offerings from Tanzanian institutions. The first related pilot effort is supporting the creation of a 'knowledge hub' in Dar Es Salaam, directed and coordinated by COSTECH, the Tanzania Commission for Science and Technology. It is hoped that this initial SMART Knowledge Hub will serve as a model for how to do similar things in other parts of the country.

What might MOOCs have to do with all of this?

The project in Tanzania is, together with Coursera, identifying a MOOC IT curriculum aligned with the needs of Tanzanian private sector employment tracks. The first stage of this process includes the design and development of the overall curriculum, informed by input from lecturers in IT and business in Dar Es Salaam, as well as from entrepreneurs and local businesses. The idea is to support students in various ways as they participate in MOOCs as part of their studies, in advance of the traditional recruiting season that kicks off at the start of the summer.

[-] How can students identify MOOCs that are relevant complements to their current areas of study -- and improve their future employment opportunities?

[-] How might MOOCs be formally incorporated into such formal study, with official credit given for the successful completion of a MOOC?

[-] More broadly, how can a higher education system align itself to help meet some of the immediate hiring needs of local industry, especially where local institutions may not currently have the capacity to develop and offer courses that help sufficient numbers of students develop the types of skills demanded by the labor market?

These are just some of the questions being explored as part of the SMART Skills project in 2013.

---

This is the second in an occasional series of new posts (here's the first one) looking at some emerging activities across Africa that are exploring new ways to utilize ICTs to help meet challenges in the education sector.

It is also the first of three EduTech blog posts about that will be published in quick succession on the topic of MOOCs. Making Sense of MOOCs -- A Reading List and Missing Perspectives on MOOCs will follow.

You might also be interested in a related online discussion that has been happening over at the EduTech Debate site.
 

Note: The image used at the top of this blog post of a Masai boy in Tanzania moving toward a group of cows ("those aren't moos you hear on the African horizon, but MOOCs") was originally posted to Flickr by Andreas Lederer; it comes via Wikimedia Commons and is used according to the terms of its Creative Commons Attribution 2.0 Generic license. The image below of what James Joyce famously called a moocow was a finalist in the Wikimedia Commons Picture of the Year 2007 competition. It comes from Daniel Schwen via Wikimedia Commons and is used according to the terms of its Creative Commons Attribution-Share Alike 3.0 Unported license.

---

---

notes from talk by
Daphne Koller, Coursera
@ The World Bank, Washington, DC
21 February 2013

To help share lessons from Coursera's activities to date (given its current size, and public profile, it is still hard to believe that the company was only formed in 2012!), Professor Koller provided a detailed overview of some of the things the MOOC provider has been doing over the past year, and what it has been learning along the way, as part of a discussion called Free Massive Open Online Courses (MOOC) to Accelerate Youth Employment in Africa. The talk is available in full through this link. Please note that you'll need to install an Adobe Connect plug-in to watch it (one of my browsers balked at doing this -- you may want to experiment with using more than one).

For those of you who do not have 90 minutes to devote to watching the video -- or who are not sure if it is worth your time to do so, or who have technical problems -- I thought I'd share some of my notes from her fascinating presentation, in case doing so might be useful. This is basically a longer, and updated, version of her celebrated Ted Talk.

---

A global challenge

Koller began her talk with a very sober example of the demand for higher education around the world that isn't being met. She spoke about the stampede that occurred in South Africa when thousands of students tried to register for a limited number of last minute places at the University of Johannesburg and then showed a graph detailing the steady rise in costs of higher education. Her point: Both demand and costs are increasing. What might we be able to do in response?

---

Speed and scale

In September 2011 Stanford University opened up three computer science courses for participation by anyone over the Internet. Over 100,000 people signed up for each. The course on machine learning offered by Andrew Ng, who co-founded Coursera with Koller, normally reaches about 400 students each time it is offered on the Stanford campus. When it was offered over the Internet, over 104,000 people registered for it. To reach that number of people with his 'traditional' course at Stanford, Ng would have to teach the course for 250 years (and of course he wouldn't reach the same geographically-dispersed population of students).

Less than a year and a half later, Coursera now registers 1.45 million course enrolments each month, in a variety of subjects, in partnerships with scores of universities, and the other MOOC platforms are growing as well. Indeed, everything is happening at warp speed: Some of the data in the slides that Koller shared, which she had presented to a different group only the day before, were already out of date as a result of new announcements. At the time of Koller's talk, Coursera had signed up 61 university partners, primarily in North America, with almost all courses to date offered in English.

---

Testimonials from students

Koller shared examples of emails from people who have benefited from participation in a course from Coursera and its partners which offered access to learning opportunities that they didn't otherwise have, for reasons of geography, disability, illness or cost.

---

General overview

MOOCs like Coursera differ from so-called open courseware  (like the MIT OCW initiative) in that they are complete courses, and not just a collection of course materials. The start and end at a pre-determined time, feature weekly assignments, and homework that is graded.

Students successfully completing a course receive a certificate. Until recently (more about this later in the talk) it was not possibly to get official 'credit' for completing a Coursera course. Koller stated that, even without official credit, students have been able to present Coursera certificates to employers and land jobs as a result.

---

There are three pillars of the Coursera experience: video; assessment; and community.

---

Video

The use of video helps break geographic boundaries. Professors have been videotaped "in the field" (and literally in fields!), conducted video chats with select numbers of randomly-chosen students (with the video archived so others can watch later), and even appeared as a character in a video game.

The length of university lectures depends largely on the availability classroom space. Online these space constraints disappear, and so there is an opportunity to break things up into smaller chunks so that the time allotted to each matched the content better -- and corresponds better to a learner's ability to follow and engage with the content (this is of course not a learning unique to the Coursera experience, lots of other online education providers do the same thing).

One professor leading a course offered through Coursera noted that the nature of MOOC discussions are different: each year his Princeton students at Princeton said things that were largely the same, but the interactions and comments made by his much larger number of globally diverse students taking his MOOC were much more diverse.

Koller then went on to talk about pedagogy and interaction in a MOOC -- and how the platform can enable the types of pedagogical approaches and interactions that Coursera feels, based on available research, are important.

(A side note: Coursera crowdsources the subtitling of videos in different languages using an online tool. This is quite interesting! Translations into additional languages can be done very quickly, as  additional translators can be brought in and would only have to do text, not audio, translation.)

---

Assessment

If you have 100,000 students in a MOOC, and no one is willing to pay for 5000 teaching assistants, how do you grade and provide meaningful feedback?

Computers are helpful in many cases. Using computers to handle things like multiple choice, short answers, and math expressions is not that difficult, but you can do a lot more than that. Anything that has a structured output, Koller said, can be graded by a computer (an example would be including things like Excel spreadsheets developed in a marketing class). A benefit of automated grading is that it allows students to achieve mastery through immediate feedback and repetition (this is not an insight unique to Coursera, it is a more general observation about the potential utility of computer use in education).

A common and criticism: 'Students could just keep doing the same problems until they get a correct answer -- are students really learning in such cases?' Koller replied that there is evidence that yes, they are. (She then showed a slide at this point that proclaimed 'for students of similar performance, mastery-based score improvements correlate with future performance' with supporting graph.)  'How do you grade things that computers can't grade, like essays?' Koller replied that you have students grade each other, noting that Coursera's approach in this regard this draws on work in calibrated peer review at UCLA). You can also use crowdsourcing. Involving students in the assessment of work by other students has a further educational value. A student learns by critiquing content, and then learns through the critiquing of the critiques of others.

Does this work? It depends on the grading rubric you construct to guide and assess this, Koller said. She then showed how peer grading can be used to help assess creative, open-ended assignments, using examples from a design course that showed prototype models made by students in the course (one example was from India, another from Spain, a third from the Philippines), who then provided input and feedback on the prototypes. Some professors participating in courses offered through Coursera are now introducing peer grading in their face to face courses, she noted, not because they can't hire enough teaching assistants (they can), but rather because they see a pedagogical value in this approach.

---

Community

Peer grading can also be used to create a sense of community and discussion among students. To illustrate this, Koller showed an example of how one essay uploaded in a course received 59 comments and 872 views -- far more than would have been possible in a traditional classroom setting. She said that, while you can't scale up instructor time to interact directly with students, you can scale up the number of interactions between students by leveraging the 'power of the crowds'. The result? Many more interactions in the virtual environment of a MOOC than in a physical environment. She then considered a potentially counterintuitive question, asking: Might larger classes actually be better (at least in this regard) than smaller classes? A provocative question, and one perhaps worth thinking about. Students also self-assemble into both physical study groups by geography: one group in Buenos Aires, for example, another in Cape Town, yet another in Kansas City. (There are presumably lessons from such experiences of relevance to the Tanzania pilot which the World Bank is supporting.)

---

Credit and credentials

What do students get at the end of a course offered by Coursera (beyond having learned something, of course)? Coursera began by awarding a letter from the course instructor which contained lots of legal boilerplate language at the bottom, stating things like this is not a Stanford degree, we can't vouch for the identity of the student, etc. Even in the early days, there were examples of universities giving credit based on this letter.

Koller was interrupted at this point by a question about why Stanford itself doesn't give credit. She mentioned that there are lots of issues at play here. Stanford students pay a lot of money to get Stanford credit, if they could pay nothing and still get Stanford credit, what would Stanford's business model look like? Stanford would argue -- and Koller would agree -- that Stanford credit for on-campus courses is a different currency than credit for a Stanford MOOC. Koller  argued that the value of a Stanford course on campus is higher than that of a Stanford MOOC, but suggested that we shouldn't be comparing these two things. Rather, we should compare the quality of a Stanford MOOC with other types of learning experiences that are available.

Now Coursera also offers a 'verified certificate', where students opt in to a scheme where their identity is confirmed (via a biometric profile). This certificate has a university brand/logo on it, and the certificate itself has a permanent place on the Coursera web site. This means that the certificate can't be misplaced over time, and that the certificate itself is verified. While you can forge a PDF or a certificate or letter that you download, Koller said, you can't forge the certificate that is on the Coursera web site itself. Students pay a small amount to be verified ($30 was mentioned).

Koller talked about American Council on Education credit, which can be used to acknowledge participation in non-traditional sorts of learning activities. If a student has ACE credit for a course, she can then take it to her university and ask for credit. This gets around the issue of needing Stanford, or another Coursera partner university, to offer the credit itself, as the student can still get credit if her home university accepts the ACE credit. Five introductory Coursera courses now qualify for ACE credit.

---

Statistics and analytics

What can you do with the data generated as a result of a Coursera course? Coursera tracks every single event on the site: What students access, when and how often, if they skip through a lecture (perhaps the professor is boring?), where they pause or rewind, which answers they submit to a quiz, do they post a question in a forum, who looks at and responds to the question, etc. When (for example) 50,000 people participate in a course, the data can offer powerful insights. Koller showed an example of a question in one MOOC which 2,000 people answered incorrectly. In a class of 100, she noted, if two people answer a question incorrectly, no one notices. In a course of 100,000 students, if the same percentage (2%) answer incorrectly, it *is* noticed! The instructor can then examine the question and try to figure out why students are getting it wrong -- and make changes if it is seen to be prudent to do so. When students get a wrong answer, they can receive a 'sorry you're wrong, here might be why, and here's how you might be able to correct your mistake' screen in response, an example of a sort of  personalized feedback.

Coursera and their partner universities can also gain insights into pedagogy. They can, for example, show two lectures to students -- one with a professor's face in front of a PowerPoint presentation, the other one with just the presentation -- and test to see if anything different happens as a result. This sort of A/B testing is common on the Internet (Google does it thousands of times a year with its search results), but it hasn't been possible and practical in the classroom because classrooms don't have the necessary scale. Where A/B tests are run in classrooms, it can take years for the study to be written up and appear in a journal, and the results thus can't be fed back into course design very quickly. In this particular example, Koller noted that they don't know which option worked 'better', seeing the professor's face or not, because they stopped the experiment halfway through. Students were complaining that they couldn't see the instructor's face, so they simply added it back in. The students, then, felt that it made an impact -- Coursera doesn't know if it actually did. (If you are asking yourself: How did some students know that other students were seeing the professors face, and they weren't? The answer: They were communicating in the online forum for the course!) In another experiment, they sent out emails to students to remind them there was an upcoming deadline for completing a problem set. Only one of the eight versions of the email was shown to have increased retention -- the other 7 actually decreased retention! (Follow-on research suggests that, in the worst case email, retention decreased because students felt nagged.)

---

Improving learning outcomes

Koller she mentioned Bloom's seminal paper on "The 2 Sigma Problem", which investigates differences in achievement as a result of instruction via lecture, mastery learning or individual tutoring, with each instruction method achieving significantly improved results. Bloom noted that mastery learning and individual tutoring don't scale. Or do they? In normal face-to-face circumstances, maybe not ... but what if everyone has a tablet or a smart phone? In such case a case, you can potentially scale mastery learning, Coursera believes. Koller then noted that the technology to scale individual tutoring isn't here ... yet.

Koller then related two quotes about education. The first was from Edwin Emery Slossom [she attributed it to mark twain in her TED Talk]: "College is a place where a professor's lecture notes got straight to the students' lecture notes, without passing through the brains of either." (Whenever I hear this quote, I am immediately reminded me of a scene from the 1980s American film comedy Back To School, where a professor just uses a cassette tape player to broadcast his lecture to the empty lecture hall ... filled with cassette tape recorders of students recording the lecture.) The second was from Plutarch: "The mind is not a vessel that needs filling, but wood that needs igniting'. Koller then proposed that we should do different things in the classroom. We shouldn't try to convey information there (that can happen outside of class), but rather to use such time to engage with students. This is, of course, famously known as 'flipping the classroom'.

---

"Blue Ocean"

Wrapping up her talk, Koller mentioned a case study of Coursera by Wharton's Christian Terwiesch which suggests that the service deliver model of MOOCs likes Coursera offer 'efficient frontier', whereby we can 'cash-in' by holding faculty productivity constant and achieving better learning, or by holding learning constant and saving money as a result of increased faculty efficiency. (If you want to follow his argument, watch him explain it on YouTube.) Terwiesch also argues that MOOCs should pursue a 'blue ocean strategy' by seeking to create new demand in an uncontested market space (a "blue ocean") rather than compete head-to-head with incumbent suppliers in an existing industry. In other words: MOOCs could be used not to compete with offerings from traditional universities, but rather work to open new markets in education that existing providers can not tap. This strategy coincides nicely with Coursera's goal of 'education for everyone'.

Koller noted that 80% of students taking courses offered through Coursera already have university college degrees, half of them at the master's level. While she expects that this mix will change as more lower level, introductory courses are offered through the Coursera platform, this suggests how MOOCs can enable lifelong learning. She concluded that students who take courses offered through Coursera come from all over the world. Only about one-third of students are from North America; a slightly lower proportion come from Europe. This suggests that MOOCs can play a real role in offering access to education to those who need it the most, she said, pointing the way to an opportunity to change education from a privilege for a few to a human right for all. (If you've seen Koller's TED Talk you'll have heard her passionate comments in regard already -- and if you haven't and have read this far, you really should -- it's about one hour shorter than the video of her talk at the World Bank in February.)

---

I have offered these notes with minimal editorial intervention. I hope I've done an accurate job of conveying some of the key information and arguments offered in Koller's talk. (If not, let please correct me in the comments section below.)

---

This is the first of three EduTech blog posts about that will be published in quick succession on the topic of MOOCs. Making Sense of MOOCs -- A Reading List and Missing Perspectives on MOOCs will follow.

You might also be interested in a related online discussion that has been happening over at the EduTech Debate site.