Syndicate content

EVOKE Reflections: Results from the World Bank's on-line educational game (part 2)

Robert Hawkins's picture

some reflections from EVOKE

On March 3, 2010, the World Bank Institute (WBI) and infoDev launched EVOKE, an online alternate reality game with the goal of supporting social innovation among young people around the world.

I’ve written previously about the EVOKE initiative here and here.  Following on a blog post from earlier this week, I wanted to provide some more data and reflections on the experience. 

At the end of July, the EVOKE team met (most for the first time face to face) to review the EVOKE experience and share lists of “what went right” and “what went wrong”.  We live blogged the meeting and the complete game analysis can be found here.  

We also met with the Natoma Group, which conducted the independent evaluation of EVOKE.  The Natoma evaluation drew on a number of different data collection tools, including a survey, a series of interviews, a review of project ideas ('evokations'), Google Analytics, and evaluator insight.  The survey yielded 500+ responses and the evaluators interviewed 15 individuals (five from South Africa, one each from Ethiopia player, the USA and Canada, as well as four 'game runners'one professor, one teacher, and one 'evokation' winner from South Africa).

The evaluation was conducted with the following overall objectives for EVOKE in mind:

  • Players recognize social challenges in their communities and in the world & are inspired to act
  • Players think creatively about ideas to address challenges
  • Players learn about social innovation
  • Players develop action plans for sustainable social enterprises
  • Players develop 21st century skills including problem solving, team work, information reasoning etc
  • Players have fun with: activities, interacting with each other and coming up with solutions to challenges
  • Players develop a sense of self confidence, community and collaboration  
  • Extensive peer review of good ideas and activities outputs occurs
  • Players interact with and learn from mentors
  • Top social innovation plans are recognized and some are funded through an incubation process

 
While the final report is not yet complete, the following are some preliminary findings:

Finding: EVOKE objectives were met

Based on the survey, EVOKE had a strong effect on:

  • Developing new ideas about global challenges
  • Developing new ideas about local challenges
  • Learning about potential solutions to….
  • Learning about people in other countries
  • Learning about sustainability

 
EVOKE had less effect on
building a network of colleagues and friends and gaining self confidence.

EVOKE evaluation

“I live in small university town so by playing EVOKE I got to look beyond the bubble I live in. You hear about stories in the news and read about them in the paper but now I actually got a chance to give my opinions, to see the bigger picture and to see things beyond the small town I live in. It opened my eyes to a lot of things let me put it that way.”
  -- EVOKE player

 
Finding: South African players participated much more actively than other players around the world

An initial concern was that as a game open to anyone, players in the developed world would dominate the discussion.  A positive finding is that African participation was around 8% of all players -- roughly equivalent to the global population percentage (around 12%).  South African players were a large portion of this total at around 5% and the 1,010 South Africa players contributed content significantly above the mean of all players in EVOKE.  In some cases contributing 78% more content than the mean (for "Act" missions) and posting blogs at 57% above the mean. 

 
Finding: EVOKE made players in Africa think differently

Evoke Evaluation -- graph 2 

Finding: EVOKE inspired players in Africa to “Think Big” and “Act”

Think big thoughts about the future

 

No effect

2

5.7%

 

Slight effect

5

 14.3%

 

Moderate effect

7

 20.0%

 

Strong effect

21

 60.0%

 

No opinion

0

 0.0%

 

 

35

   

Picture myself starting something new

 

No effect

2

 5.7%

 

Slight effect

4

 11.4%

 

Moderate effect

7

 20.0%

 

Strong effect

21

 60.0%

 

No opinion

1

 2.9%

 

 

Finding: EVOKE connected young innovators to each other

The evaluation team concluded that perhaps the most valuable contribution of EVOKE is as a platform to connect young innovators to other minds, ideas and inspiration – to connect them to social capital to help them develop new and innovative solutions to development challenges.  They summarize this goal as follows:

"…to serve as a cornerstone supporting the development of a cadre of civic-minded, socially engaged African innovators, who are able to find and organize knowledge resources and human resources to address local, national, regional and international problems in new ways." 

The EVOKE summit will be taking place at the end of September in Washington, DC.  At this summit we will bring together a number of the players, mentors and development team to celebrate the incredible work of the EVOKE community and reflect on how to build on the lessons learned for future iterations.  Please share any ideas you have!

“Evoke gave a safe place to put the idea out there and when no-one laughed but actually had good things to say it gave me the confidence to approach others…so  I can only imagine that there are many students out there with similar ideas, but without the confidence to pursue them.”
  -- EVOKE Player


 


 

Comments

Submitted by Sam Ng on
Having recently come across Evoke and reading this evaluation, I'm inspired, but also wondering if there are plans to follow up with Evoke players? It's nearly 2 years on and it'd be interested to see if the strong positive player responses to questions (eg to starting something new) have translated into any real behavior change.

Hi Sam, I know that a number of the Evoke teams are still active, but I am not sure what plans there are to follow up with them in the context of another evaluation. I do expect that a good number of them will participate in the second round of Evoke, which has a specific focus on Brazil and which will be offered in both Portuguese and English. It is meant to kick off in September; this month about a thousand players have been beta-testing the upcoming game, which will take place at http://equipeevoke.com/. -M

Add new comment