Final Gamification Research and Design Paper

Can Gamification Increase Student Educational Outcomes as Reported by Year End Assessments

 

Scott M. Haselwood

 

Oklahoma State University

 

Introduction

            Gamification is the process of applying gaming mechanics in non-game contexts and has gained popularity across several different industries. It has the potential to make a significant impact in education (Attali & Arieli-Attali, 2015). The purpose of gaming mechanics is to increase motivation or reward people for performing a specific task (Rani & Lenka, 2012). Using gamification in an educational setting can have positive outcomes (de Byl, 2013) and can increase student engagement (Kingsley & Grabner-Hagen, 2015). Studies have also shown that the use of gamification can produce results that were not as favorable as those shown by a control group (Hanus & Fox, 2015). Looking at these two studies, Kingsley and Grabner-Hagen (2015) and Hanus and Fox (2015), indicates that there is more research that needs to be done, these studies have reached opposing conclusions. Because the two studies reached such different conclusions, more research is needed to determine what impact gamification can make in the classroom. Determining what aspects of gaming have the strongest impact on student learning and retention is important, teachers will want to know exactly what gaming characteristics to implement to achieve success. The purpose of this study is to investigate what mechanics of gamification will help students retain information quicker and give the students the ability to retain that information longer which should increase test scores.

For this study, the following list contains the key term and definitions that will be used.

• Gamification – Incorporating game-like mechanics such as leaderboards, level-ups, badges and quests (assignments) into a classroom setting (Kingsley & Grabner-Hagen, 2015)
• End of instruction exams – Exams given toward the end of the instructional year in grades 3-8 and in specific courses in high school, in Oklahoma this is typically from late March through mid-April.
• Traditional classroom – A classroom where the teacher primarily uses lecture and some activities as the primary sources of instruction.
• Curriculum – “is the study of any and all educational phenomena” (Egan, 1978, p. 71), for this study, it is what the teachers will be teaching during the school year in an Algebra 2 course.

Purpose of the Study

This study seeks to help teachers by giving them a model that will increase student end of instruction (EOI) test scores. Under our current educational system, EOI’s are key markers for both a teacher and the school, it is important to have the best test scores that students are capable of earning. A direct benefit from gamifying a class would be an increase in test scores on the EOI, benefitting the student, the teacher, and the school.

Because the EOI’s are the driving factor in what teachers teach during the school year, certain items must be taught before the exams are given. Gamifying a class could give the teacher the opportunity to cover difficult educational units in more depth, or allow the teacher to teach more educational units before the test. The gamification model could give teachers some freedom in managing their time, allowing more or less time as necessary.

Assumptions

During the course of this study, teachers are expected to use a gamification model for one section of Algebra 2 and a more traditional classroom model for the other section of Algebra 2.   It is assumed that the students participating in these courses were enrolled in those courses for the full academic year. The end of instruction exam, administered by the Oklahoma State Department of Education (OSDE), is assumed to be valid and reliable testing instrument.

Limitations

Students who move into or out of a classroom during the school year could have an impact on the data. Those students would not have had the full year to participate in the gamified classroom, this would be difficult to see in the data. Teacher personality and how they interact with students could make a difference. Student will work harder for some teachers and not so hard for others. It will be difficult to determine how large of a role a specific teacher had on student learning. Previous knowledge a student has about a specific subject could have an impact. Some students have the ability to easily understand mathematics; it would be hard to isolate those students in the testing data. If a student is experiencing a significant life event during the testing window, the student could test more poorly.

Organization of the Study

            Presented above is a brief description of what gamification is and why further research is necessary. In the sections that follow, a review of relevant literature will give detailed information that relate to gamification. In the final section, the methodology that will be used to assess the impact of gamification on end of instruction exams and the ability of the teacher to teach more curriculum will be presented.

Literature Review

Introduction

            The purpose of this study is to investigate what mechanics of gamification will help students retain information more efficiently and give the students the ability to retain that information longer. The purpose of the literature review it to examine how previous studies can inform research in this study. The first section will review the theory that gives gamification its foundation. The second section will provide insight into the design of gamification as it applies to education; specifically what elements have been used. The following section will discuss how motivation and incentives are used as part of gamifying a classroom. The final section will review research that discusses how to implement gamification into the classroom.

Theoretical Perspective

Incentive Theory (IT) is rooted in B.F. Skinner’s study of behaviorism (Berridge, 2000) and motivation. If actions are seen as positive, people are more likely to act in that manner; if actions are seen as negative, then they will be less likely to repeat that action (Rani & Lenka, 2012). IT emerged as a direct result of the study of drive theory in the 1940s and 1950s (Hockenbury & Hockenbury, 2010). The use of incentives has been used as a strategy to increase motivation at work (Korman, Glickman, & Frey, 1981). The theory states that if the targeted recipient is offered a reward – one with meaning to them – then they will alter their behavior to achieve that specific reward (Korman et al., 1981). Important to IT is the understanding that the offered reward has positive value for the recipient (Rani & Lenka, 2012).

IT is most effective when the reward is given with the shortest delay possible. The larger the reward, the longer the reward can be delayed after the action and still have an impact on behavior for the targeted recipient (Killeen, 1985). If the incentive is too large, then the recipient may possibly begin to distrust the institution, or perceive threats to their freedom and feel angry (Korman et al., 1981). IT shows that if an action-reward loop occurs frequently enough, it can cause the action by the targeted recipient to become a habit (Rani & Lenka, 2012). Hockenbury and Hockenbury (2010) state that behavior by the recipient can be motivated by reaching external goals such as “rewards, money, or recognition” (Hockenbury & Hockenbury, 2010, p. 321)

This study will investigate the effectiveness of gamification in the classroom. Gamification is the process of building game-like mechanics into the fabric of the classroom culture. When playing games, “gamers” are rewarded with badges, leaderboards, level-ups, and experience points. By infusing the classroom with game-like mechanics, the teacher is trying to encourage student behavior in a fashion that will build deeper meaning from course material. IT is part of this process – in gamification, teachers reward students for performing desired behaviors or action. In a gamified classroom, teachers use IT to reward students for completing homework, passing a quiz, turning in a project, or completing a self-directed study. IT theory holds that if teachers offer the appropriate reward quickly enough, then the students actions become a learned habit (Rani & Lenka, 2012).

Gamification Design

Gamification mechanics that are used as part of gamifying a class include points and leaderboards. Posting this information in the classroom allows students to see where they rank when compared to others in their class (Christy & Fox, 2014). Teaching and learning are more complex than just leaderboards and points, so designing a system that is dynamic and uses learning theory such as the ARCS framework (attention, relevance, confidence, and satisfaction) are important (Jung Tae & Won-Hyung, 2013). Putting curriculum together with gamification takes time and must be done the right way, or students may not benefit from this educational model (Hanus & Fox, 2015).

By using several different aspects of gaming and combing them with a thorough knowledge of a specific curriculum, a gamified course structure can be developed that can be enjoyable and engaging for the students (de Byl, 2013). There needs to be a clear outcome for the game and the teacher needs to understand the characteristics of the course or specific situation (Kim, 2015). Gaming design can also influence motivation. When this is combined with Flow Theory, participants can get a lot of enjoyment from their specific experience (Deterding, 2015).

Motivation and Incentives

When using game mechanics in the classroom, the teacher is using incentives to guide the behavior of the students. Students could work to be at the top of a class leaderboard (Christy & Fox, 2014) or could level up (de Byl, 2013) as they move through a particular curriculum. However it is important to create an incentive system that will work with the targeted population– if the reward is too large, the students could respond with distrust (Korman et al., 1981).   Berridge (2000) found that reward learning and motivation could co-exist. This is in line with the mechanics of gamification, where students would be motivated by the gaming mechanics to earn the next reward in their class.

Motivation can have a positive or negative impact on overall student performance. In a longitudinal study on gamification, Hanus and Fox (2015) found that students who participated in the gamified course in their study had lower intrinsic motivation. In their study they used the gaming mechanics of leaderboards and badges. This is in contrast with a study done by de Byl (2013) that showed students were motivated to complete tasks outside of class to earn experience points in the class. In addition to leaderboards and badges, de Byl used several other mechanics in her course design including experience points, levels, and customized learning experiences. The contrasting conclusions about motivation from this study show that course design could have an impact on student motivation citation. Students who are intrinsically motivated (motivation that comes from within the individual) will work on a task on their own or seek to improve skills without being asked to do so (Rani & Lenka, 2012). Gamification could be a tool that will help students become motivated to work harder in their studies.

Implementation of Gamification in the Classroom

Implementing the gamification model can be done in several different ways. De Byl (2013) used several different aspects of gamification, including points, levels, leaderboards, badges, social engagement loops, and made her course customizable. She found that students were willing to work outside of the classroom to earn points and that overall students found the course to be enjoyable. In a different study, a course that gamified content by adding badges, power ups, awards, and levels and combining those with effective pedagogy helped students learn skills that will be important in their future (Kingsley & Grabner-Hagen, 2015). These two studies implemented several aspects of gamification and reported student success. As mention previously a study by Hanus and Fox (2015) used only leaderboards and badges. This study reported that students in the gamified class had lower motivation by the end of the course. These three studies vary in the number of gamification characteristics that were used during the classes. This could indicate that the number or type of characteristics implemented in a class could have an impact on student learning.

Hypothesis and Research Question

In the present study, the key research question is: will gaming mechanics (gamification) in a classroom increase test scores? To resolve this question two hypothesis are stated:

H1: Students in a gamified classroom will show a statistically significant increase on end of instruction exam scores when compared to students in a traditional classroom.

H2: Teachers using a gamified class model will be able to cover more units of curriculum during the school year than teachers who use a more traditional classroom model.

Summary

            In summary the review showed us that gamification has a foundation in behaviorism (Berridge, 2000) and that offering positive rewards can have value for the person receiving the reward (Rani & Lenka, 2012). When using a system of rewards that is based on gaming design, the course can be engaging for the students (de Byl, 2013) and students can get a lot of enjoyment from the experience (Deterding, 2015). There are times when motivation can be hampered by gaming a classroom (Hanus & Fox, 2015) as well as times when students can be motivated to do extra work outside of the classroom (de Byl, 2013). Literature also indicates that adding specific items such as badges, power-ups, awards, and levels can help students learn skills that they need for school (Kingsley & Grabner-Hagen, 2015).

Method

Introduction

In this section there will be a detailed description of the methods that will be used to carry out this study. The section will begin with a description of the research design. The next section will discuss the population and the sample that will be used and how the data will be collected. The last section will give the details of the instrument be used to analyze data and a brief description of the statistical lens that will be used for analysis.

Research Design

This study will happened over one school year in the Algebra 2 classes in a large suburban high school in Oklahoma. There will be six different sections of Algebra 2 and three different teachers that will be used for this study. Each teacher will teach a gamified class and a more traditional class. Only data from students who are enrolled in one of the six different sections will be used for this study, any student who does not wish to participate may enroll in an Algebra 2 course that is not participating. All participating sections will be placed in a jar. Three sections will then be pulled, one at a time, from a jar. After a section has been pulled from the jar, it will be assigned to a teacher. Once all three teachers have been assigned a gamified section, they will be randomly assigned a control section in the same way.

Teachers will follow the curriculum that will be tested by the Algebra 2 exams. They will have weekly planning meetings to ensure that the all teachers are teaching the same material. Teachers will teach all of the units of curriculum in a specific chronological order that they determine to be most appropriate for all six sections. Teachers will record how many units of instruction that they are able to complete before the Algebra 2 exams are given.

Participants

Population. The population consists of all students who are enrolled in Algebra 2, in high school, in the state of Oklahoma.

Sample. The sample will include 180 high school students enrolled in participating sections of an Algebra 2 course. There will be no disqualifying factors for student participation, if students do not want to participate, they will be enrolled in a different section of Algebra 2. The size of this sample would allow for a variety of data to be collected for analysis and for norms to be identified. This sample should be large enough to represent the overall characteristics of students in the population.

Data Collection

All students (including those on an individualized educational plan) who are enrolled in an Algebra 2 course must take the Algebra 2 end of instruction exam (EOI) in April. The EOI is administered by the state of Oklahoma and every high school in the state must give the exam. The test will be locally administered by on site staff that will ensure that testing is done according to protocols established by the state of Oklahoma. Students will have two hours to take the Algebra 2 EOI on a computer. If the two hour block ends before the student has completed the test, the student will be given additional time. The test is done completely online and all information will go directly to the testing company to be collected and scored.

All students who take Algebra 2 in the state of Oklahoma are required to take an EOI exam in April. The OSDE through local school districts administers this exam. The student will take the EOI on a computer and students will have two hours to complete the exam. If students require more time, modifications will be made to extend the testing period. The test will be given over two consecutive school days. If the test is started on a day before a break, it will be completed on the next school day. Tests are kept secure until they are taken, but the OSDE has released a blueprint of what will be on the test, which includes items such as:

• Number Sense and Algebraic Operations
• Relations and Functions
• Data Analysis, Probability, & Statistics

Something that could influence results on the EOI is the ability and/or the experience of the teacher. The culture inside of a classroom could have an impact, one teacher may be very distant to her students and one may not be. A student could have started the year in the wrong class and the situation was not corrected appropriately. A student could be on an individualized educational plan and is being required to take a test without modification. A student may be experiencing a significant life event during testing.

For the end of instruction test scores, the data that the testing company releases to the schools will be used. The data from the students who participated in the study will be used. Test scores from the three control courses will be compared to test scores from the three experimental courses. To determine the curriculum that is covered, the teachers will be asked how many units of study they were able to teach before the end of instruction exam was administered.

Potential issues include some students not taking the EOI for various reasons due to absence from school on testing days. The courses will be taught by different teachers, it is possible that some teachers will teach different topics differently and this could impact student learning. A student could have a significant life event that impacts their testing or could have impacted their attendance during the year and this would have an effect on overall performance. The testing website could crash and cause students to re-take portions of the test or the entire test.

            Validity. The testing company, Measured Progress, has not released the validity for the Algebra 2 End of Instruction exam.

Reliability. The testing company, Measured Progress, has not released the reliability for the Algebra 2 End of Instruction exam.

Data Analysis

When the testing data is released back to the school in the fall, the test scores between the experimental classes and the control classes will be compared. The raw scores will be used to determine if gamifying the classroom increased the EOI scores for students.

To determine the amount of curriculum that was taught during the school year, teachers will compare the number of units taught in the control classes and the number of units taught in the experimental classes.

Summary

Listed in the previous section was a description of the methodology that will be sued for the current study. This section explained information about the design and procedures, the population and sample that will be part of the study, the instrument used for collecting the data, and the statistical method that will be used to interpret the data.

 

References

Attali, Y., & Arieli-Attali, M. (2015). Gamification in assessment: Do points affect test performance? Computers & Education, 83, 57-63. doi: 10.1016/j.compedu.2014.12.012

Berridge, K. C. (2000). Reward learning: Reinforcement, incentives, and expectations. Psychology of learning and motivation, 40, 223-278. doi: doi:10.1016/S0079-7421(00)80022-5

Christy, K. R., & Fox, J. (2014). Leaderboards in a virtual classroom: A test of stereotype threat and social comparison explanations for women’s math performance. Computers & Education, 78, 66-77. doi: 10.1016/j.compedu.2014.05.005

de Byl, P. (2013). Factors at play in tertiary curriculum gamification. International Journal of Game-Based Learning, 3(2), 1-21. http://dx.doi.org/10.4018/ijgbl.2013040101

Deterding, S. (2015). The lens of intrinsic skill atoms: A method for gameful design. Human – Computer Interaction, 30(3-4), 294. doi: 10.1080/07370024.2014.993471

Egan, K. (1978). What Is Curriculum? Curriculum Inquiry, 8(1), 65-72. doi: 10.2307/1179791

Hanus, M. D., & Fox, J. (2015). Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education, 80, 152-161. doi: 10.1016/j.compedu.2014.08.019

Hockenbury, D. H., & Hockenbury, S. E. (2010). Discovering psychology: Macmillan.

Jung Tae, K., & Won-Hyung, L. (2013). Dynamical model and simulations for gamification of learning. International Journal of Multimedia & Ubiquitous Engineering, 8(4), 179-189. Retrieved from http://www.sersc.org/journals/IJMUE/vol8_no4_2013/18.pdf

Killeen, P. R. (1985). Incentive theory: IV. Magnitude of reward. Journal of the Experimental Analysis of Behavior, 43(3), 407-417. Retrieved from http://onlinelibrary.wiley.com/doi/10.1901/jeab.1985.43-407/abstract

Kim, B. (2015). Designing gamification in the right way. Library Technology Reports, 51(2), 29-20_23. Retrieved from http://argo.library.okstate.edu/login?url=http://search.proquest.com/docview/1658221395?accountid=4117

Kingsley, T. L., & Grabner-Hagen, M. M. (2015). Gamification: Questing to integrate content knowledge, literacy, and 21st-century learning. Journal of Adolescent & Adult Literacy, 59(1), 51. doi: 10.1002/jaal.426

Korman, A. K., Glickman, A. S., & Frey, R. L. (1981). More is not better – 2 failures of incentive theory. Journal of Applied Psychology, 66(2), 255-259. doi: http://dx.doi.org/10.1037/0021-9010.66.2.255

Rani, R., & Lenka, S. K. (2012). Motivation and work motivation: Concepts, theories, and researches. International Journal of Research in IT & Management, 2(8), 14-22. Retrieved from http://www.euroasiapub.org/pastissue.php?title=IJRIM&vol=Volume%202,Issue%208,August-2012