Assessing the Role of Python Programming Gamified Course on Students’ Knowledge, Skills Performance, Attitude, and Self-Efficacy

Introduction

As the world steadily transitions to the era of automation, computer programming is becoming an essential skill in many areas of the society. Apart from the expected labor shortage in the field of Information Technology (IT) due to the impact of Coronavirus Disease 2019 (COVID-19) pandemic [1], many introductory programming students face learning difficulties for acquiring the necessary knowledge and skills in this area [2]. A recent study on computer programming highlighted the learning challenges faced by novice programmers in terms of affective factors (e.g., motivation and attitude) and individual differences (e.g., aptitude and mathematical ability) [3]. It also provokes discontent among students, and the difficulty of the subject is considered to be an important dropout factor in IT education [4]. Thus, to provide programming students with the necessary support, many have endeavored to develop policies, strategies, and tools for teaching computer programming. For instance, proposed pedagogies to facilitate the creation of an effective learning environment includes game-based learning [5], augmented reality [6], visual block programming-based instruction [7], gamification [8], robot programming [9], and intelligent tutoring systems [10], to name a few. Nevertheless, there is no agreement on what the most efficient method is as far as how and the extent to which these proposed approaches can be used to support students in learning programming [11]. COVID-19 and online education aside, most universities still utilize traditional teaching methods (e.g., lectures, homework, and coding demonstration), which is in contrast to the fact that innovative pedagogies are evidently more effective [12].

Following the recommendations that specialized tools are needed to allow students acquire detailed knowledge and be motivated in the course [13], we installed a gamified course in teaching Python programming in higher education. By doing so, we hypothesized that programming students would have enhanced (a) basic programming knowledge, (b) self-efficacy of computer programming, (c) coding skills performance, and (d) attitude towards the course. To test these hypotheses, we adopted a quasi-experimental study design where two sections with 50 students each were randomly assigned to experimental or control groups. This study contributes to the existing thread of discussion related to programming teaching strategies and gamified courses (notably in Python programming language). In addition, understanding how programming students learn, use, and interact with this innovative pedagogy may establish a basis for educational institutions, curriculum developers, and programming professors on what supplemental strategies can be used in teaching computer programming aside from what was mentioned and adopted on existing studies.

Background of the Study

With the wide array of programming languages available in different types (e.g., procedural, functional, object-oriented, scripting, etc.), choosing the best language that would satisfy the needs and conditions of a specific problem domain can be a challenging task. Nevertheless, regardless of the preferred language of any newbie, learning basic programming concepts is still more significant. With this rationale, programming is often introduced in primary education using pedagogies (e.g., game-based learning or visual block programming) suited for this level and without a necessary focus on the language itself. Still, an exploratory survey of 100,000 open source software projects found on GitHub showed that there is a difference in terms of popularity, interoperability, and impact of languages used in computer programming [14]. According to this study, earlier programming languages like C are still a popular option in modern software projects and the rapid spread of website applications has made Ruby and JavaScript pervasive. On the other hand, the most recent annual Developer Survey of Stack Overflow revealed Python as the third most popular language among 83,439 software developers from 181 countries [15]. This is partially attributed to the rise of artificial intelligence and machine learning applications where Python is the most endorsed programming language [16]. Another study [17] also listed other applications for Python such as data science and Internet-of-Things. Figure 1 presents different emerging fields where Python can be useful as the programming language. Its wide applications are attributed to its comprehensive standard library as well as its large developer community.

Meanwhile, programming languages come and go but the teaching strategies used to teach its basic concepts stay. Thus, experts recommend to continue focusing on interventions in a regular classroom setting foster computational practices [18]. With the shift to online education due to pandemic, it is also important to modify such interventions and make it applicable to the current setup of teaching and learning process. Critical success factors for online learning during COVID-19 showed technology knowledge management (e.g., using software to facilitate learning) as one of the most significant consideration among educational institutions [19]. In addition, gamification was proposed as a technique to produce a sustainable learning environment amidst the COVID-19 pandemic [20]. With such method (e.g., the use of game elements such as leaderboards, badges, points, and virtual currencies), the positive impact on students’ self-efficacy, grade motivation, self-determination, and career motivation are evident. As well, a literature review from 2014 and 2020 on gamification applications in an online learning shows that gamified courses could be a valuable tool for acquiring knowledge and skills such as decision-making, communication, and cooperation [21]. When gamification is adopted in a computing course, there are important findings and considerations based on existing studies. First, interface of the game should be user-friendly to allow players to easily understand the game mechanics and focus on learning [22]. It is also a significant factor that affects players’ gameplay [23]. On the other hand, because the levels of difficulty (e.g., easy, moderate, hard) inculcate a competitive spirit among players [24], the gamified learning course is recommended to have a wide range of tasks with different level of complexity. This is crucial since there is a positive correlation between students’ academic competitiveness and their involvement behaviors [25]. In the core of gamified programming courses, there must also a significant role for the development of problem-solving skills as the lack thereof is one of the reasons of demotivation [26]. Lastly, the impact of gamification is not only limited to the actual gameplay but also when students were instructed to develop their own game [27]. This is further validated by the utilization of game development in a recent study as a method for acquiring Information and Communications Technology related skills such as computer programming [28].

Methodology

Research Design

Following the research design used in evaluating Jigsaw teaching strategy in a computer programming course [3], we adopted a quasi-experimental research using a nonequivalent control group pretest-posttest design. Like a true experiment, a quasi-experimental design tests causal hypotheses but with the absence of a key ingredient: random assignment. Though, instead of randomization on a student level which may cause treatment contamination, a cluster (by section) randomization was adopted to prevent selection bias. Furthermore, students enrolled in their preferred class schedule, and the researchers including computer programming professors had no control over their course and section assignments. Lastly, this study was conducted in accordance with the ethical principles in the Declaration of Helsinki and of the University. All participants were asked and agreed to be part of the experiment.

Sample and Intervention

Sophomore students who were enrolled in an Integrative Programming and Technologies (IT0011) course from April to July 2021 were recruited in the study. Because this course is the fifth programming-related subject in the curriculum, all students have prior programming experience. Among the ten sections enrolled in IT0011 during the time of the study, we randomly selected and assigned two sections with 50 students each to participate in the study (N = 100). The same syllabus was used for both groups, but the experimental group had an additional gamified tasks throughout the semester using the CheckiO classrooms [29]. This gamified course platform for Python programming provides fun coding challenges which can serves as an additional resource for teaching and learning Python programming. Puzzles that are aligned with the lesson and student outcomes were given as an additional activity for the experimental group. Nevertheless, scores from these tasks did not affect students’ final grade for lecture and laboratory.

Instruments

Two types of data (cognitive and affective scores) were collected from four different sources: (1) summative tests for knowledge, (2) technical summative assessments (laboratory activities) for skills performance, (3) Computer Programming Self-Efficacy Scale [30] instrument for self-efficacy, and (4) Attitude Scale of Computer Programming Learning [31] tool for attitude. In addition, demographic information (e.g., age, gender, college life satisfaction) was also incorporated on the final instrument. Both instruments for affective factors have a Cronbach’s alpha ranging from 0.84 to 0.96.

Data Collection and Analysis

The survey instrument was distributed one week after the course orientation (April 29, 2021) to the experimental group (pre-test), and both experimental and nonequivalent groups completed almost the same survey instrument (post-test) on the last synchronous class session (July 19, 2021). Scores on both summative tests and laboratory activities were collected as well with consent from teachers and participants alike. All statistical analyses were performed using SPSS Statistics. In reporting homogeneity and data distribution, independent ttest, descriptive statistics, chi-square tests, and Fisher's exact test were used. Finally, independent t-test and paired t-test were used to compare score differences in all factors.

Results and Discussion

The purpose of the present study was to assess the effect of a Python programming gamified course on both cognitive (knowledge and skills performance) and affective (attitude and self-efficacy) domains in higher education. Participants (see Table 1 for baseline characteristics) were dominated by male students (93.75%) and the mean age was 22.56 ± 1.33 years. All participants experienced gamification in different platforms, including those in an online course (experimental group: 68% vs. control group: 58%, χ² = 0.91, p = .334) and in a physical classroom (experimental group: 24% vs. control group: 12%, χ² = 0.34, p = .332). Majority of the participants, from both experimental and control group (96%, χ² = 0.41, p = .292), are satisfied with their college life. Furthermore, both groups (experimental group: 58% vs. control group: 64%, χ² = 0.02, p = .593) are satisfied with their present online course experience. Lastly, for previous programming course grade, the experimental group received a 2.5 (34%) while the control group received a 3.0 (30%) grade.

Aside from the between-analysis as shown on Table 2, we also performed a within-analysis in terms of affective factors. Using paired t-tests to identify whether there is a significant difference on the pre- and post-test scores of the experimental group, we determined mixed findings. First, students’ attitude did not significantly improve after using the gamified course (willingness: p = 0.104, negativity: p = 0.626, and necessity: p = 0.061). In contrary, students’ self-efficacy did improve after using the gamified course (logical thinking: p = 0.000, algorithm: p = 0.000, debug: p = 0.001, control: p = 0.022, and cooperative: p = 0.019). As a supplemental perspective, we also plotted the mean grades of the groups from their most recent programming course as pre-tests (knowledge = lecture, skills performance = laboratory) for comparison.

Knowledge and Skills Performance (Cognitive)

Although not significant (p = 0.525), the mean knowledge score in the experimental group was 82.56 ± 9.35 which is higher compared to the 79.53 ± 8.26 in the control group. This result indicates that the use of a gamified course for learning Python programming does not affect the extent of knowledge gained by students. This result echoes a previous research that evaluated gamification in C programming course, where the gamified learning approach has a positive effect but with lack of statistical significance on students’ knowledge [32]. On the other hand, the mean skills performance score in the control (84.35 ± 15.88) and experimental (94.23 ± 6.89) groups were statistically significantly different (p = 0.042). This discovery indicates that gamification in computer programming is more effective when it comes to hands-on activities (e.g., coding) than gaining theoretical understanding. This can be partially attributed to the many coding challenges readily available in the gamified course, which a player needs to solve in order to progress to another world (game level). In addition, problem solving is a common learning difficulty among students [33]. Therefore, by reinforcing coding activities, students are more likely to manifest their metacognitive control skills in search for different solutions to the present machine problems. It is therefore recommended for future adopters of gamification in computer programming (whether Python or other languages) to incorporate learning tasks within the game mechanics.

Attitude and Self-Efficacy (Affective)

Gamifying the programming course elicit a significantly different scores for both attitude (p = 0.001) and self-efficacy (p = 0.019). The attitude score (4.14 ± 0.76) of students from the gamified course of Python programming was higher than the score (2.94 ± 1.19) of students from the control group. In learning computer programming, a positive attitude towards the course is vital for students to succeed [3]. When students have a positive attitude, it improves their learning efficiency, and are more willing to learn and practice coding out of their own willingness. On the other hand, under a negative attitude, students may struggle to learn new concepts proactively and may result in a dislike of programming [34]. Therefore, the positive attitude recruited by gamifying the course holds an important implication in computer programming education. One possible explanation is the programming experience they acquired from a series of coding challenges and tasks as part of the game mechanics. In an exploratory study of computer programming in the 21st century, programming experience is an important factor that influences student’s attitude [35]. In the case of self-efficacy, the experimental group (4.22 ± 0.94) has a significantly higher score than the control group (3.88 ± 0.97). This outcome supports the literature asserting that the participation of individuals in a gamification program wields a positive influence on perceived self-efficacy [36]. Because students with high self-efficacy are more likely to accept the challenging tasks than students who have low self-efficacy, it only means that the more programming tasks they receive and solve, the higher their self-efficacy becomes. In a traditional programming classroom where time is limited, teachers give enough machine problems for students to solve in the class. This is different from a gamified course where many tasks are already available for students to solve. Moreover, because it is in the game mechanics to solve these tasks to progress the level, students are more motivated to put forth more effort to accomplish the machine problems [8]. Similar findings were found in gamifying basic Java computer programming [37].

Conclusion

The present study brings attention to the effectiveness and important role of gamification in computer programming. By using a Python programming gamified course, we found out that students from the experimental group had significantly higher scores in skills performance (hands-on activities) than students from the control group. They also had higher scores in terms of theoretical understanding (knowledge) although not significantly. Conversely, both attitude and self-efficacy of students who participated in a gamification program were significantly higher than students who had a traditional setup. It is therefore recommended to use gamification as part of an intervention strategy in teaching computer programming and for fostering computational practices. These results contribute to the existing thread of discussion in computer programming education and gamification (individually and collectively).

One advantage of this study was that it examined the use of gamification in both cognitive and affective domains. We also performed the experiment for the whole term, rather than getting a single set of pre- and post-test scores for comparison of the treatment intervention. Nevertheless, our study has still limitations that future research may address. First, we utilized an existing gamified course where we do not have control on any of its aspect from learning contents to game mechanics. Thus, we cannot tailor the gamification program according to our needs (e.g., personalized contents and challenges) and the capabilities of our learning management system (e.g., points and leaderboard within the institution). Meanwhile, caution should be exercised so as not to overgeneralize the results due to a relatively small sample size. Although, it was appropriate to still run the statistical analysis of the study. Finally, future researchers should try other game elements (e.g., badges).

Despite these limitations, our study demonstrates the vital role of gamification in the context of computer programming course, particularly during the time of a pandemic. Although online learning has been widely accepted in the past [38], the shift to emergency remote learning brought new challenges for teachers and students. As the present situation continues to demotivate students, educational leaders and policymakers should introduce interventions to inspire purpose, motivation, and confidence in their academic communities. Educators as well are encouraged to develop innovative strategies to adapt current learning to new academic needs [39]. Gamification, an engaging strategy to deliver curricula materials, is one way to encourage a positive behavior during this negative time.