Constructive Alignment in Teaching Game Research in Game Development Bachelors Programme

Petri Lankoski and Mirjam P. Eladhari

Paper presented at Teaching Games: Pedagogical Approaches – DiGRA 2019 Pre-Conference Workshop(TGPA:DiGRA2019) August 6, 2019, Kyoto, Japan.

Abstract

This paper presents a case study of a Bachelor level game research methods course (15ECTS). The course covers observations, interviews, and introduction to statistical analysis. The course set-up follows constructive alignment design where the aim is that the learning goals, learning tasks, and evaluation are aligned. During the course, students first learn re-search design and later design their research based on a set of examples and conduct data gathering and analysis. The evaluation of the pedagogical approach used is based on students’ learning diaries where the focus is the methods and applying methods. Qualitative evaluation indicates that students can better describe their research designs and analyses.

Keywords

pedagogy, constructive alignment, research methods, game design research

Introduction

Game design students in university programs are in most cases taught iterative processes of design including stages of playtesting (Game Education SIG 2008). They aim to learn and are taught, how to create well-designed games. However, when they in the last parts of their Bachelor’s programme are to write academically for their examination projects, they are met with a new set of challenges: to conduct work that meets scientific formal requirements. In order to do so, they need to work in a structured manner and use sound methods for exploring their research questions, and finally be able to build an argument that they communicate in academic text, for example, their thesis or report.

At Södertörn University while teaching game development students, we saw that students had issues in planning research and applying game research methods in analysis in the Bachelor’s essay course. Even though students had taken a course that covered research methods, it seemed that the methods appeared rather abstract to many of our students. During the supervision, it was hard to instruct the specific stages of analysis as it seemed that analysis was a black box. In 2016, we redesigned the courses teaching research method, attempting at opening the black box by designing assignments for students where they incrementally learn to go tasks that are small in their scope to larger ones. They start with small tasks of data gathering and analysis and go on to replication studies to finally be more able to conduct their own research design in their bachelors’ projects.

This paper discusses the teaching approach we have used to teach game design research and presents a case study of a course thought to students preparing for their bachelor’s projects. Here, we describe the parts of the course that concerns game research, theory and methods, focusing on teaching research methods and leave other parts of the course out. We have tough the game research course we discuss here for three years. During these three years there were 27-34 students participating each year. Before that we had similarly focused research course (with 20-24 students) from 2013-2015 but using a different pedagogical approach

The game course we are discussing is part of three year educational programme in games, and is given in the beginning of the third year. Students participating have prior knowledge of various game analysis methods (e.g, formal analysis of gameplay (Lankoski and Björk 2015) or formal analysis of art (Carroll 1999) and stimulated recall interviews (Pitkänen 2015).

We redesigned the course using the idea of constructive alignment (Biggs and Tang 2011). With constructive alignment, Biggs and Tang mean that the learning task, teaching tasks, and evaluation are aligned. Alignment, as an example, means that a goal, such as to apply a method, is connected to a learning task, and uses the method in the practical task. The following evaluation  focuses on the process of using the method—in contrast to the end result of the task (C.f. Biggs and Tang 2011).

In the course, students are completing a series of assignments, gaining hands-on experience of both gathering and analyzing data from players. They also get to practice how to design research.

Below we present the course in detail and offer some insights into the outcomes. The insights are based on supervision sessions during Bachelors essay course starting after this research course.

Case Study

The students are taught game design research over ten weeks on full time (15 ECTS). In the first half of this period, they learn game research methods through a series of workshops. In the second half, they conduct a longer study, replicating previous research of their choice, or altering the research design of a previous study. This set-up gives students an opportunity to practice designing research studies. Before the redesign, the first part of the course was based on the seminars where various research papers were discussed. Discussions included reflections on the method. The second part of the course was focused on a single method that student could choose. The method was taught during supervision sessions while student conducted a study. Below we describe the redesigned course in detail.

Part 1: Learning Game Design Research Methods

The assignments for learning and practicing methods consists of data gathering and analysis exercises. In data analysis assignment our students get the task to collect data of four participants’ play and play experience. The data gathering happens in our game lab (Figure 1) and consists of observations, surveys, and semi-structured interviews.

Figure 1. An example of recording a play session: a game stream and a two-camera set-up is used.

In forms of materials, students are given:

  • game prototype (in the first course instance, we used three AAA  games instead of the prototype)
  • a script for a data gathering session along with a message that can be used to recruit participants
  • a ready-made questionnaire
  • a semi-structured interview guide.

The game prototype is a horror game containing three cases: a level without a monster, a level with a monster, and a level with a monster and forewarning about the monster.

The first assignment is focused on data gathering. Students are given a pre-formulated research question:

“How does design influence the horror experience?”

The question is investigated with the following foci:

  • How is the the player experience influenced depending on whether a 1st person or a 3rd person point of view is used in the intro cut-scene?
  • How is use of forewarning influencing playing experience?

Students record play sessions in our game lab. The lab is equipped with  Elgato Game Capture hardware which allows for simultaneous recording of the game screen while two cameras record play (cf. Figure 1)

To prepare for the data gathering session, the students are instructed to read about conducting in-depth interviews (Cote and Raz 2015) before the sessions. The students are also instructed to write an assignment diary where they describe their data gathering and compare it against the given literature.

After the data gathering phase each student have a set of data they have collected themselves:

  • Recorded play sessions, with 3 image streams and two audio streams
  • Recorded interviews
  • Questionnaire data.

The data listed above is used in  three seminars focusing on data analysis. The themes for the seminars are:

  •  Thematic analysis of interview data
  • Coding of observation data
  • Quantitative analysis questionnaire data.

 At the beginning of each seminar, we give a short introduction to the method. After each seminar students are asked to document their work and reflect on their learning on the seminar topics.

In preparation for the thematic analysis seminar, the students are instructed to transcribe the interviews they recorded and read method literature of thematic analysis: our own handouts along with Cote and Raz (2015) and Braun and Clarke (2006). During the seminar, students analyze their data in groups using inductive thematic analysis using iterative analysis approach where each iteration is developing analysis towards themes. We provide some practical tools for helping organizing data: mind map, thematic maps, and post-it categorization. While we ask students to describe the resulting themes briefly and provide a thematic map about their results, the focus for the learning diary is in the analysis process. We instruct students to reflect and compare their analysis process to the ones in the course literature.

During the second seminar,  on the topic of the coding the observation data, we look at analysis using emergent codes. This seminar is structured similarly to the thematic analysis seminar. We ask students to read methods hands out (our own material) and conduct analysis using iterative process: finding an initial coding schema and then use that to code schema to analyze the data (and expand the schema if needed). The students are instructed to, when they write about this in their learning diary, focus on the analysis process and reflect upon it in a similar fashion to the previous seminar.

The seminar on quantitative analysis of questionnaire data, however,  focus on interpretation of analysis results, rather than conducting quantitative analysis. As mentioned, the game prototype contains three cases: a level without monster, a level with a monster, and a level with a monster and forewarning about the monster. The questionnaire measures emotional experience. During the seminar the students (in groups) get

  •  A pre-formulated hypothesis stating that the forewarning is the most scary design.
  • Box plots and violin plots of fear scores by conditions.
  • ANOVA analysis of fear scores between groups and pair-wise t-tests.
  • In order to introduce correlation, we also examine the relation between fear and number of game-overs. For this, we show scatter plots of these variables and Pearson correlation analysis results.

The tasks worked on during the seminar is providing interpretations and descriptions on what we can infer from the given figures and analyses in terms of given hypotheses. Here our goal is not to teach quantitative analysis methods per se, but rather general statistical literacy. The learning diary entry for this seminar focuses on the meaning of introduced concepts as well as the test results and their interpretation.

Above-described design is intended to follow constructive alignment (cf. Biggs and Tang 2011). All assignments and seminars include the learning activities (data gathering data analysis, analysis result interpretation) that are tied to the learning goals, and apply methods to data gathering or data analysis. In addition, the learning diary focuses on the data gathering and data analysis processes. This is intended to align learning tasks and learning goals.

Part 2: Applying Game Design Research Methods

When the second half of the course starts, student have fresh memories of conducting these structured  assignments. The task they are given is to  conduct a small study  following a model of a previously published study. We give a list (Harmon-Jones et al. 2016; Azadvar
and Canossa 2018; Busselle and Bilandzic 2009; Lankoski 2016; Hart and Staveland; Wied et al. 1997; Downs and Smith 2010; Bazzini et al. 2010) of alternatives where the method used has been observations, interviews, and stimulated recall (A method we introduced in an earlier course). The potential example studies also include quantitative studies for those who would like an extra challenge, e.g. learning more about conducting quantitative research.

The students should  build their own study based on the selected study as an example of research design. The students can change the research question to something similar and in the case of quantitative study they can also change measurement instrument.

The students write a report based on their study. The report is to focus on method, results, and discussion. The discussion is intended to include a deliberation on theoretical implications on the results as well as reflections about validity and reliability of the results.

In this second part of the course we have the aim that students should analyze the research design of the chosen study and build their own research design based on that analysis. The goal, understanding research design, and the learning task are aligned. In reporting of the study, students are not conducting a literature review. The reason for this is to help students to keep the focus on the learning task in applying research design in practical research task and reporting it.(1) The reason, again, focus on results and discussion on reporting is aligning the learning goal and learning task.

  1. Our previous courses include conducting a literature review. In addition, the Bachelor’s Thesis course is structured so that the students are required to conduct literature review first and motivate their research question on that literature review.

Reflections

The course structure described above has been iterated upon for three consecutive years. In the first iteration we required a more direct replication of the example study from the student. We also instructed the students to find example studies themselves. However, if they chose a study that had obvious methodological flaws, we asked them to choose another one. This approach turned out to be both difficult an unnecessarily restricting for the student’s learning process. Therefore we changed the approach in the latest iteration, providing students with a list of studies to choose from, and allowing for alterations of the research design.

Based on our observations during a five-year period teaching our Bachelor’s Thesis course, we see that the approach during the last three years appears to have improved students’ proficiency in research design and in application of research methods. These are the observations that we interpret as indicators of the improvement:

  •  In supervision, there is less need to discuss on general aspects of data gathering and supervision is focusing more on specific issues of research design.
  • During supervision of data analysis, students talk about specific steps and details of data analysis using relevant concepts.
  • Students enter with more considered and detailed research designs than earlier.
  • The data analyses we see in supervision are more systematic than earlier.

We must stress that the changes we see in student performance might have multiple different reasons other than the learning task design and constructive alignment: we as teachers might have gotten better at teaching the subject or the students are more skilled.

Conclusions

Above we have presented a game (design) research course focusing on research methods. The course is built on the principles of \textit{constructive alignment} by Biggs and Tang (2011) where the learning goals, learning tasks, and evaluation are aligned. Our evaluations of students skill-sets during Bachelor’s Thesis course indicate that the students have a better command on research designs and how to apply research methods in practical work than before. While we cannot claim that the constructive alignment course design is the reason for seeing the positive development, we have observed that our research methods teaching has improved since we started to use the approach.

Bibliography

Azadvar, Ahmad, and Alessandro Canossa. 2018. “UPEQ: ubisoft perceived experience questionnaire: a self-determination evaluation tool for video games” [in en]. In Proceedings of the 13th International Conference on the Foundations of Digital Games – FDG ’18, 1–7. Malmö, Sweden: ACM Press. ISBN: 978-1-4503-6571-0. http://dl.acm.org/citation.cfm?doid=3235765.3235780.

Bazzini, Doris, Lisa Curtin, Serena Joslin, Shilpa Regan, and Denise Martz. 2010. “Do Animated Disney Characters Portray and Promote the Beauty-Goodness Stereotype?: DISNEY FILMS AND BEAUTY STEREOTYPES” [in en]. Journal of Applied Social Psychology 40, no. 10 (October): 2687–2709. ISSN: 00219029. http://doi.wiley.com/10.1111/j.1559-1816.2010.00676.x.

Biggs, J., and C. Tang. 2011. Teaching for Quality Learning at University. SRHE and Open University Press imprint. McGraw-Hill Education. ISBN: 978-0-335-24275-7.

Braun, Virginia, and Victoria Clarke. 2006. “Using thematic analysis in psychology”. Qualitative Research in Psychology 3, no. 2 (January): 77–101. ISSN: 1478-0887, 1478-0895. http://www.tandfonline.com/doi/abs/10.1191/1478088706qp063oa.

Busselle, Rick, and Helena Bilandzic. 2009. “Measuring Narrative Engagement”. Media Psychology 12, no. 4 (November): 321–347. ISSN: 1521-3269, 1532-785X. http://www.tandfonline.com/doi/abs/10.1080/15213260903287259.

Carroll, Noël. 1999. Philosophy of art: a contemporary introduction. Routledge contemporary introductions to philosophy. London ; New York: Routledge. ISBN: 0415159636.

Cote, Amanda, and Julia G. Raz. 2015. “In-depth interviews for game research.” In Game Research Methods: An Overview, edited by Petri Lankoski and Staffan Björk, 93–116. Pittsburgh, PA, USA: ETC Press. ISBN: 978-1-312-88473-1.

Downs, Edward, and Stacy L. Smith. 2010. “Keeping Abreast of Hypersexuality: A Video Game Character Content Analysis.” Sex Roles 62, no. 11 (June): 721–733. ISSN: 1573.1 https://doi.org/10.1007/s11199-009-9637-1.

Game Education SIG, -. 2008. IGDA Curriculum Framework: The Study of Games and Game Development. Technical report. https://cdn.ymaws.com/www.igda.org/resource/collection/0DBC56DC- B7CB- 4140- BF3A- 22A9E92EC63A/igda_curriculum_framework_2008.pdf.

Harmon-Jones, Cindy, Brock Bastian, and Eddie Harmon-Jones. 2016. “The Discrete Emotions Questionnaire: A New Tool for Measuring State Self-Reported Emotions”, edited by André Aleman. PLOS ONE 11, no. 8 (August): e0159915. ISSN: 1932-6203. http://dx.plos.org/10.1371/journal.pone.0159915.

Hart, Sandra G., and Staveland. NASA Task Load Index. http://humansystems.arc.nasa.gov/groups/tlx/downloads/TLXScale.pdf.

Lankoski, Petri. 2016. “Embodiment in character-based videogames”. In Proceedings of the 20th International Academic Mindtrek Conference on – AcademicMindtrek’16, 358–365. Tampere, Finland: ACM Press. ISBN: 978-1-4503-4367-1. http://dl.acm.org/citation.cfm?doid=2994310.2994320.

Lankoski, Petri, and Staffan Björk. 2015. “Formal analysis of gameplay.” In Game research methods: an overview, edited by Petri Lankoski and Staffan Björk. Pittsburgh, PA: ETC Press. ISBN: 978-1-312-88473-1.

Pitkänen, Jori. 2015. “Studying thoughts. Stimulated recall as a game research method.” In Game Research Methods, edited by Petri Lankoski and Staffan Björk, 117–. ETC Press.

Wied, Minet de, Kathleen Hoffman, and David R. Roskos-Ewoldsen. 1997. “Forewarning of Graphic Portrayal of Violence and the Experience of Suspenseful Drama”. Cognition & Emotion 11, no. 4 (August): 481–494. ISSN: 0269-9931, 1464-0600. http://www.tandfonline.com/doi/abs/10.1080/026999397379890.

Constructive Alignment in Teaching Game Research in Game Development Bachelors Programme

Lankoski & P Eladhari

A paper to be presented at Teaching Games: Pedagogical Approaches (at DiGRA 2019 Conference)

This paper presents a case study of a Bachelor level game research methods course (15 ECTS). The course covers observations, interviews, and introduction to statistical analysis. The course set-up follows constructive alignment design where the aim is that the learning goals, learning tasks, and evaluation are aligned. During the course, students first learn research design and later design their research based on a set of examples and conduct data gathering and analysis. The evaluation of the pedagogical approach used is based on students’ learning diaries where the focus is the methods and applying methods. Qualitative evaluation indicates that students can better describe their research designs and analyses.

Methods, History, and Impact: Directions in Game Design Research

Forgot to blog about New Research Perspectives on Game Design and Development Education workshop paper.

Hartmut Koenitz, Christian Roth, Elisa D. Mekler, Staffan Björk, Petri Lankoski, Mirjam Palosaari Eladhari, Annakaisa Kultima, & Ben Medler
Abstract
Research into the design aspect of games has proliferated since the early 1970s. Currently, early historical overviews appear and categorical divisions within the field become more pronounced. It is therefore timely to reflect on the development until today, take stock of the current landscape, and consider future topics. This position paper does so by bringing together seasoned and emerging scholars, as well as practitioners and industry insiders. Together, they consider which topics are already engaged, and what new ones might be necessary. In addition, the paper will discuss the relationship between game design research and independent/ industry practices as well as implications for game design education.

Full-text PDF

2018 DiGRA Distinguished Scholar

I have the honor to be selected as one of 2018 DiGRA Distinguished Scholars.

In 2016 DiGRA established the Distinguished Scholars program to recognize senior scholars in the field of game studies who have been at the forefront of the development of rigorous scholarship, the establishment of game studies and game development programs, and who have made significant contributions to DiGRA itself as an organization.
The complete list of DiGRA Distinguished Scholars: http://www.digra.org/the-association/distinguished-scholars/

A simple save system in Unity

Saving game data is a task that I need regularly. Using serialization and writing data to a file is an approach that I have used much, but that does not work in WebGL builds. Instead, I wrote data in a PlayerPrefs string. In this way I do not need to keep a details about what objects have a save data to delete or find all save data. The XML string stored contains everything.

Read more

Measuring aggression, the effects of violent games (and other media)

According to Anderson et al. (2010) there is a link between violent games and aggressive behavior. However, that meta-analysis has been criticized. In addition, studies have found little evidence that violent games relates to violent behavior in real life, for example, as aggravated assault or homicide. (see my previous post). Again, this is not a systematic review, but more notes to myself on the topic.

According to Ferguson and Rueda (2009) modified Taylor competitive reaction time test of aggression (modified TCRTT) is used to violent game effects. The idea of TCRTT is to measure how much painful noise (or in unmodified version electronic shock) participants are willing to induce to another.

However, Ferguson and Rueda note that there is no standardized version of modified TCRTT. Elton (2016) reviewed the use of TCRTT in 130 publications and found that those publications used 156 different quantification approaches. Ferguson and Rueda (2009) argues that lack of standardized version has a serious consequences: “Researchers (or indeed clinicians) could choose outcomes that best suit their hypothesis and ignore outcomes that do not.”

Tedeschi and Quigley, TCRTT  have more fundamental flaw in its design:

Tedeschi and Quigley 1996 note, the iterative nature of the game may encourage aggressive responding for other reasons, such as reciprocity (fairness) and deterrence (social control), rather than desire to harm. There is also the possibility that very competitive participants might use more severe shocks, not because they want to harm theirmopponent, but to negatively affect their reaction rates and so ‘‘win’’ the game. (cited in Ritter and Eslea, 2005.)

If we “define aggression as behavior that is intended to harm another individual who does not wish to be harmed (Baron and Richardson, 1994, cited in Rajiv and Hammond, 2014) intention to harm should be also evaluated. Laboratory research have a great challenge here because participants are most likely aware that research cannot setup study so that participants could harm someone with their actions in an experiment. (C.f. Ritter and Eslea, 2005.) Also, Milgram experiment at 1960s tells a story, at least, how experiment situation can setup so that participants follow instructions, give electric shocks, even when thinking they cause pain to someone; however, the participants apparently lacked intention to harm.

Breuer, et al. (2015) studied the relation of frustration and aggression measuring aggression with modified TCRTT. They conclude:

The current results corroborate previous research and provide further support for the contention that factors other than violent content play a role in the effects of digital games on aggression. (Breuer, et al. 2015)

Elton, et al. show evidence that game speed, for example, can influence aggression measures used in laboratory research:

The results show that game speed, a feature that tends to vary across games and genres, not only interacts with displayed violence, but also has a direct effect on several outcome variables that might otherwise be misleadingly attributed to displayed violence. The findings of this study demonstrate the importance of controlling potentially confounding factors in experimental research on digital games and points to the importance of further systematic research into what other variables of a game may affect player experience and behavior […] (Elson, et al. 2015)

For the future reading: Ritter and Eslea (2005) provides review of main approaches(including TCRTT) to measure aggression in  laboratory settings as well as critique to those.

References

  • Andreson, et al. 2010. Violent Video Game Effects on Aggression, Empathy, and Prosocial Behavior in Eastern and Western Countries: A Meta-Analytic Review. Psychological Bulletin, 136(2). DOI=0.1037/a0018251.
  • Breuer, J., Scharkow, M. and Quandt, T. 2015, Sore losers? A reexamination of the frustration–aggression hypothesis for colocated video game play, Psychology of Popular Media Culture, 4(2), pp.126-137.
  • Elson, M., Breuer, J., Van Looy, J., Kneer, J. and Quandt, T. 2015, Comparing apples and oranges? Evidence for pace of action as a confound in research on digital games and aggression, Psychology of Popular Media Culture, 4(2), pp.112-125.
  • Elton, M., 2016. Flexible Measures / Competitive Reaction Time Task. Available at http://www.flexiblemeasures.com/crtt/index.php?menu=publications.
  • Ferguson, C.J. and Rueda, S.M.  2009. Examining the validity of the modified Taylor competitive reaction time test of aggression. J Exp Criminol, 5(121). DOI= 10.1007/s11292-009-9069-5.
  • Rajiv, J. and Hammond, T. 2014. Principles of social psychology. Available at https://opentextbc.ca/socialpsychology/.
  • Ritter and Eslea, 2005. Hot Sauce, Toy Guns, and Graffiti: A Critical Account of Current Laboratory Aggression Paradigms. Aggressive Behavior 31, pp.407–419.

 

Violent game effects?

This is not anything like systematic review, but more notes for myself.

The long debate of how violent games effect on behavior. Anderson, et al. (2010) argue that there are violent games increase aggression and lower empathy:

The evidence strongly suggests that exposure to violent video games is a causal risk factor for increased aggressive behavior, aggressive cognition, and aggressive affect and for decreased empathy and prosocial behavior. Moderator analyses revealed significant research design effects, weak evidence of cultural differences in susceptibility and type of measurement effects, and no evidence of sex differences in susceptibility. Results of various sensitivity analyses revealed these effects to be robust, with little evidence of selection (publication) bias.

Hilgard, Engelhardt and Rouder (2017) provide a reanalysis of Anderson, el al. (2010) meta analysis and reach different conclusion:

First, we detect substantial publication bias in experimental research on the effects of violent games on aggressive affect and aggressive behavior. Second, after adjustment for bias, the effects of violent games on aggressive behavior in experimental research are estimated as being very small, and estimates of effects on aggressive affect are much reduced. In contrast, the cross-sectional literature finds correlations that appear largely unbiased. Third, experiments meeting the original authors’ criteria for methodological quality do not yield larger adjusted effects than other experiments, but instead yield larger indications of bias, indicating that perhaps they were selected for significance.

Ferguson and Kilburn (2010) also criticize Anderson et al. (2010) meta analysis, because  studies included does not study violent aggression. Fergunson (2007) elsewhere states that evidence to publication bias in videogame violence effects publications.

On the other hand, Browne and Hamilton-Giachritsis (2008) states that there are evidence that violent imaginary in games has a short term effect to children (increasing aggressive or fearful behaviour), but inconsistent evident of any long term effect or any kind of effect to teens and adults.

Notably, studies looking at the link between violent videogames and violent real world behaviour has not found an evidence that those two are linked. Markey, et al. (2014) did not find evidence that playing violent games and aggressive assaults and homicides are positively linked. However, Browne and Hamilton-Giachritsis (2008) claim that there is “weak evidence from correlation studies links media violence directly to crime.”

A new intervention study comparing effects of playing violent videogame and non-violent videogames to aggression, by Kühn, et al. (2018a) allows draw causal conclusions. Based on their study where participants played Sims 3 (n=24) or Grand Theft Auto V (n=25) daily for two months (and have third passive control group who did not play any game, n=28), they conclude

The present results thus provide strong evidence against the frequently debated negative effects of playing violent video games in adults and will therefore help to communicate a more realistic scientific perspective on the effects of violent video gaming.

Kühn et al. (2018a) states that the sample size in the study is enough to detect the average effect sizes, r=.18 reported Anderson, et al. (2010).

In fMRI study, Kuhn et al. (2018b) where people were randomly assigned to play Grand Theft Auto V (n=26), Sims 3 (n=24) or non play group (n=30). They did not find “any evidence for desensitization in the empathy network for pain in the violent video game group at any time point”.

EDIT 2018/03/20. There is a reply to Hilgard, Engelhardt and Rouder (2017) by Kepes, Bushman and Anderson (2017) and provide another set of meta analyses and claim “As stated in our title, although the magnitude of the mean effects were reduced by publication bias and outliers, ‘violent video game effects remain a societal concern.'” However, real life impact of the violent games is another question (cf. Browne and Hamilton-Giachritsis, 2008; Markey, et al., 2014).

References

  • Andreson, et al. 2010. Violent Video Game Effects on Aggression, Empathy, and Prosocial Behavior in Eastern and Western Countries: A Meta-Analytic Review. Psychological Bulletin, 136(2). DOI=0.1037/a0018251.
  • Browne and Hamilton-Giachritsis, 2008. The influence of violent media on children and adolescents: a public-health approach. Lancet, 365(9460). DOI=10.1016/S0140-6736(05)17952-5.
  • Ferguson and Kilburn, 2010. Much ado about nothing: The misestimation and overinterpretation of violent video game effects in Eastern and Western nations: Comment on Anderson et al. (2010). Psychological Bulletin, 136(2). DOI=10.1037/a0018566.
  • Ferguson, 2007. Evidence for publication bias in video game violence effects literature: A meta-analytic review,Aggression and Violent Behavior,  12(4). DOI=10.1016/j.avb.2007.01.001.
  • Hilgard, Engelhardt and Rouder, 2017. Overstated evidence for short-term effects of violent games on affect and behavior: A reanalysis of Anderson et al. (2010). Psychological Bulletin, 143(7). DOI=10.1037/bul0000074.
  • Kepes, S., Bushman, B.J. and Anderson, C.A., 2017. Violent video game effects remain a societal concern: Reply to Hilgard, Engelhardt, and Rouder (2017). Psychological Bulletin,
  • Kühn, Kugler, Schmalen, Weichenberger, Witt, and Gallinat, 2018a. Does playing violent video games cause aggression? A longitudinal intervention study. Molecular Psychiatry. DOI=10.1038/s41380-018-0031-7.
  • Kühn, Kugler, Schmalen, Weichenberger, Witt, Gallinat, 2018b. The Myth of Blunted Gamers: No Evidence for Desensitization in Empathy for Pain after a Violent Video Game Intervention in a Longitudinal fMRI Study on Non-Gamers. Neurosignals, 26. DOI=10.1159/000487217.
  • Markey,, Markey, French, 2014. Violent Video Games and Real-World Violence:  Rhetoric Versus Data. Psychology of Popular Media Culture, 4(4) DOI=10.1037/ppm0000030.