Research

Research is a topic covered in almost any university course because of how important it is for any successful product. A good product needs appropriate data to support the decision making process. The two methods - qualitative (dealing with non-numeric data) and quantitative (dealing with numeric data), as well as a custom hybrid between the two, form the basis for all research (McCusker and Gunaydin, 2015).

When working on an online business, most of the decisions in the company are based on quantitative research. Having a large number of participants also provides enough data to give high confidence in the final outcome of the research. Using data to make educated decisions on feature prioritisation is highly sought for skill in engineers during recruitment.

Image from Wikimedia Commons

I have little experience in conducting qualitative research. I have, however, gathered feedback from peers and requirements from customers as qualitative data. While this is far from formal research, it has highlighted the importance of asking appropriate open ended questions in order to produce rich data.

Ethics

What is ethically good or bad is not only different for every person but also for every situation. History shows that even when people have good intentions the way they conduct their research can harm the participants.

Image from Wikimedia Commons

In 1968, Darley and Latané at Columbia University conducted an experiment where they subjected college student volunteers to a pre-recorded conversation (Darley and Latané, 1968). They were not informed that the conversation was a recording as this could impact their reaction. At some point, one of the pre-recorded conversation participants would suffer a seizure and the researchers would measure the time it takes the subject to seek help. The researchers would correlate that with the number of conversation participants. Based on this data they concluded that people are less likely to help when there are more people around them because they expect somebody else to step in and called this "The Bystander Effect" (Darley and Latané, 1968). However, their methods raise multiple ethical concerns - there was no informed consent given by the participants and there was no proper debrief after the experiment, i.e. the participants left believing they witnessed someone dying.

Similarly, in 1939, Mary Tudor conducted an experiment to find out how different feedback can affect children's speech fluency (Tudor, 1939). Tudor took 22 children from an orphanage, 10 of which showed early signs of stuttering. They were split into two groups and the children with speech fluency issues were split equally between them. The first group was told they do not have a stutter, and that they should ignore criticism on the way they speak. The second group was told that they have a stutter and should not speak unless they can do so fluently. The experiment showed no impact on the speech fluency of the children, however, they experienced a different lasting impact (Tudor, 1939). The group with positive feedback showed more confidence white they speak, where the group given negative feedback became more self-conscious and frustrated (Tudor, 1939)

The negative impact of these projects on their participants highlights the importance of the principles and Ethics Committee approval. While it can produce data to improve people's lives, research can also negatively impact the lives of the volunteers if not done right.

I find the "do no harm" ideology very interesting because it is something I follow as a Software Engineer. Often, when large changes to software have to be done, "do no harm" experiments are utilised to ensure the new system doesn't degrade the user's experience.

Going forward

Viewing game testing as research is a new idea to me. I am going to continue looking at it from that perspective and put more emphasys on following the research principles and ethics. Another very important point for me to take forward is the data gathering. So far I have been focusing on gathering quantitative data through the game itself rather than investigating the player and how they feel. I plan to investigate this data gathering method more to ensure I know everything there is from the players of my games.

References

Darley, J.M. and Latané, B., 1968. Bystander intervention in emergencies: diffusion of responsibility. Journal of personality and social psychology, 8(4p1), p.377.

McCusker, K. and Gunaydin, S., 2015. Research using qualitative, quantitative or mixed methods and choice based on the research. Perfusion, 30(7), pp.537-542.

Tudor, M., 1939. An experimental study of the effect of evaluative labeling of speech fluency.