Wednesday, 9 April 2014

Maths Monsters - Feedback and Evaluation

INTENDED GAME SPEC

After brainstorming this was the intended game that I had in my head.

Originally inspired by pokemon games, the intention was to have an area in which the player would randomly encounter monsters and fight them with their own monster by answering maths questions. This idea could be shifted to make it work for almost any age group but I decided I wanted to target 11-14 to ensure that I could assume a certain level of mathematics competence.
Realizing the scope of such a thing would be too great, I simplified it to a single "battle" with a succession of opponents. I added a score system to track progress and the idea was that the game would randomly generated four numbers: A, B, C and D. Using a set of those three numbers, the game would randomly use two math operations, either addition, subtraction or multiplication. I would eschew using division as it would make the game much more difficult. By equating all this information, the game would produce a Target Number, which the player would have to make by using the same set of randomly generated numbers and maths operations.
If the player get the answer correct they would gain a point towards their score and the enemy would take damage to its health, which would start at 3 and possibly increase as the player defeated more monsters. To implement a "lose state" I added a timer which would start at 15 and tick down to 0, at which point it would deal damage to the player and reset to 15. The player would also have 3 health and once it reached 0, would lose. The timer would reset whenever the player took damage or took got a question correct.
The questions would span on to infinity; there would be no win state, only an eventual loss and a highscore.

Of course, due do the nature of the time constraints, what I ended up with was slighlty less impressive.

The questions do not span infinitely: there are 9 preset questions and once the player completes them, they win. This was a great loss because the game was much less replayable and actually changed the difficulty of the game because the player could restart with prior knowledge.

FEEDBACK

Video game production is an iterative process. Once I had a playable game from start to finish I needed feedback from people not involved in its production. I recieved feedback from four play testers: Teri, Reece, John and Karl.

The question sheet I created in preparation of this stage contained what I felt were targeted but open questions, design to elicit more than one a word answer without being so vague as to overwhelm the tester into giving less useful feedback. In most cases this didn't really work but I am unsure as to whether that is a result of poor questioning or inexperienced testers.

---

What is the first thing that stands out to you about the game?

First impressions are always incredibly important, in every situation and while sometimes a poor first impression can be overcome, more often than not, the first impression is what sticks. As such, I wanted to gauge peoples initial reactions and attempt to figure out two things: Is it generally positive or negative? And, what is the stand out feature to people seeing the game for the first time?

Teri's response was :
"The art style, it’s very unique"

Reece's response was:
"The animation of the character is good because it brings it to life."

John's response was:
"The way it looks, it’s unique and the design is very professional."

Karl's response was:
"The animals you defeat and the difficulty of the questions"

From these responses I can gauge that the overall impression seems to be positive and that the aesthetics are the first thing that stands out to most people; whether that be the "unique" visual style itself or the characters. The feedback itself is short and there appears to be a reluctance to be negative or critical, as is further evidenced by later responses, which reduces the usefulness of the feedback.

---

Do the rules and gameplay make sense? Is there anything you feel could be more intuitive?

This question stemmed from a worry after writing out the instructions for the game. It seemed, long, verbose and unfortunately, unnecessarily complicated.  I wanted to figure out whether that was the case, what the main issues were and some possible solutions to those issues.

Teri's response was:
"Yes, all the rules make sense, as does the game play"

Reece's response was:
"Yes they do it took a will to get used to but it I got the hang of it."

John's response was:
"The rules were simple and easy to understand, just the time and rules makes you think very hard about the answers to the questions."

Karl's response was:
"Considering this feels like a 14 year old game not really because it makes it more addicting"

Overall, testers said that the rules and constraints of the game were not actually a problem. Some said it was difficult but got "got the hang of it" in the end. However, in practice, I noticed a great degree of difficulty in understanding and applying the rules of the game. People repeatedly didn't use the required 3 numbers and would often try to use the same math operation twice, neither of which is allowed in the final game.

---

What do you think about the visuals? Are they appealing and readable?

Here I was attempting to ascertain how distinct different visual elements were and how the visual style would be recieved. If there were any problems I would, of course, take them into account.

Teri's response was:
 "Very appealing, however the instructions seem too long/ small to read"

Reece's response was:
 "Yes the text and questions are clear and they colours are contrasting."
 
John's response was:
 "The visuals were a strong point to this game, very cartoony yet nice looking and blends in well with the games design."

Karl's response was:
"Yes they are"

First point of real contention from the feedback, Teri said that the instructions were "too long" and too "small". Looking at the instructions screen I agree entirely. As I feared previously, the insctructions are overly verbose and having the text be so small definitely wouldn't help people. I also worry that the wall of text means the player is less inclined to actually read it and take it in, whereas if it were broken up ages a few pages it would be much less intimidating.

The rest of the feedback was very positive and overall I am very proud of the quality that I achieved visually.

---

What about the sound and music? Is there anything out of place? Is the music unobtrusive?

When I created the music track for the baackground, I didn't have a great opportunity for experiementation: trying to write music without an actual keyboard in front of me was surprisingly difficult. As such, I kept the music very simple but also, very short. I was very worried that the loop would get very repetetive. 

Teri's response was:
"Very simple yet it suits the game"  

Reece's response was: "No there was nothing out of place and the music went well with the game."

John's response was:
"I liked the music, makes playing it less empty and the sound effects were simple yet effective."

Karl's response was:
"No it isn’t it suits the game very well and gives a very nice background theme which helps players concentrate because it blocks off the rest of the world"

Again, it appears my fears were unfounded as all the testers said that the audio was appropriate with the games style and wasn't at all annoying.

 ---
How is the difficulty? Does it tend towards later questions being more difficult? And are the hardest questions too hard, the easy ones too easy?

The difficulty curve in the game is quite difficult to approximate due to the relative difficulties people have with different questions. Nevertheless, I tried to frame the question in such a way that I could gauge where people were struggling.

Teri's response was:
"Later questions do seem to be more of a challenge, however that is what makes this game good, it gives people more of a challenge as you go on, so you learn how to solve more difficult equations/ problems"
 
Reece's response was:
"It was not too difficult and the questions get harder throughout the game, however because there was a timer it gave me a challenge which made it harder for myself."
 
John's response was:
"I found the game difficult due to its time limit and the rules in place but, I believe once you play for a bit, you get used to it so, the difficulty is good enough."
 
Karl's response was:
"I think the difficulty for age group is perfect because it makes it challenging which is good practise for maths"

The response was people was that actually the difficulty was just fine... which seemed to contradict the failure rate. I had intended the game to randomly generate its questions. If that had been implemented, many of the testers would not have gotten past the second question. This time, it seems, observation trumps verbal (or written) feedback. I futher tested it at home with family and both my younger brother, who is sixteen, and my mum and step-dad who are 30+ all struggled with the game. They made a few suggestions that I will touch on later.

Difficulty is definitely something that needs adressing.

---


Did you notice any glitches?The intention of this question should be blindingly obvious.

Teri's response was:
"No glitches"

Reece's response was:
"There were no glitches through the game and it ran smoothly."

John's response was:
"No."

Karl's response was:
"nope"

Currently the game seems gltich free... of course, thats not the whole picture; the music loop, for one, doesn't loop properly. That seems to be an issue with the individual browsers since the problem is much worse in Mozilla Firefox than it is in
Internet Explore. As such I'm not entirely sure what I do about it.


---

What age group would you guess this is aimed towards, keeping in mind both aesthetic, gameplay, context and difficulty?

Here I was trying to ensure that I was targeting my desired demographic correctly.

Teri's response was:
"I would say ages, 8+"

Reece's response was:
"I would say it was aimed at a 9 year old because the questions were not too difficult, the colours and the way the characters looked friendly."

John's response was:
"I would say, 11-14 due to the challenge of thinking under pressure and the rules in place."

Karl's response was:
"12-14"

My actual target was 11-14 year olds. As such, only John guessed correctly, Karl was close and Teri and Reece underestimated my targeted age group. I am unsure as to Teri's reasoning but Reece's response about "the colours and the way the characters looked friendly" leads me to think that it was the child-friendly aesthetic that led it to look like it was targetting a younger audience than intended.

---

Any further comments?

This was a chance for the testers to give criticism about the game in case they had noted something that hadn't been adressed in any of my own, targeted questions.


Teri's response was:
"No, good game!"

Reece's response was:
"No"

John's response was:
"N/A"

Karl's response was:
"Nope great game very addicting and enticing"

Again, very little in the way of useful commentary.

---

PERSONAL THOUGHTS

After recieveing a lot of feedback I realised a lot of issues with the games current incarnation. I want to first talk about less important issues and how I would fix those before moving onto some more major and conceptual issues.

One issue is the difficulty and complication of the rules. They were not intuitive as shown by me needing to explain it to every one who played. Of course, the rules were on the front page... but the text was far too compact on one page. To fix this I would increase the font size and spread the information across a number of "pages" which the player could click back and forth through with screenshots and diagrams to aid the explanation.
Also, a suggestion I received from my younger brother was very good. He said that when the player clicks "card" it should grey out or demonstrate visually that it can't be used again. I believe this is a good solution for those who skim read the instructions.

Another issue that wasn't brought up by any tester but is something I noticed myself, the player would focus entirely on the bottom section of the screen, only glancing up at the Target Number. This created two problems. The player would struggle in keeping an eye on the timer and as such would never know how much time they had left. And secondly, all the work put into designing, creating and animating characters/monsters was wasted since the player never looked at them. I came up with two solutions, the effectiveness of which are yet to be determined. One would be to move the GUI so that it surrounds the characters and also increase the size of the characters. This would also help with the issue of empty space not being utilised. The other solution would be to move the enemy monster closer to the player as the timer ticked down, visually giving the player a clue as to how close they are to losing health.

Also, the lack of attack animations for the characters means the action is essentially non-existent. The player is very removed from the characters on the screen and while the audio cues help, visual elements would improve the experience drastically.

How would I improve the game? I could implement these changes but I think the game as a whole needed to be different. If I had to time to improve it, I would start again. I would use only two numbers to create the final answer and randomly create the questions to improve replayability.This would also have the added effect of lowering the difficulty and making the game much easier to create and therefore polish. Also, it would give me time to look into creating a dynamic difficulty curve that would increase the difficulty as the player answered more and more questions. Another thing would be to change up the UI, and increase the character size to fill up more of the screen.

Overall, the final project is still a fun game but not what I had originally intended.