Findings

Once the participants answered the pre-questionnaire, we handed out the task list to them and began the usability testing.

At the start, Participant 05 (Cyndaquil) and Participant 01 (Bulbasaur) faced issues with their avatars moving on the screen without any actual movement in real life. Therefore, catching Pokémon became more difficult as their screens tracked their characters.

The GPS confusion regarding the player's location and movement led to a shortage of Pokémon spawns. Participant 01 did not receive any GPS error messages, while Participant 05 received a "slow down" message despite not being in a vehicle, resulting in user frustration.

“My guy is a little confused, he’s running around the place” (Bulbasaur 01).

“It thinks I’m a passenger, but I’m definitely not. Not even moving” (01 Bulbasaur).

“It’s just running and there’s no pokémon spawning” (05 Cyndaquil).

The beginner participants mainly focused on using the pokéball menu to go through each task of the task list. The other four button options at the top right screen in the menu was viewed much less because the main focus was on the 5 bigger buttons on the bottom. This confused and frustrated Participant 02 (Charmander) and Participant 03 (Squirtle) when they tried to complete these three tasks (favorite a postcard, check nearby raids, and check the time they caught their first Pokémon).

“I think raid should be under the pokéball button, because I didn't really know about the nearby button so I would have just looked around on the screen to see them” (03 Squirtle).

“I wish they put more things in the pokéball button, as it’s the easiest way to interact with the interface” (03 Squirtle).

Beginner players found it challenging to heal and transfer multiple Pokémon on the same page because there were no instructions provided during the tutorials. Even though it is possible, they were not aware of where to locate the feature.

The new players eventually learned that to heal their pokémon, they needed to access the item menu and choose a healing item. This would take them to a separate screen where they could heal their chosen pokémon.

“I clicked on a pokémon, I see new attack and power up. I don’t see potion, so maybe I'm gonna leave and click the pokéball, then items, then potion. Now I see pokémon.” (02 Charmander).

The beginner participants experienced a significant increase in the duration of Part 1 Task 5. This was mainly because the advanced participants were aware of the tap-and-hold function, which allows for selecting multiple Pokémon, but it is not clearly evident and is not mentioned in the tutorials.

“Ok, transfer. I’m gonna click pokémon then bulbasaur. Ok that’s not what i need.. oh okay transfer..” (02 Charmander).

Inefficient Gift Sending/Opening

Although sending gifts was not apart of the task list, participant 04 Chikorita wanted to express his struggles and frustrations with bulk gift sending. Sending each gift separately to every friend, unlike transferring or healing multiple Pokémon at once, was a tedious task for him.

• “Opening and sending gifts is hard, there’s a max limit of how many gifts you can open and send a day. I have like a hundred friends, so opening and sending gifts to a lot of people will take me over ten minutes.” (Chikorita 04).

• “Definitely should be more streamlined like how you can transfer twenty pokémon at once, you should be able to send twenty gifts at once” (Chikorita 04).