Automating Mobile Games With A Robot Arm

My Singing Monsters is one of those mobile titles that has users play simple games to earn coins and gems in the usual way. [Anykey] found that his son was a fan of the game, but that sometimes it felt a little rigged. Thus, rather than waste time playing themselves, he set up a robot to do the job for them.

The player must complete a basic but time-consuming memory game. Upon winning, the player gets to choose a prize from 17 mystery cards. The top prize of 1,000 diamonds always seemed to be hidden under another card, leading to the aforementioned frustration.

In order to test if the game was rigged, [Anykey] set up a uArm Swift Pro to play the game, with the robot arm moving a small stylus over the iPad playing the game. The iPad’s video was piped to a PC via HDMI out, going into a Camlink capture card. A Python script using OpenCV was then created to play the game automatically, and log the results of prizes gained along the way.

After over 100 attempts, the robot never managed to pick the right card to score 1,000 diamonds. Given that there are only 17 cards to choose from, one would expect the 1,000 diamond prize to come up several times in that many selections.

It seems then that the prize selection for completing the memory game may not actually be down to picking the right card. Instead, the prize given is selected by some other calculation entirely.

We love a robot playing games at Hackaday, even if it’s as simple as Tic-Tac-Toe. Video after the break.