Python GUI Automation - Google Chrome dinosaur game

Let's automate the Chrome Dino game using Python. it’s easy and very exciting. Don't worry we will write only 30 lines of code to automate the game.

Libraries used:

• PIL: Python Imaging Library (PIL) for screen image processing to detect objects.

• Pyautogui : for controlling the mouse and keyboard to automate actions.

Script capable to detecting cactus, birds and night mode.

Before we start writing code we need to install pyautogui and PIL for GUI automation.
pip3 install pyautogui
pip3 install pillow

 

Play dino games in the Chrome browser without disconnecting the Internet. > chrome://dino

Box1 : we make a single rectangle only to identify both cactus and bird. If a bird is flying above the height of our hero dino, then dino needs to do nothing. But if a bird is flying below the height of the dino, then the dino will jump to protect itself.

if any gray pixel comes inside this box area or exists then we assume that either “cactus” or “bird” is so close to dino and will press the "UP" key programmatically using PyAutoGUI. 

Box2: we make the second rectangle to identify the night mode based on the background color.

DEMO :

The automation script is still not perfect, there are many aspects that need to be fixed. So sometimes our bot scores less than 150 and sometimes crosses the score of 1000. Here is a 4 minute Demo Test of our automation script to get that idea of accuracy

Python code to automate Google Chrome dinosaur game :

import pyautogui
from PIL import Image, ImageGrab
import time

def hit(key):
    pyautogui.keyDown(key)


def isCollide(data):
    for i in range(186, 520):
        for j in range(474, 550):
            if data[i, j] < 100:
                return True
    return False

def isCollideInNight(data):
    for i in range(186, 520):
        for j in range(474, 550):
            if data[i, j] > 165:
                return True
    return False


def isNightMode(data):
    for i in range(200, 300):
        for j in range(660, 700):
            if data[i, j] < 100:
                return isCollideInNight(data)
    return isCollide(data)



if __name__ == "__main__":
    time.sleep(3)
    while True:
        image = ImageGrab.grab().convert('L')
        data = image.load()
        if isNightMode(data):
            hit("up")

    # draw a shape and show image for debug 
    # for i in range(186, 425):
    #     for j in range(480, 558):
    #         data[i, j] = 165
    # image.show()