Automating Android touchscreen interaction

Today I found myself in need of a way to automate touching the screen on my Android phone. An app I have required pressing a specific pattern on the screen with a few seconds interval for a period of time, and as you may already have guessed I decided to find a way to automate this process instead of doing it manually with my fingers.

Step 1 – ADB

First of all you need Android Debug Bridge (ADB) installed on your PC and your phone connected with a USB cable. ADB is a command-line tool included in the Android SDK Platform-Tools that lets you communicate with your phone. This tool is widely used for custom ROM’s and bootloaders so you will easily find better guides on how to install this than I will be able to cover here.

Step 2 – Catch the touch events

Once you’ve got ADB installed you can use the GETEVENT command to start catching events on your phone.

adb shell getevent -lt

This should output a list of event handlers. On my OnePlus 6 the touchscreen is called /dev/input/event2 (synaptics,s3320). ADB is now ‘listening’ for events on your phone and will output it to your PC as soon as you touch the phone screen.

Look for ABS_MT_POSITION for the X and Y coordinates. Note that the values are in HEX. You need to convert these to DECIMAL for the next step.

I’ll be using PowerShell to convert the X and Y to decimals by prepending it with “0x” like this. The result should be 696 and 1158 in decimal.

[System.Convert]::ToString(0x2b8, 10)
[System.Convert]::ToString(0x486, 10)

Step 3 – Let ADB push your screen

Once the coordinates are converted to decimal you can use ADB to touch / tap the screen programmatically.

adb shell input tap 696 1158

Repeat Step 2 until you got all the coordinates that you need to push and wrap it all up in a small PowerShell script like this, that also takes care of converting HEX to DEC on the fly.

$coordinates = @( ('0x22f','0x301'),
                  ('0x219','0x434'),
                  ('0x237','0x583'),
                  ('0x20f','0x6cf')
                )


foreach ($xy in $coordinates) {
  $decX = [System.Convert]::ToString($xy[0], 10)
  $decY = [System.Convert]::ToString($xy[1], 10)

  write-host "Pushing the screen..."
  write-host "X:" $xy[0] "($decX)"
  write-host "Y:" $xy[1] "($decY)"
  write-host ""
  adb shell input tap $decX $decY
}

Photo by MOHI SYED from Pexels