Android wasn't built for a second user. AI agents on 45 real phones

Opposable is an Android app that lets an assistant like Claude or ChatGPT use the phone it's installed on. The assistant asks for a screenshot or a list of what's on the screen, then taps, swipes and types, while the owner watches or puts the phone down. Between 8 October and 3 November, assistants made 3,324 of these actions on 45 phones through our Claude, ChatGPT, Claude Code and Codex connectors. Nearly half of what went wrong, 140 of the 305 failed actions, came from Android or the phone rather than from the assistants.

The numbers come from the activity log the app shows each phone's owner, which the relay keeps so the owner can look back at what an assistant did. An entry records the tool, which assistant called it, when, and whether it worked. It never holds an action's arguments, so there's no typed text, address or search in it. A failed action also keeps the first 120 characters of the error message, which sometimes names the app that was open. The relay keeps the latest 200 entries per phone. For this post we counted entries across all 45 phones and grouped them by the phone maker the app reports when it registers, so every figure below is a count and none of it describes a particular person.

#TL;DR

91% of the 3,324 actions worked. Android is good at being seen and pressed, with 99% of taps and swipes doing what was asked. Most of the trouble came from everything around that: of the 305 failed actions, 108 failed because the phone had dropped off the internet, and 27 of the 45 phones did that at least once while an agent was using them. Typing needs a focused text field, the same app has different names on different brands, and Android can't tell an agent's taps from the owner's, so it has no way to keep an agent off a screen or ask the owner first.

#Staying reachable

The assistant reaches the phone through our relay. The app keeps a connection open to it, and each request arrives over that connection. Android's power management is designed to stop long-lived background work on an idle phone: Doze and App Standby defer it, and several phone makers add their own battery managers that also close apps Android would leave running.

In the log, 36 of the 105 sessions ran into a phone that wasn't connected, and 16 sessions ended there with the agent giving up. 7 of 9 Xiaomi phones and 14 of 20 Samsung phones dropped off at least once, against 2 of 5 Pixels and 1 of 4 Motorolas. The log can't tell a battery manager closing the app from an owner switching the phone off or losing signal, but the split by maker lines up with dontkillmyapp.com's ranking of battery managers, where Samsung and Xiaomi are both near the top.

The app now runs as a foreground service with a permanent notification, asks once to be left out of battery optimisation, and sets a 15-minute alarm that reconnects if the service was frozen, because Android still delivers alarms in Doze's maintenance windows. On Xiaomi, Samsung, Oppo, vivo, Huawei and Tecno phones it also shows a card with the maker's own setting, such as Autostart on Xiaomi or Never sleeping apps on Samsung, which only the owner can switch on. A phone an agent can reach on demand behaves like a small server, and Android has no setting for that. The battery exemption comes closest, and not every maker's battery manager respects it.

#Permission to act

Android has one general way for an app to read other apps' screens and act in them, the accessibility service, which was built for screen readers and switch access. Turning it on means a trip to Settings › Accessibility. On Android 13 and later an app installed from a browser starts with that switch greyed out until the owner finds Allow restricted settings in the app's info page. 143 of the 192 installs that sent anonymous setup events turned the service on, and we can't tell how many of the other 49 stopped at the greyed-out switch and how many looked around and left.

Android 16 added AppFunctions, which lets an app offer actions to an agent directly without going through the screen at all. It covers only apps whose developers adopt it, so for everything else the accessibility service is still the way in.

#Seeing and pressing

Screenshots worked 95% of the time and taps and swipes 99%. Every action already returns the updated list of elements on the screen, each with a number the agent can tap by, and agents still asked for a screenshot 620 times and tapped by pixel coordinates 426 times against 525 taps by element. Most likely the list is enough where an app labels its buttons for accessibility, and the agent falls back to the picture where it doesn't, but the log doesn't record which apps these were. A tap by coordinates always lands somewhere, so its 99% says less than the 92% for taps by element, which fail when the element the agent named isn't there.

#Typing and finding apps

Typing worked 85% of the time. Android sends text to whichever field has focus, and focus can arrive a moment after the tap that gave it, so 21 times the agent typed before any field had it. The app now waits up to 600 ms for focus, and when nothing is focused and the screen has a single text field it focuses that one. With several fields it returns their names so the agent can tap the right one.

Opening an app also worked 85% of the time, partly because the same app goes by a different name and package on each brand. The clock is com.google.android.deskclock on a Pixel and com.sec.android.app.clockpackage on a Samsung, and agents sometimes asked for one on the other. For alarms, timers and switching Wi-Fi Android has standard intents that work on every phone, and the phone's own on-device agent runs about forty common tasks like these as fixed code instead of tapping through an app.

#One screen, two users

A desktop agent can work in one window while you use another. On a phone the agent takes the whole screen, so the owner can't use the phone until it's done, and the chat app they asked from goes to the background. Agents called return_to_chat 37 times to bring the assistant back at the end. The median session was 14 actions over about four minutes and the longest was 163.

A locked phone stops the agent the same way it stops anyone else, because Android lets nothing past a PIN, pattern or fingerprint. The app turns the screen on and shows the owner's normal unlock prompt, and locked phones caused 3 failed actions.

#Telling agent taps from the owner's

To Android, an accessibility service's taps look like the owner's. No flag marks an action as coming from an agent, so an app can't treat agent input differently, and the owner has no system setting that keeps an agent out of one app or screen. Android 14 added accessibilityDataSensitive, which lets an app hide views from services that aren't real accessibility tools, and FLAG_SECURE keeps a screen out of screenshots, but both are about hiding content and neither lets the owner approve an action.

We built approvals into the app instead: payments wait for the owner to approve them on the phone, and the app's own screens, including the switch that turns those approvals off, refuse agent taps. Over the four weeks agents tapped there 4 times and were refused each time. The log doesn't say what they were trying to do.

#What would help

Two changes in Android would remove most of the red in that chart. One is a supported way for an app to stay reachable on demand that phone makers' battery managers respect. The other is a notion of agent input: a permission the owner grants per app, a marker on each event saying an agent sent it, and a system prompt before actions the owner has marked as sensitive. Typing and app names are smaller problems that apps and agents can work around, and the app already does.

#Limitations

45 phones over four weeks is a small sample, and 3,048 of the 3,324 actions came through Claude, so this mostly describes Claude's habits. Because the log keeps 200 entries per phone, the earliest sessions on the five busiest phones aren't in it. Failures of our on-device model are in the chart for completeness and say nothing about Android.

#Trying it

The app is free at getopposable.com. If your phone drops off while an agent is using it, the app's home screen shows the battery setting your phone's maker needs, and I'd like to hear which phone it was at hello@getopposable.com.