Tinker Tailor Smartphone Spy: The Apps That Are Secretly Listening

Authored by Jamie Smith and Emily Forrest.

Alphonso Inc. You may not have heard of them, but they may have heard of you, or at the very least, your TV watching habits.

A recent report in the New York Times reveals Alphonso software, which can be found embedded in a multitude of apps and games on both Android and Apple products (some geared towards children), accesses the smartphone’s microphone to listen out for audio signals hidden within TV advertisements and shows – even when the app is not in use and is simply running in the background. This certainly adds credence to those who claim that “their phone is listening to them”.

The data gathered by the software is then sold on to advertisers to enable them to better target their ads based on the content the user has viewed.

A quick search for “Alphonso” on the Google Play store found “Beer Pong: Trickshot” a game developed by Dumadu Games with 50,000-100,000 installs. Indeed, deep in the description is a disclaimer for “Alphonso Automated Content Recognition (“ACR”) software”.

78% of the apps I reviewed (27) from the same publisher, Dumadu, had the Alphonso software embedded and accompanying disclaimer buried in the description.

In response to recent scrutiny on the matter, Alphonso’s CEO Ashish Chordia said “the consumer is opting in knowingly and can opt out any time”.

Although consent to access the microphone is given by the user when downloading the app, it isn’t overtly clear what this access is for without reading the full disclaimer which may be overlooked at the time when the user believes they are simply downloading a gaming app.

The intrusive nature in which the software captures information from individuals whilst enjoying downtime in their own homes or whilst visiting friends is quite unnerving – particularly when the app is capable of capturing information when the phone is locked and in your pocket.

On a more reassuring note, however, the developers do claim that the software isn’t able to recognise or understand human speech (phew) and only picks up commercial audio content. See below.

“With your permission provided at the time of downloading the app, the ACR software receives short duration audio samples from the microphone on your device,” the disclosure reads. “Access to the microphone is allowed only with your consent, and the audio samples do not leave your device but are instead hashed into digital ‘audio signatures.’

The audio signatures are compared to commercial content that is playing on your television, including content from set-top-boxes, media players, gaming consoles, broadcast, or another video source (e.g., TV shows, streaming programs, advertisements, etc.).”

This report yet again shines a spotlight on the unconscionable conduct of certain companies when it comes to the effective handling of consumer data. It is clear that more transparency is needed in relation to what and how data is collected from users.

If you think you may have given access to Alphonso, a guide to opting out is found on the Alphonso website.