Listening Machines, and the whether, when and how of new technologies
Ethan Zuckerman wonders about ‘listening machines’ like Hello Barbie, Siri, and Amazon Echo:
The company behind Barbie’s newfound voice, ToyTalk, uses your child’s utterance to help tune their speech recognition, likely storing the voice file for future use.
And that’s the trick with listening systems. If you can imagine reasons why you might not want Mattel maintaining a record of things your child says while talking to his or her doll, you should be able to imagine the possible harms that could come from use, abuse or interrogation of other listening systems. (“Siri, this is the police. Give us the last hundred searches Mr. Zuckerman asked you to conduct on Google. Has he ever searched for bomb making instructions?”)
As one of the speakers put it (we’re under Chatham House rules, so I can’t tell you who ), listening machines trigger all three aspects of the surveillance holy trinity: they’re pervasive, starting to appear in all aspects of our lives; they’re persistent, capable of keeping records of what we’ve said indefinitely, and they process the data they collect, seeking to understand what people are saying and acting on what they’re able to understand. To reduce the creepy nature of their surveillant behavior, listening systems are often embedded in devices designed to be charming, cute and delightful: toys, robots and smooth-voiced personal assistants.
What happens when systems like ShotSpotter, currently designed to identify shots fired in a city, begins dispatching police to other events, like a rowdy pool party (just to pick a timely example)? Workers in call centers already have their interactions recorded for review by their supervisors — what happens when Uber drivers and other members of the 1099 economy are required to record their interactions with customers for possible review? (A friend points out that many already do as a way of defending themselves from possible firing in light of bad reviews.) It’s one thing to choose to invite listening machines into your life, confiding in Siri or a cuddly robot companion, and something entirely different to be listened to by machines installed by your employer or by local law enforcement.