Shhhh, they’re listening: Inside the voice-profiling revolution – world expert shares insights

by

Thanks to the public’s embrace of smart speakers, intelligent car displays and voice-responsive phones – along with the rise of voice intelligence in call centres – marketers say they are on the verge of being able to use AI-assisted vocal analysis technology to achieve unprecedented insights into shoppers’ identities and inclinations, says world authority on media systems and industries, Professor Joseph Turow of the University of Pennsylvania. He sets out the implications of his in-depth research which finds that not only can people be profiled by their speech patterns, but they can also be assessed by the sound of their voices. Alexa, Siri and other voice-activated technologies could be used to spy on you for advertisers and even governments. According to some researchers, your voice is unique and can reveal feelings, personalities and even physical characteristics. He spoke to BizNews, about why we should be worried about the latest advances amid a podcasting revolution – and think twice about joining groups like Clubhouse. – Jackie Cameron

Professor Joseph Turow on companies using voices to profile consumers:

Increasingly – and this has been taking place over the last couple of decades – companies and other entities are beginning to use voice to try to discover who we are, to make inferences about us and to profile us. This happens most nowadays in contact centres, when we call up companies on the phone. A lot of times, they have software behind it on computers that listen to our voices, make decisions about us, connect that to the history of our purchases, and then maybe triage us to people in the contact centres who are supposedly good at dealing with people with that emotion or that personality. Eventually, Google, Amazon and even Pandora and Spotify have said that they reserve the right to use our voices for their marketing purposes. The question is exactly what does that mean? We’re not sure. What I’m trying to do is alert people to the idea that this, in fact, is coming down the pike, the technologies exist and we ought to worry.

On what companies like Google know about our voices:

If I have an Amazon Echo in my house, I can have the Echo – if I allow it to – know who I am compared to my wife. It can distinguish among people in the home. The question is, what will it do beyond that? Amazon has released a product called the Amazon Halo – it’s like a Fitbit. You put it on your wrist, you wear through the day. It will tell you issues about your heart and your exercise, but it also has a way to figure out what you’re saying to people and how that sounds to them. How do you sound to your boss, your spouse or your kids?

Now, they say, and I believe them, that they don’t use [that] data for any of their own purposes. But to me, this is just the beginning. Down the line, companies – whether it’s Amazon or Google – will very much work on that. There are companies that have a lot of patents in this area. For example, a famous Amazon one is where, in the patent that Amazon has, a woman walks into her apartment and says, “[coughs] Alexa, tell me a recipe for dinner tonight.” Alexa says, “it sounds like you have a cold. Would you like a recipe for chicken soup?”

The woman replies, “no, thanks anyway.” Alexa says,”well, how about I order you aspirin? I can have it delivered by Amazon in an hour.” The woman says that’s a great idea. This notion of using the voice for marketing hasn’t been implemented yet, but Google has other patents in this arena. For example, Google has a patent where you can figure out who is in what room in your house, what they’re doing at the time – based on their voices – and then have a system to control, for example, how much TV your kids watch and how much gaming they do based upon the system knowing where they are in the house and doing what at a specific time.

On other discoveries he has made while researching his book:

I’m very impressed – in a kind of chagrinned way –  about how the advertising community is really worried about getting in on it. They’re worried that Google and Amazon are going to, sort of, freeze them out of a lot of the technology, because they feel that they’re not going to be able to be given room in the Alexa speaker or the Google smart speaker, because those companies will only give room to one or two advertisers, not a whole range.

They’re worried that the whole idea of presenting advertising to people is going to change in these systems and that they won’t have the technological ability to compete. They’re starting their own research in this area. They’re trying to figure out how to get, for example, a person to hear their product when they say, “Google or Amazon, tell me about a mouthwash.” They’re actually doing research on people’s voices to see if they can use their clients apps on phones and websites to do the kinds of things that Amazon and Google might be able to do in their smart speakers.

On the potential of detecting disease via voice:

There are companies which are involved in trying to figure out how voice projects certain illnesses in the body. Alzheimer’s is an area they’re trying to figure out. Even some aspects of heart disease and Covid. There’s some evidence in medical journals that you can use the voice to plumb the body, but it remains to be seen if it’s working. If it works, it would be a terrific advance. The people in Carnegie Mellon did an interesting thing. They reverse engineered – or at least they claim to have reverse engineered – Rembrandt’s voice for the Rijksmuseum in Holland.

They did it by looking at the size of the larynx in self portraits – the cells in the throat – and making inferences about the larynx and the voice box and all the aspects of a voice that might come from it. As a consequence, [they’ve] sort of said this is how Rembrandt would have sounded. Of course, we don’t know really whether that’s true, but they argue that they’ve done it with live people and it works. Therefore, they can tell us what Rembrandt sounded like.

On his concerns surrounding voice detection:

My concern is that they will seduce us into accepting the idea of voice as just another way to learn about us, [to] give us relevant products and advertising. That we will slide into a biometric era, even more than face recognition. I think people are kind of frightened of face recognition and maybe it wouldn’t take off the way voice will. When you talk to Alexa, you talk to Google assistant – certainly to Siri – they sound pleasant and friendly – not at all threatening.

That kind of world, where we move from a targeted world based on demographics, lifestyle and other kinds of social issues to a targeted world where we’re using biometrics – which often don’t change much over time – to figure out who we are and what we’re saying. That is problematical, I think, for society, because you’re opening an iceberg of issues relating to how we define people and where we get our understanding of them from.

On privacy and surveillance:

privacy and surveillance are often interconnected – but they’re not always. if you have a political party that wants to send you particular commercials, they can make decisions in real time about your voice and send you commercials that are different, depending on what emotions they think you have. I have this idea that at one point in time, some political party will argue that people’s voice prints are public for democratic purposes –  just like a lot of data about people in the voter file. For voter files, they might be able to use my voice or somebody might collect voices in various places online – like clubhouse or other meet-ups – and use those voices connected to individuals to make inferences about them. That can be used in campaigns for politicians as well as for regular products.

Read more:

Visited 833 times, 1 visit(s) today

BizNews