“Instead of Alexa’s voice reading the book, it’s the child’s grandmother’s voice,” Rohit Prasad, Alexa’s senior vice president and chief scientist of artificial intelligence, said excitedly during a keynote speech in Las Vegas on Wednesday. (Amazon founder Jeff Bezos owns The Washington Post.)
the demo was the first glimpse of Alexa’s latest feature, which – while still in development – would allow the voice assistant to replicate people’s voices from short audio clips. The goal, Prasad said, is to build more trust among users by injecting artificial intelligence with the “human qualities of empathy and affect.”
The new function can “make” [loved ones’] memories remain,” Prasad said. But while the prospect of hearing the voice of a deceased relative may be heartbreaking, it also raises a host of safety and ethical concerns, experts say.
“I don’t feel like our world is ready for easy-to-use speech cloning technology,” Rachel Tobac, chief executive of San Francisco-based SocialProof Security, told The Washington Post. Such technology, she added, could be used to manipulate audiences through fake audio or video clips.
“If a cybercriminal can easily and credibly replicate someone else’s voice with a small voice sample, they can use that voice sample to impersonate other individuals,” added Tobac, a cybersecurity expert. “That bad actor can then trick others into believing that they are the person they’re impersonating, which can lead to fraud, data loss, account takeover and more.”
Then there’s the risk of blurring the line between what is human and what is mechanical, says Tama Leaver, a professor of Internet studies at Curtin University in Australia.
“You won’t remember talking to the depths of Amazon…and its data harvest services whether it be talking to your grandmother or the voice of your grandfather or that of a lost loved one.”
“In some ways it’s like an episode of ‘Black Mirror,'” Leaver said, referring to the sci-fi series that envisions a tech-themed future†
The new Alexa feature also raises questions about consent, Leaver added — especially for people who never thought their voice would be eavesdropped on by a personal robotic assistant after they died.
“There’s a really slippery path to using deceased people’s data in a way that’s both creepy on the one hand, and deeply unethical on the other, because they never considered using those traces that way,” said Leaver.
After recently losing his grandfather, Leaver said he felt empathy for the “temptation” of wanting to hear the voice of a loved one. But the possibility opens a floodgate of implications that society may not want to take on, he said — for example, who owns the rights to the tiny fragments people leave on the ethers of the World Wide Web?
“If my grandfather had sent me 100 messages, would I have the right to enter them into the system? And if I do, who owns it? Does Amazon own that recording?” he asked. “Have I given up the rights to my grandfather’s voice?”
Prasad did not go into such details during Wednesday’s speech. However, he argued that the ability to mimic voices was a product of “undoubtedly living in the golden age of AI, where our dreams and science fiction become reality.”
Should Amazon’s demo become a real feature, Leaver said people might have to start thinking about how their voices and likeness could be used when they die.
“Should I remember in my will that I have to say, ‘My voice and my pictorial history on social media is the property of my children, and they can decide if they want to reanimate that in a chat with me or not?'” Leaver asked wonder.
‘Now that’s a strange thing to say. But it’s probably a question we need an answer to before Alexa starts talking like me tomorrow,” he added.