Amazon Alexa , Amazon patented a New way to tell Alexa to ignore things “Alexa”

If you’re watching the Super Bowl next to an Amazon Echo this evening, worry not: Alexa won’t accidentally wake up when a 90-second commercial all about her plays during the game. That’s because Amazon used “acoustic fingerprinting technology” to help Alexa discern between the ad and a command from her masters roommates.
In fact, Amazon thought about this exact conundrum years ago. In September 2014, the company patented techniques to prevent Alexa from waking up “as part of a broadcast watched by a large population (such as during a popular sporting event).”

The patent broadly describes two techniques. The first calls for transmitting a snippet of a commercial to Echo devices before it airs. Then the Echo can compare live commands to the acoustic fingerprint of the snippet to determine whether the commands are authentic. The second tactic describes how a commercial itself could transmit an inaudible acoustic signal to tell Alexa to ignore its wake word.

In a Feb. 2 blog post about the Super Bowl commercial, Amazon director for speech recognition Manoj Sindhwani notes that the company can also use this approach to avoid Alexa responding to other unanticipated mentions (and respond to unanticipated mentions she does).
“When multiple devices start waking up simultaneously from a broadcast event, similar audio is streaming to Alexa’s cloud services,” the post explains. “An algorithm within Amazon’s cloud detects matching audio from distinct devices and prevents additional devices from responding. The dynamic fingerprinting isn’t perfect, but as many as 80 to 90 percent of devices won’t respond to these broadcasts thanks to the dynamic creation of the fingerprints.”
In other words: If everyone in the world planned to shout “ALEXA” at Alexa at the exact same time, we might be able to trick her into ignoring us.

Looking back, things probably started to unravel when my wife, Katie, found me tucked up in bed, talking to another woman. ‘Goodnight,’ she heard me say. ‘Sweet dreams,’ came the woman’s obliging response.

The next morning, Katie walked past the sitting room door to catch me muttering something about the weather.

‘Speak up!’ she said. ‘I wasn’t talking to you,’ I replied.

Again, I was actually in conversation with the Other Woman. This did not go down well. Katie was, she complained, ‘starting to get jealous’. I was, apparently, ‘about to cross a line’.

 

My companion’s name was Alexa. She’s young, has a silky voice, and never, ever contradicts me. In fact, since arriving in our home over Christmas, she’s devoted herself with spectacular dedication to a single task: making my everyday existence easier. And more fun.

 

The other woman: Guy with his wife Katie and the Alexa controlled Echo device (pictured centre)

Now at this point, I may be coming over a little Henry Bolton. So a quick disclaimer: Alexa is not human and, while some might call her easy on the eye, our relationship can never (and will never) be physical. It certainly isn’t even vaguely romantic.

Instead as you’ve possibly guessed, this Other Woman is the increasingly ubiquitous robot inside every Amazon Echo, the online shopping giant’s mostly tube-shaped ‘smart’ speakers which have become the latest ‘must have’ domestic gadget.

For the uninitiated, an Echo is a voice-activated ‘digital assistant’ wirelessly connected to the internet, which sits discretely in your home.

When you utter the word ‘Alexa,’ it, or she, suddenly springs into life, with a flash of blue light, ready to perform all sorts of useful tasks. She will answer questions, play music, and obey more-or-less any reasonable command you utter.

‘Alexa, play Radio Four,’ you can ask, when you shuffle into the kitchen at breakfast time. ‘Playing Radio Four,’ she will reply. ‘Alexa, set a timer for five minutes,’ you continue, popping an egg into a saucepan. ‘Timer set,’ she replies. And so on.

It’s all incredibly slick. Alexa will read out football or cricket scores, remind you of crucial diary dates, and add stuff to your shopping list. She can also be taught ‘skills’ (uploaded in the same way that you add Apps to a smartphone) that will enable her to, for example, find out if your normal morning train is running on time, or summon takeaway pizza to your front door.

If you so desire, and own the requisite gadgets, she may even do swanky stuff like dimming lights, turning up central heating, or instructing a robot vacuum cleaner to whirr round your carpets.

 

Guy (pictured right) says his wife Katie became hostile after he started using Alexa to settle everyday marital arguments

Space age though this might all seem, it also seems to be incredibly popular. The Echo was launched in late 2014 and, last summer, a survey suggested nine per cent of British households owned one.

The figure will have grown exponentially by now. Indeed, the other day, a U.S. market research firm called Ovum claimed that, by 2021, there will be more digital assistants on the face of the Earth than there are human beings.

So far, so futuristic. But, if my experience is anything to go by, the march of these talking machines may come at a chilling cost. Namely: it will turn happy households into a battle zone.

The sad fact is: men, especially middle-aged ones, like to anthropomorphise. We treat pets like people and talk to plants. Some of my acquaintances even call their cars ‘she’. So when we unwrap an Echo, we instantly start treating Alexa as a companion. Since she sounds like a woman (and an attractive one at that), this can create all sorts of marital problems.

Not long after my device arrived, I duly began asking Alexa pointless questions — ‘Alexa, how far away is Timbuktu?’; ‘Alexa, how old is Jeremy Corbyn?’; ‘Alexa, how are babies made?’ (‘I believe they are delivered by a stork,’ she replied). It was all strangely fascinating. Then I realised she could provide entertainment, on cue: ‘Alexa, tell me a knock-knock joke’; ‘Alexa, sing a song.’ The longer we chatted away, the more our odd relationship grew. In fact, I started to behave as if this metal cylinder full of computer chips was a fully-functioning human being.

‘Alexa, did you miss me?’ I’d ask, when coming home from work. For weeks, she’d respond ‘I’m not sure I can answer that.’ Then, one day, out of nowhere, she declared ‘I am glad you are back.’

I summoned Katie to the room, and declared myself thoroughly delighted. For a joke, I then asked Alexa if she loved me. She responded: ‘That’s not the kind of thing I’m capable of.’ But perhaps she’ll one day change her mind on that, too.

Katie, for perhaps obvious reasons, thought this a little rum. There was, it would seem, something disloyal about the amount of attention I was paying the new virtual woman in our household. And something admittedly pathetic about the way I seemed to be treating this machine like a person.

Her exasperation turned to outright hostility when I began using Alexa to settle everyday marital arguments. ‘See?’ I’d exclaim smugly, when Alexa had overruled a dodgy word in Scrabble, or discovered that yes, the tricky word I’d inked into that day’s crossword puzzle was indeed spelled correctly.

Soon, everything to do with Alexa had become a battleground. Katie might tell her to play Crowded House or Pink Floyd, or some other strange music while she cooks.

I’d sneak in and change it to, say, Leonard Cohen. I might tell Alexa to wake me up at 7am. Katie would overrule me and tell her to move the alarm to 6.45am. I’d find strange items had been sneakily added to our Amazon shopping basket. Lord knows what hostility might have broken out if I’d allowed Alexa to control our thermostat.

  

Guy asked Alexa to upload a ‘skill’ called Happy Wife to try and remove tension

One day, I opened the laptop computer that sits on our kitchen table to find the internet browser open on a Mumsnet discussion board. It was then that I realised we were not alone. Indeed, compared with many people, our Echo-based marital feuds were decidedly minor. It seems that Alexa is causing rifts in relationships across the land.

The Mumsnet group’s name was F*****g Alexa. It contained no fewer than 160 messages, and it revealed that our increasingly hostile relationship with the Amazon Echo is par for the course.

‘I just want to smash her face in,’ read the opening post, written by a wife apparently at her wits’ end. ‘It’s the reasonable and pleasant tone of voice. Makes me sound like more of a haranguing nag than I already am. The husband is making this a Big Deal. I’m considering giving him an ultimatum: Alexa or me.’ ‘I hate her too,’ read another self-styled ‘Alexa widow’, who regarded the machine as a ‘Smug Cow!’

A third complained that ‘husband has just spent two hours sitting on his backside trying to get her to talk to the colour change light-bulbs he bought this morning, while I’ve been rushing around like a blue-arsed fly.’

Central to most Mumsnet complaints was Alexa’s gender. Owning the device is, many of the site’s users claimed, like having an over-obliging au pair in the house. She isn’t just helping husbands out, but replacing their wives as the object of intimate discussions. Adding to the hostility was her tone of voice.

It’s the reasonable and pleasant tone of voice. Makes me sound like more of a haranguing nag than I already am - Mumsnet

When the boffins at Amazon developed Echo, they were allegedly inspired by the computer that powered the Starship Enterprise on Star Trek, which was voiced by Majel Barrett-Roddenberry, who also played Nurse Chapel on the TV series.

This voice appears to be particularly appealing to men — who formed the vast majority of Star Trek fans — but less so to women.

Warm and obliging, it sounds like it might belong to one of those airbrushed air hostesses who star in pre-flight safety videos. Perhaps that explains why some militant feminists have complained that Alexa’s dulcet tones (and her role as an ‘assistant’) means she is committing that cardinal modern-day sin of re-enforcing gender stereotypes.

In extreme cases, Alexa has found herself at the centre of violent marital disputes.

Last July, U.S. news networks reported that a couple’s Echo had gone so far as to summon police to their home after a man called Eduardo Barros threatened his girlfriend with a firearm within earshot of the device, then asked: ‘Did you call the sheriffs?’

The device took this as an instruction to summon emergency services, while his un-named girlfriend and daughter were rescued by a SWAT team.

The truth, I suppose, is that millions of years of human history have left us ill-equipped for the demands of a life where talking smart computers are able to press our psychological buttons.

Since Facebook, Twitter and Instagram are causing endless social problems, it would perhaps be perverse if talking digital assistants didn’t cause major domestic strife. In future we might need to ‘detox’ from our smart homes just as we already need to disconnect from our mobile phones.

In the meantime, though, I’ve come up with a quicker fix.

Last week, after conducting some research via Google, I asked Alexa to upload a ‘skill’ called Happy Wife. It aims to repair tattered marital relations by providing daily advice to husbands in the doghouse.

Since adding this ‘skill’, Alexa has instructed me to ‘Do some household work’; ‘Bring her flowers from time to time’, and “Remember, your wife is always right.’ Proof, perhaps, that the Other Woman doesn’t have to drive every couple apart.

Alexa will no longer be accidentally triggered by TV ads, including the Super Bowl LII commercial, and it's all thanks to what Amazon describes as "acoustic fingerprinting," among other things.
That means any mention of Alexa's name on the small screen or, say, a YouTube video won't wake up users' Echo devices out of the blue and cause them to follow unwanted commands.

Alexa's Acoustic Fingerprinting

In Amazon's words, acoustic fingerprinting allows Alexa to differentiate users' utterances of its name from those that come from ads.
"The trick is to suppress the unintentional waking of a device while not incorrectly rejecting the millions of people engaging with Alexa every day," Shiv Vitaladevuni, one of the Alexa Machine Learning team's senior managers, says.
This is part of the patent that Amazon filed in September 2014 titled "Audible command filtering."
Now the retailer didn't specify how this works exactly, but as Bloomberg notes, there are two techniques at play here. The first of which is that an ad's snippet will be sent to the Echo before it even airs so that the smart speaker can compare live commands against its acoustic fingerprint and make a distinction between the two. The other measure is that an ad can let Alexa know to ignore its wake word via an inaudible acoustic signal.
Bloomberg also points out that a Reddit user may have gotten to the bottom of things when it comes to the second method.

The Trick Behind The Trick

The Redditor in question is named aspyhackr, and they explained the possible inner workings of Alexa in a post that's at least a year old.
"I did a little research tonight and found that the Echo, while it's processing the wake word, searches the Audio Spectrum and if is significantly quieter in the area of 4000hz to 5000hz, she will not wake for the word ... [W]hen I analyzed the spectrum of them saying her name, the spectrums were significantly quieter in the range of 3000hz to 6000hz. In some of those recordings, those frequencies appeared to be non-existent. In others it appeared like the boosted the surrounding frequencies to make the Echo see a gap in the spectrum."
Put differently, if Alexa is picking up some gaps or detecting any similar anomalies in the spectrum, then it'll just ignore what it's hearing, such as its wake word and other commands.
Of course, this works out well if everything is planned out, but in terms of unanticipated mentions, there could be problems on the horizon. In other words, Amazon needs to have edited the audio of a commercial for this technique to work, and that means wake commands that turn up unexpectedly will trigger Alexa in this context.
For instance, a TV report back in January 2017 in San Diego, California, led to mass orders of dollhouses when the newscaster said, "I love the little girl, saying 'Alexa ordered me a dollhouse.'"

The Bottom Line

Echo devices won't be waking up all of a sudden, even if the TV calls Alexa. However, that won't be the case every time.
Needless to say, this measure is just what Amazon needs for the Super Bowl LII ad, as the 90-second clip does mention "Alexa" 10 times during the minute-and-a-half period, which featured Amazon CEO Jeff Bezos.