A recent survey found that 35% of US adult consumers own smart speakers. It’s important to understand what you are getting into if you are new to smart speakers or digital assistants.

Amazon Echo, Apple HomePod, and Google Nest are no longer gimmicks. These speakers allow you to perform many tasks more quickly than a smartphone, such as playing your favorite Spotify playlist or setting an alarm.

Each request is sent to the servers of the company concerned. They may store the information or use it to create an ad-profile that will determine the ads you see on the internet or in apps.

Smart speaker privacy is different from what you experience when searching Google or looking up products on Amazon.

These devices are always listening to you, at least in a limited way. How else would they be able to hear you saying “Alexa” and “OK, Google?” as your wake-up words?

You’ll learn what happens when you interact Amazon, Google, and Apple smart displays and speakers.

Alexa: Amazon Echo devices

Amazon has been the leading smart speaker manufacturer in the US, with around 70% of all devices sold.

Amazon has a huge selection of Echo products that are cheap enough to buy on whim. This is especially true during regular sales.

Amazon created the Alexa Privacy hub in order to educate users and alleviate privacy concerns. Take a look at the actions that Amazon takes when you use one of Alexa’s magic words.

What and when does the device record?

Alexa is activated by pressing the button or saying the wake word on your Alexa Smart Speaker. A recording is then sent to Amazon’s cloud where it is combined with other data to form a response.

When you ask to “Alexa play the top hits from Amazon Music”, Alexa will use your recorded request as well as information from Amazon Music in order to play the top hits.

The device will not store or send audio to the cloud until it detects the wake-word (or Alexa has been activated through a button press). When Alexa records and sends your request, a blue light will appear or an audio tone will sound.

The latest Amazon Echo devices, such as the Echo 4th Generation and Echo Show 10, offer on-device processing.

The data that Amazon uses is not the recording itself, but a transcription. These processors can do the transcription themselves.

How and where are recordings stored?

Amazon servers store all voice recordings in the cloud, which are securely encrypted. They are also associated with your Amazon Account.

You can review your recordings, use other Amazon services and help Alexa provide a more personal experience.

Are there any videos stored on the device if it has a camera?

Amazon claims that video calls are sent to the cloud but they’re only streamed, and not stored.

Can my recordings be used to identify me?

Amazon claims to associate requests with customer accounts so that users can access their voice recordings as well as other Amazon services.

Alexa can be asked to play audiobooks or Kindle books, for instance. It’s up to the reader whether this is a direct answer to our question.

Who else is there?

Amazon employees monitor a small percentage of Alexa interactions. The official statement is: “Humans review a very small sample of Alexa requests to help Alexa interpret a request correctly and provide an appropriate response in future.”

It’s impossible to know how it works unless we’re in Amazon’s office and doing the work ourselves. The idea is that real humans are used as a complement to the machine-learning process enabled by mass analysis of voice interactions.

Alexa’s real-world performance is being improved, and it will become smarter. You can opt out. Tap the More tab in the Alexa App, then tap Settings. In Alexa Privacy > Manage your Alexa data, you’ll see a toggle that says Help Improve Alexa. Your recordings won’t be used if you turn it off.

Amazon warns that “voice recognition and other new features may not function well if you turn it off.”

What happens to recordings that are made accidentally? What happens to recordings made by mistake?

Alexa will stop processing audio if an Echo device begins recording because it is mistaking something for a wake-word. The audio will then be streamed to the cloud. The Voice History of customers can be accessed via the Alexa App or web. A small portion of the audio will be stored.

Amazon claims that reviewing these recordings is crucial to improving the accuracy of wake words. These recordings will not be analyzed for customers who have chosen to opt out of the review.

Customers who want to know what Alexa heard during their request can ask “ Alexa tell me what I heard “, and the latest voice request will be played back.

Can an Alexa user delete recordings?

You can delete Alexa voice recordings from the Alexa App or Web. You can enable voice-activated deletion to remove the last request you made by saying “ Alexa delete what I said.

You can also delete all of the recordings from a particular day by saying “ Alexa, delete every word I said today. “ If you delete a recording, Amazon will also remove any transcript.

How long does Amazon store voice recordings of customers? Are recordings automatically deleted?

Amazon stores recordings until the customer deletes them. You can decide whether recordings are saved for 3 months, 18 month or never.

To access these options, open the Alexa App and go to Settings > Alexa privacy > Manage your Alexa data > Choose How long to save recordings.

What are the benefits of voice data for Amazon customers?

Alexa was designed to be able to learn. It is a machine-learning, artificial intelligence engine. Alexa will perform better the more data Amazon has to work with.

Alexa’s understanding is enhanced by the accents, dialects, and vocabulary of customers. Your data is also valuable for advertising and marketing.

Amazon will show you personalized ads on its networks based upon the information it has collected about you. You can turn off these ads in your Amazon Account Preferences.

Amazon claims that the majority of Alexa interaction are not used to advertise. This is partly because the majority of Alexa interactions aren’t useful in this sense.

Alexa’s experience is very similar to that of the Amazon app or website. If you play a track on Alexa, for example, you might see suggestions in the Amazon Music App of similar artists. You can order paper towels through Alexa, and then see similar products recommended on Amazon’s website.

Amazon claims that it does not use Alexa interactions, such as asking Alexa for the weather or a recipe, to make product recommendations. Amazon doesn’t allow Alexa advertising outside of third-party apps or skills such as Pandora streaming radio or CNN news.

Alexa will become more personalized as you continue to use it. It will learn your voice patterns, vocabulary, and preferences. Alexa can make recommendations based on previous requests and skill usage.

Is my face kept on file by Visual ID?

The new Echo smart displays like the Echo Show 15 have a feature known as Visual ID. It can recognize the members of your family and offer them personalized content.

Amazon has disabled Visual ID by default. All face recognition data is stored on the device and not on Amazon servers.

Can third-party apps/other Amazon properties access data recorded by voice?

Amazon states that it will not share any voice recordings to third parties. Alexa exchanges information when you use third-party services through Alexa. For example, an email address is shared to make a reservation at a restaurant.

The transcription can be used, but not the actual recording.

The Alexa app allows you to control which Alexa Skills can access data. Go to Settings> Alexa Privacy> Manage Skill Permission or the website.

What other data does Echo collect besides voice recordings?

Amazon also collects data on device usage, network diagnostics, and other information necessary for the service to be provided and improved.

Google Nest speakers: Google Assistant

Google’s privacy settings are simple, but their impact is huge. The core settings are the same for all Google services and devices, including phones, tablets and Nest smart speaker and other products with Google Assistant.

There are important options that you should know, even though there is no fine-grain control.

What and when does the device record?

Google Assistant is in standby until the words “Hey Google” and “Okay Google” are heard.

Google Smart Displays will display an icon in the screen. On Google Nest speakers, you’ll notice four colored lights that indicate the speaker is listening.

You’ll hear an audible bleep if you use a third-party device, such as Sonos One. An indicator light will flash. It all depends on the device you are using. However, there should be an audio or visual alert by default.

Google is working to free Assistant from the wake-word. Android 12 will reportedly offer “smart sentences”.

Google Assistant uses machine learning to recognize phrases.

How are recordings secured and stored? Can I be identified by the recordings?

By default, Google doesn’t retain audio recordings. Voice & Audio Activity settings (VAA), which allows customers to opt-in for the storage of audio data, can be accessed by the customer. Google stores all voice recordings encrypted. Google claims to remove personal identifiers and replace them with a unique number.

Who else is there?

In the course of the Assistant review, some recordings are sent to “language experts”, who have the task of analyzing voice data in order to improve the service. Google claims that only 0.2% of audio recordings are reviewed by reviewers.

After a massive leak of customer information, this part of the Assistant was suspended in 2019. It was however reinstated in the year 2020.

 

Are there any videos stored on the device if it has a camera?

Nest’s Nest cameras store video footage in your Google Account.

For more information, click here to access, review and delete the footage.

Google’s Nest Hub Max has an integrated camera that can be used to determine who is using it.

Google states that the video from the device will be sent to Google’s servers only during the setup, and not after that.

What happens to recordings that are made accidentally? What happens to recordings made by mistake?

Google has taken measures to prevent a repeat of 2019, a blunder in which more than 1,000 Assistant voice requests were leaked by an independent contractor. There were 153 snippets which were recorded clearly by mistake.

Google has said that it will focus on the “false acceptances” or unintended activations of Google Assistant being , apparently processing. It also says “numerous protections are in place to stop this from happening.”

Unintentional voice recordings may contain sensitive information, even if they are not directly connected to a person’s identity.

Google Assistant deletes recordings that are unintentionally activated to make sure they do not form part of “expert review processes.”

Can an individual access the recordings? And can they delete them as well? They delete themselves at any time?

You can delete all previous conversations and review the voice recordings that are associated with your account at any time by visiting ‘My activity’.

Conversations can be deleted by using voice commands, such as “Hey Google delete this week’s activities”. You can set auto-delete to occur at intervals of three or 18 months.

This is a complete guide on deleting your Google Assistant voice records.

How long does Google retain voice recordings of customers?

By default, voice recordings are not saved. However, if you have chosen to opt in, they are stored until you delete them.

Are my data used to show me advertisements?

Google insists that it will not provide voice recordings to a third party or sell personal information.

It can use the text from Assistant interactions “to inform your interests for ad personalized on Google services.”

While the audio of the conversation has no impact on the advertisements you will see, the words used during the conversation can influence the type of ads that you are shown.

Google’s Adsense network generated $147 billion by 2020 and is the most lucrative part of Alphabet.

If you are not interested in personalized ads based upon Google activity, you can turn them off on the Google Account page.

Can third-party apps or other properties owned by Google access data recorded during voice recordings?

Some information could be shared if you use Google Assistant to interact a third party service. Google says that it will share information with Uber in order to “complete a booking or confirm an Uber ride.” You must give Google permission to use this information.

Google explains a bit more in its Terms of Service. “We do not share personal information with our advertising partners such as your email or name, unless we are asked to.” If you click on an ad to contact a flower shop near you and then select the “tap-to-call” button, Google will connect your call. We may also share your phone number.

The section that asks ‘Does a third-party service provider receive an audio recording of my words?’ is found on the Google Home Privacy Terms page. Google’s answer is “In general, no.” Google will transcribe what you say, and only send the text to the third party service provider.

Google will share information with third-party providers, like smart security devices, to offer a more “helpful” experience. However, you have to give your permission.

Voice recordings will never be shared, only transcriptions. Some personal information, such as your phone number and email address, can be shared, but with your permission.

Google has a long way to go to make this clear.

What are the benefits of voice data for Google and you?

Google Assistant is able to understand your speech patterns and preferences better than Alexa. It can also recognize accents, dialects, intonations, and personal preferences.

It’s the same for music preferences. If Google knows about your music tastes, then asking Assistant to Play music will help it select songs that you are likely to enjoy.

Siri on Apple HomePod devices and iOS devices

Apple puts a lot of emphasis on privacy and uses it as a selling feature for the iPhone. Apple has made recent efforts to increase transparency about how data is used by apps and services. This is commendable.

It has, however, had its own issues over the years. A whistleblower reported in 2019 that Apple contractors were able to access accidental Siri recordings. This was caused by the smart speaker or Apple Watch activating when Hey Siri is not used.

Apple responded by updating Siri’s privacy setting, which centered around a process they call ‘grading’. Apple claimed that a small percentage of Siri requests (less than 0.2%) were flagged to be reviewed by humans. Users can now opt out.

This statement from 2019 was the last major disclosure on Siri’s privacy practices. This is how it works.

What and when does the device record?

Siri will only send audio to Apple after recognizing the “Hey, Siri” command. The audio stays on the device as long as the “Hey Siri pattern” has gone through the two-step verification procedure.

Apple’s Voice Recognition Server will only receive the data after it has been processed.

What is the policy on contractors and Siri recordings

Apple’s “Improving Siri’s Privacy Protections” post explains that the company has paused Siri’s grading while it “reviewed”.

The practice was resumed. Apple will allow users to choose whether they want to have their audio reviewed.

Apple employees can only listen to audio samples. Contractors will not be allowed.

What happens if Siri is activated by mistake?

Siri may, from time to time, incorrectly think that you have used the wake-word. Apple has changed its audio review procedure to limit the recordings that humans can review. Apple says that false triggers found on devices will be deleted, and they won’t be sent to Apple.

Apple also states that if a false trigger is detected on the Apple servers, it will be removed. However, a subset of these will be checked by Apple employees in order to verify that it works correctly.

Reviewers won’t have access to identifiable user data and will only be able to review false triggers that are sent to them for six seconds or less.

How long does Apple store voice recordings of customers? Users can they access and delete recordings?

The history of Siri requests is linked to the random identifier up to a period of six months. It could also include audio and transcripts for those who choose to opt in.

Apple may retain the history of user requests after six months, if they are not associated with the random identifier.

The random identifier can be removed from any requests that are kept after the two-year time period. This is done to improve Siri.

Apple believes that their use of random identifiers sets them apart from other digital assistants. If you opt in to the “Improve Siri” feature, you will be able delete all Siri recordings and transcriptions less than six-months old.

If you delete Siri or dictation within 24 hours, neither the audio recording nor the transcription will be sent to review.

 

Are there any videos stored on the device if it has a camera?

Apple does not store your FaceTime conversations. Apple does record the other party’s contact details and the time of the FaceTime call for a period of 30 days.

It is also recorded the time, but not if it was answered. You’d want to have access to this information, which is why it is stored on your device after the initial 30 days.

Apple says that it will store information about FaceTime “in a manner that does not identify you.”

What other data does Apple collect besides voice recordings?

Siri can also use information such as the names of your contacts, Apps installed and your location. Apple calls this data Siri Data. It is linked to a device-generated random identifier, and not an Apple ID.

What are the benefits of voice data for Apple and you?

Apple claims that Siri Data and requests from customers are never sold or used for marketing purposes.

According to the FAQ about Siri Privacy voice data is used to measure how Siri responds and improves the service.

Which smart speaker is most private?

You’ll notice some patterns if you read this article from beginning to end. Some of the most controversial stories about digital assistants are related to when our data was seen or heard by real people.

This is a challenging but necessary step in the process. It allows these companies to not rely solely on software and machine learning for improvement. You can only put so much trust in “the machine”.

Apple, Google, and Amazon each let you to some degree control the amount of information that is stored on servers. Google, in a surprising move, is not keen to keep voice recordings on the cloud. This could be due to the fact that it has a good grasp of voice transcription.

Apple, from a broader perspective, is probably the most privacy-conscious of the three assistants. Apple says that Siri Data and requests are never used to create a marketing profile and are never given to third parties.

This “marketing profile”, is an important part of the Google Assistant and Amazon Alexa strategy. This can be useful in some cases, but it’s not the best strategy for privacy.