Future AI toys could be smarter than parents, but a lot less protective

The thought of a child’s toy listening in on a family 24/7 is unsettling. While smart toys can be useful, educational tools for kids, they also pose privacy risks which toy makers and privacy experts are still learning how to balance.

Smart toys made with artificial intelligence, such as machine-learning capabilities, can collect different forms of data from kids. Whether an AI-enabled toy is personalizing lessons based on how fast your kid learns shapes, or a doll is learning your kid’s favorite ice cream flavor, toy experts expect more of these toys to be introduced in the years ahead, even though early missteps and high sales prices have limited consumer interest to date.

“As an AI toy starts to learn the child, this means the toy in the next 15 years will be smarter than the parent and gather all this data that could one day hurt the child,” said Will.i.am, singer-songwriter and chair of the World Economic Forum’s Smart Toy Awards’ judging committee, recently speaking at CNBC’s Evolve Global Summit.

AI toy concerns vary based on toy type and the capabilities it has in terms of collecting data.

In general, smart toys learn from children and provide an adaptive and responsive play experience. However, there are two main categories of smart toys that fit within this framework. First, smart companions, which learn from and interact with the child during activities. Second, programmable toys designed with machine learning that move and perform tasks to teach kids educational skills.

One example of the latter is ROYBI Robot, which creates personalized lessons to teach kids educational subjects like science, languages and math. It has a camera and microphone to detect facial and emotional reactions from kids, but all of the information collected is controlled through a parent or guardian’s account.

While ROYBI Robot is a smart toy that handles data collection ethically and responsibly, according to Seth Bergeson, an artificial intelligence and machine learning fellow at the World Economic Forum, there are other smart toys that have risked personal and data privacy.

Between 2014 and 2017, a toy company named Genesis Toys sold My Friend Cayla marketed as an interactive doll that could listen to and respond to kids. The problem: it was recording its conversations with kids, as well as conversations with parents, siblings and anyone else around the doll, Bergeson said, and able to share the data with third-party companies.

“This is a really troublesome example and cautionary tale for a toy,” Bergeson said. “There were several FTC [Federal Trade Commission] complaints filed in the U.S. The nation of Germany just banned the toy completely and called it a concealed espionage device, and ordered families to destroy the toy if they had it at home.”

Another example, better known in the U.S. for its privacy backlash: Mattel‘s Hello Barbie.

Similar to My Friend Cayla, Hello Barbie was designed to talk with children, take in information about them and create a profile of the child in order to develop better conversations in the future. For instance, if a child told Hello Barbie about their favorite ice cream flavor or the sports they play, then Barbie created a persona of the child.

“There are severe consequences around child protection laws,” said Stephanie Wissink, a senior research analyst and managing director in the consumer practice at Jefferies. “When you start creating a technology profile of a child, you are crossing a privacy line.”

Mattel’s Hello Barbie was released in 2015 even though concerns preceded the launch, but it was met with a social media campaign titled, “#HellnoBarbie” to demonstrate consumer opposition to the doll. Hello Barbie is no longer manufactured but can be found used on eBay.

Mattel did not respond to a CNBC request for comment.

There are rules in the United States to protect a child’s personal privacy as it relates to internet data.

The Children’s Online Privacy Protection Act (COPPA) protects children 13 years and younger and their personal information on the internet from being taken without express approval from parents, according to the Federal Trade Commission.

“The purpose of COPPA is to control systems, and limit and restrict the attempt to collect data on children,” said Alan Butler, executive director at the Electronic Privacy Information Center (EPIC), a public interest research center focused on privacy issues. 

EPIC, along with other parties, filed a complaint to the FTC in 2016 about Genesis Toys and its My Friend Cayla dolls along with another product, iQue intelligent robots. EPIC said these toys collect, use and disclose recordings of children’s voices without parental consent, which is a direct violation of COPPA, according to the complaint report filed.

A year after the report, these toys were still available on Amazon but were removed from other retailers like Walmart, Target and Toys “R” Us. Now, neither of these toys are sold at any U.S. retailers, except on used toy markets like eBay. According to Ailo Ravna, a senior policy advisor at the Norwegian Consumer Council, it is unclear if Genesis Toys is operational today. Ravna said My Friend Cayla dolls and iQue robots don’t appear to be sold in Europe and the company’s website has not been updated in years.

“For any new device coming onto the market, if it’s not complying with COPPA, then it’s breaking the law,” Butler said. “There’s a lot of toys on the market and there’s a need to ensure that they’re all complying with COPPA.”

There is no pre-clearance review of toys before they hit the market, and it can get harder when toys are manufactured outside of the U.S., according to Butler, who added that while the FTC is fairly aggressive at enforcing COPPA, there needs to be a better system in place regulating toys and identifying issues before they become available for sale.

Pricing may prove to be a bigger impediment to the uptake of smart toys by consumers than privacy issues, for the time being.

The majority of toys sold to consumers are at a price point as low as $10 to $15, according to Wissink, and the integration of artificial intelligence can raise toy cost to as high as $60 to $75, which is too expensive for many parents. Until smart toys become more affordable, the prevalence of these toys is going to stay relatively low, she said.

Market demand for smart toys has been low.

Between May 2020 and May 2021, the market for programmable smart toys, specifically, made up 0.1% of the whole toy industry, according to Juli Lennett, the U.S. toys industry advisor at NPD Group, and sales were roughly half of the previous year period.

There have been sales spikes when a specific toy hits, such as in 2018, when Anki, a robotics and artificial intelligence start-up, was selling a popular toy named Cozmo, similar to ROYBI Robot, designed to use programmable features to enhance learning.

According to NPD, Anki had $24 million in annual sales from Cozmo, but Lennett said the company went bankrupt in 2019, potentially due to high manufacturing costs.

Cozmo was acquired by Digital Dream Labs, where it is in the process of being rereleased in the coming months, as an AI-powered smart companion, according to Matthew Eversole, chief marketing officer of Digital Dream Labs.

“Customer demand for these devices are still high today,” Eversole said.

Lennett is more cautious about demand.

“Every now and then, you’ll see something hit the market and cause some crazy trends, and a toy might last on the market for two or three years on average, and then fade away,” she said.

“Parents tend to like it simple. If it’s something that doesn’t really enhance the play value of the toy and it adds cost, then it’s not going to be successful,” said Gerrick Johnson, an BMO Capital Markets equity research analyst following toy companies like Mattel, Hasbro, which sold a smart R2-D2 droid and later discontinued it, and Funko, which does not currently sell any smart toys.

But the success of AI toys, like other technological advancements, may be inevitable in coming years.

“I don’t think any of us have any doubt that the world these children are going to live in is going to be a world that’s enhanced by artificial intelligence,” said Richard Gottlieb, CEO of Global Toy Experts, a toy industry consultancy firm.

Gottlieb said the introduction of AI toys is similar to when kids were first introduced to books. Books are now vital forms of literacy and education, but parents were initially concerned about access to uncensored information.

“Artificial intelligence is extremely important to society, whether we like it or not,” Gottlieb said. “It’s going to be progressively more important to play.”