Facebook CEO Mark Zuckerberg is seen fencing in the “Metaverse” with an Olympic gold medal fencer during a live-streamed virtual and augmented reality conference to announce the rebrand of Facebook as Meta, in this screen grab taken from a video released October 28, 2021.
Facebook | via Reuters
Facebook co-founder Mark Zuckerberg, now CEO of Meta, claimed Monday that a new touch sensor and a plastic material could work to together to potentially support the development of a so-called “metaverse.”
Together with scientists from Carnegie Mellon University, artificial intelligence researchers at Meta created a deformable plastic “skin” less than 3mm thick.
The relatively cheap material, known as ReSkin, has magnetic particles inside that produce a magnetic field.
When the skin comes into contact with another surface, the magnetic field from the embedded particles changes. The sensor records the change in magnetic flux, before feeding the data to some AI software, which attempts to understand the force or touch that has been applied.
“We designed a high-res touch sensor and worked with Carnegie Mellon to create a thin robot skin,” wrote Zuckerberg on Facebook Monday. “This brings us one step closer to realistic virtual objects and physical interactions in the metaverse.”
The skin was tested on robots that handled soft fruit including grapes and blueberries. It was also placed inside a rubber glove while a human hand shaped a bao bun.
The AI system had to be trained on 100 human touches in order to ensure it had enough data to understand how changes in magnetic field relate to touch.
The work is set to be published in an academic journal later this month, but it is yet to be peer reviewed.
Touch has largely been neglected by AI researchers because touch sensors have been too expensive or too flimsy to get reliable data, Abhinav Gupta, a research scientist at Meta, said on a media call Friday.
“If you think of how humans or babies learn, rich multimodal data is quite critical for developing an understanding of the world,” Gupta said. “We are learning from pixels, sound, touch, taste, smell, and so on.”
“But if you look at how AI has advanced in this last decade, we have made huge advances in pixels (computer vision)…and we have made advances in sound: audio, speech and so on. But touch has been missing from this advancement even though it is very critical.”
Helping machines and robot assistants to feel will allow them to understand what humans are doing, Gupta said, adding that Meta’s ReSkin can detect forces down to 0.1 newtons from objects that are less than 1mm wide.
“We can for the first time try to have better understanding of the physics behind objects,” Gupta said, claiming that this will help with Meta’s quest to build a metaverse.
The metaverse is either the next evolution of the internet, or the latest corporate buzzword to get investors excited over some nebulous innovation that may not even come to pass over the next decade.
Either way, tech companies — primarily Facebook — are increasingly boosting the concept of the metaverse, the term for a virtual world you can live, work and play inside. If you’ve seen the movie “Ready Player One,” you have a pretty good idea of what the metaverse is: Strap on a set of computerized glasses, and you’re transported into a digital universe where anything is possible.
If Meta’s metaverse ambitions come to fruition, then it may be possible to interact with virtual objects and get some sort of physical response from a piece of hardware.
“When you’re wearing a Meta headset, you also want some haptics to be provided so users can feel even richer experiences,” Gupta said.
“How can you provide haptic feedback unless you know what kind of touch humans feel or what are the material properties and so on?”
— Additional reporting by CNBC’s Steve Kovach.
