Why a Microsoft-backed start-up pivoted to a fitness app after helping Fortune 500 firms to build machines that can ‘see’

In 2016, Roland Memisevic, Ingo Bax and Moritz Mueller-Freitag — the three entrepreneurs behind a start-up called TwentyBN — set out to create a platform that businesses could use to add a form of artificial intelligence to their products known as computer vision. 

Built off the back of research carried out at Toronto University and Mila (a Quebec AI Institute), TwentyBN’s platform is relatively technical and complex. But at a fundamental level, it allowed companies to build computer vision features into their products so that machines can “see” and understand what’s around them. The main use case has been with virtual assistants. 

The founders, who oversee a team of 30 employees spread across Toronto and Berlin, started out by licensing their platform to companies in automotive, health care, retail, and manufacturing. TwentyBN said a leading consumer device manufacturer, who cannot be named for contractual reasons, used its technology in one of their product lines.

“We thought the technology was sufficiently mature that you could give lots of Fortune 500 companies access to it and then they could make their products more interactive,” Moritz Mueller-Freitag, co-founder and chief operations officer at TwentyBN, told CNBC. 

But TwentyBN, which stands for Twenty Billion Neurons, was mistaken.

“The non-tech companies that we worked with don’t have the in-house talent and the resources to work with this platform in a self-service way,” said Mueller-Freitag.

“We realized the market didn’t understand the new kinds of applications you can build (with computer vision technology),” he said. “That’s when we started searching for our own products.”

The trio, who have raised $12.5 million from investors including Microsoft’s venture arm, M-12, decided to create a fitness app after concluding that such software could benefit from a more visual experience. The decision was also inspired by the fact that they believe that not everyone wants to pay over $1,000 for clever fitness hardware developed by companies like Tonal, Mirror and Peloton.

As the world went into lockdown, TwentyBN rolled out a fitness app on May 8 called Ally on Apple’s App Store, making it instantly accessible to hundreds of millions of iPhone and iPad owners. The company ultimately hopes that Ally will end up being as popular as fitness apps like MyFitnessPal, Nike Training Club, and Strava.  

The app contains a virtual fitness trainer called Allie who is there to guide the user through a series of workouts. The computer vision software within the app means Allie can “watch” the workout. Despite being little more than a virtual avatar, she’ll correct poor form and pull you up for slacking if she doesn’t think you’re working hard enough. 

“We wanted to create a product experience that was fun and that could hold them accountable,” said Mueller-Freitag, adding that it can really measure performance and how you improve over time.

The app, due to be launched on Android at a later date, has been downloaded by thousands of people and two well-known tech companies have made Ally part of their employee benefits offering, but they can’t be named as the deals haven’t been officially announced.

The app’s biggest public win is arguably a distribution partnership with GymPass, which allows people to access more than 40,000 gyms and studios around the world with one GymPass membership.

Through the partnership, Ally is offered to all of GymPass’s members for free.

Ally could face some very stiff competition from Apple soon.

The iPhone-maker is working on a new app with the codename “Seymour.” A person familiar with the plans told CNBC that Seymour will guide users through workouts via the iPhone and the Apple Watch. 

The app is being spearheaded by Jay Blahnik, a fitness instructor and author who joined Apple in 2013, the person added. 

In addition to the fitness app, TwentyBN is still licensing its platform to other companies but it’s being much more selective, according to Mueller-Freitag. 

TwentyBN has also created datasets that AI researchers use. AI researchers at big tech firms including Apple, Amazon and Microsoft have used TwentyBN’s datasets for publishing academic papers. TwentyBN charges a small fee for access to the datasets.