AR glasses with multimodal AI receive funding from Pokémon Go creator


In the week when gadget lovers everywhere are delighted by VisionProa young and courageous startup is trying to carve out space for its augmented reality device that has a radically different form factor from Apple’s.

Today, Singapore-based Brilliant Labs announced its new product, Frame, a pair of lightweight AR glasses powered by a multimodal AI assistant called Noa. The glasses caught the attention and investment of John Hanke, CEO of Niantic, the augmented reality platform behind games like Pokemon Go. Brilliant Labs declined to disclose the amount of funding received from Hanke.

In a video demo seen by TechCrunch, one of the founders of Brilliant Labs asked Noa by voice to introduce himself. After about three seconds, the agent generated and projected a text response onto the lenses.

In addition to voice commands, Noa is capable of visual processing, image generation and translation, thanks to the handful of AI models it has integrated: the conversational search engine Perplexity AI; Stability AI’s text-image model, Stable Diffusion; OpenAI’s latest text generation model, GPT4; and the Whisper voice recognition system. Frame lenses have a resolution of 640 x 400 for displaying videos and photos.

Using these features, a user shopping in a mall can ask Noa to check the online prices of a pair of shoes they are looking at through Frame, for example.

“The future of human/AI interaction will come to life in innovative wearables and new devices, and I am very excited to bring Perplexity’s real-time response engine to the Brilliant Labs framework,” Aravind Srinivas, CEO and founder of Perplexity, said in a statement.

The question is whether Frame will be responsive enough for any of its AI-generated responses to be useful. Brilliant Labs’ Bluetooth-enabled devices today rely on a smartphone to access different AI models. Ultimately, however, the founders want to remove the phone host and integrate lightweight machine learning models directly into the glasses.

Co-founders of Brilliant Labs 2

Co-founders of Brilliant Labs / Pictures: Brilliant laboratories

Frame came after Brilliant Labs’ first product, Monocole, a single-lens AR device that has attracted a loyal following in the open source hardware community thanks to its programmability and affordable price. A group of Stanford students once turned the device into a dating assistant that made real-time suggestions on what to say to their date.

Frame will continue to be open source like its predecessor, meaning developers will have access to resources including scalable documentation, an open source codebase, and hardware schematics. They will also be able to adjust the parameters of the AI ​​models supported by Noa.

Unlike the bulky Vision Pro, Frame is intended for everyday wear and comes with prescription lenses. Weighing 39 grams and featuring thick round nylon plastic frames, the glasses are, as the company says, a “tribute to the innovations and revolutionary ideas introduced by historical figures such as John Lennon, Steve Jobs and Gandhi.”

Frame will be available for pre-order starting today and will cost $349, the same as Monocle’s price. Electronics will begin shipping in April.

Since its inception in 2019, Brilliant Labs has managed to attract a series of notable angel investors. In June we reported its $3 million investment from Brendan Iribe, co-founder of Oculus; Adam Cheyer, co-founder of Siri; Eric Migicovsky, founder of Pebble and former partner at Y Combinator; and others. Hanke’s new capital infusion brings the company’s total funding to $6 million.


Leave a Comment

Your email address will not be published. Required fields are marked *