Are Groq’s AI chips quicker than Nvidia’s? AI startup stands out with ‘lightning-fast’ engine


AI startup Groq (not Elon Musk’s Grok) has unveiled its new synthetic intelligence (AI) chip with a language processing unit (LPU) structure that claims to supply on the spot response instances. This new innovation comes at a time when AI is booming and corporations resembling OpenAI, Meta and Google are working exhausting to develop their suite of AI instruments resembling Sora, Gemma and lots of extra . Nevertheless, Groq outright claims that it provides “the world’s quickest giant language fashions.”

Groq claims its LPUs are quicker than Nvidia’s graphics processing items (GPUs). Contemplating that Nvidia has attracted consideration when it comes to AI chips up to now, this side is stunning. Nevertheless, to again this up, Gizmodo stories that the demos Groq did had been “lightning quick” and even “…present variations of ChatGPT, Gemini, and even Grok appear sluggish.”

Groq AI chip

The AI ​​chip developed by Groq has specialised processing items that run prolonged language fashions (LLMs) offering virtually instantaneous response instances. The brand new processing unit, referred to as the Tensor Streaming Processor (TSP), has been labeled as an LPU and never a graphics processing unit (GPU). The corporate mentioned it supplies “the quickest inference for compute-intensive functions with a sequential part,” resembling AI functions or LLMs.

What are the advantages?

This eliminates the necessity for advanced planning gear and promotes a extra streamlined method to processing, the corporate claims. Groq’s LPU is designed to beat compute density and reminiscence bandwidth, two points that plague LLMs. The corporate claims that with regards to LLMs, the LPU has better computing capability than a GPU and CPU, thereby decreasing the calculation time per phrase. This ends in a lot quicker textual content era.

Calling it an “inference engine,” the corporate says its new AI processor helps commonplace machine studying (ML) frameworks resembling PyTorch, TensorFlow, and ONNX for inference. Nevertheless, its LPU inference engine doesn’t at the moment assist Machine Studying (ML) coaching.

Groq allows quicker, extra environment friendly processing, with decrease latency and constant throughput. Nevertheless, it’s not an AI chatbot and isn’t meant to interchange one. As an alternative, it claims to make them run quicker. Those that wish to strive Groq can use open supply LLMs resembling Llama-2 or Mixtral 8x7B.


In a demo sharing by Matt Shumer, CEO of HyperWrite on X, the Groq supplied a number of solutions to a question, full with quotes inside seconds. One other demo of Groq in a side-by-side comparability with GPT-3.5 discovered that it completed the identical job as GPT, solely virtually 4 instances quicker. In line with benchmarks, Groq can attain virtually 500 tokens per second, in comparison with 30-50 tokens managed by GPT-3.5.

Additionally learn different prime tales as we speak:

Demand Deepfake Regulation! Synthetic intelligence consultants and trade leaders, together with the “godfather of AI” Yoshua Bengio, have signed an open letter calling for extra regulation across the creation of deepfakes. Some attention-grabbing particulars on this article. Test it out right here.

Sora is horrifying! Since OpenAI rolled out its text-to-video AI era platform, main content material creators have feared they’re the final professionals on the verge of being changed by algorithms. Verify all the main points right here.

Microsoft will construct an in-house processor! Microsoft has turn out to be a buyer of Intel’s made-to-order chip enterprise. The corporate will use Intel’s 18A manufacturing know-how to make an upcoming chip designed in-house by the software program maker. Learn all about it right here.

Yet another factor ! We at the moment are on WhatsApp channels! Observe us there to by no means miss any updates from the tech world. ‎To comply with the HT Tech channel on WhatsApp, click on on right here be a part of now!


Leave a Comment

Your email address will not be published. Required fields are marked *