Hot Chips 2024 recap: The global unveiling of RNGD
Our Viewpoints
In the crowded, hype-filled world of AI, you never know how even a groundbreaking product will land until you actually show it to the world.
Last month at Hot Chips 2024, FuriosaAI globally unveiled RNGD (pronounced "Renegade"), our new AI accelerator designed for high-performance, highly efficient large language model (LLM) and multimodal model inference in data centers. We chose Hot Chips because it’s the premier technical conference for silicon engineers – experts who tend to ignore the marketing hype and focus instead on design, specs, software details, and benchmarks. The conference started in 1989 and has been drawing leaders in the space together for 35 years. At some conferences, companies can buy a slot presenting on the main stage, but not at Hot Chips. So when conference organizers invited Furiosa to present, we knew this was the right place to pull back the curtain and show what RNGD can do.
In addition to Furiosa CEO June Paik’s main stage talk, we let conference attendees try the first live demo of RNGD running Llama 3.1 8B and 70B, and we showcased fully functional RNGD cards and a Supermicro server with four RNGDs installed.
Now that the dust has settled, we’re incredibly pleased and excited with how RNGD was received.
Here are a few highlights:
EETimes, a leading source for semiconductor industry news, wrote a detailed overview of RNGD and Furiosa and featured the article on their front page. We also received global media coverage from leading industry publications like Semiconductor Engineering, SiliconANGLE, eeNews Europe, and IEEE Spectrum, among many others.
We were pleased to be featured on the EE Times homepage on the first day of the Hot Chips conference.
Our social media presence may just be starting to take off, but the engagement we saw was truly impressive. Our LinkedIn and Twitter posts about RNGD generated significant buzz and discussion. One veteran tech analyst shared a detailed, positive writeup on RNGD, noting that it was our company name that caught his attention. (We'll take it as a compliment, David.)
Even more gratifying was the interest we saw from the silicon engineers who came by our Hot Chips booth to try our demo and discuss RNGD. We expected thoughtful, well-informed questions from the Hot Chips crowd and we weren’t disappointed. For example, one attendee asked, “Is einsum just ‘syntactic sugar’ for matrix multiplication?”
Others wanted to know about FlashAttention support, the RNGD compiler, and how to design an architecture specifically for tensor contraction.
(We’ll dig into all of these in upcoming blog posts. But the tl;dr is: 1) no, it’s not just syntactic sugar, 2) with RNGD, you don’t actually don’t need FlashAttention, 3) we’ve built our own general compiler that treats the entire model as a single fused operation, and 4) it’s tricky to summarize but you can read this ISCA conference paper.)
The Furiosa booth featured a server with eight fully functional, highly power-efficient RNGD cards. But since our booth was situated outside in 90-degree heat, we reluctantly decided it wasn't wise to try to power it up on our presentation table.
We aim to be as transparent as possible when talking about Furiosa’s work. So we’ll acknowledge that not everything went according to plan. It was particularly hot in late August on the Stanford campus where Hot Chips was held, which meant we couldn’t run the live local RNGD demo on the server we’d brought to our booth. This was definitely disappointing.
But we made do with a live Llama 3.1 70B demo running remotely. And visitors to our booth were able to inspect an actual RNGD card and admire the snazzy all-red casing.
RNGD is just getting started
Hot Chips was a great beginning for RNGD, but there’s much more work ahead. We’ll be presenting at AI Hardware Summit this week, so please stop by our booth if you’re attending. And we’ll share more benchmarks and technical updates on RNGD soon, as well as information on availability and pricing. We’re also excited to talk about the companies that are sampling RNGD now.
Hot Chips 2024 was attended by more than 2,000 engineers and silicon industry professionals representing 20 countries.
Two weeks ago we were essentially an unknown in the U.S. chip market. And it’s only been three months since we received the first RNGD silicon from TSMC. This is a crowded, competitive space with a lot of innovative new companies as well as some of the biggest, best known technologies firms in the world.
We’re a little better known today, but we know Furiosa and RNGD will need to do much more to achieve our goal of making powerful AI accessible to all. Hot Chips is an early validation of our approach – which is to build a new kind of AI chip that delivers the “trifecta” of high performance, power efficiency, and programmability all in a single product.
The word “renegade” means diverting from the norm or mainstream. We chose “RNGD” as our flagship product's name because we’re committed to a different approach to making AI silicon – one that starts with a different fundamental operation (tensor contraction instead of matmul) and that focuses on the real world needs of the industry (performance + power efficiency + programmability).
We look forward to sharing more news soon.
Sign up here to get updates on RNGD or reach out to us to learn more.
Furiosa CEO June Paik talks through the multi hierarchies of Networks on Chip (NoCs) that RNGD uses for connectivity. Within each processing element, the NoC connects the internal CPU, Scratch Pad Memory, Tensor DMA, and Tensor Unit. The memory NoC supports full channel interleave to efficiently utilize the 16-HBM channels, allowing each processing element to utilize the full 1.5TB/s bandwidth.
