Build at any layer, from model to silicon
Serving
FURIOSA-LLM
furiosa-llm: drop-in, OpenAI-compatible inference
Point your existing client at a new endpoint and serve any pre-compiled model: no rewrite, no proprietary API to learn.
pip install furiosa-llm
furiosa-llm serve [model]
# Serving on :8000 (OpenAI-compatible)from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8000/v1"
)
client.chat.completions.create(model="[model]", ...)from furiosa_llm import LLM
llm = LLM(
model="[model]",
dtype="[precision]",
)
out = llm.generate(prompts)Model Zoo
Start from a model that already runs
OPENAI
gpt-oss
20B · 120B
FP8
Text generation
QWEN
Qwen3
4B · 8B · 32B
FP8
Text generation
UPSTAGE
Solar Open
100B
NVFp4A16
Text generation
Qwen3 | 4B · 8B · 32B FP8 TEXT GENERATION | |
Qwen3-Coder | 30B-A3B FP8 CODE GENERATION | |
Qwen3-Embedding | 0.6 · 4B · 8B BF16 TEXT GENERATION | |
gpt-oss | 20B · 120B MXFP4 TEXT GENERATION | |
Solar Open | 100B NVFP4A16 TEXT GENERATION | |
EXAONE 4.5 | 33B FP8 Vision-Language | |
EXAONE 3.5 | 7.8B · 32B BF16 TEXT GENERATION | |
Llama 3.3 | 70B FP8 · INT8 · FP16 Text Generation | |
DeepSeek-R1-Distill | 7B · 8B · 14B · 32B · 70B BF16 Reasoning | |
Qwen2.5 | 0.5B · 7B · 14B · 32B BF16 TEXT GENERATION | |
BGE-M3 | 0.6B BF16 Embedding | |
E5-Mistral | 7B FP16 EMBEDDING | |
Qwen3 MoE | 30b-A3B FP8 TEXT GENERATION | |
Qwen3-VL | 2B · 4B · 32B BF16 Vision-Language | |
Qwen3-Reranker | 0.6B · 4B · 8B BF16 Reranking | |
Gemma 4 | 31B FP8 TEXT GENERATION | |
K-EXAONE | 236B-A23B NVFP4A16 TEXT GENERATION | |
EXAONE 4.0 | 32B FP8 Text Generation | |
Mistral NeMo | 12B FP8 TEXT GENERATION | |
Llama 3.1 | 8B FP8 · BF16 TEXT GENERATION | |
QwQ | 32B BF16 Reasoning | |
Qwen2.5-Coder | 7B · 14B · 32B BF16 CODE GENERATION | |
BGE-Reranker-v2-M3 | 0.6B BF16 RERANKING | |
Harrier-OSS-v1 | 0.6B BF16 RERANKING | |
DEPLOYING
EASY DEPLOYMENT
Pick the deployment path that fits your needs
BARE METAL
A dedicated NXT RNGD Server, hosted at a data center near you.
API ENDPOINT
Call models served on RNGD through our cloud partners. Nothing to install.
PROGRAMMING
FURIOSA-TORCH
Move your model to RNGD with furiosa-torch
Bring your own PyTorch model, including ones not in the model zoo. furiosa-torch adds RNGD as a PyTorch device: run it as-is (eager mode) while you develop, compile it with torch.compile for performance, or build it ahead of time for deployment.
① Move to the device
import torch, furiosa.torch
device = torch.device("furiosa:0")
model = MyModel().to(device)② Run it your way — three paths, one backend
out = model(x.to(device)) # eager: ops run as dispatched
compiled = torch.compile(model, backend=furiosa.torch.backend)
out = compiled(x.to(device)) # whole-graph compile
runnable = CompileModule.from_module(model, inputs) # AOT → RunnableFURIOSA-TCL / FURIOSA-VISA
Two levels of kernel programming
COMING Q4 2026
TCL
Tensor Contraction Language
Declare what to compute over named symbolic axes. Kernels are generic over shape; the compiler owns tiling, layout and scheduling.
Declare what to compute over named symbolic axes. Kernels are generic over shape; the compiler owns tiling, layout and scheduling.
import furiosa.tcl as tcl
@tcl.kernel
def linear(x: tcl.bf16[M, K], w: tcl.bf16[N, K]) -> tcl.f32[M, N]:
# axes missing from the output are contracted away
return tcl.operation.contract(x, w, out=[M, N])COMING Q4 2026
vISA
TCP Virtual ISA
Hand-optimize the hot path. Reason in tensors while directly managing memory placement and Tensor Unit scheduling.
Hand-optimize the hot path. Reason in tensors while directly managing memory placement and Tensor Unit scheduling.
#[device(chip=1)]
fn gemm(a: DmTensor<bf16, m![M, K]>,
b: DmTensor<bf16, m![K, N]>) -> DmTensor<f32, m![M, N]> {
// you choose the memory tier and the schedule
a.fetch().contract(b).commit()
}Dive Deeper
TCP Paper, presented at ISCA 2024
Download the peer-reviewed architecture paper.
Read article
Keynote video by CTO and CRO
Watch a video where our engineering leads go in-depth of our HW and SW design philosophy.
Read article
Hot Chips Presentation by CEO June Paik
Watch the full presentation by our CEO at Hot Chips 2025.
Read article
