Summary
Upstage
Foundation Models
Solar Pro 3
NXT RNGD Server
power efficiency vs. H200
1.3×
context window in production
128K
endpoint change to integrate
1
Power Efficiency
Upstage develops the Solar family of foundation models and makes them available through its API and enterprise deployments. Adoption often depends on both model capability and the cost of running it. Developers need models that perform well and run efficiently, which enterprises need inference economics that make sustained use practical.
Following its acquisition of Daum, one of South Korea’s largest search portals, Upstage began introducing AI-powered features to the service. One of the first features was AI-generated summaries in search results, powered by Solar Pro 3. Serving this consumer-facing experience introduced a new engineering challenge: handling nationwide search traffic, including daily fluctuations and sudden spikes, while meeting performance targets alongside H200 systems.
Challenge
Balancing Inference Economics and Performance
Upstage faced two distinct serving challenges simultaneously, each with different demands on inference efficiency. For its API business, inference cost per token affects both customer pricing and margins, influencing the economics of scaling Solar. On Daum, Solar Pro 3 serves search queries at consumer scale, requiring low time to first token, high concurrency, and efficient handling of long, document-heavy prompts amid sharp fluctuations in traffic.
Upstage wasn’t looking to replace its H200 fleet. Rather, it needed an additional inference platform to expand serving capacity, shift traffic as demand changed, and integrate with existing interfaces without requiring application rewrites or a separate operations stack.
Solution
Optimized End to End, Built to Run Beside GPUs
Engineers from Upstage and Furiosa worked across the inference stack, aligning model compression to serving topology. Each optimization informed the next, with both teams refining the configuration to maximize serving efficiency for Solar Pro 3 on Furiosa RNGD.
From Model Quantization to Serving
Solar Pro 3's 100B-class model was quantized to NVFP4, allowing each replica to fit across four RNGD cards while leaving memory available for the KV cache at 128K context. This memory budget informed the serving layout of two tensor-parallel replicas per eight-card server, with each replica’s memory footprint determining how much concurrent traffic it could support within its latency target.
Prefix caching helps reduce repeated computation for document prefixes shared across search requests. Before go-live, Furiosa provided an offline throughput profiles using furiosa-llm 2026.3.0rc3, giving both teams a performance baseline before routing live traffic.
Integrate Once, Operate Across GPU and NPU
RNGd supports the OpenAI-compatible API already used by Upstage's applications, including compatible reasoning-content and usage fields such as cached tokens. Integration required just one endpoint change, allowing Upstage to use the same serving interface across GPU and NPU capacity.
Solar Pro 3 remains an optimization target if the Furiosa SDK, with serving improvements delivered through future releases without requiring additional work from Upstage.
Target Workload
From API Inference to AI-Powered Search
Economics
1.3× Performance per Watt Compared with H200
In production serving of Solar Pro 3, Furiosa RNGD delivers 1.3× the performance per watt of an H200 server at the same latency targets.
For a workload like Daum search, efficiency matters throughout the day, not just at peak traffic. Furiosa RNGD’s low power draw during extended periods of lighter demand helps reduce the cost of maintaining serving capacity across the full 24-hour cycle. After the service stabilized, Upstage requested additional capacity, and expansion planning is underway.
Development
Stable Under Load, Serving Live Traffic
Before cutover, the serving configuration underwent soak testing over tens of hours across multiple server configurations, maintaining its latency targets throughout. Upstage moved live traffic to Furiosa RNGD in July 2026. In production, the teams track prefix-cache hit rate alongside latency and throughput to monitor serving performance. The collaboration has also expanded across the Solar family, beginning with Solar Embedding and Document Parse.
VISION
Sovereign AI That Competes on Merit
Upstage and Furiosa believe domestic AI technology must compete on merit, not national origin alone. AI models must earn adoption through quality and price, while inference compute must demonstrate competitive performance and efficiency, measured in tokens per watt. When Korean companies can deliver on both fronts, they offer customers a compelling alternative while strengthening Korea’s sovereign AI capabilities. The partnership between Upstage and Furiosa reflects this ambition of building AI technology that can compete on the global stage.
