Announcing SEA-LION v5: Our Largest Model & Frontier Level Performance – and Open!
Today, we are thrilled to announce SEA-LION v5, our most expansive model family yet, built on three leading open foundations: Qwen, Nemotron, and Gemma. To scale capability for SEA from resource-constrained edge devices to massive enterprise reasoning clusters, the full v5 family encompasses a diverse ensemble of models and sizes: Qwen (397B, 27B, 9B), Nemotron 3.5 Bolt E4B, and Gemma 4 E2B IT. This multi-architecture approach allows us to push the boundaries of regional language understanding while offering unprecedented deployment flexibility.
The family is anchored by the massive Qwen-SEA-LION-v5-397B-A17B-Plus Mixture of Experts (MoE) flagship model fine-tuned specifically for high-performance multilingual, multicultural and agentic tasks. Qwen-SEA-LION-v5-397B-A17B-Plus, our largest SEA-LION v5 model, is built on Qwen3.5-397B-A17B and adds three components from Qwen3.8-Flash-Next: Qwen Sparse Attention (faster long-context inference), Gated Residual (more flexible information flow between layers) and N-gram Embedding (extra capacity at little compute cost, kept in host memory).
Unmatched Regional Performance Among Open-Source Models
Our flagship SEA-LION v5 397B-Plus model sets a new standard for Southeast Asian linguistic and cultural reasoning. Evaluated across a diversity of tasks integrated into our comprehensive SEA-HELM evaluation suite, the v5 family demonstrates significant leaps in instruction following, local knowledge understanding, and multilingual generation compared to previous generations and commercial alternatives.



SEA-LION v5 397B-Plus Architecture Conversion
For our SEA-LION v5 397B-Plus model, we built upon the Qwen architecture. Specifically, we converted Qwen3.5-397B-A17B to Qwen4Exp, the architecture used by Qwen3.8-Flash-Next. We used the same three core components and adapted their settings where needed:
- QSA Hybrid Attention: Qwen4Exp pairs Gated DeltaNet with Qwen Sparse Attention. In each QSA layer, a small indexer scores compressed four-token blocks of the context. For each query, attention uses up to 2,048 tokens from the highest-scoring complete blocks, plus any tokens in the incomplete trailing block. This reduces the work of full attention on long contexts. We kept Qwen3.5-397B’s 45 Gated DeltaNet layers and converted its 15 full-attention layers to QSA with Qwen4Exp’s indexer settings.
- Gated Residual: Qwen4Exp widens the residual stream into four parallel branches. Each attention and feed-forward block reads an input-dependent blend of them and writes its output back with one weight per branch, so layers can route information more flexibly. We used the same four branches, with the bottleneck scaled to our wider model.
- N-gram Embedding: Qwen4Exp adds a hashed bigram and trigram memory at layer 2. The tables use sparse lookups and can sit in host memory. We seeded our tables from Qwen3.8-Flash-Next’s trained ones, widened each head from 160 to 256 channels for our 4,096-wide backbone, and doubled the buckets to about 40M per head. That gives about 164B parameters in the n-gram tables.
We also added QSA and Gated Residual to the MTP draft layer and kept Qwen3.5-397B’s vision tower, tokenizer, and SiLU output gating in Gated DeltaNet (as opposed to the sigmoid in the Flash-Next).
Improving SEA-LION v5 for SEA with Post-training
We trained each student with a group-relative policy-gradient (PG) loss and an on-policy distillation (OPD) loss in the same update. For each prompt, the student generated eight rollouts (sampled responses). DeepSeek V4.1 Flash served as the judge, scoring each response’s reasoning and final answer against a shared rubric. The GRPO-style PG loss reinforced or discouraged the student’s choices according to how each response scored relative to the other attempts. When the scores were too close to distinguish the responses, we skipped the group’s PG contribution. In assisted rounds following eight failed attempts, the student received the start of a verified teacher solution and trained only on the continuation it generated.
Distillation from a larger teacher. OPD supplied the teacher’s next-token predictions along the student’s own response. Qwen 3.8 2.4T taught Qwen-SEA-LION-v5-397B-A17B-Plus, Qwen-SEA-LION-v5-27B and Qwen-SEA-LION-v5-9B, while Gemma 4 31B-IT taught Nemotron-SEA-LION-v5-E4B and Gemma-SEA-LION-v5-E2B. At each scored position, we kept the probabilities of the teacher’s 32 most likely next tokens and a tail bucket (the combined probability of all other tokens).
The verdict constrained distillation. We adapted OPDVR (outcome-guided feedback on sampled tokens) to our top-32-plus-tail distillation loss. When a group contained both passes and failures, we removed only the component of the distillation gradient that would locally lower a sampled token’s probability on a passing response or raise it on a failed one.
Learning when to conclude. When the judge found that the student had reached a conclusion but kept reasoning, we branched at that point, closed the reasoning, and let the student answer. We also generated an answer when the student stopped with its reasoning still open. If the branch passed, we applied targeted SFT (supervised fine-tuning) to the closing tokens. Shared reasoning received credit once, and the original ending received its own update from the point where the paths diverged. A response that reached the token limit could still earn credit for useful progress. We observed that by actively teaching SEA-LION-v5 when to conclude, the token output for its reasoning and final response was greatly reduced without affecting reasoning and response quality.
Breaking loops. For exact periodic loops (a token sequence repeating back-to-back), we followed Liquid AI’s Antidoom approach. Once the judge confirmed that useful progress had stopped, we applied a margin loss at the start of the loop, training the student to score plausible alternatives above the token that began the repetition.
Asynchronous training. While the judge scored one group, Conductor (our workflow coordinator) could send another to NVIDIA NeMo AutoModel (our training framework) for a weight update. The trainer computed shared prefixes once while preserving each branch’s targets. Awex (weight transfer) published the updated weights to SGLang (model serving). Archipelago (our deployment manager) let us resize the student, teacher, and judge fleets independently as bottlenecks shifted.
Because the trainer could take several steps while the judge scored a response, we tracked which weight version generated each token and excluded tokens beyond a set version gap. For eligible tokens, an entropy-aware filter compared the token’s log probability under the current weights with its value at sampling time. It skipped the PG update when the difference was too large, with a tighter threshold where the generating student had been more confident.
SEA-LION v5 397B-Plus note. Following Qwen 3.8 Flash Next, we use the model’s own dense attention scores to train its Qwen Sparse Attention indexers (the components that select which blocks of earlier tokens to retrieve). Training is still in progress.
Lower Inference costs for Qwen-SEA-LION-v5-397B-A17B-Plus
Post-training improved answer quality on Southeast Asian tasks while reducing how much text the model generated. On SEA-HELM, Qwen-SEA-LION-v5-397B-A17B-Plus used 1,889 tokens per prompt on average for reasoning and the final answer combined, 53.50% fewer than the base model.

Serving the BF16 model (16-bit floating point) requires a full node of eight H200 GPUs. We also release two quantized versions, FP8 and NVFP4, each of which runs on four H200s. We serve both with vLLM and SGLang.
Almost all the model’s weights sit in its routed experts (the experts selected for each token) and its n-gram tables (lookups for token sequences). Both versions quantize the routed experts, and FP8 also quantizes the n-gram tables. Attention, Gated DeltaNet (the recurrent layers), shared experts, the router, and the vision encoder remain in BF16. Together, these components account for a small share of the model’s weights.
FP8 uses the same block-wise 8-bit format as Qwen’s FP8 releases, so it runs with standard FP8 support in vLLM and SGLang. Storing the n-gram tables in FP8 halves their host memory requirement. The full model is about half the size of BF16.
NVFP4 uses NVIDIA Model Optimizer to store the routed expert weights in 4 bits. Blackwell GPUs support NVFP4 natively. On four H200s, NVFP4 leaves more than twice as much GPU memory available as FP8 for longer contexts and concurrent requests.
On SEA-HELM, both quantized versions showed little to no quality loss relative to BF16. In our H200 throughput tests, both began responding sooner and generated tokens faster than BF16.

Inference: Run SEA-LION v5 in production with Adaption
We’re partnering with Adaption as an inference partner for SEA-LION. Our Qwen-SEA-LION-v5-397B-A17B-Plus and Nemotron-SEA-LION-v4.8-120B-A12B are available on Adaption’s platform. While most inference providers stay as efficient as they were on day one, Adaption’s system learns and improves from the traffic it serves.
Getting started
- Try it with a free trial key from the SEA-LION Playground. Trial keys are rate-limited and not meant for production use.
- Take it to production with Adaption. Get started on Adaption’s free tier, then move to the Builder tier as you grow. No large upfront contracts. Enterprise teams can run on dedicated instances, on the same terms.
- Read the docs in our Inferencing guide, to find out about other inference providers and guidance for self-hosting our SEA-LION models.
More SEA-LION models are coming to Adaption, so stay tuned.
Get Started
The SEA-LION v5 models are currently rolling out. We invite the community to test the weights, explore the architectures, and push the limits of AI for Southeast Asia.
- 📁 Download Weights on Hugging Face: https://huggingface.co/collections/aisingapore/sea-lion-v5
- 💻 Documentation & Server Repositories: https://docs.sea-lion.ai/
