Uplifting AI in Southeast Asia (SEA): Announcing Nemotron-SEA-LION-v4.8 in Collaboration with NVIDIA

ByAISG | AI Products

AI Singapore is thrilled to announce the Nemotron-SEA-LION-v4.8 suite of models, developed in deep technical collaboration with NVIDIA. By combining NVIDIA NeMo Megatron Bridge and NVIDIA NeMo AutoModel training frameworks with AI Singapore’s extensive regional data expertise, the partnership has produced open-weights models built specifically for Southeast Asia’s diverse linguistic landscape.

Built on the NVIDIA Nemotron 3 family of foundation models, spanning Nano (30B-A3B) and Super (120B-A12B), this release marks two firsts for the SEA-LION series: its first Mixture-of-Experts (MoE) architecture and its first Nemotron-based models. By activating only a fraction of their total parameters at inference, 3B and 12B respectively, the models deliver enterprise-ready performance and deep regional understanding at significantly lower compute cost and latency. Quantized variants are also available for resource-constrained edge deployments, localized server hosting, and high-throughput production environments.

Crucially, the adaptation achieves major performance gains across Southeast Asian languages over the original Nemotron 3 baselines. Beyond headline scores, the v4.8 suite dramatically improves lower-resource scripts that are traditionally underrepresented in foundation models. At the 120B scale, on SEA-HELM, Burmese climbs from 4.98 to 31.35 (6.3x) and Tamil from 21.83 to 56.35 (2.6x), alongside gains in instruction following, natural language understanding, and cultural context.

Through this joint effort, AI Singapore and NVIDIA are ensuring that AI for the region is built transparently and equitably, and performs for every language it serves.

Targeted Continued Pre-Training (CPT)

To transform base Nemotron checkpoints into regional models for Southeast Asia, the aisingapore/Nemotron-SEA-LION-v4.8-30B-A3B-Base model underwent Continued Pre-Training (CPT) on 150 billion tokens, while the larger 120B variant aisingapore/Nemotron-SEA-LION-v4.8-120B-A12B-Base was trained on 33.5 billion tokens. Our primary training frameworks were NeMo Megatron Bridge and NeMo AutoModel, chosen for their ease of use, throughput, training stability, support for diverse training topologies, and rich set of out-of-the-box features, all backed by active maintenance and contributions from the NVIDIA team and the open-source community.

The Continuous Pre-Training mixture favors structured, downstream capability-oriented formats over raw web text: question answering, chain-of-thought and reasoning data, and parallel translation corpora. This is intended to prepare the model for further post-training. This training mixture combines general English data with coverage of 10 Southeast Asian languages, and includes parallel data pairing those languages with English to strengthen multilingual representation and cross-lingual transfer.

Optimization and scheduling

CPT used a peak learning rate of 3.03 × 10⁻⁵ on a Warmup-Stable-Decay (WSD) schedule, omitting the final decay phase. Holding the learning rate at its stable value keeps the loss landscape adaptable and leaves optimization headroom for post-training, rather than driving aggressive convergence at the end of pre-training.

The training mixture CPT datasets are adapted from

Pie chart displaying the breakdown of a dataset with various categories and their respective percentages, highlighting the most significant segments.

Post-Training:

Post-training was a major focus in developing aisingapore/Nemotron-SEA-LION-v4.8-30B-A3B and aisingapore/Nemotron-SEA-LION-v4.8-120B-A12B. Rather than a series of separate offline stages, we built a single continuous pipeline in which generation, scoring, and learning run at the same time and none waits for the others ie. asynchronously. Curated examples feed Supervised Fine-Tuning (SFT), and the model’s own live generations feed On-Policy Distillation (OPD). SFT exposes the model to responses it would not have produced on its own; OPD corrects the ones it does produce.

The training service is built on NeMo AutoModel, extended to run OPD and SFT concurrently. The model that is being trained, the student, generates live interaction trajectories. A much larger model, the teacher, scores them as they arrive, and the student’s weights are updated continuously. Our Conductor gateway service captures every trajectory token-exactly (avoiding tokenization drift) and tags every generated token with the weight version that produced it. The trainer never waits on generation: it takes whatever trajectories are ready, oldest first, keeping generations from the same prompt together, and skipping or masking anything beyond a specified staleness window.

Distillation depends on the teacher. Ideally, it shares the student’s tokenizer and is able to guide the student towards better responses. We found that in NVIDIA’s flagship Nemotron 3 Ultra 550B-A55B and with the RedHatAI’s Nemotron 3 Ultra 550B-A55B FP8 checkpoint, each teacher replica fits on a single 8xH200 node. The teacher does not rescore whole sequences (which would be expensive). It scores only the tokens that the student will be trained on and returns, for each token, the top-32 candidate tokens with their log-probabilities and the remaining tail mass, all in a compact binarized form (to ease movement between the various components in the pipeline).

Trajectories come from our environment-agent orchestrator, which scales environments and agent harnesses to match training, serving, and reward capacity. It comfortably sustains 3,000 active agent instances across more than 100,000 live interaction sessions, all feeding one model. Each turn yields several generations: the teacher scores every one, the agent continues with one of them, and the rest become additional training signal. Sessions may span many turns, so the model learns from whole conversations and multi-step actions.

Agentic load is uneven and unpredictable, and the serving and reward fleets are built for it. Both fleets are built atop SGLang, route traffic to the right workers to avoid duplicate prefill, and grow or shrink within a shared compute budget as generation and scoring pressure change. The serving fleet runs a copy of the student and receives new weights every time the student is updated, keeping it aligned with the trainer. Publishing the full 120B model to the serving fleet takes two to three seconds, streamed GPU to GPU over NVIDIA’s NCCL and the cluster’s InfiniBand fabric, and overlaps with training computation, so training itself never waits on it.

The result is a model trained on its own live behavior at scale, with improved localized linguistic precision, agentic adaptability, and computational efficiency across Southeast Asian languages.

A flowchart illustrating the structure of an EMV-Agent orchestrator, detailing components such as environments, agent loops, a conductor, a data pool, inference/serving routes, reward systems, and data transfer mechanisms.

Our Post-Training Pipeline

Model Performance and SEA-HELM Evaluation

A radar chart comparing overall results of Nemotron SEA-LION models with two configurations of the Nemotron baseline: 30B-A3B and NVIDIA-Nemotron-3.5-Lightning and a second chart for the 120B-A12B model.

On the SEA-HELM benchmark, aisingapore/Nemotron-SEA-LION-v4.8-30B-A3B and aisingapore/Nemotron-SEA-LION-v4.8-120B-A12B both improve markedly over their Nemotron 3 baselines. Continued pre-training and post-training together lift the overall score by 4.68 points at 30B (46.89 to 51.57) and by 14.14 points at 120B (49.30 to 63.44). The gains hold across all 7 evaluated regional languages: Burmese, Filipino, Indonesian, Malay, Tamil, Thai, and Vietnamese.

The 30B-A3B model gains most in Instruction Following, where Burmese rises from 15.47 to 53.13, and in Natural Language Understanding, where Tamil reaches 75.26 against 19.38 for the Nemotron 3 Nano baseline. The 120B-A12B model improves across all eight capability categories, most notably in Cultural Knowledge, where every one of the seven languages improves, and in General Knowledge and Natural Language Reasoning, where the largest gains are in lower-resource languages such as Burmese and Tamil.

Table comparing various AI models based on size, organisation, and performance metrics across different regions.

Complete Ecosystem Delivery

Ready for the Edge:

To ease deployment on resource-constrained edge devices and local server infrastructure, the instruct models are also released in FP8, NVFP4, and GGUF variants. The FP8 and NVFP4 checkpoints were produced with NVIDIA Model Optimizer Library. All three formats cut the memory footprint substantially with minimal loss in model quality, enabling low-latency MoE inference from single-node servers up to high-throughput production deployments.

Model VariantNumber of GPUsActual VRAM Usage (GB)Time to First Token (s)Tokens per Second
30B-A3B(BF16)1x H200 GPU(s)63.3 GB0.0456311.10 ± 25.12
30B-A3B-FP81x H200 GPU(s)33.4 GB0.0350365.18 ± 31.50
30B-A3B-NVFP41x H200 GPU(s)20.5 GB0.0339340.29 ± 8.68
120B-A12B(BF16)2x H200 GPU(s)242.5 GB0.0936150.31 ± 12.80
120B-A12B-FP82x H200 GPU(s)128.6 GB0.0449161.36 ± 3.05
120B-A12B-NVFP42x H200 GPU(s)80.4 GB0.0446156.36 ± 2.64

*Note: Benchmarks were captured on Hopper architecture (NVIDIA H200) GPUs; NVFP4 precision formats yield further hardware-level acceleration when executed natively on NVIDIA Blackwell infrastructure.

What’s Next for SEA-LION

We are grateful to NVIDIA for their collaboration on this release, and the v4.8 suite is one step on a longer roadmap. Our research and engineering teams are exploring new model architectures and quantization approaches to improve efficiency and performance further. On the post-training side, we are growing the agentic orchestration environment with more complex multi-turn and tool-use tasks, and experimenting with new training objectives and reward mechanisms to push reasoning and alignment across Southeast Asian languages.

A thriving open-source ecosystem relies on community-driven innovation. We welcome researchers, industry partners, and developers to work with us, whether by contributing high-quality regional datasets, co-designing agentic environments, or pursuing joint research, as we build the future of AI for Southeast Asia together.

Get Started Today

The model weights, detailed model cards, and starter code are now available on our Hugging Face hub:

If you are building with the SEA-LION v4.8 suite, we would love to hear from you. For deployment inquiries or technical feedback, contact our team at sealion@aisingapore.org.