AMD publishes detailed Instella‑MoE technical report; weights licensed for research use

AMD has published a technical report for Instella-MoE, a 16 billion-parameter language model that it says was trained entirely on its own Instinct GPUs and released with an unusually broad set of research materials. The caveat is central: while the code is permissively licensed, the model weights are not. They are available under a research-only license rather than a standard commercial-friendly one.

The report, titled “Instella-MoE Technical Report” and posted to arXiv as arXiv:2609.00791v1, was submitted Sept. 1. It formalizes a release that had already appeared earlier through AMD’s Hugging Face and GitHub accounts, with Hugging Face collection metadata showing updates in late July. In the paper’s abstract, AMD says “we release the complete Instella-MoE model flow.” That includes model weights, training configurations, data mixtures, training code, evaluation protocols and checkpoints from each major stage — a fuller package than the typical model release that offers only final weights or limited documentation.

Those checkpoints span the full training pipeline: Pretrain, Midtrain, Base, SFT, DPO and RL, with the final checkpoint labeled “Think.” AMD also says the Base stage comes after extending the model’s context window to 64,000 tokens, allowing it to handle much longer prompts. The model itself is a decoder-only Transformer built as a mixture-of-experts, or MoE, system — meaning each token is routed to only part of the network instead of all of it, which increases total capacity without activating every parameter each time. According to AMD’s model card and report, Instella-MoE has 16 billion total parameters, with 2.8 billion active per token, using 64 experts, including shared and routed experts. AMD says it was trained from scratch on MI300X and MI325X chips using its ROCm software stack and Primus training framework, with reinforcement learning handled through its Miles framework. Pretraining used 7.1 trillion tokens from open corpora, AMD said.

AMD’s paper presents Instella-MoE as both a research model and a demonstration of its AI hardware and software stack. The company says the system was trained entirely on AMD accelerators rather than another vendor’s chips. On Hugging Face, AMD describes it as “a state-of-the-art fully open Mixture-of-Experts (MoE) language model,” though the licensing terms for the weights place important limits on that openness.

On performance, the paper reports an average score of 76.7 across AMD’s standard pretraining benchmark set. AMD also says the final Think checkpoint scored 73.2 on average across instruction-following, reasoning, math, coding and chat benchmarks. The report says the model outperformed several open or open-weight peers in some comparisons. But those figures should be read as author-reported results: the research material reviewed explicitly notes that no independent third-party benchmark re-evaluations were identified.

The licensing split is likely to matter as much as the model’s technical details. AMD’s training code is released under the MIT license, a standard permissive software license. The model weights on Hugging Face, by contrast, use a ResearchRAIL, or research-only, model license. AMD’s model card says the models are “licensed for academic and research purposes,” and the Hugging Face license file states, “BY ACCESSING, DOWNLOADING, INSTALLING, OR USING THE ARTIFACT, YOU AGREE TO BE BOUND BY THIS LICENSE.” In practice, that means this is not a conventional commercial open-weight release.

AMD’s model card also says the models are not intended for safety-critical, health or medical use cases, and warns that outputs may be harmful, biased or inaccurate.

For researchers, the significance is less about a single benchmark score than the package AMD has put together: a fully documented, stage-by-stage MoE release trained on AMD hardware, with reproducibility materials that are rare to see in one place, even if the weights remain limited to research use.