RynnVLA-Latent-2B

Stage-1 latent-action pretraining checkpoint of RynnVLA-Latent, a Mixture-of-Transformers vision-language-action model. Given a language instruction and up to six camera views, it generates a chunk of latent actions by flow matching.

• Parameters: 2,510,917,041 (2.511 B), 1,186 tensors, all bfloat16
• Weights: EMA snapshot at the final training step 236,442
• Frozen during Stage 1: the vision tower and the language model (parameter regexes visual\., language_model\.)
• Text backbone: hidden 2048, 28 layers, 16 attention heads / 8 KV heads, intermediate 6,144
• Exported: 2026-09-28

Code, data format and training scripts: github.com/alibaba-damo-academy/RynnVLA-Latent

---

Read this first: what this checkpoint is not

This is a Stage-1 checkpoint. Its flow-matching target is the latent action, not a robot action, so Stage 1 consumes no action labels at all.

As shipped, processor_config.json carries an empty action/state schema ("schema": {"action": {}, "state": {}}, "use_state": false), and the eval adapters in the repository explicitly refuse a Stage-1 checkpoint for closed-loop control — validate_libero_schema raises "Use a Stage2 action checkpoint, not a Stage1 latent-only checkpoint" whenever use_latent_actions is true.

So: do not point LIBERO / RoboTwin / VLABench evaluation at this repo. Use it as the initialization for Stage 2, which keeps the same flow-matching objective and retargets it from latent actions to a robot-specific action space. action_dim = 81 and action_chunk_size = 30 are declared in the config and carried in the architecture, but Stage-1 training does not use them.

Latent-action protocol (608-dim ktoken_zcam)

Stage 1 trains against latent actions produced by RynnLAM, which labels raw RGB frame pairs and needs no action space — that is what lets real-robot trajectories, simulation rollouts and human egocentric video share one target representation.

Field	Value
latent_action_dim	608 = 8×64 k-tokens (512) + 64 z + 32 camera-pose
latent_action_chunk_size	6
latent_action_stride	5
Labeling cadence	gap = 4, pair_stride = 4 — one latent action per 4 RGB frames

Architecture

Config	Value
model_type	rynn_brain_vla
architectures	Qwen3VLForConditionalGeneration
action_head_type	expert
expert_backbone	qwen3_vl
expert_hidden_size	768
time_conditioning	adaln
num_view_slots	6 (head, left_wrist, right_wrist, front_third, side_left, side_right)
use_view_cond_slots	true
Vision encoder	qwen3_vl, depth 24, hidden 1024, patch 16, spatial merge 2, deepstack layers [5, 11, 17]
Tokenizer	Qwen3 BPE, 151,936 embedding rows; vocabulary identical to the public Qwen3-VL tokenizer (151,643 tokens / 151,387 merges) plus one extra special token `<
transformers_version	5.2.0

Up to 6 camera roles are carried per sample with a slot_mask marking the valid ones, so a source with one camera and a source with four train in the same batch.

Images are normalized with image_mean = image_std = [0.5, 0.5, 0.5], i.e. rescaled to [-1, 1] (processor_class = RynnBrainVLAProcessor). The separate action_norm_type field is mean_std; it governs action/state normalization and is inert at Stage 1, where schema.action and schema.state are both empty.

Loading

The custom model_type is resolved by the rynnvla package, so import it first:

import torch
from rynnvla.models.rynn_brain_vla import RynnBrainVLAModel, RynnBrainVLAProcessor

path = "./RynnVLA-Latent-2B"   # a local directory, e.g. from snapshot_download below
model = RynnBrainVLAModel.from_pretrained(
    path, dtype=torch.bfloat16, attn_implementation="flash_attention_2",
)
processor = RynnBrainVLAProcessor.from_pretrained(path)

Verified on transformers==5.2.0 / torch==2.7.1 with both attn_implementation="eager" and "flash_attention_2": the checkpoint materializes with zero missing and zero unexpected keys, the parameter count equals the safetensors header count exactly (2,510,917,041), and no tensor contains a non-finite value. The only exactly-zero tensors are the three action-head biases (state_proj.bias, action_in_proj.bias, action_out_proj.bias), which Stage 1 does not train; their weights are non-zero.

Download from ModelScope:

from modelscope import snapshot_download
model_dir = snapshot_download("DAMO_Academy/RynnVLA-Latent-2B")

Files

File	Bytes	SHA-256
LICENSE	11,358	cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
chat_template.jinja	5,292	3636d0f0bd6bef02654cdffdc447b79cb2cef8ab02cc75267345946291a489e4
config.json	3,208	8ff25f9a6a5c12661da19c270f8a6e4ba4a766e83820acde648ea59730496fd3
export_provenance.json	594	8c314bd7eab7a1bb4f5f2b71a858f684b38478be72c3fa388aefc52f0f54221d
model.safetensors	5,021,978,274	e3e7155dda69b38aa05fc6a107353d8fe3a7499bd755ea80db16ec3299992784
processor_config.json	1,501	2d3f06f58b85154a00da0905c853e92fe760e2e3a6b066935812707024a95e58
tokenizer.json	11,422,840	f9c0d33d111cb70baf6140ce788268a980ad8b6f70981c35b76193bfa5fb45aa
tokenizer_config.json	492	e6ee285337a50ab8c6572ac8967fd02fc3da292bc50bc2835b0d873c4fbe61b7

export_provenance.json records the source step, the EMA selection, the frozen-parameter regexes, the source weight digest and the tensor count. Internal absolute paths were removed from it before release, which changes only that file's own digest.

Training

Stage-1 latent-action pretraining on the full latent-action corpus, global batch size 512, run to completion at step 236,442. The released weights are the EMA snapshot of that final step.

License

Apache License 2.0.

Links

• Code & training scripts: github.com/alibaba-damo-academy/RynnVLA-Latent
• RynnLAM (produces the 608-dim latent actions): modelscope.cn/models/DAMO_Academy/RynnLAM
• Sibling checkpoints: RynnVLA-Latent-2B · RynnVLA-Latent-4B