[Feature] Add GLM-5.3-Flash F1: VL data preprocessing pipeline

[jayhenry]

Stack (bottom to top):

1. [Feature] Add GLM-5.3-Flash F0: 25B cropped reference checkpoint builder #2105 feat/glm53flash-materialize-full-f0 → main
2. [Feature] Add GLM-5.3-Flash F3: Kimi Delta Attention (KDA) #2106 feat/glm53flash-f3-kda → feat/glm53flash-materialize-full-f0
3. [Feature] Add GLM-5.3-Flash F4: mHC four-stream residual #2107 feat/glm53flash-f4-mhc → feat/glm53flash-f3-kda
4. [Feature] Add GLM-5.3-Flash F5: NoPE DSA + KPool indexer + clamped SwiGLU #2108 feat/glm53flash-f5-nope-dsa → feat/glm53flash-f4-mhc
5. [Feature] Add GLM-5.3-Flash F1: VL data preprocessing pipeline #2109 feat/glm53flash-f1-vl-data → feat/glm53flash-f5-nope-dsa ← you are here
6. [Feature] Add GLM-5.3-Flash F2: vision tower + projector (eager) #2110 feat/glm53flash-f2-vision-tower → feat/glm53flash-f1-vl-data
7. [Feature] Add GLM-5.3-Flash F6 core: text model + MTP + compose model #2111 feat/glm53flash-f6-text-moe → feat/glm53flash-f2-vision-tower

Base is #2108's branch (layer 4). Review only this PR's own diff.

---

Summary

Stack layer 5/7 of GLM-5.3-Flash support (base: layer 4, F5 NoPE DSA). The F1 milestone — chat template, TokenizeFn (cache/runtime dual path), DataItem, and Collator/SequenceContext plumbing for image/video SFT — plus the three fixes that make it actually reachable from a training config.

F1 core

• xtuner/v1/data_proto/messages/glm53_chat.py: render_glm53_chat/Glm53ChatMessages, adapted from the landed glm52_chat.py at its two documented diffs. Verified byte-for-byte against the real chat_template.jinja.
• xtuner/v1/datasets/mllm_tokenize_fn/glm53_vl_tokenize_fn.py: placeholder expansion and mm_token_type_ids reuse the real Glm5NextProcessor's own public replace_image_token/replace_video_token/create_mm_token_type_ids in both the cache and runtime paths, so both run identical HF code rather than two independent reimplementations.
• xtuner/v1/datasets/collator.py: build_text_ctx_labels also returns kept_indices; glm53_vl_sft_collator built on top.

Two discrepancies were root-caused by direct package inspection rather than trusting the design doc: transformers 5.17.0 has no Glm5NextVideoProcessor.get_number_of_video_patches (the cache-path geometry is reproduced bit-for-bit via sample_frames + smart_resize instead), and an earlier version expanded the whole <|begin_of_image|>…<|end_of_image|> marker instead of just the inner <|image|>, undercounting num_tokens by 2 per media item.

Fixes that unblock VL training

These were found by running end-to-end VL SFT; each one fired before a single training step.

1. glm53_vl_sft_collator was unreachable. DataloaderConfig.build_collator() only knew the three older collators by name, and the dotted-path fallback could not supply the two extra arguments this collator needs (image_token_id/merge_unit, used to re-check the placeholder↔patch-count invariant after pack truncation). Any VL config died in the dataloader with a TypeError. Registers the name and adds a generic collator_kwargs bound into the collator partial; the name→function dispatch collapses into one lookup table so every branch, dotted path included, goes through the same binding.
2. Glm53VLTokenizeFnConfig could not be constructed. The class documents that GLM-5.3-Flash renders through render_glm53_chat instead of a CHAT_TEMPLATE_MAP entry, and build() never reads chat_template — but the base class declares the field required, so the documented construction raised ValidationError: chat_template Field required.
3. Not exported from the package root, unlike every other VL tokenize-fn config, so configs had to reach into the submodule by path.

Samples silently training as fakes

Surfaced by the VL e2e matrix: tgs sat at roughly half of seqlen_tgs on every profile, where the text matrix keeps them within 0.05%. They differ only by padding (mask.sum() vs mask.numel()), so VL packs were ~50% padding — one step packed 131 real tokens into a 16384 slot — even though the corpus held enough tokens to fill them. Comparing each sample's cached num_tokens (what the packer budgets with) against its runtime length found 17% of samples cached at 4096 but training as 33-token fakes. Two defects in the cache/runtime split compounded:

1. Frame-folder videos were indexed against the original video. Both paths built VideoMetadata from origin_video_length/origin_fps (182 frames @30fps), but a frame folder holds the video after resampling — processed_video_length frames at processed_fps (23 @4fps). Runtime indexed a 23-file folder with original frame numbers and raised IndexError: list index out of range; the dataset swallowed it and substituted a fake. Sampling now uses the processed sequence for frame folders — qwen3_vl's calc_frame_info rule 3, which also makes the frame timestamps correct. The folder/file decision is read from the path string (the trailing-/ convention qwen3_vl's reader keys on) rather than the filesystem, so the cache path, which never touches media, reaches the same answer as the runtime path.
2. Over-length visual samples were packed at their truncated length. With the index fixed, those samples expand to 14,352 placeholders, far past max_length. Truncating text is harmless; truncating through a visual span leaves fewer placeholders than patches. The cache path reported the truncated length anyway, so the packer reserved space and runtime then rejected the sample — again a fake. The cache path now reports such samples as damaged (num_tokens=0) so they are filtered before packing, the same check qwen3_vl's cache path makes.

Both escaped the existing tests for concrete reasons: the video fixture only set origin_* on a folder holding exactly that many frames, so origin and processed coincided; and the truncation test only asserted that the runtime path raises — precisely the behavior that gets silently turned into a fake — while the cache path, where the damage is decided, was never checked.

One consequence worth stating plainly: before this fix, no video sample ever actually trained in the VL e2e runs, despite the corpus containing 24 of them.

Test Plan

• tests/datasets/test_glm53_chat.py (byte-exact vs HF jinja), test_glm53_vl_collator.py (includes build_text_ctx_labels regression coverage for the qwen3_vl/intern_s1 collators that share the builder), test_glm53_vl_tokenize_fn.py (two tests assert input_ids/mm_token_type_ids match a real end-to-end processor() call token-for-token). CPU only.
• test_glm53_collator_is_reachable_by_name_from_dataloader_config builds the collator through DataloaderConfig and calls it with only the dataloader-supplied arguments — the exact path that was broken.
• Exercised end to end by the VL SFT run in layer 7.
• test_glm53_frame_folder_samples_the_processed_frames: a folder resampled from a longer original (180 frames @30fps → 6 @2fps); cache and runtime agree and runtime does not raise. test_glm53_visual_truncation_is_dropped now also asserts the cache path returns num_tokens == 0. Both fail on the previous code with the exact production errors (IndexError: list index out of range; assert 20 == 0) and pass with the fix.
• Over the whole ci_vl corpus: 0 cached/runtime length mismatches across the kept samples, with over-length samples filtered before packing (previously 10 of the first 60 mismatched).

[jayhenry]

去掉这个判断，走 Base vl tokenize 的统一逻辑？

[jayhenry]

参考 Qwen3VLTokenizeFunction 支持 oss_loader 读取

[jayhenry]

参考 Qwen3VLTokenizeFunction 支持 oss_loader 读取