release: 1.2.34

CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## 1.2.34 (2026-11-13)
+
+Full Changelog: [v1.2.33...v1.2.34](https://github.com/fw-ai-external/python-sdk/compare/v1.2.33...v1.2.34)
+
+### Bug Fixes
+* **training-sdk:** load tokenizer metadata containing `fix_mistral_regex` on Transformers 5.0–5.3 without duplicate-keyword errors, preserving the declared flag on save.
+* **training-sdk:** keep GLM-5, GLM-5.3 and GLM-5.3-Flash tokenizer loading in installed code, including opaque overrides and saved fast-tokenizer bundles.
+* **training-sdk:** retain up to 5,000 failed TITO traces by default, with 256 MiB of history per trajectory and an aggregate byte budget that accommodates the retention count; preserve explicit smaller limits and the free-space floor.
+
+### Documentation
+* **training-sdk:** clarify that `load_adapter` takes a promoted PEFT Model resource name.
+* **supervised fine-tuning jobs, DPO jobs:** managed supervised fine-tuning, DPO and ORPO no longer accept new jobs, and existing jobs can't be resumed: `create(...)` and `resume(...)` on `client.supervised_fine_tuning_jobs` and `client.dpo_jobs` now raise `BadRequestError`. Train new models with the Training API instead; see the [migration guide](https://docs.fireworks.ai/fine-tuning/training-api/migrate-from-managed). Jobs that are already running keep running, and `get`, `list`, `cancel` and `delete` keep working. The SDK itself is unchanged: the server enforces this.
+
 ## 1.2.33 (2026-11-08)
 
 Full Changelog: [v1.2.32...v1.2.33](https://github.com/fw-ai-external/python-sdk/compare/v1.2.32...v1.2.33)

README.md

@@ -466,13 +466,15 @@ while True:
 
 ### Dataset file format
 
-Dataset files should be in JSONL format (JSON Lines), where each line is a valid JSON object. For chat-based fine-tuning, use the following format:
+Dataset files should be in JSONL format (JSON Lines), where each line is a valid JSON object. For chat data, use the following format:
 
 ```json
 {"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"}, {"role": "assistant", "content": "Hi there! How can I help you today?"}]}
 {"messages": [{"role": "user", "content": "What is 2+2?"}, {"role": "assistant", "content": "2+2 equals 4."}]}
 ```
 
+> **Note:** Managed fine-tuning no longer accepts new jobs, and existing jobs can't be resumed. To fine-tune a model, use the [Training API](https://docs.fireworks.ai/fine-tuning/training-api/migrate-from-managed), whose recipes read your own JSONL files.
+
 ### Managing datasets
 
 You can view and manage your datasets in the [Fireworks Dashboard](https://app.fireworks.ai/dashboard/datasets).

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "fireworks-ai"
-version = "1.2.33"
+version = "1.2.34"
 description = "The official Python library for the fireworks API"
 dynamic = ["readme"]
 license = "Apache-2.0"

src/fireworks/training/sdk/README.md

@@ -21,7 +21,7 @@ Inherited behavior remains the same (`forward`, `forward_backward_custom`, `opti
 Fireworks-specific additions:
 
 - `save_weights_for_sampler_ext(name, checkpoint_type=...)` — session-scoped sampler checkpoints with base/delta support, plus `merged_base` (LoRA-only) to fold the loaded adapter into the base and export a full `HF_BASE_MODEL`. Merged-base output defaults to the source storage format; use `export_precision="bf16"`, `"nvfp4"`, `"mxfp8"`, or `"fp8_block128"` only for an explicit output override.
-- `load_adapter(adapter_path)` — load HF PEFT adapter weights into a LoRA session (weights-only warm-start); required before a `merged_base` save. Prefer a Fireworks PEFT model resource (`accounts/<acct>/models/<lora-id>`); a `gs://` PEFT directory also works.
+- `load_adapter(adapter_path)` — load HF PEFT adapter weights into a LoRA session (weights-only warm-start); required before a `merged_base` save. Pass a promoted PEFT Model resource name (`accounts/<account>/models/<name>`).
 - `list_checkpoints()`
 - cross-job checkpoint references for resume (`resolve_checkpoint_path(...)`)
 

src/fireworks/training/sdk/_core/lib/public_interfaces/sampling_client.py

@@ -17,8 +17,11 @@
 from fireworks.training.sdk._core.lib.sidecar import SidecarRPC, SidecarHandle, create_sidecar_handle
 from fireworks.training.sdk.tokenizer_identity import (
     is_code_free_model_name,
+    is_minimax_m2_model_name,
     is_kimi_k3_k31_model_name,
+    is_minimax_m2_tokenizer_source,
     is_kimi_k3_k31_tokenizer_source,
+    is_standard_tokenizer_model_name,
     requires_code_free_tokenizer_loading,
 )
 from fireworks.training.sdk._core.lib.telemetry import Telemetry, capture_exceptions
@@ -503,8 +506,13 @@ def _load_tokenizer_from_model_info(
     Returns:
         The loaded PreTrainedTokenizer
     """
+    # lazy: tokenizer dependencies and their compatibility backport are optional.
     from transformers.models.auto.tokenization_auto import AutoTokenizer
 
+    from fireworks.training.sdk.tokenizer_compat import patch_duplicate_mistral_regex_kwarg
+
+    patch_duplicate_mistral_regex_kwarg()
+
     model_name = model_name.split(":")[0]
 
     # Use tokenizer_id if provided, otherwise fall back to heuristic logic
@@ -557,6 +565,10 @@ def _load_tokenizer_from_model_info(
 
     if (
         is_code_free_model_name(model_name)
+        or is_code_free_model_name(model_arch)
+        or is_standard_tokenizer_model_name(model_name)
+        or is_standard_tokenizer_model_name(model_arch)
+        or is_standard_tokenizer_model_name(tokenizer_id)
         or model_arch in {"DeepseekV4ForCausalLM", "DeepseekV41ForCausalLM", "DeepseekV41ForConditionalGeneration"}
         or requires_code_free_tokenizer_loading(tokenizer_id, revision=kwargs.get("revision"))
     ):
@@ -565,4 +577,27 @@ def _load_tokenizer_from_model_info(
         # checkpoint-staged Python inert.
         kwargs["trust_remote_code"] = False
 
+    if any(is_minimax_m2_model_name(name) for name in (model_name, model_arch)) or is_minimax_m2_tokenizer_source(
+        tokenizer_id
+    ):
+        return _load_minimax_tokenizer_json(tokenizer_id, fast=True, **kwargs)
+
     return AutoTokenizer.from_pretrained(tokenizer_id, fast=True, **kwargs)
+
+
+def _load_minimax_tokenizer_json(tokenizer_id: str, **kwargs: Any) -> PreTrainedTokenizer:
+    # lazy: both SDK clients need this optional reader only for selected MiniMax data.
+    import transformers
+    from transformers.models.auto.tokenization_auto import AutoTokenizer, get_tokenizer_config
+
+    kwargs["trust_remote_code"] = False
+    tokenizer_backend = getattr(transformers, "TokenizersBackend", None)
+    if tokenizer_backend is None:
+        # Preserve each caller's prior installed-code reader and keyword arguments.
+        return AutoTokenizer.from_pretrained(tokenizer_id, **kwargs)
+    config = get_tokenizer_config(tokenizer_id, **kwargs)
+    declared_flags = {key: config[key] for key in ("add_bos_token", "add_eos_token") if key in config}
+    tokenizer = tokenizer_backend.from_pretrained(tokenizer_id, **{**declared_flags, **kwargs})
+    # The generic installed reader needs no checkpoint-code map on export.
+    tokenizer.init_kwargs.pop("auto_map", None)
+    return tokenizer

src/fireworks/training/sdk/client.py

@@ -43,8 +43,11 @@
 from fireworks.training.sdk._core._compat import model_dump as _training_model_dump
 from fireworks.training.sdk.tokenizer_identity import (
     is_code_free_model_name,
+    is_minimax_m2_model_name,
     is_kimi_k3_k31_model_name,
+    is_minimax_m2_tokenizer_source,
     is_kimi_k3_k31_tokenizer_source,
+    is_standard_tokenizer_model_name,
     requires_code_free_tokenizer_loading,
 )
 
@@ -2809,6 +2812,11 @@ def weight_sync(self) -> APIFuture[WeightSyncResponse]:
         because legacy sampler saves do not report it. RDMA negotiation, API, and
         publication errors are surfaced without a FILE retry.
 
+        An RDMA deployment remains bound to its first trainer session. Reuse
+        that session for subsequent updates; a new trainer session requires a
+        new deployment. Resetting a dedicated trainer session after RDMA sync
+        has started is rejected before unloading its models.
+
         Save resumable checkpoints separately with ``save_state()``.
         """
         from fireworks.training.sdk._rdma import _RdmaSamplerBackend
@@ -3116,22 +3124,33 @@ def get_tokenizer(self):
         # lazy: tokenizer downloads are optional; transport needs no transformers.
         from transformers import AutoTokenizer
 
-        if is_kimi_k3_k31_model_name(
-            getattr(self, "_tokenizer_base_model", None)
-        ) or is_kimi_k3_k31_tokenizer_source(self._tokenizer_model):
+        from fireworks.training.sdk.tokenizer_compat import patch_duplicate_mistral_regex_kwarg
+
+        patch_duplicate_mistral_regex_kwarg()
+
+        if is_kimi_k3_k31_model_name(getattr(self, "_tokenizer_base_model", None)) or is_kimi_k3_k31_tokenizer_source(
+            self._tokenizer_model
+        ):
             # Recognized reviewed Kimi K3/K3.1 data: build the id-identical
             # fast tokenizer from reviewed installed code. Checkpoint Python
             # (the auto_map TikTokenTokenizer) is never imported, and an
             # explicit trust_remote_code elsewhere cannot reopen it.
             from fireworks.training.sdk.kimi_native_tokenizer import load_native_tokenizer
 
             return load_native_tokenizer(self._tokenizer_model)
-        if is_code_free_model_name(
-            getattr(self, "_tokenizer_base_model", None)
-        ) or requires_code_free_tokenizer_loading(self._tokenizer_model):
-            # Reviewed DeepSeek V4/V4.1 data ships no remote code; an
-            # explicit False resolves the identical tokenizer class and keeps
-            # any checkpoint-staged Python inert.
+        base_model = getattr(self, "_tokenizer_base_model", None)
+        if is_minimax_m2_model_name(base_model) or is_minimax_m2_tokenizer_source(self._tokenizer_model):
+            # lazy: share the literal reader without sharing the sampling client's cache.
+            from fireworks.training.sdk._core.lib.public_interfaces.sampling_client import _load_minimax_tokenizer_json
+
+            return _load_minimax_tokenizer_json(self._tokenizer_model)
+        if (
+            is_code_free_model_name(base_model)
+            or is_standard_tokenizer_model_name(base_model)
+            or is_standard_tokenizer_model_name(self._tokenizer_model)
+            or requires_code_free_tokenizer_loading(self._tokenizer_model)
+        ):
+            # Independently selected code-free context keeps checkpoint Python inert.
             return AutoTokenizer.from_pretrained(self._tokenizer_model, trust_remote_code=False)
         return AutoTokenizer.from_pretrained(self._tokenizer_model)
 
@@ -3240,9 +3259,8 @@ async def load_state_with_optimizer_async(
     def load_adapter(self, adapter_path: str) -> APIFuture[LoadAdapterResponse]:
         """Load HF PEFT adapter weights into a LoRA training session (weights-only).
 
-        ``adapter_path`` is normally a Fireworks PEFT model resource
-        (``accounts/<acct>/models/<lora-id>``). A ``gs://`` PEFT directory also
-        works when you already have the adapter files.
+        ``adapter_path`` must be a promoted PEFT Model resource name
+        (``accounts/<account>/models/<name>``).
         """
         adapter_path = (adapter_path or "").strip()
         if not adapter_path:

src/fireworks/training/sdk/tito/_failure_trace.py

@@ -28,6 +28,10 @@
 
 logger = logging.getLogger(__name__)
 
+_DEFAULT_MAX_FAILED_TRACES = 5_000
+_DEFAULT_MAX_ACTIVE_TRACES = 128
+_DEFAULT_MAX_TRAJECTORY_BYTES = 256 * 1024 * 1024
+
 
 @contextmanager
 def _open_collected_trace(source: Path, trusted_root: Path | None) -> Iterator[BinaryIO]:
@@ -64,14 +68,17 @@ class TITOFailureTraceConfig:
     """Local-only temporary traces, removed after successful terminalization.
 
     Failed traces survive shutdown within these per-writer retention bounds.
+    Defaults allow 5,000 failed traces plus active trajectories at their full
+    per-trajectory limit. The byte budget is a ceiling, not preallocated disk;
+    the free-space floor still applies. Callers can set smaller local bounds.
     They contain raw requests and tokens; do not upload them automatically.
     """
 
     root_dir: Path = field(default_factory=lambda: Path(tempfile.gettempdir()) / "fireworks-tito-traces")
-    max_local_bytes: int = 256 * 1024 * 1024
-    max_trajectory_bytes: int = 32 * 1024 * 1024
-    max_failed_traces: int = 32
-    max_active_traces: int = 128
+    max_local_bytes: int = (_DEFAULT_MAX_FAILED_TRACES + _DEFAULT_MAX_ACTIVE_TRACES) * _DEFAULT_MAX_TRAJECTORY_BYTES
+    max_trajectory_bytes: int = _DEFAULT_MAX_TRAJECTORY_BYTES
+    max_failed_traces: int = _DEFAULT_MAX_FAILED_TRACES
+    max_active_traces: int = _DEFAULT_MAX_ACTIVE_TRACES
     min_free_bytes: int = 64 * 1024 * 1024
     enabled: bool = True
     stale_writer_age_seconds: float = 7 * 24 * 60 * 60

src/fireworks/training/sdk/tokenizer_compat.py

@@ -0,0 +1,62 @@
+"""Compatibility for tokenizer metadata emitted by newer Transformers."""
+
+from __future__ import annotations
+
+import functools
+import contextvars
+from typing import Any
+
+import transformers
+from packaging.version import Version
+
+
+# TEMPORARY DUPLICATION: standalone clients share this backport until their Transformers minimum is at least 5.4.
+def patch_duplicate_mistral_regex_kwarg() -> None:
+    """Backport the 5.4 duplicate-keyword fix without changing tokenizer bytes.
+
+    Mirrors the trainer and serving loaders without requiring either runtime
+    in client environments. The explicit flag must survive save_pretrained too.
+    """
+    if not Version("5.0.0") <= Version(transformers.__version__) < Version("5.4.0"):
+        return
+    # lazy: TokenizersBackend exists only in the affected Transformers versions.
+    from transformers.tokenization_utils_tokenizers import TokenizersBackend
+
+    init = TokenizersBackend.__init__
+    if getattr(init, "_fireworks_dedup", False):
+        return
+    patch = TokenizersBackend.__dict__["_patch_mistral_regex"].__func__
+    pending = contextvars.ContextVar("training_fix_mistral_regex", default=None)
+
+    @functools.wraps(init)
+    def wrapped_init(self: Any, *args: Any, **kwargs: Any) -> None:
+        fix = kwargs.pop("fix_mistral_regex", None)
+        token = pending.set(fix)
+        try:
+            init(self, *args, **kwargs)
+        finally:
+            pending.reset(token)
+        if fix is not None:
+            self.init_kwargs["fix_mistral_regex"] = fix
+
+    @functools.wraps(patch)
+    def wrapped_patch(
+        cls: Any,
+        tokenizer: Any,
+        pretrained_model_name_or_path: str,
+        fix_mistral_regex: bool | None = None,
+        **kwargs: Any,
+    ) -> Any:
+        if fix_mistral_regex is None:
+            fix_mistral_regex = pending.get()
+        return patch(
+            cls,
+            tokenizer,
+            pretrained_model_name_or_path,
+            fix_mistral_regex=fix_mistral_regex,
+            **kwargs,
+        )
+
+    wrapped_init._fireworks_dedup = True
+    TokenizersBackend.__init__ = wrapped_init
+    TokenizersBackend._patch_mistral_regex = classmethod(wrapped_patch)