Skip to content

VoxMens Serving Topology Decision

This document records the architectural decision for serving trained adapters and selecting base models across different spokes in VoxMens.

  1. Local/Offline Base Selection: Base model selection (capability tag to VRAM-fit base ID mapping) is resolved locally/offline at training time. There is no active network coupling or central inference serving registry required during fine-tuning.
  2. Adapter Multiplexing via DomainRouter: Inference-time multiplexing of trained spoke adapters is performed by hot-swapping domain adapters on top of a shared running base model. This leverages the existing DomainRouter architecture.
  3. No S-LoRA Dependency: S-LoRA (or similar dynamic multi-adapter serving frameworks) is not introduced into the training or inference pipeline. The system maps and loads adapters using direct, simple filesystem structures.
  4. Heterogeneous Base Constraint: If different spokes target different base models (e.g. 3B vs 7B models), they cannot be loaded concurrently in a single running instance of a GPU runner without reloading the base model or running multiple backend serving processes.

To align the preset schema across different hardware specifications and configurations:

  • Canonical Preset: qwen_4080_16g is the canonical ID representing the RTX 4080 class (16GB VRAM) training profile.
  • Legacy Alias: prosumer_16g is kept as a documented alias mapping directly to qwen_4080_16g in the registry (preset_schema.rs). This ensures consistency between domain-profiles.yaml and client autodetect logic while maintaining backward compatibility with cloud sizing estimations.