Skip to content

vox-plugin-mens-candle-core API Reference (generated)

Crate: vox-plugin-mens-candle-core · Version: 0.6.0 · Target: Library (lib.rs)

Shared Candle QLoRA training/inference code for the Metal and CUDA mens plugins (device-agnostic; no cdylib entry point of its own).

vox-plugin-mens-candle-core — device-agnostic Candle QLoRA code shared by vox-plugin-mens-candle-metal and vox-plugin-mens-candle-cuda.

The two device plugins started as independent ports of vox-populi’s Candle QLoRA trainer/server and grew 44 file pairs, 22 of them byte-for-byte identical (up to 720 lines each). model_card.rs and manifest.rs were also forked a third time into vox-populi itself, but that third copy is NOT folded into this crate (see [model_card]‘s docs: vox-populi sits at a lower dependency layer than this crate and depending on it would be a layer-rule violation that needs a user-authorized exception). Per AGENTS.md §Dependency Discipline rule 3, a helper under ~50 lines may be duplicated with a marker comment — “Never fork 100+ line chunks.” This crate is the fold for the two plugins: device-agnostic training/inference code lives here once; genuinely device-specific code (CUDA memory-pool FFI, Metal’s objc2-metal peak sampler, the gradient-checkpointing feature CUDA has and Metal doesn’t yet) stays in each plugin crate. No cdylib entry point — this is an rlib-only shared library, not a loadable plugin on its own.

  • [config] — LoraTrainingConfig and friends (previously forked, save a launch_argv field CUDA had and Metal hadn’t).
  • [manifest] — on-disk training manifest (previously forked the same way).
  • [model_card] — human-readable MODEL_CARD.md writer (previously forked between the two plugins).
  • [inference_device] / [device] — the device-dispatch seam: DeviceKind, resolve_inference_device, compute_dtype_for_device. resolve_inference_device is what Metal was missing entirely (see that module’s docs for the bug).
  • Everything else (hf_layout, hf_keymap, merge, qlora_preflight, qlora_weights, telemetry*, train_log, train_jsonl_preflight, training_text, training_summary, operator_messages, finetune_contract, adapter_schema_v3, checkpoint_state, external_serving_handoff, candle_qlora_train::*) — byte-identical code moved verbatim.
FeatureDefault / Enables
cudacandle-core/cuda, candle-nn/cuda
default—
metalcandle-core/metal, candle-nn/metal
DependencyKind
vox-dbNormal
vox-evalNormal
vox-hf-layoutNormal
vox-tensorNormal
ModuleDescription
adapter_schema_v3—
candle_qlora_train—
checkpoint_state—
config—
device—
external_serving_handoff—
finetune_contract—
hf_keymap—
hf_layout—
inference_device—
manifest—
merge—
model_card—
operator_messages—
qk_norm—
qlora_preflight—
qlora_weights—
rope—
telemetry—
telemetry_schema—
train_jsonl_preflight—
train_log—
training_summary—
training_text—
weight_resolution—
Dependency Topology, diagram 17 nodes, 6 edges: vox-plugin-mens-candle-… → vox-db; vox-plugin-mens-candle-… → vox-eval; vox-plugin-mens-candle-… → vox-hf-layout; vox-plugin-mens-candle-… → vox-tensor; vox-plugin-mens-candle-… → vox-plugin-mens-candle-…; vox-plugin-mens-candle-… → vox-plugin-mens-candle-…vox-plugin-mens-candle-corevox-dbvox-evalvox-hf-layoutvox-tensorvox-plugin-mens-candle-cudavox-plugin-mens-candle-metal
Diagram source
graph {
direction: right
node vox_plugin_mens_candle_core: "vox-plugin-mens-candle-core"
node vox_db: "vox-db"
vox_plugin_mens_candle_core -> vox_db
node vox_eval: "vox-eval"
vox_plugin_mens_candle_core -> vox_eval
node vox_hf_layout: "vox-hf-layout"
vox_plugin_mens_candle_core -> vox_hf_layout
node vox_tensor: "vox-tensor"
vox_plugin_mens_candle_core -> vox_tensor
node vox_plugin_mens_candle_cuda: "vox-plugin-mens-candle-cuda"
vox_plugin_mens_candle_cuda -> vox_plugin_mens_candle_core
node vox_plugin_mens_candle_metal: "vox-plugin-mens-candle-metal"
vox_plugin_mens_candle_metal -> vox_plugin_mens_candle_core
}
Consumer CrateRole
vox-plugin-mens-candle-cudaDirect workspace dependent
vox-plugin-mens-candle-metalDirect workspace dependent
KindSymbolSignatureSourceDescription
structadapter_schema_v3::PopuliAdapterManifestV3pub struct PopuliAdapterManifestV3crates/vox-plugin-mens-candle-core/src/adapter_schema_v3.rs#L12-L28—
structadapter_schema_v3::AdapterMethodFieldspub struct AdapterMethodFieldscrates/vox-plugin-mens-candle-core/src/adapter_schema_v3.rs#L31-L33—
structadapter_schema_v3::QuantFieldspub struct QuantFieldscrates/vox-plugin-mens-candle-core/src/adapter_schema_v3.rs#L42-L46—
structadapter_schema_v3::AdapterProvenanceFieldspub struct AdapterProvenanceFieldscrates/vox-plugin-mens-candle-core/src/adapter_schema_v3.rs#L49-L58—
fnadapter_schema_v3::newpub fn new(crates/vox-plugin-mens-candle-core/src/adapter_schema_v3.rs#L65—
structcheckpoint_state::CheckpointStatepub struct CheckpointStatecrates/vox-plugin-mens-candle-core/src/checkpoint_state.rs#L38-L72All state necessary to resume a Candle QLoRA training run mid-epoch.
fncheckpoint_state::fingerprint_filepub fn fingerprint_file(path: &Path) -> Option<String>crates/vox-plugin-mens-candle-core/src/checkpoint_state.rs#L82-L88Content fingerprint for a training data file (train.jsonl / train_mixed.jsonl), used to detect resuming a checkpoint against a different dataset than the one it was trained on so far. Not a security hash — std::hash::DefaultHasher over the raw bytes is enough to catch “this is a different file” without a new dependency (same reasoning as the DefaultHasher use in validation::split_validation_by_response). Returns None if the file cannot be read (never blocks training on a fingerprinting failure — the resume check just skips itself when either side is None).
fncheckpoint_state::path_inpub fn path_in(output_dir: &Path) -> PathBufcrates/vox-plugin-mens-candle-core/src/checkpoint_state.rs#L92-L94Returns the path where the checkpoint file lives inside output_dir.
fncheckpoint_state::savepub fn save(&self, output_dir: &Path) -> anyhow::Result<()>crates/vox-plugin-mens-candle-core/src/checkpoint_state.rs#L100-L109Saves self to output_dir/checkpoint_state.json atomically. Writes to a .tmp sibling then renames, so a power-cut mid-write cannot corrupt the last good checkpoint.
fncheckpoint_state::loadpub fn load(output_dir: &Path) -> Option<Self>crates/vox-plugin-mens-candle-core/src/checkpoint_state.rs#L115-L151Loads a checkpoint from output_dir/checkpoint_state.json. Returns None when the file does not exist (fresh run) or the JSON is corrupt / the schema does not match (start fresh, warn).
fncheckpoint_state::deletepub fn delete(output_dir: &Path)crates/vox-plugin-mens-candle-core/src/checkpoint_state.rs#L154-L159Delete the checkpoint file (called on successful training completion).
fncheckpoint_state::now_utcpub fn now_utc() -> Stringcrates/vox-plugin-mens-candle-core/src/checkpoint_state.rs#L162-L176Current UTC time as an RFC 3339 string.
enumconfig::TrainingDeploymentTargetpub enum TrainingDeploymentTargetcrates/vox-plugin-mens-candle-core/src/config.rs#L14-L20—
fnconfig::as_strpub const fn as_str(self) -> &'static strcrates/vox-plugin-mens-candle-core/src/config.rs#L25-L30Stable wire / manifest label.
enumconfig::MensTokenizerModepub enum MensTokenizerModecrates/vox-plugin-mens-candle-core/src/config.rs#L38-L44—
enumconfig::OptimizerExperimentModepub enum OptimizerExperimentModecrates/vox-plugin-mens-candle-core/src/config.rs#L51-L57—
structconfig::ContextFilterpub struct ContextFiltercrates/vox-plugin-mens-candle-core/src/config.rs#L61-L66ContextFilter used to filter training pairs based on categories, difficulty, and ratings.
structconfig::CurriculumSchedulepub struct CurriculumSchedulecrates/vox-plugin-mens-candle-core/src/config.rs#L70-L76Dynamic curriculum schedule for difficult-gated training.
structconfig::ChatmlConfigpub struct ChatmlConfigcrates/vox-plugin-mens-candle-core/src/config.rs#L80-L86Dynamic ChatML separator configuration (registry-driven).
structconfig::LoraTrainingConfigpub struct LoraTrainingConfigcrates/vox-plugin-mens-candle-core/src/config.rs#L102-L198Full configuration for one LoRA / QLoRA training run.
enumdevice::DeviceKindpub enum DeviceKindcrates/vox-plugin-mens-candle-core/src/device.rs#L13-L24CLI / env device intent for the Candle backend.
structdevice::GpuInfopub struct GpuInfocrates/vox-plugin-mens-candle-core/src/device.rs#L28-L32Best-effort local GPU description.
structexternal_serving_handoff::ExternalServingHandoffV1pub struct ExternalServingHandoffV1crates/vox-plugin-mens-candle-core/src/external_serving_handoff.rs#L10-L24—
fnexternal_serving_handoff::schola_training_runpub fn schola_training_run(run_dir: &Path, base_model: &str, adapter_filename: &str) -> Selfcrates/vox-plugin-mens-candle-core/src/external_serving_handoff.rs#L30-L47—
fnexternal_serving_handoff::write_handoffpub fn write_handoff(out: &Path, handoff: &ExternalServingHandoffV1) -> anyhow::Result<()>crates/vox-plugin-mens-candle-core/src/external_serving_handoff.rs#L50-L54—
enumfinetune_contract::AdapterMethodpub enum AdapterMethodcrates/vox-plugin-mens-candle-core/src/finetune_contract.rs#L10-L13—
enumfinetune_contract::BaseQuantModepub enum BaseQuantModecrates/vox-plugin-mens-candle-core/src/finetune_contract.rs#L17-L20—
fnhf_keymap::middle_block_projection_keypub fn middle_block_projection_key(arch: HfArchitecture, layer: usize) -> Stringcrates/vox-plugin-mens-candle-core/src/hf_keymap.rs#L12-L20HF key for attention output projection per layer (shape [d_model, d_model]).
fnhf_keymap::layer_attention_weight_keyspub fn layer_attention_weight_keys(arch: HfArchitecture, layer: usize) -> Vec<String>crates/vox-plugin-mens-candle-core/src/hf_keymap.rs#L24-L47Self-attention linear weight keys for one block (inventory / diagnostics).
fnhf_keymap::layer_mlp_weight_keyspub fn layer_mlp_weight_keys(arch: HfArchitecture, layer: usize) -> Vec<String>crates/vox-plugin-mens-candle-core/src/hf_keymap.rs#L51-L64MLP / FFN linear weight keys for one block.
fnhf_keymap::layer_rope_weight_keyspub fn layer_rope_weight_keys(arch: HfArchitecture, layer: usize) -> Vec<String>crates/vox-plugin-mens-candle-core/src/hf_keymap.rs#L68-L77RoPE (Rotary Position Embedding) keys for one block.
fnhf_keymap::layer_norm_weight_keyspub fn layer_norm_weight_keys(arch: HfArchitecture, layer: usize) -> Vec<String>crates/vox-plugin-mens-candle-core/src/hf_keymap.rs#L81-L95Norm keys (RMSNorm or LayerNorm) for one block.
fnhf_keymap::ordered_full_block_weight_keyspub fn ordered_full_block_weight_keys(layout: &HfTransformerLayout) -> Vec<String>crates/vox-plugin-mens-candle-core/src/hf_keymap.rs#L187-L196All non-embedding tensor name candidates for a full transformer (attention + MLP per layer).
fnhf_keymap::ordered_full_block_weight_keys_strict_preflightpub fn ordered_full_block_weight_keys_strict_preflight(crates/vox-plugin-mens-candle-core/src/hf_keymap.rs#L204Like [ordered_full_block_weight_keys], but for strict QLoRA preflight shard checks. Qwen3.5 Hugging Face checkpoints usually omit per-layer *.rotary_emb.inv_freq; the Candle trainer synthesizes RoPE from rope_theta in config.json. Requiring those tensors here caused false failures with --qlora-require-full-proxy-stack on official Qwen/Qwen3.5-* weights.
fnhf_keymap::ordered_middle_projection_keyspub fn ordered_middle_projection_keys(layout: &HfTransformerLayout) -> Vec<String>crates/vox-plugin-mens-candle-core/src/hf_keymap.rs#L216-L220Ordered candidate keys from layer 0 .. n_layer-1 (Candle o_proj proxy stack).
fnhf_keymap::filter_keys_in_shardpub fn filter_keys_in_shard(keys: &[String], present: &HashSet<String>) -> Vec<String>crates/vox-plugin-mens-candle-core/src/hf_keymap.rs#L224-L229Keep keys in order, only those present in present.
fnhf_keymap::sample_present_keys_sorted_from_presentpub fn sample_present_keys_sorted_from_present(crates/vox-plugin-mens-candle-core/src/hf_keymap.rs#L233First max_keys sorted tensor names from an existing key union (no extra I/O).
fnhf_keymap::missing_middle_keys_reportpub fn missing_middle_keys_report(crates/vox-plugin-mens-candle-core/src/hf_keymap.rs#L245Expected middle keys missing from shard union (diagnostics). Lists up to max_missing names.
fninference_device::compute_dtype_for_devicepub fn compute_dtype_for_device(device: &Device) -> qlora_rs::ComputeDTypecrates/vox-plugin-mens-candle-core/src/inference_device.rs#L32-L38Compute dtype for QLoRA dequantization, chosen by device rather than by QLoraConfig::default()’s training-tuned BF16. Candle’s CPU backend has no BF16 matmul kernel at all (“unsupported dtype BF16 for op matmul”), so CPU inference must use F32. Metal training also dequants to F32 (no F32→F64 / BF16 kernels on the path we hit), so Metal serving must match its own trainer’s compute dtype. CUDA keeps BF16.
fninference_device::resolve_inference_devicepub fn resolve_inference_device(device_kind: &DeviceKind) -> Result<Device>crates/vox-plugin-mens-candle-core/src/inference_device.rs#L50-L101Resolve the Candle device for inference. - Cpu always succeeds. - Cuda / Metal (explicit): if the matching cargo feature isn’t compiled in, or device init fails, this is a hard error — no silent CPU fallback for an explicit accelerator request. - Best: tries the accelerator feature(s) this build has, and falls back to CPU with a tracing::warn! if none are available or init fails. This is the behavior CUDA already had; Metal previously had neither the warning nor the error path (see module docs).
structmanifest::ArchParamspub struct ArchParamscrates/vox-plugin-mens-candle-core/src/manifest.rs#L26-L31Architecture loaded from disk or defaulted for scratch training.
structmanifest::TrainingManifestpub struct TrainingManifestcrates/vox-plugin-mens-candle-core/src/manifest.rs#L46-L132On-disk training manifest (written by training loop).
structmanifest::InitialManifestRunpub struct InitialManifestRuncrates/vox-plugin-mens-candle-core/src/manifest.rs#L152-L181Snapshot of run settings for [initial_training_manifest].
fnmanifest::from_lora_configpub fn from_lora_config(c: &crate::config::LoraTrainingConfig) -> Selfcrates/vox-plugin-mens-candle-core/src/manifest.rs#L185-L222—
enummanifest::InitialTrainingKernelpub enum InitialTrainingKernelcrates/vox-plugin-mens-candle-core/src/manifest.rs#L227-L236Which native kernel is writing the manifest (kernel-specific notes / Candle fields).