audiogear — speech-dataset annotation pipeline

audiogear (https://github.com/lIkesimba9/audiogear) computes per-clip features over a speech dataset: reader → metric blocks → CSV writer, configured with Hydra, sharded across GPUs/nodes. This skill drives it from any project.

Copy this file into the consuming project as .claude/skills/audiogear/SKILL.md. Set AUDIOGEAR_HOME below to wherever the audiogear checkout lives.

Setup (once per machine)

export AUDIOGEAR_HOME=~/audiogear            # adjust; add to the project env
git clone https://github.com/lIkesimba9/audiogear.git "$AUDIOGEAR_HOME" || true
cd "$AUDIOGEAR_HOME"
uv sync --extra ru-pipeline                  # or à-la-carte: --extra mos --extra asr --extra pitch
sudo apt-get install -y espeak-ng            # needed only for SpeakingRateMetric
cp -n .env.example .env                      # put HF_TOKEN= in .env for gated models (pyannote)

Python 3.10–3.12. GigaAM pins torch <2.6 — do not "upgrade" torch inside this venv.

Input contract

One directory per dataset with a |-delimited metadata.csv:

<data_root>/<dataset>/
  metadata.csv        # one row per clip
  audio/...           # referenced by audio_path, relative to the dataset dir

Required columns: id (unique), audio_path, text (may be empty). All other columns pass through untouched; computed metric columns are appended. If the source data has a different shape, first write a converter that produces this layout (see build_betterset-style scripts) — do NOT teach audiogear the foreign schema.

Core workflow

1. Write a dataset config

Create $AUDIOGEAR_HOME/configs/feat_<ds>.yaml. Proven template (mirrors the generated feat_* configs) with CPU∥GPU lanes so DSP metrics hide under GPU time:

defaults:
  - reader: ru          # CsvReader preset: delimiter "|", audio_key=audio_path, id_key=id
  - writer: csv
  - executor: local
  - _self_
hydra:
  job: {chdir: false}
  run: {dir: .}
  output_subdir: null
device: cuda
reader:
  data_folder: /path/to/data_root/<ds>
  glob_pattern: metadata.csv
  limit: -1             # set to 10 for a dry run
writer:
  output_folder: outputs/<ds>
  output_filename: ext_$rank.csv
  sep: "|"
executor:
  tasks: 8              # shards; rows are split round-robin
  workers: 2            # concurrent processes; each pins ONE GPU (rank % gpus)
  skip_completed: true
  logging_dir: logs/<ds>
metrics:
  - _target_: audiogear.pipeline.parallel.ParallelLanes
    lanes:
      cpu:
        - _target_: audiogear.pipeline.metrics.bandwidth.BandwidthMetric
        - _target_: audiogear.pipeline.metrics.wada_snr.SnrMetric
        - _target_: audiogear.pipeline.metrics.speaking_rate.SpeakingRateMetric
          language: ru
      gpu:
        - _target_: audiogear.pipeline.metrics.distillmos.DistillMosMetric
          device: ${device}
        - _target_: audiogear.pipeline.metrics.squim.SquimMetrics
          device: ${device}
        - _target_: audiogear.pipeline.metrics.pitch.CrepePitchMetric
          device: ${device}
          crepe_model: tiny
        - _target_: audiogear.pipeline.metrics.style.StyleMetric   # after pitch (reuses pitch_*)
        - _target_: audiogear.pipeline.metrics.gender.GenderMetric
          device: ${device}

Add WhisperWer to the gpu lane only when text is human-authored.

Lane rules (violating them corrupts data silently):

lanes must write disjoint columns;
a metric that consumes another's output (StyleMetric ← pitch_mean, WhisperWer/SpeakingRateMetric ← text) goes in the SAME lane, after its producer; a producer needed by both lanes (e.g. ConsensusTranscriber) runs as an ordinary step BEFORE the lanes block.

2. Dry run, then full run

cd "$AUDIOGEAR_HOME"
uv run python process.py --config-name feat_<ds> reader.limit=10   # smoke test, downloads models
uv run python process.py --config-name feat_<ds>                   # full run

Multi-GPU: executor.workers=<n_gpus>. executor.gpus is an int (count), never a list. Never set CUDA_VISIBLE_DEVICES yourself — the executor pins it. Multi-node: launch the same command once per node under SLURM, or set AUDIOGEAR_NODE_RANK/AUDIOGEAR_NUM_NODES; put executor.logging_dir on shared storage.

3. Merge shards → extended_metadata.csv

Each shard writes outputs/<ds>/ext_<rank>.csv. Merge with id-dedup (duplicates appear if executor.tasks changed between an interrupted run and its rerun):

export AUDIOGEAR_DATA_DIR=/path/to/data_root
uv run python examples/run_batch.py <ds>     # run + merge + .FEAT_DONE marker per dataset

For many datasets, examples/gen_configs.py generates the per-dataset configs and examples/run_batch.py sweeps them smallest-first with resume.

4. Filter + QA

uv run python examples/filter_clean.py <ds>   # extended_metadata.csv -> clean_metadata.csv + drop reasons
uv run python examples/qa_report.py <ds>      # -> <data_root>/<ds>/qa_report.html

qa_report.html is self-contained: metric histograms with the filter thresholds drawn in, and playable audio just below/just above each threshold. To calibrate thresholds: listen to both sides — rejected clips sounding fine ⇒ threshold too strict; accepted clips sounding bad ⇒ too loose. Edit TH in examples/filter_clean.py (single source of truth for filter and report) and regenerate. Set HUMAN_TEXT/ASR_TEXT there: the CER filter is only valid where text was written by humans, not by an ASR model.

Metric catalogue

Class (`audiogear.pipeline.metrics.*`)	Columns	Notes
`distillmos.DistillMosMetric`	`distillmos`	no-ref MOS, GPU
`squim.SquimMetrics`	`pyt_stoi`, `pyt_pesq`, `pyt_si_sdr`	GPU, never batched (padding skews scores)
`wada_snr.SnrMetric`	`wada_snr`	blind SNR, CPU
`bandwidth.BandwidthMetric`	`bandwidth_hz`, `is_upsampled_est`	detects upsampled audio; container sample_rate lies
`pitch.CrepePitchMetric` / `pitch.PitchMetric`	`pitch_mean`, `pitch_std`	GPU torchcrepe / CPU pyin
`style.StyleMetric`	`energy_db`, `energy_dynamics`, `expressiveness`	CPU; reuses `pitch_*` if present
`speaking_rate.SpeakingRateMetric`	`speaking_rate`, `phonemes_per_word`, `char_rate`	espeak-ng; NOT thread-safe — leave it serial
`wer.WhisperWer`	`whisper_wer`, `whisper_cer`	vs `text`; -1 when no reference. Skip for ASR-derived text (CER≈0, meaningless, expensive)
`gigaam_v3.GigaAMv3`	`gigaam3_text`, `gigaam3_cer`, `text_punctuated`, opt-in words JSON	punctuated-from-audio RU transcript + punctuation transferred onto reference words; `model_name: multilingual_ctc` for ru/en/kk/ky/uz (no punctuation — set `punct_column: null`); `words_column` adds word timestamps
`gender.GenderMetric` / `emotion.EmotionMetric`	`gender_pred` / `emotion_pred`, `emotion_score`	`_pred` suffix — never overwrite curated columns
`hf.HFAudioModelMetric`	configurable	any 🤗 audio classification/regression model, config-only
`brouhaha_snr_reverb.SnrReverbMetrics`	`snr`, `c50`	pyannote, gated (HF_TOKEN)
`speaker.SpeakerLabeler`	`speaker`, `speaker_conf`, `speaker_margin`	dataset-level clustering — run with `executor.tasks=1`
`transcribers.consensus.ConsensusTranscriber`	`text`, `asr_text_*`, `asr_agreement`	multi-ASR medoid; for datasets without transcripts

Reliability semantics (what to expect downstream)

Sentinels, not crashes. A corrupt/unscorable clip never kills a run: the per-clip guard writes NaN (or -1 for WER/CER/SQUIM) and continues. Always treat NaN and negative metric values as "no value" when filtering. A long unbroken failure streak (default 50) aborts the shard — that means a systematic problem (model/config), not bad data.
Resume is two-level. Completed shards are skipped via markers in logging_dir; inside an unfinished shard, per-metric JSONL checkpoints in <output_folder>/checkpoints/ resume from the last computed clip. Rerunning the same command after any crash is always safe. Checkpoints are keyed by clip id only — after changing a metric's model/parameters, delete <output_folder>/checkpoints/ (or the run reuses stale values). Disable with resume: false in the config.

Pitfalls checklist

executor.gpus: 1 (int) — gpus: [0] breaks.
Whisper compute_type: int8_float16 when 2 workers share a GPU (halves VRAM).
Clips >25 s are truncated by GigaAMv3; SQUIM needs ≥1024 samples (else -1).
Audio must be locally readable (torchaudio); CSVs/outputs/logs may be s3://… (fsspec), audio may not — FUSE-mount buckets instead.
Models load lazily per worker process — first clips are slow, that's normal.
Adding a metric: subclass BaseMetric, implement compute_metric(segment), set metric= to the column name(s). Do NOT add try/except around it — the base class guards every path already. Load models lazily via audiogear.utils.runtime.cached_model.

Programmatic use (no CLI)

from omegaconf import OmegaConf
from audiogear.build import build_pipeline

cfg = OmegaConf.create({...})       # same shape as a yaml config
data = None
for step in build_pipeline(cfg):    # reader -> metrics -> writer
    data = step(data, 0, 1)         # (data, rank, world_size)
# data: list[AudioSegment]; computed values in segment.metadata

audiogear

SKILL.mdSKILL.md