Establishing Telemetry Baselines

When to Use

• You want to hunt for rare or first-seen behavior (uncommon process names, parent/child pairs, destinations) by comparing current activity to a historical baseline.
• You are reducing noise by establishing what "normal" looks like before alerting on outliers.

Do not use a baseline built from a compromised period as "normal" — seed it from a known-good window. This skill computes statistics from telemetry and executes nothing.

Prerequisites

• Historical telemetry (CSV/JSON) with a categorical field to baseline (e.g., process name, parent-child pair, destination host).

Workflow

Step 1: Build the baseline

python scripts/analyst.py baseline history.csv --field Image

Computes per-value counts, frequency (stacked-rank), and the set of values seen, saved as a JSON baseline.

Step 2: Score new activity against the baseline

python scripts/analyst.py compare new.csv --field Image --baseline baseline.json

Flags values not present in the baseline (first-seen) and values below a rarity threshold.

Step 3: Triage outliers

Investigate first-seen and rare values; many will be benign-but-new — corroborate with context.

Step 4: Maintain

Refresh the baseline on a rolling known-good window to avoid drift.

Validation

• The baseline captures counts and the value set from the historical window.
• First-seen values in new data are correctly identified as absent from the baseline.
• Rarity thresholds are explicit and tunable.

Pitfalls

• Baselining a compromised window, normalizing malicious activity.
• Too-short baseline windows making common items look rare.
• High-cardinality fields (full command lines) needing normalization before baselining.

References

• See references/api-reference.md for the baseliner.
• The ThreatHunting Project and ATT&CK hunting resources (linked in frontmatter).