json-pro

Overview

JSON is the default data interchange format, but production JSON work involves more than parse and stringify: schema validation, streaming huge payloads, querying/transforming nested structures, and handling the format's sharp edges (precision, duplicates, comments). This skill covers the full toolkit.

When to use

• Validating JSON payloads against schemas (JSON Schema)
• Processing JSON files too large for memory (streaming parsers, JSONL)
• Transforming nested JSON (flattening, reshaping, querying with JSONPath/jq)
• Handling precision loss, duplicate keys, and non-standard extensions
• Designing JSON APIs and config formats

Core concepts

Validate at the boundary. Every JSON input from outside your code is untrusted: validate shape and types with JSON Schema (or a typed decoder) before use. Validation errors should name the exact path ($.user.address.zip) and the violated constraint — this is what makes integrations debuggable.

Streaming for scale. Standard parsers build the whole document in memory. For large files, use streaming parsers (event-based) or newline-delimited JSON (JSONL — one object per line), which parallelizes and streams naturally. If you control the producer, prefer JSONL for anything over a few MB.

Query with the right tool. jq for shell one-liners and pipelines; JSONPath for in-code queries; your language's native mapping for typed access. Learn jq's core (select, map, group_by, to_entries) — it repays the hour invested a hundred times over in debugging and data munging.

JSON's sharp edges. Numbers are IEEE doubles — 64-bit IDs and money lose precision; use strings for those. Duplicate keys: behavior is parser-dependent (last wins, usually, silently). No comments, no trailing commas in strict JSON — but JSONC/JSON5 exist for configs; know which your parser accepts.

Canonical form for hashing/signing. Key order and whitespace affect bytes; if you hash or sign JSON, canonicalize first (sorted keys, no whitespace) or use a defined canonicalization — otherwise signatures break across implementations.

Practical workflow

1. Define the schema first (JSON Schema draft 2020-12 or your framework's typed models); generate documentation and validators from it.
2. Decode strictly: reject unknown fields in strict contexts (catches typos like emial), coerce deliberately, and preserve precision for IDs/money via strings or decimal types.
3. Transform functionally: prefer pure mapping/filtering pipelines over mutation; for shell work, compose jq filters and keep them in scripts, not one-off history.
4. Stream when large: switch to JSONL or a streaming parser before the file size forces you to; benchmark memory at 10x expected size.
5. Log and error well: on validation failure, report the JSON path, the expected vs actual, and a snippet of the offending document.
6. Version your formats: add a version or $schema field to persisted JSON so migrations are explicit, not archeological.

Common pitfalls

• Float IDs and money — 9007199254740993 parses to 9007199254740992; use strings for identifiers and decimal-safe representations for currency.
• Assuming key order — JSON objects are unordered; parsers may preserve insertion order but consumers must not depend on it.
• Deeply nested access without guards — a.b.c.d throws on missing b; use optional chaining / path queries with defaults.
• Concatenating JSON with string templates — injection and escaping bugs; always build objects and serialize.
• Huge single-line JSON in logs — unreadable and ungreppable; pretty-print in dev, keep compact (or JSONL) in production with sampled pretty output.
• Schema drift without versioning — producers add fields, consumers break; treat JSON formats as contracts with changelogs.