Database Engine Development

Overview

This skill applies when implementing a database engine, storage layer, query executor, or transaction feature. Its intended outcome is to define the data model and durability guarantees first, then build a small persistent slice with explicit recovery and resource limits.

When to Use

Preserved source section: When to Use

Use for a database implementation or a substantial low-level storage/query subsystem. State whether the goal is an educational engine, embedded store, or compatible production component; the required guarantees differ.

Scope

Does: Follow the task boundary stated under When to Use and Instructions.

Does not: See the preserved source boundaries below and under Stop Conditions.

Source boundary statements from: Safety and Acceptance

Never experiment against production data. Use disposable test directories and preserve backups before migration testing. Accept a feature only when stated durability and transaction properties are demonstrated by repeatable tests, and when corrupted or unsupported formats fail safely rather than being silently reinterpreted.

Inputs

Required: See the preserved source input guidance below.

Optional: Not specified in source skill.

Prerequisites: Not specified in source skill.

Preserved source section: Inputs

• Record and query model, expected workload, data sizes, and consistency requirements.
• Durability, isolation, crash-recovery, concurrency, and compatibility expectations.
• Storage format, supported platforms, test harness, and benchmark baseline.

Instructions

Preserved source section: Procedure

1. Specify invariants. Define key/value or relational semantics, ordering, null behavior, transaction boundaries, and what a successful write promises after a crash.
2. Build a minimal storage path. Implement encode, write, reopen, and read for a tiny dataset. Validate checksums, lengths, version tags, and corruption handling before adding complex indexes.
3. Add query and index layers separately. Compare index results with a simple reference scan. Make plans observable and preserve a correctness path when an optimization is unavailable.
4. Implement transactions deliberately. Choose a documented logging or copy-on-write approach, define atomicity and isolation scope, and test partial writes, process termination, and recovery.
5. Control concurrency and resources. Specify locking, snapshot behavior, page-cache limits, query limits, and behavior under disk-full or memory pressure.
6. Test failure, not just throughput. Use deterministic fixtures, property tests for serialization, restart tests, fault injection where safe, and benchmarks with recorded data and environment.
7. Version the format. Plan upgrades, compatibility checks, and backup/restore before changing durable structures.

Decision Rules

The following source conditional guidance is preserved verbatim; no unstated action is inferred.

Source conditional guidance from: Procedure

3. Add query and index layers separately. Compare index results with a simple reference scan. Make plans observable and preserve a correctness path when an optimization is unavailable.

Source conditional guidance from: Safety and Acceptance

Never experiment against production data. Use disposable test directories and preserve backups before migration testing. Accept a feature only when stated durability and transaction properties are demonstrated by repeatable tests, and when corrupted or unsupported formats fail safely rather than being silently reinterpreted.

Output Format

Not specified in source skill.

Validation Checklist

• Verify the source-defined success criteria above.

Edge Cases and Recovery

Source edge/failure guidance from: Inputs

• Durability, isolation, crash-recovery, concurrency, and compatibility expectations.

Source edge/failure guidance from: Procedure

4. Implement transactions deliberately. Choose a documented logging or copy-on-write approach, define atomicity and isolation scope, and test partial writes, process termination, and recovery.
5. Test failure, not just throughput. Use deterministic fixtures, property tests for serialization, restart tests, fault injection where safe, and benchmarks with recorded data and environment.

Source edge/failure guidance from: Safety and Acceptance

Never experiment against production data. Use disposable test directories and preserve backups before migration testing. Accept a feature only when stated durability and transaction properties are demonstrated by repeatable tests, and when corrupted or unsupported formats fail safely rather than being silently reinterpreted.

Examples

Not specified in source skill. The original provided no input/output example, and none has been invented.

Success Criteria

Preserved source section: Safety and Acceptance

Never experiment against production data. Use disposable test directories and preserve backups before migration testing. Accept a feature only when stated durability and transaction properties are demonstrated by repeatable tests, and when corrupted or unsupported formats fail safely rather than being silently reinterpreted.