PQL Parallel Topology

Use this skill when implementing or auditing a Parallel Q-Learning style recovery or experiment. It is appropriate when an off-policy learner should exploit many parallel simulation actors and separate data collection from policy and value updates. Do not use it for purely on-policy rollouts where learner updates must consume only the latest trajectories.

Inputs

• num_envs: positive integer count of parallel actors or environments.
• replay_capacity: positive integer replay capacity.
• actor_steps_per_tick: positive integer rollout work per actor tick.
• policy_updates_per_tick and value_updates_per_tick: non-negative update counts.
• sync_interval: positive integer policy synchronization cadence.

Outputs

• A topology JSON object with process roles, dataflow edges, and invariants.
• Validation errors when the configuration cannot exercise the PQL mechanism.

Workflow

1. Create an actor role that writes transition batches to replay.
2. Create a replay role that stores off-policy transitions independently of the latest policy.
3. Create a value learner role that samples replay and updates Q estimates.
4. Create a policy learner role that improves actions using the value learner's Q estimates.
5. Add synchronization edges from policy learner to actors and value learner to policy learner.
6. Validate that the topology contains all required PQL roles before using it as recovery evidence.

Validation

Run:

python scripts/topology.py --num-envs 8 --replay-capacity 64 --actor-steps-per-tick 2 --policy-updates-per-tick 1 --value-updates-per-tick 2 --sync-interval 4
python tests/test_topology.py

Limitations

This skill defines the topology and invariants; it does not run an RL optimizer by itself. Pair it with a recovery harness or training skill for executable results.