formal-verification-probabilistic-snn-quotient
Formal verification toolchain for probabilistic spiking neural networks using weight-discretized quotient abstractions. CogSpike framework integrates SNN design, simulation, and PRISM-based verificati
SKILL.mdcollection/skills/SKILL.md
GitHubNeuroWorld: A Latent Brain World Model for Stimulus-Conditioned Human Brain Dynamics
Overview
NeuroWorld is the first brain world model that casts naturalistic brain functional dynamics prediction as stimulus-conditioned evolution in a learned latent brain-state space. It separates endogenous neural states (measured via fMRI) from exogenous multimodal stimuli across two stages:
- Latent Dynamics Learning (LDL): Jointly learns a transition-sufficient representation and causal dynamics through next-latent prediction, without reconstructing the observed fMRI signal.
- Latent Rollout Decoding (LRD): Freezes LDL, autoregressively rolls latent states forward from an observed fMRI prefix, and decodes them into subject-specific whole-brain responses.
Core Methodology
Key Innovation
- Causal constraint: Unlike traditional brain encoding models that use stimulus-to-response regression allowing future stimuli to leak into current predictions, NeuroWorld enforces strictly causal stimulus access.
- Latent separation: Endogenous brain states are separated from exogenous stimuli in the latent space.
- Two-stage architecture: LDL learns dynamics without reconstruction; LRD handles decoding separately.
Architecture Components
Latent Dynamics Learning (LDL)
- Input: Multimodal stimuli sequence + fMRI observations
- Objective: Predict next latent state given current latent state and current stimulus
- Loss: Next-latent prediction loss (no fMRI reconstruction loss)
- Output: Transition-sufficient latent representation
Latent Rollout Decoding (LRD)
- Input: Observed fMRI prefix → initial latent state
- Process: Autoregressive rollout using learned LDL dynamics
- Output: Subject-specific whole-brain fMRI responses
Training Workflow
- Joint LDL training: Train LDL on paired stimulus-fMRI data using next-latent prediction objective
- Freeze LDL: Keep LDL parameters fixed after training
- Train LRD decoder: Learn mapping from latent states to fMRI responses
- Evaluation: Multi-step rollout under strictly causal stimulus access
Implementation Guidelines
Data Requirements
- Naturalistic stimuli: Movies, audio, or other continuous sensory inputs
- fMRI responses: Time-aligned with stimuli (TR-matched)
- Dataset size: Large-scale datasets preferred (paper uses 140.7 person-hours)
Evaluation Metrics
- Multi-step rollout accuracy: Prediction accuracy over extended horizons
- Autoregressive drift robustness: Stability during long-horizon rollouts
- Subject-specific performance: Individualized brain response prediction
Applications
- Brain activity forecasting: Predict neural responses to novel stimuli
- Extended brain-state simulation: Generate realistic neural trajectories
- Interpretability analysis: Characterize functional organization of learned dynamics
- Clinical applications: Simulate brain responses in neurological conditions
Benchmarks and Results
Datasets Used
- SG-MIND: Singapore Multimodal Imaging & Naturalistic Dataset (20 participants, 8,519 paired stimulus-response clips, 140.7 person-hours)
- Existing benchmarks: Three naturalistic movie-fMRI benchmarks spanning 30 participants total
Performance Highlights
- State-of-the-art multi-step rollout: Superior performance under strictly causal stimulus access
- Robustness to autoregressive drift: Greater stability during long-horizon predictions
- Reliable trajectory simulation: Supports extended brain-state trajectory generation
Pitfalls and Considerations
Technical Challenges
- Computational complexity: Requires significant computational resources for training
- Data alignment: Precise temporal alignment between stimuli and fMRI is critical
- Subject variability: Individual differences require careful handling in decoder design
Limitations
- Stimulus dependency: Performance depends on similarity between training and test stimuli
- Temporal resolution: Limited by fMRI temporal resolution (typically 0.5-2 Hz)
- Spatial coverage: Dependent on fMRI acquisition protocol and coverage
Activation Keywords
- brain world model
- latent brain dynamics
- stimulus-conditioned forecasting
- fMRI prediction
- neural state forecasting
- NeuroWorld
- latent dynamics learning
- brain functional dynamics
References
- Original paper: https://arxiv.org/abs/2608.01773
- SG-MIND dataset: Newly collected Singapore Multimodal Imaging & Naturalistic Dataset
- Related work: Brain encoding models, world models, latent dynamics models