babylon-havok
Engine-layer conventions for Babylon.js and Havok in this project — scene and node structure, the render/physics loop split, material and mesh budgets, glTF asset loading, WebGPU/WebGL2 dual-path rule
SKILL.mdskills/babylon-havok/SKILL.md
GitHubBabylon.js + Havok
The engine layer. Everything in here is presentation and integration — the game
rules live in the simulation layer and must not import from Babylon (see game-dev).
Side-effect imports must be in the file that calls the API
Babylon patches families of methods onto Engine and Mesh from separate modules:
thinInstanceSetBuffer from Meshes/thinInstanceMesh, createDynamicTexture
from Engines/Extensions/engine.dynamicTexture. Without the import the method
is undefined — a TypeError at best, a silent no-op at worst — and the type
checker cannot see it, because the method is declared on the interface either
way.
A sibling's import will hold yours up. These are global patches, so a file
that forgets one still works as long as anything else in the bundle imports it.
That is not working, it is coincidence: lazily load the sibling, or delete it,
and the caller breaks with no diff of its own to explain why. headlights.ts
sat in exactly that state, propped up by three other modules.
WebGPU needs its own import, always. Engines/Extensions/* patches the
WebGL engine; Engines/WebGPU/Extensions/* patches the WebGPU one. Both, every
time. Import only the first and it passes locally, passes CI and passes every
headless harness — all of which fall back to WebGL2 for want of a GPU — and
then the sky is black on the phone. This project has been bitten by it twice.
Since nothing without a GPU can catch it at runtime, catch it statically:
tests/engine-imports.test.ts scans src/engine for the calls and asserts the
caller imports the modules itself. Strip comments before matching — a file that
explains the trap is not a file that uses the API, and making that failure go
away by adding imports it doesn't need is worse than the failure.
The layering rule, concretely
src/sim/ pure TypeScript. No Babylon import. Deterministic. Vitest-testable.
src/engine/ Babylon + Havok. Reads sim state, draws it. Owns meshes and materials.
src/platform/ input, audio, storage, PWA. Browser APIs live here and nowhere else.
A lint rule forbids src/sim/** importing @babylonjs/*. If you find yourself
wanting to break it, the thing you're building is in the wrong directory.
Where does Havok sit? Physics is simulation, not presentation — but Havok is a
WASM module with its own world state, so it can't be pure. Resolution: the physics
world is owned by src/sim/physics/, wrapped behind a narrow interface that the rest
of the sim calls. Babylon's PhysicsAggregate convenience bindings tie bodies to
meshes and are therefore engine-layer only; the sim drives Havok bodies directly and
the engine layer reads transforms out to position meshes.
Loop
Babylon's runRenderLoop is the render loop. Do not put game logic in it, and do
not use Babylon's built-in physics stepping tied to render frames — that is
variable-dt physics and produces frame-rate-dependent handling.
engine.runRenderLoop(() => {
accumulator += Math.min(engine.getDeltaTime() / 1000, MAX_FRAME_S);
while (accumulator >= FIXED_DT) {
world.step(FIXED_DT, input.snapshot());
accumulator -= FIXED_DT;
}
view.sync(world, accumulator / FIXED_DT); // interpolate meshes from sim state
scene.render();
});
Step Havok manually inside world.step, at FIXED_DT, never from the render loop.
Initialisation order
Havok is WASM and loads asynchronously. Await it before creating any physics body, and load it exactly once for the process. A test suite that initialises Havok per test will be slow and will eventually flake.
Renderer selection: try WebGPUEngine and fall back to Engine (WebGL2) on failure
or on missing navigator.gpu. Both paths must be exercised — a change verified only
under WebGPU is not verified, because a large share of iOS devices are still WebGL2.
Meshes and materials
Mobile GPUs are tile-based; the costs that matter are draw calls, overdraw, and material switches — not raw triangle count.
- Instance everything repeated. Buildings, streetlights, road segments, parked
cars: thin instances (
thinInstanceAdd) for static repeats,createInstancewhen each needs its own node. A city built from individually-created meshes will not run. - Share materials aggressively. One material per texture atlas, not per object. Material count is a first-class budget.
- Freeze what doesn't change — but read
freezeWorldMatrixliterally. It freezes the world matrix, not the local one, so it is right for scenery and wrong for anything parented to something that moves. Freezing a car's merged interior left it sitting at the world origin while the body drove off. If it has a moving parent, do not freeze it.material.freeze()andscene.freezeActiveMeshes()carry no such trap. - Merge static geometry per chunk at generation time rather than drawing hundreds of small meshes.
- Keep shadows off or extremely limited on mobile. Real-time shadow maps are usually the single most expensive thing in a mobile scene; prefer baked or blob shadows.
What merging meshes actually saves
Mesh.MergeMeshes is the main draw-call lever, and it is easy to over-estimate:
- Merging parts that share a material removes draw calls. This is the real win. Collapsing one car's static parts by material took it from ~44 draws to ~20, with no visual change.
- Merging parts with different materials does not. The
multiMultiMaterialform produces one mesh carrying aMultiMaterial, but a sub-mesh per material still issues its own draw call. It saves state changes, not draws. Budget on that basis, or a plan built on "merge everything into one mesh" will miss by several times. - It bakes world positions. Merge while the parent is still at the origin with identity rotation, or every merged object is built around wherever the parent happened to be at the time.
- Attribute sets must match exactly, or it throws
Cannot merge vertex data that do not have the same set of attributes. Box and cylinder primitives carry UVs; hand-builtVertexDatawith only positions and normals does not. Group by attribute signature as well as material rather than padding with zero UVs — invented UVs break silently the first time that material gains a texture, and the extra draw call is one, not many. - Merging destroys names. Anything found by a name lookup must be held by
reference instead. Doing that is an improvement anyway:
getChildMesheswith a predicate allocates an array on every call, and these lookups tend to live in per-frame code.
Exclude from a merge: anything that moves independently, anything toggled on its own, anything alpha-blended into a later rendering group, and anything with children.
Disposal and memory
iOS is the tightest memory environment we target, and an endless streamed city allocates continuously. Leaks here are fatal, not untidy.
- Every
dispose()must also dispose materials and textures if not shared — Babylon does not cascade by default.mesh.dispose(false, true)disposes materials/textures. - "Materials are shared" is a claim to check per material, not a blanket rule.
Most of a vehicle's are shared across its whole profile; its registration
plates, door decals and lightbar are built per id and are not. Disposing the
mesh tree with
disposeMaterialAndTextures = falseon the strength of a comment leaked all of those on every cop despawn. Track per-entity resources in an explicitownedlist built where they are created — recovering them later by walking the mesh tree and matching names is the approach that breaks the first time something is renamed. - A material or texture leak is invisible in
scene.meshes.length, which is the figure a harness naturally samples. Samplescene.materials.lengthandscene.textures.lengthas well. And check that the scenario actually exercises the disposal path:city-inspect --streamteleports across the whole city without ever despawning a vehicle, sopruneRemovedis never called and the route reports a clean bill however bad the leak is. That is why--despawnexists. Reverting the fix and re-running is the only way to know a leak check has teeth. - Dispose Havok bodies when their chunk unloads. An orphaned body keeps simulating.
- Pool and reuse chunk meshes rather than create/dispose churn.
scene.blockMaterialDirtyMechanism = truearound bulk changes.- Never hold a mesh reference in sim state. Sim holds ids; the engine layer maps id → mesh.
Assets
glTF/GLB only, loaded via Babylon's SceneLoader. Compress with Draco for geometry
and KTX2/Basis for textures — KTX2 matters a lot on mobile because it stays compressed
in GPU memory, where a PNG does not.
Load asynchronously with a visible loading state, and never block the first frame on a non-essential asset.
Inspector
scene.debugLayer.show() behind a dev-only flag. It's the fastest route to
diagnosing draw-call counts, material duplication, and orphaned nodes. Do not ship it
in the production bundle — import it dynamically so it tree-shakes out.
An unlit StandardMaterial carries its colour in emissiveColor
disableLighting = true does exactly one thing: it stops Babylon emitting the
LIGHT0..n defines, so the shader's per-light loop never runs and diffuseBase
stays at its initial vec3(0.). The final colour in default.fragment is
clamp(diffuseBase * diffuseColor + emissiveColor + ambient) * baseColor
so with diffuseBase pinned at zero the entire diffuseColor term is
multiplied out, whatever it is set to. A white diffuseColor with a black
emissiveColor and lighting disabled renders pure black.
baseColor is where a thin instance's per-instance colour arrives —
baseColor.rgb *= vColor.rgb, gated on VERTEXCOLOR || (INSTANCESCOLOR && INSTANCES). So the recipe for "the instance colour is the output colour" is
black diffuse, white emissive, lighting disabled.
This cost a long debugging session on #26: the city's signage and street lighting
material had it the other way round, and every neon sign and lamp head in the
game drew as an invisible black quad. Everything upstream was correct and every
diagnostic said so — 58 to 140 thin instances a chunk, right material, in
frustum, in scene.getActiveMeshes(), bounds spanning the chunk. Nothing was
wrong anywhere except the pixels.
Two things worth taking from that beyond the recipe:
- When geometry is provably submitted and still not visible, the next suspect is
the material's colour arithmetic, not the culling, the bounds or the
transform. Read the actual shader source in
node_modulesrather than reasoning about what the property names imply —emissiveColoris added again underEMISSIVEASILLUMINATIONand not otherwise, which is not guessable. thinInstanceSetBuffer('color', ...)silently rewrites the kind toinstanceColor(VertexBuffer.ColorInstanceKind) for native's benefit. That is whyINSTANCESCOLORis keyed onisVerticesDataPresent('instanceColor')and why looking for acolorbuffer to explain a missing tint finds nothing.
Output is not gamma-corrected unless you ask for it
No image processing is enabled at the tier this game ships to, so a value the shader computes at 0.10 displays at 0.10. That looks black. The same figure through an sRGB curve would be about 0.35 — a perfectly visible mid-grey.
Tune emissive and ambient constants against linear output, and if a surface looks black when the arithmetic says it should be visible, suspect this before retuning. Two rounds of the #26 window work were spent brightening a wall that the maths said was already bright enough.
Engine extensions need a side-effect import per backend
Babylon patches many engine methods on from separate modules, and with deep imports those are not pulled in automatically. Worse, there is one module per backend, and they target different classes:
import '@babylonjs/core/Engines/Extensions/engine.dynamicTexture'; // ThinEngine
import '@babylonjs/core/Engines/WebGPU/Extensions/engine.dynamicTexture'; // WebGPU
WebGPUEngine extends ThinWebGPUEngine, not ThinEngine, so the WebGL
extension never reaches it and the method is simply undefined. Importing the
class itself — DynamicTexture — is not enough; it is the engine method that
is missing, and the failure surfaces as createX is not a function deep inside
Babylon.
This is a blind spot in every environment available here. Software rendering has no WebGPU, so the fallback to WebGL2 always applies and the WebGL extension always suffices. A missing WebGPU extension therefore looks completely fine locally, in CI, and in any headless browser — and fails only on real hardware with a working WebGPU implementation, which today means an iPhone on iOS 26.
So: when adding anything that needs an engine extension, import both variants without waiting to see a failure, because you will not see one. And treat "it renders here" as evidence about the WebGL path only. See #6.
Hand-built quads: the winding is left-handed and the atlas UVs are not obvious
Building geometry directly with VertexData — the right call for a handful of
per-chunk quads that each need their own UV rectangle, where instancing would
mean a custom attribute and a plugin to read it — hands you two conventions that
are easy to get backwards and that fail in ways which do not look like what they
are.
Winding. Babylon is left-handed by default, so a front face is wound
clockwise as seen from the front. Wind it the other way and, with
backFaceCulling on, the front face is culled and the back face renders. The
symptom is not a missing object: it is a correctly placed, correctly lit,
correctly coloured surface with its content mirrored. That reads as a UV
problem and sends you to the wrong file.
Atlas V. For a DynamicTexture atlas sampled by hand-written UVs, the cell
row comes out right with the obvious mapping while the orientation inside the
cell does not — the panel shows the right artwork, 180° round. Because the
content is correct, nothing looks like a UV bug until the artwork has
lettering in it and you get close enough to read it. Determine this by looking
rather than deriving; the interaction between invertY, the V direction and the
front-face winding is not worth reasoning about from first principles when one
screenshot settles it.
And the meta-lesson, which is the expensive one: a wide shot cannot judge
legibility. Four wide angles of the city all looked fine while every billboard
in the world was mirrored, and the bug was obvious within one second of framing
a single board square-on. When a feature's whole point is that it is read, the
review harness needs a shot that frames one — found in the scene rather than
hard-coded, since which objects exist depends on the seed. See the billboard
shot in tools/city-look.mjs.
World-space UVs on shared-material instances need a material plugin
Thin instances of one unit box, scaled non-uniformly, cannot get a
constant-size-in-metres texture through the box's own UVs: every instance has the
same 0..1 range, so a 6 m shop and a 40 m tower get windows seven times different
in size. Deriving the coordinate from vPositionW and the world normal fixes it,
and a constant floor height falls out for free.
MaterialPluginBase is the way to do that without losing the single
StandardMaterial, the single draw call per chunk, the per-instance vColor or
the fog. The stock paths do not work and it is worth knowing why:
Texture.coordinatesMode is consumed only by the reflection/refraction path and
every mode is camera- or reflection-relative; uScale/vScale are per-material
constants, which is the stretching restated; TriPlanarMaterial has no emissive
channel, is not a dependency here, and never sets THIN_INSTANCES so it drops
the world matrix.
Three details that are easy to get wrong:
- The injection point decides what the effect interacts with.
CUSTOM_FRAGMENT_BEFORE_FOGruns before fog, so an emissive addition fades into haze with the surface — correct for ordinary buildings.CUSTOM_FRAGMENT_BEFORE_FRAGCOLORruns after fog and tone mapping, so the same code punches through haze at any distance — correct for landmarks, wrong for everything else.CUSTOM_FRAGMENT_UPDATE_ALPHAsits inside#ifdef DIFFUSEand compiles out entirely, with no warning, on a material with no diffuse texture. - Write both shader languages. The varying is
vPositionWin GLSL andfragmentInputs.vPositionWin WGSL, andgetCustomCodetakes the language as its second argument for exactly this reason.isCompatible()returns true for GLSL only by default and the plugin manager throws on a WGSL material, so override it deliberately. WGSL cannot swizzle-assign, so rebuild the colour rather than adding intocolor.rgb. - Override
getClassName(). The base returns'MaterialPluginBase', and the class name keys the generatedMATERIALPLUGINdefine — two plugins reporting the base name collide.
CI has no GPU and always falls back to WebGL2, so the WGSL branch cannot be verified here at all. Same blind spot as the engine extensions above; see #6.
Side-effect stubs fail silently — turn the warnings on
Babylon 9 replaces tree-shaken augmentations with stub functions that do
nothing, throw nothing, and log nothing by default. thinInstanceSetBuffer
without import '@babylonjs/core/Meshes/thinInstanceMesh' accepts the matrices,
returns normally, and draws zero instances. A whole city was generated, uploaded
and silently discarded.
The stubs are deliberately quiet — engine internals call augmented methods as feature checks, so warning by default would spam every frame — but that means your calls are quiet too. Opt in during development:
import { SetMissingSideEffectWarningsEnabled } from '@babylonjs/core/Misc/devTools';
if (import.meta.env.DEV) SetMissingSideEffectWarningsEnabled(true);
_IsSideEffectImplemented(fn) feature-detects a specific method, since a stub is
truthy and typeof says "function".
This is the same failure as the WebGPU dynamicTexture bug in a new costume, and
the general rule now has two halves: a Babylon feature that is not part of the
core class needs a side-effect import, and its absence is invisible. When
something you built does not appear, check the import before checking your maths.
Far clip and fog distance are one decision
Exp2 fog at density 0.0075 leaves 0.6% of an object's colour at 300 m and 0.01% at 400 m. Babylon's default far plane is 10000, so the renderer was shading well over a kilometre of city that the fog then erased completely — 59 draw calls down to 50, and a fifth of the frame time back, for one line.
Set camera.maxZ from the fog falloff rather than by feel, and treat the two as
a single tunable: changing one without the other either wastes fill or clips
visible geometry. Anything that turns fog off — a debug flyover, an inspection
harness — must raise maxZ too, or it trades a fogged city for a clipped one.
Reading Havok's collision events consumes them
HP_World_GetCollisionEvents plus HP_World_GetNextCollisionEvent walks a list
that the walk itself destroys. A second walk in the same tick returns a
different, usually empty, result — measured here as 64 differing reads out of 180
steps of a box resting on the ground.
So drain once, inside step, into an array, and let everything else read that
snapshot. Exposing the walk directly means the first consumer silently starves
every other one, and the failure is indistinguishable from "no collisions
happened" — which is exactly how it presented: cars visibly shoving each other
across the map while the event list stayed empty.
Two more things about collision events specifically:
- Bodies raise nothing until opted in with
HP_Body_SetEventMask, and the omission is silent. Collisions still resolve correctly; only the reporting disappears. Opt in the handful of bodies that matter, never the whole city. - The EventType members are Emscripten objects carrying
.value, and the declared numeric enum in the .d.ts is wrong. The real bits are 1, 2 and 4 — flags, not sequence — so a literal copied from the typings both fails to OR and names the wrong events. Read.valueoff the module and verify it, the same rule asMotionType.
When the typings and the runtime disagree, believe the runtime
Havok's .d.ts has been wrong three times now in this project: MotionType
members declared as numbers but objects at runtime; EventType declared 0-based
when the real values are 1/2/4 bit flags; and
HP_World_GetNextCollisionEvent(world: number, …) declared as a plain number
while every sibling call takes HP_WorldId.
None of these fail loudly. A wrong MotionType leaves a body static, a wrong
event mask opts into nothing, and both look like a physics bug rather than a
typings bug. When something physical is not happening, probe the actual runtime
value before rereading your own maths — it takes one throwaway test and settles
it outright.
Handedness
Babylon is left-handed. A positive rotation about X carries +Z toward -Y, not toward +Y. Hinged parts rotated by the "obvious" sign swing the wrong way — the bonnet and boot both opened downward into the body and disappeared, leaving what looked like an open bay with no lid. Nothing errors; the part is simply somewhere you did not expect.
The same catches flat geometry: CreatePlane faces -Z, so a decal on the +X
flank needs -90 degrees about Y. Get it backwards and the plane is backface-
culled and the artwork silently vanishes.
Both are cheap to verify and impossible to notice from a test. Rotate the part to its extreme and look at it.
One description of the world, not two
Anything that exists in both the simulation and the view — ground extents, world bounds, chunk size, vehicle dimensions, spawn points — is published by the simulation and read by the view. Never as a constant in each.
Two constants describing the same thing will diverge, and the divergence is silent: nothing fails, nothing warns, the game simply becomes wrong in a way that is invisible from the gameplay camera. This has already happened once here — a 400 m visible ground against a 4000 m collision ground, so the car drove off the visible world and kept going. It was found by a wide-angle screenshot, not by any test.
When you catch one of these, fix the structure rather than the number: move the value into simulation state and have the view read it, so the next change cannot recreate the bug.
Never
- Import Babylon from
src/sim/. - Step physics from the render loop.
- Create a mesh or material inside the render loop or a per-tick system.
- Use
scene.registerBeforeRenderfor gameplay logic. - Ship the Inspector or debug materials in a production build.
- Duplicate a world dimension as a constant in both the sim and the view.
Screen-space geometry parented to the camera
A strip parented to the camera — a rear-view mirror, a reticle, a letterbox — has two traps that both present as "nothing draws", with no error and no warning:
- The default near clip is 1.0 and a camera that never sets
minZkeeps it. A plane placed atz = 1is clipped away entirely. It still renders its target every frame; you pay for it and see nothing. Place it well past the near plane and scale its width and height with the distance. CreatePlanealready faces −Z, which is toward a camera looking down +Z. The instinct is to rotate by PI so it "faces the camera". That points it away and it is backface-culled. No rotation is the correct orientation.
Both were hit building the mirror, the second one after this file already recorded the −Z facing rule for other geometry. Reading a rule is not the same as recognising the situation it applies to.
Second render passes are affordable only if budgeted as one
A RenderTargetTexture with renderList = null renders the whole scene again.
Left at defaults that roughly doubles draw calls, which no phone has spare.
Three knobs make it viable, and they should all be set deliberately at
construction rather than discovered later:
- A short
maxZon the target's camera — this is what actually cuts the geometry submitted, not the target's resolution. refreshRateabove 1. A glanced-at strip does not need 60 Hz.- A small target. 512×128 is more than a 90-pixel strip needs at any device ratio worth serving.
Tie all three to the quality tier and return null on the lowest one, rather
than shipping a worse version of the effect. A feature that halves the frame
rate on the device it was added for is not a feature. Note that Babylon's
scene.getEngine().drawCalls does not count render-target passes, so the stats
overlay under-reports once one exists.
Never set scene.activeCameras just to add a second camera: that switches the
whole scene to the multi-camera path. A render target drives its own camera
through target.activeCamera.
renderList = null includes the mesh that displays the target. This is a
real feedback loop and it is the third trap in the same feature: the strip's
material samples the render target, the target renders every mesh in the scene,
and the strip is a mesh in the scene. Whenever the strip enters the target
camera's frustum the driver is asked to sample a texture it is drawing into and
reports GL_INVALID_OPERATION: Feedback loop formed between Framebuffer and active Texture.
It is intermittent, because whether the strip is in that frustum depends on
where the parent camera has swung to — which makes it look like something
else's bug. Exclude the strip and any furniture around it with a layer mask,
not a renderList: default masks are 0x0fffffff, so giving the furniture a mask
of 0x1 and the target's camera 0x0fffffff & ~0x1 excludes exactly those
meshes and nothing else. A renderList would have to name every mesh in a
streaming world.
An intermittent error needs the control run in both directions
The mirror's feedback loop above was attributed to clustered street lighting for several commits, and the reasoning looked sound: the errors appeared with the lamps on, and turning the lamps off made them stop. The control that was never run is the other one — the mirror with the lamps off, which still produced fourteen errors in eight seconds.
Turning the lamps off had removed the errors because it also removed the mirror: the two were wired to the same switch by the very commit that was diagnosing them. The A/B compared "lamps and no mirror" against "lamps and mirror" and read the result as being about lamps.
When feature A and feature B are suspected of colliding, the test is four runs, not two — and if A's gate already disables B, the gate has to come out before any of them mean anything. The cost of getting this wrong was a real feature deleted for months and a comment block in three files confidently explaining a bug that did not exist.