Leaving the sandbox

Part ten of a series. The nine parts before this one are about making traffic correct: what this is, two layers, junctions, drivers, right of way in two parts, a debugging story, measurement, and where the town comes from.

None of that matters if it only runs in its own test harness.

The target is a game engine, and the engine has its own opinions. It has a physics system with real vehicles that have suspension, tyre grip and a differential. It has its own units, its own handedness, its own idea of a frame. It has a scripting layer that artists use. And it already knows where every object in the world is, to the centimetre, which is awkward, because so does the simulation.

That collision of ownership is the actual integration problem. Everything else is plumbing.

Who owns the transform

The question is this.

The simulation computes where a car should be. The engine's physics vehicle computes where the car actually ends up, after suspension, weight transfer, and a wheel that briefly lost grip on a kerb. Both of those are running every frame. Which one is right?

Three answers, and only one of them works.

The simulation wins. Cars get placed exactly where the simulation says. The physics vehicle becomes decoration, or gets deleted. This gives perfect traffic that slides. There is no weight shift in a corner, no dive under braking, no wheel that drops into a pothole, and a player driving among them can feel it immediately.

Both compute and you blend. The most common instinct and the worst option. Two systems integrating motion from the same intent fight each other, and the fight shows up as jitter that nobody can locate, because the bug lives in the seam rather than in either half.

The engine wins, and the simulation drives. The physics vehicle owns the transform. The simulation is demoted from mover to driver: it looks at where the car actually is, decides what a driver would do about it, and hands back steering, throttle and brake.

The third one is the design. A car can be marked as owned by the host, and when it is, the simulation never writes its position. The frame is a loop with three beats: push in the transforms the physics produced, advance the simulation by the frame time, read the controls back out. The controls are three numbers in the range minus one to one, which is exactly the shape a physics vehicle already eats.

The effect on the rest of the system is larger than it sounds. It forces the driver to be a controller rather than an animator, which is why part four splits decisions from control at all. A driver that emits a position has no way to be wrong. A driver that emits a pedal can be wrong, gets corrected next frame, and that correction is measurable.

Cars far away from the player do not need any of this. Those keep the built in kinematic model and move themselves, with the same driver on top. Same brain, two bodies, chosen per car.

Four rules for a boundary

The bridge between the two languages is where wrong assumptions become crashes in somebody else's process, so it has rules, and they are written at the top of the file.

Nothing touches the file system. The host reads and writes bytes. This sounds like an arbitrary restriction until you remember that the host has a package system, a virtual file system, cooked content, and a platform where the path you were given does not exist. A library that opens files is a library that works on your machine.

Anything with a fixed shape is a typed struct. A string carries text, not data. Exactly three calls still speak serialised data, and each one has a reason it can defend: the one that reaches the entire command layer without needing a bridge function per command, the one that dumps a driver's decision for debugging, whose shape changes as the driver changes, and the pair that hands the host a document to store and give back later. Everything else is a struct with named fields.

A call that can find nothing says so with a flag. Not with a sentinel, not with a zero, not with an exception. A call given a bad argument, such as the identifier of a car that does not exist, throws, because that is a programming error and not a result.

A value that may be absent travels in a wrapper with a present flag. There are no magic numbers anywhere in the interface. This one is the dullest rule here and the one that has prevented the most confusion, because the alternative is a minus one that means "none" until the day it means minus one.

Related to that second rule: any group of numbers that belong together travels as one struct. Not because it is tidy, but because no caller can then swap two arguments of the same type. Passing a width where a length belongs is a bug that compiles happily and takes a day to find.

The two objects, and why authoring is one of them

The bridge exposes exactly two things. One drives the simulation. The other builds and edits road networks.

The second one is more interesting than it looks, because it is not a separate implementation. It wraps the same command layer that the desktop studio drives, and that a script drives over a scripting interface. Three completely different front ends, one set of operations, and therefore one undo history and one definition of what "connect these two junctions" means.

The payoff shows up in small ways. Undo and redo in the engine editor work without anybody implementing undo for roads. Every operation reports its problems in the same words everywhere. A bug found in the studio is fixed in the engine at the same time, because there is only one of it.

The cost is that the command layer has to be complete enough to be the only path, with no side doors for the one operation that was easier to do directly. That discipline is worth keeping, and it is easy to lose.

Units, handedness, and the one place they change

The simulation thinks in metres, right handed, angles in radians, Z up. The engine thinks in centimetres, left handed, angles in degrees. Those are different enough to produce a mirrored world, and mirrored is a bug that renders perfectly and drives on the wrong side.

The conversion is one small file at the boundary. A scale, a sign flip on one axis, a sign flip on the angle. That is all it is.

The important part is not the arithmetic, it is the rule around it: convert exactly once, at the edge, and never halfway in. The moment two places convert, some value passes through both and some value passes through neither, and you get a world where most things are right and the signs are on the wrong side of the road. Every function that crosses the boundary converts. Nothing inside either side knows the other exists.

Traffic lights, and the trick that makes them cheap

The simulation knows the state of every signal, because it computed it. The engine has actors in the level with light meshes on them. Joining those two is a small problem with an expensive naive solution.

The naive version pushes every light state into every light actor every frame. In a city that is thousands of scripted calls per frame to tell a light it is still red.

What it does instead: each light actor carries the identifier of the lane it belongs to, baked in when the level was built. A director reads all signal states into a buffer it keeps between frames, compares each against what it last sent, and calls an actor only when its colour actually changed. A city of lights becomes a handful of calls a second, because a light changes about once every twenty seconds and there is nothing to say in between.

The part I would not have thought to add, and which has saved real debugging time: the director reports how many signal readings matched no actor in the level. That number is zero when the level was built from the same network the simulation is running. It is large and immediate when somebody rebuilt the road network and forgot to rebake the level. Without it, the symptom is "some lights do not work" with no way to tell whether the bug is in the bake, the simulation, or the Blueprint.

Publish the count of things that did not match. It is a cheap number to produce and it converts a vague symptom into a specific one.

Drawing the network, and an expensive lesson about the render thread

The editor has to draw the road network so a person can see what they are authoring. The first implementation drew every line individually every frame, which is the obvious thing and is what the engine's debug drawing exists for.

On a generated district of about a hundred and ninety thousand lines, that cost eighty eight milliseconds per frame on the render thread. Eleven frames a second. The editor was unusable on exactly the maps that were most worth looking at.

The problem was not the number of lines, it was that the work was repeated. A thick line in screen space has to be rebuilt into geometry every frame, and nothing about a static road network changes between frames. Building that geometry once into a cached buffer and drawing it as a few batches took the same scene on the same machine to a hundred and twelve frames a second, with seven milliseconds on the render thread.

Then the second pass, which is about how much geometry there needs to be in the first place. Sampling every lane at a fixed step means a dead straight lane costs as much as a tight curve, which is absurd, since the straight one needs two points. Sampling that follows the error instead, keeping points only where dropping them would visibly move the drawn line, cut the district from about two hundred and thirty five thousand segments to about thirty thousand. Nearly eight times less.

The second change did not improve the steady frame rate. The cached mesh had already made drawing effectively free. Simplification reduced memory and made geometry rebuilds cheaper instead.

The first change improved frame rate by about ten times. The second helped memory and rebuild hitches, but not steady rendering speed.

And one guard rail on top: never trust a segment count on its own. A tool that reports "eight times fewer segments" is reporting that it threw things away, not that it kept the right ones. The check that actually matters measures how far the simplified line strays from a densely sampled reference, and that number is what says a change to the sampling was safe.

The layer that was designed and then deliberately not built

The engine has a data oriented entity framework for large agent counts. The first plan was to build the traffic layer on it. That plan was dropped, and the reasoning applies to more than one engine.

The first reason is scale. That framework earns its complexity at thousands of agents, where aggressive level of detail and instanced rendering are the only way to hold frame time. The target here is one to two hundred physical vehicles. At that count it buys nothing and costs a pile of module dependencies and ordering rules.

The second reason is the one that actually settled it. A processor in that framework that also integrates motion would be a second simulation, running against the first, and we are back to the blending option from the top of this article, the one that produces jitter nobody can locate.

Any layer on the engine side is a mirror and a synchronisation layer. It is never a simulation. So the shape is deliberately dull: a map from car identifier to actor, a pool of actors to avoid spawning, and each tick a gather, a tick, and a scatter over flat arrays. If the count ever does reach thousands, that becomes a change to how the arrays are iterated, and not a change to anything the simulation exposes.

The calls that matter are already built for that shape. Transforms go in as one array. Car states come back into a buffer the host owns and reuses, so a thousand cars cost no allocation per frame. Both are bulk by default with the per car versions kept only for convenience.

To be clear about status: that engine side layer is a decision, not code. It is written down, with the reasoning, so that it is a decision that survives being forgotten.

What is not done

Being straight about the gaps, since this article is otherwise a list of things that worked.

Region streaming, which decides how many cars exist around the player and where they spawn, is fully built on the simulation side and is not on the bridge yet. The engine knows where players are, and the simulation knows where the lanes are, which spots are free, and how to route a new car. That split is right. The wiring is missing.

Recording and metrics are not exposed either. A game does not need them. A test level would.

And the density number itself is still to be measured on a real map rather than guessed, because part eight says the knee is sharp and a guessed percentage is very likely to be on the wrong side of it.

The integration is now built around one clear owner for each fact. Unreal owns physical transforms. DrivEngine owns traffic decisions and the editable road model. Unit conversion happens at the boundary, signal updates are sent only when they change, and unmatched data is counted instead of disappearing silently.

The missing work is still concrete: connect region streaming, expose recordings for test levels and measure a useful traffic density on a real map.