shacraft-core/docs/WORLD_STREAMING.md

Procedural worlds and movement diagnostics

The server creates overworld on startup if that name is unused. Open http://localhost:4000/?world=overworld, or select it in the world menu. Existing worlds, packages, edits and arenas remain available. An existing world named overworld is never automatically converted.

World dimensions and generation

• Playable horizontal coordinates: −29,999,872 through +29,999,872.
• Building layers: Y = −64 through 319, inclusive: 384 layers.
• Default seed: 20260914; generator version: 1.
• Original deterministic terrain: continents, hills, mountains, rivers, oceans, forests, plains, desert areas, snow, trees, caves, tunnels, aquifers, deep stone, ores, low lava pockets and a bedrock bottom.
• A spawn clearing and a narrow ravine near X=30 give access to the surface and underground. Fluids are generated blocks; they do not yet simulate flow.

The height range follows the modern Java Overworld convention described in Mojang's Caves & Cliffs Part II announcement. Terrain generation is independently authored. Matching Minecraft seed output, all vanilla biomes, structures and gameplay systems is outside this implementation.

The version and seed are durable in server.sqlite3. Generation depends on absolute integer coordinates, so request order and section boundaries do not change trees, caves or terrain. Keep generator version 1 stable for saved worlds; future algorithms need a new version and an explicit migration strategy.

Player simulation, action rays and spawn coordinates use double precision. Meshes store section-local float positions; draw matrices and dynamic actors use an origin near the camera. Small geometry and movement steps remain representable near the horizontal border. Arena/entity administration retains its earlier configuration limits where documented.

CPU work, streaming and storage

The server uses two CPU generation threads with 32 pending jobs at most. The simulation thread owns storage and gameplay. Requests are shared between players and ordered by distance, with extra weight on vertical distance. Old view requests are dropped; completed jobs outside current interests are discarded. Control reads retain bounded, temporary interests so remote reads can complete without a nearby player. Generation does not require a GPU.

Game menu → Render distance selects a radius from 2 to 64 chunks (32–1,024 blocks around the player), default 3. Each player chooses their own radius; changes apply without reconnecting and persist locally. The current client negotiates an extra loaded chunk ring beyond that visible radius. Its default data window is 9×9×24 sections, or 144×384×144 blocks / 1,944 sections. At radius 8 it is 19×19×24, or 304×384×304 blocks / 8,664 sections. At radius 64 it is 131×131×24, or 2,096×384×2,096 blocks / 411,864 sections, including the one-chunk preload ring. The queue covers this entire window. It is sorted once when the view changes; each tick examines a bounded lookahead of 256 pending sections plus collision neighborhoods. Up to 128 sections are sent per 50 ms interval, including at most 32 nonuniform sections, with a 256 KiB estimated section-payload budget, backpressure and time budgets.

Horizontal streaming has one chunk of hysteresis: crossing a single chunk boundary does not move the loading anchor or unload its opposite edge. The anchor moves after a two-chunk displacement. The extra ring preserves coverage of the selected radius between anchor changes. Generation, delivery and meshing prioritize nearby sections; overlapping cells and meshes remain in place. Expansion retains overlap; explicit reduction unloads excess sections. Distance fog completely blends the visible edge into the sky before the buffer is unloaded. Horizontal culling and the projection far plane follow the radius.

The current client negotiates full_height_v1: every loaded column spans Y=−64 through 319, all 24 vertical sections. Climbing and flying above the world ceiling retain ground and roof sections. Missing interior terrain still blocks prediction; space outside the world height is known void. Clients that do not negotiate this feature retain the legacy 80-block sliding window. Distant-terrain LOD is not implemented; these bounds and local benchmarks do not establish parity with Minecraft's renderer.

Already delivered distant sections do not need to remain in the server cache; only collision neighborhoods and unsent client sections trigger regeneration. Requests from multiple players still deduplicate, including a new player requesting terrain previously sent to someone else. Natural sections live in a bounded LRU: at most 8,192 decoded sections, or 128 MiB of voxel payload. This is separate from the core's encoded-section cache, SQLite, metadata, meshes and total process RSS. Visiting more terrain does not allocate a permanent world-sized array. Before editing a natural section, the server materializes its immutable baseline in generated_sections in worlds.sqlite3; the existing revision/idempotency journal then stores the edit. An explicit air deletion survives cache eviction and restart. Undo/reset use the saved baseline. World templates inherit the generator configuration and pin existing edited sections. Natural materialization does not advance the edit revision. Standalone core exports contain stored sections; exporting the entire procedural world is not supported. Back up both SQLite databases.

The browser stores nonuniform cells in Uint32Array sections: 30.38 MiB for a dense 1,944-section default voxel map, or 135.38 MiB at radius 8, excluding worker copies, lighting and GPU geometry. Uniform received sections use four bytes instead of 16 KiB, retain known-air readiness, and expand on edit. Empty sections need no mesh job. Lighting has a separate allocation cap of 7,393,280 cells; it does not scale with the maximum stream volume. With the local mesh worker, the shared actor field is limited to 112×384×112 blocks around the view center. Palette/RLE avoids JSON objects and position strings per block.

A dedicated terrain-mesh worker builds local light, AO, culling and geometry without waiting for whole-view lighting. A light tile covers 2×2 chunk columns plus an 18-block horizontal halo, at most 68×384×68 / 1,775,616 cells. The full active vertical range preserves skylight under tall roofs. An eight-entry LRU shares these fields between nearby and vertically stacked sections. Spatial invalidation and per-section tickets reject obsolete work without discarding nearby results on every distant section batch. Moving a distant view edge keeps an interior tile's light and mesh valid if its sampling bounds are unchanged.

A separate persistent worker computes a bounded local light field for actor and quick-edit sampling, starting after 225 ms without a new sync. Its completion does not gate terrain publication. Large views still need time to finish distant meshes. Uploads are staged in 64 KiB steps targeting 2 ms per frame; old meshes remain until their replacements are ready. Driver calls are not preemptible.

Unknown sections are distinct from received air. Both authoritative simulation and prediction wait for collision data instead of falling through missing terrain. The server resets the input epoch on a terrain wait, holds the body, and resumes when its neighborhood is available. Teleports also reset prediction.

Skylight includes full-height column occlusion data. A cave remains dark when its roof lies above the streamed vertical window. Editing a roof updates its column data. The background uses the camera's sky exposure to avoid bright sky appearing through unloaded underground terrain. Full lateral lighting outside the active window is not simulated.

Section protocol v2

Join with features:["chunk_stream_v2","movement_prediction_v1","view_buffer_v1","full_height_v1"] and optional view_distance:2..64 (default 3). A server advertising view_distance_v1 accepts {"type":"view_distance","chunks":64} while connected. Noninteger or out-of-range values are rejected before mutation; changes are limited to once per 500 ms. The server responds with a normal view transition containing view_distance and the new total_sections, preserving position, motion epoch and revision. Negotiating view_buffer_v1 adds the extra loaded ring and horizontal hysteresis; stream_radius reports the loaded radius, while view_distance remains the selected visible radius. Older v2 clients retain the unbuffered behavior: 245 sections by default and 1,445 at radius 8. full_height_v1 fixes the vertical bounds to the world height and adds full_height:true to view/welcome/snapshot messages. It also enables batches of up to 128 sections; older clients receive at most 32. Welcome/snapshot messages also expose max_view_distance:64. The initial welcome or explicit snapshot supplies the world, revision, motion, generation, view_center, inclusive view_min/view_max, total_sections, terrain settings and world bounds. Its blocks array is empty; sections follow incrementally. Generation numbers increase for each new view or resync.

A view message updates bounds and lists departing section coordinates in unload. Retained sections keep their current data. A sections message contains at most 128 complete sections (32 for legacy clients), each as:

{"section":[0,4,0],"palette":[0,123],"runs":[256,1,3840,0]}

Runs are (count, paletteIndex) pairs totaling exactly 4096 cells. Cell order is x + 16*z + 256*y. Section coordinates are integers; world coordinates are section coordinates multiplied by 16. Even empty sections are sent explicitly. Messages include world, revision, generation, material definitions and columns:[{column:[sectionX,sectionZ],heights:[256 values]}].

Batches run at most once per 50 ms per client. The server pauses section sends above 512 KiB of queued output; existing queue and timeout limits still apply. Packing has a soft 10 ms work budget. Clients validate the complete batch before mutation, ignore obsolete generations and resync on revision gaps. Ordinary block edits advance revisions; view changes never do.

Old clients retain v1/full-snapshot behavior in nonprocedural worlds. A client without v2 is rejected from a procedural world with an instruction to refresh.

Diagnostics

Open F3 → Movement diagnostics. Choose manual recording, automatic flight, automatic running, or flight with turns, then start a 30-second recording. Automatic scenarios use ordinary authoritative input, climb obstacles during flight, and need no pointer lock. Recording starts after terrain/meshes settle. Stop or Esc cancels automatic movement. Hiding the tab stops recording and marks the result interrupted: "tab-hidden".

The panel stays open while playing or recording; clicking the world restores control, and F3 or the close button hides it. Its controls pause manual input while focused. The panel shows a rolling frame/CPU chart and a summary. Download JSON exports bounded frame samples, network/stream events and server samples. It measures frame p50/p95/p99/max, frames over 25/50 ms, main-thread physics/update and draw submission time, long tasks, position corrections, terrain waits, distance, view changes, batches and mesh resets. Draw CPU duration is not GPU duration. The canvas DOM exposes the summary as data-dynamics-result, plus current section counts, coordinates, readiness and streaming timings for browser checks. data-first-terrain-ms measures first nonempty mesh publication after the world reset. data-near-meshes / data-near-loaded count completed and received sections in the camera's 3×3×3 neighborhood, including completed empty sections. data-terrain-meshing, data-local-light-cells and data-local-light-ms expose the independent geometry path and its local lighting work.

Authenticated Control API methods for repeatable diagnosis:

{"method":"diagnostics.snapshot","params":{}}
{"method":"terrain.inspect","params":{"world":"overworld","position":[1000000,0,-1000000]}}
{"method":"player.teleport","params":{"id":"player-id","position":[1000000.125,180,-1000000.875],"flying":true}}
{"method":"world.create","params":{"world":"another-overworld","terrain":{"seed":42,"version":1}}}

diagnostics.snapshot reports generation queues/cache, simulation timings, positions, velocity, input backlog, waits and per-player stream state, including view_distance and stream_radius. Teleport is an administrative API, not a client movement permission. /api/metrics adds generation/stream counters and the most recent 600 tick durations. world.read, world.edit, build commit and camera.capture may return terrain pending for cold sections; retry the same read or idempotent operation after a short interval. No blocking generation is performed for a cold read.

Local verification, 2026-09-14

A release server and the real browser ran against an isolated copy of the local data, using the Pixel32 Study package. Automatic flight traversed 495.08 m in 30.003 s, with 39 view changes and 78 section batches. Across 4,317 frames, frame p95 was 7.0 ms, p99 7.1 ms, maximum 13.8 ms; no frame exceeded 25 ms. The run reported no terrain-wait frames, position correction or full mesh reset. Section application p95 was 5.6 ms, maximum 7.3 ms. At the last server sample, tick work was 0.53 ms and RSS was 78.5 MiB. Three entering sections were still in transit at recording end; all 245 completed after stopping.

These measurements are one local workload, not a comparison with Minecraft or a guarantee for other devices, multiplayer loads or long sessions. The local summary is saved in ignored artifacts/large-world-flight.json.

Automated checks cover generation determinism, negative/far coordinates, deep content, skyline accuracy, durable mining/undo/restart, remote reads without players, bounded batches, overlap retention, malformed/stale data, known air, shared physics and section-local geometry precision. Browser checks also cover an open ravine at Y=10, a closed cave at Y=−29, and coordinates near ±29,999,000.

Render distance and live edit verification, 2026-09-15

These measurements predate the additional loaded ring described above.

In an isolated copy of the Overworld, the real browser expanded from radius 3 to 8 and received all 1445 sections (23,674,880 raw voxel bytes). Shrinking to radius 2 retained 125 sections (2,048,000 bytes), without a full snapshot, mesh reset, revision change or player displacement. Reloading restored the selected radius from local storage. The server regression also checks per-player isolation, invalid values, overlap retention and the world-border margin.

At radius 8, confirmed-to-visible-section publication took 121.9 ms for a break and 120.6 ms for placement in two browser samples. Including server response, they took 149.4 ms and 137.2 ms. Local packet preparation was 0.6–0.9 ms. The regular whole-view light calculation in the same session took about 4.7 seconds; the independent edit worker published geometry before it finished. These are local observations at 1280×720 with shadows and about 1.27 million triangles, not frame-time or network guarantees. Propagated light can still catch up later on large views.

F3 remained open on canvas clicks, block edits and recording start; explicit F3 toggled it closed and open. The full verification script passed (126 Rust and 171 JavaScript tests, storage crash checks and all three network suites); an additional staged edit-upload regression also passed afterwards.

Nearby mesh priority and buffered streaming verification, 2026-09-15

In the real browser at radius 8 with the buffered 1,805-section window, first nonempty mesh publication took 354 ms and 388 ms in two startup samples. All 27 nearest sections had completed while only 1,167 of 1,805 sections had arrived and whole-view lighting was still building. Subsequent local-light cache tiling improves reuse; those startup samples were taken before that tuning. A final reload with the tiled cache and a warm server published its first mesh in 189 ms, completed all 1,805 sections and logged no browser warnings/errors.

A 30-second automatic flight at 1280×720 with 2048² shadows traversed 497.12 m. Across 3,746 frames, frame p95 was 13.9 ms, p99 20.9 ms, maximum 48.6 ms. There were no frames above 50 ms, terrain-wait frames or full mesh resets during the recording. It recorded 39 view changes and 147 batches. Stream application p95 was 9.4 ms. These are local workload observations, not cross-device or Minecraft comparison results.

Server regressions exercise repeated crossings of a single chunk boundary, two-chunk anchor changes, coverage of the selected radius and departing rows outside that radius. Client regressions compare local-light mesh bytes with the whole-view result, including roofs, water, glass and negative coordinates; they also check distant-update retention, bounded caches, stale jobs, unload/re-entry tickets and unchanged interior tiles during view shifts. The verification script passed 127 Rust tests, storage crash checks and all three network suites. The final client suite passed 180 JavaScript tests.