shacraft-core/docs/WORLD_STREAMING_PLAN.md

Large world and movement diagnostics

Objective

Create a playable, deterministic Overworld with 384 block layers (-64 through 319), wide horizontal coordinates, caves and surface terrain. Keep server simulation responsive while terrain is generated and streamed. Preserve existing lobby, arenas, packages and user edits.

Implementation sequence

1. Add durable procedural section baselines to the core store. Only sections touched by edits need materialization; natural terrain is reproducible from a versioned seed. Cover air deletion, undo/reset, restart and template behavior.
2. Add a deterministic terrain generator and a bounded server worker pool/cache. Prioritize player collision neighborhoods, then nearby visible sections; never simulate unknown terrain as empty air. Add generation, cache, queue and tick timing metrics.
3. Add a negotiated compact section-stream protocol with explicit loaded sections, cancellation of obsolete view requests, bounded batches and overlap retention. Keep legacy clients supported. Use dense typed section arrays on the browser side and transferable worker payloads.
4. Support precise distant positions and rendering relative to the camera's section origin. Seed skylight from authoritative column occlusion heights so deep caves stay dark even when roofs lie above the loaded vertical window.
5. Add reusable movement diagnostics: rolling timings, trace recording/export, automatic flight/running scenarios, streaming and prediction counters, accessible controls and machine-readable results.
6. Test deterministic seams, underground content, persistence, streaming order/unknown sections, distant coordinates and worker behavior. Run live browser routes against the real server and report observed results and remaining limits.
7. Create/open overworld on the local server; keep the existing worlds available. Update documentation and the project graph.

Working choices

• Original terrain algorithm, inspired by voxel sandbox worlds; no claim of matching Minecraft's exact seed output or every biome/structure.
• Server remains CPU-only. Background work uses a small fixed worker pool and bounded queues/cache.
• 16x16x16 storage/transfer sections; independent vertical streaming instead of loading all underground layers.
• Explicit readiness for collision data, and staged GPU publication retaining old meshes.
• Seed and generator version are durable world configuration. User edits survive cache eviction and restart.
• Local benchmark routes use normal player inputs. They never clear or overwrite existing worlds.

Completion

All seven steps are implemented. The main server runs the release build at http://localhost:4000/, with the separate overworld open in the browser; the existing lobby remains at revision 210. Full verification passed: 125 Rust tests, 158 JavaScript tests, Clippy, formatting, crash recovery and HTTP/WebSocket scenarios. A focused follow-up test also verifies that legacy clients cannot enter procedural worlds through a world switch. See implementation, measured route and limits.