Root cause: after Destruct(), physics world still references dead
entities in its internal dictionary. SyncTransforms iterates dead
entity IDs → Flecs assert ecs_is_alive.
Fix:
- Pause physics during reset
- For each deleted entity: check Has<RigidBody>, call physicsWorld.RemoveBody
to clean up Jolt body + dictionary entries BEFORE Destruct
- Resume physics after CreateObjects
- New objects get IsInitialized=false, physics bodies created next frame
- renderTimer accumulates deltaTime each tick
- Render only when renderTimer >= 16.67ms
- If timer overshoots (slow frame), clamp to 0 to prevent spiral of death
- No Thread.Sleep/continue — loop runs continuously, just skips render
when not enough time has passed
- Physics + input + light animation: run at full speed every iteration
- Render: accumulator-based 60 FPS limiter — only renders when enough
time has accumulated (16.67ms). If not, Thread.Sleep(1) and continue.
- Video: captured at exactly 60 FPS, FFmpeg downsamples to 30
- Physics stays smooth even if render can't keep up
Root cause: driver returned format=58 (A2B10G10R10_PACK32) as fallback
when B8G8R8A8_SRGB wasn't found. 10-bit packed format has different
byte layout → vkCmdCopyImageToBuffer gives garbage when interpreted as
8-bit BGRA → acid colors in video.
Fix:
- QuerySurfaceFormat: try B8G8R8A8_UNORM first (NVIDIA supports it)
- Fallback: B8G8R8A8_SRGB, then first available
- Removed SRGB LUT conversion — UNORM data is already linear
- Raw BGRA bytes go directly to FFmpeg (pixel_format=bgra)
Three fixes:
1. imageCubeArray: enabled via VkPhysicalDeviceFeatures (core 1.0 feature)
instead of VkPhysicalDeviceImageCubeArrayFeatures (wrong sType in pNext
chain was interpreted as VkExternalMemoryImageCreateInfoNV)
2. Swapchain format: reverted to B8G8R8A8_SRGB (was UNORM which driver
didn't support → fell back to A2B10G10R10 10-bit format=58 → wrong
pixel layout for capture)
3. Video colors: SRGB→linear conversion via 256-entry LUT in
ReadCapturedBuffer. SRGB swapchain gives gamma-compressed bytes,
FFmpeg expects linear. LUT converts each B/G/R channel using
standard SRGB formula: s<=0.04045 ? s/12.92 : ((s+0.055)/1.055)^2.4
Also: video captures before ImGui (no UI in recording)
1. No UI in video: end render pass before ImGui, capture, start new
render pass with loadOp=LOAD for ImGui, end again. Video captures
only the 3D scene.
2. Acid colors fixed: swapchain format changed from B8G8R8A8_SRGB to
B8G8R8A8_UNORM. SRGB format stores gamma-compressed values that
vkCmdCopyImageToBuffer reads raw — FFmpeg treated them as linear
causing acid colors. UNORM stores linear values, copy gives linear
data, FFmpeg bgra→yuv420p conversion is correct.
Swapchain format is B8G8R8A8_SRGB — vkCmdCopyImageToBuffer gives BGRA
data. Was converting BGRA→RGBA in C# and telling FFmpeg 'rgba' —
but SRGB data needs to stay as-is. Now:
- No BGRA→RGBA swap in ReadCapturedBuffer
- FFmpeg pixel_format=bgra (matches swapchain)
- FFmpeg handles SRGB→yuv420p conversion correctly
Root cause: CaptureFrame returned null and was assigned to CapturedFrame,
overwriting the data that ReadCapturedBuffer had just set at the start
of the same frame.
Fix: CaptureFrame is now void — only records GPU copy commands.
ReadCapturedBuffer runs at start of next frame (after WaitFrame/fence)
and sets CapturedFrame with actual pixel data.
Program.cs reads CapturedFrame after RenderWorld — now contains data
from the previous frame's GPU copy.
- Removed skipFrames logic — capture every frame for reliability
- FFmpeg framerate set to actual FPS (capped at 60) instead of hardcoded 30
- Added -vf fps=30 filter to downsample to 30fps in output
- First frame: CapturedFrame is null (no previous capture), skipped naturally
- Second frame onwards: data available from previous frame's GPU copy
0x2000 was VK_ACCESS_MEMORY_READ_BIT in Vulkan 1.0 but maps to
VK_ACCESS_2_HOST_READ_BIT in sync2. Replaced with 0x1000
(VK_ACCESS_2_TRANSFER_WRITE_BIT) for dst access on transfer layout.
Three validation errors fixed:
1. vkCmdCopyImageToBuffer inside render pass → moved AFTER vkCmdEndRendering
2. Swapchain images missing TRANSFER_SRC usage → added to swapchain creation
3. Wrong layout transitions → ColorAttachment→TransferSrc→copy→ColorAttachment
Architecture change:
- CaptureFrame now only records commands (returns null)
- ReadCapturedBuffer called at start of NEXT frame (after WaitFrame/fence)
GPU has finished by then, safe to map memory
- CapturedFrame available 1 frame late (acceptable for video)
- Image transitions: ColorAttachmentOptimal→TransferSrcOptimal→ColorAttachmentOptimal
(present transition still works because image is back in ColorAttachmentOptimal)
- All 3 lights: intensity 8.0, range 15.5 (was 12/8/8, 60/40/40)
- Copy Parameters: also logs to console for debugging
- Clipboard text via ImGui.SetClipboardText (standard API)
Root cause: descriptor set layout binding 0 (SceneUBO) had stageFlags=Vertex
only. Fragment shader reads numLights, lights[], shadowParams[] from UBO
but couldn't access it. Changed to Vertex|Fragment.
CULL_MODE_BACK was culling faces with inconsistent winding order in
sphere.obj, torus.obj, diamond.obj, cone.obj. Many faces disappeared,
making objects look like wireframe/grids.
Reverted both main and shadow pipelines to CULL_MODE_NONE.
Shadow acne is handled by low bias (0.0001) which works fine.
- pyramid.obj: verified CCW via cross-product for all 6 faces
- diamond.obj: verified CCW for all 8 octahedron faces
- torus.obj: verified CCW — normal points outward from torus center
- cone.obj: verified CCW for side faces (radial outward) and bottom (-Y)
- Removed Grid entity — balls were falling through it (no physics)
- Removed 'Grid' from castShadow exclusion list
- All objects now render correctly with CULL_MODE_BACK
- AiCommandProcessor: checks if modelPath contains 'sphere' (case-insensitive)
- If detected as sphere: uses DynamicSphere physics body (was DynamicBox)
- Works with both shape='sphere' parameter and modelPath='Content/sphere.obj'
- No need to explicitly pass shape parameter when loading sphere.obj
- Changed from Thread to Task.Run for MCP server
- Start delay: 10 frames (was 3) — let scene fully initialize first
- Kestrel on ThreadPool instead of dedicated thread
Root cause: CULL_MODE_NONE in both main and shadow pipelines caused
back faces to render, creating self-shadowing acne on all objects:
- Shadow map: back faces written at closer depth → false shadows on front
- Main render: back faces visible through objects → wrong lighting
Fix:
- Main pipeline: CULL_MODE_BACK (was None) — only front faces render
- Shadow pipeline: CULL_MODE_BACK (was None) — only front faces cast shadows
- Removed normal flip hack (if dot(N,V) < 0 N = -N) — no longer needed
with proper culling + correct CCW winding in cube.obj
- This is the standard approach: correct geometry + back-face culling
Root cause: ALL 12 faces in cube.obj had reversed winding order.
This caused every face normal to point inward, breaking:
- Diffuse lighting (NdotL < 0 → no light on any face)
- Shadow bias (slope-dependent bias used wrong NdotL)
- Shadow self-shadowing (both sides rendered with CULL_NONE)
Fix: reversed all face indices to correct CCW winding:
- Back face: f 1 4 3, f 1 3 2 (was f 1 2 3, f 1 3 4)
- Front face: f 5 6 7, f 5 7 8 (was f 5 7 6, f 5 8 7)
- Left/Right/Bottom/Top: all reversed
Now MeshMath.ComputeFaceNormal produces correct outward normals.
Normal flip in shader (dot(N,V) < 0) is still kept as safety for
double-sided rendering, but cubes now have correct normals natively.
Previous fix flipped normal by both light AND view direction:
if (dot(N,L) < 0) N = -N;
if (dot(N,V) < 0) N = -N;
This double-flip broke spheres: when light is behind sphere, front faces
get N flipped by light (now facing away from camera), then flipped back
by view — but back faces get flipped once, causing inconsistency.
Correct approach: flip ONLY by view direction:
if (dot(N, V) < 0.0) N = -N;
This ensures the normal always faces the camera. Faces pointing away
from light naturally get NdotL=0 (no lighting) — correct behavior.
Spheres keep smooth normals, cubes get inward normals fixed.
Root cause: cube.obj has inconsistent winding order. Many faces have
normals pointing inward (e.g. back face normal = +Z instead of -Z).
With cullMode=None, both sides render, but inward normals cause:
- No diffuse lighting (NdotL < 0 → max() = 0)
- Wrong shadow bias (slope-dependent bias uses wrong NdotL)
- Inconsistent shadow edges on cube faces
Fix: in fragment shader, flip normal if dot(N,L) < 0 or dot(N,V) < 0
This is the standard approach for double-sided rendering with
non-watertight geometry or inconsistent winding order.
- ImGui.GetIO().MousePos set from IInputState.MouseX/MouseY each frame
- ImGui.GetIO().MouseDown[0-2] set from MouseLeft/Right/Middle
- DisplaySize also updated each frame (was only set at init)
- Done before BeginImGuiFrame() / ImGui.NewFrame()
Root cause: used OpenGL cubemap conventions (up=+Y) instead of Vulkan (up=-Y)
for X and Z faces. Also applied M22 *= -1 flip to shadow projection which
is wrong — that flip is only for swapchain rendering, not for offscreen
cubemap rendering.
Fix:
- Faces 0,1,4,5 (+X,-X,+Z,-Z): up = (0,-1,0) — Vulkan cubemap convention
- Face 2 (+Y): up = (0,0,1) — was (0,0,-1), now correct
- Face 3 (-Y): up = (0,0,-1) — was (0,0,1), now correct
- Removed M22 *= -1 from shadow projection (not needed for offscreen cube)
- 227 tests, all pass
Root cause of broken shadows: depth buffer stores non-linear NDC depth,
not linear distance. closestDepth * 60.0 was wrong conversion.
Fix: switch from depth-only to R32_SFLOAT color attachment approach:
- Shadow vertex shader outputs world position to fragment
- Shadow fragment shader writes length(fragPos - lightPos) / farPlane
- Main fragment shader samples cubemap, multiplies by FAR_PLANE=60
- Separate color cube (R32_SFLOAT, sampled) + depth cube (D32_SFLOAT, depth test)
- Shadow pipeline: 1 color attachment (R) + depth attachment
- Color clear = 1.0 (max distance), depth clear = 1.0
Tests: 227 total, all pass
- ShadowMapFaceDirectionTests: 6 face directions, 90° FOV, up vectors,
valid matrices, far plane consistency
- ShadowShaderTests: all 6 SPIR-V shaders exist
Root cause: TransitionImageLayoutDepth used LayerCount=1, so only
layer 0 of the 6-layer cubemap was transitioned to ShaderReadOnlyOptimal.
Layers 1-5 stayed in Undefined/DepthStencilAttachmentOptimal → sampling
returned garbage → no shadows on surfaces facing those directions.
Fix: added layerCount parameter to TransitionImageLayoutDepth (default=1).
Shadow cubemap transitions use layerCount=6.
- VulkanShadowMap: 1024x1024x6 layer D32_SFLOAT cube image
- CubeCompatible flag, cube view for sampling, 6 face views for rendering
- GetFaceViewProj: 6 directions (+X, -X, +Y, -Y, +Z, -Z) with correct up vectors
- 6 shadow render passes per frame (one per cube face)
- Fragment shader: samplerCube instead of sampler2D
- Shadow: direction from light to fragment, distance comparison
- No more perspective frustum limitation — omnidirectional shadows
- shadow.vert: same push constant block (160B), uses lightViewProj per face
- triangle.vert: removed fragLightSpacePos (not needed for cubemap)
- Reduced shadow map size to 1024 (6x memory vs single 2048)
- Floor/Grid excluded from shadow casting
- Vulkan spec: each stage can only appear in ONE push constant range
- Merged 3 ranges into 1: offset=0, size=160, stageFlags=Vertex|Fragment
- Both main and shadow pipelines use same single range
- Push constants packed into 160-byte buffer and sent in one call
- Shadow pass: restored vertex/index buffer binding + push constants
- Fixes: device lost, validation errors, missing shadow geometry
- drawCalls now include castShadow flag (bool)
- Floor and Grid entities: castShadow=false (skip in shadow pass)
- Shadow pass: foreach with 'if (!dc.castShadow) continue;'
- Main pass: renders all objects (floor receives shadows but doesn't cast)
- Light position animated with sin/cos combinations at different frequencies
- XZ plane: figure-8 pattern (sin*0.7 + cos*0.3, cos*0.6 + sin*0.4)
- Y: oscillates 7-17 with sin*0.5
- Shadows move dynamically as light orbits the scene
- Light moved to (0, 20, 0) — static, no physics body
- Intensity 50, range 60, warm white color
- Shadow map renders from this fixed position
- LightBall removed, replaced with MainLight entity
Root cause: GLSL std140 layout adds padding after mat4 for vec4 fields,
causing light data to be read at wrong offsets → NaN/artifacts.
Fix:
- UBO back to 64 bytes (mat4 vp only)
- Light data (pos + color, 32 bytes) sent via push constants at offset 64
- Two push constant ranges: Vertex (0-64, model) + Fragment (64-96, light)
- Both vertex and fragment shaders read from same push_constant block
- PBR shader fully restored with Cook-Torrance BRDF
- Dynamic point light follows physics ball
Light at (0,10,0) hardcoded in GLSL. Tests if lighting math works.
If light works: problem is UBO layout for light data (std140 padding).
If artifacts: problem is in fragWorldPos or fragNormal.
- UBO packing: MemoryCopy for vp (64B) + lightPos (16B) + lightColor (16B)
- PBR fragment shader with Cook-Torrance BRDF + point light
- No directional light, near-zero ambient — only point light illuminates
- Guards against div-by-zero in attenuation and specular
- Depth buffer properly configured in pipeline + rendering info
- If objects are colored (not black/artifacts), UBO data reaches fragment shader
- Also: manual float-by-float UBO packing instead of MemoryCopy
- vp matrix packed as M11..M44 (row-major, no row_major in GLSL = transpose)
- Marshal.StructureToPtr may not handle Matrix4x4 correctly (record struct)
- Replaced with MemoryCopy from local copies of vp, lightPos, lightColor
- Removed per-frame light debug logging
- Simple diffuse shader still active for testing
Root cause: descriptor set layout binding had stageFlags=Vertex only.
Fragment shader needs pointLightPos/pointLightColor from UBO but couldn't
access it. Changed to Vertex|Fragment.
- Only one light source: the physics LightBall
- Intensity 30 (was 20), range 40 (was 30) for stronger illumination
- Static MainLight removed so renderer picks the dynamic ball
- Removed directional light from fragment shader
- Ambient reduced to near-zero (0.01) — scene is dark
- Only light source is the physics LightBall falling from y=15
- Objects near the ball are illuminated, far objects are in darkness
- Dramatic contrast shows dynamic lighting clearly
- LightBall entity: sphere mesh + DynamicSphere physics + Light.Point
- Renderer reads light position from Transform (not static Light.Position)
so the light moves with the physics body
- Light intensity 20, range 30, warm color (1.0, 0.9, 0.7)
- Ball falls from y=15, bounces on floor, light follows it
- Renderer: prefers Light+Transform pair for dynamic position, falls back to Light only
- UBO expanded from 64 to 128 bytes: vp(64) + lightPos(16) + lightColor(16)
- Vertex shader: passes point light UBO data through
- Fragment shader: refactored calcLight() function, shared by directional + point
- Point light: Unity-style attenuation pow(1 - dist/range, 2)
- Directional light reduced intensity (0.4) to balance with point light
- Point light: position (0,8,0), warm color (1,0.9,0.7), intensity 15, range 25
- Renderer: reads Light component from ECS, packs into UBO with Marshal.StructureToPtr
- Scene: MainLight entity with Light.Point at (0,8,0)
- 0 C# struct changes — pure UBO layout + shader upgrade
- Vertex shader: passes world position, world normal, albedo to fragment
- Fragment shader: full PBR implementation
- D term: Trowbridge-Reitz GGX distribution
- G term: Smith geometry with Schlick-GGX
- F term: Schlick Fresnel approximation
- kD/kS split based on metallic
- Directional light (0.5, 0.8, 0.3) with warm color
- Ambient term (0.15, 0.18, 0.22) for fill light
- ACES filmic tonemapping
- Gamma 2.2 correction
- Fixed roughness=0.5, metallic=0.1 (per-object materials = future)
- No C# code changes — pure shader upgrade
- MCP server (Kestrel) launched on background thread after 3 frames
to avoid conflicting with Vulkan/SDL3 initialization
- --mcp-port 0 (default) = disabled, scene works as before
- --mcp-port 5000 = MCP starts after 3rd frame
- Reduced extra balls from 30 to 10 for stability
- Engine.AI/McpEngineServerHost reverted to original (no WebApplicationOptions)
- Engine.AI referenced in CortexEngine.App.csproj
- AiCommandProcessor created with world + mesh loader + screenshot callback
- AiCommandQueue marshals commands from MCP thread to main thread
- MCP HTTP server (Kestrel) started on separate background thread
- --mcp-port N flag (default 0 = disabled) controls MCP server
- queue.ProcessPending() called before render each frame
- queue.CompletePendingScreenshots() called after render
- ImGui overlay shows MCP status (port or disabled)
- DummyScreenshotProvider placeholder (real capture = future checkpoint)
- 7 MCP tools: spawn_model, set_transform, set_material, delete_entity,
list_entities, get_world_state, capture_screenshot
- MCP server has 2s startup delay to avoid blocking Vulkan init
- Default: MCP disabled (--mcp-port 0), scene + physics + ImGui work
- Engine.Physics (JoltPhysicsSharp) integrated into CortexEngine.App
- 4 cubes: DynamicBox, mass proportional to scale, fall from height 5-10
- 3 spheres: DynamicSphere, mass 1.5, fall from height 7-12
- Floor: StaticBox (20x0.5x20), friction 0.8
- Torus knot + grid: no physics (visual only)
- Physics bodies initialized lazily on first frame (IsInitialized flag)
- e.Set() moved outside world.Each to avoid mutation during iteration
- physicsWorld.Update + SyncTransforms every frame
- Removed manual rotation — physics handles movement now
- Camera at (0, 5, -15) for better view of falling objects
- 0 validation errors, ~1300 FPS
- 10 entities in ECS World with Transform + Mesh components
- Torus knot (center, scale 1.5) rotating around Y
- 4 cubes (left/right/front/back) rotating at different speeds
- 3 spheres (procedural, 32x16) rotating around X
- Floor (20x20 flat cube) + grid (ProceduralMesh.CreateGrid)
- Camera at (0, 3, -12) looking at (0, 0.5, 0)
- Mesh cache: buffers created lazily per entity ID on first frame
- Diffuse lighting on all objects
- 0 validation errors, ~2300 FPS
- Vertex shader: pass normal through mat3(model) to fragment
- Fragment shader: directional light (0.5, 0.8, 0.3) with 0.25 ambient
+ 0.75 diffuse = faces now have depth and 3D form
- Torus knot has smooth normals (vn in OBJ), cubes have face normals
- 0 validation errors, ~2300 FPS
- VulkanRenderer: mesh cache by entity ID, lazy buffer creation
- RenderWorld iterates entities with Transform+Mesh, draws each with
per-entity model matrix via push constants
- Program.cs: creates scene with 1 torus knot (center, scale 1.5) +
6 cubes at different positions, all rotating around Y at different speeds
- Camera at (0, 2, -8) looking at origin, FreeFly controls
- 0 validation errors, ~2100 FPS with 7 objects
Root cause: System.Numerics.Matrix4x4 uses row-major storage with row-vector
convention (v * M). GLSL with layout(row_major) does column-vector (M * v),
which effectively transposes the matrix, breaking the projection.
Fix: remove row_major from layout qualifiers. GLSL then interprets the
row-major data as column-major (effectively transposing it), so M * v in
GLSL equals v * M in System.Numerics — correct result.
The Y-flip (M22 *= -1) is still needed for Vulkan's Y-down clip space.
System.Numerics.CreatePerspectiveFieldOfView uses DirectX convention (Y up).
Vulkan clip space has Y down. Negating M22 corrects the projection.
Depth is already [0,1] in System.Numerics (DirectX-style), no remap needed.
- Program.cs: create Camera entity in ECS World, FreeFlyCameraController
with WASD movement, Q/E up/down, Shift boost, right-click mouse look
- VulkanRenderer.RenderWorld: queries Camera from World, computes VP
from Camera.GetViewMatrix() * GetProjectionMatrix()
- Camera aspect ratio auto-updated on window resize
- Fallback to hardcoded VP if no Camera entity found
- Torus knot still rotating via push constant model matrix
- 0 validation errors
- Push constant changed from float angle (4B) to mat4 model (64B)
- Two draw calls with different model matrices (x=-1.5 and x=+1.5)
- Both cubes rotate around Y axis at 0.2 rad/s
- 0 validation errors
emil28092005