- ImGui.GetIO().MousePos set from IInputState.MouseX/MouseY each frame
- ImGui.GetIO().MouseDown[0-2] set from MouseLeft/Right/Middle
- DisplaySize also updated each frame (was only set at init)
- Done before BeginImGuiFrame() / ImGui.NewFrame()
Root cause: used OpenGL cubemap conventions (up=+Y) instead of Vulkan (up=-Y)
for X and Z faces. Also applied M22 *= -1 flip to shadow projection which
is wrong — that flip is only for swapchain rendering, not for offscreen
cubemap rendering.
Fix:
- Faces 0,1,4,5 (+X,-X,+Z,-Z): up = (0,-1,0) — Vulkan cubemap convention
- Face 2 (+Y): up = (0,0,1) — was (0,0,-1), now correct
- Face 3 (-Y): up = (0,0,-1) — was (0,0,1), now correct
- Removed M22 *= -1 from shadow projection (not needed for offscreen cube)
- 227 tests, all pass
Root cause of broken shadows: depth buffer stores non-linear NDC depth,
not linear distance. closestDepth * 60.0 was wrong conversion.
Fix: switch from depth-only to R32_SFLOAT color attachment approach:
- Shadow vertex shader outputs world position to fragment
- Shadow fragment shader writes length(fragPos - lightPos) / farPlane
- Main fragment shader samples cubemap, multiplies by FAR_PLANE=60
- Separate color cube (R32_SFLOAT, sampled) + depth cube (D32_SFLOAT, depth test)
- Shadow pipeline: 1 color attachment (R) + depth attachment
- Color clear = 1.0 (max distance), depth clear = 1.0
Tests: 227 total, all pass
- ShadowMapFaceDirectionTests: 6 face directions, 90° FOV, up vectors,
valid matrices, far plane consistency
- ShadowShaderTests: all 6 SPIR-V shaders exist
Root cause: TransitionImageLayoutDepth used LayerCount=1, so only
layer 0 of the 6-layer cubemap was transitioned to ShaderReadOnlyOptimal.
Layers 1-5 stayed in Undefined/DepthStencilAttachmentOptimal → sampling
returned garbage → no shadows on surfaces facing those directions.
Fix: added layerCount parameter to TransitionImageLayoutDepth (default=1).
Shadow cubemap transitions use layerCount=6.
- VulkanShadowMap: 1024x1024x6 layer D32_SFLOAT cube image
- CubeCompatible flag, cube view for sampling, 6 face views for rendering
- GetFaceViewProj: 6 directions (+X, -X, +Y, -Y, +Z, -Z) with correct up vectors
- 6 shadow render passes per frame (one per cube face)
- Fragment shader: samplerCube instead of sampler2D
- Shadow: direction from light to fragment, distance comparison
- No more perspective frustum limitation — omnidirectional shadows
- shadow.vert: same push constant block (160B), uses lightViewProj per face
- triangle.vert: removed fragLightSpacePos (not needed for cubemap)
- Reduced shadow map size to 1024 (6x memory vs single 2048)
- Floor/Grid excluded from shadow casting
- Vulkan spec: each stage can only appear in ONE push constant range
- Merged 3 ranges into 1: offset=0, size=160, stageFlags=Vertex|Fragment
- Both main and shadow pipelines use same single range
- Push constants packed into 160-byte buffer and sent in one call
- Shadow pass: restored vertex/index buffer binding + push constants
- Fixes: device lost, validation errors, missing shadow geometry
- drawCalls now include castShadow flag (bool)
- Floor and Grid entities: castShadow=false (skip in shadow pass)
- Shadow pass: foreach with 'if (!dc.castShadow) continue;'
- Main pass: renders all objects (floor receives shadows but doesn't cast)
- Light position animated with sin/cos combinations at different frequencies
- XZ plane: figure-8 pattern (sin*0.7 + cos*0.3, cos*0.6 + sin*0.4)
- Y: oscillates 7-17 with sin*0.5
- Shadows move dynamically as light orbits the scene
- Light moved to (0, 20, 0) — static, no physics body
- Intensity 50, range 60, warm white color
- Shadow map renders from this fixed position
- LightBall removed, replaced with MainLight entity
Root cause: GLSL std140 layout adds padding after mat4 for vec4 fields,
causing light data to be read at wrong offsets → NaN/artifacts.
Fix:
- UBO back to 64 bytes (mat4 vp only)
- Light data (pos + color, 32 bytes) sent via push constants at offset 64
- Two push constant ranges: Vertex (0-64, model) + Fragment (64-96, light)
- Both vertex and fragment shaders read from same push_constant block
- PBR shader fully restored with Cook-Torrance BRDF
- Dynamic point light follows physics ball
Light at (0,10,0) hardcoded in GLSL. Tests if lighting math works.
If light works: problem is UBO layout for light data (std140 padding).
If artifacts: problem is in fragWorldPos or fragNormal.
- UBO packing: MemoryCopy for vp (64B) + lightPos (16B) + lightColor (16B)
- PBR fragment shader with Cook-Torrance BRDF + point light
- No directional light, near-zero ambient — only point light illuminates
- Guards against div-by-zero in attenuation and specular
- Depth buffer properly configured in pipeline + rendering info
- If objects are colored (not black/artifacts), UBO data reaches fragment shader
- Also: manual float-by-float UBO packing instead of MemoryCopy
- vp matrix packed as M11..M44 (row-major, no row_major in GLSL = transpose)
- Marshal.StructureToPtr may not handle Matrix4x4 correctly (record struct)
- Replaced with MemoryCopy from local copies of vp, lightPos, lightColor
- Removed per-frame light debug logging
- Simple diffuse shader still active for testing
Root cause: descriptor set layout binding had stageFlags=Vertex only.
Fragment shader needs pointLightPos/pointLightColor from UBO but couldn't
access it. Changed to Vertex|Fragment.
- Only one light source: the physics LightBall
- Intensity 30 (was 20), range 40 (was 30) for stronger illumination
- Static MainLight removed so renderer picks the dynamic ball
- Removed directional light from fragment shader
- Ambient reduced to near-zero (0.01) — scene is dark
- Only light source is the physics LightBall falling from y=15
- Objects near the ball are illuminated, far objects are in darkness
- Dramatic contrast shows dynamic lighting clearly
- LightBall entity: sphere mesh + DynamicSphere physics + Light.Point
- Renderer reads light position from Transform (not static Light.Position)
so the light moves with the physics body
- Light intensity 20, range 30, warm color (1.0, 0.9, 0.7)
- Ball falls from y=15, bounces on floor, light follows it
- Renderer: prefers Light+Transform pair for dynamic position, falls back to Light only
- UBO expanded from 64 to 128 bytes: vp(64) + lightPos(16) + lightColor(16)
- Vertex shader: passes point light UBO data through
- Fragment shader: refactored calcLight() function, shared by directional + point
- Point light: Unity-style attenuation pow(1 - dist/range, 2)
- Directional light reduced intensity (0.4) to balance with point light
- Point light: position (0,8,0), warm color (1,0.9,0.7), intensity 15, range 25
- Renderer: reads Light component from ECS, packs into UBO with Marshal.StructureToPtr
- Scene: MainLight entity with Light.Point at (0,8,0)
- 0 C# struct changes — pure UBO layout + shader upgrade
- Vertex shader: passes world position, world normal, albedo to fragment
- Fragment shader: full PBR implementation
- D term: Trowbridge-Reitz GGX distribution
- G term: Smith geometry with Schlick-GGX
- F term: Schlick Fresnel approximation
- kD/kS split based on metallic
- Directional light (0.5, 0.8, 0.3) with warm color
- Ambient term (0.15, 0.18, 0.22) for fill light
- ACES filmic tonemapping
- Gamma 2.2 correction
- Fixed roughness=0.5, metallic=0.1 (per-object materials = future)
- No C# code changes — pure shader upgrade
- MCP server (Kestrel) launched on background thread after 3 frames
to avoid conflicting with Vulkan/SDL3 initialization
- --mcp-port 0 (default) = disabled, scene works as before
- --mcp-port 5000 = MCP starts after 3rd frame
- Reduced extra balls from 30 to 10 for stability
- Engine.AI/McpEngineServerHost reverted to original (no WebApplicationOptions)
- Engine.AI referenced in CortexEngine.App.csproj
- AiCommandProcessor created with world + mesh loader + screenshot callback
- AiCommandQueue marshals commands from MCP thread to main thread
- MCP HTTP server (Kestrel) started on separate background thread
- --mcp-port N flag (default 0 = disabled) controls MCP server
- queue.ProcessPending() called before render each frame
- queue.CompletePendingScreenshots() called after render
- ImGui overlay shows MCP status (port or disabled)
- DummyScreenshotProvider placeholder (real capture = future checkpoint)
- 7 MCP tools: spawn_model, set_transform, set_material, delete_entity,
list_entities, get_world_state, capture_screenshot
- MCP server has 2s startup delay to avoid blocking Vulkan init
- Default: MCP disabled (--mcp-port 0), scene + physics + ImGui work
- Engine.Physics (JoltPhysicsSharp) integrated into CortexEngine.App
- 4 cubes: DynamicBox, mass proportional to scale, fall from height 5-10
- 3 spheres: DynamicSphere, mass 1.5, fall from height 7-12
- Floor: StaticBox (20x0.5x20), friction 0.8
- Torus knot + grid: no physics (visual only)
- Physics bodies initialized lazily on first frame (IsInitialized flag)
- e.Set() moved outside world.Each to avoid mutation during iteration
- physicsWorld.Update + SyncTransforms every frame
- Removed manual rotation — physics handles movement now
- Camera at (0, 5, -15) for better view of falling objects
- 0 validation errors, ~1300 FPS
- 10 entities in ECS World with Transform + Mesh components
- Torus knot (center, scale 1.5) rotating around Y
- 4 cubes (left/right/front/back) rotating at different speeds
- 3 spheres (procedural, 32x16) rotating around X
- Floor (20x20 flat cube) + grid (ProceduralMesh.CreateGrid)
- Camera at (0, 3, -12) looking at (0, 0.5, 0)
- Mesh cache: buffers created lazily per entity ID on first frame
- Diffuse lighting on all objects
- 0 validation errors, ~2300 FPS
- Vertex shader: pass normal through mat3(model) to fragment
- Fragment shader: directional light (0.5, 0.8, 0.3) with 0.25 ambient
+ 0.75 diffuse = faces now have depth and 3D form
- Torus knot has smooth normals (vn in OBJ), cubes have face normals
- 0 validation errors, ~2300 FPS
- VulkanRenderer: mesh cache by entity ID, lazy buffer creation
- RenderWorld iterates entities with Transform+Mesh, draws each with
per-entity model matrix via push constants
- Program.cs: creates scene with 1 torus knot (center, scale 1.5) +
6 cubes at different positions, all rotating around Y at different speeds
- Camera at (0, 2, -8) looking at origin, FreeFly controls
- 0 validation errors, ~2100 FPS with 7 objects
Root cause: System.Numerics.Matrix4x4 uses row-major storage with row-vector
convention (v * M). GLSL with layout(row_major) does column-vector (M * v),
which effectively transposes the matrix, breaking the projection.
Fix: remove row_major from layout qualifiers. GLSL then interprets the
row-major data as column-major (effectively transposing it), so M * v in
GLSL equals v * M in System.Numerics — correct result.
The Y-flip (M22 *= -1) is still needed for Vulkan's Y-down clip space.
System.Numerics.CreatePerspectiveFieldOfView uses DirectX convention (Y up).
Vulkan clip space has Y down. Negating M22 corrects the projection.
Depth is already [0,1] in System.Numerics (DirectX-style), no remap needed.
- Program.cs: create Camera entity in ECS World, FreeFlyCameraController
with WASD movement, Q/E up/down, Shift boost, right-click mouse look
- VulkanRenderer.RenderWorld: queries Camera from World, computes VP
from Camera.GetViewMatrix() * GetProjectionMatrix()
- Camera aspect ratio auto-updated on window resize
- Fallback to hardcoded VP if no Camera entity found
- Torus knot still rotating via push constant model matrix
- 0 validation errors
- Push constant changed from float angle (4B) to mat4 model (64B)
- Two draw calls with different model matrices (x=-1.5 and x=+1.5)
- Both cubes rotate around Y axis at 0.2 rad/s
- 0 validation errors
emil28092005