3.1.1: metals under flat ambient, visionOS PBR maps, occlusion UV set, mirrored nodes

CHANGELOG.md

@@ -1,5 +1,18 @@
 # CHANGELOG
 
+## v3.1.1
+
+### Added
+
+- **visionOS: PBR maps (`Shaders.metal`, `VROMaterialSubstrateMetal`, `VROGeometrySubstrateMetal`).** The Metal PBR path sampled only the base color, so a glTF model on visionOS lost its normal map, metallic-roughness, occlusion and emissive textures and drew flat (DamagedHelmet came out as a white silhouette). `pbr_lighting_fragment_maps` reads all five, by the same factors as the GL path, and the emissive map through its own texture transform. The normal map's tangent frame comes from screen-space derivatives, so geometry needs no tangent attribute. Maps bind at fragment texture indices 8-11 (`VROMaterialSubstrateMetal::getIndexedTextures`), past the shadow and IBL textures.
+- **An occlusion map on its own UV set (`VROGLTFLoader`, `VROShaderFactory`, `VROMaterialVisual::setTexcoordSet`).** A glTF occlusionTexture on `TEXCOORD_1` (a lightmap) was sampled with the base color's coordinates. The second set is loaded as the new `VROGeometrySourceSemantic::Texcoord1` and read only by the occlusion map; GL binds it at attribute 6, the EdgeCrease slot no shader reads, since all 16 guaranteed slots were taken. Not on visionOS, whose Metal path reads one set: there such a map is left out.
+
+### Fixed
+
+- **Metals under flat ambient light (`VROShaderFactory`, `Shaders.metal`).** Without an environment map the PBR ambient term was `ambient × albedo × occlusion`, ignoring metalness, so metal surfaces lit like matte paint (a metal visor looked light grey). It is now scaled by `1 - metalness`, as three.js and Filament do: a metal has no diffuse response and nothing to reflect.
+- **`normalTexture.scale` and `occlusionTexture.strength` apply (`VROShaderFactory`, `Shaders.metal`).** The loader read both into the visuals' intensity, which no shader used. The normal map's X and Y are scaled before normalizing, and occlusion is blended toward 1 by its strength.
+- **Mirrored glTF nodes draw their faces (`VROGLTFLoader`).** A node whose world transform has a negative determinant (a negative scale) flips its triangles' winding, so back-face culling removed the faces that should show. The loader reverses the indices of a mirrored, unskinned mesh.
+
 ## v3.1.0 — 8 November 2026
 
 ### Added

RELEASE_NOTES.md

@@ -1,46 +1,22 @@
 # Release Notes
 
-## v3.1.0
+## v3.1.1
 
-**Advanced VPS, a world mesh that persists, device depth in AR recordings, and a Meta Quest pass with hand occlusion.** The native build ships inside `@reactvision/react-viro` 3.1.0. It links Spatial 1.4.0, which adds continuous localisation against a scanned place and the VPS fuser.
+**glTF models render as glTF defines them on every platform: metals look metallic without an environment map, visionOS draws a model's normal, metallic-roughness, occlusion and emissive maps, an occlusion map can sit on its own UV set, and mirrored nodes draw their faces.** The native build ships inside `@reactvision/react-viro` 3.1.1.
 
-### Advanced VPS
-
-- **VPS map localisation in the AR session.** `VROARSession` gains map load, unload and localisation, implemented on ARKit and ARCore, and exposed on `ARScene` and through JNI. A loaded `.rvmap` is matched against camera frames while the session runs.
-- **Server mode.** The engine hands frames out for the server to localise and feeds the answers into the same fuser, without loading a map (`rvStartVPSServerSession`, `rvReportVPSServerFix`, `rvStopVPSServerSession`, `VROARCamera::getImageTransform()`).
-- **Every fix goes through Spatial's `VROVPSLocalizer`,** which keeps one smoothed map-to-world transform. `VROVPSAnchorNode` keeps content placed in map coordinates following it, and `VROARScene::setVPSMapMesh()` loads the place's mesh as occlusion and a kinematic physics body tagged `"world"`.
-- **Recordings carry GPS and a camera-forward heading** on each pose, which ViroScan turns into the map's geo-registration.
-
-### World mesh
+### Everywhere
 
-- **Depth frames fuse into a persistent room (`VROTSDFVolume`, `VROARWorldMesh`)** on the depth-image and monocular paths, re-meshing only the blocks that changed, within a memory ceiling. Fusion pauses while tracking is limited, a depth drop-out keeps the room instead of falling back to plane polygons, and `resetAccumulation()` starts a new one.
-- **The mesh reports its source** (`lidar`, `depth`, `monocular`, `plane`) and whether it is accumulated.
-- **Android: the depth mesh is geometrically correct** and no longer depends on acquiring a CPU camera image.
-- **Thread safety:** the Android snapshot no longer reads a mesh the render thread is replacing, a reset is not undone by a collision shape still being built, `voxelSize` and `maxMemoryMB` take effect mid-session, and iOS monocular depth is published whole with the pose it was inferred at.
+- **Metals under flat ambient light:** without an environment map the ambient term now leaves out the share a metal reflects, so a metal visor is dark and reflective instead of light grey.
+- **glTF factors:** `normalTexture.scale` and `occlusionTexture.strength` apply.
+- **Occlusion on `TEXCOORD_1`** (a lightmap) samples its own UV set (GL; left out on visionOS).
+- **Mirrored nodes** (a negative scale) draw their faces instead of losing them to back-face culling.
 
-### Meta Quest
+### visionOS
 
-- **Input:** the left Menu button and the palm pinch report `MenuButton`, Y goes back as B does, a menu or system pinch never ends as a select, released hands release, and a hand and a controller on one side press once. `getSourceRay` / `getControllerRayAsync` report where a ray points, and `setInputFocusListener` / `setRoomMoveListener` report focus changes and recentres.
-- **Rendering:** the right eye clears its depth buffer, each eye's framebuffer is made once and its depth discarded after drawing, the aim laser stays on top of panels, and switching AR scenes or closing one logs no errors.
-- **Room planes** stay on the real surface across a recentre, are asked for at once, face the right way, and a desk is a table.
-- **Hand occlusion in passthrough (optional, off by default):** the tracked hands are drawn into each eye's depth buffer before the scene, so the real hands show through virtual content in front of them. It uses the runtime's hand mesh, or capsules between the joints, on Quest 2, 3, 3S and Pro.
-- **Stability:** videos no longer abort the app or leak JNI references, hit tests deliver their results without leaking, and `recenterTracking()` no longer races the render thread.
+- **PBR maps:** normal, metallic-roughness, occlusion and emissive maps, which the Metal path skipped; a textured glTF model no longer draws flat.
 
-### Everywhere
+### Check when upgrading
 
-- **AR recordings can carry the device's depth:** LiDAR scene depth on iOS and the ARCore Depth API on Android, as 16-bit millimetre PNGs with confidence, about 10 a second, when the session is set to capture depth. ViroScan fuses them into the scanned place's mesh, which is what makes blank walls reconstruct.
-- **Physics runs at real time on every display,** in fixed 1/60 s substeps, instead of 1.5× fast at 90 Hz and 2× at 120 Hz.
-- **ARCore reports the ambient light again,** and a paused AR session stays paused when an option or an image or object target changes.
-- **visionOS:** spot light cones match the GL backends, and morph targets blend on the CPU.
-- **Web:** vertex colours show, as on the native backends.
-- **Co-location:** `VROColocationSession::deviceId()`, sent as `x-rv-device`, so the relay bills a device once.
-- **A click goes to the node drawn on top,** and a click in open space reaches the controller's own delegates again.
-- **Text sits where its vertical alignment puts it,** and a back-culled draw after a front-culled one culls back faces again.
-- **GLB characters with one skin per mesh animate every part,** not only the first skin's.
-- **glTF:** `KHR_texture_transform`, `KHR_node_visibility` and `KHR_animation_pointer` (node visibility, base colour and emissive factors, texture offset, rotation and scale), textures on `TEXCOORD_1` and later sets, and emissive textures. Opaque and masked materials ignore their textures' alpha, blended ones stop writing depth, skins animated through nodes between their joints move and are no longer culled by their bind-pose bounds, samplers without `interpolation` load, files with animation pointers no longer fail to load, emission shows on physically based materials, 8-bit index buffers draw, and skinned models draw on the web.
-- **Android: a `ViroARCamera` no longer crashes the scene as it mounts.**
-- **Video:** a paused iOS video stays paused and its late callbacks no longer reach a freed texture; Android play and pause keep their order.
-- **iOS:** ViroKit targets iOS 15.1, and builds for the iOS Simulator (arm64 and x86_64) as well as the device. The copy into viro now makes `ViroKit.xcframework`, replacing only the slice just built. `ios/build_ios_simulator_libs.sh` adds the simulator builds of LinearMath, protobuf-lite and GVR Audio that ViroKit links.
-- **Removed:** the macOS target, which no longer linked and nothing built, and dead code (an unused metrics client, the API key validator, the OpenCV headers, an old OpenXR loader).
+- **A metallic model without a `lightingEnvironment` looks darker than in 3.1.0.** This is how glTF and three.js light it; give metallic models an environment map.
 
 See `CHANGELOG.md` for every change.

ViroRenderer/Shaders.metal

@@ -1440,6 +1440,8 @@ typedef struct {
     float3 normal;
 
     float3 ambient_color;
+    // The untransformed texture coordinates, for a map with its own transform (emissive).
+    float2 raw_texcoord;
     // Particle instancing: which instance this vertex belongs to, so the fragment
     // stage can index particle_colors. Flat — an instance index must not interpolate.
     uint   instance_id [[ flat ]];
@@ -1490,6 +1492,7 @@ vertex VROPBRLightingVertexOut pbr_lighting_vertex(VRORendererAttributes attribu
     out.surface_position = v_position;
     out.camera_position = view.camera_position;
     out.normal = normalize((_transforms.normal_matrix * float4(_geometry.normal, 0.0)).xyz);
+    out.raw_texcoord = _geometry.texcoord;
     out.ambient_color = lighting.ambient_light_color;
 
     return out;
@@ -1498,6 +1501,11 @@ vertex VROPBRLightingVertexOut pbr_lighting_vertex(VRORendererAttributes attribu
 // Shared body: `albedo` already carries the material colour (from a uniform or a texture).
 static float4 vro_pbr_shade(VROPBRLightingVertexOut in,
                             float4 albedo,
+                            float3 normal,
+                            float metalness,
+                            float roughness,
+                            float ao,
+                            float3 emissive,
                             constant VROMaterialUniforms &material,
                             constant VROSceneLightingUniforms &lighting,
                             depth2d_array<float> shadow_map,
@@ -1511,11 +1519,11 @@ static float4 vro_pbr_shade(VROPBRLightingVertexOut in,
     _surface.specular_texcoord = in.texcoord;
     _surface.shininess = material.shininess;
     // Clamped: a roughness of zero makes the GGX denominator collapse.
-    _surface.roughness = max(material.roughness, 0.06);
-    _surface.metalness = material.metalness;
-    _surface.ao = material.ao;
+    _surface.roughness = max(roughness, 0.06);
+    _surface.metalness = metalness;
+    _surface.ao = ao;
     _surface.alpha = material.alpha;
-    _surface.normal = normalize(in.normal);
+    _surface.normal = normalize(normal);
     _surface.position = in.surface_position;
     _surface.view = normalize(in.camera_position - in.surface_position);
 
@@ -1579,10 +1587,12 @@ static float4 vro_pbr_shade(VROPBRLightingVertexOut in,
 
         ambient = (kD * diffuse_ibl + specular_ibl) * _surface.ao;
     } else {
-        ambient = in.ambient_color * base_color * _surface.ao;
+        // A metal has no diffuse response, and without an environment map nothing to
+        // reflect: only the diffusing share of the surface takes the flat ambient.
+        ambient = in.ambient_color * base_color * (1.0 - _surface.metalness) * _surface.ao;
     }
 
-    return float4(ambient + radiance_out + material.emissive_color, _surface.alpha * _surface.diffuse_color.a);
+    return float4(ambient + radiance_out + emissive, _surface.alpha * _surface.diffuse_color.a);
 }
 
 fragment VROLightingFragmentOut pbr_lighting_fragment_c(VROPBRLightingVertexOut in [[ stage_in ]],
@@ -1594,7 +1604,8 @@ fragment VROLightingFragmentOut pbr_lighting_fragment_c(VROPBRLightingVertexOut
                                              texturecube<float> irradiance_map [[ texture(5) ]],
                                              texturecube<float> prefiltered_map [[ texture(6) ]],
                                              texture2d<float> brdf_map [[ texture(7) ]]) {
-    const float4 color = vro_pbr_shade(in, material.diffuse_surface_color, material, lighting, shadow_map,
+    const float4 color = vro_pbr_shade(in, material.diffuse_surface_color, in.normal, material.metalness,
+                                      material.roughness, material.ao, material.emissive_color, material, lighting, shadow_map,
                                       irradiance_map, prefiltered_map, brdf_map);
     return VROMakeLightingOut(color, material);
 }
@@ -1610,11 +1621,79 @@ fragment VROLightingFragmentOut pbr_lighting_fragment_t(VROPBRLightingVertexOut
                                              texturecube<float> prefiltered_map [[ texture(6) ]],
                                              texture2d<float> brdf_map [[ texture(7) ]]) {
     const float4 albedo = diffuse_texture.sample(s, in.texcoord) * material.diffuse_surface_color;
-    const float4 color = vro_pbr_shade(in, albedo, material, lighting, shadow_map,
+    const float4 color = vro_pbr_shade(in, albedo, in.normal, material.metalness, material.roughness,
+                                      material.ao, material.emissive_color, material, lighting, shadow_map,
                                       irradiance_map, prefiltered_map, brdf_map);
     return VROMakeLightingOut(color, material);
 }
 
+// Tangent frame from screen-space derivatives (no tangent attribute needed): T and B
+// follow +u and +v on the surface. glTF normal maps put +Y toward -v (up in the image),
+// so the map's green is flipped before it is applied.
+static float3x3 vro_cotangent_frame(float3 N, float3 p, float2 uv) {
+    const float3 dp1 = dfdx(p);
+    const float3 dp2 = dfdy(p);
+    const float2 duv1 = dfdx(uv);
+    const float2 duv2 = dfdy(uv);
+    const float3 dp2perp = cross(dp2, N);
+    const float3 dp1perp = cross(N, dp1);
+    const float3 T = dp2perp * duv1.x + dp1perp * duv2.x;
+    const float3 B = dp2perp * duv1.y + dp1perp * duv2.y;
+    const float invmax = rsqrt(max(max(dot(T, T), dot(B, B)), 1e-12));
+    return float3x3(T * invmax, B * invmax, N);
+}
+
+// PBR with glTF's maps: base color, normal (scaled), metallic-roughness (B and G, by the
+// factors), occlusion (R, by its strength) and emissive (by the emissive color, through
+// its own texture transform). A map that is absent is bound to the blank texture and
+// skipped through material.pbr_maps.
+fragment VROLightingFragmentOut pbr_lighting_fragment_maps(VROPBRLightingVertexOut in [[ stage_in ]],
+                                             texture2d<float> diffuse_texture [[ texture(0) ]],
+                                             texture2d<float> normal_map [[ texture(8) ]],
+                                             texture2d<float> metal_rough_map [[ texture(9) ]],
+                                             texture2d<float> ao_map [[ texture(10) ]],
+                                             texture2d<float> emissive_map [[ texture(11) ]],
+                                             constant VROMaterialUniforms &material [[ buffer(2) ]],
+                                             constant VROCustomUniforms &_custom [[ buffer(3) ]],
+                                             constant VROSceneLightingUniforms &lighting [[ buffer(4) ]],
+                                             constant float4 *particle_colors [[ buffer(7) ]],
+                                             depth2d_array<float> shadow_map [[ texture(4) ]],
+                                             texturecube<float> irradiance_map [[ texture(5) ]],
+                                             texturecube<float> prefiltered_map [[ texture(6) ]],
+                                             texture2d<float> brdf_map [[ texture(7) ]]) {
+    const int maps = material.pbr_maps;
+    float4 albedo = material.diffuse_surface_color;
+    if (maps & VRO_PBR_MAP_BASE) {
+        albedo *= diffuse_texture.sample(s, in.texcoord);
+    }
+    float3 N = normalize(in.normal);
+    if (maps & VRO_PBR_MAP_NORMAL) {
+        float3 t = normal_map.sample(s, in.texcoord).xyz * 2.0 - 1.0;
+        t.xy *= material.normal_scale;
+        t.y = -t.y;
+        N = normalize(vro_cotangent_frame(N, in.surface_position, in.texcoord) * t);
+    }
+    float metalness = material.metalness;
+    float roughness = material.roughness;
+    if (maps & VRO_PBR_MAP_METAL_ROUGH) {
+        const float4 mr = metal_rough_map.sample(s, in.texcoord);
+        metalness = mr.b * material.metalness_intensity;
+        roughness = mr.g * material.roughness_intensity;
+    }
+    float ao = material.ao;
+    if (maps & VRO_PBR_MAP_AO) {
+        ao = mix(1.0, ao_map.sample(s, in.texcoord).r, material.ao_strength);
+    }
+    float3 emissive = material.emissive_color;
+    if (maps & VRO_PBR_MAP_EMISSIVE) {
+        const float2 uv = (material.emissive_texture_transform * float4(in.raw_texcoord, 0.0, 1.0)).xy;
+        emissive *= emissive_map.sample(s, uv).rgb;
+    }
+    const float4 color = vro_pbr_shade(in, albedo, N, metalness, roughness, ao, emissive, material, lighting,
+                                      shadow_map, irradiance_map, prefiltered_map, brdf_map);
+    return VROMakeLightingOut(color, material);
+}
+
 vertex VRODistortionVertexOut distortion_vertex(VRODistortionAttributes attributes [[ stage_in ]],
                                                 constant VRODistortionUniforms &uniforms [[ buffer(1) ]]) {