Release 3.1.0: glTF support

CHANGELOG.md

@@ -19,6 +19,11 @@
 - **`VROARSession::stopRecording` takes a completion receiving a `VROARRecordingResult`** (`success`, `error`, `durationSec`, `frames`, `depthFrames`, `gpsFixes`, `bytes`), called once the files are complete. The no-argument form remains. On Android, `ARScene.stopRecording()` returns an `ARScene.RecordingResult` with the same fields.
 - **`VROARSession::isRecordingDepthSupported()` and `ARScene.isRecordingSupported(boolean depth)`.** iOS checks for LiDAR scene depth and ARCore for the Depth API. On Android `ARScene.isRecordingSupported` answers false, with or without depth, until the ARCore session exists.
 
+- **glTF `KHR_texture_transform` and texture coordinate sets (`VROGLTFLoader`, `VROMaterialVisual`, `VROShaderFactory`, `VROMaterialShaderBinding`, `Shaders.metal`).** A texture's offset, rotation and scale apply when the material is drawn, and a texture can sample `TEXCOORD_1` or a later set (`texCoord`, which the extension may override). The base colour texture decides for the whole material, else its first other texture: Viro samples every texture of a material with one set of coordinates through one transform. `VROMaterialVisual::setContentsTransform` sets the transform, and the GL shaders gain a `material_texture_transform` uniform; on Metal it is applied in the vertex stage.
+- **glTF `KHR_node_visibility` (`VROGLTFLoader`).** A node with `visible: false` loads hidden, with all its descendants, as the extension asks.
+- **glTF `KHR_animation_pointer` (`VROGLTFLoader`).** An animation can drive a node's visibility, a material's `baseColorFactor` and `emissiveFactor`, and its base colour texture's offset, rotation and scale; each glTF animation's pointer channels play under its name, alongside its node and skeletal channels. A pointer at a node's translation, rotation, scale or weights is read as an ordinary channel. Other pointers are skipped with a warning naming them. `VROMaterialVisual::setColorImmediate` sets a colour without the fade or the substrate rebuild `setColor` does.
+- **Emissive textures (`VROShaderFactory`, `standard_fsh.glsl`).** A material's emission texture scales its emissive colour, through its own `KHR_texture_transform`; it was bound but no shader sampled it. Not yet on visionOS, whose Metal path draws emission as a colour only.
+
 ### Changed
 
 - **Quest: the left Menu button and the palm menu pinch report `MenuButton` and no longer go back (`VROInputControllerOpenXR`).** Meta's guidance gives the Menu button, and the palm-up pinch that stands in for it with hands, to an app's own menu, and B to going back. All three reported `BackButton` and ran the back callback, so the Menu button and the pinch ended a scene. The Menu button and the pinch now report the new `ViroOculus::MenuButton` (13) to the controller's delegates only, so it never clicks or drags a node, and the back callback no longer runs for them. The pinch is read from either hand, since the runtime sets `XR_HAND_TRACKING_AIM_MENU_PRESSED_BIT_FB` on whichever hand is not dominant; only the left hand was read before, so a left-handed wearer's pinch did nothing. B is unchanged. Pressing the Menu button now gives the left controller the short click pulse the other buttons give, which it did not get while it reported `BackButton`. An app that relied on the Menu button ending a scene has to handle `MenuButton` itself.
@@ -36,6 +41,16 @@
 
 ### Fixed
 
+- **A GLB with an animation channel that targets no node loads (`tiny_gltf.h`).** The parser required `node` on every channel target, which glTF 2.0 does not, so a file using `KHR_animation_pointer` failed to load at all. Animations made only of such channels no longer cause the skeletal animations of later ones to be skipped.
+- **A glTF without meshes or scenes loads (`VROGLTFLoader`).** The parser required scenes, nodes, accessors, buffers and buffer views, though glTF 2.0 requires only `asset`, so a file holding only a camera, or nothing yet, failed to load.
+- **Emission shows on physically based materials (`VROShaderFactory`, `Shaders.metal`).** The PBR ambient modifiers rebuilt the output colour without the emissive term, so a glTF `emissiveFactor` (and an emissive colour set on any PBR material) never showed; visionOS's PBR path had no emissive term at all.
+- **glTF `alphaMode` OPAQUE is opaque (`VROGLTFLoader`, `VROMaterial`, `VROShaderFactory`, `Shaders.metal`).** The loader mapped it to `RGBZero`, which nothing implements, and every material blends, so the alpha of a base colour texture or of vertex colours (which often hold other data) made opaque meshes see-through. `VROMaterial::setIgnoresDiffuseAlpha` drops that alpha; only the material's transparency and the node's opacity remain, so fades still work. MASK materials use it too: alpha decides only what is discarded, and what passes is opaque.
+- **glTF BLEND materials no longer write depth (`VROGLTFLoader`).** A blended surface wrote depth like an opaque one, so a nearly transparent decal hid the mesh drawn after it behind it (whole ears gone on a character with spot decals). three.js, Babylon and Filament do the same.
+- **Skins animated through nodes between their joints move (`VROGLTFLoader`).** A joint's parent was looked for only among its node's direct parent, and an animation counted as skeletal only when it targeted a joint, so a rig with an animated node above each joint (`Waist_tr` → `Waist`) had a broken bone tree and played in the bind pose. A joint's world transform is now the product of every node in its chain, animated or at rest, short of the skinned mesh's own ancestors, and an animation of any of those nodes drives the skin.
+- **An animation sampler without `interpolation` loads (`tiny_gltf.h`).** It is optional in glTF 2.0 and means LINEAR; Khronos's Fox sample failed to load without it.
+- **A skinned mesh is no longer culled while in view (`VRONode`).** The frustum test used its bind-pose bounds, which say nothing of where the bones draw it, so a model posed away from its bind pose lost parts as the camera came close (a character's eyes). A subtree holding a skinned geometry is never culled whole.
+- **A glTF with 8-bit indices draws (`VROGLTFLoader`).** The GL substrate drew a 1-byte index buffer as 32-bit indices, reading past its end, and Metal has no 8-bit index type; the loader now widens them to 16 bits.
+- **Web: skinned models draw (`VROGeometrySubstrateOpenGL`, `standard_vsh.glsl`).** The vertex shader took bone indices as `ivec4` while glTF's 8- and 16-bit joints were bound unsigned, which WebGL refuses (desktop and mobile GL let it pass). The shaders now take `uvec4`, and every bone index source is bound unsigned.
 - **Android: a world mesh snapshot no longer reads a mesh the render thread is replacing (`VROARWorldMesh`).** `rvSnapshotWorldMesh` serialises the current mesh on the UI thread while the render thread replaces it every update; a fused room takes long enough to serialise for the old mesh to be freed underneath. The current mesh now has its own lock and the snapshot holds its own reference.
 - **A reset is not undone by a collision shape still being built (`VROARWorldMesh`).** The shape is built on a background thread and installed later; one started before `resetAccumulation()` put the cleared room back as an invisible collider. A reset, a disable or a newer mesh now drops any build in flight, which also stops two overlapping builds from leaving the newer body out of the physics world.
 - **`voxelSize` and `maxMemoryMB` take effect after the first fused frame (`VROARWorldMesh`).** The volume was built once, at the first frame, so later values were ignored; a change now starts a new room at the new values. The fusion task no longer reads the config while the render thread writes it, and a frame from before a reset no longer lands in the new scan.

RELEASE_NOTES.md

@@ -37,6 +37,7 @@
 - **A click goes to the node drawn on top,** and a click in open space reaches the controller's own delegates again.
 - **Text sits where its vertical alignment puts it,** and a back-culled draw after a front-culled one culls back faces again.
 - **GLB characters with one skin per mesh animate every part,** not only the first skin's.
+- **glTF:** `KHR_texture_transform`, `KHR_node_visibility` and `KHR_animation_pointer` (node visibility, base colour and emissive factors, texture offset, rotation and scale), textures on `TEXCOORD_1` and later sets, and emissive textures. Opaque and masked materials ignore their textures' alpha, blended ones stop writing depth, skins animated through nodes between their joints move and are no longer culled by their bind-pose bounds, samplers without `interpolation` load, files with animation pointers no longer fail to load, emission shows on physically based materials, 8-bit index buffers draw, and skinned models draw on the web.
 - **Android: a `ViroARCamera` no longer crashes the scene as it mounts.**
 - **Video:** a paused iOS video stays paused and its late callbacks no longer reach a freed texture; Android play and pause keep their order.
 - **iOS:** ViroKit targets iOS 15.1, and builds for the iOS Simulator (arm64 and x86_64) as well as the device. The copy into viro now makes `ViroKit.xcframework`, replacing only the slice just built. `ios/build_ios_simulator_libs.sh` adds the simulator builds of LinearMath, protobuf-lite and GVR Audio that ViroKit links.

ViroRenderer/Shaders.metal

@@ -144,6 +144,11 @@ struct VROLightingFragmentOut {
 static VROLightingFragmentOut VROMakeLightingOut(float4 color,
                                                 constant VROMaterialUniforms &material) {
     VROLightingFragmentOut out;
+    // glTF alphaMode OPAQUE: only the material's transparency and the node's
+    // opacity remain, not the diffuse color's, texture's or vertex colors' alpha.
+    if (material.ignores_diffuse_alpha != 0) {
+        color.a = material.alpha;
+    }
     out.color = color;
     if (kVROHasToneMappingMask) {
         out.tone_mask = float4(material.tone_mapping_mask);
@@ -271,7 +276,7 @@ vertex VROConstantLightingVertexOut constant_lighting_vertex(VRORendererAttribut
 
     out.position = _vertex.position;
     out.instance_id = v_instance_id;
-    out.texcoord = _geometry.texcoord;
+    out.texcoord = (material.texture_transform * float4(_geometry.texcoord, 0.0, 1.0)).xy;
     out.ambient_color = lighting.ambient_light_color;
     out.material_color = material.diffuse_surface_color;
     out.diffuse_intensity = material.diffuse_intensity;
@@ -440,7 +445,7 @@ vertex VROLambertLightingVertexOut lambert_lighting_vertex(VRORendererAttributes
 
     out.position = _vertex.position;
     out.instance_id = v_instance_id;
-    out.texcoord = _geometry.texcoord;
+    out.texcoord = (material.texture_transform * float4(_geometry.texcoord, 0.0, 1.0)).xy;
     out.color = _geometry.color;
 
     out.surface_position = v_position;
@@ -721,7 +726,7 @@ vertex VROPhongLightingVertexOut phong_lighting_vertex(VRORendererAttributes att
 
     out.position = _vertex.position;
     out.instance_id = v_instance_id;
-    out.texcoord = _geometry.texcoord;
+    out.texcoord = (material.texture_transform * float4(_geometry.texcoord, 0.0, 1.0)).xy;
     out.color = _geometry.color;
 
     out.surface_position = v_position;
@@ -1049,7 +1054,7 @@ vertex VROBlinnLightingVertexOut blinn_lighting_vertex(VRORendererAttributes att
 
     out.position = _vertex.position;
     out.instance_id = v_instance_id;
-    out.texcoord = _geometry.texcoord;
+    out.texcoord = (material.texture_transform * float4(_geometry.texcoord, 0.0, 1.0)).xy;
     out.color = _geometry.color;
 
     out.surface_position = v_position;
@@ -1480,7 +1485,7 @@ vertex VROPBRLightingVertexOut pbr_lighting_vertex(VRORendererAttributes attribu
 
     out.position = _vertex.position;
     out.instance_id = v_instance_id;
-    out.texcoord = _geometry.texcoord;
+    out.texcoord = (material.texture_transform * float4(_geometry.texcoord, 0.0, 1.0)).xy;
     out.color = _geometry.color;
     out.surface_position = v_position;
     out.camera_position = view.camera_position;
@@ -1577,7 +1582,7 @@ static float4 vro_pbr_shade(VROPBRLightingVertexOut in,
         ambient = in.ambient_color * base_color * _surface.ao;
     }
 
-    return float4(ambient + radiance_out, _surface.alpha * _surface.diffuse_color.a);
+    return float4(ambient + radiance_out + material.emissive_color, _surface.alpha * _surface.diffuse_color.a);
 }
 
 fragment VROLightingFragmentOut pbr_lighting_fragment_c(VROPBRLightingVertexOut in [[ stage_in ]],