Release 3.1.0

.github/dependabot.yml

@@ -7,12 +7,12 @@ version: 2
 updates:
   - package-ecosystem: "gradle"
     directory: android
-    target-branch: "main"
+    target-branch: "develop"
     schedule:
       interval: "daily"
 
   - package-ecosystem: "github-actions"
     directory: "/" # Location of package manifests
-    target-branch: "main"
+    target-branch: "develop"
     schedule:
       interval: "weekly"

.github/workflows/pull-request.yml

@@ -71,6 +71,9 @@ jobs:
       - uses: maxim-lobanov/setup-xcode@v1.6.0
         with:
           xcode-version: ${{ matrix.xcode }}
+      - name: World mesh fusion test
+        run: ./scripts/run-world-mesh-test.sh
+        shell: bash
       - name: Install pod dependencies
         run: |
           cd ios
@@ -83,8 +86,15 @@ jobs:
             -workspace ViroRenderer.xcworkspace \
             -scheme ViroKit \
             -sdk iphoneos \
-            -configuration Release \
-            IPHONEOS_DEPLOYMENT_TARGET=13.0 | xcpretty
+            -configuration Release | tee xcodebuild.log | xcpretty
+
+      # xcpretty hides what the compiler said when a step dies without a diagnostic it recognises
+      - name: Upload the raw xcodebuild log
+        if: failure()
+        uses: actions/upload-artifact@v4.6.0
+        with:
+          name: ios_xcodebuild.log
+          path: ./ios/xcodebuild.log
 
       - name: Upload iOS Renderer artifact
         uses: actions/upload-artifact@v4.6.0

.github/workflows/release.yml

@@ -54,10 +54,11 @@ jobs:
           generate_release_notes: true
           prerelease: ${{ env.PRE_RELEASE }}
           name: ${{steps.tagger.outputs.tag}}
+          # the build runs in ./android, but these paths are from the repository root; only what
+          # :viroreact:assembleRelease builds (it also assembles :viroar), and a missing one fails the release
+          fail_on_unmatched_files: true
           files: |
-            ./viroreact/build/outputs/aar/viroreact-release.aar
-            ./viroar/build/outputs/aar/viroar-release.aar
-            ./virocore/build/outputs/aar/virocore-release.aar
-            ./sharedCode/build/outputs/aar/sharedCode-release.aar
+            ./android/viroreact/build/outputs/aar/viroreact-release.aar
+            ./android/viroar/build/outputs/aar/viroar-release.aar
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.gitignore

@@ -8,8 +8,6 @@
 .DS_Store
 
 # Macos
-/macos/ViroRenderer.xcodeproj/xcuserdata/*
-/macos/ViroRenderer.xcodeproj/project.xcworkspace/xcuserdata/**
 
 # WASM
 /wasm/products/

RELEASE_NOTES.md

@@ -1,25 +1,45 @@
 # Release Notes
 
-## v3.0.3
+## v3.1.0
 
-A patch release for Meta Quest, iOS and the physics and recording paths. The native build ships inside `@reactvision/react-viro` 3.0.3. It links ReactVisionCCA 1.3.1, which pins platform requests to the database's region.
+**Advanced VPS, a world mesh that persists, device depth in AR recordings, and a Meta Quest pass with hand occlusion.** The native build ships inside `@reactvision/react-viro` 3.1.0. It links Spatial 1.4.0, which adds continuous localisation against a scanned place and the VPS fuser.
 
-### Meta Quest
+### Advanced VPS
 
-- **A scene opened after another AR scene gets the room's planes.** The session published each plane only to the scene attached when it was first found, so a plane scene opened from an AR scene never fired `onAnchorFound`. A new scene now receives every plane found so far.
-- **The boundary is hidden while passthrough is on**, through `XR_META_boundary_visibility`, and comes back for a fully virtual scene. The app has to declare `com.oculus.permission.BOUNDARY_VISIBILITY`; the viro Expo plugin does.
+- **VPS map localisation in the AR session.** `VROARSession` gains map load, unload and localisation, implemented on ARKit and ARCore, and exposed on `ARScene` and through JNI. A loaded `.rvmap` is matched against camera frames while the session runs.
+- **Server mode.** The engine hands frames out for the server to localise and feeds the answers into the same fuser, without loading a map (`rvStartVPSServerSession`, `rvReportVPSServerFix`, `rvStopVPSServerSession`, `VROARCamera::getImageTransform()`).
+- **Every fix goes through Spatial's `VROVPSLocalizer`,** which keeps one smoothed map-to-world transform. `VROVPSAnchorNode` keeps content placed in map coordinates following it, and `VROARScene::setVPSMapMesh()` loads the place's mesh as occlusion and a kinematic physics body tagged `"world"`.
+- **Recordings carry GPS and a camera-forward heading** on each pose, which ViroScan turns into the map's geo-registration.
 
-### iOS
+### World mesh
 
-- **A session-only app no longer asks for location** when it opens an AR scene. Core Location starts only when `RVApiKey` and `RVProjectId` are set, as before 3.0.2. An anchor hosted on a session records no GPS fix.
-- **Camera texture recordings carry audio.** The microphone is live only while recording.
+- **Depth frames fuse into a persistent room (`VROTSDFVolume`, `VROARWorldMesh`)** on the depth-image and monocular paths, re-meshing only the blocks that changed, within a memory ceiling. Fusion pauses while tracking is limited, a depth drop-out keeps the room instead of falling back to plane polygons, and `resetAccumulation()` starts a new one.
+- **The mesh reports its source** (`lidar`, `depth`, `monocular`, `plane`) and whether it is accumulated.
+- **Android: the depth mesh is geometrically correct** and no longer depends on acquiring a CPU camera image.
+- **Thread safety:** the Android snapshot no longer reads a mesh the render thread is replacing, a reset is not undone by a collision shape still being built, `voxelSize` and `maxMemoryMB` take effect mid-session, and iOS monocular depth is published whole with the pose it was inferred at.
 
-### Android
+### Meta Quest
 
-- **`startRecording` without `RECORD_AUDIO` fails through its callback** instead of leaving the caller waiting forever, and camera-open and device errors reach it too.
+- **Input:** the left Menu button and the palm pinch report `MenuButton`, Y goes back as B does, a menu or system pinch never ends as a select, released hands release, and a hand and a controller on one side press once. `getSourceRay` / `getControllerRayAsync` report where a ray points, and `setInputFocusListener` / `setRoomMoveListener` report focus changes and recentres.
+- **Rendering:** the right eye clears its depth buffer, each eye's framebuffer is made once and its depth discarded after drawing, the aim laser stays on top of panels, and switching AR scenes or closing one logs no errors.
+- **Room planes** stay on the real surface across a recentre, are asked for at once, face the right way, and a desk is a table.
+- **Hand occlusion in passthrough (optional, off by default):** the tracked hands are drawn into each eye's depth buffer before the scene, so the real hands show through virtual content in front of them. It uses the runtime's hand mesh, or capsules between the joints, on Quest 2, 3, 3S and Pro.
+- **Stability:** videos no longer abort the app or leak JNI references, hit tests deliver their results without leaking, and `recenterTracking()` no longer races the render thread.
 
 ### Everywhere
 
-- **`onCollision` fires against the AR world mesh**, with the mesh's `collisionTag`. Ray and hit tests skip the mesh as before.
-- **A session can carry a function region** (`setSession(..., functionRegion)` on Android, `+setStudioSessionBaseUrl:accessToken:clientTag:functionRegion:` on iOS), which the cloud-anchor providers send as `x-region`. The three-argument forms still work.
-- **`VROGeospatial.h` is one header again**: both copies call the earth-tracking state `Enabled`. Nothing changes at runtime.
+- **AR recordings can carry the device's depth:** LiDAR scene depth on iOS and the ARCore Depth API on Android, as 16-bit millimetre PNGs with confidence, about 10 a second, when the session is set to capture depth. ViroScan fuses them into the scanned place's mesh, which is what makes blank walls reconstruct.
+- **Physics runs at real time on every display,** in fixed 1/60 s substeps, instead of 1.5× fast at 90 Hz and 2× at 120 Hz.
+- **ARCore reports the ambient light again,** and a paused AR session stays paused when an option or an image or object target changes.
+- **visionOS:** spot light cones match the GL backends, and morph targets blend on the CPU.
+- **Web:** vertex colours show, as on the native backends.
+- **Co-location:** `VROColocationSession::deviceId()`, sent as `x-rv-device`, so the relay bills a device once.
+- **A click goes to the node drawn on top,** and a click in open space reaches the controller's own delegates again.
+- **Text sits where its vertical alignment puts it,** and a back-culled draw after a front-culled one culls back faces again.
+- **GLB characters with one skin per mesh animate every part,** not only the first skin's.
+- **Android: a `ViroARCamera` no longer crashes the scene as it mounts.**
+- **Video:** a paused iOS video stays paused and its late callbacks no longer reach a freed texture; Android play and pause keep their order.
+- **iOS:** ViroKit targets iOS 15.1, and builds for the iOS Simulator (arm64 and x86_64) as well as the device. The copy into viro now makes `ViroKit.xcframework`, replacing only the slice just built. `ios/build_ios_simulator_libs.sh` adds the simulator builds of LinearMath, protobuf-lite and GVR Audio that ViroKit links.
+- **Removed:** the macOS target, which no longer linked and nothing built, and dead code (an unused metrics client, the API key validator, the OpenCV headers, an old OpenXR loader).
+
+See `CHANGELOG.md` for every change.

ViroRenderer/Shaders.metal

@@ -91,12 +91,12 @@ float compute_attenuation(constant VROLightUniforms &light,
         attenuation = 1.0 - pow(d, 1.0 / light.attenuation_falloff_exp);
 
         // Spot light
+        // spot_inner/outer_angle are the cosines of the half angles: full light inside the inner
+        // cone, none outside the outer one, as compute_angle_attenuation does in GL
         if (light.type == 3) {
-            float light_surface_angle = acos(dot(*surface_to_light, normalize(light.direction)));
-            if (light_surface_angle > light.spot_inner_angle) {
-                float t = (light_surface_angle - light.spot_inner_angle) / light.spot_outer_angle;
-                attenuation = clamp(mix(attenuation, 0.0, t), 0.0, 1.0);
-            }
+            float theta = dot(*surface_to_light, normalize(light.direction));
+            float epsilon = max(0.0001, light.spot_inner_angle - light.spot_outer_angle);
+            attenuation *= clamp((theta - light.spot_outer_angle) / epsilon, 0.0, 1.0);
         }
     }
 
@@ -1418,11 +1418,9 @@ static float vro_attenuation_pbr(constant VROLightUniforms &light, float3 surfac
 
     // Spot cone
     if (light.type == 3) {
-        const float angle = acos(clamp(dot(-*L, normalize(light.direction)), -1.0, 1.0));
-        if (angle > light.spot_inner_angle) {
-            const float t = (angle - light.spot_inner_angle) / max(light.spot_outer_angle, 1e-3);
-            attenuation = mix(attenuation, 0.0, clamp(t, 0.0, 1.0));
-        }
+        const float theta = dot(-*L, normalize(light.direction));
+        const float epsilon = max(0.0001, light.spot_inner_angle - light.spot_outer_angle);
+        attenuation *= clamp((theta - light.spot_outer_angle) / epsilon, 0.0, 1.0);
     }
     return attenuation;
 }

ViroRenderer/VROARCamera.h

@@ -28,10 +28,10 @@
 #define VROARCamera_h
 
 #include <stdio.h>
+#include "VROMatrix4f.h"
+#include "VROVector3f.h"
 
 class VROViewport;
-class VROMatrix4f;
-class VROVector3f;
 class VROFieldOfView;
 
 enum class VROARTrackingState {
@@ -75,6 +75,22 @@ class VROARCamera {
      */
     virtual VROMatrix4f getRotation() const = 0;
     virtual VROVector3f getPosition() const = 0;
+
+    /*
+     The camera's pose in world space (GL camera axes) for the camera image as
+     the sensor delivers it, not rotated to the display: what a pose solved
+     from that image, such as a VPS server fix, must be paired with. On a
+     portrait phone getRotation() is rolled 90 degrees from it. The default is
+     getRotation() and getPosition(), for platforms where the two agree.
+     */
+    virtual VROMatrix4f getImageTransform() const {
+        VROMatrix4f t = getRotation();
+        VROVector3f p = getPosition();
+        t[12] = p.x;
+        t[13] = p.y;
+        t[14] = p.z;
+        return t;
+    }
 
     /*
      Get the projection matrix needed to render a 3D scene to match the image