Repository navigation
Release 3.1.0 - #423
Merged
Merged
Release 3.1.0#423
Conversation
Since 3.0.0 a click completes only when its ClickDown and ClickUp resolve to the same node that takes clicks. A press and release pointing at nothing clickable sent ClickDown and ClickUp but never Clicked, so ViroController's onClick stopped firing there, and Quest tap-to-place, which places on it, could not place at all. Each source's unreleased ClickDown is now tracked, and a release that still resolves to no clickable node completes as Clicked for the controller's own delegates, with no node, as before 3.0.0. Press capture on clickable nodes is unchanged.
The formatter laid out each line as if its box started at the baseline, leaving out the part below it. Centred text, the default, drew above the middle of its box by half the font's ascent less its descent, a third of the font size in Roboto; top-aligned text sat a descent below the top and bottom-aligned text hung a descent below the bottom. A line's box now runs from the descender below its baseline, so every alignment is placed inside its box. VROTypeface gains getDescender(), read from FreeType's size metrics like getLineHeight(), on Android, iOS, macOS and wasm.
The left Menu button and the palm-up menu pinch both reported BackButton and ran the back callback, so in a ViroReact app they ended the scene. Meta's guidance gives them to an app's own menu and B to going back. They now report the new ViroOculus::MenuButton (13), to the controller's delegates only, so the press never clicks or drags a node, and the back callback runs for B alone. The pinch is read from either hand: the runtime sets MENU_PRESSED on the non-dominant one, which only the left hand was checked for.
A pinch was ignored only on the frames the runtime flagged it as a system or menu gesture. The flags can clear while the fingers are still together, and on a Quest 3 the rest of a menu pinch reached the app as a select about 0.1 s after the menu event, so closing a menu with the pinch also placed or clicked what that hand pointed at. Once flagged, a pinch is now held as a system pinch until its strength falls below 0.5, well under the 0.7 that starts one, so a wavering release cannot start a select either.
Attaching an AR scene turns passthrough on, and a switch from one AR scene to another did so while it was already running. The runtime refuses a second start with XR_ERROR_UNEXPECTED_STATE_PASSTHROUGH_FB, which was logged as a failed OpenXR call each time. The start is now skipped while passthrough is on; the layer is resumed as before.
Meta's platform treats Y as a back button too, and Y did nothing an app could rely on. It still reports YButton, and now also runs the back callback. Pressing B and Y together runs it once, since a second back could close more than the wearer meant to.
Four fixes in how the pinch, the grab, the trigger and the squeeze reach a source: - A hand that stops being located mid-pinch or mid-grab (tracking lost, hand tracking turned off) now sends its ClickUp, as a trigger already does when its action goes inactive. A drag it started stayed open. - A grab made with the palm turned toward the user, or during a palm pinch, is ignored until the hand opens (read at 7 cm, 1 cm past the grab threshold), as the pinch already was. - The trigger and the pinch press one source, as do the squeeze and the grab. With a hand and a controller active on one side each sent its own ClickDown and ClickUp; now an edge is sent only while the other input is up. - The pinch read from joints, without XR_FB_hand_tracking_aim data, is made under 2 cm and released over 3 cm instead of at 2 cm both ways, and a system pinch ends at 3 cm too.
The action set was destroyed only in VROInputControllerOpenXR's destructor, and the renderer keeps the controller alive past xrDestroyInstance, so the loader logged "No active XrInstance handle" for xrDestroyActionSet on every scene close. onDestroy now destroys it with the controller's spaces, before the session and the instance, as the OpenXR reference sample does. The destructor still covers a controller that is never torn down this way.
The beam had renderingOrder 0, and the renderer sorts on renderingOrder before depth, so a panel an app draws over the scene with a higher renderingOrder and depth reads off covered it. It now has 1000000, which keeps it last.
VRODisplayOpenGLOpenXR::bind cleared colour, depth and stencil with whatever depth mask the last draw left, and glClear skips a buffer whose writes are masked. Quest renders each eye straight into its swapchain, so when the left eye ended on a material that does not write depth, such as the aim laser, the right eye drew against stale depth. The depth clear now goes through clearDepth(), which turns depth writes on through the driver first, as VRORenderTargetOpenGL::bind does.
VROInputControllerBase::hitTest returned the nearest hit, so an object between the user and a panel drawn over the scene (a higher renderingOrder with depth reads off) took clicks aimed at the panel. Hits are now taken in draw order, nearest last within one renderingOrder. A hit that reads depth is skipped behind an earlier node that writes depth, and a node with depth reads off is drawn over everything before it, with what comes after it counted as in front of it, since bounding boxes cannot place a panel's layers a centimetre apart. Within one renderingOrder the nearest hit still wins, so a scene that sets none is unaffected. A node with ignoreEventHandling neither takes nor hides a hit, as before. On a Meta Quest 3 a Studio menu line now takes a click aimed through a nearer object.
One VRODisplayOpenGLOpenXR serves both eyes, which render into different swapchain textures, and it kept a single FBO. So each eye of every frame deleted the FBO and made another, with a new full-size depth buffer. Each swapchain image now keeps its FBO for the display's life, which is the swapchains' life too, and FBOs of one size share one depth buffer, which bind() clears for every eye.
A controller click that hits nothing reports no position, and nothing else told a caller where it was aimed: Controller's forward vector is whichever source moved last, with no origin. getSourceRay returns the origin and direction onMove last reported for the ray that carries a source. Controller.getControllerRayAsync reads it on the render thread, where the poses are written, and calls back on the application thread with both, or with nulls when that ray has never moved.
Nothing reads an eye's depth and stencil after it is drawn: no depth layer goes to the compositor, and the next eye clears them. The GPU still wrote them from tile memory back to main memory at the end of every eye, 11.8 MB per eye at Quest 3's 1680 x 1760, or up to 1.7 GB/s at 72 fps. discardDepth() now invalidates them before the swapchain image is released. It is not the display's invalidate(), which the driver also calls when it switches render targets partway through an eye. On a Meta Quest 3, in the same scene at the same GPU clock, the app's GPU time per frame went from 1.38 ms to 1.21 ms.
VROAVPlayer's constructor found com/viro/core/internal/AVPlayer with plain FindClass. It runs from the renderer queue, which Quest drains on the native OpenXR render thread, and FindClass there resolves through the system class loader. It failed with a ClassNotFoundException pending, and the next JNI call aborted the app, so every scene with a video crashed. VROPlatformFindHostClass falls back to the app context's class loader, as the AR hit-test callback already does. Phones drain the queue on the GLSurfaceView thread, where FindClass already succeeds and returns the same class. On a Meta Quest 3 a scene with two videos now opens, and screenshots taken 10 s apart show both playing.
Since 12b9e3b (2018) setCullMode set glCullFace(GL_BACK) only on the way to None. Switching from Front to Back enabled culling and left the cull face at GL_FRONT, so after one front-culled draw every back-culled draw in the frame lost its front faces until something drew with None. Text glyphs are single-sided and cull Back, so all text drawn after such a material disappeared, and closed meshes drew inside out. Back now sets GL_BACK. The driver is shared by Android, iOS's OpenGL renderer and the web renderer. On a Meta Quest 3, a scene with one front-culled material among twelve now shows all twelve text labels. Before, it showed none.
Controller.setEventDelegate disposed the installed delegate before installing the one it was given, so passing the delegate already in place disposed it and handed native a null reference, which crashed in nativeSetEventDelegate. On Quest a scene mounted between the renderer's startup callback and the context hand-off that callback posts gets its context twice, and its controller did exactly this. Installing the current delegate is now a no-op. A null check in the JNI function alone would have stopped the crash but left the delegate disposed, and the input presenter holds it weakly, so the controller would have lost its input. On a Meta Quest 3, with a plane scene's change of root delayed 1 to 30 ms to land in that window, 4 of 12 opens crashed here before and 0 of 12 after. Controller input after the change was not tested.
invokeHitTestResultsCallback found com/viro/core/HitTestResult with plain FindClass on the renderer thread, the lookup f488a16 replaced in VROAVPlayer. On Quest that thread is the native OpenXR render thread, where FindClass sees only system classes, so the lookup fails with a ClassNotFoundException pending and the next JNI call aborts the app. It now uses VROPlatformFindHostClass, as the AR hit-test callback does, and reports no results if the class still cannot be found. ViroView.performSceneHitTestWithPoint and performSceneHitTestRay are the only callers, and ViroReact calls neither, so this path was not run on a device. A Meta Quest 3 build with it opens the test scenes as before.
ErnestoMendozaHernandez
marked this pull request as draft
October 30, 2026 05:00
Find Spatial in spatial/, falling back to reactvisioncca/
ViroKit links three libraries that had no arm64 simulator slice. ios/build_ios_simulator_libs.sh builds LinearMath (Bullet 2.87) and protobuf-lite 3.2.0 from source (protobuf with -Dgoogle=google_public, the namespace the device build uses) and adds the slices to the existing x86_64 archives. GVR Audio is closed source, so retag_for_simulator.py relabels its arm64 device objects as iOS Simulator objects into Libraries/GVRAudioSDK/simulator; the code is the same on both. The simulator search paths find that GVR build first, and the device libreactvisioncca.a is no longer on the unconditional search path, where the simulator link found it before its own slice.
The "Lipo Framework and Copy to Bridge" phase now builds viro/ios/dist/ViroRenderer/ViroKit.xcframework, replacing only the slice of the platform it built and keeping the other, so device and simulator builds can run in either order. It skips a simulator build lacking arm64 or x86_64, since an app's Release simulator build asks for both and CocoaPods refuses a slice missing either. The phase runs after [CP] Copy Pods Resources, so the ARCore resource bundles are in every staged copy, and takes the framework's Info.plist as an input: a clean build otherwise staged the framework before its plist existed. The Jenkins phase lipos only when the two builds share no architecture, and ViroKit.podspec vendors the xcframework.
performARHitTestWithPoint, the ray, position and Quest OpenXR hit tests and the scene hit test kept the bridge's one-shot listener only as a weak global ref. Nothing else holds that listener, so the GC could collect it before the renderer thread answered, onHitTestFinished never ran and the JS promise stayed pending. On a Pixel 6 Pro this happened about 11 s into a recording that hit-tests every 250 ms. The listener is now a global ref, deleted after the callback runs. With the rebuilt libviro_renderer.so in a host app, 120 probes all answered.
…an be recorded VROARSession::stopRecording takes a completion receiving a VROARRecordingResult (success, error, durationSec, frames, depthFrames, gpsFixes, bytes), called once video.mp4, session.jsonl and any depth files are complete; the no-argument form remains. ARScene.stopRecording() returns an ARScene.RecordingResult with the same fields. isRecordingDepthSupported() answers LiDAR scene depth on iOS and the Depth API on ARCore, and ARScene.isRecordingSupported(boolean depth) exposes it on Android. Android reports an encoder that did not finish as a failure, as iOS does. A failure while stopping goes into the result only: it used to reach the scene after the stop had reset the status, leaving getRecordingStatus() at IOError with nothing recording. ARScene.stopRecording() may run off the UI thread. On iOS the pose counters are kept under the sidecar lock, counting only lines actually written.
Feat/recording result
Feature/ios simulator
… fails VROPlatformLastPathComponent names the temporary file a download is written to, and kept the URL's query. A signed URL's token pushed the name past the 255-byte file name limit, so writeToURL failed, and both download functions ignored that and reported success with a path to a file that did not exist. The name now drops the query and fragment, a failed write makes the synchronous download report failure, and the async one logs it and calls onFailure. ViroKit is not rebuilt in this commit.
iOS: save downloads from signed URLs, and fail a download whose write…
Fix the ViroARCamera crash on Android and multi-skin GLB animation
Fix the stack overflow when drawing a long polyline
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
virocore 3.1.0: Advanced VPS in the AR session, a world mesh that fuses depth into a persistent room, and a Meta Quest input and rendering pass. Ships inside
@reactvision/react-viro3.1.0, linking Spatial 1.4.0.What's in it
Advanced VPS: VPS map localisation on ARKit and ARCore, server mode (frames out, fixes in, same fuser),
VROVPSAnchorNode,setVPSMapMesh(), and GPS plus heading on recordings.World mesh: depth frames fuse into a persistent room (
VROTSDFVolume) within a memory ceiling, remeshing only changed blocks; the mesh reports its source and whether it is accumulated;resetAccumulation()starts over. Android's depth mesh is geometrically correct.World mesh review fixes (World mesh review fixes: threads, resets, monocular pose, ARCore image size #422): the Android snapshot no longer reads a mesh being replaced, stale collision builds are dropped,
voxelSizeandmaxMemoryMBapply mid-session, iOS monocular depth is placed with the pose it was inferred at, and ARCore reads the image size from the intrinsics.Meta Quest: menu and back buttons as Horizon OS expects, hand and controller input fixes,
getControllerRayAsync, focus and recentre listeners, room planes that stay on the surface, per-eye framebuffer and depth fixes, and video and hit-test leaks closed.Co-location with user tokens (Feature/vps advanced #424):
VROColocationSession/setAccessToken:send an app user token asAuthorization: Bearerin place of the session and API key, read at every connect so a renewed token reaches the next reconnect; Android'sColocationSessiongets the same.Everywhere: clicks go to the node drawn on top, text alignment, iOS video pause, Android video ordering, co-location device id, iOS 15.1 target.
Spatial 1.4.0 prebuilts for every platform.
RELEASE_NOTES.mdis replaced with 3.1.0's;CHANGELOG.mdhas every change (duplicated world mesh entries removed).Device depth in AR recordings (iOS: record LiDAR depth in AR session recordings #437, Record ARCore depth in the Android recorder #438): LiDAR scene depth on iOS and the ARCore Depth API on Android, written as 16-bit millimetre PNGs with confidence at about 10 Hz when the session captures depth.
Quest hand occlusion in passthrough (Quest: optional hand occlusion in passthrough #425): the tracked hands are drawn depth-only before the scene, off by default.
Fixes from the parity audit (Physics steps by the real elapsed time #426–visionOS: morph targets blend on the CPU #433): physics at real time on every display, ARCore ambient light, image tracking initialised everywhere, a paused session stays paused, web vertex colours, visionOS spot cones and CPU morph targets, release workflow.
Removed (Remove the macOS target #434, Remove dead code and files #435): the macOS target, which no longer linked, and dead code (unused metrics client, API key validator, OpenCV headers, an old OpenXR loader).
Notes
Testing
scripts/run-world-mesh-test.sh: all green, including the new voxel-size and monocular-source cases.scripts/build.sh), the visionOS xcframework (with the Metal spot cone and CPU morph changes) and the web wasm build from this branch.Work by Petko, Ernesto, Benito and Eduardo.