[TeleViz] Add XR session and xr compositing utils

[farbod-nv]

Adds XR session, depth submission, XR compositing
And utils for head_pose, xr timing and etc.
Also adds an example for viz_xr smoke test

Summary by CodeRabbit

Release Notes

• New Features
  • Added OpenXR support for developing and running VR/XR applications
  • Integrated GPU timing metrics for frame rendering performance profiling
  • New XR demonstration example showcasing stereo 3D quad rendering
  • Enhanced layer placement control for positioned virtual objects in XR space

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 74752066-c4ef-4083-aa9b-a3895a1fe0b8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fm/televiz_m5

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

• Generate code and open pull requests
• Plan features and break down work
• Investigate incidents and troubleshoot customer tickets together
• Automate recurring tasks and respond to alerts with triggers
• Summarize progress and report instantly

Built for teams:

• Shared memory across your entire org—no repeating context
• Per-thread sandboxes to safely plan and execute work
• Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 10

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/viz/session_tests/cpp/test_viz_session.cpp (1)

83-90: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

This test asserts the pre-XR contract.

This PR also adds examples/televiz/xr_smoke/main.cpp, which creates a VizSession in DisplayMode::kXr. Keeping a unit test that expects VizSession::create(cfg_xr) to throw "not yet implemented" will either fail on XR-capable hosts or force the implementation back to the old behavior.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/viz/session_tests/cpp/test_viz_session.cpp` around lines 83 - 90, The
unit test TEST_CASE "VizSession::create rejects kXr (not yet implemented)" is
now outdated because examples/televiz/xr_smoke/main.cpp creates a VizSession
with DisplayMode::kXr; remove or update this test so it no longer asserts
VizSession::create(cfg_xr) throws. Locate the test in
src/viz/session_tests/cpp/test_viz_session.cpp (the TEST_CASE referencing
VizSession::create and DisplayMode::kXr) and either delete it or change it to
assert the new behavior (e.g., ensure mode validation happens without requiring
GPU or that create succeeds/returns the expected state) rather than expecting a
std::runtime_error.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@examples/televiz/xr_smoke/main.cpp`:
- Around line 141-154: The current catch-all around viz::VizSession::create()
masks real failures; change the error handling to only treat known "no runtime /
no HMD" conditions as success and treat any other exception as a real failure:
replace the broad catch(const std::exception& e) with either (1) catching the
specific exception type(s) thrown for runtime-unavailable (if the XR wrapper
exposes one) and return EXIT_SUCCESS there, and add a second catch(const
std::exception& e) that logs the error from viz::VizSession::create and returns
EXIT_FAILURE, or (2) if only std::exception is available, inspect e.what() for
the explicit runtime/HMD-not-available message(s) and return EXIT_SUCCESS only
in those cases, otherwise log and return EXIT_FAILURE; reference
viz::VizSession::create and the catch block to locate where to implement this
branching.

In `@src/viz/core/cpp/inc/viz/core/vk_context.hpp`:
- Line 6: The public header currently exposes OpenXR via the include and by
using XrInstance/XrSystemId in viz::VkContext::Config; remove the direct
dependency by replacing those members with opaque types (e.g., uint64_t or void*
or a small XRHandle typedef) or by moving XR-specific fields into a non-public
viz::detail/XR wrapper; update VkContext::Config to use the opaque handle type
and remove `#include` <openxr/openxr.h> from the header, and perform
OpenXR-to-opaque translations inside the viz::xr boundary (e.g., where
XrInstance/XrSystemId are known) so the public API no longer exposes OpenXR
symbols.

In `@src/viz/core/cpp/render_target.cpp`:
- Around line 298-309: The depth attachment (attachments[1] using depth_format_)
is transitioned to TRANSFER_SRC for post-pass copy but deps[1] only syncs color;
update deps[1] to include fragment test stages and depth write visibility by
adding VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT |
VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT to deps[1].srcStageMask and
VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT to deps[1].srcAccessMask so depth
writes are properly visible before the transfer.

In `@src/viz/core/cpp/vk_context.cpp`:
- Around line 194-205: In init(), validate the XR configuration as an
all-or-nothing pair before branching: check config.xr_instance and
config.xr_system_id together (e.g., if one is set and the other is not), and
reject the partial XR config by returning an error/logging and aborting the XR
path instead of calling create_instance_xr, select_physical_device_xr, or
create_logical_device_xr; ensure the check is performed in init() prior to the
existing if (config.xr_instance != nullptr) branch so only fully-populated XR
configs proceed to the XR-specific functions.
- Around line 591-600: The code sets app.apiVersion = kApiVersion without
validating against the OpenXR-provided limits in reqs obtained via
xrGetVulkanGraphicsRequirements2KHR; update the logic after obtaining
XrGraphicsRequirementsVulkan2KHR reqs to verify that kApiVersion is within
[reqs.minApiVersionSupported, reqs.maxApiVersionSupported] and if not either
clamp app.apiVersion to that range or log/return an error before calling
xrCreateVulkanInstanceKHR; reference the symbols
XrGraphicsRequirementsVulkan2KHR reqs, app.apiVersion, kApiVersion,
xrGetVulkanGraphicsRequirements2KHR and xrCreateVulkanInstanceKHR when locating
where to add the check and handling.

In `@src/viz/session_tests/cpp/test_viz_session.cpp`:
- Around line 53-68: The test currently only calls SUCCEED() so it never
exercises the XR-only guard; update it to assert that the XR-only APIs throw in
non-XR mode by keeping cfg.mode = DisplayMode::kOffscreen, then call
has_xr_time_conversion on the (uninitialized) VizSession pointer to confirm it
returns false and invoke at least one of the conversion methods that are
documented to throw on non-kXr (the conversion methods declared to throw
std::logic_error) and assert it throws (e.g. REQUIRE_THROWS_AS or
CHECK_THROWS_AS expecting std::logic_error); reference VizSession,
has_xr_time_conversion and the conversion methods in your change so the test
fails if those methods stop rejecting non-kXr modes.

In `@src/viz/session/cpp/viz_session.cpp`:
- Around line 41-42: The cast from cfg.xr_environment_blend_mode to
XrEnvironmentBlendMode via static_cast is fragile; add a compile-time parity
check and replace the blind cast with an explicit mapping: introduce a helper
like mapEnvironmentBlendMode(cfg.xr_environment_blend_mode) or
ConvertToXrEnvironmentBlendMode that uses a switch over your wrapper enum and
returns the corresponding XrEnvironmentBlendMode (with a default case that
logs/handles unexpected values), and also add static_asserts that verify the
integral values of each wrapper enumerator equal the corresponding
XrEnvironmentBlendMode enumerator to catch divergence at compile time; update
the assignment to use the mapping helper instead of static_cast (target symbols:
xc.session_config.environment_blend_mode, cfg.xr_environment_blend_mode,
XrEnvironmentBlendMode).

In `@src/viz/session/cpp/xr_backend.cpp`:
- Around line 402-418: The loop acquiring swapchain images (lambda acquire_pair
used with view_swapchains_ and depth_swapchains_) is not exception-safe: if an
acquire or wait fails mid-loop, previously acquired images remain unreleased;
modify the logic to track which swapchains were successfully acquired (e.g., a
local vector of ViewSwapchain* or handles and indices) and on any check_xr
failure immediately iterate that list and call xrReleaseSwapchainImage for each
to undo the partial acquisitions, then propagate the error or return early;
alternatively set frame_renderable_ only after all acquires succeed as now but
ensure the cleanup path calls the same release logic as abort_frame(); update
acquire_pair (or its caller) to push successful acquisitions into the tracking
list and perform the cleanup on error so no acquired image is left unreleased.

In `@src/viz/xr_tests/cpp/test_openxr_instance.cpp`:
- Around line 15-24: The catch-all of std::runtime_error around constructing
viz::OpenXrInstance masks real failures; change the handler so it only SKIPs for
the known "no runtime / no HMD" conditions and re-throws or fails for any other
runtime_error. Concretely, in the try/catch blocks around viz::OpenXrInstance
(the constructors used in the blocks at the shown locations and the other
occurrences around lines 35-42 and 73-80), inspect e.what() for the expected
unavailability substrings (e.g. "no OpenXR runtime", "no HMD" or your library's
exact message) and call SKIP(...) only when matched; otherwise rethrow the
exception or let the test framework treat it as a failure. Ensure the behavior
is applied consistently to all try/catch sites that construct OpenXrInstance so
only genuine runtime-unavailable cases are skipped.

In `@src/viz/xr_tests/cpp/test_xr_runtime.cpp`:
- Around line 9-20: The two tests "OpenXR loader is linked and queryable" and
"OpenXR loader advertises XR_KHR_vulkan_enable2" should be re-tagged from [unit]
to [xr] and must early-skip when no loader is present; update the TEST_CASE tags
to "[xr][viz_xr]" and add a guard at the start of each test using
viz::openxr_loader_available() (call viz::openxr_loader_available() and
return/skip if false) so tests only run when the OpenXR loader/runtime is
available; reference the TEST_CASEs in src/viz/xr_tests/cpp/test_xr_runtime.cpp
and use the existing viz::enumerate_openxr_instance_extensions() in the second
test without changing its logic.

---

Outside diff comments:
In `@src/viz/session_tests/cpp/test_viz_session.cpp`:
- Around line 83-90: The unit test TEST_CASE "VizSession::create rejects kXr
(not yet implemented)" is now outdated because
examples/televiz/xr_smoke/main.cpp creates a VizSession with DisplayMode::kXr;
remove or update this test so it no longer asserts VizSession::create(cfg_xr)
throws. Locate the test in src/viz/session_tests/cpp/test_viz_session.cpp (the
TEST_CASE referencing VizSession::create and DisplayMode::kXr) and either delete
it or change it to assert the new behavior (e.g., ensure mode validation happens
without requiring GPU or that create succeeds/returns the expected state) rather
than expecting a std::runtime_error.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f5a4585e-eb29-4059-9e9b-f604e99bf346

📥 Commits

Reviewing files that changed from the base of the PR and between c330462 and 61bd909.

📒 Files selected for processing (36)

• examples/televiz/CMakeLists.txt
• examples/televiz/xr_smoke/CMakeLists.txt
• examples/televiz/xr_smoke/main.cpp
• src/viz/CMakeLists.txt
• src/viz/core/cpp/CMakeLists.txt
• src/viz/core/cpp/inc/viz/core/viz_types.hpp
• src/viz/core/cpp/inc/viz/core/vk_context.hpp
• src/viz/core/cpp/render_target.cpp
• src/viz/core/cpp/vk_context.cpp
• src/viz/layers/cpp/inc/viz/layers/quad_layer.hpp
• src/viz/layers/cpp/quad_layer.cpp
• src/viz/session/cpp/CMakeLists.txt
• src/viz/session/cpp/inc/viz/session/display_backend.hpp
• src/viz/session/cpp/inc/viz/session/viz_compositor.hpp
• src/viz/session/cpp/inc/viz/session/viz_session.hpp
• src/viz/session/cpp/inc/viz/session/xr_backend.hpp
• src/viz/session/cpp/viz_compositor.cpp
• src/viz/session/cpp/viz_session.cpp
• src/viz/session/cpp/xr_backend.cpp
• src/viz/session_tests/cpp/CMakeLists.txt
• src/viz/session_tests/cpp/test_perf_audit.cpp
• src/viz/session_tests/cpp/test_viz_session.cpp
• src/viz/shaders/cpp/textured_quad.vert
• src/viz/xr/CMakeLists.txt
• src/viz/xr/cpp/CMakeLists.txt
• src/viz/xr/cpp/inc/viz/xr/openxr_instance.hpp
• src/viz/xr/cpp/inc/viz/xr/openxr_session.hpp
• src/viz/xr/cpp/inc/viz/xr/xr_runtime.hpp
• src/viz/xr/cpp/openxr_instance.cpp
• src/viz/xr/cpp/openxr_session.cpp
• src/viz/xr/cpp/xr_runtime.cpp
• src/viz/xr_tests/CMakeLists.txt
• src/viz/xr_tests/cpp/CMakeLists.txt
• src/viz/xr_tests/cpp/test_openxr_instance.cpp
• src/viz/xr_tests/cpp/test_openxr_session.cpp
• src/viz/xr_tests/cpp/test_xr_runtime.cpp

[coderabbitai]

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/viz/session_tests/cpp/test_viz_session.cpp (1)

72-78: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Test will fail when OpenXR runtime is available—add runtime detection or retag.

This [unit] test assumes the host has no OpenXR runtime and will break if one is installed (e.g., Monado for other tests). The codebase already has viz::openxr_loader_available() for detection (used throughout xr_tests/cpp/); use it here to skip gracefully when a runtime is present, or retag to [xr] and invert the test logic to REQUIRE_NOTHROW.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/viz/session_tests/cpp/test_viz_session.cpp` around lines 72 - 78, Test
assumes no OpenXR runtime; update the test around VizSession::create to detect
runtime availability using viz::openxr_loader_available() and either (a)
early-skip/return the test when a runtime is present so the existing
CHECK_THROWS_AS remains valid, or (b) retag the case from [unit] to [xr] and
invert the assertion to REQUIRE_NOTHROW when viz::openxr_loader_available() is
true; locate the test by the TEST_CASE name and the VizSession::create call to
implement the conditional behavior.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/viz/session/cpp/xr_backend.cpp`:
- Around line 120-129: Before tearing down, ensure any in-flight frame and
acquired swapchain images are cleanly ended/released: in XrBackend::destroy(),
if frame_began_ is true call xrEndFrame (with an empty XrFrameEndInfo / no
layers) using the current session handle and clear frame_began_ only after a
successful call; then iterate swapchains (the same structures
destroy_swapchains() would touch) and for any swapchain with sw.acquired set
call xrReleaseSwapchainImage on that swapchain handle and clear sw.acquired,
checking xrResult and logging errors but proceeding; only after balancing
xrEndFrame and releasing acquired images proceed to call destroy_swapchains(),
session_.reset(), and xr_instance_.reset() and set ctx_ = nullptr. Ensure you
reference and use the existing symbols frame_began_, destroy_swapchains(),
sw.acquired, xrEndFrame, and xrReleaseSwapchainImage when making the changes.

---

Outside diff comments:
In `@src/viz/session_tests/cpp/test_viz_session.cpp`:
- Around line 72-78: Test assumes no OpenXR runtime; update the test around
VizSession::create to detect runtime availability using
viz::openxr_loader_available() and either (a) early-skip/return the test when a
runtime is present so the existing CHECK_THROWS_AS remains valid, or (b) retag
the case from [unit] to [xr] and invert the assertion to REQUIRE_NOTHROW when
viz::openxr_loader_available() is true; locate the test by the TEST_CASE name
and the VizSession::create call to implement the conditional behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7d32c8ef-7b6e-4dea-a008-81ad35be46bf

📥 Commits

Reviewing files that changed from the base of the PR and between 61bd909 and e4ad4e6.

📒 Files selected for processing (32)

• examples/televiz/xr_smoke/main.cpp
• src/viz/CMakeLists.txt
• src/viz/core/cpp/CMakeLists.txt
• src/viz/core/cpp/inc/viz/core/openxr_platform_compat.hpp
• src/viz/core/cpp/inc/viz/core/vk_context.hpp
• src/viz/core/cpp/render_target.cpp
• src/viz/core/cpp/vk_context.cpp
• src/viz/layers/cpp/CMakeLists.txt
• src/viz/layers/cpp/inc/viz/layers/layer_base.hpp
• src/viz/layers/cpp/inc/viz/layers/quad_layer.hpp
• src/viz/layers/cpp/quad_layer.cpp
• src/viz/layers_tests/cpp/inc/viz/layers/testing/clear_rect_layer.hpp
• src/viz/layers_tests/cpp/inc/viz/layers/testing/throwing_layer.hpp
• src/viz/session/cpp/CMakeLists.txt
• src/viz/session/cpp/inc/viz/session/display_backend.hpp
• src/viz/session/cpp/inc/viz/session/layer_base.hpp
• src/viz/session/cpp/inc/viz/session/viz_compositor.hpp
• src/viz/session/cpp/inc/viz/session/viz_session.hpp
• src/viz/session/cpp/inc/viz/session/xr_backend.hpp
• src/viz/session/cpp/viz_compositor.cpp
• src/viz/session/cpp/viz_session.cpp
• src/viz/session/cpp/xr_backend.cpp
• src/viz/session_tests/cpp/test_perf_audit.cpp
• src/viz/session_tests/cpp/test_viz_session.cpp
• src/viz/shaders/cpp/textured_quad.vert
• src/viz/xr/cpp/inc/viz/xr/openxr_instance.hpp
• src/viz/xr/cpp/inc/viz/xr/openxr_session.hpp
• src/viz/xr/cpp/openxr_instance.cpp
• src/viz/xr/cpp/openxr_session.cpp
• src/viz/xr_tests/cpp/test_openxr_instance.cpp
• src/viz/xr_tests/cpp/test_openxr_session.cpp
• src/viz/xr_tests/cpp/test_xr_runtime.cpp

💤 Files with no reviewable changes (1)

• src/viz/layers/cpp/inc/viz/layers/layer_base.hpp

✅ Files skipped from review due to trivial changes (3)

• src/viz/layers_tests/cpp/inc/viz/layers/testing/throwing_layer.hpp
• src/viz/core/cpp/inc/viz/core/openxr_platform_compat.hpp
• src/viz/session_tests/cpp/test_perf_audit.cpp

🚧 Files skipped from review as they are similar to previous changes (9)

• src/viz/xr_tests/cpp/test_xr_runtime.cpp
• src/viz/core/cpp/render_target.cpp
• src/viz/session/cpp/CMakeLists.txt
• src/viz/session/cpp/inc/viz/session/display_backend.hpp
• src/viz/session/cpp/inc/viz/session/viz_session.hpp
• src/viz/session/cpp/viz_compositor.cpp
• src/viz/session/cpp/viz_session.cpp
• src/viz/xr/cpp/inc/viz/xr/openxr_instance.hpp
• src/viz/xr/cpp/openxr_session.cpp