test: Fix Redis-backed test isolation across xdist runs

[vishal-bala]

Summary

This PR reduces flaky Redis-backed service tests by fixing test isolation issues in the test suite rather than changing product code.

The failures were caused by Redis state leaking across tests during pytest -n auto runs. Several tests reused fixed Redis resource names, and some shared HNSW index fixtures created Redis indexes without deleting them. That made unrelated tests on the same xdist worker sensitive to execution order and leftover state, which explains why some matrix failures passed on retry.

Problem

The main issue was cross-test persistence in Redis:

• Shared fixtures used Redis index names and prefixes scoped only to worker_id.
• hnsw_index and async_hnsw_index created indexes but did not tear them down.
• Several integration tests created histories, routers, caches, or indexes with fixed names like test_app, float64 history, test_pass_through_dtype, and test_multi_prefix.

In CI, that meant one test could leave Redis resources behind for another test running later on the same worker. The result was intermittent failures that were noisy, hard to trust, and often disappeared on rerun.

Solution

This PR makes Redis-backed test resources unique per test and restores missing cleanup.

A shared pytest helper now generates per-test Redis names using the worker id plus the current test node. That helper is used in the shared search-index fixtures and in the highest-risk integration tests that previously reused fixed names. The HNSW fixtures now delete their indexes after yielding, and tests that instantiate Redis-backed objects directly now clean them up explicitly.

The semantic router tests also no longer need the existing “flaky test” skips because their Redis setup is now deterministic under unique names.

Changes

• Added a per-test Redis naming fixture in tests/conftest.py
• Updated shared flat/HNSW index fixtures to use per-test index names and prefixes
• Added teardown to hnsw_index and async_hnsw_index
• Replaced fixed Redis names in test_message_history.py
• Replaced fixed Redis names in test_semantic_router.py
• Added explicit cleanup for direct SemanticMessageHistory and SemanticRouter instances
• Updated embed cache fixtures to use per-test cache names
• Updated the multiple-prefix existing-index test to use a unique index name

Scope

• Test-only changes
• No production API changes
• No CI workflow changes
• No coverage reduction

Verification

• Targeted Redis-backed suites passed under pytest -n auto
• Repeated reruns showed the isolation fixes holding at the assertion level
• There is still a separate pytest master shutdown hang during some repeated xdist reruns, but that is distinct from the Redis leakage fixed here

---

Note

Low Risk
Low risk: changes are limited to test fixtures/assertions, mainly improving Redis resource isolation and teardown to reduce CI flakiness under pytest-xdist. Main risk is unintended name/cleanup changes masking real failures or leaving less coverage for shared-state scenarios.

Overview
Improves Redis-backed test isolation under pytest -n by introducing a redis_test_name fixture (worker id + test node hash) and switching indices/caches/histories/routers to use per-test Redis names instead of fixed or worker-scoped ones.

Adds missing teardown for HNSW index fixtures and wraps several integration tests in explicit clear()/delete() cleanup blocks; also removes prior flaky test skips and updates a hybrid-search assertion to match the actual number of loaded docs.

^{Written by Cursor Bugbot for commit 7e2afe8. This will update automatically on new commits. Configure here.}

[jit-ci]

🛡️ Jit Security Scan Results

✅ No security findings were detected in this PR

---

^{Security scan by Jit}

[nkanu17]

This looks great to me. Like that you used redis-backed tests per-test resource names and fixing over the hnsw stuff properly, and like that the tests now clean up after themselves!

As a follow-up, it would be nice to apply the same per test naming pattern across the rest of the suite, to make isolation fully consistent.

Also this is on all of us, I realized we have so many repeated patterns or code blocks (maybe because of AI tests we all push?) that we should probs take time to clean up!

[vishal-bala]

@nkanu17 I catalogued a more extensive list of issues with test parallelization here for future work: #546

[applied-ai-release-bot]

🚀 PR was released in v0.17.0 🚀