Skip to content

PYTHON-5753 Add just recipes for running coverage tests locally#2727

Merged
aclark4life merged 4 commits intomongodb:masterfrom
aclark4life:PYTHON-5753
Mar 12, 2026
Merged

PYTHON-5753 Add just recipes for running coverage tests locally#2727
aclark4life merged 4 commits intomongodb:masterfrom
aclark4life:PYTHON-5753

Conversation

@aclark4life
Copy link
Contributor

@aclark4life aclark4life commented Mar 11, 2026

PYTHON-5753

Changes in this PR

Add just recipes for running coverage tests locally

Test Plan

Checklist

Checklist for Author

  • Did you update the changelog (if necessary)?
  • Is there test coverage?
  • Is any followup work tracked in a JIRA ticket? If so, add link(s).

Checklist for Reviewer

  • Does the title of the PR reference a JIRA Ticket?
  • Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
  • Is all relevant documentation (README or docstring) updated?

@aclark4life aclark4life requested a review from a team as a code owner March 11, 2026 16:12
@aclark4life aclark4life requested review from Jibola, NoahStapp and Copilot and removed request for Jibola and Copilot March 11, 2026 16:12
justfile Outdated
uv run --active --extra test --group coverage python -m coverage run -m pytest {{args}}
uv run --active --group coverage python -m coverage report
else
# Otherwise use the standard Evergreen workflow
Copy link
Contributor Author

@aclark4life aclark4life Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to make a follow up ticket PYTHON-5754 makes setup-tests run-tests support USE_ACTIVE_VENV so this could potentially wait for that to be done and then we can remove this conditional statement.

CONTRIBUTING.md Outdated
and the `<class_name>` to test a full module. For example:
`just test test/test_change_stream.py::TestUnifiedChangeStreamsErrors::test_change_stream_errors_on_ElectionInProgress`.
- Use the `-k` argument to select tests by pattern.
- Run `just test-coverage` to run tests with coverage and display a report. Use `just coverage-html` to generate an HTML report in `htmlcov/index.html`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does just coverage-html require just test-coverage to be run first? We should make that clear if so.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@codecov-commenter
Copy link

codecov-commenter commented Mar 11, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.60%. Comparing base (3d89d9f) to head (9e324e1).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2727      +/-   ##
==========================================
+ Coverage   87.57%   87.60%   +0.02%     
==========================================
  Files         141      141              
  Lines       24152    24152              
  Branches     4126     4126              
==========================================
+ Hits        21152    21159       +7     
+ Misses       2108     2105       -3     
+ Partials      892      888       -4     
Flag Coverage Δ
auth-aws-rhel8-test-auth-aws-rapid-web-identity-python3.14-cov 35.08% <ø> (ø)
auth-aws-win64-test-auth-aws-rapid-web-identity-python3.14-cov 35.08% <ø> (ø)
auth-enterprise-macos-test-standard-auth-latest-python3.11-auth-ssl-sharded-cluster-cov 43.77% <ø> (?)
auth-enterprise-rhel8-test-standard-auth-latest-python3.11-auth-ssl-sharded-cluster-cov 43.72% <ø> (-0.01%) ⬇️
auth-enterprise-win64-test-standard-auth-latest-python3.11-auth-ssl-sharded-cluster-cov 43.74% <ø> (-0.04%) ⬇️
auth-oidc-local-ubuntu-22-test-auth-oidc-default 48.65% <ø> (-0.05%) ⬇️
compression-snappy-rhel8-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 55.20% <ø> (-0.01%) ⬇️
compression-snappy-rhel8-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 57.12% <ø> (ø)
compression-snappy-rhel8-test-standard-latest-python3.13-async-auth-ssl-sharded-cluster-cov 56.53% <ø> (ø)
compression-snappy-rhel8-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.84% <ø> (-0.01%) ⬇️
compression-zlib-rhel8-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 55.20% <ø> (-0.01%) ⬇️
compression-zlib-rhel8-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 57.12% <ø> (ø)
compression-zlib-rhel8-test-standard-latest-python3.13-async-auth-ssl-sharded-cluster-cov 56.53% <ø> (ø)
compression-zlib-rhel8-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.85% <ø> (+<0.01%) ⬆️
compression-zstd-rhel8-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 55.21% <ø> (+<0.01%) ⬆️
compression-zstd-rhel8-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 57.12% <ø> (ø)
compression-zstd-rhel8-test-standard-latest-python3.13-async-auth-ssl-sharded-cluster-cov 56.53% <ø> (ø)
compression-zstd-rhel8-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.84% <ø> (ø)
compression-zstd-ubuntu-22-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.83% <ø> (ø)
coverage-report-coverage-report 87.55% <ø> (+0.04%) ⬆️
disable-test-commands-rhel8-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 55.20% <ø> (-0.01%) ⬇️
disable-test-commands-rhel8-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 57.12% <ø> (ø)
disable-test-commands-rhel8-test-standard-latest-python3.13-async-auth-ssl-sharded-cluster-cov 56.53% <ø> (ø)
disable-test-commands-rhel8-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.84% <ø> (-0.01%) ⬇️
encryption-crypt_shared-macos-test-non-standard-latest-python3.13-noauth-nossl-standalone-cov 52.80% <ø> (?)
encryption-crypt_shared-macos-test-non-standard-latest-python3.14-auth-ssl-sharded-cluster-cov 54.72% <ø> (ø)
encryption-crypt_shared-macos-test-non-standard-latest-python3.14t-noauth-ssl-replica-set-cov 54.53% <ø> (ø)
encryption-crypt_shared-rhel8-test-non-standard-latest-python3.13-noauth-nossl-standalone-cov 52.81% <ø> (-0.01%) ⬇️
encryption-crypt_shared-rhel8-test-non-standard-latest-python3.14-auth-ssl-sharded-cluster-cov 54.59% <ø> (ø)
encryption-crypt_shared-rhel8-test-non-standard-latest-python3.14t-noauth-ssl-replica-set-cov 54.52% <ø> (-0.02%) ⬇️
encryption-crypt_shared-win64-test-non-standard-latest-python3.13-noauth-nossl-standalone-cov 52.69% <ø> (-0.06%) ⬇️
encryption-crypt_shared-win64-test-non-standard-latest-python3.14-auth-ssl-sharded-cluster-cov 54.64% <ø> (+0.01%) ⬆️
encryption-crypt_shared-win64-test-non-standard-latest-python3.14t-noauth-ssl-replica-set-cov 54.64% <ø> (+0.03%) ⬆️
encryption-macos-test-non-standard-latest-python3.13-noauth-nossl-standalone-cov 52.82% <ø> (?)
encryption-macos-test-non-standard-latest-python3.14-auth-ssl-sharded-cluster-cov 54.70% <ø> (-0.03%) ⬇️
encryption-macos-test-non-standard-latest-python3.14t-noauth-ssl-replica-set-cov 54.53% <ø> (?)
encryption-pyopenssl-rhel8-test-non-standard-latest-python3.13-noauth-nossl-standalone-cov 53.48% <ø> (-0.01%) ⬇️
encryption-pyopenssl-rhel8-test-non-standard-latest-python3.14-auth-ssl-sharded-cluster-cov 55.29% <ø> (+<0.01%) ⬆️
encryption-pyopenssl-rhel8-test-non-standard-latest-python3.14t-noauth-ssl-replica-set-cov 55.21% <ø> (-0.01%) ⬇️
encryption-rhel8-test-non-standard-latest-python3.13-noauth-nossl-standalone-cov 52.82% <ø> (+<0.01%) ⬆️
encryption-rhel8-test-non-standard-latest-python3.14-auth-ssl-sharded-cluster-cov 54.60% <ø> (+0.01%) ⬆️
encryption-rhel8-test-non-standard-latest-python3.14t-noauth-ssl-replica-set-cov 54.52% <ø> (-0.01%) ⬇️
encryption-win64-test-non-standard-latest-python3.13-noauth-nossl-standalone-cov 52.69% <ø> (-0.03%) ⬇️
encryption-win64-test-non-standard-latest-python3.14-auth-ssl-sharded-cluster-cov 54.63% <ø> (-0.01%) ⬇️
encryption-win64-test-non-standard-latest-python3.14t-noauth-ssl-replica-set-cov 54.58% <ø> (-0.03%) ⬇️
load-balancer-test-non-standard-latest-python3.14-auth-ssl-sharded-cluster-cov 48.31% <ø> (ø)
mongodb-latest-test-server-version-python3.10-async-auth-ssl-sharded-cluster-min-deps-cov 56.94% <ø> (-0.02%) ⬇️
mongodb-latest-test-server-version-python3.10-async-noauth-nossl-standalone-min-deps-cov 55.23% <ø> (+<0.01%) ⬆️
mongodb-latest-test-server-version-python3.10-sync-auth-ssl-sharded-cluster-min-deps-cov 59.11% <ø> (ø)
mongodb-latest-test-server-version-python3.10-sync-noauth-nossl-replica-set-min-deps-cov 59.26% <ø> (ø)
mongodb-latest-test-server-version-python3.11-async-noauth-nossl-replica-set-cov 57.06% <ø> (-0.01%) ⬇️
mongodb-rapid-test-server-version-python3.10-async-auth-ssl-sharded-cluster-min-deps-cov 56.94% <ø> (ø)
mongodb-rapid-test-server-version-python3.10-async-noauth-nossl-standalone-min-deps-cov 55.24% <ø> (+<0.01%) ⬆️
mongodb-rapid-test-server-version-python3.10-sync-auth-ssl-sharded-cluster-min-deps-cov 59.11% <ø> (ø)
mongodb-rapid-test-server-version-python3.10-sync-noauth-nossl-replica-set-min-deps-cov 59.26% <ø> (ø)
mongodb-rapid-test-server-version-python3.11-async-noauth-nossl-replica-set-cov 57.07% <ø> (+<0.01%) ⬆️
mongodb-v4.2-test-server-version-python3.10-async-auth-ssl-sharded-cluster-min-deps-cov 54.63% <ø> (-0.06%) ⬇️
mongodb-v4.2-test-server-version-python3.10-async-noauth-nossl-standalone-min-deps-cov 53.18% <ø> (ø)
mongodb-v4.2-test-server-version-python3.10-sync-auth-ssl-sharded-cluster-min-deps-cov 56.81% <ø> (ø)
mongodb-v4.2-test-server-version-python3.10-sync-noauth-nossl-replica-set-min-deps-cov 56.93% <ø> (ø)
mongodb-v4.2-test-server-version-python3.11-async-noauth-nossl-replica-set-cov 54.75% <ø> (-0.04%) ⬇️
mongodb-v4.4-test-server-version-python3.10-async-auth-ssl-sharded-cluster-min-deps-cov 55.04% <ø> (-0.01%) ⬇️
mongodb-v4.4-test-server-version-python3.10-async-noauth-nossl-standalone-min-deps-cov 53.46% <ø> (+0.01%) ⬆️
mongodb-v4.4-test-server-version-python3.10-sync-auth-ssl-sharded-cluster-min-deps-cov 57.22% <ø> (ø)
mongodb-v4.4-test-server-version-python3.10-sync-noauth-nossl-replica-set-min-deps-cov 57.30% <ø> (ø)
mongodb-v4.4-test-server-version-python3.11-async-noauth-nossl-replica-set-cov 55.08% <ø> (-0.01%) ⬇️
mongodb-v5.0-test-server-version-python3.10-async-auth-ssl-sharded-cluster-min-deps-cov 55.25% <ø> (+0.02%) ⬆️
mongodb-v5.0-test-server-version-python3.10-async-noauth-nossl-standalone-min-deps-cov 53.63% <ø> (-0.01%) ⬇️
mongodb-v5.0-test-server-version-python3.10-sync-auth-ssl-sharded-cluster-min-deps-cov 57.41% <ø> (ø)
mongodb-v5.0-test-server-version-python3.10-sync-noauth-nossl-replica-set-min-deps-cov 57.54% <ø> (ø)
mongodb-v5.0-test-server-version-python3.11-async-noauth-nossl-replica-set-cov 55.33% <ø> (+0.01%) ⬆️
mongodb-v6.0-test-server-version-python3.10-async-auth-ssl-sharded-cluster-min-deps-cov 55.25% <ø> (-0.01%) ⬇️
mongodb-v6.0-test-server-version-python3.10-async-noauth-nossl-standalone-min-deps-cov 53.63% <ø> (-0.01%) ⬇️
mongodb-v6.0-test-server-version-python3.10-sync-auth-ssl-sharded-cluster-min-deps-cov 57.43% <ø> (ø)
mongodb-v6.0-test-server-version-python3.10-sync-noauth-nossl-replica-set-min-deps-cov 57.58% <ø> (ø)
mongodb-v6.0-test-server-version-python3.11-async-noauth-nossl-replica-set-cov 55.37% <ø> (+<0.01%) ⬆️
mongodb-v7.0-test-server-version-python3.10-async-auth-ssl-sharded-cluster-min-deps-cov 55.26% <ø> (+<0.01%) ⬆️
mongodb-v7.0-test-server-version-python3.10-async-noauth-nossl-standalone-min-deps-cov 53.62% <ø> (+<0.01%) ⬆️
mongodb-v7.0-test-server-version-python3.10-sync-auth-ssl-sharded-cluster-min-deps-cov 57.44% <ø> (ø)
mongodb-v7.0-test-server-version-python3.10-sync-noauth-nossl-replica-set-min-deps-cov 57.58% <ø> (ø)
mongodb-v7.0-test-server-version-python3.11-async-noauth-nossl-replica-set-cov 55.36% <ø> (-0.01%) ⬇️
mongodb-v8.0-test-server-version-python3.10-async-auth-ssl-sharded-cluster-min-deps-cov 56.96% <ø> (+0.02%) ⬆️
mongodb-v8.0-test-server-version-python3.10-async-noauth-nossl-standalone-min-deps-cov 55.22% <ø> (-0.02%) ⬇️
mongodb-v8.0-test-server-version-python3.10-sync-auth-ssl-sharded-cluster-min-deps-cov 59.11% <ø> (ø)
mongodb-v8.0-test-server-version-python3.10-sync-noauth-nossl-replica-set-min-deps-cov 59.26% <ø> (ø)
mongodb-v8.0-test-server-version-python3.11-async-noauth-nossl-replica-set-cov 57.05% <ø> (-0.02%) ⬇️
no-c-ext-rhel8-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 56.43% <ø> (+0.01%) ⬆️
no-c-ext-rhel8-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 58.34% <ø> (ø)
no-c-ext-rhel8-test-standard-latest-python3.13-async-auth-ssl-sharded-cluster-cov 57.75% <ø> (ø)
no-c-ext-rhel8-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 56.06% <ø> (+<0.01%) ⬆️
ocsp-rhel8-test-ocsp-ecdsa-valid-cert-server-staples-latest-python3.14-cov 34.07% <ø> (+<0.01%) ⬆️
ocsp-rhel8-test-ocsp-rsa-valid-cert-server-staples-latest-python3.14-cov 34.05% <ø> (ø)
pyopenssl-macos-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 57.03% <ø> (ø)
pyopenssl-rhel8-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 57.03% <ø> (ø)
pyopenssl-win64-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 56.96% <ø> (-0.01%) ⬇️
stable-api-accept-v2-rhel8-auth-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 55.21% <ø> (+<0.01%) ⬆️
stable-api-accept-v2-rhel8-auth-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.84% <ø> (ø)
stable-api-require-v1-rhel8-auth-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 55.19% <ø> (ø)
stable-api-require-v1-rhel8-auth-test-standard-latest-python3.13-async-auth-ssl-sharded-cluster-cov 56.41% <ø> (ø)
stable-api-require-v1-rhel8-auth-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.83% <ø> (ø)
storage-inmemory-rhel8-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 55.20% <ø> (ø)
storage-inmemory-rhel8-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.85% <ø> (+0.01%) ⬆️
test-macos-arm64-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 55.17% <ø> (-0.01%) ⬇️
test-macos-arm64-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 57.12% <ø> (ø)
test-macos-arm64-test-standard-latest-python3.13-async-auth-ssl-sharded-cluster-cov 56.54% <ø> (+<0.01%) ⬆️
test-macos-arm64-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.81% <ø> (?)
test-macos-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 55.20% <ø> (?)
test-macos-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 57.12% <ø> (+0.01%) ⬆️
test-macos-test-standard-latest-python3.13-async-auth-ssl-sharded-cluster-cov 56.53% <ø> (ø)
test-macos-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.81% <ø> (?)
test-numpy-macos-arm64-test-numpy-python3.14-python3.14-cov 32.26% <ø> (?)
test-numpy-macos-test-numpy-python3.14-python3.14-cov 32.24% <ø> (?)
test-numpy-rhel8-test-numpy-python3.14-python3.14-cov 32.26% <ø> (ø)
test-numpy-win32-test-numpy-python3.14-python3.14-cov 32.22% <ø> (-0.02%) ⬇️
test-numpy-win64-test-numpy-python3.14-python3.14-cov 32.24% <ø> (ø)
test-win32-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 55.05% <ø> (+<0.01%) ⬆️
test-win32-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 57.06% <ø> (ø)
test-win32-test-standard-latest-python3.13-async-auth-ssl-sharded-cluster-cov 56.47% <ø> (+<0.01%) ⬆️
test-win32-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.67% <ø> (ø)
test-win64-test-standard-latest-python3.11-async-noauth-nossl-standalone-cov 55.06% <ø> (+0.02%) ⬆️
test-win64-test-standard-latest-python3.12-async-noauth-ssl-replica-set-cov 57.06% <ø> (ø)
test-win64-test-standard-latest-python3.13-async-auth-ssl-sharded-cluster-cov 56.47% <ø> (ø)
test-win64-test-standard-latest-python3.14-async-noauth-nossl-standalone-cov 54.68% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copilot AI review requested due to automatic review settings March 11, 2026 18:20
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds local developer ergonomics for running the PyMongo test suite with coverage via just, and documents the new commands for contributors.

Changes:

  • Introduces a just test-coverage recipe with support for running in an active venv (USE_ACTIVE_VENV) or via the Evergreen-style setup/run workflow.
  • Adds just recipes to generate coverage reports in terminal, HTML, and XML formats.
  • Updates contributor docs to mention the new coverage recipes.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
justfile Adds test-coverage plus coverage-report/html/xml recipes for local coverage workflows.
CONTRIBUTING.md Documents just test-coverage and just coverage-html for contributors.

justfile Outdated
# Otherwise use the standard Evergreen workflow
just setup-tests --cov
just run-tests {{args}}
just coverage-report

This comment was marked as resolved.

@aclark4life aclark4life requested a review from NoahStapp March 11, 2026 18:54
@aclark4life aclark4life merged commit 80c3ff2 into mongodb:master Mar 12, 2026
38 of 40 checks passed
@aclark4life aclark4life deleted the PYTHON-5753 branch March 12, 2026 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants