Skip to content

test: support sccache for pytest builds#1610

Merged
jpnurmi merged 4 commits intomasterfrom
jpnurmi/build/sccache
Mar 31, 2026
Merged

test: support sccache for pytest builds#1610
jpnurmi merged 4 commits intomasterfrom
jpnurmi/build/sccache

Conversation

@jpnurmi
Copy link
Copy Markdown
Collaborator

@jpnurmi jpnurmi commented Mar 30, 2026

In the series of making it more pleasant to work on Windows, replace the previously undocumented ccache support with Windows-compatible sccache as the compilation cache for pytest builds, and document it in CONTRIBUTING.md.

sccache works on all platforms including Windows, making ccache redundant (added in #1529, was it used by anyone else?). On Windows, USE_SCCACHE=1 automatically switches to the Ninja generator (VS generator doesn't support compiler launchers) and uses /Z7 debug info format via CMP0141 policy (sccache can't handle /Zi shared PDBs).

YMMV, but on my old dedicated Windows laptop (Surface 4), this cuts the full pytest run from roughly 14-15 minutes to somewhere around 11-12 minutes:

pytest -v tests
(.venv) C:\Users\jpnurmi\Projects\sentry\sentry-native>pytest -v tests
=================================================================== test session starts ====================================================================
platform win32 -- Python 3.13.3, pytest-8.1.1, pluggy-1.6.0 -- C:\Users\jpnurmi\Projects\sentry\sentry-native\.venv\Scripts\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\jpnurmi\Projects\sentry\sentry-native
plugins: flaky-3.8.1, pytest_httpserver-1.0.10, repeat-0.9.3, xdist-3.5.0
collected 1144 items
[...]
====================================================================== sccache stats =======================================================================
Compile requests                   3024
Compile requests executed          3024
Cache hits                         3023
Cache hits (C/C++)                 3023
Cache misses                          1
Cache misses (C/C++)                  1
Cache hits rate                   99.97 %
Cache hits rate (C/C++)           99.97 %
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Cache errors                          0
Compilations                          1
Compilation failures                  0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.001 s
Average compiler                  0.866 s
Average cache read hit            0.009 s
Failed distributed compilations       0
Cache location                  Local disk: "C:\\Users\\jpnurmi\\AppData\\Local\\Mozilla\\sccache\\cache"
Base directories                (none)
Use direct/preprocessor mode?   yes
Version (client)                0.14.0
Cache size                           97 MiB
Max cache size                       10 GiB
===Flaky Test Report===

test_breakpad_dump_inflight passed 1 out of the required 1 times. Success!
test_shutdown_timeout passed 1 out of the required 1 times. Success!

===End Flaky Test Report===

======================================================= 1108 passed, 36 skipped in 653.14s (0:10:53) =======================================================

And in the CI (Note: these can vary a lot depending on the time of day):

image

Replace USE_CCACHE=1 with USE_SCCACHE=1 as the compiler launcher for
pytest builds. sccache works on all platforms including Windows, making
ccache redundant. On Windows, this automatically switches to the Ninja
generator (VS generator doesn't support compiler launchers) and uses /Z7
debug info format via CMP0141 policy (sccache can't handle /Zi shared
PDBs). Cache stats are printed at the end of the pytest session.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment thread tests/cmake.py Outdated
`append("-G Ninja")` passed the flag and value as a single argv entry,
causing CMake to look for a generator named " Ninja" (with leading space).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jpnurmi jpnurmi marked this pull request as draft March 30, 2026 13:52
jpnurmi and others added 2 commits March 30, 2026 16:02
Add a new CI matrix entry that builds with Ninja and sccache on Windows.
This requires setting up the MSVC environment via vcvarsall.bat since
Ninja doesn't auto-detect it like the Visual Studio generator does.

The sccache cache is persisted across CI runs using actions/cache.
Also fix install-zlib.ps1 to preserve existing CMAKE_DEFINES.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jpnurmi jpnurmi marked this pull request as ready for review March 31, 2026 06:41
@jpnurmi jpnurmi merged commit 3442a21 into master Mar 31, 2026
56 checks passed
@jpnurmi jpnurmi deleted the jpnurmi/build/sccache branch March 31, 2026 09:31
jpnurmi added a commit that referenced this pull request Apr 9, 2026
3442a21 (#1610) added CMP0141 NEW for sccache support. This policy
defaults CMAKE_MSVC_DEBUG_INFORMATION_FORMAT to ProgramDatabase (/Zi)
for Debug and RelWithDebInfo configurations:
https://cmake.org/cmake/help/latest/policy/CMP0141.html

This causes static libraries to reference an external PDB. Since
static libraries are distributed without a PDB, consumers get LNK4099
warnings when linking, which breaks builds using
MSBuildTreatWarningsAsErrors or TreatWarningsAsErrors.

Set the MSVC_DEBUG_INFORMATION_FORMAT target property to Embedded (/Z7)
for static builds so debug types are embedded directly in the .obj
files and no external PDB is needed.
jpnurmi added a commit that referenced this pull request Apr 9, 2026
3442a21 (#1610) added CMP0141 NEW for sccache support. This policy
defaults CMAKE_MSVC_DEBUG_INFORMATION_FORMAT to ProgramDatabase (/Zi)
for Debug and RelWithDebInfo configurations:
https://cmake.org/cmake/help/latest/policy/CMP0141.html

This causes static libraries to reference an external PDB. Since
static libraries are distributed without a PDB, consumers get LNK4099
warnings when linking, which breaks builds using
MSBuildTreatWarningsAsErrors or TreatWarningsAsErrors.

Set the MSVC_DEBUG_INFORMATION_FORMAT target property to Embedded (/Z7)
for static builds so debug types are embedded directly in the .obj
files and no external PDB is needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants