From b86aa0f1ad10d4b7b661566b393a246768956592 Mon Sep 17 00:00:00 2001 From: Tobias Brox Date: Tue, 13 Oct 2026 20:59:49 +0200 Subject: [PATCH 1/2] chore: collect changelog entries with towncrier Parallel pull requests keep conflicting in CHANGELOG.md. After 3.5.0, each change adds a fragment to changes/ and towncrier build merges them in at release time. A custom template keeps the existing entry style. The hand-written 3.5.0 section stays below the towncrier marker; RELEASE-HOWTO notes that open branches must move their entries into fragments. Prompt: set up caldav with towncrier, the CHANGELOG management system Followup-Prompt: We're preparing for 3.5.0 now. Things are frozen, only really important bugfixes will go in now. When 3.5.0 is released, everything should go through towncrier. The changelog as it is should go through normal QA and get stamped with 3.5.0. Hence, towncrier should not touch anything before 3.5.0 is out, but 3.5.0 should already be set up, process of using it in CONTRIBUTING.md, etc Followup-Prompt: The open PRs that goes (sic) in after release will have to be rewritten to support towncrier Followup-Prompt: PR#768 should be rebased and merged [= https://github.com/python-caldav/caldav/pull/768, this pull request, onto master after the 3.5.0 release] Assisted-By: Claude Opus 5.5 Reviewed-by: Tobias Brox --- .lycheeignore | 3 ++ CHANGELOG.md | 4 +++ CONTRIBUTING.md | 8 +++--- changes/README.md | 32 ++++++++++++++++++++++ changes/template.md.jinja | 18 ++++++++++++ docs/design/RELEASE-HOWTO.md | 13 ++++++--- docs/design/TODO.md | 17 ++---------- pyproject.toml | 53 ++++++++++++++++++++++++++++++++++++ 8 files changed, 125 insertions(+), 23 deletions(-) create mode 100644 changes/README.md create mode 100644 changes/template.md.jinja diff --git a/.lycheeignore b/.lycheeignore index 9637e3b9..ecc8d946 100644 --- a/.lycheeignore +++ b/.lycheeignore @@ -64,3 +64,6 @@ https://dav\.qq\.com/.* # pushes the code before the tag, so this view 404s until `v3.3.0` is pushed - # delete this entry once it is. https://github\.com/python-caldav/caldav/compare/v3\.2\.1\.\.\.v3\.3\.0 + +# The issue_format template in pyproject.toml (towncrier) +https://github\.com/python-caldav/caldav/issues/(%7B|\{)issue(%7D|\}) diff --git a/CHANGELOG.md b/CHANGELOG.md index 90b4965b..00d91afb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,6 +22,10 @@ Changelogs prior to v3.0 are pruned, but are available in the v3.1 release This project should adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html), though for pre-releases PEP 440 takes precedence. +After 3.5.0, changes since the last release are kept as fragments in [changes/](changes/), and merged in here at release time. + + + ## [3.5.0] - 2026-10-15 Yet another release primarily focusing on bugfixes and compatibility. iCloud has now been tested and got its compatibility profile in `compatibility_hints.py`. Test runs towards GMX, Posteo and Purelymail have been skipped for the last few releases; with 3.5.0 the tests pass towards those. More work on getting synchronization right, and on avoiding overwriting changes when multiple clients update the same object. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e6431c30..5271abc5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -9,12 +9,12 @@ Starting from v3.0.1, we'll stick to https://www.conventionalcommits.org/en/v1.0 The types used should (as for now) be one of: * "revert" - a clean revert of a previous commit (should be used very infrequently on the master branch) -* "feat" - a new feature added to the codebase/API. The commit should include test code, documentation and a CHANGELOG entry unless there are good reasons for procrastinating it. "feat" should not be used for new features that only affects the test framework, or changes only affectnig the documentation etc. -* "fix" - a bugfix. Again, documentation and CHANGELOG-entry should be included in the commit (notable exception: if a bug was introduced after the previous release and fixed before the next release, it does not need to be mentioned in the CHANGELOG). If the only "fix" is that some typo is fixed in some existing documentation, then we should use "docs" instead. +* "feat" - a new feature added to the codebase/API. The commit should include test code, documentation and a changelog fragment (see below) unless there are good reasons for procrastinating it. "feat" should not be used for new features that only affects the test framework, or changes only affecting the documentation etc. +* "fix" - a bugfix. Again, documentation and a changelog fragment should be included in the commit (notable exception: if a bug was introduced after the previous release and fixed before the next release, it does not need to be mentioned in the CHANGELOG). If the only "fix" is that some typo is fixed in some existing documentation, then we should use "docs" instead. * "perf" - a code change in the codebase that is neither a bugfix or a feature, but intends to improve performance. * "refactor" - a code change in the codebase that is neither a bugfix or a feature, but makes the code more readable, shorter, better or more maintainable. * "test" - fixes, additions or improvements that only affects the test code or the test framework. The commit may include documentation. -* "docs" - changes that *only* is done to the documentation, documentation framework - this includes minor typo fixes as well as new documentation, and it includes both the user documentation under `docs/source`, other documentation files (including CHANGELOG) as well as inline comments and docstrings in the code itself. +* "docs" - changes that *only* is done to the documentation, documentation framework - this includes minor typo fixes as well as new documentation, and it includes both the user documentation under `docs/source`, other documentation files (including CHANGELOG and changelog fragments) as well as inline comments and docstrings in the code itself. * "chore" - if nothing of the above fits The `compatibility_hints.py` has been moved from the test directory to the codebase not so very long ago. Some special rules here: @@ -72,7 +72,7 @@ Consider this procedures to be a more of a guideline than a rigid procedure. Us * Consider to write some lines in the documentation, changelog and/or examples covering your change -* Add an entry in the `CHANGELOG.md` file. +* Add a changelog fragment: a file `changes/..md`, e.g. `changes/740.fixed.md`, holding the text of the entry (without an issue: `changes/+..md`). Do not edit `CHANGELOG.md` itself - the fragments are collected into it at release time, so that parallel pull requests do not conflict there. [changes/README.md](changes/README.md) has the details. * Create a pull request diff --git a/changes/README.md b/changes/README.md new file mode 100644 index 00000000..adc59858 --- /dev/null +++ b/changes/README.md @@ -0,0 +1,32 @@ +# Changelog fragments + +Changes since the last release are not written into `CHANGELOG.md` directly. Each change adds a small file - a "fragment" - to this directory instead, and at release time [towncrier](https://towncrier.readthedocs.io/) collects them into `CHANGELOG.md`. Separate files mean that parallel pull requests no longer conflict in the CHANGELOG. + +## File naming + +`..md`, e.g. `740.fixed.md` for a fix of https://github.com/python-caldav/caldav/issues/740 - the URL is appended to the entry automatically. A pull request number works too. If two changes of the same type belong to one issue, add a counter: `740.fixed.1.md`, `740.fixed.2.md` (`towncrier create` picks one when the name is taken). + +For a change without an issue, or when the entry cites its URLs inline (several issues, a pull request and an issue, an upstream bug report ...), use `+..md`, e.g. `+async-xml-parsing.fixed.md`. Nothing is appended to such an entry. + +The types, in the order they appear in the CHANGELOG: + +| type | section | +|--------------|-------------------------| +| `breaking` | Breaking changes | +| `added` | Added | +| `changed` | Changed | +| `deprecated` | Deprecated | +| `removed` | Removed | +| `fixed` | Fixed | +| `security` | Security | +| `tests` | Tests and documentation | + +Use `tests` only for changes a user or packager notices (a new test option, a test dependency, docs that moved); a pull request that only adds or changes tests or docs needs no fragment. + +## Content + +Markdown, written as the text of one bullet point, without the leading `* `. Describe the change as seen by a user of the library: what was wrong or missing, and what it does now. Several paragraphs or a nested list are fine. + +The rules in [CONTRIBUTING.md](../CONTRIBUTING.md) for what needs an entry are unchanged - a bug introduced and fixed between two releases needs none, so delete or edit the fragment of the change that introduced it instead. + +`towncrier create 740.fixed.md --edit` creates a fragment, and `towncrier build --draft --version next` previews the next release section (`pip install towncrier`). diff --git a/changes/template.md.jinja b/changes/template.md.jinja new file mode 100644 index 00000000..c0fcec61 --- /dev/null +++ b/changes/template.md.jinja @@ -0,0 +1,18 @@ +{#- Renders entries the way CHANGELOG.md has always had them: "* " bullets, + with the issue URLs appended after two spaces (or as a paragraph of their + own when the entry spans several lines), separated by " - ". -#} +{%- set nl = "\n" -%} +{{- "## [" ~ versiondata.version ~ "] - " ~ versiondata.date ~ nl }} +{%- for section, _ in sections.items() %} + {%- for category, val in definitions.items() if category in sections[section] %} + {{- nl ~ "### " ~ definitions[category]['name'] ~ nl ~ nl }} + {%- for text, values in sections[section][category].items() %} + {{- "* " ~ text }} + {%- if values %} + {{- (nl ~ nl ~ " " if nl in text else " ") ~ values | join(" - ") }} + {%- endif %} + {{- nl }} + {%- endfor %} + {%- endfor %} +{%- endfor %} +{{- nl -}} diff --git a/docs/design/RELEASE-HOWTO.md b/docs/design/RELEASE-HOWTO.md index bf74224f..a4f10054 100644 --- a/docs/design/RELEASE-HOWTO.md +++ b/docs/design/RELEASE-HOWTO.md @@ -6,9 +6,13 @@ I have no clue on the proper procedures for doing releases, and I keep on doing (And no - we cannot do auto-releases of CalDAV. Every release has to be well-tested, some of the tests needs my private configuration with passwords and usernames for various caldav servers, and some of the docker containers included are sluggish, full test runs takes very long time). +## One-off: after the switch to towncrier + +3.5.0 was the last release whose `CHANGELOG.md` section was written by hand. Branches and pull requests still carrying an entry in `CHANGELOG.md` need it moved into a fragment in `changes/` before they are merged. Delete this section once none are left. + ## Checklist -* Go through changes since last release and compare it with the `CHANGELOG.md`. Any change should be logged. +* Go through changes since last release and compare it with the changelog fragments in `changes/` (`towncrier build --draft --version X` shows them as they will end up in the `CHANGELOG.md`, see `changes/README.md`). Any change should be logged. * Run tests towards as many servers as possible * Use the `PYTHON_CALDAV_DEBUGMODE=DEBUG_PDB` environment variable (or simply pass `--pdb` to pytest, which sets this automatically)! Should do some research if we hit any "soft asserts" or "weirdness". * Do research on breakages. If the test breaks also for the previous release of the caldav library, then it's likely to be due to some regression on the server side. For patch-level releases such breakages may be allowed, for minor-level releases we should try to work around problems @@ -17,11 +21,12 @@ I have no clue on the proper procedures for doing releases, and I keep on doing * Look through github pull requests and see if there is anything that ought to be included in the release * For minor and major releases, look through the github issues, anything urgent there that should be fixed prior to doing a new release? * Any changes done, go back to the start of this list -* Write up some release notes. (I typically keep a short summary of the changes in the CHANGELOG, and use that as the release notes). +* Write up some release notes. (I typically keep a short summary of the changes in the CHANGELOG, and use that as the release notes - see the `towncrier build` step below). * Verify that we're on the right branch - `git checkout master`. (`master` may not always be right - sometimes we may want to use a dedicated branch connected to the release-series, i.e. `v1.3`) * TODO - document needs to be updated - as the test runs on github now takes significant amounts of time, it's important to push the code first and wait for quite a while before tagging and pushing the tag. * Set the variable `VERSION=2.2.0` -* Commit the changes (typically `CHANGELOG.md`, perhaps documentation): `git commit -am "docs: preparing for v${VERSION}"` +* Collect the changelog fragments: `towncrier build --yes --version ${VERSION}` (`pip install towncrier`; without `--yes` it asks before removing the fragments). It writes a `## [${VERSION}] - ` section into `CHANGELOG.md` at the `` marker and `git rm`s the fragments. The date is the day of the build - fix it if the tag comes later. The result is plain markdown - now edit it by hand: towncrier sorts the entries by issue number, so reorder them by importance, add the summary on top, and add the `[${VERSION}]: https://github.com/python-caldav/caldav/compare/vPREVIOUS...v${VERSION}` link at the bottom of the file. +* Commit the changes (typically `CHANGELOG.md` and the removed fragments, perhaps documentation): `git commit -am "docs: preparing for v${VERSION}"` * Create a tag: `git tag -as v${VERSION}` - use the release notes in the tag message. Don't push it yet. * Make a clone: `cd ~ ; rm -rf caldav-release ; git clone caldav/ caldav-release ; cd caldav-release ; git checkout v${VERSION}` * Run tests: `pytest --pdb ; tox -e style`. @@ -56,4 +61,4 @@ This is most likely not complete, but should explain some of the "silly" steps a * Forgetting to push to pypi, or pushing something else than the tagged revision to pypi * Pushing out junk files in the pypi-release (i.e. .pyc-files, log files, temp files, `tests/conf_private.py`, `tests/caldav_test_servers.yaml`, an entire `venv/`, etc). `tox -e package` now catches this - see the build step above * Not adding the release to the "github releases" (I don't care much about this feature, but apparently some people check there to find the latest release version) -* Not committing after doing a s/unreleased/$VERSION/ on the CHANGELOG +* Not committing after running `towncrier build` (or, up to and including 3.5.0, after doing a s/unreleased/$VERSION/ on the CHANGELOG) diff --git a/docs/design/TODO.md b/docs/design/TODO.md index b327e7b7..053ed6a6 100644 --- a/docs/design/TODO.md +++ b/docs/design/TODO.md @@ -24,22 +24,9 @@ This violates CalDAV RFC expectations and is a Nextcloud bug. Restarting the Do 3. Consider adding server quirk detection in caldav_server_tester 4. Document workaround: avoid UID reuse with Nextcloud, or restart container between test runs -## Switch CHANGELOG to towncrier +## `towncrier check` in CI -**When**: right after the next release (3.5), so the Unreleased section is empty. - -Parallel PRs (third-party and own) keep conflicting in `CHANGELOG.md`. Use towncrier fragments (`changes/..md`, `changes/+slug..md` without an issue) instead. Tested OK with towncrier 26.9; nested lists in fragments render fine. - -Requirements kept: `towncrier build --version X` writes plain markdown, so the CHANGELOG can still be hand-edited and an executive summary added before the release commit. - -Steps: -1. `[tool.towncrier]` in `pyproject.toml`: `directory = "changes"`, `start_string = "\n"` (marker after the "Meta" section), `title_format = "## [{version}] - {project_date}"`, `issue_format` pointing at GitHub issues, and one `[[tool.towncrier.type]]` per Keep-a-Changelog section (Breaking changes, Added, Changed, Deprecated, Removed, Fixed, Security). -2. `changes/README.md` explaining file naming. -3. Update CONTRIBUTING.md (commit types section and "Add an entry in the CHANGELOG.md"). -4. Optional: `towncrier check` in CI, with a `no-changelog` label for test/docs-only PRs. -5. Release checklist: `towncrier build --version X` → reorder/hand-edit → add summary → commit → tag. - -Known downsides: no Unreleased section visible on master (consider `sphinxcontrib-towncrier`), entries sorted by issue number rather than importance, `(issue URL)` appended to entries — use `+slug` names where URLs are cited inline. +towncrier was set up after 3.5.0 (`changes/README.md`). Not done: a CI job running `towncrier check` on pull requests (with `--compare-with origin/master`, as towncrier defaults to `origin/main`), so that a missing changelog fragment is caught. It needs a way to skip test/docs-only pull requests, which need no entry (e.g. a `no-changelog` label), and should not be enabled before the open pull requests have been moved to fragments. Note that `towncrier check` also passes when `CHANGELOG.md` itself was edited, so it does not catch a pull request that still edits it directly. Also consider `sphinxcontrib-towncrier`, as from 3.5.0 on there is no "Unreleased" section in `CHANGELOG.md`. ## An error response with a non-XML body labelled as XML diff --git a/pyproject.toml b/pyproject.toml index 44577a0d..fb897dfb 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -219,3 +219,56 @@ filterwarnings = [ # Zimbra test server uses HTTPS with self-signed cert (ssl_verify_cert=False) "ignore:Unverified HTTPS request:urllib3_future.exceptions.InsecureRequestWarning", ] + +## Changelog fragments: changes/README.md explains the file naming, and +## docs/design/RELEASE-HOWTO.md how they are merged into CHANGELOG.md. +[tool.towncrier] +directory = "changes" +filename = "CHANGELOG.md" +template = "changes/template.md.jinja" +start_string = "\n" +## The title, "## [{version}] - {date}", is rendered by the template +title_format = false +issue_format = "https://github.com/python-caldav/caldav/issues/{issue}" +## editor leftovers too, as an unknown file name aborts the build +ignore = ["README.md", "template.md.jinja", "*~", ".#*", "#*#"] + +[[tool.towncrier.type]] +directory = "breaking" +name = "Breaking changes" +showcontent = true + +[[tool.towncrier.type]] +directory = "added" +name = "Added" +showcontent = true + +[[tool.towncrier.type]] +directory = "changed" +name = "Changed" +showcontent = true + +[[tool.towncrier.type]] +directory = "deprecated" +name = "Deprecated" +showcontent = true + +[[tool.towncrier.type]] +directory = "removed" +name = "Removed" +showcontent = true + +[[tool.towncrier.type]] +directory = "fixed" +name = "Fixed" +showcontent = true + +[[tool.towncrier.type]] +directory = "security" +name = "Security" +showcontent = true + +[[tool.towncrier.type]] +directory = "tests" +name = "Tests and documentation" +showcontent = true From ded8ef9e24578b88e37d6238aaced07be5e144ab Mon Sep 17 00:00:00 2001 From: Tobias Brox Date: Thu, 15 Oct 2026 12:35:04 +0200 Subject: [PATCH 2/2] docs: list unreviewed master commits as a mistake A CHANGELOG polished in a back-and-forth with an AI assistant and committed straight to master skips the usual QA, and leaves master with a string of commits carrying prompts nobody needs. Prompt: Do a QA on my sentence, commit it directly to master, rebase when #768 is merged, and I'll push it manually [= the sentence added to the list of release mistakes; https://github.com/python-caldav/caldav/pull/768 is the towncrier PR] Prompts (comments made inline in the document while reviewing it): > (on the review of https://github.com/python-caldav/caldav/pull/768) Please piggyback my latest commit to the RELEASE-HOWTO.md into the same PR Assisted-By: Claude Opus 5.5 (Claude Code) Reviewed-by: Tobias Brox --- docs/design/RELEASE-HOWTO.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/design/RELEASE-HOWTO.md b/docs/design/RELEASE-HOWTO.md index a4f10054..92737e9b 100644 --- a/docs/design/RELEASE-HOWTO.md +++ b/docs/design/RELEASE-HOWTO.md @@ -62,3 +62,4 @@ This is most likely not complete, but should explain some of the "silly" steps a * Pushing out junk files in the pypi-release (i.e. .pyc-files, log files, temp files, `tests/conf_private.py`, `tests/caldav_test_servers.yaml`, an entire `venv/`, etc). `tox -e package` now catches this - see the build step above * Not adding the release to the "github releases" (I don't care much about this feature, but apparently some people check there to find the latest release version) * Not committing after running `towncrier build` (or, up to and including 3.5.0, after doing a s/unreleased/$VERSION/ on the CHANGELOG) +* "Playing ball" with Claude on the CHANGELOG and committing the changes directly to master before a release, not going through the regular QA - and suddenly the master branch has 6-7 commits with various irrelevant prompts attached.