From 5a1c5869274abe35d3b348800222c9b7c6eec8e2 Mon Sep 17 00:00:00 2001 From: Tobias Brox Date: Thu, 22 Oct 2026 09:10:51 +0200 Subject: [PATCH 1/6] docs: tidy the changelog fragments for 3.6.0 One entry each for the wildcard preconditions and for the server profile updates; calendaring-config names the new dependency, and the files it reads differently move to the breaking list. The safe YAML loader is a fix, not a security issue: the attacker would need write access to the config file. Prompt: We should make a new release today. Check up the file tmp-handover.md [an untracked note with changelog decisions agreed in an earlier session] Assisted-By: Claude Opus 5.5 (1M context) Reviewed-by: Tobias Brox --- changes/+calendaring-config.breaking.md | 1 + changes/+calendaring-config.changed.md | 6 ++---- ...ig-safe-yaml.security.md => +config-safe-yaml.fixed.md} | 0 changes/+hints-2026-10-20.changed.md | 7 +++++-- changes/+hints-sync-token-probes.changed.md | 5 ----- changes/+no-overwrite-if-none-match.changed.md | 1 - changes/+wildcard-preconditions.breaking.md | 4 ++-- changes/794.fixed.md | 2 +- 8 files changed, 11 insertions(+), 15 deletions(-) create mode 100644 changes/+calendaring-config.breaking.md rename changes/{+config-safe-yaml.security.md => +config-safe-yaml.fixed.md} (100%) delete mode 100644 changes/+hints-sync-token-probes.changed.md delete mode 100644 changes/+no-overwrite-if-none-match.changed.md diff --git a/changes/+calendaring-config.breaking.md b/changes/+calendaring-config.breaking.md new file mode 100644 index 00000000..24f998d2 --- /dev/null +++ b/changes/+calendaring-config.breaking.md @@ -0,0 +1 @@ +Since the config file is now read through calendaring-config (see Changed), some files are read differently: an empty file found first is used, rather than skipped for the next one; a section named directly that is missing or disabled, or a glob that matches nothing, is an error; a name in `contains` that matches nothing gives a warning; a disabled meta-section leaves out its members too; `*` is only the sections that are not meta-sections; and a config file named explicitly must exist. diff --git a/changes/+calendaring-config.changed.md b/changes/+calendaring-config.changed.md index 1bc8219d..1817a45d 100644 --- a/changes/+calendaring-config.changed.md +++ b/changes/+calendaring-config.changed.md @@ -1,10 +1,8 @@ -The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786. New in the format: +The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786. This is a new dependency, `calendaring-config>=0.3` (it depends only on PyYAML), which matters to packagers. New in the format: - `~/.config/pycal/calendar.yaml` is searched first, before the old paths. - A section may say `kind: caldav` and drop the `caldav_` prefix. Sections of other kinds (`kind: jmap`, a directory of `.ics` files) are skipped. - `password_command` runs a program for the password, and `password_keyring` looks it up in the system keyring. - `CALDAV_*` environment variables override the section's keys one by one: a `CALDAV_PASSWORD` alone no longer makes caldav ignore the config file (https://github.com/python-caldav/caldav/issues/780). `true`/`false` in a variable is a boolean now, and `CALDAV_HEADERS` is read as JSON. - `extract_conn_params_from_section()`, which plann uses, reads the new format too; pass it the config file's path as `config_file` to let a `password_command` run, and the section's name as `section_name` for error messages. -- An unknown key in a section is logged as a warning. `test_*`, `x_*` and `extra_config` are reserved for test tooling and applications. - -Some files are read differently: an empty file found first is used, rather than skipped for the next one; a section named directly that is missing or disabled, or a glob that matches nothing, is an error; a name in `contains` that matches nothing gives a warning; a disabled meta-section leaves out its members too; `*` is only the sections that are not meta-sections; and a config file named explicitly must exist. +- An unknown key in a section is logged as a warning. `x_*` and `extra_config` are reserved for applications. diff --git a/changes/+config-safe-yaml.security.md b/changes/+config-safe-yaml.fixed.md similarity index 100% rename from changes/+config-safe-yaml.security.md rename to changes/+config-safe-yaml.fixed.md diff --git a/changes/+hints-2026-10-20.changed.md b/changes/+hints-2026-10-20.changed.md index 1df48c0c..3050cf00 100644 --- a/changes/+hints-2026-10-20.changed.md +++ b/changes/+hints-2026-10-20.changed.md @@ -1,7 +1,10 @@ -`compatibility_hints`: several server profiles are updated from a caldav-server-tester run, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. +Server compatibility: several server profiles in `compatibility_hints` are updated from caldav-server-tester runs, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. -- `calendar-color` (a hex value) and `calendar-order` are measured instead of left at the default `fragile`: both are `full`, and colour names work too, for Xandikos, Nextcloud (and so ecloud, which inherits its profile), CCS, Stalwart, Cyrus, Posteo and Robur; Bedework 3 stores both colour forms but not `calendar-order`; SOGo takes a hex colour but not a colour name; OX App Suite takes a hex colour, ignores colour names, and does not store `calendar-order`. +- `calendar-color` (a hex value) and `calendar-order` are measured instead of left at the default `fragile`: both are `full`, and colour names work too, for Xandikos, Nextcloud (and so ecloud, which inherits its profile), CCS, Stalwart, Cyrus, Posteo, Purelymail and Robur; Bedework 3 stores both colour forms but not `calendar-order`; SOGo takes a hex colour but not a colour name; OX App Suite takes a hex colour, ignores colour names, and does not store `calendar-order`. +- Sync tokens: DAViCal, SOGo and Bedework 3 now state `sync-token.invalid-token`, `sync-token.limit` and `sync-token.limit.initial-sync`, and Bedework 5 the two `limit` features, where they used to inherit `fragile` from `sync-token` or default to `full`. DAViCal and SOGo ignore `DAV:limit` and answer a never-issued sync token with a 207 and no error. Bedework 3 refuses a token it never issued, as it should, and answers a `DAV:limit` with a 500. Bedework 5 cuts the reply at `DAV:limit` without the 507 the RFC asks for: following the tokens loses nothing, but a client takes the cut reply as complete and gets the rest only on its next sync. - OX App Suite: `scheduling` is `full` (it was `unknown`), with `scheduling.schedule-tag` unsupported; inbox delivery and auto-scheduling stay `unknown`. +- Purelymail: `search.is-not-defined`, `search.time-range.event` and `search.time-range.todo` are `full` again (they were `fragile`). - Stalwart: `search.time-range.todo.no-dtstart.no-due` is `unsupported`. - GMX: `search.recurrences.includes-implicit.infinite-scope` is `fragile` (it was `full`): it failed in one run and passed in the next. +- Infomaniak: the wait after deleting a calendar (`synchronous-write.delete-calendar`) is raised from 12 to 20 seconds; a deleted calendar could still be there after 12. - `well-known`: `unsupported` for Posteo, `full` for Robur. diff --git a/changes/+hints-sync-token-probes.changed.md b/changes/+hints-sync-token-probes.changed.md deleted file mode 100644 index c54987b0..00000000 --- a/changes/+hints-sync-token-probes.changed.md +++ /dev/null @@ -1,5 +0,0 @@ -`compatibility_hints`: several server profiles are updated from a caldav-server-tester run, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. - -- DAViCal, SOGo and Bedework 3 now state `sync-token.invalid-token`, `sync-token.limit` and `sync-token.limit.initial-sync`, and Bedework 5 the two `limit` features, where they used to inherit `fragile` from `sync-token` or default to `full`. DAViCal and SOGo ignore `DAV:limit` and answer a never-issued sync token with a 207 and no error. Bedework 3 refuses a token it never issued, as it should, and answers a `DAV:limit` with a 500. Bedework 5 cuts the reply at `DAV:limit` without the 507 the RFC asks for: following the tokens loses nothing, but a client takes the cut reply as complete and gets the rest only on its next sync. -- Purelymail: `search.is-not-defined`, `search.time-range.event` and `search.time-range.todo` are `full` again (they were `fragile`), and `calendar-color` and `calendar-order` are `full`. -- Infomaniak: the wait after deleting a calendar (`synchronous-write.delete-calendar`) is raised from 12 to 20 seconds; a deleted calendar could still be there after 12. diff --git a/changes/+no-overwrite-if-none-match.changed.md b/changes/+no-overwrite-if-none-match.changed.md deleted file mode 100644 index 7a7f09c3..00000000 --- a/changes/+no-overwrite-if-none-match.changed.md +++ /dev/null @@ -1 +0,0 @@ -`save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) have improved handling on RFC-compliant servers: the server enforces the flag at the same URL in the same request as the write, so another client can no longer slip in between the check and the PUT there. A refusal still raises `ConsistencyError`. When the refusal is because another object (at another URL) holds the UID, it is the subclass `UidConflictConsistencyError`, whether the server refused the UID or caldav's own UID search found it (https://github.com/python-caldav/caldav/issues/806). diff --git a/changes/+wildcard-preconditions.breaking.md b/changes/+wildcard-preconditions.breaking.md index d15b7ef3..25d75717 100644 --- a/changes/+wildcard-preconditions.breaking.md +++ b/changes/+wildcard-preconditions.breaking.md @@ -1,6 +1,6 @@ -Potentially breaking, on servers that are themselves broken: `save(no_overwrite=True)` and `save(no_create=True)` now rely on the server honouring `If-None-Match: *` and `If-Match: *`, which every CalDAV server is required to do. Against a server that does not, an existing object may be overwritten, a missing one created, or a legitimate update refused. The fix is to tell the client that the server is broken: configure it with the server's `features` (e.g. `features: sogo`), or mark `save.if-none-match` and `save-load.mutable.if-match-wildcard` correctly in a hand-written feature list. Servers known not to honour them, as of this release: +`save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) now send `If-None-Match: *` and `If-Match: *`, so on an RFC-compliant server the flag is enforced at the same URL in the same request as the write, and another client can no longer slip in between the check and the PUT there. Potentially breaking, on servers that are themselves broken: this relies on the server honouring the wildcard preconditions, which every CalDAV server is required to do. Against a server that does not, an existing object may be overwritten, a missing one created, or a legitimate update refused. The fix is to tell the client that the server is broken: configure it with the server's `features` (e.g. `features: sogo`), or mark `save.if-none-match` and `save-load.mutable.if-match-wildcard` correctly in a hand-written feature list. Servers known not to honour them, as of this release: - `If-None-Match: *`: SOGo, Zimbra - `If-Match: *`: Bedework, GMX, iCloud, Posteo, Radicale, SOGo, Yahoo, Zimbra -A subclass overriding `_create()` or `_put()` must accept their new `wildcard=` keyword. +A refusal still raises `ConsistencyError`. When the refusal is because another object (at another URL) holds the UID, it is the new `UidConflictConsistencyError`, which is both a `UidConflictError` and a `ConsistencyError`, whether the server refused the UID or caldav's own UID search found it (https://github.com/python-caldav/caldav/issues/806). A subclass overriding `_create()` or `_put()` must accept their new `wildcard=` keyword. diff --git a/changes/794.fixed.md b/changes/794.fixed.md index c59706d5..9aadf911 100644 --- a/changes/794.fixed.md +++ b/changes/794.fixed.md @@ -1 +1 @@ -Creating an `AsyncDAVClient` failed with an `ImportError` when only an httpx-family library was installed. RFC 6764 discovery still needs niquests or requests, and is skipped with a warning without them. `aio.get_davclient()` and `aio.get_calendars()` now read the config file and do RFC 6764 discovery in a worker thread, so they no longer block the event loop. +Creating an `AsyncDAVClient` failed with an `ImportError` when only an httpx-family library was installed. It works now; RFC 6764 discovery still needs niquests or requests, and is skipped with a warning without them. `aio.get_davclient()` and `aio.get_calendars()` blocked the event loop while reading the config file and doing RFC 6764 discovery; they now do both in a worker thread. From 3f8013dca96a6c0260bc57c801e6a799fd70a9cc Mon Sep 17 00:00:00 2001 From: Tobias Brox Date: Thu, 22 Oct 2026 09:11:03 +0200 Subject: [PATCH 2/6] docs: collect the changelog fragments for 3.6.0 The output of towncrier build, unedited, so the hand-tuning that follows is a diff of its own. Prompt: We should make a new release today. Check up the file tmp-handover.md [an untracked note with changelog decisions agreed in an earlier session] Assisted-By: Claude Opus 5.5 (1M context) Reviewed-by: Tobias Brox --- CHANGELOG.md | 62 +++++++++++++++++++++ changes/+calendar-color-name.changed.md | 1 - changes/+calendaring-config.breaking.md | 1 - changes/+calendaring-config.changed.md | 8 --- changes/+config-safe-yaml.fixed.md | 1 - changes/+copy-feature-set.deprecated.md | 1 - changes/+folded-long-uid.fixed.md | 1 - changes/+hints-2026-10-20.changed.md | 10 ---- changes/+put-200-ok.fixed.md | 1 - changes/+wildcard-preconditions.breaking.md | 6 -- changes/585.added.md | 1 - changes/585.removed.md | 1 - changes/598.added.md | 1 - changes/763.added.md | 1 - changes/782.fixed.md | 1 - changes/786.deprecated.md | 1 - changes/793.fixed.md | 1 - changes/794.fixed.md | 1 - changes/795.fixed.md | 1 - changes/797.fixed.md | 1 - changes/799.changed.md | 1 - changes/805.added.md | 1 - 22 files changed, 62 insertions(+), 42 deletions(-) delete mode 100644 changes/+calendar-color-name.changed.md delete mode 100644 changes/+calendaring-config.breaking.md delete mode 100644 changes/+calendaring-config.changed.md delete mode 100644 changes/+config-safe-yaml.fixed.md delete mode 100644 changes/+copy-feature-set.deprecated.md delete mode 100644 changes/+folded-long-uid.fixed.md delete mode 100644 changes/+hints-2026-10-20.changed.md delete mode 100644 changes/+put-200-ok.fixed.md delete mode 100644 changes/+wildcard-preconditions.breaking.md delete mode 100644 changes/585.added.md delete mode 100644 changes/585.removed.md delete mode 100644 changes/598.added.md delete mode 100644 changes/763.added.md delete mode 100644 changes/782.fixed.md delete mode 100644 changes/786.deprecated.md delete mode 100644 changes/793.fixed.md delete mode 100644 changes/794.fixed.md delete mode 100644 changes/795.fixed.md delete mode 100644 changes/797.fixed.md delete mode 100644 changes/799.changed.md delete mode 100644 changes/805.added.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 824612a7..a47194cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,68 @@ After 3.5.0, changes since the last release are kept as fragments in [changes/]( +## [3.6.0] - 2026-10-22 + +### Breaking changes + +* Since the config file is now read through calendaring-config (see Changed), some files are read differently: an empty file found first is used, rather than skipped for the next one; a section named directly that is missing or disabled, or a glob that matches nothing, is an error; a name in `contains` that matches nothing gives a warning; a disabled meta-section leaves out its members too; `*` is only the sections that are not meta-sections; and a config file named explicitly must exist. +* `save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) now send `If-None-Match: *` and `If-Match: *`, so on an RFC-compliant server the flag is enforced at the same URL in the same request as the write, and another client can no longer slip in between the check and the PUT there. Potentially breaking, on servers that are themselves broken: this relies on the server honouring the wildcard preconditions, which every CalDAV server is required to do. Against a server that does not, an existing object may be overwritten, a missing one created, or a legitimate update refused. The fix is to tell the client that the server is broken: configure it with the server's `features` (e.g. `features: sogo`), or mark `save.if-none-match` and `save-load.mutable.if-match-wildcard` correctly in a hand-written feature list. Servers known not to honour them, as of this release: + + - `If-None-Match: *`: SOGo, Zimbra + - `If-Match: *`: Bedework, GMX, iCloud, Posteo, Radicale, SOGo, Yahoo, Zimbra + + A refusal still raises `ConsistencyError`. When the refusal is because another object (at another URL) holds the UID, it is the new `UidConflictConsistencyError`, which is both a `UidConflictError` and a `ConsistencyError`, whether the server refused the UID or caldav's own UID search found it (https://github.com/python-caldav/caldav/issues/806). A subclass overriding `_create()` or `_put()` must accept their new `wildcard=` keyword. + +### Added + +* Compatibility features `save-load.todo.uncomplete` (a completed task can be made pending again with `Todo.uncomplete()`; GMX keeps it completed) and `search.time-range.todo.no-dtstart.no-due` (a date search finds a task with neither DTSTART nor DUE, as RFC 4791 section 9.9 says it should; Zimbra does not). They replace the last two old-style flags in the Zimbra and GMX profiles. https://github.com/python-caldav/caldav/issues/585 +* `cancel()` marks an object, or a single occurrence from `search(..., expand=True)`, as `STATUS:CANCELLED` and saves it - for an occurrence that was planned but will not take place. A new compatibility feature, `save-load.event.recurrences.exception.cancelled`, records that Zimbra turns a cancelled occurrence into an `EXDATE`. https://github.com/python-caldav/caldav/issues/598 +* Calendar objects keep the calendar data exactly as the server sent it, before `vcal.fix()` or any other normalisation, in the new read-only `raw_data` attribute (`bytes`; `None` for objects not read from a server, copies, and after `save()`). After `load()` it is the GET response body byte for byte; from a REPORT (`search()`, `multiget()`) it is the text of the `calendar-data` element, with XML entities decoded and line breaks turned into LF. `save(raw_data=...)` sends the given (UTF-8) bytes as the PUT body unchanged - no `vcal.fix()`, no re-serialisation, no `SEQUENCE` bump - still with `If-Match` / `If-Schedule-Tag-Match`. This lets sync tools compare and copy what servers store. https://github.com/python-caldav/caldav/issues/763 +* `DAVError` and its subclasses have a new `status` attribute with the HTTP status code of the failed response (or `None`), so an `AuthorizationError` tells a 401 (bad or missing credentials) from a 403 (authenticated, but not allowed). https://github.com/python-caldav/caldav/issues/805 + +### Changed + +* `DAVClient.password` is now the `str` that was passed in, as on `AsyncDAVClient`; it used to be encoded to UTF-8 `bytes`. The return type of `Calendar.multiget()` is annotated as a list, or a coroutine of one for an async client, like `search()`. https://github.com/python-caldav/caldav/issues/799 +* Server compatibility: several server profiles in `compatibility_hints` are updated from caldav-server-tester runs, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. + + - `calendar-color` (a hex value) and `calendar-order` are measured instead of left at the default `fragile`: both are `full`, and colour names work too, for Xandikos, Nextcloud (and so ecloud, which inherits its profile), CCS, Stalwart, Cyrus, Posteo, Purelymail and Robur; Bedework 3 stores both colour forms but not `calendar-order`; SOGo takes a hex colour but not a colour name; OX App Suite takes a hex colour, ignores colour names, and does not store `calendar-order`. + - Sync tokens: DAViCal, SOGo and Bedework 3 now state `sync-token.invalid-token`, `sync-token.limit` and `sync-token.limit.initial-sync`, and Bedework 5 the two `limit` features, where they used to inherit `fragile` from `sync-token` or default to `full`. DAViCal and SOGo ignore `DAV:limit` and answer a never-issued sync token with a 207 and no error. Bedework 3 refuses a token it never issued, as it should, and answers a `DAV:limit` with a 500. Bedework 5 cuts the reply at `DAV:limit` without the 507 the RFC asks for: following the tokens loses nothing, but a client takes the cut reply as complete and gets the rest only on its next sync. + - OX App Suite: `scheduling` is `full` (it was `unknown`), with `scheduling.schedule-tag` unsupported; inbox delivery and auto-scheduling stay `unknown`. + - Purelymail: `search.is-not-defined`, `search.time-range.event` and `search.time-range.todo` are `full` again (they were `fragile`). + - Stalwart: `search.time-range.todo.no-dtstart.no-due` is `unsupported`. + - GMX: `search.recurrences.includes-implicit.infinite-scope` is `fragile` (it was `full`): it failed in one run and passed in the next. + - Infomaniak: the wait after deleting a calendar (`synchronous-write.delete-calendar`) is raised from 12 to 20 seconds; a deleted calendar could still be there after 12. + - `well-known`: `unsupported` for Posteo, `full` for Robur. +* The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786. This is a new dependency, `calendaring-config>=0.3` (it depends only on PyYAML), which matters to packagers. New in the format: + + - `~/.config/pycal/calendar.yaml` is searched first, before the old paths. + - A section may say `kind: caldav` and drop the `caldav_` prefix. Sections of other kinds (`kind: jmap`, a directory of `.ics` files) are skipped. + - `password_command` runs a program for the password, and `password_keyring` looks it up in the system keyring. + - `CALDAV_*` environment variables override the section's keys one by one: a `CALDAV_PASSWORD` alone no longer makes caldav ignore the config file (https://github.com/python-caldav/caldav/issues/780). `true`/`false` in a variable is a boolean now, and `CALDAV_HEADERS` is read as JSON. + - `extract_conn_params_from_section()`, which plann uses, reads the new format too; pass it the config file's path as `config_file` to let a `password_command` run, and the section's name as `section_name` for error messages. + - An unknown key in a section is logged as a warning. `x_*` and `extra_config` are reserved for applications. +* `compatibility_hints`: `calendar-color` is now about a hex value such as `#FF0000FF`, the format clients use in practice. Setting the colour by name (`blue`) is the new `calendar-color.name`; a server that rejects or ignores colour names is `unsupported` there, not `broken`. Zimbra, SOGo, iCloud and OX App Suite are therefore `full` for `calendar-color` and `unsupported` for `calendar-color.name`. For Synology, Purelymail, GMX and Infomaniak only colour names were measured, so their hex support is assumed. `calendar-color.hex` is deprecated: a configuration setting it still works and gives a DeprecationWarning, and its `calendar-color` is then read as `calendar-color.name`. + +### Deprecated + +* Config file keys starting with `test_` are now reserved for test tooling, so other programs sharing the config file can drop them. A section is marked for testing with `test_allowed: true` rather than `testing_allowed: true`, and the order test servers are tried in is set with `test_priority` rather than `priority`. The old names still work, but emit a `DeprecationWarning`. `test_priority` (or `priority`) in a section of the main config file was ignored before; it is now honoured. https://github.com/python-caldav/caldav/issues/786 +* `FeatureSet.copyFeatureSet()` in `caldav.compatibility_hints` has been renamed to `copy_feature_set()`, in line with the naming of the other methods. The old name still works but emits a `DeprecationWarning`. + +### Removed + +* The old-style compatibility flags are gone: `caldav.compatibility_hints.incompatibility_description`, and the `old_flags` key in a feature set. They were only ever used by the test suite to skip or adjust tests, and every flag still in use is now a compatibility feature. https://github.com/python-caldav/caldav/issues/585 + +### Fixed + +* `complete(handle_rrule=True)` on a recurring task with fixed times (`BY*` parts in the `RRULE`, e.g. `FREQ=YEARLY;BYMONTH=4,9`) did not advance the task when it was completed before its `DTSTART`: the next occurrence was looked for after the completion time, which found the occurrence just completed. It is now looked for after `DTSTART`. https://github.com/python-caldav/caldav/issues/782 +* `aio.get_calendars()` ignored meta-sections, `*` and glob patterns in `config_section`, as well as the `calendar_name` and `calendar_url` filters of a config section. It now finds the same calendars as the sync `get_calendars()`, and behaves like it on errors too: it no longer sends an OPTIONS probe first, so an unreachable server raises the error from the principal lookup rather than a `DAVError` saying "Failed to connect", and with `raise_errors=False` such an error is logged and the server skipped rather than raised. https://github.com/python-caldav/caldav/issues/793 +* Creating an `AsyncDAVClient` failed with an `ImportError` when only an httpx-family library was installed. It works now; RFC 6764 discovery still needs niquests or requests, and is skipped with a warning without them. `aio.get_davclient()` and `aio.get_calendars()` blocked the event loop while reading the config file and doing RFC 6764 discovery; they now do both in a worker thread. https://github.com/python-caldav/caldav/issues/794 +* The `supported-report-set` and `current-user-privilege-set` properties from `propfind()` were the raw lxml element. They are now lists of the report and privilege names in Clark notation, e.g. `["{urn:ietf:params:xml:ns:caldav}calendar-query", "{DAV:}sync-collection"]`. Code that walked the returned element has to read the list instead. https://github.com/python-caldav/caldav/issues/795 +* `caldav.testing`: `RadicaleServer` ignored its configured port, so Radicale always listened on 5232 and any other port timed out at startup. `RadicaleServer.stop()` now closes both ends of its shutdown socketpair, and `XandikosServer.stop()` closes its event loop, so neither leaves a `ResourceWarning` behind. https://github.com/python-caldav/caldav/issues/797 +* A `200 OK` reply to the PUT of `save()` is now taken as success (RFC 9110 section 9.3.4 allows it for an overwrite). Before, it was retried, and the retry raised `PutError` when the server answered `200` again. +* A long UID may be folded over several lines, and for an object that was not parsed yet `.id` returned only the first line. As a result `save(no_create=True)` of such an existing object raised `ConsistencyError`, `save(no_overwrite=True)` overwrote it instead of raising, and `save(increase_seqno=False, only_this_recurrence=False)` wrote the cut UID back to the server. `.id` now returns the whole UID, and a UID with a backslash escape such as `\,` is unescaped there too. +* Configuration files are now read with PyYAML's safe loader. The full loader was used before, so a YAML tag such as `!!python/object/apply:` in a config file could run arbitrary Python code when `get_davclient()` or `get_calendars()` read it - also from the system-wide `/etc/calendar.conf`. A config file using such tags is now refused with a `ValueError`; plain YAML and JSON files are unaffected. + ## [3.5.0] - 2026-10-15 Yet another release primarily focusing on bugfixes and compatibility. iCloud has now been tested and got its compatibility profile in `compatibility_hints.py`. Test runs towards GMX, Posteo and Purelymail have been skipped for the last few releases; with 3.5.0 the tests pass towards those. More work on getting synchronization right, and on avoiding overwriting changes when multiple clients update the same object. diff --git a/changes/+calendar-color-name.changed.md b/changes/+calendar-color-name.changed.md deleted file mode 100644 index 9479f280..00000000 --- a/changes/+calendar-color-name.changed.md +++ /dev/null @@ -1 +0,0 @@ -`compatibility_hints`: `calendar-color` is now about a hex value such as `#FF0000FF`, the format clients use in practice. Setting the colour by name (`blue`) is the new `calendar-color.name`; a server that rejects or ignores colour names is `unsupported` there, not `broken`. Zimbra, SOGo, iCloud and OX App Suite are therefore `full` for `calendar-color` and `unsupported` for `calendar-color.name`. For Synology, Purelymail, GMX and Infomaniak only colour names were measured, so their hex support is assumed. `calendar-color.hex` is deprecated: a configuration setting it still works and gives a DeprecationWarning, and its `calendar-color` is then read as `calendar-color.name`. diff --git a/changes/+calendaring-config.breaking.md b/changes/+calendaring-config.breaking.md deleted file mode 100644 index 24f998d2..00000000 --- a/changes/+calendaring-config.breaking.md +++ /dev/null @@ -1 +0,0 @@ -Since the config file is now read through calendaring-config (see Changed), some files are read differently: an empty file found first is used, rather than skipped for the next one; a section named directly that is missing or disabled, or a glob that matches nothing, is an error; a name in `contains` that matches nothing gives a warning; a disabled meta-section leaves out its members too; `*` is only the sections that are not meta-sections; and a config file named explicitly must exist. diff --git a/changes/+calendaring-config.changed.md b/changes/+calendaring-config.changed.md deleted file mode 100644 index 1817a45d..00000000 --- a/changes/+calendaring-config.changed.md +++ /dev/null @@ -1,8 +0,0 @@ -The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786. This is a new dependency, `calendaring-config>=0.3` (it depends only on PyYAML), which matters to packagers. New in the format: - -- `~/.config/pycal/calendar.yaml` is searched first, before the old paths. -- A section may say `kind: caldav` and drop the `caldav_` prefix. Sections of other kinds (`kind: jmap`, a directory of `.ics` files) are skipped. -- `password_command` runs a program for the password, and `password_keyring` looks it up in the system keyring. -- `CALDAV_*` environment variables override the section's keys one by one: a `CALDAV_PASSWORD` alone no longer makes caldav ignore the config file (https://github.com/python-caldav/caldav/issues/780). `true`/`false` in a variable is a boolean now, and `CALDAV_HEADERS` is read as JSON. -- `extract_conn_params_from_section()`, which plann uses, reads the new format too; pass it the config file's path as `config_file` to let a `password_command` run, and the section's name as `section_name` for error messages. -- An unknown key in a section is logged as a warning. `x_*` and `extra_config` are reserved for applications. diff --git a/changes/+config-safe-yaml.fixed.md b/changes/+config-safe-yaml.fixed.md deleted file mode 100644 index f1fbd98a..00000000 --- a/changes/+config-safe-yaml.fixed.md +++ /dev/null @@ -1 +0,0 @@ -Configuration files are now read with PyYAML's safe loader. The full loader was used before, so a YAML tag such as `!!python/object/apply:` in a config file could run arbitrary Python code when `get_davclient()` or `get_calendars()` read it - also from the system-wide `/etc/calendar.conf`. A config file using such tags is now refused with a `ValueError`; plain YAML and JSON files are unaffected. diff --git a/changes/+copy-feature-set.deprecated.md b/changes/+copy-feature-set.deprecated.md deleted file mode 100644 index 8cdfc4f2..00000000 --- a/changes/+copy-feature-set.deprecated.md +++ /dev/null @@ -1 +0,0 @@ -`FeatureSet.copyFeatureSet()` in `caldav.compatibility_hints` has been renamed to `copy_feature_set()`, in line with the naming of the other methods. The old name still works but emits a `DeprecationWarning`. diff --git a/changes/+folded-long-uid.fixed.md b/changes/+folded-long-uid.fixed.md deleted file mode 100644 index 9adff68f..00000000 --- a/changes/+folded-long-uid.fixed.md +++ /dev/null @@ -1 +0,0 @@ -A long UID may be folded over several lines, and for an object that was not parsed yet `.id` returned only the first line. As a result `save(no_create=True)` of such an existing object raised `ConsistencyError`, `save(no_overwrite=True)` overwrote it instead of raising, and `save(increase_seqno=False, only_this_recurrence=False)` wrote the cut UID back to the server. `.id` now returns the whole UID, and a UID with a backslash escape such as `\,` is unescaped there too. diff --git a/changes/+hints-2026-10-20.changed.md b/changes/+hints-2026-10-20.changed.md deleted file mode 100644 index 3050cf00..00000000 --- a/changes/+hints-2026-10-20.changed.md +++ /dev/null @@ -1,10 +0,0 @@ -Server compatibility: several server profiles in `compatibility_hints` are updated from caldav-server-tester runs, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. - -- `calendar-color` (a hex value) and `calendar-order` are measured instead of left at the default `fragile`: both are `full`, and colour names work too, for Xandikos, Nextcloud (and so ecloud, which inherits its profile), CCS, Stalwart, Cyrus, Posteo, Purelymail and Robur; Bedework 3 stores both colour forms but not `calendar-order`; SOGo takes a hex colour but not a colour name; OX App Suite takes a hex colour, ignores colour names, and does not store `calendar-order`. -- Sync tokens: DAViCal, SOGo and Bedework 3 now state `sync-token.invalid-token`, `sync-token.limit` and `sync-token.limit.initial-sync`, and Bedework 5 the two `limit` features, where they used to inherit `fragile` from `sync-token` or default to `full`. DAViCal and SOGo ignore `DAV:limit` and answer a never-issued sync token with a 207 and no error. Bedework 3 refuses a token it never issued, as it should, and answers a `DAV:limit` with a 500. Bedework 5 cuts the reply at `DAV:limit` without the 507 the RFC asks for: following the tokens loses nothing, but a client takes the cut reply as complete and gets the rest only on its next sync. -- OX App Suite: `scheduling` is `full` (it was `unknown`), with `scheduling.schedule-tag` unsupported; inbox delivery and auto-scheduling stay `unknown`. -- Purelymail: `search.is-not-defined`, `search.time-range.event` and `search.time-range.todo` are `full` again (they were `fragile`). -- Stalwart: `search.time-range.todo.no-dtstart.no-due` is `unsupported`. -- GMX: `search.recurrences.includes-implicit.infinite-scope` is `fragile` (it was `full`): it failed in one run and passed in the next. -- Infomaniak: the wait after deleting a calendar (`synchronous-write.delete-calendar`) is raised from 12 to 20 seconds; a deleted calendar could still be there after 12. -- `well-known`: `unsupported` for Posteo, `full` for Robur. diff --git a/changes/+put-200-ok.fixed.md b/changes/+put-200-ok.fixed.md deleted file mode 100644 index 7a0b3af4..00000000 --- a/changes/+put-200-ok.fixed.md +++ /dev/null @@ -1 +0,0 @@ -A `200 OK` reply to the PUT of `save()` is now taken as success (RFC 9110 section 9.3.4 allows it for an overwrite). Before, it was retried, and the retry raised `PutError` when the server answered `200` again. diff --git a/changes/+wildcard-preconditions.breaking.md b/changes/+wildcard-preconditions.breaking.md deleted file mode 100644 index 25d75717..00000000 --- a/changes/+wildcard-preconditions.breaking.md +++ /dev/null @@ -1,6 +0,0 @@ -`save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) now send `If-None-Match: *` and `If-Match: *`, so on an RFC-compliant server the flag is enforced at the same URL in the same request as the write, and another client can no longer slip in between the check and the PUT there. Potentially breaking, on servers that are themselves broken: this relies on the server honouring the wildcard preconditions, which every CalDAV server is required to do. Against a server that does not, an existing object may be overwritten, a missing one created, or a legitimate update refused. The fix is to tell the client that the server is broken: configure it with the server's `features` (e.g. `features: sogo`), or mark `save.if-none-match` and `save-load.mutable.if-match-wildcard` correctly in a hand-written feature list. Servers known not to honour them, as of this release: - -- `If-None-Match: *`: SOGo, Zimbra -- `If-Match: *`: Bedework, GMX, iCloud, Posteo, Radicale, SOGo, Yahoo, Zimbra - -A refusal still raises `ConsistencyError`. When the refusal is because another object (at another URL) holds the UID, it is the new `UidConflictConsistencyError`, which is both a `UidConflictError` and a `ConsistencyError`, whether the server refused the UID or caldav's own UID search found it (https://github.com/python-caldav/caldav/issues/806). A subclass overriding `_create()` or `_put()` must accept their new `wildcard=` keyword. diff --git a/changes/585.added.md b/changes/585.added.md deleted file mode 100644 index b94819a5..00000000 --- a/changes/585.added.md +++ /dev/null @@ -1 +0,0 @@ -Compatibility features `save-load.todo.uncomplete` (a completed task can be made pending again with `Todo.uncomplete()`; GMX keeps it completed) and `search.time-range.todo.no-dtstart.no-due` (a date search finds a task with neither DTSTART nor DUE, as RFC 4791 section 9.9 says it should; Zimbra does not). They replace the last two old-style flags in the Zimbra and GMX profiles. diff --git a/changes/585.removed.md b/changes/585.removed.md deleted file mode 100644 index c92da999..00000000 --- a/changes/585.removed.md +++ /dev/null @@ -1 +0,0 @@ -The old-style compatibility flags are gone: `caldav.compatibility_hints.incompatibility_description`, and the `old_flags` key in a feature set. They were only ever used by the test suite to skip or adjust tests, and every flag still in use is now a compatibility feature. diff --git a/changes/598.added.md b/changes/598.added.md deleted file mode 100644 index 9f6f6fd6..00000000 --- a/changes/598.added.md +++ /dev/null @@ -1 +0,0 @@ -`cancel()` marks an object, or a single occurrence from `search(..., expand=True)`, as `STATUS:CANCELLED` and saves it - for an occurrence that was planned but will not take place. A new compatibility feature, `save-load.event.recurrences.exception.cancelled`, records that Zimbra turns a cancelled occurrence into an `EXDATE`. diff --git a/changes/763.added.md b/changes/763.added.md deleted file mode 100644 index 95f4c104..00000000 --- a/changes/763.added.md +++ /dev/null @@ -1 +0,0 @@ -Calendar objects keep the calendar data exactly as the server sent it, before `vcal.fix()` or any other normalisation, in the new read-only `raw_data` attribute (`bytes`; `None` for objects not read from a server, copies, and after `save()`). After `load()` it is the GET response body byte for byte; from a REPORT (`search()`, `multiget()`) it is the text of the `calendar-data` element, with XML entities decoded and line breaks turned into LF. `save(raw_data=...)` sends the given (UTF-8) bytes as the PUT body unchanged - no `vcal.fix()`, no re-serialisation, no `SEQUENCE` bump - still with `If-Match` / `If-Schedule-Tag-Match`. This lets sync tools compare and copy what servers store. diff --git a/changes/782.fixed.md b/changes/782.fixed.md deleted file mode 100644 index ad2be0f2..00000000 --- a/changes/782.fixed.md +++ /dev/null @@ -1 +0,0 @@ -`complete(handle_rrule=True)` on a recurring task with fixed times (`BY*` parts in the `RRULE`, e.g. `FREQ=YEARLY;BYMONTH=4,9`) did not advance the task when it was completed before its `DTSTART`: the next occurrence was looked for after the completion time, which found the occurrence just completed. It is now looked for after `DTSTART`. diff --git a/changes/786.deprecated.md b/changes/786.deprecated.md deleted file mode 100644 index b68245a2..00000000 --- a/changes/786.deprecated.md +++ /dev/null @@ -1 +0,0 @@ -Config file keys starting with `test_` are now reserved for test tooling, so other programs sharing the config file can drop them. A section is marked for testing with `test_allowed: true` rather than `testing_allowed: true`, and the order test servers are tried in is set with `test_priority` rather than `priority`. The old names still work, but emit a `DeprecationWarning`. `test_priority` (or `priority`) in a section of the main config file was ignored before; it is now honoured. diff --git a/changes/793.fixed.md b/changes/793.fixed.md deleted file mode 100644 index c4cac8fe..00000000 --- a/changes/793.fixed.md +++ /dev/null @@ -1 +0,0 @@ -`aio.get_calendars()` ignored meta-sections, `*` and glob patterns in `config_section`, as well as the `calendar_name` and `calendar_url` filters of a config section. It now finds the same calendars as the sync `get_calendars()`, and behaves like it on errors too: it no longer sends an OPTIONS probe first, so an unreachable server raises the error from the principal lookup rather than a `DAVError` saying "Failed to connect", and with `raise_errors=False` such an error is logged and the server skipped rather than raised. diff --git a/changes/794.fixed.md b/changes/794.fixed.md deleted file mode 100644 index 9aadf911..00000000 --- a/changes/794.fixed.md +++ /dev/null @@ -1 +0,0 @@ -Creating an `AsyncDAVClient` failed with an `ImportError` when only an httpx-family library was installed. It works now; RFC 6764 discovery still needs niquests or requests, and is skipped with a warning without them. `aio.get_davclient()` and `aio.get_calendars()` blocked the event loop while reading the config file and doing RFC 6764 discovery; they now do both in a worker thread. diff --git a/changes/795.fixed.md b/changes/795.fixed.md deleted file mode 100644 index 8f330771..00000000 --- a/changes/795.fixed.md +++ /dev/null @@ -1 +0,0 @@ -The `supported-report-set` and `current-user-privilege-set` properties from `propfind()` were the raw lxml element. They are now lists of the report and privilege names in Clark notation, e.g. `["{urn:ietf:params:xml:ns:caldav}calendar-query", "{DAV:}sync-collection"]`. Code that walked the returned element has to read the list instead. diff --git a/changes/797.fixed.md b/changes/797.fixed.md deleted file mode 100644 index d391d52e..00000000 --- a/changes/797.fixed.md +++ /dev/null @@ -1 +0,0 @@ -`caldav.testing`: `RadicaleServer` ignored its configured port, so Radicale always listened on 5232 and any other port timed out at startup. `RadicaleServer.stop()` now closes both ends of its shutdown socketpair, and `XandikosServer.stop()` closes its event loop, so neither leaves a `ResourceWarning` behind. diff --git a/changes/799.changed.md b/changes/799.changed.md deleted file mode 100644 index 7b470f43..00000000 --- a/changes/799.changed.md +++ /dev/null @@ -1 +0,0 @@ -`DAVClient.password` is now the `str` that was passed in, as on `AsyncDAVClient`; it used to be encoded to UTF-8 `bytes`. The return type of `Calendar.multiget()` is annotated as a list, or a coroutine of one for an async client, like `search()`. diff --git a/changes/805.added.md b/changes/805.added.md deleted file mode 100644 index 81d4d627..00000000 --- a/changes/805.added.md +++ /dev/null @@ -1 +0,0 @@ -`DAVError` and its subclasses have a new `status` attribute with the HTTP status code of the failed response (or `None`), so an `AuthorizationError` tells a 401 (bad or missing credentials) from a 403 (authenticated, but not allowed). From 1d7a0780ddcbc9708e11fb34289d9aee22f62a24 Mon Sep 17 00:00:00 2001 From: Tobias Brox Date: Thu, 22 Oct 2026 09:11:40 +0200 Subject: [PATCH 3/6] docs: hand-tune the 3.6.0 changelog An intro paragraph like the one for 3.5.0, and the sections laid out like 3.5.0's: potentially breaking changes, then Added split into library API, configuration and server compatibility. The DAVClient.password/multiget entry moves to potentially breaking changes, and the "Notable changes" note no longer points at a 3.4 regression. Prompt: We should make a new release today. Check up the file tmp-handover.md [an untracked note with changelog decisions agreed in an earlier session] Assisted-By: Claude Opus 5.5 (1M context) Reviewed-by: Tobias Brox --- CHANGELOG.md | 45 +++++++++++++++++++++++++++------------------ 1 file changed, 27 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a47194cb..d728ed16 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,11 +8,9 @@ As of v3.x, **niquests** is the supported and recommended package for HTTP commu ### Configuration and "features" -This is "old news", but the 3.4-release may hold a regression making this more relevant (see "Breaking changes" for the 3.4-release below). - When configuring the server it's now possible to pass a `features`-parameter. This may contain a dict describing the server capabilities, but for end-users the intended usage pattern is to pass a server name, e.g. `features: cyrus`. -Support for configuring things like server credentials through a **configuration file** or through **environment** was added a while ago - the idea being that independent applications built using the caldav library can share the same configuration file / environment settings. Use `get_davclient()` rather than `DAVClient()` to support fetching server configuration through a config file or environment. +Support for configuring things like server credentials through a **configuration file** or through **environment** was added a while ago - the idea being that independent applications built using the caldav library can share the same configuration file / environment settings. Use `get_davclient()` rather than `DAVClient()` to support fetching server configuration through a config file or environment. As of 3.6, the config file is read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, shared with plann and other tools. ## Meta @@ -28,27 +26,47 @@ After 3.5.0, changes since the last release are kept as fragments in [changes/]( ## [3.6.0] - 2026-10-22 -### Breaking changes +The configuration file is now read through the new [calendaring-config](https://github.com/pycalendar/calendaring-config) package, shared with plann, so the two read the same file the same way - with new features like `password_command`, keyring lookups and per-key environment overrides. `save(no_overwrite=True)` and `save(no_create=True)` now leave the check to the server through wildcard preconditions (`If-None-Match: *`, `If-Match: *`), closing a race with other clients. Calendar objects keep the raw data as the server sent it (`raw_data`), and a single occurrence can be cancelled with `cancel()`. More server profiles have been checked and updated. + +Also released without the niquests dependency as 3.6.0a0 - install requests (for the sync client) and/or httpx2 (for the async client) alongside it. + +### Potentially breaking changes + +These are bug fixes or improvements, but code or configuration relying on the old behaviour may notice them. -* Since the config file is now read through calendaring-config (see Changed), some files are read differently: an empty file found first is used, rather than skipped for the next one; a section named directly that is missing or disabled, or a glob that matches nothing, is an error; a name in `contains` that matches nothing gives a warning; a disabled meta-section leaves out its members too; `*` is only the sections that are not meta-sections; and a config file named explicitly must exist. * `save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) now send `If-None-Match: *` and `If-Match: *`, so on an RFC-compliant server the flag is enforced at the same URL in the same request as the write, and another client can no longer slip in between the check and the PUT there. Potentially breaking, on servers that are themselves broken: this relies on the server honouring the wildcard preconditions, which every CalDAV server is required to do. Against a server that does not, an existing object may be overwritten, a missing one created, or a legitimate update refused. The fix is to tell the client that the server is broken: configure it with the server's `features` (e.g. `features: sogo`), or mark `save.if-none-match` and `save-load.mutable.if-match-wildcard` correctly in a hand-written feature list. Servers known not to honour them, as of this release: - `If-None-Match: *`: SOGo, Zimbra - `If-Match: *`: Bedework, GMX, iCloud, Posteo, Radicale, SOGo, Yahoo, Zimbra A refusal still raises `ConsistencyError`. When the refusal is because another object (at another URL) holds the UID, it is the new `UidConflictConsistencyError`, which is both a `UidConflictError` and a `ConsistencyError`, whether the server refused the UID or caldav's own UID search found it (https://github.com/python-caldav/caldav/issues/806). A subclass overriding `_create()` or `_put()` must accept their new `wildcard=` keyword. +* Since the config file is now read through calendaring-config (see Added), some files are read differently: an empty file found first is used, rather than skipped for the next one; a section named directly that is missing or disabled, or a glob that matches nothing, is an error; a name in `contains` that matches nothing gives a warning; a disabled meta-section leaves out its members too; `*` is only the sections that are not meta-sections; and a config file named explicitly must exist. +* `DAVClient.password` is now the `str` that was passed in, as on `AsyncDAVClient`; it used to be encoded to UTF-8 `bytes`. The return type of `Calendar.multiget()` is annotated as a list, or a coroutine of one for an async client, like `search()`. https://github.com/python-caldav/caldav/issues/799 ### Added -* Compatibility features `save-load.todo.uncomplete` (a completed task can be made pending again with `Todo.uncomplete()`; GMX keeps it completed) and `search.time-range.todo.no-dtstart.no-due` (a date search finds a task with neither DTSTART nor DUE, as RFC 4791 section 9.9 says it should; Zimbra does not). They replace the last two old-style flags in the Zimbra and GMX profiles. https://github.com/python-caldav/caldav/issues/585 +#### Library API + * `cancel()` marks an object, or a single occurrence from `search(..., expand=True)`, as `STATUS:CANCELLED` and saves it - for an occurrence that was planned but will not take place. A new compatibility feature, `save-load.event.recurrences.exception.cancelled`, records that Zimbra turns a cancelled occurrence into an `EXDATE`. https://github.com/python-caldav/caldav/issues/598 * Calendar objects keep the calendar data exactly as the server sent it, before `vcal.fix()` or any other normalisation, in the new read-only `raw_data` attribute (`bytes`; `None` for objects not read from a server, copies, and after `save()`). After `load()` it is the GET response body byte for byte; from a REPORT (`search()`, `multiget()`) it is the text of the `calendar-data` element, with XML entities decoded and line breaks turned into LF. `save(raw_data=...)` sends the given (UTF-8) bytes as the PUT body unchanged - no `vcal.fix()`, no re-serialisation, no `SEQUENCE` bump - still with `If-Match` / `If-Schedule-Tag-Match`. This lets sync tools compare and copy what servers store. https://github.com/python-caldav/caldav/issues/763 * `DAVError` and its subclasses have a new `status` attribute with the HTTP status code of the failed response (or `None`), so an `AuthorizationError` tells a 401 (bad or missing credentials) from a 403 (authenticated, but not allowed). https://github.com/python-caldav/caldav/issues/805 -### Changed +#### Configuration -* `DAVClient.password` is now the `str` that was passed in, as on `AsyncDAVClient`; it used to be encoded to UTF-8 `bytes`. The return type of `Calendar.multiget()` is annotated as a list, or a coroutine of one for an async client, like `search()`. https://github.com/python-caldav/caldav/issues/799 -* Server compatibility: several server profiles in `compatibility_hints` are updated from caldav-server-tester runs, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. +* The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786. This is a new dependency, `calendaring-config>=0.3` (it depends only on PyYAML), which matters to packagers. New in the format: + + - `~/.config/pycal/calendar.yaml` is searched first, before the old paths. + - A section may say `kind: caldav` and drop the `caldav_` prefix. Sections of other kinds (`kind: jmap`, a directory of `.ics` files) are skipped. + - `password_command` runs a program for the password, and `password_keyring` looks it up in the system keyring. + - `CALDAV_*` environment variables override the section's keys one by one: a `CALDAV_PASSWORD` alone no longer makes caldav ignore the config file (https://github.com/python-caldav/caldav/issues/780). `true`/`false` in a variable is a boolean now, and `CALDAV_HEADERS` is read as JSON. + - `extract_conn_params_from_section()`, which plann uses, reads the new format too; pass it the config file's path as `config_file` to let a `password_command` run, and the section's name as `section_name` for error messages. + - An unknown key in a section is logged as a warning. `x_*` and `extra_config` are reserved for applications. + +#### Server compatibility + +* Compatibility features `save-load.todo.uncomplete` (a completed task can be made pending again with `Todo.uncomplete()`; GMX keeps it completed) and `search.time-range.todo.no-dtstart.no-due` (a date search finds a task with neither DTSTART nor DUE, as RFC 4791 section 9.9 says it should; Zimbra does not). They replace the last two old-style flags in the Zimbra and GMX profiles. https://github.com/python-caldav/caldav/issues/585 +* `calendar-color` is now about a hex value such as `#FF0000FF`, the format clients use in practice. Setting the colour by name (`blue`) is the new `calendar-color.name`; a server that rejects or ignores colour names is `unsupported` there, not `broken`. Zimbra, SOGo, iCloud and OX App Suite are therefore `full` for `calendar-color` and `unsupported` for `calendar-color.name`. For Synology, Purelymail, GMX and Infomaniak only colour names were measured, so their hex support is assumed. `calendar-color.hex` is deprecated: a configuration setting it still works and gives a DeprecationWarning, and its `calendar-color` is then read as `calendar-color.name`. +* Several server profiles in `compatibility_hints` are updated from caldav-server-tester runs, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. - `calendar-color` (a hex value) and `calendar-order` are measured instead of left at the default `fragile`: both are `full`, and colour names work too, for Xandikos, Nextcloud (and so ecloud, which inherits its profile), CCS, Stalwart, Cyrus, Posteo, Purelymail and Robur; Bedework 3 stores both colour forms but not `calendar-order`; SOGo takes a hex colour but not a colour name; OX App Suite takes a hex colour, ignores colour names, and does not store `calendar-order`. - Sync tokens: DAViCal, SOGo and Bedework 3 now state `sync-token.invalid-token`, `sync-token.limit` and `sync-token.limit.initial-sync`, and Bedework 5 the two `limit` features, where they used to inherit `fragile` from `sync-token` or default to `full`. DAViCal and SOGo ignore `DAV:limit` and answer a never-issued sync token with a 207 and no error. Bedework 3 refuses a token it never issued, as it should, and answers a `DAV:limit` with a 500. Bedework 5 cuts the reply at `DAV:limit` without the 507 the RFC asks for: following the tokens loses nothing, but a client takes the cut reply as complete and gets the rest only on its next sync. @@ -58,15 +76,6 @@ After 3.5.0, changes since the last release are kept as fragments in [changes/]( - GMX: `search.recurrences.includes-implicit.infinite-scope` is `fragile` (it was `full`): it failed in one run and passed in the next. - Infomaniak: the wait after deleting a calendar (`synchronous-write.delete-calendar`) is raised from 12 to 20 seconds; a deleted calendar could still be there after 12. - `well-known`: `unsupported` for Posteo, `full` for Robur. -* The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786. This is a new dependency, `calendaring-config>=0.3` (it depends only on PyYAML), which matters to packagers. New in the format: - - - `~/.config/pycal/calendar.yaml` is searched first, before the old paths. - - A section may say `kind: caldav` and drop the `caldav_` prefix. Sections of other kinds (`kind: jmap`, a directory of `.ics` files) are skipped. - - `password_command` runs a program for the password, and `password_keyring` looks it up in the system keyring. - - `CALDAV_*` environment variables override the section's keys one by one: a `CALDAV_PASSWORD` alone no longer makes caldav ignore the config file (https://github.com/python-caldav/caldav/issues/780). `true`/`false` in a variable is a boolean now, and `CALDAV_HEADERS` is read as JSON. - - `extract_conn_params_from_section()`, which plann uses, reads the new format too; pass it the config file's path as `config_file` to let a `password_command` run, and the section's name as `section_name` for error messages. - - An unknown key in a section is logged as a warning. `x_*` and `extra_config` are reserved for applications. -* `compatibility_hints`: `calendar-color` is now about a hex value such as `#FF0000FF`, the format clients use in practice. Setting the colour by name (`blue`) is the new `calendar-color.name`; a server that rejects or ignores colour names is `unsupported` there, not `broken`. Zimbra, SOGo, iCloud and OX App Suite are therefore `full` for `calendar-color` and `unsupported` for `calendar-color.name`. For Synology, Purelymail, GMX and Infomaniak only colour names were measured, so their hex support is assumed. `calendar-color.hex` is deprecated: a configuration setting it still works and gives a DeprecationWarning, and its `calendar-color` is then read as `calendar-color.name`. ### Deprecated From ef3ff01b0516d41de63fcfdce0de26e5fd2cc07e Mon Sep 17 00:00:00 2001 From: Tobias Brox Date: Thu, 22 Oct 2026 10:57:18 +0200 Subject: [PATCH 4/6] fix: config_section() returns a leaf config_section() merged inherits itself and left the inherits key in, which calendaring-config's section_connection() refuses, so plann skipped, and get_all_test_servers() failed on, any section that inherits; a broken section that is not a test server no longer hides the test servers. It now delegates to calendaring_config.merged_section() (needs calendaring-config 0.5), so caldav no longer duplicates the merge. With it, a parent's contains and disable are no longer inherited, and inheriting from a section of another kind is an error; docs/source/configfile.rst says so. TODO comments mark the shims and the test tooling to move out of config.py. Prompt: A plann command for adding a task to my calendar, not giving and (sic) config options, now returns with `ERROR:root:Skipping a config section: section default has inherits: pass it merged, as a leaf Giving up: No calendars given Aborted!`. The config file is at ~/.config/calendar.conf but it contains secrets, so don't look into it directly Followup-Prompt: It should be fixed cleanly and properly. It seems to me that the caldav/config.py file needs some serious refactoring - I do understand that there is some special logic with handling the server features, etc, but it still surprises me that the file is still that big. The caldav library should not call on any private methods in calendaring-config. How much config processing is still in the caldav library - `config_section` sounds like something belonging in calendaring-config? calendaring-config releases are still cheap to do. Changes in the caldav library will delay the 3.6.0-release, but better to get it done properly now than later. Prompts (comments made inline in the document while reviewing it): > Leave [things that may be procrastinated] to 3.6.1 > Procrastinate the noisy deprecation-warnings to a later release, otherwise we'd need to change plann too. > Add /calendar-todo items for the procrastinated work, perhaps combined with ## TODO-comments inline in the code. No point with a public github issue on internal refactoring stuff. Followup-Prompt: I made a small edit in caldav [cleaning up some TODO-comments], please commit Followup-Prompt: please fix [the typos in that edit] Assisted-By: Claude Opus 5.5 (1M context) Reviewed-by: Tobias Brox --- CHANGELOG.md | 4 +- caldav/config.py | 44 +++++++++++++------- docs/source/configfile.rst | 5 ++- pyproject.toml | 2 +- tests/test_caldav_unit.py | 85 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 121 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d728ed16..a4e3f13d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -40,7 +40,7 @@ These are bug fixes or improvements, but code or configuration relying on the ol - `If-Match: *`: Bedework, GMX, iCloud, Posteo, Radicale, SOGo, Yahoo, Zimbra A refusal still raises `ConsistencyError`. When the refusal is because another object (at another URL) holds the UID, it is the new `UidConflictConsistencyError`, which is both a `UidConflictError` and a `ConsistencyError`, whether the server refused the UID or caldav's own UID search found it (https://github.com/python-caldav/caldav/issues/806). A subclass overriding `_create()` or `_put()` must accept their new `wildcard=` keyword. -* Since the config file is now read through calendaring-config (see Added), some files are read differently: an empty file found first is used, rather than skipped for the next one; a section named directly that is missing or disabled, or a glob that matches nothing, is an error; a name in `contains` that matches nothing gives a warning; a disabled meta-section leaves out its members too; `*` is only the sections that are not meta-sections; and a config file named explicitly must exist. +* Since the config file is now read through calendaring-config (see Added), some files are read differently: an empty file found first is used, rather than skipped for the next one; a section named directly that is missing or disabled, or a glob that matches nothing, is an error; a name in `contains` that matches nothing gives a warning; a disabled meta-section leaves out its members too; `*` is only the sections that are not meta-sections; `inherits` naming a missing section, or a cycle of them, is an error, where it used to be ignored or to recurse forever; `config_section()` leaves `inherits`, `contains` and `disable` out of the section it returns; and a config file named explicitly must exist. * `DAVClient.password` is now the `str` that was passed in, as on `AsyncDAVClient`; it used to be encoded to UTF-8 `bytes`. The return type of `Calendar.multiget()` is annotated as a list, or a coroutine of one for an async client, like `search()`. https://github.com/python-caldav/caldav/issues/799 ### Added @@ -53,7 +53,7 @@ These are bug fixes or improvements, but code or configuration relying on the ol #### Configuration -* The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786. This is a new dependency, `calendaring-config>=0.3` (it depends only on PyYAML), which matters to packagers. New in the format: +* The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786. This is a new dependency, `calendaring-config>=0.5` (it depends only on PyYAML), which matters to packagers. New in the format: - `~/.config/pycal/calendar.yaml` is searched first, before the old paths. - A section may say `kind: caldav` and drop the `caldav_` prefix. Sections of other kinds (`kind: jmap`, a directory of `.ics` files) are skipped. diff --git a/caldav/config.py b/caldav/config.py index f316f222..eca99920 100644 --- a/caldav/config.py +++ b/caldav/config.py @@ -11,15 +11,21 @@ log = logging.getLogger("caldav") -""" -This configuration parsing code was just copied from my plann library (and will be removed from there at some point in the future). Test coverage is poor as for now. -""" +## The config file format and its parsing belong to calendaring-config. What +## is left here is caldav's own: mapping a section to DAVClient keyword +## arguments, resolving feature profiles, and the order of the sources +## (explicit arguments, test server, environment, file). +## +## TODO: the public expand_config_section, config_section, read_config and +## extract_conn_params_from_section are kept for plann, and should be +## deprecated (with warnings) once plann gets its calendars from +## get_calendars() (https://github.com/PyCalendar/plann/issues/84). +## +## TODO: verify that we have test code for all kinds of corner cases +## +## TODO: verify that the config format is well enough documented -## This is being moved from my plann library. The code itself will be introduced into caldav 2.0, but proper test code and documentation will come in a later release (2.1?) - -## TODO TODO TODO - write test code for all the corner cases -## TODO TODO TODO - write documentation of config format def expand_config_section(config, section="default", blacklist=None): """ The leaf sections that ``section`` expands to, as a list. @@ -43,13 +49,13 @@ def expand_config_section(config, section="default", blacklist=None): def config_section(config, section="default"): - if section in config and "inherits" in config[section]: - ret = config_section(config, config[section]["inherits"]) - else: - ret = {} - if section in config: - ret.update(config[section]) - return ret + """Section ``section`` with what it ``inherits`` copied in, or ``{}`` when + there is no such section. ``inherits`` and the parents' ``contains`` and + ``disable`` are left out; the section's own are kept. The rules are + calendaring-config's ``merged_section``.""" + if section not in config: + return {} + return calendaring_config.merged_section(config, section) def read_config(fn, interactive_error=False): @@ -356,6 +362,9 @@ def _caldav_connections( return [params for params in found if params] +## TODO: the test server selection below, get_test_setting and +## get_all_test_servers are test tooling, not config; move them to +## caldav/testing.py (planned for 3.6.1). def _get_test_server_config( name: str | None, environment: bool, config_file: str | None = None ) -> dict[str, Any] | None: @@ -616,7 +625,12 @@ def get_all_test_servers( result: dict[str, dict[str, Any]] = {} for section_name in cfg: - section_data = config_section(cfg, section_name) + try: + section_data = config_section(cfg, section_name) + except calendaring_config.ConfigError as e: + ## A broken section that is not a test server must not hide the others + log.warning(f"config section {section_name} skipped: {e}") + continue if get_test_setting(section_data, "test_allowed"): conn_params = extract_conn_params_from_section(section_data, str(path), section_name) if conn_params: diff --git a/docs/source/configfile.rst b/docs/source/configfile.rst index 665094e5..dae2fa13 100644 --- a/docs/source/configfile.rst +++ b/docs/source/configfile.rst @@ -186,7 +186,10 @@ sections share the same server or credentials: calendar_name: Tasks Inheritance is recursive: a section can inherit from a section that itself -inherits from another. +inherits from another. A parent's ``contains`` and ``disable`` are not +inherited. Inheriting from a missing section, a cycle of ``inherits``, or a +parent of another kind (a ``caldav_*`` section inheriting from a +``kind: jmap`` one) is an error. Meta-sections (collections) ============================ diff --git a/pyproject.toml b/pyproject.toml index 642852a4..6be285c5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -85,7 +85,7 @@ dependencies = [ "dnspython", "python-dateutil", "PyYAML", - "calendaring-config>=0.3", ## reads the config file, see issue 786 + "calendaring-config>=0.5", ## reads the config file, see issue 786 ] dynamic = ["version"] diff --git a/tests/test_caldav_unit.py b/tests/test_caldav_unit.py index ec239f33..334b6153 100755 --- a/tests/test_caldav_unit.py +++ b/tests/test_caldav_unit.py @@ -4550,6 +4550,91 @@ def test_inherits_recursive(self): assert result["caldav_username"] == "alice" assert result["caldav_password"] == "secret" + def test_result_is_a_leaf(self): + """plann hands the result to extract_conn_params_from_section, which + refused it as long as the inherits key was left in""" + from caldav.config import config_section, extract_conn_params_from_section + + config = { + "base": {"caldav_url": "https://dav.example.com/", "contains": ["x"]}, + "default": {"inherits": "base", "caldav_user": "bob"}, + "x": {}, + } + result = config_section(config, "default") + assert "inherits" not in result and "contains" not in result + assert extract_conn_params_from_section(result) == { + "url": "https://dav.example.com/", + "username": "bob", + } + + def test_test_server_with_inherits(self, tmp_path): + import json + + from caldav.config import get_all_test_servers + + config_file = tmp_path / "calendar.conf" + config_file.write_text( + json.dumps( + { + "base": {"caldav_url": "https://dav.example.com/"}, + "srv": {"inherits": "base", "test_allowed": True}, + } + ) + ) + assert get_all_test_servers(str(config_file))["srv"]["url"] == "https://dav.example.com/" + + def test_broken_section_does_not_hide_test_servers(self, tmp_path): + import json + + from caldav.config import get_all_test_servers + + config_file = tmp_path / "calendar.conf" + config_file.write_text( + json.dumps( + { + "srv": {"caldav_url": "https://dav.example.com/", "test_allowed": True}, + "broken": {"inherits": "nosuch"}, + "a": {"inherits": "b"}, + "b": {"inherits": "a"}, + } + ) + ) + assert list(get_all_test_servers(str(config_file))) == ["srv"] + + def test_own_contains_and_disable_are_kept(self): + """only the parents' contains and disable are dropped""" + from caldav.config import config_section + + config = {"meta": {"contains": ["a"], "disable": True}, "a": {}} + assert config_section(config, "meta") == {"contains": ["a"], "disable": True} + + def test_parent_disable_is_not_inherited(self): + from caldav.config import config_section + + config = { + "base": {"caldav_url": "https://dav.example.com/", "disable": True}, + "child": {"inherits": "base"}, + } + assert config_section(config, "child") == {"caldav_url": "https://dav.example.com/"} + + def test_kind_mismatch_is_refused(self): + import calendaring_config + import pytest + + from caldav.config import config_section + + config = { + "jmap": {"kind": "jmap", "url": "https://jmap.example.com/"}, + "child": {"inherits": "jmap", "caldav_url": "https://dav.example.com/"}, + } + with pytest.raises(calendaring_config.ConfigError): + config_section(config, "child") + + def test_missing_section_is_empty(self): + from caldav.config import config_section + + assert config_section({"a": {}}, "nosuch") == {} + class TestExpandEnvVars: """Unit tests for caldav.config.expand_env_vars.""" From e660192f03631c862fc63f004c7c24096a7f1906 Mon Sep 17 00:00:00 2001 From: Tobias Brox Date: Thu, 22 Oct 2026 15:45:29 +0200 Subject: [PATCH 5/6] docs: review the 3.6.0 changelog tobixen reworded the config and wildcard-precondition entries under "Potentially breaking changes"; the details that dropped out (which header each server fails, UidConflictConsistencyError, the new wildcard parameter) move to a new "Changed" section, together with the calendar-color hex/name split. Every entry now links its PR, and the wildcard entry no longer cites issue 806, which is about UID conflicts. A Security section takes the safe_load entry and a note that password_command runs a program from the config file, guarded by an owner and permission check that the YAML tags never had, which is why safe_load is kept. The config entries now name the kind-mismatch error, that a parent's contains and disable are not inherited, and the keyring extra. Prompt: I've done some work on the CHANGELOG, do a review. If I deleted too many details about the if-(none)-match, then add another point about it under CHANGED and add a reference. Can we have links to the pull requests? Particularly for the bullet points that are missing links ot (sic) the issues. Followup-Prompt: Add the yaml.safe_load to the security, but also add information about the new password command thing under security. Due to the latter there is no security benefit from the safe_load. Perhaps we shouldn't use safe_load? Or is it still safer with safe_load than without? Prompts (comments made inline in the document while reviewing it): > (on the vague "no guarantees that things will work out 100% as expected" in the wildcard entry) do the needful, but keep it short in the "breaking changes"-section > (on the calendar-color hex/name split being filed under Added) Not breaking. Assisted-By: Claude Opus 5.5 (1M context) Reviewed-by: Tobias Brox --- CHANGELOG.md | 72 ++++++++++++++++++++++++++++++++-------------------- 1 file changed, 45 insertions(+), 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a4e3f13d..ac565c32 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,43 +30,51 @@ The configuration file is now read through the new [calendaring-config](https:// Also released without the niquests dependency as 3.6.0a0 - install requests (for the sync client) and/or httpx2 (for the async client) alongside it. -### Potentially breaking changes +### Security -These are bug fixes or improvements, but code or configuration relying on the old behaviour may notice them. +* Configuration files are now read with PyYAML's safe loader (by calendaring-config). The full loader was used before, so a YAML tag such as `!!python/object/apply:` in a config file could run arbitrary Python code when `get_davclient()` or `get_calendars()` read it - also from the system-wide `/etc/calendar.conf`. A config file using such tags is now refused with a `ValueError`; plain YAML and JSON files are unaffected. https://github.com/python-caldav/caldav/pull/785 +* The new `password_command` key runs a program named in the config file, so a config file is, by design, able to run code. It is guarded where the YAML tags were not: the command is only run if the file it comes from is owned by the user or by root and is not writable by group or others, and it must be given as a list of the program and its arguments, never through a shell. A config file written by someone else, or one others can write to, should not be trusted for the same reason as before. https://github.com/python-caldav/caldav/pull/804 -* `save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) now send `If-None-Match: *` and `If-Match: *`, so on an RFC-compliant server the flag is enforced at the same URL in the same request as the write, and another client can no longer slip in between the check and the PUT there. Potentially breaking, on servers that are themselves broken: this relies on the server honouring the wildcard preconditions, which every CalDAV server is required to do. Against a server that does not, an existing object may be overwritten, a missing one created, or a legitimate update refused. The fix is to tell the client that the server is broken: configure it with the server's `features` (e.g. `features: sogo`), or mark `save.if-none-match` and `save-load.mutable.if-match-wildcard` correctly in a hand-written feature list. Servers known not to honour them, as of this release: +### Potentially breaking changes - - `If-None-Match: *`: SOGo, Zimbra - - `If-Match: *`: Bedework, GMX, iCloud, Posteo, Radicale, SOGo, Yahoo, Zimbra +These are bug fixes or improvements, but code or configuration relying on the old behaviour may notice them. - A refusal still raises `ConsistencyError`. When the refusal is because another object (at another URL) holds the UID, it is the new `UidConflictConsistencyError`, which is both a `UidConflictError` and a `ConsistencyError`, whether the server refused the UID or caldav's own UID search found it (https://github.com/python-caldav/caldav/issues/806). A subclass overriding `_create()` or `_put()` must accept their new `wildcard=` keyword. -* Since the config file is now read through calendaring-config (see Added), some files are read differently: an empty file found first is used, rather than skipped for the next one; a section named directly that is missing or disabled, or a glob that matches nothing, is an error; a name in `contains` that matches nothing gives a warning; a disabled meta-section leaves out its members too; `*` is only the sections that are not meta-sections; `inherits` naming a missing section, or a cycle of them, is an error, where it used to be ignored or to recurse forever; `config_section()` leaves `inherits`, `contains` and `disable` out of the section it returns; and a config file named explicitly must exist. -* `DAVClient.password` is now the `str` that was passed in, as on `AsyncDAVClient`; it used to be encoded to UTF-8 `bytes`. The return type of `Calendar.multiget()` is annotated as a list, or a coroutine of one for an async client, like `search()`. https://github.com/python-caldav/caldav/issues/799 +* Most of the config file handling logic was separated out in a new package calendaring-config (see Added). The changes introduced are unlikely to break anything for anyone, and hence considered allowable for the minor release (https://github.com/python-caldav/caldav/issues/786, https://github.com/python-caldav/caldav/pull/804): + * **This introduces a new dependency**. The package has the same maintainer as the caldav library, and the same co-maintainers as the icalendar-searcher library. + * An empty config file found first in the search path is used, rather than skipped for the next one. + * If a disabled or missing section is named, or a glob matches nothing, an error is now raised. + * A name in `contains` that matches nothing gives a warning. + * A disabled meta-section leaves out its members too. + * `*` now excludes meta-sections. + * `inherits` naming a missing section or a parent of another kind (a `caldav_*` section inheriting from a `kind: jmap` one), or a cycle of `inherits`, raises an error (they used to be ignored, merged in, or recurse forever). + * A section no longer inherits its parent's `contains` and `disable`, and `config_section()` leaves `inherits` out of the section it returns. + * A config file named explicitly must exist. +* With a server that does not honour the wildcard preconditions (SOGo, Zimbra, Bedework, GMX, iCloud, Posteo, Radicale, Yahoo), `save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) may now overwrite an existing object, create a missing one or refuse a legitimate update, unless the server is configured with its `features` (e.g. `features: sogo`). See "Changed" below, https://github.com/python-caldav/caldav/pull/802 +* `DAVClient.password` is now the `str` that was passed in, as on `AsyncDAVClient`; it used to be encoded to UTF-8 `bytes`. The return type of `Calendar.multiget()` is annotated as a list, or a coroutine of one for an async client, like `search()`. https://github.com/python-caldav/caldav/issues/799, https://github.com/python-caldav/caldav/pull/807 ### Added #### Library API -* `cancel()` marks an object, or a single occurrence from `search(..., expand=True)`, as `STATUS:CANCELLED` and saves it - for an occurrence that was planned but will not take place. A new compatibility feature, `save-load.event.recurrences.exception.cancelled`, records that Zimbra turns a cancelled occurrence into an `EXDATE`. https://github.com/python-caldav/caldav/issues/598 -* Calendar objects keep the calendar data exactly as the server sent it, before `vcal.fix()` or any other normalisation, in the new read-only `raw_data` attribute (`bytes`; `None` for objects not read from a server, copies, and after `save()`). After `load()` it is the GET response body byte for byte; from a REPORT (`search()`, `multiget()`) it is the text of the `calendar-data` element, with XML entities decoded and line breaks turned into LF. `save(raw_data=...)` sends the given (UTF-8) bytes as the PUT body unchanged - no `vcal.fix()`, no re-serialisation, no `SEQUENCE` bump - still with `If-Match` / `If-Schedule-Tag-Match`. This lets sync tools compare and copy what servers store. https://github.com/python-caldav/caldav/issues/763 -* `DAVError` and its subclasses have a new `status` attribute with the HTTP status code of the failed response (or `None`), so an `AuthorizationError` tells a 401 (bad or missing credentials) from a 403 (authenticated, but not allowed). https://github.com/python-caldav/caldav/issues/805 +* `cancel()` marks an object, or a single occurrence from `search(..., expand=True)`, as `STATUS:CANCELLED` and saves it - for an occurrence that was planned but will not take place. A new compatibility feature, `save-load.event.recurrences.exception.cancelled`, records that Zimbra turns a cancelled occurrence into an `EXDATE`. https://github.com/python-caldav/caldav/issues/598, https://github.com/python-caldav/caldav/pull/810 +* Calendar objects keep the calendar data exactly as the server sent it, before `vcal.fix()` or any other normalisation, in the new read-only `raw_data` attribute (`bytes`; `None` for objects not read from a server, copies, and after `save()`). After `load()` it is the GET response body byte for byte; from a REPORT (`search()`, `multiget()`) it is the text of the `calendar-data` element, with XML entities decoded and line breaks turned into LF. `save(raw_data=...)` sends the given (UTF-8) bytes as the PUT body unchanged - no `vcal.fix()`, no re-serialisation, no `SEQUENCE` bump - still with `If-Match` / `If-Schedule-Tag-Match`. This lets sync tools compare and copy what servers store. https://github.com/python-caldav/caldav/issues/763, https://github.com/python-caldav/caldav/pull/803 +* `DAVError` and its subclasses have a new `status` attribute with the HTTP status code of the failed response (or `None`), so an `AuthorizationError` tells a 401 (bad or missing credentials) from a 403 (authenticated, but not allowed). https://github.com/python-caldav/caldav/issues/805, https://github.com/python-caldav/caldav/pull/807 #### Configuration -* The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786. This is a new dependency, `calendaring-config>=0.5` (it depends only on PyYAML), which matters to packagers. New in the format: +* The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786 and https://github.com/python-caldav/caldav/pull/804. This is a new dependency, `calendaring-config>=0.5` (it depends only on PyYAML), which matters to packagers. New in the format: - `~/.config/pycal/calendar.yaml` is searched first, before the old paths. - A section may say `kind: caldav` and drop the `caldav_` prefix. Sections of other kinds (`kind: jmap`, a directory of `.ics` files) are skipped. - - `password_command` runs a program for the password, and `password_keyring` looks it up in the system keyring. + - `password_command` runs a program for the password, and `password_keyring` looks it up in the system keyring (this needs the `keyring` package, e.g. `pip install calendaring-config[keyring]`). - `CALDAV_*` environment variables override the section's keys one by one: a `CALDAV_PASSWORD` alone no longer makes caldav ignore the config file (https://github.com/python-caldav/caldav/issues/780). `true`/`false` in a variable is a boolean now, and `CALDAV_HEADERS` is read as JSON. - `extract_conn_params_from_section()`, which plann uses, reads the new format too; pass it the config file's path as `config_file` to let a `password_command` run, and the section's name as `section_name` for error messages. - An unknown key in a section is logged as a warning. `x_*` and `extra_config` are reserved for applications. #### Server compatibility -* Compatibility features `save-load.todo.uncomplete` (a completed task can be made pending again with `Todo.uncomplete()`; GMX keeps it completed) and `search.time-range.todo.no-dtstart.no-due` (a date search finds a task with neither DTSTART nor DUE, as RFC 4791 section 9.9 says it should; Zimbra does not). They replace the last two old-style flags in the Zimbra and GMX profiles. https://github.com/python-caldav/caldav/issues/585 -* `calendar-color` is now about a hex value such as `#FF0000FF`, the format clients use in practice. Setting the colour by name (`blue`) is the new `calendar-color.name`; a server that rejects or ignores colour names is `unsupported` there, not `broken`. Zimbra, SOGo, iCloud and OX App Suite are therefore `full` for `calendar-color` and `unsupported` for `calendar-color.name`. For Synology, Purelymail, GMX and Infomaniak only colour names were measured, so their hex support is assumed. `calendar-color.hex` is deprecated: a configuration setting it still works and gives a DeprecationWarning, and its `calendar-color` is then read as `calendar-color.name`. -* Several server profiles in `compatibility_hints` are updated from caldav-server-tester runs, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. +* Compatibility features `save-load.todo.uncomplete` (a completed task can be made pending again with `Todo.uncomplete()`; GMX keeps it completed) and `search.time-range.todo.no-dtstart.no-due` (a date search finds a task with neither DTSTART nor DUE, as RFC 4791 section 9.9 says it should; Zimbra does not). They replace the last two old-style flags in the Zimbra and GMX profiles. https://github.com/python-caldav/caldav/issues/585, https://github.com/python-caldav/caldav/pull/811 +* Several server profiles in `compatibility_hints` are updated from caldav-server-tester runs, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. https://github.com/python-caldav/caldav/pull/781, https://github.com/python-caldav/caldav/pull/812 - `calendar-color` (a hex value) and `calendar-order` are measured instead of left at the default `fragile`: both are `full`, and colour names work too, for Xandikos, Nextcloud (and so ecloud, which inherits its profile), CCS, Stalwart, Cyrus, Posteo, Purelymail and Robur; Bedework 3 stores both colour forms but not `calendar-order`; SOGo takes a hex colour but not a colour name; OX App Suite takes a hex colour, ignores colour names, and does not store `calendar-order`. - Sync tokens: DAViCal, SOGo and Bedework 3 now state `sync-token.invalid-token`, `sync-token.limit` and `sync-token.limit.initial-sync`, and Bedework 5 the two `limit` features, where they used to inherit `fragile` from `sync-token` or default to `full`. DAViCal and SOGo ignore `DAV:limit` and answer a never-issued sync token with a 207 and no error. Bedework 3 refuses a token it never issued, as it should, and answers a `DAV:limit` with a 500. Bedework 5 cuts the reply at `DAV:limit` without the 507 the RFC asks for: following the tokens loses nothing, but a client takes the cut reply as complete and gets the rest only on its next sync. @@ -77,25 +85,35 @@ These are bug fixes or improvements, but code or configuration relying on the ol - Infomaniak: the wait after deleting a calendar (`synchronous-write.delete-calendar`) is raised from 12 to 20 seconds; a deleted calendar could still be there after 12. - `well-known`: `unsupported` for Posteo, `full` for Robur. +### Changed + +* `calendar-color` is now about a hex value such as `#FF0000FF`, the format clients use in practice. Setting the colour by name (`blue`) is the new `calendar-color.name`; a server that rejects or ignores colour names is `unsupported` there, not `broken`. Zimbra, SOGo, iCloud and OX App Suite are therefore `full` for `calendar-color` and `unsupported` for `calendar-color.name`. For Synology, Purelymail, GMX and Infomaniak only colour names were measured, so their hex support is assumed. `calendar-color.hex` is deprecated: a configuration setting it still works and gives a DeprecationWarning, and its `calendar-color` is then read as `calendar-color.name`. https://github.com/python-caldav/caldav/pull/813 +* `save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) now send `If-None-Match: *` and `If-Match: *`, so on an RFC-compliant server the flag is enforced at the same URL in the same request as the write, and another client can no longer slip in between the check and the PUT. Against a server that does not honour the wildcard preconditions, an existing object may be overwritten, a missing one created, or a legitimate update refused (see "Potentially breaking changes" above). Servers known not to honour them, as of this release: + + - `If-None-Match: *`: SOGo, Zimbra + - `If-Match: *`: Bedework, GMX, iCloud, Posteo, Radicale, SOGo, Yahoo, Zimbra + + A refusal still raises `ConsistencyError`. A subclass overriding `_create()` or `_put()` (or their async variants) must accept a new `wildcard` parameter after `precondition`. https://github.com/python-caldav/caldav/pull/802 +* When `save(no_overwrite=True)` or `save(no_create=True)` is refused because another object (at another URL) holds the UID, the error is now the new `UidConflictConsistencyError`, which is both a `UidConflictError` and a `ConsistencyError`, whether the server refused the UID or caldav's own UID search found it. https://github.com/python-caldav/caldav/issues/806, https://github.com/python-caldav/caldav/pull/802, https://github.com/python-caldav/caldav/pull/807 + ### Deprecated -* Config file keys starting with `test_` are now reserved for test tooling, so other programs sharing the config file can drop them. A section is marked for testing with `test_allowed: true` rather than `testing_allowed: true`, and the order test servers are tried in is set with `test_priority` rather than `priority`. The old names still work, but emit a `DeprecationWarning`. `test_priority` (or `priority`) in a section of the main config file was ignored before; it is now honoured. https://github.com/python-caldav/caldav/issues/786 -* `FeatureSet.copyFeatureSet()` in `caldav.compatibility_hints` has been renamed to `copy_feature_set()`, in line with the naming of the other methods. The old name still works but emits a `DeprecationWarning`. +* Config file keys starting with `test_` are now reserved for test tooling, so other programs sharing the config file can drop them. A section is marked for testing with `test_allowed: true` rather than `testing_allowed: true`, and the order test servers are tried in is set with `test_priority` rather than `priority`. The old names still work, but emit a `DeprecationWarning`. `test_priority` (or `priority`) in a section of the main config file was ignored before; it is now honoured. https://github.com/python-caldav/caldav/issues/786, https://github.com/python-caldav/caldav/pull/804 +* `FeatureSet.copyFeatureSet()` in `caldav.compatibility_hints` has been renamed to `copy_feature_set()`, in line with the naming of the other methods. The old name still works but emits a `DeprecationWarning`. https://github.com/python-caldav/caldav/pull/790 ### Removed -* The old-style compatibility flags are gone: `caldav.compatibility_hints.incompatibility_description`, and the `old_flags` key in a feature set. They were only ever used by the test suite to skip or adjust tests, and every flag still in use is now a compatibility feature. https://github.com/python-caldav/caldav/issues/585 +* The old-style compatibility flags are gone: `caldav.compatibility_hints.incompatibility_description`, and the `old_flags` key in a feature set. They were only ever used by the test suite to skip or adjust tests, and every flag still in use is now a compatibility feature. https://github.com/python-caldav/caldav/issues/585, https://github.com/python-caldav/caldav/pull/811 ### Fixed -* `complete(handle_rrule=True)` on a recurring task with fixed times (`BY*` parts in the `RRULE`, e.g. `FREQ=YEARLY;BYMONTH=4,9`) did not advance the task when it was completed before its `DTSTART`: the next occurrence was looked for after the completion time, which found the occurrence just completed. It is now looked for after `DTSTART`. https://github.com/python-caldav/caldav/issues/782 -* `aio.get_calendars()` ignored meta-sections, `*` and glob patterns in `config_section`, as well as the `calendar_name` and `calendar_url` filters of a config section. It now finds the same calendars as the sync `get_calendars()`, and behaves like it on errors too: it no longer sends an OPTIONS probe first, so an unreachable server raises the error from the principal lookup rather than a `DAVError` saying "Failed to connect", and with `raise_errors=False` such an error is logged and the server skipped rather than raised. https://github.com/python-caldav/caldav/issues/793 -* Creating an `AsyncDAVClient` failed with an `ImportError` when only an httpx-family library was installed. It works now; RFC 6764 discovery still needs niquests or requests, and is skipped with a warning without them. `aio.get_davclient()` and `aio.get_calendars()` blocked the event loop while reading the config file and doing RFC 6764 discovery; they now do both in a worker thread. https://github.com/python-caldav/caldav/issues/794 -* The `supported-report-set` and `current-user-privilege-set` properties from `propfind()` were the raw lxml element. They are now lists of the report and privilege names in Clark notation, e.g. `["{urn:ietf:params:xml:ns:caldav}calendar-query", "{DAV:}sync-collection"]`. Code that walked the returned element has to read the list instead. https://github.com/python-caldav/caldav/issues/795 -* `caldav.testing`: `RadicaleServer` ignored its configured port, so Radicale always listened on 5232 and any other port timed out at startup. `RadicaleServer.stop()` now closes both ends of its shutdown socketpair, and `XandikosServer.stop()` closes its event loop, so neither leaves a `ResourceWarning` behind. https://github.com/python-caldav/caldav/issues/797 -* A `200 OK` reply to the PUT of `save()` is now taken as success (RFC 9110 section 9.3.4 allows it for an overwrite). Before, it was retried, and the retry raised `PutError` when the server answered `200` again. -* A long UID may be folded over several lines, and for an object that was not parsed yet `.id` returned only the first line. As a result `save(no_create=True)` of such an existing object raised `ConsistencyError`, `save(no_overwrite=True)` overwrote it instead of raising, and `save(increase_seqno=False, only_this_recurrence=False)` wrote the cut UID back to the server. `.id` now returns the whole UID, and a UID with a backslash escape such as `\,` is unescaped there too. -* Configuration files are now read with PyYAML's safe loader. The full loader was used before, so a YAML tag such as `!!python/object/apply:` in a config file could run arbitrary Python code when `get_davclient()` or `get_calendars()` read it - also from the system-wide `/etc/calendar.conf`. A config file using such tags is now refused with a `ValueError`; plain YAML and JSON files are unaffected. +* `complete(handle_rrule=True)` on a recurring task with fixed times (`BY*` parts in the `RRULE`, e.g. `FREQ=YEARLY;BYMONTH=4,9`) did not advance the task when it was completed before its `DTSTART`: the next occurrence was looked for after the completion time, which found the occurrence just completed. It is now looked for after `DTSTART`. https://github.com/python-caldav/caldav/issues/782, https://github.com/python-caldav/caldav/pull/808 +* `aio.get_calendars()` ignored meta-sections, `*` and glob patterns in `config_section`, as well as the `calendar_name` and `calendar_url` filters of a config section. It now finds the same calendars as the sync `get_calendars()`, and behaves like it on errors too: it no longer sends an OPTIONS probe first, so an unreachable server raises the error from the principal lookup rather than a `DAVError` saying "Failed to connect", and with `raise_errors=False` such an error is logged and the server skipped rather than raised. https://github.com/python-caldav/caldav/issues/793, https://github.com/python-caldav/caldav/pull/800 +* Creating an `AsyncDAVClient` failed with an `ImportError` when only an httpx-family library was installed. It works now; RFC 6764 discovery still needs niquests or requests, and is skipped with a warning without them. `aio.get_davclient()` and `aio.get_calendars()` blocked the event loop while reading the config file and doing RFC 6764 discovery; they now do both in a worker thread. https://github.com/python-caldav/caldav/issues/794, https://github.com/python-caldav/caldav/pull/800 +* The `supported-report-set` and `current-user-privilege-set` properties from `propfind()` were the raw lxml element. They are now lists of the report and privilege names in Clark notation, e.g. `["{urn:ietf:params:xml:ns:caldav}calendar-query", "{DAV:}sync-collection"]`. Code that walked the returned element has to read the list instead. https://github.com/python-caldav/caldav/issues/795, https://github.com/python-caldav/caldav/pull/807 +* `caldav.testing`: `RadicaleServer` ignored its configured port, so Radicale always listened on 5232 and any other port timed out at startup. `RadicaleServer.stop()` now closes both ends of its shutdown socketpair, and `XandikosServer.stop()` closes its event loop, so neither leaves a `ResourceWarning` behind. https://github.com/python-caldav/caldav/issues/797, https://github.com/python-caldav/caldav/pull/800 +* A `200 OK` reply to the PUT of `save()` is now taken as success (RFC 9110 section 9.3.4 allows it for an overwrite). Before, it was retried, and the retry raised `PutError` when the server answered `200` again. https://github.com/python-caldav/caldav/pull/802 +* A long UID may be folded over several lines, and for an object that was not parsed yet `.id` returned only the first line. As a result `save(no_create=True)` of such an existing object raised `ConsistencyError`, `save(no_overwrite=True)` overwrote it instead of raising, and `save(increase_seqno=False, only_this_recurrence=False)` wrote the cut UID back to the server. `.id` now returns the whole UID, and a UID with a backslash escape such as `\,` is unescaped there too. https://github.com/python-caldav/caldav/pull/787 ## [3.5.0] - 2026-10-15 From d9849c919467269c3622001c235a2c6ffdded4c3 Mon Sep 17 00:00:00 2001 From: Tobias Brox Date: Thu, 22 Oct 2026 19:49:44 +0200 Subject: [PATCH 6/6] docs: 3.6.0 changelog intro, feature-list advice Add a note on why 3.6.0 follows 3.5.0 so closely, linking the projects that drove it. Restore the advice lost when the fragments were collected: a hand-written feature list should mark save.if-none-match and save-load.mutable.if-match-wildcard. Prompt: If we're anyway going to change something, then I'd like to do an edit to the CHANGELOG too before pushing Followup-Prompt: I've done my fix in the CHANGELOG, so please push Prompts (comments made inline in the document while reviewing it): > (on "the new CHANGELOG intro is long and chatty") I've shortened down the intro a bit > (on "grammar in the new CHANGELOG intro") The text has been changed, so the old grammar errors are gone, but fix the new grammar errors and add links. Assisted-By: Claude Opus 5.5 (1M context) Reviewed-by: Tobias Brox --- CHANGELOG.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ac565c32..8b38cb6d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,10 @@ After 3.5.0, changes since the last release are kept as fragments in [changes/]( ## [3.6.0] - 2026-10-22 +I apologize for the 3.6.0 release coming so shortly after the 3.5.0 release; it is fuelled by demands from the [calendaring-sync](https://github.com/pycalendar/calendaring-sync) and [calendaring](https://github.com/pycalendar/calendaring) projects. If you're doing packaging or maintaining dependencies and find bumping the version costly, then wait two weeks - maybe there will be a 3.7.0 release. + +### Overview + The configuration file is now read through the new [calendaring-config](https://github.com/pycalendar/calendaring-config) package, shared with plann, so the two read the same file the same way - with new features like `password_command`, keyring lookups and per-key environment overrides. `save(no_overwrite=True)` and `save(no_create=True)` now leave the check to the server through wildcard preconditions (`If-None-Match: *`, `If-Match: *`), closing a race with other clients. Calendar objects keep the raw data as the server sent it (`raw_data`), and a single occurrence can be cancelled with `cancel()`. More server profiles have been checked and updated. Also released without the niquests dependency as 3.6.0a0 - install requests (for the sync client) and/or httpx2 (for the async client) alongside it. @@ -49,7 +53,7 @@ These are bug fixes or improvements, but code or configuration relying on the ol * `inherits` naming a missing section or a parent of another kind (a `caldav_*` section inheriting from a `kind: jmap` one), or a cycle of `inherits`, raises an error (they used to be ignored, merged in, or recurse forever). * A section no longer inherits its parent's `contains` and `disable`, and `config_section()` leaves `inherits` out of the section it returns. * A config file named explicitly must exist. -* With a server that does not honour the wildcard preconditions (SOGo, Zimbra, Bedework, GMX, iCloud, Posteo, Radicale, Yahoo), `save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) may now overwrite an existing object, create a missing one or refuse a legitimate update, unless the server is configured with its `features` (e.g. `features: sogo`). See "Changed" below, https://github.com/python-caldav/caldav/pull/802 +* With a server that does not honour the wildcard preconditions (SOGo, Zimbra, Bedework, GMX, iCloud, Posteo, Radicale, Yahoo), `save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) may now overwrite an existing object, create a missing one or refuse a legitimate update, unless the server is configured with its `features` (e.g. `features: sogo`), or `save.if-none-match` and `save-load.mutable.if-match-wildcard` are marked correctly in a hand-written feature list. See "Changed" below, https://github.com/python-caldav/caldav/pull/802 * `DAVClient.password` is now the `str` that was passed in, as on `AsyncDAVClient`; it used to be encoded to UTF-8 `bytes`. The return type of `Calendar.multiget()` is annotated as a list, or a coroutine of one for an async client, like `search()`. https://github.com/python-caldav/caldav/issues/799, https://github.com/python-caldav/caldav/pull/807 ### Added