Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 16 additions & 10 deletions docs/caldav-servers.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,19 @@ nothing waits for an index that lags behind writes: three stories fail on
Bedework 3 (an event or task just written is not found yet), and
purelymail fails most stories that search.

## The conformance tests: a known failure, 2026-11-03

SOGo adds `RSVP=TRUE` and `PARTSTAT` to an ATTENDEE that had none, and Ox
adds `EMAIL=`. caldav records this
(`save-load.icalendar.attendee-params`), but no capability declaration
can say that a server *adds* parameters: LOSSY only describes a server
dropping or rewriting what it was sent. The round trip and `verify`
therefore report the attendees on these servers as changed, and the
tests fail. Whether verification should tolerate added parameters, or
`attendees` should count as LOSSY there, is undecided.
## The conformance tests (`tests/*/test_caldav_conformance.py`), 2026-11-17

One server at a time, with caldav's master branch. Zimbra, Ox, Cyrus,
Davical, Davis, Baikal, Nextcloud, Radicale, Xandikos, CCS, Bedework 3
and Stalwart passed on 2026-11-12. SOGo waits for caldav's SOGo hints;
ecloud has not been run to the end.

| Server | Result | Why |
|---|---|---|
| Bedework 5 (sync suite only) | pass, 35 skipped | the skips need tasks or journals, which its calendar does not hold. It stores each ATTENDEE twice, and caldav drops the copy |
| gmx | pass | |
| robur | passes on rerun | about one request in 60 hangs for 70 s and is then reset, and some PUTs get a 500; seeding retries a failed write, but a test that reads 120 items still fails some runs, and `--last-failed` passed them all |
| purelymail | pass | run alone, in 1 h 54 |
| infomaniak | pass, 2 skipped | its calendar holds no journals. It stores URL as `URL;VALUE=URI`, which verify took for a change before 2026-11-18 |
| posteo | pass, 10 skipped | it cannot make calendars, so the 8 tests that need a second one (move, a token from another collection, creating one) skip; and it holds no journals |
| jeanes | pass | 5 h 09: the account held 469 leftover calendars, each listing one PROPFIND per calendar. 384 were deleted through DSM's web API during the run |
2 changes: 1 addition & 1 deletion news/+added-parameters.bugfix.rst
Original file line number Diff line number Diff line change
@@ -1 +1 @@
``verify=True`` and the conformance suite no longer count a parameter the server adds as a difference where it says nothing about the value: SOGo adding ``RSVP=TRUE`` to an ATTENDEE, or Ox adding ``EMAIL=``, loses nothing, and neither does Ox adding the organizer as an attendee. A parameter the server drops or changes still is one, and so is one it adds with a meaning of its own, such as a TZID or ``PARTSTAT=DECLINED``.
``verify=True`` and the conformance suite no longer count a parameter the server adds as a difference where it says nothing about the value: SOGo adding ``RSVP=TRUE`` to an ATTENDEE, or Ox adding ``EMAIL=``, loses nothing, and neither does Ox adding the organizer as an attendee. A parameter the server drops or changes still is one, and so is one it adds with a meaning of its own, such as a TZID or ``PARTSTAT=DECLINED``. A ``VALUE`` naming the property's default type, such as infomaniak's ``URL;VALUE=URI``, is no difference either.
2 changes: 1 addition & 1 deletion news/29.documentation.rst
Original file line number Diff line number Diff line change
@@ -1 +1 @@
The design documents no longer say that recurring-task completion works like caldav's: ``task.complete()`` on a recurring task (an ``RRULE`` or ``RDATE``) or an occurrence of one raises ``UnsupportedError`` on every backend until edit-icalendar has it, with the CalDAV escape hatch described meanwhile. They also say that on CalDAV an item returned by a write has ``native`` set to ``None`` until it is read back, and list edit-icalendar under the roadmap's risks.
The design documents no longer say that recurring-task completion works like caldav's: ``task.complete()`` on a recurring task (an ``RRULE`` or ``RDATE``) or an occurrence of one raises ``UnsupportedError`` on every backend until edit-icalendar has it, with the CalDAV escape hatch described meanwhile. They also say that on CalDAV an item returned by a write has ``native`` set to ``None`` whenever the server answers with an ETag, until ``get()``, ``search()`` or ``reload()`` reads it, and list edit-icalendar under the roadmap's risks.
15 changes: 14 additions & 1 deletion src/calendaring/_mapping.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
import icalendar
import recurring_ical_events
from icalendar.parser import Contentline
from icalendar.prop import TypesFactory

from .capabilities import Capabilities, Feature, Support
from .items import X_PLANNED_END, X_PLANNED_START
Expand Down Expand Up @@ -348,11 +349,23 @@ def _without_defaults(line: bytes) -> tuple[Any, ...]:
(key.upper(), tuple(sorted(_zone(str(v)) if key.upper() == "TZID" else str(v) for v in values)))
for key, raw in params.items()
for values in [raw if isinstance(raw, list) else [raw]]
if not (len(values) == 1 and PARAMETER_DEFAULTS.get(key.upper()) == str(values[0]).upper())
if not (len(values) == 1 and _default(name, key) == str(values[0]).upper())
)
return name.upper(), tuple(kept), value


def _default(name: str, key: str) -> str | None:
"""A parameter's RFC 5545 default on property ``name``: for VALUE the property's own type (URL's is URI).

icalendar names a list property's type with a ``-list`` suffix (RDATE's is ``date-time-list``),
which no VALUE parameter carries.
"""
if key.upper() == "VALUE":
default = TypesFactory.instance().types_map.get(name)
return default.upper().removesuffix("-LIST") if default else None
return PARAMETER_DEFAULTS.get(key.upper())


@functools.cache
def _zone(tzid: str) -> str:
"""A time zone by how it behaves, so that one tzdata renamed into another (Europe/Oslo, now a link to Berlin) compares equal.
Expand Down
133 changes: 132 additions & 1 deletion tests/_async/test_caldav_conformance.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,29 @@

from __future__ import annotations

from collections.abc import Sequence
from typing import Any

import pytest
from conftest import wait_for_server

from calendaring import AsyncBackend
from calendaring import (
AlreadyExistsError,
AsyncBackend,
AsyncCollection,
AsyncItem,
BackendError,
Event,
Item,
NotFoundError,
TransportError,
)
from calendaring.loss import ALLOW
from calendaring.testing import AsyncConformance

# A failed write is tried this many times in all while seeding
SEED_ATTEMPTS = 3


class CalDAVConformance(AsyncConformance):
# A CalDAV server is probed, not known in advance (§5.1).
Expand All @@ -23,9 +38,125 @@ async def open_backend(self) -> AsyncBackend:
wait_for_server(backend, self.monkeypatch)
return backend

async def seed(self, collection: AsyncCollection, items: Sequence[Item]) -> list[AsyncItem]:
"""Store each item, again after a failed write, unless it got there after all.

Seeding is setup, not what a test is about. Robur leaves about one
request in 60 unanswered (a 70-second hang, then a reset) and answers
some PUTs with a 500, so a test that seeds 120 items fails most runs on
a write that has nothing to do with it. Every ``BackendError`` is
retried, since it carries no status to tell a 500 from a refusal; a
refusal costs two more tries and is raised all the same. A clash is
only taken for a lost answer on a retry: the UIDs are fresh.
"""
seeded = []
for item in items:
for attempt in range(SEED_ATTEMPTS):
try:
seeded.append(await collection.add(item, loss=ALLOW))
break
except AlreadyExistsError:
if attempt == 0 or not await self._seeded(collection, item, seeded):
raise
break
except BackendError:
if await self._seeded(collection, item, seeded):
break
if attempt == SEED_ATTEMPTS - 1:
raise
return seeded

@staticmethod
async def _seeded(collection: AsyncCollection, item: Item, seeded: list[AsyncItem]) -> bool:
"""Whether a write whose answer was lost stored ``item``; if so it joins ``seeded``. A failed read says no."""
try:
seeded.append(await collection.get(item.uid))
except (NotFoundError, BackendError):
return False
return True


class TestCalDAVServers(CalDAVConformance):
@pytest.fixture(autouse=True)
def _server(self, caldav_server: Any, monkeypatch: pytest.MonkeyPatch) -> None:
self.server = caldav_server
self.monkeypatch = monkeypatch


class _DroppingCollection:
"""Fails ``add`` as told: a lost answer after storing the item or before, or a 500."""

def __init__(self, outcomes: list[str], gets: list[str] | None = None) -> None:
self.outcomes = outcomes # per add: "ok", "stored-then-lost", "lost" or "refused" (a 500, nothing stored)
self.gets = gets or [] # per get: "ok", "lost" (no answer) or "missing" (not found yet, though stored)
self.stored: dict[str, Any] = {}
self.adds = 0

async def add(self, item: Any, **_: Any) -> Any:
outcome = self.outcomes[self.adds] if self.adds < len(self.outcomes) else "ok"
self.adds += 1
if outcome not in ("lost", "refused"):
if item.uid in self.stored:
raise AlreadyExistsError(item.uid, taken="uid")
self.stored[item.uid] = item
if outcome == "refused":
raise BackendError("PutError at '500 Internal Server Error'")
if outcome != "ok":
raise TransportError("read timed out")
return item

async def get(self, uid: str) -> Any:
outcome = self.gets.pop(0) if self.gets else "ok"
if outcome == "lost":
raise TransportError("read timed out")
try:
if outcome == "missing":
raise KeyError(uid)
return self.stored[uid]
except KeyError:
raise NotFoundError(uid) from None


@pytest.mark.parametrize(
("outcomes", "gets", "adds"),
[
(["stored-then-lost"], [], 1),
(["lost"], [], 2),
(["lost", "stored-then-lost"], [], 2),
(["refused"], [], 2),
(["lost"], ["lost"], 2),
(["stored-then-lost"], ["lost"], 2),
],
)
async def test_seed_gets_past_a_lost_answer(outcomes: list[str], gets: list[str], adds: int) -> None:
"""Robur leaves about one request in 60 unanswered, and answers some PUTs with a 500; seeding 120 items fails most runs without this."""
collection = _DroppingCollection(outcomes, gets)
item = Event.new(summary="paged 0")
seeded = await CalDAVConformance().seed(collection, [item]) # type: ignore[arg-type]
assert [i.uid for i in seeded] == [item.uid]
assert collection.adds == adds


async def test_seed_gives_up_on_a_server_that_never_answers() -> None:
collection = _DroppingCollection(["lost"] * 10)
with pytest.raises(TransportError):
await CalDAVConformance().seed(collection, [Event.new(summary="paged 0")]) # type: ignore[arg-type]
assert collection.adds == SEED_ATTEMPTS


async def test_seed_does_not_take_a_clash_it_did_not_cause() -> None:
"""A fresh UID that is taken before any retry is a leftover or a backend bug, not a lost answer."""
item = Event.new(summary="paged 0")
collection = _DroppingCollection([])
collection.stored[item.uid] = item
with pytest.raises(AlreadyExistsError):
await CalDAVConformance().seed(collection, [item]) # type: ignore[arg-type]
assert collection.adds == 1


async def test_seed_gives_up_on_a_clash_it_cannot_read() -> None:
"""The lost write landed, but the item cannot be read back: the clash on the retry is raised."""
collection = _DroppingCollection(["stored-then-lost"], ["missing", "missing"])
with pytest.raises(AlreadyExistsError):
await CalDAVConformance().seed(collection, [Event.new(summary="paged 0")]) # type: ignore[arg-type]
assert collection.adds == 2
133 changes: 132 additions & 1 deletion tests/_sync/test_caldav_conformance.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading