Integrate new p2panda high-level API#218
Draft
adzialocha wants to merge 1 commit intoadz/fix-testsfrom
Draft
Conversation
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
12 times, most recently
from
84af564 to
e3aa553
Compare
adzialocha
force-pushed
the
adz/fix-tests
branch
7 times, most recently
from
87b3021 to
8d83882
Compare
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
from
e3aa553 to
a79f048
Compare
adzialocha
force-pushed
the
adz/fix-tests
branch
3 times, most recently
from
49c10ba to
9adc524
Compare
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
from
a79f048 to
6e34c9e
Compare
adzialocha
force-pushed
the
adz/fix-tests
branch
from
9adc524 to
074a7f1
Compare
This patch integrates the new `p2panda` crate into the "backend" of
Reflection.
`p2panda` is the new all-in-one high-level Node API which should make
usage of p2panda features easier, removing a bunch of code in
`reflection-node` which is now handled in p2panda directly, such as:
* Encoding / Decoding of operations and their message payloads
* Make sure changes to the append-only log are strictly serialized, to
avoid breaking consistency / integrity of data due to race conditions
* Re-playing all operations on demand, this is required by Reflection to
re-materialize all state for a document
* Manage header extensions to include log ids and prune flags
* Process local and remotely incoming operations ("ingest", "pruning")
* SQLite storage handling for logs and operations
* Mapping between logs and topics
* De-duplicate messages in ephemeral streams by adding timestamp
* Add public key / signatures to give provenance to ephemeral messages
Some features implemented in `reflection-node` stay as they're adding
functionality _on top_ of `p2panda`. Some of them might be moved there
as well in the future, such as:
* Author Presence: Indicate which authors have contributed to which
topic and if they're currently online.
Related issue: p2panda/p2panda#1002
* Persisted Topics: Store to persist all previously used topics
Unfortunately we need to remove one feature for now which is to change
"connection modes", causing the network to be disabled / enabled.
`reflection-node` handled that well in the past by only disabling the
networking layer while keeping the event processing layer intact. We
will have to do something similar in `p2panda` to offer this feature as
well since the new API surface will not allow such low-level access
anymore.
See related issue: p2panda/p2panda#1093
BREAKING CHANGES
These changes introduce breaking changes to the core operation format
(`previous` field was removed) and the header extensions changed which
means that all previously created, persisted operations will _not_ be
compatible anymore.
THIS RESULTS IN LOOSING ALL PREVIOUSLY CREATED DOCUMENTS.
Since `p2panda` is not stable yet this might happen again in the near
future, though this was the most important and largest update and we're
much closer to a stable API and protocol now.
Final remarks:
* Some minor changes in `reflection-doc` where also necessary, I
specifically decided to make `p2panda-core` a concrete dependency
there since too much functionality is used to justify re-exports from
`reflection-node`.
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
from
6e34c9e to
a45d239
Compare
adzialocha
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.