Skip to content

Add bitswap_v1_get method#183

Merged
dmitry-markin merged 8 commits into
mainfrom
dm-bitswap-v1-get
Mar 27, 2026
Merged

Add bitswap_v1_get method#183
dmitry-markin merged 8 commits into
mainfrom
dm-bitswap-v1-get

Conversation

@dmitry-markin
Copy link
Copy Markdown
Contributor

This PR standardizes bitswap_v1_get RPC used to download data from Polkadot Bulletin Chain. Calling this method is equivalent to downloading the data from a full node via Bitswap protocol. Both light & full nodes must provide this endpoint.

Follow-ups

bitswap_v1_stream method for requesting streaming of multiple Bitswap chunks (multiple CIDs) will be introduced in a separate PR.

CC @ERussel

@dmitry-markin dmitry-markin requested review from bkontur and lexnv March 24, 2026 15:46
Comment thread src/api/bitswap.md
Comment thread src/api/bitswap_v1_get.md Outdated
Comment thread src/api/bitswap_v1_get.md Outdated


**Return value**: String containing the requested data chunk, hexadecimal-encoded. Always starts with
`0x...`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the CID maps to an empty block we return 0x?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but in practice it is not possible to store empty indexed transaction.

Comment thread src/api/bitswap_v1_get.md
**Parameters**:

- `cid`: CID of the data chunk requested serialized in a [string format](https://github.com/multiformats/cid/blob/edb1c5294ad2d8257812d7ded4941c3e0fafccf3/README.md#variant---stringified-form).
Only CIDv1 version in `base32` multibase encoding (string starting from `b...`) is supported.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we plan to extend these in the near future?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the default string representation of a CID and CIDv1 is the latest default as well, so we are free to define this as a requirement here and support nothing else.

Comment thread src/api/bitswap.md
Comment thread src/api/bitswap.md
@@ -0,0 +1,8 @@
# Introduction

The functions with `bitswap` prefix allow fetching data chunks from the storage given their CID,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I would add a tiny mention, "For the implementation details of the bitswap implementation see... url-link"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what implementation to include (litep2p / substrate / smoldot?), plus the implementation may change / get moved to another place and get outdated.

Comment thread src/SUMMARY.md
- [archive_v1_storage](api/archive_v1_storage.md)
- [archive_v1_storageDiff](api/archive_v1_storageDiff.md)
- [bitswap](api/bitswap.md)
- [bitswap_v1_get](api/bitswap_v1_get.md)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmitry-markin what about the stream version we discussed before?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will go as a separate PR. Going to open it when the streaming version is added to smoldot to be sure we don't miss anything important.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meanwhile, the RPC endpoint in the full node gained the priority, so I will implement it first before going back to streaming in smoldot.

Comment thread src/api/bitswap_v1_get.md
```

Please note again that only the `code` and corresponding error retry categories in the table above
are stable. Everything else is provided for debugging purposes only, subject to change, and must not
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case we could simplify the retur error code and incldue all details into the serialized message? I think chainHead follows a similar impl

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would go a JSON-RPC standard way of returning the data on success and using standard JSON-RPC error codes like -3602 for "invalid params" already familiar to web app developers.

Copy link
Copy Markdown
Contributor

@lexnv lexnv Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edit: I see what you mean, yep it makes sense

Comment thread src/api/bitswap_v1_get.md
| -32602 | `InvalidParams` | Invalid/unsupported CID was passed. Must not retry |
| -32810 | `Fail` | Permanent failure for this request, e.g. data not found |
| -32811 | `FailRetry` | Transient failure, can retry immediately |
| -32812 | `FailRetryBackoff` | Transient failure, can retry with a delay |
Copy link
Copy Markdown
Contributor Author

@dmitry-markin dmitry-markin Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only -32602 code here is standard JSON-RPC error code, and for application-defined error codes we can use something simple like -1, -2, -3. Unless we reserve "easy" codes for some common json-rpc-interface-spec errors — then we need to use unique codes per method to be consistent.

Copy link
Copy Markdown
Contributor Author

@dmitry-markin dmitry-markin Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, according to JSON-RPC 2.0 spec, the messages for standard codes (like InvalidParams in our case) seem to be standard ("Invalid params"). smoldot hardcodes a message for invalid params to "Invalid method parameter(s)." Theoretically, we can replace the hardcoded message with a human-readable description of a specific error — then we won't need to introduce a separate field data.details for a human-readable message in this particular case.

E.g., we can use a custom message Invalid params: {error} and remove data.details field completely.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"E.g., we can use a custom message Invalid params: {error} and remove data.details field completely."

Yep, lets only keep the code + message, then we can extend the message with all the other datails from data

Copy link
Copy Markdown
Contributor

@lexnv lexnv Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The design intiiatly showed the data variant:

{"code": -32812, "message": "No Bitswap peers connected", "data": {"variant": "NoPeers"}}

It could be simplified such that we place all details into the serialized message:

{"code": -32812, "message": "No Bitswap peers connected: No peers"}

Copy link
Copy Markdown
Contributor Author

@dmitry-markin dmitry-markin Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The data.variant actually makes sense, because it is suitable for structured logging in grafana etc. It is synonymous to message, not supplementing it.

But let's get rid of data.details completely and put the human-readable info into message for consistency of InvalidParams case with other cases.

lexnv
lexnv previously approved these changes Mar 26, 2026
x3c41a
x3c41a previously approved these changes Mar 26, 2026
@dmitry-markin dmitry-markin dismissed stale reviews from x3c41a and lexnv via 5fd4b58 March 26, 2026 15:18
@dmitry-markin
Copy link
Copy Markdown
Contributor Author

Added a minor clarification regarding limiting the number of retry attempts, and it rendered the approvals obsolete. @lexnv @x3c41a can you approve again, please?

Copy link
Copy Markdown
Contributor

@lexnv lexnv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks @dmitry-markin for spec here 🙏

@dmitry-markin dmitry-markin merged commit 06e3bae into main Mar 27, 2026
3 checks passed
@dmitry-markin dmitry-markin deleted the dm-bitswap-v1-get branch March 27, 2026 11:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants