Skip to content

[MEL] Use message extractor to completely replace inbox reader and tracker code#4593

Merged
ganeshvanahalli merged 15 commits intomasterfrom
replace-readerandtracker-with-messageextractor
Apr 10, 2026
Merged

[MEL] Use message extractor to completely replace inbox reader and tracker code#4593
ganeshvanahalli merged 15 commits intomasterfrom
replace-readerandtracker-with-messageextractor

Conversation

@ganeshvanahalli
Copy link
Copy Markdown
Contributor

@ganeshvanahalli ganeshvanahalli commented Apr 1, 2026

This PR aims to get rid of pending inbox reader and tracker dependency of nitro nodes, and instead to start using message extractor.

Resolves NIT-4753

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 1, 2026

Codecov Report

❌ Patch coverage is 48.94737% with 97 lines in your changes missing coverage. Please review.
✅ Project coverage is 34.30%. Comparing base (40eec39) to head (9d955a1).
⚠️ Report is 30 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4593      +/-   ##
==========================================
+ Coverage   34.12%   34.30%   +0.18%     
==========================================
  Files         497      497              
  Lines       59178    59233      +55     
==========================================
+ Hits        20193    20319     +126     
+ Misses      35432    35373      -59     
+ Partials     3553     3541      -12     

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 1, 2026

❌ 5 Tests Failed:

Tests completed Failed Passed Skipped
4801 5 4796 0
View the top 3 failed tests by shortest run time
TestDataStreaming_PositiveScenario/Many_senders,_long_messages
Stack Traces | 0.160s run time
=== RUN   TestDataStreaming_PositiveScenario/Many_senders,_long_messages
WARN [04-10|05:16:22.263] Served datastreaming_start               conn=127.0.0.1:40846 reqid=6 duration="111.84µs" err="we have already seen this request; aborting replayed protocol"
INFO [04-10|05:16:22.273] rpc response                             method=datastreaming_start logId=6 err="we have already seen this request; aborting replayed protocol" result={} attempt=0 args="[\"0x69d887a6\", \"0x31\", \"0xd9\", \"0x28bb\", \"0xa\", \"0x900690047222078d1be3f6874984e4bfbc9e908b10edb6019d9d60fd7e00d841702521c77a9ecfe43d5c49011eb790c6297621db9204c83b29da75b15f49f69f00\"]" errorData=null
    protocol_test.go:230: goroutine 282 [running]:
        runtime/debug.Stack()
        	/opt/hostedtoolcache/go/1.25.8/x64/src/runtime/debug/stack.go:26 +0x5e
        github.com/offchainlabs/nitro/util/testhelpers.RequireImpl({0x164b490, 0xc000183500}, {0x16319c0, 0xc000802960}, {0x0, 0x0, 0x0})
        	/home/runner/work/nitro/nitro/util/testhelpers/testhelpers.go:29 +0x9f
        github.com/offchainlabs/nitro/daprovider/data_streaming.testBasic.func1()
        	/home/runner/work/nitro/nitro/daprovider/data_streaming/protocol_test.go:230 +0x19b
        created by github.com/offchainlabs/nitro/daprovider/data_streaming.testBasic in goroutine 274
        	/home/runner/work/nitro/nitro/daprovider/data_streaming/protocol_test.go:223 +0x85
        
    protocol_test.go:230: �[31;1m [] we have already seen this request; aborting replayed protocol �[0;0m
--- FAIL: TestDataStreaming_PositiveScenario/Many_senders,_long_messages (0.16s)
TestDataStreaming_PositiveScenario
Stack Traces | 0.210s run time
=== RUN   TestDataStreaming_PositiveScenario
--- FAIL: TestDataStreaming_PositiveScenario (0.21s)
TestAnyTrustRekey
Stack Traces | 2.410s run time
... [CONTENT TRUNCATED: Keeping last 20 lines]
TRACE[04-10|05:18:59.988] Engine API request received              method=GetPayload                    id=0x032966b4657db2b5
DEBUG[04-10|05:18:59.987] Served eth_unsubscribe                   reqid=1671 duration="11.481µs"
DEBUG[04-10|05:18:59.988] Pushed sync data from consensus to execution synced=true  maxMessageCount=14  updatedAt=2026-04-10T05:18:59+0000 hasProgressMap=false
DEBUG[04-10|05:18:59.988] Pushed sync data from consensus to execution synced=true  maxMessageCount=3   updatedAt=2026-04-10T05:18:59+0000 hasProgressMap=false
DEBUG[04-10|05:18:59.988] Served eth_getBlockByNumber              reqid=198  duration="84.046µs"
DEBUG[04-10|05:18:59.988] Served eth_getTransactionCount           reqid=1561 duration="49.523µs"
DEBUG[04-10|05:18:59.988] Served eth_getBlockByNumber              reqid=1562 duration="24.656µs"
DEBUG[04-10|05:18:59.988] Pushed finality data from consensus to execution safeMsgCount=14 finalizedMsgCount=1 validatedMsgCount=0
TRACE[04-10|05:18:59.988] Handled RPC response                     reqid=1561 duration="1.052µs"
DEBUG[04-10|05:18:59.988] Executing EVM call finished              runtime="200.023µs"
DEBUG[04-10|05:18:59.988] Served eth_maxPriorityFeePerGas          reqid=173  duration="168.613µs"
DEBUG[04-10|05:18:59.988] Served eth_call                          reqid=389  duration="230.25µs"
TRACE[04-10|05:18:59.988] Handled RPC response                     reqid=1562 duration="1.113µs"
TRACE[04-10|05:18:59.988] Handled RPC response                     reqid=173  duration="1.072µs"
DEBUG[04-10|05:18:59.987] Served eth_blockNumber                   reqid=693  duration="22.392µs"
TRACE[04-10|05:18:59.988] Handled RPC response                     reqid=389  duration="1.383µs"
TRACE[04-10|05:18:59.988] Handled RPC response                     reqid=198  duration="1.253µs"
DEBUG[04-10|05:18:59.987] Served eth_unsubscribe                   reqid=694  duration="15.218µs"
--- FAIL: TestAnyTrustRekey (2.41s)
TRACE[04-10|05:18:59.988] Handled RPC response                     reqid=1638 duration="2.284µs"

📣 Thoughts on this report? Let Codecov know! | Powered by Codecov

Base automatically changed from mel-change-delayedmsg-accumulation to master April 2, 2026 15:29
rauljordan
rauljordan previously approved these changes Apr 2, 2026
@rauljordan rauljordan changed the title Use message extractor to completely replace inbox reader and tracker code [MEL] Use message extractor to completely replace inbox reader and tracker code Apr 6, 2026
@ganeshvanahalli ganeshvanahalli assigned eljobe and unassigned rauljordan Apr 7, 2026
@rauljordan
Copy link
Copy Markdown
Contributor

Hi @ganeshvanahalli I pushed some fixes from PR review toolkit which found the following real issues:
Code Fixes (5 files)
Critical #1
Added nil guard for InboxReader in GetParentChainDataSource() to prevent panic when both MessageExtractor and InboxReader are nil.

Critical #3
Changed return 0, nil to return m.config.RetryInterval, nil after preimage cache rebuild. Prevents unbounded CPU spin when the preimage is permanently missing from DB.

High #4
Replaced blocking channel send m.reorgEventsNotifier <- ... with a select on ctx.Done() to prevent deadlocking the MEL FSM if the reader goroutine is slow/stopped.

High #5
Extracted the error into a package-level sentinel errFindDelayedNotImplementedByMEL and downgraded the caller's log from Warn to Debug (also logging the error itself). Eliminates log spam on every message retrieval when MEL is active.

High #6
Moved ChainId validation before the BlobSchedule == nil early return, so chain ID mismatches are always detected even when BlobSchedule is nil.

Unit Tests Added 27 new test cases in mel_test.go

Copy link
Copy Markdown
Member

@eljobe eljobe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all Claude-generated PR review feedback.
I did cull it down from a longer list that didn't look like as much high-quality feedback.

Please take a look and see what can be addressed in this PR.

Comment thread arbnode/node.go
Comment thread arbnode/node.go Outdated
Comment thread arbnode/transaction_streamer.go
Comment thread arbnode/transaction_streamer.go Outdated
Comment thread arbnode/mel/runner/mel.go
Comment thread arbnode/inbox_reader.go
@eljobe eljobe assigned ganeshvanahalli and unassigned eljobe Apr 8, 2026
@rauljordan rauljordan requested a review from eljobe April 8, 2026 19:45
@rauljordan rauljordan assigned eljobe and unassigned ganeshvanahalli Apr 8, 2026
eljobe
eljobe previously approved these changes Apr 9, 2026
@ganeshvanahalli ganeshvanahalli added this pull request to the merge queue Apr 10, 2026
Merged via the queue into master with commit 540976c Apr 10, 2026
25 checks passed
@ganeshvanahalli ganeshvanahalli deleted the replace-readerandtracker-with-messageextractor branch April 10, 2026 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants