Skip to content

[DNM] feat(da): support fiber (not via c-node)#3244

Draft
julienrbrt wants to merge 36 commits intomainfrom
julien/fiber
Draft

[DNM] feat(da): support fiber (not via c-node)#3244
julienrbrt wants to merge 36 commits intomainfrom
julien/fiber

Conversation

@julienrbrt
Copy link
Copy Markdown
Member

@julienrbrt julienrbrt commented Apr 13, 2026

Overview

Support Fiber client (based on https://github.com/celestiaorg/celestia-app/blob/63fbf31cca216fc4e067a9e1b3a3431115c7009b/fibre), but not via celestia node or apex for this PoC
celestiaorg/celestia-node#4892

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 13, 2026

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b6f21e4b-5304-4c64-8304-a4408a07835a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch julien/fiber

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 13, 2026

The latest Buf updates on your PR. Results from workflow CI / buf-check (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed⏩ skipped✅ passed✅ passedApr 27, 2026, 12:15 PM

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented Apr 13, 2026

Claude finished @sysrex's task in 2m 55s —— View job


PR Review: feat(da): support fiber (not via c-node)

Tasks:

  • Gather context (AGENTS.md, PR overview)
  • Review block/internal/da/fiber_client.go
  • Review block/internal/da/fiber/types.go
  • Review block/internal/da/fibremock/mock.go
  • Review block/public.go
  • Review pkg/config/config.go and pkg/config/defaults.go
  • Review pkg/cmd/run_node.go
  • Review pkg/sequencers/solo/sequencer.go
  • Review tools/celestia-node-fiber/ package
  • Compile and post full review

Overview

This is a [DNM] PoC adding a Fiber DA backend directly (without going through celestia-node or apex). The architecture is clean — a thin fiber.DA interface, an in-memory mock, and a celestia-node adapter in tools/. The following covers the most important issues to address before this becomes production-ready.


Bugs

1. Submit ignores the caller's context — uses context.Background() instead

block/internal/da/fiber_client.go:92

result, err := c.fiber.Upload(context.Background(), namespace[len(namespace)-10:], flat)

If the caller cancels their context (e.g. on shutdown), the upload continues indefinitely. Should be ctx here.

Fix this →


2. Namespace slicing panics on short namespaces

block/internal/da/fiber_client.go:92 and :141

namespace[len(namespace)-10:]

If namespace has fewer than 10 bytes, this panics at runtime. There is no bounds check. Either validate namespace length in NewFiberClient or add a guard at the slice site.

Fix this →


3. GetLatestDAHeight panics instead of returning an error

block/internal/da/fiber_client.go:303-305

func (c *fiberDAClient) GetLatestDAHeight(context.Context) (uint64, error) {
    panic(fmt.Errorf("p2p should not be enabled"))
}

Panics in interface implementations are always wrong — they kill the whole process and bypass recover() in most callers. Return 0, fmt.Errorf("GetLatestDAHeight not supported for fiber backend") instead.

Fix this →


4. NewFiberDAClient in block/public.go panics on error

block/public.go:94

base, err := da.NewFiberClient(da.FiberConfig{...})
if err != nil {
    panic(err)
}

da.NewFiberClient only errors when cfg.Client == nil, and StartNode guards that path (line 157-159 in run_node.go), so in practice this won't fire. But a public constructor that panics is still a footgun. Change the signature to (FullDAClient, error) and let the caller handle the error.

Fix this →


5. MockDA subscriber index race on concurrent context cancellation

block/internal/da/fibremock/mock.go:196-233

The idx captured in the cleanup goroutine is the position at insertion time:

idx := len(m.subscribers)
m.subscribers = append(m.subscribers, subscriber{...})
// ...
go func() {
    <-ctx.Done()
    m.mu.Lock()
    last := len(m.subscribers) - 1
    if idx <= last {
        m.subscribers[idx] = m.subscribers[last]  // could remove wrong subscriber
    }
    m.subscribers = m.subscribers[:last]
    ...
}()

If a subscriber with a lower index is removed first (via the same swap-with-last pattern), the current subscriber's entry is now at a different index, but idx still holds the old one. The swap removes the wrong subscriber and leaks the channel or corrupts the list. This is benign for sequential tests but will break under concurrent subscribe/cancel patterns. Consider using a map keyed by an ID, or the sync/atomic pointer-to-list approach.


Design Issues

6. Height is always 0 after SubmitRetrieve is essentially broken

block/internal/da/fiber_client.go:115-124

Height: 0, /* TODO */

Because Submit returns Height: 0, any Retrieve(ctx, height, ns) call with a real height will never find blobs from Submit. This is why TestFiberClient_Retrieve_Success, TestFiberClient_Subscribe, and several others are t.Skip-ped. This is a known gap but it's worth tracking as a top-priority item — the Retrieve path is central to full-node sync.


7. P2P broadcast is silently disabled when Fiber is enabled — undocumented constraint

block/internal/executing/executor.go:631

if !e.config.DA.IsFiberEnabled() {
    // Broadcast header and data to P2P network

This is a significant architectural decision: Fiber mode and P2P sync are mutually exclusive. Full nodes in Fiber mode cannot receive blocks via P2P gossip; they must rely entirely on DA retrieval. This constraint deserves a comment (and ideally a section in the docs) because operators won't expect P2P to be disabled by a DA flag.


8. forwardFibreBlobs silently swallows errors without logging

tools/celestia-node-fiber/listen.go:72-76

event, err := a.fibreBlobToEvent(ctx, b.Blob, height)
if err != nil {
    continue  // no log
}

A blob is silently dropped. If this happens in production due to, say, a transient FSP failure, operators have no signal. Add a log.Warn() at minimum.

Fix this →


9. Listen does a Download per blob just to resolve DataSize

tools/celestia-node-fiber/listen.go (fibreBlobToEvent)

The doc comment acknowledges this tradeoff. For a PoC it's fine, but DataSize is only used for metrics/logging in BlobEvent — it's not load-bearing for correctness. Emitting DataSize: 0 (or deriving it from the v2 share) and skipping the extra round-trip would be significantly cheaper at scale.


Code Quality

10. retrieve bool parameter is unused — Retrieve and RetrieveBlobs are identical

block/internal/da/fiber_client.go:137

func (c *fiberDAClient) retrieve(ctx context.Context, height uint64, namespace []byte, _ bool) datypes.ResultRetrieve {

The bool is ignored; both callers behave identically. Either remove the parameter, or implement the distinction (include full data vs. IDs only) before shipping.


11. failOnNthUpload is defined but never used in any test

block/internal/da/fiber_client_test.go:468-481

Dead test helper. Remove or wire it up.


12. Magic constant fiberSubscribeChanSize = 42

block/internal/da/fiber_client.go:248

The value 42 is unusual. Use a named constant with a comment explaining the rationale (e.g. matches upstream blob.Subscribe buffer, or chosen to hold N seconds of expected traffic). Right now it looks like a placeholder.


13. Incorrect flag comment in config.go

pkg/config/config.go:102-103

// FlagConsensusChainID is the Chain ID of the celestia app node
FlagDAFiberConsensusChainID = FlagPrefixEvnode + "da.fiber.consensus_chain_id"

The comment says FlagConsensusChainID but the constant is FlagDAFiberConsensusChainID. Also the comment is a copy-paste of the description rather than describing the flag constant.


14. FiberDAConfig.Validate doesn't check KeyName or KeyringPath

pkg/config/config.go:311-338

When Enabled is true, signing payment promises requires a key. The validator checks ConsensusAddress, BridgeAddress, and ConsensusChainID, but not KeyName. If KeyName is empty, the adapter will fail at runtime rather than at startup validation.


Summary

Severity Count Notes
Bug 5 Context ignored, panic, namespace bounds, subscriber race, public panic
Design 4 Height unimplemented, P2P silent disable, error swallowing, per-blob Download
Code quality 5 Unused helper, magic const, dead param, wrong comment, incomplete validation

Given this is [DNM] PoC, the design gaps (height=0, Retrieve broken) are expected. The bugs and code quality issues are still worth fixing before the branch is ready for review since they'll otherwise generate noise later. Happy to help with any of these.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 13, 2026

Codecov Report

❌ Patch coverage is 90.65657% with 37 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.16%. Comparing base (2865d6d) to head (4485d91).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
block/public.go 0.00% 12 Missing ⚠️
block/internal/da/fibremock/mock.go 90.90% 5 Missing and 5 partials ⚠️
block/internal/da/fiber_client.go 96.74% 5 Missing and 3 partials ⚠️
pkg/sequencers/solo/sequencer.go 61.53% 5 Missing ⚠️
pkg/config/config.go 75.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3244      +/-   ##
==========================================
+ Coverage   62.33%   63.16%   +0.82%     
==========================================
  Files         122      124       +2     
  Lines       12873    13258     +385     
==========================================
+ Hits         8024     8374     +350     
- Misses       3968     3995      +27     
- Partials      881      889       +8     
Flag Coverage Δ
combined 63.16% <90.65%> (+0.82%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

julienrbrt and others added 7 commits April 14, 2026 15:12
Adds a fibremock package with:
- DA interface (Upload/Download/Listen) matching the fibre gRPC service
- In-memory MockDA implementation with LRU eviction and configurable retention
- Tests covering all paths

Migrated from celestiaorg/x402-risotto#16 as-is for integration.
@julienrbrt julienrbrt changed the title feat(da): support fiber (not via c-node) [DNM] feat(da): support fiber (not via c-node) Apr 20, 2026
julienrbrt and others added 15 commits April 20, 2026 14:46
Adds tools/celestia-node-fiber, a new Go sub-module that implements the
ev-node fiber.DA interface by delegating Upload, Download and Listen to a
celestia-node api/client.Client.

Upload and Download run locally against a Celestia consensus node (gRPC)
and Fibre Storage Providers (Fibre gRPC) — no bridge-node hop — using
celestia-node's self-sufficient client (celestiaorg/celestia-node#4961).
Listen subscribes to blob.Subscribe on a bridge node and forwards only
share-version-2 blobs, which is how Fibre blobs settle on-chain via
MsgPayForFibre.

The package lives in its own go.mod, parallel to tools/local-fiber, so
ev-node core does not inherit celestia-app / cosmos-sdk replace-directive
soup. A FromModules constructor accepts the Fibre and Blob Module
interfaces directly so callers can inject mocks or share an existing
*api/client.Client.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…#3280)

* test(celestia-node-fiber): showcase end-to-end Upload/Listen/Download

Adds tools/celestia-node-fiber/testing/, a single-validator in-process
showcase that boots a fibre-tagged Celestia chain + in-process Fibre
server + celestia-node bridge, registers the validator's FSP via
valaddr (with the dns:/// URI scheme the client's gRPC resolver
expects), funds an escrow account, and drives the full adapter
surface.

TestShowcase proves the round-trip: subscribe via Listen, Upload a
blob, wait for the share-version-2 BlobEvent that lands after the
async MsgPayForFibre commits, assert the BlobID from Listen matches
Upload's return, Download and diff the payload bytes.

The harness is intentionally single-validator — a 2-validator
Docker Compose showcase is planned as a follow-up for exercising real
quorum collection.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(celestia-node-fiber): scale showcase to 10 blobs, document DataSize gap

Upload 10 distinct-payload blobs through adapter.Upload, collect
BlobEvents via adapter.Listen until every BlobID is accounted for
(order-insensitive, rejects duplicates), then round-trip each blob
through adapter.Download to diff bytes. Catches routing bugs (wrong
blob returned for a BlobID) and duplicate-event bugs that a
single-blob test can't see.

Scaling the test also exposed a semantic issue: the v2 share carries
only (fibre_blob_version + commitment), so b.DataLen() — what
listen.go's fibreBlobToEvent reports today — is always 36, not the
original payload length ev-node's fibermock conveys. The adapter
can't derive the payload size from the subscription stream alone;
surfacing it correctly needs an x/fibre PaymentPromise lookup
(tracked as a TODO on fibreBlobToEvent). The test therefore asserts
DataSize is non-zero rather than matching len(payload).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…3281)

listen.go previously set BlobEvent.DataSize to b.DataLen(), which for
a share-version-2 Fibre blob is always the fixed share-data layout
(fibre_blob_version + commitment = 36 bytes) — not the original
payload length. That diverges from ev-node's fibermock contract and
misleads any consumer that uses DataSize to allocate buffers or
report progress.

The v2 share genuinely doesn't carry the original size, and x/fibre
v8 has no chain query to derive it from the commitment. The only
accurate path is to Download the blob and measure. Listen now does
exactly that before forwarding each event. The cost is one FSP
round-trip per v2 blob; can be made opt-out later if it hurts
throughput-sensitive use cases.

Tests:
- Showcase restores the strict DataSize == len(payload) assertion
  across all 10 blobs.
- Unit test TestListen_FiltersFibreOnlyAndEmitsEvent now stubs
  fakeFibre.Download to return a deterministic payload and asserts
  DataSize matches its length.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ight subscriptions (#3283)

feat(celestia-node-fiber): Listen takes fromHeight for resume subscriptions

Threads a fromHeight parameter through the Fibre DA Listen path so a
subscriber can rejoin the stream from a past block height without
missing blobs. Consumes the matching celestia-node API change landed
in celestiaorg/celestia-node#4962, which gave Blob.Subscribe a
fromHeight argument backed by a WaitForHeight loop.

Changes:

- block/internal/da/fiber/types.go: DA.Listen signature now takes
  fromHeight uint64. fromHeight == 0 preserves "follow from tip"
  semantics, >0 replays from that block forward.
- block/internal/da/fibremock/mock.go: replay matching blobs with
  height >= fromHeight before attaching the live subscriber.
- block/internal/da/fiber_client.go: outer fiberDAClient.Subscribe
  does not yet expose a starting height (datypes.DA doesn't plumb
  one), so pass 0 and defer resume-from-height wiring to a future
  datypes.DA change.
- tools/celestia-node-fiber/listen.go: propagate fromHeight to
  client.Blob.Subscribe on the celestia-node API.
- tools/celestia-node-fiber/go.mod: bump celestia-node to the merged
  pseudo-version (v0.0.0-20260423143400-194cc74ce99c) carrying #4962.
- tools/celestia-node-fiber/adapter_test.go: fakeBlob.subscribeFn
  gets the new fromHeight arg; TestListen_FiltersFibreOnlyAndEmitsEvent
  asserts that fromHeight=0 is forwarded.
- tools/celestia-node-fiber/testing/showcase_test.go: existing
  TestShowcase passes fromHeight=0. New TestShowcaseResume uploads 3
  blobs, discovers their settlement heights via a live Listen, then
  opens a fresh Listen with fromHeight at the first blob's height and
  verifies every historical blob is replayed with correct Height and
  DataSize.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…imental (#3289)

Picks up the chained celestia-app bump on celestia-node
feature/fibre-experimental, which carries the x/valaddr host:port
validation fix (celestia-app PR #7183).

Cascading changes required by the bump:

- celestia-app v8 → v9 across adapter.go, adapter_test.go, listen.go,
  testing/network.go, testing/bridge.go (the new celestia-node uses
  v9, so the consumer must too).
- testing/network.go drops the `dns:///` prefix from the in-process
  validator registration. The new x/valaddr ValidateBasic enforces
  host:port form, so `dns:///host:port` registrations are now rejected
  at tx time. gRPC's passthrough resolver dials bare `host:port`
  directly with no behavioural difference.

Verified locally:
  go vet -tags fibre ./...                       — clean
  go test -tags fibre -short -run TestShowcase   — pass

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants