Add trusted-server-adapter-axum native dev server (PR 16)

[prk-Jr]

Summary

• Adds trusted-server-adapter-axum as a native (non-WASM) dev server so the full trusted-server pipeline can be run and tested locally without Fastly Compute or Viceroy
• Promotes the axum adapter to a first-class workspace member by removing the global target = "wasm32-wasip1" override from .cargo/config.toml; Fastly-specific commands now pass --target wasm32-wasip1 explicitly
• Extends the integration test matrix so WordPress and Next.js scenarios run against both Fastly (Viceroy) and Axum, verifying identical behaviour across platforms

Changes

File	Change
crates/trusted-server-adapter-axum/src/platform.rs	PlatformConfigStore, PlatformSecretStore, PlatformBackend, PlatformGeo, PlatformHttpClient — env-var-backed implementations
crates/trusted-server-adapter-axum/src/middleware.rs	FinalizeResponseMiddleware + AuthMiddleware — mirrors Fastly adapter, always emits X-Geo-Info-Available: false
crates/trusted-server-adapter-axum/src/app.rs	TrustedServerApp implementing Hooks with all 11 routes wired
crates/trusted-server-adapter-axum/src/main.rs + axum.toml	Binary entry point; config drives port (default 8787)
crates/trusted-server-adapter-axum/tests/routes.rs	8 integration tests via EdgeZeroAxumService (no live TCP server)
crates/integration-tests/tests/environments/axum.rs	AxumDevServer runtime environment added to the matrix
crates/integration-tests/tests/environments/mod.rs	Register AxumDevServer alongside FastlyViceroy
crates/integration-tests/tests/integration.rs	test_wordpress_axum + test_nextjs_axum individual test functions
scripts/integration-tests.sh	Also builds the Axum native binary with test-specific env vars
.cargo/config.toml	Remove global target = "wasm32-wasip1"; keep only the viceroy runner
Cargo.toml	Move trusted-server-adapter-axum from [exclude] to [members]
crates/trusted-server-adapter-axum/Cargo.toml	Adopt workspace deps + lints
.github/workflows/test.yml	Add test-axum CI job; test-rust now passes --target wasm32-wasip1 explicitly
CLAUDE.md	Document axum adapter in workspace layout + build commands
.gitignore (root + adapter)	Ignore .edgezero/ (local KV store created by dev server)

Closes

Closes #497

Test plan

• cargo test --workspace (Fastly/WASM crates via Viceroy)
• cargo test -p trusted-server-adapter-axum (8 route + middleware tests)
• cargo clippy --workspace --all-targets --all-features -- -D warnings
• cargo fmt --all -- --check
• JS tests: cd crates/js/lib && npx vitest run (282 tests)
• Integration tests: test_wordpress_fastly, test_nextjs_fastly, test_wordpress_axum, test_nextjs_axum all pass
• Manual testing: cargo run -p trusted-server-adapter-axum starts on port 8787

Checklist

• Changes follow CLAUDE.md conventions
• No unwrap() in production code — use expect("should ...")
• Uses log macros (not println!)
• New code has tests
• No secrets or credentials committed

[aram356]

Summary

Introduces a native Axum dev-server adapter that runs the full trusted-server pipeline locally without Fastly Compute or Viceroy, promotes the crate to a workspace member, and extends the integration-test matrix to cover both runtimes. Requesting changes for a handful of small but concrete issues: unused deps, stale lockfile, doc/code drift, and a middleware bypass on the startup-error path.

Blocking

🔧 wrench

• Unused direct dependencies: serde_json and trusted-server-js declared in crates/trusted-server-adapter-axum/Cargo.toml but never referenced in src/ or tests/. See inline.
• Stale crate-local Cargo.lock: ~100 KB file that cargo now silently ignores (crate is a workspace member). Delete it to avoid lockfile drift.
• CLAUDE.md comment contradicts the PR: says "excluded from workspace" at line 14 while the PR makes it a workspace member. See inline.
• Log text references a non-existent NoopKvStore: the code uses UnavailableKvStore (src/platform.rs:367 vs :379). See inline.
• Startup-error router bypasses middleware: startup_error_router() has no FinalizeResponseMiddleware / AuthMiddleware, breaking the "every response has X-Geo-Info-Available" invariant and bypassing operator headers + basic auth on startup-failure responses. See inline on src/app.rs:134.

Non-blocking

🤔 thinking

• send_async/select diverges from Fastly (eager execution changes error-surface timing and ordering). Trade-off is documented, but consider a breadcrumb log or real tokio::spawn fan-out. See inline.
• Body::Stream outbound body silently truncated to empty on axum. See inline.
• Env-var namespace collisions possible due to -/./ → _ normalization. See inline.
• Fixed port 8787 baked at compile time can TIME_WAIT-flake across sequential integration tests. See inline.

♻️ refactor

• reqwest::Client rebuilt per request — defeats connection pool. Move to shared Arc<AxumPlatformHttpClient> in AppState. See inline.
• Env-var tests not isolated — use temp-env (already a workspace dep). See inline.
• bytes::Bytes → Vec<u8> round-trip on request/response bodies is wasted allocation. See inline.
• tests/routes.rs uses .unwrap() instead of .expect("should ...") — CLAUDE.md applies to test code. See inline.

🌱 seedling

• No /health endpoint — AxumDevServer::health_check_path() returns /health but the app never registers it; tests work around with wait_for_any_response. See inline.

⛏ nitpick

• Crate-local .gitignore is redundant with the workspace .gitignore. See inline.
• build_per_request_services is a no-op wrapper around build_runtime_services. See inline.

📝 note

• test-axum CI job runs without TRUSTED_SERVER__* env vars — every request goes through the startup-error path. Smoke tests still pass, but not a great signal. See inline.

CI Status

• fmt: PASS (verified locally, cargo fmt --all -- --check)
• clippy (axum crate, host target): PASS with zero warnings
• cargo test -p trusted-server-adapter-axum: 18 tests PASS
• GitHub CI on 75fe0d01: prepare artifacts + integration tests + browser tests all PASS

[ChristianPavilonis]

Summary

Nice addition overall — the Axum adapter is headed in a useful direction. I found a few correctness gaps in the platform shim plus a couple of documentation mismatches around what the Axum runtime can currently support.

[ChristianPavilonis]

Auction fan-out becomes serial on Axum. send_async() awaits self.execute(), and execute() fully buffers the upstream response before returning. The auction orchestrator in trusted-server-core assumes send_async() only launches the request and that select() yields whichever backend finishes first, so this changes response ordering, deadline enforcement, and response_time_ms accounting for multi-provider auctions.

Could we keep a real in-flight handle here (for example by storing spawned futures/joins and polling them from select()), or gate /auction off on Axum until the platform layer can preserve Fastly's concurrency semantics?

[ChristianPavilonis]

Streaming outbound bodies are dropped. On the Axum path, non-JSON inbound bodies are intentionally preserved as Body::Stream by edgezero_adapter_axum::into_core_request(). handle_publisher_request() then forwards the original request through services.http_client().send(...), but this match arm only logs and sends an empty body. That means proxied POSTs/uploads can reach the publisher origin with no payload.

Please either stream/buffer the body into reqwest here, or fail the request explicitly instead of silently truncating it.

[ChristianPavilonis]

These admin routes are wired up even though Axum can't persist the requested writes. AxumPlatformConfigStore::{put,delete} and AxumPlatformSecretStore::{create,delete} always return "not supported", but /admin/keys/rotate and /admin/keys/deactivate are still exposed here. With request signing enabled and valid admin auth, /admin/keys/rotate currently returns 500 as soon as the first store write happens.

I think we should either hide these routes on Axum or convert the unsupported-operation path into a clear 501/4xx response instead of advertising working admin endpoints.

[ChristianPavilonis]

The Axum quick-start implies .env is auto-loaded, but the adapter never does that. The new dev-server path reads process environment variables directly (plus axum.toml); nothing in the adapter sources .env. As written, cp .env.dev .env looks sufficient when users actually need to export/source those variables into their shell.

Can we update this section to show how to load the variables into the environment and mention the TRUSTED_SERVER_CONFIG_* / TRUSTED_SERVER_SECRET_* conventions the Axum platform store uses?

[ChristianPavilonis]

This overstates Axum parity with Fastly. The new adapter still has several important limitations (no KV store, no geo lookup, no config/secret-store writes, and send_async() does not preserve Fastly-style fan-out semantics). Calling it "the full trusted-server pipeline" and useful for "non-Fastly deployments" sets the wrong expectation for readers.

I'd reframe this as a local development / integration-test adapter and call out the current limitations explicitly.

[aram356]

Summary

Re-review of the Axum dev-server adapter against main. Most prior feedback (aram356, ChristianPavilonis) has been addressed, and the latest commit even fixed the auction fan-out concern via real tokio::spawn + select_all. Deep verification finds five blocking CI/correctness issues plus several non-blocking refactor and documentation items.

A compounding factor: this PR targets feature/edgezero-pr15-remove-fastly-core, not main. The Run Tests and Run Format workflows only trigger on PRs to main, so no CI run on this PR has actually exercised fmt/clippy/test-fastly/test-axum. Only Integration Tests runs. The fmt/clippy failures below would surface for the first time when this lands on main.

Blocking

🔧 wrench

• cargo fmt --all -- --check fails on crates/trusted-server-adapter-axum/src/app.rs:217-225 (auction handler Ok(...) block). See inline.
• cargo clippy ... -- -D warnings fails on crates/trusted-server-adapter-axum/src/platform.rs:194-196 with clippy::type_complexity. See inline.
• TRUSTED_SERVER__SYNTHETIC__SECRET_KEY is dead — should be TRUSTED_SERVER__EDGE_COOKIE__SECRET_KEY after PR15's rename. Two locations: scripts/integration-tests.sh:63 and .github/actions/setup-integration-test-env/action.yml:97. Effect: Axum CI integration tests run with the placeholder secret. See inline.
• CLAUDE.md self-contradicts — CLAUDE.md lines 67-69 (Build & Test Commands, modified in this PR) say cargo test-fastly / cargo test-axum; the CI Gates section near line 256 (unchanged) still says 3. `cargo test --workspace` , which AGENTS.md (also updated in this PR) explicitly forbids: "Do NOT use bare cargo test --workspace — it will attempt to compile the Fastly adapter for the host target." Update CI Gates item 3 to list both aliases. (Body-level because the affected line wasn't part of this PR's diff.)
• cargo clippy --workspace --all-targets --all-features -- -D warnings will fail post-merge to main. Removing [build] target = "wasm32-wasip1" from .cargo/config.toml means format.yml's clippy step now targets host, where the fastly crate's wasm-only APIs don't compile. The Run Format workflow doesn't trigger on PRs to feature branches, so this is hidden until rebase onto main. Fix options: add --target wasm32-wasip1 to the clippy CI step (excluding axum), or split into clippy-fastly / clippy-axum aliases analogous to the test aliases.

Non-blocking

🤔 thinking

• Fan-out parity isn't actually tested — send_async × 2 + select has no unit coverage. See inline on platform.rs:309.
• Body::Stream is silently buffered into Vec<u8> with no log emission; large-payload divergence vs Fastly will not surface in test output. See inline on platform.rs:247.

♻️ refactor

• startup_error_router's closure-of-closure (make) is convoluted. See inline on app.rs:134.
• rotate_handler / deactivate_handler are aliases for the same closure. See inline on app.rs:211.

🌱 seedling / 📌 out of scope

• health_check_path() returns /health but TrustedServerApp::routes() never registers it. See inline on environments/axum.rs:71.
• Per-request reqwest::Client revert masks a likely keep-alive bug in the empty-body branch (if !body_bytes.is_empty() { builder = builder.body(...) }). Worth a follow-up issue. See inline on platform.rs:456.
• fastly.toml:42-48 adds an [local_server.config_stores.trusted_server_config] config store with edgezero_enabled = "true". That's the EdgeZero gate, not the Axum adapter — likely a merge-resolution remnant from the Merge feature/edgezero-pr15-remove-fastly-core into PR16 commit. Confirm it belongs here or rebase it out.

📝 note

• test-axum CI job runs with no TRUSTED_SERVER__* env vars; every request flows through startup_error_router. The 8 route tests only assert status != 404 and < 500, both satisfied by 401/501 — pre-existing finding (aram356), deferred. See inline on test.yml:75.

CI Status

• fmt: FAIL (verified locally — diff in app.rs:217)
• clippy: FAIL (verified locally — type_complexity in platform.rs:194)
• cargo test-axum (host target): PASS (18 tests)
• Integration Tests workflow on 9a0d38c: not yet run for the latest commit (last green run was a780034)
• Run Tests / Run Format workflows: not exercised — this PR doesn't target main

[aram356]

🔧 wrench — cargo fmt --all -- --check fails on this block. rustfmt wants to collapse the multi-line Ok(...). Run cargo fmt --all and recommit.

Verified locally:

Diff in crates/trusted-server-adapter-axum/src/app.rs:217:
-                Ok(
-                    handle_auction(&s.settings, &s.orchestrator, &services, req)
-                        .await
-                        .unwrap_or_else(|e| http_error(&e)),
-                )
+                Ok(handle_auction(&s.settings, &s.orchestrator, &services, req)
+                    .await
+                    .unwrap_or_else(|e| http_error(&e)))

The Run Format workflow doesn't trigger on PRs to feature/edgezero-pr15-remove-fastly-core, so this fmt failure has been hidden. It will surface as soon as this PR retargets main.

[aram356]

🔧 wrench — cargo clippy --workspace --all-targets --all-features -- -D warnings fails here with clippy::type_complexity:

error: very complex type used. Consider factoring parts into `type` definitions
   --> crates/trusted-server-adapter-axum/src/platform.rs:194:13
    |
194 |       handle: tokio::task::JoinHandle<
195 | |         Result<(u16, Vec<(String, Vec<u8>)>, Vec<u8>), Report<PlatformError>>,
196 | |     >,

Fix — extract a type alias:

type SpawnedRequestResult =
    Result<(u16, Vec<(String, Vec<u8>)>, Vec<u8>), Report<PlatformError>>;

struct AxumPendingHandle {
    backend_name: String,
    handle: tokio::task::JoinHandle<SpawnedRequestResult>,
}

Like the fmt failure on app.rs:221, this is hidden because format.yml only runs on PRs to main.

[aram356]

🔧 wrench — Wrong env var name; Axum integration tests run with a placeholder secret key.

PR15 renamed the synthetic settings section to edge_cookie. The codebase reads TRUSTED_SERVER__EDGE_COOKIE__SECRET_KEY (crates/trusted-server-core/src/settings_data.rs:47), and the Fastly build at line 56 of this script sets it correctly. The Axum build was missed.

Effect: the Axum binary used by test_wordpress_axum / test_nextjs_axum falls back to the placeholder secret, only emits a log::warn, and the tests pass without exercising the real signing path.

Fix:

-TRUSTED_SERVER__SYNTHETIC__SECRET_KEY="integration-test-secret-key" \
+TRUSTED_SERVER__EDGE_COOKIE__SECRET_KEY="integration-test-secret-key" \

Same change required at .github/actions/setup-integration-test-env/action.yml:97.

[aram356]

🔧 wrench — Same synthetic→edge_cookie rename miss as scripts/integration-tests.sh:63. The Fastly build above (line 87) uses the correct name; this Axum build does not.

Fix:

-        TRUSTED_SERVER__SYNTHETIC__SECRET_KEY: integration-test-secret-key
+        TRUSTED_SERVER__EDGE_COOKIE__SECRET_KEY: integration-test-secret-key

Without this, the CI Axum binary in Integration Tests runs with the placeholder secret and the EdgeCookie/HMAC path is effectively unverified for the Axum runtime.

[aram356]

🤔 thinking — Fan-out parity isn't actually tested.

The new tokio::spawn + select_all implementation looks correct, but no unit or integration test calls send_async twice and select to verify (a) both spawn concurrently, (b) the ready handle is dropped from remaining, and (c) the backend-name mapping survives the ready_idx shift. The auction orchestrator is the canonical caller and would be the worst place to discover a bug here.

Suggestion: add a small unit test in this #[cfg(test)] module that boots two short-lived tokio::time::sleep-backed handles via send_async (skip reqwest), selects, and asserts both ready and remaining. Same harness used in edgezero-core would do.

[aram356]

♻️ refactor — Closure-of-closure is hard to read. make returns a move |msg| that returns the actual handler closure, then is called four times to bind a clone of message.

Suggestion — define a small helper fn or just inline:

let make_handler = |msg: Arc<String>| {
    move |_ctx: RequestContext| {
        let body = edgezero_core::body::Body::from((*msg).clone());
        let mut resp = Response::new(body);
        *resp.status_mut() = status;
        resp.headers_mut().insert(
            header::CONTENT_TYPE,
            HeaderValue::from_static("text/plain; charset=utf-8"),
        );
        async move { Ok::<Response, EdgeError>(resp) }
    }
};

RouterService::builder()
    .middleware(FinalizeResponseMiddleware::new(Arc::new(Settings::default())))
    .get("/", make_handler(Arc::clone(&message)))
    // ...

Drops one indirection.

[aram356]

♻️ refactor — These two bindings are aliases for the same closure. Pass the closure directly to both .post(...) calls and drop the let rotate_handler = ...; let deactivate_handler = ...; lines. Saves four lines and the cognitive load of "are these two handlers actually different?"

[aram356]

🌱 seedling — Pre-existing finding (aram356, deferred): health_check_path() returns /health but TrustedServerApp::routes() never registers it. This works only because wait_for_any_response on line 95 was added as a workaround. A future contributor swapping to the shared wait_for_ready helper would silently break the readiness gate.

The clean fix is one line in app.rs:

.get("/health", |_ctx: RequestContext| async {
    Ok::<Response, EdgeError>(Response::new(edgezero_core::body::Body::from("ok")))
})

Then wait_for_any_response can be deleted in favor of the shared helper. Worth a follow-up issue.

[aram356]

🌱 seedling — The comment says sharing a pooled client "regressed the Next.js server-action → API-route integration flow by reusing a poisoned connection after a truncated POST."

That description is consistent with the empty-body branch at lines 267-269 / 337-339:

if !body_bytes.is_empty() {
    builder = builder.body(body_bytes);
}

For a request that should have an empty body (POST with Content-Length: 0), this skips setting the body — and reqwest may send the request without Content-Length at all, which can confuse keep-alive on the server side. A subsequent reused connection then desyncs.

Suggestion — open a follow-up issue and reference it from this comment. The right long-term fix is connection-level (always set body, even when empty), not per-request client construction. Keeping a fresh client per request hides the underlying bug and burns TCP+TLS handshakes on every outbound call.

[aram356]

📝 note — Pre-existing finding (aram356, deferred to follow-up): this test-axum job runs with no TRUSTED_SERVER__* env vars. get_settings() fails at startup, so every request in tests/routes.rs flows through startup_error_router and gets 401 (basic auth) or the structured error response.

The 8 tests in tests/routes.rs only assert status != 404 and < 500, both of which are satisfied by 401/501. So the smoke tests pass without ever touching real handler logic.

Not blocking — but please document this in tests/routes.rs or wire the env vars (already centralized in setup-integration-test-env/action.yml) in a follow-up. As written, this CI job is mostly a routing-and-auth smoke check, not a handler smoke check.