[StaticWebAssets] Zstd compression and Compression Dictionary Transport (CDT) support

[javiercn]

Summary

Add zstd compression and Compression Dictionary Transport (CDT, RFC 9842) support to the Static Web Assets SDK. This enables zstd as a publish-time compression format alongside gzip and brotli, and introduces delta compression for returning users by using previous versions of assets as zstd dictionaries.

Motivation

Blazor WebAssembly and other SPA applications ship significant amounts of static content (framework assemblies, JS bundles, CSS). Currently, the SDK compresses these with gzip and brotli at publish time. Two key limitations exist:

1. No zstd support: Zstd is now supported by all major browsers and offers a better speed/ratio tradeoff than gzip. More importantly, zstd supports external dictionaries, which enables CDT.
2. No delta updates: Every time users revisit an updated application, they re-download the full compressed content even if only small portions changed. For Blazor WASM apps with ~4MB of framework DLLs, a minor app update still requires re-downloading the entire payload.

Compression Dictionary Transport (RFC 9842) solves the second problem. When a user has previously visited a site, the browser can use the cached version of a resource as a dictionary for decompressing the updated version, dramatically reducing transfer sizes for subsequent visits.

Goals

• Add zstd as a publish-time compression format for static web assets, producing .zst files alongside .gz and .br
• Generalize the compression pipeline so that C# tasks are format-agnostic and driven by CompressionFormat MSBuild item metadata (making it straightforward to add future formats)
• Implement CDT support that produces .dcz (dictionary-compressed zstd) files when a previous build's asset pack is provided
• Generate an asset pack (.zip) at publish time containing the manifest and all assets, to be used as dictionary input for future builds
• Emit correct CDT endpoint metadata per RFC 9842 so that ASP.NET Core's content encoding negotiation can serve delta-compressed responses
• Keep dictionary support orthogonal from compression — the UsesDictionary metadata on CompressionFormat items controls whether a format participates in CDT, independent of the compression logic itself

Non-goals

• Runtime zstd decompression support — this is an ASP.NET Core runtime concern (tracked separately)
• Automatic dictionary storage/retrieval — the user provides the previous pack path; CI pipeline integration is out of scope
• Build-time compression — zstd is publish-only (like brotli max quality); build-time continues to use gzip for dev server performance
• Custom user-defined compression formats — the pipeline is extensible for SDK-defined formats, not arbitrary user plugins
• N→N+2 or multi-version dictionary chains — CDT uses only the immediately previous version as dictionary

Scenarios

Scenario 1: Zstd compression at publish time

A developer publishes a Blazor WASM app and wants zstd-compressed assets served alongside gzip and brotli.

No code changes needed — zstd is automatically included in the publish compression pipeline via the CompressionFormat MSBuild items. The publish output includes .zst files for all static web assets. ASP.NET Core's content negotiation serves them to browsers that send Accept-Encoding: zstd.

Scenario 2: CDT delta compression for returning users

A developer ships updates to a Blazor WASM app and wants returning users to download only the differences, not full files.

<PropertyGroup>
  <StaticWebAssetDictionaryCompression>true</StaticWebAssetDictionaryCompression>
  <StaticWebAssetPreviousAssetPack>$(ArtifactsPath)\previous\staticwebassets.pack.zip</StaticWebAssetPreviousAssetPack>
</PropertyGroup>

At publish:

1. The SDK reads the previous asset pack and matches current assets to previous versions by their non-fingerprinted route patterns
2. For each changed asset, the SDK compresses the current version using the previous version as a zstd dictionary, producing .dcz files
3. The SDK generates endpoint metadata with:
   • Use-As-Dictionary response header on all content-negotiated responses (identity, gzip, br, zstd) so browsers cache resources for future dictionary use
   • Dictionary-Hash endpoint property containing the SHA-256 of the dictionary (:base64: per RFC 9842 §2.2 Structured Fields Byte Sequence format)
   • Content-Encoding: dcb selector on delta-compressed endpoints
   • Vary: Accept-Encoding, Available-Dictionary for correct caching
4. A new asset pack is generated for the current version at $(StaticWebAssetPublishAssetPackPath) (default: $(OutputPath)staticwebassets.pack.zip)

The CI pipeline saves the asset pack between builds. On the next publish, the saved pack is provided as StaticWebAssetPreviousAssetPack.

Scenario 3: ASP.NET Core routing integration

At request time, ASP.NET Core's ContentEncodingNegotiationMatcherPolicy:

1. Reads the Dictionary-Hash endpoint property from the manifest
2. Compares it against the Available-Dictionary request header sent by the browser
3. If the hash matches, serves the .dcz response; otherwise falls back to br/gzip/zstd/identity

This ensures delta-compressed responses are only served when the browser has the correct dictionary cached.

Detailed design

Generalized compression framework

The compression pipeline was refactored from hardcoded gzip/brotli logic to a generic, metadata-driven model:

• CompressionFormat MSBuild items: Each format declares FileExtension, ContentEncoding, and optionally UsesDictionary metadata
• ApplyCompressionNegotiation: Refactored to use EndpointGroup<CompressionGroupState> for route-based grouping. Quality ranking, selector generation, and header emission are all driven by format metadata, not format names
• ResolveCompressedAssets: Parses the Formats property and resolves format metadata generically — no hardcoded format switching

New MSBuild tasks

Task	Purpose
ZstdCompress	ToolTask that invokes the SWA CLI tool for zstd compression with optional dictionary support
GeneratePublishAssetPack	Creates a ZIP containing the publish manifest and all assets, keyed by assets/{BasePath}/{RelativePath}
ResolveDictionaryCandidates	Opens a previous asset pack, matches current assets to previous versions by route pattern, outputs dictionary candidate items

Asset pack format

staticwebassets.pack.zip
├── manifest.json               # Standard StaticWebAssetsManifest
└── assets/
    ├── _content/MyApp/js/app.js
    ├── _content/MyApp/css/site.css
    └── ...                     # All publish assets by BasePath/RelativePath

Route-based dictionary matching

Dictionary candidates are matched by non-fingerprinted route patterns, not by file paths. This ensures stable matching across versions even when fingerprints change:

1. For each current asset, compute a MatchPattern from the endpoint route (e.g., site.*.js for site.{fingerprint}.js)
2. Scan the previous pack's manifest for assets whose endpoints match the same pattern
3. If a match is found and the content differs, output the previous asset as a dictionary candidate

CDT endpoint metadata (per RFC 9842)

For assets with dictionary candidates, the negotiation task emits:

{
  "Route": "/app.js",
  "Selectors": [
    { "Name": "Content-Encoding", "Value": "dcb", "Quality": "0.9" }
  ],
  "EndpointProperties": [
    { "Name": "Dictionary-Hash", "Value": ":pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=:" }
  ],
  "ResponseHeaders": [
    { "Name": "Vary", "Value": "Accept-Encoding, Available-Dictionary" }
  ]
}

Additionally, Use-As-Dictionary response headers are added to all content-negotiated responses for each asset so browsers build up dictionary caches over time.

MSBuild properties

Property	Default	Description
StaticWebAssetDictionaryCompression	false	Enable CDT support and the dcz compression format
StaticWebAssetPreviousAssetPack	(empty)	Path to the previous version's asset pack ZIP
StaticWebAssetPublishAssetPackPath	$(OutputPath)staticwebassets.pack.zip	Output path for the current version's asset pack

CompressionFormat items (defined in targets)

<CompressionFormat Include="gzip">
  <FileExtension>.gz</FileExtension>
  <ContentEncoding>gzip</ContentEncoding>
</CompressionFormat>
<CompressionFormat Include="brotli">
  <FileExtension>.br</FileExtension>
  <ContentEncoding>br</ContentEncoding>
</CompressionFormat>
<CompressionFormat Include="zstd">
  <FileExtension>.zst</FileExtension>
  <ContentEncoding>zstd</ContentEncoding>
</CompressionFormat>
<CompressionFormat Include="dcz" Condition="'$(StaticWebAssetDictionaryCompression)' == 'true'">
  <FileExtension>.dcz</FileExtension>
  <ContentEncoding>dcb</ContentEncoding>
  <UsesDictionary>true</UsesDictionary>
</CompressionFormat>

Performance results

Scenario	vs Brotli	Notes
Standalone zstd	+7.9% larger	Expected — zstd trades size for CDT capability
CDT delta (app update)	-21.7% smaller	Clear win for update payloads
Best individual file	-98.6% smaller	System.Collections: 8,674 → 118 bytes

For a typical Blazor WASM app update (adding a feature page):

• Without CDT: 624 KB re-download (brotli)
• With CDT: 489 KB re-download (dcz) — 135 KB savings

Testing

• Unit tests for all new tasks (ZstdCompress, GeneratePublishAssetPack, ResolveDictionaryCandidates)
• Unit tests for generalized ApplyCompressionNegotiation with CDT headers
• Unit tests for ComputeMatchPattern literal preservation
• Integration tests for zstd compression in build and publish pipelines
• Integration test: Publish_WithPreviousPack_GeneratesDczForModifiedAssets — end-to-end CDT flow
• Baseline tests updated for new .zst/.dcz format registration across all app types (Blazor WASM, standalone, etc.)

Open items

• Embedding dictionary hash in .dcz file to support multiple dictionaries per asset — needs format design
• MSBuild error when StaticWebAssetDictionaryCompression=true but pack file is missing/invalid
• Introduce named types (CompressionFormatInfo, PreviousRouteMatch) to replace tuples in task internals