Open
Conversation
There was a problem hiding this comment.
Pull request overview
Adds first-class WebSocket compression support to libHttpClient across built-in providers, introducing an explicit options surface (HCWebSocketSetOptions / HCWebSocketOptions) for legacy vs deterministic semantics, compression negotiation, response-header access, and Windows-family hybrid routing to a compression-capable backend when requested.
Changes:
- Introduces
HCWebSocketOptions+HCWebSocketSetOptions(...), deterministic receive semantics (hard-cap +TooLargeclose), andHCWebSocketDisconnectWithStatus(...). - Adds upgrade response header capture + public accessors (
HCWebSocketGetResponseHeader*), and plumbs capture through WinHTTP/WinRT/Android paths. - Adds build/dependency plumbing (boost-wintls, websocketpp/asio updates), Windows hybrid provider routing, and new compression validation tests/samples.
Reviewed changes
Copilot reviewed 63 out of 64 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| libHttpClient.vs2022.sln | Adds Win32 compression test projects |
| libHttpClient.vs2019.sln | Adds Win32 compression test projects |
| hc_settings.props.example | Documents compression build knobs |
| cgmanifest.json | Updates deps; adds boost-wintls |
| Utilities/FrameworkResources/exports.exp | Exports new WebSocket APIs |
| Tests/WebSocketCompression/WebSocketCompressionTests.Win32.vcxproj | New Win32 compression test app |
| Tests/WebSocketCompression/WebSocketCompressionIntegrationTests.Win32.vcxproj | New Win32 cert-store integration app |
| Tests/UnitTests/Tests/WebsocketTests.cpp | Adds options/headers/disconnect tests |
| Source/WebSocket/websocket_publics.cpp | Adds new public WebSocket entrypoints |
| Source/WebSocket/websocket_options.h | Adds WebSocket options helpers |
| Source/WebSocket/iOS/ios_websocket.cpp | Removes stub iOS WebSocket file |
| Source/WebSocket/hcwebsocket.h | Adds options/headers APIs + state tracking |
| Source/WebSocket/hcwebsocket.cpp | Implements options/headers + deterministic logic |
| Source/WebSocket/WinRT/winrt_websocket.cpp | Caches upgrade response headers (WinRT) |
| Source/WebSocket/Websocketpp/wintls_socket.hpp | WinTLS transport adapter (websocketpp) |
| Source/WebSocket/Websocketpp/websocketpp_websocket.h | Provider lifecycle + options support surface |
| Source/WebSocket/Websocketpp/websocketpp_disabled_permessage_deflate.hpp | Wrapper for vendored websocketpp bug |
| Source/WebSocket/Websocketpp/websocketpp_configured_permessage_deflate.hpp | Wrapper to control offer params |
| Source/WebSocket/Android/AndroidWebSocketProvider.cpp | Caches upgrade response headers (Android) |
| Source/Platform/Win32/PlatformComponents_Win32.cpp | Uses hybrid provider when enabled |
| Source/Platform/IWebSocketProvider.h | Adds OptionsResult + IProviderLifecycle |
| Source/Platform/GDK/PlatformComponents_GDK.cpp | Hybrid provider + lifecycle notifications |
| Source/HTTP/WinHttp/winhttp_websocket_hybrid.h | Declares WinHTTP hybrid WS provider |
| Source/HTTP/WinHttp/winhttp_websocket_hybrid.cpp | Implements hybrid routing + lifecycle fanout |
| Source/HTTP/WinHttp/winhttp_provider.h | WinHttp WS provider implements lifecycle |
| Source/HTTP/WinHttp/winhttp_provider.cpp | Implements suspend/resume hooks (GDK) |
| Source/HTTP/WinHttp/winhttp_connection.h | Adds close + header helpers |
| Source/HTTP/WinHttp/winhttp_connection.cpp | Deterministic hard-cap + header capture |
| Source/Global/NetworkState.h | Adds WS suspend/resume notifications |
| Source/Global/NetworkState.cpp | Implements lifecycle notifications + queue dup |
| Source/Common/Apple/utils_apple.mm | Switches proxy username/pass to String |
| Source/Common/Apple/utils_apple.h | Updates proxy helper signature |
| Samples/WebSocketEchoServer/WebSocketEchoServer.vcxproj.filters | Adds zlib sources to filters |
| Samples/WebSocketEchoServer/WebSocketEchoServer.vcxproj | Adds zlib sources/include paths |
| Samples/WebSocketEchoServer/WebSocketEchoServer.cpp | Enables permessage-deflate negotiation |
| SECURITY_AUDIT_BOOST-WINTLS.md | Adds security audit writeup |
| README.md | Documents options/legacy vs deterministic behavior |
| NOTICE.txt | Updates notices for deps + boost-wintls |
| Include/httpClient/pal.h | Adds S_FALSE definition |
| Include/httpClient/httpProvider.h | Adds response header API declarations |
| Include/httpClient/httpClient.h | Adds options enum + new public WS APIs/docs |
| Build/libHttpClient.Win32/libHttpClient.Win32.vcxproj | No-websockets .def selection |
| Build/libHttpClient.Win32/libHttpClient.Win32.def | Exports new WS APIs |
| Build/libHttpClient.Win32/libHttpClient.Win32.NoWebSockets.def | New no-websockets export list |
| Build/libHttpClient.Win32.Shared/libHttpClient.Win32.Shared.vcxitems.filters | Adds hybrid/websocketpp filters |
| Build/libHttpClient.Win32.Shared/libHttpClient.Win32.Shared.vcxitems | Adds compression build gating |
| Build/libHttpClient.Linux/libHttpClient_Linux.bash | Adds websocket compression toggle |
| Build/libHttpClient.Linux/README.md | Documents compression build toggle |
| Build/libHttpClient.Linux/CMakeLists.txt | Adds compression option + test target |
| Build/libHttpClient.GDK/libHttpClient.GDK.vcxproj | No-websockets .def selection |
| Build/libHttpClient.GDK/libHttpClient.GDK.def | Exports new WS APIs |
| Build/libHttpClient.GDK/libHttpClient.GDK.NoWebSockets.def | New no-websockets export list |
| Build/libHttpClient.GDK.Shared/libHttpClient.GDK.Shared.vcxitems.filters | Adds websocketpp filters |
| Build/libHttpClient.GDK.Shared/libHttpClient.GDK.Shared.vcxitems | Adds compression + console gating |
| Build/libHttpClient.CMake/GetLibHCFlags.cmake | Adds compression + fixes nowe check |
| Build/libHttpClient.Apple.C/libHttpClient.xcodeproj/project.pbxproj | Enables compression + zlib include path |
| Build/libHttpClient.Android/src/main/java/com/xbox/httpclient/HttpClientWebSocket.java | Passes upgrade headers via JNI |
| .gitmodules | Adds boost-wintls submodule |
| .gitignore | Ignores .github/working/ |
Add the websocketpp WebSocket provider, the WinHTTP hybrid routing layer, and the platform-specific plumbing needed to select that provider on Win32 and GDK. Update the WinHTTP connection path, WinRT and Apple helpers, network-state handling, and the websocketpp configuration headers so the new transport can own deterministic and compression-capable connections.
Rename the compression-only control surface to HCWebSocketSetOptions and HCWebSocketOptions, add the LegacySemantics escape hatch, and make deterministic behavior explicit on the handle. Update the Android-facing surface, public headers, exports, and documentation so callers can reason about fragment-handler support, max receive buffer behavior, and compression negotiation through one options contract.
Add the local echo-server sample, the focused Win32 compression test projects, and the unit-test updates that lock down the final options contract. Keep the review surface centered on compression, certificate validation, and explicit options behavior by dropping the now-redundant standalone semantics harness from the branch.
- Add boost-wintls (BSL-1.0) to NOTICE.txt and cgmanifest.json - Update asio commit hash from 22afb86 to 03ae834 (asio-1-32-0) in both files to match the current submodule pointer
- Add DEFINE_ENUM_FLAG_OPERATORS(HCWebSocketOptions) so callers can combine flags with | without casting through uint32_t - Add m_stateMutex lock to SetOptions() to match the locking pattern already used in SetMaxReceiveBufferSize() - Add m_stateMutex lock and pre-connect guard to SetPingInterval() for consistency with all other pre-connect setters - Return E_INVALIDARG from GetResponseHeaderAtIndex() when headerIndex is out of range instead of returning S_OK with null output pointers
- Add Websocketpp and WinHttp Hybrid filter entries to the Win32.Shared vcxitems.filters so the new files appear under proper Solution Explorer folders (matching the existing GDK.Shared pattern) - Document the intentional raw new in wintls_socket.hpp where websocketpp's transport layer requires std::shared_ptr - Document the leak-rather-than-deadlock design choice in the detached joiner thread shutdown fallback - Note the intentional proxy URI parser duplication between websocketpp_websocket.cpp and winhttp_connection.cpp
- clarify deterministic vs legacy WebSocket behavior across platforms - document receive buffer, fragment callback, and compression behavior in the public header and README - reorder pre-connect WebSocket APIs in the header to match the intended setup flow - scope WinHTTP websocket-only proxy helpers under HC_NOWEBSOCKETS Co-authored-by: Copilot <copilot@github.com>
11e6957 to
7dad3d9
Compare
…e assert 1. SetOptions: move HasBinaryMessageFragmentHandlers() before m_stateMutex acquisition to eliminate latent m_stateMutex -> m_eventCallbacksMutex nesting hazard. 2. wintls_socket translate_ec: add comment explaining that wintls::error_code resolves to the lib::error_code identity overload (same type via WINTLS_USE_STANDALONE_ASIO), so TLS errors propagate correctly. 3. websocketpp_client_base::impl<T>(): add debug-only type tag (typeid hash) set at construction and asserted on access, catching config-type dispatch mismatches at zero release cost.
# Conflicts: # Include/httpClient/pal.h
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds first-class WebSocket compression support to
libHttpClientand includes the build, transport, API, and validation changes needed to use it safely on the important built-in paths.At a high level, this PR:
HCWebSocketSetOptions(...)andHCWebSocketOptionsas the primary public control surface for legacy preservation, deterministic behavior selection, and compression negotiation on the built-in implementations that expose that surfaceHCWebSocketCloseStatus::TooLargesocket close, effectively deprecating the fragment-handling APIs on the built-in implementations that expose the new options surface and enforcing consistent semantics across platformsWhy this is needed
The motivating problem is straightforward:
permessage-deflateThis PR closes that gap by adding compression-capable built-in WebSocket support across Linux, macOS, iOS, and the Windows-family paths that need it, while keeping the caller-facing configuration surface backend-agnostic and the deterministic receive contract consistent.
Behavior model
Public options surface
HCWebSocketSetOptions(...)is the primary control surface on the built-in implementations that expose deterministic behavior and compression negotiationHCWebSocketOptions::LegacySemanticsexplicitly preserves the existing legacy behavior by retaining the existing provider behavior and semanticsHCWebSocketOptions::Noneselects deterministic behavior without requesting compressionHCWebSocketOptions::RequestCompressionselects deterministic behavior and requestspermessage-deflateCompressionServerNoContextTakeoverandCompressionClientNoContextTakeoverrefine the compression request and requireRequestCompressionLegacy behavior
Existing behavior, aka legacy behavior, on Win32 / GDK differs from other platforms:
Other platforms:
Legacy versus deterministic behavior
HCWebSocketSetOptions(...)is never called, Win32 and GDK stay on the legacy WinHTTP-facing behaviorLegacySemanticsis an explicit no-op that preserves that same behaviorHCWebSocketSetMaxReceiveBufferSize(...)as a hard inbound message-size cap; when a message exceeds that cap, the socket is closed withHCWebSocketCloseStatus::TooLarge(1009)32,000,000byte limitHCWebSocketOptions::Nonekeeps the existing WinHTTP transport while enforcing that same deterministic hard-cap behavior; compression requests route through the hybrid provider to the compression-capable pathHCWebSocketSetOptions(...)orHCWebSocketSetMaxReceiveBufferSize(...), as the OkHttp provider automatically negotiates compression and has a fixed ~32MB message limitMessageWebSocketpath and is out of scope for this compression work: this PR does not add compression negotiation there, and the deterministicHCWebSocketSetOptions(...)/ hard-cap receive-limit behavior does not apply to that providerWhat changed
Build and dependency groundwork
asio,websocketpp, andboost-wintlsasioforward to the newest revision that still preserves the public APIsboost-wintlsdepends on, instead of taking the latest upstream breaking API removals in the same changelibzstdautodetection in the Linux vendored cURL helper so static CI-style builds do not pick up an undeclared zstd link dependencyioapi.cand thecontrib/minizipinclude path-Wdocumentationsuppression around the vendored websocketpp permessage-deflate include so strict Xcode warning-as-error builds do not fail on a third-party documentation comment defectWindows hybrid routing and receive-limit unification
WinHttpHybrid_WebSocketProviderso Windows-family backend selection stays inside a concrete provider instead of leaking into global selector stateTooLargeclose behavior as the other built-in deterministic pathsNetworkStateplumbing so the new transport integrates with the existing lifecycle and shutdown pathsExplicit WebSocket options semantics
HCWebSocketSetOptions(...), withHCWebSocketOptionsproviding the control knobsLegacySemanticsand tightened deterministic behavior across the supported built-in pathsSamples and validation coverage
LegacySemantics, fragment-handler rejection, and explicit max-receive-buffer behaviorCurrent websocketpp scale posture
m_websocketThread = std::thread(...)andstd::thread m_websocketThread)Review feedback fixes
boost-wintlstoNOTICE.txtandcgmanifest.json; corrected the staleasiocommit hash in both filesDEFINE_ENUM_FLAG_OPERATORS(HCWebSocketOptions)so callers can combine flags with|without casting throughuint32_tHCWebSocketOptionsflag-handling sites in the runtime and tests now that the enum flag operators exist; removed leftover test-only helper functions and replaced the remaining raw legacy-options equality checkm_stateMutexlock toSetOptions()andSetPingInterval()for consistency with the locking pattern inSetMaxReceiveBufferSize()and the other pre-connect settersGetResponseHeaderAtIndex()to returnE_INVALIDARGon out-of-range index, consistent with the existingHCHttpCallResponseGetHeaderAtIndex()contract.vcxitems.filtersentries for the new websocketpp and hybrid-routing filesraw newinwintls_socket.hpp, the detached-joiner shutdown fallback, and the duplicated proxy URI parserhttpClient.hand README wording aroundLegacySemantics, deterministic behavior, and receive-limit behavior; aligned the related WinHTTP wording/flow to matchHC_NOWEBSOCKETSto a first-class build option so the post-rebase no-websockets variants stay buildableWinHttpHybrid_WebSocketProviderprovider-selection helpersconstin response to late review feedbacktranslate_ec(...)compile issue so the branch still builds cleanly under stricter MSVC conformance modes used by downstream consumers/OPT:NOREFintent for the Win32 compression test executables into the linker settings and paired it with plain/LTCGso the setting actually takes effectHCWebSocketSetMaxReceiveBufferSize(0)withE_INVALIDARGto avoid invalid deterministic receive-buffer statescurl_multi_pollwarning while preserving the GDK dynamic-loader fallback behaviorValidation performed
Local Windows build matrix
libHttpClient.vs2022.slnforDebug|x64libHttpClient.vs2022.slnforRelease|x64libHttpClient.vs2022.slnforDebug|x86libHttpClient.vs2022.slnforRelease|x86libHttpClient.vs2022.slnforDebug|ARM64Release|ARM64is blocked locally by missing Spectre-mitigated libraries (MSB8040)Debug|Gaming.Desktop.x64andRelease|Gaming.Desktop.x64are blocked locally by missing Microsoft GDK integration and missing localv143GDK toolset support (MSB8020)Local Windows test runs
x64 DebugTE suite viavstest.console.exeTaskQueueTests, as expectedAsyncBlockTests::VerifyFailureInDoWorkfailed once in the full TE run but passed immediately when rerun in isolationTests\WebSocketCompression\WebSocketCompressionTests.Win32.vcxprojpermessage-deflatenegotiation, and the no-context-takeover combinations successfully against the local echo serverTests\WebSocketCompression\WebSocketCompressionIntegrationTests.Win32.vcxprojCERT_E_CN_NO_MATCH, untrusted-rootCERT_E_UNTRUSTEDROOT, and refused-connect diagnostics successfullyLinux and Android probe results
libHttpClient.Linux.soandWebSocketCompressionTests.LinuxWebSocketCompressionTests.Linuxdirectly under WSL and validated both the deterministic non-compression path (HCWebSocketOptions::None, including theTooLargeoversize close) and the negotiated compression path (RequestCompressionplus the no-context-takeover variants) successfully against the local echo serverassembleDebugcould not startPost-rebase Windows and Linux variant checks
Build\libHttpClient.Win32\libHttpClient.Win32.vcxprojforRelease|x64Out\x64\Release\WebSocketCompressionTests.Win32\WebSocketCompressionTests.Win32.exesuccessfullyBuild\libHttpClient.Win32\libHttpClient.Win32.vcxprojforRelease|x64with/p:HCEnableWebSocketCompression=falseBuild\libHttpClient.Win32\libHttpClient.Win32.vcxprojforRelease|x64with/p:HCNoWebSockets=trueTests\WebSocketCompression\WebSocketCompressionTests.Win32.vcxprojforRelease|x64with the Release/OPT:NOREF+/LTCGfollow-up appliedTests\WebSocketCompression\WebSocketCompressionIntegrationTests.Win32.vcxprojforRelease|x64with the same Release linker follow-up appliedSource\WebSocket\Websocketpp\websocketpp_websocket.cppunder/permissive-against the branch include/define set to confirm the WinTLS transport also builds under stricter MSVC conformance settingsBuild\libHttpClient.Linux\libHttpClient_Linux.bash --install-dependencies -c Releasesuccessfully inside anubuntu:22.04containerctest --output-on-failure -R websocket-compression-linuxsuccessfully under WSLCurlMulti.cppcleanup;WebSocketCompressionTests.Linuxstill linked successfully and the non-GDK warning disappearedHC_ENABLE_WEBSOCKET_COMPRESSION=OFFHC_NOWEBSOCKETS=1Not validated locally
-Wdocumentationsuppression in the configured permessage-deflate wrapper, but that rerun has not been observed locallyScope boundary for this PR
websocketppitselfMessageWebSocketprovider behaviorRisks / review focus
Internal review should focus especially on:
WinHttpHybrid_WebSocketProvideris the right long-term ownership point for Windows-family mixed-backend behaviorHCWebSocketSetOptions(...)is the right caller contract for expressing both legacy preservation and deterministic behavior selectionSuggested review path
7a3ef9b—Add WebSocket compression build and dependency plumbingDependency updates, shared-project wiring, Linux and Apple compression build plumbing, and solution/package metadata changes.
b739781—Introduce the websocketpp-backed Windows transportThe websocketpp provider, WinTLS transport adapter, WinHTTP hybrid routing, and the platform-specific wiring for Win32 and GDK.
5698ddb—Replace compression flags with explicit WebSocket optionsThe
HCWebSocketSetOptions(...)surface,LegacySemantics, deterministic behavior, Android surface cleanup, and the export/documentation updates.818751b—Add WebSocket samples and compression validation coverageThe echo-server sample, compression and certificate integration coverage, and the unit-test updates that lock down the final behavior.
a593c5b—Unify deterministic WebSocket behavior on WinHTTPThe WinHTTP deterministic receive-limit enforcement, hybrid routing adjustment for non-compression deterministic mode, deterministic oversize validation updates, and the final README/public documentation alignment.
0fa4a01—Update third-party compliance metadata (AI Review)Adds
boost-wintlstoNOTICE.txtandcgmanifest.json; corrects the staleasiocommit hash.0a76816—Fix WebSocket API issues (AI Review)Adds
DEFINE_ENUM_FLAG_OPERATORS, mutex consistency forSetOptions/SetPingInterval,E_INVALIDARGfor out-of-range header index.81436e8—Build hygiene and documentation comments (AI Review)Win32
.vcxitems.filtersentries, documentation comments for design choices and intentional duplication.d1e0f90—Simplify WebSocket option flag handling (AI Review)Low-risk cleanup of the remaining option-flag handling in the runtime and tests after the enum flag operators landed.
b1b8d97—Handle websocketpp close status consistentlyAligns websocketpp close-status handling with the rest of the deterministic behavior and removes a late clean-branch inconsistency in the websocketpp path.
c475339—Fix websocket async queue topologyKeeps the clean branch's websocket async queue usage aligned with the intended control flow.
10aed2d—Address PR feedback on WebSocket semantics docsClarifies the public
httpClient.hand README wording aroundLegacySemantics, deterministic behavior, and receive-limit expectations; includes the small aligned WinHTTP wording/flow cleanup.5ba0134—Fix websocket connect completion mergeRepairs a late merge regression in
hcwebsocket.cppso connect completion follows the intended post-rebase control flow.6b8a612—Fix no-websockets build variantsFixes the Win32 and GDK no-websockets export surfaces and the Linux
HC_NOWEBSOCKETSbuild plumbing so the post-rebase no-websockets variants remain buildable.9fbc9c1—Make hybrid websocket helpers constApplies the late review-feedback const-correctness cleanup to the non-mutating
WinHttpHybrid_WebSocketProviderprovider-selection helpers.cdaebe6—Fix strict-mode WinTLS websocket buildFixes a latent
wintls_socket.hpptransport-error translation compile issue so the websocketpp-backed Windows compression path still builds under stricter MSVC conformance settings.c90949a—Harden Linux helper scripts for container buildsHardens the Linux dependency-install, OpenSSL, and cURL helper scripts for the containerized build path used in local and CI-style validation.
6cdd639—Silence non-GDK curl multi poll warningRemoves the always-true non-GDK
curl_multi_pollcheck while preserving the GDK dynamic-loader fallback behavior.cc5fa18—Align Apple minizip Xcode wiringAdds
ioapi.cand thecontrib/minizipinclude path to the Apple Xcode project so its minizip wiring matches the Linux and Android build definitions.5f8ec91—Fix Win32 compression test linker settingsMoves the Release
/OPT:NOREFcontrol for the Win32 compression test executables into Link and forces plain/LTCGso that setting is compatible.fba869d—Reject zero WebSocket receive buffer sizeRejects
HCWebSocketSetMaxReceiveBufferSize(0)withE_INVALIDARGto avoid invalid deterministic receive-buffer states.bae2321—Disable zstd in Linux curl helperDisables optional zstd autodetection in the Linux vendored cURL helper so static builds keep a deterministic link surface and do not require undeclared
libzstdsymbols.7dad3d9—Silence Apple websocketpp documentation warningLocalizes Apple Clang
-Wdocumentationsuppression to the configured permessage-deflate wrapper so strict iOS/macOS warning-as-error builds do not fail on the vendored websocketpp header.