Skip to content

feat: opt-in HTTP keep-alive via keep_alive_connections#55

Open
erimicel wants to merge 2 commits intotypesense:masterfrom
OLIOEX:keepalive-connections
Open

feat: opt-in HTTP keep-alive via keep_alive_connections#55
erimicel wants to merge 2 commits intotypesense:masterfrom
OLIOEX:keepalive-connections

Conversation

@erimicel
Copy link
Copy Markdown
Contributor

@erimicel erimicel commented Apr 27, 2026

Motivation

Typesense::ApiCall#perform_request (lib/typesense/api_call.rb:72) constructs a fresh Faraday.new(...) on every call, which in turn builds a new Net::HTTP instance per request. For HTTPS clusters this means the TCP connection and TLS handshake are repeated on every search/index call, even when the same client instance is being reused as a process-wide singleton.

We profiled this on a hot search endpoint and the OpenSSL::X509::Store#set_default_paths plus the TLS handshake consistently dominated the request, despite the client itself being a singleton. The fix at the application layer is impossible while the gem rebuilds the Faraday connection on every call.

What

Adds an opt-in keep_alive_connections config option (default false — no behaviour change for existing users).

Typesense::Client.new(
  api_key: ENV['TYPESENSE_API_KEY'],
  nodes: [{ host: 'localhost', port: 8108, protocol: 'https' }],
  num_retries: 1,
  keep_alive_connections: true,
  keep_alive_idle_timeout_seconds: 30 # optional, default 30
)

When enabled:

  • Faraday connections are cached per (thread, node) (Net::HTTP is not thread-safe), keyed by protocol://host:port. Existing node round-robin and nearest_node logic are unchanged.
  • Uses the :net_http_persistent adapter; keep_alive_idle_timeout_seconds (default 30s) controls how long an idle socket is reused before being dropped — set this at or below your load balancer's idle timeout.
  • On any rescued network error the cached connection is discarded before retry, so a half-closed socket can't fail the retry too. Pair with num_retries >= 1 for transparent recovery.
  • Cache is keyed by ApiCall#object_id, so multiple Typesense::Client instances don't share sockets.

Default-off safety

require 'faraday/net_http_persistent' is gated on the flag, so the new dependency isn't loaded when keep-alive is off. The legacy per-request Faraday.new codepath is preserved verbatim. Rescue/retry list is unchanged.

Tests

spec/typesense/api_call_spec.rb (+10 examples, all green): connection reuse, per-node keying, per-thread isolation, per-instance isolation, eviction on network error, timeout propagation, custom + default keep_alive_idle_timeout_seconds, default-off path.

PR Checklist

Currently `Typesense::ApiCall#perform_request` builds a fresh
`Faraday.new(...)` (and therefore a new TCP and TLS handshake) on every
request. On hot endpoints this can dominate the Typesense round-trip
latency.

This adds an opt-in `keep_alive_connections` configuration option (default
`false`, so existing users see no behaviour change). When enabled:

* Faraday connections are cached per `(thread, node)` rather than
  constructed per request. Net::HTTP is not thread-safe, so per-thread
  caching keeps concurrent callers isolated while still respecting the
  existing node round-robin.
* Connections use the `:net_http_persistent` Faraday adapter with a 30s
  idle timeout, so reused sockets are dropped before most load balancers
  cull them.
* On any rescued network error, the cached connection is dropped before
  the gem retries, so a half-closed keep-alive socket cannot fail the
  retry as well. Pair with `num_retries >= 1` for transparent recovery
  from server- or load-balancer-side idle timeouts.

The `:net_http_persistent` adapter and its `net-http-persistent` runtime
dependency are listed in the gemspec, and `require 'faraday/net_http_persistent'`
is gated on the option being enabled, so loading the gem with the option
off does not import the new dependency at runtime.

New RSpec coverage:

* connection reuse on the same thread
* per-node cache keying
* per-thread cache isolation
* per-instance cache isolation
* eviction on network error
* timeouts propagate to the cached connection
* the option defaults to false and the legacy per-request connection
  path is preserved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant