Skip to content

fix(rerank): use a lazy per-loop Cohere AsyncClient in CohereReranker - #827

Open
Yarmoluk wants to merge 1 commit into
redis:mainfrom
Yarmoluk:fix/cohere-reranker-async-client
Open

Yarmoluk wants to merge 1 commit into
redis:mainfrom
Yarmoluk:fix/cohere-reranker-async-client

Conversation

@Yarmoluk

@Yarmoluk Yarmoluk commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

CohereReranker built its AsyncClient once in __init__ and passed it the same kwargs as the sync client. Two bugs resulted:

  • "Event loop is closed" — the connection pool was bound to the first event loop that used it; a reranker that outlived that loop failed on every subsequent call.
  • "object Response can't be used in 'await' expression" — a caller-supplied httpx_client (sync) was forwarded to the AsyncClient, so arank got a sync Response back and crashed on await.

Fix

  • _get_aclient() builds the AsyncClient on first use and rebuilds it when the running loop changes (same pattern CohereTextVectorizer got in fix(vectorize): use Cohere's async client for aembed and aembed_many #785).
  • A caller-supplied async_httpx_client is kept and reused across loops.
  • A sync httpx_client without a matching async one disables the async path and falls back to the sync client with a warning.

Test plan

  • pytest tests/unit/test_cohere_reranker.py — 181-line new suite covering lazy init, loop-change rebuild, sync-client fallback, and warning behaviour.
  • No changes to vectorizer, router, or connection code.

🤖 Generated with Claude Code


Note

Medium Risk
Changes async HTTP client lifecycle and httpx kwargs handling for reranking; behavior is covered by new unit tests but affects production async rerank paths.

Overview
Fixes CohereReranker.arank failures from eagerly creating AsyncClient in __init__: connection pools tied to a closed event loop caused "Event loop is closed", and forwarding a sync httpx_client to the async client caused await errors on sync responses.

CohereReranker now lazily builds the async client via _get_aclient(), reusing it within one loop and rebuilding when the running loop changes (same approach as CohereTextVectorizer). Callers can pass async_httpx_client for a dedicated async HTTP client; if only a sync httpx_client is supplied, arank logs a warning and falls back to rank. httpx_client=None still enables the async path.

Adds tests/unit/test_cohere_reranker.py with fakes covering lazy init, loop reuse/rebuild, sync fallback, and async_httpx_client wiring.

Reviewed by Cursor Bugbot for commit 3e0646b. Bugbot is set up for automated code reviews on this repo. Configure here.

CohereReranker built its AsyncClient once in __init__ and passed it the
same kwargs as the sync client. Its connection pool was bound to the
first event loop that used it, so a reranker that outlived that loop
failed on the next one with "Event loop is closed", and a user-supplied
sync httpx_client reached the AsyncClient, so arank raised "object
Response can't be used in 'await' expression".

Build the async client on first use via _get_aclient(), rebuild it when
the running loop changes, and keep a user-supplied async_httpx_client
once built. A sync httpx_client without an async one disables the async
client and arank falls back to the sync client with a warning. This is
the same handling CohereTextVectorizer got in redis#785.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant