Repository navigation
Clean up disabled connection pooler objects after operator restart - #3211
Open
sarankumar1325 wants to merge 1 commit into
Open
sarankumar1325 wants to merge 1 commit into
sarankumar1325 wants to merge 1 commit into
Conversation
When the connection pooler is disabled, cleanup only looked at the objects remembered in memory. After an operator restart those references are gone, so a Deployment, Service or auth Secret left over from a failed or interrupted cleanup was never removed. - On the first sync of a cluster with a disabled pooler, look up the pooler Deployment, Service and auth Secret by name (only objects carrying the operator's connection-pooler label) so they get deleted. - deleteConnectionPooler now tries all three objects even if one deletion fails, forgets only the ones that are gone, and returns the combined error, so the failed one is retried on the next sync. - The disabled-pooler sync path also cleans up when only the auth Secret is left. Fixes zalando#3208
sarankumar1325
requested review from
FxKu,
Jan-M,
hughcapet,
idanovinda,
jopadi and
mikkeloscar
as code owners
October 30, 2026 13:53
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem description
Fixes #3208.
When the connection pooler is disabled, cleanup only works on the objects the operator remembers in memory (
c.ConnectionPooler[role].Deployment/Service/AuthSecret). Two things went wrong:deleteConnectionPoolerreturned as soon as one deletion failed. If deleting the Service failed, the auth Secret was never attempted.Clusterobject starts with empty references.syncConnectionPoolerthen skips cleanup for a disabled pooler, because nothing is remembered. Leftover objects (the Service and auth Secret from the issue, or the whole pooler if it was disabled while the operator was down) stayed forever, and sync reported success.Changes:
syncConnectionPoolerfirst initialises the pooler objects of a role (first sync of the cluster, e.g. after a restart) and the pooler is not needed, the newdiscoverConnectionPoolerObjectslooks up the Deployment, Service and<name>-uauth Secret by name. It only remembers objects that carry the operator'sconnection-pooler: <name>label, so unrelated objects with the same name are left alone. This is a one-time lookup per cluster object, not on every sync.deleteConnectionPoolernow tries all three objects, forgets only the ones that are really deleted (or already gone), and returns the combined error. A failed object stays remembered and is retried on the next sync.Linked issues
Fixes #3208
Checklist
acid.zalan.doapi package. (n/a, no API changes)resources.goservice deletion and doesn't overlap)New unit tests in
pkg/cluster/connection_pooler_test.go:TestConnectionPoolerCleanupAfterRestart: creates the pooler, then syncs a freshClusterwith the pooler disabled (same old and new spec, as on startup) and checks that the Deployment, Service and auth Secret are gone for both roles. It fails on master.TestConnectionPoolerDeleteContinuesOnError: a fake client reactor fails the Service delete once. The auth Secret is still deleted, the Service stays remembered, and a second call removes it. It fails on master.TestConnectionPoolerCleanupKeepsForeignObjects: a Service with the pooler name but no pooler label is not deleted.go test ./...(aftergo generate ./...for mocks),go vet ./pkg/clusterandstaticcheck ./pkg/clusterall pass. I didn't run the e2e suite.I wrote this change with the help of an AI coding assistant, and I reviewed and tested it.