Skip to main content

Parallel Reconciliation

EntityQueueBackgroundService controls how many entities are reconciled simultaneously and what happens when two events for the same entity UID arrive at the same time. Both settings live on OperatorSettings.ParallelReconciliation and are configured through OperatorSettingsBuilder.ParallelReconciliation or via the WithParallelReconciliation fluent method.

Maximum Concurrency

MaxParallelReconciliations sets the upper bound on concurrently running reconciliations per entity type. The default is Environment.ProcessorCount * 2. When several controllers are registered for the same entity type, they share this single budget, so the limit is not multiplied by the controller count.

builder.Services
.AddKubernetesOperator(settings => settings
.WithParallelReconciliation(p => p.WithMaxParallelReconciliations(16)));

A SemaphoreSlim — one per entity type, shared by all of that entity's controllers — is acquired before an entry is dequeued, which means the queue acts as a natural buffer and back-pressure mechanism: producers (the watcher) can continue enqueuing while all reconciler slots are occupied.

Conflict Strategy

When a second reconciliation event arrives for an entity that is already being reconciled, the ConflictStrategy determines what happens:

StrategyBehaviourUse when
WaitForCompletion (default)Blocks until the running reconciliation finishes, then processes the new request immediatelyOrdering matters; no events should be dropped
DiscardDrops the incoming requestReconciler always reads fresh state from the API; duplicate events are safe to ignore
RequeueAfterDelayPlaces the incoming request back into the queue after RequeueDelay (default 5 s)Events must not be dropped, but immediate blocking is undesirable
builder.Services
.AddKubernetesOperator(settings => settings
.WithParallelReconciliation(p => p
.WithMaxParallelReconciliations(8)
.WithConflictStrategy(ParallelReconciliationConflictStrategy.RequeueAfterDelay)
.WithRequeueDelay(TimeSpan.FromSeconds(3))));
note

The per-UID lock is independent of the parallelism semaphore. Even with MaxParallelReconciliations = 1, two events for different entities are never serialised by the UID lock; only events for the same UID are affected by ConflictStrategy. Both the semaphore and the per-UID lock live in a coordinator shared by all controllers of an entity type, so the same object is never reconciled — or finalized — concurrently, even when two controllers match it.

Best Practices

  1. Leave defaults for most operatorsWaitForCompletion with ProcessorCount * 2 is a safe starting point
  2. Use Discard only when your reconciler always re-fetches entity state from the API server
  3. Use RequeueAfterDelay when events must not be dropped but you want to avoid blocking a worker slot
  4. Raise MaxParallelReconciliations if reconciliation is I/O-bound and CPU is not a constraint
  5. Lower MaxParallelReconciliations if the Kubernetes API server is rate-limiting your requests