Skip to content

fix(inkless): Stop replicating after migration to diskless is completed#585

Merged
giuseppelillo merged 1 commit intomainfrom
giuseppelillo/fix-follower-replication-after-migration
Apr 28, 2026
Merged

fix(inkless): Stop replicating after migration to diskless is completed#585
giuseppelillo merged 1 commit intomainfrom
giuseppelillo/fix-follower-replication-after-migration

Conversation

@giuseppelillo
Copy link
Copy Markdown
Contributor

@giuseppelillo giuseppelillo commented Apr 28, 2026

Diskless offsets should never be replicated to followers. Avoid this in two ways:

  1. Do not respond with diskless offsets on fetch requests coming from followers
  2. In case of diskless migration from classic, stop ReplicaFetcher as soon as they have caught up to classicToDisklessStartOffset.

@giuseppelillo giuseppelillo force-pushed the giuseppelillo/fix-follower-replication-after-migration branch 2 times, most recently from 278d971 to a4fc758 Compare April 28, 2026 10:06
@giuseppelillo giuseppelillo marked this pull request as ready for review April 28, 2026 10:28
@giuseppelillo giuseppelillo requested a review from Copilot April 28, 2026 10:52
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts inkless/diskless replication behavior so that followers stop replicating once classic→diskless migration is complete, and followers never receive diskless records past the migration seal offset.

Changes:

  • Return an empty fetch response (with HW clamped to classicToDisklessStartOffset) for follower fetches at/above the seal offset to prevent diskless replication into follower local logs.
  • Evict partitions from ReplicaFetcherThread once the follower’s local LEO has caught up to classicToDisklessStartOffset.
  • Add unit tests covering follower fetch behavior at/above/below classicToDisklessStartOffset.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
core/src/main/scala/kafka/server/ReplicaManager.scala Adds follower-specific handling to prevent returning diskless records after migration seal offset.
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala Tracks and evicts partitions from fetchers once classic→diskless migration catch-up is complete.
core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala Adds tests ensuring follower fetches at/above the seal offset return empty responses and do not touch diskless/local log paths.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread core/src/main/scala/kafka/server/ReplicaFetcherThread.scala
Comment thread core/src/main/scala/kafka/server/ReplicaManager.scala
Comment thread core/src/main/scala/kafka/server/ReplicaFetcherThread.scala Outdated
@giuseppelillo giuseppelillo force-pushed the giuseppelillo/fix-follower-replication-after-migration branch from a4fc758 to aa7a144 Compare April 28, 2026 13:25
Stop ReplicaFetcher once diskless migration is completed.
Fetch responses for followers never contain diskless offsets.
@giuseppelillo giuseppelillo force-pushed the giuseppelillo/fix-follower-replication-after-migration branch from aa7a144 to 2c6181c Compare April 28, 2026 13:30
@giuseppelillo giuseppelillo merged commit 2075b8c into main Apr 28, 2026
5 checks passed
@giuseppelillo giuseppelillo deleted the giuseppelillo/fix-follower-replication-after-migration branch April 28, 2026 14:22
giuseppelillo added a commit that referenced this pull request May 6, 2026
…ed (#585)

Stop ReplicaFetcher once diskless migration is completed.
Fetch responses for followers never contain diskless offsets.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants