KAFKA-17085: Handle tasks in state updater before tasks in task registry #16561
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…try (apache#16555) When a active tasks are revoked they land as suspended tasks in the task registry. If they are then reassigned, the tasks are resumed and put into restoration. On assignment, we first handle the tasks in the task registry and then the tasks in the state updater. That means that if a task is re-assigned after a revocation, we remove the suspended task from the task registry, resume it, add it to the state updater, and then remove it from the list of tasks to create. After that we iterate over the tasks in the state updater and remove from there the tasks that are not in the list of tasks to create. However, now the state updater contains the resumed tasks that we removed from the task registry before but are no more in the list of tasks to create. In other words, we remove the resumed tasks from the state updater and close them although we just got them assigned. This commit ensures that we first handle the tasks in the state updater and then the tasks in the task registry. Reviewer: Lucas Brutschy <lbrutschy@confluent.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a active tasks are revoked they land as suspended tasks in the task registry. If they are then reassigned, the tasks are resumed and put into restoration. On assignment, we first handle the tasks in the task registry and then the tasks in the state updater. That means that if a task is re-assigned after a revocation, we remove the suspended task from the task registry, resume it, add it to the state updater, and then remove it from the list of tasks to create. After that we iterate over the tasks in the state updater and remove from there the tasks that are not in the list of tasks to create. However, now the state updater contains the resumed tasks that we removed from the task registry before but are no more in the list of tasks to create. In other words, we remove the resumed tasks from the state updater and close them although we just got them assigned.
This commit ensures that we first handle the tasks in the state updater and then the tasks in the task registry.
Cherry-pick of 4ecbb75
Committer Checklist (excluded from commit message)