Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enhance: [cherry-pick] Alter compactTo segments before compactFrom to avoid data loss if crash #34532

Merged
merged 1 commit into from
Jul 11, 2024
Merged

enhance: [cherry-pick] Alter compactTo segments before compactFrom to avoid data loss if crash #34532

merged 1 commit into from
Jul 11, 2024

Conversation

wayblink
Copy link
Collaborator

@wayblink wayblink commented Jul 9, 2024

issue: #34512
pr: #34513
@sre-ci-robot sre-ci-robot added the size/L Denotes a PR that changes 100-499 lines. label Jul 9, 2024
@mergify mergify bot added dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement labels Jul 9, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2024

@wayblink E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.
@wayblink
Copy link
Collaborator Author

wayblink commented Jul 9, 2024

/run-cpu-e2e
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2024

@wayblink E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.
@wayblink
Copy link
Collaborator Author

wayblink commented Jul 9, 2024

/run-cpu-e2e
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2024

@wayblink E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.
@wayblink
Copy link
Collaborator Author

wayblink commented Jul 9, 2024

/run-cpu-e2e
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2024

@wayblink E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.
@codecov Codecov
Copy link

codecov bot commented Jul 9, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 82.60870% with 12 lines in your changes missing coverage. Please review.

Project coverage is 80.74%. Comparing base (15adb2f) to head (0a0b49d).
Report is 10 commits behind head on 2.4.

❗ Current head 0a0b49d differs from pull request most recent head 081e91e

Please upload reports for the commit 081e91e to get more accurate results.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##              2.4   #34532   +/-   ##
=======================================
  Coverage   80.74%   80.74%           
=======================================
  Files        1051     1051           
  Lines      135835   135823   -12     
=======================================
- Hits       109678   109677    -1     
+ Misses      21945    21937    -8     
+ Partials     4212     4209    -3
Files Coverage Δ
internal/datacoord/compaction_trigger.go 87.14% <75.00%> (+0.43%) ⬆️
internal/datacoord/meta.go 85.29% <83.07%> (-0.68%) ⬇️

... and 24 files with indirect coverage changes
@wayblink
Copy link
Collaborator Author

wayblink commented Jul 9, 2024

/run-cpu-e2e
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2024

@wayblink E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.
@mergify mergify bot added the ci-passed label Jul 10, 2024
@mergify mergify bot added ci-passed and removed ci-passed labels Jul 10, 2024
@czs007 czs007 added this to the 2.4.6 milestone Jul 11, 2024
czs007
czs007 reviewed Jul 11, 2024
View reviewed changes
internal/datacoord/meta.go Outdated
}
segment := NewSegmentInfo(segmentInfo)
newSegments = append(newSegments, segment)
compactToSegInfos = append(compactToSegInfos, segment)
compactToSegIDs = append(compactFromSegIDs, segment.GetID())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compactToSegIDs = append(compactToSegIDs, segment.GetID())
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved
czs007
czs007 reviewed Jul 11, 2024
View reviewed changes
internal/datacoord/meta.go Outdated
return info.GetDmlPosition()
})),
StartPosition: getMinPosition(compactFromSegInfos),
DmlPosition: getMinPosition(compactFromSegInfos),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should use DmlPosition
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved
@wayblink
Alter compactTo segments before compactFrom to avoid data loss if crash
081e91e 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
@mergify mergify bot removed the ci-passed label Jul 11, 2024
@czs007
Copy link
Contributor

czs007 commented Jul 11, 2024

/approve
/lgtm
@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: czs007, wayblink

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
@mergify mergify bot added the ci-passed label Jul 11, 2024
@sre-ci-robot sre-ci-robot merged commit 86b57b7 into milvus-io:2.4 Jul 11, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved ci-passed dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement lgtm size/L Denotes a PR that changes 100-499 lines.
3 participants
@wayblink @czs007 @sre-ci-robot