Repository navigation
Conversation
…batches A batch with two rows for one entity and the same event timestamp kept the first row, whatever its created timestamp. The offline stores break such ties on the created timestamp, and the Redis online store keeps the last row it writes. The Milvus store now keeps the row with the latest event timestamp, then the latest created timestamp, then the last row of the batch, so a correction with a newer created timestamp is not lost. Signed-off-by: jiwidi <fhjaime96@gmail.com>
This was referenced Oct 28, 2026
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #7084 +/- ##
=======================================
Coverage 51.28% 51.28%
=======================================
Files 458 458
Lines 57409 57410 +1
Branches 8415 8415
=======================================
+ Hits 29443 29445 +2
Misses 25957 25957
+ Partials 2009 2008 -1
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Signed-off-by: jiwidi <fhjaime96@gmail.com>
Collaborator
|
@jiwidi please update the PR by rebasing the master branch. |
Contributor
Author
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
When a batch holds two rows for one entity, the Milvus store keeps one of them. It kept the row
with the latest event timestamp and, on a tie, the first row of the batch, whatever its created
timestamp. The offline stores break such a tie on the created timestamp, and the Redis online
store writes rows in order so the last one wins. A correction that carries the same event
timestamp and a newer created timestamp was lost.
This orders rows within one batch. The Redis store also skips a row whose event timestamp is older
than the stored one; in Milvus that needs a read before every write and is left for a later PR.
Which issue(s) this PR fixes:
N/A
Checks
git commit -s)Testing Strategy
Unit tests
Integration tests
Manual tests
Testing is not required for this change
Unit (Milvus Lite): the four cases of the table, with the rows in both orders; a batch with
two entities keeps both.
Server: two rows with the same event timestamp; the one with the newer created timestamp is
read back with
Strongconsistency.Unit tests run on Milvus Lite 3.2.1 (pymilvus 3.0.2). The server test in
sdk/python/tests/integration/online_store/test_milvus_remote.pypasses against a localMilvus 3.0.2 server. The existing Milvus unit and server tests pass unchanged.
Misc
Part of a stack that adds to the Milvus online store the update and delete functionality the Redis
online store has, following #6882, #6895, #6917, #6921 and #6940. The other PRs of the stack are
#7082 (row deletion) and #7083 (partition options).
mutation_timeoutand a schema mismatch check follow once these merge.