-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ID-1570 - Fix dice-where and update dce-id to use the latest version #113
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #113 +/- ##
============================================
+ Coverage 58.75% 58.78% +0.03%
- Complexity 370 371 +1
============================================
Files 97 97
Lines 1913 1917 +4
Branches 167 167
============================================
+ Hits 1124 1127 +3
- Misses 730 731 +1
Partials 59 59 ☔ View full report in Codecov by Sentry. |
snackk
requested review from
gedl,
julianhowarth,
stykiaz,
styxtwo,
diogojbmoreira,
diegodouradop and
lcardito
August 20, 2024 14:50
snackk
changed the title
ID-1570 - Alternative way;
ID-1570 - Fix dice-where and update dce-id to use the latest version
Aug 20, 2024
diogojbmoreira
previously approved these changes
Aug 20, 2024
dice-where-downloader-lib/src/main/java/technology/dice/dicewhere/downloader/Download.java
Outdated
Show resolved
Hide resolved
diogojbmoreira
approved these changes
Aug 20, 2024
lcardito
reviewed
Aug 21, 2024
...er-lib/src/main/java/technology/dice/dicewhere/downloader/stream/StreamWithMD5Decorator.java
Show resolved
Hide resolved
brunocorreiaweb
approved these changes
Aug 26, 2024
gedl
approved these changes
Aug 26, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Alternative to -> #112
WHAT
Fix dice-where and update dce-id to use the newest dice-where version.
We call .md5() twice on the same StreamWithMD5Decorator object. This causes the second invocation to return the incorrect hash.
First call is newly added to the LocalFileAcceptor and S3FileAcceptor classes
Second call is already existing in S3Source.produce()
This causes dice-where to download a file successfully, but not return the correct DownloadExecutionResult when it’s finished processing. The error is triggered in the ID test DiceWhereDownloaderScheduleJobTest .
WHY
Recently dice-where was updated to add extra md5 checks prevent faulty downloads. We need to update id to use the newest version.
Sadly the above change also introduced a bug for services using dice-where as a library. This also needs to be fixed so that we can use the updated version in id.
AC
Dice-where is fixed so it no longer calls md5() twice on the same object.
dce-id testDiceWhereDownloaderScheduleJobTest passes with the updated version of dice-where
Updated dce-id can be deployed and successfully pulls in files from dce-shared.
Fix context
Override was necessary to keep the digest updated as the InputStream is being ingested in batches rather than in one sitting.