Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add java-storage implementation for createEmptyObjects implementaion. #1094

Merged
merged 2 commits into from
Jan 9, 2024
Merged

Add java-storage implementation for createEmptyObjects implementaion. #1094

merged 2 commits into from
Jan 9, 2024

Conversation

shilpi23pandey
Copy link
Contributor

No description provided.

@shilpi23pandey shilpi23pandey marked this pull request as draft December 27, 2023 07:33
@shilpi23pandey
Copy link
Contributor Author

/gcbrun
@shilpi23pandey shilpi23pandey force-pushed the move-to-java-storage-empty-objects branch from 6195ef8 to 1467538 Compare December 29, 2023 06:53
@shilpi23pandey
Copy link
Contributor Author

/gcbrun
@shilpi23pandey shilpi23pandey marked this pull request as ready for review December 29, 2023 07:17
arunkumarchacko
arunkumarchacko approved these changes Jan 2, 2024
View reviewed changes
gcsio/src/main/java/com/google/cloud/hadoop/gcsio/GoogleCloudStorageClientImpl.java Outdated
private boolean canIgnoreExceptionForEmptyObject(
StorageException exceptionOnCreate, StorageResourceId resourceId, CreateObjectOptions options)
throws IOException {
if (errorExtractor.getErrorType(exceptionOnCreate) == ErrorType.RESOURCE_EXHAUSTED
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: consider creating a local variable for errorExtractor.getErrorType(exceptionOnCreate). It will make the code a bit cleaner.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done
gcsio/src/main/java/com/google/cloud/hadoop/gcsio/GoogleCloudStorageClientImpl.java
* {@code options}, to be used to determine whether it's safe to ignore an exception that was
* thrown when trying to create the object, {@code exceptionOnCreate}.
*/
private boolean canIgnoreExceptionForEmptyObject(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are doing a lot of code duplication. Is there a way to avoid it?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe, we can avoid this duplication by refactoring the GoogleCloudStorageImpl at bit. However, given that we might deprecate GoogleCloudStorageImpl in future. I dont see merit in investing in this re-factoring at the moment. Let me know if you have any other thoughts.
arunkumarchacko
arunkumarchacko reviewed Jan 2, 2024
View reviewed changes
Copy link
Contributor

@arunkumarchacko arunkumarchacko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about tests? Code coverage is way too low.
@shilpi23pandey shilpi23pandey force-pushed the move-to-java-storage-empty-objects branch from 1467538 to fe6cbfe Compare January 4, 2024 09:34
@shilpi23pandey
Copy link
Contributor Author

/gcbrun
@shilpi23pandey shilpi23pandey force-pushed the move-to-java-storage-empty-objects branch 2 times, most recently from b0133b5 to ea2e8cd Compare January 4, 2024 13:55
@shilpi23pandey
Copy link
Contributor Author

/gcbrun
@shilpi23pandey shilpi23pandey force-pushed the move-to-java-storage-empty-objects branch from ea2e8cd to 07d6cfb Compare January 8, 2024 18:26
@shilpi23pandey shilpi23pandey force-pushed the move-to-java-storage-empty-objects branch from 07d6cfb to 73d7823 Compare January 8, 2024 18:29
@shilpi23pandey
Copy link
Contributor Author

/gcbrun
@shilpi23pandey
Copy link
Contributor Author

What about tests? Code coverage is way too low.
Some ITs relied on the common implementation so added that code in the GoogleCloudStorasgeClient. Also added unit tests for different cases.
@shilpi23pandey shilpi23pandey merged commit 4aafbba into GoogleCloudDataproc:master Jan 9, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants
@shilpi23pandey @arunkumarchacko