HADOOP-19658. ABFS:Create and rename idempotency for FNS Blob #7914
+348
−6
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
anmolanmol1234
changed the title
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
| AbfsRestOperation statusOp = null; | ||
| try { | ||
| // Check if the file already exists by calling GetPathStatus | ||
| statusOp = getPathStatus(path, false, tracingContext, null); |
There was a problem hiding this comment.
In case of override true, flow might come here with already a Head call done on path.
Can we avoid this head call in that case?
| assumeHnsDisabled(); | ||
| // Create a spy of AzureBlobFileSystemStore | ||
| AzureBlobFileSystemStore store = Mockito.spy(fs.getAbfsStore()); | ||
| assumeBlobServiceType(); |
There was a problem hiding this comment.
We can move all assume before any other statement
| // Create a spy of AzureBlobFileSystem | ||
| try (AzureBlobFileSystem fs = Mockito.spy( | ||
| (AzureBlobFileSystem) FileSystem.newInstance(getRawConfiguration()))) { | ||
| assumeHnsDisabled(); |
There was a problem hiding this comment.
Same here, move all assume to first few lines
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change introduces idempotency for create operation in the ABFS driver when using FNS-Blob. The update strengthens the client’s retry semantics to ensure that transient backend failures (e.g., HTTP 500) or ambiguous outcomes (where the client receives an error, but the operation has already succeeded on the service) are handled safely. This PR also includes test cases to validate the idempotency of rename operations.