[#2649] feat(spark): Introduce timeout mechanism when getting the decompressing data

[zuston]

What changes were proposed in this pull request?

This PR is to introduce the timeout mechanism when getting the overlapping decompression data.

Why are the changes needed?

If not having this PR, the blocking wait have the potential risk to forever hang of the tasks when the rpc hang

Does this PR introduce any user-facing change?

rss.client.read.overlappingDecompressionFetchSecondsThreshold=-1, this mechanism will be disabled by default.

How was this patch tested?

Internal job tests

[github-actions]

Test Results

 3 150 files  ±0   3 150 suites  ±0   6h 50m 42s ⏱️ +30s
 1 210 tests ±0   1 209 ✅ ±0   1 💤 ±0  0 ❌ ±0 
15 314 runs  ±0  15 299 ✅ ±0  15 💤 ±0  0 ❌ ±0 

Results for commit aed7490. ± Comparison against base commit d9815c0.

♻️ This comment has been updated with latest results.

[jerqi]

Is it ok if the data is big?

[zuston]

Is it ok if the data is big?

Normally it wont happen that the block size is controlled by the writer that the block will be flushed when reaching the threshold.

[jerqi]

Is it ok if the data is big?

Normally it wont happen that the block size is controlled by the writer that the block will be flushed when reaching the threshold.

One record may be large. For example, I ever saw 100MB record.

[zuston]

Is it ok if the data is big?

Normally it wont happen that the block size is controlled by the writer that the block will be flushed when reaching the threshold.

One record may be large. For example, I ever saw 100MB record.

Maybe the default threshold value could be more larger. I hope this PR could make task fail as fast as possible rather than hang that will be terrible

[jerqi]

Is it ok if the data is big?

Normally it wont happen that the block size is controlled by the writer that the block will be flushed when reaching the threshold.

One record may be large. For example, I ever saw 100MB record.

Maybe the default threshold value could be more larger. I hope this PR could make task fail as fast as possible rather than hang that will be terrible

Just give u some input. Should the timeout consider the data length? If u think it is ok to keep current state, I'm ok, too. Because this is the corner cases.