Skip to content

Conversation

@CarltonHere
Copy link

The vllm-server may encounter a small probability 502 error when inferring certain PDFs with a large number of pages, resulting in abnormal interruption of inference. This PR adds a retry mechanism to increase stability during the inference process

Before modification

uv run mineru -p F:\demo.pdf -o .\output\ -b vlm-http-client -u http://***:30000
2025-10-20 15:39:31.089 | INFO     | mineru.backend.vlm.vlm_analyze:get_model:128 - get http-client predictor cost: 0.85s
Two Step Extraction:  75%|████████████████████████████████████████████▏              | 315/421 [06:06<02:03,  1.16s/it]

unhandled exception during asyncio.run() shutdown
task: <Task finished name='Task-4412' coro=<gather_tasks.<locals>.indexed() done, defined at F:\mineru\.venv\Lib\site-packages\mineru_vl_utils\vlm_client\utils.py:96> exception=ReadError('')>
Traceback (most recent call last):
...
  File "F:\mineru\.venv\Lib\site-packages\mineru_vl_utils\vlm_client\http_client.py", line 447, in aio_predict
    response_data = self.get_response_data(response)
                    │    │                 └ <Response [502 Bad Gateway]>
                    │    └ <function HttpVlmClient.get_response_data at 0x000001F8A465C4A0>
                    └ <mineru_vl_utils.vlm_client.http_client.HttpVlmClient object at 0x000001F8A454EF90>
  File "F:\mineru\.venv\Lib\site-packages\mineru_vl_utils\vlm_client\http_client.py", line 218, in get_response_data
    raise ServerError(
          └ <class 'mineru_vl_utils.vlm_client.base_client.ServerError'>
...
2025-10-20 15:45:50.003 | ERROR    | mineru.cli.client:parse_doc:201 - Unexpected status code: [502], response body: ***

After modification

uv run mineru -p F:\demo.pdf -o .\output\ -b vlm-http-client -u http://***:30000
2025-10-20 16:27:15.123 | INFO     | mineru.backend.vlm.vlm_analyze:get_model:128 - get http-client predictor cost: 0.67s
Two Step Extraction:  49%|██████████████████████████████████████████████████████▎                                                        | 206/421 [05:33<00:24,  8.63it/s]

#  If debug is enabled, error messages will be printed
Attempt 1/3 failed: Unexpected status code: [502], response body: ***

Two Step Extraction: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 421/421 [07:09<00:00,  1.02s/it]
2025-10-20 16:35:04.875 | INFO     | mineru.cli.common:_process_output:155 - local output dir is .\output\demo\vlm

Copilot AI review requested due to automatic review settings October 20, 2025 08:55
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a retry mechanism to the HTTP VLM client to handle intermittent 502 errors from the vllm-server during PDF inference, improving stability by automatically retrying failed requests up to 3 times before giving up.

  • Implements retry logic with configurable max retries (set to 3) for both synchronous and asynchronous predict methods
  • Adds error logging for failed attempts when debug mode is enabled
  • Prevents inference interruption from temporary server errors

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@myhloli
Copy link
Collaborator

myhloli commented Oct 30, 2025

Thank you for submitting your code. After carefully evaluating this request, we have implemented the request retry functionality using the httpx-retries library. Therefore, we are closing this PR. We look forward to your future contributions!

@myhloli myhloli closed this Oct 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants