Discussions

• 2 You must be logged in to vote

  🙏

  What is the best way to make DocVLM work with vLLM?

  Soum-Soum announced Mar 18, 2025 in Q&A · Closed

  

  

  2
• 0 You must be logged in to vote

  🙏

  Help serving vllm with kuberay

  torsteinelv announced Jan 11, 2025 in Q&A

  

  

  

  

  8
• 13 You must be logged in to vote

  🙏

  vLLM generating repeated/duplicate responses

  saraswatmks announced Jan 21, 2025 in Q&A

  

  

  

  

  8
• 4 You must be logged in to vote

  🙏

  Generation speed suddenly became very slow, what might be the cause?

  zhuwenfei-wintech announced Sep 27, 2024 in Q&A

  

  

  

  

  3
• 3 You must be logged in to vote

  🙏

  LLaVA with GGUF files (and --mmproj argument)?

  benmayersohn announced Sep 10, 2024 in Q&A

  

  

  1
• 5 You must be logged in to vote

  💬

  Structured Generation with Reasoning Parser in offline mode.

  psych0v0yager announced May 4, 2025 in General

  

  

  

  

  2
• 1 You must be logged in to vote

  💬

  vLLM cannot connect to existing Ray cluster

  as-bain announced May 1, 2025 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  Determining Overall Speed for One Long Prompt

  chigkim announced Apr 30, 2025 in Q&A

  

  

  1
• 2 You must be logged in to vote

  🙏

  Why temperature=0,top_p=1,seed=42 is still not enough to fix the llm's output!?

  beyondguo announced Apr 25, 2025 in Q&A

  

  

  1
• 2 You must be logged in to vote

  🙏

  How to achieve maximum performance?

  gaby announced Mar 20, 2025 in Q&A · Closed

  

  

  1
• 8 You must be logged in to vote

  📣

  GitHub discussion is not used anymore, please use the forum for discussion.

  youkaichao announced Mar 20, 2025 in Announcements

  

  

  

  

  5
• 1 You must be logged in to vote

  🙏

  working example of vllm using whisper?

  silvacarl2 announced Jan 15, 2025 in Q&A · Closed

  

  

  

  

  4
• 4 You must be logged in to vote

  🙏

  MultiModal model batched inference

  aswad546 announced Feb 24, 2025 in Q&A

  

  

  3
• 17 You must be logged in to vote

  🙏

  Is there a way to set max-concurrency

  tommasofavaron1 announced Jan 31, 2024 in Q&A

  

  

  

  

  2
• 3 You must be logged in to vote

  🙏

  How can I log user id from the API request

  vinayK34 announced Feb 6, 2025 in Q&A

  

  

  1
• 1 You must be logged in to vote

  🙏

  Can vllm serving clients by using multiple model instances?

  aoyulong announced Jun 21, 2023 in Q&A

  

  

  

  

  7
• 1 You must be logged in to vote

  🙏

  Difficulty understanding sequence length and context length

  NicolasDrapier announced May 6, 2024 in Q&A · Closed

  

  

  2
• 3 You must be logged in to vote

  🙏

  What is base for logprobs?

  wongjingping announced Sep 24, 2024 in Q&A · Closed

  

  

  2
• 1 You must be logged in to vote

  🙏

  ......lib/python3.12/site-packages/vllm/_C.abi3.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKSsb

  Rex818 announced May 1, 2025 in Q&A

  

  

  1
• 3 You must be logged in to vote

  🙏

  How do let vllm v1 to use a customized stat logger extended from StatLoggerBase?

  liuzijing2014 announced Mar 5, 2025 in Q&A

  

  

  

  

  2
• 3 You must be logged in to vote

  🙏

  RAG LLM Generating the entire Prompt along with Question in the generated response

  Arpx17 announced Feb 29, 2024 in Q&A

  

  

  

  

  2
• 25 You must be logged in to vote

  💡

  /v1/embeddings please

  feature request New feature or request

  yuhai-china announced Jun 21, 2023 in Ideas

  

  

  

  

  29
• 1 You must be logged in to vote

  🙏

  Why is the GPU KV cache usage very low?

  usage How to use vllm

  tammypi announced Jun 18, 2024 in Q&A

  

  

  

  

  3
• 1 You must be logged in to vote

  💬

  is paged attention an exact attention

  CharlesJu1 announced Feb 26, 2025 in General · Closed

  

  

  

  

  5
• 2 You must be logged in to vote

  🙏

  ~50% GPU utilization of openai API server

  imoneoi announced Jul 11, 2023 in Q&A

  

  

  

  

  7