Coding
Has anyone else found vLLM outputs noticeably worse than llama.cpp for the same model?
A user has reported discrepancies in output quality between vLLM and llama.cpp while testing the same model under similar settings and quantizations. Although vLLM demonstrates superior performance and concurrency, it exhibits issues such as formatting errors, context retention failures, and lower quality code outputs. This raises questions about the impact of quantization, configuration, and template parsing on inference quality, which is critical for practitioners optimizing model deployment and performance.
vllmllama.cppcomparison