Modules:
| Name | Description |
|---|---|
api_server | |
cli_args | This file contains the command line arguments for the vLLM's |
orca_metrics | Utility functions that create ORCA endpoint load report response headers. |
protocol | |
run_batch | |
serving_chat | |
serving_classification | |
serving_completion | |
serving_embedding | |
serving_engine | |
serving_models | |
serving_pooling | |
serving_responses | |
serving_score | |
serving_tokenization | |
serving_tokens | |
serving_transcription | |
speech_to_text | |
tool_parsers | |
utils | |