INFO 06-28 19:41:58 async_llm_engine.py:529] Received request 07aba362773940d7aa2154854e6b86c9: prompt: '<|begin_of_text|><|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWrite a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>\n<|im_start|>assistant\n', sampling_params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.6, top_p=0.9, top_k=50, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[128003], include_stop_str_in_output=False, ignore_eos=False, max_tokens=512, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: None, lora_request: None. stdoutINFO 06-28 19:42:01 metrics.py:334] Avg prompt throughput: 8.5 tokens/s, Avg generation throughput: 14.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0% stdoutINFO 06-28 19:42:07 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0% stdoutINFO 06-28 19:42:12 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0% stdoutINFO 06-28 19:42:17 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.0%, CPU KV cache usage: 0.0% stdoutINFO 06-28 19:42:22 metrics.py:334] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.2%, CPU KV cache usage: 0.0% stdoutGeneration took 1719601455.61sFormatted prompt: <|begin_of_text|><|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>user Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|> <|im_start|>assistant INFO 06-28 19:42:24 async_llm_engine.py:120] Finished request 07aba362773940d7aa2154854e6b86c9. stdout