https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image8_empilog300.png
GPU Inference VM with vLLM
by Empirisch Tech GmbH
Just a moment, logging you in...
GPU-accelerated LLM inference appliance with vLLM on Ubuntu 24.04, default model ChaperoneAI-Model (84% on MedQA)
Empirischtech Inference VM serves two goals: production-style inference with a strong default model, and fast iteration on other Hugging Face models.
Default model — Chaperone-Thinking-LQ-1.0
Domain-optimized reasoning built on DeepSeek-R1-Distill-Qwen-32B (GPTQ-aware training + QLoRA). 84% MedQA — benchmark methodology and comparisons: chaperoneai.net/benchmark.
Download / model card: huggingface.co/empirischtech/DeepSeek-R1-Distill-Qwen-32B-gptq-4bit (CC-BY-4.0).
General-purpose testing
Swap to any vLLM-compatible model from Hugging Face for quick A/B tests and validation—same VM, same API, minimal ops overhead.
Platform: Ubuntu 24.04 LTS, NVIDIA GPU stack, vLLM, nginx HTTPS (self-signed TLS), API key auth, first-boot provisioning, CLI helpers (status, model swap workflow).
At a glance
https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image2_statusloaded.png
https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image5_inference1.png
https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image4_statusloading.png
https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image1_vllmcommands.png
https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image7_openhttpsport.png