Skip to main content
Don’t miss what’s next at Microsoft Ignite
Don’t miss what’s next at Microsoft IgniteGet direct access to the people, products, and expertise behind Microsoft technology, with opportunities to ask questions, build skills, and learn alongside the community.
Register now
Microsoft
separator
https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image8_empilog300.png

GPU Inference VM with vLLM

by Empirisch Tech GmbH

GPU-accelerated LLM inference appliance with vLLM on Ubuntu 24.04, default model ChaperoneAI-Model (84% on MedQA)

Empirischtech Inference VM serves two goals: production-style inference with a strong default model, and fast iteration on other Hugging Face models. Default model — Chaperone-Thinking-LQ-1.0 Domain-optimized reasoning built on DeepSeek-R1-Distill-Qwen-32B (GPTQ-aware training + QLoRA). 84% MedQA — benchmark methodology and comparisons: chaperoneai.net/benchmark. Download / model card: huggingface.co/empirischtech/DeepSeek-R1-Distill-Qwen-32B-gptq-4bit (CC-BY-4.0). General-purpose testing Swap to any vLLM-compatible model from Hugging Face for quick A/B tests and validation—same VM, same API, minimal ops overhead. Platform: Ubuntu 24.04 LTS, NVIDIA GPU stack, vLLM, nginx HTTPS (self-signed TLS), API key auth, first-boot provisioning, CLI helpers (status, model swap workflow).

At a glance

https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image2_statusloaded.png
https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image5_inference1.png
https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image4_statusloading.png
https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image1_vllmcommands.png
https://catalogartifact.azureedge.net/publicartifacts/numericalai.empirischtech-inference-9626012f-eeb0-4ec3-9f68-51ddcfbaee6b/image7_openhttpsport.png
English (United States)
Your Privacy Choices Opt-Out Icon Your Privacy Choices
Consumer Health Privacy Sitemap Contact Us Privacy & Cookies Terms of Use About our ads Manage cookies