Skip to main content
Don’t miss what’s next at Microsoft Ignite
Don’t miss what’s next at Microsoft IgniteGet direct access to the people, products, and expertise behind Microsoft technology, with opportunities to ask questions, build skills, and learn alongside the community.
Register now
Microsoft
separator
https://catalogartifact.azureedge.net/publicartifacts/bcloudllc1671615348068.litellmproxy122-94b1fbd5-4141-4627-ae99-6fdc246e6920/image3_bCloud216x216.png

LiteLLMProxy

by bCloud LLC

Version 1.104.0 + Free Support on Ubuntu 26.04

LiteLLM Proxy is a unified AI gateway and proxy server that allows developers and organizations to connect, manage, route, and monitor requests across multiple Large Language Model (LLM) providers through a single OpenAI-compatible API. It simplifies integration with different AI models while providing centralized control over authentication, routing, usage, and API access.

Version: 1.104.0

The solution supports common AI gateway workflows including multi-provider model routing, API key management, load balancing, request monitoring, authentication, and OpenAI-compatible API access. It is suitable for AI applications, development environments, enterprise AI integrations, and centralized LLM management.

Features of LiteLLM Proxy:

  • Unified OpenAI-compatible API for multiple LLM providers.
  • Supports routing requests across multiple AI models and providers.
  • Centralized API key and authentication management.
  • Provides load balancing and intelligent model routing capabilities.
  • Supports usage tracking, logging, monitoring, and cost management.
  • Provides health and readiness endpoints for service monitoring.
  • Supports integration with AI applications, agents, and development tools.
  • Runs as a centralized LiteLLM Proxy service for managing LLM traffic.

Usage instructions for LiteLLM Proxy
$ sudo su
$ systemctl status litellm
$ systemctl start litellm
$ systemctl enable litellm

LiteLLM configuration file: /opt/litellm/config.yaml

Credentials are securely saved in: /etc/litellm.env

Check LiteLLM Proxy version:
$ /opt/litellm/venv/bin/litellm --version

Check LiteLLM Proxy service:
$ systemctl status litellm

Check LiteLLM Proxy port:
$ ss -lntp | grep 4000

Test service health:
$ curl http://127.0.0.1:4000/health/liveliness

Access LiteLLM Proxy:
Open your browser or API client and navigate to:
http://your-server-ip:4000

Note: LiteLLM Proxy uses TCP port 4000 by default. Ensure that the required Azure Network Security Group (NSG) rule is configured if remote access to the service is required. Restrict network access to trusted IP addresses where possible.

Disclaimer: LiteLLM Proxy is provided “as is” under applicable open-source licenses. Users are responsible for securely configuring API credentials, model providers, network access, authentication, and data handling. Users should review their respective AI provider policies, security requirements, and usage costs before deploying the solution in production environments.

English (United States)
Your Privacy Choices Opt-Out Icon Your Privacy Choices
Consumer Health Privacy Sitemap Contact Us Privacy & Cookies Terms of Use About our ads Manage cookies