Milvus - Hardened Vector Database for AI and RAG

What is Milvus

Milvus is an open-source vector database built for similarity search over embeddings. It stores high-dimensional vectors alongside scalar metadata and answers nearest-neighbour queries across millions of items in milliseconds, which makes it the storage layer for retrieval-augmented generation (RAG), semantic search, recommendation, and image or audio similarity. It supports multiple index types and distance metrics (COSINE, L2, inner product), hybrid searches that combine vector similarity with scalar filtering, sparse and dense vectors, and full-text search. Clients connect over gRPC or a RESTful API, with official SDKs for Python, Java, Go, Node.js and C#. Milvus is a graduated project of the LF AI & Data Foundation, licensed under Apache 2.0.

Why self-host Milvus

Embeddings are derived from your source documents, and can often be inverted to recover something close to the original text. That makes a vector index as sensitive as the corpus behind it, and keeps it in scope for GDPR and internal data-residency rules. Running Milvus on your own VM keeps the vectors inside your subscription and network boundary, with no per-query egress and no third-party processor to assess. Costs stay predictable as the index grows, and the API is the upstream open-source one rather than a managed variant.

What this VM image adds

Security hardening:

• Authentication enabled from first boot - Milvus ships with authorization disabled and a documented default password; this image enables authorization and generates a unique root password before the database ever starts, so that default never exists on your instance
• Unique credentials per instance - the database, object-store and management-console credentials are all generated on your first boot into a root-only file
• Management console behind TLS and Basic Auth - the Milvus console has no authentication of its own, so it is published only through an nginx reverse proxy with its own credentials
• Internal ports bound to loopback - the metrics/management port and the object store are reachable only from the VM itself; the metadata store publishes no host port at all
• Container confinement preserved - the upstream deployment relaxes the seccomp profile; this image keeps the default profile
• CVE scan - every image is scanned for vulnerabilities with Trivy before release
• UFW firewall - only SSH, HTTPS and the database API port are open
• fail2ban - SSH brute-force protection
• AppArmor - mandatory access control

OS hardening (CIS Level 1):

• CIS Level 1 hardened - CIS Ubuntu 24.04 LTS Level 1 Benchmark via ansible-lockdown
• auditd - system call auditing for critical paths
• SSH hardening - PasswordAuthentication disabled, key-only access
• Kernel hardening - SYN cookies, ASLR, rp_filter, TCP BBR
• /tmp as tmpfs - nosuid, nodev, noexec
• Azure IMDS endpoints - egress rules pre-configured (169.254.169.254, 168.63.129.16)

Compliance artifacts (inside the VM):

• SBOM - CycloneDX 1.6 at /etc/lynxroute/sbom.json
• CIS Conformance Report - OpenSCAP HTML at /etc/lynxroute/cis-report.html
• Tailored CIS profile - /usr/share/doc/lynxroute/CIS_TAILORED_PROFILE.md
• Vendor-standard upgrade path - the Milvus version is pinned in /opt/milvus/.env, so you upgrade with the documented pull-and-restart flow

Quick Start

1. Deploy the VM from Azure Marketplace (Standard_D4s_v3 or larger recommended - Milvus holds its indexes in RAM and requires a CPU with AVX2)
2. SSH: ssh -i key.pem <username>@<PUBLIC_IP> (username set during VM creation, default: azureuser)
3. Check status: the login banner shows the Milvus version, service state and both endpoints
4. Read your generated credentials: sudo cat /root/milvus-credentials.txt
5. Connect an SDK to <PUBLIC_IP>:19530 with token root:<password>, or open https://<PUBLIC_IP>/ for the management console. The credentials file includes a copy-paste example that creates a collection, inserts vectors and runs a similarity search.

Restrict the network security group on ports 19530 and 443 to the addresses that need them before putting the instance into service. Note that in Milvus 3.x the default database is named "default" - examples written for 2.x that pass "_default" will fail.