Apache Druid - Hardened Real-Time OLAP Database

What is Apache Druid

Apache Druid is a high-performance real-time OLAP database built for sub-second analytical queries over streaming and historical data. It stores data in a columnar format with time-based partitioning, bitmap indexes and automatic rollup, answering aggregations, group-bys, top-Ns and time-series queries across billions of rows interactively. Druid runs as a set of cooperating Java services - query broker, coordinator, overlord, historical and middle manager - coordinated through Apache ZooKeeper, and offers a native JSON query API, full SQL, and a browser console for data loading, query authoring and cluster monitoring. Typical workloads are clickstream and product analytics, telemetry, operational dashboards and real-time fraud detection. Data is ingested in batch from files or object storage, or continuously from Apache Kafka.

Why self-host Apache Druid

Analytical data is often the most sensitive data an organisation holds: user behaviour, revenue, infrastructure telemetry. Running Druid on your own virtual machine keeps every event and every derived aggregate inside your own subscription and region, making data residency and GDPR obligations straightforward to evidence. There is no per-seat or per-query licence and no ingest quota, and Apache-2.0 licensing keeps the whole stack auditable and portable.

What this VM image adds

Druid runs here as a complete single-server cluster: the metadata store (embedded Apache Derby), deep storage (local disk) and coordination (bundled Apache ZooKeeper) all run on the VM, so the product works the moment it boots. Deployments that outgrow it can move deep storage to Azure Blob Storage and the metadata store to PostgreSQL or MySQL - those extensions ship in the image.

Security hardening:

• Console password generated at first boot - unique per VM, written to a root-only credentials file; nothing baked into the image
• Every service bound to localhost - broker, coordinator, overlord, historical, middle manager, the ingestion task workers, the Apache ZooKeeper client port and the metadata store are all unreachable from the network
• TLS reverse proxy as the single entry point - nginx terminates TLS on 443 and gates the console, REST API and SQL endpoint behind authentication; Druid has no built-in authentication of its own
• JavaScript query extensions disabled - untrusted query input cannot execute code inside the service JVMs
• Automatic JVM memory sizing - all six services are sized from VM memory with headroom reserved for the operating system
• CVE scan - every image is scanned for vulnerabilities before release
• UFW firewall - only 22, 80 and 443 open; every Druid, coordination and metadata port blocked externally
• fail2ban - SSH brute-force protection
• AppArmor - mandatory access control

OS hardening (CIS Level 1):

• CIS Level 1 hardened - CIS Ubuntu 24.04 LTS Level 1 Benchmark via ansible-lockdown
• auditd - system call auditing for critical paths
• SSH hardening - PasswordAuthentication disabled, key-only access
• Kernel hardening - SYN cookies, ASLR, rp_filter, TCP BBR
• /tmp as tmpfs - nosuid, nodev, noexec

Compliance artifacts (inside the VM):

• SBOM - CycloneDX 1.6 at /etc/lynxroute/sbom.json
• CIS Conformance Report - OpenSCAP HTML at /etc/lynxroute/cis-report.html
• Tailored CIS profile - /usr/share/doc/lynxroute/CIS_TAILORED_PROFILE.md

Quick Start

1. Deploy VM from Azure Marketplace (Standard_D4s_v3 recommended, Standard_D2s_v3 minimum)
2. SSH: ssh -i key.pem <username>@<PUBLIC_IP> (username set during VM creation, default: azureuser)
3. Read the console password: sudo cat /root/druid-credentials.txt
4. Open https://<PUBLIC_IP>/, accept the self-signed certificate, sign in as 'admin'. The cluster takes 2-4 minutes to start on first launch; until then the page shows a self-refreshing loading screen.
5. Load the bundled sample dataset and query it - the exact commands are in the credentials file
6. Replace the self-signed certificate before production: sudo certbot --nginx -d yourdomain.com

The console and its APIs have no authentication of their own - the TLS reverse proxy on 443 is the only access control, so keep 443 restricted to your own IP range. Data persists under /var/lib/druid across reboots and VM stop/start.