Apache Pulsar - Hardened Cloud-Native Messaging, Streaming

What is Apache Pulsar

Apache Pulsar is a cloud-native distributed messaging and event-streaming platform from the Apache Software Foundation. It combines publish-subscribe messaging with durable log storage, and supports multi-tenancy, per-topic subscriptions in four modes, and both queueing and streaming workloads on the same broker. This image runs Pulsar in standalone mode: the broker, the Apache BookKeeper storage layer and the metadata store all run inside a single JVM on OpenJDK 21 - one instance, no cluster to assemble. It ships only the Apache-2.0 distribution, with no proprietary add-ons.

Why self-host Apache Pulsar

Running Pulsar on a VM you control keeps every message - often sensitive customer, transactional or operational data - inside your own tenant rather than a managed streaming service. Self-hosting suits teams with data residency requirements, organisations under GDPR, HIPAA or ISO 27001, and any product where the event stream must stay within your perimeter with no per-message fees. Pulsar is Apache-2.0 and fully auditable.

What this VM image adds

Secure by default, not by configuration:

• Token authentication and authorization enabled - upstream ships both switched off. This image enables JWT auth, enforces authorization, and separates a superuser administrator role from the client role handed to applications
• Per-instance secrets only - the JWT signing key, both tokens and the broker TLS certificate are generated on first boot into /root/pulsar-credentials.txt (mode 0600); nothing sensitive is baked into the image
• TLS client listener on 6651 - applications connect over pulsar+ssl:// with a token; the plaintext broker port and the plaintext web port are reachable only on the instance itself
• nginx TLS perimeter on 443 for the admin REST API, with certbot pre-installed to swap in your own certificate
• Non-root service - Pulsar runs as a dedicated system user with UMask 0027, an explicitly pinned JVM heap, and its data on a persistent volume
• UFW firewall - TCP 443 and TCP 6651 open for buyer use, TCP 22 for SSH; all other inbound dropped; Azure IMDS and WireServer egress pre-configured
• fail2ban - SSH brute-force protection
• AppArmor - mandatory access control
• CVE scan - every image is scanned with Trivy before release

OS hardening (CIS Level 1):

• CIS Ubuntu 24.04 LTS Level 1 Benchmark via ansible-lockdown
• auditd for system call auditing of critical paths
• SSH hardening - PasswordAuthentication disabled, key-only access, PermitRootLogin no, LoginGraceTime 60
• Kernel hardening - SYN cookies, ASLR, rp_filter, kexec disabled, IPv6 off
• /tmp as tmpfs with nosuid, nodev, noexec

Compliance artifacts (inside the VM):

• SBOM - CycloneDX 1.6 at /etc/lynxroute/sbom.json with Pulsar pinned by version, PURL, Apache-2.0 license, supplier, and hash
• CIS Conformance Report at /etc/lynxroute/cis-report.html (OpenSCAP, Azure tailoring profile, 0 FAIL rules)
• Tailored CIS profile at /usr/share/doc/lynxroute/CIS_TAILORED_PROFILE.md
• Operator credentials file at /root/pulsar-credentials.txt (mode 0600) with the administrator and client tokens and the service URLs

Quick Start

1. Deploy VM from Azure Marketplace (Standard_D2s_v3 recommended)
2. Open NSG: TCP 443 (admin REST API) and TCP 6651 (Pulsar clients) from your trusted sources, TCP 22 from your management IPs only
3. SSH: ssh -i key.pem azureuser@<PUBLIC_IP>, then sudo cat /root/pulsar-credentials.txt for the tokens
4. Call the admin API: curl -sk -H "Authorization: Bearer $TOKEN" https://<PUBLIC_IP>/admin/v2/clusters (accept the self-signed certificate on first use)
5. Connect an application on pulsar+ssl://<PUBLIC_IP>:6651 with the client token; a ready-made client configuration is written to /root/pulsar-client.conf

Standalone runs the broker, BookKeeper and the metadata store in a single JVM, with data on the persistent volume under /var/lib/pulsar. The plaintext broker port 6650 and the plaintext web port 8080 stay bound to the instance - nginx on 443 and the TLS listener on 6651 are the only public surfaces. Replace the self-signed certificate with a CA-signed one before production. Pulsar keeps a message only while a subscription is waiting for it: create your subscription before publishing, or set a retention policy on the namespace.