Apache NiFi - Hardened Dataflow Automation Integration

What is Apache NiFi

Apache NiFi is an open-source dataflow automation and data integration server: it moves, routes, transforms and enriches data between systems, and it does it through a live browser canvas rather than code. It is a Java application running on a single JVM, built on flow-based programming - you drag processors onto a canvas, connect them into a flow, and NiFi handles queuing, back-pressure, retries and prioritisation between each step. This image ships more than 400 processors across 118 extension bundles: local and remote files, SFTP, HTTP and REST, JDBC databases, Kafka, MQTT, syslog, email, object storage, plus record-oriented readers and writers for JSON, Avro, CSV, XML and Parquet. Full data provenance records where each piece of data came from, what was done to it and where it went.

Why self-host Apache NiFi

A dataflow engine sees every record it moves, which makes it one of the most sensitive systems an organisation runs. Self-hosting keeps that data, the credentials for every connected system, and the provenance trail entirely inside your own subscription and region - a concrete answer for GDPR data residency and for auditors who ask where regulated data is processed. Apache-2.0 licensing means no seat counting, no feature gating and no vendor lock-in: the same open-source build runs on your laptop and on this VM.

What this VM image adds

Security hardening:

• Administrator password generated at first launch - NiFi would otherwise generate a random username and password and print them only to an application log; here a 24-character password for the "admin" account is written to /root/nifi-credentials.txt (mode 600)
• Per-instance flow encryption key - passwords and tokens entered into processors are encrypted with a key unique to your VM, so no two deployments share a secret
• Loopback binding behind a TLS perimeter - NiFi listens on 127.0.0.1:8443 only; nginx terminates TLS on 443 and reverse-proxies to it
• Anonymous access disabled - every request to the canvas and the REST API is authenticated
• Address re-derived on every boot - the interface keeps working after a restart changes the VM's public address
• Data kept separate from the application - flow definition, queued data, content and provenance live under /var/lib/nifi, so an upgrade does not touch them
• certbot pre-installed - one command swaps the self-signed certificate for a CA-signed one
• CVE scan - every image is scanned for vulnerabilities before release
• UFW firewall - only ports 22 and 443 open
• fail2ban - SSH brute-force protection
• AppArmor - mandatory access control

OS hardening (CIS Level 1):

• CIS Level 1 hardened - CIS Ubuntu 24.04 LTS Level 1 Benchmark via ansible-lockdown
• auditd - system call auditing for critical paths
• SSH hardening - PasswordAuthentication disabled, key-only access
• Kernel hardening - SYN cookies, ASLR, rp_filter, TCP BBR
• IMDS endpoints - egress rules pre-configured (169.254.169.254, 168.63.129.16)

Compliance artifacts (inside the VM):

• SBOM - CycloneDX 1.6 at /etc/lynxroute/sbom.json
• CIS Conformance Report - OpenSCAP HTML at /etc/lynxroute/cis-report.html
• Tailored CIS profile - /usr/share/doc/lynxroute/CIS_TAILORED_PROFILE.md

Quick Start

1. Deploy the VM (Standard_D2s_v3 or larger recommended)
2. SSH: ssh -i key.pem <username>@<PUBLIC_IP> (username set during VM creation, default: azureuser)
3. Read the generated login: sudo cat /root/nifi-credentials.txt
4. Open https://<PUBLIC_IP>/nifi/ and accept the self-signed certificate warning. First launch takes 3-6 minutes while the extension bundles are unpacked; until then the page shows a loading screen that refreshes itself.
5. Sign in as "admin", then drag a processor onto the canvas, connect it to a second one and start the flow

Restrict inbound port 443 to your own address until you have signed in and changed the administrator password, and replace the self-signed certificate before production use.