Firecrawl on Ubuntu 24.04 LTS
par cloudimg
Self hosted web scraping and crawling API, turns any website into clean LLM ready markdown
Firecrawl is a high performance open source web scraping and crawling API. This cloudimg image turns a virtual machine into a complete scraping backend running entirely within your own subscription: point the REST API at a URL and it fetches the page, renders it in a real headless browser so JavaScript heavy sites work, and returns clean markdown or structured data ready to feed to a language model or a data pipeline. It crawls whole sites, following links up to a depth and page limit you choose, so you can build search indexes, knowledge bases and retrieval augmented generation datasets from live web content. It runs the official Firecrawl 2.11.118 release, pinned by image digest, with a bundled headless Chromium renderer, a Postgres job queue, a RabbitMQ broker and a Redis cache.
Scraping and crawling run entirely on the instance with no external Firecrawl cloud key, so your requests and the pages you fetch stay inside your own cloud account. The API is published to the loopback interface and exposed through an nginx reverse proxy on port 80, so you place your own TLS termination or load balancer in front for production. The renderer, queue, broker and cache run on a private Docker network and are never published to a host port. Firecrawl is driven entirely through its REST API and the official Firecrawl SDKs for Python and Node.
Security is built in. Self hosted Firecrawl has no built in authentication and is open by default, so this image is secure by default: nothing is exposed without a token. On the first boot of every VM a unique API bearer token, and unique database, cache, broker and admin secrets, are generated before the port is reachable and recorded in a root only file, and the token is enforced at the nginx front door. As a result the public health check is open for load balancers while every scrape and crawl request must carry the bearer token, an unauthenticated or wrong token request is rejected with HTTP 401, state starts empty with no prior data, no two instances share a credential, and no known credential is ever baked into the image.
Firecrawl is distributed under the GNU Affero General Public License v3.0 (AGPL-3.0), free and open source with no per CPU or per deployment fee. cloudimg is not affiliated with or endorsed by Firecrawl; the Firecrawl name is used nominatively to identify the open source software shipped in this image. cloudimg provides packaging, the bundled renderer, queue, broker and cache, secure by default per instance secret generation, the nginx enforced bearer token, security patching, and 24/7 support with a guaranteed 24 hour response SLA.