BLOOMZ

BLOOMZ.cpp is a lightweight C++ implementation designed to run BLOOM and BLOOMZ language models efficiently on local machines. Built for efficient CPU inference, it allows developers to perform offline inference without requiring Python or heavy deep learning frameworks such as PyTorch or Transformers.

Features of BLOOMZ.cpp:

• Supports BLOOM and BLOOMZ models converted from Hugging Face formats.
• Runs fully offline with CPU optimization (AVX, FMA, multi-threading support).
• No dependency on Python, PyTorch, or Transformers.
• Command-line interface for running prompts and generating responses.
• Lightweight and portable across Linux, macOS, and embedded systems.
• Suitable for private deployments, AI research, and multilingual applications.

BLOOMZ.cpp Version Check

$ sudo su
$ cd /opt/bloomz.cpp
$ python3 --version

# Verify executable exists
$ ls -l main

# Verify BLOOMZ.cpp runs
$ ./main --help

# Check installed model
$ ls models

# Check exact BLOOMZ.cpp version (Git commit ID)
$ git log -1
$ git rev-parse HEAD

    

Disclaimer: BLOOMZ.cpp is an open-source runtime and does not include pretrained model weights by default. Users must download and convert BLOOMZ model files separately. Performance depends on system hardware resources.