BLOOMZ

BLOOMZ.cpp is a lightweight C++ implementation designed to run BLOOM and BLOOMZ language models efficiently on local machines. Built on top of the llama.cpp framework, it enables developers to perform offline inference with large models without requiring Python or heavy deep learning frameworks.

Features of BLOOMZ.cpp:

• Supports BLOOM and BLOOMZ models converted from Hugging Face formats.
• Runs fully offline with CPU/GPU optimization (AVX, FMA, etc.).
• No dependency on Python, PyTorch, or Transformers.
• Command-line interface for running prompts and generating responses.
• Lightweight and highly portable – suitable for Linux, macOS, and embedded systems.
• Ideal for private deployments, research, and multilingual AI tasks.

To check the version of BLOOMZ :

$ sudo su
$cd /opt/bloomz.cpp
$python3 --version

Disclaimer: BLOOMZ.cpp is an open-source runtime and does not include pretrained weights by default. Users must obtain model files (e.g., BLOOMZ-7B) separately from Hugging Face and convert them as required. Proper hardware resources are recommended for large models.