AI Runner: Offline AI Inference Engine for Hackers, Makers, and Builders.

Here are some of the things you can do with AI Runner:

• ✅ Voice-based chatbots for real-time conversations
• ✅ Text-to-Image generation with Stable Diffusion and ControlNet
• ✅ Image manipulation with inpainting and outpainting
• ✅ Text-to-Speech (TTS) and Speech-to-Text (STT) using OpenVoice, SpeechT5, and Whisper
• ✅ Customizable AI personalities for more engaging conversations
• ✅ Image filters and inpainting for image editing
• ✅ Retrieval-Augmented Generation (RAG) for enhanced LLM responses

For extra security, performance, and compatibility, AI Runner is built with Wayland support, Python 3.13, and the latest stable torch libraries.

System Requirements

💾 Installation Quick Start

🔧 Installation Steps

1. Install system requirements

   sudo apt update && sudo apt upgrade -y
   sudo apt install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python3-openssl git nvidia-cuda-toolkit pipewire libportaudio2 libxcb-cursor0 gnupg gpg-agent pinentry-curses espeak xclip cmake qt6-qpa-plugins qt6-wayland qt6-gtk-platformtheme mecab libmecab-dev mecab-ipadic-utf8
   sudo apt install espeak
   sudo apt install espeak-ng-espeak
   

2. Create airunner directory

   sudo mkdir ~/.local/share/airunner
   sudo chown $USER:$USER ~/.local/share/airunner
   

3. Install AI Runner - Python 3.13+ required pyenv and venv are recommended (see wiki for more info)

   pip install "typing-extensions==4.13.2"
   pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
   pip install airunner[all_dev]
   pip install -U timm
   

4. Run AI Runner

   airunner
   

For more options, including Docker, see the Installation Wiki.

Note: AI Runner does not distribute AI art models. You are responsible for obtaining and your own.

🛠️ Usage

Basic Usage

• Run AI Runner: airunner
• Build templates: airunner-build-ui

⭐ Features

Below is a high-level list of capabilities in AI Runner:

⚙️ System Requirements

System Requirements

Models

These are the sizes of the various models that power AI Runner.

AI Models

By default, AI Runner installs essential TTS/STT and minimal LLM components.
You must supply additional Stable Diffusion models (e.g., from Hugging Face or Civitai).

Organize them under your local AI Runner data directory:

~/.local/share/airunner
├── art
│   └── models
│       ├── SD 1.5
│       │   ├── lora
│       │   └── embeddings
│       ├── Flux
│       ├── SDXL 1.0
│       │   ├── lora
│       │   └── embeddings
│       └── SDXL Turbo
│           ├── lora
│           └── embeddings

Unit Tests

To run all tests:

python -m unittest discover -s src/airunner/tests

Or a single test:

python -m unittest src/airunner/tests/test_prompt_weight_convert.py

Database

AI Runner supports a simple database system. See the Wiki for how to:

• Switch engines (SQLite, etc.)
• Make schema changes
• Run migrations

Advanced Features

• Memory Optimization: TF32 Mode, VAE/Attention Slicing, Torch 2.0, sequential CPU offload, ToMe token merging.
• Experimental Integrations: Weather-based chatbot prompts, advanced command-line arguments (--perform-llm-analysis, --disable-setup-wizard, etc.).
• Safety & Guardrails: Optional NSFW content detection and adjustable guardrails for LLMs.

Contributing

We welcome pull requests for new features, bug fixes, or documentation improvements. You can also build and share extensions to expand AI Runner’s functionality. For details, see the Extensions Wiki.

Take a look at the Contributing document and the Development wiki page for detailed instructions.

Thank You!

Thanks for checking out AI Runner.
Get started with local AI inference in minutes—no more endless environment setup.
Questions or ideas? Join our Discord or open a GitHub Issue.

Happy building!