Emil_Shanaty/Imagen
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

README: make model-agnostic, document model swapping

Browse Source
This commit is contained in:
Emil
2026-06-28 14:55:12 +03:00
parent 78db0b6038
commit 6e8ab5732e
1 changed files with 51 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+51 -15
View File
@@ -1,9 +1,12 @@
# Imagen
MCP server for generating pixel-art sprites using FLUX.2-klein-4B + [pixel-art-lora](https://huggingface.co/Limbicnation/pixel-art-lora).
MCP server for generating pixel-art sprites with transparent backgrounds. Bring your own model — any [diffusers](https://github.com/huggingface/diffusers)-compatible text-to-image pipeline works.
> The default configuration uses [FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B) + [pixel-art-lora](https://huggingface.co/Limbicnation/pixel-art-lora), but you can swap in any model you like.
## Features
- **Bring your own model** — any diffusers-compatible pipeline (FLUX, SDXL, SD3, etc.)
- **Text-to-sprite generation** — describe any character, get a pixel-art PNG
- **Transparent background** — automatic background removal via flood-fill
- **Pixel-art effect** — downscale/upscale with NEAREST interpolation
@@ -21,28 +24,42 @@ source venv/bin/activate
pip install -r requirements.txt
```
### 2. Download models
### 2. Download a model
You need a base text-to-image model. Optionally, a LoRA adapter for pixel-art style.
**Example: FLUX.2-klein-4B + pixel-art-lora (default)**
```bash
# Create models directory
mkdir -p ~/models
# Download base model (~23 GB)
# Base model (~23 GB)
huggingface-cli download black-forest-labs/FLUX.2-klein-4b \
--local-dir ~/models/flux2-klein-4b
# Download LoRA adapter (~625 MB)
# LoRA adapter (~625 MB) — optional but recommended for pixel-art
huggingface-cli download Limbicnation/pixel-art-lora \
--local-dir ~/models/pixel-art-lora
```
### 3. Configure paths (optional)
**Other models that work:**
| Model | Size | LoRA support | Notes |
|---|---|---|---|
| [FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4b) | ~23 GB | Yes | Default, distilled (4 steps) |
| [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) | ~23 GB | Yes | More detail, slower (20+ steps) |
| [SDXL](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) | ~7 GB | Yes | Lighter, good for 8 GB VRAM |
| [SD3.5-large](https://huggingface.co/stabilityai/stable-diffusion-3.5-large) | ~16 GB | Yes | Good quality/speed balance |
> **Note:** You may need to adjust `LORA_SCALE` and pipeline class in `server.py` depending on your model. See [Configuration](#configuration).
### 3. Configure paths
By default, models are expected at `~/models/`. Override with environment variables:
```bash
export IMAGEGEN_MODEL_DIR=/path/to/flux2-klein-4b
export IMAGEGEN_LORA_DIR=/path/to/pixel-art-lora
export IMAGEGEN_MODEL_DIR=/path/to/your/base-model
export IMAGEGEN_LORA_DIR=/path/to/your/lora # optional, set empty to disable
export IMAGEGEN_OUTPUT_DIR=/path/to/output
```
@@ -79,7 +96,7 @@ Generate a single pixel-art sprite.
| `seed` | int? | null | Seed for reproducibility |
| `width` | int | 512 | Image width |
| `height` | int | 512 | Image height |
| `steps` | int | 4 | Inference steps (FLUX.2-klein is distilled) |
| `steps` | int | 4 | Inference steps (lower = faster, less detail) |
| `remove_bg` | bool | true | Remove background, make transparent |
| `pixel_size` | int | 4 | Pixel block size (0 = off, 4 = chunky pixel-art) |
@@ -87,26 +104,45 @@ Generate a single pixel-art sprite.
Generate multiple sprites in one call. Accepts a list of specs with the same parameters.
## Configuration
### Environment variables
| Variable | Default | Description |
|---|---|---|
| `IMAGEGEN_MODEL_DIR` | `~/models/flux2-klein-4b` | Path to base model |
| `IMAGEGEN_LORA_DIR` | `~/models/pixel-art-lora` | Path to LoRA adapter |
| `IMAGEGEN_OUTPUT_DIR` | `./output` | Default output directory |
### Swapping models
The server is configured for FLUX.2-klein by default. To use a different model, edit `server.py`:
1. **Pipeline class** — replace `Flux2KleinPipeline` with your model's pipeline (e.g. `StableDiffusionXLPipeline` for SDXL)
2. **LoRA scale** — adjust `LORA_SCALE` (rsLoRA needs ~0.1, regular LoRA typically 0.7-1.0)
3. **Guidance scale** — distilled models ignore it; standard models need 5-8
4. **Steps** — distilled models work at 4; standard models need 20-30
## How It Works
1. **Generation** — FLUX.2-klein-4B (4B params, distilled to 4 steps) with pixel-art LoRA (scale 0.1 for rsLoRA compatibility)
1. **Generation** — text-to-image model generates a 512x512 image
2. **Pixelation** — downscale with LANCZOS, upscale with NEAREST → chunky pixel-art blocks
3. **Background removal** — detect border color, normalize to magenta fill, flood-fill from edges → transparent PNG
3. **Background removal** — detect border color, normalize to solid fill, flood-fill from edges → transparent PNG
## Requirements
- **GPU:** NVIDIA with >= 8 GB VRAM (uses CPU offload)
- **GPU:** NVIDIA with >= 8 GB VRAM (uses CPU offload automatically)
- **Python:** 3.12+
- **CUDA:** 12.0+
## Credits
- Base model: [FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B) by Black Forest Labs (Apache 2.0)
- LoRA: [pixel-art-lora](https://huggingface.co/Limbicnation/pixel-art-lora) by Limbicnation (Apache 2.0)
- Default model: [FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B) by Black Forest Labs (Apache 2.0)
- Default LoRA: [pixel-art-lora](https://huggingface.co/Limbicnation/pixel-art-lora) by Limbicnation (Apache 2.0)
- MCP SDK: [modelcontextprotocol/python-sdk](https://github.com/modelcontextprotocol/python-sdk)
## License
MIT — see [LICENSE](LICENSE)
Model licenses are separate (Apache 2.0). Check model cards for details.
Model licenses are separate from this project. Check each model's license card for usage terms.