File size: 1,223 Bytes
4c1d7d9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
# flux-schnell-edge-inference
This holds the baseline for the FLUX Schnel NVIDIA GeForce RTX 4090 contest, which can be forked freely and optimized
Some recommendations are as follows:
- Installing dependencies should be done in `pyproject.toml`, including git dependencies
- HuggingFace models should be specified in the `models` array in the `pyproject.toml` file, and will be downloaded before benchmarking
- The pipeline does **not** have internet access so all dependencies and models must be included in the `pyproject.toml`
- Compiled models should be hosted on HuggingFace and included in the `models` array in the `pyproject.toml` (rather than compiling during loading). Loading time matters far more than file sizes
- Avoid changing `src/main.py`, as that includes mostly protocol logic. Most changes should be in `models` and `src/pipeline.py`
- Ensure the entire repository (excluding dependencies and HuggingFace models) is under 16MB
For testing, you need a docker container with pytorch and ubuntu 22.04.
You can download your listed dependencies with `uv`, installed with:
```bash
pipx ensurepath
pipx install uv
```
You can then relock with `uv lock`, and then run with `uv run start_inference`
|