LLaMA-Factory (llama-factory)

Fine-tuning of public and private language and vision language models (LLM and VLM: text, images, video) on the NVIDIA GPUs of the host: SFT, LoRA, QLoRA (4 and 8 bit, bitsandbytes installed by the install_qlora command), DPO, KTO, reward modeling and PPO. The LLaMA Board web interface starts, follows and evaluates the trainings, the Manager runs reviewed training configurations, and the merged models are exported to be served by vLLM behind LiteLLM. Offline: the base models, the datasets and the runs stay on the host.

Generated from catalog/services/llama-factory.yaml: do not edit by hand.

Summary

Version	0.9.5
Use	install and configure, configure and connect
Layer	functional (AI catalog)
Category	Evaluation and LLM operations
Installation methods	podman
External connection	no
Sandbox only	no

Licence

SPDX identifier	Apache-2.0
Licence policy	core licence (MIT or Apache-2.0)
Licence exception	-
Licence note	The image bundles the CUDA runtime libraries of PyTorch, under the NVIDIA CUDA licence as in the vLLM image, while the NVIDIA driver stays on the host (LEX-011); the fine-tuned weights keep the licence of their base model (LEX-012). QLoRA installs bitsandbytes (MIT), pinned by version and sha256.
Platform component	llama-factory

Packages and artifacts

A package the project of the service publishes (its image, its binary archive, its Python or OS package) carries the licence of the service; a third-party package carries the licence written on its row, read from the official upstream metadata.

Kind	Package	Version	Licence	Details
container image	docker.io/hiyouga/llamafactory	0.9.5	Apache-2.0	image key llamafactory, used by board, digest sha256:c5e769e047a19f9322800c9881a6270557ef0e6682f0145b50b322f8724de6b6

Images are pinned by tag in the descriptor, and by digest when the descriptor declares one (the offline bundle then refuses a pulled image that does not carry it); the digest is recorded in the mirror of the Manager when the offline bundle is imported, and every pull or load is checked against it.

Supported systems and architectures

Method	Operating systems	Architectures
podman	RHEL family (dnf), Debian family (apt), Arch Linux (pacman)	x86_64, aarch64 (not restricted by the descriptor)

Certified host systems: RHEL 8/9/10, Rocky Linux 8/9/10, AlmaLinux 8/9/10, Oracle Linux 8/9, CentOS Stream 9/10, Ubuntu 22.04/24.04, Debian 12, Arch Linux (rolling). The host preflight accepts these systems (other releases of a known family give a warning, end of life releases are refused) and the x86_64 and aarch64 architectures.

GPU: NVIDIA GPU given to the component as CDI devices (nvidia.com/gpu): board.

Prerequisites

Host

• at least 16384 MB of memory (blocking, memory).
• at least 1 NVIDIA GPU with 8000 MB of memory (blocking, gpu).
• the program podman installed on the host (blocking, podman).
• at least 102400 MB free for /var/lib/navin (warning, data_disk).

Programs the commands and checks of the service start on the host, outside the managed roots: add them to allowed_commands of the agent (see Security of the Navin agent), else the agent refuses them with command_not_allowed: /usr/bin/nvidia-smi, /usr/bin/podman.

Network and ports

Component	Port name	Port	Opened in the host firewall
board	board	7860/tcp	yes

Services and connections needed

This service has no dependency.

Installation methods and modes

Method	Default	What the method uses
podman	yes	the container images of the descriptor, run by Podman under systemd

Components

Component	Kind	Installation methods	Ports
LLaMA Board (board)	master	podman	7860/tcp

Single GPU host (single) (default)

One board training on the GPUs of one host (several GPUs of the host through torchrun).

What the mode deploys:

Component	Placement
board	min 1, max 1

Several training hosts (multi-host)

One board per GPU host, each one training on the GPUs of its own host with its own models, datasets and runs; trainings of separate teams run side by side.

What the mode deploys:

Component	Placement
board	min 2, recommended 2

Install this service

The install wizard of the Navin Manager plans, validates and runs the installation; you need the services:install permission on the cluster.

1. Open the wizard. In Clusters, choose the row action Install services of the target cluster. The wizard opens on its first step, Service.

2. Choose the service. On the Service step, select LLaMA-Factory. The card shows the licence Apache-2.0.

3. Choose the hosts and the mode. On the Hosts step, tick the hosts, then choose the Mode: single (default), multi-host. Each mode deploys the components listed in Installation methods and modes.

4. Bindings. This service has no dependency: the Bindings step has nothing to bind.

5. Configure. On the Configure step, review the parameters (see Configuration parameters). Passwords left empty with a generated default are generated and sealed by the Manager.

6. Review and install. On the Install step the Manager validates the plan and shows the installation order; when it says "The plan is valid. Ready to install.", choose Install. A highly available mode is installed only when its HA checks pass.

7. Verify. The operation ends when the health checks of every instance pass (see Health checks); the service then shows as running in Services.

Connect an existing service

Schemes	https, http
Default port	7860
Authentication	none
Health check	/

Authentication method	Fields	Description
none	-	LLaMA Board has no authentication of its own: publish it behind the authenticating gateway of the platform.

Official documentation: https://llamafactory.readthedocs.io/en/latest/getting_started/webui.html

High availability

Cluster	none (none)
High availability	none (none)
Failover	none (none)
Replication	none (none)
Quorum	none (none)
Discovery	none (none)
Entry point	per host (per-host)
Official documentation	https://llamafactory.readthedocs.io/en/latest/getting_started/webui.html
References	https://github.com/hiyouga/LLaMA-Factory/tree/v0.9.5/docker/docker-cuda

Topology per mode

Modes	Highest catalog level	Distribution	Quorum component	Fault domains	Description
single	standalone (standalone)	single (single)	-	1	One training host, no fault tolerance.
multi-host	standalone (standalone)	per host (per-host)	-	2	Independent training hosts, each one reached on its own address.

Notes

• A training is a batch job: it stops with its host or its board and resumes from the last checkpoint of its output_dir (resume_from_checkpoint). Multi node training (torchrun across hosts) is not deployed by the catalog: it needs the host network for NCCL.

Configuration parameters

Models and data

Parameter	Type	Default	Allowed values	Required	Secret	Restart	Description
Offline (offline)	bool	true	-	no	no	yes	Never download from the board (HF_HUB_OFFLINE, TRANSFORMERS_OFFLINE, HF_DATASETS_OFFLINE): the base models and the datasets must already be on the host. Required for a production host without Internet.
Model repository mirror (hf_endpoint)	string	(empty)	^(|https?://[A-Za-z0-9.-]+(:[0-9]{1,5})?(/[A-Za-z0-9._~/-]*)?)$	no	no	no	Internal Hugging Face compatible mirror used by download_model (HF_ENDPOINT), empty for huggingface.co.
Repository token (hf_token)	password	-	-	no	yes	no	Token of the gated or private models of the repository (HF_TOKEN), optional.
Python package index (pip_index_url)	string	https://pypi.org/simple	^https?://[A-Za-z0-9.-]+(:[0-9]{1,5})?(/[A-Za-z0-9._~/-]*)?$	no	no	no	PyPI compatible index (internal mirror) from which install_qlora downloads the pinned bitsandbytes wheel, checked against its sha256; unused when the wheel is copied into /var/lib/navin/llama-factory/wheels of the host.

GPU host

Parameter	Type	Default	Allowed values	Required	Secret	Restart	Description
GPUs (gpu_devices)	string	all	^(all|[A-Za-z0-9-]+(,[A-Za-z0-9-]+)*)$	no	no	yes	GPUs given to the board and to its trainings: all, or a comma list of indexes or UUIDs (GPUs kept for vLLM stay out).
Shared memory (shm_size)	string	16g	^[0-9]+[mg]$	no	no	yes	Size of /dev/shm of the container, used by the data loaders and by NCCL between the GPUs of the host.

Network

Parameter	Type	Default	Allowed values	Required	Secret	Restart	Description
Board port (port)	port	7860	min 1024	no	no	yes	Port of LLaMA Board (Gradio). The board has no authentication of its own: publish it behind the authenticating gateway of the platform.
TLS (tls_enabled)	bool	false	-	no	no	yes	Serve LLaMA Board over TLS with the certificate the Manager issues to the host from its CA: the board is then started by a short launcher run by the Python of the image (launch of Gradio with ssl_certfile and ssl_keyfile) instead of llamafactory-cli webui, with the same interface.
Log level (log_level)	enum	INFO	DEBUG, INFO, WARNING, ERROR	no	no	no	Verbosity of LLaMA-Factory (LLAMAFACTORY_VERBOSITY).

Endpoints

Endpoint	Scheme	Component	Port	Path	TLS parameter	Binding parameters	Binding secrets	Provisioning command
board	http	board	7860 (port)	-	tls_enabled	-	-	-

Commands

Command	Component	Target	Description
Board logs (logs)	board	chosen host	Last lines of the journal of the board and of the trainings it started.
Download a model (download_model)	board	every host	Downloads the weights of a Hugging Face model into the cache of every host of the service (from huggingface.co or the internal mirror); the board then finds it offline by its identifier.
Enable QLoRA (install_qlora)	board	every host	Installs bitsandbytes 0.50.2 (MIT), needed by QLoRA (4 and 8 bit quantization of the base model during the training) and absent from the official image, on every host of the service: the wheel is checked against its pinned sha256 and goes into /var/lib/navin/llama-factory/python, mounted read only in the board. It comes from the wheels folder of the host when it was copied there (host without access to an index), else from the Python package index of the service. No restart: the next trainings find it.
Start a training (train)	board	chosen host	Starts llamafactory-cli train in the background with a reviewed configuration of the configs folder of the host (/var/lib/navin/llama-factory/configs, /app/configs in the board); its output goes to a log of /app/saves/navin-runs (commands runs and run_log). A restart of the board stops the trainings in progress: they resume from their last checkpoint.
Export a model (export)	board	chosen host	Starts llamafactory-cli export in the background with a configuration of the configs folder (merge of the adapter into its base model, export_dir under /app/output); its output goes to a log of /app/saves/navin-runs.
Publish to vLLM (publish_to_vllm)	board	chosen host	Copies an exported model (folder of /app/output) into the imported weights of the vLLM installed on the same host (/var/lib/navin/vllm/models/local/<name>). Then set the vLLM parameters model_source to local and model to <name> and apply: LiteLLM registers the model by its binding. Another host: copy the folder there first.
Publish to Ollama (publish_to_ollama)	board	chosen host	Copies an exported model (folder of /app/output) into the import folder of the Ollama installed on the same host (/var/lib/navin/ollama/import/<name>). Then run the Ollama command import_safetensors and add the model to litellm_models. Another host: copy the folder there first.
Runs (runs)	board	chosen host	Logs of the trainings and exports started by the Manager, newest first, then the training logs (trainer_log.jsonl) of the board and of the Manager under /app/saves and /app/output.
Run log (run_log)	board	chosen host	Last 200 lines of a log of /app/saves/navin-runs (name given by runs).
Environment (env)	board	chosen host	Versions of LLaMA-Factory, PyTorch, Transformers, CUDA and the GPUs seen by the board (llamafactory-cli env).
GPU status (gpu_status)	board	every host	Usage, memory and temperature of the GPUs of every host (nvidia-smi).

Command parameters

Command	Parameter	Type	Default	Required	Description
download_model	Repository (repo)	string	-	yes	Hugging Face identifier, for example Qwen/Qwen3-8B.
download_model	Revision (revision)	string	main	no	Branch, tag or commit of the validated weights.
train	Configuration (config)	string	-	yes	File name of the YAML configuration in the configs folder, for example qwen3-lora-sft.yaml.
export	Configuration (config)	string	-	yes	File name of the YAML export configuration in the configs folder.
publish_to_vllm	Exported folder (export)	string	-	yes	Name of the export_dir folder below /app/output.
publish_to_vllm	Model folder (name)	string	-	yes	Folder name below the imported weights of vLLM.
publish_to_ollama	Exported folder (export)	string	-	yes	Name of the export_dir folder below /app/output.
publish_to_ollama	Import folder (name)	string	-	yes	Folder name below the import folder of Ollama.
run_log	Log (name)	string	-	yes	File name of the log, for example qwen3-lora-sft-20261102-093000-train.log.

Health checks

Liveness (per instance)

Component	Check	Details
board	HTTP {{ 'https' if config.tls_enabled else 'http' }}://{{ host.fqdn if config.tls_enabled else (host.ip or '127.0.0.1') }}:{{ config.port }}/	Expected 200; 90 retries; interval 5 s; timeout 5 s

Readiness (service check)

Check	Details
HTTP {{ 'https' if config.tls_enabled else 'http' }}://{{ host.fqdn if config.tls_enabled else (host.ip or '127.0.0.1') }}:{{ config.port }}/	Expected 200; 10 retries; interval 3 s; timeout 10 s

Preflight checks

Check	Type	Meaning	Severity	Components
memory	memory_mb {"min": 16384}	at least 16384 MB of memory	blocking	every component
gpu	gpu {"min_count": 1, "min_memory_mb": 8000}	at least 1 NVIDIA GPU with 8000 MB of memory	blocking	every component
podman	binary {"name": "podman"}	the program podman installed on the host	blocking	every component
data_disk	disk_free_mb {"path": "/var/lib/navin", "min": 102400}	at least 102400 MB free for /var/lib/navin	warning	every component
port_free	port_free {"param": "port"}	port 7860 free on the host	blocking	every component

Uninstall

Paths removed when the data is purged: {{ paths.data }}, {{ paths.log }}.

Quick links

• LLaMA Board: {{ 'https' if config.tls_enabled else 'http' }}://{{ host.fqdn }}:{{ config.port }}/

Upstream documentation

• Homepage: https://llamafactory.readthedocs.io
• Source: https://github.com/hiyouga/LLaMA-Factory
• High availability: https://llamafactory.readthedocs.io/en/latest/getting_started/webui.html
• High availability: https://github.com/hiyouga/LLaMA-Factory/tree/v0.9.5/docker/docker-cuda
• Authentication: https://llamafactory.readthedocs.io/en/latest/getting_started/webui.html