Skip to main content

ADR-0172: Data labeling and model fine-tuning: Label Studio and LLaMA-Factory in the catalog

  • Status: accepted
  • Date: 2026-11-02
  • Scope: catalog of the Navin Manager (navin-manager/catalog/services/label-studio.yaml, navin-manager/catalog/services/llama-factory.yaml), console registry, managed components
  • French reference: ADR-0172; guide: navin-manager/docs/en/labeling-finetuning.md

Context​

The platform serves models (vLLM behind LiteLLM) and evaluates them (Opik, promptfoo, Ragas, garak), but offered no data labeling, no human review of the model outputs and no fine-tuning of public and private models. The customer asks for two services, with their licence: Label Studio (Apache-2.0) for labeling text, images, audio and video, human review and dataset preparation; LLaMA-Factory (Apache-2.0) for fine-tuning (SFT, LoRA, QLoRA, DPO) and training management. They install like the other services of the catalog: offline, hardened as ADR-0171 requires, documented in French and English.

Decision​

  1. Label Studio 1.23.2, Community Edition, podman method. Official image heartexlabs/label-studio pinned by digest, two components as in the docker-compose.yml of the project: app (uwsgi) and nginx (static files, proxy to the application, media). No python method: label-studio start of the PyPI package runs the Django development server, and the package ships neither the nginx configuration nor the uwsgi entry of the image.
  2. State in the services of the platform. PostgreSQL database of the database binding, never SQLite; media and exports in the S3 bucket of the s3 binding (SeaweedFS by default), served by nginx through X-Accel-Redirect. The S3 endpoint of the installation is the only trusted internal endpoint (S3_ENDPOINT); the protection against server side request forgery stays on for the endpoints typed in a project, the imports by URL and the webhooks.
  3. Security of Label Studio. Isolated network of the service, --cap-drop ALL, user 1001 of the image. Sign up by invitation (LABEL_STUDIO_DISABLE_SIGNUP_WITHOUT_LINK), administrator account and secret key generated at install and kept in the vault, secure cookies and CSRF origin behind the HTTPS gateway (public_url), telemetry, Sentry reports (default DSN of the image) and version check off, local files off, internal trusted authorities for the database and the storage (trusted_ca_pem). The replicated mode shares the database, the bucket and the key: any replica serves any user, and only the first application creates the first account.
  4. LLaMA-Factory 0.9.5, podman method. PyTorch and CUDA stack of the official image hiyouga/llamafactory, pinned by digest. One LLaMA Board interface (Gradio) per GPU host, GPUs through CDI (nvidia.com/gpu), isolated network, --cap-drop ALL, offline (HF_HUB_OFFLINE, TRANSFORMERS_OFFLINE, HF_DATASETS_OFFLINE), no public share link, no Gradio analytics, no Gradio MCP server, no W&B. Models, datasets, configurations, runs and exports stay in /var/lib/navin/llama-factory of the host.
  5. Trainings driven by the Manager. Commands download_model (Hugging Face cache, internal mirror), train and export (reviewed YAML configurations of the configs folder, started in the background with their log), runs, run_log, env, gpu_status. A parameter never enters the text of a script: it is passed as an argument, and its pattern refuses the shell characters and the paths.
  6. Serving a fine-tuned model. The export merges the adapter into its base model; the exported folder is imported into models/local of a vLLM host (model_source: local), which LiteLLM registers through its binding. The agents never call LLaMA-Factory.
  7. Console without authentication. LLaMA Board has no authentication: its console is declared without authentication and with external resources (Google fonts of Gradio); the console checks only open it in production through the authenticating gateway.
  8. Licences. Apache-2.0 for both (ADR-0101). The CUDA libraries of the LLaMA-Factory image fall under LEX-011 as those of vLLM; the fine-tuned weights keep the licence of their base model (LEX-012).

Rejected alternatives​

  • Label Studio with SQLite: one server, no replica, state on the host, against the high availability and the backups of the platform.
  • Local media storage of Label Studio: it would need a volume shared between the hosts; the S3 bucket of the platform replaces it.
  • Multi node training of LLaMA-Factory (torchrun across hosts): NCCL needs the host network and dynamic ports; rejected until the need is demonstrated (ADR-0171).
  • python method for LLaMA-Factory: the PyTorch, CUDA, flash-attn and DeepSpeed stack depends on the driver and the CUDA version; the official image pins it.
  • Label Studio Enterprise (roles, SSO, review): commercial licence, outside the licence policy.

Consequences​

The catalog holds 51 services. Label Studio needs PostgreSQL and an S3 storage (SeaweedFS of the platform by default). LLaMA-Factory needs an NVIDIA GPU host (driver, NVIDIA Container Toolkit and CDI specification), 16 GB of memory or more and disk space for the models. The Community Edition of Label Studio has no roles: access is controlled by the invitation and the gateway.

Addition of 2 November 2026 (LLM and VLM, QLoRA, DPO): LLaMA-Factory also fine-tunes vision language models (for example Qwen3-VL) on sharegpt datasets with images or video, which Label Studio prepares like the DPO preferences (Pairwise comparison). The official image does not ship bitsandbytes, needed by QLoRA: the install_qlora command installs bitsandbytes 0.50.2 (MIT), a wheel checked against its pinned sha256, into a folder of the host mounted read only in the board (PYTHONPATH), from the wheels folder of the host or the Python index of the service. The image stays the one of the publisher, pinned by its digest: no image is rebuilt. The vision parameter of vLLM declares the served VLM to LiteLLM (supports_vision).

Addition of 2 November 2026 (serving with vLLM or Ollama): the publish_to_vllm and publish_to_ollama commands of LLaMA-Factory copy a merged export into the imported weights of vLLM (model_source: local) or into the import folder of Ollama of the same host, without ever overwriting a folder; the new import_safetensors command of Ollama turns that folder into a model on every instance, published in LiteLLM through litellm_models. The names are positional arguments or environment variables, never part of a script, and their pattern refuses paths.

Evidence​

navin-manager/server/tests/test_catalog_labeling_finetuning.py, navin-manager/server/tests/test_catalog.py, navin-manager/server/tests/test_consoles.py, navin-manager/server/tests/test_console_credentials.py, services/control-plane/tests/test_functional_catalog_coverage.py, services/control-plane/tests/test_platform_components.py, navin-manager/server/tests/test_catalog_inference.py.