Data labeling and model fine-tuning

This guide describes the whole journey, from raw data to the fine-tuned model served to the agents, with two services of the catalog: Label Studio for labeling and human review, and LLaMA-Factory for fine-tuning. Architecture decision: ADR-0172.

1. Label Studio: the annotators label the data (text, images, video), compare answers and review the outputs of the models.
2. Export: the annotations become a dataset in the LLaMA-Factory format: instructions (SFT), preferences (DPO), conversations with images or video (VLM).
3. LLaMA-Factory: the fine-tuning of LLM and VLM (SFT, LoRA, QLoRA, DPO) runs on the GPUs of a host, offline.
4. Model export: the adapter is merged into its base model.
5. Serving: vLLM serves the exported model, LiteLLM registers it and makes it available to the agents, which never call LLaMA-Factory.

Both services install with the podman method, on the isolated network of their service, without telemetry (hardened installation). Their service pages give every parameter, port, check and command.

1. Install Label Studio

Prerequisites. A PostgreSQL database (the postgresql service of the platform or an existing database of the customer), an S3 storage (SeaweedFS of the platform by default, or an existing bucket) and, in production, the authenticating HTTPS gateway of the platform in front of the interface. Label Studio keeps no state on its hosts: projects, tasks, annotations and users are in the database, uploaded media and exports in the bucket.

Mode	Components	Use
standalone	one application (app, uwsgi) and one interface (nginx) on the same host	trial, small team
replicated	two applications or more, one interface or more, behind HAProxy and keepalived	production

Parameter	Role
public_url	Address the users open, for example https://<label-studio-host>: links, CSRF origin, secure cookies with https
admin_email, admin_password	First account, created at the first start; the password is generated and kept in the Manager vault
secret_key	Django secret key, generated and shared by every replica
allow_signup	Off by default: only the invitation link of the organization registers a user
trusted_ca_pem	Internal authorities of the database (verify-ca, verify-full) and of an https S3 storage
allow_local_ml_backends	Allows internal machine learning backends (pre-annotation); off by default
max_upload_mb, workers	Largest upload, uwsgi processes of each application

The interface listens on the port port (8090). The uwsgi port app_port (8091) is published on 127.0.0.1 only in the standalone mode; in the replicated mode it is opened to the hosts of the cluster only, so that each interface balances over every application.

First sign in. Read the administrator password in the Manager vault, sign in, then change it in Account & Settings: it only serves at the first start. Then invite the annotators from Organization, Add Members (invitation link, Reset Link to invalidate it).

The version command of the service calls /usr/bin/curl on the host: add it to allowed_commands of the agent (Security of the Navin agent).

2. Label and review

1. Create a project and pick its labeling template (classification, named entities, free answers, comparison of two answers, image captioning and questions about an image, audio, video).
2. Import the tasks: upload from the interface, or the S3 storage of the project. For the bucket of the platform, leave the S3 endpoint of the project empty: Label Studio uses the one of the installation. An internal endpoint typed in a project is refused (protection against server side request forgery).
3. Label, then review: several annotations per task, filters and sorting of the Data Manager. The Community Edition has no roles and no review workflow: every member of the organization sees every project.
4. Export: Export, JSON format, or through the API GET /api/projects/{id}/export?exportType=JSON with a personal access token.

Labeling configurations of the three fine-tuning datasets, read by the scripts of the next section:

Dataset	Labeling configuration (inside <View>)
Instructions (SFT)	<Text name="prompt" value="$prompt"/>, <TextArea name="answer" toName="prompt"/>
Preferences (DPO)	<Text name="prompt" value="$prompt"/>, <Text name="answer1" value="$answer1"/>, <Text name="answer2" value="$answer2"/>, <Pairwise name="preference" toName="answer1,answer2"/>
Images (VLM)	<Image name="image" value="$image"/>, <Text name="prompt" value="$prompt"/>, <TextArea name="answer" toName="image"/>

3. Prepare the dataset

LLaMA-Factory reads the alpaca and sharegpt formats (json, jsonl, csv, parquet), described in dataset_info.json of the datasets folder. The scripts below turn the JSON export of Label Studio into a dataset and skip the cancelled annotations.

Instructions (SFT)

The tasks carry a prompt field; the reviewed answer is typed in the text area answer:

import json
import sys

tasks = json.load(open(sys.argv[1], encoding="utf-8"))
rows = []
for task in tasks:
    for annotation in task.get("annotations", []):
        if annotation.get("was_cancelled"):
            continue
        answers = [r["value"]["text"][0] for r in annotation["result"]
                   if r.get("from_name") == "answer" and r.get("type") == "textarea"]
        if answers:
            rows.append({"instruction": task["data"]["prompt"], "input": "", "output": answers[0]})
json.dump(rows, open(sys.argv[2], "w", encoding="utf-8"), ensure_ascii=False, indent=2)

Preferences (DPO)

The tasks carry a question prompt and two answers answer1 and answer2; the annotator picks the better one with Pairwise (left for answer1, right for answer2):

import json
import sys

tasks = json.load(open(sys.argv[1], encoding="utf-8"))
rows = []
for task in tasks:
    data = task["data"]
    for annotation in task.get("annotations", []):
        if annotation.get("was_cancelled"):
            continue
        choices = [r["value"]["selected"] for r in annotation["result"] if r.get("type") == "pairwise"]
        if choices and choices[0] in ("left", "right"):
            chosen, rejected = ("answer1", "answer2") if choices[0] == "left" else ("answer2", "answer1")
            rows.append({"instruction": data["prompt"], "input": "", "chosen": data[chosen],
                         "rejected": data[rejected]})
json.dump(rows, open(sys.argv[2], "w", encoding="utf-8"), ensure_ascii=False, indent=2)

Images (VLM)

The tasks carry an image image (uploaded, or an object of the S3 storage of the project) and a question prompt; the reviewed answer is typed in answer. The script downloads each image through the Label Studio API (address in LS_URL, personal access token in LS_TOKEN), then writes support_images.json and the support_images folder of its images:

import base64
import json
import os
import sys
import urllib.request
from pathlib import PurePosixPath

base, token = os.environ["LS_URL"].rstrip("/"), os.environ["LS_TOKEN"]


def call(path, access=None, body=None):
    headers = {"Authorization": f"Bearer {access}"} if access else {"Content-Type": "application/json"}
    with urllib.request.urlopen(urllib.request.Request(base + path, data=body, headers=headers)) as response:
        return response.read()


def source(task):
    uri = task["data"]["image"]
    if uri.startswith("/"):  # file uploaded into Label Studio
        return uri
    # object of the S3 storage of the project, read by Label Studio
    return f"/tasks/{task['id']}/resolve/?fileuri={base64.urlsafe_b64encode(uri.encode()).decode()}"


tasks = json.load(open(sys.argv[1], encoding="utf-8"))
name = sys.argv[2]  # for example support_images
os.makedirs(f"{name}/images", exist_ok=True)
rows = []
for task in tasks:
    answers = [r["value"]["text"][0] for a in task.get("annotations", []) if not a.get("was_cancelled")
               for r in a["result"] if r.get("from_name") == "answer" and r.get("type") == "textarea"]
    if not answers:
        continue
    access = json.loads(call("/api/token/refresh/", body=json.dumps({"refresh": token}).encode()))["access"]
    image = f"{name}/images/{task['id']}-{PurePosixPath(task['data']['image'].split('?')[0]).name}"
    with open(image, "wb") as out:
        out.write(call(source(task), access=access))
    rows.append({"messages": [{"role": "user", "content": "<image>" + task["data"]["prompt"]},
                              {"role": "assistant", "content": answers[0]}],
                 "images": [image]})
json.dump(rows, open(f"{name}.json", "w", encoding="utf-8"), ensure_ascii=False, indent=2)

The personal access token gives an access token valid about 5 minutes: the script asks for one per image. For an S3 storage of the project, untick Use pre-signed URLs: Label Studio then reads the objects itself, and the token never leaves Label Studio. A video is described the same way, with the videos column and the <video> tag in the message.

Copy the resulting files, and the support_images folder, into /var/lib/navin/llama-factory/datasets of the GPU host, with their description in dataset_info.json of the same folder. The image paths are relative to this folder (media_dir, the datasets folder by default):

{
  "support_answers": {
    "file_name": "support_answers.json",
    "formatting": "alpaca",
    "columns": {"prompt": "instruction", "query": "input", "response": "output"}
  },
  "support_preferences": {
    "file_name": "support_preferences.json",
    "ranking": true,
    "columns": {"prompt": "instruction", "query": "input", "chosen": "chosen", "rejected": "rejected"}
  },
  "support_images": {
    "file_name": "support_images.json",
    "formatting": "sharegpt",
    "columns": {"messages": "messages", "images": "images"},
    "tags": {"role_tag": "role", "content_tag": "content", "user_tag": "user", "assistant_tag": "assistant"}
  }
}

4. Install LLaMA-Factory

Prerequisites. An NVIDIA GPU host with its driver, the NVIDIA Container Toolkit and the CDI specification (nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml), 16 GB of memory or more and disk space for the models (100 GB advised). The preflight checks refuse a host without a GPU of 8 GB or more.

Mode	Use
single	one LLaMA Board interface training on the GPUs of its host (several GPUs through torchrun)
multi-host	one interface per GPU host, each one with its own models, datasets and runs

Parameter	Role
offline	On by default: no download from the interface, models and datasets already on the host
hf_endpoint, hf_token	Internal Hugging Face compatible mirror and token of the gated models, for download_model
pip_index_url	PyPI compatible index (internal mirror) of the install_qlora command, unused when the wheel is copied into wheels
gpu_devices	GPUs given to the interface: all, or a list of indexes or UUIDs (the GPUs of vLLM stay out)
shm_size	Shared memory of the container (data loaders, NCCL between the GPUs of the host)
port	Port of LLaMA Board (7860)

Folders of the host, below /var/lib/navin/llama-factory:

Folder	In the interface	Content
models/hf, models/local	/models/hf, /models/local	Hugging Face cache, imported weights
datasets	/app/shared_data	Datasets and dataset_info.json
configs	/app/configs (read only)	Reviewed configurations of the train and export commands
saves	/app/saves	Checkpoints and adapters, logs of the commands in navin-runs
output	/app/output	Exported models
python	/opt/navin/python (read only)	bitsandbytes of QLoRA, written by the install_qlora command only
wheels	install_qlora command only	bitsandbytes wheel copied for a host without access to an index

QLoRA. The official LLaMA-Factory image does not ship bitsandbytes, needed by the 4 and 8 bit quantization of QLoRA. The install_qlora command installs it once on every host of the service: bitsandbytes 0.50.2 (MIT), a wheel checked against its pinned sha256, into the python folder mounted read only in the board, without restart; it then shows the installed version. It takes the wheel from the wheels folder of the host when it is there, else from the pip_index_url index. Host without access to an index: copy bitsandbytes-0.50.2-py3-none-manylinux_2_24_x86_64.whl (sha256 55348a9a4a21bfd99cf8c7b32fe67b4030ae5c2a05738e03c1747f65fa6ec283) into /var/lib/navin/llama-factory/wheels before running the command.

LLaMA Board has no authentication of its own: in production its console only opens through the authenticating gateway of the platform. Gradio fetches its fonts from Google: without Internet access the interface uses the fonts of the workstation, without losing any function.

The commands of the service start /usr/bin/podman (download, training, export, logs), /usr/bin/nvidia-smi (GPU status) and /bin/sh (publication to vLLM and Ollama) on the host: add them to allowed_commands of the agent (Security of the Navin agent).

5. Base model

Two sources, offline afterwards:

• the download_model command (repository, revision): the weights go into the Hugging Face cache of every host of the service, from the internal mirror; the interface finds them by their identifier, for example Qwen/Qwen3-4B-Instruct-2507 (LLM) or Qwen/Qwen3-VL-4B-Instruct (VLM);
• validated weights copied into /var/lib/navin/llama-factory/models/local/<name>: path /models/local/<name>.

6. Train

From LLaMA Board: set the data folder (Data dir) to /app/shared_data, pick the model, the method (LoRA, QLoRA, full), the stage (SFT, DPO, KTO...) and the dataset, then start.

From the Manager: put a reviewed configuration into /var/lib/navin/llama-factory/configs, for example qwen3-lora-support.yaml:

model_name_or_path: Qwen/Qwen3-4B-Instruct-2507
trust_remote_code: true
stage: sft
do_train: true
finetuning_type: lora
lora_rank: 8
lora_target: all
dataset_dir: /app/shared_data
dataset: support_answers
template: qwen3_nothink
cutoff_len: 2048
preprocessing_num_workers: 16
dataloader_num_workers: 4
output_dir: saves/qwen3-4b/lora/support
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
report_to: none
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 1.0e-4
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000
resume_from_checkpoint: null

Run the train command of the service with the file name, then follow the training with runs (logs of the Manager and trainer_log.jsonl) and run_log. A configuration name only accepts letters, digits, dots, hyphens and underscores: it is passed as an argument, never inside the text of the script. A restart of the interface stops the trainings in progress: starting again with resume_from_checkpoint resumes from the last checkpoint.

QLoRA, DPO and VLM. The other methods keep the same configuration, with these lines changed:

Method	Lines to change
QLoRA (after install_qlora)	quantization_bit: 4 (or 8), quantization_method: bnb
DPO	stage: dpo, pref_beta: 0.1, pref_loss: sigmoid, dataset: support_preferences
VLM (images, video)	model_name_or_path: Qwen/Qwen3-VL-4B-Instruct, template: qwen3_vl_nothink, dataset: support_images, image_max_pixels: 262144, video_max_pixels: 16384, freeze_vision_tower: true

The methods combine: a VLM is also fine-tuned with QLoRA or DPO (sharegpt dataset with the chosen, rejected and images columns). DPO usually starts from the SFT fine-tuned model: export it first, then give it as model_name_or_path.

7. Export and serve with vLLM or Ollama

Export configuration, for example qwen3-merge-support.yaml (never merge a quantized model):

model_name_or_path: Qwen/Qwen3-4B-Instruct-2507
adapter_name_or_path: saves/qwen3-4b/lora/support
template: qwen3_nothink
trust_remote_code: true
export_dir: output/qwen3-4b-support
export_size: 5
export_device: cpu
export_legacy_format: false

The export command writes the merged model into /var/lib/navin/llama-factory/output/qwen3-4b-support. To serve it:

1. vLLM on the same host: publish_to_vllm command of LLaMA-Factory (export: qwen3-4b-support, name: qwen3-4b-support), which copies the folder into /var/lib/navin/vllm/models/local/qwen3-4b-support and never overwrites an existing folder; vLLM on another host: copy this folder there, to the same place;
2. set the vLLM service: model to qwen3-4b-support, model_source to local, served_model_name to qwen3-4b-support;
3. LiteLLM registers the model through its vllm binding; evaluate it (Opik, promptfoo) before opening it to the agents.

To serve it with Ollama (architectures Ollama imports from Safetensors: Llama, Mistral, Gemma, Qwen):

1. Ollama on the same host: publish_to_ollama command of LLaMA-Factory, which copies the folder into /var/lib/navin/ollama/import/qwen3-4b-support; Ollama on other hosts: copy this folder there, to the same place, on every instance;
2. import_safetensors command of the Ollama service (model: qwen3-4b-support, folder: qwen3-4b-support), run on every instance;
3. add qwen3-4b-support to litellm_models of the Ollama service and apply: LiteLLM registers it through its ollama binding.

A model already converted to GGUF goes through the import_model command of Ollama (.gguf file of the same import folder).

A QLoRA adapter is merged into the base model without quantization: the export configuration keeps model_name_or_path without quantization_bit. A VLM is exported the same way, with its template (template: qwen3_vl_nothink). To serve it, also set vision of the vLLM service to true, so that LiteLLM declares it with supports_vision, and if needed the number of images per request in extra_args: --limit-mm-per-prompt then {"image": 4, "video": 1}.

The fine-tuned weights keep the licence of their base model (LEX-012): record the licence and the sha256 digest of every model in the model registry.

8. Security and operations

Topic	Label Studio	LLaMA-Factory
Network	Isolated, interface published, uwsgi on 127.0.0.1 or opened to the cluster only	Isolated, LLaMA Board published behind the gateway
Privileges	--cap-drop ALL, user 1001 of the image	--cap-drop ALL, GPUs through CDI
Telemetry	Analytics, Sentry and version check off	Gradio analytics, public share, MCP and W&B off
Quantization	Not applicable	bitsandbytes pinned (version and sha256), installed by install_qlora, mounted read only
Accounts	First account generated, sign up by invitation	None: authenticating gateway
State	PostgreSQL database and S3 bucket, backed up by their services	/var/lib/navin/llama-factory, host backup
High availability	Replicated mode behind HAProxy and keepalived	No: a training resumes from its last checkpoint

Multi node training (torchrun across hosts) is not deployed by the catalog: NCCL would need the host network.