F-02 Image generation and editing

Brick F of the Enterprise AI Platform. Technology: Hugging Face Diffusers (SDXL, FLUX.1 schnell under compatible license) on GPU workers + LiteLLM image generation for external providers. French version: ../../fr/bricks/F-02-image-generation.md.

Role and responsibilities

• Generate images from text, edit images with a mask (inpainting), produce variations.
• Apply output moderation and provenance (C2PA manifest, invisible watermark when the model supports it).

Position and interactions

• Execution path: asynchronous job (queue) with synchronous wrapper for small images.
• A quota reservation (H-01) is taken before the GPU job starts, committed with the measured GPU-seconds.
• Results are stored as new media (F-12) and linked to the run in the Execution Contract.

Open source and developed parts

• Open source reused: Diffusers, PyTorch, safety checker models, LiteLLM image_generation.
• Developed in the platform: Job API, prompt policy (brand and forbidden subjects), provenance, cost estimate per size/quality/count.

Data model

Control Plane tables (schema eai): eai.media, eai.multimodal_jobs, eai.usage_events, eai.quota_reservations.

Every tenant table has tenant_id uuid NOT NULL and a row level security policy on current_setting('app.tenant_id', true)::uuid. ClickHouse usage_events is the analytics replica of eai.usage_events (ReplacingMergeTree keyed by event_id).

Contract: contracts/v1/multimodal-contract.schema.json (schema_version 1.0). Usage is always reported with contracts/v1/usage-event.schema.json, audit with contracts/v1/audit-event.schema.json.

API endpoints

Method	Path
POST	/v1/multimodal/images/generate
POST	/v1/multimodal/images/edit
GET	/v1/multimodal/jobs/{id}

POST /v1/multimodal/images/generate

Request:

{
  "model": "sdxl-local",
  "prompt": "Isometric illustration of a secure data center, corporate colors",
  "size": "1024x1024",
  "n": 2
}

Response:

{
  "job_id": "mmj_9A1",
  "status": "succeeded",
  "media_ids": [
    "med_IMG1",
    "med_IMG2"
  ],
  "provenance": {
    "c2pa": true,
    "model_id": "sdxl-local"
  },
  "usage": {
    "images": 2,
    "image_size": "1024x1024",
    "image_quality": "standard",
    "gpu_seconds": 7.4
  }
}

POST /v1/multimodal/images/edit

Request:

{
  "model": "sdxl-inpaint",
  "media_id": "med_IMG1",
  "prompt": "Replace the logo with a neutral shape",
  "size": "1024x1024"
}

Response:

{
  "job_id": "mmj_9A2",
  "status": "succeeded",
  "media_ids": [
    "med_IMG3"
  ],
  "usage": {
    "images": 1,
    "image_size": "1024x1024"
  }
}

GET /v1/multimodal/jobs/{id}

Response:

{
  "job_id": "mmj_V01",
  "operation": "video_process",
  "status": "succeeded",
  "outputs": {
    "frames": 118,
    "audio_media_id": "med_AUD7"
  },
  "attempts": 1
}

SDK examples

Python

from enterprise_ai import EnterpriseAI

client = EnterpriseAI(base_url="<control-plane-url>", api_key="eai_live_xxx", tenant="acme", locale="en")

with client.quotas.reservation(agent_id="agt_ap", amount="1.80", currency="EUR") as r:
    result = client.multimodal.video.generate("Teaser", model="video-gen-gpu", seconds=15)
    r.commit(cost=result.get("cost_measured"))  # released automatically on exception
estimate = client.finops.estimate("openai", usage_type="llm", model_id="gpt-4o-mini", units={"input_tokens": 12000, "output_tokens": 2000})
client.multimodal.image.generate("Isometric secure data center", model="sdxl-local", size="1024x1024", n=2)
client.multimodal.image.edit("med_IMG1", "Replace the logo", size="1024x1024")
client.multimodal.job(job_id="mmj_V01")

TypeScript

import { EnterpriseAI } from "@enterprise-ai/sdk";

const client = new EnterpriseAI({ baseUrl: "<control-plane-url>", apiKey: process.env.EAI_API_KEY, tenant: "acme", locale: "en" });
await client.multimodal.generateImage({ model: "sdxl-local", prompt: "Isometric secure data center", size: "1024x1024", n: 2 });
await client.multimodal.editImage({ media_id: "med_IMG1", prompt: "Replace the logo" });
await client.multimodal.getJob("mmj_V01");

Java

EnterpriseAIClient client = EnterpriseAIClient.builder()
    .baseUrl("<control-plane-url>")
    .apiKey(System.getenv("EAI_API_KEY"))
    .tenant("acme")
    .locale(Locale.ENGLISH)
    .build();
client.multimodal().generateImage(Map.of("model", "sdxl-local", "prompt", "Isometric secure data center", "size", "1024x1024", "n", 2));
client.multimodal().editImage(Map.of("media_id", "med_IMG1", "prompt", "Replace the logo"));
client.multimodal().job("mmj_V01");

Go

client, err := eai.NewClient(eai.WithBaseURL("<control-plane-url>"), eai.WithAPIKey(os.Getenv("EAI_API_KEY")), eai.WithTenant("acme"), eai.WithLocale("en"))
if err != nil { log.Fatal(err) }
ctx := context.Background()
_, err = client.Multimodal.GenerateImage(ctx, map[string]any{"model": "sdxl-local", "prompt": "Isometric secure data center", "size": "1024x1024", "n": 2})
_, err = client.Multimodal.EditImage(ctx, map[string]any{"media_id": "med_IMG1", "prompt": "Replace the logo"})
_, err = client.Multimodal.Job(ctx, "mmj_V01")

// worker package: run a media worker consuming multimodal jobs
w, err := worker.New(worker.Config{Source: jobs, Handler: handle, Client: client, Concurrency: 2})
if err != nil { log.Fatal(err) }
err = w.Run(ctx)

Configuration

Environment variables: EAI_IMAGE_MODEL.

Helm values keys (deploy/helm/enterprise-ai/values.yaml): workers.items[name=image-gen].resources.limits.nvidia.com/gpu, workers.items[name=image-gen].keda.

Security

• Prompt and output moderation; denied subjects are policy rules (POST /v1/policies/evaluate).
• Input images for editing pass F-11 DLP (faces, documents) before use.
• Tenant resolved from the credential, never from the body; PostgreSQL row level security on every tenant table.
• Every mutation and policy decision writes a hash chained audit event (G-11).
• Errors use the standard envelope with message_i18n en and fr.

Observability

• Metrics eai_image_jobs_total, eai_image_job_seconds (histogram), queue depth per worker pool.
• W3C traceparent propagated, X-Request-Id returned, spans carry eai.tenant_id, eai.project, eai.agent_id, eai.run_id.

FinOps units

• usage_type image_generation: images, image_size, image_quality, count; for local models gpu_seconds and energy_wh are also recorded and valued by the allocation policy (H-04).

Failure modes and resilience

• GPU OOM -> retry with lower batch size, then fail the job with a clear error and release the reservation.
• Moderation blocks the output -> job status blocked, no media stored, measured cost still recorded.

Standalone adoption

Usable alone as an internal image service: a marketing tool calls POST /v1/multimodal/images/generate and polls GET /v1/multimodal/jobs/{id}.

Acceptance tests

1. n=2 at 1024x1024 creates two media with provenance and one Usage Event with images=2.
2. A request above the quota returns quota_exceeded before any GPU time is used.

Implementation roadmap