Client storage administration (brick E-13)

French version: storage-administration. API reference: control-plane-v1 section "Client object storage". Brick page: E-13-object-storage.

The platform never imposes a storage backend. The client declares its own bucket or container in a storage profile; the platform discovers what the backend really offers (versioning, Object Lock, legal hold, retention, multipart, presigned URLs, encryption, KMS, storage classes, lifecycle, replication, integrity) and refuses any usage that needs a capability that was not proved. WORM is never simulated.

1. Supported backends and authentication

Provider (provider)	API used	Authentication (auth.method)	Presigned URLs
aws_s3	S3 REST, virtual hosted URLs (https://<bucket>.s3.<region>.amazonaws.com)	sigv4 (access key, optional session token)	SigV4 query string, at most 7 days
ibm_cos	S3 REST	sigv4 with HMAC keys, or ibm_iam: API key exchanged at https://iam.cloud.ibm.com/identity/token for a bearer token, header ibm-service-instance-id from auth.service_instance_id	HMAC keys only (ibm_iam: capability presign false)
seaweedfs, s3_compatible	S3 REST, path style URLs	sigv4	SigV4 query string
azure_blob	Blob REST, version 2023-11-03	shared_key (account key), sas (SAS token), entra_client_secret (Microsoft Entra ID client credentials, auth.tenant_id, auth.client_id), workload_identity (federated token file AZURE_FEDERATED_TOKEN_FILE, platform profiles only)	service SAS (shared key) or user delegation SAS (Entra); none with a SAS credential
gcs	JSON API	service_account (JWT assertion signed with the key, exchanged at https://oauth2.googleapis.com/token)	V4 signed URL (GOOG4-RSA-SHA256), at most 7 days

Every call goes through the outbound guard (eai.security.egress): no vendor SDK, no proxy variable, no redirect, cloud metadata addresses always refused. A private endpoint (on premises SeaweedFS, IBM COS appliance) is reachable only when the operator lists it in EAI_EGRESS_PRIVATE_ALLOWLIST; the tenant restriction egress_allowed_hosts also applies to tenant profiles. Outside dev and test, endpoints, token endpoints and presigned URLs must be https; EAI_STORAGE_CA_BUNDLE adds the CA of an internal endpoint.

2. Store the credential, then create the profile

A profile holds a reference, never a secret: tenant profiles use openbao://<path> (secret written with POST /v1/secrets), platform profiles use env://EAI_CRED_<NAME> (variable injected by the operator, for example from a Kubernetes Secret). The secret value is either a JSON object with the fields of the method, or the plain value of its single field.

Method	Secret value
sigv4	{"access_key_id": "...", "secret_access_key": "...", "session_token": "..."} (session token optional)
ibm_iam	the API key, or {"api_key": "..."}
shared_key	the account key (base64), or {"account_key": "..."}
sas	the SAS token, or {"sas_token": "..."}
entra_client_secret	the client secret, or {"client_secret": "..."}
service_account	the Google service account key file (client_email, private_key, private_key_id, token_uri)
workload_identity	none (credential_ref absent)

curl -X POST <control-plane-url>/v1/secrets -H "Authorization: Bearer $TOKEN" \
  -d '{"path": "storage/audit", "value": "{\"access_key_id\": \"AKIA...\", \"secret_access_key\": \"...\"}"}'
curl -X POST <control-plane-url>/v1/storage/profiles -H "Authorization: Bearer $TOKEN" -d '{
  "name": "audit-evidence", "provider": "aws_s3", "region": "eu-west-3", "bucket": "bank-audit-worm",
  "key_prefix": "eai/", "auth": {"method": "sigv4"}, "credential_ref": "openbao://storage/audit",
  "encryption": {"mode": "sse-kms", "kms_key_id": "arn:aws:kms:eu-west-3:111122223333:key/..."},
  "purposes": ["audit_worm"]}'

A secret field in auth, options or encryption is refused (422 storage_secret_in_profile). Roles: tenant_admin manages the tenant profiles, platform_admin the platform profiles ("scope": "platform", not visible to the tenants).

3. Verify the capabilities

POST /v1/storage/profiles/{id}/verify (tenant_admin, security_officer) asks the backend with real calls and stores each capability with its method and time: supported true (proved), false (proved absent) or null (not determined: refused like an absent one). With probe_write (default), a probe object is written under .eai-probe/ with the configured encryption, storage class and legal hold, read back, compared by sha256 and removed; the probe is skipped on a bucket with a default retention or an immutability policy, since it could not be removed.

Capability	S3 (AWS, IBM COS, SeaweedFS)	Azure Blob	Google Cloud Storage
object_lock (qualified WORM)	GetObjectLockConfiguration Enabled	container x-ms-immutable-storage-with-versioning-enabled (version level immutability)	bucket objectRetention.mode: Enabled or locked retention policy
retention, legal_hold	Object Lock (legal hold set, read and released by the probe)	version level immutability, container policy	retention policy or object retention; temporary hold set and released by the probe
versioning	GetBucketVersioning	implied by version level immutability, else x-ms-version-id of the probe	bucket versioning.enabled
multipart	multipart upload initiated then aborted	Put Block and Put Block List (probe)	resumable session opened then cancelled
presign	presigned GET of an absent key answered 404	SAS GET answered 404	signed GET answered 404
sse, kms	GetBucketEncryption, response headers of the probe	x-ms-request-server-encrypted, encryption scope	default KMS key of the bucket
storage_classes	storage class of the probe	access tier of the probe	default storage class
lifecycle, replication	GetBucketLifecycleConfiguration, GetBucketReplication	management plane only (unknown), account SKU (GRS, GZRS)	lifecycle rules, location type (dual-region, multi-region)
integrity	probe sha256 round trip, server checked SigV4 payload hash and Content-MD5	probe sha256, Content-MD5 per block	probe sha256, server checked md5Hash

A verification older than EAI_STORAGE_CAPABILITY_MAX_AGE_SECONDS (default 1 day) no longer counts: verify again, for example from a scheduled job of the operator. A change of endpoint, bucket, credential, encryption or options clears the capabilities. GET /v1/storage/profiles/{id}/checks keeps every verification for the evidence.

The status is verified, degraded (a capability needed by the purposes or by required_capabilities is missing, listed in missing_capabilities), or failed (error: storage_credential_invalid, storage_auth_failed, storage_unreachable, storage_backend_error, egress_denied).

4. Presigned URLs

POST /v1/storage/profiles/{id}/presign (tenant_admin, developer, agent) {"scope": "read" | "write", "key", "expires_seconds"} returns the URL once. The key must start with one of the presign_prefixes of the profile (403 storage_key_out_of_scope), it is placed under key_prefix; the expiry is bounded by EAI_STORAGE_PRESIGN_MAX_SECONDS (422 storage_presign_expiry_invalid); the capability presign must be verified (409 storage_capability_missing). A read URL cannot write and the reverse (the method is signed). Only the sha256 of the URL, its key, scope and expiry are stored (storage_presigned_grants) and audited (storage.presign.issue).

5. WORM audit exports (AUDI-011)

The signed audit exports (POST /v1/audit/export and the audit_worm worker job) go to the tenant profile with purpose audit_worm, else to the platform profile named by EAI_AUDIT_WORM_STORAGE_PROFILE. The export is refused with 409 storage_capability_missing unless object_lock and retention were verified on that profile; each object is written in compliance mode until retain_until and is accepted only when the backend reports that retention back (502 storage_worm_not_honoured otherwise, the object being removed). The manifest then carries worm.qualified: true, the profile, the reported retention and the version id; audit_exports.worm_qualified is true. Without any profile, the legacy EAI_S3_* store is used and the manifest says worm.qualified: false: it is not qualified immutability.

Adding the purpose audit_worm to a verified profile without Object Lock is refused. To qualify a bucket: AWS and IBM COS, create the bucket with Object Lock; SeaweedFS, create it with x-amz-bucket-object-lock-enabled: true; Azure, enable version level immutability on the container; Google Cloud Storage, enable object retention on the bucket.

6. Policy, audit and isolation

Every action is decided by the policy engine first (storage.profile.create, storage.profile.update, storage.profile.delete, storage.profile.verify, storage.presign.read, storage.presign.write) and audited (storage.profile.*, storage.presign.issue, storage.capability.refused). The tables storage_profiles, storage_capability_checks and storage_presigned_grants have forced row level security: a tenant never sees another tenant's profile (404), nor the platform profiles.

7. Tests and emulators

bash tools/emulators/fetch.sh installs pinned, sha256 verified emulators into /tmp/toolchains: SeaweedFS weed 4.51 (Apache-2.0), Azurite 3.37.0 (MIT, npm, every dependency pinned by tools/emulators/azurite/package-lock.json, telemetry disabled by the tests), fake-gcs-server 1.56.1 (BSD-2-Clause, test tool only, not distributed). The tests services/control-plane/tests/test_storage_*.py start them on free ports and fail with the command above when a binary is missing.

What an emulator cannot prove is tested against a faithful mock and stays to be validated on the real cloud of the client (BLOCKED until a test account exists): AWS S3 itself (the S3 code path runs against SeaweedFS and the AWS signature examples), IBM IAM tokens, Microsoft Entra ID tokens (Azurite checks the token audience, not its signature), Azure version level immutability, Google OAuth2 tokens, the V4 signature checks and the object retention of Google Cloud Storage (fake-gcs-server checks neither).

8. Procedures in the portal

The screen is Data and knowledge > Storage profiles (portal/src/screens/kit-data.tsx), with the tabs Profiles, Objects of a profile and Presigned URL.

1. Store the credential. The profile only holds a reference, so the secret goes first into OpenBao.

   Screen: Security and governance > Secrets, or POST /v1/secrets as in section 2.

   Expected result: the secret path, for example storage/audit, used as openbao://storage/audit.

   If it fails: 403 forbidden means your role cannot write secrets.

2. Create the profile. It declares the bucket or container of the client, its purposes and its encryption.

   Screen: Storage profiles > Profiles > New profile: name, provider, bucket, scope, endpoint, region, key prefix, auth, credential_ref, purposes. API: POST /v1/storage/profiles.

   Expected result: the profile is listed; its capabilities are not verified yet.

   If it fails: 422 storage_secret_in_profile (a secret field in auth, options or encryption), 409 storage_profile_exists (name taken), 422 storage_tls_required (an http endpoint outside dev and test), 403 storage_scope_forbidden (scope platform without platform_admin).

3. Verify the capabilities. Only verified capabilities count, so verify before any use.

   Screen: Profiles, action verify on the profile, tick probe_write to write the probe object. API: POST /v1/storage/profiles/{id}/verify.

   Expected result: status verified; the checks action lists every verification.

   If it fails: status degraded lists missing_capabilities; status failed gives error (see the table below).

4. Issue a presigned URL. A client application then reads or writes one object directly, for a bounded time.

   Screen: Storage profiles > Presigned URL: profile, scope read or write, key, expiry. API: POST /v1/storage/profiles/{id}/presign.

   Expected result: the URL, shown once.

   If it fails: 403 storage_key_out_of_scope, 422 storage_presign_expiry_invalid, 409 storage_capability_missing, 422 storage_prefix_required (the profile has several presign_prefixes and the key starts with none).

5. Disable or delete a profile. A disabled profile refuses every use until it is enabled again.

   Screen: Profiles, action disable, enable or delete.

   Expected result: the new status shows in the list.

   If it fails: 409 storage_profile_disabled when a disabled profile is used; 409 storage_object_protected when an object of an evidence purpose is deleted through the API.

9. Troubleshooting

Status	Code	Cause	Fix
422	storage_secret_in_profile	a secret field in the profile	move the secret to OpenBao, give credential_ref
422	storage_credential_invalid	the secret value does not match the method	write the fields of the method (section 2)
502	storage_auth_failed	the backend refused the credential	rotate the key at the provider, write it again
502	storage_unreachable	endpoint not reachable	check the network and EAI_EGRESS_PRIVATE_ALLOWLIST for a private endpoint
422	storage_tls_required	http endpoint outside dev and test	use https, add the CA with EAI_STORAGE_CA_BUNDLE
409	storage_capability_missing	capability not verified (presign, Object Lock, retention)	enable it on the bucket, verify again
502	storage_worm_not_honoured	the backend did not report the retention	enable Object Lock or immutability (section 5)
403	storage_key_out_of_scope	key outside presign_prefixes	use a key under an allowed prefix
404	storage_profile_not_found	unknown profile or other tenant	check the identifier

10. Next steps

• File security
• API Reference
• Brick page: E-13-object-storage