Clef-omni

Cloudflare Clef-omni: one decision API for text, images, audio, and video

Cloudflare has added Clef-omni to its open-weight Clef decision-model family. The release accepts text, images, WAV or MP3 audio, and MP4 or WebM video in one request. Rather than generating long prose, the model scores fields against a defined schema. That makes it relevant to teams building classification, moderation, routing, or automated inspection workflows on Workers AI.

What changes in a multimodal workflow

A conventional pipeline may extract frames from video, transcribe speech, classify images, then combine the results with text. Cloudflare says Clef-omni can assess the modalities jointly in one pipeline. Its example is equipment inspection: a serial-label photograph, operating audio, and fan video can answer separate structured questions about label visibility, abnormal sounds, and fan activity. This reduces integration steps, but it does not remove the need to validate each answer class.

Published architecture and performance details

• The foundation is Qwen3-Omni-30B-A3B-Instruct, a mixture-of-experts model. Cloudflare keeps the comprehension backbone and removes text-to-speech output components.
• Training uses a frozen backbone, LoRA adapters, label-smoothed cross-entropy, and Brier-score calibration.
• Clef models are not text-generating LLMs. Cloudflare describes a prefill pass over the payload followed by schema-constrained scoring of valid options.
• Reported median latency is about 130 ms for text, 150 ms for images, and a few hundred milliseconds for audio. A 21-second video with sound is reported at about 1.5 seconds in one API call.
• Weights are available on Hugging Face; the hosted Workers AI model ID is @cf/cloudflare/clef-omni.

Pricing, context windows, and serving changes

Cloudflare lists Clef-omni at $0.15 per million input tokens. Clef-flash drops from $0.09 to $0.038 per million input tokens, which Cloudflare says is below Jev. The important trade-off is its hosted context window: Clef-flash moves from 64k to 24k, while the self-hosted weights are described as supporting 256k. Standard Clef remains at 64k. A workload handling long documents or transcripts should not change models for the lower unit price without measuring how many requests exceed 24k.

Cloudflare also improved hosted Clef serving. Published median latency changed from 262 to 152 ms around 800 tokens, 616 to 305 ms around 3,400 tokens, and 2,721 to 1,635 ms around 16,000 tokens. These are largely serving-layer changes rather than new weights. Cloudflare cites a move to SGLang and contributions scheduled for SGLang 0.5.22.

Read benchmarks carefully

Clef-omni reports 98.2 on BFCL case exact, 94.8 macro-F1 on BANKING77, and 97.7 macro-F1 on CLINC150+OOS. Clef or Clef-flash lead several other rows. In TypeSafe workflow evaluations, Clef leads Clef-omni for invoice processing and security incidents, while Jev leads agent-trace observability. Those figures are useful for shortlisting; they are not an accuracy guarantee for an organization’s data, labels, media quality, or policy.

Practical rollout checklist

1. Define the schema, allowed labels, and ground-truth acceptance rules before integrating the API.
2. Build a test set containing noisy audio, blurred images, shaky video, real languages, and insufficient-evidence cases.
3. Measure per-label accuracy, calibration, p95 latency, input-token cost, and fallback rate.
4. Control sensitive media: transmission authorization, retention, PII redaction, logging, and API-token access.
5. For Clef-flash, route requests over 24k tokens to Clef or another explicit path rather than silently losing context.

Bottom line

Clef-omni can simplify multimodal, schema-bound decisions. It is not a general chatbot and does not eliminate testing or human review. Start with a narrow workflow such as media-backed ticket triage, form inspection, or content moderation; retain confidence thresholds, human escalation, and a fallback path before expanding use.

Governance and monitoring

Production integration should record the model version, schema version, confidence, input modality availability, and the eventual human disposition where applicable. Review disagreement samples regularly; they reveal label ambiguity, degraded media quality, and policy gaps earlier than aggregate accuracy alone. Rate limits, retry behavior, timeout handling, and a safe manual queue should be designed before launch. If a decision can affect a customer, account, security event, or compliance record, define a human escalation rule rather than using model output as the sole authority.

Source

Original reporting: https://blog.cloudflare.com/clef-faster-cheaper-multimodal/.