Integrate next
Living candidate queue for new adapters. This is not a commitment and it is not the v0.1.0 product roadmap.
| File | Role |
|---|---|
| This page | Canonical list (git + docs site) |
ROADMAP.md | Product slices (train / eval / one local VTON / agents) |
.cursor/skills/integrate-model/ | How to integrate once you pick a row |
Surveyed: 29 August 2026 · Sources: build.nvidia.com/models, Google virtual-try-on-001, Alibaba OutfitAnyone-Plus, Photoroom Virtual Try-On, CatVTON, Leffa, Nemotron, Cosmos 3.
How to use: pick a next row → follow the integrate-model skill (Path A first-party API, Path B local). After ship, move the row to Shipped and bump the date.
| Status | Meaning |
|---|---|
next | Worth integrating when someone is ready |
watch | Interesting; wait for a clearer API or fashion fit |
blocked | No first-party API / weights / license yet |
skip | Out of scope or we already cover it another way |
NVIDIA access is usually NVIDIA_API_KEY on build.nvidia.com (hosted NIM) and/or a self-hosted NIM container. Prefer the hosted NIM for Path A; HF weights + Diffusers/vLLM for Path B. Do not wrap FLUX / Qwen-Image / Ideogram a second time through NIM — we already have first-party adapters.
OpenTryOn services today: vton · generate · edit · understand · video-generate · bg-remove. Rows marked new service need a new CLI/MCP category before an adapter.
Wave 1 — NVIDIA (fits existing services)
Highest leverage: one NIM provider key unlocks understand + video. Nemotron is not a T2I/VTON family — it is open multimodal understanding / agents. Cosmos is NVIDIA’s generation stack.
| Candidate | Category | Path | Suggested id | Why | Status |
|---|---|---|---|---|---|
| Nemotron 3 Nano Omni 30B-A3B Reasoning | Understand (image + video + audio + text) | A NIM | nemotron-omni | Hosted NIM chat. Path B local weights later. | shipped |
| Nemotron 3.5 Super VL 120B-A12B | Understand (image + video + text, agentic reasoning) | A NIM | nemotron-3.5-super-vl | GA Oct 2026 on build.nvidia.com; no audio (use Omni). | shipped |
| Cosmos 3 Generator (nano, 8B) | Text-to-video, image-to-video | A NIM (POST infer) | cosmos3 | T2V if prompt only; I2V if image set. Optional COSMOS3_INFER_URL. | shipped |
| Cosmos 3 Reasoner (nano) | Understand (physical / video) | A NIM | cosmos3-reasoner | World-model VLM. | shipped |
| Cosmos 3 Nano local weights | T2V / I2V | B Diffusers | cosmos3-local | nvidia/Cosmos3-Nano, Cosmos3OmniPipeline, BF16/CUDA. Shipped Oct 2026. | shipped |
| Nemotron Nano 12B V2 VL | Understand (image + video) | A NIM | — | Predecessor to Omni. Only if Omni is too large. | watch |
| Cosmos 3 Generator super (32B) | T2V / I2V | A NIM (NIM_MODEL_SIZE=super) | cosmos3-super | Same API as nano; heavier GPU. | watch |
| Cosmos Predict 2.5 2B | T2V / I2V / V2V | A NIM | cosmos-predict-2.5 | Older WFM; prefer Cosmos 3 unless Predict-only features are needed. | watch |
| Nemotron 3 Nano / Super / Ultra (text) | Text-only agents | A NIM | — | Coding/planning MoE. No image/video I/O. Planner already uses other LLMs. | skip |
| Nemotron OCR / Parse / page-table | Document OCR | A NIM | — | Useful later for PDP/catalog agents, not invoke-layer media. | watch |
| Nemotron 3.5 Content Safety | Safety classifier | A NIM | — | Guardrail, not a generation/understand tool. | watch |
| Nemotron 3.5 Super VL 120B-A12B | Understand (image + video + text, agentic reasoning) | A NIM | nemotron-3.5-super-vl | Hybrid Mamba–Transformer MoE; 1M context; GA on build.nvidia.com. HF weights for self-host. | shipped |
Nemotron 3 family (context, Aug 2026)
| Model | Role | OpenTryOn fit |
|---|---|---|
| Nemotron 3 Nano 30B-A3B | Text agents, 1M context | Low (no media) |
| Nemotron 3 Super 120B-A12B | Larger text agents | Low |
| Nemotron 3 Ultra 550B-A55B | Flagship text | Low |
| Nemotron 3.5 Lightning 30B-A3B | Fast text agents | Low |
| Nemotron 3 Nano Omni 30B-A3B | Image / video / speech / text | High — Wave 1 |
| Nemotron Voicechat / ASR / Magpie TTS | Speech | tts service now exists (ElevenLabs Eleven v4/v4 Turbo shipped 21 Sep 2026) — re-evaluate as an NVIDIA-hosted alternative/local option |
| Llama-Nemotron embed/rerank VL | RAG embeddings | Out of scope |
Wave 2 — Virtual try-on (fashion / D2C / marketplace)
Compile-only as of 29 Aug 2026 — do not integrate until asked. Developers building fitting rooms, PDP/catalog on-model shots, and marketplace listing tools need dedicated person+garment try-on, not another general I2I compose.
Trust order for Path A: hyperscalers and durable public platforms first (Google, Amazon, Alibaba, BFL, Kuaishou, Meta). Fashion specialists we already ship (FASHN, Pruna) stay. Smaller photo-API vendors are watch unless a customer names them. Do not add Fal / Replicate / PiAPI wrappers of models we already call first-party.
NVIDIA has no dedicated VTON NIM. Product roadmap Slice D is still: pick one local OSS path for v0.1.0.
Already in OpenTryOn (do not re-add)
| Registry id | Vendor | Kind | Notes |
|---|---|---|---|
flux-vto | Black Forest Labs | Dedicated VTON API | First-party FLUX VTO |
google-vton | Google Cloud Vertex | Dedicated VTON API | virtual-try-on-001; ADC + GOOGLE_CLOUD_PROJECT, not GEMINI_API_KEY |
outfitanyone-plus | Alibaba Cloud Model Studio | Dedicated VTON API | aitryon-plus; Beijing-region DASHSCOPE_API_KEY, not Qwen-Image compose |
photoroom-vton / photoroom-virtual-model | Photoroom | Dedicated VTON + catalog API | Image Editing /v2/edit; shopper try-on or flat-lay → on-model |
nova-canvas | Amazon Bedrock | Dedicated VTON API | Garment classes incl. footwear |
kling-ai | Kuaishou (Kling / Kolors) | Dedicated VTON API | First-party Kolors v1 / v1.5 |
fashn-tryon-max / fashn-tryon-v1.6 | FASHN | Dedicated VTON API | Fashion suite; v1.6 is the fast e-comm path |
p-image-tryon | Pruna | Dedicated VTON API | Multi-garment (up to 11 refs) |
segmind | Segmind | Hosted try-on diffusion | Third-party hoster; keep, do not add more hosters |
nano-banana-2-lite | Google Gemini | Composition I2I | Not Vertex virtual-try-on-001 |
qwen-image / qwen-image-local | Alibaba Qwen | Composition I2I | Not OutfitAnyone aitryon-plus |
muse-image | Meta | Composition I2I | Multi-ref edit, not a garment-fit model |
Path A — dedicated VTON APIs to add
Prefer first-party APIs. Related catalog jobs (product→model, model-swap, parsing) are listed only when they are that vendor’s try-on product.
| Candidate | Vendor durability | Task | Suggested id | Why | Status |
|---|---|---|---|---|---|
Google virtual-try-on-001 | Google Cloud (GA 20 Jan 2026; listed discontinue 20 Jan 2027) | Shopper / catalog image VTON | google-vton | Dedicated Vertex / Gemini Enterprise predict API. Person + product image, 1–4 samples, C2PA watermark. Auth is ADC / GCP project, not GEMINI_API_KEY. Distinct from Nano Banana compose. Docs | shipped |
Alibaba OutfitAnyone-Plus (aitryon-plus) | Alibaba Cloud Model Studio | Image VTON + combo top/bottom | outfitanyone-plus | Dedicated DashScope try-on (async). Top, bottoms, dress, face restore, parsing companion aitryon-parsing-v1. Same company as Qwen; Beijing-region key, not the Qwen-Image compose path. Docs | shipped |
| Photoroom Virtual Try-On / Virtual Model | Photoroom (widely used e-comm photo API) | Fitting room or garment→lifestyle model | photoroom-vton / photoroom-virtual-model | API-first catalog/shopper flows. Virtual Model is “flat-lay in, on-model out” (no person photo). Complements dedicated person+SKU VTON. API · Product | shipped |
| Pixelcut Try-On | Pixelcut (e-comm photo; Shopify-heavy) | Image VTON + garment transfer | pixelcut-vton | REST /v1/try-on; upper/lower/full. Smaller than Google/Alibaba. API | watch |
| Fitroom | Specialist startup | Combo top+bottom in one request | fitroom | Strong e-comm DX; weaker long-term vendor signal. | watch |
| Claid | Specialist | Catalog try-on | — | Photo-API vendor; overlap with Photoroom. | watch |
| BytePlus Effects / live AR try-on | ByteDance | Real-time AR, not still VTON | — | SDK/effects stack. Seedream image/edit is already in the registry (seedream). | watch |
| Tencent Cloud FitDiT (hosted) | Tencent | Commercial FitDiT | — | Open weights are NC; Tencent Cloud is the commercial door. Confirm a public REST API before Path A. | watch |
| Adobe Firefly Services | Adobe | General gen/edit | — | Commercially durable Creative Cloud; no dedicated person+garment VTON API found. | skip |
| Shopify / Google Shopping / Walmart Zeekit | Platform lock-in | In-app try-on | — | Not a developer API we can register. | skip |
| Kling via Fal / PiAPI / Replicate | Aggregators | Same Kolors VTON | — | We already have first-party kling-ai. | skip |
| Snap Camera Kit / glasses AR | Snap | Accessory AR | — | Different modality (mesh/AR), not image VTON. | watch |
Related try-on jobs (same developers, not a second vton clone):
| Job | What they call | Prefer |
|---|---|---|
| Shopper fitting room | Person selfie + SKU photo | Google VTO, FASHN, Kling, BFL, Amazon, OutfitAnyone |
| Catalog on-model | Flat-lay → generated model | Photoroom Virtual Model; FASHN product-to-model (vendor suite — do not invent a parallel adapter until asked) |
| Multi-SKU outfit | Top + bottoms one call | OutfitAnyone combo, Fitroom combo, Pruna multi-ref (shipped) |
| Model swap / consistent model | Face/body swap, keep garment | FASHN model-swap (vendor suite) |
| Video try-on | Temporal garment on a clip | CatV2TON local; no durable first-party video-VTON API picked yet |
| Parsing / hotspots | Garment masks, bboxes | Alibaba aitryon-parsing-v1; OpenTryOn already has preprocess helpers |
Path B — local / open-weight VTON
Pick one for v0.1.0 Slice D (tryon.models + opentryon[local]). Many research checkpoints are CC BY-NC-SA — fine for OSS demos, a problem for D2C/marketplace production. Confirm license before making one the default.
| Candidate | Origin | Suggested id | VRAM / notes | License (typical) | Status |
|---|---|---|---|---|---|
| Leffa | CVPR 2025; HF franciszzj/Leffa | leffa | Diffusers; VITON-HD + DressCode try-on + pose transfer; strong detail/logo story | Code MIT; confirm weight card for commercial D2C | shipped |
| CatVTON + CatVTON-FLUX LoRA | ICLR 2025; FLUX.1-Fill LoRA ~37M | catvton | <8GB @ 1024×768; SD 1.5 concatenation pipeline (FLUX LoRA weights exist; official FLUX infer code not released) | CC BY-NC-SA 4.0 (code + checkpoints); FLUX-Fill base has its own terms | shipped |
| IDM-VTON | ECCV 2024; yisol/IDM-VTON | idm-vton | Higher fidelity; ~18–24GB typical | CC BY-NC-SA | watch |
| OOTDiffusion | levihsu/OOTDiffusion | ootdiffusion | Community baseline; setup scripts already under tryon/ | Check repo | watch |
| FitDiT | Tencent-affiliated DiT; BoyuanJiang/FitDiT | fitdit | High garment-detail DiT; ComfyUI exists | CC BY-NC-SA; commercial via Tencent Cloud | watch |
| CatV2TON | Same lab as CatVTON | catv2ton | Video try-on; needs a video-VTON design pass | Check repo (likely NC like CatVTON) | watch |
| FLUX-fill LoRA (train slice) | BFL Fill + brand LoRA | — | Not a third cloud VTON; opentryon train path | FLUX terms | watch |
| OutfitAnyone weights | HumanAIGC / Alibaba paper | — | Demos lock person upload; use aitryon-plus API instead | Restricted demos | skip |
| VITON-HD / StableVITON | 2022–2023 warping/diffusion | — | Superseded for new work | Mixed | skip |
| Qwen-Image-Edit-2511 local | Already qwen-image-local | — | Composition I2I, not a VTON specialist | — | shipped |
By capability (NVIDIA + fashion-relevant others)
Text-to-image
| Candidate | Path | Notes | Status |
|---|---|---|---|
| SANA-Sprint (NVLabs, HF Diffusers) | B | Fast local T2I (1–4 steps). Not a NIM. | shipped (sana-sprint) |
| Cosmos 3 T2I | B / vLLM-Omni | NIM Generator does not expose T2I (one-frame video only in TRT-LLM). | watch |
| FLUX.1-dev / schnell / Kontext via NIM | A | We already have BFL FLUX.2. Do not add a second FLUX stack. | skip |
| FLUX.2 Klein 4B via NIM | A | Distilled FLUX.2; only if BFL does not offer Klein. | watch |
| SD 3.5 Large via NIM | A | Crowded T2I table; low fashion differentiation. | skip |
| Qwen-Image via NIM | A | Already qwen-image (DashScope) + local. | skip |
| NVIDIA Edify (Getty/Shutterstock) | — | NIM preview retired 6 June 2025. | skip |
| Janus-Pro-7B (DeepSeek) | B (HF, single 24GB GPU) | Unified understand + T2I, autoregressive. Only DeepSeek model that generates images; quality trails FLUX.2/Nano Banana/Qwen-Image — OSS option, not a differentiator. Third-party host: Fal if Path A is preferred over local. HF | watch |
Image-to-image / edit
| Candidate | Path | Notes | Status |
|---|---|---|---|
| FLUX.1 Kontext via NIM | A | In-context edit. Skip unless we want NIM as a fallback host. | skip |
| Cosmos Transfer 2.5 2B | A NIM | Video control transfer (edge/depth/seg/vis), not still I2I. See video. | — |
| SANA-Sprint ControlNet | B | Local realtime I2I if we take SANA. | watch |
Virtual try-on
NVIDIA / Nemotron has no VTON NIM. Cloud dedicated VTON is already broad (FLUX VTO, Google Vertex, OutfitAnyone-Plus, Photoroom, Amazon Nova, Kling, FASHN, Pruna). Local weights shipped: Leffa (leffa, MIT code) and CatVTON (catvton, CC BY-NC-SA). Full tables: Wave 2 above.
Text-to-video
| Candidate | Path | Notes | Status |
|---|---|---|---|
| Cosmos 3 Generator nano | A | Wave 1 shipped (cosmos3). Physics-aware; fashion lookbooks are a stretch but the API is clean. | shipped |
| Cosmos Predict 2.5 | A | Legacy WFM. | watch |
| Muse Video | — | Consumer preview only; no Meta Model API / weights. | blocked |
Audio-to-video / talking head
New CLI service (e.g. audio-to-video) — do not stuff this into video-generate without a design pass.
| Candidate | Path | Notes | Status |
|---|---|---|---|
| NVIDIA LipSync NIM | A | Audio → lip-dubbed video. Self-hosted gRPC (no hosted REST). | shipped (nvidia-lipsync on audio-to-video) |
| Cosmos 3 + sound | B / vLLM-Omni | T2V/I2V with synchronized audio. Not on Generator NIM. | watch |
| Pruna P-Video-Avatar | — | Already shipped (p-video-avatar). | shipped |
Text-to-speech
The tts service now exists (shipped 21 Sep 2026 with ElevenLabs) — new candidates go in this table, not a "new CLI service" note.
| Candidate | Path | Suggested id | Notes | Status |
|---|---|---|---|---|
| ElevenLabs Eleven v4 | A — first-party, synchronous REST | eleven-v4 | Most expressive TTS; 90+ languages, 10K char limit. ELEVENLABS_API_KEY. | shipped |
| ElevenLabs Eleven v4 Turbo | A — first-party, synchronous REST | eleven-v4-turbo | ~100ms latency; audio tags ([whispers], [laughs]) for delivery control. Same key. | shipped |
| NVIDIA Magpie TTS / Voicechat | A NIM | — | Re-evaluate as an NVIDIA-hosted alternative or local option now that tts exists. | watch |
| Qwen-TTS (DashScope) | A | — | Same DASHSCOPE_API_KEY family as Qwen3.8/Wan. Only if a customer asks for a second TTS vendor. | watch |
OpenAI TTS (gpt-4o-mini-tts etc.) | A | — | Same OPENAI_API_KEY as GPT Image. Only if asked. | watch |
| Local open-weight TTS (e.g. Kokoro, F5-TTS) | B | — | No local TTS shipped yet; worth a Path B pass if someone wants offline/no-per-request-cost voiceover. | watch |
Image & video understanding / multimodal
| Gemini 4 Argon (gemini-4-argon) | A — GEMINI_API_KEY, understand | Flagship reasoning multimodal (text/image/video/audio/PDF → text); agentic video + long decode. Plan: Gemini 4 Argon integration plan. Keep shipped gemini-3.8-flash for fast Flash work. | next |
| Candidate | Path | Notes | Status |
|---|---|---|---|
Gemini 4 Argon (gemini-4-argon) | A — GEMINI_API_KEY, understand | Flagship reasoning, agentic video, long decode + background Interactions. Plan: gemini-4-argon-integration-plan. | shipped |
| Nemotron 3 Nano Omni | A | Wave 1 shipped. Native audio (unlike current Kimi/Qwen understand tools). | shipped |
| Cosmos 3 Reasoner / Reason2 8B | A | Reasoner shipped (cosmos3-reasoner). Reason2 remains watch. | shipped / watch |
| Muse Glimmer 30B (on NIM) | A | Meta multimodal on NVIDIA’s catalog; we already have first-party Muse Image. Glimmer is understand, not gen. | watch |
| Llama 3.2 11B/90B Vision on NIM | A | Older VLMs; Omni supersedes for new work. | skip |
DeepSeek-V4.1-Flash (deepseek-flash) | A — first-party api-docs.deepseek.com, OpenAI-compatible | Cheapest frontier-class VLM available ($0.15/$0.60 per 1M off-peak). Text+image (no video), 1M context, thinking on by default + reasoning_effort (none/low/high/max). Same shape as kimi-k2.6/qwen3.8-max/glm-5.3-flashx. DEEPSEEK_API_KEY. | shipped |
DeepSeek-VL2 (deepseek-vl2) | B — Transformers + DeepSeek's own deepseek_vl2 package (not on PyPI). No first-party hosted API exists (confirmed: deepseek-flash/deepseek-v4-pro are the only hosted models). | Local counterpart to deepseek-flash, same pairing pattern as qwen3.8-max/qwen3.8. Default deepseek-ai/deepseek-vl2-tiny (single-GPU friendly). | shipped |
DeepSeek-OCR (deepseek-ocr) | B — HF weights only, no first-party hosted endpoint (confirmed) | "Contexts Optical Compression" — real fashion fit: care labels, size tags, SKU/product-copy sheets. Needs flash-attn; distinct value from general captioning even though it shares the understand service. | shipped |
DeepSeek-V4-Pro (deepseek-v4-pro) | A — first-party, text-only (no image input) | Flagship reasoning (R-series folded into V4) but no vision — doesn't fit understand's media-first contract. Better framed as an alternate OPENTRYON_AGENT_LLM_PROVIDER backend for the planner than a registry model, mirroring Nemotron's text-only agents. | skip |
| Kimi-K3 (2.8T MoE) | A — Moonshot | 2.8T total / 104B active MoE, Kimi Delta Attention, 1M context, native multimodal. kimi-k3 is shipped — re-verify Moonshot API model id and caps match this generation; if a distinct endpoint appears, add e.g. kimi-k3-moe. | watch |
| Muse Spark Open | Understand / agents (coding + perception) | Meta — 816B MoE / 42B active; agent- and coding-oriented; 1M context; native audio/video perception; open weights (distinct from muse-image gen). | watch |
| Ling-3.1-flash | Understand / reasoning | InclusionAI — 560B / 25B active hybrid-linear MoE; 1M context. Needs first-party API or open-weight hosting path. | watch |
| Beam | Understand / agents | Reflection — 501B / 23B active frontier MoE; 1M context; Apache-2.0 open weights. Path A hosted chat shipped (beam). Local serve needs four B300s — not registered. | shipped |
| Nemotron 3.5 Super VL | Understand | See Wave 1 row — nemotron-3.5-super-vl on NIM (NVIDIA_API_KEY). | shipped |
3D model generation
New CLI service (e.g. generate-3d). Roadmap lists 3D VTON under Later.
| Candidate | Path | Notes | Status |
|---|---|---|---|
| Microsoft TRELLIS NIM | A | Text-to-3D and image-to-3D meshes; active NVIDIA 3D NIM (Edify 3D is gone). | shipped (trellis on generate-3d) |
| TRELLIS.2 4B | B | Local; Windows-skewed wheels as of 2026 — verify Linux before Path B. | watch |
| Video VTON / 3D VTON | — | Research; no product API picked. | watch |
Out of scope for this list
Biology (AlphaFold, Evo2), CFD, weather, routing, chip sim, protein design — on the NIM catalog, not fashion media. DeepSeek-Prover-V2 (formal theorem proving) and DeepSeek-Coder (code LLM) — no fashion/media surface.
Exception shipped anyway: Limite 1B - Violetto (Paradigma Inc, limite-1b-violetto, Path B local) is a text-only 1B math-reasoning model with the same "no fashion/media surface" profile as DeepSeek-Prover-V2/Coder above — it was integrated on explicit request anyway (29 Sep 2026) as a reference point for a possible future generic model gateway direction for OpenTryOn, not because it fits the current fashion/media roadmap. It's named-model-only: no new planner intent, no capability-default change. If that "generic gateway" direction becomes real, re-evaluate whether text-only reasoning/coding/agent models (Nemotron 3 Nano/Super/Ultra, DeepSeek-Coder, DeepSeek-Prover-V2, etc. — currently skip/out-of-scope above) should move into this backlog properly rather than being one-off exceptions.
Frontier long-context MoE survey — Oct 2026
Public frontier models with ~1M token context and large MoE footprints (industry round-up, Oct 2026). Fashion fit is understand (lookbooks, long PDP copy, video) and planner/agent backends — not VTON/T2I unless noted.
| Candidate | Vendor | Scale (total / active) | Highlights | Path | Suggested id | Status |
|---|---|---|---|---|---|---|
| Kimi-K3 | Moonshot AI | 2.8T / 104B (896 experts, 16 active) | Kimi Delta Attention; native multimodal | A | kimi-k3 (shipped) and kimi-k3.1-pro (+ -highspeed, shipped 24 Oct 2026). Open weights promised by 4 Nov 2026 — add a Path B local entry when they land. | watch |
| Muse Spark Open | Meta | 816B / 42B | Coding + agents; Blackwell Ultra training; audio/video in | A or B | muse-spark-open | watch |
| Ling-3.1-flash | InclusionAI | 560B / 25B | Hybrid-linear MoE reasoning | A (or B if weights land) | ling-3.1-flash | watch |
| Beam | Reflection | 501B / 23B | Open frontier MoE; Apache-2.0 | A (hosted). Local weights need four B300s | beam | shipped |
| Nemotron 3.5 Super VL | NVIDIA | 120B / 12B | Multimodal VLM; Mamba–Transformer hybrid; HF + NIM | A NIM | nemotron-3.5-super-vl | shipped |
Suggested integrate order within this survey: shipped (Path A, beamunderstand, Oct 2026). Remaining: muse-spark-open / ling-3.1-flash when API or weights are documented → re-check kimi-k3 against Moonshot's latest model card. nemotron-3.5-super-vl is already in the registry.
Fashion-domain survey — 13 Oct 2026
Fashion-specific models and APIs found on Hugging Face (model API, sorted by recency and likes, 2026), GitHub topic searches and the web. Licences were read from model cards; benchmark numbers are the authors' own and were not reproduced. "Unverified" means a single secondary source. All six rows in this table have shipped.
Fits today's services
| Candidate | Service / path | Why | Licence / caveats | Status |
|---|---|---|---|---|
| FASHN catalogue endpoints — product-to-model, model-create, model-swap, face-to-model, reframe, edit, image-to-video, background-remove | generate / edit / video-generate / bg-remove, Path A, same FASHN_API_KEY | We only wrapped FASHN's try-on. These cover the catalogue workflow (flat-lay → on-model, swap the model, reframe). | Hosted; credits per generation_mode × resolution. Integrated without live calls. | shipped (fashn-product-to-model, fashn-model-create, fashn-model-swap, fashn-face-to-model, fashn-reframe, fashn-edit, fashn-image-to-video, fashn-bg-remove) |
Fashion embeddings — Marqo-FashionSigLIP, MODA (moda-fashion-distilled, moda-pro-lite), ZooClaw-FashionSigLIP2 | embed, Path B, open_clip / transformers | Catalogue search by text or photo with fashion-tuned encoders. | Apache-2.0 / MIT. MODA distilled is image-only here. Benchmarks vendor-reported. MODA Plus multi-view recipe, Matryoshka and fp16 variants not wired. | shipped (marqo-fashionsiglip, moda-fashion-distilled, moda-pro-lite, zooclaw-fashionsiglip2) |
Fashion Florence (anushreeberlia/fashion-florence) | understand, Path B | Florence-2-large LoRA (0.77B) → JSON tags: category, colour, material, style. Cheap catalogue tagging. | MIT, trust_remote_code. Card: 89.5% category, 42.6% material (exact match); the paper (arXiv 2605.09827) reports 94.6% / 63.0% on a different set. | shipped (fashion-florence) |
prithivMLmods/QIE-2511-Extract-Outfit | edit, Path B | Worn photo → flat-lay mockup (garment extraction) on Qwen-Image-Edit-2511; fits our qwen-image-local setup. | Apache-2.0 LoRA on an Apache-2.0 base. Community LoRA; needs a full-outfit photo. | shipped (qie-2511-extract-outfit) |
xocialize/tryon-FLUX.2-klein-4B-lora | vton, Path B | Try-on LoRA on FLUX.2 klein 4B — commercially clean stack. Inputs: masked person + top + bottom garment. | Apache-2.0 LoRA + Apache-2.0 base. ~273 training tuples, no metrics, weakest on all-dark outfits. | shipped (klein-4b-tryon) |
Genlook tryon-safety-classifier | gating; needs a design choice | Gate garments before rendering them onto a shopper photo (ok / revealing / lingerie / adult + swimwear, subject). 94.6% four-level accuracy (their cross-validation). | Apache-2.0. Fixed-label classifier — does not fit the decide question schema; needs a small classify service or a fixed-label adapter. | shipped (tryon-safety-classifier, new classify service) |
Needs a new service (design pass first)
| Candidate | What it is | Notes | Status |
|---|---|---|---|
segment — garment / human parsing | fashn-ai/fashn-human-parser (SegFormer-B4, 18 classes, 244 MB). | Inherits the NVIDIA SegFormer licence (terms not confirmed — possibly non-commercial); segformer_b2_clothes is also "other". Find a commercial-clean parser first. Needs a mask output type. | watch — queue: new-services-backlog |
measure — body measurements / sizing | Clad API (clad-body-photo, clad-body-questionnaire) + local clad-body-local on datar-psa/clad-body. | New measure service; ISO 8559-1 keys in JSON. CLAD_API_KEY at clad.you/developers. | shipped (Oct 2026) |
detect — e-commerce detection | Trendyol YOLOv9 (84 product classes) and a DINOv2 256-d embedder. | CC BY-SA 4.0 share-alike; paper in review. | watch |
| 3D garments / sewing patterns | pygarment / GarmentCode (MIT); DressWild and GarmentCrafter (2026). | Code / weights for the 2026 papers unverified. Large scope. | watch |
Watch: licence or maturity blockers
| Candidate | Why it matters | Blocker |
|---|---|---|
| OmniTry (shoes, bags, jewellery, accessories try-on) | Only open model for non-clothing wearables. | Apache-2.0 LoRA on FLUX.1-Fill-dev, which is non-commercial. |
| Mobile-VTON (CVPR 2026), CatV2TON (video try-on) | Mobile and video try-on. | CC BY-NC-SA / CC BY-NC-ND. |
fal/flux-klein-9b-virtual-tryon-lora, fal/virtual-tryoff-lora | Most-liked try-on LoRA (156) and its reverse (try-off). | Base FLUX.2 klein 9B is non-commercial (~29 GB VRAM); fal-hosted terms not checked. |
| Garments2Look LoRA (outfit-level try-on with accessories, CVPR 2026) | Multi-garment looks. | Gated, no licence stated; Qwen bases. |
| Qwen-Image-2.1 outfit-swap LoRA, Krea-2 outfit-transfer LoRA | Popular outfit transfer. | Research-licensed base / no licence stated. |
| FitVTON (MIT, fit-aware), ReGra-VTON | Fit-aware try-on. | Empty README / no model card. |
| Eevee (CVPR 2026), LiveVVT | Video try-on research. | Code and weights unconfirmed. |
| Outfit compatibility (Polyvore baselines) | Styling recommendations. | No strong open model; Polyvore is gated. |
| Hosted vendors: The New Black (API + MCP), Claid, Uwear, Lalaland | Product-to-model alternatives. | Evidence is vendor blogs only; prefer first-party FASHN for now. |
Suggested integration order
-
shipped (Path A, 29 Aug 2026).nemotron-omni/cosmos3/cosmos3-reasoner -
shipped (Path A Vertexgoogle-vtonvirtual-try-on-001, 29 Aug 2026). -
shipped (Path A, 29 Aug 2026).outfitanyone-plus/photoroom-vton/photoroom-virtual-model -
shipped (Path B local VTON, 1 Sep 2026).leffa/catvton -
New services LipSync (A2V), TRELLIS (3D)shipped (audio-to-video/generate-3d, Oct 2026). -
SANA-Sprintshipped (sana-sprintongenerate, local Diffusers). -
Optional:
nemotron-omni-localif someone will run 30B-A3B. -
shipped (Path A,deepseek-flashunderstand, 21 Sep 2026). -
shipped (Path B local, 21 Sep 2026). Both confirmed no first-party hosted API exists for either.deepseek-vl2/deepseek-ocr -
shipped (Path A, neweleven-v4/eleven-v4-turbottsservice, 28 Sep 2026). First non-media-generation registry category — newoutput_kind="audio_bytes"intryon/cli/runner.py. No dedicated Studio capability screen yet (tracked separately). -
shipped (Path A Fal,fal-h3-max-relightvideo-generate, 8 Oct 2026). Video + lighting-sphere image in, relit video out; no prompt field. -
shipped (Path A,claude-haiku-5-5understand, 8 Oct 2026; model released 7 Oct 2026). First Anthropic adapter intryon/api/claude. -
shipped (newjev/d1-3b/d1-omni-600mdecideservice, 8 Oct 2026). "System One" decision models: typed questions in, calibrated probabilities out. Jev is API (early access, sign-ups paused 2026-09-22) and text-only; Liquid d1 is Path B local with images (and speech for omni). 13b.shipped (Path B localvideojev-4b/videojev-35b-a3bdecide, Oct 2026). MCG-NJU VideoJev — video grounding + MC QA; no hosted API (Typesafe Jev/THX remain text-only). Native JSON question array, not Jevnoul/choice/scoremaps. -
shipped (newembeddinggemma-2/pplx-embed-v2-late-0.6b/pplx-embed-v2-late-9bembedservice withoutput_kind="embeddings", 8 Oct 2026). Both local-only: Perplexity's hosted Embeddings API serves the v1 dense models, not v2-late. Hosted counterpartgemini-embedding-2shipped 13 Oct 2026 (stable id,GEMINI_API_KEY, text/image/audio/video/PDF,--aggregate). -
pplx-embed-v2-context-9b-preview—watch: text-only contextual preview, weights/interface may change before GA. -
NVIDIA PivotOPD —
watch, not integratable today. It is a training method (on-policy distillation for multi-turn agents, arXiv 2609.40285), not a model: the project page lists code as "coming soon" and releases no checkpoints or API. Nothing to wrap in the registry; revisit if NVIDIA publishes student checkpoints. -
shipped (Path A,nano-banana-2.1generate+edit, 8 Oct 2026). Successor tonano-banana-2, which Google deprecated. -
shipped (Path B local VTON, 8 Oct 2026). Apache-2.0 code + weights; first commercial-friendly local try-on. VRAM not published by FASHN — measure on first GPU run.fashn-vton-1.5 -
shipped (Path A,gemini-3.8-flashunderstand) and 20.shipped (Path A, secondgemini-3.8-flash-tts/gemini-3.8-flash-lite-ttsttsvendor), 8 Oct 2026. Voices endpoint / voice design / replication not wired yet. 19b.shipped (Path A,gemini-4-argonunderstand, Oct 2026): multimodal parity with 3.8 Flash,--video-processing agentic, long decode (max_tokens> 64K + SDK continuation),--backgroundInteractions for text-only jobs. Image/video generation stay on existingnano-banana-*/gemini-omni; video-to-image on Nano Banana 2 / 2.1 / 2 Lite viagenerate --video. 19c.shipped (Path A,gemini-4-argon-cyber-permissiveunderstand, allowlist): Fairwind defensive-cyber twin of Argon.
Next candidates from the 8 Oct 2026 survey (not started): Hy Image 3.5 Preview (TokenHub, generate/edit), local TTS (Tencent AuK, Voxtral, Breeze 2), Lyria 3.5 (needs a music service).
Oct 2026 batch: shipped (Path A video, 2026 REST API). kling-v4 + nemotron-3.5-embed-1b shipped (NIM hosted + local NIM twins). nemotron-parse / flint-local and flint-openrouter shipped (text sage-a1.5-localunderstand, local SGLang/vLLM or OpenRouter). Laplace 3B — watch only (laplace-3b-watch; no public weights/API). GAIA-2 — watch only (gaia-2-watch; Wayve latent-diffusion driving world model, no public weights/API). Laguna XS.2 / M.1 shipped (laguna-xs.2, laguna-xs.2.1, laguna-m.1). Arcee Trinity Large shipped (trinity-large 400B Preview, trinity-large-thinking, trinity-mini, trinity-nano local). Recraft V3 shipped (recraft-v3 generate/edit/bg-remove). Ideogram 4.0 was already ideogram.
-
shipped (Path A,beamunderstand, Oct 2026): ReflectionBeam-1.0-501B-A23B, text chat,REFLECTION_API_KEY. Local four-B300 serve not registered.watch: Muse Spark Open, Ling-3.1-flash, Kimi-K3 2.8T (vs shippedkimi-k3). See Frontier long-context MoE survey. -
FASHN catalogue endpointsshipped (Path A, 13 Oct 2026):fashn-product-to-model,fashn-model-create,fashn-model-swap,fashn-face-to-model,fashn-reframe,fashn-edit,fashn-image-to-video,fashn-bg-removeviatryon.api.fashn.FashnAdapter. -
Fashion embeddingsshipped (Path B, 13 Oct 2026):marqo-fashionsiglip,moda-fashion-distilled,moda-pro-lite,zooclaw-fashionsiglip2in theembedservice. -
shipped (Path B local,fashion-florence/qie-2511-extract-outfitunderstand/edit, 13 Oct 2026). -
shipped (Path B local,klein-4b-tryon/tryon-safety-classifiervton/ newclassifyservice, 13 Oct 2026). -
shipped (newmeasure(Clad body photo / questionnaire / local)measureservice, Oct 2026). Remaining new-service ideas:segment,detect, 3D garments; plus licence-blocked models. -
shipped (6 Nov 2026): default forgemini-omni-1.2-flashgemini-omni, adds--duration3-15s; 1.1 stays selectable via--omni-model(Google routes it to 1.2 from 3 Dec 2026). -
shipped (6 Nov 2026,ripple-65b-openrouter/ripple-65b-localunderstand, text-only): Ricursive Ripple-65B Preview, Apache-2.0. No first-party API exists, so the hosted path is OpenRouter and the local path is a client for your own vLLM/SGLang server. Outside the fashion/media scope, named-model-only. -
Inkling 2 Small (Thinking Machines, 594B MoE / 17B active, text + image in, text out) —
watch. Launched 5 Nov 2026 on Tinker only with closed weights; Apache-2.0 weights promised after safety testing (reports say roughly the week of 9 Nov). Tinker's OpenAI-compatible endpoint is beta and its docs show checkpoint paths as themodelvalue, serverless sampling is waitlisted, and image input over that endpoint is undocumented. Integrate as Path B (local) when weights publish, and a Path A entry once the hosted endpoint is confirmed. Context length is also inconsistent (Tinker 64K/256K vs model card 1M). -
Hima ASR (
Prahaladha/hima-asr, ~0.46B, speech-to-text for 11 Indian languages + English, plus a 17.5M "decision head") —needs design. OpenTryOn has no speech-to-text service; this needs a newtranscribeservice (audio in, text out) and the decision head overlapsdecide. The PyTorch path requirestrust_remote_code, the licence is Apache-2.0 plus the Gemma Terms of Use, and benchmarks are author-reported. Fashion relevance is limited to voice commerce and support calls. -
VFlash (NVIDIA, visual planning for robots and driving) —
watch, not integratable: the project page links no VFlash weights, code or API (its "Code" button points at the unrelated LocateAnything repo). Re-check for a release. -
Decision-model waveshipped (11 Nov 2026,decide):pplx-decider+pplx-decider-local(Perplexity, first-party Decisions API),clef+clef-local(Cloudflare Workers AI),kev-openrouter+kev-local(Jared Palmer; no first-party API, so OpenRouter),autotrust-jev-local(AutoTrust; no hosted API anywhere). Already present:jev,d1-*,d3-*.watch: Kev 9B/27B on OpenRouter, a hosted AutoTrust endpoint, open weights for Decider v1.2. -
Seedream 5.5 (ByteDance, launched 9 Nov 2026) —
watch, pending a first-party API id. It is live in Dreamina, CapCut, Doubao and Jimeng, but the BytePlus ModelArk model list (updated 9 Nov) and Volcano Ark list only 5.0 and older. Add it toSEEDREAM_MODELSand theseedream--model-versionchoices once ModelArk publishes the id. Meanwhileseedreamgained 5.0 Flash and corrected 5.0 ids (11 Nov 2026). -
shipped (11 Nov 2026,lightonocr-3-local/lightonocr-3-vllmparse): LightOnOCR-3 0.8B / 4B, Apache-2.0, local only (no hosted API). The 1B sibling (LightOnOCR-2 architecture) is not wired.
Follow-ups for the new services: Studio capability screens for decide (question builder + probability bars) and embed (upload + similarity), and a vector-store hand-off for embed output.
Shipped (do not re-add)
Invoke-layer highlights already in the registry: FLUX.2 (+ Turbo local), FLUX 3 Image (flux3-image), FLUX 3 Image Klein (flux3-image-klein, flux3-image-klein-local), Nano Banana family, GPT Image (1.5 + ChatGPT Images 2.5 Flare/Sunburst), Muse Image, Ideogram 4.0, P-Image-Ideogram, Qwen-Image API+local, Veo 3.1, Sora (deprecated — API removed 2026-09-24), LTX-2.5, Hailuo 2.3, MiniMax H3 / H3 Max, Fal H3 Max (+ Fal H3 Max Lip Sync, fal-h3-max-lipsync), Wan, Runway Gen-4.5, Nemotron Omni, Cosmos 3 Reasoner, Cosmos 3 Generator (+ Cosmos 3 Nano local, cosmos3-local), Kimi K2.6/K2.7/K3, Qwen3.8 (+ Qwen3.8-Omni-Flash, qwen3.8-omni-flash), Hy4 preview (hy4-preview TokenHub + hy4-preview-local vLLM/SGLang), Reflection Beam (beam, text chat), GLM-5.3-FlashX (glm-5.3-flashx, Zhipu/Z.ai — new tryon.api.zai), Ternary Bonsai 2 27B (ternary-bonsai-2-27b, PrismML local server — new tryon.models.ternary_bonsai), P-Video-2-Pro (p-video-2-pro, Pruna MiniMax H3-based), DeepSeek-V4.1-Flash (deepseek-flash, new tryon.api.deepseek), DeepSeek-VL2 (deepseek-vl2, local — new tryon.models.deepseek_vl2), DeepSeek-OCR (deepseek-ocr, local — new tryon.models.deepseek_ocr), ElevenLabs Eleven v4 / Eleven v4 Turbo (eleven-v4 / eleven-v4-turbo, new tts service + tryon.api.elevenlabs), Claude Haiku 5.5 (claude-haiku-5-5, new tryon.api.claude), H3 Max Relight (fal-h3-max-relight), Typesafe Jev / Liquid d1-3B / d1-omni-600M (new decide service), EmbeddingGemma 2 / pplx-embed-v2-late (new embed service), Limite 1B - Violetto (limite-1b-violetto, Paradigma Inc, local — new tryon.models.limite, out-of-scope exception, see above), BEN2, dedicated cloud VTON (flux-vto, google-vton, outfitanyone-plus, photoroom-vton, photoroom-virtual-model, nova-canvas, kling-ai, FASHN, p-image-tryon, Segmind) plus composition try-on (nano-banana-2-lite, qwen-image, muse-image) and local dedicated VTON (leffa, catvton). Full table: CLI --help / registry.
Maintenance
When adding a row: vendor, modality, Path A/B, proposed registry id, license/API URL, status.
When shipping: move to Shipped, delete the next row, note the registry id.
Re-survey NVIDIA: build.nvidia.com/models + Cosmos / Nemotron blogs. Do not paste the entire NIM catalog.
Re-survey VTON: Vertex Imagen try-on, DashScope OutfitAnyone, Photoroom/Pixelcut, Hugging Face CatVTON / Leffa / IDM-VTON. Prefer first-party APIs over aggregators.