If you have tried generating AI video locally in the last few months, you already know the two pain points: it is slow, and it eats VRAM. The workflow that took over the ComfyUI community is the direct answer to both. It pairs Wan 2.2's dual-expert (Mixture-of-Experts) architecture with distilled Lightning LoRAs, bringing a 5-second clip down from a 20-step grind to roughly four sampling steps per expert — on hardware most creators actually own.
This is a full walkthrough: what the architecture does, the exact node graph, the parameters that matter, and how to tune it for your GPU.
Why This Workflow Matters
Wan 2.2 is Alibaba's open-weight video family, released under Apache 2.0 — the first reason it dominates local pipelines, since you can use it commercially without a license negotiation. The second reason is architectural: the larger variants split the denoising timeline across two experts.
The high-noise expert runs the early steps, deciding composition, subject placement, and motion trajectory — it blocks out the shot.
The low-noise expert takes the later steps, resolving detail, texture, faces, and temporal stability — the polish pass.
Because only one expert is active at any point in the schedule, you get the behavior of a much larger model while keeping per-step compute manageable. The practical consequence: your graph has two model loaders and two samplers, not one. That structure trips up people coming from image workflows, and it is the number one reason a Wan 2.2 graph produces mush.
Where Lightning LoRAs Come In
Base Wan 2.2 sampling wants 20–30 steps with CFG active (two model evaluations per step). Lightning LoRAs are step-distillation adapters — the same idea as LCM or Turbo on the image side. They compress the trajectory so the model converges in about 4 steps, and let you drop CFG to 1.0, removing the negative-prompt pass entirely. Stack both effects and you approach a 10x reduction in compute per clip.
The catch: you need a separate Lightning LoRA per expert. A high-noise LoRA on the low-noise model is a silent failure — no error, just smeared output.
What You Need First
ComfyUI, updated. Wan 2.2 support is native in current builds. Update via ComfyUI Manager or git pull, then restart.
Both experts (high-noise and low-noise) in ComfyUI/models/diffusion_models/. Use the FP8 scaled versions on consumer GPUs — they roughly halve VRAM versus FP16.
The UMT5-XXL text encoder (FP8 recommended) in ComfyUI/models/text_encoders/.
The Wan VAE in ComfyUI/models/vae/. The 5B variant uses a different VAE than the 14B models — do not mix them.
Lightning LoRAs, one per expert, in ComfyUI/models/loras/.
Pick Your Variant First
5B hybrid (TI2V) — a single model for text-to-video and image-to-video, tuned for 8–12 GB cards. Not a two-expert model, so one loader, one sampler. Start here on 8 GB.
14B (T2V and I2V) — the dual-expert models this guide targets. Better motion and prompt adherence; realistic on 12 GB with offloading, comfortable at 16–24 GB.
Building the Workflow, Step by Step
The fastest start is a template: Workflow > Browse Templates > Wan 2.2. It arrives pre-wired with the correct topology, and ComfyUI will offer to download any missing model. Here is what each stage does so you can repair it when it breaks.
Load both experts. Add two Load Diffusion Model (UNETLoader) nodes — one high-noise, one low-noise. Set weight_dtype to fp8_e4m3fn if VRAM-constrained. Label the nodes; the branches are easy to confuse.
Attach matching Lightning LoRAs. Two LoraLoaderModelOnly nodes — high-noise LoRA on the high-noise branch, low-noise on low-noise. Start both at strength_model: 1.0.
Text encoding. Load CLIP pointed at UMT5-XXL, type wan; then positive and negative CLIP Text Encode. At CFG 1.0 the negative is not evaluated — leave it connected but empty.
Latent setup. For text-to-video use EmptyHunyuanLatentVideo (Wan's latent format in current builds); set width, height, and length. A 5-second clip at 16 fps is 81 frames (the 4n+1 pattern). For image-to-video use WanImageToVideo and feed it your start frame + VAE.
The two-stage sampler chain — the core of the workflow (see below).
Decode and save. VAE Decode with the Wan VAE, then Create Video > Save Video. Set fps to 16 for 14B, 24 for the 5B variant.
The Two-Stage Sampler Chain
Use two KSampler (Advanced) nodes so you can split the schedule between experts.
Sampler 1 — high-noise expert: steps: 4, cfg: 1.0, sampler_name: euler, scheduler: simple, start_at_step: 0, end_at_step: 2, add_noise: enable, and — critically — return_with_leftover_noise: enable so it hands an unfinished latent to stage two.
Sampler 2 — low-noise expert: latent_image = output of Sampler 1, steps: 4 (must match), cfg: 1.0, same sampler/scheduler, start_at_step: 2, end_at_step: 4, add_noise: disable, return_with_leftover_noise: disable.
The mental model: both samplers share one 4-step schedule; the first runs the front half, the second finishes it. The handoff step is your main quality dial — push it later (3) for more motion authority, earlier for more detail.
Three failure modes to check first: mismatched steps between samplers (broken schedule), return_with_leftover_noise off on stage 1 (stage 2 gets a finished latent), and add_noise on for stage 2 (washed-out result).
Prompt Structure That Works
Wan responds to camera and motion language, not tag soup. Write a shot description in this order: [shot type + subject] + [one clear action] + [camera movement] + [environment + lighting] + [style].
Medium tracking shot of a woman in a red raincoat walking through a neon-lit Tokyo alley at night. She turns toward the camera and smiles. The camera dollies backward smoothly. Rain-slicked pavement reflects pink and cyan signage. Shallow depth of field, cinematic 35mm film look.
Keep to one primary action per clip, name the camera move explicitly (say "static camera" if you want none), describe lighting direction, and skip weight syntax like (masterpiece:1.4) — it does nothing here.
Optimization: Speed, VRAM, and Upscaling
8 GB: 5B variant, 480p, 49–81 frames, launch with --lowvram.
12 GB: 14B FP8 dual-expert at 480p with block swapping (WanVideoWrapper moves transformer blocks to system RAM between evaluations).
16 GB: 14B FP8 at 480p cleanly; 720p feasible on shorter clips.
24 GB: 14B at 720p, 81 frames, comfortably — the sweet spot.
Other levers: Sage Attention (near-default in the Wan community) and torch.compile for repeated runs; GGUF quantized models for finer VRAM control than FP8. For finishing, generate low and upscale after, then use frame interpolation (RIFE/FILM) to lift 16 fps to 32 — often a bigger perceived quality gain than raw resolution, at a fraction of the cost.
Conclusion
This workflow spread because it moved local video from "start it and come back later" into an iterative creative loop. Set it up in order: one 5B clip end to end, then the 14B dual-expert graph, then Lightning LoRAs, then interpolation and upscaling — each stage a working baseline to fall back to when the next breaks. And with a two-branch graph, something will break; isolating which branch is the whole skill.
Downloadable Workflow & References
Use the built-in Workflow > Browse Templates > Wan 2.2 template as your starting JSON.
Match the high-noise LoRA to the high-noise expert and low-noise to low-noise.
|
0 Comments