Mastering Wan 2.2 Local AI Video: The Dual-Expert & Lightning LoRA ComfyUI Workflow


If you have tried generating AI video locally in the last few months, you already know the two things that break the experience: it is slow, and it runs out of VRAM. The workflow below is the community's answer to both. It combines Wan 2.2's dual-expert (MoE) architecture with distilled Lightning LoRAs to bring a 5-second clip down from a multi-minute, 20-step grind to roughly four sampling steps per expert - on hardware most creators actually own.

This is a full walkthrough: what the architecture is doing, the exact ComfyUI node graph, the parameters that matter, a prompt structure that works, and the optimization ladder for 8 GB, 12 GB, and 24 GB cards.

Why the Wan 2.2 Workflow Matters



Wan 2.2 is Alibaba's open-weight video generation family, released under Apache 2.0 - which is the first reason it took over local video pipelines. You can use it commercially without a license negotiation, and it ships in variants that target radically different hardware budgets.

The second reason is architectural. Wan 2.2's larger variants use a Mixture-of-Experts diffusion design split across the denoising timeline, and this is the part most tutorials gloss over. Instead of one model handling the entire denoise, there are two:

  • The high-noise expert runs during the early steps. This is where composition, subject placement, and overall motion trajectory get decided. Think of it as blocking out the shot.
  • The low-noise expert takes over for the later steps and resolves detail, texture, faces, and temporal stability - the polish pass.

Because only one expert is active at any point in the schedule, you get the quality behavior of a much larger model while keeping the active parameter count per step manageable. The practical consequence for a ComfyUI user: your graph has two model loaders and two samplers chained together, not one. That structure confuses people coming from image workflows, and it is the single most common reason a Wan 2.2 graph produces mush.

Where Lightning LoRAs Come In

Base Wan 2.2 sampling wants somewhere in the region of 20-30 steps with classifier-free guidance active, which means two model evaluations per step. That is the slow path.

Lightning LoRAs are step-distillation adapters - the same family of idea as LCM, Turbo, and Hyper adapters on the image side. They compress the sampling trajectory so the model reaches a converged result in roughly 4 steps instead of 20+, and they let you drop CFG to 1.0, which removes the negative-prompt pass entirely. Stack those two effects and you are looking at something in the range of a 10x reduction in raw compute per clip.

The catch, and you should design around it: you need a separate Lightning LoRA for each expert. A high-noise LoRA applied to the low-noise model is a classic silent failure - no error, just smeared, low-detail output.

What You Need Before You Start

  • ComfyUI, updated. Wan 2.2 support is native in current builds. Update via ComfyUI Manager or git pull in the ComfyUI directory, then restart. Old builds will not have the required video nodes.
  • The Wan 2.2 diffusion models. For the MoE workflow you need both experts. For a consumer GPU, grab the FP8 scaled versions - they roughly halve VRAM against FP16 with minimal quality cost. Place them in ComfyUI/models/diffusion_models/.
  • The text encoder (UMT5-XXL, FP8 scaled recommended) in ComfyUI/models/text_encoders/.
  • The Wan VAE in ComfyUI/models/vae/. Note that the 5B variant uses a different, higher-compression VAE than the 14B models - do not mix them.
  • Lightning LoRAs, one per expert, in ComfyUI/models/loras/.

Filenames change between releases, so match the expert and the LoRA by their high_noise / low_noise designation rather than by memorizing a specific filename.

Picking Your Variant First

Choose the model tier before you build the graph, because it determines whether you need the two-expert structure at all.

  • The 5B hybrid variant (TI2V) - a single model that handles both text-to-video and image-to-video. It is designed for consumer cards and runs comfortably in the 8-12 GB range. It is not a two-expert model, so you use one loader and one sampler. Start here if you are on 8 GB.
  • The 14B variants (T2V and I2V) - these are the dual-expert models this guide focuses on. Noticeably better motion coherence and prompt adherence. Realistic on 12 GB with offloading, comfortable on 16-24 GB.


Building the Workflow: Step by Step

The fastest start is a template rather than a blank canvas. In ComfyUI, open Workflow → Browse Templates and pick a Wan 2.2 video template - a current build ships them, and they arrive pre-wired with the correct node topology. If any model is missing, ComfyUI will prompt you to download it. Below is what each part is doing, so you can fix it when it breaks.

Step 1: Load Both Experts

Add two Load Diffusion Model nodes (also labeled UNETLoader).

  • Node A → the high-noise model file.
  • Node B → the low-noise model file.
  • Set weight_dtype to fp8_e4m3fn on both if you are VRAM-constrained. Leave it at default if you have 24 GB and want maximum fidelity.

Label these nodes in the graph. Once you add LoRAs and two samplers, mixing up the branches becomes very easy and the failure is silent.

Step 2: Attach the Matching Lightning LoRAs

Add two LoraLoaderModelOnly nodes, one on each branch.

  • High-noise model → high-noise Lightning LoRA.
  • Low-noise model → low-noise Lightning LoRA.
  • Start both at strength_model: 1.0.

Use the model-only loader variant, not the standard one that also patches CLIP - you are conditioning text separately and only want the UNet patched.

Tuning note: if motion feels stiff or overly damped, lower the high-noise LoRA to around 0.7-0.8. Motion is decided early in the schedule, so that is the lever. If fine detail looks waxy, adjust the low-noise side instead. Change one at a time.

Step 3: Text Encoding

  • Add Load CLIP, point it at the UMT5-XXL encoder, and set type to wan.
  • Add two CLIP Text Encode nodes - positive and negative.
  • With Lightning LoRAs at CFG 1.0 the negative prompt is not evaluated. Leave it connected but empty. Do not spend time writing an elaborate negative and then wonder why it has no effect.

Step 4: Latent Setup - T2V vs I2V

For text-to-video: add an EmptyHunyuanLatentVideo node (Wan uses this latent format in current ComfyUI builds). Set width, height, and length in frames. A 5-second clip at 16 fps is 81 frames - the familiar 4n+1 pattern.

For image-to-video: use WanImageToVideo instead. Feed it your Load Image output, the VAE, and your positive and negative conditioning. It emits conditioning plus the initial latent. This node is what makes the start frame actually govern the clip rather than acting as a loose style hint.

Resolution guidance: the 14B models are trained around 480p and 720p. Generate at native training resolution and upscale afterward. Pushing straight to 1080p in the sampler produces duplicated limbs and looping artifacts, and costs far more VRAM than upscaling later.

Step 5: The Two-Stage Sampler Chain

This is the core of the workflow. You need two KSampler (Advanced) nodes so you can split the denoise schedule between the experts.

Sampler 1 - high-noise expert:

  • model: high-noise branch (after its LoRA)
  • steps: 4
  • cfg: 1.0
  • sampler_name: euler
  • scheduler: simple
  • start_at_step: 0
  • end_at_step: 2
  • add_noise: enable
  • return_with_leftover_noise: enable - this is mandatory. It hands an unfinished latent to the second stage.

Sampler 2 - low-noise expert:

  • model: low-noise branch (after its LoRA)
  • latent_image: output of Sampler 1
  • steps: 4 (must match Sampler 1 - this defines the shared schedule)
  • cfg: 1.0
  • same sampler and scheduler as stage 1
  • start_at_step: 2
  • end_at_step: 4 (or higher, to run to completion)
  • add_noise: disable
  • return_with_leftover_noise: disable

The mental model: both samplers share one 4-step schedule. The first runs the front half, the second finishes the back half. The handoff step is your main quality dial - shifting it later (3 instead of 2) gives the high-noise expert more authority over motion; earlier gives the low-noise expert more room for detail.

Three failure modes to check first when output is wrong:

  • steps differs between the two samplers → broken, discontinuous schedule.
  • return_with_leftover_noise disabled on stage 1 → stage 2 receives a finished latent and has nothing to do.
  • add_noise enabled on stage 2 → re-noised latent, washed-out result.

Step 6: Decode and Save

  • VAE Decode → feed the final latent and the Wan VAE.
  • Then either Create VideoSave Video (current native nodes), or a VideoCombine node if you use the widely installed VideoHelperSuite.
  • Set fps to 16 for the 14B models and 24 for the 5B variant. Mismatched fps is why some clips play at the wrong speed.


Prompt Structure That Actually Works

Wan responds to camera and motion language, not to image-generation tag soup. Comma-separated keyword lists underperform badly here. Write a shot description, in this order:

[Shot type + subject] + [subject action, one clear verb] + [camera movement] + [environment and lighting] + [style or film stock]

A working example:

Medium tracking shot of a woman in a red raincoat walking through a neon-lit Tokyo alley at night. She turns her head toward the camera and smiles. The camera dollies backward smoothly, maintaining distance. Rain-slicked pavement reflects pink and cyan signage. Shallow depth of field, cinematic 35mm film look.

Guidelines that hold up in practice:

  • One primary action per clip. Chaining "she walks, then sits, then opens a book" into 5 seconds produces incoherent motion. Generate separate clips and cut them together.
  • Name the camera move explicitly - dolly in, pan left, static locked-off shot, handheld follow. Wan understands this vocabulary. If you want no camera motion, say "static camera" or you will often get drift.
  • Describe lighting direction, not just mood. "Backlit by a low sun" beats "beautiful lighting."
  • Skip weights and quality spam. (masterpiece:1.4) and "best quality, 8k" do nothing useful here, and at CFG 1.0 the negative prompt is inert anyway.
  • For image-to-video, describe only what changes. The start frame already establishes appearance and composition - spend the prompt budget on motion.

Optimization: Speed, VRAM, and Upscaling

VRAM Tuning by Card

  • 8 GB - use the 5B variant, 480p, 49-81 frames. Skip the dual-expert setup. Add --lowvram to your ComfyUI launch arguments.
  • 12 GB - 14B FP8 dual-expert is reachable at 480p with block swapping enabled (a WanVideoWrapper feature that moves transformer blocks to system RAM between evaluations). Expect slower generation in exchange for fitting.
  • 16 GB - 14B FP8 at 480p runs cleanly; 720p is feasible with shorter clips.
  • 24 GB - 14B at 720p, 81 frames, comfortably. This is the sweet spot for the full workflow.

Additional levers worth knowing:

  • GGUF quantized models (via ComfyUI-GGUF) let you trade precision for footprint in finer increments than FP8 - Q8 is near-lossless, Q4 fits far smaller cards with visible quality loss.
  • Sage Attention / Flash Attention deliver meaningful speedups on supported hardware. Sage Attention in particular has become a near-default in the Wan community.
  • Torch compile adds a one-time compilation cost per session and pays back across repeated generations - worth it for batch work, not for a single test clip.
  • Close everything else using the GPU. Browser hardware acceleration alone can hold a gigabyte hostage.

The Upscaling Strategy

Generate low, finish high. The reliable path:

  • Sample at 480p or 720p native.
  • Decode, then run a video-aware upscale. A latent upscale plus a short low-denoise second pass preserves temporal consistency better than a naive per-frame image upscaler, which introduces flicker because each frame is enhanced independently.
  • Optionally apply frame interpolation (RIFE or FILM, available as ComfyUI nodes) to lift 16 fps to 32 or 48 fps. This is often a bigger perceived quality gain than raw resolution - motion reads as smooth rather than choppy.

Interpolating from 16 to 32 fps costs a fraction of what generating twice the frames would, and it is the cheapest quality win in the entire pipeline.

Iteration Discipline

  • Test at low frame counts. Dial in prompt and seed at 33 frames, then raise to 81 once the shot works. Do not debug prompts at full length.
  • Lock your seed when comparing parameters, or you cannot attribute a change to anything.
  • Change one variable at a time - LoRA strength, handoff step, or prompt. Not all three.

Conclusion

The reason this workflow spread so quickly is not that it produces the best video available - hosted commercial models still hold an edge on raw fidelity. It spread because it moved local video generation from "start it and come back later" into an iterative creative loop. When a clip takes tens of seconds instead of many minutes, you can actually explore: try eight prompt variations, compare handoff steps, find the seed that works.

Set it up in this order: get a single 5B text-to-video clip rendering end to end, then move to the 14B dual-expert graph, then add Lightning LoRAs, then layer on interpolation and upscaling. Each stage gives you a working baseline to fall back to when the next one breaks - and with a two-branch graph, something will break. Being able to isolate which branch is the whole skill.

The architecture is the real takeaway. Split experts across the denoise schedule plus step distillation is a pattern you will keep seeing in open video models, so learning to read and repair this graph pays off well beyond this one release.

📥 Download Wan 2.2 ComfyUI Workflow: You can grab the official community workflow JSON template from [Civitai] .

Post a Comment

0 Comments