Skip to main content

​
MemFlow Pipeline

MemFlow is a streaming pipeline and autoregressive video diffusion model from Kling. The model is trained using Self-Forcing on Wan2.1 1.3b based on the LongLive training and inference pipeline with the additions of a memory bank to improve long context consistency and using sparse memory activation to maintain generation efficiency.

​
At a Glance

Base ModelWan2.1 1.3B
Estimated VRAM~20GB
TrainingSelf-Forcing
LoRA Support1.3B LoRAs
VACE SupportYes
T2V / V2VYes / Yes
Memory BankYes
The sparse memory activation technique is discussed in the paper, but not implemented right now as mentioned in this issue.

​
Examples

The following examples include timeline JSON files with the prompts used so you can try them as well.

​
Beekeeper

Timeline JSON file

Download timeline

​
Detective

Timeline JSON file

Download timeline

​
Resolution

The generation will be faster for smaller resolutions resulting in smoother video. The visual quality will be better at 832x480 which is the resolution that the model was trained on, but you may need a more powerful GPU in order to achieve a higher FPS.

​
Seed

The seed parameter in the UI can be used to reproduce generations. If you like the generation for a certain seed value and sequence of prompts you can re-use that value later with those same prompts to reproduce the generation.

​
Prompting

Subject and Background/Setting Anchors The model works better if you include a subject (who/what) and background/setting (where) in each prompt. If you want continuity in the next scene then you can continue referencing the same subject and/or background/setting. For example: 
"A 3D animated scene. A **panda** walks along a path towards the camera in a park on a spring day."

"A 3D animated scene. A **panda** halts along a path in a park on a spring day."
Cinematic Long Takes The model works better for scene transitions that involve long cinematic long takes and works less well with rapid shot-by-shot transitions or fast cutscenes. Long, Detailed Prompts The model works better with long, detailed prompts. A helpful technique to extend prompts is to take a base prompt and then ask a LLM chatbot (eg. ChatGPT, Claude, Gemini, etc.) to write a more detailed version. If your base prompt is: 
"A cartoon dog jumping and then running."
Then, the extended prompt could be: 
"A cartoon dog with big expressive eyes and floppy ears suddenly leaps into the frame, tail wagging, and then sprints joyfully toward the camera. Its oversized paws pound playfully on the ground, tongue hanging out in excitement. The animation style is colorful, smooth, and bouncy, with exaggerated motion to emphasize energy and fun. The background blurs slightly with speed lines, giving a lively, comic-style effect as if the dog is about to jump right into the viewer."

​
Offline Generation

A test script can be used for offline generation. If the model weights are not downloaded yet: 
# Run from scope directory
uv run download_models --pipeline memflow
Then: 
# Run from scope directory
uv run -m scope.core.pipelines.memflow.test
This will create an output.mp4 file in the memflow directory.

​
See Also

​
Other Pipelines

StreamDiffusion V2

Real-time streaming from the original StreamDiffusion creators

LongLive

Smooth prompt transitions and extended generation from Nvidia

Krea Realtime

14B model for highest quality generation

RewardForcing

Reward-matched training for improved output quality