Skip to main content
Be careful not to use too many words. CLIP (Contrastive Language-Image Pre-training) maps text to latent space and will cut off your prompt after a certain number of characters.Focus on important nouns and adjectives. Avoid lengthy sentence structures. Keep it direct and to the point.
Be mindful of how each step balances the image input against the text prompt. Experiment with strategies that emphasize prompt adherence in early steps and become more abstract in later ones—or reverse the approach depending on your goal.
Seeds don’t produce the same results across different machines, so avoid “locking in” a single look. Instead, ensure your prompt works well across a wide range of seeds.
Pay attention to the texture of your image input. Adding a small amount of noise can encourage the model to generate richer detail, but too much noise may result in a muddy or unclear output.
A useful approach is to initialize all ControlNets during stream creation with conditioning_scale set to 0, and include every ControlNet in subsequent updates.This ensures the number of ControlNets remains unchanged between updates, allowing only the relevant conditioning_scale values to be adjusted. Setting conditioning_scale to 0 effectively disables that ControlNet without triggering pipeline reloads.Example for SDXL:
{
  "pipeline": "streamdiffusion",
  "params": {
    "model_id": "stabilityai/sdxl-turbo",
    "prompt": "your prompt",
    "controlnets": [
      {
        "enabled": true,
        "model_id": "xinsir/controlnet-depth-sdxl-1.0",
        "preprocessor": "depth_tensorrt",
        "conditioning_scale": 0.5
      },
      {
        "enabled": true,
        "model_id": "xinsir/controlnet-canny-sdxl-1.0",
        "preprocessor": "canny",
        "conditioning_scale": 0
      },
      {
        "enabled": true,
        "model_id": "xinsir/controlnet-tile-sdxl-1.0",
        "preprocessor": "feedback",
        "conditioning_scale": 0
      }
    ]
  }
}
When working with multiple prompts, you can blend them for smoother, more continuous transitions.Instead of switching abruptly:
prompt: "A beautiful tree"
prompt: "Stands by the river"
Merge the prompts with weights:
{
  "prompt": [["A beautiful tree", 0.6], ["Stands by the river", 0.4]]
}
Each value represents the weight of its corresponding prompt.
Use a sliding window technique for sequential prompting by selecting a subset of recent context.Example:
  • Transcript 1: A beautiful → Queue: A beautiful
  • Transcript 2: scene with → Queue: A beautiful scene with
  • Transcript 3: mountains → Queue: A beautiful scene with mountains
  • Transcript 4: and lakes → Queue: scene with mountains and lakes (removed “A beautiful”)
  • Transcript 5: by the river → Queue: mountains and lakes by the river (removed “scene with”)
To hide controls on the Livepeer player for playback, append &controls=false to the playback URL:
https://lvpr.tv/?v=<playback_id>&controls=false