Skip to main content

​
Scripting API

The Daydream Unity plugin exposes a C# API for scripting, automation, and building custom interactions.

​
Accessing the Component

// Get the Daydream component from a Camera
var daydream = GetComponent<Daydream>();

​
State Machine

The component follows a simple state machine:

​
States

StateDescription
IdleComponent created, not yet started
CreatingStream endpoint being created via API
ConnectingWebRTC handshake in progress
StreamingVideo streaming and AI processing active
ReconnectingAuto-reconnecting after disconnect
ErrorConnection failed
var daydream = GetComponent<Daydream>();

// Check current state
if (daydream.State == DaydreamState.Streaming)
{
    Debug.Log("AI processing is active!");
}

​
Output Texture

Access the AI-transformed video as a Texture for use on any surface: 
var daydream = GetComponent<Daydream>();

// Get the AI output texture
Texture aiOutput = daydream.OutputTexture;

// Apply to a material
GetComponent<Renderer>().material.mainTexture = aiOutput;
Set showOverlay = false to disable the built-in fullscreen overlay when you want to render the AI output on custom surfaces like 3D objects or UI elements.

​
Changing Prompts at Runtime

var daydream = GetComponent<Daydream>();

// Change the prompt — updates are sent to the server automatically
daydream.SetPrompt("cyberpunk cityscape, neon lights");
Parameters are synced to the server every 100ms via debounced PATCH requests. You can also modify any public field directly: 
var daydream = GetComponent<Daydream>();

// Adjust generation parameters
daydream.guidanceScale = 2.0f;
daydream.delta = 0.5f;
daydream.seed = 123;

// Toggle ControlNets
daydream.controlnets[0].conditioningScale = 0.6f; // Depth
daydream.controlnets[1].conditioningScale = 0.3f; // Canny

// Toggle display
daydream.showOverlay = false;
daydream.showOriginalPIP = false;

​
Inspector Fields

All fields on the Daydream component are public and can be modified from scripts:
FieldTypeDescription
apiUrlstringServer endpoint
apiKeystringAuthentication token
modelIdstringDiffusion model ID
resolutionintOutput size (384-1024)
promptstringText prompt
negativePromptstringNegative prompt
guidanceScalefloatPrompt adherence (0.1-20.0)
deltafloatDiffusion strength (0.0-1.0)
seedintRNG seed (-1 for random)
numInferenceStepsintDiffusion steps (1-100)
controlnetsControlNetConfig[]ControlNet configurations
ipAdapterIPAdapterConfigIP Adapter configuration
showOverlayboolShow fullscreen overlay
showOriginalPIPboolShow PIP camera view
pipSizefloatPIP size ratio (0.15-0.4)

​
Types

​
ControlNetConfig

[Serializable]
public class ControlNetConfig
{
    public bool enabled;
    public string modelId;
    public float conditioningScale;    // 0.0-1.0
    public string preprocessor;        // "depth", "canny", "tile", etc.
    public float controlGuidanceStart; // 0.0-1.0
    public float controlGuidanceEnd;   // 0.0-1.0
}

​
IPAdapterConfig

[Serializable]
public class IPAdapterConfig
{
    public bool enabled;
    public float scale;
    public string type;       // "regular" or "faceid"
    public string weightType; // "linear", "style transfer", etc.
}

​
PromptEntry / SeedEntry

[Serializable]
public class PromptEntry
{
    public string text;
    public float weight;
}

[Serializable]
public class SeedEntry
{
    public int seed;
    public float weight;
}

​
Example: Audio-Reactive Effects

Modulate AI parameters based on audio analysis: 
using UnityEngine;

public class AudioReactiveDaydream : MonoBehaviour
{
    public Daydream daydream;
    public AudioSource audioSource;

    private float[] spectrum = new float[256];

    void Update()
    {
        audioSource.GetSpectrumData(spectrum, 0, FFTWindow.Rectangular);

        // Map bass frequencies to guidance scale
        float bass = 0f;
        for (int i = 0; i < 8; i++) bass += spectrum[i];
        daydream.guidanceScale = Mathf.Lerp(1.0f, 3.0f, bass * 10f);

        // Map mid frequencies to delta
        float mids = 0f;
        for (int i = 8; i < 64; i++) mids += spectrum[i];
        daydream.delta = Mathf.Lerp(0.3f, 0.8f, mids * 20f);
    }
}

​
Example: Prompt Cycling

Cycle through prompts on a timer: 
using UnityEngine;

public class PromptCycler : MonoBehaviour
{
    public Daydream daydream;
    public float interval = 10f;
    public string[] prompts = {
        "anime style, vibrant colors",
        "oil painting, impressionist",
        "cyberpunk, neon lights",
        "watercolor, soft pastels"
    };

    private int index;
    private float timer;

    void Update()
    {
        timer += Time.deltaTime;
        if (timer >= interval)
        {
            timer = 0f;
            index = (index + 1) % prompts.Length;
            daydream.SetPrompt(prompts[index]);
        }
    }
}

​
Example: Render to 3D Surface

Apply AI output to a 3D object instead of the screen overlay: 
using UnityEngine;

public class DaydreamOnMesh : MonoBehaviour
{
    public Daydream daydream;
    public Renderer targetRenderer;

    void Start()
    {
        // Disable the built-in overlay
        daydream.showOverlay = false;
    }

    void Update()
    {
        if (daydream.OutputTexture != null)
        {
            targetRenderer.material.mainTexture = daydream.OutputTexture;
        }
    }
}

​
Next Steps