> ## Documentation Index
> Fetch the complete documentation index at: https://docs.daydream.live/llms.txt
> Use this file to discover all available pages before exploring further.

# Pipeline Architecture

> Technical reference for the Scope pipeline system

# Understanding pipeline architecture

Pipelines are the core abstraction for handling streaming video in Scope. A pipeline encapsulates model loading, inference logic, configuration schemas, and metadata.

***

## Pipeline Definition

### The Pipeline Base Class

All pipelines inherit from the abstract `Pipeline` class:

```python theme={null}
from abc import ABC, abstractmethod

class Pipeline(ABC):
    @classmethod
    def get_config_class(cls) -> type["BasePipelineConfig"]:
        """Return the Pydantic config class for this pipeline."""
        ...

    @abstractmethod
    def __call__(self, **kwargs) -> dict:
        """Process video frames and return results."""
        pass
```

| Method               | Purpose                                                                |
| :------------------- | :--------------------------------------------------------------------- |
| `get_config_class()` | Returns the Pydantic config class that defines parameters and metadata |
| `__call__()`         | Processes input frames and returns generated video                     |

***

## Configuration Schema

Every pipeline defines a Pydantic configuration class that inherits from `BasePipelineConfig`. This class serves as the **single source of truth** for:

1. Pipeline metadata (ID, name, description, version)
2. Feature flags (LoRA support, VACE support, quantization)
3. Parameter definitions with validation constraints
4. UI rendering hints for the frontend

### Example Configuration

```python theme={null}
from pydantic import Field
from scope.core.pipelines.base_schema import BasePipelineConfig, ModeDefaults, ui_field_config

class LongLiveConfig(BasePipelineConfig):
    # Metadata (class variables)
    pipeline_id = "longlive"
    pipeline_name = "LongLive"
    pipeline_description = "A streaming autoregressive video diffusion model..."
    estimated_vram_gb = 20.0
    supports_lora = True
    supports_vace = True

    # Parameters (instance fields with validation)
    height: int = Field(
        default=320,
        ge=1,
        description="Output height in pixels",
        json_schema_extra=ui_field_config(
            order=4, component="resolution", is_load_param=True
        ),
    )

    noise_scale: float = Field(
        default=0.7,
        ge=0.0,
        le=1.0,
        description="Amount of noise to add during video generation",
        json_schema_extra=ui_field_config(
            order=7, component="noise", modes=["video"], is_load_param=True
        ),
    )
```

***

## Pipeline Metadata

Configuration classes declare metadata as class variables:

| Metadata               | Type             | Description                                |
| :--------------------- | :--------------- | :----------------------------------------- |
| `pipeline_id`          | `str`            | Unique identifier used for registry lookup |
| `pipeline_name`        | `str`            | Human-readable display name                |
| `pipeline_description` | `str`            | Description of capabilities                |
| `pipeline_version`     | `str`            | Semantic version string                    |
| `docs_url`             | `str \| None`    | Link to pipeline documentation             |
| `estimated_vram_gb`    | `float \| None`  | Estimated VRAM requirement in GB           |
| `artifacts`            | `list[Artifact]` | Model files required by the pipeline       |

### Feature Flags

Feature flags control which UI controls are shown:

| Flag                        | Effect                          |
| :-------------------------- | :------------------------------ |
| `supports_lora`             | Enables LoRA management UI      |
| `supports_vace`             | Enables VACE reference image UI |
| `supports_cache_management` | Enables cache controls          |
| `supports_quantization`     | Enables quantization selector   |
| `supports_kv_cache_bias`    | Enables KV cache bias slider    |

***

## Artifacts

Artifacts declare model files and resources that a pipeline requires. The system downloads these automatically before the pipeline loads.

### Available Artifact Types

| Type                      | Source          | Attributes                                       |
| :------------------------ | :-------------- | :----------------------------------------------- |
| `HuggingfaceRepoArtifact` | HuggingFace Hub | `repo_id`, `files`                               |
| `GoogleDriveArtifact`     | Google Drive    | `file_id`, `files` (optional), `name` (optional) |

### Example

```python theme={null}
from scope.core.pipelines.artifacts import HuggingfaceRepoArtifact, GoogleDriveArtifact

class MyPipelineConfig(BasePipelineConfig):
    pipeline_id = "my-pipeline"

    artifacts = [
        HuggingfaceRepoArtifact(
            repo_id="depth-anything/Video-Depth-Anything-Small",
            files=["video_depth_anything_vits.pth"],
        ),
        GoogleDriveArtifact(
            file_id="1Smy6gY7BkS_RzCjPCbMEy-TsX8Ma5B0R",
            files=["flownet.pkl"],
            name="RIFE",
        ),
    ]
```

Common artifacts shared across pipelines can be reused:

```python theme={null}
from scope.core.pipelines.common_artifacts import WAN_1_3B_ARTIFACT, VACE_ARTIFACT

class MyPipelineConfig(BasePipelineConfig):
    artifacts = [WAN_1_3B_ARTIFACT, VACE_ARTIFACT]
```

***

## Input Requirements

The `prepare()` method declares input frame requirements before processing. **Pipelines that accept video input must implement it.**

| Return Value                 | Meaning                                                     |
| :--------------------------- | :---------------------------------------------------------- |
| `Requirements(input_size=N)` | Pipeline needs N input frames before `__call__()`           |
| `None`                       | Pipeline operates in text-only mode (no video input needed) |

### Example

```python theme={null}
from scope.core.pipelines.interface import Requirements

class MyPipeline(Pipeline):
    def prepare(self, **kwargs) -> Requirements:
        return Requirements(input_size=4)

    def __call__(self, **kwargs) -> dict:
        video = kwargs.get("video")  # List of 4 frames
        # ... process video ...
```

### Multi-mode Pipeline Example

Pipelines that support both text-to-video and video-to-video modes use the `prepare_for_mode()` helper from `defaults.py`:

```python theme={null}
from scope.core.pipelines.defaults import prepare_for_mode
from scope.core.pipelines.interface import Requirements

class LongLivePipeline(Pipeline):
    def prepare(self, **kwargs) -> Requirements | None:
        """Return input requirements based on current mode."""
        return prepare_for_mode(self.__class__, self.components.config, kwargs)
```

The helper returns `Requirements` when video mode is active (indicated by `video=True` in kwargs) and `None` for text mode. The `input_size` is calculated from the pipeline's configuration.

**Why implement `prepare()`:**

* Without it, the frame processor cannot know how many frames to buffer before calling `__call__()`
* Enables efficient queue management - the processor sizes queues based on requirements
* Allows multi-mode pipelines to dynamically switch between text and video input modes

***

## Mode System

Pipelines can support multiple input modes with different default parameters:

```python theme={null}
class LongLiveConfig(BasePipelineConfig):
    # Base defaults
    height: int = Field(default=320, ...)
    width: int = Field(default=576, ...)

    # Mode-specific overrides
    modes = {
        "text": ModeDefaults(default=True),
        "video": ModeDefaults(
            height=512,
            width=512,
            noise_scale=0.7,
        ),
    }
```

| Mode    | Description                                |
| :------ | :----------------------------------------- |
| `text`  | Text-to-video generation from prompts only |
| `video` | Video-to-video with input conditioning     |

The `default=True` flag marks which mode is selected initially.

***

## Preprocessors and Postprocessors

Pipelines can be declared as preprocessors or postprocessors using the `usage` class variable:

```python theme={null}
from scope.core.pipelines.base_schema import BasePipelineConfig, ModeDefaults, UsageType

class MyPreprocessorConfig(BasePipelineConfig):
    pipeline_id = "my-preprocessor"
    pipeline_name = "My Preprocessor"
    usage = [UsageType.PREPROCESSOR]

    modes = {"video": ModeDefaults(default=True)}
```

| Type                      | Purpose                                  | UI Location            |
| :------------------------ | :--------------------------------------- | :--------------------- |
| `UsageType.PREPROCESSOR`  | Process input video before main pipeline | Preprocessor dropdown  |
| `UsageType.POSTPROCESSOR` | Process output video after main pipeline | Postprocessor dropdown |
| (empty list)              | Standard pipeline                        | Main pipeline selector |

***

## Dynamic UI Rendering

### JSON Schema Generation

Pydantic models automatically generate JSON Schema via `model_json_schema()`. The backend exposes this schema through the `/pipelines` endpoint, which the frontend consumes for dynamic UI rendering.

The schema includes:

* Field types and validation constraints (`minimum`, `maximum`, `enum`)
* Default values
* Descriptions (used as tooltips)
* Custom UI metadata (via `json_schema_extra`)

### Schema-to-UI Flow

```
Backend (Python)                    Frontend (React)
─────────────────                   ─────────────────
BasePipelineConfig
       │
       ▼
model_json_schema()
       │
       ▼
GET /pipelines  ───────────────────▶  configSchema
                                           │
                                           ▼
                                    parseConfigurationFields()
                                           │
                                           ▼
                                    Field type inference
                                           │
                                    ┌──────┴──────┐
                                    ▼             ▼
                              Primitive      Complex
                              widgets        components
```

### Field Type Inference

For fields without a `component` specified, the frontend automatically renders an appropriate widget based on the JSON Schema type. The frontend (`schemaSettings.ts`) infers widget types from schema properties:

| Schema Property                           | Inferred Type | Widget            |
| :---------------------------------------- | :------------ | :---------------- |
| `type: "boolean"`                         | `toggle`      | Toggle switch     |
| `type: "string"`                          | `text`        | Text input        |
| `type: "number"` with `minimum`/`maximum` | `slider`      | Slider with input |
| `type: "number"` without bounds           | `number`      | Number input      |
| `enum` or `$ref`                          | `enum`        | Select dropdown   |

### Two-Tier Component System

The UI uses a two-tier approach: **primitive fields** render as individual widgets based on inferred type, while **complex components** group related fields into unified UI blocks.

***

## UI Metadata

The `ui_field_config()` helper attaches rendering hints to schema fields:

```python theme={null}
height: int = Field(
    default=320,
    ge=1,
    description="Output height in pixels",
    json_schema_extra=ui_field_config(
        order=4,              # Display order (lower = higher)
        component="resolution", # Complex component grouping
        is_load_param=True,   # Requires pipeline reload
        modes=["video"],      # Only show in video mode
        label="Height",       # Short label
        category="configuration", # Panel placement
    ),
)
```

| Field           | Type        | Description                                                          |
| :-------------- | :---------- | :------------------------------------------------------------------- |
| `order`         | `int`       | Sort order for display (lower values appear first)                   |
| `component`     | `str`       | Groups fields into complex widgets (e.g., "resolution", "noise")     |
| `modes`         | `list[str]` | Restrict visibility to specific input modes                          |
| `is_load_param` | `bool`      | If `True`, field is disabled during streaming                        |
| `label`         | `str`       | Short display label; description becomes tooltip                     |
| `category`      | `str`       | `"configuration"` for Settings panel, `"input"` for Input & Controls |

### Complex Components

Fields with the same `component` value are grouped and rendered together:

| Component         | Fields                            | Rendered As              |
| :---------------- | :-------------------------------- | :----------------------- |
| `resolution`      | `height`, `width`                 | Linked dimension inputs  |
| `noise`           | `noise_scale`, `noise_controller` | Slider + toggle          |
| `vace`            | `vace_context_scale`              | VACE configuration panel |
| `lora`            | `lora_merge_strategy`             | LoRA manager             |
| `denoising_steps` | `denoising_steps`                 | Multi-step slider        |

Fields with the same `component` value are grouped and rendered once.

### Mode-Aware Filtering

Fields specify which modes they appear in via `modes`:

```python theme={null}
noise_scale: float = Field(
    ...,
    json_schema_extra=ui_field_config(modes=["video"]),
)
```

The frontend filters fields based on the current input mode, hiding irrelevant controls.

***

## Load-time vs Runtime Parameters

| Type              | `is_load_param` | Editable During Streaming | Example                        |
| :---------------- | :-------------- | :------------------------ | :----------------------------- |
| Load parameter    | `True`          | No                        | Resolution, quantization, seed |
| Runtime parameter | `False`         | Yes                       | Prompt strength, noise scale   |

Load parameters are passed when the pipeline loads and require a restart to change. Runtime parameters can be adjusted while streaming.

***

## Pipeline Registry

Pipelines register with the central `PipelineRegistry` at startup:

```python theme={null}
from scope.core.pipelines.registry import PipelineRegistry

PipelineRegistry.register(config_class.pipeline_id, pipeline_class)
```

### Built-in vs Node Pipelines

**Built-in pipelines** are registered automatically when the registry module is imported.

**Node pipelines** register through the pluggy hook system:

```python theme={null}
from scope.core.plugins.hookspecs import hookimpl

@hookimpl
def register_pipelines(register):
    from .pipelines import MyCustomPipeline
    register(MyCustomPipeline)
```

### GPU-Based Filtering

Pipelines with `estimated_vram_gb` set are only registered if a compatible GPU is detected. This prevents showing pipelines that cannot run on the current hardware.

***

## See Also

<CardGroup cols={2}>
  <Card title="Developing Nodes" icon="code" href="/scope/guides/node-development">
    Create your own custom pipelines
  </Card>

  <Card title="Node Architecture" icon="puzzle-piece" href="/scope/reference/architecture/nodes">
    Technical details of the node system
  </Card>
</CardGroup>