Understanding node system

The Scope node system enables third-party extensions to provide custom pipelines. This document describes the architectural design and data flows that enable node discovery, installation, and lifecycle management.

Architecture Layers

Desktop App (Electron) → Frontend (React) → Backend (FastAPI) → Nodes

Each layer has distinct responsibilities, communicating through well-defined interfaces.

Key Technologies

pluggy: Python hook system for pipeline registration and discovery
uv: Fast Python package manager for dependency resolution and installation
Electron IPC: Communication bridge between desktop app and frontend

System Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        Desktop App (Electron)                    │
│  ┌─────────────┐    ┌──────────────┐    ┌───────────────────┐  │
│  │ Deep Links  │───▶│  IPC Bridge  │◀──▶│ Python Process    │  │
│  │ File Browse │    │              │    │ Manager           │  │
│  └─────────────┘    └──────────────┘    └───────────────────┘  │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                         Frontend (React)                         │
│  ┌─────────────┐    ┌──────────────┐    ┌───────────────────┐  │
│  │ Settings UI │───▶│  API Client  │───▶│ State Management  │  │
│  └─────────────┘    └──────────────┘    └───────────────────┘  │
└─────────────────────────────────────────────────────────────────┘
                              │ HTTP
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                      Backend (FastAPI)                           │
│  ┌─────────────┐    ┌──────────────┐    ┌───────────────────┐  │
│  │  REST API   │───▶│ Node Manager │───▶│ Pipeline Registry │  │
│  └─────────────┘    └──────────────┘    └───────────────────┘  │
│                              │                                   │
│                     ┌────────┴────────┐                         │
│                     ▼                 ▼                         │
│              ┌────────────┐    ┌────────────┐                   │
│              │ Dependency │    │   Venv     │                   │
│              │ Validator  │    │ Snapshot   │                   │
│              └────────────┘    └────────────┘                   │
└─────────────────────────────────────────────────────────────────┘

Node Discovery

Nodes integrate with Scope through Python’s entry point mechanism:

Nodes declare entry points in their pyproject.toml under [project.entry-points."scope"]
The backend uses pluggy hooks to discover installed nodes at startup
Each node implements a register_pipelines hook to register its pipeline implementations
The Pipeline Registry maintains a mapping of pipeline IDs to their implementations

Node Sources

Source	Description
PyPI	Standard Python packages
Git	Direct repository installation
Local	File system paths (editable mode for development)

Installation Flow

User                Frontend              Backend              Desktop App
 │                    │                     │                      │
 │ Click Install      │                     │                      │
 │───────────────────▶│                     │                      │
 │                    │ POST /nodes       │                      │
 │                    │────────────────────▶│                      │
 │                    │                     │ Validate deps        │
 │                    │                     │────────┐             │
 │                    │                     │◀───────┘             │
 │                    │                     │ Capture venv         │
 │                    │                     │────────┐             │
 │                    │                     │◀───────┘             │
 │                    │                     │ Install via uv       │
 │                    │                     │────────┐             │
 │                    │                     │◀───────┘             │
 │                    │     200 OK          │                      │
 │                    │◀────────────────────│                      │
 │                    │ Request restart     │                      │
 │                    │─────────────────────────────────────────────▶
 │                    │                     │                      │ Respawn
 │                    │                     │                      │────────┐
 │                    │                     │                      │◀───────┘
 │                    │ Poll health         │                      │
 │                    │────────────────────▶│                      │
 │                    │ Refresh pipelines   │                      │
 │                    │────────────────────▶│                      │

Installation steps:

User initiates install (UI or deep link)
Frontend sends install request to backend API
Backend validates dependencies won’t conflict with existing environment
Backend captures current venv state (for rollback)
Backend resolves and installs dependencies via uv
Backend updates node registry
Frontend triggers server restart
Server restarts with new node loaded
Frontend polls until server is healthy
Frontend refreshes pipeline list

Rollback: If installation fails at any step, the venv is restored to its captured state.

Node Update Flow

The update flow has two phases: detection (happens automatically when nodes are listed) and execution (triggered by the user). The execution phase reuses the installation flow with an upgrade: true flag.

Update Detection

Frontend                        Backend
 │                                │
 │ GET /nodes                     │
 │───────────────────────────────▶│
 │                                │ For each node:
 │                                │ _check_node_update()
 │                                │────────┐
 │                                │        │ Compare installed
 │                                │        │ vs latest version
 │                                │◀───────┘
 │  Node list                     │
 │  (with update_available flags) │
 │◀───────────────────────────────│

Step by step:

Frontend fetches the node list via GET /nodes
Backend iterates over installed nodes and calls _check_node_update() for each
For PyPI nodes, the installed version is compared against the latest version on PyPI
For Git nodes, the installed commit hash is compared against the latest commit on the remote
Local nodes are skipped (use Reload instead)
Each node in the response includes an update_available flag
Frontend displays an update badge on nodes where the flag is true

Update Execution

User       Frontend              Backend              Desktop App
 │           │                     │                      │
 │ Click     │                     │                      │
 │ Update    │                     │                      │
 │──────────▶│                     │                      │
 │           │ POST /nodes         │                      │
 │           │ {upgrade: true}     │                      │
 │           │────────────────────▶│                      │
 │           │                     │ Capture venv         │
 │           │                     │────────┐             │
 │           │                     │◀───────┘             │
 │           │                     │ Compile with         │
 │           │                     │ --upgrade-package    │
 │           │                     │────────┐             │
 │           │                     │◀───────┘             │
 │           │                     │ Sync deps            │
 │           │                     │────────┐             │
 │           │                     │◀───────┘             │
 │           │     200 OK          │                      │
 │           │◀────────────────────│                      │
 │           │ Request restart     │                      │
 │           │─────────────────────────────────────────────▶
 │           │                     │                      │ Respawn
 │           │                     │                      │────────┐
 │           │                     │                      │◀───────┘
 │           │ Poll health         │                      │
 │           │────────────────────▶│                      │
 │           │ Refresh pipelines   │                      │
 │           │────────────────────▶│                      │

Step by step:

User clicks the Update button on a node
Frontend sends POST /nodes with the node spec and upgrade: true
Backend captures the current venv state (for rollback)
Backend runs uv pip compile with --upgrade-package targeting only the node package
Backend syncs the environment with the newly resolved dependencies
Backend updates the node registry
Frontend triggers server restart
Server restarts with the updated node loaded
Frontend polls until server is healthy
Frontend refreshes pipeline list

Rollback: If the update fails at any step, the venv is restored to its pre-update state, just like a failed installation.

Source-Specific Update Behavior

Source	Detection Method	Notes
PyPI	Compares installed version against latest version on PyPI	Standard version comparison
Git	Compares installed commit hash against latest remote commit	Detects new commits on the default branch
Local	Skipped	Local nodes use Reload instead

Uninstallation Flow

User initiates uninstall
Backend unloads any active pipelines from the node
Backend removes node from registry
Backend uninstalls package via uv
Frontend triggers server restart
Frontend refreshes pipeline list

Manual Reload Flow

For local/editable nodes, developers can trigger a reload to pick up code changes:

Steps
Sequence Diagram

Developer modifies code
Developer clicks Reload button
Frontend requests server restart
Server restarts with fresh Python module imports
Code changes take effect

In standalone mode (without the desktop app), the reload flow is the same except the server performs a self-restart instead of being respawned by the desktop app (see Standalone Mode below).

Developer      Frontend              Backend              Desktop App
 │                │                     │                      │
 │ Modify code    │                     │                      │
 │────────┐       │                     │                      │
 │◀───────┘       │                     │                      │
 │ Click Reload   │                     │                      │
 │───────────────▶│                     │                      │
 │                │ Request restart     │                      │
 │                │───────────────────────────────────────────▶│
 │                │                     │                      │ Respawn server
 │                │                     │                      │────────┐
 │                │                     │                      │◀───────┘
 │                │ Poll health         │                      │
 │                │────────────────────▶│                      │
 │                │ Refresh pipelines   │                      │
 │                │────────────────────▶│                      │

This triggers a server restart to ensure Python modules are fully reloaded.

Deep Link Installation

External sources can facilitate node installation via protocol URLs:

daydream-scope://install-node?package=<spec>

Source	Raw Spec	Encoded URL
PyPI	`my-node`	`daydream-scope://install-node?package=my-node`
Git	`git+https://github.com/user/repo.git`	`daydream-scope://install-node?package=git%2Bhttps%3A%2F%2Fgithub.com%2Fuser%2Frepo.git`

Flow:

External source opens the deep link URL
Desktop app receives URL via OS protocol handler
If app is starting: stores pending deep link for later processing
Once frontend is loaded: sends action via IPC to renderer
Frontend opens settings with Nodes tab and pre-filled package spec
User confirms installation

Component Responsibilities

Backend Components

Component	Responsibility
Node Manager	Singleton that manages node lifecycle (install, uninstall, update, reload)
Dependency Validator	Pre-validates that new packages won’t break existing environment
Venv Snapshot	Captures and restores environment state for safe rollback
Pipeline Registry	Maps pipeline IDs to their implementations and source nodes
REST API	Exposes node operations to frontend

Frontend Components

Component	Responsibility
Settings Dialog	User interface for node management
API Client	HTTP calls to backend node endpoints
Restart Coordinator	Handles server restart and health polling
Pipeline Context	Refreshes available pipelines after changes

Desktop App Components

Component	Responsibility
Python Process Manager	Spawns/respawns backend server, handles restart signals
Deep Link Handler	Receives and parses protocol URLs
IPC Bridge	Communicates between main process and renderer
File Browser	Native dialog for selecting local node directories

The File Browser is a desktop convenience feature. In standalone mode, users can type local paths directly into the node installation input field.

Server Restart Protocol

Managed Mode (Desktop App)

When running in the desktop app, server restarts are handled automatically:

Summary
Sequence Diagram

Backend exits with code 42 (signals intentional restart)
Desktop app waits for port release
Desktop app respawns server process
Frontend polls health endpoint until ready

Frontend              Backend              Desktop App
 │                     │                      │
 │ POST /restart       │                      │
 │────────────────────▶│                      │
 │                     │ Exit code 42         │
 │                     │─────────────────────▶│
 │                     │                      │ Wait for port
 │                     │                      │────────┐
 │                     │                      │◀───────┘
 │                     │                      │ Respawn server
 │                     │◀─────────────────────│
 │ Poll /health        │                      │
 │────────────────────▶│                      │
 │     200 OK          │                      │
 │◀────────────────────│                      │
 │ Resume ops          │                      │
 │────────┐            │                      │
 │◀───────┘            │                      │

Key points:

Exit code 42 signals intentional restart (not a crash)
Brief wait ensures the port is released before respawn
Frontend polls health endpoint until server is ready
This ensures proper Python module reloading since imports are cached

Standalone Mode

When running the server directly (uv run daydream-scope):

Unix/macOS: Uses os.execv() to replace the current process in-place
Windows: Spawns a new subprocess and exits the old one

Data Storage

Node state is persisted in the user’s data directory:

Data	Location	Purpose
Node list	`~/.daydream-scope/nodes/nodes.txt`	Installed package specs
Resolved deps	`~/.daydream-scope/nodes/resolved.txt`	Lock file for reproducibility; baseline for update detection
Venv backup	`~/.daydream-scope/nodes/freeze.txt`	Rollback state

Error Handling

The node system uses defensive error handling at each stage:

Error Type	Detection	Recovery
Dependency conflicts	Dry-run compilation before install	Installation blocked with clear error message
Installation failures	Exception during uv install	Venv rolled back to pre-install state
Runtime errors	Pipeline execution failure	Pipeline unloaded without affecting others
Network errors	Health check timeouts	Frontend retries with exponential backoff

This multi-layer approach ensures that node operations cannot corrupt the base Scope installation.

Using Nodes

Install and manage nodes

Developing Nodes

Create your own custom pipelines

Pipeline Architecture

Technical details of the pipeline system

Tutorial: Build a VFX Node

Step-by-step tutorial building a complete node from scratch

Node Architecture

Understanding node system

Architecture Layers

Key Technologies

System Architecture

Node Discovery

Node Sources

Installation Flow

Node Update Flow

Update Detection

Update Execution

Source-Specific Update Behavior

Uninstallation Flow

Manual Reload Flow

Deep Link Installation

Component Responsibilities

Backend Components

Frontend Components

Desktop App Components

Server Restart Protocol

Managed Mode (Desktop App)

Standalone Mode

Data Storage

Error Handling

See Also

Using Nodes

Developing Nodes

Pipeline Architecture

Tutorial: Build a VFX Node

​Understanding node system

​Architecture Layers

​Key Technologies

​System Architecture

​Node Discovery

​Node Sources

​Installation Flow

​Node Update Flow

​Update Detection

​Update Execution

​Source-Specific Update Behavior

​Uninstallation Flow

​Manual Reload Flow

​Deep Link Installation

​Component Responsibilities

​Backend Components

​Frontend Components

​Desktop App Components

​Server Restart Protocol

​Managed Mode (Desktop App)

​Standalone Mode

​Data Storage

​Error Handling

​See Also

Using Nodes

Developing Nodes

Pipeline Architecture

Tutorial: Build a VFX Node

Understanding node system

Architecture Layers

Key Technologies

System Architecture

Node Discovery

Node Sources

Installation Flow

Node Update Flow

Update Detection

Update Execution

Source-Specific Update Behavior

Uninstallation Flow

Manual Reload Flow

Deep Link Installation

Component Responsibilities

Backend Components

Frontend Components

Desktop App Components

Server Restart Protocol

Managed Mode (Desktop App)

Standalone Mode

Data Storage

Error Handling

See Also