How Scrollr Works
From source API to your desktop in milliseconds. A decoupled, CDC-driven pipeline built on Go, Rust, React, and Redis.
The Pipeline
Four stages from data source to your screen
Data Sources
TwelveData WebSocket for market data, ESPN API for scores, RSS/Atom feeds for news, Yahoo Fantasy API for leagues.
Ingestion Services
Three independent Rust services collect, normalize, and write data to PostgreSQL. Each runs its own schedule and connection strategy.
PostgreSQL + CDC
All data lands in PostgreSQL. Sequin monitors table changes via CDC (Change Data Capture) and fires webhooks to the core API.
Real-time Delivery
Core API routes CDC records to channel APIs, which return affected user lists. Core publishes to per-user Redis channels via SSE.
CDC Record Flow
How a single data change reaches the right user
Rust Service
Writes to PostgreSQL
Sequin CDC
Detects row changes
Core API
POST /webhooks/sequin
Channel API
POST /internal/cdc → users[]
Redis Pub/Sub
events:user:{sub}
SSE → Client
Desktop App
Design Principles
The rules that shape every architectural decision
Decoupled Channels
Each channel is a fully self-contained unit with its own Go API, Rust service, frontend components, and config. No shared code between channels.
Zero-trust Proxying
Core API validates JWTs and injects X-User-Sub headers. Integration APIs never see tokens — they trust the core gateway.
Self-registration
Channel APIs register in Redis on startup with a 30s TTL heartbeat. Core discovers them dynamically — no hardcoded routes.
Convention-based UI
Desktop app discovers channel components at build time via import.meta.glob. Drop a file in the right folder and it appears.
Tech Stack
What powers each layer
Core API
Go 1.22
Fiber v2, pgx, Redis
SSE Hub
Per-user Redis Pub/Sub channels
Logto
Self-hosted OIDC, JWT validation
Ingestion
Rust
tokio async runtime
WebSocket
TwelveData persistent connection
HTTP Polling
ESPN 60s, RSS 5min, Yahoo 120s
Frontend
React 19
Vite 7, TanStack Router
Tailwind v4
daisyUI theme system
Motion
Production-grade animations
Desktop
Tauri v2
Cross-platform native shell
React 19
Multi-window UI
SSE + Polling
Real-time data delivery
Infrastructure
PostgreSQL
Shared DB, natural table isolation
Redis
Cache, Pub/Sub, registration
Sequin
CDC webhooks from PostgreSQL
Deployment
Coolify
Self-hosted PaaS
Docker Compose
Per-integration bundles
Nixpacks
Frontend builds