Deephaven architecture overview

From Python/Java code to real-time web UIs

Deephaven query engine (Java)
Directed Acyclic Graph (DAG)
Tables and operations form a dependency graph with automatic update propagation
T1
Source
T2
Filter
T3
Join
• Incremental updates
• Logical clock for consistency
• Garbage collection
Update Graph (UG) Cycles:
Batch changes per cycle (1000ms target, configurable) →
Propagate in topological order →
Notify dependents →
UI updates
Table Data Structures
RowSet
Sparse 64-bit keys
RSP / Roaring Bitmaps
ColumnSources
Column data by row key
Shared across tables
Zero-copy optimization: filters and views share ColumnSources instead of copying column data
Chunk-Oriented Processing
Chunks (thousands of elements)
Bulk data transfer: getChunk(), fillChunk()
Enables SIMD vectorization, one call per chunk instead of per cell
Performance: a filter makes one call per chunk, not one per row
Formula Engine
JavaParser → Compiled classes
Chunked kernels for Python function calls (Python sessions)
Parameter substitution & reuse
Table Operations
Joins • Aggregations • Sorts
Columnar hash tables
Incremental algorithms
Data Sources
Parquet • Kafka • CSV
Streaming & Batch
Unified API for both
Key performance characteristics
Incremental updates: only the rows a change affects are recomputed
Shared structures: filtered and joined views reuse their source's column data
Chunk processing: orders of magnitude fewer method calls, and SIMD processes several values per instruction
Language Integration
jpy (Python ↔ Java)
JNI bridge, minimal crossings
Java and Groovy
Direct engine access
Network Protocols
gRPC / Arrow
Flight service
Barrage
Ticking data
WebSocket
Real-time updates
deephaven.ui (Python sessions only)
Pure Python components
@ui.component decorator
Server-side rendering
→ React in browser
No JavaScript required
JavaScript / TypeScript Client
web-client-ui components
Direct gRPC-Web
Grid, charts, dashboards
Custom integration
Maximum flexibility
Both approaches leverage same DAG engine, gRPC protocols, and real-time capabilities