Table structure: RowSets + ColumnSources

Efficient sharing enables zero-copy operations and reduced memory usage

Table: source
time_table("PT1S")
RowSet
Keys: [0, 1, 2, 3, ...]
All rows in table
ColumnSource
"A"
[0, 1, 2, 3, ...]
ColumnSource
"B"
[0, 2, 4, 6, ...]
Table: filtered
.where("B > 10")
RowSet (new)
Keys: [6, 7, 8, ...]
Subset where B > 10
ColumnSource
"A" (shared)
→ source.A
ColumnSource
"B" (shared)
→ source.B
Table: renamed
.view(["A", "C=B"])
RowSet (shared)
→ source.RowSet
Same keys
ColumnSource
"A" (shared)
→ source.A
ColumnSource
"C" (shared)
→ source.B
Columns A & B shared with filtered RowSet, Column A & B shared with renamed
Memory efficiency through sharing
  • Column A is stored once in memory, referenced by all three tables
  • RowSet for "renamed" table is shared with "source" (zero-copy view operation)
  • Updates to source.A automatically appear in filtered.A and renamed.A
Key concepts
RowSet
Sparse set of 64-bit row keys
Can be shared, or derived by a filter or sort
ColumnSource
Column data indexed by row keys
Shared across tables when possible
Shared reference
Points to parent table's data structure
Zero memory overhead for the data itself
Table
Combines RowSet + map of ColumnSources
Lightweight structure, heavy sharing