Overview

Tabular Data

The TabularData class (from System.Data.Tabular) provides high-performance, memory-efficient handling of large column-oriented datasets.

TabularData Class

Method Description
CreateFromDataSet(ds [, opts]) Initializes from a database DataSet.
EvaluateNewColumn(name, rpn_expr) Adds a calculated column using an RPN expression array.
EvaluateAggregate(op, rpn_expr) Calculates a scalar value (currently supports only sum).
LockAndShare(name) Makes the dataset globally available in shared memory.
ConnectToShared(name) Connects to an existing shared dataset.
ExportToSeparated(sep) Exports data to a delimited string (CSV).

RPN Expression Reference

The EvaluateNewColumn and EvaluateAggregate methods use an array of strings/variants representing an RPN expression. These are JIT-compiled for maximum performance.

Field and Constant Access

Syntax Description
'"ColumnName"' Pushes the value of the specified column onto the stack.
'"ColumnName" DefValue' Pushes column value, using DefValue (float) if the cell is null.
'"Col" Def {"K":V, ...}' Pushes a mapped value based on a JSON lookup table.
123.45 Pushes a literal numeric constant onto the stack.

Arithmetic and Comparison

Opcode Operation
+, -, *, / Standard binary arithmetic operations.
=, >=, <> Comparison operators (returns 1.0 for True, 0.0 for False).

Mathematical Functions

Opcode Description
abs Absolute value.
sqr, sqrt Square and square root.
exp, ln Exponential and natural logarithm.
min, max Pops two values and pushes the minimum or maximum (row-wise).
relu Rectified Linear Unit: fused binary operator max(0, a + b).
smape Symmetric Mean Absolute Percentage Error component.

Rounding and Stack

Opcode Description
round Rounds to the nearest integer.
floor, ceil Floor and ceiling functions.
dup Duplicates the top value on the stack.
dup0..dup9 Duplicates the value at the specified stack depth.

Example: Normalization

uses System.Data, System.Data.Tabular;

var data := TabularData.Create;
data.AddColumn('Signal', [0.5, 1.2, 3.4, 2.1, 0.8]);

// Pre-calculate constants (min/max aggregates not supported in current binary)
var sMin := 0.5;
var sMax := 3.4;
var sRange := sMax - sMin;

// JIT expression: (Signal - Min) / Range
data.EvaluateNewColumn('Normalized', [ '"Signal"', sMin, '-', sRange, '/' ]);

// Aggregate sum
var total := data.EvaluateAggregate('sum', [ '"Normalized"' ]);
PrintLn('Total Normalized Sum: ' + FloatToStr(total));
Result
Total Normalized Sum: 1.89655172413793

Performance Features

  • JIT Compilation: Expressions in EvaluateNewColumn and EvaluateAggregate are JIT-compiled for native execution speed.
  • Memory Efficiency: Data is stored in a column-oriented format, reducing overhead and improving cache locality.
  • Zero-Copy Sharing: ConnectToShared provides access to global datasets without duplicating memory.
← Databases (SQL) Graphics (Pixmap) →
On this page