Skip to main content

Arrow Flight SQL

Arrow Flight SQL is the high-throughput, columnar path into the same governed engine that the PostgreSQL wire protocol serves, a modern alternative to ODBC/JDBC for clients such as DuckDB, Polars, pandas, and PyArrow that consume Arrow record batches directly.

Direction: inbound. System9 is the server. Status: Live. A real pyarrow.flight server (FlightServerBase) serves the genuine Flight RPCs; a stock pyarrow.flight client round-trip, authenticate, run SQL, read Arrow batches, is proven end to end by a standing test. No named BI product is certified.

What is served​

The gateway is a real pyarrow.flight.FlightServerBase implementation, not an advertised-but-empty port:

  • A real auth handler. A basic handshake authenticates the caller and mints a short-lived bearer token; an unauthenticated or bad-credential client is denied closed.
  • GetFlightInfo returns one Flight endpoint per live cell, so a client that follows the endpoints performs a native multi-cell read. The endpoint set is re-planned against current cell membership, so an elastic expand or contract is reflected on the next GetFlightInfo rather than served from a stale list.
  • DoGet streams pyarrow.RecordBatch results carrying the same governance as the SQL wire, column masking is applied on the served stream, so an unauthorized column is masked in the bytes the client receives, not filtered client-side.

The surface is a second reach into one governed engine, not a second policy: the same row filters, column masks, and audit record apply as on the PostgreSQL wire.

Ports and connection​

PortTransportWhen
59950gRPC, clearTrusted intra-network segment
59951gRPC, TLSEnabled when server certificates are configured

Point an Arrow Flight SQL client at grpc://CELL_HOST:59950 (or grpc+tls://CELL_HOST:59951), authenticate, and query. The external ports terminate on the deny-by-default node front door and proxy to the data engine; see ports & protocols.

Configured through Config Policy​

Arrow Flight SQL is the first service governed by the unified Config Policy plane (data.flight_sql, operator-only): an operator enables and tunes it as a signed, topology-scoped config document, the same way the nine administrative sections are authored.