Skip to main content

Arrow Flows

Arrow is the most nuanced data-plane area currently documented in the Go SDK.

The SDK already supports:

  • Arrow row append
  • direct arrow.Record append
  • Arrow log fetch and decode
  • projection-aware fetch on ARROW log tables

Bootstrapped Arrow Writes​

The dev harness uses BootstrapRowWriter(...) to pin table metadata, schema ID, compression settings, and optional writer state:

bootstrap, err := arrowClient.BootstrapRowWriter(ctx, client.WriterBootstrapOptions{
InitWriter: true,
})
if err != nil {
log.Fatal(err)
}

Then it appends Arrow rows:

result, err := bootstrap.AppendArrowRows(ctx, 0, []client.Row{row1, row2})

Direct arrow.Record Append​

The SDK also supports direct record append through:

  • AppendArrowRecord(...)
  • AppendArrowRecordAuto(...)

This path is useful when your application already builds Arrow records directly.

Fetch and Decode​

After fetch, the current public decode helper is:

rows, err := client.DecodeArrowLogBatchRows(logSchema, fetched[0].Records)

Projection Caveat​

Projection support is aligned with upstream/server behavior:

  • projection is supported for ARROW log format
  • projection should not be documented as a universal log-table feature across all encodings

That caveat should remain explicit anywhere Arrow projection is shown.