std.text.stream

std.text.stream

std.text.stream is the shipped v1 text-stream facade for :script text work. It is bridge-backed, bounded to concrete operators, and intentionally smaller than the long-term stream algebra.

The current module exposes real string streams plus bridge-backed file handles, grep-style selection, awk-style field extraction (field, cols, fields), literal replacement, materializing sort/unique, count, and joinLines. It does not yet expose callback map, callback filter, materializing collect, sort_by/group_by, parallelism, or automatic fusion — those wait on a real callback bridge.

Types

pub struct Provenance {
    path: str,
    line: u64,
    byte_offset: u64,
}

pub struct Line {
    text: str,
    provenance: Provenance,
}

pub struct TextStream[T] {
    _handle: usize,
}

TextStream[T] wraps an opaque bridge handle. Terminal operators consume the current stream state.

Constructors

pub func fromString(input: []const u8) -> TextStream[Line]
pub func fromFile(path: []const u8) -> TextStream[Line]
pub func fromGlob(pattern: []const u8) -> TextStream[Line]
pub func fromStdin() -> TextStream[Line]

fromString is the proven v1 constructor. fromFile reads a whole file (bounded at 10 MiB) and splits it into lines. fromGlob and fromStdin exist as bridge handles but are not yet covered by a live smoke.

Selection

pub func grep[T](self: TextStream[T], pattern: []const u8) -> TextStream[T]
pub func grepV[T](self: TextStream[T], pattern: []const u8) -> TextStream[T]
pub func head[T](self: TextStream[T], n: usize) -> TextStream[T]
pub func skip[T](self: TextStream[T], n: usize) -> TextStream[T]
pub func selectField[T](self: TextStream[T], n: usize, sep: []const u8, value: []const u8) -> TextStream[T]

grep keeps matching records. grepV excludes matching records. head limits the emitted records; skip consumes emitted records before returning the rest. selectField keeps only records whose nth (1-based) field equals value (awk $N == value).

Field Extraction

pub func field[T](self: TextStream[T], n: usize, sep: []const u8) -> TextStream[T]
pub func cols[T](self: TextStream[T], start: usize, end: usize, sep: []const u8) -> TextStream[T]
pub func fields[T](self: TextStream[T], sep: []const u8) -> TextStream[T]

All three split each record on a literal sep (byte scanning, no callback):

• field(n, sep) replaces each record with its nth (1-based) field; an out-of-range field yields an empty record. This is awk {print $N}.
• cols(start, end, sep) replaces each record with its fields start..end (1-based, inclusive) rejoined by sep, skipping fields that do not exist. This is awk {print $start ... $end}.
• fields(sep) is cardinality-expanding: it emits each field as its own record. This is awk {for(i=1; i<=NF; i++) print $i} and is the primitive behind sort -u-style word pipelines: fields(" ") |> sort() |> unique().

Replacement

pub func replace[T](
    self: TextStream[T],
    pattern: []const u8,
    replacement: []const u8,
) -> TextStream[T]

pub func replaceFirst[T](
    self: TextStream[T],
    pattern: []const u8,
    replacement: []const u8,
) -> TextStream[T]

The bridge currently performs literal byte-pattern replacement. Regex-grade replacement can be layered later once the stream bridge and std.text.rex share a common matcher contract.

Materialization

pub func sort[T](self: TextStream[T]) -> TextStream[T]
pub func unique[T](self: TextStream[T]) -> TextStream[T]

sort materializes the stream and returns a fresh stream of records sorted in ascending byte order. unique materializes the stream and deduplicates consecutive-equal records in first-occurrence order. Both consume the source.

Terminal Operators

pub func count[T](self: TextStream[T]) -> u64
pub func joinLines[T](self: TextStream[T], sep: []const u8) -> *u8

count consumes the stream and returns the number of emitted records. joinLines consumes the stream and returns a NUL-terminated bridge-owned byte buffer.

Convenience

pub func grepFile(path: []const u8, pattern: []const u8) -> TextStream[Line]
pub func countMatches(path: []const u8, pattern: []const u8) -> u64
pub func headMatches(path: []const u8, pattern: []const u8, n: usize) -> TextStream[Line]

These are thin compositions over fromFile, grep, count, and head.

Deferred Surface

The following names are not part of the shipped v1 surface:

• callback map
• callback filter
• materializing collect
• sort_by(fn) / group_by(fn)
• parallel stream algebra
• compile-time fusion guarantees

Those require a real callback/materialization bridge. The module now refuses to pretend they exist.

Verification

./scripts/zb test-text-stream

The target runs bridge unit tests and the Janus AOT smoke std/text/stream_smoke.jan, proving fromString, fromFile, grep, grepV, head, skip, replace, replaceFirst, field, selectField, cols, fields, sort, unique, and count.