Function: tokenizeHeader()

tokenizeHeader(record, d, onUnknownEscape?): AstmField[]

Tokenize an H (header) record into its fields.

A header cannot go through tokenizeRecord: its second field is the delimiter declaration, which carries the escape character literally rather than as an escape sequence. The generic escape-aware split treats that lone escape character as the opening of an unterminated escape and swallows the whole remainder of the record into one field, so every data field after the declaration is lost. This tokenizer instead takes the declaration verbatim as one opaque field and applies the ordinary escape-aware tokenizer to the data portion that follows it.

fields[0] is the type-letter field and fields[1] is the delimiter declaration (verbatim, never escape-decoded); the header's ASTM data fields follow from fields[2].

Parameters

record

string

The raw H record text (terminator already stripped).

d

Delimiters

The delimiters declared by this header.

onUnknownEscape?

(fieldIndex) => void

Called with the 0-based whole-record field index for each unrecognized escape sequence in the data portion.

Returns

AstmField[]

The header's fields.

Example

import { tokenizeHeader, CANONICAL_DELIMITERS } from "@cosyte/astm";
const fields = tokenizeHeader("H|\\^&|||sender", CANONICAL_DELIMITERS);
fields[1].raw; // "\\^&"
fields[4].raw; // "sender"