Function: tokenizeHeader()

tokenizeHeader(record, d, onUnknownEscape?, onUnpairedEscape?): AstmField[]

Tokenize an H (header) record into its fields.

A header cannot go through tokenizeRecord: its second field is the delimiter declaration, which carries all three non-field delimiters literally rather than as escape sequences. Run through the generic tokenizer the declaration would be split on its own repeat and component characters and its escape character would be decoded and reported as unpaired, so what the header declares would come back as fragments plus a spurious warning. This tokenizer instead takes the declaration verbatim as one opaque field, never decoded, and applies the ordinary escape-aware tokenizer to the data portion that follows it.

fields[0] is the type-letter field and fields[1] is the delimiter declaration (verbatim, never escape-decoded); the header's ASTM data fields follow from fields[2].

Parameters

record

string

The raw H record text (terminator already stripped).

d

Delimiters

The delimiters declared by this header.

onUnknownEscape?

(fieldIndex) => void

Called with the 0-based whole-record field index for each unrecognized escape sequence in the data portion.

onUnpairedEscape?

(fieldIndex) => void

Called with the 0-based whole-record field index for each unpaired escape character in the data portion. The declaration itself is opaque, so the escape character it names never reports here.

Returns

AstmField[]

The header's fields.

Example

import { tokenizeHeader, CANONICAL_DELIMITERS } from "@cosyte/astm";
const fields = tokenizeHeader("H|\\^&|||sender", CANONICAL_DELIMITERS);
fields[1].raw; // "\\^&"
fields[4].raw; // "sender"