Documentation

compiler

Compile .tf source files to .cfm mmap format.

This module provides the Compiler class that converts Text-Fabric (.tf) source files into the Context Fabric memory-mapped (.cfm) format. The cfm format uses numpy arrays with memory mapping for efficient multi-process access.

Classes

class

Compiler

Compile .tf source files to CF mmap format.

Compile .tf source files to CF mmap format. The compiler reads .tf plain text feature files and converts them to memory-mapped numpy arrays organized in the .cfm directory structure. Usage ----- compiler = Compiler(source_dir='/path/to/tf/files') success = compiler.compile(output_dir='/path/to/output/.cfm/1/') Parameters ---------- source_dir : str Path to directory containing .tf source files

Attributes

NameTypeDescription
_edge_featuresdict[(str, tuple[(dict[(int, Any)], bool)])]—
_feature_metadict[(str, dict[(str, str)])]—
_levdown_datalist[tuple[(int, ...)]] | None—
_levels_datalist[tuple[(str, float, int, int)]] | None—
_levup_datalist[tuple[(int, ...)]] | None—
_node_featuresdict[(str, dict[(int, str | int)])]—
_order_datalist[int] | None—
_oslots_datatuple[(tuple[(tuple[(int, ...)], ...)], int, int)] | None—
_otext_metadict[(str, str)]—
_otype_datatuple[(tuple[(str, ...)], int, int, str)] | None—
_rank_datalist[int] | None—
error——
info——
max_nodeint—
max_slotint—
node_typeslist[str]—
slot_typestr—
source_dirPath—
type_orderlist[str]—
warning——

Methods

__init__(self, source_dir: str) → None
Parameters
  • source_dir: str
compile(self, output_dir: str | Path | None = None, precomputed: dict[(str, Any)] | None = None) → bool

Compile all .tf files to .cfm format.

Parameters
  • output_dir: str | Path | None= None
  • precomputed: dict[(str, Any)] | None= None

Functions

function
compile_corpus(source_dir: str, output_dir: str | None = None) → bool

Convenience function to compile a .tf corpus to CFM format.

Convenience function to compile a .tf corpus to CFM format. Parameters ---------- source_dir : str Path to directory containing .tf source files output_dir : str, optional Output directory. Defaults to {source_dir}/.cfm/{CFM_VERSION}/ Returns ------- bool True if compilation succeeded
Parameters
  • source_dir: str
  • output_dir: str | None= None