Architecture

Context-Fabric uses memory-mapped storage for predictable performance and efficient multi-corpus analysis.

Memory-Mapped Storage

Instead of loading corpus data into Python objects, Context-Fabric maps compiled files directly into the process's address space. The operating system handles paging data in and out as needed.

This enables:

• Multi-corpus analysis: Load Hebrew Bible, Septuagint, Dead Sea Scrolls, and Greek New Testament simultaneously on a laptop
• Production deployments: Predictable resource usage across concurrent requests

Multi-Process Sharing

Multiple processes reading the same corpus share physical memory pages at the OS level:

Process 1 ─┐
Process 2 ──┼── Shared mmap pages ── .cfm files
Process 3 ─┘

Four workers don't use four times the memory—they share read-only data transparently.

Benchmark Summary

With 10 corpora loaded simultaneously:

For detailed benchmarks and methodology, see the technical paper.