progressive
Progressive corpus loading models.
Classes
class
ProgressiveLoadResult
(BaseModel)Complete progressive loading results.
Attributes
| Name | Type | Description |
|---|---|---|
| cf_memory_by_step | list[float] | None | — |
| cf_scaling | ScalingAnalysis | None | — |
| corpora_order | list[str] | — |
| max_corpora | int | — |
| num_runs | int | — |
| steps | list[ProgressiveLoadStep] | — |
| tf_memory_by_step | list[float] | None | — |
| tf_scaling | ScalingAnalysis | None | — |
Methods
get_steps_by_impl(self, implementation: Literal[('TF', 'CF')]) → list[ProgressiveLoadStep]Get steps for a specific implementation.
Parameters
implementation: Literal[('TF', 'CF')]
get_steps_by_run(self, run_id: int) → list[ProgressiveLoadStep]Get steps for a specific run.
Parameters
run_id: int
to_dict(self) → dictConvert to JSON-serializable dictionary.
to_json(self) → strSerialize to JSON string.
class
ProgressiveLoadStep
(BaseModel)Single step in progressive loading test.
Attributes
| Name | Type | Description |
|---|---|---|
| corpora_loaded | list[str] | — |
| corpus_added | str | — |
| cumulative_load_time_s | float | — |
| implementation | Literal[('TF', 'CF')] | — |
| incremental_rss_mb | float | — |
| run_id | int | — |
| step | int | — |
| step_load_time_s | float | — |
| total_rss_mb | float | — |
Methods
to_dict(self) → dictConvert to JSON-serializable dictionary.
class
ScalingAnalysis
(BaseModel)Linear scaling analysis results.
Attributes
| Name | Type | Description |
|---|---|---|
| implementation | Literal[('TF', 'CF')] | — |
| intercept_mb | float | — |
| predicted_10_corpora_mb | float | — |
| predicted_50_corpora_mb | float | — |
| r_squared | float | — |
| slope_mb_per_corpus | float | — |
Methods
to_dict(self) → dictConvert to JSON-serializable dictionary.