Batch Processing

Parallel multi-replay parsing via ProcessPoolExecutor. Each worker process parses one replay independently, so performance scales with CPU cores.

options:
  members:
    - ParseResult
    - parse_many
    - parse_many_to_parquet
    - read_parquet_table
    - parse_many_to_dataframe
  show_source: true

Generated API

Module gem.replays.batch

Bulk replay parsing — process many .dem files in parallel.

Source: src/gem/replays/batch.py

Top-level functions

parse_many

def parse_many(source: str | Path | Sequence[str | Path], *, workers: int | None = None, recursive: bool = False, progress: bool = True, timeout: float | None = None) -> list[ParseResult]

Parse multiple replays in parallel and return a result per replay.

Source: src/gem/replays/batch.py:136

parse_many_to_dataframe

def parse_many_to_dataframe(source: str | Path | Sequence[str | Path], *, workers: int | None = None, recursive: bool = False, progress: bool = True, timeout: float | None = None) -> dict[str, pd.DataFrame]

Parse multiple replays and concatenate results into per-table DataFrames.

Source: src/gem/replays/batch.py:209

parse_many_to_parquet

def parse_many_to_parquet(source: str | Path | Sequence[str | Path], output_dir: str | Path, *, workers: int | None = None, recursive: bool = False, progress: bool = True, timeout: float | None = None, include: Iterable[str] = (), index: bool = False) -> list[Path]

Parse multiple replays and write each to its own parquet subdirectory.

Source: src/gem/replays/batch.py:266

read_parquet_table

def read_parquet_table(output_dir: str | Path, table: str) -> pd.DataFrame

Load one table across every replay written by :func:parse_many_to_parquet.

Source: src/gem/replays/batch.py:414

Top-level classes

ParseResult

class ParseResult

Outcome of parsing a single replay.

Source: src/gem/replays/batch.py:50

Dataclass fields

Properties

ok

Signature: def ParseResult.ok(self) -> bool

Return True when parsing succeeded.

Source: src/gem/replays/batch.py:64

complete

Signature: def ParseResult.complete(self) -> bool

Return True when parsing succeeded and read the whole replay.

Source: src/gem/replays/batch.py:69