core_lens.schema.detection#

Schema auto-detection from Parquet file metadata.

Exceptions#

SchemaDetectionError

Raised when the detection layer cannot produce a valid SchemaProfile.

Functions#

detect(→ core_lens.schema.profile.SchemaProfile)

Introspect Parquet file schemas and return a validated SchemaProfile.

Module Contents#

exception core_lens.schema.detection.SchemaDetectionError#

Bases: Exception

Raised when the detection layer cannot produce a valid SchemaProfile.

The message identifies which file and which inference step failed so entity authors have an actionable starting point.

core_lens.schema.detection.detect(static_path: str, key_cols: list[str], geometry_col: str, annual_path: str | None = None, sub_annual_path: str | None = None, storage_options: dict[str, Any] | None = None) → core_lens.schema.profile.SchemaProfile#

Introspect Parquet file schemas and return a validated SchemaProfile.

Reads only Parquet footer metadata — no row data is scanned. Intended to be called once per entity at AoI.register() time and the result cached on the entity instance.

Detection order for each inferred field:

  • geometry_type — derived from the dtype of geometry_col: Binary → "wkb", String → "wkt", Float with a detected lon companion column → "latlon".

  • bbox_cols — checked against a fixed set of four-column name patterns. None if no known pattern is present in the static schema.

  • annual_time_col / sub_annual_time_col — the first column with a Date, Datetime, or Int dtype whose name matches a known set of time-column names. Falls back to the first Date/Datetime column regardless of name.

  • extra_*_cols — every remaining column after key, geometry, bbox, and time columns are accounted for.

Parameters:

  • static_path (str) – Path or cloud URI to the static GeoParquet file.

  • key_cols (list[str]) – Entity key column name(s), as declared on the entity.

  • geometry_col (str) – Geometry column name, as declared on the entity.

  • annual_path (str | None, optional) – Path or cloud URI to the annual Parquet file, or None.

  • sub_annual_path (str | None, optional) – Path or cloud URI to the sub_annual Parquet file, or None.

  • storage_options (dict[str, Any] | None, optional) – Cloud credential / configuration options forwarded to pl.scan_parquet. None uses ambient credentials.

Returns:

A validated SchemaProfile.

Return type:

SchemaProfile

Raises:

SchemaDetectionError – If a required column is absent, the geometry type cannot be inferred, or any other detection step fails.