core_lens.base.namespaces.stats

Statistical analysis namespace for core_lens Result objects.

Exceptions

CorrelationError

Raised when correlation cannot be computed (e.g. fewer than 2 columns).

Classes

CorrelateMethod	Correlation methods.
TestMethod	Statistical hypothesis testing methods.
ChangeMethod	Methods for calculating change over time.
AnomalyCrossMethod	Methods for detecting cross-sectional anomalies.
AnomalyTsMethod	Methods for detecting time-series anomalies.
SimilarityMethod	Methods for calculating similarity or distance.
StatsNamespace	Statistical methods on result.stats.*.

Module Contents

class core_lens.base.namespaces.stats.CorrelateMethod(*args, **kwds)

Bases: enum.Enum

Correlation methods.

Variables:

• PEARSON – Pearson correlation coefficient.

• SPEARMAN – Spearman rank correlation.

• KENDALL – Kendall Tau correlation.

PEARSON = 'pearson'

SPEARMAN = 'spearman'

KENDALL = 'kendall'

class core_lens.base.namespaces.stats.TestMethod(*args, **kwds)

Bases: enum.Enum

Statistical hypothesis testing methods.

Variables:

• T_TEST – Student’s t-test.

• MANN_WHITNEY – Mann-Whitney U test.

• WILCOXON – Wilcoxon signed-rank test.

• KS – Kolmogorov-Smirnov test.

• CHI_SQUARE – Chi-square test.

__test__ = False

T_TEST = 't-test'

MANN_WHITNEY = 'mann-whitney'

WILCOXON = 'wilcoxon'

KS = 'ks'

CHI_SQUARE = 'chi-square'

class core_lens.base.namespaces.stats.ChangeMethod(*args, **kwds)

Bases: enum.Enum

Methods for calculating change over time.

Variables:

• ABSOLUTE – Absolute difference between periods.

• PERCENTAGE – Percentage change between periods.

• TREND – Linear trend over time.

ABSOLUTE = 'absolute'

PERCENTAGE = 'percentage'

TREND = 'trend'

class core_lens.base.namespaces.stats.AnomalyCrossMethod(*args, **kwds)

Bases: enum.Enum

Methods for detecting cross-sectional anomalies.

Variables:

• ZSCORE – Z-score method.

• IQR – Interquartile range method.

• PERCENTILE – Percentile-based method.

• THRESHOLD – Fixed threshold method.

ZSCORE = 'zscore'

IQR = 'iqr'

PERCENTILE = 'percentile'

THRESHOLD = 'threshold'

class core_lens.base.namespaces.stats.AnomalyTsMethod(*args, **kwds)

Bases: enum.Enum

Methods for detecting time-series anomalies.

Variables:

• STL – Seasonal-Trend decomposition using LOESS.

• CUSUM – Cumulative sum control chart.

• MAD – Median Absolute Deviation.

STL = 'stl'

CUSUM = 'cusum'

MAD = 'mad'

class core_lens.base.namespaces.stats.SimilarityMethod(*args, **kwds)

Bases: enum.Enum

Methods for calculating similarity or distance.

Variables:

• EUCLIDEAN – Euclidean distance.

• COSINE – Cosine similarity.

• MAHALANOBIS – Mahalanobis distance.

• MANHATTAN – Manhattan distance.

EUCLIDEAN = 'euclidean'

COSINE = 'cosine'

MAHALANOBIS = 'mahalanobis'

MANHATTAN = 'manhattan'

exception core_lens.base.namespaces.stats.CorrelationError

Bases: ValueError

Raised when correlation cannot be computed (e.g. fewer than 2 columns).

class core_lens.base.namespaces.stats.StatsNamespace(result: core_lens.base.result.Result)

Statistical methods on result.stats.*.

All methods return a fresh Result. Computed values always go in data; method parameters go in metadata.

describe(columns: list[str] | None = None, by: str = 'column') → core_lens.base.result.Result

Per-column or per-entity descriptive statistics.

Uses polars’ in-built methods for mean, std, min, max, quantiles etc.

Parameters:

• columns (list[str] | None, optional) – Numeric columns to describe. Defaults to all numeric cols.

• by (str, optional) – "column" (one row per column, design default) or "entity" (one row per entity with mean/std per column).

Returns:

New Result with descriptive stats.

Return type:

Result

Raises:

ValueError – If by is not \"column\" or \"entity\".

correlate(columns: list[str], method: CorrelateMethod = CorrelateMethod.PEARSON, across: str = 'entity') → core_lens.base.result.Result

Pairwise correlations between columns.

Parameters:

• columns (list[str]) – At least 2 column names.

• method (CorrelateMethod, optional) – A CorrelateMethod enum value.

• across (str, optional) – "entity" or "time" — recorded in metadata only.

Returns:

Result whose data has columns column_a | column_b | correlation | p_value.

Return type:

Result

Raises:

• CorrelationError – If fewer than 2 columns supplied.

• ValueError – If method is not recognised.

Under the hood:

Calls scipy.stats.pearsonr, scipy.stats.spearmanr, or scipy.stats.kendalltau to compute the corresponding correlation coefficients and p-values.

test(column: str, groups: str | None = None, periods: list[tuple[int, int]] | None = None, against: float | None = None, method: TestMethod | None = None, significance_level: float = 0.05) → core_lens.base.result.Result

Hypothesis test in three modes: group-based, period-based, single-sample.

Parameters:

• column (str) – Numeric column to test.

• groups (str | None, optional) – Categorical column to split groups on.

• periods (list[tuple[int, int]] | None, optional) – List of (from_year, to_year) period tuples.

• against (float | None, optional) – Reference value for a one-sample test.

• method (TestMethod | None, optional) – A TestMethod enum value. Auto-selected via Shapiro-Wilk if None.

• significance_level (float, optional) – Alpha level for significant flag (default 0.05).

Returns:

Result whose data has per-group descriptive stats group | n | mean | std | median and metadata with statistic, p_value, significant.

Return type:

Result

Raises:

ValueError – If none of groups, periods, against supplied, or if method is not recognised.

Under the hood:

Uses scipy.stats functions for statistical testing. - shapiro is used for normality testing to auto-select parametric vs non-parametric methods. - Parametric tests: ttest_1samp, ttest_ind. - Non-parametric tests: wilcoxon, mannwhitneyu, ks_2samp.

change(column: str, from_period: int, to_period: int, method: ChangeMethod = ChangeMethod.ABSOLUTE) → core_lens.base.result.Result

Change between two time periods per entity.

Parameters:

• column (str) – Value column to compute change for.

• from_period (int) – Start year/period integer.

• to_period (int) – End year/period integer.

• method (ChangeMethod, optional) – A ChangeMethod enum value.

Returns:

For absolute / percentage: data has key_col | value_from | value_to | change | pct_change. For trend: key_col | slope | r_squared | direction.

Return type:

Result

Raises:

ValueError – If method is not recognised or year column absent.

Under the hood:

• Absolute and Percentage methods use native Polars expressions.

• Trend method uses scipy.stats.linregress to perform linear regression over the time period.

anomaly(column: str, mode: str, method: AnomalyCrossMethod | AnomalyTsMethod, baseline: tuple[int, int] | None = None, threshold: float = 2.0) → core_lens.base.result.Result

Anomaly detection in cross-sectional or timeseries mode.

Parameters:

• column (str) – Value column to analyse.

• mode (str) – "cross_sectional" or "timeseries".

• method (AnomalyCrossMethod | AnomalyTsMethod) – An AnomalyCrossMethod or AnomalyTsMethod enum value.

• baseline (tuple[int, int] | None, optional) – (from_year, to_year) inclusive. Required for timeseries; optional for cross-sectional.

• threshold (float, optional) – Sigma / score threshold for anomaly flag (default 2.0).

Returns:

Result whose data has key_col | anomaly_score | is_anomaly (cross-sectional) or key_col | year | anomaly_score | is_anomaly (timeseries, baseline period excluded).

Return type:

Result

Raises:

ValueError – If mode, method, or observation count invalid.

Under the hood:

• Most anomaly methods (ZSCORE, IQR, PERCENTILE, THRESHOLD, MAD, CUSUM) are implemented using native Polars aggregations for high performance.

• The STL method calls statsmodels.tsa.seasonal.STL to decompose timeseries data.

similarity(target: str, columns: dict[str, Any], method: SimilarityMethod = SimilarityMethod.EUCLIDEAN, top_n: int = 10) → core_lens.base.result.Result

Find entities most similar to target across columns.

Parameters:

• target – Key value of the reference entity.

• columns –

  Mapping {column_name: None | (resolution, filter_dict)}.

  • None — use the column directly from result.data.

  • (resolution, filter_dict) — fetch from the entity’s file at the given resolution ("static", "annual", "sub_annual"). Supported filter_dict keys:

    • "year": int | (int, int) — year equality or range.

    • "season": str — season label ("kharif" etc.).

    • "agg": str — time-aggregation before joining ("mean" default, "sum", "min", "max").

• method – Distance metric. A SimilarityMethod enum value.

• top_n – Number of most-similar entities to return (default 10).

Returns:

Result with key_col | similarity_score | rank.

Raises:

ValueError – If method is invalid, no columns can be resolved, or target is not found.

Under the hood:

• Euclidean, Manhattan, and Cosine distances are computed using optimized NumPy operations.

• Mahalanobis distance uses scipy.spatial.distance.mahalanobis.