Field Definitions and Constants#

Market data field definitions and constants.

This module provides standardized field name constants and enums to ensure type safety and prevent magic strings throughout the codebase.

class CalculatedMetric(*values)

Bases: StrEnum

Calculated metrics derived from the loaded base fields.

These metrics are not present in the input data; they are computed by MarketDataProcessingService.add_calculated_metrics after the raw price tables have been loaded, and are appended to the data dictionary alongside the source fields. Each enum value is the column name used in the resulting PyArrow tables.

Variables:

  • RETURNS – Simple period returns derived from the adjusted close price ((p_t - p_t_minus_1) / p_t_minus_1). Column name "c_returns".

  • LOG_RETURNS – Natural-log returns derived from the adjusted close price (log(p_t / p_t_minus_1)). Column name "c_log_returns".

  • VOLATILITY – Rolling realized volatility derived from the returns series: the sample standard deviation of RETURNS over the execution-cost window, ending on the previous trading day. Column name "c_volatility".

  • ADV – Average daily volume in shares over the execution-cost window, ending on the previous trading day. Column name "c_adv".

  • HALF_SPREAD – Half the relative bid-ask spread as a fraction of the price, either taken from a mapped spread column or estimated from daily highs and lows, averaged over the execution-cost window and ending on the previous trading day. Column name "c_half_spread".

Notes

The "c_" prefix in every value is a convention to flag the column as calculated, so callers can distinguish at a glance between raw source data and engineered features.

Every window metric ends on the trading day before the one it is stamped on. A window that included the current day would let a model see the very bar it is trading into, which is look-ahead.

See also

StandardField

Required source fields used to derive these metrics.

RETURNS = 'c_returns'
LOG_RETURNS = 'c_log_returns'
VOLATILITY = 'c_volatility'
ADV = 'c_adv'
HALF_SPREAD = 'c_half_spread'
class DataPipelineKeys

Bases: object

Keys used in data pipeline result dictionaries.

Centralise string constants so callers never rely on magic strings when accessing DataPipelineResult fields.

Variables:

  • PORTFOLIO_ENTITY (str) – Key for the portfolio entity object.

  • PORTFOLIO_DF (str) – Key for the portfolio DataFrame.

  • START_DATE_BACKTEST (str) – Key for the backtest start date.

  • MARKET_DATA_BUNDLE (str) – Key for the market data bundle.

  • DATA_DICT (str) – Key for the data dictionary.

  • BENCHMARK (str) – Key for the benchmark table.

  • PORTFOLIO_DICT (str) – Key for the portfolio weights dictionary.

  • EQUIVALENCE_DICT (str) – Key for the field equivalence mapping.

PORTFOLIO_ENTITY = 'portfolio_entity'
PORTFOLIO_DF = 'portfolio_df'
START_DATE_BACKTEST = 'start_date_backtest'
MARKET_DATA_BUNDLE = 'market_data_bundle'
DATA_DICT = 'data_dict'
BENCHMARK = 'benchmark'
PORTFOLIO_DICT = 'portfolio_dict'
EQUIVALENCE_DICT = 'equivalence_dict'
class RegisterKeys

Bases: object

Keys used in register dictionaries.

Centralise string constants for accessing register-related results produced by the backtesting engine.

Variables:

  • REGISTER_DF (str) – Key for the main register DataFrame.

  • COMMISSION_DF (str) – Key for the commissions DataFrame.

  • SHARES_DF (str) – Key for the shares DataFrame.

  • DAILY_WEIGHTS (str) – Key for the daily weights table.

  • ORDERS_DF (str) – Key for the orders DataFrame.

  • FINAL_TOTAL_PORTFOLIO_VALUE (str) – Key for the final portfolio value.

  • TOTAL_COMMISSIONS (str) – Key for total commissions paid.

  • YEARS (str) – Key for the number of years in the backtest.

  • PORTFOLIO_STATS (str) – Key for portfolio statistics.

  • BENCHMARK_STATS (str) – Key for benchmark statistics.

REGISTER_DF = 'Register_df'
COMMISSION_DF = 'Commission_df'
SHARES_DF = 'Shares_df'
DAILY_WEIGHTS = 'Daily_Weights'
ORDERS_DF = 'orders_df'
FINAL_TOTAL_PORTFOLIO_VALUE = 'final_total_portfolio_value'
TOTAL_COMMISSIONS = 'Total_commissions'
YEARS = 'years'
PORTFOLIO_STATS = 'portfolio_stats'
BENCHMARK_STATS = 'benchmark_stats'
class StandardField(*values)

Bases: StrEnum

Standard field names used internally by the backtest engine.

These are the canonical field names that the system uses for all operations. All input data is mapped to these standard field names. Each field name describes the role the column plays in the backtest, not the data source.

Variables:

  • COMMISSION_PRICE – Price column used to calculate the number of shares for commission fees (typically unadjusted VWAP).

  • TRADE_EXECUTION_PRICE – Price column used as the trade execution price for buying/selling shares (typically split/dividend-adjusted VWAP).

  • MARK_TO_MARKET_PRICE – Price column used for daily portfolio valuation between rebalancing dates (typically split/dividend-adjusted close).

  • DATE – Column containing the trading dates.

  • BORROW_FEE_RATE – Optional column with the ticker’s own annual borrow fee, as a fraction. Not a price: it may be absent from a file, and a null value means “use the global borrow fee”.

  • VOLUME – Optional column with the shares traded on the day. Feeds the average daily volume that market-impact models divide the order size by.

  • HIGH – Optional column with the day’s highest price. Feeds the bid-ask spread estimators.

  • LOW – Optional column with the day’s lowest price. Feeds the bid-ask spread estimators.

  • CLOSE – Optional column with the day’s closing price. Only the Abdi-Ranaldo spread estimator needs it; Corwin-Schultz does not.

  • SPREAD – Optional column with the full relative bid-ask spread provided by the data vendor, as a fraction of the price (0.001 for 10 basis points). When mapped it takes precedence over the estimators, and the engine halves it to obtain the half spread a single side pays.

COMMISSION_PRICE = 'commission_price_column'
TRADE_EXECUTION_PRICE = 'trade_execution_price_column'
MARK_TO_MARKET_PRICE = 'mark_to_market_price_column'
DATE = 'date_column'
BORROW_FEE_RATE = 'borrow_fee_rate_column'
VOLUME = 'volume_column'
HIGH = 'high_column'
LOW = 'low_column'
CLOSE = 'close_column'
SPREAD = 'spread_column'
classmethod get_optional_fields() → list[StandardField]

Return the fields a market data file may carry but need not.

Returns:

The optional fields in canonical order: BORROW_FEE_RATE followed by the execution-cost columns VOLUME, HIGH, LOW, CLOSE and SPREAD.

Return type:

list[StandardField]

Notes

Every one of these may be absent from a file, absent from the configuration, or null on a given day, and none of them is ever treated as a price by the data integrity check.

classmethod get_numeric_fields() → list[StandardField]

Return the required numeric standard fields, excluding the date column.

Useful for routines that operate only on price-like columns (commission, execution, mark-to-market) and need to skip the date column. The order matches the canonical price-field ordering used across the pipeline. Optional fields are not included; see get_optional_fields().

Returns:

[COMMISSION_PRICE, TRADE_EXECUTION_PRICE, MARK_TO_MARKET_PRICE].

Return type:

list[StandardField]

Examples

>>> [f.value for f in StandardField.get_numeric_fields()]
['commission_price_column', 'trade_execution_price_column', 'mark_to_market_price_column']

See also

get_all_fields

Same list plus the DATE field.

classmethod get_all_fields() → list[StandardField]

Return every required standard field, including the date column.

Used by validation routines that need to confirm a fully specified field-mapping configuration. The order is canonical (three price fields followed by the date field) and matches STANDARD_FIELDS_ORDER at module level.

Returns:

Every StandardField member in canonical order.

Return type:

list[StandardField]

Examples

>>> len(StandardField.get_all_fields())
4
>>> StandardField.get_all_fields()[-1]
<StandardField.DATE: 'date_column'>

See also

get_numeric_fields

Same list without the DATE field.

get_optional_fields

Fields that are not required.

classmethod values() → list[str]

Return every field enum as its plain string value.

Convenience wrapper around iterating the enum and collecting .value on each member. Returned as a list (not a set) so the canonical order is preserved.

Returns:

String values of every StandardField member, in definition order (required fields first, then optional).

Return type:

list[str]

Examples

>>> StandardField.values()[:4]
['commission_price_column', 'trade_execution_price_column', 'mark_to_market_price_column', 'date_column']

See also

get_all_fields

The required members only.