calc_nozawa_portfolio

Functions to process and merge bond data in order to construct portfolio approximations following Nozawa (2017) as used in He, Manela, and Kelly (2017).

This module performs the following steps:

1. Processes the open treasury data (from pull_bondret_treasury.load_bondret_treasury_file):

   • Converts ‘DATE’ to datetime,

   • Scales ‘tr_return’ and ‘tr_ytm_match’ by 1/100,

   • Creates a unique identifier ‘cusip_date’ (CUSIP + ‘_’ + formatted DATE),

   • Sorts the data by ‘DATE’.

2. Processes the CRSP bond returns data (from pull_CRSP_bond_returns.load_bondret):

   • Converts ‘date’ to datetime,

   • Casts ‘year’ to int,

   • Scales ‘t_yld_pt’ by 1/100,

   • Creates ‘cusip_date’ (cusip + ‘_’ + formatted date),

   • Sorts by [‘cusip’, ‘date’],

   • Computes a forward return column ‘ret_eom_fwd’ (using groupby shift(-1)).

3. Merges the two datasets on ‘cusip_date’ via an outer join, filling missing ‘date’ and ‘cusip’ values from the counterpart columns, and then reorders columns using a custom helper from misc_tools.

4. Processes the merged DataFrame:

   • Computes ‘yield_spread’ = (yield - tr_ytm_match),

   • Computes ‘TTM_diff’ = (tmt - tau),

   • Drops rows with missing yield_spread,

   • Adjusts negative ‘amount_outstanding’ by multiplying by -0.001,

   • Sorts by ‘date’ and resets the index,

   • Computes decile rankings for yield_spread within each date (using qcut), then adds 11 so deciles run from 11 to 20.

5. calculate_decil_returns(merged): A vectorized function that computes value–weighted portfolio returns for each decile. It uses the pre–computed forward return ‘ret_eom_fwd’ and the weights based on ‘amount_outstanding’. It returns a DataFrame with index as ‘date’ and columns corresponding to decile numbers (11 to 20) representing the forward returns.

6. process_all_data(open_df, crsp_df): Processes the raw open and CRSP data and returns a dictionary containing: ‘open_df’ : processed open treasury data, ‘crsp_df’ : processed CRSP bond data, ‘merged’ : the merged and fully processed DataFrame.

When running this module directly, it computes the decile returns and saves the resulting DataFrame (with forward returns) as a parquet file.

Module Contents

Functions

process_open_source_bond_data	Process the open treasury data.
process_crsp_bond_data	Process the CRSP bond returns data.
merge_bond_data	Merge processed open treasury data and CRSP bond data on ‘cusip_date’ using an outer join. Fills missing ‘date’ and ‘cusip’ using values from the counterpart. Uses move_columns_to_front to bring ‘date’, ‘cusip’, and ‘cusip_date’ to the front.
process_merged_bond_data	Process the merged bond data:
process_all_data	Process the raw open treasury and CRSP bond data and return a dictionary with:
calculate_decile_returns	Vectorized calculation of decile portfolio returns.
plot_avg_yield_tr_ytm	Plot the average ‘yield’ and average ‘tr_ytm_match’ over time.
load_nozawa
load_merged_data

Data

DATA_DIR
OUTPUT_DIR
START_DATE
END_DATE

API

calc_nozawa_portfolio.DATA_DIR

‘Path(…)’

calc_nozawa_portfolio.OUTPUT_DIR

‘Path(…)’

calc_nozawa_portfolio.START_DATE

‘config(…)’

calc_nozawa_portfolio.END_DATE

‘config(…)’

calc_nozawa_portfolio.process_open_source_bond_data(open_df)

Process the open treasury data.

Expected columns: ‘DATE’, ‘CUSIP’, ‘tr_return’, ‘tr_ytm_match’

• Converts ‘DATE’ to datetime.

• Scales ‘tr_return’ and ‘tr_ytm_match’ by 1/100.

• Creates a unique identifier ‘cusip_date’ (CUSIP + ‘_’ + formatted DATE).

• Sorts by ‘DATE’.

calc_nozawa_portfolio.process_crsp_bond_data(crsp_df)

Process the CRSP bond returns data.

Expected columns: ‘date’, ‘cusip’, ‘year’, ‘t_yld_pt’, ‘ret_eom’, and ‘yield’

• Converts ‘date’ to datetime.

• Casts ‘year’ to int.

• Scales ‘t_yld_pt’ by 1/100.

• Forward fills the ‘yield’ column for each CUSIP.

• Creates ‘cusip_date’ (cusip + ‘_’ + formatted date).

• Sorts by [‘cusip’, ‘date’].

• Computes a forward return column ‘ret_eom_fwd’ (shift(-1) for each cusip).

calc_nozawa_portfolio.merge_bond_data(proc_open, proc_crsp)

Merge processed open treasury data and CRSP bond data on ‘cusip_date’ using an outer join. Fills missing ‘date’ and ‘cusip’ using values from the counterpart. Uses move_columns_to_front to bring ‘date’, ‘cusip’, and ‘cusip_date’ to the front.

calc_nozawa_portfolio.process_merged_bond_data(merged)

Process the merged bond data:

• Computes ‘yield_spread’ = (yield - tr_ytm_match)

• Computes ‘TTM_diff’ = (tmt - tau)

• Drops rows with missing ‘yield_spread’

• Adjusts negative ‘amount_outstanding’ by multiplying by -0.001 (fixes one incorrect data entry)

• Sorts by ‘date’ and resets the index

• Computes decile rankings for ‘yield_spread’ within each date using qcut, then adds 11 so deciles run from 11 to 20.

calc_nozawa_portfolio.process_all_data(open_df, crsp_df)

Process the raw open treasury and CRSP bond data and return a dictionary with:

• proc_open: Processed open treasury data.

• proc_crsp: Processed CRSP bond data.

• merged: The merged and fully processed DataFrame.

calc_nozawa_portfolio.calculate_decile_returns(merged)

Vectorized calculation of decile portfolio returns.

Assumes that the ‘merged’ DataFrame contains:

• ‘date’: rebalancing date,

• ‘decile’: decile assignment (values from 11 to 20),

• ‘amount_outstanding’: used as proxy for market cap,

• ‘ret_eom_fwd’: the forward return (return realized from t to t+1).

Computes the value–weighted portfolio return for each decile (using amount_outstanding as weights).

Returns

portfolio_returns_fwd : pd.DataFrame DataFrame with index as ‘date’ and columns corresponding to decile values (11, 12, …, 20) representing the forward returns. portfolio_returns_norm : pd.DataFrame Same as portfolio_returns_fwd, but with the date shifted forward by one period.

calc_nozawa_portfolio.plot_avg_yield_tr_ytm(merged, save_path=None, show=True)

Plot the average ‘yield’ and average ‘tr_ytm_match’ over time.

This function assumes that the input DataFrame contains:

• a ‘date’ column with datetime values,

• a ‘yield’ column containing periodic yield values (in decimal form), and

• a ‘tr_ytm_match’ column containing the corresponding treasury yield (in decimal form).

It computes the daily (or monthly) average of these two columns by grouping on the date, and then plots them.

Parameters

merged : pd.DataFrame DataFrame containing at least the columns ‘date’, ‘yield’, and ‘tr_ytm_match’. save_path : str or Path, optional If provided, the plot is saved to this file. show : bool, optional If True (default), the plot is displayed immediately.

Returns

fig : matplotlib.figure.Figure The figure object for the plot. ax : matplotlib.axes.Axes The axes object for the plot.

calc_nozawa_portfolio.load_nozawa(output_dir=OUTPUT_DIR)

calc_nozawa_portfolio.load_merged_data(output_dir=DATA_DIR)