diagnostics

Offline diagnostic reports for Puzzletron artifacts.

Functions

generate_campaign_report	Generate a strict JSON/HTML report for one completed model campaign.
generate_replace_block_report	Generate an offline replace-one block/subblock score diagnostic and sidecars.
generate_vllm_stats_report	Generate the offline vLLM block-runtime diagnostic report and sidecars.

generate_campaign_report(puzzle_dir, *, model_name='Puzzletron model', expected_kd_scenarios)

Generate a strict JSON/HTML report for one completed model campaign.

Parameters:

• puzzle_dir (str | Path)

• model_name (str)

• expected_kd_scenarios (int)

Return type:

dict[str, Any]

generate_replace_block_report(puzzle_dir, *, scores_dir=None, output_dir=None, granularity='block', default_metric='normalized_mse_loss_hidden_states', default_layer_count=5, anchor_count=3, trend_relative_tolerance=0.02)

Generate an offline replace-one block/subblock score diagnostic and sidecars.

Parameters:

• puzzle_dir (str | Path)

• scores_dir (str | Path | None)

• output_dir (str | Path | None)

• granularity (str)

• default_metric (str)

• default_layer_count (int)

• anchor_count (int)

• trend_relative_tolerance (float)

Return type:

dict[str, Any]

generate_vllm_stats_report(puzzle_dir, *, stats_path=None, output_dir=None, granularity='block', anchor_count=3, trend_relative_tolerance=0.05)

Generate the offline vLLM block-runtime diagnostic report and sidecars.

Parameters:

• puzzle_dir (str | Path)

• stats_path (str | Path | None)

• output_dir (str | Path | None)

• granularity (str)

• anchor_count (int)

• trend_relative_tolerance (float)

Return type:

dict[str, Any]