Analyze study results

Analyze a study

After running a pipeline, generate statistical analysis and plots:

just analyze format-comparison

This loads data/metrics/format-comparison/quality.json, computes statistics, and writes outputs to data/analysis/format-comparison/.

What it produces

<study-id>_statistics.csv — aggregated statistics: mean, median, 5th/25th/50th/75th/95th percentiles for all metrics, grouped by format and sweep parameters
Quality plots (SVG) — SSIMULACRA2, PSNR, SSIM, Butteraugli vs the sweep parameter, showing mean and 5th-percentile (worst case)
Rate-distortion plots (SVG) — quality metrics vs bits per pixel, grouped by format
Efficiency plots (SVG) — bits per quality unit per pixel
Bits-per-pixel plots (SVG) — file size distribution with percentile bands

For crop-impact studies, analysis uses the crop parameter as the natural x-axis and reports bits_per_pixel rather than absolute file size, which makes values comparable across different crop areas.

All plots are SVG vector graphics, suitable for papers and presentations.

List studies with available results

python3 scripts/analyze_study.py --list

Advanced options

Use the script directly for more control:

# Analyze by study ID
python3 scripts/analyze_study.py format-comparison

# Analyze from a specific quality.json path
python3 scripts/analyze_study.py data/metrics/format-comparison/quality.json

# Custom output directory
python3 scripts/analyze_study.py format-comparison --output data/analysis/custom

Troubleshooting

No quality measurements found: Run the pipeline first — just pipeline <study-id> <time-budget>
Missing plots: Some plots require at least two distinct parameter values. Check the CSV statistics to see which metrics have data.