5. Verify Model Performance

5.1. wxvx Overview

EAGLE uses wxvx to verify model forecasts against either gridded analyses or point observations.

Within the EAGLE workflow, wxvx is responsible for running the underlying verification tasks and producing the statistics and plots used to evaluate forecast performance. In practice, this means you can compare both global and LAM forecasts against:

Before running verification, complete the preprocessing steps described in the Quickstart Guide so that forecast output has already been prepared for wxvx.

See the wxvx repository for further information about the project itself.

5.2. wxvx Quick Tips

  • Run the provided PreWXVX steps before starting verification.

  • Use the vx-grid-* targets to verify against gridded analyses.

  • Use the vx-obs-* targets to verify against PrepBUFR observations.

  • The global and lam targets are independent, so they can be run in parallel.

  • Check the run/<expname>/vx/*.log files if a verification job fails.

5.2.1. Running Verification

EAGLE provides four standard verification targets:

make vx-grid-global config=eagle.yaml
make vx-grid-lam config=eagle.yaml
make vx-obs-global config=eagle.yaml
make vx-obs-lam config=eagle.yaml

These commands submit batch jobs for:

  • grid-global: global forecasts verified against gridded analyses

  • grid-lam: limited-area forecasts verified against gridded analyses

  • obs-global: global forecasts verified against point observations

  • obs-lam: limited-area forecasts verified against point observations

Because these are separate jobs, they can be launched in quick succession to run in parallel.

5.2.2. Verification Output

When a verification job completes successfully, wxvx writes output beneath the corresponding verification run directory:

run/<expname>/vx/grid2{grid,obs}/{global,lam}/run/

The most useful outputs are:

  • stats/ for MET .stat files

  • plots/ for generated .png plots

  • run/<expname>/vx/*.log for verification logs

5.2.3. Additional Visualization

After verification is complete, you can generate additional postprocessed visualization outputs with:

make vis-grid-global config=eagle.yaml
make vis-grid-lam config=eagle.yaml
make vis-obs-global config=eagle.yaml
make vis-obs-lam config=eagle.yaml

These targets run eagle-tools postwxvx to create netCDF summary files and additional plots under:

run/<expname>/visualization/grid2{grid,obs}/{global,lam}/