Inference pipeline (Part 1)

One cycle fetches the latest GFS f000, builds both encoder inputs, runs one six-hour WeatherMesh-3 step, de-normalizes the output, checks it, and writes the artifacts.

flowchart TD
  A[GFS f000 on NOAA S3] -->|read .idx and fetch ranges| B[138 GRIB fields<br/>721 x 1440]
  B --> C[Normalize and assemble]
  C --> D[GFS-shaped encoder<br/>25 input levels]
  C --> E[HRES-shaped encoder<br/>20 input levels from GFS]
  D --> F[WeatherMesh-3<br/>encode, process, decode]
  E --> F
  F --> G[De-normalize<br/>720 x 1440 x 157]
  G --> H{Core validation passes?}
  H -->|yes| I[netCDF, plot, metadata]
  H -->|no| Q[quarantine artifacts]

Fetching GFS

• The f000 GRIB file is about 500 MB; the fetcher reads the .idx, finds the records it needs, and pulls only those byte ranges.
• It starts at the nominal cycle and walks back in six-hour steps for up to 36 hours, so normal publication delay is handled.
• A fetch of 138 fields takes about 7.5 s.

Building both inputs

• WeatherMesh-3 expects a 25-level GFS input and a 20-level HRES input; here GFS supplies both.
• Each field is normalized at its source level, mapped to the model’s 28 levels via interp_levels, and surface fields plus padding complete the 157 channels.
• Each encoder gets a (720, 1440, 157) tensor; the South Pole row is dropped to match the mesh.

Running the model

• The runner loads the released checkpoint, moves model and inputs to CUDA, and asks for lead hour 6.
• The forward pass takes about 7.1 s on an A10G.
• Fetch, preprocessing, output, metrics, and upload bring the whole cycle to under a minute.

Validation and publication

• The validator counts NaN and Inf over all 157 raw channels and applies broad bounds to temperature, MSLP, 10 m wind, the 250 hPa jet, negative humidity, and dewpoint above temperature.
• A pass writes under forecasts/ and updates latest.json; a failure writes under quarantine/, leaves the pointer alone, and fires an SNS alert.
• Precipitation is published but marked experimental (precip_reliable=false); see Limits and tradeoffs.

Measured result

init                    2026-07-10T18Z
valid time              2026-07-11T00:00Z
fetch                   7.5 s
preprocess              6.4 s
inference               7.1 s
whole cycle             54.5 s
nonfinite raw values    0
latent L2               8.579