NomadsGDASObsConv

GlobalNOAA

Import path: earth2studio.data.NomadsGDASObsConv

View source on GitHub

Documentation

Real-time GDAS conventional observations from NOAA NOMADS.

Provides near-real-time access to quality-controlled conventional (in-situ) observations from the NOAA Global Data Assimilation System (GDAS). Data is sourced from merged PrepBUFR and separate GPSRO files on NOMADS, updated 4 times daily (00z, 06z, 12z, 18z) with approximately 6-10 hours latency.

Observation types include radiosondes (ADPUPA), surface stations (ADPSFC), aircraft (AIRCAR/AIRCFT), ships and buoys (SFCSHP), wind profilers (PROFLR), satellite-derived winds (SATWND), and GPS precipitable water (GPSIPW). The gps variable reads only the combined ionosphere-corrected bending-angle observation from the separate GPSRO dump.

GPSRO rows use the shared columns with product-specific meanings: type is receiver SAID, station combines receiver/transmitter identifiers, quality is the QFRO flag table, pres is null, and elev is impact parameter minus Earth radius of curvature for gps rows. The gps_refractivity variable exposes the message's refractivity levels (ARFR, N-units) with elev set to the level height (HEIT) so consumers can derive their own vertical coordinate. Both carry the occultation's radius_curvature (ELRC) and geoid_undulation (GEODU); a level without its own tangent point sits at the occultation's reference point.

The output schema matches UFSObsConv with additional PrepBUFR quality, pressure_quality, and level_cat metadata.

Parameters:

• time_tolerance (TimeTolerance, default: timedelta64(10, 'm') ) –

  Time tolerance window for filtering observations. Accepts a single value (symmetric +/- window) or a tuple (lower, upper) for asymmetric windows, by default np.timedelta64(10, "m").

• cache (bool, default: True ) –

  Cache downloaded observation files locally, by default True.

• verbose (bool, default: True ) –

  Print download progress, by default True.

• async_timeout (int, default: 600 ) –

  Total timeout in seconds for the entire fetch, by default 600.

• async_workers (int, default: 4 ) –

  Maximum concurrent async download tasks, by default 4.

• decode_workers (int, default: 8 ) –

  Number of parallel processes for BUFR message decoding. Set to 1 to disable multiprocessing, by default 8.

• retries (int, default: 3 ) –

  Number of retry attempts per failed download with exponential backoff, by default 3.

Warning

This is a remote data source and can potentially download a large amount of data to your local machine for large requests. Each 6-hourly PrepBUFR file is approximately 60-100 MB.

Note

Additional information on the data:

• nomads.ncep.noaa.gov/pub/data/nccf/com/obsproc/prod/
• www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/document.htm
• www.emc.ncep.noaa.gov/emc/pages/numerical_forecast_systems/gfs.php

Data is retained on the NOMADS production server for approximately 2 days. Older data should be retrieved from the UFS GEFSv13 Replay dataset via UFSObsConv.

__call__

__call__(
    time: datetime | list[datetime] | TimeArray,
    variable: str | list[str] | VariableArray,
    fields: str | list[str] | Schema | None = None,
) -> DataFrame

Fetch conventional observation data.

Parameters:

• time (datetime | list[datetime] | TimeArray) –

  Timestamps to return data for (UTC).

• variable (str | list[str] | VariableArray) –

  Variables to return. Must be in GDASObsConvLexicon.

• fields (str | list[str] | Schema | None, default: None ) –

  Schema fields to include in output. None returns all fields.

Returns:

• DataFrame –

  Observation data matching the requested time/variable window.

Raises:

• KeyError –

  If a variable is not found in the lexicon.

• ValueError –

  If requested time is out of valid range.

fetch async

fetch(
    time: datetime | list[datetime] | TimeArray,
    variable: str | list[str] | VariableArray,
    fields: str | list[str] | Schema | None = None,
) -> DataFrame

Async fetch of conventional observation data.

Parameters:

• time (datetime | list[datetime] | TimeArray) –

  Timestamps to return data for (UTC).

• variable (str | list[str] | VariableArray) –

  Variables to return.

• fields (str | list[str] | Schema | None, default: None ) –

  Schema fields to include in output.

Returns:

• DataFrame –

  Observation data.

available classmethod

available(time: datetime | datetime64) -> bool

Check if date time is available on NOMADS.

Parameters:

• time (datetime | datetime64) –

  Date time to check.

Returns:

• bool –

  True if the time falls within the NOMADS retention window.