.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "examples/02_diagnostic_workflow.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_examples_02_diagnostic_workflow.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_examples_02_diagnostic_workflow.py:


Running Diagnostic Inference
============================

Basic prognostic + diagnostic inference workflow.

This example will demonstrate how to run a deterministic inference workflow that couples
a prognostic model with a diagnostic model. This diagnostic model will predict a new
atmospheric quantity from the predicted fields of the prognostic.

In this example you will learn:

- How to instantiate a prognostic model
- How to instantiate a diagnostic model
- Creating a data source and IO object
- Running the built in diagnostic workflow
- Post-processing results

.. GENERATED FROM PYTHON SOURCE LINES 38-42

Set Up
------
For this example, the built in diagnostic workflow :py:meth:`earth2studio.run.diagnostic`
will be used.

.. GENERATED FROM PYTHON SOURCE LINES 44-48

.. literalinclude:: ../../earth2studio/run.py
   :language: python
   :start-after: # sphinx - diagnostic start
   :end-before: # sphinx - diagnostic end

.. GENERATED FROM PYTHON SOURCE LINES 51-57

Thus, we need the following:

- Prognostic Model: Use the built in FourCastNet Model :py:class:`earth2studio.models.px.FCN`.
- Diagnostic Model: Use the built in precipitation AFNO model :py:class:`earth2studio.models.dx.PrecipitationAFNO`.
- Datasource: Pull data from the GFS data api :py:class:`earth2studio.data.GFS`.
- IO Backend: Save the outputs into a Zarr store :py:class:`earth2studio.io.ZarrBackend`.

.. GENERATED FROM PYTHON SOURCE LINES 59-84

.. code-block:: Python

    import os

    os.makedirs("outputs", exist_ok=True)
    from dotenv import load_dotenv

    load_dotenv()  # TODO: make common example prep function

    from earth2studio.data import GFS
    from earth2studio.io import ZarrBackend
    from earth2studio.models.dx import PrecipitationAFNO
    from earth2studio.models.px import FCN

    # Load the default model package which downloads the check point from NGC
    package = FCN.load_default_package()
    prognostic_model = FCN.load_model(package)

    package = PrecipitationAFNO.load_default_package()
    diagnostic_model = PrecipitationAFNO.load_model(package)

    # Create the data source
    data = GFS()

    # Create the IO handler, store in memory
    io = ZarrBackend()





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /localhome/user/miniconda3/envs/e2studio/lib/python3.10/site-packages/modulus/models/module.py:360: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
      model_dict = torch.load(




.. GENERATED FROM PYTHON SOURCE LINES 85-91

Execute the Workflow
--------------------
With all components initialized, running the workflow is a single line of Python code.
Workflow will return the provided IO object back to the user, which can be used to
then post process. Some have additional APIs that can be handy for post-processing or
saving to file. Check the API docs for more information.

.. GENERATED FROM PYTHON SOURCE LINES 93-102

.. code-block:: Python

    import earth2studio.run as run

    nsteps = 8
    io = run.diagnostic(
        ["2021-06-01"], nsteps, prognostic_model, diagnostic_model, data, io
    )

    print(io.root.tree())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    2024-09-20 15:59:00.610 | INFO     | earth2studio.run:diagnostic:179 - Running diagnostic workflow!
    2024-09-20 15:59:00.611 | INFO     | earth2studio.run:diagnostic:186 - Inference device: cuda
    2024-09-20 15:59:00.668 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:178 - Fetching GFS index file: 2021-06-01 00:00:00 lead 0:00:00

    Fetching GFS for 2021-06-01 00:00:00:   0%|          | 0/26 [00:00<?, ?it/s]
                                                                            
    2024-09-20 15:59:01.556 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: u10m at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:   0%|          | 0/26 [00:00<?, ?it/s]
                                                                            
    2024-09-20 15:59:01.596 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: v10m at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:   0%|          | 0/26 [00:00<?, ?it/s]
                                                                            
    2024-09-20 15:59:01.615 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: t2m at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:   0%|          | 0/26 [00:00<?, ?it/s]
                                                                            
    2024-09-20 15:59:01.635 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: sp at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:   0%|          | 0/26 [00:00<?, ?it/s]
                                                                            
    2024-09-20 15:59:01.654 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: msl at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:   0%|          | 0/26 [00:00<?, ?it/s]
    Fetching GFS for 2021-06-01 00:00:00:  19%|█▉        | 5/26 [00:00<00:00, 42.67it/s]
                                                                                    
    2024-09-20 15:59:01.674 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: t850 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  19%|█▉        | 5/26 [00:00<00:00, 42.67it/s]
                                                                                    
    2024-09-20 15:59:01.693 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: u1000 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  19%|█▉        | 5/26 [00:00<00:00, 42.67it/s]
                                                                                    
    2024-09-20 15:59:01.712 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: v1000 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  19%|█▉        | 5/26 [00:00<00:00, 42.67it/s]
                                                                                    
    2024-09-20 15:59:01.730 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: z1000 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  19%|█▉        | 5/26 [00:00<00:00, 42.67it/s]
                                                                                    
    2024-09-20 15:59:01.749 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: u850 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  19%|█▉        | 5/26 [00:00<00:00, 42.67it/s]
                                                                                    
    2024-09-20 15:59:01.768 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: v850 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  19%|█▉        | 5/26 [00:00<00:00, 42.67it/s]
    Fetching GFS for 2021-06-01 00:00:00:  42%|████▏     | 11/26 [00:00<00:00, 48.56it/s]
                                                                                     
    2024-09-20 15:59:01.787 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: z850 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  42%|████▏     | 11/26 [00:00<00:00, 48.56it/s]
                                                                                     
    2024-09-20 15:59:01.806 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: u500 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  42%|████▏     | 11/26 [00:00<00:00, 48.56it/s]
                                                                                     
    2024-09-20 15:59:01.826 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: v500 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  42%|████▏     | 11/26 [00:00<00:00, 48.56it/s]
                                                                                     
    2024-09-20 15:59:01.844 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: z500 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  42%|████▏     | 11/26 [00:00<00:00, 48.56it/s]
                                                                                     
    2024-09-20 15:59:01.863 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: t500 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  42%|████▏     | 11/26 [00:00<00:00, 48.56it/s]
                                                                                     
    2024-09-20 15:59:01.882 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: z50 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  42%|████▏     | 11/26 [00:00<00:00, 48.56it/s]
    Fetching GFS for 2021-06-01 00:00:00:  65%|██████▌   | 17/26 [00:00<00:00, 50.53it/s]
                                                                                     
    2024-09-20 15:59:01.901 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: r500 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  65%|██████▌   | 17/26 [00:00<00:00, 50.53it/s]
                                                                                     
    2024-09-20 15:59:01.919 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: r850 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  65%|██████▌   | 17/26 [00:00<00:00, 50.53it/s]
                                                                                     
    2024-09-20 15:59:01.938 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: tcwv at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  65%|██████▌   | 17/26 [00:00<00:00, 50.53it/s]
                                                                                     
    2024-09-20 15:59:01.959 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: u100m at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  65%|██████▌   | 17/26 [00:00<00:00, 50.53it/s]
                                                                                     
    2024-09-20 15:59:01.978 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: v100m at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  65%|██████▌   | 17/26 [00:00<00:00, 50.53it/s]
                                                                                     
    2024-09-20 15:59:01.998 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: u250 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  65%|██████▌   | 17/26 [00:00<00:00, 50.53it/s]
    Fetching GFS for 2021-06-01 00:00:00:  88%|████████▊ | 23/26 [00:00<00:00, 51.05it/s]
                                                                                     
    2024-09-20 15:59:02.016 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: v250 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  88%|████████▊ | 23/26 [00:00<00:00, 51.05it/s]
                                                                                     
    2024-09-20 15:59:02.035 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: z250 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  88%|████████▊ | 23/26 [00:00<00:00, 51.05it/s]
                                                                                     
    2024-09-20 15:59:02.053 | DEBUG    | earth2studio.data.gfs:_fetch_gfs_dataarray:226 - Fetching GFS grib file for variable: t250 at 2021-06-01 00:00:00_0

    Fetching GFS for 2021-06-01 00:00:00:  88%|████████▊ | 23/26 [00:00<00:00, 51.05it/s]
    Fetching GFS for 2021-06-01 00:00:00: 100%|██████████| 26/26 [00:00<00:00, 50.42it/s]
    2024-09-20 15:59:02.159 | SUCCESS  | earth2studio.run:diagnostic:200 - Fetched data from GFS
    2024-09-20 15:59:02.165 | INFO     | earth2studio.run:diagnostic:231 - Inference starting!

    Running inference:   0%|          | 0/9 [00:00<?, ?it/s]
    Running inference:  11%|█         | 1/9 [00:00<00:01,  6.05it/s]
    Running inference:  22%|██▏       | 2/9 [00:00<00:01,  4.41it/s]
    Running inference:  33%|███▎      | 3/9 [00:00<00:01,  4.10it/s]
    Running inference:  44%|████▍     | 4/9 [00:00<00:01,  3.99it/s]
    Running inference:  56%|█████▌    | 5/9 [00:01<00:01,  3.93it/s]
    Running inference:  67%|██████▋   | 6/9 [00:01<00:00,  3.82it/s]
    Running inference:  78%|███████▊  | 7/9 [00:01<00:00,  3.76it/s]
    Running inference:  89%|████████▉ | 8/9 [00:02<00:00,  3.79it/s]
    Running inference: 100%|██████████| 9/9 [00:02<00:00,  3.84it/s]
    Running inference: 100%|██████████| 9/9 [00:02<00:00,  3.94it/s]
    2024-09-20 15:59:04.450 | SUCCESS  | earth2studio.run:diagnostic:245 - Inference complete
    /
     ├── lat (720,) float64
     ├── lead_time (9,) timedelta64[h]
     ├── lon (1440,) float64
     ├── time (1,) datetime64[ns]
     └── tp (1, 9, 720, 1440) float32




.. GENERATED FROM PYTHON SOURCE LINES 103-114

Post Processing
---------------
The last step is to plot the resulting predicted total precipitation. The power of
diagnostic models is that they allow the prediction of any variable from a pre-trained
prognostic model.

.. note::
  The built in workflow will only save the direct outputs of the diagnostic. In this
  example only total precipitation is accessible for plotting. If you wish to save
  outputs of both the prognostic and diagnostic, we recommend writing a custom
  workflow.

.. GENERATED FROM PYTHON SOURCE LINES 116-158

.. code-block:: Python

    from datetime import datetime

    import cartopy.crs as ccrs
    import matplotlib.pyplot as plt
    import numpy as np

    forecast = datetime(2021, 6, 1)
    variable = "tp"
    step = 8  # lead time = 48 hrs

    plt.close("all")
    # Create a Orthographic projection of USA
    projection = ccrs.Orthographic(-100, 40)

    # Create a figure and axes with the specified projection
    fig, ax = plt.subplots(subplot_kw={"projection": projection}, figsize=(10, 6))

    # Plot the field using pcolormesh
    levels = np.arange(0.0, 0.01, 0.001)
    im = ax.contourf(
        io["lon"][:],
        io["lat"][:],
        io[variable][0, step],
        levels,
        transform=ccrs.PlateCarree(),
        vmax=0.01,
        vmin=0.00,
        cmap="terrain",
    )

    # Set title
    ax.set_title(f"{forecast.strftime('%Y-%m-%d')} - Lead time: {6*step}hrs")

    # Add coastlines and gridlines6
    ax.set_extent([220, 340, 20, 70])  # [lat min, lat max, lon min, lon max]
    ax.coastlines()
    ax.gridlines()
    plt.colorbar(
        im, ax=ax, ticks=levels, shrink=0.75, pad=0.04, label="Total precipitation (m)"
    )

    plt.savefig("outputs/02_tp_prediction.jpg")



.. image-sg:: /examples/images/sphx_glr_02_diagnostic_workflow_001.png
   :alt: 2021-06-01 - Lead time: 48hrs
   :srcset: /examples/images/sphx_glr_02_diagnostic_workflow_001.png, /examples/images/sphx_glr_02_diagnostic_workflow_001_2_00x.png 2.00x
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 18.271 seconds)

**Estimated memory usage:**  537 MB


.. _sphx_glr_download_examples_02_diagnostic_workflow.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: 02_diagnostic_workflow.ipynb <02_diagnostic_workflow.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: 02_diagnostic_workflow.py <02_diagnostic_workflow.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: 02_diagnostic_workflow.zip <02_diagnostic_workflow.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_