Single Indexing

import pyearthtools.data
import warnings

with warnings.catch_warnings(action="ignore"):
    import site_archive_nci

Variables

var = 'tcwv'

Advanced Single Indexing

In addition to retrieving a single timestep, when calling a DataIndex, if the time is not fully defined, as in that the data is hourly but the time provided is merely day, all timesteps which could fit that criterion are automatically loaded. In this call an aggregation function can be provided, which will also be run.

First setting up the ERA5 DataIndex

era5 = pyearthtools.data.archive.ERA5(var)

There are a variety of ways to now index into this DataIndex. We could specify a particular time.

era5('2021-01-01T01')  # Fetch on hour of ERA5 data at a specific time and date

<xarray.Dataset> Size: 8MB
Dimensions:    (longitude: 1440, latitude: 721, time: 1)
Coordinates:
  * longitude  (longitude) float32 6kB -180.0 -179.8 -179.5 ... 179.5 179.8
  * latitude   (latitude) float32 3kB 90.0 89.75 89.5 ... -89.5 -89.75 -90.0
  * time       (time) datetime64[ns] 8B 2021-01-01T01:00:00
Data variables:
    tcwv       (time, latitude, longitude) float64 8MB dask.array<chunksize=(1, 182, 360), meta=np.ndarray>
Attributes:
    Conventions:  CF-1.6
    history:      2021-07-04 08:07:33 UTC+1000 by era5_replication_tools-1.9....
    license:      Licence to use Copernicus Products: https://apps.ecmwf.int/...
    summary:      ERA5 is the fifth generation ECMWF atmospheric reanalysis o...
    title:        ERA5 single-levels reanalysis total_column_water_vapour 202...

xarray.Dataset

• Dimensions:
  • longitude: 1440
  • latitude: 721
  • time: 1
• Coordinates: (3)
  • longitude
    (longitude)
    float32
    -180.0 -179.8 ... 179.5 179.8

    units :

    degrees_east

    long_name :

    longitude

    array([-180.  , -179.75, -179.5 , ...,  179.25,  179.5 ,  179.75],
          dtype=float32)

  • latitude
    (latitude)
    float32
    90.0 89.75 89.5 ... -89.75 -90.0

    units :

    degrees_north

    long_name :

    latitude

    array([ 90.  ,  89.75,  89.5 , ..., -89.5 , -89.75, -90.  ], dtype=float32)

  • time
    (time)
    datetime64[ns]
    2021-01-01T01:00:00

    long_name :

    time

    array(['2021-01-01T01:00:00.000000000'], dtype='datetime64[ns]')

• Data variables: (1)
  • tcwv
    (time, latitude, longitude)
    float64
    dask.array<chunksize=(1, 182, 360), meta=np.ndarray>

    units :

    kg m**-2

    long_name :

    Total column water vapour

    standard_name :

    lwe_thickness_of_atmosphere_mass_content_of_water_vapor

    Array Chunk Bytes 7.92 MiB 511.88 kiB Shape (1, 721, 1440) (1, 182, 360) Dask graph 16 chunks in 4 graph layers Data type float64 numpy.ndarray

• Indexes: (3)
  • longitude
    PandasIndex

    PandasIndex(Index([ -180.0, -179.75,  -179.5, -179.25,  -179.0, -178.75,  -178.5, -178.25,
            -178.0, -177.75,
           ...
             177.5,  177.75,   178.0,  178.25,   178.5,  178.75,   179.0,  179.25,
             179.5,  179.75],
          dtype='float32', name='longitude', length=1440))

  • latitude
    PandasIndex

    PandasIndex(Index([  90.0,  89.75,   89.5,  89.25,   89.0,  88.75,   88.5,  88.25,   88.0,
            87.75,
           ...
           -87.75,  -88.0, -88.25,  -88.5, -88.75,  -89.0, -89.25,  -89.5, -89.75,
            -90.0],
          dtype='float32', name='latitude', length=721))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2021-01-01 01:00:00'], dtype='datetime64[ns]', name='time', freq=None))

• Attributes: (5)

  Conventions :

  CF-1.6

  history :

  2021-07-04 08:07:33 UTC+1000 by era5_replication_tools-1.9.5: mv /g/data/rt52/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.tmp /g/data/rt52/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.nc 2021-07-04 08:06:34 UTC+1000 by era5_replication_tools-1.9.5: nccopy -m1G -h1G -k4 -d5 -s /g/data/id28/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.nc /g/data/rt52/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.tmp 2021-07-04 08:02:22 UTC+1000 by era5_replication_tools-1.9.5: curl --connect-timeout 20 --show-error --silent --max-time 36000 -o /g/data/id28/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.nc https://download-0000.copernicus-climate.eu/cache-compute-0000/cache/data1/adaptor.mars.internal-1625349576.8791792-11800-2-0e2a7c89-0712-4ea1-8f01-a174a40af9dd.nc 2021-07-03 21:59:46 GMT by grib_to_netcdf-2.20.0: /opt/ecmwf/mars-client/bin/grib_to_netcdf -S param -o /cache/data1/adaptor.mars.internal-1625349576.8791792-11800-2-0e2a7c89-0712-4ea1-8f01-a174a40af9dd.nc /cache/tmp/0e2a7c89-0712-4ea1-8f01-a174a40af9dd-adaptor.mars.internal-1625349543.215349-11800-2-tmp.grib

  license :

  Licence to use Copernicus Products: https://apps.ecmwf.int/datasets/licences/copernicus/

  summary :

  ERA5 is the fifth generation ECMWF atmospheric reanalysis of the global climate. This file is part of the ERA5 replica hosted at NCI Australia. For more information please see http://dx.doi.org/10.25914/5f48874388857

  title :

  ERA5 single-levels reanalysis total_column_water_vapour 20210101-20210131

Retrieving all data on a certain day.

era5('2021-01-01')  # Fetches 24 hours of data from a specific date.

<xarray.Dataset> Size: 199MB
Dimensions:    (longitude: 1440, latitude: 721, time: 24)
Coordinates:
  * longitude  (longitude) float32 6kB -180.0 -179.8 -179.5 ... 179.5 179.8
  * latitude   (latitude) float32 3kB 90.0 89.75 89.5 ... -89.5 -89.75 -90.0
  * time       (time) datetime64[ns] 192B 2021-01-01 ... 2021-01-01T23:00:00
Data variables:
    tcwv       (time, latitude, longitude) float64 199MB dask.array<chunksize=(24, 182, 360), meta=np.ndarray>
Attributes:
    Conventions:  CF-1.6
    history:      2021-07-04 08:07:33 UTC+1000 by era5_replication_tools-1.9....
    license:      Licence to use Copernicus Products: https://apps.ecmwf.int/...
    summary:      ERA5 is the fifth generation ECMWF atmospheric reanalysis o...
    title:        ERA5 single-levels reanalysis total_column_water_vapour 202...

xarray.Dataset

• Dimensions:
  • longitude: 1440
  • latitude: 721
  • time: 24
• Coordinates: (3)
  • longitude
    (longitude)
    float32
    -180.0 -179.8 ... 179.5 179.8

    units :

    degrees_east

    long_name :

    longitude

    array([-180.  , -179.75, -179.5 , ...,  179.25,  179.5 ,  179.75],
          dtype=float32)

  • latitude
    (latitude)
    float32
    90.0 89.75 89.5 ... -89.75 -90.0

    units :

    degrees_north

    long_name :

    latitude

    array([ 90.  ,  89.75,  89.5 , ..., -89.5 , -89.75, -90.  ], dtype=float32)

  • time
    (time)
    datetime64[ns]
    2021-01-01 ... 2021-01-01T23:00:00

    long_name :

    time

    array(['2021-01-01T00:00:00.000000000', '2021-01-01T01:00:00.000000000',
           '2021-01-01T02:00:00.000000000', '2021-01-01T03:00:00.000000000',
           '2021-01-01T04:00:00.000000000', '2021-01-01T05:00:00.000000000',
           '2021-01-01T06:00:00.000000000', '2021-01-01T07:00:00.000000000',
           '2021-01-01T08:00:00.000000000', '2021-01-01T09:00:00.000000000',
           '2021-01-01T10:00:00.000000000', '2021-01-01T11:00:00.000000000',
           '2021-01-01T12:00:00.000000000', '2021-01-01T13:00:00.000000000',
           '2021-01-01T14:00:00.000000000', '2021-01-01T15:00:00.000000000',
           '2021-01-01T16:00:00.000000000', '2021-01-01T17:00:00.000000000',
           '2021-01-01T18:00:00.000000000', '2021-01-01T19:00:00.000000000',
           '2021-01-01T20:00:00.000000000', '2021-01-01T21:00:00.000000000',
           '2021-01-01T22:00:00.000000000', '2021-01-01T23:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (1)
  • tcwv
    (time, latitude, longitude)
    float64
    dask.array<chunksize=(24, 182, 360), meta=np.ndarray>

    units :

    kg m**-2

    long_name :

    Total column water vapour

    standard_name :

    lwe_thickness_of_atmosphere_mass_content_of_water_vapor

    Array Chunk Bytes 190.11 MiB 12.00 MiB Shape (24, 721, 1440) (24, 182, 360) Dask graph 16 chunks in 4 graph layers Data type float64 numpy.ndarray

• Indexes: (3)
  • longitude
    PandasIndex

    PandasIndex(Index([ -180.0, -179.75,  -179.5, -179.25,  -179.0, -178.75,  -178.5, -178.25,
            -178.0, -177.75,
           ...
             177.5,  177.75,   178.0,  178.25,   178.5,  178.75,   179.0,  179.25,
             179.5,  179.75],
          dtype='float32', name='longitude', length=1440))

  • latitude
    PandasIndex

    PandasIndex(Index([  90.0,  89.75,   89.5,  89.25,   89.0,  88.75,   88.5,  88.25,   88.0,
            87.75,
           ...
           -87.75,  -88.0, -88.25,  -88.5, -88.75,  -89.0, -89.25,  -89.5, -89.75,
            -90.0],
          dtype='float32', name='latitude', length=721))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2021-01-01 00:00:00', '2021-01-01 01:00:00',
                   '2021-01-01 02:00:00', '2021-01-01 03:00:00',
                   '2021-01-01 04:00:00', '2021-01-01 05:00:00',
                   '2021-01-01 06:00:00', '2021-01-01 07:00:00',
                   '2021-01-01 08:00:00', '2021-01-01 09:00:00',
                   '2021-01-01 10:00:00', '2021-01-01 11:00:00',
                   '2021-01-01 12:00:00', '2021-01-01 13:00:00',
                   '2021-01-01 14:00:00', '2021-01-01 15:00:00',
                   '2021-01-01 16:00:00', '2021-01-01 17:00:00',
                   '2021-01-01 18:00:00', '2021-01-01 19:00:00',
                   '2021-01-01 20:00:00', '2021-01-01 21:00:00',
                   '2021-01-01 22:00:00', '2021-01-01 23:00:00'],
                  dtype='datetime64[ns]', name='time', freq=None))

• Attributes: (5)

  Conventions :

  CF-1.6

  history :

  2021-07-04 08:07:33 UTC+1000 by era5_replication_tools-1.9.5: mv /g/data/rt52/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.tmp /g/data/rt52/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.nc 2021-07-04 08:06:34 UTC+1000 by era5_replication_tools-1.9.5: nccopy -m1G -h1G -k4 -d5 -s /g/data/id28/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.nc /g/data/rt52/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.tmp 2021-07-04 08:02:22 UTC+1000 by era5_replication_tools-1.9.5: curl --connect-timeout 20 --show-error --silent --max-time 36000 -o /g/data/id28/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.nc https://download-0000.copernicus-climate.eu/cache-compute-0000/cache/data1/adaptor.mars.internal-1625349576.8791792-11800-2-0e2a7c89-0712-4ea1-8f01-a174a40af9dd.nc 2021-07-03 21:59:46 GMT by grib_to_netcdf-2.20.0: /opt/ecmwf/mars-client/bin/grib_to_netcdf -S param -o /cache/data1/adaptor.mars.internal-1625349576.8791792-11800-2-0e2a7c89-0712-4ea1-8f01-a174a40af9dd.nc /cache/tmp/0e2a7c89-0712-4ea1-8f01-a174a40af9dd-adaptor.mars.internal-1625349543.215349-11800-2-tmp.grib

  license :

  Licence to use Copernicus Products: https://apps.ecmwf.int/datasets/licences/copernicus/

  summary :

  ERA5 is the fifth generation ECMWF atmospheric reanalysis of the global climate. This file is part of the ERA5 replica hosted at NCI Australia. For more information please see http://dx.doi.org/10.25914/5f48874388857

  title :

  ERA5 single-levels reanalysis total_column_water_vapour 20210101-20210131

Providing an aggreagation function to find the ‘mean’

era5('2021-01-01', aggregation = 'mean')

<xarray.Dataset> Size: 8MB
Dimensions:    (longitude: 1440, latitude: 721)
Coordinates:
  * longitude  (longitude) float32 6kB -180.0 -179.8 -179.5 ... 179.5 179.8
  * latitude   (latitude) float32 3kB 90.0 89.75 89.5 ... -89.5 -89.75 -90.0
Data variables:
    tcwv       (latitude, longitude) float64 8MB dask.array<chunksize=(182, 360), meta=np.ndarray>
Attributes:
    Conventions:  CF-1.6
    history:      2021-07-04 08:07:33 UTC+1000 by era5_replication_tools-1.9....
    license:      Licence to use Copernicus Products: https://apps.ecmwf.int/...
    summary:      ERA5 is the fifth generation ECMWF atmospheric reanalysis o...
    title:        ERA5 single-levels reanalysis total_column_water_vapour 202...

xarray.Dataset

• Dimensions:
  • longitude: 1440
  • latitude: 721
• Coordinates: (2)
  • longitude
    (longitude)
    float32
    -180.0 -179.8 ... 179.5 179.8

    units :

    degrees_east

    long_name :

    longitude

    array([-180.  , -179.75, -179.5 , ...,  179.25,  179.5 ,  179.75],
          dtype=float32)

  • latitude
    (latitude)
    float32
    90.0 89.75 89.5 ... -89.75 -90.0

    units :

    degrees_north

    long_name :

    latitude

    array([ 90.  ,  89.75,  89.5 , ..., -89.5 , -89.75, -90.  ], dtype=float32)

• Data variables: (1)
  • tcwv
    (latitude, longitude)
    float64
    dask.array<chunksize=(182, 360), meta=np.ndarray>

    units :

    kg m**-2

    long_name :

    Total column water vapour

    standard_name :

    lwe_thickness_of_atmosphere_mass_content_of_water_vapor

    Array Chunk Bytes 7.92 MiB 511.88 kiB Shape (721, 1440) (182, 360) Dask graph 16 chunks in 6 graph layers Data type float64 numpy.ndarray

• Indexes: (2)
  • longitude
    PandasIndex

    PandasIndex(Index([ -180.0, -179.75,  -179.5, -179.25,  -179.0, -178.75,  -178.5, -178.25,
            -178.0, -177.75,
           ...
             177.5,  177.75,   178.0,  178.25,   178.5,  178.75,   179.0,  179.25,
             179.5,  179.75],
          dtype='float32', name='longitude', length=1440))

  • latitude
    PandasIndex

    PandasIndex(Index([  90.0,  89.75,   89.5,  89.25,   89.0,  88.75,   88.5,  88.25,   88.0,
            87.75,
           ...
           -87.75,  -88.0, -88.25,  -88.5, -88.75,  -89.0, -89.25,  -89.5, -89.75,
            -90.0],
          dtype='float32', name='latitude', length=721))

• Attributes: (5)

  Conventions :

  CF-1.6

  history :

  2021-07-04 08:07:33 UTC+1000 by era5_replication_tools-1.9.5: mv /g/data/rt52/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.tmp /g/data/rt52/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.nc 2021-07-04 08:06:34 UTC+1000 by era5_replication_tools-1.9.5: nccopy -m1G -h1G -k4 -d5 -s /g/data/id28/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.nc /g/data/rt52/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.tmp 2021-07-04 08:02:22 UTC+1000 by era5_replication_tools-1.9.5: curl --connect-timeout 20 --show-error --silent --max-time 36000 -o /g/data/id28/admin/incoming/era5/single-levels/reanalysis/tcwv/2021/tcwv_era5_oper_sfc_20210101-20210131.nc https://download-0000.copernicus-climate.eu/cache-compute-0000/cache/data1/adaptor.mars.internal-1625349576.8791792-11800-2-0e2a7c89-0712-4ea1-8f01-a174a40af9dd.nc 2021-07-03 21:59:46 GMT by grib_to_netcdf-2.20.0: /opt/ecmwf/mars-client/bin/grib_to_netcdf -S param -o /cache/data1/adaptor.mars.internal-1625349576.8791792-11800-2-0e2a7c89-0712-4ea1-8f01-a174a40af9dd.nc /cache/tmp/0e2a7c89-0712-4ea1-8f01-a174a40af9dd-adaptor.mars.internal-1625349543.215349-11800-2-tmp.grib

  license :

  Licence to use Copernicus Products: https://apps.ecmwf.int/datasets/licences/copernicus/

  summary :

  ERA5 is the fifth generation ECMWF atmospheric reanalysis of the global climate. This file is part of the ERA5 replica hosted at NCI Australia. For more information please see http://dx.doi.org/10.25914/5f48874388857

  title :

  ERA5 single-levels reanalysis total_column_water_vapour 20210101-20210131

Another Data source

sat = pyearthtools.data.archive.Himawari()

sat('2021-01-01T01')

<xarray.Dataset> Size: 2GB
Dimensions:                     (time: 6, latitude: 1726, longitude: 2214)
Coordinates:
  * time                        (time) datetime64[ns] 48B 2021-01-01T01:00:00...
  * latitude                    (latitude) float32 7kB -44.5 -44.48 ... -10.0
  * longitude                   (longitude) float32 9kB 112.0 112.0 ... 156.3
Data variables:
    crs                         (time) int32 24B -2147483647 ... -2147483647
    surface_global_irradiance   (time, latitude, longitude) float64 183MB dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>
    direct_normal_irradiance    (time, latitude, longitude) float64 183MB dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>
    surface_diffuse_irradiance  (time, latitude, longitude) float64 183MB dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>
    quality_mask                (time, latitude, longitude) float64 183MB dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>
    cloud_type                  (time, latitude, longitude) float32 92MB dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>
    cloud_optical_depth         (time, latitude, longitude) float64 183MB dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>
    solar_elevation             (time, latitude, longitude) float64 183MB dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>
    solar_azimuth               (time, latitude, longitude) float64 183MB dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>
    julian_date                 (time, latitude, longitude) float64 183MB dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>
Attributes: (12/52)
    Conventions:                      CF-1.7
    Metadata_Conventions:             Unidata Dataset Discovery v1.0
    acknowledgment:                   The following acknowledgement is requir...
    cdm_data_type:                    grid
    comment:                          Solar radiation data derived from satel...
    contributor_name:                 Mines ParisTech; Commonwealth of Austra...
    ...                               ...
    bias_correction_applied_meaning:  0: not applied; 1:applied
    quality:                          0
    quality_meaning:                  0: no_known_issues 1: known_issue
    project:                          Gridded Solar Observations
    references:                       Poulsen C., Majewski L. J. (2022) Gridd...
    NCO:                              netCDF Operators version 4.7.7 (Homepag...

xarray.Dataset

• Dimensions:
  • time: 6
  • latitude: 1726
  • longitude: 2214
• Coordinates: (3)
  • time
    (time)
    datetime64[ns]
    2021-01-01T01:00:00 ... 2021-01-...

    long_name :

    time

    comment :

    Start time of the corresponding satellite observation period

    array(['2021-01-01T01:00:00.000000000', '2021-01-01T01:10:00.000000000',
           '2021-01-01T01:20:00.000000000', '2021-01-01T01:30:00.000000000',
           '2021-01-01T01:40:00.000000000', '2021-01-01T01:50:00.000000000'],
          dtype='datetime64[ns]')

  • latitude
    (latitude)
    float32
    -44.5 -44.48 ... -10.02 -10.0

    standard_name :

    latitude

    long_name :

    latitude

    units :

    degrees_north

    comment :

    latitude at the centre of the pixel

    array([-44.5 , -44.48, -44.46, ..., -10.04, -10.02, -10.  ], dtype=float32)

  • longitude
    (longitude)
    float32
    112.0 112.0 112.0 ... 156.2 156.3

    standard_name :

    longitude

    long_name :

    longitude

    units :

    degrees_east

    comment :

    longitude at the centre of the pixel

    array([112.  , 112.02, 112.04, ..., 156.22, 156.24, 156.26], dtype=float32)

• Data variables: (10)
  • crs
    (time)
    int32
    -2147483647 ... -2147483647

    grid_mapping_name :

    latitude_longitude

    long_name :

    CRS definition

    semi_major_axis :

    6378137.0

    inverse_flattening :

    298.257223563

    array([-2147483647, -2147483647, -2147483647, -2147483647, -2147483647,
           -2147483647], dtype=int32)

  • surface_global_irradiance
    (time, latitude, longitude)
    float64
    dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>

    standard_name :

    surface_downwelling_shortwave_flux_in_air

    long_name :

    Total horizontal solar irradiance at surface with bias correction applied

    comment :

    Bias correction has been applied. Surface downwelling shortwave is the sum of direct and diffuse solar radiation incident on the surface, and is sometimes called "global radiation"

    units :

    W m-2

    valid_min :

    0

    valid_max :

    140000

    grid_mapping :

    crs

    Array Chunk Bytes 174.93 MiB 29.15 MiB Shape (6, 1726, 2214) (1, 1726, 2214) Dask graph 6 chunks in 15 graph layers Data type float64 numpy.ndarray

  • direct_normal_irradiance
    (time, latitude, longitude)
    float64
    dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>

    long_name :

    Direct normal irradiance at surface

    comment :

    Direct component of the surface downwelling shortwave flux on the surface

    units :

    W m-2

    valid_min :

    0

    valid_max :

    140000

    grid_mapping :

    crs

    Array Chunk Bytes 174.93 MiB 29.15 MiB Shape (6, 1726, 2214) (1, 1726, 2214) Dask graph 6 chunks in 15 graph layers Data type float64 numpy.ndarray

  • surface_diffuse_irradiance
    (time, latitude, longitude)
    float64
    dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>

    long_name :

    Diffuse component of total horizontal solar irradiance at surface

    comment :

    Diffuse component of the surface downwelling shortwave flux on the surface

    units :

    W m-2

    valid_min :

    -140000

    valid_max :

    140000

    grid_mapping :

    crs

    Array Chunk Bytes 174.93 MiB 29.15 MiB Shape (6, 1726, 2214) (1, 1726, 2214) Dask graph 6 chunks in 15 graph layers Data type float64 numpy.ndarray

  • quality_mask
    (time, latitude, longitude)
    float64
    dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>

    long_name :

    quality mask

    flag_meanings :

    not_computed no_known_issues_land known_issues_salt_lake known_issues_lake known_issues_nearshore_ocean

    flag_values :

    [0 1 2 3 4]

    grid_mapping :

    crs

    comment :

    There are known issues with some surface types (salt lakes, lakes, nearshore ocean) and some observations (not observed, ocean)

    Array Chunk Bytes 174.93 MiB 29.15 MiB Shape (6, 1726, 2214) (1, 1726, 2214) Dask graph 6 chunks in 15 graph layers Data type float64 numpy.ndarray

  • cloud_type
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>

    long_name :

    Cloud type

    flag_meanings :

    clear low medium high high_thin

    flag_values :

    [0 5 6 7 8]

    grid_mapping :

    crs

    comment :

    Cloud type retrieved from corresponding satellite observations using NWCSAF GEO. 5: Low; 6: Medium; 7: High; 8: High Thin

    Array Chunk Bytes 87.46 MiB 14.58 MiB Shape (6, 1726, 2214) (1, 1726, 2214) Dask graph 6 chunks in 15 graph layers Data type float32 numpy.ndarray

  • cloud_optical_depth
    (time, latitude, longitude)
    float64
    dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>

    long_name :

    Cloud optical depth

    units :

    1

    valid_min :

    0

    valid_max :

    25600

    grid_mapping :

    crs

    comment :

    Cloud optical depth retrieved from corresponding satellite observations using NWCSAF GEO

    Array Chunk Bytes 174.93 MiB 29.15 MiB Shape (6, 1726, 2214) (1, 1726, 2214) Dask graph 6 chunks in 15 graph layers Data type float64 numpy.ndarray

  • solar_elevation
    (time, latitude, longitude)
    float64
    dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>

    long_name :

    Solar Elevation

    units :

    degrees

    valid_min :

    -900

    valid_max :

    900

    grid_mapping :

    crs

    Array Chunk Bytes 174.93 MiB 29.15 MiB Shape (6, 1726, 2214) (1, 1726, 2214) Dask graph 6 chunks in 15 graph layers Data type float64 numpy.ndarray

  • solar_azimuth
    (time, latitude, longitude)
    float64
    dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>

    long_name :

    Solar Azimuth

    units :

    degrees

    valid_min :

    -3600

    valid_max :

    3600

    grid_mapping :

    crs

    Array Chunk Bytes 174.93 MiB 29.15 MiB Shape (6, 1726, 2214) (1, 1726, 2214) Dask graph 6 chunks in 15 graph layers Data type float64 numpy.ndarray

  • julian_date
    (time, latitude, longitude)
    float64
    dask.array<chunksize=(1, 1726, 2214), meta=np.ndarray>

    long_name :

    Julian Date

    comment :

    Julian date in units of days since noon on Monday, January 1, 4713 BC, proleptic Julian calendar

    units :

    day

    grid_mapping :

    crs

    Array Chunk Bytes 174.93 MiB 29.15 MiB Shape (6, 1726, 2214) (1, 1726, 2214) Dask graph 6 chunks in 15 graph layers Data type float64 numpy.ndarray

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2021-01-01 01:00:00', '2021-01-01 01:10:00',
                   '2021-01-01 01:20:00', '2021-01-01 01:30:00',
                   '2021-01-01 01:40:00', '2021-01-01 01:50:00'],
                  dtype='datetime64[ns]', name='time', freq=None))

  • latitude
    PandasIndex

    PandasIndex(Index([              -44.5,  -44.47999954223633, -44.459999084472656,
           -44.439998626708984,  -44.41999816894531, -44.400001525878906,
           -44.380001068115234,  -44.36000061035156,  -44.34000015258789,
            -44.31999969482422,
           ...
           -10.180000305175781,  -10.15999984741211, -10.140000343322754,
           -10.119999885559082, -10.100000381469727, -10.079999923706055,
             -10.0600004196167, -10.039999961853027, -10.020000457763672,
                         -10.0],
          dtype='float32', name='latitude', length=1726))

  • longitude
    PandasIndex

    PandasIndex(Index([             112.0,  112.0199966430664, 112.04000091552734,
           112.05999755859375, 112.08000183105469,  112.0999984741211,
           112.12000274658203, 112.13999938964844, 112.16000366210938,
           112.18000030517578,
           ...
            156.0800018310547, 156.10000610351562,  156.1199951171875,
           156.13999938964844, 156.16000366210938, 156.17999267578125,
            156.1999969482422, 156.22000122070312, 156.24000549316406,
           156.25999450683594],
          dtype='float32', name='longitude', length=2214))

• Attributes: (52)

  Conventions :

  CF-1.7

  Metadata_Conventions :

  Unidata Dataset Discovery v1.0

  acknowledgment :

  The following acknowledgement is required "Solar radiation data derived from satellite imagery processed by the Bureau of Meteorology from the geostationary satellite Himawari-8 operated by the Japan Meteorological Agency using Heliosat-4 software provided by Mines ParisTech. Contains modified Copernicus Atmosphere Monitoring Service Information 2021."

  cdm_data_type :

  grid

  comment :

  Solar radiation data derived from satellite imagery processed by the Bureau of Meteorology from the geostationary satellite Himawari-8 operated by the Japan Meteorological Agency using Heliosat-4 software provided by Mines ParisTech

  contributor_name :

  Mines ParisTech; Commonwealth of Australia, Bureau of Meteorology, Science and Innovation, Research, Weather and Environmental Prediction, Satellite Applications; Commonwealth of Australia, Bureau of Meteorology, Data and Digital, Digital Services, Business Solutions Services

  contributor_role :

  Development; Development; Operations

  creator_email :

  sat_support@bom.gov.au

  creator_name :

  Commonwealth of Australia, Bureau of Meteorology, Data and Digital, Digital Services, Business Solutions Services

  creator_url :

  http://www.bom.gov.au/satellite/

  disclaimer :

  You are solely responsible for your use of this information and you accept all risks and consequences that might arise from your use of this information. The Bureau of Meteorology ('Bureau') is not in any way liable for losses, damages, costs, expenses and liability of any kind that you or any other person may suffer or incur directly or indirectly from you using this information. This information: is subject to the uncertainties of scientific and technical research; may not be accurate, current or complete; is subject to change without notice. Please refer to http://www.bom.gov.au/other/disclaimer.shtml for disclaimer details. Neither the European Commission nor ECMWF is responsible for any use that may be made of the information it contains.

  geospatial_lat_units :

  degree_north

  geospatial_lon_units :

  degree_east

  geospatial_vertical_max :

  0

  geospatial_vertical_min :

  0

  geospatial_vertical_positive :

  up

  geospatial_vertical_resolution :

  0

  geospatial_vertical_units :

  km

  id :

  ABOM_HS4_SSI_P1S_GRID_v1.0

  institution :

  ABOM

  instrument :

  AHI

  keywords :

  EARTH SCIENCE

  keywords_vocabulary :

  NASA Global Change Master Directory (GCMD) Science Keywords

  license :

  Unless provided under a different license by the Bureau of Meteorology, this information in this data file is provided under the CC BY-NC 4.0 International Licence. The CC BY-NC 4.0 International (Creative Commons Attribution Non-Commercial CC BY-NC v.4.0 international) license summary may be found here: [ https://creativecommons.org/licenses/by-nc/4.0/]. The full license text may be found here: [ https://creativecommons.org/licenses/by-nc/4.0/legalcode ]. For commercial use, please contact Bureau of Meteorology Climate Data Services at climatedata@bom.gov.au

  naming_authority :

  au.gov.bom

  platform :

  Himawari-8

  processing_level :

  L3

  processing_version :

  heliosat-4

  product_specification :

  ABOM_HS4_SSI_P1S_GRID_v1.0

  product_suite :

  ABOM_HS4_SSI

  product_version :

  v1.1

  publisher_email :

  sat_support@bom.gov.au

  publisher_name :

  Commonwealth of Australia, Bureau of Meteorology, Data and Digital, Digital Services, Business Solutions Services

  publisher_url :

  http://www.bom.gov.au/satellite

  standard_name_vocabulary :

  CF-1.7

  summary :

  Solar radiation data derived from satellite imagery processed by the Bureau of Meteorology from the geostationary satellite Himawari-8 operated by the Japan Meteorological Agency using Heliosat-4 software provided by Mines ParisTech. Contains modified Copernicus Atmosphere Monitoring Service Information 2021.

  time_coverage_duration :

  P1S

  time_coverage_resolution :

  P1S

  title :

  Heliosat-4 Solar Radiation - Instantaneous

  geospatial_lat_max :

  -9.99

  geospatial_lat_min :

  -44.51

  geospatial_lon_max :

  156.26999999999998

  geospatial_lon_min :

  111.99

  geospatial_lat_resolution :

  0.02

  geospatial_lon_resolution :

  0.02

  bias_correction_applied :

  1

  bias_correction_applied_meaning :

  0: not applied; 1:applied

  quality :

  0

  quality_meaning :

  0: no_known_issues 1: known_issue

  project :

  Gridded Solar Observations

  references :

  Poulsen C., Majewski L. J. (2022) Gridded satellite solar observations Project: Implementation of Heliosat-4 and application of bias correction, Australian Bureau of Meteorology, Melbourne, Victoria. Qu A., Oumbe A., Blanc P., Espinar B., Gesell G., et al. (2017) Fast radiative transfer parameterisation for assessing the surface solar irradiance: The Heliosat-4 method. Meteorologische Zeitschrift, v26, p33-57. 10.1127/metz/2016/0781 . hal-01512589

  NCO :

  netCDF Operators version 4.7.7 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)