input

read Sentinel-2 (meta)data

Sentinel-2 is a high-resolution (10-60 m) multi-spectral satellite constellation from European Space Agency and the European Union. The following functions make is easier to read such data.

dhdt.input.read_sentinel2.dn2toa_s2(dn)

convert the digital numbers of Sentinel-2 to top of atmosphere (TOA)

Notes

https://sentinel.esa.int/web/sentinel/technical-guides/sentinel-2-msi/level-1c/algorithm

dhdt.input.read_sentinel2.get_flight_bearing_from_detector_mask_s2(Det)

Parameters

Det (numpy.array, size=(m,n), ndim={2,3}, dtype=integer) – array with numbers of the different detectors in Sentinel-2

Returns

ψ – general bearing of the satellite

Return type

float, unit=degrees

dhdt.input.read_sentinel2.get_flight_bearing_from_gnss_s2(path, spatialRef, rec_tim, fname='MTD_DS.xml')

get the direction/argument/heading of the Sentinel-2 acquisition

Parameters

• path (string) – directory where the meta-data is located

• spatialRef (osgeo.osr.SpatialReference) – projection system used

• rec_tim ({integer,float}) – time stamp of interest

• fname (string) – filename of the meta-data file

Returns

az – array with argument values, based upon the map projection given

Return type

numpy.array, size=(m,1), float

See also

get_s2_image_locations

to get the datastrip id

Notes

The metadata is scattered over the file structure of Sentinel-2, L1C

* S2X_MSIL1C_20XX...
├ AUX_DATA
├ DATASTRIP
│  └ DS_XXX_XXXX...
│     └ QI_DATA
│        └ MTD_DS.xml <- metadata about the data-strip
├ GRANULE
│  └ L1C_TXXXX_XXXX...
│     ├ AUX_DATA
│     ├ IMG_DATA
│     ├ QI_DATA
│     └ MTD_TL.xml <- metadata about the tile
├ HTML
├ rep_info
├ manifest.safe
├ INSPIRE.xml
└ MTD_MSIL1C.xml <- metadata about the product

The following acronyms are used:

• DS : datastrip

• TL : tile

• QI : quality information

• AUX : auxiliary

• MTD : metadata

• MSI : multi spectral instrument

• L1C : product specification,i.e.: level 1, processing step C

dhdt.input.read_sentinel2.get_flight_orientation_s2(ds_path, fname='MTD_DS.xml', s2_dict=None)

get the flight path and orientations of the Sentinel-2 satellite during acquisition.

It is also possible to only give the dictionary, as this has such metadata within.

Parameters

• ds_path (string) – directory where the meta-data is located

• fname (string, default='MTD_DS.xml') – filename of the meta-data file

• s2_dict (dictonary, default=None) – metadata of the Sentinel-2 platform

Returns

• sat_time (numpy.array, size=(m,1), unit=nanosec) – satellite timestamps

• sat_angles (numpy.array, size=(m,3)) – satellite orientation angles, given in ECEF

• s2_dict (dictonary) – updated with keys: “time”, “quat”

See also

get_flight_path_s2

get positions of the flight path

dhdt.input.read_sentinel2.get_flight_path_s2(ds_path, fname='MTD_DS.xml', s2_dict=None)

It is also possible to only give the dictionary, as this has such metadata within.

Parameters

• ds_path (string) – location of the metadata file

• fname (string, default='MTD_DS.xml') – name of the xml-file that has the metadata

• s2_dict (dictonary) – metadata of the Sentinel-2 platform

Returns

• sat_time (numpy.array, size=(m,1), dtype=np.datetime64, unit=ns) – time stamp of the satellite positions

• sat_xyz (numpy.array, size=(m,3), dtype=float, unit=meter) – 3D coordinates of the satellite within an Earth centered Earth fixed (ECEF) frame.

• sat_err (numpy.array, size=(m,3), dtype=float, unit=meter) – error estimate of the 3D coordinates given by “sat_xyz”

• sat_uvw (numpy.array, size=(m,3), dtype=float, unit=meter sec-1) – 3D velocity vectors of the satellite within an Earth centered Earth fixed (ECEF) frame.

• s2_dict (dictonary) – updated with keys: “gps_xyz”, “gps_uvw”, “gps_tim”, “gps_err”, “altitude”, “speed”

See also

get_flight_orientation_s2

get the quaternions of the flight path

Examples

Following the file and metadata structure of scihub:

>>> import os
>>> import numpy as np

>>> S2_dir = '/data-dump/examples/'
>>> S2_name = 'S2A_MSIL1C_20200923T163311_N0209_R140_T15MXV_20200923T200821.SAFE'
>>> fname = os.path.join(S2_dir, S2_name, 'MTD_MSIL1C.xml')
>>> s2_df = list_central_wavelength_s2()

>>> s2_df, datastrip_id = get_s2_image_locations(fname, s2_df)
>>> path_det = os.path.join(S2_dir, S2_name, 'DATASTRIP', datastrip_id[17:-7])

>>> sat_tim, sat_xyz, sat_err, sat_uvw = get_flight_path_s2(path_det)

Notes

The metadata structure of MTD_DS looks like:

* MTD_DS.xml
└ n1:Level-1C_DataStrip_ID
   ├ n1:General_Info
   ├ n1:Image_Data_Info
   ├ n1:Satellite_Ancillary_Data_Info
   │  ├ Time_Correlation_Data_List
   │  ├ Ephemeris
   │  │  ├ GPS_Number_List
   │  │  ├ GPS_Points_List
   │  │  │  └ GPS_Point
   │  │  │     ├ POSITION_VALUES
   │  │  │     ├ POSITION_ERRORS
   │  │  │     ├ VELOCITY_VALUES
   │  │  │     ├ VELOCITY_ERRORS
   │  │  │     ├ GPS_TIME
   │  │  │     ├ NSM
   │  │  │     ├ QUALITY_INDEX
   │  │  │     ├ GDOP
   │  │  │     ├ PDOP
   │  │  │     ├ TDOP
   │  │  │     ├ NOF_SV
   │  │  │     └ TIME_ERROR
   │  │  └ AOCS_Ephemeris_List
   │  ├ Attitudes
   │  ├ Thermal_Data
   │  └ ANC_DATA_REF
   │
   ├ n1:Quality_Indicators_Info
   └ n1:Auxiliary_Data_Info

The following acronyms are used:

• AOCS : attitude and orbit control system

• CAMS : Copernicus atmosphere monitoring service

• DEM : digital elevation model

• GDOP : geometric dilution of precision

• GPS : global positioning system

• GRI : global reference image

• IERS : international earth rotation and reference systems service

• IMT : instrument measurement time

• NSM : navigation solution method

• NOF SV : number of space vehicles

• PDOP : position dilution of precision

• TDOP : time dilution of precision

dhdt.input.read_sentinel2.get_integration_and_sampling_time_s2(ds_path, fname='MTD_DS.xml', s2_dict=None)

Parameters

• ds_path (string) – location where metadata of datastrip is situated

• fname (string, default='MTD_DS.xml') – metadata filename

• s2_dict (dictionary, default=None) – metadata dictionary, can be used instead of the string based method

Notes

The metadata is scattered over the file structure of Sentinel-2, L1C

* S2X_MSIL1C_20XX...
├ AUX_DATA
│  └ DS_XXX_XXXX...
│     └ QI_DATA
│        └ MTD_DS.xml <- metadata about the data-strip
├ GRANULE
│  └ L1C_TXXXX_XXXX...
│     ├ AUX_DATA
│     ├ IMG_DATA
│     ├ QI_DATA
│     └ MTD_TL.xml <- metadata about the tile
├ HTML
├ rep_info
├ manifest.safe
├ INSPIRE.xml
└ MTD_MSIL1C.xml <- metadata about the product

The following acronyms are used:

• DS : datastrip

• TL : tile

• QI : quality information

• AUX : auxiliary

• MTD : metadata

• MSI : multi spectral instrument

• L1C : product specification,i.e.: level 1, processing step C

dhdt.input.read_sentinel2.get_intrinsic_camera_mat_s2(s2_df, det, boi)

Parameters

• s2_df (pandas.DataFrame) – metadata and general multispectral information about the MSI instrument that is onboard Sentinel-2

• det (list) – list with detector numbers that are within the tile

• boi (list) – list with bands of interest

Returns

F – fundamental matrix

Return type

numpy.array, size=(3,3)

Notes

The pushbroom detector onboard Sentinel-2 has the following configuration:

                      nadir
det1                    |
====#   #===#   #===#   #===#   #===#   #===#
   #===#   #===#   #===#   #===#   #===#   #====
                        |                   det12
===== : 2592 pixels for 10 meter, 1296 pixels for 20-60 meter
   # : 98 pixels overlap for 20 meter data

References

HG94

Hartley & Gupta, “Linear pushbroom cameras” Proceedings of the European conference on computer vision, 1994.

Mo10

Moons et al. “3D reconstruction from multiple images part 1: Principles.” Foundations and trends® in computer graphics and vision, vol.4(4) pp.287-404, 2010.

dhdt.input.read_sentinel2.list_central_wavelength_msi()

create dataframe with metadata about Sentinel-2

Returns

df – metadata and general multispectral information about the MSI instrument that is onboard Sentinel-2, having the following collumns:

• center_wavelength, unit=µm : central wavelength of the band

• full_width_half_max, unit=µm : extent of the spectral sensativity

• bandid : number for identification in the meta data

• resolution, unit=m : spatial resolution of a pixel

• along_pixel_size, unit=µm : physical size of the sensor

• across_pixel_size, unit=µm : size of the photosensative sensor

• focal_length, unit=m : lens characteristic of the telescope

• field_of_view, unit=degrees : angle of swath of instrument

• crossdetector_parallax, unit=degress : in along-track direction

• name : general name of the band, if applicable

• solar_illumination, unit=W m-2 µm-1 :

mostly following the naming convension of the STAC EO extension [stac]

Return type

pandas.dataframe

See also

dhdt.input.read_landsat.list_central_wavelength_oli

for the instrument data of Landsat 8 or 9

dhdt.input.read_aster.list_central_wavelength_as

for the instrument data of ASTER on the Terra satellite

dhdt.input.read_venus.list_central_wavelength_vssc

for the instrument data of VSSC on the VENµS satellite

Notes

The Multi Spectral instrument (MSI) has a Tri Mirror Anastigmat (TMA) telescope, which is opened at F/4 and has a focal length of 60 centimeter.

The crossdetector parallex is given here as well, see [La15]. While the crossband parallax is 0.018 for VNIR and 0.010 for the SWIR detector, respectively. The dimensions of the Silicon CMOS detector are given in [MG10]. While the SWIR detector is based on a MCT sensor.

The following acronyms are used:

• MSI : multi-spectral instrument

• MCT : mercury cadmium telluride, HgCdTe

• TMA : trim mirror anastigmat

• VNIR : very near infra-red

• SWIR : short wave infra-red

• CMOS : silicon complementary metal oxide semiconductor, Si

Examples

make a selection by name:

>>> boi = ['red', 'green', 'blue', 'near infrared']
>>> s2_df = list_central_wavelength_msi()
>>> s2_df = s2_df[s2_df['common_name'].isin(boi)]
>>> s2_df
         wavelength  bandwidth  resolution       name  bandid
B02         492         66          10           blue       1
B03         560         36          10          green       2
B04         665         31          10            red       3
B08         833        106          10  near infrared       7

similarly you can also select by pixel resolution:

>>> s2_df = list_central_wavelength_msi()
>>> tw_df = s2_df[s2_df['gsd']==20]
>>> tw_df.index
Index(['B05', 'B06', 'B07', 'B8A', 'B11', 'B12'], dtype='object')

References

La15

Languille et al. “Sentinel-2 geometric image quality commissioning: first results” Proceedings of the SPIE, 2015.

MG10

Martin-Gonthier et al. “CMOS detectors for space applications: From R&D to operational program with large volume foundry”, Proceedings of the SPIE conference on sensors, systems, and next generation satellites XIV, 2010.

stac

https://github.com/stac-extensions/eo

dhdt.input.read_sentinel2.read_band_s2(path, band=None)

This function takes as input the Sentinel-2 band name and the path of the folder that the images are stored, reads the image and returns the data as an array

Parameters

• path (string) – path of the folder, or full path with filename as well

• band (string, optional) – Sentinel-2 band name, for example ‘04’, ‘8A’.

Returns

• data (numpy.array, size=(_,_)) – array of the band image

• spatialRef (string) – projection

• geoTransform (tuple) – affine transformation coefficients

• targetprj (osr.SpatialReference object) – spatial reference

See also

list_central_wavelength_msi

creates a dataframe for the MSI instrument

read_stack_s2

reading several Sentinel-2 bands at once into a stack

dhdt.generic.read_geo_image

basic function to import geographic imagery

data

Examples

>>> path = '/GRANULE/L1C_T15MXV_A027450_20200923T163313/IMG_DATA/'
>>> band = '02'
>>> _,spatialRef,geoTransform,targetprj = read_band_s2(path, band)

>>> spatialRef
'PROJCS["WGS 84 / UTM zone 15S",GEOGCS["WGS 84",DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],
UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",-93],
PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],
PARAMETER["false_northing",10000000],UNIT["metre",1,
AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],
AUTHORITY["EPSG","32715"]]'
>>> geoTransform
(600000.0, 10.0, 0.0, 10000000.0, 0.0, -10.0)
>>> targetprj
<osgeo.osr.SpatialReference; proxy of <Swig Object of type
'OSRSpatialReferenceShadow *' at 0x7f9a63ffe450> >

dhdt.input.read_sentinel2.read_cloud_mask(path_meta, geoTransform)

Parameters

• path_meta (string) – directory where meta data is situated, ‘MSK_CLOUDS_B00.gml’ is typically the file of interest

• geoTransform (tuple) – affine transformation coefficients

Returns

msk_clouds – array which highlights where clouds could be

Return type

numpy.array, size=(m,n), dtype=int

dhdt.input.read_sentinel2.read_detector_mask(path_meta, boi, geoTransform)

create array of with detector identification

Sentinel-2 records in a pushbroom fasion, collecting reflectance with a ground resolution of more than 270 kilometers. This data is stacked in the flight direction, and then cut into granules. However, the sensorstrip inside the Multi Spectral Imager (MSI) is composed of several CCD arrays.

This function collects the geometry of these sensor arrays from the meta- data. Since this is stored in a gml-file.

Parameters

• path_meta (string) – path where the meta-data is situated.

• boi (pandas.DataFrame) – list with bands of interest

• geoTransform (tuple, size={(6,),(8,)}) – affine transformation coefficients

Returns

det_stack – array where each pixel has the ID of the detector, of a specific band

Return type

numpy.ma.array, size=(msk_dim[0:1],len(boi)), dtype=int8

See also

list_central_wavelength_msi

creates a dataframe for the MSI instrument

read_view_angles_s2

read grid of Sentinel-2 observation angles

Notes

The metadata is scattered over the file structure of Sentinel-2, L1C

* S2X_MSIL1C_20XX...
├ AUX_DATA
├ DATASTRIP
│  └ DS_XXX_XXXX...
│     └ QI_DATA
│        └ MTD_DS.xml <- metadata about the data-strip
├ GRANULE
│  └ L1C_TXXXX_XXXX...
│     ├ AUX_DATA
│     ├ IMG_DATA
│     ├ QI_DATA
│     └ MTD_TL.xml <- metadata about the tile
├ HTML
├ rep_info
├ manifest.safe
├ INSPIRE.xml
└ MTD_MSIL1C.xml <- metadata about the product

The following acronyms are used:

• DS : datastrip

• TL : tile

• QI : quality information

• AUX : auxiliary

• MTD : metadata

• MSI : multi spectral instrument

• L1C : product specification,i.e.: level 1, processing step C

Examples

>>> path_meta = '/GRANULE/L1C_T15MXV_A027450_20200923T163313/QI_DATA'
>>> boi = ['red', 'green', 'blue', 'near infrared']
>>> s2_df = list_central_wavelength_msi()
>>> boi_df = s2_df[s2_df['common_name'].isin(boi)]
>>> geoTransform = (600000.0, 10.0, 0.0, 10000000.0, 0.0, -10.0)
>>>
>>> det_stack = read_detector_mask(path_meta, boi_df, geoTransform)

dhdt.input.read_sentinel2.read_detector_time_s2(path, fname='MTD_DS.xml', s2_df=None)

get the detector metadata of the relative detector timing

Parameters

• path (string) – path where the meta-data is situated

• fname (string) – file name of the metadata.

Returns

• det_time (numpy.array, numpy.datetime64) – detector time

• det_name (list, size=(13,)) – list of the different detector codes

• det_meta (numpy.array, size=(13,4)) –

  metadata of the individual detectors, each detector has the following:

  1. spatial resolution [m]

  2. minimal spectral range [nm]

  3. maximum spectral range [nm]

  4. mean spectral range [nm]

• line_period (float, numpy.timedelta64) – temporal sampling distance

Notes

The sensor blocks are arranged as follows, with ~98 pixels overlap:

    ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐
    |DET02|   |DET04|   |DET06|   |SCA08|   |SCA10|   |SCA12|
    └-----┘   └-----┘   └-----┘   └-----┘   └-----┘   └-----┘
┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐
|DET01|   |DET03|   |SCA05|   |SCA07|   |SCA09|   |SCA11|
└-----┘   └-----┘   └-----┘   └-----┘   └-----┘   └-----┘

a forward looking array, while band order is reverse for aft looking
<-2592 10m pixels->
┌-----------------┐    #*# satellite
|       B02       |     |
├-----------------┤     | flight
|       B08       |     | direction
├-----------------┤     |
|       B03       |     v
├-----------------┤
       etc.
the detector order is B02, B08, B03, B10, B04, B05,
B11, B06, B07, B8A, B12, B01 and B09

dhdt.input.read_sentinel2.read_geotransform_s2(path, fname='MTD_TL.xml', resolution=10)

Parameters

• path (string) – location where the meta data is situated

• fname (string) – file name of the meta-data file

• resolution ({float,integer}, unit=meters, default=10) – resolution of the grid

Returns

geoTransform – affine transformation coefficients

Return type

tuple, size=(1,6)

Notes

The metadata is scattered over the file structure of Sentinel-2, L1C

* S2X_MSIL1C_20XX...
├ AUX_DATA
├ DATASTRIP
│  └ DS_XXX_XXXX...
│     └ QI_DATA
│        └ MTD_DS.xml <- metadata about the data-strip
├ GRANULE
│  └ L1C_TXXXX_XXXX...
│     ├ AUX_DATA
│     ├ IMG_DATA
│     ├ QI_DATA
│     └ MTD_TL.xml <- metadata about the tile
├ HTML
├ rep_info
├ manifest.safe
├ INSPIRE.xml
└ MTD_MSIL1C.xml <- metadata about the product

The metadata structure is as follows:

• MTD_TL.xml

└ n1:Level-1C_Tile_ID

├ n1:General_Info ├ n1:Geometric_Info │ ├ Tile_Geocoding │ │ ├ HORIZONTAL_CS_NAME │ │ ├ HORIZONTAL_CS_CODE │ │ ├ Size : resolution={“10”,”20”,”60”} │ │ │ ├ NROWS │ │ │ └ NCOLS │ │ └ Geoposition │ │ ├ ULX │ │ ├ ULY │ │ ├ XDIM │ │ └ YDIM │ └ Tile_Angles └ n1:Quality_Indicators_Info

The following acronyms are used:

• DS : datastrip

• TL : tile

• QI : quality information

• AUX : auxiliary

• MTD : metadata

• MSI : multi spectral instrument

• L1C : product specification,i.e.: level 1, processing step C

dhdt.input.read_sentinel2.read_mean_sun_angles_s2(path, fname='MTD_TL.xml')

Read the xml-file of the Sentinel-2 scene and extract the mean sun angles.

Parameters

path (string) – path where xml-file of Sentinel-2 is situated

Returns

• Zn (float) – Mean solar zentih angle of the scene, in degrees.

• Az (float) – Mean solar azimuth angle of the scene, in degrees

Notes

The azimuth angle declared in the following coordinate frame:

     ^ North & y
     |
- <--|--> +
     |
     └----> East & x

The angles related to the sun are as follows:

surface normal              * sun
^                     ^    /
|                     |   /
|-- zenith angle      |  /
| /                   | /|
|/                    |/ | elevation angle
└----                 └------

The metadata is scattered over the file structure of Sentinel-2, L1C

* S2X_MSIL1C_20XX...
├ AUX_DATA
├ DATASTRIP
│  └ DS_XXX_XXXX...
│     └ QI_DATA
│        └ MTD_DS.xml <- metadata about the data-strip
├ GRANULE
│  └ L1C_TXXXX_XXXX...
│     ├ AUX_DATA
│     ├ IMG_DATA
│     ├ QI_DATA
│     └ MTD_TL.xml <- metadata about the tile
├ HTML
├ rep_info
├ manifest.safe
├ INSPIRE.xml
└ MTD_MSIL1C.xml <- metadata about the product

The following acronyms are used:

• s2 : Sentinel-2

• DS : datastrip

• TL : tile

• QI : quality information

• AUX : auxiliary

• MTD : metadata

• MSI : multi spectral instrument

• L1C : product specification,i.e.: level 1, processing step C

dhdt.input.read_sentinel2.read_orbit_number_s2(path, fname='MTD_MSIL1C.xml')

read the orbit number of the image

dhdt.input.read_sentinel2.read_sensing_time_s2(path, fname='MTD_TL.xml')

Parameters

• path (string) – path where the meta-data is situated

• fname (string) – file name of the metadata.

Returns

rec_time – time of image acquisition

Return type

numpy.datetime64

See also

get_s2_granual_id

Notes

The recording time is the average sensing within the tile, which is a combination of forward and backward looking datastrips. Schematically, this might look like:

x recording time of datastrip, i.e.: the start of the strip
× recording time of tile, i.e.: the weighted average
                           _________  datastrip
                 ____x____/        /_________
                /        /        /        /
               /        /        /        /
              /        /        /        /
             /        /        /        /
            /        /        /        /
           /        /        /        /
          /    ┌---/------┐ /        /
         /     |  /       |/        /
        /      | /  ×     /        /
       /       |/        /|       /
      /        /--------/-┘      /
     /        /  tile  /        /
    /        /        /        /
   /        /        /        /
  /        /        /        /
 /        /        /        /
/        _________/        /
_________        /________/

Examples

demonstrate the code when in a Scihub data structure

>>> import os
>>> fpath = '/Users/Data/'
>>> sname = 'S2A_MSIL1C_20200923T163311_N0209_R140_T15MXV_20200923T200821.SAFE'
>>> fname = 'MTD_MSIL1C.xml'
>>> s2_path = os.path.join(fpath, sname, fname)
>>> s2_df,_ = get_s2_image_locations(fname, s2_df)
>>> s2_df['filepath']
B02 'GRANULE/L1C_T15MXV_A027450_20200923T163313/IMG_DATA/T115MXV...'
B03 'GRANULE/L1C_T15MXV_A027450_20200923T163313/IMG_DATA/T15MXV...'
>>> full_path = '/'.join(s2_df['filepath'][0].split('/')[:-2])
'GRANULE/L1C_T15MXV_A027450_20200923T163313'
>>> rec_time = read_sensing_time_s2(path, fname='MTD_TL.xml')
>>> rec_time

dhdt.input.read_sentinel2.read_stack_s2(s2_df)

read imagery data of interest into an three dimensional np.array

Parameters

s2_df (pandas.Dataframe) – metadata and general multispectral information about the MSI instrument that is onboard Sentinel-2

Returns

• im_stack (numpy.array, size=(_,_,_)) – array of the band image

• spatialRef (string) – projection

• geoTransform (tuple, size=(8,1)) – affine transformation coefficients

• targetprj (osr.SpatialReference object) – spatial reference

See also

list_central_wavelength_msi

creates a dataframe for the MSI instrument

get_s2_image_locations

provides dataframe with specific file locations

read_band_s2

reading a single Sentinel-2 band

Examples

>>> s2_df = list_central_wavelength_msi()
>>> s2_df = s2_df[s2_df['gsd'] == 10]
>>> s2_df,_ = get_s2_image_locations(IM_PATH, s2_df)

>>> im_stack, spatialRef, geoTransform, targetprj = read_stack_s2(s2_df)

dhdt.input.read_sentinel2.read_sun_angles_s2(path, fname='MTD_TL.xml')

This function reads the xml-file of the Sentinel-2 scene and extracts an array with sun angles, as these vary along the scene.

Parameters

path (string) – path where xml-file of Sentinel-2 is situated

Returns

• Zn (numpy.array, size=(m,n), dtype=float) – array of the solar zenith angles, in degrees.

• Az (numpy.array, size=(m,n), dtype=float) – array of the solar azimuth angles, in degrees.

Notes

The computation of the solar angles is done in two steps: 1) Computation of the solar angles in J2000; 2) Transformation of the vector to the mapping frame.

• 1. The outputs of the first step is the solar direction normalized vector with the Earth-Sun distance, considering that the direction of the sun is the same at the centre of the Earth and at the centre of the Sentinel-2 satellite.

• 2. Attitude of the satellite platform are used to rotate the solar vector to the mapping frame. Also Ground Image Calibration Parameters (GICP Diffuser Model) are used to transform from the satellite to the diffuser, as Sentinel-2 has a forward and backward looking sensor configuration. The diffuser model also has stray-light correction and a Bi-Directional Reflection Function model.

The angle(s) are declared in the following coordinate frame:

     ^ North & y
     |
- <--|--> +
     |
     +----> East & x

The angles related to the sun are as follows:

surface normal              * sun
^                     ^    /
|                     |   /
|-- zenith angle      |  /
| /                   | /|
|/                    |/ | elevation angle
+----                 +------

Two different coordinate system are used here:

indexing   |           indexing    ^ y
system 'ij'|           system 'xy' |
           |                       |
           |       i               |       x
   --------+-------->      --------+-------->
           |                       |
           |                       |
image      | j         map         |
based      v           based       |

The metadata is scattered over the file structure of Sentinel-2, L1C

* S2X_MSIL1C_20XX...
├ AUX_DATA
├ DATASTRIP
│  └ DS_XXX_XXXX...
│     └ QI_DATA
│        └ MTD_DS.xml <- metadata about the data-strip
├ GRANULE
│  └ L1C_TXXXX_XXXX...
│     ├ AUX_DATA
│     ├ IMG_DATA
│     ├ QI_DATA
│     └ MTD_TL.xml <- metadata about the tile
├ HTML
├ rep_info
├ manifest.safe
├ INSPIRE.xml
└ MTD_MSIL1C.xml <- metadata about the product

The metadata structure looks like:

* MTD_TL.xml
└ n1:Level-1C_Tile_ID
   ├ n1:General_Info
   ├ n1:Geometric_Info
   │  ├ Tile_Geocoding
   │  └ Tile_Angles
   │     ├ Sun_Angles_Grid
   │     │  ├ Zenith
   │     │  │  ├ COL_STEP
   │     │  │  ├ ROW_STEP
   │     │  │  └ Values_List
   │     │  └ Azimuth
   │     │     ├ COL_STEP
   │     │     ├ ROW_STEP
   │     │     └ Values_List
   │     ├ Mean_Sun_Angle
   │     └ Viewing_Incidence_Angles_Grids
   └ n1:Quality_Indicators_Info

The following acronyms are used:

• s2 : Sentinel-2

• DS : datastrip

• TL : tile

• QI : quality information

• AUX : auxiliary

• MTD : metadata

• MSI : multi spectral instrument

• L1C : product specification,i.e.: level 1, processing step C

dhdt.input.read_sentinel2.read_view_angles_s2(path, fname='MTD_TL.xml', det_stack=array([], dtype=float64), boi_df=None)

This function reads the xml-file of the Sentinel-2 scene and extracts an array with viewing angles of the MSI instrument.

Parameters

• path (string) – path where xml-file of Sentinel-2 is situated

• fname (string) – the name of the metadata file, sometimes this is changed

• boi_df (pandas.dataframe, default=4) – each band has a somewhat minute but different view angle

Returns

• Zn (numpy.ma.array, size=(m,n), dtype=float) – masked array of the solar zenith angles, in degrees.

• Az (numpy.ma.array, size=(m,n), dtype=float) – masked array of the solar azimuth angles, in degrees.

See also

list_central_wavelength_s2

Notes

The azimuth angle is declared in the following coordinate frame:

     ^ North & y
     |
- <--|--> +
     |
     └----> East & x

The angles related to the satellite are as follows:

     #*#                   #*# satellite
^    /                ^    /|
|   /                 |   / | nadir
|-- zenith angle      |  /  v
| /                   | /|
|/                    |/ | elevation angle
└----- surface        └------

The metadata is scattered over the file structure of Sentinel-2, L1C

* S2X_MSIL1C_20XX...
├ AUX_DATA
├ DATASTRIP
│  └ DS_XXX_XXXX...
│     └ QI_DATA
│        └ MTD_DS.xml <- metadata about the data-strip
├ GRANULE
│  └ L1C_TXXXX_XXXX...
│     ├ AUX_DATA
│     ├ IMG_DATA
│     ├ QI_DATA
│     └ MTD_TL.xml <- metadata about the tile
├ HTML
├ rep_info
├ manifest.safe
├ INSPIRE.xml
└ MTD_MSIL1C.xml <- metadata about the product

The metadata structure looks like:

* MTD_TL.xml
└ n1:Level-1C_Tile_ID
   ├ n1:General_Info
   ├ n1:Geometric_Info
   │  ├ Tile_Geocoding
   │  └ Tile_Angles
   │     ├ Sun_Angles_Grid
   │     ├ Mean_Sun_Angle
   │     └ Viewing_Incidence_Angles_Grids
   │        ├ Zenith
   │        │  ├ COL_STEP
   │        │  ├ ROW_STEP
   │        │  └ Values_List
   │        └ Azimuth
   │           ├ COL_STEP
   │           ├ ROW_STEP
   │           └ Values_List
   └ n1:Quality_Indicators_Info

The following acronyms are used:

• s2 : Sentinel-2

• DS : datastrip

• TL : tile

• QI : quality information

• AUX : auxiliary

• MTD : metadata

• MSI : multi spectral instrument

• L1C : product specification,i.e.: level 1, processing step C

read Landsat (meta)data

Landsat 8 & 9 are the latest fleet of satellites that are part of a legacy. Both satellites are build by NASA and data provision is done through USGS. The following function makes reading such data easier.

dhdt.input.read_landsat.get_sca_numbering_ls(ang, boi='BAND04')

Parameters

• ang (dictonary) – metadata of the Landsat scene

• boi (string) – band of intertest, typically the data of band 4 is given by VAA.tif

Returns

Sca – array with detector numbers

Return type

np.array, size=(m,n), dtype=int, range=0…14

Notes

The sensor blocks are arranged as follows, with ~28 pixels overlap:

    ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐
    |SCA02|   |SCA04|   |SCA06|   |SCA08|   |SCA10|   |SCA12|   |SCA14|
    └-----┘   └-----┘   └-----┘   └-----┘   └-----┘   └-----┘   └-----┘
┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐   ┌-----┐
|SCA01|   |SCA03|   |SCA05|   |SCA07|   |SCA09|   |SCA11|   |SCA13|
└-----┘   └-----┘   └-----┘   └-----┘   └-----┘   └-----┘   └-----┘

See also

read_metadata_ls

dhdt.input.read_landsat.list_central_wavelength_oli()

create dataframe with metadata about OLI on Landsat8 or Landsat9

Returns

df – metadata and general multispectral information about the OLI instrument that is onboard Landsat8, having the following collumns:

• wavelength, unit=µm : central wavelength of the band

• bandwidth, unit=µm : extent of the spectral sensativity

• bandid : number for identification in the meta data

• resolution, unit=m : spatial resolution of a pixel

• field_of_view, unit=degrees : angle of swath of instrument

• name : general name of the band, if applicable

Return type

pandas.DataFrame

See also

dhdt.input.read_sentinel2.list_central_wavelength_msi

for the instrument data of MSI on Sentinel-2

dhdt.input.read_aster.list_central_wavelength_as

for the instrument data of ASTER on the Terra satellite

dhdt.input.read_venus.list_central_wavelength_vssc

for the instrument data of VSSC on the VENµS satellite

Notes

The time lag between panchromatic and red is 0.52 seconds, while the time difference between panchromatic and the green band is 0.65 sec., see [dM16]. The detector samping time is 4.2 msec, while the total staring time is 1.2 seconds. The detector arrays (or senso chip assemblies: SCA) of landsat are configured as follows:

a forward looking SCA, while bandorder is reverse for aft looking
<--494 pixels-->
┌--------------┐    #*# satellite
|    green     |     |
├--------------┤     | flight
|     red      |     | direction
├--------------┤     |
|near infrared |     v
├--------------┤
|  not in use  |
├--------------┤
|    blue      |
├--------------┤
| panchromatic |
└--------------┘
<--988 pixels-->

References

dM16

de Michele, et al. “Volcanic plume elevation model and its velocity derived from Landsat 8” Remote sensing of environment, vol.176 pp.219-224, 2016.

Examples

make a selection by name:

>>> boi = ['red', 'green', 'blue', 'near infrared']
>>> ls_df = list_central_wavelength_oli()
>>> ls_df = ls_df[ls_df['name'].isin(boi)]
>>> ls_df
         wavelength  bandwidth  resolution   common_name  bandid
B02         482         60          30           blue       2
B03         561         57          30          green       3
B04         655         37          30            red       4
B05         865         28          30  near infrared       5

similarly you can also select by pixel resolution:

>>> ls_df = list_central_wavelength_oli()
>>> pan_df = ls_df[ls_df['resolution']==15]
>>> pan_df.index
Index(['B08'], dtype='object')

dhdt.input.read_landsat.read_band_ls(path, band='B8')

read landsat image from path

pathstring

path of the folder, or full path with filename as well

bandstring, optional

Landsat8 band name, for example ‘4’, ‘B8’.

datanp.array, size=(_,_)

array of the band image

spatialRefstring

projection

geoTransformtuple

affine transformation coefficients

targetprjosr.SpatialReference object

spatial reference

list_central_wavelength_ls

>>> path = '/LC08_L1TP_018060_20200904_20200918_02_T1/'
>>> band = '2'
>>> _,spatialRef,geoTransform,targetprj = read_band_ls(path, band)

>>> spatialRef

‘PROJCS[“WGS 84 / UTM zone 15N”,GEOGCS[“WGS 84”,DATUM[“WGS_1984”, …

>>> geoTransform
(626385.0, 30.0, 0.0, 115815.0, 0.0, -30.0)
>>> targetprj
<osgeo.osr.SpatialReference; proxy of <Swig Object of type 'OSRSpatial ...

dhdt.input.read_landsat.read_metadata_ls(dir_path, suffix='MTL.txt')

Convert Landsat MTL file to dictionary of metadata values

read Terra ASTER data

The ASTER instrument onboard of the Terra satellite has been collecting data since 1999, but will be decommisioned soon. Nonetheless, its archive is very valuable and the following functions make reading such data easier.

dhdt.input.read_aster.get_flight_path_as(as_path, fname='AST_L1A_*.Ancillary_Data.txt', as_dict=None)

Parameters

• as_path (string) –

• fname (string) – file name of the accillary data

• as_dict (dictionary) –

See also

dhdt.input.read_aster.list_central_wavelength_as

dhdt.input.read_aster.list_central_wavelength_as()

create dataframe with metadata about Terra’s ASTER

Returns

df – metadata and general multispectral information about the MSI instrument that is onboard Sentinel-2, having the following collumns:

• wavelength : central wavelength of the band

• bandwidth : extent of the spectral sensativity

• bandid : number for identification in the meta data

• resolution : spatial resolution of a pixel

• name : general name of the band, if applicable

• irradiance :

Return type

pandas.DataFrame

See also

dhdt.input.read_sentinel2.list_central_wavelength_msi

for the instrument data of MSI on Sentinel-2

dhdt.input.read_landsat.list_central_wavelength_oli

for the instrument data of Landsat 8 or 9

dhdt.input.read_venus.list_central_wavelength_vssc

for the instrument data of VSSC on the VENµS satellite

Examples

make a selection by name:

>>> boi = ['red', 'green', 'blue']
>>> as_df = list_central_wavelength_as()
>>> as_df = as_df[as_df['name'].isin(boi)]
>>> as_df
     wavelength  bandwidth  resolution   name  bandid  irradiance
B01         560         80          15   blue       0      1848.0
B02         660         60          15  green       1      1549.0
B3B         810        100          15    red       2      1114.0
B3N         810        100          15    red       3      1114.0

similarly you can also select by pixel resolution:

>>> as_df = list_central_wavelength_as()
>>> sw_df = as_df[as_df['gsd']==30]
>>> sw_df.index
Index(['B04', 'B05', 'B06', 'B07', 'B08', 'B09'], dtype='object')

dhdt.input.read_aster.read_band_as_hdf(path, band='3N')

This function takes as input the ASTER L1T hdf-filename and the path of the folder that the images are stored, reads the image and returns the data as an array, plus associated geographical/projection metadata.

Parameters

• path (string) – path of the folder, or full path with filename as well

• band (string, optional) – Terra ASTER band index, for example ‘02’, ‘3N’.

Returns

• data (numpy.ndarray, size=(_,_)) – array of the band image

• spatialRef (string) – projection

• geoTransform (tuple) – affine transformation coefficients

• targetprj (osr.SpatialReference object) – spatial reference

References

wwwASTR

https://asterweb.jpl.nasa.gov/content/03_data/04_Documents/ASTERHigherLevelUserGuideVer2May01.pdf

wwwGCTP

https://www.cmascenter.org/ioapi/documentation/all_versions/html/GCTP.html

dhdt.input.read_aster.read_stack_as_l1(path, as_df)

read the band stack of Level-1 data

Parameters

• path (string) – location where the imagery data is situated

• as_df (pandas.DataFrame) – metadata and general multispectral information about the instrument that is onboard Terra

Returns

• im_stack (numpy.ndarray, size=(_,_,_)) – array of the band image

• spatialRef (string) – projection

• geoTransform (tuple, size=(8,1)) – affine transformation coefficients

• targetprj (osr.SpatialReference object) – spatial reference

See also

list_central_wavelength_as

read RapidEye data

The RapidEye constellation composed of five commercial mini-satellites, image collection started in 2008 and halted in 2020. The following functions make reading such data easier.

dhdt.input.read_rapideye.list_central_wavelength_re()

create dataframe with metadata about RapidEye

Returns

df – metadata and general multispectral information about the MSI instrument that is onboard Sentinel-2, having the following collumns:

• center_wavelength, unit=µm : central wavelength of the band

• full_width_half_max, unit=µm : extent of the spectral sensativity

• bandid : number for identification in the meta data

• resolution, unit=m : spatial resolution of a pixel

• common_name : general name of the band, if applicable

• irradiance, unit=W m-2 μm-1 : exo-atmospheric radiance

• relative_timing, unit=ms : relative sampling time difference

Return type

pandas.dataframe

Notes

The detector arrays of the satellite are configured as follows[Kr13]_:

      78 mm
<-------------->
┌--------------┐    #*# satellite
|      red     |     |
├--------------┤     | flight
|   red edge   |     | direction
├--------------┤     |
|near infrared |     v
├--------------┤
|              |
|              |
|              |
|              |
├--------------┤  ^
|  not in use  |  | 6.5 μm
├--------------┤  v
|   green      |
├--------------┤
|   blue       |
└--------------┘

References

Kr13

Krauß, et al. “Traffic flow estimation from single satellite images” Archives of the ISPRS, vol.XL-1/WL3, 2013.

Examples

make a selection by name:

>>> boi = ['red', 'green', 'blue']
>>> re_df = list_central_wavelength_re()
>>> re_df = re_df[re_df['common_name'].isin(boi)]
>>> re_df
     wavelength  bandwidth  resolution   name  bandid  irradiance
B01         475         70           5   blue       0      1997.8
B02         555         70           5  green       1      1863.5
B03         657         55           5    red       2      1560.4

similarly you can also select by bandwidth:

>>> re_df = list_central_wavelength_re()
>>> sb_df = re_df[re_df['bandwidth']<=60]
>>> sb_df.index
Index(['B03', 'B04'], dtype='object')

dhdt.input.read_rapideye.read_band_re(band, path)

read specific band of RapidEye image

This function takes as input the RapidEye band number and the path of the folder that the images are stored, reads the image and returns the data as an array

Parameters

• band (string) – RapidEye band number.

• path (string) – path to folder with imagery.

Returns

• data (np.array, size=(m,n), dtype=integer) – data of one RadidEye band.

• spatialRef (string) – osr.SpatialReference in well known text

• geoTransform (tuple, size=(6,1)) – affine transformation coefficients.

• targetprj (osgeo.osr.SpatialReference() object) – coordinate reference system (CRS)

dhdt.input.read_rapideye.read_stack_re(path)

read specific band of RapidEye image

This function takes as input the RapidEye band number and the path of the folder that the images are stored, reads the image and returns the data as an array

Parameters

path (string) – path to folder with imagery.

Returns

• data (np.array, size=(m,n,_), dtype=integer) – data of one RadidEye band.

• spatialRef (string) – osr.SpatialReference in well known text

• geoTransform (tuple, size=(6,1)) – affine transformation coefficients.

• targetprj (osgeo.osr.SpatialReference() object) – coordinate reference system (CRS)

dhdt.input.read_rapideye.read_sun_angles_re(path)

read the sun angles, corresponding to the RapidEye acquisition

Parameters

path (string) – path where xml-file of RapidEye is situated.

Returns

• Zn (np.array, size=(m,n), dtype=float) – array of the Zenith angles

• Az (np.array, size=(m,n), dtype=float) – array of the Azimtuh angles

Notes

The azimuth angle declared in the following coordinate frame:

     ^ North & y
     |
- <--┼--> +
     |
     +----> East & x

The angles related to the sun are as follows:

surface normal              * sun
^                     ^    /
|                     |   /
├-- zenith angle      |  /
| /                   | /|
|/                    |/ | elevation angle
+---- surface         +--┴---

read PlanetScope data

The PlanetScope constellation is a commercial fleet of micro-satellites. The following function makes reading such data easier.

dhdt.input.read_planetscope.read_band_ps(dir_path, fname, no_dat=None)

This function takes as input the PlanetScope file name and the path of the folder that the images are stored, reads the image and returns the data as an array

Parameters

• dir_path (string) – path of the folder

• fname (string) – PlanetScope image name

Returns

• data (np.array, size=(_,_)) – array of the band image

• spatialRef (string) – projection

• geoTransform (tuple) – affine transformation coefficients

• targetprj (osr.SpatialReference object) – spatial reference

See also

list_central_wavelength_ps

dhdt.input.read_planetscope.read_detector_type_ps(path, fname='*_metadata.xml')

Notes

Dove Pilot has the following sensor array:

  <---- 4032 pixels ---->
┌------------------------┐ ^
|                        | |
|                        | | 2672 pixels
|         RGB            | |
|     (Bayer pattern)    | |
|                        | |
└------------------------┘ v

PlanetScope (PS Instrument) has the following sensor array:

  <---- 6600 pixels ---->
┌------------------------┐ ^
|                        | |
|         RGB            | | 4400 pixels
|     (Bayer pattern)    | |
├------------------------┤ |
|                        | |
|         NIR            | |
|                        | |
└------------------------┘ v

Dove-R (PS2.SD Instrument) has the following sensor array:

  <---- 6600 pixels ---->
┌------------------------┐ ^
|         Green II (B04) | |
├------------------------┤ | 4400 pixels
|         Red      (B05) | |
├------------------------┤ |
|         NIR      (B13) | |
├------------------------┤ | ^
|         Blue     (B02) | | | 1100 pixels
└------------------------┘ v v

Superdove(PSB.SD Instrument) has the following bands: Coastal Blue, Blue, Green, Green I, Green II, Yellow, Red, Rededge, NIR

  <---- 8800 pixels ---->
┌------------------------+ ^
|         Blue   (B02)   | |
├------------------------┤ | 5280 pixels
|         Red    (B05)   | |
├------------------------┤ |
|         Green  (B03)   | |
├------------------------┤ |
|         GreenII(B04)   | |
├------------------------┤ |
|         Yellow (B06)   | |
├------------------------┤ |
|         Rededge(B10)   | |
├------------------------┤ |
|         NIR    (B13)   | |
├------------------------┤ | ^
|         Coastal Blue   | | | 660 pixels
└------------------------┘ v v

read VENµS data

The VENµS satellite is a demonstration satellite, that acquires over specific pre-defined regions. It has a high repeat rate in the order of one or two days, if cloud cover permits. The following functions make reading of such data easier.

dhdt.input.read_venus.list_central_wavelength_vssc()

create dataframe with metadata about VENµS

Returns

df – metadata and general multispectral information about the SuperSpectral Camera that is onboard VENµS, having the following collumns:

• center_wavelength, unit=µm : central wavelength of the band

• full_width_half_max, unit=µm : extent of the spectral sensativity

• bandid : number for identification in the meta data

• gsd, unit=m : spatial resolution of a pixel

• common_name : general name of the band, if applicable

Return type

pandas.dataframe

See also

dhdt.input.read_sentinel2.list_central_wavelength_msi

for the instrument data of MSI on Sentinel-2

dhdt.input.read_landsat.list_central_wavelength_oli

for the instrument data of Landsat 8 or 9

dhdt.input.read_aster.list_central_wavelength_as

for the instrument data of ASTER on the Terra satellite

Notes

The sensor is composed of four detectors, which each have three arrays. These multi-spectral bands (BXX) are arranged as follows:

┌-----B06------┐                      #*# satellite
├-----B04------┤                      | flight
└-----B03------┘ detector id: IV      | direction
┌-----B09------┐                      v
├-----B08------┤
└-----B07------┘ detector id: III
┌-----B11------┐
├-----B12------┤
└-----B10------┘ detector id: II
┌-----B01------┐
├-----B02------┤
└-----B05------┘ detector id: I

References

Di22

Dick, et al. “VENμS: mission characteristics, final evaluation of the first phase and data production” Remote sensing vol.14(14) pp.3281, 2022.

dhdt.input.read_venus.read_stack_vn(vn_df)

read imagery data of interest into an three dimensional np.array

Parameters

vn_df (pandas.Dataframe) – metadata and general multispectral information about the VSSC instrument that is onboard VENµS

Returns

• im_stack (numpy.array, size=(_,_,_)) – array of the band image

• spatialRef (string) – projection

• geoTransform (tuple, size=(8,1)) – affine transformation coefficients

• targetprj (osr.SpatialReference object) – spatial reference