The DL4ES book

"Deep learning for the Earth Sciences -- A comprehensive approach to remote sensing, climate science and geosciences"
Editors: Gustau Camps-Valls, Devis Tuia, Xiao Xiang Zhu, Markus Reichstein
Publisher: Wiley & Sons, inc., 2021

Cite it

@Book{CampsValls21wiley,
Title = {Deep learning for the Earth Sciences: A comprehensive approach to remote sensing, climate science and geosciences},
Author = {Camps-Valls, G. and Tuia, D. and Zhu, X.X. and Reichstein, M.},
Publisher = {Wiley & Sons},
isbn = {978-1-119-64614-3},
Year = {2021},
}

Preview it

https://books.google.es/books?id=5P5DzgEACAAJ&dq=%22Deep+learning+for+the+Earth+Sciences%22&hl=en&sa=X&ved=2ahUKEwimm8GygYXwAhW4BGMBHWpJDf8Q6AEwAHoECAIQAg

Buy it

https://www.amazon.com/Deep-learning-Earth-Sciences-comprehensive/dp/1119646146/

Links to toolboxes, code and data

Chapter 01: Introduction

by Gustau Camps-Valls, Xiao Xiang Zhu, Devis Tuia, and Markus Reichstein

Some review papers of interest:

• Remote sensing
  • X. X. Zhu et al., "Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources," in IEEE Geoscience and Remote Sensing Magazine, vol. 5, no. 4, pp. 8-36, Dec. 2017, doi: 10.1109/MGRS.2017.2762307. https://ieeexplore.ieee.org/abstract/document/8113128
  • L. Zhang, L. Zhang and B. Du, "Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art," in IEEE Geoscience and Remote Sensing Magazine, vol. 4, no. 2, pp. 22-40, June 2016, doi: 10.1109/MGRS.2016.2540798. https://ieeexplore.ieee.org/abstract/document/7486259
  • Ma, Lei, et al. "Deep learning in remote sensing applications: A meta-analysis and review." ISPRS journal of photogrammetry and remote sensing 152 (2019): 166-177. https://www.sciencedirect.com/science/article/pii/S0924271619301108
  • Yuan, Qiangqiang, et al. "Deep learning in environmental remote sensing: Achievements and challenges." Remote Sensing of Environment 241 (2020): 111716. https://www.sciencedirect.com/science/article/pii/S0034425720300857
• Climate science
  • Rasp, Stephan, Michael S. Pritchard, and Pierre Gentine. "Deep learning to represent subgrid processes in climate models." Proceedings of the National Academy of Sciences 115.39 (2018): 9684-9689. https://www.pnas.org/content/115/39/9684.short
  • Jones, Nicola. "How machine learning could help to improve climate forecasts." Nature News 548.7668 (2017): 379. https://www.nature.com/articles/548379a
  • Dueben, Peter D., and Peter Bauer. "Challenges and design choices for global weather and climate models based on machine learning." Geoscientific Model Development 11.10 (2018): 3999-4009. https://gmd.copernicus.org/articles/11/3999/2018/
• Geosciences
  • Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., & Carvalhais, N. (2019). Deep learning and process understanding for data-driven Earth system science. Nature, 566(7743), 195-204. https://www.nature.com/articles/s41586-019-0912-1.
  • Bergen, Karianne J., et al. "Machine learning for data-driven discovery in solid Earth geoscience." Science 363.6433 (2019). https://science.sciencemag.org/content/363/6433/eaau0323.abstract

Chapter 02: Learning Unsupervised Feature Representations of Remote Sensing Data with Sparse Convolutional Networks

by Jose E. Adsuara, Manuel Campos-Taberner, Javier García-Haro, Carlo Gatta, Adriana Romero, and Gustau Camps-Valls

• EPLS algorithm in https://sites.google.com/site/adriromsor/epls

Chapter 03: Generative Adversarial Networks in the Geosciences

Gonzalo Mateo-García, Valero Laparra, Christian Requena-Mesa, and Luis Gómez-Chova

• GANs zoo: https://github.com/hindupuravinash/the-gan-zoo

Chapter 04: Deep Self-taught Learning in Remote Sensing

by Ribana Roscher

Chapter 05: Deep Learning-based Semantic Segmentation in Remote Sensing

by Devis Tuia, Diego Marcos, Konrad Schindler, and Bertrand Le Saux

• http://www2.isprs.org/commissions/comm3/wg4/semantic-labeling.html
• http://www.grss-ieee.org/community/technical-committees/data-fusion/2015-ieee-grss-data-fusion-contest/
• https://sites.google.com/site/michelevolpiresearch/data/zurich-dataset
• https://project.inria.fr/aerialimagelabeling/
• https://spacenetchallenge.github.io
• http://deepglobe.org/
• https://spacenet.ai/sn6-challenge/
• https://github.com/cloudtostreet/Sen1Floods11
• http://www.grss-ieee.org/community/technical-committees/data-fusion/2018-ieee-grss-data-fusion-contest/
• https://udayton.edu/engineering/research/centers/vision_lab/research/was_data_analysis_and_processing/dale.php
• http://www.grss-ieee.org/community/technical-committees/data-fusion/2017-ieee-grss-data-fusion-contest-2/
• https://mediatum.ub.tum.de/1454690
• http://dx.doi.org/10.21227/b9pt-8x03
• https://github.com/SorourMo/38-Cloud-A-Cloud-Segmentation-Dataset
• https://rcdaudt.github.io/oscd/

Chapter 06: Object Detection in Remote Sensing

by Jian Ding, Jinwang Wang, Wen Yang, and Gui-Song Xia

• https://captain-whu.github.io/DOTA/
• http://aiskyeye.com/
• http://xviewdataset.org/
• https://github.com/dingjiansw101/RoITransformer_DOTA
• https://github.com/dingjiansw101/AerialDetection

Chapter 07: Deep Domain adaptation in Earth Observation

by Benjamin Kellenberger, Onur Tasar, Bharath Bhushan Damodaran, Nicolas Courty, and Devis Tuia

• https://github.com/bkellenb/da-dl4eo
• https://kuzikus-namibia.de/xe_index.html

Chapter 08: Recurrent Neural Networks and the Temporal Component

by Marco Körner and Marc Rußwurm

Chapter 09: Deep Learning for Image Matching and Co-registration

by Maria Vakalopoulou, Stergios Christodoulidis, Mihir Sahasrabudhe, and Nikos Paragios

• https://github.com/stergioc/smooth-transformer

Chapter 10: Multisource Remote Sensing Image Fusion

by Wei He, Danfeng Hong, Giuseppe Scarpa, Tatsumi Uezato, and Naoto Yokoya

• BDSD: http://openremotesensing.net/wp-content/uploads/2015/01/pansharpeningtoolver_1_3.rar
• MTF-GLP: http://openremotesensing.net/wp-content/uploads/2015/01/pansharpeningtoolver_1_3.rar
• SIRF: http://cchen156.web.engr.illinois.edu/code/CODE_SIRF.zip
• PNN: http://www.grip.unina.it/download/prog/PNN/PNN_v0.1.zip
• DRPNN: https://github.com/Decri/DRPNN-Deep-Residual-Pan-sharpening-Neural-Network
• PanNet: https://xueyangfu.github.io/paper/2017/iccv/ICCV17_training_code.zip
• PNN+: https://github.com/sergiovitale/pansharpening-cnn
• http://naotoyokoya.com/Download.html
• https://github.com/qw245/BlindFuse
• https://github.com/alfaiate
• https://sites.google.com/site/harikanats/
• https://sites.google.com/view/renweidian/
• https://github.com/aicip/uSDN
• https://github.com/XieQi2015/MHF-net
• http://naotoyokoya.com/Download.html
• http://www1.cs.columbia.edu/CAVE/databases/

Chapter 11: Deep Learning for Image Search and Retrieval in Large Remote Sensing Archives

by Gencer Sumbul, Jian Kang, and Begüm Demir

• https://git.tu-berlin.de/rsim

Chapter 12: Deep Learning for Detecting Extreme Weather Patterns

by Mayur Mudigonda, Prabhat, Karthik Kashinath, Evan Racah, Ankur Mahesh, Yunjie Liu, Christopher Beckham, Jim Biard, Thorsten Kurth, Sookyung Kim, Samira Kahou, Tegan Maharaj, Burlen Loring, Christopher Pal, Travis O’Brien, Ken Kunkel, Michael F. Wehner, and William D. Collins

• Coded Surface Bulletin (CSB): http://www.nws.noaa.gov/noaaport/html/noaaport.shtml
• ClimateNet: https://www.nersc.gov/research-and-development/data-analytics/big-data-center/climatenet
• ClimateContoursTool: http://labelmegold.services.nersc.gov/climatecontoursgold/tool.html

Chapter 13: Spatio-temporal Autoencoders in Weather and Climate Research

by Xavier-Andoni Tibau, Christian Reimers, Christian Requena-Mesa, and Jakob Runge

• https://github.com/ClimateInformatics/SupernoVAE

Chapter 14: Deep Learning to Improve Weather Predictions

by Peter D. Dueben, Peter Bauer, and Samantha Adams

• https://www.top500.org/
• https://www.ecmwf.int/en/about/what-we-do/scalability
• https://www.ecmwf.int/en/computing/our-facilities/data-handling-system

Chapter 15: Deep Learning and the Weather Forecasting Problem: Precipitation Nowcasting

by Zhihan Gao, Xingjian Shi, Hao Wang, Dit-Yan Yeung, Wang-chun Woo, and Wai-Kin Wong

• WMO-No.1198 report: https://library.wmo.int/doc_num.php?explnum_id=3795
• HKO: https://rsmc.hko.gov.hk
• Com-SWIRLS: https://com-swirls.org

Chapter 16: Deep Learning for High-dimensional Parameter Retrieval

by David Malmgren-Hansen

• IASI - Atmospheric Temperature Profiles Code: https://github.com/damaha/iasi-atmosphere Data: https://data.dtu.dk/articles/dataset/IASI_dataset_v1/12999642

• ASIP - Sea Ice Concentrations from Satellite Sensor Fusion Code: https://github.com/damaha/asip-v1 Data: https://data.dtu.dk/articles/dataset/ASIP_Sea_Ice_Dataset_-_version_1/11920416

Automatic Ice charting in early operational use at the Danish Meteorological Institute: http://ocean.dmi.dk/asip/.

Chapter 17: A review of Deep Learning for cryospheric studies

by Lin Liu

Here are the major data centers, repositories, and providers for cryospheric studies:

• US National Snow and Ice Data Center (https://nsidc.org)
• US National Science Foundation Arctic Data Center (https://arcticdata.io)
• US Antarctic Program Data Center (http://www.usap-dc.org})
• European Space Agency Climate Change Initiative (http://cci.esa.int)
  • Antarctic Ice Sheet (http://esa-icesheets-antarctica-cci.org)
  • Greenland Ice Sheet (http://esa-icesheets-greenland-cci.org)
  • Glaciers (http://www.esa-glaciers-cci.org)
  • Permafrost (http://cci.esa.int/Permafrost)
  • Sea ice (http://cci.esa.int/seaice)
  • Snow (http://cci.esa.int/node/274/)
• Canadian Cryospheric Information Network (https://www.ccin.ca)
• China National Tibetan Plateau Data Center (https://data.tpdc.ac.cn/en/)

Below we list the data and codes published in the cryospheric studies reviewed in this chapter, grouped by the cryospheric components.

• Glaciers

  • Detection of glacier calving margins with convolutional neural networks \citep{mohajerani2019} Code and data: https://github.com/yaramohajerani/FrontLearning
  • Automatically delineating the calving front of Jakobshavn Isbr{\ae} from multitemporal TerraSAR-X images \citep{zhang2019} Training and test data: https://doi.org/10.1594/PANGAEA.897066
  • ALpine Parameterized Glacier Model (ALPGM) \citep{bolibar2020} Code and sample data: https://github.com/JordiBolibar/ALPGM

• Ice sheet

  • DeepBedMap: Antarctica Ice Sheet bed elevation using a super resolution deep neural network \citep{leong2020} Code: https://github.com/weiji14/deepbedmap Training experiments: https://www.comet.ml/weiji14/deepbedmap Digital bed elevation model: https://doi.org/10.17605/OSF.IO/96APW

• Snow

  • Estimating snow depth on Arctic sea ice using satellite microwave radiometry and a neural network \citep{Braakmann-Folgmann2019} Sample code and data: https://github.com/AnneBF/snownet

• Permafrost

  • Automatic mapping of thermokarst landforms from remote sensing images using deep learning \citep{huang2018} Code: https://github.com/yghlc/DeeplabforRS

  • Using deep learning to map retrogressive thaw slumps from CubeSat images \citep{huang2020} Code: https://github.com/yghlc/Landuse\_DL Training and test data: https://doi.pangaea.de/10.1594/PANGAEA.908909

    • High-resolution mapping of spatial heterogeneity in ice wedge polygon geomorphology near Prudhoe Bay, Alaska \citep{abolt2020} Code and data: https://doi.org/10.1594/PANGAEA.910178

• River ice

  • River ice segmentation with deep learning \citep{singh2019} Code: https://github.com/abhineet123/river\_ice\_segmentation Data: https://ieee-dataport.org/open-access/alberta-river-ice-segmentation-dataset

Chapter 18: Emulating Ecological Memory with Recurrent Neural Networks

by Basil Kraft, Simon Besnard, and Sujan Koirala

• Code: https://github.com/bask0/dl4es_ch18
• For access to data / simulations contact the authors

Chapter 19: Applications of Deep Learning in Hydrology

by Chaopeng Shen and Kathryn Lawson

Chapter 20: Deep Learning of Unresolved Turbulent Ocean Processes in Climate Models

by Laure Zanna and Thomas Bolton

Chapter 21: Deep Learning for the Parametrization of Subgrid Processes in Climate Models

by Pierre Gentine, Veronika Eyring, and Tom Beucler

Chapter 22: Using Deep Learning to Correct Theoretically-Derived Models

by Peter A. G. Watson

• https://github.com/PAGWatson/Lorenz96_and_neural_networks

Chapter 23: Outlook

by Markus Reichstein, Gustau Camps-Valls, Devis Tuia, and Xiao Xiang Zhu

• Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., & Carvalhais, N. (2019). Deep learning and process understanding for data-driven Earth system science. Nature, 566(7743), 195-204. https://www.nature.com/articles/s41586-019-0912-1.
• Tuia, Devis, Ribana Roscher, Jan Dirk Wegner, Nathan Jacobs, Xiao Xiang Zhu, and Gustau Camps-Valls. "Towards a Collective Agenda on AI for Earth Science Data Analysis." arXiv preprint arXiv:2104.05107 (2021). https://arxiv.org/abs/2104.05107
• Camps-Valls, Gustau. "Perspective on Deep Learning for Earth Sciences." Generalization With Deep Learning: For Improvement On Sensing Capability (2021): 159-173. https://www.worldscientific.com/doi/abs/10.1142/9789811218842_0007