On-going projects
- DECOL – ANR JCJC (2024 – , lead):
Participants: Charlotte Pelletier (PI), Laetitia Chapel, Thomas Corpetti, … - AXOLOTL – HORIZON EUROPE (2024 – 2027): From sky to seafloor observation: Achieving excellence in oceanic surveillance and conservation through deep learning. AXOLOTL is proposed as a transformational international endeavour capable of enhancing the R&I capacity of CMMI, Cyprus and Europe in the interdisciplinary fields of DL-enhanced in-situ biodiversity assessment and DL-enhanced remote sensing for maritime surveillance. The project will contribute to closing the gap between a H2020 Teaming Centre of excellence and 2 strong innovators from France and Belgium through capacity-building, knowledge transfer, networking, and outreach activities at regional and international levels. Activities will go beyond the strictly scientific scope and support the mutual development, consolidation, and reinforcement of administrative, dissemination and entrepreneurial competencies, access to networks of excellence and the sustainable linkage between partners. The project’s R&I component will develop new strategies for improving data quality, standardization, and synchronization issues, devise novel interdisciplinary methodologies, develop robust DL models from state-of-the-art computer vision methods, and validate its proposed solutions in 2 relevant real-world contexts (biodiversity assessment and maritime surveillance).
Participants: Minh-Tan Pham (local PI), Nicolas Courty, Charlotte Pelletier, Abdelbadie Belmouhcine - OTTOPIA – ANR/Chair AI (2021-2025, lead): Earth Observation, whether it be by satellites, airborne captors or drones, allows a better understanding of the dynamics of environmental systems or our human society. It is a decisive tool to measure the impact of mankind on earth. In the last 50 years, the fast development of spatial missions and of the technology of the associated captors yields an unprecedented amount of data, largely under-exploited. Artificial intelligence can become a major help toward exploiting this wealth of information, by automatizing tasks cantoned to human operators, or even combining them to produce novel knowledges. Yet, the earth observation data come with specific challenges not only related to their volume but also their complexity. The OTTOPIA Chair project proposes to tackle some of them through the prism of Optimal Transport theory applied to machine learning. This mathematical tool makes it possible to apprehend the data through their distributions, and no longer as a sum of distinct individuals. Following significant advances in computational aspects, it has recently emerged as a tool of choice for multiple learning problems. We propose to exploit its principles on four challenges: 1. multi-modality and considering the heterogeneity of the data at transfer of learning, 2. Learning with few data, possibly corrupted by label noise, 3. Security of AI algorithms in Earth observation; and 4. Visual Question Answering, i.e. interacting with remote sensing data through natural language questions. The contributions of the Chair will naturally aim at fundamental developments in AI but also new applied methodologies for which a strong industrial transfer potential is envisaged.
Participants: Nicolas Courty (PI), Chloé Friguet, Minh-Tan Pham, Charlotte Pelletier, Huy Tran (PhD), Paul Berg (PhD), Renan Bernard (PhD), Hoang-An Le (Postdoc) - DynaLearn – Labex CominLabs (2020-2023, lead): Neural networks are powerful objects used in machine learning, but poorly understood from a theoretical point of view. A recent line of research consist in studying the flow of information through or in these networks through the lens of dynamical systems and their associated Physics. The Dynalearn project aims at contributing on those aspects in a two-fold way: (1) By exploring how dynamical formulation of learning process can help in understanding better learning deep neural architectures, as well as proposing new learning paradigms based on the regularization of the flows of information; (2) By leveraging on novel neural architectures and available data to devise new data-driven dynamical simulation models, with applications in Earth Observation and Medical Imaging.
Participants: Nicolas Courty (PI), Thomas Corpetti, Clément Bonet (PhD), Diego di Carlo (Postdoc) - OWFSOMM – ANR/FEM (2020-2023): The project OWFSOMM (Offshore Wind Farm Surveys Of Marine Megafauna: standardization of tools and methods for monitoring at OWF scales) aims to provide, (i) a method for conducting a robust inter-calibration of surveys at sea from mobile platforms using historical and novel technologies and, (ii) an AI suite to optimize the use of multiple sensors in order to improve their efficiency in detecting, identifying and characterizing marine megafauna.
Participants: Sébastien Lefèvre (WP lead), Minh-Tan Pham - SAD 2021 – ROMMEO (2022-2024, Lead): The objective of ROMMEO (Robust Multitask learning via mutual knowledge distillation for earth observation) is to develop a robust and compact model by coupling two powerful tools in machine learning, i.e. multitask learning framework and knowledge distillation, in the context of earth observation.
Participants: Minh-Tan Pham (PI), Hoang-An Le (Postdoc) - DeepChange – ANR/JCJC (2021-2025): Accurate and up-to-date land cover information constitutes key environmental data for developing efficient policies in this era of resource scarcity and climate change. New Satellite Image Times Series offer new opportunities for detecting land cover class transitions. Nevertheless, the challenges of the “Big Data” have become imminent for the exploitation of this massive flow of data. Deep generative models are one of the most promising tools for big data analysis. The use of such models has just started to emerge in the remote sensing. In this project, Generative Adversarial Networks and Variational Autoencoders want to be explored to face common remote sensing challenges, which are the lack of reference data and the exploitation of complex and heterogeneous information. The originality of the project relies on the development of new online change detection methodologies by using generative models, which incorporate the temporal dynamics of the data and physical knowledge constraints.
Participants: Charlotte Pelletier - MATS – ANR/JCJC (2019-2023, lead): A huge trend in recent earth observation missions is to target high temporal and spatial resolutions (e.g. SENTINEL-2 mission by ESA). Data resulting from these missions can then be used for fine grained studies in many applications. This project focus on three key environmental issues: agricultural practices and their impact, forest preservation and air quality monitoring. Based on identified key requirements for these application settings, MATS project aims to feature a complete rethinking of the literature in machine learning for time series, with a focus on large-scale methods that could operate even when little supervised information is available. In more details, MATS introduce new paradigms in large-scale time series classification, spatio-temporal modeling and weakly supervised approaches for time series. Proposed methods cover a wide range of machine learning problems including domain adaptation, clustering, metric learning and (semi-)supervised classification, for which dedicated methodology is lacking when time series data is at stake. Methods developed in the project are made available to the scientific community as well as to practitioners through an open-source toolbox in order to help dissemination to a wide range of application areas. Moreover, the application settings considered in the project are used to showcase benefits offered by methodologies developed in MATS in terms of time series analysis.
Participants: Romain Tavenard (PI), Laetitia Chapel, Thomas Corpetti, Nicolas Courty, Chloé Friguet, François Painblanc (PhD) - SixP – ANR/PRCI (2019-2023): SixP aims: i) to characterize the variation of plant-plant interactions along gradients of metal phyto-availability, while explaining the specific role of metallicolous species in these interactions; ii) to better identify the effects of multiple stress factors on these interactions; iii) to specify the plant functional strategies at stake; and iv) to assess the effect of plant-plant interactions at the community scale. The project will be implemented in several mine tailings in the Pyrénées at different altitudes (in the montane zone, and at the subalpine-alpine zone). At each site, several areas will be specified from peripheral low-contaminated areas towards tailings centers corresponding to a gradient of metal phyto-availability. The first three research directions will then be addressed by experimentations manipulating species in interaction. As for the last direction, the combination of very high resolution airborne data (lidar, multispectral images) covering the studied areas with in situ observations in a deep learning framework will be used to map species distribution and their geomorphological position. Spatial patterns of the different interacting species (aggregation vs repulsion) will exhibit the effects of plant-plant interactions on the long-term.
Participants: Sébastien Lefèvre (WP Leader), Thomas Corpetti, Javiera Castillo Nvarro (Postdoc) - MULTISCALE – ANR/PRCI (2019-2022, lead): This research project aims at providing a complete and integrated framework for multiscale image analysis and learning with hierarchical representations of complex remote sensing images. While hierarchical representations of RS images has led to an effective and efficient scheme to deal with panchromatic or at most multiband data, their application to complex data is still to be explored. In addition, despite their ability to encode structural and multiscale information, their so far exploitation have not reached beyond a mere superposition of monoscale analysis. In this context, the MULTISCALE project defines new methods for the construction of hierarchical image representations from multivariate, multi-source, multi-resolution and multi-temporal data, and provides some dedicated image analysis and machine learning tools to perform multiscale analysis. The new methodology will be implemented in various toolboxes used by the community to favor the dissemination of the results. Success of the project will be assessed by benchmarking the proposed framework on two remote sensing applications. Substantial breakthroughs over classical methods are expected, both in terms of efficiency and effectiveness.
Participants: Laetitia Chapel (PI), Sébastien Lefèvre, Thomas Corpetti, Minh-Tan Pham, François Merciol, Manal Hamzaoui (PhD) - SEMMACAPE – ADEME (2019-2022, lead): The analysis of the development impacts of a Marine renewable energies project generally requires aerial observations of marine megafauna (marine mammals and birds) to better characterize the species that frequent these sites. The Semmacape project aims to demonstrate the relevance of software solutions for processing and analyzing aerial photographs to ensure the automated census of marine megafauna. The importance of such monitoring has been reinforced by the need for impact studies, which are required for any wind power project subject to environmental authorization. Computer vision has undergone a recent upheaval with “deep learning” in the form of deep convolutional networks. The application of these networks to aerial images for the automated observation of marine megafauna is promising, but adaptations of existing algorithms are to be expected. In particular, these animals evolve in a context (sea) characterized by a highly variable visual content, which is detrimental to the performance of these deep networks. The Semmacape project aims to respond to these scientific obstacles in order to provide a technological leap forward in the field of aerial census of marine megafauna and its application to the environmental monitoring of offshore wind farms. The main gain will lie in the completeness of the observations, while minimising the risk of identification errors and allowing a reduction in analysis time.
Participants: Sébastien Lefèvre (PI), Minh-Tan Pham, Deise Santana Mai (Postdoc), Hugo Gangloff (Postdoc) - Game of Trawls – FEAMP (2019-2022): The main goal of the project is to allow future fishing boats to detect in real-time, with a network of sensors, the different species of fish before catching them to sort them in the trawl and thus limit discards. We will focus on underwater detection and recognition of fish species. Our data are diverse: underwater images, history of captures in a logbook, multi beam sounders, GPS, depth sensors, temperatures. We therefore propose to design neural networks specialized in the detection and tracking of objects, taking advantage of multimodal data input while also taking care of efficiency for real-time processing of these data.
Participants: Luc Courtrai, Sébastien Lefèvre, Jean-Christophe Burnel (PhD), Abdelbadie Belmouhcine (Postdoc), Hugo Gangloff (Postdoc) - SESURE – GdR ISIS (2021-2023): The SEntinel time series SUper REsolution (SESURE) project is interested in developing super-resolution approaches for satellite image sequences that make the most of the temporal structure of the data. By exploiting deep learning on the mass of Sentinel-2 data acquired in France since 2015, SESURE will make it possible to infer a subpixel structure of pixels in different colours. Unlike methods of the current state of the art, which is often limited to evaluations on synthetic data, the project will rely on SPOT-6 and -7 open data as a reference for high-resolution images and Sentinel-2 for low-resolution image series.
SESURE aims to quantify the informational gap between Sentinel-2 time series and very high-resolution RGB SPOT acquisitions. In particular, we will study the existence of transformations, reversible or not, allowing to pass from one modality to another, and thus to solve the frequency-resolution dilemma currently faced by the computer vision community in remote sensing.
Participant: Charlotte Pelletier - MACLEAN – GDR MADICS (2019-2022, co-lead): Machine learning for Earth Observation (MACLEAN) is an action within the GDR MADICS (BigData, Data Science) http://www.madics.fr/, started in January 2019. We organise regularly workshops, seminars, tutorials, young researcher days at national and international levels to promote scientifique exchanges and discussions from both Earth Observation and machine learning communities.
Participants: Sébastien Lefèvre, Minh-Tan Pham, Thomas Corpetti
Past projects
- OATMIL – ANR (2017-2021, lead): OptimAl Transport for MachIne Learning
- PHC PAMOJA (Phase 2, 2019-2021, with TU Kenya)
- DEEPDETECT – ANR/ASTRID (2018-2020) : Deep learning for multiple targets detection and recognition in variable background
- CNES TOSCA Floodscape (2018-2020)
- CNES TOSCA PARCELLE (2018-2020)
- SESAME – ANR/ASTRID (2017-2019): big data infraStructurE and analytics for AIS & Sentinel SAtellite Data for the surveillance of the MaritimE Traffic
- CNES R&T Trees (2018-2019, lead): Hierarchical representation of satellite images for optimization of the processing chains
- CNES R&T Vehicles (2018-2019): Vehicle counting from VHR satellite images using deep learning
- DGA MMT (2018-2019): Semantic segmenation of a scene
- PHC Cai YuanPei (2017-2019, with China)
- AAP PME I&R DELORA (2016-2018): Detection and localization of buried objects from ground penetrating radar
- ASTERIX – ANR/JCJC (2013-2017, lead): Analyse Spatio-temporelle pour la Télédétection de l’Environnement par Reconnaissance dans les Images compleXes
- CNES TOSCA VEGIDAR (2014-2017, lead): Urban vegetation analysis by coupling very high resolution optical remote sensing and Lidar data
- PHC PAMOJA (Phase 1, 2016-2017, with TU Kenya)
