Coral Reef Mapping and Unmixing
Skills: Python, Machine Learning, Planning, TensorFlow, ENVI
My research as a graduate student in the Master of Science in Robotics at CMU focused on the mapping and unmixing of coral reef habitats. Specifically, I classified spectral images of coral reefs into coral, sand, and algae and disambiguating the data when a single pixel contains multiple classes. The research end goal is to have a more efficient and accessible pipeline than traditional methods to map the Great Barrier Reef and other coral habitats over time for monitoring.
​
I presented the first year results of this research at WHISPERS 2022 (Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing). I also competed as a top 10 finalist in the IGARSS 2023 Three Minute Thesis Competition.
This research was developed from prior work by Alberto Candela as part of his PhD dissertation, also under David Wettergreen.
The data products used consist of multispectral Landsat-8 and WorldView-3 data at 30m and 1.6m resolution respectively, as well as hyperspectral PRISM labels at 7.9m resolution of the ratios between classes per pixel. ENVI, an image processing and analysis software by L3 Harris, was used to georegister the data together as well as align their resolutions, such that the PRISM labels could be used with Landsat and WorldView inputs for supervised learning.
​
When considering spectral data, individual pixels cover different areas depending on the resolution of the data. As such, each pixel may be composed of a mixture of classes as opposed to containing only a single class. As such, it becomes possible to improve the accuracy of the data classification by developing an understanding of how the desired classes are mixed together from the spectral data inputs. For this research, I primarily trained two neural network based models for the purpose of not only classifying pixels, but also unmixing them to improve performance precision.
The two primary networks developed for this research are a Regression Model and Deep Conditional Dirichlet Model (DCDM), both of which were used to classify the data into coral, sand, and algae for Landsat and WorldView inputs as well as perform unmixing of the data.
​
The Regression Model I trained in this research is a five layer fully connected network with SELU activation and a softmax output and optimized using Kullback-Leibler Divergence (KLD) loss. This was used as a baseline approach using neural networks for classification.
The second model I trained was the DCDM, which was developed as an expansion of the Regression Model and thus also consisted of a five layer fully connected network with SELU activation, but using a sigmoid output instead. The Dirichlet components of the DCDM fit a Dirichlet distribution to model the mixing ratios of classes per pixel. This distribution is then used by the network estimate class mixtures and unmix them when performing classification. In addition to the KLD Loss which directly optimizes the network output with respect to the labels, the mixing ratio output was optimized using the Negative Log Likelihood loss.
Aside from these two networks, I also trained a Support Vector Machine on the data to serve as a baseline machine learning performance metric.
After the models have been trained and the spectral data has been processed, it is important to consider how to best improve the classification and unmixing confidences. One method to approach this problem is to supplement the model predictions with on-site expeditions. However, these expeditions are a costly expenditure. As such, I calculate a set of information priors from the model outputs and pass them into a path planner, thus determining where to efficiently sample in order to yield the most information gain.
​
By training the same model on different subsets of the training dataset, the variance in the predictions can be calculated. This variance quantifies the model uncertainty, where high variance indicates the model is less confident about the prediction, and serves as the information prior for the planner. Weighting the variance of the predicted classes allows the model to optimize for various goals.
An ergodic trajectory is one that traverses to and spends time in regions proportional to their expected information gain. I apply an ergodic path planner based on the information prior to determine an ergodic trajectory for on-site expedition sampling. The result is a path that prioritizes areas where the model has high prediction uncertainty. This yields the greatest amount of information gain over a finite trajectory, thus optimizing the cost of the expedition.
A recording of my presentation of "Spectral Unmixing and Mapping of Coral Reef Benthic Cover with Deep Learning" at the WHISPERS 2022 conference in Rome is given here, where I also had the opportunity to chair the session I presented in.
​
If a copy of the final conference paper is desired, please use the contact form to let me know, and I would be happy to send it over!
A recording of my Master of Science in Robotics Thesis Defense "Spectral Unmixing and Mapping of Coral Reef Benthic Cover" in included here.
​
The final thesis document can be found in the CMU Robotics Institute publications page here.