Rock Detection for SLAM
Skills: Python, Machine Learning, PyTorch, TensorFlow
During my senior year at CMU starting Fall 2020, I conducted research on a semantic segmentation Rock Detection neural network for lunar Simultaneous Localization and Mapping (SLAM) under David Wettergreen.
I also worked on this project under the supervision of Samuel Yim, who had initiated this project in 2015 with David.
In sparse regions like the lunar surface, there are a lack of distinct features that allow robots to localize themselves relative to the environment. By detecting rocks and connecting them together to create unique constellations of rocks, you obtain unique features to localize to. My focus for this project was the semantic segmentation of the rocks in the SLAM pipeline. Samuel previously conducted this project in Tensorflow, so I adapted that work into PyTorch, a newer machine learning library at the time, in order to determine if we could obtain improved results.
I primarily trained Fully Convolutional Networks and Convolutional Neural Networks for the purpose of semantic segmentation, or the pixel wise classification of the image. Several architectures I implemented included DenseNet, ResNet-50, ResNet-101, and DeepLab-V3 networks. In addition, I developed custom PyTorch DataSet and DataLoader classes to preprocess, augment, and load the data in batches for network training.
After the models were trained, I worked with Edge Case Research to post process the model and evaluate the model performance using their Hologram software. Hologram processes input videos and visualizes statistics such as confidence scores and true/false positives/ negatives. I used these results to inform hyperparameter selection as well as my augmentation and aggregation of the data set in further rounds of training and fine tuning, targeting weaknesses in prior models to improve model performance.
