Convolutional Neural Network Classification of Hyperspectral Imagery in the San Francisco Bay Area, California

Purpose: Hyperspectral image (HSI) data has the potential to have a high degree of intraclass variability and low inter-class variability; this makes engineering features and creating robust classifiers challenging. Convolutional Neural Networks (CNNs) have found success in classifying classical 2-d imagery and recently HSI data. CNNs, unlike other neural network topologies learn features from the data in the form of convolutional kernels. These trained convolutional kernels bring out or filter the class-specific discriminating features from the data. These features are then used to classify the data. To date there has been little discussion on visualization and interpretation of these learned convolutional kernels and their respective impact on classification as applied to hyperspectral data. This work will introduce an architecture as well as provide methods to visualize and determine how important these learned convolutional kernels are on the classification task. Procedure: A convolutional neural network architecture was developed, trained and utilized to classify hyperspectral imagery. As a comparison, Random Forests (RF) and Support Vector Machine (SVM) classifiers were also trained on the same dataset. These data are from the San Francisco Bay area in year 2015, across three temporal seasons. Comparisons between these classifiers and the application of temporal data within the data are provided. The analysis of the convolutional network's inner data product visualizations are provided to show insight into what significant features exist in the data and how important these features are for classification accuracy. Findings: The CNN developed provided classification accuracies comparable to SVM. Both CNN and SVM performed better than RF. Analysis of the inner products of the CNN provided insight to the distinctive features within the spectral and temporal domain. All classification methods perform well when generating land cover maps. The Random Forest classifier mis-classified some obvious land cover areas. The Support Vector Machine (SVM) and CNN generated maps disagree on harder to classify areas. Conclusions: Described within is a classifier implementation that can be applied to hyperspectral imagery data that produces classification accuracies comparable to existing classification methods. Unbalanced/Balanced accuracies of 91.0%/74.0% for SVM, 89.8%/75.5% for CNN and 84.2%/65.8% for RF were developed. Visualizations methods were developed to show the distinguishing characteristics between the classes across the dimension of the convolution defined within this topology. This type of classifier is a good candidate for HSI applications because of the interpretability of these visualizations and relatively high classification accuracy. Kernel importance is provided as a method and metric to determine how important a learned feature is with respect to the classification task.