Abstract
Environmental monitoring guides conservation and is particularly important for aquatic habitats which are heavily impacted by human activities. Underwater cameras and uncrewed devices monitor aquatic wildlife, but manual processing of footage is a significant bottleneck to rapid data processing and dissemination of results. Deep learning has emerged as a solution, but its ability to accurately detect animals across habitat types and locations is largely untested for coastal environments. Here, we produce five deep learning models using an object detection framework to detect an ecologically important fish, luderick (Girella tricuspidata). We trained two models on footage from single habitats (seagrass or reef) and three on footage from both habitats. All models were subjected to tests from both habitat types. Models performed well on test data from the same habitat type (object detection measure: mAP50: 91.7 and 86.9% performance for seagrass and reef, respectively) but poorly on test sets from a different habitat type (73.3 and 58.4%, respectively). The model trained on a combination of both habitats produced the highest object detection results for both tests (an average of 92.4 and 87.8%, respectively). The ability of the combination trained models to correctly estimate the ecological abundance metric, MaxN, showed similar patterns. The findings demonstrate that deep learning models extract ecologically useful information from video footage accurately and consistently and can perform across habitat types when trained on footage from the variety of habitat types.
This is a preview of subscription content, log in via an institution to check access.
References
Abrantes, K. G., Barnett, A., Baker, R., & Sheaves, M. (2015). Habitat-specific food webs and trophic interactions supporting coastal-dependent fishery species: An Australian case study. Reviews in Fish Biology and Fisheries, 25(2), 337–363.
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., van Essen, B. C., Awwal, A. A. S., & Asari, V. K. (2019). A state-of-the-art survey on deep learning theory and architectures. Electronics, 8(3), 292.
Beijbom, O., Edmunds, P. J., Kline, D. I., Mitchell, B. G., & Kriegman, D. (2012). Automated annotation of coral reef survey images. IEEE Conference on Computer Vision and Pattern Recognition, 2012, 1170–1177.
Buckland, M., & Gey, F. (1994). The relationship between recall and precision. Journal of the American Society for Information Science, 45(1), 12–19.
Christin, S., Hervet, E., & Lecomte, N. (2019). Applications for deep learning in ecology. Methods in Ecology and Evolution, 10(10), 1632–1644.
Davidson, N. C. (2014). How much wetland has the world lost? Long-term and recent trends in global wetland area. Marine and Freshwater Research, 65(10), 934–941.
Ditria, E. M., Lopez-Marcano, S., Sievers, M., Jinks, E. L., Brown, C. J., & Connolly, R. M. (2020). Automating the analysis of fish abundance using object detection: Optimizing animal ecology with deep learning. Frontiers in Marine Science, 7, 429.
dos Santos, A. A., & Goncalves, W. N. (2019). Improving Pantanal fish species recognition through taxonomic ranks in convolutional neural networks. Ecological Informatics, 53, 100977. https://doi.org/10.1016/j.ecoinf.2019.100977.
Everingham, M., Van Gool, L., Williams, C. K., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2), 303–338.
Ferguson, A., Harvey, E. S., Rees, M., & Knott, N. A. (2015). Does the abundance of girellids and kyphosids correlate with cover of the palatable green algae, Ulva spp.? A test on temperate rocky intertidal reefs. Journal of Fish Biology, 86(1), 375–384.
Frid, A., & Dill, L. (2002). Human-caused disturbance stimuli as a form of predation risk. Conservation Ecology, 6(1).
Goldsmith, F. B. (2012). Monitoring for conservation and ecology (Vol. 3). Springer Science & Business Media.
Goutte, C., & Gaussier, E. (2005). A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. Paper presented at the European Conference on Information Retrieval.
Hobday, A. J., & Pecl, G. T. (2014). Identification of global marine hotspots: Sentinels for change and vanguards for adaptation action. Reviews in Fish Biology and Fisheries, 24(2), 415–425.
Igulu, M. M., Nagelkerken, I., Dorenbosch, M., Grol, M. G., Harborne, A. R., Kimirei, I. A., et al. (2014). Mangrove habitat use by juvenile reef fish: Meta-analysis reveals that tidal regime matters more than biogeographic region. PLoS One, 9(12), e114715.
Kalogeiton, V., Ferrari, V., & Schmid, C. (2016). Analysing domain shift factors between videos and images for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(11), 2327–2334.
Lecchini, D., & Galzin, R. (2005). Spatial repartition and ontogenetic shifts in habitat use by coral reef fishes (Moorea, French Polynesia). Marine Biology, 147(1), 47–58.
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.
Mandal, R., Connolly, R. M., Schlacher, T. A., & Stantic, B. (2018). Assessing fish abundance from underwater video using deep neural networks. Paper presented at the 2018 international joint conference on neural networks (IJCNN).
Massa, F., & Girshick, R. (2018). Maskrcnn-benchmark: Fast, modular reference implementation of instance segmentation and object detection algorithms in PyTorch.
Maxwell, S. L., Fuller, R. A., Brooks, T. M., & Watson, J. E. (2016). Biodiversity: The ravages of guns, nets and bulldozers. Nature News, 536(7615), 143–145.
Moniruzzaman, M., Islam, S. M. S., Bennamoun, M., & Lavery, P. (2017). Deep learning on underwater marine object detection: A survey. Paper presented at the International Conference on Advanced Concepts for Intelligent Vision Systems.
Podder, T. K., Sibenac, M., & Bellingham, J. G. (2019). Applications and challenges of AUV docking systems deployed for long-term science missions. Monterey Bay Aquarium Research Institute.
Pollock, B. (2017). Latitudinal change in the distribution of luderick Girella tricuspidata (Pisces: Girellidae) associated with increasing coastal water temperature in eastern Australia. Marine and Freshwater Research, 68(6), 1187–1192.
Prechelt, L. (1998). Early stopping-but when? In Neural Networks: Tricks of the trade (pp. 55-69): Springer.
Rawat, W., & Wang, Z. (2017). Deep convolutional neural networks for image classification: A comprehensive review. Neural Computation, 29(9), 2352–2449.
Ridgway, K. (2007). Long-term trend and decadal variability of the southward penetration of the east Australian current. Geophysical Research Letters, 34(13).
Salman, A., Maqbool, S., Khan, A. H., Jalal, A., & Shafait, F. (2019a). Real-time fish detection in complex backgrounds using probabilistic background modelling. Ecological Informatics, 51, 44–51.
Salman, A., Siddiqui, S. A., Shafait, F., Mian, A., Shortis, M. R., Khurshid, K., et al. (2019b). Automatic fish detection in underwater videos by a deep neural network-based hybrid motion learning system. ICES Journal of Marine Science.
Sarwar, S. S., Ankit, A., & Roy, K. (2019). Incremental learning in deep convolutional neural networks using partial network sharing. IEEE Access.
Sheaves, M., Bradley, M., Herrera, C., Mattone, C., Lennard, C., Sheaves, J., & Konovalov, D. A. (2020). Optimizing video sampling for juvenile fish surveys: Using deep learning and evaluation of assumptions to produce critical fisheries parameters. Fish and Fisheries.
Sievers, M., Brown, C. J., Tulloch, V. J., Pearson, R. M., Haig, J. A., Turschwell, M. P., et al. (2019). The role of vegetated coastal wetlands for marine megafauna conservation. Trends in Ecology & Evolution, 34, 807–817.
Silliman, B. R., He, Q., Angelini, C., Smith, C. S., Kirwan, M. L., Daleo, P., et al. (2019). Field experiments and meta-analysis reveal wetland vegetation as a crucial element in the coastal protection paradigm. Current Biology, 29(11), 1800–1806. e1803.
Spampinato, C., Giordano, D., Di Salvo, R., Chen-Burger, Y.-H. J., Fisher, R. B., & Nadarajan, G. (2010). Automatic fish classification for underwater species behavior understanding. In Proceedings of the first ACM international workshop on Analysis and retrieval of tracked events and motion in imagery streams (pp. 45–50).
Tao, Y., Tu, Y., & Shyu, M.-L. (2019). Efficient Incremental Training for Deep Convolutional Neural Networks. In 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) (pp. 286–291): IEEE.
Townhill, B. L., Radford, Z., Pecl, G., van Putten, I., Pinnegar, J. K., & Hyder, K. (2019). Marine recreational fishing and the implications of climate change. Fish and Fisheries, 20(5), 977–992.
Tulloch, V. J., Turschwell, M. P., Giffin, A. L., Halpern, B. S., Connolly, R., Griffiths, L., et al. (2020). Linking threat maps with management to guide conservation investment. Biological Conservation, 245, 108527.
Vergés, A., Doropoulos, C., Czarnik, R., McMahon, K., Llonch, N., & Poore, A. G. (2018). Latitudinal variation in seagrass herbivory: Global patterns and explanatory mechanisms. Global Ecology and Biogeography, 27(9), 1068–1079.
Villon, S., Chaumont, M., Subsol, G., Villéger, S., Claverie, T., & Mouillot, D. (2016). Coral reef fish detection and recognition in underwater videos by supervised machine learning: Comparison between Deep Learning and HOG+ SVM methods. In International Conference on Advanced Concepts for Intelligent Vision Systems (pp. 160–171). Springer.
Villon, S., Mouillot, D., Chaumont, M., Darling, E. S., Subsol, G., Claverie, T., & Villéger, S. (2018). A deep learning method for accurate and fast identification of coral reef fishes in underwater images. Ecological Informatics, 48, 238–244.
Weinstein, B. G. (2018). A computer vision for animal ecology. Journal of Animal Ecology, 87(3), 533–545.
Wendländer, N. S., Lange, T., Connolly, R. M., Kristensen, E., Pearson, R. M., Valdemarsen, T., & Flindt, M. R. (2020). Assessing methods for restoring seagrass (Zostera muelleri) in Australia’s subtropical waters. Marine and Freshwater Research, 71(8), 996–1005.
Whitmarsh, S. K., Fairweather, P. G., & Huveneers, C. (2017). What is big BRUVver up to? Methods and uses of baited underwater video. Reviews in Fish Biology and Fisheries, 27(1), 53–73.
Xu, W., & Matzner, S. (2018). Underwater fish detection using deep learning for water power applications. Paper presented at the 2018 international conference on computational science and computational intelligence (CSCI),
Acknowledgements
We thank the volunteers who assisted in manual training of the deep learning algorithm, M. Turner and A. Shand.
Funding
RC was supported by a Discovery Project from the Australian Research Council (DP180103124). All authors were supported by the Global Wetlands Project, with support by a charitable organisation which neither seeks nor permits publicity for its efforts.
Author information
Authors and Affiliations
Contributions
ED and RC designed the study. ED and SL conducted the fieldwork. ED and EJ developed the deep learning architecture and user interface. RC provided resources. All authors helped interpret results. ED led the writing of the manuscript, with input from all authors.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
ESM 1
(PDF 314 kb)
Rights and permissions
About this article
Cite this article
Ditria, E.M., Sievers, M., Lopez-Marcano, S. et al. Deep learning for automated analysis of fish abundance: the benefits of training across multiple habitats. Environ Monit Assess 192, 698 (2020). https://doi.org/10.1007/s10661-020-08653-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10661-020-08653-z