Using biological features to improve deep neural network models for vision

Convolutional neural networks and vision transformers represent the state of the art in artificial neural network (ANN) models for vision problems, such as classification, segmentation, and object detection. However, the performance of these models still falls behind human performance in many problems and is highly susceptible to image variation, lighting conditions, and deliberate attacks. Recent results have shown that it is possible to draw inspiration from the architecture and function of the visual cortex to improve the performance of ANNs and to make these systems more robust to a wide range of image perturbations. The objective of this dissertation is to study how structural and functional characteristics of the primate visual pathways can be used to derive new layers and optimization goals in deep neural networks that contribute to improving their robustness and performance in image classification tasks. Efficient coding algorithms, used in the retina and the primary visual cortex, and different connection patterns between layers are some of the approaches that will be tested. The novel models will be assessed both in terms of their performance in existing computer vision benchmarks and on how well their internal components and behavioral output match those of real primate brains using the Brain-Score platform. The work will be co-supervised by Tiago Marques, currently at the Champalimaud Foundation. The selected student will have access to the facilities of INESC-ID and the MLKD group ( https://mlkd.idss.inesc-id.pt/ ), including computing facilities that include four DELL PowerEdge C41402 servers, eight NVIDIA 32GB Tesla V100S and eight NVIDIA 64GB Tesla A100, among other computing servers ( https://mlkd.idss.inesc-id.pt/cluster )