Class Integrated Artificial Intelligence

The “Embedded AI” curricular unit addresses the practical application of advanced deep learning techniques in Embedded computing contexts. This emerging field combines the fundamental principles of deep neural networks with the resource constraints and efficiency demands of embedded devices, such as IoT sensors, wearables, or mobile devices. The UC explores methods of model compression, performance optimization and efficient implementation of deep neural networks in embedded devices, preparing students to face the challenges of developing intelligent applications in low computing power and low latency environments.

Not applicable

Part I: Deep Learning Frameworks and Definitions Feedforward neural networks (FNN). Convolutional neural networks (CNN) and convolution operations. Recurrent neural networks, long-term memory (LSTM) and closed recurrent units (GRU). Representations of Bidirectional Transformers and Encoders (BERT). The attention mechanism. Definition and exposure of Hyperparameters Part II: Pruning Static Pruning, Pruning Criteria, Pruning Combined with Tuning or Re-training. Dynamic Pruning, Conditional Computing, Reinforcement Learning Adaptive Networks, Differential Adaptive Networks. Part III: Quantization Reduced Numerical Precision, Linear and Non-Linear Quantization, Uniform and Non-Uniform Quantization, Symmetric and Asymmetric Quantization, Simulated and Integer Quantization, Mixed Precision Quantization. Post-training quantization and conscious quantization training. Part VI: Case Studies TensorFlow and TensorFlow Lite to improve deep neural networks.

The teaching methodologies take a blended approach, where there is a first phase of the curricular unit with the exposure of theoretical contents, and a second focused on the tutorial monitoring of practical cases of implementation and analysis of algorithms. Collaborative programming tools such as Google Colaboratory will be used in both.

Vadera, Sunil, and Salem Ameen. "Methods for pruning deep neural networks." IEEE Access 10 (2022): 63280-63300. Han, Song, Huizi Mao, and William J. Dally. "Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding." arXiv preprint arXiv:1510.00149 (2015). Molchanov, Pavlo, et al. "Pruning convolutional neural networks for resource efficient inference." arXiv preprint arXiv:1611.06440 (2016). Novac, Pierre-Emmanuel, et al. "Quantization and deployment of deep neural networks on microcontrollers." Sensors 21.9 (2021): 2984. Gholami, Amir, et al. "A survey of quantization methods for efficient neural network inference." Low-Power Computer Vision. Chapman and Hall/CRC, 2022. 291-326. Nagel, Markus, et al. "A white paper on neural network quantization." arXiv preprint arXiv:2106.08295 (2021).