General information:
Code:
UBPJO-128
Name:
Introduction to Deep Learning
Profile of education:
Academic (A)
Lecture language:
English
Semester:
Fall
Responsible teacher:
Kurdziel Marcin (kurdziel@agh.edu.pl)
Academic teachers:
Kurdziel Marcin (kurdziel@agh.edu.pl)
Module summary

Description of learning outcomes for module
MLO code Student after module completion has the knowledge/ knows how to/is able to Connections with FLO Method of learning outcomes verification (form of completion)
Skills
M_U001 Students can design and implement a deep neural network with unsupervised pretraining and supervised fine-tuning. Completion of laboratory classes
M_U002 Students can train a deep neural network to solve a given machine learning task. Students can asses performance of the final model. Completion of laboratory classes
Knowledge
M_W001 Students have knowledge on feed-forward neural networks and selected generative models. Examination
M_W002 Students know methods and algorithms for unsupervised and supervised training of deep neural networks. Examination
FLO matrix in relation to forms of classes
MLO code Student after module completion has the knowledge/ knows how to/is able to Form of classes
Lecture
Audit. classes
Lab. classes
Project classes
Conv. seminar
Seminar classes
Pract. classes
Zaj. terenowe
Zaj. warsztatowe
Others
E-learning
Skills
M_U001 Students can design and implement a deep neural network with unsupervised pretraining and supervised fine-tuning. - - + - - - - - - - -
M_U002 Students can train a deep neural network to solve a given machine learning task. Students can asses performance of the final model. - - + - - - - - - - -
Knowledge
M_W001 Students have knowledge on feed-forward neural networks and selected generative models. + - - - - - - - - - -
M_W002 Students know methods and algorithms for unsupervised and supervised training of deep neural networks. + - - - - - - - - - -
Module content
Lectures:

  1. Machine learning and artificial neural networks.
  2. Generative models.
  3. Restricted Boltzmann Machine.
  4. Contrastive Divergence algorithm.
  5. Deep Belief Network.
  6. Backpropagation algorithm.
  7. Logistic regression and Softmax regression.
  8. Unsupervised pretraining in deep neural networks.
  9. Regularization in neural networks. Dropout.

Laboratory classes:

  1. Introductory lab – matrix algebra in Matlab.
  2. Restricted Boltzmann Machine. Implementation of the Contrastive Divergence algorithm.
  3. Layer-wise training of Deep Belief Networks. Sampling from a Deep Belief Network.
  4. Backpropagation in a Multilayer Perceptron with Softmax regression.
  5. Unsupervised pretraining of a Multilayer Perceptron.
  6. Dropout.

Student workload (ECTS credits balance)
Student activity form Student workload
Summary student workload 100 h
Module ECTS credits 4 ECTS
Participation in lectures 14 h
Preparation for classes 35 h
Realization of independently performed tasks 27 h
Participation in laboratory classes 14 h
Contact hours 8 h
Examination or Final test 2 h
Additional information
Method of calculating the final grade:

To get a positive final grade students must complete all laboratory classes and pass the final test. The final grade is then an average of grades from the test (30%) and the laboratory classes (70%).

Prerequisites and additional requirements:

Elementary knowledge of matrix algebra and probability theory. Basic programming skills.

Recommended literature and teaching resources:

1. Christopher M. Bishop: Neural Networks for Pattern Recognition, Oxford University Press, 1996
2. Christopher M. Bishop: Pattern Recognition and Machine Learning, Springer, 2007
3. Yoshua Bengio: Learning Deep Architectures for AI, Foundations & Trends in Machine Learning, 2009

Scientific publications of module course instructors related to the topic of the module:

1. K. Grzegorczyk, M. Kurdziel, P.I Wójcik “Effects of Sparse Initialization in Deep Belief Networks”, Computer Science (in print).
2. K. Grzegorczyk, M. Kurdziel, P.I Wójcik “Implementing deep learning algorithms on graphics processor units”, Parallel Processing and Applied Mathematics 2015 (accepted).

Additional information:

None