Lecturer
Sergios Theodoridis ( Dept. of Informatics and Telecommunication, National and Kapodistrian University of Athens, Greece And Chinese University of Hong Kong, Shenzhen, China)
Schedule and place
This 15-hour course will take place in 5 sessions over five days on October 14,15,16,17,18, 2019 at KU Leuven - Auditorium Arenberg Castle, Kasteelpark Arenberg 1, 3001 Heverlee
Schedule: 3 hours/day.
From 10:00 to 13:00 (including a 30-minute coffee break at 11:15)
From 14:00 to 17:00 (including a 30-minute coffee break at 16:15)
- October 14: from 14:00 to 17:00
- October 15: from 14:00 to 17:00
- October 16: from 14:00 to 17:00
- October 17: from 14:00 to 17:00
- October 18: from 10:00 to 13:00
Including a 30 minute coffee break in each session in the salons of the Castle
Parking is possible at Parking The Molen (see the map here)
For the parking there is a daily code:
14 october:8771#
15 october:7017#
16 october:1170#
17 october:7565#
18 october:5395#
Please note this code should be used to enter AND to leave the parking
Description
In this short course, a brief tour to Machine Learning (ML) “land” will be attempted. We will
begin from the “beginning”. Two of the major ML pillars will be introduced, namely the
regression, in its simplest Least Squares formulation, and the classification tasks. The notion
of model overfitting will be outlined, which will bring into the scene Occam’s razor rule and
the need for parameter regularization. The Bayes classifier and the logistic regression will offer
us their own optimal flavor in grasping the basics. We will move on to present two classical
parameter estimation methods, namely the Maximum Likelihood (ML) and Maximum APosteriori
(MAP) techniques. The basic “secrets” behind the powerful probabilistic view of
parameter learning via the Bayesian palette of tools will be briefly established. The EM
algorithm and its variational approximation version will be discussed. Depending on time and
the audience’ s background, the basic rationale of learning in reproducing kernel Hilbert spaces
(RKHS) will be presented.
In thesequel, the tour will follow more recent advances. Neural networks will be “visited”,
starting from their late 19th century “spring”, with the discovery of the neuron, and then we
will “stop” at the major milestones. The artificial neuron, the perceptron and the multilayer
feedforward NN will be the very first to “look” at. Backpropagation and some up to date related
optimization algorithms will be discussed. Nonlinear activation functions will be presented, in
the context of their effect on the training algorithm’s convergence. In the sequel, techniques
guarding against overfitting will be outlined, such as the dropout approach. The final path will
evolve along the most modern advances in the terrain. Convolutional networks (CNN) and
recurrent neural networks (RNN) will be “visited” and discussed. Adversarial examples,
generative adversarial networks (GANs) and the basics on capsule modules will also be part of
the tour. If time allows, some “bridges” will be established that bring together deep networks
and the Bayesian spirit.
The Classics
1) Parametric modelling: Regression and Classification
2) Bias Variance Tradeoff and Overfitting
3) Ridge Regression
4) Bayes Classifiers, Minimum Distance Classifiers and Logistic Regression
5) Maximum Likelihood, Maximum a-Posteriori Estimation
6) Bayesian Learning: Basic Principles and the EM algorithm
7) Variational EM and the Mean Field approximation
8) Reproducing Hilbert Kernel Spaces and the Kernel Trick
9) Support Vector Regression and Machines.
Neural Networks and Deep Learning
1) Perceptron and the Perceptron rule
2) Feedforward Neural Networks
3) The Backpropagation and the Gradient Diminishing/Expanding Effect
4) Sigmoid and ReLU Activation Units
5) Regularizing a network and the Dropout method
6) The need for Deep Networks
7) Convolutional Networks
8) High-way and Residual Networks
9) Recurrent Neural Networks and the LSTM
10) Building Attention Mechanism in a Network
11) Adversarial Examples
12) Generative Adversarial Networks
13) Capsule Networks
14) NN with a Bayesian flavor (if time allows)
15) Open Issues and Future Challenges.
Course material
To be determined
Evaluation
To be determined.