Course Description

Prototype-based machine learning and vector quantization methods : recent developments and applications

2-3 April 2007, UCL, Louvain-la-Neuve, Belgium.

This course is coordinated by Michel Verleysen, in the framework of the CIL Doctoral School and the MUSICS Graduate School.

Speakers

Prof. Barbara Hammer, Clausthal University (Germany), and Dr. Thomas Villmann, University of Lepizig (Germany).

Abstract

Prototype-based machine learning methods have a great potential as an intuitive and flexible toolbox for mining, visualization, and inspection of large data sets. They combine simple and human-understandable principles, such as distance-based classification, prototypes, or Hebbian learning, with a large variety of different, problem adapted design choices, such as a data-optimum topology, similarity measure, or learning mode. Classical prototype-based methods include well known supervised as well as unsupervised paradigms, such as learning vector quantization, neural gas, self-organizing map, and other. In recent years, a variety of new results could be achieved in this area with respect to both, theoretical foundations as well as algorithms.

The aim of the course is to give an overview about several recent major developments in this field related to new learning paradigms which are particularly important when dealing with complex large scale applications. Thereby, algorithmic development and theoretical background are accompanied by demonstrations for practical applications.

Detailed contents

A detailed list of topics tackled in the course is as follows:

• Supervised learning: Learning vector quantization (LVQ)
  • Dynamics of LVQ:
    Classical LVQ has been proposed as a powerful heuristic which shows instable behavior in particular for overlapping classes because of which a variety of (also heuristic) window rules has been introduced to overcome this problem. For these classical heuristics, a theoretical insight into the learning behavior is hardly available. Recently, different approaches have been proposed in the literature which, on the one side, allow to formally analyse the dynamic of LVQ training using methods of statistical physics, and, on the other side, introduce alternative LVQ learning rules which are based on mathematical principles such as a cost function.
  • Metric adaptation:
    LVQ depends heavily on the underlying (usually Euclidean) metric. To make it appropriate for large scale applications for complex data sets, extensions of LVQ to more general and adaptive metrics have been introduced, including the principle of relevance determination and arbitrary differentiable similarity measures and general kernels.
  • Generalization ability:
    It has recently been shown that LVQ can be interpreted as a large margin classifier, similar to support vector machines. Thereby, it is possible to derive explicit bounds on the generalization ability of LVQ classifiers with adaptive metric parameters. Learning strategies to actively select training samples can be built based on these bounds.
  • Application to clinical proteomics
• Unsupervised Learning: Neural gas (NG) and self-organizing map (SOM)
  • General data structures:
    NG and SOM have been proposed for vectorial data. Recently, a modification of these methods to deal with more general data structures has been introduced by substituting the metric in these methods. On the one hand, so-called median versions open the way towards general data for which only pairwise distances are available. On the other hand, recurrence can be included such that spatiotemporal data can be dealt with.
  • Enhancing by (fuzzy) labels:
    General clustering often suffers from the garbage-in-garbage-out problem: noisy data and missing knowledge about an appropriate metric yield to a result without useful information. This problem can be prevented by incorporating additional information into the model. One possibility which has recently proposed consists in the incorporation of (possibly fuzzy) label information in the cost function of the model.
  • Magnification control:
    NG and SOM can be used to represent a given data distribution, whereby the resulting prototypes are linked to the data distribution by a magnification exponent which is different from the information theoretically optimum exponent one. Vector quantizers with optimum exponent one can be based on information theoretic principles. Alternatively, a variety of methods exist which allow to control the exponent, thereby also enabling the suppression or, conversely, enhancing of rare events.
  • Use case:
    Satelite remote sensing image analysis

PhD students can benefit from the course in several ways:

• Topics:
  The course provides an overview about state-of-the-art algorithms and developments in the important subject of prototype-based machine learning such that it is relevant for PhD students working in the area of data mining, visualization, classification, clustering, or related.
• Methods:
  The course gives insights into important learning paradigms and techniques which are relevant and can be transferred to other areas of machine learning. Thus, PhD students who are involved in developments of methods or theoretical background in pattern recognition or statistical machine learning can benefit from the demonstration of techniques and ideas.
• Applications:
  The course presents several large scale applications of the methods in relevant areas (bioinformatics, satelite sensing), such that the way in which the presented methods can be applied to advanced problems becomes clear. Therefore, the course is relevant to PhD students who work in applications of machine learning to bioinformatics, image processing, or other applications in particular involving high dimensional and noisy data

Registration

Registration deadline: 26 March 2007.