9th Workshop "Theoretical Foundations of Computer Vision"

Evaluation and Validation of Computer Vision Algorithms

Schloß Dagstuhl/Wadern (near Saarbrücken/Germany)

March 16 - 20, 1998

Chairmen

Robert Haralick (Seattle), Reinhard Klette (Auckland),

H. Siegfried Stiehl (Hamburg), Max Viergever (Utrecht)


This workshop addressed a subject which has been under active discussion in computer vision for several years. The evaluation and validation of algorithms is of basic importance for the configuration of computer vision applications.

"...the authors should clearly state their assumptions and justify their claims based on results that can be obtained under those assumptions. Vague justifications, such as subjective evaluations of images ... should not be allowed." (R.C. Jain, Th.O. Binford: Ignorance, myopia, and naivetÈ in computer vision systems. Computer Vision, Graphics, and Image Processing: Image Understanding 53 (1991), pp. 112 - 117. )
In the ideal case certain "data sheets" should allow to qualify algorithmic solutions in a specific context, e.g. defined by image data, goal of image analysis, or software environment ("edge detection is not equal to edge detection"). There is a lack of methodological fundamentals in the field of performance analysis.

Workshop Program (pdf file, 25k)

Final Workshop Report (pdf file, 261k)

Participants presented ideas towards general methodology of evaluation and validation of algorithms or/and the application of these techniques to computer vision algorithms.

The proceedings of the workshop have been published: Klette, R., Stiehl, S., Viergever, M., Vincken, V. (eds.): Performance Evaluation of Computer Vision Algorithms. Kluwer Academic Publishers, Amsterdam 2000.

Related Web Sites

Algorithm Performance Contest, ICPR 2000
ECV NET Benchmarking and Performance Evaluation
IEEE Computer Society Workshop 1998
DAGM-Working Group Quality Characteristics of Pattern Recognition Algorithms

Workshops "Theoretical Foundations of Computer Vision":

10th (2000): Multi-Image Search, Filtering, Reasoning and Visualization (chairs: Alfred Bruckstein, Thomas Huang, Reinhard Klette, SongDe Ma).
8th (1996): Advances in Computer Vision (ed. by F. Solina, W. Kropatsch, R. Klette and R. Bajcsy), Advances in Computing Science, Springer Wien New York, 1997 (ISSN 1433-0113).
7th (1994): Theoretical Foundations of Computer Vision (ed. by W. Kropatsch, R. Klette and F. Solina), Computing Supplement 11, Springer Wien New York, 1996 (ISSN 0344-8029).
6th (1992): Theoretical Foundations of Computer Vision (ed. by R. Klette and W. Kropatsch), Mathematical Research, Akademie Verlag Berlin, 1992 (ISSN 0138-3019).
5th (1990): Theoretische Grundlagen der Digitalen Bildverarbeitung, Mägdesprung/Harz
(no proceedings published).
4th (1988): Theoretische Grundlagen der Digitalen Bildverarbeitung (ed. by R. Klette and W.-D. Klix), Heft 104/88, Technische Universität Dresden, Sektion Mathematik, 1988.
3rd (1986): Digitale Bildverarbeitung (Theoretische Grundlagen) (ed. by R. Klette and W.-D. Klix), Heft 93/86, Technische Universität Dresden, Sektion Mathematik, 1986.
2nd (1984): Methoden und Algorithmen der Digitalen Bildverarbeitung (ed. by R. Klette and W.-D. Klix), Heft 77/84, Technische Universität Dresden, Sektion Mathematik, 1984.
1st (1982): Digitale Bildverarbeitung (ed. by R. Klette and W.-D. Klix), Heft 60/82, Technische Universität Dresden, Sektion Mathematik, 1982.

CITR: last update: 4 April 2000