TY - JOUR A1 - Stuhlsatz, André A1 - Lippel, Jens A1 - Zielke, Thomas T1 - Feature extraction with deep neural networks by a generalized discriminant analysis JF - IEEE transactions on neural networks and learning systems N2 - We present an approach to feature extraction that is a generalization of the classical linear discriminant analysis (LDA) on the basis of deep neural networks (DNNs). As for LDA, discriminative features generated from independent Gaussian class conditionals are assumed. This modeling has the advantages that the intrinsic dimensionality of the feature space is bounded by the number of classes and that the optimal discriminant function is linear. Unfortunately, linear transformations are insufficient to extract optimal discriminative features from arbitrarily distributed raw measurements. The generalized discriminant analysis (GerDA) proposed in this paper uses nonlinear transformations that are learnt by DNNs in a semisupervised fashion. We show that the feature extraction based on our approach displays excellent performance on real-world recognition and detection tasks, such as handwritten digit recognition and face detection. In a series of experiments, we evaluate GerDA features with respect to dimensionality reduction, visualization, classification, and detection. Moreover, we show that GerDA DNNs can preprocess truly high-dimensional input data to low-dimensional representations that facilitate accurate predictions even if simple linear predictors or measures of similarity are used. Y1 - 2012 U6 - https://doi.org/10.1109/tnnls.2012.2183645 VL - 23 IS - 4 SP - 596 EP - 608 PB - IEEE ER - TY - CHAP A1 - Becker, Martin A1 - Lippel, Jens A1 - Stuhlsatz, André T1 - Regularized Nonlinear Discriminant Analysis - An Approach to Robust Dimensionality Reduction for Data Visualization T2 - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 27.02.2017 - 01.03.2017, Porto, Portugal Y1 - 2017 SN - 978-989-758-228-8 U6 - https://doi.org/10.5220/0006167501160127 SP - 116 EP - 127 PB - SCITEPRESS - Science and Technology Publications ER - TY - JOUR A1 - Wolf, Christian A1 - Gaida, Daniel A1 - Stuhlsatz, André A1 - Ludwig, Thomas A1 - McLoone, Seán A1 - Bongards, Michael T1 - Predicting organic acid concentration from UV/vis spectrometry measurements – a comparison of machine learning techniques JF - Transactions of the Institute of Measurement and Control Y1 - 2013 U6 - https://doi.org/10.1177/0142331211403797 VL - 35 IS - 1 SP - 5 EP - 15 PB - sage journals ER - TY - CHAP A1 - Stuhlsatz, André A1 - Lippel, Jens A1 - Zielke, Thomas T1 - Discriminative feature extraction with Deep Neural Networks T2 - The 2010 International Joint Conference on Neural Networks (IJCNN), 18.07.2010 - 23.07.2010, Barcelona, Spain Y1 - 2010 UR - http://ieeexplore.ieee.org/document/5596329/ SN - 978-1-4244-6916-1 U6 - https://doi.org/10.1109/IJCNN.2010.5596329 SP - 1 EP - 8 PB - IEEE ER - TY - CHAP A1 - Wolf, Christian A1 - Gaida, Daniel A1 - Stuhlsatz, André A1 - McLoone, S. A1 - Bongards, Michael T1 - A Comparison of Performance of Advanced Pattern Recognition Methods for Organic Acid Prediction in Biogas Plants Using UV/vis Spectroscopic Online-Measurements T2 - Proceedings of the 2010 International Conference on Life System Modeling and Simulation (LSMS), Wuxi, China, 2010 Y1 - 2010 ER - TY - CHAP A1 - Gaida, Daniel A1 - Wolf, Christian A1 - Stuhlsatz, André A1 - Lippel, Jens A1 - Bäck, T. A1 - Bongards, Michael A1 - McLoone, S. T1 - State Estimation for Anaerobic Digesters using the ADM1 T2 - Proceedings of International IWA-Symposium on Anaerobic Digestion of Solid Waste and Energy Crops, Vienna, Austria, 2011 Y1 - 2011 ER - TY - CHAP A1 - Stuhlsatz, André A1 - Meier, Hans-Günter A1 - Katz, Michael A1 - Wendemuth, Andreas T1 - Support Vector Machines for Postprocessing of Speech Recognition Hypotheses T2 - Proceedings of the International Conference on Telecommunications and Multimedia (TEMU), Heraklion (Crete), Greece, 2006 Y1 - 2006 ER - TY - PAT A1 - Heimann, Thomas A1 - Plociennik, Uwe A1 - Müller, Jürgen A1 - Stuhlsatz, André T1 - Verfahren und Anlage zum Stranggiessen eines metallischen Produkts Y1 - 2009 PB - Europäisches Patentamt ER - TY - THES A1 - Stuhlsatz, André T1 - Machine learning with Lipschitz classifiers N2 - complex image, like for example that of a known person, and to distinguish it from other objects within half a second. While for a solution of this task the brain has access to a massive parallelism and a vast, hierarchically organized, and auto-associative memory, common computer architectures are just able to a sequential processing of information stored in a non auto-associative memory. Even modern, parallelly operating, multi-processor systems are far away from the performance of our brain. However, nowadays, it is possible to solve complex and memory extensive pattern recognition problems, like the recognition of handwritten digits or the transcription of speech, satisfactorily with a common computer by the use of modern statistical and algorithmic learning approaches. One of the most successful pattern recognition methods is the so-called Support Vector Machine (SVM). The SVM is based on the learning paradigm of structural risk minimization, which outperforms empirical approaches if only few data is available for solving the considered classification problem. Although the SVM has proven very good recognition performances in many cases, the SVM also comes up with limitations, for example if specific a priori knowledge shall be used. In particular, the increasing complexity of applications requires a high adaptivity of the classification method to the specific problem. Also concerning this point, the SVM is limited due to a restricted variety of implementable classification functions. The objective of the present thesis is the development of new learning algorithms for the classification of patterns, that on the one hand overcome the limitations of the SVM, but on the other hand are based on the same theoretical concepts facilitating the good performance of the SVM. Two new algorithms will be presented that are justified by a theoretical generalization of the SVM, and which will be utilized for the first time for a practical implementation. In contrast to the SVM, the new methods make accessible a much larger function class for constructing a classifier. This is an important prerequisite for flexible adaptation of the classifier to difficult classification tasks with particular requirements as well as for the integration of a priori knowledge about the problem at hand. In this work, the way to implementable algorithms leads across different mathematical reformulations of the original problem. Starting with the theoretical generalization of the SVM, it results a restricted optimization problem that is difficult to solve in general. In a first step, this problem is expressed in terms of a restricted minimax-problem by a modification of the suitable classification functions to a still very large function class consisting of (affine-)linear combinations of at least one-time continuously differentiable functions. In the next step, the minimax-problem is converted into a so-called Semi-Infinite Problem (SIP). It turns out, that this particular mathematical problem is appropriate in order to obtain a solution of the original problem for the considered function class using well-known optimization methods. To further exploit the problem structure, an equivalent dual problem is derived from the SIP. Therefore, we prove a duality theorem about the equality of the optimal values of the dual and the original problem. For solving the dual problem, a multilevel iterative approach is developed from which the proposed algorithms follow by pursuing different solution strategies. Moreover, all sub-optimization methods of any stage necessary for an implementation in software are developed. Namely, these are an adapted interior-point-method, a simulated annealing based search heuristics and a particular gradient decent approach. Furthermore, options are depicted for an improvement of efficiency for future implementations. Besides the emphasis on the theoretical development of new learning methods and their practical implementations, all algorithms were implemented in the MATLAB(R) programming environment for the experimental part of the present thesis. Hence, they are also available for further research purposes in future. For the first time, classification results are explored and evaluated in comparison to the SVM on different data sets. As test data, an artificial 2d-dataset as well as two real-world datasets were used. In the concluding experiment, a scenario is prototypically considered to which the SVM is only inadequately applicable and which shall precisely prove the capability of the new methods in that case. It follows, regarding the considered datasets, the proposed learning methods reach comparably good classification accuracy like the SVM in standard applications. Moreover, the particular benefit of the new methods is reflected theoretically and experimentally in the ability to solve classification problems using decision functions that are not accessible to SVMs. Thereby, the underlying ideas, which make the SVM excel compared to other approaches with respect to generalization performances in case of few available learning information, are adequately transported into the proposed new environment. This opens the way for a design and a use of new classifiers that have not been implementable in a robust and generalizing basic concept so far. Y1 - 2010 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:101:1-201104192945 PB - Otto-von-Guericke-Universität Magdeburg CY - Magdeburg ER - TY - JOUR A1 - Stuhlsatz, André ED - Fellbaum, Klaus T1 - HSVM - A SVM Toolkit for Segmented Speech Data JF - Studientexte zur Sprachkommunikation: Elektronische Sprachsignalverarbeitung 2007 KW - Spracherkennung Y1 - 2007 UR - https://web.archive.org/web/20250718065024/https://www.essv.de/pdf/2007_107_114.pdf SN - 978-3-940046-40-6 SN - 0940-6832 SP - 107 EP - 114 ER - TY - CHAP A1 - Stuhlsatz, André A1 - Meier, Hans-Günter A1 - Katz, M. A1 - Krüger, S. E. A1 - Wendemuth, Andreas ED - Wendemuth, Andreas T1 - Classification of Speech Recognition Hypotheses with Support Vector Machines T2 - Proceedings of the Speech Processing Workshop: Magdeburg, Germany, September 09, 2003 ; in connection with the symposium of the German Association for Pattern Recognition (DAGM) 2003 Y1 - 2003 SN - 9783929757590 PB - Otto-von-Guericke-Universität Magdeburg CY - Magdeburg ER - TY - CHAP A1 - Gaida, Daniel A1 - Stuhlsatz, André A1 - Meier, Hans-Günter ED - Wendemuth, Andreas ED - Meier, Hans-Günter T1 - Fusion of Visual and Inertial Measurements for Pose Estimation T2 - Proceedings of the Workshop on Emotion-, Speech- and Face Recognition with Advanced Classifiers: Düsseldorf, Germany, May 16 - 17, 2008 ; 3rd joined research workshop between Otto-von-Guericke University Magdeburg and the University of Applied Sciences FH Düsseldorf Y1 - 2008 SN - 9783940961242 SP - 39 EP - 48 PB - Otto-von-Guericke University Magdeburg CY - Magdeburg ER -