FG Graphische Systeme
Refine
Year of publication
Document Type
Way of publication
- Open Access (2)
Keywords
- Pflanzenmodellierung (8)
- L-Systeme (6)
- FSPM (5)
- graph grammars (5)
- functional-structural plant models (3)
- 3-D structure (2)
- Driver emotion recognition (2)
- GroIMP (2)
- L-system (2)
- L-systems (2)
Institute
BTU
Research profile line
The emergence of Vision-Language Models (VLMs) like CLIP (Contrastive Language-Image Pretraining) provides appealing solutions to various vision problems including Dynamic Facial Expression Recognition (DFER). However, most of the proposed approaches face major challenges, particularly related to inefficient full fine-tuning of the encoders and the complexity of the models. Moreover, some of the proposed methods seem to struggle with suboptimal performance due to (i) poor alignment between textual and visual representations, and (ii) ineffective temporal modeling. To address these challenges, we propose PE-CLIP, a parameter-efficient fine-tuning (PEFT) framework that elegantly adapts CLIP for dynamic facial expression recognition, requiring significantly reduced number of trainable parameters while maintaining high accuracy. At its core, to enhance efficiency and performance, PE-CLIP introduces two specialized adapters namely a Temporal Dynamic Adapter (TDA) and a Shared Adapter (ShA). The temporal dynamic adapter is a GRU-based module with a dynamic scaling mechanism, capturing sequential dependencies while adaptively modulating the contribution of each temporal feature to emphasize the most informative ones while mitigating irrelevant variations. The shared adapter is a lightweight adapter refine representations within both textual and visual encoders, ensuring consistent feature processing while maintaining parameter efficiency. Additionally, we leverage Multi-modal Prompt Learning (MaPLe), which introduces learnable prompts to both visual and action unit-based textual description inputs, further improving the semantic alignment between modalities and enabling the efficient adaptation of CLIP for dynamic tasks. We evaluate our proposed PE-CLIP on two benchmark datasets, namely DFEW, FERV39K, and AFEW, achieving competitive performance compared to state-of-the-art methods while requiring fewer trainable parameters. By striking an optimal balance between parameter efficiency and performance, PE-CLIP sets a new benchmark in resource-efficient DFER. The source code of the proposed PE-CLIP will be publicly available at
https://github.com/Ibtissam-SAADI/PE-CLIP
.
We are witnessing an increasing interest in vision-language models (VLMs) as reflected in the impressive results across a large spectrum of tasks. In this context, we introduce in this paper a novel architecture that exploits the capabilities of VLMs for facial expression recognition in driving environment to enhance road safety. We present an approach called CLIVP-FER, which uses the Contrastive Language-Image Pretraining (CLIP) and combines both visual and textual data to overcome the environmental challenges and ambiguities in facial expression interpretation. In addition, we apply average pooling to improve the accuracy and the computational efficiency. The proposed approach is thoroughly evaluated on a benchmark driving dataset called KMU-FED. The experiments showed superior performance compared to state-of-the-art methods, achieving an average accuracy of 97.36%. Cross-database evaluation is also provided showing good generalization abilities. The ablation study gives more insights into the performance of our proposed architecture. The obtained results are interesting and confirm the capabilities of vision-language models in vision tasks, demonstrating their promising applications in efficient driver assistance and intervention systems. We are making the code of this work publicly available for research purposes at https://github.com/Ibtissam-SAADI/CLIVP-FER.
3D reconstruction of partial foot scans using different state of the art neural network approaches
(2024)
As online shopping continues to grow in popularity, shoes are increasingly being purchased without being physically tried on. This has resulted in a significant surge in returns, causing both financial and environmental consequences. To tackle this issue, several systems are available to measure foot dimensions accurately either in-store or at home. By obtaining precise foot measurements, individuals can determine their ideal shoe size and prevent unnecessary returns. In order to make such a system as simple as possible for the user, only a single image should be sufficient to measure the foot. To make this possible, point clouds from one side of the foot, which are generated by taking a depth image, are to be used. Since these point clouds represent only one side of the foot, the other side has to be generated. For this purpose, different existing state of the art networks were tested and compared to determine which architecture is best suited for this task. After implementing, re-training on our own dataset and testing the different architectures, it can be concluded that the point/transormer-based network SnowflakeNet is the most efficient to be used for our task.
Recent advances in laser scanning systems have enabled the acquisition of 3D point cloud representations of scenes, revolutionizing the fields of Architecture, Engineering, and Construction (AEC). This paper presents a novel pipeline for the automatic generation of 3D semantic models of multi-level buildings from indoor point clouds. The architectural components are extracted hierarchically. After segmenting the point clouds into potential building floors, a wall detection process is performed on each floor segment. Then, room, ground, and ceiling extraction are conducted using the walls 2D constellation obtained from the projection of the walls onto the ground plan. The identification of the openings in the walls is performed using a deep learning-based classifier that separates doors and windows from non-consistent holes. Based on the geometric and semantic information from previously detected elements, the final model is generated in IFC format. The effectiveness and reliability of the proposed pipeline are demonstrated through extensive experiments and visual inspections. The results reveal high precision and recall values in the extraction of architectural elements, ensuring the fidelity of the generated models. In addition, the pipeline’s efficiency and accuracy offer valuable contributions to future advancements in point cloud processing.
Driving is an integral part of daily life for millions of people worldwide, and it has a profound impact on road safety and human health. The emotional state of the driver, including feelings of anger, happiness, or fear, can significantly affect their ability to make safe driving decisions. Recognizing the facial expressions of drivers(DFER) has emerged as a promising technique for improving road safety and can provide valuable information about their emotions, This information can be used by intelligent transportation systems (ITS), like advanced driver assistance systems (ADAS) to take appropriate decision, such as alerting the driver or intervening in the driving process, to prevent the potential risks. This survey paper presents a comprehensive survey of recent studies that focus on the problem of recognizing the facial expression of driver recognition in the driving context from 2018 to March 2023. Specifically, we examine studies that address the recognition of the driver’s emotion using facial expressions and explore the challenges that exist in this field, such as illumination conditions, occlusion, and head poses. Our survey includes an analysis of different techniques and methods used to identify and categorize specific expressions or emotions of the driver. We begin by reviewing and comparing available datasets and summarizing state-of-the-art methods, including machine learning-based methods, deep learning-based methods, and hybrid methods. We also identify limitations and potential areas for improvement. Overall, our survey highlights the importance of recognizing driver facial expressions in improving road safety and provides valuable insights into recent developments and future research directions in this field.
Shuffle vision transformer : lightweight, fast and efficient recognition of driver facial expression
(2024)
Existing methods for driver facial expression recognition (DFER) are often computationally intensive, rendering them unsuitable for real-time applications. In this work, we introduce a novel transfer learning-based dual architecture, named ShuffViT-DFER, which elegantly combines computational efficiency and accuracy. This is achieved by harnessing the strengths of two lightweight and efficient models using convolutional neural network (CNN) and vision transformers (ViT). We efficiently fuse the extracted features to enhance the performance of the model in accurately recognizing the facial expressions of the driver. Our experimental results on two benchmarking and public datasets, KMU-FED and KDEF, highlight the validity of our proposed method for real-time application with superior performance when compared to state-of-the-art methods.
Driver's facial expression recognition plays a critical role in enhancing driver safety, comfort, and overall driving experience by proactively mitigating potential road risks. While most existing works in this domain rely on CNN - based approaches, this paper proposes a novel method for driver facial expression recognition using Global Context Vision Transformer (DFER-GCViT). With its inherent capabilities of transformer-based architectures and global context modeling, the proposed method handles challenges commonly encountered in real-world driving scenarios, including occlusions, head pose variations, and illumination conditions. Our method consists of three modules: preprocessing for face detection and data augmentation, facial feature extraction of local and global features, and expression classification using a modified GC-ViT classifier. To evaluate the performance of DFER-GCViT, extensive experiments are conducted on two benchmarking datasets namely the KMU-FED driver facial expression dataset and FER2013 general facial expression dataset. The experimental results demonstrate the superiority of DFER-GCViT in accurately recognizing driver's facial expressions, achieving an average accuracy of 98.27 % on the KMU-FED dataset and 73.78% on the FER2013 dataset, outperforming several state-of-the-art methods on these two benchmarking datasets.
The human voice is one of our most important tools for communicating with other people. Besides pure semantic meaning it also conveys syntactical information such as emphasis as well as personal information such as emotional state, gender, and age. While the physical changes that occur to a person’s voice are well studied, there is surprisingly little work on the perception of those changes. To hold the range of subtleties present in a given utterance constant and thus focus on the changes caused by age, this paper takes adult recordings (three males, and three females) and artificially resynthesizes them (using values from measurements of real children’s voices) to create a childlike versions of the utterance at different target ages. In particular, we focus on a systematic, factorial combination pitch shifting and formant shifting. To get an insight about the influence of these factors on the estimated age, we performed a perceptual experiment. Since the resynthesis method we used can produce a wide range of voices, not all of which are physically consistent, we also asked the participants to rate how natural the voices sounded. Furthermore, since former studies suggest that people are not able to distinguish between males and females of young ages, participants were also asked to rate how male or female the voices sounded. Overall, we found that although the synthesis method produced physically plausible signals (compared average values for real children), the degree of signal manipulation was correlated with perceived unnaturalness. We also found that pitch shift had only a small affect on perceived age, that formant shift had a strong affect on perceived age, and that these effects depended on the original gender of the recording. As expected, people had difficulty guessing the gender of younger sounding voices.
Perceptual Evaluation of Common Line Variables for Displaying Uncertainty on Molecular Surfaces
(2022)
Data are often subject to some degree of uncertainty, whether aleatory or epistemic. This applies both to experimental data acquired with sensors as well as to simulation data. Displaying these data and their uncertainty faithfully is crucial for gaining knowledge. Specifically, the effective communication of the uncertainty can influence the interpretation of the data and the users' trust in the visualization. However, uncertainty-aware visualization has gotten little attention in molecular visualization. When using the established molecular representations, the physicochemical attributes of the molecular data usually already occupy the common visual channels like shape, size, and color. Consequently, to encode uncertainty information, we need to open up another channel by using feature lines. Even though various line variables have been proposed for uncertainty visualizations, they have so far been primarily used for two-dimensional data and there has been little perceptual evaluation. Therefore, we conducted a perceptual study to determine the suitability of the line variables sketchiness, dashing, grayscale, and width for distinguishing several uncertainty values on molecular surfaces.
Erosion as a novel Approach for removing Semantics and Comparison of different State-of-Art-Methods
(2022)
Through language, people convey not only pure semantics, but also information about themselves, such as age, gender, state of mind or health. The supralingual features that carry this information have been a subject of research for a long time. Various procedures have been proposed to remove unneeded semantics from speech recordings, in order to study supralingual information in natural speech. In this paper, we propose a new method for removing sematics, based on erosion, a morphological operator. We compare its effectiveness to different state-of-the-art methods. As established methods we consider two low pass filters with cut off frequencies of 450Hz and 1150Hz and Brownian noise. As a newer method we investigate a filter for spectro-temporal frequencies. To evaluate each method, appropriately processed recordings were presented to a group of participants in a perceptual experiment. The intelligibility was measured by means of the Levenshtein distance. Our results show that erosion itself performs similarly to the established methods, while a combination of erosion and low-pass filter outperforms all other methods.