TY - INPR A1 - Rajan, Kohulan A1 - Brinkhaus, Henning Otto A1 - Zielesny, Achim A1 - Steinbeck, Christoph T1 - Advancements in Hand-Drawn Chemical Structure Recognition through an Enhanced DECIMER Architecture N2 - Accurate recognition of hand-drawn chemical structures is crucial for digitising hand-written chemical information found in traditional laboratory notebooks or for facilitating stylus-based structure entry on tablets or smartphones. However, the inherent variability in hand-drawn structures poses challenges for existing Optical Chemical Structure Recognition (OCSR) software. To address this, we present an enhanced Deep lEarning for Chemical ImagE Recognition (DECIMER) architecture that leverages a combination of Convolutional Neural Networks (CNNs) and Transformers to improve the recognition of hand-drawn chemical structures. The model incorporates an EfficientNetV2 CNN encoder that extracts features from hand-drawn images, followed by a Transformer decoder that converts the extracted features into Simplified Molecular Input Line Entry System (SMILES) strings. Our models were trained using synthetic hand-drawn images generated by RanDepict, a tool for depicting chemical structures with different style elements. To evaluate the model's performance, a benchmark was performed using a real-world dataset of hand-drawn chemical structures. The results indicate that our improved DECIMER architecture exhibits a significantly enhanced recognition accuracy compared to other approaches. KW - OCSR, Optical Chemical Structure Recognition KW - DECIMER KW - Deep Learning KW - Transformer Y1 - 2024 U6 - https://doi.org/10.26434/chemrxiv-2024-7ch9f ER - TY - JOUR A1 - Rajan, Kohulan A1 - Brinkhaus, Henning Otto A1 - Steinbeck, Christoph A1 - Zielesny, Achim T1 - Advancements in hand-drawn chemical structure recognition through an enhanced DECIMER architecture JF - Journal of Cheminformatics N2 - Accurate recognition of hand-drawn chemical structures is crucial for digitising hand-written chemical information in traditional laboratory notebooks or facilitating stylus-based structure entry on tablets or smartphones. However, the inherent variability in hand-drawn structures poses challenges for existing Optical Chemical Structure Recognition (OCSR) software. To address this, we present an enhanced Deep lEarning for Chemical ImagE Recognition (DECIMER) architecture that leverages a combination of Convolutional Neural Networks (CNNs) and Transformers to improve the recognition of hand-drawn chemical structures. The model incorporates an EfficientNetV2 CNN encoder that extracts features from hand-drawn images, followed by a Transformer decoder that converts the extracted features into Simplified Molecular Input Line Entry System (SMILES) strings. Our models were trained using synthetic hand-drawn images generated by RanDepict, a tool for depicting chemical structures with different style elements. A benchmark was performed using a real-world dataset of hand-drawn chemical structures to evaluate the model's performance. The results indicate that our improved DECIMER architecture exhibits a significantly enhanced recognition accuracy compared to other approaches. KW - DECIMER KW - Hand-drawn chemical structures KW - OCSR, Optical Chemical Structure Recognition KW - Transformer KW - Deep Learning Y1 - 2024 U6 - https://doi.org/10.1186/s13321-024-00872-7 VL - 2024 IS - 16: 78 ER -