TY - JOUR A1 - Rajan, Kohulan A1 - Brinkhaus, Henning Otto A1 - Steinbeck, Christoph A1 - Zielesny, Achim T1 - Advancements in hand-drawn chemical structure recognition through an enhanced DECIMER architecture JF - Journal of Cheminformatics N2 - Accurate recognition of hand-drawn chemical structures is crucial for digitising hand-written chemical information in traditional laboratory notebooks or facilitating stylus-based structure entry on tablets or smartphones. However, the inherent variability in hand-drawn structures poses challenges for existing Optical Chemical Structure Recognition (OCSR) software. To address this, we present an enhanced Deep lEarning for Chemical ImagE Recognition (DECIMER) architecture that leverages a combination of Convolutional Neural Networks (CNNs) and Transformers to improve the recognition of hand-drawn chemical structures. The model incorporates an EfficientNetV2 CNN encoder that extracts features from hand-drawn images, followed by a Transformer decoder that converts the extracted features into Simplified Molecular Input Line Entry System (SMILES) strings. Our models were trained using synthetic hand-drawn images generated by RanDepict, a tool for depicting chemical structures with different style elements. A benchmark was performed using a real-world dataset of hand-drawn chemical structures to evaluate the model's performance. The results indicate that our improved DECIMER architecture exhibits a significantly enhanced recognition accuracy compared to other approaches. KW - DECIMER KW - Hand-drawn chemical structures KW - OCSR, Optical Chemical Structure Recognition KW - Transformer KW - Deep Learning Y1 - 2024 U6 - https://doi.org/10.1186/s13321-024-00872-7 VL - 2024 IS - 16: 78 ER - TY - INPR A1 - Rajan, Kohulan A1 - Brinkhaus, Henning Otto A1 - Zielesny, Achim A1 - Steinbeck, Christoph T1 - Advancements in Hand-Drawn Chemical Structure Recognition through an Enhanced DECIMER Architecture N2 - Accurate recognition of hand-drawn chemical structures is crucial for digitising hand-written chemical information found in traditional laboratory notebooks or for facilitating stylus-based structure entry on tablets or smartphones. However, the inherent variability in hand-drawn structures poses challenges for existing Optical Chemical Structure Recognition (OCSR) software. To address this, we present an enhanced Deep lEarning for Chemical ImagE Recognition (DECIMER) architecture that leverages a combination of Convolutional Neural Networks (CNNs) and Transformers to improve the recognition of hand-drawn chemical structures. The model incorporates an EfficientNetV2 CNN encoder that extracts features from hand-drawn images, followed by a Transformer decoder that converts the extracted features into Simplified Molecular Input Line Entry System (SMILES) strings. Our models were trained using synthetic hand-drawn images generated by RanDepict, a tool for depicting chemical structures with different style elements. To evaluate the model's performance, a benchmark was performed using a real-world dataset of hand-drawn chemical structures. The results indicate that our improved DECIMER architecture exhibits a significantly enhanced recognition accuracy compared to other approaches. KW - OCSR, Optical Chemical Structure Recognition KW - DECIMER KW - Deep Learning KW - Transformer Y1 - 2024 U6 - https://doi.org/10.26434/chemrxiv-2024-7ch9f ER - TY - JOUR A1 - Rajan, Kohulan A1 - Brinkhaus, Henning Otto A1 - Sorokina, Maria A1 - Zielesny, Achim A1 - Steinbeck, Christoph T1 - DECIMER-Segmentation: Automated extraction of chemical structure depictions from scientific literature JF - Journal of Cheminformatics Y1 - 2021 U6 - https://doi.org/10.1186/s13321-021-00496-1 VL - 13 SP - Artikelnr. 20 ER - TY - JOUR A1 - Rajan, Kohulan A1 - Brinkhaus, Henning Otto A1 - Agea, M. Isabel A1 - Zielesny, Achim A1 - Steinbeck, Christoph T1 - DECIMER.ai: an open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications JF - Nature Communications N2 - The number of publications describing chemical structures has increased steadily over the last decades. However, the majority of published chemical information is currently not available in machine-readable form in public databases. It remains a challenge to automate the process of information extraction in a way that requires less manual intervention - especially the mining of chemical structure depictions. As an open-source platform that leverages recent advancements in deep learning, computer vision, and natural language processing, DECIMER.ai (Deep lEarning for Chemical IMagE Recognition) strives to automatically segment, classify, and translate chemical structure depictions from the printed literature. The segmentation and classification tools are the only openly available packages of their kind, and the optical chemical structure recognition (OCSR) core application yields outstanding performance on all benchmark datasets. The source code, the trained models and the datasets developed in this work have been published under permissive licences. An instance of the DECIMER web application is available at https://decimer.ai. KW - machine learning KW - artificial intelligence KW - AI KW - optical chemical structure recognition KW - OCSR Y1 - 2023 U6 - https://doi.org/10.1038/s41467-023-40782-0 VL - 2023 IS - 14: 5045 ER - TY - INPR A1 - Brinkhaus, Henning Otto A1 - Rajan, Kohulan A1 - Zielesny, Achim A1 - Steinbeck, Christoph T1 - RanDepict - Random Chemical Structure Depiction Generator T2 - ChemRxiv N2 - The development of deep learning-based optical chemical structure recognition (OCSR) systems has led to a need for datasets of chemical structure depictions. The diversity of the features in the training data is an important factor for the generation of deep learning systems that generalise well and are not overfit to a specific type of input. In the case of chemical structure depictions, these features are defined by the depiction parameters such as bond length, line thickness, label font style and many others. Here we present RanDepict, a toolkit for the creation of diverse sets of chemical structure depictions. The diversity of the image features is generated by making use of all available depiction parameters in the depiction functionalities of the CDK, RDKit, and Indigo. Furthermore, there is the option to enhance and augment the image with features such as curved arrows, chemical labels around the structure, or other kinds of distortions. Using depiction feature fingerprints, RanDepict ensures diversely picked image features. Here, the depiction and augmentation features are summarised in binary vectors and the MaxMin algorithm is used to pick diverse samples out of all valid options. By making all resources described herein publicly available, we hope to contribute to the development of deep learning-based OCSR systems. Y1 - 2022 U6 - https://doi.org/10.26434/chemrxiv-2022-t1kbb ER - TY - JOUR A1 - Brinkhaus, Henning Otto A1 - Zielesny, Achim A1 - Steinbeck, Christoph A1 - Rajan, Kohulan T1 - DECIMER—hand-drawn molecule images dataset JF - Journal of Cheminformatics N2 - The translation of images of chemical structures into machine-readable representations of the depicted molecules is known as optical chemical structure recognition (OCSR). There has been a lot of progress over the last three decades in this field, but the development of systems for the recognition of complex hand-drawn structure depictions is still at the beginning. Currently, there is no data for the systematic evaluation of OCSR methods on hand-drawn structures available. Here we present DECIMER — Hand-drawn molecule images, a standardised, openly available benchmark dataset of 5088 hand-drawn depictions of diversely picked chemical structures. Every structure depiction in the dataset is mapped to a machine-readable representation of the underlying molecule. The dataset is openly available and published under the CC-BY 4.0 licence which applies very few limitations. We hope that it will contribute to the further development of the field. Y1 - 2022 SN - 1758-2946 VL - 14.2022 SP - 1 EP - 5 PB - BioMed Central CY - London ER - TY - INPR A1 - Brinkhaus, Henning Otto A1 - Zielesny, Achim A1 - Steinbeck, Christoph A1 - Rajan, Kohulan T1 - DECIMER - Hand-drawn molecule images dataset N2 - The translation of images of chemical structures into machine-readable representations of the depicted molecules is known as optical chemical structure recognition (OCSR). There has been a lot of progress over the last three decades in this field, but the development of systems for the recognition of complex hand-drawn structure depictions is still at the beginning. Currently, there is no data for the systematic evaluation of OCSR methods on hand-drawn structures available. Here we present DECIMER - Hand-drawn molecule images, a standardised, openly available benchmark dataset of 5088 hand-drawn depictions of diversely picked chemical structures. Every structure depiction in the dataset is mapped to a machine-readable representation of the underlying molecule. The dataset is openly available and published under the CC-BY 4.0 licence which applies very few limitations. We hope that it will contribute to the further development of the field. KW - OCSR KW - Hand-drawn images KW - Molecule images KW - Deep learning KW - Chemical structure depictions KW - Optical Chemical Structure Recognition Y1 - 2022 ER - TY - CHAP A1 - Rajan, Kohulan A1 - Brinkhaus, Henning Otto A1 - Zielesny, Achim A1 - Steinbeck, Christoph T1 - The DECIMER (Deep lEarning for Chemical IMagE Recognition) project BT - May 8 -12, 2022 in Garmisch-Partenkirchen/Germany T2 - 17th German Conference on Cheminformatics Y1 - 2022 PB - Gesellschaft Deutscher Chemiker CY - Frankfurt a.M. ER - TY - CHAP A1 - Rajan, Kohulan A1 - Brinkhaus, Henning Otto A1 - Zielesny, Achim A1 - Steinbeck, Christoph T1 - The DECIMER (Deep lEarning for Chemical IMagE Recognition) project BT - 12-16 June 2022, Noordwijkerhout, The Netherlands T2 - 12th International Conference on Chemical Structures Y1 - 2022 ER - TY - JOUR A1 - Brinkhaus, Henning Otto A1 - Rajan, Kohulan A1 - Schaub, Jonas A1 - Zielesny, Achim A1 - Steinbeck, Christoph T1 - Open data and algorithms for open science in AI-driven molecular informatics JF - Current Opinion in Structural Biology N2 - Recent years have seen a sharp increase in the development of deep learning and artificial intelligence-based molecular informatics. There has been a growing interest in applying deep learning to several subfields, including the digital transformation of synthetic chemistry, extraction of chemical information from the scientific literature, and AI in natural product-based drug discovery. The application of AI to molecular informatics is still constrained by the fact that most of the data used for training and testing deep learning models are not available as FAIR and open data. As open science practices continue to grow in popularity, initiatives which support FAIR and open data as well as open-source software have emerged. It is becoming increasingly important for researchers in the field of molecular informatics to embrace open science and to submit data and software in open repositories. With the advent of open-source deep learning frameworks and cloud computing platforms, academic researchers are now able to deploy and test their own deep learning models with ease. With the development of new and faster hardware for deep learning and the increasing number of initiatives towards digital research data management infrastructures, as well as a culture promoting open data, open source, and open science, AI-driven molecular informatics will continue to grow. This review examines the current state of open data and open algorithms in molecular informatics, as well as ways in which they could be improved in future. Y1 - 2023 U6 - https://doi.org/https://doi.org/10.1016/j.sbi.2023.102542 VL - 2023 IS - 79 ER -