TY - CHAP A1 - Pepe, Alberto A1 - Buchholz, Sven A1 - Lasenby, Joan T1 - Clifford Group Equivariant Neural Network Layers for Protein Structure Prediction T2 - Northern Lights Deep Learning Conference, VOL 233, 09-11 Januar 2024, Tromso, Norwegen N2 - We employ Clifford Group Equivariant Neural Network (CGENN) layers to predict protein coordinates in a Protein Structure Prediction (PSP) pipeline. PSP is the estimation of the 3D structure of a protein, generally through deep learning architectures. Information about the geometry of the protein chain has been proven to be crucial for accurate predictions of 3D structures. However, this information is usually flattened as machine learning features that are not representative of the geometric nature of the problem. Leveraging recent advances in geometric deep learning, we redesign the 3D projector part of a PSP architecture with the addition of CGENN layers . CGENNs can achieve better generalization and robustness when dealing with data that show rotational or translational invariance such as protein coordinates, which are independent of the chosen reference frame. CGENNs inputs, outputs, weights and biases are objects in the Geometric Algebra of 3D Euclidean space, i.e. G3,0,0, and hence are interpretable from a geometrical perspective. We test 6 approaches to PSP and show that CGENN layers increase the accuracy in term of GDT scores by up to 2.1\%, with fewer trainable parameters compared to linear layers and give a clear geometric interpretation of their outputs. Y1 - 2024 UR - https://proceedings.mlr.press/v233/pepe24a/pepe24a.pdf SP - 1 EP - 7 PB - PMLR ER - TY - CHAP A1 - Pepe, Alberto A1 - Lasenby, Joan A1 - Buchholz, Sven T1 - CGAPoseNet+GCAN: A Geometric Clifford Algebra Network for Geometry-aware Camera Pose Regression T2 - Winter Conference on Applications of Computer Vision, Waikoloa, Hawaii, US, 2024 N2 - We introduce CGAPoseNet+ GCAN, which enhances CGAPoseNet, an architecture for camera pose regression, with a Geometric Clifford Algebra Network (GCAN). With the addition of the GCAN we obtain a geometry-aware pipeline for camera pose regression from RGB images only. CGAPoseNet employs Clifford Geometric Algebra to unify quaternions and translation vectors into a single mathematical object, the motor, which can be used to uniquely describe camera poses. CGAPoseNet solves the issue of balancing rotation and translation components in the loss function, and can obtain comparable results to other approaches without the need of expensive tuning of the loss function or additional information about the scene, such as 3D point clouds, which might not always be available. CGAPoseNet, however, like several approaches in the literature, only learns to predict motor coefficients, and it is unaware of the mathematical space in which predictions sit in and of their geometrical meaning. By leveraging recent advances in Geometric Deep Learning, we modify CGAPoseNet with a GCAN: proposals of possible motor coefficients associated with a camera frame are obtained from the InceptionV3 backbone, and the GCAN downsamples them to a single motor through a sequence of layers that work in G_ 4, 0. The network is hence geometry-aware, has multivector-valued inputs, weights and biases and preserves the grade of the objects that it receives in input. CGAPoseNet+ GCAN has almost 4 million fewer trainable parameters, it reduces the average rotation error by 41% and the average translation error by 8.8% compared to CGAPoseNet. Similarly, it reduces rotation and translation errors by 32.6% and 19.9%, respectively, compared to the best performing PoseNet strategy. CGAPoseNet+ GCAN reaches the state-of-the-art results on 13 commonly employed datasets. To the best of our knowledge, it is the first experiment in GCANs applied to the problem of camera pose regression. Y1 - 2024 U6 - https://doi.org/10.1109/WACV57701.2024.00646 SP - 6593 EP - 6603 PB - IEEE ER - TY - INPR A1 - Pepe, Alberto A1 - Lasenby, Joan A1 - Buchholz, Sven T1 - STAResNet: A Network in Spacetime Algebra to solve Maxwell's PDEs T2 - arXiv N2 - We introduce STAResNet, a ResNet architecture in Spacetime Algebra (STA) to solve Maxwell's partial differential equations (PDEs). Recently, networks in Geometric Algebra (GA) have been demonstrated to be an asset for truly geometric machine learning. In [1], GA networks have been employed for the first time to solve partial differential equations (PDEs), demonstrating an increased accuracy over real-valued networks. In this work we solve Maxwell's PDEs both in GA and STA employing the same ResNet architecture and dataset, to discuss the impact that the choice of the right algebra has on the accuracy of GA networks. Our study on STAResNet shows how the correct geometric embedding in Clifford Networks gives a mean square error (MSE), between ground truth and estimated fields, up to 2.6 times lower than than obtained with a standard Clifford ResNet with 6 times fewer trainable parameters. STAREsNet demonstrates consistently lower MSE and higher correlation regardless of scenario. The scenarios tested are: sampling period of the dataset; presence of obstacles with either seen or unseen configurations; the number of channels in the ResNet architecture; the number of rollout steps; whether the field is in 2D or 3D space. This demonstrates how choosing the right algebra in Clifford networks is a crucial factor for more compact, accurate, descriptive and better generalising pipelines. Y1 - 2024 UR - https://arxiv.org/pdf/2408.13619 SP - 1 EP - 17 PB - arXiv ER - TY - CHAP A1 - Pepe, Alberto A1 - Lasenby, Joan A1 - Buchholz, Sven T1 - GA-ReLU: an activation function for Geometric Algebra Networks applied to 2D Navier-Stokes PDEs T2 - ICLR 2024 Workshop on AI4DifferentialEquations In Science, Wien, Österreich N2 - Many differential equations describing physical phenomena are intrinsically geometric in nature. It has been demonstrated how this geometric structure of data can be captured effectively through networks sitting in Geometric Algebra (GA) that work with multivectors, making them suitable candidates to solve differential equations. GA networks however, are still mostly uncharted territory. In this paper we focus on non-linearities, since applying them to multivectors is not a trivial task: they are generally applied in a point-wise fashion over each real-valued component of a multivector. This approach discards interactions between different elements of the multivector input and compromises the geometric nature of GA networks. To bridge this gap, we propose GA-ReLU, a GA approach to the rectified linear unit (ReLU), and show how it can improve the solution of Navier-Stokes PDEs. Y1 - 2024 UR - https://openreview.net/pdf?id=T54Hn1e98P SP - 1 EP - 6 ER -