TY - CHAP A1 - Surmann, Hartmut A1 - Digakis, Niklas A1 - Kremer, Jan-Niklas A1 - Meine, Julien A1 - Schulte, Max A1 - Voigt, Niklas T1 - Redefining Recon: Bridging Gaps with UAVs, 360° Cameras, and Neural Radiance Fields N2 - In the realm of digital situational awareness during disaster situations, accurate digital representations, like 3D models, play an indispensable role. To ensure the safety of rescue teams, robotic platforms are often deployed to generate these models. In this paper, we introduce an innovative approach that synergizes the capabilities of compact Unmaned Arial Vehicles (UAVs), smaller than 30 cm, equipped with 360° cameras and the advances of Neural Radiance Fields (NeRFs). A NeRF, a specialized neural network, can deduce a 3D representation of any scene using 2D images and then synthesize it from various angles upon request. This method is especially tailored for urban environments which have experienced significant destruction, where the structural integrity of buildings is compromised to the point of barring entry—commonly observed post-earthquakes and after severe fires. We have tested our approach through recent post-fire scenario, underlining the efficacy of NeRFs even in challenging outdoor environments characterized by water, snow, varying light conditions, and reflective surfaces. KW - Small UAVs KW - 360° Panorama KW - Visual Monocular SLAM KW - Rescue Robotics KW - NeRF Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:1010-opus4-44603 SP - 1 EP - 6 ER - TY - VIDEO A1 - Meine, Julien A1 - Voigt, Niklas A1 - Kremer, Jan-Niklas A1 - Digakis, Niklas A1 - Schulte, Max A1 - Surmann, Hartmut T1 - 3D point cloud model from the DRZ trainings site N2 - From the 360° images of the former video ( • German rescue robotic center captured... ) we now generate the 3D point cloud. The UAV needs 3 minutes to capture the outdoor scenario and the hall from inside and outside. The 3D point cloud generation is 5x slower than the video. It uses a VSLAM algorithm to localize the k-frames (green) and with 3 k-frames it use a 360° PatchMatch algorithm implemented at a NVIDIA graphic card (CUDA) to calculated the dense point clouds.The hall ist about 70 x 20 meters. Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:1010-opus4-43134 ER - TY - VIDEO A1 - Meine, Julien A1 - Voigt, Niklas A1 - Kremer, Jan-Niklas A1 - Digakis, Niklas A1 - Schulte, Max A1 - Surmann, Hartmut T1 - Drone Flight Documentation with ARGUS at rescue operations N2 - ARGUS is a tool for the systematic acquisition, documentation and evaluation of drone flights in rescue operations. In addition to the very fast generation of RGB and IR orthophotos, a trained AI can automatically detect fire, people and cars in the images captured by the drones. Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:1010-opus4-43141 ER - TY - VIDEO A1 - Meine, Julien A1 - Voigt, Niklas A1 - Kremer, Jan-Niklas A1 - Digakis, Niklas A1 - Schulte, Max A1 - Surmann, Hartmut T1 - NeRF: 3D Reconstruction of Chemical Company After Tank Explosion in Kempen (August 17, 2023) N2 - The video showcases a 3D model of a chemical company following a tank explosion that occurred on August 17, 2023, in Kempen computed with the AI algorithm Neural Radiance Field (NeRF). Captured by a compact mini drone measuring 18cm x 18cm and equipped with a 360° camera, these images offer an intricate perspective of the aftermath. After a comprehensive aerial survey and inspection of the 360° images taken within the facility, authorities confirmed that it was safe for the evacuated residents to return to their homes. See also: https://www1.wdr.de/fernsehen/aktuelle-stunde/alle-videos/video-grosser-chemieunfall-in-kempen-100.html Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:1010-opus4-43236 ER - TY - VIDEO A1 - Meine, Julien A1 - Voigt, Niklas A1 - Kremer, Jan-Niklas A1 - Digakis, Niklas A1 - Schulte, Max A1 - Surmann, Hartmut T1 - Nerf(acto) for 3D modeling of the DRZ outside testing area N2 - The video shows a very high resolution 3D point cloud !!! of the outdoor area of the German Rescue Robotics Center. For the recording, a 25-second POI flight was performed with a Mavic 3. From the 4K video footage captured during this flight, 77 images were cropped and localized within 4 minutes using colmap and processed using Neural Radiance Fields (NeRF). The nerfacto model of Nerfstudio was trained on an Nvidia RTX 4090 for 8 minutes. In summary, a top 3D model is available to task forces after about 13 minutes. The calculation is performed locally on site by the RobLW of the DRZ. The video shown here shows a free camera path rendered at 60 hz (Full HD). Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:1010-opus4-43156 ER - TY - VIDEO A1 - Meine, Julien A1 - Voigt, Niklas A1 - Kremer, Jan-Niklas A1 - Digakis, Niklas A1 - Schulte, Max A1 - Surmann, Hartmut T1 - Nerf(acto) for the 3D modeling of the Computer Science building of Westfälische Hochschule GE N2 - The video shows a very high resolution 3D point cloud !!! of the computer science building of the University of Applied Science Gelsenkirchen. For the recording a 3 minute flight with a M30T was performed. The 105 images taken by the wide-angle camera during this flight were localized within 3 minutes using colmap and processed using Neural Radiance Fields (NeRF). The nerfacto model of Nerfstudio was trained on an Nvidia RTX 4090 for 8 minutes. Thus, a top 3D model is available after about 15 minutes. The video shown here shows a free camera path rendered at 60 hz (Full HD). Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:1010-opus4-43165 ER - TY - VIDEO A1 - Meine, Julien A1 - Voigt, Niklas A1 - Kremer, Jan-Niklas A1 - Digakis, Niklas A1 - Schulte, Max A1 - Surmann, Hartmut A1 - Goos, Richard A1 - Grafe, Robert T1 - Challenging Dataset: 3D point cloud generated with the nerfacto neural network model for 360° videos N2 - Challenging Dataset: 3D point cloud generated with the nerfacto neural network model for 360° videos Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:1010-opus4-43199 ER - TY - VIDEO A1 - Meine, Julien A1 - Voigt, Niklas A1 - Kremer, Jan-Niklas A1 - Digakis, Niklas A1 - Schulte, Max A1 - Surmann, Hartmut A1 - Goos, Richard A1 - Grafe, Robert T1 - Challenging Dataset: 3D Pointcloud generated with the PatchMatch algorithm for 360° videos N2 - Challenging Dataset: 3D Pointcloud generated with the PatchMatch algorithm for 360° videos Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:1010-opus4-43188 ER - TY - VIDEO A1 - Meine, Julien A1 - Voigt, Niklas A1 - Kremer, Jan-Niklas A1 - Digakis, Niklas A1 - Schulte, Max A1 - Surmann, Hartmut T1 - 3D model generated with the nerfacto AI algorithm after a fire based on 360° camera images N2 - Essen 2 2022, indoor, DJI FPV with insta360 Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:1010-opus4-43212 ER - TY - VIDEO A1 - Meine, Julien A1 - Voigt, Niklas A1 - Kremer, Jan-Niklas A1 - Digakis, Niklas A1 - Schulte, Max A1 - Surmann, Hartmut T1 - 3D model generated with the nerfacto AI algorithm after a fire N2 - Essen 2 / 2022, outdoor, based on a meander flight with a P4 and Mavic Enterprise. Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:1010-opus4-43228 ER - TY - VIDEO A1 - Meine, Julien A1 - Voigt, Niklas A1 - Kremer, Jan-Niklas A1 - Digakis, Niklas A1 - Schulte, Max A1 - Surmann, Hartmut T1 - Gaussian Splatting: 3D Reconstruction of a Chemical Company After a Tank Explosion in Kempen 8/2023 N2 - The video showcases a 3D model of a chemical company following a tank explosion that occurred on August 17, 2023, in Kempen computed with the gaussian splatting algorithm. Captured by a compact mini drone measuring 18cm x 18cm and equipped with a 360° camera, these images offer an intricate perspective of the aftermath. The computation need 29 minutes and uses 2770 images (~350 equirectangular images). After a comprehensive aerial survey and inspection of the 360° images taken within the facility, authorities confirmed that it was safe for the evacuated residents to return to their homes. See also: https://www1.wdr.de/fernsehen/aktuelle-stunde/alle-videos/video-grosser-chemieunfall-in-kempen-100.html Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:1010-opus4-43262 ER -