ROV-Based Multi-Sensor Dataset: Synchronized Camera and Sonar images taken in the Tropical Waters of the Red Sea, Eilat

Description: This dataset consists of approximately 46,928 synchronized image pairs collected by the  Blue-ROV2. The images were captured using a machine-vision camera (IDS UI-3260CP-C-HQ)  and a BluePrint Oculus M1200d Forward-Looking Sonar (FLS). Both sensors were installed with the FLS tilted 15 degrees downward to achieve optimal coverage of the terrain and optimal FOV overlap. The data was collected to train and evaluate a comprehensive perception and obstacle avoidance framework.   Context: This dataset is the second installment in our collection of synchronized multi-sensor underwater  datasets, aimed at enabling advanced research in multi-modal sensor fusion, obstacle  detection, and navigation for autonomous underwater vehicles (AUVs). The data was collected  using the Blue-ROV2 Remotely Operated Vehicle (ROV) in the tropical waters of the Red Sea,  off the coast of Eilat, Israel. This data captures diverse underwater environments and is part of a  research project focused on developing fusion models for improved obstacle detection and  navigation in AUVs.   Content: The data encompasses several sites within the tropical waters of the Red Sea, Eilat, including  corals, rocks, shipwrecks, man-made structures, piers, and caves. The ROV platform was  operated by divers, ensuring accurate positioning and coverage. Data was acquired at depths  ranging from 3 to 12 meters at different times from dawn to dusk.   Dataset Composition: Site Recording Session Image Pairs Description Tropical Site 1 20221211_092506 20221211_133252 10,915 7,978 Pier, rocks, corals Tropical Site 2 20221212_095821 20221212_141308 9,900 8,475 Man-made structure,  rocks, corals Tropical Site 3 20221213_102542 9,390 Rocks, corals Total   46,928      The dataset is organized into separate sessions, each representing a specific dive or  experiment. Within each session, data is further categorized into modalities: camera (FLC  images), sonar (FLS images), and depth. Each modality directory contains the corresponding  data files in PNG format for images and CSV format for depth data.   Each modality directory includes: A `camera.csv` file for the camera modality that maps each image file to its respective  timestamp. A `sonar.csv` file for the sonar modality that maps each image file to its respective timestamp. The depth data in `depth.csv` formatted with `timestamp` and `value`. Additionally, a `samples.json` file documents the relationship between uni-modal and  multi-modal samples, enabling easy association of data from different modalities.   Technical Details: Camera: IDS UI-3260CP-C-HQ Image dimensions: 1936x1216 pixels (downscaled to 968 × 608 for this dataset) Sensor type: Sony IMX249 1/1.2" CMOS Lens: Tamron M112FM06 Captured bit depth: 8-bit Frame rate: 5 Hz Sonar: BluePrint Oculus M1200d Operating frequency: 1.2 MHz (low frequency mode) Maximum range: 40 m (set to 15 m for this dataset) Horizontal aperture: 130° Vertical aperture: 20° Number of beams: 512 Angular resolution: 0.6° Beam separation: 0.25° Image resolution: 544x300 pixels Coordinate system: Polar Frame rate: 5 Hz Depth: Blue-Robotics Ping2 Sonar Altimeter and Echosounder Frequency: 115 kHz Source Level: 198 dB re 1µPa @ 1m Beamwidth: 25 degrees Typical Minimum Range: 0.3 m (1 ft) Typical Usable Range: 100 m (328 ft) Range Resolution: 0.5% of range Depth Rating: 300 m (984 ft) Data format: CSV Columns: timestamp: Unix timestamp (seconds) value: Depth value (meters) Sample rate: 5 Hz   Example File Tree Layout: ```${session}/${dataset}/camera/camera.csv00000001.png00000002.png…sonar/sonar.csv00000001.png00000002.png…depth/depth.csvsamples.json``` Example File Content:  camera.csv```timestamp,filename1644234340.181234,00000001.png1644234343.375667,00000002.png```  sonar.csv```timestamp,filename1644234340.181234,00000001.png1644234343.375667,00000002.png```  depth.csv```timestamp,value1644234340.181234,5.41644234343.375667,6.1```  samples.json ```{    "samples": [        {            "camera": [                0            ],            "depth": [                0            ],            "sonar": [                0            ]        },        {            "camera": [                1            ],            "depth": [                1            ],            "sonar": [                1            ]        }] ``` By providing synchronized and aligned camera, sonar imagery, and depth data, this dataset  enables researchers to explore novel algorithms and techniques for multi-modal sensor fusion in  the context of autonomous underwater vehicles operating in the tropical waters of the Red Sea. Acknowledgements The data in this repository is part of the DeeperSense project that received funding from the European Commission, Program H2020-ICT-2020-2 ICT-47-2020, Project Number: 101016958.