Assembly Dataset for Contact Based tasks for Learning from Demonstration

Assembly Dataset for Contact Based Tasks for Learning from Demonstration The Dataset is a detailed collection of motion and visual data during kinesthetic demonstration gathered for the HARTU project, specifically for two use cases: PCL (Philips Consumer Lifestyle) assembly of electric shaver's body to lackering fixture and Tofas (Türk Otomobil Fabrikası A.Ş) bearing assembly of the brakes. This dataset supports research and development for Learning from Demonstrations of contact based tasks in robotics, focusing on motion planning, control, and computer vision applications related to contact based assembly tasks. Dataset Contents 1. Use Cases PCL Assembly: Demonstrations related to the two assembly tasks in the PCL assembly process. Tofas Assembly: Demonstrations related to the two assembly tasks in the Tofas assembly process. 2. Motion Information The motion data provides detailed information about the movement and operation of the dual arm KUKA robot. This includes: Cartesian Positions: The positions of the robot's end effectors in 6D Cartesian coordinates, position in 3D (x, y, z) and orientation represented as quaternion (x,y,z,w). Joint Positions: The joint angles or positions of each joint in the robot's arms. Velocity: The speed of movement in both Cartesian positions and joint positions. Force/Torque: The forces and torques experienced by the robot's end effectors during the demonstrations measured using FT sensor and torques measured from joints.  Gripper Actions: Information about the actions performed by the robot's grippers, such as open, close during assembly operation. 3. Camera Information The visual data is captured using an intel Realsense d455 RGB-D camera, which provides both color and depth information for each frame. This includes: RGB Data: Color images captured during the demonstrations, providing visual context for the robot's actions. Depth Data: Depth images that represent the distance of objects from the camera, useful for 3D reconstruction and spatial analysis. 4. Data Format The dataset is stored in HDF5 files, a hierarchical data format that allows for efficient storage and retrieval of large datasets. Each HDF5 file contains: Motion Data: Cartesian positions, joint positions, velocities, forces, and gripper actions. Camera Data: RGB and depth images. In addition, ROS bags are recorded during each demonstration, capturing all the relavant ROS topics published during the demonstrations for easy playback and analysis. Format and Structure The dataset is organized into directories corresponding to individual demonstrations for each use case. Again each use case is recorded for complete execution and step wise execution. Each directory contains the following files: demo.hdf5: An HDF5 file containing motion and camera data recorded during demonstration. rosbag2*: A ROS bag folder containing metadata and db3 file capturing required ROS topics during the demonstration. Resources for Working with HDF5 and ROS Bags HDF5 Resources HDF5 Viewer: HDFView is a tool for browsing and editing HDF5 files.  Python Library: h5py is a Python library for interacting with HDF5 files. ROS Bag Resources ROS Bag Tutorials: ROS Wiki rqt_bag: A GUI plugin for visualizing ROS bag files. Usage This dataset can be used for LfD applications, including but not limited to: Robotics Motion Planning: Developing and testing algorithms for controlling dual arm robots in industrial assembly tasks. Robot Vision: Training and evaluating models for object recognition, tracking, and 3D reconstruction in assembly processes. Force Control: Studying the interaction forces between the robot and its environment during assembly tasks. Licensing and Citation Please refer to the repository's LICENSE file for terms of use. If you use this dataset in your research, please cite it as per the citation details given. Contact For questions or further information, please contact Vamsi Origanti at vamsi.origanti@dfki.de.The HARTU project supports this work. This project has received funding from the European Union’s research and innovation programme Horizon Europe under the grant agreement No. 101092100.