Imitation learning framework for the TM5-900 collaborative robot arm equipped with the OnRobot RG6 gripper. The package interfaces Isaac Sim 4.5 with ROS 2 Humble and can be used as an expert demonstration generator.
The resulting demonstrations are saved in the LeRobot dataset format.
- Ubuntu 22.04 PC
- ROS 2 Humble (Python3)
- Isaac Sim 4.5
- LeRobot Dataset Format
- SceneHandler node using Omniverse libraries
- AnalyticSolver node using ground truth information to solve the randomized task
- TrajectoryRecorder node records robot state - action pairs
Using the previous section's links, install every prerequisites.
Navigate into your ROS 2 workspace and copy this repo to your source folder
cd src && git clone https://github.com/ABC-iRobotics/isaac_sim_imitation_learning.git && cd ..
Build the package (and installing python prerequisites)
colcon build --packages-select isaac_sim_msgs isaac_sim_scene_handler analytic_solver
After installation use the TMFlow to make a simulated robot. Set up the Ethernet Slave settings.
The expert demonstration generation demo can be initialized with the included launch file.
ros2 launch isaac_sim_imitation_learning bringup_robot_scene.launch.py robot_ip:=<robot_ip>
where robot_ip is the simulated robot's ip address.
Using a separate terminal the demonstration generation can be called via a ROS 2 service.
ros2 service call /TrajectoryRecorder/GetTrajectory isaac_sim_msgs/srv/Demonstration "{amount:<num_of_demos>, path: '<save_path>'}"
where num_of_demos is the number of requested successfull demonstrations and save_path is the absolute path ("~" can be used) where the completed demonstrations are saved.
In case of any issues, check the official resources:
Makány András - Graduate student at Obuda University
This software is released under the MIT License, see LICENSE.