How I built a language-controlled multi-robot system for hospitals | Isaac Sim devlog

Описание: In this video, I showcase how I built a multi-robot system controlled by natural language inside Isaac Sim and how you can use this in your own project.

The project is open source and by following this tutorial / devlog you will understand all the details and be able to use this approach in your next project.

I realized this project in 2 weeks during the REVELxLYCHEEAI Hackaton and won the 3rd place with 600$. The project brings together a Franka Research 3 arm and the A2D robot from agibot, both collaborating to assist nurses and doctors.

Thanks ​⁠‪@LycheeAI‬ and ​⁠‪@REVELSTUDIOSINC‬

🎬 Timestamps
00:00 - Introduction
00:32 - Context
01:32 - How to build an Extension
02:39 - User Interface
03:45 - Picking objects from a shelf
06:29 - Configure a mobile manipulator
08:30 - Controlling a mobile manipulator
10:10 - Synchronization & Language
11:45 - Conlusion

🔗 Links
The github repository : https://github.com/loanBRNT/delivery_...
Nvidia Extension Creation Tutorial : https://docs.isaacsim.omniverse.nvidi...
Nvidia Lula COnfiguration Tutorial : https://docs.isaacsim.omniverse.nvidi...

❓ If you have any questions, reach me on Linkedin :   / loan-bernat-928b9a1b0  

👮‍♂️ Copyrights
Music by MacLeod and MokkaMusic

#hackaton #lycheeai #isaac #robotics #ai #llm