In this project we’ve created a robot that can drive in a route that was initially defined by the user in a very intuitive, simulator-like way. Then, while driving, the robot searches for balls and when a ball has been detected – it navigates to it, emulate pick-up and on user’s order, returns to the initial route for further search.
The robot based on Kubuki base robot, which delivers basic necessary movement interface, combined with ROS (Robotic Operating System) packages that gives many different tools for communication and operation that have been the foundation for the first phase of the whole process – driving autonomously in a user’s pre-defined route. The second phase starts when a ball has been detected. With the aid of a 2D – RGB camera we’ve developed an algorithm that with the power of YOLO – an open source trained CNN for object detection, enables the ability to declare that a ball was found and we’re ready for the second phase – navigate to the ball.
The robot remembers the position where it left the initial route, and starts a controlled and modified navigation towards the ball, all while managing distance, adapting speed and keeping in the right direction. When reached a certain threshold in terms of distance and angle, the robot stops, emulate a pick-up and return to the initial pre-defined route.
A couple of hardware elements that were crucial for proper workflow are:
– LiDAR sensor
– Camera
– Wi-Fi adapter
– Voltage converter
– Nvidia’s jetson nano
– Kubuki base
Last but not least, we would like to mention the challenge we faced at the very first stage of the project in the form of software bring up that required a non-negligible amount of work, that included OS installation and versions coordination between different software packages and hardware elements and so on.