Sawyer Arm Plays Ping Pong
Using Stereo Vision and Model Predictive Control
Using Stereo Vision and Model Predictive Control
In this project, we aim to use stereo vision and Model Predictive Control (MPC) to enable Sawyer arm to play ping pong against an opponent. Due to time and hardware constraints, we define "playing ping pong" as having the sawyer robot see an incoming ball, predict its trajectory, and quickly move to strike the ball.
Unlike many other projects, we look to make a robot that achieves a simple tasks but requires impressive physical movements that is fast and accurate. As most team members are studying advanced Model Predictive Control concepts, we are interested to see it applied in real world hardware to make such movements possible.
Designing a controller from scratch is challenging, but offers a unique chance to experiment with different parameters like horizon length, time-steps, and cost functions to try to extract as much performance out of the robot arm as possible. Since we are space limited in the lab, we can only throw balls from about 3 meters away from the arm, providing a maximum 0.75 second time window for the arm to reach the strike point.
While our project looks at playing ping pong as the main objective, our setup can be easily extended to other similar tasks that require
strong "hand-eye coordination" with both fast reaction and movement speeds. For example, instead of detecting and hitting an object, the setup
can be modified to detecting and catching an object.
Additionally, by utilizing advanced control design, we look to make the Sawyer robot, which is not specifically built for speed,
able to perform tasks with very tight time constraints.
We demonstrate the power of optimal control design in pushing the limitations of hardware.
