Unlocking Robotic Manipulator Potential:

Exploring Model-Free Reinforcement Learning for Adaptive Task Performance

Abstract

Robotic manipulators hold significant promise across various industrial applications, yet their adaptability to dynamic and evolving environments remains a challenge. Leveraging artificial intelligence, particularly reinforcement learning (RL), presents a compelling avenue to enhance the capabilities of these manipulators. In this study, we investigate the efficacy of model-free RL algorithms in training robotic manipulators for pushing tasks, crucial for applications like high-mix-low-volume manufacturing and household assistance. We explore five prominent model-free RL algorithms: Proximal Primal Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), Advantage actor critic (A2C), and Soft Actor Critic (SAC). These algorithms offer distinct approaches to training agents by iteratively interacting with their environments to maximize cumulative rewards. We study their mechanisms, focusing on policy optimization and value function estimation. Furthermore, we provide a detailed setup elucidating the components of RL, including agents, environments, observations, actions, and reward functions. We discuss the nuances between fully observed and partially observed environments, as well as discrete and continuous action spaces, crucial for training robotic manipulators. Utilizing Python libraries such as stable-baselines3 and panda-gym, we conduct experiments with a Franka Panda robot simulated in the pybullet physics environment.Ultimately, this study contributes to advancing the field of robotic manipulation through the integration of cutting-edge RL techniques.