The demand for intelligent automation in factories has been steadily increasing. While traditional robotic arms perform simple automated tasks, deep reinforcement learning enables them to execute more complex operations. However, deep reinforcement learning in the field of robotics often encounters challenging learning tasks, especially in three-dimensional and continuous environments where obtaining rewards becomes sparse. To address this issue, this article proposes the Hindsight Proximal Policy Optimization (HPPO) method for intelligent robotic control.