Grasp intention recognition is a vital problem for controlling assistive robots to help the elderly and infirm people restore arm and hand function. This dataset contains gaze data and scene image data of healthy individuals and hemiplegic patients while performing different grasping tasks. It can be used for gaze-based grasp intention recognition studies.