The date fruit dataset was created to address the requirements of many applications in the pre-harvesting and harvesting stages. The two most important applications are automatic harvesting and visual yield estimation. The dataset is divided into two subsets and each of them is oriented into one of these two applications. The first dataset consists of 8079 images of more than 350 date bunches captured from 29 date palms. The date bunches belong to five date types: Naboot Saif, Khalas, Barhi, Meneifi, and Sullaj.