There is an increasing demand for automated systems capable of accurately diagnosing paddy diseases, which would help lower pesticide usage and prevent yield loss. Yet, the absence of publicly available datasets with annotated disease labels has posed a challenge to the development and benchmarking of advanced deep learning models. To address this issue, we created and open-sourced the Paddy Doctor dataset, facilitating the development of reliable and effective paddy disease diagnosis systems.