"ChatGPT vs. Student: A Dataset for Source Classification of Computer Science Answers

A dataset comprising 500 data points was gathered by collecting answers to 250  computer science problems assigned in classes and quizzes from students. To generate this dataset, a response was selected from a random student for each question. The same questions were then asked to ChatGPT 3.0, and the answers were recorded. Based on the source of the response (either student or GPT), the dataset was labeled accordingly. The resulting labeled dataset includes the list of assignment and quiz questions, along with the corresponding answers from students and ChatGPT.

Half of the assignment questions were essay prompts that required students to compose an essay-style answer. GPT was also tasked with answering these questions, with a constraint to limit the length of the responses to be similar to that of the student answers. This was done to prevent the classifier from relying solely on the length of the response to distinguish between student and GPT-generated answers. The other half of the assignment consisted of programming prompts that required students and GPT to write programs in both C and Python. The difficulty of these programming prompts ranged from simple programs to more complex ones that involved multiple functions and classes.