Osteoarthritis (OA) is a prevalent degenerative joint disease,particularly affecting the knees. Early and accurate detection of OA and its severity, often graded using the Kellgren-Lawrence (KL) scale, is crucial for timely intervention and management. This study explores the application of deep learning techniques to automatically detect OA and assign KL grades from knee X-ray images. We propose a novel deep learning architecture that effectively extracts relevant features from X-ray images and classifies them into different KL grades.