Evaluation of the degree of agreement in the diagnosis of diabetic retinopathy between ophthalmologists and EyeArt®

Abstract

Objective or purpose: To evaluate the diagnostic performance and agreement of the EyeArt (R) Artificial Intelligence (AI) system for detecting Diabetic Retinopathy (DR), comparing its results with ophthalmologists' assessments in a regional screening program.DesignCross-sectional observational study. Subjects, participants, and/or controlsA total of 498 diabetic patients aged 18 years or older were enrolled between June and September 2023 through the Retisalud DR screening program in the Canary Islands. No separate control group was included. Methods: All participants underwent non-mydriatic fundus photography using the TRC-NW400 camera. Retinal images were analyzed by the EyeArt (R) AI system (version 2.1.0), and results were compared with assessments by ophthalmologists based on the International Clinical Diabetic Retinopathy scale (ICDR). Agreement was quantified using Cohen's kappa coefficient. Additionally, mixed-effects logistic regression was used to explore associations between DR and clinical risk factors. Main outcome measures: Sensitivity, specificity, and agreement (Cohen's kappa) of the AI system compared to clinical diagnosis; predictors of DR such as age, diabetes duration, presence of Diabetic Macular Edema (DME), and central retinal thickness (CRT-OCT).ResultsThe EyeArt (R) system achieved a binocular sensitivity of 100% (95% CI: 98.1-100) and a specificity of 93.5% (95% CI: 90.2-96.0). Agreement with ophthalmologist grading was excellent, with kappa values of 0.966 (right eye) and 0.978 (left eye). Younger age, longer diabetes duration, DME presence, and higher CRT were significantly associated with DR diagnosis. Conclusions: The EyeArt (R) AI system showed excellent diagnostic accuracy and strong agreement with clinical evaluations in DR screening. Nonetheless, its tendency to overestimate DR severity indicates the need for further refinement of its grading algorithm. These findings support the potential integration of AI systems into large-scale DR screening programs, pending further validation.