Grape Leaf Disease Identification Using Image Retrieval and Expert Referral of Doubtful Images

Image-based tools for plant disease identification are typically evaluated on images that resemble their training data, and they produce a diagnosis for every image regardless of whether it resembles anything they have seen before. This paper describes diagnose-or-defer, a content-based image retrieval method for grape leaves that either reports the disease indicated by the most similar labelled reference images or refers the image to an expert. The leaf and its lesions are segmented, and each image is represented by how its lesions differ in color from the healthy tissue of the same leaf, by leaf texture, and by lesion shape. The ten most similar reference images are retrieved, and the method defers when too few of them agree or when the query lies farther from its nearest reference image than 95% of calibration images do. In acquisition-blocked cross-validation on 4,046 Plant-Village images covering black rot, esca, leaf blight, and healthy leaves, accuracy without deferral was 89.0%. With deferral, 60.6% of images received a diagnosis, 97.9% of those diagnoses were correct, and 1.3% of all images received a wrong diagnosis, compared with 11.0% without deferral. On 133 internet images from Plant-Doc, retrieval alone was wrong for 64%; the method referred 96% of them and gave a wrong diagnosis for 1.2%. The method is suited to photographs taken under conditions similar to its reference images, and it recognizes most photographs that are not. Lesion color is measured relative to the healthy tissue of the same leaf, and every diagnosis is presented together with the reference images on which it rests. A two-part rule, calibrated on held-out images, defers to an expert when the retrieved images disagree or when the photograph is unfamiliar. Validation employed acquisition-blocked splits, a separate internet test set, and controlled image changes, and all parameters and code are openly available.

This work presents a reliable approach to grape leaf disease identification by integrating image retrieval with an expert-referral mechanism. Unlike conventional classifiers that generate predictions for every image, the proposed diagnose-or-defer method recognizes uncertain and unfamiliar photographs, reducing the risk of incorrect diagnoses. Its use of lesion color relative to healthy leaf tissue, texture, and shape improves image representation, while calibrated referral criteria enhance diagnostic reliability. This approach offers potential for safer plant disease screening and more trustworthy agricultural decision-support systems.

This study introduces a diagnose-or-defer image retrieval method for identifying four grape leaf conditions: black rot, esca, leaf blight, and healthy leaves. A 56-value descriptor combines relative lesion color, leaf texture, and lesion shape to retrieve ten similar reference images. Calibrated agreement and familiarity checks determine whether to provide a diagnosis or refer the image to an expert. Testing on 4,046 PlantVillage images achieved 97.9% accuracy among diagnosed cases, while wrong diagnoses decreased to 1.3% of all images.