Diabetic retinopathy (DR) is one of the leading causes of blindness in a significant portion of the working population, and its damage on vision is irreversible. Therefore, rapid diagnosis on DR is crucial for saving the patient’s eyesight. Since Transformer shows superior performance in the field of computer vision compared with Convolutional Neural Networks (CNNs), it has been proposed and applied in computer aided diagnosis of DR. However, a large number of images should be used for training due to the lack of inductive bias in Transformers. It has been demonstrated that the retinal vessels follow self-similar fractal scaling law, and the fractal dimension of DR patients shows an evident difference from that of normal people. Based on this, the fractal dimension is introduced as a prior into Transformers to mitigate the adverse influence of lack of inductive bias on model performance. A new Transformer method pretrained with Masked Autoencoders and fractal dimension (MAEFD) is developed and proposed in this paper. The experiments on the APTOS dataset show that the classification performance for DR by the proposed MAEFD can be substantially improved. Additionally, the present model pretrained with 100,000 retinal images outperforms that pretrained with 1 million natural images in terms of DR classification performance.

糖尿病视网膜病变 (DR) 是导致很大一部分工作人群失明的主要原因之一，其对视力的损害是不可逆转的。因此，快速诊断DR对于挽救患者的视力至关重要。由于 Transformer 与卷积神经网络（CNN）相比在计算机视觉领域表现出优越的性能，因此被提出并应用于 DR 的计算机辅助诊断。然而，由于 Transformers 缺乏归纳偏差，需要使用大量图像进行训练。研究表明，视网膜血管遵循自相似分形尺度规律，DR患者的分形维数与正常人存在明显差异。基于此，将分形维数作为先验引入到 Transformers 中，以减轻缺乏归纳偏差对模型性能的不利影响。本文开发并提出了一种使用 Masked Autoencoders 和分形维数 (MAEFD) 进行预训练的新 Transformer 方法。 APTOS数据集上的实验表明，所提出的MAEFD对DR的分类性能可以得到显着提高。此外，就 DR 分类性能而言，使用 100,000 张视网膜图像预训练的当前模型优于使用 100 万张自然图像预训练的模型。