Objective:To develop an artificial intelligence (AI)-guided intravitreal injection robot system to accurately detect the injection point on the ocular surface and guide the robotic arm to complete the intravitreal injection positioning task through 3D position calculation.Methods:The Dikablis subset of the TEyeD dataset was used.Training set, testing set, and validation set were constructed by using equal interval sampling strategy.The system read the ocular surface color RGB image with an RGBD camera, then used a PatchCrop-Transformer-based injection point detection algorithm to detect and locate key points such as the pupil, iris, and eyelid in the image.Next, it extracted the local 3D point cloud data near the injection point based on the depth information obtained by the camera.Through principal component analysis (PCA) of the local area point cloud data, the injection point and injection direction were determined.The key information was then passed to the robotic arm system.The end of the robotic arm adopted a remote center of motion (RCM) mechanism.After solving the forward and inverse kinematics, the joint movement path was obtained, and the robotic arm was controlled to move to 2 cm above the injection point.After confirmation by the doctor, the insertion, injection, and withdrawal operations were completed to ensure the stability and repeatability of the injection process.The mean square error (MSE) of key points localization and the success detection rate (SDR) within different pixel error ranges (2, 5, and 10 pixels) of the study method were compared with those of the NFDP, SLPT, and StarLoss methods, and the effects of random weight enhancement, fixed weight enhancement, and no enhancement methods on the MSE of key points localization were evaluated.The repeatability and absolute positioning accuracy of the robotic arm system were also evaluated.Results:After adding random weight enhancement, the model of this study outperformed the fixed weight enhancement and no enhancement methods in both MSE and SDR.The MSEs of the model proposed in this study for overall eye, pupil, and iris localization were 4.25, 2.41, and 1.54, respectively, which were lower than those of the NFDP, StarLoss, and SLPT methods.Within the error ranges of 5 and 10 pixels, the SDRs of the model proposed in this study were 72.09% and 92.68%, respectively, which were higher than those of the NFDP, StarLoss, and SLPT methods.The single-axis repeatability errors and absolute positioning errors of the robotic arm were within ±5 μm.Conclusions:The AI-guided intravitreal injection robot system integrates RGBD images to achieve automatic recognition of the ocular injection point and high-precision motion control through RCM mechanism design and corresponding kinematic solution methods.

Objective:To develop an artificial intelligence (AI)-guided intravitreal injection robot system to accurately detect the injection point on the ocular surface and guide the robotic arm to complete the intravitreal injection positioning task through 3D position calculation.Methods:The Dikablis subset of the TEyeD dataset was used.Training set, testing set, and validation set were constructed by using equal interval sampling strategy.The system read the ocular surface color RGB image with an RGBD camera, then used a PatchCrop-Transformer-based injection point detection algorithm to detect and locate key points such as the pupil, iris, and eyelid in the image.Next, it extracted the local 3D point cloud data near the injection point based on the depth information obtained by the camera.Through principal component analysis (PCA) of the local area point cloud data, the injection point and injection direction were determined.The key information was then passed to the robotic arm system.The end of the robotic arm adopted a remote center of motion (RCM) mechanism.After solving the forward and inverse kinematics, the joint movement path was obtained, and the robotic arm was controlled to move to 2 cm above the injection point.After confirmation by the doctor, the insertion, injection, and withdrawal operations were completed to ensure the stability and repeatability of the injection process.The mean square error (MSE) of key points localization and the success detection rate (SDR) within different pixel error ranges (2, 5, and 10 pixels) of the study method were compared with those of the NFDP, SLPT, and StarLoss methods, and the effects of random weight enhancement, fixed weight enhancement, and no enhancement methods on the MSE of key points localization were evaluated.The repeatability and absolute positioning accuracy of the robotic arm system were also evaluated.Results:After adding random weight enhancement, the model of this study outperformed the fixed weight enhancement and no enhancement methods in both MSE and SDR.The MSEs of the model proposed in this study for overall eye, pupil, and iris localization were 4.25, 2.41, and 1.54, respectively, which were lower than those of the NFDP, StarLoss, and SLPT methods.Within the error ranges of 5 and 10 pixels, the SDRs of the model proposed in this study were 72.09% and 92.68%, respectively, which were higher than those of the NFDP, StarLoss, and SLPT methods.The single-axis repeatability errors and absolute positioning errors of the robotic arm were within ±5 μm.Conclusions:The AI-guided intravitreal injection robot system integrates RGBD images to achieve automatic recognition of the ocular injection point and high-precision motion control through RCM mechanism design and corresponding kinematic solution methods.