MIRNet: Integrating Constrained Graph-Based Reasoning with Pre-training for Diagnostic Medical Imaging

Automated interpretation of medical images demands robust modeling of complex visual-semantic relationships while addressing annotation scarcity, label imbalance, and clinical plausibility constraints. We introduce MIRNet (Medical Image Reasoner Network), a novel framework that integrates self-supervised pre-training with constrained graph-based reasoning. Tongue image diagnosis is a particularly challenging domain that requires fine-grained visual and semantic understanding. Our approach leverages self-supervised masked autoencoder (MAE) to learn transferable visual representations from unlabeled data; employs graph attention networks (GAT) to model label correlations through expert-defined structured graphs; enforces clinical priors via constraint-aware optimization using KL divergence and regularization losses; and mitigates imbalance using asymmetric loss (ASL) and boosting ensembles. To address annotation scarcity, we also introduce TongueAtlas-4K, a comprehensive expert-curated benchmark comprising 4,000 images annotated with 22 diagnostic labels--representing the largest public dataset in tongue analysis. Validation shows our method achieves state-of-the-art performance. While optimized for tongue diagnosis, the framework readily generalizes to broader diagnostic medical imaging tasks.

Biomimetic Evaluation of an Underwater Soft Hand Through Deep Learning-based 3D Pose Reconstruction

Soft robotic hand shows considerable promise for various grasping applications. However, the sensing and reconstruction of the robot pose will cause limitation during the design and fabrication. In this work, we present a novel 3D pose reconstruction approach to analyze the grasping motion of a bidirectional soft robotic hand using experiment videos. The images from top, front, back, left, right view were collected using an one-camera-multiple-mirror imaging device. The coordinate and orientation information of soft fingers are detected based on deep learning methods. Faster RCNN model is used to detect the position of fingertips, while U-Net model is applied to calculate the side boundary of the fingers. Based on the kinematics, the corresponding coordinate and orientation databases are established. The 3D pose reconstructed result presents a satisfactory performance and good accuracy. Using efficacy coefficient method, the finger contribution of the bending angle and distance between fingers of soft robot hand is analyzed by compared with that of human hand. The results show that the soft robot hand perform a human-like motion in both single-direction and bidirectional grasping.