Medical cross-modal retrieval aims to achieve semantic similarity search between different modalities of medical cases, such as quickly locating relevant ultrasound images through ultrasound reports, or using ultrasound images to retrieve matching reports. However, existing medical cross-modal hash retrieval methods face significant challenges, including semantic and visual differences between modalities and the scalability issues of hash algorithms in handling large-scale data. To address these challenges, this paper proposes a Medical image Semantic Alignment Cross-modal Hashing based on Transformer (MSACH). The algorithm employed a segmented training strategy, combining modality feature extraction and hash function learning, effectively extracting low-dimensional features containing important semantic information. A Transformer encoder was used for cross-modal semantic learning. By introducing manifold similarity constraints, balance constraints, and a linear classification network constraint, the algorithm enhanced the discriminability of the hash codes. Experimental results demonstrated that the MSACH algorithm improved the mean average precision (MAP) by 11.8% and 12.8% on two datasets compared to traditional methods. The algorithm exhibits outstanding performance in enhancing retrieval accuracy and handling large-scale medical data, showing promising potential for practical applications.
Algorithms ; Semantics ; Humans ; Ultrasonography ; Information Storage and Retrieval/methods* ; Image Processing, Computer-Assisted/methods*

Automatic classification of medical questions is of great significance in improving the quality and efficiency of online medical services, and belongs to the task of intent recognition. Joint entity recognition and intent recognition perform better than single task models. Currently, most publicly available medical text intent recognition datasets lack entity annotation, and manual annotation of these entities requires a lot of time and manpower. To solve this problem, this paper proposes a medical text classification model, bidirectional encoder representation based on transformer-recurrent convolutional neural network-entity-label-semantics (BRELS), which integrates medical entity label semantics. This model firstly utilizes an adaptive fusion mechanism to absorb prior knowledge of medical entity labels, achieving local feature enhancement. Then in global feature extraction, a lightweight recurrent convolutional neural network (LRCNN) is used to suppress parameter growth while preserving the original semantics of the text. The ablation and comparison experiments are conducted on three public medical text intent recognition datasets to validate the performance of the model. The results show that F1 score reaches 87.34%, 81.71%, and 77.74% on each dataset, respectively. The results show that the BRELS model can effectively identify and understand medical terminology, thereby effectively identifying users' intentions, which can improve the quality and efficiency of online medical services.
Semantics ; Neural Networks, Computer ; Humans ; Natural Language Processing

Online hashing methods are receiving increasing attention in cross modal medical image retrieval research. However, existing online methods often lack the learning ability to maintain semantic correlation between new and existing data. To this end, we proposed online semantic similarity cross-modal hashing (OSCMH) learning framework to incrementally learn compact binary hash codes of medical stream data. Within it, a sparse representation of existing data based on online anchor datasets was designed to avoid semantic forgetting of the data and adaptively update hash codes, which effectively maintained semantic correlation between existing and arriving data and reduced information loss as well as improved training efficiency. Besides, an online discrete optimization method was proposed to solve the binary optimization problem of hash code by incrementally updating hash function and optimizing hash code on medical stream data. Compared with existing online or offline hashing methods, the proposed algorithm achieved average retrieval accuracy improvements of 12.5% and 14.3% on two datasets, respectively, effectively enhancing the retrieval efficiency in the field of medical images.
Semantics ; Humans ; Algorithms ; Information Storage and Retrieval/methods* ; Diagnostic Imaging ; Image Processing, Computer-Assisted/methods*

OBJECTIVES: To develop a heterogeneous graph prediction method based on the fusion of multi-layer semantics and topological information for addressing the challenges in drug-target interaction prediction, including insufficient modeling of high-order semantic dependencies, lack of adaptive fusion of semantic paths, and over-smoothing of node features. METHODS: A heterogeneous graph network with multiple types of entities such as drugs, proteins, side effects, and diseases was constructed, and graph embedding techniques were used to obtain low-dimensional feature representations. An adaptive metapath search module was introduced to automatically discover semantic path combinations for guiding the propagation of high-order semantic information. A semantic aggregation mechanism integrating multi-head attention was designed to automatically learn the importance of each semantic path based on contextual information and achieve differentiated aggregation and dynamic fusion among paths. A structure-aware gated graph convolutional module was then incorporated to regulate the feature propagation intensity for suppressing redundant information and redcuing over-smoothing. Finally, the potential interactions between drugs and targets were predicted through an inner product operation. RESULTS: Compared with existing drug-target interaction prediction methods, the proposed method achieved an average improvement of 3.4% and 2.4%, 3.0% and 3.8% in terms of the area under the receiver operating characteristic curve (AUC) and the area under the precision-recall curve (AUPRC) on public datasets, respectively. CONCLUSIONS The drug-target interaction prediction method developed in this study can effectively extract complex high-order semantic and topological information from heterogeneous biological networks, thereby improving the accuracy and stability of drug-target interaction prediction. This method provides technical support and theoretical foundation for precise drug target discovery and targeted treatment of complex diseases.
Semantics ; Humans ; Drug Interactions ; Neural Networks, Computer ; Algorithms

In order to address the issues of spatial induction bias and lack of effective representation of global contextual information in colon polyp image segmentation, which lead to the loss of edge details and mis-segmentation of lesion areas, a colon polyp segmentation method that combines Transformer and cross-level phase-awareness is proposed. The method started from the perspective of global feature transformation, and used a hierarchical Transformer encoder to extract semantic information and spatial details of lesion areas layer by layer. Secondly, a phase-aware fusion module (PAFM) was designed to capture cross-level interaction information and effectively aggregate multi-scale contextual information. Thirdly, a position oriented functional module (POF) was designed to effectively integrate global and local feature information, fill in semantic gaps, and suppress background noise. Fourthly, a residual axis reverse attention module (RA-IA) was used to improve the network's ability to recognize edge pixels. The proposed method was experimentally tested on public datasets CVC-ClinicDB, Kvasir, CVC-ColonDB, and EITS, with Dice similarity coefficients of 94.04%, 92.04%, 80.78%, and 76.80%, respectively, and mean intersection over union of 89.31%, 86.81%, 73.55%, and 69.10%, respectively. The simulation experimental results show that the proposed method can effectively segment colon polyp images, providing a new window for the diagnosis of colon polyps.
Humans ; Colonic Polyps/diagnostic imaging* ; Computer Simulation ; Electric Power Supplies ; Semantics ; Image Processing, Computer-Assisted

The past twenty years have seen the increasingly important role of ontology in traditional Chinese medicine (TCM). However, the development of TCM ontology faces many challenges. Since the epistemologies dramatically differ between TCM and contemporary biomedicine, it is hard to apply the existing top-level ontology mechanically. "Data silos" are widely present in the currently available terminology standards, term sets, and ontologies. The formal representation of ontology needs to be further improved in TCM. Therefore, we propose a unified basic semantic framework of TCM based on in-depth theoretical research on the existing top-level ontology and a re-study of important concepts in TCM. Under such a framework, ontologies in TCM sub-domains should be built collaboratively and be represented formally in a common format. Besides, extensive cooperation should be encouraged by establishing ontology research communities to promote ontology peer review and reuse.
Medicine, Chinese Traditional ; Semantics ; Drugs, Chinese Herbal

Based on the top-level ontology and the existing ontology methodology, the related concepts of meridians and acupoints were discriminated, defined and classified; the relationship of core concepts were established, e.g. meridians, acupoints and zangfu. It was attempted to build an ontological semantic framework of meridians and acupoints. Through the investigation on the classification mode of the top-level ontology, it is proposed that the meridians and acupoints, as the unique concepts of traditional Chinese medicine, exist in the form of "emptiness" and belong to "immaterial entity". Meridians refer to the three-dimensional channels in the human body, and acupoints are divided into ontological acupoints and body surface ones. Ontological acupoints are regarded as a three-dimensional structure within the human body, whereas, body surface ones are the optimal sites for acupuncture needle insertion on the body surface, meaning, the zero-dimensional point on the body surface. The main relationships between meridians and acupoints include is-a, exterior-interior, located-in, correspondent-to, mapping, etc. The exploration of the semantic framework of meridians and acupoints is conductive to understanding the connotation of meridians, acupoints and their relationship.
Acupuncture ; Acupuncture Points ; Acupuncture Therapy/methods* ; Humans ; Meridians ; Semantics

Glioma is a primary brain tumor with high incidence rate. High-grade gliomas (HGG) are those with the highest degree of malignancy and the lowest degree of survival. Surgical resection and postoperative adjuvant chemoradiotherapy are often used in clinical treatment, so accurate segmentation of tumor-related areas is of great significance for the treatment of patients. In order to improve the segmentation accuracy of HGG, this paper proposes a multi-modal glioma semantic segmentation network with multi-scale feature extraction and multi-attention fusion mechanism. The main contributions are, (1) Multi-scale residual structures were used to extract features from multi-modal gliomas magnetic resonance imaging (MRI); (2) Two types of attention modules were used for features aggregating in channel and spatial; (3) In order to improve the segmentation performance of the whole network, the branch classifier was constructed using ensemble learning strategy to adjust and correct the classification results of the backbone classifier. The experimental results showed that the Dice coefficient values of the proposed segmentation method in this article were 0.909 7, 0.877 3 and 0.839 6 for whole tumor, tumor core and enhanced tumor respectively, and the segmentation results had good boundary continuity in the three-dimensional direction. Therefore, the proposed semantic segmentation network has good segmentation performance for high-grade gliomas lesions.
Attention ; Glioma/diagnostic imaging* ; Humans ; Magnetic Resonance Imaging/methods* ; Semantics

Ultrasonic examination is a common method in thyroid examination, and the results are mainly composed of thyroid ultrasound images and text reports. Implementation of cross modal retrieval method of images and text reports can provide great convenience for doctors and patients, but currently there is no retrieval method to correlate thyroid ultrasound images with text reports. This paper proposes a cross-modal method based on the deep learning and improved cross-modal generative adversarial network: ①the weight sharing constraints between the fully connection layers used to construct the public representation space in the original network are changed to cosine similarity constraints, so that the network can better learn the common representation of different modal data; ②the fully connection layer is added before the cross-modal discriminator to merge the full connection layer of image and text in the original network with weight sharing. Semantic regularization is realized on the basis of inheriting the advantages of the original network weight sharing. The experimental results show that the mean average precision of cross modal retrieval method for thyroid ultrasound image and text report in this paper can reach 0.508, which is significantly higher than the traditional cross-modal method, providing a new method for cross-modal retrieval of thyroid ultrasound image and text report.
Humans ; Image Processing, Computer-Assisted ; Semantics ; Thyroid Gland

BACKGROUND AND PURPOSE: There are three distinct subtypes of primary progressive aphasia (PPA): the nonfluent/agrammatic variant (nfvPPA), the semantic variant (svPPA), and the logopenic variant (lvPPA). We sought to characterize the pattern of [¹⁸F]-THK5351 retention across all three subtypes and determine the topography of [¹⁸F]-THK5351 retention correlated with each neurolinguistic score. METHODS: We enrolled 50 participants, comprising 13 PPA patients (3 nfvPPA, 5 svPPA, and 5 lvPPA) and 37 subjects with normal cognition (NC) who underwent 3.0-tesla magnetic resonance imaging, [¹⁸F]-THK5351 positron-emission tomography scans, and detailed neuropsychological tests. The PPA patients additionally participated in extensive neurolinguistic tests. Voxel-wise and region-of-interest-based analyses were performed to analyze [¹⁸F]-THK5351 retention. RESULTS: The nfvPPA patients exhibited higher [¹⁸F]-THK5351 retention in the the left inferior frontal and precentral gyri. In svPPA patients, [¹⁸F]-THK5351 retention was elevated in the anteroinferior and lateral temporal cortices compared to the NC group (left>right). The lvPPA patients exhibited predominant [¹⁸F]-THK5351 retention in the inferior parietal, lateral temporal, and dorsolateral prefrontal cortices, and the precuneus (left>right). [¹⁸F]-THK5351 retention in the left inferior frontal area was associated with lower fluency scores. Comprehension was correlated with [¹⁸F]-THK5351 retention in the left temporal cortices. Repetition was associated with [¹⁸F]-THK5351 retention in the left inferior parietal and posterior temporal areas, while naming difficulty was correlated with retention in the left fusiform and temporal cortices. CONCLUSIONS: The pattern of [¹⁸F]-THK5351 retention was well matched with clinical and radiological findings for each PPA subtype, in agreement with the anatomical and functional location of each language domain.
Aphasia, Primary Progressive ; Cognition ; Comprehension ; Humans ; Magnetic Resonance Imaging ; Neurofibrillary Tangles ; Neuropsychological Tests ; Parietal Lobe ; Positron-Emission Tomography ; Prefrontal Cortex ; Rabeprazole ; Semantics ; Temporal Lobe