1.Adherence of Studies on Large Language Models for Medical Applications Published in Leading Medical Journals According to the MI-CLEAR-LLM Checklist
Ji Su KO ; Hwon HEO ; Chong Hyun SUH ; Jeho YI ; Woo Hyun SHIM
Korean Journal of Radiology 2025;26(4):304-312
Objective:
To evaluate the adherence of large language model (LLM)-based healthcare research to the Minimum Reporting Items for Clear Evaluation of Accuracy Reports of Large Language Models in Healthcare (MI-CLEAR-LLM) checklist, a framework designed to enhance the transparency and reproducibility of studies on the accuracy of LLMs for medical applications.
Materials and Methods:
A systematic PubMed search was conducted to identify articles on LLM performance published in high-ranking clinical medicine journals (the top 10% in each of the 59 specialties according to the 2023 Journal Impact Factor) from November 30, 2022, through June 25, 2024. Data on the six MI-CLEAR-LLM checklist items: 1) identification and specification of the LLM used, 2) stochasticity handling, 3) prompt wording and syntax, 4) prompt structuring, 5) prompt testing and optimization, and 6) independence of the test data—were independently extracted by two reviewers, and adherence was calculated for each item.
Results:
Of 159 studies, 100% (159/159) reported the name of the LLM, 96.9% (154/159) reported the version, and 91.8% (146/159) reported the manufacturer. However, only 54.1% (86/159) reported the training data cutoff date, 6.3% (10/159) documented access to web-based information, and 50.9% (81/159) provided the date of the query attempts. Clear documentation regarding stochasticity management was provided in 15.1% (24/159) of the studies. Regarding prompt details, 49.1% (78/159) provided exact prompt wording and syntax but only 34.0% (54/159) documented prompt-structuring practices. While 46.5% (74/159) of the studies detailed prompt testing, only 15.7% (25/159) explained the rationale for specific word choices. Test data independence was reported for only 13.2% (21/159) of the studies, and 56.6% (43/76) provided URLs for internet-sourced test data.
Conclusion
Although basic LLM identification details were relatively well reported, other key aspects, including stochasticity, prompts, and test data, were frequently underreported. Enhancing adherence to the MI-CLEAR-LLM checklist will allow LLM research to achieve greater transparency and will foster more credible and reliable future studies.
2.Adherence of Studies on Large Language Models for Medical Applications Published in Leading Medical Journals According to the MI-CLEAR-LLM Checklist
Ji Su KO ; Hwon HEO ; Chong Hyun SUH ; Jeho YI ; Woo Hyun SHIM
Korean Journal of Radiology 2025;26(4):304-312
Objective:
To evaluate the adherence of large language model (LLM)-based healthcare research to the Minimum Reporting Items for Clear Evaluation of Accuracy Reports of Large Language Models in Healthcare (MI-CLEAR-LLM) checklist, a framework designed to enhance the transparency and reproducibility of studies on the accuracy of LLMs for medical applications.
Materials and Methods:
A systematic PubMed search was conducted to identify articles on LLM performance published in high-ranking clinical medicine journals (the top 10% in each of the 59 specialties according to the 2023 Journal Impact Factor) from November 30, 2022, through June 25, 2024. Data on the six MI-CLEAR-LLM checklist items: 1) identification and specification of the LLM used, 2) stochasticity handling, 3) prompt wording and syntax, 4) prompt structuring, 5) prompt testing and optimization, and 6) independence of the test data—were independently extracted by two reviewers, and adherence was calculated for each item.
Results:
Of 159 studies, 100% (159/159) reported the name of the LLM, 96.9% (154/159) reported the version, and 91.8% (146/159) reported the manufacturer. However, only 54.1% (86/159) reported the training data cutoff date, 6.3% (10/159) documented access to web-based information, and 50.9% (81/159) provided the date of the query attempts. Clear documentation regarding stochasticity management was provided in 15.1% (24/159) of the studies. Regarding prompt details, 49.1% (78/159) provided exact prompt wording and syntax but only 34.0% (54/159) documented prompt-structuring practices. While 46.5% (74/159) of the studies detailed prompt testing, only 15.7% (25/159) explained the rationale for specific word choices. Test data independence was reported for only 13.2% (21/159) of the studies, and 56.6% (43/76) provided URLs for internet-sourced test data.
Conclusion
Although basic LLM identification details were relatively well reported, other key aspects, including stochasticity, prompts, and test data, were frequently underreported. Enhancing adherence to the MI-CLEAR-LLM checklist will allow LLM research to achieve greater transparency and will foster more credible and reliable future studies.
3.Medication rules and mechanisms of treating chronic renal failure by Jinling medical school based on data mining, network pharmacology, and experimental validation.
Jin-Long WANG ; Wei WU ; Yi-Gang WAN ; Qi-Jun FANG ; Yu WANG ; Ya-Jing LI ; Fee-Lan CHONG ; Sen-Lin MU ; Chu-Bo HUANG ; Huang HUANG
China Journal of Chinese Materia Medica 2025;50(6):1637-1649
This study aims to explore the medication rules and mechanisms of treating chronic renal failure(CRF) by Jinling medical school based on data mining, network pharmacology, and experimental validation systematically and deeply. Firstly, the study selected the papers published by the inherited clinicians in Jinling medical school in Chinese journals using the subject headings named "traditional Chinese medicine(TCM) + chronic renal failure", "TCM + chronic renal inefficiency", or "TCM + consumptive disease" in China National Knowledge Infrastructure, Wanfang, and VIP Chinese Science and Technology Periodical Database and screened TCM formulas for treating CRF according to inclusion and exclusion criteria. The study analyzed the frequency of use of single TCM and the four properties, five tastes, channel tropism, and efficacy of TCM used with high frequency and performed association rule and clustering analysis, respectively. As a result, a total of 215 TCM formulas and 235 different single TCM were screened, respectively. The TCM used with high frequency included Astragali Radix, Rhei Radix et Rhizoma, Salviae Miltiorrhizae Radix et Rhizoma, Poria, and Atractylodis Macrocephalae Rhizoma(top 5). The single TCM characterized by "cold properties, sweet flavor, and restoring spleen channel" and the TCM with the efficacy of tonifying deficiency had the highest frequency of use, respectively. Then, the TCM with the rules of "blood-activating and stasis-removing" and "diuretic and dampness-penetrating" appeared. In addition, the core combination of TCM [(Hexin Formula, HXF)] included "Astragali Radix, Rhei Radix et Rhizoma, Poria, Salviae Miltiorrhizae Radix, and Angelicae Sinensis Radix". The network pharmacology analysis showed that HXF had 91 active compounds and 250 corresponding protein targets including prostaglandin-endoperoxide synthase 2(PTGS2), PTGS1, sodium voltage-gated channel alpha subunit 5(SCN5A), cholinergic receptor muscarinic 1(CHRM1), and heat shock protein 90 alpha family class A member 1(HSP90AA1)(top 5). Gene Ontology(GO) function analysis revealed that the core targets of HXF predominantly affected biological processes, cellular components, and molecular functions such as positive regulation of transcription by ribonucleic acid polymerase Ⅱ and DNA template transcription, formation of cytosol, nucleus, and plasma membrane, and identical protein binding and enzyme binding. Kyoto Encyclopedia of Genes and Genomes(KEGG) analysis revealed that CRF-related genes were involved in a variety of signaling pathways and cellular metabolic pathways, primarily involving "phosphatidylinositol 3-kinase(PI3K)-protein kinase B(Akt) pathway" and "advanced glycation end products-receptor for advanced glycation end products". Molecular docking results showed that the active components in HXF such as isomucronulatol 7-O-glucoside, betulinic acid, sitosterol, and przewaquinone B might be crucial in the treatment of CRF. Finally, a modified rat model with renal failure induced by adenine was used, and the in vivo experimental confirmation was performed based on the above-mentioned predictions. The results verify that HXF can regulate mitochondrial autophagy in the kidneys and the PI3K-Akt-mammalian target of rapamycin(mTOR) signaling pathway activation at upstream, so as to alleviate renal tubulointerstitial fibrosis and then delay the progression of CRF.
Data Mining
;
Drugs, Chinese Herbal/chemistry*
;
Network Pharmacology
;
Humans
;
Kidney Failure, Chronic/metabolism*
;
Medicine, Chinese Traditional
;
China
4.Effect of Chaihu Jia Longgu Muli Decoction on apoptosis in rats with heart failure after myocardial infarction through IκBα/NF-κB pathway.
Miao-Yu SONG ; Cui-Ling ZHU ; Yi-Zhuo LI ; Xing-Yuan LI ; Gang LIU ; Xiao-Hui LI ; Yan-Qin SUN ; Ming-Yuan DU ; Lei JIANG ; Chao-Chong YUE
China Journal of Chinese Materia Medica 2025;50(8):2184-2192
This study aims to explore the protective effect of Chaihu Jia Longgu Muli Decoction on rats with heart failure after myocardial infarction, and to clarify its possible mechanisms, providing a new basis for basic research on the mechanism of classic Chinese medicinal formula-mediated inflammatory response in preventing and treating heart failure induced by apoptosis after myocardial infarction. A heart failure model after myocardial infarction was established in rats by coronary artery ligation. The rats were divided into sham group, model group, and low, medium, and high-dose groups of Chaihu Jia Longgu Muli Decoction, with 10 rats in each group. The low-dose, medium-dose, and high-dose groups of Chaihu Jia Longgu Muli Decoction were given 6.3, 12.6, and 25.2 g·kg~(-1) doses by gavage, respectively. The sham group and model group were given an equal volume of distilled water by gavage once daily for four consecutive weeks. Cardiac function was assessed using color Doppler echocardiography. Myocardial pathology was detected by hematoxylin-eosin(HE) staining, apoptosis was measured by TUNEL assay, and mitophagy was observed by transmission electron microscopy. The levels of tumor necrosis factor-α(TNF-α), interleukin(IL)-1β, and N-terminal pro-B-type natriuretic peptide(NT-proBNP) in serum were detected by enzyme-linked immunosorbent assay(ELISA). The expression of apoptosis-related proteins B-cell lymphoma 2(Bcl-2), Bcl-2-associated X protein(Bax), and cleaved caspase-3 was detected by Western blot. Additionally, the expression of phosphorylated nuclear transcription factor-κB(NF-κB) p65(p-NF-κB p65)(upstream) and nuclear factor kappa B inhibitor alpha(IκBα)(downstream) in the NF-κB signaling pathway was assessed by Western blot. The results showed that compared with the sham group, left ventricular ejection fraction(LVEF) and left ventricular short axis shortening(LVFS) in the model group were significantly reduced, while left ventricular end diastolic diameter(LVEDD) and left ventricular end systolic diameter(LVESD) increased significantly. Myocardial tissue damage was severe, with widened intercellular spaces and disorganized cell arrangement. The apoptosis rate was increased, and mitochondria were enlarged with increased vacuoles. Levels of TNF-α, IL-1β, and NT-proBNP were elevated, indicating an obvious inflammatory response. The expression of pro-apoptotic factors Bax and cleaved caspase-3 increased, while the anti-apoptotic factor Bcl-2 decreased. The expression of p-NF-κB p65 was upregulated, and the expression of IκBα was downregulated. In contrast, the Chaihu Jia Longgu Muli Decoction groups showed significantly improved of LVEF, LVFS and decreased LVEDD, LVESD compared to the model group. Myocardial tissue damage was alleviated, and intercellular spaces were reduced. The apoptosis rate decreased, mitochondrial volume decreased, and the levels of TNF-α, IL-1β, and NT-proBNP were lower. The expression of pro-apoptotic factors Bax and cleaved caspase-3 decreased, while the expression of the anti-apoptotic factor Bcl-2 increased. Additionally, the expression of p-NF-κB p65 decreased, while IκBα expression increased. In summary, this experimental study shows that Chaihu Jia Longgu Muli Decoction can reduce the inflammatory response and apoptosis rate in rats with heart failure after myocardial infarction, which may be related to the regulation of the IκBα/NF-κB signaling pathway.
Animals
;
Apoptosis/drug effects*
;
Drugs, Chinese Herbal/administration & dosage*
;
Rats
;
Myocardial Infarction/physiopathology*
;
Male
;
NF-kappa B/genetics*
;
Heart Failure/etiology*
;
Rats, Sprague-Dawley
;
Signal Transduction/drug effects*
;
NF-KappaB Inhibitor alpha/genetics*
;
Humans
;
Tumor Necrosis Factor-alpha/genetics*
5.Adherence of Studies on Large Language Models for Medical Applications Published in Leading Medical Journals According to the MI-CLEAR-LLM Checklist
Ji Su KO ; Hwon HEO ; Chong Hyun SUH ; Jeho YI ; Woo Hyun SHIM
Korean Journal of Radiology 2025;26(4):304-312
Objective:
To evaluate the adherence of large language model (LLM)-based healthcare research to the Minimum Reporting Items for Clear Evaluation of Accuracy Reports of Large Language Models in Healthcare (MI-CLEAR-LLM) checklist, a framework designed to enhance the transparency and reproducibility of studies on the accuracy of LLMs for medical applications.
Materials and Methods:
A systematic PubMed search was conducted to identify articles on LLM performance published in high-ranking clinical medicine journals (the top 10% in each of the 59 specialties according to the 2023 Journal Impact Factor) from November 30, 2022, through June 25, 2024. Data on the six MI-CLEAR-LLM checklist items: 1) identification and specification of the LLM used, 2) stochasticity handling, 3) prompt wording and syntax, 4) prompt structuring, 5) prompt testing and optimization, and 6) independence of the test data—were independently extracted by two reviewers, and adherence was calculated for each item.
Results:
Of 159 studies, 100% (159/159) reported the name of the LLM, 96.9% (154/159) reported the version, and 91.8% (146/159) reported the manufacturer. However, only 54.1% (86/159) reported the training data cutoff date, 6.3% (10/159) documented access to web-based information, and 50.9% (81/159) provided the date of the query attempts. Clear documentation regarding stochasticity management was provided in 15.1% (24/159) of the studies. Regarding prompt details, 49.1% (78/159) provided exact prompt wording and syntax but only 34.0% (54/159) documented prompt-structuring practices. While 46.5% (74/159) of the studies detailed prompt testing, only 15.7% (25/159) explained the rationale for specific word choices. Test data independence was reported for only 13.2% (21/159) of the studies, and 56.6% (43/76) provided URLs for internet-sourced test data.
Conclusion
Although basic LLM identification details were relatively well reported, other key aspects, including stochasticity, prompts, and test data, were frequently underreported. Enhancing adherence to the MI-CLEAR-LLM checklist will allow LLM research to achieve greater transparency and will foster more credible and reliable future studies.
6.Intelligent handheld ultrasound improving the ability of non-expert general practitioners in carotid examinations for community populations: a prospective and parallel controlled trial
Pei SUN ; Hong HAN ; Yi-Kang SUN ; Xi WANG ; Xiao-Chuan LIU ; Bo-Yang ZHOU ; Li-Fan WANG ; Ya-Qin ZHANG ; Zhi-Gang PAN ; Bei-Jian HUANG ; Hui-Xiong XU ; Chong-Ke ZHAO
Ultrasonography 2025;44(2):112-123
Purpose:
The aim of this study was to investigate the feasibility of an intelligent handheld ultrasound (US) device for assisting non-expert general practitioners (GPs) in detecting carotid plaques (CPs) in community populations.
Methods:
This prospective parallel controlled trial recruited 111 consecutive community residents. All of them underwent examinations by non-expert GPs and specialist doctors using handheld US devices (setting A, setting B, and setting C). The results of setting C with specialist doctors were considered the gold standard. Carotid intima-media thickness (CIMT) and the features of CPs were measured and recorded. The diagnostic performance of GPs in distinguishing CPs was evaluated using a receiver operating characteristic curve. Inter-observer agreement was compared using the intragroup correlation coefficient (ICC). Questionnaires were completed to evaluate clinical benefits.
Results:
Among the 111 community residents, 80, 96, and 112 CPs were detected in settings A, B, and C, respectively. Setting B exhibited better diagnostic performance than setting A for detecting CPs (area under the curve, 0.856 vs. 0.749; P<0.01). Setting B had better consistency with setting C than setting A in CIMT measurement and the assessment of CPs (ICC, 0.731 to 0.923). Moreover, measurements in setting B required less time than the other two settings (44.59 seconds vs. 108.87 seconds vs. 126.13 seconds, both P<0.01).
Conclusion
Using an intelligent handheld US device, GPs can perform CP screening and achieve a diagnostic capability comparable to that of specialist doctors.
7.Adherence of Studies on Large Language Models for Medical Applications Published in Leading Medical Journals According to the MI-CLEAR-LLM Checklist
Ji Su KO ; Hwon HEO ; Chong Hyun SUH ; Jeho YI ; Woo Hyun SHIM
Korean Journal of Radiology 2025;26(4):304-312
Objective:
To evaluate the adherence of large language model (LLM)-based healthcare research to the Minimum Reporting Items for Clear Evaluation of Accuracy Reports of Large Language Models in Healthcare (MI-CLEAR-LLM) checklist, a framework designed to enhance the transparency and reproducibility of studies on the accuracy of LLMs for medical applications.
Materials and Methods:
A systematic PubMed search was conducted to identify articles on LLM performance published in high-ranking clinical medicine journals (the top 10% in each of the 59 specialties according to the 2023 Journal Impact Factor) from November 30, 2022, through June 25, 2024. Data on the six MI-CLEAR-LLM checklist items: 1) identification and specification of the LLM used, 2) stochasticity handling, 3) prompt wording and syntax, 4) prompt structuring, 5) prompt testing and optimization, and 6) independence of the test data—were independently extracted by two reviewers, and adherence was calculated for each item.
Results:
Of 159 studies, 100% (159/159) reported the name of the LLM, 96.9% (154/159) reported the version, and 91.8% (146/159) reported the manufacturer. However, only 54.1% (86/159) reported the training data cutoff date, 6.3% (10/159) documented access to web-based information, and 50.9% (81/159) provided the date of the query attempts. Clear documentation regarding stochasticity management was provided in 15.1% (24/159) of the studies. Regarding prompt details, 49.1% (78/159) provided exact prompt wording and syntax but only 34.0% (54/159) documented prompt-structuring practices. While 46.5% (74/159) of the studies detailed prompt testing, only 15.7% (25/159) explained the rationale for specific word choices. Test data independence was reported for only 13.2% (21/159) of the studies, and 56.6% (43/76) provided URLs for internet-sourced test data.
Conclusion
Although basic LLM identification details were relatively well reported, other key aspects, including stochasticity, prompts, and test data, were frequently underreported. Enhancing adherence to the MI-CLEAR-LLM checklist will allow LLM research to achieve greater transparency and will foster more credible and reliable future studies.
8.Intelligent handheld ultrasound improving the ability of non-expert general practitioners in carotid examinations for community populations: a prospective and parallel controlled trial
Pei SUN ; Hong HAN ; Yi-Kang SUN ; Xi WANG ; Xiao-Chuan LIU ; Bo-Yang ZHOU ; Li-Fan WANG ; Ya-Qin ZHANG ; Zhi-Gang PAN ; Bei-Jian HUANG ; Hui-Xiong XU ; Chong-Ke ZHAO
Ultrasonography 2025;44(2):112-123
Purpose:
The aim of this study was to investigate the feasibility of an intelligent handheld ultrasound (US) device for assisting non-expert general practitioners (GPs) in detecting carotid plaques (CPs) in community populations.
Methods:
This prospective parallel controlled trial recruited 111 consecutive community residents. All of them underwent examinations by non-expert GPs and specialist doctors using handheld US devices (setting A, setting B, and setting C). The results of setting C with specialist doctors were considered the gold standard. Carotid intima-media thickness (CIMT) and the features of CPs were measured and recorded. The diagnostic performance of GPs in distinguishing CPs was evaluated using a receiver operating characteristic curve. Inter-observer agreement was compared using the intragroup correlation coefficient (ICC). Questionnaires were completed to evaluate clinical benefits.
Results:
Among the 111 community residents, 80, 96, and 112 CPs were detected in settings A, B, and C, respectively. Setting B exhibited better diagnostic performance than setting A for detecting CPs (area under the curve, 0.856 vs. 0.749; P<0.01). Setting B had better consistency with setting C than setting A in CIMT measurement and the assessment of CPs (ICC, 0.731 to 0.923). Moreover, measurements in setting B required less time than the other two settings (44.59 seconds vs. 108.87 seconds vs. 126.13 seconds, both P<0.01).
Conclusion
Using an intelligent handheld US device, GPs can perform CP screening and achieve a diagnostic capability comparable to that of specialist doctors.
9.Adherence of Studies on Large Language Models for Medical Applications Published in Leading Medical Journals According to the MI-CLEAR-LLM Checklist
Ji Su KO ; Hwon HEO ; Chong Hyun SUH ; Jeho YI ; Woo Hyun SHIM
Korean Journal of Radiology 2025;26(4):304-312
Objective:
To evaluate the adherence of large language model (LLM)-based healthcare research to the Minimum Reporting Items for Clear Evaluation of Accuracy Reports of Large Language Models in Healthcare (MI-CLEAR-LLM) checklist, a framework designed to enhance the transparency and reproducibility of studies on the accuracy of LLMs for medical applications.
Materials and Methods:
A systematic PubMed search was conducted to identify articles on LLM performance published in high-ranking clinical medicine journals (the top 10% in each of the 59 specialties according to the 2023 Journal Impact Factor) from November 30, 2022, through June 25, 2024. Data on the six MI-CLEAR-LLM checklist items: 1) identification and specification of the LLM used, 2) stochasticity handling, 3) prompt wording and syntax, 4) prompt structuring, 5) prompt testing and optimization, and 6) independence of the test data—were independently extracted by two reviewers, and adherence was calculated for each item.
Results:
Of 159 studies, 100% (159/159) reported the name of the LLM, 96.9% (154/159) reported the version, and 91.8% (146/159) reported the manufacturer. However, only 54.1% (86/159) reported the training data cutoff date, 6.3% (10/159) documented access to web-based information, and 50.9% (81/159) provided the date of the query attempts. Clear documentation regarding stochasticity management was provided in 15.1% (24/159) of the studies. Regarding prompt details, 49.1% (78/159) provided exact prompt wording and syntax but only 34.0% (54/159) documented prompt-structuring practices. While 46.5% (74/159) of the studies detailed prompt testing, only 15.7% (25/159) explained the rationale for specific word choices. Test data independence was reported for only 13.2% (21/159) of the studies, and 56.6% (43/76) provided URLs for internet-sourced test data.
Conclusion
Although basic LLM identification details were relatively well reported, other key aspects, including stochasticity, prompts, and test data, were frequently underreported. Enhancing adherence to the MI-CLEAR-LLM checklist will allow LLM research to achieve greater transparency and will foster more credible and reliable future studies.
10.Intelligent handheld ultrasound improving the ability of non-expert general practitioners in carotid examinations for community populations: a prospective and parallel controlled trial
Pei SUN ; Hong HAN ; Yi-Kang SUN ; Xi WANG ; Xiao-Chuan LIU ; Bo-Yang ZHOU ; Li-Fan WANG ; Ya-Qin ZHANG ; Zhi-Gang PAN ; Bei-Jian HUANG ; Hui-Xiong XU ; Chong-Ke ZHAO
Ultrasonography 2025;44(2):112-123
Purpose:
The aim of this study was to investigate the feasibility of an intelligent handheld ultrasound (US) device for assisting non-expert general practitioners (GPs) in detecting carotid plaques (CPs) in community populations.
Methods:
This prospective parallel controlled trial recruited 111 consecutive community residents. All of them underwent examinations by non-expert GPs and specialist doctors using handheld US devices (setting A, setting B, and setting C). The results of setting C with specialist doctors were considered the gold standard. Carotid intima-media thickness (CIMT) and the features of CPs were measured and recorded. The diagnostic performance of GPs in distinguishing CPs was evaluated using a receiver operating characteristic curve. Inter-observer agreement was compared using the intragroup correlation coefficient (ICC). Questionnaires were completed to evaluate clinical benefits.
Results:
Among the 111 community residents, 80, 96, and 112 CPs were detected in settings A, B, and C, respectively. Setting B exhibited better diagnostic performance than setting A for detecting CPs (area under the curve, 0.856 vs. 0.749; P<0.01). Setting B had better consistency with setting C than setting A in CIMT measurement and the assessment of CPs (ICC, 0.731 to 0.923). Moreover, measurements in setting B required less time than the other two settings (44.59 seconds vs. 108.87 seconds vs. 126.13 seconds, both P<0.01).
Conclusion
Using an intelligent handheld US device, GPs can perform CP screening and achieve a diagnostic capability comparable to that of specialist doctors.

Result Analysis
Print
Save
E-mail