BACKGROUND: The STAR (Scientific, Transparent, and Applicable Rankings) working group conducts regular evaluations of Chinese guidelines and consensus statements. This study gathered insights from STAR working group members using qualitative interviews. METHODS: From March to August 2023, members of the STAR specialist committees were interviewed using semi-structured interview outline. The interviewees were selected through purpose-based sampling. Subject analysis was employed to summarize the findings. RESULTS: We conducted interviews with 37 members from 36 committees and summarized the contents into four main themes and 16 specific topics. The value of STAR in enhancing the development and selection of high-quality guidelines in China was commonly mentioned. Challenges identified included the lack of resources and suboptimal organizational structures, collaboration, and evaluation efficiency. Suggestions for the STAR tool included developing extensions for different guideline types, adjusting certain items, and better covering guideline applicability. The promotion of STAR and the consideration of an international committee for global outreach were also highlighted. CONCLUSION STAR has exerted a substantial influence on the evaluation of Chinese guidelines, and the insights gained from interviews offer valuable directions for its further enhancement.
Humans ; China ; Qualitative Research ; Practice Guidelines as Topic ; Interviews as Topic

BACKGROUND: This study aimed to develop a comprehensive instrument for evaluating and ranking clinical practice guidelines, named Scientific, Transparent and Applicable Rankings tool (STAR), and test its reliability, validity, and usability. METHODS: This study set up a multidisciplinary working group including guideline methodologists, statisticians, journal editors, clinicians, and other experts. Scoping review, Delphi methods, and hierarchical analysis were used to develop the STAR tool. We evaluated the instrument's intrinsic and interrater reliability, content and criterion validity, and usability. RESULTS: STAR contained 39 items grouped into 11 domains. The mean intrinsic reliability of the domains, indicated by Cronbach's α coefficient, was 0.588 (95% confidence interval [CI]: 0.414, 0.762). Interrater reliability as assessed with Cohen's kappa coefficient was 0.774 (95% CI: 0.740, 0.807) for methodological evaluators and 0.618 (95% CI: 0.587, 0.648) for clinical evaluators. The overall content validity index was 0.905. Pearson's r correlation for criterion validity was 0.885 (95% CI: 0.804, 0.932). The mean usability score of the items was 4.6 and the median time spent to evaluate each guideline was 20 min. CONCLUSION The instrument performed well in terms of reliability, validity, and efficiency, and can be used for comprehensively evaluating and ranking guidelines.
Reproducibility of Results ; Surveys and Questionnaires ; Practice Guidelines as Topic ; Humans