Abstract
Artificial intelligence (AI), particularly large language models (LLMs) such as ChatGPT, has expanded across various fields, including medical education and professional applications. However, the extent to which AI is utilized in writing personal statements (PSs) for adult reconstruction fellowship applications remains unclear. This study aimed to analyze the prevalence of AI-generated text in PSs submitted to our institution before and after the release of ChatGPT.
We retrospectively reviewed PSs submitted to our institution's adult reconstruction fellowship from 2021 to 2025. The PSs were divided into two cohorts: Pre-PS (2021 to 2022) and Post-PS (2024 to 2025). The PSs from 2023 were excluded due to uncertainty in AI adoption. All PSs were analyzed using GPTZero, an AI detection software, to determine the proportion of AI-generated versus human-generated text. Descriptive statistics and comparative analyses were conducted.
A total of 421 PSs were analyzed. The Pre-PS cohort had an average GPTZero score of 99.5% (SD 1.9) human, 0.4% (SD 0.8) AI, and 0.1% (SD 1.8) mixed, while the Post-PS cohort had scores of 83.8% (SD 29.9) human, 15.1% (SD 28.9) AI, and 1.1% mixed (SD 5.2) (P < 0.001). The AI-generated text was significantly more prevalent in the Post-PS cohort compared to the Pre-PS cohort. Additionally, international medical graduates (IMGs) and applicants from non-U.S. residencies demonstrated a higher proportion of AI-generated text in their PSs compared to U.S. applicants (P < 0.001).
The use of AI in PS writing has increased significantly since the release of ChatGPT. Given the role of PSs in candidate selection, these findings highlight the need for transparency, standardized guidelines regarding AI-assisted writing, and re-evaluation of the importance placed on personal statements in candidate selection. Further research should expand to other subspecialties and institutions to assess the broader implications of AI in postgraduate medical education.