• Computer Science > Computation and Language [Submitted on 23 Jan 2026] Title:Can LLMs Assess Personality? • Validating Conversational AI for Trait Profiling View PDF HTML (experimental)Abstract:This study validates Large Language Models (LLMs) as a dynamic alternative to questionnaire-based personality assessment. • Using a within-subjects experiment (N=33), we compared Big Five personality scores derived from guided LLM conversations against the gold-standard IPIP-50 questionnaire, while also measuring user-perceived accuracy. • Results indicate moderate convergent validity (r=0.38-0.58), with Conscientiousness, Openness, and Neuroticism scores statistically equivalent between methods. • Agreeableness and Extraversion showed significant differences, suggesting trait-specific calibration is needed. • Notably, participants rated LLM-generated profiles as equally accurate as traditional questionnaire results.
Article Summaries:
- A recent study tested whether large language models (LLMs) can reliably assess personality traits compared to traditional questionnaires. In a within‑subject experiment with 33 participants, researchers compared Big Five scores derived from guided LLM conversations to scores from the gold‑standard IPIP‑50 questionnaire. The results showed moderate convergent validity (r = 0.38‑0.58). Conscientiousness, Openness, and Neuroticism scores were statistically equivalent across methods, while Agreeableness and Extraversion differed, indicating a need for trait‑specific calibration. Participants rated the LLM‑generated profiles as equally accurate to questionnaire results, suggesting conversational AI could become a viable alternative for psychometric assessment.
Sources: