ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions

ThoughtTrace is a new dataset of real user-AI chats paired with users’ self-reported thoughts.

It includes 1,058 users, 2,155 conversations, 17,058 turns, and 10,174 thought annotations across 20 language models.

The data shows that user thoughts often differ from their chat messages, revealing latent goals and preferences.

Researchers say these thought annotations can improve user-behavior prediction and help train more personalized assistants.