AI Papers Reader

Personalized digests of latest AI research

View on GitHub

Personalized AI Safety: New Benchmark and Approach Enhance LLM Response

Seattle, WA – In a new study, researchers have unveiled a novel approach to enhancing the safety of large language models (LLMs) by personalizing their responses based on individual user vulnerabilities. Published on ArXiv, the study introduces a new benchmark called PENGUIN and a framework called RAISE, addressing critical gaps in existing safety evaluations that often overlook the diverse needs of users.

LLMs, like GPT-4 and LLaMA-3, typically generate similar responses to identical prompts, irrespective of the user’s emotional state or background. This “one-size-fits-all” approach can pose serious safety risks, particularly in high-stakes domains such as health counseling or financial advising. For instance, a seemingly harmless empathetic response could trigger a fatal action in a user with suicidal tendencies, as illustrated in the paper.

To tackle this challenge, researchers developed PENGUIN, a large-scale benchmark comprising 14,000 scenarios across seven sensitive domains. These scenarios include both context-rich and context-free variants, allowing for a controlled evaluation of personalized safety.

“PENGUIN allowed us to systematically evaluate how including user context significantly improves safety scores,” explained Yuchen Wu, a lead author from the University of Washington. “We found that personalized user information can improve safety scores by 43.2%.”

However, the study also revealed that not all user attributes contribute equally to safety enhancement. To address this, the team developed RAISE (Risk-Aware Information Selection Engine), a two-stage agent framework that strategically acquires user-specific background information.

RAISE operates in a training-free manner, meaning it doesn’t require retraining the underlying LLM. In the offline phase, it employs a sophisticated search algorithm to discover optimal acquisition paths for user attributes. Then, during online use, it dynamically retrieves the most relevant information with minimal user interaction.

For example, imagine a user asking an LLM for advice on managing financial stress. A standard LLM might provide generic tips on budgeting. RAISE, however, would first gather relevant information such as the user’s employment status, debt levels, and history of financial planning. This allows the LLM to generate a more tailored and safer response, such as suggesting specific debt consolidation strategies or connecting the user with local financial counseling resources.

RAISE significantly improves safety scores by up to 31.6% over vanilla LLMs while maintaining a low interaction cost of just 2.7 user queries on average.

“The key is selective information gathering,” said Kaijie Zhu from the University of California, Santa Barbara. “RAISE prioritizes the most informative user attributes, enabling personalized responses without overwhelming the user.”

The researchers emphasize that this work establishes a foundation for safety research that adapts to individual user contexts rather than assuming a universal harm standard. By integrating personalized safety mechanisms into LLMs, we can ensure AI systems are more responsible and user-aware, particularly in sensitive domains.