Boundary-pair training cuts false refusals in safety study

Researchers describe safety training that aims to help a model distinguish harmful requests from benign ones about the same subject. The issue is framed as a practical challenge for deployed assistants, rather than a question of whether an entire topic should be allowed or blocked. For example, a civics tutor and a public-sector assistant might share a model, while only one needs to refuse requests for political manipulation; both can still be expected to answer factual election questions. The example illustrates why a broad topic label may not capture the difference between a harmful request and a safe one on that topic.

The researchers say they trained a model to refuse the harmful subset and assessed its behavior on both sides of that distinction. In the configuration with the lowest harmful-response rate, it also rejected 74% of prompts described as plainly safe. That result shows how a system can look safer when judged by harmful responses alone, even as it declines many benign requests. The reported 74% finding is separate from the false-refusal comparison that follows.

In that comparison, adding examples that pair harmful and benign requests at the boundary reduced the false-refusal rate from 32.94% to 4.16%. The report says genuine refusals barely changed, though it does not give a precise rate for that movement in the source text. The findings point to the composition of safety-training data as a factor in the balance between blocking harmful requests and answering safe ones. They also argue against treating a higher refusal rate, by itself, as proof of better safety.

For teams building assistants, the study makes evaluation on both sides of a safety boundary an important part of assessing a model. Builders working with a shared model across different settings may need to distinguish harmful requests from factual or otherwise benign requests on the same subject. The researchers’ example shows why a single topic-wide rule could be too broad for such deployments. Their central practical message is to measure both harmful responses and false refusals, rather than relying on only one side of the trade-off.