TY - CHAP A1 - Lee, Seanie A1 - Seong, Haebin A1 - Bok Lee, Dong A1 - Kang, Minki A1 - Chen, Xiaoyin A1 - Wagner, Dominik A1 - Bengio, Joshua A1 - Lee, Juho A1 - Hwang, Sung Ju T1 - HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models N2 - Safety guard models that detect malicious queries aimed at large language models(LLMs) are essential for ensuring the secure and responsible deployment of LLMs in real-world applications. However, deploying existing safety guard models with billions of parameters alongside LLMs on mobile devices is impractical due to substantial memory requirements and latency. To reduce this cost, we distill a large teacher safety guard model into a smaller one using a labeled dataset of instruction-response pairs with binary harmfulness labels. Due to the limited diversity of harmful instructions in the existing labeled dataset, naively distilled models tend to underperform compared to larger models. To bridge the gap between small and large models, we propose HarmAug, a simple yet effective data augmentation method that involves jailbreaking an LLM and prompting it to generate harmful instructions. Given a prompt such as, “Make a single harmful instruction prompt that would elicit offensive content”, we add an affirmative prefix (e.g., “I have an idea for a prompt:”) to the LLM’s response. This encourages the LLM to continue generating the rest of the response, leading to sampling harmful instructions. Another LLM generates a response to the harmful instruction, and the teacher model labels the instruction-response pair. We empirically show that our HarmAug outperforms other relevant baselines. Moreover, a 435-millionparameter safety guard model trained with HarmAug achieves an F1 score comparable to larger models with over 7 billion parameters, and even outperforms them in AUPRC, while operating at less than 25% of their computational cost. Our code, safety guard model, and synthetic dataset are publicly available. Y1 - 2025 U6 - https://doi.org/10.48550/arXiv.2410.01524 SP - 1 EP - 20 PB - ICLR ER - TY - JOUR A1 - Lee, Seanie A1 - Bok Lee, Dong A1 - Wagner, Dominik A1 - Kang, Minki A1 - Seong, Haebin A1 - Bocklet, Tobias A1 - Lee, Juho A1 - Hwang, Sung Ju T1 - SafeRoute: Adaptive Model Selection for Efficient and Accurate Safety Guardrails in Large Language Models, JF - Findings of the Association for Computational Linguistics: ACL 2025 N2 - Deploying large language models (LLMs) in real-world applications requires robust safety guard models to detect and block harmful user prompts. While large safety guard models achieve strong performance, their computational cost is substantial. To mitigate this, smaller distilled models are used, but they often underperform on “hard” examples where the larger model provides accurate predictions. We observe that many inputs can be reliablyhandled by the smaller model, while only a small fraction require the larger model’s capacity. Motivated by this, we propose SafeRoute, a binary router that distinguishes hard examples from easy ones. Our method selectively applies the larger safety guard model to the data that the router considers hard, improving efficiency while maintaining accuracy compared to solely using the larger safety guard model. Experimental results on multiple benchmark datasets demonstrate that our adaptive model selection significantly enhances the trade-off between computational cost and safety performance, outperforming relevant baselines. Y1 - 2025 UR - https://aclanthology.org/2025.findings-acl.105.pdf PB - Association for Computational Linguistics CY - Vienna, Austria ER -