Isolation-First Safety Management for AI Agents
Project Description
Large Language Model-based agents are increasingly capable of autonomously interacting with external environments and completing complex tasks. However, their growing autonomy also introduces safety risks that may not be fully addressed by model-level safeguards or external monitoring alone. In this project, we will explore isolation-first safety management for AI agents, using system-level isolation to limit the impact of compromised or misbehaving agents. Students will investigate the design of isolation mechanisms, build a prototype framework, and evaluate its effectiveness and practical trade-offs in representative agent tasks.
Supervisor
SONG Yangqiu
Quota
5
Course type
UROP1000
UROP1100
UROP2100
UROP3100
UROP3200
UROP4100
Applicant's Roles
Working with PhD and MPhil student mentors to formulate the threat model, implement isolation and permission mechanisms, construct attack and defense scenarios, run experiments, analyze safety and utility trade-offs, and contribute to research papers.
Applicant's Learning Objectives
1. Gain hands-on experience in building tool-using LLM agents and agentruntimes in Python.
2. Learn practical security concepts including sandboxing, least privilege, capability-based access control, credential isolation, and audit logging.
3. Learn how to design reproducible evaluations for prompt injection, unauthorized tool use, cross-agent data leakage, and failure propagation.
4. Develop an understanding of the trade-offs between agent safety, task utility, system latency, and resource overhead.
Complexity of the project
Challenging