Wesley H. Deng
Microsoft Research
Supporting Responsible AI in Industry Practice: Case studies on Generative AI Auditing and Red-Teaming
Abstract
There has been growing interest among both practitioners and researchers in engaging end users in AI auditing and red-teaming to draw upon their unique knowledge and lived experiences. However, we still know little about how to effectively scaffold end users in conducting audits that produce actionable insights for AI practitioners.
At the same time, automated red-teaming can complement human efforts by scaling evaluations and minimizing the need for people to be exposed to harmful content. However, existing automated approaches have not adequately considered the role of people’s identity.
In this talk, I present two systems—WeAudit and PersonaTeaming—that explore new ways to coordinate and enhance AI auditing and red-teaming efforts. Through these two case studies, I aim to demonstrate empirically grounded approaches to addressing on-the-ground needs for responsible AI in industry, and to highlight future directions for more effective and sustainable responsible AI and AI safety practices. To conclude, I will discuss my current work within the AI and Human Systems Group, a newly established team at Microsoft Research, and how this work builds on insights from my earlier research.
About
Wesley Hanwen Deng is a Senior Researcher at Microsoft Research New York City where he is a member of the Fairness, Accountability, Transparency, and Ethics (FATE) group and a close collaborator of the Sociotechnical Alignment Center (STAC).
Wesley received his Ph.D. from the Human-Computer Interaction Institute(opens in new tab) at Carnegie Mellon University(opens in new tab) (CMU). His research focuses on responsible AI (RAI), AI safety, and human-AI collaboration. His work aims to build human-centered tools to support the design, development, and governance of safer, more responsible AI systems. At Microsoft Research, he is currently focused on supporting on-the-ground AI agent evaluation and measurement across Microsoft’s AI products and services.
Wesley’s Ph.D. work has been published at top-tier venues in human-computer interaction and RAI, including CHI, CSCW, FAccT, HCOMP, and AIES, receiving four best paper awards (top 1% of all submissions) and one honorable mention (top 3% of all submissions). His work has also been recognized through the Microsoft AI & Society Fellowship and a K&L Gates Presidential Fellowship in Ethics and Computational Technologies. Prior to CMU, Wesley earned his undergraduate degree in Computer Science from UC Berkeley, graduating with highest honors and receiving an EECS Research Honor.