The 2023 Defcon hacker conference held in Las Vegas marked a significant turning point in the world of AI technology. At this event, prominent AI tech companies partnered with algorithmic integrity and transparency groups to engage thousands of attendees in a red-teaming exercise aimed at identifying weaknesses in generative AI platforms. With support from the US government, this exercise sought to bring greater transparency to these increasingly influential yet opaque systems.
Subsequently, the ethical AI and algorithmic assessment nonprofit, Humane Intelligence, took this initiative a step further. In a recent announcement, the group introduced a call for participation in collaboration with the US National Institute of Standards and Technology (NIST). This initiative invited US residents to partake in a qualifying round of a nationwide red-teaming effort focused on evaluating AI office productivity software.
The qualifying round, set to take place online, is open to developers and members of the general public. Successful participants will advance to an in-person red-teaming event at the Conference on Applied Machine Learning in Information Security (CAMLIS) later in October. The primary objective is to enhance capabilities for conducting rigorous testing of the security, resilience, and ethics of generative AI technologies.
The Chief of Staff at Humane Intelligence, Theo Skeadas, highlighted the importance of democratizing the evaluation process. He emphasized that the average person using these models lacks the necessary expertise to determine their fitness for purpose. By involving a broader range of individuals in the evaluation process, the initiative aims to empower users to assess whether AI models meet their specific needs.
During the final event at CAMLIS, participants will be divided into a red team responsible for attacking AI systems and a blue team focused on defense. The AI 600-1 profile, part of NIST’s AI risk management framework, will serve as a rubric for evaluating the red team’s performance. This structured approach leverages user feedback to assess the real-world applications of AI models and ensures a rigorous scientific evaluation of generative AI technologies.
Rumman Chowdhury, founder of Humane Intelligence and a contractor at NIST’s Office of Emerging Technologies, underscored the significance of partnerships in enhancing accountability and transparency in AI systems. The collaboration with NIST is just the beginning of a series of red team collaborations that Humane Intelligence plans to announce in the coming weeks. These collaborations will extend to US government agencies, international governments, and NGOs, fostering a culture of transparency and accountability in the development of AI algorithms.
Skeadas emphasized the importance of broadening the community involved in testing and evaluating AI systems. Beyond programmers, policymakers, journalists, civil society members, and non-technical individuals should all play a role in scrutinizing these systems. By engaging a diverse set of stakeholders, the initiative aims to ensure thorough evaluation and accountability in the deployment of AI technologies.
Leave a Reply