What ChatGPT Passing an Ethical Hacking Exam Means for Cyber
A groundbreaking study led by US University of Missouri (Mizzou) and collaborators from Amrita University in India has revealed that AI chatbots like ChatGPT and Google's Gemini can successfully pass certified ethical hacking exams.
The study evaluated these leading Gen AI tools using a standard certified ethical hacking exam, which typically assesses a professional's understanding of various attack types, protection methods, and responses to security breaches.
Mizzou lead researcher Prasad Calyam and Professor of Cyber Security in Electrical Engineering and Computer Science, explained that both AI tools took multiple exam scenarios to assess their ability to respond to queries.
Both ChatGPT and Gemini demonstrated the ability to describe complex attack scenarios, such as man-in-the-middle attacks, and recommend preventive security measures.
This discovery has significant implications for the future of cybersecurity and the role of AI in protecting digital infrastructure.
Ethical hacking and AI
This development highlights the rapid advancement of AI capabilities in specialised fields like cybersecurity.
Ethical hacking, a critical component of modern cybersecurity practices, involves using hacker-like techniques to identify and remedy security vulnerabilities.
The fact that AI chatbots can now pass these exams suggests a potential shift in how cybersecurity assessments and training might be conducted in the future.
On one hand, it presents opportunities for enhanced red teaming – the practice of simulating cyberattacks to test an organisation's defences.
AI-powered tools could potentially conduct more frequent and comprehensive security checks, leading to the identification of vulnerabilities in systems that human testers might miss or be unaware of, making red teaming tests more refined when they occur.
However, the study also revealed important limitations of these AI tools.
While both chatbots passed the exam and provided comprehensible responses, when they occasionally gave incorrect answers. Yet, the study showed that when prompted to confirm their responses, both systems frequently revised their answers, rectifying prior errors.
Issues with Gen AI in cybersecurity
Yet, this instance of inaccuracy in cybersecurity setting can prove a critical concern in a field where mistakes can have severe consequences.
These AI hallucinations, where AI systems, particularly large language models (LLMs) generate false, inaccurate, or nonsensical information not grounded in reality, are a broader issue that industry are currently grappling with when it comes to Gen AI.
The study therefore used the unreliability of these to underscore the continued importance of human expertise in cybersecurity.
The results also raises concerns about the potential misuse of such technology. If AI can pass ethical hacking exams, it could theoretically be used by malicious actors to automate and scale up their attacks.
While the research demonstrates the potential for AI models to contribute significantly to ethical hacking, Prasad noted the accuracy of these AI tools would have to be enhance to meaningfully contribute to better cybersecurity practices.
While this study opens up new possibilities for enhancing digital defences and training, it also underscores the continued importance of human expertise and the developments that will need to take place outside of a cybersecurity context for them to be implemented in a safe way.
******
Make sure you check out the latest edition of Cyber Magazine and also sign up to our global conference series - Tech & AI LIVE 2024
******
Cyber Magazine is a BizClik brand
