Read Time: 5 minutes
GenAI tools like ChatGPT are transforming how businesses operate, but they also introduce serious cyber risks. Attackers are using AI to create more convincing deepfakes, generate sophisticated malware, and bypass traditional security controls. At the same time, organizations deploying GenAI face risks of data leakage, model manipulation, and compliance violations. The Monetary Authority of Singapore (MAS) has identified four critical threat areas in their July 2024 information paper. This article outlines practical mitigation strategies for organizations navigating this evolving landscape.
Key Risk Areas and Mitigation
1. Deepfakes and GenAI-Enabled Phishing
Deepfakes leverage AI to create convincing fake audio, videos, images, and text that defeat biometric authentication and enable sophisticated social engineering. Fraudsters use synthetic faces to bypass facial recognition during customer onboarding and impersonate executives to manipulate employees into transferring funds or sharing sensitive information.
Mitigating measures:
- Implement liveness detection in facial recognition systems
- Conduct deepfake awareness campaigns and simulation exercises
- Enable multi-factor authentication for high-risk transactions and privileged roles
- Deploy endpoint-based deepfake detection tools on corporate devices
- Include deepfake attack scenarios in incident response plans
2. Malware Generation and Enhancement
GenAI tools like WormGPT and DarkBard enable even low-skill attackers to create sophisticated malware quickly and cheaply. Some malware use GenAI to implement polymorphism, constantly changing code to evade traditional signature-based detection, making conventional security defenses less effective.
Mitigating measures:
- Adopt a multi-layered cyber defense strategy
- Implement AI-powered malware detection for polymorphic threats
- Integrate threat intelligence into log monitoring for real-time anomaly detection
- Maintain basic cyber hygiene fundamentals
3. Data Leakage from GenAI Deployment
Organizations face data leaks when employees submit sensitive information to public GenAI tools or when attackers use prompt injection and jailbreak attacks to extract confidential data. Third-party and open-source GenAI models may also introduce vulnerabilities or backdoors that result in data exposure.
Mitigating measures:
- Establish clear GenAI usage policies and data classification guidelines
- Adopt security-by-design when developing in-house GenAI models
- Conduct due diligence and vulnerability assessments on third-party GenAI solutions
- Implement data loss prevention (DLP) tools and GenAI-specific firewalls
4. GenAI Model and Output Manipulation
Threat actors can manipulate GenAI models through data poisoning attacks, introducing malicious or inaccurate data during training or use. This can lead to biased outputs, false investment recommendations, inaccurate trading predictions, and compromised decision-making processes.
Mitigating measures:
- Establish proper GenAI model and data governance
- Implement strict access controls and maker-checker processes for training data
- Continuously monitor models for performance degradation or unexpected behavior
- Incorporate GenAI contingency measures into business continuity plans
- Participate in GenAI information sharing through industry forums
Building GenAI Resilience with Crowe
The growing sophistication of GenAI threats demands continuous vigilance and adaptation of security measures. Crowe helps organizations assess their GenAI risk exposure and implement layered defensive measures. Our cybersecurity services test technology defenses through vulnerability assessments and red team exercises, develop governance frameworks for safe AI adoption, train teams to recognize GenAI-enabled attacks, and provide rapid response capabilities when incidents occur.