The EU AI Act is the world's first comprehensive legal framework for artificial intelligence. It introduces binding requirements for AI systems across risk tiers — and for companies building or deploying AI in the European market, understanding where your system sits in that taxonomy and what security obligations flow from it is no longer optional.
The Four Risk Tiers
Unacceptable Risk (Prohibited)
AI systems that pose clear threats to fundamental rights or safety are prohibited outright. This includes social scoring systems, real-time biometric surveillance in public spaces (with narrow law enforcement exceptions), and AI systems that exploit psychological vulnerabilities to manipulate behaviour.
High Risk
The most heavily regulated tier. High-risk AI systems are those used in: critical infrastructure, educational assessment, employment decisions, essential services (credit scoring, insurance), law enforcement, migration management, and administration of justice. AI systems that are safety components of products already subject to EU safety legislation (medical devices, vehicles, industrial machinery) are also classified as high risk.
If your AI product operates in any of these domains or is integrated into products in these domains, you are almost certainly in the high-risk tier.
Limited Risk
AI systems with specific transparency obligations — primarily chatbots and deepfake generators that must disclose their AI nature to users.
Minimal Risk
All other AI systems, including most consumer applications, AI-enabled spam filters, and AI in video games. No mandatory requirements, though voluntary compliance with codes of practice is encouraged.
Security Requirements for High-Risk AI Systems
Article 9 of the EU AI Act requires providers of high-risk AI systems to implement a risk management system that includes:
- Identification and analysis of known and foreseeable risks associated with the AI system.
- Estimation and evaluation of risks that may emerge when the system is used in accordance with its intended purpose and under conditions of reasonably foreseeable misuse.
- Evaluation of risks in light of data gathered from post-market monitoring.
- Adoption of appropriate risk management measures.
Article 15 requires high-risk AI systems to achieve appropriate levels of accuracy, robustness, and cybersecurity. Specifically:
- Systems must be resilient against attempts to alter their use or performance by third parties (adversarial attacks).
- Technical solutions must address AI-specific attacks including data poisoning, model poisoning, adversarial examples, and model extraction attacks.
- Systems must have appropriate backup plans and failover mechanisms.
What Security Testing Must Cover
For AI Act compliance, your security assessment needs to go beyond standard cybersecurity testing to address AI-specific attack vectors:
| Attack Category | AI Act Reference | Test Requirement |
|---|---|---|
| Adversarial input attacks | Article 15(3) | Test robustness of outputs to adversarially crafted inputs |
| Data poisoning | Article 15(3) | Assess training data integrity controls and provenance verification |
| Model extraction | Article 15(3) | Test resistance to membership inference and model inversion |
| Prompt manipulation (for LLM-based systems) | Article 9(2)(b) | Test for prompt injection and instruction override |
| Availability attacks | Article 15(1) | Test rate limiting and DoS resilience |
Technical Documentation Requirements
Article 11 requires providers to maintain technical documentation that includes, for security purposes:
- A description of the measures taken to ensure cybersecurity of the AI system.
- Results of testing and validation, including security testing.
- Description of known or foreseeable circumstances that may affect the system's security.
- Post-market monitoring plan including security incident reporting procedures.
General Purpose AI Models (GPAI)
The AI Act introduced specific obligations for providers of general-purpose AI models — foundation models used as components in other AI systems. GPAI providers with systemic risk (defined by training compute thresholds) must conduct adversarial testing and report serious incidents to the AI Office. If your product is built on a GPAI, the obligations cascade: you must ensure you are using the GPAI in accordance with the provider's terms and conduct your own risk assessment of how the GPAI behaves in your specific deployment context.
Preparing for Compliance
For AI companies operating in or entering the European market: classify your systems against the risk tiers now. High-risk designation triggers obligations that require significant documentation, testing, and conformity assessment work. Start with an AI security risk assessment that maps to OWASP LLM Top 10 and EU AI Act Article 15 requirements. Commission independent adversarial testing. Implement post-market monitoring. The AI Act's enforcement provisions include fines of up to €30 million or 6% of global annual turnover for violations — comparable to GDPR penalties.
Key Takeaways
- This post covers practical, actionable guidance for security and engineering teams.
- All findings and techniques are mapped to recognised frameworks (OWASP, NIST, ISO).
- Contact Vynox Security to test your systems against the vulnerabilities described here.