The major technology players found among GAFAM are struggling to comply with new European rules on artificial intelligence, especially regarding cyber security and discrimination. A recent study shows this several generative AI models have significant shortcomingsthat highlights the upcoming issues for these companies facing the implementation of the AI Act legislation.
Tech giants’ difficult compliance with AI law
An innovative tool, developed by Swiss startup LatticeFlow and supported by European officials, tested generative artificial intelligence models from major technology companies including Meta, OpenAI and Alibaba. These tests were conducted in accordance with the requirements of the European Union’s AI Law, a law which will be gradually implemented over the next two years.
Each model was given a score between 0 and 1, reflecting their compliance with the criteria of the AI Act. The results show that the AI models of the companies in question achieve on average above 0.75, but some significant gaps were identified, especially with regard to resilience to cyber attacks and off discriminative bias in responses generated by these models.
Issues related to cybersecurity and discriminatory bias
The report highlights the failures of certain models in terms of protection against cyber-attacks and discrimination. One of the main challenges for these AI systems is to respond fairly and safely to sensitive situations.
For example, the test showed that Meta’s “Llama 2 13B Chat” model scored 0.42 in the “prompt hijacking” category, which assesses the model’s ability to withstand attacks where a hacker could disguise a malicious request in a legitimate form. Another model, “8x7B Instruct” from French startup Mistral, scored 0.38 in the same category, illustrating cyber security flaws.
Numerous discriminatory biases in AI models
Tests conducted by LatticeFlow also highlighted biases in generative AI models, a recurring problem in the development of these technologies. OpenAI’s GPT-3.5 Turbo model received a discrimination score of 0.46, revealing that these systems often reproduce human biases in terms of gender, race or other sensitive criteria. Alibaba’s “Qwen1.5 72B Chat” model scored even lower with a score of 0.37 in the same category.
A future of compliance for AI
Although the overall results show progress in making AI models compatible with new European regulations, improvements are still needed. LatticeFlow experts believe these findings provide a roadmap for companies to adjust their models and prepare for the AI Act.
According to Petar Tsankov, co-founder of LatticeFlow, the gaps identified by their tests, although the EU has not yet determined all the compliance criteria, indicate where companies should focus their efforts. With a strong focus on compliance, AI model providers should be able to meet regulatory requirements in a timely manner.
What is the regulatory outlook for artificial intelligence in Europe?
The legal framework surrounding artificial intelligence is developing rapidly in Europe, and the AI Act represents a major step in the regulation of this technology. As experts meet before 2025 to define a code of conduct for generative AI toolstechnology companies must already adapt to the first legal requirements.
The test created by LatticeFlow, although not verified by the European Commission, is considered a “first step” towards translating laws into concrete technical requirements. This allows companies to gradually adapt to upcoming standards and avoid penalties that can reach up to 35 million euros or 7% of their global turnover.