Or why the excuse of the “computer bug” no longer protects anyone
There was a time, not so long ago, when an executive could hide behind a convenient phrase whenever their systems behaved badly. “It’s the computer’s fault.” The sentence had something reassuring about it, almost childlike, turning the machine into an autonomous third party whose whims we merely suffered without bearing any responsibility. That era is coming to an end. When a recruitment algorithm silently excludes candidates of a certain age, when a chatbot promises a grieving customer a refund that does not exist, there is no longer anyone left to take the blame on the company’s behalf. The convenient third party has disappeared. What remains is the responsible party.
I have been observing this shift for several years, and it seems to me far deeper than a simple tightening of regulation. What is at stake is the end of a fiction: the idea that technology decides in our place and, by the same token, absolves us.
It would be a mistake to believe, as the frenzy around ChatGPT might suggest, that artificial intelligence was born with our teenagers’ smartphones. It was already occupying laboratories when our parents were learning to program, and the questions of ethics, governance and responsibility it raises are far from new. They were feeding academic conferences long before the general public took hold of them. What has changed is not the nature of the problem, but its scale. As long as a handful of researchers were handling these systems inside closed environments, the debate remained theoretical, almost confidential. From the moment hundreds of millions of people began questioning generative AI tools on a daily basis, the same question became a concrete legal, economic and social issue. Scale has shifted everything: what once belonged to the seminar room has become a matter of survival for many organizations.
This transformation is also redrawing the map of power. Europe legislates, methodically and at times fussily. The United States bets on innovation and intervenes only after the fact, once the damage is already done. China, for its part, tightly frames an innovation it wants to be both fast and obedient. Caught between these three irreconcilable logics, companies are trying to hold a steady course. And the question of who will answer for the missteps of their machines is not a minor legal detail: it will weigh heavily on the distribution of wealth in the decades to come.
Europe positions itself as the digital regulator
When the European Union adopted its Artificial Intelligence Act in 2024, published in the Official Journal on July 12 and entering into force on August 1 [1], many saw it as yet another layer of bureaucracy, another European reflex. That was to misunderstand its scope. For the first time, a major power was setting explicit legal limits on what automated systems are allowed to do. One may find the text heavy-handed, one may consider it premature, but it establishes a principle that had been missing: some things will not be decided by algorithm, whatever the machine’s performance.
The regulation does not merely prohibit or authorize in bulk. It grades. It classifies uses according to four levels of risk, from unacceptable to minimal, and calibrates obligations accordingly [1]. Practices deemed contrary to fundamental rights, such as the social scoring of citizens, manipulation through subliminal techniques, or certain forms of biometric surveillance in public spaces, fall into the prohibited category [1]. This hierarchy says something quite new: not all algorithms are born equal before the law.
For companies, this subtlety conceals a real constraint. Developing a system classified as “high risk” now means entering a compliance process whose burden recalls, in its weight, the approval of a medical device. Detailed documentation, data governance, human oversight, traceability. All of it backed by deterrent sanctions: up to 35 million euros, or 7% of the company’s worldwide annual turnover if that amount is higher, for the most serious infringements [1]. Enough to make even the most daring think twice.
And yet there is something slightly dizzying about this race. While Brussels lawyers were chiseling their articles, the technology they were trying to regulate changed face several times. This is the paradox of all technological regulation: it aims at a moving target and almost always arrives one step late. Institutions put up road signs when the road itself has already moved.
In France, the Commission nationale de l’informatique et des libertés, the CNIL, inherits this new role. Its president, Marie-Laure Denis, reappointed in early 2024 for a second term, has made artificial intelligence one of her stated priorities; the authority has also created a dedicated department and regularly publishes recommendations to help actors reconcile innovation and data protection [2]. The CNIL has also experimented with regulatory “sandboxes”, controlled spaces where a company can test a system under the regulator’s eye before deploying it widely. The idea is clever: rather than discovering problems once the harm has been done, they are anticipated within a controlled framework.
When machines go off the rails in public
The best lessons do not come from legal texts but from accidents. And in this respect, recent years have offered a collection of case studies as instructive as they are embarrassing for the companies involved.
Take the Air Canada case. A man who has just lost his grandmother consults the airline’s chatbot to find out whether he can benefit from a bereavement fare. The chatbot tells him that he can request a retroactive refund within ninety days of purchasing his ticket. The information is false: the company’s actual policy requires the discount to be granted before travel. When the customer asks for what he believes he is owed, Air Canada refuses, then, before the tribunal, attempts an astonishing defense: the chatbot is said to be a separate entity, responsible for its own statements, for which the company cannot be held liable. In February 2024, the Civil Resolution Tribunal of British Columbia swept the argument aside. In substance, it ruled that a chatbot is merely a component of the company’s website, and that the company remains responsible for all information it provides, whether it comes from a static page or an automated conversation [3]. The amount at stake was insignificant, a few hundred Canadian dollars, but the principle established was not. Your algorithms speak in your name. What they promise binds you.
The McDonald’s case tells another story, more burlesque, but just as revealing. Since 2021, the hamburger giant had been testing with IBM a voice-ordering system in more than a hundred of its American restaurants [4]. On paper, the idea was attractive: smooth the drive-thru experience, relieve teams. In reality, the system struggled to understand the diversity of accents and drowned in ambient noise [4]. Social networks feasted on videos where the machine imperturbably added items no customer had requested. In June 2024, McDonald’s ended the experiment [4]. The lesson is clear: a drive-thru at rush hour, with its engines, horns and overlapping orders, has nothing in common with the silent laboratory where the algorithm learned its manners. Between the two lies the full distance between the world as imagined and the world as it resists.
Other abuses are less amusing. In 2023, the tutoring group iTutorGroup agreed to pay $365,000 to settle an action brought by the American agency responsible for employment equality [5]. At issue was recruitment software programmed to automatically exclude female applicants aged 55 and over, and male applicants aged 60 and over, leading to the rejection of more than two hundred candidates [5]. It was the first settlement of this kind involving automated discrimination in hiring. What is striking here is not only the illegality, but the mechanism. A human recruiter modulates their decisions, reconsiders, takes individual cases into account. The algorithm applies the rule it has learned with metronomic regularity, without exception or qualm. It does not discriminate more than a bad-faith human would; it discriminates more systematically.
This same systematization appears in a frequently cited precedent, Amazon’s. Starting in 2014, the company had developed an experimental tool intended to rank résumés automatically. Trained on a decade of applications that were mostly male, the system had “learned” to prefer men, penalizing, for example, résumés mentioning certain terms associated with women. Revealed by Reuters in 2018, the project was abandoned [6]. Amazon clarified that the tool had never been used to actually evaluate candidates [6], an important nuance, but the episode remains emblematic: AI does not invent its prejudices, it inherits ours and reproduces them at scale.
The Apple Card case deserves attention precisely because it is more ambiguous. In late 2019, several users, including entrepreneur David Heinemeier Hansson and then Apple co-founder Steve Wozniak, publicly expressed outrage: the card, backed by Goldman Sachs, was said to have granted women credit limits far lower than those of their spouses, sometimes under comparable financial circumstances [7]. The financial regulator of the State of New York opened an investigation. Yet its report, published in 2021, found no violation of fair lending laws [7]. No illegal discrimination demonstrated, then. And yet the reputational damage had very much occurred. That is the whole paradox: once an algorithm decides opaquely, mere suspicion is enough to undermine trust, even when the accusation is not confirmed. Opacity alone is a risk.
Disinformation offers one final category of abuses. In April 2024, Grok, the chatbot integrated into the social network X, turned a joke by basketball fans into false news. Fans were mocking a player, Klay Thompson, for poor shooting by playing on the expression “shooting bricks.” Taking the image literally, Grok generated a story about vandalism, accusing the player of breaking windows with bricks [8]. Seemingly anecdotal, the episode says something serious about these systems’ ability to manufacture falsehood with the assurance of truth. A few months earlier, in November 2023, another scandal had shaken journalism: Sports Illustrated was accused of publishing articles signed by authors who did not exist, complete with fictional biographies and profile pictures generated by artificial intelligence. The content came from an external provider, and the publisher ended the collaboration once the matter was revealed [9]. Once again, trust took the blow, that fragile raw material without which neither a media outlet nor a brand is worth very much.
France is not immune to these debates. The administration’s use of profiling algorithms is raising growing concerns. The association La Quadrature du Net has documented, since 2024, France Travail’s use of scoring systems for job seekers, including a “suspicion score” intended to detect potential fraud and an “employability score”, mechanisms that have fueled criticism from institutions such as the Defender of Rights regarding the logic of control underlying them [10]. One can see here the particular sensitivity of the issue when public authority automates its decisions: these are no longer consumers being affected, but citizens facing their rights.
From this series of accidents emerges a somewhat uncomfortable truth. Our machines are not neutral. They amplify our biases, perpetuate our inequalities, and do so with a consistency that no human malice could match. It is precisely this mechanical regularity that makes their supervision so necessary.
Anatomy of algorithmic failure
After examining these failures, one eventually recognizes recurring patterns. Always the same flaws, or almost, returning from one fiasco to the next.
The first, and most insidious, lies in the illusion of the controlled environment. A system can behave perfectly in the calm of a laboratory and collapse on contact with reality. McDonald’s paid dearly for this: months of conclusive tests, then failure once the machine had to deal with the disorder of an actual drive-thru. Designing AI is not about building a device that works under ideal conditions, but creating one that withstands ordinary chaos. The nuance may seem slight; it makes all the difference.
Next comes the temptation of full automation. Intoxicated by efficiency, some companies remove humans from the loop and create autonomous systems without a safety valve. We saw this in McDonald’s orders spiraling out of control without any safeguard to stop them. This obsession with total automation has a fatal flaw: it removes the capacity for real-time correction, that distinctly human ability to sense that something is going wrong and take control again.
There is also the amplified reproduction of past biases. An algorithm does not merely reflect existing discrimination, it freezes it into a rule and applies it on an industrial scale. iTutorGroup offers the clearest illustration: a discriminatory preference that might have remained the behavior of individuals becomes a systematic policy applied flawlessly to hundreds of files.
Finally, the absence of a clear chain of responsibility poisons the governance of most organizations. In large structures, no one truly assumes the final algorithmic decision. This ambiguity creates a void that some companies even try to exploit, as Air Canada did when it attempted to assign blame to its own chatbot. As long as no executive agrees to personally answer for the choices made by their machines, abuses remain inevitable.
These four shortcomings are not independent. They feed one another: the illusion of control encourages total automation, which conceals inherited biases, which then prosper in the fog of responsibility. This mechanism cannot be broken by correcting a single link in the chain. It must be addressed as a whole, otherwise the problem is merely displaced.
Three powers, three ways of holding the reins
A broad-brush summary of the American, European and Chinese approaches would look roughly like this: the United States moves fast and repairs afterward, Europe thinks so much that it risks missing the train, China innovates within a framework firmly held by political power. The caricature has its limits, but it points to philosophies that, deep down, do not converge.
On the American side, the doctrine remains one of the primacy of innovation and trust in the market to eliminate irresponsible actors by itself. This orientation has recently intensified: the executive order on artificial intelligence issued under the Biden administration in 2023 was revoked in early 2025 by the Trump administration, which dismantled part of the inherited regulatory structure in the name of competitiveness [11]. The advantage is real: a sustained pace of innovation that attracts talent and capital. The downside is just as real: damage is discovered once it is already there, when it is often too late to repair it without breakage.
Europe, for its part, relies on the only weapon it still truly masters: the norm. Its wager is not only to protect its citizens, but to impose its standards on the rest of the world by seepage, hoping to reproduce what happened with the data protection regulation, which became a reference far beyond its borders. It is a strategy of influence through law. It has its strengths, robust protection and the ability to set rules that others eventually adopt. But it carries a risk that cannot be ignored: forcefully regulating a technology that one barely produces, and watching talent and investment flow toward more accommodating skies. I have already had occasion elsewhere to write how vulnerable this dependence makes us; I will not return to it here, but it hangs over this entire debate.
China, finally, follows a third path, often reduced, wrongly, to simple authoritarian control. The reality is more subtle. Companies there are encouraged to develop powerful systems, provided they respect the ideological orientations of the authorities; the rules adopted in 2023 on generative AI thus require produced content to remain consistent with the values promoted by the Party [12]. This framework enables rapid innovation massively supported by the state, within a rigid but predictable corset. Strategic coherence and considerable resources on one side, the risk of sclerosis and international distrust on the other.
For a company operating across several continents, these divergences create a genuinely complex puzzle. How can one design an AI system that complies at once with European transparency requirements, American flexibility and Chinese ideological imperatives? The squaring of the circle seems impossible, and each possible exit has its cost. Aligning all products with the strictest rule simplifies management but restrains innovation everywhere. Creating a version for each market optimizes locally but fragments and burdens development. Concentrating efforts where the rules are loosest accelerates the pace but closes markets and invites criticism. None of these paths is comfortable; one must simply choose which discomfort to live with.
What a company can do, concretely
Faced with this shifting landscape, the painful experience of early movers nevertheless outlines a few principles of conduct. Nothing miraculous, but landmarks that already separate, in practice, those who stand firm from those who stumble.
The first is to treat every AI deployment as a binding decision, deserving a serious impact assessment before launch, not after. Technical reliability, legal compliance, effects on people, social acceptability: better to examine these angles calmly than to discover them in a courtroom or on social networks. A single mistake, as we have seen, can durably damage a reputation patiently built.
The second principle, confirmed by all these failures, is to keep a human in the loop. The organizations that manage best are invariably those that have resisted the temptation to automate everything. Maintaining human oversight is costly in the short term and complicates operations, but it is precisely this redundancy that prevents disaster when the machine is wrong. This safeguard must still be designed to function under pressure, at the exact moment when it is needed.
Then comes documentation. The era of algorithmic tinkering, when one assembled a model in a corner without worrying too much about traceability, is nearing its end. Regulators now expect organizations to be able to account for training data, testing methods, performance and its limits, and responsibilities in the event of a problem. This rigor may seem tedious. Yet it is the best protection on the day explanations must be given.
Responsibility, finally, can no longer rest solely on the shoulders of technical teams. It requires a shared culture, where sales teams understand the possible biases of their recommendation tool, where human resources teams know how to detect discriminatory drift in recruitment software, where management fully assumes the technological choices it validates. Anticipating changes in rules rather than enduring them, participating in consultations and sector initiatives, is better than discovering tomorrow’s constraints in the Official Journal.
It should be added that these requirements are not only a matter of constraint. In France as elsewhere, initiatives have emerged to make responsible AI a field of collective action rather than a mere obligation. Associations such as Data for Good have for years mobilized volunteers around public-interest projects, proving that technical excellence and awareness of the issues are not mutually exclusive. Other business groups have chosen to anticipate regulation by imposing rules on themselves before others impose them. The calculation is not only moral: it is often more comfortable to write the rules of the game oneself than to inherit them from others.
A new outline of responsibility
We are witnessing, in real time, the formation of an unprecedented regime of responsibility. The comparison with the automobile seems illuminating to me. At the beginning of the twentieth century, the car transformed notions of insurance and civil liability by creating risks that the law of the time did not yet know how to name. Rules, solidarities and mechanisms of compensation had to be invented. Artificial intelligence places us before a comparable worksite, except that this time it touches decision-making itself, what had until now remained the reserved domain of human judgment.
This transformation is giving rise to new professions. The AI officer, the algorithm auditor, the compliance specialist are becoming sought-after functions as the issues they master grow in importance. Paradoxically, the very complexity of this framework could end up democratizing access to it. Rather than developing costly in-house expertise beyond the reach of most medium-sized structures, many are already turning to specialized providers in auditing, training or monitoring. This pooling enables modest actors to access know-how long reserved for giants.
Technology itself could, ironically, ease some of the difficulties that today seem insurmountable. Methods designed to make algorithmic decisions more explainable are progressing, automatic bias-detection tools are spreading, and generative AI could facilitate the production of the documentation that weighs so heavily on teams. Nothing is settled, but the trajectory is not solely one of endless hardening.
Let us not lull ourselves into illusion, however. This transition will cause damage. Some companies will pay for their past negligence, others will discover that their model cannot withstand the new constraints. This selection, harsh on a human level, will no doubt contribute to the emergence of a more mature and more durable ecosystem. It is cold comfort for those who will bear the cost.
The real question is no longer whether companies will have to answer for their systems. On that point, the debate is settled, and the courts have already made it clear. The question is how quickly, and with what lucidity, they prepare for it. Those that manage to turn this constraint into an asset will gain a head start. The others will learn, at their own expense, that in the economy now taking shape, technological irresponsibility is paid for in cash: in money, in reputation, in market share, sometimes in more than that.
Because in the end, when our machines make mistakes, someone must assume the consequences. That someone, whether we wanted it or not, is us. With our decisions, our budgets, and the trust we grant or refuse to these tools. The question I would willingly ask in closing is therefore not whether regulation goes too far or not far enough. It is more concrete, and it is addressed to anyone deploying AI in their organization today: would you be ready, tomorrow if necessary, to explain and defend every decision your machine makes? Those who can answer yes have little to fear. The others would do well to get started.
References
As always, the sources that informed this text are gathered here, for anyone who wishes to verify, compare and form their own view. At a time when AI-generated content is proliferating, the simple habit of tracing claims back to facts has never been more valuable.
[1] Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonized rules on artificial intelligence. Published in the Official Journal of the European Union on July 12, 2024, entered into force on August 1, 2024. Four levels of risk; fines of up to 35 million euros or 7% of annual worldwide turnover for prohibited practices, whichever is higher, under Article 99. Text and implementation timeline: https://eur-lex.europa.eu/eli/reg/2024/1689/oj and https://artificialintelligenceact.eu/article/99/
[2] CNIL, “Marie-Laure Denis est reconduite dans ses fonctions de présidente de la CNIL,” January 31, 2024, and the CNIL’s work on artificial intelligence. https://www.cnil.fr/fr/marie-laure-denis-est-reconduite-dans-ses-fonctions-de-presidente-de-la-cnil and https://www.cnil.fr/fr/intelligence-artificielle
[3] Moffatt v. Air Canada, 2024 BCCRT 149, Civil Resolution Tribunal of British Columbia, decision of February 14, 2024. On the company’s responsibility for information provided by its chatbot: Le Monde du Droit, February 20, 2024. https://www.lemondedudroit.fr/decryptages/91527-air-canada-est-responsable-des-erreurs-de-son-chatbot.html
[4] “McDonald’s to end AI drive-thru test with IBM,” CNBC, June 17, 2024, partnership launched in 2021, test in more than 100 restaurants, ended in July 2024, difficulties linked to accents and noise. https://www.cnbc.com/2024/06/17/mcdonalds-to-end-ibm-ai-drive-thru-test.html
[5] U.S. Equal Employment Opportunity Commission, “iTutorGroup to Pay $365,000 to Settle EEOC Discriminatory Hiring Suit,” August 9, 2023, software excluding female applicants aged 55 and over and male applicants aged 60 and over; more than 200 applicants affected; first EEOC settlement involving AI-based discrimination. https://www.eeoc.gov/newsroom/itutorgroup-pay-365000-settle-eeoc-discriminatory-hiring-suit
[6] Jeffrey Dastin, “Amazon scraps secret AI recruiting tool that showed bias against women,” Reuters, October 10, 2018. Amazon stated that the tool had never been used to actually evaluate candidates. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G
[7] New York State Department of Financial Services, “DFS Issues Findings on the Apple Card and Its Underwriter Goldman Sachs Bank,” report of March 23, 2021: no violation of fair lending laws found, following the controversy that emerged in November 2019. https://www.dfs.ny.gov/reports_and_publications/press_releases/pr202103231
[8] “Grok AI Spreads False Vandalism Accusation Against NBA Star Klay Thompson,” April 2024, the chatbot interpreted the phrase “shooting bricks” literally. https://oecd.ai/en/incidents/2024-04-17-f827
[9] Maggie Harrison Dupré, “Sports Illustrated Published Articles by Fake, AI-Generated Writers,” Futurism, November 27, 2023; content provided by a third-party vendor, partnership terminated after the revelations. https://futurism.com/sports-illustrated-ai-generated-writers
[10] La Quadrature du Net, investigation into the algorithmic profiling of job seekers and benefit recipients by France Travail, “suspicion score,” “employability score,” 2024-2025; criticism from the Defender of Rights and the CNCDH regarding the “Full Employment” law. https://www.laquadrature.net/2025/05/22/france-travail-des-robots-pour-controler-les-chomeurs%C2%B7euses-et-les-personnes-au-rsa/
[11] On the revocation of the Biden administration’s artificial intelligence executive order, Executive Order 14110, October 2023, by the Trump administration in early 2025, and the deregulatory orientation that followed. To be checked against official U.S. sources for the details of successive texts.
[12] China’s Interim Measures for the Management of Generative Artificial Intelligence Services, entered into force in August 2023, notably requiring produced content to comply with values promoted by the authorities. For a general overview: Cyberspace Administration of China and specialized analyses.