The transition of AI from providing recommendations to executing decisions presents organisations with an urgent accountability puzzle. This challenge formed the centrepiece of a La Tribune discussion at Tech Show Paris, chaired by Jérôme Lucereau, Co-Founder of La Fresque de l'IA. The panel brought together Paul Richiardi, CISO at BACCARAT; Mathieu David, Chief Data & AI Officer and former member of the Joint Chiefs of Staff; and Claire Lerondeau, Founder of Artenovia.

Lucereau opened the conversation by distinguishing between making a decision and establishing the framework within which decisions occur. He observed that AI typically functions in the deliberative space, sifting through numerous possibilities and narrowing them to a manageable list of options. "It doesn't make our decision, but it probably takes up our space for deliberation," he noted. This shift fundamentally alters how responsibility should be allocated. Rather than residing with a single person or unit, accountability now extends across a network encompassing executives, technical staff, security specialists, end-users, external partners, and the systems themselves.

Measuring Autonomy

David proposed a framework consisting of three questions to assess the degree of autonomy an AI system possesses: does it perform actions that cannot be undone, does it determine its own methods of operation, and does it assess the validity of its own actions? These benchmarks enable organisations to identify what protections must be in place before deployment. An AI offering recommendations may demand a distinct governance structure compared to one capable of executing decisions independently.

Richiardi contended that AI agents warrant treatment as operational entities bearing defined accountability. "An AI agent must be seen as a pure identity, meaning it must have a real name, a real leader, a real associated need," he stated. This identity requires periodic examination of the agent's access rights, intended function, and actual performance.

Responsibility Follows the Risk Decision

The panellists rejected the notion that accountability can be handed off to vendors or technical departments once problems arise. Deployment involves distributed responsibility: management assumes the business risks, the CIO oversees technical integration, and the CISO surfaces and communicates security concerns. External providers may bear contractual and legal obligations, while staff must adhere to established policies and guidelines. Yet the individual who endorses the risk remains pivotal in any subsequent accountability determination.

Documentation emerged as critical. Records of approvals, cautions, system activity, and compliance checks establish a chain showing who possessed what knowledge, when choices were authorised, and whether the system followed recommendations. "Everything that is not written does not exist for a machine," David emphasised.

Safeguards Before Systems Act

The panel examined a scenario involving an AI marketing agent generating and distributing promotional material, including audio mimicking a recognised performer. In this situation, the company might face the initial complaint from the rights holder, yet liability could implicate the creative agency, marketing division, IT operations, and technology vendor. The speakers urged organisations to rehearse such situations before going live. Deployment should impose caps on volume and scope, while irreversible operations must demand human approval. Shutdown procedures must remain available should a system exceed its authorised parameters.

Richiardi drew a parallel to disaster recovery protocols: "It's like an untested backup; it's useless."

Preserving Human Judgement

The discussion turned to a subtle risk: treating human oversight as mere bureaucratic procedure. Meaningful supervision of an AI system demands sufficient time, knowledge, and data for a person to question its recommendations. "If they don't have time to check, it's as if the person is useless," Lerondeau observed. The panellists acknowledged that AI outpaces humans in specific domains, including threat detection in cybersecurity and certain diagnostic tasks in medicine. Yet accelerated processing can erode opportunities for careful thought and dissent. David highlighted a deeper hazard: not merely that AI might fail, but that humans could gradually vanish from decision-making altogether. "We need to keep the human deep," he said.

The panel converged on a central principle: organisations should not grant an AI agent greater autonomy than they can manage. Prior to deployment, leadership must clarify the system's capabilities, identify who authorised it, determine who accepted the risk, and establish how it can be halted. Lucereau captured this succinctly: "The question is more about knowing, fundamentally, who is responsible when the AI decides, but rather who let them decide."