What is the current role of explainability (XAI) in Machine Learning and Data Science? For the past decade, the challenge in data science has primarily been finding the right "algorithmic recipe" to create increasingly powerful and complex ML models, which are consequently becoming less and less understandable.
AI systems based on these algorithms often make predictions without the designer or user being able to understand the reasoning behind them. These systems are nicknamed "black boxes" due to the opacity of the data processing that occurs within them.
The black box phenomenon and the natural distrust it generates are major barriers to AI adoption. R&D efforts must focus on trustworthy AI to find a form of explainability for every algorithm, even the most complex ones.
Among the actions required to create trustworthy AI (Explainability / Privacy / Frugality / Fairness), XAI is particularly important because it concerns the model as a whole, including both algorithms and data. Trust then becomes the responsibility of the AI creator—the person who designed the model and chose the algorithm.
Many software publishers whose solutions are used maliciously hide behind the excuse of algorithmic neutrality. They seek to shift the blame onto the end user and their use of the tool, absolving themselves of any ethical responsibility.
Social media platforms are the most striking example of this. They prefer not to have to explain their algorithms, even at the risk of losing control of them,rather than taking ethical responsibility for their creation. The very purpose of explainability is to prevent these abuses and enable trustworthy AI.
Beyond these ethical and regulatory aspects, explainability is primarily of operational interest. The example of the autonomous vehicle is arguably the most telling regarding what explainability (XAI) can and must bring to our daily lives. Whether from the perspective of the designer or the end user, XAI provides access to three levels of understanding AI:
The process of creating a machine learning model usually consists of three phases: design, training, and evaluation, with an iterative logic until the model is "finalized" as desired for production.
This is where explainability reveals its first major operational benefit. With a better understanding of the algorithmic recipe, the designer can choose not to move a model into production if the explanations are unsatisfactory, despite good statistical results. They can then restart the "design, training, evaluation" iteration and avoid deploying an uncertain AI.
XAI makes it possible to understand what the algorithm is doing during the training phase and provides the opportunity to:
XAI allows you to verify that the model you are building truly corresponds to what you intended to model, thereby accelerating the development phase by helping Data Scientists "finalize" the model with confidence before moving it into production.
During the model deployment phase in production, it will be confronted with dynamic data, and explainability will once again be required to:
From the end-user's perspective, explainability also holds major value. We believe that XAI plays a decisive role in the adoption of AI by the general public by enabling trustworthy AI and streamlining the user experience. Widespread adoption is an essential step in the development of AI today.
If software publishers want to continue promoting artificial intelligence, they must open the black box, expose its inner workings, and explain them. They must provide the end user with elements of understanding to keep them in the decision-making loop: should they follow the AI's predictions or recommendations or not?
Just like designers, AI users must have access to the three levels of AI comprehension:
To understand a service that uses artificial intelligence, every user must be able to question the decisions made by the algorithm:
It is possible to keep humans in the AI loop. This is achieved through the use of XAI methods.
To master an artificial intelligence, one must know what it is capable of doing and understand the interactions between data and their causes:
The challenge of mastering AI lies in creating an empowering complementarity between humans and algorithms. Enhancing humans through machines is only feasible within the framework of trustworthy AI, particularly through explainability.
XAI is the pillar of trustworthy AI, encompassing fairness (eliminating biases and discrimination present in data), frugality (minimizing the energy required to train models), and confidentiality (respecting privacy and data security).
Trust is built on tangible evidence; it is the result of understanding and mastery.
A machine learning model is built on a limited set of historical data. Having explanations for how it works allows us to trust its behavior in production, especially when it encounters data it has never seen before.
It is the role of explainability to go beyond statistical guarantees to convince the widest possible audience that trustworthy AI, which keeps humans at the heart of technology, is both possible and already operational.
Developing trust to encourage adoption is also the approach taken by the European Union. Committed to creating a safe environment conducive to innovation, the European Commission proposed in 2021 a series of initiatives that will help strengthen trustworthy AI and make explainability the standard in artificial intelligence.
_Craft%20AI.jpg)
