This has been a critical problem across machine learning applications in the last years. To break open the black box of AI models, and understand how they make decisions, the concept of explainability was introduced.

Then, LLMs entered the chat. They answer our questions confidently and with a beautiful prose, even when they are making up data. Explainability then becomes essential to trust -or not- their output. But when the existing explainable AI methods cannot be directly applied to these models, what do we do?

In this talk, we will delve into the topic of explainable AI, and its importance in the current context of LLMs and agents. Starting from traditional machine learning to then focus on generative AI, we will cover the different methods that can be implemented, from well-known ones to novel proposals stemming from our internal research. We will go through the main risks and challenges we have encountered when implementing explainability, and how we have solved them. Finally, we will share some tips and tricks to integrate explanations into LLM-based workflows (also when using existing third-party services that are not natively explainable).