We're going to talk about um an interesting, I would say um aspect of A I that a lot of people are talking about and working on. So what is it before we delve into? I'll give a quick introduction about myself.I currently work as a software engineer at Goldman Sachs and I love participating in hackathons, one couple of them in the past. Uh Apart from that, I also like to write and I'm a technical writer with, then I've been a podcast host host as well. So let's dive in. So uh what are, what all are we going to talk about? And, you know, we will start with an introduction. What, what is hybrid decision making? Then we'll dive into understanding this whole concept of interpretable A I that the entire A I community thinking about. We'll be looking at line which is a great library which is going to make our lives easier. And we can also lo uh be looking at really interesting examples of how you can make your pipelines um existing pipelines more explainable. So let's begin. So honestly, I'm pretty sure we all have been here, right. We've trained models, we have uh built amazing models, but when it comes to testing them out, right, there are very ambiguous and spurious results and we're just thinking like what just went wrong.

We, we, we've all been in his position and what, what we're trying to do is trying to, you know, take smarter and better decisions. So if you look at today's world, right? Um A I with increasing compute power and more and more algorithms and an ever increasing amount of data, right? A I has come out of the labs and more and more businesses companies are adopting and joining the bandwagon. But as we are moving towards more and more uh better models, we're also moving away from explainability and incompatibility. So if you look at this right? This is where we started, this is where we are, we have amazingly uh well performing models, but we do not understand how they work. I mean um sometimes it so happens that your iphone is able to recognize you without a mask, but sometimes uh with a mask, but sometimes it's not even able to recognize you without a mask. So that that's just one application, right? But if you think of it from a perspective of let's say the medical domain or the fintech domain where your loan is getting declined or um the scan say that you have cancer, you don't want false positives in those critical cases, right?

That's why this whole concept of credibility is very important. Another aspect that uh people miss out a lot is that if a model is working very well, it doesn't mean it's working as we expected it to be. Now, this might sound very trivial. But as we look at certain examples, as we go forward, you will um you'll be shocked to see how A I can actually go haywire. But before we go forward, right, let's just understand explainability from both the data and an algorithmic perspective. When you know all along when we have built data pipelines, we've understood the data really well. When it comes to algorithms, we just use the word whatever best is available, right? That's why ML has become actually a black box. And we're trying to move towards a world where it becomes a glass box. And then finally, we're able to understand it how we understand the problems as a human mind. So when it comes to algorithm, right, we we should be very thorough with, you know how it works, how it's trained, how it's tested and when we're sending in the data, right, what data points are contributing and influencing the outcome. So these are, these are the questions we are trying to get answers for. So if you look at how ML pipelines are designed in today's world, this is how we do it.

But with the entire training ML process and the test functions in place, we still have these questions unanswered. We don't know why it worked. Why did it fail in a certain case? And we always think, can I trust this A I, you know, uh if your loan is getting declined in the bank, you don't know why it got declined. You don't have answers to your questions, right? And if you think from a developer perspective, right? Many a times the models Gore Wii and you're just left thinking, how do I correct this set of? Where do I start explainable A I or Xa I as we call it is a way that answers all of these questions for you and helps you take better decisions. So you have a path to start from and essentially you're not doing something groundbreaking, you're just augmenting it into your existing pipelines. So you can consider explainable as an auxiliary support that helps you take smarter decisions, helps you navigate your way with complex models, right?

And essentially how it fits in the current pipeline is that when we are taking data from the real world and you know, giving it any black box model between the model and when humans interpret that result, we're trying to fit in this, we're trying to uh you know, use maths again to explain all that uh statistical interpretation that goes inside the model.

And the idea is not to like uh you know, again, have certain metrics idea is to give explanations that even, you know, a non technical person or someone who's not a subject matter expert in whatever your building can understand. So the explanations have to be that simple and people would say that, you know, what are you going to get out of making? Um A I explainable. The idea is to make A I more responsible, right? We are advocating for better practices and methodologies in building and deploying these animals. So a lot of things come under this umbrella and most important of them being secure, ethical A I and explainable and interpretable A I actually drives the adoption of all of these things because we are building this A I around, you know, it's a human computer interaction mo so it has to be centered around the people who are going to use it as well as um you know, in the process of making it more secure and ethical.

It's also a very good approach at achieving data transparency. You know, we are living in a tech savvy world and everybody is aware of how their data is going to be used. So you build an A I that's trustworthy, you build systems that are more resilient to protect their privacy. But at the same time, when you build systems that are explainable, you ensure that A I has learned what you expected it to learn. Now, you all understood what explainability is, you know, you understood why is it important? But how do we create it? And I mean, you've already invested so much time into building a machine learning model and you've tested it rigorously and it still doesn't work. And now you don't want to do much of, you know, you don't want to delve deep into again, doing that, repeating that like, well, we have line for us and what is line stands for? Line stands for local interpretable model, agnostic explanations. Now, why, why do we use Line? It's because if I'm saying that it's a cat and I'm 93% sure it's a cat. I want to know why it's a cat because it has fires, it has whiskers and clouds. That's how my mind would work right now. This might be a very trivial example, think of it that your company is building a child prediction model and it's saying that OK, 30% of the employees are going to leave in the month of July. But what good is a child prediction model?

If it cannot tell you why so many people are leaving, that's, that's what we're trying to, we're going towards a future where we get output plus reasons for that output and how it does. This is very simple essentially. Uh I mean, all you do is you learn a simple model on top of your existing model by taking certain samples around your existing sample. I won't delve deep into the mathematics of this given the quantity of time. But the essence is we're trying to minimize the loss function and the loss function is trying to measure the unfaithfulness of your model and how it does that it's, it's very simple. I mean, we, we call it locality aware loss. It takes into consideration the neighborhood of what you are trying to explain. For the simple model. It has learned and the complexity of the model. And there are various links that are linked into the presentation. So you can go further and read about it. But you know, before we understand, you know, and start implementing, there's a very fundamental question that will come in each one of your mind is that I was doing good already with all my ed A and my, you know, model performance evaluation metrics. Why should I use link? How is it advantages? It's because it's explaining decision boundary for you, which no, um which none of the metrics can do.

I mean you could argue that simple models does, it does give you explanations like decision trees and other models, but it does not um you know, this is not valid when you move towards more complex and bad models. Like, you know, when you combine XG boost and your classifiers, you cannot expect that someone um who's not an expert can understand it. Also. The best thing about line is that the explanations are crisp, they're short and it's actually helping you approximate your global model using very simple local explanations. Let's see it in action, right? But if you consider the classic problem of digit classification. This is how traditionally people evaluate their outputs, they'll do a TSNE plot. But if I do not understand digital classification, I cannot understand what's happening here. I mean, I do see clusters are scattered. There's something definitely wrong with what, right when we, when we want the answers for what is wrong and where do I start? Line comes into picture? Let's take an example that we are trying to classify this number eight. My human mind knows it's an eight. My model says, OK, I'm 50% sure this is a three and you're like, what is happening? Why is it even calling it a three? Now, what line helps you do is it helps you trace the boundary of the model.

Why it thought certain pixels were important in contributing it to A three? And you actually, if you look at it closely, it does resemble three and this tells you that you need to probably augment more data instead of wasting your time in hyper parameter optimization, you might want to go back to your data, correct it come back tested. So it, it's helping you take decisions in a much more informed manner because there's any plot, you would naturally be a little lost and it's not just text that line, it's not just image that line helps you uh you know, understand it's even text and even your talar data. So any kind of ML application that you're building line actually helps you achieve like, you know, build um explanations which anybody can understand. There, there's a couple of collab notebooks that are present here at this particular link, you can go and try them out or, you know, see this in practice. And it, it is an application of all the three cases that I've discussed, which includes text ML and deep learning. And um I'll also skip this part since it's, it's, it's more deep into how it actually works. But if you see four lines of code and you are done, you have an explainer model with you, which actually helps you understand.

So essentially, if you look at it right, if six is getting identified as zero and it's also getting identified as five and six, you see that model has not trained properly, you probably need to train more iterations of it. So this is a model which I just stopped at five iterations. So it has essentially learned nothing. So if you see the process is really simple, right? You have your model line is training another model on top of your model. But it's a very simple model because it's trying to explain for one explanation and it's working in a local linear space, it's not working in the non nonlinear space. So it simplifies the scope of your problem. Now, let's, you know, look at this application, I'm trying to differentiate between wolves and dogs and this is what happened with my model, if you look at the gray area, that is the uh decision boundary, the model has learned. So essentially how my model is differentiating between wolves and dogs is only the background. The reason for this bias is because wolves are generally present in the wild while dogs are present primarily in households as well as wild. So one fine day, if you give an image of a dog in a wild area, it might just get identified as wolf.

Now, this is a classic example of, you know what you were trying to do what actually happened and you do not want that when you're dealing with critical applications, you know where let's say you're trying to summarize legal documents and that uh summarization goes haywire and the end users are reading something that's not even relevant to your documents or you're working in a printed domain, we are trying to predict the risk involved in a particular transaction and you like that bypass that will be a false negative.

And in domains like this, it becomes very critical. So essentially line is helping you do validation. It's helping you uh decide your next part. It's also helping you understand what, what went wrong. And it's also answering all of your howls and where's and what's and, and what's better than that. I mean, if you look at uh this particular model differentiating between dogs and cats, you can easily see why a dog is a dog even I as a human would see that left a part of the image is a dog and line helps you validate your decision boundaries. And essentially, if you're trying to, you know, sell your idea, sell your um um algorithm to someone or you're trying to pitch it to someone. It's very easy to explain what your A I does. Rather than delving into the mathematics of functionality of it, you can explain them from a working perspective. So when, when you're proposing this to someone who's from the business aspect of it, they need to understand it even a more abstracted manner and even for text, right?

The explanations are very, very simple. Let's say I'm just trying to classify questions on staff overflow. If I'm saying that the question belongs to the SQL category, it's because these keywords were important while these were not. And in a similar way, you can also highlight in the same.

Um you know, at the same time, what, what part of the text was important? Where did it occur? How many time did it occur? So it helps you in easy dashboarding as well. And you know, you can do pretty much anything if you're doing sentiment analysis. This is a great way to demonstrate why your sentiment analysis works. You can um you know, talk about how it's able to identify the proper words with proper importance and their occurrence. So you can talk about their correlation with other words in the text. And while um line great library, you also have sha as a library available, it's more suited for um um you know, tabular data because the explanations are much more involved for tabular data. And if you're someone who's interested to explore this further, I would recommend reading this book by Christopher Mulan. It's a great book, someone who's um you know, and trying to understand more interpretable ML and here we are coming towards the conclusion, you know, line as a library but rather explainable A I as a concept is very, very important because it's helping you induce trust and transparency in your existing applications while helping you increase the public perception of what you are building.

Because essentially A I being a black box people uh would necessarily doubt that should they really use it? How are, how is my data going to be used? Is it really safe? Are the decision it's going to make? Do I know how it works? So, you know, you are um using an auxiliary support to make informed decisions. And at the same time to people who are your end users, you're able to give explanations which are very short and selective and well, everything is really great. There are few drawbacks currently since it's in development is that um you know, you need to know the complexity of your model beforehand as in um how deep is your neural network or how deep is your um decision tree. And since the sampling is happening from a Gaussian distribution, it does introduce certain bias in your models which you have to tune by and you know, by tuning how many number of samples you want to take for your local model. And this brings us towards the end of the session. The most important thing that we've learned is that do not trust your models without explanations because you do not know what they're actually learning, right? And um we all know interpreting machines, intelligence stuff even for us humans.

So it it's a, it's a process, it's a journey to understand um you know, to understand how these explanations can be made more and more verbose and simpler. And the most important thing is for any business centric use case, right? Where there are stakeholders with regulators or consumers, it's very, very essential that we ensure model is learning what we expected it to irrespective of the accuracy. So accuracy shouldn't be the only metric that we track at any given time. And why we have come a long way.

What the future holds for us is that we are looking at a world where focus will be on model agnostic interpret ability tools right now, how how we do is we analyze the data. We do not analyze models from a qualitative perspective much. So we'll have partial dependence functions like line helps us do to, you know, say model A is better than model B in certain cases or for a certain test set, right? And with all of this happening, we are essentially going to automate ourselves. The auto ML has done that for us, but that has introduced another problem which interpret ability is going to solve. And since all of this is happening, there's a great boom in machine intelligence research that's happening. And there are a couple of references for you to look at this uh the original paper by the authors of this library and certain blogs. So that brings us to the end of the session. Thank you for being patient audience. And um I'll be open for questions. You can put your questions in the chat. You can also reach out to me either on Twitter or on linkedin. Uh If you have any further questions, if you're working on a project and you would like to know more. So I'd be happy to answer your questions. All right, then. That's a wrap. Uh Thank you so much for having me today. Bye bye everyone.