I hope you are enjoying your day so far. My name is Nicola Martin. I'm head of quality engineering at a company called Adaa. We're in the A I and Data Science Space.And today I'm going to be talking to you about A I biases in particular, disrupting A I biases in software engineering. It was estimated in 2020 that 59 trillion gigabytes of data was collected and processed for used in A I models. And according to the international Data Corporation or I DC, by 2025 it is estimated to increase to 100 and 75 Z bytes. So we see the use of A I models in a lot of products now. And so it's not rocket science to us anymore. Or this curiously strange thing that we don't really understand. We see it mostly in areas such as products with digital access, um entertainment, education, the environment, finance, health hiring and internet of things. So things that you use around the house, for example, and what exactly is A I bias according to levity A I bias refers to the tendency of algorithms to reflect human biases. It's a phenomenon that arises when an algorithm delivers systematically biased results as a consequence of erroneous assumptions of the machine learning process. What are some of the causes of bias in A I models?

So it could be um related to bias in training data, which is a special area of mine because I work in quality and we're not just testing quality looks at areas such as data. So we are really interested to understand the training data that's being used in models. It can be around algorithmic bias human bias, which is something that we will obviously come on to um and lack of transparency from um teams or companies that are um employing people to put data into these models. So where we talk about biased training data, if the training data that is used to develop an A I model is biased, the model may reproduce and ultimately amplify that bias, which is a a real problem. This can happen if the data used to train the model contains inherent biases or doesn't give a true reflection of the eventual end users. So it may start as an idea for a particular um user. But actually the u the user group is actually going to be much wider and there have been no considerations made for those initial uh or additional users. So when we talk about algorithmic bias, it's algorithms used in A I models that can also be biased. So um especially if they are developed with assumptions or criteria that are based on biased data or indeed our own human biases in terms of human bias A I models can also be biased if they are developed by people with certain biases as I've mentioned.

So that could be consciously or unconsciously, this can happen if say a development team has a specific goal or agenda or if they make decisions based on their own personal biases or assumptions, and there are no checks in place to ensure that doesn't happen. Engineering bias in teams can mean that team set up is not fully representative of the products or the client base that the organization is trying to reach and where we talk about a lack of transparency, it can be in the decision making process of A I models. So this can make it difficult to detect or even address the bias. So where logic and decision making processes of an A I model are unclear, it can be difficult to assess whether or not the outcomes are biased or not. So we have several different examples that I've got on screen here of A I bias in the news. And I will say when you search for stories like this, you almost always come up with facial recognition. If I go back as far as maybe 2019, they're the first kind of stories that we used to see. We are used to seeing in the media around A I bias. So it's usually about facial recognition software that's gone terribly wrong. Um So it's very high profile. Um, you know, here, I've got some examples. Um, one in particular around the US immigration system, um and CBP one which I'll talk about a bit, a bit more.

Um, when we move on to future slide, um there are some that seem quite funny on the surface but um there have been several stories in the last few years about um the use of soap dispensers or in this instance here force it tap. But actually it seems funny because they're on um platforms like tiktok or Twitter and um you know, they may be um going viral and for, you know, lighthearted reasons essentially. But underneath it, there's actually something more serious here and it's actually a bit sinister, the fact that, you know, as a black person, I could potentially use a tap or um as I've said, soap dispenser and find that I can't use it because it doesn't recognize me. Um And in the terms of the facial recognition again, that's an issue that that's come up. Um in particular this one that I found around um reproducing inequality. It's around the fact that A I image generators. So you'll see a lot of those. Now you've got Microsoft version, you can go in and put in a request for um a picture that looks like something or looks like you and potentially get something back or you could put in some specific requirements around a picture that you want to generate and it will give you a um you know, its own responses back based on your your query.

So in this instance, what has been highlighted is that there are inequalities around the images that are returned. When say for example, someone says show me pictures of women in stem and you know, for the most part, what's been coming back in the news is that the pictures will all represent white people, for example. Um And I've been very lucky this year to have been allowed to be part of the UN UK women's delegation for the 67th meeting of the Commission for the status of Women. And their main um topic this year was around um inequalities in technology. And a large part of the conversations that I was in um listening to meetings I attended were related to um A I and uh gender inequality. So that's a massive issue that we need to try and resolve. Um And you know, it's going to take a while to try and resolve those, but we need to be working to make that happen. So organizations like the UN are working with groups and NGO S and um other groups and obviously governments to try and fix some of these issues with, with companies and to try and hold them accountable.

So I have a lovely picture here of George Clooney and Barack Obama and we're talking about facial recognition technology. So facial recognition technology depends on machine learning technology. Um Some of the vendors that are famous for having um software in this space are Amazon.

Amazon themselves have a software called Amazon Recognition. There's also been some more controversial um examples such as Idem A's facial recognition algorithm that was supposedly biased um against black women because again, it wasn't recognizing it and Idem are actually a security company.

And as I've mentioned, the CBP One app which was hailed as you know, revolutionary and going to solve a lot of the administration issues around people trying to apply for um um uh to move to the States. Um And one of the issues that came up almost immediately was the fact that people at the Haiti and us Mexico border were reporting that they couldn't um register onto the app. Part of the registration process involves taking your photo and uploading that and because it wasn't recognizing darker skin tones, unfortunately, it meant that these people couldn't register to start the um application process. Now you have to bear in mind that some of these issues were raised at the UN meetings this year um that it's families that are using these apps. So in some instances, you've got people in the family who can register and you've got other people who can't register. This has been highlighted by lots of um groups who work in this space as a major issue because families are forced with the um, decision to have to leave certain family members behind or abandon their application process altogether. Or they're looking at a long blockage because of the fact that they just can't register. And until these problems are solved, there's going to be a massive backlog of people trying to apply um, to get into the state.

So it was highlighted within days of release and it's looking very bad for the um government at the moment. So that's something that they really need to pick up quickly because the press are obviously, you know, it's highlighting bad press for them at the moment. Um I've also talked about automated hand uh soap dispensers earlier. Um There are design flaws basically that mean that the automated soap dispensers are not recognizing darker skin tones. So as I've mentioned, there was a first version of uh you know, these stories that happened around 2017 where a machine only recognized lighter skin tones, a man and his colleague were in uh a bathroom. I think it was in Saudi Arabia and they um tried this um soap dispenser and at first it was funny, but actually, it's not funny anymore when you realize that the reason it can't recognize you is down to your skin tone. We have more serious stories around racial bias in oximetry. Um If you're not aware this, this machine that I'm showing you on screen right now is called a blood oximetry meter. Um It's a medical device that's placed on the end of someone's finger and it checks the oxygen levels and allows nurses or doctors to understand, you know, how, how ill a person is or how much oxygen they need to bring their levels back up.

Um during the pandemic, um the UK government actually announced that they were going to be doing an investigation and a report should have been uh produced off the back of that. Um It was initiated because of um very urgent reviews that were called for um the systematic systematic racism that was found around gender bias and um and gender bias found around the um medical devices. Um and the fact that for certain groups of people, it was actually meaning that their level of medical care was affected and in some cases, they actually reported deaths. So it was very serious, especially during COVID um that, you know, the the information being given was correct and unfortunately, they failed in this instance. And that's why an investigation and report was initiated. We also have um allegations of gender bias now, um I haven't mentioned it yet, but obviously we now have um LL MS which are large language models and the most famous of those is Chat GP T. It's dependent like all A I models on the quality of the data it ingests. So you know, if the quality going in is bad, what you get out of the other end is equally going to be as bad.

And um it's been described as mansplaining as a service and um there were quite a few articles written when it first came out. Um, you know, alluding to that. Um We've also got recruitment tools which um obviously with models trained to predict um who should be hired or fired from jobs. Um, Amazon had um a piece of software that actually um was grading people within the tech teams um based on their skill sets, et cetera. And that should have been a good and positive thing and should have helped with building skills, recognizing skill gaps and then um allowing managers to figure out what their teams needed in and individuals needed in terms of training. But actually, they found another application for it and they realized that um Amazon realized that they could use it to um figure out whether or not somebody had enough skills to be kept on or whether or not they needed to be um made redundant. But what they found was that the tool was actually um the software was actually recognizing women predominantly as um not meeting the standards that they had. And actually that meant that they were being um highlighted more predominantly than their male counterparts for possible redundancy, which was bad.

And so apparently the uh tool was abandoned. So what are the major implications then for society if we have A I with bad data? And um you know, uh poor um set up in terms of the data being used and the training data that's being used and then ultimately the products, right? So, um you know, it's the implications can mean poor decision making for organizations. There's a lack of trust from the public in terms of the products we have. As I mentioned, poor quality of products, potential discrimination and exclusion and reinforcement of stereotypes for um groups, certain groups in society. So they will be singled out by these uh by the software just because the A I has been built badly. So there are ways to mitigate the bias. Um We can use better data sets and ensuring that we're um using larger data sets as well. We need to make sure that they're more diverse and inclusive. We need to include regular internal audits. So holding ourselves accountable if you like rather than waiting until an external auditor, which is a new area that's coming in. Um you know, is able to come into a company and do that audit. We should be doing that regularly ourselves. Um It, as I said, it holds us accountable.

Um And then we can increase transparency and giving visibility of how our models are made, the data that we decide to use, that gives more trust. So it's a, you know, it's a circular thing. And then, you know, I will mention this a few times during this presentation, but more thorough testing of models. And then, um, obviously before and after deployment, it's really important that we're doing monitoring as well. Um, of those models, not just leaving them to, um, you know, once they've gone into production, we need to make sure that we're still monitoring those for, um, you know, to make sure that they're working correctly and as we would expect, so what's the future around A I models?

And, you know, essentially trying to mitigate that bias. We've got the introduction of new regulations. So the new eu regulations are coming and um you know, that should make the use of A I models within organizations more robust. Um There's more research obviously being done, whether that's by individual organizations or governments are definitely doing more research. Um You know, there will be increased diversity because more talks like this, more conversations happening more widely about the lack of diversity within teams.

There will definitely be something that will, you know, decrease the social bias. And, you know, there are definitely more calls for transparency and that will become part of the regulations, I'm sure. Um and you know, that will mean that companies can't hide information anymore.

They'll have to be more honest about what they do, whether they're buying those models off the shelf or whether they're building them themselves, the ethical considerations are increased regulation. As I've, I've mentioned the A I regulation in Europe and the standards A I audit.

So there will actually now be qualifications when there are now some. Um but there will be more um around um A I audit. So you could train as an A I auditor if you felt that you had the background and skills. Um you know, we are looking at addressing A I bias by, you know, a concerted effort by engineers, government and other stakeholders to ensure that A I systems are developed and implemented in an ethical and unbiased manner as unbiased as you can because you can't get rid of bias altogether.

I should say that. So what can you do as a software engineer? You can take a proactive approach in the development of models, get involved in the conversations, get involved in things like data collection and prep. Don't wait for the model to be ready for you to apply to your code. Make sure that you're getting involved in the data and collection prep conversations and understanding more about that. We should ensure that the quality of data is diverse and inclusive check for bias during pre processing of the data um and encourage monitoring of models in production work with your platform teams to ensure that that happens and with your development teams, join the ethics team within your organization.

It's really important that you get involved in those conversations and understand what's expected of you and your team contribute to best practice initiatives and ensure you feed into the principles around fairness, transparency and accountability and actively follow those in your work. Again.

I am going to say, ensure thorough testing and validation of data is carried out and I'm gonna make a call here for you to make a call for more diverse and inclusive teams. We want people from different backgrounds to minimize that social bias and ultimately, we will all produce better products. Thank you.