Just because we're all in the same boat, at least for a little while longer. I am broadcasting directly from my daughter's bedroom. There's stuffed animals back there, her clothes. Um So I just thought you all would appreciate that.Um Just in case you're wondering, so, are we building products that are ruining the world? Yes, we are in case you were thinking I was gonna take a long time to get to the answer to that question. Um We can all leave now. Just kidding. There's more to it. Um I used to have a long preamble that I would give um at the beginning of this talk about the current state of technology, the unintended consequences, the harmful effects, but honestly, it was exhausting. Uh and we all know them and I suspect you knew how I was going to answer this question when you came to this talk because are we building products that are ruining the world? Yes. Yes, we are. But we don't have to. Technology is capable of making the world a better place. But the technologists have to choose that. We have trained a generation of entrepreneurs and product creators that the only thing that matters is massive growth and engagement at any cost. We have taught them to work for the highest bidder and that they bear little responsibility for their individual contributions.

We have crafted an environment where V CS make some of the most important early product decisions and the quest for attention erodes exploration in the areas that most need our creativity. We have to untrain ourselves. If you're curious who I am, I am the CEO of substantial, which is a research innovation and build studio. We're headquartered in Seattle, Washington. It is closed right now with many, many businesses, although soon to reopen. But until that moment, we are fully remote and distributed workforce, workforce of 50 strategists, designers, product managers and developers. And for 14 years, substantial has been conducting research, creating innovation and building digital products and services for start ups, foundations and enterprises.

We created products for some of the largest companies in the world and some really small ones and occasionally we even build products for ourselves. So I consider myself optimistic about what technology can do for the world. But these are trying times and I find myself desperate to use whatever voice I'm given to convince people we can change. So this talk is not about passing judgment. It's about fostering a sense of individual responsibility and purpose that we can bring to every product we create.

So today I'm gonna share a case study with you and then we're gonna talk about how we can approach our work with integrity and a more ethical mindset. So there are a lot of really amazing technology companies out there. I get to work with many of them, but many I hope to work with someday. These are companies that are really trying to do everything right. Companies finding remarkable success, building products we use every single day, but even the companies doing everything right can lose sight of the power imbalance that technology presents. I'm encouraged that design is becoming a more prevalent voice and advocate for equity and responsibility.

A constant reminder to understand the customer. The design is not a field that is immune from doing harm and placing the burden of ethical interrogation on one department or role or even on a process, ignores the realities of the decisions big and small that happen outside of design.

Design is a skill and an approach and it can be employed for many purposes. Remember, most design departments still report in to the business. So we shouldn't expect products to be inherently ethically designed just because we put them through a process. We can't expect ethical products to be created without embedding the practice of ethics throughout every department and in every step of creation, a singular focus on the customer will not ensure technology that is responsible without interrogating who the customer is and why?

So I'm going to tell you the story of a start up that did everything right, Acme Health Tech is a start up that has made a name for itself by tapping into the enormous demand for digital health information, particularly health records. Their free platform allows physicians to document patient visits, order prescriptions and lab tests and easily access their records online. And this this is very useful for doctors, particularly in rural and more small offices that really needed this kind of infrastructure and support.

Now, Acme had experimented with other revenue models but research oriented businesses in the health care space take years to develop. And in the meantime, Pharma advertising is a huge pool of spend and a straightforward sales opportunity. But once you choose that ad driven path, commercial gravity tugs in the direct direction of selling more ad inventory and creating more effective advertising tools backed by a high profile venture firms, Acme had reeled in over 100 and 50 million in private funding. They had some really big mouths to feed. So what if they turned all of their data into insightful information for their advertisers, particularly pharmaceutical companies that never had access to granular weekly data on their drugs utilizing something called clinical decision support is not new.

It, it can be applied to patient care in an infinite number of ways and it has been vaccinations is one such way. Previously, Acme worked with a Pharma company to develop workflow tools based on CDC guidelines that would try to boost pneumonia vaccinations in the elderly and that work was viewed very positively. So why not apply it to other uses? Partnering with a Pharma company, they started with a research study to see what kind of C DS would work with patients suffering from chronic pain. The idea was to get the pharma companies pain drugs to certain kinds of patients, patients who weren't currently taking pain drugs or those being prescribed. The company's less profitable products. It also aimed to secure longer or consistent prescriptions. Their research found some very clear and actionable insights.

One of the things they found that overuse of medication and pain treatment was already a national issue and gaining attention. They also found that reminders have a sustained influence on the rate of opioid prescribing. So the Acne design team got to work creating a seamless U I to help doctors in real time patient care by providing helpful suggestions based on certain triggers. Despite the suggestions, looking like unbiased medical information, Acme allowed the pharmaceutical companies to participate in designing the C DS alerts, including selecting the guidelines used to develop the alerts, setting the criteria that would determine when a health care provider received an alert.

And in some cases even drafting the language used in the alert for this product, the drug maker paid ACME a million dollars and from 2016 to 2019, it went off 230 million times. It may be difficult to imagine the internal decision cycle that approved this work without triggering some ethical or legal alarm bells in a business of this size, perhaps a $1 million deal might not have been discussed at the board level, but 14 of them. Certainly. What I want you to think about how many leaders, managers, analysts, researchers, consultants, designers, developers Q A people worked on this product. Imagine all the meetings to discuss KPIS and customer journeys and use cases and features. This wasn't an unintended consequence. This was very intended and very effective.

And in how many of those meetings are the objectives, intent and success, metrics, deeply discussed or even questioned. And how many meetings did anyone raise objections? We don't honestly know exploitation is a really strong word. But technologists laugh when we say if the product is free, then you are the product. I have a lot of concerns about technology companies. But my biggest concern is an industry culture that prioritizes revenue and speed above all else and deprioritizes the lives of users in general and vulnerable populations in particular. And if you're thinking regulatory progress and legal requirements are going to solve ethical friction, know that compliance and ethics are not the same thing. Compliance is a minimum bar for social responsibility, but it's mostly about risk mitigation. Ethics is not as black and white as regulations and law, but it is about how you contribute influence and read acting ethically takes practice and courage. This is why ethical mishaps are thornier and frankly more damaging they can happen in such tiny steps. We don't know we are moving until the line we have crossed is miles behind us. Now, it's worth mentioning that the employees of Acne Health Tech seem just like you and me. I don't get the sense they chose worsening opioid addiction as their career paths.

And for what I could tell, their design team seemed diverse and close knit and focused on quality design practices, their design principles while this product was being created and deployed were published on their website as three words, simple, focused and trusted. And I imagine the pharmaceutical companies would wholeheartedly agree that they were those three things. There's no point hoping to burn this whole thing down. Technology companies are amassing more and more power start ups are being created every hour. The economy and marketplace reflect the quest for the next unicorn and users increasingly reliant on products but companies are made up of individuals. What if we try to convince each and every technologist that they have a personal responsibility to practice more ethical decision making?

How might we rethink our everyday decisions to ensure critical thinking is the most important objective, an integrity framework is broader, deeper and more demanding than a legal compliance initiative. It doesn't replace design thinking or compliance. I am not an ethicist and it's very possible that your work could benefit from that expertise. But I believe strongly that every technologist should be empowered to think and act critically. So what might that look like in practice in many companies? The very structure employees and teams are in diminishes a feeling of agency and personal responsibility. That is not a coincidence. We need to change the operating system that products are being built in are directives and mandates coming from above without discussion or involvement at every level.

What is the message being sent from incentives in your company is a large percentage of compensation and equity, for instance, are people bonus for hitting only revenue and profits? The result is going to be a bias towards shortcuts speed status quo and poorly thought out implications.

What kind of accountability is framed in your structure? Huge competitive atmosphere, growth and high turnover all the time, constant reorgs and leadership changes. We may be telling ourselves that the meritocracy will serve us, but it has proven not particularly adept at ethical behavior.

Are your teams working in silos? Because they're gonna have a much harder time connecting to the intention of the work. There are competing priorities or competition for resources? A very clear response would be to value only objectives without factoring in the larger picture.

Now, the most important question to answer up here is actually who will be the most affected by your product, not who will pay for the product, not necessarily who will use the product and to condition ourselves to ask that question above all else requires rethinking the operating system that most companies create products in.

And this question can't only be asked by designers. So should we sell more pain meds or gain efficiency for physicians? You can imagine this might have been the question asked early on at Acne Health Tech. And it seems an easy question to answer if the pharmaceutical company is paying for the product. But remember, efficiency for physicians is why the company was founded and you can increase efficiency for physicians without keeping one very important mandate in mind. First do no harm.

So selling more pay meds was the easy answer. But and the road to more revenue, but it lost sight of their company mission and traded in their values for cash. Had they chosen efficiency for physicians realized the connection to positive patient outcomes that would have created a very different product. And if the leaders in the company don't set a very clear example of what is important. Teams will make their own assumptions. Every use case.

Every KP I, every feature is an opportunity to lead from why? What does successful engagement mean in your product? Are you engaging in open dialogue and critical thinking across all of your teams? How do you define success? And does it factor in humanity, community and society?

And most importantly, could you disclose your intentions openly to all of your teams, all of your customers and all of the users? If the answer is no, then it's pretty obvious something is amiss. You need to reduce your blind spots and believe me, you have them, Acme did robust research, but they did it with an agenda to support an already troubling objective with a customer that had an obvious conflict of interest. What if their research had focused on doctors and patients?

Many within the tech industry positioned themselves with the actors best suited to address ethical challenges rather than less technically inclined stakeholders and advocacy groups. This can manifest in relying on technologists to use their personal judgment to discern and evaluate the ethical mistakes of their own products.

No matter how skilled you are at critical thinking. There are times when getting neutral expert opinions is warranted, you could certainly hire an internal expert that immediately creates a conflict of interest and fiction. The most public recent example of this is with the Google A it.

So how much do you know about the most vulnerable people in your system? Is your research agnostic of any agenda? Because if your customer POV is driving what you look for, you will miss important information either when you're conducting research or when synthesizing. And that is how you can do hardy research and still not create an ethical product. And for many companies, independent specialists empowered to do equity centered audits, research and systems thinking is the most effective solution. The products are not isolated. They are part of an ecosystem along with the intended users like doctors and other actors like patients, insurance companies and existing infrastructure and protocols. So this is a very rudimentary version of a map of a healthcare system. But do you understand the system your product is living in?

Do you can you map out all the dependencies, all of the connections because system changes take place gradually and are interconnected. Expectations of speed and scale and adoption can cause imbalances and unexpected consequences. What is the key to the system? Your product is living in the entity whose existence, all other actors depend. In the case of Acne health tech product, the key was prescribers. And if we're asking ourselves if the product improves the system changes it or eliminates part of the system. In the case of ACME, the product changed the way the prescribers made decisions and improved the speed of those decisions. But if we're thinking about how the product might change the system in unintended ways, the acne product made the quality of the doctor's decisions worse.

And it deprived the patients of an opportunity for different kinds of care and tipped the scales even further in addiction to painkillers. So what if the key to the acne product system had actually been the patients? And it could be argued that any product in health care should center the patient because that is in fact the whole point of health care. So once you understand the system, you can map out how it might evolve and this isn't an empty exercise. There are trends that are affecting the system locally and globally. You need to know what the system is gonna look like in 135 years. You need to know how your product is gonna grow within that system and how your success metrics will work within that. And you need to know how much runway you have to actually bring value because I can't tell you how many products are vastly not built the way they should be because they did not have enough time and money to really work on the value stream. That was most helpful for their users. So we all know good products have been built in bad environments. There is a cost to product builders or to the users, but someone paid for better or worse technologies are driving the economy the way we approach our work, our responsibility, our accountability will affect research, innovation regulations and the very future of work.

We owe it to ourselves, our customers, our teammates to create an environment where the richest thorniest and most important questions can be explored without impunity. That is how you change the world with great products.