Quality Engineering: Automate Your Way to Excellence by Devon Jones


Video Transcription

Welcome everybody. Um My name is uh Devin Jones. I'm a lead quality engineer at Fanatics. Um And uh uh my presentation is on quality engineer, engineering specifically, um Q A automation a little bit about me. Uh I am a boot camp graduate.I graduated at a uh female only coding boot camp in San Francisco called Hack Bright in 2014. Uh pretty much immediately became a contract to hire a Python developer. Uh I ended up moving to Denver in 2015 after my contract was over. Uh I initially got a quality engineer job as kind of a foot in the door. Um I was a Python Dev when I moved to Denver. There just weren't a lot of Python shops. Uh I needed a job. There were, there was a company hiring um quality engineers, entry level quality engineers and they told me that it would be a good way to learn the product and the code base figured why not went ahead and accepted it. Um And I've been doing it for the past seven years. Um I love Q A automation. Um and I'm very excited to hopefully get some of you to get excited about it as well. Uh Recently accepted a position as a Q A lead for fanatics. It's kind of been my career goal for a while to uh get into leadership. Um And I finally got to do it. So what is Q A automation and why am I so excited to give a presentation about it?

Let's talk a little bit about um where Q A just general manual, Q A fits in to uh the development process. So the developers do their work on their tickets. Uh Usually I'll start testing iteratively as functionality is implemented. DEV turns a ticket over to QAQ. A tests it. Um When the ticket passes, we mark the ticket as pass, the DEV is cleared to merge their code into master. And then we have to do something called a full or partial manual regression and a regression is retesting things that have already been working on our site to make sure that the newly merged code didn't break something that was working previously. So you can imagine eventually how the more complex your app gets, how companies don't really want to take the time to do a full regression every time code is merged. Uh When you also take into the fact that a lot of companies, they want things tested on multiple browsers, they wanna make sure that it works on mobile web and web, they wanna make sure that it works on their native app. Um What about different devices. What if you're on a tablet versus a phone or an android versus an apple and then even different Os versions, like there's a lot of people that just don't update on a regular basis and then also different data sets.

What about the difference between a logged in user versus a non logged in user or Amazon, a prime user versus a non prime user? You could see how this could get extremely difficult to scale. So a lot of companies that don't have automation, they just don't do a full regression. And when there's no regression, you have the potential for a lot more bugs to be introduced. So let's see what happens when we add automated um regression. So now the work pipeline looks a little more like this. So the DEVS do their work. Uh The quality engineer is still gonna be testing iteratively on things that are in progress. Um We're still going to manually test the ticket. Once the ticket is manually tested and cleared, the DEV will merge their branch into master. Um I will automate the test that I just did manually and I will merge that into my framework branch and then we can run the entire framework repo against the master branch. Um If any bugs are found, same thing with manual, we create defects. But this way, instead of a human doing all of this automated uh or all this manual regression, um we can write code that does it for us. So obviously, there's a lot of benefits to quality automation. Um The first of which being get back on my slide here. Ah Here we go. The first of which being you only have to automate a test case once um you can also have multiple data sets per test.

So if I wanted to test log in with different types of users, I could easily do that. Um I can run my test against multiple browsers machines, browser sizes, all sorts of things. Um I can do a full suite or I can do smaller subsets. Um For example, a lot of companies have a smoke test suite which is does the absolute basic functionality of the site work. Um by product area say we're only dealing with the cart. So maybe we only run the cart test. Uh The biggie here is that tests can be multi threaded, meaning you can run multiple tests at one time in different threads. So um it would be like having several humans simultaneously testing. You can also set your regression suite to run on an automatic cycle. So you can run it nightly, you can run it every time a new um build is completed or every time something is merged. Um And this really allows quality engineers to focus on manual exploratory testing and things that just can't be automated or aren't yet automated. There's a lot of things you still need a human eye for, there's different types of automated testing. We're gonna talk about U I testing um in this workshop, um I'll be doing future workshops and some of the other ones um basically U I testing simulates the actions of the manual user.

So you can write code that opens up a browser, navigates to a website, types things into fields, clicks buttons and then asserts if certain things are present, not present or in a certain state. Um There's also api testing, native mobile testing and load testing. Um I've included some example frameworks. Um There are different frameworks depending on what features you need in your testing framework, what language you want it in. Uh These are examples of front end automation frameworks. Um And this is not an exhaustive list, but this kind of breaks it down by programming language. But one thing that you'll notice is that you can use any of these languages with Selenium Webdriver, which is why we're gonna talk about that specifically today. Selenium is kind of like the burger and fries or the chicken strip of test automation. Um It's very well known uh if you know how to um use Selenium Webdriver. A lot of these other testing frameworks are built on top of it. So a little bit about Selenium Webdriver. Uh It's been around for a really long time. It was created in uh 2004. Well, Selenium was created in 2004 by a programmer at thoughtworks. Um It was immediately open source, which is another reason it's popular. Um In 2011, Selenium 2.0 was created. It's a combination of Selenium and then Webdriver, which was another project that a Thoughtworks engineer was working on. It works on Windows, Mac Linux, multiple programming languages. You can test the multiple browsers.

And as I said before, many, many other test frameworks sit on top of Selenium. OK? So some basics on how Selenium Webdriver works. Uh Basically your test commands are converted to an H TT request and that's sent via Json wire protocol before executing any test cases. Um Every browser has its own driver. So you can see there in that browser driver, there's Firefox driver, chrome driver, um edge driver, um whichever drivers, whichever browsers you're gonna wanna run your test against, you'll need the corresponding drivers and then the browser starts receiving the request through its driver. OK?

Um So the one that we're gonna be talking about today, we're gonna be using Selenide, which it's a Java testing framework that sits on top of Selenium Webdriver. Um There's a lot of java isms and a lot of work with Selenium uh Vanilla Selenium in setting up the drivers, getting everything instantiated and cen I just kind of takes care of all that for you, which is why um I prefer to use CEN. Now it just takes a lot of the pain points out of vanilla Um Selenium test NG is a uh framework that makes selenium tests and reports easier to understand. Um Test NG is also really great for organizing and grouping your tests. So I preferred to use Java before cyanide came out just because I really preferred working with S and G. And some of the things we're gonna be discussing is the page object model that's kind of the best practices um way to structure your testing framework. I'll demo working project. We'll talk a little bit about how C I continuous integration can improve quality of your company, different third party services for browsers and devices that you can use, how to get started and where to start including how to pick a framework. And then I'll also give you some additional resources you can use. OK? So the basics of the page object model is that every U I view in your application is going to have a page class and that page class can be connected to one or more test classes.

So for example, log in, you're probably gonna need to call that page class in multiple test classes just because you're gonna log in for pretty much anything if it's like an ecommerce site with a login. So your page class, you're going to define any of the elements you're going to be interacting with in your tests and you're going to use CS S or X PA selectors also in this page class, you're gonna write all of the methods that interact with or query those elements. So this is when you'll define your buttons, your fields, you'll write the methods that will click things and intertex into things and um verify things are or aren't present in your test class. All you're going to do is call those methods and make assertions based on the expected behavior. There's gonna be very little original code in your test class. You really want all of your logic to be in your page class. The test class should really just be calling page object methods and adding assertions and you might have some um some ways of handling data in your test class. Um Selenide syntax. So if you look here, when what you're going to be doing with Selenide or pretty much any of your page object code is opening the page, finding an element and doing something with it and then checking a condition if you look on the right.

This is a little excerpt from the selenide cheat sheet. Uh I have this linked in my um resources at the end of this, but you can see it's fairly basic. There's how to navigate. Um you do set a base URL, you can do open, you can do switch to if you're switching to different frames, how to locate your elements. Um Those of you that are familiar with Jquery, this will look very familiar single dollar sign for a single element, double for double, you can see for clicking on things. It's literally just dot click or dot submit. You can even do a dot M double click setting the values within text fields, getting selected texts and then querying um things like is displayed or is not displayed. So I'm gonna show you now a working project. Let me change my chair here. We are going to do. Yes. OK. See if I can make this bigger and I'll zoom in a little bit. So this is my page class right here. Now, I'll just go ahead and show you one at a time. That way you can see everything a little bit better. So you can see at the top I'm defining it's my page class. Um These are where my elements live. So I've got the see all tools button, search button, search field, search results pop down. These are my page object methods.

So searching for item, I'm literally first asserting that a button should be visible, clicking on it, the field should then be visible and then I'm going to send the past and string into the search field down here. I'm just returning the pop down text from this element and then I'm clicking the see all tools button. So as you can see element definition, everything that's dealing with the elements. When I go to my test, I'm importing my page. This is a data provider. So these are two different search terms that I'm going to send into the test. So it will run the test one time for each item in this data provider. This is all you have to do for cyanide for setting up cide handles all of the importing of drivers. Um Pretty much all of the setup that you would normally need to do yourself if you were doing Vanilla Selenium. So this is extremely easy. I can set my browser size if I wanted to do mobile web, I could just determine the size of my ipad screen or whatever and I could set that. So that's a really good way of mimicking mobile web. This here just navigates directly to this o this URL and opens the browser and this is my test. So you can see it's just a test named search. I'm passing in a search term, the search terms I'm pulling from the data provider. I showed you above.

I'm calling my page object method of searching for an item. And then all I'm doing is asserting this is also a page object method where I'm getting the text from the pop down. And then I'm just doing an assert that it matches the search term that I passed in. And then down here, I'm just asserting that I am able to navigate to this, see all tools page and that I'm able to see the title that I'm expecting. So pretty simple test. If you notice these groups, this is a test in G functionality where you can create different groups and you can run just groups or you can run the entirety of the test. So I'm gonna start by just running this. So you can see it. Excuse all my red output. This is all just logging that I haven't hidden yet. Do you see the browser opens? Oh You can't, you guys can't see the browser. Let me fix this. I'll run it again here in a second. So you can see the whole page. So, so you see over to the left down here, my test pass, I am gonna do a full screen share just so you can all see the uh all of the windows. OK? I'll do that again. This time, you can actually see everything. There you go. So you can see the browser opening, it's entering the first search term and then it's entering the second search term. And you could see that was super fast um automation even if you're not running it multi threaded moves a lot faster than a human being would.

So you're able to test a lot more efficiently. And again, as I said before, you can see on the right hand side, you could see the search, the search and then it has my search term in parenti. So I can see which one it is. And then there's my search with testing G as my second one. So you can see what it is. OK? So you've seen what the automated framework looks like basically. And you heard me talk about continuous integration, but let's talk about what that looks like. So we have this entire repository of automated tests, what will typically happen or what can happen at various companies, depending on how your company likes to do their DEV ops, you'll commit the change to your build, that'll trigger the build, your um app will build, the build will pass or fail once the build is done.

So if you're merging something to master or you're deploying things to a different environment, the test will run against that. You'll get a test report. If the test passes, the build goes to staging and is deployed to production. Some companies like to have this cadence more frequently.

A lot of companies will run regression every night against whatever is in master and they may run just smoke tests every 15 minutes or so. Um If it's a smaller company and there's, you don't have a ton of tests, you may run it against every commit. It just depends on what your company wants to do. Uh different ways that having this in continuous integration can improve quality. You can run your regression on a schedule, however, you wanna do it but releases nightly, et cetera. So you get to see the bugs sooner. You can get a report every morning. Look at the bugs immediately. Non-technical, stakeholders can trigger jobs manually. They can go into Jenkins or whatever you use for your C I and they can just press a button to um kick off another job, you can run your subsets. So again, if you're only worried about a very specific portion of your app, maybe it's super buggy, maybe there have been a lot of hot fixes pushed in. You can create groups with testing G and just run against those groups and you can use various reporting libraries for charts, graphs and logging. Um My allred output with my two little check check marks wasn't super readable. So this is an example of allure. It's a very popular reporting tool. Um You can see at the top you can click on Allure report and then you get this beautiful chart with pass and fail and all of the different test suites that you've created.

You can see the environment you can click in and see console output. You can even set it up so that it takes screenshots upon failure. So someone who doesn't know how to write code, a project manager can look through and see the screenshots of what was failing. And there are tons of different tools, but this is basically how they all work together. You plan, you do your continuous feedback, you operate your deploy, you run your tests, you build, it's just sort of a never ending chain. And you can see there's so many different things you can use depending on the tech stack, your company likes your continuous integration may look different, but this is kind of the basic process of it. So you remember me earlier saying that we might need to test on different browsers and screen sizes of browser versions and mobile devices and all sorts of things. There are tools that you can use to do that. Now, the top three in this list are all paid third party tools and then the bottom is something you would need to build yourself. So Sauce Labs, browser stack, Perfecto there's more all of them basically host either real devices or simulators. And when you have a paid subscription with them, you can run your code and it will also run it against whatever browsers devices versions.

OS S you select Selenium grid is something that you would set up or you could do this yourself, but you would need all of those devices and browsers. So for example, if you wanted to run against Windows and apple, you would need to have a Windows VM, you would need to have a Mac mini. If you wanted to run it against iphone and Android, you would need to actually have those physical devices. Um And again, some companies, they do it themselves with the grid and they have their own devices and some companies prefer to go the paid route that um screenshot there on the slide is an example from Sauce Labs. So you can see on the right all the commands that were run, you can view the logs, the metadata, it actually records the test so you can watch it in real time. Super helpful. But again, it's a paid tool. So this is all sounding great. And you're just kind of wondering where do you even get started? There are a few different steps that you need to take and things you need to think about. Um you need to figure out what exactly you want to test and what the best way to test it would be. It doesn't usually make sense to automate everything. So figuring that out is something that your team would need to do.

Um Soon after that, you need to figure out what tool you're gonna use and we're gonna talk about that more. You may need to do a few different proof of concepts play around with it. See which one works best. Uh After you design your framework, there's lots of different design patterns, page object model is kind of standard, but there's lots of different ways you can pull your data and you can incorporate API calls into your automation. If you want to getting your environment set up, do you wanna use Docker containers? Are you gonna be incorporating aws? Um Are you going to be running this against a single environment or multiple environments? Your test data? Are you gonna be using real data? How are you gonna create that data? How are you gonna clean the data up? Um There are test data generation uh tools that you can use. Um some companies use API calls to create test data on the fly, developing your test scripts. So actually writing the test. The nice thing about the page object methods, once you write them, you can call them from any test. So you can easily write code that interacts with things and then any anywhere in your test, you need to interact with those elements, that code is done. So usually the more code you write, the less you end up having to write in the long run, executing your scripts, any debugging, gotchas, making sure that they work, analyzing your results and then just supporting it and maintaining it.

Once you get your base automated, you can include automation as part of the definition of done for depth tickets, test the ticket, write the automated test for it when both the automated test and the new code is merged, that ticket can be considered done. But these are all things you would need to discuss with your team, your company, you may have existing automation that you may wanna clean up or you may want to keep the design patterns you're already using when it comes to things like deciding a testing framework. This is extremely important. I've worked for a few companies where they pick a framework, they build a ton of stuff out and then they decide or they discover that there's just a huge pain point. But at this point, there's so much in there that the cost of completely ripping it out and starting over again is just too great. And then they're stuck dealing with the framework that is a pain in the butt to use. So think about who's going to be writing and maintaining these, if you're gonna have devs develop a framework and then you wanna turn it over to an automation engineer that might be problematic because devs and automation engineers learn different things. Um And we tend to construct our test differently. If you have um developers are going to be writing and maintaining, you probably want something, they're going to be comfortable using. Uh nothing's worse than having to contact switch.

If you're gonna be hiring somebody, you wanna keep that in mind as well. Testing needs. Do you just need web? Do you need mobile web? Do you need native web? Do you need to do multiple tabs and browsers? Uh Does your app have pop ups? Things like that are all need to be considered because test different testing frameworks may or may not support some of this framework, maturity and longevity. There's a huge tendency in the tech world to say, oh, I read about this brand new thing and I think we should use it or Facebook just developed this new thing and I think we should use it or I use this at my last company, I think we should use it. There are a lot of frameworks that just haven't been around long enough. And if you get stuck, there's not going to be a lot of resources. Plus, if it's not super mature, there may still be a lot of features that you don't realize you need that that framework doesn't support. Uh Protractor was a very popular framework that just got deprecated and they routed everybody that was using it over to uh a different framework.

So that's definitely something that you want to think about. Um When you choose your framework is if it's brand shiny and new, there may not be a lot of support and there may still be a lot of functionality that um doesn't exist yet, flexibility and customization. Um Everybody loves Cypress mostly because it's shiny and new. Um Cypress was actually a test framework written by and for developers. Um It's very similar to an iphone. It basically took all of the things that they assumed you'd need and they built it into the framework and that's what you have to use. It's not something where you can kind of pick and choose what libraries you want. Also, you need a um an understanding of the development code to use a lot of the features. So it's not super flexible or customizable, the size of the community and the ease of finding support and resources. One of the nice things about Selenium is it's literally been around for 20 years. Um almost 20 years. So the community is huge, finding support and resources is super easy. Selenide just is a library that sits on top of Selenium. So you can still use all the Selenium functionality. Selenide just makes it easier. But you're not going to have to worry about not being able to find information you need or not being able to do something.

It may not be the easiest way to do it, but it's a pretty safe framework to start with. Oh, did I get everything? Yes. Ok. So if all of this sounds good, um Some of the next steps you can take, I do have a tutorial for the demo code. Um There's also an additional selenide in testing. G tutorial that I found very helpful. There is an extensive Cellan wiki cen documentation cen cheat sheet. Uh I will, I did submit the slide deck. Um If there isn't a way for me to get this to you, uh You guys can just message me, I'm gonna drop my name in the chat and I am gonna do questions. This is my email if anybody up that should be.com and also just in case anyone is looking for a job, um Fanatics is hiring uh I have to say again, being a, a woman in tech, it's not always easy finding a tech company that's extremely passionate about inclusion and diversity, but this company really is so uh I can highly recommend it.

Um But Yeah, I'm gonna go through here and answer some of these questions and chat and if anybody has any more, if you wanna see anything closer up, let me know. So let me see. Yes, the little ghost browser that is the magic. Um When you do this in your test page, when you do this open, it literally pulls up a browser. Uh by default cyanide uses chrome, but you can configure it so you can do it in Firefox or edge or whatever else you want. But it literally just opens the browser for you and navigates to uh whatever URL you put in here, you can also put in a base URL and then you could just add, you know, whatever path you wanna go to from that base URL. Yes, I will share the presentation going here and regression. So, regression is just the name for making sure that everything that used to work still works. Basically, it's like going back and being like OK, cool. We just added this new widget to our site. But did we break login? Did we break search, did we break checkout um prod testing. Sometimes you can do it. You wanna be careful with prod because uh it's live data. Um Most companies I've worked for don't have automation in prod, but some do a lot of times in prod, what you're going to be doing is stuff like searching for things, adding things to cart.

Um My current company, we don't have automation in prod, but we do do live testing in prod. So we actually do order things, real things with a real credit card and then we just go into our system and cancel it afterwards. OK. Thoughts on how to juggle demands between different tools. Focus on each one, struggling being proficient with a single tool. Um Honestly, what I did when I was first started, I was very lucky. My very first quality engineer job was Java Selenium and Java Selenium is probably one of the most popular. Uh Selenium is probably the most popular testing framework. Java is probably the most popular language to work with it. My next company used javascript with uh web driver IO which is something completely different, but the nice thing is locating selectors and defining elements is gonna be the same, the page object model where you're gonna have a page class and a test class is gonna be the same.

The only thing that's going to be different is the syntax of your tests. Um One thing I would recommend is if you have any say over it, do not use a javascript testing framework for U I testing. And the reason for that is because javascript is asynchronous and U I testing is synchronous by nature, you need to do something and then immediately after you do it, you're going to check something or you need to do another thing. Um So I prefer synchronous languages and the frameworks that go with them. Um Selenium you could use with just about any programming language and there is a tutorial for Selenium with any programming language. I prefer Java because I love testing G but I have done Selenium with Python with C# um I've done it with Vanilla javascript. Uh Java just offers the most helpful libraries because it's the most common. And even those of you that don't use Java, you can see there's not a ton of Java ISMs in here. You might like you pretty much everything is public for the most part. Um You might use protected for some of your data files. Um You need to specify your elements, but aside from a few little things, it's not super Java heavy. Let's see. Awesome. I love this sounds interesting. I do have a linkedin page. OK. So the record and play automation tools, those are, they're tricky.

So, playwright is a framework that is supposed to quote unquote. Um You're supposed to be able to record and then it's supposed to be able to generate code. I don't know if you guys have ever used like text to speak. Um I've tried tools like that and the amount of time it takes me to go in and debug and adjust everything I could have just written it myself. Uh I have yet to find one that works as well as it's supposed to. One of the things you can do with selenide too is you can capture screenshots of your tests so that it'll take a screenshot upon error. Uh And there are tools where you can record the browser and things like um sauce labs. Some of those C I tools that I told you about, they also record the tools. Cypress has a recording tool. But again, with Cypress, I was fighting that framework so much. Um because again, it's very obviously written by developers for developers. And a lot of the really cool things that you can use in Cyprus, you have to know you have to be able to interact with the developer code. And as a test engineer, I don't want to do that. I wanna come at this and see the site and interact with it the way a user would. So I need to interact with what I can see in the DM.

I don't wanna have to like go into a class and call a method to get my U I into a certain state for developers. That's great. I would highly recommend Cyprus if your developers are going to be setting it up and using it and maintaining it. But I would uh I would recommend a Synchronous U I framework which means avoiding javascript. And uh P test is basically sits on top of Selenium, Python. Um Capybara is sits on top of Selenium with Ruby Cen, sits upon um Selenium with Java uh ex PA uh X PA, you can do it. So it used to be, to be able to find things by the text they contained, you had to use X PA. But now um cen and other test frameworks are getting better at coming up with um CS S that allows you to do everything you used to be able to do with expat X PA. You basically have to traverse the dom to find your element where CS S you can go directly there. So think of X pa like written directions to someone's house and CS S is like sending uh a Google Maps, share my location link. So it's a lot more direct and it's a lot quicker. Yeah, I hate Cyprus. I really do. Um I do have a linkedin page. Let me send you guys my linkedin. I can also drop the uh Google slide deck here into this page, exit presentation mode here.

Just share this. OK. So in the chat, there is the link to the slide deck, get to my linkedin, send this to you. But yes, in linkedin, if you go to it's literally Q A automation engineer. Uh there's a lot of different words for this. Um But you can see there's Q A automation engineer, this might also be called S de uh software engineering test. Um It might just be quality engineer, it might be automation engineer, it might be Q A automation engineer. So there's a lot of different ways uh They may list this job. It kind of just depends on the company what they call it. Um So it could be, I'm gonna put et lots for se software development engineering test. So yeah, this is all Q A automation. The there's a massive demand for this but there's not a lot of people doing it because most people that know how to code are developers and most people that do testing are manual testers. Um And especially after the pandemic happened and every company with a website started, you know, the website started crashing or bugs started being reported because everything went online and people were using online tools to an extent that they hadn't before. There's a massive demand to get uh automated tests in place. And the beautiful thing about it is you get to write code every day and when you find a bug, it's not your problem. You just tell the developers that their stuff doesn't work in chrome.

And then when they tell you it's fixed, you press your button and run your test again. So highly recommend uh it's a great field to get into. You get to pick things apart. Testing. YYYY. Yeah. What I would do is um Anna I would create, if you don't have existing automation, I would just create a proof of concept framework. Uh Take some of the things that are the biggest pain points. A really good thing to automate. First is your top navigation U I styles now. So that's one of those things that you need humans for, if all of the sudden this linkedin page, all of the text turn green. My automation wouldn't have a, have a way of figuring it out. Uh There is a type of automation that's like visual comparisons where it can take a screenshot of things and then it, it basically takes another screenshot and compares the two. Um I still think people are better for that, but that's definitely something that um you could research if you wanted. But also like the the codes for different colors are different depending on browsers. So uh but that's the nice thing is if you automate as much functionality as you can, then you can be like huh this, this color looks a little bit off or this, this this looks like it's a few pixels to the right. Spying on dead tools you can.

But again, if you look at, if you look at the um like the CS S in chrome and then Firefox, it still looks different. And again, there are specific tools. So let me see, visual testing automation frameworks, there are frameworks that like that's what they do, visual regress or regression testing tools. This is what you want. Um What I'm mostly talking about is um actual um functionality, right? But yeah, just Google um visual automation, visual visual testing tools. And uh if that's what you really want, if your biggest pain point for your company is your pixels look weird and your, you know, CS S is off or your things aren't displaying correctly then this is what you want. Um What I do is more about functionality. It's more about things like, oh, when I change the language to Japanese, um all of a sudden I can't add things to cart or I'm able to submit more than the maximum amount of characters or when I enter in no user name and no password, I should get an error message popping up. And I'm not, that's the kind of thing that the test I write do and then back end testing as well is um things like, you know, getting a 404 and stuff like that.

Um You know, if I hit this post with an empty request body, I should get this error code in this message. Anything else, please feel free to reach out? Like I said, I put my email, my linkedin and the slide deck into the chat. Grab those if you need to. Um Just last week, there was somebody that was, had their first Q A automation interview and we did an interview prep session. So I'm more than happy to help anybody that needs help. Um And if you need more resources or um I've, my last job was in consulting. So I have U I automated test framework, PO CS for a few different languages so I can send you a repo to get you started. If you wanna play around any other questions, automation mentors. Yeah, I would say if there's someone at your company that does it, I would reach out to them just because they can tell you about the tech stack, the tools that your company uses. Like I said, I am absolutely happy to help anybody. Um If you wanna reach out to me, I can get you started with some tools and tutorials. The, the thing I would recommend the most is pick an automation tool. Um And then just write automation that just goes through and clicks through your top navigation and asserts that it takes you to the correct page, stick that in your github repo add Selenium to your linkedin and just watch all of the messages. All right.

Well, thank you guys so much.