Good afternoon, everyone. I'm Federica Neera. I'm a principal software engineering lead at Microsoft in an organization called commercial software Engineering.And I'm really excited to be here today to talk to you about the code with engineering playbook and share some of the lessons that we've learned from our customer engagements that have really inspired and led us to create the playbook. So what is the code with engineering playbook?

You might ask well to fully understand how and why the playbook started and some of the sort of essence behind it, it's really important to understand really the department, the organization that I work for is part of commercial software engineering at Micro Soft. We work directly with large companies and as your customers to help them tackle some of the most significant technical challenges. Um This means that it means that we work side by side with customers and their engineering teams and basically sort of integrating ourselves into one team with them to scope design and execute on projects. We're a really global team and we've grown significantly in the last few years.

So that's, that's sort of also additional context. So during our time together we'll be discussing the development of the playbook. I'll go into some select tips and tricks that hopefully you might be useful to you all and can be found directly in the playbook. And finally I'll discuss, you know, how to make the most of it also a small note before we get into the content. Um, although there will be a bunch of QR codes that I've added to the presentation, um, and also plenty of memes. I'm a bit of a meme fan. Um Please don't worry if you don't manage to sort of scan the QR code, all of the links will be shared in the chat at the end of the session. Um So enjoy the ride, the development of the playbook. Why, why did it even come about? As I said, we coen engineer with our customers um on projects that can vary in length between 3 to 6 months. And as you can imagine, you know, there are some challenges that we've encountered being such a global team. Um For each new project, we have to basically re-establish an engineering team with DEVS machine learning engineers, customer uh engineering teams as well as needing sort of the customers to be able to continue iterating. Um And really owning the solutions that we coen engineer with them um after we leave and this is where the playbook came in.

It started as a centralized way for us to collate together our learnings um and share them to suggest new, you know, not best but suggested practices with one another within the organization as well as upskilling customers and really taking them along the journey of learning with us um side by side and quickly transferring knowledge between different teams and ourselves.

You know, particularly we do tend to use lots of different technologies. So it's really a place to document and evolve our understanding and foundation around core engineering practices. If any of you are involved with cloud work, you'll know that Azure and Aws and you know all these these cloud providers that the products that are being offered are evolving so quickly and so rapidly that it's really important to, you know, keep iterating and keep updating and always sort of striving to know what the you know, what the best practice is to implement something.

Um So with so many projects and so many different technologies, we really see a huge variety from data lake solutions to migrations and redeployments of apps, re architecting of apps from on prem to being cloud native integrations with teams, machine learning model deployments or even autonomous driving solutions around internet of things.

The variety is, is is really huge. Um And some of the projects we work with can have steep learning curves and some that come mind for me are around uh machine learning. We have seen that customers tend to be early in their journey of integrating machine learning fully into solutions. And into their businesses. And based on these repeated experiences around machine learning projects, we built a section in the playbook around ML fundamentals that I'll talk about a little bit in more detail later um to help kick start customers on this journey. And this includes embracing the principles of responsible A I um to ensure ML is put in place in a responsible manner. So now let's dive into some essential sort of tips, tricks and uh suggestions that have proven useful. At least to us in CS E across many customer uh projects from the playbook DEV experience. Um As you can see in the meme, I mean, who hasn't heard someone say oh, but it works on my computer. Why doesn't it work on yours? Um It's one of those perennial jokes in um in software development and it really is one of the key areas that can be a pain point with projects like ours that start up fast, integrate developers from, you know, all kinds of different areas with familiarity in different languages that run different machines and tend to be projects that we try and integrate or have a sort of mono repo approach because we develop both the solution code and the infrastructure code needed.

So we really need the DEV experience to be as seamless and as easy for new devs to contribute to the solution. So what do we do? How good is Dev X? Well, we always make sure we have a clear onboarding guide. Um And obviously, the more you can streamline the dev experience, the simpler the onboarding guide. Um So that kind of, you know, feeds itself, we've also found that two metrics can be useful in judging and evaluating, you know, whether you've been successful in setting up a good DEV experience. And that is the time to first commit. So how quickly can you get someone up and running to make their first commit? And secondly, uh time to first end to end result, this is where the concept of an F five contract emerged. And this is great because it developed through an engagement that we had with a really large automotive manufacturer. We set up vs code launch configurations and tasks to allow the solution to run in debug just by clicking F five. which to be honest was truly life changing, everything being up and running at the click of, you know, 11 keystroke and I'm going to talk a little bit more about the S code because we do really, we love to use it in our engagements.

So DEV containers um like I said, we often use VS code and it has really good support for DEV containers which really make it super easy to spin up local development environments with all the dependencies just through specifying the DEV container. So what are de containers? So development container allows DEVS to package the local development tools stack into the internals of the docker container. This is done with just two files. Uh really the docker file and the DEV container JSON, the DE container JSON orchestrates the artifacts needed to support the base image VS code UI on the other hand, remains local, but all the files that you're working with are volume mounted into the container. And this is all possible. Thanks to the great VS code, remote desktop uh remote development extension pack. Um You can find it in the links later, simply using a DEV container, makes onboarding significantly faster and removes inconsistencies in DEV experience within the team cloud solutions.

Sometimes testing can be challenging. So we do try for things that can't be run on a local machine to sort of swap that dependency out for one that can be run locally and creating an abstracting layer that removes that sort of remote dependency and sort of toggle between the two extra extra points. All right. So we've talked about DEV experience. Um you know, and apart from really working together with a seamless developer experience, another area that we've had to adapt and learn given our variety of projects is really agile. Um And you know, I'm sure you relate to this meme of everyone sort of thinking like they've got, you know, totally they're all over agile, they know what it is, they can do it well and, and customers feel like they have a pretty good grasp. Um However, you know, it can vary a lot from customer to customer. Um You really never stop learning how to do it better and it really evolves with every new challenge and every new customer that we work with. Um And one thing I want to talk about here is sort of focusing a little bit more on the category of collaboration during agile.

And so I, you know, I definitely recommend starting any engagement where you have to pull a new team together um by setting really clear expectations right at the start uh through a working or team agreement. Another thing that we've seen work really awesome is having a rotating agile facilitator or process lead instead of relying on one person to act as from master. This means everyone gets a chance to run the process, including the customer engineers and everyone else. And this helps everyone to learn and foster not only a sense of learning, but also of ownership for the process. Having a new customer team to work with also means that we have to sort of build those key relationships and trust between one another. These projects can really be, you know, a bonding experience uh back in the day when we used to travel and go to the customer site and meet in person. And we all know that being in person does really accelerate relationship building. So, you know, what have we done in this past year? How can we do that from a remote setting since the pandemic. Uh, we've, uh, started using a social question that we have added to stand up and everyone gives an answer. Um, additionally to their usual stand up update, um, things could be, what's your favorite Starbucks order or Glaze or Apple Cider donuts or even something, like, would you rather live under water or in space for a year? Um, now everyone kind of listening into the presentation.

Um, you know, what would you prefer? Um type it in the chat. Uh For me, it would definitely be underwater. And to be honest, the expansiveness of space has always sort of scared me. Um So please share so that I can get to know you all a little bit better. So with these apparently sort of silly questions, it's actually very easy to kind of slowly day by day, stand up by stand up, start to get to know one another better and it helps to spark of ongoing conversations that you can pick up after. So I definitely recommend that you try it continuing on the theme of collaboration, given the sort of essential need for both Microsoft and customer engineers to get upskilled in technologies through our projects. Um We find that pair programming and swarming when deployed in the right way, really helps to encourage and grow collaboration. This is especially useful when engineers bring diverse knowledge to the table, for example, an expert in data pipelines um versus you know, someone who's an expert in uh data lakes pairing is more helpful when the engineers do not have perfect clarity about what is needed to be done or how it can be done.

So it helps to have some uncertainty in the user story that one is pairing on. Um I guess somewhat in contrast with uh with the hello world example that we see here, swarming is done. On the other hand, when two engineers assigned to a story, need an additional sounding board or need expertise that other team members could provide and other help that could be provided or when a story is particularly complex or at risk of becoming a large blocker for others in this way, sort of to help with collaboration.

Our top tips are to um you know, set up obviously a voice or video call when remote um and to ensure that the pair come together to decide how best to work together and how to break up the story, either sort of sharing screens via teams call or using collaborative tools such as VSVS code lives share.

And I've got the little QR code for the VS code lives share extension that lets you sort of share your VS code ID, you know, remotely without sharing your screen, which is really fantastic for pairing work. An additional thing that we've done recently in projects is tailoring the product backlog item or user story template to make sure that the card layout shows both engineers, this helps increase visibility of pairing during stand up. Um And on the Kanban board helping to drive greater accountability by both engineers. Um And as you can see there, this is a, an Azure devops uh card. Uh And there's a recipe in the playbook on how to uh edit and amend these backlog items in in Azure devops. So check it out a final additional methodology that seems to work well with teams that are remote is red team testing. Um where one developer writes the tests using the public interface and attempting to perform, you know, edge case testing, input validation and stress testing the interface. While the second developer is simultaneously writing the implementation which will eventually be tested.

We definitely couldn't have a project without considering continuous integration and delivery carefully. Everyone's sort of hot topic, I guess and it's obviously a huge, huge area. So I am only absolutely scraping the surface and there's lots more in the playbook. But one thing that we've learned is that it's never too early to have an automated and repeatable C I pipeline um integrated within the project with all the appropriate steps, checks and even before the code is written, um this reduces code integration errors and maximizes the velocity of the team by removing the need to remember checks by the developer.

And we all really like to be kind of lazy developers in reality and have plenty more to worry about you know, that can be automated by the C I. And this um you know, the things that we, we try and integrate um are, you know, always maintaining sort of AC I pipeline definition within the source code so that it's version controlled, ensuring testing is integrated using as many code style checks as is possible and integrating security checks as well as part of the overall devops process.

We also try and integrate to commit hooks to try and have some of that testing be sort of more local and easier for the developed for her to adapt. Because the last thing that we all want is to kind of push up big commit for it to fail on the C I. So definitely recommend pre commit hooks too and making sure they run as quickly as possible. And this is the sort of template or the um check check marks, I guess um of some of those uh to have in all continuous integration projects. Sorry. So we always try and also define our architecture through infrastructure as code just like we try and have our C I pipeline definitions um within the source code. An infrastructure code is really something that we live and breathe in every single project. We try to manage as much as much as possible as code and make extensive use of terror form for building our different environments. This makes it truly repeatable. It's also really easy to audit changes made to the infrastructure or roll back to known good configurations. We try to make all cloud resources that are provisioned uh through code templates including secrets service config settings, even role assignments and monitoring conditions.

Uh through the CIA we can then build and validate uh the target infrastructures, environment's current state um with the expected state and use task integrations in our pipelines um such as the ones listed on the slide. So we use Azure devops pipeline Terraform tasks which happens to be written by someone in in CS E um and also forget hub actions, the setup terraform um action and these help to run with terraform co I. So a final area that is near and dear to my heart that I wanted to talk us through. Um As I have a background in both machine learning and software development is the machine learning fundamentals, which I alluded to a little bit earlier, we've come across a large number of engagements where customers want to adopt all the ML best practices and they sort of come to us and they're, and they're like, tell us, teach us everything, but sometimes there can be roadblocks along the way.

Um And when it comes to A L, it really does all start with having the right data um preferably labeled and then bringing a little bit of engineering fundamentals to data science. We put extra emphasis on data quality and proving out the feasibility of a model. Um to gauge what is possible with an ML solution um sort of different from the approach that these two did with the with the big data pile. Um And to help with all of this in the playbook, there is a super handy ML fundamentals checklist that helps to work through all of the stages of developing an ML model to determine whether you're ready to move forward through the various stages of the development process. Another thing we always do is to ensure that every project undergoes a responsible A I screening and we encourage the use of tools such as these packages called Fair Learn or interpret ML that really help with understanding and questioning of the outputs of ML models to make sure that their behaving is expected.

And here we have our ML fundamentals checklist, um you know, with those points around data quality um and and sort of feasibility of the model. So overall insights sort of from from this whole journey of the engineering playbook. Um The engineering playbook has been a huge effort from a diverse set of people within the organization um with lots of different backgrounds expertise. Um And so I wanted to share four big points um that we've seen along the way. The first one is feedback as part of our learnings and efforts, we always try to share feedback directly with the product, the Microsoft product teams to drive improvements. Um And when you know, and when things can't directly be sort of built within the product we try and open source and share any useful assets that we develop through our projects ever changing. It really is true that the playbook is always a work in progress because we keep updating and adapting as we keep learning more and wanting to sort of extend both our understanding and our, our our sharing. Um So, for example, you know, there were two new sections added recently. Um One was around uh documentation guidance and one around inclusive code reviews.

And it's really this continual development, that also means we try and link as much as possible to external repos when there are already very good sources of information that are managed externally. And we really try not to replicate existing resources when we do already have go tos different perspectives. So coordinating some of the effort around the playbook, you know, the effort has certainly taken time and practice a huge number of people over 130 people contributed.

So you can imagine how many opinions there's been over all the contributions and the direction of the playbook. And as you all know, engineers love to be opinionated me included. Um So there's been many discussions to find the right balance between integrating useful snippets of code and making it sort of maintainable and sustainable into the future. Um Guide versus law. Uh The playbook is intended to be taken as a guide, you know, it certainly is somewhat opinionated.

Uh But for every project and scenario, it's up to the user to pick and choose what's relevant based on engineering, maturity and project scope. And also to give back when there is something that you could do with a refresh or an update. So I wanted to give a big thank you to all the contributions and it has really been a huge effort. Um And um now that I sort of shared some of our recommendations from the playbook, I wanted to leave you with an open question. Um What do you see as a pain point or an improvement area around engineering fundamentals in your projects? And what is one new practice that you can adopt to see if it improves? Thanks, questions. All right, just catching up on the chat. Give me a second. How do you manage um All the different ideas of what should be in the playbook. Um It's, uh it's definitely a challenge. We have a great team of folks that sort of lead one of the different areas. Um And we all come together and we each, every two weeks and we try and brainstorm and come up with new ideas and new directions, but it's also really driven by everyone within the department contributing as they're finishing projects and sharing the most up to date learnings that they have.

So I see there's a question around why the shift from machine learning to software engineer. Um I gladly answer that. Um to be honest, um what I saw as a, as a machine learning in machine learning when I was sort of a data scientist, I was working in another part of Microsoft. And one of the big pain points was getting models into production in a way where models were well tested and hardened to the point that they were really reliable. And so, you know, that made me sort of think that perhaps moving to the software engineering side would help me understand and gather some of those learnings and bring them back to machine learning. And we definitely see in our projects a lot more synergy happening and a lot more synergy still needed between machine learning and software engineering. All right. So Jessica has a question around retrospectives. Um We retrospect, um and one of the tools that I would recommend is integrated as an ad in, in Azure devops. Um And that is the retrospective tool. Um And it's actually managed and written by, by someone within CS E. Um And it's, it's a really good tool.

I'll find the link and share it with you. Um And let me share all the links. Like I said, I would, it's too long. Let's see in the chat. What else? Any other questions? Let's see who, who's underwater and who's uh space Jennifer. I'm with you for the water. And Jessica too, I guess, you know, does anyone wanna kind of answer that that question. Um You know, is there any one kind of engineering fundamental area that you see in your projects needing kind of investment and some improvement? Yeah. Is it code reviews, design reviews and the advice for female um engineering career development. Code reviews. Yeah. Code reviews are a big pain. Y you, I, I I'm sorry if I'm pronouncing that wrong. Um I think you're going to have to be a little bit more specific on, on what advice around female engineers, career development. Um You know, generally I would say um finding mentors um and being able to grow connections within the organization um that would set you up uh for potentially useful um you know, sponsorship opportunities. That would be a great way to start. Yeah, design reviews are, are a real, are a real pain. Um Let me share some links associated with that from the playbook.

Um So we, we have some sort of standard templates and sort of standard ways of going about our, our design reviews and it really depends on the size of the component that's being scoped. Um Whether it's something that is sort of project level or something that is very, very specific and on a feature level. And uh going back to Thanks Claris. Any other pain points? Apart from code reviews? How about observ ability or security? Yes, security say it's a real, real pain. We, I mean, I actually work with a lot of financial services customers. Um And so, you know, getting through their, their security processes, um can take a, you know, a good amount of time. Um And really what we try and do is to really think through it incrementally um in one step at a time. Um And, and sort of changing that mindset from, from on prem to cloud, um can be pretty, pretty challenging. Thanks. That's very kind of you. Let me see if I can finally share all these links. Give me a second. I have too many in the chat limits me. Um Do, do you often use um working agreements? Anyone on the call? Um used working agreements before for, for their teams? Ok. I guess it could be kind of like a pack contract. It's really about, I guess aligning on kind of how you're going to show up to work.

So probably similar to a packed contract, figuring out sort of working hours, even collaboration preferences, especially around the time, you know, when we're all remote, it's really important to not sort of overdo the whole meeting burden and uh and, and being sort of mindful of, of the most effective ways um that that people can work together.

Um I'll share the links to the collaboration and question of the day and that's really changed a lot of our dynamics. It's funny that such a simple question can really sort of foster growth and camaraderie. Um You mentioned that sometimes you have engineers changing in and out of projects depending on the stage. Um How do you make sure everyone stays aligned? Um We actually try and avoid that. Um Jennifer, um It's more that at the beginning of engagements, there's lots of different people with different backgrounds and expertise and so aligning on, I don't know, one way of doing agile, one way of pushing the project forward is really essential. Um Where do you get most pushback from customers on the playbook? Um You know, as I mentioned, the playbook is more of a guide. Um It definitely should never be used as a like be all and end all. Um It's really just a resource um to kind of help along the journey. Um And so, you know, I guess sometimes customers have, you know, different ways of, of doing agile. Um And it takes a while for, for them to get, uh you know, get aligned with how we would like to do it, but I wouldn't say there's, there's too much push back.

Um You know, it's surprising how much growth and how much change working together and collaborating and really kind of, I guess demonstrating the difference that adopting some of these practices can have on velocity on productivity and on generally like feeling like you're really working well towards a common goal.

Um So I would say customers are sometimes skeptical. Um We all like, we all prefer the things that we know. Um But they, you know, when you, when you see the evidence, um it's hard to, hard to deny any last minute questions, pain points. Um How many folks have used de containers? I'm curious. Um And have you considered using them that one? How many folks um I guess on the session um have, you know, have, have been involved with uh machine learning projects? Maybe we should have created a poll for that and there are some pretty useful things around source control as well that I'm just sharing. Oh, yes. Um My linkedin is FN. Um Let me get the link, actually give me one second. Um So please reach out a Neil linkedin. Um Yes, that's right. Um Please add me on linkedin. Um And you know, if you ever want to talk about uh the engineering fundamentals in more detail, um Please just, you know, reach out. Um Thanks so much.