Accelerate your API Led Digital Transformation with Azure API Management Service by Teena Idnani


Video Transcription

So, hello and welcome everyone. Thank you very much for joining this session. I'm Tina Anani. I'm a principal engineer at Investec. I'm an Azure certified solution architect passionate about developing and architecting applications, in.net and Azure.In my current role, I focus on uh uplifting engineering across Investec and enabling various teams in Azure as they migrate and modernize their applications to assure in today's demo heavy session. I'll be discussing how companies can accelerate their API led uh their, their uh digital transformation uh using Azure Pass offering which is Azure API Management Services. So a quick look at the agenda, we'll start by trying to understand what is digital transformation and uh how can API S benefit in accelerating the digital transformation for companies? We will also look at some challenges with API based approach and how an API management to the one which we will be discussing today is assure API management. How can it help us overcome those challenges? We will jump into a demo of Azure API Management via the Azure portal as well as the C I CD way. And after the demo, we will look at some useful patterns, you know, when considering integrations between API using API management.

Um After that, I will leave you with some considerations for designing your ATM strategy and some resources for interested folks to continue their exploration of visual API management service. All right then. So what is digital transformation? And why does it matter? A very simple definition?

Digital transformation is a process of change where we live, which makes use of technologies and its innovation and to change the way we do things and communicate, maybe make things more productive, generate new business models from the opportunities that we identify in the digital society.

So to give you a few examples of digital transformation, if you're in sales, then moving from spreadsheets to cloud CRM. It's digital transformation. If you are in hr then moving from in-person training to online elearning is digital transformation. And let's say you're in customer support, then moving from a call center support to online knowledge base basis and self-service portals. It's digital transformation. So digital transformation is soon becoming the first and most preferred choice of companies and customers for doing engaging with the business.

And the reason for that is because it improves customer experience because it allows you to deliver a better experience to your tech savvy customers and also gives you more ways to connect with them. It increases competitiveness. Now, you know the companies in in the industry are not going to wait around to digitally transform their processes. So you shouldn't wait either if you want to be stay competitive and it reduces cost and raises productivity because you know, digital transformation makes heavy use of automation which leads to reduced cost and increased productivity. So what are benefits of using API S and digital transformation?

And for that, I'll tell you my personal story in my previous organization, I was helping a bank in their digital transformation journey and I would often talk to their leaders and try to understand more of their developer program. So what I found out was that they had many staff developers and they also had many external developers, but they had an it stack which was heavily bloated. And it was uh you know, it it included several layers of platform now because of this bloated ID stack and because of the inconsistent developer experience, the innovation, it was limited to just limited, it was limited to certain pockets of the enterprise portfolio. And this is exactly where an API led approach can help you smoothen, you know the bumpy right in your industry. So like API S are not not just connections between systems which are intelligible only to the creator and that too for a single purpose. No, it's not API S are rather products for developers which are standardized and managed to make innovation easier and to increase digital transformation to accelerate digital transformation. So with that in mind, let's take a look at some benefits of using API S and digital transformation.

So as you can see, they lead to easier app development, better connectivity, uh more personalization for different audience, it increases, it helps increasing your brand image adds flexibility, reduces cost increases productivity. And with all of this, it's bound to lead to an increased functionality and eventually a better customer satisfaction. But then there are some challenges with this api led integration challenges like discoverability. So how how how many API S are available and what do they do on boarding?

Let's say I want to request access to an API, how can I request access and how can I learn how to use it security? Let's say I have an API how do I ensure that it is secured against unauthorized access and overload monitoring? OK. Uh How do I know who is using my API and how much and are my API S actually online and available and working as expected? And the life cycle, which is how um how do how do I continuously evolve my API S without impacting without adversely impacting my consumers of the API. So this is where your API management tools comes into picture to solve these challenges. And the one that we are going to talk about today is Microsoft's offering, which is Azure API Management Services. So what is Azure API Management service? It is a central place for designing, publishing, documenting and analyzing API S in a secure environment. You can bring your own swagger definition, your open API definition. WSTL definition for your soap API S, it acts like a fad or a proxy to your backend API S. You can keep like a very big, major advantage is that you can keep all your API S behind a single static IP or a domain. You can apply complex authorization of transformation policies.

There is a user friendly developer portal completely customized and then you can manage API S not just across clouds but also on premises and, and you know, it's not just for one API. So when, when you're employing a lot of API S to digitize or maybe you are adopting a microservice architecture or you're building your business strategy around API S, you need to control not just one aspect of one API but the full life cycle including like tasks such as defining and publishing the schema, maintaining the security, traffic management, uh monitoring, maintenance, analytics on boarding of, you know, uh users to API S.

So you can see that it's it's a whole life cycle for all the API S that are sitting in your estate and as your API management just makes it easy to do all this. So the next thing, let's look at the different components of Azure API management. So there are three main components of Azure API management. You have an API gateway, you have a management plan and you have a developer portal. So API gateway is, it's the end point that accepts your API calls and routes them to your back end. So let's imagine that a call is made to your API uh to your uh API management as your A P management, it will land at the API gateway and it's the responsibility of the API gateway to route it to an appropriate backend. And while it does that, it can also verify your API keys, JWD tokens, certificates and other credentials and even apply a lot of transformations at this stage. Then you have your management plan. Now this is like an administrative interface for setting up your API program. It, it can be used to define or improve your API schemas package, your API S into products, manage users uh set up your analytics and you know, do all those things and then you have your developer portal. Now this is like the main web presence for your developers to you know where they can read your API documentation about your API.

They can even test an API on an interactive console, create an account, subscribe to different API S using keys and access analytics for their own usage. So with this, let us deep dive into a demo for Azure API management and for the demo, I am going to take you to Azure portal. Uh Yeah. So we're going to divide the demo into three parts. In the first part. I'm going to show how can we create an Azure API management instance from an Azure portal. In the second part of the demo, I'm gonna show how can we import an API into Azure API management instance. And in the third part, in the last part of the demo, I'm going to show how you can do both of these things, the automated way using C I CD. So let's jump into the first part of the demo. So we'll go into API, this is Azure portal. We'll go into Azure API Management services. You can see that I already have few existing API management instances. But what I'm going to do is I'm going to create a new one. So it's going to open um yeah, an interface for us. So quite many of these things that you have to fill here are expected, but we'll discuss them one by one, irrespective.

So you first have to choose which subscription you want to deploy your instance to, you choose a resource group where you want to deploy your instance, a region where you want to deploy to a resource name. Now, just need to be a little careful here. Now, this resource name, whatever you add here that forms the part of the API gateway which will then be used to route calls to your API. So um uh so just be careful that will become the part of the week and second, it should be globally unique. So I've entered that now in organization name. Uh this is useful for your developer portal. For now, I'm just going into first here and it expects uh administrator email. So I'm just gonna put my email ID. Now this is where this is basically so that they can send you notifications for your API management instance. Now the next part is the price is the pricing tier. Now this is very important. Now, you can see that there are different options which are available, consumption developer, basic standard premium. Now, consumption tier is lightweight and soulus version of API management service. It is built for execution but it has very limited features. The other tiers which is developer basic standard premium, they also have tangible differences between them which are very well documented in the Microsoft documentation which include features like virtual network integration, developer, portal integration just to name a couple.

Now at an enterprise level, your requirements will truly influence which tier fits your need and cost best. So make sure you review all the different tiers and their you know, do their future comparisons so that you can choose appropriately for the purpose of this demo. I'm going to choose developer next tab you have is monitoring. Now, this is basically to confirm from you whether you would like to link your app insights instance to your API management gateway instance or not. This is so that all the API calls which are routed to your API management instance can be locked. So for the purpose of this demo, I'm not going to choose it. The next part is scaling. Now, scaling is available only in basic standard and premium tiers. Now, developer tier doesn't support scaling and in consumption tier, it is handled automatically. So not going to choose scaling managed identity. Now, like many Azure resources, it's possible to associate a managed identity with um Azure API management instance. Now, why would you want to do that?

Maybe because uh you want to retrieve certificates from an hr key vault or maybe you want to authenticate your backend API using the system managed identity key of your API management gateway. So irrespective you can decide whether you want this managed identity to be set or not.

So I'm not going to set it for the purpose of this demo. The next is very important in the discussions which is virtual network. Now, it's a very common discussion to have with enterprises which is around network integration. Now, from a network perspective, um vnet integration is allowed only in two tiers which is developer tier and premium tier. Now, though developer tier allows us to experiment with the vnet integration, but it's not covered under an nesle whereas um the premium tier it it supports SL A but then it comes with high costs. So you know, if you configure the virtual network option, then your API management instance will be deployed straight inside a virtual network as against a multi-tenant deployment. Let's say where maybe you would connect via private endpoint. So uh we're not gonna choose any virtual network integration here.

We move next to the protocol settings. Sorry. Yeah, this is basically where you decide, you know about your client site protocols, client site transport security back inside transport security. We will just leave the defaults. As is the next option you have is uh to tag uh your Azure API management instance while using tags not going to do that now. And uh once you feel that yes, you have chosen all the options uh for your API management instance, you can click on create and um uh in the back end, uh it will create an issue API management instance for you. Now, if you've chosen a consumption tier, then probably you just take a couple of minutes to create the API management instance. But um since we have chosen develop a tier and even for the rest of the tier, it does take time like mostly around 20 to 30 minutes to create uh the API management instance for you. So this was the first part of the demo where we saw how can we create an API management instance straight from the A R portal? And the second part of the demo.

Now, what we are going to do is we are going to connect to an existing API management instance and uh we are going to see how can we import an API to this man API management instance for that? Uh this is the API management instance that we are connecting to. You can see that uh I talked about the components. So this is the gateway URL. This is the management URL and this is the developer portal. Uh You can see that I have installed a developer tier and uh let's jump straight into API section. And this is where all the API S that you want to publish to API will, will sit here. So let's, let's assume that we want to I to import our API, we want to publish our API to API. You can do it, you can define a new API straight over here or you can create it from an open API definition that you have or like AWS TL definition that I mentioned earlier or you can even create it from an Azure resource like a logic app app service function app or a container app.

Uh For the purpose of this demo, we are going to use an open API specification. So what I have, what I've done alongside is I have an Azure function which I have uh deployed inside a function app. It has this one single operation. So this is the function app that I have deployed and it has just one single operation here called hello. Let's take a quick look at uh what this function does. It is, it's a very simple function. It's a get uh method it expects and it takes a credit parameter. Let's uh which is called name and it just takes any name of it. And let's see what happens when you click run. So uh when you click run, what you get is a 200 OK response and a content here saying that hello Tina, this H A TB trigger function executed successfully. So this is just very simply what that uh function does. What we've also done is we have exposed an open API definition for this uh function using an open API extension. So this is the, this is the swagger, this is how you know, we have exposed a swagger for it. And uh let's, you can see that it has just one operation. Let's go and grab its open API definition which sits here. I'm gonna copy this link, go back to my API management instance.

And over here we have this create from definition, I'm gonna choose the open API option. So when I choose that, it asks for a path to the open API specification or a file, I already have the path. So I'm going to past the part for the open API specification. You need to give a name to the API. So let's just say hello API, we need to give API UR suffix. Uh This is very important because um this API management service, it's, it's like a shared service which will be used by all your teams to deploy their API S into API. So this API UR suffix it, it works, you know, it, it uniquely identifies the API that you have published with API. So let's just say this is all about greeting. So I'm just going to give the API URX si as greeting and note what the base URL has now become. It is the base. It's the API gateway URL of the API management instance appended by the API UR suffix. And whenever you will have to call an operation of this API, it will also get appended over here as we will see in the next section. So with that in mind, let's just click on create so that um our API can be published to API.

So you can see how simple it is in just like one step, we have imported our, our function which is sitting inside a function app in Azure over here in A PM. And again, you can see that it just has one operation over here. Now how, how the Azure A PM works is that whenever a call is made to an Azure API management instance, it's first routed to the front end. What the front end does is it passes it to the inbound processing. At this particular stage, you can apply a lot of process informations or validations and checks to it. So just look at some examples, you can filter, you know your incoming request based on your allowed or block type addresses. You can limit the call rate, you can walk responses, you can cash responses, you can validate JWT tokens, you can validate parameters. So there's a whole lot of things that you can do. And so, so just for the purpose of this demo, let's add, let's add a policy here. Uh which is, which limits your call rate to control the number of requests reaching the backend service. So let's say we want to limit the request to five requests in 300 seconds. And um yeah, let's go on safe. So what we have done is we have added a policy in the inbound processing section. So whenever a request comes to the API management gateway, if the front end first outset to the inbound processing at this stage, the policy is applied.

Let's say if it's all good, then the request is passed to the back end where the function is executed, your backend API is executed. The result on the way back is again passed via the outbound processing stage. At this. You can again apply some transformations, maybe convert your XML to JSON, you know, and then before you send it back to the caller. So this is how an HO API management works. Let's quickly see it by testing it. So I'm going to do the same thing. The way I tested the function app, I'm going to uh pass a parameter called name with the value called TNA note the request URL which is made. So whoever needs to call this API now has to call this request ur path. So it is the base URL followed by the API U I suffix that you mentioned, followed by the operation name and then any query parameters that you have. So with that, let's get consent. So when you click on send, you can see that we have received a 200. OK? And the response that we were seeing originally in our function app, now we have clicked this once. Let's try to click it five more times and see what the response is. Yeah. Second time, 200. Ok. Third time, 200. Ok. Fourth time 200. Ok. And fifth time 200. Ok. Now, sixth time, if I will attempt to do the same, it should ideally stop me.

Uh It should not send the request to the backend API because we have put a policy here which limits the call rate coming from one subscription uh to just five requests in 300 seconds. So let's click on send and yes, what we see is that this time we have not got to on it. Ok. We have got a 429 to more too many requests. And in the message, it has clearly called out that your rate limit is exceeded, try again in 2 62 seconds. So you can see uh how simple and easy it is to deploy an API to API M and then use it to uh reach out to you. Back in API S. So this was the second part of the demo where we saw, how can we publish an API to API now in the third part of the demo, which is very exciting. What we are going to see is how can we do just the same thing, whatever we covered in part one and part two. But this time uh using infrastructure is code like and, and you know, the issue pipelines and we are, we are using Bicep as a tool uh for our um infrastructure as code uh deploying our resources to assure. So let's jump into that.

But before we jump into that, I will take you back to the presentation here to talk a little bit about the deployment architecture. So there are two parts of this deployment architecture. The first, the first part is where we are deploying our API our API management instance.

So let's imagine that you are an enterprise and you want to maintain a shared API management instance for all your teams. So you have two tasks at hand. The first part, deploy the required infrastructure to Azure, which is the creation of the API management instance, which we saw in part one of the demo that I showed. And the second part of uh the task that the A PM team has is how can we make the developer experience easy and smooth for the different developer teams? For different application teams that want to publish their API S into API. So for that, uh we have divided it into certain tasks. The first task deals with uh deploying your Azure API management instance uh uh infrastructure to Azure. I'm going to take you straight to the code for that. So this is uh this is the pipeline Azure pipeline which will deploy our Azure uh API management instance into Azure. So you can see that it has three stages. The first stage is we have written some bicep code for deploying our API management instance that is converted into arm templates. And then we validate those arm templates. And then we deploy those arm templates a quick look at the code for uh the infrastructure code part for this.

So you can see that we have two parts here. The first part is Infra it's, it's a very simplified uh simplified deployment of API. You can see that all we are providing is an API name, a location, uh some tags, the pricing tier here we're mentioning the scaling capacity. We're talking about publisher, email, publisher name. So essentially everything that we mentioned in the Azure portal, we are mentioning it here, but just that we are using BICEP and um BICEP is like a declarative. It's a, it's a, it's a language for declaring uh for decoratively deploying your Azure resources. You can use Bicep instead of JSON templates for developing your arm templates basically. So this simple bicep file is going to is going to deploy your API management instance in eure. And uh for that, we're going to look into the second part, which is the A O pipelines. Now, the pipelines, they expect a service connection so that uh which has the rights to deploy to that particular resource group, your API management instance to that particular subscriptions resource group. After that, you can see that there are just a couple of stages that are there.

So the first stage is Bicep dot Yaml. What this stage does is it converts your bicep templates into an arm template, uh J zone format, which can then be directly deployed to assure in the second stage, we validate those arm templates that we have just converted from the BICEP state um just for the syntax, whether it's it's syntactically correct or not.

And after that, we deploy the arm template that we have just that we have obtained here to the insure environment. So once this pipeline completes running what we will have in our Azure portal is the, the part one that I showed, which is an end to end API management instance uh for your for your enterprise. That's the first part. The second part is, is very important. It is to make the developers work easy when they want to publish their API to API. So for that, what we have done is we have connected, we have, we have created two custom rules to deploy and uh to uh to right to create and publish API S to A PM. But uh but there is a catch here. So what we've done is via our third pipeline over here where we do the role assignments. What we do is the teams already have their own pipelines, which they're using to deploy their Azure resources to assure what we are doing is we are granting permissions to their pipelines service principles.

Uh We are, we are granting them permission to be able to deploy to our shared API management instance, their API S. But the, the, the catch here is that we are just giving them only limited permissions to be able to deploy just their API they do not have permissions to interfere with other teams API S nor they have permissions to interfere with the API management instance that we have created and that is achieved via this pipeline, which I'm just going to show now.

So let's go back here. So we have a second pipeline called um as your pipeline rules. Now, all this uh second pipeline does is it maintains the rule assignments. So if you go to this stage, which is non pro rules where we will just go in a minute. So stages non pro rules. So you can see that all this expects is it's just a powershell script, it expects an object ID of the service principle of another team's service. Uh three teams pipeline and well along with that, it expects an API name. So if you pass these two parameters, which is the object id of the pipeline service principle and the API which the team wants to deploy, and you can see that we are passing it to this method here. And what this method does is an API rule. What this uh function does is it, it gives you uh these the custom rules that we talked about. But the scope of those custom rules is limited just to the API that they want to create. So it's like we have sliced the API management instance where we, we have got different slices and we have passed it to different teams. So what we have achieved is we have achieved a decentralization of our API management instance uh pipeline. The teams don't, the, the simple A PM team doesn't become the bottleneck for maintaining and deploying uh API S of different application teams that the ownership uh and responsibility accountability lies with the different application teams and they do not interfere with other teams.

API S which are deployed to API management instance. So that was the first part which is uh deploying the A PM part. The second part is, let's say you are from the developer side, you want to publish your API to API and you will see how easy the work has become if you are on the developer side and you have an API which you want to publish to API. So all you have to do is you already have an existing Azure pipeline over here which will uh deploy your function app uh to a your environment. Let's say you already have a backend API. Let's uh let's say you have a function app that you have deployed. Uh So you only have the pipeline, which does that, the only thing that you need to do extra is add a step to fetch the open API definition of your backend API. And once you have that open API definition, and then again, you need a little bit of infrastructure as a code, another bicep template to publish that API to API which we will just see in a minute. So you have this second uh let's, let's say you belong to a team A so you can see that we have created another pipeline here.

What I'm going to do is I'm just going to click uh a one just to show you I am I already have the function which I've deployed. So I'm just going to run the compiled B stage, the swagger and the deploy API to API. But before that, let's delete this API, which we created manually and run this pipeline. So while this pipeline is running in the background, I'm gonna show the code behind this. So um let's go to report, you can see that we have a source folder here. This is where my hello function sits. It's, it's the same function which I showed um the uh which, which just says hello, this actually be of function executed successfully. So let's say you have this hello function that you want to deploy to Azure. What you will do is you will write a bit of infrastructure as a code. Like again, we are using bicep. So what the first part will do is it will deploy a storage account, an app service plan and the function app to assure. Now, uh once the infrastructure is available, let's go into our pipeline. Let's look at this main pipeline. So again, it requires our service connection. So this is the service connection of the pipeline of the application team. OK. The different stages that are being used here is the build.

This is to build your.net code, your application code into a zip file which then which then can be published to your function app. The second stage is you are converting the bicep into an arm template, the BICEP which has the infrastructure code for your storage account, your app access plan and your function app to deploy to Azure. Uh Once you have uh once you have the arm template converted from a bicep over here, you validate that arm template and you deploy. So once you have, you have run the pipeline till this stage, what you will have in Azure is a storage account, an app service plan and a working function app