Thank you so much, everyone uh for joining this session. My name is Shi Astana. Uh I'm a research senior software engineer at IBM Research Almaden. Uh I work in cloud services and analytics team and today I would be giving a talk on the transforming the order management process.

Uh This work was done along with my colleagues, Pawan Taika. So um a short introduction of myself. So I work at IBM Research Almain. Uh This is a view of a lab uh in the spring. Uh My research and development is in the area of cloud services, IOT and productive analytics. Uh I've been doing research and development for a few years now. Um And I've published my work in top tier conferences um And I've recently become an I triple senior member. So uh our agenda of today is transforming order management, the cloud service providers, they perform transaction of goods and services with their clients and most of them are usually contracts and orders driven. The other way of doing. Uh uh This is doing a pay pay as you go method. You know, like how much of a cloud solution you used that much, you pay. But the other method is to sign uh a contract uh which includes like a purchase order, document, that details what are the products and services that you would be providing to the client. And then based on this purchase order or contract that we signed, uh we pay, uh we look at how the cu customer is uh utilizing the services and we keep billing them invoices based on usage.

So every uh IBM and other service providers, we process more than 10,000 purchase orders uh which have to be monitored and managed routinely. Um since the the systems which house contracts are not always in the same system and a dis uh not uh they're disintegrated from the invoices system. So it's harder for us to manage all contracts and the invoices. And then also look at what invoices have been settled by the customer which have been put into dispute. So because of this purchase orders get into a dispute, but the overall end to end flow is not uh there. So it's not easy to find the uh which are the purchase orders in dispute. So as you see, um the number of stakeholders here which are involved in the orders management like the purchase order management are too many. We have the customer, we have the customer service representative, the services delivery person, the contract of order management, the invoice management person and engaged support. The one which is handling all the dispute tickets, the customer. So they are all um uh talking using different systems to log their work.

So uh this is another way to look into this issue when a customer raises a um request, um it uh the request goes to the co code to cash team, the team which is going to provide the services and the services delivery delivers the products to them. Then another team looks into what are the services provided, uh bill them invoices. Um make sure the invoices are paid which is stored uh and sent back to the customer. So there are a lot of steps involved in this monitoring and all the actors here do not have the full visibility into the entire solution. So there can be disputes, disputes that can occur because we do not have the uh complete knowledge of how the solution should be or is looking as end to end. So uh the invoices can get into a dispute or there can be a correspondence with the customer through representatives that can change the status of an invoice, but that may not be directly uh reflected in the orders dashboard. So there is a need for building a trustful dashboard uh which incorporates all the purchase orders, the customer data, the invoices data together and detect over usage of services and what are the risks associated with those purchase orders or contracts?

So I hope you got the motivation of the problem till now. So this is what we're trying to solve here is how we can transform the way we bill our ca we transform the way we have our purchase orders, information, how we monitor and manage them in order to uh improve the overall orders increase. So we increase the renewal of these contracts and we decrease their disputes. So as you can see, I do use the terms contracts and orders interchangeably. Uh A contract is the in the in the hierarchical structure. Contract encompasses many purchase orders. So what are the likely outcomes we are trying to get here? We want to be able to provide all the stakeholders visibility into the process. We want to do more customer profiling. So if a customer is seeking more than one product or services, we want to see how, how is the usage across the different services they have. And then we want to um show those purchase orders which are, which require your attention. So suppose an invoice in the order got into dispute, we want to look into it. So um so what are the risks and associated with this process? There's the risk of purchase order funds getting exhausted.

Uh You need enterprise purchase orders that needs monitoring invoices may not be settled by customers on time. And do the team may have more than 1000 purchase orders to look at at one point of time, they can't simply monitor all of them on a daily basis. So they will help build a priority list of those purchase orders that need higher attention. So our first step over here was to do the unique purchase order to invoice matching. Now, this is a unique problem that is there in the financial services industry is how do we map to financial data sets? In this case, a con contracts data set with the invoices, data set. So we found out those features that were unique to put the data sets and use them to get the matching. The amount of each invoice is not fixed. So we think that a purchase order is of a dollar, say $1000 amount. But the invoice usage may be more than that or less than that. So the invoices bill would be not be exactly $1000. So we have to look into how much is the usage and use it to build the custom. Now, um once we did that matching of those purchase orders to invoices, we will we have a set criteria that suppose I have a contract for a year. And um I want to build the invoices monthly. I do expect to see around 12 invoices for that entire year.

So we have an expected number of invoices and expected billing amount, but we also have the actual invoices and actual build amount. So we can compare these two to perform the risk analytics. So how do we do that? We do that using the risk analytics model So initially, we thought we could just use a simple machine learning model, um a regression system or use a time series to predict the risk of a purchase order. But we realized that um an in house risk and algorithm was necessary because of machine learning did not dynamically update which uh features it has to pick up. So we used a number of features like customer features, po features features that we derived. like what is the trend in invoices usage. So we plotted the usage of invoices over time uh using the profit model. So profit is an open source algorithm in Facebook, we use that to plot. Um And then we also looked at looking at the, we looked at how are the customer is utilizing them based on that, we calculated the risk factor, which is the remaining po amount upon the expected amount uh per invoice. So we have a calculation of how much amount we expect to invoice and how much amount of that contract is still remaining intuitively, it's saying that say if it's a value six, that means I have a risk factor six.

That means I have six invoice cycles amount enough to cover the contract amount. And uh I can uh build six invoices without uh an issue. But if it's 1/7 invoice, it may get into a dispute. So we want to be able to use that risk score into translating into how uh uh the risk is of that purchase order and we use that uh we generate these predictions from the snapshot of purchase order that we have and we keep updating it based on the invoices data, based on the invoices settlement data, et cetera.

Uh This is a more um uh so we did dig in into further and refine the model. We looked into how risk can be categorized by purchase orders, which are being tracked by end date, like based on when they are going to expire versus those which are just being tracked by amount, like how much po amount is left because a po amount is a fixed amount at the beginning of the contract, it does not change and we do not want that amount to get exhausted through the invoices.

So this is a snapshot um of a tool. Uh We had a good around 283 users who have been using this tool across many countries like us, Canada, Asia Pacific Australia. Um And we have a different line of businesses. So we built this uh uh dashboard tool which showed the risk along with the po invoice matching. The one of the uh outputs of the matching was that we saw that sometimes we were billing an invoice to a purchase order or a contract which had already expired or it was inaccurate. So those invoices were highlighted before the customer could dispute them. The second was we saw how the risk score looked like for different purchase orders. So for example, in this purchase order, uh if we look at the right side, we have the actual uh usage of invoices and the predicted usage of invoices. So more or less we were able to track what the actual usage, what would look like and use it to come up with a risk chart. So this chart we trained on 4500 purchase orders um with more than 1 million invoice records because each purchase order at least had 20 to 50 invoices being built to it.

Uh Using this, we calculated the risk factor are and our criteria for, for uh giving an insight into purchase order was how likely it is that this customer will renew this po on time. So firstly, of course, it matters whether the car customer had an existing contract to dispute or not. How does this customer portfolio look like? Like whether he's a new customer or an old customer? Uh When is the purchase order expiring? How does the past trend look like? Except um lastly, these are some of the things we are working on right now, which is customer profile. Now, data driven customer profiling is very important because you want to be able to not just know whether this is an old customer or not, but also look at how has the old customer behaved in cases of disputes uh and see if we want to renew the customer or not. So we want to be able to maximize as many renewals as possible, minimize the cost it takes for a certain action uh while working on enterprise high-value purchase orders and then maximize the revenue for the given brand or customer. So here we want, we could not build one single customer profiling tool as each geography, each business unit uh tackles the customer separately.

So we build different customer profiling methods based on the constraints that each geography and business unit had and came up with a reasonable set of actionable and non actionable insights that they can use. So reflecting on the past, we found many challenges in adoption of order management workflow um where we had to build an integrated experience for all the stakeholders, all the actors uh around the purchase order, we build a trash, a trustful dashboard for the same. Uh We observed that there were separate tools, maintaining purchase orders, customers and invoice data. And we provided an automated uh data mapping system between all the data sets. And this way, we build the transformative workflow process which show the order management data to all the relevant deeps.

So our key takeaways from this transformation of order management process is how to can we provide better visibility of the process to all the teams to perform more, better customer profiling so that we can help our customers purchase more products and services provide risk and in insights.

So we prompt our teams to take actions with customers. Uh Thus, we are reducing the disputes. We are increasing renewals on time and we are ensuring that our data is high quality and is also create data collection and a monitoring mechanism. Uh Thank you so much. Uh I hope you found this dog insightful and I'll be happy to take any questions that you may have. OK. So I see one question from Eleni. Uh Thank you for the question. So the question is, how can we better monitor any discrepancies in the order versus what was delivered? So that's a great question, Eleni. And sometimes it may not always be discrepancies, but the customer thinks that they would require 1000 servers um physical servers, but they actually end up needing more than that. So the best way to do is to uh first of all, as we did was keep the orders and invoices data at the same place uh or have a better ma matching mechanism. So you know that what was the order uh went for and whether is the invoice complying with the constraints of the contract or not? So, um comply and compare is a very important method.

A very important way to monitor what uh what was the order about and what are we providing to our client? So I hope that answers your question. Oh Any other question that I may answer? Do we need fulfillment Kpis? Um So the fulfillment Kpis can be added. Uh Also I in uh in the end to end flow process in our current flow, we did not add that we had it only till invoice settlement, whether the invoice was settled or not. Uh But a fulfillment. KP I can definitely add value to this process. So, thank you. OK. So I think that's my time, but I'll be happy to take any questions that you may have regarding this work that we are doing at IBM research or any other questions around IBM research that you may have? Uh Thank you and thanks for the opportunity.