Modern Data Platforms
Aditi Sharma
Managing DirectorTransforming Businesses with Modern Data Platform: An Insight
Welcome to this well-crafted piece on modern data platforms, featuring a rich conversation with Ati Sharma, a managing director with Exchanges, Data and AI Group. The focus point of our discussion highlights some of the challenges firms are experiencing in realizing the full potential of a modern data platform.
About Ati Sharma
Ati Sharma has worked in the technology industry for over 17 years, focusing on tech transformations, especially cloud modernizations in the past seven years. Based in Raleigh, North Carolina, he brings a wealth of experience in healthcare, telecom, energy utilities, and finance. His work in different platforms is client-oriented, addressing their unique data needs and business drivers.
Riding the wave of Cloud Migration and Modernization
The surge of the COVID-19 pandemic brought about drastic changes in how we work, leading to a sudden need for scalability and easy access to data. This need sparked what was arguably the significant migration and modernization of firms across the globe. The process focuses on scalability, flexibility, and providing quick value to the market.
As time progressed, there was a visible increase in seamless API integration due to firms exploring more than one cloud. However, as migration and modernization became more common, business leaders quickly identified the need for a harmonized, centralized, and accessible data layer. This need led to a move towards decentralization via the concept of Data Mesh Architecture.
An Understanding Modern Data Platform
Although firms are bringing data under one roof to streamline processes, it is paramount that they communicate the business value within it. Therefore, a focus on data ownership began to take precedence. The modern data platform thrives on harmonization. It's every bit as essential to ensure the credibility of the data because the business value ties directly into the quality of the data and how it's managed. However, the challenge lies in moving humongous amounts of data to the cloud while maintaining its integrity, a challenge many companies are progressively tackling.
A significant aspiration of a modern data platform is self-service: allowing businesses to access data quickly without relying on their IT counterparts. This need led to the creation of data portal and data marketplace concepts. However, firms are still struggling to extract full value from the modern data platform.
Turning Investments into Value
As 2023 unfolds, quite a few clients express concerns about high costs incurred without matching the pace of realizing data value. The focus shifted towards operational efficiency, especially in managing costs.
Cloud continuum has introduced an abundance of opportunity to evolve and scale data according to business needs. However, recurring costs and ensuring data security has been a stumbling block for many. Data integrity and efficiency have taken precedence in the modern data platform. Firms are evaluating their RoI on the investments made on the cloud and realizing that only putting data on the cloud doesn't automatically convert into value.
{{h3}}Conclusion: The Modern Data Platform Way{{/h3}The modern data platform is more than simply a centralized data platform. It's about realizing the full potential of your data and monetizing it effectively while managing internal costs. These platforms only thrive and provide value when there is strong collaboration between IT and the rest of the business.
Firms are moving towards bringing business and IT together to drive a broad change across the entire enterprise. In a nutshell, transforming through a modern data platform is not a mere technology transformation play. Instead, it's an ensemble of technology, process, and people, ensuring the new approach to data is being driven across the organization.
Stay engaged for more talks on modern data platforms and how they can transform businesses. For any further discussions, feel free to connect with Ati Sharma on LinkedIn. We look forward to helping you navigate your data transformation journey!
Video Transcription
OK, so I think I'll get started. So once again, good morning everyone. Um This is Ati Sharma here. Uh quick brief background about me. Uh I'm based out of Raleigh North Carolina.I'm a managing director with exchanges, data and A I group uh focused on migration organization and definitely touching upon one of the topic today as well. Uh A quick context about my background. I have been in it for around 17 years now. Uh completely focused on technology uh transformations and past seven plus years more on cloud modernizations, uh different platforms we touch upon based on the work we do uh and definitely different use cases uh based on client domains that we work upon. So I do bring a lot of experience around healthcare, telecommunications and uh energy utilities, uh financial domain. Uh So uh kind of combination across all with different uh data needs, different business drivers. So today, uh I would like to talk about uh modern data platform and some of the challenges we are seeing or repetitively hearing with clients and how uh a lot of our clients feel that uh they have not yet realized the
value of cloud modern organizations, the way it all
picked up buzz. Uh So uh quickly going back to uh 2020 definitely the year that changed a lot of the ways we were working and a lot of our clients did feel the need to quickly uh bring options of scalability, easy access to data and uh how the it transformation looked like. And that was the pioneer year when a lot of our clients uh across globe uh took uh or initiated their journeys to either migrate or modernize. So uh definitely 2020 kind of set the stone, a lot of people who were not looking at cloud organizations as a pathway, started exploring it and began their journey. But if you talk about uh moving to cloud, right? And uh the visual that we see here is a very quick glimpse of what cloud was expected or anticipated
to provide in terms of technical capability, right?
It it did bring in scalability, it brought in flexibility. It gave you an option
of high availability like
specifically in times like pandemic where uh people were working remote, people were not sure of when the downtime can happen due to personal uh uh unavoidable circumstances. Then we were also looking at opportunities to maintain the right security. Even
if now we are trying to do things on cloud, we're trying to move a lot of our data sets on cloud.
So a lot of that was the initial uh focus uh around cloud migrations organization. So definitely top 23 things that comes to everyone's mind is scalability,
flexibility and quick uh uh
uh market to
uh value kind of theme, right?
Um With evolution, uh definitely aspects of uh seamless API integration scheme because a lot of clients did want to explore more than one cloud. Uh No one wanted to just lock in just on one cloud capability if they have a bigger
data and it footprint.
So a lot of integration and aspect was also a key uh contributing factor to the success of your cloud modernization, data ops, data security definitely evolved as migrations organizations were on their good uh uh journey. Uh can you? Uh But uh as cloud was picking up more, we were talking more of migrations, we were talking more of modernization. Uh That's
where modern data platform picked up more as a team,
right? And if we talk about modern data platform and definitely this is different than modernizing your apps or different capabilities, we are talking of a platform that gives the unique aspect of harmonizing your data, right?
And when we say harmonization,
right? Uh it's not just about bringing data to
one location or
making sure your silos are removed. It's beyond that, it's talking about what is that uniform layer which makes data seamless, centralized and accessible to everyone. And uh that definitely is one of the foundational aspects
of modern data platform too,
but with data harmonization also came the responsibility of platform and data ownership. And uh this became more relevant because uh everyone focused on centralized data platform. But at the same time, you bring everything on a common area, you try to streamline it, but you have to eventually look at what business value with it, right? And that's where the focus of data ownership start building up last two years. Uh definitely also brought more momentum on uh data domain uh data product or you can also call
data mesh architecture concept.
Most of our clients are focused on bringing um business along because you won't be able to implement it successfully if you do not have the right coalition between it and business there. So that was another challenge where um
data mesh architecture was picking up a lot of momentum, a
lot of firms, a lot of our clients were trying to go towards
it. But uh
the key question that always uh remained as open was uh were the firms able to bring the right cadence of change or were the firms able to bring the right mix of process people and technology together? And definitely it's not a a few months journey, it takes time for you to bring that change in how your organization operates, how people
see the value of data.
Um Touching upon this theme uh also brings the aspect of data governance. Um you can bring tons terabytes,
megabytes of data and
cloud you can modernize your platforms. But if there is no way to authenticate the quality of your data, your uh there is no due to all the modernization, right? So that also has been one of the repetitive challenge we have heard from clients. They are progressing towards it.
But uh just moving terabytes gigabytes of data onto cloud will not solve the problem if if the data integrity the right data governance is not
taken care of. So what I'm
talking about different aspects of modern data platform, I'm also trying to share some areas where uh there are very repetitive gaps across
industry and across clients. And um one
of the biggest aspiration out of modern data platform was also around self serve uh which was how business can have access to the data quickly, right? Business teams do not want to wait, rely
on
uh their it counterparts to request data, they wanted to have mechanics or ways to access data or do
self serve options.
And that's
where a lot of data portal data marketplace consider with them.
But the question again, stay that have, have we we been able to uh really get the value that we were trying to get
out of modern data platform,
right? Were we able to give the right quality right, uh governance, right? Cadence to data will be able to have right, sponsorship, adoption of the themes or different architectures. We are trying to build on
cloud and calling it modern data platform.
And that's where uh 2023 has been a year where a lot of
clients have expressed their concerns
challenges in terms of incurring a lot of costs but not being at the same pace where they would have uh thought
they would have realized the value of data.
So advanced analytics, if you look at it, right, your business value directly ties
back to this theme of modern data platform
and uh even generative A I. These days, we're talking of a lot of buzz around
chab GT
or what generative A I can do what LL
M uh modules can do for you. But uh in
the end, if the quality of your data or your data augmentation mechanics are not, right. No
algorithm can bring the value that you
are aspiring to get. So that's where the basic foundational elements of modern data
platform come into play. If some of the basic
challenges or some of the basic uh synchrony between uh people processing technology is not taken care of. Uh This becomes more
of like you're putting data on the cloud but really not getting the value out of it.
So this is more of a theme around what modern data platform was expected to be and how different clients are facing challenges on going on that journey but not being there. And on the top left, if you see a very high level, when when we talk about cloud organizations
right? And
more from a data aspect uh for a business person, the thought process is that this will enable them to have right data at the right point of time. They're not waiting on any insight or any dependency on teams to avail any data, right. They have quick ways to materialize or monetize data for business value. And uh definitely another very uh aspirational goal of any modernization. And specifically the modern data platform has been in a way faster, but it all ties back to you will only innovate faster if
your data is credible,
right? Or if you have data defined in the right manner for your teams, business or business use cases to even innovate faster. And that's where the reliability governance is very much important. Uh A lot of clients are in the journey but uh definitely not there yet where they feel this this all initiative of data modernization
or migration enable them uh in realizing the value that they want it to be. And definitely the summing up all this concept is around business value, right? If you, if
you are putting data, you you do not have right, sponsorship or right
ownership on data
from business teams, you will be uh creating different domains, different data products, but not maintaining or sustaining them well for a continued business value. So that's another challenge. And we are very repetitively hearing from clients where they are building architectures, but that that right mindset or the way the organization should drive the change in terms of understanding that data now doesn't belong to it. So that gap
uh is continuing to evolve and a lot of clients
are facing it. So a very high level picture on um how migration organization has evolved. But uh going back to data platform, some of the key themes where the majority of the market is and definitely centralized data platform is where a lot of clients have definitely reached a stage of 65 70%. But remaining areas on uh ownership, self serve, evolving to the right majority of data products or even leveraging data for the right analytics, considering
data
has right, set of uh integrity and checks and balances. It's it's still pre uh early stages of maturing. And that's where uh many of our clients being in year two year three of the journeys, they feel they have spent a lot of money on migration organizations, but not yet. Uh
seeing the value of data coming out of it. Uh So I'll quickly
pause to see if there are any questions. Uh And uh then I'll take it for for uh next
section. Mhm OK.
So now quickly moving about uh talking about cloud continuum, right? And also talking about uh what is changing in uh 2023 for a lot of our clients uh as uh in past two years, a lot of cloud uh modernization journeys picked up pace, a lot of different vendors. And in fact, I'm not talking of just the top players
on cloud, but a lot
of niche platform vendors, they evolve their platforms in a
very different direction,
right, which gave uh the clients or the end users an immense amount of opportunity to expand, evolve in whatever way they want, right? And this brought the beauty of uh uh a lot of use cases leveraging different platforms together or even leveraging different cloud platforms together.
And that's where the aspect of seamless integration or how different clouds are integrating with each other, how
the data sharing aspect is working, that will take momentum and again, going back
to how mature it is right now. Um uh definitely a lot of uh organizations are doing very good work in terms of data sharing or data exchange, but uh still
some aspects of
data security, how the integrations have to work or how the cloud comp has to evolve. It's still uh in the pace of evolving and that's where uh clients have been in a mood of procuring a lot of uh a cost
uh
setting up or uh standing up cloud platforms, putting the data there, but not being able to manage how much cost they're. So 2023 has been a theme of uh clients definitely looking at how much they have spent and
what Roy they
received out of this modernization journey, right? And that's where operational uh efficiency
has come into play. A lot of solutions are being built in
house. A lot of solutions are being given uh outsourced by key platform providers that this is how you can manage, monitor your cost, how FOPS can evolve for you. But uh this definitely is another pin point for a lot of modernization of projects right now. Uh They are in the journey, they are trying to pick pace, they are trying to see how they deliver quick business value.
But at the same time, they're incurring heavy costs
as well. So, uh again, operational cost uh or I would say fops suspect a plot
of momentum in late 2022 and
2023 provided the realization has come that cloud may seem scalable cloud may sound cheap. But when you have data there and your uh teams are leveraging processing data, it's not that cheap. So again, another uh big pain point, we
we have felt that uh uh modern
data platforms are incurring on clients. They, they are going through the journey of delivering, bringing
business on board. But at the same time finding it hard to justify costs on different cloud or platform providers they have
incurred. So that's another aspect of uh where uh if any,
any pathway or any road map has to be built on modern data platform, there
has to be a very uh very clear direction on how you will look at uh your Roy, how you will look at the cost, you are incurring and what basic structure or governance you will be around managing the cost as well and definitely doesn't tie back to just it. Again, this is a place where business and it come
together.
So uh this touches upon the aspect of uh cloud continuum will give you uh a different level of integration or bringing different iotot or different set of data which may have never existed on cloud, right? And specifically for energy utility clients or chemical clients, where uh bringing this
data is very much
valuable to some of the goals beyond just the data business cases, right? Uh um But incurring the cost
having the right mechanisms
to uh integrate m making sure that the data you bring in it is of the right
quality is some of the foundational aspect of data platform.
And I think that pretty much sums up uh where clients wanted to be and how the journey has been ongoing. But uh from maturity point of view, 2023 definitely has been a reflection on where clients
are and what are some of the pain points they are still uh looking at.
So again, I'm quickly pausing to see if there are uh any questions. Uh and also uh most I'm just doing a quick time check as well.
OK.
So I think uh uh this pretty much sums up uh what I want to cover, but uh I'll do a quick recap on um where we are on modern data
platform. When we talk of modern data platform, it's, it's beyond the centralized
data platform, it's beyond different cloud providers. Integrating it's about uh looking at waste monetize the data
if you have moved the data. But at the same time internally, uh ensuring you
have right cadence rigor on the cost. How much
cost you are incurring on these uh cloud platforms? Because
always the intent of modernizing was to have right, scalability availability, but also have options of cheaper uh platform
combinations, right?
And um what definitely plays a key role in any of the firm to uh make it more adaptable is bringing your business it together. So this this is not a simple technology transformation play. It's more about technology process and people how how you ensure that the change is being uh driven across the enterprise, how people are changing or even looking at
it in a different way
and then acting as sponsors
for the data that you're building for them on mobile data platform. So
that was all pretty much from my side. Uh very happy to be part of the discussion today. Feel free to reach out to me through linkedin or uh other way that I can uh help and be part of any
discussion. Thank you so much.