Krishna Subramanian - The Cloud Gold Rush: How Businesses Manage Data in a Cloud-First World
Paving a Path in Cloud Technology and Data Management
Christian Subramanian, a seasoned professional and the co-founder of Comprise, sat down at a virtual Women in Technology conference to share how businesses manage data in the cloud and why it's a promising career opportunity for women.
The Significance of Cloud Adoption
Cloud adoption is growing rapidly. Businesses today strive to shift a larger fraction of their business workloads to the cloud. In fact, a recent survey found that 94% of enterprises are resorting to using multiple cloud models. The surge of opportunity within the cloud, particularly with respect to data management, is truly intriguing.
The Challenge and Opportunity of Data Management
Today, businesses encounter challenges with resource and space limitations in their data centers, compounded by new issues such as supporting a remote workforce due to the pandemic. The cloud provides a transformative solution for these IT challenges.
Initially, companies started with writing new applications for the cloud and experienced how easy it was to use the public clouds. From there, more software was moved into the cloud. Moving on from these initial steps, many companies have adopted a "cloud-first" model, migrating tasks that would usually run within their own IT infrastructure to the cloud.
This transition to a cloud-first model is what needs to be realized and understood - particularly in terms of the implications it has on data generated by these applications.
Explosion of Unstructured Data
Data generation has experienced an exponential growth in recent years. Compared to the amount of data the world had in 1970, today's data volume is monstrous – with 90% of the data generated in just the last two years.
Much of this data is what we would term as unstructured data – it doesn't fit into a database; instead, it comprises things like an X-ray file, genomic sequence or IoT data from a self-driving car.
Why Data Management is a Great Oppurtunity for Women
The flourishing market of data management, increasing at a steam of 20-30% each year, creates an abundance of opportunities in tech careers. Genomics data in particular is seeing its amount of data more than doubling each year.
The broad field of data science, from analyzing trends to technical roles, is a golden opportunity for women, especially since there is a significant gender gap.
How Companies are Maximizing the Benefits of Unstructured Data
Comprise and Unstructured Data
Our company, Comprise, focuses on understanding, harnessing, and managing this unstructured data, an emerging area of focus in the data management sector. We liken the current rush towards managing and understanding this data to a gold rush – it's an area of explosive growth and immense opportunity, particularly for women looking to carve out a career in cloud technology or data science.
The Future of Unstructured Data
The sphere of unstructured data is wide, diverse, and rapidly expanding. For instance, patterns within data are now being used for analysis. From genomic sequences to video content to seismic analysis, the potential of unstructured data is vast and intriguing.
Whether you're already employed in the tech industry or are considering stepping into the field, now is the time to explore careers in cloud technology and data science. Feel free to reach out on LinkedIn for any further queries, and let's unlock the world of data together!
Video Transcription
Hi, everyone. Uh It's great to um participate in this uh virtual women in technology conference. And um I'm honored to talk to you about uh the cloud and particularly how businesses manage data in the cloud.Um I'm gonna walk through a bunch of concepts around data and how it's growing and why managing data in the cloud is a great career opportunity for women uh and share with you my own personal story. Um And the story of our company comprise, um I'd love for this session to be interactive. So if you have any questions as we go along, feel free to type them in and I'd be happy to answer them for you. My name is Christian Subramanian. I am a co-founder and coo of comprise. Um So let's get started. Um You probably have heard this a lot. Um Cloud adoption is growing. A lot of businesses today are trying to put more and more of their business workloads into the cloud. Um And uh in fact, a recent survey found that over 90% 94% actually of enterprises are not just using one cloud, they're using multiple cloud models. Uh And um many of them are using multiple public clouds. So all of this is very interesting. Um It's fascinating actually because uh there's a huge explosion of opportunity uh in the cloud. Uh And there's a lot of opportunity with data that goes into the cloud. Uh and what you can do with that data and I wanted to share a perspective of this with you.
Um Because if any of you are considering careers in uh cloud or data management or high technology, uh you might find this uh interesting and relevant. So uh let's give you some background. Um companies today are really running out of resources. Um They're running out of space in their data center. We saw with the pandemic, all the issues uh of supporting a remote workforce and the challenges that businesses had to face. Uh Most companies have limited resources and limited budgets. So cloud provides a great way to transform their it technology. Um And several years ago, when people like Amazon first launched their public cloud with Amazon web services companies started experimenting with the cloud. Uh They had their developers write new applications for the cloud.
Uh And that went pretty well. They saw how easy it was to use the public clouds. And then companies started moving more software as a service into the cloud. So they started moving things like the sales force customer management application or their back office application and other database centric transactional applications to the cloud. And now what's happening is a lot of companies want to go beyond these first two steps. And they're really saying before we add anything in our data center in the cloud first, so they're going to a cloud first model, meaning we're trying to shut down data centers and move things that normally would have run within their own it infrastructure into the cloud. So I wanna walk you through what it means to be cloud first and particularly what it does to the data that these applications use and what happens to it. OK, so a little bit about myself and why I'm speaking about this topic. Um I have almost 30 years of experience uh in computer technologies. Uh I have a master's in Computer Science from the University of Illinois in urbana-champaign. Um And uh I met my two co-founders there uh about 27 years ago. Uh And uh then I, I worked in Silicon Valley for many companies including Sun Microsystems for about 14 years and then did three start ups. This is uh the third start up that I'm a co-founder of uh with my other two co-founders.
Um And our last two companies were acquired. Uh So I have a lot of history in uh technology uh particularly in enterprise uh information technology. Uh And uh I've been pretty involved with the whole movement to the cloud. Uh And um so let me uh uh uh uh a little bit of uh facts about comprise. Um you know, we are a, a venture backed company, we're based in Silicon Valley, California and we help customers manage their data, a particular what is called unstructured data across their cloud environments and their data centers. And we work with a lot of the large enterprises and we partner with a lot of the cloud providers and major vendors like IBM and HP and pure storage. So I wanted to give you a background uh from our perspective of the problem that customers are facing and why this is a huge opportunity. So, uh first, uh you know, uh I wanna talk to you a little bit about data and the two kinds of data that's out there.
Um All of you probably have a smartphone and I'm sure you take, take uh pictures and videos, uh you take photos of your family or your pets. Uh And um you know, if you think about it, you have a lot of photos on your phone. How often do you go in and delete your photos? Um If you're like me, probably take a lot of photos and videos. Um and you barely ever delete them because I'm always thinking, well, one day my kid might like to see this picture or one day I'd like to show this to their grandmother. And you know, that's kind of what everybody's doing from holders by nature. We tend to keep uh data around for a long time and businesses are no different. They're also keeping data for a long time and they're generating a lot of data. Pretty much everything we use today generates data. You know, every time we jump into a electric car, it's generating data. You know, every time you take a video or you take a photo on your phone, it's generating data, you know, uh every time you go to your doctor's office, they run an X ray or they uh you know uh do a lab test that's generating data, genomics analysis is generating data.
So everywhere we look in every facet of our life, we are generating a lot of data and a lot of this data today is digital. So we're keeping it on digital technologies and we're keeping it around for a very long amount of time. That's why if you look at the amount of data that the whole world had uh in 1970 compared to today, it's a massive explosion, you can see how this curve is exponentially growing because we are generating a lot more data. And in fact, 90% of the data that we have in the world today was generated in the last two years. Uh believe it or not. Um And most of this data is what we would call unstructured data. And what we mean by that is it's not data that fits in a database. So it's not like a, a transactional record of your sales volume uh but it's things like an X ray file or a genomic sequence or a self driving cars, IOT data, you know, or a seismic analysis of, of a, of a, of geography. All of those is what we call unstructured data. There's no specific format. It's a variety of formats and most of the data in the world today is unstructured and that's what I'm going to talk to you about.
I'm gonna talk to you about what it takes to manage this unstructured data in today's cloud world and why? That's a great opportunity for women. Um Another thing I wanted to mention is, you know, you saw that big explosion of data just to kind of put it in more, you know, uh understandable terms, you know, data is measured in, you know, gigabytes or megabytes or terabytes or petabytes, etcetera. So if you own a smartphone, you probably have a few terabytes of data. Um Many businesses uh are today having terabytes of data and just to give you an idea, a terabyte of data would be about 11,004 K movies. So, you know, I imagine, and you know, every business probably has 11,000, you know, Titanic movies kind of uh size uh today. That's where a lot of companies are today and they're rapidly approaching an exabyte. So if a petabyte is 11,000 movies, an exabyte is 11 million movies.
So you can see the explosion that's coming from going from a petabyte to an exabyte. And analysts like the firm I DC are saying that actually where we're headed is not in petabytes or exabytes, it's in Zetta bytes. They're saying that the amount of data we're gonna generate by the year 2025 is going to be 100 and 75 zab bytes and a Zab byte is 11,000,000,004 K movies. So you can just imagine how big 100 and 75 zab bytes is. Um So that's really the explosion of data that's happening in the world around us. Um And just a little bit of a practical way to think about it. So all that data is coming our way. Uh And most of that data is unstructured and this unstructured data because it doesn't have a specific format. Uh It's more complicated to store it and manage it. Uh In fact, if you look at all the cloud providers, they may have two or three ways of storing structured data, but they have 16 plus ways of storing unstructured data. Uh And the unstructured data can be billions of files, billions of objects. And so you really need a way to manage all that complexity. You need a way to understand what it is. You need a way to move it and manage it and that's where a lot of work is going into.
That's for example, the company that I co-founded, you know, we work in this space, but we're not alone, you know, there's a lot of companies doing big data analytics in this market. Um uh There's a lot of uh companies doing storage for this unstructured data. So there's many angles of unstructured data, you know, from starting from how do you store it to, how do you manage it to, how do you protect it? Uh How do you secure it? How do you analyze it, et cetera? And all of those are great opportunities. Um If you're looking at a career in high technology, so some of the things that customers really care about in the with this unstructured data is, how do they really optimize the cost of managing so much data? And how do they leverage the cloud in a smarter way for their data? Um And this is where you know, providing ways to understand all that data and then move the right data to the right place and then do things like artificial intelligence on it or big data analytics on the data so they can extract value from that data are all important. And these are some of the areas we are looking at as a company. Uh But these are areas that the industry overall is also focused on.
Uh so there's a big movement in the technology industry towards data and you know, data analytics, data management, um you know, extracting value from data A I on data, big data, et cetera and those are all excellent opportunities and some of the reasons why businesses are adopting these solutions is because they can cut their costs, they can be more agile, they can move faster and they can generate more value.
Um And uh because you know, their users need an easy way to find the data that they have across different environments so they can do things like big data analytics on it. Um So, um I wanted to highlight a little bit for you. Uh This is a pretty uh um rapidly growing market. Um The data management market overall is over $18 billion a year. Uh And it's growing at over 20 to 30% every single year. In fact, in some markets, it's more than doubling every year. So if you look at genomics data, we're just starting to analyze our genome sequences. Genomic sequencing is gonna be a bigger and bigger thing and their data is more than doubling every single. Um So there's a very large opportunity around data and data management and what's broadly known as data science. Uh Data science is analyzing the data looking for trends, things like that. Um There's a lot of career opportunities in this market. Um There's a large gender gap.
Uh you know, there's not as many women uh as uh as there are uh graduates of college uh in the whole data market. Um And it's an excellent opportunity really because uh you can apply strong analytical thinking skills um as well as you know, um technical skills. Um it's a great market uh to be in uh there's a variety of careers uh in data management. Uh You can be on the infrastructure side, you can be uh in the data science side, doing analytics and doing things like, you know, A I on data, if that's an interest or you could be uh kind of on the architecture si side and you know, helping businesses transform, become more agile, et cetera.
Um And within the whole market of data, this notion of unstructured data, you know, data that isn't sitting inside a database is really where a lot of the growth is. Uh because uh we have a pretty good understanding of how to manage structured data, but unstructured data is a brand new world. And you know, since 90% of the world's data is unstructured, that's really where a lot of the innovation is going in. Uh That's where companies like ourselves are focused on finding better ways uh to understand unstructured data, finding better ways to harness unstructured data and manage unstructured data. So I would say this is kind of like a gold rush, you know, um like how the West was established uh during the eras of gold mining, that's a bit of what's happening right now with data and the cloud, you know, the market is really exploding uh for data management. Uh And there is a gold rush in a way uh towards this area. So if you are considering a career um in cloud or if you're uh interested in things like data science, I would say there's no better time than now. Uh There's a pretty significant um opportunity there. Uh I know I'm close to the end of my time. So I wanted to keep the presentation uh short and sweet and I hope uh this gave you a bit of a background uh on what's happening in the market.
You know, why companies are moving to the cloud? Uh why businesses are looking at what's called unstructured data, which is, you know, like data, like video and genomics, uh data and other things that are not fitting in a database. Uh Why this unstructured data is a pretty large market. Uh And there are m and why there are many opportunities in it starting from data management to data science uh to cloud architecture, et cetera and why this could be a great career choice. So I hope this was all helpful to you uh and feel free um to um reach out if you want. Uh somebody asked a question, can you share some examples of the type of analys is done with unstructured data? Absolutely. I mean, there's a lot of things that you can do with unstructured data. Um For example, uh you know, any kind of genomic analysis um requires uh unstructured. Uh all genome sequencing is unstructured data. So they're looking for things like uh you know, do you have uh you know, particular variants of mutations of genes can cause certain kinds of cancer more frequently. That kind of analysis is done on unstructured data. Uh Or if somebody is um you know, looking at uh different videos to try to understand um some pattern within them, any kind of pattern recognition on video or audio that is dealing with unstructured data, any seismic analysis is done on unstructured data, you know, log analysis, but most of these things are done on unstructured data.
Yeah. Um So I hope this was helpful. I appreciate all your time and feel free to reach out to me on linkedin if you have any further questions and I hope you enjoy the rest of this session. Thank you.