Data Science, Artificial Intelligence and Machine Learning
Unveiling the World of Data Science, AI, Machine Learning and Deep Learning
Welcome to an instructive journey where we delve into the captivating realms of data science, AI, machine-learning and deep learning. Forked over about a decade in IT, I am your guide in this journey, Sandhya Krishna.
Understanding Data in the Digital Era
In this digital epoch, data is the modern oil fuelling the economy. This treasure trove is characterized into open data with unrestricted access and closed data requiring certain permissions. The mammoth influx of data offers us a challenge. The data we collect may be raw, inconsistent, incomplete, without trends and filled with errors, but after refinement and application of statistical advanced tools, it transforms into valuable insights. These insights further morph into knowledge which is beneficial in resolving issues related to business analytics and technical aspects.
The World of Business Intelligence (BI) and Business Analytics (BA)
BI and BA play pivotal roles in dealing with data. BI helps in the delivery of information timely and accurately, aiding in better decision-making. Business Analytics is a larger spectrum that engages artificial intelligence, statistical tactics, IT and management strategies to generate meaningful insights.
It primarily focuses on four analytical areas:
- Descriptive Analytics: Records the events.
- Diagnostic Analytics: Determines why did the event occur.
- Predictive Analytics: Forecasts future events.
- Prescriptive Analytics: Guides on future course of action.
With the integration of predictive techniques in BI, the field of Business Analytics emerges.
Artificial Intelligence: The Game Changer
Artificial Intelligence (AI) is often misconstrued as a job killer, but it is fundamentally a job transformer. AI is integrating into our daily work lives, automating tedious work and providing better insights. According to great personalities like Elon Musk, Mark Cuban and Jeffrey Hinton, the sheer understanding and correct application of AI can be a game changer in businesses.
AI emulates human behavior in decision-making, text processing, visual perception, etc. AI has been branched into various capabilities like narrow intelligence (best example being Siri), general intelligence and superintelligence. It functions via reactive machines, limited memory, theory of mind and self-awareness.
What is Machine Learning (ML)?
Machine Learning leverages algorithms to identify patterns in data and generate insights. One can think of AI and ML in terms of Venn diagrams, where ML is the path to achieve AI. ML algorithms can be broadly classified into three categories:
- Supervised Learning: The model is trained with labeled data.
- Unsupervised Learning: Here, we don't have labeled data and the model learns to figure out patterns.
- Reinforcement Learning: This is a mixture of supervised and unsupervised learning and is used when training data is minimal.
A Peek into Deep Learning (DL)
Deep Learning, much like ML, also imitates the neural network of the human brain, but the difference being DL works on unstructured data. DL uses neural networks to learn from unstructured and unlabeled data. It segments the data into input, hidden (or feature) layer and output layer, identifying patterns, and predicting outputs.
In conclusion, these are not just technological advancements; they are powering solutions to complex problems, transforming the way businesses operate, and influencing how data is driven. For any queries, feel free to reach out, and let's continue our voyage in this ocean of knowledge.
Email: [email]
References:
- AI & Machine Learning: The Basics. (n.d.). Digital Marketing Institute.
- Expert System. (2017, April 3). AI & Machine Learning: The Benefits and Challenges to Business.
- SAS. (n.d.). What is Artificial Intelligence?
- SAS. (n.d.). What is Machine Learning?
- SAS. (n.d.). What is Deep Learning?
Video Transcription
Hello everyone myself, Sandhya Krishna. I'm having around 10 years of experience in it. Today, I will be dealing with session data science, artificial intelligence, machine learning and deep learning. First, we will go through data science. As you all know, data is the new oil in digital economy.
There are different types of data. It can be categorical or qualitative numerical data, but we can classify them into broader way that is open data and closed data. Open data are those data which we can acc access. Anyone can access without any restrictions like survey research data and customer studies which are published. Whereas closed data, we should have some rest. It will have a restriction and we should have some permission to access those data like sensitive and important record.
This is a graph where data is uh plotted with data volume in terabyte, terabyte and ears. And you can see from 2015 there is a massive increase or exponential increase in data. This data can be collected from Twitter or Facebook or whatever you search in from your mobile. So whatever we do in this digital era, we are living in digital footprint and these are raw data. These raw data are inconsistent in nature, incomplete does not have any certain or particular behavior or trends and most likely it will harm noise, there is error. So these data can be refined and distilled. And uh after that, if you put some statistical advanced tool in that, then we can get valuable insights and this valuable insights we can convert into information and which in turn can be converted to knowledge. And this knowledge can be used to solve both uh technical and productivity problem of analytic business. Business. And these are mainly done by two that is business intelligence and business analytics. Business intelligence is used to deliver relevant and reliable information to right people in the right format at right time. Here, it is very important that we are delivering the information at the right time. So bie to make better decisions faster. Whereas business intelligence, it is having uh uh artificial intelligence, statistical and operation research technique, information technology and management strategies. And by using these knowledge, we are making some valuable insights. This can be done by correlation analysis, regression analysis, text mining and many much thing like that.
Mainly business intelligence deal with descriptive and diagnostic analysis. Whereas business analytics lead with perspective and predictive analytics. So in a nutshell, we can explain these for analytics that is description analytics is what happened. Diagnostic is why did it happen and predictive is what will happen.
And prescriptive is what should I do? So you can see the first two that is descriptive and diagnostic, is that something already happened? And then we are analyzing the data, where are the later two, it is forecasting the future. So the first two is unreactive method, something happened, then we are working on that. Whereas later two is proactive approach. So we can say that when we put some prediction into business intelligence, we get business analytics that is business analytics, a bigger uh umbrella and business intelligence is a subset to that descriptive. We can use for revenue of company or sales report and the diagnostic we can use for employee performance analysis with hr department predictive. We can use for forecasting weather predic predict uh weather. And the also for company customer churn, we can see the likelihood of that prescriptive analysis always deals with when optimization come that is optimization of bed capacity and overtime shift in a hospital price and promotion optimization for retail. Then we can say artificial intelligence, artificial intelligence is the science and engineering of making intelligent machine, any technique which enable computer to mimic human intelligence such as decision making, text processing, visual representation, sorry, visual perception A I is a broader field for a big umbrella which contains several subfields such as machine learning, robotics, computer vision, et cetera.
So we have data sense where we deal with data and if we use data sense to make intelligent decisions, then we are using artificial intelligence. So we can see that the Venn diagram, the intersection A I A is now being used in almost every sector like transportation, health care, re banking, retail, entertainment, e-commerce. And we all all have and hype that if artificial intelligence come, then the our job will be at risk.
But actually it is not like that human germ won't go anyway, but they will change role will be more creative and specialized as A I is integrated into workday, better data lead to better Macs which lead to better prediction. So people using A I can automate the tedious work and track action to the inside. Artificial intelligence can be classified based on capabilities and functionalities. When we classify it based on capabilities, we have narrow intelligence, general intelligence and superintelligence.
Narrow intelligence is also known as weak intelligence and it is only specific to one area and it cannot work beyond that. It is having little limitation and it is widely used for natural language processing, IBM Watson and Siri is best example and we are also having many other like image recognition system recommendation system spam filtering the natural. When we come. When human communicate with each other with the help of text or via verbal communication, we are using natural language. The same frontal is used here. Natural language processing or N LP refers to branch of artificial intelligence that give the machine ability to read, understand and derive meaning from human language. Robots such as Sophia or human assistance, used uses natural languaging process to sound like human and understand what we are saying.
Then comes general intelligence. It is a strong one, it can understand and learn any individual intellectual task that a human being can and we are not completely developed into it. But we are in the phase of development. Fujitsu has built K computer which is one of the fastest supercomputer in the world. It is a significant attempt towards strong intelligence and it takes around 40 minutes to stimulate a simple second of neural activity. Then last come artificial superintelligence. It is only hypothetical it should be above human activity in all sense that it should surpass human intelligence and can perform any task better than human. It should be smarter than human in every way. And when we classify intelligent artificial intelligence based on functionality, it can be divided into reactive machine, limited theory, mind and self-awareness, reactive. It is like the primary form of artificial intelligence. It does not store any memory, it does not have any cache. It works on present data.
Best example is IBM Deep Blue that defeated chess grandmaster Gary Kasparov and then comes limited theory. Limit theory is like it has limited uh memory access, but it is very short term short-lived and they can use this past data for specific period while they work on past data and it works only for short period. This is best example is like self-driving car. They are included with vehicle devices which went to change lane, avoid cutting off another driver or hit a nearby vehicle. Then theory of mind, it is also like not completely developed kismet and can mimic human emotion and recognize them is one of the world, uh real world example of this, but it cannot follow grace or convey attention to human theory of mind. The example is Robert Sophia. She has an uh combined computer algorithm which allow in her eye which allow her to see she can sustain eye contact, recognize individuals and follow face self awareness. It also is an hypothetical one. It all at uh self system, understand the internal right state and condition and pres human emotion. These machines will be smarter than human mind.
Then these are the famous quotes of great personality, Elon Musk says that A I does not have to be able to destroy human if A A I has a goal and humanity just happens in the way it will destroy human as a matter of course without even thinking about it, no heart failings. Whereas Mark Cuban says that artificial intelligence, deep learning and machine learning, whatever you are doing if you don't understand it, learn it because otherwise you are going to be a Dino sir within three years. And father of deep deep learning, Jeffrey Hinton says that in deep learning, the algorithm which we are using it is of 19 eighties and 19 nineties people were very optimistic about them, but it turned out they did not work too well. And he has worked lots of lots and lots of work in deep learning. And he has published many articles in the A the learn machine learning machine learning provides A I the ability to learn without explicit programming. Here we meet machine to work. Machine will program. Hm. Machine learning is achieved by algorithm that discovers pattern and generate insights from data. They are exposed to machine learning access, vast amount of data, both structured and unstructured and learn it from to predict the future. When we class uh mark artificial intelligence, machine learning and deep learning by an Venn diagram. We can see that artificial intelligence is a broader umbrella and machine learning is a way of achieving artificial intelligence. Likewise, deep learning is a way of achieving machine learning artificial intelligence.
We know that anything that imitate human behavior and application of a that allows the system to automatically learn and improve from experience and deep learning mimic our neural network of the human brain. Machine learning algorithm can be classified into supervised learning, unsupervised learning and reinforcement learning.
Supervised learning is like making a child learn. If you want to want them to identify some cat or a dog, we have to show them a few pics the same as work for supervised learning, your data is already labeled. So we know that what our target is it required at least an input and output variable be given to the model for it to be trained. It is somewhat like this. We have input data and annotations that we feed to the machine. So when we put an input to the machine, it will predict that what the input is. Example of supervised learning is linear regression logistic regression support vector machine, new base and decision trade, unsupervised learning. It is like we don't have label data and we make the machine to understand the design pattern. The system can identify hidden features from input data provided. Once the data is more readable, the pattern and similarity become more evident. It will be like we will be putting lots of input without any annotations and the model will learn and then it can divide and classify into the group. Example of unsupervised learning is gaming, clustering, hierarchical clustering and anomaly detection reinforcement learning. Here.
It is a combination of supervised and unsupervised learning here. Minimum data training data available. At that time, we use reinforcement learning and also we don't have much idea of the environment and it learned the environment by interacting with it algorithm. Act on the environment example, make a trade in financial portfolio algorithm optimizes for best series of action by correcting itself over time. And when we have to design a machine learning algorithm, we have to first uh collect the data data processing, choose the model train the model test the model feature scaling and predictions. We will learn one by one. First one is data collection. There are five common methods by which we can collect data that is closed survey and cues other is open ended survey and question or 1 to 1 interview focus group or direct observation. Once we have collected data, we have to check identifying and handling missing data. That is if any numerical value is missing, then we can go for mean of the all value and put it in the column that inconsistent value. For example, we are having someone is entering the data in computer that time they may have made a mistake like address and pin number are not matching. In such cases, it become inconsistent values. So we have to treat that manually then comes duplicate value.
There is a chance that two people enter the same value if duplicate value is that we have to delete one value that encoding categorical data. For example, we are taking a population of three main three country. For example, India us and UK. In that case, we uh we cannot feed the system categorical value. We have to make better like we have to put India as 100 us, S 010 and UK as 001. Then next is choose the model for choosing the model. We can uh we can have it first have a scattered plot of the independent variable, each independent variable and dependent variable. When we have a scattered plot. We can see the uh relation between two if they are related or they are scattered. And then we can go based on scattered report, we can decide which model we have to take that we have already learned supervised and unsupervised learning many uh like in a regression logistic. So we can choose from one of the model. Then resampling method, it is an advanced like more pre Carlo stimulations can be used for this. Then after that, we have to split the test, we have having a test data. So we are having 1000 test data. In that we have to split into training set and test set. Training set, we have to divide into 80% test to 20%.
Then training set, we have to evaluate, we have to train the system with training set and test set is used for evaluation and evaluation of model uh is done to check the performance of system. Then come feature scaling for all the algorithm we don't go for feature scaling feature scaling are used such so that all the features that is variable are scaled to make sure that all take same scale. It is done to prevent that one feature does not dominate the other. For example, if you are having multiple linear recre uh regression, it can be represented by this equation. So by using normalization or standardization, we can go for feature scale then prediction once our model is there we have train test and we have feature scale, then we can go for prediction, predicting the future value. This can be done by putting system and life examples of machine learning is automatic uh tagging of friend in Facebook. When we put a photo, we automatically we know that we get a tag. This person is name is this. So like it is by using the face algorithm, then Uber, Uber uh calculates shortest distance and the demand. They are surprising. Everything is calculated basing on machine learning application.
Then virtual personal assistant as Siri Alexa Google Cortana, everything are using MLML learn is used for speech recognition, speech to text conversion, natural language processing text to speech conversion. Then Google translate, use neural machine translation and N LP for uh translation.
Then Netflix, we know that when you log into Netflix, it gives us the recommendation. It is done by uh collecting the data. We do that is searching browsing, start stop rewind by collecting all this data. We they recommend the film for us. Then next is fraud detection when we use credit card or debit card. Whenever a customer carries out the transaction. ML model thoroughly scan accel their profile searching for suspicious activity. Then at sports here we don't have joystick but by seeing our actions and move uh and then uh play with this, then it is virtual reality set as we move the picture also move the robot dog. It is used for uh making kids learn walking and also it is appears in fiction. Then other example are audible recommendation, Amazon recommendation, Facebook, a recommendation we use ML for mass exploration program. Then in drugs, we use behavior modeling we use. Then in lead planning, the major difference between ML and AD L is that Mle uses our mimic our neural network. But whereas ML works only on structured data, this is the example of neural network and provides a, a ability to mimic human brain. It is like input layer we feed and the fetal layer identifies the pattern and get the output with this uh retina.
By scanning retina, they identify ML, sorry DL identifies if the patient is diabetic or not, then it is calculated by when we put, provide an input, it provide a weight for each one. Then it goes to an activation function. Activation function identifies the the input and provide a bias. It determines whether the input should be uh filtered or not. Once the input according to the activation, the value enters and once it enters output uh layer predict the value. These are the other applications DLR used. Thank you.
You can uh tell uh write down my email ID. If there is any doubt or anything, which I can help. Please let me know I will try to contact soon. Thank you, everyone.