Data science involves a plethora of disciplines and expertise areas to produce a holistic, thorough and refined look into raw data. Data scientists must be skilled in everything from data engineering, math, statistics, advanced computing and visualizations to be able to effectively sift through muddled masses of information and communicate only the most vital bits that will help drive innovation and efficiency.

Data scientists also rely heavily on artificial intelligence, especially its subfields of machine learning and deep learning, to create models and make predictions using algorithms and other techniques. 

Reference

https://builtin.com/data-science

Machine learns through experience

Like humans,

machines also learn from experience. They analyse the existing data, figure out the pattern, and find what to do with the new data.

Learning from Medical records

based on combination of attributes i.e., symptoms, it determines an outcome variable i.e., whether the patient is covid positive or not by learning from the existing data.

Learning from Energy usage patterns

smart homes optimize the energy usage by learning from the energy usage patterns of its residents (by capturing the motion of residents), hence saving the electricity bills.

Learning from User interests

learns from user interests and gives suggestions and recommendations accordingly

Learning to target customers

learns from the travel history of customer and gives discounts, coupons, offers to the customers travelling more often during a certain period, to attract them to travel more (to retain the customer to themselves)

Machine learning in finance

• Financial Monitoring

Simultaneously multiple micro transactions to multiple accounts.

• Risk Management

Giving loans based on credit score of the customer. (Credit worthiness)

• Money laundering prevention

Machine learning in medicine

• Personalized medical reports and treatments
• Smart watches
• Predicting whether a cell is cancerous cell or not
• Based on the attributes (lifestyle, food habits, exercise, sleep etc.), determining whether a person is prone to certain disease (say cardiac arrest).

Equipment maintenance records

Combustion chamber, rockets

The unsupervised learning algorithm can be further categorized into two types of problems:

Clustering

Clustering is a method of grouping the objects into clusters such that objects with most similarities remains into a group and has less or no similarities with the objects of another group.

Cluster analysis finds the commonalities between the data objects and categorizes them as per the presence and absence of those commonalities.

Association

An association rule is an unsupervised learning method which is used for finding the relationships between variables in the large database. It determines the set of items that occurs together in the dataset.

Association rule makes marketing strategy more effective.

Such as people who buy X item (suppose a bread) also tend to purchase Y (Butter/Jam) item. A typical example of Association rule is Market Basket Analysis.

Unsupervised Learning algorithms:

K-means clustering

KNN (k-nearest neighbors)

Hierarchal clustering

Anomaly detection

Neural Networks

Principle Component Analysis

Independent Component Analysis

Apriori algorithm

Singular value decomposition

Advantages of Unsupervised Learning

Fast process as task of data labelling is not involved.

Provides unique and disruptive insights.

Disadvantages of Unsupervised Learning

The result of the unsupervised learning algorithm might be less accurate as input data is not labelled, and algorithms do not know the exact output in advance.

Difficult to measure accuracy

Dealing with high-dimensional data- when the dimension of data and the number of variables is more, the process becomes difficult.

As the name suggests, unsupervised learning is a machine learning technique in which models are not supervised using training dataset.

Instead, models itself find the hidden patterns and insights from the given data.

It is much similar as a human learns to think by their own experiences, which makes it closer to the real AI.

In real-world, we do not always have input data with the corresponding output (training or labelled data referred to as supervisor). So, to solve such cases, we need unsupervised learning.

Steps:

Step 1: The very first step is to load the unlabeled data into the system.

Step 2: Once the data is loaded into the system, the algorithm analyses the data.

Step 3: As the analysis gets completed, the algorithm will look for patterns depending upon the behavior or attributes of the dataset.

Step 4: Once pattern identification and grouping are done, it gives the output.

Example: Input dataset containing images of different types of fruits.

Now, let’s take these fruits and feed them to an unsupervised learning model.

The model determines the features associated with the data and understands that all the apples are similar in nature and thus groups them together.

Similarly, it understands that all the oranges have the same features and thus group them together and the same is the case with all the mangoes (in case we have mangoes in this example)

Here, the unsupervised learning algorithm will perform this task by clustering the image dataset into the groups according to similarities between images.

Example 2:

For instance, given a data base of movie reviews, you could identify clusters of users who rate action movies similarly, and use those correlations to predict how one member might like a particular movie he had not yet seen, but others have rated.

1. Classification

Supervised learning can be further divided into two types of problems:

Classification algorithms are used when the output variable is categorical, which means there are two classes such as Yes-No, Male-Female, True-false, etc.

Multiple classes may also be present.

The output variable or the dependent variable should be categorical in nature.

Example:  Diagnosis

“Prone to lung cancer” (output variable) is the dependent variable and “Weight” and “Number of cigarettes smoked” are the independent variables.

1. Regression

Regression algorithms are used if there is a relationship between the input variable and the output variable. It is used for the prediction of continuous variables, such as Weather forecasting, Market Trends, etc.

• Analyse the existing data and
• Predict the future data parts.

Let’s say you have two variables, “Number of hours studied” & “Number of marks scored”. Here we want to understand how the number of marks scored by a student change with the number of hours studied by the student, i.e.

“Marks scored” is the dependent variable, and “Hours studied” is the independent variable.

You need to note that “marks scored” is the dependent variable and it is a continuous numerical.

Question: “How many hours should a student learn to get 60 points?” 

Ans: The regression model would understand that there is an increment of 10 marks for    every extra hour studied and to score 60 marks the student must study for 6 hours.

Example: Weather app in our mobile

                This app predicts the weather of the entire next week. How does it do?

By analysing the previous data (say past 10 years weather report data) and predicts the pattern for the next week.

Here, since we deal with large amount of data, it may be difficult for humans to work on it. Hence, the machines are fed with large amount of data and made to predict the future data parts.

In supervised learning, models are trained using labelled dataset, where the model learns about each type of data.

Once the training process is completed, the model is tested based on test data (a subset of the training set), and then it predicts the output.

Example:

Task

Suppose we have a dataset of different types of shapes which includes square, rectangle, triangle, and Polygon. Now the first step is that we need to train the model for each shape.

Training Experience

If the given shape has four sides, and all the sides are equal, then it will be labelled as a Square.

If the given shape has three equal sides, then it will be labelled as a triangle.

If the given shape has six equal sides, then it will be labelled as hexagon.

Now, after training, we test our model using the test set, and the task of the model is to identify the shape.

Performance

The machine is already trained on all types of shapes, and when it finds a new shape, how well it classifies the shape based on number of sides and predicts the output.

Steps involved

• First, determine the type of training dataset
• Collect/Gather the labelled training data.
• Split the training dataset into training dataset, test dataset, and validation dataset.
• Determine the input features of the training dataset, which should have enough knowledge so that the model can accurately predict the output.
• Determine the suitable algorithm for the model, such as support vector machine, decision tree, etc.
• Execute the algorithm on the training dataset. Sometimes we need validation sets as the control parameters, which are the subset of training datasets.
• Evaluate the accuracy of the model by providing the providing the test set. If the model predicts the correct output, which means our model is accurate.

References

https://www.javatpoint.com/supervised-machine-learning

The machines are trained using well “labelled” training data, and on basis of that data, machines predict the output.

The labelled data means some input data is already tagged with the correct output.

Supervisor is this training data (labelled data) which helps to predict the output correctly when a new input data point is given as input.

The aim of a supervised learning algorithm is to find a mapping function to map the input variable(x) with the output variable(y).

Step 1: The very first step of Supervised Machine Learning is to load labelled data into the system. This step is a bit time-consuming because the preparation of labelled data is often done by a human trainer.

Step 2: The next step is to train and build connections between inputs and outputs(function). This step is also known as the training model.

Step 3: Then comes the step known as the testing model. As the name suggests, you test the model by introducing it to a set of new data.

Here, the input is an independent variable, and the output is a dependent variable. The goal is to generate a mapping function that is accurate enough so that the algorithm can predict the output when we feed new input.

Example of labelled data:

We have a labelled dataset that consists of images of apples and oranges, with different attributes such shape, colour etc.

Consider the image of an apple shown above with the labels- shape, colour, and apple.

We train the model with this image. Then, we repeat the same training process with other images of both apples and oranges with their attributes.

What we are doing is-

Here, the input data is the independent variable and “Apple” or “Orange” is dependent variable as it is dependent on the input picture given.

Our goal is to generate a mapping function between the dependent and independent variable so we can determine the output when we feed a new data point.          

            

Once the model is trained and the algorithm is built, the accuracy can be tested with the help of a test dataset.

When we feed the model with a new apple image, it scans the image and matches the attributes of the image with other trained images. Then depending upon the accuracy of the model, it returns the output ‘apple’.

When new data point is given as input, say,

The machine should be able to guess the output as  “Apple”.

This labelled data or the training data (acts as supervisor), helps to predict the output as “Apple”.