Machine learning for beginners: A brief introduction
This post is for you who have no knowledge about machine learning, want to learn but do not know where to start. In this post, I will begin with a didactic example of our daily life showing how we do to learn to estimate the value of a house and then show how we can make the machine "imitate" our learning.
Starting with an example
Imagine that you are a real state agent and your job is to estimate the value of a house or apartment to put it up for sale. So, for each new property, you need to evaluate its different features and then, based on your experience with other sells, estimate a suitable value for sale.
We can consider many features when evaluating a property: size, number of rooms, number of bathrooms, size of the yard, number of parking spaces, the neighborhood where it is located, etc, etc, etc. To make things easier, let's imagine that the value of a property is defined considering only one feature: the total size of the property in m².
It would be great to train a machine to "learn" how to estimate the value of a property and help you in this task (especially if we consider all possible characteristics). But before we see how we can do this with machines, let's check how we (humans) do to estimate this value. Notice that I highlighted two key points when predicting the price of a property: (1) Use historical information of previously sold properties and (2) analyze the characteristics of the property to be evaluated. Let's see how to combine these two pieces of information.
How we learn
Over the years selling houses, you have arranged in a spreadsheet all the properties you sold, relating its features and the final sale price, as follows:
Historical sales data - You will learn from them
To help us visualize these data, let's plot a scatter plot:
Analyzing a chart is easier than a spreadsheet full of numbers
In the figure above, each point represents a property. On the x-axis, we have the size information in m² and the y-axis we have the sale price. Analyzing the graph as a whole, we can see that there is a particular pattern between the size of the property and its price, that is, the larger the property, the higher its price is. (For this example this is pretty obvious, but remember that we are only considering ONE feature. Imagine if we had 60?!).
With the chart in hand, it is easy to define the value of a new property. For example, how much would cost an 1100 m² property?
From the chart we can estimate that a house of 1100m² costs approximately US$500,000.00
This means that we can draw a line that fits along the data we have and this will allow us to predict the value of new properties by simply checking at which point of the line is the value of y (property price) according to x (size of property). Of course that the straight line will not accurately predict the value of all properties in the world. In our example, we have numerous properties that are a little below or above the line, we also have properties with the same size but were sold for different prices. No human being would be able to hit 100% of the time the exact amount that a property will be sold, but it is possible to find a line that is as close as possible to the true value. Which of the following lines do you think best estimates the price of a property?
Which of the lines best fits the data?
Hope you have chosen the line from the right! :)
Okay, we used historical data to plot a graph that allowed us to "learn" how the size of the house relates to its selling price (finding the line) to be able to predict the price of a new property. We did all this in a visual and intuitive manner, but for the machine to do the same, we need to use some math. In the next section, we'll see how.
How the machine learns
Analyzing the historical data, we realized that if we define a line that fits the data, we will be able to predict the value of a new property. When we talk about training a machine to learn a model capable of predicting house prices, we are just talking about learning the equation of a line that fits over the existing data and will allow us to predict new data. For this, we will use a technique known as supervised learning. In this type of learning, we use data with the "correct answer" so that the machine learns to make inferences about them. In our case, the data are the houses features that have already been sold, and the "answer" is the sale price. During training with these data, the machine will learn how the houses features relate to their final sale price and will determine the line that best represents those data. The big question is how to make the machine learn how this line will be: what will be its inclination and where will it cut the y-axis.
First, let's recall the line equation with the conventional machine learning notation:
\(h(x) = \theta_0 + \theta_1 x\)
Don't be scared by these letters, we are looking at the line equation we already know! :) Bringing this equation to our example, h(x) would be the "y" in the line equation, it represents the value we want to find (the price of the property) and, x is a characteristic of the property (its size in m²). The others, theta0 and theta1, are the parameters of the line equation that we have to learn to better fit the line into our data. Let's see how they influence the behavior of the line:
Note that theta0 represents the point where the line intersects the y-axis and theta1 represents the slope of the line. In the first example of the figure above we have that theta0 = 1.5 (precisely the point where the line intersects the y-axis) and theta1 = 0, that is, it is a straight line without any inclination. The other two examples work analogously.
During the learning process, the machine needs to find out which theta values form the line that best fits the data. This means that by experiencing theta values, we have to check with our training data how far they are from the line. To exemplify, let's assume that we want to try values for two sets of theta: (theta0 = 60000, theta1 = 468) or (theta0 = 80000, theta1 = 528) and we are using information from only 4 houses as training. See the figures below the houses information in the worksheet and the graph of those data with the straight lines using these theta values we want to try:
To know which is the best set of theta, we have to check which one has fewer predictions errors. We do this through the difference between the value predicted by the model (on the line) and the real value of the training. See the spreadsheet below for the calculations:
| Size in m² | Real price |
Predicted price using |
Real price - predicted price |
| 260 | $179,900 | $181,680 | $1,780 |
| 915 | $539,900 | $488,220 | $51,680 |
| 1186 | $573,900 | $615,048 | $41,148 |
| 1365 | $699,900 | $698,820 | $1,080 |
| $95,688 | |||
Note that when using the first set of theta, our model predicted that a 260m² house would cost $ 181,680.00, but in fact, it cost $ 179,900. This means that we are off by $ 1,780.00. By summing up all the differences in our training data, we have the total error of the model, which in this case was $ 95,688.00. Let's look at the error for the second set of thetas:
| Size in m² | Real price |
Predicted price using |
Real price - predicted price |
| 260 | $179.900 | $217.280 | $37.380 |
| 915 | $539.900 | $563.120 | $23.220 |
| 1186 | $573.900 | $706.208 | $132.308 |
| 1365 | $699.900 | $800.720 | $100.820 |
| $293.728 | |||
For the second set of theta's, the total error is $ 293,728.00, which is much higher than the first case. So, the first configuration of theta's is best to estimate the value of the houses.
Let's zoom in on the first case to understand better how this difference between what was predicted and the real value is reflected in the graph:
Note that the prediction error represents the distance between the real training data and the line we are testing. For a house with 915 m², the line predicts that it would have the value of $ 488,220.00, while its real price is $ 539,900.00, we err in $ 51,680.00.
In practice, we do not go around testing all possible combinations of theta to choose the one that has the lowest error. Some techniques allow us to "walk toward" the theta that will minimize the model error. When we find the theta's which line has the lowest error during the training process, the prediction model is ready to be used! That is, the machine "learned" the best parameters to solve this particular problem.
This process that we have just seen in this post, which defines a line that best fits the data is called linear regression, and I will present more in-depth how to estimate its theta parameters in the next post.
Much more ahead
This was a short introduction to machine learning, showing how we can teach a machine to estimate the value of a house. The technique we used here (linear regression) is just one of several existing methods that we can apply in machine learning. Choosing which approach to use will depend on the nature of the problem you want to solve. The most important of this little introduction is to understand the learning mechanism used by the machine. In general, everything works in the same way we have seen here: choose a technique and make the machine learn the necessary parameters. In the example of this introductory post, we decided to use linear regression to solve our problem and the machine just had to learn the theta parameters. For other types of problems, there are other techniques that we can use and the parameters that the machine needs to learn.
