Logistic regression confusion

When you try to acquire knowledge about machine learning you will most likely find out that it consists of two main groups of algorithms called supervised machine learning and unsupervised machine learning. Furthermore supervised machine learning can be divided into regression and classification.

The difference between classification and regression is that classification output is discreet (for example if we have an image at the input and want to have 1 at the output if that is a dog and 0 if that is a cat) while the regression output is continuous (this is useful if for example we want to predict stock price based on historical data about that stock because the stock price is a continuous number).

Machine learning algorithms have different names. One of them is called a “logistic regression”. In summary it can take any number of continuous inputs and return a continuous number between 0 and 1. It produces that number by passing weighted sum of the inputs through a sigmoid activation function.

The confusing part is that many sources treat logistic regression as a classification algorithm! This can sound strange for at least two reasons:
1. it has “regression” in the name
2. it doesn’t return discreet values (and discreet values are needed if we want to solve classification problems).

So logistic regression a regression or a classification algorithm? The answer is it can be both depending on how you look at it! It is a regression algorithm because it produces a continuous number but it is can become a classification algorithm if we add just one step: return 1 if the output is greater or equal to 0.5 and 0 if the output is less than 0.5. This means that if we interpret the outcome slightly differently it becomes a classification instead of regression.

I think that above example shows how regression and classification are not really completely different things but instead they can have a lot in common in a sense that classification can actually be based on regression which may be confusing if you’re just starting to learn machine learning!