Fit and Transform

This is a very simple, yet useful concept to understand.

.fit is used to execute the operation being commanded, such as calculating the mean, most frequent, or median.

After the operation is conducted, it is then entered to the data by using .transform method.

Backward Elimination

Backward Elimination is selection technique to remove those features that don’t have a significant effect on the dependent variable or prediction of output.

Dummy Variable Trap in Regression

Assume we have variable of sex: female and male. If in the analysis we use two dummy variable, let’s say female is represented by f and male represented by m, there will be a problem of multicollinearity (one value can be predicted from the other values). This is caused by the fact that the the row of f and m in the matrix is highly correlated: The i-th row in the f is correlated with the i-th row in the m. For example: if i-th row in f is 0, so the i-th is 1, vice versa. Thus, the determinant will be 0.

Categorical and Continuous Data

Categorical data is a type of data that can be distinguished between groups and we can list a small number of categories for it. This includes product type, gender, age group, country, etc. It has a finite amount of number.

Label Encoder and One Hot Encoder

Some of you might confuse about these two encoders which are parts of Scikit learn library in Python. The encoder is used to convert categorical data or text data into numbers. Why it should be converted? Because model only understand the numbers.

Confusion Matrix

After building a model, of course we will test it. And yeah we got the result. But wait, how do we measure the effectiveness of our model? That’s what confusion matrix made for.

Super Important Shortcut in Jupyter Notebook

1. A to insert a new cell above the current cell
2. B to insert a new cell below.
3. M to change the current cell to Markdown

iloc

iloc is used for filtering rows and columns based on integer position. For example:

Using the Terminal

Terminal is absolutely important for every programmer. We have to be tremendously familiar with it. Here is the list of foremost keywords for using the terminal.

Underfitting and Overfitting

It is a common problem for machine learning to overgeneralize or too specific on the training data. When a model is too generalize it is called underfitting. While when the model is too specific or follow the exact