09_feature_scaling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1ae22583",
   "metadata": {},
   "source": [
    "<table align=\"left\">\n",
    "  <td>\n",
    "    <a href=\"https://colab.research.google.com/github/nyandwi/machine_learning_complete/blob/main/4_real_world_data_analysis_and_preparation/4_feature_scaling.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45497309",
   "metadata": {},
   "source": [
    "*This notebook was created by [Jean de Dieu Nyandwi](https://twitter.com/jeande_d) for the love of machine learning community. For any feedback, errors or suggestion, he can be reached on email (johnjw7084 at gmail dot com), [Twitter](https://twitter.com/jeande_d), or [LinkedIn](https://linkedin.com/in/nyandwi).*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73eca2c6",
   "metadata": {
    "id": "73eca2c6"
   },
   "source": [
    "#  A Handy Notes about Feature Scaling"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f755b200",
   "metadata": {
    "id": "f755b200"
   },
   "source": [
    "Machine Learning models are very selective about the type and range of values that have to go for their input in order to work well. With the exception of decision trees, most ML models will expect you to scale the input features. What is feature scaling?\n",
    "\n",
    "Feature scaling is nothing other than transforming the numerical features into a small range of values. In this notebook, we will see the following scaling technique:\n",
    "\n",
    "* Normalization\n",
    "* Standardizatiom\n",
    "* Robust Scaling\n",
    "\n",
    "Normalization and Standardization can be used or applied interchangeably, but they are quite different and they are suited for different purposes. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b21c5e41",
   "metadata": {
    "id": "b21c5e41"
   },
   "source": [
    "## 1. Normalization"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cbaae7d",
   "metadata": {
    "id": "3cbaae7d"
   },
   "source": [
    "Normalization is a scaling techniques that transform the numerical feature to the range of values between 0 and 1. \n",
    "\n",
    "Here is a formula that is followed when normalizing the data. $Xmin$ is the minimum value of feature X, and $Xmax$ is the maximum value of X. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "EXImNPvtADvi",
   "metadata": {
    "id": "EXImNPvtADvi"
   },
   "source": [
    "$$\n",
    "Xnorm = \\frac {X-Xmin} {Xmax-Xmin}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14c25a1d",
   "metadata": {
    "id": "14c25a1d"
   },
   "source": [
    "#### When Should you Normalize the Features?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2K8Ky-aW6Sw1",
   "metadata": {
    "id": "2K8Ky-aW6Sw1"
   },
   "source": [
    "When you have features that have different ranges of values, normalizing these features can be a good practice. \n",
    "\n",
    "Take an example. If you have two features that have different ranges (say one feature from 1-100, other vary from 5-300), you will to scale them so they have the same range of values. \n",
    "\n",
    "More specifically, normalization is a preferrable scaling technique when the data at hand has not a normal or gaussian distribution. If the data's distribution is gaussian, standardization is a preferrable scaling technique. If you don't know the distribution of the data, still, normalization is a good choice at first. \n",
    "\n",
    "With that said, when the ML algorithm of choice is either neural network or K-Nearest Neighbors(KNN), normalization is a good choice for these type of algorithms because they don't make any assumption of the input data. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "I67mjU6m56MO",
   "metadata": {
    "id": "I67mjU6m56MO"
   },
   "source": [
    "Most popular ML frameworks provide functions to normalize the numerical data. \n",
    "\n",
    "For illustration purpose, I will use tips data available in Seaborn. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d20d3476",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 203
    },
    "id": "d20d3476",
    "outputId": "ec767f94-6682-4df8-8155-3dc13faef941"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>total_bill</th>\n",
       "      <th>tip</th>\n",
       "      <th>sex</th>\n",
       "      <th>smoker</th>\n",
       "      <th>day</th>\n",
       "      <th>time</th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>16.99</td>\n",
       "      <td>1.01</td>\n",
       "      <td>Female</td>\n",
       "      <td>No</td>\n",
       "      <td>Sun</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10.34</td>\n",
       "      <td>1.66</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Sun</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21.01</td>\n",
       "      <td>3.50</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Sun</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>23.68</td>\n",
       "      <td>3.31</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Sun</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>24.59</td>\n",
       "      <td>3.61</td>\n",
       "      <td>Female</td>\n",
       "      <td>No</td>\n",
       "      <td>Sun</td>\n",
       "      <td>Dinner</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   total_bill   tip     sex smoker  day    time  size\n",
       "0       16.99  1.01  Female     No  Sun  Dinner     2\n",
       "1       10.34  1.66    Male     No  Sun  Dinner     3\n",
       "2       21.01  3.50    Male     No  Sun  Dinner     3\n",
       "3       23.68  3.31    Male     No  Sun  Dinner     2\n",
       "4       24.59  3.61  Female     No  Sun  Dinner     4"
      ]
     },
     "execution_count": 2,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from seaborn import load_dataset\n",
    "\n",
    "tip_data = load_dataset('tips')\n",
    "tip_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c87d399",
   "metadata": {
    "id": "0c87d399"
   },
   "source": [
    "Let's take all numerical features from the above data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f49318c8",
   "metadata": {
    "id": "f49318c8"
   },
   "outputs": [],
   "source": [
    "num_feats = tip_data[['total_bill', 'tip', 'size']]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e806119",
   "metadata": {
    "id": "2e806119"
   },
   "source": [
    "For now let's scale those numerical features with Scikit-Learn preprocessing functions. We will use `MinMaxScaler` which scale the data to the range between 0 and 1 by default. If you want a different range, you can change that. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d63dd722",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "d63dd722",
    "outputId": "ee8c8547-be58-4408-a0fc-f6793b589675"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.29157939, 0.00111111, 0.2       ],\n",
       "       [0.1522832 , 0.07333333, 0.4       ],\n",
       "       [0.3757855 , 0.27777778, 0.4       ],\n",
       "       [0.43171345, 0.25666667, 0.2       ],\n",
       "       [0.45077503, 0.29      , 0.6       ]])"
      ]
     },
     "execution_count": 7,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "scaler = MinMaxScaler()\n",
    "\n",
    "num_scaled = scaler.fit_transform(num_feats)\n",
    "num_scaled[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a850c99",
   "metadata": {
    "id": "2a850c99"
   },
   "source": [
    "The output of the scaler is a NumPy array. You can convert it back to a Pandas DataFrame. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6a757301",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 203
    },
    "id": "6a757301",
    "outputId": "b40c142f-78f3-4f53-f665-e56b0fb2df73"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>total_bill</th>\n",
       "      <th>tip</th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.291579</td>\n",
       "      <td>0.001111</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.152283</td>\n",
       "      <td>0.073333</td>\n",
       "      <td>0.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.375786</td>\n",
       "      <td>0.277778</td>\n",
       "      <td>0.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.431713</td>\n",
       "      <td>0.256667</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.450775</td>\n",
       "      <td>0.290000</td>\n",
       "      <td>0.6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   total_bill       tip  size\n",
       "0    0.291579  0.001111   0.2\n",
       "1    0.152283  0.073333   0.4\n",
       "2    0.375786  0.277778   0.4\n",
       "3    0.431713  0.256667   0.2\n",
       "4    0.450775  0.290000   0.6"
      ]
     },
     "execution_count": 5,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd \n",
    "\n",
    "num_scaled_df = pd.DataFrame(num_scaled, columns=num_feats.columns)\n",
    "num_scaled_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1161d3b5",
   "metadata": {
    "id": "1161d3b5"
   },
   "source": [
    "As you can see above, all the values are scaled to the values between 0 and 1. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27354aa2",
   "metadata": {
    "id": "27354aa2"
   },
   "source": [
    "## 2. Standardization"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "BfCJFRN-Cb2A",
   "metadata": {
    "id": "BfCJFRN-Cb2A"
   },
   "source": [
    "In standardization, the numerical features are rescaled to have the 0 mean($u$) and unity standard deviation(std or $\\sigma$ ). \n",
    "\n",
    "Here is the formula of standardization. Xstd is the standardized feature, X is the feature, $u$ is mean of the feature, and $\\sigma$ is the standard deviation. \n",
    "\n",
    "$$\n",
    "Xstd = \\frac {X - u} {\\sigma}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae039d4b",
   "metadata": {
    "id": "ae039d4b"
   },
   "source": [
    "#### When Should you Standardize the Features?\n",
    "\n",
    "When you know that the training data at hand has a normal or gaussian distribution, you should standardize such data. \n",
    "\n",
    "Some ML algorithms such as Support Vector Machines(with rbf kernel) and linear models expect that the input data to have a normal distribution. \n",
    "\n",
    "In most cases, whether you choose normalization or standardization, it won't make much difference, but it can. So, it makes sense to try both especially if you are not sure about the distribution of the data.\n",
    "\n",
    "Here is how Standardization is implemented in Scikit-Learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "QNsSSG-SHi6U",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "QNsSSG-SHi6U",
    "outputId": "42e103cd-8a49-4d9f-c73b-1d60eedc049a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.31471131, -1.43994695, -0.60019263],\n",
       "       [-1.06323531, -0.96920534,  0.45338292],\n",
       "       [ 0.1377799 ,  0.36335554,  0.45338292],\n",
       "       [ 0.4383151 ,  0.22575414, -0.60019263],\n",
       "       [ 0.5407447 ,  0.4430195 ,  1.50695847]])"
      ]
     },
     "execution_count": 19,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "std_scaler = StandardScaler()\n",
    "num_std = std_scaler.fit_transform(num_feats)\n",
    "num_std[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "X6JC0b2uLuwq",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "X6JC0b2uLuwq",
    "outputId": "6f2f99fe-bec8-4f8f-e376-f65cde9da0ea"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([19.78594262,  2.99827869,  2.56967213])"
      ]
     },
     "execution_count": 14,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The mean of each feature in the scaled data\n",
    "\n",
    "std_scaler.mean_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "eQ5huG8aL-UJ",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "eQ5huG8aL-UJ",
    "outputId": "e0df31e0-86e0-40c4-ebc3-23b8e9598e3c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([78.92813149,  1.90660851,  0.9008835 ])"
      ]
     },
     "execution_count": 15,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Variance of the scaled features\n",
    "\n",
    "std_scaler.var_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Gdniw_yyNXVH",
   "metadata": {
    "id": "Gdniw_yyNXVH"
   },
   "source": [
    "Scaled data has zero mean an unit variance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "tfBfcO_CNbZ7",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tfBfcO_CNbZ7",
    "outputId": "eb214052-34d8-4e24-e7fb-3515645549f9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The mean of scaled data: [-0.  0. -0.]\n",
      "The standard deviation of scaled data: [1. 1. 1.]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "print(f'The mean of scaled data: {np.round(num_std.mean(axis=0))}')\n",
    "print(f'The standard deviation of scaled data: {num_std.std(axis=0)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "94oLxVmCIFBD",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 203
    },
    "id": "94oLxVmCIFBD",
    "outputId": "c73e7131-04c9-4189-8faf-c17c05246d57"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>total_bill</th>\n",
       "      <th>tip</th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.314711</td>\n",
       "      <td>-1.439947</td>\n",
       "      <td>-0.600193</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1.063235</td>\n",
       "      <td>-0.969205</td>\n",
       "      <td>0.453383</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.137780</td>\n",
       "      <td>0.363356</td>\n",
       "      <td>0.453383</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.438315</td>\n",
       "      <td>0.225754</td>\n",
       "      <td>-0.600193</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.540745</td>\n",
       "      <td>0.443020</td>\n",
       "      <td>1.506958</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   total_bill       tip      size\n",
       "0   -0.314711 -1.439947 -0.600193\n",
       "1   -1.063235 -0.969205  0.453383\n",
       "2    0.137780  0.363356  0.453383\n",
       "3    0.438315  0.225754 -0.600193\n",
       "4    0.540745  0.443020  1.506958"
      ]
     },
     "execution_count": 11,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Converting the scaled values back to datframe \n",
    "\n",
    "num_std_scaled_df = pd.DataFrame(num_std, columns=num_feats.columns)\n",
    "num_std_scaled_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "q_EfBa0kMw6o",
   "metadata": {
    "id": "q_EfBa0kMw6o"
   },
   "source": [
    "## 3. Robust Scaler"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "oDu_7GCPM2Ne",
   "metadata": {
    "id": "oDu_7GCPM2Ne"
   },
   "source": [
    "[Robust scaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.RobustScaler.html#sklearn.preprocessing.RobustScaler) is kind of similar to standardization but is used when the data contains many outliers. \n",
    "\n",
    "Instead of dropping mean, the median is dropped and the data is scaled to the Interquartile Range(IQR). The IQR is the range between the 1st quartile (25th quantile) and the 3rd quartile (75th quantile).\n",
    "\n",
    "Like normalization and standardization, Roust scaler is also implemented easily in Scikit-Learn. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "C2tPWb_jPSRy",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "C2tPWb_jPSRy",
    "outputId": "f8f18e7c-7797-4f53-a988-a60471b508f3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.07467532, -1.2096    ,  0.        ],\n",
       "       [-0.69155844, -0.7936    ,  1.        ],\n",
       "       [ 0.29823748,  0.384     ,  1.        ],\n",
       "       [ 0.54591837,  0.2624    ,  0.        ],\n",
       "       [ 0.63033395,  0.4544    ,  2.        ]])"
      ]
     },
     "execution_count": 23,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import RobustScaler\n",
    "\n",
    "rob_scaler = RobustScaler()\n",
    "num_rob_scaled = rob_scaler.fit_transform(num_feats)\n",
    "\n",
    "num_rob_scaled[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eGKJqjlLRJ0_",
   "metadata": {
    "id": "eGKJqjlLRJ0_"
   },
   "source": [
    "By scaling the data with Robust Scaler, the median of the resulting values will have a median of zero. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "A6dBrs_wQJ4M",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "A6dBrs_wQJ4M",
    "outputId": "fae7bfcc-3fb3-42f9-c723-08fcf773993a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The median of scaled data: [-0.  0.  0.]\n"
     ]
    }
   ],
   "source": [
    "print(f'The median of scaled data: {np.round(np.median(num_rob_scaled, axis=0))}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "zf59BqHWS9aR",
   "metadata": {
    "id": "zf59BqHWS9aR"
   },
   "source": [
    "### Final Notes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "m36tAB_zTCIx",
   "metadata": {
    "id": "m36tAB_zTCIx"
   },
   "source": [
    "Scaling the input data before feeding it to a machine learning model is always a good practice. \n",
    "\n",
    "Here are the punchlines: \n",
    "\n",
    "* **Scaling** the features helps the model to converge faster.\n",
    "* **Normalization** is scaling the data to be between 0 and 1. It is preferred when the data has not a normal distribution\n",
    "* **Standardization** is scaling the data to have 0 mean and unit standard deviation. It is preferred when the data has a normal or gaussian distribution.\n",
    "* **Robust scaling** technique is used if the data has many outliers. \n",
    "* In most cases, the choice of scaling technique won't make much difference (or it can). **Try all of them** and see what work best with your data.\n",
    "* Only the features are scaled. The labels should not be scaled. \n",
    "* Make sure to not fit the scaler on test data. Only transfom. \n",
    "\n",
    "```\n",
    "Don't: scaler.fit_transfrom(X_test)\n",
    "Do: scaler.transform(X_test)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a6de3d9",
   "metadata": {
    "id": "7a6de3d9"
   },
   "source": [
    "### Futher Learning\n",
    "\n",
    "* http://www.faqs.org/faqs/ai-faq/neural-nets/part2/section-16.html\n",
    "* https://scikit-learn.org/stable/modules/preprocessing.html#standardization-or-mean-removal-and-variance-scaling\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0288daeb",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "4.4 Feature Scaling.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3.7.10 64-bit ('tensor': conda)",
   "language": "python",
   "name": "python3710jvsc74a57bd034ac5db714c5906ee087fcf6e2d00ee4febf096586592b6ba3662ed3b7e7a5f6"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}