Peer_Grade_Assignment.ipynb

{
    "nbformat_minor": 2, 
    "cells": [
        {
            "source": "<a href=\"https://www.bigdatauniversity.com\"><img src=\"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width=\"400\" align=\"center\"></a>\n\n<h1 align=\"center\"><font size=\"5\">Classification with Python</font></h1>", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "In this notebook we try to practice all the classification algorithms that we learned in this course.\n\nWe load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n\nLets first load required libraries:", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 1, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [], 
            "source": "import itertools\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib.ticker import NullFormatter\nimport pandas as pd\nimport numpy as np\nimport matplotlib.ticker as ticker\nfrom sklearn import preprocessing\n%matplotlib inline"
        }, 
        {
            "source": "### About dataset", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n\n| Field          | Description                                                                           |\n|----------------|---------------------------------------------------------------------------------------|\n| Loan_status    | Whether a loan is paid off on in collection                                           |\n| Principal      | Basic principal loan amount at the                                                    |\n| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n| Effective_date | When the loan got originated and took effects                                         |\n| Due_date       | Since it\u2019s one-time payoff schedule, each loan has one single due date                |\n| Age            | Age of applicant                                                                      |\n| Education      | Education of applicant                                                                |\n| Gender         | The gender of applicant                                                               |", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets download the dataset", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 2, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "--2019-07-10 17:39:55--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.193\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.193|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 23101 (23K) [text/csv]\nSaving to: \u2018loan_train.csv\u2019\n\n100%[======================================>] 23,101      --.-K/s   in 0.002s  \n\n2019-07-10 17:39:55 (10.3 MB/s) - \u2018loan_train.csv\u2019 saved [23101/23101]\n\n"
                }
            ], 
            "source": "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
        }, 
        {
            "source": "### Load Data From CSV File  ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 3, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 3, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>9/8/2016</td>\n      <td>9/22/2016</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30       9/8/2016   \n1           2             2     PAIDOFF       1000     30       9/8/2016   \n2           3             3     PAIDOFF       1000     15       9/8/2016   \n3           4             4     PAIDOFF       1000     30       9/9/2016   \n4           6             6     PAIDOFF       1000     30       9/9/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   45  High School or Below    male  \n1  10/7/2016   33              Bechalor  female  \n2  9/22/2016   27               college    male  \n3  10/8/2016   28               college  female  \n4  10/8/2016   29               college    male  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df = pd.read_csv('loan_train.csv')\ndf.head()"
        }, 
        {
            "execution_count": 4, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 4, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "(346, 10)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df.shape"
        }, 
        {
            "source": "### Convert to date time object ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 5, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 5, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  \n0 2016-10-07   45  High School or Below    male  \n1 2016-10-07   33              Bechalor  female  \n2 2016-09-22   27               college    male  \n3 2016-10-08   28               college  female  \n4 2016-10-08   29               college    male  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['due_date'] = pd.to_datetime(df['due_date'])\ndf['effective_date'] = pd.to_datetime(df['effective_date'])\ndf.head()"
        }, 
        {
            "source": "# Data visualization and pre-processing\n\n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Let\u2019s see how many of each class is in our data set ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 6, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 6, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "PAIDOFF       260\nCOLLECTION     86\nName: loan_status, dtype: int64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['loan_status'].value_counts()"
        }, 
        {
            "source": "260 people have paid off the loan on time while 86 have gone into collection \n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets plot some columns to underestand data better:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 7, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Solving environment: / ^C\nfailed\n\nCondaError: KeyboardInterrupt\n\n"
                }
            ], 
            "source": "# notice: installing seaborn might takes a few minutes\n!conda install -c anaconda seaborn -y"
        }, 
        {
            "execution_count": 7, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n", 
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    }, 
                    "metadata": {
                        "needs_background": "light"
                    }
                }
            ], 
            "source": "import seaborn as sns\n\nbins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        }, 
        {
            "execution_count": 8, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n", 
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    }, 
                    "metadata": {
                        "needs_background": "light"
                    }
                }
            ], 
            "source": "bins = np.linspace(df.age.min(), df.age.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'age', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        }, 
        {
            "source": "# Pre-processing:  Feature selection/extraction", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "### Lets look at the day of the week people get the loan ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 9, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGepJREFUeJzt3XmcVPW55/HPV2gvIriC2tIBWkQQldtgR+OCQUh4EdzwuoTEKGTMdTQuYQyDSzImN84YF8YlcSVq8EbEhUTMJTcaVIjgztKCiCFebbEVFJgYYxQFfeaPOt1poKGr6VPU6erv+/WqV1edOud3ntNdTz91fnXq91NEYGZmljU7FDsAMzOzprhAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlApUTS3pLuk/S6pAWSnpV0ckptD5U0M422tgdJcyRVFzsOK75SygtJ3SU9L2mRpCEF3M+HhWq7rXGBSoEkATOApyJiv4g4FBgDVBQpno7F2K9ZYyWYF8OBVyNiUETMTSMm2zoXqHQMAz6NiNvrF0TEmxHxcwBJHSRdJ+lFSYsl/fdk+dDkbGO6pFclTU2SGkkjk2XzgH+pb1fSzpLuTtpaJOmkZPk4SQ9J+g/gD605GElTJN0maXbyzvfLyT6XSZrSaL3bJM2XtFTSv22hrRHJu+aFSXxdWhObtSklkxeSqoBrgVGSaiTttKXXtqRaSVclz82XNFjSY5L+S9K5yTpdJD2RbLukPt4m9vs/G/1+msyxkhYRvrXyBlwE3LCV588Bfpjc/ydgPlAJDAX+Su4d5Q7As8DRQCfgLaAvIOBBYGay/VXAt5L7uwHLgZ2BcUAdsMcWYpgL1DRx+0oT604B7k/2fRLwAXBIEuMCoCpZb4/kZwdgDjAweTwHqAa6AU8BOyfLLwGuKPbfy7ftcyvBvBgH3Jzc3+JrG6gFzkvu3wAsBroC3YH3kuUdgV0atfUaoOTxh8nPEcDk5Fh3AGYCxxT777o9b+4KKgBJt5BLqE8j4ovkXmgDJZ2arLIruST7FHghIuqS7WqA3sCHwBsR8edk+b3kkpmkrRMlTUgedwJ6JvdnRcT/ayqmiGhpn/l/RERIWgK8GxFLkliWJjHWAKdLOodcspUDA8glY70vJcueTt4A70jun421QyWSF/Wae23/Nvm5BOgSEX8D/iZpnaTdgL8DV0k6Bvgc6AHsDaxq1MaI5LYoedyF3O/nqW2Muc1xgUrHUuCU+gcRcb6kbuTeEULuHdCFEfFY440kDQU+abToM/7xN9nSIIkCTomIP23S1uHkXvRNbyTNJfcublMTIuLxJpbXx/X5JjF+DnSUVAlMAL4YEX9Juv46NRHrrIj4xpbispJWinnReH9be21vNX+AM8idUR0aEesl1dJ0/vw0Iu7YShwlzZ9BpeNJoJOk8xot69zo/mPAeZLKACQdIGnnrbT3KlApqU/yuHESPAZc2KhPflA+AUbEkIioauK2tSTcml3IJf5fJe0NfK2JdZ4DjpK0fxJrZ0kHbOP+rO0p5bxo7Wt7V3LdfeslHQv0amKdx4D/1uizrR6S9mrBPto8F6gURK7DeDTwZUlvSHoBuIdcvzTAncArwEJJLwN3sJWz14hYR67r4nfJh8FvNnr6SqAMWJy0dWXax5OPiHiJXNfDUuBu4Okm1llNrt9+mqTF5JK6/3YM04qolPMihdf2VKBa0nxyZ1OvNrGPPwD3Ac8mXe3Tafpsr2TVfyhnZmaWKT6DMjOzTHKBMjOzTHKBMjOzTHKBMjOzTNquBWrkyJFB7nsMvvlWqrdWc5741g5uedmuBWrNmjXbc3dmbZLzxCzHXXxmZpZJLlBmZpZJLlBmZpZJHizWzErO+vXrqaurY926dcUOpV3r1KkTFRUVlJWVbdP2LlBmVnLq6uro2rUrvXv3Jhk/1raziGDt2rXU1dVRWVm5TW24i8/MSs66devYc889XZyKSBJ77rlnq85iXaCs5PUqL0dSq2+9ysuLfSjWAi5Oxdfav4G7+KzkrVi1irp9K1rdTsU7dSlEY2b58hmUmZW8tM6iW3I23aFDB6qqqjj44IM57bTT+Oijjxqee/jhh5HEq6/+Yxqo2tpaDj74YADmzJnDrrvuyqBBg+jXrx/HHHMMM2fO3Kj9yZMn079/f/r3789hhx3GvHnzGp4bOnQo/fr1o6qqiqqqKqZPn75RTPW32tra1vxaCy6vMyhJ/wP4DrkhKpYA3wbKgfuBPYCFwJkR8WmB4jQz22ZpnUXXy+dseqeddqKmpgaAM844g9tvv52LL74YgGnTpnH00Udz//338+Mf/7jJ7YcMGdJQlGpqahg9ejQ77bQTw4cPZ+bMmdxxxx3MmzePbt26sXDhQkaPHs0LL7zAPvvsA8DUqVOprq7eYkxtQbNnUJJ6ABcB1RFxMNABGANcA9wQEX2BvwBnFzJQM7O2asiQIbz22msAfPjhhzz99NPcdddd3H///XltX1VVxRVXXMHNN98MwDXXXMN1111Ht27dABg8eDBjx47llltuKcwBFEm+XXwdgZ0kdQQ6AyuBYeSmIIbcNM6j0w/PzKxt27BhA7///e855JBDAJgxYwYjR47kgAMOYI899mDhwoV5tTN48OCGLsGlS5dy6KGHbvR8dXU1S5cubXh8xhlnNHTlrV27FoCPP/64YdnJJ5+cxuEVVLNdfBHxtqRJwArgY+APwALg/YjYkKxWB/RoantJ5wDnAPTs2TONmM1KjvOk9NQXA8idQZ19dq6Tadq0aYwfPx6AMWPGMG3aNAYPHtxsexFbHwQ8Ija6aq4UuviaLVCSdgdOAiqB94GHgK81sWqTv72ImAxMBqiurs57mHWz9sR5UnqaKgZr167lySef5OWXX0YSn332GZK49tprm21v0aJFHHjggQAMGDCABQsWMGzYsIbnFy5cyIABA9I9iCLLp4vvK8AbEbE6ItYDvwGOBHZLuvwAKoB3ChSjmVlJmD59OmeddRZvvvkmtbW1vPXWW1RWVm50BV5TFi9ezJVXXsn5558PwMSJE7nkkksauu5qamqYMmUK3/3udwt+DNtTPlfxrQC+JKkzuS6+4cB8YDZwKrkr+cYCjxQqSDOz1ui5zz6pfo+tZ3KlXEtNmzaNSy+9dKNlp5xyCvfddx+XXHLJRsvnzp3LoEGD+Oijj9hrr7342c9+xvDhwwE48cQTefvttznyyCORRNeuXbn33nspL7Evk6u5fk0ASf8GfB3YACwid8l5D/5xmfki4FsR8cnW2qmuro758+e3NmazFpGU2hd188iXVg9f4DxpvWXLljV0h1lxbeFvkVee5PU9qIj4EfCjTRa/DhyWz/ZmZmYt5ZEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzk7VvRM9XpNvatyG84qlWrVjFmzBj69OnDgAEDGDVqFMuXL2fp0qUMGzaMAw44gL59+3LllVc2fIVhypQpXHDBBZu11bt3b9asWbPRsilTptC9e/eNptB45ZVXAFi+fDmjRo1i//3358ADD+T000/ngQceaFivS5cuDVNynHXWWcyZM4fjjz++oe0ZM2YwcOBA+vfvzyGHHMKMGTManhs3bhw9evTgk09y3yxas2YNvXv3btHfJB+esNDMSt7Kt9/i8CseTa29538ystl1IoKTTz6ZsWPHNoxaXlNTw7vvvsu4ceO47bbbGDFiBB999BGnnHIKt956a8NIES3x9a9/vWGU83rr1q3juOOO4/rrr+eEE04AYPbs2XTv3r1h+KWhQ4cyadKkhvH65syZ07D9Sy+9xIQJE5g1axaVlZW88cYbfPWrX2W//fZj4MCBQG5uqbvvvpvzzjuvxTHny2dQZmYFMHv2bMrKyjj33HMbllVVVbF8+XKOOuooRowYAUDnzp25+eabufrqq1Pb93333ccRRxzRUJwAjj322IYJEZszadIkLr/8ciorKwGorKzksssu47rrrmtYZ/z48dxwww1s2LBhS820mguUmVkBvPzyy5tNiQFNT5XRp08fPvzwQz744IMW76dxt11VVRUff/zxFvedr3ym8+jZsydHH300v/rVr7Z5P81xF5+Z2Xa06bQYjW1p+dY01cXXWk3F2NSyyy+/nBNPPJHjjjsu1f3X8xmUmVkBHHTQQSxYsKDJ5ZuOtfj666/TpUsXunbtWtB9t2T7TWNsajqP/fffn6qqKh588MFt3tfWuECZmRXAsGHD+OSTT/jFL37RsOzFF1+kb9++zJs3j8cffxzITWx40UUXMXHixNT2/c1vfpNnnnmG3/3udw3LHn30UZYsWZLX9hMmTOCnP/0ptbW1ANTW1nLVVVfx/e9/f7N1f/CDHzBp0qRU4t6Uu/jMrOSV9/hCXlfetaS95kji4YcfZvz48Vx99dV06tSJ3r17c+ONN/LII49w4YUXcv755/PZZ59x5plnbnRp+ZQpUza6rPu5554DYODAgeywQ+684vTTT2fgwIE88MADG80ndeutt3LkkUcyc+ZMxo8fz/jx4ykrK2PgwIHcdNNNeR1fVVUV11xzDSeccALr16+nrKyMa6+9tmGG4MYOOuggBg8enPfU9S2R13QbafE0AlYMnm6j/fF0G9nRmuk23MVnZmaZlKkC1au8PLVvevcqsZklzczam0x9BrVi1apUumKAVKd3NrO2Z2uXc9v20dqPkDJ1BmVmloZOnTqxdu3aVv+DtG0XEaxdu5ZOnTptcxuZOoMyM0tDRUUFdXV1rF69utihtGudOnWiomLbe8VcoMys5JSVlTWMI2dtl7v4zMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk/IqUJJ2kzRd0quSlkk6QtIekmZJ+nPyc/dCB2tmZu1HvmdQNwGPRkR/4J+BZcClwBMR0Rd4InlsZmaWimYLlKRdgGOAuwAi4tOIeB84CbgnWe0eYHShgjQzs/YnnzOo/YDVwC8lLZJ0p6Sdgb0jYiVA8nOvpjaWdI6k+ZLm+1vdZk1znphtLp8C1REYDNwWEYOAv9OC7ryImBwR1RFR3b17920M06y0OU/MNpdPgaoD6iLi+eTxdHIF611J5QDJz/cKE6KZmbVHzRaoiFgFvCWpX7JoOPAK8FtgbLJsLPBIQSI0M7N2Kd/BYi8EpkraEXgd+Da54vagpLOBFcBphQnRrHXUoSyV+cHUoSyFaMwsX3kVqIioAaqbeGp4uuGYpS8+W8/hVzza6nae/8nIFKIxs3x5JAkzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8ukvAuUpA6SFkmamTyulPS8pD9LekDSjoUL08zM2puWnEF9D1jW6PE1wA0R0Rf4C3B2moGZmVn7lleBklQBHAfcmTwWMAyYnqxyDzC6EAGamVn7lO8Z1I3ARODz5PGewPsRsSF5XAf0aGpDSedImi9p/urVq1sVrFmpcp6Yba7ZAiXpeOC9iFjQeHETq0ZT20fE5Iiojojq7t27b2OYZqXNeWK2uY55rHMUcKKkUUAnYBdyZ1S7SeqYnEVVAO8ULkwzM2tvmj2DiojLIqIiInoDY4AnI+IMYDZwarLaWOCRgkVpZmbtTmu+B3UJcLGk18h9JnVXOiGZmZnl18XXICLmAHOS+68Dh6UfkpmZmUeSMDOzjHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKB2o56lZcjKZVbr/LyYh+OmVlBtWg+KGudFatWUbdvRSptVbxTl0o7ZmZZ5TMoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLpGYLlKQvSJotaZmkpZK+lyzfQ9IsSX9Ofu5e+HDNzKy9yOcMagPw/Yg4EPgScL6kAcClwBMR0Rd4InlsZmaWimYLVESsjIiFyf2/AcuAHsBJwD3JavcAowsVpJmZtT8t+gxKUm9gEPA8sHdErIRcEQP22sI250iaL2n+6tWrWxetWYlynphtLu8CJakL8GtgfER8kO92ETE5Iqojorp79+7bEqNZyXOemG0urwIlqYxccZoaEb9JFr8rqTx5vhx4rzAhmplZe5TPVXwC7gKWRcT1jZ76LTA2uT8WeCT98MzMrL3KZ8LCo4AzgSWSapJllwNXAw9KOhtYAZxWmBDNzKw9arZARcQ8QFt4eni64ZiZWTH0Ki9nxapVqbTVc599eHPlyla34ynfzcyMFatWUbdvRSptVbxTl0o7HurIMqlXeTmSUrmVorR+P73Ky4t9KGZb5DMoy6QsvpvLkrR+P6X4u7HS4TMoMzPLpJI9g/onSK17J60P/Cx/6lDmd/dm7VzJFqhPwF1EbVh8tp7Dr3g0lbae/8nIVNoxs+3LXXxmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJJTvUkZmZ5S/N8S/VoSyVdlygzMwsk+NfuovPrB2rH/Xfkx9aFvkMyqwd86j/lmU+gzIzs0xygbLU7FvRM7XuIjMzd/FZala+/VbmPmQ1s7YrUwUqi5c5mtn216u8nBWrVrW6nZ777MObK1emEJEVQ6YKVBYvc8yq+quv0uAktqxZsWpVKhdv+MKNtq1VBUrSSOAmoANwZ0RcnUpU1ixffWVmpW6bL5KQ1AG4BfgaMAD4hqQBaQVmZtZaWf2eV6/y8lRi6tyhY0lfmNSaM6jDgNci4nUASfcDJwGvpBGYmVlrZbWnIc0uzCweX1oUEdu2oXQqMDIivpM8PhM4PCIu2GS9c4Bzkof9gD9tpdluwJptCqht8PG1bfkc35qIaPEHoC3Mk3xjact8fG1bc8eXV5605gyqqXPCzapdREwGJufVoDQ/IqpbEVOm+fjatkIeX0vypNCxZIGPr21L6/ha80XdOuALjR5XAO+0LhwzM7Oc1hSoF4G+kiol7QiMAX6bTlhmZtbebXMXX0RskHQB8Bi5y8zvjoilrYwn7y6ONsrH17Zl6fiyFEsh+PjatlSOb5svkjAzMyskDxZrZmaZ5AJlZmaZlJkCJWmkpD9Jek3SpcWOJ02SviBptqRlkpZK+l6xY0qbpA6SFkmaWexYCkHSbpKmS3o1+TseUaQ4nCdtXCnnStp5konPoJJhk5YDXyV3+fqLwDcioiRGpZBUDpRHxEJJXYEFwOhSOT4ASRcD1cAuEXF8seNJm6R7gLkRcWdy1WrniHh/O8fgPCkBpZwraedJVs6gGoZNiohPgfphk0pCRKyMiIXJ/b8By4AexY0qPZIqgOOAO4sdSyFI2gU4BrgLICI+3d7FKeE8aeNKOVcKkSdZKVA9gLcaPa6jxF6Y9ST1BgYBzxc3klTdCEwEPi92IAWyH7Aa+GXSNXOnpJ2LEIfzpO0r5VxJPU+yUqDyGjaprZPUBfg1MD4iPih2PGmQdDzwXkQsKHYsBdQRGAzcFhGDgL8Dxfj8x3nShrWDXEk9T7JSoEp+2CRJZeSSbmpE/KbY8aToKOBESbXkupyGSbq3uCGlrg6oi4j6d/PTySViMeJwnrRdpZ4rqedJVgpUSQ+bpNxkK3cByyLi+mLHk6aIuCwiKiKiN7m/25MR8a0ih5WqiFgFvCWpX7JoOMWZVsZ50oaVeq4UIk8yMeV7gYZNypKjgDOBJZJqkmWXR8R/FjEma5kLgalJYXgd+Pb2DsB5Ym1AqnmSicvMzczMNpWVLj4zM7ONuECZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUBlgKQfS5qQYnv9JdUkw430SavdRu3PkVSddrtmW+M8aX9coErTaOCRiBgUEf9V7GDMMsp5knEuUEUi6QfJvD6PA/2SZf8q6UVJL0n6taTOkrpKeiMZAgZJu0iqlVQmqUrSc5IWS3pY0u6SRgHjge8kc+tMlHRRsu0Nkp5M7g+vH2ZF0ghJz0paKOmhZCw0JB0q6Y+SFkh6LJkOofEx7CDpHkn/e7v94qxdcZ60by5QRSDpUHJDnQwC/gX4YvLUbyLiixHxz+SmGjg7mXZgDrkh+km2+3VErAf+HbgkIgYCS4AfJd+6vx24ISKOBZ4ChiTbVgNdkiQ+GpgrqRvwQ+ArETEYmA9cnKzzc+DUiDgUuBv4P40OoyMwFVgeET9M8ddjBjhPLCNDHbVDQ4CHI+IjAEn146kdnLzL2g3oQm5IG8jNHTMRmEFu6JB/lbQrsFtE/DFZ5x7goSb2tQA4VLkJ4D4BFpJLwCHARcCXgAHA07mh0NgReJbcu9WDgVnJ8g7Aykbt3gE8GBGNk9EsTc6Tds4FqniaGmNqCrkZRF+SNA4YChART0vqLenLQIeIeDlJvOZ3ErFeudGTvw08AywGjgX6kHv32QeYFRHfaLydpEOApRGxpSmbnwGOlfR/I2JdPrGYbQPnSTvmLr7ieAo4WdJOyTu2E5LlXYGVSbfBGZts8+/ANOCXABHxV+Avkuq7Jc4E/kjTngImJD/nAucCNZEbiPE54ChJ+wMk/fkHAH8Cuks6IlleJumgRm3eBfwn8JAkv9GxQnCetHMuUEWQTGv9AFBDbu6buclT/4vcDKKzgFc32WwqsDu55Ks3FrhO0mKgCvjJFnY5FygHno2Id4F19fuMiNXAOGBa0s5zQP9kSvFTgWskvZTEeuQmx3E9ua6QX0nya8lS5Twxj2beRkg6FTgpIs4sdixmWeU8KS0+5WwDJP0c+BowqtixmGWV86T0+AzKzMwyyf2hZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSf8feZ3K8s9z83MAAAAASUVORK5CYII=\n", 
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    }, 
                    "metadata": {
                        "needs_background": "light"
                    }
                }
            ], 
            "source": "df['dayofweek'] = df['effective_date'].dt.dayofweek\nbins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\ng.axes[-1].legend()\nplt.show()\n"
        }, 
        {
            "source": "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 10, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 10, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below    male          3        0  \n1 2016-10-07   33              Bechalor  female          3        0  \n2 2016-09-22   27               college    male          3        0  \n3 2016-10-08   28               college  female          4        1  \n4 2016-10-08   29               college    male          4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ndf.head()"
        }, 
        {
            "source": "## Convert Categorical features to numerical values", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets look at gender:", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 11, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 11, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "Gender  loan_status\nfemale  PAIDOFF        0.865385\n        COLLECTION     0.134615\nmale    PAIDOFF        0.731293\n        COLLECTION     0.268707\nName: loan_status, dtype: float64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
        }, 
        {
            "source": "86 % of female pay there loans while only 73 % of males pay there loan\n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets convert male to 0 and female to 1:\n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 12, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 12, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>1</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below       0          3        0  \n1 2016-10-07   33              Bechalor       1          3        0  \n2 2016-09-22   27               college       0          3        0  \n3 2016-10-08   28               college       1          4        1  \n4 2016-10-08   29               college       0          4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ndf.head()"
        }, 
        {
            "source": "## One Hot Encoding  \n#### How about education?", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 13, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 13, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "education             loan_status\nBechalor              PAIDOFF        0.750000\n                      COLLECTION     0.250000\nHigh School or Below  PAIDOFF        0.741722\n                      COLLECTION     0.258278\nMaster or Above       COLLECTION     0.500000\n                      PAIDOFF        0.500000\ncollege               PAIDOFF        0.765101\n                      COLLECTION     0.234899\nName: loan_status, dtype: float64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
        }, 
        {
            "source": "#### Feature befor One Hot Encoding", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 14, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 14, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>education</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>High School or Below</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>Bechalor</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender             education\n0       1000     30   45       0  High School or Below\n1       1000     30   33       1              Bechalor\n2       1000     15   27       0               college\n3       1000     30   28       1               college\n4       1000     30   29       0               college"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df[['Principal','terms','age','Gender','education']].head()"
        }, 
        {
            "source": "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 15, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 15, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "Feature = df[['Principal','terms','age','Gender','weekend']]\nFeature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\nFeature.drop(['Master or Above'], axis = 1,inplace=True)\nFeature.head()\n"
        }, 
        {
            "source": "### Feature selection", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets defind feature sets, X:", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 16, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 16, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "X = Feature\nX[0:5]"
        }, 
        {
            "source": "What are our lables?", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 17, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 17, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n      dtype=object)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "y = df['loan_status'].values\ny[0:5]"
        }, 
        {
            "source": "## Normalize Data ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Data Standardization give data zero mean and unit variance (technically should be done after train test split )", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 18, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  return self.partial_fit(X, y)\n/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  if __name__ == '__main__':\n"
                }, 
                {
                    "execution_count": 18, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n        -0.38170062,  1.13639374, -0.86968108],\n       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n         2.61985426, -0.87997669, -0.86968108],\n       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679]])"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "X= preprocessing.StandardScaler().fit(X).transform(X)\nX[0:5]"
        }, 
        {
            "source": "# Classification ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\nYou should use the following algorithm:\n- K Nearest Neighbor(KNN)\n- Decision Tree\n- Support Vector Machine\n- Logistic Regression\n\n\n\n__ Notice:__ \n- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n- You should include the code of the algorithm in the following cells.", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "# K Nearest Neighbor(KNN)\nNotice: You should find the best k to build the model with the best accuracy.  \n**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Spliting training dataset and test dataset ", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 19, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Train set: (242, 8) (242,)\nTest set: (104, 8) (104,)\n"
                }
            ], 
            "source": "from sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.model_selection import train_test_split\nX_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.3, random_state=4)\nprint ('Train set:', X_train.shape,  y_train.shape)\nprint ('Test set:', X_test.shape,  y_test.shape)"
        }, 
        {
            "source": "#### Randomly try the K in the range 1 to 10", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 23, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "while n = 1\nTrain set Accuracy:  0.8966942148760331\nTest set Accuracy:  0.7115384615384616\n\n\nwhile n = 2\nTrain set Accuracy:  0.8140495867768595\nTest set Accuracy:  0.625\n\n\nwhile n = 3\nTrain set Accuracy:  0.8347107438016529\nTest set Accuracy:  0.7211538461538461\n\n\nwhile n = 4\nTrain set Accuracy:  0.8181818181818182\nTest set Accuracy:  0.7211538461538461\n\n\nwhile n = 5\nTrain set Accuracy:  0.8057851239669421\nTest set Accuracy:  0.7307692307692307\n\n\nwhile n = 6\nTrain set Accuracy:  0.8099173553719008\nTest set Accuracy:  0.7115384615384616\n\n\nwhile n = 7\nTrain set Accuracy:  0.7975206611570248\nTest set Accuracy:  0.7211538461538461\n\n\nwhile n = 8\nTrain set Accuracy:  0.7892561983471075\nTest set Accuracy:  0.7211538461538461\n\n\nwhile n = 9\nTrain set Accuracy:  0.7933884297520661\nTest set Accuracy:  0.75\n\n\n"
                }, 
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8VOX1+PHPmS0hYd9XBRUUUAgQFqVatypW61oVtFUUwQ0VXKr91l/bbzeXr62o4AKIWFSoS6VaFXdrtQIJ+yIIAkJk37MnM3N+f8wkDSEhQ3Incyc579crLzM39z5zJoY589x77nlEVTHGGGPcxpPoAIwxxpiqWIIyxhjjSpagjDHGuJIlKGOMMa5kCcoYY4wrWYIyxhjjSpagjDHGuJIlKGOMMa5kCcoYY4wr+eI5uIiMAJ4AvMB0VX240s+PAV4EWkb3eUBV343+7JfAGCAE3Kmq7x/pudq2bavdu3d3/DUYY4xx1qJFi3ararua9otbghIRLzAF+BGQA2SJyFuqurrCbg8Cr6rqMyLSB3gX6B79fiTQF+gMfCQivVQ1VN3zde/enezs7Hi9HGOMMQ4Rke9i2S+ep/iGAOtVdYOqlgBzgEsq7aNA8+j3LYCt0e8vAeaoarGqbgTWR8czxhjTSMQzQXUBtlR4nBPdVtFvgZ+JSA6R2dMdR3EsIjJORLJFJHvXrl1OxW2MMcYF4pmgpIptlVunjwJmqmpX4MfALBHxxHgsqjpVVTNVNbNduxpPZxpjjEki8SySyAG6VXjclf+ewiszBhgBoKpfiUgq0DbGY40xxlGlpaXk5ORQVFSU6FAahNTUVLp27Yrf76/V8fFMUFlATxHpAXxPpOjhmkr7bAbOAWaKSG8gFdgFvAW8IiJ/IVIk0RNYGMdYjTGGnJwcmjVrRvfu3RGp6kSOiZWqsmfPHnJycujRo0etxojbKT5VDQLjgfeBr4lU660Skd+JyMXR3e4BxorIMmA2MFojVgGvAquBecDtR6rgM8YYJxQVFdGmTRtLTg4QEdq0aVOn2Whc74OK3tP0bqVtv67w/WpgeDXH/hH4YzzjM8aYyiw5Oaeuv0vrJGGMMcaVLEEZY4zLvPnmm4gIa9asSXQoh1BVwnpYQXXcWIIyxhiXmT17Nj/4wQ+YM2dOXJ8nFIr90n5YlWBY0XpMUHG9BmWMMclqwrwJLN2+1NExMzpmMGnEpCPuk5eXx5dffsmnn37KxRdfzG9/+9vynz366KPMmjULj8fDBRdcwMMPP8z69eu55ZZb2LVrF16vl9dee40tW7bw2GOP8c9//hOA8ePHk5mZyejRo+nevTs33ngjH3zwAePHjyc3N5epU6dSUlLCCSecwKxZs0hLS2PHjh3ccsstbNiwAYCnJk9h3rx5tGvXlokTJgDwq1/9ig4dOnDnnXc6+nsqYwnKGGNcZO7cuYwYMYJevXrRunVrFi9ezMCBA3nvvfeYO3cuCxYsIC0tjb179wJw7bXX8sADD3DZZZdRVFREOBxmy5YtR3yO1NRUvvjiCwD27NnD2LFjAXjwwQd5/vnnueOOO7jzzjs544wzePX1NwiFQuTl5dGpc2euvvKnTJwwgXA4zJw5c1i4MH53AFmCMsaYKtQ004mX2bNnMyE6Qxk5ciSzZ89m4MCBfPTRR9xwww2kpaUB0Lp1a3Jzc/n++++57LLLgEjiicXVV19d/v3KlSt58MEH2b9/P3l5eZx//vmoKp988gnPvzATAK/XS4sWLWjRogWtW7dmyZIl7NixgwEDBtCmTRsHX/2hLEEZY4xL7Nmzh08++YSVK1ciIoRCIUSERx99FFU9rGy7uutBPp+PcDhc/rjyvUjp6enl348ePZq5c+fSv39/Zs6cyaeffkYwXP11pjFjxjBz5ky2b9/OjTfeWJuXGTMrkjDGGJd4/fXXue666/juu+/YtGkTW7ZsoUePHnzxxRecd955zJgxg4KCAgD27t1L8+bN6dq1K3PnzgWguLiYgoICjj32WFavXk1xcTEHDhzg448/rvY5c3Nz6dSpEyUlJbz00ktotO3pWWefzXPPPgtEiikOHjwIwKWXXca8efPIysri/PPPj+evwxKUMca4xezZs8tP15W54ooreOWVVxgxYgQXX3wxmZmZZGRk8NhjjwEwa9YsnnzySfr168dpp53G9u3b6datG1dddRX9+vXj2muvZcCAAdU+5+9//3uGDh3Kj350HieedFL59r88PonPPvuUARn9GTpkMKtXrQIgEAhw1llncdVVV+H1euPwW/gvqc+SwXjKzMxUW7DQGFMXX3/9Nb179050GPVGVQmF9fClIo54UJjBmZm89tpr9OzZs8bdq/qdisgiVc2s6VibQRljTCMUCkfvazqKY1avXs2JvXpxzjnnxJSc6sqKJIwxphGp1awpqk+fPqxbvx6vp37mNpagjDGmEYi0KaJeWxXVlSUoY4xp4MKqhGs5a0okS1DGGNNAlTV3PcJtTa5mCcoYYxqgcPRaUzKzKj5jjKnGjoNFjn455auvvirvn1dZpAgizDPPPMOAjP5kDhrImWecwerVq4/6eTZt2sTs2a9U+/MzzzyTeN7eYwnKGGNc6LPPPmP06NFV/mzevHmMGDHisO3haOl4WGHkqGtYsnQZ2YsWc8999/KLe+856hi+27SJv82efdTHOcUSlDHGJJmPP/6Yc889t/yxqhIMhQlVqNBr3rx5+ff5+fnlffwmPf44Y28aA8CKFSvI6N+PgoICPv/Xv8gcNJDMQQMZnDmI3NxcfvU//8MXX3xB5qCBPDFpEoWFhVwzahT9+vXj6quvprCwMK6v065BGeMyobBSGgpTGgqjQBO/F7/XPkuaiN27d+P3+2nRogUQmTWFqikdf+bpp3li0uOUlJTw/ocfAXDnXXdx7tlnM3fumzz80EM8/fQzpKWl8Ze//Jknn3yK04YPJy8vj9TUVP74pz/x+F/+zNy33gYiyS0tLY3ly5ezfPlyBg4cGNfXan/1xiSQqlISDJNfHGR/QQm7covZnVfMgcJSCkpCFJaE2Jtfwr78EopKY1/91CSvoUOHkpGRwU033cRbb71FRkYGGRkZvP/++wB88MEHnHfeeVXOmiq79bbbWPPNOv740EM89Kc/AuDxeJg+YwY3XH89Z5x+BqcNHw7AaacN575772XyU0+xf/9+fL7D5y///vfnXHPttQD069ePfv36Of3yD2EzKGPqUTAUpjSklIbDlAbDR1zWoKKSUJiSwjC5RUHSAl7SAt7Dll4wDcOCBQuAyDWomTNnMnPmzEN+/t5773HXhIkEw8pNY25k6dKldO7Uibf++U61Y1599UjuuP328sfr162jadOmbN22tXzbL+6/nwt+/GPmvfcepw8/jffe/6DKserz785mUMbESTisFJWGyCsOsi+/hJ25RezJL+FgUSmFJaGYk9MhY6qSVxxkV24xB4tKCYbCNR9kGoxwOMyy5cvp178/ANOfn0H2osVVJqd169aVf//uO+9wQrR33oEDB7j77ol8/Oln7N2zhzfeeB2Ab7/9llNOOYX7fvELBg4axNq1a2jWrBm5uXnl45x++hnMfiVS1bdy5UqWL18et9cKNoMyxhGqkeqp0lCY0mBkhhTPe1AUKIyeAkzxeWgS8JLii+/SB41Rh+axrVBbH0JhJSsrm4yMjJhmMc88PYWPP/4Yv99Pq5ateH7GCwDce8/d3HLLrfTq1Yvnpk3nvHPP4fTTz+CpJ5/gs88+w+v10rt3b0aMuACPx4PP52PQwAFcd9313HzLLYwdcyP9+vUjIyODIUOGxPU123IbxtRCWSFDSShMMBS5FpDof0lej5Ae8JHq99jpv1py43IbFdsU/emPf+T4E47n6qtHJiwej3BUzWLrstyGzaCMqYGqlieisqTkxs91obBysKiU3OJI5V9awIfXY4kqWVXV3PV/fvWrBEZU/yxBGVNJWSFDSbTUO9naxahCQUmIgpIQqT4vTQJeAj673JxMGkKbIidYgjKNWjj830RUNkNqSG8LRcEQRcEQPo+QnuIjxWen/2qiqgn7HSV7c9fK6noJyRKUcVxBSZCiUvdXl4XCmlRr49RFMKwcKCxFBNICPtL8XjwN/PSfqkZK+qMfPo50v1AZrz/Azl27ad26dUKSVEP6a1RV9uzZQ2pq7QtNLEEZR2m0DLqRvO8nHVXILw5SUBwkxR+5n6qhdKkIhiL3lZWEIveY1WbV2KatO7Bvzw527doVlxgbAhHwxJi8U1NT6dq1a62fyxKUcVRRqTsLCMyhFCgqDVFUGsLv9ZAW8JLqT54y9bLCldJoBaVThSten49WHbrUfaAGrEnAS/NUf708lyUo46j8kmCiQzBHqTQU5kCFLhVNXHj6r+w0XbIWrpjasQRlHFNUGrI3jiRW1qUivzhIasBLmt+LLwGn/xp64YqJnSUo45iCEmtm2hBU7FIR8HpIS4lfl4rKhQwloXCjKVwxNbMEZRxREox84jUNS0koTElBGK/nv6f/6lLdVrEDR20LGUzjYQnKOKLQZk8NWiis5BYFySsOxtylIl6FDKbxsARl6iwUVoqClqAag4pdKlJ8HtICvvIuFVbIYJxmCaqCUFitd1ktWOVe41QcDFMcLMHrkfJmpsY4qWHcoeeAshtMzdEJh5UiO73XqNl1JBMvlqAqKCoN2QJwR6mgNGRvTsaYuIhrghKRESKyVkTWi8gDVfz8cRFZGv36RkT2V/hZqMLP3opnnBXl22wgZqpKgZ3eM8bESdyuQYmIF5gC/AjIAbJE5C1VXV22j6pOrLD/HcCACkMUqmpGvOKrTlFpiPRAYm5QTDbW1sgYE0/xfBceAqxX1Q2qWgLMAS45wv6jgNlxjCdm+cU2i4qFFUcY07hs2r+BK179CRv3bayX54tnguoCbKnwOCe67TAicizQA/ikwuZUEckWkfkicmk1x42L7pPtZPfhoqBdi6qJtTUypvFQVV5cNo2zXxpC1tb5fLPnm3p53niWmVdVr13dO9pI4HVVrTh1OUZVt4rIccAnIrJCVb89ZDDVqcBUgMzMTEffLfOLQ7RIs9N81bG2RsY0DtvyvufuD27l0+8+5IfHnMPTF06lT/vj6uW545mgcoBuFR53BbZWs+9I4PaKG1R1a/S/G0TkMyLXp749/ND4KAqGSA/ZtaiqWFsjYxo+VeXva+bwP5/eTUmohIfOnsTofuNIS6m/22fj+e6bBfQUkR4iEiCShA6rxhORE4FWwFcVtrUSkZTo922B4cDqysfGm12LqppV7hnTsO0p3M3Yd67l9nk30rP1iXz8swXc0P/mel9lOG6pUFWDIjIeeB/wAjNUdZWI/A7IVtWyZDUKmKOHLl7fG3hORMJEkujDFav/6ovNog4XCivFQZs9GdNQfbDhXe758Db2F+3lV8N/x22Zd+P1JGYxy7jO1VT1XeDdStt+Xenxb6s47j/AKfGMLVZ2LepQVrlnTMOUW3yQX//rF8xe9SJ92p7CnMvfom+7fgmNyXrx1cBmUf9lbY2MaZi+3PI5d30wlq25Odw5+D7uGfYrUnwpiQ7LElQsbBYVYW2NjGlYCoOFPPTFr5m6ZDI9Wh7PW1d9TGbnYYkOq5wlqBgUBUOkhbz4G/EsytoaGdOwLNmezZ3v38S6vWu5of/NPHj6H0n3pyc6rENYgopRfnGQlmmBRIeRMNbWyJiGoTRUyl8WPMSTCx+lQ3pHXr38Hc449uxEh1UlS1AxKo7e+9NYZ1FWHGFM8luzezV3vn8Ty3cu4cre1/KHMx+jRWrLRIdVLUtQR6GxzqKsrZExyS0UDvHc4id55D//S9NAc2b8ZA4/PuFIrVHdwRLUUWissyhra2RM8vpu/0bu+mAs87//kguO/wmPnjuZdmntEx1WTCxBHaXGNouytkbGJCdV5aUVM/jN5/fjFS9Pnj+dK3tfU+/dIOrCEtRRamyzKKvcMyb5bM/byt0f3sYnm97n9G5n8fh5z9K1+TGJDuuoWYKqhcYyiwqGwtbWyJgkM3ftqzzw8QSKQ0X88ay/cEP/m/FIcn6gtgRVC8XBMCXBMAFfcv5Pj1VBqV17MiZZ7C3cwwOf3MVb37zBwI6DeWrE8xzfqmeiw6oTS1C1lF8cJOBruLMoa2tkTPL4aOM87v7gVvYV7eGXw/+X2zPvxudJ/rf35H8FCVISatizKGtrlBjBcJA1u1excOtXZG+bz/6i/fzslBs4/7iLEtZR2rhXXkkuv/nX/by88gVOatOXVy6by8nt+yc6LMdYgqqDhjqLsrZG9edg8QEWbVtI1tb5ZG39isXbs8gvzQOgQ3pHfB4/N749kuNansAtg+7iyj7X0sTXJMFRGzf4T86/uev9sXyfu4Xxmfdw36n/zxUNXp1kCaoOGuosytoaxYeqsvnAJhZu/YqsbZGEtGb3KhTFIx76tD2Fq/pcy+DOw8jsdCrdmh9DSEO8s24uTy96nF98fAeP/Od3jMm4ldH9x9G6SZtEvySTAEXBIh768jdMXfwUx7bowdwrP2RIl9MSHVZciDaQd6LMzEzNzs6u9fGqys7c4qM+LuD10Cq9Yc2iducVW+cIBxQHi1mxc0l5MsrauoBdBTsAaBZozqBOQxjc+VQGdxrGwE6DaRpoVu1YqspX33/B09mP89HG92jia8LIvtdxy8C7OLZlj/p6SSbBlu1YzPh5Y1i3dw3X9xvHr0//I+mBpvUaQ5OAl+ap/jqNISKLVDWzpv1sBlVHDW0WZW2Nam93wS6yt80vP123bMdiikORDz3HtujBD489h8GdhjG48zBObNPnqK4piQindT2d07qezprdq3l28RO8tGIGLy6fxoUnXMptmRMZ0LHGf+8mSZWGSnli4aNMWvgw7dLaM+fytznz2HMTHVbc2QwqqrYzKGhYs6i9+SXWOSIGYQ3zzZ41ZG+bHylo2DqfDfvXA+D3+OnfYSCZnYeVJ6T26R0dj2F73laeX/oMLy6fxsHiAwzr8gNuz7ybc3qcn7T3vZjDfbNnDXe8P4ZlOxbz05NG8Yez/kzL1FYJi6c+Z1CWoKLqkqAAWqUFkn4WVRIMs6+gJNFhuFJ+aT5LtmeTvTWSkBZtW8CB4v0AtGnSjsGdh5LZaRhDOp9Kvw4DSfWl1ltseSW5vLzyBaYuforvc3Po1bo3twy6iytOGtngLpo3JmENM23JZP70xa9J9zfl0XOf4qKelyU6LEtQtZHoBOX3emid5LOo/QUl1jki6vvcLeWn6rK2LmDVrmWENHJf2Ilt+kRnRqcyuPMwerQ83hX9zUpDpbz1zes8vWgSq3Ytp0N6R27KuI3r+o119ZIK5nCbD3zHXR+M5aucf3P+cRfx2LmTaZfeIdFhAZagaiXRCQqgZZqfFF9y3qsSDIXZk984Z0/BcJBVu5ZHk9F8srfN5/vcHACa+NIY2GlweUIa1GlIQk+vxEJV+XzzJzyd/Tj/2vwx6f6m/OyUGxg7YHxS9mNrTFSV2ate5P99dh8iwh/O/DNX9/mZKz4AlbEEVQtuSFDJPIs6WFRKoUOdIz7aOI+9hXscGSteVJWN+9eTtW0+i7dlURgsAKBLs65kVpgd9W3XL6nvyF+1aznPZE9i7jevoapccuJPuW3QxAZ1M2dN9hXtZdG2ha7/mwR4+5u/8+HGdxne7YdMOu85ujU/NtEhHcYSVC24IUFBcs6iwmFld16xI50jvsr5gste+5EDI8WfV7z0bdefIZ2HRQoaOg+jS7NuiQ4rLr7P3cLUxZN5acUM8kvzOOOYs7ktcyI/POYcV306rytV5dt96yqU9s9n3d41iQ4rZqneVB48/Q/cmHGrawtdLEHVglsSVDLOovKKg+QXO9M54oa3rmLB9//h7ZGf4hN3zzzaprcn3Z+e6DDq1YGi/fx1xXSmL5nCjvzt9G3Xj1sHTeCSXj/F763bm04iFAYLWbZ9USQZbVtA9tb57C2KzJRaprQq/+AxuPOpdGnaNcHR1qxlaivXXy+0BFULbklQkFyzKFVlV16xI50jvtu/kWEv9OWuIb/ggeG/rfuAJm5KQiX8fc3feGbRJNbuWU3npl0YO3A8Pzv5RpqlNE90eNXakbftkNnRip1LKQ2XAnBCq15kdo5UUmZ2GsYJrXu5dhaSzCxB1YKbElQyzaIKSoLkFjkze/r1Z/cxY9mzZI9ZS8emnR0Z08RXWMN8sukDns5+nP/kfE7zlBb8/JQxjB1we8L/H4bCIdbsWVV+n1nW1vlsPrgJiJwKy+g4qLy0f1DnobRp0jah8TYWlqBqwU0JCpJnFuVUW6Pc4oMMmH4C5x93IVMueMGByEx9W7p9EU8vepx/rnsTr3i5/KSR3DpoAie17VMvz59bfJDF27PKE9Ki7QvJK8kFoH1aRwZ3HsaQLpHWUCe3zyDgTY4PgQ2NtTpqAPKKgqQ0dXeCcrKt0ZxVfyWvJJexA8Y7Mp6pfxkdBzH1wpf4bv9Gpi55ildWvsjfVs/i7O7nc1vmRIZ3PcOxggpVZfPB78pP1WVt/Yqvd68sb5zbu+3JXHHSqGgBy6kc0/zYBlXMYWJjM6gop2dQAC2a+En1uzdJOdXWKBQOceoLJ9OxaSfeuvoTByIzbrC3cA8vLp/G80ufYXfBTvq1H8htmRO4qOdlR116XxIqiTTOrXDz886C7QA0DTRjUMch5aX9AzsOdvV1sMbOZlANRH5x0LUJqiQYdqzn3gcb3mHzwU38+ow/OTKecYfWTdowcegD3DpoAq+tfplnFz/BLe9exzHNuzNu4B2MOvn6aqsg9xTuLr9ulLX1K5buWFTeOPeY5t0545izyivsTmrT1xZjNFWyGVRUOBxm0969jreud+ssysm2Rpe9dh5bDn7H/BtWJfVNrebIwhrm/W//ydPZj5O1bT6tUlszuv84buh/M/uK9pX3KczeNp9v960DIo1zT2mfUT47GtxpGB2adkrwKzF1YUUStVCXBKWqnDfrPNL9LXnuwlmOxuXzCG2auqthp5NtjVbsXMqPXj6V35zxELcOmuDImMb9srZ+xTOLJvHe+rfRCrd4t05tw+DOp5LZeWh541xbAbhhsVN89UxEyOycySNfPsK9pz5Iz9YnOjZ2MKwUlYZcNYsqKHWmpRHAtCVTSPOnc03f0Y6NadwvMiM6lW/3rePNNa/SpXk3hnQ+leNanmDFDMYxdhdb1MRhE0n1NeGJhY86PrZTXRqcEA4rRQ713NuZv525a19lZN+fu/7udxMfx7fqyb2n/opRfa/j+FY9LTkZR1mCimqX3o7r+t3Em2v+xqb9Gxwdu2wW5QYFpSFHeu4BzFw2jdJQKTdl3O7QiMYY81+WoCq4bdAEfB4fT2U95vjYbphFqSoFJc7EURQs4sXl0/jRcT/muFYnODKmMcb96nOObAmqgg5NO3HNyaN5dfVL5Bzc7OjYbphFFZaGHOm5B/Dm2lfZU7iLcXZjrjGNRnqKj2Z1LJA4GpagKhk/+B4ApmT/xfGx8xI8iypw6NqTqjJt8WR6tz2Z4d1+6MiYxhj38ojQKi1A05T6raurMUGJyHgRcfcSog7q0qwbV/f5Oa+snMmOvG2Ojh1K4CzKybZGX+Z8zurdKxg7YLxdFDemgUv1eWmTHiDgq//5TCzP2BHIEpFXRWSENIJ3pDsG30swHOTpRY87PnaiZlFOzZ4Api2eTOsmbbn8pKsdG9MY4y4CNE/10yLNj8eTmLf9GhOUqj4I9ASeB0YD60TkTyJyfE3HRhPaWhFZLyIPVPHzx0VkafTrGxHZX+Fn14vIuujX9Uf1quro2JY9uPykkfx1+XR2F+xydOxEzKKcbGu0cf+3fLDhHa7vN5ZUX6ojYxpj3MXnEVqnB2gSSOz9mzHN2TTSbmJ79CsItAJeF5FqbxoSES8wBbgA6AOMEpFD+var6kRVzVDVDOAp4O/RY1sDvwGGAkOA39T3acY7h9xHUbCI5xY/6fjY9T2LcqpyD2D6kin4PD5u6D/OsTGNMe6RFvDSOj2Az5v4EoVYrkHdKSKLgEeBL4FTVPVWYBBwxREOHQKsV9UNqloCzAEuOcL+o4DZ0e/PBz5U1b2qug/4EBhR46txUM/WJ3JxryuYsfRZ9hXtdXTs+pxFBUNhx3ruHSjaz+xVf+XSE6+ifXpHR8Y0xriDSGQdu2apftdcW44lRbYFLlfV81X1NVUtBVDVMHDREY7rAmyp8Dgnuu0wInIs0AMoW6shpmNFZJyIZItI9q5dzp6KA5gw9H7yS/OYvuRpx8eur1mUk22NZq96kYLSfMYOsBtzjWlIUnwe2qanuG6R1VgS1LtA+RRCRJqJyFAAVf36CMdVlYKrKyMbCbyuqmXvpjEdq6pTVTVTVTPbtWt3hFBqp3fbk7ng+J8wfckUcosPOjp2KKwUOli4UBUn2xoFw0GmL32aYV1+QL8OAxwZ0xiTWAI0S/XRMi2QsEKII4klQT0D5FV4nB/dVpMcoFuFx12BrdXsO5L/nt472mPjasLQBzhQvJ8Xlj3n+Nj5Dl4bqoqTbY3e//af5BzczLiBdmOuMQ2B1yO0Sg+QFnBvz/BYEpRohTU5oqf2YnlFWUBPEekhIgEiSeitwwYXOZFI0cVXFTa/D5wnIq2ixRHnRbfVu/4dBnJOjxE8u/hJ8kvzHR07nrMoJ9saAUxd/BTHNO/O+ccd6ayuMSYZNAlE7m3yu6AQ4khiiW5DtFDCH/26C6ixm6qqBoHxRBLL18CrqrpKRH4nIhdX2HUUMKdSEtwL/J5IkssCfhfdlhATh9zP3sLd/HX5NMfHjte1KCfbGi3dvogFW//DTQNus5VPjUliIpFFVJu7qBDiSGpcsFBE2gNPAmcTuQ70MTBBVXfGP7zY1XVFXVVlZ25xtT+/8vUfs3bP1ywYs9rxBdiap/odv99gd16xY50jxr93I/M2/JMlN62nWUpzR8Y0xtSvgNdD8yZ+vC641hTrgoWx3Ki7U1VHqmp7Ve2gqte4LTnVhwlDH2BnwXZeWTnT8bHzioM4ubKxk22Ntudt5R/fvM6ovtdZcjImSTVN8dEqPeCK5HQ0aryWJCKpwBigL1DeOkBVb4xjXK5zWtfTGdr5NKZd/opJAAAe2UlEQVRk/ZmfnzKGgDfg2NhhVYpKw47Nopxsa/Ti8mkEw0HGZNzm2JjGmPrh9QjNU/0J6aPnhFiinkWkH9/5wL+IVNTlxjMoNxIRJg77JVvzvufV1S85Pr5Tsygn2xoVBgt5cfl0zj/+Qrq3PM6RMY0x9SORTV6dEkvkJ6jq/wPyVfVF4ELglPiG5U4/POYcMjoM4smF/0dpqNTRscOqFDpwU62TlXt//3oOewt3M3bAHY6NaYyJLyFSCNEiLTkKIY4klgRV9k68X0ROBloA3eMWkYuJCHcP/SWbD27izbWvOj5+fnGoTrMoJ9saqSpTl0ymb7t+nNb1dEfGNMbEl9/roU3TFFL9DaPaNpYENTV6L9KDRO5jWg08EteoXOxHx/2Yvu368cTCRwiFnb2Hqa6zKCfbGv1786es3bOacQPuSPpPYcY0BukpPlonYSHEkRwxQYmIBzioqvtU9XNVPS5azed8W4UkISJMGHI/3+5bx9vr3nB8/NrOopxsawQwdclk2qa159ITr3RsTHN0fB4h4PIbKU3iJWq12/pwxL/+aNcI621TyYU9L6VX695MWvAoYXXmlFqZ2s6inGxr9O2+dXy08T1G9xtHii/FoVHNkQiR0zPpKT5apvlp3yyFNk1TaJUeoHV6gJQkvtBt4ifF50n6QogjieVVfSgi94pINxFpXfYV98hczCMeJgz9BWv2rGLet287Pv7RzqKcbms0bckUAt4A1/e7ybExzaGEyI2TTVN8tEoL0K5ZCq3TI5+CU3zeQ06r+r0eWqYFaJMeINVl3aZNYpStduvWJq9OiSVB3QjcDnwOLIp+1b5lQwNxca+f0qPl8Uxa8IijN9nC0c+inGxrtL9oH39bNYvLTryadukdnBnUIBIp+22WGrlO0L55Kq3SA6Sn+Aj4PDFd5/N5PbRI89O2aQpNAt4qW/6bhs8tq93Wh1g6SfSo4qvR3xTj8/i4c8gvWL5zCR9vcr6P7dHMopy8MffllS9QGCywruV15BEh1e+leaqfNukB2jdLpUWan7SAr84NOstuvmzTNIU0S1SNShMXrXZbH2LpxXddVdtV9a9xiaiW4t2LryqloVJOnXkyHdI78s+rP3O82q1pio/0Gi58FpWGOFDozD1ZwXCQoTN6073l8bzx03mOjNlYeD1CwOch4PXg93rqtZIqHFYKSkMUlAQdm0kbdylr8uq2BQVry7FefMDgCl+nA78FLj7SAY2F3+vnjsx7WbRtIV9s+czx8fNLau4u4eTs6d31/+D73BzGDbDZU038Xg9pAS8tmvhp1zSFtk1TaJ7qJ9XvrfcyX49HaJrio13TFJqm+LC7AhqWgNedq93WhxpnUIcdINICmKWqrkpSiZhBARQHixk6ozc9Wp3Am1d+UOvnr86RZlElwTD7Ckoce66fzDmLXQU7+XL0cltWo4KyCju/z4PfGyn9dvO9YRq9hplfHCJsU6qkJUTubarpLEoycnIGVVkB0LMWxzVIKb4Ubs+8m69y/s38nC8cH/9IsygnK/cWb88ia9t8W/OJyOmUFN+hFXatqqmwcyMRIS3go23TAM1T3bG8gjk6ZavdNsTkdDRqTFAi8raIvBX9+iewFvhH/ENLHteeciNt09ozaaHzDTZUqz6N52RbI4BpiyfTLNCckX2rvOTYoHlEDq2wa5ZKy7Sjq7BzIxGhScBL26YptGjix2eJKimk+pNjtdv6EEt6fqzC90HgO1XNiVM8SSnNn8atg+7i9//+FYu3ZzGw42BHx88vCUaqtSq8UTrZ1mhrbg5vr/s7YzJupWmgWZ3H83s9SXHjoFcEv1caRUVUqt9Lqt9LUWmIgpKQYx3vk4HXI/VeuFJbPo80mD56ToglQW0GtqlqEYCINBGR7qq6Ka6RJZnr+41lctafeXz+w8y61NkWSGWzqLLpvtNtjWYum0pYw9zo0JpPzVLrXkpt4qMsUZUEw+QXBylpgInK5xH80YrKgNfToG9kbehieRd5Daj4VxyKbjMVNA00Y9zAO/hw47us2LnU8fErXotysq1RQWkBs1Y8zwXH/4RjW3Sv83j+aJm1cbeAz9Mg2iiVFbCkBbyHtIgqq6i05JTcYvnL9KlqealY9HvnlpNtQMZk3ErzlBZMWhC/a1FOtzV6/etX2Fe0l7EOlZanNYK72xuSZGujVF2LqGap/qQoYDFHJ5YEtUtEykvKReQSYHf8QkpezVNaMCbjVt5ZP5c1u1c7Pn5+STCapJwZL6xhpi2ZTL/2AxjaZXidxyurfjPJx61tlMr+purSIsokr1jeTW4B/kdENovIZuB+4Ob4hpW8xg4YT5o/nSezHnV8bNXI0vBO+dd3H7Nu71rGDhjvyD/0Jn77BJvsEt1GqaoWUS3TAo60iDLJJ5ZefN+q6jCgD9BXVU9T1fXxDy05tW7ShtH9xjF37Wts2OfuX9PUJZNpn9aRS078qSPjpQUa9z0bDYnXIzRLjcyo0uPYncLniZTCt2gSea52zSIl8U0C3kZRXWmOLJb7oP4kIi1VNU9Vc0WklYj8oT6CS1a3DrqLFG8KT2b9X6JDqdY3e9bw6aYPuCHjZgLeul9STPElRxmvOTpOtlGqWNBQ1iKqTQJbRBn3i+UjygWqur/sgaruA34cv5CSX7v0DvzslBt5/etX2Hzgu0SHU6XpS58mxZvCz08Z48h4jaH1f2MmIqRHE1WzVB+eGDJVWUFD2SKMFQsarMLOxCKWBOUVkfJlVUWkCWDLrNbgtsyJeMTD5KzHat65nu0t3MNrq1/iit6jaJvWrs7jeT3SKBtZNkZHaqNUuUVUWUFDsrSIMu4TS4J6CfhYRMaIyI3Ah4Crltpwgog4ep69U9MujOx7PXNW/5Wtue5qvPHyihkUBgsZO+B2R8az0vLGp2IbpZZphxY0lFXYGVNXsRRJPAr8AegN9AV+r6rO3+jjAk5f5B+feQ9hDfN09uOOjlsXpaFSZix7ltO7nUXvtifXeTyRSPWeabxSfFbQYOIjpr8qVZ2nqveq6j1AnohMiXNcCZHmd7as9pgWx/LT3tfw0ooZ7Mrf4eDItffO+jfZlreVsQ6tmJtqpeXGmDiJKUGJSIaIPCIim4jMptbENaoE8URLXp105+D7KAmX8MyiJxwdt7amLp7McS1P4NweIxwZL81mT8aYOKk2QYlILxH5tYh8DUwGcogscHiWqj5VbxHWs7SAz9FZ1HGtTuDSE69k5vKp7ClMbAOORdsWsHh7FjcNuB2P1P2UTMDrsVM7xpi4OdK7yxrgHOAnqvqDaFJyroW2S3k9QorDs4K7htxPYWkB05ck9szoc4ufonlKC67u8zNHxrPScmNMPB0pQV0BbAc+FZFpInIOuKZFV1ylO/zGe2Kb3lzY81KmL3maA0X7az4gDnIObuaddXO59uQbSA80rfN4Xlu3xhgTZ9UmKFV9U1WvBk4CPgMmAh1E5BkROa+e4ksIn9fjeNPTCUPuJ7fkIM8vfcbRcWP1wrLnUJQbM25xZDyr3DPGxFssZeb5qvqyql4EdAWWAg/EPbIEK1sc0Cknt+/PecddyLQlk8kryXV07Jrkl+bz0ooZ/PiES+jW/Ng6jydYgjLGxN9RTRNUda+qPqeqZ8crILfwR1fjdNKEofezr2gvM5dNdXTcmry2+mUOFO/n5oF3ODJeirWpMcbUAyvBOoK0FGdnCQM7DubMY8/l2cVPUlBa4OjY1Slb8ymjwyAyOw1zZEzrHGGMqQ+WoI4gxefF5/BMYeLQB9hdsJOXV8xwdNzqfLLpA77dt45xA51Z88mWdDfG1Bd7p6mB09eihnYZzmldz2BK9l8oChY5OnZVpi2eTMf0TlzU83JHxrPZkzGmvliCqkE81qmZOPQBtudv42+rZjk6bmVrdq/mX5s/5oaMWxxZ86lstVNjjKkPcU1QIjJCRNaKyHoRqbLyT0SuEpHVIrJKRF6psD0kIkujX2/FM86apDvcRPYH3c4ks9NQnsp+jNJQqaNjVzR9yRRSvan87JQbHRnPbsw1xtSnuCUoEfECU4ALiCwXP0pE+lTapyfwS2C4qvYFJlT4caGqZkS/Lo5XnLFI9XtiWqAtViLChKEPkHNwM699/UrNB9TCnsLdvP71K1zZ51raNGlb5/EE67tnjKlf8ZxBDQHWq+oGVS0B5gCXVNpnLDAlukovqrozjvHUWmSRNmffnM/pfj792g/kqaz/IxgOOjo2wKzlz1MUKuImh9Z8SvFZabkxpn7FM0F1AbZUeJwT3VZRL6CXiHwpIvNFpGKL7VQRyY5uv7SqJxCRcdF9snft2uVs9JWkBbyOLmgoIkwcej8b93/LP9a+5tzAQEmohBeWPcuZx57LiW16OzKmnd4zxtS3eCaoqt7OtdJjH9ATOBMYBUwXkZbRnx2jqpnANcAkETn+sMFUp6pqpqpmtmtX96XLj6RsqWsnnX/8RZzUpi+TFj5KWMOOjfv2N2+wI387Ywc4s+aTzyO2Qqoxpt7F810nB+hW4XFXYGsV+/xDVUtVdSOwlkjCQlW3Rv+7gUgvwAFxjDUmTi9o6BEPE4bez7q9a3hn3VxHxlRVpi6ZzAmtenFW9x85MqbTidkYY2IRzwSVBfQUkR4iEgBGApWr8eYCZwGISFsip/w2iEgrEUmpsH04sDqOscbE4xFSHT7V9ZOel3NCq15MWvgIqpUnmEcva+tXLNux2LE1n0QiRSLGGFPf4vbOo6pBYDzwPvA18KqqrhKR34lIWVXe+8AeEVkNfArcp6p7gN5Atogsi25/WFUTnqAgUnLu5CzK6/Fy55BfsGrXcj7c8G6dx5u6ZDItU1pxZZ9rHYgu0hTWlnQ3xiSCOPGp3Q0yMzM1Ozu7Xp7rQGEpRaXOrd0YDAcZPrMfrVJb896of9c6IWw+8B3DXujDbYMm8uDpf3AktrZNUxy/UdkY07iJyKJojcER2bmbWnB6QUOfx8edg+9j6Y5FfPbdR7UeZ8ayZxCEGzJudiSuFJ/HkpMxJmEsQdVCPBY0vLLPtXRp1pXHFzxUq2tReSW5vLJyJhf1vIwuzbrVfEAMrLTcGJNIlqBqyenKtoA3wO2Z97Bw61f8J+ffR33831a/xMHiA4wb6ExpudcjpPgsQRljEscSVC0FfM4vaDjq5Otpn9aRxxc8dFTHhTXM9CVTGNhxMIM6DXUkFutaboxJNEtQdeD0goZNfE24LXMCX2z5jKytX8V83Ecb57Fx/7eOzZ5EbEl3Y0ziWYKqg3gsaPjzfjfRuklbHl/wSMzHTF38FJ2bduHCEy5zJIZUKy03xriAJag6cnpBw3R/OrcMvJNPNr3Psh2La9x/9a4VfLHlM27IuAW/1+9IDNa13BjjBpag6igeCxre0P9mWqa0YtKCh2vcd9qSKTTxNXFszaeA14PPlnQ3xriAvRM5wOkFDZulNOemAbfz3rdvs3rXimr321Wwk7+vmcOVfX5Gq9TWjjy3lZYbY9zCEpQDnF7QEOCmAbfRNNCMJxY+Wu0+s5Y/T3GomJsybnPkOW1Jd2OMm1iCckA8FjRsmdqKG/vfwlvfvMG6vWsP+3lxsJgXlj3HWd3Po1ebkxx5TistN8a4iSUohzi9oCHAuIF3kOprUuUs6h/fvM6ugh3cPPAOR55LsNJyY4y7WIJySDwWNGyb1o7r+t3Em2v+xqb9G8q3qyrTlkymZ+uT+OEx5zjyXCl+W9LdGOMulqAc5PSChgC3DZqAz+PjqazHyrfN//5LVuxcyrgB4x27X8lO7xlj3MYSlIPisaBhh6aduObk0by6+iVyDm4GYNqSp2iV2poreo9y5Dn8Xg9+Ky03xriMvSs5zOkFDQHGD74HgCnZf+G7/Rt5b/3b/PyUMaT50xwZ32ZPxhg3cvaiiYl0Afd7HV3QsEuzblzd5+e8snIm+4r24vV4HVvzyUrLjTFuZTOoOHB6QUOAOwbfSzAcZO7a17i41xV0atrFkXHtxlxjjFtZgoqDeCxoeGzLHlx+0kgAbhpwuyNjCtZ3zxjjXnaKL07SAj6KgyWOjvm/P3yEC3tewsCOgx0ZL8VnpeXGGPeyGVScxGNBw9ZN2jDi+J84Np6d3jPGuJklqDhyekFDJ/k8QsDh05DGGOMke4eKo3gsaOgUp7teGGOM0yxBxZnTCxo6QSTSgd0YY9zM3qXiLB4LGtZVE1vS3RiTBCxB1QOnFzSsKzu9Z4xJBpag6kE8FjSsrRSfx3UzOmOMqYolqHoQjwUNa8tKy40xycISVD2Jx4KGR8vrEVJ8lqCMMcnBElQ9iceChkfLLbM4Y4yJhSWoehSPBQ1jZUu6G2OSjSWoehSPBQ1jlRqw0nJjTHKxBFXPElVybl3LjTHJxhJUPfN6hNR6LlQIeD34bEl3Y0ySsXetBEiv5yayVlpujElGlqASIB4LGlbHlnQ3xiQrS1AJUl8l51ZaboxJVpagEiTg8+CP83UhKy03xiQzS1AJFO9rUSl+W9LdGJO8LEElULwXNLTTe8aYZBbXBCUiI0RkrYisF5EHqtnnKhFZLSKrROSVCtuvF5F10a/r4xlnIsVrQUO/N/6nEI0xJp7idqVeRLzAFOBHQA6QJSJvqerqCvv0BH4JDFfVfSLSPrq9NfAbIBNQYFH02H3xijdRUv1e8oqDhMLq6Lg2ezLGJLt4fsQeAqxX1Q2qWgLMAS6ptM9YYEpZ4lHVndHt5wMfqure6M8+BEbEMdaEcrq7hEek3srYjTEmXuL5LtYF2FLhcU50W0W9gF4i8qWIzBeREUdxLCIyTkSyRSR7165dDoZev5xe0LCJ9d0zxjQA8UxQVb1DVj6P5QN6AmcCo4DpItIyxmNR1amqmqmqme3atatjuInj5IKGVlpujGko4pmgcoBuFR53BbZWsc8/VLVUVTcCa4kkrFiObVCcWtAwxee1Jd2NMQ1CPBNUFtBTRHqISAAYCbxVaZ+5wFkAItKWyCm/DcD7wHki0kpEWgHnRbc1WCLiyMzH+u4ZYxqKuFXxqWpQRMYTSSxeYIaqrhKR3wHZqvoW/01Eq4EQcJ+q7gEQkd8TSXIAv1PVvfGK1S3SAz4KS0KHn8uMkc8jBKw4whjTQIiqs+XNiZKZmanZ2dmJDqPODhaVUlgSqtWxzVP9NoMyxrieiCxS1cya9rOP2y5T25JzkUg1oDHGNBT2juYytV3QsInfSsuNMQ2LJSgXqk0T2fpavsMYY+qLJSgXOtoFDVN8HistN8Y0OJagXOpoZkRWGGGMaYgsQblUrAsaej1CSi2uWRljjNtZgnKxWK5FWddyY0xDZQnKxWpa0ND67hljGjJLUC53pAUNU61ruTGmAbME5XKp/uqbv6bZ7MkY04BZgkoCVV1nCng9+GxJd2NMA2bvcEkg0iWi0jYrjjDGNHCWoJKAiBzSo88jQqqd3jPGNHCWoJJExQUNrbTcGNMYWIJKEmULGlppuTGmsbAOo0kkPeBDAY/13TPGNAI2g0oiHo/Q7Aj3RRljTENiCSrJ2I25xpjGwhKUMcYYV7IEZYwxxpUsQRljjHElS1DGGGNcyRKUMcYYV7IEZYwxxpUsQRljjHElS1DGGGNcyRKUMcYYVxJVTXQMjhCRXcB3dRymLbDbgXDiKRlihOSI02J0TjLEaTE6p65xHquq7WraqcEkKCeISLaqZiY6jiNJhhghOeK0GJ2TDHFajM6przjtFJ8xxhhXsgRljDHGlSxBHWpqogOIQTLECMkRp8XonGSI02J0Tr3EadegjDHGuJLNoIwxxriSJShjjDGuZAkKEJEZIrJTRFYmOpbqiEg3EflURL4WkVUicleiY6pMRFJFZKGILIvG+L+Jjqk6IuIVkSUi8s9Ex1IdEdkkIitEZKmIZCc6nqqISEsReV1E1kT/Nk9NdEyViciJ0d9h2ddBEZmQ6LgqE5GJ0X83K0VktoikJjqmykTkrmh8q+rjd2jXoAAROQPIA/6qqicnOp6qiEgnoJOqLhaRZsAi4FJVXZ3g0MpJZD36dFXNExE/8AVwl6rOT3BohxGRu4FMoLmqXpToeKoiIpuATFV17Y2bIvIi8G9VnS4iASBNVfcnOq7qiIgX+B4Yqqp1vbHfMSLShci/lz6qWigirwLvqurMxEb2XyJyMjAHGAKUAPOAW1V1Xbye02ZQgKp+DuxNdBxHoqrbVHVx9Ptc4GugS2KjOpRG5EUf+qNfrvsEJCJdgQuB6YmOJZmJSHPgDOB5AFUtcXNyijoH+NZNyakCH9BERHxAGrA1wfFU1huYr6oFqhoE/gVcFs8ntASVhESkOzAAWJDYSA4XPXW2FNgJfKiqrosRmAT8AggnOpAaKPCBiCwSkXGJDqYKxwG7gBeip0uni0h6ooOqwUhgdqKDqExVvwceAzYD24ADqvpBYqM6zErgDBFpIyJpwI+BbvF8QktQSUZEmgJvABNU9WCi46lMVUOqmgF0BYZETwu4hohcBOxU1UWJjiUGw1V1IHABcHv0VLSb+ICBwDOqOgDIBx5IbEjVi56CvBh4LdGxVCYirYBLgB5AZyBdRH6W2KgOpapfA48AHxI5vbcMCMbzOS1BJZHodZ03gJdV9e+JjudIoqd6PgNGJDiUyoYDF0ev78wBzhaRlxIbUtVUdWv0vzuBN4mc+3eTHCCnwiz5dSIJy60uABar6o5EB1KFc4GNqrpLVUuBvwOnJTimw6jq86o6UFXPIHJZJG7Xn8ASVNKIFiA8D3ytqn9JdDxVEZF2ItIy+n0TIv/o1iQ2qkOp6i9VtauqdidyuucTVXXVJ1UAEUmPFsMQPW12HpFTLK6hqtuBLSJyYnTTOYBrinaqMAoXnt6L2gwME5G06L/1c4hcZ3YVEWkf/e8xwOXE+ffpi+fgyUJEZgNnAm1FJAf4jao+n9ioDjMc+DmwInqNB+B/VPXdBMZUWSfgxWillAd4VVVdW8btch2ANyPvVfiAV1R1XmJDqtIdwMvR02cbgBsSHE+VotdMfgTcnOhYqqKqC0TkdWAxkdNmS3Bn26M3RKQNUArcrqr74vlkVmZujDHGlewUnzHGGFeyBGWMMcaVLEEZY4xxJUtQxhhjXMkSlDHGGFeyBGUaDBFREflzhcf3ishvHRp7poj81ImxanieK6NdwT+ttL179PXdUWHbZBEZXcN49RX36yJyXPT7TSLSNvr9IBHZKCIDROQiN3e4N+5jCco0JMXA5WVvjm4RvS8sVmOA21T1rCp+thO4K3rPUdxFm5bGsl9fwKuqGypt70eku8TVqroEeIdIF480x4M1DZIlKNOQBInc3Dix8g8qzyREJC/63zNF5F8i8qqIfCMiD4vItRJZ12qFiBxfYZhzReTf0f0uih7vFZH/E5EsEVkuIjdXGPdTEXkFWFFFPKOi468UkUei234N/AB4VkT+r4rXtwv4GLi+ivHGRmNYJiJvVEoCVcWdKiIvRGNYIiJnRbePFpHXRORtIo1qO4nI5xJZR2mliJxeRVzXAv+otK03MBf4uaouhEi3eyLtr1y5vIlxH0tQpqGZAlwrIi2O4pj+wF3AKUS6dfRS1SFEluO4o8J+3YEfElmq41mJLCg3hkjn6cHAYGCsiPSI7j8E+JWq9qn4ZCLSmUjTzbOBDGCwiFyqqr8DsoFrVfW+amJ9GLinilnZ31V1sKr2J9IiZ0wNcd8OoKqnEGkB9KL8d4G8U4HrVfVs4Brg/WgD4P7AUg43nMj6ZBX9Axivql9U2p4NVJXkjDmMJSjToEQ7vP8VuPMoDsuKrrdVDHwLlC1zsILIm3uZV1U1HF2gbQNwEpEeeddF208tANoAPaP7L1TVjVU832Dgs2hj0CDwMpF1lWJ5fRuBhUQSR0UnR2dJK4jMaPrWEPcPgFnRMdcA3wG9ovt/qKpl66NlATdEr+WdEl2LrLJORGZ3FX0E3FRFIt1JpFu3MTWyBGUaoklEZhAV1yYKEv17jzbjrHgdp7jC9+EKj8Mc2q+ycl8wBQS4Q1Uzol89Kqzjk19NfBLrC6nGn4D7OfTf70wiM5ZTgP8FKi4XXl3c1SmPO7qY5xlEVqGdJSLXVbF/YaXnAxgf/e/TlbanRvc3pkaWoEyDE/30/yqHnubaBAyKfn8JkdV+j9aVIuKJXpc6DlgLvA/cKpGlUBCRXlLzon0LgB+KSNvoDGMUkdVJYxKd8azm0Gs5zYBt0TiujSHuz8v2E5FewDHR7YcQkWOJrJ81jUg3/aqW0/gaOKHStnD0dZ0oIr+rsL0XLuvKbtzLEpRpqP4MVKzmm0YkKSwEhlL97OZI1hJJJO8Bt6hqEZHrVKuBxSKyEniOGlYJUNVtwC+BT4ks+rZYVSsXGdTkj0QWhSzz/4gkvg85fImTquJ+GvBGTwn+DRgdPcVZ2ZnAUhFZAlwBPFHFPu9E9ztEdLxLiFTu3R7dfFZ0f2NqZN3MjTF1IpG1vz4lsgJw6Aj7dSCybMg59RacSWqWoIwxdSYi5xNZTHPzEfYZDJSqalWVgMYcxhKUMcYYV7JrUMYYY1zJEpQxxhhXsgRljDHGlSxBGWOMcSVLUMYYY1zp/wPBsI7pV3cQagAAAABJRU5ErkJggg==\n", 
                        "text/plain": "<Figure size 432x288 with 1 Axes>"
                    }, 
                    "metadata": {
                        "needs_background": "light"
                    }
                }, 
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "The best accuracy was with 0.75 with k= 9\n"
                }
            ], 
            "source": "from sklearn import metrics\n\nKs = 10\nmean_acc = np.zeros((Ks-1))\nstd_acc = np.zeros((Ks-1))\nConfustionMx = [];\nfor n in range(1,Ks):\n    \n    #Train Model and Predict  \n    neigh = KNeighborsClassifier(n_neighbors = n).fit(X_train,y_train)\n    yhat_knn=neigh.predict(X_test)\n    mean_acc[n-1] = metrics.accuracy_score(y_test, yhat_knn)\n\n    \n    std_acc[n-1]=np.std(yhat_knn==y_test)/np.sqrt(yhat_knn.shape[0])\n    \n    print(\"while n = {0}\".format(n))\n    print(\"Train set Accuracy: \", metrics.accuracy_score(y_train, neigh.predict(X_train)))\n    print(\"Test set Accuracy: \", metrics.accuracy_score(y_test, yhat_knn))\n    print('\\n')\n\n#mean_acc\n\nplt.plot(range(1,Ks),mean_acc,'g')\nplt.fill_between(range(1,Ks),mean_acc - 1 * std_acc,mean_acc + 1 * std_acc, alpha=0.10)\nplt.legend(('Accuracy ', '+/- 3xstd'))\nplt.ylabel('Accuracy ')\nplt.xlabel('Number of Nabors (K)')\nplt.tight_layout()\nplt.show()\n\nprint( \"The best accuracy was with\", mean_acc.max(), \"with k=\", mean_acc.argmax()+1) "
        }, 
        {
            "source": "#### The best K is 9\nLet us train and run predicting", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 24, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF']\n"
                }
            ], 
            "source": "k = 9\nneigh = KNeighborsClassifier(n_neighbors = k).fit(X_train,y_train)\n#neigh\nyhat_knn = neigh.predict(X_test)\nprint(yhat_knn[0:5])"
        }, 
        {
            "source": "# Decision Tree", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Build DecisionTree model\nTrain the model with given datasets", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 27, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 27, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=6,\n            max_features=None, max_leaf_nodes=None,\n            min_impurity_decrease=0.0, min_impurity_split=None,\n            min_samples_leaf=1, min_samples_split=2,\n            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n            splitter='best')"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "from sklearn.tree import DecisionTreeClassifier\n#Classification model training\nloan_decision_tree = DecisionTreeClassifier(criterion=\"entropy\", max_depth = 6)\nloan_decision_tree.fit(X_train, y_train)"
        }, 
        {
            "source": "#### Using the DecisionTree model predict the test dataset", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 28, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF']\nDecisionTrees's Accuracy:  0.7403846153846154\n"
                }
            ], 
            "source": "predTree = loan_decision_tree.predict(X_test)\nprint (predTree [0:5])\nprint(\"DecisionTrees's Accuracy: \", metrics.accuracy_score(y_test, predTree))\n"
        }, 
        {
            "source": "# Support Vector Machine", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Build the SVM model and Train it with given datasets", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 30, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/svm/base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n  \"avoid this warning.\", FutureWarning)\n"
                }, 
                {
                    "execution_count": 30, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n  decision_function_shape='ovr', degree=3, gamma='auto_deprecated',\n  kernel='rbf', max_iter=-1, probability=False, random_state=None,\n  shrinking=True, tol=0.001, verbose=False)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "from sklearn import svm\nclf = svm.SVC(kernel='rbf')\nclf.fit(X_train, y_train) "
        }, 
        {
            "source": "#### Run predict with the trained SVM model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 31, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 31, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n      dtype=object)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "yhat_svm = clf.predict(X_test)\nyhat_svm [0:5]"
        }, 
        {
            "execution_count": 32, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# The code was removed by Watson Studio for sharing."
        }, 
        {
            "source": "#### Check the SVM model classification report", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 37, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "              precision    recall  f1-score   support\n\n  COLLECTION       1.00      0.04      0.07        27\n     PAIDOFF       0.75      1.00      0.86        77\n\n   micro avg       0.75      0.75      0.75       104\n   macro avg       0.87      0.52      0.46       104\nweighted avg       0.81      0.75      0.65       104\n\n"
                }
            ], 
            "source": "# Compute confusion matrix\n#cnf_matrix = confusion_matrix(y_test, yhat_svm, labels=['COLLECTION', 'PAIDOFF'])\nnp.set_printoptions(precision=2)\n\nprint (classification_report(y_test, yhat_svm))\n\n# Plot non-normalized confusion matrix\n#plt.figure()\n#plot_confusion_matrix(cnf_matrix, classes=['COLLECTION','PAIDOFF'],normalize= False,  title='Confusion matrix')"
        }, 
        {
            "source": "# Logistic Regression", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Build the Logistic Regression model and train it with the given datasets", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 39, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "import scipy.optimize as opt\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.metrics import confusion_matrix\nLR = LogisticRegression(C=0.01, solver='liblinear').fit(X_train,y_train)"
        }, 
        {
            "source": "#### Run prediction with the trained model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 40, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 40, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n      dtype=object)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "yhat_lg = LR.predict(X_test)\nyhat_lg[0:5]\n"
        }, 
        {
            "source": "#### Chek the Probability values for classes\n**predict_proba** returns estimates for all classes, ordered by the label of classes. So, the first column is the probability of class 1, P(Y=1|X), and second column is probability of class 0, P(Y=0|X):", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 41, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 41, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "array([[0.49, 0.51],\n       [0.45, 0.55],\n       [0.32, 0.68],\n       [0.36, 0.64],\n       [0.34, 0.66]])"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "yhat_prob = LR.predict_proba(X_test)\nyhat_prob[0:5]"
        }, 
        {
            "source": "# Model Evaluation using Test set", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Get required package ready", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 42, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn.metrics import jaccard_similarity_score\nfrom sklearn.metrics import f1_score\nfrom sklearn.metrics import log_loss"
        }, 
        {
            "source": "First, download and load the test set:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 43, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "--2019-07-10 18:05:33--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.193\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.193|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 3642 (3.6K) [text/csv]\nSaving to: \u2018loan_test.csv\u2019\n\n100%[======================================>] 3,642       --.-K/s   in 0s      \n\n2019-07-10 18:05:33 (700 MB/s) - \u2018loan_test.csv\u2019 saved [3642/3642]\n\n"
                }
            ], 
            "source": "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
        }, 
        {
            "source": "### Load Test set for evaluation ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 44, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 44, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "(54, 10)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "test_df = pd.read_csv('loan_test.csv')\n#test_df.dropna(axis=0, inplace=True)\ntest_df.shape"
        }, 
        {
            "source": "#### Preprocessing the test data", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 45, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 45, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>2016-09-09</td>\n      <td>2016-09-15</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>1</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>26</td>\n      <td>college</td>\n      <td>0</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>2016-09-11</td>\n      <td>2016-09-25</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>0</td>\n      <td>6</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30     2016-09-08   \n1           5             5     PAIDOFF        300      7     2016-09-09   \n2          21            21     PAIDOFF       1000     30     2016-09-10   \n3          24            24     PAIDOFF       1000     30     2016-09-10   \n4          35            35     PAIDOFF        800     15     2016-09-11   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   50              Bechalor       1          3        0  \n1 2016-09-15   35       Master or Above       0          4        1  \n2 2016-10-09   43  High School or Below       1          5        1  \n3 2016-10-09   26               college       0          5        1  \n4 2016-09-25   29              Bechalor       0          6        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "test_df['due_date'] = pd.to_datetime(test_df['due_date'])\ntest_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\n#test_df.head()\n\ntest_df['loan_status'].value_counts()\n\ntest_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\ntest_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n\ntest_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n\ntest_df.head()\n"
        }, 
        {
            "execution_count": 46, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 46, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>50.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>300.0</td>\n      <td>7.0</td>\n      <td>35.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>43.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>26.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>800.0</td>\n      <td>15.0</td>\n      <td>29.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms   age  Gender  weekend  Bechalor  High School or Below  \\\n0     1000.0   30.0  50.0     1.0      0.0         0                     1   \n1      300.0    7.0  35.0     0.0      1.0         1                     0   \n2     1000.0   30.0  43.0     1.0      1.0         0                     0   \n3     1000.0   30.0  26.0     0.0      1.0         0                     0   \n4      800.0   15.0  29.0     0.0      1.0         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "test_Feature = test_df[['Principal','terms','age','Gender','weekend']]\ntest_Feature = pd.concat([test_Feature,pd.get_dummies(df['education'])], axis=1)\ntest_Feature.drop(['Master or Above'], axis = 1,inplace=True)\ntest_Feature.head()"
        }, 
        {
            "execution_count": 47, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 47, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "PAIDOFF       40\nCOLLECTION    14\nName: loan_status, dtype: int64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "test_df['loan_status'].value_counts()\n"
        }, 
        {
            "source": "#### Remove all rows with NaN value", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 48, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": false
            }, 
            "outputs": [
                {
                    "execution_count": 48, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>50.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>300.0</td>\n      <td>7.0</td>\n      <td>35.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>43.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>26.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>800.0</td>\n      <td>15.0</td>\n      <td>29.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>700.0</td>\n      <td>15.0</td>\n      <td>33.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>1000.0</td>\n      <td>15.0</td>\n      <td>24.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>32.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>800.0</td>\n      <td>15.0</td>\n      <td>27.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>1000.0</td>\n      <td>15.0</td>\n      <td>37.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>10</th>\n      <td>800.0</td>\n      <td>15.0</td>\n      <td>24.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>11</th>\n      <td>300.0</td>\n      <td>7.0</td>\n      <td>35.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>12</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>31.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>13</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>37.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>14</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>37.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>15</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>33.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>16</th>\n      <td>800.0</td>\n      <td>15.0</td>\n      <td>43.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>17</th>\n      <td>1000.0</td>\n      <td>7.0</td>\n      <td>32.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>18</th>\n      <td>1000.0</td>\n      <td>15.0</td>\n      <td>26.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>19</th>\n      <td>1000.0</td>\n      <td>7.0</td>\n      <td>29.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>20</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>30.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>21</th>\n      <td>1000.0</td>\n      <td>7.0</td>\n      <td>27.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>22</th>\n      <td>300.0</td>\n      <td>7.0</td>\n      <td>37.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>23</th>\n      <td>1000.0</td>\n      <td>15.0</td>\n      <td>29.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>24</th>\n      <td>1000.0</td>\n      <td>15.0</td>\n      <td>26.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>25</th>\n      <td>800.0</td>\n      <td>30.0</td>\n      <td>28.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>26</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>38.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>27</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>46.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>28</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>33.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>29</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>29.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>30</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>29.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>31</th>\n      <td>1000.0</td>\n      <td>15.0</td>\n      <td>36.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>32</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>29.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>33</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>30.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>34</th>\n      <td>1000.0</td>\n      <td>15.0</td>\n      <td>36.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>35</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>29.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>36</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>28.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>37</th>\n      <td>800.0</td>\n      <td>15.0</td>\n      <td>23.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>38</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>38.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>39</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>30.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>40</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>33.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>41</th>\n      <td>1000.0</td>\n      <td>15.0</td>\n      <td>31.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>42</th>\n      <td>800.0</td>\n      <td>15.0</td>\n      <td>41.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>43</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>30.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>44</th>\n      <td>800.0</td>\n      <td>15.0</td>\n      <td>26.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>45</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>20.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>46</th>\n      <td>1000.0</td>\n      <td>15.0</td>\n      <td>26.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>47</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>24.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>48</th>\n      <td>800.0</td>\n      <td>15.0</td>\n      <td>27.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>49</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>32.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>50</th>\n      <td>800.0</td>\n      <td>15.0</td>\n      <td>29.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>51</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>37.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>52</th>\n      <td>800.0</td>\n      <td>15.0</td>\n      <td>36.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>53</th>\n      <td>1000.0</td>\n      <td>30.0</td>\n      <td>33.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "    Principal  terms   age  Gender  weekend  Bechalor  High School or Below  \\\n0      1000.0   30.0  50.0     1.0      0.0         0                     1   \n1       300.0    7.0  35.0     0.0      1.0         1                     0   \n2      1000.0   30.0  43.0     1.0      1.0         0                     0   \n3      1000.0   30.0  26.0     0.0      1.0         0                     0   \n4       800.0   15.0  29.0     0.0      1.0         0                     0   \n5       700.0   15.0  33.0     0.0      1.0         0                     0   \n6      1000.0   15.0  24.0     0.0      1.0         0                     0   \n7      1000.0   30.0  32.0     0.0      1.0         0                     0   \n8       800.0   15.0  27.0     1.0      1.0         0                     0   \n9      1000.0   15.0  37.0     0.0      1.0         0                     1   \n10      800.0   15.0  24.0     0.0      1.0         0                     0   \n11      300.0    7.0  35.0     0.0      1.0         0                     0   \n12     1000.0   30.0  31.0     0.0      1.0         0                     1   \n13     1000.0   30.0  37.0     1.0      1.0         0                     0   \n14     1000.0   30.0  37.0     1.0      1.0         0                     1   \n15     1000.0   30.0  33.0     0.0      1.0         0                     1   \n16      800.0   15.0  43.0     0.0      1.0         0                     1   \n17     1000.0    7.0  32.0     1.0      1.0         0                     0   \n18     1000.0   15.0  26.0     0.0      1.0         0                     0   \n19     1000.0    7.0  29.0     0.0      1.0         0                     1   \n20     1000.0   30.0  30.0     0.0      1.0         0                     0   \n21     1000.0    7.0  27.0     0.0      1.0         0                     1   \n22      300.0    7.0  37.0     0.0      0.0         1                     0   \n23     1000.0   15.0  29.0     0.0      0.0         1                     0   \n24     1000.0   15.0  26.0     0.0      0.0         0                     0   \n25      800.0   30.0  28.0     0.0      0.0         0                     0   \n26     1000.0   30.0  38.0     0.0      0.0         0                     1   \n27     1000.0   30.0  46.0     0.0      0.0         0                     0   \n28     1000.0   30.0  33.0     0.0      0.0         0                     0   \n29     1000.0   30.0  29.0     0.0      0.0         0                     0   \n30     1000.0   30.0  29.0     0.0      0.0         0                     0   \n31     1000.0   15.0  36.0     0.0      0.0         0                     1   \n32     1000.0   30.0  29.0     0.0      0.0         1                     0   \n33     1000.0   30.0  30.0     0.0      0.0         0                     0   \n34     1000.0   15.0  36.0     0.0      0.0         0                     0   \n35     1000.0   30.0  29.0     0.0      0.0         1                     0   \n36     1000.0   30.0  28.0     0.0      0.0         0                     0   \n37      800.0   15.0  23.0     0.0      0.0         1                     0   \n38     1000.0   30.0  38.0     1.0      0.0         0                     1   \n39     1000.0   30.0  30.0     1.0      0.0         0                     1   \n40     1000.0   30.0  33.0     0.0      1.0         0                     0   \n41     1000.0   15.0  31.0     1.0      1.0         0                     1   \n42      800.0   15.0  41.0     0.0      1.0         1                     0   \n43     1000.0   30.0  30.0     0.0      1.0         1                     0   \n44      800.0   15.0  26.0     1.0      1.0         0                     1   \n45     1000.0   30.0  20.0     0.0      1.0         0                     1   \n46     1000.0   15.0  26.0     0.0      1.0         0                     0   \n47     1000.0   30.0  24.0     1.0      1.0         0                     1   \n48      800.0   15.0  27.0     0.0      1.0         1                     0   \n49     1000.0   30.0  32.0     0.0      1.0         0                     1   \n50      800.0   15.0  29.0     0.0      1.0         0                     0   \n51     1000.0   30.0  37.0     0.0      1.0         1                     0   \n52      800.0   15.0  36.0     0.0      1.0         0                     1   \n53     1000.0   30.0  33.0     0.0      0.0         0                     0   \n\n    college  \n0         0  \n1         0  \n2         1  \n3         1  \n4         1  \n5         1  \n6         1  \n7         1  \n8         1  \n9         0  \n10        1  \n11        1  \n12        0  \n13        1  \n14        0  \n15        0  \n16        0  \n17        1  \n18        1  \n19        0  \n20        1  \n21        0  \n22        0  \n23        0  \n24        1  \n25        1  \n26        0  \n27        1  \n28        1  \n29        1  \n30        1  \n31        0  \n32        0  \n33        1  \n34        1  \n35        0  \n36        1  \n37        0  \n38        0  \n39        0  \n40        1  \n41        0  \n42        0  \n43        0  \n44        0  \n45        0  \n46        1  \n47        0  \n48        0  \n49        0  \n50        0  \n51        0  \n52        0  \n53        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "test_Feature.dropna(inplace=True)\ntest_Feature"
        }, 
        {
            "source": "#### Finishe the test datasets", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 50, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, float64 were all converted to float64 by StandardScaler.\n  return self.partial_fit(X, y)\n/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:3: DataConversionWarning: Data with input dtype uint8, float64 were all converted to float64 by StandardScaler.\n  app.launch_new_instance()\n"
                }
            ], 
            "source": "y_testset = test_df['loan_status'].values\nX_testset = test_Feature\nX_testset= preprocessing.StandardScaler().fit(X_testset).transform(X_testset)\n"
        }, 
        {
            "source": "#### Run prediction with the same test datset for each individual model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 51, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "yhat_knn_test = neigh.predict(X_testset)"
        }, 
        {
            "execution_count": 52, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "predTree_test = loan_decision_tree.predict(X_testset)"
        }, 
        {
            "execution_count": 53, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "yhat_svm_test = clf.predict(X_testset)"
        }, 
        {
            "execution_count": 56, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "\nyhat_LR_test = LR.predict(X_testset)\n\nyhat_prob_test = LR.predict_proba(X_testset)\n\n#f1_score(y_testset, yhat_LR, average='weighted') "
        }, 
        {
            "source": "#### Get Jaccard accuracy for each prediction", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 64, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "knn_jaccard_test = jaccard_similarity_score(y_testset, yhat_knn_test)\ntree_jaccard_test = jaccard_similarity_score(y_testset, predTree_test)\nsvm_jaccard_test = jaccard_similarity_score(y_testset, yhat_svm_test)\nlr_jaccard_test = jaccard_similarity_score(y_testset, yhat_LR_test)\n#jaccard_similarity_score(y_test, yhat)"
        }, 
        {
            "source": "#### Get F1-Score accuracy fro each prediction", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 66, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "knn_F1_score_test = f1_score(y_testset, yhat_knn_test, average='weighted') \ntree_F1_score_test = f1_score(y_testset, predTree_test, average='weighted') \nsvm_F1_score_test = f1_score(y_testset, yhat_svm_test, average='weighted') \nlr_F1_score_test = f1_score(y_testset, yhat_LR_test, average='weighted') "
        }, 
        {
            "source": "#### Get the LogLoss accuracy for Logistic Regression model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 67, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "\nlr_LogLoss_test = log_loss(y_test, yhat_prob_test)"
        }, 
        {
            "source": "#### Creat the accuracy report for all models", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 68, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 68, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Algorithm</th>\n      <th>Jaccard</th>\n      <th>F1-score</th>\n      <th>LogLoss</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>KNN</td>\n      <td>0.740741</td>\n      <td>0.700673</td>\n      <td>NA</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>Decision Tree</td>\n      <td>0.796296</td>\n      <td>0.742704</td>\n      <td>NA</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>SVM</td>\n      <td>0.759259</td>\n      <td>0.671764</td>\n      <td>NA</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>LogisticRegression</td>\n      <td>0.759259</td>\n      <td>0.671764</td>\n      <td>0.578272</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "            Algorithm   Jaccard  F1-score   LogLoss\n0                 KNN  0.740741  0.700673        NA\n1       Decision Tree  0.796296  0.742704        NA\n2                 SVM  0.759259  0.671764        NA\n3  LogisticRegression  0.759259  0.671764  0.578272"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "# create a Pandas DataFrame for accuracy report\ndataset = {\"Algorithm\": [\"KNN\", \"Decision Tree\", \"SVM\", \"LogisticRegression\"],\n           \"Jaccard\": [knn_jaccard_test, tree_jaccard_test, svm_jaccard_test, lr_jaccard_test],\n           \"F1-score\": [knn_F1_score_test, tree_F1_score_test,  svm_F1_score_test, lr_F1_score_test],\n           \"LogLoss\": [\"NA\", \"NA\", \"NA\", lr_LogLoss_test]}\ndf = pd.DataFrame(dataset, columns = ['Algorithm', 'Jaccard', 'F1-score', 'LogLoss'])\ndf"
        }, 
        {
            "source": "# Report\nYou should be able to report the accuracy of the built model using different evaluation metrics:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "| Algorithm          | Jaccard | F1-score | LogLoss |\n|--------------------|---------|----------|---------|\n| KNN                | ?       | ?        | NA      |\n| Decision Tree      | ?       | ?        | NA      |\n| SVM                | ?       | ?        | NA      |\n| LogisticRegression | ?       | ?        | ?       |", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<h2>Want to learn more?</h2>\n\nIBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems \u2013 by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"http://cocl.us/ML0101EN-SPSSModeler\">SPSS Modeler</a>\n\nAlso, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://cocl.us/ML0101EN_DSX\">Watson Studio</a>\n\n<h3>Thanks for completing this lesson!</h3>\n\n<h4>Author:  <a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a></h4>\n<p><a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients\u2019 ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.</p>\n\n<hr>\n\n<p>Copyright &copy; 2018 <a href=\"https://cocl.us/DX0108EN_CC\">Cognitive Class</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.</p>", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }
    ], 
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.6", 
            "name": "python3", 
            "language": "python"
        }, 
        "language_info": {
            "mimetype": "text/x-python", 
            "nbconvert_exporter": "python", 
            "version": "3.6.8", 
            "name": "python", 
            "file_extension": ".py", 
            "pygments_lexer": "ipython3", 
            "codemirror_mode": {
                "version": 3, 
                "name": "ipython"
            }
        }
    }, 
    "nbformat": 4
}