notebooks/03_Preprocessing_and_Training.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "93e83bfd",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# 3.0 Pre-processing and Training\n",
    "\n",
    "Contents\n",
    "\n",
    "3.1 [Introduction](#3.1)\n",
    "\n",
    "  * [3.1.1 Problem Recap](#3.1.1)\n",
    "  * [3.1.2 Notebook Goals](#3.1.2)\n",
    " \n",
    "3.2 [Load the data](#3.2)\n",
    "\n",
    "  * [3.2.1 Imports](#3.2.1)\n",
    "  * [3.2.2 Load the data](#3.2.2)\n",
    "\n",
    "3.3 [Examine Class Split](#3.3)\n",
    "\n",
    "3.4 [Pre-processing](#3.4)\n",
    "\n",
    "  * [3.4.1 Set Random Seed for Reproducability](#3.4.1)\n",
    "  * [3.4.2 Train/test Split](#3.4.2)\n",
    "  * [3.4.3 Examine Class Split for Train/Test Data](#3.4.3)\n",
    "  \n",
    "\n",
    "3.5 [Building some Initial Models](#3.5)\n",
    "  * 3.5.1 [Logistic Regression with Count Vectorization](#3.5.1)\n",
    "<br/><br/>\n",
    "    * [3.5.1.1 Training and Fitting the Model](#3.5.1.1)\n",
    "    * [3.5.1.2 Evaluating the Model](#3.5.1.2)\n",
    "<br/><br/>\n",
    "  * 3.5.2 [Logistic Regression with TFIDF Vectorization](#3.5.2)\n",
    "<br/><br/>\n",
    "    * [3.5.2.1 Training and Fitting Model](#3.5.2.1)\n",
    "    * [3.5.2.2 Evaluating the Model](#3.5.2.2)\n",
    "<br/><br/>\n",
    "  * 3.5.3 [Naive Bayes](#3.5.3)\n",
    "<br/> <br/> \n",
    "    * [3.5.3.1 Training and Fitting the Model](#3.5.3.1)\n",
    "    * [3.5.3.2 Evaluating the Model](#3.5.3.2)\n",
    "<br/> <br/>\n",
    "  * 3.5.4 [Random Forest Classifier](#3.5.4)\n",
    "<br/>  <br/>\n",
    "    * [3.5.4.1 Training and Fitting the Model](#3.5.4.1)\n",
    "    * [3.5.4.2 Evaluating the Model](#3.5.4.2)\n",
    "\n",
    "3.6 [Conclusions](#3.6)\n",
    "  * 3.6.1 [Conclusions](#3.6.1)\n",
    "  * 3.6.2 [Next Steps](#3.6.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75b9c841",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 3.1 Introduction <a name=\"3.1\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3d02b97",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 3.1.1 Problem Recap <a name=\"3.1.1\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cadbd87",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Using customer text data about amazon products, we will build, evaluate and compare models to estimate the probability that a given text review can be classified as “positive” or “negative”.\n",
    "\n",
    "Our goal is to build a text classifier using Amazon product review data which can be used to analyze customer sentiment which does not have accompanying numeric data. The metric we will be primarily interested in will be Recall on the positive class. This is the proportion of the positive class (negative reviews coded as \"1\" in the data) we correctly predict."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27bc4983",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 3.1.2 Notebook Goals <a name=\"3.1.2\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aeda4730",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "1. Examine the split between the class we're trying to predict.\n",
    "\n",
    "2. Split the data into train and test sets.\n",
    "\n",
    "3. Convert our text review data into numeric features. (CountVectorization, TFIDF, and word2vec are options)\n",
    "\n",
    "4. Build a few simple, initial predictive models.\n",
    "\n",
    "5. Logistic Regression predicts probabilities. With \"positive\" (0) and \"negative\" (1) review, we can do a binary classification.\n",
    "\n",
    "6. Measure the performance of the model with a Classification Report, Confusion Matrix and ROC + AUC."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71d646fd",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 3.2 Load the data <a name=\"3.2\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f718e2c",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 3.2.1 Imports <a name=\"3.2.1\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "589120c8",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from random import seed\n",
    "\n",
    "#reading/processing data\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pyarrow.parquet as pq\n",
    "\n",
    "#splitting the dataset\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "#scaling/vectorization\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
    "\n",
    "\n",
    "# models\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.naive_bayes import ComplementNB\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "#metrics\n",
    "from sklearn.metrics import accuracy_score, classification_report, ConfusionMatrixDisplay\n",
    "from sklearn.metrics import confusion_matrix, RocCurveDisplay"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "041448da",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 3.2.2 Load the data <a name=\"3.2.2\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "23d6a53a",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "data = pq.read_table(\"../data/edited/fashion.parquet\")\n",
    "fashion = data.to_pandas()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2c4d67c6",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>review</th>\n",
       "      <th>neg_sentiment</th>\n",
       "      <th>stars</th>\n",
       "      <th>review_length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>exactly need</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>agree review opening small bent hook expensiv...</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>love going order pack work including losing ea...</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>tiny open</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>okay</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              review  neg_sentiment  stars  \\\n",
       "0                                       exactly need              0      5   \n",
       "1   agree review opening small bent hook expensiv...              1      2   \n",
       "2  love going order pack work including losing ea...              0      4   \n",
       "3                                          tiny open              1      2   \n",
       "4                                               okay              1      3   \n",
       "\n",
       "   review_length  \n",
       "0              4  \n",
       "1             49  \n",
       "2             50  \n",
       "3              4  \n",
       "4              1  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fashion.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f498bccd",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>neg_sentiment</th>\n",
       "      <th>stars</th>\n",
       "      <th>review_length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>873352.000000</td>\n",
       "      <td>873352.000000</td>\n",
       "      <td>873352.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.304939</td>\n",
       "      <td>3.904786</td>\n",
       "      <td>29.131591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.460382</td>\n",
       "      <td>1.419361</td>\n",
       "      <td>39.372047</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>7.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>17.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>36.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>2196.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       neg_sentiment          stars  review_length\n",
       "count  873352.000000  873352.000000  873352.000000\n",
       "mean        0.304939       3.904786      29.131591\n",
       "std         0.460382       1.419361      39.372047\n",
       "min         0.000000       1.000000       1.000000\n",
       "25%         0.000000       3.000000       7.000000\n",
       "50%         0.000000       5.000000      17.000000\n",
       "75%         1.000000       5.000000      36.000000\n",
       "max         1.000000       5.000000    2196.000000"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fashion.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "9e57eb28",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(873352, 4)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fashion.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b279c482",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 3.3 Examine Class Split <a name=\"3.3\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b10a27b2",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "def plot_sentiment_split(df_col:pd.Series):\n",
    "    \"\"\"\n",
    "    Plot the split between our positive and negative classes\n",
    "    \"\"\"\n",
    "    df_col.value_counts().plot(kind=\"pie\")\n",
    "    plt.legend(labels=(\"Positive Reviews\",\"Negative Reviews\"))\n",
    "    plt.title(\"Review Sentiment Split\")\n",
    "    plt.show()\n",
    "\n",
    "    positive, negative = df_col.value_counts(normalize=True).values\n",
    "\n",
    "    pos_perc = round(positive * 100, 2)\n",
    "    neg_perc = round(negative * 100, 2)\n",
    "    print(f\"Positive reviews: {pos_perc}%\\n\", f\"Negative reviews: {neg_perc}%\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e96a82d1-111b-49bf-83b4-06086e2e80bc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAD3CAYAAAA0cknjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqp0lEQVR4nO3de3xT9f3H8dcnSS+UXihQRKAa5WIpAnMg4ABFhKmrIIrgBX/iFEVx8z6N++nsFLXq2Bi/idcp6hRFlIlEQVGnojhFAaFYuRm5o9zSlkJLmu/vj3OKEWmblCQnzfk+H48+msvJOe8m+fR7rt+vKKXQNC25OawOoGla7OlC1zQb0IWuaTagC13TbEAXuqbZgC50TbMBXegxJiKPichdVueIFhEpFZEhVueINhFRItLFvJ1UnxnoQj9IRHwisk9EKkVkm4jMEJHMI52vUuoapdS90chYHxG5UkTKRKRCRLaLyJsikhWF+c4Qkcmhjymleiil/nOk825Clv+IyIRGponK+xD6mYnIEBHZ1NTciUIX+k+NUEplAr8ATgLusDZO40TkNOB+4GKlVBbQHXjZ2lTxp9+HRiil9I9xdqAPGBZy/yHAG3J/APAJsAdYDgwxH78QWHLIvG4C5pq3ZwCTQ547B1hmzucToJf5+G+BN0KmWwO8EnJ/I/CLw+S+Ffh3A39XGvAXYAOwHXgMaGE+NwTYBNwCfA9sBX5rPnc1cACoASrrsoW+T0Ax8ArwL6ACWAF0w/gH+b2Z+dchWXKAf5rL2QxMBpzmc5cDi8ysu4FvgbPN5+4DaoH9ZpZ/NOF9mGH+7e+YWT8Ajg15XgFdQj8zoCWwDwiay60EOlj9XW3S99vqAInyc8gXuJP5pf27eb8jsBP4DcZa0HDzfh6QYX5xuobM63PgotAvjXn7JLMA+gNOYLy53DTgeIzidwAdgO+ATebrjje//I7D5B5sfhn/DAwE0g55/m/AXKA1kAW8ATxgPjcECAD3ACnm31cF5B6avZ73qdgsvjMBF/CcWaD/a87vKuDbkNfOAR43C6gd8Bkw0Xzucox/LFeZ7821wBZAzOf/A0xo4PNr7H2YYX5Op5rv99+BRSHP/6zQQ96jTVZ/P4/4+211gET5Mb/AleaXQQHvAq3M524Hnj9k+gXAePP2v4A/mbe7mvPIOMyX5lHg3kPm8w1wmnl7I/BL4CLgCbMQCjBa+7kNZD/bLOA95t/wV7NYBNgLdA6Z9pS64jO/xPsAV8jz3wMDDs1+yPsUWujvhDw3wlx+XSudZb6XrYCjgGrMtQnz+YuB983blwNrQ57LMF/b3rzfYKE39D6E/C0vhUybibGWkG/eT+pCd6GFGqWUWmhu770ItMX40hwLjBGRESHTpgDvm7dfBKZgtIyXYKxCVh1m/scC40Xk9yGPpWK04GCsTg4Bupi39wCnYRTnB/WFVkq9BbwlIg7gdIzV6W8wWtAM4AsRqZtcMP4J1NmplAqE3K/CKIJwbQ+5vQ/YoZSqDbmPOb8OGO/Z1pAsDox/bnW2hfxNVeZ0YWdp4H143JxkY8i0lSKyy8y18dB5JRtd6IehlPpARGZgbC+OwvgiPK+Uuqqel7wD5InILzBaqZvqmW4jcJ9S6r56nv8Ao1U8DmPH0h5gHEah/yOM3EHgXRF5DzgReBKj2HoopTY39vrDzbIJr6nPRowWve0h/1iinuUw70Od/Lob5hGV1hibB1FZbiLTe93rNxUYLiK9MVbNR4jImSLiFJF087BLJwCl1AGM1uNhjC/PO/XM80ngGhHpL4aWIlIUcgjoA4yWqIVSahPwEXAW0AZYergZisi5InKRiOSa8+yHsRbwqfmFfxL4m4i0M6fvKCJnhvkebMfYP3DElFJbgbeBKSKSLSIOEelsrj0dcZaG3oeQyX4jIoNEJBW4F+M9aqw13w60EZGcMHMmJF3o9VBK/YCxc+lP5pfhXOCPwA8YrdMf+On79yIwDGNP+WFbLKXUEoydTf/A2Lm2FmPbtO751Rjblh+Z98uB9cDHIavDh9ptznMNUI7xT+lhpdQL5vO3m8v5VETKgYXACWG+Df8ECkVkj4j8O8zXNOQyjE2VVWbu2cDRYb7278AFIrJbRKYd5vnG3gcwPqO7gV1AH+DSxhaqlCoDZgLrzfehQ2OvSUR1ezQ1LamZm2KblFJ3Wp3FCrpF1zQb0IWuaTagV901zQZ0i65pNqALXdNsQBe6ptmALnRNswFd6JpmA7rQNc0G9EUtSeCLL75o53K5nsK4gEP/805uQWBlIBCY0KdPn+/DfZEu9CTgcrmeat++ffe8vLzdDodDnxiRxILBoPzwww+F27ZtewoYGe7r9H//5HBiXl5euS7y5OdwOFReXp6fn15+2/jrYpRHiy+HLnL7MD/riGpXF7qm2YDeRk9Cbo+3TzTn5ysp+qKxaZxOZ5+uXbvuq62tlS5duuybNWuWLysrKxj2Mny+lGuuuSZ//vz56z/55JMWGzduTL3wwgv9AC+88EJOaWlpi/vvv39bY/NpyLRp09rcfffdnY466qgD1dXVcvnll/9w9913h71DK9RJJ51UsHTp0rIjyRNPukXXoiItLS1YVla2as2aNaUpKSlqypQpeZG83u12H5g/f/56gCVLlmR4vd6DPbqMGzfOf6RFXmfEiBG7y8rKVi1evLhs6tSpR69duzalKfNpTkUOutC1GBg0aFDl2rVr07Zv3+4cNmxY527duhX27t274L///W8LAK/Xm1lQUFBYUFBQ2L1798Ldu3c7vvnmm9SuXbv22L9/vzzwwAMd3njjjdyCgoLCJ598MnfatGltLrvssmN27tzp7NChQ8/aWqOznfLyckf79u17VVdXS2lpadrgwYO79ujRo3ufPn1OWLp0aXpDGdu3b197zDHHVG/cuDEFYPr06a179uzZvaCgoPCSSy45NhAI8NBDD+VNnDixU91r6nIAZGRknFT3+F133XXUiSee2L1bt26FN910U4e6xyZPntwO4Morr8wfMGBAN4C5c+dmjRw58rhAIMDo0aPdXbt27dGtW7fCP//5z+2i+iEcQhe6FlUHDhxgwYIF2T179tx32223dejdu3fV6tWrV917772bx48ffxzAlClT2k+bNu27srKyVZ9++mlZZmbmwVX89PR0dccdd2ypa3mvuuqq3XXPtWnTprZ79+5Vb775ZhbAyy+/nHPaaaf509LS1IQJE46dPn36htLS0q8ffvjhTddee+0xDeVcs2ZNanV1taN///77vvzyy/TZs2e3XrJkSVlZWdkqh8OhHnvssTaXXnrp7rfeeqtV3Wtmz57dety4cbtC5/Paa69lr127Nv2rr776+uuvv161bNmyjLfeeitzyJAhlR9//HEmwLJlyzL27t3rrK6ulg8++CBz8ODBFYsXL87YunVrypo1a0pXr1696rrrrtsZlQ+gHrrQtaiorq52FBQUFPbs2bOwU6dONTfccMOOzz77LOvKK6/cCTBy5MiKPXv2uHbt2uUYMGBA5a233po/efLkdjt27HCmpIS/9jxmzJjdM2fOzAWYNWtW64suumi33+93LF26NHPMmDGdCwoKCidNmnTs999/f9iZvvHGG7ndunUrLCwsPHHChAnfZ2RkqPnz52etXLkyo3fv3t0LCgoKFy1alL1+/fq0Dh06BPLz86vffffdltu2bXOuW7cuffjw4ZWh85s/f372hx9+mF1YWFjYo0ePwnXr1qWXlZWlDxo0qGrFihUtd+3a5UhLS1N9+/at/OijjzIWL16cNXTo0MqCgoLqjRs3po0fPz5/9uzZ2bm5ufX1CRgVemecFhV12+jhTHv//fdvGzVqlP/111/PGTx4cIHX612TkZER1o67iy++eM+9997bcfv27c6VK1dmjBgxory8vNyRlZUVCGf5I0aM2P3cc89t+PDDDzNGjBjRbezYsXuUUjJmzJidjzzyyM+6xB4zZsyumTNn5hYUFOw/++yzdzscP20blVLceOONW//whz/sOPS1+fn51dOnT2/br1+/yt69e+9buHBh1nfffZd20kkn7Xc4HKxcuXLVnDlzsh977LG8l19+ufUrr7ziC+c9aArdomsx079//4pnnnmmDcC8efOycnNzA61btw6Wlpam9evXb9999923rVevXntXrlz5k+3p7Ozs2srKysN+N3NycoK9evXaO3HixGPOOOMMv8vlonXr1sFOnTrVPP3007kAwWCQxYsXt2go26mnnlp1/vnn73zwwQePOuuss8rnzZuXu3nzZhfA9u3bnatXr04FGDdu3J4FCxa0euWVV3622g5w9tlnlz///PNt/X6/A+Dbb79NqZvPKaecUvnII48cNWTIkIphw4ZVPPvss3mFhYVVDoeDrVu3umpra7n88sv3PPDAA5tXrFiREfk7HD7doiehcA6HxcODDz64Zdy4ce5u3boVtmjRIjhjxoxvAR566KF2n3zySbaIqBNOOGHfBRdc4N+wYcPBVe2zzz674i9/+cvRBQUFhbfccsvWQ+c7duzY3VdcccXx8+bN+6busZkzZ66/6qqrjn3wwQePDgQCct555+065ZRT9h362lB33333tr59+xZOnjx565133rn5jDPO6BYMBklJSVHTpk3b0K1bt5q8vLzaLl267F+zZk2L008//Wej75x//vnlpaWl6SeffHIBQEZGRvCFF174tmPHjoHTTjutYtq0ae2HDh26Nzs7O5iWlqYGDhxYCcbhxCuvvNIdDAYF4J577onp0My6z7gksHz5cl/v3r1/tuqoJa/ly5e37d27tzvc6fWqu6bZgC50TbMBXeiaZgN6Z5xNBIJBR/WBYFp1IJheEwimBoLBlNqgcpk/zqDCGVTKgTGs8k923Agoh0NqnQ6pdYoEnA4JOB1S63LIgVSXozo9xVmd6nLUOET0Dp8EpQs9yQSDSvbWBDKqampb7j9Qm1EdCKYfqA2m1QbVkX3WjZzOIYDL6ahJcTr2p7kc+zNSnXszUl1701Mc1SHjoWsW0YXezLk93vYvjemYuWFnVct9B2pb1gSC6QoV98pSwIHaYOqB2mBqVQ3Zu80DUQ6R2vQUZ1WLFOfezHRXRVaaq8Lh0C1/vOlCb2bcHm8qMAg4E2Ps9F4V1UH27Ks5OE2vp46N6jK/mvBdo9P0zs/lf666jlv/NBmAZx/7P6qq9nLtzR5nVU0gq6omkLVzb3V7EQlmpDgrM9Nd/uz0lPIWqc79kWTxeDztS0pKDl7JFq3LRUMvs83Pz6+eNWvWt23bto34tNQbb7yxw5AhQypGjRpVcaSZoknvjGsG3B5vitvjHeH2eGcCO4F3gduAXtYm+1FqWhrvzn+D3bsavjZDKeXYWxPI3l6+P3/N9xU9vt5a3nPz7qqOVTWBBs9kqzNt2rSfjKcerctFQy+zbdWqVeDhhx+O6DLbOlOnTt2SaEUOutAjIiJnicg3IrJWRDyxXJbb4xW3xzvY7fE+BmwF5gIXAZmxXG5TOZ0uLrhkPP96cvrPntu1cwc3X30ZlxQN5ZKioSz9/NODj19x4bmppw7o0/7S8VcUduiU/8sVazd0qA7UpgwbNqxzjx49unfp0qXHX/7yl7YAkyZN6lh38czIkSOPgx8vFz3nnHOOf+mllw5ewz569Gj3M888kxsIBJg4cWKnustIH3744baN/S0DBgzYu3nz5lSAw13+2tDlsnXLBfjoo48yTj755BN69OjRfdCgQV2/++67lM2bN7t69OjRHWDx4sUtRKTPmjVrUgHy8/NPrKiocDz99NO5Xbt27XHCCScU9u3b94Qj+FgO0oUeJhFxAo8AZwOFwMUiUhjt5bg93lZuj/cWYC3wITARaBPt5cTCheMn8Oa/X6Gi3P+Txx+628OlE67lRe97THniWf582w0APPa3B+n3q8HMeXcxw38zkq2bN8muqgNHf7OtotcD//in8+PPl25ZunTpqscff/yobdu2OadPn765ruWdO3fut6HLGDt27K5Zs2blAuzfv18+/vjj7DFjxuyZOnVq25ycnNqVK1d+vXz58q+fffbZvLKystT6/oZAIMD777+fNWrUqD0Ah7v8taHLZevmU11dLddff/0xr7/++rrS0tKvx48fv+PWW2/t2LFjx0B1dbVj165djvfffz+zR48eVQsXLsxcvXp1aps2bQJZWVnBkpKSo99+++3V33zzzar58+evjcZno7fRw9cPWKuUWg8gIi8B5wJhXbHVGLfHWwBcD1wGtIzGPOMtMyubc0ZfxItPP0F6+o/XqXy66APWrzl4WjqVFRVU7a1k2eef8tcn/wXAwNOHkZ3T6uA0Tz/xaNZ78+dlAWrbtm2sWFnaon379j+5RDTUBRdc4L/99tvz9+3bJ6+++mpOv379KjIzM9XChQuzy8rKMubOnZsLUFFR4Vy1alV6QUFBTejr69YUtm/fntK5c+f9o0aNKg+9/LVuupqaGoEfL5cdMWJExaxZs1pPmjTph9D5ffXVV2lr1qxpMXTo0G5gXGiTl5d3AKBv376VCxcuzFy0aFHWbbfdtnX+/Pk5SikGDBhQWff8uHHj3KNHj949bty43USBLvTwdQQ2htzfBPQ/0pm6Pd5BwJ3ArzGOUjVrl155LRf95jTOHTvu4GMqGOT5198hLb3BTl8O+nzxIj5d9B+ee/1tWrTIkCvHnMPG3fu6btpdVe/5/BkZGWrAgAEVr732WvbLL7+ce9FFF+0CUErJlClTNowePbq8oWXWrSlUVFQ4hgwZ0rWkpKTdpEmTdtR3+evhLpcNfV4pJV26dNm3bNmyn+1DGDx4cMWHH36YtWnTptRx48btmTJlSntAnXPOOX6AF198ccN7773Xcu7cuTl9+vQp/OKLL1a1b9/+iK5X16vuFnF7vCe7Pd4FwEcYe9CbfZED5OTm8utzRjHnpecPPnbKqaczc8YTB++Xla4A4Bd9+/P2vDkAfPLBe5T79wBQWV5Odk4rWrTI4Nu1q/lq6RIUOHbtrWnnSk11rN+2J78mEPxZxxIXXnjh7hkzZrT9/PPPs+oKe/jw4f5HH300r7q6WsBoacvLy+v93mdlZQWnTZu2Yfr06UdlZWXVe/nr4S6XDdWrV6/9u3btci1cuLAlGKvyS5YsSQcYNmxY5auvvtr6uOOOq3Y6nbRq1Srw/vvv59R1alFaWpo2dOjQvVOnTt2Sm5sbWL9+fb2bGuHSLXr4NgP5Ifc7mY9FxO3x9gLuJYJRNiIVzuGwWLrs6t/x0oynDt6//Z4Huf9//8AFwwdSW1vLL/ufwl0P/I2JN92O53cTmPfqy/Tu04+27Y6iZctMBg45g1f+9TSjTu+P+/gu9Dqp78F5jb5kPMMG92/X/cReeU88/dz20OWed9555RMnTjxu+PDhe9LT0xXATTfdtMPn86X17Nmzu1JKWrdufeDNN99c11D+gQMH7isoKNj3xBNPtG7o8tfDXS5bJz09Xb300kvrrr/++mMqKiqctbW1cu21127v27fv/hNOOKFGKSWDBw+uAOO69a1bt6bm5eXVmpk7+Xy+NKWUDBo0qHzAgAENXm4bDn2ZaphExAWsBs7AKPDPgUuUUqXhvN7t8bYDHsLYBo9q6/3kyKM56pjjoznLuKiprsbhdOJyuVj+xWfc98dbmLXgo4jm4XRIbV5m2ua8rLQf7HQGXqSXqeoWPUxKqYCI/A5YADiBp8MpcrfH6wCuBSYDrWIaspnZumUTf7j2t6hgkJSUVP704N8jnkdtUDm3le8/Zk/VgbYdclt8l5nm+lnnEJou9Igopd4E3gx3erfH2x+YDvwyZqGasWOP68ys+R9GZV77A7UZ63+o7N4qI/WHDjnpm11OR0w7W2xudKHHgNvjzQAeBK4jDjvZFAqlFHZada3PnqqavIr9B3KPzmmxoXXL1Kgcmko0ZvdTYY+CA3qve9S5Pd4BwDLgd8RpT/p3ew4QqCpH728x1AaVa9PuquO/27nXXRtUSfUdN4dNzgFWRvI6vTMuStwerxPjePhdGNvwcZOd5uD3/XM5tlUKkhxH6aLGKQSy0xw7UpxSbXWWKAkCKwOBwIQ+ffqEPW6cLvQocHu8nYCZGFeVaYmnFrgf+LOvpMiW2+660I+Q2+MdCLwKHGV1Fq1R7wAX+kqKknLbvSFJtf0Sb26P92rgPXSRNxfDgc/cHm/UL0ZKdLpFbwK3x5sCTAOusTqL1iQVwKW+kqK5VgeJF92iR8jt8eZgrALqIm++soB/uz3emPYpkEh0ix4Bt8d7FMaZcb2tzqJFzV99JUW3WB0i1nShh8nt8R6L0ZJ3tTqLFnVPA1cn8x55XehhcHu83TGKvKPVWbSYeRW4xFdSVNPolM2QLvRGuD3e3sBCoNG+xrRmbwEwyldSFFHPtM2BLvQGuD3ersAioJ3VWbS4eQM431dSFLA6SDTpve71cHu8+RgtuS5yexkBPOP2eJPqXGJd6Ifh9njzMLbJj7E6i2aJSzHOk0gautAP4fZ4szG21aLSn7bWbP3O7fHeY3WIaNHb6CHM3mDeAH5jdRYtYfzWV1I0w+oQR0q36D91P7rItZ96zOxjoFnTLbrJ7fFeArxgdQ4tIW0F+vpKirZYHaSpdKEDbo+3L8bwR2EN9KfZ0mfAqb6SombZgYXtV93dHm9rYA66yLWG9QMetTpEU9m+0IEnMQZj0LTG/Nbt8Y61OkRT2HrV3e3xXgk81eiEmvaj3UAvX0nRJquDRMK2hW6e3rqUZjpyqWap94BhvpKiZlM8tlx1d3u8Low97LrItaYYCjSra9htWeiABzjZ6hBas3af2+PtYXWIcNlu1d3t8R4PlALhDdatafX7GBjcHFbh7dii/wNd5Fp0DAQmWB0iHLZq0d0e72hgttU5tKSyC+jmKynaaXWQhtimRXd7vJnAVKtzaEmnNcaQ2AnNNoUO3IE+MUaLjavNLscSli0K3eym+Qarc2hJy0GCt+q2KHTgj+hj5lpsneP2ePtbHaI+SV/oZt9vE63OodnCfVYHqE/SFzpwN5BmdQjNFs5we7xDrA5xOEld6G6PtzMw3uocmq0k5LZ6Uhc6cDPgsjqEZisD3R7vr6wOcaikLXS3x9sGuNzqHJotJdwRnrALXUSOC+exBDIRyLA6hGZL57s93oQ6ZyOSFv3VwzyWkKeTuj1eJ3Ct1Tk023IB11kdIlSjhS4iBSIyGsgRkfNDfi4ncS8OORd9FpxmravdHm/C9EMYzo6qE4BzgFYY41LVqQCuikGmaEjUXJp9tAbGAs9aHQQiuHpNRE5RSi2OcZ4j5vZ42wFbAKfVWTTbe8dXUvRrq0NAZIee1orIHwF36OuUUldEO9QRughd5FpiGOr2eNv7Soq2WR0kkkJ/HfgIYyjh2tjEiYpLrQ6gaSYnRsMz1eIcERV6hlLq9pgliQKzZ1fdF5yWSMaRAIUeyeG1eSKS6AMQXmJ1AE07RF+zAbJUJIV+A0ax7xeRchGpEJHyWAVronOsDqBphzHS6gBhF7pSKksp5VBKpSulss372bEMFwm3x5sH9LE6h6YdxplWB4jkFFgRkUtF5C7zfr6I9ItdtIidCYjVITTtME51e7yWno4dyar7dOAUftwOrgQeiXqipjvL6gCaVo80YIiVASIp9P5KqeuA/QBKqd1AakxSRcjt8TpIgNUjTWuApd/PSAr9gIg4AQUgInlAMCapItcLaGt1CE1rwHArFx5JoU8D5gDtROQ+YBFwf0xSRS5hO+XTNFOB2+PNsWrhYZ8wo5R6QUS+AM7A2Ok1Sin1dcySRSaRdgpq2uEI0Bd414qFR9rDzHaM02A/AVqIyC+jH6lJdKFrzYFlZ22G3aKLyL0YXTOtw9xON38PjX6s8JlDLRVamUHTwpT4hY5xbW1npVRNrMI0UR+SuO87LalYVuiRFMhKjM4nEk1PqwNoWpjyzU5L4y6SFv0BYKmIrASq6x5USll9Hq/lFwxoWgS6AHEfYjmSQn8WeBBYQeIcPwfjjdO05qIz8N94LzSSQq9SSk2LWZKm04WuNSedrVhoJIX+kYg8AMzlp6vuX0Y9VZjMU1/dVi1f05og4Qv9JPP3gJDHrD68lk+CnG+vaWGyZA00kjPjTo9lkCbqaHUATYtQvhULbbTQReRSpdS/ROTmwz2vlPpr9GOFrbWFy9a0prDkOxvOcfSW5u+sw/xkxihXuCw5JqlpRyDT7fGmxHuhjbboSqnHzZsLlVIfhz4nIgNjkip8ukXXmqNc4Pt4LjCSM+P+L8zH4kkXutYcxf17G842+inAr4C8Q7bTs7F+RJRci5evaU2ReIWOcfgq05w2K+TxcuCCWISKQKKO5qppDWnZ+CTRFc42+gfAByIyQyn1XRwyaVqyi/uacCQnzKSJyBP8fJBFK0+Y0d07a81RQhf6K8BjwFMkziCLutBjZHnahBVZ7NMnJMVAFWkBo7Om+Imk0ANKqUdjlqRpdKHHwC9k7eocqdLX+cdIJvvjfvVnJIfX3hCRSSJytIi0rvuJWTLNMje5Zm+1OkOSOxDvBUbSoo83f/8h5DEFHB+9OBHba+Gyk5IQDA5yrOxmdY4kl7iFrpQ6LpZBmmiP1QGSzVmOz5c7JXhS41NqRyDuhR7JIIsZInKnuecdEekqIlYPU7zb4uUnnd+75lRZncEG4r4mGsk2+jNADcZZcgCbgclRTxQZXehRlE71vu6yoZfVOWxgQ7wXGEmhd1ZKPYS52qGUqsL6vd57LF5+UhnnfHeZyE/OftSibzfF/op4LzSSQq8RkRb8OMhiZ0K6lLLILouXn1QmuN6MZOes1jSWnF0ayQd7NzAfyBeRF4CBGCO3WGmzxctPGq3x72zPLr0TLvbivtoOke11f0dEvsToM06AG5RSO2KWLDzfYaxhWL0J0exNcs0tFeFUq3PYgCUteiR73QcC+5VSXowRW/4oIsfGKlg4fCVFNcAWKzMkiwud/9G99cRHYhc68ChQJSK9gZsxBlt8LiapIrPO6gDNXWfZ/F2W7OthdQ6bSPhCDyilFHAu8IhS6hFIiD20q60O0Nzd7JrtszqDjST8zrgKEbkDuBQ4VUQcQNw7uTsMXehHaLhjidvqDDaS8C36hRiH065USm0DOgEPxyRVZJZaHaA5G+hYWZoqtZbua7GRKor9ce0Usk4ke923AX8Nub+BkG10EVmslDoluvHCsgS9573JbnS9GveRPW0s7oMr1omkRW+MJf23+UqK9qB3yDWJk9pAH1mtd8LFzwdWLTiaha6iOK9IfW7hsput85yLljpE6cNq8ZMUhW6lJVYHaI4mOefG/XJJG6sBPrVq4dEsdCu3kS17A5urTKrKj5Ot+pTX+PmMYv9+qxYezUL/nyjOK1KfYfQzr4XpCudbX4nQwuocNvKhlQuP5BTYChEpP+Rno4jMEZHjlVIrYxm0Ib6SogDwrlXLb44uc70T90EEbM6y7XOI7ISZqcAm4EWM1fSLgM7Al8DTwJAoZ4vUAuA8izM0Cx3YsbUN5b2tzmEjAeATKwNEsuo+Uin1uFKqQilVrpR6AjhTKfUyiTEG2gKrAzQX17vmrBZJmh2xzcGXFPsrrQwQyYddJSJjRcRh/owF6nYuWHloDQBfSZEPfTpsWM51fny01RlsxtLVdois0Mdh7HD7HmOYif8BLjV7nfldDLI1xVyrAyS6XrJuTQup0d05x9fLVgeI5BTY9cCIep5eFJ04R2wmcKvVIRLZTa7ZW4CuVuewkRUU+7+wOkQke927ici7IrLSvN9LRO6MXbTI+UqKvgTKrM6RuJQa7FihW/P4etbqABDZqvuTwB382AvsVxh73hPNi1YHSFS/dnyx3CVBvX0ePwHgX1aHgMgKPUMp9dkhjwWiGSZKdKHX43rXa5bu+bWhtyj2x3fY1HpEUug7zC6e67p7vgBIuMH4fCVF67DwcsBElUbN/h7i0yOkxtcMqwPUiaTQrwMeBwpEZDNwI3BNLEJFwWNWB0g0FznfXyZCjtU5bGQH8IbVIepEUuibMYZlug94CXiHH0dYTTQzgR+sDpFIrnJ59Qky8TWTYn/CXB0YyYf/OsbhtQMYXSxXkqDDFvtKiqqBJ6zOkShyKd/VkR36SrX4mmF1gFCRnOveSSl1VsySRN904DYSowNLS13jemOlHpwhrr6k2P+l1SFCRdKifyIizWZnjq+kaAvwqtU5EsHFzvdbW53BZu63OsChIin0QcAXIvKNiHwlIitE5KtYBYuSROil1lLHyZYN2VJ1otU5bGQF8JrVIQ4lxpgMYUxYz/BLSilL+qkOl9vjnQOMsjqHVaal/N8HI52LT7M6h41cSLF/ltUhDhXJue4JXdANuAsYSfL0jxeRsxyfH2N1BhtZBcy2OsThJP2X31dStBJIuP+w8TDAUboqVQLHWZ3DRoop9getDnE4SV/opmKg1uoQ8XaT61Wrh7W2k/9S7H/F6hD1sUWh+0qKviHBjmvGmpPawMnyTXerc9jIbVYHaIgtCt10B7DH6hDxMtLxyTKHqDyrc9jEGxT7Le3ltTG2KXRfSdEPwP9anSNernO9XmN1BpsIAB6rQzTGNoVuegyj19qk1pJ9lZ1lyy+szmET91HsX2V1iMbYqtB9JUVBYBIJ0JllLF3uXLBchAyrc9jAl8Bkq0OEw1aFDuArKfovSX7By+WuBXoEltirBi6j2J+Ina/8jO0K3XQrSTrUcnt2bW+LXw/OEHt3UuwvtTpEuGxZ6L6SokrgUpLw2PrvXHPKRHBanSPJfQT81eoQkbBloQP4Soo+JQGvMjpS5zkXHWV1hiS3F7g8Uc+Aq49tC910D8ZIrEmhUHzrWkp1gdU5ktytFPvXWx0iUrYudHMU1nEkyZDLN7te2Wh1hiS3gGJ/s+yP0NaFDuArKVqL0fddMz/kptQQx3I9AkvsbAGusDpEU9m+0AF8JUX/pplvr5/h+PIrlwQ7Wp0jSVUARRT7t1gdpKl0of/oTzTjQRpvcL2WFJsfCSgAjKXYv8zqIEdCF7rJPGtuHEZXQM1KKgeqe8q3vazOkaSupdg/3+oQR0oXegjz+HoR0Kx2al3o/M9SPThDTNxHsf8pq0NEgy70Q/hKijYCw2lGA0Bc7ZwnVmdIQs9T7E+o0YKPhC70wzA7qjiLZnDYLYfKPZ3kBz04Q3S9D1xpdYho0oVeD3Os9ZHAfquzNGSia94KEVKtzpFESoHzE2k4pWjQhd4AX0nRB8BoErjYL3G+q7fNo6cUOJNi/x6rg0SbLvRG+EqK3sTYQZdwY4sfK9s25bC32Yyek+A+AgZT7N9sdZBY0IUeBl9J0XvAMGC31VlC3eh6dZ0Iekfckfs38GuK/Qn1+UaTLvQwmR1WnAZsszpLnd84PutkdYYk8DhwAcX+hN08iwZd6BHwlRStAAYDa6zOcrKUlaXJgc5W52jm7qbYfw3F/qTrl+BQutAjZF4E0w9428ocN7lmb7dy+c1cLXA1xf57rA4SL7rQm8BXUrQH+A3wNyuW7yBY29/xtR6coWn2YRw+e9LqIPGkC72JfCVFtb6SopuB32J0FBg3RY5PlzpFtYvnMpPEOuBUiv3N9uKlptKFfoR8JUUzMLbb18Zrmb93zdGDM0RuJnASxf4lVgexQtjjo2sNc3u8mcA0jBY+ZjLYv7c07QpEaBnL5SSRKuD3FPuftjqIlXSLHiW+kqJKX0nRFcBYYni8/TLn28t1kYftM6CP3YscdKFHna+k6BWgF7AwFvP/rWt+Wizmm2RqgD8Cv6LYX2Z1mESgV91jyO3xXgJMAdpHY37t2P3Df9Oua637bW/Ql8B4iv0rrQ6SSHSLHkO+kqIXgQLgH8AR9wN+nevfX+sir9f3wO+B/uEUuYg8LSLfi4gt/iHoFj1O3B5vH+ARoH9T57Ey7YqvM2W/Pn7+UxUYa01TKPaHfeGRiJyKcaHSc0qpE2MVLlHoQo8zt8d7HnAv0COS1xXIhvXz0zzHxyZVs1SDcZ76vRT7m9QbkIi4gXl2KHS96h5nvpKiORg76y4Dwh7x4yY9OEMdBbwIdKfYf31Ti9xudItuIbfHm4LRZdEtQJf6p1RqTdplm1KkNj9O0RLVAsATra6X7dSi60JPAG6P14HRucWNwNBDnz/NsWzFs6kP2bWDiQqMs9qeotj/eTRnrAtds4zb4+2FUfAXA+kAc1L/9NFJjrWDrcxlgcXAU8DLFPv3xmIButA1y7k93lbAhS4C41anXXaiQ8i1OlMc7ASex2i9S2O5IBGZCQwB2gLbgbuVUv+M5TKtpAu9OSjOOQ7j1NqxwC8tThNtAeA/GK33HIr9+oKdGNCF3twU53QBxmD0Ydcfmt157wr4CngPeBf4kGJ/hbWRkp8u9OasOMcF9AZ+BQw0fxKxH7nVGIX9HvA+xf4dFuexHV3oyaY4J58fi/5XQCHmTr04UMAG4BugDPgCeI9i/6Y4LV+rhy50OyjOaQN0NH86hdwOfay1OXUtxnZzIOR26O8DGOeVb8AYjLLuZx2wmmJ/VVz+Ji0iutA1Q3GOUOzXX4YkpQtd02xAn+uuaTagC13TbEAXuqbZgC50TbMBXeiaZgO60DXNBnSha5oN6ELXNBvQha5pNqALXdNsQBe6ptmALnRNswFd6JpmA7rQNc0GdKFrmg3oQtc0G9CFrmk2oAtd02xAF7qm2YAudE2zAV3ommYDutA1zQZ0oWuaDehC1zQb0IWuaTagC13TbEAXuqbZwP8Dd3BjYpiVyDMAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Positive reviews: 69.51%\n",
      " Negative reviews: 30.49%\n"
     ]
    }
   ],
   "source": [
    "plot_sentiment_split(fashion[\"neg_sentiment\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96e4329c-a5d8-4c47-93c0-c9b11a2440f7",
   "metadata": {},
   "source": [
    "We have more than double the positive reviews vs. negative reviews. The class we are trying to predict is negative reviews which are coded as \"1\"."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2a20099-9c59-4751-a268-ac411ef8dd06",
   "metadata": {},
   "source": [
    "## 3.4 Pre-Processing <a name=\"3.4\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a53ec675",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 3.4.1 Set Random Seed for Reproducability <a name=\"3.4.1\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c521e2ea",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "seed(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6196a8de",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 3.4.2 Train/Test Split <a name=\"3.4.2\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d8332659",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "#split into train and test with a 90% / 10% ratio\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(fashion[\"review\"].values, fashion[\"neg_sentiment\"], test_size = .1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1e0baf4-2d20-4553-b1f8-af5f7bc8522f",
   "metadata": {},
   "source": [
    "### 3.4.3 Examine Class Split for Train/Test Data <a name=\"3.4.3\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "cc3cf2f1",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAD3CAYAAAA0cknjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqkklEQVR4nO3deXhU1f3H8fd3ZrIQEkKAAALRUbYQBLQgYAFFhLpEEEUUxYoLSkGrUq1OWy2posaF1vKruJbiiiJuyCgqanHDuiFLMKyOLAIKhElCIGEy5/fHvcERIZmBmdyZuef1PHmS2e79ZDLfnLudc0QphaZpyc1hdQBN02JPF7qm2YAudE2zAV3ommYDutA1zQZ0oWuaDehCjzEReUREbrc6R7SISImIDLY6R7SJiBKRTubPSfU3A13o+4mIT0T2iEiliGwVkVkiknmky1VK/U4pdWc0Mh6KiFwlIqUiUiEi20TkDRHJisJyZ4nI1ND7lFLdlVL/PdJlH0aW/4rI+AaeE5X3IfRvJiKDRWTT4eaOF7rQf264UioTOAE4EfiTtXEaJiKnAncDFyulsoBuwAvWpmp8+n1ogFJKfxlXB/qAoSG37wO8Ibf7A58Au4ClwGDz/ouALw5Y1mRgnvnzLGBqyGPnAF+by/kE6GnefwXwesjz1gAvhtzeCJxwkNw3A6/W83ulAQ8AG4BtwCNAE/OxwcAm4CbgB2ALcIX52DXAPqAGqKzLFvo+AUXAi8AzQAWwHOiC8Q/yBzPzb0KyZAP/NtezGZgKOM3HLgc+MrOWAd8CZ5mP3QXUAnvNLP86jPdhlvm7v2NmXQQcE/K4AjqF/s2ApsAeIGiutxJoZ/Vn9bA+31YHiJevAz7AHcwP7T/N2+2BHcDZGFtBw8zbuUCG+cHpHLKsz4ExoR8a8+cTzQLoBziBceZ604DjMIrfAbQDvgM2ma87zvzwOw6Se5D5YfwbMABIO+DxfwDzgBZAFvA6cI/52GAgANwBpJi/XxWQc2D2Q7xPRWbxnQG4gKfMAv2LubyrgW9DXvsK8KhZQK2Bz4AJ5mOXY/xjudp8byYC3wNiPv5fYHw9f7+G3odZ5t/pFPP9/ifwUcjjvyj0kPdok9WfzyP+fFsdIF6+zA9wpflhUMC7QHPzsVuBpw94/lvAOPPnZ4C/mj93NpeRcZAPzcPAnQcsZxVwqvnzRuBXwBjgMbMQ8jFa+3n1ZD/LLOBd5u/wd7NYBNgNdAx57sl1xWd+iPcArpDHfwD6H5j9gPcptNDfCXlsuLn+ulY6y3wvmwNtgGrMrQnz8YuB982fLwfWhjyWYb62rXm73kKv730I+V2eD3luJsZWQp55O6kL3YUWaqRSaqG5v/cc0ArjQ3MMMFpEhoc8NwV43/z5OWAaRst4CcYmZNVBln8MME5Efh9yXypGCw7G5uRgoJP58y7gVIziXHSo0EqpN4E3RcQBnIaxOb0KowXNAL4UkbqnC8Y/gTo7lFKBkNtVGEUQrm0hP+8BtiulakNuYy6vHcZ7tiUkiwPjn1udrSG/U5X5vLCz1PM+PGo+ZWPIcytFZKeZa+OBy0o2utAPQim1SERmYewvjsT4IDytlLr6EC95B8gVkRMwWqnJh3jeRuAupdRdh3h8EUareCzGgaVdwFiMQv9XGLmDwLsi8h5wPPA4RrF1V0ptbuj1B1vkYbzmUDZitOitDvjHEvUsB3kf6uTV/WCeUWmBsXsQlfXGM33U/dAeBIaJSC+MTfPhInKGiDhFJN087dIBQCm1D6P1uB/jw/POIZb5OPA7EeknhqYiUhhyCmgRRkvURCm1CfgQOBNoCSw52AJF5FwRGSMiOeYy+2JsBXxqfuAfB/4hIq3N57cXkTPCfA+2YRwfOGJKqS3A28A0EWkmIg4R6WhuPR1xlvreh5CnnS0iA0UkFbgT4z1qqDXfBrQUkewwc8YlXeiHoJT6EePg0l/ND8O5wJ+BHzFapz/y8/fvOWAoxpHyg7ZYSqkvMA42/Qvj4NpajH3TusdXY+xbfmjeLgfWAx+HbA4fqMxc5hqgHOOf0v1KqWfNx2811/OpiJQDC4GuYb4N/wYKRGSXiLwa5mvqcxnGrspKM/dc4KgwX/tP4AIRKROR6Qd5vKH3AYy/0RRgJ9AbuLShlSqlSoHZwHrzfWjX0GviUd0RTU1Lauau2Cal1G1WZ7GCbtE1zQZ0oWuaDehNd02zAd2ia5oN6ELXNBvQha5pNqALXdNsQBe6ptmALnRNswHdqSUJfPnll61dLtcTGB049D/v5BYEVgQCgfG9e/f+IdwX6UJPAi6X64m2bdt2y83NLXM4HPrCiCQWDAblxx9/LNi6desTwIhwX6f/+yeH43Nzc8t1kSc/h8OhcnNz/fy8+23Dr4tRHq1xOXSR24f5t46odnWha5oN6H30JOT2eHtHc3m+4sIvG3qO0+ns3blz5z21tbXSqVOnPXPmzPFlZWUFw16Hz5fyu9/9Lm/BggXrP/nkkyYbN25Mveiii/wAzz77bHZJSUmTu+++e2tDy6nP9OnTW06ZMqVDmzZt9lVXV8vll1/+45QpU8I+oBXqxBNPzF+yZEnpkeRpTLpF16IiLS0tWFpaunLNmjUlKSkpatq0abmRvN7tdu9bsGDBeoAvvvgiw+v17h/RZezYsf4jLfI6w4cPLystLV25ePHi0gcffPCotWvXphzOchKpyEEXuhYDAwcOrFy7dm3atm3bnEOHDu3YpUuXgl69euX/73//awLg9Xoz8/PzC/Lz8wu6detWUFZW5li1alVq586du+/du1fuueeedq+//npOfn5+weOPP54zffr0lpdddtnRO3bscLZr165Hba0x2E55ebmjbdu2Paurq6WkpCRt0KBBnbt3796td+/eXZcsWZJeX8a2bdvWHn300dUbN25MAZgxY0aLHj16dMvPzy+45JJLjgkEAtx33325EyZM6FD3mrocABkZGSfW3X/77be3Of7447t16dKlYPLkye3q7ps6dWprgKuuuiqvf//+XQDmzZuXNWLEiGMDgQCjRo1yd+7cuXuXLl0K/va3v7WO6h/hALrQtajat28fb731VrMePXrsueWWW9r16tWravXq1SvvvPPOzePGjTsWYNq0aW2nT5/+XWlp6cpPP/20NDMzc/8mfnp6uvrTn/70fV3Le/XVV5fVPdayZcvabt26Vb3xxhtZAC+88EL2qaee6k9LS1Pjx48/ZsaMGRtKSkq+uf/++zdNnDjx6PpyrlmzJrW6utrRr1+/PV999VX63LlzW3zxxRelpaWlKx0Oh3rkkUdaXnrppWVvvvlm87rXzJ07t8XYsWN3hi7n5ZdfbrZ27dr0ZcuWffPNN9+s/PrrrzPefPPNzMGDB1d+/PHHmQBff/11xu7du53V1dWyaNGizEGDBlUsXrw4Y8uWLSlr1qwpWb169cprr712R1T+AIegC12Liurqakd+fn5Bjx49Cjp06FBzww03bP/ss8+yrrrqqh0AI0aMqNi1a5dr586djv79+1fefPPNeVOnTm29fft2Z0pK+FvPo0ePLps9e3YOwJw5c1qMGTOmzO/3O5YsWZI5evTojvn5+QWTJk065ocffjjoQl9//fWcLl26FBQUFBw/fvz4HzIyMtSCBQuyVqxYkdGrV69u+fn5BR999FGz9evXp7Vr1y6Ql5dX/e677zbdunWrc926denDhg2rDF3eggULmn3wwQfNCgoKCrp3716wbt269NLS0vSBAwdWLV++vOnOnTsdaWlpqk+fPpUffvhhxuLFi7OGDBlSmZ+fX71x48a0cePG5c2dO7dZTk7OocYEjAp9ME6Lirp99HCee/fdd28dOXKk/7XXXsseNGhQvtfrXZORkRHWgbuLL75415133tl+27ZtzhUrVmQMHz68vLy83JGVlRUIZ/3Dhw8ve+qppzZ88MEHGcOHD+9y4YUX7lJKyejRo3c89NBDvxgSe/To0Ttnz56dk5+fv/ess84qczh+3jYqpbjxxhu3/PGPf9x+4Gvz8vKqZ8yY0apv376VvXr12rNw4cKs7777Lu3EE0/c63A4WLFixcpXXnml2SOPPJL7wgsvtHjxxRd94bwHh0O36FrM9OvXr+I///lPS4D58+dn5eTkBFq0aBEsKSlJ69u375677rpra8+ePXevWLHiZ/vTzZo1q62srDzoZzM7OzvYs2fP3RMmTDj69NNP97tcLlq0aBHs0KFDzcyZM3MAgsEgixcvblJftlNOOaXq/PPP33Hvvfe2OfPMM8vnz5+fs3nzZhfAtm3bnKtXr04FGDt27K633nqr+YsvvviLzXaAs846q/zpp59u5ff7HQDffvttSt1yTj755MqHHnqozeDBgyuGDh1a8eSTT+YWFBRUORwOtmzZ4qqtreXyyy/fdc8992xevnx5RuTvcPh0i56Ewjkd1hjuvffe78eOHevu0qVLQZMmTYKzZs36FuC+++5r/cknnzQTEdW1a9c9F1xwgX/Dhg37N7XPOuusigceeOCo/Pz8gptuumnLgcu98MILy6688srj5s+fv6ruvtmzZ6+/+uqrj7n33nuPCgQCct555+08+eST9xz42lBTpkzZ2qdPn4KpU6duue222zaffvrpXYLBICkpKWr69OkbunTpUpObm1vbqVOnvWvWrGly2mmn/WL2nfPPP7+8pKQk/aSTTsoHyMjICD777LPftm/fPnDqqadWTJ8+ve2QIUN2N2vWLJiWlqYGDBhQCcbpxKuuusodDAYF4I477ojp1Mx6zLgksHTpUl+vXr1+semoJa+lS5e26tWrlzvc5+tNd02zAV3ommYDutA1zQb0wTibCASDjup9wbTqQDC9JhBMDQSDKbVB5TK/nEGFM6iUA2NaZQiZRVQAh0MCTofUOkUCTvNnl0P2pboc1ekpzupUl6PGIaIP+MQpXehJJhhUsrsmkFFVU9t0777ajOpAMH1fbTCtNqiO7G9dS1p9DwvgcjpqUpyOvWkux96MVOfujFTX7vQUR3XIfOiaRXShJzi3x9v2+dHtMzfsqGq6Z19t05pAMF2hGr2yFLCvNpi6rzaYWlVDszLzRJRDpDY9xVnVJMW5OzPNWZGVnlLhcOiWv7HpQk8wbo83FRgInIExd3rPiuogu/bU7H9OzyeOieo6l43/rsHn9MrL4bdXX8vNf50KwJOP/B9VVbuZ+AePs6omkFVVE8jasZu2IhLMSHFWZqa7/M3SU8qbpDr3RpLF4/G0LS4u3t+TLVrdRUO72ebl5VXPmTPn21atWkV8WeqNN97YbvDgwRUjR46sONJM0aQPxiUAt8eb4vZ4h7s93tnADuBd4Bagp7XJfpKalsa7C16nbGf9fTOUUo7dNYFm28r35q35oaL7N1vKe2wu29O+qiZQ75VsdaZPn/6z+dSj1V00tJtt8+bNA/fff39E3WzrPPjgg9/HW5GDLvSIiMiZIrJKRNaKiCeW63J7vOL2eAe6Pd6HgS3APGAMkBnL9R4up9PFBZeM45nHZ/zisZ07tvOHay7jksIhXFI4hCWff7r//isvOjf1lP6/anvpuCsL2nXI+9XytRvaVQdqU4YOHdqxe/fu3Tp16tT9gQceaAUwadKk9nWdZ0aMGHEs/NRd9Jxzzjnu+eef39+HfdSoUe7//Oc/OYFAgAkTJnSo60Z6//33t2rod+nfv//uzZs3pwIcrPtrfd1l69YL8OGHH2acdNJJXbt3795t4MCBnb/77ruUzZs3u7p3794NYPHixU1EpPeaNWtSAfLy8o6vqKhwzJw5M6dz587du3btWtCnT5+uR/Bn2U8XephExAk8BJwFFAAXi0hBtNfj9nibuz3em4C1wIfA74CW0V5PLFw0bjxvvPoiFeX+n91/3xQPl46fyHPe95j22JP87ZYbAHjkH/fS99eDeOXdxQw7ewRbNm+SnVX7jlq1taLnPf/6t/Pjz5d8v2TJkpWPPvpom61btzpnzJixua7lnTdv3reh67jwwgt3zpkzJwdg79698vHHHzcbPXr0rgcffLBVdnZ27YoVK75ZunTpN08++WRuaWlp6qF+h0AgwPvvv581cuTIXQAH6/5aX3fZuuVUV1fL9ddff/Rrr722rqSk5Jtx48Ztv/nmm9u3b98+UF1d7di5c6fj/fffz+zevXvVwoULM1evXp3asmXLQFZWVrC4uPiot99+e/WqVatWLliwYG00/jZ6Hz18fYG1Sqn1ACLyPHAuEFaPrYa4Pd584HrgMqBpNJbZ2DKzmnHOqDE8N/Mx0tN/6qfy6UeLWL9m/2XpVFZUULW7kq8//5S/P/4MAANOG0qz7Ob7nzPzsYez3lswPwtQW7duZfmKkiZt27b9WRfRUBdccIH/1ltvzduzZ4+89NJL2X379q3IzMxUCxcubFZaWpoxb968HICKigrnypUr0/Pz82tCX1+3pbBt27aUjh077h05cmR5aPfXuufV1NQI/NRddvjw4RVz5sxpMWnSpB9Dl7ds2bK0NWvWNBkyZEgXMDra5Obm7gPo06dP5cKFCzM/+uijrFtuuWXLggULspVS9O/fv7Lu8bFjx7pHjRpVNnbs2DKiQBd6+NoDG0NubwL6HelC3R7vIOAvwG/46Rx2wrr0qomMOftUzr1w7P77VDDI06+9Q1p6vYO+7Pf54o/49KP/8tRrb9OkSYZcNfocNpbt6byprOqQ1/NnZGSo/v37V7z88svNXnjhhZwxY8bsBFBKybRp0zaMGjWqvL511m0pVFRUOAYPHty5uLi49aRJk7YfqvvrwbrLhj6ulJJOnTrt+frrr39xDGHQoEEVH3zwQdamTZtSx44du2vatGltAXXOOef4AZ577rkN7733XtN58+Zl9+7du+DLL79c2bZt2yPqr6433S3i9nhPcnu8bwEfYBxBT/giB8jOyeE354zkleef3n/fyaecxuxZj+2/XVqyHIAT+vTj7fmvAPDJovco9+8CoLK8nGbZzWnSJINv165m2ZIvUODYubumtSs11bF+6668mkDwF43URRddVDZr1qxWn3/+eVZdYQ8bNsz/8MMP51ZXVwsYLW15efkhP/dZWVnB6dOnb5gxY0abrKysQ3Z/PVh32VA9e/bcu3PnTtfChQubgrEp/8UXX6QDDB06tPKll15qceyxx1Y7nU6aN28eeP/997PrBrUoKSlJGzJkyO4HH3zw+5ycnMD69esPuasRLt2ih28zkBdyu4N5X0TcHm9P4E4imGUjUuGcDouly665judnPbH/9q133Mvdf/kjFwwbQG1tLb/qdzK33/MPJky+Fc9145n/0gv06t2XVq3b0LRpJgMGn86Lz8xk5Gn9cB/XiZ4n9tm/rFGXjGPooH6tux3fM/exmU9tC13veeedVz5hwoRjhw0btis9PV0BTJ48ebvP50vr0aNHN6WUtGjRYt8bb7yxrr78AwYM2JOfn7/nsccea1Ff99eDdZetk56erp5//vl1119//dEVFRXO2tpamThx4rY+ffrs7dq1a41SSgYNGlQBRr/1LVu2pObm5taamTv4fL40pZQMHDiwvH///vV2tw2H7qYaJhFxAauB0zEK/HPgEqVUSTivd3u8rYH7MPbBo9p6Pz7iKNocfVw0F9koaqqrcTiduFwuln75GXf9+SbmvPVhRMtwitTmZqVtzs1K+9FOV+BF2k1Vt+hhUkoFROQ64C3ACcwMp8jdHq8D48j5XUDzmIZMMFu+38QfJ16BCgZJSUnlr/f+M+Jl1Crl3Fq+9+hdVftatctp8l1mmusXg0NoutAjopR6A3gj3Oe7Pd6+wAwgqhMqJItjju3InAUfRGVZewO1Get/rOzWPCP1x3bZ6ZtdTkdMB1tMNLrQY8Dt8WYA9wKTaIQDngqFUgo7bboeyq6qmtyKvftyjspusqFF09SonJqKN+bwU2HPggP6qHvUuT3e/sDXwHU00vv73a59BKrK0cdbDLVB5dpUVnXcdzt2u2uDKqk+4+a0ydnAikhepw/GRYnb43UCtwG3Y+zDN5pmaQ5+3y+HY5qnIMlxli5qnA4CzVId21OcUm11ligJAisCgcD43r17hz1vnC70KHB7vB2A2Ri9yrT4UwvcDfzNV1xoy313XehHyO3xDgBeAtpYnUVr0DvARb7iwqTcd69PUu2/NDa3x3sN8B66yBPFMOAzt8cb9c5I8U636IfB7fGmANMxzo9riacCuNRXXDjP6iCNRbfoEXJ7vM0xNgF1kSeuLOBVt8cb0zEF4olu0SPg9njbYFwZ18vqLFrU/N1XXHiT1SFiTRd6mNwe7zEYLXlnq7NoUTcTuCaZj8jrQg+D2+PthlHk7a3OosXMS8AlvuLCmgafmYB0oTfA7fH2AhYCDY41piW8t4FzfcWFEY1Mmwh0odfD7fF2Bj4CWludRWs084BRvuLCgNVBokkfdT8Et8ebh9GS6yK3lxHATLfHm1TXEutCPwi3x5uLsU9+tNVZNEv8FuM6iaShC/0Abo+3GcYptKiMp60lrOvcHu8dVoeIFr2PHsIcDWY+xtjtmgZwha+4cJbVIY6UbtF/7h50kWs/94jb4z3iYb2tplt0k9vjHQs8Y3UOLS5tAfr4igu/tzrI4dKFDrg93j4Y0x+FN8OAZkf/A071FRcm5AAWtt90d3u8LYFX0EWu1a8fxkCfCcn2hQ48gTEZg6Y15Eq3x3uh1SEOh6033d0e73jgcatzaAmlDOjpKy7cZHWQSNi20M3LW5eQoDOXapZ6DxjqKy5MmOKx5aa72+N1Ac+ii1w7PEOAhOrDbstCBzzASVaH0BLaXW6Pt7vVIcJlu013t8fbEWPwe32UXTtSHwODEmET3o4t+r/QRa5FxwBgvNUhwmGrFt3t8Y4C5lqdQ0sqO4EuvuLCHVYHqY9tWnS3x5sJPGh1Di3ptACmWh2iIbYpdODP6AtjtNi4xhxyLG7ZotDNYZpvsDqHlrQcxHmrbotCx2jNM6wOoSW1c+K5O2vSF7rb4z0amGB1Ds0W7rI6wKEkfaEDfwXSrA6h2cLpbo93sNUhDiapC928OGac1Tk0W4nLffWkLnSM65FdVofQbGWA2+P9tdUhDpS0hW4OKHG51Tk0W4q7MzxhF7qIHBvOfXFkAtDE6hCaLZ3v9njj6pqNSFr0lw5yX1xeTur2eJ3ARKtzaLblAq61OkSoBgtdRPJFZBSQLSLnh3xdTvx2DhmJvgpOs9Y1bo83brYowzlQ1RU4B2gODA+5vwK4OgaZoiEhehRpSa0FcCHwpNVBIILeayJyslJqcYzzHDG3x9sa+B5wWp1Fs713fMWFv7E6BER26mmtiPwZcIe+Til1ZbRDHaEx6CLX4sMQt8fbxldcuM3qIJEU+msYkxwsBGpjEycqxlodQNNMToyG559WB4mk0DOUUrfGLEkUmCO79rU6h6aFGEscFHokp9fmi8jZMUsSHRdbHUDTDnCS2QBZKpJCvwGj2PeKSLmIVIhIeayCHabhDT9F0xqd5Z/LsAtdKZWllHIopdKVUs3M281iGS4Sbo83F+htdQ5NO4gzrQ4QySWwIiKXisjt5u08EYmn/eHfAGJ1CE07iFPcHq+lA59Esuk+AzgZuMS8XQk8FPVEh+8sqwNo2iGkAYOtDBBJofdTSl0L7AVQSpUBqTFJFSG3xysYLbqmxaszrFx5JIW+T0ScgAIQkVwgGJNUkesJ5FodQtPqYWlDFEmhTwdeAVqLyF3AR8DdMUkVuf5WB9C0BnR1e7zZVq087AtmlFLPisiXwOkYB71GKqW+iVmyyOgJE7V4J0Af4F0rVh7pCDPbMC6D/QRoIiK/in6kwxJPR/817VAsa5DCbtFF5E6MoZnWYe6nm9+HRD9W+Nweb1OgwMoMmham+C90jL61HZVSNbEKc5h+he6tpiUGywo9kk33FRiDT8SbnlYH0LQw5ZmDlja6SFr0e4AlIrICqK67Uyk1IuqpImN5hwFNi0AnoNGnWI6k0J8E7gWWEz/nz8F44zQtUXQE/tfYK42k0KuUUtNjluTw6ULXEklHK1YaSaF/KCL3APP4+ab7V1FPFSa3x+sA4nlseU07kCUNUySFfqL5PfQqNKtPr+URJ9fba1qY4rtFV0qdFssgh0mP3a4lmjwrVtpgoYvIpUqpZ0TkDwd7XCn19+jHClsLC9etaYfDks9sOOfRm5rfsw7ylRmjXOHSha4lmky3x5vS2CttsEVXSj1q/rhQKfVx6GMiMiAmqcKnC11LRDnAD425wkiujPu/MO9rTLrQtUTU6J/bcPbRTwZ+DeQesJ/eDOuvMc+xeP2adjjir9AxTl9lms/NCrm/HLggFqEiEDezVWpaBJo2/JToCmcffRGwSERmKaW+a4RMmpbsGn1LOJILZtJE5DF+OcmilRfM6OGdtUQU14X+IvAI8ATxPcmiFgVL08Yvz2JPe6tzJKMq0gLGYE2NJ5JCDyilHo5ZksOjW/QYOEHWrs6Wqh5W50hWmext9N6fkZxee11EJonIUSLSou4rZsnCoxp+ihapya65W6zOkOT2NfYKI2nRx5nf/xhynwKOi16ciFVZuO6kJASDAx0rulidI8nFb6ErpeKxO2iZ1QGSzRmOL5Y6JXhiw8/UjkCjF3okkyxmiMht5pF3RKSziJwTu2hh0YUeZb93vbLb6gw20OjvcST76P8BajCukgPYDEyNeqLI6EKPojRq9hbId3qwzdjb0NgrjKTQOyql7sPc7FBKVWH9UW9d6FF0sfO9JSLEzZz3SaqMIn9FY680kkKvEZEm/DTJYkdChpSyiC70KLra5bW674IdWHJ1aSRH3acAC4A8EXkWGIAxc4uVNlu8/qSRQ/nOduzQB+FiL74LXSn1joh8hTFmnAA3KKW2xyxZeDZgbGFYvQuR8Ca6Xl8hwilW57CBRt8/h8iOug8A9iqlvBgztvxZRI6JVbBw+IoLawB9cUcUXOR8X3f5bRyWtOiR7KM/DFSJSC/gDxiTLT4Vk1SRWWt1gETnli0b9SWvjSbuCz2glFLAucBDSqmH+Hn/dKussTpAopvsmrve6gw2Et/76ECFiPwJuBQ4RUQcQKMPcncQq6wOkOjOdHx+tNUZbCTuW/SLME6nXaWU2ooxpvr9MUkVmSVWB0hk/WTlyjQJxOPlzcloN0X+Rh0Usk4kR923An8Pub2BkH10EVmslDo5uvHC8gX6yPthm5zyktVnTuyk0SdXrBNJi96Q9CguK2y+4sJdGAcGtQg5CNaeJKXdrM5hIx9YteJoFrqVfcM/t3DdCescx+KvnaJyrc5hI4usWnE0C91KutAPw3WuV62+hNlOqoFPrVp5NAvdyn1ky/Z9ElUGe3d3ls29rM5hI59T5N9r1cqjWei/jeKyIvUZxjjzWpguc769VKTxxxe3Mcs22yGyS2ArRKT8gK+NIvKKiBynlFoRy6D18RUXBoD3rFp/IrrCtSDN6gw2Y9mBOIjsgpkHgU3Acxib6WMwJnX/CpgJDI5ytkgtAEZanCEhtKbsx9bsOsHqHDYSAD6xMkAkm+4jlFKPKqUqlFLlSqnHgDOUUi8QH3OgvWV1gEQxyfXaNyKWz5tnJ19R5K+0MkAkhV4lIheKiMP8uhCoO7hg+bDLvuJCH7Da6hyJ4ALnB62szmAz/7U6QCSFPhbjgNsPGNNM/Ba41Bx15roYZDscr1sdIN51lQ3fZsreAqtz2MwLVgeI5BLY9cDwQzz8UXTiHLHngJusDhHPJrvmbgD0te2NZxlF/q+sDhHJUfcuIvKuiKwwb/cUkdtiFy1yvuLCr4BSq3PEs9MdS6yccMOOnrQ6AES26f448Cd+GgV2GcaR93jznNUB4tUpjqXLU6Q2z+ocNhIAnrE6BERW6BlKqc8OuC8QzTBRogv9EG5wvaxHzW1cb1rVLfVAkRT6dnOI57rhni8gDsdr8xUXrsPCa4rjlYvAvhNl7fFW57CZWVYHqBPJBTPXAo8B+SKyGfgW40h8PHoUY7RazXS+88MlDlF9rc5hI9uJo7NAkbTomzGmZboLeB54h59mWI03s4EfrQ4RTyY6X6+1OoPNzKbI3+iTKR5KJIX+GsbptX3A90AlFkwWFw5fcWE1RquuAZlUlbtlq56coXHNsjpAqEg23Tsopc6MWZLoexi4lfgYwNJS411vLBNhoNU5bGRJPJw7DxVJi/6JiCTM2N++4sLvgblW54gHv3W+o7ujNq67rA5woEgKfSDwpYisEpFlIrJcRJbFKliUPGB1AKu1Y/uWFlToASYaz3LgZatDHCiSTfezYpYiRnzFhV+5Pd5XsXH31etdr6wW4Sirc9jInRT5Le/kdaBIrnW3ZOD5KLgdGEHyjI8XkXOdH+sibzwlxOnuYtJ/+H3FhSuAOVbnsEJPWbemidR0sTqHjRTFY2sONih00xTAdueRJ7vmfm91Bhv5lCJ/XLbmYJNC9xUXribOzmvGnlKDHMs7W53CRm6xOkB9bFHopj8Bu6wO0ViGOr5c5pJgO6tz2MTrFPk/tDpEfWxT6L7iwh+Bv1ido7Hc4HqlwuoMNhEAPFaHaIhtCt30CMaotUktlX3Vx8u3CXNxU4K7iyL/SqtDNMRWhe4rLgwCk4iDwSxjaYzz/SUiZFudwwa+BKZaHSIctip0AF9x4f8wutsmrWtc8233d7VANXAZRf54HHzlF+z6gbiZJJ1qOZvKXe3ZfoLVOWzgtkTYZK9jy0L3FRdWApeShOfWJ7jmLxch1eocSe5D4O9Wh4iELQsdwFdc+Clwt9U5ou0S57t63zy2dgOXU+QPWh0kErYtdNMdGDOxJoVjZOumbHbro+2xdTNF/vVWh4iUrQvdnIV1LEky5fKNrpfWiVg6T32ye4si/yNWhzgcti50AF9x4VqMse8S/pTb2Y7POlidIYl9D1xpdYjDZftCB/AVF75Kgu+v95FV36TJvo5W50hSFUAhRf6E7SSkC/0nfwXmWR3icE12zY2LiQKSUAAYTZH/a6uDHAld6CbzqrmxGEMBJRQHwdr+jpX5VudIUhMp8r9ldYgjpQs9hHl+vRDYaHWWSJzt+N9Sp6g2VudIQndR5H/C6hDRoAv9AL7iwo3AMBJoAojrXK/usTpDEnqGIn9czRZ8JHShH4SvuHAVcCYJcNqtCdVVXWWjHuU1ut4HrrI6RDTpQj8Ec671EcBeq7PU57fOd74WIdPqHEmkBDifIn+N1UGiSRd6PXzFhYuACzB6KsWlK11v2n4mmigqAc6gyL/L6iDRpgu9Ab7iQi9wNsZcc3GlJf7tbSjTc6pFx4fAIIr8m60OEgu60MPgKy58DxgKlFmdJdS1rtdWikQ0CYd2cK8Cv6HIH1d/32jShR4mc8CKU4GtVmepM9q5qKXVGZLAo8AFFPnj+ljMkdKFHgFfceFyjDno1lidpbNs8mXJnu5W50hwUyjy/44if9KNS3AgXegR8hUXrgP6Ae9YmWOya26iTpEVD2qBayjy32F1kMaiC/0w+IoLyzAmnfyHVRmGOb50W7XuBLcHGEWR/3GrgzQmUSrhe2dayu3xXo4xjHRaY61zgGNFybOpd+vN9sitA8ZQ5P/C6iCNTbfoR8hXXDgLOIVGHGzyRtdLOxprXUlkNnCiHYscdIseNW6PNxP4P+DyWK7HSW1gTdplfocofcQ9PFXA7ynyz7Q6iJV0ix4lvuLCSl9x4RXAaGJ4vv0850dLdJGH7TOgt92LHHShR52vuHAu0BNYGIvlT3TOS4gJAyxWA/wZ+DVF/lKrw8QDvekeQ26P9xJgGtA2Gstryp6KFWlXuURoEo3lJamvgHEU+VdYHSSe6BY9hnzFhc8B+cC/gCMeB/wK54JlusgP6Qfg90D/cItcRGaKyA8ikvT/FHSL3kjcHm9vYAbQ93CX8XnaxK9yxf+r6KVKCpXAA8A0ivwRdTwSkVPM1z+llDo+FuHihS70Rub2eM8D7gQiOg/elp3bFqddlyuit8JM+zCuU7+TIv9hD4wpIm5gfrIXuv7QNDJfceErGAfrLgPCnvHj966XS3WRA8b4+7OBfIr8vz+SIrcT3aJbyO3xpmBMCnAz0Km+565Mu2JVhlR3bZRg8ettwEORf0m0FmiXFl0Xehxwe7wOjNFnbwSGHPh4gfjWvZH2Z7tOzlCB0YI/Hour2nSha5Zwe7w9MAr+EiAd4ImUBxYNdX51qpW5LLAYeAJ4gSL/7litRBe6Zim3x9scuAgYtzbt0jyXBO0wr9oO4GngCYr8JbFemYjMBgYDrYBtwBSl1L9jvV4r6EJPBEXZxwIXml/JdnotAPwX+DfwCkX+uB2IM5HpQk80RdkdMQr+Nxjn5DOsDRQxBSwD3gXeAz6gyF9hbaTkpws9kRVlu4ATgF8DA8zv8biJvwajqN8F3qfIv93iPLajCz3ZFGUfjVHwdcXfncYbFENhzFtXan59BbxHkT+h5rJLRrrQ7aAouxXQvoGvuq6vteZXIOTrwNs/YBT0RmCD+X09sIoif1Wj/E5aRHSha4aibAdF/iPueKPFJ13ommYD+tppTbMBXeiaZgO60DXNBnSha5oN6ELXNBvQha5pNqALXdNsQBe6ptmALnRNswFd6JpmA7rQNc0GdKFrmg3oQtc0G9CFrmk2oAtd02xAF7qm2YAudE2zAV3ommYDutA1zQZ0oWuaDehC1zQb0IWuaTagC13TbEAXuqbZgC50TbMBXeiaZgO60DXNBv4f8bBdxZFXtYQAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Positive reviews: 69.52%\n",
      " Negative reviews: 30.48%\n"
     ]
    }
   ],
   "source": [
    "plot_sentiment_split(y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d08d057b",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAD3CAYAAAA0cknjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqoUlEQVR4nO2deXhTVf7/X58kpaXQlgJVBKpRWUoR0AFZBEZEHJeKogiiMOIIDi4z7ktmfjp2FLVuMwzfEddR3BV0HJAoKu4oLiAoFMsiRBZZBEraUmhJc35/3FuMCG1Sktwk97yeJ09zc+89592b+85Z7jmfI0opNBpNauOwWoBGo4k92ugajQ3QRtdobIA2ukZjA7TRNRoboI2u0dgAbfQYIyKPisjtVuuIFiJSKiJDrNYRbUREiUgn831KfWegjb4PEfGJyG4RqRKRzSIyXURaHmq6SqkrlFJ3RUPjwRCRCSJSJiKVIrJFRN4UkawopDtdRCaHfqaU6q6U+vBQ026Clg9FZGIjx0TlOoR+ZyIyREQ2NFV3oqCN/kuGK6VaAscDJwB/sVZO44jIycA9wEVKqSygG/CKtarij74OjaCU0i9jdKAPGBayfT/gDdnuD3wG7AS+AYaYn18ILNwvreuB2eb76cDkkH1nA0vMdD4Depqf/wF4I+S4VcDMkO31wPEH0H0T8L8G/q904EFgHbAFeBRobu4bAmwAbgS2ApuAP5j7/gjsBWqBqnptodcJKAZmAs8DlcBSoAvGD+RWU/PvQrTkAP8x89kITAac5r5Lgfmm1nJgLXCmue9uoA7YY2r5dxOuw3Tzf3/X1PoRcFTIfgV0Cv3OgBbAbiBo5lsFtLf6Xm3S/W21gER57XcDdzRv2n+Z2x2A7cBZGLWg08ztPCDTvHE6h6T1FTAm9KYx359gGqAf4ATGm/mmA8dgmN8BtAd+ADaY5x1j3vyOA+gebN6MfwcGAun77f8nMBtoDWQBbwD3mvuGAAHgTiDN/P+qgdz9tR/kOhWb5jsdcAHPmgb9f2Z6lwNrQ859HXjMNNBhwJfAJHPfpRg/LJeb1+ZK4EdAzP0fAhMb+P4auw7Tze/pt+b1/hcwP2T/r4weco02WH1/HvL9bbWARHmZN3CVeTMo4D2glbnvVuC5/Y5/Gxhvvn8e+Jv5vrOZRuYBbppHgLv2S2cFcLL5fj3wG2AM8LhphAKM0n52A9rPNA280/wf/mGaRYBdwLEhxw6oN595E+8GXCH7twL999e+33UKNfq7IfuGm/nXl9JZ5rVsBRwO1GDWJsz9FwEfmO8vBVaH7Ms0z21nbjdo9IauQ8j/8nLIsS0xagn55nZKG92FJpQRSql5ZnvvRaAtxk1zFDBKRIaHHJsGfGC+fxF4CKNkvBijCll9gPSPAsaLyJ9DPmuGUYKDUZ0cAnQy3+8ETsYw50cHE62Uegt4S0QcwCkY1ekVGCVoJrBIROoPF4wfgXq2K6UCIdvVGCYIly0h73cD25RSdSHbmOm1x7hmm0K0ODB+3OrZHPI/VZvHha2lgevwmHnI+pBjq0Rkh6lr/f5ppRra6AdAKfWRiEzHaC+OwLgRnlNKXX6QU94F8kTkeIxS6vqDHLceuFspdfdB9n+EUSoejdGxtBMYi2H0f4ehOwi8JyLvA8cBT2CYrbtSamNj5x8oySacczDWY5Tobff7YYm6lgNch3ry69+YT1RaYzQPopJvIqN73Q/OFOA0EemFUTUfLiKni4hTRDLMxy4dAZRSezFKjwcwbp53D5LmE8AVItJPDFqISFHII6CPMEqi5kqpDcAnwBlAG2DxgRIUkXNFZIyI5Jpp9sWoBXxu3vBPAP8UkcPM4zuIyOlhXoMtGP0Dh4xSahPwDvCQiGSLiENEjjVrT4espaHrEHLYWSIySESaAXdhXKPGSvMtQBsRyQlTZ0KijX4QlFI/YXQu/c28Gc4F/gr8hFE63cwvr9+LwDCMnvIDllhKqYUYnU3/xuhcW43RNq3fvxKjbfmJuV0BrAE+DakO70+5meYqoALjR+kBpdQL5v5bzXw+F5EKYB7QNczL8B+gUER2isj/wjynIS7BaKosN3W/ChwR5rn/Ai4QkXIRmXqA/Y1dBzC+ozuAHUBvYFxjmSqlyoCXgDXmdWjf2DmJSH2PpkaT0phNsQ1Kqdus1mIFukTXaGyANrpGYwN01V2jsQG6RNdobIA2ukZjA7TRNRoboI2u0dgAbXSNxgZoo2s0NkBPakkBFi1adJjL5XoSYwKH/vFObYLAskAgMLF3795bwz1JGz0FcLlcT7Zr165bXl5eucPh0AMjUphgMCg//fRT4ebNm58Ezgn3PP3rnxocl5eXV6FNnvo4HA6Vl5fn55fTbxs/L0Z6NPHFoU1uH8zvOiLvaqNrNDZAt9FTELfH2zua6flKihY1dozT6ezduXPn3XV1ddKpU6fdM2bM8GVlZQXDzsPnS7viiivy586du+azzz5rvn79+mYXXnihH+CFF17IKS0tbX7PPfdsbiydhpg6dWqbO+64o+Phhx++t6amRi699NKf7rjjjrA7tEI54YQTChYvXlx2KHriiS7RNVEhPT09WFZWtnzVqlWlaWlp6qGHHsqL5Hy327137ty5awAWLlyY6fV690V0GTt2rP9QTV7P8OHDy8vKypYvWLCgbMqUKUesXr06rSnpJJPJQRtdEwMGDRpUtXr16vQtW7Y4hw0bdmyXLl0Ke/XqVfDFF180B/B6vS0LCgoKCwoKCrt161ZYXl7uWLFiRbPOnTt337Nnj9x7773t33jjjdyCgoLCJ554Infq1KltLrnkkiO3b9/ubN++fY+6OiPYTkVFhaNdu3Y9a2pqpLS0NH3w4MGdu3fv3q13795dFy9enNGQxnbt2tUdeeSRNevXr08DmDZtWusePXp0KygoKLz44ouPCgQC3H///XmTJk3qWH9OvQ6AzMzME+o/v/322w8/7rjjunXp0qXw+uuvb1//2eTJkw8DmDBhQn7//v27AMyePTvrnHPOOToQCDBy5Eh3586du3fp0qXw73//+2FR/RL2QxtdE1X27t3L22+/nd2jR4/dt9xyS/tevXpVr1y5cvldd921cfz48UcDPPTQQ+2mTp36Q1lZ2fLPP/+8rGXLlvuq+BkZGeovf/nLj/Ul7+WXX15ev69NmzZ13bp1q37zzTezAF555ZWck08+2Z+enq4mTpx41LRp09aVlpZ+98ADD2y48sorj2xI56pVq5rV1NQ4+vXrt/vrr7/OePXVV1svXLiwrKysbLnD4VCPPvpom3HjxpW/9dZbrerPefXVV1uPHTt2R2g6//3vf7NXr16d8e2333733XffLV+yZEnmW2+91XLIkCFVn376aUuAJUuWZO7atctZU1MjH330UcvBgwdXLliwIHPTpk1pq1atKl25cuXyq6++entUvoCDoI2uiQo1NTWOgoKCwh49ehR27Nix9tprr9325ZdfZk2YMGE7wDnnnFO5c+dO144dOxz9+/evuummm/InT5582LZt25xpaeHXnkeNGlX+0ksv5QLMmDGj9ZgxY8r9fr9j8eLFLUeNGnVsQUFB4VVXXXXU1q1bD5joG2+8kdulS5fCwsLC4yZOnLg1MzNTzZ07N2vZsmWZvXr16lZQUFA4f/787DVr1qS3b98+kJ+fX/Pee++12Lx5s/P777/POO2006pC05s7d272xx9/nF1YWFjYvXv3wu+//z6jrKwsY9CgQdVLly5tsWPHDkd6errq06dP1SeffJK5YMGCrKFDh1YVFBTUrF+/Pn38+PH5r776anZubu7BYgJGBd0Zp4kK9W30cI695557No8YMcI/a9asnMGDBxd4vd5VmZmZYXXcXXTRRTvvuuuuDlu2bHEuW7Ysc/jw4RUVFRWOrKysQDj5Dx8+vPzZZ59d9/HHH2cOHz68y+jRo3cqpWTUqFHbH3744V+FxB41atSOl156KbegoGDPmWeeWe5w/LJsVEpx3XXXbbr55pu37X9ufn5+zbRp09r27du3qlevXrvnzZuX9cMPP6SfcMIJexwOB8uWLVv++uuvZz/66KN5r7zySuuZM2f6wrkGTUGX6JqY0a9fv8qnn366DcCcOXOycnNzA61btw6Wlpam9+3bd/fdd9+9uWfPnruWLVv2i/Z0dnZ2XVVV1QHvzZycnGDPnj13TZo06chTTz3V73K5aN26dbBjx461Tz31VC5AMBhkwYIFzRvS9tvf/rb6/PPP337fffcdfsYZZ1TMmTMnd+PGjS6ALVu2OFeuXNkMYOzYsTvffvvtVjNnzvxVtR3gzDPPrHjuuefa+v1+B8DatWvT6tMZMGBA1cMPP3z4kCFDKocNG1b5zDPP5BUWFlY7HA42bdrkqqur49JLL9157733bly6dGlm5Fc4fHSJnoKE8zgsHtx3330/jh071t2lS5fC5s2bB6dPn74W4P777z/ss88+yxYR1bVr190XXHCBf926dfuq2meeeWblgw8+eERBQUHhjTfeuGn/dEePHl1+2WWXHTNnzpwV9Z+99NJLay6//PKj7rvvviMCgYCcd955OwYMGLB7/3NDueOOOzb36dOncPLkyZtuu+22jaeeemqXYDBIWlqamjp16rouXbrU5uXl1XXq1GnPqlWrmp9yyim/Wn3n/PPPrygtLc048cQTCwAyMzODL7zwwtoOHToETj755MqpU6e2Gzp06K7s7Oxgenq6GjhwYBUYjxMnTJjgDgaDAnDnnXfGdGlmHTMuBfjmm298vXr1+lXVUZO6fPPNN2179erlDvd4XXXXaGyANrpGYwO00TUaG6A742xCIBh01OwNptcEgum1gWB6IBhMqwsql/lyBhXOoFIOjGWVwVxF1NxQDofUOR1S5xQJOB0ScDqkzuWQvc1cjtr0NOeedJej1iGiO3wSFG30FCMYVLKrNpC5u7auxe69dZk1gWDzvXXBZnVBdWjfdSPDOQRwOR01zZyOmmYux57MZs5dmc1cuzLSHDUh66FrLEIbPclxe7ztXhnVoeUP23e12LM32KI2EMxQqLg7SwF764Lpe+uC6btqyS43H0Q5ROoy0pzVzdOcu1qkOyuzM9IqHQ5d8scbbfQkw+3xpgODgNPNV4+KmqA037133zE9nzwqqnl+O/GHRo/plZ/L7y+/mpv+NhmAZx79P6qrd3HlDR5ndW0gq7o2kLV9F+1ERGWmOStbZrj82Rlp/ubNnDWRaPF4PO1KSkr2zWSL1nTR0Gm2+fn5NTNmzFjbtm3biIelXnfdde2HDBlSOWLEiMpD1RRNdGdcBIjIGSKyQkRWi4gnXvm6Pd40t8d7ttvjfRHYjrHG+c1AT35uU1tKs/R03pv7BuU7Gp6boZSSXbWB7C0Ve/JXba087rtNFcdtKK/uWF0baHAkWz1Tp079xXrq0ZouGjrNtlWrVoEHHnggomm29UyZMuXHRDM5aKOHjYg4gYeBM4FC4CIRKYxVfm6PV9we78luj/cxYBPwBnAR0CJWeR4KTqeLCy4ez/NPTPvVvh3bt3HDHy/h4qKhXFw0lMVffb7v88suPDf95P69Dx83/rLC9h3zf7N09bojagJ1acOGDTu2e/fu3Tp16tT9wQcfbAtw1VVXdaifPHPOOeccDT9PFz377LOPefnll/fNYR85cqT76aefzg0EAkyaNKlj/TTSBx54oG1j/0v//v13bdy4sRnAgaa/NjRdtj5fgE8++STzxBNP7Nq9e/dugwYN6vzDDz+kbdy40dW9e/duAAsWLGguIr1XrVrVDCA/P/+4yspKx1NPPZXbuXPn7l27di3s06dP10P4WvahjR4+fYHVSqk1Sqla4GXg3Ghn4vZ4c9we7w3AauBD4I9Am2jnEwsuHD+RN/83k8oK/y8+v/8OD+MmXsmL3vd56PFn+Pst1wLw6D/vo+9Jg3n9vQWcdtY5bNq4QXZU722/YnNlz3v//R/n/K8W/7h48eLljz322OGbN292Tps2bWN9yTt79uy1oXmMHj16x4wZM3IB9uzZI59++mn2qFGjdk6ZMqVtTk5O3bJly7775ptvvnvmmWfyysrKmh3sfwgEAnzwwQdZI0aM2AlwoOmvDU2XrU+npqZGrrnmmiNnzZr1fWlp6Xfjx4/fdtNNN3Xo0KFDoKamxrFjxw7HBx980LJ79+7V8+bNa7ly5cpmbdq0CWRlZQVLSkqOeOedd1auWLFi+dy5c1dH47vRbfTw6QCsD9neAPSLVuJuj7cb8GfgEhK01G6MllnZnD1yDC8+9TgZGT/PU/l8/kesWbVvWDpVlZVU76piyVef848nngdg4CnDyM5pte+Ypx5/JOv9uXOyALV582aWLitt3q5du19MEQ3lggsu8N966635u3fvltdeey2nb9++lS1btlTz5s3LLisry5w9e3YuQGVlpXP58uUZBQUFtaHn19cUtmzZknbsscfuGTFiREXo9Nf642prawV+ni47fPjwyhkzZrS+6qqrfgpN79tvv01ftWpV86FDh3YBY6JNXl7eXoA+ffpUzZs3r+X8+fOzbrnllk1z587NUUrRv3//qvr9Y8eOdY8cObJ87Nix5UQBbXSLcXu8A4G/Ab+zWks0GDfhSsacdTLnjh677zMVDPLcrHdJz2gw6Ms+vlown8/nf8izs96hefNMmTDqbNaX7+68obz6p4Odk5mZqfr371/53//+N/uVV17JHTNmzA4w+gQeeuihdSNHjqxoKM/6mkJlZaVjyJAhnUtKSg676qqrth1s+uuBpsuG7ldKSadOnXYvWbLkV30IgwcPrvz444+zNmzY0Gzs2LE7H3rooXaAOvvss/0AL7744rr333+/xezZs3N69+5duGjRouXt2rU7pPnquuoePhuB/JDtjuZnTcLt8Z7o9njnAvNJEZMD5OTm8ruzR/D6y8/t+2zAb0/hpemP79suK10KwPF9+vHOnNcB+Oyj96nw7wSgqqKC7JxWNG+eydrVK/l28UIUOHbsqj3c1ayZY83mnfm1geCvAktceOGF5dOnT2/71VdfZdUb+7TTTvM/8sgjeTU1NQJGSVtRUXHQ+z4rKys4derUddOmTTs8KyvroNNfDzRdNpSePXvu2bFjh2vevHktwKjKL1y4MANg2LBhVa+99lrro48+usbpdNKqVavABx98kFMf1KK0tDR96NChu6ZMmfJjbm5uYM2aNQdtaoSLLtHD5yugs4gcjWHwMcDFkSbi9niPB+4EhkdVXQjhPA6LJZf88U+8PP3Jfdu33nkf9/y/m7ngtIHU1dXxm34DuP3efzLp+lvx/Gkic157hV69+9L2sMNp0aIlA4ecysznn2LEKf1wH9OJnif02ZfWyIvHM2xwv8O6Hdcz7/Gnnt0Smu95551XMWnSpKNPO+20nRkZGQrg+uuv3+bz+dJ79OjRTSklrVu33vvmm29+35D+gQMH7i4oKNj9+OOPt25o+uuBpsvWk5GRoV5++eXvr7nmmiMrKyuddXV1cuWVV27p06fPnq5du9YqpWTw4MGVYMxb37RpU7O8vLw6U3NHn8+XrpSSQYMGVfTv37/B6bbhoKepRoCInAVMAZzAU0qpu8M91+3x5gH3A+OJ8iOxJ845gsOPPCaaScaF2poaHE4nLpeLbxZ9yd1/vZEZb38SURpOh9TltUz/MS8rfaudRuBFOk1Vl+gRoJR6E3gzknPcHq8TuAKYDLSKgaykZdOPG7j5yj+ggkHS0prxt/v+FXEadUHl3FyxJ39n9d427XOb/9Ay3fWr4BAabfSY4vZ4B2A8ez+hsWPtyFFHH8uMuR9HJa09gbrMNT9VdWuV2eyn9jkZG11OR0yDLSYb2ugxwO3xZgL3AVcTh5FrCoVSCjtVXQ/GzuravMo9e3OPyGm+rnWLZlF5NJVomOGnwl4FB3Sve9Rxe7z9gSXAn4jT8NQfdu4lUF2B7m8xqAsq14by6mN823e564Iqpe5xc9nkHGBZJOfpzrgoYbbFbwNux+isixvZ6Q7+3C+Xo1qlIYkx9D1hcAqB7HTHtjSnRDR5JoEJAssCgcDE3r17h71unDZ6FHB7vB0whsQOslqL5oDUAXcDd/pKimzZdtdGP0TcHu9JwGtAO6u1aBplHnChr6ToV/HZU52Uar/EG7fH+0fgA7TJk4VhwJdujzdmsw4TFV2iNwG3x5sG/BtjZpkm+agExvpKit6wWki80CV6hLg93mzgXbTJk5ks4H9ujzduwUOsRpfoEeD2eA8D3gaOt1iKJnr8E7jRV1KU0kbQRg8Tt8d7FEZJ3tlqLZqo8xTwx1TukddGDwO3x9sdoyTvYLUWTcx4FaPdXtvokUmINnojuD3eHhg960kRzklzSLwNjPCVFO2xWki00Z1xDeD2eDsB76BNbhdOB2a4Pd6UmwOijX4Q3B5vPsYAC/2M3F4MB552e7wpNZZYG/0AmL3r84DoroSgSRbGAZFPjk9gtNH3w+3xZmFU17tYrUVjKX92e7x/t1pEtNCdcSG4PV4HMAs422otmoThD76SoulWizhUdIn+S+5Bm1zzSx51e7xRi99vFbpEN3F7vOOA5xo9UGNHNgF9fCVFP1otpKloowNuj7cv8BEQ3goDGjvyBXCyr6QoKQNY2L7q7vZ4W2PMJ9cm1zREP+DXK0gmCbY3OvA4xqorGk1jXOb2eEdbLaIp2Lrq7vZ4JwBPNnqgRvMz5UBPX0nRBquFRIJtje72eLsAX5OkK5dqLOV9YFgyTW21ZdXdjBDzAtrkmqYxFLjBahGRYEujAzcDfRo9SqM5OHeb05eTAttV3d0e79FAKdDcai2apOdTYHAyVOHtWKI/jDa5JjoMBCZYLSIcbFWiuz3eC4CZVuvQpBQ7gC6+kqLtVgtpCNuU6OastClW69CkHK0xlsROaGxjdIwOOB3zTRML/uj2eHtZLaIhbGF0M5DE9Vbr0KQsDoy13RIWWxgd+CvQ0moRmpSmyFwyOyFJeaO7Pd4jgSus1qGxBQnbVk95owPFQLrVIjS24FS3xzvEahEHIqWN7vZ4jwEusVqHxlYkZKme0kbHGI/stFqExlYMdHu8J1ktYn9S1uhuj7cN8AerdWhsybVWC9ifsI0uIkeH81kCMQnItFqExpac7/Z4EyqYSSQl+msH+OzVaAmJJuaSOldarUNjW1zA1VaLCKVRo4tIgYiMBHJE5PyQ16Ukbpy1EejwUBprudzt8SbM5KlwFpPrihHrvBXGulT1VAKXx0BTNNBtc43VtAFGA89YLQQimL0mIgOUUgtirOeQMYe7biS8HzGNJpa86ysp+p3VIiAyM6wWkb8C7tDzlFKXRVvUITIGbXJNYjDU7fG285UUbbZaSCSGmAV8grHKaF1s5ESF31stQKMxcWIUPFMs1hGR0TOVUrfGTEkUcHu8BehYcJrE4mISwOiRPF6bIyJnxUxJdLjQagEazX6c6PZ4O1stIhKjX4th9j0iUiEilSJSESthTSTRf4g09uQcqwWEbXSlVJZSyqGUylBKZZvb2bEUFwluj7ctutquSUzOsFpAJENgRUTGicjt5na+iPSNnbSIOYMUHruvSWoGuz1eS4djR2KMacAAjM4FgCqM0MmJwplWC9BoDkI6MMRKAZEYvZ9S6mpgD4BSqhxoFhNVEeL2eB3A6Vbr0GgawNL7MxKj7xURJ6AARCQPCMZEVeR0xxhyqNEkKpaOkIvE6FOB14HDRORuYD5wT0xURU4/qwVoNI3Q1e3x5liVedgDZpRSL4jIIuBUQIARSqnvYqYsMrTRNYmOYDwVes+KzCPtpd6CMQz2M6C5iPwm+pKahDa6Jhk40aqMwy7RReQu4FLge8x2uvl3aPRlhY/b422J0UbXaBKdxDc6xtzaY5VStbES00R6o5+fa5IDy4weiUGWYQSfSDQKrRag0YRJvhm0NO5EUqLfCywWkWVATf2HSimrx/FaPmFAo4mATkDcl1iOxOjPAPcBS0mc5+egja5JLo4Fvoh3ppEYvVopNTVmSpqONrommehkRaaRGP0TEbkXmM0vq+5fR11VmLg9XidwjFX5azRN4FgrMo3E6CeYf0OXhrX68Vo+kGZh/hpNpCS20ZVSp8RSSBM53GoBGk2EHGlFpo0aXUTGKaWeF5EbDrRfKfWP6MsKGz2RRZNstLYi03Ceo7cw/2Yd4NUyRrrCpa3F+Ws0kdLC7fHGvbnZaImulHrMfDtPKfVp6D4RGRgTVeGjja5JRlpjzBuJG5GMjPu/MD+LJ7rqrklGcuOdYTht9AHASUDefu30bIwA9VbSyuL8NZqmEPd2eji97s0w2uIujHZ5PRXABbEQFQH60ZomGWnR+CHRJZw2+kfARyIyXSn1Qxw0RYKetaZJRuJeE45kwEy6iDzOrxdZtHLAjDa6JhlJaKPPBB4FniRxFlnURo8R36ZPXNqS3R2s1pGKVJMeiHOne0RGDyilHomZkqahjR4DesuKsmyp7mG1jlSlJXviPvszEqO8ISJXicgRItK6/hUzZeGRKDWLlOIm18ytVmtIcfbGO8NISvTx5t+bQz5TWDt7LNEWeUx6nNQF+jm+62a1jhQn7uHYIpnUcnQshTQRv9UCUo3znPMXO0RZFtvMJsS9RI9kkcVMEbnN7HlHRDqLyNmxkxYWOy3OP+W42jkrYLUGG1Ad7wwjaaM/jVHlOMnc3ghMjrqiyNAlehTJpsrvls0nNH6k5hBZF+8MIzH6sUqp+zGrHUqpaozVJ6xkp8X5pxSTXN5vRciwWkeKU06xvzLemUZi9FoRac7PiyweS0hIKYuIezTNVGac813L1gazEZaMLo2k1/0OYC6QLyIvAAMxVm6xkvUW558yHCM//pAj1T2t1mEDEtvoSql3ReRrjJhxAlyrlNoWM2XhsR6jhmF1EyLpuck1wwccZbUOG2CJ0SPpdR8I7FFKeTGmh/5VRCy9MXwlRbXAj1ZqSA2UOs2xSEfTjQ+JbXTgEaBaRHoBN2AstvhsTFRFxvdWC0h2TnEsWZomdflW67AJCW/0gFJKAecCDyulHuaX89OtYrXVApKd612v6seU8SOx2+hApYj8BRgH/FZEHCRG4IfvrBaQzKRTu6eHrNWdcPEj4Uv0CzEep01QSm0GOgIPxERVZCyyWkAyM9b53mIR9GO1+LCLYv9PVmQcSa/7ZuAfIdvrCGmji8gCpdSA6MoLi6/RPe9N5nLXnEhqdZpD40urMo7mfG5LRlT5Sor86A65JnEY5T+1o1wPeY0fH1mVcTSNrqKYVqTo6nsT+LPr9eUiEfXTaA6NlDC6lSy0WkAyMtL5STurNdiIWuBzqzKPptGtbCPPtzDvpKSnfL8qU2q6Wq3DRnxJsX+PVZlH0+i/j2JakfIVeiZbRNzomqlHFMYXy6rtENkQ2EoRqdjvtV5EXheRY5RSy2IptCF8JUV1wAdW5Z9sOAjWDXIsLbBah8342MrMI+mImQJsAF7EqKaPwVjU/WvgKWBIlLVFyjvAeRZrSAqKHJ8vcYrqbbUOGxEAPrNSQCRV93OUUo8ppSqVUhVKqceB05VSr2DBonEH4F2rBSQL17hetzqOgN1YRLG/ykoBkRi9WkRGi4jDfI0G6jsXrHy0BoCvpOh79PP0RmnB7spOsvF4q3XYDEvb5xCZ0cdidLhtxVhm4vfAODPqzJ9ioK0pvGa1gERngvPNb0TItFqHzZhptYBIhsCuAYYfZHeiPN56CbjFahGJzKWud1parcFmLKPYb/k4j0h63buIyHsisszc7ikit8VOWuT4SoqWAGVW60hU8mXrxlwqe1mtw2ZMt1oARFZ1fwL4Cz9Hgf0Wo+c90XjZagGJyg2umatF9OSfOBIAnrdaBERm9Eyl1P6zbxIx2L82+kE4y/HFkVZrsBlvUeyP77KpByESo28zQzzXh3u+ANgUE1WHgK+kaAUWTgdMVAY4SkvTJZCIy2qlMtOtFlBPJANmrgYeBwpEZCOwFqMnPhF5GOhrtYhE4kbXTB0DP75sA96wWkQ9kZToGzGWZbobo3r8Lj+vsJpovAJYEskjEUkjUPsbWanXO48vL1Lsj/tiigcjEqPPwni8thcjxHIVsCsWog4VX0lRDUbtQwOMdn642CEJMXrRTjxttYBQIqm6d1RKnREzJdHnEeBWIvsfU5IrXbN1T3t8+Zpi/xKrRYQSSYn+mYgkTfXPV1K0EXjdah1Wk0vFjg5s0+Gi4ss9VgvYn0iMPghYJCIrRORbEVkqIt/GSliUuIcEGIdvJVe7Zi0TSYiw3HZhKfBfq0XsjxhrMoRx4EGWX1JKWRKnOlzcHu9rwPlW67CKZemXLW8pewqt1mEjRlPst3xs+/5EMtY9oQ3dAHdgzFO3XTu1QNat0SaPK6XAq1aLOBCpEhzyoPhKipaRALOHrOAm1wy9rHR8KabYn5BNxZQ3usnfgaDVIuKLUkMcSzpbrcJGfEGxPyFLc7CJ0X0lRcsxwl3ZhtMdC79xSbC91TpsxM1WC2gIWxjd5C9AudUi4sW1rtcScjBTijKLYv8nVotoCNsY3VdStA1IqPnzsaI5NdXdZJ2edx4fAoDHahGNYRujmzwKLLZaRKwZ73x7iQg6kkx8KKHYn/DBTmxldF9JURAjvl1C9oxGiwmutyxZ8NKGfA3cabWIcLCV0QF8JUWfYZTsKckRbN/cFv/xVuuwATXAJYk0Q60hbGd0k5uAVVaLiAXXuV5bIWLb7zWe3Eaxv9RqEeFiyxvCV1JUjRGuus5qLdHmXOenHazWYAM+Af5htYhIsKXRAXwlRV8A91qtI5r0lhVlGbK3k9U6UpwqYDzF/qQagGVbo5vcCSyyWkS0uMk1c6vVGmzAjRT711otIlJsbXRfSdFe4CLAb7WWQ8VJXaCf47tuVutIcd6k2J+UkYtsbXQAX0nRKuASkvyR23nO+YsdovKs1pHCbAYmWi2iqdje6AC+kqLZGEEvk5arnbMSMcZ+qlAFFFHsT7jw5uGijf4zfwNmWy2iKWRT5XfLZh0uKjYEMIJJfG21kENBG93EV1KkgHHAMqu1RMok15xvRdCj4WLDlRT737JaxKGijR6Cr6SoEjgDSKpoOuOc81pZrSFFmUyx/0mrRUQDbfT9MKPH/o4kWQDiGPnxhxypTprovEnEsxT7b7daRLTQRj8AvpKilcCZQKXVWhrjJtcMn9UaUpB5JHEP+4HQRj8IvpKiRcAIjMkLCYpSpzkWHWO1ihTjW2BkskxWCRdt9AbwlRS9jxFBdrfVWg7EKY4lS9OkLt9qHSlEGXAWxf4Kq4VEG230RvCVFL0FnIXxLDWhuN71atKP6EsgPgMGUuzfaLWQWKCNHga+kqIPgWEkUMy5dGr39JC1Pa3WkSLMAoZR7N9htZBYoY0eJuZst1OAhJg4Mtb53mIRcqzWkQI8htEmT8jmWbTQRo8AX0nRN8BAjLacpVzummP7VWKjwO0U+6+g2J9ycQn2Rxs9QnwlRauB/sDbVmk4jPKf2lGuh7w2nQAwkWL/ZKuFxAtt9CbgKynyA0XAv6zI/8+u15eL6HXfm0g1MIJi/3+sFhJPwl5NVXNg3B7vROBhoFm88lye/ocVmVLTNV75pRBrMSaoLLRaSLzRJfoh4ispehI4CVgZj/x6yvertMmbxMvA8XY0OWijRwVzFN1vgKdjndeNrpk/xjqPFGMXMIFi/0WpOBAmXHTVPcq4Pd7RGI9sWkU7bQfBulXpv9/mFHV4tNNOUb7CiL1u+VMSq9ElepTxlRTNAHoBc6OddpHj8yXa5GFRg7Go5gBtcgNdoscQs3SfAhwRjfTebXbzZ50dG0+KRlopzELg0mRaXCEe6BI9hpilezdgGnBIccBbsLuyk2w8Phq6UpTtwA0YpXijJheRp0Rkq4gkXUShpqBL9Djh9nj7Af8EBjTl/Gucr82/Ie21QdFVlRLswlg15cFIOttE5LcYE5WeVUodFytxiYI2epxxe7znYkSc7R7JeV+nT1rSWiqPj4mo5KQWeBy4i2J/k+YfiIgbmGMHo+uqe5zxlRTNAnoClxJmbLp82boxl8pesdSVRASB54ECiv1/bqrJ7YY2ugX4SoqCvpKiZ4AuwCRgRUPH3+CauVoEiYu4xMYLnECx//fJuCySleiqewLg9ngFY+z8DRhTYX/BivRL1qZL4Oi4C0sMdgEzgMco9n8RzYTtVHXXRk8w3B7vCcB1wCig+UmOZaUvNrsnovZ8ivA58B/gFYr9MQnSqY2usRy3x5sDjHku7Z4Bg53LxlutJ078BDwH/Idi//JYZiQiLwFDgLbAFuAOpVTKzmjTRk8GinPcwBjzlWqdckGMuf3/AWanWvTVREEbPdkozinAiEz7W4xn8skWTkphLHv1gfn6iGJ/wsTiS1W00ZOZ4hwHxqO6QeZrMNDeUk0H5jt+NvaHFPu3WazHdmijpxrFOUfzs/H7A24gO065K2ADxuPCMuBTDGNvjlP+moOgjW4HinOygY5Afsir437vm2O0l/d/1e23vRvYiGHo+td6jMAbq1I9mmqyoo2u0dgAPTJOo7EB2ugajQ3QRtdobIA2ukZjA7TRNRoboI2u0dgAbXSNxgZoo2s0NkAbXaOxAdroGo0N0EbXaGyANrpGYwO00TUaG6CNrtHYAG10jcYGaKNrNDZAG12jsQHa6BqNDdBG12hsgDa6RmMDtNE1Ghugja7R2ABtdI3GBmijazQ2QBtdo7EB2ugajQ3QRtdobIA2ukZjA/4/bfhbTvL+84oAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Positive reviews: 69.4%\n",
      " Negative reviews: 30.6%\n"
     ]
    }
   ],
   "source": [
    "plot_sentiment_split(y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b373584c-9a1e-44a1-ad7d-354d9b4d79f6",
   "metadata": {},
   "source": [
    "#### Both the training and test data after splitting is very representative of the split of the entire dataset (roughly .1 percent difference)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "94ceef53-ab3f-411c-9f02-abf2d5c961e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "#convert to 1d arrays for modeling\n",
    "y_train, y_test = np.ravel(y_train), np.ravel(y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dedd5bf-0a9d-4124-a09e-692e4f4fb68e",
   "metadata": {},
   "source": [
    "## 3.5 Building Models <a name=\"3.5\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d19fb63b-d126-4963-a50e-39420101fafc",
   "metadata": {},
   "source": [
    "### 3.5.1 Logistic Regression with Count Vectorization <a name=\"3.5.1\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "995c8e44",
   "metadata": {},
   "source": [
    "#### 3.5.1.1 Training and Fitting the Model <a name=\"3.5.1.1\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f7e79147-630d-42e5-9065-ffb43c3d309e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(786016, 77986) <class 'scipy.sparse._csr.csr_matrix'> (87336,)\n",
      "(786016,) (87336,)\n"
     ]
    }
   ],
   "source": [
    "#Trying a logistic model with simple bag-of-words Count Vectorization\n",
    "\n",
    "count_vec = CountVectorizer()\n",
    "\n",
    "#fit the vectorizer on the training data\n",
    "count_vec.fit(X_train)\n",
    "\n",
    "#transform both the training and test data\n",
    "X_train_vec = count_vec.transform(X_train)\n",
    "X_test_vec = count_vec.transform(X_test)\n",
    "\n",
    "\n",
    "\n",
    "print(X_train_vec.shape, type(X_train_vec), X_test.shape)\n",
    "print(y_train.shape, y_test.shape)\n",
    "\n",
    "#Using the sklearn StandardScaler to normalize the data, with a \"with_mean\" parameter for sparse matrices\n",
    "scaler = StandardScaler(with_mean=False)\n",
    "\n",
    "#fit on the training data, scale both train and text\n",
    "X_train_vec = scaler.fit_transform(X_train_vec)\n",
    "X_test_vec = scaler.transform(X_test_vec)\n",
    "\n",
    "#using the default solver\n",
    "lr = LogisticRegression(solver = \"lbfgs\", n_jobs=-1, max_iter=1000)\n",
    "\n",
    "#train the lr model with the training data\n",
    "lr.fit(X_train_vec, y_train)\n",
    "\n",
    "#predict on the training data\n",
    "train_preds = lr.predict(X_train_vec)\n",
    "\n",
    "\n",
    "#predict on the test data\n",
    "test_preds = lr.predict(X_test_vec)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68e29c8f-b9ce-4ab3-8e24-3cdb8f203fc4",
   "metadata": {},
   "source": [
    "##### Count Vectorized-Logistic Regression Train Set Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "affd3bee-1e52-4387-8bf5-5cb03e7c7e39",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.92      0.94      0.93    546424\n",
      "           1       0.87      0.81      0.84    239592\n",
      "\n",
      "    accuracy                           0.90    786016\n",
      "   macro avg       0.89      0.88      0.88    786016\n",
      "weighted avg       0.90      0.90      0.90    786016\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_train, train_preds))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cf45419",
   "metadata": {},
   "source": [
    "### Evaluating the model <a name=\"3.5.1.2\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "217af7d4-026f-4b63-bd09-eefdf0c373e9",
   "metadata": {},
   "source": [
    "##### Count Vectorized-Logistic Regression Test Set Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9c68738a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Create a dictionary to store the results for each model to compare at the end of the notebook.\n",
    "results_dict = {}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "450d408c-6b0b-408a-b87c-ff67867a59cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.90      0.92      0.91     60609\n",
      "           1       0.81      0.76      0.78     26727\n",
      "\n",
      "    accuracy                           0.87     87336\n",
      "   macro avg       0.85      0.84      0.85     87336\n",
      "weighted avg       0.87      0.87      0.87     87336\n",
      "\n",
      "Confusion matrix values: \n",
      "[[55949  4660]\n",
      " [ 6493 20234]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEGCAYAAADyuIefAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAg5ElEQVR4nO3deZgV1bnv8e/bEzPNKCKNCAoa1KCIiHL0OuQyeDxBE41TAjHGIaLGnOQ6nqOJxsRoEoxjQpSIc5xFwyAXNWKOiOCAAkK3KDIKzWwzdvd7/qi1oRi6e2/o3b179+/zPPV01apVVatoeFlD1Spzd0REJJJT3wUQEckkCooiIjEKiiIiMQqKIiIxCooiIjF59V2AuA7tcv2grvn1XQxJwfxZzeu7CJKCzZSx1bfYvpxj8CktfNXqiqTyzpy1ZZK7D9mX69W1jAqKB3XNZ/qkrvVdDEnB4AOOqu8iSAre9Sn7fI7S1RW8O6koqbz5nT/rsM8XrGMZFRRFpCFwKryyvguRNgqKIpISByrJ3pc+FBRFJGWVqKYoIgKA42xT81lEJOJAhZrPIiI7qE9RRCRwoCKLZ9dSUBSRlGVvj6KCooikyHH1KYqIJLjDtuyNiQqKIpIqo4J9en06oykoikhKHKhUTVFEZAfVFEVEgujhbQVFEREgCorbPHvnp1ZQFJGUOEZFFk/ar6AoIimrdDWfRUQA9SmKiOzCqFCfoohIJJp5O3uDYvbemYikhbux1XOTWmpiZl+Y2cdm9qGZzQhp7cxsspkVh59tQ7qZ2T1mVmJms8ysb+w8I0L+YjMbEUs/Jpy/JBxbY7tfQVFEUlaJJbUk6RR3P8rd+4Xt64Ep7t4TmBK2AYYCPcNyKfAgREEUuAU4DugP3JIIpCHPJbHjavzcqoKiiKQkGmjJSWrZS8OAsWF9LHBmLP1Rj0wD2phZZ2AwMNndV7v7GmAyMCTsa+3u09zdgUdj56qS+hRFJEW1OtDiwGtm5sBf3H000Mndl4X9y4FOYb0LsCh27OKQVl364j2kV0tBUURSkuJAS4dEX2EwOgS+hH9z9yVmth8w2cw+3ela7h4CZp1RUBSRlFUk//B2aayvcDfuviT8XGFmLxL1CX5lZp3dfVloAq8I2ZcAXWOHF4W0JcDJu6S/GdKL9pC/WupTFJGUOMY2z0tqqY6ZtTCzVol1YBDwCTAOSIwgjwBeDuvjgOFhFHoAsC40sycBg8ysbRhgGQRMCvvWm9mAMOo8PHauKqmmKCIpSQy01IJOwIvhKZk84El3n2hm7wHPmNnFwELgeyH/eOB0oATYCFwE4O6rzew24L2Q71Z3Xx3WrwAeAZoBE8JSLQVFEUmJY6k0n6s+j/sCoM8e0lcBp+0h3YGRVZxrDDBmD+kzgCNSKZeCooikLJvfaFFQFJGUuKN3n0VEEqKBlppf4WuoFBRFJGWaZFZEJHBMk8yKiMSppigiEkTffVZQFBEJTJ8jEBFJiD5xqtFnEREgmnlbzWcRkRg9vC0iEkTzKapPUUQk0CdORUS2ix7JUU1RRATQu88iIrvR1GEiIkE0dZiazyIi26lPUUQkiGbJUfNZRARIvOanoCjA8P69adaygpwcyM1z7ps4n8d+vz8TnmxHYbsKAC66YSn9T9vAtq3Gn64tonhWcywHfnLrEvqc8PVO57tlRHeWfVnA6DfmAfDZ7Kbce31XNpXl0KloK9fdv5AWrSrr/D6zVU6Oc+/E+axals/NI3oAzg+vW86JZ6ylstJ49dH2vPxwRwC+efzXXH7rEvLynHWr8/h/3z0EgH4nr+fy25aSm+NMeKodz9zXqR7vqL6oprjXzGwI8CcgF3jI3e9I5/Xqwp3PllDYvmKntLMuWck5P1m5U9qEJ9oD8JfX57G2NI+bLuzBvRPmkxP+Lr09vpCmLXYOeHf/4kAuuXkJ3zy+jElPteO5B/djxLXL03czjcyZPy5lUXFTmreMfn+Dzl1DxwO28eOTDsPdKGy/DYAWrSu48reLuenCHqxcUrA9PSfHGfmbJdxwXg9Kl+Vz7/hipk0q5MvipvV2T/Ulm99oSVu4N7Nc4H5gKNAbON/Meqfrepnmy/lNOOrfopphmw7ltCysYP5HzQHYVJbDC3/pyAXX7BzwFi9owpEDygA4+qQNvP2PNnVa5mzWofNW+p+2nglPttuedsbwUp4Y1QkPgwbrVuUDcMpZa/jX+EJWLinYKf3Qozey9IsCln/ZhPJtObz5chuOH7yuju+k/iVGn5NZGqJ01oH7AyXuvsDdtwJPA8PSeL30M+fG8w9m5OBejH+8/fbkV/7WkctPO5Q//KwrG9ZGD7X2OHwz014rpKIcln9ZQPGs5qxcGv3jGnvn/nz38pU0aeY7nb5br828M7EQgKmvttmeX/bd5b9aykO/7oxX7viH2rnbVv7Pt9dy74T5/PrxBRzQfQsART220LJNBXc+V8J9E+fzrbOj76q3338bK5cWbD++dFk+HTpvq9sbyRCVnpPU0hCls9RdgEWx7cUhbSdmdqmZzTCzGStXVey6O6P88aUS7n9tPrc/sYBxj3Tg42ktOGNEKX97Zw4PTJ5Hu07bGP2rAwAYfN4qOnTeypVDDuXBm7vQu18ZuTnw2SfNWPZFEwYO3b2G8Z9//JJXxrZn5OBebPo6h7wC3y2PpO64b61nbWkeJR833yk9v4mzdYtx1dBeTHiiHT//Y/TXNTfP6XnkJv77B9258YIeXHDNV3TpsaU+ip6REt9oSWZpiOp9oMXdRwOjAfr1aZrRUSBRK2jToZyBQ9bx6QfNtzd3AYZeuJqbh3cHIDcvqp0kXPMfPely8GZmvdOS+bOaM7x/byoqYG1p1Il/1/MlHNhzC799egEAiz9rwrtTWtfh3WWv3seWMWDQeo49bQ4FTZzmrSq49t6FlC7L5+3xUc38XxMK+fmoKCiuXJbP+jV5bNmUy5ZN8PG7LenRexOly/LpeMDW7eft0HkbpcsaX23egfIGWgtMRjrvbAnQNbZdFNIapM0bc9j4dc729Zn/bMVBh21m1Vc7/l/5nwmFHHTo5pDH2Lwxyj/zny3JzXO69drCf4xYxVMfzObR6XP4w0sldOmxhbueLwGiAAlQWQlP/qkTZ/xgVV3eYtb622878/1+vRlxXG9++5NufPR2S+68qhv/M7E1fQZG/b7fPL6MxQuaAPDOxEIOP7aMnFynSbNKDjt6I18WN2Heh83p0n0rnbpuIS+/kpOHrWXaa4X1eWv1Jpubz+msKb4H9DSz7kTB8DzggjReL63WrMzjVxdHtcCKcjjlrLUce8oG7rzqQD6b3Qwz6FS0lavvjGoba1flc9P5PbCcqC/q2nsX1niNN15qwyuPdABg4NB1DDpvdfpuSPj7fZ247r6FfOeSUjaV5XD3L6L/wxeVNGXGm63485R5eKUx8cl2LJzXDID7b+rCb55cQE4uvPZ0OxbOb3wjzzTgpnEyzD19LVYzOx24m+iRnDHufnt1+fv1aerTJ3WtLotkmMEHHFXfRZAUvOtTWO+r9ymitT1sPz91zNlJ5X1h4IMz3b3fvlyvrqW1T9HdxwPj03kNEal72VxTrPeBFhFpWDTJrIhIjGOUVzbMQZRkKCiKSMqy+TU/BUURSY1nd/M5e+vAIpIWiT7F2nqjxcxyzewDM3s1bHc3s3fNrMTM/m5mBSG9SdguCfsPip3jhpA+z8wGx9KHhLQSM7s+mfIoKIpIymr5Nb+fAnNj278DRrn7IcAa4OKQfjGwJqSPCvkIE82cBxwODAEeCIF2ryalUVAUkZQ4RkVlTlJLTcysCPh34KGwbcCpwHMhy1jgzLA+LGwT9p8W8g8Dnnb3Le7+OVBCNCHNXk1Ko6AoIimrxJJagA6JCV/Ccukup7obuBZITC7aHljr7uVhOz6RzPZJZsL+dSF/VZPPJDUpza400CIiKfHUBlpKq3qjxczOAFa4+0wzO7mWirfPFBRFJGVeO6PPA4Fvh9eBmwKtiWbqb2NmeaE2GJ9IJjHJzGIzywMKgVVUP/lMypPSqPksIimqnfkU3f0Gdy9y94OIBkped/cLgTeAxMvVI4CXw/q4sE3Y/7pHkzeMA84Lo9PdgZ7AdGKT0oQR7PNC3mqppigiKaulmmJVrgOeNrNfAx8AD4f0h4HHzKwEWE0U5HD32Wb2DDAHKAdGunsFgJldCUxix6Q0s2u6uIKiiKTEHSoqazcouvubwJthfQHRyPGueTYD51Rx/O3AbrNw7c2kNAqKIpIyveYnIhI4aW8+1ysFRRFJUXbPvK2gKCIpS+OE/fVOQVFEUqbms4hIEI0+Z+8jzgqKIpIyNZ9FRGLUfBYRCRxTUBQRicvi1rOCooikyMFr+TW/TKKgKCIpU/NZRCSmUY4+m9m9VNN14O5Xp6VEIpLRGvO7zzPqrBQi0nA40BiDoruPjW+bWXN335j+IolIpsvm5nON7+qY2fFmNgf4NGz3MbMH0l4yEclQhlcmtzREybzAeDcwmOgDMbj7R8BJaSyTiGQ6T3JpgJIafXb3RdE3p7erSE9xRCTjeeMdaElYZGYnAG5m+cBPgbnpLZaIZLQGWgtMRjLN58uBkUAXYClwVNgWkUbLklwanhpriu5eClxYB2URkYaisr4LkD7JjD73MLNXzGylma0ws5fNrEddFE5EMlDiOcVklgYomebzk8AzQGfgAOBZ4Kl0FkpEMpt7cktDlExQbO7uj7l7eVgeB5qmu2AiksEa4yM5ZtYurE4ws+uBp4lu81xgfB2UTUQyVQNtGiejuoGWmURBMHH3l8X2OXBDugolIpnNGmgtMBnVvfvcvS4LIiINhBs00Ff4kpHUGy1mdgTQm1hfors/mq5CiUiGa4w1xQQzuwU4mSgojgeGAm8DCooijVUWB8VkRp/PBk4Dlrv7RUAfoDCtpRKRzNYYR59jNrl7pZmVm1lrYAXQNc3lEpFM1VgnmY2ZYWZtgL8SjUh/DbyTzkKJSGbL5tHnGpvP7n6Fu6919z8D/xcYEZrRItJY1ULz2cyamtl0M/vIzGab2a9Cencze9fMSszs72ZWENKbhO2SsP+g2LluCOnzzGxwLH1ISCsJz1vXqMqgaGZ9d12AdkBeWBeRRso8uaUGW4BT3b0P0exbQ8xsAPA7YJS7HwKsAS4O+S8G1oT0USEfZtYbOA84HBgCPGBmuWaWC9xPNDjcGzg/5K1Wdc3nP1Szz4FTazp5qoo/acnQQ0+s7dNKGq0dfkR9F0FSUPHqtNo5US30Kbq7E3XHAeSHJRFbLgjpY4FfAg8Cw8I6wHPAfRbNfj0MeNrdtwCfm1kJ0D/kK3H3BQBm9nTIO6e6clX38PYpyd+eiDQaqY0sdzCz+JdBR7v76MRGqM3NBA4hqtV9Bqx19/KQZTHRXK6En4sA3L3czNYB7UN6PNrHj1m0S/pxNRU4qYe3RUR2knxQLHX3flWexr0COCoM5r4IHLbPZdtHCooikjKr5Ulm3X2tmb0BHA+0MbO8UFssApaEbEuIHgdcbGZ5RM9Lr4qlJ8SPqSq9Ssk8vC0isrPaGX3uGGqImFkzoqdb5gJvEL00AjACeDmsjwvbhP2vh37JccB5YXS6O9ATmA68B/QMo9kFRIMx42q6tWRe8zOizxH0cPdbzexAYH93n17TsSKSfZIcWU5GZ2Bs6FfMAZ5x91fDd+afNrNfAx8AD4f8DwOPhYGU1URBDnefbWbPEA2glAMjQ7McM7sSmATkAmPcfXZNhUqm+fwA0RcZTgVuBTYAzwPHJnXbIpJ9amf0eRZw9B7SF7Bj9Dievhk4p4pz3Q7cvof08aQ4/2syQfE4d+9rZh+Ei6xJPEwpIo1UFr/RkkxQ3Baqtw5RPwBZ/S0vEalJNr/ml0xQvIdoqHw/M7udqIPzv9JaKhHJXF77o8+ZJJnvPj9hZjOJpg8z4Ex3n5v2kolI5mrMNcUw2rwReCWe5u5fprNgIpLBGnNQBP7Bjg9YNQW6A/OIXr4WkUaoUfcpuvuR8e0wQ84VaSuRiEg9Svk1P3d/38xqfKlaRLJYY64pmtl/xjZzgL7A0rSVSEQyW2MffQZaxdbLifoYn09PcUSkQWisNcXw0HYrd/9FHZVHRDKc0UgHWhJT95jZwLoskIg0AI0xKBJNvdMX+NDMxgHPAmWJne7+QprLJiKZqPZmyclIyfQpNiWayPFUdjyv6ICCokhj1UgHWvYLI8+fsCMYJmTx/xMiUpPGWlPMBVqyczBMyOI/EhGpURZHgOqC4jJ3v7XOSiIiDUNqX/NrcKoLivs+ta6IZKXG2nw+rc5KISINS2MMiu6+ui4LIiINR2N/zU9EZIdG3KcoIrIbI7sHHBQURSR1qimKiOzQWEefRUT2TEFRRCTQJLMiIrtQTVFEZAf1KYqIxCkoiojsoJqiiEiC02gnmRUR2U2j/XCViEiVFBRFRHYwz96omFPfBRCRBsZTWKphZl3N7A0zm2Nms83spyG9nZlNNrPi8LNtSDczu8fMSsxslpn1jZ1rRMhfbGYjYunHmNnH4Zh7zKzGuSwUFEUkZebJLTUoB37u7r2BAcBIM+sNXA9McfeewJSwDTAU6BmWS4EHIQqiwC3AcUB/4JZEIA15LokdN6SmQikoikjKrDK5pTruvszd3w/rG4C5QBdgGDA2ZBsLnBnWhwGPemQa0MbMOgODgcnuvtrd1wCTgSFhX2t3n+buDjwaO1eV1KcoIqlLvkuxg5nNiG2PdvfRu2Yys4OAo4F3gU7uvizsWg50CutdgEWxwxaHtOrSF+8hvVoKiiKSmuSaxgml7t6vugxm1hJ4HrjG3dfHu/3c3c3q9gEgNZ9FJHW1MNACYGb5RAHxCXd/ISR/FZq+hJ8rQvoSoGvs8KKQVl160R7Sq6WgKCIpSTy8va8DLWEk+GFgrrv/MbZrHJAYQR4BvBxLHx5GoQcA60IzexIwyMzahgGWQcCksG+9mQ0I1xoeO1eV1HwWkZRZZa20aAcCPwA+NrMPQ9qNwB3AM2Z2MbAQ+F7YNx44HSgBNgIXQfTlUTO7DXgv5Ls19jXSK4BHgGbAhLBUS0FRRFJTS1/zc/e3qfobWLt9dz6MII+s4lxjgDF7SJ8BHJFKuRQU91KLVuVc8+tiuvXaiDuMurEnn37YGoDvXLSYS67/gnMHHMf6Nfm0bF3Oz34zn84HbmbrlhxG3diThcUtyC+o5K4nZpFfUEluLrw9qT2P39utnu8sO+xX+DW//N7rtGu5CQdemv4N/v6vb+7TOU/vO48fnToTgDGvH8P49w/daf9dwyfQpd16Lrj73H26TkOgmbf3gpmNAc4AVrh7SpG6Ibj8pgXMmNqW23/6DfLyK2nSNPpb0mH/LfQduJavljTZnvfcyxfx2dyW3HZlb4p6bGTkzZ9xww+PZNtW4/oRR7J5Yy65eZX8/slZzHirLZ9+1Lq+bitrVFQaf/rH8cxb2pHmBVsZe9XzTC8u4vMV7Wo89oFLX+a2Z09h2Zodv4fWzTbz49Nm8MP7votjjL3yOabOPYgNm6Lf88mHL2DT1vy03U/Gyd63/NI60PIISTw93hA1b1nOEceuY9Jz0eNT5dtyKNsQ/f9y2Q0LePiug3b6S3PgwRv5aFohAIsXNKdTly20ab8VMDZvzAUgL8/Jy3Pcs/mLunVn1YYWzFvaEYCNWwv4YmVbOrYuo0u7ddx90T8Ye+Vz/OWyl+jWcU1S5xvQaxHTS4pYv6kpGzY1YXpJEcf3+hKAZgXbuODEWfzt9b41nCV71NIbLRkpbTVFd38rPJCZdfYv2sy61fn852+L6XFYGcWzW/Ln23tw9AlrKV1RwOfzWu6Uf8GnLRg4aBWzZxbS68gN7HfAZjrsv5W1qwrIyXHueeFDDjhwE68+2Zl5s1rV011lr85t19PrgFJmL+rE734wid+9eCKLVrXh8K5fce2ZUxn512/XeI6Orcv4at2O3+uKdS3p2LoMgMsGTeeJqX3YvK2R9EY5kMUTQtT7b9HMLiV6j5Gm1qKeS5Oc3DznkN5f8+BtBzNvVisuu+kzvn/VlxzRbx03/Wj3noJnRxdx2U0LuO+lD/hifnM+m9uSyopoX2WlceWZR9OiVTn/ff9cuvUsY2Fxw/hzaAiaFWzjjgtfY9QrJ1DpxpHdlvObCydv35+fF/0izjjmU84d+DEARe3XMeqHE9hWkcPSNa247rGqGzw9O5fSpd167n51IJ3brk/vzWQQ9SmmUXjlZzRAYW6HBvHfT+nyJpQub7K9Vvf2xA58/6ov2b9oCw+8/AEQ9S3e+8KHXHNOH9aUFjDqxl7haOeRKTNYvqjpTucs25DHrHcL6XfiGgXFWpKbU8Ed35/ExA978ubsHrRospWvNzXhB/ecs1veV2cexqszDwP23Ke4cn0L+vZYun17v8KveX/BARx54Fd8o2glL173OHk5TtsWm3jg0pe5YvSw9N9gPdEks7KbNaUFrFzehC7dN7Lk8+YcdfxaSua05IYfHrk9zyNT3uPqs49i/Zp8WrQqZ8vmHMq35TDknK/4eEZrNpblUdh2G+XlRtmGPAqaVHD0CWt59q9F1VxZkuf819n/5IsVbXnq7T4AlG0pYOmaVpx65Ge8/vHBgNOz8yqKl3Wo8WzT5nflJ4On06rZFgCO67mYByYex/pNTXnh3cOBqJn+hxETsjogAlHTWc1n2dWDt/Xg2t/PJz+/kmWLmjLqhl5V5u168EZ+fsd8wFhY3Jy7b+oJQNv9tvKLO+aTk+uYwdSJHZj+Zs2jo1KzPt2Wc3rf+RQva8djVz8LwIOT+nPz06dx3ZlT+dGp75OXU8nkWQcnFRTXb2rKmNeP4W8jnwfg4SnHsH5T0xqOyl7ZXFM0T1PEN7OngJOBDsBXwC3u/nB1xxTmdvABLWvu9JbMsfqsrHvaKqvNeXUUZaWL9ukRh1Ztivzok36aVN6pr1w7s6YJITJNOkefz0/XuUWkfmVzTVHNZxFJjQMV2RsVFRRFJGWqKYqIxGn0WURkB9UURUQSamnqsEyloCgiKTHANNAiIrKDqU9RRCRQ81lEJE7vPouI7ESjzyIicaopiogErtFnEZGdZW9MVFAUkdTpkRwRkTgFRRGRwAF9uEpEJGK4ms8iIjupzN6qooKiiKRGzWcRkZ2p+SwiEqegKCKSoAkhRER2yPKv+eXUdwFEpOEx96SWGs9jNsbMVpjZJ7G0dmY22cyKw8+2Id3M7B4zKzGzWWbWN3bMiJC/2MxGxNKPMbOPwzH3mJnVVCYFRRFJnXtyS80eAYbsknY9MMXdewJTwjbAUKBnWC4FHoQoiAK3AMcB/YFbEoE05Lkkdtyu19qNgqKIpMaBSk9uqelU7m8Bq3dJHgaMDetjgTNj6Y96ZBrQxsw6A4OBye6+2t3XAJOBIWFfa3ef5u4OPBo7V5XUpygiKUppoKWDmc2IbY9299E1HNPJ3ZeF9eVAp7DeBVgUy7c4pFWXvngP6dVSUBSR1CUfFEvdvd/eX8bdrG7n+VbzWURS40BFZXLL3vkqNH0JP1eE9CVA11i+opBWXXrRHtKrpaAoIily8Mrklr0zDkiMII8AXo6lDw+j0AOAdaGZPQkYZGZtwwDLIGBS2LfezAaEUefhsXNVSc1nEUldLT28bWZPAScT9T0uJhpFvgN4xswuBhYC3wvZxwOnAyXARuCiqCi+2sxuA94L+W5198TgzRVEI9zNgAlhqZaCooikJjH6XBuncj+/il2n7SGvAyOrOM8YYMwe0mcAR6RSJgVFEUmdXvMTEYlRUBQRCdyhoqK+S5E2CooikjrVFEVEYhQURUQSknuvuaFSUBSR1Dj43j+YnfEUFEUkdXv/Cl/GU1AUkdS46xOnIiI70UCLiMgOrpqiiEiCvuYnIrJDLU4IkYkUFEUkJQ64XvMTEQnc92UC2YynoCgiKXM1n0VEYrK4pmieQaNIZraSaPrxbNMBKK3vQkhKsvV31s3dO+7LCcxsItGfTzJK3b3GD9BnkowKitnKzGbsy2cepe7pd9Z46Wt+IiIxCooiIjEKinVjdH0XQFKm31kjpT5FEZEY1RRFRGIUFEVEYhQU08jMhpjZPDMrMbPr67s8UjMzG2NmK8zsk/oui9QPBcU0MbNc4H5gKNAbON/MetdvqSQJjwAN6mFjqV0KiunTHyhx9wXuvhV4GhhWz2WSGrj7W8Dq+i6H1B8FxfTpAiyKbS8OaSKSwRQURURiFBTTZwnQNbZdFNJEJIMpKKbPe0BPM+tuZgXAecC4ei6TiNRAQTFN3L0cuBKYBMwFnnH32fVbKqmJmT0FvAMcamaLzezi+i6T1C295iciEqOaoohIjIKiiEiMgqKISIyCoohIjIKiiEiMgmIDYmYVZvahmX1iZs+aWfN9ONcjZnZ2WH+ouskqzOxkMzthL67xhZnt9tW3qtJ3yfN1itf6pZn9ItUyiuxKQbFh2eTuR7n7EcBW4PL4TjPbq+94u/uP3X1ONVlOBlIOiiINkYJiwzUVOCTU4qaa2ThgjpnlmtldZvaemc0ys8sALHJfmN/x/wP7JU5kZm+aWb+wPsTM3jezj8xsipkdRBR8fxZqqSeaWUczez5c4z0zGxiObW9mr5nZbDN7CLCabsLMXjKzmeGYS3fZNyqkTzGzjiHtYDObGI6ZamaH1cqfpkiwVzULqV+hRjgUmBiS+gJHuPvnIbCsc/djzawJ8C8zew04GjiUaG7HTsAcYMwu5+0I/BU4KZyrnbuvNrM/A1+7++9DvieBUe7+tpkdSPTWzjeAW4C33f1WM/t3IJm3QX4UrtEMeM/Mnnf3VUALYIa7/8zMbg7nvpLog1KXu3uxmR0HPACcuhd/jCJ7pKDYsDQzsw/D+lTgYaJm7XR3/zykDwK+megvBAqBnsBJwFPuXgEsNbPX93D+AcBbiXO5e1XzCn4L6G22vSLY2sxahmt8Jxz7DzNbk8Q9XW1mZ4X1rqGsq4BK4O8h/XHghXCNE4BnY9duksQ1RJKmoNiwbHL3o+IJITiUxZOAq9x90i75Tq/FcuQAA9x98x7KkjQzO5kowB7v7hvN7E2gaRXZPVx37a5/BiK1SX2K2WcS8BMzywcws15m1gJ4Czg39Dl2Bk7Zw7HTgJPMrHs4tl1I3wC0iuV7DbgqsWFmR4XVt4ALQtpQoG0NZS0E1oSAeBhRTTUhB0jUdi8gapavBz43s3PCNczM+tRwDZGUKChmn4eI+gvfDx9f+gtRi+BFoDjse5RoJpiduPtK4FKipupH7Gi+vgKclRhoAa4G+oWBnDnsGAX/FVFQnU3UjP6yhrJOBPLMbC5wB1FQTigD+od7OBW4NaRfCFwcyjcbfeJBaplmyRERiVFNUUQkRkFRRCRGQVFEJEZBUUQkRkFRRCRGQVFEJEZBUUQk5n8BmNfKkxChUm0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(classification_report(y_test, test_preds))\n",
    "\n",
    "results_dict[\"Logistic Regression with Count Vectorization\"] = classification_report(y_test, test_preds)\n",
    "\n",
    "\n",
    "print(f\"Confusion matrix values: \\n{confusion_matrix(y_test, test_preds)}\")\n",
    "ConfusionMatrixDisplay.from_predictions(y_test, test_preds)\n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93fe550a",
   "metadata": {},
   "source": [
    "##### Summary"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b676c3a6",
   "metadata": {},
   "source": [
    "Our first Logistic regression model has fairly good metrics on our majority class (0) but worse on the minority class (1) that we really care about predicting. \n",
    "\n",
    "Recall  =  True Positives / (True Positives + False Negatives)\n",
    "\n",
    "Our recall is only .76 on the test set for the positive class. We are predicting 20234 of our positives data points correctly vs 6493 of them incorrectly. We'll try using TF-IDF vectorization instead and see if that impoves our results. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc4c7f6a-c85c-4e96-994c-5ff8a4267474",
   "metadata": {},
   "source": [
    "### 3.5.2 Logistic Regression with Term-Frequency Inverse Document Frequency Vectorization <a name=\"3.5.2\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3d219e8",
   "metadata": {},
   "source": [
    "#### 3.5.2.1 Training and Fitting the Model <a name=\"3.5.2.1\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "86d3b3cf-7690-4786-8c9a-06f27f00d583",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(786016, 2358146) (87336,)\n",
      "(786016,) (87336,)\n"
     ]
    }
   ],
   "source": [
    "#Trying a logistic model with Tfidf Vectorization\n",
    "\n",
    "#instantiate tfidf using a max_df to eliminate the most common words (uni-gram and bi-grams) in 90%+ of reviews\n",
    "tfidf = TfidfVectorizer(ngram_range=(1,2), max_df=0.9)\n",
    "\n",
    "#fit the vectorizer on the training data\n",
    "tfidf.fit(X_train)\n",
    "\n",
    "#transform both the training and test data\n",
    "X_train_vec = tfidf.transform(X_train)\n",
    "X_test_vec = tfidf.transform(X_test)\n",
    "\n",
    "print(X_train_vec.shape, X_test.shape)\n",
    "print(y_train.shape, y_test.shape)\n",
    "\n",
    "#using the default solver\n",
    "lr = LogisticRegression(solver = \"lbfgs\", n_jobs=-1, max_iter=1000)\n",
    "\n",
    "#train the lr model with the training data\n",
    "lr.fit(X_train_vec, y_train)\n",
    "\n",
    "#predict on the training data\n",
    "train_preds = lr.predict(X_train_vec)\n",
    "\n",
    "#predict on the test data\n",
    "test_preds = lr.predict(X_test_vec)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b022230f",
   "metadata": {},
   "source": [
    "Note the MUCH larger size of the sparse matrix created by the TFIDF model. Because of the additional size added by using bigrams, we have over 2 million column features instead of the 75,000 created by the Count Vectorizor. This proved to be a problem for some sklearn solvers (which did not converge) as well as the random forest model (out-of-memory errors). The small Count Vectorized data worked better for those models. A possible solution would be to switch to a local pyspark cluster to increase the memory available."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "600519a6-527c-4aae-82fb-a604fb9007ca",
   "metadata": {},
   "source": [
    "##### Tfidf-Logistic Regression Training set results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "fb04d17c-858a-4150-bdd8-81a04511e5ad",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.93      0.96      0.95    546424\n",
      "           1       0.90      0.84      0.87    239592\n",
      "\n",
      "    accuracy                           0.92    786016\n",
      "   macro avg       0.92      0.90      0.91    786016\n",
      "weighted avg       0.92      0.92      0.92    786016\n",
      "\n",
      "Confusion matrix values: \n",
      "[[523902  22522]\n",
      " [ 37289 202303]]\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_train, train_preds))\n",
    "\n",
    "print(f\"Confusion matrix values: \\n{confusion_matrix(y_train, train_preds)}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "595273a5-d6b0-43ef-9016-dc4f7b3026ef",
   "metadata": {},
   "source": [
    "##### Tfidf-Logistic Regression Test Set Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "593a75bc-b118-40e3-90b4-49a38dfcb643",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.92      0.94      0.93     60609\n",
      "           1       0.86      0.80      0.83     26727\n",
      "\n",
      "    accuracy                           0.90     87336\n",
      "   macro avg       0.89      0.87      0.88     87336\n",
      "weighted avg       0.90      0.90      0.90     87336\n",
      "\n",
      "Confusion matrix values: \n",
      "[[57111  3498]\n",
      " [ 5300 21427]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEGCAYAAADyuIefAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAfbElEQVR4nO3deZwV1Zn/8c/TTTf70ggim4JKVCSCioBhYnAFHCOaX2JcJvIzJMRxyzJJRjNjnGCccSaLieOSEEVR4xqjYqIiIi5RQUDcAAktqIAgtM1is3f3M3/UabqA7r63oG/f27e/79erXl116tSpU40+fU6dqlPm7oiISKQg2xUQEcklCooiIjEKiiIiMQqKIiIxCooiIjGtsl2BuG5dC71f36JsV0MS+Pvb7bJdBUlgG5vZ4dttf8oYfXJ7/7S8Kq2889/ePt3dx+zP+ZpaTgXFfn2LeH1632xXQxIY3WtItqsgCczxmftdRll5FXOm90krb1HP97vt9wmbWE4FRRFpDpwqr852JTJGQVFEEnGgmvx96UNBUUQSq0YtRRERABxnp7rPIiIRB6rUfRYRqaV7iiIigQNVeTy7loKiiCSWv3cUFRRFJCHHdU9RRKSGO+zM35iooCgiSRlV7Nfr0zlNQVFEEnGgWi1FEZFaaimKiATRw9sKiiIiQBQUd3r+zk+toCgiiThGVR5P2q+gKCKJVbu6zyIigO4piojswajSPUURkUg087aCoogIAO7GDi/MdjUyRkFRRBKr1j1FEZFINNCi7rOISJDfAy35e2UikhE1Ay3pLKmY2Qdm9o6ZvWlm80JaVzObYWZLw8+SkG5mdrOZlZrZ22Z2XKyc8SH/UjMbH0s/PpRfGo5N2e9XUBSRxKrc0lrSdLK7D3H3oWH7amCmuw8AZoZtgLHAgLBMBG6HKIgC1wHDgWHAdTWBNOT5duy4Makqo6AoIok4xk5vldayj8YBU8P6VOCcWPo9HpkNdDGznsBoYIa7l7v7emAGMCbs6+Tus93dgXtiZdVLQVFEEqkZaElnAbqZ2bzYMrGO4p41s/mxfT3cfXVYXwP0COu9gRWxY1eGtIbSV9aR3iANtIhIIk6irnFZrFtcl39w91VmdiAww8ze2+1c7m5mTTqlrVqKIpJYYw20uPuq8HMt8BjRPcFPQteX8HNtyL4K6Bs7vE9Iayi9Tx3pDVJQFJFE3KHKC9JaGmJm7c2sY806cAbwLjANqBlBHg88EdanAReHUegRwMbQzZ4OnGFmJWGA5Qxgeti3ycxGhFHni2Nl1UvdZxFJJBpoaZTX/HoAj4WnZFoB97v7M2Y2F3jYzCYAHwLnhfxPAWcCpcAW4BIAdy83s+uBuSHfJHcvD+uXAXcDbYGnw9IgBUURSawx3mhx92XA4DrSPwVOrSPdgcvrKWsKMKWO9HnAoCT1UlAUkUQc0ySzIiJxevdZRCSIvvusoCgiEpg+RyAiUiP6xKkmmRURAaKZt9V9FhGJyef5FBUURSSRaD5F3VMUEQnye+ZtBUURSSR6JEctRRERoFHffc5JCooiklg604I1VwqKIpJINHWYus8iIrvonqKISBDNkqPus4gIUPOan4KiABcPG0jbDlUUFEBhK+eWZ/7ODd85hJXvtwFg86ZC2neq4vbnlrCpvJDrJ/bj72+24/TzyrniP2s/DXHXjQfx3CNdqdhYyBOl7+xKf2d2e373094sW9yWn9z+AV88a2OTX2O+Kmpdza/+XEpRsVPYynn5r12495cH7dr/z9evYvT55Zwz4PMAHNh7Bz/49Qo6H1DJZxsK+Z8rD6ZsdTEAE/79Y4afugkrgDde6sjt1/aCPH6YeW9qKe4zMxsD/BYoBO5w9xszeb6m8D+PlNL5gKpd2//2+w93rf/+Z71o3zHaV9zGGf+jNXywpA0fvNdmtzJGnL6Jsy8p45sjj9otvXvvnfzLbz7iT787MINX0DLt3G78+GuHsW1LIYWtnF8/Xsrc5zvy3hvtGXDMFjp0rtot/7d/+jHP/amE5x7pyuCRn3HJNWv4xVUHM3DoZo4+YTOXnnoEAL96vJRjTtzM2691yMZlZU0+v9GSsXBvZoXArcBYYCBwgZkNzNT5ss0dXprWhZPPWQ9Am3bVDBq+meLWe3+d8ajjt3BAj8q90g/qu4NDB26jIH//CGeRsW1L9GxdqyKnsMhxh4IC59vXfsydP++5W+5DPreNt16JAt1br3TgxNFRq90dils7rYqdotZOqyJn/bqW1eGqGX1OZ2mOMvm/3zCg1N2XufsO4EFgXAbPl3nm/OSCw7h89Od46r4Ddtv17pz2lHSvpPehO7JUOUmloMC5bcYSHnp7IQte6sCSBe05+5IyXnu2M+Vri3bLu2xRW0aOjQLhyLEbad+xmo4llSye3563Xu3AAwsW8sCChcx/oSMrStvUdbq8Vu0FaS3NUSb/xPUGVsS2VwLD98xkZhOBiQAH987tv7i/fryUbj13sqGsFVeffxh9D9/G50dsBmDW4yWMCq1EyU3V1cZlpx9B+05VXHfncgYNr+CLX97Aj/7f4XvlnTypF5ffsIrTv17OO7M7sO7jIqqrjF79ttP38G1cdHzU6fmvB5cxaFgF777ecrrP+f6NlqyHcnef7O5D3X1o9wNy+9Whbj13AtClWyUjx2zkvQXtAKiqhFee6syXzt6QxdpJujZvKuStVzsweGQFvfrt4K5XFzN1ziJat63mrlcWA1D+SRHXf6sfl59xBHffeNCu474wdiPvvdGebVsK2balkHmzOnLU0C3ZvJwm50ClF6S1NEeZrPUqoG9su09Ia5a2bSlgS0XBrvX5L3ak35HbAHjj5Y70PXw73XvtzGYVpQGdu1bSvlPNIFg1x51UQenb7bhgyNGMHz6Q8cMHsn1rAZeEwa9OXSsxi+4Hn3/lWp59qCsA61YVccyJFRQURqPYnx+xmY+Wts7ORWWRus/7Zi4wwMz6EwXD84ELM3i+jFq/rhU/m9AfiFqGJ5+7gRNO/gyAF5+ou+t88bCBbK4ooHKH8dr0zvznA+9zyOe2c8f1PZn1eAnbtxZw0fEDGXNBOd/44RqWvNmWSRP689mGQmbP6MQ9vzyIP7ywpEmvM1917bGTH/72IwoKoKAAXnqyM3Oe61Rv/mNOrOCb16zGHd6Z04Fbf9IbgJf/0oXBIyv4/fNLcId5szoyZ0bnprqM3OD53X226PvSGSrc7EzgN0SP5Exx9xsayj90cBt/fXrfhrJIjhnda0i2qyAJzPGZbPLy/YpoJUce6KdM+Wpaef888vb57j50f87X1DI6suHuTwFPZfIcItL08rmlmNvDvSKSczTJrIhIjGNUVjfPQZR0KCiKSGL5/JqfgqKIJOPqPouI7KJ7iiIie8jnoJi/d0tFJCMco6q6IK0lHWZWaGYLzOwvYbu/mc0xs1Ize8jMikN667BdGvb3i5VxTUhfYmajY+ljQlqpmV2dTn0UFEUksWosrSVN3wUWx7b/G7jJ3Q8H1gMTQvoEYH1IvynkI0xJeD5wNDAGuC0E2n2avlBBUUQS8TDQks6Sipn1Af4RuCNsG3AK8KeQZSpwTlgfF7YJ+08N+ccBD7r7dndfDpQSTV24T9MXKiiKSGLultYCdDOzebFl4h5F/Qb4MVAdtg8ANrh7zSzMK4mmIYTYdIRh/8aQv65pCns3kN4gDbSISEKJJoQoq+/dZzM7C1jr7vPNbFQjVW6/KSiKSGLeOKPPI4Gzw8QxbYBORN906mJmrUJrMD7lYM10hCvNrBXQGfiUhqcpTDx9obrPIpKIO1RVW1pLw+X4Ne7ex937EQ2UPO/uFwGzgJppeMYDT4T1aWGbsP95j6b5mgacH0an+wMDgNeJTV8YRrDPD3kbpJaiiCSW4df8/hV40Mx+DiwA7gzpdwL3mlkpUE4U5HD3hWb2MLAIqAQud/cqADO7AphO7fSFC1OdXEFRRBJxGq37XFum+wvAC2F9GdHI8Z55tgFfq+f4G4C95mvdl+kLFRRFJKH8nnlbQVFEEsvghP1Zp6AoIok1dvc5lygoikgi0ehz/j64oqAoIomp+ywiEqPus4hI4JiCoohIXB73nhUURSQhB0/xCl9zpqAoIomp+ywiEtMiR5/N7H9p4NaBu1+VkRqJSE7LxLvPuaShluK8JquFiDQfDrTEoOjuU+PbZtbO3bdkvkoikuvyufuc8l0dMzvRzBYB74XtwWZ2W8ZrJiI5yvDq9JbmKJ0XGH8DjCaa9ht3fws4KYN1EpFc52kuzVBao8/uviL6kuAuVZmpjojkPG+5Ay01VpjZFwA3syL2/nC1iLQ0zbQVmI50us+XApcTfS/1Y2BI2BaRFsvSXJqflC1Fdy8DLmqCuohIc1GdOktzlc7o86Fm9qSZrTOztWb2hJkd2hSVE5EcVPOcYjpLM5RO9/l+4GGgJ9ALeAR4IJOVEpHc5p7e0hylExTbufu97l4ZlvuANpmumIjksJb4SI6ZdQ2rT5vZ1cCDRJf5dRJ+R1VE8kwz7Rqno6GBlvlEQbDm6r8T2+fANZmqlIjkNmumrcB0NPTuc/+mrIiINBNu0Exf4UtHWm+0mNkgYCCxe4nufk+mKiUiOa4lthRrmNl1wCiioPgUMBb4G6CgKNJS5XFQTGf0+avAqcAad78EGAx0zmitRCS3tcTR55it7l5tZpVm1glYC/TNcL1EJFe11ElmY+aZWRfgD0Qj0hXAa5mslIjktnwefU7ZfXb3y9x9g7v/DjgdGB+60SLSUjVC99nM2pjZ62b2lpktNLOfhfT+ZjbHzErN7CEzKw7prcN2adjfL1bWNSF9iZmNjqWPCWml4XnrlOoNimZ23J4L0BVoFdZFpIUyT29JYTtwirsPJpp9a4yZjQD+G7jJ3Q8H1gMTQv4JwPqQflPIh5kNBM4HjgbGALeZWaGZFQK3Eg0ODwQuCHkb1FD3+VcN7HPglFSFJ7X03Q6MHTCysYuVDCqbeEy2qyAJVD46u3EKaoR7iu7uRLfjAIrCUhNbLgzpU4H/AG4HxoV1gD8Bt1g0+/U44EF33w4sN7NSYFjIV+ruywDM7MGQd1FD9Wro4e2T0788EWkxko0sdzOz+JdBJ7v75JqN0JqbDxxO1Kp7H9jg7pUhy0qiuVwJP1cAuHulmW0EDgjp8WgfP2bFHunDU1U4rYe3RUR2k35QLHP3ofUW414FDAmDuY8BR+533faTgqKIJGaNPMmsu28ws1nAiUAXM2sVWot9gFUh2yqixwFXmlkrouelP42l14gfU196vdJ5eFtEZHeNM/rcPbQQMbO2RE+3LAZmEb00AjAeeCKsTwvbhP3Ph/uS04Dzw+h0f2AA8DowFxgQRrOLiQZjpqW6tHRe8zOizxEc6u6TzOxg4CB3fz3VsSKSf9IcWU5HT2BquK9YADzs7n8J35l/0Mx+DiwA7gz57wTuDQMp5URBDndfaGYPEw2gVAKXh245ZnYFMB0oBKa4+8JUlUqn+3wb0RcZTgEmAZ8BjwInpHXZIpJ/Gmf0+W3g2DrSl1E7ehxP3wZ8rZ6ybgBuqCP9KRLO/5pOUBzu7seZ2YJwkvU1D1OKSAuVx2+0pBMUd4bmrUN0H4C8/paXiKSSz6/5pRMUbyYaKj/QzG4gusH57xmtlYjkLm/80edcks53n/9oZvOJpg8z4Bx3X5zxmolI7mrJLcUw2rwFeDKe5u4fZbJiIpLDWnJQBP5K7Qes2gD9gSVEL1+LSAvUou8puvvn49thhpzLMlYjEZEsSvyan7u/YWYpX6oWkTzWkluKZvaD2GYBcBzwccZqJCK5raWPPgMdY+uVRPcYH81MdUSkWWipLcXw0HZHd/9hE9VHRHKc0UIHWmqm7jEzTYUtIrtriUGRaOqd44A3zWwa8AiwuWanu/85w3UTkVzUeLPk5KR07im2IZrI8RRqn1d0QEFRpKVqoQMtB4aR53epDYY18vjvhIik0lJbioVAB3YPhjXy+FciIinlcQRoKCiudvdJTVYTEWkekn3Nr9lpKCju/9S6IpKXWmr3+dQmq4WINC8tMSi6e3lTVkREmo+W/pqfiEitFnxPUURkL0Z+DzgoKIpIcmopiojUaqmjzyIidVNQFBEJNMmsiMge1FIUEamle4oiInEKiiIitdRSFBGp4bTYSWZFRPaS7x+uKsh2BUSkGfI0lwaYWV8zm2Vmi8xsoZl9N6R3NbMZZrY0/CwJ6WZmN5tZqZm9bWbHxcoaH/IvNbPxsfTjzeydcMzNZpbyDUUFRRFJzNzTWlKoBP7F3QcCI4DLzWwgcDUw090HADPDNsBYYEBYJgK3QxREgeuA4cAw4LqaQBryfDt23JhUlVJQFJFk0m0lpoiJ7r7a3d8I658Bi4HewDhgasg2FTgnrI8D7vHIbKCLmfUERgMz3L3c3dcDM4AxYV8nd5/t7g7cEyurXrqnKCKJJbin2M3M5sW2J7v75L3KM+sHHAvMAXq4++qwaw3QI6z3BlbEDlsZ0hpKX1lHeoMUFEUksQSv+ZW5+9AGyzLrADwKfM/dN8Vv+7m7mzXtsI66zyKSXCN0nwHMrIgoIP7R3Wu+Jf9J6PoSfq4N6auAvrHD+4S0htL71JHeIAVFEUnGo+5zOktDwkjwncBid/91bNc0oGYEeTzwRCz94jAKPQLYGLrZ04EzzKwkDLCcAUwP+zaZ2YhwrotjZdVL3WcRSa5xOrQjgW8A75jZmyHtJ8CNwMNmNgH4EDgv7HsKOBMoBbYAl0D0PSkzux6YG/JNin1j6jLgbqAt8HRYGqSgKCKJNNbD2+7+N+r/ssFeXxMNI8iX11PWFGBKHenzgEFJ6qWgKCKJWXX+vtKioCgiyehrflKXu2fNZ8vmQqqroarS+O5XBvON733EiaeWU+2w8dMifvWvAyhfWww4l167nBO+tIHtWwv41b8ezvuLOgBw2rlrOf+y6FGqB2/rw3OPHZjFq8ovPTpVMOnc5+naYSvu8Nj8o3hgzjGcNvB9Jo6aR//u67n4D19h8ce7/84P6vwZj1z+EJNfGMq9rw6ptxyA//rqDA7ptgGAjm2289m21lz4u6819aU2Oc28vQ/MbApwFrDW3RP16ZuLq79xNJvWF+3afvSOXtz7m4MBOPvi1Vx4xQpu+elhnPClDfQ6ZBsTTjuWI4dUcMWkZXz/q8fQofNOLrxyBVedewy4cfPjbzF7ZlcqNulvVWOoqjZuevZE3lvdnXbFO7jvO48ye1kfStd25UcPjeYnX36xzuO+P/o1Xl16cMpylq/ryjV/Or32uDNepWJ7ccavKyfkcUsxk4/k3E0a7xnmky0VtcGsTduqXf/hjDitnJmPdweM997sSIeOlZR038HxX9zAgle6ULGxiIpNrVjwSheOP2l9diqfh8oq2vPe6u4AbNlRzPJ1JRzYcTMflJXw4add6jxm1JHL+Xh9R95fV5KynN05px39Ps+8c3gmLiXnNMYjObkqY0HR3V8CylNmbKbc4Ya7FnHzY28x9utrdqWP//6H3PPSPE4+ex33/jZqbRzQYwdlq1vvylO2pjXdeuygW48drFtdHEsvpluPHU13ES1Izy6bOLJnGe+u6lFvnrbFOxk/8k0mv1j/Cxj1lXPsIasp39yOFeVdGqvKucuJ/gdIZ2mGsv7wtplNNLN5ZjZvh2/LdnXS9sMLBnHlOYO5dsJRnHXRGgadsBGAqTcdwsUnDWXWtO58+Z9WpyhFmkLb4p384rxn+eUzX2BzA93b74yax/2zP8/WHUV17m+onDGDSpneQlqJEN1TTGdpjrIeFN19srsPdfehxdYm29VJ26efRC2/jeXFvDqjK0ccU7Hb/lnTujNy9KchbzHdem7fta/bQdsp+6SYsk+K6d5zRyx9B2WftJB7Uk2kVUEVvzhvOk+/M4BZiw9tMO+g3p9w1emzefJ793HhiHe45IsLOG/YuynLKSyo5uSjlvPswsMydh25pOY5xXztPuuO/j5o3baKggLYurmQ1m2rOO4fNnL/LX3odchWPv6wLQAnnlbOymXR+uyZJXz5n9bw4l+6ceSQCjZ/1or164qZ/3IX/v8PPqJDp0oAjhu5gbt+eXC955WknGvHvcjyshL++NrglLm/ddc5u9YnjprL1h1FPPz6oJTlDDt0JR+UdWHtpg6NWPcc1oy7xulQUNwHJd12cu2t7wFQ2Mp54cnuzH+5hH+75T369N+KVxtrP27N//40alHMfaGEE760gSkz32Db1kJuujrqZlVsLOKB2/rw2z+/DcD9t/ahYmPdXTdJbsjBazhr8N9Z+klX7r/0EQBunTmM4sJqfnTm3yhpt5XfXvg0f19zAFfcd1bicl5ZeggAoweVMv3dltN1hubbCkyHeYYivpk9AIwCugGfANe5+50NHdO5sJuPaFf/f5ySe9ZedEy2qyAJLHn0JrasW5FySv6GdOzSx4896btp5X35yR/PTzV1WK7JWEvR3S/IVNkikl353FJU91lEknGgKn+jooKiiCSmlqKISJxGn0VEaqmlKCJSQ1OHiYjUMsA00CIiUst0T1FEJFD3WUQkTu8+i4jsRqPPIiJxaimKiASu0WcRkd3lb0xUUBSR5PRIjohInIKiiEjgQDP9KFU6FBRFJBHD1X0WEdlNdf42FRUURSSZPO8+Z/27zyLS/Jh7WkvKcsymmNlaM3s3ltbVzGaY2dLwsySkm5ndbGalZva2mR0XO2Z8yL/UzMbH0o83s3fCMTebWcqPdikoikhyNd9+TrWkdjcwZo+0q4GZ7j4AmBm2AcYCA8IyEbgdoiAKXAcMB4YB19UE0pDn27Hj9jzXXhQURSShNANiGkHR3V8CyvdIHgdMDetTgXNi6fd4ZDbQxcx6AqOBGe5e7u7rgRnAmLCvk7vP9uhbzvfEyqqX7imKSDLJvubXzczmxbYnu/vkFMf0cPfVYX0N0COs9wZWxPKtDGkNpa+sI71BCooikliCR3LK3H3ovp7H3d2saefkUfdZRJJrvHuKdfkkdH0JP9eG9FVA31i+PiGtofQ+daQ3SEFRRJJxoNrTW/bNNKBmBHk88EQs/eIwCj0C2Bi62dOBM8ysJAywnAFMD/s2mdmIMOp8cayseqn7LCIJNd7M22b2ADCK6N7jSqJR5BuBh81sAvAhcF7I/hRwJlAKbAEuAXD3cjO7Hpgb8k1y95rBm8uIRrjbAk+HpUEKiiKSXCMFRXe/oJ5dp9aR14HL6ylnCjCljvR5wKAkdVJQFJFkHKjK31daFBRFJCEHV1AUEamlWXJERIKa0ec8paAoIsmppSgiEqOgKCISuENVVbZrkTEKiiKSnFqKIiIxCooiIjX2673mnKegKCLJOLge3hYRidFrfiIigbs+cSoishsNtIiI1HK1FEVEajTeJLO5SEFRRJLRhBAiIrUccL3mJyISuCaZFRHZjav7LCISk8ctRfMcGkUys3VEnzTMN92AsmxXQhLJ13+zQ9y9+/4UYGbPEP1+0lHm7mP253xNLaeCYr4ys3nuPjTb9ZD06d+s5SrIdgVERHKJgqKISIyCYtOYnO0KSGL6N2uhdE9RRCRGLUURkRgFRRGRGAXFDDKzMWa2xMxKzezqbNdHUjOzKWa21szezXZdJDsUFDPEzAqBW4GxwEDgAjMbmN1aSRruBprVw8bSuBQUM2cYUOruy9x9B/AgMC7LdZIU3P0loDzb9ZDsUVDMnN7Aitj2ypAmIjlMQVFEJEZBMXNWAX1j231CmojkMAXFzJkLDDCz/mZWDJwPTMtynUQkBQXFDHH3SuAKYDqwGHjY3Rdmt1aSipk9ALwGHGFmK81sQrbrJE1Lr/mJiMSopSgiEqOgKCISo6AoIhKjoCgiEqOgKCISo6DYjJhZlZm9aWbvmtkjZtZuP8q628y+GtbvaGiyCjMbZWZf2IdzfGBme331rb70PfJUJDzXf5jZD5PWUWRPCorNy1Z3H+Lug4AdwKXxnWa2T9/xdvdvufuiBrKMAhIHRZHmSEGx+XoZODy04l42s2nAIjMrNLNfmNlcM3vbzL4DYJFbwvyOzwEH1hRkZi+Y2dCwPsbM3jCzt8xsppn1Iwq+3w+t1C+aWXczezScY66ZjQzHHmBmz5rZQjO7A7BUF2Fmj5vZ/HDMxD323RTSZ5pZ95B2mJk9E4552cyObJTfpkiwTy0Lya7QIhwLPBOSjgMGufvyEFg2uvsJZtYaeMXMngWOBY4gmtuxB7AImLJHud2BPwAnhbK6unu5mf0OqHD3X4Z89wM3ufvfzOxgord2jgKuA/7m7pPM7B+BdN4G+WY4R1tgrpk96u6fAu2Bee7+fTP7aSj7CqIPSl3q7kvNbDhwG3DKPvwaReqkoNi8tDWzN8P6y8CdRN3a1919eUg/Azim5n4h0BkYAJwEPODuVcDHZvZ8HeWPAF6qKcvd65tX8DRgoNmuhmAnM+sQzvGVcOxfzWx9Gtd0lZmdG9b7hrp+ClQDD4X0+4A/h3N8AXgkdu7WaZxDJG0Kis3LVncfEk8IwWFzPAm40t2n75HvzEasRwEwwt231VGXtJnZKKIAe6K7bzGzF4A29WT3cN4Ne/4ORBqT7inmn+nAP5tZEYCZfc7M2gMvAV8P9xx7AifXcexs4CQz6x+O7RrSPwM6xvI9C1xZs2FmQ8LqS8CFIW0sUJKirp2B9SEgHknUUq1RANS0di8k6pZvApab2dfCOczMBqc4h0giCor55w6i+4VvhI8v/Z6oR/AYsDTsu4doJpjduPs6YCJRV/UtaruvTwLn1gy0AFcBQ8NAziJqR8F/RhRUFxJ1oz9KUddngFZmthi4kSgo19gMDAvXcAowKaRfBEwI9VuIPvEgjUyz5IiIxKilKCISo6AoIhKjoCgiEqOgKCISo6AoIhKjoCgiEqOgKCIS83+q+57v3EO3vAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(classification_report(y_test, test_preds))\n",
    "\n",
    "results_dict[\"Logistic Regression with TFIDF Vectorization\"] = classification_report(y_test, test_preds)\n",
    "\n",
    "\n",
    "print(f\"Confusion matrix values: \\n{confusion_matrix(y_test, test_preds)}\")\n",
    "ConfusionMatrixDisplay.from_predictions(y_test, test_preds)\n",
    "\n",
    "plt.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "161d66f1",
   "metadata": {},
   "source": [
    "#### Evaluating the Model <a name=\"3.5.2.2\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44618fdd",
   "metadata": {},
   "source": [
    "Logistic regression with TFIDF vectorization has better metrics than with Count Vectorization. Due to including bigrams (and possibly disallowing the most common words via max_df of .9) the TFIDF vectorization encodes more information about the data, but results in a much larger input matrix to the model.\n",
    "\n",
    "Recall  =  True Positives / (True Positives + False Negatives)\n",
    "\n",
    "Our recall is only .81 (vs .76 for Count Vec.) on the test set for the positive class. We are predicting 21427 of our positives data points correctly vs 5300 of them incorrectly. Though we don't care quite as much about the negative class, our metrics for it are also better than the LR model with Count Vectorization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "ae593f88",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<sklearn.metrics._plot.roc_curve.RocCurveDisplay at 0x7f23d3d9f640>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAA2GUlEQVR4nO3deXhU5dn48e89k8lKCLvKjghVAUGNIFXEFVFxpyJWqxW1vmrdrfTV16391dpaa7VUxaWiVtSqtIhUrS0q1iqLBGQRiYgQFgkBQkLWydy/P86ZYRKSzAnJZEjm/lxXrszZ7zOTnHue5znneURVMcYYk7x8iQ7AGGNMYlkiMMaYJGeJwBhjkpwlAmOMSXKWCIwxJsmlJDqApurWrZv2798/0WEYY0ybsnjx4m2q2r2+ZW0uEfTv359FixYlOgxjjGlTROTbhpZZ1ZAxxiQ5SwTGGJPkLBEYY0ySs0RgjDFJzhKBMcYkubglAhF5TkS2isjyBpaLiDwmIvkiskxEjopXLMYYYxoWzxLB88D4RpafAQxyf64BnohjLMYYYxoQt+cIVPUjEenfyCrnAi+o0w/2pyLSSUQOUtXN8YrJGLP/UlVUQYGQKqHwtO6ZDinUhNRZ110PhZCC4izXOtuF13N63A+vA5XBGnwi7jH3HFvd41Br3p59KLWPGb1d9L52VVSTEUiJHDMcf3i98HRIlapgiMpgiJpQKDJPFUIhjZpWTjnsAIb36dTi730iHyjrBWyImi5w5+2VCETkGpxSA3379m2V4IzZ31QFQxSXV1NcXs2uimqqgyGCISUYUmpCIYI1ys7yavwiAARD7vIaZ52qYIiqYIjqmhDV7vobtpfRtUMq1TVKsGbP+t9uL6N7dhqhkBIMhQiFoEad/YRCSo07PxhyLlg1oT0X7pCqO+1c9GrU2abuxa/+C7VpiAj06Jje7hKBZ6o6HZgOkJuba38uptUEa0LsLK9mx+4qSiqDkQtpVTBEUWkVaQEf1TUamV9eXUNldYjKYA3fbi+jc2aAmpDWutBW14Sorgn/dl5/W7Sbbh3SCGn4wr7nJxhSdlcGKauqaZFzEoGA30fAJ/h8QklFkJ456aT4faT4hRSf4Pf5WLV5F327ZOIXwe8TUn0+fD5nuU/c9fyCXwSfgE+c/dX72v0RwZnn7kMILweJLK9/2ucmuB1lVfTITscnzsn4BITwNu527n7D27ub1p4HlFYG6ZSZirurWtvv+b1nfWody9nXnm333i4YUjIC/pjnlJbiIy3gJ+CXyLzodcL7j5dEJoKNQJ+o6d7uPGP2SU1IKa0IsrsqyPbdVewoq2JbaSUlFUHWbSsjK81PeVUN5dU1lFYGIxf0ymCIiuoayqpqKK9yfpdVBamoDlFVE9qnWAJ+QRCqakL0yE4jxSfOhdYnpPiFgN9HaoqPgM9HesDHkJ45bCutZECXLPw+2fPjXoSz0lLolBEgJzNATkaAjukB0lJ8+OvsN8XnNPtlBPz4/ULAXe73CanuMf2++F1QTNuUyEQwG7hBRF4BRgHF1j5gwqprQuwsq2ZzcTm7K2vYUVbFpp3lFOwoZ1tpJWVVNRSXV1NUWsmuiiDlVTVUBGtiVi9kpfrJSPWTmZpCWoovckHOCPg5sGOAjFQ/GQE/mal+MlJTyAj46ZQZoHNWKtnpKaT5fQRSfKT6fSiQnZ4SucAG/M5FPS3Fbxdb06bELRGIyEzgRKCbiBQA9wIBAFV9EpgLnAnkA2XAj+MVi0msUEgp2u1cyHdXBimtDLKzrJrC0kq2FFewo6yK/K2lpKb4KCqtYld5NSWVwXr3lZXqp0fHdLLS/GSnBRjaK4dOmQEyAs4FPCczlcxUPwADu3egW4dUOmYE6JDmXPjjWbw2pq2K511Dk2MsV+D6eB3ftA5V5yK/eWcF28uqWL+9jI07ylmyfgci8N2uSjZsLyMYqv+rek5GgK5ZqWSlpVAVDDFqQBdyMgN0ykilc1aArllpBPxCr84Z9O6USceMFLuYG9PC2kRjsUmsymANW3dV8m1RGV99V8LabaVs2llBYUklX2ws3mt9v0/o1SmDqmCIo/p1YtyQA+iWlUZORoAuWakcmJNOTkaAbh3SyHC/vRtjEscSgYnYVVHNyk27+O/XRRSXV7Nw3XZ27K5i866Keuvej+zbiR+N7kdORoAhPXPo2iGVXp0yOLBjOj6rIzemzbBEkKS2llSwYtMuVm7axYpNxSzfuIv128tqrXPogdkcckA2E4/uTc9OGfTtmslBORl07ZBKx/RAgiI3xrQ0SwTtXFUwRN6GnXyxsZhF67ZTXF5N/tZStpZURtbp2yWTob06MumYPhzesyNDDupI9+w0q4s3JklYImhHVJUVm3ax4JvtrNy8izVbS1m9ZRcV1c698NnpKWSnpXD8Id04vGdHhvbK4fCeHe3bvTFJzhJBGxYKKZ9+U8T8Ndv4oqCYFZuK2VFWDUDXrFQGdu/ApNw+HHtwV44Z0IWuWan2Ld8YsxdPiUBEfMBwoCdQDixX1a3xDMzUb3dlkPlrtvH+qu/4YPVWtpVWEfALgw/I5pTDDmDkgC6cMKg7B3S0qh1jjDeNJgIRGQjcCZwKrAEKgXRgsIiUAU8BM1R1357DN55s313Ff/K38Y/lm/nXqq1UBkN0TE9h7Pd6cOphPTj1sAPISrPCnTFm38S6evwSZ5yAn7gPgEWISA/gEuAyYEZ8wktu3+2q4HfvrWbWko1U1yidMgNclNuH04ccyKiDuxDw2wBzxpjmazQRNPZ0sFs19GhLB2RgZ1kVT320lmc//gYUJh7dhwuP6sXwPp3s4m+MaXH7XJ8gIqep6j9bMphkV1Fdw/OfrGPav/MpqQxy/pG9uPGUQQzolpXo0Iwx7VhzKpafBWyUmBYQCilzl2/ml3NWsWVXBccf0o2fn3koQ3rmJDo0Y0wSiNVYPLuhRUDXlg8n+Xz0VSH3v7WCrwt3c3D3LGZcOZKxg7snOixjTBKJVSIYA1wKlNaZL8DIuESUJHZXBvnl26uYuWA9A7tn8ftJwzn7iJ6kWBuAMaaVxUoEnwJlqvph3QUisjo+IbVvoZDy7y+3cv+cFWzYXs5lx/bjrrMOIz1gvXAaYxIj1l1DZzSy7ISWD6d9W7GpmKlvfMEXG4vp1SmDl6aM4vhB3RIdljEmydlTSK2guibEQ//4kmf/8w1dMlP5zcQjOP/IXnYrqDFmv2CJIM6WbtjJ7X9dypqtpVwyqi93nn4oOZnWyZsxZv9hiSCOPvl6G1c+v5DOmak8eelRjB96UKJDMsaYvVgiiJP5awqZMmMR/bpk8perR9EjOz3RIRljTL08V1KLyH2NTZs9Pl+/g6tfWET/rpm8+pPRlgSMMfu1prRWLo4xbYBVm3dxzQuL6J6dxstXH0uXrNREh2SMMY3ynAhU9a3Gpg389+siLp7+KT4R/nzFSLp1SEt0SMYYE1OsLiYeB7Sh5ap6Y4tH1EZ9uWUXl/95AX27ZPLs5bn062odxRlj2oZYjcWLWiWKNu67XRX8+M8L6ZiewivXHGslAWNMmxLryeJaA86ISKaqlsU3pLalorqGa19azK7yal79yWhLAsaYNsdTG4GIjBaRlcCX7vRwEflTXCNrI+7+23KWrN/JgxcewdBe1m20Mabt8dpY/ChwOlAEoKpLgaTva+ijrwp5fXEBPz6uP+cM75nocIwxZp805a6hDXVm1bRwLG3Kd7squPnVPHrmpHPbuO8lOhxjjNlnXp8s3iAi3wdURALATcCq+IW1fwuFlJ++vISyqiCvXnM8HdLsAW1jTNvltURwLXA90AvYBIxwp5PSS599y4J127nv7CEMOiA70eEYY0yzeEoEqrpNVX+oqgeoandVvVRVi2JtJyLjRWS1iOSLyNR6lvcVkXkiskRElonImftyEq2porqGx/6Vz7EHd2HSMX0SHY4xxjSb17uGDhaRt0SkUES2isjfReTgGNv4gWnAGcDhwGQRObzOancDr6nqkcDFwH5/J9Lj/17DttJKbjplMCKS6HCMMabZvFYNvQy8BhwE9AT+CsyMsc1IIF9V16pqFfAKcG6ddRTo6L7Owal22m+t2ryLpz5cy4VH9Wb0wK6JDscYY1qE10SQqaovqmrQ/XkJiNWlZi8g+k6jAndetPuAS0WkAJgL/LS+HYnINSKySEQWFRYWegy5ZYVCytQ3ltExI8DdZx2WkBiMMSYeGk0EItJFRLoA/xCRqSLSX0T6icjPcC7czTUZeF5VewNnAi+KyF4xqep0Vc1V1dzu3bu3wGGbbtaSjSwtKOaO079HZ+tR1BjTjsS673ExTvVNuDL8J1HLFPh5I9tuBKJbU3u786JNAcYDqOp/RSQd6AZsjRFXqyqrCvLwe6sZ2qsjk3KtgdgY077E6mtoQDP2vRAYJCIDcBLAxcAlddZZD5wCPC8ih+FUNyWm7qcRv3vvKzYXV/C7HwzH57MGYmNM++L5SSgRGYpz90+kbUBVX2hofVUNisgNwLuAH3hOVVeIyAPAIlWdDdwGPC0it+CUMK5Q1Qa7vU6E1VtKePbjb5g8si/fP6RbosMxxpgW5ykRiMi9wIk4iWAuzi2hHwMNJgIAVZ1LnbYEVb0n6vVK4LgmRdzKHv/3GgBuOmVQgiMxxpj48HrX0EScKpwtqvpjYDjO7Z7t2uJvdzBn2WZuPGUQB+bYuMPGmPbJayIoV9UQEBSRjjiNue2+1fSxf62hc2aAn5zQ6LNzxhjTpnltI1gkIp2Ap3HuJCoF/huvoPYHawtL+fCrQm48+RCyrFM5Y0w75ukKp6rXuS+fFJF3gI6quix+YSXe3/M2IQI/PLZfokMxxpi4ijV4/VGNLVPVz1s+pP3DO8u3cEy/LhzQ0doGjDHtW6wSwe8aWabAyS0Yy37jyy27WP1dCfdMqNtHnjHGtD+xHig7qbUC2Z+8t+I7RGDCEQclOhRjjIk7z0NVJpN/fbmVI3p3oodVCxljkoAlgjq2FFewdMNOTj20R6JDMcaYVmGJoI6XF6wH4EyrFjLGJAmvI5SJiFwqIve4031FZGR8Q0uMlZuK6ZiewsDuHRIdijHGtAqvJYI/AaNxxg8AKMEZhrJdCYWUvA3FnPg9qxYyxiQPr4/MjlLVo0RkCYCq7hCRdjc6y+rvSthWWsnx1suoMSaJeC0RVLuD0SuAiHQHQnGLKkEWf7sDgJEDuiQ4EmOMaT1eE8FjwCygh4j8P5wuqH8Vt6gS5JOvt9E9O41+XTMTHYoxxrQar30N/UVEFuN0RS3Aeaq6Kq6RJcCygmJGDuiCiI1CZoxJHl7vGnoM6KKq01T1j+0xCWzaWU7BjnKG9Wr3wywYY0wtXquGFgN3i8jXIvKwiOTGM6hE+PArZ6jkk+1BMmNMkvGUCFR1hqqeCRwDrAYeEpE1cY2slS38ZjvdOqQyqIc9P2CMSS5NfbL4EOBQoB/wZcuHkziffbOd3H7WPmCMST5e2wh+45YAHgCWA7mqenZcI2tFW4or2LiznGPstlFjTBLy+kDZ18BoVd0Wz2AS5YuNxQAc0dsaio0xySfWCGWHquqXwEKgr4j0jV7eXkYo+3z9DlJ8wtCelgiMMcknVongVuAa6h+prN2MULasYCeDD8gmI9Wf6FCMMabVxRqh7Br35RmqWhG9TETaxagtoZCydEMx54zomehQjDEmIbzeNfSJx3ltzqbickorgxx+UMdEh2KMMQkRq43gQKAXkCEiR+J0LwHQEWgXHfKs2LQLgMN7WiIwxiSnWG0EpwNXAL2BR6LmlwD/G6eYWtXyjcWIwKEHZic6FGOMSYhYbQQzgBkicqGqvtFKMbWqVZtLOKR7BzJTvd5Ja4wx7UusqqFLVfUloL+I3Fp3uao+Us9mbcrawlIOsW4ljDFJLFZjcZb7uwOQXc9Po0RkvIisFpF8EZnawDoXichKEVkhIi83IfZmKy6vZu223Qzv06k1D2uMMfuVWFVDT7m/72/qjt0RzaYBpwEFwEIRma2qK6PWGQT8HDjOHf6yVbv+XF9UBmAlAmNMUmtKX0MdRSQgIv8SkUIRuTTGZiOBfFVdq6pVwCvAuXXWuRqYpqo7AFR1a1NPoDk27nQSQa9OGa15WGOM2a94fY5gnKruAiYA63B6Ib0jxja9gA1R0wXuvGiDgcEi8h8R+VRExte3IxG5RkQWiciiwsJCjyHHts4tEfTp0i7uhDXGmH3iNRGEq5DOAv6qqsUtdPwUYBBwIjAZeFpEOtVdSVWnq2ququZ27969hQ4Nm3eWk52eQk5GoMX2aYwxbY3XRDBHRL4Ejgb+JSLdgYoY22wE+kRN93bnRSsAZqtqtap+A3yFkxhaRcGOcqsWMsYkPa8jlE0Fvo8zDkE1sJu96/vrWggMEpEBIpIKXAzMrrPO33BKA4hIN5yqorVeg2+ub7btZkC3rNgrGmNMO+a1sTgAXAq8KiKvA1OAosa2UdUgcAPwLrAKeE1VV4jIAyJyjrvau0CRiKwE5gF3qGqj+20poZBSsLPc2geMMUnP6+O0TwAB4E/u9GXuvKsa20hV5wJz68y7J+q14nR1vdfDavFWWFpJVTBE785WNWSMSW5eE8Exqjo8avrfIrI0HgG1lk07ywG7ddQYY7w2FteIyMDwhIgcDNTEJ6TWEe51tKclAmNMkvNaIrgDmCcia3G6ou4H/DhuUbWC4vJqAPp3tcZiY0xyi5kI3FtFi3GeFA53AbFaVSvjGVi8FZVWkRHw2/CUxpik12jVkIhcBawAHgfygP6quqytJwGAJRt20CUrNdFhGGNMwsUqEdwMDFHVQrdd4C/s/SxAm1RcXm1PFBtjDLEbi6tUtRBAVdcCafEPqXUUllTSy24dNcaYmCWC3iLyWEPTqnpjfMKKr+qaECUVQRue0hhjiJ0I6vYwujhegbSmotIqAA7omJ7gSIwxJvG8jFnc7mwqdh4mOyjHEoExxsS6a+hpERnawLIsEblSRH4Yn9DiZ1uJc9NTj2xLBMYYE6tqaBpwj4gMA5YDhUA6TlfRHYHncO4kalN2lDlVQ50y7a4hY4yJVTWUB1wkIh2AXOAgoBxYpaqr4x9efBTtdhJB1w72HIExxnjqYkJVS4EP4htK69nuPlWcmeq1hw1jjGm/vHY6164U7a6yp4qNMcaVlIlgV3m1tQ8YY4yrSYlARNrFcF4lFUE6pFm1kDHGgPehKr/vDif5pTs9XET+FGOz/VbR7kq6dWg3vWUYY0yzeC0R/B44HXecYlVdCpwQr6DibUdZNTlWNWSMMUATqoZUdUOdWW1yhDJVpbi8ms6WCIwxBvA+QtkGEfk+oCISAG4CVsUvrPjZXVVDTUitC2pjjHF5LRFcC1wP9AI2AiOA6+IUU1yFh6jMTrdEYIwx4L1E8D1VrdWnkIgcB/yn5UOKr5IKJxGEVBMciTHG7B+8lgge9zhvv1daEQSgd+d2cSesMcY0W6MlAhEZDXwf6C4it0Yt6gi0yVHfd5Y5JYJO1kZgjDFA7KqhVKCDu170cF67gInxCiqetrs9j1oXE8YY44jV++iHwIci8ryqfttKMcXVDrfn0c6WCIwxBvDeWFwmIr8FhuCMRwCAqp4cl6jiaHtZFal+H1mpbbJmyxhjWpzXxuK/4HQvMQC4H1gHLIxTTHG1q7yajhkBRCTRoRhjzH7BayLoqqrPAtWq+qGqXgm0udIAQFlVDVlpVhowxpgwr1VD1e7vzSJyFrAJ6BKfkOKrrKqGjIAlAmOMCfNaIviliOQAtwG3A88AN8faSETGi8hqEckXkamNrHehiKiI5HqMZ5+VVFSTnW5dUBtjTJjXoSrnuC+LgZMg8mRxg0TED0wDTgMKgIUiMltVV9ZZLxun76LPmhb6vtldWWNjFRtjTJRGSwQi4heRySJyu4gMdedNEJFPgD/G2PdIIF9V16pqFfAKcG496/0CeAioaHr4Tberotr6GTLGmCixqoaeBa4CugKPichLwMPAb1T1yBjb9gKiu64ucOdFiMhRQB9VfbuxHYnINSKySEQWFRYWxjhs43ZX2uhkxhgTLdYVMRc4QlVDIpIObAEGqmpRcw8sIj7gEeCKWOuq6nRgOkBubm6zeosrrQzaMwTGGBMlVomgSlVDAKpaAaxtQhLYCPSJmu7tzgvLBoYCH4jIOuBYYHY8G4xDIaWiOkSmlQiMMSYi1hXxUBFZ5r4WYKA7LYCq6hGNbLsQGCQiA3ASwMXAJeGFqloMdAtPi8gHwO2quqjJZ+FRZTAEQKaVCIwxJiJWIjhsX3esqkERuQF4F6en0udUdYWIPAAsUtXZ+7rvfVVW5XRBnZ7ieYROY4xp92J1OtesjuZUdS4wt868expY98TmHMuLCrdEkGElAmOMiUiqr8aV1TUApKVYIjDGmLCkSgQl7uhk6YGkOm1jjGmU5yuiiGSIyPfiGUy8hccprqqx8YqNMSbMUyIQkbOBPOAdd3qEiLR6Y29zhe8a6mZdTBhjTITXEsF9OF1G7ARQ1TycsQnalHAiSLO7howxJsLrFbHave8/WpurXwk3Fqf6rbHYGGPCvD5iu0JELgH8IjIIuBH4JH5hxYfdPmqMMXvzWiL4Kc54xZXAyzjdUd8cp5jipiJy+6hVDRljTJjXEsGhqnoXcFc8g4m3wpJKANJthDJjjInw+tX4dyKySkR+ER6XoC1S9/bRVCsRGGNMhKcroqqehDMyWSHwlIh8ISJ3xzWyOBARwB4oM8aYaJ6viKq6RVUfA67Feaag3j6D9mfh20dT/ZYIjDEmzOsDZYeJyH0i8gXwOM4dQ73jGlkcVAVDpKb4IiUDY4wx3huLnwNeBU5X1U1xjCeuKqpr7I4hY4ypw1MiUNXR8Q6kNVQGa+yOIWOMqaPRRCAir6nqRW6VUPSTxF5GKNvvVFaHrERgjDF1xCoR3OT+nhDvQFrD2m278fusfcAYY6I1+vVYVTe7L69T1W+jf4Dr4h9ey+qalUqpOyaBMcYYh9d6ktPqmXdGSwbSGqpDSu8umYkOwxhj9iux2gj+B+eb/8EisixqUTbwn3gGFg/BmhABqxoyxphaYrURvAz8A3gQmBo1v0RVt8ctqjgJ1igpfksExhgTLVbVkKrqOuB6oCTqBxHpEt/QWt7GneUE7KliY4ypxUuJYAKwGOf20eiv0wocHKe44iI7PYVtpVWJDsMYY/YrjSYCVZ3g/m5zw1LWpyakDDogK9FhGGPMfsVrX0PHiUiW+/pSEXlERPrGN7SWV1UTsg7njDGmDq9XxSeAMhEZDtwGfA28GLeo4uTbojIbi8AYY+rwelUMqjOqy7nAH1V1Gs4tpG1KTkaA7butjcAYY6J57X20RER+DlwGjBERHxCIX1jxEVKld2d7oMwYY6J5LRFMwhm4/kpV3YIzFsFv4xZVnARrlIA9R2CMMbV4HapyC/AXIEdEJgAVqvpCXCOLg5qQ4vdZG4ExxkTzetfQRcAC4AfARcBnIjLRw3bjRWS1iOSLyNR6lt8qIitFZJmI/EtE+jX1BJqiOhSyEoExxtThtY3gLuAYVd0KICLdgfeB1xvaQET8wDScDusKgIUiMltVV0attgTIVdUyt1+j3+BUQ7W4UEhRxbqhNsaYOrzWk/jCScBV5GHbkUC+qq5V1SrgFZy7jiJUdZ6qlrmTnxLHcZCrQ87A9dbFhDHG1Oa1RPCOiLwLzHSnJwFzY2zTC9gQNV0AjGpk/Sk4HdztRUSuAa4B6Nt3355jqwo6icAeKDPGmNq8jll8h4hcABzvzpquqrNaKggRuRTIBcY2cPzpwHSA3NxcrW+dWCrdRJAWsERgjDHRYo1HMAh4GBgIfAHcrqobPe57I9Anarq3O6/uMU7FaYMYq6qVHvfdZFYiMMaY+sW6Kj4HzAEuxOmB9PEm7HshMEhEBohIKnAxMDt6BRE5EngKOKdOG0SLC9Y4BYkUSwTGGFNLrKqhbFV92n29WkQ+97pjVQ2KyA3Au4AfeE5VV4jIA8AiVZ2N81BaB+CvIgKwXlXPafJZeFCjTiKwPGCMMbXFSgTp7rf28D2XGdHTqtpoYlDVudRpVFbVe6Jen9rkiPdRTchJBD6x20eNMSZarESwGXgkanpL1LQCJ8cjqHgIRUoElgiMMSZarIFpTmqtQOItXCLwW4nAGGNqSZoa80jVkJUIjDGmlqRJBJGqISsRGGNMLUmTCCJVQ1YiMMaYWrz2PiruWMX3uNN9RWRkfENrWbsra4A9CcEYY4zDa4ngT8BoYLI7XYLTs2ibEe5+2m/dUBtjTC1eO50bpapHicgSAFXd4T4t3GaECwLpKf7EBmKMMfsZryWCand8AYXIeAShuEUVB+HGYmsiMMaY2rwmgseAWUAPEfl/wMfAr+IWVRzYA2XGGFM/r91Q/0VEFgOn4HQvcZ6qroprZC0s3EgsdvuoMcbU4ikRiEhfoAx4K3qeqq6PV2AtzS0QWInAGGPq8NpY/DZO+4AA6cAAYDUwJE5xtbg9nc4lOBBjjNnPeK0aGhY9LSJHAdfFJaI42dNYbJnAGGOi7dOTxW73042NP7zfsURgjDH189pGcGvUpA84CtgUl4jiJPwcgS9pOtUwxhhvvLYRZEe9DuK0GbzR8uHEj3VDbYwx9YuZCNwHybJV9fZWiCduIlVD1lpsjDG1NFpRIiIpqloDHNdK8cSNdUNtjDH1i1UiWIDTHpAnIrOBvwK7wwtV9c04xtaiQm6HGJYHjDGmNq9tBOlAEc4YxeHnCRRoM4kg3Pm03TVkjDG1xUoEPdw7hpazJwGEtamO/cNVQ5YH2pfq6moKCgqoqKhIdCjG7BfS09Pp3bs3gUDA8zaxEoEf6EDtBBDWphKBqvU11B4VFBSQnZ1N//797bM1SU9VKSoqoqCggAEDBnjeLlYi2KyqDzQvtP1DuK8hu2mofamoqLAkYIxLROjatSuFhYVN2i7W41Xt5r8r8kCZXTDaHUsCxuyxL/8PsRLBKfsWyv4n0kaQ4DiMMWZ/02giUNXtrRVIvIUbNOzbo2lpHTp0aPY+Fi1axI033tjg8nXr1vHyyy97Xh+gf//+DBs2jCOOOIKxY8fy7bffNjvOlvLkk0/ywgsvtMi+Nm/ezIQJE2rNu/nmm+nVqxeh0J6BFO+77z4efvjhWuv179+fbdu2AbBlyxYuvvhiBg4cyNFHH82ZZ57JV1991azYKisrmTRpEocccgijRo1i3bp19a73hz/8gaFDhzJkyBAeffTRWssef/xxDj30UIYMGcLPfvYzAL744guuuOKKZsUWLWl63lEbqtLsx3Jzc3nssccaXF43EcRaP2zevHksW7aME088kV/+8pfNjlNVa11c99W1117Lj370o2bvB+CRRx7h6quvjkyHQiFmzZpFnz59+PDDDz3tQ1U5//zzOfHEE/n6669ZvHgxDz74IN99912zYnv22Wfp3Lkz+fn53HLLLdx55517rbN8+XKefvppFixYwNKlS5kzZw75+fmA8/n9/e9/Z+nSpaxYsYLbb3c6eBg2bBgFBQWsX98yQ8J4fY6gzQvZCGXt3v1vrWDlpl0tus/De3bk3rObPuxGXl4e1157LWVlZQwcOJDnnnuOzp07s3DhQqZMmYLP5+O0007jH//4B8uXL+eDDz7g4YcfZs6cOXz44YfcdNNNgPP3+tFHHzF16lRWrVrFiBEjuPzyyznyyCMj65eWlvLTn/6URYsWISLce++9XHjhhbXiGT16dCRxFBYWcu2110YuIo8++ijHHXcchYWFXHLJJWzatInRo0fzz3/+k8WLF1NaWsrpp5/OqFGjWLx4MXPnzuW1117jtddeo7KykvPPP5/777+f3bt3c9FFF1FQUEBNTQ3/93//x6RJk5g6dSqzZ88mJSWFcePG8fDDD3PffffRoUMHbr/99gbfqxNPPJFRo0Yxb948du7cybPPPsuYMWP2eq/feOONWknugw8+YMiQIUyaNImZM2dy0kknxfy85s2bRyAQ4Nprr43MGz58eJM/97r+/ve/c9999wEwceJEbrjhBlS11nVo1apVjBo1iszMTADGjh3Lm2++yc9+9jOeeOIJpk6dSlpaGgA9evSIbHf22WfzyiuvREoJzZE8JQL3t5UITGv40Y9+xEMPPcSyZcsYNmwY999/PwA//vGPeeqpp8jLy8Pv99e77cMPP8y0adPIy8tj/vz5ZGRk8Otf/5oxY8aQl5fHLbfcUmv9X/ziF+Tk5PDFF1+wbNkyTj755L32+c4773DeeecBcNNNN3HLLbewcOFC3njjDa666ioA7r//fk4++WRWrFjBxIkTa33bXLNmDddddx0rVqxg9erVrFmzhgULFpCXl8fixYv56KOPeOedd+jZsydLly5l+fLljB8/nqKiImbNmsWKFStYtmwZd999t+f3CiAYDLJgwQIeffTRWvPDvvnmGzp37hy5UALMnDmTyZMnc/755/P2229TXV3d0McUsXz5co4++uiY6wGMGTOGESNG7PXz/vvv77Xuxo0b6dOnDwApKSnk5ORQVFRUa52hQ4cyf/58ioqKKCsrY+7cuWzYsAGAr776ivnz5zNq1CjGjh3LwoULI9vl5uYyf/58TzHHkjwlAjcTiDUXt1v78s09HoqLi9m5cydjx44F4PLLL+cHP/gBO3fupKSkhNGjRwNwySWXMGfOnL22P+6447j11lv54Q9/yAUXXEDv3r0bPd7777/PK6+8Epnu3Llz5PVJJ53E9u3b6dChA7/4xS8i669cuTKyzq5duygtLeXjjz9m1qxZAIwfP77Wfvr168exxx4LwHvvvcd7773HkUceCUBpaSlr1qxhzJgx3Hbbbdx5551MmDCBMWPGEAwGSU9PZ8qUKUyYMGGvuvyG3quwCy64AICjjz663vr1zZs3071798h0VVUVc+fO5ZFHHiE7O5tRo0bx7rvvMmHChAZrA5paS9BSF9+www47jDvvvJNx48aRlZXFiBEjIl8SgsEg27dv59NPP2XhwoVcdNFFrF27FhGhR48ebNrUMqMBxLVEICLjRWS1iOSLyNR6lqeJyKvu8s9EpH+8Yok8UJY0ZSDTVk2dOpVnnnmG8vJyjjvuOL788st93te8efP49ttvGTFiBPfeey/g1KF/+umn5OXlkZeXx8aNG2M2eGdlZUVeqyo///nPI9vn5+czZcoUBg8ezOeff86wYcO4++67eeCBB0hJSWHBggVMnDiROXPmMH78+CbFH/6m7/f7CQaDey3PyMio9VT5u+++y86dOxk2bBj9+/fn448/ZubMmQB07dqVHTt21Nq+pKSETp06MWTIEBYvXuwppqaUCHr16hX5dh8MBikuLqZr1657rTdlypRIyapz584MHjwYgN69e3PBBRcgIowcORKfzxdp3K6oqCAjI8NTzLHE7bLodl89DTgDOByYLCKH11ltCrBDVQ8Bfg88FK941J4jMK0kJyeHzp07R745vvjii4wdO5ZOnTqRnZ3NZ599BlDrW3y0r7/+mmHDhnHnnXdyzDHH8OWXX5KdnU1JSUm965922mlMmzYtMl33YpeSksKjjz7KCy+8wPbt2xk3bhyPP/54ZHleXh7glERee+01wPnWX3c/YaeffjrPPfccpaWlgFP9sXXrVjZt2kRmZiaXXnopd9xxB59//jmlpaUUFxdz5pln8vvf/56lS5d6eq+8Gjx4cK2SwsyZM3nmmWdYt24d69at45tvvuGf//wnZWVlnHDCCcyePTvyPr755psMHz4cv9/PySefTGVlJdOnT4/sa9myZfV++58/f34kCUb/nHrqqXute8455zBjxgwAXn/9dU4++eR6SyBbt24FYP369bz55ptccsklAJx33nnMmzcPcKqJqqqq6NatW2R66NChnt+rxsSzamgkkK+qawFE5BXgXGBl1DrnAve5r18H/igiouGv7y3IniMw8VJWVlar+ubWW29lxowZkQbQgw8+mD//+c+AcxfJ1Vdfjc/nY+zYseTk5Oy1v0cffZR58+bh8/kYMmQIZ5xxBj6fD7/fz/Dhw7niiisi1TIAd999N9dffz1Dhw7F7/dz7733RqpUwg466CAmT57MtGnTeOyxx7j++us54ogjCAaDnHDCCTz55JPce++9TJ48mRdffJHRo0dz4IEHkp2dHbngh40bN45Vq1ZFqrg6dOjASy+9RH5+PnfccQc+n49AIMATTzxBSUkJ5557LhUVFagqjzzyyF7n29B75UVWVhYDBw4kPz+fnj178s477/Dkk0/WWn788cfz1ltvMWnSJG644QaOP/74SNXKM888AzjVQ7NmzeLmm2/moYceIj09nf79++91K2dTTZkyhcsuu4xDDjmELl26RJL/pk2buOqqq5g7dy4AF154IUVFRQQCAaZNm0anTp0AuPLKK7nyyisZOnQoqampzJgxI5JI5s2bx1lnndWs+CJUNS4/wETgmajpy4A/1llnOdA7avproFs9+7oGWAQs6tu3r+6L91Zs0eteWqzlVcF92t7sn1auXJnoEJqkpKQk8vrBBx/UG2+8MYHR1FZRUaHV1dWqqvrJJ5/o8OHDExuQR2+++abeddddiQ6jVVVUVOioUaMin1dd9f1fAIu0get1m2gsVtXpwHSA3NzcfSotnHb4AZx2+AEtGpcxTfX222/z4IMPEgwG6devH88//3yiQ4pYv349F110EaFQiNTUVJ5++ulEh+TJ+eefv9edOO3d+vXr+fWvf01KSstcwuOZCDYCfaKme7vz6lunQERSgByccQ+MaZcmTZrEpEmTEh1GvQYNGsSSJUsSHcY+Cd8CmywGDRrEoEGDWmx/8byHZiEwSEQGiEgqcDEwu846s4HL3dcTgX+7RRhjPLM/GWP22Jf/h7glAlUNAjcA7wKrgNdUdYWIPCAi57irPQt0FZF84FZgr1tMjWlMeno6RUVFlgyMYc94BOnp6U3aTtraP1Bubq4uWrQo0WGY/YSNUGZMbQ2NUCYii1U1t75t2kRjsTENCQQCTRqJyRizN3vO1hhjkpwlAmOMSXKWCIwxJsm1ucZiESkE9nWopW7AthYMpy2wc04Ods7JoTnn3E9Vu9e3oM0lguYQkUUNtZq3V3bOycHOOTnE65ytasgYY5KcJQJjjElyyZYIpsdepd2xc04Ods7JIS7nnFRtBMYYY/aWbCUCY4wxdVgiMMaYJNcuE4GIjBeR1SKSLyJ79WgqImki8qq7/DMR6Z+AMFuUh3O+VURWisgyEfmXiPRLRJwtKdY5R613oYioiLT5Ww29nLOIXOR+1itE5OXWjrGlefjb7isi80Rkifv3fWYi4mwpIvKciGwVkeUNLBcRecx9P5aJyFHNPmhDQ5e11R/AjzPk5cFAKrAUOLzOOtcBT7qvLwZeTXTcrXDOJwGZ7uv/SYZzdtfLBj4CPgVyEx13K3zOg4AlQGd3ukei426Fc54O/I/7+nBgXaLjbuY5nwAcBSxvYPmZwD9whmA/FvisucdsjyWCkUC+qq5V1SrgFeDcOuucC8xwX78OnCLhEaHbppjnrKrzVLXMnfwUZ8S4tszL5wzwC+AhoD30U+3lnK8GpqnqDgBV3drKMbY0L+esQEf3dQ6wqRXja3Gq+hGwvZFVzgVeUMenQCcROag5x2yPiaAXsCFqusCdV+866gygUwx0bZXo4sPLOUebgvONoi2Lec5ukbmPqr7dmoHFkZfPeTAwWET+IyKfisj4VosuPryc833ApSJSAMwFfto6oSVMU//fY7LxCJKMiFwK5AJjEx1LPImID3gEuCLBobS2FJzqoRNxSn0ficgwVd2ZyKDibDLwvKr+TkRGAy+KyFBVDSU6sLaiPZYINgJ9oqZ7u/PqXUdEUnCKk0WtEl18eDlnRORU4C7gHFWtbKXY4iXWOWcDQ4EPRGQdTl3q7DbeYOzlcy4AZqtqtap+A3yFkxjaKi/nPAV4DUBV/wuk43TO1l55+n9vivaYCBYCg0RkgIik4jQGz66zzmzgcvf1RODf6rbCtFExz1lEjgSewkkCbb3eGGKcs6oWq2o3Ve2vqv1x2kXOUdW2PM6pl7/tv+GUBhCRbjhVRWtbMcaW5uWc1wOnAIjIYTiJoLBVo2xds4EfuXcPHQsUq+rm5uyw3VUNqWpQRG4A3sW54+A5VV0hIg8Ai1R1NvAsTvExH6dR5uLERdx8Hs/5t0AH4K9uu/h6VT0nYUE3k8dzblc8nvO7wDgRWQnUAHeoapst7Xo859uAp0XkFpyG4yva8hc7EZmJk8y7ue0e9wIBAFV9Eqcd5EwgHygDftzsY7bh98sYY0wLaI9VQ8YYY5rAEoExxiQ5SwTGGJPkLBEYY0ySs0RgjDFJzhJBEhCRGhHJi/rp38i6pS1wvOdF5Bv3WJ+7T3s2dR/PiMjh7uv/rbPsk+bG6O4n/L4sF5G3RKRTjPVH7EvPliJykIjMcV+fKCLF7nFXici9+7C/c8K9cIrIeeH3yZ1+wH1wsFncz3BijHU+aMoDeu65z/GwXr29b4rIwyJystfjGe8sESSHclUdEfWzrhWOeYeqjgCm4jzI1iSqepWqrnQn/7fOsu83Pzxgz/syFOd5kutjrD8C5/7tproVeDpqer773uTi9JHTpG6EVXW2qv7anTwPp8fN8LJ7VPX9fYhxf/I8UF8fSY/j/D2ZFmaJIAmJSAdxxiT4XES+EJG9eu10v8V+FPWNeYw7f5yI/Nfd9q8i0iHG4T4CDnG3vdXd13IRudmdlyUib4vIUnf+JHf+ByKSKyK/BjLcOP7iLit1f78iImdFxfy8iEwUEb+I/FZEForTX/tPPLwt/8XtuEtERrrnuEREPhGR77lPtT4ATHJjmeTG/pyILHDXra/3U4ALgXfqzlTV3cBi4BC3tPGpG+8sEensxnKj7BlH4hV33hUi8kcR+T5wDvBbN6aBUe/BeBH5a9R7E/k23tTPUETucd/L5SIyXaRWT72XRf2NjHTX9/q+1Kuh3jdV9Vugq4gc2JT9GQ8S0d+2/bTuD84TpnnuzyycJ8o7usu64TyhGH64sNT9fRtwl/vaj9N3TzecC3uWO/9O4J56jvc8MNF9/QPgM+Bo4AsgC+cJ5xXAkTgXyaejts1xf3+AO35AOKaodcIxng/McF+n4vTImAFcA9ztzk8DFgED6omzNOr8/gqMd6c7Ainu61OBN9zXVwB/jNr+V8Cl7utOOP36ZNU5xgBgcdT0icAc93VXYB0wBFgGjHXnPwA86r7eBKSFj1E3juj3Onra/YzXR31WTwCX7uNn2CVq/ovA2VGf0dPu6xNw+89v6H2pc+65wDON/M32p57++HFKVhcm+n+qvf20uy4mTL3K1amKAEBEAsCvROQEIITzTfgAYEvUNguB59x1/6aqeSIyFqca4j/ul8JUnG/S9fmtiNyN0+fLFJy+YGap8y0YEXkTGIPzTfl3IvIQzkVifhPO6x/AH0QkDacq4SNVLReRccARUXXcOTgdr31TZ/sMEclzz38V8M+o9WeIyCCcLgsCDRx/HHCOiNzuTqcDfd19hR3E3v3ejBGRJTjv/a9xOorrpKofustn4CQmcBLEX0Tkbzj9CHmiTtcM7wBni8jrwFnAz3B6nfX6GYadJCI/AzKBLjhJ/C132Uz3eB+JSEdx2lkael+i41sEXOX1fKJsBXruw3amEZYIktMPge7A0apaLU7vnOnRK7j/2CfgXECeF5FHgB3AP1V1sodj3KGqr4cnROSU+lZS1a/cOvIzgV+KyL9U9QEvJ6GqFSLyAXA6MAln0BJwRm76qaq+G2MX5ao6QkQycfqyuR54DGcwm3mqer44DesfNLC94Hw7Xd3YMajz3uK0EUyI7EQkp5Htz8L5tn02cJeIDGtk3bpeAW7AqWZZpKolbrWO188QEUkH/oRTOtsgIvdR+3zq9lGjNPC+iMgBTYi9Iek476lpQdZGkJxygK1uEjgJ2Gv8YnHGNP5OVZ8GnsEZOu9T4DgRCdf5Z4nIYI/HnA+cJyKZIpKFU60zX0R6AmWq+hJOx3j1NZxWuyWT+ryK0+lWuHQBzkX9f8LbiMhg95j1UmfkthuB22RPt+Thbn2viFq1BKeKLOxd4KfhOnNxenit6yucao4GqWoxsEPcdhjgMuBDccZU6KOq83CqcHJwqtWi1Y0p2oc47+fV7EmSTf0Mwxf9bW5bQt07icJtOsfj9IJZjLf3ZV8NBuody9fsO0sEyekvQK6IfAH8CPiynnVOBJa6VRiTgD+oaiHOhXGmiCzDqVI41MsBVfVznHrnBThtBs+o6hJgGLDAraK5F/hlPZtPB5aJ21hcx3s41R3vqzOUITiJayXwuTi3ID5FjNKvG8synEFOfgM86J579HbzgMPDjcU4JYeAG9sKd7rufncDX4cvvI24HKc6bRnO3UkP4LRdvOR+TkuAx3TvAWZeAe5wG2UH1jl2DTAHOMP9TVM/Q/d4T+NcfN/FqTKMVuG+T0/iVAGCh/dFnBsBnqnvmOL0vvlf4HsiUiAiU9z5AZwbD9pyV+L7Jet91Jg4E5Hzcarh7k50LG2Z+z4epar/l+hY2htrIzAmzlR1loi05TGx9xcpwO8SHUR7ZCUCY4xJctZGYIwxSc4SgTHGJDlLBMYYk+QsERhjTJKzRGCMMUnu/wPuPqwhG3/BMAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "RocCurveDisplay.from_estimator(lr, X_test_vec, y_test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bb7ceed",
   "metadata": {},
   "source": [
    "Examining the ROC curve we see a very high AUC (.96). The curve is showing the TPR vs FPR for each possible cut-off for the probabilities output by the Logistic Regression model. \n",
    "\n",
    "The model gives a probability for a positive or negative prediction for each individual data point and then chooses the best cutoff. The curve shows how this varies across a range of different cut offs points ranging from 0-1."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ae40141-c859-42e8-a0f0-7d196305efbb",
   "metadata": {},
   "source": [
    "## 3.5.3 Naive Bayes with Count Vectorization <a name=\"3.5.3\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d382aefc",
   "metadata": {},
   "source": [
    "#### 3.5.3.1 Training and Fitting the Model <a name=\"3.5.3.1\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "a99a6c79-9f7c-42ea-a2bf-1a348b5693bc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(786016, 77986) <class 'scipy.sparse._csr.csr_matrix'> (87336,)\n",
      "(786016,) (87336,)\n"
     ]
    }
   ],
   "source": [
    "#Trying a Naive Bayesian model with simple bag-of-words Count Vectorization\n",
    "\n",
    "count_vec = CountVectorizer()\n",
    "\n",
    "#fit the vectorizer on the training data\n",
    "count_vec.fit(X_train)\n",
    "\n",
    "#transform both the training and test data\n",
    "X_train_vec = count_vec.transform(X_train)\n",
    "X_test_vec = count_vec.transform(X_test)\n",
    "\n",
    "print(X_train_vec.shape, type(X_train_vec), X_test.shape)\n",
    "print(y_train.shape, y_test.shape)\n",
    "\n",
    "#using the sklearn StandardScaler to normalize the data, with a \"with_mean\" parameter for sparse matrices\n",
    "scaler = StandardScaler(with_mean=False)\n",
    "\n",
    "#fit on the training data, scale both train and text\n",
    "X_train_vec = scaler.fit_transform(X_train_vec)\n",
    "X_test_vec = scaler.transform(X_test_vec)\n",
    "\n",
    "#using the complement version of naive bayes which accepts sparse matrices as input\n",
    "#it also is supposed to work well for imbalanced datasets\n",
    "comp_nb = ComplementNB()\n",
    "\n",
    "#train the lr model with the training data\n",
    "comp_nb.fit(X_train_vec, y_train)\n",
    "\n",
    "#predict on the training data\n",
    "train_preds = comp_nb.predict(X_train_vec)\n",
    "\n",
    "#predict on the test data\n",
    "test_preds = comp_nb.predict(X_test_vec)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d71410-4828-4544-9830-8051cc252fb7",
   "metadata": {},
   "source": [
    "##### Naive Bayes Train Set Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "5a8c7e21-0bcf-4fec-bea1-6b9f9770c597",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.92      0.81      0.86    546424\n",
      "           1       0.66      0.85      0.75    239592\n",
      "\n",
      "    accuracy                           0.82    786016\n",
      "   macro avg       0.79      0.83      0.80    786016\n",
      "weighted avg       0.85      0.82      0.83    786016\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_train, train_preds))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "997ae026",
   "metadata": {},
   "source": [
    "#### 3.5.3.2 Evaluating the Model <a name=\"3.5.3.2\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c1cf5db",
   "metadata": {},
   "source": [
    "##### Naive Bayes Test Set Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "b2f2e46d-ee80-4591-a90a-2646ad38a2d5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.90      0.79      0.84     60609\n",
      "           1       0.63      0.81      0.71     26727\n",
      "\n",
      "    accuracy                           0.80     87336\n",
      "   macro avg       0.77      0.80      0.78     87336\n",
      "weighted avg       0.82      0.80      0.80     87336\n",
      "\n",
      "Confusion matrix values: \n",
      "[[47984 12625]\n",
      " [ 5168 21559]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEGCAYAAADyuIefAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAl5UlEQVR4nO3deZwV1Zn/8c/TzY7sICJgQEERUVFQUBNDNAISRzRj3EZljNG4m2gmipMZjEt+OmM0IW5DhLjEiMa1VZQQFLeEXQQBlRaUfV+bvbuf3x91GgrovvcW9Hr7+3696uWtU6eqTqE+nKpTdR5zd0REJJJT1Q0QEalOFBRFRGIUFEVEYhQURURiFBRFRGLqVHUD4lq3zPVOHetWdTMkgS+/alXVTZAEtm1fz46dm+1AjjHge419zdqijOpOm7l9rLsPPJDzVbZqFRQ7dazL5LEdq7oZkkD/C4ZUdRMkgckzHj/gY6xeW8SksR0yqlu33VetD/iElaxaBUURqQmcIi+u6kZUGAVFEUnEgWKy96MPBUURSawY9RRFRABwnJ26fRYRiThQpNtnEZHd9ExRRCRwoCiLZ9dSUBSRxLL3iaKCoogk5LieKYqIlHCHndkbEzUhhIgkZRRluGR0NLNcM/vEzN4M60+Z2QIzmxGWnqHczGy4meWb2UwzOzF2jCFmNi8sQ2LlvcxsVthnuJmlbZSCoogk4kCxZ7Zk6BZg7l5l/+HuPcMyI5SdDXQNyzXA4wBm1hIYBvQBTgaGmVmLsM/jwNWx/dJOTqGgKCKJlVdP0cw6AD8AnszgtIOBZzwyEWhuZu2AAcA4d1/r7uuAccDAsK2pu0/0KBnVM8B56U6ioCgiiUQvb5fb7fPvgF+y74D2feEW+WEzqx/K2gOLYnUWh7JU5YtLKU9JQVFEEnFgp+dktACtzWxqbLmm5Dhmdg6w0t2n7XWKoUA34CSgJXB7JV0aoNFnEUnIMYoy70+tdvfeZWw7DTjXzAYBDYCmZvZnd78sbN9uZn8CfhHWlwDxCVc7hLIlQL+9yieE8g6l1E9JPUURSazYLaMlFXcf6u4d3L0TcDHwrrtfFp4FEkaKzwM+C7vkAVeEUei+wAZ3XwaMBfqbWYswwNIfGBu2bTSzvuFYVwCvp7s29RRFJJGSZ4oV6DkzawMYMAO4NpSPAQYB+cAW4EoAd19rZvcAU0K9u919bfh9PfAU0BB4OywpKSiKSEJGkZfvTaa7TyC65cXdzyijjgM3lLFtFDCqlPKpQI8kbVFQFJFEopm3s/fJm4KiiCTibuzw3KpuRoVRUBSRxIor9plilVJQFJFEooEW3T6LiATlP9BSnSgoikgiGmgREdlLUZoXs2syBUURScQxdnr2ho7svTIRqRAaaBERiXFMt88iInEaaBERCdzRKzkiIiWigRZ95icisosGWkREAif9BLI1mYKiiCSmnqKISBDlfc7eoJi9VyYiFSSz9KaZpiwws1wz+8TM3gzrnc1skpnlm9kLZlYvlNcP6/lhe6fYMYaG8i/MbECsfGAoyzezOzJpj4KiiCQSpTjNzWjJ0C3A3Nj6A8DD7t4FWAdcFcqvAtaF8odDPcysO1Hiq2OAgcBjIdDmAo8CZwPdgUtC3ZQUFEUkEXej2HMyWtIxsw7AD4Anw7oBZwAvhSpPE2X0Axgc1gnbzwz1BwOj3X27uy8gSmx1cljy3X2+u+8ARoe6KemZoogkluDl7dZmNjW2PsLdR8TWfwf8EmgS1lsB6929MKwvBtqH3+2BRQDuXmhmG0L99sDE2DHj+yzaq7xPugYrKIpIItF8ihm/krPa3XuXtsHMzgFWuvs0M+tXPq07cAqKIpJQuc28fRpwrpkNAhoATYHfA83NrE7oLXYAloT6S4COwGIzqwM0A9bEykvE9ymrvEx6pigiiUSv5FhGS8rjuA919w7u3olooORdd/834D3gglBtCPB6+J0X1gnb3w25oPOAi8PodGegKzAZmAJ0DaPZ9cI58tJdn3qKIpJIJXz7fDsw2szuBT4BRobykcCzZpYPrCUKcrj7bDN7EZgDFAI3uHsRgJndCIwFcoFR7j473ckVFEUksfKeOszdJwATwu/5RCPHe9fZBvyojP3vA+4rpXwMMCZJWxQURSSRaOowffssIrKLJoQQEQmiWXKyd4xWQVFEEok+81NQlKCoCG4aeCSt2u3knmcWcOt5XdhaEI3ErV9Th6N6buGuPy1g0/pcHrq1I8u+qU/d+sXc9tAiOnXbBsArI9rw9l9aYgadu23jtocXUq+B7zrHY79qz9jRLXk9f1aVXGO2uPX6j+nbawnrNzTgmlvPBeDqy6fSt/didhbmsGx5Ex589DQ2b6kHQOdvreOWa/5Jo0Y78WLjxjt+QI45v7rtfQ49ZBNFxcbEqR0Y9VwvAM7ql8/Vl09jzdpGALz+TjfeGd+1ai62UqmnuN/MbCDRy5i5wJPufn9Fnq8yvPZkGzp23c6Wgug/iodey9+17e6fdOKUARsAGD28LUccs5Vho75m4bz6PPqfHXjgxa9Yvawur41szR8nfE79hs69P/0WE15vQf+L1gLw5acNKdiQvVO9V6Zx73Uh7+1u/PKmj3eVTZ95KCOfO5Hi4hyuumwaF/9wFiP/3IucnGJuv/lD/mf4t5n/TUuaHLSNoiIjp47zUt4xfDr7EOrUKeKBYeM46YQlTPkk+ors/X904tGRab8cyzoJvmipcSos3O/vDBXV2aqldZk8vilnX7pmn22bN+Xw6ccHcerAKCgunFef479dAMBhXbezYlE91q2K/g4qKjS2b8uhqBC2b82hVdudUXkR/PGeQ7nqV0sr6Yqy26y5bdlUUH+PsmmfHkpxcfSf/edftqFNqy0A9Dp+KQu+acH8b1oCsKmgAcXFOWzfUYdPZx8CQGFhLvnzW9K61eZKvIrqp2T0OZOlJqrInuKuGSoAzKxkhoo5FXjOCvXEsPb85FdL2VKwb0/uH+80o+e3C2jcpBiAzt238fGYZhzbZzOff9KIFYvrsXpZXboet5ULrlvJ5Sd1p34D58TvbqRXv00A5P2pNaf030irtoX7HF/K34Az8nn/404AdDh0I+7Gb341jmZNtzPh40789fUee9Rv3GgHfXsv5tW3jt5V9u2+Czm2+wqWLG3KE0+dxKo1jSvzEqpMNt8+V+SV7ZrRIojPXLGLmV1jZlPNbOqqNUUV2JwDM3FcU5q3LqTrcVtL3T7htRb0O2/drvWLblxBwYZcrvv+UeSNak2XHlvJyYFN63P559hmPD1pDn/55DO2bcll/MstWLO8Dh++0ZzBP15VWZdUq13yw5kUFRnjP+wMQG6u06PbSu7//Xe49VcDOe3khfQ8dtmu+jk5xdz58w94bUw3lq+MJnSZOLUDV1z3Q6697VymzzyU/7jx41LPlW1KcrQc6Gd+1VWVD7SEaYRGAPQ+PjbaUM3MmdKYiX9rypTx3dmx3diyKZcHbjyM2x9ZyIY1uXwxoxHDRi7YVb9xk2J+8bvo7wR3GNKnO4d8azvTJjThkI47aN4q+gvgtEHrmTO1MQc1K2Lp1/W58tToCcP2rTn8+6lH89Q/5u7bGDkgZ/XLp0+vxdz+6/4Qno2tXtOIWXMPZuOmBgBM+aQDXTuvYcasdgD87Np/smRZU159a/cToE0FDXb9fnt8F35y2bTKu4gq5EBhFvcUKzIoppq5osb58Z3L+PGdUc/h038cxEtPtOH2RxYC8OFbzenz/Y17jCAXbMilfsNi6tZz3v5LS3r0jW6tD26/k7nTG7Fti1G/oTPjoyYcedwW+nx/I6M/3f1Z5uAuxyogVoDePZdw4eDZ/GLYALbv2P2f/9QZh/KjwbOpX6+QnYU5HNt9Oa+8GQXAf7/4Exo32snDj5+6x7FaNt/C2vXRyPMpvRezcEmzyruQKpbNt88VGRR3zVBBFAwvBi6twPNVmfdfb8GFN67Yo2zhvPo8+LPDMOBbR23j57+Neo3dTtzCd36wgRsGHEVuHadLj62cfdm+Azdy4Ib+7AOOO2YFzZps47n/e4lnXziei87/jHp1i7j/v8YBMHdeG4aP6EvB5vq88kZ3/vDAW+DG5OntmTy9A61bbubSC2axcHEzHvufN4Hdr96cN+hz+p60iKKiHDYV1OPBR06rysutPDX41jgTFs28U0EHj+ZJ+x27Z6jY54PtuN7HN/DJYzumqiLVTP8LhqSvJNXG5BmPs7FgyQFFtBbdDvYzRl2QviLwymmPTytrktnqqkKfKe7PDBUiUv1lc0+xygdaRKRmKZlkNlspKIpIIo5RWJy9Ay3Ze2UiUmGKsYyWVMysgZlNNrNPzWy2mf06lD9lZgvMbEZYeoZyM7PhIbH9TDM7MXasIWY2LyxDYuW9zGxW2Gd4SImaknqKIpKMl9vt83bgDHcvMLO6wEdm9nbY9h/u/tJe9c8myr/SlShV6eNAHzNrCQwDeketY5qZ5bn7ulDnamAS0fjGQOBtUlBPUUQSKcfEVe7uBWG1blhSvQ4zGHgm7DeRKOtfO2AAMM7d14ZAOA4YGLY1dfeJIcHVM8B56a5PQVFEEksQFFuXfMYblmvixzGzXDObAawkCmyTwqb7wi3yw2ZWMqtHWZ8OpypfXEp5Srp9FpFEHKMo84GW1aneUwxZ93qaWXPgVTPrAQwFlgP1iD4Bvh24+4AanYB6iiKSWHkMtMS5+3qifM8D3X1ZuEXeDvyJ3Zn9yvp0OFV5h1LKU1JQFJFE3MvnmaKZtQk9RMysIXAW8Hl4FkgYKT4P+CzskgdcEUah+wIb3H0ZUV7n/mbWwsxaAP2BsWHbRjPrG451BfB6uuvT7bOIJOblM/rcDng6TEidA7zo7m+a2btm1oZoCqMZwLWh/hhgEJAPbAGujNria83sHqL5FgDudve14ff1wFNAQ6JR55Qjz6CgKCKJlc+EEO4+EzihlPIzyqjvwA1lbBsFjCqlfCrQY989yqagKCKJlVNPsVpSUBSRRNyhqFhBUURkl2zO5qegKCKJOLp9FhGJye6ZtxUURSSxCpywv8opKIpIYrp9FhEJotHn7P0YTkFRRBLT7bOISIxun0VEAscUFEVE4rL47llBUUQScnB95icisptun0VEYmrl6LOZ/YEUjw7c/eYKaZGIVGu1+dvnqZXWChGpORyojUHR3Z+Or5tZI3ffUvFNEpHqrjxun82sAfABUJ8oFr3k7sPMrDMwGmgFTAMud/cdIdXpM0AvYA1wkbt/HY41FLgKKAJudvexoXwg8HsgF3jS3e9P16603+qY2SlmNgf4PKwfb2aPJbl4EckmhhdntqSxHTjD3Y8HehIlsO8LPAA87O5dgHVEwY7wz3Wh/OFQDzPrDlwMHAMMBB4L+aRzgUeBs4HuwCWhbkqZfMD4O2AAUWTG3T8FTs9gPxHJVp7hkuoQkYKwWjcsDpwBvBTKnybK6AcwOKwTtp8ZsvQNBka7+3Z3X0CU2OrksOS7+3x330HU+xyc7tIy+qrb3RftVVSUyX4ikoU8GmjJZAFam9nU2HJN/FChRzcDWAmMA74C1rt7YaiyGGgffrcHFgGE7RuIbrF3le+1T1nlKWXySs4iMzsVcDOrC9wCzM1gPxHJVpk/U1zt7r3LPIx7EdAz5H9+Feh2wG07QJn0FK8lSivYHlhKdO9fappBEaktLMMlM+6+HngPOAVobmYlHbYOwJLwewnQESBsb0b0WG9X+V77lFWeUtqg6O6r3f3f3L2tu7dx98vcfU26/UQkixVnuKRgZm1CDxEzawicRXQX+h5wQag2BHg9/M4L64Tt74Zc0HnAxWZWP4xcdwUmA1OArmbW2czqEQ3G5KW7tLS3z2Z2ONGQdl+iTvM/gZ+7+/x0+4pIFiq/9xTbAU+HUeIc4EV3fzO87TLazO4FPgFGhvojgWfNLB9YSxTkcPfZZvYiMAcoBG4It+WY2Y3AWKJXcka5++x0jcrkmeJfiIa1zw/rFwPPA30y2FdEslB5vKfo7jOBE0opn080crx3+TbgR2Uc6z7gvlLKxwBjkrQrk2eKjdz9WXcvDMufgQZJTiIiWaYcXsmprlJ9+9wy/HzbzO4gesfHgYtIGHlFJMvUxs/8iD6vcXYPIf00ts2BoRXVKBGp3qyG9gIzkerb586V2RARqSHcoLZPMmtmPYi+Hdz1LNHdn6moRolINVcbe4olzGwY0I8oKI4h+rj6I6LZKkSkNsrioJjJ6PMFwJnAcne/Ejie6E1yEamtauPoc8xWdy82s0Iza0r04XbHdDuJSJaqrZPMxkwNn+L8kWhEuoDoqxYRqaVq5ehzCXe/Pvx8wszeAZqGN9FFpLaqjUHRzE5Mtc3dp1dMk0SkuqutPcXfpthWMjtuufpyZiMGHNqzvA8rFWj5bY2rugmSwM78jOaVTq82PlN09+9VZkNEpIaowSPLmcjo5W0RkT0oKIqI7GZpJpCtyRQURSS5LO4pZpL32czsMjP777B+mJntMwGkiNQO5pkvKY9j1tHM3jOzOWY228xuCeV3mdkSM5sRlkGxfYaaWb6ZfWFmA2LlA0NZfpjqsKS8s5lNCuUvhLQEKWUyFPUYUTKZS8L6JqKZuEWktnLLbEmtELjN3bsTpTu5IZas/mF37xmWMbDfSe8fCMfqAqwDrkrXqEyCYh93vwHYBuDu64C00VZEslg5fPvs7stK3nd2901ESatS5WVOlPTezIzo1cGXwv5PA+elu7RMguLOEIkdogxcpM3TJSLZLMHtc2szmxpbrin1eGadiPK1TApFN5rZTDMbZWYtQlnSpPetgPXuXrhXeUqZBMXhREmqDzaz+4imDftNBvuJSDbyaPQ5kwVY7e69Y8uIvQ9nZgcBLwM/c/eNwOPAEUQ55peR+kOScpfJt8/Pmdk0ounDDDjP3edWeMtEpPoqp9FnM6tLFBCfc/dXANx9RWz7H4E3w2qq5Palla8BmptZndBbjNcvUyajz4cBW4A3iBJJbw5lIlJblcMzxfDMbyQw190fipW3i1U7H/gs/E6U9N7dHXiPaE5YgCHA6+kuLZP3FN9idwKrBkBn4AuiESARqYXKaUKI04DLgVlmNiOU3Uk0etyTKO58TUiat59J728HRpvZvcAnREE4pUxun4+Nr4fZc64vo7qISEbc/SN2ZwuNKzOFctKk9+4+n2h0OmOJv2hx9+lm1ifpfiKSRbL4i5ZMElfdGlvNAU4EllZYi0SkenN9+9wk9ruQ6BnjyxXTHBGpEWprTzG8tN3E3X9RSe0RkWrOqKUzb5e822Nmp1Vmg0SkBqiNQZHo/Z8TgRlmlgf8FdhcsrHkRUsRqWUymAGnJsvkmWIDojfDz2D3+4oOKCiK1Fa1dKDl4DDy/Bm7g2GJLP57QkTSqa09xVzgIEp/uTKL/0hEJK0sjgCpguIyd7+70loiIjVDLc7ml72JXUXkgNTW2+czK60VIlKz1Mag6O5rK7MhIlJz1PbP/EREdqvFzxRFRPZhZPeAg4KiiCSXxT3FTBJXiYjsIUE2v7KPYdbRzN4zszlmNtvMbgnlLc1snJnNC/9sEcrNzIaHxPYzw4TXJccaEurPM7MhsfJeZjYr7DM8pEBISUFRRJIrhxwtRFMR3ubu3YG+wA0hif0dwHh37wqMD+sQJbvvGpZriLL+YWYtgWFAH6JZtofF0qI+Dlwd229gukYpKIpIMslSnJZ9GPdl7j49/N4EzCXKyzyYKHE97JnAfjDwjEcmEmXqawcMAMa5+1p3XweMAwaGbU3dfWJIYvVM7Fhl0jNFEUku82eKrc1samx9RBm5nzsBJwCTgLbuvixsWg60Db/LSnqfqnxxKeUpKSiKSGIJvmhZ7e69Ux7L7CCi2fx/5u4b44/93N3NKvf7Gd0+i0hy5fNMETOrSxQQn4vN0bqiJPdz+OfKUL6E0pPepyrvUEp5SgqKIpJYOY0+G1Ee5rnu/lBsUx5R4nrYM4F9HnBFGIXuC2wIt9ljgf5m1iIMsPQHxoZtG82sbzjXFbFjlUm3zyKSjFNek8yeBlwOzDKzGaHsTuB+4EUzuwr4BrgwbBsDDALygS3AlRB9kmxm9wBTQr27Y58pXw88BTQE3g5LSgqKIpJIeSWucvePKPvjmH0mpAkjyDeUcaxRwKhSyqcCPZK0S0FRRJLL4i9aFBRFJDHz7I2KCooikoxmyRER2VNtnXlbRKRUmmRWRCROPUURkSCDF7NrMgVFEUlOQVFEJFJeL29XVwqKIpKYFWdvVFRQFJFk9J6ilObpSXPYWpBLcTEUFRo3nX0k3zlnPZfftpyOXbdz86CuzJvZaFf9zkdv5eYHFtO4SRHFxcZNg7qyc3sO/c5bx8U3rcQd1q6oywM3HcbGtfrXcqDaNingvoHjadV4K+7w8szuPPfJcZx15Fdcd8oUDm+1jkuf+1fmrDgYgEObbuS1fx/N1+uaAzBzWVvu/ft3ARh54eu0abyZbYXRv5drXzqHtVsb0a7JJu4e8B4tGm1lw7YG3DnmTFYUHFQl11vZ9ErOfjCzUcA5wEp3T/RBdk3xyx8dsUcA+/rzBtz9k07c/MDiPerl5Dq//MNC/vfmw5g/pyFNWhRStNPIyXWuu3spV/c7io1r63DVr5Zy7pWr+fNvD6nsS8k6RcXGb98/lbkr29Co7g5GX/YS//ymA/mrW3Jr3gD+66wP9tln8YamXPjshaUcDe4Y8/1dAbTEbd/9B2/MOZK8Od04ueNibv7OJP7z7X3mMchOWdxTrMj5FJ8igyQx2WRRfgMWf9Vgn/Je393EgrkNmD+nIQCb1tWhuNiw8MS6QcNiwGl8UDFrltet3EZnqdWbGzN3ZRsAtuysx4K1LTi4yWYWrG3B1+tapNk7M4e3WsekRdEcppMXted7Rywol+PWBOUxn2J1VWE9RXf/IORdyE5u/Ob5+eDw1rOtePu5VmVW7XD4dtyN+/7yFc1aFfH+683562MHU1Ro/OGODjzx7hds25LD0gX1eeTOtCkkJKFDm26k28GrmbWsbcp67Ztt4oXL/8rm7XV55OOTmb7k0F3b7hnwHkVu/H3e4YyY2AswvlzViu93mc9znxzHmV0WcFD9nTRrsI0N2/b9izGrOKAJISqOmV1DlK6QBjRKU7v6uPW8LqxZXpdmrXZy/+j5LMqvz2eTSn+elFvH6XHyZm4a1JXtW3O4/4WvmDezIbMmHsQ5V6zhhv5Hsuybetxw3xIuumklz/8+9f+8krmGdXfy0Llj+Z/3TmPzjnpl1lu1uTH9R1zOhm0NOPrgVfz+vLc5/6mL2byjHkPHnMnKgoNoVHcHD507ln/p/iVvzDmK375/KkPP/JBze3zB9MXtWLGpMcWeNq1wVsjmZ4pVno7A3Ue4e293712X+lXdnIyV3OZuWFOXj99pRrcTtpRZd9Wyusya2JiNa+uwfWsOU95tSpdjt3LEMVsBWPZNfcB4P6853Xtvrozm1wp1cop46NyxvDX3SMbnH56y7s6i3F09vLkr27BofTO+1WI9ACvD4MmWnfUY83lXehwSpQxZtbkxt+YN5KJnf8Twj/oAsGl7zflveH+VvKeYrbfPVR4Ua6L6DYto2Lho1+9e393E15+Xfcs0bUITOh29jfoNi8nJdY47pYCFXzZg9fK6HHbkNpq1LATgxNM3sWhelt96VRrn1/0nsGBNc56ddnza2i0abiUndH/aN9vIYc03sHhDU3KtmOYNo7+86uQU8d3DvyF/dUsAmjfcioURh5+cPJ1XP+tWQddSzbhnvqRhZqPMbKWZfRYru8vMlpjZjLAMim0bamb5ZvaFmQ2IlQ8MZflmdkesvLOZTQrlL5hZ2bcLQZXfPtdELdoUMmzk10B0a/zeqy2YOqEppw7cwPX3LqFZq0LueXYBX81uwH9eegQFG+rwyv+14Q9jvsTdmPxuEyaPbwrAcw+15cFX8yncaaxcUo8Hf9YxxZklUye0X86/HPMlX65qyYuXvwjA8I/6UC+3iKFnfESLhlt59PwxfL6qNde9fA69Oizl+lOnUFicg7tx799PZ+O2BjSss5Mn/vVN6uQUk2POpIUdeHnW0QCc1GEpN39nEu4wfUk77ht/elVecqUqx17gU8AjRInq4x529wf3OKdZd+Bi4BjgUODvZnZk2PwocBZRbucpZpbn7nOAB8KxRpvZE8BVwOOpGmReQQ9Mzex5oB/QGlgBDHP3kan2aWotvY/VklcassSy206t6iZIAl89+xBbly86oAefTZp38BNOvyWjuh++8ctpGeR97gS8WfLqnpndBRSUEhSHArj7/wvrY4G7wua73H1AvB5RAqxVwCHuXmhmp8TrlaUiR58vqahji0jVStBTbG1mU2PrI9x9RAb73WhmVwBTgdvcfR3QHpgYq7M4lAEs2qu8D9AKWO/uhaXUL5Nun0UkGQeKMo6Kq9P1FEvxOHBPONM9wG+BHyc8xn5TUBSRxCpyZNndV+w6j9kfgTfD6hIg/tC9QyijjPI1QHMzqxN6i/H6ZdLos4gkV06jz6Uxs3ax1fOBkpHpPOBiM6tvZp2BrsBkYArQNYw01yMajMkLeaLfAy4I+w8BXk93fvUURSSx8uopxgdkzWwxMAzoZ2Y9iW6fvwZ+CuDus83sRWAOUAjc4O5F4Tg3AmOBXGCUu88Op7gdGG1m9wKfACkHe0FBUUSSKsepw8oYkC0zcLn7fcB9pZSPAcaUUj4fODlJmxQURSQRAyzzgZYaR0FRRBIzTQghIhJo5m0Rkbj9H1muCRQURSSxmjoDTiYUFEUkOfUURUQC1+iziMiesjcmKiiKSHJ6JUdEJE5BUUQkcCCLE1cpKIpIIobr9llEZA/F2dtVVFAUkWR0+ywisifdPouIxGVxUFQ6AhFJKMNUBBkETjMbZWYrzeyzWFlLMxtnZvPCP1uEcjOz4SGx/UwzOzG2z5BQf56ZDYmV9zKzWWGf4WaWNr2rgqKIJFOSzS+TJb2ngIF7ld0BjHf3rsD4sA5wNlFelq7ANYSk9mbWkiiNQR+iWbaHlQTSUOfq2H57n2sfCooikpi5Z7Sk4+4fAGv3Kh4MPB1+Pw2cFyt/xiMTiTL1tQMGAOPcfW3IDz0OGBi2NXX3iSGJ1TOxY5VJzxRFJLnMnym2NrOpsfUR7j4izT5t3X1Z+L0caBt+t2ffpPft05QvLqU8JQVFEUnGgeKMg+Jqd++936dyd7PKnb1Rt88iklD5DbSUYUVJ7ufwz5WhfAmlJ71PVd6hlPKUFBRFJLmKDYp5RInrYc8E9nnAFWEUui+wIdxmjwX6m1mLMMDSHxgbtm00s75h1PmK2LHKpNtnEUnGgaLy+aTFzJ4H+hE9e1xMNIp8P/CimV0FfANcGKqPAQYB+cAW4EoAd19rZvcAU0K9u929ZPDmeqIR7obA22FJSUFRRBJy8PIJiu5+SRmbziylrgM3lHGcUcCoUsqnAj2StElBUUSSy+IvWhQURSSZZKPPNY6Coogkp56iiEiMgqKISOAORUVV3YoKo6AoIsmppygiEqOgKCJSwjX6LCKyi4OX08vb1ZGCoogkV06f+VVHCooikoy7UpyKiOxBAy0iIru5eooiIiUOaK7Eak9BUUSS0YQQIiK7OeD6zE9EJPDym2S2OlJQFJHEXLfPIiIxWdxTNK9Go0hmtoooUU22aQ2srupGSCLZ+u/sW+7e5kAOYGbvEP35ZGK1uw88kPNVtmoVFLOVmU09kITgUvn076z2Ut5nEZEYBUURkRgFxcoxoqobIInp31ktpWeKIiIx6imKiMQoKIqIxCgoViAzG2hmX5hZvpndUdXtkfTMbJSZrTSzz6q6LVI1FBQriJnlAo8CZwPdgUvMrHvVtkoy8BRQo142lvKloFhxTgby3X2+u+8ARgODq7hNkoa7fwCsrep2SNVRUKw47YFFsfXFoUxEqjEFRRGRGAXFirME6Bhb7xDKRKQaU1CsOFOArmbW2czqARcDeVXcJhFJQ0Gxgrh7IXAjMBaYC7zo7rOrtlWSjpk9D/wTOMrMFpvZVVXdJqlc+sxPRCRGPUURkRgFRRGRGAVFEZEYBUURkRgFRRGRGAXFGsTMisxshpl9ZmZ/NbNGB3Csp8zsgvD7yVSTVZhZPzM7dT/O8bWZ7ZP1razyveoUJDzXXWb2i6RtFNmbgmLNstXde7p7D2AHcG18o5ntVx5vd/+Ju89JUaUfkDgoitRECoo114dAl9CL+9DM8oA5ZpZrZv9rZlPMbKaZ/RTAIo+E+R3/DhxcciAzm2BmvcPvgWY23cw+NbPxZtaJKPj+PPRSv2Nmbczs5XCOKWZ2Wti3lZn9zcxmm9mTgKW7CDN7zcymhX2u2Wvbw6F8vJm1CWVHmNk7YZ8PzaxbufxpigT71bOQqhV6hGcD74SiE4Ee7r4gBJYN7n6SmdUHPjazvwEnAEcRze3YFpgDjNrruG2APwKnh2O1dPe1ZvYEUODuD4Z6fwEedvePzOwwoq92jgaGAR+5+91m9gMgk69BfhzO0RCYYmYvu/saoDEw1d1/bmb/HY59I1FCqWvdfZ6Z9QEeA87Yjz9GkVIpKNYsDc1sRvj9ITCS6LZ2srsvCOX9geNKnhcCzYCuwOnA8+5eBCw1s3dLOX5f4IOSY7l7WfMKfh/obrarI9jUzA4K5/hh2PctM1uXwTXdbGbnh98dQ1vXAMXAC6H8z8Ar4RynAn+Nnbt+BucQyZiCYs2y1d17xgtCcNgcLwJucvexe9UbVI7tyAH6uvu2UtqSMTPrRxRgT3H3LWY2AWhQRnUP512/95+BSHnSM8XsMxa4zszqApjZkWbWGPgAuCg8c2wHfK+UfScCp5tZ57Bvy1C+CWgSq/c34KaSFTPrGX5+AFways4GWqRpazNgXQiI3Yh6qiVygJLe7qVEt+UbgQVm9qNwDjOz49OcQyQRBcXs8yTR88LpIfnS/xHdEbwKzAvbniGaCWYP7r4KuIboVvVTdt++vgGcXzLQAtwM9A4DOXPYPQr+a6KgOpvoNnphmra+A9Qxs7nA/URBucRm4ORwDWcAd4fyfwOuCu2bjVI8SDnTLDkiIjHqKYqIxCgoiojEKCiKiMQoKIqIxCgoiojEKCiKiMQoKIqIxPx/TPCQlkHjn5YAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(classification_report(y_test, test_preds))\n",
    "\n",
    "#saving the results\n",
    "results_dict[\"Naive Bayes with Count Vectorization\"] = classification_report(y_test, test_preds)\n",
    "\n",
    "\n",
    "print(f\"Confusion matrix values: \\n{confusion_matrix(y_test, test_preds)}\")\n",
    "ConfusionMatrixDisplay.from_predictions(y_test, test_preds)\n",
    "\n",
    "plt.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "556c1e0f",
   "metadata": {},
   "source": [
    "The Naive Bayes model is giving a fairly good recall of .80 on the positive class we are predicting, but interestingly predicts a much higher number of the negative class incorrectly (notice the low precision).\n",
    "\n",
    "\n",
    "\n",
    "Precision  =  True Positives / (True Positives + False Positives)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "095c445d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Precision of our Naive Bayes model: \t0.64192839755388\n"
     ]
    }
   ],
   "source": [
    "precision = 21624 / (21624 + 12062)\n",
    "\n",
    "print(f\"Precision of our Naive Bayes model: \\t{precision}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5668f0c8-705f-4edb-aafc-303614118f6b",
   "metadata": {},
   "source": [
    "## 3.5.4 Random Forest Classifier with Count Vectorizations <a name=\"3.5.4\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd6d1b8c",
   "metadata": {},
   "source": [
    "#### 3.5.4.1 Training and Fitting the Model <a name=\"3.5.4.1\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "4bbfaf44-a84f-4997-959d-d9924043ec8e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(786016, 77986) <class 'scipy.sparse._csr.csr_matrix'> (87336,)\n",
      "(786016,) (87336,)\n"
     ]
    }
   ],
   "source": [
    "#Trying a Random Forest model with simple bag-of-words Count Vectorization\n",
    "\n",
    "count_vec = CountVectorizer()\n",
    "\n",
    "#fit the vectorizer on the training data\n",
    "count_vec.fit(X_train)\n",
    "\n",
    "#transform both the training and test data\n",
    "X_train_vec = count_vec.transform(X_train)\n",
    "X_test_vec = count_vec.transform(X_test)\n",
    "\n",
    "print(X_train_vec.shape, type(X_train_vec), X_test.shape)\n",
    "print(y_train.shape, y_test.shape)\n",
    "\n",
    "#Since Random Forest models use linear decision boundaries, it should not need scaled input data\n",
    "\n",
    "#using the default solver\n",
    "rfc = RandomForestClassifier(max_leaf_nodes=2000, min_samples_leaf=3)\n",
    "\n",
    "#train the lr model with the training data\n",
    "rfc.fit(X_train_vec, y_train)\n",
    "\n",
    "#predict on the training data\n",
    "train_preds = rfc.predict(X_train_vec)\n",
    "\n",
    "\n",
    "#predict on the test data\n",
    "test_preds = rfc.predict(X_test_vec)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83d8d574",
   "metadata": {},
   "source": [
    "#### 3.5.4.2 Evaluating the Model <a name=\"3.5.4.2\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "c7cd9f26-bbbd-4cce-97f4-3093d0581c2a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.85      0.97      0.90     60609\n",
      "           1       0.89      0.61      0.72     26727\n",
      "\n",
      "    accuracy                           0.86     87336\n",
      "   macro avg       0.87      0.79      0.81     87336\n",
      "weighted avg       0.86      0.86      0.85     87336\n",
      "\n",
      "Confusion matrix values: \n",
      "[[58543  2066]\n",
      " [10371 16356]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEGCAYAAADyuIefAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgcklEQVR4nO3dd5hV1b3/8fd3Cgww9BYQFFCiEuyIWK4/WxTNvUFzYzfwGK9oBGPiNYmaRJToNaZoNLYQ5QqJDWPDiog9V6SIoGBhYqHXoQ19Zr6/P/aaYYNTzoY5M3POfF7Ps585e+21116H8p219tp7LXN3REQkktPQFRARaUwUFEVEYhQURURiFBRFRGIUFEVEYvIaugJxnTrkeq+e+Q1dDUngszktG7oKksAWNrLNt9qelHHaia18dXFZSnlnztk6yd0H78n16lujCoq9euYzbVLPhq6GJHBa90MbugqSwHs+ZY/LWFVcxnuTeqSUN7/bvzrt8QXrWaMKiiKSCZwyL2/oSqSNgqKIJOJAOdn70oeCoogkVo5aiiIiADjOdnWfRUQiDpSp+ywisoPuKYqIBA6UZfHsWgqKIpJY9t5RVFAUkYQc1z1FEZEK7rA9e2OigqKIJGWUsUevTzdqCooikogD5WopiojsoJaiiEgQPbytoCgiAkRBcbtn7/zUCooikohjlGXxpP0KiiKSWLmr+ywiAuieoojILowy3VMUEYlEM28rKIqIAOBubPPchq5G2igoikhi5bqnKCISiQZa1H0WEQk00CIiUkkDLSIiuyjTw9siIhHH2O7ZGzqy95uJSFpk+0BL9n4zEUkLxyjz1LbamNmXZvahmX1gZjNCWgczm2xm88PP9iHdzOwuMysyszlmdnisnGEh/3wzGxZLPyKUXxTOrbVSCooiklg5OSltKTrR3Q919wFh/1pgirv3BaaEfYDTgb5hGw7cB1EQBUYBRwEDgVEVgTTkuTR23uDaKqOgKCKJuEOZ56S07aYhwLjweRxwZix9vEemAu3MrBtwGjDZ3YvdfQ0wGRgcjrVx96nu7sD4WFnV0j1FEUkkGmhJ+TW/ThXd4mCMu4/ZqTh4xcwc+Es41tXdl4bjy4Cu4fNewMLYuYtCWk3pi6pIr5GCoogklmCgZVWsW1yV49x9sZl1ASab2Sfxg+7uIWDWG3WfRSQRxyj31LZay3JfHH6uAJ4muie4PHR9CT9XhOyLgZ6x03uEtJrSe1SRXiMFRRFJrIyclLaamFkrM2td8Rk4FfgImAhUjCAPA54NnycCQ8Mo9CBgXehmTwJONbP2YYDlVGBSOLbezAaFUeehsbKqpe6ziCQSrftcJ+2prsDT4SmZPOARd3/ZzKYDE8zsEuAr4JyQ/0XgDKAI2ARcDODuxWb2G2B6yDfa3YvD5yuAh4AWwEthq5GCoogkZHWyHIG7fw4cUkX6auDkKtIdGFFNWWOBsVWkzwD6J6mXgqKIJBItcapJZkVEgGjm7TrqPjdKCooikpjmUxQRCaL5FDV1mIhIoJm3RUQqRY/kqKUoIgIkfvc54ygoikhiWqNFRCSIpg5T91lEpJLuKYqIBNEsOeo+i4gAFa/5KSgKMHRgP1oUlpGTA7l5zt0vf8a/PmrBXdf2YNuWHHLznJG3LuKAwzYx+/8KufHi3nyj5zYAjj1jLRddvbyyrLIyuHLwN+nYbTu/Gf8FALdf3ZPP5rQEh736bOWaPy2gRavyBvmu2aZz92387M4FtOtcCg4v/r0jzzzYmdbtSrn+/q/o2mMbyxc145bL9qFkXfTf4uCjS7h89GLy8px1xXn87D/3A6BVmzJ++oeF9DpgC+7R39vHM1s15NerZ2op7jYzGwzcCeQCD7j7b9N5vfrwuyeKaNuxrHL/gZu7cdHVyzjypA1Mm9KaB2/uzu+fLAKg/1EllQFvV8880JmefbeyqWTHP67LblpMq9ZREPzLjd2ZOLYT5165osrzJZmyUmPM6O4UfdiSFq3KuPvlz3j/rdZ8+9xiZr1TyIS7u3LOyOWcO3IFD97SnVZtyhh56yJ+eWEfVi5uRtuO2yvL+tHoxcx4ozU3D+9FXn45zVvU68TQjUI2v9GStnBvZrnAPUQrcPUDzjezfum6XkMxg40bome2Nq7PpUPX7bWcASuX5DNtShtOv2D1TukVAdEdtm7JIYv/3dW74hX5FH3YEoDNG3NZWFRAp27bOfq09bw6oQMAr07owNGD1wNw4llr+OeLbVm5uBkA61bnA9CydRkHDdrIy49E55Ruz2Hj+ux9Zq8qFaPPdbHEaWOUzpbiQKAozJmGmT1GtBrXvDReM73Muf78fcHgOz9YzRkXreby0Yu5/vx9+evo7rjDHRPnV2b/eGYrLj9lfzp23c6lNyyh1/5bALh/1F7816+WsKnk6/+Z/vCTnkx/rQ17f3MLw2+odeZ02Q1de2xj3/6b+eT9lrTvtJ3iFVHAK16RR/tO0S+1Hn22kpvv/O4fRbQsLOeZBzrx6j868I29t7FudS7/fcdC+nxrM/PntOS+X3dn6+amFRizufuczm9W3QpbOzGz4WY2w8xmrFxdtuvhRuX2Z4q455XPuOXhz5n4UCc+nNqK58d14rKbFvPwzHlcduMSbr96bwD2O2gTf5s2j/tf/ZQhP1zJTT/sDcDUyW1o16mUvgdvrvIa1/xpIY/Mmsvefbfy5sT2VeaR3VfQsoxfP/Al99/QvYpfSoaH1k1untP3oM38+ge9uf6CPlzwk+Xs1WcrubnOfgdt5vnxHRlx6v5s2ZTDuSOb1i2OulyjpTFq8HDv7mPcfYC7D+jcsXH/tu3ULWpFtOtUyrGD1/HJrJZMfqIDx52xDoDj/2Mtn30QddFatS6vHCQZePIGyrYb61bnMm96K6a+0oahA/tx64/2YfY7rblt5N47XSc3F04YsoZ3Xmxbj98u++XmOb9+4Etee6o9/3ypHQBrVuXToUv099qhy3bWro46TyuX5jPzzdZs3ZzL+uI8PnyvkD79NrNqaT4rl+bz6axoYOWd59uy30FV/4LLVg6Uek5KWyZKZ62rW2ErI23ZlFM5KLJlUw4z32xNrwO20LHrdua8WwjAB+8U0r33ViDqinm4//7JrJaUl0ObDmX88PqlPDxzHuOnzeO6+77ikOM28Iu7F+AOi7+I7l+5w7uT2tJz3631/0WzlnP1HxeycH4BT43pXJk69ZU2nHJOtJzHKecU8+6kNgC8+3JbvnXkRnJyneYtyjngsE0smN+cNSvzWbWkGT32jW6FHPpvJSyYX1D/X6eBlXtOSlsmSuc9xelAXzPrTRQMzwMuSOP10mrNyjxuuiTqApeVwolnreXIEzfQouVC7rthL8rKjGbNy/nJ76M7Bm8/347nx3ckNw+aF5Rz3X1fYjX0JtzhD1ftzaaSXNyhT7/NXPnbRdWfIIl8a+BGTjl7DZ/PK+DeyZ8C8L+3duPxu7vwy/u/YvB5xaxYHD2SA7CwqIAZb7Tm/imf4uXGy4904KtPWwBwz6/24hd3LyAv31m2oBl//GnPaq+blTK4a5wKc0/f4wRmdgbwJ6JHcsa6+y015R9wSIFPm9TE/oFluNO6H9rQVZAE3vMprPfiPYpo7Q/o4ieN/X5KeZ869r6Z7j5gT65X39L6nKK7v0i0LKGIZJFsbinqjRYRSUSTzIqIxDhGaXlmDqKkQkFRRBLL5tf8FBRFJBlX91lEpJLuKYqI7EJBUUQkcIwyDbSIiOyQzQMt2RvuRSQtPAy01NUsOWaWa2azzOz5sN/bzN4zsyIze9zMmoX05mG/KBzvFSvjupD+qZmdFksfHNKKzOzaVOqjoCgiiblbSluKrgI+ju3fBtzh7vsBa4BLQvolwJqQfkfIR5i8+jzgW8Bg4N4QaHdromsFRRFJqO7mUzSzHsB3gAfCvgEnAf8IWcYBZ4bPQ8I+4fjJIf8Q4DF33+ruXwBFRJNcV0507e7bgIqJrmukoCgiiSVoKXaqmEQ6bMN3KepPwM+BihXaOgJr3b007Mcnp66cuDocXxfyVzehdUoTXe9KAy0ikog7lJWn3DVeVd0sOWb278AKd59pZifUUfX2mIKiiCRWR6PPxwLfDVMMFgBtiFb/bGdmeaE1GJ+cumLi6kVmlge0BVZT84TWiSe6VvdZRBJx6magxd2vc/ce7t6LaKDkNXe/EHgdqJiwcRjwbPg8MewTjr/m0YSwE4Hzwuh0b6AvMI3YRNdhBPu8kLdGaimKSEJpn3n7F8BjZnYzMAt4MKQ/CPzNzIqAYqIgh7vPNbMJRCuFlgIj3L0MwMxGApPYMdH13NourqAoIonV9YT97v4G8Eb4/DnRyPGuebYAZ1dz/i3A12b2352JrhUURSSxBM8gZhwFRRFJJBp9zt7hCAVFEUksjevdNTgFRRFJTN1nEZHASfRec8ZRUBSRxLK496ygKCIJOXjqr/llHAVFEUlM3WcRkZgmOfpsZn+mhlsH7v7jtNRIRBq1inefs1VNLcUZ9VYLEckcDjTFoOju4+L7ZtbS3Telv0oi0thlc/e51nd1zOxoM5sHfBL2DzGze9NeMxFppAwvT23LRKm8wPgn4DSiyRxx99nA8Wmsk4g0dp7iloFSGn1294XR+jCVytJTHRFp9LzpDrRUWGhmxwBuZvl8fTlCEWlqMrQVmIpUus+XAyOIVsFaAhwa9kWkybIUt8xTa0vR3VcBF9ZDXUQkU5TXniVTpTL63MfMnjOzlWa2wsyeNbM+9VE5EWmEKp5TTGXLQKl0nx8BJgDdgO7AE8Cj6ayUiDRu7qltmSiVoNjS3f/m7qVh+zvRGq0i0lQ1xUdyzKxD+PiSmV0LPEb0Nc8l4epYIpJlMrRrnIqaBlpmEgXBim9/WeyYA9elq1Ii0rhZhrYCU1HTu8+967MiIpIh3CBDX+FLRUpvtJhZf6AfsXuJ7j4+XZUSkUauKbYUK5jZKOAEoqD4InA68A6goCjSVGVxUExl9Pn7wMnAMne/GDgEaJvWWolI49YUR59jNrt7uZmVmlkbYAXQM831EpHGqqlOMhszw8zaAX8lGpEuAd5NZ6VEpHHL5tHnWrvP7n6Fu6919/uBbwPDQjdaRJqqOug+m1mBmU0zs9lmNtfMbgrpvc3sPTMrMrPHzaxZSG8e9ovC8V6xsq4L6Z+a2Wmx9MEhrSg8b12raoOimR2+6wZ0APLCZxFposxT22qxFTjJ3Q8hmn1rsJkNAm4D7nD3/YA1wCUh/yXAmpB+R8iHmfUDzgO+BQwG7jWzXDPLBe4hGhzuB5wf8taopu7zH2s45sBJtRWe1GdFHRj8HU3Ik0k2nVXY0FWQBMpfm1o3BdXBPUV3d6LbcQD5YauILReE9HHAjcB9wJDwGeAfwN0WzX49BHjM3bcCX5hZETAw5Cty988BzOyxkHdeTfWq6eHtE1P/eiLSZCQbWe5kZvGVQce4+5iKndCamwnsR9Sq+xew1t1LQ5ZFRHO5En4uBHD3UjNbB3QM6fFoHz9n4S7pR9VW4ZQe3hYR2UnqQXGVuw+othj3MuDQMJj7NHDAHtdtDykoikhiVseTzLr7WjN7HTgaaGdmeaG12ANYHLItJnoccJGZ5RE9L706ll4hfk516dVK5eFtEZGd1c3oc+fQQsTMWhA93fIx8DrRSyMAw4Bnw+eJYZ9w/LVwX3IicF4Yne4N9AWmAdOBvmE0uxnRYMzE2r5aKq/5GdFyBH3cfbSZ7Q18w92n1XauiGSfFEeWU9ENGBfuK+YAE9z9+bDO/GNmdjMwC3gw5H8Q+FsYSCkmCnK4+1wzm0A0gFIKjAjdcsxsJDAJyAXGuvvc2iqVSvf5XqIVGU4CRgMbgCeBI1P62iKSfepm9HkOcFgV6Z+zY/Q4nr4FOLuasm4Bbqki/UUSzv+aSlA8yt0PN7NZ4SJrKh6mFJEmKovfaEklKG4PzVuH6D4AWb2Wl4jUJptf80slKN5FNFTexcxuIbrB+au01kpEGi+v+9HnxiSVdZ8fNrOZRNOHGXCmu3+c9pqJSOPVlFuKYbR5E/BcPM3dF6SzYiLSiDXloAi8wI4FrAqA3sCnRC9fi0gT1KTvKbr7QfH9MEPOFWmrkYhIA0r8mp+7v29mtb5ULSJZrCm3FM3s6thuDnA4sCRtNRKRxq2pjz4DrWOfS4nuMT6ZnuqISEZoqi3F8NB2a3e/pp7qIyKNnNFEB1oqpu4xs2Prs0IikgGaYlAkmnrncOADM5sIPAFsrDjo7k+luW4i0hjV3Sw5jVIq9xQLiCZyPIkdzys6oKAo0lQ10YGWLmHk+SN2BMMKWfx7QkRq01RbirlAITsHwwpZ/EciIrXK4ghQU1Bc6u6j660mIpIZkq3ml3FqCop7PrWuiGSlptp9PrneaiEimaUpBkV3L67PiohI5mjqr/mJiOzQhO8pioh8jZHdAw4KiiKSnFqKIiI7NNXRZxGRqikoiogEmmRWRGQXaimKiOyge4oiInEKiiIiO2RzSzGnoSsgIhnGiSaZTWWrgZn1NLPXzWyemc01s6tCegczm2xm88PP9iHdzOwuMysyszlhDfqKsoaF/PPNbFgs/Qgz+zCcc5eZ1frcuYKiiCRSsXBVKlstSoH/dvd+wCBghJn1A64Fprh7X2BK2Ac4HegbtuHAfRAFUWAUcBQwEBhVEUhDnktj5w2urVIKiiKSnKe41VSE+1J3fz983gB8DOwFDAHGhWzjgDPD5yHAeI9MBdqZWTfgNGCyuxe7+xpgMjA4HGvj7lPd3YHxsbKqpXuKIpKYeco3FTuZ2YzY/hh3H/O18sx6AYcB7wFd3X1pOLQM6Bo+7wUsjJ22KKTVlL6oivQaKSiKSDLJZslZ5e4DaspgZoXAk8BP3H19/Lafu7tZ/Q7rqPssIonV0T1FzCyfKCA+HFs2eXno+hJ+rgjpi4GesdN7hLSa0ntUkV4jBUURSczKU9tqLCNqEj4IfOzut8cOTQQqRpCHAc/G0oeGUehBwLrQzZ4EnGpm7cMAy6nApHBsvZkNCtcaGiurWuo+i0hyddOhPRb4AfChmX0Q0q4HfgtMMLNLgK+Ac8KxF4EzgCJgE3AxRKsEmNlvgOkh3+jYygFXAA8BLYCXwlYjBUURSSbFrnGtxbi/Q/Xz1X5tjagwgjyimrLGAmOrSJ8B9E9SLwVFEUkui99oUVAUkUQqHt7OVgqKIpKYlWdvVFRQFJFktJqf/PSqqRw1cDFr1xZw+Yjv7HF5p5z8Oeef+xEAjz7en1en9Nnp+I03vMk3upbUybWasmsveoNj+i9gzYYWDLvl7Mr0//x/H3HW8XMp9xze/agn9z0ziAP3WcHPLngbAMMZ++IRvD27NwATRj/Cpi35lHsOZWXGpb/7Xo1lNQWaeXs3mNlY4N+BFe6eaPSnsZn8ah+ee/6bXHP1u4nO+92tr/LHOwaxfEVhZVph4VYuvOBDrrxqMGD8+c6XmPpeD0pKmgFw7DEL2bxZv6vqwktT9+epN/vzy6GvV6Yd1ncJxx38FRff+n22l+bSrnAzAJ8v6cClt51FWXkOHdts4n+v/wf/9+E+lJVHj/Jeded/sG5jwU7lV1dWk5DFLcV0Prz9ECnMSJEJPprbhQ0bmu2U1u0bG7h59Ov8+c6X+MNtk+nRY11KZQ04YimzZnWjpKQ5JSXNmDWrGwOOWAJAQcF2vnfmJzz6WEb/Dmk0Zhd1Y/3G5julnXn8PP7+yiFsL80FYG1JCwC2bs+rDIDN8ktxr31l4+rKagrq6o2WxihtTRJ3fyu85J2VfnzlNP58z5EsWdKG/fdfxcgrZnDt9V97tOprOnbczMqVLSv3V61qQceOUQtj6A/m8OTTB7B1a27a6t3U9eyyjkP2W8bw705n2/Y87nnqKD5Z0AWAfr1WcO1Fb9K1wwZuHndiZZB0N24f+QKO8ew7B/LcPw+stays5kDqE0JknAbvp5nZcKK50Sho1raBa5OagoLt9DtwFb+87p3KtPz86CbLt0/5F2cO+RSA7t1KGH3TG5SW5rBsWSG/ueX4asvs02cN3buVMOavR9C1S0l6v0ATlptTTpuWW7ns92dy4D4ruemSKZw76jzAmPdlF4befDb7dF3D9UPf4L25PdlWmseI27/LqnWtaFe4mTuufIEFy9sxu6hbjWVlO91TTKMwjdAYgDatumfEr58cg40b8xlx5RlfOzb51X2Z/Oq+QNX3FFevbsHBB62o3O/UaTNzPuzCgQesou9+xYwb+yw5ueW0a7uV3936Kj+/7pT0f6EmZOXaVrz5QW/A+PirLrhDu8ItO3V9v1rens1b8+ndfQ2fLujMqnWtgKh7/NbsXhy4zwpmF3VLqaxslO3PKWpCiN2waXM+y5YX8m/HLQgpTu/ea1I6d8bMbhx+2FIKC7dRWLiNww9byoyZ3Xjhxb5cOPQshv1wCNf87NssXtxaATEN3p7di8O/Gd3D7dllLXl55awtKaBbx/Xk5kTNn64dNrBP17UsW92agmbbadF8GwAFzbZz5IGL+XxphxrLynruqW8ZqMFbipng2p//k4MPWk6bNlv527in+fvDB3Pb74/hyhHTOf/cj8jNK+fNt/bhiy/a11pWSUlzHnmsP3fd8TIADz/an5KS5rWcJbtj1MVTOKzvEtoWbuHJmx9m7AtH8MK7+3PdRW8y7pdPUFqaw/+MPwEwDt53GReeOpvSshy8HG5//DjWbYyC5f8MfwWA3Fxn8vR9mTYvmqWqurKagmxuKZqnKZqb2aPACUAnYDkwyt0frOmcNq26+6ADhqelPpIeG3sV1p5JGo3Zr91JyZqFexS5W7fr4Ycdf1VKed9+7ucza5tktrFJ5+jz+ekqW0QaVja3FNV9FpFkHCjL3qiooCgiiamlKCISl6Ejy6lQUBSRxNRSFBGpoKnDRER2MMA00CIisoPpnqKISKDus4hIXOa+15wKBUURSUyjzyIicWopiogErtFnEZGdZW9MVFAUkeT0SI6ISFwWB0UtRyAiyThQnuJWCzMba2YrzOyjWFoHM5tsZvPDz/Yh3czsLjMrMrM5ZnZ47JxhIf98MxsWSz/CzD4M59xlZrVOsKugKCKJGI55alsKHuLr68NfC0xx977AlLAPcDrQN2zDgfsgCqLAKOAoYCAwqiKQhjyXxs6rdS16BUURSa68PLWtFu7+FlC8S/IQYFz4PA44M5Y+3iNTgXZm1g04DZjs7sXuvgaYDAwOx9q4+1SP1l0ZHyurWrqnKCLJVHSfU9PJzGbE9seEZY1r0tXdl4bPy4Cu4fNewMJYvkUhrab0RVWk10hBUUQSSzD6vGpPFq5ydzer3/dn1H0WkeTSu+7z8tD1JfxcEdIXAz1j+XqEtJrSe1SRXiMFRRFJKMWAuPtBcSJQMYI8DHg2lj40jEIPAtaFbvYk4FQzax8GWE4FJoVj681sUBh1Hhorq1rqPotIMnW4ml98fXgzW0Q0ivxbYIKZXQJ8BZwTsr8InAEUAZuAiwHcvdjMfgNMD/lGu3vF4M0VRCPcLYCXwlYjBUURSayu3mipYX34k6vI68CIasoZC4ytIn0G0D9JnRQURSS5LH6jRUFRRJJxoFxBUUQk0MzbIiI7U1AUEQkcKEv9lZZMo6AoIgk5uIKiiMgO6j6LiAQafRYR2YVaiiIiMQqKIiKBO5SVNXQt0kZBUUSSU0tRRCRGQVFEpIJr9FlEpJKD6+FtEZEYveYnIhK4p7R8aaZSUBSR5DTQIiKyg6ulKCJSQZPMiojsoAkhRER2cMD1mp+ISOCaZFZEZCeu7rOISEwWtxTNG9EokpmtBL5q6HqkQSdgVUNXQhLJ1r+zfdy9854UYGYvE/35pGKVuw/ek+vVt0YVFLOVmc1w9wENXQ9Jnf7Omq6chq6AiEhjoqAoIhKjoFg/xjR0BSQx/Z01UbqnKCISo5aiiEiMgqKISIyCYhqZ2WAz+9TMiszs2oauj9TOzMaa2Qoz+6ih6yINQ0ExTcwsF7gHOB3oB5xvZv0atlaSgoeAjHrYWOqWgmL6DASK3P1zd98GPAYMaeA6SS3c/S2guKHrIQ1HQTF99gIWxvYXhTQRacQUFEVEYhQU02cx0DO23yOkiUgjpqCYPtOBvmbW28yaAecBExu4TiJSCwXFNHH3UmAkMAn4GJjg7nMbtlZSGzN7FHgX2N/MFpnZJQ1dJ6lfes1PRCRGLUURkRgFRRGRGAVFEZEYBUURkRgFRRGRGAXFDGJmZWb2gZl9ZGZPmFnLPSjrITP7fvj8QE2TVZjZCWZ2zG5c40sz+9qqb9Wl75KnJOG1bjSza5LWUWRXCoqZZbO7H+ru/YFtwOXxg2a2W+t4u/t/ufu8GrKcACQOiiKZSEExc70N7BdacW+b2URgnpnlmtnvzWy6mc0xs8sALHJ3mN/xVaBLRUFm9oaZDQifB5vZ+2Y228ymmFkvouD709BK/Tcz62xmT4ZrTDezY8O5Hc3sFTOba2YPAFbblzCzZ8xsZjhn+C7H7gjpU8ysc0jb18xeDue8bWYH1MmfpkiwWy0LaVihRXg68HJIOhzo7+5fhMCyzt2PNLPmwD/N7BXgMGB/orkduwLzgLG7lNsZ+CtwfCirg7sXm9n9QIm7/yHkewS4w93fMbO9id7aORAYBbzj7qPN7DtAKm+D/DBcowUw3cyedPfVQCtghrv/1MxuCGWPJFpQ6nJ3n29mRwH3Aiftxh+jSJUUFDNLCzP7IHx+G3iQqFs7zd2/COmnAgdX3C8E2gJ9geOBR929DFhiZq9VUf4g4K2Ksty9unkFTwH6mVU2BNuYWWG4xvfCuS+Y2ZoUvtOPzeys8LlnqOtqoBx4PKT/HXgqXOMY4InYtZuncA2RlCkoZpbN7n5oPCEEh43xJOBKd5+0S74z6rAeOcAgd99SRV1SZmYnEAXYo919k5m9ARRUk93Dddfu+mcgUpd0TzH7TAJ+ZGb5AGb2TTNrBbwFnBvuOXYDTqzi3KnA8WbWO5zbIaRvAFrH8r0CXFmxY2aHho9vAReEtNOB9rXUtS2wJgTEA4haqhVygIrW7gVE3fL1wBdmdna4hpnZIbVcQyQRBcXs8wDR/cL3w+JLfyHqETwNzA/HxhPNBLMTd18JDCfqqs5mR/f1OeCsioEW4MfAgDCQM48do+A3EQXVuUTd6AW11PVlIM/MPgZ+SxSUK2wEBobvcBIwOqRfCFwS6jcXLfEgdUyz5IiIxKilKCISo6AoIhKjoCgiEqOgKCISo6AoIhKjoCgiEqOgKCIS8/8BBzAESRrhH9cAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(classification_report(y_test, test_preds))\n",
    "\n",
    "#saving the results\n",
    "results_dict[\"Random Forest with Count Vectorization\"] = classification_report(y_test, test_preds)\n",
    "\n",
    "\n",
    "print(f\"Confusion matrix values: \\n{confusion_matrix(y_test, test_preds)}\")\n",
    "ConfusionMatrixDisplay.from_predictions(y_test, test_preds)\n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e484393b",
   "metadata": {},
   "source": [
    "##### Summary\n",
    "It seems like this model is STONGLY overfitting our training data. The recall on the test set for our positive class (1) is only .61 which is very poor. We are seeing a very high number of False Negatives (~10000) which results in the poor recall for our positive class. This could possibly be due to leaf sizes and might also be mitigated by adjusting the proportion of our positive class in the training data (under or over sampling)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85bdf5f0",
   "metadata": {},
   "source": [
    "## 3.6 Conclusions <a name=\"3.6\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44c21fd2",
   "metadata": {},
   "source": [
    "### 3.6.1 Preprocessing and Training Summary <a name=\"3.6.1\"><a/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "4968db06",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Logistic Regression with Count Vectorization\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.90      0.92      0.91     60609\n",
      "           1       0.81      0.76      0.78     26727\n",
      "\n",
      "    accuracy                           0.87     87336\n",
      "   macro avg       0.85      0.84      0.85     87336\n",
      "weighted avg       0.87      0.87      0.87     87336\n",
      "\n",
      "Logistic Regression with TFIDF Vectorization\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.92      0.94      0.93     60609\n",
      "           1       0.86      0.80      0.83     26727\n",
      "\n",
      "    accuracy                           0.90     87336\n",
      "   macro avg       0.89      0.87      0.88     87336\n",
      "weighted avg       0.90      0.90      0.90     87336\n",
      "\n",
      "Naive Bayes with Count Vectorization\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.90      0.79      0.84     60609\n",
      "           1       0.63      0.81      0.71     26727\n",
      "\n",
      "    accuracy                           0.80     87336\n",
      "   macro avg       0.77      0.80      0.78     87336\n",
      "weighted avg       0.82      0.80      0.80     87336\n",
      "\n",
      "Random Forest with Count Vectorization\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.85      0.97      0.90     60609\n",
      "           1       0.89      0.61      0.72     26727\n",
      "\n",
      "    accuracy                           0.86     87336\n",
      "   macro avg       0.87      0.79      0.81     87336\n",
      "weighted avg       0.86      0.86      0.85     87336\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for model in results_dict:\n",
    "    print(model, results_dict[model], sep=\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b5eec50",
   "metadata": {},
   "source": [
    "#### Model Results on Test Set:\n",
    "Examining the four models we built, using the Recall on the positive class as our primary metric, it looks like the most successful model was the Logistic Regression with TFIDF. Both it and the Naive Bayes model had a recall of around .80 on the positive class. The Naive Bayes overall had worse metrics on the other (\"0\") class, incorrectly predicting a higher number of them as the positive class."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "afcbf3fc",
   "metadata": {},
   "source": [
    "### 3.6.2 Next Steps <a name=\"3.6.2\"><a/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88983a10",
   "metadata": {},
   "source": [
    "1. Check if under-sampling the majority class (\"0\"s) or over-sampling the minority class (\"1\"s) can improve our results.\n",
    "\n",
    "2. Test some other models such as XGBoost or SVMs. XGBoost could give better results than the strongly-overfit Random Forest model.\n",
    "\n",
    "3. We should try using cross-validation to get a better sense of how well our models will generalize.\n",
    "\n",
    "4. One of the challenges in this notebook was how much the models struggled with the larger vectors created by the TFIDF model. We could further experiment with TFIDF parameters or possibly train on a smaller sample size.\n",
    "\n",
    "5. Hyper-parameter tuning with grid-search or bayesian search optimization could be used for our best/most promising models in the next notebook.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10.0 ('pyspark')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  },
  "vscode": {
   "interpreter": {
    "hash": "1d73df188c3ed28f9117694899638a2078581279f5400990dc36b7edf848ab19"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}