examples/PyCaret 2 Text Similarity Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text Similarity Classification\n",
    "<i>Last Update: 08/02/2020</i><br>\n",
    "<i>PyCaret Version: 2.0</i><br>\n",
    "<i>Author: Alexandre Farias</i><br>\n",
    "<i>Email: afarias@tuta.io</i>\n",
    "\n",
    "# Introduction\n",
    "This task consists in compare two sentences present on the dataset and identify if both have the same meaning.<br>\n",
    "An Exploratory Data Analysis is made to gain insights about the data, a Topic Modelling to get the features and the classification step, these last two steps are made with the Python Module PyCaret.<br>\n",
    "Importing the requires modules, PyCaret is imported later to avoid conflicts on the experiments.\n",
    "The dataset used on this work is the [Text Similarity](https://www.kaggle.com/rishisankineni/text-similarity) hosted on Kaggle, only using the data present on the train set, since the test set has a few samples doesn't has the labels do validate the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "        <script type=\"text/javascript\">\n",
       "        window.PlotlyConfig = {MathJaxConfig: 'local'};\n",
       "        if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}\n",
       "        if (typeof require !== 'undefined') {\n",
       "        require.undef(\"plotly\");\n",
       "        requirejs.config({\n",
       "            paths: {\n",
       "                'plotly': ['https://cdn.plot.ly/plotly-latest.min']\n",
       "            }\n",
       "        });\n",
       "        require(['plotly'], function(Plotly) {\n",
       "            window._Plotly = Plotly;\n",
       "        });\n",
       "        }\n",
       "        </script>\n",
       "        "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Standard\n",
    "import pandas as pd\n",
    "# Plots\n",
    "from plotly.offline import init_notebook_mode, iplot\n",
    "import plotly.graph_objects as go\n",
    "from plotly.subplots import make_subplots\n",
    "init_notebook_mode(connected=True)\n",
    "import seaborn as sns\n",
    "# Sklearn tools\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.metrics import *\n",
    "# PATH and setup\n",
    "import os\n",
    "os.chdir(\"..\")\n",
    "PATH = os.getcwd()+os.sep\n",
    "RANDOM_SEED = 42\n",
    "K_FOLDS = 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The helper functions used to plots, data sampling and scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Dataset Sampling\n",
    "def data_sampling(dataset, frac: float, random_seed: int):\n",
    "    data_sampled_a = dataset.sample(frac=frac,\n",
    "                                    random_state=random_seed)\n",
    "    data_sampled_b =  dataset.drop(data_sampled_a.index).\\\n",
    "    reset_index(drop=True)\n",
    "    data_sampled_a.reset_index(drop=True, inplace=True)\n",
    "    return data_sampled_a, data_sampled_b  \n",
    "## Pie Chart\n",
    "def pie_plot(data, plot_title: str):\n",
    "    trace = go.Pie(labels=data.index, values=data.values)\n",
    "    layout = go.Layout(autosize=False,\n",
    "    title={\"text\" : plot_title,\n",
    "            \"y\" : 0.9,\n",
    "            \"x\" : 0.5,\n",
    "            \"xanchor\" : \"center\",\n",
    "            \"yanchor\" : \"top\"})\n",
    "    fig = go.Figure(data=trace, layout=layout)\n",
    "    fig.update_traces(textfont_size=14,\n",
    "                      marker=dict(line=dict(color=\"#000000\",\n",
    "                      width=2)))\n",
    "    fig.update_yaxes(automargin=True)\n",
    "    iplot(fig)\n",
    "## Histogram subplots\n",
    "def histogram_subplot(dataset_a, dataset_b, feature_a: str,\n",
    "feature_b: str, title: str, title_a:\n",
    "str, title_b: str):\n",
    "    fig = make_subplots(rows=1, cols=2, subplot_titles=(\n",
    "    title_a,\n",
    "    title_b\n",
    "    )\n",
    "    )\n",
    "    fig.add_trace(go.Histogram(x=dataset_a[feature_a],\n",
    "    showlegend=False), row=1, col=1)\n",
    "    fig.add_trace(go.Histogram(x=dataset_b[feature_b],\n",
    "    showlegend=False), row=1, col=2)\n",
    "    fig.update_layout(template=\"simple_white\")\n",
    "    fig.update_layout(autosize=False,\n",
    "    title={\"text\" : title,\n",
    "            \"y\" : 0.9,\n",
    "            \"x\" : 0.5,\n",
    "            \"xanchor\" : \"center\",\n",
    "            \"yanchor\" : \"top\"},\n",
    "            yaxis={\"title\" : \"<i>Frequency</i>\"})\n",
    "    fig.update_traces(marker=dict(line=dict(color=\"#000000\",\n",
    "    width=2)))\n",
    "    fig.update_yaxes(automargin=True)\n",
    "    iplot(fig)\n",
    "# Calculate scores with Test/Unseen labeled data\n",
    "def test_score_report(data_unseen, predict_unseen):\n",
    "    le = LabelEncoder()\n",
    "    data_unseen[\"Label\"] = le.fit_transform(data_unseen.same_security.values)\n",
    "    data_unseen[\"Label\"] = data_unseen[\"Label\"].astype(int)\n",
    "    accuracy = accuracy_score(data_unseen[\"Label\"],\n",
    "    predict_unseen[\"Label\"])\n",
    "    roc_auc = roc_auc_score(data_unseen[\"Label\"],\n",
    "    predict_unseen[\"Label\"])\n",
    "    precision = precision_score(data_unseen[\"Label\"],\n",
    "    predict_unseen[\"Label\"])\n",
    "    recall = recall_score(data_unseen[\"Label\"],\n",
    "    predict_unseen[\"Label\"])\n",
    "    f1 = f1_score(data_unseen[\"Label\"],\n",
    "    predict_unseen[\"Label\"])\n",
    "    df_unseen = pd.DataFrame({\n",
    "    \"Accuracy\" : [accuracy],\n",
    "    \"AUC\" : [roc_auc],\n",
    "    \"Recall\" : [recall],\n",
    "    \"Precision\" : [precision],\n",
    "    \"F1 Score\" : [f1]\n",
    "    })\n",
    "    return df_unseen\n",
    "# Confusion Matrix\n",
    "def conf_mat(data_unseen, predict_unseen):\n",
    "    unique_label = data_unseen[\"Label\"].unique()\n",
    "    cmtx = pd.DataFrame(\n",
    "    confusion_matrix(data_unseen[\"Label\"],\n",
    "                    predict_unseen[\"Label\"], labels=unique_label),\n",
    "                    index=['{:}'.format(x) for x in unique_label],\n",
    "                    columns=['{:}'.format(x) for x in unique_label])\n",
    "    ax = sns.heatmap(cmtx, annot=True, fmt=\"d\", cmap=\"YlGnBu\")\n",
    "    ax.set_ylabel('Predicted')\n",
    "    ax.set_xlabel('Target');\n",
    "    ax.set_title(\"Predict Unseen Confusion Matrix\", size=14);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Data Loading and Initial Infos\n",
    "The data is loaded in a Dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>description_x</th>\n",
       "      <th>description_y</th>\n",
       "      <th>ticker_x</th>\n",
       "      <th>ticker_y</th>\n",
       "      <th>same_security</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>first trust dow jones internet</td>\n",
       "      <td>first trust dj internet idx</td>\n",
       "      <td>FDN</td>\n",
       "      <td>FDN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>schwab intl large company index etf</td>\n",
       "      <td>schwab strategic tr fundamental intl large co ...</td>\n",
       "      <td>FNDF</td>\n",
       "      <td>FNDF</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>vanguard small cap index adm</td>\n",
       "      <td>vanguard small-cap index fund inst</td>\n",
       "      <td>VSMAX</td>\n",
       "      <td>VSCIX</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0                        description_x  \\\n",
       "0           0       first trust dow jones internet   \n",
       "1           1  schwab intl large company index etf   \n",
       "2           2         vanguard small cap index adm   \n",
       "\n",
       "                                       description_y ticker_x ticker_y  \\\n",
       "0                        first trust dj internet idx      FDN      FDN   \n",
       "1  schwab strategic tr fundamental intl large co ...     FNDF     FNDF   \n",
       "2                 vanguard small-cap index fund inst    VSMAX    VSCIX   \n",
       "\n",
       "   same_security  \n",
       "0           True  \n",
       "1           True  \n",
       "2          False  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv(PATH+\"data\"+os.sep+\"train.csv\")\n",
    "data.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A check for duplicated samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.drop_duplicates(inplace=True, keep='first')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There're no duplicated samples on the dataset.<br>\n",
    "Time to drop features that will not help on the prediction and get some information about the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 2142 entries, 0 to 2141\n",
      "Data columns (total 3 columns):\n",
      " #   Column         Non-Null Count  Dtype \n",
      "---  ------         --------------  ----- \n",
      " 0   description_x  2142 non-null   object\n",
      " 1   description_y  2142 non-null   object\n",
      " 2   same_security  2142 non-null   bool  \n",
      "dtypes: bool(1), object(2)\n",
      "memory usage: 52.3+ KB\n"
     ]
    }
   ],
   "source": [
    "data.drop(columns=['Unnamed: 0', 'ticker_x', 'ticker_y'],\n",
    "          inplace=True)\n",
    "data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And check for missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "description_x    0\n",
       "description_y    0\n",
       "same_security    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset doesn't contain any missing value, what is very good!\n",
    "A good approach to compare the sentences is check its characters length, let's create these features for each sentence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['len_x'] = data['description_x'].apply(len)\n",
    "data['len_y'] = data['description_y'].apply(len)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. EDA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check the distribution of the target feature, `same_security`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "linkText": "Export to plot.ly",
        "plotlyServerURL": "https://plot.ly",
        "showLink": false
       },
       "data": [
        {
         "labels": [
          true,
          false
         ],
         "marker": {
          "line": {
           "color": "#000000",
           "width": 2
          }
         },
         "textfont": {
          "size": 14
         },
         "type": "pie",
         "values": [
          1613,
          529
         ]
        }
       ],
       "layout": {
        "autosize": false,
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Security Values",
         "x": 0.5,
         "xanchor": "center",
         "y": 0.9,
         "yanchor": "top"
        },
        "yaxis": {
         "automargin": true
        }
       }
      },
      "text/html": [
       "<div>\n",
       "        \n",
       "        \n",
       "            <div id=\"dfd96959-865b-4891-99a2-52f4faa21faa\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                require([\"plotly\"], function(Plotly) {\n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"dfd96959-865b-4891-99a2-52f4faa21faa\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        'dfd96959-865b-4891-99a2-52f4faa21faa',\n",
       "                        [{\"labels\": [true, false], \"marker\": {\"line\": {\"color\": \"#000000\", \"width\": 2}}, \"textfont\": {\"size\": 14}, \"type\": \"pie\", \"values\": [1613, 529]}],\n",
       "                        {\"autosize\": false, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Security Values\", \"x\": 0.5, \"xanchor\": \"center\", \"y\": 0.9, \"yanchor\": \"top\"}, \"yaxis\": {\"automargin\": true}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('dfd96959-865b-4891-99a2-52f4faa21faa');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                });\n",
       "            </script>\n",
       "        </div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "series_aux = data['same_security'].value_counts()\n",
    "pie_plot(series_aux, 'Security Values')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's a case of imbalanced classes and it will be handled with a model evaluation using F1-Score. I will not use any resampling technique, as random oversampling, SMOTE, etc. <br>\n",
    "A good insight for this problem is see the distribution of character length of each sentence for the security values, starting with the True values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "linkText": "Export to plot.ly",
        "plotlyServerURL": "https://plot.ly",
        "showLink": false
       },
       "data": [
        {
         "marker": {
          "line": {
           "color": "#000000",
           "width": 2
          }
         },
         "showlegend": false,
         "type": "histogram",
         "x": [
          30,
          35,
          53,
          16,
          28,
          24,
          43,
          28,
          17,
          31,
          14,
          31,
          19,
          64,
          18,
          16,
          12,
          28,
          15,
          22,
          10,
          39,
          19,
          18,
          25,
          20,
          26,
          36,
          20,
          20,
          33,
          29,
          27,
          13,
          35,
          35,
          32,
          25,
          30,
          39,
          29,
          33,
          10,
          39,
          25,
          66,
          14,
          44,
          40,
          15,
          33,
          19,
          18,
          24,
          24,
          31,
          33,
          38,
          25,
          33,
          18,
          40,
          37,
          22,
          16,
          30,
          18,
          17,
          20,
          40,
          24,
          23,
          32,
          13,
          31,
          32,
          32,
          27,
          21,
          32,
          12,
          17,
          28,
          14,
          24,
          27,
          36,
          21,
          45,
          36,
          29,
          29,
          34,
          14,
          24,
          19,
          8,
          32,
          15,
          39,
          22,
          27,
          29,
          44,
          35,
          42,
          18,
          8,
          15,
          21,
          19,
          17,
          19,
          24,
          27,
          17,
          33,
          25,
          23,
          20,
          33,
          29,
          10,
          23,
          31,
          24,
          35,
          26,
          16,
          38,
          18,
          34,
          21,
          22,
          36,
          28,
          26,
          31,
          19,
          28,
          16,
          20,
          41,
          34,
          23,
          54,
          17,
          32,
          21,
          28,
          18,
          28,
          24,
          32,
          53,
          32,
          60,
          46,
          27,
          58,
          27,
          30,
          22,
          35,
          30,
          29,
          26,
          34,
          55,
          35,
          12,
          48,
          27,
          18,
          28,
          18,
          27,
          35,
          22,
          22,
          20,
          51,
          34,
          39,
          13,
          20,
          32,
          25,
          24,
          30,
          18,
          30,
          18,
          12,
          31,
          21,
          20,
          23,
          24,
          40,
          18,
          26,
          12,
          27,
          33,
          21,
          27,
          16,
          48,
          26,
          11,
          37,
          28,
          35,
          35,
          22,
          36,
          35,
          14,
          32,
          15,
          12,
          14,
          15,
          17,
          35,
          37,
          16,
          32,
          40,
          16,
          23,
          33,
          20,
          31,
          22,
          31,
          44,
          12,
          34,
          19,
          28,
          47,
          24,
          19,
          53,
          20,
          32,
          19,
          25,
          18,
          22,
          17,
          29,
          21,
          45,
          18,
          11,
          20,
          41,
          28,
          14,
          20,
          30,
          18,
          23,
          13,
          21,
          25,
          37,
          19,
          25,
          26,
          10,
          27,
          31,
          19,
          17,
          32,
          34,
          19,
          19,
          24,
          21,
          35,
          26,
          58,
          17,
          28,
          40,
          21,
          50,
          27,
          30,
          43,
          34,
          12,
          18,
          27,
          32,
          26,
          40,
          34,
          20,
          22,
          35,
          34,
          26,
          14,
          20,
          28,
          22,
          40,
          54,
          32,
          15,
          12,
          19,
          34,
          29,
          24,
          13,
          60,
          19,
          16,
          25,
          29,
          29,
          27,
          20,
          34,
          23,
          24,
          34,
          19,
          11,
          24,
          10,
          36,
          33,
          35,
          44,
          29,
          18,
          19,
          37,
          26,
          9,
          27,
          31,
          36,
          43,
          21,
          41,
          29,
          20,
          29,
          12,
          31,
          14,
          51,
          59,
          29,
          34,
          32,
          10,
          23,
          36,
          23,
          30,
          14,
          38,
          32,
          21,
          28,
          25,
          19,
          22,
          19,
          18,
          33,
          17,
          15,
          11,
          35,
          21,
          17,
          39,
          30,
          33,
          17,
          30,
          33,
          29,
          20,
          14,
          21,
          10,
          20,
          22,
          52,
          10,
          31,
          32,
          24,
          45,
          52,
          29,
          28,
          24,
          22,
          16,
          30,
          22,
          19,
          34,
          30,
          24,
          26,
          19,
          15,
          28,
          16,
          43,
          14,
          26,
          13,
          26,
          15,
          23,
          26,
          30,
          30,
          45,
          19,
          53,
          29,
          32,
          32,
          22,
          38,
          21,
          20,
          27,
          27,
          26,
          17,
          18,
          19,
          20,
          32,
          24,
          26,
          58,
          40,
          20,
          47,
          36,
          39,
          25,
          59,
          21,
          10,
          20,
          37,
          35,
          34,
          21,
          15,
          36,
          34,
          60,
          27,
          36,
          15,
          30,
          34,
          56,
          31,
          27,
          17,
          50,
          21,
          30,
          31,
          12,
          34,
          18,
          28,
          13,
          18,
          24,
          16,
          21,
          17,
          38,
          16,
          24,
          29,
          18,
          31,
          33,
          25,
          31,
          40,
          22,
          28,
          20,
          35,
          28,
          48,
          25,
          15,
          7,
          47,
          12,
          24,
          24,
          27,
          28,
          34,
          23,
          16,
          22,
          31,
          14,
          25,
          34,
          50,
          13,
          29,
          21,
          24,
          23,
          31,
          36,
          51,
          29,
          23,
          29,
          40,
          31,
          33,
          44,
          29,
          31,
          49,
          19,
          34,
          17,
          41,
          18,
          26,
          26,
          30,
          28,
          15,
          20,
          48,
          32,
          58,
          34,
          24,
          11,
          15,
          18,
          22,
          18,
          8,
          17,
          29,
          53,
          34,
          31,
          44,
          19,
          30,
          27,
          18,
          31,
          41,
          28,
          33,
          32,
          33,
          30,
          29,
          20,
          27,
          10,
          49,
          14,
          26,
          24,
          30,
          39,
          18,
          79,
          48,
          44,
          23,
          29,
          23,
          14,
          24,
          27,
          11,
          29,
          22,
          59,
          18,
          33,
          18,
          36,
          59,
          34,
          15,
          29,
          41,
          36,
          30,
          37,
          31,
          42,
          27,
          19,
          50,
          18,
          15,
          25,
          27,
          29,
          26,
          37,
          25,
          16,
          60,
          23,
          38,
          22,
          50,
          34,
          36,
          18,
          30,
          19,
          22,
          22,
          15,
          20,
          38,
          28,
          45,
          44,
          39,
          28,
          36,
          18,
          30,
          30,
          31,
          29,
          29,
          53,
          19,
          31,
          25,
          11,
          20,
          19,
          38,
          24,
          32,
          30,
          18,
          30,
          28,
          26,
          30,
          20,
          23,
          29,
          39,
          28,
          23,
          29,
          37,
          11,
          27,
          18,
          11,
          39,
          27,
          31,
          20,
          17,
          18,
          25,
          20,
          20,
          16,
          30,
          19,
          22,
          55,
          58,
          30,
          23,
          28,
          30,
          28,
          17,
          23,
          50,
          15,
          17,
          21,
          40,
          19,
          28,
          28,
          24,
          25,
          17,
          19,
          60,
          13,
          39,
          23,
          44,
          28,
          15,
          68,
          23,
          8,
          37,
          31,
          60,
          56,
          22,
          60,
          14,
          34,
          38,
          32,
          20,
          19,
          25,
          23,
          48,
          36,
          21,
          35,
          18,
          13,
          28,
          20,
          36,
          32,
          14,
          32,
          28,
          30,
          17,
          24,
          30,
          30,
          53,
          51,
          36,
          9,
          22,
          27,
          27,
          23,
          26,
          24,
          36,
          15,
          20,
          30,
          39,
          13,
          44,
          30,
          35,
          46,
          15,
          79,
          22,
          25,
          34,
          28,
          31,
          31,
          20,
          12,
          21,
          18,
          24,
          36,
          28,
          18,
          26,
          19,
          59,
          29,
          17,
          11,
          20,
          32,
          18,
          27,
          27,
          21,
          26,
          14,
          28,
          39,
          31,
          23,
          47,
          16,
          49,
          22,
          38,
          52,
          30,
          23,
          36,
          9,
          78,
          23,
          32,
          19,
          23,
          26,
          27,
          19,
          28,
          30,
          27,
          50,
          28,
          19,
          40,
          28,
          27,
          24,
          18,
          22,
          30,
          23,
          16,
          14,
          15,
          22,
          43,
          25,
          12,
          20,
          21,
          40,
          28,
          34,
          37,
          32,
          26,
          32,
          25,
          31,
          17,
          40,
          48,
          41,
          30,
          27,
          33,
          17,
          48,
          35,
          48,
          21,
          18,
          64,
          34,
          17,
          13,
          19,
          17,
          38,
          37,
          31,
          36,
          18,
          16,
          17,
          27,
          11,
          19,
          20,
          22,
          47,
          39,
          25,
          46,
          30,
          30,
          19,
          39,
          24,
          34,
          15,
          21,
          33,
          16,
          23,
          17,
          37,
          28,
          44,
          41,
          31,
          28,
          18,
          36,
          28,
          49,
          11,
          27,
          40,
          33,
          32,
          18,
          30,
          17,
          26,
          37,
          26,
          35,
          18,
          20,
          20,
          47,
          15,
          23,
          39,
          52,
          24,
          18,
          18,
          22,
          26,
          24,
          14,
          29,
          29,
          32,
          23,
          26,
          27,
          30,
          27,
          29,
          17,
          57,
          33,
          28,
          23,
          25,
          38,
          41,
          23,
          20,
          27,
          20,
          30,
          21,
          35,
          38,
          14,
          20,
          35,
          28,
          37,
          21,
          36,
          17,
          18,
          30,
          44,
          32,
          23,
          42,
          44,
          60,
          31,
          44,
          28,
          17,
          27,
          18,
          12,
          23,
          22,
          20,
          21,
          15,
          20,
          18,
          40,
          27,
          18,
          23,
          21,
          35,
          40,
          18,
          25,
          64,
          36,
          24,
          34,
          9,
          31,
          15,
          68,
          26,
          28,
          16,
          38,
          32,
          50,
          27,
          14,
          27,
          39,
          29,
          52,
          20,
          19,
          32,
          39,
          39,
          23,
          38,
          26,
          24,
          30,
          24,
          56,
          29,
          23,
          34,
          20,
          13,
          36,
          60,
          15,
          42,
          23,
          53,
          25,
          20,
          18,
          22,
          18,
          24,
          27,
          30,
          60,
          25,
          21,
          27,
          35,
          18,
          15,
          22,
          41,
          23,
          19,
          35,
          13,
          35,
          14,
          32,
          17,
          37,
          35,
          29,
          40,
          36,
          36,
          30,
          7,
          21,
          58,
          24,
          28,
          27,
          40,
          46,
          16,
          30,
          18,
          21,
          18,
          21,
          25,
          28,
          30,
          28,
          27,
          17,
          59,
          20,
          12,
          31,
          18,
          49,
          17,
          18,
          20,
          25,
          28,
          32,
          29,
          26,
          34,
          24,
          26,
          22,
          30,
          33,
          40,
          56,
          33,
          19,
          36,
          15,
          42,
          35,
          29,
          34,
          18,
          11,
          14,
          18,
          33,
          24,
          43,
          22,
          25,
          33,
          16,
          25,
          52,
          19,
          16,
          18,
          34,
          28,
          12,
          37,
          30,
          28,
          26,
          27,
          25,
          39,
          18,
          35,
          15,
          17,
          22,
          26,
          60,
          53,
          17,
          51,
          20,
          16,
          10,
          20,
          27,
          11,
          30,
          19,
          28,
          36,
          31,
          51,
          28,
          29,
          11,
          30,
          25,
          35,
          39,
          18,
          29,
          14,
          25,
          36,
          30,
          19,
          28,
          21,
          14,
          22,
          45,
          32,
          30,
          23,
          30,
          32,
          15,
          34,
          25,
          52,
          34,
          29,
          27,
          36,
          10,
          13,
          33,
          21,
          37,
          12,
          20,
          18,
          30,
          35,
          34,
          14,
          19,
          25,
          30,
          44,
          36,
          32,
          28,
          13,
          30,
          18,
          21,
          32,
          26,
          17,
          68,
          23,
          27,
          35,
          57,
          21,
          35,
          50,
          36,
          27,
          60,
          11,
          23,
          16,
          22,
          67,
          20,
          30,
          21,
          29,
          17,
          16,
          28,
          24,
          35,
          20,
          28,
          32,
          28,
          20,
          60,
          22,
          31,
          41,
          33,
          20,
          25,
          9,
          36,
          34,
          51,
          26,
          46,
          34,
          26,
          30,
          17,
          18,
          41,
          20,
          17,
          26,
          15,
          53,
          28,
          33,
          23,
          17,
          23,
          18,
          25,
          23,
          22,
          39,
          18,
          27,
          26,
          19,
          33,
          32,
          19,
          23,
          22,
          36,
          38,
          20,
          28,
          30,
          26,
          31,
          20,
          26,
          9,
          41,
          26,
          26,
          31,
          13,
          34,
          38,
          15,
          20,
          24,
          25,
          32,
          55,
          36,
          27,
          20,
          30,
          23,
          18,
          71,
          23,
          29,
          29,
          19,
          38,
          24,
          39,
          32,
          20,
          36,
          11,
          50,
          25,
          31,
          16,
          14,
          24,
          33,
          14,
          38,
          19,
          24,
          35,
          23,
          47,
          39,
          20,
          30,
          36,
          46,
          18,
          22,
          14,
          16,
          21,
          33,
          18,
          25,
          28,
          12,
          18,
          17,
          18,
          21,
          27,
          22,
          27,
          14,
          36,
          26,
          18,
          28,
          18,
          19,
          27,
          23,
          35,
          16,
          26,
          21,
          33,
          22,
          40,
          10,
          48,
          20,
          9,
          17,
          37,
          23,
          19,
          43,
          32,
          35,
          20,
          19,
          19,
          48,
          15,
          36,
          29,
          23,
          27,
          32,
          29,
          33,
          22,
          28,
          24,
          33,
          25,
          41,
          32,
          33,
          31,
          10,
          33,
          41,
          20,
          21,
          18,
          32,
          25,
          20,
          31,
          20,
          21,
          58,
          36,
          42,
          28,
          26,
          24,
          10,
          24,
          36,
          36,
          20,
          33,
          18,
          33,
          26,
          27,
          45,
          26,
          17,
          45,
          21,
          28,
          32,
          20,
          27,
          41,
          19,
          36,
          13,
          28,
          33,
          31,
          14,
          36,
          18,
          19,
          40,
          28,
          17,
          22,
          21,
          11,
          33,
          37,
          20,
          35,
          37,
          19,
          24,
          53,
          27,
          18,
          12,
          24,
          11,
          29,
          28,
          24,
          31,
          29,
          29,
          79,
          24,
          23,
          43,
          25,
          34,
          21,
          27,
          43,
          14,
          38,
          27,
          25,
          33,
          18,
          16,
          17,
          35,
          29,
          22,
          38,
          12,
          33,
          17,
          31,
          25,
          7,
          59,
          21,
          9,
          21,
          28,
          27,
          13,
          60,
          23,
          36,
          60,
          20,
          28,
          13,
          26,
          31,
          24,
          23,
          53,
          33,
          24,
          29,
          38,
          20,
          10,
          54,
          23,
          37,
          12,
          27,
          17,
          47,
          32,
          22,
          14,
          14,
          34,
          21,
          33,
          30,
          14,
          26,
          30,
          33,
          27,
          18,
          32,
          9,
          23,
          40,
          41,
          30,
          35,
          44,
          29,
          21,
          30,
          27,
          20,
          9,
          39,
          23,
          24,
          18,
          19
         ],
         "xaxis": "x",
         "yaxis": "y"
        },
        {
         "marker": {
          "line": {
           "color": "#000000",
           "width": 2
          }
         },
         "showlegend": false,
         "type": "histogram",
         "x": [
          27,
          55,
          60,
          9,
          13,
          23,
          34,
          23,
          19,
          35,
          22,
          40,
          18,
          36,
          24,
          11,
          23,
          23,
          20,
          19,
          17,
          20,
          20,
          27,
          10,
          59,
          34,
          24,
          45,
          23,
          36,
          42,
          45,
          17,
          30,
          36,
          36,
          38,
          29,
          18,
          26,
          60,
          15,
          20,
          22,
          59,
          15,
          33,
          53,
          11,
          41,
          24,
          28,
          16,
          57,
          25,
          21,
          29,
          30,
          49,
          27,
          40,
          35,
          38,
          25,
          32,
          20,
          13,
          48,
          15,
          39,
          14,
          34,
          16,
          26,
          18,
          27,
          32,
          25,
          38,
          16,
          16,
          45,
          13,
          9,
          17,
          32,
          27,
          34,
          38,
          13,
          31,
          15,
          6,
          19,
          52,
          4,
          30,
          19,
          54,
          39,
          34,
          43,
          39,
          35,
          27,
          16,
          9,
          29,
          15,
          20,
          27,
          27,
          59,
          23,
          16,
          22,
          24,
          24,
          25,
          22,
          35,
          9,
          21,
          28,
          35,
          28,
          41,
          13,
          27,
          21,
          58,
          17,
          24,
          35,
          26,
          28,
          34,
          44,
          25,
          11,
          20,
          33,
          28,
          27,
          30,
          26,
          59,
          11,
          30,
          22,
          34,
          18,
          31,
          37,
          27,
          24,
          39,
          42,
          25,
          23,
          44,
          18,
          18,
          60,
          23,
          49,
          40,
          30,
          38,
          17,
          31,
          22,
          22,
          67,
          31,
          31,
          20,
          33,
          32,
          42,
          39,
          59,
          46,
          11,
          23,
          20,
          32,
          29,
          34,
          33,
          15,
          29,
          17,
          26,
          20,
          36,
          33,
          54,
          38,
          13,
          22,
          10,
          25,
          27,
          28,
          35,
          17,
          30,
          37,
          15,
          59,
          32,
          35,
          31,
          22,
          20,
          34,
          27,
          30,
          30,
          16,
          23,
          11,
          22,
          22,
          39,
          27,
          39,
          24,
          20,
          23,
          30,
          35,
          26,
          11,
          35,
          25,
          13,
          23,
          15,
          26,
          26,
          28,
          11,
          31,
          29,
          36,
          26,
          27,
          32,
          18,
          17,
          25,
          52,
          48,
          37,
          16,
          32,
          28,
          19,
          20,
          41,
          24,
          22,
          31,
          16,
          10,
          19,
          36,
          48,
          21,
          30,
          14,
          53,
          12,
          15,
          25,
          25,
          60,
          38,
          15,
          20,
          30,
          60,
          34,
          35,
          26,
          25,
          38,
          22,
          31,
          19,
          36,
          47,
          35,
          16,
          15,
          27,
          36,
          38,
          40,
          40,
          39,
          32,
          38,
          40,
          43,
          33,
          28,
          18,
          31,
          24,
          22,
          22,
          17,
          17,
          53,
          58,
          26,
          28,
          10,
          35,
          34,
          18,
          38,
          28,
          30,
          31,
          13,
          45,
          33,
          32,
          18,
          20,
          24,
          24,
          18,
          40,
          18,
          42,
          39,
          29,
          31,
          31,
          24,
          16,
          13,
          30,
          56,
          45,
          79,
          17,
          32,
          22,
          27,
          27,
          19,
          19,
          10,
          49,
          37,
          78,
          25,
          26,
          5,
          19,
          52,
          19,
          33,
          15,
          39,
          28,
          36,
          23,
          29,
          26,
          19,
          34,
          32,
          18,
          9,
          11,
          30,
          32,
          22,
          33,
          25,
          20,
          40,
          21,
          30,
          25,
          36,
          24,
          21,
          9,
          21,
          33,
          19,
          27,
          13,
          22,
          38,
          22,
          16,
          38,
          39,
          45,
          37,
          32,
          22,
          34,
          32,
          15,
          27,
          55,
          29,
          25,
          15,
          9,
          36,
          16,
          25,
          10,
          51,
          9,
          31,
          19,
          12,
          30,
          24,
          20,
          46,
          15,
          19,
          23,
          34,
          20,
          18,
          26,
          29,
          29,
          26,
          15,
          16,
          13,
          14,
          27,
          24,
          33,
          32,
          18,
          49,
          41,
          39,
          28,
          28,
          26,
          15,
          20,
          21,
          23,
          19,
          47,
          36,
          31,
          46,
          10,
          32,
          33,
          22,
          23,
          32,
          30,
          27,
          35,
          34,
          35,
          24,
          17,
          30,
          29,
          36,
          23,
          15,
          27,
          19,
          24,
          27,
          23,
          29,
          39,
          31,
          18,
          30,
          34,
          17,
          39,
          17,
          22,
          18,
          20,
          37,
          33,
          20,
          27,
          17,
          19,
          27,
          34,
          24,
          38,
          11,
          59,
          22,
          60,
          35,
          18,
          25,
          28,
          19,
          24,
          20,
          35,
          9,
          58,
          36,
          41,
          20,
          33,
          32,
          46,
          21,
          36,
          28,
          37,
          29,
          8,
          30,
          14,
          41,
          39,
          37,
          44,
          21,
          49,
          23,
          40,
          21,
          20,
          38,
          34,
          29,
          22,
          20,
          14,
          59,
          32,
          47,
          30,
          27,
          32,
          7,
          13,
          19,
          22,
          24,
          10,
          20,
          15,
          29,
          23,
          51,
          26,
          24,
          25,
          19,
          14,
          27,
          16,
          32,
          78,
          60,
          20,
          39,
          31,
          32,
          40,
          16,
          29,
          41,
          33,
          20,
          44,
          25,
          13,
          30,
          23,
          23,
          27,
          22,
          37,
          8,
          13,
          22,
          16,
          15,
          18,
          40,
          25,
          30,
          14,
          30,
          34,
          39,
          23,
          33,
          58,
          38,
          33,
          20,
          34,
          39,
          26,
          23,
          21,
          24,
          19,
          28,
          26,
          38,
          25,
          41,
          33,
          23,
          40,
          31,
          32,
          18,
          36,
          17,
          19,
          21,
          31,
          30,
          35,
          33,
          15,
          22,
          45,
          31,
          64,
          33,
          38,
          24,
          20,
          36,
          23,
          30,
          15,
          26,
          21,
          35,
          15,
          22,
          34,
          9,
          25,
          38,
          36,
          22,
          18,
          23,
          14,
          19,
          24,
          16,
          28,
          42,
          24,
          16,
          25,
          27,
          13,
          28,
          23,
          15,
          27,
          25,
          13,
          27,
          29,
          32,
          37,
          19,
          23,
          34,
          28,
          30,
          29,
          29,
          26,
          15,
          35,
          28,
          26,
          24,
          20,
          32,
          60,
          27,
          29,
          30,
          19,
          16,
          20,
          35,
          8,
          20,
          20,
          19,
          15,
          15,
          21,
          44,
          18,
          27,
          26,
          30,
          16,
          39,
          59,
          31,
          12,
          62,
          22,
          72,
          63,
          18,
          45,
          22,
          44,
          28,
          21,
          32,
          27,
          23,
          57,
          22,
          18,
          25,
          29,
          33,
          17,
          19,
          31,
          24,
          49,
          15,
          36,
          35,
          29,
          42,
          22,
          28,
          21,
          20,
          27,
          40,
          14,
          33,
          23,
          32,
          28,
          26,
          18,
          21,
          44,
          11,
          26,
          17,
          17,
          22,
          24,
          24,
          55,
          19,
          78,
          42,
          16,
          27,
          52,
          26,
          33,
          19,
          8,
          25,
          25,
          30,
          19,
          20,
          22,
          18,
          25,
          35,
          24,
          22,
          14,
          28,
          45,
          14,
          31,
          28,
          25,
          28,
          15,
          44,
          59,
          31,
          20,
          38,
          36,
          59,
          11,
          39,
          43,
          18,
          22,
          20,
          25,
          35,
          38,
          30,
          19,
          30,
          20,
          32,
          48,
          28,
          44,
          32,
          32,
          24,
          24,
          33,
          31,
          32,
          21,
          11,
          34,
          26,
          41,
          24,
          20,
          18,
          14,
          55,
          20,
          11,
          27,
          15,
          29,
          20,
          35,
          26,
          51,
          44,
          36,
          23,
          46,
          31,
          29,
          21,
          53,
          39,
          33,
          40,
          21,
          19,
          32,
          21,
          29,
          10,
          27,
          38,
          33,
          20,
          37,
          31,
          36,
          40,
          25,
          41,
          22,
          26,
          21,
          27,
          13,
          22,
          15,
          26,
          31,
          44,
          38,
          21,
          25,
          30,
          23,
          23,
          16,
          30,
          18,
          17,
          30,
          37,
          24,
          21,
          36,
          29,
          30,
          41,
          27,
          21,
          10,
          32,
          24,
          29,
          15,
          61,
          40,
          33,
          21,
          15,
          16,
          16,
          30,
          44,
          16,
          31,
          14,
          17,
          22,
          53,
          23,
          27,
          28,
          28,
          35,
          16,
          20,
          15,
          20,
          32,
          15,
          23,
          36,
          26,
          26,
          30,
          21,
          30,
          39,
          58,
          9,
          23,
          50,
          34,
          27,
          17,
          35,
          30,
          36,
          19,
          28,
          34,
          28,
          28,
          46,
          48,
          9,
          22,
          48,
          38,
          36,
          15,
          35,
          13,
          33,
          38,
          53,
          30,
          19,
          26,
          23,
          29,
          29,
          40,
          35,
          25,
          26,
          23,
          7,
          22,
          26,
          30,
          21,
          15,
          37,
          17,
          17,
          23,
          27,
          19,
          31,
          48,
          27,
          16,
          21,
          36,
          34,
          26,
          13,
          16,
          17,
          30,
          40,
          19,
          35,
          31,
          20,
          27,
          23,
          23,
          27,
          30,
          36,
          42,
          20,
          57,
          15,
          45,
          21,
          33,
          19,
          40,
          12,
          39,
          33,
          27,
          33,
          27,
          16,
          33,
          19,
          18,
          29,
          39,
          26,
          26,
          37,
          45,
          21,
          19,
          22,
          11,
          28,
          17,
          30,
          18,
          21,
          17,
          21,
          35,
          20,
          24,
          20,
          28,
          35,
          59,
          15,
          38,
          17,
          32,
          12,
          17,
          39,
          35,
          22,
          34,
          53,
          29,
          37,
          32,
          9,
          19,
          62,
          37,
          54,
          25,
          37,
          33,
          25,
          34,
          14,
          17,
          26,
          29,
          22,
          68,
          16,
          53,
          59,
          16,
          32,
          18,
          11,
          30,
          18,
          36,
          21,
          22,
          16,
          34,
          22,
          27,
          36,
          40,
          22,
          20,
          27,
          21,
          35,
          20,
          45,
          20,
          30,
          21,
          35,
          12,
          30,
          37,
          23,
          28,
          24,
          21,
          21,
          49,
          24,
          26,
          24,
          39,
          14,
          20,
          30,
          27,
          63,
          23,
          15,
          26,
          43,
          24,
          21,
          47,
          24,
          40,
          38,
          30,
          32,
          20,
          24,
          31,
          22,
          20,
          14,
          34,
          47,
          35,
          12,
          31,
          25,
          26,
          9,
          47,
          20,
          20,
          20,
          27,
          29,
          15,
          23,
          27,
          22,
          34,
          15,
          16,
          14,
          29,
          28,
          27,
          23,
          9,
          36,
          32,
          32,
          17,
          37,
          28,
          13,
          31,
          69,
          36,
          22,
          38,
          26,
          33,
          10,
          30,
          22,
          36,
          30,
          25,
          23,
          28,
          5,
          21,
          40,
          25,
          24,
          33,
          38,
          22,
          19,
          52,
          45,
          23,
          15,
          38,
          22,
          22,
          39,
          22,
          20,
          33,
          29,
          15,
          17,
          24,
          31,
          15,
          39,
          14,
          20,
          19,
          20,
          23,
          60,
          52,
          27,
          25,
          26,
          19,
          16,
          12,
          18,
          60,
          52,
          26,
          19,
          30,
          21,
          29,
          30,
          29,
          40,
          14,
          20,
          35,
          29,
          19,
          21,
          46,
          53,
          35,
          23,
          28,
          18,
          18,
          30,
          17,
          26,
          22,
          25,
          29,
          30,
          33,
          11,
          11,
          28,
          15,
          17,
          15,
          23,
          27,
          29,
          46,
          23,
          18,
          19,
          33,
          60,
          19,
          22,
          26,
          22,
          22,
          20,
          44,
          25,
          33,
          23,
          36,
          36,
          38,
          45,
          24,
          24,
          23,
          31,
          47,
          15,
          15,
          22,
          31,
          19,
          20,
          26,
          10,
          34,
          17,
          11,
          48,
          28,
          20,
          24,
          20,
          21,
          18,
          20,
          23,
          14,
          23,
          32,
          27,
          20,
          40,
          23,
          39,
          37,
          18,
          26,
          29,
          43,
          7,
          35,
          18,
          19,
          20,
          18,
          26,
          27,
          14,
          19,
          19,
          18,
          13,
          12,
          29,
          45,
          25,
          36,
          35,
          29,
          22,
          15,
          15,
          12,
          12,
          30,
          9,
          43,
          32,
          9,
          20,
          21,
          21,
          26,
          29,
          18,
          22,
          10,
          50,
          35,
          39,
          24,
          30,
          33,
          54,
          57,
          25,
          15,
          22,
          24,
          23,
          19,
          63,
          9,
          43,
          21,
          13,
          21,
          13,
          44,
          15,
          68,
          18,
          22,
          23,
          15,
          28,
          33,
          26,
          29,
          20,
          19,
          51,
          21,
          19,
          28,
          26,
          39,
          25,
          33,
          33,
          56,
          34,
          24,
          24,
          11,
          35,
          35,
          15,
          21,
          23,
          19,
          37,
          26,
          49,
          32,
          19,
          29,
          26,
          26,
          32,
          45,
          28,
          21,
          24,
          20,
          35,
          17,
          29,
          26,
          25,
          31,
          18,
          53,
          17,
          15,
          31,
          25,
          28,
          36,
          39,
          19,
          18,
          23,
          26,
          23,
          24,
          22,
          38,
          9,
          26,
          31,
          27,
          58,
          39,
          25,
          34,
          17,
          15,
          39,
          27,
          21,
          34,
          34,
          39,
          20,
          44,
          20,
          14,
          13,
          34,
          16,
          34,
          36,
          59,
          20,
          32,
          39,
          59,
          28,
          21,
          45,
          22,
          34,
          8,
          14,
          43,
          22,
          70,
          16,
          31,
          26,
          34,
          23,
          21,
          24,
          20,
          37,
          50,
          8,
          40,
          33,
          33,
          20,
          11,
          23,
          22,
          13,
          30,
          34,
          36,
          18,
          40,
          20,
          53,
          24,
          24,
          18,
          14,
          24,
          38,
          20,
          48,
          22,
          39,
          32,
          22,
          50,
          11,
          18,
          35,
          19,
          45,
          24,
          37,
          20,
          28,
          18,
          19,
          23,
          15,
          27,
          19,
          28,
          22,
          19,
          37,
          20,
          31,
          27,
          26,
          38,
          11,
          26,
          36,
          41,
          39,
          36,
          38,
          21,
          25,
          38,
          28,
          22,
          7,
          21,
          28,
          29,
          21,
          27
         ],
         "xaxis": "x2",
         "yaxis": "y2"
        }
       ],
       "layout": {
        "annotations": [
         {
          "font": {
           "size": 16
          },
          "showarrow": false,
          "text": "Description X Length",
          "x": 0.225,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         },
         {
          "font": {
           "size": 16
          },
          "showarrow": false,
          "text": "Description Y Length",
          "x": 0.775,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         }
        ],
        "autosize": false,
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "rgb(36,36,36)"
            },
            "error_y": {
             "color": "rgb(36,36,36)"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "baxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.6
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "rgb(237,237,237)"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "rgb(217,217,217)"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 1,
            "tickcolor": "rgb(36,36,36)",
            "ticks": "outside"
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "rgb(103,0,31)"
            ],
            [
             0.1,
             "rgb(178,24,43)"
            ],
            [
             0.2,
             "rgb(214,96,77)"
            ],
            [
             0.3,
             "rgb(244,165,130)"
            ],
            [
             0.4,
             "rgb(253,219,199)"
            ],
            [
             0.5,
             "rgb(247,247,247)"
            ],
            [
             0.6,
             "rgb(209,229,240)"
            ],
            [
             0.7,
             "rgb(146,197,222)"
            ],
            [
             0.8,
             "rgb(67,147,195)"
            ],
            [
             0.9,
             "rgb(33,102,172)"
            ],
            [
             1,
             "rgb(5,48,97)"
            ]
           ],
           "sequential": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ]
          },
          "colorway": [
           "#1F77B4",
           "#FF7F0E",
           "#2CA02C",
           "#D62728",
           "#9467BD",
           "#8C564B",
           "#E377C2",
           "#7F7F7F",
           "#BCBD22",
           "#17BECF"
          ],
          "font": {
           "color": "rgb(36,36,36)"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           }
          },
          "shapedefaults": {
           "fillcolor": "black",
           "line": {
            "width": 0
           },
           "opacity": 0.3
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "baxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          }
         }
        },
        "title": {
         "text": "<b>Character length distribution for True Security",
         "x": 0.5,
         "xanchor": "center",
         "y": 0.9,
         "yanchor": "top"
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          0.45
         ]
        },
        "xaxis2": {
         "anchor": "y2",
         "domain": [
          0.55,
          1
         ]
        },
        "yaxis": {
         "anchor": "x",
         "automargin": true,
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "<i>Frequency</i>"
         }
        },
        "yaxis2": {
         "anchor": "x2",
         "automargin": true,
         "domain": [
          0,
          1
         ]
        }
       }
      },
      "text/html": [
       "<div>\n",
       "        \n",
       "        \n",
       "            <div id=\"16ec371f-47a1-4dfe-bf13-685266929687\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                require([\"plotly\"], function(Plotly) {\n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"16ec371f-47a1-4dfe-bf13-685266929687\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        '16ec371f-47a1-4dfe-bf13-685266929687',\n",
       "                        [{\"marker\": {\"line\": {\"color\": \"#000000\", \"width\": 2}}, \"showlegend\": false, \"type\": \"histogram\", \"x\": [30, 35, 53, 16, 28, 24, 43, 28, 17, 31, 14, 31, 19, 64, 18, 16, 12, 28, 15, 22, 10, 39, 19, 18, 25, 20, 26, 36, 20, 20, 33, 29, 27, 13, 35, 35, 32, 25, 30, 39, 29, 33, 10, 39, 25, 66, 14, 44, 40, 15, 33, 19, 18, 24, 24, 31, 33, 38, 25, 33, 18, 40, 37, 22, 16, 30, 18, 17, 20, 40, 24, 23, 32, 13, 31, 32, 32, 27, 21, 32, 12, 17, 28, 14, 24, 27, 36, 21, 45, 36, 29, 29, 34, 14, 24, 19, 8, 32, 15, 39, 22, 27, 29, 44, 35, 42, 18, 8, 15, 21, 19, 17, 19, 24, 27, 17, 33, 25, 23, 20, 33, 29, 10, 23, 31, 24, 35, 26, 16, 38, 18, 34, 21, 22, 36, 28, 26, 31, 19, 28, 16, 20, 41, 34, 23, 54, 17, 32, 21, 28, 18, 28, 24, 32, 53, 32, 60, 46, 27, 58, 27, 30, 22, 35, 30, 29, 26, 34, 55, 35, 12, 48, 27, 18, 28, 18, 27, 35, 22, 22, 20, 51, 34, 39, 13, 20, 32, 25, 24, 30, 18, 30, 18, 12, 31, 21, 20, 23, 24, 40, 18, 26, 12, 27, 33, 21, 27, 16, 48, 26, 11, 37, 28, 35, 35, 22, 36, 35, 14, 32, 15, 12, 14, 15, 17, 35, 37, 16, 32, 40, 16, 23, 33, 20, 31, 22, 31, 44, 12, 34, 19, 28, 47, 24, 19, 53, 20, 32, 19, 25, 18, 22, 17, 29, 21, 45, 18, 11, 20, 41, 28, 14, 20, 30, 18, 23, 13, 21, 25, 37, 19, 25, 26, 10, 27, 31, 19, 17, 32, 34, 19, 19, 24, 21, 35, 26, 58, 17, 28, 40, 21, 50, 27, 30, 43, 34, 12, 18, 27, 32, 26, 40, 34, 20, 22, 35, 34, 26, 14, 20, 28, 22, 40, 54, 32, 15, 12, 19, 34, 29, 24, 13, 60, 19, 16, 25, 29, 29, 27, 20, 34, 23, 24, 34, 19, 11, 24, 10, 36, 33, 35, 44, 29, 18, 19, 37, 26, 9, 27, 31, 36, 43, 21, 41, 29, 20, 29, 12, 31, 14, 51, 59, 29, 34, 32, 10, 23, 36, 23, 30, 14, 38, 32, 21, 28, 25, 19, 22, 19, 18, 33, 17, 15, 11, 35, 21, 17, 39, 30, 33, 17, 30, 33, 29, 20, 14, 21, 10, 20, 22, 52, 10, 31, 32, 24, 45, 52, 29, 28, 24, 22, 16, 30, 22, 19, 34, 30, 24, 26, 19, 15, 28, 16, 43, 14, 26, 13, 26, 15, 23, 26, 30, 30, 45, 19, 53, 29, 32, 32, 22, 38, 21, 20, 27, 27, 26, 17, 18, 19, 20, 32, 24, 26, 58, 40, 20, 47, 36, 39, 25, 59, 21, 10, 20, 37, 35, 34, 21, 15, 36, 34, 60, 27, 36, 15, 30, 34, 56, 31, 27, 17, 50, 21, 30, 31, 12, 34, 18, 28, 13, 18, 24, 16, 21, 17, 38, 16, 24, 29, 18, 31, 33, 25, 31, 40, 22, 28, 20, 35, 28, 48, 25, 15, 7, 47, 12, 24, 24, 27, 28, 34, 23, 16, 22, 31, 14, 25, 34, 50, 13, 29, 21, 24, 23, 31, 36, 51, 29, 23, 29, 40, 31, 33, 44, 29, 31, 49, 19, 34, 17, 41, 18, 26, 26, 30, 28, 15, 20, 48, 32, 58, 34, 24, 11, 15, 18, 22, 18, 8, 17, 29, 53, 34, 31, 44, 19, 30, 27, 18, 31, 41, 28, 33, 32, 33, 30, 29, 20, 27, 10, 49, 14, 26, 24, 30, 39, 18, 79, 48, 44, 23, 29, 23, 14, 24, 27, 11, 29, 22, 59, 18, 33, 18, 36, 59, 34, 15, 29, 41, 36, 30, 37, 31, 42, 27, 19, 50, 18, 15, 25, 27, 29, 26, 37, 25, 16, 60, 23, 38, 22, 50, 34, 36, 18, 30, 19, 22, 22, 15, 20, 38, 28, 45, 44, 39, 28, 36, 18, 30, 30, 31, 29, 29, 53, 19, 31, 25, 11, 20, 19, 38, 24, 32, 30, 18, 30, 28, 26, 30, 20, 23, 29, 39, 28, 23, 29, 37, 11, 27, 18, 11, 39, 27, 31, 20, 17, 18, 25, 20, 20, 16, 30, 19, 22, 55, 58, 30, 23, 28, 30, 28, 17, 23, 50, 15, 17, 21, 40, 19, 28, 28, 24, 25, 17, 19, 60, 13, 39, 23, 44, 28, 15, 68, 23, 8, 37, 31, 60, 56, 22, 60, 14, 34, 38, 32, 20, 19, 25, 23, 48, 36, 21, 35, 18, 13, 28, 20, 36, 32, 14, 32, 28, 30, 17, 24, 30, 30, 53, 51, 36, 9, 22, 27, 27, 23, 26, 24, 36, 15, 20, 30, 39, 13, 44, 30, 35, 46, 15, 79, 22, 25, 34, 28, 31, 31, 20, 12, 21, 18, 24, 36, 28, 18, 26, 19, 59, 29, 17, 11, 20, 32, 18, 27, 27, 21, 26, 14, 28, 39, 31, 23, 47, 16, 49, 22, 38, 52, 30, 23, 36, 9, 78, 23, 32, 19, 23, 26, 27, 19, 28, 30, 27, 50, 28, 19, 40, 28, 27, 24, 18, 22, 30, 23, 16, 14, 15, 22, 43, 25, 12, 20, 21, 40, 28, 34, 37, 32, 26, 32, 25, 31, 17, 40, 48, 41, 30, 27, 33, 17, 48, 35, 48, 21, 18, 64, 34, 17, 13, 19, 17, 38, 37, 31, 36, 18, 16, 17, 27, 11, 19, 20, 22, 47, 39, 25, 46, 30, 30, 19, 39, 24, 34, 15, 21, 33, 16, 23, 17, 37, 28, 44, 41, 31, 28, 18, 36, 28, 49, 11, 27, 40, 33, 32, 18, 30, 17, 26, 37, 26, 35, 18, 20, 20, 47, 15, 23, 39, 52, 24, 18, 18, 22, 26, 24, 14, 29, 29, 32, 23, 26, 27, 30, 27, 29, 17, 57, 33, 28, 23, 25, 38, 41, 23, 20, 27, 20, 30, 21, 35, 38, 14, 20, 35, 28, 37, 21, 36, 17, 18, 30, 44, 32, 23, 42, 44, 60, 31, 44, 28, 17, 27, 18, 12, 23, 22, 20, 21, 15, 20, 18, 40, 27, 18, 23, 21, 35, 40, 18, 25, 64, 36, 24, 34, 9, 31, 15, 68, 26, 28, 16, 38, 32, 50, 27, 14, 27, 39, 29, 52, 20, 19, 32, 39, 39, 23, 38, 26, 24, 30, 24, 56, 29, 23, 34, 20, 13, 36, 60, 15, 42, 23, 53, 25, 20, 18, 22, 18, 24, 27, 30, 60, 25, 21, 27, 35, 18, 15, 22, 41, 23, 19, 35, 13, 35, 14, 32, 17, 37, 35, 29, 40, 36, 36, 30, 7, 21, 58, 24, 28, 27, 40, 46, 16, 30, 18, 21, 18, 21, 25, 28, 30, 28, 27, 17, 59, 20, 12, 31, 18, 49, 17, 18, 20, 25, 28, 32, 29, 26, 34, 24, 26, 22, 30, 33, 40, 56, 33, 19, 36, 15, 42, 35, 29, 34, 18, 11, 14, 18, 33, 24, 43, 22, 25, 33, 16, 25, 52, 19, 16, 18, 34, 28, 12, 37, 30, 28, 26, 27, 25, 39, 18, 35, 15, 17, 22, 26, 60, 53, 17, 51, 20, 16, 10, 20, 27, 11, 30, 19, 28, 36, 31, 51, 28, 29, 11, 30, 25, 35, 39, 18, 29, 14, 25, 36, 30, 19, 28, 21, 14, 22, 45, 32, 30, 23, 30, 32, 15, 34, 25, 52, 34, 29, 27, 36, 10, 13, 33, 21, 37, 12, 20, 18, 30, 35, 34, 14, 19, 25, 30, 44, 36, 32, 28, 13, 30, 18, 21, 32, 26, 17, 68, 23, 27, 35, 57, 21, 35, 50, 36, 27, 60, 11, 23, 16, 22, 67, 20, 30, 21, 29, 17, 16, 28, 24, 35, 20, 28, 32, 28, 20, 60, 22, 31, 41, 33, 20, 25, 9, 36, 34, 51, 26, 46, 34, 26, 30, 17, 18, 41, 20, 17, 26, 15, 53, 28, 33, 23, 17, 23, 18, 25, 23, 22, 39, 18, 27, 26, 19, 33, 32, 19, 23, 22, 36, 38, 20, 28, 30, 26, 31, 20, 26, 9, 41, 26, 26, 31, 13, 34, 38, 15, 20, 24, 25, 32, 55, 36, 27, 20, 30, 23, 18, 71, 23, 29, 29, 19, 38, 24, 39, 32, 20, 36, 11, 50, 25, 31, 16, 14, 24, 33, 14, 38, 19, 24, 35, 23, 47, 39, 20, 30, 36, 46, 18, 22, 14, 16, 21, 33, 18, 25, 28, 12, 18, 17, 18, 21, 27, 22, 27, 14, 36, 26, 18, 28, 18, 19, 27, 23, 35, 16, 26, 21, 33, 22, 40, 10, 48, 20, 9, 17, 37, 23, 19, 43, 32, 35, 20, 19, 19, 48, 15, 36, 29, 23, 27, 32, 29, 33, 22, 28, 24, 33, 25, 41, 32, 33, 31, 10, 33, 41, 20, 21, 18, 32, 25, 20, 31, 20, 21, 58, 36, 42, 28, 26, 24, 10, 24, 36, 36, 20, 33, 18, 33, 26, 27, 45, 26, 17, 45, 21, 28, 32, 20, 27, 41, 19, 36, 13, 28, 33, 31, 14, 36, 18, 19, 40, 28, 17, 22, 21, 11, 33, 37, 20, 35, 37, 19, 24, 53, 27, 18, 12, 24, 11, 29, 28, 24, 31, 29, 29, 79, 24, 23, 43, 25, 34, 21, 27, 43, 14, 38, 27, 25, 33, 18, 16, 17, 35, 29, 22, 38, 12, 33, 17, 31, 25, 7, 59, 21, 9, 21, 28, 27, 13, 60, 23, 36, 60, 20, 28, 13, 26, 31, 24, 23, 53, 33, 24, 29, 38, 20, 10, 54, 23, 37, 12, 27, 17, 47, 32, 22, 14, 14, 34, 21, 33, 30, 14, 26, 30, 33, 27, 18, 32, 9, 23, 40, 41, 30, 35, 44, 29, 21, 30, 27, 20, 9, 39, 23, 24, 18, 19], \"xaxis\": \"x\", \"yaxis\": \"y\"}, {\"marker\": {\"line\": {\"color\": \"#000000\", \"width\": 2}}, \"showlegend\": false, \"type\": \"histogram\", \"x\": [27, 55, 60, 9, 13, 23, 34, 23, 19, 35, 22, 40, 18, 36, 24, 11, 23, 23, 20, 19, 17, 20, 20, 27, 10, 59, 34, 24, 45, 23, 36, 42, 45, 17, 30, 36, 36, 38, 29, 18, 26, 60, 15, 20, 22, 59, 15, 33, 53, 11, 41, 24, 28, 16, 57, 25, 21, 29, 30, 49, 27, 40, 35, 38, 25, 32, 20, 13, 48, 15, 39, 14, 34, 16, 26, 18, 27, 32, 25, 38, 16, 16, 45, 13, 9, 17, 32, 27, 34, 38, 13, 31, 15, 6, 19, 52, 4, 30, 19, 54, 39, 34, 43, 39, 35, 27, 16, 9, 29, 15, 20, 27, 27, 59, 23, 16, 22, 24, 24, 25, 22, 35, 9, 21, 28, 35, 28, 41, 13, 27, 21, 58, 17, 24, 35, 26, 28, 34, 44, 25, 11, 20, 33, 28, 27, 30, 26, 59, 11, 30, 22, 34, 18, 31, 37, 27, 24, 39, 42, 25, 23, 44, 18, 18, 60, 23, 49, 40, 30, 38, 17, 31, 22, 22, 67, 31, 31, 20, 33, 32, 42, 39, 59, 46, 11, 23, 20, 32, 29, 34, 33, 15, 29, 17, 26, 20, 36, 33, 54, 38, 13, 22, 10, 25, 27, 28, 35, 17, 30, 37, 15, 59, 32, 35, 31, 22, 20, 34, 27, 30, 30, 16, 23, 11, 22, 22, 39, 27, 39, 24, 20, 23, 30, 35, 26, 11, 35, 25, 13, 23, 15, 26, 26, 28, 11, 31, 29, 36, 26, 27, 32, 18, 17, 25, 52, 48, 37, 16, 32, 28, 19, 20, 41, 24, 22, 31, 16, 10, 19, 36, 48, 21, 30, 14, 53, 12, 15, 25, 25, 60, 38, 15, 20, 30, 60, 34, 35, 26, 25, 38, 22, 31, 19, 36, 47, 35, 16, 15, 27, 36, 38, 40, 40, 39, 32, 38, 40, 43, 33, 28, 18, 31, 24, 22, 22, 17, 17, 53, 58, 26, 28, 10, 35, 34, 18, 38, 28, 30, 31, 13, 45, 33, 32, 18, 20, 24, 24, 18, 40, 18, 42, 39, 29, 31, 31, 24, 16, 13, 30, 56, 45, 79, 17, 32, 22, 27, 27, 19, 19, 10, 49, 37, 78, 25, 26, 5, 19, 52, 19, 33, 15, 39, 28, 36, 23, 29, 26, 19, 34, 32, 18, 9, 11, 30, 32, 22, 33, 25, 20, 40, 21, 30, 25, 36, 24, 21, 9, 21, 33, 19, 27, 13, 22, 38, 22, 16, 38, 39, 45, 37, 32, 22, 34, 32, 15, 27, 55, 29, 25, 15, 9, 36, 16, 25, 10, 51, 9, 31, 19, 12, 30, 24, 20, 46, 15, 19, 23, 34, 20, 18, 26, 29, 29, 26, 15, 16, 13, 14, 27, 24, 33, 32, 18, 49, 41, 39, 28, 28, 26, 15, 20, 21, 23, 19, 47, 36, 31, 46, 10, 32, 33, 22, 23, 32, 30, 27, 35, 34, 35, 24, 17, 30, 29, 36, 23, 15, 27, 19, 24, 27, 23, 29, 39, 31, 18, 30, 34, 17, 39, 17, 22, 18, 20, 37, 33, 20, 27, 17, 19, 27, 34, 24, 38, 11, 59, 22, 60, 35, 18, 25, 28, 19, 24, 20, 35, 9, 58, 36, 41, 20, 33, 32, 46, 21, 36, 28, 37, 29, 8, 30, 14, 41, 39, 37, 44, 21, 49, 23, 40, 21, 20, 38, 34, 29, 22, 20, 14, 59, 32, 47, 30, 27, 32, 7, 13, 19, 22, 24, 10, 20, 15, 29, 23, 51, 26, 24, 25, 19, 14, 27, 16, 32, 78, 60, 20, 39, 31, 32, 40, 16, 29, 41, 33, 20, 44, 25, 13, 30, 23, 23, 27, 22, 37, 8, 13, 22, 16, 15, 18, 40, 25, 30, 14, 30, 34, 39, 23, 33, 58, 38, 33, 20, 34, 39, 26, 23, 21, 24, 19, 28, 26, 38, 25, 41, 33, 23, 40, 31, 32, 18, 36, 17, 19, 21, 31, 30, 35, 33, 15, 22, 45, 31, 64, 33, 38, 24, 20, 36, 23, 30, 15, 26, 21, 35, 15, 22, 34, 9, 25, 38, 36, 22, 18, 23, 14, 19, 24, 16, 28, 42, 24, 16, 25, 27, 13, 28, 23, 15, 27, 25, 13, 27, 29, 32, 37, 19, 23, 34, 28, 30, 29, 29, 26, 15, 35, 28, 26, 24, 20, 32, 60, 27, 29, 30, 19, 16, 20, 35, 8, 20, 20, 19, 15, 15, 21, 44, 18, 27, 26, 30, 16, 39, 59, 31, 12, 62, 22, 72, 63, 18, 45, 22, 44, 28, 21, 32, 27, 23, 57, 22, 18, 25, 29, 33, 17, 19, 31, 24, 49, 15, 36, 35, 29, 42, 22, 28, 21, 20, 27, 40, 14, 33, 23, 32, 28, 26, 18, 21, 44, 11, 26, 17, 17, 22, 24, 24, 55, 19, 78, 42, 16, 27, 52, 26, 33, 19, 8, 25, 25, 30, 19, 20, 22, 18, 25, 35, 24, 22, 14, 28, 45, 14, 31, 28, 25, 28, 15, 44, 59, 31, 20, 38, 36, 59, 11, 39, 43, 18, 22, 20, 25, 35, 38, 30, 19, 30, 20, 32, 48, 28, 44, 32, 32, 24, 24, 33, 31, 32, 21, 11, 34, 26, 41, 24, 20, 18, 14, 55, 20, 11, 27, 15, 29, 20, 35, 26, 51, 44, 36, 23, 46, 31, 29, 21, 53, 39, 33, 40, 21, 19, 32, 21, 29, 10, 27, 38, 33, 20, 37, 31, 36, 40, 25, 41, 22, 26, 21, 27, 13, 22, 15, 26, 31, 44, 38, 21, 25, 30, 23, 23, 16, 30, 18, 17, 30, 37, 24, 21, 36, 29, 30, 41, 27, 21, 10, 32, 24, 29, 15, 61, 40, 33, 21, 15, 16, 16, 30, 44, 16, 31, 14, 17, 22, 53, 23, 27, 28, 28, 35, 16, 20, 15, 20, 32, 15, 23, 36, 26, 26, 30, 21, 30, 39, 58, 9, 23, 50, 34, 27, 17, 35, 30, 36, 19, 28, 34, 28, 28, 46, 48, 9, 22, 48, 38, 36, 15, 35, 13, 33, 38, 53, 30, 19, 26, 23, 29, 29, 40, 35, 25, 26, 23, 7, 22, 26, 30, 21, 15, 37, 17, 17, 23, 27, 19, 31, 48, 27, 16, 21, 36, 34, 26, 13, 16, 17, 30, 40, 19, 35, 31, 20, 27, 23, 23, 27, 30, 36, 42, 20, 57, 15, 45, 21, 33, 19, 40, 12, 39, 33, 27, 33, 27, 16, 33, 19, 18, 29, 39, 26, 26, 37, 45, 21, 19, 22, 11, 28, 17, 30, 18, 21, 17, 21, 35, 20, 24, 20, 28, 35, 59, 15, 38, 17, 32, 12, 17, 39, 35, 22, 34, 53, 29, 37, 32, 9, 19, 62, 37, 54, 25, 37, 33, 25, 34, 14, 17, 26, 29, 22, 68, 16, 53, 59, 16, 32, 18, 11, 30, 18, 36, 21, 22, 16, 34, 22, 27, 36, 40, 22, 20, 27, 21, 35, 20, 45, 20, 30, 21, 35, 12, 30, 37, 23, 28, 24, 21, 21, 49, 24, 26, 24, 39, 14, 20, 30, 27, 63, 23, 15, 26, 43, 24, 21, 47, 24, 40, 38, 30, 32, 20, 24, 31, 22, 20, 14, 34, 47, 35, 12, 31, 25, 26, 9, 47, 20, 20, 20, 27, 29, 15, 23, 27, 22, 34, 15, 16, 14, 29, 28, 27, 23, 9, 36, 32, 32, 17, 37, 28, 13, 31, 69, 36, 22, 38, 26, 33, 10, 30, 22, 36, 30, 25, 23, 28, 5, 21, 40, 25, 24, 33, 38, 22, 19, 52, 45, 23, 15, 38, 22, 22, 39, 22, 20, 33, 29, 15, 17, 24, 31, 15, 39, 14, 20, 19, 20, 23, 60, 52, 27, 25, 26, 19, 16, 12, 18, 60, 52, 26, 19, 30, 21, 29, 30, 29, 40, 14, 20, 35, 29, 19, 21, 46, 53, 35, 23, 28, 18, 18, 30, 17, 26, 22, 25, 29, 30, 33, 11, 11, 28, 15, 17, 15, 23, 27, 29, 46, 23, 18, 19, 33, 60, 19, 22, 26, 22, 22, 20, 44, 25, 33, 23, 36, 36, 38, 45, 24, 24, 23, 31, 47, 15, 15, 22, 31, 19, 20, 26, 10, 34, 17, 11, 48, 28, 20, 24, 20, 21, 18, 20, 23, 14, 23, 32, 27, 20, 40, 23, 39, 37, 18, 26, 29, 43, 7, 35, 18, 19, 20, 18, 26, 27, 14, 19, 19, 18, 13, 12, 29, 45, 25, 36, 35, 29, 22, 15, 15, 12, 12, 30, 9, 43, 32, 9, 20, 21, 21, 26, 29, 18, 22, 10, 50, 35, 39, 24, 30, 33, 54, 57, 25, 15, 22, 24, 23, 19, 63, 9, 43, 21, 13, 21, 13, 44, 15, 68, 18, 22, 23, 15, 28, 33, 26, 29, 20, 19, 51, 21, 19, 28, 26, 39, 25, 33, 33, 56, 34, 24, 24, 11, 35, 35, 15, 21, 23, 19, 37, 26, 49, 32, 19, 29, 26, 26, 32, 45, 28, 21, 24, 20, 35, 17, 29, 26, 25, 31, 18, 53, 17, 15, 31, 25, 28, 36, 39, 19, 18, 23, 26, 23, 24, 22, 38, 9, 26, 31, 27, 58, 39, 25, 34, 17, 15, 39, 27, 21, 34, 34, 39, 20, 44, 20, 14, 13, 34, 16, 34, 36, 59, 20, 32, 39, 59, 28, 21, 45, 22, 34, 8, 14, 43, 22, 70, 16, 31, 26, 34, 23, 21, 24, 20, 37, 50, 8, 40, 33, 33, 20, 11, 23, 22, 13, 30, 34, 36, 18, 40, 20, 53, 24, 24, 18, 14, 24, 38, 20, 48, 22, 39, 32, 22, 50, 11, 18, 35, 19, 45, 24, 37, 20, 28, 18, 19, 23, 15, 27, 19, 28, 22, 19, 37, 20, 31, 27, 26, 38, 11, 26, 36, 41, 39, 36, 38, 21, 25, 38, 28, 22, 7, 21, 28, 29, 21, 27], \"xaxis\": \"x2\", \"yaxis\": \"y2\"}],\n",
       "                        {\"annotations\": [{\"font\": {\"size\": 16}, \"showarrow\": false, \"text\": \"Description X Length\", \"x\": 0.225, \"xanchor\": \"center\", \"xref\": \"paper\", \"y\": 1.0, \"yanchor\": \"bottom\", \"yref\": \"paper\"}, {\"font\": {\"size\": 16}, \"showarrow\": false, \"text\": \"Description Y Length\", \"x\": 0.775, \"xanchor\": \"center\", \"xref\": \"paper\", \"y\": 1.0, \"yanchor\": \"bottom\", \"yref\": \"paper\"}], \"autosize\": false, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"rgb(36,36,36)\"}, \"error_y\": {\"color\": \"rgb(36,36,36)\"}, \"marker\": {\"line\": {\"color\": \"white\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"white\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"rgb(36,36,36)\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"rgb(36,36,36)\"}, \"baxis\": {\"endlinecolor\": \"rgb(36,36,36)\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"rgb(36,36,36)\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"line\": {\"color\": \"white\", \"width\": 0.6}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"rgb(237,237,237)\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"rgb(217,217,217)\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"colorscale\": {\"diverging\": [[0.0, \"rgb(103,0,31)\"], [0.1, \"rgb(178,24,43)\"], [0.2, \"rgb(214,96,77)\"], [0.3, \"rgb(244,165,130)\"], [0.4, \"rgb(253,219,199)\"], [0.5, \"rgb(247,247,247)\"], [0.6, \"rgb(209,229,240)\"], [0.7, \"rgb(146,197,222)\"], [0.8, \"rgb(67,147,195)\"], [0.9, \"rgb(33,102,172)\"], [1.0, \"rgb(5,48,97)\"]], \"sequential\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"sequentialminus\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]]}, \"colorway\": [\"#1F77B4\", \"#FF7F0E\", \"#2CA02C\", \"#D62728\", \"#9467BD\", \"#8C564B\", \"#E377C2\", \"#7F7F7F\", \"#BCBD22\", \"#17BECF\"], \"font\": {\"color\": \"rgb(36,36,36)\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"white\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"white\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\"}, \"bgcolor\": \"white\", \"radialaxis\": {\"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"white\", \"gridcolor\": \"rgb(232,232,232)\", \"gridwidth\": 2, \"linecolor\": \"rgb(36,36,36)\", \"showbackground\": true, \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\", \"zeroline\": false, \"zerolinecolor\": \"rgb(36,36,36)\"}, \"yaxis\": {\"backgroundcolor\": \"white\", \"gridcolor\": \"rgb(232,232,232)\", \"gridwidth\": 2, \"linecolor\": \"rgb(36,36,36)\", \"showbackground\": true, \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\", \"zeroline\": false, \"zerolinecolor\": \"rgb(36,36,36)\"}, \"zaxis\": {\"backgroundcolor\": \"white\", \"gridcolor\": \"rgb(232,232,232)\", \"gridwidth\": 2, \"linecolor\": \"rgb(36,36,36)\", \"showbackground\": true, \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\", \"zeroline\": false, \"zerolinecolor\": \"rgb(36,36,36)\"}}, \"shapedefaults\": {\"fillcolor\": \"black\", \"line\": {\"width\": 0}, \"opacity\": 0.3}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\"}, \"baxis\": {\"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\"}, \"bgcolor\": \"white\", \"caxis\": {\"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\", \"title\": {\"standoff\": 15}, \"zeroline\": false, \"zerolinecolor\": \"rgb(36,36,36)\"}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\", \"title\": {\"standoff\": 15}, \"zeroline\": false, \"zerolinecolor\": \"rgb(36,36,36)\"}}}, \"title\": {\"text\": \"<b>Character length distribution for True Security\", \"x\": 0.5, \"xanchor\": \"center\", \"y\": 0.9, \"yanchor\": \"top\"}, \"xaxis\": {\"anchor\": \"y\", \"domain\": [0.0, 0.45]}, \"xaxis2\": {\"anchor\": \"y2\", \"domain\": [0.55, 1.0]}, \"yaxis\": {\"anchor\": \"x\", \"automargin\": true, \"domain\": [0.0, 1.0], \"title\": {\"text\": \"<i>Frequency</i>\"}}, \"yaxis2\": {\"anchor\": \"x2\", \"automargin\": true, \"domain\": [0.0, 1.0]}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('16ec371f-47a1-4dfe-bf13-685266929687');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                });\n",
       "            </script>\n",
       "        </div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data_true = data.query('same_security == True')\n",
    "histogram_subplot(data_true, data_true, 'len_x', 'len_y', \n",
    "                  '<b>Character length distribution for True Security', \n",
    "                  'Description X Length', 'Description Y Length')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sentences that are True on the comparing, have a similar distribution, even on the frequency for the values, for now, seems a good idea use the character length as a feature.<br>\n",
    "To make this idea better, let's check it for the False values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "linkText": "Export to plot.ly",
        "plotlyServerURL": "https://plot.ly",
        "showLink": false
       },
       "data": [
        {
         "marker": {
          "line": {
           "color": "#000000",
           "width": 2
          }
         },
         "showlegend": false,
         "type": "histogram",
         "x": [
          28,
          52,
          40,
          39,
          56,
          20,
          17,
          24,
          38,
          31,
          19,
          31,
          37,
          40,
          56,
          39,
          31,
          47,
          34,
          28,
          23,
          22,
          32,
          30,
          36,
          40,
          35,
          49,
          56,
          30,
          24,
          63,
          20,
          27,
          17,
          36,
          43,
          34,
          50,
          26,
          29,
          40,
          26,
          35,
          15,
          44,
          17,
          23,
          38,
          17,
          19,
          20,
          31,
          46,
          36,
          49,
          29,
          37,
          40,
          36,
          19,
          28,
          19,
          43,
          39,
          46,
          29,
          32,
          23,
          20,
          26,
          38,
          15,
          24,
          24,
          28,
          28,
          60,
          38,
          28,
          40,
          25,
          36,
          29,
          36,
          24,
          29,
          28,
          46,
          33,
          49,
          32,
          52,
          33,
          40,
          36,
          63,
          51,
          40,
          22,
          33,
          25,
          20,
          33,
          33,
          34,
          62,
          28,
          31,
          24,
          36,
          32,
          39,
          27,
          24,
          33,
          52,
          35,
          29,
          29,
          40,
          17,
          35,
          49,
          16,
          28,
          28,
          32,
          29,
          48,
          39,
          31,
          32,
          40,
          28,
          24,
          56,
          21,
          43,
          27,
          63,
          15,
          34,
          35,
          45,
          22,
          45,
          35,
          33,
          25,
          17,
          24,
          24,
          20,
          31,
          50,
          22,
          18,
          37,
          31,
          44,
          28,
          24,
          34,
          23,
          44,
          20,
          29,
          15,
          30,
          23,
          50,
          36,
          17,
          25,
          34,
          28,
          17,
          22,
          34,
          30,
          23,
          22,
          28,
          28,
          29,
          42,
          17,
          26,
          19,
          24,
          24,
          32,
          30,
          34,
          38,
          20,
          38,
          24,
          40,
          24,
          63,
          33,
          42,
          17,
          42,
          33,
          45,
          35,
          20,
          25,
          30,
          51,
          44,
          30,
          17,
          27,
          27,
          20,
          19,
          18,
          39,
          62,
          11,
          36,
          29,
          20,
          19,
          25,
          44,
          31,
          56,
          20,
          30,
          45,
          26,
          24,
          57,
          30,
          18,
          28,
          28,
          28,
          31,
          31,
          25,
          34,
          52,
          40,
          25,
          34,
          39,
          14,
          34,
          29,
          36,
          28,
          17,
          34,
          38,
          71,
          39,
          69,
          40,
          40,
          44,
          19,
          49,
          43,
          29,
          20,
          38,
          32,
          24,
          34,
          33,
          19,
          38,
          37,
          38,
          19,
          20,
          38,
          14,
          45,
          52,
          20,
          26,
          33,
          22,
          29,
          27,
          28,
          31,
          57,
          30,
          22,
          25,
          29,
          19,
          25,
          32,
          19,
          33,
          33,
          36,
          48,
          52,
          30,
          35,
          37,
          28,
          24,
          31,
          32,
          19,
          26,
          32,
          20,
          26,
          18,
          18,
          28,
          32,
          50,
          14,
          24,
          24,
          28,
          33,
          36,
          23,
          34,
          40,
          28,
          35,
          52,
          35,
          29,
          25,
          25,
          15,
          37,
          30,
          57,
          24,
          45,
          36,
          22,
          24,
          43,
          26,
          62,
          25,
          34,
          16,
          11,
          34,
          31,
          30,
          20,
          44,
          36,
          29,
          30,
          45,
          30,
          24,
          32,
          49,
          16,
          17,
          24,
          22,
          24,
          36,
          58,
          20,
          56,
          20,
          34,
          29,
          36,
          20,
          36,
          22,
          24,
          40,
          28,
          63,
          49,
          32,
          55,
          31,
          37,
          30,
          32,
          28,
          33,
          29,
          39,
          40,
          40,
          38,
          29,
          21,
          40,
          19,
          39,
          20,
          43,
          20,
          41,
          20,
          44,
          20,
          24,
          22,
          31,
          26,
          40,
          26,
          24,
          23,
          33,
          38,
          35,
          31,
          32,
          35,
          30,
          29,
          31,
          30,
          20,
          28,
          31,
          39,
          37,
          36,
          34,
          38,
          57,
          24,
          36,
          17,
          17,
          50,
          29,
          34,
          58,
          33,
          20,
          50,
          40,
          32,
          39,
          49,
          28,
          39,
          56,
          42,
          31,
          17,
          24,
          36,
          35,
          32,
          21,
          33,
          36,
          31,
          29,
          49,
          20,
          24,
          35,
          35,
          40,
          40,
          30,
          26,
          30,
          30,
          20,
          33,
          17,
          18,
          39,
          23,
          25,
          28,
          21,
          28,
          29,
          39,
          28,
          34,
          24,
          33,
          20,
          44,
          22,
          29,
          16,
          52,
          40,
          25,
          29,
          24,
          60,
          40,
          28,
          25,
          40,
          26,
          22,
          33,
          26,
          20,
          32,
          20,
          28,
          17,
          26,
          39,
          37,
          26,
          52
         ],
         "xaxis": "x",
         "yaxis": "y"
        },
        {
         "marker": {
          "line": {
           "color": "#000000",
           "width": 2
          }
         },
         "showlegend": false,
         "type": "histogram",
         "x": [
          34,
          40,
          38,
          31,
          32,
          30,
          33,
          18,
          20,
          28,
          39,
          23,
          36,
          53,
          32,
          39,
          29,
          19,
          31,
          20,
          20,
          18,
          26,
          50,
          58,
          40,
          53,
          32,
          53,
          27,
          33,
          38,
          38,
          30,
          14,
          38,
          48,
          31,
          28,
          37,
          20,
          36,
          25,
          30,
          14,
          29,
          20,
          25,
          21,
          18,
          26,
          28,
          41,
          38,
          63,
          33,
          37,
          52,
          38,
          63,
          36,
          39,
          20,
          35,
          38,
          36,
          53,
          53,
          24,
          39,
          20,
          39,
          22,
          30,
          38,
          34,
          37,
          59,
          20,
          25,
          63,
          38,
          34,
          56,
          26,
          26,
          20,
          20,
          50,
          42,
          36,
          22,
          56,
          26,
          24,
          19,
          32,
          38,
          64,
          17,
          21,
          13,
          27,
          31,
          18,
          28,
          36,
          39,
          30,
          28,
          20,
          23,
          20,
          30,
          34,
          32,
          37,
          30,
          38,
          36,
          40,
          18,
          58,
          32,
          29,
          47,
          20,
          20,
          34,
          33,
          39,
          42,
          38,
          20,
          39,
          27,
          38,
          13,
          33,
          22,
          36,
          18,
          20,
          20,
          44,
          18,
          50,
          56,
          18,
          19,
          30,
          20,
          29,
          41,
          33,
          50,
          20,
          37,
          30,
          23,
          49,
          20,
          25,
          13,
          26,
          20,
          27,
          40,
          27,
          20,
          31,
          39,
          53,
          30,
          39,
          20,
          20,
          20,
          14,
          20,
          40,
          33,
          27,
          24,
          18,
          56,
          25,
          24,
          20,
          38,
          28,
          14,
          20,
          36,
          53,
          31,
          38,
          31,
          36,
          38,
          29,
          36,
          24,
          46,
          28,
          33,
          16,
          31,
          23,
          36,
          50,
          53,
          20,
          35,
          21,
          20,
          37,
          30,
          32,
          32,
          23,
          36,
          27,
          19,
          20,
          50,
          38,
          34,
          28,
          46,
          21,
          36,
          25,
          27,
          68,
          24,
          33,
          32,
          50,
          27,
          38,
          50,
          20,
          37,
          24,
          34,
          38,
          34,
          53,
          21,
          30,
          34,
          22,
          35,
          30,
          30,
          20,
          27,
          38,
          20,
          38,
          30,
          45,
          56,
          34,
          47,
          15,
          50,
          33,
          58,
          28,
          42,
          24,
          37,
          63,
          25,
          38,
          29,
          40,
          38,
          20,
          32,
          36,
          18,
          38,
          32,
          38,
          18,
          27,
          27,
          19,
          39,
          39,
          20,
          46,
          56,
          40,
          39,
          28,
          19,
          29,
          20,
          29,
          19,
          31,
          58,
          43,
          20,
          63,
          20,
          45,
          25,
          39,
          36,
          27,
          36,
          22,
          36,
          25,
          18,
          27,
          21,
          37,
          47,
          46,
          17,
          37,
          19,
          18,
          27,
          56,
          41,
          58,
          53,
          50,
          63,
          63,
          40,
          20,
          36,
          20,
          18,
          39,
          39,
          38,
          23,
          45,
          38,
          15,
          24,
          43,
          17,
          64,
          19,
          46,
          20,
          17,
          63,
          31,
          25,
          40,
          31,
          56,
          34,
          28,
          30,
          32,
          53,
          33,
          28,
          39,
          30,
          25,
          36,
          25,
          52,
          36,
          30,
          32,
          50,
          57,
          63,
          20,
          40,
          31,
          27,
          20,
          50,
          24,
          27,
          57,
          16,
          39,
          49,
          20,
          38,
          32,
          20,
          34,
          39,
          31,
          19,
          20,
          32,
          38,
          21,
          39,
          46,
          20,
          36,
          21,
          25,
          31,
          36,
          39,
          40,
          28,
          29,
          31,
          19,
          30,
          27,
          20,
          19,
          41,
          32,
          20,
          43,
          27,
          28,
          20,
          63,
          31,
          25,
          53,
          36,
          21,
          50,
          19,
          38,
          56,
          41,
          39,
          30,
          33,
          22,
          26,
          30,
          32,
          32,
          38,
          40,
          37,
          32,
          20,
          30,
          33,
          20,
          39,
          19,
          28,
          33,
          33,
          27,
          28,
          40,
          58,
          20,
          14,
          30,
          20,
          52,
          20,
          32,
          19,
          20,
          38,
          20,
          30,
          20,
          36,
          20,
          26,
          28,
          56,
          17,
          23,
          17,
          20,
          31,
          40,
          30,
          17,
          20,
          35,
          50,
          20,
          56,
          28,
          39,
          33,
          39,
          22,
          28,
          27,
          30,
          30,
          26,
          58,
          20,
          60,
          27,
          39,
          34,
          58,
          14,
          18,
          20,
          18,
          36,
          18,
          37,
          30,
          28,
          27,
          30,
          20,
          30,
          33
         ],
         "xaxis": "x2",
         "yaxis": "y2"
        }
       ],
       "layout": {
        "annotations": [
         {
          "font": {
           "size": 16
          },
          "showarrow": false,
          "text": "Description X Length",
          "x": 0.225,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         },
         {
          "font": {
           "size": 16
          },
          "showarrow": false,
          "text": "Description Y Length",
          "x": 0.775,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         }
        ],
        "autosize": false,
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "rgb(36,36,36)"
            },
            "error_y": {
             "color": "rgb(36,36,36)"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "baxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.6
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "rgb(237,237,237)"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "rgb(217,217,217)"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 1,
            "tickcolor": "rgb(36,36,36)",
            "ticks": "outside"
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "rgb(103,0,31)"
            ],
            [
             0.1,
             "rgb(178,24,43)"
            ],
            [
             0.2,
             "rgb(214,96,77)"
            ],
            [
             0.3,
             "rgb(244,165,130)"
            ],
            [
             0.4,
             "rgb(253,219,199)"
            ],
            [
             0.5,
             "rgb(247,247,247)"
            ],
            [
             0.6,
             "rgb(209,229,240)"
            ],
            [
             0.7,
             "rgb(146,197,222)"
            ],
            [
             0.8,
             "rgb(67,147,195)"
            ],
            [
             0.9,
             "rgb(33,102,172)"
            ],
            [
             1,
             "rgb(5,48,97)"
            ]
           ],
           "sequential": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ]
          },
          "colorway": [
           "#1F77B4",
           "#FF7F0E",
           "#2CA02C",
           "#D62728",
           "#9467BD",
           "#8C564B",
           "#E377C2",
           "#7F7F7F",
           "#BCBD22",
           "#17BECF"
          ],
          "font": {
           "color": "rgb(36,36,36)"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           }
          },
          "shapedefaults": {
           "fillcolor": "black",
           "line": {
            "width": 0
           },
           "opacity": 0.3
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "baxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          }
         }
        },
        "title": {
         "text": "<b>Character length distribution for False Security",
         "x": 0.5,
         "xanchor": "center",
         "y": 0.9,
         "yanchor": "top"
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          0.45
         ]
        },
        "xaxis2": {
         "anchor": "y2",
         "domain": [
          0.55,
          1
         ]
        },
        "yaxis": {
         "anchor": "x",
         "automargin": true,
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "<i>Frequency</i>"
         }
        },
        "yaxis2": {
         "anchor": "x2",
         "automargin": true,
         "domain": [
          0,
          1
         ]
        }
       }
      },
      "text/html": [
       "<div>\n",
       "        \n",
       "        \n",
       "            <div id=\"ba9a4422-6046-4357-976e-8b623a69a0df\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                require([\"plotly\"], function(Plotly) {\n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"ba9a4422-6046-4357-976e-8b623a69a0df\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        'ba9a4422-6046-4357-976e-8b623a69a0df',\n",
       "                        [{\"marker\": {\"line\": {\"color\": \"#000000\", \"width\": 2}}, \"showlegend\": false, \"type\": \"histogram\", \"x\": [28, 52, 40, 39, 56, 20, 17, 24, 38, 31, 19, 31, 37, 40, 56, 39, 31, 47, 34, 28, 23, 22, 32, 30, 36, 40, 35, 49, 56, 30, 24, 63, 20, 27, 17, 36, 43, 34, 50, 26, 29, 40, 26, 35, 15, 44, 17, 23, 38, 17, 19, 20, 31, 46, 36, 49, 29, 37, 40, 36, 19, 28, 19, 43, 39, 46, 29, 32, 23, 20, 26, 38, 15, 24, 24, 28, 28, 60, 38, 28, 40, 25, 36, 29, 36, 24, 29, 28, 46, 33, 49, 32, 52, 33, 40, 36, 63, 51, 40, 22, 33, 25, 20, 33, 33, 34, 62, 28, 31, 24, 36, 32, 39, 27, 24, 33, 52, 35, 29, 29, 40, 17, 35, 49, 16, 28, 28, 32, 29, 48, 39, 31, 32, 40, 28, 24, 56, 21, 43, 27, 63, 15, 34, 35, 45, 22, 45, 35, 33, 25, 17, 24, 24, 20, 31, 50, 22, 18, 37, 31, 44, 28, 24, 34, 23, 44, 20, 29, 15, 30, 23, 50, 36, 17, 25, 34, 28, 17, 22, 34, 30, 23, 22, 28, 28, 29, 42, 17, 26, 19, 24, 24, 32, 30, 34, 38, 20, 38, 24, 40, 24, 63, 33, 42, 17, 42, 33, 45, 35, 20, 25, 30, 51, 44, 30, 17, 27, 27, 20, 19, 18, 39, 62, 11, 36, 29, 20, 19, 25, 44, 31, 56, 20, 30, 45, 26, 24, 57, 30, 18, 28, 28, 28, 31, 31, 25, 34, 52, 40, 25, 34, 39, 14, 34, 29, 36, 28, 17, 34, 38, 71, 39, 69, 40, 40, 44, 19, 49, 43, 29, 20, 38, 32, 24, 34, 33, 19, 38, 37, 38, 19, 20, 38, 14, 45, 52, 20, 26, 33, 22, 29, 27, 28, 31, 57, 30, 22, 25, 29, 19, 25, 32, 19, 33, 33, 36, 48, 52, 30, 35, 37, 28, 24, 31, 32, 19, 26, 32, 20, 26, 18, 18, 28, 32, 50, 14, 24, 24, 28, 33, 36, 23, 34, 40, 28, 35, 52, 35, 29, 25, 25, 15, 37, 30, 57, 24, 45, 36, 22, 24, 43, 26, 62, 25, 34, 16, 11, 34, 31, 30, 20, 44, 36, 29, 30, 45, 30, 24, 32, 49, 16, 17, 24, 22, 24, 36, 58, 20, 56, 20, 34, 29, 36, 20, 36, 22, 24, 40, 28, 63, 49, 32, 55, 31, 37, 30, 32, 28, 33, 29, 39, 40, 40, 38, 29, 21, 40, 19, 39, 20, 43, 20, 41, 20, 44, 20, 24, 22, 31, 26, 40, 26, 24, 23, 33, 38, 35, 31, 32, 35, 30, 29, 31, 30, 20, 28, 31, 39, 37, 36, 34, 38, 57, 24, 36, 17, 17, 50, 29, 34, 58, 33, 20, 50, 40, 32, 39, 49, 28, 39, 56, 42, 31, 17, 24, 36, 35, 32, 21, 33, 36, 31, 29, 49, 20, 24, 35, 35, 40, 40, 30, 26, 30, 30, 20, 33, 17, 18, 39, 23, 25, 28, 21, 28, 29, 39, 28, 34, 24, 33, 20, 44, 22, 29, 16, 52, 40, 25, 29, 24, 60, 40, 28, 25, 40, 26, 22, 33, 26, 20, 32, 20, 28, 17, 26, 39, 37, 26, 52], \"xaxis\": \"x\", \"yaxis\": \"y\"}, {\"marker\": {\"line\": {\"color\": \"#000000\", \"width\": 2}}, \"showlegend\": false, \"type\": \"histogram\", \"x\": [34, 40, 38, 31, 32, 30, 33, 18, 20, 28, 39, 23, 36, 53, 32, 39, 29, 19, 31, 20, 20, 18, 26, 50, 58, 40, 53, 32, 53, 27, 33, 38, 38, 30, 14, 38, 48, 31, 28, 37, 20, 36, 25, 30, 14, 29, 20, 25, 21, 18, 26, 28, 41, 38, 63, 33, 37, 52, 38, 63, 36, 39, 20, 35, 38, 36, 53, 53, 24, 39, 20, 39, 22, 30, 38, 34, 37, 59, 20, 25, 63, 38, 34, 56, 26, 26, 20, 20, 50, 42, 36, 22, 56, 26, 24, 19, 32, 38, 64, 17, 21, 13, 27, 31, 18, 28, 36, 39, 30, 28, 20, 23, 20, 30, 34, 32, 37, 30, 38, 36, 40, 18, 58, 32, 29, 47, 20, 20, 34, 33, 39, 42, 38, 20, 39, 27, 38, 13, 33, 22, 36, 18, 20, 20, 44, 18, 50, 56, 18, 19, 30, 20, 29, 41, 33, 50, 20, 37, 30, 23, 49, 20, 25, 13, 26, 20, 27, 40, 27, 20, 31, 39, 53, 30, 39, 20, 20, 20, 14, 20, 40, 33, 27, 24, 18, 56, 25, 24, 20, 38, 28, 14, 20, 36, 53, 31, 38, 31, 36, 38, 29, 36, 24, 46, 28, 33, 16, 31, 23, 36, 50, 53, 20, 35, 21, 20, 37, 30, 32, 32, 23, 36, 27, 19, 20, 50, 38, 34, 28, 46, 21, 36, 25, 27, 68, 24, 33, 32, 50, 27, 38, 50, 20, 37, 24, 34, 38, 34, 53, 21, 30, 34, 22, 35, 30, 30, 20, 27, 38, 20, 38, 30, 45, 56, 34, 47, 15, 50, 33, 58, 28, 42, 24, 37, 63, 25, 38, 29, 40, 38, 20, 32, 36, 18, 38, 32, 38, 18, 27, 27, 19, 39, 39, 20, 46, 56, 40, 39, 28, 19, 29, 20, 29, 19, 31, 58, 43, 20, 63, 20, 45, 25, 39, 36, 27, 36, 22, 36, 25, 18, 27, 21, 37, 47, 46, 17, 37, 19, 18, 27, 56, 41, 58, 53, 50, 63, 63, 40, 20, 36, 20, 18, 39, 39, 38, 23, 45, 38, 15, 24, 43, 17, 64, 19, 46, 20, 17, 63, 31, 25, 40, 31, 56, 34, 28, 30, 32, 53, 33, 28, 39, 30, 25, 36, 25, 52, 36, 30, 32, 50, 57, 63, 20, 40, 31, 27, 20, 50, 24, 27, 57, 16, 39, 49, 20, 38, 32, 20, 34, 39, 31, 19, 20, 32, 38, 21, 39, 46, 20, 36, 21, 25, 31, 36, 39, 40, 28, 29, 31, 19, 30, 27, 20, 19, 41, 32, 20, 43, 27, 28, 20, 63, 31, 25, 53, 36, 21, 50, 19, 38, 56, 41, 39, 30, 33, 22, 26, 30, 32, 32, 38, 40, 37, 32, 20, 30, 33, 20, 39, 19, 28, 33, 33, 27, 28, 40, 58, 20, 14, 30, 20, 52, 20, 32, 19, 20, 38, 20, 30, 20, 36, 20, 26, 28, 56, 17, 23, 17, 20, 31, 40, 30, 17, 20, 35, 50, 20, 56, 28, 39, 33, 39, 22, 28, 27, 30, 30, 26, 58, 20, 60, 27, 39, 34, 58, 14, 18, 20, 18, 36, 18, 37, 30, 28, 27, 30, 20, 30, 33], \"xaxis\": \"x2\", \"yaxis\": \"y2\"}],\n",
       "                        {\"annotations\": [{\"font\": {\"size\": 16}, \"showarrow\": false, \"text\": \"Description X Length\", \"x\": 0.225, \"xanchor\": \"center\", \"xref\": \"paper\", \"y\": 1.0, \"yanchor\": \"bottom\", \"yref\": \"paper\"}, {\"font\": {\"size\": 16}, \"showarrow\": false, \"text\": \"Description Y Length\", \"x\": 0.775, \"xanchor\": \"center\", \"xref\": \"paper\", \"y\": 1.0, \"yanchor\": \"bottom\", \"yref\": \"paper\"}], \"autosize\": false, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"rgb(36,36,36)\"}, \"error_y\": {\"color\": \"rgb(36,36,36)\"}, \"marker\": {\"line\": {\"color\": \"white\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"white\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"rgb(36,36,36)\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"rgb(36,36,36)\"}, \"baxis\": {\"endlinecolor\": \"rgb(36,36,36)\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"rgb(36,36,36)\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"line\": {\"color\": \"white\", \"width\": 0.6}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}, \"colorscale\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"rgb(237,237,237)\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"rgb(217,217,217)\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 1, \"tickcolor\": \"rgb(36,36,36)\", \"ticks\": \"outside\"}}, \"colorscale\": {\"diverging\": [[0.0, \"rgb(103,0,31)\"], [0.1, \"rgb(178,24,43)\"], [0.2, \"rgb(214,96,77)\"], [0.3, \"rgb(244,165,130)\"], [0.4, \"rgb(253,219,199)\"], [0.5, \"rgb(247,247,247)\"], [0.6, \"rgb(209,229,240)\"], [0.7, \"rgb(146,197,222)\"], [0.8, \"rgb(67,147,195)\"], [0.9, \"rgb(33,102,172)\"], [1.0, \"rgb(5,48,97)\"]], \"sequential\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]], \"sequentialminus\": [[0.0, \"#440154\"], [0.1111111111111111, \"#482878\"], [0.2222222222222222, \"#3e4989\"], [0.3333333333333333, \"#31688e\"], [0.4444444444444444, \"#26828e\"], [0.5555555555555556, \"#1f9e89\"], [0.6666666666666666, \"#35b779\"], [0.7777777777777778, \"#6ece58\"], [0.8888888888888888, \"#b5de2b\"], [1.0, \"#fde725\"]]}, \"colorway\": [\"#1F77B4\", \"#FF7F0E\", \"#2CA02C\", \"#D62728\", \"#9467BD\", \"#8C564B\", \"#E377C2\", \"#7F7F7F\", \"#BCBD22\", \"#17BECF\"], \"font\": {\"color\": \"rgb(36,36,36)\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"white\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"white\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\"}, \"bgcolor\": \"white\", \"radialaxis\": {\"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"white\", \"gridcolor\": \"rgb(232,232,232)\", \"gridwidth\": 2, \"linecolor\": \"rgb(36,36,36)\", \"showbackground\": true, \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\", \"zeroline\": false, \"zerolinecolor\": \"rgb(36,36,36)\"}, \"yaxis\": {\"backgroundcolor\": \"white\", \"gridcolor\": \"rgb(232,232,232)\", \"gridwidth\": 2, \"linecolor\": \"rgb(36,36,36)\", \"showbackground\": true, \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\", \"zeroline\": false, \"zerolinecolor\": \"rgb(36,36,36)\"}, \"zaxis\": {\"backgroundcolor\": \"white\", \"gridcolor\": \"rgb(232,232,232)\", \"gridwidth\": 2, \"linecolor\": \"rgb(36,36,36)\", \"showbackground\": true, \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\", \"zeroline\": false, \"zerolinecolor\": \"rgb(36,36,36)\"}}, \"shapedefaults\": {\"fillcolor\": \"black\", \"line\": {\"width\": 0}, \"opacity\": 0.3}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\"}, \"baxis\": {\"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\"}, \"bgcolor\": \"white\", \"caxis\": {\"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\", \"title\": {\"standoff\": 15}, \"zeroline\": false, \"zerolinecolor\": \"rgb(36,36,36)\"}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"rgb(232,232,232)\", \"linecolor\": \"rgb(36,36,36)\", \"showgrid\": false, \"showline\": true, \"ticks\": \"outside\", \"title\": {\"standoff\": 15}, \"zeroline\": false, \"zerolinecolor\": \"rgb(36,36,36)\"}}}, \"title\": {\"text\": \"<b>Character length distribution for False Security\", \"x\": 0.5, \"xanchor\": \"center\", \"y\": 0.9, \"yanchor\": \"top\"}, \"xaxis\": {\"anchor\": \"y\", \"domain\": [0.0, 0.45]}, \"xaxis2\": {\"anchor\": \"y2\", \"domain\": [0.55, 1.0]}, \"yaxis\": {\"anchor\": \"x\", \"automargin\": true, \"domain\": [0.0, 1.0], \"title\": {\"text\": \"<i>Frequency</i>\"}}, \"yaxis2\": {\"anchor\": \"x2\", \"automargin\": true, \"domain\": [0.0, 1.0]}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('ba9a4422-6046-4357-976e-8b623a69a0df');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                });\n",
       "            </script>\n",
       "        </div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data_false = data.query('same_security == False')\n",
    "histogram_subplot(data_false, data_false, 'len_x', 'len_y', \n",
    "                  '<b>Character length distribution for False Security', \n",
    "                  'Description X Length', 'Description Y Length')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And for the False values the distribution for each sentence is different, I will use both features about the character length to feed the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Model Build\n",
    "PyCaret has a NLP module that can automatize most of the boring stuff, like lowering case, remove stop words, stemming, etc. So, a good amount of this part is just setting up PyCaret to run.<br>\n",
    "Let's import the module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pycaret.nlp import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The PyCaret setup is simple, just enter data, the text target to process and a random seed for the session.<br>\n",
    "It's possible to add custom stop words, but I will use the default from PyCaret.<br>\n",
    "Let's start with the text from `description_x`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_cb2fa3f4_d51b_11ea_99d8_e006e683d306\" ><thead>    <tr>        <th class=\"col_heading level0 col0\" >Description</th>        <th class=\"col_heading level0 col1\" >Value</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                                <td id=\"T_cb2fa3f4_d51b_11ea_99d8_e006e683d306row0_col0\" class=\"data row0 col0\" >session_id</td>\n",
       "                        <td id=\"T_cb2fa3f4_d51b_11ea_99d8_e006e683d306row0_col1\" class=\"data row0 col1\" >42</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_cb2fa3f4_d51b_11ea_99d8_e006e683d306row1_col0\" class=\"data row1 col0\" >Documents</td>\n",
       "                        <td id=\"T_cb2fa3f4_d51b_11ea_99d8_e006e683d306row1_col1\" class=\"data row1 col1\" >2142</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_cb2fa3f4_d51b_11ea_99d8_e006e683d306row2_col0\" class=\"data row2 col0\" >Vocab Size</td>\n",
       "                        <td id=\"T_cb2fa3f4_d51b_11ea_99d8_e006e683d306row2_col1\" class=\"data row2 col1\" >310</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_cb2fa3f4_d51b_11ea_99d8_e006e683d306row3_col0\" class=\"data row3 col0\" >Custom Stopwords</td>\n",
       "                        <td id=\"T_cb2fa3f4_d51b_11ea_99d8_e006e683d306row3_col1\" class=\"data row3 col1\" >False</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x1df1ee07908>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "exp_x = setup(data=data, target='description_x', session_id=RANDOM_SEED)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 310 words present in the vocabulary for `sentence_x`.<br>\n",
    "To get the features for the model, is used a Topic Modelling with PyCaret, for it, the model is tuned (LDA Model) to see what is the best number of topics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "IntProgress(value=0, description='Processing: ', max=25)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9c4d66da6add4e4fadd70930b483a6ec",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "        <script type=\"text/javascript\">\n",
       "        window.PlotlyConfig = {MathJaxConfig: 'local'};\n",
       "        if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}\n",
       "        if (typeof require !== 'undefined') {\n",
       "        require.undef(\"plotly\");\n",
       "        requirejs.config({\n",
       "            paths: {\n",
       "                'plotly': ['https://cdn.plot.ly/plotly-latest.min']\n",
       "            }\n",
       "        });\n",
       "        require(['plotly'], function(Plotly) {\n",
       "            window._Plotly = Plotly;\n",
       "        });\n",
       "        }\n",
       "        </script>\n",
       "        "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hovertemplate": "Metric=Accuracy<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "Accuracy",
         "line": {
          "color": "#636efa",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "Accuracy",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.8697478991596639,
          0.861811391223156,
          0.8408029878618114,
          0.8524743230625583,
          0.8585434173669467,
          0.8552754435107376,
          0.8632119514472456,
          0.8674136321195145,
          0.8646125116713352,
          0.8650793650793651
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "Metric=AUC<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "AUC",
         "line": {
          "color": "#EF553B",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "AUC",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.8883908742413072,
          0.8811997745163647,
          0.8552504052025309,
          0.8707846338293428,
          0.8904535104075231,
          0.8887694148559027,
          0.8940865627457437,
          0.8964169900278574,
          0.8953048072314149,
          0.8966713036915328
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "Metric=Recall<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "Recall",
         "line": {
          "color": "#00cc96",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "Recall",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.9820210787352759,
          0.9845009299442034,
          0.9783013019218847,
          0.9714817110973342,
          0.9758214507129572,
          0.977061376317421,
          0.9851208927464352,
          0.9869807811531308,
          0.9851208927464352,
          0.9857408555486671
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "Metric=Precision<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "Precision",
         "line": {
          "color": "#ab63fa",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "Precision",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.8636859323882224,
          0.8542227003765466,
          0.8375796178343949,
          0.8530212302667393,
          0.8563656147986942,
          0.8523526230394808,
          0.8552206673842842,
          0.8582210242587601,
          0.8566037735849057,
          0.8566810344827587
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "Metric=F1<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "F1",
         "line": {
          "color": "#FFA15A",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "F1",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.9190600522193212,
          0.9147465437788018,
          0.902487846725765,
          0.9084057971014493,
          0.9121993625036221,
          0.9104563835932986,
          0.9155862863728033,
          0.9181084198385238,
          0.9163783160322952,
          0.9166906889593543
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "Metric=Kappa<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "Kappa",
         "line": {
          "color": "#19d3f3",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "Kappa",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.592622989931765,
          0.5596663634027599,
          0.48299322616949436,
          0.5375729809569745,
          0.5562682630341831,
          0.5422856079353153,
          0.5644570963806426,
          0.5787861456586435,
          0.5698872614119951,
          0.5710484484548298
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "legend": {
         "title": {
          "text": "Metric"
         },
         "tracegroupgap": 0
        },
        "margin": {
         "t": 60
        },
        "plot_bgcolor": "rgb(245,245,245)",
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Logistic Regression Metrics and # of Topics",
         "x": 0.45,
         "xanchor": "center",
         "y": 0.95,
         "yanchor": "top"
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "# Topics"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "range": [
          0,
          1
         ],
         "title": {
          "text": "Score"
         }
        }
       }
      },
      "text/html": [
       "<div>\n",
       "        \n",
       "        \n",
       "            <div id=\"256a9864-762f-4739-bfe1-0e9ce5352c26\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                require([\"plotly\"], function(Plotly) {\n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"256a9864-762f-4739-bfe1-0e9ce5352c26\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        '256a9864-762f-4739-bfe1-0e9ce5352c26',\n",
       "                        [{\"hovertemplate\": \"Metric=Accuracy<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"Accuracy\", \"line\": {\"color\": \"#636efa\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"Accuracy\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.8697478991596639, 0.861811391223156, 0.8408029878618114, 0.8524743230625583, 0.8585434173669467, 0.8552754435107376, 0.8632119514472456, 0.8674136321195145, 0.8646125116713352, 0.8650793650793651], \"yaxis\": \"y\"}, {\"hovertemplate\": \"Metric=AUC<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"AUC\", \"line\": {\"color\": \"#EF553B\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"AUC\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.8883908742413072, 0.8811997745163647, 0.8552504052025309, 0.8707846338293428, 0.8904535104075231, 0.8887694148559027, 0.8940865627457437, 0.8964169900278574, 0.8953048072314149, 0.8966713036915328], \"yaxis\": \"y\"}, {\"hovertemplate\": \"Metric=Recall<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"Recall\", \"line\": {\"color\": \"#00cc96\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"Recall\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.9820210787352759, 0.9845009299442034, 0.9783013019218847, 0.9714817110973342, 0.9758214507129572, 0.977061376317421, 0.9851208927464352, 0.9869807811531308, 0.9851208927464352, 0.9857408555486671], \"yaxis\": \"y\"}, {\"hovertemplate\": \"Metric=Precision<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"Precision\", \"line\": {\"color\": \"#ab63fa\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"Precision\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.8636859323882224, 0.8542227003765466, 0.8375796178343949, 0.8530212302667393, 0.8563656147986942, 0.8523526230394808, 0.8552206673842842, 0.8582210242587601, 0.8566037735849057, 0.8566810344827587], \"yaxis\": \"y\"}, {\"hovertemplate\": \"Metric=F1<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"F1\", \"line\": {\"color\": \"#FFA15A\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"F1\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.9190600522193212, 0.9147465437788018, 0.902487846725765, 0.9084057971014493, 0.9121993625036221, 0.9104563835932986, 0.9155862863728033, 0.9181084198385238, 0.9163783160322952, 0.9166906889593543], \"yaxis\": \"y\"}, {\"hovertemplate\": \"Metric=Kappa<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"Kappa\", \"line\": {\"color\": \"#19d3f3\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"Kappa\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.592622989931765, 0.5596663634027599, 0.48299322616949436, 0.5375729809569745, 0.5562682630341831, 0.5422856079353153, 0.5644570963806426, 0.5787861456586435, 0.5698872614119951, 0.5710484484548298], \"yaxis\": \"y\"}],\n",
       "                        {\"legend\": {\"title\": {\"text\": \"Metric\"}, \"tracegroupgap\": 0}, \"margin\": {\"t\": 60}, \"plot_bgcolor\": \"rgb(245,245,245)\", \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Logistic Regression Metrics and # of Topics\", \"x\": 0.45, \"xanchor\": \"center\", \"y\": 0.95, \"yanchor\": \"top\"}, \"xaxis\": {\"anchor\": \"y\", \"domain\": [0.0, 1.0], \"title\": {\"text\": \"# Topics\"}}, \"yaxis\": {\"anchor\": \"x\", \"domain\": [0.0, 1.0], \"range\": [0, 1], \"title\": {\"text\": \"Score\"}}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('256a9864-762f-4739-bfe1-0e9ce5352c26');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                });\n",
       "            </script>\n",
       "        </div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best Model: Latent Dirichlet Allocation | # Topics: 2 | Accuracy : 0.8697\n"
     ]
    }
   ],
   "source": [
    "tuned_model_x = tune_model(model='lda', supervised_target='same_security')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on the F1-Score, the model doesn't have a high variation on the score, so I will use 4 topics, to keep it simples.<br>\n",
    "Time to create a LDA model with 4 topics and see some informations about the topic modelled data.<br>\n",
    "You can click on each box to see infos like frequency of a word, word cloud, etc.<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_x = create_model('lda', num_topics=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model is assigned to the topic weights and other infos to use after in the classification task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_x_results = assign_model(model_x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, all the steps are made again for the `description_y`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_f9c719fa_d51c_11ea_8f3d_e006e683d306\" ><thead>    <tr>        <th class=\"col_heading level0 col0\" >Description</th>        <th class=\"col_heading level0 col1\" >Value</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                                <td id=\"T_f9c719fa_d51c_11ea_8f3d_e006e683d306row0_col0\" class=\"data row0 col0\" >session_id</td>\n",
       "                        <td id=\"T_f9c719fa_d51c_11ea_8f3d_e006e683d306row0_col1\" class=\"data row0 col1\" >42</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_f9c719fa_d51c_11ea_8f3d_e006e683d306row1_col0\" class=\"data row1 col0\" >Documents</td>\n",
       "                        <td id=\"T_f9c719fa_d51c_11ea_8f3d_e006e683d306row1_col1\" class=\"data row1 col1\" >2142</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_f9c719fa_d51c_11ea_8f3d_e006e683d306row2_col0\" class=\"data row2 col0\" >Vocab Size</td>\n",
       "                        <td id=\"T_f9c719fa_d51c_11ea_8f3d_e006e683d306row2_col1\" class=\"data row2 col1\" >310</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_f9c719fa_d51c_11ea_8f3d_e006e683d306row3_col0\" class=\"data row3 col0\" >Custom Stopwords</td>\n",
       "                        <td id=\"T_f9c719fa_d51c_11ea_8f3d_e006e683d306row3_col1\" class=\"data row3 col1\" >False</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x1df23feb688>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "exp_y = setup(data=data, target='description_y', session_id=RANDOM_SEED)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, 310 words are present in the vocabulary for `sentence_y`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "IntProgress(value=0, description='Processing: ', max=25)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0994d638bdac48d2a680f9252651ebdd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "        <script type=\"text/javascript\">\n",
       "        window.PlotlyConfig = {MathJaxConfig: 'local'};\n",
       "        if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}\n",
       "        if (typeof require !== 'undefined') {\n",
       "        require.undef(\"plotly\");\n",
       "        requirejs.config({\n",
       "            paths: {\n",
       "                'plotly': ['https://cdn.plot.ly/plotly-latest.min']\n",
       "            }\n",
       "        });\n",
       "        require(['plotly'], function(Plotly) {\n",
       "            window._Plotly = Plotly;\n",
       "        });\n",
       "        }\n",
       "        </script>\n",
       "        "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hovertemplate": "Metric=Accuracy<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "Accuracy",
         "line": {
          "color": "#636efa",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "Accuracy",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.8389355742296919,
          0.8277310924369747,
          0.8361344537815126,
          0.8361344537815126,
          0.8380018674136321,
          0.8440709617180205,
          0.8417366946778712,
          0.8412698412698413,
          0.8417366946778712,
          0.842203548085901
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "Metric=AUC<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "AUC",
         "line": {
          "color": "#EF553B",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "AUC",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.8582394697149929,
          0.8400226421197337,
          0.8410164577270922,
          0.8647273980196349,
          0.8619686221473216,
          0.8498295395281954,
          0.8633450802025603,
          0.8680088646477052,
          0.8681202001225863,
          0.8680264439332128
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "Metric=Recall<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "Recall",
         "line": {
          "color": "#00cc96",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "Recall",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.9820210787352759,
          0.9646621202727836,
          0.972101673899566,
          0.9615623062616243,
          0.9578425294482331,
          0.9764414135151891,
          0.9807811531308122,
          0.977061376317421,
          0.9776813391196528,
          0.9776813391196528
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "Metric=Precision<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "Precision",
         "line": {
          "color": "#ab63fa",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "Precision",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.8336842105263158,
          0.8329764453961456,
          0.8367129135538954,
          0.8429347826086957,
          0.8470394736842105,
          0.8417958311063602,
          0.837037037037037,
          0.8387440127727515,
          0.8388297872340426,
          0.8392762107503992
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "Metric=F1<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "F1",
         "line": {
          "color": "#FFA15A",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "F1",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.9017933390264732,
          0.8939959781671935,
          0.8993404072268426,
          0.898349261511729,
          0.8990398603433226,
          0.9041331802525833,
          0.9032258064516129,
          0.9026345933562429,
          0.9029487546521615,
          0.9032073310423825
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "Metric=Kappa<br># Topics=%{x}<br>Score=%{y}<extra></extra>",
         "legendgroup": "Kappa",
         "line": {
          "color": "#19d3f3",
          "dash": "solid",
          "shape": "linear"
         },
         "mode": "lines",
         "name": "Kappa",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2,
          4,
          8,
          16,
          32,
          64,
          100,
          200,
          300,
          400
         ],
         "xaxis": "x",
         "y": [
          0.47042737590705774,
          0.44732591491426676,
          0.4718838462240891,
          0.4852110872682107,
          0.4970504049856884,
          0.4986074531741237,
          0.4836484186004838,
          0.48647943534455407,
          0.4875981387681829,
          0.4895001445484096
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "legend": {
         "title": {
          "text": "Metric"
         },
         "tracegroupgap": 0
        },
        "margin": {
         "t": 60
        },
        "plot_bgcolor": "rgb(245,245,245)",
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Logistic Regression Metrics and # of Topics",
         "x": 0.45,
         "xanchor": "center",
         "y": 0.95,
         "yanchor": "top"
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "# Topics"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "range": [
          0,
          1
         ],
         "title": {
          "text": "Score"
         }
        }
       }
      },
      "text/html": [
       "<div>\n",
       "        \n",
       "        \n",
       "            <div id=\"26463e44-880e-434d-b75b-3a7b701c7ed9\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                require([\"plotly\"], function(Plotly) {\n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"26463e44-880e-434d-b75b-3a7b701c7ed9\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        '26463e44-880e-434d-b75b-3a7b701c7ed9',\n",
       "                        [{\"hovertemplate\": \"Metric=Accuracy<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"Accuracy\", \"line\": {\"color\": \"#636efa\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"Accuracy\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.8389355742296919, 0.8277310924369747, 0.8361344537815126, 0.8361344537815126, 0.8380018674136321, 0.8440709617180205, 0.8417366946778712, 0.8412698412698413, 0.8417366946778712, 0.842203548085901], \"yaxis\": \"y\"}, {\"hovertemplate\": \"Metric=AUC<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"AUC\", \"line\": {\"color\": \"#EF553B\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"AUC\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.8582394697149929, 0.8400226421197337, 0.8410164577270922, 0.8647273980196349, 0.8619686221473216, 0.8498295395281954, 0.8633450802025603, 0.8680088646477052, 0.8681202001225863, 0.8680264439332128], \"yaxis\": \"y\"}, {\"hovertemplate\": \"Metric=Recall<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"Recall\", \"line\": {\"color\": \"#00cc96\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"Recall\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.9820210787352759, 0.9646621202727836, 0.972101673899566, 0.9615623062616243, 0.9578425294482331, 0.9764414135151891, 0.9807811531308122, 0.977061376317421, 0.9776813391196528, 0.9776813391196528], \"yaxis\": \"y\"}, {\"hovertemplate\": \"Metric=Precision<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"Precision\", \"line\": {\"color\": \"#ab63fa\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"Precision\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.8336842105263158, 0.8329764453961456, 0.8367129135538954, 0.8429347826086957, 0.8470394736842105, 0.8417958311063602, 0.837037037037037, 0.8387440127727515, 0.8388297872340426, 0.8392762107503992], \"yaxis\": \"y\"}, {\"hovertemplate\": \"Metric=F1<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"F1\", \"line\": {\"color\": \"#FFA15A\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"F1\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.9017933390264732, 0.8939959781671935, 0.8993404072268426, 0.898349261511729, 0.8990398603433226, 0.9041331802525833, 0.9032258064516129, 0.9026345933562429, 0.9029487546521615, 0.9032073310423825], \"yaxis\": \"y\"}, {\"hovertemplate\": \"Metric=Kappa<br># Topics=%{x}<br>Score=%{y}<extra></extra>\", \"legendgroup\": \"Kappa\", \"line\": {\"color\": \"#19d3f3\", \"dash\": \"solid\", \"shape\": \"linear\"}, \"mode\": \"lines\", \"name\": \"Kappa\", \"orientation\": \"v\", \"showlegend\": true, \"type\": \"scatter\", \"x\": [2, 4, 8, 16, 32, 64, 100, 200, 300, 400], \"xaxis\": \"x\", \"y\": [0.47042737590705774, 0.44732591491426676, 0.4718838462240891, 0.4852110872682107, 0.4970504049856884, 0.4986074531741237, 0.4836484186004838, 0.48647943534455407, 0.4875981387681829, 0.4895001445484096], \"yaxis\": \"y\"}],\n",
       "                        {\"legend\": {\"title\": {\"text\": \"Metric\"}, \"tracegroupgap\": 0}, \"margin\": {\"t\": 60}, \"plot_bgcolor\": \"rgb(245,245,245)\", \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Logistic Regression Metrics and # of Topics\", \"x\": 0.45, \"xanchor\": \"center\", \"y\": 0.95, \"yanchor\": \"top\"}, \"xaxis\": {\"anchor\": \"y\", \"domain\": [0.0, 1.0], \"title\": {\"text\": \"# Topics\"}}, \"yaxis\": {\"anchor\": \"x\", \"domain\": [0.0, 1.0], \"range\": [0, 1], \"title\": {\"text\": \"Score\"}}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('26463e44-880e-434d-b75b-3a7b701c7ed9');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                });\n",
       "            </script>\n",
       "        </div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best Model: Latent Dirichlet Allocation | # Topics: 64 | Accuracy : 0.8441\n"
     ]
    }
   ],
   "source": [
    "tuned_model_y = tune_model(model='lda', supervised_target='same_security')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result from the tuned model is similar from before, so 4 topics is used again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_y = create_model('lda', num_topics=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And assign the topics to the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_y_results = assign_model(model_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Topic Modelling made and is time to make the classification."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Classification\n",
    "To start, the unused columns are dropped and the results are concatenated in one dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_topics = model_x_results.drop(columns=['description_x', 'description_y', \n",
    "                                         'Dominant_Topic', 'Perc_Dominant_Topic'])\n",
    "data_topics['Topic_0_y'] = model_y_results['Topic_0']\n",
    "data_topics['Topic_1_y'] = model_y_results['Topic_1']\n",
    "data_topics['Topic_2_y'] = model_y_results['Topic_2']\n",
    "data_topics['Topic_3_y'] = model_y_results['Topic_3']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To avoid errors with PyCaret on the label feature, `same_security` is converted to object and the labels are renamed to `Yes` for `True` and `No` for `False`.<br>\n",
    "10% of the data is sampled to be used as unseen data to validate the final model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>same_security</th>\n",
       "      <th>len_x</th>\n",
       "      <th>len_y</th>\n",
       "      <th>Topic_0</th>\n",
       "      <th>Topic_1</th>\n",
       "      <th>Topic_2</th>\n",
       "      <th>Topic_3</th>\n",
       "      <th>Topic_0_y</th>\n",
       "      <th>Topic_1_y</th>\n",
       "      <th>Topic_2_y</th>\n",
       "      <th>Topic_3_y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Yes</td>\n",
       "      <td>30</td>\n",
       "      <td>27</td>\n",
       "      <td>0.272233</td>\n",
       "      <td>0.162907</td>\n",
       "      <td>0.156305</td>\n",
       "      <td>0.408555</td>\n",
       "      <td>0.056694</td>\n",
       "      <td>0.123293</td>\n",
       "      <td>0.066383</td>\n",
       "      <td>0.753630</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Yes</td>\n",
       "      <td>35</td>\n",
       "      <td>55</td>\n",
       "      <td>0.088963</td>\n",
       "      <td>0.053199</td>\n",
       "      <td>0.219810</td>\n",
       "      <td>0.638028</td>\n",
       "      <td>0.460700</td>\n",
       "      <td>0.177548</td>\n",
       "      <td>0.189938</td>\n",
       "      <td>0.171814</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>No</td>\n",
       "      <td>28</td>\n",
       "      <td>34</td>\n",
       "      <td>0.387918</td>\n",
       "      <td>0.080165</td>\n",
       "      <td>0.076917</td>\n",
       "      <td>0.455000</td>\n",
       "      <td>0.056584</td>\n",
       "      <td>0.745060</td>\n",
       "      <td>0.066187</td>\n",
       "      <td>0.132170</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Yes</td>\n",
       "      <td>53</td>\n",
       "      <td>60</td>\n",
       "      <td>0.272233</td>\n",
       "      <td>0.162907</td>\n",
       "      <td>0.156305</td>\n",
       "      <td>0.408555</td>\n",
       "      <td>0.034559</td>\n",
       "      <td>0.320199</td>\n",
       "      <td>0.578683</td>\n",
       "      <td>0.066559</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Yes</td>\n",
       "      <td>16</td>\n",
       "      <td>9</td>\n",
       "      <td>0.272233</td>\n",
       "      <td>0.162907</td>\n",
       "      <td>0.156305</td>\n",
       "      <td>0.408555</td>\n",
       "      <td>0.161175</td>\n",
       "      <td>0.347640</td>\n",
       "      <td>0.188525</td>\n",
       "      <td>0.302660</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  same_security  len_x  len_y   Topic_0   Topic_1   Topic_2   Topic_3  \\\n",
       "0           Yes     30     27  0.272233  0.162907  0.156305  0.408555   \n",
       "1           Yes     35     55  0.088963  0.053199  0.219810  0.638028   \n",
       "2            No     28     34  0.387918  0.080165  0.076917  0.455000   \n",
       "3           Yes     53     60  0.272233  0.162907  0.156305  0.408555   \n",
       "4           Yes     16      9  0.272233  0.162907  0.156305  0.408555   \n",
       "\n",
       "   Topic_0_y  Topic_1_y  Topic_2_y  Topic_3_y  \n",
       "0   0.056694   0.123293   0.066383   0.753630  \n",
       "1   0.460700   0.177548   0.189938   0.171814  \n",
       "2   0.056584   0.745060   0.066187   0.132170  \n",
       "3   0.034559   0.320199   0.578683   0.066559  \n",
       "4   0.161175   0.347640   0.188525   0.302660  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from pycaret.classification import *\n",
    "\n",
    "data_topics['same_security'] = data_topics['same_security'].astype('str')\n",
    "data_topics_dict = {'True' : 'Yes',  'False' : 'No'}\n",
    "data_topics['same_security'] = data_topics['same_security'].replace(data_topics_dict)\n",
    "train, unseen = data_sampling(data_topics, 0.9, RANDOM_SEED)\n",
    "data_topics.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same setting up from before, but now PyCaret split the dataset in 70% for Train and 30% for Test.<br>\n",
    "No feature transformation, scalling or normalization are used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Setup Succesfully Completed!\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >Description</th>        <th class=\"col_heading level0 col1\" >Value</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row0_col0\" class=\"data row0 col0\" >session_id</td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row0_col1\" class=\"data row0 col1\" >42</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row1_col0\" class=\"data row1 col0\" >Target Type</td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row1_col1\" class=\"data row1 col1\" >Binary</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row2_col0\" class=\"data row2 col0\" >Label Encoded</td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row2_col1\" class=\"data row2 col1\" >No: 0, Yes: 1</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row3_col0\" class=\"data row3 col0\" >Original Data</td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row3_col1\" class=\"data row3 col1\" >(1928, 11)</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row4_col0\" class=\"data row4 col0\" >Missing Values </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row4_col1\" class=\"data row4 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row5_col0\" class=\"data row5 col0\" >Numeric Features </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row5_col1\" class=\"data row5 col1\" >10</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row6_col0\" class=\"data row6 col0\" >Categorical Features </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row6_col1\" class=\"data row6 col1\" >0</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row7_col0\" class=\"data row7 col0\" >Ordinal Features </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row7_col1\" class=\"data row7 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row8_col0\" class=\"data row8 col0\" >High Cardinality Features </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row8_col1\" class=\"data row8 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row9_col0\" class=\"data row9 col0\" >High Cardinality Method </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row9_col1\" class=\"data row9 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row10_col0\" class=\"data row10 col0\" >Sampled Data</td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row10_col1\" class=\"data row10 col1\" >(1928, 11)</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row11_col0\" class=\"data row11 col0\" >Transformed Train Set</td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row11_col1\" class=\"data row11 col1\" >(1349, 10)</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row12_col0\" class=\"data row12 col0\" >Transformed Test Set</td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row12_col1\" class=\"data row12 col1\" >(579, 10)</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row13\" class=\"row_heading level0 row13\" >13</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row13_col0\" class=\"data row13 col0\" >Numeric Imputer </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row13_col1\" class=\"data row13 col1\" >mean</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row14\" class=\"row_heading level0 row14\" >14</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row14_col0\" class=\"data row14 col0\" >Categorical Imputer </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row14_col1\" class=\"data row14 col1\" >constant</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row15\" class=\"row_heading level0 row15\" >15</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row15_col0\" class=\"data row15 col0\" >Normalize </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row15_col1\" class=\"data row15 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row16\" class=\"row_heading level0 row16\" >16</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row16_col0\" class=\"data row16 col0\" >Normalize Method </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row16_col1\" class=\"data row16 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row17\" class=\"row_heading level0 row17\" >17</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row17_col0\" class=\"data row17 col0\" >Transformation </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row17_col1\" class=\"data row17 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row18\" class=\"row_heading level0 row18\" >18</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row18_col0\" class=\"data row18 col0\" >Transformation Method </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row18_col1\" class=\"data row18 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row19\" class=\"row_heading level0 row19\" >19</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row19_col0\" class=\"data row19 col0\" >PCA </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row19_col1\" class=\"data row19 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row20\" class=\"row_heading level0 row20\" >20</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row20_col0\" class=\"data row20 col0\" >PCA Method </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row20_col1\" class=\"data row20 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row21\" class=\"row_heading level0 row21\" >21</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row21_col0\" class=\"data row21 col0\" >PCA Components </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row21_col1\" class=\"data row21 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row22\" class=\"row_heading level0 row22\" >22</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row22_col0\" class=\"data row22 col0\" >Ignore Low Variance </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row22_col1\" class=\"data row22 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row23\" class=\"row_heading level0 row23\" >23</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row23_col0\" class=\"data row23 col0\" >Combine Rare Levels </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row23_col1\" class=\"data row23 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row24\" class=\"row_heading level0 row24\" >24</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row24_col0\" class=\"data row24 col0\" >Rare Level Threshold </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row24_col1\" class=\"data row24 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row25\" class=\"row_heading level0 row25\" >25</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row25_col0\" class=\"data row25 col0\" >Numeric Binning </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row25_col1\" class=\"data row25 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row26\" class=\"row_heading level0 row26\" >26</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row26_col0\" class=\"data row26 col0\" >Remove Outliers </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row26_col1\" class=\"data row26 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row27\" class=\"row_heading level0 row27\" >27</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row27_col0\" class=\"data row27 col0\" >Outliers Threshold </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row27_col1\" class=\"data row27 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row28\" class=\"row_heading level0 row28\" >28</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row28_col0\" class=\"data row28 col0\" >Remove Multicollinearity </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row28_col1\" class=\"data row28 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row29\" class=\"row_heading level0 row29\" >29</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row29_col0\" class=\"data row29 col0\" >Multicollinearity Threshold </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row29_col1\" class=\"data row29 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row30\" class=\"row_heading level0 row30\" >30</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row30_col0\" class=\"data row30 col0\" >Clustering </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row30_col1\" class=\"data row30 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row31\" class=\"row_heading level0 row31\" >31</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row31_col0\" class=\"data row31 col0\" >Clustering Iteration </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row31_col1\" class=\"data row31 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row32\" class=\"row_heading level0 row32\" >32</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row32_col0\" class=\"data row32 col0\" >Polynomial Features </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row32_col1\" class=\"data row32 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row33\" class=\"row_heading level0 row33\" >33</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row33_col0\" class=\"data row33 col0\" >Polynomial Degree </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row33_col1\" class=\"data row33 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row34\" class=\"row_heading level0 row34\" >34</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row34_col0\" class=\"data row34 col0\" >Trignometry Features </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row34_col1\" class=\"data row34 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row35\" class=\"row_heading level0 row35\" >35</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row35_col0\" class=\"data row35 col0\" >Polynomial Threshold </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row35_col1\" class=\"data row35 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row36\" class=\"row_heading level0 row36\" >36</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row36_col0\" class=\"data row36 col0\" >Group Features </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row36_col1\" class=\"data row36 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row37\" class=\"row_heading level0 row37\" >37</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row37_col0\" class=\"data row37 col0\" >Feature Selection </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row37_col1\" class=\"data row37 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row38\" class=\"row_heading level0 row38\" >38</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row38_col0\" class=\"data row38 col0\" >Features Selection Threshold </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row38_col1\" class=\"data row38 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row39\" class=\"row_heading level0 row39\" >39</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row39_col0\" class=\"data row39 col0\" >Feature Interaction </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row39_col1\" class=\"data row39 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row40\" class=\"row_heading level0 row40\" >40</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row40_col0\" class=\"data row40 col0\" >Feature Ratio </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row40_col1\" class=\"data row40 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row41\" class=\"row_heading level0 row41\" >41</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row41_col0\" class=\"data row41 col0\" >Interaction Threshold </td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row41_col1\" class=\"data row41 col1\" >None</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row42\" class=\"row_heading level0 row42\" >42</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row42_col0\" class=\"data row42 col0\" >Fix Imbalance</td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row42_col1\" class=\"data row42 col1\" >False</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306level0_row43\" class=\"row_heading level0 row43\" >43</th>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row43_col0\" class=\"data row43 col0\" >Fix Imbalance Method</td>\n",
       "                        <td id=\"T_5754c95c_d51e_11ea_8fc0_e006e683d306row43_col1\" class=\"data row43 col1\" >SMOTE</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x1df238af108>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "exp_clf = setup(data=train, \n",
    "                target='same_security',\n",
    "                session_id=RANDOM_SEED, \n",
    "                experiment_name=\"text-similarity\",\n",
    "                log_experiment=True,\n",
    "                silent=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare the models sorted by F1-Score to get the best.`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "    #T_b2d959f6_d51e_11ea_8043_e006e683d306 th {\n",
       "          text-align: left;\n",
       "    }#T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col0,#T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col3,#T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col7{\n",
       "            text-align:  left;\n",
       "        }#T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col5,#T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col1,#T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col2,#T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col4,#T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col6,#T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col7,#T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col3{\n",
       "            background-color:  yellow;\n",
       "            text-align:  left;\n",
       "        }#T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col8,#T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col8{\n",
       "            background-color:  lightgrey;\n",
       "            text-align:  left;\n",
       "        }</style><table id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >Model</th>        <th class=\"col_heading level0 col1\" >Accuracy</th>        <th class=\"col_heading level0 col2\" >AUC</th>        <th class=\"col_heading level0 col3\" >Recall</th>        <th class=\"col_heading level0 col4\" >Prec.</th>        <th class=\"col_heading level0 col5\" >F1</th>        <th class=\"col_heading level0 col6\" >Kappa</th>        <th class=\"col_heading level0 col7\" >MCC</th>        <th class=\"col_heading level0 col8\" >TT (Sec)</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col0\" class=\"data row0 col0\" >CatBoost Classifier</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col1\" class=\"data row0 col1\" >0.8480</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col2\" class=\"data row0 col2\" >0.8613</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col3\" class=\"data row0 col3\" >0.9397</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col4\" class=\"data row0 col4\" >0.8687</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col5\" class=\"data row0 col5\" >0.9026</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col6\" class=\"data row0 col6\" >0.5585</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col7\" class=\"data row0 col7\" >0.5692</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row0_col8\" class=\"data row0 col8\" >7.4354</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col0\" class=\"data row1 col0\" >Extreme Gradient Boosting</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col1\" class=\"data row1 col1\" >0.8480</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col2\" class=\"data row1 col2\" >0.8696</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col3\" class=\"data row1 col3\" >0.9159</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col4\" class=\"data row1 col4\" >0.8857</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col5\" class=\"data row1 col5\" >0.9003</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col6\" class=\"data row1 col6\" >0.5804</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col7\" class=\"data row1 col7\" >0.5833</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row1_col8\" class=\"data row1 col8\" >0.1665</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col0\" class=\"data row2 col0\" >Light Gradient Boosting Machine</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col1\" class=\"data row2 col1\" >0.8436</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col2\" class=\"data row2 col2\" >0.8665</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col3\" class=\"data row2 col3\" >0.9179</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col4\" class=\"data row2 col4\" >0.8791</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col5\" class=\"data row2 col5\" >0.8979</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col6\" class=\"data row2 col6\" >0.5638</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col7\" class=\"data row2 col7\" >0.5681</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row2_col8\" class=\"data row2 col8\" >0.2272</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col0\" class=\"data row3 col0\" >Gradient Boosting Classifier</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col1\" class=\"data row3 col1\" >0.8384</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col2\" class=\"data row3 col2\" >0.8578</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col3\" class=\"data row3 col3\" >0.9367</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col4\" class=\"data row3 col4\" >0.8602</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col5\" class=\"data row3 col5\" >0.8967</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col6\" class=\"data row3 col6\" >0.5283</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col7\" class=\"data row3 col7\" >0.5397</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row3_col8\" class=\"data row3 col8\" >0.2618</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col0\" class=\"data row4 col0\" >Extra Trees Classifier</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col1\" class=\"data row4 col1\" >0.8221</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col2\" class=\"data row4 col2\" >0.8237</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col3\" class=\"data row4 col3\" >0.9021</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col4\" class=\"data row4 col4\" >0.8664</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col5\" class=\"data row4 col5\" >0.8838</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col6\" class=\"data row4 col6\" >0.5051</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col7\" class=\"data row4 col7\" >0.5076</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row4_col8\" class=\"data row4 col8\" >0.3498</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col0\" class=\"data row5 col0\" >Ada Boost Classifier</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col1\" class=\"data row5 col1\" >0.8073</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col2\" class=\"data row5 col2\" >0.8133</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col3\" class=\"data row5 col3\" >0.9179</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col4\" class=\"data row5 col4\" >0.8398</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col5\" class=\"data row5 col5\" >0.8770</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col6\" class=\"data row5 col6\" >0.4351</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col7\" class=\"data row5 col7\" >0.4454</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row5_col8\" class=\"data row5 col8\" >0.1626</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col0\" class=\"data row6 col0\" >Random Forest Classifier</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col1\" class=\"data row6 col1\" >0.8028</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col2\" class=\"data row6 col2\" >0.8187</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col3\" class=\"data row6 col3\" >0.8635</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col4\" class=\"data row6 col4\" >0.8722</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col5\" class=\"data row6 col5\" >0.8675</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col6\" class=\"data row6 col6\" >0.4810</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col7\" class=\"data row6 col7\" >0.4821</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row6_col8\" class=\"data row6 col8\" >0.1271</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col0\" class=\"data row7 col0\" >Ridge Classifier</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col1\" class=\"data row7 col1\" >0.7621</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col2\" class=\"data row7 col2\" >0.0000</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col3\" class=\"data row7 col3\" >0.9634</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col4\" class=\"data row7 col4\" >0.7742</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col5\" class=\"data row7 col5\" >0.8584</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col6\" class=\"data row7 col6\" >0.1636</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col7\" class=\"data row7 col7\" >0.2199</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row7_col8\" class=\"data row7 col8\" >0.0085</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col0\" class=\"data row8 col0\" >Logistic Regression</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col1\" class=\"data row8 col1\" >0.7583</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col2\" class=\"data row8 col2\" >0.7011</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col3\" class=\"data row8 col3\" >0.9594</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col4\" class=\"data row8 col4\" >0.7728</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col5\" class=\"data row8 col5\" >0.8561</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col6\" class=\"data row8 col6\" >0.1538</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col7\" class=\"data row8 col7\" >0.2025</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row8_col8\" class=\"data row8 col8\" >0.0924</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col0\" class=\"data row9 col0\" >Linear Discriminant Analysis</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col1\" class=\"data row9 col1\" >0.7554</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col2\" class=\"data row9 col2\" >0.6951</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col3\" class=\"data row9 col3\" >0.9376</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col4\" class=\"data row9 col4\" >0.7802</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col5\" class=\"data row9 col5\" >0.8516</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col6\" class=\"data row9 col6\" >0.1859</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col7\" class=\"data row9 col7\" >0.2202</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row9_col8\" class=\"data row9 col8\" >0.0165</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col0\" class=\"data row10 col0\" >Decision Tree Classifier</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col1\" class=\"data row10 col1\" >0.7687</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col2\" class=\"data row10 col2\" >0.7171</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col3\" class=\"data row10 col3\" >0.8230</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col4\" class=\"data row10 col4\" >0.8626</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col5\" class=\"data row10 col5\" >0.8421</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col6\" class=\"data row10 col6\" >0.4105</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col7\" class=\"data row10 col7\" >0.4129</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row10_col8\" class=\"data row10 col8\" >0.0116</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col0\" class=\"data row11 col0\" >K Neighbors Classifier</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col1\" class=\"data row11 col1\" >0.7398</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col2\" class=\"data row11 col2\" >0.6488</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col3\" class=\"data row11 col3\" >0.8843</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col4\" class=\"data row11 col4\" >0.7927</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col5\" class=\"data row11 col5\" >0.8359</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col6\" class=\"data row11 col6\" >0.2162</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col7\" class=\"data row11 col7\" >0.2237</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row11_col8\" class=\"data row11 col8\" >0.0083</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col0\" class=\"data row12 col0\" >Quadratic Discriminant Analysis</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col1\" class=\"data row12 col1\" >0.7494</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col2\" class=\"data row12 col2\" >0.7744</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col3\" class=\"data row12 col3\" >0.8249</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col4\" class=\"data row12 col4\" >0.8383</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col5\" class=\"data row12 col5\" >0.8315</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col6\" class=\"data row12 col6\" >0.3432</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col7\" class=\"data row12 col7\" >0.3436</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row12_col8\" class=\"data row12 col8\" >0.0078</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row13\" class=\"row_heading level0 row13\" >13</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col0\" class=\"data row13 col0\" >Naive Bayes</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col1\" class=\"data row13 col1\" >0.7324</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col2\" class=\"data row13 col2\" >0.7417</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col3\" class=\"data row13 col3\" >0.8268</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col4\" class=\"data row13 col4\" >0.8177</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col5\" class=\"data row13 col5\" >0.8220</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col6\" class=\"data row13 col6\" >0.2817</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col7\" class=\"data row13 col7\" >0.2825</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row13_col8\" class=\"data row13 col8\" >0.0054</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306level0_row14\" class=\"row_heading level0 row14\" >14</th>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col0\" class=\"data row14 col0\" >SVM - Linear Kernel</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col1\" class=\"data row14 col1\" >0.6114</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col2\" class=\"data row14 col2\" >0.0000</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col3\" class=\"data row14 col3\" >0.6576</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col4\" class=\"data row14 col4\" >0.8290</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col5\" class=\"data row14 col5\" >0.6588</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col6\" class=\"data row14 col6\" >0.1025</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col7\" class=\"data row14 col7\" >0.1357</td>\n",
       "                        <td id=\"T_b2d959f6_d51e_11ea_8043_e006e683d306row14_col8\" class=\"data row14 col8\" >0.0150</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x1df23af3b88>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "top_model = compare_models(sort='F1',\n",
    "                           fold=K_FOLDS,\n",
    "                           n_select=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best model by F1-Score is the Catboost Classifier, but the difference from XGBoost is minimal and this model got a better score on the other metrics, let's take XGBoost as the base model.\n",
    "Tune the model to see if can get any improvement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "#T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col0,#T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col1,#T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col2,#T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col3,#T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col4,#T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col5,#T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col6{\n",
       "            background:  yellow;\n",
       "        }</style><table id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >Accuracy</th>        <th class=\"col_heading level0 col1\" >AUC</th>        <th class=\"col_heading level0 col2\" >Recall</th>        <th class=\"col_heading level0 col3\" >Prec.</th>        <th class=\"col_heading level0 col4\" >F1</th>        <th class=\"col_heading level0 col5\" >Kappa</th>        <th class=\"col_heading level0 col6\" >MCC</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                        <th id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row0_col0\" class=\"data row0 col0\" >0.8444</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row0_col1\" class=\"data row0 col1\" >0.8865</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row0_col2\" class=\"data row0 col2\" >0.9163</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row0_col3\" class=\"data row0 col3\" >0.8815</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row0_col4\" class=\"data row0 col4\" >0.8986</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row0_col5\" class=\"data row0 col5\" >0.5658</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row0_col6\" class=\"data row0 col6\" >0.5677</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row1_col0\" class=\"data row1 col0\" >0.8444</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row1_col1\" class=\"data row1 col1\" >0.8241</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row1_col2\" class=\"data row1 col2\" >0.9109</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row1_col3\" class=\"data row1 col3\" >0.8846</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row1_col4\" class=\"data row1 col4\" >0.8976</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row1_col5\" class=\"data row1 col5\" >0.5748</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row1_col6\" class=\"data row1 col6\" >0.5758</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row2_col0\" class=\"data row2 col0\" >0.8519</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row2_col1\" class=\"data row2 col1\" >0.8944</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row2_col2\" class=\"data row2 col2\" >0.9109</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row2_col3\" class=\"data row2 col3\" >0.8932</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row2_col4\" class=\"data row2 col4\" >0.9020</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row2_col5\" class=\"data row2 col5\" >0.5990</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row2_col6\" class=\"data row2 col6\" >0.5995</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row3_col0\" class=\"data row3 col0\" >0.8407</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row3_col1\" class=\"data row3 col1\" >0.8421</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row3_col2\" class=\"data row3 col2\" >0.9406</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row3_col3\" class=\"data row3 col3\" >0.8597</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row3_col4\" class=\"data row3 col4\" >0.8983</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row3_col5\" class=\"data row3 col5\" >0.5342</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row3_col6\" class=\"data row3 col6\" >0.5459</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row4_col0\" class=\"data row4 col0\" >0.8216</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row4_col1\" class=\"data row4 col1\" >0.8442</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row4_col2\" class=\"data row4 col2\" >0.9059</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row4_col3\" class=\"data row4 col3\" >0.8632</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row4_col4\" class=\"data row4 col4\" >0.8841</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row4_col5\" class=\"data row4 col5\" >0.4979</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row4_col6\" class=\"data row4 col6\" >0.5007</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306level0_row5\" class=\"row_heading level0 row5\" >Mean</th>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col0\" class=\"data row5 col0\" >0.8406</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col1\" class=\"data row5 col1\" >0.8583</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col2\" class=\"data row5 col2\" >0.9169</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col3\" class=\"data row5 col3\" >0.8765</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col4\" class=\"data row5 col4\" >0.8961</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col5\" class=\"data row5 col5\" >0.5543</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row5_col6\" class=\"data row5 col6\" >0.5579</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306level0_row6\" class=\"row_heading level0 row6\" >SD</th>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row6_col0\" class=\"data row6 col0\" >0.0102</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row6_col1\" class=\"data row6 col1\" >0.0273</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row6_col2\" class=\"data row6 col2\" >0.0123</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row6_col3\" class=\"data row6 col3\" >0.0129</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row6_col4\" class=\"data row6 col4\" >0.0062</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row6_col5\" class=\"data row6 col5\" >0.0350</td>\n",
       "                        <td id=\"T_0c7f7a92_d51f_11ea_a2fe_e006e683d306row6_col6\" class=\"data row6 col6\" >0.0334</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x1df242aa548>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tuned_model = tune_model(top_model[1], optimize='F1',\n",
    "                         choose_better=True, fold=K_FOLDS);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The tuned model doesn't get any improvement, so the base model is the best.<br>\n",
    "Time to build a Bagging Ensemble."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "#T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col0,#T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col1,#T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col2,#T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col3,#T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col4,#T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col5,#T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col6{\n",
       "            background:  yellow;\n",
       "        }</style><table id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >Accuracy</th>        <th class=\"col_heading level0 col1\" >AUC</th>        <th class=\"col_heading level0 col2\" >Recall</th>        <th class=\"col_heading level0 col3\" >Prec.</th>        <th class=\"col_heading level0 col4\" >F1</th>        <th class=\"col_heading level0 col5\" >Kappa</th>        <th class=\"col_heading level0 col6\" >MCC</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                        <th id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row0_col0\" class=\"data row0 col0\" >0.8593</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row0_col1\" class=\"data row0 col1\" >0.8879</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row0_col2\" class=\"data row0 col2\" >0.9360</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row0_col3\" class=\"data row0 col3\" >0.8837</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row0_col4\" class=\"data row0 col4\" >0.9091</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row0_col5\" class=\"data row0 col5\" >0.5987</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row0_col6\" class=\"data row0 col6\" >0.6036</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row1_col0\" class=\"data row1 col0\" >0.8444</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row1_col1\" class=\"data row1 col1\" >0.8363</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row1_col2\" class=\"data row1 col2\" >0.9208</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row1_col3\" class=\"data row1 col3\" >0.8774</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row1_col4\" class=\"data row1 col4\" >0.8986</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row1_col5\" class=\"data row1 col5\" >0.5660</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row1_col6\" class=\"data row1 col6\" >0.5691</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row2_col0\" class=\"data row2 col0\" >0.8556</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row2_col1\" class=\"data row2 col1\" >0.8987</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row2_col2\" class=\"data row2 col2\" >0.9356</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row2_col3\" class=\"data row2 col3\" >0.8791</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row2_col4\" class=\"data row2 col4\" >0.9065</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row2_col5\" class=\"data row2 col5\" >0.5908</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row2_col6\" class=\"data row2 col6\" >0.5963</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row3_col0\" class=\"data row3 col0\" >0.8370</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row3_col1\" class=\"data row3 col1\" >0.8463</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row3_col2\" class=\"data row3 col2\" >0.9505</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row3_col3\" class=\"data row3 col3\" >0.8496</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row3_col4\" class=\"data row3 col4\" >0.8972</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row3_col5\" class=\"data row3 col5\" >0.5102</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row3_col6\" class=\"data row3 col6\" >0.5295</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row4_col0\" class=\"data row4 col0\" >0.8550</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row4_col1\" class=\"data row4 col1\" >0.8693</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row4_col2\" class=\"data row4 col2\" >0.9257</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row4_col3\" class=\"data row4 col3\" >0.8863</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row4_col4\" class=\"data row4 col4\" >0.9056</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row4_col5\" class=\"data row4 col5\" >0.5942</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row4_col6\" class=\"data row4 col6\" >0.5968</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306level0_row5\" class=\"row_heading level0 row5\" >Mean</th>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col0\" class=\"data row5 col0\" >0.8503</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col1\" class=\"data row5 col1\" >0.8677</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col2\" class=\"data row5 col2\" >0.9337</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col3\" class=\"data row5 col3\" >0.8752</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col4\" class=\"data row5 col4\" >0.9034</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col5\" class=\"data row5 col5\" >0.5720</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row5_col6\" class=\"data row5 col6\" >0.5791</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306level0_row6\" class=\"row_heading level0 row6\" >SD</th>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row6_col0\" class=\"data row6 col0\" >0.0083</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row6_col1\" class=\"data row6 col1\" >0.0237</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row6_col2\" class=\"data row6 col2\" >0.0102</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row6_col3\" class=\"data row6 col3\" >0.0132</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row6_col4\" class=\"data row6 col4\" >0.0047</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row6_col5\" class=\"data row6 col5\" >0.0329</td>\n",
       "                        <td id=\"T_5f89aa9e_d520_11ea_a2fd_e006e683d306row6_col6\" class=\"data row6 col6\" >0.0275</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x1df28973808>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "bagged_model = ensemble_model(tuned_model, optimize=\"F1\",\n",
    "                              fold=K_FOLDS) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now a Boosting Ensemble."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "#T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col0,#T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col1,#T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col2,#T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col3,#T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col4,#T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col5,#T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col6{\n",
       "            background:  yellow;\n",
       "        }</style><table id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >Accuracy</th>        <th class=\"col_heading level0 col1\" >AUC</th>        <th class=\"col_heading level0 col2\" >Recall</th>        <th class=\"col_heading level0 col3\" >Prec.</th>        <th class=\"col_heading level0 col4\" >F1</th>        <th class=\"col_heading level0 col5\" >Kappa</th>        <th class=\"col_heading level0 col6\" >MCC</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                        <th id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row0_col0\" class=\"data row0 col0\" >0.7519</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row0_col1\" class=\"data row0 col1\" >0.5000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row0_col2\" class=\"data row0 col2\" >1.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row0_col3\" class=\"data row0 col3\" >0.7519</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row0_col4\" class=\"data row0 col4\" >0.8584</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row0_col5\" class=\"data row0 col5\" >0.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row0_col6\" class=\"data row0 col6\" >0.0000</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row1_col0\" class=\"data row1 col0\" >0.7481</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row1_col1\" class=\"data row1 col1\" >0.5000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row1_col2\" class=\"data row1 col2\" >1.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row1_col3\" class=\"data row1 col3\" >0.7481</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row1_col4\" class=\"data row1 col4\" >0.8559</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row1_col5\" class=\"data row1 col5\" >0.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row1_col6\" class=\"data row1 col6\" >0.0000</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row2_col0\" class=\"data row2 col0\" >0.7481</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row2_col1\" class=\"data row2 col1\" >0.5000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row2_col2\" class=\"data row2 col2\" >1.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row2_col3\" class=\"data row2 col3\" >0.7481</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row2_col4\" class=\"data row2 col4\" >0.8559</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row2_col5\" class=\"data row2 col5\" >0.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row2_col6\" class=\"data row2 col6\" >0.0000</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row3_col0\" class=\"data row3 col0\" >0.7481</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row3_col1\" class=\"data row3 col1\" >0.5000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row3_col2\" class=\"data row3 col2\" >1.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row3_col3\" class=\"data row3 col3\" >0.7481</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row3_col4\" class=\"data row3 col4\" >0.8559</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row3_col5\" class=\"data row3 col5\" >0.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row3_col6\" class=\"data row3 col6\" >0.0000</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row4_col0\" class=\"data row4 col0\" >0.7509</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row4_col1\" class=\"data row4 col1\" >0.5000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row4_col2\" class=\"data row4 col2\" >1.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row4_col3\" class=\"data row4 col3\" >0.7509</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row4_col4\" class=\"data row4 col4\" >0.8577</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row4_col5\" class=\"data row4 col5\" >0.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row4_col6\" class=\"data row4 col6\" >0.0000</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306level0_row5\" class=\"row_heading level0 row5\" >Mean</th>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col0\" class=\"data row5 col0\" >0.7494</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col1\" class=\"data row5 col1\" >0.5000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col2\" class=\"data row5 col2\" >1.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col3\" class=\"data row5 col3\" >0.7494</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col4\" class=\"data row5 col4\" >0.8568</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col5\" class=\"data row5 col5\" >0.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row5_col6\" class=\"data row5 col6\" >0.0000</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306level0_row6\" class=\"row_heading level0 row6\" >SD</th>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row6_col0\" class=\"data row6 col0\" >0.0016</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row6_col1\" class=\"data row6 col1\" >0.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row6_col2\" class=\"data row6 col2\" >0.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row6_col3\" class=\"data row6 col3\" >0.0016</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row6_col4\" class=\"data row6 col4\" >0.0011</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row6_col5\" class=\"data row6 col5\" >0.0000</td>\n",
       "                        <td id=\"T_75ce801e_d520_11ea_8ab7_e006e683d306row6_col6\" class=\"data row6 col6\" >0.0000</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x1df2894a148>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "boosted_model = ensemble_model(tuned_model, optimize=\"F1\",\n",
    "                               fold=K_FOLDS, method=\"Boosting\") "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Bagged Model is the best and is saved as the best model and used to predict on the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>Accuracy</th>\n",
       "      <th>AUC</th>\n",
       "      <th>Recall</th>\n",
       "      <th>Prec.</th>\n",
       "      <th>F1</th>\n",
       "      <th>Kappa</th>\n",
       "      <th>MCC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bagging Classifier</td>\n",
       "      <td>0.8601</td>\n",
       "      <td>0.8719</td>\n",
       "      <td>0.924</td>\n",
       "      <td>0.8931</td>\n",
       "      <td>0.9083</td>\n",
       "      <td>0.6141</td>\n",
       "      <td>0.6157</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                Model  Accuracy     AUC  Recall   Prec.      F1   Kappa  \\\n",
       "0  Bagging Classifier    0.8601  0.8719   0.924  0.8931  0.9083  0.6141   \n",
       "\n",
       "      MCC  \n",
       "0  0.6157  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "best_model = bagged_model\n",
    "predict_model(best_model);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check the model hyperparameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Parameters</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>base_estimator__objective</th>\n",
       "      <td>binary:logistic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__base_score</th>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__booster</th>\n",
       "      <td>gbtree</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__colsample_bylevel</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__colsample_bynode</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__colsample_bytree</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__gamma</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__gpu_id</th>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__importance_type</th>\n",
       "      <td>gain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__interaction_constraints</th>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__learning_rate</th>\n",
       "      <td>0.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__max_delta_step</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__max_depth</th>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__min_child_weight</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__missing</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__monotone_constraints</th>\n",
       "      <td>()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__n_estimators</th>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__n_jobs</th>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__num_parallel_tree</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__random_state</th>\n",
       "      <td>42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__reg_alpha</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__reg_lambda</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__scale_pos_weight</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__subsample</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__tree_method</th>\n",
       "      <td>exact</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__validate_parameters</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator__verbosity</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>base_estimator</th>\n",
       "      <td>XGBClassifier(base_score=0.5, booster='gbtree'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bootstrap</th>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bootstrap_features</th>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max_features</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max_samples</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>n_estimators</th>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>n_jobs</th>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>oob_score</th>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>random_state</th>\n",
       "      <td>42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>verbose</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>warm_start</th>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                Parameters\n",
       "base_estimator__objective                                                  binary:logistic\n",
       "base_estimator__base_score                                                             0.5\n",
       "base_estimator__booster                                                             gbtree\n",
       "base_estimator__colsample_bylevel                                                        1\n",
       "base_estimator__colsample_bynode                                                         1\n",
       "base_estimator__colsample_bytree                                                         1\n",
       "base_estimator__gamma                                                                    0\n",
       "base_estimator__gpu_id                                                                  -1\n",
       "base_estimator__importance_type                                                       gain\n",
       "base_estimator__interaction_constraints                                                   \n",
       "base_estimator__learning_rate                                                          0.3\n",
       "base_estimator__max_delta_step                                                           0\n",
       "base_estimator__max_depth                                                                6\n",
       "base_estimator__min_child_weight                                                         1\n",
       "base_estimator__missing                                                                NaN\n",
       "base_estimator__monotone_constraints                                                    ()\n",
       "base_estimator__n_estimators                                                           100\n",
       "base_estimator__n_jobs                                                                  -1\n",
       "base_estimator__num_parallel_tree                                                        1\n",
       "base_estimator__random_state                                                            42\n",
       "base_estimator__reg_alpha                                                                0\n",
       "base_estimator__reg_lambda                                                               1\n",
       "base_estimator__scale_pos_weight                                                         1\n",
       "base_estimator__subsample                                                                1\n",
       "base_estimator__tree_method                                                          exact\n",
       "base_estimator__validate_parameters                                                      1\n",
       "base_estimator__verbosity                                                                0\n",
       "base_estimator                           XGBClassifier(base_score=0.5, booster='gbtree'...\n",
       "bootstrap                                                                             True\n",
       "bootstrap_features                                                                   False\n",
       "max_features                                                                             1\n",
       "max_samples                                                                              1\n",
       "n_estimators                                                                            10\n",
       "n_jobs                                                                                  -1\n",
       "oob_score                                                                            False\n",
       "random_state                                                                            42\n",
       "verbose                                                                                  0\n",
       "warm_start                                                                           False"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_model(best_model, plot=\"parameter\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, plot the AUC Score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAFlCAYAAAAZGcpRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAACGa0lEQVR4nO3dd3hT5RfA8W+SNulelLJpoUzZRRBZsiobZA8FQRRcuJApILKHioii8AMBEQVBEBAEWQIyBAoUQZZQyqYUukeacX9/1MaW7tI2TTmf5/GR3NzcnFxKT9573/cclaIoCkIIIYQoFtTWDkAIIYQQ+UcSuxBCCFGMSGIXQgghihFJ7EIIIUQxIoldCCGEKEYksQshhBDFiJ21AxAir6pXr061atVQq9WoVCoSEhJwcXFhypQp1KlTB4D4+HgWLlzInj170Gq1ALRp04bXXnsNBwcHy7E2btzImjVrSExMxGAw0LBhQ0aPHo2bm1uG753b/Qvb7du3eeWVV9BoNEyZMoUGDRrk+hg3btwgMDCQatWqAWA2m3FwcGDcuHE0bNgwv0Nm9+7dHD58mIkTJ+b5GNHR0Xz++ef8+eeflp+L559/nj59+gDJf/cLFiyw/HzkhwULFuDr68tzzz3Hhg0b+Pzzz/H396du3bqW7UIUKkUIG1WtWjXl/v37abYtXbpU6du3r6IoimIwGJS+ffsqM2fOVOLj4xVFUZT4+Hhl2rRpysCBAxWDwaAoiqJ89dVXyoABA5R79+4piqIoSUlJypQpU5QBAwZk+L653d8aNm7cqLz44ouPdIzr168r9evXT7Nt69atSmBg4CMdt6AkJiYqXbt2VZYsWWL5u71x44bSrl075ccff1QURVFat26tnD59usBiGDRokPLzzz8X2PGFyAkZsYtiw2g0cvv2bdzd3QHYvn07ZrOZ8ePHW/ZxdHTkgw8+4LnnnmPnzp0888wzLF68mI0bN+Lt7Q2Avb09Y8aMYefOnSQlJVlG+pB8BSC7/RcvXkxERASTJ08GYOHChZbHgwYNwt3dnStXrtCvXz8WLVrEgQMH0Gq1mEwmWrVqxYoVK/Dx8WHGjBlcvHgRg8HA008/zZgxY7Czs+Pzzz9n586d2Nvb4+npyaxZs/Dx8bHEeOTIET777DNiYmIYNGgQq1atYu3ataxatQq1Wo23tzeTJk2iUqVKjBs3jsjISK5fv06rVq0YPXp0luc4MjKSkiVLAskj+JkzZxIcHExcXByKojB9+nQaNmzIgwcPGD9+PNeuXcPDw4OSJUtStWpVRo4cyb59+/j4449Rq9XUrFmTQ4cO8f3333P06FF27NjB4sWLGTRoEPXr1+fEiRPcvn2bp59+mmnTpqFWq9mwYQNLlizBwcGBJk2a8O233/L333+zbds2nJyceOWVVyzxlitXjs8++wyDwZDmc2QV+/Hjx5k9ezZmsxmAESNG0L59+0y3jxs3jqpVq3L37l3++usvbty4QUREBOfPn6dq1aoMGzaMy5cvM2PGDCIjIzGZTAwaNIjevXvz559/MmPGDJycnIiLi+Onn35K8/MmRF5IYhc27cUXXwQgIiICnU5H69atmTVrFgAnT57kySefTPcalUrF008/TVBQEBUqVMDBwQE/P780+zg6OtKtW7d0r71y5Uqu9s+Im5sb27ZtA2Dnzp3s2bOHDh068Mcff1C+fHn8/f0ZP348tWrVYvbs2ZhMJsaNG8fy5cvp0qULK1eu5PDhw2i1Wr755htOnz5Nu3btLMdv0qQJb731liVJHj58mKVLl7J27Vq8vLzYsGEDb7zxBlu3bgUgMTHR8ueHJSYm0r17dyD5Mve9e/f48ssvAQgODiYsLIy1a9eiVqtZsmQJ//vf/2jYsCHTp0+nSpUqLF68mLCwMHr27EnVqlWJiIhgzJgxrFy5kho1arBx40Y2btyY4Xtfu3aNVatWER8fT8eOHTl69Cje3t58/PHHbNiwgdKlS/PFF19gMpkAOHPmDAEBAemOU6tWrXTbsop94cKFDB06lM6dO3P+/HnWrl1L+/btM92eYsKECZw7d47nn3+eDh06MG7cOCD5C+dbb73F3LlzqVWrFjExMfTr148qVaoAcOnSJXbt2kW5cuWy+KkRIucksQubtnLlSry8vDh79izDhw/nqaeeokSJEpbnjUZjhq9LSkpCo9GgVqstI7CcyO3+GUn9ZaN3795s3LiRDh06sGHDBvr27QvA77//zl9//cX69euB5AQLUKpUKWrUqEGPHj1o2bIlLVu25Omnn87y/Q4cOECnTp3w8vICoGfPnsyYMYMbN24AZHm/3MHBgU2bNlkeHzp0iDfeeIPNmzfToEED3N3dWbNmDdevX+fPP//E2dkZgH379lkSto+PDx06dADg+PHj+Pv7U6NGDQB69OjB9OnTM3zv1q1bo1arcXFxwdfXl6ioKM6fP0+zZs0oXbo0AC+88AILFy4Ekr+wKTmskJ1V7B07dmTq1Kns2bOHpk2b8t5772W5PTtXr17l2rVrTJgwwbItMTGRv//+G39/f8qUKSNJXeQrmRUvioVatWoxfvx4xo0bZ0lYAQEBHD9+PF0iNpvNHDt2jAYNGlClShWMRiNXr15Ns49er+eVV17h7t27abbnZP+HE8zDl4GdnJwsf+7YsSPBwcFcvnyZY8eOWRKg2WxmwYIFbNq0iU2bNrFu3TomT56MWq3mu+++Y9asWXh4eDBz5kzmzp2b5bnJ6IuIoiiWLz2p48lO06ZNqVixIn/99Re///47I0aMAKBt27YMGDDAsp+dnV2ac6BWJ/+q0Wg06ZJvynMPSz25MeWcPvx6jUZj+XP9+vU5depUuuPs3r2bOXPmpNmWVez9+/dn8+bNNGvWjD/++INu3bqh1+sz3Z4dk8mEq6ur5e9y06ZN/Pjjj/Tq1QvI3fkXIicksYtio0uXLtStW9dyKb59+/Y4Ojoyc+ZMy4g3MTGRadOm4ezsTGBgIFqtlldeeYUPPviA8PBwIHk0P3PmTBISEihVqlSa98jJ/p6enpw9exZFUYiNjWXv3r2ZxqzT6ejcuTPjxo3j2WefxdHREYDmzZuzYsUKFEUhKSmJ1157je+++47z58/TpUsX/P39GTFiBEOGDOGvv/7K8ry0aNGCbdu28eDBAwB++uknPDw88PX1zfU5DgkJ4ebNm9SsWZODBw/SunVrBg4cSO3atdm1a5flsvgzzzxjudoQERHBrl27UKlUBAQEcPXqVc6fPw/Ajh07iI6ORqVS5ej9mzdvzuHDhy1fuNatW2d57tlnnyU2Npb//e9/ljiuX7/O7Nmz8ff3T3OcrGLv378/586do2fPnkybNs1yCyKz7dmpVKlSmisft2/fpkuXLpw5cyZHn1mI3JJL8aJYmTRpEt26dePAgQO0aNGCb775hkWLFtGzZ0/UajUmk4k2bdrwzTffYG9vD8Crr76Ko6Mjw4YNA5JH340bN2bRokUZvkd2+6e8/7PPPkupUqVo3LhxlpeI+/Tpw3fffceUKVMs2z744ANmzJhB165dMRgMNG3alJdffhl7e3s6duxIr169cHJywsHBIdvlYc2aNWPIkCG8+OKLmM1mvLy8WLx4caYj5dRS32OH5NH/1KlTqVSpEv3792fUqFF07doVo9FIs2bN+O233ywTFidOnEjXrl3x8PCgbNmyODg44OHhwaeffsrYsWNRq9XUrl0bOzs7yxea7FSqVInx48czbNgwtFotNWvWtLxWq9WyfPly5s2bR9euXdFoNGg0Gl577TV69uyZ5jhZxf7+++8zc+ZMPvvsM1QqFW+++Sbly5fPdHt2tFotixYtYsaMGSxduhSj0cjbb79Nw4YN+fPPP3P0uYXIDZWS05tSQgiRQ6tXr+aJJ56gQYMGJCUlMXDgQEaOHEnDhg1ZtGgRI0eOxNHRkbNnzzJixAgOHDiQo1H79evX2bRpE6+//jpqtZrffvuN//3vf2lG7kI87mTELoTId1WqVGHatGmYzWYMBgMdOnTgmWeeAZKXB/bu3Rs7Ozvs7OwsI+CcKF26NGFhYZYRuaurKzNnzizIjyKEzZERuxBCCFGMFOjkueDgYAYNGpRu+549e+jVqxf9+vXjxx9/LMgQhBBCiMdKgV2K/9///sfmzZvTTYoxGAzMmjWL9evX4+joyIABA2jdurWlmpUQQggh8q7AEnvFihVZuHAhY8aMSbP98uXLVKxY0VL2M6WEY8eOHTM9ltlsJi4uDnt7+xzfixNCCCFsmaIoGAwGnJ2dc7SKJUWBJfb27dtbCoWkFhsbi6urq+Wxs7MzsbGxWR4rLi6Oixcv5nuMQgghRFFXrVq1NHkzO4U+K97FxYW4uDjL47i4uGwDTllvXK1aNWmQUIDOnDlD7dq1rR1GsSfnueDJOc6d09f3ciPifKbPG4yJGM1pKygqgEZlh84++XarwZSEwZRcCEpF2iurjtrk3/ElXX25FxOa4Xs0qtSZkq4VAdh5drnlWKlV8q5HzbJNATgVuotbUZfS7ePqUIIW1ZJLM9+IuMDp63syfL82NQfjYO9MoiGOPee+zXCfuhXa4KrzYt+FNSSZ4kkwaDCY1ZR2Tf7M8UnRVCnZkPq+7Th36xAGk566FVpneKycSExMpGPHjuj1en799VccHR25ePGiJQfmVKEndn9/f0JDQ4mMjMTJyYnjx49bCn1kJuXyu1arRafTFUaYjy05v4VDznPBk3Oc3rGQbVwNP51ue5w+EgBnnQcAScYEjKYk7O0csNfo0CtxmBRDmoStAGaVHZp/txnRY1QSsdNosbdLO7eqR6N3AIiIu8Ouvy9kGJudvcbyd2ZS6TGSPrGrNOb//l41pgz3MaFHbZf8RUNvjkVvTr4irEKFWp1cgthoNvBP+FEAEpJiiTM8QEFBhQqdfXKJX6PJwJGQDZjMBsyKCXdHH+4lRP77Wf/LSWp7FTqdjvqV8p7QTSYTGk3y5x87dizOzs74+PhYShbn9hZ0oSX2LVu2EB8fT79+/Rg3bhzDhg1DURR69eqVrmynEEIUBZklwsKUkmRTaDRadP8mzoefS6FSqS2jZJPZgN4QD4BC8urmlATtYO+CWq3BSeueZoRsMCUnlCf9OlKtdOMM4woKCkrXQOhB3G3+uXscHzc//LzrAHDu1mG2BX9FdOJ9zIrJsq+Xcxk61BkOwKW7x9lzblWG79PrydHo7JyITYxg86nPOXvrj3T7NKvSC1/v5Cs064/NYfXhD9PtU7VUI5pVTa7Pf/ifjfx14/d0+zhp3ejTKLkr382Ii/x5ZTNajQPBd8sy9Xe4ERVPeXcnrkzsme61eaEoCp9++inbt29n27Zt2Nvbp6n0mFcFmtjLly9vWc7WtWtXy/Y2bdrQpk2bgnxrIYTItYcT+cMj2dx6OPE6al1RqdSYFTOJSTEZvkZr54SdJvnSa0JSLGYluVnPw5e380KFCjuNFq2dI4qiUKlkXWIS7hOVEE5CUjR2aq0l0ScZEzh5bScnr+1Mc4x6FdpSo0wTAPae+46wVJfWE5Nik0e+KrUlsYdFX+VezHVcHDyxU/93SdnB3sXyZzu1FmeteyYxJ08aU6nUme6jSXXcUu6V8DD4YKfRYa/RWv7v7VLBsk+NMk9TsUSt5KsLKfupk/+fopxnNXo2fB+At6ZvsCT13vVy32MhMyqVipCQEG7dukVISAjVqlXLl+NK5TkhhM1ISbxlParQrGpvAM7e/IO/HxrFJSUlcfX4HiqWqMXV8NOYzEb0hriMDonO3hmNOvlXYUoiT0miKYmwRpmnqVM+uXLeoX82cjMi/eVkT6fStKs1BIArYacICt1uGfmmfDHo1uAtdHZOxOmj2Hb6qwzjeapyVyqWSO4hv/2vJcQkPsDPuy6NKnXKySnKkNFsAAXLF4ar4acJDT/L2ZsHALDXOODi4EW7J4bg7lQSk9nIphOfZXgsteq/jnoatT326v9uebi6eVG73DOUdP0viTau3JXm1fpYznFGKpWsS6WSdbP8DM46d7oHvJPdR7XcX8+Kp3NpPJ1LZ7sfwJgtQYRGxOHr6ZwvI3Wj0ci+ffto27YtADNnzsRsNuPh4fHIx04hiV0IUaSlJPMkY4IlUYbcO82tyH+A5IldKdtTKIBRn8jZm/uBtKPDrKhUauzU9mjtctaUJjvOOo8Mk7Kzzt1yyTcrKZeqc8toSuLCnT+5dPc48fookkyJNK7clSfKNgPg3K1D3I2+ipuDN+3rvIKT1i3NfVyN2o6eT76f7fu0rN4v230ctTk790XV+uDkKxL5NVIfOXIka9euZdOmTbRo0QI3N7d8OW5qktiFEHmWn/egH75snTJxK2UUbdmu0aVJvPZ2DtjbOaTZJykpybKC5lFHuw9rWqVHtvtU9qlPZZ/6+faeuXEr4hL7L64l0RCLRm2Pq4MXJbTlcUz15aaB77OYzSYW/PGA97fvztP7JCUlof014xnuxcmNqHh8PZ2Z27Vh9jvnwCuvvIJKpSrQVRuS2IV4zGWUnBOSYlAUc7p9U+7PAmlG0Knv/6aeuGU0GUgyxmf4vg5aF9QqDYqikJAUnW5iVwpnnQdqlQazYspxks5oYtfjQFEUTt/YS5IxkboVWlOrbAvLLO/USrtXBmBd8H/3jkXGHvW++sWLF5k8eTKff/45Pj4+BAQEZNoSOr9IYheiGMnLCPpRJoilnoyVwlnnTqe6rwFw48EFDl/emOFrO9R5BVeHEhhMen4+MR/I/9G1tSiKGbNittxXvnjnGOGx14nTR2IyGy37eTmXpXHlLgCE3Avmwp2M+7M/W3sYapWG6IT7HPrnpwz3CfDtgI9bRdrUHExE/B0+2Xef9au2Zxnno8zyfly/POXW3r17+e2339i4cSMjRowolPeUxC6EDcpuPXJmM6gd/72XalZMJCb9t77X7t/ZwE5aN+pVaEt5r+r5Emd5r+r08cr6XrK9Rpej+81FkaKYuRdzncj4u//+d49YfQRx+gjqVWhrKVZy7f4ZbmQw4S61OH0kd6KuZPJGgAqM5qRM99H/e2VEa+dAKTc/1gefyHY0nt+zvEWyq1evUrFiRdRqNa+88go1a9akZcuWhfb+ktiFKARh0dfYd+F7IPne5JVj/y0hal3zBbxdygPw0/F5adb6WgqFaHSW+8h6QzxGc/K96NQJXK22w1nngbujD1EJYRnG8VzAu9hrdMQk3mf7X/9L93x8UjS3oy7nW2K3JYqikGiIxcHeBZVKRZw+iuBru0k0pp9N36Bi4L+zqlXpqqTp7JzxcCqNzu6/hNrQrxNPVuqEs84jzZKv1GqVa0mtci0yiS7579nTqTQvNsus/3zyPmO2BLE+ODTf11yLnNmxYwdDhw5l0qRJvPbaa6jV6kJN6iCJXYh8lzKajtNH4uHkw3MB72W4X0rS3nV2heWSbXxSNCiKZZ+U+84Py+gSeAmXcrSpmb5NckZcHUrY7Cg5P0Ql3OP6/XPE6iOITYwgVv+A2MQIjGYD/Z+ahIO9MwAX7x7N8PXlPavj6VwalUpFA99A7NVa3J1K4eHkg/ahiXwAns7ZF+FKnpWe9Vr1nOyTOqnLaLzwBQQE4OvrS4UKFbLfuYBIYhciB3adXUFE/J1028t5VrfMkv7rxj7O3z6c5p51Oc8aAPi4VaRPo3EcC9nGxVvHgf+qe6Ve3+ukTb/0pbjcd84PYdGh6I0JmM1GTGYjJiX5/64OXpTzTC7ucf3BOW4bgjly+QZmswm9MY7YxEhUKjVd6r8BJJc2PX51m+W49hoH3By9cXHwstwDd9K68lzAu8kj+IeSaeqCKClLyIoSGakXHrPZzJIlS2jUqBENGzakZMmSHDx4MFfd2PKbJHYhHpL6/nVD3w65XraU2dplSC4MYlAS0KLNcr+iJl4fTUzifSC5oIuHkw8Al8NOZtjUQ6txJMCvPZBcZvRiJpPC6lZog5PWDZPZyLGQX9I9n5AUR40yTSjj4Q/AwUvriUq4l24/P++6lsR+J+oK4caLhN/+73mN2h53R28URUGlUuHj5kurGs/j6uCFi4NnmsvmKVQqNR5OUu5aZO3UqVNMmDCBpk2b8ssvyT/D1kzqIIldCCBtMs9olnhKRbGs1Cn/jKU6WWbHjtdHY69ytKnL4CH3gjlwcZ2ltGkl73o8U2MAAPdjb3D+9pF0r3HSulkSe2zigwz3geTSnk5aNxRFyXQfL+fSlsReu1xLEo3xaFQa1Go7NP/+56LzSnPMxHAttWrWQq22Q2vniOO/981Tx5dS8rQwpdz/LkiyfK3gmc1m9Ho9jo6OBAQE8MUXXxAYGGjtsCwksYvHVkrC7VT3NepWaIWimAi9fzZfRtKZfVFw0rnhaPLJj/ALXGT8Xc7ePMA/d4Ow02ipWfppNGp7PJ3+K8VZq1xLqpZqxMP3fdWq/0Yspd396d7g3Qzfw8UhOSFr1JoM97HXaNN8wapaulG2cbs6eOGk9sLLpWy2+xa21Pe/C4rcWy9Y4eHhDB06FF9fX7744gsABg4caOWo0pLELh4b1+6f5c8rWyyT1lImpv0S/GWaRATJl8wfpaJa6mT+8BeFoKCgPB83rxRFwWg2WJpcRMTd4UbEeeL10SQYYohPirEUkulY91V0dk4kGRO5dPc4zjp3WtV43tIrOzVnnTvOuowbc6TQ2jlkOKEsNZVKnaMJZkVZTkbjMlPd9rm7uxMbG0tUVBQGgyHXvdILgyR2YZNSRsRVfBrSwDfw321buRr+V5r94vSRlHavnKbmdkpST5lZ/nBSzw8Fef/cbDaRZEquj24wJpJkSsTB3sVy3/tq+F/cibpCQlIM8UnR//4/BnenknRv8DYA4bE3CLr6X/ESFep/k68K5d9Z+SVdK9Cp7muUdK2AqgDOUXGTk9G4jKZt07Vr17hw4QKBgYHY29vz888/4+bmlus+6YVFEruwuqyqpTWr2ouyHlUB+OXUFyQYkouqPFw/PDPOOg9K/LtGvGKJWlQsUYt1x2YDWP0+t8GUhJ3aHpVKRZIxkesPzqE3xqM3xKM3xpFoiEdvjKdehTaWEqA/HZ9nmcSWWrXSjWlaJXkUeCvyEhfvJC/TUqHGUeuCl3PpNBPBynpUpe0TQ3DSuuKodcPB3jndFxyVSo2PW/FOQvl5z1tG48WTXq+nY8eOxMbGcvz4cUqWLIm7e9ZXqaxNEruwipRk3rPh+1Qt9STxSVGERef8F2zKiDhltA7QqFJnGlXqXBDh5puQe8H8k7ibi0e2oTfGMeCpyejsndAb4zlwcW2Gr6ni81/ZzhIu5XDReVgapGg1yQ1QSv775QWSJ5hVL90EJ60bDvZOGY62c3IJ/XGQn/e8ZTRevKSsoNDpdHz44YcAeHt7WzmqnJHELgrFxTtHCb6+x/I49Yjbw8mHZ6oPyPYYXeq/WRChFQqjKYkbERf488oWEpVY3Oy8KeFS1lJlzlHrytP+PdDZO6Gzc0Jn74SDnTM6e6c069xb1ch+ko6bo2388ikqZJQtUlMUhZUrV/Lzzz+zfv167Ozs6Ns3+x7vRYkkdlFgjoVsI04fmWEyShlxp05axYnZbCLRGI/Tv13O/ri0znL/38euFp2eTFshzk5tT/UyTxV6nDlRGEu08ltOW4rK0jDxMJVKxdGjRzl16hQXLlygVq1a1g4p14rnb1VhFfsvrOVudIjl8X+j8oFUK92YaqUbWyWughZ0dTt3oq4kT2Yz6TGakkgyJVLG3Z9naw8Dki+nuzl44+ddhyvnb1k54twpjCVa1iKXzwUkj9KPHz9Oo0bJyylnzpzJxIkTKVu26C2ZzAlJ7OKRHAvZhovOk5pln073XMqovCh4uEiMky596dacUBQzx0O2obN3tnTuik4IJzz2RnKjFo0OR60r7pqSeDmXsbyuvFcNynsll5cNUd3O8NhF0ZgtQYRGxOHr6WxTl6ulpajIjbFjx7J06VK2bNlCs2bN8PDwwMPDw9ph5ZkkdpGplGTY7okh/3aywjKjPEWcPhIHexdqln2altX7WSPMHLkaftqS0J10bvh51yXJmJhh/XdI7qKVsvY6LDoUBQWz2cTtqMucvfUHJZzLUad8K1QqFc/UGIAKdZFd+vIoUi7By6hWFGd9+vQhNDQUX9/i8XMuiV2kk7o7WXaK0qg8O046tzRL3O5GXeXXv77OcN8OdYZblpj9dmYpRrMhzfNPV3nOksjVKk0BRVxwcnrf/EZUPL6ezsztKqNfUXzcunWLjz76iBkzZuDt7U2jRo1YuzbjVSm2SBK7SCdldJuStFNG62D9td+5kdHl94i4O4TFhFK99FM46dyoW751hq9NXca0VrmWKIoZVODm4I2Pm6/NzzzP6X1zuQctiqPNmzezbt06qlevznvvZdxW2ZZJYhccC9nGrchLlKc5kJy8oOgl8awK2WTk4Rrtpd392XHmfyQa4vFx9cXTubSlUUlWUq+Vf1RjtgTx/bFLOZqxXZCkmIp43Ny7d48SJUqgVqt55ZVXKFu2LF27drV2WAVC6kQ+5o6FbOPszf1ExP03oatexTZF8vJ6ypWEnHLWeVCrXEv6NBpHYK2h3Iy4QKIhjqerPJfmKkRhWh8cSli8IfsdC5iMxMXj5MCBAzRu3JhvvvkGAI1GQ7du3YrlvBiQEftjK/jabi7ePWYZ1dYq1xIeJD9X2r2y5f5yUfPwffKcMJmN/HbmGxINsTTxf47qpfNnvXhe1nffiIrHx8leRspCFKJq1arh5uaGTqezdiiFQhL7YyLlMnZFryd4yr+bZXvqZiVBDwq/69ijSjImkmRMwGQ2YlKMmMxGzGYjoKKUux+QXPUuPimKmmWaUqNMk3x777ys7y7v7kSL0ll3OhNCPBpFUfjpp5+oUqUK9evXp1SpUhw7dgytVmvt0AqFJPbHxMOXsetVbEu9im2tGFHOmcxGQsPPkGiIQ1HMnAjdQYBv8r3xoKu/cuHOn+le46R1o2/jCUByuVZQUSODtfZ5kTJSz+t9amu0bRXicXLmzBmGDx/Ok08+yY4dO1CpVI9NUgdJ7MVeykg9ZVZ46tF6UZR6gpxZMWE0JWEwJcG/vdMB9IYEy59LulbEaEpCo7ZHrdagUdujUduhs3NMtU8FOtR5BXfHkvkSY+qkLvephSgaFEXBaDRib29PnTp1mDVrFu3bty+299GzIondhmU1S7xRpc74edchIu42cfrIIr3ePPXnSD2TPcmYiMlsAFTYa3TYqbVUKlmfxpX/6+BWpVRDqpTKeo21s84jzfK1R619LjPKhShaoqKiGDlyJB4eHnz++ecAjBgxwspRWY8kdhtWu1wLTOYkrj84n+k+KbXKrSUnS9TSLEvTulOpZD0aVerE/dibRMTdwc+7LnYa+3yL6VFrn8tIXYiixdHRkatXr+Lq6kpiYiIODo/3PBZJ7DbMUetKE//naOJv7UjSymwEnhlnnQcVvWrh4uDBuVuH+ePKNV76cS1xhpRknr9rvmXELYTtCw8P58KFCzRr1gytVsv69evx9vZGrZZV3JLYbVhCUgyQMjms6Eh9Tz/1rPvMxCQ+YFvwVyQYYrDTaDl924EL4UmUd8+/UXpqMuIWwrYZDAaeffZZIiIiOHLkCKVKlcLHx8faYRUZkthtVEphGWedR5GoEJdR+dacxHUi9DdCw8+QYIihRpkmbDrnw7cnL9tcNzEhROGxt7fnvffeIyoqCm9v2y7vXBAksduYhxu0FJUJcRl1T3tYoiGOy2EnKOnqi49bRQAexN4iKuEens5laOjbkYE/bAWkm5gQIq1ffvmFH374gZUrV2JnZ8cLL7xg7ZCKLEnsRdyxkG1ExN22TIK7FXExzSz3rC5xF7asRumKonDq2k7O3z6Cm6M3PQJGoVKpaFGtL3YaLeO3BvPu1q3STUwIkaHNmzezZ88egoODadhQfj9kRRJ7EZZyuT21ehXbci/mepFI6Bldfn9YSnGZszcPcD/uJgBPVe5G8rp0FTr75JnpsjZcCPGw8+fPU6NGDQBmz57NqFGjqF69upWjKvoksRdRqZN6rXItLdv9vOvg513HWmFlOuM9s8vvv59fzfUH51ChwrdEbepWaEMJl7Lp1pLLTHUhRGrTp09n/vz5bN26lSZNmuDl5YWXl5e1w7IJktiLqNRJvSiMzlNkNePdrJi5E3WF8JjrVCpZD2edB6Xd/XF1KEHNsk/j6lDCcpyH15LLSF0IkVq7du3Yt28fnp6e1g7F5khiL6JqlmmKWm1XJJJ6djPeYxLvc+7WYULuBZNgSF6CFx57k1Y1BlKrXHPLfqlH6TJCF0KkFhUVxaxZsxgzZgxeXl40adKE33777bEsCfuoJLFbWUaV2aqValSkarpnNuNdURT2X1zD1XunUVDQ2TlRrVRj/ErWybDta+pRuozQhRCprVmzhiVLluDq6soHH3wAIEk9jySxW1nKJfesKrMVBSmjdEUxozfGA8n/6NSo8XIuS61yLfDzroNarcnw9WO2BBEaESfr04UQFtHR0bi6uqJSqXj55ZdxdHRkwIAB1g7L5klitzJP5zKU9ahaJC65Z0VRFP65G8SZm/vR2TnRsW5yg4Wnq/RAo7bP9pt1yiV4GaULISC5ffGLL77IqFGjGDp0KBqNhsGDB1s7rGJBEruV3Im6AkD3Bm9bOZKMpdwiUBSF+KRoVMAfl9ahUqmp7F0Pk9mIRm2HnSZ9j+OMuqfJ+nQhRGply5bFYDAQExNj7VCKHUnsVnLg4o8ARaIcbEaSq9tFoVHbAQqo1NQs05RaZVvg4uCR5Wsz6p4m99SFEHv27MHHx4fatWtTpkwZgoKCcHFxsXZYxY4kdpGOwagHwEnrikZtj87OkTZPvIhTqmYzWfU0lxnvQoiHnT9/nt69e9OgQQN27dqFSqWSpF5AJLFbwbGQbZaysEVFdEI4YYZzbD75h2WbSqWmU93XUKvUlgpxKbLqaS6jcyFECrPZjFqtpkaNGkycOJF27drJbPcCJondClKWt1m7gcvYLX9y88ExnvB5QFnX5JnupiQVlyPc8HGOB1TUmvtbhq+VUbkQIivx8fFMnjwZgI8//hiA9957z5ohPTYksReSYyHbLDPfvV0qAFh9JryTaift/GNRFDCYVCSZ1BjNanycE3DTGYjWp58Yl0JG5UKIrGg0Gg4fPoxKpSI+Ph4np/RX90TBkMReCFLqvttrtNSv2I76FdvyT9gJq8SiKArjtx7hx1O3eb1xImYFXHRuqFRqkpKS0GpTkrkTdcrX5Z12RXsZnhCi6IiNjeXChQs0bNgQnU7HmjVr8PHxQafTWTu0x4ok9gKWupmLwZQEgKdzaauN1nf9vYJKLpd4vbEaL0cjRrMTfRtPAJLXlUo7RCFEXphMJjp06MCtW7c4fPgwpUqVokKFCtYO67FUYIndbDYzZcoULly4gFarZfr06fj6/nfpdvPmzSxfvhy1Wk2vXr0YOHBgQYViNQ93aLNWMj/0z0auhJ3AZDb+W/oVkkxqXB08rH6fXwhRPGg0Gl566SWuX7+Ou7u7tcN5rBVYYt+1axdJSUmsXbuWU6dOMXv2bL766ivL83PnzuWXX37BycmJzp0707lz52L3w+Ci88TB3gV/nwCr3k//+9YJ7NQGzAoYzSou3Xfip7+f4MrEXlaLSQhh+w4dOsS3337Ll19+aUnswvoKLLEHBQXRokULAOrXr8+ZM2fSPF+9enViYmKws7NDUZRitfzhWMg2EpJiaFm9HzXLPm3tcDCZTdipYWlQLe7EOgNS2lUI8eiWLVvGpk2bGDx4ME2bNrV2OOJfBZbYY2Nj0xQf0Gg0GI1G7OyS37Jq1ar06tULR0dHAgMDcXNzK6hQCl1y1bZIWlbvZ+1QgOTL7hq1wp43euJg72ztcIQQNuzGjRuUL18eSL7y+uqrr9KoUSMrRyVSK7DE7uLiQlxcnOWx2Wy2JPXz58/z+++/s3v3bpycnBg9ejS//vorHTt2zPKYD4/6i5LbhmCiTDcAMCgJ2KucCAoKsnJUyYxmFTF6DWdPn89236ISc3En57ngyTnOf+vWrWPZsmV8+umnPPHEE1y9ehW1Wi3nuogpsMQeEBDA3r176dSpE6dOnaJatWqW51xdXXFwcECn06HRaPDy8iI6OjrbY9auXbvILpv4/fwFwsPjcdZ5oEWLn3ddGlYq3BnmD5d5ddUmobUz0a822KnV2c54l1nxhUPOc8GTc1ww9Ho9e/fu5YknngCQc1zA9Hp9nga0BZbYAwMDOXjwIP3790dRFGbOnMmWLVuIj4+nX79+9OvXj4EDB2Jvb0/FihXp0aNHQYVSKFrVGAhYd2b/w2Ven61yjRreEcQbNOjs7K0amxDC9iQkJLBgwQJeffVVPDw8aNq0KQcPHsTOzk5G6UVYgSV2tVrN1KlT02zz9/e3/HnAgAEMGDCgoN6+UF28cxSAaqUbF8r7ZdaAJXWZ11uR//DbmaOUdK1InD6qWE1OFEIUjtWrVzN37lzi4uKYNm0agOWWqii65G/oEaWsVXfWeRRaYs+sAUtKmddbEZf4/fxqQMVT/t3Ye+67QolLCGH79Ho9Wq0WlUrFkCFDiI+PZ9iwYdYOS+SCJPZHVBgNXR4eoWfWgEVRFM7fPsTOsz+hUqloUa0v3i7lCywuIUTx8vfff/PSSy/x+uuvM3jwYOzs7HjrrbesHZbIJUns+cBZ51GgBWgeHqFn1oBFwUzo/bPo7J0o416FE6E7OBG6g3h9NE664rOcUAhRMNzd3bl79y5Xr161dijiEUhifwQms7HA+6qP2RJEaEQcvp7OWbZINStm1CoNrWo8j9Fk4Ne/vrYkdCedm5SOFUJk6OTJkzg6OlKjRg3KlSvH8ePHKVGihLXDEo9AEvsjctYVbL31lEvwGY3QjaYkYhIf8PetPzCYkmhVY2ByAZp/J8A76dzo02hcgcUmhLBtV65c4dlnn6VOnTrs2rULtVotSb0YkMSeB8dCthGfFEW9Cm3zPXFmdD/d19OZuV3/Wy8afG03f986hN74XwEgD6dSGEx67DVFc52/EKLoqVy5Mm+//TYtWrRArVZbOxyRTySx50FKyVgnrXu+31vPyf30S3eD0BvjKOtRFWedOz5ufvj7NECt0uRrLEKI4iUpKYlPPvmEmJgYZs6cCcDEiROtHJXIb5LY8yg/J8ylHqVnNOPdaDJwP/YmJVzKAVC7fEuiE+7RuHLXNMc5FrLNMktfJswJIR5mNpvZvHkz8fHxjB8/HldXV2uHJAqAJPYiIPUo/eERutls4sDFH7n+4Bzt67xMKTc/qpd+KsOCM1fDT8uEOSFEGgaDgUuXLvHEE0/g4ODAqlWr8PHxkaRejEliz6VbkZfydTSc1ax3o9nAvvPfc/3BOUq5VbKsSc+qipxMmBNCpDCbzXTt2pXLly9z6NAhSpYsSZUqVawdlihgktjzID9HwxnNeo9NjODag7+5EnaK8NjrlPWoSuuag9Co5a9LCJFzarWa5557jr///rvINtAS+U8yRQ79cuoLALrUfzPfRsSpR+upZ72Hx17n6JUtAPh516FFtX5pknrqe+mpyX11IcTff//N8uXLmTNnDmq1mhEjRkiviMeMJPYcSjDE5vsxk0frCq83iWLHX0vR2TvRqsZASrv707xqH3zc/HB18Er3jzL1vfTU5L66EOKTTz5h48aNdOjQgbZt20pSfwxJYs+BYyHbCqzCXMNyGrwd/uZ2VPJadAAHe2eqlErb5zijGe9yL10IAfDgwQO8vLwAmDVrFv369aNt27ZWjkpYi1QkyIGCbPTirtMD0KBiIM8FvJtlDPH6aEBG5kKI/6xcuZK6dety4sQJAHx8fHj22WetHJWwJhmx50DKaD0/i9Gk3F9vUCYJIMOrATJKF0Jkp3Llyri5uRETE2PtUEQRISP2HKhXoW2+j5BTZsM3qZhcYc5J555uHxmlCyEeZjKZWLx4MVFRUQC0aNGCoKAgnnnmGStHJooKGbHnQAPfwHw9XurZ8P0DanAi9AEHLv6IWpX2e5aM0oUQD1u1ahXjx48nJCSE2bNnA+Do6GjlqERRIok9G8dCtgLQqFLnfDtm6rXrFUo8wZErm0lMipVZ7kKIDJlMJtRqNSqViueff54bN27w+uuvWzssUURJYs/CsZBtnL154N/76/mX2IF0a9dlZC6EyMjVq1cZMWIEL774IgMHDsTe3l4at4gsyT32LBTkbPgUv59fTZIxscCOL4SwbRqNhnPnznH06FFrhyJshIzYs5GX2fAP91R/WErDlyRjIlfD/5JSsUKINK5cuYLZbKZKlSpUqFCBgwcPUqFCBWuHJWyEjNgLQEq3tsykdHD7+9YfAKjkr0EI8a/r16/TokULRowYgclkApCkLnJFhopZcHXwyvVrsurWltqtyH/47cw6XHRemBTjo4QphChGKlSowODBg2ncuDEajcba4QgbJIk9Cx3qDM/1azLq1vYwRVE4dW0XAK1qDGTv+e/yFqAQwuaZzWaWLVvG3bt3LZPiZs2aZeWohC2Ta8CZuHb/LNfun83Tax+e8f6wB3G3CYu+SnnPGni7ls9riEKIYiApKYmlS5eycuVKIiMjrR2OKAZkxJ6JP/9tm1qxRK18P7aXc2l6PTkaFXKZTYjHkdls5tq1a/j5+eHg4MDy5cspUaIEHh4e1g5NFAMyYrcClUqNq0MJXBw8rB2KEKKQKYrCwIEDad++Pffv3wfgiSeeoFSpUlaOTBQXMmK3guiE+zhpXbHTaK0dihCikKlUKpo3b45arbbMehciP8mI3Qp2nPkfG4I+tnYYQohCcv36daZOnYrZbAbg9ddfZ/Xq1fj4+Fg5MlEcSWLPwLGQbcTpI3P9upSlblkxmPTE6SNxd5R/0EI8LqZOncpnn33G9u3bASx134UoCHIpPgNPlG2GwZiIvZ1Drl6Xk6VuoeFnAPBwksQuRHEWFxeHs7MzANOmTaNNmzZ07NjRylGJx4Ek9gw469xpWjXz4jKQcdnYG1HxWS51++duEAcvrcdeo6NKqSfzLV4hRNGyceNGRo8ezU8//US9evUoXbo0AwYMsHZY4jEhl+IzoDfGozdmXhIWMi4bm1IqNiMRcXf449I67O0caF/7ZUq4lM23eIUQRYuHhwdGo5EbN25YOxTxGJIR+0OSW7XuJzJRy+dH6me6X0ojl6zKxt6JuoLJbKSUeyU8nHyo4PUEDXwD8XIuw7GQbZbucfH66HS92IUQtkNRFNavX0/79u1xc3OjdevWBAcH4+7ubu3QxGNIRuyppCR1gD+vZ51osxqdpzhzYx87z36D0aRHpVLT9onBeDmXAZJbwsbro4HkXuwF2RpWCFGw1q5dy4gRI5g+fbplmyR1YS0yYk8lZQR96Fpp/rxZMcvReHYURSE89gbOOg8c7F0y3MdJ50afRuPy/B5CCOtRFAVIXpfes2dPgoKCGDlypJWjEkJG7Glo1PY46zzYdaXiIx8rTh9FoiEObxdptyhEcXP37l2ef/551q1bB4BWq2XevHnSXlUUCTJiT6V+hbbcj7sFJD7ScW5GXGTn2eUA0uRFiGIoMTGRAwcOYG9vT9++fa0djhBpPPaJPeW+ennPGrSrNYTK1Ac2PNIx70RdQYUKJ507Fbxq5kucQgjrunv3LomJifj6+uLr68tvv/1G9erVrR2WEOk89ok95b66ez4WjAnwbU/VUk/i5uidb8cUQljP7du3ad68Of7+/vz6669oNBpq1pQv7aJoeuwTO4CzzoNGlTrl2/FUKpUkdSGKkTJlytClSxdq164tpWBFkfdYJ/aUmvDOOo98O2bIvdOERV+ldvlncNbJchchbNXGjRu5fPky77//PgALFiywckRC5EyOZsXfuHGD33//HZPJxPXr1ws6pkJjMOkB8nUN+a3Ii5y7fQijKSnfjimEKFx6vZ4ZM2bw+eefEx4ebu1whMiVbEfs27Zt46uvviIhIYG1a9fSv39/xowZQ/fu3QsjvgLVtEoPmlbpkWZbSoc2X0/nPB0z0ZDc3c3BPu3rU1eaA6k2J0RRdPfuXUqVKoVOp2Pp0qW4urri7S231YRtyXbE/r///Y8ffvgBFxcXSpQowcaNG1myZElhxFag/rqxj79u7Eu3PScd2rKSaIhDhRrtQ53hUleaA6k2J0RRoigKr7/+Oq1atSIiIgKA+vXr4+/vb+XIhMi9bEfsarUaF5f/Kqf5+PigVtt+XZvztw8DUKf8M+mey6pDW3YSDXHo7J1QqdQZ1oOXSnNCFD0qlYrq1atz5coVYmNj8fT0tHZIQuRZthm6atWqfPfddxiNRs6dO8ekSZOoUaNGYcRmk/SGOMtleKkHL0TR9eDBAz777DNLadg333yTrVu3SvU4YfOyHbFPnjyZr776Cp1Ox4QJE2jSpAnjxsmoMyNmxYyDvQuuDl6WbTJKF6Jo+uCDD1i7di1+fn4899xzaDQaa4ckRL7INrHv2LGDUaNGMWrUKMu21atX8/zzzxdoYLZIrVLT88n3rR2GECITBoMBe3t7IHnQUrt2bbp27WrlqITIX5km9hUrVhAbG8uaNWu4efOmZbvJZGLLli02n9gzWr/+qDPihRBF1+7du3n33XdZvXo1derUoUyZMrzxxhvWDkuIfJfpPXY/P78Mt2u1WmbPnl1Q8RQaDyefdPe8H3VGfJw+kpB7wcQkPnjk+IQQ+ctsNnPv3j3Onj1r7VCEKFCZjthbtWpFq1at6NixY7olH4mJ2Xc/M5vNTJkyhQsXLqDVapk+fTq+vv8lzNOnTzN79mwURaFkyZLMmzcPnU73CB8lZ8KirwHwXMB7lm1jtgSxPjiUG1HxjzQj/m50KPsv/EAT/+7UKPN0vsQrhMi73bt307hxY1xdXQkMDOTkyZOULl3a2mEJUaCynRUfGhpK165dadeuHW3btqV169a0atUq2wPv2rWLpKQk1q5dy6hRo9KM8hVFYdKkScyaNYsffviBFi1apLncX5D2XfiefRe+T7MtJamXd3fK82gdQG+IBdIXpxFCFL7NmzfTp08fpk2bZtkmSV08DrKdPDdr1iymTZvG8uXLefXVV9m1axcJCQnZHjgoKIgWLVoAyYUezpw5Y3kuJCQEDw8PVq5cycWLF3nmmWeoXLnyI3yM3EsZpQOWpH5lYs88HUtviOevm/u4cPsIAI72UlFOCGtr3749ffr0YciQIdYORYhClW1id3V1pUmTJpw4cYKYmBhGjx5Np07Zd0KLjY1NU9hGo9FgNBqxs7MjIiKCkydPMmnSJHx9fXn11VepXbs2Tz+d9eXr1F8O8iopKbmG+/fHLhEWb8DHyR4fRztalHYgKCgo18czKQau6PeSqERhhwPO6pLs/GslKpUKg5KAvcoxT8e1FluK1ZbJec5/cXFxfP311wQEBNC6dWvOnDnD8OHDSUhIkPNdQOS8Fk3ZJnYHBwdCQkLw9/fn6NGjNGnSBIPBkO2BXVxciIuLszw2m83Y2SW/nYeHB76+vlSpUgWAFi1acObMmWwTe+3atR/5PvyVYzuJiE/idpwBX0/nPI/SU5gVM6qQcAzmJBpX7srGoE8w6fU4ad3QosXPuy4NK+Xtnn1hCwoKomFD24jVlsl5LhghISHs37+fhIQEWrVqxZNPPmntkIo1+TkueHq9Pk8D2mzvsb/zzjt89tlntG7dmsOHD9OsWTPatWuX7YEDAgLYv38/AKdOnaJatWqW5ypUqEBcXByhocmXwo8fP07VqlVzHXxexRuMQN5nv6emVqlpVLkLT/v3wE6dvD42pShNn0bj8rXPuxAirejoaMv8nEqVKrFp0yZ+/PFH6ZkuHmvZjtgbN25M48aNAfjpp5+IiooiJiYm2wMHBgZy8OBB+vfvj6IozJw5ky1bthAfH0+/fv2YMWMGo0aNQlEUGjRokKMJefnpUWa/A5y/fQS9MZ665VujUqnkF4kQhSw8PJzWrVtTvnx5fvnlFzQajYwghSCLxB4SEsLChQtxd3fn/fffx9nZmdjYWBYvXszq1asJDg7O8sBqtZqpU6em2ZZ62dzTTz/N+vXrHzH83Gtd8wW6L92bp9fei7lOaPgZ4pKS16sDVPFpiLPOPT9DFELkgLe3N02aNKFy5cqYzWYpCSvEvzJN7OPHj6du3brcu3ePRYsW0bx5c8aMGUO5cuVYvnx5YcaYr7xdynM7Nm/L0Q5eWkdkfBgAKlSUdq+Mk1ZmwAtRWPbv389ff/1lqRi3ZMkSuVomxEMyTewRERFMmDCBpKQkunTpwq+//sq4cePo3LlzYcaXr346Pu/fP+W+x7LJbCQqPpwSLuVoU3MQjvauqNUyQhCisBgMBt59911u3rxJr169KF26tCR1ITKQaWJ3dHQEkkvI6vV6VqxYQaVKlQotsIJgVkx5fq1apaF3ozEkGfXpaswLIQpOdHQ0bm5u2Nvb8/XXX2NnZyeFZoTIQqaz4lN/E/b09LT5pA4QEZ/Ezah4bkTF5/q1KpUKZ50Hns6lCiAyIURGJk6cyNNPP01kZCQAjRo1okGDBtYNSogiLtMRe2RkJD///DOKohAVFcXPP/+c5vnnnnuugEPLf/EGI0azkqfSsXH6KLQaB+ztCr6evRAimaenJ66urty7dw8PDw9rhyOETcg0sTdp0oQ///wz3Z9T2GJiB7BTq/JUlObI5U1cf/A3/Z+aiIO9S/YvEELkWlxcHGvXrmXo0KGoVCrefvtt3njjDRwcHKwdmhA2I9PEPmvWrMKMo8iLjL+Lzs4JnZ00eBGioEyYMIFVq1bh7u5Or169sLOzs1SsFELkzGPzL2bMliCCr3vh5aTN9WuNJgMxiQ8o5eYns3CFyGdmsxm1Onm6z+jRo/H29s5RPwohRMayLSlbXKwPDmXX5RJUKPFUrl8blRAGKHg4ycQ5IfLTsWPHaNGiBX///TcA5cuXZ9KkSZZVOUKI3HtsEjvkvYxsSlEamREvRP568OAB58+f5+DBg9YORYhiI9vEfvPmTYYOHcqzzz5LWFgYgwcP5saNG4URW77rWv0KBy/lvoxtRPwdABmxC5EPjh8/bun82L59e44ePcorr7xi5aiEKD6yTeyTJ09m2LBhODs7U7JkSbp06cLYsWMLI7Z8M2ZLEI3LXaZBmXBuRf6T69dXK9WYltX64+VctgCiE+LxsXPnTjp06MC0adMs21L3kBBCPLpsE3tERATNmzdHURRUKhV9+/YlNja2MGLLN5ExB+hQ9T4Aft51c/16N8cSVPapj9ZOltwI8ShatGhBYGAg3bt3t3YoQhRb2c6Kd3Bw4M6dO5bZ4MePH0erzf3Mcmt6wucBALXKtcx1f3SzYgaS+64LIXInMTGROXPmUL9+fbp3746DgwM//PCDtcMSoljLNrGPGzeOESNGcO3aNbp3705UVBQLFiwojNjyze0YZ27HODOkee6X0DyIvcmvfy2mXoV21K3QKv+DE6IYu3XrFosXL6ZWrVp069ZNlosKUQiyTezly5dn/fr1XL16FZPJROXKlW1uxL7ubFUA5vTI/Wsj4u9iMhvR2cnyGyFyQq/XExUVhY+PD5UrV2bNmjUEBARIUheikGR7fblHjx6MHDmSS5cuUalSJZtL6o8qZambzIgXInuRkZG0adOGl156CbM5+TZWy5YtcXGRMsxCFJZsR+x79+7lyJEj/PLLL3z88cc89dRTdOvWjaeffrow4ssX9Urfy/NrI+PvAuDh7JPpPsdCtnE1/DQA8fponHRueX4/IWyZu7s7/v7+lCxZkqSkJKnxLoQVZJvY1Wo1TZs2pWnTpvz555/MmTOHN998k6CgoMKIL18843czT68zmY3cj72Jk9YNnZ1TpvtdDT9tSehOOrc8zbwXwladPn2aoKAgS+OWb775Ruq7C2FF2f7r+/vvv9myZQs7d+6kUqVKDB06lMDAwMKILV+M2RKEj1bBTpP7+3sh94JJNMTyRNlm2e7rpHOjT6NxeQlRCJtlMpkYNmwYoaGhBAYGUr58eUnqQlhZtv8CJ06cSPfu3VmzZg3e3t6FEVO+Wh8cyuuNwMk+979sfEvURm9MwLdE7QKITAjblZiYiIODAxqNhgULFpCYmEj58uWtHZYQgiwS+7179yhZsiQLFy5EpVKRlJTErVu3LM+XLWsbVdjaVb6Gt7MBZ13u263a2+moVa55AUQlhO2aP38+q1at4vfff8fNzY2mTZtaOyQhRCqZJvaJEyeyePFiBg0ahEqlQlEUy3MqlYrdu3cXSoD5Jbf3vUPDz1DOszp2GvsCikgI25SUlERSUhKhoaHUqVPH2uEIIR6SaWJfvHgxABs2bMDDwyPNc7bQBOZYyDZqlHmKfaHlAHJVnCYs+hp7z39HJe96PFNjQEGFKIRNMBgMbN68mZ49e6JSqXj33Xd59dVXcXd3t3ZoQogMZJrYb9++jaIoDB8+nP/973+WEbvJZOKVV15h+/bthRZkXlwNP02iIRaDyY5dVyrm6rVnb+4HoFrpxpnuI0vcxONi0qRJLFmyBLPZTJ8+fdBqtY9dPQshbEmmif3zzz/nzz//JCwsjOeff/6/F9jZ0apVq8KI7ZGdu32e0IhK+Hrm7v763eirOOs8KO1eOc321Mk8Th8JgLPOQ5a4iWLtjTfeQK/X29RqGCEeZ5km9lmzZgGwZMkShg8fXmgB5ad4gxGA3vV8c/U6RTFjr9GlK4GZer26s84DP++6uW4qI0RRd/78eUaNGsUnn3xCjRo1qFChAvPnz7d2WEKIHMq2pGz79u3ZvHkziqIwefJkevXqxZkzZwojtnzh6+nM3K4Nc/Uas2JGRcbr3lPWq/dpNE6SuiiWrly5wuHDh9m6dau1QxFC5EG2iX3ChAmYzWZ2795NSEgI48ePZ/r06YURm9UomFFJm1bxGLl48SLx8fEAdOrUid27dzNq1CgrRyWEyItss5der+e5555j7969dO3alSeffJKkpKTCiO2RRMQnYTQp2e+Ygd5PjuHZ2sPyOSIhiqY//viDZ555hhkzZli2NWjQwIoRCSEeRbbl2DQaDTt27OD333/n7bffZteuXajVRX80u/5see7GJub6/jqAg710ohKPj4YNGxIQEGBTjZ2EEJnLNrFPnTqVFStW8OGHH+Lj48PWrVtt4lL8Pw88AHJ9fz085jp2Gq20aRXFlslk4uuvv6ZSpUp06tQJR0dHfvnlF+mXLkQxke3Qu3r16gwZMoSwsDBWrFjB8OHDqVGjRmHEZhXHQrbx84nPiE+KsXYoQhSIa9euMX36dGbOnGnpmS5JXYjiI9vE/vPPP/PGG29w48YNbt26xZtvvsn69esLI7ZHMqTB3wxp8HeuXhOvj+Zu9FVKufnipHUtoMiEKHxms5nIyEgAKlWqxIoVK/j5559t4raaECJ3sr0Uv3z5ctatW4enpycAr776KoMHD6Z3794FHtyjcNPlfoJfcvEZBT/vevkfkBBWEh0dTf/+/dFqtWzYsAG1Wk379u2tHZYQooBk+3XdbDZbkjqAl5dXsbxsZ1bMXL53ChUq/LylTasoPlxdXXF3d8fd3Z2EhARrhyOEKGDZjtirV6/OjBkzLCP09evXF8t77Acvred+7A0qeNXEUS7DCxt39epV/vzzT/r164dKpeKbb77BwcGhWH4pF0Kkle2Iffr06Wi1WiZMmMD48eOxt7fnww8/LIzYCo3JbMTR3pWqpZ6kRbV+1g5HiEdiNpvp378/I0eOJCQkBABHR0dJ6kI8JrIcscfGxnLt2jXefPNNRo8eXVgxFTqN2o4nK3W0dhhCPBKTyYRGo0GtVjNnzhzCw8Px8/OzdlhCiEKW6Yj9119/pWnTprz00ku0adOGo0ePFmZcj+zM3RKcuVvC2mEIUShWrFhBixYtiI6OBuCZZ56hV69eMkoX4jGUaWL/6quvWL9+PUeOHGHu3LksXLiwMON6ZHtCKrAnpEKO9g0NP8O+Cz/wIPZWAUclRMG4efMmt2/f5vz589YORQhhZZkmdpVKRbVq1QBo0aKFZQ1scXQv5hoh94IxmIt+DXwhABRFYefOnShKcj+E0aNHc/jwYRo3bmzlyIQQ1pZpYn+4cIWdXbYT6IuUNpWu06bS9RztG6ePAsBZ61GAEQmRf6ZPn06/fv0sxaK0Wi2lS5e2clRCiKIg02wdFxfH8ePHLSOC+Pj4NI8bNWpUOBHmUe1S93O8b1xSFCpUOOlkmZuwDYMGDeLy5cs0a9bM2qEIIYqYTBN7qVKlWLBggeWxj4+P5bFKpeLbb78t+OgKSZw+EketK2qVxtqhCJGhGzduMG7cOD788EOqVq2Kn58fK1assHZYQogiKNPEvmrVqsKMw2oi4+8Sp4+kpGtFa4ciRKZOnjzJtm3bqFKlClOmTLF2OEKIIsy2bpzn0JgtQfhoFew0GS/1iU64j4vOA7Vag5ujNyVcyvNEWbmkKYqWW7du4eXlhYODA127dmXjxo20bNnS2mEJIYq4YtnaaX1wKN7OBpzs039v0Rvj2XZ6Efsu/ICiKKhVGrrWf5NKJaXxiyg6jh07RtOmTZk1a5Zl2zPPPCPr0oUQ2So2I/ZjIds4ee0468+W50aUlmtRLnR8IiDdfqHhZ0g0xOHu5CO/JEWR9cQTT+Dn54e/v7+1QxFC2JhsR+xRUVFMnDiRwYMHExkZyfjx44mKiiqM2HLlavhp7NTxhMfpKe/uRKy5I40qdUq3X8i9YACqlSras/rF40VRFNauXcvOnTsBcHZ2Zs+ePQwePNjKkQkhbE22iX3SpEnUqVOHyMhInJyc8PHxKbJ146P1WiL1pbgysSdzuzZM9/ytyEvcjrqCj6svLg6eGRxBCOu4fv06b7/9NhMmTMBkMgHpa0kIIUROZPub48aNG/Tr1w+1Wo1Wq+Xdd9/lzp07hRFbvgqPucHOM9+gUqmoVV4mIAnrUxSFuLg4ACpWrMiiRYv46aef0Ghk2aUQIu+yvceu0WiIiYmx3I++evVqjkYSZrOZKVOmcOHCBbRaLdOnT8fX1zfdfpMmTcLd3Z33338/D+HnnEqlwsfNj9rln6GCV/HrJy9sS3x8PCNGjCAuLo6ffvoJlUpFz549rR2WEKIYyDZDjxw5kkGDBnHr1i1ef/11Bg4cyDvvvJPtgXft2kVSUhJr165l1KhRzJ49O90+a9as4eLFi3kKPLdKuJSjY90RlPesXijvJ0RWHB0d0ev1GAwGYmJirB2OEKIYyXbE3rJlS2rXrs3p06cxmUxMnToVb2/vbA8cFBREixYtAKhfvz5nzpxJ8/zJkycJDg6mX79+XLlyJY/h/6dDnVd45osdGT4XnxSDkza5XKzMhBfWcu/ePY4cOULZsmVRqVQsXboUFxcXuZcuhMhX2Sb2L774Is3jc+fOAfDmm29m+brY2FhcXFwsjzUaDUajETs7O8LCwvjiiy/44osv+PXXX3Mc7MNfDh4WFq0GkggKCrJs05tjuKjfQWm7OpS0f7TRelJScve31McvborzZ7MmRVF47bXXCA0NZcmSJdYO57EgP8sFT85x0ZSrdewGg4EDBw5Qr172xVxcXFwsE4Mg+Z57Soe47du3ExERwfDhw7l37x6JiYlUrlw523uMtWvXRqfTZfjc2C1/8iAxkdJubjRs+N+M+NuRl7l4RqFUmZI08E0/Uz43rhxLXoqU+vjFSVBQULH9bNaiKIrlKtGsWbP4559/KFeunJznAiY/ywVPznHB0+v12Q5oM5JtYn94ZP7GG2/w0ksvZXvggIAA9u7dS6dOnTh16pSltzvA4MGDLetzN2zYwJUrVx554lAZx61MbasQlvRcmu0Kyd3o1Cq53CkK1+bNm1m4cCEbN27ExcWFdu3a0a5dOxnlCCEKVK6zXVxcHLdu3cp2v8DAQLRaLf3792fWrFmMHz+eLVu2sHbt2jwFmhN2GlW69euKYk7+g9xbF4UsODiYs2fPSiIXQhSqbEfsbdq0sVxKVBSFqKgoXn755WwPrFarmTp1apptGZXHLOglPimJXYUkdlHwjh07xpNPPolKpWLMmDEMHDhQysIKIQpVton9s88+o0SJEkDyjHI3N7c0k+KKujvRIQC4O5a0ciSiuJs/fz7Tpk1j6dKl9OzZE51OJ0ldCFHosk3sY8eOzdXM9aLGz7sOZrORcrJ+XRSw7t27s2/fPmrVqmXtUIQQj7FsE3uNGjX4+eefqVu3Lg4ODpbtZcuWLdDA8ou3S3m8Xcrn+fXHQrZxNfw0APH6aJx0bvkVmrBxERERTJ48mXfffZfKlStTuXJlfv75Z2uHJYR4zGWb2IODgwkODk6zTaVSsXv37gILKi/2XS0HwDuptiUa4nCwd36k414NP21J6E46N/y86z7S8UTxsX//flavXo2DgwPz5s2zdjhCCAFkkdg3btxIjx492LNnT2HGk2fBd9LeQzcrZjad+AwPp1K0r5P9ZL+sOOnc6NNo3CMdQxQPkZGRODg44ODgQLdu3VixYgWdOqVvDyyEENaS6XK3b7/9tjDjyLMxW4KoPH0DN6Li02y/FXGJBEMMro4lrBSZKG7OnDlDs2bNmDt3LpB85apbt26WwktCCFEU2PxvpPXBodyIiufdpjcp5+4EJC/LO309+UpD9dKNrRmeKEb8/PxwcXHBzU3mWQghiq5ME/ulS5do27Ztuu0pJTKLwj32MVuCCI2Iw9fTmSfLK0ByCdt7MdcIiwmlgldNSriUs26Qwqbt2rULOzs7WrVqhYuLCwcOHECr1Vo7LCGEyFSmid3X17fIN6tYHxwKQO96vsB/7V/j9FEAlPWoltHLhMiRW7du8fzzz1O2bFmOHTuGnZ2dJHUhRJGXaWK3t7enXLmiP9r19XRmbteGrPu3QUuy5PrwUkVW5IXBYMDe3p6yZcvy6aefUq9ePbmPLoSwGZn+tgoICCjMOPJVWY+qdKn3Bs46T2uHImyIXq9n7Nix3L59mzVr1qBSqXj++eetHZYQQuRKpol98uTJhRlHvtLZO6Gzd7J2GMLGaLVarl+/zt27d4mIiMDLy8vaIQkhRK4Vm+uLZT2qWDsEYYNiYmI4evQobdu2RaVSsWTJElxdXeVeuhDCZhWbxN6sam/Lny+HneTAxbU0rdKTarLcTWRCURR69+5NcHAw+/bto3r16paGR0IIYauKTWIXIrdUKhXvvvsuQUFB+Pn5WTscIYTIF8UmsZ+9+QcAtco1f+RjSeOX4uuPP/5g/vz5fPvttzg7O9OhQwc6dOhg7bCEECLfZFpS1tb8fesP/r6VnNzNigkAlSpvHy+l8QsgjV+KmZ07d7Jv3z7++OMPa4cihBAFotiM2FMzmpIAsNfkfQKUNH4pPs6fP0+NGjUAGD9+PD169KB+/frWDUoIIQpIsRixHwvZRpw+0vLYaE5O7Bq1zGx+3C1dupRmzZqxadMmABwcHCSpCyGKtWKR2FPuh6dcMi/lVokGFQPxcCqZ1cvEY+CZZ56hdu3alC1b1tqhCCFEoSgWl+LVKg2uDiVoVCm5L7aPmy8+br65OoZMmCse4uPjmTNnDkOHDsXPz4+qVavy+++/o5L6wkKIx0SxSOy9nhz9yMdImTDnpHOTCXM2bNeuXSxcuJDw8HC+/PJLAEnqQojHSrFI7A87fX0vd6NDaF61L45alxy/TibM2aaEhAQ0Gg1arZauXbuyYMECevXqZe2whBDCKorFPfbw2BuEx96wPL4fe4ubERezeIUoLv755x9atWrFvHnzgOTR+aBBg3Bykl4BQojHU7FI7HvPfcfec99ZHpv+nRVvp7G3VkiikJQqVYqkpCQSExOtHYoQQhQJxfJSvOHfdex2aknsxVFQUBB6vZ6mTZvi6urKgQMHcHHJ+S0XIYQozoplYjeaDWjUdnmuPCeKrrCwMLp06ULJkiU5duwYOp1OkroQQqRSLBO7yZyEnRSnKVbMZjNqtRofHx+mTZtG9erV0el01g5LCCGKnGKZ2Es4l8PgmGTtMEQ+MBqNzJkzhwsXLrBy5UpUKhUvv/yytcMSQogiq1gkdg+nUkTG37U8blG9nxWjEflJrVZz7NgxQkJCCAsLo1SpUtYOSQghirRikdjrVWhL6P0z1g5D5JOkpCROnDhBkyZNUKvVfP311zg7O+Pq6mrt0IQQosgrFrPLfNwqWsrJKorC6et7uRr+l5WjEnk1YMAAevTowYULFwAoXbq0JHUhhMghm0/srzb6i59PfGp5bDIbORG6g4t3jlkxKvEoXnrpJfr160eZMmWsHYoQQtgcm0/sWo3Jsm4d/mvZKsVpbMeZM2cYMmQICQkJAHTu3JnPPvsMNzdpxCOEELll04m9XeVreDiknf1uNBkAZLmbDVmzZg2bN29m+/bt1g5FCCFsnk1PnnvC5wFAmk5sMmK3DTdu3KB8+fIATJgwgXbt2tGqVSvrBiWEEMWATY/YASITtZaJc5AqscuIvchau3YtDRs2ZOvWrQA4OTlJUhdCiHxi0yP24zfTr2k2mYxA5iP2YyHbuBp+Ot32lF7souDVr1+fcuXK4ejoaO1QhBCi2LHpxH7oevpZ06Xc/RjcbCYoSoavuRp+OsMk7qRzS3NJX+Qfo9HIokWL6NGjBxUqVKB69eocPXoUOzub/vETQogiqVj+ZlWr1KD673HqUXpKUu/TaJyVonv87NixgylTpnD69GmWLl0KIEldCCEKiE3/du1ULSTNY7Ni4sjlzdQs8zSezqUt21OP0mVkXjhMJhNmsxl7e3s6derE9OnTGTBggLXDEkKIYs+mE3sVr6g0jy+HneTinT/RG2JpXXNQmudklF54bty4wdChQ2nXrh1jx45FpVLx+uuvWzssIYR4LNjsrPgxW4IwmtLeR78VeQmABr7trRGS+Jebmxu3b9/m2rVrKJnMdRBCCFEwbHbEvj44lNcbgZN98kdQFIU7UVdwsHfB3bGklaN7/Pzzzz9ERETQqFEj3Nzc+P333/H29rZ2WEII8dixycQ+ZksQoRFx2GlUeDolr1ePTgwnISkGP++6qFSqDCfMiYIRERFBmzZt8PT05MiRIzg6OkpSF0IIK7HJxL4+OBT4b7QOEBadvK20eyVAJswVJk9PT0aPHk3FihVlbboQQliZzSX2lNG6r6czlb0rWrbr7Bwp414FD6f/itbIhLmCYTabWbJkCSdPnuTrr79GpVIxcuRIa4clhBACG0zsKaP13vV8aVeroWV7xRK1qFiilrXCeuxs3bqVc+fOcfPmTUvNdyGEENZnc4kdwNfTmbldG6bbrigKKpUqg1eIR2U2mzl79ix16tRBrVbz1VdfodVq8fHxsXZoQgghUrHJxJ7iStgpohPuExZzlfuxN6lRpgn+PgG4OcrErfw2bNgwduzYwf79+6lSpYqM0oUQooiy2cR+LGQbZ2/utzx20XkSGX8PrZ1M3ioIzz33HAaDATc3WV0g/mM0GjGbzdYOI0NJSUnWDqHYk3OcP9Rqdb6W2bbJxN6u8jXO3rxjedzE/zlqlGlixYiKn2vXrvHJJ58wZ84cHBwc6N69O926dZNbHcIiJiYGjUZTJOv++/v7WzuEYk/Ocf5JSkoiISEBV1fXfDlegf2LNJvNTJkyhQsXLqDVapk+fTq+vr6W53/55RdWrlyJRqOhWrVqTJkyBbU6Z4XwnvB5AEBJ14rci7mGh5Pc581vS5YsYdWqVTRu3Jjnn38eQJK6sDAajWg0GpycnKwdSoYMBgNardbaYRRrco7zj1arJT4+HqPRmC9flAuspOyuXbtISkpi7dq1jBo1itmzZ1ueS0xM5LPPPuPbb79lzZo1xMbGsnfv3lwd31nngZ93XVwdvNCoi96IwRZFR0db/jx+/HiWLVvGwIEDrRiRKKrMZnORHKkLYas0Gk2+3dYqsMQeFBREixYtAKhfvz5nzpyxPKfValmzZo2lmInRaESn0+X42EuO16Zbg7eoVa45vZ4cQ0nXitm/SGRp69atDBo0iB07dgDg7OxMjx49ZJQuhBCFID9/1xbYV+7Y2FhcXFwsjzUajeUyg1qttpQcXbVqFfHx8TRr1izbY545c4akpCSSkuBM8Lks902Z1BEUFPQIn+LxkZCQgFar5ezZs1IOthAUh59Lf39/DAaDtcPIVFxcnLVDKPbkHOcfg8HA5cuX8+VYBZbYXVxc0vylP3zpzmw2M2/ePEJCQli4cGGOvq3Url2bThcOoLMzUaN2R5KMCUTGh1HGozIO9i5p9r1ybCcADRumX+8uktf8r169mlatWlG+fHkaNmxI2bJladq0qbVDK/aCgoJs/ucy5YuzNe+x/vnnn7zzzjtUqVIFSE4y5cuX5+OPP8ZgMKDX65kzZw63bt3CZDJRpkwZxo0bR8mSyU2ijh8/zpdffonRaCQ+Pp6ePXta5pNkZ/Xq1fz444+MGDGCTp06Zbrfhg0buHLlCu+///6jf+B/JSYmMnr0aO7fv4+zszNz5szBy8sr0/31ej1t2rRh6NChvPzyy0Bya+X33nuPH3/80bLfDz/8QHh4OCNHjiQqKoo5c+YQGhpqOXdTp05NM7krLi4OZ2fnLGN98OAB77//PomJifj4+DBr1qx0ZadnzZpFUFAQarWasWPH0rBhQ2bMmMH58+cBuHfvHm5ubmliLY6SkpKoU6dOmn9Ter0+zdXunCqwxB4QEMDevXvp1KkTp06dolq1ammenzx5MlqtlkWLFuV40hwkT5zzcEji71sH0ajtOH19Dx3rjMDB3SX7FwuLnTt38tZbb9G1a1dWrlwJkKvbIUKkNmZLkKUqZH7pXc83w0JUqTVp0oT58+dbHo8aNYo9e/bQvHlz3nzzTV566SXatWsHwKFDhxgxYgTr1q3j1q1bTJ8+naVLl+Lt7U1iYiKDBw+mQoUKtGzZMtvYdu7cydy5c6levfqjfcg8+OGHH6hWrRojR45k69atLFq0iIkTJ2a6/44dO+jUqRMbN27kpZdeytHv2/fee4/+/fsTGBgIwIoVK5g8eXKac50TixYtokuXLvTs2ZMlS5awdu1ahgwZYnn+/PnznDx5knXr1hEaGsp7773Hhg0b+OCDD4DkUezAgQOZNm1art73cVdgiT0wMJCDBw/Sv39/FEVh5syZbNmyhfj4eGrXrs369et58sknefHFFwEYPHiw5YcoMy2/2E6fOgqRiVoaVerEidDfCir8YklRFMxmMxqNhsDAQMaMGcMLL7xg7bCEyBdJSUmEhYXh7u7OuXPncHV1tSR1gKZNm1KxYkWOHTvG8ePHee655yy3nRwcHFi2bFm6Wf43btzggw8+wGg0olKpmDhxIsHBwZw5c4YPPviA+fPnU6FCBSB5JD1+/Hhu3bqFwWBg0qRJaY71ySefcObMGeLi4vD397eMVOfMmYOdnR1ubm58/PHH3Lt3j/Hjx2NnZ4dGo2Hu3LmUKvVfD4ygoCDLyLtly5YsWrQoy/Oybt06PvjgAx48eMC+ffto3bp1lvvfvHmT8PDwNL+PBw0aRK9evdLsd/LkSRYvXpxm25AhQ2jbtm2aWEeMGGGJ9dNPP02T2H18fHBwcCApKYnY2Nh0EzK/++47mjVrZpUvULaswBK7Wq1m6tSpabalXveYcpklN27HJGKnVqXp6iZyJiwsjJEjR9K4cWNGjRqFSqVi3DhpkCPyx9yuDbMdXReEI0eOMGjQIO7fv49araZv3748/fTTbNy40ZJwU6tQoQK3bt0iLCyMGjVqpHkuozXEc+fOZdCgQbRr145z584xYcIENmzYwC+//MKUKVPSvMeaNWsoV64c8+fP5+LFixw6dMhS0Ck2NhY3NzeWL1+O2Wymc+fO3L17l127dhEYGMiwYcPYs2cP0dHRHDp0iFq1ajFu3DiOHz9OVFRUmsQeGxtridXZ2ZmYmJhMz8/Vq1dJSEigRo0a9OrVi2+++SbLxK5SqQgLC0tXWVKj0aQ7Pw0aNGDVqlWZHisnsabMuerYsSMxMTFpRuZJSUmsWbOG9evXZ/keIj2bypBlXB0o5140180WdVqtljNnzqBSqaSmvig2Ui7FR0RE8NJLL1kSko+PD7/9lv6KXmhoKE2bNiUsLIw7d+6kee78+fMoikLNmjUt2y5fvkyjRo0AqFmzZrrXpHblyhXLZfxq1apRrVo1NmzYACTf5nrw4AHvvfceTk5OxMfHYzAYePXVV/n666958cUXKVWqFHXr1qV3797873//4+WXX8bV1ZV33303zfuknr8UFxeXZTXIdevWkZCQwLBhwwA4ceIEoaGhODs7p6saFx8fj06no2zZsuk+p8FgYPv27XTt2tWyLScj9pRYHRwcMoz1559/xtvbm2XLlhEXF8fAgQNp0KABpUqV4vDhwzRq1CjfirY8TgpsuZuwvtu3b3P69GkAPDw82LFjBz/88IMkdVHseHp6Mm/ePCZOnEhYWBj16tUjPDycPXv2WPbZv38/oaGhNG7cmC5durBu3ToePEgudhUXF8fkyZMJCwtLc1x/f3+OHz8OwLlz57JcMeLv789ff/0FwPXr1xk1alSa9759+zaffvop7733HomJiSiKwpYtW+jRowerVq2iatWq/Pjjj+zevZuGDRuycuVKOnTowNKlS9O8T0BAAPv27bMcN7OJmEajkW3btrF69WqWLVvGsmXLGD58ON9//z0lSpQgLi6Of/75BwCTycShQ4eoU6cOpUqVwtPTk127dlmO9e2336Z5DP+N2FP/lzqp5yRWNzc3nJyc0Gg0ODs7o9VqLV9aDh06lKP5DiI9mxqxAzxVuWv2Owmio6Np2bIlLi4u/PHHHzg7O0vjFlGsValShUGDBjF9+nRmzZrF119/zcyZMy2jytKlS7NkyRI0Gg3ly5dn9OjRvPnmm2g0GuLi4ujduzfPPPNMmmOOGTOGSZMm8c0332A0GpkxY0am79+/f38mTJjACy+8gMlkYsKECVy6dAmAunXrsmjRIvr27YtWq6VChQqEhYVRp04dxo0bh5OTE/b29kydOhVFURg9ejQLFy5ErVYzfvz4NO8zYMAAxo4dy4ABA7C3t+eTTz4BkqtF1qhRw5IM9+zZQ61atfDw8LC8tmfPnnTv3p133nmHWbNmMWHCBNRqNQaDgbZt29KkSXJp7rlz5zJ16lS++eYbDAYDFStWZPr06bn+O3nttdcYO3YsP/74I56enpZY586dS4cOHejatSsnTpygf//+mEwmunbtSuXKlQEICQnhueeey/V7ClApiqJYO4jspEz533IXpnT67xuf3hCP3piAk9YNO419mtesO5Zc6a5Po8f3PvK8efMoUaIEQ4YMydFM2OKwDMsWFIfzXBSWu2UlJ0uxipvdu3fj5OTE008/XSjv9zie44KU0b+plNxXu3btXK1asqkR+/i2tdM81tk7obOXe+6QPOP9p59+4uDBg5YlKaNHj7ZyVEKIwlKzZk3Kli1r7TBEEWBz99i3/7WE7X8tAcBoMqA3JmBWTFaOyvoURWHZsmWsW7eOq1evWjscIUQhk6QuUtjUiB0gJvGB5c+nb+zh9PW9dKz7KqXc/KwXlJUoisLVq1epVKkSarWar776CkVR8PPzs3ZoQgghrMTmRuziP++99x4tWrQgJCQEAD8/PypVqmTlqIQQQliTzY3YU8Tpo7gZcREAtUpj5Wiso3nz5ly4cCFXJXmFEEIUbzaZEcyKmV9PL+Z+7E18S9TG26WctUMqFOHh4UyePBm9Xg8kL1355Zdf8PX1tXJkQgghigqbG7ErioLeEIdZMVGvQlvqV2z32BRc+fzzz/niiy/w9fVl2LBhqFSqx+azC/GwrLq7QXJnseLY3S3Fzp072b59u2VteGYKqrtbTkh3N+uwucReqWRd9IZ4PJxKUbt88a9KlHqt6JgxY/Dz87M0zhGiqDgWso2r4afz9Zh+3nVpVCnzpAmPZ3c3gOnTp/PHH3+kKX+bGenu9vixucTeqFJna4dQaPbt28eIESP44osvaNeuHS4uLrz00kvWDkuIIulx6e4GyaVa27Vrx9q1a7M9L9Ld7fFjU4k9Iu42+tg4ynlWw05tn/0LbFyJEiWIj4/n5s2b1g5FiCw1qtQp29F1QXgcu7sBdOrUiT///DPb8yPd3R5PNjV57tLd4+w9t4o/L2+2digFZvv27dy6dQuA2rVr89dff8mldyEy0aRJE1atWsXq1auxt7dP090toy/EoaGhlClTJsMOZufPn+fcuXNptuW2u1v9+vWB5O5uqUemqbu7TZ48OU13twcPHvDiiy+yfft27Ozs6N27N56enrz88susXr0ajSbvq35Sd3dbtmwZQUFBhIaGWkbJqWXX3W3Lli1ptp08eZJBgwal+W/37t1p9smuE11Kd7edO3eye/duvvjiC+7evQsg3d0egU0l9tuR/6BCbVnmVtzs37+fgQMHMnbsWMs2d3d3K0YkhG14nLq75ZR0d3t82dSleIM5CTuNfbGbCZ7SH71Fixa89tprvPDCC9YOSQib87h0d8uMdHcTKWyqu9v5xK1o7NRo1HbZdm2zhe5u0dHRjB8/nmrVqvH2229bO5xi0XXMFhSH8yzd3Yoe6e5m2/Kzu5tNXYr3cipTrKrMGY1Gdu/eza+//orJJI1shBB5V7NmzUJL6qJos6nE/kTZZjZ/GT46OtpSeMHLy4vNmzezZcuWR5ogI4QQ0t1NpLCpe+yl3CvjaO9i7TDyLC4ujhYtWmBvb8/+/ftxcnKiWrVq1g5LCCFEMWJTiV2lUtGl/pvWDiPPnJ2d6d27NzqdDnv74r8OXwghROGzqcRui/bu3cvu3bstM0ofrkYlhBBC5CebS+y3IpOXj5T1qGrlSLKnKAqzZ8/m5MmTDB48WC67C1HI3nzzTb744gtrh1HgsmqIo9VqC7QhTmEIDg7m+eef5/vvv6du3bpAxg123n33Xfr3789TTz3FpUuXmDdvHgkJCcTHx/PMM88wcuTIXM/TOnXqFDNmzECj0Vh6EKQWGRnJ6NGjiY2NxcPDg+nTp1OiRAkOHz7MZ599hp2dHSVKlGDOnDnpGuAUFJuaPAdw8NJPHLz0k7XDyFJK5SSVSsWXX37Jzp07JakLYQWPQ1JPkVKFb9WqVWzYsAF7e3v27NmDoii8+eabBAYGsmrVKr7//nt69erFiBEjMJlMXL9+nenTpzNv3jxLFb9Nmzaxf/9+a38ki3Xr1jF06FC+//77HO0fHR3Ne++9x4QJE1i1ahU//vgjFy9eZM2aNbl+7w8//JBPPvmEH374geDgYM6ePZvm+cWLF9OwYUN++OEHBg0axKeffgrAlClT+PLLL1m9ejW+vr6sW7cu1++dVzY3Yi/qpk6dytKlSzlw4AC+vr6Wb9BCFHcptSMeVrvcM9Qsm7wMa/+FtdyNDkm3T0nXirSqMRCAi3eOEnx9T7Y1KDZs2MDevXtJTEzk3r17DB48mN27d3Pp0iXefvttOnfuTLNmzTh48CDBwcHMmDEDRVEoVaoUH3/8Ma+88gqenp5ER0ezZMkSPvjgA65fv47JZGLo0KHp2rHGxsbywQcfEBMTQ0REBH369KFDhw48//zzbNu2DZVKxUcffWRpNpNy+83Dw4OZM2fy999/8/HHH2Nvb0/fvn1xcHBg9erVluMvWLAAT09PPvroI86cOYO3tzc3b97kq6++QqPRMGnSJPR6PTqdjmnTplGmTJlMz03qhjhnzpzJl4Y4V69eZeLEiRgMBhwcHJgxYwbTpk2jU6dOtGzZkv3797Nt2zZmz55N69atqVy5MhUrVuSPP/5g06ZNODk5sXTpUuzs7Gjfvn2OP09cXBxHjhxh69atdO3alQcPHuDl5ZXlz8bu3bt56qmn8PPzA5Jr3c+ZMyfd3KbvvvuOHTt2pNk2Z84cywqD2NhYkpKSqFixIgDNmzfn8OHD1KpVy7L/P//8w7vvvgskV9qbOnUqAKtWrbKcU6PRmKt16I9KEns+q169OhUqVCA+Pt7aoQhR7MXFxfHNN9+wdetWVqxYwY8//siff/7J8uXL6dz5v06QkyZNYv78+fj7+7N69WouX74MQNeuXQkMDOS7776zlKWNjY2lZ8+eNGnSJE0CCQ0NpXPnzjz77LPcvXuXQYMGMXDgQKpXr87x48epV68eR48e5YMPPmDgwIHMnDmTKlWqsG7dOpYuXUrTpk3R6/WWkdvXX3/NkiVLcHR0ZPLkyfzxxx84OTkRGRnJ+vXrefDgAc8++yyQnGwGDRrEM888w+HDh/n444/T9WHPrCHOtm3b8qUhzpw5cxg+fDgtW7Zk27ZtlmW7Gbl9+zYbNmywnNPffvuN5557jm3btrFs2TI++uijbD9Pim3bthEYGIhOp6Njx46sX7+e4cOHZ/reKY1sHv7MGRXTeeGFF7Ks9BkbG4uLy38rsZydnbl+/XqafWrWrMmePXt44okn2LNnD4mJiUByvwJIbvGbcquksEhif0SxsbEsXryYkSNHotVq6du3Lz169CiyFbmEKCg5qfLYsnq/bPepVrox1Uo3ztF7pvQjd3V1xd/fH5VKhbu7O3q9Ps1+9+/fx9/fHyDNveNKlSoByc1emjZtCiQ3LvH39+eff/5h4cKFQPIIt2fPnqxcuZLffvsNFxcXjEYjAH379mXjxo3cu3ePNm3aYGdnx+XLl/noo4+A5AYqKe+T8n9I7t44duxYnJ2dLQ1kUjeS8fLyspRXvXjxIosXL2bp0qUoipLhqpqU3vQRERG89NJLloY4pUqVyrQhTtOmTQkLC8uwIY6iKGn6vYeEhNCgQQMgubtcXFxcmvrxqYuYenp64unpCUCfPn2YMmUKlStXxs/PD09Pzxx9nhTr1q1Do9EwbNgwEhMTuXPnDi+//HKmjWwcHBwoW7Ysf//9d5rnrl+/zp07dyxNfSD7EXvqJjaQcSOb4cOHM2PGDIYMGUKLFi0oXbq05bkVK1awfft2li5dKiN2WzJ//nzmz5+Pk5MTr732GiqVSpK6EIUkpxOhfHx8uHr1Kn5+fixZssSSYFNen9LsJTAwkNjYWC5evIi/v3+atqSzZs2ifv36DBw4kCNHjliamzz99NPMmzePu3fvMnnyZCA5gackiKCgIO7duweAWp08rSkmJobPP/+c33//HYChQ4eiKApVq1Zl06ZNAERFRXH16lUAKleuzEsvvURAQACXL1/m2LFjmX7WlFHy4MGD+fnnnwkICLA0xGnTpg2QtiFOhQoVeOONN+jUqRNeXl6WhjhvvPFGmsSe0uSmadOmbN68mXv37qHVai2fLXUiTfmcAH5+fiiKwtKlSxkwYECuPs+FCxcwmUz8+OOPlm1Dhw5l79691KhRg0WLFllK20ZGRnLp0iX8/f2pXLkyixcvZsCAAVSsWBGDwcDs2bNp2rRpmsSe3YjdxcUFe3t7rl27RoUKFfjjjz/STZ47fvw43bt3p0mTJuzYsYOAgAAAvvrqK86ePcuKFStwcHDI9D0KgiT2PDAYDJZvmG+//TZOTk4MHTrUylEJITLz0UcfWRqelCxZkiFDhvDtt99anu/bty+TJk1iwIAB6PV63nzzTUqUKJHmGK1bt2bKlCls2bIFDw8PNBoNSUlJaLVa2rdvz6FDh/D19QWSJ06NHTvWUip6xowZaTrHubi4EBAQQI8ePXBycsLNzY2wsDB69uzJ/v376d+/P97e3jg4OGBvb8/YsWOZMmUKer2exMREPvjggyw/b+qGOJ9//nm+NcSZPHkyX331FQ4ODnz00Ufcv3+fCRMmsGXLFsv97Iz07t2bBQsWWJrMZPZ53n33XSZMmGCZrb9u3Tq6d++e5lh9+vRh9erVfPPNNwwcOJCBAwfi7OyM0Wjkgw8+sFxynz17NhMnTkRRFOLi4mjdujUDBw7M8rxl5KOPPuL999/HZDLRvHlz6tWrB8BLL73E119/TaVKlSwdOX18fJg5cybh4eF8+eWXPPHEE7zyyisAdOzYMU/vnxc21QSmdu3aJJiiAPBw8snyNQXVBCYoKIgRI0Ywb948Wrduna/Htrbi0JzEFhSH8yxNYArG5cuXOX/+PJ07dyYiIoIuXbqwd+/eInmeC+Icf/rpp7z66qvpJu49Dh7bJjCQnNCzS+oFyc7Ojhs3blj6LgshRH4pU6YMv/zyC3379uXll1/m/fffL5JJvaD079//sUzq+c3mLsWbzMkTVjTqwgv9yJEj+Pn5Ubp0aerVq8fJkyezXGoihBB54eTkxFdffWXtMKxGGtnkD5tL7BuCPgbSX2I/FrKNq+GnLY/j9dE46dLOXsyLI0eO0LlzZzp16mSZSCNJXQghRFFlc4k9tdTJPE4fCYCzzgMAJ50bft51H/k9GjduzAsvvGCZzSmEEEIUZTaV2Led/op4ffLkuXXHZqdJ5s46D/y869KoUqcsjpC9hIQEZs2aRenSpXn99ddRq9UsWLDgUUMXQgghCoVNJfaH5VcyTy0+Pp61a9fi7e3N8OHDsbOz6VMkhBDiMWNTWatT3dfYfHo+kL/L2BITE7l9+zaVKlWiRIkSrFu3Dn9/f0nqQgib1KZNG8qUKYNarcZkMhEfH8+0adOoU6cOiqLw/fff88svv1h+x7388suWdetRUVHMmTOH0NBQSye4qVOnZlhm1lq6d+9OQEAAH374oWVbSl+AFKlr1+v1ej777DOCg4NRqVQ4OTkxderUXM+XSkxMZPTo0dy/fx9nZ2fmzJmTrm79smXL2Lp1KyqVildffZXAwECWLFnCgQMHgOQGNeHh4WlizW+PfeZKTEykbdu2GI1Gfv/9dxwdHS1tAYUQwlZ98803lrXPBw4c4IsvvmDx4sWsXbuWEydOsGLFCnQ6HREREQwfPhx3d3fq16/Pe++9R//+/QkMDASSy6JOnjyZ+fPnW/PjWAQFBVGtWjWOHDmSrpZ7ZmbMmEHlypUt3eF27tzJO++8w9q1a3P13j/88APVqlVj5MiRbN26lUWLFjFx4kTL89HR0axatYrffvuNhIQEnnvuOQIDAxk+fLilvv2IESPStJotCDaX2OtVaJOvx3NwcKBVq1YYDAZsoFaPEEVWUezu9t133/Hbb79hNBpxdXVl4cKFmM1mxo8fz61btzAYDEyaNImQkBB++uknzGYzb731Fvfu3WPlypVotVr8/PyYOnVqhp3BHj72e++9x+DBg2ncuDGnT5/mq6++4vPPP+fDDz8kNDQUs9nMO++8w1NPPUWXLl3w8/NDq9UyZswYSyW2yMhI3njjDdq1a8fevXv5/PPPcXFxwd3dnerVqzNy5Eg++eQTjh07hqIoDBkyhI4dO2Z5rm7dumWpcf7dd9/x7bffWpK+p6cnb775Jj/88AMlS5YkPDzcktQBBg0aRK9evdIcz2w2M3v2bM6dO4fBYGDkyJG4urqyZs0ayxeAlBH0uHHjiIyMJDIykkqVKtG4cWN69OjBvXv3GDFiBBs2bMjV51m3bh3t27enTJky/Pzzz1mWhIXkwi979uyx1O4HCAwM5Mknn0yzX2hoaJokDdClSxf69fuvv0FQUBAvv/wyAC1btmTRokVp9nd0dKRs2bIkJCSQkJCQruTxb7/9hpubGy1atMgy5kdlU4n9yOVNPPNE30c+zqlTp/j1118ZP348ANOnT89xzWkhRNGRVXe3jh07EhkZyYoVK1Cr1QwbNoy//vqLv/76i3LlyjF//nwuXrzIoUOHcHNzw83Nja+++oqIiAgmT57Mxo0bcXFxYebMmaxduzZNAjGbzRkeu0+fPmzcuJHGjRuzceNG+vbty7p16/D09GTmzJlERETwwgsvsHXrVuLj43n99dd54oknOHToEEOHDuWpp57ixIkTLFy4kNatWzN9+nTLnJ9Ro0YBsG/fPm7cuMGaNWvQ6/X07duXZs2apWtO8tJLL6HX6wkLC6NFixaWsqcRERHpLh+n7vSW0jwmhUajSXcZfvfu3ZYudPfu3eO7776zNNHJSJMmTRgyZIilOU6PHj3YtGkTPXv2zPHngeSmW0FBQUyfPp2qVavy+uuvZ5nYVSoVkZGReHt7p/sdn9KkJoWvr2+a3gAZiY2NtZwLZ2dnYmJi0u1TpkwZOnfujMlkYsSIEWmeW7x4saVfe0GyqcSeHxRFYdy4cRw9epRu3bpRq1YtSepC5IOi1t1NrVZjb2/Pe++9h5OTE3fu3MFoNHLlyhVatmyZ/F7VqlGtWjU2bNhgaQxz/fp1qlSpYrnE26hRI/744w/mz5/PiRMngOTL0xkdu0WLFsybN4/IyEiOHz/OxIkTmTZtGkFBQZw+nbw012g0EhERAfzX7a1kyZJ89dVXrF+/HpVKhdFo5MGDB7i4uFh6ej/55JOEh4dz8eJFzp49y6BBgyzHSz0iT5FyKf7TTz/lxo0bltr3Li4uREZG4uHhYdk3NDSUMmXKULZs2XSd3gwGA9u3b6dr166WbSEhIZZbliVLluTdd9/lzz//TPO61FdAUz6nv78/JpOJmzdvsm3bNlasWMHatWtz9HkANm/ejNlstiTMe/fucfjwYZ5++ul0v8fj4+PR6XR4enoSHR2Noihp9tmyZQsdOnSwXInJyYg9dbe3jDq97d+/n7CwMHbv3g3AsGHDCAgIoG7duvzzzz+4ublZ+gkUJJsqKdvEv3v2O2UiOjoaSP4Gt2DBAjZu3EitWrXyKzQhhBVk9aX8/Pnz7Nq1i88++4xJkyZhNptRFMXSpQySk3jKSDilI1n58uW5fPky8fHxABw9epRKlSrx7rvvsmrVKlatWsWlS5cyPLZaraZDhw5MmTKFdu3aodFoqFy5Mp07d2bVqlX873//o0OHDri7u6d5zwULFtC9e3fmzZvHU089haIolChRgri4OB48eABAcHAwkNwZ7amnnmLVqlWsXLmSjh07phtlp/bOO+8QFhZmub/8wgsvMH36dEtt8vv37/PFF1/Qv39/SpUqhaenZ5p2rN9++22axykxnD17FkjuVDds2DB0Op2l09vNmzeJiorK8O+pd+/ezJs3jypVquDm5parz7N+/Xq+/vprli1bxrJly5g4cSKrV6+2/L0dPnzYsu+BAweoU6cO9vb2NG/ePM1ofPv27axcuTLN7ZWUEXvq/1IndYCAgABLV7/9+/en6/ng7u6Og4MDWq0WnU6Hq6urJfccOnTI8oWyoD0WI/bPP/+cBQsW8Pvvv1OhQgWqV69O9erVrR2WEKIA+fr64ujoSM+ePdFqtZQsWZKwsDD69+/PhAkTeOGFFzCZTEyYMIFLly5ZXufl5cXIkSMZPHgwarWaihUrppvslNmxAXr16kW7du0sfb779+/PxIkTeeGFF4iNjWXgwIFp2poCdOjQgRkzZrB48WLKlClDREQEarWaSZMm8corr+Dq6orZbMbX15c2bdpw9OhRBg4cSHx8PO3atctyAplarWbGjBk8//zztGvXjkGDBmEymXj++eexs7NDpVLx+uuvW9qNzp07l6lTp/LNN99gMBioWLEi06dPT3PMtm3bsn//fgYMGIDJZOKNN96gdu3auLq60qdPH/z9/TNNzimfNaV0bmafZ8OGDQD07NkTSG4Lm9LaNkX79u2ZNWsWt2/fZvr06Xz00UfMnz8fs9lM/fr1LZ3hxo8fz6xZs+jfvz+QnIAXLlyY6TnLzIABAxg7diwDBgzA3t6eTz75BIDly5dTsWJF2rZty6FDh+jbty9qtZqAgACaNWsGJF/lSPlzQbO57m55aVb//fffM2vWLJYvX55uwoT4T3HoOmYLisN5lu5uhWPx4sUMHToUrVbL+++/T/PmzXnuueesHRZQ8Of4/PnznDlzht69exfYexQlj3V3t5wwGAwsW7YMg8EAJH/LOnz4sCR1IYRNcXZ2pm/fvvTv3x9FUejUKf+KcRV1Hh4e6Wbji5wplpfiP/nkE+bOnUt8fDwjR45EpVLlaK2jEEIUJS+88EK2y7mKq9KlS1s7BJtVbBK72Wy23Ld67bXXiI2N5cUXX7RyVEIIIUT2Hp61/yiKxaX48+fPExgYyP79+4HkiRHTp0/PcLmEEOLRqdVqjEajtcMQotgwmUzpJlXmVbEYscfHx3P69Gn27dtXaMsJhHic2dnZkZCQQHx8PBqNpsjVgjAYDJbJSKJgyDnOH4qiYDKZMJlM+dafxGZH7OfPn7csLwkICODw4cNMmjTJylEJ8fhwdXVFq9UWuaQOcPnyZWuHUOzJOc4fKpUKrVabr012bHLEfurUKTp06MCzzz7Lt99+C0CVKlWsHJUQj5+i3AGxqC7FK07kHBdNBTZiN5vNTJ48mX79+jFo0CBCQ0PTPL9nzx569epFv379+PHHH3N17Lp169K5c2cGDhyYnyELIYQQNq/AEvuuXbtISkpi7dq1jBo1itmz/+v8ZDAYmDVrFt988w2rVq1i7dq1llKEWUn5AqBWq1m2bBkdOnQoqPCFEEIIm1Rg19GCgoIsrenq16/PmTNnLM9dvnyZihUrWuolN2zYkOPHj2faqi+lON6mTZvo0aNHuvaJIv/o9Xprh/BYkPNc8OQcFzw5xwUrZXJibgvEFlhij42NTVMURqPRYDQasbOzS9P6DpKrK8XGxmZ6rJQKcuPHj+fChQsFFbKANF/ARMGR81zw5BwXPDnHhcNgMODg4JDj/QsssadubwfJ99xTJto8/FxcXFyWMwKdnZ2pVq0a9vb2RXIGrhBCCJHfFEXBYDDkuiZ/gSX2gIAA9u7dS6dOnTh16hTVqlWzPOfv709oaCiRkZE4OTlx/Phxhg0blumx1Gp1vi4FEEIIIWxBbkbqKQqsu5vZbGbKlClcvHgRRVGYOXMmf//9N/Hx8fTr1489e/bw5ZdfoigKvXr14vnnny+IMIQQQojHik20bRVCCCFEzths5TkhhBBCpCeJXQghhChGilxiL8iKdSJZduf4l19+oU+fPvTv35/JkydjNputFKntyu4cp5g0aRIff/xxIUdXPGR3jk+fPs3AgQMZMGAAb731lqy5zqPszvPmzZvp0aMHvXr14vvvv7dSlMVDcHAwgwYNSrc913lPKWJ27NihjB07VlEURTl58qTy6quvWp5LSkpS2rVrp0RGRip6vV7p2bOnEhYWZq1QbVZW5zghIUFp27atEh8fryiKorz77rvKrl27rBKnLcvqHKf44YcflL59+yrz5s0r7PCKhazOsdlsVrp166ZcvXpVURRF+fHHH5XLly9bJU5bl93PcrNmzZSIiAhFr9dbfj+L3FuyZInSpUsXpU+fPmm25yXvFbkRe04r1mm1WkvFOpE7WZ1jrVbLmjVrcHR0BMBoNKLT6awSpy3L6hwDnDx5kuDgYPr162eN8IqFrM5xSEgIHh4erFy5khdeeIHIyEgqV65srVBtWnY/y9WrVycmJoakpCQURZFaI3lUsWJFFi5cmG57XvJekUvsmVWsS3kuNxXrRMayOsdqtRpvb28AVq1aRXx8PM2aNbNKnLYsq3McFhbGF198weTJk60VXrGQ1TmOiIjg5MmTDBw4kOXLl3PkyBEOHz5srVBtWlbnGaBq1ar06tWLzp0706pVK9zc3KwRps1r3759ht0S85L3ilxiz8+KdSJjWZ3jlMdz5szh4MGDLFy4UL6B50FW53j79u1EREQwfPhwlixZwi+//MKGDRusFarNyuoce3h44OvrS5UqVbC3t6dFixZS/jSPsjrP58+f5/fff2f37t3s2bOHBw8e8Ouvv1or1GIpL3mvyCX2gIAA9u/fD5BlxbqkpCSOHz9OgwYNrBWqzcrqHANMnjwZvV7PokWLLJfkRe5kdY4HDx7Mhg0bWLVqFcOHD6dLly707NnTWqHarKzOcYUKFYiLi7NM9Dp+/DhVq1a1Spy2Lqvz7OrqioODAzqdDo1Gg5eXF9HR0dYKtVjKS94rsJKyeRUYGMjBgwfp37+/pWLdli1bLBXrxo0bx7BhwywV60qVKmXtkG1OVue4du3arF+/nieffJIXX3wRSE5EgYGBVo7atmT3cyweXXbneMaMGYwaNQpFUWjQoAGtWrWydsg2Kbvz3K9fPwYOHIi9vT0VK1akR48e1g65WHiUvCeV54QQQohipMhdihdCCCFE3kliF0IIIYoRSexCCCFEMSKJXQghhChGJLELIYQQxUiRW+4mRHF048YNOnTogL+/f5rtX3/9NWXKlMnwNSnlJUeOHJnn992wYQOzZ8+2vEdiYiKNGzfmww8/zLDKVVYWLFhA7dq1adu2LYMGDWLVqlUAdO/enU2bNuU5RoBBgwZx584dnJycgORqWxUqVODjjz+2VELMyI8//oiTkxNdunR5pPcXojiRxC5EIfHx8XnkBJgXbdq0Yfbs2QCYTCb69+/P+vXr6d+/f66O8/bbb1v+fPToUcuf8+szTZ8+naeeegpIrm721ltvsXz5ckaPHp3pa06cOEHjxo3z5f2FKC4ksQthZRcvXmTatGnEx8fz4MEDhg8fzoABAyzPGwwGJkyYwKVLlwAYOHAgffv2JTw8nMmTJ3Pnzh1UKhWjRo2iadOmWb6XRqPhySeftBzrp59+Yvny5ahUKmrVqsWkSZPQarUZvt+4ceNo3Lgxf//9NwB9+vRh3bp1VK9enbNnz9KqVSt+/vlnvL29iYyMpEuXLuzdu5fDhw/z+eefYzQaKV++PNOmTcPT0zPLOOPj44mIiKBu3boA/PrrryxfvpzExESSkpKYOXMmiYmJ7NmzhyNHjlCyZElq1qyZ6/MhRHEk99iFKCRhYWF0797d8t/SpUsBWLduHa+//jo//fQT3377LXPnzk3zupMnTxIVFcXPP//M4sWLLZ2dZsyYQa9evdiwYQNfffUVkydPzrY5REREBH/88Qf169fnwoULfP3116xatYotW7bg6OjIF198ken7pZg4caIl7hR2dnZ06NCB7du3A/Dbb78RGBhITEwMn3zyCcuWLePnn3+mefPmmfafnzhxIt26daN58+b069ePpk2bMmTIEMxmM2vWrOHrr79m8+bNvPzyyyxZsoSmTZvSpk0b3nrrLVq0aJGn8yFEcSQjdiEKSWaX4seNG8eBAwdYvHgxFy9eJD4+Ps3zVatWJSQkhGHDhtGyZUvGjBkDwKFDh7hy5Qqff/45kNxi9/r169SsWTPN6/fs2UP37t1RFAVFUQgMDKRLly6sXr2a1q1bW0bP/fr1Y/z48QwfPjzD98tOt27dmDVrFi+88AK//PIL7777LsHBwdy+fZvBgwcDyZfY3d3dM3x9yqX4EydO8NZbbxEYGIhWqwXgyy+/ZM+ePYSEhHD06FHU6vRjkpyeDyGKO0nsQljZO++8g5ubG61bt6ZTp0788ssvaZ739PRk69atHDx4kH379tGjRw+2bt2K2Wxm5cqVeHh4AMlXBEqUKJHu+KnvsadmNpvTPFYUBaPRmOn7Zadu3bpERUVx+vRp7t69S4MGDdi1axcBAQF8/fXXAOj1+jSdqjISEBDAoEGDGDVqFBs3bkSv19O7d2+6detGo0aNqF69OqtXr87w8+TkfAhR3MmleCGs7ODBg7z11lu0a9fO0kXLZDJZnt+9ezejR4+mVatWTJw4EScnJ27fvk2TJk34/vvvAfjnn3/o2rUrCQkJOX7fxo0bs2fPHiIjI4HkGeZPPfVUpu+X2sM9uVN07dqVDz/8kM6dOwNQr149Tp06RUhICACLFi1Kd6shI0OHDiUuLo61a9dy9epVVCoVr776Kk899RQ7d+60nB+NRmP586OeDyGKCxmxC2FlI0eOZODAgeh0OmrUqEG5cuW4ceOG5fmWLVvy22+/0blzZ3Q6Hd26daN69epMnDiRyZMn07VrVwDmzp2Li4tLjt+3Ro0ajBgxgkGDBmEwGKhVqxYfffQROp0uw/dLrW3btnTv3j1dH/lu3bqxYMEC5s+fD0DJkiWZOXMm77zzDmazmVKlSjFv3rxsY9NqtbzzzjvMnDmTnTt3UrNmTTp27IhKpaJ58+YEBQUB0LRpUz799FNcXV0f+XwIUVxIdzchhBCiGJFL8UIIIUQxIoldCCGEKEYksQshhBDFiCR2IYQQohiRxC6EEEIUI5LYhRBCiGJEErsQQghRjEhiF0IIIYqR/wNSdeiVpxqm1AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x396 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_model(best_model, plot=\"auc\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The AUC Score for both classes was good, 0.87.<br>\n",
    "Now, the confusion matrix and class report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeoAAAFkCAYAAADv13iSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAoPUlEQVR4nO3de3zP9f//8fvOM5PDxjDxEcswa8ghhxyLZIYKX4eshI9jRTKFxPjk+HFKH4dOU0IxEzIhh5A55LBMUTnMbDSHT23M7P3+/eHnXTMzPnm/92S36+XicmnP1/v9ej022m2v1/swJ6vVahUAADCSc34PAAAAckeoAQAwGKEGAMBghBoAAIMRagAADEaoAQAwGKGGsRITE1W1alWFhYUpLCxMoaGheu6557Rnzx67HG/Dhg2KjIz8W/v473//q8jISIWGhiosLEzt27fX559/btvevHlzHTx48O+Oms2MGTO0YsUKSdLy5cvVtGlT9erVK9v637F371716tXL9nfQp08f/fTTT39rn3+d83/Ru3dvHT169G/NcN2sWbNUpUoVLVu2LNt6enq6atasqb59++a5j9mzZ2v9+vU33Xa3/h5QgFkBQ508edIaEhKSbW316tXWJ554Ip8murXLly9bQ0NDrfPmzbNmZmZarVarNTEx0dqyZUvr0qVLrVar1dqsWTPrgQMH7DZDjx49rCtWrLhr+4uLi7M2adLEevDgQdtaTEyMtW7dutbU1NT/eb93e86/Y+bMmdamTZtae/TokW09Ojra2qBBA2ufPn3y3Ef37t2tX331lb1GRAHnmt8/KAB34sKFCypZsqQkyWKxaMKECdq/f7/S0tJktVoVGRmp2rVr69y5cxoxYoROnDihYsWKqWTJkgoICNCgQYO0efNmTZkyRc7Ozqpataq2b9+uRYsWKS4uTrGxsZo7d6569OihkJAQ7d27V6dPn9Zjjz2mcePGydnZWcuXL9e8efPk6emp+vXrKyoqSocOHdKaNWvk5eWl3r172+b19/fX9OnTlZmZme3zuNXsu3fv1jvvvCOLxSJJ6tu3r1q1apXrekREhAICApSSkqKDBw8qMTFR58+f1+HDhxUQEKBevXrp559/1vjx43XhwgVlZWWpR48eevbZZ7Vz506NHz9eXl5eSktL07Jly+Tu7m6bc+bMmerfv7+CgoJsa+3atZOHh4eysrIkSUuWLNHChQvl7OwsX19fjRo1ShUrVlRERIS8vb31448/Kjk5WVWqVNHEiRM1Y8aMXOeUZPt8evXqpUWLFmnx4sVyc3OTh4eHxo4dq8qVK6t58+aaMWOGatSoccfHL1y4cI5/V40bN9b69euVnJys0qVLS5Kio6PVrl07/fLLL5KkX3/9VWPHjlVaWprOnj2rwMBATZ8+XV988YXi4+M1adIkubi4aMOGDbpw4YJOnjyppk2bKjU1VQEBAWratKm6dOmiqKgoVa1aVa+//rpcXV01YcKEv/c/Be57hBpGu3z5ssLCwiRdu6x89uxZvfvuu5Kk/fv368yZM1qyZImcnZ01b948zZ8/X7Vr11ZkZKQqV66suXPn6syZM+rYsaMCAgJ0/vx5vf766/r4448VGBio6OhoRUdH3/TYJ06c0MKFC5Wenq6nnnpKcXFx8vX11ZQpU7R8+XKVLl1as2fPtgUrPj5etWrVyrGf6tWr51i71eyzZs3SCy+8oKefflqHDx/WkiVL1KpVq1zXr3vjjTeUkJCgbt26qXXr1oqIiJAkXb16VYMHD9akSZNUvXp1/f777+rcubMqV64sSTpy5IjWr18vf3//HHPGx8frrbfeyrF+/bg7duzQggULtGTJEpUoUULLly/XgAEDtHr1atv9o6Ki5OTkpE6dOmnt2rW5znmjrKwsTZgwQRs3blSpUqW0YsUK7dmzxzb3/3r8Z555JsexXF1d9dRTT2nlypXq06ePkpKSlJaWpoCAAFuoly5dqvbt2yssLEyZmZnq2LGjNm3apG7dumnt2rXq1q2bnnjiCW3YsEGXL1+2zXD986tUqZKGDRum4cOHq0ePHjp8+LCWLl16088d+CtCDaN5enoqJibG9vH27ds1YMAArVy5UjVr1lTRokW1ePFinTx5Ujt37rSdLW3evNkW4FKlSql169aSpN27d6tSpUoKDAyUJHXo0CHXx6WbNWsmZ2dneXt7q0KFCrp48aIOHz6shg0b2s66unfvrlmzZkmSnJycZL3Nd+S91exPPfWUxo4dq40bN6pBgwYaMmTILdfzcuzYMZ04cUJvvPGGbe3y5cs6dOiQKlWqpDJlytw00pLk7OxsO4O/ma1bt6pNmzYqUaKEJKljx44aP368EhMTJV07U71+hv7www/r4sWLtzWzJLm4uKh169bq0qWLmjZtqkaNGqlJkyZ2O35YWJjefPNN9enTRzExMWrfvn227cOGDdO2bds0f/58HTt2TGfOnFF6evpN91W7du2brnfq1EnffvutIiMjFRMTI09Pz9v6WqBg48lkuKc0aNBA5cuX18GDB7Vp0ybbE31atGih//u//7PdztXVNVs0nZ2v/VN3cXHJEdPr227012+i1yN84/1dXFxs/x0SEqJ9+/bl2M+GDRs0ceLEbGu3mr1Lly5auXKlGjZsqG+//Vbt2rVTRkZGrut5ycrKUpEiRRQTE2P7s3TpUtuZpZeXV673DQkJ0f79+3Osv/3229q+fftNI261WnX16lVJN/8a3ujG9b8+TDBlyhT95z//Ufny5TVv3rwcP5zcjeNfFxwcrKysLCUkJGjNmjVq27Zttu1DhgzR0qVL5e/vr/DwcFWvXj3X/eX2Nb1y5YqOHz+uIkWKKCEhIddZgL8i1Lin/Prrrzp16pSqVq2qbdu2qVmzZuratauCgoK0fv1622XoJk2a6IsvvpAknT9/XuvXr5eTk5Nq1aqlY8eO6fDhw5Kk2NhY/fe//5WTk9NtHb9Ro0basWOHUlJSJCnbM7qffPJJ/fHHH5o/f75tjpMnT+qdd95RpUqVsu3nVrN36dJFCQkJ6tixo8aNG2e75J/bel4qVqyY7crE6dOn1bZtW8XHx+d53379+mn27NnZbrt8+XLFxsbq4YcfVuPGjbVmzRqdO3dOkrRs2TIVK1ZMFSpUyHPf1xUvXty2/5SUFMXFxUmSzp07pyZNmqhYsWIKDw/XK6+8kuMZ83fj+H8VFhamCRMmqGLFiipWrFi2bd9++60GDBigNm3aSLr28MX1vzMXFxfbDwe3MmnSJAUEBOj9999XZGSkTp069T/NiYKFS98w2l8fo5aunUGNHTtWFStWVJcuXTR06FCFhobq6tWratiwodatWyeLxaIRI0Zo5MiRCg0NVbFixVS2bFl5enqqWLFimjZtmoYPHy5nZ2cFBQXJ1dVVhQoVuq15KlasqBEjRqhXr15yd3dX1apVbfd1d3fXhx9+qMmTJys0NFQuLi5ycXFRv3791LFjx2z7udXsr732miZMmKDp06fLyclJAwcOVLly5XJdz4u7u7vmzJmj8ePHa8GCBbp69apefvll1a5dWzt37rzlfR999FFFRkZq/PjxSk9PV2ZmpsqXL6+oqCj5+vrK19dX4eHh6tmzpywWi0qUKKG5c+fmepXiZnr06KHXXntNrVq1Urly5VS/fn1JUokSJdSvXz+Fh4fL09NTLi4uOR6maNiw4d8+/l+1a9dO06dP15w5c3Jse/XVVzVgwAB5eXnJ29tbderU0YkTJyRde9ndtGnTcjxp8K82bdqkr7/+Wl9++aUeeOAB9ezZU0OHDtUnn3wiV1e+FSN3TtbbfVANuId8+umnqlatmmrWrKkrV66oa9euGjRokGrXrq05c+Zo0KBBKlSokH744Qf17dtXW7duva2z6pMnTyomJkb9+/eXs7Oz1q1bp/nz52c7swaAu4kf43Bfqly5ssaNGyeLxaLMzEy1bt3a9kQkNzc3Pfvss3J1dZWrq6vtDPV2lC5dWmfOnLGdMRcpUoSX1wCwK86oAQAwGE8mAwDAYMZd+rZYLEpLS5Obm9ttX44EAOBeZbValZmZqcKFC9/0iZDGhTotLe1vv+E/AAD3mocfflhFihTJsW5cqN3c3CRJmSXTJVcePgccpaZPPaVc4nW9gKMVdympn376yda/GxkXatvlblcroQYcyMPDQ65ZLnnfEMBd5e5y7W1uc3u4lyeTAQBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDXuWNrvaZo4bKpCg5/Rk1VCNaZ/pM6dPS9J6vBoZz3m1/Smf5ITU/J5cuD+sSdur8o/UEnbt+ywrX25fLVa1GmlyiWrqmntlloStTQfJ8Td4mrPnWdlZWn69OmKjo5WWlqaGjdurNGjR8vX19eeh4WdvfnSGB0/ekJvzhiuUmVKau6/FmjgM6/qo6/n6YO1c2WxZNlueyn9sgZ2fFUh9YNVupxfPk4N3D/S09I1+KUhysr68/+1ndviNOjFVzRu6hg93ryxtmzcqmEDR8inlK9atm6ej9Pi77LrGfWsWbMUHR2tiRMn6pNPPlFycrIGDRpkz0PCzn6KP6Kdm3ZpxLRhqt+srh4KrKi35ryp35J/09crNqq4bzH5lPKx/Vk0Z4lcXFwUMeW1/B4duG+MiRinMv6ls63FrvpagdWrqEevbqpQsbx69OqmGiFB2rx+Sz5NibvFbqG+cuWKoqKiNGTIEDVs2FDVq1fXtGnTtHfvXu3du9deh4WdnfzllCTpkXo1bGtehb1UrmI5fb99f7bbHvnhqGIWrtLQf70sTy9Ph84J3K82rP1GG9Z+o3GTx2Rb9/EtoZ8SftK2zdtltVr13bc7dfjQjwquWePmO8I9w26Xvg8fPqy0tDTVrVvXtlauXDn5+/tr9+7dqlWrlr0ODTvyLe0jSTpz+qwerFhO0rWHOM6ePqvivsWy3XbB5I8UXK+GGrSs7+gxgfvSud/OadiA4Zr63iQVLV4027bwvs9r13d71KlNV7m4uCgrK0v/fLmPnuv2TD5Ni7vFbmfUycnJkiQ/v+yPS5YqVcq2DfeeaiGBqhBQXpOGTdNvKam6fClD742fr/OpF3Q186rtdknHT+vb2O3q+XK3fJwWuL8MH/yGWrZpoWZPNs2x7bezqfrtzFmNjByhNVtXatyUMfpoXpQWf8wTyu51djujvnTpkpydneXm5pZt3d3dXRkZGfY6LOzMzd1N73w4Tm8PGK/Q4Gfk5u6mJzu2VIMW9eXq/uc/p9hlX6uUf0nVa1onH6cF7h9LP/lC8ft/0Pqda2+6fdiACFV/pLr6vdpXkhT0SHWl/nZOkaP+pc7PPycnJydHjou7yG6h9vT0lMVi0dWrV+Xq+udhrly5okKFCtnrsHCAfwRU0Ifr5uniuYtydXdTYW8v9WzZW/WaPGq7zdbYbWoZ1pxvDsBdsvTTL3T6VLJCHrr2w6/VapUk9egQrme7PaO9u75X+07tst2nZp0QTX9npi5e+K+K3XCpHPcOu136LlOmjCTp7Nmz2dbPnDmT43I47h1pf6SrX/uX9XPCLypaoqgKe3vp9InTOvrDz6r7/8+eL6Vd0k8Hj6h2o5r5PC1w/5j1/nR9s+drrduxRut2rNGimChJ0uR3J2rYyCEq419GCfGHs93nxx9+VHGf4kT6Hme3M+rAwEAVLlxYcXFxCgsLkyQlJibq1KlTqlOHy6H3qsLeXrJkZWn6qNkaMn6w0tMuacIrE/Vo41p6tPG1JwgePfSLsrIsqlT1oXyeFrh/lCmb/eVYHp4ekqTSZf3kW8pXvfq/oLeHj1NAYGU1bdlEe3bu1awpc/RKxOD8GBd3kd1C7e7urq5du2rSpEkqXry4fHx89Pbbb6tu3boKCQmx12HhAOPmjtbUETPU++kB8vB0V9O2TTRgVB/b9tQzqZKkosUfyK8RgQInvE8Pubu7acHsDzR2xHg9WKGcIsYMU88+PfJ7NPxNTtbrD3TYwdWrVzVlyhRFR0fr6tWrtncmK1GiRK73ycjIUHx8vDLLpEmudhsNwA3ql2qipPTj+T0GUOD4uJRWfHy8goKC5OHhkWO7Xd9C1NXVVREREYqIiLDnYQAAuG/xSzkAADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAgxFqAAAMRqgBADAYoQYAwGCEGgAAg91WqK9cuSJJOn78uDZt2iSLxWLXoQAAwDV5hnr27NmKiIhQUlKSunXrpo8++kgTJkxwxGwAABR4eYZ648aNmjBhglatWqV27drpo48+0t69ex0xGwAABV6eobZYLPL09NQ333yjJk2ayGKx6NKlS46YDQCAAi/PUD/22GNq27atMjMzVadOHXXv3l3Nmzd3xGwAABR4Tlar1ZrXjZKSkuTn5ycXFxcdPnxYgYGBdhsoIyND8fHxyiyTJrnmORqAu6R+qSZKSj+e32MABY6PS2nFx8crKChIHh4eObbneUZ94MABxcbGKisrSy+++KLCw8O1ZcsWuwwLAACyyzPUkZGRCggIUGxsrDw9PRUdHa0ZM2Y4YjYAAAq823oyWaNGjbRp0yY9+eSTKlOmjLKyshwxGwAABV6eoS5UqJA++OADfffdd2rWrJmioqJUuHBhR8wGAECBl2eop0yZovT0dM2aNUtFixZVSkqKpk2b5ojZAAAo8PIMtZ+fn5o3by6LxaJdu3apcePG2rp1qyNmAwCgwHPN6wYjR45UXFycLl68qIceekiHDx9WrVq19OyzzzpiPgAACrQ8z6i3b9+u1atXq1WrVho3bpyioqJ0+fJlR8wGAECBl2eoS5UqJTc3N1WqVEk//vijatSood9//90RswEAUODleenbz89Pc+fO1WOPPabJkydL+vPXXgIAAPvK84x6/PjxKleunIKDg/Xkk09q1apVGjNmjANGAwAAuZ5RJyUl2f67Zs2aSkpKUosWLdSiRQuHDAYAAG4R6u7du+d6JycnJ23YsMEuAwEAgD/lGuqNGzc6cg4AAHATt3yMetmyZTpw4IDt42nTpmnZsmV2HwoAAFyTa6gXLlyoxYsXy9vb27bWqFEjLVq0SIsWLXLIcAAAFHS5hvqLL77Qhx9+qIceesi2VrduXc2fP1+LFy92yHAAABR0uYba2dk529n0dSVKlJCzc56v6gIAAHdBrk8mc3FxUWpqqnx8fLKt//bbbw75fdQvtOirlJQUux8HwDXnzp1TWa8K+T0GUOBkZGTccvstX57Vu3dvvf7666pWrZo8PDx08OBBTZw4UV26dLnrg95o554dcvdwt/txAFxTokQJna/tld9jAAVOYGIRffLJJ7luzzXU7du3V0ZGhkaMGKHk5GRJ0oMPPqgXX3zRIaEGAAB5vNd3586d1blzZ50/f17Ozs4qWrSoo+YCAAC6jV/KIUnFixe39xwAAOAmePo2AAAGI9QAABgsz1BfvHhRI0eO1PPPP68LFy5oxIgRunjxoiNmAwCgwMsz1KNGjVKNGjV04cIFeXl5qVSpUho2bJgjZgMAoMDLM9SJiYnq3LmznJ2d5e7urldffdX2ci0AAGBfeYbaxcVFv//+u5ycnCRJx44d4y1EAQBwkDxfnjVo0CD16NFDp0+fVv/+/bVv3z5NmDDBEbMBAFDg5Rnqxx9/XEFBQTpw4ICysrI0duxY+fr6OmI2AAAKvDxDPXv27GwfJyQkSJIGDhxon4kAAIDNHT3YnJmZqY0bNyo1NdVe8wAAgL/I84z6xjPnAQMG6MUXX7TbQAAA4E93/PTttLQ0JSUl2WMWAABwgzzPqJs3b257aZbVatXFixf10ksv2X0wAABwG6GePn26fHx8JElOTk564IEH5O3tbffBAADAbYR6+PDh+uqrrxwxCwAAuEGeoQ4MDNSKFSsUHBwsT09P23rZsmXtOhgAALiNUO/fv1/79+/Ptubk5KQNGzbYbSgAAHBNrqGOjo5Whw4dtHHjRkfOAwAA/iLXl2dFRUU5cg4AAHAT/BosAAAMluul7yNHjqhFixY51q1WK49RAwDgILmGukKFCpo3b54jZwEAADfINdRubm7y9/d35CwAAOAGuT5GXatWLUfOAQAAbiLXUI8ePdqRcwAAgJvgWd8AABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwQg17lhi4il17dxdZUuWU2mfsurRtaeSkk5LkqxWqyZPnKLAytVU3NtHDeo21Fer1+bzxMC9r17VWspce0xNgh+zrT1R+3F9/59Ypa86qv1zv1brOs1uel93N3ft+886dWvR0VHj4i4i1LgjVqtVHdt11IXz57V2/Rqt27hWyaeT9Wz7ZyVJM/49U1MnTdPkaZO098BuhbUP03MdO+n7vd/n8+TAvcvLs5AWDp8hVxdX21rV8gFaOfYDfb5llWr2a6WY7bFaMWaBqlV4ONt9vQsVVvSYBXqkUjVHj427xGGhHj16tN58801HHQ52kpKSoiqBgXpv3hwFPxKs4EeCNfjVQfp+7z6dP39el9Iv6Z3J/1Jou7aq+FBFDX/jdRUpUkRbt3yb36MD96xpfd9S4tnT2dZe7tBL3yV8rwmLZunHkz9r9MdTtP3QHr3coZftNi1qNtK+/8TKr5ivo0fGXWT3UFutVs2YMUNLliyx96HgAKVLl9bCRR+rwj8qSLp2Gfz9ee+r9qO1Vbx4cY0YGaHwF3tKki5fvqy5781Tenq6Hm/SOD/HBu5ZT9VtrqfrtdDgd0dnW29co6427d+RbW3T/h1qXKOu7eM2dZvrg9glavBKe0eMCjtxzfsm/7uTJ0/qjTfe0JEjR1S2bFl7Hgr54LmOnbVq5SoVL15csRu+yrZtRXSMunbqJqvVqrfeHq2QmiH5MyRwD/N5oLgWDJmkF6e8pvN/XMy2rZxvGZ36LTnbWlJqih4s+ef32qFzxzpkTtiXXc+ov//+ez344IP68ssvVa5cOXseCvlg9JiR2rJ9sxo0fExPt2qrU6eSbNvq1K2j73Zv18Qp72hC5L/0wYIP83FS4N4095WJ+nLHesXu3pRjm5dnIV3OzMi2lpGZIU93DwdNB0ex6xl1u3bt1K5dO3seAvmoRnANSVLUoo8V8I8q+jTqU70+Ypgkyd+/rPz9yyr4kWAdPXJU/546XS++9EJ+jgvcU55/4lnVrFxdwX2euOn2SxmX5eHmnm3Nw81DaZfTHTEeHIhnfeOOpKSkaOmSz7OteXl5qeJDFZWUlKSvVq/Vz0d/zra9elB1Jf3lbBtA3sKf7KRyvmWUvPR7/b7yR/34wWZJ0lcTFuq9l/+lk2eTVKZEqWz3Kevjl+NyOO59hBp35MTxk+rZLVx7du+1rV28eFFHfjqiwKqBeiPiTc2a+W62++zetVuB1QIdPSpwT+v+zmBVe6m5Qv7ZSiH/bKVWI7pJkl6aNkyjP56ib+N3qUlw/Wz3aRbSQFsO7syPcWFHdr30jftP7UdrqWGjhurft79mvzdbbm6uGvXGaPmW9FX357vJw8NDLw98RbVr11SDhg0Us2KlFi9aos+X86x/4E4kpWY/M7585drj0ad+S9bZC6mateJD7ZmzRmOeH6rPvlmhrs07qF5gTfWbOSI/xoUdEWrcEWdnZy3+YpFGvP6Gngl7RpcvZ6jlky21bmOsvL299UKvcF29elWTJ07VieMv6+EqAfrs80Vq3aZ1fo8O3Ffijx1WhzEvaVLvNzW8cz8dPvmzQkeF6/CJo/k9Gu4yQo075uvrq/kfzMt1e+++L6l335ccOBFw/zv122k5PZH91TNr4jZqTdzG27r/jffFvYPHqAEAMJjDzqgXLlzoqEMBAHDf4IwaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAxGqAEAMBihBgDAYIQaAACDEWoAAAzmmt8D3MhqtUqSMq9k5vMkQMHi5+cnz+KF8nsMoMDxvVxY0p/9u5GTNbct+eT333/XTz/9lN9jAADgUA8//LCKFCmSY924UFssFqWlpcnNzU1OTk75PQ4AAHZltVqVmZmpwoULy9k55yPSxoUaAAD8iSeTAQBgMEINAIDBCDUAAAYj1AAAGIxQAwBgMEINAIDBCDUAAAYz7i1Ece/Yt2+fdu/ereTkZGVkZKhQoULy8/NTnTp1FBwcnN/jAcB9gTc8wR1LTU3V4MGDtWfPHvn7+6tUqVJyd3fXlStXdObMGZ06dUr16tXTjBkzVKxYsfweFwDuaYQad2zw4MFKSUnRtGnT5O/vn2N7YmKihg4dqnLlymnq1Kn5MCEA3D8INe5YzZo1tWjRIlWtWjXX2/zwww8KDw/Xrl27HDgZcP8bNWrUbd923LhxdpwEjsJj1LhjhQsX1oULF255m3PnzsnVlX9ewN3m5OSkpUuXqmzZsje9ovXX2+H+wHdS3LEOHTpo+PDhGjJkiOrWrSs/Pz+5uLjIYrHo7NmziouL08SJE9W+ffv8HhW474wdO1Y+Pj5asmSJpk+fLh8fn/weCXbGpW/cMYvFopkzZ2rhwoVKT0+XJDk7O8tisUiSvLy81LVrV73yyitycXHJz1GB+9bzzz+v8uXLKzIyMr9HgZ0RavzPMjMzlZCQoJSUFF26dEmenp4qXbq0AgMD5e7unt/jAfe1kydPat++fQoNDc3vUWBnhBoAAIPxzmQAABiMUAMAYDBCDThYYmKigoKCFBYWpvbt2+vpp5/WCy+8oOTk5P95n8uXL1dERIQkqXfv3kpJScn1tjNnztTu3bvvaP9VqlS56fovv/yif/7znwoNDVVoaKiGDh2qc+fOSZJmzZqlWbNm3dFxAOREqIF8UKpUKcXExGjFihVavXq1qlSpokmTJt2Vfc+fP19+fn65bt+1a5eysrL+9nFSUlL0/PPPq1OnTvryyy+1cuVKBQQEaODAgX973wD+xOuoAQPUq1dP06ZNkyQ1b95cwcHBSkhI0KJFi7R161Z9/PHHslgsql69ut566y15eHhoxYoVeu+99+Tt7S1/f395eXnZ7h8VFaWSJUvq7bff1p49e+Tm5qb+/fvrypUrio+P18iRIzV79mx5enpqzJgxunDhgjw9PTVq1ChVq1ZNiYmJGjZsmNLT0/XII4/cdObPPvtM9evXV/PmzSVde4ON3r17q1y5crp69Wq2237yySeKiYnRpUuX5ObmpqlTp+qhhx7SxIkTtW3bNjk7O6tly5YaOHCgduzYocmTJ0uSihYtqqlTp6pEiRL2+tIDxuOMGshnmZmZio2NVUhIiG3t8ccfV2xsrM6dO6elS5dq8eLFiomJkY+Pj95//32lpKRoypQp+vTTT7VkyRKlpaXl2O/117l/9dVX+vDDD/Xuu++qTZs2CgoKUmRkpKpUqaLhw4dr2LBhio6O1rhx4/Tqq69KuvbWkx07dlRMTIxq1ap107kTEhJUvXr1bGsuLi5q27Zttnel++OPP7R+/XotXLhQq1atUtOmTfXpp5/q1KlT2rJli1auXKnPPvtMR48eVUZGhubMmaMxY8Zo+fLlatCggQ4dOnQXvsrAvYszaiAfnDlzRmFhYZKkK1euKDg4WEOHDrVtv34Wu3PnTh0/flydOnWSdC3q1apV0/fff6+aNWvK19dXkhQaGqrvvvsu2zF27dqlTp06ydnZWSVLltTq1auzbU9LS1N8fLxGjBhhW0tPT9f58+cVFxdn+4Uq7dq108iRI3N8Dk5OTrf1enlvb29NnTpVq1ev1rFjx7R161ZVrVpVfn5+8vDwUJcuXdSsWTO99tpr8vDwUIsWLTRw4EC1bNlSLVq0UMOGDfM8BnA/I9RAPrj+GHVuPDw8JElZWVl66qmnbKFMS0tTVlaWduzYob++BcLN3lfd1dU12/s9Hz9+XGXKlLF9bLFY5O7unm2O5ORk268mvb5/JycnOTvnvPgWFBSk+Pj4bGsWi0WDBw/WmDFjbGunT59Wjx491L17dz3++OPy9fVVQkKCXF1d9fnnnysuLk5btmxRly5dtHDhQoWHh6tZs2b65ptvNHnyZB04cED9+vXL9WsF3O+49A0YrF69evr666+Vmpoqq9WqMWPG6OOPP1bt2rW1b98+paSkyGKxaM2aNTnuW6dOHa1Zs0ZWq1Wpqanq3r27rly5IhcXF2VlZalIkSL6xz/+YQv1tm3b1K1bN0lSgwYNtHLlSknSunXrlJGRkWP/nTt31ubNm7V582ZJ18I+Z84cpaam2s70JengwYOqUKGCwsPDVaNGDa1fv15ZWVk6dOiQunfvrjp16mj48OGqVKmSfv31Vz333HNKS0tTeHi4wsPDufSNAo8zasBggYGBGjhwoHr27CmLxaKqVauqT58+8vDw0MiRIxUeHq5ChQqpcuXKOe7btWtXRUZGql27dpKu/XpEb29vNW7cWG+99ZYmTpyoyZMna8yYMVqwYIHc3Nz073//W05OTho9erSGDRumJUuWKCgoSIULF86x/5IlS2r+/PmaNGmSpkyZoqysLFWrVk3vvvtutts1bNhQn332mdq0aSOr1ao6deroyJEjqlatmkJCQtS2bVsVKlRItWrV0uOPP65ChQopIiJCrq6u8vLy4r2sUeDxFqIAABiMS98AABiMUAMAYDBCDQCAwQg1AAAGI9QAABiMUAMAYDBCDQCAwf4fuofRKvwvEMoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x396 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_model(best_model, plot=\"confusion_matrix\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAggAAAGACAYAAADI95pLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAA6y0lEQVR4nO3deZzNZf/H8fdZZjELQ4aoGTEZZGJsFUJkz9i7DUV1K5KlNAotEhoTfsmSklay3ky2so0lZd8zlaWyDDdGxhgzY7Zzvr8/3HMyfcMUM5OZ1/Px8Hg43+U6n2sux3nP9b3O91gMwzAEAABwBWtBFwAAAP55CAgAAMCEgAAAAEwICAAAwISAAAAATAgIAADAhICAHE6cOKFq1aqpQ4cO6tChg8LCwvToo49q165defJ8a9eu1ZgxY26ojaSkJI0ZM0ZhYWHq0KGDOnbsqP/85z+u/c2aNdP+/ftvtNQcJk2apMWLF0uSoqOj9dBDD6l37945tt+I3bt3q3fv3q4x6NOnjw4dOiRJ2rZtm9q1a3fDz/FHHTp0UFJSkhwOh/r166dWrVrpiy++cG3/OzZs2KBJkyZJujljfaVmzZqpVatWrjFv27at2rVrp40bN96057iWixcvqlevXvnyXECBMIArxMXFGaGhoTm2ffXVV0aLFi0KqKJrS0tLM8LCwowPP/zQyMzMNAzDME6cOGE0b97cWLBggWEYhtG0aVPj+++/z7MaevbsaSxevPimtbd9+3ajSZMmxv79+13blixZYtx3333GuXPnjK1btxqPPPLITXu+Pzp58qQREhJiZGVl3XBbkydPNt58882bUJXZn43rihUrjIYNG+bJ8/3Rn71WgMLEXtABBf98iYmJ8vf3lyQ5nU5FRkZq3759SklJkWEYGjNmjOrUqaOEhAQNHz5cx48fl5+fn/z9/VW5cmUNHDhQ33zzjSZMmCCr1apq1app8+bNmjNnjrZv365Vq1Zp+vTp6tmzp0JDQ7V7926dOnVK9evX1+jRo2W1WhUdHa0PP/xQnp6eeuCBBzRz5kz9+OOP+vrrr+Xl5aVnnnnGVe8dd9yhd999V5mZmTn6ca3ad+7cqaioKDmdTklS37591apVq6tuHzZsmCpXrqwzZ85o//79OnHihM6fP68DBw6ocuXK6t27t3755Re99dZbSkxMlMPhUM+ePdW1a1dt27ZNb731lry8vJSSkqJFixbJ3d3dVefkyZP13HPPKSQkxLWtffv28vDwkMPhyNGnI0eOaNSoUUpJSdHZs2dVtWpVvfvuu/Lw8NDkyZO1Zs0aubm5qWTJkho7dqzKlClz1e1VqlTRhg0b9PTTTysrK0udO3fWlClT1KJFC23ZskWlSpXS9OnT9eWXX8put6tChQqKioqSzWbTyJEjdezYMSUmJsrb21sTJkzQxYsXNW/ePDkcDvn6+qpChQqusT59+rRGjhypkydPyjAMdezYUU8//bROnDihJ598Uk2aNNG+ffuUlJSkl156SS1atLjuv1PDMHTixAmVKFHCte0///mP5s6dK6fTKT8/P73++usKCgrSsGHD5OHhoQMHDujcuXNq2LChXnvtNbm5uWnnzp0aN26cLl26JDc3N73wwgtq3LixoqOjtXDhQl26dEk+Pj6SpLS0NHXo0EHR0dGy2Wy5ej0Bt4yCzSf4p4mLizOqVq1qtG/f3mjfvr3x0EMPGdWrVzc2bNhgGIZh7N692xg4cKDhcDgMwzCM6dOnG3379jUMwzAGDx5sjBs3zjAMwzhz5ozRsGFDY/LkyUZCQoJx3333GT/99JNhGIYRHR1tBAcHG3FxccaiRYuMPn36GIZhGI8//rgxaNAgw+FwGBcvXjQefPBBY8uWLcbhw4eN+vXrG6dOnTIMwzCmTJliBAcHG4ZhGKNGjTLefvvta/Yp+zfNa9Xeq1cvY/ny5YZhGMZPP/1kjBw58prbhw4danz00UeuulesWJFje2ZmptG2bVsjNjbWMAzDSEpKMtq0aWPs2bPH2Lp1q1G1alXjxIkTf1pvaGiocfjw4av258oZhKioKNfsRUZGhtGuXTtj5cqVxn//+1+jdu3aRnp6umEYhvHxxx8ba9asuep2wzCM4OBg49y5c6bfjLO3x8TEGC1btjQSExMNwzCMyMhIY9q0acaKFSuM0aNHu45//fXXjVGjRhmGkXMG4cqxfuyxx4xPPvnE9bMJCwszli9fbsTFxRnBwcHGunXrDMMwjJUrVxoPPfTQn/4cmjZtarRs2dIICwszGjVqZDRq1MgYPny4cfz4ccMwDGPbtm1Gjx49jNTUVMMwDOPbb781Wrdu7Rqnjh07GsnJyUZ6errx2GOPGbNmzTISEhKM+vXrG3v37jUMwzAOHTpk3Hfffcbx48eNRYsWGfXq1TMuXrxoGAYzCCj8mEGAiaenp5YsWeJ6vHnzZvXv319Lly5VrVq1VKJECc2bN09xcXHatm2bvL29JUnffPONvvzyS0lSmTJl1Lp1a0nSzp07FRQUpKpVq0qSOnXqdNVr0U2bNpXVapWPj48qVKigCxcu6MCBA2rYsKFuv/12SdLjjz+uKVOmSJIsFouMXN4t/Fq1t2nTRqNGjdK6devUoEEDvfjii9fcfj1Hjx7V8ePH9corr7i2paWl6ccff1RQUJDKlSunO+6440/PtVqtrhmL63nppZe0adMmzZgxQ0ePHlV8fLxSU1NVtmxZVa1aVZ06dVLjxo3VuHFj1a9fX06n80+358aWLVvUunVr12/ow4cPd+0LCAjQrFmzdOzYMW3fvl21atW6ajupqanavXu3PvnkE0mSr6+vOnfurI0bN6pmzZpyc3NTkyZNJEn33HOPEhMTr9rWhAkTdO+99youLk5PPfWUqlWrpoCAAEmX1z8cO3ZM4eHhruOTkpJc7XXq1Mk1/h06dNDatWsVEBCgwMBA1axZU5JUuXJl1a5dW9u3b5fFYlGVKlVcswdAYcciRVxXgwYNFBgYqP3792vDhg3q27evJOnhhx9W9+7dXcfZ7fYcb9ZW6+V/XjabzfQmnr3vjzw9PV1/z37z/+P5V07lhoaGau/evaZ21q5dq7fffjvHtmvVHh4erqVLl6phw4b67rvv1L59e6Wnp191+/VkT6svWbLE9WfBggXq0qWLJMnLy+uq54aGhmrfvn2m7W+++aY2b96cY9uLL76oBQsW6I477tCTTz6p6tWryzAMWa1WffHFFxo7dqz8/PwUGRmpcePGXXV7bthsNlksFtfjpKQknThxQnPmzNGrr74qT09PhYWFqV27dtcMbU6n07Tf6XQqKytLkuTm5ub693Hl811LQECAxo0bp7ffflvff/+9q80OHTq4fv5ffvmlFi1a5Ao4V/47yv6ZORwO03MahuGq7VrjBhQ2BARc15EjR3Ty5ElVq1ZNmzZtUtOmTdWjRw+FhIQoJibGdV28SZMmWrhwoSTp/PnziomJkcViUe3atXX06FEdOHBAkrRq1SolJSXl+j//Bx98UFu2bNGZM2ckKccnFFq2bKnk5GTNmDHDVUdcXJyioqIUFBSUo51r1R4eHq6ffvpJnTt31ujRo5WUlKSzZ89edfv1VKxYMcdMzKlTp9SuXTvFxsZe99x+/fpp6tSpOY6Njo7WqlWrFBwcnOPY7777Tv3791fbtm0lSfv27ZPD4dCBAwfUrl07BQUFqW/fvnryySe1f//+q27PjQYNGmjNmjVKTk6WJE2ZMkWfffaZvvvuO3Xq1EmPPvqoKlasqHXr1rl+rjabzfXmms3Hx0c1a9bU7NmzJV3+NMDixYvVoEGDXNVxNbVr11bHjh01cuRIOZ1OPfjgg/rqq68UHx8vSZo7d66eeOIJ1/ErVqxQRkaG0tPT9eWXX6pp06YKDQ3Vr7/+6goZhw8f1o4dO3TfffeZns9ut8vhcOR6Bgu41XCJASbZC6+yOZ1OjRo1ShUrVlR4eLgiIiIUFhamrKwsNWzYUKtXr5bT6dTw4cP12muvKSwsTH5+fipfvrw8PT3l5+end955R0OHDpXValVISIjsdruKFSuWq3oqVqyo4cOHq3fv3nJ3d1e1atVc57q7u+vTTz/V+PHjFRYWJpvNJpvNpn79+qlz58452rlW7UOGDFFkZKTeffddWSwWDRgwQHfeeedVt1+Pu7u7pk2bprfeeksfffSRsrKy9Pzzz6tOnTratm3bNc+tW7euxowZo7feekupqanKzMxUYGCgZs6cqdKlS+uXX35xHTt48GD1799fXl5e8vHxUb169XT8+HE9+uijatOmjbp06SIvLy95enrqtddeU9WqVf90e240adJEP//8s2vm5e6779bo0aN14MABjRgxwhUOQ0NDXR/JfOCBBzRkyBCNHj1a1atXd7U1YcIEjRo1StHR0crIyFBYWJg6d+6skydP5qqWq3nxxRfVpk0bLViwQOHh4XrmmWf073//WxaLRT4+Ppo6daormHp6eqpHjx5KSkpSq1at1KVLF1mtVk2aNEmjR49WWlqaLBaLxo4dq4oVK2rPnj05nsvf3181atTQI488otmzZ6tkyZI3VDvwT2MxiL+4SWbPnq177rlHtWrVUkZGhnr06KGBAweqTp06mjZtmgYOHKhixYrphx9+UN++ffXtt9/mahYhLi5OS5Ys0XPPPSer1arVq1drxowZOWYSgL8i+1MovXv3LuhSgH8sZhBw02T/Rul0OpWZmanWrVu7Fpu5ubmpa9eustvtstvtrt/Ic+P2229XfHy8a4bA19dXkZGRedkVACjymEEAAAAmLFIEAKCQ2Ldvn3r27Gnavm7dOnXp0kXdunXTggULctUWlxgAACgEZsyYoaVLl5oWgGdmZmrs2LFauHChihUrpu7du6tp06auO+RezU0NCE6nUykpKXJzc8v19WUAAAqKYRjKzMyUt7f3Ve/PcjNlZWWZbpmeWzabTXb71d+2AwMDNWXKFL388ss5tv/yyy8KDAx03QMk+/bybdq0uebz3dSAkJKS4vp4EwAAt4rg4GD5+vrm6XNkZWVpx8ZNci/x9+7GabPZXB8T/zOtWrXSiRMnTNuTk5Nz9M3b29t1P5NruakBwc3NTZK0qfdIpcUn3MymkQeeP7JOkyo2K+gykAuM1a3l+SPrdPqBG7vxE/JHqY0bdOjQIdf7V15yOBxyL+GjTb3fVFr8ub90rmeZ29Tw4zfkcDiuOYvwZ3x8fJSSkuJ6nJKSkqswdFMDQvZlhbT4BF069dvNbBp5wMPDg3G6RTBWtxYPDw/Z/ncHR/yzZX+Tan5eFk+LP5evr+egoCDXt616eXlp586duboHCIsUAQAohJYtW6bU1FR169ZNw4YNU+/evWUYhrp06aKyZcte93wCAgAAhcSdd97p+hhjWFiYa3uzZs3UrNlfu0zJfRAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACY2Au6AAAAipJ6koy/eI4lLwq5DmYQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAAAoBJxOp0aMGKFu3bqpZ8+eOnbsWI79S5cuVadOndSlSxfNmTPnuu3Z86pQAACQf2JiYpSRkaH58+dr7969ioqK0vvvv+/aP27cOC1fvlxeXl565JFH9Mgjj6hEiRJXbY+AAABAIbBr1y41atRIkhQaGqrY2Ngc+6tUqaKLFy/KbrfLMAxZLJZrtkdAAACgEEhOTpaPj4/rsc1mU1ZWluz2y2/1lStXVpcuXVSsWDG1aNFCxYsXv2Z7rEEAAKAQ8PHxUUpKiuux0+l0hYMDBw5ow4YNWrt2rdatW6eEhAStWLHimu0REAAAKARq166tjRs3SpL27t2r4OBg1z5fX195enrKw8NDNptNpUqVUlJS0jXb4xIDAACFQIsWLbRp0yaFh4fLMAxFRkZq2bJlSk1NVbdu3dStWzf16NFDbm5uCgwMVKdOna7ZHgEBAIBCwGq1atSoUTm2BQUFuf7evXt3de/ePfft3bTKAABAoUFAAAAAJlxiAAAgH1UqmyabJfUvneMok6bf8qieq2EGAQAAmBAQAACACQEBAACYFLmAULltEz27b6n6H1iprgsmyd3X23RM1Y7N9ey+peq7Z7F6rf1cJSsFSJI8S5ZQl3kT1f/ASvXZFa37BjzuOueuh+7XMzsWqe/eJeq9Zb7K17s33/pUmOVmvO4b8Lj6H1ipvnsWq/Oc/5NnyctfPmL39FD7jyPVb/8y9YtdrvYfR8ru6ZHjXL+77tTL57apXJ2QfOlPYXYjY2WxWtVq4ivq/9MKDTy8WnX6hrvOKV0tSE99O0d99yxWn91fKqjlg/nWp6LIs1UrlT/4kyTJ4uurUh9+oLJrY1R2/Vr5PtfPdLwtIEDlY/fLrUaN/C4VeSxXAWHfvn3q2bNnXteS57xKl1SHT8dqQZeBeq9qayX+GqfmUUNyHGP39FCnL8ZrfucBml6row4tW6fWk1+TJLWaOFyZyamadk9bffRAN93dprEqP/KQrG5u6jp/opY985qmh3bQxjHvq9Os8QXRxUIlN+N110P3q+HQZzTz4Sc0vVZH/fz1RoV9ePlzwI1e7Ser3ab3a7TXBzXay17MQw8O7+s61+bhrk5fjJfN3S1f+1UY3ehY1ekbrtuC79K0kHaaUa+rHnjhCVfIfmTaG9rzySJNr9VRS//9iroueFcWmy3f+1gU2CveJb/XX5P+9yU+xV8aIsepUzrzcHPFt20n71495V6n9u8neHio1JRJEq+hQum6AWHGjBl67bXXlJ6enh/15Kmglg/q5I79Svj58ndk73h/ru59LCzHMRabTRaLRZ4lfCVJ7j7eykq73Pfydapr36wlMpxOOTMzdfirDbqnays5MzP1zh2NdXrv5dRdslKALp07n489K5xyM17l6lTXrzGbdfHkGUnST9GrFRzWTFY3Nx3buEMbx7wvGYYMp1On9/ykEhXKu8595L03tO+zaKX+xljdqBsdq6qdmmvvp9EyHA6lJSYpdt5XqvF4e0mXX5PFSl7+Uhl3399fj7i5LJ6eKjV5shLf/P1GOxdGvKELo8ZIkqxly8ri7i5n0kXX/pJvjVHqgv/ImZCQ7/Ui7103IAQGBmrKlCn5UUueKx5wu5LiTrseJ504Lc8SvjmmQjNTUrX82Tf0783z9OLJb1VvwGOKGTpBknRy2/eq2bODrHa73Ly9VK1LK/mU85ckObOy5F3mNg0+sVEtxr+sTeM+yt/OFUK5Ga+T2/apYrMHVCLw8ht/6FOdZfdwl9dtfvp1zSYlHD4qSSoRWF4PvPCEfvzPSklSrd5dZXWza/dH/8m/DhViNzpWJQLK6ULcqRznF7/zdknS1/1H6cHhfTU47hv1ivlUX/UbKcPhyKeeFR1+46KU/MVsZf70U84dDodKTp6k29euUfqWrcr65RdJklf3cMnNrpQ5cwugWuSH6waEVq1aub4N6lZnsVolwzBtNxxO19/LhASryYj+mnZPW71zRyN9+9YH+teiywFpVUSUDMNQ3z1fKnzxe/p1zSY5MjJd56bEn9PEOxvr4/rd1OHTsSpV+a4871NhlpvxOv7dLn3z5nvq9uVUPbNjkQynodRz53OMS7na1fXUt7O1feoXOvzVBt1e6x7Vfba7lj/7Rr70oyi40bGyWC05zrdYLHI6nLJ5uKvr/Ila/OQwTQxook8bP65200e5wgNuDu8neklZDqXOn/+n+88Pel7/vbemrH5+Kj74BbmFhMinZ08lDh2ez5UiPxWpRYoXjp+ST/kyrsfF7yirSwmJyky95NoW1OpBHd+0W+d/jZMk7XhvtsqEVFax20rKo7iP1rw8Xu/fG6ZZLZ6SLBad//m4PIr7qGrH5q42Tu/5UWf2HVDZe3//Ji38dbkZL3cfbx39Zrs+rNNZM+p10cElMZKkSwmJkqTq3dqq55pPFDPs//Td2OmSpJq9OsqjuLd6b56nvnsWy7d8GXWePUHBYc3yr3OFzI2O1YXjp+R7xfm+5cvo4onTKhMSLDcvTx3+aoOky7MQZ384rDvur5k/HSsivB99VO41a6rM6pUqPetzWTw9VWb1Snk92lXWsmUlSUZqqlKXLJHbvSHyerSLLL4+8l+6WGVWr5StbFmVmjpZni1aFHBPcDMVqYDwy+rvdOcDNVXq7gqSpLrPhuvAkrU5jjm1+0fd1aSevMvcJunyJxoSj5zQpXPnVffZcDUdNUiS5F3mNtV++lHtn7NcTodT7T+JVECDy4t3/O+5W6WrVtKJbfvysXeFT27Gy7d8GT25YZZrKrvRq/0UO/crSVJwu6ZqM/k1zWrZW7Fzl7vOWTU4UlOrtNb0Wh01vVZHXfxvvKIfG6JDy9blU88Knxsdq4NL1ir0311ksdnkUcJX1cMf0YHFMUr4+Zg8S/jqzvq1JF1e3+N/z906vefHfOxd4RffLuzyQsSWrfVbzydkpKUpvmVredx/v4q/+MLlg9zdVaxdO6Vv2qwLb7ypM42aKL5la8W3bC3HmTNKGDBIaWvWFGg/cHMVjmsHuZR6NkFLnhquRxdOls3dTed/Oa4vew1VuTohav/RGE2v1VFH12/V5vEf64kNs+TIyNSlhAua1+E5SdJ3Yz9Up1nj1G//Msli0foRk/XfnfslSfM79lerd1+Rzc2urPQMLeoxxLUYC39Pbsbr3KEj+i7qQz297T+yWK2K+26Xvh5weZFViwlDJYtF7T8a42ozbtNu137cPDc6Vjven6uSQYF6dt8S2dzdtGv6fB3buEOSNL/TALWe9Krsnu5yZjm0rM/rrhk+5K3EUaNVMmqsyq7932zPypVK/ujjAq4K+cViGH9y4fBvSk9PV2xsrNaGDdKlU/l912j8VW8YB/WmpUpBl4FcYKxuLW8YB3XijoCCLgO54P/rz4qNjVVISIg8PDyuf8INyH6PLP10H9ni4//SuY4yZfTbRx/mS53ZitQlBgAAkDsEBAAAYEJAAAAAJgQEAABgQkAAAAAmBAQAAGBCQAAAACYEBAAAYEJAAAAAJgQEAABgQkAAAAAmBAQAAGBCQAAAACYEBAAAYEJAAAAAJgQEAABgQkAAAAAmBAQAAGBCQAAAACYEBAAAYEJAAAAAJgQEAABgYi/oAgAAKEr87/WRx4W0v3ROegkf/ZZH9VwNMwgAAMCEgAAAAEwICAAAwISAAAAATAgIAADAhIAAAABMCAgAAMCEgAAAAEwICAAAwISAAAAATAgIAADAhIAAAABMCAgAAMCEgAAAAEwICAAAwISAAAAATOwFXQAAALhxTqdTI0eO1MGDB+Xu7q4xY8aoQoUKrv3ff/+9oqKiZBiG/P39NX78eHl4eFy1PWYQAAAoBGJiYpSRkaH58+crIiJCUVFRrn2GYej111/X2LFjNXfuXDVq1EgnT568ZnvMIAAAUAjs2rVLjRo1kiSFhoYqNjbWte/IkSPy8/PT559/rkOHDqlJkyaqVKnSNdtjBgEAgEIgOTlZPj4+rsc2m01ZWVmSpPPnz2vPnj3q0aOHPv30U23dulVbtmy5ZnsEBAAACgEfHx+lpKS4HjudTtntly8U+Pn5qUKFCrr77rvl5uamRo0a5Zhh+DMEBAAACoHatWtr48aNkqS9e/cqODjYtS8gIEApKSk6duyYJGnnzp2qXLnyNdtjDQIAAIVAixYttGnTJoWHh8swDEVGRmrZsmVKTU1Vt27d9NZbbykiIkKGYahWrVp66KGHrtkeAQEAgELAarVq1KhRObYFBQW5/l6/fn0tXLgw9+3dtMoAAEChQUAAAAAmBAQAAGBCQAAAACYEBAAAYEJAAAAAJgQEAABgQkAAAAAmBAQAAGBCQAAAACbcahkAgHxkud9PlnTnXzvHwy9virkGZhAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgIk9Lxp9/sg6eXh45EXTuMneMA4WdAnIJcbq1nLnybiCLgG5kJ6eXtAl/GMxgwAAAEzyZAbh9AMNZIuPz4umcRPdeTJOJ+4IKOgykAuM1a3lzpNxMg4NLegykBsVRhV0Bf9YzCAAAAATAgIAADAhIAAAABMCAgAAMCEgAAAAEwICAAAwISAAAAATAgIAADAhIAAAABMCAgAAMCEgAAAAEwICAAAwISAAAAATAgIAADAhIAAAABMCAgAAMLEXdAEAABQllorFZXE4/9o5tuJ5VM3VMYMAAABMCAgAAMCEgAAAAEwICAAAwISAAAAATAgIAADAhIAAAABMCAgAAMCEgAAAAEwICAAAFAJOp1MjRoxQt27d1LNnTx07duxPj3v99dc1YcKE67ZHQAAAoBCIiYlRRkaG5s+fr4iICEVFRZmOmTdvng4dOpSr9ggIAAAUArt27VKjRo0kSaGhoYqNjc2xf8+ePdq3b5+6deuWq/YICAAAFALJycny8fFxPbbZbMrKypIkxcfHa+rUqRoxYkSu2+PbHAEAKAR8fHyUkpLieux0OmW3X36bX7lypc6fP68+ffro7NmzSktLU6VKldS5c+ertkdAAACgEKhdu7bWr1+vtm3bau/evQoODnbt69Wrl3r16iVJio6O1q+//nrNcCAREAAAKBRatGihTZs2KTw8XIZhKDIyUsuWLVNqamqu1x1ciYAAAEAhYLVaNWrUqBzbgoKCTMddb+bA1d5NqQoAABQqBAQAAGBCQAAAACYEBAAAYEJAAAAAJgQEAABgQkAAAAAmBAQAAGBCQAAAACYEBAAAYEJAAAAAJgQEAABgQkAAAAAmBAQAAGBCQAAAACYEBAAAYEJAAAAAJvaCLgAAgCKl0t2SNfWvneP0klLyppyrYQYBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAICJvaALyG+eDzdT8WHDZPFwV+ZPP+l8xEsykpNd+726dpFPn2dcj62+vrKVK6dTde+T87ffXNtvm/GhHGfOKPG11yVJFj8/lRw9SvbgyrJ4euri5ClKXRSdfx0rpK43XpJkr1pVJUePkqW4r+Rw6vzQYcrcv1+S5P1EL3l3D5fF01MZ3+/X+SEvSRkZ8mhQXyVGvC7Z7HImnteFN0Yq88efCqKLhcYNvbYSElTijdfl+dBDstjsujh9ulJmfXG53RbNVWriRGX996Tr3LOdushIScm/zhURhmHoqWFfKyTYX0N635djX5cBX6pcGR9NHdFCkrRs3c96ctjXCixX3HXMxtnd5evjka81I+9cdwbB6XRqxIgR6tatm3r27Kljx47lR115wlqqlEq+839K6NNHZxo/JMex4yrxyvAcx6QuXKT4lq0v/2nbTo6zZ5X42us5woFPv2flfn/OF0+pie/IceqU4lu10W/h3eU36k3Zyt2eL/0qrHIzXhZPT/nP+UIX339f8a3aKOndSSo1dbIkybNNa/k89aTOhvfQmaYPy+LpKd9nnpbF11e3zfhQF8a8pfgWLZU47BWV+uB9yd29ILpZKNzoa8u75+OyV6ykM82a68wj7eTzdG+5hYZKktzr1NHF6dN/P7dla8JBHvjpl3Nq/sR8LVx1yLRv3Ixt+nbniRzbNu85qYh/19OeJU+6/hAOCpfrBoSYmBhlZGRo/vz5ioiIUFRUVH7UlSc8mjRWxr59yjpyVJKUPHOWvDp1vOrxvv2fk/O3c0r5YvbvbdR/QJ5NH3L9diNdnj3wbNRISe9MlCQ5Tp1WfFh7Oc8n5kEvio7cjJdHkybKOnZMaevWS5LSVq9WwrP9JEneXbvq4vQPZSQmSoahxGHDlbIoWvaKFeW8eFHp322SJGX98ouMi8nyqFM7v7pW6Nzoa6tY61ZKXbBAcjhkXLigS0uWyqtzp8tt160rj4YNVGbNKvlHL5L7/ffndXeKpPdm71bvR2vo0dZVcmzfsO24Vn17RH3DQ3Ns37LnpNZvPa7Q9p+qcY852rgjLh+rRX64bkDYtWuXGjVqJEkKDQ1VbGxsnheVV+zly8vx3/+6HjtOnZK1eHFZfHxMx1pLlpRvn2eUOPLN37eVLasSo95UwoBBksPxe7t33SVH/Bn59O0j/8XRKvP1V3ILuVdGWlredqiQy8142StVlOPsWZWcMF5lvv5KpefNkWx21z5b6dIq/cUslVmzWsUjXpRx4YKyfv1VFi8veTRuLElyq1lT9irBspYpm78dLERu9LVl+8P5WadOyV6unCTJef68UmZ9ofgWrXRhbJRu+3gGs3N5YOqIFuoRdk+Obf89c1EvvLVWX0xoJ5vNkmPfbX7F1Dc8VHuWPKnIFxur84AvdeL0xfwsGXnsugEhOTlZPle8yG02m7KysvK0qDxjtUrGn2y/4s0+m/fjj+nS6jVyHD9+eYPdrtvem6oLI9+UMz4+x7EWu132ChVkXLyosx0769xz/eU3coTc7r03DzpRhORivCxubvJs1kwps2crvu0jSv7kM5We9fnlywVubvJo3Ejnnu2n+LaPyOrnp+LDXpaRnKxz/35axQcOUJk1q+TdtYvSN22SMjPyr2+FzY28tlzn/96AxWKR4bx87rln+ujSV19LkjJ27FDGzp3yaNT4ppYPs8xMh3pELNM7w5upXBlz0Fs0tZO6tq4ii8WiB+veqQa17tCaTUfzv1DkmesuUvTx8VHKFdf7nE6n7PZbc22j4+RJudeq5Xpsu/12Oc8nyrh0yXRssfZhSnz9Dddj95o1ZKsQqBJvjLh8rr+/ZLPJ4uGhpMlTJEkp8xdcfp6jR5W+Y4fca4W6Fsvhr8vNeDlOn1HW4cPK2LNX0uVLDJowTvbAQDlPn9Glr1e4FsqlRker+AsvSBaLjNQUnX30X652yn77jbKOHs2PbhVKN/Layj7fWvb3WQFr2bJynDotS/Hi8nmily5Omfr7wRaLlJV58zuBHHbGntavcRcUEXX58t3p31LkcDiVlp6lCUObatqcPRre9wFZLJdnFgzDkJudD8YVJtcdzdq1a2vjxo2SpL179yo4ODjPi8orad9slHvtWrJXvEuS5N3zcV1avdp0nKVECdnvuksZO3e6tmXs2q3T9e53LZJKmfWFLi1dpvMvvSxHXJwyvv9eXo92lSRZS5eWR526ytj3fb70q7DKzXilrV8ve0Cga7bG/f77JcNQVlycUr/6Sl5hYZKnpyTJs1UrZezbJxmGSs+cKbcaNSRdfsMy0tL5FMMNuJHXliRdWrVa3uH/uhy6ixeXV4f2urRylYzkZPk88YSKtW0jSXKrXl3uoaFKW78hr7tU5NWvdYeOf9PPtQCxb3hN/attVX30Vhv5ertr2uw9il59eUHjnh/PaPv3p9W6UcUCrho303WnAlq0aKFNmzYpPDxchmEoMjIyP+rKE85z53T+xQiV+nC6LG5uyjp2TAnPD5ZbjRoqOWGc4lu2lnR5TYHzTLz0Fy6lnOv9jPwix8inV0/JalXSu+8qc9++vOpKkZCb8XKePavfej+tkpFvyeLlJSMjXeee7iOlpyvl85my+vmp7IqvJJtNmftjdX7UaEnSuQEDVXL827K4uckRH69zvZ8u4N7e2m70tZUyc5bsd1VQ2TWrJHd3pcyarYytWyVJv/27t/zGjFLxiAgZjiwl9HtOzvPn872P+J3NZtXiaZ00aMxajZyySXabVfMmhql0Ka+CLg03kcUwjD+7cvi3pKenKzY2VqWf7iPbH67T45/nzpNxOnFHQEGXgVxgrG4td56Mk3FoaEGXgVzIqDBKsbGxCgkJkYdH3n5MM/s9srr3cnlYU//auU4v/ZDSLl/qzMYFIwAAYEJAAAAAJgQEAABgQkAAAAAmBAQAAGBya97xCACAW5SlZGVZ3P7azb4smW5SPn9HGTMIAADAhIAAAABMuMQAAEAh4HQ6NXLkSB08eFDu7u4aM2aMKlSo4Nq/fPlyff7557LZbAoODtbIkSNltV59noAZBAAACoGYmBhlZGRo/vz5ioiIUFRUlGtfWlqa3n33Xc2cOVPz5s1TcnKy1q9ff832CAgAABQCu3btUqNGjSRJoaGhio2Nde1zd3fXvHnzVKxYMUlSVlbWdW/ZTEAAAKAQSE5Olo+Pj+uxzWZT1v++GM1qtap06dKSpFmzZik1NVUNGza8ZnusQQAAoBDw8fFRSsrvn4V0Op2y2+05Ho8fP15HjhzRlClTZLFYrtkeMwgAABQCtWvX1saNGyVJe/fuVXBwcI79I0aMUHp6uqZNm+a61HAtzCAAAFAItGjRQps2bVJ4eLgMw1BkZKSWLVum1NRUhYSEaOHChapbt66eeOIJSVKvXr3UokWLq7ZHQAAAoBCwWq0aNWpUjm1BQUGuvx84cOCvtXdTqgIAAIUKAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACYEBAAAIAJAQEAAJgQEAAAgAkBAQAAmBAQAACACQEBAACY2Au6AAAAihS/KpLHXzwnXdKJvCjm6phBAAAAJgQEAABgQkAAAAAmBAQAAGBCQAAAACYEBAAAYEJAAAAAJgQEAABgQkAAAAAmBAQAAGBCQAAAACYEBAAAYEJAAAAAJgQEAABgQkAAAAAmBAQAAGBCQAAAACYEBAAAYEJAAAAAJgQEAABgQkAAAAAmBAQAAGBCQAAAACYEBAAAYEJAAAAAJgQEAABgQkAAAAAmBAQAAGBCQAAAACYEBAAAYEJAAACgEHA6nRoxYoS6deumnj176tixYzn2r1u3Tl26dFG3bt20YMGC67ZHQAAAoBCIiYlRRkaG5s+fr4iICEVFRbn2ZWZmauzYsfrkk080a9YszZ8/X2fPnr1me/abWZxhGJKkUhs3yN3d/WY2jTyQnp4u/19/LugykAuM1a0lPT1dqjCqoMtALmRkZEj6/f0rP2Rm5s05u3btUqNGjSRJoaGhio2Nde375ZdfFBgYqBIlSkiS6tSpo507d6pNmzZXbe+mBoTM//Xg0KFDN7NZAADyVGZmpjw9PfP0OWw2m2w2mw4edNzQ+VeTnJwsHx+fHMdnZWXJbrcrOTlZvr6+rn3e3t5KTk6+5vPd1IDg7e2t4OBgubm5yWKx3MymAQC46QzDUGZmpry9vfP8uex2u0JCQuRw/P2AYLdf/W3bx8dHKSkprsdOp9N1/B/3paSk5AgMf1rv36ryKqxW63WfEACAf5K8njm4kt1uv+ab/I2oXbu21q9fr7Zt22rv3r0KDg527QsKCtKxY8eUmJgoLy8v7dy5U717975mexYjPy+8AACAPOF0OjVy5EgdOnRIhmEoMjJSP/74o1JTU9WtWzetW7dO7733ngzDUJcuXfTYY49dsz0CAgAAMOFjjgAAwISAAAAATAgIAADAhICAQu/nn39WVlZWQZeBP3A6nQVdAvIAy9oKDwLCTfJ3P9eKvLVixQq9//77+uGHHwgJ/xCffvqpLly4IKvVSkgoRBISEiSJe+AUIgSEm8DpdMpms8kwDO3bt0+nT58u6JKKPMMwNHHiRD388MOqUaOGlixZQkj4B0hOTtbq1av1wQcfKDExkZBQSGRkZGjz5s1auHChdu7cKYfDwbgWAgSEG+RwOGS1WmUYhgYNGqSoqCjNmDFDMTExBV1akWaxWHTw4EFFRESoe/fuCgwM1OLFiwkJBcTpdOrjjz/Wjh07lJ6eLrvdrv/7v/8jJBQChmHI3d1dFotF7777rqKjo2Wz2RjXQoCAcIOyZw4+/vhj1alTR5999pmqVKmivXv3atWqVQVdXpGU/Z0gH3zwgXx9fTVo0CD16NFDFSpU0LJly7R3714uCeWzadOmad++fapVq5bCw8P1+OOPq1SpUnrnnXcICbcwp9PpuqRw6NAh1atXT7fddpsWL14s6fLddXHrYvT+piv/M9u5c6fmz58vDw8PeXh46OGHH1a5cuW0a9cunTt3rgCrLHqcTqfc3NyUkJCgEydOKDIyUhUqVNDAgQPVo0cP+fv7a+3atcwi5LOwsDAdPXpUw4YNU/Xq1VW2bFl16tRJ/v7+GjNmjGtNAm4t2cFu8ODBqlSpkiZMmKAaNWro0KFDmjhxotasWUMYv4XlzQ2hCzmHw+GaOTh48KCCg4M1dOhQzZkzR5UrV1bdunXVpk0bpaen67bbbivocosMwzBktVoVHx+vgQMHqmLFisrIyNA777yjqKgoPfnkk/rss8+UmpoqDw+Pgi63SAkICJDdbtfPP//sWsxWoUIFtW3bVmvWrHF95S5uDYZhuGYO9uzZoxUrVuixxx6TzWZT3bp15ebmpkWLFql+/frX/PZB/LNxq+W/yel0qm/fvipVqpT27t2r4cOH6/z585o7d64GDx6s+vXrF3SJRdLFixc1ePBg1yWF1157TcHBwXrjjTc0adIkhYeHq1y5cgVdZpGUkJCguLg4jR07Vk899ZRatWol6fIlITc3twKuDrnldDpd665+++03+fv7KyYmRiNGjNAHH3ygGjVqyOFwyOFwyN3dvaDLxQ0gIPwFcXFxKleunOx2u8aOHSur1aqhQ4dqx44dGjt2rMaPH6+ffvpJAQEBqlmzZkGXW2Rkz+hIl1fJf/3116pYsaKio6PVqFEjffbZZwoMDNSECRMKuFJI0saNGxUVFaWIiAg9/PDDBV0O/oLsmQOn06lBgwbJ19dXu3fvVlRUlNLS0tS/f399/vnnuvfeewu6VNwEXGLIpS1btig5OVkBAQGSpHLlyrl+66lXr56aNWum/fv3q2PHjgVYZdGT/RHTM2fOaMOGDQoMDFSVKlX07bffqkuXLnI6nQoODlafPn0KulT8T+PGjeXm5uZ6LeHWkX1ZYfjw4apYsaIiIiK0detW9e/fX4sXL9Yrr7yi5OTkAq4SNwsBIZeyLxl8/vnnuvvuu+Xl5aWTJ08qJiZGxYsX1+rVqzVy5MiCLbIIslqtOnv2rF5++WUFBwcrODhYNWvW1MqVK7V06VJt3rxZ06dPV2BgYEGXiitwCe7WcuWaA0ny9/dXhw4dJEkPPPCA/vWvf+n48ePq2rXrnx6PWxPLhq/jjytwU1JS9O2336p8+fLy8vLS/v379dFHH2no0KGqXbt2AVVZNGV/kiQ6Olr33HOPXn31VdWqVUvbt2+Xj4+P2rRpo88++0xBQUEFXClw68p+szcMQ3v27FFycrKSkpI0d+5cJScna/fu3dqwYUOO9QaEg8KBGYRryF485XQ6NXr0aFWvXl3PPfecPvvsM+3YsUP333+/GjRooKSkJBUvXrygyy0yrlwkJV1eIW+1WpWamiovLy/t2rVL5cqV47dU4AZdGQ769++vtLQ0+fn5qXnz5lq6dKkmTJiggwcP6qWXXlKNGjUKulzcZASEa8heYzBw4EDVq1dPZcqU0erVq9WwYUN99dVXWr58uUJCQuTr61vAlRYd2eEgPj5es2fPVpUqVZSWlqZt27bJzc1Nqamp2rp1K5d7gJsgOxy8+uqrqlOnjnr37q2FCxdq8+bNGjFihMqWLauEhAT5+/sXdKnIA1xi+BOLFy/W3LlzJUmnT59WcnKyGjVqpDlz5mjHjh2aOnWq+vXrpwEDBqh48eJMp+WT7HCQkJCgZ555RhUrVtTKlSt15MgRtW/fXn5+fnI6nXrzzTdVqVKlgi4XuGVdeSO406dP64cfflBcXJwkqWvXrkpPT9cPP/wgm82m0qVLF1SZyGMEhD/RsWNHde/eXe+8845uv/121axZU9u3b9ezzz6r559/XufPn1dCQoLKly9f0KUWGdnhIDExUQcPHlTXrl3VsWNHnT17Vn5+fvL391fHjh01YMAAVaxYsaDLBW5Z2Tccczqd2rJlixISEjR16lQlJydr6tSpOnbsmI4eParbb79dEusNCjPug3CFKz9PL0n9+vWTxWLRtGnT9MMPP2jlypXasGGDhgwZoiZNmhRgpUXT+fPn1bdvX7Vs2VKff/65/Pz8NHPmTO3du1eLFi3S22+/LS8vL/7DAv6mK9ccDBo0yPXYy8tLAwYM0AsvvKDk5GSNHz/edUMk7pRYeDGD8D/Zn6fPXpA4adIkTZo0SV5eXurTp4+qV6+utm3bKioqinBQAJxOp1auXKmsrCw1b95cgwYNUkpKimJiYjRp0iS98MIL8vb2JhwANyA7HEybNk2BgYGaOnWqJk6cqOTkZMXGxmrcuHGqUKGCtm7dKkmEg0KOgPA/2V8U079/f3l7e+u+++6Tu7u7xowZo2LFiqlnz56qVq2aqlevXsCVFk1Wq1WtWrVSy5YtNW/ePIWEhOi9996Tj4+PJk+erLvvvrugSwQKhXPnzmnHjh2SpKSkJNntdtWrV08ZGRkKCgrSkCFDtH//ftd3aqDwKvKXGK68ocepU6c0evRoTZs2TZL0888/68MPP9S4ceN08OBBValSpSBLhS7fz3/p0qX69ddf9dRTT7HeALgJstf49O/fX48//rgCAgI0adIk1atXT35+fvrkk0/0wgsv6IEHHpAkZWRk8D0LRUCRnkFwOBw5pqTLlCkjHx8fvfPOO5IkDw8PJSUlKTExkXDwD1GqVCmFhYWpcuXK3HsCuEHZvx9mz6CGhYVp2LBhrm9E/e677zR37lw988wzrnAgiXBQRBTZGYQ/fulIpUqV9PPPP6tv376aO3eukpKSdPbsWfXr10/NmjUr6HLxByyOAm7MlbOnH330kTp27KjSpUvrm2++0Ysvvqhp06apfPnymjp1qurXr6/GjRurVKlSBVw18lORDAhXvjCGDBmiWrVqqU2bNvrXv/6lNm3aaMCAATp8+LB8fHx01113FWyxAHCTXXk30vXr12vdunVyd3fXc889p9KlS+vtt9/WqlWrtG7dOm3YsEHr169XREQEs3ZFTJELCNkvjGxTp05V3bp1NWvWLHXp0kUJCQkKCQlR1apVC7BKAMhbTqdTgwcPVpkyZVSjRg0dPnxY58+fV4sWLbR06VJ1795dderUkSRdunRJxYoVK+CKkd+K1BqE7BuAGIahV155RYsXL1ZCQoLefvttNWjQQNWrV9fMmTNVxDITgCIoMjJS/v7+GjhwoDZv3qzExEQdOHBA06dPV/v27VWnTh3XHRUJB0VTkfkuhiuvWQ8ZMkTFihVTx44dde+99+q3335TcnKyIiIiFBERoWrVqhVwtQCQtwICAhQXF6c33nhDzZs315kzZ9SsWTPVrl3bdSnhytlWFD1FIiBceROk7du36+TJk8rKytKxY8cUFBSk4cOHy9PTU82bN+ergQEUCWFhYUpJSdGRI0dUunRpzZw5U0OGDGGdAVwK/RqEK28d2q9fP5UpU0bx8fHasGGDQkJCNHHiRAUEBBR0mQCQ7y5duqTly5dr+fLleuqpp/TQQw8VdEn4Byn0ASHbe++9pxMnTmjs2LEyDEMvvPCCNm7cqDvuuEMLFy6Up6dnQZcIAPnOMAxdvHiRmQOYFIlLDBcvXlRaWprOnTunAwcOqGrVqgoLC1Pr1q0VEhJCOABQZFksFsIB/lSRmUG4cOGCoqOjdeLECVWqVElLlizR888/r4YNGxZ0aQAA/OMUmSWqJUqUUIcOHVSyZEl9/fXXevTRR9WwYUM+0ggAwJ8oEpcYspUqVUqPP/64vLy8dPDgQdflBgAAkFORmUHI5ufnpw4dOuiuu+5S6dKlC7ocAAD+kYrMGoQ/4st+AAC4uiIbEAAAwNUVuUsMAADg+ggIAADAhIAAAABMCAgAAMCEgAAAAEz+H88MOMbifTu7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x396 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_model(best_model, plot=\"class_report\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the results are excellent for the class True (1), but good for the class False (0). <br>\n",
    "As the test data is well fitted on the model, let's use it to fit a final model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:logs:Creating MLFlow logs\n",
      "INFO:logs:SubProcess create_model() called ==================================\n",
      "INFO:logs:Initializing create_model()\n",
      "INFO:logs:create_model(estimator=BaggingClassifier(base_estimator=XGBClassifier(base_score=0.5, booster='gbtree',\n",
      "                                               colsample_bylevel=1,\n",
      "                                               colsample_bynode=1,\n",
      "                                               colsample_bytree=1, gamma=0,\n",
      "                                               gpu_id=-1,\n",
      "                                               importance_type='gain',\n",
      "                                               interaction_constraints='',\n",
      "                                               learning_rate=0.300000012,\n",
      "                                               max_delta_step=0, max_depth=6,\n",
      "                                               min_child_weight=1, missing=nan,\n",
      "                                               monotone_constraints='()',\n",
      "                                               n_estimators=100, n_jobs=-1,\n",
      "                                               num_parallel_tree=1,\n",
      "                                               objective='binary:logistic',\n",
      "                                               random_state=42, reg_alpha=0,\n",
      "                                               reg_lambda=1, scale_pos_weight=1,\n",
      "                                               subsample=1, tree_method='exact',\n",
      "                                               validate_parameters=1,\n",
      "                                               verbosity=0),\n",
      "                  bootstrap=True, bootstrap_features=False, max_features=1.0,\n",
      "                  max_samples=1.0, n_estimators=10, n_jobs=-1, oob_score=False,\n",
      "                  random_state=42, verbose=0, warm_start=False), ensemble=False, method=None, fold=10, round=4, cross_validation=True, verbose=False, system=False)\n",
      "INFO:logs:Checking exceptions\n",
      "INFO:logs:Preloading libraries\n",
      "INFO:logs:Preparing display monitor\n",
      "INFO:logs:Copying training dataset\n",
      "INFO:logs:Importing libraries\n",
      "INFO:logs:Defining folds\n",
      "INFO:logs:Declaring metric variables\n",
      "INFO:logs:Importing untrained model\n",
      "INFO:logs:Declaring custom model\n",
      "INFO:logs:Bagging Classifier Imported succesfully\n",
      "INFO:logs:Checking ensemble method\n",
      "INFO:logs:Initializing Fold 1\n",
      "INFO:logs:Fitting Model\n",
      "INFO:logs:Evaluating Metrics\n",
      "INFO:logs:Compiling Metrics\n",
      "INFO:logs:Initializing Fold 2\n",
      "INFO:logs:Fitting Model\n",
      "INFO:logs:Evaluating Metrics\n",
      "INFO:logs:Compiling Metrics\n",
      "INFO:logs:Initializing Fold 3\n",
      "INFO:logs:Fitting Model\n",
      "INFO:logs:Evaluating Metrics\n",
      "INFO:logs:Compiling Metrics\n",
      "INFO:logs:Initializing Fold 4\n",
      "INFO:logs:Fitting Model\n",
      "INFO:logs:Evaluating Metrics\n",
      "INFO:logs:Compiling Metrics\n",
      "INFO:logs:Initializing Fold 5\n",
      "INFO:logs:Fitting Model\n",
      "INFO:logs:Evaluating Metrics\n",
      "INFO:logs:Compiling Metrics\n",
      "INFO:logs:Initializing Fold 6\n",
      "INFO:logs:Fitting Model\n",
      "INFO:logs:Evaluating Metrics\n",
      "INFO:logs:Compiling Metrics\n",
      "INFO:logs:Initializing Fold 7\n",
      "INFO:logs:Fitting Model\n",
      "INFO:logs:Evaluating Metrics\n",
      "INFO:logs:Compiling Metrics\n",
      "INFO:logs:Initializing Fold 8\n",
      "INFO:logs:Fitting Model\n",
      "INFO:logs:Evaluating Metrics\n",
      "INFO:logs:Compiling Metrics\n",
      "INFO:logs:Initializing Fold 9\n",
      "INFO:logs:Fitting Model\n",
      "INFO:logs:Evaluating Metrics\n",
      "INFO:logs:Compiling Metrics\n",
      "INFO:logs:Initializing Fold 10\n",
      "INFO:logs:Fitting Model\n",
      "INFO:logs:Evaluating Metrics\n",
      "INFO:logs:Compiling Metrics\n",
      "INFO:logs:Calculating mean and std\n",
      "INFO:logs:Creating metrics dataframe\n",
      "INFO:logs:Finalizing model\n",
      "INFO:logs:Uploading results into container\n",
      "INFO:logs:Uploading model into container now\n",
      "INFO:logs:create_model_container: 17\n",
      "INFO:logs:master_model_container: 17\n",
      "INFO:logs:display_container: 21\n",
      "INFO:logs:BaggingClassifier(base_estimator=XGBClassifier(base_score=0.5, booster='gbtree',\n",
      "                                               colsample_bylevel=1,\n",
      "                                               colsample_bynode=1,\n",
      "                                               colsample_bytree=1, gamma=0,\n",
      "                                               gpu_id=-1,\n",
      "                                               importance_type='gain',\n",
      "                                               interaction_constraints='',\n",
      "                                               learning_rate=0.300000012,\n",
      "                                               max_delta_step=0, max_depth=6,\n",
      "                                               min_child_weight=1, missing=nan,\n",
      "                                               monotone_constraints='()',\n",
      "                                               n_estimators=100, n_jobs=-1,\n",
      "                                               num_parallel_tree=1,\n",
      "                                               objective='binary:logistic',\n",
      "                                               random_state=42, reg_alpha=0,\n",
      "                                               reg_lambda=1, scale_pos_weight=1,\n",
      "                                               subsample=1, tree_method='exact',\n",
      "                                               validate_parameters=1,\n",
      "                                               verbosity=0),\n",
      "                  bootstrap=True, bootstrap_features=False, max_features=1.0,\n",
      "                  max_samples=1.0, n_estimators=10, n_jobs=-1, oob_score=False,\n",
      "                  random_state=42, verbose=0, warm_start=False)\n",
      "INFO:logs:create_model() succesfully completed......................................\n",
      "INFO:logs:SubProcess create_model() end ==================================\n",
      "INFO:logs:SubProcess save_model() called ==================================\n",
      "INFO:logs:Initializing save_model()\n",
      "INFO:logs:save_model(model=BaggingClassifier(base_estimator=XGBClassifier(base_score=0.5, booster='gbtree',\n",
      "                                               colsample_bylevel=1,\n",
      "                                               colsample_bynode=1,\n",
      "                                               colsample_bytree=1, gamma=0,\n",
      "                                               gpu_id=-1,\n",
      "                                               importance_type='gain',\n",
      "                                               interaction_constraints='',\n",
      "                                               learning_rate=0.300000012,\n",
      "                                               max_delta_step=0, max_depth=6,\n",
      "                                               min_child_weight=1, missing=nan,\n",
      "                                               monotone_constraints='()',\n",
      "                                               n_estimators=100, n_jobs=-1,\n",
      "                                               num_parallel_tree=1,\n",
      "                                               objective='binary:logistic',\n",
      "                                               random_state=42, reg_alpha=0,\n",
      "                                               reg_lambda=1, scale_pos_weight=1,\n",
      "                                               subsample=1, tree_method='exact',\n",
      "                                               validate_parameters=1,\n",
      "                                               verbosity=0),\n",
      "                  bootstrap=True, bootstrap_features=False, max_features=1.0,\n",
      "                  max_samples=1.0, n_estimators=10, n_jobs=-1, oob_score=False,\n",
      "                  random_state=42, verbose=0, warm_start=False), model_name=Trained Model, verbose=False)\n",
      "INFO:logs:Appending prep pipeline\n",
      "INFO:logs:Trained Model.pkl saved in current working directory\n",
      "INFO:logs:[Pipeline(memory=None,\n",
      "         steps=[('dtypes',\n",
      "                 DataTypes_Auto_infer(categorical_features=[],\n",
      "                                      display_types=False, features_todrop=[],\n",
      "                                      ml_usecase='classification',\n",
      "                                      numerical_features=[],\n",
      "                                      target='same_security',\n",
      "                                      time_features=[])),\n",
      "                ('imputer',\n",
      "                 Simple_Imputer(categorical_strategy='not_available',\n",
      "                                numeric_strategy='mean',\n",
      "                                target_variable=None)),\n",
      "                ('new_levels1',\n",
      "                 New_Catagor...\n",
      "                ('group', Empty()), ('nonliner', Empty()), ('scaling', Empty()),\n",
      "                ('P_transform', Empty()), ('pt_target', Empty()),\n",
      "                ('binn', Empty()), ('rem_outliers', Empty()),\n",
      "                ('cluster_all', Empty()),\n",
      "                ('dummy', Dummify(target='same_security')),\n",
      "                ('fix_perfect', Empty()), ('clean_names', Clean_Colum_Names()),\n",
      "                ('feature_select', Empty()), ('fix_multi', Empty()),\n",
      "                ('dfs', Empty()), ('pca', Empty())],\n",
      "         verbose=False), BaggingClassifier(base_estimator=XGBClassifier(base_score=0.5, booster='gbtree',\n",
      "                                               colsample_bylevel=1,\n",
      "                                               colsample_bynode=1,\n",
      "                                               colsample_bytree=1, gamma=0,\n",
      "                                               gpu_id=-1,\n",
      "                                               importance_type='gain',\n",
      "                                               interaction_constraints='',\n",
      "                                               learning_rate=0.300000012,\n",
      "                                               max_delta_step=0, max_depth=6,\n",
      "                                               min_child_weight=1, missing=nan,\n",
      "                                               monotone_constraints='()',\n",
      "                                               n_estimators=100, n_jobs=-1,\n",
      "                                               num_parallel_tree=1,\n",
      "                                               objective='binary:logistic',\n",
      "                                               random_state=42, reg_alpha=0,\n",
      "                                               reg_lambda=1, scale_pos_weight=1,\n",
      "                                               subsample=1, tree_method='exact',\n",
      "                                               validate_parameters=1,\n",
      "                                               verbosity=0),\n",
      "                  bootstrap=True, bootstrap_features=False, max_features=1.0,\n",
      "                  max_samples=1.0, n_estimators=10, n_jobs=-1, oob_score=False,\n",
      "                  random_state=42, verbose=0, warm_start=False)]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:logs:save_model() succesfully completed......................................\n",
      "INFO:logs:SubProcess save_model() end ==================================\n",
      "INFO:logs:create_model_container: 17\n",
      "INFO:logs:master_model_container: 17\n",
      "INFO:logs:display_container: 21\n",
      "INFO:logs:BaggingClassifier(base_estimator=XGBClassifier(base_score=0.5, booster='gbtree',\n",
      "                                               colsample_bylevel=1,\n",
      "                                               colsample_bynode=1,\n",
      "                                               colsample_bytree=1, gamma=0,\n",
      "                                               gpu_id=-1,\n",
      "                                               importance_type='gain',\n",
      "                                               interaction_constraints='',\n",
      "                                               learning_rate=0.300000012,\n",
      "                                               max_delta_step=0, max_depth=6,\n",
      "                                               min_child_weight=1, missing=nan,\n",
      "                                               monotone_constraints='()',\n",
      "                                               n_estimators=100, n_jobs=-1,\n",
      "                                               num_parallel_tree=1,\n",
      "                                               objective='binary:logistic',\n",
      "                                               random_state=42, reg_alpha=0,\n",
      "                                               reg_lambda=1, scale_pos_weight=1,\n",
      "                                               subsample=1, tree_method='exact',\n",
      "                                               validate_parameters=1,\n",
      "                                               verbosity=0),\n",
      "                  bootstrap=True, bootstrap_features=False, max_features=1.0,\n",
      "                  max_samples=1.0, n_estimators=10, n_jobs=-1, oob_score=False,\n",
      "                  random_state=42, verbose=0, warm_start=False)\n",
      "INFO:logs:finalize_model() succesfully completed......................................\n"
     ]
    }
   ],
   "source": [
    "final_model = finalize_model(best_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Validation on Unseen Data\n",
    "To validadte the model, let's see the prediction with unseen data, which was not included on the final model fit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Accuracy       AUC    Recall  Precision  F1 Score\n",
      " 0.878505  0.780538  0.952381   0.898876  0.924855\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAc4AAAFoCAYAAADeoxtDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAvNElEQVR4nO3de1hU5d7/8c8AAgooeWqbhqZ5rFDRPCSSJzZpaWqWimHZbpumlZoFngAVcpfmzrCUrMd6wEOa/NSOlkhSWmrmWTGl1LQ0K9wCKgywfn90OY9sEZjBcTn4fnXNFbNmZq0vIxcfvvd9rzUWwzAMAQCAcnEzuwAAAFwJwQkAgB0ITgAA7EBwAgBgB4ITAAA7EJwAANiB4KxEevTooebNm9tud9xxh3r27Km33nrrqh5n6NChSkhIkCRFRUVp4sSJZb7GMAwtW7ZMRUVFJT6ekJCgoUOHlvhYSEiIUlJSHC/YRGvXrtXgwYPVtm1bBQcHa8KECTp27NhV2/+BAwfUs2dPtW7dWhs3bnR4PykpKQoJCblqdV0qIiJCzZs31wcffHDZYz/++KOaN29+xX/7/1bWz5EkNW/eXJs3b3a4XqAsHmYXgKsrKipKDzzwgCSpoKBA3377raZMmaK6deuqf//+V/14U6ZMKdfztm3bptjYWD388MNyc7sx/l57+eWXlZKSovHjx6tTp07Kzs7WG2+8ofDwcK1cuVL16tWr8DESEhLUqFEjvffee6pdu7bD++nTp4+6detW4XqupEqVKkpLS9OgQYOKbV+/fr0sFku591Oen6Ovv/5aNWrUqFC9QGlujN9gNxBfX1/VqVNHderUUb169TRgwAB17txZn3/+uVOO5+fnJz8/vzKfd6NdZ+O7777T4sWLNX/+fA0ZMkSNGjXSXXfdpddff10+Pj5auHDhVTlOdna27rzzTjVo0EDe3t4O78fb21s1a9a8KjWVpH379tq8ebPy8vKKbV+/fr3atGlT7v2U5+eoTp068vT0tLdEoNwIzhuAh4eH7RdJRESEZsyYodDQUHXt2lV//vmnTp48qaefflpt2rRRt27dNGfOHOXn59te/8UXXygsLExt2rRRXFxcsV9e/z1U+/HHH+v+++9X69atNWjQIO3YsUPHjx/X8OHDJUl33HGHtmzZ4vD3EhUVpbi4OE2YMEFt2rRRWFhYsWHcLVu2aODAgQoMDFS3bt2UmJhoeyw/P1/x8fHq1KmTOnbsqOeee06///677fHS3oeUlBQNHTpU8+fPV6dOnRQcHKz4+PgrDhmuXr1agYGBuvvuu4tt9/T01Lx58zRq1CjbtpSUFPXp00eBgYEaOHBgsfenR48eSk5O1pAhQ2zv6e7du22Pbd26VQsXLlSPHj10/PhxNW/eXEePHrW9/tIhcKvVqpiYGHXu3FmtW7fWiBEj9OOPP9pquHSoNjMzU//4xz8UFBSk4OBgJSQk2L7XhIQEjR8/XjNmzFC7du3UvXv3Yu9zSVq3bi0fHx99++23tm2//fabjhw5oo4dOxZ7blpamgYMGKC77rpL7dq107hx45STk1Piz1FUVJQiIyPVv39/dezYUQcPHrQN1WZmZurOO+/UypUrbd9/v379NGnSpFJrBcpCcFZiVqtVn3/+uTZt2qQePXrYtqekpGjWrFl68803ddNNN2nMmDGqUaOGVq1apTlz5ujLL7/U3LlzJUmHDx/WuHHjNHToUK1atUr5+fnasWNHicf75ptv9OKLLyo8PFxr165Vx44d9dRTT6lGjRq2OdH09HS1bdu2Qt/X8uXL1bJlS6WkpCg4OFixsbE6c+aMCgsL9eyzz6p79+765JNPFB0drTfeeENfffWVJGnu3LnauXOnEhMTlZSUJMMw9NRTT8kwDBmGUer7IEl79uxRZmamlixZonHjxik5Odm27/+WkZGhu+66q8THWrRoYRumTUlJ0YwZMzRy5EitWbNGXbp00ciRI/XLL7/Ynj9//nw9+eSTev/99+Xl5aWZM2dKkj744AO1bdtWjz32WInzh/9tyZIl2rRpkxITE7V27Vr5+PiUGCJ//vmnwsPDVbduXa1cuVKxsbFasmSJ/ud//sf2nC+++ELu7u5auXKlBg8erLlz5+rw4cOlHr979+5KTU213d+wYYO6du0qD4//mzH6+eef9cwzz2jIkCH69NNPNW/ePH377bdatmyZ6tWrV+LP0dq1azVmzBgtWrRITZs2te2rSZMmGjVqlObOnauzZ89q0aJFysrKIjhRYcxxVjIzZszQSy+9JEm6cOGCvL299dhjj6lfv36254SEhKh9+/aS/gq748ePa8WKFXJ3d5ckRUdH64knntDEiRO1atUqBQUF6fHHH5ckTZs2TRs2bCjx2MuWLVPv3r01bNgwSdLzzz8vwzCUnZ1tm3OqVatWsV+UjmjWrJn++c9/SpLGjx+v5ORkHTp0SE2bNtWZM2dUq1YtNWjQQA0aNNC7776rW2+9VefPn1dycrJWrFihVq1aSZJeeeUVdezYUdu3b5fVai31fZD+mjOeMWOG/Pz81KRJEy1ZskR79uzRvffee1mN2dnZ8vX1LfN7SUpK0rBhw2zzz88//7y2bt2qpKQkRUZGSpL69++vXr16SZKeeOIJjR07VpJUs2ZNValSRVWrVlXNmjV17ty5Uo91/PhxeXt7q0GDBqpZs6ZiY2N15MiRy5730UcfycvLSzNmzFCVKlXUpEkTnT59WvPmzdOTTz4p6a8h+qioKLm7u2vUqFFavHix9u7dq9tvv/2Kx+/Zs6eio6Nt99evX69Bgwbp0KFDtm2FhYWaMmWKBg8eLElq0KCB7rnnHh0+fFju7u4l/hy1bNlSoaGhJR5z5MiR+vTTTzV16lSlpaUpISFB1atXL/V9AspCcFYyY8eO1X333SdJ8vLyUp06dWxBcFH9+vVtX2dmZurs2bO2IJX+mkeyWq365ZdflJmZqebNm9seq1KlSrH7l8rMzNTDDz9su+/m5qYXX3xR0l+dRGk8PDyuOH9VVFRULGxvvfVW29cXw6mgoED+/v4aOXKkpk+frgULFqh79+7q16+f6tSpox9++EFWq9UW6hfl5eXpp59+Ul5eXqnvgyTddNNNxeZzfX19VVBQUGLNN910k86ePVvq9yz99Z6NHj262LY2bdrYhlBL+n6LiopUWFh42b9rWR599FGtX79eXbt2VVBQkHr27KmHHnqoxJpatWqlKlWq2La1bdtWWVlZ+vPPPyX99TN06fF9fHxktVpLPX7nzp2VnZ2tvXv3qlGjRtq5c6fmzZtXLDgbNWokT09PLViwQIcOHdKhQ4d0+PBh3X///Vfcb4MGDa74mKenp6ZPn65hw4bpvvvuc+oCKNw4CM5KpmbNmmrYsGGpz7l04URBQYEaNmxY4hzV3/72txJff+kv1PJsL4/q1asrJyfnsu0XO9ZLu4SSjnMxdJ9//nkNGDBAqampSktLU0REhOLi4mxdZlJS0mWLmWrWrKlVq1aV+T6Udtz/duedd15xSHvlypXav3+/YmJiSlzQU1hYqMLCQtv9kha6lHTcklanXhrsjRo10rp165Senq4vv/xSCxcu1IoVKy471cfLy+uy/Vyc37z4f0f+rb28vBQcHKy0tDQ1adJE7du3l4+PT7HnZGRkaOjQoerevbvatWunxx9/XO+9916p+y1rIdDBgwfl7u6uPXv26Ny5c6pWrZrdtQOXYo7zBnfbbbfp5MmT8vf3V8OGDdWwYUOdPn1ar776qgzDUNOmTbVnzx7b8wsLC3Xw4MES99WwYUPt37/fdt8wDPXp00fp6ellnnLQvHlzHTlyRFlZWcW2Z2Rk6MKFC2rZsmWZ38vp06cVGxur+vXr65///KeWLl2qgQMH6tNPP9Wtt94qd3d3ZWVl2b7PmjVratasWTpx4kSZ74O9+vXrp3379mnr1q3Ftl+4cEHvvPOOrTtr3Lixdu3aVew5u3bt0m233Wb3MS+G2aV/gBw/ftz29erVq7V+/XqFhoYqPj5eq1evVmZm5mX/nk2aNNH+/fuLdZA7duyQv79/hVfe9uzZU2lpaUpNTS1xeHXNmjUKCgrS3LlzNWzYMAUGBuro0aO2fwN7Tl2RpFOnTmnu3LmKj4+Xp6enXnvttQrVD0gE5w0vODhYt956qyZOnKiMjAzt2LFDU6dOlZubm7y8vPTwww9r//79mj9/vn788UfNmjVLJ0+eLHFfw4cP1yeffKKVK1fq6NGjmjNnjv7zn/+obdu2tr/y9+/ff9kpCZIUFBSk5s2b69lnn9X333+vn3/+WRs3btQLL7ygvn376uabby7ze6lRo4bWr1+v+Ph4HT16VLt379Z3332nO+64Q76+vnr44Yc1c+ZMffPNN8rMzFRkZKR++OEHNWrUqMz3wV6BgYEKDw/X2LFjtXLlSh07dkzfffedRo4cqXPnzumZZ56RJI0YMUJLly7V6tWr9dNPP+nVV1/VgQMH9Mgjj9h9zNq1a6tevXpKTEzUzz//rNWrV+vLL7+0PZ6dna34+Hh9/fXXOn78uFatWqVq1aqpUaNGxfbzwAMPqKioSNHR0crMzFRqaqoSEhI0ZMiQCp+D261bNx08eFAbN25U9+7dL3vc399fP/zwg3bt2qUjR47oX//6l/bs2WML8bJ+jv7b9OnT1apVKw0YMEDR0dFKTk62rUoGHEVw3uDc3d315ptvyt3dXUOGDNGoUaPUvn17xcXFSfpreG/hwoX67LPP1L9/f2VlZalr164l7qtdu3aaOXOmEhMT1bdvX23fvl2JiYny8/NTs2bNFBwcrPDwcKWnp1/2Wjc3N7399tsKCAjQc889p969e2v69Onq0aOHbbFTWS6dG+vfv79GjhypLl26aMyYMZKkSZMmKTg4WOPHj9egQYOUl5end955R97e3mW+D46YNm2ann32WSUlJenBBx/UuHHjVLduXS1btsz2h0BYWJief/55vf766+rXr5+2bNmixYsXF1sdWl5ubm6Kj4/XDz/8oD59+uijjz7S008/bXt82LBhGjhwoCZNmqTevXsrNTVViYmJl10swMfHR2+//bZ+/vln9e/fXzNmzNDw4cP13HPPOfxeXOTv76+goCC1bNmyxO41IiJCQUFBGjFihIYMGaITJ05o7NixOnDggCSV+XN0qYvD0jExMZKke+65R3//+981derUK85NA+VhMW60M9MBAKgAOk4AAOxAcAIAYAeCEwBQae3atUsRERGSpD/++EOjR4/WsGHDNGTIENsnFa1YsUIDBw7UI488orS0tDL3yXmcAIBKadGiRVq7dq2qVq0qSZo9e7b69u2rPn366Ntvv9WPP/6oqlWrKikpSatWrVJeXp7Cw8PVpUuXUs8PpuMEAFRKAQEBtusbS9L333+vU6dO6fHHH9eHH36oDh06aPfu3Wrbtq08PT3l5+engIAAZWRklLrf67LjrBpQvg+1Ba5n549NN7sE4Cpp5rQ9V+T3/fljy0p9PCwsrNhFQE6cOKHq1avr3Xff1fz587Vo0SI1atSo2NXEfHx8SryK2aXoOAEAprFY3By+2cvf39/2SVE9evTQ3r175evrq9zcXNtzcnNzy/yMYYITAHBDaNeunTZu3ChJ2rZtm26//XYFBgZq+/btysvLU3Z2tjIzM9WsWekd9nU5VAsAuDFYrmH/FhkZqalTp2r58uXy9fXVq6++qho1aigiIkLh4eEyDEPjx48v8zKb1+WVg5jjRGXAHCcqD+fNcfo2eszh1+YcKf2Tc5yFjhMAYBpH5irNRnACAExj70fFXQ8ITgCAieg4AQAoN1ccqnW9igEAMBEdJwDANK7YcRKcAADTXMvzOK8WghMAYBo6TgAA7EBwAgBgB4ITAAA7WOR6F0BwvagHAMBEdJwAANMwVAsAgB0ITgAA7EBwAgBgF4ITAIByo+MEAMAOrhicrlcxAAAmouMEAJiGi7wDAGAHVxyqJTgBAKaxWFzvknsEJwDANHScAADYgTlOAADs4Iodp+tVDACAieg4AQCmccWOk+AEAJiGOU4AAOxBxwkAQPkxVAsAgB1c8QIIrhf1AIBKwyI3h2/lsWvXLkVERBTb9uGHH2rw4MG2+ytWrNDAgQP1yCOPKC0trcx90nECACqlRYsWae3atapatapt24EDB/TBBx/IMAxJ0unTp5WUlKRVq1YpLy9P4eHh6tKlizw9Pa+4XzpOAIBpLBY3h29lCQgIUEJCgu1+VlaW5syZo8mTJ9u27d69W23btpWnp6f8/PwUEBCgjIyMUvdLcAIAzGOxOH4rQ1hYmDw8/hpYLSws1JQpUzR58mT5+PjYnpOTkyM/Pz/bfR8fH+Xk5JS6X4ZqAQDmuUbt2759+3T06FHFxsYqLy9Phw8fVnx8vDp16qTc3Fzb83Jzc4sFaUkITgCAea7RqtrAwEB9/PHHkqTjx49rwoQJmjJlik6fPq3XXntNeXl5ys/PV2Zmppo1a1bqvghOAIB5TD4dpU6dOoqIiFB4eLgMw9D48ePl5eVV6mssxsWlRdeRqgFDzS4BqLDzx6abXQJwlZTegVVoz8ELHX7tD1+PuoqVlB+LgwAAsANDtQAA0xgueOUgghMAYB7Xy02CEwBgIjfXS06CEwBgHoZqAQCwg+vlJsEJADCRCw7VcjoKAAB2oOMEAJiHOU4AAOzgerlJcAIATOSCc5wEJwDAPK6XmwQnAMA8XHIPAAB7uOBQLaejAABgBzpOAIB5XK/hJDgBACZijhMAADu44BwnwQkAMI/r5SbBCQAwkQsO1bKqFgAAO9BxAgDM44IdJ8EJADCPC457EpwAAPPQcQIAYAfXy02CEwBgHoPzOHG9WTR3tPZlHNNrb30sSRoZEarHh3RXVW9P7djzk0a9mKj8/AI1afQ3LZw9UrVq+ik394L+MX6Bfsj8xeTqgSv74otv9PrrS+XmZlGNGn6KixurgIB6ZpcFe7ngUK0LTsuiPJrffos+XTZVA/p0sG178L67NfrxMN0fHq+gXi+oqncVPftkH0nSu6+P0dvJ6xXU8wXNnPuBli4cZ1LlQNkuXMjTCy+8qvnzJ2nNmtfVo8fdiot7y+yycIMgOCupUcP/rneXb1DKx1ts24Y91FXzFn2srP/kyjAMPTP5HS1d9ZVuufkmNWtyi1as/UaS9PmXu+RbzVtt7mxkUvVA6QoLi2QYhrKzz0mScnMvyMurislVwSGWCtxMwlBtJTU++l1JUs+QQNu22xvXU52dmVrzv1Gqd/NN2rQ1Q1NeWqo7W9yqX09lyTAM23NPnPxD9evV0s69R65x5UDZfHyqavr0MRoy5AX5+1dXUVGRli172eyy4AgXnOOk47yBVPFwV8+ud+nRp+epywOTVdPfV9NfHCw3N4suyUxJksViUWFhkTmFAmU4ePCI3nhjuT755E19/fV7GjXqYT3zzKxif/zBRVgsjt9M4pSOMyIiQlartdg2wzBksVi0fPlyZxwS5fDrqTNa89k2ZeeclyQt+39fa/JzA/Va4kf6W13/Ys+tV/cmnfj1DxOqBMr29dffKyiopW0x0LBh92vWrHeUlXVWNWvWMLk62MXJ+bdr1y7NmTNHSUlJOnDggGbOnCl3d3d5enrq5ZdfVu3atbVixQotX75cHh4eGj16tLp3717qPp0SnBMnTtTUqVP1xhtvyN3d3RmHgAP+3ydb9NADnbR42QZdyLOqb1h7bd+VqRMn/1Tm0VN6uG9nrfzwG/UKCVSRYWhvxs9mlwyUqFWrJlqy5GP9/nuWate+SevXf6sGDW4mNF2RE4dqFy1apLVr16pq1aqSpPj4eE2bNk0tW7bU8uXLtWjRIj355JNKSkrSqlWrlJeXp/DwcHXp0kWenp5X3K9TgrN169Z68MEHdfDgQYWGhjrjEHBA4v9+rpv8fbX5k5fk7uamnXuPaOzMZEnSY2MT9ObL/1TkswN0Ic+qYaNfY9gL163OnVvrH/8YqIiIyapSxUM1avjpzTenmF0WHOHE4AwICFBCQoJefPFFSdLcuXNVt25dSVJhYaG8vLy0e/dutW3bVp6envL09FRAQIAyMjIUGBh4xf06bXHQk08+6axdww4jn19o+7qoyNBLr63SS6+tuux5mUdOKmzwzGtZGlAhw4bdr2HD7je7DFzHwsLCdPz4cdv9i6H5/fffKzk5WUuWLNFXX30lPz8/23N8fHyUk5NT6n5ZVQsAMI1xjdf4fPLJJ1qwYIHeeust1axZU76+vsrNzbU9npubWyxIS8KqWgCAedwsjt/stGbNGiUnJyspKUm33nqrJCkwMFDbt29XXl6esrOzlZmZqWbNmpW6HzpOAIB5rtFpJYWFhYqPj1e9evX0zDPPSJLuvvtuPfvss4qIiFB4eLgMw9D48ePl5eVV6r4sxnW4AqRqwFCzSwAq7Pyx6WaXAFwlpXdgFdF4dIrDr/1xwcCrWEn50XECAMzjghOGBCcAwDx8OgoAAJUbHScAwDwueJF3ghMAYBrDBYdqCU4AgHlccMKQ4AQAmIehWgAA7MBQLQAAdnDBjtMFR5cBADAPHScAwDyu13ASnAAA8xguOFRLcAIAzENwAgBgB1bVAgBgBxdcokpwAgDM44IdpwtmPQAA5qHjBACYh8VBAADYgeAEAKD8+FgxAADs4YIrbQhOAIB56DgBALCDC85xumCTDACAeeg4AQDmccGOk+AEAJjH9XKT4AQAmIePFQMAwB6sqgUAwA50nAAA2MH1cpPTUQAAsAcdJwDANG4u2L65YMkAgMrCYnH8Vh67du1SRESEJOno0aMaOnSowsPDFRMTo6KiIknSihUrNHDgQD3yyCNKS0src58EJwDANM4MzkWLFmnq1KnKy8uTJM2aNUvjxo3T0qVLZRiGUlNTdfr0aSUlJWn58uV65513NHfuXOXn55e6X4ITAGAai8Xi8K0sAQEBSkhIsN3ft2+fOnToIEkKCQnR5s2btXv3brVt21aenp7y8/NTQECAMjIySt0vwQkAMI0zO86wsDB5ePzfUh7DMGyB6+Pjo+zsbOXk5MjPz8/2HB8fH+Xk5JS6XxYHAQBMcy2vf+B2yUqk3NxcVa9eXb6+vsrNzS22/dIgLXE/TqsQAIDrSKtWrbRlyxZJUnp6utq3b6/AwEBt375deXl5ys7OVmZmppo1a1bqfug4AQCmsVzD9i0yMlLTpk3T3Llz1bhxY4WFhcnd3V0REREKDw+XYRgaP368vLy8St2PxTAM4xrVXG5VA4aaXQJQYeePTTe7BOAqKb0Dq4jmb6c7/NqDT4ZcxUrKj44TAGAaF7xULcEJADCPC344CsEJADAPwQkAgB3KcyGD6w2nowAAYAc6TgCAaa7l6ShXS6nB2aJFi2JttIeHh9zd3ZWXlydfX19t27bN6QUCACovFxypLT04L17oNiYmRkFBQerXr58sFovWrVunr7766poUCACovFwxOMvVJO/evVsPPvigrfsMCwvT3r17nVoYAKDyc/bncTpDuYKzatWqWrVqlc6dO6ecnBwtWbJENWrUcHZtAIBKzs3i+M20msvzpNmzZ+uLL75Qly5ddO+99+rbb7/VK6+84uzaAACVnCt2nOVaVVu/fn0tXLhQZ86ckb+/v5NLAgDg+lWujvPAgQO677771L9/f506dUqhoaHat2+fs2sDAFRyrthxlis44+Li9MYbb8jf318333yzYmNjFRMT4+zaAACVnMXN4vDNLOUKzvPnz6tJkya2+126dFF+fr7TigIA3BhcseMs1xynv7+/MjIybKejrF27llW1AIAKc8XzOMsVnLGxsYqMjNShQ4fUvn17NWzYUHPmzHF2bQCASq7SBmdeXp6WLVumc+fOqaioSL6+vtq5c6eTSwMAVHaV7oOst2/frqKiIk2dOlXx8fEyDEOSVFBQoNjYWK1bt+6aFAkAwPWi1ODcvHmztm7dqt9++03z5s37vxd5eGjw4MFOLw4AULlVuqHaZ555RpK0evVqPfDAA/Lw8JDVapXValW1atWuSYEAgMrLFT9WrFwle3p6asCAAZKkX3/9Vb1799b69eudWhgAoPJzxdNRyhWcCxYs0OLFiyVJAQEBSklJUUJCglMLAwBUfhaLxeGbWcq1qtZqtap27dq2+7Vq1bItFAIAwFGVbo7zonbt2mnChAnq27evLBaLPvnkE7Vp08bJpQEAKrtKG5wxMTFKSkrS+++/Lw8PD7Vv317h4eHOrg0AgOuOxShlzPX06dOqU6eOfvnllxIfv+WWW5xS1PmCzU7ZL3AtnSv43ewSgKuilnc/p+27+yebHH5tWp8uV7GS8iu145w6daoSExP16KOPymKxyDCMYv9PTU29VnUCACqhSnfloMTEREnShg0brkkxAIAbS6ULzkmTJpX64lmzZl3VYgAANxY3i+udoVHqeZwdOnRQhw4dlJubq99++02dOnVScHCwzp49y+koAIAKc7M4fjNLqR3nxasFLV26VO+//77c3P7K2d69e+uRRx5xfnUAgErNWVfcs1qtioqK0okTJ+Tm5qaZM2fKw8NDUVFRslgsatq0qWJiYmy5Zo9yvSI7O1tnzpyx3f/999917tw5uw8GAMC1sHHjRhUUFGj58uUaM2aMXnvtNc2aNUvjxo3T0qVLZRiGwwtcy3Ue56hRo9SvXz8FBQXJMAzt3LlT06ZNc+iAAABc5Kw5zttuu02FhYUqKipSTk6OPDw8tHPnTnXo0EGSFBISok2bNik0NNTufZcrOPv376977rlHO3bskMViUWxsrGrVqmX3wQAAuJSz5iqrVaumEydOqHfv3srKytLChQu1bds22zVufXx8lJ2d7dC+yzVUm5+fr5SUFKWmpqpz585atmyZ8vPzHTogAAAXuVXgVpp3331XwcHBWrdundasWaOoqChZrVbb47m5uapevbrDNZdpxowZOnfunPbv3y8PDw8dO3ZMkydPduiAAABc5KxVtdWrV5efn58kqUaNGiooKFCrVq20ZcsWSVJ6errat2/vWM3ledK+ffs0YcIEeXh4qGrVqnr55ZeVkZHh0AEBALjIYjEcvpXm8ccf1759+xQeHq7HHntM48ePV3R0tBISEjR48GBZrVaFhYU5VHO55jgtFovy8/NtY8NZWVmmfhYaAKBycNYcp4+Pj+bNm3fZ9uTk5Arvu1zBOXz4cI0YMUKnT59WfHy81q9frzFjxlT44AAAuJpyBWdISIjuvPNObdmyRYWFhVqwYIFatGjh7NoAAJWcsy6A4EzlCs5hw4bp008/1e233+7segAANxBXvFZtuYKzRYsWWr16tQIDA+Xt7W3b7qzP4wQA3Bgq3aejXLRr1y7t3r272IXd+TxOAEBFVbqh2lOnTumVV16Rj4+P2rZtq4kTJzp8wigAAP/NFTvOUsN+8uTJqlu3rp5//nlZrVY+fxMAcFW5WQyHb2Yps+N85513JEldunRR//79r0VNAABct0oNzipVqhT7+tL7AABUlCsO1ZZrcdBFXC0IAHA1VbrFQYcOHVLPnj1t90+dOqWePXvKMAxW1QIAKqzSnce5bt26a1UHAOAGVOmGauvXr3+t6gAA3IAqXXACAOBMrjjH6Yo1AwBgGjpOAIBpKt3iIAAAnIk5TgAA7OCK84UEJwDANHScAADYwcIcJwAA5eeKHacrDi8DAGAaOk4AgGlcsXsjOAEApuE8TgAA7OCKc5wEJwDANAQnAAB2cDe7AAcQnAAA07jiHKcrLmgCAMA0dJwAANMwxwkAgB0ITgAA7ODuxOBMTEzUhg0bZLVaNXToUHXo0EFRUVGyWCxq2rSpYmJi5OZm/4wlc5wAANO4WRy/lWbLli3asWOHli1bpqSkJJ08eVKzZs3SuHHjtHTpUhmGodTUVMdqduhVAABcBW4Ww+Fbab7++ms1a9ZMY8aM0ahRo9StWzft27dPHTp0kCSFhIRo8+bNDtXMUC0AwDTOmuPMysrSL7/8ooULF+r48eMaPXq0DMOQxfLXAX18fJSdne3QvglOAECl4+/vr8aNG8vT01ONGzeWl5eXTp48aXs8NzdX1atXd2jfDNUCAEzjXoFbadq1a6evvvpKhmHo1KlTOn/+vDp37qwtW7ZIktLT09W+fXuHaqbjBACYxllDtd27d9e2bds0aNAgGYah6OhoNWjQQNOmTdPcuXPVuHFjhYWFObRvghMAYBpnXnLvxRdfvGxbcnJyhfdLcAIATOPM8zidheAEAJiGKwcBAGAHVwxOVtUCAGAHOk4AgGlcseMkOAEApnF3wQ+yJjgBAKZxxflCghMAYBqGagEAsAPBCQCAHVxxjtMVh5cBADANHScAwDQM1QIAYAeCEwAAOxCcAADYgU9HAQDADs78PE5nITgBAKZxxVM7XLFmAABMQ8d5gzAMQ9Mmv62mzRrosRG9VVhYpH/FJ2v7toOSpOCQuzR+4mBZLC444YAbxmcfbdfS9zbKYpG8vD01PvJBNWtRX3Nn/T/t2P6jJKlzcAuNnfAAP8sugsVBuC79mPmLZsUlac+eH9W0WQNJ0kdrN+vIT79q5eqZKioq0mPD4vXF59/p72F3m1wtULKjR37TG//+WIuXj1PtOtW1+asDmjzhf/Xk03/X0SOnlfTB8zKKDI18bL7SvtitHn9vbXbJKAcWB+G69P6yVA14KER/q1fLtq2oqEjnz+crP98qo8hQgbVQXp5VTKwSKJ1nFQ9FxTys2nWqS5JatLpVf/yeLWt+gS6cz5c1v0BFxl8/y578LLsMFgeVoKioSG5uTKWaadLUCEnSN5v32bb16x+sL9Zt0997TFBhQZE633OH7u3exqQKgbLVq19T9erXlPTX1MPrc9YquFsr9R3YURs37NWDoXEqLCxSh85NFdytlcnVorxccajWKYn2888/6+mnn1ZISIh69eqlbt26aeTIkfrpp5+ccTg4IPHNNbqppp82bJyndRte1X/+k6v/ffczs8sCynT+XL6mvpCsEz//oUkxD+t/Fn4h/5t89VFatFZ/PkVn/3NeS9/baHaZKCc3i+M302p2xk6nTJmip556Sunp6dqwYYO+/PJLPf3005o0aZIzDgcHpK7frv4DuqqKp4f8/Kqp74NdtG1rhtllAaU6+WuWnnpsvtzcLJr/9ij5Va+qL1P36IH+d6tKFQ/5+lVV737t9P22TLNLRSXmlODMz89X69bFJ+bbtGnjjEPBQS1bNdTn67ZJkqzWAm38cocCAxubXBVwZbm5FzT2Hwt1b887NfOVR+Xl/dc8ZvOW9bXh812SpAJrob7+cr/uCAwws1TYwa0CN7M4ZY6zefPmmjRpkrp27So/Pz/l5uZq48aNat68uTMOBwdMjByqf8Ulq/8Dk+Tm5qaOnVrp8Sf6mF0WcEWrlm/WyV+zlL5hr9I37LVtf/2tp/TqrNUa8uArcndzU7uOt+vREd1MqxP2ccWzhiyGYVz1JU2GYWj9+vXavn27cnJy5Ovrq6CgIIWGhpbr3KrzBZuvdknANXeu4HezSwCuilre/Zy2722nP3b4tXfXuf8qVlJ+Tuk4LRaLQkNDFRoa6ozdAwAqCVfsODmPEwBgGlc8WZHgBACYxuKCF0BwxbAHAMA0dJwAANO44BQnHScAwDwWi+O38vjjjz907733KjMzU0ePHtXQoUMVHh6umJgYFRUVOVQzwQkAMI2lAreyWK1WRUdHy9vbW5I0a9YsjRs3TkuXLpVhGEpNTXWoZoITAGAaZ16r9uWXX9aQIUNUt25dSdK+ffvUoUMHSVJISIg2b3bsmgEEJwDANM7qOFNSUlSzZk117drVts0wDNtFeHx8fJSdne1QzSwOAgCYxlkXQFi1apUsFou++eYbHThwQJGRkfrzzz9tj+fm5qp69eoO7ZvgBABUOkuWLLF9HRERodjYWM2ePVtbtmxRx44dlZ6erk6dOjm0b4ZqAQCmcebioP8WGRmphIQEDR48WFarVWFhYY7V7IyLvFcUF3lHZcBF3lFZOPMi7xlnPnL4tS38H7iKlZQfQ7UAANOUZ3Xs9YbgBACYxgVzk+AEAJjHFS/yTnACAEzjih0nq2oBALADHScAwDTOugCCMxGcAADTuOKwJ8EJADANHScAAHZwwdwkOAEA5qHjBADADi6Ymy45LwsAgGnoOAEApuFatQAA2MEFc5PgBACYh2vVAgBgBzpOAADswOkoAADYwQVzk9NRAACwBx0nAMA0rti9EZwAANMwxwkAgF1cLzkJTgCAaSwEJwAA5WexuN4sJ8EJADCR63Wcrhf1AACYiI4TAGAa5jgBALALwQkAQLmxOAgAALvQcQIAUG7McQIAYAdnBafVatXkyZN14sQJ5efna/To0br99tsVFRUli8Wipk2bKiYmRm5u9g8VE5wAgEpn7dq18vf31+zZs5WVlaUBAwaoRYsWGjdunDp27Kjo6GilpqYqNDTU7n273qwsAKAScavA7cruu+8+Pffcc7b77u7u2rdvnzp06CBJCgkJ0ebNmx2uGAAAU1gsFodvpfHx8ZGvr69ycnL07LPPaty4cTIMw/Y6Hx8fZWdnO1QzwQkAMJGlArfS/frrrxo+fLgefPBB9e3bt9h8Zm5urqpXr+5QxQQnAMA0lgr8V5rff/9dTzzxhF544QUNGjRIktSqVStt2bJFkpSenq727ds7VrNhGIZDr3Si8wWOjTsD15NzBb+bXQJwVdTy7ue0fZ8r2OTwa6t5dLniY3Fxcfr000/VuHFj27YpU6YoLi5OVqtVjRs3VlxcnNzd3e0+LsEJOAnBicrCmcFZkd/3VT3uuYqVlB9DtQAA2IHzOAEApilrdez1iOAEAJiI4AQAoNwsLjhjSHACAExExwkAQLkxxwkAgF1cLzhdb3AZAAAT0XECAEzD4iAAAOziekO1BCcAwDRlXaz9ekRwAgBMw6paAADswhwnAADl5opDta4X9QAAmIiOEwBgItfrOAlOAIBpWBwEAIBdXG/GkOAEAJjGFRcHWQzDMMwuAgAAV+F6PTIAACYiOAEAsAPBCQCAHQhOAADsQHACAGAHghMAADsQnDeoXbt2KSIiwuwyAIcVFRUpOjpagwcPVkREhI4ePWp2SbhBcAGEG9CiRYu0du1aVa1a1exSAIetX79e+fn5ev/997Vz507961//0oIFC8wuCzcAOs4bUEBAgBISEswuA6iQ7du3q2vXrpKkNm3aaO/evSZXhBsFwXkDCgsLk4cHgw1wbTk5OfL19bXdd3d3V0FBgYkV4UZBcAJwSb6+vsrNzbXdLyoq4g9CXBMEJwCXFBQUpPT0dEnSzp071axZM5Mrwo2CP88AuKTQ0FBt2rRJQ4YMkWEYeumll8wuCTcIPh0FAAA7MFQLAIAdCE4AAOxAcAIAYAeCEwAAOxCcAADYgdNRgDJMnz5d33//vaxWq44dO6YmTZpIkoYPH66HHnroqh4rLS1NR44c0YgRI67qfgFcPQQnUIaYmBhJ0vHjxzV8+HCtWbPGacfieqvA9Y/gBBywdetW/fvf/9aFCxd09uxZTZo0Sb169VJUVJTOnDmjo0eP6oUXXpCPj4/i4uLk7u6uNm3aKDMzU0lJSTp69KhiY2N15swZeXt7a9q0afL09NTy5cslSbfccstV72YBXB0EJ+CA5ORkxcXFqUmTJvrmm2/00ksvqVevXpIkf39/LVy4UFarVb169VJiYqJatGihuLg42+sjIyMVHR2tVq1a6fDhwxozZozWrVunIUOGSBKhCVzHCE7AAbNnz1ZaWpo+++wz7dq1q9jFxgMDAyVJP/zwg2rVqqUWLVpIkgYNGqT4+Hjl5uZq7969mjRpku01586dU1ZW1rX9JgA4hOAEHBAeHq6OHTuqY8eO6ty5syZOnGh7zNvbW9JfH3NVVFR02WuLiork6elZbK705MmT8vf3d3rdACqO01EAO505c0ZHjhzRc889p5CQEKWmpqqwsPCy5zVu3Fhnz57VwYMHJUkffvihJMnPz0+NGjWyBeemTZs0bNgwSXymJOAK6DgBO/n7+2vQoEG6//775eHhoU6dOunChQs6d+5csed5enrqlVdeUWRkpNzc3HTbbbfZutHZs2crNjZWb7/9tqpUqaJ///vfslgsuvvuuxUZGanatWsrIiLCjG8PQBn4dBTASYqKijRnzhyNHTtW1apV0+LFi3Xq1ClFRUWZXRqACqDjBJzEzc3N1p1WqVJF9evXV3x8vNllAaggOk4AAOzA4iAAAOxAcAIAYAeCEwAAOxCcAADYgeAEAMAOBCcAAHb4/wCyH9YSIeK4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x396 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "predict_unseen = predict_model(final_model, data=unseen);\n",
    "score_unseen = test_score_report(unseen, predict_unseen)\n",
    "print(score_unseen.to_string(index=False))\n",
    "conf_mat(unseen, predict_unseen);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, the unseen data was well predicted on the model, with a high F1-Score showing a good balance between Recall and precision."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Conclusion\n",
    "The following conclusions can be drawed:\n",
    "* The Model just needed 4 Topics Modelled to compare the sentences.\n",
    "* Character length is important to compare the sentences.\n",
    "* Recall was excellent, getting about 95% of the positive labels.\n",
    "* Precision was excellent too, predicting correctly about 90% of the values labeled as positives.\n",
    "* F1-Score was about 92% with a good balance between recall and precision.\n",
    "* The Length on the `sentence_y` is important to model prediction performance."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "nteract": {
   "version": "0.24.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}