deep-learning-with-python-notebooks/6.2-understanding-recurrent-neural-networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "原文代码作者：François Chollet\n",
    "\n",
    "github：https://github.com/fchollet/deep-learning-with-python-notebooks\n",
    "\n",
    "中文注释制作：黄海广\n",
    "\n",
    "github：https://github.com/fengdu78\n",
    "\n",
    "代码全部测试通过。\n",
    "\n",
    "配置环境：keras 2.2.1（原文是2.0.8，运行结果一致），tensorflow 1.8，python 3.6，\n",
    "\n",
    "主机：显卡：一块1080ti；内存：32g（注：绝大部分代码不需要GPU）\n",
    "![公众号](data/gongzhong.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-15T10:04:37.730109Z",
     "start_time": "2019-02-15T10:04:37.710809Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.2.4'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Understanding recurrent neural networks\n",
    "# 理解循环神经网络\n",
    "\n",
    "This notebook contains the code samples found in Chapter 6, Section 2 of [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff). Note that the original text features far more content, in particular further explanations and figures: in this notebook, you will only find source code and related comments.\n",
    "\n",
    "---\n",
    "\n",
    "[...]\n",
    "\n",
    "## A first recurrent layer in Keras\n",
    "\n",
    "The process we just naively implemented in Numpy corresponds to an actual Keras layer: the `SimpleRNN` layer:\n",
    "## Keras 中的循环层\n",
    "上面 Numpy 的简单实现，对应一个实际的 Keras 层，即 SimpleRNN 层。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import SimpleRNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is just one minor difference: `SimpleRNN` processes batches of sequences, like all other Keras layers, not just a single sequence like \n",
    "in our Numpy example. This means that it takes inputs of shape `(batch_size, timesteps, input_features)`, rather than `(timesteps,input_features)`.\n",
    "\n",
    "Like all recurrent layers in Keras, `SimpleRNN` can be run in two different modes: it can return either the full sequences of successive \n",
    "outputs for each timestep (a 3D tensor of shape `(batch_size, timesteps, output_features)`), or it can return only the last output for each \n",
    "input sequence (a 2D tensor of shape `(batch_size, output_features)`). These two modes are controlled by the `return_sequences` constructor \n",
    "argument. Let's take a look at an example:\n",
    "\n",
    "二者有一点小小的区别：SimpleRNN 层能够像其他 Keras 层一样处理序列批量，而不是像 Numpy 示例那样只能处理单个序列。因此，它接收形状为 (batch_size, timesteps, input_features) 的输入，而不是 (timesteps, input_features)。\n",
    "\n",
    "与 Keras 中的所有循环层一样，SimpleRNN 可以在两种不同的模式下运行：一种是返回每个时间步连续输出的完整序列，即形状为 (batch_size, timesteps, output_features) 的三维张量；另一种是只返回每个输入序列的最终输出，即形状为 (batch_size, output_ features) 的二维张量。这两种模式由 return_sequences 这个构造函数参数来控制。我们来看一个使用 SimpleRNN 的例子，它只返回最后一个时间步的输出。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_1 (SimpleRNN)     (None, 32)                2080      \n",
      "=================================================================\n",
      "Total params: 322,080\n",
      "Trainable params: 322,080\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Embedding, SimpleRNN\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_2 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_2 (SimpleRNN)     (None, None, 32)          2080      \n",
      "=================================================================\n",
      "Total params: 322,080\n",
      "Trainable params: 322,080\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is sometimes useful to stack several recurrent layers one after the other in order to increase the representational power of a network. \n",
    "In such a setup, you have to get all intermediate layers to return full sequences:\n",
    "\n",
    "为了提高网络的表示能力，将多个循环层逐个堆叠有时也是很有用的。在这种情况下，你需要让所有中间层都返回完整的输出序列。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_3 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_3 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_4 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_5 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_6 (SimpleRNN)     (None, 32)                2080      \n",
      "=================================================================\n",
      "Total params: 328,320\n",
      "Trainable params: 328,320\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32))  \n",
    "# This last layer only returns the last outputs.（最后一层仅返回最终输出）\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's try to use such a model on the IMDB movie review classification problem. First, let's preprocess the data:\n",
    "\n",
    "接下来，我们将这个模型应用于 IMDB 电影评论分类问题。首先，对数据进行预处理。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading data...\n",
      "25000 train sequences\n",
      "25000 test sequences\n",
      "Pad sequences (samples x time)\n",
      "input_train shape: (25000, 500)\n",
      "input_test shape: (25000, 500)\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras.preprocessing import sequence\n",
    "\n",
    "max_features = 10000  # number of words to consider as features（作为特征的单词个数）\n",
    "maxlen = 500  \n",
    "# cut texts after this number of words (among top max_features most common words)\n",
    "# 在这么多单词之后截断文本（这些单词都属于前 max_features 个最常见的单词）\n",
    "batch_size = 32\n",
    "\n",
    "print('Loading data...')\n",
    "(input_train, y_train), (input_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "print(len(input_train), 'train sequences')\n",
    "print(len(input_test), 'test sequences')\n",
    "\n",
    "print('Pad sequences (samples x time)')\n",
    "input_train = sequence.pad_sequences(input_train, maxlen=maxlen)\n",
    "input_test = sequence.pad_sequences(input_test, maxlen=maxlen)\n",
    "print('input_train shape:', input_train.shape)\n",
    "print('input_test shape:', input_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's train a simple recurrent network using an `Embedding` layer and a `SimpleRNN` layer:\n",
    "\n",
    "我们用一个 Embedding 层和一个 SimpleRNN 层来训练一个简单的循环网络。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 39s 2ms/step - loss: 0.6418 - acc: 0.6152 - val_loss: 0.6204 - val_acc: 0.6346\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 37s 2ms/step - loss: 0.4293 - acc: 0.8168 - val_loss: 0.4122 - val_acc: 0.8220\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 37s 2ms/step - loss: 0.3153 - acc: 0.8729 - val_loss: 0.3616 - val_acc: 0.8478\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 36s 2ms/step - loss: 0.2472 - acc: 0.9047 - val_loss: 0.3814 - val_acc: 0.8434\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 37s 2ms/step - loss: 0.1735 - acc: 0.9357 - val_loss: 0.3583 - val_acc: 0.8648\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 36s 2ms/step - loss: 0.1173 - acc: 0.9592 - val_loss: 0.6933 - val_acc: 0.7828\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 37s 2ms/step - loss: 0.0722 - acc: 0.9766 - val_loss: 0.4537 - val_acc: 0.8488\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 37s 2ms/step - loss: 0.0434 - acc: 0.9873 - val_loss: 0.5678 - val_acc: 0.8290\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 37s 2ms/step - loss: 0.0225 - acc: 0.9941 - val_loss: 0.5894 - val_acc: 0.8264\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 37s 2ms/step - loss: 0.0182 - acc: 0.9949 - val_loss: 0.6602 - val_acc: 0.8126\n"
     ]
    }
   ],
   "source": [
    "from keras.layers import Dense\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_features, 32))\n",
    "model.add(SimpleRNN(32))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
    "history = model.fit(input_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's display the training and validation loss and accuracy:\n",
    "\n",
    "接下来显示训练和验证的损失和精度："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VNXZwPHfQ1gi+6oia7S8yg4hBiyoFBSBKiBuIFpxQ1Swr0sVhVZfWqV1X6viggtRpCBKq0irBfeFADFsssiiAcSA7Hvgef84N2EyTJKbMJOZyTzfz2c+mXvvuWeeuUnmmXvuueeIqmKMMcZUinYAxhhjYoMlBGOMMYAlBGOMMR5LCMYYYwBLCMYYYzyWEIwxxgCWEEwAEUkSkV0i0jycZaNJRH4lImHvWy0i54jI2oDl5SJypp+yZXitF0XknrLub4xflaMdgCk7EdkVsFgd2A8c8pZvUNWM0tSnqoeAmuEumwhU9dRw1CMi1wFXqGrPgLqvC0fdxpTEEkIcU9WCD2TvG+h1qvphUeVFpLKq5pVHbMaUxP4eY481GVVgIvIXEXlLRN4UkZ3AFSJyhoh8JSLbRGSjiDwpIlW88pVFREWkpbc82ds+S0R2isiXIpJS2rLe9n4iskJEtovIUyLyuYgMLyJuPzHeICKrRGSriDwZsG+SiDwmIltE5HugbzHHZ5yITAla94yIPOo9v05Elnnv53vv23tRdeWISE/veXURed2LbQnQJcTrrvbqXSIiA7z17YGngTO95rjNAcf2voD9R3rvfYuIvCMijf0cm9Ic5/x4RORDEflFRH4SkTsDXueP3jHZISKZInJSqOY5Efks//fsHc9PvNf5BRgnIq1EZI73XjZ7x61OwP4tvPeY621/QkSSvZhbB5RrLCJ7RKRBUe/X+KCq9qgAD2AtcE7Qur8AB4ALcMn/OOB0oCvu7PBkYAUwyitfGVCgpbc8GdgMpAFVgLeAyWUoezywExjobbsNOAgML+K9+InxXaAO0BL4Jf+9A6OAJUBToAHwifszD/k6JwO7gBoBdf8MpHnLF3hlBOgF7AU6eNvOAdYG1JUD9PSePwzMBeoBLYClQWUvBRp7v5PLvRhO8LZdB8wNinMycJ/3vI8XYycgGfg78F8/x6aUx7kOsAn4PVANqA2ke9vuBr4FWnnvoRNQH/hV8LEGPsv/PXvvLQ+4EUjC/T3+D9AbqOr9nXwOPBzwfhZ7x7OGV767t20icH/A69wOzIj2/2G8P6IegD3C9IssOiH8t4T97gD+4T0P9SH/XEDZAcDiMpS9Bvg0YJsAGykiIfiMsVvA9reBO7znn+CazvK39Q/+kAqq+yvgcu95P2BFMWX/BdzsPS8uIfwQ+LsAbgosG6LexcBvveclJYRXgQcCttXGXTdqWtKxKeVxvhLILKLc9/nxBq33kxBWlxDDxcA87/mZwE9AUohy3YE1gHjLWcDgcP9fJdrDmowqvh8DF0TkNBF5z2sC2AGMBxoWs/9PAc/3UPyF5KLKnhQYh7r/4JyiKvEZo6/XAtYVEy/AG8BQ7/nlQMGFeBE5X0S+9ppMtuG+nRd3rPI1Li4GERkuIt96zR7bgNN81gvu/RXUp6o7gK1Ak4Ayvn5nJRznZsCqImJohksKZRH893iiiEwVkfVeDK8ExbBWXQeGQlT1c9zZRg8RaQc0B94rY0zGYwmh4gvucvk87hvpr1S1NvAn3Df2SNqI+wYLgIgIhT/Agh1LjBtxHyT5SuoW+xZwjog0xTVpveHFeBwwDZiAa86pC/zbZxw/FRWDiJwMPItrNmng1ftdQL0ldZHdgGuGyq+vFq5par2PuIIVd5x/BE4pYr+itu32YqoesO7EoDLB7+9vuN5x7b0YhgfF0EJEkoqI4zXgCtzZzFRV3V9EOeOTJYTEUwvYDuz2LsrdUA6v+S8gVUQuEJHKuHbpRhGKcSrwvyLSxLvAeFdxhVV1E65ZYxKwXFVXepuq4dq1c4FDInI+rq3bbwz3iEhdcfdpjArYVhP3oZiLy43X4c4Q8m0CmgZe3A3yJnCtiHQQkWq4hPWpqhZ5xlWM4o7zTKC5iIwSkaoiUltE0r1tLwJ/EZFTxOkkIvVxifAnXOeFJBEZQUDyKiaG3cB2EWmGa7bK9yWwBXhA3IX640Ske8D213FNTJfjkoM5RpYQEs/twFW4i7zP474hR5T3oXsZ8CjuH/wUYCHum2G4Y3wW+AhYBMzDfcsvyRu4awJvBMS8DbgVmIG7MHsxLrH5cS/uTGUtMIuADytVzQaeBL7xypwGfB2w73+AlcAmEQls+snf/wNc084Mb//mwDCfcQUr8jir6nbgXOAi3EXsFcDZ3uaHgHdwx3kH7gJvstcUeD1wD66Dwa+C3lso9wLpuMQ0E5geEEMecD7QGne28APu95C/fS3u93xAVb8o5Xs3IeRfkDGm3HhNABuAi1X102jHY+KXiLyGu1B9X7RjqQjsxjRTLkSkL64JYB+u22Ie7luyMWXiXY8ZCLSPdiwVhTUZmfLSA1iNa0roCwyyi4CmrERkAu5eiAdU9Ydox1NRWJORMcYYwM4QjDHGeOLqGkLDhg21ZcuW0Q7DGGPiyvz58zeranFdvYE4SwgtW7YkMzMz2mEYY0xcEZGS7tgHrMnIGGOMxxKCMcYYwBKCMcYYT1xdQwjl4MGD5OTksG/fvmiHYoqQnJxM06ZNqVKlqOF5jDGxIO4TQk5ODrVq1aJly5a4QTRNLFFVtmzZQk5ODikpKSXvYIyJmrhvMtq3bx8NGjSwZBCjRIQGDRrYGZwxZZCRAS1bQqVK7mdGRkl7HJu4P0MALBnEOPv9GFN6GRkwYgTs2eOW161zywDDyjq+bQni/gzBGGMqorFjjySDfHv2uPWRYgnhGG3ZsoVOnTrRqVMnTjzxRJo0aVKwfODAAV91XH311SxfvrzYMs888wwZkT5fNMbEjB+KGLKvqPXhkHAJIdxtcg0aNCArK4usrCxGjhzJrbfeWrBctWpVwF1YPXz4cJF1TJo0iVNPPbXY17n55psZFqnzRGPMUcq7/T5Y8yImfy1qfTgkVELIb5Nbtw5Uj7TJReIXvWrVKtq1a8fIkSNJTU1l48aNjBgxgrS0NNq2bcv48eMLyvbo0YOsrCzy8vKoW7cuY8aMoWPHjpxxxhn8/PPPAIwbN47HH3+8oPyYMWNIT0/n1FNP5Ysv3GRRu3fv5qKLLqJjx44MHTqUtLQ0srKyjort3nvv5fTTTy+IL3/E2xUrVtCrVy86duxIamoqa9euBeCBBx6gffv2dOzYkbGRPF81JkaU52dFUe6/H6pXL7yuenW3PmJUNW4eXbp00WBLly49al1RWrRQdb/ewo8WLXxXUax7771XH3roIVVVXblypYqIfvPNNwXbt2zZoqqqBw8e1B49euiSJUtUVbV79+66cOFCPXjwoAL6/vvvq6rqrbfeqhMmTFBV1bFjx+pjjz1WUP7OO+9UVdV3331XzzvvPFVVnTBhgt50002qqpqVlaWVKlXShQsXHhVnfhyHDx/WIUOGFLxeamqqzpw5U1VV9+7dq7t379aZM2dqjx49dM+ePYX2La3S/J6MibZIf1b4NXmye00R93Py5LLVA2Sqj8/YhDpDKO82uVNOOYXTTz+9YPnNN98kNTWV1NRUli1bxtKlS4/a57jjjqNfv34AdOnSpeBberDBgwcfVeazzz5jyJAhAHTs2JG2bduG3Pejjz4iPT2djh078vHHH7NkyRK2bt3K5s2bueCCCwB3M1n16tX58MMPueaaazjuuOMAqF+/fukPhDGlEO2mGohO+30ow4bB2rVw+LD7GelW4wrR7dSv5s3dqV+o9ZFQo0aNgucrV67kiSee4JtvvqFu3bpcccUVIfvm5193AEhKSiIvLy9k3dWqVTuqjPqY7GjPnj2MGjWKBQsW0KRJE8aNG1cQR6juoapq3UZNuYlGV8tQyvuzIlYk1BlCVNrkPDt27KBWrVrUrl2bjRs3Mnv27LC/Ro8ePZg6dSoAixYtCnkGsnfvXipVqkTDhg3ZuXMn06dPB6BevXo0bNiQf/7zn4C74W/Pnj306dOHl156ib179wLwyy+/hD1uY/JFo6tlKNH8rIimhEoIw4bBxInQogWIuJ8TJ5bPN4/U1FTatGlDu3btuP766+nevXvYX2P06NGsX7+eDh068Mgjj9CuXTvq1KlTqEyDBg246qqraNeuHRdeeCFdu3Yt2JaRkcEjjzxChw4d6NGjB7m5uZx//vn07duXtLQ0OnXqxGOPPRb2uI3JF0tNNdH6rIimuJpTOS0tTYMnyFm2bBmtW7eOUkSxJS8vj7y8PJKTk1m5ciV9+vRh5cqVVK4c/ZZB+z0ZP1q2DN1U06KFa0M3ZSMi81U1raRyvs4QRKSviCwXkVUiMibE9hYi8pGIZIvIXBFpGrDtkIhkeY+ZAetTRORrEVkpIm+JSNXgek3p7Nq1i+7du9OxY0cuuuginn/++ZhIBsb4lahNNbGixIQgIknAM0A/oA0wVETaBBV7GHhNVTsA44EJAdv2qmon7zEgYP3fgMdUtRWwFbj2GN6HAerWrcv8+fP59ttvyc7Opk+fPtEOycSRWOjdk6hNNbHCzxlCOrBKVVer6gFgCjAwqEwb4CPv+ZwQ2wsR122lFzDNW/UqMMhv0MaY8IqFG7HylXdXS3OEn4TQBPgxYDnHWxfoW+Ai7/mFQC0RaeAtJ4tIpoh8JSL5H/oNgG2qmt+nMlSdAIjICG//zNzcXB/hGmNKK1Z695jo8pMQQnVCD74SfQdwtogsBM4G1gP5H/bNvYsZlwOPi8gpPut0K1UnqmqaqqY1atTIR7jGmNKKld49Jrr8JIQcoFnAclNgQ2ABVd2gqoNVtTMw1lu3PX+b93M1MBfoDGwG6opI5aLqNMaUn2gMpGZij5+EMA9o5fUKqgoMAWYGFhCRhiKSX9fdwMve+noiUi2/DNAdWOqNrTEHuNjb5yrg3WN9M9HQs2fPo24ye/zxx7npppuK3a9mzZoAbNiwgYsvvjhkmZ49exLczTbY448/zp6Ac/3+/fuzbds2P6EbU8B69xjwkRC8dv5RwGxgGTBVVZeIyHgRye811BNYLiIrgBOA/D+j1kCmiHyLSwB/VdX822fvAm4TkVW4awovhek9lauhQ4cyZcqUQuumTJnC0KFDfe1/0kknMW3atJILFiE4Ibz//vvUrVu3zPWZ8me9e0zM8DMCXqw8jnW000jYvHmzNmzYUPft26eqqmvWrNFmzZrp4cOHdefOndqrVy/t3LmztmvXTt95552C/WrUqFFQvm3btqqqumfPHr3sssu0ffv2eumll2p6errOmzdPVVVHjhypXbp00TZt2uif/vQnVVV94okntEqVKtquXTvt2bOnqqq2aNFCc3NzVVX1kUce0bZt22rbtm0LRkpds2aNnnbaaXrddddpmzZt9Nxzzy0YyTTQzJkzNT09XTt16qS9e/fWn376SVVVd+7cqcOHD9d27dpp+/btddq0aaqqOmvWLO3cubN26NBBe/XqdVR90f49xarJk1WrVy88omb16mUf1dKYUPA52mmFumvpf/8XQgz/f0w6dQJvGoKQGjRoQHp6Oh988AEDBw5kypQpXHbZZYgIycnJzJgxg9q1a7N582a6devGgAEDihws7tlnn6V69epkZ2eTnZ1Nampqwbb777+f+vXrc+jQIXr37k12dja33HILjz76KHPmzKFhw4aF6po/fz6TJk3i66+/RlXp2rUrZ599NvXq1WPlypW8+eabvPDCC1x66aVMnz6dK664otD+PXr04KuvvkJEePHFF3nwwQd55JFH+POf/0ydOnVYtGgRAFu3biU3N5frr7+eTz75hJSUFBvvqBSK691j385NeUuosYwiJbDZKLC5SFW555576NChA+eccw7r169n06ZNRdbzySefFHwwd+jQgQ4dOhRsmzp1KqmpqXTu3JklS5aEHLgu0GeffcaFF15IjRo1qFmzJoMHD+bTTz8FICUlhU6dOgFFD7Gdk5PDeeedR/v27XnooYdYsmQJAB9++CE333xzQbl69erx1VdfcdZZZ5GSkgLYENmlYb17TCypUGcIxX2Tj6RBgwZx2223sWDBAvbu3VvwzT4jI4Pc3Fzmz59PlSpVaNmyZcghrwOFOntYs2YNDz/8MPPmzaNevXoMHz68xHq0mDGq8ofOBjd8dv5IpoFGjx7NbbfdxoABA5g7dy733XdfQb3BMYZaZ/xJ1GGWTWyyM4QwqFmzJj179uSaa64pdDF5+/btHH/88VSpUoU5c+awLtR/foCzzjqLDO+K4uLFi8nOzgbc0Nk1atSgTp06bNq0iVmzZhXsU6tWLXbu3BmyrnfeeYc9e/awe/duZsyYwZlnnun7PW3fvp0mTdy9gq+++mrB+j59+vD0008XLG/dupUzzjiDjz/+mDVr1gA2RHZpWO8eE0ssIYTJ0KFD+fbbbwtmLAMYNmwYmZmZpKWlkZGRwWmnnVZsHTfeeCO7du2iQ4cOPPjgg6SnpwNu9rPOnTvTtm1brrnmmkJDZ48YMYJ+/frxm9/8plBdqampDB8+nPT0dLp27cp1111H586dfb+f++67j0suuYQzzzyz0PWJcePGsXXrVtq1a0fHjh2ZM2cOjRo1YuLEiQwePJiOHTty2WWX+X6daLLePcYUZsNfm3IRa7+n4Jm5wH0ztw9jUxGFdfhrYyoaG7vHmKNZQjAJyXr3GHO0CpEQ4qnZKxHF4u/Hxu4x5mhxnxCSk5PZsmVLTH7oGJcMtmzZQnJycrRDKcR69xhztLi/D6Fp06bk5ORgcyXEruTkZJo2bVpywXKUf+F47FjXTNS8uUsGdkHZJLK472VkjDGmeNbLyBhjTKlYQjDGGANYQjDGGOPxlRBEpK+ILBeRVSIyJsT2FiLykYhki8hcEWnqre8kIl+KyBJv22UB+7wiImtEJMt7dArf2zKxLBaGjDDGHK3EXkYikgQ8A5yLm195nojM1CMznwE8DLymqq+KSC9gAnAlsAf4naquFJGTgPkiMltV8+d4/IOqln26MBN3goeMWLfOLYP18DEm2vycIaQDq1R1taoeAKYAA4PKtAE+8p7Pyd+uqitUdaX3fAPwM9AoHIGb+GRDRhgTu/wkhCbAjwHLOd66QN8CF3nPLwRqiUiDwAIikg5UBb4PWH2/15T0mIhUIwQRGSEimSKSafcaxD8bMsKY2OUnIYSa+ST45oU7gLNFZCFwNrAeyCuoQKQx8Dpwtaoe9lbfDZwGnA7UB+4K9eKqOlFV01Q1rVEjO7mIdzZkhDGxy09CyAGaBSw3BTYEFlDVDao6WFU7A2O9ddsBRKQ28B4wTlW/Cthnozf/835gEq5pylRwNmSEMbHLT0KYB7QSkRQRqQoMAWYGFhCRhiKSX9fdwMve+qrADNwF538E7dPY+ynAIGDxsbwREx9sQhhjYleJvYxUNU9ERgGzgSTgZVVdIiLjgUxVnQn0BCaIiAKfAPmzsF8KnAU0EJHh3rrhqpoFZIhII1yTVBYwMnxvy8SyYcMsARgTi2wsI2OMqeBsLCNjjDGlYgnBGGMMYAnBGGOMxxKCMcYYwBKCMcYYjyUEY4wxgCWEhGNDTxtjilLijWmm4rChp40xxbEzhARiQ08bY4pjCSGB2NDTxpjiWEJIIDb0tDGmOJYQEogNPW2MKY4lhARiQ08bY4pjvYwSjA09bYwpip0hGGOMAXwmBBHpKyLLRWSViIwJsb2FiHwkItkiMldEmgZsu0pEVnqPqwLWdxGRRV6dT3ozpxljjImSEhOCiCQBzwD9gDbAUBFpE1TsYdw0mR2A8cAEb9/6wL1AV9ycyfeKSD1vn2eBEUAr79H3mN+NMcaYMvNzhpAOrFLV1ap6AJgCDAwq0wb4yHs+J2D7ecB/VPUXVd0K/Afo682nXFtVv1Q3ZdtruHmVjTHGRImfhNAE+DFgOcdbF+hb4CLv+YVALRFpUMy+TbznxdVpjDGmHPlJCKHa9oMnYr4DOFtEFgJnA+uBvGL29VOne3GRESKSKSKZubm5PsI1xhhTFn4SQg7QLGC5KbAhsICqblDVwaraGRjrrdtezL453vMi6wyoe6KqpqlqWqNGjXyEa4wxpiz8JIR5QCsRSRGRqsAQYGZgARFpKCL5dd0NvOw9nw30EZF63sXkPsBsVd0I7BSRbl7vot8B74bh/RhjjCmjEhOCquYBo3Af7suAqaq6RETGi8gAr1hPYLmIrABOAO739v0F+DMuqcwDxnvrAG4EXgRWAd8Ds8L1powxxpSeuE4+8SEtLU0zMzOjHUZc27sXbrkFrr0WunWLdjTGmPIgIvNVNa2kcnancoJ56CF48UUYMgR27ox2NMaYWGIJIYGsWwcTJkB6upsD4Q9/iHZExphYYgkhgdx+u5tLedo0uO02eP55+M9/oh2VMSZWWEJIEB99BNOnwz33QLNm8Oc/w6mnumsJO3ZEOzpjTCywhJAADh6E0aPh5JPdWQLAccfBK6/A+vVwxx1RDc8YEyMsISSAp5+GZcvg8cchOfnI+m7dXIJ44QWYPTt68RkTKDcXPvjAXfMy5cu6nVZwmzbB//wPdO8O773nZkoLtG8fdO4Mu3bB4sVQp0504jSJSRW++w4+/9w9vvgCVqxw2ypVggsugJtvhnPOOfpv1/hn3U5jTEYGtGzp/shbtnTL5WHMGHfvweOPh/6HSk52TUcbNhxpTjImUvbuhU8/hb/+1X3YN2wIbdrA9dfDP//prmv99a+us8OYMS5J9OkDrVvDk0/C9u3RfgcVnKrGzaNLly4ajyZPVq1eXdV9H3KP6tXd+kj68kv3WnfdVXLZu+5yZd9/P7IxmcTy00+q06er3nabateuqlWqHPkfOPVU1WuuUX3pJdXvvlM9fPjo/ffuVX3tNbcvqNaooTpypOqiReX/XuIZkKk+PmOtyagctGwZuj20RQtYuzYyr3n4MHTt6r75L18ONWsWX37fPujSxX0DW7wY6taNTFym4jp8GJYuLdz88/33blu1anD66a7psnt3OOMMd3ZQGpmZ8Mwz8OabsH8/nH22a04aNAiqVAn/+6lI/DYZWUIoB5Uque9EwUTcP1EkvPiiOw3PyIDLL/e3zzffuH/Uq66Cl18uubxJbLt3w7x5RxLAl1/Ctm1uW6NGRz78u3eH1FSXFMJh82b39/nss+4L1UknwQ03uL/3xo3D8xoVjSWEGFLeZwhbt7oLya1bw8cfl+5i3D33uLuZ33sP+vcPf2wmfm3YcOTD//PPISsL8vLctjZtCieAU06J/EXgQ4fg/ffdWcPs2VC5Mlx8sTtr6N7dLkIHsoQQQzIyYMQI2LPnyLrq1WHiRBg2LPyvd8st7p9kwQLo2LF0++7f75qOtm51TUf16pW8j6l4Dh1yv//A5p/8Ly/HHeeGP+neHX79a3dWWb9+VMNlxQp3xjBpkmv27NABRo1yZ8c1akQ3tljgNyFE/UJxaR7xelFZ1V1AbtFCVcT9jNQF5exs1aQk1ZtuKnsd8+a5Oq66KmxhmTjx5puqffqo1q595OLviSeqXnSR6qOPqn79ter+/dGOsmi7dqk+/7xqhw4u9jp1VG+9VXXlymhHFl3YReXEowq/+Y37ZrdixbF9axs3Du6/33UFPP/88MVoYtfkyXDlla65sVevI2cAKSnx1/yiCp995s6Up093TVt9+7rmpH79ICkp2hGWL2sySkBvveWGtX7uOXeR7Vjs3w9pabBlCyxZYk1HFd0HH7j7As46y7XLh+sCcCzYuNE1zz7/vHuekgI33gjXXAMNGkQ7uvIR1hvTRKSviCwXkVUiMibE9uYiMkdEFopItoj099YPE5GsgMdhEenkbZvr1Zm/7fjSvklzxO7dbkyi1FS47rpjr69aNXfD2s8/w+9/f+z1xbL8C6OJ6uuv4aKLoF07mDGjYiUDcD2P7r3Xdex46y03uOOdd0LTpi4pzJ8f7QhjSEltSkASborLk4GqwLdAm6AyE4EbvedtgLUh6mkPrA5Ynguk+WnXyn/E8zWESLvnHtdm+vnn4a33j3909b77bnjrjQWHDqmOHu3ay+fOjXY00fHdd6oNGqiefLLqxo3Rjqb8ZGer3nDDkRtGu3VTff111X37oh1ZZODzGoKfM4R0YJWqrlbVA8AUYGBwXgFqe8/rABtC1DMUeNPH65lSWrUKHn7Ytf/++tfhrXvcONdj44Yb4JdfSi4fLw4dcu/pqadcd8X+/V0X3USyfr0bFiIpyXXbPPHEaEdUftq3d02r69e7YV22bHH/P82awdixbgKphFRSxgAuBl4MWL4SeDqoTGNgEZADbAW6hKjne6BdwPJcb58s4I941zNC7DcCyAQymzdvHuE8Gp/OP1+1Vi3VDRsiU/+CBaqVK6sOGxaZ+svbwYOqV17pvhmOG+eGV2jd2n1bTJQzha1bVdu1U61ZU3X+/GhHE32HDqnOnq06YIBqpUruMWiQ6gcfqK5Yobp6teoPP7izqNxc1W3bXI+m/fvdvrEOn2cIlX3kjFD9C4KvRA8FXlHVR0TkDOB1EWmnqocBRKQrsEdVFwfsM0xV14tILWC6l2heC5GwJuKapEhLS4ufK+Dl5P334V//cnMlR+ouzc6d3bem//s/uOQSGBh8fhhHDh503wTfestNEjRunFs/Z47rodW/vzumZ58d3Tgjae9eGDDADWkya5a77pToKlVyZ0t9+rj7LZ57zt3t/847/vYXccNnVK585Gfgcz/bSio/blzkz+JK7GXkfcDfp6rnect3A6jqhIAyS4C+qvqjt7wa6KaqP3vLjwG5qvpAEa8xHHc9YVRxsVgvo8L273cXApOSIDsbqlaN3GsdOOBuRvrpJ9frKB57Zxw44HphzZgBDz549JzSmza5pLBuXcVNCnl5Lqm/+y5MmQKXXhrtiGLXvn1u1NUdO9xxO3iw8E+/68K17auvoFWrsr2XsN2YBlQGVgMpHLmo3DaozCxguPe8Ne4aQn6yqYRrSjo5qM6G3vMqwDRgZEmx2EXlwiZMcM0es2eXz+stXOiajoYOLZ/eBS58AAAV6UlEQVTXC6e9e1V/+1t3vJ54ouhyFbn56PBh1euvd8fgySejHY0pT/hsMvLVuwfoD6zAXQcY660bDwzQIz2LPveSRRbQJ2DfnsBXQfXVAOYD2cAS4AkgqaQ4LCEckZPjhgIeNKh8X/f//s/91bz9dvm+7rHYvVv13HNd3M89V3L5ipoU8nuMjR0b7UhMeQtrQoiVhyWEI4YOVU1Odhe7ytOBA6qdO6sef7y7uBbrdu5U7dnTDRny8sv+96toSeHpp91/+7XXhp53wFRsfhOCzZgWhz75xI0Jf+ed7q7L8lSlirthbetWN3hYLNuxww1X8OmnbliGq6/2v+8JJ7gLzS1axH+X1H/8A0aPdheSn3su/oahMOXHEkKcyctz/9zNm8Ndd0Unhg4d4E9/cj11pk+PTgwl2boVzj3X3YU7ZYr/OSECVYSk8N//whVXuHGJpkxxvVWMKYolhDjz/POuR9Gjj7ohtKPlrrtcd8Ubb4Tc3OjFEcrmzdC7txuvf/p0N0Z+WcVzUli40M0m1qoVzJzphq02pjiWEOLI5s3wxz+6D7vBg6MbS37T0bZtbgTJWLFpkxupc+lS17VywIBjrzMek8L337tRPevVc3ch2+CExg9LCHFk7FjYuROefDI22oHbt4f77nNt1P/4R7SjcTN69ezphvJ47z13/SBc4ikpbNoE553n+q7Png1NmkQ7IhMvLCHEifnz4YUX3PWDNm2iHc0Rd97phsm+6SY3Mmq0/Piju5EsJ8cN5dy7d/hfIx6Swo4d7sxg40Z3c91pp0U7IhNPLCHEgcOHXSI4/ng3jG8sqVzZNR3t2OGSgkZhcJE1a9w4/j//DP/+t3seKbGcFPbvd02J2dkwbRp07RrtiEy8sYQQByZPhi+/hL/+FerUiXY0R2vb1o1zNH06TJ1avq+9cqVLANu3w0cfufl9Iy0Wk8Lhw/C737lj8PLL7izBmNKyGdNi3I4dbkrDlBQ32XmlGE3heXmua+P337uxjk44IfKvuWyZaxo6eBA+/BA6doz8awaKlbGPVOGWW+Dpp90gh3fcEZ04TOwK64xpJnrGj3dNIU89FbvJAFzT0aRJ7qL3jTdGvulo0SL3AXz4MMydW/7JAI4+U/jkk/KPAWDCBJcMbr/dkoE5NjH8EWOWLYMnnoBrr3UXbmNdmzZuSOkZM9xNUJGyYIHrTVSlimuuads2cq9VksCk0K9f+SeFF190vc+uuMKN4GrMsbAmoxil6roOzpsHK1ZAo0bRjsifQ4dc09HKla7pKNzjt3/zjTsutWu7u3BPOSW89ZdVYPPRrFmRvbCdb+ZMuPBCN4b/zJkuQRoTijUZxbl33nFjsY8fHz/JANzcDK+8Art3w8iR4W06+vxzOOccqF/ffROPlWQA5X+m8NlncNll7szxH/+wZGDCwxJCDNq7F267zd34deON0Y6m9E47Df7yF3en8BtvhKfOuXPdmUHjxu7DtkWL8NQbTuWVFBYvhgsucK/z3ntQs2ZkXscknoRICPfe64ZXWL8+2pH48+CDbhq//Ang49Gtt7ouoKNHu5ukjsW//+0+YFu0cNcMYvnO20gnhXXrXGKsXt3dhdywYXjrN4nNV0IQkb4islxEVonImBDbm4vIHBFZKCLZItLfW99SRPaKSJb3eC5gny4issir80mRyA3GsGsXTJzomhhuvdVNAxmr1q519xtcdll8T+GYlOR6He3dCzfcUPamo/fec9+GTz3VnSVEek7ZcIhUUti82SWD3bvd3dixeJZk4lxJEyYASbiZ0k7myBSabYLKTARu1COzp631nrcEFhdR7zfAGYDgpuDsV1IsxzJBzurVqldfrZqUpHrccap/+ENsTvAyeLCblOXHH6MdSXg88oibmOW110q/79tvq1apotqli+qWLeGPLdICJ9n5+ONjq2vXLtWuXVWrVVP95JPwxGcSB2GcICcdWKWqq1X1ADAFGBicV4Da3vM6uDmViyQijYHaqvqlF+xrwCAfsZRZSoq7g3PZMnd7/8MPu3Xjxrmx82PBhx/C22+7boRNm0Y7mvD4/e9dr6NbbnGDz/n11ltuMvguXdzdt/XrRy7GSAnXmcLBg+5YzJvnuvOeeWZ44zSmQEkZA7gYeDFg+Urg6aAyjYFFQA6wFeiiR84QdgMLgY+BM731acCHAfufCfyriNcfAWQCmc2bNw9bxlyyRPWSS9y31zp13FzB27eHrfpSO3DAfZs85RTVffuiF0ckLF/upvs8/3x/0ze++qpqpUqqZ56pumNH5OOLtGM5Uzh0SPXKK93f6cSJkYnPVHyEa05l4JIQCeGpoDK3Abd7z88AluKuT1QDGnjruwA/4s4kTg+REP5ZUiyRmFM5K0t14EB3JOrXV50wwZ2el7f8ppV//rP8X7s8PPaYe3+vvlp8uRdecPMf9+4dnd9DpJQ1KfzhD+64/fnPkYvNVHzhTAhnALMDlu8G7g4qswRoFrC8Gjg+RF1zvbODxsB3AeuHAs+XFEskEkK+efNU+/VzR6RRI/cBvWdPxF6ukI0bVWvVUu3fv3xeLxoOHVLt0cOdjeXkhC6TPxF8377ld+zLU2mTwsMPu+Nx883+zqyMKUo4E0Jl7wM+hSMXldsGlZkFDPeet8ZdQxCgEZDkrT8ZWA/U95bnAd04clG5f0mxRDIh5PviC9VzznFHpnFj1aeeinwTzlVXqVatqrpiRWRfJ9pWrnQX9Pv3P/oD7tFH3TEfMKDiNZkF8psUXn/dHY9LLlHNyyu/+EzFFLaE4OqiP7AC19torLduPDDAe94G+NxLFllAH2/9Rd7Zw7fAAuCCgDrTgMVenU/jDaNR3KM8EkK+uXNdGzaoNmum+vzzrp0/3L74wr3GmDHhrzsWPfGEe7+TJh1Z98ADbt3FF0fmGMeakpLCrFmqlSur9upVsZOjKT9hTQix8ijPhKDqvsX+5z+q3bq5I5WS4j7IDh4MT/15ea5LZZMmqjt3hqfOWHfokOpZZ6nWru261t57rzu2l18evuMaD4pKCl995dZ16hTdTg6mYvGbEBLiTuWyEnFj53zxhbtBql49uPpqN7rmG2+4gdyOxcsvu6kxH3oocYYfqFTJve+8POjWzU2sM3w4vPZa/N6VXRahuqQuXw6//a27+W7WLDeAnzHlyRKCDyJuvPvMTDe0c7VqMGwYdOjgpio8fLj0dW7dCvfc40bFHDIk/DHHslNOgb/9zQ0lcsMN8NJL7s7mRBOcFM45xx2H2bPj445sU/FYQigFERg0CLKy3I1Thw+7G4ZSU93ww1qK4Rn+9Cf45Rd48klXb6K5+WZYuhSefTa2J/6JtMCksG2bOzP41a+iHZVJVAn8r1h2lSrBpZe6USdff92NLTNwoJvU/IMPSk4M2dnw97+7kUyjMdNXLBCB1q0TMxkGO+EEdxfy8uXuy4Ux0WIJ4RgkJbmZqpYtc80eP//sTv179HCTt4Si6kYArVfPzXVgDECNGnDSSdGOwiQ6SwhhULkyXHONm9ns2WfdEMW9e7sZtD77rHDZt95yFxAfeCA+x+cxxlRclhDCqGpVN0vYqlVuLuRly9xAZOed56Z+3LXLTYKemurmSTbGmFhiCSECkpPd6J6rV7supQsWuOsLnTu7njVPP52YvWqMMbHNEkIEVa/uzghWr4b774ctW+D6691MYsYYE2tES9NXMsrS0tI0MzMz2mGU2aFDrldNInezNMaUPxGZr6ppJZVLoHtDo8+aiYwxscy+qxpjjAEsIRhjjPFYQjDGGANYQjDGGOPxlRBEpK+ILBeRVSIyJsT25iIyR0QWiki2iPT31p8rIvNFZJH3s1fAPnO9OrO8x/Hhe1vGGGNKq8ReRiKSBDwDnAvkAPNEZKaqLg0oNg6YqqrPikgb4H2gJbAZN0vaBhFpB8wGmgTsN0xV47cfqTHGVCB+zhDSgVWqulpVDwBTgIFBZRTIn86jDm5OZVR1oapu8NYvAZJFpNqxh22MMSbc/CSEJsCPAcs5FP6WD3AfcIWI5ODODkaHqOciYKGq7g9YN8lrLvqjSOiBkEVkhIhkikhmbm6uj3CNMcaUhZ+EEOqDOvj25qHAK6raFOgPvC4iBXWLSFvgb8ANAfsMU9X2wJne48pQL66qE1U1TVXTGjVq5CNcY4wxZeEnIeQAzQKWm+I1CQW4FpgKoKpfAslAQwARaQrMAH6nqt/n76Cq672fO4E3cE1TxhhjosRPQpgHtBKRFBGpCgwBZgaV+QHoDSAirXEJIVdE6gLvAXer6uf5hUWksojkJ4wqwPnA4mN9M8YYY8quxISgqnnAKFwPoWW43kRLRGS8iAzwit0OXC8i3wJvAsPVjZo3CvgV8Meg7qXVgNkikg1kAeuBF8L95owxxvhno50aY0wF53e0U7tT2RhjDGAJwRhjjMcSgjHGGMASgjHGGI8lBGOMMYAlBGOMMR5LCMYYYwBLCMYYYzyWEIwxxgCWEIwxxngsIRhjjAEsIRhjjPFYQjDGGANYQjDGGOOxhGCMMQawhGCMMcbjKyGISF8RWS4iq0RkTIjtzUVkjogsFJFsEekfsO1ub7/lInKe3zqNMcaUrxITgogkAc8A/YA2wFARaRNUbBxuas3OuDmX/+7t28Zbbgv0Bf4uIkk+6zTGGFOO/JwhpAOrVHW1qh4ApgADg8ooUNt7XgfY4D0fCExR1f2qugZY5dXnp05jjDHlyE9CaAL8GLCc460LdB9whYjkAO8Do0vY10+dAIjICBHJFJHM3NxcH+EaY4wpCz8JQUKs06DlocArqtoU6A+8LiKVitnXT51upepEVU1T1bRGjRr5CNcYY0xZVPZRJgdoFrDclCNNQvmuxV0jQFW/FJFkoGEJ+5ZUpzHGmHLk5wxhHtBKRFJEpCruIvHMoDI/AL0BRKQ1kAzkeuWGiEg1EUkBWgHf+KzTGGNMOSrxDEFV80RkFDAbSAJeVtUlIjIeyFTVmcDtwAsiciuu6We4qiqwRESmAkuBPOBmVT0EEKrOCLw/Y4wxPon73I4PaWlpmpmZGe0wjDEmrojIfFVNK6mc3alsjDEGsIRgjDHGYwnBGGMMYAnBGGOMxxKCMcYYwBKCMcYYjyUEY4wxgCUEY4wxHksIxhhjAEsIxhhjPJYQjDHGAJYQjDHGeCwhGGOMASwhGGOM8VhCMMYYA/hMCCLSV0SWi8gqERkTYvtjIpLlPVaIyDZv/W8C1meJyD4RGeRte0VE1gRs6xTet2aMMaY0SpwxTUSSgGeAc3FzJM8TkZmqujS/jKreGlB+NNDZWz8H6OStrw+sAv4dUP0fVHVaGN6HMcaYY+TnDCEdWKWqq1X1ADAFGFhM+aHAmyHWXwzMUtU9pQ/TGGNMpPlJCE2AHwOWc7x1RxGRFkAK8N8Qm4dwdKK4X0SyvSanakXUOUJEMkUkMzc310e4xhhjysJPQpAQ64qaiHkIME1VDxWqQKQx0B6YHbD6buA04HSgPnBXqApVdaKqpqlqWqNGjXyEa4wxpiz8JIQcoFnAclNgQxFlQ50FAFwKzFDVg/krVHWjOvuBSbimKWOMMVHiJyHMA1qJSIqIVMV96M8MLiQipwL1gC9D1HHUdQXvrAEREWAQsLh0oRtjjAmnEnsZqWqeiIzCNfckAS+r6hIRGQ9kqmp+chgKTFHVQs1JItISd4bxcVDVGSLSCNcklQWMPJY3Yowx5thI0Od3TEtLS9PMzMxoh2GMMXFFROaralpJ5exOZWOMMYAlBGOMMR5LCMYYYwBLCMYYYzyWEIwxxgCWEIwxxngsIRhjjAESICFkZEDLllCpkvuZkRHtiIwxJjaVeKdyPMvIgBEjYI834Pa6dW4ZYNiw6MVljDGxqEKfIYwdeyQZ5Nuzx603xhhTWIVOCD/8ULr1xhiTyCp0QmjevHTrjTEmkVXohHD//VC9euF11au79cYYYwqr0Alh2DCYOBFatAAR93PiRLugbIwxoVToXkbgPvwtARhjTMl8nSGISF8RWS4iq0RkTIjtj4lIlvdYISLbArYdCtg2M2B9ioh8LSIrReQtbzY2Y4wxUVJiQhCRJOAZoB/QBhgqIm0Cy6jqraraSVU7AU8Bbwds3pu/TVUHBKz/G/CYqrYCtgLXHuN7McYYcwz8nCGkA6tUdbWqHgCmAAOLKX/U/MnBvHmUewHTvFWv4uZVNsYYEyV+EkIT4MeA5Rxv3VFEpAWQAvw3YHWyiGSKyFcikv+h3wDYpqp5Puoc4e2fmZub6yNcY4wxZeHnorKEWFfURMxDgGmqeihgXXNV3SAiJwP/FZFFwA6/darqRGAiuDmVfcRrjDGmDPwkhBygWcByU2BDEWWHADcHrlDVDd7P1SIyF+gMTAfqikhl7yyhuDoLzJ8/f7OIrPMRcygNgc1l3LcisuNxhB2Lwux4FFYRjkcLP4X8JIR5QCsRSQHW4z70Lw8uJCKnAvWALwPW1QP2qOp+EWkIdAceVFUVkTnAxbhrElcB75YUiKo28hFvSCKSqappZd2/orHjcYQdi8LseBSWSMejxGsI3jf4UcBsYBkwVVWXiMh4EQnsNTQUmKKqgc06rYFMEfkWmAP8VVWXetvuAm4TkVW4awovHfvbMcYYU1ZS+PO74kqkLO+HHY8j7FgUZsejsEQ6HhV66IogE6MdQIyx43GEHYvC7HgUljDHI2HOEIwxxhQvkc4QjDHGFMMSgjHGGCBBEkJJg/MlChFpJiJzRGSZiCwRkd9HO6ZYICJJIrJQRP4V7ViiTUTqisg0EfnO+zs5I9oxRYuI3Or9nywWkTdFJDnaMUVahU8IfgbnSyB5wO2q2hroBtycwMci0O9xXaoNPAF8oKqnAR1J0OMiIk2AW4A0VW0HJOHuwarQKnxCoPSD81VYqrpRVRd4z3fi/tlDjiGVKESkKfBb4MVoxxJtIlIbOAvvniBVPaCq24rfq0KrDBwnIpWB6vgYTSHeJUJC8D04XyIRkZa4YUS+jm4kUfc4cCdwONqBxICTgVxgkteE9qKI1Ih2UNGgquuBh4EfgI3AdlX9d3SjirxESAilGZwvIYhITdx4Uv+rqqEGGkwIInI+8LOqzo92LDGiMpAKPKuqnYHdQEJec/OG3RmIG735JKCGiFwR3agiLxESQmkG56vwRKQKLhlkqOrbJZWv4LoDA0RkLa4psZeITI5uSFGVA+Soav5Z4zRcgkhE5wBrVDVXVQ/iJv36dZRjirhESAgFg/N503QOAWaWsE+F5E1M9BKwTFUfjXY80aaqd6tqU1Vtifu7+K+qVvhvgUVR1Z+AH72BKgF6A0uL2aUi+wHoJiLVvf+b3iTABXY/o53GNVXNE5H8wfmSgJdVdUmUw4qW7sCVwCIRyfLW3aOq70cxJhNbRgMZ3pen1cDVUY4nKlT1axGZBizA9c5bSAIMYWFDVxhjjAESo8nIGGOMD5YQjDHGAJYQjDHGeCwhGGOMASwhGGOM8VhCMMYYA1hCMMYY4/l/MSFSvWzaNroAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VNXZwPHfE/Z9CSiWLaCoBAwQI6KgqCiiFlCKCoKKQhEVtaJ9QUQQUF+q1AXFhbrUVhQpvlaqKNqKRaosYZVFCrJIBBWQfTXkef84N2ESh2SSzMydzDzfz2c+mXvn3Hufmck8c+bcc88RVcUYY0xiSPI7AGOMMdFjSd8YYxKIJX1jjEkglvSNMSaBWNI3xpgEYknfGGMSiCV9UywiUk5E9otIk3CW9ZOInCYiYe+7LCKXisimgOW1InJBKGVLcKyXRWRkSbcvZL+PiMifw71f45/yfgdgIktE9gcsVgWOAMe85dtUdWpx9qeqx4Dq4S6bCFT1jHDsR0QGAf1V9aKAfQ8Kx75N/LOkH+dUNS/pejXJQar6zxOVF5HyqpodjdiMMdFnzTsJzvv5/raIvCUi+4D+InKeiMwXkd0isk1EJolIBa98eRFREUnxlt/wHv9QRPaJyJci0qy4Zb3HrxCR/4rIHhF5VkT+IyIDThB3KDHeJiLrRWSXiEwK2LaciDwlIjtF5BugWyGvzygRmVZg3WQRedK7P0hE1njP5xuvFn6ifWWJyEXe/aoi8lcvtlXA2UGOu8Hb7yoR6eGtPwt4DrjAazrbEfDaPhyw/RDvue8Ukb+LyCmhvDZFEZGrvXh2i8inInJGwGMjRWSriOwVka8DnmsHEVnirf9BRJ4I9XgmAlTVbglyAzYBlxZY9whwFOiOqwRUAc4BzsX9EmwO/BcY6pUvDyiQ4i2/AewAMoAKwNvAGyUoexKwD+jpPTYM+BkYcILnEkqM7wG1gBTgp9znDgwFVgGNgGRgrvsoBD1Oc2A/UC1g3z8CGd5yd6+MAJcAh4A077FLgU0B+8oCLvLuTwQ+A+oATYHVBcpeB5zivSc3eDGc7D02CPisQJxvAA9797t6MbYFKgPPA5+G8toEef6PAH/27rf04rjEe49Geq97BaAVsBlo4JVtBjT37i8C+nr3awDn+v1ZSOSb1fQNwDxV/Yeq5qjqIVVdpKoLVDVbVTcAU4DOhWw/Q1UzVfVnYCou2RS37K+BZar6nvfYU7gviKBCjPF/VXWPqm7CJdjcY10HPKWqWaq6E5hQyHE2ACtxX0YAlwG7VTXTe/wfqrpBnU+BfwFBT9YWcB3wiKruUtXNuNp74HGnq+o27z15E/eFnRHCfgH6AS+r6jJVPQyMADqLSKOAMid6bQrTB5ipqp9679EEoCbuyzcb9wXTymsi3Oi9duC+vFuISLKq7lPVBSE+DxMBlvQNwJbABRE5U0Q+EJHvRWQvMA6oV8j23wfcP0jhJ29PVPZXgXGoquJqxkGFGGNIx8LVUAvzJtDXu38D7ssqN45fi8gCEflJRHbjatmFvVa5TiksBhEZICLLvWaU3cCZIe4X3PPL25+q7gV2AQ0DyhTnPTvRfnNw71FDVV0L3Id7H370mgsbeEVvAVKBtSKyUESuDPF5mAiwpG/A/dwP9BKudnuaqtYERuOaLyJpG665BQAREfInqYJKE+M2oHHAclFdSt8GLvVqyj1xXwKISBVgBvC/uKaX2sDHIcbx/YliEJHmwAvA7UCyt9+vA/ZbVPfSrbgmo9z91cA1I30XQlzF2W8S7j37DkBV31DVjrimnXK41wVVXauqfXBNeH8E3hGRyqWMxZSQJX0TTA1gD3BARFoCt0XhmO8D6SLSXUTKA/cA9SMU43TgdyLSUESSgeGFFVbVH4B5wGvAWlVd5z1UCagIbAeOicivgS7FiGGkiNQWdx3D0IDHquMS+3bc998gXE0/1w9Ao9wT10G8BQwUkTQRqYRLvp+r6gl/ORUj5h4icpF37N/jzsMsEJGWInKxd7xD3u0Y7gncKCL1vF8Ge7znllPKWEwJWdI3wdwH3Iz7QL+Eq+lGlJdYrweeBHYCpwJLcdcVhDvGF3Bt71/hTjLOCGGbN3EnZt8MiHk3cC/wLu5kaG/cl1coxuB+cWwCPgT+ErDfFcAkYKFX5kwgsB38E2Ad8IOIBDbT5G7/Ea6Z5V1v+ya4dv5SUdVVuNf8BdwXUjegh9e+Xwl4HHce5nvcL4tR3qZXAmvE9Q6bCFyvqkdLG48pGXFNp8bEFhEph2tO6K2qn/sdjzHxwmr6JmaISDcRqeU1ETyE6xGy0OewjIkrlvRNLOkEbMA1EXQDrlbVEzXvGGNKwJp3jDEmgVhN3xhjEkjMDbhWr149TUlJ8TsMY4wpUxYvXrxDVQvr5gzEYNJPSUkhMzPT7zCMMaZMEZGiriwHrHnHGGMSiiV9Y4xJIJb0jTEmgcRcm74xJrp+/vlnsrKyOHz4sN+hmBBUrlyZRo0aUaHCiYZeKpwlfWMSXFZWFjVq1CAlJQU3uKmJVarKzp07ycrKolmzZkVvEETcNO9MnQopKZCU5P5OLdZ038YkrsOHD5OcnGwJvwwQEZKTk0v1qywuavpTp8LgwXDwoFvevNktA/Qr9diCxsQ/S/hlR2nfq7io6T/44PGEn+vgQbfeGGPMcXGR9L/9tnjrjTGxY+fOnbRt25a2bdvSoEEDGjZsmLd89Ghow+7fcsstrF27ttAykydPZmqY2n07derEsmXLwrKvaIuL5p0mTVyTTrD1xpjwmjrV/Yr+9lv3GXv00dI1oyYnJ+cl0Icffpjq1atz//335yujqqgqSUnB66mvvfZakce58847Sx5kHImLmv6jj0LVqvnXVa3q1htjwif3/NnmzaB6/PxZJDpOrF+/ntatWzNkyBDS09PZtm0bgwcPJiMjg1atWjFu3Li8srk17+zsbGrXrs2IESNo06YN5513Hj/++CMAo0aN4umnn84rP2LECNq3b88ZZ5zBF198AcCBAwf4zW9+Q5s2bejbty8ZGRlF1ujfeOMNzjrrLFq3bs3IkSMByM7O5sYbb8xbP2nSJACeeuopUlNTadOmDf379w/7axaKuEj6/frBlCnQtCmIuL9TpthJXGPCLdrnz1avXs3AgQNZunQpDRs2ZMKECWRmZrJ8+XI++eQTVq9e/Ytt9uzZQ+fOnVm+fDnnnXcer776atB9qyoLFy7kiSeeyPsCefbZZ2nQoAHLly9nxIgRLF26tND4srKyGDVqFHPmzGHp0qX85z//4f3332fx4sXs2LGDr776ipUrV3LTTTcB8Pjjj7Ns2TKWL1/Oc889V8pXp2RCSvrejEZrRWS9iIwI8vgAEdkuIsu826CAx24WkXXe7eZwBh+oXz/YtAlyctxfS/jGhF+0z5+deuqpnHPOOXnLb731Funp6aSnp7NmzZqgSb9KlSpcccUVAJx99tls2rQp6L579er1izLz5s2jT58+ALRp04ZWrVoVGt+CBQu45JJLqFevHhUqVOCGG25g7ty5nHbaaaxdu5Z77rmH2bNnU6tWLQBatWpF//79mTp1aokvriqtIpO+N1fpZOAKIBXoKyKpQYq+raptvdvL3rZ1cRNAnwu0B8aISJ2wRW+MiaoTnSeL1PmzatWq5d1ft24dzzzzDJ9++ikrVqygW7duQfurV6xYMe9+uXLlyM7ODrrvSpUq/aJMcSeVOlH55ORkVqxYQadOnZg0aRK33XYbALNnz2bIkCEsXLiQjIwMjh07VqzjhUMoNf32wHpV3eDNYD8N6Bni/i8HPlHVn1R1F/AJbho8Y0wZ5Of5s71791KjRg1q1qzJtm3bmD17dtiP0alTJ6ZPnw7AV199FfSXRKAOHTowZ84cdu7cSXZ2NtOmTaNz585s374dVeXaa69l7NixLFmyhGPHjpGVlcUll1zCE088wfbt2zlYsK0sCkLpvdMQ2BKwnIWruRf0GxG5EPgvcK+qbjnBtg0Lbigig4HBAE2sy40xMSu32TScvXdClZ6eTmpqKq1bt6Z58+Z07Ngx7Me46667uOmmm0hLSyM9PZ3WrVvnNc0E06hRI8aNG8dFF12EqtK9e3euuuoqlixZwsCBA1FVRIQ//OEPZGdnc8MNN7Bv3z5ycnIYPnw4NWrUCPtzKEqRc+SKyLXA5ao6yFu+EWivqncFlEkG9qvqEREZAlynqpeIyO+BSqr6iFfuIeCgqv7xRMfLyMhQm0TFmOhZs2YNLVu29DuMmJCdnU12djaVK1dm3bp1dO3alXXr1lG+fGz1bg/2nonIYlXNKGrbUJ5JFtA4YLkRsDWwgKruDFj8E/CHgG0vKrDtZyEc0xhjom7//v106dKF7OxsVJWXXnop5hJ+aYXybBYBLUSkGfAd0Ae4IbCAiJyiqtu8xR7AGu/+bOCxgJO3XYEHSh21McZEQO3atVm8eLHfYURUkUlfVbNFZCgugZcDXlXVVSIyDshU1ZnA3SLSA8gGfgIGeNv+JCLjcV8cAONU9acIPA9jjDEhCOl3i6rOAmYVWDc64P4DnKAGr6qvAsGvjjDGGBNVcXFFrjHGmNBY0jfGmARiSd8Y46uLLrroFxdaPf3009xxxx2Fble9enUAtm7dSu/evU+476K6gD/99NP5LpK68sor2b17dyihF+rhhx9m4sSJpd5PuFnSN8b4qm/fvkybNi3fumnTptG3b9+Qtv/Vr37FjBkzSnz8gkl/1qxZ1K5du8T7i3WW9I0xvurduzfvv/8+R44cAWDTpk1s3bqVTp065fWbT09P56yzzuK99977xfabNm2idevWABw6dIg+ffqQlpbG9ddfz6FDh/LK3X777XnDMo8ZMwaASZMmsXXrVi6++GIuvvhiAFJSUtixYwcATz75JK1bt6Z169Z5wzJv2rSJli1b8tvf/pZWrVrRtWvXfMcJZtmyZXTo0IG0tDSuueYadu3alXf81NRU0tLS8gZ6+/e//503iUy7du3Yt29fiV/bYOLrqgNjTKn87ncQ7gmh2rYFL18GlZycTPv27fnoo4/o2bMn06ZN4/rrr0dEqFy5Mu+++y41a9Zkx44ddOjQgR49epxwntgXXniBqlWrsmLFClasWEF6enreY48++ih169bl2LFjdOnShRUrVnD33Xfz5JNPMmfOHOrVq5dvX4sXL+a1115jwYIFqCrnnnsunTt3pk6dOqxbt4633nqLP/3pT1x33XW88847hY6Pf9NNN/Hss8/SuXNnRo8ezdixY3n66aeZMGECGzdupFKlSnlNShMnTmTy5Ml07NiR/fv3U7ly5WK82kWzmr4xxneBTTyBTTuqysiRI0lLS+PSSy/lu+++44cffjjhfubOnZuXfNPS0khLS8t7bPr06aSnp9OuXTtWrVpV5GBq8+bN45prrqFatWpUr16dXr168fnnnwPQrFkz2rZtCxQ+fDO48f13795N586dAbj55puZO3duXoz9+vXjjTfeyLvyt2PHjgwbNoxJkyaxe/fusF8RbDV9Y0yewmrkkXT11VczbNgwlixZwqFDh/Jq6FOnTmX79u0sXryYChUqkJKSEnQ45UDBfgVs3LiRiRMnsmjRIurUqcOAAQOK3E9h45LlDssMbmjmopp3TuSDDz5g7ty5zJw5k/Hjx7Nq1SpGjBjBVVddxaxZs+jQoQP//Oc/OfPMM0u0/2Cspm+M8V316tW56KKLuPXWW/OdwN2zZw8nnXQSFSpUYM6cOWwONhl2gAsvvDBv8vOVK1eyYsUKwA3LXK1aNWrVqsUPP/zAhx9+mLdNjRo1grabX3jhhfz973/n4MGDHDhwgHfffZcLLrig2M+tVq1a1KlTJ+9Xwl//+lc6d+5MTk4OW7Zs4eKLL+bxxx9n9+7d7N+/n2+++YazzjqL4cOHk5GRwddff13sYxbGavrGmJjQt29fevXqla8nT79+/ejevTsZGRm0bdu2yBrv7bffzi233EJaWhpt27alffv2gJsFq127drRq1eoXwzIPHjyYK664glNOOYU5c+bkrU9PT2fAgAF5+xg0aBDt2rUrtCnnRF5//XWGDBnCwYMHad68Oa+99hrHjh2jf//+7NmzB1Xl3nvvpXbt2jz00EPMmTOHcuXKkZqamjcLWLgUObRytNnQysZElw2tXPaUZmhla94xxpgEYknfGGMSiCV9Y0yxJwQ3/inte2VJ35gEV7lyZXbu3GmJvwxQVXbu3FmqC7ZC6r0jIt2AZ3CTqLysqhNOUK438DfgHFXNFJEU3Cxaa70i81V1SImjNcaEXaNGjcjKymL79u1+h2JCULlyZRo1alTi7YtM+iJSDpgMXIab83aRiMxU1dUFytUA7gYWFNjFN6ratsQRGmMiqkKFCjRr1szvMEyUhNK80x5Yr6obVPUoMA3oGaTceOBxoPDL3IwxxvgmlKTfENgSsJzlrcsjIu2Axqr6fpDtm4nIUhH5t4gEvZxNRAaLSKaIZNpPTGOMiZxQkn6w4ezyzviISBLwFHBfkHLbgCaq2g4YBrwpIjV/sTPVKaqaoaoZ9evXDy1yY4wxxRZK0s8CGgcsNwK2BizXAFoDn4nIJqADMFNEMlT1iKruBFDVxcA3wOnhCNwYY0zxhZL0FwEtRKSZiFQE+gAzcx9U1T2qWk9VU1Q1BZgP9PB679T3TgQjIs2BFsCGsD8LY4wxISmy946qZovIUGA2rsvmq6q6SkTGAZmqOrOQzS8ExolINnAMGKKqP4UjcGOMMcVnA64ZY0wcsAHXjDHG/IIlfWOMSSCW9I2JohhrTTUJyJK+MVGybx+0agXDh1vyN/6x6RKNiZJnnoE1a9ytbl2X/I2JNkv6xkTBrl0wcSL8+tdQsyaMGAEnnwwDBvgdmUk0lvSNiYInn4Q9e+CRR6BlS9i+HQYNgvr14aqr/I7OJBJr0zcmwnbsgKefhmuvhTZtoGJFeOcdaNvWrfvyS78jNInEkr4xEfbEE3DgADz88PF1NWrArFnQsKFr8lmzxrfwTIKxpG9MBH3/PTz7LPTrB6mp+R876SSYPRsqVIDLL4esLH9iNInFkr4xETRhAhw9CmPGBH+8eXP48EPYvRu6dXMnfI2JJEv6xkRIVha8+CLcfDOcdtqJy7VrB3//O6xbBz16wKFD0YvRJB5L+sZEyKOPQk4OPPRQ0WUvuQT++lf4z3+gb1/Izo58fCYxWdI3JgI2bYJXXnHdMlNSQtvmuuvcBVzvvQd33GFX7ZrIsH76xkTA+PGQlAQPPli87e66y538fewxOOUUGDs2MvGZxGVJ35gwW7cOXn8dhg51XTKL65FHXOIfNw4aNIDbbw9/jCZxhdS8IyLdRGStiKwXkRGFlOstIioiGQHrHvC2Wysil4cjaGNi2dixUKmSG2qhJETgpZege3e4806YMSO88ZnEVmTS9+a4nQxcAaQCfUUkNUi5GsDdwIKAdam4OXVbAd2A53PnzDUmHq1eDW++6Wr5DRqUfD/ly8O0adChg+vj/9lnYQvRJLhQavrtgfWqukFVjwLTgJ5Byo0HHgcOB6zrCUxT1SOquhFY7+3PmLg0ZgxUqwa//33p91W1Krz/Ppx6KvTsCcuXl36fxoSS9BsCWwKWs7x1eUSkHdBYVd8v7rbe9oNFJFNEMrdv3x5S4MbEmmXLXFPMvfdCvXrh2Wfduu6q3Zo13cVbGzeGZ78mcYWS9CXIurzOZCKSBDwF3FfcbfNWqE5R1QxVzahfv34IIRkTe8aMgdq1Ydiw8O63cWP46CM4csQN12D1IlMaoST9LKBxwHIjYGvAcg2gNfCZiGwCOgAzvZO5RW1rTFxYuBBmzoT77nOJP9xatXJNPVu2uKGY9+8P/zFMYggl6S8CWohIMxGpiDsxOzP3QVXdo6r1VDVFVVOA+UAPVc30yvURkUoi0gxoASwM+7MwxmejR0NyMtxzT+SOcf758PbbsHgx9O4NP/8cuWOZ+FVk0lfVbGAoMBtYA0xX1VUiMk5EehSx7SpgOrAa+Ai4U1WPlT5sY2LHvHmu3X34cDdkciT16AFTprjj3XqrG+bBmOIQjbFrvTMyMjQzM9PvMIwJ2cUXu/HwN2xwPW6i4dFHYdQouP9+N16/MSKyWFUziipnV+QaUwqffur60D/zTPQSPsDIke6q3YkT3fUA9wXrRmHKnOxsd41GJFnSN6aEVN0Imo0aweDB0T22iJuC8YcfXG3/5JOhf//oxmBK59gxWLUKFiyA+fPdLbenViRZ0jemhD76CL74Al54ASpXjv7xy5VzwzHv2AG33OImWb/cBjqJWT/8kD/BL1p0vBdWcrK7+rpLl8jHYW36xpSAKrRv7xLu2rVusnO/7NkDF13kBnqbMwfOOce/WIxz5Ii7WC83wc+f74bbBtd807atS/Lnnuv+nnqq+/VWGtamb0wEzZwJmZnw6qv+JnyAWrXclIvnnw9XXukmYjn9dH9jSiSqsHlz/gS/dKmbJhNck02HDm48pg4dID0dqlTxL16r6RtTTDk5rqZ2+LAbYC3SJ95CtW4ddOzoxv754gs3Hr8Jv3373Bd+YJL/8Uf3WJUq7pdWbg3+3HNLNrx2SVhNP0pUXRIIvJUr53/tz0TOjBnw1VcwdWrsJHyAFi1g1izX1HPFFfDvf7tfAabkcnLg66/zJ/hVq45fH3H66W5MpA4d3K11a6hQwd+YixI3Nf1du9yLn5t4gyXj4txC3T6YatXclHfROCljouvYMffBTkqCFSvcF3ys+fhjN1RDp06u2cePk8xl1Y4d+U+2LlwIe/e6x2rXPl6D79DBndOpW9ffeAMlXE0/Kcm9AUlJ7oRIUlLxbyXdruC2f/6z6z63YoXrUWHix5tvuprf3/4WmwkfoGtXN3NXv37u//Dtt2M3Vr+pui/JN95wSX79erc+KQnS0uCGG44n+RYt3PqyLm5q+rFkxQpXC7j0UvjHP0p/Vt7Ehp9/hpYt3VALixfHfgJ46ik34ucdd8Bzz9n/YUFz57o5jOfNc5WzTp2Ot8OffTZUr+53hMWTcDX9WJKW5i6Nv/tu92G76y6/IzLh8Je/wDffuJ47sZ7wwY3rv22b+19s0MBdSGZc//hRo1wN/5RT4PnnYeDAxDkPZzX9CFF1g2N9/LFrF2zTxu+ITGkcOeJO2jVo4JoBykqtOSfHXbj1l7+4gdp++1u/I/LPypXui+/vf3cXQz3wgPsV5Gf3yXAKtaZfBuorZZOI68OdnAx9+sDBg35HZErjlVfg229h/Piyk/DB/SJ5+WXXm2fIENfBINGsX+/Ob6SlubGSxo51g+Pdd1/8JPzisKQfQfXru8vk1651P7VN2XTokBvVslMnuOwyv6MpvgoV3InnjAxXAZk3z++IomPLFvfL5swz4d134X/+x003OXq0m34yUVnSj7AuXdw/25Qp8M47fkdjSuLFF2Hr1rJXyw9UrRp88AE0bQrdu7umjnj1ww9uMpvTTnPNWnfc4Wr2EybEVhdLv4SU9EWkm4isFZH1IjIiyONDROQrEVkmIvNEJNVbnyIih7z1y0TkxXA/gbJg/Hh3ld6gQa6JwJQdBw64ZNGli7voqSyrV89NvlKlirumZepU2L3b76jCZ9cuN+R08+YweTLceCP8978waZI7F2M8qlroDSgHfAM0ByoCy4HUAmVqBtzvAXzk3U8BVhZ1jMDb2WefrfFo/XrV6tVVL7hANTvb72iiZ80a1Y8/9juKkpswQRVUv/jC70jCZ/ly1SZN3PMqX171sstUn39eNSvL78hKZu9e1fHjVWvVcs+pb1/VtWv9jir6gEwNIceGUtNvD6xX1Q2qehSYBvQs8MWxN2CxGhBbXYJiwKmnuq5hn3/u2ocTwddfu3bwrl3dSbPsbL8jKp69e+Hxx91J0PPO8zua8ElLc23b8+e792XzZtcE0qiR66c+YYI7DxXrDh2CJ590NfuHHoLOnWH5cncBnQ04V4iivhWA3sDLAcs3As8FKXcn7hfBFqCFHq/pHwCWAv8GLjjBMQYDmUBmkyZNIv+V6KP+/VWTklTnzfM7ksjaskW1cWPVk05SHTjQ1cAuu0x1506/Iwvd2LEu7sxMvyOJvNWrVR97TPWcc9xzBtUzz1R94AHVBQtUjx3zO8LjjhxRfeEF1YYNXZyXXqo6f77fUfmPEGv6oST9a4Mk/WcLKX8D8Lp3vxKQ7N0/2/tCqFnY8eK1eSfXnj2qzZu7n9e7dvkdTWTs3Kmamqpao4bqkiVu3auvqlas6J77V1/5G18odu5UrVlT9eqr/Y4k+rZsUX3uOdUuXVTLlXNZomFD1TvuUP3kE9WjR/2JKztb9fXXVZs1czGdf77qnDn+xBKLwpn0zwNmByw/ADxQSPkkYM8JHvsMyCjsePGe9FVdzal8edVrr1XNyfE7mvA6cMB9GCtWVP300/yPffml6imnqFarpvrOO/7EF6qRI92nY/lyvyPx108/qf7lL6q9eqlWrepek9q13S/WGTNU9+2LfAzHjqn+7W+qLVu647dtq/rBB/H32SmtcCb98sAGoBnHT+S2KlCmRcD97rkHB+oD5bz7zYHvgLqFHS8Rkr6q6v/+r3v1X37Z70jC5+hR1auuUhVxH9JgvvtOtUMH99wfeii2mg1y/fij+2K6/nq/I4ktBw+qvvee6oABqsnJ7j2sXFm1e3fVV15xr1s45eSozpqlmp5+vLlp+vTY/J+JBWFL+m5fXAn812uzf9BbNw7o4d1/BlgFLAPm5H4pAL/x1i8HlgDdizpWoiT9Y8dUL7nE1Z7WrPE7mtLLyVG9+Wb3H/XCC4WXPXxY9dZbXdkePVyTVyy5/3533iUe3pdI+fln17Ryzz3HewIlJaleeKHqU0+pbtxYuv1/9plqx45uvykpqn/+c2L1eiuJsCb9aN4SJemrulpvcrL7uXr4sN/RlM7vf+/+mx5+OLTyOTmqzz7r2oxbtoydLnZbt7ra6403+h1J2ZGT487dPPSQ6llnad6J4LZt3f/D8uWhN8UsXOhO+INrCnz+eXfi1hTNkn4ZMXOmexfuvdfvSEpu4kT3HO64o/jtrHPmqNar5/pYz5oVkfCK5a673BfR+vV+R1J2rVvn/icldQ45AAAS4klEQVQ6dnRNfeBO4A8bpjp3bvAa+4oVqj17urLJyW77gwejH3tZZkm/DBk61L0TsZD0iuv1113svXuX/Of3pk2uVijiLoby6wTdt9+6E9CDBvlz/Hj0/feqU6aoXnmle21BtX591433/fdVV65UveEG997XrKk6bpy72MoUnyX9MuTQIfezuH591W3b/I4mdB984GrFl1xS+uapAwfciVNQ7dPHLUfb4MGqFSq4LyETfnv2qL79tnt/a9TQvGagKlVUhw8vW9dwxKJQk76Npx8jVq92oyBecIGb1zTWJ+n48ks3Hk3LljBnTnhGLVR1V8A+8ICbf+DddyElpfT7DcWGDXDGGXDbbW7iGxNZR464/5vVq92UhDY2TunZePplTGqqm97u44/d31i2erWbePtXv4JZs8I3TK0IDB/uRoPcuNENUvfZZ+HZd1HGj4fy5d2AXSbyKlVyg74NG2YJP9os6ceQwYOhVy9X01282O9ogtuyBS6/3H1oP/4YTj45/Me44go321j9+m6e4eeec78CImXtWjcE7+23uy8yY+KZJf0YIgJ/+pNLpH37wv79fkeU386dbvC0vXvho4/cQFeRcvrpbkCwK690cwwPGuSaBCJh7FioXBlG/GLQcGPijyX9GFO3LrzxhpviLZYmVD9wwDXpbNzoJgaPxpy/NWu6+UwfeshNPdm5s5vMJJxWroRp09wk9iedFN59GxOLLOnHoM6dYdQo+POfXULy288/w7XXwqJF8NZbLr5oSUqCcePcrGMrV7qT3fPnh2//Y8ZA9epw//3h26cxscySfowaPRrOP9/1Jtm40b84cnLg1ltdj6IXX4RrrvEnjl69XI+hKlXcl86rr5Z+n0uXwv/9n5u/ODm59PszpiywpB+jypd309mJuC5tP/8c/RhU4fe/d81N48e7Sab9dNZZ7tfGhRfCwIGu+as0r8vo0VCnjk1abxKLJf0YlpICL73kmjPGjo3+8Z94ws1MNHQoPPhg9I8fTN267lfHsGGuV0/XrrB9e/H3M38+vP++a9apXTv8cRoTqyzpx7jrr4dbboHHHoten3Vw5xOGD3fHf+YZ94sjVpQvD3/8o+tm+eWXrj//smXF28fo0W6i8LvvjkyMxsQqS/plwKRJ0KIF9O/vuk1G2j/+4bpIXnopvP567F4dfOONMG8eHDvmzn+8/XZo282dC5984rpoVq8e2RiNiTUx+nE2gapXd71mfvzRJeNIXqj0n//AdddBu3buJGelSpE7VjhkZEBmJqSnQ58+LpEfO3bi8qquC2iDBu5iLGMSjSX9MiI9HSZMcP3WX3wxMsdYuRJ+/Wto3NgNr1CjRmSOE24nnwyffup6Ov3hD9C9O+zeHbzsv/7lavojR0LVqtGN05hYEFLSF5FuIrJWRNaLyC+uWxSRISLylYgsE5F5IpIa8NgD3nZrReTycAafaH73OzcEwrBhLkGH0+bNbt9VqrjhFerXD+/+I61iRfdl+OKLrummfXtYsyZ/mdxafuPGbsgLYxJRkUlfRMoBk4ErgFSgb2BS97ypqmepalvgceBJb9tUoA/QCugGPO/tz5RAUpJrY69Z0w3TcOhQePa7Y4dL+AcOuOEVojWyZSTcdpsbvXHPHjj3XHf1cK5Zs1yvnVGjYr/ZyphICaWm3x5Yr6obVPUoMA3oGVhAVfcGLFYDcludewLTVPWIqm4E1nv7MyV08sku8a9c6frQl9b+/W58m82b3QnctLTS79NvnTq5dv7TT4eePd01Bjk5rsdOs2auN5QxiSqUpN8Q2BKwnOWty0dE7hSRb3A1/buLue1gEckUkcztJel0HSOmTnW15KQk93fq1MgcJ3dI2smT89dki+voUfjNb9yInm+/7cbyjxeNG8Pnn7seT6NHu26dS5a4YRcqVPA7OmP8E0rSD9ZD+xf9R1R1sqqeCgwHRhVz2ymqmqGqGfXLWmOyZ+pU1068ebNrO9682S1HKvE/9pjrYXPLLfDdd8XfPicHBgxw7fdTpkCPHmEP0XdVqri+/H/8o+vHf/rp0K+f31EZ469Qkn4W0DhguRFQ2FiH04CrS7htmfXgg3DwYP51Bw9G7krWSpVcN87Dh11/9cK6KRak6n4pvPWW+/IYODAyMcYCEfdcFy92V/KWL+93RMb4K5SkvwhoISLNRKQi7sRsvkYFEWkRsHgVsM67PxPoIyKVRKQZ0AJYWPqwY8+33xZvfTiccQY8+6w7cfn446FvN2GCu8r2nnsSZwz5tm0jO/6/MWVFkUlfVbOBocBsYA0wXVVXicg4EcltFBgqIqtEZBkwDLjZ23YVMB1YDXwE3KmqxaiTlh1NmhRvfbjccosbKuGhh0IbcviVV1wf9RtucOPqxNLwCsaYyLOJ0cMkt00/sImnalXXXh7pduTdu11NNinJDRdcq1bwcu+954YovuwydwK4YsXIxmWMiR6bGD3K+vVzCb5pU1d7bto0Ogkf3CiRb77pmpLuuCP4MA2ff+6GKcjIgBkzLOEbk6gs6YdRv36waZPrGbNpU3R7ipx/vuuO+Oab8Ne/5n9sxQo3NEHTpvDBBzbImDGJzJJ+HBk50k0wcuedbo5dcLNudevmEv3HH7vhhI0xicuSfhwpV87NclWhghum4bvv3PAKhw/D7NmRP6lsjIl91ms5zjRu7Hro9OoFqaluOsF//hNatfI7MmNMLLCafhy65ho3VvyBAzB9umvvN8YYsKQftyZPhq1b3fj4xhiTy5J+nBKBk07yOwpjTKyxpG+MMQnEkr4xxiQQS/rGGJNALOkbY0wCsaRvjDEJxJK+McYkEEv6xhiTQEJK+iLSTUTWish6EfnFXEsiMkxEVovIChH5l4g0DXjsmIgs826lmMbbGGNMaRU59o6IlAMmA5fh5rxdJCIzVXV1QLGlQIaqHhSR24HHgeu9xw6patswx22MMaYEQqnptwfWq+oGVT2Km/i8Z2ABVZ2jqrlzRs3HTYBujDEmxoSS9BsCWwKWs7x1JzIQ+DBgubKIZIrIfBG5OtgGIjLYK5O5ffv2EEIyxhhTEqEMrRxs6uygE+uKSH8gA+gcsLqJqm4VkebApyLylap+k29nqlOAKeDmyA0pcmOMMcUWSk0/C2gcsNwI2FqwkIhcCjwI9FDVI7nrVXWr93cD8BnQrhTxGmOMKYVQkv4ioIWINBORikAfIF8vHBFpB7yES/g/BqyvIyKVvPv1gI5A4AlgY4wxUVRk0lfVbGAoMBtYA0xX1VUiMk5EenjFngCqA38r0DWzJZApIsuBOcCEAr1+TARMnQopKZCU5P5Onep3RMaYWCGqsdWEnpGRoZmZmX6HUWZNnQqDB8PBg8fXVa0KU6ZAv37+xWWMiSwRWayqGUWVsyty48yDD+ZP+OCWH3zQn3iMMbHFkn6c+fbb4q03xiQWS/pxpkmT4q03xiQWS/px5tFHXRt+oKpV3XpjjLGkH2f69XMnbZs2dZOjN21qJ3GNMceFckWuKWP69bMkb4wJzmr6xhiTQCzpG2NMArGkb4wxCcSSvjHGJBBL+sYYk0As6ZuIsYHfjIk91mXTRETBgd82b3bLYN1JjfGT1fRNRNjAb8bEJkv6JiJs4DdjYlNISV9EuonIWhFZLyIjgjw+TERWi8gKEfmXiDQNeOxmEVnn3W4OZ/AmdtnAb8bEpiKTvoiUAyYDVwCpQF8RSS1QbCmQoappwAzgcW/busAY4FygPTBGROqEL3wTq2zgN2NiUyg1/fbAelXdoKpHgWlAz8ACqjpHVXNbcOfjJk8HuBz4RFV/UtVdwCdAt/CEbmKZDfxmTGwKpfdOQ2BLwHIWruZ+IgOBDwvZtmFxAjRllw38ZkzsCSXpS5B1QSfWFZH+QAbQuTjbishgYDBAE2v0NcaYiAmleScLaByw3AjYWrCQiFwKPAj0UNUjxdlWVaeoaoaqZtSvXz/U2I0xxhRTKEl/EdBCRJqJSEWgDzAzsICItANewiX8HwMemg10FZE63gncrt46Y4wxPiiyeUdVs0VkKC5ZlwNeVdVVIjIOyFTVmcATQHXgbyIC8K2q9lDVn0RkPO6LA2Ccqv4UkWdijDGmSKIatHneNxkZGZqZmel3GMYYU6aIyGJVzSiqnF2Ra4wxCcSSvjHGJBBL+sYYk0As6RtjTAKxpG/ink3mYsxxNomKiWs2mYsx+VlN38Q1m8zFmPws6Zu4ZpO5GJOfJX0T12wyF2Pys6Rv4ppN5mJMfpb0TVyzyVyMyc9675i4Z5O5GHOc1fSNMSaBWNI3JkrsIjETC6x5x5gosIvETKywmr4xUWAXiZlYEVLSF5FuIrJWRNaLyIggj18oIktEJFtEehd47JiILPNuMwtua0wisIvETKwosnlHRMoBk4HLcBOdLxKRmaq6OqDYt8AA4P4guzikqm3DEKsxZVaTJq5JJ9h6Y6IplJp+e2C9qm5Q1aPANKBnYAFV3aSqK4CcCMRoTJlnF4mZWBFK0m8IbAlYzvLWhaqyiGSKyHwRuTpYAREZ7JXJ3L59ezF2bUzZEEsXiVkvosQWSu8dCbKuOLOpN1HVrSLSHPhURL5S1W/y7Ux1CjAF3MToxdi3MWVGLFwkZr2ITCg1/SygccByI2BrqAdQ1a3e3w3AZ0C7YsRnjAkj60VkQkn6i4AWItJMRCoCfYCQeuGISB0RqeTdrwd0BFYXvpUxJlKsF5EpMumrajYwFJgNrAGmq+oqERknIj0AROQcEckCrgVeEpFV3uYtgUwRWQ7MASYU6PVjjIkiG2rahHRFrqrOAmYVWDc64P4iXLNPwe2+AM4qZYzGmDB59NH8bfpgvYgSjV2Ra0wCiZVeRNaDyD829o4xCcbvXkTWg8hfVtM3xkSV9SDylyV9Y0xUWQ8if1nSN8ZElfUg8pclfWNMVNk4RP6ypG+MiapY6UEEidmLyHrvGGOizu8eRJC4vYispm+MSUiJ2ovIkr4xJiElai8iS/rGmISUqL2ILOkbYxJSLPUiiuYJZUv6xpiEFCu9iHJPKG/eDKrHTyhHKvGLamxNVJWRkaGZmZl+h2GMMVGRkuISfUFNm8KmTaHvR0QWq2pGUeWspm+MMT6K9gnlkJK+iHQTkbUisl5ERgR5/EIRWSIi2SLSu8BjN4vIOu92c7gCN8aYeBDtE8pFJn0RKQdMBq4AUoG+IpJaoNi3wADgzQLb1gXGAOcC7YExIlKn9GEbY0x8iPYJ5VBq+u2B9aq6QVWPAtOAnoEFVHWTqq4Acgpseznwiar+pKq7gE+AbmGI2xhj4kK0TyiHMgxDQ2BLwHIWruYeimDbNixYSEQGA4MBmsR7J1ljjCkgmsNShFLTlyDrQu3yE9K2qjpFVTNUNaN+/foh7toYY0xxhZL0s4DGAcuNgK0h7r802xpjjAmzUJL+IqCFiDQTkYpAH2BmiPufDXQVkTreCdyu3jpjjDE+KDLpq2o2MBSXrNcA01V1lYiME5EeACJyjohkAdcCL4nIKm/bn4DxuC+ORcA4b50xxhgf2BW5xhgTB0K9Ijfmkr6IbAeCXJQcsnrAjjCFU9bZa5GfvR752etxXDy8Fk1VtcieMDGX9EtLRDJD+bZLBPZa5GevR372ehyXSK+Fjb1jjDEJxJK+McYkkHhM+lP8DiCG2GuRn70e+dnrcVzCvBZx16ZvjDHmxOKxpm+MMeYELOkbY0wCiZukX9REL4lERBqLyBwRWSMiq0TkHr9j8puIlBORpSLyvt+x+E1EaovIDBH52vsfOc/vmPwkIvd6n5OVIvKWiFT2O6ZIioukH+JEL4kkG7hPVVsCHYA7E/z1ALgHN4yIgWeAj1T1TKANCfy6iEhD4G4gQ1VbA+Vw44vFrbhI+oQw0UsiUdVtqrrEu78P96H+xTwGiUJEGgFXAS/7HYvfRKQmcCHwCoCqHlXV3f5G5bvyQBURKQ9UJc5HAo6XpB/SZC2JSERSgHbAAn8j8dXTwP/wy5ndElFzYDvwmtfc9bKIVPM7KL+o6nfARNyUr9uAPar6sb9RRVa8JP3STPQSt0SkOvAO8DtV3et3PH4QkV8DP6rqYr9jiRHlgXTgBVVtBxwAEvYcmDfke0+gGfAroJqI9Pc3qsiKl6Rvk7UUICIVcAl/qqr+n9/x+Kgj0ENENuGa/S4RkTf8DclXWUCWqub+8puB+xJIVJcCG1V1u6r+DPwfcL7PMUVUvCT90kz0EndERHBttmtU9Um/4/GTqj6gqo1UNQX3f/GpqsZ1Ta4wqvo9sEVEzvBWdQFW+xiS374FOohIVe9z04U4P7EdysToMU9Vs0Ukd6KXcsCrqrrK57D81BG4EfhKRJZ560aq6iwfYzKx4y5gqldB2gDc4nM8vlHVBSIyA1iC6/W2lDgfksGGYTDGmAQSL807xhhjQmBJ3xhjEoglfWOMSSCW9I0xJoFY0jfGmARiSd8YYxKIJX1jjEkg/w9vPq3feGhK2QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(len(acc))\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a reminder, in chapter 3, our very first naive approach to this very dataset got us to 88% test accuracy. Unfortunately, our small \n",
    "recurrent network doesn't perform very well at all compared to this baseline (only up to 85% validation accuracy). Part of the problem is \n",
    "that our inputs only consider the first 500 words rather the full sequences -- \n",
    "hence our RNN has access to less information than our earlier baseline model. The remainder of the problem is simply that `SimpleRNN` isn't very good at processing long sequences, like text. Other types of recurrent layers perform much better. Let's take a look at some \n",
    "more advanced layers.\n",
    "\n",
    "提醒一下，在第 3 章，处理这个数据集的第一个简单方法得到的测试精度是 88%。不幸的是， 与这个基准相比，这个小型循环网络的表现并不好（验证精度只有 85%）。问题的部分原因在于， 输入只考虑了前 500 个单词，而不是整个序列，因此，RNN 获得的信息比前面的基准模型更少。 另一部分原因在于，SimpleRNN 不擅长处理长序列，比如文本。\n",
    "\n",
    "其他类型的循环层的表现要好得多。我们来看几个更高级的循环层。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[...]\n",
    "\n",
    "## A concrete LSTM example in Keras\n",
    "\n",
    "Now let's switch to more practical concerns: we will set up a model using a LSTM layer and train it on the IMDB data. Here's the network, \n",
    "similar to the one with `SimpleRNN` that we just presented. We only specify the output dimensionality of the LSTM layer, and leave every \n",
    "other argument (there are lots) to the Keras defaults. Keras has good defaults, and things will almost always \"just work\" without you \n",
    "having to spend time tuning parameters by hand.\n",
    "\n",
    "## Keras 中一个 LSTM 的具体例子\n",
    "\n",
    "现在我们来看一个更实际的问题：使用 LSTM 层来创建一个模型，然后在 IMDB 数据上 训练模型（见图 6-16 和图 6-17）。这个网络与前面介绍的 SimpleRNN 网络类似。你只需指定 LSTM 层的输出维度，其他所有参数（有很多）都使用 Keras 默认值。Keras 具有很好的默认值， 无须手动调参，模型通常也能正常运行。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 159s 8ms/step - loss: 0.5138 - acc: 0.7586 - val_loss: 0.3711 - val_acc: 0.8532\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 157s 8ms/step - loss: 0.2932 - acc: 0.8845 - val_loss: 0.3716 - val_acc: 0.8354\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 158s 8ms/step - loss: 0.2352 - acc: 0.9094 - val_loss: 0.2759 - val_acc: 0.8944\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 152s 8ms/step - loss: 0.2097 - acc: 0.9252 - val_loss: 0.3520 - val_acc: 0.8424\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 158s 8ms/step - loss: 0.1744 - acc: 0.9365 - val_loss: 0.2960 - val_acc: 0.8762\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 157s 8ms/step - loss: 0.1556 - acc: 0.9423 - val_loss: 0.4309 - val_acc: 0.8414\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 156s 8ms/step - loss: 0.1435 - acc: 0.9491 - val_loss: 0.3543 - val_acc: 0.8798\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 157s 8ms/step - loss: 0.1312 - acc: 0.9546 - val_loss: 0.3326 - val_acc: 0.8878\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 155s 8ms/step - loss: 0.1183 - acc: 0.9579 - val_loss: 0.3686 - val_acc: 0.8732\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 154s 8ms/step - loss: 0.1092 - acc: 0.9620 - val_loss: 0.3836 - val_acc: 0.8726\n"
     ]
    }
   ],
   "source": [
    "from keras.layers import LSTM\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_features, 32))\n",
    "model.add(LSTM(32))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='rmsprop',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(input_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VNXZwPHfQ1gi+6oia7S8yg4hBiyoFBSBKiBuIFpxQ1Swr0sVhVZfWqV1X6viggtRpCBKq0irBfeFADFsssiiAcSA7Hvgef84N2EyTJKbMJOZyTzfz2c+mXvvuWeeuUnmmXvuueeIqmKMMcZUinYAxhhjYoMlBGOMMYAlBGOMMR5LCMYYYwBLCMYYYzyWEIwxxgCWEEwAEUkSkV0i0jycZaNJRH4lImHvWy0i54jI2oDl5SJypp+yZXitF0XknrLub4xflaMdgCk7EdkVsFgd2A8c8pZvUNWM0tSnqoeAmuEumwhU9dRw1CMi1wFXqGrPgLqvC0fdxpTEEkIcU9WCD2TvG+h1qvphUeVFpLKq5pVHbMaUxP4eY481GVVgIvIXEXlLRN4UkZ3AFSJyhoh8JSLbRGSjiDwpIlW88pVFREWkpbc82ds+S0R2isiXIpJS2rLe9n4iskJEtovIUyLyuYgMLyJuPzHeICKrRGSriDwZsG+SiDwmIltE5HugbzHHZ5yITAla94yIPOo9v05Elnnv53vv23tRdeWISE/veXURed2LbQnQJcTrrvbqXSIiA7z17YGngTO95rjNAcf2voD9R3rvfYuIvCMijf0cm9Ic5/x4RORDEflFRH4SkTsDXueP3jHZISKZInJSqOY5Efks//fsHc9PvNf5BRgnIq1EZI73XjZ7x61OwP4tvPeY621/QkSSvZhbB5RrLCJ7RKRBUe/X+KCq9qgAD2AtcE7Qur8AB4ALcMn/OOB0oCvu7PBkYAUwyitfGVCgpbc8GdgMpAFVgLeAyWUoezywExjobbsNOAgML+K9+InxXaAO0BL4Jf+9A6OAJUBToAHwifszD/k6JwO7gBoBdf8MpHnLF3hlBOgF7AU6eNvOAdYG1JUD9PSePwzMBeoBLYClQWUvBRp7v5PLvRhO8LZdB8wNinMycJ/3vI8XYycgGfg78F8/x6aUx7kOsAn4PVANqA2ke9vuBr4FWnnvoRNQH/hV8LEGPsv/PXvvLQ+4EUjC/T3+D9AbqOr9nXwOPBzwfhZ7x7OGV767t20icH/A69wOzIj2/2G8P6IegD3C9IssOiH8t4T97gD+4T0P9SH/XEDZAcDiMpS9Bvg0YJsAGykiIfiMsVvA9reBO7znn+CazvK39Q/+kAqq+yvgcu95P2BFMWX/BdzsPS8uIfwQ+LsAbgosG6LexcBvveclJYRXgQcCttXGXTdqWtKxKeVxvhLILKLc9/nxBq33kxBWlxDDxcA87/mZwE9AUohy3YE1gHjLWcDgcP9fJdrDmowqvh8DF0TkNBF5z2sC2AGMBxoWs/9PAc/3UPyF5KLKnhQYh7r/4JyiKvEZo6/XAtYVEy/AG8BQ7/nlQMGFeBE5X0S+9ppMtuG+nRd3rPI1Li4GERkuIt96zR7bgNN81gvu/RXUp6o7gK1Ak4Ayvn5nJRznZsCqImJohksKZRH893iiiEwVkfVeDK8ExbBWXQeGQlT1c9zZRg8RaQc0B94rY0zGYwmh4gvucvk87hvpr1S1NvAn3Df2SNqI+wYLgIgIhT/Agh1LjBtxHyT5SuoW+xZwjog0xTVpveHFeBwwDZiAa86pC/zbZxw/FRWDiJwMPItrNmng1ftdQL0ldZHdgGuGyq+vFq5par2PuIIVd5x/BE4pYr+itu32YqoesO7EoDLB7+9vuN5x7b0YhgfF0EJEkoqI4zXgCtzZzFRV3V9EOeOTJYTEUwvYDuz2LsrdUA6v+S8gVUQuEJHKuHbpRhGKcSrwvyLSxLvAeFdxhVV1E65ZYxKwXFVXepuq4dq1c4FDInI+rq3bbwz3iEhdcfdpjArYVhP3oZiLy43X4c4Q8m0CmgZe3A3yJnCtiHQQkWq4hPWpqhZ5xlWM4o7zTKC5iIwSkaoiUltE0r1tLwJ/EZFTxOkkIvVxifAnXOeFJBEZQUDyKiaG3cB2EWmGa7bK9yWwBXhA3IX640Ske8D213FNTJfjkoM5RpYQEs/twFW4i7zP474hR5T3oXsZ8CjuH/wUYCHum2G4Y3wW+AhYBMzDfcsvyRu4awJvBMS8DbgVmIG7MHsxLrH5cS/uTGUtMIuADytVzQaeBL7xypwGfB2w73+AlcAmEQls+snf/wNc084Mb//mwDCfcQUr8jir6nbgXOAi3EXsFcDZ3uaHgHdwx3kH7gJvstcUeD1wD66Dwa+C3lso9wLpuMQ0E5geEEMecD7QGne28APu95C/fS3u93xAVb8o5Xs3IeRfkDGm3HhNABuAi1X102jHY+KXiLyGu1B9X7RjqQjsxjRTLkSkL64JYB+u22Ie7luyMWXiXY8ZCLSPdiwVhTUZmfLSA1iNa0roCwyyi4CmrERkAu5eiAdU9Ydox1NRWJORMcYYwM4QjDHGeOLqGkLDhg21ZcuW0Q7DGGPiyvz58zeranFdvYE4SwgtW7YkMzMz2mEYY0xcEZGS7tgHrMnIGGOMxxKCMcYYwBKCMcYYT1xdQwjl4MGD5OTksG/fvmiHYoqQnJxM06ZNqVKlqOF5jDGxIO4TQk5ODrVq1aJly5a4QTRNLFFVtmzZQk5ODikpKSXvYIyJmrhvMtq3bx8NGjSwZBCjRIQGDRrYGZwxZZCRAS1bQqVK7mdGRkl7HJu4P0MALBnEOPv9GFN6GRkwYgTs2eOW161zywDDyjq+bQni/gzBGGMqorFjjySDfHv2uPWRYgnhGG3ZsoVOnTrRqVMnTjzxRJo0aVKwfODAAV91XH311SxfvrzYMs888wwZkT5fNMbEjB+KGLKvqPXhkHAJIdxtcg0aNCArK4usrCxGjhzJrbfeWrBctWpVwF1YPXz4cJF1TJo0iVNPPbXY17n55psZFqnzRGPMUcq7/T5Y8yImfy1qfTgkVELIb5Nbtw5Uj7TJReIXvWrVKtq1a8fIkSNJTU1l48aNjBgxgrS0NNq2bcv48eMLyvbo0YOsrCzy8vKoW7cuY8aMoWPHjpxxxhn8/PPPAIwbN47HH3+8oPyYMWNIT0/n1FNP5Ysv3GRRu3fv5qKLLqJjx44MHTqUtLQ0srKyjort3nvv5fTTTy+IL3/E2xUrVtCrVy86duxIamoqa9euBeCBBx6gffv2dOzYkbGRPF81JkaU52dFUe6/H6pXL7yuenW3PmJUNW4eXbp00WBLly49al1RWrRQdb/ewo8WLXxXUax7771XH3roIVVVXblypYqIfvPNNwXbt2zZoqqqBw8e1B49euiSJUtUVbV79+66cOFCPXjwoAL6/vvvq6rqrbfeqhMmTFBV1bFjx+pjjz1WUP7OO+9UVdV3331XzzvvPFVVnTBhgt50002qqpqVlaWVKlXShQsXHhVnfhyHDx/WIUOGFLxeamqqzpw5U1VV9+7dq7t379aZM2dqjx49dM+ePYX2La3S/J6MibZIf1b4NXmye00R93Py5LLVA2Sqj8/YhDpDKO82uVNOOYXTTz+9YPnNN98kNTWV1NRUli1bxtKlS4/a57jjjqNfv34AdOnSpeBberDBgwcfVeazzz5jyJAhAHTs2JG2bduG3Pejjz4iPT2djh078vHHH7NkyRK2bt3K5s2bueCCCwB3M1n16tX58MMPueaaazjuuOMAqF+/fukPhDGlEO2mGohO+30ow4bB2rVw+LD7GelW4wrR7dSv5s3dqV+o9ZFQo0aNgucrV67kiSee4JtvvqFu3bpcccUVIfvm5193AEhKSiIvLy9k3dWqVTuqjPqY7GjPnj2MGjWKBQsW0KRJE8aNG1cQR6juoapq3UZNuYlGV8tQyvuzIlYk1BlCVNrkPDt27KBWrVrUrl2bjRs3Mnv27LC/Ro8ePZg6dSoAixYtCnkGsnfvXipVqkTDhg3ZuXMn06dPB6BevXo0bNiQf/7zn4C74W/Pnj306dOHl156ib179wLwyy+/hD1uY/JFo6tlKNH8rIimhEoIw4bBxInQogWIuJ8TJ5bPN4/U1FTatGlDu3btuP766+nevXvYX2P06NGsX7+eDh068Mgjj9CuXTvq1KlTqEyDBg246qqraNeuHRdeeCFdu3Yt2JaRkcEjjzxChw4d6NGjB7m5uZx//vn07duXtLQ0OnXqxGOPPRb2uI3JF0tNNdH6rIimuJpTOS0tTYMnyFm2bBmtW7eOUkSxJS8vj7y8PJKTk1m5ciV9+vRh5cqVVK4c/ZZB+z0ZP1q2DN1U06KFa0M3ZSMi81U1raRyvs4QRKSviCwXkVUiMibE9hYi8pGIZIvIXBFpGrDtkIhkeY+ZAetTRORrEVkpIm+JSNXgek3p7Nq1i+7du9OxY0cuuuginn/++ZhIBsb4lahNNbGixIQgIknAM0A/oA0wVETaBBV7GHhNVTsA44EJAdv2qmon7zEgYP3fgMdUtRWwFbj2GN6HAerWrcv8+fP59ttvyc7Opk+fPtEOycSRWOjdk6hNNbHCzxlCOrBKVVer6gFgCjAwqEwb4CPv+ZwQ2wsR122lFzDNW/UqMMhv0MaY8IqFG7HylXdXS3OEn4TQBPgxYDnHWxfoW+Ai7/mFQC0RaeAtJ4tIpoh8JSL5H/oNgG2qmt+nMlSdAIjICG//zNzcXB/hGmNKK1Z695jo8pMQQnVCD74SfQdwtogsBM4G1gP5H/bNvYsZlwOPi8gpPut0K1UnqmqaqqY1atTIR7jGmNKKld49Jrr8JIQcoFnAclNgQ2ABVd2gqoNVtTMw1lu3PX+b93M1MBfoDGwG6opI5aLqNMaUn2gMpGZij5+EMA9o5fUKqgoMAWYGFhCRhiKSX9fdwMve+noiUi2/DNAdWOqNrTEHuNjb5yrg3WN9M9HQs2fPo24ye/zxx7npppuK3a9mzZoAbNiwgYsvvjhkmZ49exLczTbY448/zp6Ac/3+/fuzbds2P6EbU8B69xjwkRC8dv5RwGxgGTBVVZeIyHgRye811BNYLiIrgBOA/D+j1kCmiHyLSwB/VdX822fvAm4TkVW4awovhek9lauhQ4cyZcqUQuumTJnC0KFDfe1/0kknMW3atJILFiE4Ibz//vvUrVu3zPWZ8me9e0zM8DMCXqw8jnW000jYvHmzNmzYUPft26eqqmvWrNFmzZrp4cOHdefOndqrVy/t3LmztmvXTt95552C/WrUqFFQvm3btqqqumfPHr3sssu0ffv2eumll2p6errOmzdPVVVHjhypXbp00TZt2uif/vQnVVV94okntEqVKtquXTvt2bOnqqq2aNFCc3NzVVX1kUce0bZt22rbtm0LRkpds2aNnnbaaXrddddpmzZt9Nxzzy0YyTTQzJkzNT09XTt16qS9e/fWn376SVVVd+7cqcOHD9d27dpp+/btddq0aaqqOmvWLO3cubN26NBBe/XqdVR90f49xarJk1WrVy88omb16mUf1dKYUPA52mmFumvpf/8XQgz/f0w6dQJvGoKQGjRoQHp6Oh988AEDBw5kypQpXHbZZYgIycnJzJgxg9q1a7N582a6devGgAEDihws7tlnn6V69epkZ2eTnZ1Nampqwbb777+f+vXrc+jQIXr37k12dja33HILjz76KHPmzKFhw4aF6po/fz6TJk3i66+/RlXp2rUrZ599NvXq1WPlypW8+eabvPDCC1x66aVMnz6dK664otD+PXr04KuvvkJEePHFF3nwwQd55JFH+POf/0ydOnVYtGgRAFu3biU3N5frr7+eTz75hJSUFBvvqBSK691j385NeUuosYwiJbDZKLC5SFW555576NChA+eccw7r169n06ZNRdbzySefFHwwd+jQgQ4dOhRsmzp1KqmpqXTu3JklS5aEHLgu0GeffcaFF15IjRo1qFmzJoMHD+bTTz8FICUlhU6dOgFFD7Gdk5PDeeedR/v27XnooYdYsmQJAB9++CE333xzQbl69erx1VdfcdZZZ5GSkgLYENmlYb17TCypUGcIxX2Tj6RBgwZx2223sWDBAvbu3VvwzT4jI4Pc3Fzmz59PlSpVaNmyZcghrwOFOntYs2YNDz/8MPPmzaNevXoMHz68xHq0mDGq8ofOBjd8dv5IpoFGjx7NbbfdxoABA5g7dy733XdfQb3BMYZaZ/xJ1GGWTWyyM4QwqFmzJj179uSaa64pdDF5+/btHH/88VSpUoU5c+awLtR/foCzzjqLDO+K4uLFi8nOzgbc0Nk1atSgTp06bNq0iVmzZhXsU6tWLXbu3BmyrnfeeYc9e/awe/duZsyYwZlnnun7PW3fvp0mTdy9gq+++mrB+j59+vD0008XLG/dupUzzjiDjz/+mDVr1gA2RHZpWO8eE0ssIYTJ0KFD+fbbbwtmLAMYNmwYmZmZpKWlkZGRwWmnnVZsHTfeeCO7du2iQ4cOPPjgg6SnpwNu9rPOnTvTtm1brrnmmkJDZ48YMYJ+/frxm9/8plBdqampDB8+nPT0dLp27cp1111H586dfb+f++67j0suuYQzzzyz0PWJcePGsXXrVtq1a0fHjh2ZM2cOjRo1YuLEiQwePJiOHTty2WWX+X6daLLePcYUZsNfm3IRa7+n4Jm5wH0ztw9jUxGFdfhrYyoaG7vHmKNZQjAJyXr3GHO0CpEQ4qnZKxHF4u/Hxu4x5mhxnxCSk5PZsmVLTH7oGJcMtmzZQnJycrRDKcR69xhztLi/D6Fp06bk5ORgcyXEruTkZJo2bVpywXKUf+F47FjXTNS8uUsGdkHZJLK472VkjDGmeNbLyBhjTKlYQjDGGANYQjDGGOPxlRBEpK+ILBeRVSIyJsT2FiLykYhki8hcEWnqre8kIl+KyBJv22UB+7wiImtEJMt7dArf2zKxLBaGjDDGHK3EXkYikgQ8A5yLm195nojM1CMznwE8DLymqq+KSC9gAnAlsAf4naquFJGTgPkiMltV8+d4/IOqln26MBN3goeMWLfOLYP18DEm2vycIaQDq1R1taoeAKYAA4PKtAE+8p7Pyd+uqitUdaX3fAPwM9AoHIGb+GRDRhgTu/wkhCbAjwHLOd66QN8CF3nPLwRqiUiDwAIikg5UBb4PWH2/15T0mIhUIwQRGSEimSKSafcaxD8bMsKY2OUnIYSa+ST45oU7gLNFZCFwNrAeyCuoQKQx8Dpwtaoe9lbfDZwGnA7UB+4K9eKqOlFV01Q1rVEjO7mIdzZkhDGxy09CyAGaBSw3BTYEFlDVDao6WFU7A2O9ddsBRKQ28B4wTlW/Cthnozf/835gEq5pylRwNmSEMbHLT0KYB7QSkRQRqQoMAWYGFhCRhiKSX9fdwMve+qrADNwF538E7dPY+ynAIGDxsbwREx9sQhhjYleJvYxUNU9ERgGzgSTgZVVdIiLjgUxVnQn0BCaIiAKfAPmzsF8KnAU0EJHh3rrhqpoFZIhII1yTVBYwMnxvy8SyYcMsARgTi2wsI2OMqeBsLCNjjDGlYgnBGGMMYAnBGGOMxxKCMcYYwBKCMcYYjyUEY4wxgCWEhGNDTxtjilLijWmm4rChp40xxbEzhARiQ08bY4pjCSGB2NDTxpjiWEJIIDb0tDGmOJYQEogNPW2MKY4lhARiQ08bY4pjvYwSjA09bYwpip0hGGOMAXwmBBHpKyLLRWSViIwJsb2FiHwkItkiMldEmgZsu0pEVnqPqwLWdxGRRV6dT3ozpxljjImSEhOCiCQBzwD9gDbAUBFpE1TsYdw0mR2A8cAEb9/6wL1AV9ycyfeKSD1vn2eBEUAr79H3mN+NMcaYMvNzhpAOrFLV1ap6AJgCDAwq0wb4yHs+J2D7ecB/VPUXVd0K/Afo682nXFtVv1Q3ZdtruHmVjTHGRImfhNAE+DFgOcdbF+hb4CLv+YVALRFpUMy+TbznxdVpjDGmHPlJCKHa9oMnYr4DOFtEFgJnA+uBvGL29VOne3GRESKSKSKZubm5PsI1xhhTFn4SQg7QLGC5KbAhsICqblDVwaraGRjrrdtezL453vMi6wyoe6KqpqlqWqNGjXyEa4wxpiz8JIR5QCsRSRGRqsAQYGZgARFpKCL5dd0NvOw9nw30EZF63sXkPsBsVd0I7BSRbl7vot8B74bh/RhjjCmjEhOCquYBo3Af7suAqaq6RETGi8gAr1hPYLmIrABOAO739v0F+DMuqcwDxnvrAG4EXgRWAd8Ds8L1powxxpSeuE4+8SEtLU0zMzOjHUZc27sXbrkFrr0WunWLdjTGmPIgIvNVNa2kcnancoJ56CF48UUYMgR27ox2NMaYWGIJIYGsWwcTJkB6upsD4Q9/iHZExphYYgkhgdx+u5tLedo0uO02eP55+M9/oh2VMSZWWEJIEB99BNOnwz33QLNm8Oc/w6mnumsJO3ZEOzpjTCywhJAADh6E0aPh5JPdWQLAccfBK6/A+vVwxx1RDc8YEyMsISSAp5+GZcvg8cchOfnI+m7dXIJ44QWYPTt68RkTKDcXPvjAXfMy5cu6nVZwmzbB//wPdO8O773nZkoLtG8fdO4Mu3bB4sVQp0504jSJSRW++w4+/9w9vvgCVqxw2ypVggsugJtvhnPOOfpv1/hn3U5jTEYGtGzp/shbtnTL5WHMGHfvweOPh/6HSk52TUcbNhxpTjImUvbuhU8/hb/+1X3YN2wIbdrA9dfDP//prmv99a+us8OYMS5J9OkDrVvDk0/C9u3RfgcVnKrGzaNLly4ajyZPVq1eXdV9H3KP6tXd+kj68kv3WnfdVXLZu+5yZd9/P7IxmcTy00+q06er3nabateuqlWqHPkfOPVU1WuuUX3pJdXvvlM9fPjo/ffuVX3tNbcvqNaooTpypOqiReX/XuIZkKk+PmOtyagctGwZuj20RQtYuzYyr3n4MHTt6r75L18ONWsWX37fPujSxX0DW7wY6taNTFym4jp8GJYuLdz88/33blu1anD66a7psnt3OOMMd3ZQGpmZ8Mwz8OabsH8/nH22a04aNAiqVAn/+6lI/DYZWUIoB5Uque9EwUTcP1EkvPiiOw3PyIDLL/e3zzffuH/Uq66Cl18uubxJbLt3w7x5RxLAl1/Ctm1uW6NGRz78u3eH1FSXFMJh82b39/nss+4L1UknwQ03uL/3xo3D8xoVjSWEGFLeZwhbt7oLya1bw8cfl+5i3D33uLuZ33sP+vcPf2wmfm3YcOTD//PPISsL8vLctjZtCieAU06J/EXgQ4fg/ffdWcPs2VC5Mlx8sTtr6N7dLkIHsoQQQzIyYMQI2LPnyLrq1WHiRBg2LPyvd8st7p9kwQLo2LF0++7f75qOtm51TUf16pW8j6l4Dh1yv//A5p/8Ly/HHeeGP+neHX79a3dWWb9+VMNlxQp3xjBpkmv27NABRo1yZ8c1akQ3tljgNyFE/UJxaR7xelFZ1V1AbtFCVcT9jNQF5exs1aQk1ZtuKnsd8+a5Oq66KmxhmTjx5puqffqo1q595OLviSeqXnSR6qOPqn79ter+/dGOsmi7dqk+/7xqhw4u9jp1VG+9VXXlymhHFl3YReXEowq/+Y37ZrdixbF9axs3Du6/33UFPP/88MVoYtfkyXDlla65sVevI2cAKSnx1/yiCp995s6Up093TVt9+7rmpH79ICkp2hGWL2sySkBvveWGtX7uOXeR7Vjs3w9pabBlCyxZYk1HFd0HH7j7As46y7XLh+sCcCzYuNE1zz7/vHuekgI33gjXXAMNGkQ7uvIR1hvTRKSviCwXkVUiMibE9uYiMkdEFopItoj099YPE5GsgMdhEenkbZvr1Zm/7fjSvklzxO7dbkyi1FS47rpjr69aNXfD2s8/w+9/f+z1xbL8C6OJ6uuv4aKLoF07mDGjYiUDcD2P7r3Xdex46y03uOOdd0LTpi4pzJ8f7QhjSEltSkASborLk4GqwLdAm6AyE4EbvedtgLUh6mkPrA5Ynguk+WnXyn/E8zWESLvnHtdm+vnn4a33j3909b77bnjrjQWHDqmOHu3ay+fOjXY00fHdd6oNGqiefLLqxo3Rjqb8ZGer3nDDkRtGu3VTff111X37oh1ZZODzGoKfM4R0YJWqrlbVA8AUYGBwXgFqe8/rABtC1DMUeNPH65lSWrUKHn7Ytf/++tfhrXvcONdj44Yb4JdfSi4fLw4dcu/pqadcd8X+/V0X3USyfr0bFiIpyXXbPPHEaEdUftq3d02r69e7YV22bHH/P82awdixbgKphFRSxgAuBl4MWL4SeDqoTGNgEZADbAW6hKjne6BdwPJcb58s4I941zNC7DcCyAQymzdvHuE8Gp/OP1+1Vi3VDRsiU/+CBaqVK6sOGxaZ+svbwYOqV17pvhmOG+eGV2jd2n1bTJQzha1bVdu1U61ZU3X+/GhHE32HDqnOnq06YIBqpUruMWiQ6gcfqK5Yobp6teoPP7izqNxc1W3bXI+m/fvdvrEOn2cIlX3kjFD9C4KvRA8FXlHVR0TkDOB1EWmnqocBRKQrsEdVFwfsM0xV14tILWC6l2heC5GwJuKapEhLS4ufK+Dl5P334V//cnMlR+ouzc6d3bem//s/uOQSGBh8fhhHDh503wTfestNEjRunFs/Z47rodW/vzumZ58d3Tgjae9eGDDADWkya5a77pToKlVyZ0t9+rj7LZ57zt3t/847/vYXccNnVK585Gfgcz/bSio/blzkz+JK7GXkfcDfp6rnect3A6jqhIAyS4C+qvqjt7wa6KaqP3vLjwG5qvpAEa8xHHc9YVRxsVgvo8L273cXApOSIDsbqlaN3GsdOOBuRvrpJ9frKB57Zxw44HphzZgBDz549JzSmza5pLBuXcVNCnl5Lqm/+y5MmQKXXhrtiGLXvn1u1NUdO9xxO3iw8E+/68K17auvoFWrsr2XsN2YBlQGVgMpHLmo3DaozCxguPe8Ne4aQn6yqYRrSjo5qM6G3vMqwDRgZEmx2EXlwiZMcM0es2eXz+stXOiajoYOLZ/eBS58AAAV6UlEQVTXC6e9e1V/+1t3vJ54ouhyFbn56PBh1euvd8fgySejHY0pT/hsMvLVuwfoD6zAXQcY660bDwzQIz2LPveSRRbQJ2DfnsBXQfXVAOYD2cAS4AkgqaQ4LCEckZPjhgIeNKh8X/f//s/91bz9dvm+7rHYvVv13HNd3M89V3L5ipoU8nuMjR0b7UhMeQtrQoiVhyWEI4YOVU1Odhe7ytOBA6qdO6sef7y7uBbrdu5U7dnTDRny8sv+96toSeHpp91/+7XXhp53wFRsfhOCzZgWhz75xI0Jf+ed7q7L8lSlirthbetWN3hYLNuxww1X8OmnbliGq6/2v+8JJ7gLzS1axH+X1H/8A0aPdheSn3su/oahMOXHEkKcyctz/9zNm8Ndd0Unhg4d4E9/cj11pk+PTgwl2boVzj3X3YU7ZYr/OSECVYSk8N//whVXuHGJpkxxvVWMKYolhDjz/POuR9Gjj7ohtKPlrrtcd8Ubb4Tc3OjFEcrmzdC7txuvf/p0N0Z+WcVzUli40M0m1qoVzJzphq02pjiWEOLI5s3wxz+6D7vBg6MbS37T0bZtbgTJWLFpkxupc+lS17VywIBjrzMek8L337tRPevVc3ch2+CExg9LCHFk7FjYuROefDI22oHbt4f77nNt1P/4R7SjcTN69ezphvJ47z13/SBc4ikpbNoE553n+q7Png1NmkQ7IhMvLCHEifnz4YUX3PWDNm2iHc0Rd97phsm+6SY3Mmq0/Piju5EsJ8cN5dy7d/hfIx6Swo4d7sxg40Z3c91pp0U7IhNPLCHEgcOHXSI4/ng3jG8sqVzZNR3t2OGSgkZhcJE1a9w4/j//DP/+t3seKbGcFPbvd02J2dkwbRp07RrtiEy8sYQQByZPhi+/hL/+FerUiXY0R2vb1o1zNH06TJ1avq+9cqVLANu3w0cfufl9Iy0Wk8Lhw/C737lj8PLL7izBmNKyGdNi3I4dbkrDlBQ32XmlGE3heXmua+P337uxjk44IfKvuWyZaxo6eBA+/BA6doz8awaKlbGPVOGWW+Dpp90gh3fcEZ04TOwK64xpJnrGj3dNIU89FbvJAFzT0aRJ7qL3jTdGvulo0SL3AXz4MMydW/7JAI4+U/jkk/KPAWDCBJcMbr/dkoE5NjH8EWOWLYMnnoBrr3UXbmNdmzZuSOkZM9xNUJGyYIHrTVSlimuuads2cq9VksCk0K9f+SeFF190vc+uuMKN4GrMsbAmoxil6roOzpsHK1ZAo0bRjsifQ4dc09HKla7pKNzjt3/zjTsutWu7u3BPOSW89ZdVYPPRrFmRvbCdb+ZMuPBCN4b/zJkuQRoTijUZxbl33nFjsY8fHz/JANzcDK+8Art3w8iR4W06+vxzOOccqF/ffROPlWQA5X+m8NlncNll7szxH/+wZGDCwxJCDNq7F267zd34deON0Y6m9E47Df7yF3en8BtvhKfOuXPdmUHjxu7DtkWL8NQbTuWVFBYvhgsucK/z3ntQs2ZkXscknoRICPfe64ZXWL8+2pH48+CDbhq//Ang49Gtt7ouoKNHu5ukjsW//+0+YFu0cNcMYvnO20gnhXXrXGKsXt3dhdywYXjrN4nNV0IQkb4islxEVonImBDbm4vIHBFZKCLZItLfW99SRPaKSJb3eC5gny4issir80mRyA3GsGsXTJzomhhuvdVNAxmr1q519xtcdll8T+GYlOR6He3dCzfcUPamo/fec9+GTz3VnSVEek7ZcIhUUti82SWD3bvd3dixeJZk4lxJEyYASbiZ0k7myBSabYLKTARu1COzp631nrcEFhdR7zfAGYDgpuDsV1IsxzJBzurVqldfrZqUpHrccap/+ENsTvAyeLCblOXHH6MdSXg88oibmOW110q/79tvq1apotqli+qWLeGPLdICJ9n5+ONjq2vXLtWuXVWrVVP95JPwxGcSB2GcICcdWKWqq1X1ADAFGBicV4Da3vM6uDmViyQijYHaqvqlF+xrwCAfsZRZSoq7g3PZMnd7/8MPu3Xjxrmx82PBhx/C22+7boRNm0Y7mvD4/e9dr6NbbnGDz/n11ltuMvguXdzdt/XrRy7GSAnXmcLBg+5YzJvnuvOeeWZ44zSmQEkZA7gYeDFg+Urg6aAyjYFFQA6wFeiiR84QdgMLgY+BM731acCHAfufCfyriNcfAWQCmc2bNw9bxlyyRPWSS9y31zp13FzB27eHrfpSO3DAfZs85RTVffuiF0ckLF/upvs8/3x/0ze++qpqpUqqZ56pumNH5OOLtGM5Uzh0SPXKK93f6cSJkYnPVHyEa05l4JIQCeGpoDK3Abd7z88AluKuT1QDGnjruwA/4s4kTg+REP5ZUiyRmFM5K0t14EB3JOrXV50wwZ2el7f8ppV//rP8X7s8PPaYe3+vvlp8uRdecPMf9+4dnd9DpJQ1KfzhD+64/fnPkYvNVHzhTAhnALMDlu8G7g4qswRoFrC8Gjg+RF1zvbODxsB3AeuHAs+XFEskEkK+efNU+/VzR6RRI/cBvWdPxF6ukI0bVWvVUu3fv3xeLxoOHVLt0cOdjeXkhC6TPxF8377ld+zLU2mTwsMPu+Nx883+zqyMKUo4E0Jl7wM+hSMXldsGlZkFDPeet8ZdQxCgEZDkrT8ZWA/U95bnAd04clG5f0mxRDIh5PviC9VzznFHpnFj1aeeinwTzlVXqVatqrpiRWRfJ9pWrnQX9Pv3P/oD7tFH3TEfMKDiNZkF8psUXn/dHY9LLlHNyyu/+EzFFLaE4OqiP7AC19torLduPDDAe94G+NxLFllAH2/9Rd7Zw7fAAuCCgDrTgMVenU/jDaNR3KM8EkK+uXNdGzaoNmum+vzzrp0/3L74wr3GmDHhrzsWPfGEe7+TJh1Z98ADbt3FF0fmGMeakpLCrFmqlSur9upVsZOjKT9hTQix8ijPhKDqvsX+5z+q3bq5I5WS4j7IDh4MT/15ea5LZZMmqjt3hqfOWHfokOpZZ6nWru261t57rzu2l18evuMaD4pKCl995dZ16hTdTg6mYvGbEBLiTuWyEnFj53zxhbtBql49uPpqN7rmG2+4gdyOxcsvu6kxH3oocYYfqFTJve+8POjWzU2sM3w4vPZa/N6VXRahuqQuXw6//a27+W7WLDeAnzHlyRKCDyJuvPvMTDe0c7VqMGwYdOjgpio8fLj0dW7dCvfc40bFHDIk/DHHslNOgb/9zQ0lcsMN8NJL7s7mRBOcFM45xx2H2bPj445sU/FYQigFERg0CLKy3I1Thw+7G4ZSU93ww1qK4Rn+9Cf45Rd48klXb6K5+WZYuhSefTa2J/6JtMCksG2bOzP41a+iHZVJVAn8r1h2lSrBpZe6USdff92NLTNwoJvU/IMPSk4M2dnw97+7kUyjMdNXLBCB1q0TMxkGO+EEdxfy8uXuy4Ux0WIJ4RgkJbmZqpYtc80eP//sTv179HCTt4Si6kYArVfPzXVgDECNGnDSSdGOwiQ6SwhhULkyXHONm9ns2WfdEMW9e7sZtD77rHDZt95yFxAfeCA+x+cxxlRclhDCqGpVN0vYqlVuLuRly9xAZOed56Z+3LXLTYKemurmSTbGmFhiCSECkpPd6J6rV7supQsWuOsLnTu7njVPP52YvWqMMbHNEkIEVa/uzghWr4b774ctW+D6691MYsYYE2tES9NXMsrS0tI0MzMz2mGU2aFDrldNInezNMaUPxGZr6ppJZVLoHtDo8+aiYwxscy+qxpjjAEsIRhjjPFYQjDGGANYQjDGGOPxlRBEpK+ILBeRVSIyJsT25iIyR0QWiki2iPT31p8rIvNFZJH3s1fAPnO9OrO8x/Hhe1vGGGNKq8ReRiKSBDwDnAvkAPNEZKaqLg0oNg6YqqrPikgb4H2gJbAZN0vaBhFpB8wGmgTsN0xV47cfqTHGVCB+zhDSgVWqulpVDwBTgIFBZRTIn86jDm5OZVR1oapu8NYvAZJFpNqxh22MMSbc/CSEJsCPAcs5FP6WD3AfcIWI5ODODkaHqOciYKGq7g9YN8lrLvqjSOiBkEVkhIhkikhmbm6uj3CNMcaUhZ+EEOqDOvj25qHAK6raFOgPvC4iBXWLSFvgb8ANAfsMU9X2wJne48pQL66qE1U1TVXTGjVq5CNcY4wxZeEnIeQAzQKWm+I1CQW4FpgKoKpfAslAQwARaQrMAH6nqt/n76Cq672fO4E3cE1TxhhjosRPQpgHtBKRFBGpCgwBZgaV+QHoDSAirXEJIVdE6gLvAXer6uf5hUWksojkJ4wqwPnA4mN9M8YYY8quxISgqnnAKFwPoWW43kRLRGS8iAzwit0OXC8i3wJvAsPVjZo3CvgV8Meg7qXVgNkikg1kAeuBF8L95owxxvhno50aY0wF53e0U7tT2RhjDGAJwRhjjMcSgjHGGMASgjHGGI8lBGOMMYAlBGOMMR5LCMYYYwBLCMYYYzyWEIwxxgCWEIwxxngsIRhjjAEsIRhjjPFYQjDGGANYQjDGGOOxhGCMMQawhGCMMcbjKyGISF8RWS4iq0RkTIjtzUVkjogsFJFsEekfsO1ub7/lInKe3zqNMcaUrxITgogkAc8A/YA2wFARaRNUbBxuas3OuDmX/+7t28Zbbgv0Bf4uIkk+6zTGGFOO/JwhpAOrVHW1qh4ApgADg8ooUNt7XgfY4D0fCExR1f2qugZY5dXnp05jjDHlyE9CaAL8GLCc460LdB9whYjkAO8Do0vY10+dAIjICBHJFJHM3NxcH+EaY4wpCz8JQUKs06DlocArqtoU6A+8LiKVitnXT51upepEVU1T1bRGjRr5CNcYY0xZVPZRJgdoFrDclCNNQvmuxV0jQFW/FJFkoGEJ+5ZUpzHGmHLk5wxhHtBKRFJEpCruIvHMoDI/AL0BRKQ1kAzkeuWGiEg1EUkBWgHf+KzTGGNMOSrxDEFV80RkFDAbSAJeVtUlIjIeyFTVmcDtwAsiciuu6We4qiqwRESmAkuBPOBmVT0EEKrOCLw/Y4wxPon73I4PaWlpmpmZGe0wjDEmrojIfFVNK6mc3alsjDEGsIRgjDHGYwnBGGMMYAnBGGOMxxKCMcYYwBKCMcYYjyUEY4wxgCUEY4wxHksIxhhjAEsIxhhjPJYQjDHGAJYQjDHGeCwhGGOMASwhGGOM8VhCMMYYA/hMCCLSV0SWi8gqERkTYvtjIpLlPVaIyDZv/W8C1meJyD4RGeRte0VE1gRs6xTet2aMMaY0SpwxTUSSgGeAc3FzJM8TkZmqujS/jKreGlB+NNDZWz8H6OStrw+sAv4dUP0fVHVaGN6HMcaYY+TnDCEdWKWqq1X1ADAFGFhM+aHAmyHWXwzMUtU9pQ/TGGNMpPlJCE2AHwOWc7x1RxGRFkAK8N8Qm4dwdKK4X0SyvSanakXUOUJEMkUkMzc310e4xhhjysJPQpAQ64qaiHkIME1VDxWqQKQx0B6YHbD6buA04HSgPnBXqApVdaKqpqlqWqNGjXyEa4wxpiz8JIQcoFnAclNgQxFlQ50FAFwKzFDVg/krVHWjOvuBSbimKWOMMVHiJyHMA1qJSIqIVMV96M8MLiQipwL1gC9D1HHUdQXvrAEREWAQsLh0oRtjjAmnEnsZqWqeiIzCNfckAS+r6hIRGQ9kqmp+chgKTFHVQs1JItISd4bxcVDVGSLSCNcklQWMPJY3Yowx5thI0Od3TEtLS9PMzMxoh2GMMXFFROaralpJ5exOZWOMMYAlBGOMMR5LCMYYYwBLCMYYYzyWEIwxxgCWEIwxxngsIRhjjAESICFkZEDLllCpkvuZkRHtiIwxJjaVeKdyPMvIgBEjYI834Pa6dW4ZYNiw6MVljDGxqEKfIYwdeyQZ5Nuzx603xhhTWIVOCD/8ULr1xhiTyCp0QmjevHTrjTEmkVXohHD//VC9euF11au79cYYYwqr0Alh2DCYOBFatAAR93PiRLugbIwxoVToXkbgPvwtARhjTMl8nSGISF8RWS4iq0RkTIjtj4lIlvdYISLbArYdCtg2M2B9ioh8LSIrReQtbzY2Y4wxUVJiQhCRJOAZoB/QBhgqIm0Cy6jqraraSVU7AU8Bbwds3pu/TVUHBKz/G/CYqrYCtgLXHuN7McYYcwz8nCGkA6tUdbWqHgCmAAOLKX/U/MnBvHmUewHTvFWv4uZVNsYYEyV+EkIT4MeA5Rxv3VFEpAWQAvw3YHWyiGSKyFcikv+h3wDYpqp5Puoc4e2fmZub6yNcY4wxZeHnorKEWFfURMxDgGmqeihgXXNV3SAiJwP/FZFFwA6/darqRGAiuDmVfcRrjDGmDPwkhBygWcByU2BDEWWHADcHrlDVDd7P1SIyF+gMTAfqikhl7yyhuDoLzJ8/f7OIrPMRcygNgc1l3LcisuNxhB2Lwux4FFYRjkcLP4X8JIR5QCsRSQHW4z70Lw8uJCKnAvWALwPW1QP2qOp+EWkIdAceVFUVkTnAxbhrElcB75YUiKo28hFvSCKSqappZd2/orHjcYQdi8LseBSWSMejxGsI3jf4UcBsYBkwVVWXiMh4EQnsNTQUmKKqgc06rYFMEfkWmAP8VVWXetvuAm4TkVW4awovHfvbMcYYU1ZS+PO74kqkLO+HHY8j7FgUZsejsEQ6HhV66IogE6MdQIyx43GEHYvC7HgUljDHI2HOEIwxxhQvkc4QjDHGFMMSgjHGGCBBEkJJg/MlChFpJiJzRGSZiCwRkd9HO6ZYICJJIrJQRP4V7ViiTUTqisg0EfnO+zs5I9oxRYuI3Or9nywWkTdFJDnaMUVahU8IfgbnSyB5wO2q2hroBtycwMci0O9xXaoNPAF8oKqnAR1J0OMiIk2AW4A0VW0HJOHuwarQKnxCoPSD81VYqrpRVRd4z3fi/tlDjiGVKESkKfBb4MVoxxJtIlIbOAvvniBVPaCq24rfq0KrDBwnIpWB6vgYTSHeJUJC8D04XyIRkZa4YUS+jm4kUfc4cCdwONqBxICTgVxgkteE9qKI1Ih2UNGgquuBh4EfgI3AdlX9d3SjirxESAilGZwvIYhITdx4Uv+rqqEGGkwIInI+8LOqzo92LDGiMpAKPKuqnYHdQEJec/OG3RmIG735JKCGiFwR3agiLxESQmkG56vwRKQKLhlkqOrbJZWv4LoDA0RkLa4psZeITI5uSFGVA+Soav5Z4zRcgkhE5wBrVDVXVQ/iJv36dZRjirhESAgFg/N503QOAWaWsE+F5E1M9BKwTFUfjXY80aaqd6tqU1Vtifu7+K+qVvhvgUVR1Z+AH72BKgF6A0uL2aUi+wHoJiLVvf+b3iTABXY/o53GNVXNE5H8wfmSgJdVdUmUw4qW7sCVwCIRyfLW3aOq70cxJhNbRgMZ3pen1cDVUY4nKlT1axGZBizA9c5bSAIMYWFDVxhjjAESo8nIGGOMD5YQjDHGAJYQjDHGeCwhGGOMASwhGGOM8VhCMMYYA1hCMMYY4/l/MSFSvWzaNroAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VNXZwPHfE/Z9CSiWLaCoBAwQI6KgqCiiFlCKCoKKQhEVtaJ9QUQQUF+q1AXFhbrUVhQpvlaqKNqKRaosYZVFCrJIBBWQfTXkef84N2ESh2SSzMydzDzfz2c+mXvn3Hufmck8c+bcc88RVcUYY0xiSPI7AGOMMdFjSd8YYxKIJX1jjEkglvSNMSaBWNI3xpgEYknfGGMSiCV9UywiUk5E9otIk3CW9ZOInCYiYe+7LCKXisimgOW1InJBKGVLcKyXRWRkSbcvZL+PiMifw71f45/yfgdgIktE9gcsVgWOAMe85dtUdWpx9qeqx4Dq4S6bCFT1jHDsR0QGAf1V9aKAfQ8Kx75N/LOkH+dUNS/pejXJQar6zxOVF5HyqpodjdiMMdFnzTsJzvv5/raIvCUi+4D+InKeiMwXkd0isk1EJolIBa98eRFREUnxlt/wHv9QRPaJyJci0qy4Zb3HrxCR/4rIHhF5VkT+IyIDThB3KDHeJiLrRWSXiEwK2LaciDwlIjtF5BugWyGvzygRmVZg3WQRedK7P0hE1njP5xuvFn6ifWWJyEXe/aoi8lcvtlXA2UGOu8Hb7yoR6eGtPwt4DrjAazrbEfDaPhyw/RDvue8Ukb+LyCmhvDZFEZGrvXh2i8inInJGwGMjRWSriOwVka8DnmsHEVnirf9BRJ4I9XgmAlTVbglyAzYBlxZY9whwFOiOqwRUAc4BzsX9EmwO/BcY6pUvDyiQ4i2/AewAMoAKwNvAGyUoexKwD+jpPTYM+BkYcILnEkqM7wG1gBTgp9znDgwFVgGNgGRgrvsoBD1Oc2A/UC1g3z8CGd5yd6+MAJcAh4A077FLgU0B+8oCLvLuTwQ+A+oATYHVBcpeB5zivSc3eDGc7D02CPisQJxvAA9797t6MbYFKgPPA5+G8toEef6PAH/27rf04rjEe49Geq97BaAVsBlo4JVtBjT37i8C+nr3awDn+v1ZSOSb1fQNwDxV/Yeq5qjqIVVdpKoLVDVbVTcAU4DOhWw/Q1UzVfVnYCou2RS37K+BZar6nvfYU7gviKBCjPF/VXWPqm7CJdjcY10HPKWqWaq6E5hQyHE2ACtxX0YAlwG7VTXTe/wfqrpBnU+BfwFBT9YWcB3wiKruUtXNuNp74HGnq+o27z15E/eFnRHCfgH6AS+r6jJVPQyMADqLSKOAMid6bQrTB5ipqp9679EEoCbuyzcb9wXTymsi3Oi9duC+vFuISLKq7lPVBSE+DxMBlvQNwJbABRE5U0Q+EJHvRWQvMA6oV8j23wfcP0jhJ29PVPZXgXGoquJqxkGFGGNIx8LVUAvzJtDXu38D7ssqN45fi8gCEflJRHbjatmFvVa5TiksBhEZICLLvWaU3cCZIe4X3PPL25+q7gV2AQ0DyhTnPTvRfnNw71FDVV0L3Id7H370mgsbeEVvAVKBtSKyUESuDPF5mAiwpG/A/dwP9BKudnuaqtYERuOaLyJpG665BQAREfInqYJKE+M2oHHAclFdSt8GLvVqyj1xXwKISBVgBvC/uKaX2sDHIcbx/YliEJHmwAvA7UCyt9+vA/ZbVPfSrbgmo9z91cA1I30XQlzF2W8S7j37DkBV31DVjrimnXK41wVVXauqfXBNeH8E3hGRyqWMxZSQJX0TTA1gD3BARFoCt0XhmO8D6SLSXUTKA/cA9SMU43TgdyLSUESSgeGFFVbVH4B5wGvAWlVd5z1UCagIbAeOicivgS7FiGGkiNQWdx3D0IDHquMS+3bc998gXE0/1w9Ao9wT10G8BQwUkTQRqYRLvp+r6gl/ORUj5h4icpF37N/jzsMsEJGWInKxd7xD3u0Y7gncKCL1vF8Ge7znllPKWEwJWdI3wdwH3Iz7QL+Eq+lGlJdYrweeBHYCpwJLcdcVhDvGF3Bt71/hTjLOCGGbN3EnZt8MiHk3cC/wLu5kaG/cl1coxuB+cWwCPgT+ErDfFcAkYKFX5kwgsB38E2Ad8IOIBDbT5G7/Ea6Z5V1v+ya4dv5SUdVVuNf8BdwXUjegh9e+Xwl4HHce5nvcL4tR3qZXAmvE9Q6bCFyvqkdLG48pGXFNp8bEFhEph2tO6K2qn/sdjzHxwmr6JmaISDcRqeU1ETyE6xGy0OewjIkrlvRNLOkEbMA1EXQDrlbVEzXvGGNKwJp3jDEmgVhN3xhjEkjMDbhWr149TUlJ8TsMY4wpUxYvXrxDVQvr5gzEYNJPSUkhMzPT7zCMMaZMEZGiriwHrHnHGGMSiiV9Y4xJIJb0jTEmgcRcm74xJrp+/vlnsrKyOHz4sN+hmBBUrlyZRo0aUaHCiYZeKpwlfWMSXFZWFjVq1CAlJQU3uKmJVarKzp07ycrKolmzZkVvEETcNO9MnQopKZCU5P5OLdZ038YkrsOHD5OcnGwJvwwQEZKTk0v1qywuavpTp8LgwXDwoFvevNktA/Qr9diCxsQ/S/hlR2nfq7io6T/44PGEn+vgQbfeGGPMcXGR9L/9tnjrjTGxY+fOnbRt25a2bdvSoEEDGjZsmLd89Ghow+7fcsstrF27ttAykydPZmqY2n07derEsmXLwrKvaIuL5p0mTVyTTrD1xpjwmjrV/Yr+9lv3GXv00dI1oyYnJ+cl0Icffpjq1atz//335yujqqgqSUnB66mvvfZakce58847Sx5kHImLmv6jj0LVqvnXVa3q1htjwif3/NnmzaB6/PxZJDpOrF+/ntatWzNkyBDS09PZtm0bgwcPJiMjg1atWjFu3Li8srk17+zsbGrXrs2IESNo06YN5513Hj/++CMAo0aN4umnn84rP2LECNq3b88ZZ5zBF198AcCBAwf4zW9+Q5s2bejbty8ZGRlF1ujfeOMNzjrrLFq3bs3IkSMByM7O5sYbb8xbP2nSJACeeuopUlNTadOmDf379w/7axaKuEj6/frBlCnQtCmIuL9TpthJXGPCLdrnz1avXs3AgQNZunQpDRs2ZMKECWRmZrJ8+XI++eQTVq9e/Ytt9uzZQ+fOnVm+fDnnnXcer776atB9qyoLFy7kiSeeyPsCefbZZ2nQoAHLly9nxIgRLF26tND4srKyGDVqFHPmzGHp0qX85z//4f3332fx4sXs2LGDr776ipUrV3LTTTcB8Pjjj7Ns2TKWL1/Oc889V8pXp2RCSvrejEZrRWS9iIwI8vgAEdkuIsu826CAx24WkXXe7eZwBh+oXz/YtAlyctxfS/jGhF+0z5+deuqpnHPOOXnLb731Funp6aSnp7NmzZqgSb9KlSpcccUVAJx99tls2rQp6L579er1izLz5s2jT58+ALRp04ZWrVoVGt+CBQu45JJLqFevHhUqVOCGG25g7ty5nHbaaaxdu5Z77rmH2bNnU6tWLQBatWpF//79mTp1aokvriqtIpO+N1fpZOAKIBXoKyKpQYq+raptvdvL3rZ1cRNAnwu0B8aISJ2wRW+MiaoTnSeL1PmzatWq5d1ft24dzzzzDJ9++ikrVqygW7duQfurV6xYMe9+uXLlyM7ODrrvSpUq/aJMcSeVOlH55ORkVqxYQadOnZg0aRK33XYbALNnz2bIkCEsXLiQjIwMjh07VqzjhUMoNf32wHpV3eDNYD8N6Bni/i8HPlHVn1R1F/AJbho8Y0wZ5Of5s71791KjRg1q1qzJtm3bmD17dtiP0alTJ6ZPnw7AV199FfSXRKAOHTowZ84cdu7cSXZ2NtOmTaNz585s374dVeXaa69l7NixLFmyhGPHjpGVlcUll1zCE088wfbt2zlYsK0sCkLpvdMQ2BKwnIWruRf0GxG5EPgvcK+qbjnBtg0Lbigig4HBAE2sy40xMSu32TScvXdClZ6eTmpqKq1bt6Z58+Z07Ngx7Me46667uOmmm0hLSyM9PZ3WrVvnNc0E06hRI8aNG8dFF12EqtK9e3euuuoqlixZwsCBA1FVRIQ//OEPZGdnc8MNN7Bv3z5ycnIYPnw4NWrUCPtzKEqRc+SKyLXA5ao6yFu+EWivqncFlEkG9qvqEREZAlynqpeIyO+BSqr6iFfuIeCgqv7xRMfLyMhQm0TFmOhZs2YNLVu29DuMmJCdnU12djaVK1dm3bp1dO3alXXr1lG+fGz1bg/2nonIYlXNKGrbUJ5JFtA4YLkRsDWwgKruDFj8E/CHgG0vKrDtZyEc0xhjom7//v106dKF7OxsVJWXXnop5hJ+aYXybBYBLUSkGfAd0Ae4IbCAiJyiqtu8xR7AGu/+bOCxgJO3XYEHSh21McZEQO3atVm8eLHfYURUkUlfVbNFZCgugZcDXlXVVSIyDshU1ZnA3SLSA8gGfgIGeNv+JCLjcV8cAONU9acIPA9jjDEhCOl3i6rOAmYVWDc64P4DnKAGr6qvAsGvjjDGGBNVcXFFrjHGmNBY0jfGmARiSd8Y46uLLrroFxdaPf3009xxxx2Fble9enUAtm7dSu/evU+476K6gD/99NP5LpK68sor2b17dyihF+rhhx9m4sSJpd5PuFnSN8b4qm/fvkybNi3fumnTptG3b9+Qtv/Vr37FjBkzSnz8gkl/1qxZ1K5du8T7i3WW9I0xvurduzfvv/8+R44cAWDTpk1s3bqVTp065fWbT09P56yzzuK99977xfabNm2idevWABw6dIg+ffqQlpbG9ddfz6FDh/LK3X777XnDMo8ZMwaASZMmsXXrVi6++GIuvvhiAFJSUtixYwcATz75JK1bt6Z169Z5wzJv2rSJli1b8tvf/pZWrVrRtWvXfMcJZtmyZXTo0IG0tDSuueYadu3alXf81NRU0tLS8gZ6+/e//503iUy7du3Yt29fiV/bYOLrqgNjTKn87ncQ7gmh2rYFL18GlZycTPv27fnoo4/o2bMn06ZN4/rrr0dEqFy5Mu+++y41a9Zkx44ddOjQgR49epxwntgXXniBqlWrsmLFClasWEF6enreY48++ih169bl2LFjdOnShRUrVnD33Xfz5JNPMmfOHOrVq5dvX4sXL+a1115jwYIFqCrnnnsunTt3pk6dOqxbt4633nqLP/3pT1x33XW88847hY6Pf9NNN/Hss8/SuXNnRo8ezdixY3n66aeZMGECGzdupFKlSnlNShMnTmTy5Ml07NiR/fv3U7ly5WK82kWzmr4xxneBTTyBTTuqysiRI0lLS+PSSy/lu+++44cffjjhfubOnZuXfNPS0khLS8t7bPr06aSnp9OuXTtWrVpV5GBq8+bN45prrqFatWpUr16dXr168fnnnwPQrFkz2rZtCxQ+fDO48f13795N586dAbj55puZO3duXoz9+vXjjTfeyLvyt2PHjgwbNoxJkyaxe/fusF8RbDV9Y0yewmrkkXT11VczbNgwlixZwqFDh/Jq6FOnTmX79u0sXryYChUqkJKSEnQ45UDBfgVs3LiRiRMnsmjRIurUqcOAAQOK3E9h45LlDssMbmjmopp3TuSDDz5g7ty5zJw5k/Hjx7Nq1SpGjBjBVVddxaxZs+jQoQP//Oc/OfPMM0u0/2Cspm+M8V316tW56KKLuPXWW/OdwN2zZw8nnXQSFSpUYM6cOWwONhl2gAsvvDBv8vOVK1eyYsUKwA3LXK1aNWrVqsUPP/zAhx9+mLdNjRo1grabX3jhhfz973/n4MGDHDhwgHfffZcLLrig2M+tVq1a1KlTJ+9Xwl//+lc6d+5MTk4OW7Zs4eKLL+bxxx9n9+7d7N+/n2+++YazzjqL4cOHk5GRwddff13sYxbGavrGmJjQt29fevXqla8nT79+/ejevTsZGRm0bdu2yBrv7bffzi233EJaWhpt27alffv2gJsFq127drRq1eoXwzIPHjyYK664glNOOYU5c+bkrU9PT2fAgAF5+xg0aBDt2rUrtCnnRF5//XWGDBnCwYMHad68Oa+99hrHjh2jf//+7NmzB1Xl3nvvpXbt2jz00EPMmTOHcuXKkZqamjcLWLgUObRytNnQysZElw2tXPaUZmhla94xxpgEYknfGGMSiCV9Y0yxJwQ3/inte2VJ35gEV7lyZXbu3GmJvwxQVXbu3FmqC7ZC6r0jIt2AZ3CTqLysqhNOUK438DfgHFXNFJEU3Cxaa70i81V1SImjNcaEXaNGjcjKymL79u1+h2JCULlyZRo1alTi7YtM+iJSDpgMXIab83aRiMxU1dUFytUA7gYWFNjFN6ratsQRGmMiqkKFCjRr1szvMEyUhNK80x5Yr6obVPUoMA3oGaTceOBxoPDL3IwxxvgmlKTfENgSsJzlrcsjIu2Axqr6fpDtm4nIUhH5t4gEvZxNRAaLSKaIZNpPTGOMiZxQkn6w4ezyzviISBLwFHBfkHLbgCaq2g4YBrwpIjV/sTPVKaqaoaoZ9evXDy1yY4wxxRZK0s8CGgcsNwK2BizXAFoDn4nIJqADMFNEMlT1iKruBFDVxcA3wOnhCNwYY0zxhZL0FwEtRKSZiFQE+gAzcx9U1T2qWk9VU1Q1BZgP9PB679T3TgQjIs2BFsCGsD8LY4wxISmy946qZovIUGA2rsvmq6q6SkTGAZmqOrOQzS8ExolINnAMGKKqP4UjcGOMMcVnA64ZY0wcsAHXjDHG/IIlfWOMSSCW9I2JohhrTTUJyJK+MVGybx+0agXDh1vyN/6x6RKNiZJnnoE1a9ytbl2X/I2JNkv6xkTBrl0wcSL8+tdQsyaMGAEnnwwDBvgdmUk0lvSNiYInn4Q9e+CRR6BlS9i+HQYNgvr14aqr/I7OJBJr0zcmwnbsgKefhmuvhTZtoGJFeOcdaNvWrfvyS78jNInEkr4xEfbEE3DgADz88PF1NWrArFnQsKFr8lmzxrfwTIKxpG9MBH3/PTz7LPTrB6mp+R876SSYPRsqVIDLL4esLH9iNInFkr4xETRhAhw9CmPGBH+8eXP48EPYvRu6dXMnfI2JJEv6xkRIVha8+CLcfDOcdtqJy7VrB3//O6xbBz16wKFD0YvRJB5L+sZEyKOPQk4OPPRQ0WUvuQT++lf4z3+gb1/Izo58fCYxWdI3JgI2bYJXXnHdMlNSQtvmuuvcBVzvvQd33GFX7ZrIsH76xkTA+PGQlAQPPli87e66y538fewxOOUUGDs2MvGZxGVJ35gwW7cOXn8dhg51XTKL65FHXOIfNw4aNIDbbw9/jCZxhdS8IyLdRGStiKwXkRGFlOstIioiGQHrHvC2Wysil4cjaGNi2dixUKmSG2qhJETgpZege3e4806YMSO88ZnEVmTS9+a4nQxcAaQCfUUkNUi5GsDdwIKAdam4OXVbAd2A53PnzDUmHq1eDW++6Wr5DRqUfD/ly8O0adChg+vj/9lnYQvRJLhQavrtgfWqukFVjwLTgJ5Byo0HHgcOB6zrCUxT1SOquhFY7+3PmLg0ZgxUqwa//33p91W1Krz/Ppx6KvTsCcuXl36fxoSS9BsCWwKWs7x1eUSkHdBYVd8v7rbe9oNFJFNEMrdv3x5S4MbEmmXLXFPMvfdCvXrh2Wfduu6q3Zo13cVbGzeGZ78mcYWS9CXIurzOZCKSBDwF3FfcbfNWqE5R1QxVzahfv34IIRkTe8aMgdq1Ydiw8O63cWP46CM4csQN12D1IlMaoST9LKBxwHIjYGvAcg2gNfCZiGwCOgAzvZO5RW1rTFxYuBBmzoT77nOJP9xatXJNPVu2uKGY9+8P/zFMYggl6S8CWohIMxGpiDsxOzP3QVXdo6r1VDVFVVOA+UAPVc30yvURkUoi0gxoASwM+7MwxmejR0NyMtxzT+SOcf758PbbsHgx9O4NP/8cuWOZ+FVk0lfVbGAoMBtYA0xX1VUiMk5EehSx7SpgOrAa+Ai4U1WPlT5sY2LHvHmu3X34cDdkciT16AFTprjj3XqrG+bBmOIQjbFrvTMyMjQzM9PvMIwJ2cUXu/HwN2xwPW6i4dFHYdQouP9+N16/MSKyWFUziipnV+QaUwqffur60D/zTPQSPsDIke6q3YkT3fUA9wXrRmHKnOxsd41GJFnSN6aEVN0Imo0aweDB0T22iJuC8YcfXG3/5JOhf//oxmBK59gxWLUKFiyA+fPdLbenViRZ0jemhD76CL74Al54ASpXjv7xy5VzwzHv2AG33OImWb/cBjqJWT/8kD/BL1p0vBdWcrK7+rpLl8jHYW36xpSAKrRv7xLu2rVusnO/7NkDF13kBnqbMwfOOce/WIxz5Ii7WC83wc+f74bbBtd807atS/Lnnuv+nnqq+/VWGtamb0wEzZwJmZnw6qv+JnyAWrXclIvnnw9XXukmYjn9dH9jSiSqsHlz/gS/dKmbJhNck02HDm48pg4dID0dqlTxL16r6RtTTDk5rqZ2+LAbYC3SJ95CtW4ddOzoxv754gs3Hr8Jv3373Bd+YJL/8Uf3WJUq7pdWbg3+3HNLNrx2SVhNP0pUXRIIvJUr53/tz0TOjBnw1VcwdWrsJHyAFi1g1izX1HPFFfDvf7tfAabkcnLg66/zJ/hVq45fH3H66W5MpA4d3K11a6hQwd+YixI3Nf1du9yLn5t4gyXj4txC3T6YatXclHfROCljouvYMffBTkqCFSvcF3ys+fhjN1RDp06u2cePk8xl1Y4d+U+2LlwIe/e6x2rXPl6D79DBndOpW9ffeAMlXE0/Kcm9AUlJ7oRIUlLxbyXdruC2f/6z6z63YoXrUWHix5tvuprf3/4WmwkfoGtXN3NXv37u//Dtt2M3Vr+pui/JN95wSX79erc+KQnS0uCGG44n+RYt3PqyLm5q+rFkxQpXC7j0UvjHP0p/Vt7Ehp9/hpYt3VALixfHfgJ46ik34ucdd8Bzz9n/YUFz57o5jOfNc5WzTp2Ot8OffTZUr+53hMWTcDX9WJKW5i6Nv/tu92G76y6/IzLh8Je/wDffuJ47sZ7wwY3rv22b+19s0MBdSGZc//hRo1wN/5RT4PnnYeDAxDkPZzX9CFF1g2N9/LFrF2zTxu+ITGkcOeJO2jVo4JoBykqtOSfHXbj1l7+4gdp++1u/I/LPypXui+/vf3cXQz3wgPsV5Gf3yXAKtaZfBuorZZOI68OdnAx9+sDBg35HZErjlVfg229h/Piyk/DB/SJ5+WXXm2fIENfBINGsX+/Ob6SlubGSxo51g+Pdd1/8JPzisKQfQfXru8vk1651P7VN2XTokBvVslMnuOwyv6MpvgoV3InnjAxXAZk3z++IomPLFvfL5swz4d134X/+x003OXq0m34yUVnSj7AuXdw/25Qp8M47fkdjSuLFF2Hr1rJXyw9UrRp88AE0bQrdu7umjnj1ww9uMpvTTnPNWnfc4Wr2EybEVhdLv4SU9EWkm4isFZH1IjIiyONDROQrEVkmIvNEJNVbnyIih7z1y0TkxXA/gbJg/Hh3ld6gQa6JwJQdBw64ZNGli7voqSyrV89NvlKlirumZepU2L3b76jCZ9cuN+R08+YweTLceCP8978waZI7F2M8qlroDSgHfAM0ByoCy4HUAmVqBtzvAXzk3U8BVhZ1jMDb2WefrfFo/XrV6tVVL7hANTvb72iiZ80a1Y8/9juKkpswQRVUv/jC70jCZ/ly1SZN3PMqX171sstUn39eNSvL78hKZu9e1fHjVWvVcs+pb1/VtWv9jir6gEwNIceGUtNvD6xX1Q2qehSYBvQs8MWxN2CxGhBbXYJiwKmnuq5hn3/u2ocTwddfu3bwrl3dSbPsbL8jKp69e+Hxx91J0PPO8zua8ElLc23b8+e792XzZtcE0qiR66c+YYI7DxXrDh2CJ590NfuHHoLOnWH5cncBnQ04V4iivhWA3sDLAcs3As8FKXcn7hfBFqCFHq/pHwCWAv8GLjjBMQYDmUBmkyZNIv+V6KP+/VWTklTnzfM7ksjaskW1cWPVk05SHTjQ1cAuu0x1506/Iwvd2LEu7sxMvyOJvNWrVR97TPWcc9xzBtUzz1R94AHVBQtUjx3zO8LjjhxRfeEF1YYNXZyXXqo6f77fUfmPEGv6oST9a4Mk/WcLKX8D8Lp3vxKQ7N0/2/tCqFnY8eK1eSfXnj2qzZu7n9e7dvkdTWTs3Kmamqpao4bqkiVu3auvqlas6J77V1/5G18odu5UrVlT9eqr/Y4k+rZsUX3uOdUuXVTLlXNZomFD1TvuUP3kE9WjR/2JKztb9fXXVZs1czGdf77qnDn+xBKLwpn0zwNmByw/ADxQSPkkYM8JHvsMyCjsePGe9FVdzal8edVrr1XNyfE7mvA6cMB9GCtWVP300/yPffml6imnqFarpvrOO/7EF6qRI92nY/lyvyPx108/qf7lL6q9eqlWrepek9q13S/WGTNU9+2LfAzHjqn+7W+qLVu647dtq/rBB/H32SmtcCb98sAGoBnHT+S2KlCmRcD97rkHB+oD5bz7zYHvgLqFHS8Rkr6q6v/+r3v1X37Z70jC5+hR1auuUhVxH9JgvvtOtUMH99wfeii2mg1y/fij+2K6/nq/I4ktBw+qvvee6oABqsnJ7j2sXFm1e3fVV15xr1s45eSozpqlmp5+vLlp+vTY/J+JBWFL+m5fXAn812uzf9BbNw7o4d1/BlgFLAPm5H4pAL/x1i8HlgDdizpWoiT9Y8dUL7nE1Z7WrPE7mtLLyVG9+Wb3H/XCC4WXPXxY9dZbXdkePVyTVyy5/3533iUe3pdI+fln17Ryzz3HewIlJaleeKHqU0+pbtxYuv1/9plqx45uvykpqn/+c2L1eiuJsCb9aN4SJemrulpvcrL7uXr4sN/RlM7vf+/+mx5+OLTyOTmqzz7r2oxbtoydLnZbt7ra6403+h1J2ZGT487dPPSQ6llnad6J4LZt3f/D8uWhN8UsXOhO+INrCnz+eXfi1hTNkn4ZMXOmexfuvdfvSEpu4kT3HO64o/jtrHPmqNar5/pYz5oVkfCK5a673BfR+vV+R1J2rVvn/icldQ45AAAS4klEQVQ6dnRNfeBO4A8bpjp3bvAa+4oVqj17urLJyW77gwejH3tZZkm/DBk61L0TsZD0iuv1113svXuX/Of3pk2uVijiLoby6wTdt9+6E9CDBvlz/Hj0/feqU6aoXnmle21BtX591433/fdVV65UveEG997XrKk6bpy72MoUnyX9MuTQIfezuH591W3b/I4mdB984GrFl1xS+uapAwfciVNQ7dPHLUfb4MGqFSq4LyETfnv2qL79tnt/a9TQvGagKlVUhw8vW9dwxKJQk76Npx8jVq92oyBecIGb1zTWJ+n48ks3Hk3LljBnTnhGLVR1V8A+8ICbf+DddyElpfT7DcWGDXDGGXDbbW7iGxNZR464/5vVq92UhDY2TunZePplTGqqm97u44/d31i2erWbePtXv4JZs8I3TK0IDB/uRoPcuNENUvfZZ+HZd1HGj4fy5d2AXSbyKlVyg74NG2YJP9os6ceQwYOhVy9X01282O9ogtuyBS6/3H1oP/4YTj45/Me44go321j9+m6e4eeec78CImXtWjcE7+23uy8yY+KZJf0YIgJ/+pNLpH37wv79fkeU386dbvC0vXvho4/cQFeRcvrpbkCwK690cwwPGuSaBCJh7FioXBlG/GLQcGPijyX9GFO3LrzxhpviLZYmVD9wwDXpbNzoJgaPxpy/NWu6+UwfeshNPdm5s5vMJJxWroRp09wk9iedFN59GxOLLOnHoM6dYdQo+POfXULy288/w7XXwqJF8NZbLr5oSUqCcePcrGMrV7qT3fPnh2//Y8ZA9epw//3h26cxscySfowaPRrOP9/1Jtm40b84cnLg1ltdj6IXX4RrrvEnjl69XI+hKlXcl86rr5Z+n0uXwv/9n5u/ODm59PszpiywpB+jypd309mJuC5tP/8c/RhU4fe/d81N48e7Sab9dNZZ7tfGhRfCwIGu+as0r8vo0VCnjk1abxKLJf0YlpICL73kmjPGjo3+8Z94ws1MNHQoPPhg9I8fTN267lfHsGGuV0/XrrB9e/H3M38+vP++a9apXTv8cRoTqyzpx7jrr4dbboHHHoten3Vw5xOGD3fHf+YZ94sjVpQvD3/8o+tm+eWXrj//smXF28fo0W6i8LvvjkyMxsQqS/plwKRJ0KIF9O/vuk1G2j/+4bpIXnopvP567F4dfOONMG8eHDvmzn+8/XZo282dC5984rpoVq8e2RiNiTUx+nE2gapXd71mfvzRJeNIXqj0n//AdddBu3buJGelSpE7VjhkZEBmJqSnQ58+LpEfO3bi8qquC2iDBu5iLGMSjSX9MiI9HSZMcP3WX3wxMsdYuRJ+/Wto3NgNr1CjRmSOE24nnwyffup6Ov3hD9C9O+zeHbzsv/7lavojR0LVqtGN05hYEFLSF5FuIrJWRNaLyC+uWxSRISLylYgsE5F5IpIa8NgD3nZrReTycAafaH73OzcEwrBhLkGH0+bNbt9VqrjhFerXD+/+I61iRfdl+OKLrummfXtYsyZ/mdxafuPGbsgLYxJRkUlfRMoBk4ErgFSgb2BS97ypqmepalvgceBJb9tUoA/QCugGPO/tz5RAUpJrY69Z0w3TcOhQePa7Y4dL+AcOuOEVojWyZSTcdpsbvXHPHjj3XHf1cK5Zs1yvnVGjYr/ZyphICaWm3x5Yr6obVPUoMA3oGVhAVfcGLFYDcludewLTVPWIqm4E1nv7MyV08sku8a9c6frQl9b+/W58m82b3QnctLTS79NvnTq5dv7TT4eePd01Bjk5rsdOs2auN5QxiSqUpN8Q2BKwnOWty0dE7hSRb3A1/buLue1gEckUkcztJel0HSOmTnW15KQk93fq1MgcJ3dI2smT89dki+voUfjNb9yInm+/7cbyjxeNG8Pnn7seT6NHu26dS5a4YRcqVPA7OmP8E0rSD9ZD+xf9R1R1sqqeCgwHRhVz2ymqmqGqGfXLWmOyZ+pU1068ebNrO9682S1HKvE/9pjrYXPLLfDdd8XfPicHBgxw7fdTpkCPHmEP0XdVqri+/H/8o+vHf/rp0K+f31EZ469Qkn4W0DhguRFQ2FiH04CrS7htmfXgg3DwYP51Bw9G7krWSpVcN87Dh11/9cK6KRak6n4pvPWW+/IYODAyMcYCEfdcFy92V/KWL+93RMb4K5SkvwhoISLNRKQi7sRsvkYFEWkRsHgVsM67PxPoIyKVRKQZ0AJYWPqwY8+33xZvfTiccQY8+6w7cfn446FvN2GCu8r2nnsSZwz5tm0jO/6/MWVFkUlfVbOBocBsYA0wXVVXicg4EcltFBgqIqtEZBkwDLjZ23YVMB1YDXwE3KmqxaiTlh1NmhRvfbjccosbKuGhh0IbcviVV1wf9RtucOPqxNLwCsaYyLOJ0cMkt00/sImnalXXXh7pduTdu11NNinJDRdcq1bwcu+954YovuwydwK4YsXIxmWMiR6bGD3K+vVzCb5pU1d7bto0Ogkf3CiRb77pmpLuuCP4MA2ff+6GKcjIgBkzLOEbk6gs6YdRv36waZPrGbNpU3R7ipx/vuuO+Oab8Ne/5n9sxQo3NEHTpvDBBzbImDGJzJJ+HBk50k0wcuedbo5dcLNudevmEv3HH7vhhI0xicuSfhwpV87NclWhghum4bvv3PAKhw/D7NmRP6lsjIl91ms5zjRu7Hro9OoFqaluOsF//hNatfI7MmNMLLCafhy65ho3VvyBAzB9umvvN8YYsKQftyZPhq1b3fj4xhiTy5J+nBKBk07yOwpjTKyxpG+MMQnEkr4xxiQQS/rGGJNALOkbY0wCsaRvjDEJxJK+McYkEEv6xhiTQEJK+iLSTUTWish6EfnFXEsiMkxEVovIChH5l4g0DXjsmIgs826lmMbbGGNMaRU59o6IlAMmA5fh5rxdJCIzVXV1QLGlQIaqHhSR24HHgeu9xw6patswx22MMaYEQqnptwfWq+oGVT2Km/i8Z2ABVZ2jqrlzRs3HTYBujDEmxoSS9BsCWwKWs7x1JzIQ+DBgubKIZIrIfBG5OtgGIjLYK5O5ffv2EEIyxhhTEqEMrRxs6uygE+uKSH8gA+gcsLqJqm4VkebApyLylap+k29nqlOAKeDmyA0pcmOMMcUWSk0/C2gcsNwI2FqwkIhcCjwI9FDVI7nrVXWr93cD8BnQrhTxGmOMKYVQkv4ioIWINBORikAfIF8vHBFpB7yES/g/BqyvIyKVvPv1gI5A4AlgY4wxUVRk0lfVbGAoMBtYA0xX1VUiMk5EenjFngCqA38r0DWzJZApIsuBOcCEAr1+TARMnQopKZCU5P5Onep3RMaYWCGqsdWEnpGRoZmZmX6HUWZNnQqDB8PBg8fXVa0KU6ZAv37+xWWMiSwRWayqGUWVsyty48yDD+ZP+OCWH3zQn3iMMbHFkn6c+fbb4q03xiQWS/pxpkmT4q03xiQWS/px5tFHXRt+oKpV3XpjjLGkH2f69XMnbZs2dZOjN21qJ3GNMceFckWuKWP69bMkb4wJzmr6xhiTQCzpG2NMArGkb4wxCcSSvjHGJBBL+sYYk0As6ZuIsYHfjIk91mXTRETBgd82b3bLYN1JjfGT1fRNRNjAb8bEJkv6JiJs4DdjYlNISV9EuonIWhFZLyIjgjw+TERWi8gKEfmXiDQNeOxmEVnn3W4OZ/AmdtnAb8bEpiKTvoiUAyYDVwCpQF8RSS1QbCmQoappwAzgcW/busAY4FygPTBGROqEL3wTq2zgN2NiUyg1/fbAelXdoKpHgWlAz8ACqjpHVXNbcOfjJk8HuBz4RFV/UtVdwCdAt/CEbmKZDfxmTGwKpfdOQ2BLwHIWruZ+IgOBDwvZtmFxAjRllw38ZkzsCSXpS5B1QSfWFZH+QAbQuTjbishgYDBAE2v0NcaYiAmleScLaByw3AjYWrCQiFwKPAj0UNUjxdlWVaeoaoaqZtSvXz/U2I0xxhRTKEl/EdBCRJqJSEWgDzAzsICItANewiX8HwMemg10FZE63gncrt46Y4wxPiiyeUdVs0VkKC5ZlwNeVdVVIjIOyFTVmcATQHXgbyIC8K2q9lDVn0RkPO6LA2Ccqv4UkWdijDGmSKIatHneNxkZGZqZmel3GMYYU6aIyGJVzSiqnF2Ra4wxCcSSvjHGJBBL+sYYk0As6RtjTAKxpG/ink3mYsxxNomKiWs2mYsx+VlN38Q1m8zFmPws6Zu4ZpO5GJOfJX0T12wyF2Pys6Rv4ppN5mJMfpb0TVyzyVyMyc9675i4Z5O5GHOc1fSNMSaBWNI3JkrsIjETC6x5x5gosIvETKywmr4xUWAXiZlYEVLSF5FuIrJWRNaLyIggj18oIktEJFtEehd47JiILPNuMwtua0wisIvETKwosnlHRMoBk4HLcBOdLxKRmaq6OqDYt8AA4P4guzikqm3DEKsxZVaTJq5JJ9h6Y6IplJp+e2C9qm5Q1aPANKBnYAFV3aSqK4CcCMRoTJlnF4mZWBFK0m8IbAlYzvLWhaqyiGSKyHwRuTpYAREZ7JXJ3L59ezF2bUzZEEsXiVkvosQWSu8dCbKuOLOpN1HVrSLSHPhURL5S1W/y7Ux1CjAF3MToxdi3MWVGLFwkZr2ITCg1/SygccByI2BrqAdQ1a3e3w3AZ0C7YsRnjAkj60VkQkn6i4AWItJMRCoCfYCQeuGISB0RqeTdrwd0BFYXvpUxJlKsF5EpMumrajYwFJgNrAGmq+oqERknIj0AROQcEckCrgVeEpFV3uYtgUwRWQ7MASYU6PVjjIkiG2rahHRFrqrOAmYVWDc64P4iXLNPwe2+AM4qZYzGmDB59NH8bfpgvYgSjV2Ra0wCiZVeRNaDyD829o4xCcbvXkTWg8hfVtM3xkSV9SDylyV9Y0xUWQ8if1nSN8ZElfUg8pclfWNMVNk4RP6ypG+MiapY6UEEidmLyHrvGGOizu8eRJC4vYispm+MSUiJ2ovIkr4xJiElai8iS/rGmISUqL2ILOkbYxJSLPUiiuYJZUv6xpiEFCu9iHJPKG/eDKrHTyhHKvGLamxNVJWRkaGZmZl+h2GMMVGRkuISfUFNm8KmTaHvR0QWq2pGUeWspm+MMT6K9gnlkJK+iHQTkbUisl5ERgR5/EIRWSIi2SLSu8BjN4vIOu92c7gCN8aYeBDtE8pFJn0RKQdMBq4AUoG+IpJaoNi3wADgzQLb1gXGAOcC7YExIlKn9GEbY0x8iPYJ5VBq+u2B9aq6QVWPAtOAnoEFVHWTqq4Acgpseznwiar+pKq7gE+AbmGI2xhj4kK0TyiHMgxDQ2BLwHIWruYeimDbNixYSEQGA4MBmsR7J1ljjCkgmsNShFLTlyDrQu3yE9K2qjpFVTNUNaN+/foh7toYY0xxhZL0s4DGAcuNgK0h7r802xpjjAmzUJL+IqCFiDQTkYpAH2BmiPufDXQVkTreCdyu3jpjjDE+KDLpq2o2MBSXrNcA01V1lYiME5EeACJyjohkAdcCL4nIKm/bn4DxuC+ORcA4b50xxhgf2BW5xhgTB0K9Ijfmkr6IbAeCXJQcsnrAjjCFU9bZa5GfvR752etxXDy8Fk1VtcieMDGX9EtLRDJD+bZLBPZa5GevR372ehyXSK+Fjb1jjDEJxJK+McYkkHhM+lP8DiCG2GuRn70e+dnrcVzCvBZx16ZvjDHmxOKxpm+MMeYELOkbY0wCiZukX9REL4lERBqLyBwRWSMiq0TkHr9j8puIlBORpSLyvt+x+E1EaovIDBH52vsfOc/vmPwkIvd6n5OVIvKWiFT2O6ZIioukH+JEL4kkG7hPVVsCHYA7E/z1ALgHN4yIgWeAj1T1TKANCfy6iEhD4G4gQ1VbA+Vw44vFrbhI+oQw0UsiUdVtqrrEu78P96H+xTwGiUJEGgFXAS/7HYvfRKQmcCHwCoCqHlXV3f5G5bvyQBURKQ9UJc5HAo6XpB/SZC2JSERSgHbAAn8j8dXTwP/wy5ndElFzYDvwmtfc9bKIVPM7KL+o6nfARNyUr9uAPar6sb9RRVa8JP3STPQSt0SkOvAO8DtV3et3PH4QkV8DP6rqYr9jiRHlgXTgBVVtBxwAEvYcmDfke0+gGfAroJqI9Pc3qsiKl6Rvk7UUICIVcAl/qqr+n9/x+Kgj0ENENuGa/S4RkTf8DclXWUCWqub+8puB+xJIVJcCG1V1u6r+DPwfcL7PMUVUvCT90kz0EndERHBttmtU9Um/4/GTqj6gqo1UNQX3f/GpqsZ1Ta4wqvo9sEVEzvBWdQFW+xiS374FOohIVe9z04U4P7EdysToMU9Vs0Ukd6KXcsCrqrrK57D81BG4EfhKRJZ560aq6iwfYzKx4y5gqldB2gDc4nM8vlHVBSIyA1iC6/W2lDgfksGGYTDGmAQSL807xhhjQmBJ3xhjEoglfWOMSSCW9I0xJoFY0jfGmARiSd8YYxKIJX1jjEkg/w9vPq3feGhK2QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(len(acc))\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一次，验证精度达到了 87%。还不错，肯定比 SimpleRNN 网络好多了，这主要是因为LSTM 受梯度消失问题的影响要小得多。这个结果也比第 3 章的全连接网络略好，虽然使用的数据量比第 3 章要少。此处在 500 个时间步之后将序列截断，而在第 3 章是读取整个序列。 \n",
    "\n",
    "但对于一种计算量如此之大的方法而言，这个结果也说不上是突破性的。为什么 LSTM 不能表现得更好？一个原因是你没有花力气来调节超参数，比如嵌入维度或 LSTM 输出维度。另 一个原因可能是缺少正则化。\n",
    "\n",
    "但说实话，主要原因在于，适用于评论分析全局的长期性结构（这 正是 LSTM 所擅长的），对情感分析问题帮助不大。对于这样的基本问题，观察每条评论中出现了哪些词及其出现频率就可以很好地解决。这也正是第一个全连接方法的做法。但还有更加困 难的自然语言处理问题，特别是问答和机器翻译，这时 LSTM 的优势就明显了。\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "nbTranslate": {
   "displayLangs": [
    "*"
   ],
   "hotkey": "alt-t",
   "langInMainMenu": true,
   "sourceLang": "en",
   "targetLang": "fr",
   "useGoogleTranslate": true
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}