From d2a0ddfe2733db8f59a1bea484786894cd38482e Mon Sep 17 00:00:00 2001 From: colcarroll Date: Mon, 31 Aug 2020 08:11:34 -0700 Subject: [PATCH] Fix math rendering in probabilistic PCA. PiperOrigin-RevId: 329304301 --- .../jupyter_notebooks/Probabilistic_PCA.ipynb | 1165 +++++++++-------- 1 file changed, 586 insertions(+), 579 deletions(-) diff --git a/tensorflow_probability/examples/jupyter_notebooks/Probabilistic_PCA.ipynb b/tensorflow_probability/examples/jupyter_notebooks/Probabilistic_PCA.ipynb index bbd0daf75b..a8f512d231 100644 --- a/tensorflow_probability/examples/jupyter_notebooks/Probabilistic_PCA.ipynb +++ b/tensorflow_probability/examples/jupyter_notebooks/Probabilistic_PCA.ipynb @@ -1,607 +1,614 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "P20Z1RhWhPzX" - }, - "source": [ - "##### Copyright 2018 The TensorFlow Probability Authors.\n", - "\n", - "Licensed under the Apache License, Version 2.0 (the \"License\");" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": {}, - "colab_type": "code", - "id": "qS8MroChhSxR" - }, - "outputs": [], - "source": [ - "#@title Licensed under the Apache License, Version 2.0 (the \"License\"); { display-mode: \"form\" }\n", - "# you may not use this file except in compliance with the License.\n", - "# You may obtain a copy of the License at\n", - "#\n", - "# https://www.apache.org/licenses/LICENSE-2.0\n", - "#\n", - "# Unless required by applicable law or agreed to in writing, software\n", - "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", - "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", - "# See the License for the specific language governing permissions and\n", - "# limitations under the License." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "0ufLLTrrPIhi" - }, - "source": [ - "# Probabilistic PCA\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " View on TensorFlow.org\n", - " \n", - " Run in Google Colab\n", - " \n", - " View source on GitHub\n", - " \n", - " Download notebook\n", - "
" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "Py-bl6M32ZXY" - }, - "source": [ - "Probabilistic principal components analysis (PCA) is a\n", - "dimensionality reduction technique that\n", - "analyzes data via a lower dimensional latent space\n", - "([Tipping and Bishop 1999](#1)). It is often\n", - "used when there are missing values in the data or for multidimensional\n", - "scaling." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "WHNNrlLNPbpA" - }, - "source": [ - "## Imports" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": {}, - "colab_type": "code", - "id": "mbM5dCFdUior" - }, - "outputs": [], - "source": [ - "import functools\n", - "import warnings\n", - "\n", - "import matplotlib.pyplot as plt\n", - "import numpy as np\n", - "import seaborn as sns\n", - "\n", - "import tensorflow.compat.v2 as tf\n", - "import tensorflow_probability as tfp\n", - "\n", - "from tensorflow_probability import bijectors as tfb\n", - "from tensorflow_probability import distributions as tfd\n", - "\n", - "tf.enable_v2_behavior()\n", - "\n", - "plt.style.use(\"ggplot\")\n", - "warnings.filterwarnings('ignore')" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "QjIXsU55eq-d" - }, - "source": [ - "## The Model\n", - "\n", - "Consider a data set $\\mathbf{X} = \\{\\mathbf{x}_n\\}$ of $N$ data\n", - "points, where each data point is $D$-dimensional, $\\mathbf{x}_n \\in\n", - "\\mathbb{R}^D$. We aim to represent each $\\mathbf{x}_n$ under a latent\n", - "variable $\\mathbf{z}_n \\in \\mathbb{R}^K$ with lower dimension, $K <\n", - "D$. The set of principal axes $\\mathbf{W}$ relates the latent variables to\n", - "the data.\n", - "\n", - "Specifically, we assume that each latent variable is normally distributed,\n", - "\n", - "\\begin{equation*}\n", - "\\mathbf{z}_n \\sim N(\\mathbf{0}, \\mathbf{I}).\n", - "\\end{equation*}\n", - "\n", - "The corresponding data point is generated via a projection,\n", - "\n", - "\\begin{equation*}\n", - "\\mathbf{x}_n \\mid \\mathbf{z}_n\n", - "\\sim N(\\mathbf{W}\\mathbf{z}_n, \\sigma^2\\mathbf{I}),\n", - "\\end{equation*}\n", - "\n", - "where the matrix $\\mathbf{W}\\in\\mathbb{R}^{D\\times K}$ are known as\n", - "the principal axes. In probabilistic PCA, we are typically interested in\n", - "estimating the principal axes $\\mathbf{W}$ and the noise term\n", - "$\\sigma^2$.\n", - "\n", - "Probabilistic PCA generalizes classical PCA. Marginalizing out the the\n", - "latent variable, the distribution of each data point is\n", - "\n", - "\\begin{equation*}\n", - "\\mathbf{x}_n \\sim N(\\mathbf{0}, \\mathbf{W}\\mathbf{W}^\\top + \\sigma^2\\mathbf{I}).\n", - "\\end{equation*}\n", - "\n", - "Classical PCA is the specific case of probabilistic PCA when the\n", - "covariance of the noise becomes infinitesimally small, $\\sigma^2 \\to 0$.\n", - "\n", - "We set up our model below. In our analysis, we assume $\\sigma$ is known, and\n", - "instead of point estimating $\\mathbf{W}$ as a model parameter, we\n", - "place a prior over it in order to infer a distribution over principal\n", - "axes. We'll express the model as a TFP JointDistribution, specifically, we'll\n", - "use [JointDistributionCoroutineAutoBatched](https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/JointDistributionCoroutineAutoBatched)." - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": {}, - "colab_type": "code", - "id": "Gwc6maZifKnb" - }, - "outputs": [], - "source": [ - "def probabilistic_pca(data_dim, latent_dim, num_datapoints, stddv_datapoints):\n", - " w = yield tfd.Normal(loc=tf.zeros([data_dim, latent_dim]),\n", - " scale=2.0 * tf.ones([data_dim, latent_dim]),\n", - " name=\"w\")\n", - " z = yield tfd.Normal(loc=tf.zeros([latent_dim, num_datapoints]),\n", - " scale=tf.ones([latent_dim, num_datapoints]),\n", - " name=\"z\")\n", - " x = yield tfd.Normal(loc=tf.matmul(w, z),\n", - " scale=stddv_datapoints,\n", - " name=\"x\")" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": {}, - "colab_type": "code", - "id": "2whVRGHPdXNm" - }, - "outputs": [], - "source": [ - "num_datapoints = 5000\n", - "data_dim = 2\n", - "latent_dim = 1\n", - "stddv_datapoints = 0.5\n", - "\n", - "concrete_ppca_model = functools.partial(probabilistic_pca,\n", - " data_dim=data_dim,\n", - " latent_dim=latent_dim,\n", - " num_datapoints=num_datapoints,\n", - " stddv_datapoints=stddv_datapoints)\n", - "\n", - "model = tfd.JointDistributionCoroutineAutoBatched(concrete_ppca_model)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "x2zF-wrTVSK4" - }, - "source": [ - "## The Data" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "n4HYhXlTVZ1_" - }, - "source": [ - "We can use the model to generate data by sampling from the joint prior distribution." - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": { - "height": 85 + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "P20Z1RhWhPzX" + }, + "source": [ + "##### Copyright 2018 The TensorFlow Probability Authors.\n", + "\n", + "Licensed under the Apache License, Version 2.0 (the \"License\");" + ] }, - "colab_type": "code", - "id": "b23iIkX8VVyn", - "outputId": "acae7840-0adf-43cd-f92d-550b2a7e42dc" - }, - "outputs": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Principal axes:\n", - "tf.Tensor(\n", - "[[ 2.2801023]\n", - " [-1.1619819]], shape=(2, 1), dtype=float32)\n" - ] - } - ], - "source": [ - "actual_w, actual_z, x_train = model.sample()\n", - "\n", - "print(\"Principal axes:\")\n", - "print(actual_w)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "O2ZdIFz7VuSO" - }, - "source": [ - "We visualize the dataset." - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": { - "height": 282 + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "qS8MroChhSxR" + }, + "outputs": [], + "source": [ + "#@title Licensed under the Apache License, Version 2.0 (the \"License\"); { display-mode: \"form\" }\n", + "# you may not use this file except in compliance with the License.\n", + "# You may obtain a copy of the License at\n", + "#\n", + "# https://www.apache.org/licenses/LICENSE-2.0\n", + "#\n", + "# Unless required by applicable law or agreed to in writing, software\n", + "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", + "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", + "# See the License for the specific language governing permissions and\n", + "# limitations under the License." + ] }, - "colab_type": "code", - "id": "ubJJvk0KVyVW", - "outputId": "b0ecbbc5-1045-499d-d45a-6da1848969ff" - }, - "outputs": [ { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEJCAYAAAB8Pye7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt0VeWd//H33vvkkIRcOMnJxYQg\nBhBBa9UCKj8ttkR7c7lYtnXBXLzN0uWo7ai1HUdGrQt1xSpip6PtXNSZsTOzhFUv80+nayIj2kEl\nFSnKRQgYriG3c3K/nr2f3x8bDiAghJxDctyf11pZ5Fz3N4fks/d+nmc/j2WMMYiISKDYY12AiIic\neQp/EZEAUviLiASQwl9EJIAU/iIiAaTwFxEJIIW/iEgAKfwl49x8881YloVlWYRCIYqKirj88st5\n9NFHicViI36/6dOn89Of/jT1hZ6GX//611iWNdZlSAAo/CUjXXnllTQ1NbF7927eeecdbrvtNv7z\nP/+T888/n23bto11eSLjnsJfMlI4HKa8vJyKigrOP/98br31VtatW0dubi533HFH8nnr16/nW9/6\nFqWlpeTl5TF37lz++7//O/n4VVddxY4dO3j00UeTZxONjY0YY7jtttuYNm0aOTk5VFdX8+CDDzI4\nOPi5db3xxhtcfPHF5ObmMmnSJObNm8eHH36YfLyhoYHvfve7TJo0iUgkwjXXXMNHH30EwFtvvcWf\n//mfAyRrufnmm1P4qYkcpvCXL4yCggL+8i//krfeeovW1lYAurq6WLx4MW+99Rbr16/nG9/4Btdd\nd13y7ODVV19l6tSp/OhHP6KpqYmmpiaqqqowxlBWVsZ//Md/sGXLFp599lleeuklnnjiiRNu/8CB\nA3z/+99nyZIlbNq0iXfffZd77rmHUCgEQHNzM1dccQWlpaW88847vPfee8ycOZOrrrqK1tZW5s+f\nz9///d8DJGv5+c9/nuZPTQLLiGSYm266ySxcuPC4j/32t781gHn//fdP+PoLL7zQPPbYY8nb06ZN\nM4888shJt/vMM8+Y6dOnn/Dx9evXG8B8+umnx338kUceMZdeeulR93meZ6qrq82KFSuMMca8/PLL\nRn+WciaExnjfI5JS5uA8hYc6TVtbW3nkkUdYvXo1Bw4cIJFIMDAwwK5du076Xv/0T//EP//zP9PY\n2Ehvby+JRALP8074/AsvvJBvfOMbXHDBBVx99dVcddVVXH/99VRVVQFQX1/PBx98QF5e3lGv6+/v\nZ/v27af7I4ucFoW/fKF8/PHHWJZFdXU14I8M2r17Nz/72c8455xzyMnJYfHixQwNDX3u+6xatYq7\n7rqL2tpaFixYQEFBAatWrWLp0qUnfI3jOPz2t7+lvr6euro6fvOb3/DAAw+watUqrr32WjzPY+HC\nhcmmnSMVFhaO7gcXGSGFv3xhdHV18ctf/pKFCxdSXFwMwNtvv83PfvYzrrvuOgB6e3vZuXMnF1xw\nQfJ14XAY13WPeq+3336biy++mPvuuy95X2Nj40lrsCyLefPmMW/ePB588EG++c1v8tJLL3Httdcy\nZ84c/uVf/oXKykpycnKO+/pwOAyA67o4jjOin19kJNThKxlpaGiIAwcO0NTUxObNm3nxxReZN28e\ng4OD/PKXv0w+b+bMmfz7v/87H330ERs2bGDJkiXHBP0555zD//3f/7F7927a2trwPI+ZM2fy0Ucf\n8cYbb7Bjxw5+/vOf8+qrr35uTWvXrmXZsmW8//777N69mzfffJONGzcye/ZsAO6++25c12XRokW8\n8847NDY28vvf/56lS5eydu3aZC0A//Vf/0Vrays9PT2p/NhEDhvrTgeRkbrpppsMYADjOI6ZNGmS\nufTSS82jjz5qYrHYUc/duHGjufzyy012drY5++yzzXPPPWcWLlxobrrppuRz6uvrzSWXXGKys7OT\nHbZDQ0Pm9ttvN5FIxOTn55slS5aYX/ziF5/bGfvxxx+bb33rW6asrMyEw2EzZcoUc//995vBwcHk\ncxobG82f/MmfmGg0mnzOn/7pn5qdO3cmn/NXf/VXprS01FiWdVSdIqlkGaOVvEREgkbNPiIiATTq\nDt+2tjaee+45Ojo6sCyLmpoavv3tb9PT08OKFStobW2lpKSEe++995ghbiIiMjZG3ewTj8eJx+NU\nV1fT39/PAw88wI9//GPeeust8vLyWLRoEa+//jo9PT382Z/9WarqFhGRURh1s08kEkmOqc7JyaGy\nspJYLEZ9fT0LFiwAYMGCBdTX1492UyIikiIpHeff0tLCp59+yvTp0+ns7CQSiQD+DqKrq+u4r6mr\nq6Ourg6A2traVJYjIiInkLLwHxgYYPny5dx8883k5uae8utqamqoqalJ3t6/f3+qSkqbaDRKW1vb\nWJdxUqoztVRn6mRCjZA5dVZUVIz4NSkZ7ZNIJFi+fDlXXnkll156KeBfrh6PxwG/X6CgoCAVmxIR\nkRQYdfgbY/jVr35FZWUl1157bfL+OXPmsGbNGgDWrFnD3LlzR7spERFJkVE3+3zyySe8/fbbTJky\nhR//+McALFmyhEWLFrFixQpWr15NNBo9ao4UEREZW6MO//POO4+VK1ce97GHH354tG8vIiJpoCt8\nRUQCSOEvIhJACn8RkQBS+IuIBJDCX0QkgBT+IiIBpPAXEQkghb+ISAAp/EVEAkjhLyISQAp/EZEA\nUviLiASQwl9EJIAU/iIiAaTwFxEJIIW/iEgAKfxFRAJI4S8iEkCjXsYR4Pnnn2f9+vUUFhayfPly\nAFauXMmbb75JQUEB4K/re8kll6RicyIiMkopCf+rrrqKb37zmzz33HNH3f+d73yH6667LhWbEBGR\nFEpJs8/s2bPJy8tLxVuJiMgZkJIj/xP53e9+x9tvv011dTU33nijdhAiIuOEZYwxqXijlpYWnnzy\nyWSbf0dHR7K9/5VXXiEej3PnnXce87q6ujrq6uoAqK2tZWhoKBXlpFUoFCKRSIx1GSelOlNLdaZO\nJtQImVNnOBwe8WvSduQ/adKk5PcLFy7kySefPO7zampqqKmpSd5ua2tLV0kpE41GVWcKqc7UyoQ6\nM6FGyJw6KyoqRvyatA31jMfjye/XrVtHVVVVujYlIiIjlJIj/2effZbNmzfT3d3NHXfcwQ033MCm\nTZtobGzEsixKSkq4/fbbU7EpERFJgZSE/z333HPMfV//+tdT8dYiIpIGusJXRCSAFP4iIgGk8BcR\nCSCFv4hIACn8RUQCSOEvIhJACn8RkQBS+IuIBJDCX0QkgBT+IiIBpPAXEQkghb+ISAAp/EVEAkjh\nLyISQAp/EZEAUviLiASQwl9EJIAU/iIiAaTwFxEJoJSs4fv888+zfv16CgsLWb58OQA9PT2sWLGC\n1tZWSkpKuPfee8nLy0vF5kREZJRScuR/1VVX8eCDDx513+uvv86XvvQl/u7v/o4vfelLvP7666nY\nlIiIpEBKwn/27NnHHNXX19ezYMECABYsWEB9fX0qNiUiIimQkmaf4+ns7CQSiQAQiUTo6uo67vPq\n6uqoq6sDoLa2lmg0mq6SUiYUCqnOFFKdqZUJdWZCjZA5dZ6OtIX/qaqpqaGmpiZ5u62tbQyrOTXR\naFR1ppDqTK1MqDMTaoTMqbOiomLEr0nbaJ/CwkLi8TgA8XicgoKCdG1KRERGKG3hP2fOHNasWQPA\nmjVrmDt3bro2JSIiI5SSZp9nn32WzZs3093dzR133MENN9zAokWLWLFiBatXryYajXLfffelYlMi\nIpICKQn/e+6557j3P/zww6l4exERSTFd4SsiEkAKfxGRAFL4i4gEkMJfRCSAFP4iIgGk8BcRCSCF\nv4hIACn8RUQCSOEvIhJACn8RkQBS+IuIBJDCX0QkgBT+IiIBpPAXEQkghb+ISAAp/EVEAkjhLyIS\nQAp/EZEASskyjp/nrrvuIjs7G9u2cRyH2tradG9SREROIu3hD/DII49QUFBwJjYlIiKnQM0+IiIB\nZBljTDo3cNddd5GXlwfA1VdfTU1NzVGP19XVUVdXB0BtbS1DQ0PpLCclQqEQiURirMs4KdWZWqoz\ndTKhRsicOsPh8Ihfk/bwj8ViFBUV0dnZyWOPPcYtt9zC7NmzT/j8/fv3p7OclIhGo7S1tY11GSeV\nyjpdF+Jxi0TCIhQyRCIGx0nJWwfy80ynTKgzE2qEzKmzoqJixK9Je7NPUVERAIWFhcydO5eGhoZ0\nb1JSzHVh/36b4WELgOFhi/37bVx3dO/Z1mZx4IBNayujei8RGbm0dvgODAxgjCEnJ4eBgQE2btzI\n9773vXRuUtKgrc2iq8vC8yzA4HkQj9t8/LHD5MkepaUe0ejhM4HPniUUFBi6uo6+vX+/TW+v/56J\nBLS2OkSjhv5+mDgRJkxI7dmFiBwtreHf2dnJ008/DYDrulxxxRVcdNFF6dykpJjrwr59DqEQDA/D\nzp0hWlosJkyAwkI4cMCiv9+iv9+jqsoD/LOEUAgsC3p7LT78MEQ0asjKMhhjsWGDf8JZXGywLNi7\nF+Jxh/Z2Q0mJoacHSks99u+3qKjwtAMQSYO0hn9ZWRlPPfVUOjchaRaPW0yYYOjvt9i0ySEWs4nF\nLMJh6Okx2LZFb6/D8DDk5PjdR4fOEjzPsGOHg+fZ9PV5hEIwMOCfRTiOwfMMEyYY8vJgcNBicNCi\nrMzFsvz3iEQM8bhFNJrWbimRQDoj4/wlcyUSFvn5hk2bQgwO+sE9PGzT1wfhsMfu3TZVVR5DQzZ7\n9oAxMGECeB40NISIxSwmTYLdu236+mxKSz26uy2Mgf5+m+Fhj/Jy2LfPIhSCggIbxzGAxdCQobsb\nior8M4Ti4qObl0Tk9Cn85XOFQob2dpuiIkMs5ge04xhs228GGh72m34mTPBD2XH88O/utrBti6ws\n2LPHoq0NYjGLrVsdBgY42Axk0dsLTU1+s09BgXcw/CEry9Dbaw42KRlKSjz27nUYGPCYPFlNQSKj\npfCXzxWJGHbvtgiHDdEoWJYhHoe+PouODouCAsjK8r/6+21KSjwSCf+MwbYNg4MWW7b4o4M8L4Rt\nGwYGDPG43+TT0eE3GeXlWUSjNrEYVFW55OX5ncpTpvj9Dq2tDhMmWGzd6pGX5zc3OY6hstKjvPzz\nO5zVcSxyLIW/fC7HgcpKl+Zmm9ZWm2jU3wls3+6P0qmshJISP4A7Oy36+2HqVI+BAQiHbTZudGht\ntejrsxkags5Om9xc8DxDX58hFLLJy4OmpixaWiAUShCLGfLz/aahHTugo8MiN9dQWmoYGnKYONEw\nY4bfFLRrl2HKFI+KCpfzz3dxnMMdzp4H7e02u3dbVFa6ajISOYLCX04qGjUMDRnmzBlm+3aH7m6b\naBTOOcdQWGgoL/ewbcjN9TtmbRsmT/b4+OMQiYRFd7dFb69NPG4xPAzd3QAWrmvhONDZ6W8nLw96\ne0P88Y82eXl+k9DwsIVl+U1JrmsIhw0TJ8LOnS7V1R4TJsCBAzYlJTY7dzqUlblMnGiRleU3GU2Y\n4I86amhw2LcP7QREDlL4y0k5DlRUeLS1WUyaZMjJcYlELDzPP7p2Xejrg5wcyM72sG1DT4/fdu83\nATl0dFgMDPgdwkc68uKuQzsB8DuUjzQ4CGDR2+u/T0uLzZYtHjk5HgUFhunTDeXlNtGoQ0GBIRbz\nm55mznTJzvabgfLy/P6J885zOfts9RtIsCn85ZQc6sytrPSwLD+0m5v98fodHRbFxQbXhfx8fzin\n5x0O86Ehv4koVROJ9Pcfel9/BFFnJ/T2uuzf7x3shPb7KAoLDX/4g0VBgc2MGS5nnWXo6XHYssVm\n2jSP6dM9Sko0gkiCSeEvpyyR8JtgwN8RlJV57NplY1kWWVlecjx+W5vNwAAUFEBrq43r+tcFpGvO\nvkQCWlocWlr8WoyBcNhvKgqFIDfX74PYuNFvkgLYtMljwQKX8nKPSZP85qO+PouCAqMdggSCwl9O\nWShkkm3w4O8A8vJg0iSXoiI/+GMxf3in40A4bMjP9+jsNHR3W2egQit5djE0dHhnE4/bNDVBfr7f\n2WyMwbI8YjGb8893KSz0RwQVFflXGxcUGMrK/A7k05gsUSQjKPzllEUihv37reTUDX7Q+nP1HOK6\nfsiHw4ZZs1z+8Aeb/n6XwUELYxx6esamds/zm4f8pigL23bo7LTZu9emstKltNQjK8tvrqqs9Kiq\nsojFLGbN8giH/Z2DyBeJwl9O2aGO30Nj6LOyDOed59HcbGOMv0Pwx/H7fQAA3/52gjfesMnJgf37\nDQcOhBgY8C8OG0ue5zfz9PU5tLc7FBZ6FBV5hMM227a55OdDTo7D177mcd55CSZONJx/vj9Xka4b\nkC8Chb+MiONwsG3/8JHwkTuEaNSjv9/CtqGw0FBUZJg/P0FTk8X06R5/+INHV5dDR4c/9HM8GBry\n+yZaW+2DHds2eXn+dQW/+Y3LzJk2BQWGjz4CY8JkZfk/U2HhsTOWascgmULhL6P22R2C65rkzuDs\ns13OOstl27YQHR02Z5/t8eabBssK4bp+M5DnjW39R3Jd/ysWOzQBnUVTk98h/OGHMGOGg+fB1q0O\nCxYMMzxskZ0NPT3+jiwvzzBrVoKyMu0EZHxT+EvKHf/sYJi2NoutW0Ps3WsRDifIzfXYuzd08GKu\nsav3RA5NUzE46Kd4UxNs2RKmoAAiEZdNmywKCizKyjzKyvx1CAoL/fUOhoZcqqp0LYGMXwp/OSP8\noaEGx0nQ2emHZjjsnyV0dkJHB8dc2DUeDQ35k9S1tYUAv68gJ8ehvNxQUeExc6aL69oHm79sjPHP\nCDQjqYw3Cn85oyIRQ0WFS2enzYUXujiOYe/eLGw7QWGhPzdQX5+DMePzbOBoFp2dDp2dEIvB7t0e\n27ZZzJiRYOPGMOecY5g92x9KeuSMpKCJ52TsKfzljHIcuPBCf06HffsccnM9IhG/mcQYgzE227d7\ntLb6i8YMDdkZsBPwO439hW5surpsJk2CTz7x2LrVZsoUj7PP9ti716ax0cPzIBLxh8MWFMD+/X7T\nkTqO5UxS+MsZFw7DxRe7TJ3qkUhYtLZCe/swu3Y5dHZ6FBVBc7Nh/Xp/NtCOjvRdHZwO/tXGYNsO\nDQ3+iKDy8gSzZ3vk5toUFVlEIh45OYbhYZuSkgTNzRaVlf5MpcPDlpawlLRLe/hv2LCBl156Cc/z\nWLhwIYsWLUr3JiUDHNkpHI3CH/9omDLF5dNPHVzXv0r4S18ybN3qkp9v09Fh0dNzaIK3zODPb2TR\n2WmxZ0+YrVtdystdzjrLIivLprra7yfYuTMLxzF8/evDyVFCoRBawlLSKq3h73keL7zwAn/7t39L\ncXExf/M3f8OcOXOYPHlyOjcrGaakBKJRj3DYYmDAwhi/wzQ725Cd7c/Q6Tj+QjJtbbBrV+jgZHH+\nmUGm6O526O52iMUMOTnw8cf+zz1xokdFheG112D+/MPrErS02CQSRs1AkhZpDf+GhgbKy8spKysD\nYP78+dTX1yv85SiO48//H4/7VwZPneoHX1eXzQUXuPT0WBQW+ks7Tpzo0dDg8v77Dt3dFnv3WsRi\n/tw9rpsZrZjt7f4IoFDIX6EsL8+mvd1wzjn+1dG7d/vLZpaUGDzPwxhLC9JIyqX1ryUWi1FcXJy8\nXVxczPbt2496Tl1dHXV1dQDU1tYSjUbTWVJKhEIh1ZlCoVCIsrIoB48RAP9Cq717Sa7I1dHhT+U8\nZQp89atwxRXw9tvQ2uo/9sc/wief+MNF09c0lNo/l0TC/7erC3p6oLU1i+pqf+TQl78MZWX+Y57n\nj3zq6IDiYpg/31874YRVZsD/eybUCJlT5+lIa/ib40zgbllHX9JfU1NDTU1N8nZbW1s6S0qJaDSq\nOlPoRHUeWoTl0DxCpaX+UW9vr7985He/C9u22axfHyYchtzcEJ98YtHd7dDVla6pI9yTP+U0eJ4f\n9Bs2QFFRAsvy6Ox0sW1/gjnwm34qK2HPHpdrrhk+eJ3EscNGy8rG//97pv9ujjcVFRUjfk1aw7+4\nuJj29vbk7fb2diKRSDo3KV8gx7tS+EjhMFxwgce55w6wY4fNrFkef/yjRUODzbZtDt3dIfr6rIwa\nKQQQi4V4803D1Kn+hXDGOEya5JFIeIRChr4+2LbNZdYsL7le8ZGjhL6gB6qSYmkN/2nTptHU1ERL\nSwtFRUWsXbuWH/7wh+ncpARQOAyzZnmce+4Ql11msWOHw+uve3z6qd++3t7uj78fT3MInczQkMW2\nbVlkZ/tnBaEQ7NhhUV7uMXmywXWz6O5OkJ1tMWGCv2rZoVFCsRhY42POPBnH0hr+juNw66238vjj\nj+N5Hl/72teoqqpK5yYlwA5NIRGNJvA8w8aNFp9+atHXZ7Fzp0V3t83QkKG93WZgIDN6TQcG/H/9\nJSuho8NhcNCmpyeBZdlMnGgoK/Nob7coLzfk5Xnk5/uL7Ih8nrQPj7jkkku45JJL0r0ZkSTHgXnz\n/PbyWbMsWlttvvpV2L7dJjfXY8sWm337XA4csBkethgaGrtFZkZqYAC2bLHo7LT59FN/RFBursOM\nGR5tbYbKSpueHrjyys/vFBbJjLFxIiMUDsOcOQkaG/1hk4ODhjlzDC0tFtOnO2zZAuGwR0ODw/bt\n0NTkB6qbnv7cFLM5cMAmHIa2NkNhIcRiHpGI3w9w6aXw/vshrrwykRwWeryOYQ0ZDTaFv3xhhcNw\n7rkecLixf9o0cF2Xiy+22LgxREGBx5Qp/vw6W7Y49PRYxGIO3d3j++IxzzvUJGTR1we9vQ62DW1t\n0N4OluWwbZtHYaG/dGVPj01urmHSJIMxmj5CFP4SQI4DVVWGqqrDM8a5LmzaZLN+fRbvvz/Mrl02\nvb1ZdHRAR4dNIjF+e1CN8YeJtrZahEL+2cvwsE1fX5ipU10sy590znH8506fnqC42Gj6iIBT+Ivg\n7xDOP98jEhnmoossVq/OIh5P0N4Otm2xa5fF3r0OHR0ew8M2x7mEZUwZA01NNj09NvE4FBc7B4eL\n+juFnByoqDBUV0NDQ4ihIZfKSo8TDaOVLz6Fv8hBhxaoD4ctLr88wYcfhpg925+cbcoU2L0bWlsT\nxGIOg4OGtjabzs7x1TzU3e1fOdzZaWOMze7dNtGoxcSJhqEhA3hMm+bR12dlTCe3pIfCX+QIRw4X\nnTLF5aOPQmRn25xzjsuiRbB+/TBbt7oMDFg0N1v09tp0dlrE4/76BDD2zUP9/dDf7+A40NcXIicH\nhocNrptg1y4L2wbwm4fa2mwKCw3RqFYaCxqFv8hxOA5UVhoqKw/3C0QiMGFCgpkzLfbtc2hrs+js\n9MjK8tcbePddl+Zmf/rp/n5/IZpEwhqzi8sOLUa/ezdUVEA4bNHRARs3+lNGRCIWU6f6E+ZVVrr0\n93tadzhAFP4ip+jQ7KMTJ1qcdZbBsvxZOB0HurosqqocNm1ycF1zsH/A0Ntr0dVl6OiwGBqy8Dz7\njK9J0N/vjwIaHLQJhTxCoRCxGEyZ4uG6/kVibW0hdu70p5auqPAX2gmHz2ydcmYp/EVG4LPzDbmu\nP2omFIL584cpK/OIxWxmzoSPP/ZDv7zco60N4nGb1lb/385Of1jpmeg49jzo6bFwXb9JqrDQYmAA\nHMemudlwwQXD2LZNUZE/fHR42GLnTsMFFyQIhXRdwBeVwl9kFD67MyguTrBjh82+fSGuuSbB8LCF\n41i0t1sMDXns3m2zb5+ht9dlzx6b9nYb1zX09joMDqZ3Z2AMhEI2nZ3+jKktLRCNWtTXh5g71+XA\nAYu+PkNfn0U4bNPba3HZZQldF/AFpfAXSaEjJ5mLx/2mnp4eKCnxg/2CC/yFarZssSkvNxjjMjAA\nu3d7NDbaWFaIwUHIyvLn8B8e9tcosCx/R+O6foifaj/CobH9/uygh+cKMsYfwjowAK5rsWOHX092\ntqGz099+d7c/ZLS01KOgwD/DiUSMrhT+glD4i6TBsc1DJKdfLitzmTXLZd8+i+3b/WmnZ850aWz0\n2LXLYNuG7m6LUMhicNCis9MwPOyQn+8vaemvcWzo7fVH7NgHR5smEv5Owbb9cHecw48d+nd4+PCO\nIBTyGB6G/HwXy/JXUTu0s+jutgCLzZsdOjps8vM9pk1z6e+3jplCWmcEmUnhL3IGHLqG4NBRc3a2\nYc4cj4svdtmxw6a52aGigoMzcx5a3MUiHrcpKPDYvz9BPG4xfbrBtv2Ltvbu9V83NGTwPL//wbJs\nsrMdBgddcnNtenoMw8OQk+Nf4dvZ6dcTDhvy8z2KivxRTV1dkEhYuK6/s+rqsigosBgeNnieRUuL\nP/ndV77iJqeLtiwtNJ/JFP4iZ8jxFqdxHL+ZaNYsvx2nv3+Y+voQQ0M2AwP+Qu/d3Rbf+pZLSYl/\nhN3fb7F1q01jo8OBA8O0t9uAxeCgob/fYsKELCwrQU+PTVYW9PcbXNcmHreYPNmjq8ti0iSLggKP\nK67w6OiA6dM9WlosCgr8s4PCQn9bBQWH63Rdm64uj6Kiw/VbFgenvlD4ZxqFv8g4kpMD/+//JU7S\nrm6YOtVjaCjBpk0Omzdn0d4OubmQSHi4LliWRzjsUlho2LrVpq3Nn+ohFPKboFpabCIRj9xcw5w5\nCdrbbaqrYeJEw86dDu3tUFBgKCgAxzEUFBgGBo4NemP86wQk8yj8RcaZky1feUg4DBde6HcY5+aG\nsG3/iL2qCvbudbFtfwz/3Lmwbl0IYyw6Omw6OqCoyB/Tn5fnf1VUJOjtBWMsbNuQn29TWOjvLIzx\nzwYmT/bnuzbGP+I3xu9nKC1V+Gcihb9IBnMcOPtsw4QJbrIjNisL8vMN5eUeFRV+c5Jt+2cJWVlw\n1lmGUOjwYi+W5X+de66XHFHU2Og3Kxlj4TiGiRP9M5CzzvKbjRIJv9O5tFSjfTKVwl8kw4XDhtJS\nP5T9kUBQWuoRDh8+Is/NNVRWGizLP3p3Xb9TF/zgP3LEjuPA1KkeeXnmYH8CFBcfnvvnVM5KZPxL\nW/ivXLmSN998k4KDPUZLlizRco4iaRCJGPbv98fgW5YhEvEv4IpEzGee44/csW1/6Gdenr/TyM4+\n9uj90AR3ZWUZsbSZnIa0Hvk5OuLyAAAMJUlEQVR/5zvf4brrrkvnJkQC77PDSMNhjhl77zhw3nku\nW7f6TTmhkKGoyGDM0TsJCQ41+4h8ARzZHBON+hO5fVY4DOef7+oKXQHAMiY9s4msXLmSNWvWkJOT\nQ3V1NTfeeCN5eXnHPK+uro66ujoAamtrGRoaSkc5KRUKhUgkEmNdxkmpztRSnamTCTVC5tQZPo0p\nWEcV/suWLaOjo+OY+xcvXsyMGTOS7f2vvPIK8XicO++886TvuX///tMt54yJRqO0He/QapxRnaml\nOlMnE2qEzKmzoqJixK8ZVbPPQw89dErPW7hwIU8++eRoNiUiIimUtgVI4/F48vt169ZRVVWVrk2J\niMgIpa3D99e//jWNjY1YlkVJSQm33357ujYlIiIjlLbw/8EPfpCutxYRkVFKW7OPiIiMXwp/EZEA\nUviLiASQwl9EJIAU/iIiAaTwFxEJIIW/iEgAKfxFRAJI4S8iEkAKfxGRAFL4i4gEkMJfRCSAFP4i\nIgGk8BcRCSCFv4hIACn8RUQCSOEvIhJACn8RkQAa1TKO7777LqtWrWLfvn088cQTTJs2LfnYa6+9\nxurVq7Ftm1tuuYWLLrpo1MWKiEhqjOrIv6qqivvvv59Zs2Yddf/evXtZu3YtzzzzDEuXLuWFF17A\n87xRFSoiIqkzqvCfPHkyFRUVx9xfX1/P/PnzycrKorS0lPLychoaGkazKRERSaFRNfucSCwWY8aM\nGcnbRUVFxGKx4z63rq6Ouro6AGpra4lGo+koKaVCoZDqTCHVmVqZUGcm1AiZU+fpOGn4L1u2jI6O\njmPuX7x4MXPnzj3ua4wxp1xATU0NNTU1ydttbW2n/NqxEo1GVWcKqc7UyoQ6M6FGyJw6j9cCczIn\nDf+HHnpoxG9aXFxMe3t78nYsFqOoqGjE7yMiIumRlqGec+bMYe3atQwPD9PS0kJTUxPTp09Px6ZE\nROQ0jKrNf926dbz44ot0dXVRW1vL1KlTWbp0KVVVVVx++eXcd9992LbNX/zFX2DbuqRARGS8GFX4\nz5s3j3nz5h33seuvv57rr79+NG8vIiJposNxEZEAUviLiASQwl9EJIAU/iIiAaTwFxEJIIW/iEgA\nKfxFRAJI4S8iEkAKfxGRAFL4i4gEkMJfRCSAFP4iIgGk8BcRCSCFv4hIACn8RUQCSOEvIhJACn8R\nkQBS+IuIBNColnF89913WbVqFfv27eOJJ55g2rRpALS0tHDvvfdSUVEBwIwZM7j99ttHX62IiKTE\nqMK/qqqK+++/n3/8x3885rHy8nKeeuqp0by9iIikyajCf/LkyamqQ0REzqBRhf/naWlp4Sc/+Qk5\nOTksXryYWbNmpWtTIiIyQicN/2XLltHR0XHM/YsXL2bu3LnHfU0kEuH5558nPz+fnTt38tRTT7F8\n+XJyc3OPeW5dXR11dXUA1NbWEo1GR/oznHGhUEh1ppDqTK1MqDMTaoTMqfN0nDT8H3rooRG/aVZW\nFllZWQBUV1dTVlZGU1NTskP4SDU1NdTU1CRvt7W1jXh7Z1o0GlWdKaQ6UysT6syEGiFz6jw0uGYk\n0jLUs6urC8/zAGhubqapqYmysrJ0bEpERE7DqNr8161bx4svvkhXVxe1tbVMnTqVpUuXsnnzZlau\nXInjONi2zW233UZeXl6qahYRkVEaVfjPmzePefPmHXP/ZZddxmWXXTaatxYRkTTSFb4iIgGk8BcR\nCSCFv4hIACn8RUQCSOEvIhJACn8RkQBS+IuIBJDCX0QkgBT+IiIBpPAXEQkghb+ISAAp/EVEAkjh\nLyISQAp/EZEAUviLiASQwl9EJIAU/iIiAaTwFxEJIIW/iEgAjWoN35dffpkPPviAUChEWVkZd955\nJxMnTgTgtddeY/Xq1di2zS233MJFF12UkoJFRGT0RnXkf+GFF7J8+XKefvppzjrrLF577TUA9u7d\ny9q1a3nmmWdYunQpL7zwAp7npaRgEREZvVGF/5e//GUcxwHg3HPPJRaLAVBfX8/8+fPJysqitLSU\n8vJyGhoaRl+tiIikxKiafY60evVq5s+fD0AsFmPGjBnJx4qKipI7hs+qq6ujrq4OgNraWioqKlJV\nUlqpztRSnamVCXVmQo2QOXWO1EmP/JctW8aPfvSjY77q6+uTz3n11VdxHIcrr7wSAGPMKRdQU1ND\nbW0ttbW1PPDAA6fxI5x5qjO1VGdqZUKdmVAjfLHrPOmR/0MPPfS5j7/11lt88MEHPPzww1iWBUBx\ncTHt7e3J58RiMYqKikZcnIiIpMeo2vw3bNjAG2+8wV//9V8zYcKE5P1z5sxh7dq1DA8P09LSQlNT\nE9OnTx91sSIikhrOT3/605+e7osff/xxhoaGeO+99/if//kfGhsb+cpXvkJhYSE9PT38wz/8A7//\n/e+59dZbT7ndrLq6+nTLOaNUZ2qpztTKhDozoUb44tZpmZE00IuIyBeCrvAVEQkghb+ISAClbJz/\naGTKNBHvvvsuq1atYt++fTzxxBNMmzYNgJaWFu69995kv8aMGTO4/fbbx12dML4+zyOtXLmSN998\nk4KCAgCWLFnCJZdcMsZV+TZs2MBLL72E53ksXLiQRYsWjXVJx3XXXXeRnZ2Nbds4jkNtbe1YlwTA\n888/z/r16yksLGT58uUA9PT0sGLFClpbWykpKeHee+8lLy9v3NU5Hn8v29raeO655+jo6MCyLGpq\navj2t7898s/UjAMbNmwwiUTCGGPMyy+/bF5++WVjjDF79uwx999/vxkaGjLNzc3m7rvvNq7rjlmd\ne/bsMfv27TOPPPKIaWhoSN7f3Nxs7rvvvjGr67NOVOd4+zyP9Morr5g33nhjrMs4huu65u677zYH\nDhwww8PD5v777zd79uwZ67KO68477zSdnZ1jXcYxNm3aZHbs2HHU38jLL79sXnvtNWOMMa+99lry\nb34sHa/O8fh7GYvFzI4dO4wxxvT19Zkf/vCHZs+ePSP+TMdFs0+mTBMxefLkjLja70R1jrfPMxM0\nNDRQXl5OWVkZoVCI+fPnH3WBo5zc7NmzjzkCra+vZ8GCBQAsWLBgXHymx6tzPIpEIsmRPTk5OVRW\nVhKLxUb8mY6LZp8jne40EWOtpaWFn/zkJ+Tk5LB48WJmzZo11iUdY7x/nr/73e94++23qa6u5sYb\nbxwXf4ixWIzi4uLk7eLiYrZv3z6GFX2+xx9/HICrr76ampqaMa7mxDo7O4lEIoAfZl1dXWNc0YmN\nx9/LQ1paWvj000+ZPn36iD/TMxb+y5Yto6Oj45j7Fy9ezNy5c4HRTRORKqdS52dFIhGef/558vPz\n2blzJ0899RTLly8nNzd3XNU5Fp/nkT6v5muuuYbvfe97ALzyyiv827/9G3feeeeZLvEYx/vMDl3J\nPt4sW7aMoqIiOjs7eeyxx6ioqGD27NljXVZGG6+/lwADAwMsX76cm2+++bSy5oyFf6ZME3GyOo8n\nKyuLrKwswL/QoqysjKampqM6WlPtdOoc62k3TrXmhQsX8uSTT6a5mlPz2c+svb09eXQ13hz6vyws\nLGTu3Lk0NDSM2/AvLCwkHo8TiUSIx+PJDtXxZtKkScnvx9PvZSKRYPny5Vx55ZVceumlwMg/03HR\n5p/p00R0dXUl1ytobm6mqamJsrKyMa7qWOP584zH48nv161bR1VV1RhWc9i0adNoamqipaWFRCLB\n2rVrmTNnzliXdYyBgQH6+/uT32/cuJEpU6aMcVUnNmfOHNasWQPAmjVrTni2OtbG4++lMYZf/epX\nVFZWcu211ybvH+lnOi6u8P3BD35AIpFItqUdOVTy1Vdf5X//93+xbZubb76Ziy++eMzqXLduHS++\n+CJdXV1MnDiRqVOnsnTpUt577z1WrlyJ4zjYts33v//9MQ2IE9UJ4+vzPNIvfvELGhsbsSyLkpIS\nbr/99nFzhL1+/Xr+9V//Fc/z+NrXvsb1118/1iUdo7m5maeffhoA13W54oorxk2dzz77LJs3b6a7\nu5vCwkJuuOEG5s6dy4oVK2hrayMajXLfffeNeVv68erctGnTuPu93Lp1Kw8//DBTpkxJtpIsWbKE\nGTNmjOgzHRfhLyIiZ9a4aPYREZEzS+EvIhJACn8RkQBS+IuIBJDCX0QkgBT+IiIBpPAXEQmg/w+r\nFE9a9OxXDgAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "0ufLLTrrPIhi" + }, + "source": [ + "# Probabilistic PCA\n", + "\n", + "\u003ctable class=\"tfo-notebook-buttons\" align=\"left\"\u003e\n", + " \u003ctd\u003e\n", + " \u003ca target=\"_blank\" href=\"https://www.tensorflow.org/probability/examples/Probabilistic_PCA\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" /\u003eView on TensorFlow.org\u003c/a\u003e\n", + " \u003c/td\u003e\n", + " \u003ctd\u003e\n", + " \u003ca target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/probability/blob/master/tensorflow_probability/examples/jupyter_notebooks/Probabilistic_PCA.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" /\u003eRun in Google Colab\u003c/a\u003e\n", + " \u003c/td\u003e\n", + " \u003ctd\u003e\n", + " \u003ca target=\"_blank\" href=\"https://github.com/tensorflow/probability/blob/master/tensorflow_probability/examples/jupyter_notebooks/Probabilistic_PCA.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" /\u003eView source on GitHub\u003c/a\u003e\n", + " \u003c/td\u003e\n", + " \u003ctd\u003e\n", + " \u003ca href=\"https://storage.googleapis.com/tensorflow_docs/probability/tensorflow_probability/examples/jupyter_notebooks/Probabilistic_PCA.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/download_logo_32px.png\" /\u003eDownload notebook\u003c/a\u003e\n", + " \u003c/td\u003e\n", + "\u003c/table\u003e" ] - }, - "metadata": { - "tags": [] - }, - "output_type": "display_data" - } - ], - "source": [ - "plt.scatter(x_train[0, :], x_train[1, :], color='blue', alpha=0.1)\n", - "plt.axis([-20, 20, -20, 20])\n", - "plt.title(\"Data set\")\n", - "plt.show()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "qIZepLMgItfq" - }, - "source": [ - "## Maximum a Posteriori Inference\n", - "\n", - "We first search for the point estimate of latent variables that maximizes the posterior probability density. This is known as maximum a posteriori (MAP) inference, and is done by calculating the values of $\\mathbf{W}$ and $\\mathbf{Z}$ that maximise the posterior density $p(\\mathbf{W}, \\mathbf{Z} \\mid \\mathbf{X}) \\propto p(\\mathbf{W}, \\mathbf{Z}, \\mathbf{X})$.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": {}, - "colab_type": "code", - "id": "s2AvQAYqIh6K" - }, - "outputs": [], - "source": [ - "w = tf.Variable(tf.ones([data_dim, latent_dim]))\n", - "z = tf.Variable(tf.ones([latent_dim, num_datapoints]))\n", - "\n", - "target_log_prob_fn = lambda w, z: model.log_prob((w, z, x_train))\n", - "losses = tfp.math.minimize(\n", - " lambda: -target_log_prob_fn(w, z),\n", - " optimizer=tf.optimizers.Adam(learning_rate=0.05),\n", - " num_steps=200)" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": { - "height": 286 }, - "colab_type": "code", - "id": "ya-XoAtpY474", - "outputId": "d59fcbd3-5df7-4b0d-9213-d3248c36ae9e" - }, - "outputs": [ { - "data": { - "text/plain": [ - "[]" + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "Py-bl6M32ZXY" + }, + "source": [ + "Probabilistic principal components analysis (PCA) is a\n", + "dimensionality reduction technique that\n", + "analyzes data via a lower dimensional latent space\n", + "([Tipping and Bishop 1999](#1)). It is often\n", + "used when there are missing values in the data or for multidimensional\n", + "scaling." ] - }, - "execution_count": 0, - "metadata": { - "tags": [] - }, - "output_type": "execute_result" }, { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAD8CAYAAACLrvgBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt8lPWB7/HPbyYBcmcuCWkQVAyo\n0GAiyQqohGL0nEq7tWo9tdt2TeGllVZPyNbW1r523bZaXrUQGxLW1sOx1tJtrUfSbretuzQltKbU\nAEZBUYzxRrkEMjEkJJDJzO/8MTBySSD3Z5J836/XvDLzzPOb+c7DkG9+zzMXY621iIiI9IHL6QAi\nIjJ6qDRERKTPVBoiItJnKg0REekzlYaIiPSZSkNERPpMpSEiIn2m0hARkT5TaYiISJ+pNEREpM/i\nnA4wHPbt2zfgsX6/n8OHDw9hmqGhXP0Tq7kgdrMpV//Eai4YWLasrKw+raeZhoiI9JlKQ0RE+kyl\nISIifabSEBGRPlNpiIhIn6k0RESkz1QaIiLSZyqNE+zRdsL/8XOCDbudjiIiErPO++a+devWsWPH\nDtLS0li9ejUA7e3tlJWVcejQIdLT01m5ciXJyckAbNy4kerqalwuF8XFxeTm5gLQ2NhIZWUlXV1d\n5OXlUVxcjDGGYDBIRUUFjY2NpKSkUFJSQkZGBgCbN2/m2WefBeDmm29m8eLFw7ENIlwu7K9/Rldq\nKhTeOHz3IyIyip13prF48WK+8Y1vnLasqqqKnJwcysvLycnJoaqqCoC9e/dSW1vLmjVreOCBB1i/\nfj3hcBiAxx9/nLvuuovy8nIOHDhAfX09ANXV1SQlJbF27VqWLl3Khg0bgEgxPfPMMzz88MM8/PDD\nPPPMM7S3tw/pgz+VSUgEXwbd77w5bPchIjLanbc0Zs+eHZ1FnFRXV0dhYSEAhYWF1NXVRZcvXLiQ\n+Ph4MjIyyMzMpKGhgZaWFjo7O5k1axbGGBYtWhQds23btugMYv78+ezatQtrLfX19cydO5fk5GSS\nk5OZO3dutGiGTdZ0ut9tHN77EBEZxQZ0TKO1tRWPxwOAx+PhyJEjAAQCAXw+X3Q9r9dLIBA4a7nP\n5yMQCJw1xu12k5iYSFtbW6+3NZzMBRfS/bd3sN3dw3o/IiKj1ZB+YKG1tl/Le7vOGNPjur0t37Rp\nE5s2bQJg1apV+P3+80XtUedlORz53f/Dc/wocZmXDOg2hktcXNyAH9dwUq7+i9VsytU/sZoLhjfb\ngEojLS2NlpYWPB4PLS0tpKamApEZRHNzc3S9QCCA1+s9a3lzczNer/e0MT6fj1AoREdHB8nJyXi9\nXl599dXTbmv27Nk95ikqKqKoqCh6eaCfPGlTI7OnwCsv4UpKG9BtDJdY/URN5eq/WM2mXP0Tq7kg\nBj/lNj8/n5qaGgBqamooKCiILq+trSUYDNLU1MT+/fvJzs7G4/GQkJDAnj17sNayZcsW8vPzAZg3\nbx6bN28GYOvWrcyZMwdjDLm5ubz00ku0t7fT3t7OSy+9FH0l1rDJvADcbvjbO8N7PyIio9R5ZxqP\nPvoor776Km1tbXzxi1/ktttu46abbqKsrIzq6mr8fj+lpaUATJs2jQULFlBaWorL5WLZsmW4XJFe\nWr58OevWraOrq4vc3Fzy8vIAWLJkCRUVFdxzzz0kJydTUlICQHJyMrfccgtf//rXAbj11lvPOiA/\n1ExcPO6s6YRUGiIiPTL2XAccRqnBfAlT3JM/4PjunbhX/Z8hTDR4sToVVq7+i9VsytU/sZoLYnD3\n1FgWf9FMaG7CHh2+94SIiIxWKo0zxGVfHjnzToOzQUREYpBK4wzxl1wKgFVpiIicRaVxBldyKqRn\nYt9WaYiInEml0QNz0UztnhIR6YFKoycXZkcOhre1Op1ERCSmqDR6YC7KjpzRbENE5DQqjZ5MvwSM\nwb71htNJRERiikqjByYhEaZeiG149fwri4iMIyqNXpjs2fDm69hQyOkoIiIxQ6XRm5mz4XgnvKcv\nZRIROUml0Qszcw4A9g3tohIROUml0Qvj8YF/CvaNV5yOIiISM1Qa52BmzoY3XsWGw05HERGJCSqN\nc7l0LrQfgb1vO51ERCQmqDTOwcyJfFGUfWWHw0lERGKDSuMczGQvXHAxdpdKQ0QEVBrnZXKuhDd3\nYzs7nI4iIuI4lcZ5mDnzIBSC1152OoqIiONUGudzyaWQkIh96QWnk4iIOE6lcR4mLh5zxd9hX9yK\n7e52Oo6IiKNUGn1g8q+BjnZ47SWno4iIOEql0Rez8yK7qLY973QSERFHqTT6wMTHY664KrKLKhh0\nOo6IiGNUGn1kriqEjnZs/V+djiIi4hiVRl/NzgVfBnbL751OIiLiGJVGHxmXC3PtDfDay9iD+5yO\nIyLiCJVGP5hrrge3G7v5d05HERFxhEqjH0yaB5N/DfZPz2HbjjgdR0RkxKk0+snc+Ck4fgz7h187\nHUVEZMSpNPrJZE2HKxdgq/8Te7TN6TgiIiNKpTEAro9/Go51Yv/j505HEREZUSqNATAXXIxZdAP2\nj/+J/du7TscRERkxKo0BMp/4LExKIPzTddhwyOk4IiIjIm4wg3/zm99QXV2NMYZp06axYsUKurq6\nKCsr49ChQ6Snp7Ny5UqSk5MB2LhxI9XV1bhcLoqLi8nNzQWgsbGRyspKurq6yMvLo7i4GGMMwWCQ\niooKGhsbSUlJoaSkhIyMjME/6iFgUlIxn74T+3/LsM9txHz0VqcjiYgMuwHPNAKBAL/73e9YtWoV\nq1evJhwOU1tbS1VVFTk5OZSXl5OTk0NVVRUAe/fupba2ljVr1vDAAw+wfv16wuEwAI8//jh33XUX\n5eXlHDhwgPr6egCqq6tJSkpi7dq1LF26lA0bNgzBQx46Zv5izLyrsb/agG3Y7XQcEZFhN6jdU+Fw\nmK6uLkKhEF1dXXg8Hurq6igsLASgsLCQuro6AOrq6li4cCHx8fFkZGSQmZlJQ0MDLS0tdHZ2MmvW\nLIwxLFq0KDpm27ZtLF68GID58+eza9curLWDiTykjDGYz30JfBmE1z2MbW5yOpKIyLAacGl4vV4+\n/vGPc/fdd3PnnXeSmJjIFVdcQWtrKx6PBwCPx8ORI5E3wQUCAXw+32njA4HAWct9Ph+BQOCsMW63\nm8TERNraYutlriYpGdeXvwndQcI/+FdsW6vTkUREhs2Aj2m0t7dTV1dHZWUliYmJrFmzhi1btvS6\nfm8zhHPNHHq6zhhz1rJNmzaxadMmAFatWoXf7z9f/F7FxcX1f7zfT9c3vkfLt0txlX8Lz7/+AFfq\n5AFnGLJcI0C5+i9WsylX/8RqLhjebAMujZ07d5KRkUFqaioAV111FXv27CEtLY2WlhY8Hg8tLS3R\n630+H83NzdHxgUAAr9d71vLm5ma8Xu9pY3w+H6FQiI6OjuhB9VMVFRVRVFQUvXz48OGBPiz8fv/A\nxmdOx7XiAborH+LQfctxlTyISc8ccI4hyzXMlKv/YjWbcvVPrOaCgWXLysrq03oD3j3l9/t54403\nOH78ONZadu7cydSpU8nPz6empgaAmpoaCgoKAMjPz6e2tpZgMEhTUxP79+8nOzsbj8dDQkICe/bs\nwVrLli1byM/PB2DevHls3rwZgK1btzJnzpweZxqxwszJw1X6LWg/Qvi792HfaXA6kojIkBrwTGPm\nzJnMnz+fr33ta7jdbi666CKKioo4duwYZWVlVFdX4/f7KS0tBWDatGksWLCA0tJSXC4Xy5Ytw+WK\ndNby5ctZt24dXV1d5ObmkpeXB8CSJUuoqKjgnnvuITk5mZKSkiF4yMPLZM/Gdf/3CP/gQcKPfAPX\nnfdh5hY4HUtEZEgYG0svRxoi+/YN/PsuhmrKad8PEF77bXjvLcz/Wo7ruo8N6vZidSqsXP0Xq9mU\nq39iNRfE6O4pOTcz2Yvrq9+FufnYn/+I8L//SO8cF5FRT6UxjMzESbhWfB1z/Sew1b8h/MNHsMGg\n07FERAZMpTHMjMuN67ZlmNuWwY5awpXfwQa7nI4lIjIgKo0R4rr+E5jPfxleeZHwD7+H7e52OpKI\nSL+pNEaQ69obMJ/5Irz0AvanlTH1kSgiIn0xqE+5lf5zfeRGwkfex/7m5zDlAsxHb3E6kohIn2mm\n4QDz97djCq7FbvwJdvdLTscREekzlYYDjDGR4xtTphJevwZ7pMXpSCIifaLScIiZlIDrrq9Cx1HC\nT/2bjm+IyKig0nCQueAizN/fDvVbYUet03FERM5LpeEwc/1NMP0Swj/7IbbjqNNxRETOSaXhMON2\n4/r8l6CtFfvbXzodR0TknFQaMcBcmI1ZsAT7h19jm/Y7HUdEpFcqjRhhPvlZcLmxVT91OoqISK9U\nGjHCTPZhlnwMu+3P2H3vOh1HRKRHKo0YYm74JEyYhP3NL5yOIiLSI5VGDDEpqZglSyOzjQN/czqO\niMhZVBoxxhR9HNxx2E2/cjqKiMhZVBoxxqR6MPMXY2ursW1HnI4jInIalUYMMtd/AoJd2JrfOR1F\nROQ0Ko0YZLKmw+w87J+e0/eKi0hMUWnEKFfh/4DAYdi5w+koIiJRKo1YNffvIM1DWLuoRCSGqDRi\nlImLw1xzPezajg0ccjqOiAig0ohp5uoisBb71xqno4iIACqNmGbSMyH7cuxf/qgvaRKRmKDSiHFm\nwUdg/3t0N77udBQREZVGrDPzroG4eDr/qAPiIuI8lUaMM0nJkDOP47V/1Hs2RMRxKo1RwORfQ7jl\nMDTsdjqKiIxzKo1RwMwtgAkTsNv+7HQUERnnVBqjgJmUwMR5C7Hba7WLSkQcpdIYJSYtXAJH3tcu\nKhFxlEpjlJhw5YLI92y8VOd0FBEZx1Qao4QrMQku/TD25RecjiIi41jcYAYfPXqUxx57jPfeew9j\nDHfffTdZWVmUlZVx6NAh0tPTWblyJcnJyQBs3LiR6upqXC4XxcXF5ObmAtDY2EhlZSVdXV3k5eVR\nXFyMMYZgMEhFRQWNjY2kpKRQUlJCRkbG4B/1KGXm/h325z/CHtyHmZLldBwRGYcGNdN44oknyM3N\n5dFHH+WRRx5h6tSpVFVVkZOTQ3l5OTk5OVRVVQGwd+9eamtrWbNmDQ888ADr168nHA4D8Pjjj3PX\nXXdRXl7OgQMHqK+vB6C6upqkpCTWrl3L0qVL2bBhwyAf7uhmrigAwL6k2YaIOGPApdHR0cHu3btZ\nsmQJAHFxcSQlJVFXV0dhYSEAhYWF1NVF9sHX1dWxcOFC4uPjycjIIDMzk4aGBlpaWujs7GTWrFkY\nY1i0aFF0zLZt21i8eDEA8+fPZ9euXeP6M5iMfwpMvRD7so5riIgzBrx7qqmpidTUVNatW8c777zD\njBkzuOOOO2htbcXj8QDg8Xg4ciTyPdeBQICZM2dGx3u9XgKBAG63G5/PF13u8/kIBALRMSevc7vd\nJCYm0tbWRmpq6kBjj3pmbgH2uWexR9sj7xYXERlBAy6NUCjEW2+9xRe+8AVmzpzJE088Ed0V1ZPe\nZgjnmjn0dJ0x5qxlmzZtYtOmTQCsWrUKv99/vvi9iouLG9T44XIyV9ei62n53TMkv7uHhGtvcDpW\nzG+vWBSr2ZSrf2I1FwxvtgGXhs/nw+fzRWcP8+fPp6qqirS0NFpaWvB4PLS0tERnBT6fj+bm5uj4\nQCCA1+s9a3lzczNer/e0MT6fj1AoREdHR/Sg+qmKioooKiqKXj58+PBAHxZ+v39Q44fLyVzWmwEp\nabT9uZqjl1/pdKyY316xKFazKVf/xGouGFi2rKy+vbhmwMc0Jk+ejM/nY9++fQDs3LmTCy64gPz8\nfGpqIl8aVFNTQ0FB5OBtfn4+tbW1BINBmpqa2L9/P9nZ2Xg8HhISEtizZw/WWrZs2UJ+fj4A8+bN\nY/PmzQBs3bqVOXPm9DjTGE+My43Jycfu2o7t7nY6joiMM4N6ye0XvvAFysvL6e7uJiMjgxUrVmCt\npaysjOrqavx+P6WlpQBMmzaNBQsWUFpaisvlYtmyZbhckc5avnw569ato6uri9zcXPLy8gBYsmQJ\nFRUV3HPPPSQnJ1NSUjLIhzs2mCsKsLV/gDdfg0s/7HQcERlHjB2DL0c6OfsZiFidcp6ay3YcJbzy\nHzD/81Zcn/xszOSKJbGaC2I3m3L1T6zmghjdPSXOMYlJcPEs7O56p6OIyDij0hilzOxcePsN7NE2\np6OIyDii0hilzOxcsBZee9npKCIyjqg0RquLZkFCIvZV7aISkZGj0hilTFwcXJqj0hCREaXSGMXM\n7Fw4fBDbtN/pKCIyTqg0RjFzeeSj5TXbEJGRotIYzaZkgTddL70VkRGj0hjFjDGRXVS7X8aGQk7H\nEZFxQKUx2s3Ohc6j8PYbTicRkXFApTHKmcuuAMDq/RoiMgJUGqOcSUmFCy7Cvr7T6SgiMg6oNMYA\nc9lcaNiNDQadjiIiY5xKYwwwl82FYBc0vu50FBEZ41QaY8HMOWBcOq4hIsNOpTEGmMQkuChbpSEi\nw06lMUaYS3Pgrdexx485HUVExjCVxhhhLpsLoRC88arTUURkDFNpjBXZl4M7TruoRGRYqTTGCDNx\nEsyYpfdriMiwUmmMIeayufDOm9iOdqejiMgYpdIYQ8ylc8GGYc8rTkcRkTFKpTGWzLgU4ifouIaI\nDBuVxhhi4uMh+3KVhogMG5XGGGMumwt/ewfb1up0FBEZg1QaY4y5bG7kjF5FJSLDQKUx1lyYDZMS\ntItKRIaFSmOMMW43zJyDfU0zDREZeiqNMchcNhcO/g3b0ux0FBEZY1QaY9DJ4xr2de2iEpGhpdIY\niy64CJJSQMc1RGSIqTTGIONywaU52N0vY611Oo6IjCEqjTHKXJYDgUNw+KDTUURkDFFpjFHR4xra\nRSUiQ0ilMVZlXgBpHh3XEJEhFTfYGwiHw9x///14vV7uv/9+2tvbKSsr49ChQ6Snp7Ny5UqSk5MB\n2LhxI9XV1bhcLoqLi8nNzQWgsbGRyspKurq6yMvLo7i4GGMMwWCQiooKGhsbSUlJoaSkhIyMjMFG\nHheMMZhL52JfjxzXMMY4HUlExoBBzzR++9vfMnXq1OjlqqoqcnJyKC8vJycnh6qqKgD27t1LbW0t\na9as4YEHHmD9+vWEw2EAHn/8ce666y7Ky8s5cOAA9fX1AFRXV5OUlMTatWtZunQpGzZsGGzc8eWy\nHGhtgQN7nU4iImPEoEqjubmZHTt2cN1110WX1dXVUVhYCEBhYSF1dXXR5QsXLiQ+Pp6MjAwyMzNp\naGigpaWFzs5OZs2ahTGGRYsWRcds27aNxYsXAzB//nx27dqlVwP1g45riMhQG9TuqR//+Md89rOf\npbOzM7qstbUVj8cDgMfj4ciRIwAEAgFmzpwZXc/r9RIIBHC73fh8vuhyn89HIBCIjjl5ndvtJjEx\nkba2NlJTU0/LsWnTJjZt2gTAqlWr8Pv9A35McXFxgxo/XAaSy/p8HE7PJL7xdSZ/6h9jJtdIiNVc\nELvZlKt/YjUXDG+2AZfG9u3bSUtLY8aMGbzyyvm/Ka63GcK5Zg49XdfTvvmioiKKioqilw8fPnze\nPL3x+/2DGj9cBprLzprD8foXONTUFHn/RozkGm6xmgtiN5ty9U+s5oKBZcvKyurTegMujddff51t\n27bx4osv0tXVRWdnJ+Xl5aSlpdHS0oLH46GlpSU6K/D5fDQ3f/BZSIFAAK/Xe9by5uZmvF7vaWN8\nPh+hUIiOjo7oQXXpo8vmwvN/gL1vw/QZTqcRkVFuwH96fuYzn+Gxxx6jsrKSkpISPvzhD3PvvfeS\nn59PTU0NADU1NRQUFACQn59PbW0twWCQpqYm9u/fT3Z2Nh6Ph4SEBPbs2YO1li1btpCfnw/AvHnz\n2Lx5MwBbt25lzpw5ehVQP5lLdVxDRIbOoF9ye6abbrqJsrIyqqur8fv9lJaWAjBt2jQWLFhAaWkp\nLpeLZcuW4Tqxu2T58uWsW7eOrq4ucnNzycvLA2DJkiVUVFRwzz33kJycTElJyVDHHfOMxwdTpkZK\n44abnI4jIqOcsWPw5Uj79u0b8NhY3U85mFzhn67Dbq3B9egGTNzQ/p0wFrfXcIvVbMrVP7GaC4b3\nmIbeET4OmMvmwvFOeKfB6SgiMsqpNMaDS3MAsLtfcjiIiIx2Ko1xwKSkwUUzsbu2Ox1FREY5lcY4\nYeYWQOPr2LZWp6OIyCim0hgnzNwCsBa7U7MNERk4lcZ4MX0GTPbCy3VOJxGRUUylMU4YYzA5+dhX\nX8R2B52OIyKjlEpjHDFzC6CzA9541ekoIjJKqTTGk8uvgLh47MvbnE4iIqOUSmMcMRMnwWVzsTqu\nISIDpNIYZ8zcAmjahz3wN6ejiMgopNIYZ8zcyCcI25dfcDiJiIxGKo1xxvgyYPoM7PZap6OIyCik\n0hiHTP41kXeHNzc5HUVERhmVxjhk5l0NgN3+vMNJRGS0UWmMQybjQ3BhNrbuz05HEZFRRqUxTpn8\nq+HtN7AHB/6FVSIy/qg0xilz1WIwLuxfqp2OIiKjiEpjnDIeH8y+AvuXamw47HQcERklVBrjmFl4\nHQQOw+s7nY4iIqOESmMcM7lXQUIS9k//5XQUERklVBrjmJkwEXP1ddgdtdj3A07HEZFRQKUxzpnF\nN0IopNmGiPSJSmOcM1Oy4MNXYmt+ry9nEpHzUmkIriUfh9YA9q81TkcRkRin0hD48JWRDzH87TPY\ncMjpNCISw1QagjEG1423Rb5nQx8tIiLnoNKQiLz5kDUd++t/x3Z3O51GRGKUSkMAMC4Xrlv+MTLb\nqPm903FEJEapNOQDOfmR7xD/zb9jj7Y7nUZEYpBKQ6KMMbhuWwYdR7HPPOF0HBGJQSoNOY2ZdjHm\nhk9i//zf2N0vOR1HRGKMSkPOYj7+acj4EOEfl2s3lYicRqUhZzETJuJa/k/QGiD8ZDnWWqcjiUiM\niBvowMOHD1NZWcn777+PMYaioiJuvPFG2tvbKSsr49ChQ6Snp7Ny5UqSk5MB2LhxI9XV1bhcLoqL\ni8nNzQWgsbGRyspKurq6yMvLo7i4GGMMwWCQiooKGhsbSUlJoaSkhIyMjKF55HJO5uJZmE9+HvvM\nE9jfP4v56C1ORxKRGDDgmYbb7eZzn/scZWVlPPTQQzz33HPs3buXqqoqcnJyKC8vJycnh6qqKgD2\n7t1LbW0ta9as4YEHHmD9+vWET3z5z+OPP85dd91FeXk5Bw4coL6+HoDq6mqSkpJYu3YtS5cuZcOG\nDUPwkKWvzA03YQquxW78CbZ+q9NxRCQGDLg0PB4PM2bMACAhIYGpU6cSCASoq6ujsLAQgMLCQurq\n6gCoq6tj4cKFxMfHk5GRQWZmJg0NDbS0tNDZ2cmsWbMwxrBo0aLomG3btrF48WIA5s+fz65du7Sr\nZAQZYzB33AsXZhP+0fd1YFxEhuaYRlNTE2+99RbZ2dm0trbi8XiASLEcOXIEgEAggM/ni47xer0E\nAoGzlvt8PgKBwFlj3G43iYmJtLW1DUVk6SMzYSKue/8F0jMJV3wH2/Cq05FExEEDPqZx0rFjx1i9\nejV33HEHiYmJva7X2wzhXDOHnq4zxpy1bNOmTWzatAmAVatW4ff7zxe7V3FxcYMaP1wczeX3E/pO\nJS3f/BLh8m8x+cFy4mfNdj7XOcRqLojdbMrVP7GaC4Y326BKo7u7m9WrV3Pttddy1VVXAZCWlkZL\nSwsej4eWlhZSU1OByAyiubk5OjYQCOD1es9a3tzcjNfrPW2Mz+cjFArR0dERPah+qqKiIoqKiqKX\nDx8+PODH5Pf7BzV+uMRCLvu/H8Q+8nUCD96L60vfxFz64ZjI1ZNYzQWxm025+idWc8HAsmVlZfVp\nvQHvnrLW8thjjzF16lQ+9rGPRZfn5+dTUxP5XoaamhoKCgqiy2trawkGgzQ1NbF//36ys7PxeDwk\nJCSwZ88erLVs2bKF/Px8AObNm8fmzZsB2Lp1K3PmzOlxpiEjw3j9uO57GCb7CD/6L9gXdXBcZLwZ\n8Ezj9ddfZ8uWLUyfPp377rsPgNtvv52bbrqJsrIyqqur8fv9lJaWAjBt2jQWLFhAaWkpLpeLZcuW\n4XJFOmv58uWsW7eOrq4ucnNzycvLA2DJkiVUVFRwzz33kJycTElJyWAfrwyS8abj+up3CZd/i/C/\nraLDhuDKq52OJSIjxNgx+HKkffv2DXhsrE45Yy2XPX6M8GOrYNeOyEtzb/lHjMvtdKyoWNtep4rV\nbMrVP7GaC2J095SMb2biJFxf+iYJH70F+19VhCsewnZ2OB1LRIaZSkMGzMTFkXrnP2H+4Yvwyg7C\n370P+7d3nY4lIsNIpSGD5lp8I66Sf4X2I4QfLiX8p//SmzBFxiiVhgwJc/kVuP6lHC65HPuTCuzj\n38ceaXE6logMMZWGDBmT5sFV8iDmps9id/yF8DdXEP7jf2LDIaejicgQUWnIkDIuN66lt0VmHRdl\nY3/2Q8LfKcVu+zM2pPIQGe0G/TEiIj0xH7oA18pvRcqi6qeEf/g98GVgrvs4Jv8ajMd3/hsRkZij\n0pBhY4yJfLT6vIVQ/wLh/67CPr0e+/R6uDAbc8XfYWZcCtNnYFLSnI4rIn2g0pBhZ1xuuHIB7isX\nYPe9i63/a+T0658RfY3VZB+kT8F408GbHpmVTPZBSiqkpEVOEyfpY2REHKbSkBFlsqZjsqbDjZ/C\ndrTDu43Yd9+E997GBpqwDbvh/T9DKMRZL9qNn/BBgaSkYU4tlJS0yGwlJQ2SU7HJSU48PJExT6Uh\njjGJyXDZXMxlc09bbsMhaH0f3m+GtlZs2xFoex9O/IxcbsXuexfaWiHYFRl3ym00AUyYAIkpkJAY\nPZlJiZCYBJMSICEJEhJgYgJMmIiZOBEmTIT4Ez9PO02A+AkYl147IuObSkNijnG5weOLnIBz7ZCy\n1sLxY5HyaGuFtiPY9laSQkGOHtgHHUcjH29yrCNyvvlQ5HxnR2TcqbfVl3DxE04vk/gJMHEixMVD\nfDzExWNOOc+Z5+PiOTp5MuHjxyNjT64fFw/xcaetFx138qc7DuLiwO2Oqc/5kvFFpSGjmjEmMmuY\nlADpmZFlQJLfT+d5PrDNhkKRuKCIAAAIt0lEQVRwrDNyCh6HruPQ1XXi53HsiZ8ETyw7fjx63cll\ntut4pHy6g3A0stx2ByEYhO7uyPLuYGT9cBiA9jNzDOiBu6IFEvkZd0qpnDydct0py00v49qSUwgH\ng5Hr3PEfrHPabfY03g0u94nzrsh5l/vE7fRy3uUGl0vHqEYhlYaMW8bthqTkyKmn64f4/mw4BMFu\nfGkpNB88eKJMgtDdBcFTCubE8g/K58Qp1B0polDoxM9TTt0fXGdDwbPXOdZ5YlkwUpY9jOsInchw\nvscxlBvlzCLpoWQOT5hAyJ5c13XiurgPzp9Y3/Qy/rTzLveJYnNFivdEeUVOp54/ZZ0T92uiYyKn\n45M92Pb2s9c/7T56uM0z7+fkbbrd0fOxvBtUpSEyQozLDRPduJJTMce6zr/+CGQ6ld/v59ChQ5EZ\nUbSgunsvq5Pnw6HIdaEQhEMnSil01vLI5fApY8InfnZ/cP7MMaEQcXFuQp2dp99mOARdwdOW2ZO5\nQ6HIYzjztk/erg0PaPucWZbvD36Tn1tvRXNW2X1QPGbaxbjuvG9YY6k0RCTKGPPBX+YTJg7sNoY4\n0+Qh/t4Ka+2JUjlxsqFTLodOvy66TviDMrKRZWkpKbS2tPQ87pTbtL3drg334X5PluvJ9c9cLwSn\nPp4Tu2iHk0pDRMaV04pxECb4/Zg+lNlYO2oTuzvOREQk5qg0RESkz1QaIiLSZyoNERHpM5WGiIj0\nmUpDRET6TKUhIiJ9ptIQEZE+M9baIf0oGRERGbs00zjD/fff73SEHilX/8RqLojdbMrVP7GaC4Y3\nm0pDRET6TKUhIiJ95n7wwQcfdDpErJkxY4bTEXqkXP0Tq7kgdrMpV//Eai4Yvmw6EC4iIn2m3VMi\nItJn+j6NE+rr63niiScIh8Ncd9113HTTTY7kOHz4MJWVlbz//vsYYygqKuLGG2/k6aef5g9/+AOp\nqakA3H777Vx55ZUjnu9LX/oSkyZNwuVy4Xa7WbVqFe3t7ZSVlXHo0CHS09NZuXIlyck9f4XqcNi3\nbx9lZWXRy01NTdx2220cPXp0xLfZunXr2LFjB2lpaaxevRrgnNtn48aNVFdX43K5KC4uJjc3d8Ry\nPfXUU2zfvp24uDimTJnCihUrSEpKoqmpiZUrV5KVlQXAzJkzufPOO4clV2/ZzvV8d3KblZWVsW/f\nPgA6OjpITEzkkUceGdFt1tvviBF7nlmxoVDIfvnLX7YHDhywwWDQfuUrX7HvvfeeI1kCgYB98803\nrbXWdnR02Hvvvde+99579he/+IX91a9+5UimU61YscK2traetuypp56yGzdutNZau3HjRvvUU085\nEc1aG/m3XL58uW1qanJkm73yyiv2zTfftKWlpdFlvW2f9957z37lK1+xXV1d9uDBg/bLX/6yDYVC\nI5arvr7ednd3RzOezHXw4MHT1htuPWXr7d/O6W12qieffNL+8pe/tNaO7Dbr7XfESD3PtHsKaGho\nIDMzkylTphAXF8fChQupq6tzJIvH44kewEpISGDq1KkEAgFHsvRVXV0dhYWFABQWFjq27QB27txJ\nZmYm6enpjtz/7Nmzz5pl9bZ96urqWLhwIfHx8WRkZJCZmUlDQ8OI5briiitwn/j2ulmzZjn2POsp\nW2+c3mYnWWv5y1/+wtVXXz0s930uvf2OGKnnmXZPAYFAAJ/PF73s8/l44403HEwU0dTUxFtvvUV2\ndjavvfYazz33HFu2bGHGjBl8/vOfH9FdQKd66KGHALj++uspKiqitbUVj8cDRJ7QR44ccSQXwPPP\nP3/af+RY2Ga9bZ9AIMDMmTOj63m9Xsd+cVdXV7Nw4cLo5aamJr761a+SkJDApz/9aS6//PIRz9TT\nv12sbLPdu3eTlpbGhz70oegyJ7bZqb8jRup5ptLgxBfNn8EYZ7/Z99ixY6xevZo77riDxMREbrjh\nBm699VYAfvGLX/CTn/yEFStWjHiub3/723i9XlpbW/nOd74T3YcbC7q7u9m+fTuf+cxnAGJmm/Wm\np+edE5599lncbjfXXnstEPmFs27dOlJSUmhsbOSRRx5h9erVJCYmjlim3v7tYmWbnfnHiRPb7Mzf\nEb0Z6m2m3VNEZhbNzc3Ry83NzdHGdkJ3dzerV6/m2muv5aqrrgJg8uTJuFwuXC4X1113HW+++aYj\n2bxeLwBpaWkUFBTQ0NBAWloaLS0tALS0tEQPXo60F198kYsvvpjJkycDsbPNets+Zz7vAoFAdPuO\nlM2bN7N9+3buvffe6B9K8fHxpKSkAJHX+k+ZMoX9+/ePaK7e/u1iYZuFQiFeeOGF02ZmI73Nevod\nMVLPM5UGcMkll7B//36ampro7u6mtraW/Px8R7JYa3nssceYOnUqH/vYx6LLTz4ZAF544QWmTZs2\n4tmOHTtGZ2dn9PzLL7/M9OnTyc/Pp6amBoCamhoKCgpGPBuc/ddfLGwzoNftk5+fT21tLcFgkKam\nJvbv3092dvaI5aqvr+dXv/oVX/va15g4cWJ0+ZEjRwiHwwAcPHiQ/fv3M2XKlBHLBb3/2zm9zSBy\n3CwrK+u0Xdojuc16+x0xUs8zvbnvhB07dvDkk08SDof5yEc+ws033+xIjtdee41//ud/Zvr06dG/\n/G6//Xaef/553n77bYwxpKenc+edd474bOjgwYN8//vfByJ/bV1zzTXcfPPNtLW1UVZWxuHDh/H7\n/ZSWlo74sYPjx49z9913U1FREZ2qr127dsS32aOPPsqrr75KW1sbaWlp3HbbbRQUFPS6fZ599ln+\n+Mc/4nK5uOOOO8jLyxuxXBs3bqS7uzua5eTLRLdu3crTTz+N2+3G5XLxqU99alj/iOop2yuvvNLr\nv52T22zJkiVUVlYyc+ZMbrjhhui6I7nNevsdMXPmzBF5nqk0RESkz7R7SkRE+kylISIifabSEBGR\nPlNpiIhIn6k0RESkz1QaIiLSZyoNERHpM5WGiIj02f8HTMy5BylMm1sAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "WHNNrlLNPbpA" + }, + "source": [ + "## Imports" ] - }, - "metadata": { - "tags": [] - }, - "output_type": "display_data" - } - ], - "source": [ - "plt.plot(losses)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "TqTuhkTsP90b" - }, - "source": [ - "We can use the model to sample data for the inferred values for $\\mathbf{W}$ and $\\mathbf{Z}$, and compare to the actual dataset we conditioned on." - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": { - "height": 337 }, - "colab_type": "code", - "id": "T3O6PHe3XX8a", - "outputId": "4443cdf0-42e7-47d8-bc00-000067292a36" - }, - "outputs": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "MAP-estimated axes:\n", - "\n" - ] + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "mbM5dCFdUior" + }, + "outputs": [], + "source": [ + "import functools\n", + "import warnings\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import seaborn as sns\n", + "\n", + "import tensorflow.compat.v2 as tf\n", + "import tensorflow_probability as tfp\n", + "\n", + "from tensorflow_probability import bijectors as tfb\n", + "from tensorflow_probability import distributions as tfd\n", + "\n", + "tf.enable_v2_behavior()\n", + "\n", + "plt.style.use(\"ggplot\")\n", + "warnings.filterwarnings('ignore')" + ] }, { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt0XHW9///nvswtmWQyk8k9aUt6\no8VigRZqBQs2cLi4CiIgfHVxBA8sBHVRRAURdK0KBGnF46XHG3iU4zmWumjxskSNlaK/Ai1CAVta\nem+a+2QmmZlkbvvy+yPttKXXNDNN0nk/1spqsmfP/rwznbz2ns/+7M9WbNu2EUIIUVDU0S5ACCHE\n6SfhL4QQBUjCXwghCpCEvxBCFCAJfyGEKEAS/kIIUYD0kW4gFArxwx/+kL6+PhRFoampiauvvpp4\nPM5TTz1FT08PFRUVLF68GK/Xm4uahRBCjJAy0nH+kUiESCRCY2MjiUSCBx54gC9/+cu89NJLeL1e\nrrvuOlavXk08HufTn/50ruoWQggxAiPu9vH7/TQ2NgLg8Xioq6sjHA6zYcMGFixYAMCCBQvYsGHD\nSJsSQgiRIyPu9jlUd3c3u3btYsqUKfT39+P3+4GhHUQ0Gj3qc1paWmhpaQGgubk5l+UIIYQ4hpyF\nfzKZZNmyZXzmM5+hqKjopJ/X1NREU1NT9uf29vZclZQ3wWCQUCg02mWckNSZW1Jn7oyHGmH81Flb\nWzvs5+RktI9hGCxbtoxLLrmEiy66CACfz0ckEgGGzguUlpbmoikhhBA5MOLwt22bH/3oR9TV1fGx\nj30su3zOnDmsXbsWgLVr1zJ37tyRNiWEECJHRtzts3XrVl5++WUmTJjAl7/8ZQBuueUWrrvuOp56\n6inWrFlDMBjkvvvuG3GxQgghcmPEQz1zTfr8c0fqzK1c1WnbNslkEsuyUBQlB5UdzuVykUqlcr7d\nXBoPNcLYqtO2bVRVxe12H/G+OZU+/5yO9hFCnFgymcThcKDr+fnz03UdTdPysu1cGQ81wtir0zAM\nkskkHo9nxNuS6R2EOM0sy8pb8Iszm67rWJaVk21J+AtxmuWjq0cUjly9fyT8hRCiAEn4C1Gg/vjH\nP1JXV8f27dtPuO6KFSvo7Ow85bbWrVvHrbfeesL1brjhBt56663jrvPTn/6URCJxyrWIIRL+QhSo\n1atXc+GFF/LCCy+ccN2VK1fS1dV1Gqo6sZ/97GcS/jkg4S/EGGeaEAopdHaqhEIKpjnybQ4MDPD6\n66+zdOnSI8J/+fLlLFy4kKamJh577DF+//vf89Zbb/H5z3+eyy+/nEQiwUUXXUQ4HAbgrbfe4oYb\nbgDgzTffZNGiRVxxxRUsWrTohJ8qEokEn/vc52hqauKuu+4imUxmH3vggQe46qqruOyyy1i6dCkA\nTz/9NF1dXdx4443ZNo+2njgxGXIgxBhmmtDerqLroCiQySi0tyvU1lqMZATiiy++yKWXXsrkyZMp\nKyvjnXfeYdasWaxZs4YXX3yR3//+93g8HiKRCH6/n//+7//m4Ycf5oMf/OBxtztlyhSef/55dF3n\n5Zdf5oknnuCnP/3pMdf/5S9/icfjoaWlhc2bN3PllVdmH/vqV7+K3+/HNE0++clPsnnzZj772c/y\nk5/8hJUrVxIIBI653syZM0/9xSkQEv5CjGGRiJINfhj6V9eHlgeDp3595urVq7njjjsAuPbaa1m9\nejWzZs3i73//O5/85Cez48gPzMx7sqLRKPfeey+7du1CURQymcxx13/ttde4/fbbAZg5cyYzZszI\nPva73/2OX/3qV5imSVdXF9u2bTtqqJ/seuJwEv5CjGGGofD+kX2KMrQcTi38w+Ew69atY+vWrSiK\ngmmaKIrC17/+dWzbPqmhhIeONz/0Ctgnn3yS+fPn8/TTT9Pa2prtmjmeo7W3d+9efvzjH/OHP/yB\nsrIy7r333sO6hIa7njiS9PkLMYbpus37J2Cx7aHlp+oPf/gDn/jEJ1i/fj2vvfYar7/+OhMmTGD9\n+vUsWLCAX//619kTqgdm5i0uLiYej2e3UV9fz9tvv53d3gGxWIzq6moAnnvuuRPWctFFF7Fq1SoA\ntmzZwrvvvpvdjsfjobS0lJ6eHv72t79ln+P1erO1HG89cXwS/kKMYX6/jWGQ3QHYNhjG0PJT9cIL\nL3DVVVcdtuzqq69m1apVXHbZZVxxxRVcddVVXH755fzoRz8C4KabbuKBBx7InvC97777eOSRR/j4\nxz9+2PQHn/vc53j88ce59tprMU/izPStt97KwMAATU1NLF++nNmzZwNwzjnn8IEPfIDLLruM++67\n77BZgT/1qU/x6U9/mhtuuOG464njk4ndTkGhTUSWb4VW5+Dg4LBueGSaQ338hqGg6zZ+v33ck726\nrmMYxojrzKfxUCOMzTqP9v6Rid2EOANpGvtP7o6p4zQxzkm3jxBCFCAJfyGEKEAS/kIIUYAk/IUQ\nogDl5ITv8uXLeeONN/D5fCxbtgwYGuP717/+ldLSUmDovr7nn39+LpoTQggxQjkJ/0svvZQrr7yS\nH/7wh4ctv+aaa1i0aFEumhBCCJFDOen2mTlzJl6vNxebEkKcBv/5n//JZZddRlNTE5dffjlvvPEG\nAPfffz/vvfdeTto4dObPY/ne97437O2uWLGChx56aNTaB7jjjjvYs2dPtp2Pf/zjhz1++eWX89GP\nfvSwZY888ggXXHDBYbdhXLFiBbNmzeLyyy/n0ksv5Ve/+hUAf/nLX/I+Q2le+/z/9Kc/cf/997N8\n+fLDLg0XQgyDaaKEQqidnSihECOd0/n111+npaWFF198kZaWFlasWJG9SGjp0qVMmzYtF1WflO9/\n//unra1ctb9161Ysy2LixInZZfF4nLa2NgC2bdt2xHMsy+KPf/wjNTU1vPrqq4c9tmjRIv7yl7/w\nm9/8hubmZnp6emhqauLPf/5zXu9bkLeLvK644orspE4rVqzgl7/8JXffffcR67W0tNDS0gJAc3Mz\nwWAwXyXljK7rUmcOFVqdXV1dJ38Dd9NE7e4emspT14fmd+juxqqt5XiX+R5v+729vZSXl1NcXAxA\nZWVl9rGPf/zjfOMb32D27NmcddZZ3H777bz88sv4fD6+9rWvsWTJEvbt28eSJUu48sor+fWvf81b\nb73F448/DgxNvXD33Xfz4Q9/GEVR0DQNXdf593//d9rb20mlUvzHf/wHt956K48//jjJZJIrrriC\n6dOn81//9V/85je/4ac//SmZTIbzzz+fJ554Ak3T+L//+z++973vUVVVRWNjIy6X64jfMRwOc9dd\nd9Hb28t5550HcNz2lyxZckT7R1vv/e2sXr2aq666KrtcURSuvfZa/vCHP3D33Xfz29/+luuvv56V\nK1dm13n55ZeZMWMG1157Lb/97W/5yEc+kq1PVVV0Xae6uppJkybR0dFBTU0NH/7wh1mzZg3XXnvt\nYe27XK6cvA/zFv5lZWXZ7xcuXMgTTzxx1PWamppoamrK/lxIl/nnm9SZW7mqM5VKHTYfzvEooRAW\nHHG0b/f0YB8jAE40JcHFF1/M0qVLmTdvHpdccgmLFi3iQx/60NB2bRvTNDEMg8HBQS666CIefPBB\nPvvZz/L444/zv//7v7z33nvce++9NDU1YZomlmVl2zv0+Yd+v3TpUvx+P4lEgmuuuYaPfexjPPjg\ngzzzzDP8+c9/BuDdd99l1apVrF69GofDwYMPPshzzz3HRz7yEb797W/z4osvUlJSwo033sgHPvCB\nI37HJ598krlz57J48WJaWlp49tlnj9n+lVdeeUT7x6rzwKCVA9avX8+iRYsO+52vuuoqFi9ezJ13\n3smf/vQnfvCDH7By5crsOs8//3z2JjePPfYYiUQCh8Nx2Ou3Z88e9uzZQ0NDA4ZhMGvWLF555RWu\nueaaw9pPpVJHvA/H1PQOB24CAUMvVkNDQ76aEuKMpRgGR5vTWTGMU57sobi4mBdffJHXXnuNdevW\n8bnPfY4HH3yQT37yk4et53Q6ueyyywA4++yzcTqdOBwOZsyYwb59+4bV5jPPPMMf//hHYGj+rp07\nd2YncTvgH//4B++88w5XX301AMlkkmAwyJtvvsmHPvQhysvLgaFukp07dx7RxquvvsrPfvYzYOig\n8tAD0Pe3v2vXruzNYIZbZ1dXV7aWA/x+Pz6fjxdeeIGpU6dm74cAkE6nWbNmDd/85jfxer2cd955\nrF27NnvQ+9vf/pb169fjcrl44oknsrkZDAbzeuvMnIT/d7/7XTZv3kwsFuOuu+7ipptuYtOmTeze\nvRtFUaioqODOO+/MRVNCFBRb11EymcN3ALaN7XCMaLuapjF//nzmz5/P2WefzcqVK48If13Xs3Pt\nq6qKy+XKfn/giPbQef3h8Ln9D1i3bh1///vf+d3vfofH4+GGG2446nq2bXPjjTfy4IMPHrb8xRdf\nPKl7DMDR7w1wsu2f7Hput/uoyxctWsTXvvY1nnrqqcOWv/TSS0SjURYuXAgM3brS4/Fkw3/RokU8\n+uijR2wvmUzidrtP6vc+FTkJ/3vvvfeIZe8/0y2EGD7b70dpbyd7O6/9czrbh/TTD9f27dtRVZXG\nxkYANm3aRH19/Sltq6GhgV/84hdYlkVHRwcbN248Yp1YLIbP58Pj8bB9+/bsyCIAh8NBJpPB4XBw\n8cUXc9ttt3HHHXcQDAaJRCIMDAxw3nnn8cgjjxAOhykpKeH3v//9Ue/UNW/ePJ5//nnuvfde1qxZ\nQ19f37DaP956h5o6dSq7du06ojfjqquuoru7m0svvfSwI/bVq1ezdOlSrrvuOmBoVs558+ad8GTu\nzp07mT59+nHXGQmZ1VOIsUzTsGprUSKRoa4eh2Mo+EdwA9/BwUG+/vWvE41G0XWdSZMm8e1vf/uU\ntjV37lwmTJjAwoULmT59OrNmzTpinUsvvZRnn32WpqYmGhsbD7vY81Of+hRNTU3MmjWLH/zgB3zl\nK1/hlltuwbZtdF3n0Ucf5YILLuBLX/oSixYtoqqqilmzZh31XgGLFy/mnnvu4d/+7d+YN28edXV1\nw2p/2bJlx1zvUAsXLuSVV17JnrQ9wOv1cs899xy2LJFIsHbt2sPOeRYVFXHhhRdmzzUcy7p16474\nFJRLMp//KSi0E5T5Vmh1Dnc+/+Eai3PQv994qBGOXmcikeDGG2/khRdeOOkT98PV09PDPffcc9S7\noeVqPn+Z20cIIYbB4/Fw//3309nZmbc22traeOSRR/K2fZBuHyFOuzH2YVucgksvvTSv23//CKND\n5er9I0f+Qpxmh46WEWI4DMNAVXMT23LkL8Rp5na7SSaTpFKpkx7COBwul+uoQxHHkvFQI4ytOm3b\nRlXVnA3/lPAX4jRTFOWwi4BybTycQB8PNcL4qfNUSLePEEIUIAl/IYQoQBL+QghRgCT8hRCiAEn4\nCyFEAZLwF0KIAiThL4QQBUjCXwghCpCEvxBCFCAJfyGEKEAS/kIIUYAk/IUQogDlZGK35cuX88Yb\nb+Dz+Vi2bBkA8Xicp556ip6eHioqKli8eDFerzcXzQkhhBihnBz5X3rppXzta187bNnq1auZNWsW\n3/ve95g1axarV6/ORVNCCCFyICfhP3PmzCOO6jds2MCCBQsAWLBgARs2bMhFU0IIIXIgb/P59/f3\n4/f7AfD7/USj0aOu19LSQktLCwDNzc0Eg8F8lZQzuq5LnTkkdebWeKhzPNQI46fOUzHqN3Npamqi\nqakp+/N4uHHCeLnBg9SZW1Jn7oyHGmH81FlbWzvs5+RttI/P5yMSiQAQiUQoLS3NV1NCCCGGKW/h\nP2fOHNauXQvA2rVrmTt3br6aEkIIMUw56fb57ne/y+bNm4nFYtx1113cdNNNXHfddTz11FOsWbOG\nYDDIfffdl4umhBBC5EBOwv/ee+896vJHHnkkF5sXQgiRY3KFrxBCFCAJfyGEKEAS/kIIUYAk/IUQ\nogBJ+AshRAGS8BdCiAIk4S+EEAVIwl8IIQqQhL8QQhQgCX8hhChAEv5CCFGAJPyFEKIASfgLIUQB\nkvAXQogCJOEvhBAFSMJfCCEKkIS/EEIUIAl/IYQoQDm5jePx3HPPPbjdblRVRdM0mpub892kEEKI\nE8h7+AN84xvfoLS09HQ0JYQQ4iRIt48QQhQgxbZtO58N3HPPPXi9XgAuv/xympqaDnu8paWFlpYW\nAJqbm0mn0/ksJyd0XccwjNEu44RyWqdpQjgMmQw4HBAIgKad/OOnq848kjpzZzzUCOOnTqfTOezn\n5D38w+EwgUCA/v5+vvWtb3Hbbbcxc+bMY67f3t6ez3JyIhgMEgqFRruME8pZnaYJ+9rpizswLRVN\ntSjzpFCK3Si2ja0oKMkkOJ2gKGDbYBhYtbUntQMouNczz8ZDneOhRhg/ddbW1g77OXnv9gkEAgD4\nfD7mzp3L9u3b892kyDGrO0TPjjh2dwg1EsZImPRuCWN3D/1RqKEQaigEljX0BEUBXUeJRI65TdOE\nUEihs1Olp2foZyHE6ZPXE77JZBLbtvF4PCSTSd5++21uuOGGfDYpcs00iW9qJxF3YNkq2AZa3w7C\npo+970bxN0KQXkp9FtrAAFYwiFHioz+mY5gWoFBabDDQ2ocZS+AOt1NU5qJroJiI7yws3YVhQE+P\nRjBok0hAcTG4XDZ+v32yPUdCiGHKa/j39/ezdOlSAEzT5OKLL2b27Nn5bFLkmBWK0NlfhAMDw4C2\nNp3UHvAae1CqK7G6wdHfi11kUDa9HCtlEN7Tg11Vie12Mxi12PrnHvx+E//ezaR0B3s2DpIqUSnz\nbiQ67Tz2RU0G2yIMbM8QqNLo8viprFFob1eorbVkByBEHuQ1/KuqqnjyySfz2YTIs2iviVruw9jX\nzY49DvpjKlpPEtO2GPD6aFD76bACGKlutK4BMs5KBpIK5u4I0YazCf1tD46BKNqrb5BOx0hbOkm7\nlIw3TrwhgG1toySYIBZLYiaiFJHAO7CJZGwSpVUeIs4yglUyKE2IXDst4/zF+JXBQbE3w5sDNaSN\nCGZvH1rKJGo7MFPQ02VRFVRJFJXTmXCQ6VBxuHWSRV46X+9G29lKZWwL2t5dxJUiit0mZQO7SLl8\nJMK1eMKtpKacjzsyQLhiKpWpGKoKVnQnvcZUBrd20TWlBkXXKC+3CAalK0iIXJDwF8ellJcx0NGJ\nz6fQuy9DxuPD9poMqKX4B/oYVBR6+hyYZX7cbieWx4ff6sPevovK1ghKbwfpcB+JQY2icCuGlcCV\njuNwu/Du2kTKUvF2tOLMOCn3rGPg/EvIlJXjHWhHCUdJuioYyDgomVbNvj2Q6QhTV5lGderYfv9J\nDyfFNFEiERTDwNaH+VwhzkAS/uJw7wtJv9/P3tI6PF07KfVrWJpGVC3HF+kiFtNxl6i4rUFKwyGS\naj3lg50YHg+u9jaKO/fgaN1LvDOKGUtRYbZRTJwEXlyJKG4yZNBI9HvQCFKiDxD7825itZPpq5lJ\nclec0mAIZ+sW3u37BBUDrYTSNjtLdYwiHw6lk/IJLqoqTPwVGmpwKNBNEyIRBcNQ0HUbf6mBo6sd\ndB0UBSWTQWlvP+mhqEKciST8xUGmidp+eEg6utqpq6sjEvfSM1hCwO7E79PYZ/sJdOygWsvgCJRT\nUleCu2MP6dJSao3tqO/9A2Mgw0BbD9WxbjSSuBjAgUExfThIYwFuHJQRIUMXccOHywij7YpT3LaT\nAXc56R1OYikHlS9vZu+ki+lwN+IssjmrugN3OkrsHQeRqUH8E4uZPrkVxeuhu0NBc+mYJX56e3U6\n3olRG3QcvO7skKGodjA42q+6EKNCwl9kKZFINviHFgyFZIUaxizXmG12sW+XysCgQkXRACUz/Hi1\nQUrqnZhlftSUjSsWprj9bVR1kIGeCEWZOB5iuIjjIIXK0JtO2f+lkgFAw8BNL2l0HNYgRtJDVzJC\nKTbVmLgSadyRfdThp9XRSJV7F65SFwMlVYT2TSVe7mUgCCWVLrTaSiguwuztQKsM4ujvZ3fUR2en\nRnWNScBnoMf6UU0TE6QLSBQkCX+RpRjGweDPLlTQbYOqsysY6N5DideJy22h2SkwTUzbA4bJ4CD4\nzAReox892ofdHx8KdzODAwOdDBpDVxUe+PdQOmADOgbFGCRJUcsgFhpekhhYXEQXCbxclvkTvZkA\neiyDhUpmiwtbLyYRqCE08TwSdVOIDur0FtXTMKUPbyqEvms9ab+PXeUBzKlOqiY4Ud0OlGQSddMm\n7GAQ2+mUHYEoGBL+IsvWdZRM5vAdgG1jOxxoTg2zro4KZy+qZWH7nIQGivAMxojFFYrKwYo5KUr3\nYZoKqmWRNhWKMzFsQGNoFpGjBf8BCnBghhIHNgYpNIZ2Ck5AxcRLPxZQwgA2kL0w2IBM9xaqut8k\ntKGSLqWBwYrz6HjVTY3STbU3jsPnIar7eHdjBdvPm07FeUGq2rvxldlou3eD14vd3o45dSrKwICc\nHBZnNAl/kWX7/SiH9PkfmKPHrqwEIFVagSORwdJ1KAsQ6OykM+HBdGvoqknANUCmrp7BziglVg+6\namJio2EwFPknP4eDtv/r/Y434t+NgZsQZYRosLfT2P0vEvgYpBi3SyWtOSlSXLR5P8j2aCPTzQR9\npTpBqwfdpRN1qJSZvZS/9nvcc89GqQig2PbBk8NCnEEk/MVBmoZVW3twtI/DMRT8+496dZdKurIW\nPRpBMQ2s+noUt0oxGbzOOIa3gcHWCI6aelLpJObgLhTFSco2sLDxMIgCWBzs98+lA58cnICbNMW0\nkaINAyfpVBEZXCTxYg0M8E7/RLZGohSV7cZ02mRqavAEUljRECk9hTOxl7q5SbQJtQfnKaqqynHF\nQoweCX9xOE0b6v8+ykN+v017uwb+YPaDQcZSKAnYpHXQwiHiag3eWBxjsB9XNMy+2DSKY73YmQQV\ndKGRwkNyf1fQ0Y/uc/Jr7P9yAZAmTRoTsOgiyA4mJrfQsaueBH76S+rw7tyMrpikVA+xhgnoGR/R\nVJSquUVolUF8ZWN/Wl8hhkPCX5w0TYPaWis7ht7hsDn7bIuuLhXbBtPnR+3oIFVcjm+6ScbMUFKc\nZsc2g8rwVrriOj7CxDAoIY6DNDoWKuAgv1PMHjrbeTHgJUI5EQxUzJgFMQcGbtK4iXWWsu+tqXSW\n1ZN6swrH/NnsaZzE2XYPif5uSssPXlMgxHgl4S+GRdMgGLThkM8GB3cIGiUzqjGVGOr2dphQA64U\nfpIkulXMaIjuzgHMtE3QbqOcHhRsihnAQwI3ds67go7Fsf9rqBMKLDL7v2KU00N1uo2e7ho6whMI\ntcZJVOxmx7yrMZwOnKrBRZPfxVPtpTTZzaDhJu0owpo0EX+VQ/YJYlyQ8BcjdvgOQcUMTiGhJqC3\nnxJPJ6VFKXq9PhKpchxnKfxrb4BoT4bzYmspp50aOtDIoJDBPUq/g8rBTx4m7L8obRcBo4vwzt10\n76ygd9ObdE25kPaiWv6/f0SYV7aViO5B9bjoc1ZguPbwr4sWMO2DDqqqZA4iMbZJ+Iuc05wa3vkz\n0bZsQWksRWn3UlbeSizlZKc9CZfLCV6T5yNf5COt/4eBC41dJEkC8VHbAWTr3//lADwM4mMPE9hD\nKL6NfRs3ksZHm3Myre4UmtOFy6sQb9CxSwzUV15js3oJ6bRCQ21m6OS4DBkVY5CEv8gPpxPznHNQ\nIhHU/dcPFBcXM2FAI56wKfZAX4+XN42ryYRKCGWqqGM309gKZNAZO2/OA7VU00+AfgZw0JB+j1Ta\nSwwf8ZAfpTdFe+1cfJkQqdJdDCgu2t8KkSotB03HX5qivHUTamU5tsslOwIx6sbK35c4Ex0YOWQY\nGD4f2o4deL0WgUqdXjvAjNIkb7qnE3k3RHF3nIQRYzeTmcY2DEws8jsiaLg0wAN4yBCklwS9JCgi\nTgnRWBfRrQrGviJ6OlS63qygPphiSsUO9Eo/MSMDAY2KWD+2P0B/6wCJygnoLlXuWCZGhYS/yDtb\n11FsG3PaNJT+fiYFTFI7NPYmqphZoxNxXUCqIwWhFOmUh63dDsrpoph+dFI4OHhl8Fh6ww7tCAYp\nYZAKumhkC+0DlXT8603caOiaQsofJF1ZjbfcQ1fdZHoaakiWQqmnDzNejOesStrbFaqqLKLRQ2Yi\nlR2CyLOx9LckzlCHXjlsBwJots3Z5QZBtw/DUIhNtzH/VU1P9yKMnf9CqzyLUGSQ9l6dicl3aWAn\nVYSwMdGx90/1MHYcHDkEjXQzke6hawpMSIXcDIZK6FLqcVduIxKcxuDEmaTKXFg7ogz8s5+A3yRc\nE6ByZjmKrpHJyC0sRf7lPfw3btzIz3/+cyzLYuHChVx33XX5blKMNUe5cpjKSoLaUIQHgxPZFu/C\nr/cQSU0hUWYSd6dwFIXZ0lGKNeCmz+6mlAhuUpQRwsXBwB1LDowaOlBbEUlKSFJm9zDQ5cXuCjHQ\nlcCwDaqKo3gnTKHTcRaWy4HjmukEJpfhUG00VZdbWIq8ymv4W5bF008/zde//nXKy8t58MEHmTNn\nDvX19flsVoxFx7lyuKJao+28KaR9xVjlOsb2buJulZrEDgZ8VewOeamLbyNp9uEYjNOb6qKWdhxk\ncJNG5fCLuMaaAyeMXcQpYiN1od3EKGGgtwzX3k7qvRvpKj8b3l1D4txJBM+pIj11Gn37Yti1RTg0\nSy4sEzmX1/Dfvn071dXVVO2fE2X+/Pls2LBBwl8cRtOgfqJCpLia0po+4g3lVO1LMNg3jRnpXqLW\nRPzxKkq6t1Oc6iPeWc2mrtkUD/Ti6e+iht0UMZA9JzBW41EHvICTPsroY5AIAxQTi5cwMb6JQdVP\nKtpG7L1yHO6XcE+cQvzij2B7S+l5x6CysgPvvOlonrG8qxPjRV7DPxwOU15env25vLycbdu2HbZO\nS0sLLS0tADQ3NxMcB3dW0nVd6swhXdepqgrunzdtaAZR04R9e0zc770NlsVA+2Qygw34yi1qvAq+\n13ay/T2LeE8Ie/ebOLv2UGT34aMXD5kx2SV0wIHo9hDHR5w0XVhoOKwkRqSfvnQGR6mTUjaRTMTY\ne/H/w9RK2bnDwhfuYtanzsVOsGsKAAAY6klEQVTjPfYubjz8v4+HGmH81Hkq8hr+tn3kh3zlfTcL\naWpqoqmpKftzKBTKZ0k5EQwGpc4cOlad7mKI1Fajb9mKXmZTWRJD8ftImCYl8xqYXb2X1uQ5dKwP\nUPruGwy276VnsJZadlJGX3aU0Fh2oEvIwsRFHyn68Qz0og04MToc9AdCRPZ46fSfjcvnwDmxll1P\n72DBRy2cioGh6IQJYNh6dpRQVdXY/38f7+/Nsab2FKYcz2v4l5eX09vbm/25t7cXv9+fzybFGUTT\nIFjrgKoZQ/fbTVdgx+NQXIyt63DWWTREIlSdW07Peh/pV9+BrTtoD7sZGNxHMVGU/ZPIaRycRnos\nnkI9eKLYJkMKhRQm4AjHsMJRIjUOikoclPVuYaB9ErvKZzO1foBwu4mmdWJMPZuM7aS9XeEMPVAV\nOZbX8J88eTIdHR10d3cTCARYt24dX/ziF/PZpDgTHedksVlTgwZUnzsD+0MNGL/7K7G9MbZt/yCu\njn1oqUESGYtpvEMJCVwMoDM0GmesjnN2QHbKa8gwkW3oHb9kb8dM+rx1eFtbcW1robcxgDGxkXR9\nAy7DJPOBc9F1jXD4yLtxCvF+eX3/a5rG7bffzqOPPoplWVx22WU0NDTks0lRqJxOlPM+iOPsaZS/\n8hqxlzsIv1tGsqMHJZlmR48HX7oTL1FcZNAYoJg4RSRxM/Y+DRzIbhdD847W04mbNEb8HbS4k+7o\nWRiZOMqeHuyGLnZ6GrFa3Wh1VZTM9+H1jWLxYlzI+8HP+eefz/nnn5/vZoQY4vFgXzyfBu9mkhUx\nlL4+jD3dYE0ktX0nuhWjL2Kg90dwmlHSqNTSRgl9eLCOe4/h0aIydDVxNWHSQBI3jcl+wruC9Dlq\nYGeIkqo2evoGsc8+m3d3hTj33zy4fYfMIWSaB6+zkEnmBGP3k68Qp0yJRtEbqphSU01rq4o1YKN2\nduC9Zi7qq+vR2yOEu2qJuzz0RRzs6ejg/OTfKaYPnQTFmGPyZLGDoU8Ejv13QnMyiCcTJ00Jbe1O\nSGzDHDAoP8vB9tQEzmkKosbjQ/cf7ugiEndgWiqaalA20A71tbIDKGAS/uKMoxgGKAoOBzQ2Dt2s\nhZkVKP392Bdcg71zN+5kKVt3ODB2h/FtfZ2NfZ+gcvebFJl9lNBHMf0UEcd1yI5gLHSjH/oHq2BR\nTog4/VQY4OzLYKq9KFotRKN0O+vxOgz62ELMWUFJupfiMgd4PHQG66h0R1Cr5OxwoZLwF2cce/8U\n0oed9VRVrMpK7GAQ6uqoiESonG1A1IGSasJKG7T9fRrpl96gd1cnrTETJW1Qb2+nhF6cGGgkKcIe\nM9cQHLheQCeDg1bUjEF/yKAo3oamOemNXkCmXMcx2I93IEPKV0XC4cI1ewql/f1Ei86hTO5JX7Ak\n/MUZ59CJ5LJ3mjcM7MqhC8gOHT2kArZponV1UXfxROLONKnWStrf7sOOD7IvWkZJOkJxfxeGbaCQ\nYgJ78GCNmZ3A0NQRBlW0UmJGsQZcxBU/5ptvohdlyCQtYp4qnOUpHJUB7HV9hC84n8CWLahVupwD\nKFAS/uLMc5SJ5OzKyqOG24Hppq2qKtT+fkrOrkQpdlFSWsvWzgDVqd1YbTuIxqvoMirRwu309dXR\nwA7K6cGJgc7BOxqPVnweuFjMSf/Q5xPbQE8aGEkbPwnKUt0MJv3EzCm4Ah5KXn+J9Ly5kMmg9vai\n7N2LWVc39MlIdgIFQcJfnJmOc23Aod4/3TSlpZQWt+H5SDXevYNs/1c5g3WTcRWrTK0p4e12N2/t\ngIve/iX93VspHeigiAhOklhY6EOTOePEPuzCsnzTGbr3sAq4SKBhMoiNAwMNSJjFFA+GKd7zTxKx\nKhL1k8jsjrDtuU04JlQR0GP4Wtejlnoxpk/HrqqSncAZTsJfFLb3f0pwuzHOOw81GqViuovgOTq2\nfyZoGn5/kNTGMBU9FoOv3Exq7W/pjUykX7Vw9XdDdw9KJsUgbmIU0chWAvRl7wec7yGkB4apWtjo\npCjBwEDFwIWXfjK2g7ThRunuQckodCiTcGeiqB2DqGUGeBL4J6Rx9K/HOPdcrIkTZQdwBpPwF+Io\nnxKO9qlB06C+3qK4WMGoOQfnBS60d97B2R8mGZtCZE+CRGsvKbWIPVYDe8M9fCD0EkF24yWOgo2C\niWv/DWny8YnA4tCdjImFhQMTGx3H/k8lOg52DpZjdveRVFQ8JQrdITe9xU5KBjUCzn4C6TdRUyms\nadNkB3CGkvAXYhg0DYJBG1CguhFrUikDO3rQwv1UTepmX/RcXDv3MiVjE95XwsbAJziLHfRafipC\nWymOd1Fi9FBONx4S2aDWOHjeQN3//an8cSr7tzUU82CgoqOgYmGjYaOQwI031UdPpJT+QSf1vgj9\ngeloqhvnllZiAY3ekEqADpROEyrKKa1wyP0EzjAS/kKcKk1Dra2ipHb/eMl0mmnvbGFfQ5DU9l4q\npihURfuxPB8h2N0P/V5i7SH6esP09e/DP7ALJyYaJkNxraBjoJLGTQp9/4gilaFQNzn+H6y5fz17\n/5cF6Kgo2KRwo2GgAAoqSXSc8T58ikms3cBZ1IfV1k/cBXHFi8uM4mhvJXDp2ZjRKroHz6I6KReG\nnUkk/IXIFacTZfY5TJgYwZpfQ1/cQcLpI7UvhHPLuyiDLuzZdfSmSnlvt4G2ayeTu17DlYigDcSJ\nGh5SePAQJ0AvBjY+BikiipMMKhbG/qYOzAKqMBTyJgcnhDP2L0vgxsCJhombNAncDOAliZMAIfbS\nSNgOoKoWgfYuikt0Mv7JaH1hlMwA3R4f2rvdFLf1UtzTQ/ysCRQPJugNTDlsCmnZF4xPEv5C5NL+\n8wcK4N//xaRqzAsr6NrSjxODGl0nUOynbe8Cdq69iJK9mynu72AglKZ1IICiQHXvJoJmDz2mRkVm\nDy6SuMiQRKWY1P5RRSY6aXTSqCikUPYPPLVI4aKHIF6SFDFAHC+g7O8WSjNACZbqwLSdtJY24mEL\n3qI0rngbZsqATBrFVOlttUmXaSiJKH5jL72Zs2CwA6prydia3Gh+HJPwF+I00JwaVecEiEQUDEPB\nrdvMma9gzplJx18dRLrOQQnFcbVZFLnBjlfSHYmRiSUZUOPYoT7cfe00pPYQtxXc6iDJJBi2Qhod\nB2lKiKNikKaIvUygVE2RtHpRsFHR0UiRwIMDCxs3yWIfdkkJZ1UMEh5ooFTtwJfpB9uBmbQodvSj\nxwbIVE4lHVOI7YG6mgS2Q0frj2AGgug6RCLK/vMgYjyR8BfiNDl4svhgUGoejUnzgpyVyYASJJmE\njRt1CFeQTtk4KsqJRS2mKluo6tuBo70O+vro61NI9g4ST7sYTGkUZeL0mQYYFgmHH1UziSYG0ZMm\nabsUl5XGsBQGtWIsE1Rdwe2yKZtRRCIBNd5+zN4MyeoGrOggeiaMwxjELvKhGSkUpxPLVolbxRQr\nCoo51AGlKGAYymG/kxgfJPyFGGWHXmjmdivMnZMh2uPEMBS0ogy+MnD0+sA4Fyv4UZRolJreXsye\nMB3v9BGJVRKJQakRQk2lSDkCFCfbcGTiQIBEdwIj3k/SUYKTNFgKqaSNUlqEw6XQ0KiQbrfI1NSi\nVpYT39WLiUWiKIhfiZMxkjiqa4gHJlGsuMC2sfWhyS1sGxwOCf7xSMJfiNH2vgvNVLcD37lD8xAd\nWGa53VBUNHQlcjCIGQzCNJuK+Sr73nYS7bCJaCql3gwTiw2628PoXbspopuq1m10hsopMVKkUiqZ\nhM1AcSXumgqsQBlquUG5D6Jl9SioGCVlDLYH8cf2gh4kXV5PNHgWFQGLTHEJdsbALK88MGUSlZUS\n/uORhL8QY8ExpqPILquoQG1vHzrUPmSyOrW2lppzHSiVanYeO9NXhl3RR1FgOsHet3C8Okjl7gjd\nsQrIqLg1DaNuMgQCGMWlJAMWZk0Z9bSh9/dhqg66g2V07S3CchcxWN9IkZLA1eBhQoODPjUAtoZD\nt6mslNE+45WEvxDjwXEmq3M6bSorLaJRBdMcuo9BZaWF0+3AbJiNVV2N1bIeR3sCtcRHqqqeIlsl\n7KjC1F0MTPIxfboJHQZmkQc1EqGi1kavqqCtfBZep5PycotgcCjoh+4AYI3yCyJGKm/h/9xzz/HX\nv/6V0tJSAG655Ra5naMQI3GMTwd+v017u4Lfb6MoNn4/dHcPLUfTsOvqcNyyiPCrUTBMFIdO2utH\nszVqKi3cbhvNqWHV16NEItgVFdi6js/vx6cduF5YnGnyeuR/zTXXsGjRonw2IUTB0zSorbWyw0id\nTo4Ye685NSbP87Nly9A9yXTdJlBqYdv7dxL7N3QyM6GKM4N0+whxBjh0GGkwCKHQkes4nXDOOWZ2\nJyFX6BY2xbbtvOzon3vuOdauXYvH46GxsZFbb70Vr9d7xHotLS20tLQA0NzcTDqdzkc5OaXrOoZh\nnHjFUSZ15pbUmTvjoUYYP3U6nc4Tr/Q+Iwr/JUuW0NfXd8Tym2++malTp2b7+1esWEEkEuHuu+8+\n4Tbb29tPtZzTJhgMEjraodUYI3XmltSZO+OhRhg/ddbW1g77OSPq9nn44YdPar2FCxfyxBNPjKQp\nIYQQOZS3mwtFIpHs9+vXr6ehoSFfTQkhhBimvJ3w/Z//+R92796NoihUVFRw55135qspIYQQw5S3\n8P/CF76Qr00LIYQYoXzfU1oIIcQYJOEvhBAFSMJfCCEKkIS/EEIUIAl/IYQoQBL+QghRgCT8hRCi\nAEn4CyFEAZLwF0KIAiThL4QQBUjCXwghCpCEvxBCFCAJfyGEKEAS/kIIUYAk/IUQogBJ+AshRAGS\n8BdCiAIk4S+EEAVoRLdxfOWVV1i5ciVtbW089thjTJ48OfvYqlWrWLNmDaqqcttttzF79uwRFyuE\nECI3RnTk39DQwP3338+MGTMOW75v3z7WrVvHd77zHR566CGefvppLMsaUaFCCCFyZ0ThX19fT21t\n7RHLN2zYwPz583E4HFRWVlJdXc327dtH0pQQQogcGlG3z7GEw2GmTp2a/TkQCBAOh4+6bktLCy0t\nLQA0NzcTDAbzUVJO6boudeaQ1Jlb46HO8VAjjJ86T8UJw3/JkiX09fUdsfzmm29m7ty5R32Obdsn\nXUBTUxNNTU3Zn0Oh0Ek/d7QEg0GpM4ekztwaD3WOhxph/NR5tB6YEzlh+D/88MPD3mh5eTm9vb3Z\nn8PhMIFAYNjbEUIIkR95Geo5Z84c1q1bRyaTobu7m46ODqZMmZKPpoQQQpyCEfX5r1+/nmeeeYZo\nNEpzczOTJk3ioYceoqGhgQ996EPcd999qKrKZz/7WVRVLikQQoixYkThf+GFF3LhhRce9bHrr7+e\n66+/fiSbF0IIkSdyOC6EEAVIwl8IIQqQhL8QQhQgCX8hhChAEv5CCFGAJPyFEKIASfgLIUQBkvAX\nQogCJOEvhBAFSMJfCCEKkIS/EEIUIAl/IYQoQBL+QghRgCT8hRCiAEn4CyFEAZLwF0KIAiThL4QQ\nBUjCXwghCtCIbuP4yiuvsHLlStra2njssceYPHkyAN3d3SxevJja2loApk6dyp133jnyaoUQQuTE\niMK/oaGB+++/n5/85CdHPFZdXc2TTz45ks0LIYTIkxGFf319fa7qEEIIcRqNKPyPp7u7m6985St4\nPB5uvvlmZsyYka+mhBBCDNMJw3/JkiX09fUdsfzmm29m7ty5R32O3+9n+fLllJSUsHPnTp588kmW\nLVtGUVHREeu2tLTQ0tICQHNzM8FgcLi/w2mn67rUmUNSZ26NhzrHQ40wfuo8FScM/4cffnjYG3U4\nHDgcDgAaGxupqqqio6Mje0L4UE1NTTQ1NWV/DoVCw27vdAsGg1JnDkmduTUe6hwPNcL4qfPA4Jrh\nyMtQz2g0imVZAHR1ddHR0UFVVVU+mhJCCHEKRtTnv379ep555hmi0SjNzc1MmjSJhx56iM2bN/Pc\nc8+haRqqqnLHHXfg9XpzVbMQQogRGlH4X3jhhVx44YVHLJ83bx7z5s0byaaFEELkkVzhK4QQBUjC\nXwghCpCEvxBCFCAJfyGEKEAS/kIIUYAk/IUQogBJ+AshRAGS8BdCiAIk4S+EEAVIwl8IIQqQhL8Q\nQhQgCX8hhChAEv5CCFGAJPyFEKIASfgLIUQBkvAXQogCJOEvhBAFSMJfCCEKkIS/EEIUoBHdw/fZ\nZ5/ln//8J7quU1VVxd13301xcTEAq1atYs2aNaiqym233cbs2bNzUrAQQoiRG9GR/7nnnsuyZctY\nunQpNTU1rFq1CoB9+/axbt06vvOd7/DQQw/x9NNPY1lWTgoWQggxciMK/w9+8INomgbAtGnTCIfD\nAGzYsIH58+fjcDiorKykurqa7du3j7xaIYQQOTGibp9DrVmzhvnz5wMQDoeZOnVq9rFAIJDdMbxf\nS0sLLS0tADQ3N1NbW5urkvJK6swtqTO3xkOd46FGGD91DtcJj/yXLFnCl770pSO+NmzYkF3n+eef\nR9M0LrnkEgBs2z7pApqammhubqa5uZkHHnjgFH6F00/qzC2pM7fGQ53joUY4s+s84ZH/ww8/fNzH\nX3rpJf75z3/yyCOPoCgKAOXl5fT29mbXCYfDBAKBYRcnhBAiP0bU579x40ZeeOEFvvrVr+JyubLL\n58yZw7p168hkMnR3d9PR0cGUKVNGXKwQQojc0L75zW9+81Sf/Oijj5JOp3n11Vf5y1/+wu7du7ng\nggvw+XzE43F+/OMf849//IPbb7/9pPvNGhsbT7Wc00rqzC2pM7fGQ53joUY4c+tU7OF00AshhDgj\nyBW+QghRgCT8hRCiAOVsnP9IjJdpIl555RVWrlxJW1sbjz32GJMnTwagu7ubxYsXZ89rTJ06lTvv\nvHPM1Qlj6/U81HPPPcdf//pXSktLAbjllls4//zzR7mqIRs3buTnP/85lmWxcOFCrrvuutEu6aju\nuece3G43qqqiaRrNzc2jXRIAy5cv54033sDn87Fs2TIA4vE4Tz31FD09PVRUVLB48WK8Xu+Yq3Ms\nvi9DoRA//OEP6evrQ1EUmpqauPrqq4f/mtpjwMaNG23DMGzbtu1nn33WfvbZZ23btu3W1lb7/vvv\nt9PptN3V1WV//vOft03THLU6W1tb7ba2Nvsb3/iGvX379uzyrq4u+7777hu1ut7vWHWOtdfzUCtW\nrLBfeOGF0S7jCKZp2p///Oftzs5OO5PJ2Pfff7/d2to62mUd1d1332339/ePdhlH2LRpk71jx47D\n/kaeffZZe9WqVbZt2/aqVauyf/Oj6Wh1jsX3ZTgctnfs2GHbtm0PDg7aX/ziF+3W1tZhv6Zjottn\nvEwTUV9fPy6u9jtWnWPt9RwPtm/fTnV1NVVVVei6zvz58w+7wFGc2MyZM484At2wYQMLFiwAYMGC\nBWPiNT1anWOR3+/PjuzxeDzU1dURDoeH/ZqOiW6fQ53qNBGjrbu7m6985St4PB5uvvlmZsyYMdol\nHWGsv55/+tOfePnll2lsbOTWW28dE3+I4XCY8vLy7M/l5eVs27ZtFCs6vkcffRSAyy+/nKamplGu\n5tj6+/vx+/3AUJhFo9FRrujYxuL78oDu7m527drFlClThv2anrbwX7JkCX19fUcsv/nmm5k7dy4w\nsmkicuVk6nw/v9/P8uXLKSkpYefOnTz55JMsW7aMoqKiMVXnaLyehzpezVdccQU33HADACtWrOCX\nv/wld9999+ku8QhHe80OXMk+1ixZsoRAIEB/fz/f+ta3qK2tZebMmaNd1rg2Vt+XAMlkkmXLlvGZ\nz3zmlLLmtIX/eJkm4kR1Ho3D4cDhcABDF1pUVVXR0dFx2InWXDuVOkd72o2TrXnhwoU88cQTea7m\n5Lz/Nevt7c0eXY01B/4vfT4fc+fOZfv27WM2/H0+H5FIBL/fTyQSyZ5QHWvKysqy34+l96VhGCxb\ntoxLLrmEiy66CBj+azom+vzH+zQR0Wg0e7+Crq4uOjo6qKqqGuWqjjSWX89IJJL9fv369TQ0NIxi\nNQdNnjyZjo4Ouru7MQyDdevWMWfOnNEu6wjJZJJEIpH9/u2332bChAmjXNWxzZkzh7Vr1wKwdu3a\nY35aHW1j8X1p2zY/+tGPqKur42Mf+1h2+XBf0zFxhe8XvvAFDMPI9qUdOlTy+eef529/+xuqqvKZ\nz3yG8847b9TqXL9+Pc888wzRaJTi4mImTZrEQw89xKuvvspzzz2HpmmoqsqNN944qgFxrDphbL2e\nh/r+97/P7t27URSFiooK7rzzzjFzhP3GG2/wi1/8AsuyuOyyy7j++utHu6QjdHV1sXTpUgBM0+Ti\niy8eM3V+97vfZfPmzcRiMXw+HzfddBNz587lqaeeIhQKEQwGue+++0a9L/1odW7atGnMvS+3bNnC\nI488woQJE7K9JLfccgtTp04d1ms6JsJfCCHE6TUmun2EEEKcXhL+QghRgCT8hRCiAEn4CyFEAZLw\nF0KIAiThL4QQBUjCXwghCtD/DwO+toNAbPnLAAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "QjIXsU55eq-d" + }, + "source": [ + "## The Model\n", + "\n", + "Consider a data set $\\mathbf{X} = \\{\\mathbf{x}_n\\}$ of $N$ data\n", + "points, where each data point is $D$-dimensional, $\\mathbf{x}_n \\in\n", + "\\mathbb{R}^D$. We aim to represent each $\\mathbf{x}_n$ under a latent\n", + "variable $\\mathbf{z}_n \\in \\mathbb{R}^K$ with lower dimension, $K \u003c\n", + "D$. The set of principal axes $\\mathbf{W}$ relates the latent variables to\n", + "the data.\n", + "\n", + "Specifically, we assume that each latent variable is normally distributed,\n", + "\n", + "$$\n", + "\\begin{equation*}\n", + "\\mathbf{z}_n \\sim N(\\mathbf{0}, \\mathbf{I}).\n", + "\\end{equation*}\n", + "$$\n", + "\n", + "The corresponding data point is generated via a projection,\n", + "\n", + "$$\n", + "\\begin{equation*}\n", + "\\mathbf{x}_n \\mid \\mathbf{z}_n\n", + "\\sim N(\\mathbf{W}\\mathbf{z}_n, \\sigma^2\\mathbf{I}),\n", + "\\end{equation*}\n", + "$$\n", + "\n", + "where the matrix $\\mathbf{W}\\in\\mathbb{R}^{D\\times K}$ are known as\n", + "the principal axes. In probabilistic PCA, we are typically interested in\n", + "estimating the principal axes $\\mathbf{W}$ and the noise term\n", + "$\\sigma^2$.\n", + "\n", + "Probabilistic PCA generalizes classical PCA. Marginalizing out the the\n", + "latent variable, the distribution of each data point is\n", + "\n", + "$$\n", + "\\begin{equation*}\n", + "\\mathbf{x}_n \\sim N(\\mathbf{0}, \\mathbf{W}\\mathbf{W}^\\top + \\sigma^2\\mathbf{I}).\n", + "\\end{equation*}\n", + "$$\n", + "\n", + "Classical PCA is the specific case of probabilistic PCA when the\n", + "covariance of the noise becomes infinitesimally small, $\\sigma^2 \\to 0$.\n", + "\n", + "We set up our model below. In our analysis, we assume $\\sigma$ is known, and\n", + "instead of point estimating $\\mathbf{W}$ as a model parameter, we\n", + "place a prior over it in order to infer a distribution over principal\n", + "axes. We'll express the model as a TFP JointDistribution, specifically, we'll\n", + "use [JointDistributionCoroutineAutoBatched](https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/JointDistributionCoroutineAutoBatched)." ] - }, - "metadata": { - "tags": [] - }, - "output_type": "display_data" - } - ], - "source": [ - "print(\"MAP-estimated axes:\")\n", - "print(w)\n", - "\n", - "_, _, x_generated = model.sample(value=(w, z, None))\n", - "\n", - "plt.scatter(x_train[0, :], x_train[1, :], color='blue', alpha=0.1, label='Actual data')\n", - "plt.scatter(x_generated[0, :], x_generated[1, :], color='red', alpha=0.1, label='Simulated data (MAP)')\n", - "plt.legend()\n", - "plt.axis([-20, 20, -20, 20])\n", - "plt.show()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "YdtvGLvmg341" - }, - "source": [ - "## Variational Inference\n", - "\n", - "MAP can be used to find the mode (or one of the modes) of the posterior distribution, but does not provide any other insights about it. We next use variational inference, where the posterior distribtion $p(\\mathbf{W}, \\mathbf{Z} \\mid \\mathbf{X})$ is approximated using a variational distribution $q(\\mathbf{W}, \\mathbf{Z})$ parametrised by $\\boldsymbol{\\lambda}$. The aim is to find the variational parameters $\\boldsymbol{\\lambda}$ that _minimize_ the KL divergence between q and the posterior, $\\mathrm{KL}(q(\\mathbf{W}, \\mathbf{Z}) \\mid\\mid p(\\mathbf{W}, \\mathbf{Z} \\mid \\mathbf{X}))$, or equivalently, that _maximize_ the evidence lower bound, $\\mathbb{E}_{q(\\mathbf{W},\\mathbf{Z};\\boldsymbol{\\lambda})}\\left[ \\log p(\\mathbf{W},\\mathbf{Z},\\mathbf{X}) - \\log q(\\mathbf{W},\\mathbf{Z}; \\boldsymbol{\\lambda}) \\right]$.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": {}, - "colab_type": "code", - "id": "BfXGOtI9g7bG" - }, - "outputs": [], - "source": [ - "qw_mean = tf.Variable(tf.ones([data_dim, latent_dim]))\n", - "qz_mean = tf.Variable(tf.ones([latent_dim, num_datapoints]))\n", - "qw_stddv = tfp.util.TransformedVariable(1e-4 * tf.ones([data_dim, latent_dim]),\n", - " bijector=tfb.Softplus())\n", - "qz_stddv = tfp.util.TransformedVariable(\n", - " 1e-4 * tf.ones([latent_dim, num_datapoints]),\n", - " bijector=tfb.Softplus())\n", - "def factored_normal_variational_model():\n", - " qw = yield tfd.Normal(loc=qw_mean, scale=qw_stddv, name=\"qw\")\n", - " qz = yield tfd.Normal(loc=qz_mean, scale=qz_stddv, name=\"qz\")\n", - "\n", - "surrogate_posterior = tfd.JointDistributionCoroutineAutoBatched(\n", - " factored_normal_variational_model)\n", - "\n", - "losses = tfp.vi.fit_surrogate_posterior(\n", - " target_log_prob_fn,\n", - " surrogate_posterior=surrogate_posterior,\n", - " optimizer=tf.optimizers.Adam(learning_rate=0.05),\n", - " num_steps=200)" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": { - "height": 405 }, - "colab_type": "code", - "id": "iTmklbFhdHiV", - "outputId": "7597bd72-44a9-4f4a-e52e-f35fdcd62e92" - }, - "outputs": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Inferred axes:\n", - "\n", - "Standard Deviation:\n", - "\n" - ] + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "Gwc6maZifKnb" + }, + "outputs": [], + "source": [ + "def probabilistic_pca(data_dim, latent_dim, num_datapoints, stddv_datapoints):\n", + " w = yield tfd.Normal(loc=tf.zeros([data_dim, latent_dim]),\n", + " scale=2.0 * tf.ones([data_dim, latent_dim]),\n", + " name=\"w\")\n", + " z = yield tfd.Normal(loc=tf.zeros([latent_dim, num_datapoints]),\n", + " scale=tf.ones([latent_dim, num_datapoints]),\n", + " name=\"z\")\n", + " x = yield tfd.Normal(loc=tf.matmul(w, z),\n", + " scale=stddv_datapoints,\n", + " name=\"x\")" + ] }, { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAD8CAYAAACLrvgBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X1clXWe//HX9zoHFTiA50YkTEsF\nMhHkBCaSiRm1M9nOtFbu3LSzUW6WWw3ym6kmZ9t2ZzNnGqVFYNt1nKZp3J0aV2xumm2GCJyRTFAx\nbzJDnJkcUYRDCoIe4Hx/fxw7owkKCFwH+DwfDx96rnNdh/e5OJ73+V53R2mtNUIIIUQPGGYHEEII\nMXRIaQghhOgxKQ0hhBA9JqUhhBCix6Q0hBBC9JiUhhBCiB6T0hBCCNFjUhpCCCF6TEpDCCFEj0lp\nCCGE6DGr2QEGwtGjR/u8rMvloqGhoR/T9A/J1TvBmguCN5vk6p1gzQV9yxYbG9uj+WSkIYQQosek\nNIQQQvSYlIYQQogek9IQQgjRY1IaQggheuyyR08VFRWxc+dOoqKiWL169QX3/fznP+cnP/kJP/jB\nD4iMjASguLiY0tJSDMMgOzublJQUAGprayksLMTr9eJ2u8nOzkYpRXt7OwUFBdTW1hIREUFOTg7R\n0dEAlJWVsWnTJgAWLVrE/Pnz+/O5CyGE6KXLjjTmz5/P008/fdH0hoYG9uzZg8vlCkw7cuQIFRUV\nrFmzhhUrVrB+/Xp8Ph8A69atY+nSpeTn53Ps2DGqq6sBKC0tJTw8nLVr17Jw4UI2bNgAQEtLCxs3\nbmTlypWsXLmSjRs30tLS0i9PWgghRN9ctjSmT5+OzWa7aPorr7zCV7/6VZRSgWmVlZVkZGQQEhJC\ndHQ0MTEx1NTU0NTURFtbGwkJCSilmDdvHpWVlQBUVVUFRhDp6ens3bsXrTXV1dUkJydjs9mw2Wwk\nJycHimYg6NPN+H7xU9oPfThgP0MIIYa6Pu3TqKqqwuFwcO21114w3ePx4HQ6A7cdDgcej+ei6U6n\nE4/Hc9EyFouFsLAwmpubu32sAWNY0L/4H85WbR24nyGEEENcr88IP3v2LJs2beLb3/72Rfdprbtc\nprvp3d13/uilJ9NLSkooKSkBYNWqVRdsMuuNxklT6Di4F9ffPtCn5QeS1Wrt8/MaSJKr94I1m+Tq\nnWDNBQObrdelcfz4cerr6/nmN78JQGNjI08++STPP/88TqeTxsbGwLwejweHw3HR9MbGRhwOB0Dg\nPqfTSWdnJ62trdhsNhwOB/v377/gsaZPn95lpqysLLKysgK3+3pqv++aeDqrfs+J+nqUEVwHlgXr\nJQskV+8FazbJ1TvBmguC7DIikyZN4gc/+AGFhYUUFhbidDr57ne/y9ixY0lLS6OiooL29nbq6+up\nq6sjLi4Ou91OaGgoBw8eRGvNli1bSEtLAyA1NZWysjIAtm3bRmJiIkopUlJS2L17Ny0tLbS0tLB7\n9+7AkVgDZuo0dGsL1B0Z2J8jhBBD1GVHGi+++CL79++nubmZhx9+mMWLF7NgwYIu5504cSJz5swh\nNzcXwzB48MEHMc59Yl+yZAlFRUV4vV5SUlJwu90ALFiwgIKCAh577DFsNhs5OTkA2Gw27r77br71\nrW8BcM8993S5Q74/qanT0IA+9AFqwqQB/VlCCDEUKX2pHQ5DVF+vcqu1Rn/j7yEpFeP+r/dzqisT\nrENhydV7wZpNcvVOsOaCINs8NZwppRh13Qx0zQGzowghRFCS0viMkOtnwvE/o082mR1FCCGCjpTG\nZ4xKTgVAH3jf5CRCCBF8pDQ+w3ptPISFg5SGEEJcRErjM5TFAglJMtIQQoguSGl0QU1Lhobj6BPH\nzI4ihBBBRUqjC2paMiD7NYQQ4rOkNLoSOxHGOtF7qsxOIoQQQUVKowtKKVTKjbBvF7rda3YcIYQI\nGlIa3VAzbwTvWTmKSgghziOl0Z3rkmF0KLr6PbOTCCFE0JDS6IYKCYEZbvTu7ehzX1krhBAjnZTG\nJSj3HDjZBDX7Lz+zEEKMAFIal6BSZsOo0ej3tpgdRQghgoKUxiWo0WNQKbPRO7aiO9rNjiOEEKaT\n0rgMdWMmnG6GfdVmRxFCCNNJaVxOYgrYIvBVvG12EiGEMJ2UxmUoawhqzgLY/Z58x4YQYsST0ugB\nNe+voLMTvbXE7ChCCGEqKY0eUDFXw3VJ6N/9Bu3rNDuOEEKYRkqjh4wFC/2XS5fDb4UQI5iURk+l\npMOkqeg3Nsjht0KIEUtKo4eUYWD8zd9BYz16y1tmxxFCCFNYLzdDUVERO3fuJCoqitWrVwPw6quv\nsmPHDqxWK+PHj2fZsmWEh4cDUFxcTGlpKYZhkJ2dTUpKCgC1tbUUFhbi9Xpxu91kZ2ejlKK9vZ2C\nggJqa2uJiIggJyeH6OhoAMrKyti0aRMAixYtYv78+QOxDnou0e3ft/HL19AZt6LGhJqbRwghBtll\nRxrz58/n6aefvmBacnIyq1ev5vvf/z5XXXUVxcXFABw5coSKigrWrFnDihUrWL9+Pb5zF/tbt24d\nS5cuJT8/n2PHjlFd7T9ZrrS0lPDwcNauXcvChQvZsGEDAC0tLWzcuJGVK1eycuVKNm7cSEtLS78+\n+d5SSvlHG80n0W//wtQsQghhhsuWxvTp07HZbBdMmzlzJhaLBYCEhAQ8Hg8AlZWVZGRkEBISQnR0\nNDExMdTU1NDU1ERbWxsJCQkopZg3bx6VlZUAVFVVBUYQ6enp7N27F6011dXVJCcnY7PZsNlsJCcn\nB4rGTGrqNEiZjX5rE/oTj9lxhBBiUF3xPo3S0tLAJiiPx4PT6Qzc53A48Hg8F013Op2Bojn/PovF\nQlhYGM3Nzd0+VjAw7v576OzA98M8uWy6EGJEuew+jUvZtGkTFouFm2++GQCtdZfzdTe9u/uUUl3O\n2930kpISSkr8J96tWrUKl8t1ydyXYrVaL7+8y0Xrg8tp/o/vErb1N4T/zX19/nn9mssEkqv3gjWb\n5OqdYM0FA5utz6VRVlbGjh07eOaZZwJv5k6nk8bGxsA8Ho8Hh8Nx0fTGxkYcDscFyzidTjo7O2lt\nbcVms+FwONi/f/8FjzV9+vQus2RlZZGVlRW43dDQ0Nenhcvl6tHy2p0BN2TQsuE/ab16CmpyQp9/\nZn/mGmySq/eCNZvk6p1gzQV9yxYbG9uj+fq0eaq6upo33niDJ598ktGjRwemp6WlUVFRQXt7O/X1\n9dTV1REXF4fdbic0NJSDBw+itWbLli2kpaUBkJqaSllZGQDbtm0jMTERpRQpKSns3r2blpYWWlpa\n2L17d2AzWDBQSmF87VGIsuNb9330aXN30gshxGBQ+lLbjoAXX3yR/fv309zcTFRUFIsXL6a4uJiO\njo7ADvL4+HgeeughwL/J6p133sEwDO6//37cbjcAhw4doqioCK/XS0pKCg888ABKKbxeLwUFBRw+\nfBibzUZOTg7jx48H/PtLPj0ya9GiRdxyyy09elJHjx7t29qg9w2ta/bjW/1tuDYeI+dfUeeVaH8K\n1k81kqv3gjWb5OqdYM0FAzvSuGxpDEWDWRoAesdWfP/5PZiRirHsaZT1inYV9VuuwSC5ei9Ys0mu\n3gnWXBCEm6fEhVTqTaivPgJ7qtCv5MsRVUKIYav/PxKPUEbm5/C1nEJv/gn4NGQ/jrKGmB1LCCH6\nlZRGP1J33AuGgd70Y/TpUxgPPyWXGhFCDCuyeaofKaUwPn8P6muPwv7d+Nb8E7r5lNmxhBCi30hp\nDADj5tsxln0LjvwB3/eeRDfWmx1JCCH6hZTGAFEpszFy/gVOfYJv1RPoP//R7EhCCHHFpDQGkEpI\nxPjm8wD4vvcU+qP9l1lCCCGCm5TGAFNXX4vx5HchYiy+vGfQu7ebHUkIIfpMSmMQKNd4jCdXwYRr\n8P3H8+j3K82OJIQQfSKlMUhURBTG8n+Fqyfj+49V6A92mx1JCCF6TUpjEKmwcIyvPwvRV+ErfA59\n6IDZkYQQolekNAaZioj0jzii7Pjy/xVd3/frZAkhxGCT0jCBGuvwH46rFL7ClegzrWZHEkKIHpHS\nMIkaF4Ox9Ak4dgT93/9pdhwhhOgRKQ0Tqetnou64F/3uO+jq98yOI4QQlyWlYTK1cDFMnIzv1UK5\nTpUQIuhJaZhMWUMwHsiB0y3o/37J7DhCCHFJUhpBQF09GfWFL6Orfo+u+r3ZcYQQoltSGkFC/dUi\nuCYO32s/kKOphBBBS0ojSCiLBeOrD8PJJvQvXjM7jhBCdElKI4ioyQmoubeh3/45+ric9CeECD5S\nGkFG3fVVsFjRb2wwO4oQQlxESiPIqEg7KuuL6Mrfof90yOw4QghxAevlZigqKmLnzp1ERUWxevVq\nAFpaWsjLy+PEiROMGzeO5cuXY7PZACguLqa0tBTDMMjOziYlJQWA2tpaCgsL8Xq9uN1usrOzUUrR\n3t5OQUEBtbW1REREkJOTQ3R0NABlZWVs2rQJgEWLFjF//vyBWAdBR/3V36DL3sT3xn9jeeyfzI4j\nhBABlx1pzJ8/n6effvqCaZs3byYpKYn8/HySkpLYvHkzAEeOHKGiooI1a9awYsUK1q9fj8/nA2Dd\nunUsXbqU/Px8jh07RnV1NQClpaWEh4ezdu1aFi5cyIYN/s0yLS0tbNy4kZUrV7Jy5Uo2btxIS0tL\nvz75YKXCwlG3fRHer5TRhhAiqFy2NKZPnx4YRXyqsrKSzMxMADIzM6msrAxMz8jIICQkhOjoaGJi\nYqipqaGpqYm2tjYSEhJQSjFv3rzAMlVVVYERRHp6Onv37kVrTXV1NcnJydhsNmw2G8nJyYGiGQnU\ngjshNBzfL+VIKiFE8Ljs5qmunDx5ErvdDoDdbufUKf/lLzweD/Hx8YH5HA4HHo8Hi8WC0+kMTHc6\nnXg8nsAyn95nsVgICwujubn5gunnP1ZXSkpKKCkpAWDVqlW4XK6+PC0ArFbrFS3ff1y03LmY0z97\nmbFnWrBaY4Ik14WCZ31dKFhzQfBmk1y9E6y5YGCz9ak0uqO17tX07u5TSnU5b3fTs7KyyMrKCtxu\naGi4VMxLcrlcV7R8f9Lpt0Dxq3g2/YTox78dNLnOF0zr63zBmguCN5vk6p1gzQV9yxYbG9uj+fp0\n9FRUVBRNTU0ANDU1ERkZCfhHEI2NjYH5PB4PDofjoumNjY04HI6Lluns7KS1tRWbzYbD4bjosT4d\n3YwUKnIsatbN6Ip38LWeNjuOEEL0rTTS0tIoLy8HoLy8nFmzZgWmV1RU0N7eTn19PXV1dcTFxWG3\n2wkNDeXgwYNordmyZQtpaWkApKamUlZWBsC2bdtITExEKUVKSgq7d++mpaWFlpYWdu/eHTgSayRR\nt9wJZ9s4U/ors6MIIcTlN0+9+OKL7N+/n+bmZh5++GEWL17MXXfdRV5eHqWlpbhcLnJzcwGYOHEi\nc+bMITc3F8MwePDBBzEMfy8tWbKEoqIivF4vKSkpuN1uABYsWEBBQQGPPfYYNpuNnJwcAGw2G3ff\nfTff+ta3ALjnnnsu2iE/EqjJ8TA5gdY3/xd943yUIafWCCHMo/SldjgMUUeP9v0SHMG4ndK3rQy9\nfg3G1/8ZNSPV7DgXCMb1BcGbC4I3m+TqnWDNBUG4T0MMLpV2E8ZYBz7ZRCWEMJmUxhCgrCGE3n4X\n7N2Brq8zO44QYgST0hgiQm//IqDQW0vMjiKEGMGkNIYIi3MczLgBXfE2urPT7DhCiBFKSmMIMebe\nBp94YO9Os6MIIUYoKY2hJHkWRETh2/pbs5MIIUYoKY0hRFmtqNmZsKcK3ToyrvgrhAguUhpDjLox\nEzo60DvfNTuKEGIEktIYaq6Ng+ir0Nu3mJ1ECDECSWkMMUop1I3z4MD76E+6vlS8EEIMFCmNIUjd\nOA+0Rlf9zuwoQogRRkpjCFJXTYRJU9DbpTSEEINLSmOIUjdmwuGD6Pq+X5xRCCF6S0pjiFKz5gLI\nDnEhxKCS0hiilGMcJCSi39tyya/TFUKI/iSlMYSpGzPh2BH4+LDZUYQQI4SUxhCmUjPAYkFvLzc7\nihBihJDSGMKULRKmu9Hbf4f2+cyOI4QYAaQ0hjg1OxOaGqBmv9lRhBAjgJTGEKdmzgJrCHrXNrOj\nCCFGACmNIU6NCYPrZ6J3bZOjqIQQA05KYxhQ7nRorIePa82OIoQY5qQ0hgGVMhuUIZuohBADznol\nC//yl7+ktLQUpRQTJ05k2bJleL1e8vLyOHHiBOPGjWP58uXYbDYAiouLKS0txTAMsrOzSUlJAaC2\ntpbCwkK8Xi9ut5vs7GyUUrS3t1NQUEBtbS0RERHk5OQQHR195c96mFERURB/vb80vvhVs+MIIYax\nPo80PB4Pv/71r1m1ahWrV6/G5/NRUVHB5s2bSUpKIj8/n6SkJDZv3gzAkSNHqKioYM2aNaxYsYL1\n69fjO3eY6Lp161i6dCn5+fkcO3aM6upqAEpLSwkPD2ft2rUsXLiQDRs29MNTHp5USjr8+Y9yLSoh\nxIC6os1TPp8Pr9dLZ2cnXq8Xu91OZWUlmZmZAGRmZlJZWQlAZWUlGRkZhISEEB0dTUxMDDU1NTQ1\nNdHW1kZCQgJKKebNmxdYpqqqivnz5wOQnp7O3r17ZWdvN5Q7HQC96z2TkwghhrM+b55yOBz89V//\nNY888gijRo1i5syZzJw5k5MnT2K32wGw2+2cOnUK8I9M4uPjL1je4/FgsVhwOp2B6U6nE4/HE1jm\n0/ssFgthYWE0NzcTGRl5QZaSkhJKSkoAWLVqFS6Xq69PC6vVekXLD5TL5nK5aJwcj9pbheOr/xA8\nuUwSrLkgeLNJrt4J1lwwsNn6XBotLS1UVlZSWFhIWFgYa9asYcuW7q+42t0I4VIjh67uU0pdNC0r\nK4usrKzA7YaGhktFvySXy3VFyw+UnuTyJc1C/+J/OHHoI1SUPWhymSFYc0HwZpNcvROsuaBv2WJj\nY3s0X583T+3Zs4fo6GgiIyOxWq3Mnj2bgwcPEhUVRVNTEwBNTU2BUYHT6aSxsTGwvMfjweFwXDS9\nsbERh8Nx0TKdnZ20trYGdqqLiyl3uv8b/aplE5UQYmD0uTRcLhcfffQRZ8+eRWvNnj17mDBhAmlp\naZSX+y+gV15ezqxZswBIS0ujoqKC9vZ26uvrqaurIy4uDrvdTmhoKAcPHkRrzZYtW0hLSwMgNTWV\nsrIyALZt20ZiYmKXIw1xzoRrYFwMete7ZicRQgxTfd48FR8fT3p6Ok8++SQWi4Vrr72WrKwszpw5\nQ15eHqWlpbhcLnJzcwGYOHEic+bMITc3F8MwePDBBzEMf2ctWbKEoqIivF4vKSkpuN1uABYsWEBB\nQQGPPfYYNpuNnJycfnjKw5dSCuVOR7/9S3TraVRYuNmRhBDDjNLD8HCko0f7fthpsG6n7GkuXbMf\n33efQi35fxizM4Mm12AL1lwQvNkkV+8Eay4I0n0aIkhNmQaRY0HODhdCDAApjWFGGQYqZTZ67w60\n96zZcYQQw4yUxjCkUjPg7BnYt8vsKEKIYUZKYzhKSILwCPSOrWYnEUIMM1Iaw5CyWv2bqHZvR7e3\nmx1HCDGMSGkMUyr1JjjTBvurzY4ihBhGpDSGq+uTISxcNlEJIfqVlMYwpawhqJmz0bvfQ3fIJioh\nRP+Q0hjGVOpN0HoaDrxvdhQhxDAhpTGcTU+BMaHoHRVmJxFCDBNSGsOYCglBzbwRvWsbuqPD7DhC\niGFASmOYU6k3welmOLjH7ChCiGFASmO4S3TD6DGyiUoI0S+kNIY5NWo0KnmWfxNVZ6fZcYQQQ5yU\nxgigUjOg+SR8tM/sKEKIIU5KYySYkQqjRqO3d/8d7kII0RNSGiOAGj0GdUMGumorut1rdhwhxBAm\npTFCqDm3QNtp2L3d7ChCiCFMSmOkmJYEY5343n3H7CRCiCFMSmOEUIYFlT4f9u5An/rE7DhCiCFK\nSmMEUem3gM8nO8SFEH0mpTGCqAmT4Jo4tGyiEkL0kfVKFj59+jQvvfQSH3/8MUopHnnkEWJjY8nL\ny+PEiROMGzeO5cuXY7PZACguLqa0tBTDMMjOziYlJQWA2tpaCgsL8Xq9uN1usrOzUUrR3t5OQUEB\ntbW1REREkJOTQ3R09JU/6xFMzbkF/dN16D//ETXhGrPjCCGGmCsaabz88sukpKTw4osv8sILLzBh\nwgQ2b95MUlIS+fn5JCUlsXnzZgCOHDlCRUUFa9asYcWKFaxfvx6fzwfAunXrWLp0Kfn5+Rw7dozq\nav+3zZWWlhIeHs7atWtZuHAhGzZsuMKnK9Ssm8FiQVe8bXYUIcQQ1OfSaG1t5YMPPmDBggUAWK1W\nwsPDqaysJDMzE4DMzEwqKysBqKysJCMjg5CQEKKjo4mJiaGmpoampiba2tpISEhAKcW8efMCy1RV\nVTF//nwA0tPT2bt3L1rrK3m+I56KHAszZ6Mr3pZzNoQQvdbn0qivrycyMpKioiKeeOIJXnrpJc6c\nOcPJkyex2+0A2O12Tp06BYDH48HpdAaWdzgceDyei6Y7nU48Hs9Fy1gsFsLCwmhubu5rZHGOMf/z\n0NKMrpKvghVC9E6f92l0dnZy+PBhHnjgAeLj43n55ZcDm6K60t0I4VIjh67uU0pdNK2kpISSkhIA\nVq1ahcvlulz8blmt1itafqD0Zy49dwGNP12HsfW3OP763qDJ1Z+CNRcEbzbJ1TvBmgsGNlufS8Pp\ndOJ0OomPjwf8m482b95MVFQUTU1N2O12mpqaiIyMDMzf2NgYWN7j8eBwOC6a3tjYiMPhuGAZp9NJ\nZ2cnra2tgZ3q58vKyiIrKytwu6Ghoa9PC5fLdUXLD5T+zuW7+XY6X/sBJ977PWrqtKDJ1V+CNRcE\nbzbJ1TvBmgv6li02NrZH8/V589TYsWNxOp0cPXoUgD179nD11VeTlpZGeXk5AOXl5cyaNQuAtLQ0\nKioqaG9vp76+nrq6OuLi4rDb7YSGhnLw4EG01mzZsoW0tDQAUlNTKSsrA2Dbtm0kJiZ2OdIQvafm\n3gbhEfje/JnZUYQQQ8gVHXL7wAMPkJ+fT0dHB9HR0SxbtgytNXl5eZSWluJyucjNzQVg4sSJzJkz\nh9zcXAzD4MEHH8Qw/J21ZMkSioqK8Hq9pKSk4Ha7AViwYAEFBQU89thj2Gw2cnJyrvDpik+pMaGo\nrC+g39iA/lMtatIUsyMJIYYApYfh4Uifjn76IliHnAORS7e24HtqCWq6G+PhJ4MmV38I1lwQvNkk\nV+8Eay4I0s1TYuhTYTbULQvROyvQdR+bHUcIMQRIaYxwKusLEDIK/eZGs6MIIYYAKY0RTkVEoTI/\nh95ejj52xOw4QoggJ6UhUJ+7G0aPwff6D82OIoQIclIaAhU5FrXwb2FPFXrvTrPjCCGCmJSGAEDd\neidEX4Xv9fXojg6z4wghgpSUhgBAWUMw7n0A6j5Gl/+f2XGEEEFKSkP8xcwb4fqZ6J//N/pUk9lp\nhBBBSEpDBCilML70D9DuxfefL6A7O82OJIQIMlIa4gIqdhLqa/8IB/ei//dHZscRQgQZKQ1xESP9\nFv+Z4r99A1/l78yOI4QIIlIaoktq8QMwdRr6lbXoPx4yO44QIkhIaYguKWsIxsNPgS0S34v/jK6T\ns8WFEFIa4hLUWAfG8n8Fw8C35p/QDcfNjiSEMJmUhrgkNT4WI+dfwHsGX94z6E8aL7+QEGLYktIQ\nl6UmTsZ47Bk42YTve99CnzhmdiQhhEmkNESPqLjrMXK/A6db/MUh+ziEGJGkNESPqSnXYXzzOfB1\n4vveU+g/1ZodSQgxyKQ0RK+oqydjPLEKRo3Ct/rbtH+03+xIQohBJKUhek2Nj8X45vMQFk7TM4+j\nd283O5IQYpBIaYg+Ua7xGE+uwjJhEr7C5/D9phittdmxhBADTEpD9Jka68TxXBG456B/9jL6xwXo\njnazYwkhBpCUhrgiavQYjKVPoO5YjP79b/Hl/TO65ZTZsYQQA8R6pQ/g8/l46qmncDgcPPXUU7S0\ntJCXl8eJEycYN24cy5cvx2azAVBcXExpaSmGYZCdnU1KSgoAtbW1FBYW4vV6cbvdZGdno5Sivb2d\ngoICamtriYiIICcnh+jo6CuNLPqZMgzU39yH76qr0a+sxbfyGxiPPYO66mqzowkh+tkVjzTefPNN\nJkyYELi9efNmkpKSyM/PJykpic2bNwNw5MgRKioqWLNmDStWrGD9+vX4fD4A1q1bx9KlS8nPz+fY\nsWNUV1cDUFpaSnh4OGvXrmXhwoVs2LDhSuOKAWSkz8f4xnNwpg3fyv+H73e/kf0cQgwzV1QajY2N\n7Ny5k1tvvTUwrbKykszMTAAyMzOprKwMTM/IyCAkJITo6GhiYmKoqamhqamJtrY2EhISUEoxb968\nwDJVVVXMnz8fgPT0dPbu3StvQkFOTZ2GsWINXBOH/nEBvvx/QXsazI4lhOgnV1QaP/rRj7jvvvtQ\nSgWmnTx5ErvdDoDdbufUKf/2bY/Hg9PpDMzncDjweDwXTXc6nXg8nouWsVgshIWF0dzcfCWRxSBQ\nznEYud9BfWUpHNyH79lH8W0tkcIXYhjo8z6NHTt2EBUVxZQpU9i3b99l5+/uDeNSbyRd3Xd+QX2q\npKSEkpISAFatWoXL5bpsnu5YrdYrWn6gDMlc9/49HXNv5VTBc7T/KB/re+WE/+0DjEpO6/L3OGi5\nTBas2SRX7wRrLhjYbH0ujQ8//JCqqip27dqF1+ulra2N/Px8oqKiaGpqwm6309TURGRkJOAfQTQ2\n/uUKqR6PB4fDcdH0xsZGHA7HBcs4nU46OztpbW0N7FQ/X1ZWFllZWYHbDQ193xzicrmuaPmBMmRz\nhYxBf/1fUFveov1Xr/PJs1+HqdMw7vwSJLoHrDyCdX1B8GaTXL0TrLmgb9liY2N7NF+fN0995Stf\n4aWXXqKwsJCcnBxmzJjB448/TlpaGuXl5QCUl5cza9YsANLS0qioqKC9vZ36+nrq6uqIi4vDbrcT\nGhrKwYMH0VqzZcsW0tLSAEhNTaWsrAyAbdu2kZiYOOCfUEX/U4aBMf/zGCv/C/XVh6GpAd+/P4vv\n+W/ie/cdtPes2RGFED10xYcPRSNCAAASnElEQVTcftZdd91FXl4epaWluFwucnNzAZg4cSJz5swh\nNzcXwzB48MEHMQx/Zy1ZsoSioiK8Xi8pKSm43W4AFixYQEFBAY899hg2m42cnJz+jisGkQoJQc2/\nAz33NnTF2+i3itE/zEP/9L9Q6begbr4ddfW1ZscUQlyC0sNw7+TRo0f7vGywDjmHYy6tNXy4B73l\nLfSud6GjAyZORs1IRc24AaZMQ1n79rkmWNcXBG82ydU7wZoLBnbzVL+PNIToKaUUTEtGTUtGN59C\nb3sHXb0N/dYm9K83QuRY1O13oWZnosY6L/+AQogBJ6UhgoKKiETd9kW47Yvo1tNwYDe+sl+jN/4I\nvfFHcPVk1Iwb/COQqdNQ1hCzIwsxIklpiKCjwsLhhgwsN2Sg//xH9PtV6H070b/djP6//4UxoTBt\nJkbGApg5C2VYzI4sxIghpSGCmppwDWrCNfD5u9FtrXDgffTenej3K/FVbwPXeNSNmajkNLgmrs/7\nQIQQPSP/w8SQoULDwJ2OcqejOzth17v4tryF/vVG9Juvw6jR/k1XKbPx3bHI7LhCDEtSGmJIUhYL\npM3FkjYX3XwSDu5DH9yLPvA++n/+ixOvr4dr41HTU1Cpc1ETJpkdWYhhQUpDDHkqIgpSM1CpGQDo\nPx1izL6dtFZvR//yNfQvfgpXTUSl3uSfZ8I1cpKoEH0kpSGGHTVpKhE3zObs5+9Ff+JB73oXXbUV\n/avX0L/8KUTZUdcl+Q/3Tb3Jv+NdCNEjUhpiWFNjHahbFsItC9Enm9DvV/p3pn+4B7ZvQb+2HpV2\nE8QnoqZOg/GxMgoR4hKkNMSIoaLsqJtvh5tv95+N/qdD6JJf+M9G31qCBhjrRM25BeWeA9dMkcN5\nhfgMKQ0xIiml/IfoPrgc7fPBsSPomg/Q1e+h/+/cGenhETAtyb8z/foU1LgYs2MLYTopDTHiKcOA\n2Emo2Ekw76/Qpz5Bf7AbPqhG79+N3lHhH4WMi/GflZ40y18mIaPMji7EoJPSEOIzVORY1OxMmJ3p\n34x17M/o/dXo/bvQW99Gv/Om/5yQ62eiktJQiW6Ua7zZsYUYFFIaQlyCUgquuhp11dVw653odq//\nyrzvV6H3VKF3b/ePQsZP8I9CEt2QMAM1eozZ0YUYEFIaQvSCChkFM1JRM1LR+iH/KGTfTv+f372F\nfvsXYLX6j8ZKdPtLZMK1ckSWGDakNIToowtGIVlf8I9CPtqH3rfL/+fTK/RGOfzlkehGTU+BIP1e\naSF6QkpDiH6iQkbBdDdquhvuBe1pQO/fBft2oavfg4q30UrROHUavmkzUTfMgQmT5LBeMaRIaQgx\nQJTDhZp7G8y9De3rhD/UoPftQn34PvpXr/vPTrdYIXYiKm0uKiERYq+RM9RFUJPSEGIQKMMCU65D\nTbkOR/ajnKj9CL1np//8kI/2oYtf9e9Qt1gh5UaMtLn+o7PCI8yOLsQFpDSEMIGKtKNuujVwWzc1\nwse16A/eR28rxbejAj49AXG6G3V9MkyaggqzmZhaCCkNIYKCsjvB7kQlz0Lfcz8cPug/N+SDavT/\nnfu+EPBfoTc5DZU8CyZf579EvBCDSEpDiCCjLBaIux4Vdz184cv+70w/dAD9p0PoD3ajf7MZ/ev/\nhdGhcG0cauo0VMIMuG6GfHe6GHBSGkIEORUWDkmpqKRUWLjYXyL7d6EP7kPXfoh+axP6zZ9BaLj/\na2+vn4lyxcCUBLnUieh3fS6NhoYGCgsL+eSTT1BKkZWVxR133EFLSwt5eXmcOHGCcePGsXz5cmw2\n/3bY4uJiSktLMQyD7OxsUlJSAKitraWwsBCv14vb7SY7OxulFO3t7RQUFFBbW0tERAQ5OTlER0f3\nzzMXYohSYeGQNheVNhcAffYsHNiN3vkuevd2eK/cv1M9NAw1IxXip/sv+z7hWtmcJa5Yn0vDYrHw\nd3/3d0yZMoW2tjaeeuopkpOTKSsrIykpibvuuovNmzezefNm7rvvPo4cOUJFRQVr1qyhqamJ73zn\nO/z7v/87hmGwbt06li5dSnx8PM8//zzV1dW43W5KS0sJDw9n7dq1bN26lQ0bNrB8+fL+fP5CDHlq\n9GiYeSNq5o3+705vPA51f0bvqkDv3QWVvztXIuH+kwynTkNNmgJTpqGssrFB9E6fXzF2ux273Q5A\naGgoEyZMwOPxUFlZybPPPgtAZmYmzz77LPfddx+VlZVkZGQQEhJCdHQ0MTEx1NTUMG7cONra2khI\nSABg3rx5VFZW4na7qaqq4t577wUgPT2dH/7wh2it5ZIMQnRDWSwQHQvRsaiZs/wXXPScQNd84P/y\nqT07oOr3gZEIkxNQE6fApCl0ps8zO74YAvrlY0Z9fT2HDx8mLi6OkydPBsrEbrdz6tQpADweD/Hx\n8YFlHA4HHo8Hi8WC0+kMTHc6nXg8nsAyn95nsVgICwujubmZyMjIC35+SUkJJSUlAKxatQrXFVym\nwWq1XtHyA0Vy9U6w5gITso0bB9dNh4V3o7XGd7KJ9g/34N35Hu01++l4++fQ0UHDD1YTct0MQq6f\nyahEN6NmuIPiwovB+rsM1lwwsNmuuDTOnDnD6tWruf/++wkLC+t2Pq11r6Z3d19Xo4ysrCyysrIC\ntxsaGi4V+ZJcLtcVLT9QJFfvBGsuCJJsUxP9fwCjox2O/onQj/Zxels57T//H1qLfwLWEP8Ve6+Z\nCpFR/u8buTZ+0M8VCYr11YVgzQV9yxYbG9uj+a6oNDo6Oli9ejU333wzs2fPBiAqKoqmpibsdjtN\nTU2BUYHT6aSxsTGwrMfjweFwXDS9sbERh8NxwTJOp5POzk5aW1sDO9WFEP1DWUNg0lRsN8zmzK1f\nQHvPwkf70XvPXb33wG7w+fybtAwjcGa7mpwgZ62PQH0uDa01L730EhMmTODOO+8MTE9LS6O8vJy7\n7rqL8vJyZs2aFZien5/PnXfeSVNTE3V1dcTFxWEYBqGhoRw8eJD4+Hi2bNnC5z73OQBSU1MpKysj\nISGBbdu2kZiYKPszhBhgatRo/xV5E93Ag/4Rf8sp+Pgw+sO9/hMOS3+F7tjsX8BihbEO/9fiJt4A\nU6dBWLj/ccSwo/Sltg9dwoEDB3jmmWeYNGlS4I38y1/+MvHx8eTl5dHQ0IDL5SI3NzcwOti0aRPv\nvPMOhmFw//3343a7ATh06BBFRUV4vV5SUlJ44IEHUErh9XopKCjg8OHD2Gw2cnJyGD/+8t+QdvTo\n0b48JSB4h5ySq3eCNRcEb7be5NIdHfCHj9AH98KZVvTxo/DBbmhr/ctMDpf/a3TDbGB3wfhY1PhY\n/78jolBjQvs912AK1lwwsJun+lwawUxKY/BIrt4L1mxXmkt3dEDth+gjh/3lcfRP/jJpbYGmBujo\nuHCBcTEQczXKFQ3OaJQz2l8o2geWEIgaCyGjcUZH03jyJISMumhLg9baPz+ABrQ+9w/AYg3Mr7WG\ndi94z0JHO3R2QmfHZ/7uBJ/P/xhn29CfeOATj3+ZKLt/H0+71z/qCg0n4qoJNNfXgbLAmFDwdfqf\nY2e7/+9Ro8EWCc0n/cuNGg1nz/rnGz3af0Z/SAgoAwwLylDn/n3en/NvKwUo/9+Kv/z70+esgTGh\nqImTg3efhhBCfEpZrZCQ6L/E+2doXyc0noDjR9GnmqCp0b+560Qd+tABaG2hu0+vJwI/QPnfeEeN\nhtFj/G/wJz3+N/uuWCz++Trawevt+xMzDP/P+oxTfX/ELvXLp/fJCVie/n5/PFK3pDSEEANOGRb/\nyGJcDF3tldStp8FTD00esBjg9aJPfQId7YSPHs3pUyf9n/jPnvF/WveeARSMdfhLJPCg5z59g3/e\nM20wahSEnCubUaP8n+4Ni39fjMWCOvc3Fsu5T/bnymms0z/CsFjhdLO/nKxWCLPBmVbsIVaavO3+\nQjnT5l/eavWPSCxWONsGzc0QEel/vHYvjBrjf35nzz2Xdq9/eZ/PP2I6/++L/q0vHFUERlXnjT5C\nB/67WKQ0hBCmU2HhEDYZrp78l2nn/g53uWgze3NeRNSFt8NsWF0u1Ke5Pns/+Ec5kfa/3D5/H441\nBMKH5pGghtkBhBBCDB1SGkIIIXpMSkMIIUSPSWkIIYToMSkNIYQQPSalIYQQosekNIQQQvSYlIYQ\nQogeG5bXnhJCCDEwZKTxGU899ZTZEbokuXonWHNB8GaTXL0TrLlgYLNJaQghhOgxKQ0hhBA9Znn2\n2WefNTtEsJkyZYrZEbokuXonWHNB8GaTXL0TrLlg4LLJjnAhhBA9JpunhBBC9Jh8n8Y51dXVvPzy\ny/h8Pm699VbuuusuU3I0NDRQWFjIJ598glKKrKws7rjjDl5//XXefvttIiMjAf/3sd9www2Dnu8f\n//EfGTNmDIZhYLFYWLVqFS0tLeTl5XHixAnGjRvH8uXLA98LPxiOHj1KXl5e4HZ9fT2LFy/m9OnT\ng77OioqK2LlzJ1FRUaxevRrgkuunuLiY0tJSDMMgOzublJSUQcv16quvsmPHDqxWK+PHj2fZsmWE\nh4dTX1/P8uXLA1//GR8fz0MPPTQgubrLdqnXu5nrLC8vL/B10q2trYSFhfHCCy8M6jrr7j1i0F5n\nWujOzk796KOP6mPHjun29nb9jW98Q3/88cemZPF4PPrQoUNaa61bW1v1448/rj/++GP92muv6Tfe\neMOUTOdbtmyZPnny5AXTXn31VV1cXKy11rq4uFi/+uqrZkTTWvt/l0uWLNH19fWmrLN9+/bpQ4cO\n6dzc3MC07tbPxx9/rL/xjW9or9erjx8/rh999FHd2dk5aLmqq6t1R0dHIOOnuY4fP37BfAOtq2zd\n/e7MXmfne+WVV/TPfvYzrfXgrrPu3iMG63Umm6eAmpoaYmJiGD9+PFarlYyMDCorK03JYrfbAzuw\nQkNDmTBhAh6Px5QsPVVZWUlmZiYAmZmZpq07gD179hATE8O4ceNM+fnTp0+/aJTV3fqprKwkIyOD\nkJAQoqOjiYmJoaamZtByzZw5E4vFAkBCQoJpr7OusnXH7HX2Ka017777LjfddNOA/OxL6e49YrBe\nZ7J5CvB4PDidzsBtp9PJRx99ZGIiv/r6eg4fPkxcXBwHDhzgrbfeYsuWLUyZMoWvfe1rg7oJ6HzP\nPfccALfddhtZWVmcPHkSu93/tZZ2u51Tp06Zkgtg69atF/xHDoZ11t368Xg8xMfHB+ZzOBymvXGX\nlpaSkZERuF1fX88TTzxBaGgoX/rSl7j++usHPVNXv7tgWWcffPABUVFRXHXVVYFpZqyz898jBut1\nJqWB/1PDZymluphz8Jw5c4bVq1dz//33ExYWxu23384999wDwGuvvcaPf/xjli1bNui5vvOd7+Bw\nODh58iT/9m//FtiGGww6OjrYsWMHX/nKVwCCZp11p6vXnRk2bdqExWLh5ptvBvxvOEVFRURERFBb\nW8sLL7zA6tWrCQsLG7RM3f3ugmWdffbDiRnr7LPvEd3p73Umm6fwjywaGxsDtxsbGwONbYaOjg5W\nr17NzTffzOzZswEYO3YshmFgGAa33norhw4dMiWbw+EAICoqilmzZlFTU0NUVBRNTU0ANDU1BXZe\nDrZdu3YxefJkxo4dCwTPOutu/Xz2defxeALrd7CUlZWxY8cOHn/88cAHpZCQECIiIgD/sf7jx4+n\nrq5uUHN197sLhnXW2dnJ9u3bLxiZDfY66+o9YrBeZ1IawNSpU6mrq6O+vp6Ojg4qKipIS0szJYvW\nmpdeeokJEyZw5513BqZ/+mIA2L59OxMnThz0bGfOnKGtrS3w7/fff59JkyaRlpZGeXk5AOXl5cya\nNWvQs8HFn/6CYZ0B3a6ftLQ0KioqaG9vp76+nrq6OuLi4gYtV3V1NW+88QZPPvkko0ePDkw/deoU\nPp8PgOPHj1NXV8f48eMHLRd0/7sze52Bf79ZbGzsBZu0B3OddfceMVivMzm575ydO3fyyiuv4PP5\nuOWWW1i0aJEpOQ4cOMAzzzzDpEmTAp/8vvzlL7N161b+8Ic/oJRi3LhxPPTQQ4M+Gjp+/Djf//73\nAf+nrblz57Jo0SKam5vJy8ujoaEBl8tFbm7uoO87OHv2LI888ggFBQWBofratWsHfZ29+OKL7N+/\nn+bmZqKioli8eDGzZs3qdv1s2rSJd955B8MwuP/++3G73YOWq7i4mI6OjkCWTw8T3bZtG6+//joW\niwXDMLj33nsH9ENUV9n27dvX7e/OzHW2YMECCgsLiY+P5/bbbw/MO5jrrLv3iPj4+EF5nUlpCCGE\n6DHZPCWEEKLHpDSEEEL0mJSGEEKIHpPSEEII0WNSGkIIIXpMSkMIIUSPSWkIIYToMSkNIYQQPfb/\nAakVbkxFIbkUAAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "2whVRGHPdXNm" + }, + "outputs": [], + "source": [ + "num_datapoints = 5000\n", + "data_dim = 2\n", + "latent_dim = 1\n", + "stddv_datapoints = 0.5\n", + "\n", + "concrete_ppca_model = functools.partial(probabilistic_pca,\n", + " data_dim=data_dim,\n", + " latent_dim=latent_dim,\n", + " num_datapoints=num_datapoints,\n", + " stddv_datapoints=stddv_datapoints)\n", + "\n", + "model = tfd.JointDistributionCoroutineAutoBatched(concrete_ppca_model)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "x2zF-wrTVSK4" + }, + "source": [ + "## The Data" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "n4HYhXlTVZ1_" + }, + "source": [ + "We can use the model to generate data by sampling from the joint prior distribution." ] - }, - "metadata": { - "tags": [] - }, - "output_type": "display_data" - } - ], - "source": [ - "print(\"Inferred axes:\")\n", - "print(qw_mean)\n", - "print(\"Standard Deviation:\")\n", - "print(qw_stddv)\n", - "\n", - "plt.plot(losses)\n", - "plt.show()" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab": { - "height": 269 }, - "colab_type": "code", - "id": "CpSsruVIqAv5", - "outputId": "aa6a557d-68b6-49e3-dc57-0e348d238d81" - }, - "outputs": [ { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt0VPW9///nvswtmUtmMsmEkHAH\nEcV6AS8cLVKiR20PpR61+mu/ttrqstp2gbWtHqs9a1FtrFB72lN61Z7Wc6l6foK/tqv2nNQW2y+2\n4N2CIAHCJffJ3O8ze+/fH4EIAkLIhCTM+7FWFpmZPfvzzmbmtWc+n70/W7Esy0IIIURFUce6ACGE\nEKeehL8QQlQgCX8hhKhAEv5CCFGBJPyFEKICSfgLIUQF0ke6gnA4zPe//31isRiKotDS0sI111xD\nKpXiscceo7+/n7q6OlauXInb7S5HzUIIIUZIGelx/tFolGg0yowZM8hms9x77718+ctf5o9//CNu\nt5vly5ezfv16UqkUn/zkJ8tVtxBCiBEYcbeP3+9nxowZALhcLiZPnkwkEmHz5s0sXrwYgMWLF7N5\n8+aRNiWEEKJMRtztc6i+vj52797NrFmziMfj+P1+YHAHkUgkjvqctrY22traAGhtbS1nOUIIIY6h\nbOGfy+VYs2YNn/70p6mqqjrh57W0tNDS0jJ0u6urq1wljZpgMEg4HB7rMo5L6iwvqbN8JkKNMHHq\nbGxsHPZzynK0T6lUYs2aNVx22WVcdNFFAPh8PqLRKDA4LuD1esvRlBBCiDIYcfhblsUPf/hDJk+e\nzEc+8pGh+xcsWMCGDRsA2LBhAwsXLhxpU0IIIcpkxN0+27dv58UXX2TKlCl8+ctfBuCmm25i+fLl\nPPbYY7zwwgsEg0HuvvvuERcrhBCiPEZ8qGe5SZ9/+Uid5TUWdVqWRS6XwzRNFEU5oec4HA7y+fwo\nVzYyE6FGGF91WpaFqqo4nc4jXgsn0+df1qN9hBDllcvlsNls6PqJv1V1XUfTtFGsauQmQo0w/uos\nlUrkcjlcLteI1yXTOwgxjpmmOazgF6c3XdcxTbMs65LwF2IcO9GuHlE5yvWakPAXQogKJOEvhDiu\n3/72t0yePJn29vbjLvvUU0/R09Nz0m1t3LiRm2+++bjLXXfddbzxxhvvu8xPfvITstnsSddyOpPw\nF0Ic1/r167nwwgt57rnnjrvsM888Q29v7ymo6vh++tOfSvgfg4S/EKcRw4BwWKGnRyUcVjCMka8z\nnU7z8ssvs3r16iPCf+3atSxdupSWlhYefvhhfv3rX/PGG2/w+c9/niuuuIJsNstFF11EJBIB4I03\n3uC6664D4LXXXmPZsmVceeWVLFu27LjfKrLZLJ/73OdoaWnhjjvuIJfLDT127733cvXVV7NkyRJW\nr14NwOOPP05vby/XX3/9UJtHW65SyWEEQpwmDAO6ulSczsEBwWJRoatLobHRZCRHKz7//PNcfvnl\nzJw5k5qaGt566y3mz5/PCy+8wPPPP8+vf/1rXC4X0WgUv9/Pv/3bv/HAAw/wgQ984H3XO2vWLJ59\n9ll0XefFF1/kkUce4Sc/+ckxl//FL36By+Wira2NrVu3ctVVVw099tWvfhW/349hGHz84x9n69at\nfOYzn+HHP/4xzzzzDIFA4JjLzZs37+Q3zgQm4S/EaSIaVdB1OHgwiKKArg/eHwye/Lmc69ev57bb\nbgPgox/9KOvXr2f+/Pn86U9/4uMf//jQMecHZ/E9UYlEghUrVrB7924URaFYLL7v8n/961+59dZb\nAZg3bx5nnnnm0GO/+tWv+I//+A8Mw6C3t5cdO3YcNdRPdLlKIOEvxGmiVFJ471GAijJ4P5xc+Eci\nETZu3Mj27dtRFAXDMFAUha997WtYlnVChx0eemz6oWfLPvrooyxatIjHH3+cffv2DXXNvJ+jtbd3\n715+9KMf8Zvf/IaamhpWrFhxWJfQcJerFNLnL8RpQtct3jtZi2UN3n+yfvOb3/CP//iPbNq0ib/+\n9a+8/PLLTJkyhU2bNrF48WJ++ctfDg2oHpzFt7q6mlQqNbSOpqYm3nzzzaH1HZRMJmloaADg6aef\nPm4tF110EevWrQNg27ZtvP3220PrcblceL1e+vv7+cMf/jD0HLfbPVTL+y1XiST8hThN+P0WpRJD\nOwDLglJp8P6T9dxzz3H11Vcfdt8111zDunXrWLJkCVdeeSVXX301V1xxBT/84Q8BuOGGG7j33nuH\nBnzvvvtuHnzwQT72sY8dNlXC5z73Ob75zW/y0Y9+FOMERqZvvvlm0uk0LS0trF27lnPPPReAs846\ni7PPPpslS5Zw9913HzaD8Cc+8Qk++clPct11173vcpVIJnY7CTIRWXlJnceWyWSGdXEkw4Bk0kYu\nZ6DrFn6/NaLB3tGi6zqlUmmsyziu8Vjn0V4TMrGbEBVO0yAYtCiVyjP/izh9SbePEEJUIAl/IYSo\nQBL+QghRgST8hRCiApVlwHft2rW8+uqr+Hw+1qxZAwwet/v73/8er9cLDF7X9/zzzy9Hc0IIIUao\nLOF/+eWXc9VVV/H973//sPs//OEPs2zZsnI0IYQQoozK0u0zb9483G53OVYlhBhn/uVf/oUlS5bQ\n0tLCFVdcwauvvgrAPffcwzvvvFOWNg6d+fNYvvvd7w57vU899RT333//mLUPcNttt7Fnzx5WrFjB\nk08+edhjzz//PP/n//wfAGbPng3AwMAAn/jEJ06qreEY1T7/3/3ud9xzzz2sXbv2sNO9hRCjxDBQ\nwmHUnh6UcJiRzun88ssv09bWxvPPP09bWxtPPfXU0AlFq1evZs6cOeWo+oR873vfO2Vtlav97du3\nY5omU6dOZfny5UdMif3cc8/x0Y9+9LD7amtrqa+vZ/PmzSOq93hG7SSvK6+8cmiipqeeeopf/OIX\n3HnnnUcs19bWRltbGwCtra0Eg8HRKqlsdF2XOstI6jy23t7eE7+Au2Gg9vWBrg8+x7Kgrw+zsZGT\nPc13YGCA2tpaqqurAaivrx967GMf+xhf//rXOffcc5k+fTq33norL774Ij6fj3/6p39i1apV7N+/\nn1WrVnHVVVfxy1/+kjfeeINvfvObAHzqU5/izjvv5O/+7u9QFAVN09B1nU996lN0dXWRz+f57Gc/\ny80338yqVavI5XJceeWVnHHGGfzgBz/gv//7v/nJT35CsVjk/PPP55FHHkHTNP7rv/6L7373u4RC\nIWbMmIHD4ThiG0YiEe644w4GBgY477zzAIbd/tGWe6/169dz9dVXo+s6S5YsYeXKlQwMDBAKhchk\nMvz5z3/mscceG6rv4L/XXHMN69ev55JLLjlinQ6Hoyyvw1EL/5qamqHfly5dyiOPPHLU5VpaWmhp\naRm6Laf5l4/UWV5jUWc+nz9sPpz3o4TDmICuKIdNSWD192OdZFhceumlrF69mosvvpjLLruMZcuW\nDQWSZVkYhkGpVCKTyXDRRRdx33338ZnPfIZvfvOb/Od//ifvvPMOK1asoKWlBcMwME2TUqmEruuH\nPf/Q31evXo3f7yebzfLhD3+Yq666ivvuu48nnniC//mf/wHg7bffZt26daxfvx6bzcZ9993H008/\nzQc/+EG+9a1v8fzzz+PxeLj++us5++yzj5ii4dFHH2XhwoWsXLmStrY2nnzyyWG1f6zlDl434KBN\nmzaxbNmyofavvvpq1q1bx2c/+1l++9vfcskll+B0OoceP/jv2WefTWtr61Gnlsjn80e8DsfV9A4H\nL+wAgxugubl5tJoSQgBKqcTR5nRWSqWTnNB5cIbO559/nr/+9a9s3LiRz33uc9x33318/OMfP2w5\nu93OkiVLAJg7dy52ux2bzcaZZ57J/v37h9XmE088wW9/+1tgcK6v3bt3HxGqf/7zn3nrrbe45ppr\nAMjlcgSDQV577TUuueQSamtrAVi2bBm7du06oo2//OUv/PSnPwUGP4Ae+mH1RNo/0eV6e3uHagFY\nvnw5q1at4rOf/SzPPffcMaexDgaDI7oO8okoS/h/5zvfYevWrSSTSe644w5uuOEGtmzZQkdHB4qi\nUFdXx+23316OpoQQx2DpOsp7L4hiWVg224jWq2kaixYtYtGiRcydO5dnnnnmiPDXdX1orn1VVXE4\nHEO/H/z0eui8/nD43P4Hbdy4kT/96U/86le/wuVycd111x11OcuyuP7667nvvvsOu//5558/oWsM\nwNGvDfDe9q+//voR1el0Og+7f+HChfT19bFlyxZefvll1q5de9Ta8vk8TqfzhP6Ok1WW8F+xYsUR\n933oQx8qx6qFECfI8vtRuroGL98FQ3M6W4f00w9Xe3s7qqoyY8YMALZs2UJTU9NJrau5uZmf//zn\nmKZJZ2cnr7/++hHLJJNJfD4fLpeL9vb2oSOLAGw2G8ViEZvNxqWXXsott9zCbbfdRjAYJBqNkk6n\nOe+883jwwQeJRCJ4PB5+/etfH/VKXRdffDHPPvssK1as4IUXXiAWix21/VdeeeWo7b9fnYeaPXs2\nu3fvHur5UBSFf/iHf2DFihV86EMfOmbA79q1i7lz5574xj0JMqunEKcLTcNsbMRKJsEwsGy2weAf\nwZzOmUyGr33tayQSCXRdZ9q0aXzrW986qXUtXLiQKVOmsHTpUubOncv8+fOPWObyyy/nySefpKWl\nhRkzZhx2YugnPvEJWlpamD9/Pv/6r//KV77yFW666SYsy0LXdR566CEuuOACvvSlL7Fs2TJCoRDz\n588/6rUCVq5cyV133cXf//3fc/HFFzN58uSjtn/BBRcctf01a9Ycs85DLV26lJdeeokPfvCDQ/ct\nX76cH/zgB/zTP/3TMbfVxo0bWbp06fE36gjIfP4nQQYoy0vqPLbhzucP43MO+veaCDXCyOvMZrNc\nf/31PPfccyc8cA9w7bXX8sQTTxw2FnFQuebzl7l9hBBilLhcLu65555hDd4ODAxw++23HzX4y0m6\nfYQYx8bZF3NxEi6//PJhLV9bW8tVV111zMfL9ZqQT/5CjGOHHi0jRKlUQlXLE9vyyV+IcczpdJLL\n5cjn8yd8CKPD4TjqYYfjyUSoEcZXnZZloapq2Q4BlfAXYhxTFAWXyzWs50yEAfSJUCNMnDpPhnT7\nCCFEBZLwF0KICiThL4QQFUjCXwghKpCEvxBCVCAJfyGEqEAS/kIIUYEk/IUQogJJ+AshRAWS8BdC\niAok4S+EEBVIwl8IISpQWSZ2W7t2La+++io+n481a9YAkEqleOyxx+jv76euro6VK1fidrvL0ZwQ\nQogRKssn/8svv/yI61GuX7+e+fPn893vfpf58+ezfv36cjQlhBCiDMoS/vPmzTviU/3mzZtZvHgx\nAIsXL2bz5s3laEoIIUQZjNp8/vF4HL/fD4Df7yeRSBx1uba2Ntra2gBobW0lGAyOVkllo+u61FlG\nUmd5TYQ6J0KNMHHqPBljfjGXlpYWWlpahm5PhAsnTJQLPEid5SV1ls9EqBEmTp2NjY3Dfs6oHe3j\n8/mIRqMARKNRvF7vaDUlhBBimEYt/BcsWMCGDRsA2LBhAwsXLhytpoQQQgxTWbp9vvOd77B161aS\nySR33HEHN9xwA8uXL+exxx7jhRdeIBgMcvfdd5ejKSGEEGVQlvBfsWLFUe9/8MEHy7F6IYQQZSZn\n+AohRAWS8BdCiAok4S+EEBVIwl8IISqQhL8QQlQgCX8hhKhAEv5CCFGBJPyFEKICSfgLIUQFkvAX\nQogKJOEvhBAVSMJfCCEqkIS/EEJUIAl/IYSoQBL+QghRgST8hRCiAkn4CyFEBZLwF0KIClSWyzi+\nn7vuugun04mqqmiaRmtr62g3KYQQ4jhGPfwBvv71r+P1ek9FU0IIIU6AdPsIIUQFUizLskazgbvu\nugu32w3AFVdcQUtLy2GPt7W10dbWBkBrayuFQmE0yykLXdcplUpjXcZxndI6DQMiESgWwWaDQAA0\n7YSeKtuzvCZCnROhRpg4ddrt9mE/Z9TDPxKJEAgEiMfjfOMb3+CWW25h3rx5x1y+q6trNMspi2Aw\nSDgcHusyjqucdRoGRKMKpZKCrlv4/da72W4YqF1doOugKGBZUCphNjae0A6gErfnaJoIdU6EGmHi\n1NnY2Djs54x6n38gEADA5/OxcOFC2tvb3zf8xfhjGNDVpQ5le7Go0NWl0NhoommgRKODwW+aKPE4\nimFgqSqKpoHdjlIqYek6lt8/tDM4dGdiWWCaJ/xFQQhRBqMa/rlcDsuycLlc5HI53nzzTa677rrR\nbFKMgnCvSXZvFKtoYGka+Wo/0YSNv/1No6nJpFExCPhMbP29oOsYpkJyoIjyxlsU55+Np0YjOVDC\n2NuDNXkSXr9K1z6LQs+BdfYb9OQVgiGVbBaqq8HheM+3CyFEWY1q+MfjcVavXg2AYRhceumlnHvu\nuaPZpCgzo2Aw8Fo3jkIao2Cwv8tGTypPvq6ROnuCeDyLPbcfy5GifpKC5fESjujYU0lwucj3Jfnb\n23UEAhY2xULt2s32tAdHIkxVcwDFptO7t0AyEiMSmUQwpJJKQX29edi3CyFEeY1q+IdCIR599NHR\nbEKMssTOAdyZMLmSjV27NeJxk3ykH6UnSri+jhp3L/2KHWeym3RGR03tpISfomGSmTSVvi0DqMRJ\nayU0SiTtQVKZIjXRPXj2bqcYmoQ+42xyJR2rO4LDBZRKZHMa7ik1RKMqweB7hqUMAyUaPWp3khDi\nxJyS4/zFBNYfocqrs/NvGoWiQjqtouVzFNNgcznpytiYVFeAdI78q90YVR60Gg0zX0R5vR1fFsz6\nEJlInmxeobp2D65EjqK9mnCumkJ/GHffm3Qr0/Hl+0mbLnSlhKlqGPE0A85GIvY4qlGipk7FP82D\nLdw7NLisFIsoXV0nPLgshBgk4S/el65bJDPg9UE8AZoOulICm46Vz6Pkozh62jGrMhQUO1axhKd9\nC4Wchb0/Qtg7hdLuDpyJDI5siewuHcwSpq8O01FNVKkhHrFhT/6FVI2flGbDlYvgS3ThVDIElCqy\n08/A2+Ag0umGLXFqz29GsymDBSoK6DpKNIoVDI7txhJiApHwF++relqA3t092DQVf42GgomRsehX\nvdj7Izj1GKoNbEYeLdGL10igFPPomRI5xYYr0kV30ouZyGCzDOqNXtKmGy2Vxk6B6iJkFDe1Sj9Z\ndxBzh4sGtZ8qK4uSyVDl82J0byPlnoxa5aSvCOE/tZOYNAfT46Wu0UbNDA+BuhIHdgfvf1iqEAKQ\n8BfHoYaCBM7Ioe7LEolbeAMapdAUMtuiWDaDJnsvPi1FMLUbK5ECpYTTo1GMZ6BgkRqw8BciGHkD\nL1EK6PhJYZVKqJRwUqBAFSoFtFgHjlgBFQ1NBSwwY/sx92hoqpeMp46MPYiqazinRMl469jtnUTD\na1kGzmpi5tkJVK+b3qgTxedFSSaIRgy6DRv1Z3gJhtTj7wRkPEFUCAl/8f40Dd85TeTscWY3l9i5\n10HCqMHfE8UX1HAWA9TFelBjRRSnilpU0NJJbGaCgYwdbzZBVcnATYoUDuyAgwIOMqgY2ChhYsfC\nwEkJMAFQTLAY/DHRqDF7ycQ78eOmk2a0WAybv56Zxh8oOJ3YXqrilQuvoHq6H3u1HW9XO2lPPVpd\nEK3KT+emLJ0NIab4otT6iqh2HcvrRUkk3g16rxe1V8YTRGWQ8BfHpdk1QmcFCIcVqtBw9YXRZzqg\npFFKGOStekrFElXpCHqyHyVbIO/woJWSqKUcOgYmBi4yWICTAk4y6HCgqyaLBWgcOdmUCVgYWICN\nJFWk8TBANuvCkTXIoxMnhOmoZvr+7WRqG7BrFl3OZtKuXgKNTqyaAGHHVLxv7aa9tobiLJWG2VXY\ndu7EnDx5cMygWETdtg0rEBgcRwAZTxCnNQl/cUI0bfBn8mQTm6OAGfKQ2N6P3TCJpYMEfVmsXBwm\nTyaf8WEOJKA0QNbuQymk0AEHWezksFNAh0PC/9gO3RnogA0TFwWqKaAceH4tMXJ5G0peoZTYTowA\nk2knZq9n/445aM79TPJsRK+tZXfoQnredtIU6qFurh9fMYlvuh9NUwZrSSQGdwAHKcrgN4Nybkwh\nxgEJf3HCSiVlcOoeTUezWXjPqCPyag6zykepughBOzlVR3n1LdQi5HwN9Bd0vCjYSWEnj50SGgwF\n93Ad3Bkc2gljA5wUsYASBdykKKDQUNhFU2ELGTxoPRrJXQHmON5gl2c+vS43mdR8nNNDOJIadjuU\n+px4bDlqQlH83hKqTcPyerGczhFuOSHGHwl/ccJ03aJYVDB8ftTeLjSbjjJjKp5oN3Z3NcVcHbl9\nMcwPXEzVnndI7R7AHukkjp8geYrYKKGjMjoztyoM7ggAHFiYlHARxSRKEQWf0U8m48af2UMSD6ne\nzew760OYtQ3Ygh6cdV7S8T76GhsJ1KnMnF7EluykdN55o1KvEGNJwl+cML/foqtLQdc1iqFG1FgU\nSzXQZ02moINSKpE2IxS8teRmnUH1n18g2ZujoJgMpCxqyWPhw02CKvKj/uJTgYMT3TqwKJHCTQoD\nyOIkl91P9cthOjznkK5vxm6m2etqwh2Mk22uJd3loHleNbaXd1I9sw78/lGuWIhTR8JfnDBNg8ZG\n88Ax9BpqqJYZsy16e1U4MONnRp2EEY5RM7WeYvICzOppDGzYja0qQiHiwCplCRJmKu3oHN59M5oO\n/VYAYCeHmxxu4tQl91JIVhEjSIPqJqn6yFYH2OVpJn7BdCY1g/ZmH8zqJHLGGfgbne9/8I8cLiom\nAAl/MSyaxoG5dt4dAn13h6AQDClkvQFMB1iBOlT9barTKpnuRvrT06ja/ibdmWrsZo7J7MfJyfX9\nj9TBAWc7Ji6iQJQmOjFNiJke4vF6EvEdJHtr6GyeS9Ffj6n1Yir/H9umL2TKsnl4JlVT3egh3ZXE\nyBtoDg1vs0w/ISYGCX8xYu/dIRiGNbgzUOz4l5yF94wgA398B2XvflKTFjPw8jsU+x1oRoEG+sZs\nB8CBdt97DSQXSWpJYrETswiRXS8Rp5GsWoPPVaS4s5Nd23bjuHYpzt43MIN1pEsuCk4vfrOdKef5\nCDYog1kvh4uKcUrCX5Td4TsDFRonETqnnui2Pvr+sIPudBVxZz37B2ZxWfL/pYnOocM+bYzdjuBQ\nh+4QJpOgngQFUyWZ9qCkw7jCneS2/Yku39nYahw4A26cLgeqx0a40wPz6wme14jmssvhomJckvAX\np4Rm1wieMwlCIToDcQovvcFAR4jn9gRZFHueqeygigzVpIfGAWzvu8ZTy8bgOQbVxKklTpY9pHPV\nTM/9jUKvnTxu8q4Azho72eY5WNZ0BvbsIXbWhei5PDV+C6+io/qPPKv4sNsyPiBOEQl/cUr5gyr+\nM4N0JM7DVdOLqdl5Zd+HSfdvYAbbMVCwU0QnDxz9rN+xZjvw4yFNifSB8wvAzGoMZINU93awd9/5\nVLlNgi/+CaZMIXLG+eRTRSb52jEmNxFP2zDyBZzh16k6oxHNrsv4gDilJPzFKaVpcM45Bhg+4q4c\n1J5L/O3pvB2fgmPvb7Ai23BlwmgU0MnhJoWFiZ0TOyP4VDr0CKLBbiIDB70UTfB37yONBxMX/d0p\narbvJDJlPnl/NcmGAo7pDYNTY7tcZHbEqa1TyMRMSmhYqQG8c+ol/8WokvAXp5zdDuctgOi0OqyB\nGJHzmkh26ezuXknib3/C2b2Txv2vk4yb5LFRSw81xKkhjB0LjfH7wj34rWDw9yQGSdR9fyGxr57G\nnW+zL3QexcYU0QGoKfQTVlz4XWmSTZMITPOiaBbs76LLVUdjo4mekENGxegY9ffQ66+/zs9+9jNM\n02Tp0qUsX758tJsUE4CmQTCkQiiAf06QN97w4JussNM3BefOt3nTv4S6gXdIdqaYnd1KfzHJdN7G\nQQ4PSewYOMb6jzgO29BPEidZzIxK4+40kc49JHb0kmoIUGtPki46iG/P4J4LjjmNqG43jlgvqUQJ\nf50mh4yKUTGq4W+aJo8//jhf+9rXqK2t5b777mPBggU0NTWNZrNigqmrg2DQxG5XyOV0upWzSWsD\neBurMNx9ZPYNEMpm6bfORs8myRX3E6Qfk8KYHiZ6ojTAQ4kS4KAHT2GAQN9euvuaMBUnOc8knL4k\nuyIBGrtjeC47k6rXNtEz5XxKioZGiRolhm6WIJvFnDlTdgBixEY1/Nvb22loaCAUCgGwaNEiNm/e\nLOEvDqNp0NQ0eKJYba3FtGnQNydAMuan/tIIxb551G34ProNHKUaSvsdRPa78BUGsEihUhyaLG68\ndgfBuyeWOSlSRS9BehmwatATUWwJHW3AQbI/RLIvg0cvUNWwl/S552C5XITVKuoaVGp8OVT5BiDK\nYFTfK5FIhNra2qHbtbW17Nix47Bl2traaGtrA6C1tZXgBDgRRtd1qbOMdF0nFApy4DMCMHgpxv37\nQdf9mCYU6q9Hf3sLNcoAzgtmENkTJ7ytD9/e17CnUhQpYlDEe+A6Ae89cWu8OdhlZSdGPTHyQDJT\nQ02mk0xsH5kps6nWC5Q25ch7G4jNvJBkyomn1sa50wM4VRWO8X87Ef7fJ0KNMHHqPBmjGv6WdeRp\nLYpy+Jf0lpYWWlpahm6Hw+HRLKksgsGg1FlGx6rT6Xz3WryOMxrxOnqxnI3ko1Gq3Q6qAwqdl3yC\n3KZ20hkDR08Pvel+GunETQob4/ubALz7bcAGuIhRJIY9H8O5K0YiMZWMZxL9ikn69T/QO/MinHOa\n2ZtK8cHFRXRFOer1ikOh8f//PtFfm+NNY2PjsJ8zqu+N2tpaBgYGhm4PDAzgl5kRxQk67Ezhunos\n1xmYPT2gqmCaMGUK9VOmUPh//p5I21ukXtlFfscetvXPIJTaSohunAcuHDPeO0jUAz+DA8QZ4kYv\nem8GozdGlTMBTj8+xUXOzJDrMOhLZ2i8pJ9eowGlLoiiaxSLCl1dyrG+EAhxmFEN/5kzZ9Ld3U1f\nXx+BQICNGzfyxS9+cTSbFKcrTcOcNg3L58MslbAUBSWXA7sdm6IQWr6Auouayb34KsldETpf8RAO\nD+DJ9lBDNz4iQ1NH2BjfOwOHx6vZAAAXnklEQVQbUEuOPDl8hEnm9hDO1eKO7cHa7kL1uenpnEOp\nL05xag4aczjnNKHZNXQdIpF3r0QpxLGMavhrmsatt97KQw89hGmaLFmyhObm5tFsUpzONA0rGBya\nI8c6dOpkpxPOP5fqKidV/WFSjQP0v5OkI2xRnegi0PEmNWYfdrJ4iVBDCjvjdyegAIPXD7NwEKeG\nOJPZT6xYRywcIl3KY0Y6SDTHMM4+h93bPNTMqsVvDeCbk6PalzjyvACZalocYtS7RM8//3zOP//8\n0W5GVKL37AwAzGAQLZdj+iVgeAMEEilSA17iSz/Ijt0ZmhNvMWnLiyTSCVwkqSFMFcVxPTagD/0U\ncNKJnzC52B76Yg3YOgboeLuAt2kHuS3NJKYG2b4dPjC3H3uVRmnhQnC5wDBQu7pkqmkxZDy/5oUY\nNisYxMxm0ex25szqoSviQ5vhRvU0MPVDRXoTHyDeVofy5laC6X10RhupTvdRTT8ucoedoTveHNwJ\nOMjjIo+LBJl8J3RmeSd1EfreKJ7dOWrObGaH2cC8s4vYNmygdM45WNEY8RgUqv1odgWfDzSZarqi\nSfiL04umYTY3D3ZvBAI0ptPgdmPZbVj+epo0DeOya8n+QqWnfQ5qRwepaBwrniAXzjKJDmoYwE0G\nB+NvUjkYrMkB6BTRiTGdLfjjAyhxnWS+GctjofXtI5yppdpIEOkwiLubcJfi1BjvYAQChD21BM+o\nQVNkqulKJeEvTj9H6Q467GGfG/dn/5Ez3noLLTUNs1DAqPbQscNgz5/bifz1D3hj+/ERRsGiigR2\njHF36KgGuACNOA5KaGjURAbIvNZD2BEi12XHaAxARxrT1knSpjNQVUNdU4yq+gIZK4P7nClj/WeI\nMTKeXstCnDouF8aFF2IcvG0YNM8M43MWSVz0aXb+cR+Bva/jjIWpKsZQ0wmqiFJDBAfjp2tI5eDA\ncBoTBY0sWjaNMztALuMjG08TNoOkqhXcHgVvQ5JoPkBeryKY60JtcGJqmgz+ViAJfyEANA21MYT3\ncmDnADWuEB2vz6Y5tZ10OEx1uItYJE0usZtqo5dqktgoAu+epDWWnICBhU4RO0USqKQLdozeCDaX\nTigbIZ/2kDf8uDNRHEaY/NwpKIYhg78VSsJfiEOooSABo4Bvuk7dOXVsf2s69o6d2OO7mFwKE9/l\nJtXZhC/dRT6n4C92YWBRfaBr6OAcQ2OxM9AO/OQAOzlsZLFToCqbwVDsmKjovVlK+WoMpwf2p0j8\nz37UGRBwZvBE4yiNDe9+C5BDQ09rEv5CHErTMBsbUaNRGhpKhJrsWP5FYFyEfyBCdsM2rL19dPYb\nGD0RIuFuGnvepGAqZKJZNKOAToJaYtjID30rOJUDx4NtFvGRJIeDKlIoloo9lSNX9GFlwmRyJZIR\ni+I5U3C8vJfSFB/FeC9+Xw1atgszFELt7ZVDQ09jEv5CvNfRBow1DW3+2fjrGwZnH82bODq2Y+7v\nRQufTaEvRvGdfnp6SiiWyZakyeyel3CTxEkKyFF9yFQToxmfFqBh4iADqKgUMbFRRRQrb1HAQbBz\nC/bkAMlCCquujo6ch/2+KrzFFMF6i6bd+1BqfVg1NYNhryggh4aeViT8hRiGd+cbUlC1GoxpNcSj\ns1EiCTyLspRe30+0UA1FB7/ecx3+ri3MsDrQIr2U8gae2B6a6cBFGteBqajL/SZUhn4soIiBhgU4\nyWMQx4EDO3mshEWiw4OWTWNPqPTPuhBray9mrpZ4b5TmMwtoWpjS9GnU2NPoloGlaRjS/XNakPAX\n4iRZuo5mFQkEVQjWADV459Shbs+Ri/s4d06SnH4lGbuNaEShlCuSf2sX2995jWl9r+CK7WMqu3GQ\nw8Hg8fYq716c5uD4wcEjklTA5PhvWuvAcwevfWyQxnHgORY1pMhRII8bjSJ1+S529HixPCZaRwfh\nugYKf+vFU8yyJ1Vg9uQktl3biU49g8BML6pTlesJnCYk/IU4SZbfj3LIlAlYFjbdYtoV02lGIxqt\np1BQSKXAm1ewLNDP0Ynua6br/84k3xUjHNvG9O6/4kwNoJNGw0DFxE4WOxYGFgVcWAwGv+vAnEQH\nZwE9dP42i8EdxYF5UAHI4TqwrEEJhSIOTGwHvhWo5E0HTiNDsieOHqpC0UMwECPsrkXZt4tYMYPH\nUcRWEyG7M4vjgrmk9iYpdeyAUB3eaTVodtkJTEQS/kKcrAODw0NHxNhsWPX1g+MDHDIdNYMXp+nq\nUtEDNYTcGUoz55J56W8ktk1lIKChFbKU+mN4wp3kFAceI45ayOInQgQv/TRSwkEzO/AQw0YBBRUb\nOVRMitjRUTCwMLChUgIssrixULGRR6OAgY6CSQGdGLVkFRcuJUefVo3q8+GwsiSr6tEyGfqtII6+\nTtIeDbUjh/uiqdhe2YMSrEVVLYrZEv2v9VB3XoPsACYgCX8hRuI4ZxMfshiNjSbRqEox1IgjHaHh\nQ00YS6eyf88FRHuKeF75v4S7phDK7CVvm4yWiLPZNp3GYic9ttnYklEw/NRn92Mjj5rNohlJLAzU\nA29lF1l0LLJUHdhhKDjJUMB34LE8DvIYgKop6E4d0+Mh7msmHPo7phu7sBSTXMqgyqWRV6tx2VUK\nMUi80sf0uXZQLCxNR1EHn5/oiOGfU/v+G0CMOxL+Qpwihw4WQy2ELfRikWm1JjN6e1Fmnonxzi72\ndNRhYCNWXU+tlWd/9TXMrC9Qn9yJnk1j9TeR37aHdDRPqmCjT2kgmOmkKhvFzPViM0sUbFX0uObi\nUHKkC3lKio19RQchswe9mMaGga5buGoNOnyTcZ81hU7NTbp+Lo72rTiqVbBMVG8VWilGyd2Ans9R\n6E3jslmUAkHUaATT68PIG8f708U4JOEvxBg5dMzADIVQbDb0bJYpZ3pIWj4Cqh3FU8WsswYv1KL0\nTkXt60ONRPAuOgN/psDuqA/X2wn2Fi5h8sAbGLEuPPFuEjWNeLwhHFUa4X6TRN4Bmo101oNeSKFn\nUhRdPmxTm2ia6aMmswXfBdPQG2rpV2aQ6UkyNbsdm9tFwu6jyl5EHxiABFgEUFQNxSih9PZgTZk8\n1ptSnAQJfyHGynvGDMxJkzDOOAMlkcB7lLNqrWAQq1DACAYHB5hLJab2R4g0nEFVe4T01EWYXh39\nzNlk3t6OMxWmxp6kxuvmnb1eLFUj091AVecOSoUSXreBTUlRZdlwnj8Hu6efEgWUCxqIdVTjLBSx\nFZJouTQF1UN1QzXE+tAjYQqBABYqhgF+n8wLOhFJ+Asxlo4yZnDMMYT3DjA7nXDOWYRCNnqmzqAq\n3oOqgO6pRpkzC1/9DILBImp3N0p1in17DIyADY9Wg7OYRnPqmJqNYl0dzlyaaY15lEaF6WqE3c31\ndL1pocb70Ku9OJ3gVsLUNgXIRnJoyRRWXQDfrACaamGeos0lykfCX4iJ5Cg7C7vdor7eJJvTsPIl\nbDYIBk1sNgtUFaO5GZwmgfQOqPPh6I1iqAGMaJKiuwanYuJrdqPkM5iKgm6VmDbNxO3ykH+pAzQN\nnx/8FugFE21e0+COqM4PloWlS4xMRKP2v/b000/z+9//Hq/XC8BNN90kl3MUYhT4/RZdXQruKTXY\n+7rw+SzCA+DzmlAqYTU24g1q7CtOQk/GcHm8uLr3ga4TDChoPi+aZmJWVQ2Guc2GpkGoUUW5tAE1\nHEYxTSwlgJnLgaJgaRpY1uD66+vHehOIkzCqu+wPf/jDLFu2bDSbEKLivfcwUt0OIZuJardh+t89\n72DuWbBtWy05wAjWU2NGKPX14A3YMH0eLJvtiDA/OM5gHTiRzSqVUCIRLJ/vsPMaxMQj39eEOA0c\nehhpMBgkHFaOGDew2+GsswyiYS9KZxrNUYtvTg1KIgb5PGYwODhp26FhfpRxBvOssyTwTwOKZVmj\nMlT/9NNPs2HDBlwuFzNmzODmm2/G7XYfsVxbWxttbW0AtLa2UigURqOcstJ1nVKpNNZlHJfUWV6n\nVZ2GAZEIFItgs0EgcEoD/bTaluOA3W4f9nNGFP6rVq0iFosdcf+NN97I7Nmzh/r7n3rqKaLRKHfe\needx19nV1XWy5Zwyg5+swmNdxnFJneUldZbPRKgRJk6djY2Nw37OiLp9HnjggRNabunSpTzyyCMj\naUoIIUQZjdoFhqLR6NDvmzZtorm5ebSaEkIIMUyjNuD77//+73R0dKAoCnV1ddx+++2j1ZQQQohh\nGrXw/8IXvjBaqxZCCDFCp/K60kIIIcYJCX8hhKhAEv5CCFGBJPyFEKICSfgLIUQFkvAXQogKJOEv\nhBAVSMJfCCEqkIS/EEJUIAl/IYSoQBL+QghRgST8hRCiAkn4CyFEBZLwF0KICiThL4QQFUjCXwgh\nKpCEvxBCVCAJfyGEqEAjuozjSy+9xDPPPENnZycPP/wwM2fOHHps3bp1vPDCC6iqyi233MK55547\n4mKFEEKUx4g++Tc3N3PPPfdw5plnHnb//v372bhxI9/+9re5//77efzxxzFNc0SFCiGEKJ8RhX9T\nUxONjY1H3L9582YWLVqEzWajvr6ehoYG2tvbR9KUEEKIMhpRt8+xRCIRZs+ePXQ7EAgQiUSOumxb\nWxttbW0AtLa2EgwGR6OkstJ1XeosI6mzvCZCnROhRpg4dZ6M44b/qlWriMViR9x/4403snDhwqM+\nx7KsEy6gpaWFlpaWodvhcPiEnztWgsGg1FlGUmd5TYQ6J0KNMHHqPFoPzPEcN/wfeOCBYa+0traW\ngYGBoduRSIRAIDDs9QghhBgdo3Ko54IFC9i4cSPFYpG+vj66u7uZNWvWaDQlhBDiJIyoz3/Tpk08\n8cQTJBIJWltbmTZtGvfffz/Nzc1ccskl3H333aiqymc+8xlUVU4pEEKI8WJE4X/hhRdy4YUXHvWx\na6+9lmuvvXYkqxdCCDFK5OO4EEJUIAl/IYSoQBL+QghRgST8hRCiAkn4CyFEBZLwF0KICiThL4QQ\nFUjCXwghKpCEvxBCVCAJfyGEqEAS/kIIUYEk/IUQogJJ+AshRAWS8BdCiAok4S+EEBVIwl8IISqQ\nhL8QQlQgCX8hhKhAI7qM40svvcQzzzxDZ2cnDz/8MDNnzgSgr6+PlStX0tjYCMDs2bO5/fbbR16t\nEEKIshhR+Dc3N3PPPffw4x//+IjHGhoaePTRR0eyeiGEEKNkROHf1NRUrjqEEEKcQiMK//fT19fH\nV77yFVwuFzfeeCNnnnnmaDUlhBBimI4b/qtWrSIWix1x/4033sjChQuP+hy/38/atWvxeDzs2rWL\nRx99lDVr1lBVVXXEsm1tbbS1tQHQ2tpKMBgc7t9wyum6LnWWkdRZXhOhzolQI0ycOk/GccP/gQce\nGPZKbTYbNpsNgBkzZhAKheju7h4aED5US0sLLS0tQ7fD4fCw2zvVgsGg1FlGUmd5TYQ6J0KNMHHq\nPHhwzXCMyqGeiUQC0zQB6O3tpbu7m1AoNBpNCSGEOAkj6vPftGkTTzzxBIlEgtbWVqZNm8b999/P\n1q1befrpp9E0DVVVue2223C73eWqWQghxAiNKPwvvPBCLrzwwiPuv/jii7n44otHsmohhBCjSM7w\nFUKICiThL4QQFUjCXwghKpCEvxBCVCAJfyGEqEAS/kIIUYEk/IUQogJJ+AshRAWS8BdCiAok4S+E\nEBVIwl8IISqQhL8QQlQgCX8hhKhAEv5CCFGBJPyFEKICSfgLIUQFkvAXQogKJOEvhBAVSMJfCCEq\n0Iiu4fvkk0/yyiuvoOs6oVCIO++8k+rqagDWrVvHCy+8gKqq3HLLLZx77rllKVgIIcTIjeiT/znn\nnMOaNWtYvXo1kyZNYt26dQDs37+fjRs38u1vf5v777+fxx9/HNM0y1KwEEKIkRtR+H/gAx9A0zQA\n5syZQyQSAWDz5s0sWrQIm81GfX09DQ0NtLe3j7xaIYQQZTGibp9DvfDCCyxatAiASCTC7Nmzhx4L\nBAJDO4b3amtro62tDYDW1lYaGxvLVdKokjrLS+osr4lQ50SoESZOncN13E/+q1at4ktf+tIRP5s3\nbx5a5tlnn0XTNC677DIALMs64QJaWlpobW2ltbWVe++99yT+hFNP6iwvqbO8JkKdE6FGOL3rPO4n\n/wceeOB9H//jH//IK6+8woMPPoiiKADU1tYyMDAwtEwkEiEQCAy7OCGEEKNjRH3+r7/+Os899xxf\n/epXcTgcQ/cvWLCAjRs3UiwW6evro7u7m1mzZo24WCGEEOWh/fM///M/n+yTH3roIQqFAn/5y1/4\n3//9Xzo6Orjgggvw+XykUil+9KMf8ec//5lbb731hPvNZsyYcbLlnFJSZ3lJneU1EeqcCDXC6Vun\nYg2ng14IIcRpQc7wFUKICiThL4QQFahsx/mPxESZJuKll17imWeeobOzk4cffpiZM2cC0NfXx8qV\nK4fGNWbPns3tt98+7uqE8bU9D/X000/z+9//Hq/XC8BNN93E+eefP8ZVDXr99df52c9+hmmaLF26\nlOXLl491SUd111134XQ6UVUVTdNobW0d65IAWLt2La+++io+n481a9YAkEqleOyxx+jv76euro6V\nK1fidrvHXZ3j8XUZDof5/ve/TywWQ1EUWlpauOaaa4a/Ta1x4PXXX7dKpZJlWZb15JNPWk8++aRl\nWZa1b98+65577rEKhYLV29trff7zn7cMwxizOvft22d1dnZaX//616329vah+3t7e6277757zOp6\nr2PVOd6256Geeuop67nnnhvrMo5gGIb1+c9/3urp6bGKxaJ1zz33WPv27Rvrso7qzjvvtOLx+FiX\ncYQtW7ZYO3fuPOw98uSTT1rr1q2zLMuy1q1bN/SeH0tHq3M8vi4jkYi1c+dOy7IsK5PJWF/84het\nffv2DXubjotun4kyTURTU9OEONvvWHWOt+05EbS3t9PQ0EAoFELXdRYtWnTYCY7i+ObNm3fEJ9DN\nmzezePFiABYvXjwutunR6hyP/H7/0JE9LpeLyZMnE4lEhr1Nx0W3z6FOdpqIsdbX18dXvvIVXC4X\nN954I2eeeeZYl3SE8b49f/e73/Hiiy8yY8YMbr755nHxRoxEItTW1g7drq2tZceOHWNY0ft76KGH\nALjiiitoaWkZ42qOLR6P4/f7gcEwSyQSY1zRsY3H1+VBfX197N69m1mzZg17m56y8F+1ahWxWOyI\n+2+88UYWLlwIjGyaiHI5kTrfy+/3s3btWjweD7t27eLRRx9lzZo1VFVVjas6x2J7Hur9ar7yyiu5\n7rrrAHjqqaf4xS9+wZ133nmqSzzC0bbZwTPZx5tVq1YRCASIx+N84xvfoLGxkXnz5o11WRPaeH1d\nAuRyOdasWcOnP/3pk8qaUxb+E2WaiOPVeTQ2mw2bzQYMnmgRCoXo7u4+bKC13E6mzrGeduNEa166\ndCmPPPLIKFdzYt67zQYGBoY+XY03B/8vfT4fCxcupL29fdyGv8/nIxqN4vf7iUajQwOq401NTc3Q\n7+PpdVkqlVizZg2XXXYZF110ETD8bTou+vwn+jQRiURi6HoFvb29dHd3EwqFxriqI43n7RmNRod+\n37RpE83NzWNYzbtmzpxJd3c3fX19lEolNm7cyIIFC8a6rCPkcjmy2ezQ72+++SZTpkwZ46qObcGC\nBWzYsAGADRs2HPPb6lgbj69Ly7L44Q9/yOTJk/nIRz4ydP9wt+m4OMP3C1/4AqVSaagv7dBDJZ99\n9ln+8Ic/oKoqn/70pznvvPPGrM5NmzbxxBNPkEgkqK6uZtq0adx///385S9/4emnn0bTNFRV5frr\nrx/TgDhWnTC+tuehvve979HR0YGiKNTV1XH77bePm0/Yr776Kj//+c8xTZMlS5Zw7bXXjnVJR+jt\n7WX16tUAGIbBpZdeOm7q/M53vsPWrVtJJpP4fD5uuOEGFi5cyGOPPUY4HCYYDHL33XePeV/60erc\nsmXLuHtdbtu2jQcffJApU6YM9ZLcdNNNzJ49e1jbdFyEvxBCiFNrXHT7CCGEOLUk/IUQogJJ+Ash\nRAWS8BdCiAok4S+EEBVIwl8IISqQhL8QQlSg/x/NeCjXbAiX0AAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "height": 85 + }, + "colab_type": "code", + "id": "b23iIkX8VVyn", + "outputId": "acae7840-0adf-43cd-f92d-550b2a7e42dc" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Principal axes:\n", + "tf.Tensor(\n", + "[[ 2.2801023]\n", + " [-1.1619819]], shape=(2, 1), dtype=float32)\n" + ] + } + ], + "source": [ + "actual_w, actual_z, x_train = model.sample()\n", + "\n", + "print(\"Principal axes:\")\n", + "print(actual_w)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "O2ZdIFz7VuSO" + }, + "source": [ + "We visualize the dataset." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "height": 282 + }, + "colab_type": "code", + "id": "ubJJvk0KVyVW", + "outputId": "b0ecbbc5-1045-499d-d45a-6da1848969ff" + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEJCAYAAAB8Pye7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt0VeWd//H33vvkkIRcOMnJxYQg\nBhBBa9UCKj8ttkR7c7lYtnXBXLzN0uWo7ai1HUdGrQt1xSpip6PtXNSZsTOzhFUv80+nayIj2kEl\nFSnKRQgYriG3c3K/nr2f3x8bDiAghJxDctyf11pZ5Fz3N4fks/d+nmc/j2WMMYiISKDYY12AiIic\neQp/EZEAUviLiASQwl9EJIAU/iIiAaTwFxEJIIW/iEgAKfwl49x8881YloVlWYRCIYqKirj88st5\n9NFHicViI36/6dOn89Of/jT1hZ6GX//611iWNdZlSAAo/CUjXXnllTQ1NbF7927eeecdbrvtNv7z\nP/+T888/n23bto11eSLjnsJfMlI4HKa8vJyKigrOP/98br31VtatW0dubi533HFH8nnr16/nW9/6\nFqWlpeTl5TF37lz++7//O/n4VVddxY4dO3j00UeTZxONjY0YY7jtttuYNm0aOTk5VFdX8+CDDzI4\nOPi5db3xxhtcfPHF5ObmMmnSJObNm8eHH36YfLyhoYHvfve7TJo0iUgkwjXXXMNHH30EwFtvvcWf\n//mfAyRrufnmm1P4qYkcpvCXL4yCggL+8i//krfeeovW1lYAurq6WLx4MW+99Rbr16/nG9/4Btdd\nd13y7ODVV19l6tSp/OhHP6KpqYmmpiaqqqowxlBWVsZ//Md/sGXLFp599lleeuklnnjiiRNu/8CB\nA3z/+99nyZIlbNq0iXfffZd77rmHUCgEQHNzM1dccQWlpaW88847vPfee8ycOZOrrrqK1tZW5s+f\nz9///d8DJGv5+c9/nuZPTQLLiGSYm266ySxcuPC4j/32t781gHn//fdP+PoLL7zQPPbYY8nb06ZN\nM4888shJt/vMM8+Y6dOnn/Dx9evXG8B8+umnx338kUceMZdeeulR93meZ6qrq82KFSuMMca8/PLL\nRn+WciaExnjfI5JS5uA8hYc6TVtbW3nkkUdYvXo1Bw4cIJFIMDAwwK5du076Xv/0T//EP//zP9PY\n2Ehvby+JRALP8074/AsvvJBvfOMbXHDBBVx99dVcddVVXH/99VRVVQFQX1/PBx98QF5e3lGv6+/v\nZ/v27af7I4ucFoW/fKF8/PHHWJZFdXU14I8M2r17Nz/72c8455xzyMnJYfHixQwNDX3u+6xatYq7\n7rqL2tpaFixYQEFBAatWrWLp0qUnfI3jOPz2t7+lvr6euro6fvOb3/DAAw+watUqrr32WjzPY+HC\nhcmmnSMVFhaO7gcXGSGFv3xhdHV18ctf/pKFCxdSXFwMwNtvv83PfvYzrrvuOgB6e3vZuXMnF1xw\nQfJ14XAY13WPeq+3336biy++mPvuuy95X2Nj40lrsCyLefPmMW/ePB588EG++c1v8tJLL3Httdcy\nZ84c/uVf/oXKykpycnKO+/pwOAyA67o4jjOin19kJNThKxlpaGiIAwcO0NTUxObNm3nxxReZN28e\ng4OD/PKXv0w+b+bMmfz7v/87H330ERs2bGDJkiXHBP0555zD//3f/7F7927a2trwPI+ZM2fy0Ucf\n8cYbb7Bjxw5+/vOf8+qrr35uTWvXrmXZsmW8//777N69mzfffJONGzcye/ZsAO6++25c12XRokW8\n8847NDY28vvf/56lS5eydu3aZC0A//Vf/0Vrays9PT2p/NhEDhvrTgeRkbrpppsMYADjOI6ZNGmS\nufTSS82jjz5qYrHYUc/duHGjufzyy012drY5++yzzXPPPWcWLlxobrrppuRz6uvrzSWXXGKys7OT\nHbZDQ0Pm9ttvN5FIxOTn55slS5aYX/ziF5/bGfvxxx+bb33rW6asrMyEw2EzZcoUc//995vBwcHk\ncxobG82f/MmfmGg0mnzOn/7pn5qdO3cmn/NXf/VXprS01FiWdVSdIqlkGaOVvEREgkbNPiIiATTq\nDt+2tjaee+45Ojo6sCyLmpoavv3tb9PT08OKFStobW2lpKSEe++995ghbiIiMjZG3ewTj8eJx+NU\nV1fT39/PAw88wI9//GPeeust8vLyWLRoEa+//jo9PT382Z/9WarqFhGRURh1s08kEkmOqc7JyaGy\nspJYLEZ9fT0LFiwAYMGCBdTX1492UyIikiIpHeff0tLCp59+yvTp0+ns7CQSiQD+DqKrq+u4r6mr\nq6Ourg6A2traVJYjIiInkLLwHxgYYPny5dx8883k5uae8utqamqoqalJ3t6/f3+qSkqbaDRKW1vb\nWJdxUqoztVRn6mRCjZA5dVZUVIz4NSkZ7ZNIJFi+fDlXXnkll156KeBfrh6PxwG/X6CgoCAVmxIR\nkRQYdfgbY/jVr35FZWUl1157bfL+OXPmsGbNGgDWrFnD3LlzR7spERFJkVE3+3zyySe8/fbbTJky\nhR//+McALFmyhEWLFrFixQpWr15NNBo9ao4UEREZW6MO//POO4+VK1ce97GHH354tG8vIiJpoCt8\nRUQCSOEvIhJACn8RkQBS+IuIBJDCX0QkgBT+IiIBpPAXEQkghb+ISAAp/EVEAkjhLyISQAp/EZEA\nUviLiASQwl9EJIAU/iIiAaTwFxEJIIW/iEgAKfxFRAJI4S8iEkCjXsYR4Pnnn2f9+vUUFhayfPly\nAFauXMmbb75JQUEB4K/re8kll6RicyIiMkopCf+rrrqKb37zmzz33HNH3f+d73yH6667LhWbEBGR\nFEpJs8/s2bPJy8tLxVuJiMgZkJIj/xP53e9+x9tvv011dTU33nijdhAiIuOEZYwxqXijlpYWnnzy\nyWSbf0dHR7K9/5VXXiEej3PnnXce87q6ujrq6uoAqK2tZWhoKBXlpFUoFCKRSIx1GSelOlNLdaZO\nJtQImVNnOBwe8WvSduQ/adKk5PcLFy7kySefPO7zampqqKmpSd5ua2tLV0kpE41GVWcKqc7UyoQ6\nM6FGyJw6KyoqRvyatA31jMfjye/XrVtHVVVVujYlIiIjlJIj/2effZbNmzfT3d3NHXfcwQ033MCm\nTZtobGzEsixKSkq4/fbbU7EpERFJgZSE/z333HPMfV//+tdT8dYiIpIGusJXRCSAFP4iIgGk8BcR\nCSCFv4hIACn8RUQCSOEvIhJACn8RkQBS+IuIBJDCX0QkgBT+IiIBpPAXEQkghb+ISAAp/EVEAkjh\nLyISQAp/EZEAUviLiASQwl9EJIAU/iIiAaTwFxEJoJSs4fv888+zfv16CgsLWb58OQA9PT2sWLGC\n1tZWSkpKuPfee8nLy0vF5kREZJRScuR/1VVX8eCDDx513+uvv86XvvQl/u7v/o4vfelLvP7666nY\nlIiIpEBKwn/27NnHHNXX19ezYMECABYsWEB9fX0qNiUiIimQkmaf4+ns7CQSiQAQiUTo6uo67vPq\n6uqoq6sDoLa2lmg0mq6SUiYUCqnOFFKdqZUJdWZCjZA5dZ6OtIX/qaqpqaGmpiZ5u62tbQyrOTXR\naFR1ppDqTK1MqDMTaoTMqbOiomLEr0nbaJ/CwkLi8TgA8XicgoKCdG1KRERGKG3hP2fOHNasWQPA\nmjVrmDt3bro2JSIiI5SSZp9nn32WzZs3093dzR133MENN9zAokWLWLFiBatXryYajXLfffelYlMi\nIpICKQn/e+6557j3P/zww6l4exERSTFd4SsiEkAKfxGRAFL4i4gEkMJfRCSAFP4iIgGk8BcRCSCF\nv4hIACn8RUQCSOEvIhJACn8RkQBS+IuIBJDCX0QkgBT+IiIBpPAXEQkghb+ISAAp/EVEAkjhLyIS\nQAp/EZEASskyjp/nrrvuIjs7G9u2cRyH2tradG9SREROIu3hD/DII49QUFBwJjYlIiKnQM0+IiIB\nZBljTDo3cNddd5GXlwfA1VdfTU1NzVGP19XVUVdXB0BtbS1DQ0PpLCclQqEQiURirMs4KdWZWqoz\ndTKhRsicOsPh8Ihfk/bwj8ViFBUV0dnZyWOPPcYtt9zC7NmzT/j8/fv3p7OclIhGo7S1tY11GSeV\nyjpdF+Jxi0TCIhQyRCIGx0nJWwfy80ynTKgzE2qEzKmzoqJixK9Je7NPUVERAIWFhcydO5eGhoZ0\nb1JSzHVh/36b4WELgOFhi/37bVx3dO/Z1mZx4IBNayujei8RGbm0dvgODAxgjCEnJ4eBgQE2btzI\n9773vXRuUtKgrc2iq8vC8yzA4HkQj9t8/LHD5MkepaUe0ejhM4HPniUUFBi6uo6+vX+/TW+v/56J\nBLS2OkSjhv5+mDgRJkxI7dmFiBwtreHf2dnJ008/DYDrulxxxRVcdNFF6dykpJjrwr59DqEQDA/D\nzp0hWlosJkyAwkI4cMCiv9+iv9+jqsoD/LOEUAgsC3p7LT78MEQ0asjKMhhjsWGDf8JZXGywLNi7\nF+Jxh/Z2Q0mJoacHSks99u+3qKjwtAMQSYO0hn9ZWRlPPfVUOjchaRaPW0yYYOjvt9i0ySEWs4nF\nLMJh6Okx2LZFb6/D8DDk5PjdR4fOEjzPsGOHg+fZ9PV5hEIwMOCfRTiOwfMMEyYY8vJgcNBicNCi\nrMzFsvz3iEQM8bhFNJrWbimRQDoj4/wlcyUSFvn5hk2bQgwO+sE9PGzT1wfhsMfu3TZVVR5DQzZ7\n9oAxMGECeB40NISIxSwmTYLdu236+mxKSz26uy2Mgf5+m+Fhj/Jy2LfPIhSCggIbxzGAxdCQobsb\nior8M4Ti4qObl0Tk9Cn85XOFQob2dpuiIkMs5ge04xhs228GGh72m34mTPBD2XH88O/utrBti6ws\n2LPHoq0NYjGLrVsdBgY42Axk0dsLTU1+s09BgXcw/CEry9Dbaw42KRlKSjz27nUYGPCYPFlNQSKj\npfCXzxWJGHbvtgiHDdEoWJYhHoe+PouODouCAsjK8r/6+21KSjwSCf+MwbYNg4MWW7b4o4M8L4Rt\nGwYGDPG43+TT0eE3GeXlWUSjNrEYVFW55OX5ncpTpvj9Dq2tDhMmWGzd6pGX5zc3OY6hstKjvPzz\nO5zVcSxyLIW/fC7HgcpKl+Zmm9ZWm2jU3wls3+6P0qmshJISP4A7Oy36+2HqVI+BAQiHbTZudGht\ntejrsxkags5Om9xc8DxDX58hFLLJy4OmpixaWiAUShCLGfLz/aahHTugo8MiN9dQWmoYGnKYONEw\nY4bfFLRrl2HKFI+KCpfzz3dxnMMdzp4H7e02u3dbVFa6ajISOYLCX04qGjUMDRnmzBlm+3aH7m6b\naBTOOcdQWGgoL/ewbcjN9TtmbRsmT/b4+OMQiYRFd7dFb69NPG4xPAzd3QAWrmvhONDZ6W8nLw96\ne0P88Y82eXl+k9DwsIVl+U1JrmsIhw0TJ8LOnS7V1R4TJsCBAzYlJTY7dzqUlblMnGiRleU3GU2Y\n4I86amhw2LcP7QREDlL4y0k5DlRUeLS1WUyaZMjJcYlELDzPP7p2Xejrg5wcyM72sG1DT4/fdu83\nATl0dFgMDPgdwkc68uKuQzsB8DuUjzQ4CGDR2+u/T0uLzZYtHjk5HgUFhunTDeXlNtGoQ0GBIRbz\nm55mznTJzvabgfLy/P6J885zOfts9RtIsCn85ZQc6sytrPSwLD+0m5v98fodHRbFxQbXhfx8fzin\n5x0O86Ehv4koVROJ9Pcfel9/BFFnJ/T2uuzf7x3shPb7KAoLDX/4g0VBgc2MGS5nnWXo6XHYssVm\n2jSP6dM9Sko0gkiCSeEvpyyR8JtgwN8RlJV57NplY1kWWVlecjx+W5vNwAAUFEBrq43r+tcFpGvO\nvkQCWlocWlr8WoyBcNhvKgqFIDfX74PYuNFvkgLYtMljwQKX8nKPSZP85qO+PouCAqMdggSCwl9O\nWShkkm3w4O8A8vJg0iSXoiI/+GMxf3in40A4bMjP9+jsNHR3W2egQit5djE0dHhnE4/bNDVBfr7f\n2WyMwbI8YjGb8893KSz0RwQVFflXGxcUGMrK/A7k05gsUSQjKPzllEUihv37reTUDX7Q+nP1HOK6\nfsiHw4ZZs1z+8Aeb/n6XwUELYxx6esamds/zm4f8pigL23bo7LTZu9emstKltNQjK8tvrqqs9Kiq\nsojFLGbN8giH/Z2DyBeJwl9O2aGO30Nj6LOyDOed59HcbGOMv0Pwx/H7fQAA3/52gjfesMnJgf37\nDQcOhBgY8C8OG0ue5zfz9PU5tLc7FBZ6FBV5hMM227a55OdDTo7D177mcd55CSZONJx/vj9Xka4b\nkC8Chb+MiONwsG3/8JHwkTuEaNSjv9/CtqGw0FBUZJg/P0FTk8X06R5/+INHV5dDR4c/9HM8GBry\n+yZaW+2DHds2eXn+dQW/+Y3LzJk2BQWGjz4CY8JkZfk/U2HhsTOWascgmULhL6P22R2C65rkzuDs\ns13OOstl27YQHR02Z5/t8eabBssK4bp+M5DnjW39R3Jd/ysWOzQBnUVTk98h/OGHMGOGg+fB1q0O\nCxYMMzxskZ0NPT3+jiwvzzBrVoKyMu0EZHxT+EvKHf/sYJi2NoutW0Ps3WsRDifIzfXYuzd08GKu\nsav3RA5NUzE46Kd4UxNs2RKmoAAiEZdNmywKCizKyjzKyvx1CAoL/fUOhoZcqqp0LYGMXwp/OSP8\noaEGx0nQ2emHZjjsnyV0dkJHB8dc2DUeDQ35k9S1tYUAv68gJ8ehvNxQUeExc6aL69oHm79sjPHP\nCDQjqYw3Cn85oyIRQ0WFS2enzYUXujiOYe/eLGw7QWGhPzdQX5+DMePzbOBoFp2dDp2dEIvB7t0e\n27ZZzJiRYOPGMOecY5g92x9KeuSMpKCJ52TsKfzljHIcuPBCf06HffsccnM9IhG/mcQYgzE227d7\ntLb6i8YMDdkZsBPwO439hW5surpsJk2CTz7x2LrVZsoUj7PP9ti716ax0cPzIBLxh8MWFMD+/X7T\nkTqO5UxS+MsZFw7DxRe7TJ3qkUhYtLZCe/swu3Y5dHZ6FBVBc7Nh/Xp/NtCOjvRdHZwO/tXGYNsO\nDQ3+iKDy8gSzZ3vk5toUFVlEIh45OYbhYZuSkgTNzRaVlf5MpcPDlpawlLRLe/hv2LCBl156Cc/z\nWLhwIYsWLUr3JiUDHNkpHI3CH/9omDLF5dNPHVzXv0r4S18ybN3qkp9v09Fh0dNzaIK3zODPb2TR\n2WmxZ0+YrVtdystdzjrLIivLprra7yfYuTMLxzF8/evDyVFCoRBawlLSKq3h73keL7zwAn/7t39L\ncXExf/M3f8OcOXOYPHlyOjcrGaakBKJRj3DYYmDAwhi/wzQ725Cd7c/Q6Tj+QjJtbbBrV+jgZHH+\nmUGm6O526O52iMUMOTnw8cf+zz1xokdFheG112D+/MPrErS02CQSRs1AkhZpDf+GhgbKy8spKysD\nYP78+dTX1yv85SiO48//H4/7VwZPneoHX1eXzQUXuPT0WBQW+ks7Tpzo0dDg8v77Dt3dFnv3WsRi\n/tw9rpsZrZjt7f4IoFDIX6EsL8+mvd1wzjn+1dG7d/vLZpaUGDzPwxhLC9JIyqX1ryUWi1FcXJy8\nXVxczPbt2496Tl1dHXV1dQDU1tYSjUbTWVJKhEIh1ZlCoVCIsrIoB48RAP9Cq717Sa7I1dHhT+U8\nZQp89atwxRXw9tvQ2uo/9sc/wief+MNF09c0lNo/l0TC/7erC3p6oLU1i+pqf+TQl78MZWX+Y57n\nj3zq6IDiYpg/31874YRVZsD/eybUCJlT5+lIa/ib40zgbllHX9JfU1NDTU1N8nZbW1s6S0qJaDSq\nOlPoRHUeWoTl0DxCpaX+UW9vr7985He/C9u22axfHyYchtzcEJ98YtHd7dDVla6pI9yTP+U0eJ4f\n9Bs2QFFRAsvy6Ox0sW1/gjnwm34qK2HPHpdrrhk+eJ3EscNGy8rG//97pv9ujjcVFRUjfk1aw7+4\nuJj29vbk7fb2diKRSDo3KV8gx7tS+EjhMFxwgce55w6wY4fNrFkef/yjRUODzbZtDt3dIfr6rIwa\nKQQQi4V4803D1Kn+hXDGOEya5JFIeIRChr4+2LbNZdYsL7le8ZGjhL6gB6qSYmkN/2nTptHU1ERL\nSwtFRUWsXbuWH/7wh+ncpARQOAyzZnmce+4Ql11msWOHw+uve3z6qd++3t7uj78fT3MInczQkMW2\nbVlkZ/tnBaEQ7NhhUV7uMXmywXWz6O5OkJ1tMWGCv2rZoVFCsRhY42POPBnH0hr+juNw66238vjj\nj+N5Hl/72teoqqpK5yYlwA5NIRGNJvA8w8aNFp9+atHXZ7Fzp0V3t83QkKG93WZgIDN6TQcG/H/9\nJSuho8NhcNCmpyeBZdlMnGgoK/Nob7coLzfk5Xnk5/uL7Ih8nrQPj7jkkku45JJL0r0ZkSTHgXnz\n/PbyWbMsWlttvvpV2L7dJjfXY8sWm337XA4csBkethgaGrtFZkZqYAC2bLHo7LT59FN/RFBursOM\nGR5tbYbKSpueHrjyys/vFBbJjLFxIiMUDsOcOQkaG/1hk4ODhjlzDC0tFtOnO2zZAuGwR0ODw/bt\n0NTkB6qbnv7cFLM5cMAmHIa2NkNhIcRiHpGI3w9w6aXw/vshrrwykRwWeryOYQ0ZDTaFv3xhhcNw\n7rkecLixf9o0cF2Xiy+22LgxREGBx5Qp/vw6W7Y49PRYxGIO3d3j++IxzzvUJGTR1we9vQ62DW1t\n0N4OluWwbZtHYaG/dGVPj01urmHSJIMxmj5CFP4SQI4DVVWGqqrDM8a5LmzaZLN+fRbvvz/Mrl02\nvb1ZdHRAR4dNIjF+e1CN8YeJtrZahEL+2cvwsE1fX5ipU10sy590znH8506fnqC42Gj6iIBT+Ivg\n7xDOP98jEhnmoossVq/OIh5P0N4Otm2xa5fF3r0OHR0ew8M2x7mEZUwZA01NNj09NvE4FBc7B4eL\n+juFnByoqDBUV0NDQ4ihIZfKSo8TDaOVLz6Fv8hBhxaoD4ctLr88wYcfhpg925+cbcoU2L0bWlsT\nxGIOg4OGtjabzs7x1TzU3e1fOdzZaWOMze7dNtGoxcSJhqEhA3hMm+bR12dlTCe3pIfCX+QIRw4X\nnTLF5aOPQmRn25xzjsuiRbB+/TBbt7oMDFg0N1v09tp0dlrE4/76BDD2zUP9/dDf7+A40NcXIicH\nhocNrptg1y4L2wbwm4fa2mwKCw3RqFYaCxqFv8hxOA5UVhoqKw/3C0QiMGFCgpkzLfbtc2hrs+js\n9MjK8tcbePddl+Zmf/rp/n5/IZpEwhqzi8sOLUa/ezdUVEA4bNHRARs3+lNGRCIWU6f6E+ZVVrr0\n93tadzhAFP4ip+jQ7KMTJ1qcdZbBsvxZOB0HurosqqocNm1ycF1zsH/A0Ntr0dVl6OiwGBqy8Dz7\njK9J0N/vjwIaHLQJhTxCoRCxGEyZ4uG6/kVibW0hdu70p5auqPAX2gmHz2ydcmYp/EVG4LPzDbmu\nP2omFIL584cpK/OIxWxmzoSPP/ZDv7zco60N4nGb1lb/385Of1jpmeg49jzo6bFwXb9JqrDQYmAA\nHMemudlwwQXD2LZNUZE/fHR42GLnTsMFFyQIhXRdwBeVwl9kFD67MyguTrBjh82+fSGuuSbB8LCF\n41i0t1sMDXns3m2zb5+ht9dlzx6b9nYb1zX09joMDqZ3Z2AMhEI2nZ3+jKktLRCNWtTXh5g71+XA\nAYu+PkNfn0U4bNPba3HZZQldF/AFpfAXSaEjJ5mLx/2mnp4eKCnxg/2CC/yFarZssSkvNxjjMjAA\nu3d7NDbaWFaIwUHIyvLn8B8e9tcosCx/R+O6foifaj/CobH9/uygh+cKMsYfwjowAK5rsWOHX092\ntqGz099+d7c/ZLS01KOgwD/DiUSMrhT+glD4i6TBsc1DJKdfLitzmTXLZd8+i+3b/WmnZ850aWz0\n2LXLYNuG7m6LUMhicNCis9MwPOyQn+8vaemvcWzo7fVH7NgHR5smEv5Owbb9cHecw48d+nd4+PCO\nIBTyGB6G/HwXy/JXUTu0s+jutgCLzZsdOjps8vM9pk1z6e+3jplCWmcEmUnhL3IGHLqG4NBRc3a2\nYc4cj4svdtmxw6a52aGigoMzcx5a3MUiHrcpKPDYvz9BPG4xfbrBtv2Ltvbu9V83NGTwPL//wbJs\nsrMdBgddcnNtenoMw8OQk+Nf4dvZ6dcTDhvy8z2KivxRTV1dkEhYuK6/s+rqsigosBgeNnieRUuL\nP/ndV77iJqeLtiwtNJ/JFP4iZ8jxFqdxHL+ZaNYsvx2nv3+Y+voQQ0M2AwP+Qu/d3Rbf+pZLSYl/\nhN3fb7F1q01jo8OBA8O0t9uAxeCgob/fYsKELCwrQU+PTVYW9PcbXNcmHreYPNmjq8ti0iSLggKP\nK67w6OiA6dM9WlosCgr8s4PCQn9bBQWH63Rdm64uj6Kiw/VbFgenvlD4ZxqFv8g4kpMD/+//JU7S\nrm6YOtVjaCjBpk0Omzdn0d4OubmQSHi4LliWRzjsUlho2LrVpq3Nn+ohFPKboFpabCIRj9xcw5w5\nCdrbbaqrYeJEw86dDu3tUFBgKCgAxzEUFBgGBo4NemP86wQk8yj8RcaZky1feUg4DBde6HcY5+aG\nsG3/iL2qCvbudbFtfwz/3Lmwbl0IYyw6Omw6OqCoyB/Tn5fnf1VUJOjtBWMsbNuQn29TWOjvLIzx\nzwYmT/bnuzbGP+I3xu9nKC1V+Gcihb9IBnMcOPtsw4QJbrIjNisL8vMN5eUeFRV+c5Jt+2cJWVlw\n1lmGUOjwYi+W5X+de66XHFHU2Og3Kxlj4TiGiRP9M5CzzvKbjRIJv9O5tFSjfTKVwl8kw4XDhtJS\nP5T9kUBQWuoRDh8+Is/NNVRWGizLP3p3Xb9TF/zgP3LEjuPA1KkeeXnmYH8CFBcfnvvnVM5KZPxL\nW/ivXLmSN998k4KDPUZLlizRco4iaRCJGPbv98fgW5YhEvEv4IpEzGee44/csW1/6Gdenr/TyM4+\n9uj90AR3ZWUZsbSZnIa0Hvk5OuLyAAAMJUlEQVR/5zvf4brrrkvnJkQC77PDSMNhjhl77zhw3nku\nW7f6TTmhkKGoyGDM0TsJCQ41+4h8ARzZHBON+hO5fVY4DOef7+oKXQHAMiY9s4msXLmSNWvWkJOT\nQ3V1NTfeeCN5eXnHPK+uro66ujoAamtrGRoaSkc5KRUKhUgkEmNdxkmpztRSnamTCTVC5tQZPo0p\nWEcV/suWLaOjo+OY+xcvXsyMGTOS7f2vvPIK8XicO++886TvuX///tMt54yJRqO0He/QapxRnaml\nOlMnE2qEzKmzoqJixK8ZVbPPQw89dErPW7hwIU8++eRoNiUiIimUtgVI4/F48vt169ZRVVWVrk2J\niMgIpa3D99e//jWNjY1YlkVJSQm33357ujYlIiIjlLbw/8EPfpCutxYRkVFKW7OPiIiMXwp/EZEA\nUviLiASQwl9EJIAU/iIiAaTwFxEJIIW/iEgAKfxFRAJI4S8iEkAKfxGRAFL4i4gEkMJfRCSAFP4i\nIgGk8BcRCSCFv4hIACn8RUQCSOEvIhJACn8RkQAa1TKO7777LqtWrWLfvn088cQTTJs2LfnYa6+9\nxurVq7Ftm1tuuYWLLrpo1MWKiEhqjOrIv6qqivvvv59Zs2Yddf/evXtZu3YtzzzzDEuXLuWFF17A\n87xRFSoiIqkzqvCfPHkyFRUVx9xfX1/P/PnzycrKorS0lPLychoaGkazKRERSaFRNfucSCwWY8aM\nGcnbRUVFxGKx4z63rq6Ouro6AGpra4lGo+koKaVCoZDqTCHVmVqZUGcm1AiZU+fpOGn4L1u2jI6O\njmPuX7x4MXPnzj3ua4wxp1xATU0NNTU1ydttbW2n/NqxEo1GVWcKqc7UyoQ6M6FGyJw6j9cCczIn\nDf+HHnpoxG9aXFxMe3t78nYsFqOoqGjE7yMiIumRlqGec+bMYe3atQwPD9PS0kJTUxPTp09Px6ZE\nROQ0jKrNf926dbz44ot0dXVRW1vL1KlTWbp0KVVVVVx++eXcd9992LbNX/zFX2DbuqRARGS8GFX4\nz5s3j3nz5h33seuvv57rr79+NG8vIiJposNxEZEAUviLiASQwl9EJIAU/iIiAaTwFxEJIIW/iEgA\nKfxFRAJI4S8iEkAKfxGRAFL4i4gEkMJfRCSAFP4iIgGk8BcRCSCFv4hIACn8RUQCSOEvIhJACn8R\nkQBS+IuIBNColnF89913WbVqFfv27eOJJ55g2rRpALS0tHDvvfdSUVEBwIwZM7j99ttHX62IiKTE\nqMK/qqqK+++/n3/8x3885rHy8nKeeuqp0by9iIikyajCf/LkyamqQ0REzqBRhf/naWlp4Sc/+Qk5\nOTksXryYWbNmpWtTIiIyQicN/2XLltHR0XHM/YsXL2bu3LnHfU0kEuH5558nPz+fnTt38tRTT7F8\n+XJyc3OPeW5dXR11dXUA1NbWEo1GR/oznHGhUEh1ppDqTK1MqDMTaoTMqfN0nDT8H3rooRG/aVZW\nFllZWQBUV1dTVlZGU1NTskP4SDU1NdTU1CRvt7W1jXh7Z1o0GlWdKaQ6UysT6syEGiFz6jw0uGYk\n0jLUs6urC8/zAGhubqapqYmysrJ0bEpERE7DqNr8161bx4svvkhXVxe1tbVMnTqVpUuXsnnzZlau\nXInjONi2zW233UZeXl6qahYRkVEaVfjPmzePefPmHXP/ZZddxmWXXTaatxYRkTTSFb4iIgGk8BcR\nCSCFv4hIACn8RUQCSOEvIhJACn8RkQBS+IuIBJDCX0QkgBT+IiIBpPAXEQkghb+ISAAp/EVEAkjh\nLyISQAp/EZEAUviLiASQwl9EJIAU/iIiAaTwFxEJIIW/iEgAjWoN35dffpkPPviAUChEWVkZd955\nJxMnTgTgtddeY/Xq1di2zS233MJFF12UkoJFRGT0RnXkf+GFF7J8+XKefvppzjrrLF577TUA9u7d\ny9q1a3nmmWdYunQpL7zwAp7npaRgEREZvVGF/5e//GUcxwHg3HPPJRaLAVBfX8/8+fPJysqitLSU\n8vJyGhoaRl+tiIikxKiafY60evVq5s+fD0AsFmPGjBnJx4qKipI7hs+qq6ujrq4OgNraWioqKlJV\nUlqpztRSnamVCXVmQo2QOXWO1EmP/JctW8aPfvSjY77q6+uTz3n11VdxHIcrr7wSAGPMKRdQU1ND\nbW0ttbW1PPDAA6fxI5x5qjO1VGdqZUKdmVAjfLHrPOmR/0MPPfS5j7/11lt88MEHPPzww1iWBUBx\ncTHt7e3J58RiMYqKikZcnIiIpMeo2vw3bNjAG2+8wV//9V8zYcKE5P1z5sxh7dq1DA8P09LSQlNT\nE9OnTx91sSIikhrOT3/605+e7osff/xxhoaGeO+99/if//kfGhsb+cpXvkJhYSE9PT38wz/8A7//\n/e+59dZbT7ndrLq6+nTLOaNUZ2qpztTKhDozoUb44tZpmZE00IuIyBeCrvAVEQkghb+ISAClbJz/\naGTKNBHvvvsuq1atYt++fTzxxBNMmzYNgJaWFu69995kv8aMGTO4/fbbx12dML4+zyOtXLmSN998\nk4KCAgCWLFnCJZdcMsZV+TZs2MBLL72E53ksXLiQRYsWjXVJx3XXXXeRnZ2Nbds4jkNtbe1YlwTA\n888/z/r16yksLGT58uUA9PT0sGLFClpbWykpKeHee+8lLy9v3NU5Hn8v29raeO655+jo6MCyLGpq\navj2t7898s/UjAMbNmwwiUTCGGPMyy+/bF5++WVjjDF79uwx999/vxkaGjLNzc3m7rvvNq7rjlmd\ne/bsMfv27TOPPPKIaWhoSN7f3Nxs7rvvvjGr67NOVOd4+zyP9Morr5g33nhjrMs4huu65u677zYH\nDhwww8PD5v777zd79uwZ67KO68477zSdnZ1jXcYxNm3aZHbs2HHU38jLL79sXnvtNWOMMa+99lry\nb34sHa/O8fh7GYvFzI4dO4wxxvT19Zkf/vCHZs+ePSP+TMdFs0+mTBMxefLkjLja70R1jrfPMxM0\nNDRQXl5OWVkZoVCI+fPnH3WBo5zc7NmzjzkCra+vZ8GCBQAsWLBgXHymx6tzPIpEIsmRPTk5OVRW\nVhKLxUb8mY6LZp8jne40EWOtpaWFn/zkJ+Tk5LB48WJmzZo11iUdY7x/nr/73e94++23qa6u5sYb\nbxwXf4ixWIzi4uLk7eLiYrZv3z6GFX2+xx9/HICrr76ampqaMa7mxDo7O4lEIoAfZl1dXWNc0YmN\nx9/LQ1paWvj000+ZPn36iD/TMxb+y5Yto6Oj45j7Fy9ezNy5c4HRTRORKqdS52dFIhGef/558vPz\n2blzJ0899RTLly8nNzd3XNU5Fp/nkT6v5muuuYbvfe97ALzyyiv827/9G3feeeeZLvEYx/vMDl3J\nPt4sW7aMoqIiOjs7eeyxx6ioqGD27NljXVZGG6+/lwADAwMsX76cm2+++bSy5oyFf6ZME3GyOo8n\nKyuLrKwswL/QoqysjKampqM6WlPtdOoc62k3TrXmhQsX8uSTT6a5mlPz2c+svb09eXQ13hz6vyws\nLGTu3Lk0NDSM2/AvLCwkHo8TiUSIx+PJDtXxZtKkScnvx9PvZSKRYPny5Vx55ZVceumlwMg/03HR\n5p/p00R0dXUl1ytobm6mqamJsrKyMa7qWOP584zH48nv161bR1VV1RhWc9i0adNoamqipaWFRCLB\n2rVrmTNnzliXdYyBgQH6+/uT32/cuJEpU6aMcVUnNmfOHNasWQPAmjVrTni2OtbG4++lMYZf/epX\nVFZWcu211ybvH+lnOi6u8P3BD35AIpFItqUdOVTy1Vdf5X//93+xbZubb76Ziy++eMzqXLduHS++\n+CJdXV1MnDiRqVOnsnTpUt577z1WrlyJ4zjYts33v//9MQ2IE9UJ4+vzPNIvfvELGhsbsSyLkpIS\nbr/99nFzhL1+/Xr+9V//Fc/z+NrXvsb1118/1iUdo7m5maeffhoA13W54oorxk2dzz77LJs3b6a7\nu5vCwkJuuOEG5s6dy4oVK2hrayMajXLfffeNeVv68erctGnTuPu93Lp1Kw8//DBTpkxJtpIsWbKE\nGTNmjOgzHRfhLyIiZ9a4aPYREZEzS+EvIhJACn8RkQBS+IuIBJDCX0QkgBT+IiIBpPAXEQmg/w+r\nFE9a9OxXDgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "\u003cFigure size 600x400 with 1 Axes\u003e" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "plt.scatter(x_train[0, :], x_train[1, :], color='blue', alpha=0.1)\n", + "plt.axis([-20, 20, -20, 20])\n", + "plt.title(\"Data set\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "qIZepLMgItfq" + }, + "source": [ + "## Maximum a Posteriori Inference\n", + "\n", + "We first search for the point estimate of latent variables that maximizes the posterior probability density. This is known as maximum a posteriori (MAP) inference, and is done by calculating the values of $\\mathbf{W}$ and $\\mathbf{Z}$ that maximise the posterior density $p(\\mathbf{W}, \\mathbf{Z} \\mid \\mathbf{X}) \\propto p(\\mathbf{W}, \\mathbf{Z}, \\mathbf{X})$.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "s2AvQAYqIh6K" + }, + "outputs": [], + "source": [ + "w = tf.Variable(tf.ones([data_dim, latent_dim]))\n", + "z = tf.Variable(tf.ones([latent_dim, num_datapoints]))\n", + "\n", + "target_log_prob_fn = lambda w, z: model.log_prob((w, z, x_train))\n", + "losses = tfp.math.minimize(\n", + " lambda: -target_log_prob_fn(w, z),\n", + " optimizer=tf.optimizers.Adam(learning_rate=0.05),\n", + " num_steps=200)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "height": 286 + }, + "colab_type": "code", + "id": "ya-XoAtpY474", + "outputId": "d59fcbd3-5df7-4b0d-9213-d3248c36ae9e" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[\u003cmatplotlib.lines.Line2D at 0x7f19897a42e8\u003e]" + ] + }, + "execution_count": 0, + "metadata": { + "tags": [] + }, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAD8CAYAAACLrvgBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt8lPWB7/HPbyYBcmcuCWkQVAyo\n0GAiyQqohGL0nEq7tWo9tdt2TeGllVZPyNbW1r523bZaXrUQGxLW1sOx1tJtrUfSbretuzQltKbU\nAEZBUYzxRrkEMjEkJJDJzO/8MTBySSD3Z5J836/XvDLzzPOb+c7DkG9+zzMXY621iIiI9IHL6QAi\nIjJ6qDRERKTPVBoiItJnKg0REekzlYaIiPSZSkNERPpMpSEiIn2m0hARkT5TaYiISJ+pNEREpM/i\nnA4wHPbt2zfgsX6/n8OHDw9hmqGhXP0Tq7kgdrMpV//Eai4YWLasrKw+raeZhoiI9JlKQ0RE+kyl\nISIifabSEBGRPlNpiIhIn6k0RESkz1QaIiLSZyqNE+zRdsL/8XOCDbudjiIiErPO++a+devWsWPH\nDtLS0li9ejUA7e3tlJWVcejQIdLT01m5ciXJyckAbNy4kerqalwuF8XFxeTm5gLQ2NhIZWUlXV1d\n5OXlUVxcjDGGYDBIRUUFjY2NpKSkUFJSQkZGBgCbN2/m2WefBeDmm29m8eLFw7ENIlwu7K9/Rldq\nKhTeOHz3IyIyip13prF48WK+8Y1vnLasqqqKnJwcysvLycnJoaqqCoC9e/dSW1vLmjVreOCBB1i/\nfj3hcBiAxx9/nLvuuovy8nIOHDhAfX09ANXV1SQlJbF27VqWLl3Khg0bgEgxPfPMMzz88MM8/PDD\nPPPMM7S3tw/pgz+VSUgEXwbd77w5bPchIjLanbc0Zs+eHZ1FnFRXV0dhYSEAhYWF1NXVRZcvXLiQ\n+Ph4MjIyyMzMpKGhgZaWFjo7O5k1axbGGBYtWhQds23btugMYv78+ezatQtrLfX19cydO5fk5GSS\nk5OZO3dutGiGTdZ0ut9tHN77EBEZxQZ0TKO1tRWPxwOAx+PhyJEjAAQCAXw+X3Q9r9dLIBA4a7nP\n5yMQCJw1xu12k5iYSFtbW6+3NZzMBRfS/bd3sN3dw3o/IiKj1ZB+YKG1tl/Le7vOGNPjur0t37Rp\nE5s2bQJg1apV+P3+80XtUedlORz53f/Dc/wocZmXDOg2hktcXNyAH9dwUq7+i9VsytU/sZoLhjfb\ngEojLS2NlpYWPB4PLS0tpKamApEZRHNzc3S9QCCA1+s9a3lzczNer/e0MT6fj1AoREdHB8nJyXi9\nXl599dXTbmv27Nk95ikqKqKoqCh6eaCfPGlTI7OnwCsv4UpKG9BtDJdY/URN5eq/WM2mXP0Tq7kg\nBj/lNj8/n5qaGgBqamooKCiILq+trSUYDNLU1MT+/fvJzs7G4/GQkJDAnj17sNayZcsW8vPzAZg3\nbx6bN28GYOvWrcyZMwdjDLm5ubz00ku0t7fT3t7OSy+9FH0l1rDJvADcbvjbO8N7PyIio9R5ZxqP\nPvoor776Km1tbXzxi1/ktttu46abbqKsrIzq6mr8fj+lpaUATJs2jQULFlBaWorL5WLZsmW4XJFe\nWr58OevWraOrq4vc3Fzy8vIAWLJkCRUVFdxzzz0kJydTUlICQHJyMrfccgtf//rXAbj11lvPOiA/\n1ExcPO6s6YRUGiIiPTL2XAccRqnBfAlT3JM/4PjunbhX/Z8hTDR4sToVVq7+i9VsytU/sZoLYnD3\n1FgWf9FMaG7CHh2+94SIiIxWKo0zxGVfHjnzToOzQUREYpBK4wzxl1wKgFVpiIicRaVxBldyKqRn\nYt9WaYiInEml0QNz0UztnhIR6YFKoycXZkcOhre1Op1ERCSmqDR6YC7KjpzRbENE5DQqjZ5MvwSM\nwb71htNJRERiikqjByYhEaZeiG149fwri4iMIyqNXpjs2fDm69hQyOkoIiIxQ6XRm5mz4XgnvKcv\nZRIROUml0Qszcw4A9g3tohIROUml0Qvj8YF/CvaNV5yOIiISM1Qa52BmzoY3XsWGw05HERGJCSqN\nc7l0LrQfgb1vO51ERCQmqDTOwcyJfFGUfWWHw0lERGKDSuMczGQvXHAxdpdKQ0QEVBrnZXKuhDd3\nYzs7nI4iIuI4lcZ5mDnzIBSC1152OoqIiONUGudzyaWQkIh96QWnk4iIOE6lcR4mLh5zxd9hX9yK\n7e52Oo6IiKNUGn1g8q+BjnZ47SWno4iIOEql0Rez8yK7qLY973QSERFHqTT6wMTHY664KrKLKhh0\nOo6IiGNUGn1kriqEjnZs/V+djiIi4hiVRl/NzgVfBnbL751OIiLiGJVGHxmXC3PtDfDay9iD+5yO\nIyLiCJVGP5hrrge3G7v5d05HERFxhEqjH0yaB5N/DfZPz2HbjjgdR0RkxKk0+snc+Ck4fgz7h187\nHUVEZMSpNPrJZE2HKxdgq/8Te7TN6TgiIiNKpTEAro9/Go51Yv/j505HEREZUSqNATAXXIxZdAP2\nj/+J/du7TscRERkxKo0BMp/4LExKIPzTddhwyOk4IiIjIm4wg3/zm99QXV2NMYZp06axYsUKurq6\nKCsr49ChQ6Snp7Ny5UqSk5MB2LhxI9XV1bhcLoqLi8nNzQWgsbGRyspKurq6yMvLo7i4GGMMwWCQ\niooKGhsbSUlJoaSkhIyMjME/6iFgUlIxn74T+3/LsM9txHz0VqcjiYgMuwHPNAKBAL/73e9YtWoV\nq1evJhwOU1tbS1VVFTk5OZSXl5OTk0NVVRUAe/fupba2ljVr1vDAAw+wfv16wuEwAI8//jh33XUX\n5eXlHDhwgPr6egCqq6tJSkpi7dq1LF26lA0bNgzBQx46Zv5izLyrsb/agG3Y7XQcEZFhN6jdU+Fw\nmK6uLkKhEF1dXXg8Hurq6igsLASgsLCQuro6AOrq6li4cCHx8fFkZGSQmZlJQ0MDLS0tdHZ2MmvW\nLIwxLFq0KDpm27ZtLF68GID58+eza9curLWDiTykjDGYz30JfBmE1z2MbW5yOpKIyLAacGl4vV4+\n/vGPc/fdd3PnnXeSmJjIFVdcQWtrKx6PBwCPx8ORI5E3wQUCAXw+32njA4HAWct9Ph+BQOCsMW63\nm8TERNraYutlriYpGdeXvwndQcI/+FdsW6vTkUREhs2Aj2m0t7dTV1dHZWUliYmJrFmzhi1btvS6\nfm8zhHPNHHq6zhhz1rJNmzaxadMmAFatWoXf7z9f/F7FxcX1f7zfT9c3vkfLt0txlX8Lz7/+AFfq\n5AFnGLJcI0C5+i9WsylX/8RqLhjebAMujZ07d5KRkUFqaioAV111FXv27CEtLY2WlhY8Hg8tLS3R\n630+H83NzdHxgUAAr9d71vLm5ma8Xu9pY3w+H6FQiI6OjuhB9VMVFRVRVFQUvXz48OGBPiz8fv/A\nxmdOx7XiAborH+LQfctxlTyISc8ccI4hyzXMlKv/YjWbcvVPrOaCgWXLysrq03oD3j3l9/t54403\nOH78ONZadu7cydSpU8nPz6empgaAmpoaCgoKAMjPz6e2tpZgMEhTUxP79+8nOzsbj8dDQkICe/bs\nwVrLli1byM/PB2DevHls3rwZgK1btzJnzpweZxqxwszJw1X6LWg/Qvi792HfaXA6kojIkBrwTGPm\nzJnMnz+fr33ta7jdbi666CKKioo4duwYZWVlVFdX4/f7KS0tBWDatGksWLCA0tJSXC4Xy5Ytw+WK\ndNby5ctZt24dXV1d5ObmkpeXB8CSJUuoqKjgnnvuITk5mZKSkiF4yMPLZM/Gdf/3CP/gQcKPfAPX\nnfdh5hY4HUtEZEgYG0svRxoi+/YN/PsuhmrKad8PEF77bXjvLcz/Wo7ruo8N6vZidSqsXP0Xq9mU\nq39iNRfE6O4pOTcz2Yvrq9+FufnYn/+I8L//SO8cF5FRT6UxjMzESbhWfB1z/Sew1b8h/MNHsMGg\n07FERAZMpTHMjMuN67ZlmNuWwY5awpXfwQa7nI4lIjIgKo0R4rr+E5jPfxleeZHwD7+H7e52OpKI\nSL+pNEaQ69obMJ/5Irz0AvanlTH1kSgiIn0xqE+5lf5zfeRGwkfex/7m5zDlAsxHb3E6kohIn2mm\n4QDz97djCq7FbvwJdvdLTscREekzlYYDjDGR4xtTphJevwZ7pMXpSCIifaLScIiZlIDrrq9Cx1HC\nT/2bjm+IyKig0nCQueAizN/fDvVbYUet03FERM5LpeEwc/1NMP0Swj/7IbbjqNNxRETOSaXhMON2\n4/r8l6CtFfvbXzodR0TknFQaMcBcmI1ZsAT7h19jm/Y7HUdEpFcqjRhhPvlZcLmxVT91OoqISK9U\nGjHCTPZhlnwMu+3P2H3vOh1HRKRHKo0YYm74JEyYhP3NL5yOIiLSI5VGDDEpqZglSyOzjQN/czqO\niMhZVBoxxhR9HNxx2E2/cjqKiMhZVBoxxqR6MPMXY2ursW1HnI4jInIalUYMMtd/AoJd2JrfOR1F\nROQ0Ko0YZLKmw+w87J+e0/eKi0hMUWnEKFfh/4DAYdi5w+koIiJRKo1YNffvIM1DWLuoRCSGqDRi\nlImLw1xzPezajg0ccjqOiAig0ohp5uoisBb71xqno4iIACqNmGbSMyH7cuxf/qgvaRKRmKDSiHFm\nwUdg/3t0N77udBQREZVGrDPzroG4eDr/qAPiIuI8lUaMM0nJkDOP47V/1Hs2RMRxKo1RwORfQ7jl\nMDTsdjqKiIxzKo1RwMwtgAkTsNv+7HQUERnnVBqjgJmUwMR5C7Hba7WLSkQcpdIYJSYtXAJH3tcu\nKhFxlEpjlJhw5YLI92y8VOd0FBEZx1Qao4QrMQku/TD25RecjiIi41jcYAYfPXqUxx57jPfeew9j\nDHfffTdZWVmUlZVx6NAh0tPTWblyJcnJyQBs3LiR6upqXC4XxcXF5ObmAtDY2EhlZSVdXV3k5eVR\nXFyMMYZgMEhFRQWNjY2kpKRQUlJCRkbG4B/1KGXm/h325z/CHtyHmZLldBwRGYcGNdN44oknyM3N\n5dFHH+WRRx5h6tSpVFVVkZOTQ3l5OTk5OVRVVQGwd+9eamtrWbNmDQ888ADr168nHA4D8Pjjj3PX\nXXdRXl7OgQMHqK+vB6C6upqkpCTWrl3L0qVL2bBhwyAf7uhmrigAwL6k2YaIOGPApdHR0cHu3btZ\nsmQJAHFxcSQlJVFXV0dhYSEAhYWF1NVF9sHX1dWxcOFC4uPjycjIIDMzk4aGBlpaWujs7GTWrFkY\nY1i0aFF0zLZt21i8eDEA8+fPZ9euXeP6M5iMfwpMvRD7so5riIgzBrx7qqmpidTUVNatW8c777zD\njBkzuOOOO2htbcXj8QDg8Xg4ciTyPdeBQICZM2dGx3u9XgKBAG63G5/PF13u8/kIBALRMSevc7vd\nJCYm0tbWRmpq6kBjj3pmbgH2uWexR9sj7xYXERlBAy6NUCjEW2+9xRe+8AVmzpzJE088Ed0V1ZPe\nZgjnmjn0dJ0x5qxlmzZtYtOmTQCsWrUKv99/vvi9iouLG9T44XIyV9ei62n53TMkv7uHhGtvcDpW\nzG+vWBSr2ZSrf2I1FwxvtgGXhs/nw+fzRWcP8+fPp6qqirS0NFpaWvB4PLS0tERnBT6fj+bm5uj4\nQCCA1+s9a3lzczNer/e0MT6fj1AoREdHR/Sg+qmKioooKiqKXj58+PBAHxZ+v39Q44fLyVzWmwEp\nabT9uZqjl1/pdKyY316xKFazKVf/xGouGFi2rKy+vbhmwMc0Jk+ejM/nY9++fQDs3LmTCy64gPz8\nfGpqIl8aVFNTQ0FB5OBtfn4+tbW1BINBmpqa2L9/P9nZ2Xg8HhISEtizZw/WWrZs2UJ+fj4A8+bN\nY/PmzQBs3bqVOXPm9DjTGE+My43Jycfu2o7t7nY6joiMM4N6ye0XvvAFysvL6e7uJiMjgxUrVmCt\npaysjOrqavx+P6WlpQBMmzaNBQsWUFpaisvlYtmyZbhckc5avnw569ato6uri9zcXPLy8gBYsmQJ\nFRUV3HPPPSQnJ1NSUjLIhzs2mCsKsLV/gDdfg0s/7HQcERlHjB2DL0c6OfsZiFidcp6ay3YcJbzy\nHzD/81Zcn/xszOSKJbGaC2I3m3L1T6zmghjdPSXOMYlJcPEs7O56p6OIyDij0hilzOxcePsN7NE2\np6OIyDii0hilzOxcsBZee9npKCIyjqg0RquLZkFCIvZV7aISkZGj0hilTFwcXJqj0hCREaXSGMXM\n7Fw4fBDbtN/pKCIyTqg0RjFzeeSj5TXbEJGRotIYzaZkgTddL70VkRGj0hjFjDGRXVS7X8aGQk7H\nEZFxQKUx2s3Ohc6j8PYbTicRkXFApTHKmcuuAMDq/RoiMgJUGqOcSUmFCy7Cvr7T6SgiMg6oNMYA\nc9lcaNiNDQadjiIiY5xKYwwwl82FYBc0vu50FBEZ41QaY8HMOWBcOq4hIsNOpTEGmMQkuChbpSEi\nw06lMUaYS3Pgrdexx485HUVExjCVxhhhLpsLoRC88arTUURkDFNpjBXZl4M7TruoRGRYqTTGCDNx\nEsyYpfdriMiwUmmMIeayufDOm9iOdqejiMgYpdIYQ8ylc8GGYc8rTkcRkTFKpTGWzLgU4ifouIaI\nDBuVxhhi4uMh+3KVhogMG5XGGGMumwt/ewfb1up0FBEZg1QaY4y5bG7kjF5FJSLDQKUx1lyYDZMS\ntItKRIaFSmOMMW43zJyDfU0zDREZeiqNMchcNhcO/g3b0ux0FBEZY1QaY9DJ4xr2de2iEpGhpdIY\niy64CJJSQMc1RGSIqTTGIONywaU52N0vY611Oo6IjCEqjTHKXJYDgUNw+KDTUURkDFFpjFHR4xra\nRSUiQ0ilMVZlXgBpHh3XEJEhFTfYGwiHw9x///14vV7uv/9+2tvbKSsr49ChQ6Snp7Ny5UqSk5MB\n2LhxI9XV1bhcLoqLi8nNzQWgsbGRyspKurq6yMvLo7i4GGMMwWCQiooKGhsbSUlJoaSkhIyMjMFG\nHheMMZhL52JfjxzXMMY4HUlExoBBzzR++9vfMnXq1OjlqqoqcnJyKC8vJycnh6qqKgD27t1LbW0t\na9as4YEHHmD9+vWEw2EAHn/8ce666y7Ky8s5cOAA9fX1AFRXV5OUlMTatWtZunQpGzZsGGzc8eWy\nHGhtgQN7nU4iImPEoEqjubmZHTt2cN1110WX1dXVUVhYCEBhYSF1dXXR5QsXLiQ+Pp6MjAwyMzNp\naGigpaWFzs5OZs2ahTGGRYsWRcds27aNxYsXAzB//nx27dqlVwP1g45riMhQG9TuqR//+Md89rOf\npbOzM7qstbUVj8cDgMfj4ciRIwAEAgFmzpwZXc/r9RIIBHC73fh8vuhyn89HIBCIjjl5ndvtJjEx\nkba2NlJTU0/LsWnTJjZt2gTAqlWr8Pv9A35McXFxgxo/XAaSy/p8HE7PJL7xdSZ/6h9jJtdIiNVc\nELvZlKt/YjUXDG+2AZfG9u3bSUtLY8aMGbzyyvm/Ka63GcK5Zg49XdfTvvmioiKKioqilw8fPnze\nPL3x+/2DGj9cBprLzprD8foXONTUFHn/RozkGm6xmgtiN5ty9U+s5oKBZcvKyurTegMujddff51t\n27bx4osv0tXVRWdnJ+Xl5aSlpdHS0oLH46GlpSU6K/D5fDQ3f/BZSIFAAK/Xe9by5uZmvF7vaWN8\nPh+hUIiOjo7oQXXpo8vmwvN/gL1vw/QZTqcRkVFuwH96fuYzn+Gxxx6jsrKSkpISPvzhD3PvvfeS\nn59PTU0NADU1NRQUFACQn59PbW0twWCQpqYm9u/fT3Z2Nh6Ph4SEBPbs2YO1li1btpCfnw/AvHnz\n2Lx5MwBbt25lzpw5ehVQP5lLdVxDRIbOoF9ye6abbrqJsrIyqqur8fv9lJaWAjBt2jQWLFhAaWkp\nLpeLZcuW4Tqxu2T58uWsW7eOrq4ucnNzycvLA2DJkiVUVFRwzz33kJycTElJyVDHHfOMxwdTpkZK\n44abnI4jIqOcsWPw5Uj79u0b8NhY3U85mFzhn67Dbq3B9egGTNzQ/p0wFrfXcIvVbMrVP7GaC4b3\nmIbeET4OmMvmwvFOeKfB6SgiMsqpNMaDS3MAsLtfcjiIiIx2Ko1xwKSkwUUzsbu2Ox1FREY5lcY4\nYeYWQOPr2LZWp6OIyCim0hgnzNwCsBa7U7MNERk4lcZ4MX0GTPbCy3VOJxGRUUylMU4YYzA5+dhX\nX8R2B52OIyKjlEpjHDFzC6CzA9541ekoIjJKqTTGk8uvgLh47MvbnE4iIqOUSmMcMRMnwWVzsTqu\nISIDpNIYZ8zcAmjahz3wN6ejiMgopNIYZ8zcyCcI25dfcDiJiIxGKo1xxvgyYPoM7PZap6OIyCik\n0hiHTP41kXeHNzc5HUVERhmVxjhk5l0NgN3+vMNJRGS0UWmMQybjQ3BhNrbuz05HEZFRRqUxTpn8\nq+HtN7AHB/6FVSIy/qg0xilz1WIwLuxfqp2OIiKjiEpjnDIeH8y+AvuXamw47HQcERklVBrjmFl4\nHQQOw+s7nY4iIqOESmMcM7lXQUIS9k//5XQUERklVBrjmJkwEXP1ddgdtdj3A07HEZFRQKUxzpnF\nN0IopNmGiPSJSmOcM1Oy4MNXYmt+ry9nEpHzUmkIriUfh9YA9q81TkcRkRin0hD48JWRDzH87TPY\ncMjpNCISw1QagjEG1423Rb5nQx8tIiLnoNKQiLz5kDUd++t/x3Z3O51GRGKUSkMAMC4Xrlv+MTLb\nqPm903FEJEapNOQDOfmR7xD/zb9jj7Y7nUZEYpBKQ6KMMbhuWwYdR7HPPOF0HBGJQSoNOY2ZdjHm\nhk9i//zf2N0vOR1HRGKMSkPOYj7+acj4EOEfl2s3lYicRqUhZzETJuJa/k/QGiD8ZDnWWqcjiUiM\niBvowMOHD1NZWcn777+PMYaioiJuvPFG2tvbKSsr49ChQ6Snp7Ny5UqSk5MB2LhxI9XV1bhcLoqL\ni8nNzQWgsbGRyspKurq6yMvLo7i4GGMMwWCQiooKGhsbSUlJoaSkhIyMjKF55HJO5uJZmE9+HvvM\nE9jfP4v56C1ORxKRGDDgmYbb7eZzn/scZWVlPPTQQzz33HPs3buXqqoqcnJyKC8vJycnh6qqKgD2\n7t1LbW0ta9as4YEHHmD9+vWET3z5z+OPP85dd91FeXk5Bw4coL6+HoDq6mqSkpJYu3YtS5cuZcOG\nDUPwkKWvzA03YQquxW78CbZ+q9NxRCQGDLg0PB4PM2bMACAhIYGpU6cSCASoq6ujsLAQgMLCQurq\n6gCoq6tj4cKFxMfHk5GRQWZmJg0NDbS0tNDZ2cmsWbMwxrBo0aLomG3btrF48WIA5s+fz65du7Sr\nZAQZYzB33AsXZhP+0fd1YFxEhuaYRlNTE2+99RbZ2dm0trbi8XiASLEcOXIEgEAggM/ni47xer0E\nAoGzlvt8PgKBwFlj3G43iYmJtLW1DUVk6SMzYSKue/8F0jMJV3wH2/Cq05FExEEDPqZx0rFjx1i9\nejV33HEHiYmJva7X2wzhXDOHnq4zxpy1bNOmTWzatAmAVatW4ff7zxe7V3FxcYMaP1wczeX3E/pO\nJS3f/BLh8m8x+cFy4mfNdj7XOcRqLojdbMrVP7GaC4Y326BKo7u7m9WrV3Pttddy1VVXAZCWlkZL\nSwsej4eWlhZSU1OByAyiubk5OjYQCOD1es9a3tzcjNfrPW2Mz+cjFArR0dERPah+qqKiIoqKiqKX\nDx8+PODH5Pf7BzV+uMRCLvu/H8Q+8nUCD96L60vfxFz64ZjI1ZNYzQWxm025+idWc8HAsmVlZfVp\nvQHvnrLW8thjjzF16lQ+9rGPRZfn5+dTUxP5XoaamhoKCgqiy2trawkGgzQ1NbF//36ys7PxeDwk\nJCSwZ88erLVs2bKF/Px8AObNm8fmzZsB2Lp1K3PmzOlxpiEjw3j9uO57GCb7CD/6L9gXdXBcZLwZ\n8Ezj9ddfZ8uWLUyfPp377rsPgNtvv52bbrqJsrIyqqur8fv9lJaWAjBt2jQWLFhAaWkpLpeLZcuW\n4XJFOmv58uWsW7eOrq4ucnNzycvLA2DJkiVUVFRwzz33kJycTElJyWAfrwyS8abj+up3CZd/i/C/\nraLDhuDKq52OJSIjxNgx+HKkffv2DXhsrE45Yy2XPX6M8GOrYNeOyEtzb/lHjMvtdKyoWNtep4rV\nbMrVP7GaC2J095SMb2biJFxf+iYJH70F+19VhCsewnZ2OB1LRIaZSkMGzMTFkXrnP2H+4Yvwyg7C\n370P+7d3nY4lIsNIpSGD5lp8I66Sf4X2I4QfLiX8p//SmzBFxiiVhgwJc/kVuP6lHC65HPuTCuzj\n38ceaXE6logMMZWGDBmT5sFV8iDmps9id/yF8DdXEP7jf2LDIaejicgQUWnIkDIuN66lt0VmHRdl\nY3/2Q8LfKcVu+zM2pPIQGe0G/TEiIj0xH7oA18pvRcqi6qeEf/g98GVgrvs4Jv8ajMd3/hsRkZij\n0pBhY4yJfLT6vIVQ/wLh/67CPr0e+/R6uDAbc8XfYWZcCtNnYFLSnI4rIn2g0pBhZ1xuuHIB7isX\nYPe9i63/a+T0658RfY3VZB+kT8F408GbHpmVTPZBSiqkpEVOEyfpY2REHKbSkBFlsqZjsqbDjZ/C\ndrTDu43Yd9+E997GBpqwDbvh/T9DKMRZL9qNn/BBgaSkYU4tlJS0yGwlJQ2SU7HJSU48PJExT6Uh\njjGJyXDZXMxlc09bbsMhaH0f3m+GtlZs2xFoex9O/IxcbsXuexfaWiHYFRl3ym00AUyYAIkpkJAY\nPZlJiZCYBJMSICEJEhJgYgJMmIiZOBEmTIT4Ez9PO02A+AkYl147IuObSkNijnG5weOLnIBz7ZCy\n1sLxY5HyaGuFtiPY9laSQkGOHtgHHUcjH29yrCNyvvlQ5HxnR2TcqbfVl3DxE04vk/gJMHEixMVD\nfDzExWNOOc+Z5+PiOTp5MuHjxyNjT64fFw/xcaetFx138qc7DuLiwO2Oqc/5kvFFpSGjmjEmMmuY\nlADpmZFlQJLfT+d5PrDNhkKRuKCIAAAIt0lEQVRwrDNyCh6HruPQ1XXi53HsiZ8ETyw7fjx63cll\ntut4pHy6g3A0stx2ByEYhO7uyPLuYGT9cBiA9jNzDOiBu6IFEvkZd0qpnDydct0py00v49qSUwgH\ng5Hr3PEfrHPabfY03g0u94nzrsh5l/vE7fRy3uUGl0vHqEYhlYaMW8bthqTkyKmn64f4/mw4BMFu\nfGkpNB88eKJMgtDdBcFTCubE8g/K58Qp1B0polDoxM9TTt0fXGdDwbPXOdZ5YlkwUpY9jOsInchw\nvscxlBvlzCLpoWQOT5hAyJ5c13XiurgPzp9Y3/Qy/rTzLveJYnNFivdEeUVOp54/ZZ0T92uiYyKn\n45M92Pb2s9c/7T56uM0z7+fkbbrd0fOxvBtUpSEyQozLDRPduJJTMce6zr/+CGQ6ld/v59ChQ5EZ\nUbSgunsvq5Pnw6HIdaEQhEMnSil01vLI5fApY8InfnZ/cP7MMaEQcXFuQp2dp99mOARdwdOW2ZO5\nQ6HIYzjztk/erg0PaPucWZbvD36Tn1tvRXNW2X1QPGbaxbjuvG9YY6k0RCTKGPPBX+YTJg7sNoY4\n0+Qh/t4Ka+2JUjlxsqFTLodOvy66TviDMrKRZWkpKbS2tPQ87pTbtL3drg334X5PluvJ9c9cLwSn\nPp4Tu2iHk0pDRMaV04pxECb4/Zg+lNlYO2oTuzvOREQk5qg0RESkz1QaIiLSZyoNERHpM5WGiIj0\nmUpDRET6TKUhIiJ9ptIQEZE+M9baIf0oGRERGbs00zjD/fff73SEHilX/8RqLojdbMrVP7GaC4Y3\nm0pDRET6TKUhIiJ95n7wwQcfdDpErJkxY4bTEXqkXP0Tq7kgdrMpV//Eai4Yvmw6EC4iIn2m3VMi\nItJn+j6NE+rr63niiScIh8Ncd9113HTTTY7kOHz4MJWVlbz//vsYYygqKuLGG2/k6aef5g9/+AOp\nqakA3H777Vx55ZUjnu9LX/oSkyZNwuVy4Xa7WbVqFe3t7ZSVlXHo0CHS09NZuXIlyck9f4XqcNi3\nbx9lZWXRy01NTdx2220cPXp0xLfZunXr2LFjB2lpaaxevRrgnNtn48aNVFdX43K5KC4uJjc3d8Ry\nPfXUU2zfvp24uDimTJnCihUrSEpKoqmpiZUrV5KVlQXAzJkzufPOO4clV2/ZzvV8d3KblZWVsW/f\nPgA6OjpITEzkkUceGdFt1tvviBF7nlmxoVDIfvnLX7YHDhywwWDQfuUrX7HvvfeeI1kCgYB98803\nrbXWdnR02Hvvvde+99579he/+IX91a9+5UimU61YscK2traetuypp56yGzdutNZau3HjRvvUU085\nEc1aG/m3XL58uW1qanJkm73yyiv2zTfftKWlpdFlvW2f9957z37lK1+xXV1d9uDBg/bLX/6yDYVC\nI5arvr7ednd3RzOezHXw4MHT1htuPWXr7d/O6W12qieffNL+8pe/tNaO7Dbr7XfESD3PtHsKaGho\nIDMzkylTphAXF8fChQupq6tzJIvH44kewEpISGDq1KkEAgFHsvRVXV0dhYWFABQWFjq27QB27txJ\nZmYm6enpjtz/7Nmzz5pl9bZ96urqWLhwIfHx8WRkZJCZmUlDQ8OI5briiitwn/j2ulmzZjn2POsp\nW2+c3mYnWWv5y1/+wtVXXz0s930uvf2OGKnnmXZPAYFAAJ/PF73s8/l44403HEwU0dTUxFtvvUV2\ndjavvfYazz33HFu2bGHGjBl8/vOfH9FdQKd66KGHALj++uspKiqitbUVj8cDRJ7QR44ccSQXwPPP\nP3/af+RY2Ga9bZ9AIMDMmTOj63m9Xsd+cVdXV7Nw4cLo5aamJr761a+SkJDApz/9aS6//PIRz9TT\nv12sbLPdu3eTlpbGhz70oegyJ7bZqb8jRup5ptLgxBfNn8EYZ7/Z99ixY6xevZo77riDxMREbrjh\nBm699VYAfvGLX/CTn/yEFStWjHiub3/723i9XlpbW/nOd74T3YcbC7q7u9m+fTuf+cxnAGJmm/Wm\np+edE5599lncbjfXXnstEPmFs27dOlJSUmhsbOSRRx5h9erVJCYmjlim3v7tYmWbnfnHiRPb7Mzf\nEb0Z6m2m3VNEZhbNzc3Ry83NzdHGdkJ3dzerV6/m2muv5aqrrgJg8uTJuFwuXC4X1113HW+++aYj\n2bxeLwBpaWkUFBTQ0NBAWloaLS0tALS0tEQPXo60F198kYsvvpjJkycDsbPNets+Zz7vAoFAdPuO\nlM2bN7N9+3buvffe6B9K8fHxpKSkAJHX+k+ZMoX9+/ePaK7e/u1iYZuFQiFeeOGF02ZmI73Nevod\nMVLPM5UGcMkll7B//36ampro7u6mtraW/Px8R7JYa3nssceYOnUqH/vYx6LLTz4ZAF544QWmTZs2\n4tmOHTtGZ2dn9PzLL7/M9OnTyc/Pp6amBoCamhoKCgpGPBuc/ddfLGwzoNftk5+fT21tLcFgkKam\nJvbv3092dvaI5aqvr+dXv/oVX/va15g4cWJ0+ZEjRwiHwwAcPHiQ/fv3M2XKlBHLBb3/2zm9zSBy\n3CwrK+u0Xdojuc16+x0xUs8zvbnvhB07dvDkk08SDof5yEc+ws033+xIjtdee41//ud/Zvr06dG/\n/G6//Xaef/553n77bYwxpKenc+edd474bOjgwYN8//vfByJ/bV1zzTXcfPPNtLW1UVZWxuHDh/H7\n/ZSWlo74sYPjx49z9913U1FREZ2qr127dsS32aOPPsqrr75KW1sbaWlp3HbbbRQUFPS6fZ599ln+\n+Mc/4nK5uOOOO8jLyxuxXBs3bqS7uzua5eTLRLdu3crTTz+N2+3G5XLxqU99alj/iOop2yuvvNLr\nv52T22zJkiVUVlYyc+ZMbrjhhui6I7nNevsdMXPmzBF5nqk0RESkz7R7SkRE+kylISIifabSEBGR\nPlNpiIhIn6k0RESkz1QaIiLSZyoNERHpM5WGiIj02f8HTMy5BylMm1sAAAAASUVORK5CYII=\n", + "text/plain": [ + "\u003cFigure size 600x400 with 1 Axes\u003e" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "plt.plot(losses)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "TqTuhkTsP90b" + }, + "source": [ + "We can use the model to sample data for the inferred values for $\\mathbf{W}$ and $\\mathbf{Z}$, and compare to the actual dataset we conditioned on." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "height": 337 + }, + "colab_type": "code", + "id": "T3O6PHe3XX8a", + "outputId": "4443cdf0-42e7-47d8-bc00-000067292a36" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "MAP-estimated axes:\n", + "\u003ctf.Variable 'Variable:0' shape=(2, 1) dtype=float32, numpy=\n", + "array([[ 2.9135954],\n", + " [-1.4826864]], dtype=float32)\u003e\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt0XHW9///nvswtmWQyk8k9aUt6\no8VigRZqBQs2cLi4CiIgfHVxBA8sBHVRRAURdK0KBGnF46XHG3iU4zmWumjxskSNlaK/Ai1CAVta\nem+a+2QmmZlkbvvy+yPttKXXNDNN0nk/1spqsmfP/rwznbz2ns/+7M9WbNu2EUIIUVDU0S5ACCHE\n6SfhL4QQBUjCXwghCpCEvxBCFCAJfyGEKEAS/kIIUYD0kW4gFArxwx/+kL6+PhRFoampiauvvpp4\nPM5TTz1FT08PFRUVLF68GK/Xm4uahRBCjJAy0nH+kUiESCRCY2MjiUSCBx54gC9/+cu89NJLeL1e\nrrvuOlavXk08HufTn/50ruoWQggxAiPu9vH7/TQ2NgLg8Xioq6sjHA6zYcMGFixYAMCCBQvYsGHD\nSJsSQgiRIyPu9jlUd3c3u3btYsqUKfT39+P3+4GhHUQ0Gj3qc1paWmhpaQGgubk5l+UIIYQ4hpyF\nfzKZZNmyZXzmM5+hqKjopJ/X1NREU1NT9uf29vZclZQ3wWCQUCg02mWckNSZW1Jn7oyHGmH81Flb\nWzvs5+RktI9hGCxbtoxLLrmEiy66CACfz0ckEgGGzguUlpbmoikhhBA5MOLwt22bH/3oR9TV1fGx\nj30su3zOnDmsXbsWgLVr1zJ37tyRNiWEECJHRtzts3XrVl5++WUmTJjAl7/8ZQBuueUWrrvuOp56\n6inWrFlDMBjkvvvuG3GxQgghcmPEQz1zTfr8c0fqzK1c1WnbNslkEsuyUBQlB5UdzuVykUqlcr7d\nXBoPNcLYqtO2bVRVxe12H/G+OZU+/5yO9hFCnFgymcThcKDr+fnz03UdTdPysu1cGQ81wtir0zAM\nkskkHo9nxNuS6R2EOM0sy8pb8Iszm67rWJaVk21J+AtxmuWjq0cUjly9fyT8hRCiAEn4C1Gg/vjH\nP1JXV8f27dtPuO6KFSvo7Ow85bbWrVvHrbfeesL1brjhBt56663jrvPTn/6URCJxyrWIIRL+QhSo\n1atXc+GFF/LCCy+ccN2VK1fS1dV1Gqo6sZ/97GcS/jkg4S/EGGeaEAopdHaqhEIKpjnybQ4MDPD6\n66+zdOnSI8J/+fLlLFy4kKamJh577DF+//vf89Zbb/H5z3+eyy+/nEQiwUUXXUQ4HAbgrbfe4oYb\nbgDgzTffZNGiRVxxxRUsWrTohJ8qEokEn/vc52hqauKuu+4imUxmH3vggQe46qqruOyyy1i6dCkA\nTz/9NF1dXdx4443ZNo+2njgxGXIgxBhmmtDerqLroCiQySi0tyvU1lqMZATiiy++yKWXXsrkyZMp\nKyvjnXfeYdasWaxZs4YXX3yR3//+93g8HiKRCH6/n//+7//m4Ycf5oMf/OBxtztlyhSef/55dF3n\n5Zdf5oknnuCnP/3pMdf/5S9/icfjoaWlhc2bN3PllVdmH/vqV7+K3+/HNE0++clPsnnzZj772c/y\nk5/8hJUrVxIIBI653syZM0/9xSkQEv5CjGGRiJINfhj6V9eHlgeDp3595urVq7njjjsAuPbaa1m9\nejWzZs3i73//O5/85Cez48gPzMx7sqLRKPfeey+7du1CURQymcxx13/ttde4/fbbAZg5cyYzZszI\nPva73/2OX/3qV5imSVdXF9u2bTtqqJ/seuJwEv5CjGGGofD+kX2KMrQcTi38w+Ew69atY+vWrSiK\ngmmaKIrC17/+dWzbPqmhhIeONz/0Ctgnn3yS+fPn8/TTT9Pa2prtmjmeo7W3d+9efvzjH/OHP/yB\nsrIy7r333sO6hIa7njiS9PkLMYbpus37J2Cx7aHlp+oPf/gDn/jEJ1i/fj2vvfYar7/+OhMmTGD9\n+vUsWLCAX//619kTqgdm5i0uLiYej2e3UV9fz9tvv53d3gGxWIzq6moAnnvuuRPWctFFF7Fq1SoA\ntmzZwrvvvpvdjsfjobS0lJ6eHv72t79ln+P1erO1HG89cXwS/kKMYX6/jWGQ3QHYNhjG0PJT9cIL\nL3DVVVcdtuzqq69m1apVXHbZZVxxxRVcddVVXH755fzoRz8C4KabbuKBBx7InvC97777eOSRR/j4\nxz9+2PQHn/vc53j88ce59tprMU/izPStt97KwMAATU1NLF++nNmzZwNwzjnn8IEPfIDLLruM++67\n77BZgT/1qU/x6U9/mhtuuOG464njk4ndTkGhTUSWb4VW5+Dg4LBueGSaQ338hqGg6zZ+v33ck726\nrmMYxojrzKfxUCOMzTqP9v6Rid2EOANpGvtP7o6p4zQxzkm3jxBCFCAJfyGEKEAS/kIIUYAk/IUQ\nogDl5ITv8uXLeeONN/D5fCxbtgwYGuP717/+ldLSUmDovr7nn39+LpoTQggxQjkJ/0svvZQrr7yS\nH/7wh4ctv+aaa1i0aFEumhBCCJFDOen2mTlzJl6vNxebEkKcBv/5n//JZZddRlNTE5dffjlvvPEG\nAPfffz/vvfdeTto4dObPY/ne97437O2uWLGChx56aNTaB7jjjjvYs2dPtp2Pf/zjhz1++eWX89GP\nfvSwZY888ggXXHDBYbdhXLFiBbNmzeLyyy/n0ksv5Ve/+hUAf/nLX/I+Q2le+/z/9Kc/cf/997N8\n+fLDLg0XQgyDaaKEQqidnSihECOd0/n111+npaWFF198kZaWFlasWJG9SGjp0qVMmzYtF1WflO9/\n//unra1ctb9161Ysy2LixInZZfF4nLa2NgC2bdt2xHMsy+KPf/wjNTU1vPrqq4c9tmjRIv7yl7/w\nm9/8hubmZnp6emhqauLPf/5zXu9bkLeLvK644orspE4rVqzgl7/8JXffffcR67W0tNDS0gJAc3Mz\nwWAwXyXljK7rUmcOFVqdXV1dJ38Dd9NE7e4emspT14fmd+juxqqt5XiX+R5v+729vZSXl1NcXAxA\nZWVl9rGPf/zjfOMb32D27NmcddZZ3H777bz88sv4fD6+9rWvsWTJEvbt28eSJUu48sor+fWvf81b\nb73F448/DgxNvXD33Xfz4Q9/GEVR0DQNXdf593//d9rb20mlUvzHf/wHt956K48//jjJZJIrrriC\n6dOn81//9V/85je/4ac//SmZTIbzzz+fJ554Ak3T+L//+z++973vUVVVRWNjIy6X64jfMRwOc9dd\nd9Hb28t5550HcNz2lyxZckT7R1vv/e2sXr2aq666KrtcURSuvfZa/vCHP3D33Xfz29/+luuvv56V\nK1dm13n55ZeZMWMG1157Lb/97W/5yEc+kq1PVVV0Xae6uppJkybR0dFBTU0NH/7wh1mzZg3XXnvt\nYe27XK6cvA/zFv5lZWXZ7xcuXMgTTzxx1PWamppoamrK/lxIl/nnm9SZW7mqM5VKHTYfzvEooRAW\nHHG0b/f0YB8jAE40JcHFF1/M0qVLmTdvHpdccgmLFi3iQx/60NB2bRvTNDEMg8HBQS666CIefPBB\nPvvZz/L444/zv//7v7z33nvce++9NDU1YZomlmVl2zv0+Yd+v3TpUvx+P4lEgmuuuYaPfexjPPjg\ngzzzzDP8+c9/BuDdd99l1apVrF69GofDwYMPPshzzz3HRz7yEb797W/z4osvUlJSwo033sgHPvCB\nI37HJ598krlz57J48WJaWlp49tlnj9n+lVdeeUT7x6rzwKCVA9avX8+iRYsO+52vuuoqFi9ezJ13\n3smf/vQnfvCDH7By5crsOs8//3z2JjePPfYYiUQCh8Nx2Ou3Z88e9uzZQ0NDA4ZhMGvWLF555RWu\nueaaw9pPpVJHvA/H1PQOB24CAUMvVkNDQ76aEuKMpRgGR5vTWTGMU57sobi4mBdffJHXXnuNdevW\n8bnPfY4HH3yQT37yk4et53Q6ueyyywA4++yzcTqdOBwOZsyYwb59+4bV5jPPPMMf//hHYGj+rp07\nd2YncTvgH//4B++88w5XX301AMlkkmAwyJtvvsmHPvQhysvLgaFukp07dx7RxquvvsrPfvYzYOig\n8tAD0Pe3v2vXruzNYIZbZ1dXV7aWA/x+Pz6fjxdeeIGpU6dm74cAkE6nWbNmDd/85jfxer2cd955\nrF27NnvQ+9vf/pb169fjcrl44oknsrkZDAbzeuvMnIT/d7/7XTZv3kwsFuOuu+7ipptuYtOmTeze\nvRtFUaioqODOO+/MRVNCFBRb11EymcN3ALaN7XCMaLuapjF//nzmz5/P2WefzcqVK48If13Xs3Pt\nq6qKy+XKfn/giPbQef3h8Ln9D1i3bh1///vf+d3vfofH4+GGG2446nq2bXPjjTfy4IMPHrb8xRdf\nPKl7DMDR7w1wsu2f7Hput/uoyxctWsTXvvY1nnrqqcOWv/TSS0SjURYuXAgM3brS4/Fkw3/RokU8\n+uijR2wvmUzidrtP6vc+FTkJ/3vvvfeIZe8/0y2EGD7b70dpbyd7O6/9czrbh/TTD9f27dtRVZXG\nxkYANm3aRH19/Sltq6GhgV/84hdYlkVHRwcbN248Yp1YLIbP58Pj8bB9+/bsyCIAh8NBJpPB4XBw\n8cUXc9ttt3HHHXcQDAaJRCIMDAxw3nnn8cgjjxAOhykpKeH3v//9Ue/UNW/ePJ5//nnuvfde1qxZ\nQ19f37DaP956h5o6dSq7du06ojfjqquuoru7m0svvfSwI/bVq1ezdOlSrrvuOmBoVs558+ad8GTu\nzp07mT59+nHXGQmZ1VOIsUzTsGprUSKRoa4eh2Mo+EdwA9/BwUG+/vWvE41G0XWdSZMm8e1vf/uU\ntjV37lwmTJjAwoULmT59OrNmzTpinUsvvZRnn32WpqYmGhsbD7vY81Of+hRNTU3MmjWLH/zgB3zl\nK1/hlltuwbZtdF3n0Ucf5YILLuBLX/oSixYtoqqqilmzZh31XgGLFy/mnnvu4d/+7d+YN28edXV1\nw2p/2bJlx1zvUAsXLuSVV17JnrQ9wOv1cs899xy2LJFIsHbt2sPOeRYVFXHhhRdmzzUcy7p16474\nFJRLMp//KSi0E5T5Vmh1Dnc+/+Eai3PQv994qBGOXmcikeDGG2/khRdeOOkT98PV09PDPffcc9S7\noeVqPn+Z20cIIYbB4/Fw//3309nZmbc22traeOSRR/K2fZBuHyFOuzH2YVucgksvvTSv23//CKND\n5er9I0f+Qpxmh46WEWI4DMNAVXMT23LkL8Rp5na7SSaTpFKpkx7COBwul+uoQxHHkvFQI4ytOm3b\nRlXVnA3/lPAX4jRTFOWwi4BybTycQB8PNcL4qfNUSLePEEIUIAl/IYQoQBL+QghRgCT8hRCiAEn4\nCyFEAZLwF0KIAiThL4QQBUjCXwghCpCEvxBCFCAJfyGEKEAS/kIIUYAk/IUQogDlZGK35cuX88Yb\nb+Dz+Vi2bBkA8Xicp556ip6eHioqKli8eDFerzcXzQkhhBihnBz5X3rppXzta187bNnq1auZNWsW\n3/ve95g1axarV6/ORVNCCCFyICfhP3PmzCOO6jds2MCCBQsAWLBgARs2bMhFU0IIIXIgb/P59/f3\n4/f7AfD7/USj0aOu19LSQktLCwDNzc0Eg8F8lZQzuq5LnTkkdebWeKhzPNQI46fOUzHqN3Npamqi\nqakp+/N4uHHCeLnBg9SZW1Jn7oyHGmH81FlbWzvs5+RttI/P5yMSiQAQiUQoLS3NV1NCCCGGKW/h\nP2fOHNauXQvA2rVrmTt3br6aEkIIMUw56fb57ne/y+bNm4nFYtx1113cdNNNXHfddTz11FOsWbOG\nYDDIfffdl4umhBBC5EBOwv/ee+896vJHHnkkF5sXQgiRY3KFrxBCFCAJfyGEKEAS/kIIUYAk/IUQ\nogBJ+AshRAGS8BdCiAIk4S+EEAVIwl8IIQqQhL8QQhQgCX8hhChAEv5CCFGAJPyFEKIASfgLIUQB\nkvAXQogCJOEvhBAFSMJfCCEKkIS/EEIUIAl/IYQoQDm5jePx3HPPPbjdblRVRdM0mpub892kEEKI\nE8h7+AN84xvfoLS09HQ0JYQQ4iRIt48QQhQgxbZtO58N3HPPPXi9XgAuv/xympqaDnu8paWFlpYW\nAJqbm0mn0/ksJyd0XccwjNEu44RyWqdpQjgMmQw4HBAIgKad/OOnq848kjpzZzzUCOOnTqfTOezn\n5D38w+EwgUCA/v5+vvWtb3Hbbbcxc+bMY67f3t6ez3JyIhgMEgqFRruME8pZnaYJ+9rpizswLRVN\ntSjzpFCK3Si2ja0oKMkkOJ2gKGDbYBhYtbUntQMouNczz8ZDneOhRhg/ddbW1g77OXnv9gkEAgD4\nfD7mzp3L9u3b892kyDGrO0TPjjh2dwg1EsZImPRuCWN3D/1RqKEQaigEljX0BEUBXUeJRI65TdOE\nUEihs1Olp2foZyHE6ZPXE77JZBLbtvF4PCSTSd5++21uuOGGfDYpcs00iW9qJxF3YNkq2AZa3w7C\npo+970bxN0KQXkp9FtrAAFYwiFHioz+mY5gWoFBabDDQ2ocZS+AOt1NU5qJroJiI7yws3YVhQE+P\nRjBok0hAcTG4XDZ+v32yPUdCiGHKa/j39/ezdOlSAEzT5OKLL2b27Nn5bFLkmBWK0NlfhAMDw4C2\nNp3UHvAae1CqK7G6wdHfi11kUDa9HCtlEN7Tg11Vie12Mxi12PrnHvx+E//ezaR0B3s2DpIqUSnz\nbiQ67Tz2RU0G2yIMbM8QqNLo8viprFFob1eorbVkByBEHuQ1/KuqqnjyySfz2YTIs2iviVruw9jX\nzY49DvpjKlpPEtO2GPD6aFD76bACGKlutK4BMs5KBpIK5u4I0YazCf1tD46BKNqrb5BOx0hbOkm7\nlIw3TrwhgG1toySYIBZLYiaiFJHAO7CJZGwSpVUeIs4yglUyKE2IXDst4/zF+JXBQbE3w5sDNaSN\nCGZvH1rKJGo7MFPQ02VRFVRJFJXTmXCQ6VBxuHWSRV46X+9G29lKZWwL2t5dxJUiit0mZQO7SLl8\nJMK1eMKtpKacjzsyQLhiKpWpGKoKVnQnvcZUBrd20TWlBkXXKC+3CAalK0iIXJDwF8ellJcx0NGJ\nz6fQuy9DxuPD9poMqKX4B/oYVBR6+hyYZX7cbieWx4ff6sPevovK1ghKbwfpcB+JQY2icCuGlcCV\njuNwu/Du2kTKUvF2tOLMOCn3rGPg/EvIlJXjHWhHCUdJuioYyDgomVbNvj2Q6QhTV5lGderYfv9J\nDyfFNFEiERTDwNaH+VwhzkAS/uJw7wtJv9/P3tI6PF07KfVrWJpGVC3HF+kiFtNxl6i4rUFKwyGS\naj3lg50YHg+u9jaKO/fgaN1LvDOKGUtRYbZRTJwEXlyJKG4yZNBI9HvQCFKiDxD7825itZPpq5lJ\nclec0mAIZ+sW3u37BBUDrYTSNjtLdYwiHw6lk/IJLqoqTPwVGmpwKNBNEyIRBcNQ0HUbf6mBo6sd\ndB0UBSWTQWlvP+mhqEKciST8xUGmidp+eEg6utqpq6sjEvfSM1hCwO7E79PYZ/sJdOygWsvgCJRT\nUleCu2MP6dJSao3tqO/9A2Mgw0BbD9WxbjSSuBjAgUExfThIYwFuHJQRIUMXccOHywij7YpT3LaT\nAXc56R1OYikHlS9vZu+ki+lwN+IssjmrugN3OkrsHQeRqUH8E4uZPrkVxeuhu0NBc+mYJX56e3U6\n3olRG3QcvO7skKGodjA42q+6EKNCwl9kKZFINviHFgyFZIUaxizXmG12sW+XysCgQkXRACUz/Hi1\nQUrqnZhlftSUjSsWprj9bVR1kIGeCEWZOB5iuIjjIIXK0JtO2f+lkgFAw8BNL2l0HNYgRtJDVzJC\nKTbVmLgSadyRfdThp9XRSJV7F65SFwMlVYT2TSVe7mUgCCWVLrTaSiguwuztQKsM4ujvZ3fUR2en\nRnWNScBnoMf6UU0TE6QLSBQkCX+RpRjGweDPLlTQbYOqsysY6N5DideJy22h2SkwTUzbA4bJ4CD4\nzAReox892ofdHx8KdzODAwOdDBpDVxUe+PdQOmADOgbFGCRJUcsgFhpekhhYXEQXCbxclvkTvZkA\neiyDhUpmiwtbLyYRqCE08TwSdVOIDur0FtXTMKUPbyqEvms9ab+PXeUBzKlOqiY4Ud0OlGQSddMm\n7GAQ2+mUHYEoGBL+IsvWdZRM5vAdgG1jOxxoTg2zro4KZy+qZWH7nIQGivAMxojFFYrKwYo5KUr3\nYZoKqmWRNhWKMzFsQGNoFpGjBf8BCnBghhIHNgYpNIZ2Ck5AxcRLPxZQwgA2kL0w2IBM9xaqut8k\ntKGSLqWBwYrz6HjVTY3STbU3jsPnIar7eHdjBdvPm07FeUGq2rvxldlou3eD14vd3o45dSrKwICc\nHBZnNAl/kWX7/SiH9PkfmKPHrqwEIFVagSORwdJ1KAsQ6OykM+HBdGvoqknANUCmrp7BziglVg+6\namJio2EwFPknP4eDtv/r/Y434t+NgZsQZYRosLfT2P0vEvgYpBi3SyWtOSlSXLR5P8j2aCPTzQR9\npTpBqwfdpRN1qJSZvZS/9nvcc89GqQig2PbBk8NCnEEk/MVBmoZVW3twtI/DMRT8+496dZdKurIW\nPRpBMQ2s+noUt0oxGbzOOIa3gcHWCI6aelLpJObgLhTFSco2sLDxMIgCWBzs98+lA58cnICbNMW0\nkaINAyfpVBEZXCTxYg0M8E7/RLZGohSV7cZ02mRqavAEUljRECk9hTOxl7q5SbQJtQfnKaqqynHF\nQoweCX9xOE0b6v8+ykN+v017uwb+YPaDQcZSKAnYpHXQwiHiag3eWBxjsB9XNMy+2DSKY73YmQQV\ndKGRwkNyf1fQ0Y/uc/Jr7P9yAZAmTRoTsOgiyA4mJrfQsaueBH76S+rw7tyMrpikVA+xhgnoGR/R\nVJSquUVolUF8ZWN/Wl8hhkPCX5w0TYPaWis7ht7hsDn7bIuuLhXbBtPnR+3oIFVcjm+6ScbMUFKc\nZsc2g8rwVrriOj7CxDAoIY6DNDoWKuAgv1PMHjrbeTHgJUI5EQxUzJgFMQcGbtK4iXWWsu+tqXSW\n1ZN6swrH/NnsaZzE2XYPif5uSssPXlMgxHgl4S+GRdMgGLThkM8GB3cIGiUzqjGVGOr2dphQA64U\nfpIkulXMaIjuzgHMtE3QbqOcHhRsihnAQwI3ds67go7Fsf9rqBMKLDL7v2KU00N1uo2e7ho6whMI\ntcZJVOxmx7yrMZwOnKrBRZPfxVPtpTTZzaDhJu0owpo0EX+VQ/YJYlyQ8BcjdvgOQcUMTiGhJqC3\nnxJPJ6VFKXq9PhKpchxnKfxrb4BoT4bzYmspp50aOtDIoJDBPUq/g8rBTx4m7L8obRcBo4vwzt10\n76ygd9ObdE25kPaiWv6/f0SYV7aViO5B9bjoc1ZguPbwr4sWMO2DDqqqZA4iMbZJ+Iuc05wa3vkz\n0bZsQWksRWn3UlbeSizlZKc9CZfLCV6T5yNf5COt/4eBC41dJEkC8VHbAWTr3//lADwM4mMPE9hD\nKL6NfRs3ksZHm3Myre4UmtOFy6sQb9CxSwzUV15js3oJ6bRCQ21m6OS4DBkVY5CEv8gPpxPznHNQ\nIhHU/dcPFBcXM2FAI56wKfZAX4+XN42ryYRKCGWqqGM309gKZNAZO2/OA7VU00+AfgZw0JB+j1Ta\nSwwf8ZAfpTdFe+1cfJkQqdJdDCgu2t8KkSotB03HX5qivHUTamU5tsslOwIx6sbK35c4Ex0YOWQY\nGD4f2o4deL0WgUqdXjvAjNIkb7qnE3k3RHF3nIQRYzeTmcY2DEws8jsiaLg0wAN4yBCklwS9JCgi\nTgnRWBfRrQrGviJ6OlS63qygPphiSsUO9Eo/MSMDAY2KWD+2P0B/6wCJygnoLlXuWCZGhYS/yDtb\n11FsG3PaNJT+fiYFTFI7NPYmqphZoxNxXUCqIwWhFOmUh63dDsrpoph+dFI4OHhl8Fh6ww7tCAYp\nYZAKumhkC+0DlXT8603caOiaQsofJF1ZjbfcQ1fdZHoaakiWQqmnDzNejOesStrbFaqqLKLRQ2Yi\nlR2CyLOx9LckzlCHXjlsBwJots3Z5QZBtw/DUIhNtzH/VU1P9yKMnf9CqzyLUGSQ9l6dicl3aWAn\nVYSwMdGx90/1MHYcHDkEjXQzke6hawpMSIXcDIZK6FLqcVduIxKcxuDEmaTKXFg7ogz8s5+A3yRc\nE6ByZjmKrpHJyC0sRf7lPfw3btzIz3/+cyzLYuHChVx33XX5blKMNUe5cpjKSoLaUIQHgxPZFu/C\nr/cQSU0hUWYSd6dwFIXZ0lGKNeCmz+6mlAhuUpQRwsXBwB1LDowaOlBbEUlKSFJm9zDQ5cXuCjHQ\nlcCwDaqKo3gnTKHTcRaWy4HjmukEJpfhUG00VZdbWIq8ymv4W5bF008/zde//nXKy8t58MEHmTNn\nDvX19flsVoxFx7lyuKJao+28KaR9xVjlOsb2buJulZrEDgZ8VewOeamLbyNp9uEYjNOb6qKWdhxk\ncJNG5fCLuMaaAyeMXcQpYiN1od3EKGGgtwzX3k7qvRvpKj8b3l1D4txJBM+pIj11Gn37Yti1RTg0\nSy4sEzmX1/Dfvn071dXVVO2fE2X+/Pls2LBBwl8cRtOgfqJCpLia0po+4g3lVO1LMNg3jRnpXqLW\nRPzxKkq6t1Oc6iPeWc2mrtkUD/Ti6e+iht0UMZA9JzBW41EHvICTPsroY5AIAxQTi5cwMb6JQdVP\nKtpG7L1yHO6XcE+cQvzij2B7S+l5x6CysgPvvOlonrG8qxPjRV7DPxwOU15env25vLycbdu2HbZO\nS0sLLS0tADQ3NxMcB3dW0nVd6swhXdepqgrunzdtaAZR04R9e0zc770NlsVA+2Qygw34yi1qvAq+\n13ay/T2LeE8Ie/ebOLv2UGT34aMXD5kx2SV0wIHo9hDHR5w0XVhoOKwkRqSfvnQGR6mTUjaRTMTY\ne/H/w9RK2bnDwhfuYtanzsVOsGsKAAAY6klEQVTjPfYubjz8v4+HGmH81Hkq8hr+tn3kh3zlfTcL\naWpqoqmpKftzKBTKZ0k5EQwGpc4cOlad7mKI1Fajb9mKXmZTWRJD8ftImCYl8xqYXb2X1uQ5dKwP\nUPruGwy276VnsJZadlJGX3aU0Fh2oEvIwsRFHyn68Qz0og04MToc9AdCRPZ46fSfjcvnwDmxll1P\n72DBRy2cioGh6IQJYNh6dpRQVdXY/38f7+/Nsab2FKYcz2v4l5eX09vbm/25t7cXv9+fzybFGUTT\nIFjrgKoZQ/fbTVdgx+NQXIyt63DWWTREIlSdW07Peh/pV9+BrTtoD7sZGNxHMVGU/ZPIaRycRnos\nnkI9eKLYJkMKhRQm4AjHsMJRIjUOikoclPVuYaB9ErvKZzO1foBwu4mmdWJMPZuM7aS9XeEMPVAV\nOZbX8J88eTIdHR10d3cTCARYt24dX/ziF/PZpDgTHedksVlTgwZUnzsD+0MNGL/7K7G9MbZt/yCu\njn1oqUESGYtpvEMJCVwMoDM0GmesjnN2QHbKa8gwkW3oHb9kb8dM+rx1eFtbcW1robcxgDGxkXR9\nAy7DJPOBc9F1jXD4yLtxCvF+eX3/a5rG7bffzqOPPoplWVx22WU0NDTks0lRqJxOlPM+iOPsaZS/\n8hqxlzsIv1tGsqMHJZlmR48HX7oTL1FcZNAYoJg4RSRxM/Y+DRzIbhdD847W04mbNEb8HbS4k+7o\nWRiZOMqeHuyGLnZ6GrFa3Wh1VZTM9+H1jWLxYlzI+8HP+eefz/nnn5/vZoQY4vFgXzyfBu9mkhUx\nlL4+jD3dYE0ktX0nuhWjL2Kg90dwmlHSqNTSRgl9eLCOe4/h0aIydDVxNWHSQBI3jcl+wruC9Dlq\nYGeIkqo2evoGsc8+m3d3hTj33zy4fYfMIWSaB6+zkEnmBGP3k68Qp0yJRtEbqphSU01rq4o1YKN2\nduC9Zi7qq+vR2yOEu2qJuzz0RRzs6ejg/OTfKaYPnQTFmGPyZLGDoU8Ejv13QnMyiCcTJ00Jbe1O\nSGzDHDAoP8vB9tQEzmkKosbjQ/cf7ugiEndgWiqaalA20A71tbIDKGAS/uKMoxgGKAoOBzQ2Dt2s\nhZkVKP392Bdcg71zN+5kKVt3ODB2h/FtfZ2NfZ+gcvebFJl9lNBHMf0UEcd1yI5gLHSjH/oHq2BR\nTog4/VQY4OzLYKq9KFotRKN0O+vxOgz62ELMWUFJupfiMgd4PHQG66h0R1Cr5OxwoZLwF2cce/8U\n0oed9VRVrMpK7GAQ6uqoiESonG1A1IGSasJKG7T9fRrpl96gd1cnrTETJW1Qb2+nhF6cGGgkKcIe\nM9cQHLheQCeDg1bUjEF/yKAo3oamOemNXkCmXMcx2I93IEPKV0XC4cI1ewql/f1Ei86hTO5JX7Ak\n/MUZ59CJ5LJ3mjcM7MqhC8gOHT2kArZponV1UXfxROLONKnWStrf7sOOD7IvWkZJOkJxfxeGbaCQ\nYgJ78GCNmZ3A0NQRBlW0UmJGsQZcxBU/5ptvohdlyCQtYp4qnOUpHJUB7HV9hC84n8CWLahVupwD\nKFAS/uLMc5SJ5OzKyqOG24Hppq2qKtT+fkrOrkQpdlFSWsvWzgDVqd1YbTuIxqvoMirRwu309dXR\nwA7K6cGJgc7BOxqPVnweuFjMSf/Q5xPbQE8aGEkbPwnKUt0MJv3EzCm4Ah5KXn+J9Ly5kMmg9vai\n7N2LWVc39MlIdgIFQcJfnJmOc23Aod4/3TSlpZQWt+H5SDXevYNs/1c5g3WTcRWrTK0p4e12N2/t\ngIve/iX93VspHeigiAhOklhY6EOTOePEPuzCsnzTGbr3sAq4SKBhMoiNAwMNSJjFFA+GKd7zTxKx\nKhL1k8jsjrDtuU04JlQR0GP4Wtejlnoxpk/HrqqSncAZTsJfFLb3f0pwuzHOOw81GqViuovgOTq2\nfyZoGn5/kNTGMBU9FoOv3Exq7W/pjUykX7Vw9XdDdw9KJsUgbmIU0chWAvRl7wec7yGkB4apWtjo\npCjBwEDFwIWXfjK2g7ThRunuQckodCiTcGeiqB2DqGUGeBL4J6Rx9K/HOPdcrIkTZQdwBpPwF+Io\nnxKO9qlB06C+3qK4WMGoOQfnBS60d97B2R8mGZtCZE+CRGsvKbWIPVYDe8M9fCD0EkF24yWOgo2C\niWv/DWny8YnA4tCdjImFhQMTGx3H/k8lOg52DpZjdveRVFQ8JQrdITe9xU5KBjUCzn4C6TdRUyms\nadNkB3CGkvAXYhg0DYJBG1CguhFrUikDO3rQwv1UTepmX/RcXDv3MiVjE95XwsbAJziLHfRafipC\nWymOd1Fi9FBONx4S2aDWOHjeQN3//an8cSr7tzUU82CgoqOgYmGjYaOQwI031UdPpJT+QSf1vgj9\ngeloqhvnllZiAY3ekEqADpROEyrKKa1wyP0EzjAS/kKcKk1Dra2ipHb/eMl0mmnvbGFfQ5DU9l4q\npihURfuxPB8h2N0P/V5i7SH6esP09e/DP7ALJyYaJkNxraBjoJLGTQp9/4gilaFQNzn+H6y5fz17\n/5cF6Kgo2KRwo2GgAAoqSXSc8T58ikms3cBZ1IfV1k/cBXHFi8uM4mhvJXDp2ZjRKroHz6I6KReG\nnUkk/IXIFacTZfY5TJgYwZpfQ1/cQcLpI7UvhHPLuyiDLuzZdfSmSnlvt4G2ayeTu17DlYigDcSJ\nGh5SePAQJ0AvBjY+BikiipMMKhbG/qYOzAKqMBTyJgcnhDP2L0vgxsCJhombNAncDOAliZMAIfbS\nSNgOoKoWgfYuikt0Mv7JaH1hlMwA3R4f2rvdFLf1UtzTQ/ysCRQPJugNTDlsCmnZF4xPEv5C5NL+\n8wcK4N//xaRqzAsr6NrSjxODGl0nUOynbe8Cdq69iJK9mynu72AglKZ1IICiQHXvJoJmDz2mRkVm\nDy6SuMiQRKWY1P5RRSY6aXTSqCikUPYPPLVI4aKHIF6SFDFAHC+g7O8WSjNACZbqwLSdtJY24mEL\n3qI0rngbZsqATBrFVOlttUmXaSiJKH5jL72Zs2CwA6prydia3Gh+HJPwF+I00JwaVecEiEQUDEPB\nrdvMma9gzplJx18dRLrOQQnFcbVZFLnBjlfSHYmRiSUZUOPYoT7cfe00pPYQtxXc6iDJJBi2Qhod\nB2lKiKNikKaIvUygVE2RtHpRsFHR0UiRwIMDCxs3yWIfdkkJZ1UMEh5ooFTtwJfpB9uBmbQodvSj\nxwbIVE4lHVOI7YG6mgS2Q0frj2AGgug6RCLK/vMgYjyR8BfiNDl4svhgUGoejUnzgpyVyYASJJmE\njRt1CFeQTtk4KsqJRS2mKluo6tuBo70O+vro61NI9g4ST7sYTGkUZeL0mQYYFgmHH1UziSYG0ZMm\nabsUl5XGsBQGtWIsE1Rdwe2yKZtRRCIBNd5+zN4MyeoGrOggeiaMwxjELvKhGSkUpxPLVolbxRQr\nCoo51AGlKGAYymG/kxgfJPyFGGWHXmjmdivMnZMh2uPEMBS0ogy+MnD0+sA4Fyv4UZRolJreXsye\nMB3v9BGJVRKJQakRQk2lSDkCFCfbcGTiQIBEdwIj3k/SUYKTNFgKqaSNUlqEw6XQ0KiQbrfI1NSi\nVpYT39WLiUWiKIhfiZMxkjiqa4gHJlGsuMC2sfWhyS1sGxwOCf7xSMJfiNH2vgvNVLcD37lD8xAd\nWGa53VBUNHQlcjCIGQzCNJuK+Sr73nYS7bCJaCql3gwTiw2628PoXbspopuq1m10hsopMVKkUiqZ\nhM1AcSXumgqsQBlquUG5D6Jl9SioGCVlDLYH8cf2gh4kXV5PNHgWFQGLTHEJdsbALK88MGUSlZUS\n/uORhL8QY8ExpqPILquoQG1vHzrUPmSyOrW2lppzHSiVanYeO9NXhl3RR1FgOsHet3C8Okjl7gjd\nsQrIqLg1DaNuMgQCGMWlJAMWZk0Z9bSh9/dhqg66g2V07S3CchcxWN9IkZLA1eBhQoODPjUAtoZD\nt6mslNE+45WEvxDjwXEmq3M6bSorLaJRBdMcuo9BZaWF0+3AbJiNVV2N1bIeR3sCtcRHqqqeIlsl\n7KjC1F0MTPIxfboJHQZmkQc1EqGi1kavqqCtfBZep5PycotgcCjoh+4AYI3yCyJGKm/h/9xzz/HX\nv/6V0tJSAG655Ra5naMQI3GMTwd+v017u4Lfb6MoNn4/dHcPLUfTsOvqcNyyiPCrUTBMFIdO2utH\nszVqKi3cbhvNqWHV16NEItgVFdi6js/vx6cduF5YnGnyeuR/zTXXsGjRonw2IUTB0zSorbWyw0id\nTo4Ye685NSbP87Nly9A9yXTdJlBqYdv7dxL7N3QyM6GKM4N0+whxBjh0GGkwCKHQkes4nXDOOWZ2\nJyFX6BY2xbbtvOzon3vuOdauXYvH46GxsZFbb70Vr9d7xHotLS20tLQA0NzcTDqdzkc5OaXrOoZh\nnHjFUSZ15pbUmTvjoUYYP3U6nc4Tr/Q+Iwr/JUuW0NfXd8Tym2++malTp2b7+1esWEEkEuHuu+8+\n4Tbb29tPtZzTJhgMEjraodUYI3XmltSZO+OhRhg/ddbW1g77OSPq9nn44YdPar2FCxfyxBNPjKQp\nIYQQOZS3mwtFIpHs9+vXr6ehoSFfTQkhhBimvJ3w/Z//+R92796NoihUVFRw55135qspIYQQw5S3\n8P/CF76Qr00LIYQYoXzfU1oIIcQYJOEvhBAFSMJfCCEKkIS/EEIUIAl/IYQoQBL+QghRgCT8hRCi\nAEn4CyFEAZLwF0KIAiThL4QQBUjCXwghCpCEvxBCFCAJfyGEKEAS/kIIUYAk/IUQogBJ+AshRAGS\n8BdCiAIk4S+EEAVoRLdxfOWVV1i5ciVtbW089thjTJ48OfvYqlWrWLNmDaqqcttttzF79uwRFyuE\nECI3RnTk39DQwP3338+MGTMOW75v3z7WrVvHd77zHR566CGefvppLMsaUaFCCCFyZ0ThX19fT21t\n7RHLN2zYwPz583E4HFRWVlJdXc327dtH0pQQQogcGlG3z7GEw2GmTp2a/TkQCBAOh4+6bktLCy0t\nLQA0NzcTDAbzUVJO6boudeaQ1Jlb46HO8VAjjJ86T8UJw3/JkiX09fUdsfzmm29m7ty5R32Obdsn\nXUBTUxNNTU3Zn0Oh0Ek/d7QEg0GpM4ekztwaD3WOhxph/NR5tB6YEzlh+D/88MPD3mh5eTm9vb3Z\nn8PhMIFAYNjbEUIIkR95Geo5Z84c1q1bRyaTobu7m46ODqZMmZKPpoQQQpyCEfX5r1+/nmeeeYZo\nNEpzczOTJk3ioYceoqGhgQ996EPcd999qKrKZz/7WVRVLikQQoixYkThf+GFF3LhhRce9bHrr7+e\n66+/fiSbF0IIkSdyOC6EEAVIwl8IIQqQhL8QQhQgCX8hhChAEv5CCFGAJPyFEKIASfgLIUQBkvAX\nQogCJOEvhBAFSMJfCCEKkIS/EEIUIAl/IYQoQBL+QghRgCT8hRCiAEn4CyFEAZLwF0KIAiThL4QQ\nBUjCXwghCtCIbuP4yiuvsHLlStra2njssceYPHkyAN3d3SxevJja2loApk6dyp133jnyaoUQQuTE\niMK/oaGB+++/n5/85CdHPFZdXc2TTz45ks0LIYTIkxGFf319fa7qEEIIcRqNKPyPp7u7m6985St4\nPB5uvvlmZsyYka+mhBBCDNMJw3/JkiX09fUdsfzmm29m7ty5R32O3+9n+fLllJSUsHPnTp588kmW\nLVtGUVHREeu2tLTQ0tICQHNzM8FgcLi/w2mn67rUmUNSZ26NhzrHQ40wfuo8FScM/4cffnjYG3U4\nHDgcDgAaGxupqqqio6Mje0L4UE1NTTQ1NWV/DoVCw27vdAsGg1JnDkmduTUe6hwPNcL4qfPA4Jrh\nyMtQz2g0imVZAHR1ddHR0UFVVVU+mhJCCHEKRtTnv379ep555hmi0SjNzc1MmjSJhx56iM2bN/Pc\nc8+haRqqqnLHHXfg9XpzVbMQQogRGlH4X3jhhVx44YVHLJ83bx7z5s0byaaFEELkkVzhK4QQBUjC\nXwghCpCEvxBCFCAJfyGEKEAS/kIIUYAk/IUQogBJ+AshRAGS8BdCiAIk4S+EEAVIwl8IIQqQhL8Q\nQhQgCX8hhChAEv5CCFGAJPyFEKIASfgLIUQBkvAXQogCJOEvhBAFSMJfCCEKkIS/EEIUoBHdw/fZ\nZ5/ln//8J7quU1VVxd13301xcTEAq1atYs2aNaiqym233cbs2bNzUrAQQoiRG9GR/7nnnsuyZctY\nunQpNTU1rFq1CoB9+/axbt06vvOd7/DQQw/x9NNPY1lWTgoWQggxciMK/w9+8INomgbAtGnTCIfD\nAGzYsIH58+fjcDiorKykurqa7du3j7xaIYQQOTGibp9DrVmzhvnz5wMQDoeZOnVq9rFAIJDdMbxf\nS0sLLS0tADQ3N1NbW5urkvJK6swtqTO3xkOd46FGGD91DtcJj/yXLFnCl770pSO+NmzYkF3n+eef\nR9M0LrnkEgBs2z7pApqammhubqa5uZkHHnjgFH6F00/qzC2pM7fGQ53joUY4s+s84ZH/ww8/fNzH\nX3rpJf75z3/yyCOPoCgKAOXl5fT29mbXCYfDBAKBYRcnhBAiP0bU579x40ZeeOEFvvrVr+JyubLL\n58yZw7p168hkMnR3d9PR0cGUKVNGXKwQQojc0L75zW9+81Sf/Oijj5JOp3n11Vf5y1/+wu7du7ng\nggvw+XzE43F+/OMf849//IPbb7/9pPvNGhsbT7Wc00rqzC2pM7fGQ53joUY4c+tU7OF00AshhDgj\nyBW+QghRgCT8hRCiAOVsnP9IjJdpIl555RVWrlxJW1sbjz32GJMnTwagu7ubxYsXZ89rTJ06lTvv\nvHPM1Qlj6/U81HPPPcdf//pXSktLAbjllls4//zzR7mqIRs3buTnP/85lmWxcOFCrrvuutEu6aju\nuece3G43qqqiaRrNzc2jXRIAy5cv54033sDn87Fs2TIA4vE4Tz31FD09PVRUVLB48WK8Xu+Yq3Ms\nvi9DoRA//OEP6evrQ1EUmpqauPrqq4f/mtpjwMaNG23DMGzbtu1nn33WfvbZZ23btu3W1lb7/vvv\nt9PptN3V1WV//vOft03THLU6W1tb7ba2Nvsb3/iGvX379uzyrq4u+7777hu1ut7vWHWOtdfzUCtW\nrLBfeOGF0S7jCKZp2p///Oftzs5OO5PJ2Pfff7/d2to62mUd1d1332339/ePdhlH2LRpk71jx47D\n/kaeffZZe9WqVbZt2/aqVauyf/Oj6Wh1jsX3ZTgctnfs2GHbtm0PDg7aX/ziF+3W1tZhv6Zjottn\nvEwTUV9fPy6u9jtWnWPt9RwPtm/fTnV1NVVVVei6zvz58w+7wFGc2MyZM484At2wYQMLFiwAYMGC\nBWPiNT1anWOR3+/PjuzxeDzU1dURDoeH/ZqOiW6fQ53qNBGjrbu7m6985St4PB5uvvlmZsyYMdol\nHWGsv55/+tOfePnll2lsbOTWW28dE3+I4XCY8vLy7M/l5eVs27ZtFCs6vkcffRSAyy+/nKamplGu\n5tj6+/vx+/3AUJhFo9FRrujYxuL78oDu7m527drFlClThv2anrbwX7JkCX19fUcsv/nmm5k7dy4w\nsmkicuVk6nw/v9/P8uXLKSkpYefOnTz55JMsW7aMoqKiMVXnaLyehzpezVdccQU33HADACtWrOCX\nv/wld9999+ku8QhHe80OXMk+1ixZsoRAIEB/fz/f+ta3qK2tZebMmaNd1rg2Vt+XAMlkkmXLlvGZ\nz3zmlLLmtIX/eJkm4kR1Ho3D4cDhcABDF1pUVVXR0dFx2InWXDuVOkd72o2TrXnhwoU88cQTea7m\n5Lz/Nevt7c0eXY01B/4vfT4fc+fOZfv27WM2/H0+H5FIBL/fTyQSyZ5QHWvKysqy34+l96VhGCxb\ntoxLLrmEiy66CBj+azom+vzH+zQR0Wg0e7+Crq4uOjo6qKqqGuWqjjSWX89IJJL9fv369TQ0NIxi\nNQdNnjyZjo4Ouru7MQyDdevWMWfOnNEu6wjJZJJEIpH9/u2332bChAmjXNWxzZkzh7Vr1wKwdu3a\nY35aHW1j8X1p2zY/+tGPqKur42Mf+1h2+XBf0zFxhe8XvvAFDMPI9qUdOlTy+eef529/+xuqqvKZ\nz3yG8847b9TqXL9+Pc888wzRaJTi4mImTZrEQw89xKuvvspzzz2HpmmoqsqNN944qgFxrDphbL2e\nh/r+97/P7t27URSFiooK7rzzzjFzhP3GG2/wi1/8AsuyuOyyy7j++utHu6QjdHV1sXTpUgBM0+Ti\niy8eM3V+97vfZfPmzcRiMXw+HzfddBNz587lqaeeIhQKEQwGue+++0a9L/1odW7atGnMvS+3bNnC\nI488woQJE7K9JLfccgtTp04d1ms6JsJfCCHE6TUmun2EEEKcXhL+QghRgCT8hRCiAEn4CyFEAZLw\nF0KIAiThL4QQBUjCXwghCtD/DwO+toNAbPnLAAAAAElFTkSuQmCC\n", + "text/plain": [ + "\u003cFigure size 600x400 with 1 Axes\u003e" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "print(\"MAP-estimated axes:\")\n", + "print(w)\n", + "\n", + "_, _, x_generated = model.sample(value=(w, z, None))\n", + "\n", + "plt.scatter(x_train[0, :], x_train[1, :], color='blue', alpha=0.1, label='Actual data')\n", + "plt.scatter(x_generated[0, :], x_generated[1, :], color='red', alpha=0.1, label='Simulated data (MAP)')\n", + "plt.legend()\n", + "plt.axis([-20, 20, -20, 20])\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "YdtvGLvmg341" + }, + "source": [ + "## Variational Inference\n", + "\n", + "MAP can be used to find the mode (or one of the modes) of the posterior distribution, but does not provide any other insights about it. We next use variational inference, where the posterior distribtion $p(\\mathbf{W}, \\mathbf{Z} \\mid \\mathbf{X})$ is approximated using a variational distribution $q(\\mathbf{W}, \\mathbf{Z})$ parametrised by $\\boldsymbol{\\lambda}$. The aim is to find the variational parameters $\\boldsymbol{\\lambda}$ that _minimize_ the KL divergence between q and the posterior, $\\mathrm{KL}(q(\\mathbf{W}, \\mathbf{Z}) \\mid\\mid p(\\mathbf{W}, \\mathbf{Z} \\mid \\mathbf{X}))$, or equivalently, that _maximize_ the evidence lower bound, $\\mathbb{E}_{q(\\mathbf{W},\\mathbf{Z};\\boldsymbol{\\lambda})}\\left[ \\log p(\\mathbf{W},\\mathbf{Z},\\mathbf{X}) - \\log q(\\mathbf{W},\\mathbf{Z}; \\boldsymbol{\\lambda}) \\right]$.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "BfXGOtI9g7bG" + }, + "outputs": [], + "source": [ + "qw_mean = tf.Variable(tf.ones([data_dim, latent_dim]))\n", + "qz_mean = tf.Variable(tf.ones([latent_dim, num_datapoints]))\n", + "qw_stddv = tfp.util.TransformedVariable(1e-4 * tf.ones([data_dim, latent_dim]),\n", + " bijector=tfb.Softplus())\n", + "qz_stddv = tfp.util.TransformedVariable(\n", + " 1e-4 * tf.ones([latent_dim, num_datapoints]),\n", + " bijector=tfb.Softplus())\n", + "def factored_normal_variational_model():\n", + " qw = yield tfd.Normal(loc=qw_mean, scale=qw_stddv, name=\"qw\")\n", + " qz = yield tfd.Normal(loc=qz_mean, scale=qz_stddv, name=\"qz\")\n", + "\n", + "surrogate_posterior = tfd.JointDistributionCoroutineAutoBatched(\n", + " factored_normal_variational_model)\n", + "\n", + "losses = tfp.vi.fit_surrogate_posterior(\n", + " target_log_prob_fn,\n", + " surrogate_posterior=surrogate_posterior,\n", + " optimizer=tf.optimizers.Adam(learning_rate=0.05),\n", + " num_steps=200)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "height": 405 + }, + "colab_type": "code", + "id": "iTmklbFhdHiV", + "outputId": "7597bd72-44a9-4f4a-e52e-f35fdcd62e92" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Inferred axes:\n", + "\u003ctf.Variable 'Variable:0' shape=(2, 1) dtype=float32, numpy=\n", + "array([[ 2.4168603],\n", + " [-1.2236133]], dtype=float32)\u003e\n", + "Standard Deviation:\n", + "\u003cTransformedVariable: dtype=float32, shape=[2, 1], fn=\"softplus\", numpy=\n", + "array([[0.0042499 ],\n", + " [0.00598824]], dtype=float32)\u003e\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAD8CAYAAACLrvgBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X1clXWe//HX9zoHFTiA50YkTEsF\nMhHkBCaSiRm1M9nOtFbu3LSzUW6WWw3ym6kmZ9t2ZzNnGqVFYNt1nKZp3J0aV2xumm2GCJyRTFAx\nbzJDnJkcUYRDCoIe4Hx/fxw7owkKCFwH+DwfDx96rnNdh/e5OJ73+V53R2mtNUIIIUQPGGYHEEII\nMXRIaQghhOgxKQ0hhBA9JqUhhBCix6Q0hBBC9JiUhhBCiB6T0hBCCNFjUhpCCCF6TEpDCCFEj0lp\nCCGE6DGr2QEGwtGjR/u8rMvloqGhoR/T9A/J1TvBmguCN5vk6p1gzQV9yxYbG9uj+WSkIYQQosek\nNIQQQvSYlIYQQogek9IQQgjRY1IaQggheuyyR08VFRWxc+dOoqKiWL169QX3/fznP+cnP/kJP/jB\nD4iMjASguLiY0tJSDMMgOzublJQUAGprayksLMTr9eJ2u8nOzkYpRXt7OwUFBdTW1hIREUFOTg7R\n0dEAlJWVsWnTJgAWLVrE/Pnz+/O5CyGE6KXLjjTmz5/P008/fdH0hoYG9uzZg8vlCkw7cuQIFRUV\nrFmzhhUrVrB+/Xp8Ph8A69atY+nSpeTn53Ps2DGqq6sBKC0tJTw8nLVr17Jw4UI2bNgAQEtLCxs3\nbmTlypWsXLmSjRs30tLS0i9PWgghRN9ctjSmT5+OzWa7aPorr7zCV7/6VZRSgWmVlZVkZGQQEhJC\ndHQ0MTEx1NTU0NTURFtbGwkJCSilmDdvHpWVlQBUVVUFRhDp6ens3bsXrTXV1dUkJydjs9mw2Wwk\nJycHimYg6NPN+H7xU9oPfThgP0MIIYa6Pu3TqKqqwuFwcO21114w3ePx4HQ6A7cdDgcej+ei6U6n\nE4/Hc9EyFouFsLAwmpubu32sAWNY0L/4H85WbR24nyGEEENcr88IP3v2LJs2beLb3/72Rfdprbtc\nprvp3d13/uilJ9NLSkooKSkBYNWqVRdsMuuNxklT6Di4F9ffPtCn5QeS1Wrt8/MaSJKr94I1m+Tq\nnWDNBQObrdelcfz4cerr6/nmN78JQGNjI08++STPP/88TqeTxsbGwLwejweHw3HR9MbGRhwOB0Dg\nPqfTSWdnJ62trdhsNhwOB/v377/gsaZPn95lpqysLLKysgK3+3pqv++aeDqrfs+J+nqUEVwHlgXr\nJQskV+8FazbJ1TvBmguC7DIikyZN4gc/+AGFhYUUFhbidDr57ne/y9ixY0lLS6OiooL29nbq6+up\nq6sjLi4Ou91OaGgoBw8eRGvNli1bSEtLAyA1NZWysjIAtm3bRmJiIkopUlJS2L17Ny0tLbS0tLB7\n9+7AkVgDZuo0dGsL1B0Z2J8jhBBD1GVHGi+++CL79++nubmZhx9+mMWLF7NgwYIu5504cSJz5swh\nNzcXwzB48MEHMc59Yl+yZAlFRUV4vV5SUlJwu90ALFiwgIKCAh577DFsNhs5OTkA2Gw27r77br71\nrW8BcM8993S5Q74/qanT0IA+9AFqwqQB/VlCCDEUKX2pHQ5DVF+vcqu1Rn/j7yEpFeP+r/dzqisT\nrENhydV7wZpNcvVOsOaCINs8NZwppRh13Qx0zQGzowghRFCS0viMkOtnwvE/o082mR1FCCGCjpTG\nZ4xKTgVAH3jf5CRCCBF8pDQ+w3ptPISFg5SGEEJcRErjM5TFAglJMtIQQoguSGl0QU1Lhobj6BPH\nzI4ihBBBRUqjC2paMiD7NYQQ4rOkNLoSOxHGOtF7qsxOIoQQQUVKowtKKVTKjbBvF7rda3YcIYQI\nGlIa3VAzbwTvWTmKSgghziOl0Z3rkmF0KLr6PbOTCCFE0JDS6IYKCYEZbvTu7ehzX1krhBAjnZTG\nJSj3HDjZBDX7Lz+zEEKMAFIal6BSZsOo0ej3tpgdRQghgoKUxiWo0WNQKbPRO7aiO9rNjiOEEKaT\n0rgMdWMmnG6GfdVmRxFCCNNJaVxOYgrYIvBVvG12EiGEMJ2UxmUoawhqzgLY/Z58x4YQYsST0ugB\nNe+voLMTvbXE7ChCCGEqKY0eUDFXw3VJ6N/9Bu3rNDuOEEKYRkqjh4wFC/2XS5fDb4UQI5iURk+l\npMOkqeg3Nsjht0KIEUtKo4eUYWD8zd9BYz16y1tmxxFCCFNYLzdDUVERO3fuJCoqitWrVwPw6quv\nsmPHDqxWK+PHj2fZsmWEh4cDUFxcTGlpKYZhkJ2dTUpKCgC1tbUUFhbi9Xpxu91kZ2ejlKK9vZ2C\nggJqa2uJiIggJyeH6OhoAMrKyti0aRMAixYtYv78+QOxDnou0e3ft/HL19AZt6LGhJqbRwghBtll\nRxrz58/n6aefvmBacnIyq1ev5vvf/z5XXXUVxcXFABw5coSKigrWrFnDihUrWL9+Pb5zF/tbt24d\nS5cuJT8/n2PHjlFd7T9ZrrS0lPDwcNauXcvChQvZsGEDAC0tLWzcuJGVK1eycuVKNm7cSEtLS78+\n+d5SSvlHG80n0W//wtQsQghhhsuWxvTp07HZbBdMmzlzJhaLBYCEhAQ8Hg8AlZWVZGRkEBISQnR0\nNDExMdTU1NDU1ERbWxsJCQkopZg3bx6VlZUAVFVVBUYQ6enp7N27F6011dXVJCcnY7PZsNlsJCcn\nB4rGTGrqNEiZjX5rE/oTj9lxhBBiUF3xPo3S0tLAJiiPx4PT6Qzc53A48Hg8F013Op2Bojn/PovF\nQlhYGM3Nzd0+VjAw7v576OzA98M8uWy6EGJEuew+jUvZtGkTFouFm2++GQCtdZfzdTe9u/uUUl3O\n2930kpISSkr8J96tWrUKl8t1ydyXYrVaL7+8y0Xrg8tp/o/vErb1N4T/zX19/nn9mssEkqv3gjWb\n5OqdYM0FA5utz6VRVlbGjh07eOaZZwJv5k6nk8bGxsA8Ho8Hh8Nx0fTGxkYcDscFyzidTjo7O2lt\nbcVms+FwONi/f/8FjzV9+vQus2RlZZGVlRW43dDQ0Nenhcvl6tHy2p0BN2TQsuE/ab16CmpyQp9/\nZn/mGmySq/eCNZvk6p1gzQV9yxYbG9uj+fq0eaq6upo33niDJ598ktGjRwemp6WlUVFRQXt7O/X1\n9dTV1REXF4fdbic0NJSDBw+itWbLli2kpaUBkJqaSllZGQDbtm0jMTERpRQpKSns3r2blpYWWlpa\n2L17d2AzWDBQSmF87VGIsuNb9330aXN30gshxGBQ+lLbjoAXX3yR/fv309zcTFRUFIsXL6a4uJiO\njo7ADvL4+HgeeughwL/J6p133sEwDO6//37cbjcAhw4doqioCK/XS0pKCg888ABKKbxeLwUFBRw+\nfBibzUZOTg7jx48H/PtLPj0ya9GiRdxyyy09elJHjx7t29qg9w2ta/bjW/1tuDYeI+dfUeeVaH8K\n1k81kqv3gjWb5OqdYM0FAzvSuGxpDEWDWRoAesdWfP/5PZiRirHsaZT1inYV9VuuwSC5ei9Ys0mu\n3gnWXBCEm6fEhVTqTaivPgJ7qtCv5MsRVUKIYav/PxKPUEbm5/C1nEJv/gn4NGQ/jrKGmB1LCCH6\nlZRGP1J33AuGgd70Y/TpUxgPPyWXGhFCDCuyeaofKaUwPn8P6muPwv7d+Nb8E7r5lNmxhBCi30hp\nDADj5tsxln0LjvwB3/eeRDfWmx1JCCH6hZTGAFEpszFy/gVOfYJv1RPoP//R7EhCCHHFpDQGkEpI\nxPjm8wD4vvcU+qP9l1lCCCGCm5TGAFNXX4vx5HchYiy+vGfQu7ebHUkIIfpMSmMQKNd4jCdXwYRr\n8P3H8+j3K82OJIQQfSKlMUhURBTG8n+Fqyfj+49V6A92mx1JCCF6TUpjEKmwcIyvPwvRV+ErfA59\n6IDZkYQQolekNAaZioj0jzii7Pjy/xVd3/frZAkhxGCT0jCBGuvwH46rFL7ClegzrWZHEkKIHpHS\nMIkaF4Ox9Ak4dgT93/9pdhwhhOgRKQ0Tqetnou64F/3uO+jq98yOI4QQlyWlYTK1cDFMnIzv1UK5\nTpUQIuhJaZhMWUMwHsiB0y3o/37J7DhCCHFJUhpBQF09GfWFL6Orfo+u+r3ZcYQQoltSGkFC/dUi\nuCYO32s/kKOphBBBS0ojSCiLBeOrD8PJJvQvXjM7jhBCdElKI4ioyQmoubeh3/45+ric9CeECD5S\nGkFG3fVVsFjRb2wwO4oQQlxESiPIqEg7KuuL6Mrfof90yOw4QghxAevlZigqKmLnzp1ERUWxevVq\nAFpaWsjLy+PEiROMGzeO5cuXY7PZACguLqa0tBTDMMjOziYlJQWA2tpaCgsL8Xq9uN1usrOzUUrR\n3t5OQUEBtbW1REREkJOTQ3R0NABlZWVs2rQJgEWLFjF//vyBWAdBR/3V36DL3sT3xn9jeeyfzI4j\nhBABlx1pzJ8/n6effvqCaZs3byYpKYn8/HySkpLYvHkzAEeOHKGiooI1a9awYsUK1q9fj8/nA2Dd\nunUsXbqU/Px8jh07RnV1NQClpaWEh4ezdu1aFi5cyIYN/s0yLS0tbNy4kZUrV7Jy5Uo2btxIS0tL\nvz75YKXCwlG3fRHer5TRhhAiqFy2NKZPnx4YRXyqsrKSzMxMADIzM6msrAxMz8jIICQkhOjoaGJi\nYqipqaGpqYm2tjYSEhJQSjFv3rzAMlVVVYERRHp6Onv37kVrTXV1NcnJydhsNmw2G8nJyYGiGQnU\ngjshNBzfL+VIKiFE8Ljs5qmunDx5ErvdDoDdbufUKf/lLzweD/Hx8YH5HA4HHo8Hi8WC0+kMTHc6\nnXg8nsAyn95nsVgICwujubn5gunnP1ZXSkpKKCkpAWDVqlW4XK6+PC0ArFbrFS3ff1y03LmY0z97\nmbFnWrBaY4Ik14WCZ31dKFhzQfBmk1y9E6y5YGCz9ak0uqO17tX07u5TSnU5b3fTs7KyyMrKCtxu\naGi4VMxLcrlcV7R8f9Lpt0Dxq3g2/YTox78dNLnOF0zr63zBmguCN5vk6p1gzQV9yxYbG9uj+fp0\n9FRUVBRNTU0ANDU1ERkZCfhHEI2NjYH5PB4PDofjoumNjY04HI6Lluns7KS1tRWbzYbD4bjosT4d\n3YwUKnIsatbN6Ip38LWeNjuOEEL0rTTS0tIoLy8HoLy8nFmzZgWmV1RU0N7eTn19PXV1dcTFxWG3\n2wkNDeXgwYNordmyZQtpaWkApKamUlZWBsC2bdtITExEKUVKSgq7d++mpaWFlpYWdu/eHTgSayRR\nt9wJZ9s4U/ors6MIIcTlN0+9+OKL7N+/n+bmZh5++GEWL17MXXfdRV5eHqWlpbhcLnJzcwGYOHEi\nc+bMITc3F8MwePDBBzEMfy8tWbKEoqIivF4vKSkpuN1uABYsWEBBQQGPPfYYNpuNnJwcAGw2G3ff\nfTff+ta3ALjnnnsu2iE/EqjJ8TA5gdY3/xd943yUIafWCCHMo/SldjgMUUeP9v0SHMG4ndK3rQy9\nfg3G1/8ZNSPV7DgXCMb1BcGbC4I3m+TqnWDNBUG4T0MMLpV2E8ZYBz7ZRCWEMJmUxhCgrCGE3n4X\n7N2Brq8zO44QYgST0hgiQm//IqDQW0vMjiKEGMGkNIYIi3MczLgBXfE2urPT7DhCiBFKSmMIMebe\nBp94YO9Os6MIIUYoKY2hJHkWRETh2/pbs5MIIUYoKY0hRFmtqNmZsKcK3ToyrvgrhAguUhpDjLox\nEzo60DvfNTuKEGIEktIYaq6Ng+ir0Nu3mJ1ECDECSWkMMUop1I3z4MD76E+6vlS8EEIMFCmNIUjd\nOA+0Rlf9zuwoQogRRkpjCFJXTYRJU9DbpTSEEINLSmOIUjdmwuGD6Pq+X5xRCCF6S0pjiFKz5gLI\nDnEhxKCS0hiilGMcJCSi39tyya/TFUKI/iSlMYSpGzPh2BH4+LDZUYQQI4SUxhCmUjPAYkFvLzc7\nihBihJDSGMKULRKmu9Hbf4f2+cyOI4QYAaQ0hjg1OxOaGqBmv9lRhBAjgJTGEKdmzgJrCHrXNrOj\nCCFGACmNIU6NCYPrZ6J3bZOjqIQQA05KYxhQ7nRorIePa82OIoQY5qQ0hgGVMhuUIZuohBADznol\nC//yl7+ktLQUpRQTJ05k2bJleL1e8vLyOHHiBOPGjWP58uXYbDYAiouLKS0txTAMsrOzSUlJAaC2\ntpbCwkK8Xi9ut5vs7GyUUrS3t1NQUEBtbS0RERHk5OQQHR195c96mFERURB/vb80vvhVs+MIIYax\nPo80PB4Pv/71r1m1ahWrV6/G5/NRUVHB5s2bSUpKIj8/n6SkJDZv3gzAkSNHqKioYM2aNaxYsYL1\n69fjO3eY6Lp161i6dCn5+fkcO3aM6upqAEpLSwkPD2ft2rUsXLiQDRs29MNTHp5USjr8+Y9yLSoh\nxIC6os1TPp8Pr9dLZ2cnXq8Xu91OZWUlmZmZAGRmZlJZWQlAZWUlGRkZhISEEB0dTUxMDDU1NTQ1\nNdHW1kZCQgJKKebNmxdYpqqqivnz5wOQnp7O3r17ZWdvN5Q7HQC96z2TkwghhrM+b55yOBz89V//\nNY888gijRo1i5syZzJw5k5MnT2K32wGw2+2cOnUK8I9M4uPjL1je4/FgsVhwOp2B6U6nE4/HE1jm\n0/ssFgthYWE0NzcTGRl5QZaSkhJKSkoAWLVqFS6Xq69PC6vVekXLD5TL5nK5aJwcj9pbheOr/xA8\nuUwSrLkgeLNJrt4J1lwwsNn6XBotLS1UVlZSWFhIWFgYa9asYcuW7q+42t0I4VIjh67uU0pdNC0r\nK4usrKzA7YaGhktFvySXy3VFyw+UnuTyJc1C/+J/OHHoI1SUPWhymSFYc0HwZpNcvROsuaBv2WJj\nY3s0X583T+3Zs4fo6GgiIyOxWq3Mnj2bgwcPEhUVRVNTEwBNTU2BUYHT6aSxsTGwvMfjweFwXDS9\nsbERh8Nx0TKdnZ20trYGdqqLiyl3uv8b/aplE5UQYmD0uTRcLhcfffQRZ8+eRWvNnj17mDBhAmlp\naZSX+y+gV15ezqxZswBIS0ujoqKC9vZ26uvrqaurIy4uDrvdTmhoKAcPHkRrzZYtW0hLSwMgNTWV\nsrIyALZt20ZiYmKXIw1xzoRrYFwMete7ZicRQgxTfd48FR8fT3p6Ok8++SQWi4Vrr72WrKwszpw5\nQ15eHqWlpbhcLnJzcwGYOHEic+bMITc3F8MwePDBBzEMf2ctWbKEoqIivF4vKSkpuN1uABYsWEBB\nQQGPPfYYNpuNnJycfnjKw5dSCuVOR7/9S3TraVRYuNmRhBDDjNLD8HCko0f7fthpsG6n7GkuXbMf\n33efQi35fxizM4Mm12AL1lwQvNkkV+8Eay4I0n0aIkhNmQaRY0HODhdCDAApjWFGGQYqZTZ67w60\n96zZcYQQw4yUxjCkUjPg7BnYt8vsKEKIYUZKYzhKSILwCPSOrWYnEUIMM1Iaw5CyWv2bqHZvR7e3\nmx1HCDGMSGkMUyr1JjjTBvurzY4ihBhGpDSGq+uTISxcNlEJIfqVlMYwpawhqJmz0bvfQ3fIJioh\nRP+Q0hjGVOpN0HoaDrxvdhQhxDAhpTGcTU+BMaHoHRVmJxFCDBNSGsOYCglBzbwRvWsbuqPD7DhC\niGFASmOYU6k3welmOLjH7ChCiGFASmO4S3TD6DGyiUoI0S+kNIY5NWo0KnmWfxNVZ6fZcYQQQ5yU\nxgigUjOg+SR8tM/sKEKIIU5KYySYkQqjRqO3d/8d7kII0RNSGiOAGj0GdUMGumorut1rdhwhxBAm\npTFCqDm3QNtp2L3d7ChCiCFMSmOkmJYEY5343n3H7CRCiCFMSmOEUIYFlT4f9u5An/rE7DhCiCFK\nSmMEUem3gM8nO8SFEH0mpTGCqAmT4Jo4tGyiEkL0kfVKFj59+jQvvfQSH3/8MUopHnnkEWJjY8nL\ny+PEiROMGzeO5cuXY7PZACguLqa0tBTDMMjOziYlJQWA2tpaCgsL8Xq9uN1usrOzUUrR3t5OQUEB\ntbW1REREkJOTQ3R09JU/6xFMzbkF/dN16D//ETXhGrPjCCGGmCsaabz88sukpKTw4osv8sILLzBh\nwgQ2b95MUlIS+fn5JCUlsXnzZgCOHDlCRUUFa9asYcWKFaxfvx6fzwfAunXrWLp0Kfn5+Rw7dozq\nav+3zZWWlhIeHs7atWtZuHAhGzZsuMKnK9Ssm8FiQVe8bXYUIcQQ1OfSaG1t5YMPPmDBggUAWK1W\nwsPDqaysJDMzE4DMzEwqKysBqKysJCMjg5CQEKKjo4mJiaGmpoampiba2tpISEhAKcW8efMCy1RV\nVTF//nwA0tPT2bt3L1rrK3m+I56KHAszZ6Mr3pZzNoQQvdbn0qivrycyMpKioiKeeOIJXnrpJc6c\nOcPJkyex2+0A2O12Tp06BYDH48HpdAaWdzgceDyei6Y7nU48Hs9Fy1gsFsLCwmhubu5rZHGOMf/z\n0NKMrpKvghVC9E6f92l0dnZy+PBhHnjgAeLj43n55ZcDm6K60t0I4VIjh67uU0pdNK2kpISSkhIA\nVq1ahcvlulz8blmt1itafqD0Zy49dwGNP12HsfW3OP763qDJ1Z+CNRcEbzbJ1TvBmgsGNlufS8Pp\ndOJ0OomPjwf8m482b95MVFQUTU1N2O12mpqaiIyMDMzf2NgYWN7j8eBwOC6a3tjYiMPhuGAZp9NJ\nZ2cnra2tgZ3q58vKyiIrKytwu6Ghoa9PC5fLdUXLD5T+zuW7+XY6X/sBJ977PWrqtKDJ1V+CNRcE\nbzbJ1TvBmgv6li02NrZH8/V589TYsWNxOp0cPXoUgD179nD11VeTlpZGeXk5AOXl5cyaNQuAtLQ0\nKioqaG9vp76+nrq6OuLi4rDb7YSGhnLw4EG01mzZsoW0tDQAUlNTKSsrA2Dbtm0kJiZ2OdIQvafm\n3gbhEfje/JnZUYQQQ8gVHXL7wAMPkJ+fT0dHB9HR0SxbtgytNXl5eZSWluJyucjNzQVg4sSJzJkz\nh9zcXAzD4MEHH8Qw/J21ZMkSioqK8Hq9pKSk4Ha7AViwYAEFBQU89thj2Gw2cnJyrvDpik+pMaGo\nrC+g39iA/lMtatIUsyMJIYYApYfh4Uifjn76IliHnAORS7e24HtqCWq6G+PhJ4MmV38I1lwQvNkk\nV+8Eay4I0s1TYuhTYTbULQvROyvQdR+bHUcIMQRIaYxwKusLEDIK/eZGs6MIIYYAKY0RTkVEoTI/\nh95ejj52xOw4QoggJ6UhUJ+7G0aPwff6D82OIoQIclIaAhU5FrXwb2FPFXrvTrPjCCGCmJSGAEDd\neidEX4Xv9fXojg6z4wghgpSUhgBAWUMw7n0A6j5Gl/+f2XGEEEFKSkP8xcwb4fqZ6J//N/pUk9lp\nhBBBSEpDBCilML70D9DuxfefL6A7O82OJIQIMlIa4gIqdhLqa/8IB/ei//dHZscRQgQZKQ1xESP9\nFv+Z4r99A1/l78yOI4QIIlIaoktq8QMwdRr6lbXoPx4yO44QIkhIaYguKWsIxsNPgS0S34v/jK6T\ns8WFEFIa4hLUWAfG8n8Fw8C35p/QDcfNjiSEMJmUhrgkNT4WI+dfwHsGX94z6E8aL7+QEGLYktIQ\nl6UmTsZ47Bk42YTve99CnzhmdiQhhEmkNESPqLjrMXK/A6db/MUh+ziEGJGkNESPqSnXYXzzOfB1\n4vveU+g/1ZodSQgxyKQ0RK+oqydjPLEKRo3Ct/rbtH+03+xIQohBJKUhek2Nj8X45vMQFk7TM4+j\nd283O5IQYpBIaYg+Ua7xGE+uwjJhEr7C5/D9phittdmxhBADTEpD9Jka68TxXBG456B/9jL6xwXo\njnazYwkhBpCUhrgiavQYjKVPoO5YjP79b/Hl/TO65ZTZsYQQA8R6pQ/g8/l46qmncDgcPPXUU7S0\ntJCXl8eJEycYN24cy5cvx2azAVBcXExpaSmGYZCdnU1KSgoAtbW1FBYW4vV6cbvdZGdno5Sivb2d\ngoICamtriYiIICcnh+jo6CuNLPqZMgzU39yH76qr0a+sxbfyGxiPPYO66mqzowkh+tkVjzTefPNN\nJkyYELi9efNmkpKSyM/PJykpic2bNwNw5MgRKioqWLNmDStWrGD9+vX4fD4A1q1bx9KlS8nPz+fY\nsWNUV1cDUFpaSnh4OGvXrmXhwoVs2LDhSuOKAWSkz8f4xnNwpg3fyv+H73e/kf0cQgwzV1QajY2N\n7Ny5k1tvvTUwrbKykszMTAAyMzOprKwMTM/IyCAkJITo6GhiYmKoqamhqamJtrY2EhISUEoxb968\nwDJVVVXMnz8fgPT0dPbu3StvQkFOTZ2GsWINXBOH/nEBvvx/QXsazI4lhOgnV1QaP/rRj7jvvvtQ\nSgWmnTx5ErvdDoDdbufUKf/2bY/Hg9PpDMzncDjweDwXTXc6nXg8nouWsVgshIWF0dzcfCWRxSBQ\nznEYud9BfWUpHNyH79lH8W0tkcIXYhjo8z6NHTt2EBUVxZQpU9i3b99l5+/uDeNSbyRd3Xd+QX2q\npKSEkpISAFatWoXL5bpsnu5YrdYrWn6gDMlc9/49HXNv5VTBc7T/KB/re+WE/+0DjEpO6/L3OGi5\nTBas2SRX7wRrLhjYbH0ujQ8//JCqqip27dqF1+ulra2N/Px8oqKiaGpqwm6309TURGRkJOAfQTQ2\n/uUKqR6PB4fDcdH0xsZGHA7HBcs4nU46OztpbW0N7FQ/X1ZWFllZWYHbDQ193xzicrmuaPmBMmRz\nhYxBf/1fUFveov1Xr/PJs1+HqdMw7vwSJLoHrDyCdX1B8GaTXL0TrLmgb9liY2N7NF+fN0995Stf\n4aWXXqKwsJCcnBxmzJjB448/TlpaGuXl5QCUl5cza9YsANLS0qioqKC9vZ36+nrq6uqIi4vDbrcT\nGhrKwYMH0VqzZcsW0tLSAEhNTaWsrAyAbdu2kZiYOOCfUEX/U4aBMf/zGCv/C/XVh6GpAd+/P4vv\n+W/ie/cdtPes2RGFED10xYcPRSNCAAASnElEQVTcftZdd91FXl4epaWluFwucnNzAZg4cSJz5swh\nNzcXwzB48MEHMQx/Zy1ZsoSioiK8Xi8pKSm43W4AFixYQEFBAY899hg2m42cnJz+jisGkQoJQc2/\nAz33NnTF2+i3itE/zEP/9L9Q6begbr4ddfW1ZscUQlyC0sNw7+TRo0f7vGywDjmHYy6tNXy4B73l\nLfSud6GjAyZORs1IRc24AaZMQ1n79rkmWNcXBG82ydU7wZoLBnbzVL+PNIToKaUUTEtGTUtGN59C\nb3sHXb0N/dYm9K83QuRY1O13oWZnosY6L/+AQogBJ6UhgoKKiETd9kW47Yvo1tNwYDe+sl+jN/4I\nvfFHcPVk1Iwb/COQqdNQ1hCzIwsxIklpiKCjwsLhhgwsN2Sg//xH9PtV6H070b/djP6//4UxoTBt\nJkbGApg5C2VYzI4sxIghpSGCmppwDWrCNfD5u9FtrXDgffTenej3K/FVbwPXeNSNmajkNLgmrs/7\nQIQQPSP/w8SQoULDwJ2OcqejOzth17v4tryF/vVG9Juvw6jR/k1XKbPx3bHI7LhCDEtSGmJIUhYL\npM3FkjYX3XwSDu5DH9yLPvA++n/+ixOvr4dr41HTU1Cpc1ETJpkdWYhhQUpDDHkqIgpSM1CpGQDo\nPx1izL6dtFZvR//yNfQvfgpXTUSl3uSfZ8I1cpKoEH0kpSGGHTVpKhE3zObs5+9Ff+JB73oXXbUV\n/avX0L/8KUTZUdcl+Q/3Tb3Jv+NdCNEjUhpiWFNjHahbFsItC9Enm9DvV/p3pn+4B7ZvQb+2HpV2\nE8QnoqZOg/GxMgoR4hKkNMSIoaLsqJtvh5tv95+N/qdD6JJf+M9G31qCBhjrRM25BeWeA9dMkcN5\nhfgMKQ0xIiml/IfoPrgc7fPBsSPomg/Q1e+h/+/cGenhETAtyb8z/foU1LgYs2MLYTopDTHiKcOA\n2Emo2Ekw76/Qpz5Bf7AbPqhG79+N3lHhH4WMi/GflZ40y18mIaPMji7EoJPSEOIzVORY1OxMmJ3p\n34x17M/o/dXo/bvQW99Gv/Om/5yQ62eiktJQiW6Ua7zZsYUYFFIaQlyCUgquuhp11dVw653odq//\nyrzvV6H3VKF3b/ePQsZP8I9CEt2QMAM1eozZ0YUYEFIaQvSCChkFM1JRM1LR+iH/KGTfTv+f372F\nfvsXYLX6j8ZKdPtLZMK1ckSWGDakNIToowtGIVlf8I9CPtqH3rfL/+fTK/RGOfzlkehGTU+BIP1e\naSF6QkpDiH6iQkbBdDdquhvuBe1pQO/fBft2oavfg4q30UrROHUavmkzUTfMgQmT5LBeMaRIaQgx\nQJTDhZp7G8y9De3rhD/UoPftQn34PvpXr/vPTrdYIXYiKm0uKiERYq+RM9RFUJPSEGIQKMMCU65D\nTbkOR/ajnKj9CL1np//8kI/2oYtf9e9Qt1gh5UaMtLn+o7PCI8yOLsQFpDSEMIGKtKNuujVwWzc1\nwse16A/eR28rxbejAj49AXG6G3V9MkyaggqzmZhaCCkNIYKCsjvB7kQlz0Lfcz8cPug/N+SDavT/\nnfu+EPBfoTc5DZU8CyZf579EvBCDSEpDiCCjLBaIux4Vdz184cv+70w/dAD9p0PoD3ajf7MZ/ev/\nhdGhcG0cauo0VMIMuG6GfHe6GHBSGkIEORUWDkmpqKRUWLjYXyL7d6EP7kPXfoh+axP6zZ9BaLj/\na2+vn4lyxcCUBLnUieh3fS6NhoYGCgsL+eSTT1BKkZWVxR133EFLSwt5eXmcOHGCcePGsXz5cmw2\n/3bY4uJiSktLMQyD7OxsUlJSAKitraWwsBCv14vb7SY7OxulFO3t7RQUFFBbW0tERAQ5OTlER0f3\nzzMXYohSYeGQNheVNhcAffYsHNiN3vkuevd2eK/cv1M9NAw1IxXip/sv+z7hWtmcJa5Yn0vDYrHw\nd3/3d0yZMoW2tjaeeuopkpOTKSsrIykpibvuuovNmzezefNm7rvvPo4cOUJFRQVr1qyhqamJ73zn\nO/z7v/87hmGwbt06li5dSnx8PM8//zzV1dW43W5KS0sJDw9n7dq1bN26lQ0bNrB8+fL+fP5CDHlq\n9GiYeSNq5o3+705vPA51f0bvqkDv3QWVvztXIuH+kwynTkNNmgJTpqGssrFB9E6fXzF2ux273Q5A\naGgoEyZMwOPxUFlZybPPPgtAZmYmzz77LPfddx+VlZVkZGQQEhJCdHQ0MTEx1NTUMG7cONra2khI\nSABg3rx5VFZW4na7qaqq4t577wUgPT2dH/7wh2it5ZIMQnRDWSwQHQvRsaiZs/wXXPScQNd84P/y\nqT07oOr3gZEIkxNQE6fApCl0ps8zO74YAvrlY0Z9fT2HDx8mLi6OkydPBsrEbrdz6tQpADweD/Hx\n8YFlHA4HHo8Hi8WC0+kMTHc6nXg8nsAyn95nsVgICwujubmZyMjIC35+SUkJJSUlAKxatQrXFVym\nwWq1XtHyA0Vy9U6w5gITso0bB9dNh4V3o7XGd7KJ9g/34N35Hu01++l4++fQ0UHDD1YTct0MQq6f\nyahEN6NmuIPiwovB+rsM1lwwsNmuuDTOnDnD6tWruf/++wkLC+t2Pq11r6Z3d19Xo4ysrCyysrIC\ntxsaGi4V+ZJcLtcVLT9QJFfvBGsuCJJsUxP9fwCjox2O/onQj/Zxels57T//H1qLfwLWEP8Ve6+Z\nCpFR/u8buTZ+0M8VCYr11YVgzQV9yxYbG9uj+a6oNDo6Oli9ejU333wzs2fPBiAqKoqmpibsdjtN\nTU2BUYHT6aSxsTGwrMfjweFwXDS9sbERh8NxwTJOp5POzk5aW1sDO9WFEP1DWUNg0lRsN8zmzK1f\nQHvPwkf70XvPXb33wG7w+fybtAwjcGa7mpwgZ62PQH0uDa01L730EhMmTODOO+8MTE9LS6O8vJy7\n7rqL8vJyZs2aFZien5/PnXfeSVNTE3V1dcTFxWEYBqGhoRw8eJD4+Hi2bNnC5z73OQBSU1MpKysj\nISGBbdu2kZiYKPszhBhgatRo/xV5E93Ag/4Rf8sp+Pgw+sO9/hMOS3+F7tjsX8BihbEO/9fiJt4A\nU6dBWLj/ccSwo/Sltg9dwoEDB3jmmWeYNGlS4I38y1/+MvHx8eTl5dHQ0IDL5SI3NzcwOti0aRPv\nvPMOhmFw//3343a7ATh06BBFRUV4vV5SUlJ44IEHUErh9XopKCjg8OHD2Gw2cnJyGD/+8t+QdvTo\n0b48JSB4h5ySq3eCNRcEb7be5NIdHfCHj9AH98KZVvTxo/DBbmhr/ctMDpf/a3TDbGB3wfhY1PhY\n/78jolBjQvs912AK1lwwsJun+lwawUxKY/BIrt4L1mxXmkt3dEDth+gjh/3lcfRP/jJpbYGmBujo\nuHCBcTEQczXKFQ3OaJQz2l8o2geWEIgaCyGjcUZH03jyJISMumhLg9baPz+ABrQ+9w/AYg3Mr7WG\ndi94z0JHO3R2QmfHZ/7uBJ/P/xhn29CfeOATj3+ZKLt/H0+71z/qCg0n4qoJNNfXgbLAmFDwdfqf\nY2e7/+9Ro8EWCc0n/cuNGg1nz/rnGz3af0Z/SAgoAwwLylDn/n3en/NvKwUo/9+Kv/z70+esgTGh\nqImTg3efhhBCfEpZrZCQ6L/E+2doXyc0noDjR9GnmqCp0b+560Qd+tABaG2hu0+vJwI/QPnfeEeN\nhtFj/G/wJz3+N/uuWCz++Trawevt+xMzDP/P+oxTfX/ELvXLp/fJCVie/n5/PFK3pDSEEANOGRb/\nyGJcDF3tldStp8FTD00esBjg9aJPfQId7YSPHs3pUyf9n/jPnvF/WveeARSMdfhLJPCg5z59g3/e\nM20wahSEnCubUaP8n+4Ni39fjMWCOvc3Fsu5T/bnymms0z/CsFjhdLO/nKxWCLPBmVbsIVaavO3+\nQjnT5l/eavWPSCxWONsGzc0QEel/vHYvjBrjf35nzz2Xdq9/eZ/PP2I6/++L/q0vHFUERlXnjT5C\nB/67WKQ0hBCmU2HhEDYZrp78l2nn/g53uWgze3NeRNSFt8NsWF0u1Ke5Pns/+Ec5kfa/3D5/H441\nBMKH5pGghtkBhBBCDB1SGkIIIXpMSkMIIUSPSWkIIYToMSkNIYQQPSalIYQQosekNIQQQvSYlIYQ\nQogeG5bXnhJCCDEwZKTxGU899ZTZEbokuXonWHNB8GaTXL0TrLlgYLNJaQghhOgxKQ0hhBA9Znn2\n2WefNTtEsJkyZYrZEbokuXonWHNB8GaTXL0TrLlg4LLJjnAhhBA9JpunhBBC9Jh8n8Y51dXVvPzy\ny/h8Pm699VbuuusuU3I0NDRQWFjIJ598glKKrKws7rjjDl5//XXefvttIiMjAf/3sd9www2Dnu8f\n//EfGTNmDIZhYLFYWLVqFS0tLeTl5XHixAnGjRvH8uXLA98LPxiOHj1KXl5e4HZ9fT2LFy/m9OnT\ng77OioqK2LlzJ1FRUaxevRrgkuunuLiY0tJSDMMgOzublJSUQcv16quvsmPHDqxWK+PHj2fZsmWE\nh4dTX1/P8uXLA1//GR8fz0MPPTQgubrLdqnXu5nrLC8vL/B10q2trYSFhfHCCy8M6jrr7j1i0F5n\nWujOzk796KOP6mPHjun29nb9jW98Q3/88cemZPF4PPrQoUNaa61bW1v1448/rj/++GP92muv6Tfe\neMOUTOdbtmyZPnny5AXTXn31VV1cXKy11rq4uFi/+uqrZkTTWvt/l0uWLNH19fWmrLN9+/bpQ4cO\n6dzc3MC07tbPxx9/rL/xjW9or9erjx8/rh999FHd2dk5aLmqq6t1R0dHIOOnuY4fP37BfAOtq2zd\n/e7MXmfne+WVV/TPfvYzrfXgrrPu3iMG63Umm6eAmpoaYmJiGD9+PFarlYyMDCorK03JYrfbAzuw\nQkNDmTBhAh6Px5QsPVVZWUlmZiYAmZmZpq07gD179hATE8O4ceNM+fnTp0+/aJTV3fqprKwkIyOD\nkJAQoqOjiYmJoaamZtByzZw5E4vFAkBCQoJpr7OusnXH7HX2Ka017777LjfddNOA/OxL6e49YrBe\nZ7J5CvB4PDidzsBtp9PJRx99ZGIiv/r6eg4fPkxcXBwHDhzgrbfeYsuWLUyZMoWvfe1rg7oJ6HzP\nPfccALfddhtZWVmcPHkSu93/tZZ2u51Tp06Zkgtg69atF/xHDoZ11t368Xg8xMfHB+ZzOBymvXGX\nlpaSkZERuF1fX88TTzxBaGgoX/rSl7j++usHPVNXv7tgWWcffPABUVFRXHXVVYFpZqyz898jBut1\nJqWB/1PDZymluphz8Jw5c4bVq1dz//33ExYWxu23384999wDwGuvvcaPf/xjli1bNui5vvOd7+Bw\nODh58iT/9m//FtiGGww6OjrYsWMHX/nKVwCCZp11p6vXnRk2bdqExWLh5ptvBvxvOEVFRURERFBb\nW8sLL7zA6tWrCQsLG7RM3f3ugmWdffbDiRnr7LPvEd3p73Umm6fwjywaGxsDtxsbGwONbYaOjg5W\nr17NzTffzOzZswEYO3YshmFgGAa33norhw4dMiWbw+EAICoqilmzZlFTU0NUVBRNTU0ANDU1BXZe\nDrZdu3YxefJkxo4dCwTPOutu/Xz2defxeALrd7CUlZWxY8cOHn/88cAHpZCQECIiIgD/sf7jx4+n\nrq5uUHN197sLhnXW2dnJ9u3bLxiZDfY66+o9YrBeZ1IawNSpU6mrq6O+vp6Ojg4qKipIS0szJYvW\nmpdeeokJEyZw5513BqZ/+mIA2L59OxMnThz0bGfOnKGtrS3w7/fff59JkyaRlpZGeXk5AOXl5cya\nNWvQs8HFn/6CYZ0B3a6ftLQ0KioqaG9vp76+nrq6OuLi4gYtV3V1NW+88QZPPvkko0ePDkw/deoU\nPp8PgOPHj1NXV8f48eMHLRd0/7sze52Bf79ZbGzsBZu0B3OddfceMVivMzm575ydO3fyyiuv4PP5\nuOWWW1i0aJEpOQ4cOMAzzzzDpEmTAp/8vvzlL7N161b+8Ic/oJRi3LhxPPTQQ4M+Gjp+/Djf//73\nAf+nrblz57Jo0SKam5vJy8ujoaEBl8tFbm7uoO87OHv2LI888ggFBQWBofratWsHfZ29+OKL7N+/\nn+bmZqKioli8eDGzZs3qdv1s2rSJd955B8MwuP/++3G73YOWq7i4mI6OjkCWTw8T3bZtG6+//joW\niwXDMLj33nsH9ENUV9n27dvX7e/OzHW2YMECCgsLiY+P5/bbbw/MO5jrrLv3iPj4+EF5nUlpCCGE\n6DHZPCWEEKLHpDSEEEL0mJSGEEKIHpPSEEII0WNSGkIIIXpMSkMIIUSPSWkIIYToMSkNIYQQPfb/\nAakVbkxFIbkUAAAAAElFTkSuQmCC\n", + "text/plain": [ + "\u003cFigure size 600x400 with 1 Axes\u003e" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "print(\"Inferred axes:\")\n", + "print(qw_mean)\n", + "print(\"Standard Deviation:\")\n", + "print(qw_stddv)\n", + "\n", + "plt.plot(losses)\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "height": 269 + }, + "colab_type": "code", + "id": "CpSsruVIqAv5", + "outputId": "aa6a557d-68b6-49e3-dc57-0e348d238d81" + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt0VPW9///nvswtmUtmMsmEkHAH\nEcV6AS8cLVKiR20PpR61+mu/ttrqstp2gbWtHqs9a1FtrFB72lN61Z7Wc6l6foK/tqv2nNQW2y+2\n4N2CIAHCJffJ3O8ze+/fH4EIAkLIhCTM+7FWFpmZPfvzzmbmtWc+n70/W7Esy0IIIURFUce6ACGE\nEKeehL8QQlQgCX8hhKhAEv5CCFGBJPyFEKICSfgLIUQF0ke6gnA4zPe//31isRiKotDS0sI111xD\nKpXiscceo7+/n7q6OlauXInb7S5HzUIIIUZIGelx/tFolGg0yowZM8hms9x77718+ctf5o9//CNu\nt5vly5ezfv16UqkUn/zkJ8tVtxBCiBEYcbeP3+9nxowZALhcLiZPnkwkEmHz5s0sXrwYgMWLF7N5\n8+aRNiWEEKJMRtztc6i+vj52797NrFmziMfj+P1+YHAHkUgkjvqctrY22traAGhtbS1nOUIIIY6h\nbOGfy+VYs2YNn/70p6mqqjrh57W0tNDS0jJ0u6urq1wljZpgMEg4HB7rMo5L6iwvqbN8JkKNMHHq\nbGxsHPZzynK0T6lUYs2aNVx22WVcdNFFAPh8PqLRKDA4LuD1esvRlBBCiDIYcfhblsUPf/hDJk+e\nzEc+8pGh+xcsWMCGDRsA2LBhAwsXLhxpU0IIIcpkxN0+27dv58UXX2TKlCl8+ctfBuCmm25i+fLl\nPPbYY7zwwgsEg0HuvvvuERcrhBCiPEZ8qGe5SZ9/+Uid5TUWdVqWRS6XwzRNFEU5oec4HA7y+fwo\nVzYyE6FGGF91WpaFqqo4nc4jXgsn0+df1qN9hBDllcvlsNls6PqJv1V1XUfTtFGsauQmQo0w/uos\nlUrkcjlcLteI1yXTOwgxjpmmOazgF6c3XdcxTbMs65LwF2IcO9GuHlE5yvWakPAXQogKJOEvhDiu\n3/72t0yePJn29vbjLvvUU0/R09Nz0m1t3LiRm2+++bjLXXfddbzxxhvvu8xPfvITstnsSddyOpPw\nF0Ic1/r167nwwgt57rnnjrvsM888Q29v7ymo6vh++tOfSvgfg4S/EKcRw4BwWKGnRyUcVjCMka8z\nnU7z8ssvs3r16iPCf+3atSxdupSWlhYefvhhfv3rX/PGG2/w+c9/niuuuIJsNstFF11EJBIB4I03\n3uC6664D4LXXXmPZsmVceeWVLFu27LjfKrLZLJ/73OdoaWnhjjvuIJfLDT127733cvXVV7NkyRJW\nr14NwOOPP05vby/XX3/9UJtHW65SyWEEQpwmDAO6ulSczsEBwWJRoatLobHRZCRHKz7//PNcfvnl\nzJw5k5qaGt566y3mz5/PCy+8wPPPP8+vf/1rXC4X0WgUv9/Pv/3bv/HAAw/wgQ984H3XO2vWLJ59\n9ll0XefFF1/kkUce4Sc/+ckxl//FL36By+Wira2NrVu3ctVVVw099tWvfhW/349hGHz84x9n69at\nfOYzn+HHP/4xzzzzDIFA4JjLzZs37+Q3zgQm4S/EaSIaVdB1OHgwiKKArg/eHwye/Lmc69ev57bb\nbgPgox/9KOvXr2f+/Pn86U9/4uMf//jQMecHZ/E9UYlEghUrVrB7924URaFYLL7v8n/961+59dZb\nAZg3bx5nnnnm0GO/+tWv+I//+A8Mw6C3t5cdO3YcNdRPdLlKIOEvxGmiVFJ471GAijJ4P5xc+Eci\nETZu3Mj27dtRFAXDMFAUha997WtYlnVChx0eemz6oWfLPvrooyxatIjHH3+cffv2DXXNvJ+jtbd3\n715+9KMf8Zvf/IaamhpWrFhxWJfQcJerFNLnL8RpQtct3jtZi2UN3n+yfvOb3/CP//iPbNq0ib/+\n9a+8/PLLTJkyhU2bNrF48WJ++ctfDg2oHpzFt7q6mlQqNbSOpqYm3nzzzaH1HZRMJmloaADg6aef\nPm4tF110EevWrQNg27ZtvP3220PrcblceL1e+vv7+cMf/jD0HLfbPVTL+y1XiST8hThN+P0WpRJD\nOwDLglJp8P6T9dxzz3H11Vcfdt8111zDunXrWLJkCVdeeSVXX301V1xxBT/84Q8BuOGGG7j33nuH\nBnzvvvtuHnzwQT72sY8dNlXC5z73Ob75zW/y0Y9+FOMERqZvvvlm0uk0LS0trF27lnPPPReAs846\ni7PPPpslS5Zw9913HzaD8Cc+8Qk++clPct11173vcpVIJnY7CTIRWXlJnceWyWSGdXEkw4Bk0kYu\nZ6DrFn6/NaLB3tGi6zqlUmmsyziu8Vjn0V4TMrGbEBVO0yAYtCiVyjP/izh9SbePEEJUIAl/IYSo\nQBL+QghRgST8hRCiApVlwHft2rW8+uqr+Hw+1qxZAwwet/v73/8er9cLDF7X9/zzzy9Hc0IIIUao\nLOF/+eWXc9VVV/H973//sPs//OEPs2zZsnI0IYQQoozK0u0zb9483G53OVYlhBhn/uVf/oUlS5bQ\n0tLCFVdcwauvvgrAPffcwzvvvFOWNg6d+fNYvvvd7w57vU899RT333//mLUPcNttt7Fnzx5WrFjB\nk08+edhjzz//PP/n//wfAGbPng3AwMAAn/jEJ06qreEY1T7/3/3ud9xzzz2sXbv2sNO9hRCjxDBQ\nwmHUnh6UcJiRzun88ssv09bWxvPPP09bWxtPPfXU0AlFq1evZs6cOeWo+oR873vfO2Vtlav97du3\nY5omU6dOZfny5UdMif3cc8/x0Y9+9LD7amtrqa+vZ/PmzSOq93hG7SSvK6+8cmiipqeeeopf/OIX\n3HnnnUcs19bWRltbGwCtra0Eg8HRKqlsdF2XOstI6jy23t7eE7+Au2Gg9vWBrg8+x7Kgrw+zsZGT\nPc13YGCA2tpaqqurAaivrx967GMf+xhf//rXOffcc5k+fTq33norL774Ij6fj3/6p39i1apV7N+/\nn1WrVnHVVVfxy1/+kjfeeINvfvObAHzqU5/izjvv5O/+7u9QFAVN09B1nU996lN0dXWRz+f57Gc/\ny80338yqVavI5XJceeWVnHHGGfzgBz/gv//7v/nJT35CsVjk/PPP55FHHkHTNP7rv/6L7373u4RC\nIWbMmIHD4ThiG0YiEe644w4GBgY477zzAIbd/tGWe6/169dz9dVXo+s6S5YsYeXKlQwMDBAKhchk\nMvz5z3/mscceG6rv4L/XXHMN69ev55JLLjlinQ6Hoyyvw1EL/5qamqHfly5dyiOPPHLU5VpaWmhp\naRm6Laf5l4/UWV5jUWc+nz9sPpz3o4TDmICuKIdNSWD192OdZFhceumlrF69mosvvpjLLruMZcuW\nDQWSZVkYhkGpVCKTyXDRRRdx33338ZnPfIZvfvOb/Od//ifvvPMOK1asoKWlBcMwME2TUqmEruuH\nPf/Q31evXo3f7yebzfLhD3+Yq666ivvuu48nnniC//mf/wHg7bffZt26daxfvx6bzcZ9993H008/\nzQc/+EG+9a1v8fzzz+PxeLj++us5++yzj5ii4dFHH2XhwoWsXLmStrY2nnzyyWG1f6zlDl434KBN\nmzaxbNmyofavvvpq1q1bx2c/+1l++9vfcskll+B0OoceP/jv2WefTWtr61Gnlsjn80e8DsfV9A4H\nL+wAgxugubl5tJoSQgBKqcTR5nRWSqWTnNB5cIbO559/nr/+9a9s3LiRz33uc9x33318/OMfP2w5\nu93OkiVLAJg7dy52ux2bzcaZZ57J/v37h9XmE088wW9/+1tgcK6v3bt3HxGqf/7zn3nrrbe45ppr\nAMjlcgSDQV577TUuueQSamtrAVi2bBm7du06oo2//OUv/PSnPwUGP4Ae+mH1RNo/0eV6e3uHagFY\nvnw5q1at4rOf/SzPPffcMaexDgaDI7oO8okoS/h/5zvfYevWrSSTSe644w5uuOEGtmzZQkdHB4qi\nUFdXx+23316OpoQQx2DpOsp7L4hiWVg224jWq2kaixYtYtGiRcydO5dnnnnmiPDXdX1orn1VVXE4\nHEO/H/z0eui8/nD43P4Hbdy4kT/96U/86le/wuVycd111x11OcuyuP7667nvvvsOu//5558/oWsM\nwNGvDfDe9q+//voR1el0Og+7f+HChfT19bFlyxZefvll1q5de9Ta8vk8TqfzhP6Ok1WW8F+xYsUR\n933oQx8qx6qFECfI8vtRuroGL98FQ3M6W4f00w9Xe3s7qqoyY8YMALZs2UJTU9NJrau5uZmf//zn\nmKZJZ2cnr7/++hHLJJNJfD4fLpeL9vb2oSOLAGw2G8ViEZvNxqWXXsott9zCbbfdRjAYJBqNkk6n\nOe+883jwwQeJRCJ4PB5+/etfH/VKXRdffDHPPvssK1as4IUXXiAWix21/VdeeeWo7b9fnYeaPXs2\nu3fvHur5UBSFf/iHf2DFihV86EMfOmbA79q1i7lz5574xj0JMqunEKcLTcNsbMRKJsEwsGy2weAf\nwZzOmUyGr33tayQSCXRdZ9q0aXzrW986qXUtXLiQKVOmsHTpUubOncv8+fOPWObyyy/nySefpKWl\nhRkzZhx2YugnPvEJWlpamD9/Pv/6r//KV77yFW666SYsy0LXdR566CEuuOACvvSlL7Fs2TJCoRDz\n588/6rUCVq5cyV133cXf//3fc/HFFzN58uSjtn/BBRcctf01a9Ycs85DLV26lJdeeokPfvCDQ/ct\nX76cH/zgB/zTP/3TMbfVxo0bWbp06fE36gjIfP4nQQYoy0vqPLbhzucP43MO+veaCDXCyOvMZrNc\nf/31PPfccyc8cA9w7bXX8sQTTxw2FnFQuebzl7l9hBBilLhcLu65555hDd4ODAxw++23HzX4y0m6\nfYQYx8bZF3NxEi6//PJhLV9bW8tVV111zMfL9ZqQT/5CjGOHHi0jRKlUQlXLE9vyyV+IcczpdJLL\n5cjn8yd8CKPD4TjqYYfjyUSoEcZXnZZloapq2Q4BlfAXYhxTFAWXyzWs50yEAfSJUCNMnDpPhnT7\nCCFEBZLwF0KICiThL4QQFUjCXwghKpCEvxBCVCAJfyGEqEAS/kIIUYEk/IUQogJJ+AshRAWS8BdC\niAok4S+EEBVIwl8IISpQWSZ2W7t2La+++io+n481a9YAkEqleOyxx+jv76euro6VK1fidrvL0ZwQ\nQogRKssn/8svv/yI61GuX7+e+fPn893vfpf58+ezfv36cjQlhBCiDMoS/vPmzTviU/3mzZtZvHgx\nAIsXL2bz5s3laEoIIUQZjNp8/vF4HL/fD4Df7yeRSBx1uba2Ntra2gBobW0lGAyOVkllo+u61FlG\nUmd5TYQ6J0KNMHHqPBljfjGXlpYWWlpahm5PhAsnTJQLPEid5SV1ls9EqBEmTp2NjY3Dfs6oHe3j\n8/mIRqMARKNRvF7vaDUlhBBimEYt/BcsWMCGDRsA2LBhAwsXLhytpoQQQgxTWbp9vvOd77B161aS\nySR33HEHN9xwA8uXL+exxx7jhRdeIBgMcvfdd5ejKSGEEGVQlvBfsWLFUe9/8MEHy7F6IYQQZSZn\n+AohRAWS8BdCiAok4S+EEBVIwl8IISqQhL8QQlQgCX8hhKhAEv5CCFGBJPyFEKICSfgLIUQFkvAX\nQogKJOEvhBAVSMJfCCEqkIS/EEJUIAl/IYSoQBL+QghRgST8hRCiAkn4CyFEBZLwF0KIClSWyzi+\nn7vuugun04mqqmiaRmtr62g3KYQQ4jhGPfwBvv71r+P1ek9FU0IIIU6AdPsIIUQFUizLskazgbvu\nugu32w3AFVdcQUtLy2GPt7W10dbWBkBrayuFQmE0yykLXdcplUpjXcZxndI6DQMiESgWwWaDQAA0\n7YSeKtuzvCZCnROhRpg4ddrt9mE/Z9TDPxKJEAgEiMfjfOMb3+CWW25h3rx5x1y+q6trNMspi2Aw\nSDgcHusyjqucdRoGRKMKpZKCrlv4/da72W4YqF1doOugKGBZUCphNjae0A6gErfnaJoIdU6EGmHi\n1NnY2Djs54x6n38gEADA5/OxcOFC2tvb3zf8xfhjGNDVpQ5le7Go0NWl0NhoommgRKODwW+aKPE4\nimFgqSqKpoHdjlIqYek6lt8/tDM4dGdiWWCaJ/xFQQhRBqMa/rlcDsuycLlc5HI53nzzTa677rrR\nbFKMgnCvSXZvFKtoYGka+Wo/0YSNv/1No6nJpFExCPhMbP29oOsYpkJyoIjyxlsU55+Np0YjOVDC\n2NuDNXkSXr9K1z6LQs+BdfYb9OQVgiGVbBaqq8HheM+3CyFEWY1q+MfjcVavXg2AYRhceumlnHvu\nuaPZpCgzo2Aw8Fo3jkIao2Cwv8tGTypPvq6ROnuCeDyLPbcfy5GifpKC5fESjujYU0lwucj3Jfnb\n23UEAhY2xULt2s32tAdHIkxVcwDFptO7t0AyEiMSmUQwpJJKQX29edi3CyFEeY1q+IdCIR599NHR\nbEKMssTOAdyZMLmSjV27NeJxk3ykH6UnSri+jhp3L/2KHWeym3RGR03tpISfomGSmTSVvi0DqMRJ\nayU0SiTtQVKZIjXRPXj2bqcYmoQ+42xyJR2rO4LDBZRKZHMa7ik1RKMqweB7hqUMAyUaPWp3khDi\nxJyS4/zFBNYfocqrs/NvGoWiQjqtouVzFNNgcznpytiYVFeAdI78q90YVR60Gg0zX0R5vR1fFsz6\nEJlInmxeobp2D65EjqK9mnCumkJ/GHffm3Qr0/Hl+0mbLnSlhKlqGPE0A85GIvY4qlGipk7FP82D\nLdw7NLisFIsoXV0nPLgshBgk4S/el65bJDPg9UE8AZoOulICm46Vz6Pkozh62jGrMhQUO1axhKd9\nC4Wchb0/Qtg7hdLuDpyJDI5siewuHcwSpq8O01FNVKkhHrFhT/6FVI2flGbDlYvgS3ThVDIElCqy\n08/A2+Ag0umGLXFqz29GsymDBSoK6DpKNIoVDI7txhJiApHwF++relqA3t092DQVf42GgomRsehX\nvdj7Izj1GKoNbEYeLdGL10igFPPomRI5xYYr0kV30ouZyGCzDOqNXtKmGy2Vxk6B6iJkFDe1Sj9Z\ndxBzh4sGtZ8qK4uSyVDl82J0byPlnoxa5aSvCOE/tZOYNAfT46Wu0UbNDA+BuhIHdgfvf1iqEAKQ\n8BfHoYaCBM7Ioe7LEolbeAMapdAUMtuiWDaDJnsvPi1FMLUbK5ECpYTTo1GMZ6BgkRqw8BciGHkD\nL1EK6PhJYZVKqJRwUqBAFSoFtFgHjlgBFQ1NBSwwY/sx92hoqpeMp46MPYiqazinRMl469jtnUTD\na1kGzmpi5tkJVK+b3qgTxedFSSaIRgy6DRv1Z3gJhtTj7wRkPEFUCAl/8f40Dd85TeTscWY3l9i5\n10HCqMHfE8UX1HAWA9TFelBjRRSnilpU0NJJbGaCgYwdbzZBVcnATYoUDuyAgwIOMqgY2ChhYsfC\nwEkJMAFQTLAY/DHRqDF7ycQ78eOmk2a0WAybv56Zxh8oOJ3YXqrilQuvoHq6H3u1HW9XO2lPPVpd\nEK3KT+emLJ0NIab4otT6iqh2HcvrRUkk3g16rxe1V8YTRGWQ8BfHpdk1QmcFCIcVqtBw9YXRZzqg\npFFKGOStekrFElXpCHqyHyVbIO/woJWSqKUcOgYmBi4yWICTAk4y6HCgqyaLBWgcOdmUCVgYWICN\nJFWk8TBANuvCkTXIoxMnhOmoZvr+7WRqG7BrFl3OZtKuXgKNTqyaAGHHVLxv7aa9tobiLJWG2VXY\ndu7EnDx5cMygWETdtg0rEBgcRwAZTxCnNQl/cUI0bfBn8mQTm6OAGfKQ2N6P3TCJpYMEfVmsXBwm\nTyaf8WEOJKA0QNbuQymk0AEHWezksFNAh0PC/9gO3RnogA0TFwWqKaAceH4tMXJ5G0peoZTYTowA\nk2knZq9n/445aM79TPJsRK+tZXfoQnredtIU6qFurh9fMYlvuh9NUwZrSSQGdwAHKcrgN4Nybkwh\nxgEJf3HCSiVlcOoeTUezWXjPqCPyag6zykepughBOzlVR3n1LdQi5HwN9Bd0vCjYSWEnj50SGgwF\n93Ad3Bkc2gljA5wUsYASBdykKKDQUNhFU2ELGTxoPRrJXQHmON5gl2c+vS43mdR8nNNDOJIadjuU\n+px4bDlqQlH83hKqTcPyerGczhFuOSHGHwl/ccJ03aJYVDB8ftTeLjSbjjJjKp5oN3Z3NcVcHbl9\nMcwPXEzVnndI7R7AHukkjp8geYrYKKGjMjoztyoM7ggAHFiYlHARxSRKEQWf0U8m48af2UMSD6ne\nzew760OYtQ3Ygh6cdV7S8T76GhsJ1KnMnF7EluykdN55o1KvEGNJwl+cML/foqtLQdc1iqFG1FgU\nSzXQZ02moINSKpE2IxS8teRmnUH1n18g2ZujoJgMpCxqyWPhw02CKvKj/uJTgYMT3TqwKJHCTQoD\nyOIkl91P9cthOjznkK5vxm6m2etqwh2Mk22uJd3loHleNbaXd1I9sw78/lGuWIhTR8JfnDBNg8ZG\n88Ax9BpqqJYZsy16e1U4MONnRp2EEY5RM7WeYvICzOppDGzYja0qQiHiwCplCRJmKu3oHN59M5oO\n/VYAYCeHmxxu4tQl91JIVhEjSIPqJqn6yFYH2OVpJn7BdCY1g/ZmH8zqJHLGGfgbne9/8I8cLiom\nAAl/MSyaxoG5dt4dAn13h6AQDClkvQFMB1iBOlT9barTKpnuRvrT06ja/ibdmWrsZo7J7MfJyfX9\nj9TBAWc7Ji6iQJQmOjFNiJke4vF6EvEdJHtr6GyeS9Ffj6n1Yir/H9umL2TKsnl4JlVT3egh3ZXE\nyBtoDg1vs0w/ISYGCX8xYu/dIRiGNbgzUOz4l5yF94wgA398B2XvflKTFjPw8jsU+x1oRoEG+sZs\nB8CBdt97DSQXSWpJYrETswiRXS8Rp5GsWoPPVaS4s5Nd23bjuHYpzt43MIN1pEsuCk4vfrOdKef5\nCDYog1kvh4uKcUrCX5Td4TsDFRonETqnnui2Pvr+sIPudBVxZz37B2ZxWfL/pYnOocM+bYzdjuBQ\nh+4QJpOgngQFUyWZ9qCkw7jCneS2/Yku39nYahw4A26cLgeqx0a40wPz6wme14jmssvhomJckvAX\np4Rm1wieMwlCIToDcQovvcFAR4jn9gRZFHueqeygigzVpIfGAWzvu8ZTy8bgOQbVxKklTpY9pHPV\nTM/9jUKvnTxu8q4Azho72eY5WNZ0BvbsIXbWhei5PDV+C6+io/qPPKv4sNsyPiBOEQl/cUr5gyr+\nM4N0JM7DVdOLqdl5Zd+HSfdvYAbbMVCwU0QnDxz9rN+xZjvw4yFNifSB8wvAzGoMZINU93awd9/5\nVLlNgi/+CaZMIXLG+eRTRSb52jEmNxFP2zDyBZzh16k6oxHNrsv4gDilJPzFKaVpcM45Bhg+4q4c\n1J5L/O3pvB2fgmPvb7Ai23BlwmgU0MnhJoWFiZ0TOyP4VDr0CKLBbiIDB70UTfB37yONBxMX/d0p\narbvJDJlPnl/NcmGAo7pDYNTY7tcZHbEqa1TyMRMSmhYqQG8c+ol/8WokvAXp5zdDuctgOi0OqyB\nGJHzmkh26ezuXknib3/C2b2Txv2vk4yb5LFRSw81xKkhjB0LjfH7wj34rWDw9yQGSdR9fyGxr57G\nnW+zL3QexcYU0QGoKfQTVlz4XWmSTZMITPOiaBbs76LLVUdjo4mekENGxegY9ffQ66+/zs9+9jNM\n02Tp0qUsX758tJsUE4CmQTCkQiiAf06QN97w4JussNM3BefOt3nTv4S6gXdIdqaYnd1KfzHJdN7G\nQQ4PSewYOMb6jzgO29BPEidZzIxK4+40kc49JHb0kmoIUGtPki46iG/P4J4LjjmNqG43jlgvqUQJ\nf50mh4yKUTGq4W+aJo8//jhf+9rXqK2t5b777mPBggU0NTWNZrNigqmrg2DQxG5XyOV0upWzSWsD\neBurMNx9ZPYNEMpm6bfORs8myRX3E6Qfk8KYHiZ6ojTAQ4kS4KAHT2GAQN9euvuaMBUnOc8knL4k\nuyIBGrtjeC47k6rXNtEz5XxKioZGiRolhm6WIJvFnDlTdgBixEY1/Nvb22loaCAUCgGwaNEiNm/e\nLOEvDqNp0NQ0eKJYba3FtGnQNydAMuan/tIIxb551G34ProNHKUaSvsdRPa78BUGsEihUhyaLG68\ndgfBuyeWOSlSRS9BehmwatATUWwJHW3AQbI/RLIvg0cvUNWwl/S552C5XITVKuoaVGp8OVT5BiDK\nYFTfK5FIhNra2qHbtbW17Nix47Bl2traaGtrA6C1tZXgBDgRRtd1qbOMdF0nFApy4DMCMHgpxv37\nQdf9mCYU6q9Hf3sLNcoAzgtmENkTJ7ytD9/e17CnUhQpYlDEe+A6Ae89cWu8OdhlZSdGPTHyQDJT\nQ02mk0xsH5kps6nWC5Q25ch7G4jNvJBkyomn1sa50wM4VRWO8X87Ef7fJ0KNMHHqPBmjGv6WdeRp\nLYpy+Jf0lpYWWlpahm6Hw+HRLKksgsGg1FlGx6rT6Xz3WryOMxrxOnqxnI3ko1Gq3Q6qAwqdl3yC\n3KZ20hkDR08Pvel+GunETQob4/ubALz7bcAGuIhRJIY9H8O5K0YiMZWMZxL9ikn69T/QO/MinHOa\n2ZtK8cHFRXRFOer1ikOh8f//PtFfm+NNY2PjsJ8zqu+N2tpaBgYGhm4PDAzgl5kRxQk67Ezhunos\n1xmYPT2gqmCaMGUK9VOmUPh//p5I21ukXtlFfscetvXPIJTaSohunAcuHDPeO0jUAz+DA8QZ4kYv\nem8GozdGlTMBTj8+xUXOzJDrMOhLZ2i8pJ9eowGlLoiiaxSLCl1dyrG+EAhxmFEN/5kzZ9Ld3U1f\nXx+BQICNGzfyxS9+cTSbFKcrTcOcNg3L58MslbAUBSWXA7sdm6IQWr6Auouayb34KsldETpf8RAO\nD+DJ9lBDNz4iQ1NH2BjfOwOHx6vZAAAXnklEQVQbUEuOPDl8hEnm9hDO1eKO7cHa7kL1uenpnEOp\nL05xag4aczjnNKHZNXQdIpF3r0QpxLGMavhrmsatt97KQw89hGmaLFmyhObm5tFsUpzONA0rGBya\nI8c6dOpkpxPOP5fqKidV/WFSjQP0v5OkI2xRnegi0PEmNWYfdrJ4iVBDCjvjdyegAIPXD7NwEKeG\nOJPZT6xYRywcIl3KY0Y6SDTHMM4+h93bPNTMqsVvDeCbk6PalzjyvACZalocYtS7RM8//3zOP//8\n0W5GVKL37AwAzGAQLZdj+iVgeAMEEilSA17iSz/Ijt0ZmhNvMWnLiyTSCVwkqSFMFcVxPTagD/0U\ncNKJnzC52B76Yg3YOgboeLuAt2kHuS3NJKYG2b4dPjC3H3uVRmnhQnC5wDBQu7pkqmkxZDy/5oUY\nNisYxMxm0ex25szqoSviQ5vhRvU0MPVDRXoTHyDeVofy5laC6X10RhupTvdRTT8ucoedoTveHNwJ\nOMjjIo+LBJl8J3RmeSd1EfreKJ7dOWrObGaH2cC8s4vYNmygdM45WNEY8RgUqv1odgWfDzSZarqi\nSfiL04umYTY3D3ZvBAI0ptPgdmPZbVj+epo0DeOya8n+QqWnfQ5qRwepaBwrniAXzjKJDmoYwE0G\nB+NvUjkYrMkB6BTRiTGdLfjjAyhxnWS+GctjofXtI5yppdpIEOkwiLubcJfi1BjvYAQChD21BM+o\nQVNkqulKJeEvTj9H6Q467GGfG/dn/5Ez3noLLTUNs1DAqPbQscNgz5/bifz1D3hj+/ERRsGiigR2\njHF36KgGuACNOA5KaGjURAbIvNZD2BEi12XHaAxARxrT1knSpjNQVUNdU4yq+gIZK4P7nClj/WeI\nMTKeXstCnDouF8aFF2IcvG0YNM8M43MWSVz0aXb+cR+Bva/jjIWpKsZQ0wmqiFJDBAfjp2tI5eDA\ncBoTBY0sWjaNMztALuMjG08TNoOkqhXcHgVvQ5JoPkBeryKY60JtcGJqmgz+ViAJfyEANA21MYT3\ncmDnADWuEB2vz6Y5tZ10OEx1uItYJE0usZtqo5dqktgoAu+epDWWnICBhU4RO0USqKQLdozeCDaX\nTigbIZ/2kDf8uDNRHEaY/NwpKIYhg78VSsJfiEOooSABo4Bvuk7dOXVsf2s69o6d2OO7mFwKE9/l\nJtXZhC/dRT6n4C92YWBRfaBr6OAcQ2OxM9AO/OQAOzlsZLFToCqbwVDsmKjovVlK+WoMpwf2p0j8\nz37UGRBwZvBE4yiNDe9+C5BDQ09rEv5CHErTMBsbUaNRGhpKhJrsWP5FYFyEfyBCdsM2rL19dPYb\nGD0RIuFuGnvepGAqZKJZNKOAToJaYtjID30rOJUDx4NtFvGRJIeDKlIoloo9lSNX9GFlwmRyJZIR\ni+I5U3C8vJfSFB/FeC9+Xw1atgszFELt7ZVDQ09jEv5CvNfRBow1DW3+2fjrGwZnH82bODq2Y+7v\nRQufTaEvRvGdfnp6SiiWyZakyeyel3CTxEkKyFF9yFQToxmfFqBh4iADqKgUMbFRRRQrb1HAQbBz\nC/bkAMlCCquujo6ch/2+KrzFFMF6i6bd+1BqfVg1NYNhryggh4aeViT8hRiGd+cbUlC1GoxpNcSj\ns1EiCTyLspRe30+0UA1FB7/ecx3+ri3MsDrQIr2U8gae2B6a6cBFGteBqajL/SZUhn4soIiBhgU4\nyWMQx4EDO3mshEWiw4OWTWNPqPTPuhBray9mrpZ4b5TmMwtoWpjS9GnU2NPoloGlaRjS/XNakPAX\n4iRZuo5mFQkEVQjWADV459Shbs+Ri/s4d06SnH4lGbuNaEShlCuSf2sX2995jWl9r+CK7WMqu3GQ\nw8Hg8fYq716c5uD4wcEjklTA5PhvWuvAcwevfWyQxnHgORY1pMhRII8bjSJ1+S529HixPCZaRwfh\nugYKf+vFU8yyJ1Vg9uQktl3biU49g8BML6pTlesJnCYk/IU4SZbfj3LIlAlYFjbdYtoV02lGIxqt\np1BQSKXAm1ewLNDP0Ynua6br/84k3xUjHNvG9O6/4kwNoJNGw0DFxE4WOxYGFgVcWAwGv+vAnEQH\nZwE9dP42i8EdxYF5UAHI4TqwrEEJhSIOTGwHvhWo5E0HTiNDsieOHqpC0UMwECPsrkXZt4tYMYPH\nUcRWEyG7M4vjgrmk9iYpdeyAUB3eaTVodtkJTEQS/kKcrAODw0NHxNhsWPX1g+MDHDIdNYMXp+nq\nUtEDNYTcGUoz55J56W8ktk1lIKChFbKU+mN4wp3kFAceI45ayOInQgQv/TRSwkEzO/AQw0YBBRUb\nOVRMitjRUTCwMLChUgIssrixULGRR6OAgY6CSQGdGLVkFRcuJUefVo3q8+GwsiSr6tEyGfqtII6+\nTtIeDbUjh/uiqdhe2YMSrEVVLYrZEv2v9VB3XoPsACYgCX8hRuI4ZxMfshiNjSbRqEox1IgjHaHh\nQ00YS6eyf88FRHuKeF75v4S7phDK7CVvm4yWiLPZNp3GYic9ttnYklEw/NRn92Mjj5rNohlJLAzU\nA29lF1l0LLJUHdhhKDjJUMB34LE8DvIYgKop6E4d0+Mh7msmHPo7phu7sBSTXMqgyqWRV6tx2VUK\nMUi80sf0uXZQLCxNR1EHn5/oiOGfU/v+G0CMOxL+Qpwihw4WQy2ELfRikWm1JjN6e1Fmnonxzi72\ndNRhYCNWXU+tlWd/9TXMrC9Qn9yJnk1j9TeR37aHdDRPqmCjT2kgmOmkKhvFzPViM0sUbFX0uObi\nUHKkC3lKio19RQchswe9mMaGga5buGoNOnyTcZ81hU7NTbp+Lo72rTiqVbBMVG8VWilGyd2Ans9R\n6E3jslmUAkHUaATT68PIG8f708U4JOEvxBg5dMzADIVQbDb0bJYpZ3pIWj4Cqh3FU8WsswYv1KL0\nTkXt60ONRPAuOgN/psDuqA/X2wn2Fi5h8sAbGLEuPPFuEjWNeLwhHFUa4X6TRN4Bmo101oNeSKFn\nUhRdPmxTm2ia6aMmswXfBdPQG2rpV2aQ6UkyNbsdm9tFwu6jyl5EHxiABFgEUFQNxSih9PZgTZk8\n1ptSnAQJfyHGynvGDMxJkzDOOAMlkcB7lLNqrWAQq1DACAYHB5hLJab2R4g0nEFVe4T01EWYXh39\nzNlk3t6OMxWmxp6kxuvmnb1eLFUj091AVecOSoUSXreBTUlRZdlwnj8Hu6efEgWUCxqIdVTjLBSx\nFZJouTQF1UN1QzXE+tAjYQqBABYqhgF+n8wLOhFJ+Asxlo4yZnDMMYT3DjA7nXDOWYRCNnqmzqAq\n3oOqgO6pRpkzC1/9DILBImp3N0p1in17DIyADY9Wg7OYRnPqmJqNYl0dzlyaaY15lEaF6WqE3c31\ndL1pocb70Ku9OJ3gVsLUNgXIRnJoyRRWXQDfrACaamGeos0lykfCX4iJ5Cg7C7vdor7eJJvTsPIl\nbDYIBk1sNgtUFaO5GZwmgfQOqPPh6I1iqAGMaJKiuwanYuJrdqPkM5iKgm6VmDbNxO3ykH+pAzQN\nnx/8FugFE21e0+COqM4PloWlS4xMRKP2v/b000/z+9//Hq/XC8BNN90kl3MUYhT4/RZdXQruKTXY\n+7rw+SzCA+DzmlAqYTU24g1q7CtOQk/GcHm8uLr3ga4TDChoPi+aZmJWVQ2Guc2GpkGoUUW5tAE1\nHEYxTSwlgJnLgaJgaRpY1uD66+vHehOIkzCqu+wPf/jDLFu2bDSbEKLivfcwUt0OIZuJardh+t89\n72DuWbBtWy05wAjWU2NGKPX14A3YMH0eLJvtiDA/OM5gHTiRzSqVUCIRLJ/vsPMaxMQj39eEOA0c\nehhpMBgkHFaOGDew2+GsswyiYS9KZxrNUYtvTg1KIgb5PGYwODhp26FhfpRxBvOssyTwTwOKZVmj\nMlT/9NNPs2HDBlwuFzNmzODmm2/G7XYfsVxbWxttbW0AtLa2UigURqOcstJ1nVKpNNZlHJfUWV6n\nVZ2GAZEIFItgs0EgcEoD/bTaluOA3W4f9nNGFP6rVq0iFosdcf+NN97I7Nmzh/r7n3rqKaLRKHfe\needx19nV1XWy5Zwyg5+swmNdxnFJneUldZbPRKgRJk6djY2Nw37OiLp9HnjggRNabunSpTzyyCMj\naUoIIUQZjdoFhqLR6NDvmzZtorm5ebSaEkIIMUyjNuD77//+73R0dKAoCnV1ddx+++2j1ZQQQohh\nGrXw/8IXvjBaqxZCCDFCp/K60kIIIcYJCX8hhKhAEv5CCFGBJPyFEKICSfgLIUQFkvAXQogKJOEv\nhBAVSMJfCCEqkIS/EEJUIAl/IYSoQBL+QghRgST8hRCiAkn4CyFEBZLwF0KICiThL4QQFUjCXwgh\nKpCEvxBCVCAJfyGEqEAjuozjSy+9xDPPPENnZycPP/wwM2fOHHps3bp1vPDCC6iqyi233MK55547\n4mKFEEKUx4g++Tc3N3PPPfdw5plnHnb//v372bhxI9/+9re5//77efzxxzFNc0SFCiGEKJ8RhX9T\nUxONjY1H3L9582YWLVqEzWajvr6ehoYG2tvbR9KUEEKIMhpRt8+xRCIRZs+ePXQ7EAgQiUSOumxb\nWxttbW0AtLa2EgwGR6OkstJ1XeosI6mzvCZCnROhRpg4dZ6M44b/qlWriMViR9x/4403snDhwqM+\nx7KsEy6gpaWFlpaWodvhcPiEnztWgsGg1FlGUmd5TYQ6J0KNMHHqPFoPzPEcN/wfeOCBYa+0traW\ngYGBoduRSIRAIDDs9QghhBgdo3Ko54IFC9i4cSPFYpG+vj66u7uZNWvWaDQlhBDiJIyoz3/Tpk08\n8cQTJBIJWltbmTZtGvfffz/Nzc1ccskl3H333aiqymc+8xlUVU4pEEKI8WJE4X/hhRdy4YUXHvWx\na6+9lmuvvXYkqxdCCDFK5OO4EEJUIAl/IYSoQBL+QghRgST8hRCiAkn4CyFEBZLwF0KICiThL4QQ\nFUjCXwghKpCEvxBCVCAJfyGEqEAS/kIIUYEk/IUQogJJ+AshRAWS8BdCiAok4S+EEBVIwl8IISqQ\nhL8QQlQgCX8hhKhAI7qM40svvcQzzzxDZ2cnDz/8MDNnzgSgr6+PlStX0tjYCMDs2bO5/fbbR16t\nEEKIshhR+Dc3N3PPPffw4x//+IjHGhoaePTRR0eyeiGEEKNkROHf1NRUrjqEEEKcQiMK//fT19fH\nV77yFVwuFzfeeCNnnnnmaDUlhBBimI4b/qtWrSIWix1x/4033sjChQuP+hy/38/atWvxeDzs2rWL\nRx99lDVr1lBVVXXEsm1tbbS1tQHQ2tpKMBgc7t9wyum6LnWWkdRZXhOhzolQI0ycOk/GccP/gQce\nGPZKbTYbNpsNgBkzZhAKheju7h4aED5US0sLLS0tQ7fD4fCw2zvVgsGg1FlGUmd5TYQ6J0KNMHHq\nPHhwzXCMyqGeiUQC0zQB6O3tpbu7m1AoNBpNCSGEOAkj6vPftGkTTzzxBIlEgtbWVqZNm8b999/P\n1q1befrpp9E0DVVVue2223C73eWqWQghxAiNKPwvvPBCLrzwwiPuv/jii7n44otHsmohhBCjSM7w\nFUKICiThL4QQFUjCXwghKpCEvxBCVCAJfyGEqEAS/kIIUYEk/IUQogJJ+AshRAWS8BdCiAok4S+E\nEBVIwl8IISqQhL8QQlQgCX8hhKhAEv5CCFGBJPyFEKICSfgLIUQFkvAXQogKJOEvhBAVSMJfCCEq\n0Iiu4fvkk0/yyiuvoOs6oVCIO++8k+rqagDWrVvHCy+8gKqq3HLLLZx77rllKVgIIcTIjeiT/znn\nnMOaNWtYvXo1kyZNYt26dQDs37+fjRs38u1vf5v777+fxx9/HNM0y1KwEEKIkRtR+H/gAx9A0zQA\n5syZQyQSAWDz5s0sWrQIm81GfX09DQ0NtLe3j7xaIYQQZTGibp9DvfDCCyxatAiASCTC7Nmzhx4L\nBAJDO4b3amtro62tDYDW1lYaGxvLVdKokjrLS+osr4lQ50SoESZOncN13E/+q1at4ktf+tIRP5s3\nbx5a5tlnn0XTNC677DIALMs64QJaWlpobW2ltbWVe++99yT+hFNP6iwvqbO8JkKdE6FGOL3rPO4n\n/wceeOB9H//jH//IK6+8woMPPoiiKADU1tYyMDAwtEwkEiEQCAy7OCGEEKNjRH3+r7/+Os899xxf\n/epXcTgcQ/cvWLCAjRs3UiwW6evro7u7m1mzZo24WCGEEOWh/fM///M/n+yTH3roIQqFAn/5y1/4\n3//9Xzo6Orjgggvw+XykUil+9KMf8ec//5lbb731hPvNZsyYcbLlnFJSZ3lJneU1EeqcCDXC6Vun\nYg2ng14IIcRpQc7wFUKICiThL4QQFahsx/mPxESZJuKll17imWeeobOzk4cffpiZM2cC0NfXx8qV\nK4fGNWbPns3tt98+7uqE8bU9D/X000/z+9//Hq/XC8BNN93E+eefP8ZVDXr99df52c9+hmmaLF26\nlOXLl491SUd111134XQ6UVUVTdNobW0d65IAWLt2La+++io+n481a9YAkEqleOyxx+jv76euro6V\nK1fidrvHXZ3j8XUZDof5/ve/TywWQ1EUWlpauOaaa4a/Ta1x4PXXX7dKpZJlWZb15JNPWk8++aRl\nWZa1b98+65577rEKhYLV29trff7zn7cMwxizOvft22d1dnZaX//616329vah+3t7e6277757zOp6\nr2PVOd6256Geeuop67nnnhvrMo5gGIb1+c9/3urp6bGKxaJ1zz33WPv27Rvrso7qzjvvtOLx+FiX\ncYQtW7ZYO3fuPOw98uSTT1rr1q2zLMuy1q1bN/SeH0tHq3M8vi4jkYi1c+dOy7IsK5PJWF/84het\nffv2DXubjotun4kyTURTU9OEONvvWHWOt+05EbS3t9PQ0EAoFELXdRYtWnTYCY7i+ObNm3fEJ9DN\nmzezePFiABYvXjwutunR6hyP/H7/0JE9LpeLyZMnE4lEhr1Nx0W3z6FOdpqIsdbX18dXvvIVXC4X\nN954I2eeeeZYl3SE8b49f/e73/Hiiy8yY8YMbr755nHxRoxEItTW1g7drq2tZceOHWNY0ft76KGH\nALjiiitoaWkZ42qOLR6P4/f7gcEwSyQSY1zRsY3H1+VBfX197N69m1mzZg17m56y8F+1ahWxWOyI\n+2+88UYWLlwIjGyaiHI5kTrfy+/3s3btWjweD7t27eLRRx9lzZo1VFVVjas6x2J7Hur9ar7yyiu5\n7rrrAHjqqaf4xS9+wZ133nmqSzzC0bbZwTPZx5tVq1YRCASIx+N84xvfoLGxkXnz5o11WRPaeH1d\nAuRyOdasWcOnP/3pk8qaUxb+E2WaiOPVeTQ2mw2bzQYMnmgRCoXo7u4+bKC13E6mzrGeduNEa166\ndCmPPPLIKFdzYt67zQYGBoY+XY03B/8vfT4fCxcupL29fdyGv8/nIxqN4vf7iUajQwOq401NTc3Q\n7+PpdVkqlVizZg2XXXYZF110ETD8bTou+vwn+jQRiURi6HoFvb29dHd3EwqFxriqI43n7RmNRod+\n37RpE83NzWNYzbtmzpxJd3c3fX19lEolNm7cyIIFC8a6rCPkcjmy2ezQ72+++SZTpkwZ46qObcGC\nBWzYsAGADRs2HPPb6lgbj69Ly7L44Q9/yOTJk/nIRz4ydP9wt+m4OMP3C1/4AqVSaagv7dBDJZ99\n9ln+8Ic/oKoqn/70pznvvPPGrM5NmzbxxBNPkEgkqK6uZtq0adx///385S9/4emnn0bTNFRV5frr\nrx/TgDhWnTC+tuehvve979HR0YGiKNTV1XH77bePm0/Yr776Kj//+c8xTZMlS5Zw7bXXjnVJR+jt\n7WX16tUAGIbBpZdeOm7q/M53vsPWrVtJJpP4fD5uuOEGFi5cyGOPPUY4HCYYDHL33XePeV/60erc\nsmXLuHtdbtu2jQcffJApU6YM9ZLcdNNNzJ49e1jbdFyEvxBCiFNrXHT7CCGEOLUk/IUQogJJ+Ash\nRAWS8BdCiAok4S+EEBVIwl8IISqQhL8QQlSg/x/NeCjXbAiX0AAAAABJRU5ErkJggg==\n", + "text/plain": [ + "\u003cFigure size 600x400 with 1 Axes\u003e" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "posterior_samples = surrogate_posterior.sample(50)\n", + "_, _, x_generated = model.sample(value=(posterior_samples))\n", + "\n", + "# It's a pain to plot all 5000 points for each of our 50 posterior samples, so\n", + "# let's subsample to get the gist of the distribution.\n", + "x_generated = tf.reshape(tf.transpose(x_generated, [1, 0, 2]), (2, -1))[:, ::47]\n", + "\n", + "plt.scatter(x_train[0, :], x_train[1, :], color='blue', alpha=0.1, label='Actual data')\n", + "plt.scatter(x_generated[0, :], x_generated[1, :], color='red', alpha=0.1, label='Simulated data (VI)')\n", + "plt.legend()\n", + "plt.axis([-20, 20, -20, 20])\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "rWrL1mCNIcNy" + }, + "source": [ + "## Acknowledgements\n", + "\n", + "This tutorial was originally written in Edward 1.0 ([source](https://github.com/blei-lab/edward/blob/master/notebooks/probabilistic_pca.ipynb)). We thank all contributors to writing and revising that version." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "ZJd8RbQxvyp6" + }, + "source": [ + "#### References\n", + "\n", + "\u003ca name='1'\u003e\u003c/a\u003e[1]: Michael E. Tipping and Christopher M. Bishop. Probabilistic principal component analysis. _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_, 61(3): 611-622, 1999." ] - }, - "metadata": { - "tags": [] - }, - "output_type": "display_data" } - ], - "source": [ - "posterior_samples = surrogate_posterior.sample(50)\n", - "_, _, x_generated = model.sample(value=(posterior_samples))\n", - "\n", - "# It's a pain to plot all 5000 points for each of our 50 posterior samples, so\n", - "# let's subsample to get the gist of the distribution.\n", - "x_generated = tf.reshape(tf.transpose(x_generated, [1, 0, 2]), (2, -1))[:, ::47]\n", - "\n", - "plt.scatter(x_train[0, :], x_train[1, :], color='blue', alpha=0.1, label='Actual data')\n", - "plt.scatter(x_generated[0, :], x_generated[1, :], color='red', alpha=0.1, label='Simulated data (VI)')\n", - "plt.legend()\n", - "plt.axis([-20, 20, -20, 20])\n", - "plt.show()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "rWrL1mCNIcNy" - }, - "source": [ - "## Acknowledgements\n", - "\n", - "This tutorial was originally written in Edward 1.0 ([source](https://github.com/blei-lab/edward/blob/master/notebooks/probabilistic_pca.ipynb)). We thank all contributors to writing and revising that version." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text", - "id": "ZJd8RbQxvyp6" - }, - "source": [ - "#### References\n", - "\n", - "[1]: Michael E. Tipping and Christopher M. Bishop. Probabilistic principal component analysis. _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_, 61(3): 611-622, 1999." - ] - } - ], - "metadata": { - "colab": { - "collapsed_sections": [], - "name": "Probabilistic PCA", - "toc_visible": true + ], + "metadata": { + "colab": { + "collapsed_sections": [], + "name": "Probabilistic PCA", + "provenance": [], + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + } }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - } - }, - "nbformat": 4, - "nbformat_minor": 0 + "nbformat": 4, + "nbformat_minor": 0 }