{"cells": [{"cell_type": "markdown", "id": "32cc9c24", "metadata": {"papermill": {"duration": 0.008019, "end_time": "2025-04-03T19:27:05.257930", "exception": false, "start_time": "2025-04-03T19:27:05.249911", "status": "completed"}, "tags": []}, "source": ["\n", "# Tutorial 10: Autoregressive Image Modeling\n", "\n", "* **Author:** Phillip Lippe\n", "* **License:** CC BY-SA\n", "* **Generated:** 2025-04-03T19:26:58.602358\n", "\n", "In this tutorial, we implement an autoregressive likelihood model for the task of image modeling.\n", "Autoregressive models are naturally strong generative models that constitute one of the current\n", "state-of-the-art architectures on likelihood-based image modeling,\n", "and are also the basis for large language generation models such as GPT3.\n", "We will focus on the PixelCNN architecture in this tutorial, and apply it to MNIST modeling.\n", "This notebook is part of a lecture series on Deep Learning at the University of Amsterdam.\n", "The full list of tutorials can be found at https://uvadlc-notebooks.rtfd.io.\n", "\n", "\n", "---\n", "Open in [![Open In Colab](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHUAAAAUCAYAAACzrHJDAAAIuUlEQVRoQ+1ZaVRURxb+qhdolmbTUVSURpZgmLhHbQVFZIlGQBEXcMvJhKiTEzfigjQg7oNEJ9GMGidnjnNMBs2czIzajksEFRE1xklCTKJiQLRFsUGkoUWw+82pamn79etGYoKek1B/4NW99/tu3e/dquJBAGD27NkHALxKf39WY39gyrOi+i3xqGtUoePJrFmznrmgtModorbTu8YRNZk5cybXTvCtwh7o6NR2KzuZMWNGh6jtVt7nA0ymT5/eJlF9POrh7PAQl6s8bGYa3PUum//htmebVtLRqW0q01M5keTk5FZFzU0oRle3+zxwg5Hgtb+PZiL/ZVohxCI+hL5JgjmfjPxZ26+33BG3dA+ealHPM4gQAo5rU59gsI8bRvl54t3Ca62mvHyUAhtOlLd5WSQpKcluBjumnoCLs1EARkVd9E8l3p9y2i7RbQ1B6pFwu/YDgW8KbHJHMTQrwnjz2oZm9M4pavOCfo5jWrgCaaMVcMs6/pNhDr0+AMN93XlxV7R6DNpyzi7W/OE+yIrsjU6rTrbKV5cd/pNyItOmTbMp6sbBB+EqaYJY4cWE3VUciNt1TpgfcRFv71Fi54xT5kSoyLvOBEJMOMxWXkFlBeBSX4u6Zkcs+3KszYRtiapbNRqF31UgetVuc8z9vBXIv1qD+F1f83B6uDlCUyfsZGepGPpmg01OB7EITQbhS9ribKy+DmP1DUiClLz4bnIHVOqa7BY+Z1wg5g3zgUvyehiNpnJKxSLc/ts76LKm0BzX3c0RNy1yXjDcB5lWoro4iNHQxM+f1kWeWQARAWQS++trISJTp061Kep25X/MycwtjuctSC5rxo7ppi7VNUox5+PhPHtrsS2O1qJ6yx1QujQUzm9sh6hbkBlvvGcN8hYnwjUjH6kjfZEd5c/jitz5Jc5U3ENnFynKl4eB7nyEgP2UZ+Yz3/rVEbyYr27qELrtC4FIC0J7sc7xWnmccdHfRRTs0VB+cA4lt+oFcRR/wUeH8FG5w2Mbx8FQ8TXEvv1xYf4wBP3O2WyL3/UVjpXWgIqaFeUPr+wTmDvUB7njH6/bOv+HRg4SqioAg5GDe1aB3ZeMTJkyRSBqkLsWqSEm0fZVBEN94zEZnYvrdx1JL5cxe+a+AbhSJecRRHW/ikTFRTa38dtQlNZ5CRKwFvUtZU/kvBoEF9Uxni/XqIM+dwKbTw3rhcxIf7gmr2M+H6SMwx8iBzJbw5oxeG3Lv5FX9B3AGaHPS8e8z77H7v9VMpvPG5ug1enh7eGK8h0LBTwUb+GInqzInlRUK65DmTPQu4c3+uQKjwKK77zwUxBX4Tq7yR1RuiwUsqlrABCM6esHdXoy47fk4+prYKy8ZF574x4V5BnHQBuf4g9Z9ld8U36L2aktZNNplNfw7zotwWTy5MkCUft4aLEopJj5/OPHl1BQqeAVOnHgNSQOqmBzq9V9cfEm/yx5ubMGKS9cYPZ3vx2OS/c6PVHUuUO7Y1Pci3BO/1zgq18byebfGemLtNF+6JRtOvMk926ibussZqM+1mNz4TWkH7rCbM5phwGRGDAaoF8fY5OHFnlldAA8sgoEXKnDukA1NgSeNjqkJT9brbN4pC9WRweYXyLugR73c+MYvyWfu0yC6+mjzN1Isfw3FKJS98CU/zI1IHFkFPR52cHL2FJk0sB6kMTERIGo9GzcPkLNfA0cwdwi/hfEYO86ZMd9w+y1egfM2T2Eh/vesMNwljSzuZRT420SW3eqy8N6aHMmwmnFUZ7/PGVPbIoNZvNU1BURdHs0bT2+HjL8sDSM2e6vi4Lj5NW8WOLVA6RTT2azxLV+bglaFNqLieqemS/gWkw7NyoAHo+2dEsiivengjKsPFoqWOvbSh/kxPaxyW/JRzH2Fl3EzD9/xjAefJqB3usKUFn/0Gb+S/d/jy3FN2yLOmnSJJtn6oehByEiHPSeXnDxFGPRnoFoaBJjcdQlbDwcjL1zTNuQpoxD7R0OG0uUTMi0fkVwdzBdYIwcwZunxrVJVLplNm54BZp7jfDfYLoNyqQi1K6KxIdHzmN+QQ2WjFIwUT2zTGdlRXo4NFXVUO4sgX5dFC7f0aP/ZlNeUjFBuL8Xjl6uRuP6aMjSjpjzsH62FDU7JhBuGccEXIvDfJFFBc/gHw80dklfCVYnRaDfpiJcutPA4F7qJsfJeUPQI+1fqMlNhFx1FM0GDqkjFVg7NojlQ0Vt4aM5ReSqcbpaCg8nCW5lRsBvbT4T1TLfFptsfh7gItzuKTdJSEiwKSrt1vcmnEXXrsLbYnWDA1bu+z2WKy9Arq+1KRqdfKsoBo0GcdtEpS/B1bO4v0cFiUhkjskvKcMrWwtAPHuwQq8Z+4LZ1vTQANfXt4J0DwZX9gWa9qh4XDM/voC9JXfwYEMMHJcfNtusn82ihvliVUwg5KrPGVf6GH94ZJpEZBen6EC4qYTHA1dXhW0JIex8txzv//c8lhzXIi/BFxOH9jGbQhZsRalTIBZZ8KkGyZAxeRQvXkFF1TWz/Hm46jNYUnjPbt3JxIkT7f6dSj8qfJJyVvBxgaIlblOyjtysNHWN9fjjqWi7glJfW3/S0Hlj2XnA8PhKT9w6g3Qx3XiXhvuxQsuT1proxBKI/AaZqY1Xz5muvY8G8XkRRCaHsfQsRAFDH/tZPbcYuHotOG0FRIqB4HR3wNVoIPLtz8ycTguu+jpEigE218vd1YCr5m+HpHMvEI9u4LTXwNWaLjl0iPwGAmIpeHx1VeCqTJdPs1/vweweQPO3HC24NhOhnTphwoQnfv6QSY2ICbkNmdSA4h87oaLaiYfn5diIEd4att2erOwJXbPUHp953p6orQVSUVWRAXBT8c/dJ5L9xhzaJGp71GR/wFP8P5V2z10NSC9T93QM2xUg8fHxT+zU9ijeU4naHon8CjFJXFzc8/kn+dN06q9QgF98SYSo2Xen2NjYZy5sR6f+4nLSK5Iam2PH/x87a1YN/t5sBgAAAABJRU5ErkJggg==){height=\"20px\" width=\"117px\"}](https://colab.research.google.com/github/PytorchLightning/lightning-tutorials/blob/publication/.notebooks/course_UvA-DL/10-autoregressive-image-modeling.ipynb)\n", "\n", "Give us a \u2b50 [on Github](https://www.github.com/Lightning-AI/lightning/)\n", "| Check out [the documentation](https://lightning.ai/docs/)\n", "| Join us [on Discord](https://discord.com/invite/tfXFetEZxv)"]}, {"cell_type": "markdown", "id": "9dd2d5ea", "metadata": {"papermill": {"duration": 0.007183, "end_time": "2025-04-03T19:27:05.271868", "exception": false, "start_time": "2025-04-03T19:27:05.264685", "status": "completed"}, "tags": []}, "source": ["## Setup\n", "This notebook requires some packages besides pytorch-lightning."]}, {"cell_type": "code", "execution_count": 1, "id": "cead9f66", "metadata": {"colab": {}, "colab_type": "code", "execution": {"iopub.execute_input": "2025-04-03T19:27:05.286464Z", "iopub.status.busy": "2025-04-03T19:27:05.286109Z", "iopub.status.idle": "2025-04-03T19:27:06.470559Z", "shell.execute_reply": "2025-04-03T19:27:06.469307Z"}, "id": "LfrJLKPFyhsK", "lines_to_next_cell": 0, "papermill": {"duration": 1.194088, "end_time": "2025-04-03T19:27:06.472558", "exception": false, "start_time": "2025-04-03T19:27:05.278470", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.\u001b[0m\u001b[33m\r\n", "\u001b[0m"]}, {"name": "stdout", "output_type": "stream", "text": ["\r\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.0.1\u001b[0m\r\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython -m pip install --upgrade pip\u001b[0m\r\n"]}], "source": ["! pip install --quiet \"matplotlib\" \"numpy <3.0\" \"torchvision\" \"torch >=1.8.1,<2.7\" \"seaborn\" \"pytorch-lightning >=2.0,<2.6\" \"torchmetrics >=1.0,<1.8\""]}, {"cell_type": "markdown", "id": "5cbad2c8", "metadata": {"papermill": {"duration": 0.006773, "end_time": "2025-04-03T19:27:06.486621", "exception": false, "start_time": "2025-04-03T19:27:06.479848", "status": "completed"}, "tags": []}, "source": ["
\n", "\n", "Similar to the language generation you have seen in assignment 2, autoregressive models work on images by modeling the likelihood of a pixel given all previous ones.\n", "For instance, in the picture below, we model the pixel $x_i$ as a conditional probability distribution\n", "based on all previous (here blue) pixels (figure credit - [Aaron van den Oord et al. ](https://arxiv.org/abs/1601.06759)):\n", "\n", "
\n", "\n", "Generally, autoregressive model over high-dimensional data $\\mathbf{x}$ factor the joint distribution as the following product of conditionals:\n", "\n", "$$p(\\mathbf{x})=p(x_1, ..., x_n)=\\prod_{i=1}^{n} p(x_i|x_1,...,x_{i-1})$$\n", "\n", "Learning these conditionals is often much simpler than learning the joint distribution $p(\\mathbf{x})$ all together.\n", "However, disadvantages of autoregressive models include slow sampling, especially for large images,\n", "as we need height-times-width forward passes through the model.\n", "In addition, for some applications, we require a latent space as modeled in VAEs and Normalizing Flows.\n", "For instance, in autoregressive models, we cannot interpolate between two images because of the lack of a latent representation.\n", "We will explore and discuss these benefits and drawbacks alongside with our implementation.\n", "\n", "Our implementation will focus on the [PixelCNN](https://arxiv.org/abs/1606.05328) [2] model which has been discussed in detail in the lecture.\n", "Most current SOTA models use PixelCNN as their fundamental architecture,\n", "and various additions have been proposed to improve the performance\n", "(e.g. [PixelCNN++](https://arxiv.org/abs/1701.05517) and [PixelSNAIL](http://proceedings.mlr.press/v80/chen18h/chen18h.pdf)).\n", "Hence, implementing PixelCNN is a good starting point for our short tutorial.\n", "\n", "First of all, we need to import our standard libraries. Similarly as in\n", "the last couple of tutorials, we will use [PyTorch\n", "Lightning](https://lightning.ai/docs/pytorch/stable/) here as\n", "well."]}, {"cell_type": "code", "execution_count": 2, "id": "4ff56ec6", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:06.501834Z", "iopub.status.busy": "2025-04-03T19:27:06.501464Z", "iopub.status.idle": "2025-04-03T19:27:10.057972Z", "shell.execute_reply": "2025-04-03T19:27:10.056814Z"}, "papermill": {"duration": 3.566953, "end_time": "2025-04-03T19:27:10.060293", "exception": false, "start_time": "2025-04-03T19:27:06.493340", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Seed set to 42\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Using device cuda:0\n"]}, {"data": {"text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["\n", "import math\n", "import os\n", "import urllib.request\n", "from urllib.error import HTTPError\n", "\n", "# Imports for plotting\n", "import matplotlib.pyplot as plt\n", "import matplotlib_inline.backend_inline\n", "import numpy as np\n", "import pytorch_lightning as pl\n", "import seaborn as sns\n", "import torch\n", "import torch.nn as nn\n", "import torch.nn.functional as F\n", "import torch.optim as optim\n", "import torch.utils.data as data\n", "import torchvision\n", "from matplotlib.colors import to_rgb\n", "from pytorch_lightning.callbacks import LearningRateMonitor, ModelCheckpoint\n", "from torch import Tensor\n", "from torchvision import transforms\n", "from torchvision.datasets import MNIST\n", "from tqdm.notebook import tqdm\n", "\n", "plt.set_cmap(\"cividis\")\n", "%matplotlib inline\n", "matplotlib_inline.backend_inline.set_matplotlib_formats(\"svg\", \"pdf\") # For export\n", "\n", "# Path to the folder where the datasets are/should be downloaded (e.g. MNIST)\n", "DATASET_PATH = os.environ.get(\"PATH_DATASETS\", \"data\")\n", "# Path to the folder where the pretrained models are saved\n", "CHECKPOINT_PATH = os.environ.get(\"PATH_CHECKPOINT\", \"saved_models/tutorial12\")\n", "\n", "# Setting the seed\n", "pl.seed_everything(42)\n", "\n", "# Ensure that all operations are deterministic on GPU (if used) for reproducibility\n", "torch.backends.cudnn.deterministic = True\n", "torch.backends.cudnn.benchmark = False\n", "\n", "# Fetching the device that will be used throughout this notebook\n", "device = torch.device(\"cpu\") if not torch.cuda.is_available() else torch.device(\"cuda:0\")\n", "print(\"Using device\", device)"]}, {"cell_type": "markdown", "id": "0b61f8ce", "metadata": {"papermill": {"duration": 0.015212, "end_time": "2025-04-03T19:27:10.091135", "exception": false, "start_time": "2025-04-03T19:27:10.075923", "status": "completed"}, "tags": []}, "source": ["We again provide a pretrained model, which is downloaded below:"]}, {"cell_type": "code", "execution_count": 3, "id": "be4c7b14", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:10.117748Z", "iopub.status.busy": "2025-04-03T19:27:10.117213Z", "iopub.status.idle": "2025-04-03T19:27:10.313199Z", "shell.execute_reply": "2025-04-03T19:27:10.311791Z"}, "papermill": {"duration": 0.209633, "end_time": "2025-04-03T19:27:10.315655", "exception": false, "start_time": "2025-04-03T19:27:10.106022", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Downloading https://raw.githubusercontent.com/phlippe/saved_models/main/tutorial12/PixelCNN.ckpt...\n"]}], "source": ["# Github URL where saved models are stored for this tutorial\n", "base_url = \"https://raw.githubusercontent.com/phlippe/saved_models/main/tutorial12/\"\n", "# Files to download\n", "pretrained_files = [\"PixelCNN.ckpt\"]\n", "# Create checkpoint path if it doesn't exist yet\n", "os.makedirs(CHECKPOINT_PATH, exist_ok=True)\n", "\n", "# For each file, check whether it already exists. If not, try downloading it.\n", "for file_name in pretrained_files:\n", " file_path = os.path.join(CHECKPOINT_PATH, file_name)\n", " if not os.path.isfile(file_path):\n", " file_url = base_url + file_name\n", " print(f\"Downloading {file_url}...\")\n", " try:\n", " urllib.request.urlretrieve(file_url, file_path)\n", " except HTTPError as e:\n", " print(\n", " \"Something went wrong. Please try to download the file from the GDrive folder, or contact the author with the full output including the following error:\\n\",\n", " e,\n", " )"]}, {"cell_type": "markdown", "id": "4b7a8eb9", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.015372, "end_time": "2025-04-03T19:27:10.346595", "exception": false, "start_time": "2025-04-03T19:27:10.331223", "status": "completed"}, "tags": []}, "source": ["Similar to the Normalizing Flows in Tutorial 11, we will work on the\n", "MNIST dataset and use 8-bits per pixel (values between 0 and 255). The\n", "dataset is loaded below:"]}, {"cell_type": "code", "execution_count": 4, "id": "18b2dc41", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:10.374985Z", "iopub.status.busy": "2025-04-03T19:27:10.374608Z", "iopub.status.idle": "2025-04-03T19:27:12.473405Z", "shell.execute_reply": "2025-04-03T19:27:12.472091Z"}, "papermill": {"duration": 2.11339, "end_time": "2025-04-03T19:27:12.474970", "exception": false, "start_time": "2025-04-03T19:27:10.361580", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n", "Failed to download (trying next):\n", "HTTP Error 404: Not Found\n", "\n", "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz\n", "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz to /__w/11/s/.datasets/MNIST/raw/train-images-idx3-ubyte.gz\n"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", " 0%| | 0/9912422 [00:00 only make them a tensor\n", "transform = transforms.Compose([transforms.ToTensor(), discretize])\n", "\n", "# Loading the training dataset. We need to split it into a training and validation part\n", "train_dataset = MNIST(root=DATASET_PATH, train=True, transform=transform, download=True)\n", "pl.seed_everything(42)\n", "train_set, val_set = torch.utils.data.random_split(train_dataset, [50000, 10000])\n", "\n", "# Loading the test set\n", "test_set = MNIST(root=DATASET_PATH, train=False, transform=transform, download=True)\n", "\n", "# We define a set of data loaders that we can use for various purposes later.\n", "train_loader = data.DataLoader(train_set, batch_size=128, shuffle=True, drop_last=True, pin_memory=True, num_workers=4)\n", "val_loader = data.DataLoader(val_set, batch_size=128, shuffle=False, drop_last=False, num_workers=4)\n", "test_loader = data.DataLoader(test_set, batch_size=128, shuffle=False, drop_last=False, num_workers=4)"]}, {"cell_type": "markdown", "id": "6a57a100", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.010974, "end_time": "2025-04-03T19:27:12.497246", "exception": false, "start_time": "2025-04-03T19:27:12.486272", "status": "completed"}, "tags": []}, "source": ["A good practice is to always visualize some data examples to get an intuition of the data:"]}, {"cell_type": "code", "execution_count": 5, "id": "6f129402", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:12.520496Z", "iopub.status.busy": "2025-04-03T19:27:12.520220Z", "iopub.status.idle": "2025-04-03T19:27:12.645255Z", "shell.execute_reply": "2025-04-03T19:27:12.644115Z"}, "papermill": {"duration": 0.13853, "end_time": "2025-04-03T19:27:12.646814", "exception": false, "start_time": "2025-04-03T19:27:12.508284", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzQxLjY3NDgzODcwOTcgMTgwLjcyIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nFWOSw7CMAxE9z7FnCDfKkmXQKWIZWHBAaJQiCioVKLXx61AhcWzPJbHHtnk1zXlQ9xidyS5qjSSRmE6KBRmgkZkOlKserKVFs5XwdYsb79SByW84Zla2wvRmQZ4YRas4TpvB69qD+2csAbPjBPukBv+MvKrwkx8PeI/2LD4HeYgH+v3cOoh9xrNAy219AYPKzF0CmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMTQ4CmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjMgMCBvYmoKPDwgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCA0NTUgL0hlaWdodCAyMzEKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDIyMgoo/////f39+/v7+fn59/f39fX18/Pz8fHx7+/v7e3t6+vr6enp5+fn5eXl4+Pj4eHh39/f3d3d29vb2dnZ19fX1dXV09PT0dHRz8/Pzc3Ny8vLycnJx8fHxcXFw8PDwcHBv7+/vb29u7u7ubm5t7e3tbW1s7Ozr6+vra2tqampp6enpaWloaGhmZmZl5eXlZWVk5OTkZGRj4+PjY2Ni4uLiYmJh4eHhYWFg4ODfX19e3t7eXl5d3d3dXV1c3NzcXFxb29vZ2dnY2NjX19fXV1dW1tbWVlZV1dXVVVVU1NTUVFRT09PS0tLSUlJR0dHRUVFQ0NDQUFBPz8/Ozs7OTk5Nzc3NTU1MzMzMTExLy8vKysrJycnJSUlIyMjHx8fHR0dGxsbGRkZFxcXFRUVExMTERERDw8PXHJcclxyCwsLCQkJBwcHBQUFAwMDAQEB/v7+/Pz8+vr6+Pj49PT08vLy8PDw7u7u7Ozs6urq6Ojo5ubm5OTk4uLi4ODg3t7e3Nzc2tra2NjY1tbW1NTU0NDQzs7OzMzMxsbGxMTEwsLCwMDAvr6+vLy8urq6uLi4tra2tLS0srKysLCwrq6urKysqqqqqKiopKSkoqKioKCgnp6enJycmpqamJiYlpaWlJSUioqKiIiIhoaGhISEgoKCgICAfn5+fHx8eHh4dnZ2dHR0cnJycHBwbm5uampqZmZmZGRkYmJiYGBgXl5eXFxcXFxcWlpaWFhYVlZWVFRUUFBQTExMSkpKSEhIRkZGREREQkJCQEBAPj4+PDw8Ojo6ODg4NDQ0MjIyMDAwLi4uLCwsKioqXChcKFwoJiYmJCQkIiIiICAgHh4eHBwcGhoaGBgYFhYWFBQUEhISEBAQDg4ODAwMXG5cblxuCAgIBgYGBAQEAgICAAAAKQpdCi9CaXRzUGVyQ29tcG9uZW50IDggL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDQ1NSAvQml0c1BlckNvbXBvbmVudCA4ID4+Ci9MZW5ndGggMTQgMCBSID4+CnN0cmVhbQp4nO2d/58WRQHHOUIoSkwNAoqyiAL6AgRFREREJwlpGET0BSlM0JCzODwBM0PlLoOKhKDSS6wsoKgEosvEwzs7EdPokoMj4W/p8+F2X+ztM7M7++U5juHz/gXcZ3c/O/t+8Jmd2ZkZ8FvhAwMu9gWIUpBHP5BHP5BHP5BHP5BHPwg9nusrlFeVPHn0I08e/ciTRz/y5NGPPHn0I08e/ciTRz/y5NFCZ2fnO8GzoE/y8iKPycijU26/z5NHp9x+n3eZejwBFi5cWA/qwAAwZswYe27RPGfy5HWDrVu3siDbQdXziiCPduRRHi9hj58A40BNb5qamuy5RfIykTXvVrAF1PUwB1Q1ryjyaEYe5TFPXlHk0czl5PF/gHWBlpaWRjAI1FRy9913/wWYc8srSApZ87aBwOE9gEWtal5R5NGMPMpjnryiyKOZy8Xjf4ChUmOgQD3nKcCmhMmTJ9t34i1+DSSfKmv53gICj4ZvYTpZ8z4D7ge4Ycx8BLD0p06dypQnjzHkUR7z5MmjPEbysnmcNWvWe0G6Q1JbW5unnCjBqQcBT3H48GHDHv8Eu3fvfgMYCVDbakktp1v5zl3wWF9fvwA4H5c5b+XKlTsBK4l39cASB3+9a8SIEQuBc548xpBHecyU1+ce16xZ800wA6xduzZB3Lp16xaBFwLOnj2bp5zfAOEZ586dy01Uy0tYsmTJh8F6EIl9HKSWM/2GBIQet2zZ4nxMrjz8CFLYG8GfwZ49e6IeAW722qPAKU8eY8ijPGbKk0d5rMxz8Njd3Z0gD/wU/Ay0t7eXUE7UbQ6HZx46dOgk8GZgjn4HYEtvgbwos2fPHgjoccqUKW7H5Mn7OsAzI221B3dt2bJlfwvYCgKX94Gurq70PHmMIo/ymCVPHuXRnFfIY1NTE99fuR2UV05qYSsjX4WN5V1xxRUs42QQbEmu4zjlhfDFnM2bNwdtq3WdnZ3OZcqctx/g6j8CDC3E3IRvFL7EQ1nG5ubm9Dx5DJFHecya16cezwB+1NOpEqUBjAYrVqyIHcP/8Xcl/g/d+b7yUXTfvn3fD+Dj65EjR/jJz0HN+cfVdexFSz6Nc948UHeBwON8cEMPnwPpp0nNewnwBm7YsIF3y7IX/mlcA3glqHmk58ljiDxGkEd5jCGPJXlk/1dl3eZq0ApYuZlbASsnPwb8+9NPP52nnMncBNjxiCtx6x50zuMbnBGP48aNGw/4EBdsYU8hN01JfLRMzWMLcdDDmHA1uMNhl+T111+fsKM8xpBHecySJ49EHs152TwOGTKEPWJ8MW8HqLQcZf369aweZC1nMm8HQVVrOUg/IK9HO1eCjo6OvHkTABXt37/fssdZMHHiRO70EFi9enV6+eQxRB7lMUuePBJ5NOcleAzbaSMMHDjwXmBowjbDR/U2kKWcyYQep02b5nZAXo/4Fm6oBF/kIRywgMpep7klPTFvJbgKUJH9SjhOP+h/TH43N5InjyHyKI+ueRfFI0vjZCuZm4FrOVOYP3/+68EPwOnTp92OyePxOrBq1SrDTnxy5puK2OkVkDXvvyB8rbHysfAkmApeB4Kd0i9bHmPIozy65snjeeTRmnfJeMQTWwcc8oyZ5mDM4/GTwLwTRydsAlBp6TxMzePzI0sR6Vbkk/C8efNYgYrcuMmJo7FjefIYIo/ymCVPHuXRnpfbI3vjRo0axbdaONKNtlA7iO3ER+clIEs5LXwaBGf9O3A+Lo/Hd4PKeg62rOJMkKznLFq0KG/edMAqzODBg38YwIGA2MI6HMdXvA3gP+XRUM70HeVRHrPkyaM82vMSPCb3MPIlmX09PAHMO/0GZC2nAT4gc4YVnJFroyQPlMub9wEQaSfHd2VF8H4u+wPb2tp+BPjJcFAgj9NwPQYaGhrCJgH+m8BXJxxEHmg9AJzLJ48h8iiPWfL63KP5vUd3HnjgAb4Dn7WcBvL+NmbK46QgF+bQJe8CHA7Nux7ZPAuUkxcOQP4CCLayBZb/RGD5FuBcPnkMkUd5zJknj+7I4/OAb+Nk9TcM8MrMM7Tn8fgTwDNv3LhxGch0bKa8M2fOXAvqjLA2wgHhd4KS8ir5LGDVx3m6EHk0II/ymCevEnlMLqfbzl56JJ8HWVRyHpR/gNRc54K9Ct4KePaxY8c6H5c3jxUpfm8iAvm+zKFDhzjEvPy8GNcFjdSo/bgdII9m5FEe8+TFqJJH0tbWlizvu+DmHlYnDw46l72cnFSeIfzVtY9yKi+vKAXy+CYlnx35+yiPBfOKIo9m5NEpTx5Lpv96LJesee8H9MgKQF/kFaVA3ldA2CUpjwXziiKPZuTRKU8eS0YeDbz44ovfBvTI+d8nTZqUMEViCXklII8G5NE1Tx5LRh4NyKNrXv/2eOzYsbCxgWuxvPzyy9XNK4ECed8CvwRcoGzp0qWZ8uSxZOTRgDy65vVvj8pzzZNHP/Lk0Y88efQjTx79yJNHP/Lk0Y88efQjL/QoLm3k0Q/k0Q/k0Q/k0Q/k0Q/k0Q/0/OhHnjz6kSePfuTJox958uhHnjz6kSePfuTJox958uhHnjz6kSePfuTJox958uhHnjz6kSePfuT1X4//Ak/1cA/g2LnI+oi1tbUl51XCdVgeBJ8CzgdV5r0ARoG6urqrwfvA78HevXtZNs7My5Wstm3bFk4zyXUgx48f/yGwEjjlyaMdeUzPTd9RHuUxT14l8piea9/hCOCiiFyGeGBvgmk7SENDA3f6GDgDCuTZ+RPgfeUcjM4HxfJuvPHGEeAP4NChQ8mXugZwkUnegZkzZ3IZyMFgwoQJXJY5OU8e7chjAvJoRR7lMbPHxSC26TBwPj45r7W1FYYaAmk1do8XttRMBXnzEuEyyfT4HeB8UCwP4m4HeeI5tzSXSa07v6iobXU2eUxFHhOQR1eq6LGxsZEPN7GtXASLgU6nSM7r6uq6ClARV1/c0ptgy/dAxCNvs/0J7yJ7LAAXUv4gkMf0vETkMQF5dEUe3fMS8dfj8uXLDR7p0LA5Mde+ww3gALDv0QGmT58erQw9+eSTefMsdHd3NwF6vBI4H1eSx+PHj48BjN+5cye+3l3JefJoQR6TkUcn5DFrnoXL0CPbAcrz6EZra2vQI3m+S7J8j88880y4jucfgfNxBcrH+ZBfAh8HqFpx+VC2H992223pefJoQR6TkUcrfecRP4R8WDR8chE8Vvf3sS893gQeffTRNwHm8c/m5uavAuc8ebQgj065bnl25LF3njxa8NpjY2Ojof+R0CNqQc65bnl2qu5x8+bN1fUId5DVHK7fhVvL9x65yPSdIGuePFqQR6dc5+u0II+98+TRgtced+zYYfmEbaxOr+lcKh5HjhxZXY+jR48OAyaAtra2zNcYyZNHC/LolOt8nRbksXeePFrw1iNfE7W/hMNP7ZYrczNdq4Fdu3Zd4h7vuOOOL4GPgg0AhbgPcBjdSZA1Tx4tyKNTrvN1WpDH3nnuHu0tqOx/vLcHqNzBHfn3AReIGC7o8TR4BATDknnyIWDBggWWA/LkLQT19fV0yKHQ/wbOx+bJuxVA6ROAA6yGAXx1OC7AOU8eDcijPMqjE/JoznPzSFONILYJdZsBvWF1h58sB4d7UaScEZ4HkXFzdPgwSC1nppBagMLQ426Q6diC5WOn4+ieR0u3aULk0Y48yqM8OiGP5jw3j6y2UM3ixYunRuSh+sJ21eCTxdaKkCHXrVQxOjo6OCNUxOMvQLl5rwG2d+JGsr7h9pJMgTwDnJ0DKlHTqufUHU558hhDHuUxT54BeSwhz1+PEEePjT2DyvmHZaKV8jweB78Dc+bMaQW7AP/EM3nEIeeJSpjoKUteBNYugvbxh4DzcXnzLJw8eZKt5xw755QnjzHkUR7z5Fko2ePyYKRc0GRqfucxxDxiwJxr+bSlpYU/edeC5Pms4HAZKJpXybNgLYBHlvhrwPlYe14nyHQa8GvAm/Dcc8+l58ljDHmUR3uePMpjGR4DgW4zOuJx0jJqoDLX/mlNpGuxpjeRLcOHD+cs7ekXlafesRHA40TA/s5Mx1bmTQPO031F4LQkrG4dPXo0PU8eDcijPMpj8Kk8BpTpMdNUx0FTa/JO9jzWBSKtp8n1nAAuaJI3z8xfAWc8wQ1Mb9x0ynsP2AqWLl2a6UxfBPQ4Y8aM9Dx5jCGP8mjO6w8ey8ee9wroLS3dY/n9VlzQKmhcfRVkK5wxj2//XwcGDRrE5lIuZjULHDx4kO87Wk6DZ+P7AcvY3t6eniePMeTRDXmUR3m05vUjj3yjYurUqTGP14BfgVtAbW0t57YNdqhO/2NQx6kbO3ZstoIl5vExFM+BbDLl6DgGbNq0iYMOHg5ARWhOAGfYHzZsGMs4EzjlyWMMeXRDHuVRHq15/cgj6ejoWA8eA/SIYn8ZRHbgDTnQQ3X6Hzkor3yPETgKj+uqnDhx4hh4HNRVsn379oTFVyrz5DGGPLohj/J4Th5tef3Mo/Jy5smjH3ny6EeePPqRJ49+5MmjH3ny6EeePPqRJ49+5MmjH3ny6EeePPqRJ49+5MmjH3ny6Ede6FFc2sijH8ijH8ijH8ijH8ijH8ijH/wfjEm7MQplbmRzdHJlYW0KZW5kb2JqCjE0IDAgb2JqCjM0NDMKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjE1IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjkuMikgL0NyZWF0aW9uRGF0ZSAoRDoyMDI1MDQwMzE5MjcxMlopCj4+CmVuZG9iagp4cmVmCjAgMTYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDUxNzYgMDAwMDAgbiAKMDAwMDAwMDYwNyAwMDAwMCBuIAowMDAwMDAwNjI4IDAwMDAwIG4gCjAwMDAwMDA2ODggMDAwMDAgbiAKMDAwMDAwMDcwOSAwMDAwMCBuIAowMDAwMDAwNzMwIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNTg3IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDU2NyAwMDAwMCBuIAowMDAwMDAwNzYyIDAwMDAwIG4gCjAwMDAwMDUxNTUgMDAwMDAgbiAKMDAwMDAwNTIzNiAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDE2IC9Sb290IDEgMCBSIC9JbmZvIDE1IDAgUiA+PgpzdGFydHhyZWYKNTM4NwolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:27:12.551569\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["def show_imgs(imgs):\n", " num_imgs = imgs.shape[0] if isinstance(imgs, Tensor) else len(imgs)\n", " nrow = min(num_imgs, 4)\n", " ncol = int(math.ceil(num_imgs / nrow))\n", " imgs = torchvision.utils.make_grid(imgs, nrow=nrow, pad_value=128)\n", " imgs = imgs.clamp(min=0, max=255)\n", " np_imgs = imgs.cpu().numpy()\n", " plt.figure(figsize=(1.5 * nrow, 1.5 * ncol))\n", " plt.imshow(np.transpose(np_imgs, (1, 2, 0)), interpolation=\"nearest\")\n", " plt.axis(\"off\")\n", " plt.show()\n", " plt.close()\n", "\n", "\n", "show_imgs([train_set[i][0] for i in range(8)])"]}, {"cell_type": "markdown", "id": "4e3ff948", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.011095, "end_time": "2025-04-03T19:27:12.669368", "exception": false, "start_time": "2025-04-03T19:27:12.658273", "status": "completed"}, "tags": []}, "source": ["## Masked autoregressive convolutions\n", "\n", "The core module of PixelCNN is its masked convolutions.\n", "In contrast to language models, we don't apply an LSTM on each pixel one-by-one.\n", "This would be inefficient because images are grids instead of sequences.\n", "Thus, it is better to rely on convolutions that have shown great success in deep CNN classification models.\n", "\n", "Nevertheless, we cannot just apply standard convolutions without any changes.\n", "Remember that during training of autoregressive models, we want to use teacher forcing which both helps the model training, and significantly reduces the time needed for training.\n", "For image modeling, teacher forcing is implemented by using a training image as input to the model, and we want to obtain as output the prediction for each pixel based on *only* its predecessors.\n", "Thus, we need to ensure that the prediction for a specific pixel can only be influenced by its predecessors and not by its own value or any \"future\" pixels.\n", "For this, we apply convolutions with a mask.\n", "\n", "Which mask we use depends on the ordering of pixels we decide on, i.e. which is the first pixel we predict,\n", "which is the second one, etc.\n", "The most commonly used ordering is to denote the upper left pixel as the start pixel,\n", "and sort the pixels row by row, as shown in the visualization at the top of the tutorial.\n", "Thus, the second pixel is on the right of the first one (first row, second column),\n", "and once we reach the end of the row, we start in the second row, first column.\n", "If we now want to apply this to our convolutions, we need to ensure that the prediction of pixel 1\n", "is not influenced by its own \"true\" input, and all pixels on its right and in any lower row.\n", "In convolutions, this means that we want to set those entries of the weight matrix to zero that take pixels on the right and below into account.\n", "As an example for a 5x5 kernel, see a mask below (figure credit - [Aaron van den Oord](https://arxiv.org/abs/1606.05328)):\n", "\n", "
\n", "\n", "Before looking into the application of masked convolutions in PixelCNN\n", "in detail, let's first implement a module that allows us to apply an\n", "arbitrary mask to a convolution:"]}, {"cell_type": "code", "execution_count": 6, "id": "f214a7ef", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:12.692864Z", "iopub.status.busy": "2025-04-03T19:27:12.692656Z", "iopub.status.idle": "2025-04-03T19:27:12.698785Z", "shell.execute_reply": "2025-04-03T19:27:12.697930Z"}, "lines_to_next_cell": 2, "papermill": {"duration": 0.019597, "end_time": "2025-04-03T19:27:12.700106", "exception": false, "start_time": "2025-04-03T19:27:12.680509", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class MaskedConvolution(nn.Module):\n", " def __init__(self, c_in, c_out, mask, **kwargs):\n", " \"\"\"Implements a convolution with mask applied on its weights.\n", "\n", " Args:\n", " c_in: Number of input channels\n", " c_out: Number of output channels\n", " mask: Tensor of shape [kernel_size_H, kernel_size_W] with 0s where\n", " the convolution should be masked, and 1s otherwise.\n", " kwargs: Additional arguments for the convolution\n", "\n", " \"\"\"\n", " super().__init__()\n", " # For simplicity: calculate padding automatically\n", " kernel_size = (mask.shape[0], mask.shape[1])\n", " dilation = 1 if \"dilation\" not in kwargs else kwargs[\"dilation\"]\n", " padding = tuple(dilation * (kernel_size[i] - 1) // 2 for i in range(2))\n", " # Actual convolution\n", " self.conv = nn.Conv2d(c_in, c_out, kernel_size, padding=padding, **kwargs)\n", "\n", " # Mask as buffer => it is no parameter but still a tensor of the module\n", " # (must be moved with the devices)\n", " self.register_buffer(\"mask\", mask[None, None])\n", "\n", " def forward(self, x):\n", " self.conv.weight.data *= self.mask # Ensures zero's at masked positions\n", " return self.conv(x)"]}, {"cell_type": "markdown", "id": "67077b9a", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.011143, "end_time": "2025-04-03T19:27:12.722441", "exception": false, "start_time": "2025-04-03T19:27:12.711298", "status": "completed"}, "tags": []}, "source": ["### Vertical and horizontal convolution stacks\n", "\n", "To build our own autoregressive image model, we could simply stack a few masked convolutions on top of each other.\n", "This was actually the case for the original PixelCNN model, discussed in the paper\n", "[Pixel Recurrent Neural Networks](https://arxiv.org/abs/1601.06759), but this leads to a considerable issue.\n", "When sequentially applying a couple of masked convolutions, the receptive field of a pixel\n", "show to have a \"blind spot\" on the right upper side, as shown in the figure below\n", "(figure credit - [Aaron van den Oord et al. ](https://arxiv.org/abs/1606.05328)):\n", "\n", "
\n", "\n", "Although a pixel should be able to take into account all other pixels above and left of it,\n", "a stack of masked convolutions does not allow us to look to the upper pixels on the right.\n", "This is because the features of the pixels above, which we use for convolution,\n", "do not contain any information of the pixels on the right of the same row.\n", "If they would, we would be \"cheating\" and actually looking into the future.\n", "To overcome this issue, van den Oord et.\n", "al [2] proposed to split the convolutions into a vertical and a horizontal stack.\n", "The vertical stack looks at all pixels above the current one, while the horizontal takes into account all on the left.\n", "While keeping both of them separate, we can actually look at the pixels on the right with the vertical stack without breaking any of our assumptions.\n", "The two convolutions are also shown in the figure above.\n", "\n", "Let us implement them here as follows:"]}, {"cell_type": "code", "execution_count": 7, "id": "b27b8278", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:12.742215Z", "iopub.status.busy": "2025-04-03T19:27:12.741696Z", "iopub.status.idle": "2025-04-03T19:27:12.748571Z", "shell.execute_reply": "2025-04-03T19:27:12.747568Z"}, "papermill": {"duration": 0.017021, "end_time": "2025-04-03T19:27:12.749819", "exception": false, "start_time": "2025-04-03T19:27:12.732798", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class VerticalStackConvolution(MaskedConvolution):\n", " def __init__(self, c_in, c_out, kernel_size=3, mask_center=False, **kwargs):\n", " # Mask out all pixels below. For efficiency, we could also reduce the kernel\n", " # size in height, but for simplicity, we stick with masking here.\n", " mask = torch.ones(kernel_size, kernel_size)\n", " mask[kernel_size // 2 + 1 :, :] = 0\n", "\n", " # For the very first convolution, we will also mask the center row\n", " if mask_center:\n", " mask[kernel_size // 2, :] = 0\n", "\n", " super().__init__(c_in, c_out, mask, **kwargs)\n", "\n", "\n", "class HorizontalStackConvolution(MaskedConvolution):\n", " def __init__(self, c_in, c_out, kernel_size=3, mask_center=False, **kwargs):\n", " # Mask out all pixels on the left. Note that our kernel has a size of 1\n", " # in height because we only look at the pixel in the same row.\n", " mask = torch.ones(1, kernel_size)\n", " mask[0, kernel_size // 2 + 1 :] = 0\n", "\n", " # For the very first convolution, we will also mask the center pixel\n", " if mask_center:\n", " mask[0, kernel_size // 2] = 0\n", "\n", " super().__init__(c_in, c_out, mask, **kwargs)"]}, {"cell_type": "markdown", "id": "3daa06c8", "metadata": {"papermill": {"duration": 0.00812, "end_time": "2025-04-03T19:27:12.766137", "exception": false, "start_time": "2025-04-03T19:27:12.758017", "status": "completed"}, "tags": []}, "source": ["Note that we have an input argument called `mask_center`. Remember that\n", "the input to the model is the actual input image. Hence, the very first\n", "convolution we apply cannot use the center pixel as input, but must be\n", "masked. All consecutive convolutions, however, should use the center\n", "pixel as we otherwise lose the features of the previous layer. Hence,\n", "the input argument `mask_center` is True for the very first\n", "convolutions, and False for all others."]}, {"cell_type": "markdown", "id": "816ab39a", "metadata": {"papermill": {"duration": 0.008126, "end_time": "2025-04-03T19:27:12.782473", "exception": false, "start_time": "2025-04-03T19:27:12.774347", "status": "completed"}, "tags": []}, "source": ["### Visualizing the receptive field\n", "\n", "To validate our implementation of masked convolutions, we can visualize the receptive field we obtain with such convolutions.\n", "We should see that with increasing number of convolutional layers, the receptive field grows in both vertical and horizontal direction, without the issue of a blind spot.\n", "The receptive field can be empirically measured by backpropagating an arbitrary loss for the output features of a speicifc pixel with respect to the input.\n", "We implement this idea below, and visualize the receptive field below."]}, {"cell_type": "code", "execution_count": 8, "id": "a2654698", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:12.802833Z", "iopub.status.busy": "2025-04-03T19:27:12.802602Z", "iopub.status.idle": "2025-04-03T19:27:13.349777Z", "shell.execute_reply": "2025-04-03T19:27:13.348637Z"}, "papermill": {"duration": 0.560659, "end_time": "2025-04-03T19:27:13.351258", "exception": false, "start_time": "2025-04-03T19:27:12.790599", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1Qy27CMBC871fMEQ51vHYdJ8dSWtTeaCNxQD1UwYREhDaAQP37LkHpA1Uqra2Rduy1Z3aiYdiVeXgYDXD9SNEnyzfEqAQFNCrBHoyRoCAtrCbrWTkj5bIrOWWVOGO8lTP9nS6I5tTAK9OCY6Os8cdtT+k6YIIVoitR24hkJdiL0ujEYNM+9AcTuis7hbxGdMcYvmBMYzTSUHz96cCpkfE0LuQt2CnWLjE2iW0M9k4G+vCT1zTIKLqVNoNs3gaQzWiK3qQPlyiXWu/QC2Wx2IYZ1n0YrRLbLbnJw+u23AXMy7Cc9fGE7J5uMjpaM5pVmnZy/82HDlP9ns8PYm1U5uyoDKdKa81sJarL87MalKvn9dvf0qF3+fSWJgplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjI4NAplbmRvYmoKMTAgMCBvYmoKWyBdCmVuZG9iagoxOSAwIG9iago8PCAvTGVuZ3RoIDI2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UrmRAzEMy7cKlsBfUj2+uXFg958ewD07MTFLEQBB925RORs/bSXLj/zYZWdJ5Jb3oG3yuqLqBqmbIHPJcckVYpbyuBIkFi1lJtZnqoPycQ1qFb7wEzMT0yFJxBJyUo8irI+vg9f1HNxfN+n8GhkfdGxQekuSq6BUw75ytBI7lupdg+yDppvS6jPTruyApfGGrNSkTn8d9b8jLMKk3khFByEWv9PLHbIspBzU27l+A+Fd7YJYT6087BBp3lZ6SxXM5swETBltO6yAtVljwlQJ8BbNIdRaiMwXOq2I+eTc0cE0VXkaIsNShYPtPaM1XOgaEkvD+UnGBOa/8PqsyG1//wBwaGe6CmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL0xlbmd0aCA5MiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9jcENwDAIA/9MwQgQAsT7VFUf6f7fJhHqBx8G2RhgYbM14MHZwJfS2je9pEWT2ghWtUXdUJ67FKVYXUelTMJPmTt/UnQc7XAO29/W5ThN4+hf99D9AQ9KHgsKZW5kc3RyZWFtCmVuZG9iagoyMSAwIG9iago8PCAvTGVuZ3RoIDMwNyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9kktuAzEMQ/c+hS4QwPrZnvOkKLqY3n/bJyXpihzZFkVqlrpMWVMekDSThH/p8HCxnfI7bM9mZuBaopeJ5ZTn0BVi7qJ82cxGXVknxeqEZjq36FE5Fwc2Taqfqyyl3S54Dtcmnlv2ET+80KAe1DUuCTd0V6NlKTRjqvt/0nv8jDLgakxdbFKrex88XkRV6OgHR4kiY5cX5+NBCelKwmhaiJV3RQNB7vK0ynsJ7tveasiyB6mYzjspZrDrdFIubheHIR7I8qjw5aPYa0LP+LArJfRI2IYzcifuaMbm1MjikP7ejQRLj65oIfPgr27WLmC8UzpFYmROcqxpi1VO91AU07nDvQwQ9WxFQylzkdXqX8POC2uWbBZ4SvoFHqPdJksOVtnbqE7vrTzZ0PcfWtd0HwplbmRzdHJlYW0KZW5kb2JqCjIyIDAgb2JqCjw8IC9MZW5ndGggMjMyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRSW7EMAy7+xX8wADW7rwnxaCH9v/XUsoUCEAltrglYmMjAi8x+DmI3PiSNaMmfmdyV/wsT4VHwq3gSRSBl+FedoLLG8ZlPw4zH7yXVs6kxpMMyEU2PTwRMtglEDowuwZ12Gbaib4h4bMjUs1GltPXEvTSKgTKU7bf6YISbav6c/usC2372hNOdnvqSeUTiOeWrMBl4xWTxVgGPVG5SzF9kOpsoSehvCifg2w+aohElyhn4InBwSjQDuy57WfiVSFoXd2nbWOoRkrH078NTU2SCPlECWe2NO4W/n/Pvb7X+w9OIVQRCmVuZHN0cmVhbQplbmRvYmoKMjMgMCBvYmoKPDwgL0xlbmd0aCAyMzEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNU85kgQhDMt5hT4wVRjbQL+np7Y22Pl/upKZTpDwIcnTEx2ZeJkjI7Bmx9taZCBm4FNMxb/2tA8TqvfgHiKUiwthhpFw1qzjbp6OF/92lc9YB+82+IpZXhDYwkzWVxZnLtsFY2mcxDnJboxdE7GNda2nU1hHMKEMhHS2w5Qgc1Sk9MmOMuboOJEnnovv9tssdjl+DusLNo0hFef4KnqCNoOi7HnvAhpyQf9d3fgeRbvoJSAbCRbWUWLunOWEX712dB61KBJzQppBLhMhzekqphCaUKyzo6BSUXCpPqforJ9/5V9cLQplbmRzdHJlYW0KZW5kb2JqCjI0IDAgb2JqCjw8IC9MZW5ndGggMjQ5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1QO45EIQzrOYUv8CTyI3AeRqstZu/frgOaKVBMfrYzJNARgUcMMZSv4yWtoK6Bv4tC8W7i64PCIKtDUiDOeg+IdOymNpETOh2cMz9hN2OOwEUxBpzpdKY9ByY5+8IKhHMbZexWSCeJqiKO6jOOKZ4qe594FiztyDZbJ5I95CDhUlKJyaWflMo/bcqUCjpm0QQsErngZBNNOMu7SVKMGZQy6h6mdiJ9rDzIozroZE3OrCOZ2dNP25n4HHC3X9pkTpXHdB7M+Jy0zoM5Fbr344k2B02N2ujs9xNpKi9Sux1anX51EpXdGOcYEpdnfxnfZP/5B/6HWiIKZW5kc3RyZWFtCmVuZG9iagoyNSAwIG9iago8PCAvTGVuZ3RoIDEzNiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNj0EOAzEIA+95hZ9AIEB4z1ZVD9v/X0vYdtMLHsmAbFEGgSWHeIcb4dHbD99FNhVn45xfUiliIZhPcJ8wUxyNKXfyY4+AcZRqLKdoeF5Lzk3DFy13Ey2lrZeTGW+47pf3R5VtkQ1Fzy0LQtdskvkygQd8GJhHdeNppcfd9myv9vwAzmw0SQplbmRzdHJlYW0KZW5kb2JqCjI2IDAgb2JqCjw8IC9MZW5ndGggMzQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEVSS25EMQjbv1NwgUjhl5DztKq6mN5/W5tM1c3gCWBseMtTpmTKsLklIyTXlE99IkOspvw0ciQipvhJCQV2lY/Ha0usjeyRqBSf2vHjsfRGptkVWvXu0aXNolHNysg5yBChnhW6snvUDtnwelxIuu+UzSEcy/9QgSxl3XIKJUFb0HfsEd8PHa6CK4JhsGsug+1lMtT/+ocWXO9992LHLoAWrOe+wQ4AqKcTtAXIGdruNiloAFW6i0nCo/J6bnaibKNV6fkcADMOMHLAiCVbHb7R3gCWfV3oRY2K/StAUVlA/MjVdsHeMclIcBbmBo69cDzFmXBLOMYCQIq94hh68CXY5i9Xroia8Al1umQvvMKe2ubnQpMId60ADl5kw62ro6iW7ek8gvZnRXJGjNSLODohklrSOYLi0qAeWuNcN7HibSOxuVff7h/hnC9c9usXS+yExAplbmRzdHJlYW0KZW5kb2JqCjI3IDAgb2JqCjw8IC9MZW5ndGggMTY0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQx3EFMQxD76oCJTCACvWsx/MP6/6vhvTTQXoYQgxiT8KwXFdxYXTDj7ctMw1/RxnuxvoyY7zVWCAn6AMMkYmr0aT6dsUZqvTk1WKuo6JcLzoiEsyS46tAI3w6sseTtrYz/XReH+wh7xP/KirnbmEBLqruQPlSH/HUj9lR6pqhjyorax5q2leEXRFK2z4upzJO3b0DWuG9las92u8/HnY68gplbmRzdHJlYW0KZW5kb2JqCjI4IDAgb2JqCjw8IC9MZW5ndGggNzIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZcQL6piblCLhdIDMTKAbMMgLQlnIKIZ4CYIG0QxSAWRLGZiRlEHZwBkcvgSgMAJdsWyQplbmRzdHJlYW0KZW5kb2JqCjI5IDAgb2JqCjw8IC9MZW5ndGggNDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZclhBWLhdMLAfMAtGWcAoinsGVBgC5Zw0nCmVuZHN0cmVhbQplbmRvYmoKMzAgMCBvYmoKPDwgL0xlbmd0aCAxNjMgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA7EgMhDEN7TqEj+CMDPs9mMik2929j2GxSwNNYIIO7E4LU2oKJ6IKHtiXdBe+tBGdj/Ok2bjUS5AR1gFak42iUUn25xWmVdPFoNnMrC60THWYOepSjGaAQOhXe7aLkcqbuzvlDcPVf9b9i3TmbiYHJyh0IzepT3Pk2O6K6usn+pMfcrNd+K+xVYWlZS8sJt527ZkAJ3FM52qs9Px8KOvYKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvTGVuZ3RoIDIzOSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNUMltBDEM+7sKNTDA6By7HgeLPLL9f0PKCZKXaEviofKUW5bKZfcjOW/JuuVDh06VafJu0M2vsf6jDAJ2/1BUEK0lsUrMXNJusTRJL9nDOI2Xa7WO56l7hFmjePDj2NMpgek9MsFms705MKs9zg6QTrjGr+rTO5UkA4m6kPNCpQrrHtQloo8r25hSnU4t5RiXn+h7fI4APcXejdzRx8sXjEa1LajRapU4DzATU9GVcauRgZQTBkNnR1c0C6XIynpCNcKNOaGZvcNwYAPLs4Skpa1SvA9lAegCXdo64zRKgo4Awt8ojPX6Bqr8XjcKZW5kc3RyZWFtCmVuZG9iagozMiAwIG9iago8PCAvTGVuZ3RoIDE2MCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkDkSAzEIBHO9gidIXIL3rMu1wfr/qQfWR6LpAjQcuhZNynoUaD7psUahutBr6CxKkkTBFpIdUKdjiDsoSExIY5JIth6DI5pYs12YmVQqs1LhtGnFwr/ZWtXIRI1wjfyJ6QZU/E/qXJTwTYOvkjH6GFS8O4OMSfheRdxaMe3+RDCxGfYJb0UmBYSJsanZvs9ghsz3Ctc4x/MNTII36wplbmRzdHJlYW0KZW5kb2JqCjMzIDAgb2JqCjw8IC9MZW5ndGggMTggL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMza0UDCAwxRDrjQAHeYDUgplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9MZW5ndGggMTMzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWPSw4EIQhE95yijsDHH+dxMumFc//tgJ1uE2M9hVSBuYKhPS5rA50VHyEZtvG3qZaORVk+VHpSVg/J4Iesxssh3KAs8IJJKoYhUIuYGpEtZW63gNs2DbKylVOljrCLozCP9rRsFR5folsidZI/g8QqL9zjuh3Ipda73qKLvn+kATEJCmVuZHN0cmVhbQplbmRvYmoKMzUgMCBvYmoKPDwgL0xlbmd0aCA3NSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwztTRSMFAwNgASpmZGCqYm5gophlxAPoiVy2VoZApm5XAZWZopWFgAGSZm5lAhmIYcLmNTc6ABQEXGpmAaqj+HK4MrDQCVkBLvCmVuZHN0cmVhbQplbmRvYmoKMzYgMCBvYmoKPDwgL0xlbmd0aCAxNDEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY/BDsMwCEPv+Qr/QKTYKaF8T6dqh+7/ryNLuwt6AmOMhdDQG6qaw4Zgm+PF0iVUa/gUxUAlN8iZYA6lpNIdR5F6YjgYXB60G47isej6EbuSZn3QxkK6JWiAe6xTadymcRPEHTUF6inqnKO8ELmfqWfYNJLdNLOSc7gNv3vPU9f/p6u8y/kFvXcu/gplbmRzdHJlYW0KZW5kb2JqCjE3IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQk1RUURWK0RlamFWdVNhbnMgL0ZpcnN0Q2hhciAwIC9MYXN0Q2hhciAyNTUKL0ZvbnREZXNjcmlwdG9yIDE2IDAgUiAvU3VidHlwZSAvVHlwZTMgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0KL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNvZGluZwovRGlmZmVyZW5jZXMgWyAzMiAvc3BhY2UgNjYgL0IgODcgL1cgOTcgL2EgOTkgL2MgL2QgL2UgL2YgL2cgL2ggL2kgMTA4IC9sIDExMCAvbiAxMTIgL3AKMTE0IC9yIDExNiAvdCAxMTggL3YgMTIxIC95IF0KPj4KL1dpZHRocyAxNSAwIFIgPj4KZW5kb2JqCjE2IDAgb2JqCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRm9udE5hbWUgL0JNUVFEVitEZWphVnVTYW5zIC9GbGFncyAzMgovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Bc2NlbnQgOTI5IC9EZXNjZW50IC0yMzYgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDAgL1N0ZW1WIDAgL01heFdpZHRoIDEzNDIgPj4KZW5kb2JqCjE1IDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE4IDAgb2JqCjw8IC9CIDE5IDAgUiAvVyAyMCAwIFIgL2EgMjEgMCBSIC9jIDIyIDAgUiAvZCAyMyAwIFIgL2UgMjQgMCBSIC9mIDI1IDAgUgovZyAyNiAwIFIgL2ggMjcgMCBSIC9pIDI4IDAgUiAvbCAyOSAwIFIgL24gMzAgMCBSIC9wIDMxIDAgUiAvciAzMiAwIFIKL3NwYWNlIDMzIDAgUiAvdCAzNCAwIFIgL3YgMzUgMCBSIC95IDM2IDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTcgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAxID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9JMSAxMyAwIFIgL0kyIDE0IDAgUiA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjYgL0hlaWdodCAyMjYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDEgKP3nNwAiTSkgXSAvQml0c1BlckNvbXBvbmVudCAxCi9GaWx0ZXIgL0ZsYXRlRGVjb2RlCi9EZWNvZGVQYXJtcyA8PCAvUHJlZGljdG9yIDEwIC9Db2xvcnMgMSAvQ29sdW1ucyAyMjYgL0JpdHNQZXJDb21wb25lbnQgMSA+PgovTGVuZ3RoIDM3IDAgUiA+PgpzdHJlYW0KeJztySEBACAQBLBrQv8WJHvwCBTq2eyyLmastdZaa621/2wlw9rXe7LWWmuttdY23w351RDUCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKNTQKZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAxICj95zcAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMQovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDEgPj4KL0xlbmd0aCAzOCAwIFIgPj4Kc3RyZWFtCnic7ckhAQAgEASwa0L/FiR78AgU6tnssi5mrLXWWmuttf9sJcPa13uy1lprrbXWNt8N+dUQ1AplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjU0CmVuZG9iagoyIDAgb2JqCjw8IC9UeXBlIC9QYWdlcyAvS2lkcyBbIDExIDAgUiBdIC9Db3VudCAxID4+CmVuZG9iagozOSAwIG9iago8PCAvQ3JlYXRvciAoTWF0cGxvdGxpYiB2My45LjIsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My45LjIpIC9DcmVhdGlvbkRhdGUgKEQ6MjAyNTA0MDMxOTI3MTNaKQo+PgplbmRvYmoKeHJlZgowIDQwCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDA3ODU5IDAwMDAwIG4gCjAwMDAwMDY5OTAgMDAwMDAgbiAKMDAwMDAwNzAyMiAwMDAwMCBuIAowMDAwMDA3MDgyIDAwMDAwIG4gCjAwMDAwMDcxMDMgMDAwMDAgbiAKMDAwMDAwNzEyNCAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzNDQgMDAwMDAgbiAKMDAwMDAwMDcyMyAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA3MDMgMDAwMDAgbiAKMDAwMDAwNzE2NyAwMDAwMCBuIAowMDAwMDA3NTEzIDAwMDAwIG4gCjAwMDAwMDU3MzEgMDAwMDAgbiAKMDAwMDAwNTUyNCAwMDAwMCBuIAowMDAwMDA1MTE2IDAwMDAwIG4gCjAwMDAwMDY3ODQgMDAwMDAgbiAKMDAwMDAwMDc0MyAwMDAwMCBuIAowMDAwMDAxMDgwIDAwMDAwIG4gCjAwMDAwMDEyNDQgMDAwMDAgbiAKMDAwMDAwMTYyNCAwMDAwMCBuIAowMDAwMDAxOTI5IDAwMDAwIG4gCjAwMDAwMDIyMzMgMDAwMDAgbiAKMDAwMDAwMjU1NSAwMDAwMCBuIAowMDAwMDAyNzY0IDAwMDAwIG4gCjAwMDAwMDMxNzggMDAwMDAgbiAKMDAwMDAwMzQxNSAwMDAwMCBuIAowMDAwMDAzNTU5IDAwMDAwIG4gCjAwMDAwMDM2NzggMDAwMDAgbiAKMDAwMDAwMzkxNCAwMDAwMCBuIAowMDAwMDA0MjI2IDAwMDAwIG4gCjAwMDAwMDQ0NTkgMDAwMDAgbiAKMDAwMDAwNDU0OSAwMDAwMCBuIAowMDAwMDA0NzU1IDAwMDAwIG4gCjAwMDAwMDQ5MDIgMDAwMDAgbiAKMDAwMDAwNzQ5NCAwMDAwMCBuIAowMDAwMDA3ODQwIDAwMDAwIG4gCjAwMDAwMDc5MTkgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSA0MCAvUm9vdCAxIDAgUiAvSW5mbyAzOSAwIFIgPj4Kc3RhcnR4cmVmCjgwNzAKJSVFT0YK", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:27:13.259656\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["inp_img = torch.zeros(1, 1, 11, 11)\n", "inp_img.requires_grad_()\n", "\n", "\n", "def show_center_recep_field(img, out):\n", " \"\"\"Calculates the gradients of the input with respect to the output center pixel, and visualizes the overall\n", " receptive field.\n", "\n", " Args:\n", " img: Input image for which we want to calculate the receptive field on.\n", " out: Output features/loss which is used for backpropagation, and should be\n", " the output of the network/computation graph.\n", "\n", " \"\"\"\n", " # Determine gradients\n", " loss = out[0, :, img.shape[2] // 2, img.shape[3] // 2].sum() # L1 loss for simplicity\n", " # Retain graph as we want to stack multiple layers and show the receptive field of all of them\n", " loss.backward(retain_graph=True)\n", " img_grads = img.grad.abs()\n", " img.grad.fill_(0) # Reset grads\n", "\n", " # Plot receptive field\n", " img = img_grads.squeeze().cpu().numpy()\n", " fig, ax = plt.subplots(1, 2)\n", " _ = ax[0].imshow(img)\n", " ax[1].imshow(img > 0)\n", " # Mark the center pixel in red if it doesn't have any gradients (should be\n", " # the case for standard autoregressive models)\n", " show_center = img[img.shape[0] // 2, img.shape[1] // 2] == 0\n", " if show_center:\n", " center_pixel = np.zeros(img.shape + (4,))\n", " center_pixel[center_pixel.shape[0] // 2, center_pixel.shape[1] // 2, :] = np.array([1.0, 0.0, 0.0, 1.0])\n", " for i in range(2):\n", " ax[i].axis(\"off\")\n", " if show_center:\n", " ax[i].imshow(center_pixel)\n", " ax[0].set_title(\"Weighted receptive field\")\n", " ax[1].set_title(\"Binary receptive field\")\n", " plt.show()\n", " plt.close()\n", "\n", "\n", "show_center_recep_field(inp_img, inp_img)"]}, {"cell_type": "markdown", "id": "e921efb9", "metadata": {"papermill": {"duration": 0.011938, "end_time": "2025-04-03T19:27:13.376098", "exception": false, "start_time": "2025-04-03T19:27:13.364160", "status": "completed"}, "tags": []}, "source": ["Let's first visualize the receptive field of a horizontal convolution\n", "without the center pixel. We use a small, arbitrary input image\n", "($11\\times 11$ pixels), and calculate the loss for the center pixel. For\n", "simplicity, we initialize all weights with 1 and the bias with 0, and\n", "use a single channel. This is sufficient for our visualization purposes."]}, {"cell_type": "code", "execution_count": 9, "id": "95c1ecfc", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:13.401575Z", "iopub.status.busy": "2025-04-03T19:27:13.401368Z", "iopub.status.idle": "2025-04-03T19:27:13.583774Z", "shell.execute_reply": "2025-04-03T19:27:13.582650Z"}, "papermill": {"duration": 0.197382, "end_time": "2025-04-03T19:27:13.585470", "exception": false, "start_time": "2025-04-03T19:27:13.388088", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzNyAwIFIgPj4Kc3RyZWFtCnic7cuxDQAgDMCwciTcx8yVtB8gFgacMZJjXTaDJEmSJEmSJEmSJEmSJEmSJMm6WevZIEmSfCJPIkmSJEmSJEmSJEmSJEmSJMnv5AaKEP9FCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKODAKZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzOCAwIFIgPj4Kc3RyZWFtCnic7cuxDQAgDMCwciTcx8yVtB8gFgacMZJjXTaDJEmSJEmSJEmSJEmSJEmSJMm6WevZIEmSfCJPIkmSJEmSJEmSJEmSJEmSJMnv5AaKEP9FCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKODAKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjkuMikgL0NyZWF0aW9uRGF0ZSAoRDoyMDI1MDQwMzE5MjcxM1opCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDc5MTUgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDc1NDAgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA3NTIxIDAwMDAwIG4gCjAwMDAwMDc4OTYgMDAwMDAgbiAKMDAwMDAwNzk3NSAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODEyNgolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:27:13.479673\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["horiz_conv = HorizontalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=True)\n", "horiz_conv.conv.weight.data.fill_(1)\n", "horiz_conv.conv.bias.data.fill_(0)\n", "horiz_img = horiz_conv(inp_img)\n", "show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "336a2436", "metadata": {"papermill": {"duration": 0.013132, "end_time": "2025-04-03T19:27:13.646033", "exception": false, "start_time": "2025-04-03T19:27:13.632901", "status": "completed"}, "tags": []}, "source": ["The receptive field is shown in yellow, the center pixel in red, and all other pixels outside of the receptive field are dark blue.\n", "As expected, the receptive field of a single horizontal convolution with the center pixel masked and a $3\\times3$ kernel is only the pixel on the left.\n", "If we use a larger kernel size, more pixels would be taken into account on the left.\n", "\n", "Next, let's take a look at the vertical convolution:"]}, {"cell_type": "code", "execution_count": 10, "id": "32dab764", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:13.669671Z", "iopub.status.busy": "2025-04-03T19:27:13.669464Z", "iopub.status.idle": "2025-04-03T19:27:13.851955Z", "shell.execute_reply": "2025-04-03T19:27:13.850847Z"}, "papermill": {"duration": 0.194394, "end_time": "2025-04-03T19:27:13.853256", "exception": false, "start_time": "2025-04-03T19:27:13.658862", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzNyAwIFIgPj4Kc3RyZWFtCnic7csxCsAgEADB+6R5n615ZcTOQoSghTjbLhPvz3KQJEmSJEmSJEmSJEmS5DqZo48kSfIc2Sqp9gw3SZLkdjmJJEmSJEmSJEmSJEmSJEmSJK+THzKLcE0KZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago4OAplbmRvYmoKMTQgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjYgL0hlaWdodCAyMjYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nN/8AAAAiTSkgXSAvQml0c1BlckNvbXBvbmVudCAyCi9GaWx0ZXIgL0ZsYXRlRGVjb2RlCi9EZWNvZGVQYXJtcyA8PCAvUHJlZGljdG9yIDEwIC9Db2xvcnMgMSAvQ29sdW1ucyAyMjYgL0JpdHNQZXJDb21wb25lbnQgMiA+PgovTGVuZ3RoIDM4IDAgUiA+PgpzdHJlYW0KeJztyzEKwCAQAMH7pHmfrXllxM5ChKCFONsuE+/PcpAkSZIkSZIkSZIkSZLkOpmjjyRJ8hzZKqn2DDdJkuR2OYkkSZIkSZIkSZIkSZIkSZIkr5MfMotwTQplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjg4CmVuZG9iagoyIDAgb2JqCjw8IC9UeXBlIC9QYWdlcyAvS2lkcyBbIDExIDAgUiBdIC9Db3VudCAxID4+CmVuZG9iagozOSAwIG9iago8PCAvQ3JlYXRvciAoTWF0cGxvdGxpYiB2My45LjIsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My45LjIpIC9DcmVhdGlvbkRhdGUgKEQ6MjAyNTA0MDMxOTI3MTNaKQo+PgplbmRvYmoKeHJlZgowIDQwCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDA3OTMxIDAwMDAwIG4gCjAwMDAwMDY5ODggMDAwMDAgbiAKMDAwMDAwNzAyMCAwMDAwMCBuIAowMDAwMDA3MDgwIDAwMDAwIG4gCjAwMDAwMDcxMDEgMDAwMDAgbiAKMDAwMDAwNzEyMiAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzNDQgMDAwMDAgbiAKMDAwMDAwMDcyMSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA3MDEgMDAwMDAgbiAKMDAwMDAwNzE2NSAwMDAwMCBuIAowMDAwMDA3NTQ4IDAwMDAwIG4gCjAwMDAwMDU3MjkgMDAwMDAgbiAKMDAwMDAwNTUyMiAwMDAwMCBuIAowMDAwMDA1MTE0IDAwMDAwIG4gCjAwMDAwMDY3ODIgMDAwMDAgbiAKMDAwMDAwMDc0MSAwMDAwMCBuIAowMDAwMDAxMDc4IDAwMDAwIG4gCjAwMDAwMDEyNDIgMDAwMDAgbiAKMDAwMDAwMTYyMiAwMDAwMCBuIAowMDAwMDAxOTI3IDAwMDAwIG4gCjAwMDAwMDIyMzEgMDAwMDAgbiAKMDAwMDAwMjU1MyAwMDAwMCBuIAowMDAwMDAyNzYyIDAwMDAwIG4gCjAwMDAwMDMxNzYgMDAwMDAgbiAKMDAwMDAwMzQxMyAwMDAwMCBuIAowMDAwMDAzNTU3IDAwMDAwIG4gCjAwMDAwMDM2NzYgMDAwMDAgbiAKMDAwMDAwMzkxMiAwMDAwMCBuIAowMDAwMDA0MjI0IDAwMDAwIG4gCjAwMDAwMDQ0NTcgMDAwMDAgbiAKMDAwMDAwNDU0NyAwMDAwMCBuIAowMDAwMDA0NzUzIDAwMDAwIG4gCjAwMDAwMDQ5MDAgMDAwMDAgbiAKMDAwMDAwNzUyOSAwMDAwMCBuIAowMDAwMDA3OTEyIDAwMDAwIG4gCjAwMDAwMDc5OTEgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSA0MCAvUm9vdCAxIDAgUiAvSW5mbyAzOSAwIFIgPj4Kc3RhcnR4cmVmCjgxNDIKJSVFT0YK", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:27:13.747993\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["vert_conv = VerticalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=True)\n", "vert_conv.conv.weight.data.fill_(1)\n", "vert_conv.conv.bias.data.fill_(0)\n", "vert_img = vert_conv(inp_img)\n", "show_center_recep_field(inp_img, vert_img)"]}, {"cell_type": "markdown", "id": "d3b369f0", "metadata": {"papermill": {"duration": 0.01383, "end_time": "2025-04-03T19:27:13.881158", "exception": false, "start_time": "2025-04-03T19:27:13.867328", "status": "completed"}, "tags": []}, "source": ["The vertical convolution takes all pixels above into account. Combining\n", "these two, we get the L-shaped receptive field of the original masked\n", "convolution:"]}, {"cell_type": "code", "execution_count": 11, "id": "4bd9e793", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:13.910685Z", "iopub.status.busy": "2025-04-03T19:27:13.909881Z", "iopub.status.idle": "2025-04-03T19:27:14.078310Z", "shell.execute_reply": "2025-04-03T19:27:14.077168Z"}, "papermill": {"duration": 0.184966, "end_time": "2025-04-03T19:27:14.079931", "exception": false, "start_time": "2025-04-03T19:27:13.894965", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzNyAwIFIgPj4Kc3RyZWFtCnic7csxDgARFEDB75CcT+2UtBKFbFYh5rUvE+1jNUiSJEmSJEmSJEmSJEnyP1ljjiRJ8jKZ8qisNkmS5HG5E0mSJEmSJEmSJEmSJEmSJEk+JzthQS1FCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKODYKZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzOCAwIFIgPj4Kc3RyZWFtCnic7csxDgARFEDB75CcT+2UtBKFbFYh5rUvE+1jNUiSJEmSJEmSJEmSJEnyP1ljjiRJ8jKZ8qisNkmS5HG5E0mSJEmSJEmSJEmSJEmSJEk+JzthQS1FCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKODYKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjkuMikgL0NyZWF0aW9uRGF0ZSAoRDoyMDI1MDQwMzE5MjcxNFopCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDc5MjcgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDc1NDYgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA3NTI3IDAwMDAwIG4gCjAwMDAwMDc5MDggMDAwMDAgbiAKMDAwMDAwNzk4NyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODEzOAolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:27:13.974372\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["horiz_img = vert_img + horiz_img\n", "show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "30f070dc", "metadata": {"papermill": {"duration": 0.014508, "end_time": "2025-04-03T19:27:14.109364", "exception": false, "start_time": "2025-04-03T19:27:14.094856", "status": "completed"}, "tags": []}, "source": ["If we stack multiple horizontal and vertical convolutions, we need to take two aspects into account:\n", "\n", "1.\n", "The center should not be masked anymore for the following convolutions as the features at the pixel's position are already independent of its actual value.\n", "If it is hard to imagine why we can do this, just change the value below to `mask_center=True` and see what happens.\n", "2.\n", "The vertical convolution is not allowed to work on features from the horizontal convolution.\n", "In the feature map of the horizontal convolutions, a pixel contains information about all of the \"true\" pixels on the left.\n", "If we apply a vertical convolution which also uses features from the right, we effectively expand our receptive field to the true input which we want to prevent.\n", "Thus, the feature maps can only be merged for the horizontal convolution.\n", "\n", "Using this, we can stack the convolutions in the following way. We have\n", "two feature streams: one for the vertical stack, and one for the\n", "horizontal stack. The horizontal convolutions can operate on the joint\n", "features of the previous horizontals and vertical convolutions, while\n", "the vertical stack only takes its own previous features as input. For a\n", "quick implementation, we can therefore sum the horizontal and vertical\n", "output features at each layer, and use those as final output features to\n", "calculate the loss on. An implementation of 4 consecutive layers is\n", "shown below. Note that we reuse the features from the other convolutions\n", "with `mask_center=True` from above."]}, {"cell_type": "code", "execution_count": 12, "id": "45c3bfea", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:14.139699Z", "iopub.status.busy": "2025-04-03T19:27:14.139490Z", "iopub.status.idle": "2025-04-03T19:27:15.034395Z", "shell.execute_reply": "2025-04-03T19:27:15.032826Z"}, "papermill": {"duration": 0.912953, "end_time": "2025-04-03T19:27:15.036905", "exception": false, "start_time": "2025-04-03T19:27:14.123952", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Layer 2\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiA2ICj95zfIt2VmaXA1RWz/AACUjncAIk0pIF0KL0JpdHNQZXJDb21wb25lbnQgNCAvRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDQgPj4KL0xlbmd0aCAzNyAwIFIgPj4Kc3RyZWFtCnic7c2xCYAwEEDRNPa6guAEcQRxACEruP8IVkkKA2lEFN4vL3d54Xy7QCQSiUQikUgkEolEIpFIJBKJRCKRSCQSG8XSXFpS7qjDukgkEolEIpH4C7F+OZXGkBuKnYhEIpFIJBKJnffYaN1ye+eaSCQSiUQi8bvi8xGJRCKRSCQSiUQikUgkEolEIpFIJBKJRCKRSCQSiUQikUgkEolEIvHeBejld1UKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iagoxNjYKZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzOCAwIFIgPj4Kc3RyZWFtCnic7cuxDQAQEADANyTzqU3JAp+IRMNdfzEO9TBN0zRN0zRN0zQfmT0ypmmapmma5u4sdWkjY5qmeWluM03TNE3TNE3TNE3TNE3TNM3v5gSmHBJ7CmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKODMKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjkuMikgL0NyZWF0aW9uRGF0ZSAoRDoyMDI1MDQwMzE5MjcxNFopCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgwMTcgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDc2MzkgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA3NjE5IDAwMDAwIG4gCjAwMDAwMDc5OTggMDAwMDAgbiAKMDAwMDAwODA3NyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODIyOAolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:27:14.217024\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 3\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAxMiAo/ec3481SxrVnX2NuUllsJz1tCzNwACtk/wAAgoB4dnZ2N0ZsACJNKSBdCi9CaXRzUGVyQ29tcG9uZW50IDQgL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDIyNiAvQml0c1BlckNvbXBvbmVudCA0ID4+Ci9MZW5ndGggMzcgMCBSID4+CnN0cmVhbQp4nO3WrQ0CQRCA0Q0VQAvn0EcH0AItoPAgrgAEDdACLaDJGYpC7YzgEgw/OXifnOzkrZxy+3SFSCQSiUQikUgkEolEIpE4frHfRetoFS3b2iKH+TCXOyKRSCQSicS/F6+5fo5m0bTUJk0Uv2g30ZZIJBKJRCJxDGIz0PwUXaKXXVZEIpFIJBKJPyD2uZSHWc72h9pxaPtJRCKRSCQSiV8U3xqRSCQSiUQikUgkEolEIpFIJBKJRCKRSCQSiUQikUgkEolEIpFIJBIfuwNrVoLyCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKMjA4CmVuZG9iagoxNCAwIG9iago8PCAvVHlwZSAvWE9iamVjdCAvU3VidHlwZSAvSW1hZ2UgL1dpZHRoIDIyNiAvSGVpZ2h0IDIyNgovQ29sb3JTcGFjZSBbIC9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3/wAAACJNKSBdIC9CaXRzUGVyQ29tcG9uZW50IDIKL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDIyNiAvQml0c1BlckNvbXBvbmVudCAyID4+Ci9MZW5ndGggMzggMCBSID4+CnN0cmVhbQp4nO3LsRGAIBAAsGcd98F5HMX6p4TGkvPkrCDpEznpDtM0TdM0TfNxxbvDNE3TNE3TNE3TXHmW2p05Ypqm+f/8xjRN0zRN0zRN0zRN0zRN0zS3mw0E/K1rCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKODgKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjkuMikgL0NyZWF0aW9uRGF0ZSAoRDoyMDI1MDQwMzE5MjcxNFopCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgwODMgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDc3MDAgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA3NjgwIDAwMDAwIG4gCjAwMDAwMDgwNjQgMDAwMDAgbiAKMDAwMDAwODE0MyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODI5NAolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:27:14.582027\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 4\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyNAoo/ec3m5N2mJF2bG1yLkFsKj9tJT1tITtuAC1pETVvACNQ/wAAAFwoW+vUSwAqYL+waqaccwAsZn18eHh4dmBkblZcXG1GUGs3RmwAIk0pCl0KL0JpdHNQZXJDb21wb25lbnQgOCAvRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDggPj4KL0xlbmd0aCAzNyAwIFIgPj4Kc3RyZWFtCnic7dpJUiQxFAXBooFmbObp/iflAm/BogGzKI91SvqupSwPH/kOvz3A94dYCLEQYiHEQoiFEAshFkIshFjoaInXo7vV1ehs9T6aH64d59FrRkRERERERERERERERERERMRjIK4t50Cno4fV/ehk9TK6WN2OEBERERERERERERERERERERETxMvR2+rP6N/qMFrs+3Vpf1eIiIiIiIiIiIiIiIiIiIiIx0r88vPUekx6Wj2PXlfno8fVT/09hYiIiIiIiIiIiIiIiIiIiIj448S1fL14zdbi65vRPPr/h4iIiIiIiIiIiIiIiIiIiBgjpkIshFgIsRBiIcRCiIUQCyEWQiyEWAixEGIhxEKIhRALIRZCLIRYCLEQYiHEQoiFEAshFkIshFgIsRBiIcRCiIUQCyEWQiyEWAixEGIhxEKIhRALIRZCLIRYCLEQYiHEQoiFEAshFkIshFgIsdAn62RUogplbmRzdHJlYW0KZW5kb2JqCjM3IDAgb2JqCjM1NAplbmRvYmoKMTQgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjYgL0hlaWdodCAyMjYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nN/8AAAAiTSkgXSAvQml0c1BlckNvbXBvbmVudCAyCi9GaWx0ZXIgL0ZsYXRlRGVjb2RlCi9EZWNvZGVQYXJtcyA8PCAvUHJlZGljdG9yIDEwIC9Db2xvcnMgMSAvQ29sdW1ucyAyMjYgL0JpdHNQZXJDb21wb25lbnQgMiA+PgovTGVuZ3RoIDM4IDAgUiA+PgpzdHJlYW0KeJzty7ENgDAMALD0Hf4p93AKc66kDzA0aiUk7N2RRXeYpml+Yl4x4zBN0zRN0zRN0zRN0zTNLbP14cw3pmmaS2eBaZqmaZqmaZqmaZqmaZqmaf5uPqnfbB8KZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iago4OQplbmRvYmoKMiAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0tpZHMgWyAxMSAwIFIgXSAvQ291bnQgMSA+PgplbmRvYmoKMzkgMCBvYmoKPDwgL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuOS4yLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuOS4yKSAvQ3JlYXRpb25EYXRlIChEOjIwMjUwNDAzMTkyNzE0WikKPj4KZW5kb2JqCnhyZWYKMCA0MAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwODI2OCAwMDAwMCBuIAowMDAwMDA2OTg4IDAwMDAwIG4gCjAwMDAwMDcwMjAgMDAwMDAgbiAKMDAwMDAwNzA4MCAwMDAwMCBuIAowMDAwMDA3MTAxIDAwMDAwIG4gCjAwMDAwMDcxMjIgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzQ0IDAwMDAwIG4gCjAwMDAwMDA3MjEgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNzAxIDAwMDAwIG4gCjAwMDAwMDcxNjUgMDAwMDAgbiAKMDAwMDAwNzg4NCAwMDAwMCBuIAowMDAwMDA1NzI5IDAwMDAwIG4gCjAwMDAwMDU1MjIgMDAwMDAgbiAKMDAwMDAwNTExNCAwMDAwMCBuIAowMDAwMDA2NzgyIDAwMDAwIG4gCjAwMDAwMDA3NDEgMDAwMDAgbiAKMDAwMDAwMTA3OCAwMDAwMCBuIAowMDAwMDAxMjQyIDAwMDAwIG4gCjAwMDAwMDE2MjIgMDAwMDAgbiAKMDAwMDAwMTkyNyAwMDAwMCBuIAowMDAwMDAyMjMxIDAwMDAwIG4gCjAwMDAwMDI1NTMgMDAwMDAgbiAKMDAwMDAwMjc2MiAwMDAwMCBuIAowMDAwMDAzMTc2IDAwMDAwIG4gCjAwMDAwMDM0MTMgMDAwMDAgbiAKMDAwMDAwMzU1NyAwMDAwMCBuIAowMDAwMDAzNjc2IDAwMDAwIG4gCjAwMDAwMDM5MTIgMDAwMDAgbiAKMDAwMDAwNDIyNCAwMDAwMCBuIAowMDAwMDA0NDU3IDAwMDAwIG4gCjAwMDAwMDQ1NDcgMDAwMDAgbiAKMDAwMDAwNDc1MyAwMDAwMCBuIAowMDAwMDA0OTAwIDAwMDAwIG4gCjAwMDAwMDc4NjQgMDAwMDAgbiAKMDAwMDAwODI0OSAwMDAwMCBuIAowMDAwMDA4MzI4IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgNDAgL1Jvb3QgMSAwIFIgL0luZm8gMzkgMCBSID4+CnN0YXJ0eHJlZgo4NDc5CiUlRU9GCg==", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:27:14.747207\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 5\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAzNwoo/ec37NVK1MFe0r9gpZtzoJd1iIV4ACRSe3t3ZmlwWF1tV11tRE9rQU1rACZVLkFsKz9tCzNwACNQ/wAAAFwoW828YgAqYgArZJSOd4KAeAAsZnZ2dlRabFFYbExUbAAuazdGbCQ8bgAvbxY2bxQ2bwAiTSkKXQovQml0c1BlckNvbXBvbmVudCA4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlCi9EZWNvZGVQYXJtcyA8PCAvUHJlZGljdG9yIDEwIC9Db2xvcnMgMSAvQ29sdW1ucyAyMjYgL0JpdHNQZXJDb21wb25lbnQgOCA+PgovTGVuZ3RoIDM3IDAgUiA+PgpzdHJlYW0KeJzt27lOw0AYhVGzhn0JENYQ1vd/RVoXtwtG5up85Wg09nE5+j18p5ah29QmdJ36DMWN6cT46PSOETMgIiIiIiIiIiIiIiIiIiIiIs6XmHYun0JXqZPQUeow9Jxah95Tq1DEICIiIiIiIiIiIiIiIiIiIv5f4kXqK3SaOgjtpIbQXuoxdJ76CC1SiIiIiIiIiIiIiIiIiIiIiIgNxJfUQ+gutRvaTx2HLlP3oS1v4BARERERERERERERERERERHnRoyraXoqjjCdhdL80/o19JZKXzcNSq3STdSWf74hIiIiIiIiIiIiIiIiIiIiIv45Ma4mdjpykTbGE29CcePvh4iIOApxyhAREUchThkiIuIoxClDREQcNTtiVYgNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2NAPuU5lsQplbmRzdHJlYW0KZW5kb2JqCjM3IDAgb2JqCjQwMwplbmRvYmoKMTQgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjYgL0hlaWdodCAyMjYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nN/8AAAAiTSkgXSAvQml0c1BlckNvbXBvbmVudCAyCi9GaWx0ZXIgL0ZsYXRlRGVjb2RlCi9EZWNvZGVQYXJtcyA8PCAvUHJlZGljdG9yIDEwIC9Db2xvcnMgMSAvQ29sdW1ucyAyMjYgL0JpdHNQZXJDb21wb25lbnQgMiA+PgovTGVuZ3RoIDM4IDAgUiA+PgpzdHJlYW0KeJzty7ERABAQAMFXJPWJVekb+BGQ2UtvNkKSJEmSJEmSJN3WejZW1SRJknwky3WIJEmSJEmSJEmSJEmSJEmSJL+TGx1KsYUKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iago3MQplbmRvYmoKMiAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0tpZHMgWyAxMSAwIFIgXSAvQ291bnQgMSA+PgplbmRvYmoKMzkgMCBvYmoKPDwgL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuOS4yLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuOS4yKSAvQ3JlYXRpb25EYXRlIChEOjIwMjUwNDAzMTkyNzE0WikKPj4KZW5kb2JqCnhyZWYKMCA0MAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwODMzNyAwMDAwMCBuIAowMDAwMDA2OTg4IDAwMDAwIG4gCjAwMDAwMDcwMjAgMDAwMDAgbiAKMDAwMDAwNzA4MCAwMDAwMCBuIAowMDAwMDA3MTAxIDAwMDAwIG4gCjAwMDAwMDcxMjIgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzQ0IDAwMDAwIG4gCjAwMDAwMDA3MjEgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNzAxIDAwMDAwIG4gCjAwMDAwMDcxNjUgMDAwMDAgbiAKMDAwMDAwNzk3MSAwMDAwMCBuIAowMDAwMDA1NzI5IDAwMDAwIG4gCjAwMDAwMDU1MjIgMDAwMDAgbiAKMDAwMDAwNTExNCAwMDAwMCBuIAowMDAwMDA2NzgyIDAwMDAwIG4gCjAwMDAwMDA3NDEgMDAwMDAgbiAKMDAwMDAwMTA3OCAwMDAwMCBuIAowMDAwMDAxMjQyIDAwMDAwIG4gCjAwMDAwMDE2MjIgMDAwMDAgbiAKMDAwMDAwMTkyNyAwMDAwMCBuIAowMDAwMDAyMjMxIDAwMDAwIG4gCjAwMDAwMDI1NTMgMDAwMDAgbiAKMDAwMDAwMjc2MiAwMDAwMCBuIAowMDAwMDAzMTc2IDAwMDAwIG4gCjAwMDAwMDM0MTMgMDAwMDAgbiAKMDAwMDAwMzU1NyAwMDAwMCBuIAowMDAwMDAzNjc2IDAwMDAwIG4gCjAwMDAwMDM5MTIgMDAwMDAgbiAKMDAwMDAwNDIyNCAwMDAwMCBuIAowMDAwMDA0NDU3IDAwMDAwIG4gCjAwMDAwMDQ1NDcgMDAwMDAgbiAKMDAwMDAwNDc1MyAwMDAwMCBuIAowMDAwMDA0OTAwIDAwMDAwIG4gCjAwMDAwMDc5NTEgMDAwMDAgbiAKMDAwMDAwODMxOCAwMDAwMCBuIAowMDAwMDA4Mzk3IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgNDAgL1Jvb3QgMSAwIFIgL0luZm8gMzkgMCBSID4+CnN0YXJ0eHJlZgo4NTQ4CiUlRU9GCg==", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:27:14.924254\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["# Initialize convolutions with equal weight to all input pixels\n", "horiz_conv = HorizontalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=False)\n", "horiz_conv.conv.weight.data.fill_(1)\n", "horiz_conv.conv.bias.data.fill_(0)\n", "vert_conv = VerticalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=False)\n", "vert_conv.conv.weight.data.fill_(1)\n", "vert_conv.conv.bias.data.fill_(0)\n", "\n", "# We reuse our convolutions for the 4 layers here. Note that in a standard network,\n", "# we don't do that, and instead learn 4 separate convolution. As this cell is only for\n", "# visualization purposes, we reuse the convolutions for all layers.\n", "for l_idx in range(4):\n", " vert_img = vert_conv(vert_img)\n", " horiz_img = horiz_conv(horiz_img) + vert_img\n", " print(f\"Layer {l_idx + 2}\")\n", " show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "01ac5e73", "metadata": {"papermill": {"duration": 0.0252, "end_time": "2025-04-03T19:27:15.092774", "exception": false, "start_time": "2025-04-03T19:27:15.067574", "status": "completed"}, "tags": []}, "source": ["The receptive field above it visualized for the horizontal stack, which includes the features of the vertical convolutions.\n", "It grows over layers without any blind spot as we had before.\n", "The difference between \"weighted\" and \"binary\" receptive field is that for the latter, we check whether there are any gradients flowing back to this pixel.\n", "This indicates that the center pixel indeed can use information from this pixel.\n", "Nevertheless, due to the convolution weights, some pixels have a stronger effect on the prediction than others.\n", "This is visualized in the weighted receptive field by plotting the gradient magnitude for each pixel instead of a binary yes/no.\n", "\n", "\n", "Another receptive field we can check is the one for the vertical stack\n", "as the one above is for the horizontal stack. Let's visualize it below:"]}, {"cell_type": "code", "execution_count": 13, "id": "3675c065", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:15.127558Z", "iopub.status.busy": "2025-04-03T19:27:15.127325Z", "iopub.status.idle": "2025-04-03T19:27:15.298608Z", "shell.execute_reply": "2025-04-03T19:27:15.297482Z"}, "papermill": {"duration": 0.189487, "end_time": "2025-04-03T19:27:15.300070", "exception": false, "start_time": "2025-04-03T19:27:15.110583", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAxNgoo/ec3pZtzACRSZGdvACZVKj9tIjtuCDNwACVU/wAA38pWACpikYx3AC1pTlZsNERsACJNKSBdCi9CaXRzUGVyQ29tcG9uZW50IDggL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDIyNiAvQml0c1BlckNvbXBvbmVudCA4ID4+Ci9MZW5ndGggMzcgMCBSID4+CnN0cmVhbQp4nO3buY3gABDEQN7/P/lHewmMeRR3sV22oGG7AsTfy9fDj8uXw+fLp8P54PXG8/TVeI5hEzdxEzdxEzdxEzdxEzdxEzdxEzdxEzdxE1/uxPeXX4c/lw+Hn5d3h/PB643n6avxHLOJm7iJm7iJm7iJm7iJm7iJm7iJm7iJm/h6J368XN+Ifl+uyO8XDueD1xvP01fjOWYTN3ETN3ETN3ETN3ETN3ETN3ETN3ETN3ETX/DEN/AFbhM3cRM3cRM3cRM3cRM3cRM3cRM3cRM38dVOfAN/vm3iJm7iJm7iJm7iJm7iJm7iJm7iJm7iJm7iK5v4/307PHSah+5soomH7myiiYfubKKJh+5soomH7myiiYfubKKJh+5soomH7myiiYfubKKJh+68gYkh6gAfdYCPOsBHHeCjDvBRB/ioA3zUAT7qAB91gI86wEcd4KMO8FEH+KgDfNQBPuoAH3WAjzrARx3gow7wUQf4qAN81AE+6gAfdYCPOsBHHeCjDvBRB/ioA3zUAT7qAB91gI86wEcd4KMO8FEH+KgDfNQBPuoAH3WAjzrARx3gow7wUQf4qAN81AE+6gAfdYCPOsBHHeCjDvBRB/ioA3zUAT7qAB91gI86wEcd4KMO8FEH+KgDfNQBPuoAH3WAjzrARx3gow7wUQf4qAN81AE+6gAfdYCPOsBHHeCjDvBRB/ioA3zUAT7qAB91gI86wEcd4KMO8FEH+KgDfNQBPuoAH3WAjzrARx3gow7wUQf4qAN81AE+6gAfdYCPOsBHHeCjDvBRB/j+AY1R2XQKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago1OTcKZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzOCAwIFIgPj4Kc3RyZWFtCnic7cuxEQAQEADBb5L6pFTpG/ghEdlLbzZCkiRJkiRJkiRdtqpmy3q5B0mS5Gt5iCRJkiRJkiRJkiRJkiRJkiS/kxuWTwrLCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKNzEKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjkuMikgL0NyZWF0aW9uRGF0ZSAoRDoyMDI1MDQwMzE5MjcxNVopCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDg0NjcgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDgxMDEgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA4MDgxIDAwMDAwIG4gCjAwMDAwMDg0NDggMDAwMDAgbiAKMDAwMDAwODUyNyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODY3OAolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:27:15.194563\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["show_center_recep_field(inp_img, vert_img)"]}, {"cell_type": "markdown", "id": "c7b045fb", "metadata": {"papermill": {"duration": 0.025733, "end_time": "2025-04-03T19:27:15.354795", "exception": false, "start_time": "2025-04-03T19:27:15.329062", "status": "completed"}, "tags": []}, "source": ["As we have discussed before, the vertical stack only looks at pixels above the one we want to predict.\n", "Hence, we can validate that our implementation works as we initially expected it to.\n", "As a final step, let's clean up the computation graph we still had kept\n", "in memory for the visualization of the receptive field:"]}, {"cell_type": "code", "execution_count": 14, "id": "e776f892", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:15.389836Z", "iopub.status.busy": "2025-04-03T19:27:15.389636Z", "iopub.status.idle": "2025-04-03T19:27:15.394414Z", "shell.execute_reply": "2025-04-03T19:27:15.393361Z"}, "papermill": {"duration": 0.022424, "end_time": "2025-04-03T19:27:15.395928", "exception": false, "start_time": "2025-04-03T19:27:15.373504", "status": "completed"}, "tags": []}, "outputs": [], "source": ["del inp_img, horiz_conv, vert_conv"]}, {"cell_type": "markdown", "id": "9b41e5e3", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.013603, "end_time": "2025-04-03T19:27:15.423214", "exception": false, "start_time": "2025-04-03T19:27:15.409611", "status": "completed"}, "tags": []}, "source": ["## Gated PixelCNN\n", "\n", "
\n", "\n", "In the next step, we will use the masked convolutions to build a full autoregressive model, called Gated PixelCNN.\n", "The difference between the original PixelCNN and Gated PixelCNN is the use of separate horizontal and vertical stacks.\n", "However, in literature, you often see that people refer to the Gated PixelCNN simply as \"PixelCNN\".\n", "Hence, in the following, if we say \"PixelCNN\", we usually mean the gated version.\n", "What \"Gated\" refers to in the model name is explained next.\n", "\n", "### Gated Convolutions\n", "\n", "For visualizing the receptive field, we assumed a very simplified stack of vertical and horizontal convolutions.\n", "Obviously, there are more sophisticated ways of doing it, and PixelCNN uses gated convolutions for this.\n", "Specifically, the Gated Convolution block in PixelCNN looks as follows\n", "(figure credit - [Aaron van den Oord et al. ](https://arxiv.org/abs/1606.05328)):\n", "\n", "
\n", "\n", "The left path is the vertical stack (the $N\\times N$ convolution is masked correspondingly),\n", "and the right path is the horizontal stack.\n", "Gated convolutions are implemented by having a twice as large output channel size,\n", "and combine them by a element-wise multiplication of $\\tanh$ and a sigmoid.\n", "For a linear layer, we can express a gated activation unit as follows:\n", "\n", "$$\\mathbf{y} = \\tanh\\left(\\mathbf{W}_{f}\\mathbf{x}\\right)\\odot\\sigma\\left(\\mathbf{W}_{g}\\mathbf{x}\\right)$$\n", "\n", "For simplicity, biases have been neglected and the linear layer split into two part, $\\mathbf{W}_{f}$ and $\\mathbf{W}_{g}$.\n", "This concept resembles the input and modulation gate in an LSTM, and has been used in many other architectures as well.\n", "The main motivation behind this gated activation is that it might allow to model more complex interactions and simplifies learning.\n", "But as in any other architecture, this is mostly a design choice and can be considered a hyperparameters.\n", "\n", "Besides the gated convolutions, we also see that the horizontal stack uses a residual connection while the vertical stack does not.\n", "This is because we use the output of the horizontal stack for prediction.\n", "Each convolution in the vertical stack also receives a strong gradient signal\n", "as it is only two $1\\times 1$ convolutions away from the residual connection,\n", "and does not require another residual connection to all its earleri layers.\n", "\n", "The implementation in PyTorch is fairly straight forward for this block,\n", "because the visualization above gives us a computation graph to follow:"]}, {"cell_type": "code", "execution_count": 15, "id": "0e047bc9", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:15.452509Z", "iopub.status.busy": "2025-04-03T19:27:15.451532Z", "iopub.status.idle": "2025-04-03T19:27:15.462667Z", "shell.execute_reply": "2025-04-03T19:27:15.461465Z"}, "lines_to_next_cell": 2, "papermill": {"duration": 0.027269, "end_time": "2025-04-03T19:27:15.464189", "exception": false, "start_time": "2025-04-03T19:27:15.436920", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class GatedMaskedConv(nn.Module):\n", " def __init__(self, c_in, **kwargs):\n", " \"\"\"Gated Convolution block implemented the computation graph shown above.\"\"\"\n", " super().__init__()\n", " self.conv_vert = VerticalStackConvolution(c_in, c_out=2 * c_in, **kwargs)\n", " self.conv_horiz = HorizontalStackConvolution(c_in, c_out=2 * c_in, **kwargs)\n", " self.conv_vert_to_horiz = nn.Conv2d(2 * c_in, 2 * c_in, kernel_size=1, padding=0)\n", " self.conv_horiz_1x1 = nn.Conv2d(c_in, c_in, kernel_size=1, padding=0)\n", "\n", " def forward(self, v_stack, h_stack):\n", " # Vertical stack (left)\n", " v_stack_feat = self.conv_vert(v_stack)\n", " v_val, v_gate = v_stack_feat.chunk(2, dim=1)\n", " v_stack_out = torch.tanh(v_val) * torch.sigmoid(v_gate)\n", "\n", " # Horizontal stack (right)\n", " h_stack_feat = self.conv_horiz(h_stack)\n", " h_stack_feat = h_stack_feat + self.conv_vert_to_horiz(v_stack_feat)\n", " h_val, h_gate = h_stack_feat.chunk(2, dim=1)\n", " h_stack_feat = torch.tanh(h_val) * torch.sigmoid(h_gate)\n", " h_stack_out = self.conv_horiz_1x1(h_stack_feat)\n", " h_stack_out = h_stack_out + h_stack\n", "\n", " return v_stack_out, h_stack_out"]}, {"cell_type": "markdown", "id": "da5b8cb8", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.013636, "end_time": "2025-04-03T19:27:15.491498", "exception": false, "start_time": "2025-04-03T19:27:15.477862", "status": "completed"}, "tags": []}, "source": ["### Building the model\n", "\n", "Using the gated convolutions, we can now build our PixelCNN model.\n", "The architecture consists of multiple stacked GatedMaskedConv blocks, where we add an additional dilation factor to a few convolutions.\n", "This is used to increase the receptive field of the model and allows to take a larger context into account during generation.\n", "As a reminder, dilation on a convolution works looks as follows\n", "(figure credit - [Vincent Dumoulin and Francesco Visin](https://arxiv.org/abs/1603.07285)):\n", "\n", "
\n", "\n", "Note that the smaller output size is only because the animation assumes no padding.\n", "In our implementation, we will pad the input image correspondingly.\n", "Alternatively to dilated convolutions, we could downsample the input and use a encoder-decoder architecture as in PixelCNN++ [3].\n", "This is especially beneficial if we want to build a very deep autoregressive model.\n", "Nonetheless, as we seek to train a reasonably small model, dilated convolutions are the more efficient option to use here.\n", "\n", "Below, we implement the PixelCNN model as a PyTorch Lightning module.\n", "Besides the stack of gated convolutions, we also have the initial\n", "horizontal and vertical convolutions which mask the center pixel, and a\n", "final $1\\times 1$ convolution which maps the output features to class\n", "predictions. To determine the likelihood of a batch of images, we first\n", "create our initial features using the masked horizontal and vertical\n", "input convolution. Next, we forward the features through the stack of\n", "gated convolutions. Finally, we take the output features of the\n", "horizontal stack, and apply the $1\\times 1$ convolution for\n", "classification. We use the bits per dimension metric for the likelihood,\n", "similarly to Tutorial 11 and assignment 3."]}, {"cell_type": "code", "execution_count": 16, "id": "3cb904f0", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:15.520806Z", "iopub.status.busy": "2025-04-03T19:27:15.519990Z", "iopub.status.idle": "2025-04-03T19:27:15.543526Z", "shell.execute_reply": "2025-04-03T19:27:15.542356Z"}, "papermill": {"duration": 0.039604, "end_time": "2025-04-03T19:27:15.544747", "exception": false, "start_time": "2025-04-03T19:27:15.505143", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class PixelCNN(pl.LightningModule):\n", " def __init__(self, c_in, c_hidden):\n", " super().__init__()\n", " self.save_hyperparameters()\n", "\n", " # Initial convolutions skipping the center pixel\n", " self.conv_vstack = VerticalStackConvolution(c_in, c_hidden, mask_center=True)\n", " self.conv_hstack = HorizontalStackConvolution(c_in, c_hidden, mask_center=True)\n", " # Convolution block of PixelCNN. We use dilation instead of downscaling\n", " self.conv_layers = nn.ModuleList(\n", " [\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=2),\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=4),\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=2),\n", " GatedMaskedConv(c_hidden),\n", " ]\n", " )\n", " # Output classification convolution (1x1)\n", " self.conv_out = nn.Conv2d(c_hidden, c_in * 256, kernel_size=1, padding=0)\n", "\n", " self.example_input_array = train_set[0][0][None]\n", "\n", " def forward(self, x):\n", " \"\"\"Forward image through model and return logits for each pixel.\n", "\n", " Args:\n", " x: Image tensor with integer values between 0 and 255.\n", "\n", " \"\"\"\n", " # Scale input from 0 to 255 back to -1 to 1\n", " x = (x.float() / 255.0) * 2 - 1\n", "\n", " # Initial convolutions\n", " v_stack = self.conv_vstack(x)\n", " h_stack = self.conv_hstack(x)\n", " # Gated Convolutions\n", " for layer in self.conv_layers:\n", " v_stack, h_stack = layer(v_stack, h_stack)\n", " # 1x1 classification convolution\n", " # Apply ELU before 1x1 convolution for non-linearity on residual connection\n", " out = self.conv_out(F.elu(h_stack))\n", "\n", " # Output dimensions: [Batch, Classes, Channels, Height, Width]\n", " out = out.reshape(out.shape[0], 256, out.shape[1] // 256, out.shape[2], out.shape[3])\n", " return out\n", "\n", " def calc_likelihood(self, x):\n", " # Forward pass with bpd likelihood calculation\n", " pred = self.forward(x)\n", " nll = F.cross_entropy(pred, x, reduction=\"none\")\n", " bpd = nll.mean(dim=[1, 2, 3]) * np.log2(np.exp(1))\n", " return bpd.mean()\n", "\n", " @torch.no_grad()\n", " def sample(self, img_shape, img=None):\n", " \"\"\"Sampling function for the autoregressive model.\n", "\n", " Args:\n", " img_shape: Shape of the image to generate (B,C,H,W)\n", " img (optional): If given, this tensor will be used as\n", " a starting image. The pixels to fill\n", " should be -1 in the input tensor.\n", "\n", " \"\"\"\n", " # Create empty image\n", " if img is None:\n", " img = torch.zeros(img_shape, dtype=torch.long).to(device) - 1\n", " # Generation loop\n", " for h in tqdm(range(img_shape[2]), leave=False):\n", " for w in range(img_shape[3]):\n", " for c in range(img_shape[1]):\n", " # Skip if not to be filled (-1)\n", " if (img[:, c, h, w] != -1).all().item():\n", " continue\n", " # For efficiency, we only have to input the upper part of the image\n", " # as all other parts will be skipped by the masked convolutions anyways\n", " pred = self.forward(img[:, :, : h + 1, :])\n", " probs = F.softmax(pred[:, :, c, h, w], dim=-1)\n", " img[:, c, h, w] = torch.multinomial(probs, num_samples=1).squeeze(dim=-1)\n", " return img\n", "\n", " def configure_optimizers(self):\n", " optimizer = optim.Adam(self.parameters(), lr=1e-3)\n", " scheduler = optim.lr_scheduler.StepLR(optimizer, 1, gamma=0.99)\n", " return [optimizer], [scheduler]\n", "\n", " def training_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"train_bpd\", loss)\n", " return loss\n", "\n", " def validation_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"val_bpd\", loss)\n", "\n", " def test_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"test_bpd\", loss)"]}, {"cell_type": "markdown", "id": "45ca33f7", "metadata": {"papermill": {"duration": 0.013619, "end_time": "2025-04-03T19:27:15.572163", "exception": false, "start_time": "2025-04-03T19:27:15.558544", "status": "completed"}, "tags": []}, "source": ["To sample from the autoregressive model, we need to iterate over all dimensions of the input.\n", "We start with an empty image, and fill the pixels one by one, starting from the upper left corner.\n", "Note that as for predicting $x_i$, all pixels below it have no influence on the prediction.\n", "Hence, we can cut the image in height without changing the prediction while increasing efficiency.\n", "Nevertheless, all the loops in the sampling function already show that it will take us quite some time to sample.\n", "A lot of computation could be reused across loop iterations as those the features on the already predicted pixels will not change over iterations.\n", "Nevertheless, this takes quite some effort to implement, and is often not done in implementations because in the end,\n", "autoregressive sampling remains sequential and slow.\n", "Hence, we settle with the default implementation here.\n", "\n", "Before training the model, we can check the full receptive field of the model on an MNIST image of size $28\\times 28$:"]}, {"cell_type": "code", "execution_count": 17, "id": "779ae7d4", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:15.601400Z", "iopub.status.busy": "2025-04-03T19:27:15.600584Z", "iopub.status.idle": "2025-04-03T19:27:15.933171Z", "shell.execute_reply": "2025-04-03T19:27:15.932011Z"}, "papermill": {"duration": 0.348753, "end_time": "2025-04-03T19:27:15.934581", "exception": false, "start_time": "2025-04-03T19:27:15.585828", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAxNQoo/ec3ADFwACZVACNPACRSACZXITtuHTluACNQ/wAAAFwoWwBcKFxcACtkACxnBDJwACJNKSBdCi9CaXRzUGVyQ29tcG9uZW50IDQgL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDIyNiAvQml0c1BlckNvbXBvbmVudCA0ID4+Ci9MZW5ndGggMzcgMCBSID4+CnN0cmVhbQp4nO3VoU7EQBAG4LXIOjQGTw3uTAUOQSpJUH2FKhQhIUGgq/ANb1CBRdTwAD15suYgIZhjG3pJJVzSQ/D9ZiYj5luzmbDZdwKRSCQSiUQikUgkEolEIpFIJBKJRCKRSCQSiUQikUj8e/Hj7jsPsV+naXo69m9Zlp3F/ibOFkQicS/iZiIN29NR2M62ryESifOJ63Hz8NPPY/2s6/qpbduXKNzmeX4xecWCSCTOKg6ZXuP3qFx2XfeaJMlhnN03TfP8kyVEInEncdh+FDNc3ZNYj8uyvI7Ksu/7VVEUVyGEg6qqHn8jEYnEncRZQyQSiUQikUgkEolEIpFIJBKJRCKRSCQSiUQikUgkEolEIpFIJBKJxP8pfgEnzHpfCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKMjgzCmVuZG9iagoxNCAwIG9iago8PCAvVHlwZSAvWE9iamVjdCAvU3VidHlwZSAvSW1hZ2UgL1dpZHRoIDIyNiAvSGVpZ2h0IDIyNgovQ29sb3JTcGFjZSBbIC9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3/wAAACJNKSBdIC9CaXRzUGVyQ29tcG9uZW50IDIKL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDIyNiAvQml0c1BlckNvbXBvbmVudCAyID4+Ci9MZW5ndGggMzggMCBSID4+CnN0cmVhbQp4nO3ZMQEAIAzEwJpAGiZQxYxKPPxAGS77KUidsF3kn7LCSJIkSZIkSZIkSZIkSZIkSZLslWOu8AGQfTKLJEmSJEmSJEmSJEmSJEmSJMkH8gLidNhUCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKODUKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjkuMikgL0NyZWF0aW9uRGF0ZSAoRDoyMDI1MDQwMzE5MjcxNVopCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgxNjcgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDc3ODcgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA3NzY3IDAwMDAwIG4gCjAwMDAwMDgxNDggMDAwMDAgbiAKMDAwMDAwODIyNyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODM3OAolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:27:15.825733\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["test_model = PixelCNN(c_in=1, c_hidden=64)\n", "inp = torch.zeros(1, 1, 28, 28)\n", "inp.requires_grad_()\n", "out = test_model(inp)\n", "show_center_recep_field(inp, out.squeeze(dim=2))\n", "del inp, out, test_model"]}, {"cell_type": "markdown", "id": "8855f815", "metadata": {"papermill": {"duration": 0.020458, "end_time": "2025-04-03T19:27:15.976520", "exception": false, "start_time": "2025-04-03T19:27:15.956062", "status": "completed"}, "tags": []}, "source": ["The visualization shows that for predicting any pixel, we can take almost half of the image into account.\n", "However, keep in mind that this is the \"theoretical\" receptive field and not necessarily\n", "the [effective receptive field](https://arxiv.org/abs/1701.04128), which is usually much smaller.\n", "For a stronger model, we should therefore try to increase the receptive\n", "field even further. Especially, for the pixel on the bottom right, the\n", "very last pixel, we would be allowed to take into account the whole\n", "image. However, our current receptive field only spans across 1/4 of the\n", "image. An encoder-decoder architecture can help with this, but it also\n", "shows that we require a much deeper, more complex network in\n", "autoregressive models than in VAEs or energy-based models."]}, {"cell_type": "markdown", "id": "aec9878a", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.014885, "end_time": "2025-04-03T19:27:16.011048", "exception": false, "start_time": "2025-04-03T19:27:15.996163", "status": "completed"}, "tags": []}, "source": ["### Training loop\n", "\n", "To train the model, we again can rely on PyTorch Lightning and write a\n", "function below for loading the pretrained model if it exists. To reduce\n", "the computational cost, we have saved the validation and test score in\n", "the checkpoint already:"]}, {"cell_type": "code", "execution_count": 18, "id": "3b18e3d6", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:16.041197Z", "iopub.status.busy": "2025-04-03T19:27:16.040847Z", "iopub.status.idle": "2025-04-03T19:27:16.047963Z", "shell.execute_reply": "2025-04-03T19:27:16.046922Z"}, "papermill": {"duration": 0.024015, "end_time": "2025-04-03T19:27:16.049285", "exception": false, "start_time": "2025-04-03T19:27:16.025270", "status": "completed"}, "tags": []}, "outputs": [], "source": ["def train_model(**kwargs):\n", " # Create a PyTorch Lightning trainer with the generation callback\n", " trainer = pl.Trainer(\n", " default_root_dir=os.path.join(CHECKPOINT_PATH, \"PixelCNN\"),\n", " accelerator=\"auto\",\n", " devices=1,\n", " max_epochs=150,\n", " callbacks=[\n", " ModelCheckpoint(save_weights_only=True, mode=\"min\", monitor=\"val_bpd\"),\n", " LearningRateMonitor(\"epoch\"),\n", " ],\n", " )\n", " result = None\n", " # Check whether pretrained model exists. If yes, load it and skip training\n", " pretrained_filename = os.path.join(CHECKPOINT_PATH, \"PixelCNN.ckpt\")\n", " if os.path.isfile(pretrained_filename):\n", " print(\"Found pretrained model, loading...\")\n", " model = PixelCNN.load_from_checkpoint(pretrained_filename)\n", " ckpt = torch.load(pretrained_filename, map_location=device)\n", " result = ckpt.get(\"result\", None)\n", " else:\n", " model = PixelCNN(**kwargs)\n", " trainer.fit(model, train_loader, val_loader)\n", " model = model.to(device)\n", "\n", " if result is None:\n", " # Test best model on validation and test set\n", " val_result = trainer.test(model, dataloaders=val_loader, verbose=False)\n", " test_result = trainer.test(model, dataloaders=test_loader, verbose=False)\n", " result = {\"test\": test_result, \"val\": val_result}\n", " return model, result"]}, {"cell_type": "markdown", "id": "eb9b181c", "metadata": {"papermill": {"duration": 0.014227, "end_time": "2025-04-03T19:27:16.077915", "exception": false, "start_time": "2025-04-03T19:27:16.063688", "status": "completed"}, "tags": []}, "source": ["Training the model is time consuming and we recommend using the provided pre-trained model for going through this notebook.\n", "However, feel free to play around with the hyperparameter like number of layers etc.\n", "if you want to get a feeling for those.\n", "\n", "When calling the training function with a pre-trained model, we automatically load it and print its test performance:"]}, {"cell_type": "code", "execution_count": 19, "id": "c4e06ed6", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:16.107614Z", "iopub.status.busy": "2025-04-03T19:27:16.107372Z", "iopub.status.idle": "2025-04-03T19:27:16.402307Z", "shell.execute_reply": "2025-04-03T19:27:16.401053Z"}, "papermill": {"duration": 0.312517, "end_time": "2025-04-03T19:27:16.404712", "exception": false, "start_time": "2025-04-03T19:27:16.092195", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["GPU available: True (cuda), used: True\n"]}, {"name": "stderr", "output_type": "stream", "text": ["TPU available: False, using: 0 TPU cores\n"]}, {"name": "stderr", "output_type": "stream", "text": ["HPU available: False, using: 0 HPUs\n"]}, {"name": "stderr", "output_type": "stream", "text": ["/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py:75: Starting from v1.9.0, `tensorboardX` has been removed as a dependency of the `pytorch_lightning` package, due to potential conflicts with other packages in the ML ecosystem. For this reason, `logger=True` will use `CSVLogger` as the default logger, unless the `tensorboard` or `tensorboardX` packages are found. Please `pip install lightning[extra]` or one of them to enable TensorBoard support by default\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Found pretrained model, loading...\n"]}, {"name": "stderr", "output_type": "stream", "text": ["Lightning automatically upgraded your loaded checkpoint from v0.9.0 to v2.4.0. To apply the upgrade to your files permanently, run `python -m pytorch_lightning.utilities.upgrade_checkpoint saved_models/tutorial12/PixelCNN.ckpt`\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Test bits per dimension: 0.808bpd\n"]}, {"name": "stderr", "output_type": "stream", "text": ["/tmp/ipykernel_676/1489443601.py:19: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.\n", " ckpt = torch.load(pretrained_filename, map_location=device)\n"]}], "source": ["model, result = train_model(c_in=1, c_hidden=64)\n", "test_res = result[\"test\"][0]\n", "print(\n", " \"Test bits per dimension: %4.3fbpd\" % (test_res[\"test_loss\"] if \"test_loss\" in test_res else test_res[\"test_bpd\"])\n", ")"]}, {"cell_type": "markdown", "id": "bf18d8a4", "metadata": {"papermill": {"duration": 0.029364, "end_time": "2025-04-03T19:27:16.466870", "exception": false, "start_time": "2025-04-03T19:27:16.437506", "status": "completed"}, "tags": []}, "source": ["With a test performance of 0.809bpd, the PixelCNN significantly outperforms the normalizing flows we have seen in Tutorial 11.\n", "Considering image modeling as an autoregressive problem simplifies the learning process as predicting\n", "one pixel given the ground truth of all others is much easier than predicting all pixels at once.\n", "In addition, PixelCNN can explicitly predict the pixel values by a discrete softmax while\n", "Normalizing Flows have to learn transformations in continuous latent space.\n", "These two aspects allow the PixelCNN to achieve a notably better performance.\n", "\n", "To fully compare the models, let's also measure the number of parameters of the PixelCNN:"]}, {"cell_type": "code", "execution_count": 20, "id": "83bf68da", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:16.504770Z", "iopub.status.busy": "2025-04-03T19:27:16.504568Z", "iopub.status.idle": "2025-04-03T19:27:16.510450Z", "shell.execute_reply": "2025-04-03T19:27:16.509487Z"}, "papermill": {"duration": 0.024785, "end_time": "2025-04-03T19:27:16.511706", "exception": false, "start_time": "2025-04-03T19:27:16.486921", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Number of parameters: 852,160\n"]}], "source": ["num_params = sum(np.prod(param.shape) for param in model.parameters())\n", "print(f\"Number of parameters: {num_params:,}\")"]}, {"cell_type": "markdown", "id": "a5bffdd3", "metadata": {"papermill": {"duration": 0.014631, "end_time": "2025-04-03T19:27:16.541088", "exception": false, "start_time": "2025-04-03T19:27:16.526457", "status": "completed"}, "tags": []}, "source": ["Compared to the multi-scale normalizing flows, the PixelCNN has considerably less parameters.\n", "Of course, the number of parameters depend on our hyperparameter choices.\n", "Nevertheless, in general, it can be said that autoregressive models\n", "require considerably less parameters than normalizing flows to reach\n", "good performance, based on the reasons stated above. Still,\n", "autoregressive models are much slower in sampling than normalizing\n", "flows, which limits their possible applications."]}, {"cell_type": "markdown", "id": "5a08b2ce", "metadata": {"papermill": {"duration": 0.014655, "end_time": "2025-04-03T19:27:16.570515", "exception": false, "start_time": "2025-04-03T19:27:16.555860", "status": "completed"}, "tags": []}, "source": ["## Sampling\n", "\n", "One way of qualitatively analysing generative models is by looking at the actual samples.\n", "Let's therefore use our sampling function to generate a few digits:"]}, {"cell_type": "code", "execution_count": 21, "id": "b6f1847b", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:27:16.601120Z", "iopub.status.busy": "2025-04-03T19:27:16.600739Z", "iopub.status.idle": "2025-04-03T19:28:38.941181Z", "shell.execute_reply": "2025-04-03T19:28:38.940274Z"}, "papermill": {"duration": 82.357361, "end_time": "2025-04-03T19:28:38.942538", "exception": false, "start_time": "2025-04-03T19:27:16.585177", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "c5fa6d3178fe4940ad9f698e0821d952", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:28:38.902406\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["pl.seed_everything(1)\n", "samples = model.sample(img_shape=(16, 1, 28, 28))\n", "show_imgs(samples.cpu())"]}, {"cell_type": "markdown", "id": "9799736e", "metadata": {"papermill": {"duration": 0.024469, "end_time": "2025-04-03T19:28:38.983811", "exception": false, "start_time": "2025-04-03T19:28:38.959342", "status": "completed"}, "tags": []}, "source": ["Most of the samples can be identified as digits, and overall we achieve a better quality than we had in normalizing flows.\n", "This goes along with the lower likelihood we achieved with autoregressive models.\n", "Nevertheless, we also see that there is still place for improvement\n", "as a considerable amount of samples cannot be identified (for example the first row).\n", "Deeper autoregressive models are expected to achieve better quality,\n", "as they can take more context into account for generating the pixels.\n", "\n", "Note that on Google Colab, you might see different results, specifically with a white line at the top.\n", "After some debugging, it seemed that the difference occurs inside the dilated convolution,\n", "as it gives different results for different batch sizes.\n", "However, it is hard to debug this further as it might be a bug of the installed PyTorch version on Google Colab.\n", "\n", "The trained model itself is not restricted to any specific image size.\n", "However, what happens if we actually sample a larger image than we had\n", "seen in our training dataset? Let's try below to sample images of size\n", "$64\\times64$ instead of $28\\times28$:"]}, {"cell_type": "code", "execution_count": 22, "id": "fe1521a7", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:28:39.021720Z", "iopub.status.busy": "2025-04-03T19:28:39.021524Z", "iopub.status.idle": "2025-04-03T19:32:19.852785Z", "shell.execute_reply": "2025-04-03T19:32:19.851923Z"}, "papermill": {"duration": 220.849199, "end_time": "2025-04-03T19:32:19.854162", "exception": false, "start_time": "2025-04-03T19:28:39.004963", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "ee9dd5ee3bd84912a6cdd48f511d3bc4", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/64 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:32:19.821339\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["pl.seed_everything(1)\n", "samples = model.sample(img_shape=(8, 1, 64, 64))\n", "show_imgs(samples.cpu())"]}, {"cell_type": "markdown", "id": "31c85c02", "metadata": {"papermill": {"duration": 0.01659, "end_time": "2025-04-03T19:32:19.888668", "exception": false, "start_time": "2025-04-03T19:32:19.872078", "status": "completed"}, "tags": []}, "source": ["The larger images show that changing the size of the image during testing confuses the model\n", "and generates abstract figures (you can sometimes spot a digit in the upper left corner).\n", "In addition, sampling for images of 64x64 pixels take more than a minute on a GPU.\n", "Clearly, autoregressive models cannot be scaled to large images without changing the sampling procedure such as with [forecasting](https://arxiv.org/abs/2002.09928).\n", "Our implementation is also not the most efficient as many computations can be stored and reused throughout the sampling process.\n", "Nevertheless, the sampling procedure stays sequential which is\n", "inherently slower than parallel generation like done in normalizing\n", "flows."]}, {"cell_type": "markdown", "id": "60b206cb", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.016486, "end_time": "2025-04-03T19:32:19.921658", "exception": false, "start_time": "2025-04-03T19:32:19.905172", "status": "completed"}, "tags": []}, "source": ["### Autocompletion\n", "\n", "One common application done with autoregressive models is\n", "auto-completing an image. As autoregressive models predict pixels one by\n", "one, we can set the first $N$ pixels to predefined values and check how\n", "the model completes the image. For implementing this, we just need to\n", "skip the iterations in the sampling loop that already have a value\n", "unequals -1. See above in our PyTorch Lightning module for the specific\n", "implementation. In the cell below, we randomly take three images from\n", "the training set, mask about the lower half of the image, and let the\n", "model autocomplete it. To see the diversity of samples, we do this 12\n", "times for each image:"]}, {"cell_type": "code", "execution_count": 23, "id": "abc09fb5", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:32:19.956008Z", "iopub.status.busy": "2025-04-03T19:32:19.955818Z", "iopub.status.idle": "2025-04-03T19:32:25.716629Z", "shell.execute_reply": "2025-04-03T19:32:25.715650Z"}, "papermill": {"duration": 5.780364, "end_time": "2025-04-03T19:32:25.718552", "exception": false, "start_time": "2025-04-03T19:32:19.938188", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMTc1LjUyMjUgOTcuNTYgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicVY4xDsIwDEV3n+KfIImDkpQRqBQxFgYOEIVCRItKJXp93A6tGJ7kZ9nf1nX+PlO+xCNOV9KbpZEYRWhhUIQJjCi0ZMQ64uCUs9aJvDbZB+W8NMxaPYjuNCAou8CeFc+T1U6xxyfjhh76IMGjpBdhksiI/1+GZdFWmI/LqrNrYuqgz4z6jYYa+gGxYy7BCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMTQzCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjMgMCBvYmoKPDwgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjQgL0hlaWdodCAxMTYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDcwCij////9/f37+/v19fXv7+/t7e3r6+vj4+Pf39/R0dHPz8/Hx8fDw8PBwcG/v7+5ubmlpaWVlZWTk5OPj4+Li4t9fX1jY2NfX19RUVFPT09HR0c7Ozs1NTUjIyMdHR0bGxsZGRkTExP6+vry8vLw8PDs7Ozo6Ojk5OTY2NjQ0NDCwsK6urq0tLSurq6srKygoKCKioqIiIiAgIBycnJgYGBWVlZQUFBMTExEREQyMjIwMDAsLCwqKioiIiIaGhoUFBQSEhIODg4MDAxcblxuXG4GBgYEBAQAAAApCl0KL0JpdHNQZXJDb21wb25lbnQgOCAvRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI0IC9CaXRzUGVyQ29tcG9uZW50IDggPj4KL0xlbmd0aCAxNCAwIFIgPj4Kc3RyZWFtCnic7dxrU9pAGMXxRFoLFEUUWyxUUYpoixe8lWJBQOH7f6SenY0dlSRuMs/qZDn/l8Bw+L0Id/AuHM977xtgOwKzHoFZb2mA87fo4h3GCLSwSaDkGIEWNgmUHCPQwiaBkmMEWtgkUHKMQAubBEqOEWhhk0DJMQItbBIoOWYEvEKe522iyMs8oNlsZrL56pinx6Ivo7eMxggMNgnMLnAymRwj3/ev0eL5f1Cv1/uIisXiDXptM3psosd8PRa51dNbxbgtApcH2Gw2fd1XFNyMfdRoNLbQIfL/d47SA5vPxh7JequxsOXHbRFIoDNAHO3B9a2iCsrn835opVLpHqUHXj8bC7bCx0p6jEAC57iWE9RqtV5M5HK5X0g9gQtOiT0mTICSEfgYgVkHBtXr9Ut0pNtG/X5fnXGGgPuBRqORySaBIhH4IueBoX1D6oUggDvIcJNAkQg06TNSD4DtdnsXGW4SKBKBJi0NsFqtJtgkUCQCTXIbWKvVPqCfaDqdJtgkUCQCYxsg4NTxd4qSbRIoEoGxOQ/8goL3Cn+jZJsEikRgdOPxWH0FALg1FPt5UugmgSIRGJ3zwFR3ME82CRSJwOicB3a7XaXrdDrfUfJNAkUiMLw7VCgUFLBcLqfbJFAkAsNzHqi+NQ3cJzQYDNJtEigSgeE5D9xAAK6j1JsEikTgYn/RAQJwBVUqlT2UfJNAkQhczHngLQreq1A/ZxgOh+k2CRSJQHubBEqOEWhhk0DJMQItbBIoObY0f0PtbARmPQKznvPAfz0XsDAKZW5kc3RyZWFtCmVuZG9iagoxNCAwIG9iago1ODYKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjE1IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjkuMikgL0NyZWF0aW9uRGF0ZSAoRDoyMDI1MDQwMzE5MzIxOVopCj4+CmVuZG9iagp4cmVmCjAgMTYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDE4NDAgMDAwMDAgbiAKMDAwMDAwMDU5NSAwMDAwMCBuIAowMDAwMDAwNjE2IDAwMDAwIG4gCjAwMDAwMDA2NzYgMDAwMDAgbiAKMDAwMDAwMDY5NyAwMDAwMCBuIAowMDAwMDAwNzE4IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDMzNyAwMDAwMCBuIAowMDAwMDAwNTc1IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDU1NSAwMDAwMCBuIAowMDAwMDAwNzUwIDAwMDAwIG4gCjAwMDAwMDE4MjAgMDAwMDAgbiAKMDAwMDAwMTkwMCAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDE2IC9Sb290IDEgMCBSIC9JbmZvIDE1IDAgUiA+PgpzdGFydHhyZWYKMjA1MQolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:32:19.979127\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "55de96dee29b4139b173f55b1978a7c6", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:32:22.037354\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMTc1LjUyMjUgOTcuNTYgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicVY4xDsIwDEV3n+KfIImDkpQRqBQxFgYOEIVCRItKJXp93A6tGJ7kZ9nf1nX+PlO+xCNOV9KbpZEYRWhhUIQJjCi0ZMQ64uCUs9aJvDbZB+W8NMxaPYjuNCAou8CeFc+T1U6xxyfjhh76IMGjpBdhksiI/1+GZdFWmI/LqrNrYuqgz4z6jYYa+gGxYy7BCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMTQzCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjMgMCBvYmoKPDwgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjQgL0hlaWdodCAxMTYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDY4Cij////7+/v5+fn39/fx8fHp6enl5eXd3d3Z2dnX19fV1dXBwcG/v7+9vb27u7u1tbWtra2hoaGVlZWRkZF1dXVzc3NxcXFnZ2dXV1dTU1NRUVFPT09LS0s7OzsbGxsXFxcVFRUTExNcclxyXHIDAwP+/v78/Pzy8vLq6uro6Ojm5ubW1tbMzMzGxsa8vLy6urq4uLi0tLSysrKsrKyYmJiAgIBqampiYmJaWlpMTExCQkJAQEA6OjowMDAgICAYGBgODg5cblxuXG4ICAgGBgYCAgIAAAApCl0KL0JpdHNQZXJDb21wb25lbnQgOCAvRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI0IC9CaXRzUGVyQ29tcG9uZW50IDggPj4KL0xlbmd0aCAxNCAwIFIgPj4Kc3RyZWFtCnic7ZrZktJAGEZR0VFZBBVFUZYxigtR1AEURBjl/Z/JryuxBkJIL3Q3lfCdy06nTs5N8leS0lXBKZ36AlzDwLzDwLxzNoEbH1ydQMZAB06vMgY6cHqVMdCB06vMMPA3qFQqP4G+00BWiWQ6ZzFQ5mTgQaeBzG/ger3+DIIg+Ab0nTqydSQLIpnOVTIwU8rALKeOzHtgH4RhGEQ8B/pOVVns2pJpuBiYLS1u4FcQC4fD4R+g71SV7bqGei4GKkkZmOpUlZ0k8AGIpdPpVEe40Q7cdRnKGJgpNXOqyhioIjMMfAc6nY6ZU1X2PzByGcoYmC4tfKAYETWFG+NAIxcDs6UMzHKqyrwHPgO3AJxPgLFTVRa7AiMXA9OdBQ7s9/ufQDw6iTfbxk65zAYMTMBAdadcZgPdwHa7HdwQB74ELyJeAUWnXGYDBiZgYN4DR6PRVuAj0Gg03oJ45Q7ASuYEwECbMDDBbuBB7oFfIMspl9mAgQkYWKzA9+DDHuVyuQQuQOqsykCbMDDBOQVWq9VLsL9nPp/fBWLPcrk86JTLbMDABOcU2Gq10vcMBoOPQER2u92DTrnMBgxMwMBNcQLr9fr+TUashGEo4l6DLKdcZgMGJih8YLPZ3BrVfoBerycO/AWLxeILwIGHQOaUy2zAwAQMzHvgarXa+n9TUKvVHoPv4GbxKZA55TIbMDABA/MeCK7BfRCkIV7dTyaTN0DmVJIdDQP3KXygQDwAx+PxVtltMJvNxLdQRaey7CgYmA4DFZzKsqMwDLTh9CpjoAOnVxkDHTi9yhjowOlVxkAHTq8yBjpwepUx0IHTq4yBDpxeZQx04PQqY6ADp1cZAx04vcrOJrCwMDDvMDDvMDDv/ANQi27OCmVuZHN0cmVhbQplbmRvYmoKMTQgMCBvYmoKNjU5CmVuZG9iagoyIDAgb2JqCjw8IC9UeXBlIC9QYWdlcyAvS2lkcyBbIDExIDAgUiBdIC9Db3VudCAxID4+CmVuZG9iagoxNSAwIG9iago8PCAvQ3JlYXRvciAoTWF0cGxvdGxpYiB2My45LjIsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My45LjIpIC9DcmVhdGlvbkRhdGUgKEQ6MjAyNTA0MDMxOTMyMjJaKQo+PgplbmRvYmoKeHJlZgowIDE2CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDAxOTEwIDAwMDAwIG4gCjAwMDAwMDA1OTUgMDAwMDAgbiAKMDAwMDAwMDYxNiAwMDAwMCBuIAowMDAwMDAwNjc2IDAwMDAwIG4gCjAwMDAwMDA2OTcgMDAwMDAgbiAKMDAwMDAwMDcxOCAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzMzcgMDAwMDAgbiAKMDAwMDAwMDU3NSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA1NTUgMDAwMDAgbiAKMDAwMDAwMDc1MCAwMDAwMCBuIAowMDAwMDAxODkwIDAwMDAwIG4gCjAwMDAwMDE5NzAgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSAxNiAvUm9vdCAxIDAgUiAvSW5mbyAxNSAwIFIgPj4Kc3RhcnR4cmVmCjIxMjEKJSVFT0YK", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:32:22.077686\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "74a0f4fb9234459bb1d762be9325aeb8", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:32:23.864434\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMTc1LjUyMjUgOTcuNTYgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicVY4xDsIwDEV3n+KfIImDkpQRqBQxFgYOEIVCRItKJXp93A6tGJ7kZ9nf1nX+PlO+xCNOV9KbpZEYRWhhUIQJjCi0ZMQ64uCUs9aJvDbZB+W8NMxaPYjuNCAou8CeFc+T1U6xxyfjhh76IMGjpBdhksiI/1+GZdFWmI/LqrNrYuqgz4z6jYYa+gGxYy7BCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMTQzCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjMgMCBvYmoKPDwgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjQgL0hlaWdodCAxMTYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDc4Cij////9/f35+fn39/fz8/Pv7+/t7e3r6+vp6enh4eHb29vT09PJycm/v7+7u7upqamnp6eTk5ORkZGPj4+NjY2JiYmFhYV3d3dxcXFdXV1ZWVlXV1dVVVVDQ0M/Pz8xMTErKysjIyMfHx8dHR0bGxsPDw8LCwsHBwcFBQX+/v78/Pz09PTs7Ozo6Ojm5ube3t7a2trOzs6+vr66urqurq6oqKikpKScnJyYmJiWlpaAgIB2dnZqampkZGRgYGBeXl5UVFRQUFBAQEA+Pj44ODgmJiYgICAWFhYSEhIQEBAODg4GBgYEBAQCAgIAAAApCl0KL0JpdHNQZXJDb21wb25lbnQgOCAvRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI0IC9CaXRzUGVyQ29tcG9uZW50IDggPj4KL0xlbmd0aCAxNCAwIFIgPj4Kc3RyZWFtCnic7Zxrc9JAGEZBQEVUEBTxVlAEBLl6t0otIIhB+f8/x2cnmwEqNJvhXSvLcz61YScnZ6bsdpOB2KnjxK76AmzDwEOHgYfO0QQu/wWnVyBjoAUnAyVlDLTgZKCkjIEWnAyUlDHQgpOBkjIGWnAyUFLGQAtOBkrKGGjByUBJWdTAR6Df78dBs9l8B86A53kRnKYy7er7rubKFUXGwO1SBl7m/D8Da7XaB5AALR949U+tTCbzDBg6Q2U1X7bpiq9cGQMXA48uEH/96vQ3wXA4/ALWAsFrMJlMRALPfJl2DX3XBdnEUMZABroQ+BJgPVKn/g5w5Dk493kLtLfX6/0E+wX6rv7Ktdzl6l3mYuBRBX4FeBM8AL/A5ovqSKlUugEw5jPYL1AKBq7BwIMOnE6nbdDtdtVcs31MvV6/BbCneQ/CnAwUgYEBzgcWCgW9Dds9ZjweqzGYZB6DMCcDRWBgQDabVRePpXD7679BLpdTYwaDwSsQ5mSgCAwMcDpQ3b9Lp9Pq4nee6CHQe7TRaGTiZKAIDFQ4H/gD6Iv/ewGfz+f3wTWgxxg6GSgCAwOwDqr7ymvboGq1+gTEYrH4invA0MlAERgY4HxgsVhUE0gymXyj6XQ66kgqlVI3KG4D/MbAJQNlMQ5cLBafQLvd1qtdK5FI3AX6CWTH7/0GDJ0MFIGBAc4HajzP049Zz09OTvRB9b/qdYD6F8DQyUARGHgB5wO38hSoaSefz0dwMlAEBppwB6i9Et6YEZwMFIGBJrgdOJvN1BqoJhkGbjgZKAIDQ6lUKsEOkYEbTgaKwMBQGLjLyUAR9g9sNBofgfpMT7lcjuBkoAgMtOdkoKSMgRacDJSUMdCCk4GSsqP5GmpnYeChw8BDx/nAPyUA2gAKZW5kc3RyZWFtCmVuZG9iagoxNCAwIG9iago2NjcKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjE1IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjkuMikgL0NyZWF0aW9uRGF0ZSAoRDoyMDI1MDQwMzE5MzIyM1opCj4+CmVuZG9iagp4cmVmCjAgMTYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDE5NDIgMDAwMDAgbiAKMDAwMDAwMDU5NSAwMDAwMCBuIAowMDAwMDAwNjE2IDAwMDAwIG4gCjAwMDAwMDA2NzYgMDAwMDAgbiAKMDAwMDAwMDY5NyAwMDAwMCBuIAowMDAwMDAwNzE4IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDMzNyAwMDAwMCBuIAowMDAwMDAwNTc1IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDU1NSAwMDAwMCBuIAowMDAwMDAwNzUwIDAwMDAwIG4gCjAwMDAwMDE5MjIgMDAwMDAgbiAKMDAwMDAwMjAwMiAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDE2IC9Sb290IDEgMCBSIC9JbmZvIDE1IDAgUiA+PgpzdGFydHhyZWYKMjE1MwolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:32:23.905525\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "9dfb9ce840e94079b6afc17d7f525c1e", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:32:25.687953\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["def autocomplete_image(img):\n", " # Remove lower half of the image\n", " img_init = img.clone()\n", " img_init[:, 10:, :] = -1\n", " print(\"Original image and input image to sampling:\")\n", " show_imgs([img, img_init])\n", " # Generate 12 example completions\n", " img_init = img_init.unsqueeze(dim=0).expand(12, -1, -1, -1).to(device)\n", " pl.seed_everything(1)\n", " img_generated = model.sample(img_init.shape, img_init)\n", " print(\"Autocompletion samples:\")\n", " show_imgs(img_generated)\n", "\n", "\n", "for i in range(1, 4):\n", " img = train_set[i][0]\n", " autocomplete_image(img)"]}, {"cell_type": "markdown", "id": "a91f1554", "metadata": {"papermill": {"duration": 0.019183, "end_time": "2025-04-03T19:32:25.758236", "exception": false, "start_time": "2025-04-03T19:32:25.739053", "status": "completed"}, "tags": []}, "source": ["For the first two digits (7 and 6), we see that the 12 samples all\n", "result in a shape which resemble the original digit. Nevertheless, there\n", "are some style difference in writing the 7, and some deformed sixes in\n", "the samples. When autocompleting the 9 below, we see that the model can\n", "fit multiple digits to it. We obtain diverse samples from 0, 3, 8 and 9.\n", "This shows that despite having no latent space, we can still obtain\n", "diverse samples from an autoregressive model."]}, {"cell_type": "markdown", "id": "a8ec0a48", "metadata": {"papermill": {"duration": 0.018707, "end_time": "2025-04-03T19:32:25.795749", "exception": false, "start_time": "2025-04-03T19:32:25.777042", "status": "completed"}, "tags": []}, "source": ["### Visualization of the predictive distribution (softmax)\n", "\n", "Autoregressive models use a softmax over 256 values to predict the next pixel.\n", "This gives the model a large flexibility as the probabilities for each pixel value can be learned independently if necessary.\n", "However, the values are actually not independent because the values 32 and 33 are much closer than 32 and 255.\n", "In the following, we visualize the softmax distribution that the model predicts to gain insights how it has learned the relationships of close-by pixels.\n", "\n", "To do this, we first run the model on a batch of images and store the output softmax distributions:"]}, {"cell_type": "code", "execution_count": 24, "id": "a77602f0", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:32:25.834605Z", "iopub.status.busy": "2025-04-03T19:32:25.834420Z", "iopub.status.idle": "2025-04-03T19:32:26.770079Z", "shell.execute_reply": "2025-04-03T19:32:26.768939Z"}, "papermill": {"duration": 0.957329, "end_time": "2025-04-03T19:32:26.771812", "exception": false, "start_time": "2025-04-03T19:32:25.814483", "status": "completed"}, "tags": []}, "outputs": [], "source": ["det_loader = data.DataLoader(train_set, batch_size=128, shuffle=False, drop_last=False)\n", "imgs, _ = next(iter(det_loader))\n", "imgs = imgs.to(device)\n", "with torch.no_grad():\n", " out = model(imgs)\n", " out = F.softmax(out, dim=1)\n", " mean_out = out.mean(dim=[0, 2, 3, 4]).cpu().numpy()\n", " out = out.cpu().numpy()"]}, {"cell_type": "markdown", "id": "1dae9714", "metadata": {"papermill": {"duration": 0.018978, "end_time": "2025-04-03T19:32:26.809968", "exception": false, "start_time": "2025-04-03T19:32:26.790990", "status": "completed"}, "tags": []}, "source": ["Before diving into the model, let's visualize the distribution of the pixel values in the whole dataset:"]}, {"cell_type": "code", "execution_count": 25, "id": "a6601dcc", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:32:26.853016Z", "iopub.status.busy": "2025-04-03T19:32:26.852827Z", "iopub.status.idle": "2025-04-03T19:32:27.648364Z", "shell.execute_reply": "2025-04-03T19:32:27.647389Z"}, "papermill": {"duration": 0.81799, "end_time": "2025-04-03T19:32:27.649786", "exception": false, "start_time": "2025-04-03T19:32:26.831796", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNDA3LjAyIDMwMC42NTI2MjUgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic3V1Nj1xHcrz3r+jj7kHF+v44SpZNQDdJhH0wfFjQ3LUEUoZW2PXfd2bV9MssdpRBL5c2qgQQmsmYjunI6fe6oiLf61ffvvvrT2/f/fD6m/s//Hh7Jd+9/e3m7j/Tvz/d7f1n+vdfd3d/Tf/+dLP03YdbtMVYT1++f3wZrDU5+ewT1ez87X/cbn+8vfqaHv7b3ZrmSo7FplqfvonNupZtqfc/829+Pf3A7X/66dstelPuvpgQLf/WD7fQmqleld5fpRZMfDy58Thd6U/31/tHhCEV4+ixORtH///zu/u/3H+5v/rasyp3/47+/Uz/RqeklzfqZaqmBV/8/PxUVf/224+37++/PoitcYn+Cg/u/u3rl+rtV/oL2ftXlqCUTOxknbCZWB2xvf1w++bN7dU/ubtz9zd/7H+8N/9++9f77+zv7/92f/Pd7R/f3L7vv+/vp9ZFa6KL9Nef5Ory5+t1IbywEX/8BME5fjnF3jtTW070StSKdfnzFXvnTGC2kqJ1n6DY+frlJAcbTCg12TAfc6r8d5DcvKnM5nMJ9VMkN/8FJddoarbRz39lXf58yaFEE5itVFvSJ0j2KX85yZ2pVlNsq02dU6X0t+oshlhIarLJtRJSfpFIwozNlaSTPNH7O/f7+5ufb9m0VlNILmU/OsAnMgJcMIn+y5kl36PxzrYgJIVkMEvoP0yN+pV/jS3lzo18+dJF46wPkR9HDKX1p/H2w50Yvvr23c9/+Oe//PiHX3776sNPv/zlt/u3/3n//sv02yV6QdXqUpCGq9rndNwla5KvpZQavf8/abnfoeV8prakhJTLukFqn9Ny7zIdK8WG1CItW6TlrvQj+0v03H16z0NtvYP/Dz2v1J4Yc26q51L72xcG3PRKbxw1+RYttevzXuek0oZqa6JOr1tuHy3/X/UqDGW+VRuad4n+UCaOBa4PiRa8+bHEjX1FbFKXvvj5+/TzP7y+feJP8upZFqNf5VQbvfQdvUQq/WkysfSHfYS8V4ivtDahl2V/41PrWl2mVfU3e/am67TNtufeOBNLX4A+9eZC6CvCSnaPjnUmXd24M6SSSs09d4ak0FtMzM+duZBSqUfWJ/fSr8Gkqxt3hp5woGMgPHcm0td9ufjUmQspfJ7PpbSXfg0mXd24M6TSpkin5afOJFoC1Pzxi+m9AHRazznxq+S94tHVjfvCIkMOHx8x1JdsUszl6Sh7rxCyDclFcpuPdnUmXd24M6zSVffxEUOdKaa62J6OsvcKKc3kkoP0qzPp6sadYZXPXWnG1uCejrD3CimZVYb+zjxYdGXjjpBCWlg/rVY+3IqltYgL4eMVznuF0HuPpTOsLS+9Gky6um9nWGWkk8LHaxXqDK1FXCMf/tyZC6nBhJwdmcb3iklXN+4Mq4z0VvLcGbYRpdjnM68gvItRqQnp0a/OpKsbd4ZVQldA7yx0GkGuQBB9jhWmM868XSX0BCWZ6KEnEESv5ITpjPUdq3TQE9A7TG7QEwiiz7XCdMgZmFRW6AlKMS1DTyAIdyN699KtwSO1jbtCChP0A3Sm8B75gQugs0rKKbvy0qnBo6sb94VFQj9Q6YEN+gFBqAd0xDSXH+0afkBV9+1MVwn9QHWmZOgHBCHfGPmr+OhXZ9LVjTvDKp+7EowN0A8IQu9CrdKrpDx6NVU27ggp9NAP8ArNQj8gCHmA5nOr7aVXg0lXN+5MpJ+FfqAmOn9CPyCIPpMI0yHnF1YJ/UDNpgboBwTR61thOmPV21VCP1CrcTglECQ18tmV5D361Zl0dePOsEroByrpwxmBINkaso0l5Ee/Rkagqht3hlTijKDRz+KMQJDMj661lZd+DSZd3bczrBJnBPR1wxmBIMVywhR7TiJMurpxZ0glzgha4OAceIILyJ53XmJf/QuPrm7cFxYJPUGj9SvOCATJyZTisvWPdnUmXd24M6wSeoJG+nBGIEimtV3LdFw9+jU8gapu3BlW+dyVTKt76AkEofOttSV0TzRYdGXjjpBCnBG0ajzOCATRu3TCdMbeHavEGUFrJuGMQBCdIAnTGblSVwk9AS3WTMEhgYKm3lxcZ/RmyIS2wFl2hNAXKChRuVmrejacgSrv3B3WCa2B42lpnBUoKCU6yQRb3dW0TqbLO3eHdOK4gN54TcJ5gYL0xp0iO2M/r+vEkQF56sfY+XN3LkgbJUV2hn/qOnF04Cw/eeQTBCn06BZLD68VlS7v3Bt+htAqOFv5QhLkFRRUoqHfVMPVssdA0aO6c29YJTQLzjaTcYKgIFJYXIo1Xi3rZLq8c3dY53NnnDUNpwgK0mbyhegMf9k14iCBnrHxOElQEEnjlV6Lj4a9ZAmqvHN3vHE4THAu8GQvcg4Kml43QnbKa4d1YvPgIk/3QvMgUC4mezr1xKtpnUyXd+4O68TuwWWe8YXuQaAUjKOib1fThntQ5Z27wzqxe3DFeJwsKGjuzkV2SndIJw4XnKsm4XRBQYm/osWgezRtkOnyzt0hGThgcK6ZihMGBU3eSsgO8VasE4cMji/9gymDIPTW5HwqLjxa9pIzqPLGvekysXvwnqd/oXsQaHrHErJD3rG6TuwffOAJYOgfBNIXfymyM64JGzpBZ/gKZuwfBNKjNS9EZ0zbdI04dHCenjVOHRSkhwEU2RkzAl0nDh4c6Yg4eVBQ5nvQOL4vxntNpss7d4d1Yv/gK88CQ/8gUOJLv1OI7WpaJ9PlnbvDOrF/8I1ngqF/EGjuzkV2SndYJ/YPwRm/SB8EmnZJheyQXVLWuUgfgjdxkT4INHVHyE7pDulcpA8hmLJIHwRK3riQfEmPpg0yXd65O6RzkT6ExLPCyD9cyLQGFKpD1oBdJvYPIfO0MPQPAqVsaiqxpqtnnUyXd+4O68T+IRSeGIb+QSA9zKbIzphxGzpBZ6qpi/xBID1M/EJ0xnxx17jIHyJ5gEX+INDkrITsEHdFOu0if4iOp4ahfxCoJfoqlOhemvZCpss7d4d1Yv9AYvIifxAo8503ve+DkIpMl3fuDuvE/iEGnh+G/kGgaZdUyA7ZJe06sX+IybhF/iBQrrwoHhvswqWqO/eGVC7Sh5jpodg9CJRoYZNt6rdnUWS6vHN3SOcifYg8KYHdg0DTlIWQHTJlwToX6UOsPFWM3MOF6NF0RXXGxPqQid1DsjxXDN2DQNP6WMgOWR93ndg9JMeTxdA9CDRlnkJ2SObZdYLOkJBF+iDQlOkNokPyPNa4SB9SNHaRPgg07eQI2SE7OaxzkT7Qe3FYpA8Cza+bi+yU1w7rxO4hZZ4ohu5BoGmPVMgO2SPtOrF7oCdfF+mDQNMaUMgOWQN2ndg9pEZPHrsHgabUU8gOST1Z5yJ9IIVhkT4INO2SCtkhu6Ssc5E+8C75In0QaLpuRsgOuW6mBwXYP9A6t+H04UKmPQuhOmTPosvE/oEUukX6IBDZqFJrrtKz4R9UeefusE7sH2jFEhfpg0AxGW+jLf5qWifT5Z27wzpBZ7Ipi/RBoMrrZVr8uathU2nnrpDGRfqQq7GL9EGgaR5QyA6ZByQZbZE+5MYzxdA/CFSK8c4F3/crhEyXd+4O68T+oVieKYb+QaBpgl3IDplg7zqxfyiOp4qhfxCob/7RWi9cTetkurxzd1gn9g8lGLtIHwSapiyE7JApC9a5yB/4Y6QW+YNAMZia6WVSHk0bZLq8c3dI5yJ/KMnkRf4g0OSuhOwQd8U6F/lDyTxdjPzDhUzZjFAdks10mdg/lMqzxdA/CDQ5TyE7xHl2ndg/lMazxdA/CDR5TyE7xHt2nc+dqfSTi/xBoGm3axAdstPFGhf5Q3WmLfIHgaZET8gOSfRY5yJ/qIEHJaB/EGhy5EJ2iCPvOrF/IHcdF/mDQFPeKWSH5J1dJ/YPNfFUMfQPAk3+QcgO8Q9dJ/YPla8Nwv5BoOmdXMgOeSdnnYv8oVbiwf5BoCkVFrJDUmHWucgfajNpkT8INF2TJmSHXJPGOhf5Q7M8XYz8w4VMqadQHZJ6dpnYPzTPE1rQPwgUaOkXXj42U5Hp8s7dYZ3YPzT+PCnsHwSaVjtCdshqp+sEnYkmL/IHgabcahAdklmxxkX+0Oi9eJE/CJQiSaPHt0fDBpku79wdHqXB/qEVniqG/kGgyV0J2SHuquvE/qFVniqG/kGgadZWyA6Zte06sX9ozZRF/iDQfGRdZKccWawT+gdvLXkA6B8UFGu/UYPzV9M6mS7v252uE+cP3nrjcf6gIG2jFNkZ7qrrxPmDt8EknD8oSN/8TpGdcU+8rhPnD7SK4/li4B8EKYWvTXN9KayodHnn3rBM6B+8zTxdjPyDgnSAp8jOyPWGTugfvC08XYz8g4L0lroiO2OnfegEnakm4fxBQXrj5oXojL2crhHnD942U3H+oCC9nFFkZ6xyuk6cP3jneKoY+QcFaZOpyM7wnkMn9A/esUOC/kFBuZrSWulb6YpMl3fuDuuE/sG7wHPFyD8oaDqyhOyQI6vrxP7BRdNw/qAgbRQU2SH+gXXi/ME7etY4f1CQjvAU2RnJXteJ8wfviok4f1CQHpBUZGfMTXadOH/wrvJ8MfIPFzI5T6E6xHl2mdg/uMbTxdA/CDQ5TyE7xHl2ndg/eMfTxdA/CDQfVxfZIcdV1wk6403C+YOC+geF0HqvfyL9C9NU27kvpBInEN6Tr8YJhIKm9yohO+S9inQWnEB4z5kudhACTc5TyA5xn10ndhA+81wxdBAC6ctkFNkZV88MndhBkI6MEwgF6aEBRXbGLMHQiR2EZxeAHYRA85F1kZ1yZJHORQIReD8POwiB9C0aFNkZd27oOhcJRHAmLhIIgfStExXZGXdU7DoXCUTwPGGMHMSFTLvIQnXILnKXiR1ECDxfDB2EQPMr5yI75ZXDOrGDCInni6GDEEiPDiiyMyYKhk7QmWziIoEQaH7ddKJTXjOkcZFAhGLKIoEQaHofF7JD3sdZ5yKBCI3niqF/EEhfKq3IzriCeujE/iFaniuG/kGg+XxzkR1yvuk6sX+IjieLoX8QSF/qoMjOuAJi6MT+gcTURQIh0JTPCNkh+QzrXCQQMdLRgf2DQPqDlBXZGZ+v3HUuEojId8bG/kGg+ci6yE45skjnIoGImSeMkX+4EH3zbEV1xj21h0zsH+ivXxcJhED6QnJFdsb15UMn9g+x8Xwx9A8CJSo3a1XTRgKhyjt3h3U+dybRwxYJhEDTfs4gOmQvhzUu8gf6u5dF/iDQlOkJ2SGZHunMi/yBHFJb5A8CTROTQnbIxGTXif0DreP8In8QaHLlQnaIM+86sX9IiSeLoX8QaDobC9khZ+OuE/uHlOncgf2DQJN/ELJD/APrXOQPqRq7yB8EmuZQhOyQORTWucgf+NyxyB8E4g99qC4P/yBkurxzd0jnIn8gF5Bw/nAh036gUB2yH9hlYv+QHc8XQ/8g0JTOCNkh6UzXif1DDjxfDP2DQPqjTRXZGZ94OnSCzkQTFvmDQNNRNYhOOaJI4yJ/IA+QF/mDQJOzErJD3BXrXOQP9KzbIn8QSN9QUpGdcZ/JoRP7h1x5rhj6B4Hm7lxkp3SHdWL/kBtPFkP/INDkH4TsEP/QdWL/UKwpi/xBoGhNazblcDVtXAGhyht3h3Uu8ofiSCH2DwJNu8hCdsguMutc5A8lGL/IHwSix6fQbLyaNsh0eefukM5F/lAizxcj/3Ah9OiUcint0bJBpcs794ZlYv9QEk8XQ/8gULX8icI1pqtnnUyXd+4O68T+oRSeLob+QaDJPwjZIf6h6wSdqSYs8geBpt2uQXTIThdrXOQPpZm8yB8Emq4RFrJDrhEmnWmRP/A5Y5E/CDR35yI7pDtdJ/YPld019g8CTWmwkB2SBned2D/UwHPF0D8IpG9Ir8jOuE/90In9Q40mL/IHgaYddiE7ZIeddS7yh0oKF/mDQNN5R8hOOe+QzkX+UHkyFPsHgchQxRJjco+mDTJd3rk7pHORP9TK88XIP1xI8Ybsd3P50bKXOzCp8s69YZnYP9TG08XQPwg0+XIhO8SXd53YP/RvsH8QiKRFG/y4nlHIdHnj7nSdoDPe+EX+INDkHwbRIf6BNS7yh8Z7Dtg/CDR5TiE7xHOyzkX+0CJPFUP/INA0oSNkh0zodJ3YP7TMU8XQPwg0pXpCdkiy13Vi/9AKzxVD/yDQdI8hITvkHkNdJ/YPrZq8yB8Emo+si+yUI4t0LvKH1kxd5A8CTamwkB2SCrNOnD8E64zD+YOC9EdaKbIzPumq68T5Q7Ce54uBfxCETjTF2tIPIEWlyzv3hmVC/xBs4Oli5B8UpA8gRXbGcTV0Qv8QbOTpYuQfFJQrf/JVv2eO4lLVnXvDKkFfsvE4fVCQvqXkC9EZd5nsGnH6EGwxCacPCtLWUpGd4ThZZ8TpQ7CVZ4qRe1CQ3tBSZGfscw2d0D0EZ3mmGLkHBelIXJGdkZQPndA9BOd4qhi5BwVVenwuoeSraZ1Ml3fuDuuE7iE4bxJOHxSkP5RIkZ3xWUVdJ04fggum4vRBQfoGBIrsjPsSdJ04faA3a3rW2D0IpG8oqcjOuM9k14nTh+AyTxcj93Ah00pHqA5Z7XSZ2D24wrPF0D0IpD9iUJGd8cmDQyd2D67ybDF0DwLp0VFFdsZE6dD53BlvjcPpg4IKl10eK51BpEsbd4U14vQheGciTh8UpMcFFNkZUwRdJ04fgvc8Uwz9g0D6sipFdsbVVkMn9g8+8Ewx9A8C6dBXkZ2RBQ+d2D/4xFPF0D8IpDfSFdkZ++tDJ/YPPpuE0wcF6UtkFNkZV850nTh9IBNJKznsHwTSt1lSZGfcfanrXKQPvvG9jaF/EEgPcSmyM2a7us5F+hAsTxcj/3Ah81nnojrkrNNlYv8QHM8WQ/8gUGErnnL/4FdFpss7d4d1Yv8QPM8WQ/8gUImGflMNV886l6ru3BtWCfoSjVukDwJN65xBdMgahzUu0oeQTFykDwKVQOfkFMujX4NLVXfuTTJhkT2EzPPE0DsINJ2JheyUMzHrxN4hFJ4nht5BIP0hBorsjM82GDqxdwiNJ4qhdxBI58DCdUYO3FVi5xB7Lgedg0DTDqmQHbJDyjoXyUN0piySB4H0jaEV2Rn3i+46F8lDpDedRfIg0JSWC9khaTnrXCQPMfJcMXIOF6I/dEdRnfFZPEMmdg4x8VQxdA4CTZMEQnbIJEHXiZ1DzDxVDJ2DQJWMpqWDKV5N62S6vHN3WCfoTDV2kTwINB9VneiUI4o0LpKHyGtc7B0EmjIZITskl2Gdi+QhWZ4mhu5BIH0JsCI748rgoRO7h+R4mhi6B4FK5O10H8rVtDG5pMo7d4d1YvfAMxKL5EEgfRsYRXbG3WGGTuwfUjRxkTwINPkHITvEP7DORfJA3jovkgeBCl/Saf1Y5wiZLu/cHdK5SB5SNm2RPAik97uE64z9Lla5yB1S5bli5B4uZErKheqQpLzLxO4hNZ4qhu5BoOqMt+3xwrnIdHnn7rBO7B7o718WuYNAhRbEoTifrqaN4EGVN+5O1wk6441dJA8CTVfLDKJDrpRhjYvkIQcTFsmDQNM+l5Adss9FOv0ie8iRp4mhexBIJ3nCdUaS11Vi70DHRl0kDwJV4k/B5XS1rJPp8s7dYZ3YO+TC08TQOwhEb0mp2tg/NkWR6fLO3WGd2DvQ+j8ssgeBSjOeXIItV9M6mS7v3B3SucgecjN5kT0IxLeTdLGOTE/IdHnn7jRejkDvUHgaAHsHgegQajGnEB5NG2S6vHF3WOcieyiep4qRe7iQad5NqA6Zd+sysXsg6xgX2YNAfK/JkoPq2cgeVHnn7rBO7B7ozFEW2YNA016gkB2yF9h1gs4kUofdg0CVH1vrWAYOIl3auSt8eQJ2D+wAFtmDQJX3u4KP/tGwQabLO3eHdC6yh1J5lhi6B4Gcd4boQvCPrg22qb5zf1gpdhB0Ti2L9EEgR2ub1ug35KtvnW2q79wfVoo9BK1W7CJ/EMjVwPc6qa1dfetsU33j/nSl2EVUb8IigRDI0Xt4LbQm9FffOttU37k/pHSRQdArIC0yCIFc4j6k4dCFTJev7rz62rNwf//u7ujsTprc/fX91bfv/vrT23c/vP7m/va320cd/PBSaLRWSD73/cXbj7fA9zHy0889Sh/95BPf00PVj8ljl3zf3/4brljuhQplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjYxMjkKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTcgMCBvYmoKPDwgL0xlbmd0aCAzOTUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVJLbsVACNvnFFyg0vCbz3lSVd28+29rQ1KpKryJMcYwfcqQueVLXRJxhcm3Xq5bPKZ8LltamXmIu4uNJT623JfuIbZddC6xOB1H8gsynSpEqM2q0aH4QpaFB5BO8KELwn05/uMvgMHXsA244T0yQbAk5ilCxm5RGZoSQRFh55EVqKRQn1nC31Hu6/cyBWpvjKULYxz0CbQFQm1IxALqQABE7JRUrZCOZyQTvxXdZ2IcYOfRsgGuGVRElnvsx4ipzqiMvETEPk9N+iiWTC1Wxm5TGV/8lIzUfHQFKqk08pTy0FWz0AtYiXkS9jn8SPjn1mwhhjpu1vKJ5R8zxTISzmBLOWChl+NH4NtZdRGuHbm4znSBH5XWcEy0637I9U/+dNtazXW8cgiiQOVNQfC7Dq5GscTEMj6djSl6oiywGpq8RjPBYRAR1vfDyAMa/XK8EDSnayK0WCKbtWJEjYpscz29BNZM78U51sMTwmzvndahsjMzKiGC2rqGautAdrO+83C2nz8z6KJtCmVuZHN0cmVhbQplbmRvYmoKMTggMCBvYmoKPDwgL0xlbmd0aCAyNDkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVFJigMwDLvnFfpAIV6TvKdDmUPn/9fKDoU5BAmvkpOWmFgLDzGEHyw9+JEhczf9G36i2btZepLJ2f+Y5yJTUfhSqC5iQl2IG8+hEfA9oWsSWbG98Tkso5lzvgcfhbgEM6EBY31JMrmo5pUhE04MdRwOWqTCuGtiw+Ja0TyN3G77RmZlJoQNj2RC3BiAiCDrArIYLJQ2NhMyWc4D7Q3JDVpg16kbUYuCK5TWCXSiVsSqzOCz5tZ2N0Mt8uCoffH6aFaXYIXRS/VYeF+FPpipmXbukkJ64U07IsweCqQyOy0rtXvE6m6B+j/LUvD9yff4Ha8PzfxcnAplbmRzdHJlYW0KZW5kb2JqCjE5IDAgb2JqCjw8IC9MZW5ndGggOTQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRY3BEcAgCAT/VEEJCgraTyaTh/b/jRAyfGDnDu6EBQu2eUYfBZUmXhVYB0pj3FCPQL3hci3J3AUPcCd/2tBUnJbTd2mRSVUp3KQSef8OZyaQqHnRY533C2P7IzwKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvVHlwZSAvWE9iamVjdCAvU3VidHlwZSAvRm9ybSAvQkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0xlbmd0aCAzOQovRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzjMjQwUzA2NVXI5TI3NgKzcsAsI3MjIAski2BBZDO40gAV8wp8CmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMjIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVG7bcUwDOw1BRcwIH4lzeMgSJG3f5s72qlI07wfVV4ypVwudckqWWHypUN1iqZ8nmam/A71kOOYHtkhulPWlnsYFpaJeUodsZos93ALNr4AmhJzC/H3CPArgFHARKBu8fcPulkSQBoU/BTomquWWGICDYuFrdkV4lbdKVi4q/h2JLkHCXIxWehTDkWKKbfAfBks2ZFanOtyWQr/bn0CGmGFOOyzi0TgecADTCT+ZIBszz5b7OrqRTZ2hjjp0ICLgJvNJAFBUzirPrhh+2q75ueZKCc4OdavojG+DU7mS1LeV7nHz6BB3vgzPGd3jlAOmlAI9N0CIIfdwEaEPrXPwC4Dtkm7d2NK+ZxkKb4ENgr2qFMdyvBi7MxWb9j8x+jKZlFskJX10ekOytygE2Ieb2ShW7K2+zcPs33/AV8Ze2QKZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDgzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWMuw3AMAhEe6ZgBH4m9j5RlMLevw0QJW64J909XB0JmSluM8NDBp4MLIZdcYH0ljALXEdQjp3so2HVvuoEjfWmUvPvD5Se7KzihusBAkIaZgplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMzIwIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSS24FMQjbzym4QKXwT87zqqqLvvtvaxO9FUwwYOMpL1nSS77UJdulw+RbH/clsULej+2azFLF9xazFM8tr0fPEbctCgRREz1YmS8VItTP9Og6qHBKn4FXCLcUG7yDSQCDavgHHqUzIFDnQMa7YjJSA4Ik2HNpcQiJciaJf6S8nt8nraSh9D1Zmcvfk0ul0B1NTugBxcrFSaBdSfmgmZhKRJKX632xQvSGwJI8PkcxyYDsNoltogUm5x6lJczEFDqwxwK8ZprVVehgwh6HKYxXC7OoHmzyWxOVpB2t4xnZMN7LMFNioeGwBdTmYmWC7uXjNa/CiO1Rk13DcO6WzXcI0Wj+GxbK4GMVkoBHp7ESDWk4wIjAnl44xV7zEzkOwIhjnZosDGNoJqd6jonA0J6zpWHGxx5a9fMPVOl8hwplbmRzdHJlYW0KZW5kb2JqCjI0IDAgb2JqCjw8IC9MZW5ndGggMzQwIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSOW4EMQzr/Qp9IIBu2+/ZIEiR/L8NqdkUA3F0UpQ7WlR2y4eFVLXsdPm0ldoSN+R3ZYXECcmrEu1ShkiovFYh1e+ZMq+3NWcEyFKlwuSk5HHJgj/DpacLx/m2sa/lyB2PHlgVI6FEwDLFxOgals7usGZbfpZpwI94hJwr1i3HWAVSG9047Yr3oXktsgaIvZmWigodVokWfkHxoEeNffYYVFgg0e0cSXCMiVCRgHaB2kgMOXssdlEf9DMoMRPo2htF3EGBJZKYOcW6dPTf+NCxoP7YjDe/OirpW1pZY9I+G+2Uxiwy6XpY9HTz1seDCzTvovzn1QwSNGWNksYHrdo5hqKZUVZ4t0OTDc0xxyHzDp7DGQlK+jwUv48lEx2UyN8ODaF/Xx6jjJw23gLmoj9tFQcO4rPDXrmBFUoXa5L3AalM6IHp/6/xtb7X1x8d7YDGCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAyNTEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLVFJcgNBCLvPK/SEZqffY5crh+T/1wjKBwYNi0B0WuKgjJ8gLFe85ZGraMPfMzGC3wWHfivXbVjkQFQgSWNQNaF28Xr0HthxmAnMk9awDGasD/yMKdzoxeExGWe312XUEOxdrz2ZQcmsXMQlExdM1WEjZw4/mTIutHM9NyDnRliXYZBuVhozEo40hUghhaqbpM4EQRKMrkaNNnIU+6Uvj3SGVY2oMexzLW1fz004a9DsWKzy5JQeXXEuJxcvrBz09TYDF1FprPJASMD9bg/1c7KT33hL584W0+N7zcnywlRgxZvXbkA21eLfvIjj+4yv5+f5/ANfYFuICmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0xlbmd0aCAyMTUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVE5DgMhDOz3Ff5AJIwveE+iKM3+v82M0VYewVyGtJQhmfJSk6gh5VM+epkunLrc18xqNOeWtC1zgLi2vC+tksCJZoiDwWmYuAGaPAFD19GoUUMXHtDUpVMosNwEPoq3bg/dY7WBl7Yh54kgYigZLEHNqUUTFm3PJ6Q1v16LG96X7d3IU6XGlhiBBgFWOBzX6NfwlT1PJtF0FTLUqzXLGAkTRSI8+Y6m1RPrWjTSMhLUxhGsagO8O/0wTgAAE3HLAmSfSpSz5MRvsfSzBlf6/gGfR1SWCmVuZHN0cmVhbQplbmRvYmoKMTUgMCBvYmoKPDwgL1R5cGUgL0ZvbnQgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvRmlyc3RDaGFyIDAgL0xhc3RDaGFyIDI1NQovRm9udERlc2NyaXB0b3IgMTQgMCBSIC9TdWJ0eXBlIC9UeXBlMyAvTmFtZSAvQk1RUURWK0RlamFWdVNhbnMKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXQovQ2hhclByb2NzIDE2IDAgUgovRW5jb2RpbmcgPDwgL1R5cGUgL0VuY29kaW5nCi9EaWZmZXJlbmNlcyBbIDQ4IC96ZXJvIC9vbmUgL3R3byAvdGhyZWUgL2ZvdXIgL2ZpdmUgL3NpeCA1NiAvZWlnaHQgL25pbmUgXQo+PgovV2lkdGhzIDEzIDAgUiA+PgplbmRvYmoKMTQgMCBvYmoKPDwgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9Gb250TmFtZSAvQk1RUURWK0RlamFWdVNhbnMgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0FzY2VudCA5MjkgL0Rlc2NlbnQgLTIzNiAvQ2FwSGVpZ2h0IDAKL1hIZWlnaHQgMCAvSXRhbGljQW5nbGUgMCAvU3RlbVYgMCAvTWF4V2lkdGggMTM0MiA+PgplbmRvYmoKMTMgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTYgMCBvYmoKPDwgL2VpZ2h0IDE3IDAgUiAvZml2ZSAxOCAwIFIgL2ZvdXIgMTkgMCBSIC9uaW5lIDIxIDAgUiAvb25lIDIyIDAgUgovc2l4IDIzIDAgUiAvdGhyZWUgMjQgMCBSIC90d28gMjUgMCBSIC96ZXJvIDI2IDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTUgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAwIC9jYSAxID4+Ci9BMiA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAxID4+Ci9BMyA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAwLjUgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0YxLURlamFWdVNhbnMtbWludXMgMjAgMCBSID4+CmVuZG9iagoyIDAgb2JqCjw8IC9UeXBlIC9QYWdlcyAvS2lkcyBbIDExIDAgUiBdIC9Db3VudCAxID4+CmVuZG9iagoyNyAwIG9iago8PCAvQ3JlYXRvciAoTWF0cGxvdGxpYiB2My45LjIsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My45LjIpIC9DcmVhdGlvbkRhdGUgKEQ6MjAyNTA0MDMxOTMyMjdaKQo+PgplbmRvYmoKeHJlZgowIDI4CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDExNzA4IDAwMDAwIG4gCjAwMDAwMTE0NDUgMDAwMDAgbiAKMDAwMDAxMTQ3NyAwMDAwMCBuIAowMDAwMDExNjE3IDAwMDAwIG4gCjAwMDAwMTE2MzggMDAwMDAgbiAKMDAwMDAxMTY1OSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzNDAgMDAwMDAgbiAKMDAwMDAwNjU2NSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDY1NDQgMDAwMDAgbiAKMDAwMDAxMDI1NCAwMDAwMCBuIAowMDAwMDEwMDQ3IDAwMDAwIG4gCjAwMDAwMDk2ODEgMDAwMDAgbiAKMDAwMDAxMTMwNyAwMDAwMCBuIAowMDAwMDA2NTg1IDAwMDAwIG4gCjAwMDAwMDcwNTMgMDAwMDAgbiAKMDAwMDAwNzM3NSAwMDAwMCBuIAowMDAwMDA3NTQxIDAwMDAwIG4gCjAwMDAwMDc3MTMgMDAwMDAgbiAKMDAwMDAwODEwOCAwMDAwMCBuIAowMDAwMDA4MjYzIDAwMDAwIG4gCjAwMDAwMDg2NTYgMDAwMDAgbiAKMDAwMDAwOTA2OSAwMDAwMCBuIAowMDAwMDA5MzkzIDAwMDAwIG4gCjAwMDAwMTE3NjggMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSAyOCAvUm9vdCAxIDAgUiAvSW5mbyAyNyAwIFIgPj4Kc3RhcnR4cmVmCjExOTE5CiUlRU9GCg==", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:32:27.250997\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["sns.set()\n", "plot_args = {\"color\": to_rgb(\"C0\") + (0.5,), \"edgecolor\": \"C0\", \"linewidth\": 0.5, \"width\": 1.0}\n", "plt.hist(imgs.view(-1).cpu().numpy(), bins=256, density=True, **plot_args)\n", "plt.yscale(\"log\")\n", "plt.xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "da9736db", "metadata": {"papermill": {"duration": 0.021155, "end_time": "2025-04-03T19:32:27.693824", "exception": false, "start_time": "2025-04-03T19:32:27.672669", "status": "completed"}, "tags": []}, "source": ["As we would expect from the seen images, the pixel value 0 (black) is the dominant value, followed by a batch of values between 250 and 255.\n", "Note that we use a log scale on the y-axis due to the big imbalance in the dataset.\n", "Interestingly, the pixel values 64, 128 and 191 also stand out which is likely due to the quantization used during the creation of the dataset.\n", "For RGB images, we would also see two peaks around 0 and 255,\n", "but the values in between would be much more frequent than in MNIST\n", "(see Figure 1 in the [PixelCNN++](https://arxiv.org/abs/1701.05517) for a visualization on CIFAR10).\n", "\n", "Next, we can visualize the distribution our model predicts (in average):"]}, {"cell_type": "code", "execution_count": 26, "id": "1c9cf42b", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:32:27.736877Z", "iopub.status.busy": "2025-04-03T19:32:27.736687Z", "iopub.status.idle": "2025-04-03T19:32:28.487714Z", "shell.execute_reply": "2025-04-03T19:32:28.486747Z"}, "papermill": {"duration": 0.77431, "end_time": "2025-04-03T19:32:28.489128", "exception": false, "start_time": "2025-04-03T19:32:27.714818", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNDA3LjAyIDMwMC42NTI2MjUgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic1d1NkxzHcQbg+/yKOUoHJur740iaNiJ4I4mwDw4fFDAkE8Glg2JI/vvOt3p3Mru30gELgh1pBOXFm7u1Ww9me6omu3vefPvhrz+9//DD22/u//Dj7Y387f1vt3j/yP/96R7uH/m//7rH+1v+70+3wH97upXQKST+8OeXD3MI1GpqqXIWzn/9j9vtj7c3X/OX/3YPNGNvpYc6xqu/lBnibKGP+5/xnd+ePuH2P3327VYS9XvqlEvAd3265TlpJBX9/IhmpvLywx1fp5P14/56vwyYa6fIX9saRf7/f/5w/5f7L/c3XyfMKt6/4/8+8n+HlFje2LJOqq2NOE4/n0r1d7/9ePv+/uvLwIFi5X+Fl7HXX98+p7df+V8o3L8KXKqNwhoMf+6Tyog82vun2zfvbm/+Kd5jvL/74/rHe/fvt3+9/y78/v5v93ff3f7x3e379f3+frONJVDPodR6mq6OP3++MedjNPypnzDhVr7cjFOKNGYJM55mrOPPn3GKkTJGw5/4CTOOaXy5KeeQKdTe0+V3TsV/hynPRHWNhj+fMuWZvuCURyH+J82tnKes4s+fcu78uMZo+FM+Ycqpti835TXS4P/h/ytJjqkS/a3z7JTufVCJI7fGR/L5PEWeGB/IRq+Ynsz3d/H393cfb43mHDXXWFs6BHAg40LM/ECpfDjFlO/8DxLDzDJI52lglLw+maF+xbcJvd8B+fxhLBRDygVfxyP0uX6M9093HuGrbz98/MM//+XHP/zy21dPP/3yl9/u3/7n/fsv4x1rpJBjm1nAVfY54rE0aj3l2EcP/zfkyQN5ipPSqK0UtW6Q7HPIE89x8iGBZzajJud/A/xmfwnz+Onmecwl+P9gPjrVPOvUazXJ/vaFAdBHpjgC/xP2mD7zcc6zDHmEUVnaJg8v5P8rq3zMLM0R8kyx8j8UlWOBm3LlBW97WeKWtSKmuqZufP799Pk/vL194mdi9VwHzbyeZb9q/GvY+Mgcc2r8T9N4lIA1x6Xys6qkwWsTfliuJz4Z6hTzqvobnzaWANtEKm0tx17ZPCq1Uh0l5XkS06ljGWP+LMNTyfwYyK9lHpXGG6CZZ8gnL506ljHmzzKFj8Nr0/JK5lHp/HvDG906T146dSxjzJ9lKi8BsO59LfOoYHnc+ggXL5U6ljHmzzKNalk7+1cyjwobFH6aCq+8HqljGWP+LNNpbI6+K+1YUobGT/MnJ5U6FtnMmzUm8WdEXkm9EnlUeP4Bq9qpnR6JYxFj7k+3HngWazt9VZEKS4RaRyonK536lbHmzzKRWsQraK9lHhU8OlJdCxUZSIWOXYzZs0si/prXq+GfVYUJeKswc7xqPVLHMsb8WYZ3j3W7G5AKf8SrlFHj1euROpYx5s8ylT/e7gWkgnTWdMg8RtKpYxlj/izTiLfdu72AVKDRSlqrfhlJp45ljPmzTKdZt3sBqbRMYeBbnbx06ljGmD/LTEppuxeQCq9yWwyx9bOXSh3LGPN/ug3+wrndC0iF1y015OP1eeWlUr8y1vxZJvIu8LXKSnmvyAgllrOTSh2LbObNGnyESNu9gFRa4jnlmKI4qcSxiDF3VuF/8bndC0ilDeol1jbPVip1LGPMn2Uqr1y3ewGp8BEkR/boZy+VOpYx5s8yjcbutfGfVYXXuaWFOcvJS6eOZYz5s8yguO8NSIVXKjO3EdPJS6eOZYz5s8ykvO8MSIWnN1sK9cSlQscuxuyfbpM/d98XkApe8R58RJknLZ36lbHmzzKR5r4vIBW8ANNbWB0AGUmnjmWM+bNMprjvC0ilDh6vpfVKnYykU8cyxvxZpvAXbvcCUmmr6328uiAj6dSxjDF/lqnUXx97j5TnXkcKaV6dHqljkc28WaPR3PcFpDISpdJSyQ8nnTgWMebOKgPz2+0FpILpRd4jnq106ljGmD/LTKr7voBUeO9cKx9bT1wqdOxizP7pxms0XoRstwKqxDvFPEZs4+ylUr82pgBwEoV9b0CVSqeAbzXPZjr2rGMQQCfzLnm7I1ClWqmMEso4o+nYs45BAB0+cOx7BKqEp+jQw4hnNB171jEIoFNp7PsEqlQnhVTquKDp2LOOQQAd/PDb/YEqtcpP3iH2C5qOPesYBNDh5519v0CVGn99y8dLeBpNxZ51DALoTGq7x82KC85kHb2OM5aOPatsps4iMdDc9w1UqTZqKc7VOHkeSEeOVazpQyZR3PcOVKnhqXqMddWAGkzHnnUMAuhkKvv+gSrxDPkb1X42U6lnGwMANoX6voOgSvxEPXB2b72QqdizjkEAncZr//2+QUq8BObJzhIvaCr2rGMQQKdT2ncSVIkPvjGlZxwxk9SzjQEAm0F1301QpZr5OMPP1/NCpmLPOgYBdPjIse8oqFJLOD0kX8xU6tnGAGCbFCnsewqqxIfeWCevhc9kOnasYxFAJ/E89nsGKdXEy+DIW8sLmoo96xgE0MnUNkfjI+YHR6qltHTBUrFnlc3UIVJo7PsLqtQyjZjDsZs6BtKRZxVj+pDhn3rfY1ClUvkjXuRdwHTsWccggE7nv+z3DFLiJV6bZcR6RtOxZx2DADqDmtFskFKNxHtuPGef0VTsWccggM6kaXQbpFQDRTwvXcwk9WxjALBNjpSMXoOU2qCSRh3pTKZjxzoWAXQSFaPXICWcUM5jHzoymI496xgE0MnUjV6DlErBhf3jeP1PBtOxZx2DADqVgtFrkBIu7i9jHq+my2A69qxjEECn8Tj7fYOUeONdcLrAxUxSzzYGAGw61d2jZsX8jNRaSzFdqFTsWWUzdYgMGkanQUq1UY68h+oKS0WeVYzpH7fKC0anQUotUuRvVs5eKnVsYwHAZt0FY7tnkNIY/M1yaRcyHXvWMQigk6gZnQYp8WElpIrlzAlNx551DALoZJpGp0FKtRB/cZj9gqZizzoGAXQqRaPTICW8bM5fX/MFTcWedQwC6PBfjF6DlMpYlxaWfEbTsWcdgwA6nbrRa5BSxSUcvR/nb8lgOvasYxBAZ9A0ug1Sqvwg4eNvihc0FXvWMQhYh593ktFtkBIfYHoZPOYFTcWOdSwC6ESqmyPyEfOvTlr3JrxgqdizymbqEMHdZ/f7BinxZjvVmo6zKZ6xVORZxZg+ZAoFo9sgJX7SLnnw9zuD6dizjkEAnYp943bnICX8EvFHzzqPwXTsWccggE7D9U/bnYOUcCJ6SfxkfUFTsWcdgwA6nYbRbZASlsN1ll7PaDr2rGMQQGfyT73fOUiJf4VGTCmnM5qOPesYBKzTAmWj3yAlPhC3GPrzcUfQVOxYxyKATqRm9BukxAu+0GMa44ymY886BgF0Ek2j3yAl3ixkPhKv2/FpNBV71jEIoFMoGv0GKa2zIPvzGdkymI496xgE0OFn5d3jZsVlUq88v3HG0rFnlc3UIdKoGx0HKbXJT01tzilYOvKsYkwfMp2m0XGQEh9aZh193TpXg6nYs45BAJ1Jyeg5SKnzMm/wiOWMpmPPOgYB63T+vkbPQUp58vby+a0A1GA6dqxjEUAn0jB6DlLihQxuQxfHGU3HnnUMAuhkCkbPQUqVlzQppR7PaDr2rGMQQKfwCm6/c5BSCVjwHW+YoAbTsWcdgwA6lZrRc5BSzfz1tV/RdOxZxyCATqNh9BykhM13wWm0FzQVe9YxCKAzKBo9BymhqTnS8eYJGk3FnnUMAujws/LuiLxinlnJORzrHIWlYs8qm6mzyODPNHoOUio4vWT2mQRLR45VrOlDhlf/Rs9BSrXwgq+3Ws5gOvasYxBAJ6++wW7nICWeMR9Y+nrTADWYjj3rGATQKVSMnoOUSsdLNj3nC5qKPesYBNCp1I2eg5TWIjjEGs9oOvasYxBAp1Mweg5SYoYYej86Mie0R+xZxyCADj/vGD0HKeF0/Ynzks5oOvasYxBAZ+J9NLc7BylVtO1SPc7lksF07FnHIGCdyat/o+cgJX6QTLzjwtlMpY5tLADYJApGx0FK/NTUej1ezFFiknq2MQBgw3vq3WNmxWiCx1TnlUrFnlU2U4dI4Wfh/a5BSjjJZMR5XMF5DKQjzyrG9CFTaRj9BikVnloO6TjIyGA69qxjEEAH913b7xqkhFs4R958xzOajj3rGATQGVSMfoOUKu6CM2MYZzQde9YxCKAzqRv9BimVSSW1mPsZTceedQyCp1sKgea+36BKZfDUxjiuAFFoKvarYxJAJ1Ha9xtUiRfAuIx1XeqhBtOxZx2DADo8w32/QZV4hr3klvoFTcWedQwC6BTq+36DKjW8nUeOaZzRdOxZxyCADj/v7PsNqlRwskkJq6GpBtOxZx2DADp4B6mNzIrXO+Lw2qacsXTsWWUzdYgMqvt+gyrx5gmLvtwFS0eeVYzpQwb3w9zuHFQJFwiNftw6SYOp2LOOQcA6MfJPvd05qFJdJ5iEVM5oOnasYxFAJ1He9xtUiRfBodS8bmKiBtOxZx2DADqZ2r7foErYIsw51lt8nNEesWcdgwA6hea+36BKBQuZ1sZrtEfsWccggA7/1Pt+gypVjN/W/WjVWCr1bGMAwKZT2XcbVImfuFMvx5u4qcF07FnHIIDOoL7vNqgSmr2zj3UZjEZTsWcdgwA6k+a+36BKteAU4+M2QWowHXvWMQhYJ0VKm8fNEePyxLxuKKWpHqFjkd20oZGo7rsNqoR3FYq9zgPmGOmUeXYxAGCTqe/7DapUcJPeUHM9k+nYs45BAJ1KYd9vUCXcUrTP421P1GA69qxjEECn8TFjv2uQEh9YAlrfFzNJPdsYALDp1PbdBlXCRayVv0+9kKnYs45BAJ1Bw+g2SIkXMbzAO96kVg2mY886BgHr5EDR6DZIiXcHOZS0Lr1Tg+nYsY5FAJ2If//trkFKZeBimLreFFsNpmPPOgYBdPjYYXQbpMRP3G2msG4UqdFU7FnHIIBOpml0G6TEv0J9hhTiGU3HnnUMAuhUSpsj8hHzcgZvjB2uWCr2rLKZOkT4L0a3QUrrNm0pj6ywVORZxZg+ZNC33e8bpMQMIYfc6iuwR+xZxyCAzqRgdBuk1HGLqVbyBU3HnnUMAtYpPI7RbZAS7xBixHhnNB071rEIoBOpGt0GKRXcCGc8v2Qsg+nYs45BAJ1Ew+g2SAk3tAuht3lG07FnHYMAOjg/dr9zkFLFCzg1HOeaKDQVe9YxCKBTKRv9BinhJFreLaybIarBdOxZxyCADt6ndr9zkBKeukdb9yRTY6nUs40BAJtOw+g2SAmXzofwfJaSIlOxZx2DADr8rLN71KyYt9w18vTLGUvHnlU2U2cRtG6NjoOUcMCNI7QoWDpyrGJNHzKRmtFvkFKLlFKd8wKmY886BgF0Ek2j3yAl3Np5Zl4Fn9F07FnHIIBOoWT0G6SEk9FTzOuNqNRgOvasYxBAp1I1Og5SqljuhXjsyGUwHXvWMQig06gbHQcp4WUbXtasSxHPaI/Ys45BAJ1Bweg4SIkPMLzGa+GKpmLPOgYBdCav4Pb7Binxanj02OcFTceedQwC1mn8fY2Og5TwKxRKPM5TksF07FjHIoBOpGF0HKS0ThUo8+h0KjQVe9YxCKCT+afeyKy4rjcIrOtmWxpLxZ5VNlOHSKFsdBykhCVxxw1NFJaKPKsY04dMpWZ0HKSEJ6PKT0z5AqZizzoGAXQa3td4u3OQEt6OIOJCjwuaij3rGATQGbgz3XbnICXego8Wa+1nNB171jEIoDOpGB0HKRW8ETI/SPIrtEfsWccgYJ0eqBsdBykVvAHn4FXxBU3FjnUsAuhEXDm23TlICads1Xa80ccZ7RF71jEIoJPxzkrbnYOUOE5j9uMKBxlMx551DALoFKpGx0FKDTfC6a2nC5qKPesYBNCp1I2eg5T6wA1xjkuAZSyVerYxAGDTKeweNSteJ2qNsG5EpgbRsWeVzdQhMigbHQcp8TI4hLHex/bFShLPJsbk4TKpGv0GKeGlPj7UpnrhUrFnHYOAdUbgte1+1yAlXP7MH+WLmaSObSwA2CSKRrdBSjjdupdwHIBPZI/Ys45BAJ1Mxeg2SKniJji5t9doj9izjkEAnULN6DZIab2FWT9aMcpMUs82BgBsKk2j1yAlPvS2WEZJFzIVe9YxCKDTKRm9BinVwr9KzzeaUmaSerYxAGAzqBidBimhqTni81m1J7JH7FnHIIDOpG50GqRU14n75bhqSKGp2LOOQcA6M1LYHI2PGDfpwNflM5aOHavspg6RRMnoNEiJD7i9zNiSYOnIs4oxfcjgjn37PYOU8NbZo+fjfi8KTMWedQwC6BQaRqdBSmhdlucbKZ3MXlLPNgYAbBoFo88gJV7+xo7bPF/IVOxZxyCADs/D6DNICS/ThJpCeYX2iD3rGATQGby23e8ZpIRbl9TxykxSzzYGAGwmDaPLICW8ce0c5bgOT5Gp2LOOQfB0yyFS3HcZVKny1/P24Pn5StBU7FfHJIBOorLvMqgSL20qr4dDPaPp2LOOQQCdzJ+93TWoEv8K8WizX9B07FnHIIAOH1h3j5sVV5xeUsqsFywVe1bZTB0ijdK+z6BKeCPAkMt6meYFS0WeVYzpQ6ZT2XcaVIlnmGIpqwmlwVTsWccggM6gvu80qFIdlEdI/YqmYs86BgHrxEBh32tQpVb4o1JjP6Pp2LGORQCdiCvodvsGVWqTWq6j1Auaij3rGATQSVT3vQZVwmVCPc82zmg69qxjEEAn09h3G1SpJuKdZcrzgqZizzoGAXQqPyvvdw5Swta7zLlu8KzRVOxZxyCATuOV3H7nICVcCMNP4fmKpmLPOgYBdDq1fb9BlXByep2x1jOajj3rGATQGbQ9Iq94bRF6GOOMpWPPKpups0jC65v7nYOUmGD02nM5YT0ixyrW9CET+Uv3Owcp4W3Ea07H+kaBqdizjkEAHZ7Mvt+gSry9LGm2fkVTsWcdgwA6mea+46BKuGQqpvXebmezl9SzjQEAm4r7E2/3DVLCxS+l1XVpkCZTsWcdgwA6jeq+36BKPMOxvtMFTcWedQwC6HTq+46DKoFh4hLN12gvsWcdgwA6k4LRcZASLvNtta+bkWk0FXvWMQhYJwfKRsdBSo3Hz7m3fkbTsWMdiwA6karRcZASLvflnfixI1doKvasYxBAJ9HYPW5W3HCuX5vrcnmNpWLPKpupQ6TwD77fN0iJDyq83c7H89QxkI48qxjTh0ylbHQcpNQSldFGjWcwHXvWMQig06gZHQcp8Qz50JKPpbEMpmPPOgYBdDofM/b7BinhvJLc2ryYSerZxgCAzcR9Qrf7BinhjWtHPhY3SkxSzzYGANsU/lKj2yClxoeZ0tcZN2oslTq2sQBgE6kbvQYpsUKPtRwPnBPZI/asYxBAB5ca7vcMUuKlTM9pzCuaij3rGATQKZSMXoOUGi4OT7z+vaCp2LOOQQCdStXoNUip4YZtZRyHHYWmYs86BgF0Go3d8XjFI/DDZObjrAEZRMeeVTZTh8igYPQapNQy8QK4B4WlI88qxvQhM3ke+z2DlNqglFp8/k0SMBV71jEIWKcGXF+43TNIaZ0ym/q69YQaTMeOdSwC6EQaRq9BSryxzKnk4/V0GUzHnnUMAuigp73fNUiJ44qrg/IFTcWedQwC6BT+y37fICUcYMLI/WImqWcbAwA2lZ919vsGKXXcrDeG0s9kOvasYxBAp9E0eg1SapNiLzXFM5qOPesYBNDBc/J+3yAl3OqvzrLe2kOjqdizjkEAnUnF6DVIiR8kM4SWxwVNxZ51DALWaYH65nFzxBz1PstxJo7CUrFjld3UIZIoGL0GKfFHbfauqB6BZxFj6lDJPM5+zyAlNBRSnzG+wnrEnnUMAuiUdSOk3Z5BSj3iVvvtuCJGBtOxZx2DADqVhtFnkBIzjNbSuhHkGe0Re9YxCKDDP7zRaZASf33B66DpjKZjzzoGAXQGZaPXICV+kLRa29HBk8F07FnHIIAOrmzZ7xqk1PnwO0Nel81rNBV71jEIWKcHXr3tdw1S4o/4VygdVzYoNBU71rEIoJN4X7TfNUiJvz7UVI7XuU5oj9izjkEAnUzF6DZICb9CZTyfj63QVOxZxyCADjqVG5kV8y/RyO1CJaFnkc20oVFpGp0GKfGDIsWWjvO2joF05FnFmD5kOiWj0yAl/mi03o/XQxWYij3rGATQwfsf7ncNUooRvZfYjtPaZLRT7tnHQIDPxDuob/cNUoq8V5hhpONlP+Wmc88+BgL7jMjPwvudg5TiCBRxr7ZycdO5Yx8LAT6JstFvkFJseCeqVq5sKvasYxBAJ+Muddu9g5RiGfzd5jhuVqHUdP7wefN1wtTT/bt7pIRZxfvb+5tvP/z1p/cffnj7zf39b7eL4dNzMHmlUFNb99m+/XjLuDtROn3eS3T5zFfjvfpS9WnyteZ439/+G3p2zqkKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iago2MzA3CmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE3IDAgb2JqCjw8IC9MZW5ndGggMzk1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1SS27FQAjb5xRcoNLwm895UlXdvPtva0NSqSq8iTHGMH3KkLnlS10ScYXJt16uWzymfC5bWpl5iLuLjSU+ttyX7iG2XXQusTgdR/ILMp0qRKjNqtGh+EKWhQeQTvChC8J9Of7jL4DB17ANuOE9MkGwJOYpQsZuURmaEkERYeeRFaikUJ9Zwt9R7uv3MgVqb4ylC2Mc9Am0BUJtSMQC6kAAROyUVK2QjmckE78V3WdiHGDn0bIBrhlURJZ77MeIqc6ojLxExD5PTfoolkwtVsZuUxlf/JSM1Hx0BSqpNPKU8tBVs9ALWIl5EvY5/Ej459ZsIYY6btbyieUfM8UyEs5gSzlgoZfjR+DbWXURrh25uM50gR+V1nBMtOt+yPVP/nTbWs11vHIIokDlTUHwuw6uRrHExDI+nY0peqIssBqavEYzwWEQEdb3w8gDGv1yvBA0p2sitFgim7ViRI2KbHM9vQTWTO/FOdbDE8Js753WobIzMyohgtq6hmrrQHazvvNwtp8/M+iibQplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9MZW5ndGggMjQ5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nE1RSYoDMAy75xX6QCFek7ynQ5lD5//Xyg6FOQQJr5KTlphYCw8xhB8sPfiRIXM3/Rt+otm7WXqSydn/mOciU1H4UqguYkJdiBvPoRHwPaFrElmxvfE5LKOZc74HH4W4BDOhAWN9STK5qOaVIRNODHUcDlqkwrhrYsPiWtE8jdxu+0ZmZSaEDY9kQtwYgIgg6wKyGCyUNjYTMlnOA+0NyQ1aYNepG1GLgiuU1gl0olbEqszgs+bWdjdDLfLgqH3x+mhWl2CF0Uv1WHhfhT6YqZl27pJCeuFNOyLMHgqkMjstK7V7xOpugfo/y1Lw/cn3+B2vD838XJwKZW5kc3RyZWFtCmVuZG9iagoxOSAwIG9iago8PCAvTGVuZ3RoIDk0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWNwRHAIAgE/1RBCQoK2k8mk4f2/40QMnxg5w7uhAULtnlGHwWVJl4VWAdKY9xQj0C94XItydwFD3Anf9rQVJyW03dpkUlVKdykEnn/DmcmkKh50WOd9wtj+yM8CmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0Zvcm0gL0JCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9MZW5ndGggMzkKL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic4zI0MFMwNjVVyOUyNzYCs3LALCNzIyALJItgQWQzuNIAFfMKfAplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9MZW5ndGggMzIyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRu23FMAzsNQUXMCB+Jc3jIEiRt3+bO9qpSNO8H1VeMqVcLnXJKllh8qVDdYqmfJ5mpvwO9ZDjmB7ZIbpT1pZ7GBaWiXlKHbGaLPdwCza+AJoScwvx9wjwK4BRwESgbvH3D7pZEkAaFPwU6JqrllhiAg2Lha3ZFeJW3SlYuKv4diS5BwlyMVnoUw5Fiim3wHwZLNmRWpzrclkK/259AhphhTjss4tE4HnAA0wk/mSAbM8+W+zq6kU2doY46dCAi4CbzSQBQVM4qz64Yftqu+bnmSgnODnWr6Ixvg1O5ktS3le5x8+gQd74Mzxnd45QDppQCPTdAiCH3cBGhD61z8AuA7ZJu3djSvmcZCm+BDYK9qhTHcrwYuzMVm/Y/MfoymZRbJCV9dHpDsrcoBNiHm9koVuytvs3D7N9/wFfGXtkCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0xlbmd0aCA4MyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFjLsNwDAIRHumYAR+JvY+UZTC3r8NECVuuCfdPVwdCZkpbjPDQwaeDCyGXXGB9JYwC1xHUI6d7KNh1b7qBI31plLz7w+Unuys4obrAQJCGmYKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvTGVuZ3RoIDMyMCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDM0MCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UjluBDEM6/0KfSCAbtvv2SBIkfy/DanZFANxdFKUO1pUdsuHhVS17HT5tJXaEjfkd2WFxAnJqxLtUoZIqLxWIdXvmTKvtzVnBMhSpcLkpORxyYI/w6WnC8f5trGv5cgdjx5YFSOhRMAyxcToGpbO7rBmW36WacCPeIScK9Ytx1gFUhvdOO2K96F5LbIGiL2ZlooKHVaJFn5B8aBHjX32GFRYINHtHElwjIlQkYB2gdpIDDl7LHZRH/QzKDET6NobRdxBgSWSmDnFunT03/jQsaD+2Iw3vzoq6VtaWWPSPhvtlMYsMul6WPR089bHgws076L859UMEjRljZLGB63aOYaimVFWeLdDkw3NMcch8w6ewxkJSvo8FL+PJRMdlMjfDg2hf18eo4ycNt4C5qI/bRUHDuKzw165gRVKF2uS9wGpTOiB6f+v8bW+19cfHe2AxgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9MZW5ndGggMjUxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nC1RSXIDQQi7zyv0hGan32OXK4fk/9cIygcGDYtAdFrioIyfICxXvOWRq2jD3zMxgt8Fh34r121Y5EBUIEljUDWhdvF69B7YcZgJzJPWsAxmrA/8jCnc6MXhMRlnt9dl1BDsXa89mUHJrFzEJRMXTNVhI2cOP5kyLrRzPTcg50ZYl2GQblYaMxKONIVIIYWqm6TOBEESjK5GjTZyFPulL490hlWNqDHscy1tX89NOGvQ7Fis8uSUHl1xLicXL6wc9PU2AxdRaazyQEjA/W4P9XOyk994S+fOFtPje83J8sJUYMWb125ANtXi37yI4/uMr+fn+fwDX2BbiAplbmRzdHJlYW0KZW5kb2JqCjI2IDAgb2JqCjw8IC9MZW5ndGggMjE1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQk1RUURWK0RlamFWdVNhbnMgL0ZpcnN0Q2hhciAwIC9MYXN0Q2hhciAyNTUKL0ZvbnREZXNjcmlwdG9yIDE0IDAgUiAvU3VidHlwZSAvVHlwZTMgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0KL0NoYXJQcm9jcyAxNiAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNvZGluZwovRGlmZmVyZW5jZXMgWyA0OCAvemVybyAvb25lIC90d28gL3RocmVlIC9mb3VyIC9maXZlIC9zaXggNTYgL2VpZ2h0IC9uaW5lIF0KPj4KL1dpZHRocyAxMyAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRm9udE5hbWUgL0JNUVFEVitEZWphVnVTYW5zIC9GbGFncyAzMgovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Bc2NlbnQgOTI5IC9EZXNjZW50IC0yMzYgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDAgL1N0ZW1WIDAgL01heFdpZHRoIDEzNDIgPj4KZW5kb2JqCjEzIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9laWdodCAxNyAwIFIgL2ZpdmUgMTggMCBSIC9mb3VyIDE5IDAgUiAvbmluZSAyMSAwIFIgL29uZSAyMiAwIFIKL3NpeCAyMyAwIFIgL3RocmVlIDI0IDAgUiAvdHdvIDI1IDAgUiAvemVybyAyNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE1IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMCAvY2EgMSA+PgovQTIgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PgovQTMgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMC41ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1EZWphVnVTYW5zLW1pbnVzIDIwIDAgUiA+PgplbmRvYmoKMiAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0tpZHMgWyAxMSAwIFIgXSAvQ291bnQgMSA+PgplbmRvYmoKMjcgMCBvYmoKPDwgL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuOS4yLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuOS4yKSAvQ3JlYXRpb25EYXRlIChEOjIwMjUwNDAzMTkzMjI4WikKPj4KZW5kb2JqCnhyZWYKMCAyOAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAxMTg4NiAwMDAwMCBuIAowMDAwMDExNjIzIDAwMDAwIG4gCjAwMDAwMTE2NTUgMDAwMDAgbiAKMDAwMDAxMTc5NSAwMDAwMCBuIAowMDAwMDExODE2IDAwMDAwIG4gCjAwMDAwMTE4MzcgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzQwIDAwMDAwIG4gCjAwMDAwMDY3NDMgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDA2NzIyIDAwMDAwIG4gCjAwMDAwMTA0MzIgMDAwMDAgbiAKMDAwMDAxMDIyNSAwMDAwMCBuIAowMDAwMDA5ODU5IDAwMDAwIG4gCjAwMDAwMTE0ODUgMDAwMDAgbiAKMDAwMDAwNjc2MyAwMDAwMCBuIAowMDAwMDA3MjMxIDAwMDAwIG4gCjAwMDAwMDc1NTMgMDAwMDAgbiAKMDAwMDAwNzcxOSAwMDAwMCBuIAowMDAwMDA3ODkxIDAwMDAwIG4gCjAwMDAwMDgyODYgMDAwMDAgbiAKMDAwMDAwODQ0MSAwMDAwMCBuIAowMDAwMDA4ODM0IDAwMDAwIG4gCjAwMDAwMDkyNDcgMDAwMDAgbiAKMDAwMDAwOTU3MSAwMDAwMCBuIAowMDAwMDExOTQ2IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgMjggL1Jvb3QgMSAwIFIgL0luZm8gMjcgMCBSID4+CnN0YXJ0eHJlZgoxMjA5NwolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:32:28.090269\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["plt.bar(np.arange(mean_out.shape[0]), mean_out, **plot_args)\n", "plt.yscale(\"log\")\n", "plt.xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "14b35777", "metadata": {"papermill": {"duration": 0.023412, "end_time": "2025-04-03T19:32:28.537627", "exception": false, "start_time": "2025-04-03T19:32:28.514215", "status": "completed"}, "tags": []}, "source": ["This distribution is very close to the actual dataset distribution.\n", "This is in general a good sign, but we can see a slightly smoother histogram than above.\n", "\n", "Finally, to take a closer look at learned value relations, we can\n", "visualize the distribution for individual pixel predictions to get a\n", "better intuition. For this, we pick 4 random images and pixels, and\n", "visualize their distribution below:"]}, {"cell_type": "code", "execution_count": 27, "id": "e4a2a68c", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:32:28.585547Z", "iopub.status.busy": "2025-04-03T19:32:28.585363Z", "iopub.status.idle": "2025-04-03T19:32:31.258105Z", "shell.execute_reply": "2025-04-03T19:32:31.257122Z"}, "papermill": {"duration": 2.699713, "end_time": "2025-04-03T19:32:31.260770", "exception": false, "start_time": "2025-04-03T19:32:28.561057", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNjEyLjkgMzY3LjE4MDYyNSBdIC9Db250ZW50cyA5IDAgUiAvQW5ub3RzIDEwIDAgUiA+PgplbmRvYmoKOSAwIG9iago8PCAvTGVuZ3RoIDEyIDAgUiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzc3U2TLLd1Lup5/YoaSgOCwML3ULq0GaGZpB2+A4cHCprSEaO3HDwM+f79u15kdWIBWCnR5OY+JyGF7N3v6s7ufKoKBWQhgS+/+va///rNt3/4+rfP/+ePjy/7V9/88HDP7/h/f3na53f8v//v6Z5f8//+8rD81cdHcmQq/+vt9S+fsnHFJooc2fHL//V4/Pnx5W/4h394WlNdTiHbWMryRajW1WRzef5v/N6vh294/KPvfjxCNvlJtph8/NqPD2+d8T75FGT+JnMfq6nveTvCkLQ//PvnfGiK3iT/+m94uugMPf/3t8//9/m355e/IZyme/6O//cd/6/BTbT8O1KulMP4B/d4+Csef3z8/vn9+5GtcZEflfeDty+/fqWP7/kRs88vLJdiMq4djHKqT1cZjBwf75uPj99+eHz5r+7p3PPDn9vD+eE/H//+/JX99fM/nh9+9/iXD4/ft9/4yc/buWw8+WppPHGRf4Izd86+juZz+lFnnsIvf+o5mJpK9NOZn/GnOPEUTMDBbMiRftSZOyq/+KmTJ5Ocd7GO5y7yT3DyRGQcjlZiDuHHnXylX/7k+e+gmHOZGieRf4qTz9VkHC2l7H/cc55i+sVP/jggZZOs9XZuk3v+U087899D5A0FW/nIfNzXCVt8cy4Q4LPtp/8r9+vnh+8eydTKL5PoYqIDBI0fF5w3kf+TUmCAZzDk+MD5PAhOpx8lmFJLzSHZGl5Hia3AoN/zH2Btzk+Av/7pgnHcJgUcMBhfakP+5uOTD/3FV99+96d/+/sf//S3H774+Ne//f2H51f/9fz9L/u4RDKZX/+uTo9Lz3/W4xKK8dGH4GtyWTwuNpX8mR8W+vEPC7+S2t/3f+5hydkUPpT308PS85/8/kB4XLLnN4lYW9tDn+VxqTfi95Z7sz6XkEd+kf88/lpMxGEc95vTZ+FPd+KnYmy0/B426p/xz8LnXqFJ1bJm8MV9Fnx/J3w+iktUeAAy6vf8J/MX8PNbQrE+1ErJx5/Hz8NL64stkdGv9e27/k9h88cJ8ivWeu5Nxcwj0HAMP4n/fIun0jEADW28amIT6N//vPz+xx++/pHf2ca23EEkKqz2/KJYHiulHJkMo3CWpprIj4U3UWgPHh+pIu4HGmIe5/721kAXDrhMYbiH6Wnxec+p8Dg5Vj7nQW2I78+jKjSd6p11ReF5FShxyxs8v/MNajLdgUdjYB8yxbvMjdnscxYwjvMxUKwjm4zvD6Q7MJA32VNw6+vrLFB1poRQ3Owm4vsD6Q4MFPjv9/yWNvu851SDiSl4qpOaiO/PoyqwTjQ8oCk2LzxnwVs+GvcAfJRqQ3p/Hp2BfZIJPkQbFp+zQMXzS6pyJ3RgG+L7A+kODJS5jxrJrq+vs0A141qLy3F0k/H9gXQHBuJ+t4+l1gXoLFDh9/LsfMqjm4zvD6Q7MFA1fI6xpgXoLHCHmbvKwXK7M7qJ+P5AusPHR+bhmc9UlyFGL1B2Bp84+Dq6yfj2QBcODYiHnrUubVAv8NDZkI0p0+gm4x2ANAcG4p6eL6kso4xewHCLz7MmN7mJ+P5AugMDEXf7qi/LMKMXqPDQooYY4+Qm4vsD6Q4MxH9/sLasbdBZ4GGpoRQT0eA2xPcH0h0YiEcLATMhZp/33Ntsgo02T2g9vT+OasA2kX/MhbwMMnqBG2PuF2bXLnCexxnS+/PoDOyTjA+EjvDscxYoF+Mq1XY1Q7DJ+P5AugMD8bt0IHxYNgOdBQy3eGBhsx/dZHx/IN2BgfhpEHxMyyCjF9DVyej0xMlNxPcH0h0YqBobAqW1CToLVCsPLPh3TG4yvj+Q7tCAagg1LRc6eoGbGVNTSu2KmHQT8Q5AmsPHR7GmhJjSMsjoBUrB1By9ndxkfHugCwcGciaHxCPzBegsMAH3dlytA5tM78+jKzAPmTZzdrnM0QvULs4Tv7cPakN8fyDdgYE89/oyv04WoLOAS4bW8S8Z2UR6fx5dgXkC/xy/Yhad9xwzEKO1NtfRTMb351EVWCdyj6+6sDY+Z4H4n9xX5tfUoCbT+/PoDOyTuMNXtafPWfAe82n4twxqMr0/j67APNm4aGNYBhi9gM+TSyo10aA2xPcH0h0YCLO/HC29wzOnTAYXmo+2uavJ+P48qkLTqdFhHtTK8yp4x6eYKMdBbUh34NEY2KeaEin59elzFngkYWJNrrqBbYjvD6Q7fHxwI5IxsXhpf3oBwyxKFm3y4Cbj2wNdODCQMykGS8sFjl7AtbBonWsfVQg3Gd8fSHdgIG5nucc3T1B8EwVcyciBSkijm4zvD6Q7MJA3gSvrW1gv4ONSF1wsNLrJ+P5AugMDBeNj4nHUAnQWMCM5xtcT6DyOTO/PoyswD7ezMZV1pmYvoC0OBXNaBrUhvj+Q7sBAmK+b4zpVsxfajJZqbbuDRrjJ+P5AugMDZe71FXLLBY5e8Daa6hNm/0i3Ib4/kO7QgGos6zSOM6cccPuCax939cMM8Q48igLrFO7z4fsWnrPQ5qxafsrUQU2m9+fRGdiHuzGp3cw6+5wFPFFidbjxZ2CT8f2BdIePD4cpu4n/7zLKEBUei5qYa3aznIhvT3QlASPuyySX1umaooIbOF2k2uYdDnYi3wBJpwASmZDI16UpEhV8qpNcqM6PeEO+AZJOASRv+I/HLdAL0lnxLsElZj/hyXwDJJ0CSMFQ8nl9zxcV3OoUrY3VjXhDvgGSTgGkaFwKoSht0lmhFHlA7zEem/BEvgGSTgGkZGyKriwXQESFuEsdnSs048l8AySd4kCqiX9uGYKICrqLtQbvZ7seb0GkQYAom5K4pLRIZ4UKlrjgf4WJTuYbIOkUQOKTSZnW6a6iQoWM9y7E2a7HGxDpECCq3C3kHvMyHBGVdunVl9TuEZN0Mt8ASafAqmP8a1NJeW2PegWf0FcWCXXCk/n9kS4ogOS4W6h8uNgL3mOE763PI92Qb0CkQkCIm5Rs7Tq7U1Tw/oUVX4If6IZ4AyJdAkbeULZ5neApKq1ttpmfNpOdzDdA0imAxE+G7HgQtiKdFUrcOFse54cJT+YbIOkUQIrGZsLV1gXprOC+sVJ4/JonPJlvgKRTHEg181vUOhzpFVxUs6WWtuygxJP5FkgaBZD4yZB9jEqbdFZwg0INydc44cl8AySd4liZNedAcR2Q9Irnf5ZMxyV+cawh3wBJpwBS4S9CXed/igrW7ws5upRGvCHfAEmnAFI1MccUlDbprFDBzdEp+BlP5hsg6RSMRPyTOfmwDkl6BXOqU3a4g3zEk/n9kS4ogOS4a5jXVVF6wfNALTvvj6FtP9KQb0CkQkCIuF+YeRC/Ep0VitzwuBDb59b9UEO8AZEuASOPJYKDXy+R9ApuH8+pVJdGuyHfAEmnAFLgbmF169RQUcHSqBTta0gi8US+AZJOcSDVXJXL2mcBNwJTTan4kW7ItyBSICAUedzFz4T1EkmvYB6EDdnXNNLJeAMiXQJGyWDF8nWKqKjw7zGeqEaa7GS+AZJOAST+ori6ThMVFfJtGwTrZjyZb4CkUwCpmFgoufUSSa9g9aqQ3SR3hhvw6AjgqZgQ6906EOkVLE/JZ+0dTWwy3wBJp2Akb40vwa7TRUWFsJye41Y6T3gyvz/SBQWQ+E/GUu5ra9QruD84t41JJjyZb4CkUwCJjCsx2HUs0iusYHK1x1W2figZb0CkQ4DIG1uSWyePigo+3bc+lraanjjWkG+ApFMcSLXw27fSIp0VzOXnrmN1C57It0DSKIAUuEuofOr/nuMTkFw95otOcCLfAEhjAA+3J4X7Ous4pFfQCfI1p2M42+FEugGQDgGixL3BUqryHDoreO+KkftDYaKT+QZIOgWQ+Dw5r+tlkV7hrpCx/AZfwog35Bsg6RRAKiZUS8rE0V45XlaF2hKE4lhDvgGSTgEknKdFV3lBOiuULXcVY3299Xc8mW+ApFMwUsBhXCrrpZFeoYS5EEQpTHgyvz/SBQWQHJaS8crU0V7BfGxfQyxpxhP5Bkg6BZDIYKvAvLZJveItVgB9rVstjjXkGyDpFAdSrT7n9QJJr2DmWgo5tP3QxLGGfAskjQJI3pTKP7sOR3oFk0RLiSVPdiLegEiHAFHgMUV061vbWYBETjzUV+DOfAMiFQJCkb+XW9312kivoCuULEXrJzoRb0CkS8AoYQ2emNYBSa9gIeuUrDve/PuxhnwDJJ0CSJk7hZmUqaO9gpVAqVIubsQb8g2QdAogFQzh160+egE39bmS6bg8IulEvgGRCgGhiqOkqDTZZwUzjUtIrwuRgk7EGxDpEmwULXcIq1emjfYKGud8vLpGO5nfH+mCAkjca7bWxrU96hVg1FCP5eMnvDPfAEmnABL3mq3NYR2M9AoWEMsxH91sYSfiDYh0iIOIvzcok0Z7pT1fSrKVJjqZb4GkUQCJ+82WuLQinRV8cG0D5hxPeDLfAEmnABJ3nC2VsF4e6RUsp8qjkGOf6QFP5Bsg6RRA4p6z9VGZN9or3FXEnUb5mKcl8US+AZJOASTuOdtAXmmTzgq1rWCTOz7RFngy3wBJpwASdhYK1a+XR3oFs/qJavI04cl8AySdAkjcebYxKUbvhTZB1BbXFggf6ES+AZEKASHuOlt+IqwDkl5Bw+NL8sfk2k4n4w2IdAk2wq7llhvh9QJJr+DeEEt8/DTZyfz+SBcUQOJujs2Z1gFJr7TNdngIe3QkBZ7MN0DSKQ6kavm9fB2Q9ApOM+fUFoOUdiLegkiDABFxp7A65aJ2r3hKhgcf1U50Q74Bkk4BJH4u2IqFaBaks4Ld9bIL2ccRb8g3QNIpgBQM5qYpk0d7BZ+DYE224xMkgSfzDZB0CiBFLI1FVmmRzgoFHufXFI4ZtgJP5hsg6RRAwnRrV+06IOkVLKLBB6OcJzyZb4CkUwApG+8oKdNHewUYpQaitOKd+QZIOgWQiiHcHqu0SWcFl9UKuXB8RiLxRL4Bkk4BpGqc87WuF0l65VhUPPm84Il8AySdgpGwy7kLSZlG2iu41RgLQ9s44cn8/kgXFAdSddHPe8q/yQr2N8jkU04j3pBvgaRRAMmZwoOxtQdwFtraRz46SiPdkG9ApEJAiLjHnLIyh7RX0PAU7/JxIanTyXgDIl0CRnwGLod5b/k3WcFNEIFfV8c6NtJO5Bsg6RRACtwtLK6sF0l6BW/zMcRKNOOJfAMknQJIkbuFpShTSHsFO6vFZEv0I96Qb4CkUwApcbewRmUKaa9gdzXi30MT3pBvgKRTAInPkyzl9TJJr2BVf489+uKEJ/MNkHQKIOHOPVvzOiTpFexxFP2J1PFkvgGSTgEkXMXnAes6JOkVwp7O1kc748l8AySd4kDidyeflDbprPBIll9WLhwrRwo8mW+BpFEwEjZGJ2+VzyR7hfB5tufjuwlP5vdHuqAAUht6KW9uZwGfq5VKOdNEJ/MNiFQICJFJFMK8Ef2brOBKf4pkj+1HOp2MNyDSJWDEfUGKLq5Dkl6hWE2wthwX24SdzDdA0imAFEygWJSJpL2CyZDJxnDMbRN4Mt8ASacAUjSekvZEei/galEIIR03kfQjDfkGRCoEhBL3CTOF9RJJr+B22mDD6yOS81BDvAGRLgGjzF3CXJVppL3CnWlTiz/2MxbHGvINkHQKIBXuEpY071H/Jivt4yI+5dcT6TzWkG+ApFMcSJXfxv16iaRX2r692VnvRrwh3wJJowAS9wS9tco00l7BWuyeG6FjS4R+rCHfAEmnYCTspN72fliQegVrsXvn/DGNVODJ/P5IFxRAcnwGPDZdL5H0CpaJcj7UHGY8kW+ApFMAiXApCBcVF6Szgk9Dsq3puI4k8US+AZJOASRvgqeiTCXtFVx5zHinjxOezDdA0imAFIz3PpLSJp0V3DFSbXTHuE3gyXwDJJ0CSNGQD8p027OANUcqtkCmiU7mGxCpEBBKreFVZpL2Clpnz2/0x6YRnU7GGxDpEjDKWNswzbvZv8kK2p3i+AmTJzuZb4CkUxxIlf96p7zWzgow+Jnjj/0QR7wz3wJJowAS5jlkq0wj6RXcd+RTsilOeDLfAEmnABLmy+Q8727/JivYrZZ/lZ3tRLwBkQ7x8UHYbx0r+C4XSUSlDTxyTMeN7YJO5rdHuqIAEg/hfXXrNFJRwWW1QuhgT3gy3wBJpwAScaew5nmD+zdZwWchtvLwY8aT+QZIOgWQvPG45XG5SCIqWHXUWpzzjCfyDZB0CiAFQ5ipvgxIRIUidtDKx9KsEk/mGyDpFECKxgW3LkbaC8cmY5HaZyGSTuYbEKkQEErG4mOh5QqJqPCvweK1Nk90Mt6ASJc4jCqG8suARFQ83sySC7aOdkO+BZJGAaRsSsDqvSvSWcHAo6YS2qJj4lhDvgGSTgEkPpkQ1hlbvUCRTKTjFn8pJ+INgFQG+FSTQuSO8wp0VignU2KIOYxwMt6ASJdgI+y1Hvg718FIr2DmWoi2xDzZyfz+SBcUQMIMh4QZVwvSWcEzJkZXY5jwZL4Bkk4BJOIOYQ5p7R71ShuZhRzb2jUDnsg3QNIpgOS5S1jcOoFUVDx5E3jM3y7NimMN+QZIOgWQAvcJS5m3t3+TFXyeRolfVRPekG+ApFMAKXKnsMa0XiDpFcp8Xj5Ru8df4sl8AySd4kCq0dI6h1RUMKXGlRIXPJlvgaRRAAldHVvn7e3fZKXd408ptslZE96Zb4CkUwCp3U+c4nqBpFewXFTkLja5CU/mGyDpFEAqmHbt15HtWcAbvcc6CGGik/kGRCoEhCp3Br2d97Z/kxXcc5wSHzyOdDLegEiXYCPstc7ji7A+jXoFk9dzdTlNdCK+P9EFBIic8TGEdQqpqOD5QpiiXSc6mW+ApFMACXuHR7dOIRUVDGFrCOX4CEniiXwDJJ0CSN64GItfr5H0Ci7wF6rHoE3YiXgDIh0CRMHYmDBRdiE6K+gHWbLJ+YlO5hsg6RQHUo2Z1gmkotKmHOdU2h2QEk/mWyBpFECK3CXMlZQW6azg2mxOLiSa8GS+AZJOAaTEncLCg4oV6ay0TRCco5gmPJlvgKRTAIm/iNWvE0hFBXto55J9TROezDdA0imAVLBShp23t3+TFezt58rZSxJ4It8ASacAUjW4UO3WEUmvEJ93renYsl7iyXwDJJ2Ckfxx6/46iVRUsOYIZmjbOOHJ/P5IFxRA4j85kZs3uX+TFdy8HgJWIJ3wZL4Bkk4BJDIu0brDZi94i5vXU223HIsjDfkGRCoEhLyxycd1BqmoUMZ+YyX4ONAN8QZEusRhVFOgeYv7N1nBhmN8uum4+C/sZL4FkkYBpGBKwj7HK9JZwRg28ZDfuwlP5hsg6RRAwi6ZMSpTSHulzfAvKbWtfSWezDdA0imAlExKiZQppL2CbVlswf61o52INyDSIUCUuVOYal0vkvQKZdzpx2cXJzqZb4CkUwCpcKcwp6q8+58V79ot2un11nYea8g3QNIpgFS5U1i8Moe0V/jtyxQfQowj3pBvgKRTMBL2XefvLmuL1CvoFxWX4vFxpMCT+f2RLiiA5LhfWHNZByS9gvcwwhb2YcKT+QZIOgWQyNisfSB5FnBRjcchqd0FKelkvgGRCnEI1excVoleFYw8MoV6dJE6nYy3INIkYORNya7kdUDSK9hwLNZyrIIk7WS+AZJOASR+MmSKyjTSXiE+r8rvZe3WdYkn8w2QdAogRf5C2a2tFzB7jd/h6fisTdDJfAMiFQJCyURcvVeIzgrWG+eYaJQT6QZAugOEssGUWWUCaa9gGQ3cU1NokpP5Bkg6BZAK7iT28yb3b7KCQZq11lGc8GS+AZJOASTu4uRk43p5pFcwdy3xi6rOeDLfAEmnYCTsu575y3Uw0it48jiydNzRJvBkfn+kCwogOe4T5jBvdP8mK1hHO3BH6HjzF3gy3wBJpziQeGjBf7mG9Krg7SxH1/Y/kHYi3oJIgwBRWwCyKAO2XmmfppH3bW0/SSfzDZB0CiBhJ9EalSmkvYIpfvyez23RhCfzDZB0CiAFk4rlMcaKdFYoEQ9grYsLnsg3QNIpgBRNLFa5rH0WcIMfv77sMRwRdDLfgEiFgFAyuPVKmUHaK2h4iN/AFjoRb0CkS8AoG88/5JX26KxgaR/Lw9ejry3sZL4Bkk4BpMJPBm/9OiDpFQz3qRT/Qup4Mt8ASacAUjWu+KxMIe0VDPMTj2VTnPBkvgGSTsFI/A5uSwjKRaRewbV+W3woYcKT+f2RLigOpFqio/WtrVdwN22M2R0fRgo8mW+BpFEAyZlSYlGmkPYKVhtNoaQy48l8AySdAkjcFywpzlvdv8lKuxjiInaLGvFkvgGSTgEkPoOSya0XSXrFu2Ry8DmkEW/IN0DSKYAUuGeYqzKFtFcwns011WOmTT/WkG+ApFMAKRrcxj9vdf8mK208m2w6lrWReCLfAEmnAFLC/cTerpdJegUzIYOtRHXCk/kGSDoFkLBLprXKNNJewT+LI39cuJV4It8ASacAUjGu2qxMI+0VzGDjXxIrTXgy3wBJpwBSNbY6pSv5ytEtckSviyQCTuYbAGkMB0+t2CtTA3pVsNUYfskx7/+Ek+kWQBoEE2EFqIpNjhaiXsGHRTGRP9azE3Qyvz/SBQWQ0MXxoa5tUa+0zQ+sL8fyUQJP5hsg6RRAIv5mdZ3WXsHErBgphjDhyXwDJJ0CSJhzhf7OinRWDowQotPwXvkGSDoFkIIJNcaitElnBWPZGKulPOHJfAMknQJI0fiq3tLWK7jyyCccjuWjBJ7MN0DSKYCU+DCYNrsinRWMzVzGtmOjnYg3INIhQJS5U5jTuiWyqOA6bcV9bHWmE/kGSDoFkAp3C4vP6wWSXml31sTkj0XIBZ7MN0DSKQ6kWqtyA+lZ4OMzBTaLmuhkvgWRAgEhbCFe8bnrQnRWcJaRKMx0Mt6ASJdgI+y4bm1I68WRXsFQn9/vnZ/tZH5/pAsKIHHP2TqnTCLtFUyuTZVPNk54Mt8ASacAEvecrVO62mcBdx67yN3FMNHJfAMiFQJC3KZYinFtsnsFdx6FHF5TkTqdjDcg0iVgxL1m60mZQNormDaSHYXjNiRhJ/MNkHQKIHG3mQ+iTCDtFdwvGqPLccaT+QZIOgWQuN/MLXBYL5D0Cp4x1dbwapE6nsw3QNIpgMT9Zht9UFrts4J57NHGfKxmK/BkvgGSTgEk7jnbZJUppL2Cz2V9wf39E57MN0DSKQ6kik9clTbprOASJEayx91IAk/mWyBpFEDivrPNwa8jkl5pSyDYmo/FWvuxhvxE+vI3hBOn5++ezIFzcs+vn19+9e1///Wbb//w9W+f3/zwmCE/HonH5YQjeXs8/vjw1mGzXNgO3yzy+UfWI+sH6d8sjvAPj/zlb1x7QPm8rMFaxwf48kV4t28PKh7S59fDNz3+0U88HjwIxpJB/N/xPJKN46m9vaLhDxU/PeT88Py5PTz9G+Zf8Y+fze1BdXhQn9893UN7VPmnsGFWxUf5wyPQ8xlVPGEs97iffzkO314I/OXXr/Tx/cOx4xeWSx73CeBoNgUbnw7bchPuiPvm4+O3Hx5f/qt78i/68OeH5Z/48J+Pf3/+yv76+R/PD797/MuHx+9/aYRADjNjIhbqHZ7gPf8ECMGF42hYTzL/KIQUPqcCj8m9J5eWl/mZfwqFhE1s+WjHAhs/RsFR+YwM0eNehJRrHBlE/gkYIiUeivHRQvU/UqHS51So/GQlF7C856DQ80+hwCMK145mfaYfxcCd6s/IIBt2b3HmSbTrZ/KT20XnTBR/OT8pKBHl0D7mORD41PlPwLoGDNBFfuV+/fzw3YOfk5U7htHFRIcRWk4uOB7Q8n8SN7hcCYacrb4fBO+i/SjBlFoq/1rcaXIchVqBjb/H77eZ+ytP+/5PbI9jyQcckA+da/v7vvn45EN/8dW33/3p3/7+xz/97YcvPv71b3//4fnVfz1//3kfqoReDI/BxGPVo5/+YKXhwcL6STVEPvkY6bM8WPXOj0nFPGasqXw+JGfyqR4RHjBxu1oK7rv/PI9IuvEj4olMdCWKFk1En+gxwXjDxepKTdn7z/KY+Ds/JhEXsW3J/SE5k5/8iOBBFY9IKNj3JVTuiVH4eY8ILlD4Ykvkx+H6AbHvD8jPk8TY/PGjR9D/bGz+GL7/fzY25z/PxErQ+IJheATu+L0zv4ZN3KMjrPwgK2+y0h5SPlhb7EIca8jPsfkOXDoKuDB1g9+V88p1VvBGG+NrvWtxrCHfiktHARdmcYSEi/kz11kBi+cxm6L4Hm+FpZMAC7M50GqtWGelLV3o/bGXmkSU+VZcOgq4MLMj2aC0XGcFi6yFaG37GFoyynwrLh3l4Krcp8GF/pXrVcGM4lo8d4kmRplvxqWhgAvzPXLAJf+Z66y0aaAUkq8To8y34tJRmIufILliR7VZ6yy06bPk2ocd0lDEO1HpIJDChz6l4MOQmeqs4HMRX7L1fiSU8VZYugm0eChQ0WavWmcFE7IsP4fax9ZSUeZbceko4MKsEB4u0sp1VtraNZELdWKU+VZcOgq4MD+Eh4Cr1nsBVwdKrtR2HRVHGvKtsFQSWGGaiEu0jnx6xVtsaB/fX4knooy3wtJNoIX5IvxEUd4Qzwo+/8dmm7aOikO+FZeOAq52YQaL1y5cZwUfm0dbgg8j45BvxaWjgAtzSPidbh369AquVnOHgSiNjEO+FZeOcnBVG4Jbhz69gne/SNzLoolR5ptxaSjgwqSc6LTO/FnBfPhoY223CQ6MIt+KS0dhrsLdc/45t1626ZU2NY6qf70xdkaZ78R1gQIu7p/bFO06/ukV3I8aLLfoYWKU+VZcOgq4uH9uM9m17eoV9K9yzlgnf2Ac8q24dBRweX6C5GrXLn2veOwpXGpbUEAqingrLJ0EWIG7nEV5Zr1yzHq2LtQcR8Ah34pKAwFU5M5m5eOsVGcFU2NqzqFESTikW1HpJMBK3NWsta6XbHqlTYHl51JbPk8iynwrLh0FXJlfSzbVdejTK7hrrHpyLo6MQ74Vl45ycFXnfFmHPr3ibeSmikL1I+OQb8aloYCrYCEvzMZYuM4KVdxAxk+iNDIO+VZcOgq4Km6lwzcuXGcFfXb+MiWaGGW+FZeOwlyVv9n5UNahT6+gSS8lh+MzxX6sId+J6wIFXLiuF1xe265eaTd2YoSTRsYh34pLRwEXD/ZcKHntoPYKVSzqQOkY+ghGmW/FpaOAyxvvYszr0KdX+PfgbqpEZWYU+VZcOgq4MP0vUVbarrPSLv65mq2bGGW+FZeOAq5onEs1rUOgXsF6NHywfFwUlIwi34pLRwEXRsg5pXUQ1CtUK/8uV93MKPOtuHSUg6u6wn+/xvWqeNzcVpM/rkb0Yw35ZlwaCriwPG9VrgmeBe8sd7RCjmFClPlWWCoJrPhcuKlWZuz2Cl5xfLRS64A4xFth6SbQqiaRxR7ri9ZZIW7ZK+4LjbOiyLfi0lE+PoK1mLzm4nLxRlQweTJyx73GiVHmG3FdoYDLmUCurFN2RQWfT4ecfJvJLBllvhWXjgIuMp4orlN2RcXzGWMCl6sj45BvxaWjgMsbwoyQ5eKNqKCNKjV470bGId+KS0cBF25l5l76MgASFbzoUgo5xJlR5Ftx6SjgisZiTvcyABKV9vGFjdmmkXHIt+LSUQ4uLP/nlbbrrGA7bkvVlzAyDvlmXBoKuJIplDCze+E6K7jATClSmBllvhWXjgKubDKl9VJXL+D9rzgX2u4c4khDvhWWSgKrwj3OHEhpuM4K+qKED6zTgDjEW2HpJtCqbbofLQMgUcGSeda69Grmu6LMt+LSUZjL8U8S/9g6AOoVfOyao80+jIxDvhPXBQq4uB9AVRkungV0RKMj1+6pFkca8q2wVBJYEdoecsuFG1HBFZoUayqTYU+3otJFYOWN88fuRLPVWcFV+BCDL2UylPlWXDoKuIKx3iWnvA7PCvc/+V2w1lInRplvxaWjHFzVt93gFK5XBfcmVmaxdWKU+WZcGgq4oine2/XyvKhQ5T574De/MDHKfCsuHQVcyWTvs1XarrOCEaEPZGOYGUW+FZeOAi7+wodg18s2vYKJ39HZTH5kHPKtuHQUcBUTPRZdWbnOCj5BzBRqdROjzLfi0lHAVU3wMa+Td0XF46K8z+/vjJ1R5ltx6ShYb9ViJd+w3kEsKmBxJVCsK+OZ78R1gXIsT0vabYu9wO+BxidbPM2IIt8KSyWBFXGPM5d15q6o4NN857COzYgo462wdBNoee5wlljWdqtXcF81VvVrH+eLYw35Vlw6ysFV+RhlHQD1yrF7huOn1MQo8824NBRwBe5w1rpO3BUV7FpTIpUUZkaRb8Wlo4ArmhxsykrTdVbQcy++1ONdUTDKfCsuHQVcyaTAbdE6AOoVfILosis1jIxDvhWXjgKujKX57TpxV1QwQbeQ9amOjEO+FZeOAq7CP8nnrbRdZwUXl23CQsQTo8y34tJRwMX9zOBDWi/e9AquL/MBPS2MIt+KS0dhLs+HwcIP6wCoV7CebwmOaFSU8U5YFyTA4rY6hJLWlqtXsBlnyfyaiyPikG/FpaOAi88txBjXSze90jqjlQc9cWKU+VZcOsrBVQOfyzoE6hX0rrJ/rdgsGWW+GZeGAi7PrU/SPovtFexDReTcMWIUjDLfiktHAVfgLmde72fpBW8Dx8n50VDGW1GpIJCK3NssPqyjn15BT8H7GNqqUv1QQ7wVlm4CrcSdTSwetWqdFdyITtaWXGdFkW/FpaOAK/NP1hyUVuustEFOSo7KxCjzrbh0FHAV46MNfh389EobQsdIZWIc8q24dBRwVUPROWXSbq+gI5pDjXliHPKtuHQU5grYu8RxU7Rw9Qo6ojVkF/3EKPOduC5QwOWwHHP06/CnVzC7ho9IcWaU+VZcOsrBVSMPktfhT6/gHZB/Tzruz5CMIt+MS0MBFy6z45xXrrPStnHOqQY/Mcp8Ky4dBVze5MijmfXCTa9geamQeczjZ0aRb8Wlo4CLBzExruuq9wLm1lTnj2WIxZGGfCsslQRW0cSYrFMarrPSegvR2eNz/vNQQ7wVlm4CrWRCTNwRWLXOCmbKl0ghTIpDvhWXjgKuzB3OHJSpu73iLfYls7gOODGKfCsuHQVchTucytrEvcC/xdi2X8ZgKOOtqFQQSFXua5Zi18s2vYLrfiVgKY2RUMZbYekmrBWtwSrfyqTdXkGfvfIo+rgFTyjKfCeuC5SDi7+RlEm7veIpm8BDnOMj/n6sId+MS0MBlzMlWay7snCdFcyrcSHmmXHIt+LSUcBFJicX6zr06RXcm8j/Cotij7fC0kmAxWeQiJQpu73SVuCKqS1NKQ1FvBWWTgKsYGIiTIBfsM4KBjih1BjCiDjkW3HpKOCKhgfFqawjn17B3frZZ9wAOzAO+VZcOgq4kuE/3SuTdnvFW/4nuRQmxiHfiktHAVdb5sEWpeU6K1gmlnuidHzA34815Ftx6SjgKsalmJXX4nsBq9jEQvG4CCEQZb4VlkoCq4p5fkGZsdsr3hXja3phvR9JpltR6SKHVU3ZZaXVOitt+hHlEuNoOOSbcWkozJUs9zVzyesFm15pjXmMr3F1P9aQ78R1gQIux73NEpXpur1CxRqHEWEaGYd8Ky4dBVzE3c1KaW24egXvfRSjbcsPD4wi34pLRwGX5+5mrWm9bNMruAbvM71u7xSMMt+KS0cBF7fWmZvudfjTK3gW+dCuA46MMt+KS0cBV+QniPPKhN1eQRtVXY2vtkswinwrLh0FXPjQmWxcL9v0SlsPgvtZbctvySjzrbh0FHBl4zLluA5/eqUttEi+tD1YxLGGfCsuHQVcxWBprbWXehZwHR4D6DIayngrKhXkkKoZG2FoVK8K8XMp2zbrdCCU8WZYmgm0qik5FO2JdVbaADqE1xxBoSjzrbh0FObK1uQcozJht1fabC1n66sLIRhFvhPXBQq4nEk5rdum9wKaJx5Cv5Yxk4gi3wpLJYEVmZhT9evYp1eON77yulXxPNQQb4Wlm0CLnx45J2Wybq+gV+XeP7Duh5LxVlg6CbACdzS5c6m0WmeFX2vGcmNuw4Qo8624dBRwRe5o4sLLynVW2uSjHLHb1sA45Ftx6SjgSnzGNStTdXulbT7pcz5u7OzHGvKtuHQUcGVjiw3am+JZwWJSPllfaGQc8q24dJSDqxbnaL1o0yuY8J1cwD8HxiHfjEtDAVcxpbiizNbtFfTbffWvhfoFo8y34tJRwFX5ZCgq03V7pa0Dwc37cRVCMop8Ky4dhbkKfzN3M9160aZX0GHIseZME6PMd+K6QAEXPnj2ynSbs4AWyvJL7tjvRyDKfCsslQRWmF4UkjJht1faHcI+VB9HRBlvhaWbQMsbX6K3a7vVK20I7eNr7RGpKPKtuHQUcAXDryhr1wFQrxB34p0P8bjjRzDKfCsuHQVc0biSkjJht1fa/dTep2N+cz/WkG/FpaOAi9vqkn1Vmq6zciz4E6KjkXHIt+LSUQ6uWoqt6wCoV7CIp+VnVKSZUeSbcWko4Mrc4SxZ60WcFeIRNb/9vdZNkowi34pLRwEXTqaGorRdZwU3FWD50+PGYcEo8624dBRw1fbNZb140ytYNzBVT8fyi4JR5ltx6SjMVfnXVtxIt3D1CmHqKR+rzIwy34nrAgVcDtOUY1nbrl5B7z3xk+h4dglGmW/FpaOAi7DMAymTnHsFHYZoY6A0Mcp8Ky4dBVyeD0NVmbrbKxhIR1vjsaSGYJT5Vlw6Crh4HFO9ts1ir7SbOV3kLyZGmW/FpaOAKxruQikfx54FdBdwy/CxjY1AlPlWWCrJYVVrtMq83V6hhAVaKL2eWieijDfD0kyglUypMSvTdnsF27il80YpoSjzrbh0FHBlk2sKab140yu43S5Sqcc8VMEo8624dBRwFf7mzG31ynVWwMIvPpv9ynjmW3HpKOCq3OHMRZm22yuNxQZni8L4nm/FpaN8fETLP8lRXNouUWnrb1l+Qwwzo8g34rpCARc+Ua0UlwGQqKBv5UMK3k2MMt+KS0cBF3ecaq3r3F1RocQvOrLtU1epKOKtsHQSYHksl8vt9op1VnC9NGTv4mQo4q2wdBJgBcz582EZ/IiKd+2THmvdiDjkW3HpKODC1ue0rgfUC1gPonge6oQJUeZbYakkh1W1mKqsYb0q6LbHkGNb27ofaog3w9JMoMV9c8tPlOWijahQrvx0ojgbnulWVDoIqLhfzr3LddquqOCe6pJSbJf9xLGGfCsuHQVc3C+3Yb003wtYHsk5X30YEYd8KyyVBFYVF4sjLRdsRIUycY+qkI0jooy3wtJNWMvxT/K5rFN2RaU15dYFl2ZFke/EdYECLu6U21TXKbuigpnMuGGlzXsQxxryrbh0FHBxt9zmtO6jKyqUuDfqIqZADIxDvhWXjgIurPpavFuHPb1CqRr+Re3DnUGxx1th6STACtzT5O9e3xR7BavYuJz48BOizLfi0lEOrmprXhfYFRXMqck+09HQC0aZb8aloYArmuJssOvIp1faBxaZD00zo8i34tJRwJUMn7lbJ+2KSutd2WxLnBlFvhWXjgIufOH4OCvXWWnXSH2NvkyMMt+KS0cBVzERy00uWK8c7VOIDOMmQplvRaWBAKoaHvW5dbquqFCyJlMIPg2EMt2KSidhLLLGO2zgvWD1CpXKPaxaS5gQZb4T1wUKuPhPdiHW9ZJNr+BTQ+sKHf0tySjyrbh0FHCRcQ6fz69cZwWbmVZHx87CklHmW3HpKODy/ASJtawtV69g8lHI0badyCSjzLfi0lEOrupSKutlm17B3T3ce3fBT4wy34xLQwFX4M5m9utkXVF5sZQ2n2ZlLLvNs7lCARcP97gDlZW266xgUq7NLuWFUeRbceko4MKt+CXn9bJNr4Cl1pxyWRnPfCsuHQVcmbucNayTdUUF74DF11LSxCjzrbh0FHAVTP5wyhvjewEozlrf1judEM98KyyVBFbVeD6htF616ZW2aiC/9o5ZNgJRxFth6Sas5S32ho/rVumiQgwQElU3IYp4J6wLEmA5w/1xWqfqigpl7rezTygTosy34tJRwMXnRrRu9NMLuDhjyfMgekYU+VZYKslhVcmnuDbxvYLrMylFe3y+3xFlvBmWZgItbwoFr0zT7RWKbSVPf4wUhaLMt+LSUcAVTCYeGyvt1llpA2isJhUmRplvxaWjgCsaPpMc1qFPr1BOpmQ6VjuVjDLfiktHAVcy3DsPykTdXmlrb/Ex2lYiklHmW3HpKODCRO7sgtJ2nRV8HJacrbZMjDLfiktHAVfh/iZ/33rZplewpSm//x3LSUlGmW/FpaOAq3KHs0Rltm6vYEwYcCtnmBhlvhWXjsJcgUfHhI7BwtUrx7qB8TX+kYwi34nrAgVcWJ6mVr9etukVzGMOqYY6M8p8Ky4d5eCqns9O0zoKFJPJWAg8TIgy3wxLIYEV7qtwXpmy2yvH3cHFljAiyngrLN0EWt5kT1aZD9ErxC17ZQGaFWW+FZeOAq6AtVYyrQOgXqFYjS8pHhPfBKPMt+LSUcAVDY8AgzJlt1dwicZhfVg/Mcp8Ky4dBVyJu03BKZfne6UtJmVDCnVilPlWXDoKuLLhcy9uHQD1CqYyZ3esmy4VRbwVlk4CrGIIm0quw59eocL9hkDh9VLsiDLfiktHAVc1zieySst1VihhhQMO/MQo8624dBTmitZYz9++Dn96hVIw3sV03DYsGUW+E9cFysHF72xJmbLbKxSjKdxxT3FilPlmXBoKuBx3OYu3a9vVK3gWuUg2zIwy34pLRwEXcZez1LpeuukVDHVCjrHtAisZZb4Vl44CLpxBTcrU3V7BJZpaPLWN5CWjzLfi0lHAFUwMfBSl7TorxP1S4gOHmVHmW3HpKOCK/JNO6UechXY9Kyd33OIpEGW+FZZKAqtkuHuelXm7vYJ18HiQU487WTqijLfC0k2glQ2hU660W2el7eyDVXVpUpT5Vlw6CriKccG7sl666RXCpCR+Ph23swhGmW/FpaOAq/KQzxdl2m6v4DNqbqXCsWySYJT5Vlw6ysFVscWrMgA6K5gVaDENPEyMMt+MS0NhrmRNCZHyeummV3DVoaTsXJgYZb4T1wUKuJzJIda8DoB6BfNrHO7lnBV7vBWWTgIs4p5ASsq83V6hwN2szAOdOiHKfCsuHQVcHktr+bS2XL2COxR9dDbOjDLfiktHAVfgnyzaZ7G9gguAkQ9+zBgUjDLfiktHAVfkjlNR3hbPAvG/ko8vK2HY462oVBBIcZ8p1BCVZuus4PaxQCkfPYhOKOOtsHQTaGXjonVxHfz0CsaEgduml1ZXlPlWXDoKuIqx0RZl4m6v4DMxfvHxKGdmFPlWXDrKwVWj+uR6L2CXTh5Jp2O5FoEo882wFBJYVcMnTmG9aNMruAiPj/LDhCjjrbB0E9bK1uRIVZm02ytoztEfTXVSlPlOXBco4HImRf7j13arVyjw77LxdceiYJT5Vlw6CrjIxBh4tLdynRVqO205d1wSFIwy34pLRwGXNyFGq0za7RXcnFi8y8cqu4JR5ltx6SjgCjzY4/o69OkVfGjBtRppYpT5Vlw6Criiodi2JVi4zgrWDszcWB1zBQWjzLfi0lHAlbjDmR2t459ewQCaw1DKxCjzrbh0FHBl7nJiy/iV66xQYBb+eVsnRplvxaWjHFw1cj9KabvOSrv/NdvXrHDBKPPNuDQUcBVurasyzfkstLVGMCmQJkSZb4WlksCqco+zVmXWbq9g7jf/i0IaEWW8FZZuwlqFvznZ5NZ2q1fa+oE1xWPlN6Eo8524LlDA5UxMzrv1ldgr6Ir6UGqdGWW+FZeOAq62kohVptv0CobR5IuPM6PMt+LSUcDljU+U7dp09QqWTg+2HJ/CCkURb4WlkwCLnx7JB2Wzn14h/nFs75PShCjzrbh0FHBF41JwyqTdXuFjGhusS35ilPlWXDoKuBKfW8hVabnOCoXKHdHo7cwo8624dJSDizsCQVkTqFfIV+NqOPpcQlHEm2FpJMDK2OPIKVN2e4USA1lMfZgQZb4Vl44CLj6ZlJTFds8CeR4cWh+PwY9AlPlWWCoJrLjl4aanrFdteoV/AS4y++OaYEeU8VZYuglrVf61qZAyZbdX2scW1edjpzKhKPOduC5QwOW4t1mqMmW3V/CBq/cpHivtCkaZb8Wlo4CLh3upKvPezgI67Ync6yYpgSjzrbBUElh5wx1zn9eRT69gkaSacrUTooy3wtJNoIUdjpxVpuv2CmEDa4vypCjzk+vL3xCf+JOev3syB87JPb9+fvnVt//912++/cPXv31+88Ojk0rOjzL3mBZ85G+Pxx8fyUaTx+8+ovkbr469HGD+bnGgq2N/+RuHc7N8blh6P6cDffkivPu3BxYP6/Pr4Zse/+gnHo+Q8adm48M7DLZ5Pp5lPX6TscsFtxcdefv5IeGH58/t4ZkO/Y+fwu2RdHgkn9/x/9pD+RgfyogBSm3zkcXf29Phr2BC8RSxxsXnX85jty+/fqWP7x+O1b6wXIqJn4k4GOVUn9WEgq1Ivvn4+O2Hx5f/6p7OPT/8+WH52z/85+Pfn7+yv37+x/PD7x7/8uHx+1/knJ3jY/Fgw9Jw0iL+BGftnH0dzef0I846hV/8tHPg5hFzXIezPtNPcdKYJoqD8RsW9x7/+Vk7Kr/0aZPHinrexTqct4g/wYlT26eCj1ba7MUfceKVfvET5z4XxdzuKpMn3uNPceLtQwA+GneG/Y95nlNMv/SJt+Phk1TuDEyt75n+1PPN/LcE7KaefcYC7OV1qnyC/EZU+Mh8mv28f+V+/fzw3YMH7fw+66OLiQ4JtHJccN5E/k9KOPVnMNzV45HEeRCcSD9KMNwnrPwGb2sYjsKS3+P3W35Xh/Trn9j1nZufgAMG3PfV/r5vPj750F989e13f/q3v//xT3/74YuPf/3b3394fvVfz9//kg9IbtvR8zvm+Ij0+Cc3OsR/TiqmBOf5wHzCn+UxKfehrxmf/KY0yp/pz4KvuIki8sDO8svhs8Cn+8A7fltItuAzv4Fe5D8L37mCufPczeBubhT63LKjGf4l+MOP5/elNsz/Y/xYKYkd8qR/xj8PP/BwM3qHzv/neebTjZ75PBa3LsV5+CPyn4xfgJ94sBuzC7Wgf/2z9DPD+WILj43zNf75VvsT1Pxxfn3E//g51xKG73/8T68lYHHbSun5hSMelnHfn8+2tiEz1nFNmHM0VN5EpUQTi0vHOrnnkWR6Xke4qc+VAvs4Y731tPK8FzyeO6/L5v04Mr29jm7QcKp31hVN51UJHgMu7DggzGS4gY6KwDzHlHRMI5p5zgoPTVzBOg0jmkhv73OhwD5tLdbglBfXWeGTDO5YiqEfSIS317kwYB0sveqxFuKM816IPPrxIbcFiYSYSG+voxswTltotWBa0KxzVvAs8Ta70UyEt9e5QGCetrJqxDSgmeesYH0YyrZ9Ni7QRHp7nwsF9sFaqpGs8uI6KzHhNlsf06gm0tv7XCiwT1s8tWCiz+xzVtATTAlzzAc1kd7e50KBfdpqqfgwe/E5KwGju1zaGlX9SDK9vc+FwsdHxvKomeo6qugV7u04HsFWGtRkenefK4Xmw4PNWtf2p1fa57qeu4GDmkw38FEV2KctgJrKOrDoFfR4Usx+QBPh7XUuDFgH651WX9ZxRa/wK8ranAqNZiK9vc+FAvvwCQRri9L6nJWI60Ku3bnSDyTC2+tcGLBOW9E0Ky+t90IMuMJZ2wIjQkykt9fRDRinLWAa8jqu6BVugrl/XCkMZiK8vc4FAvNgxVKOlJbnrCT+TS5R6+H0I8n09j4XCuzTligteb2i0SvcF3S42TePaiK9vc+FAvu0NUljWscVvcIjUD692Da36EeS6e19LhTYB4uQYumw1eesYJvbEgrRrHamt/e5UGg+NYSa1usavYIb4ZK37epgP5JMN/BRFT4+SltnNKV1XNErPFbPpeTgRjWR3t3nSoF9sLJo8nHt/PQKLjMH50oe1UR6e58LBfbBUqL8g+t1jV5JEav1lbY9jFAT6e19LhTYp60dmuM6sugV7DbkUm5LxQg1kd7e50KBfdp6oWHtHp4FTNZvq8QMZjK9vY5uwDhYH7S6oDQ+Z4WbmUgV5yfMZHh7nQsE5mkLgqpPnrOCaYM8xmqfi/YjyfT2PhcK7NNWAMW9jIvPWcHnFK7U1kvuR5Lp7X0uFNgHi346WruGZyFb4/iwIQ5mMr29jm7QcGrE9p2azqvCT5PkfWwfHJ8HkuEGOioC87RFPZNXnjxnhUcQpbjYRhACTaS397lQ+PioWMXTe2UaVK/wCCLw+baZxUJNpHf3uVJgHyzbGSyt1zV6JTvjqcY8qsn09j4XCuzT1unMy/zDN1FJwWSy3NEZ1GR6e58LBfZpC3MG5c2rV/BMiUxSBjWZ3t7nQoF9sBJn4nD1OSs54TmT2+R6oSbS2/tcKLBPW3qzKBMxeyXhWnMp3g9qMr29z4UC+7S1NqMyE7NXuB/oa/TtPup+JJne3udCgX2wuGYht17X6BUeoxNueE6jmkhv73Oh0HxqLMp0jbOAGT58sHb9tB9HphvoaAaM05bPTMpczF4pbb/k9unWeRyR3d7mgoBxqsG+x1Zpec4K+jiOio8DmUxv73Oh8PHhLH9z4v+7Di1EKReTccJ5cJPp3YUuHUDU1sRMymxMUQIGhUJllXuP7290AQGjthKmr2sjJErcWc4lp3apR9KJ+P5GFxAwwvqXHreRr0ZnCZfiueuTJ7me3l/oggFCbcnLrLzRixI/XZy3+fUs6nAivr/RBQSM2jqXoWit0VnCRdVEtX0cKOlEfH+jCwgYYXXL6Mp6vUOUChmbqy9ppJPx/Y0uIA6jmvgH1zGHKHHsuf0JeaST8Q5GKgSM2lKWsWjt0VnK0cRaaprkenp/oQsGCGH5ykzKdFZRyt5EyzlNcCK+v9EFBIzaopVVmbUpSpjKQTGOcD28v88FAlYAa6tUpqy0RL2UIgb1pX2IKg4m49sbXUHACItTap8f9gq/nIpP+EB+gJPx/YV0BgCR8dlaZeqmKOXELY6t7f4UASfS+wtdOICoLUKZldmbolS8ydG1eeLiWCK9v9AFA4Ta8p085lKEzhIP6S22CAwjnIzvb3QBASNucTNhBb/V6CzhYx9b+HgTnYjvb3QBcRjVTEWZxylK3ENMlH3wE52IdzBSIWCUTMk+Rq01OkupmuximuREen+hC4ZjZdScA0Vl9NFL7Q6o5NotheJgMr6/0QUEjIpJOVRlUqcocbND1pdAE52I7290AQGjamKOKWit0VnCUAPPnDzSyfj+RhcQbET8ozmh1V2MegkdRcwhp5FOxrc3uoKAkePuYFYWNemVwj+dXrdBiUPJ+P5COgOAiDuDOXulMeolHo/x//dtv51+LJneX+jCAUQeq2QGr1wL6SXELuW2fa2UE/H9jS4gYBRwBdopsz1FiRtmS5F8HulkfH+jC4jDqOaqXbw+K9wFsp7s8VFaP5SMdxDSGAAUTSk2knI9pJcwZi0Je4kOcCK9v9CFA4iSwYK9yqxPUeKYnyyF3CQn4vsbXUDAKJtUXFVmfooSJqERTnqiE/H9jS4gYFRMLJScckWklzh2AUsHTXQivr/RBQSMsEeB904Zg/RSzoxRcywTnYjvb3QBwUbeGh6NWmUWqCgxRog2Rlrozvj2RlcQMOK/uYSsLMspSvwGxi+vfLzWBJ2I7290AQEjMq7EYJVhSC+VgEW7wTLQyfj+RhcQMPLo4zhlXqgo8cC+2pqP9/5+MBnf3+gC4jCqhZ8MWnt0lrCEDg/tbVnozngHIxUCRoF7hNon+++FYvnJkmsY1UR6fx2VADYRa5o4bU5oL+Vjx7wUJZoM7+9zwQChxO9IpVTt+XOWsExDyXTMfOgHk/H9jS4gYJT5PGusytWQXuI48qDjGIH0g8n4/kYXEDDCBluWtDmhvZSjIefyMUNd0In4/kYXEDCqxvPbdtFao7OED6j5N71ao04n4vsbXUCwUdtT2aWiXBHpJdw3lPPrE8d+MBnf3ugKAkbO8Mjda3NCe4lj7inSTCfj+xtdQMCIDP+TOziK0VkqxRRumo8PQvrBZHx/owuIw6hWn7NyRaSXeDiWS/Xt9mBJJ+IdjFQIGHlTKv+wMgLpJSy0wyNYHyc6Ed/f6AICRvxcqNEpb2tnpTgTq43Hp7ICTsT3F9IZABT5m2NJyuWQXuJni7MhJD/CifT+QhcOIEr8TEgxKeOQXuKnC1H1x/u+kBPx/Y0uIGCUuTeYSZsb2kul3dLgjptBBJ2I7290AQGjwr3BrGzL0SuVsBUH5dFNpPf30RHAU3GYFLW2+ixxm1NjOHYo6ceS6f2FLhyYKFruB1avzQrtpcJv8s6VEiY5Ed/e6AoCRtijxfIbt2J0lgp26oyhLQwi6UR8f6MLCBhxX9naHJQRSC+15WHTsXClpBPx/Y0uIA4j/uagzQvtJXSBPNk804l4ByMVAkbcW7bENcXoLJWIXXH9cUeIoBPx/Y0uIGDE3WVLJShXRHqpcBzcMdgXcj29v9AFA4T4mWB91CaG9hIW1LXsQhOciO9vdAEBI9yFF7BDyWp0lhjDlbYixkx3xvc3uoCAEXeYbaheuR7SSwXb2Aaf3EQn4vsbXUDAiLvMNiaN6L1SgqFgyzHtUcCJ+P5COgOAMHUxeW1maC/xK4o7QPn1lnbCifT+QhcOTIR9xC23v8r1kF7irnSu5I87ZoSciG9vdAUBI4e97DIpo5Be4pdUjFi5aKIT8f2NLiAOo2r5TVwZhfQSMHJMx03pI917vIORCgEj4s4gj9uV5qiXKj91MrVdN8WxRHp/oQsGCHnuC9bilDFIL3GzQ9ljSYwBTsb3N7qAgFEwmHemzQvtJR60RgqplolOxPc3uoCAUTSck9Vao7NU+L2e3u9xFHQivr/RBQSMsFeHq1YZhfQSPqLOKbmV7ozvb3QBASM+G0dJmxfaSxXTrusxvbgfS6T3F7pggFDBZ89e2d9elEo1MWD3yRFOxvc3uoCAUTXO+VqVayK9xG9fPpNNeaIT8f2NLiDYCPuOO35nUsYhvcTNTs6WZrme3l7oiuEQqi56Zad7UeI3+FBCPkYhAk7EOxipEDBypvDgS3nbPyuVT6vGfKyQ0Q8l4/sL6QwAIpNxY7kyBOkl7GmRQrI0won0/kIXDiDiU3A5KHvei1LBtP1cPY1yMr6/0QUEjHjE7gpmwK5GZ6ldQuMOo5voRHx/owsIGEXshVe0maG9VLHaXHSvl9p5MBnf3+gCAkaJ+4Lc7CpjkF7i93cK5Ee4Ht7f5wIBPtkQWcrK9ZBe4ieLTS4GN7LJ+P5GFxAwKgZj0qyMQHqJG+aUeVjvRzoZ39/oAgJG1VjiwakyAuklbFSZuWWe6UR8f6MLiMOoEvmktUVniZud6HN8tdedTsQ7GKkQbIRdyslb7SPHXqoJC6lVF0Y6Gd/e6AoCRljJ2mtvaWelojPkwnF/moAT8f2FdAYAkUkUwrIp/JsssYXjt69jWwcJd6b3F7pwAJE3kaKLyhiklzCo58Mf02f7wWR8f6MLCBgFEygWbW5oL5WKt/pyzA0VdCK+v9EFBIyi8ZTUp9F7pfLgtQQceYCT8f2FdAYAJX4iZArK9ZBeqo77QvgQf4QT6f2FLhxAxP0/ylWbF9pLNZqMXiJNciK+v9EFBIwK9wRLWvaKf5MlbnSwT/Xgdmb317kgOHQqVe+VKyG9BJNSsRHqiCbiHYxUCBhVU7y12pzQXsK/0in0DneG9/e5QGAf7GPubdbmg/ZS+8wVi4ONbDK+vdEVBIwcn4LD3Ryr0VnCPHRny7Gaaj+YjO9vdAEBI+KROjlSBh+9xCwlVFwgGuhkfH+jCwgYeRM8FW1OaC/xcJ5b5XSshiXoRHx/owsIGAXjPXdstPboLFVMdciveQ+CTsT3N7qAgFE05IM2bfasYFfLEMn6EU7G9xfSGQCEW8tC1SaE9hKrWJfqMT/9PJZM7y904QCibKyPadlJ/k2WKlabyzw0m+REfH+jC4jDqPKf77QX2lmqmCibjtuJhFxPdxBSGSBUuC+YrTYxpJfYgl9f7tgldYA74/sbXUDAqHJfMOdlZ/k3WULDTNwuH8vP9aMN+f2VLig+Pgj7nWOR3fVqiCi1vZqjL8cCIqPemd9d6ZICSo77g9Ups0JFCXvH52pDXPREfn+lCwooEfcIa142mX+TJX728PnWtiCfOJiM7290AQEjj7MJykIrosQghl9fNebRbsjvr3RBASXcz+mcMjdUlPC68iH4tj/qoCfy+ytdUEApGhecsnRor+AZw0P9GN1sJ/L7G+kQIErG4hOg9fKIKOEJ41xb5nG0E/H9jS4kDqSKofw6JhElvNdXz+9jbsYT+Q5KKgWUsincDitzREWptlUxihvtRHp/oQsGCBWTQ1AmZfVKe03F4l0Y3Yb8/kY6BIi4mxMi954Vo7MEjMJvX22INtqd8f2NLiQYCTugB/5WZUzSS9CoxR+7gU14Z357pSsKKPE7eEg5K+1RL1WsQlePpebFwWR8f6MLCBgRn00OSeki9RJjOJ/5AAvdGd/f6AICRp47hcUp80RFCf3FVGv1I52M7290AQGjwH1Cbni1FuksoeXh7nUMcbYT+f2VLiigFLlTWGNSrpL0Er+sKKbQFsUSB5Px/Y0uIA6jGi0pk0VFqT1jfLIpjXZDvoOSSgGlZEq0ddl8/k2WcB0yUco04sn4/kYXEDDiU4ouReUaSS85SyYlV2Ic7Yb8/koXFFAqJkX81Ir0XmnX+pPjs53tRH5/Ix0CRNXE6O2y//ybLOFVFYNtdzn2Y8n0/kIXDkyE/dB5dBGUp1EvoTMUseh8GemG/PZKVxRQcsbHEJRZo6LUNGLMxzWSSe89v7/SBQWUiHs60SmzR0UJ7XNIbe7IpCfy+ytdUEDJGxcj5u+tSmfJ2dTuU2uzRQc9kd9f6YICSsHYyO9Vyrikl5zN2CfMukVP5PdXuqA4lGrMpMwjFSV+KwspJiojnox3MFIhYBS5a5graa3SWcL4g5shFya7Ib+/0gUFlBL3dwqPLhSls4R+Yy6Z2t2zg57I7690QQEl7FRQvTKXVJTwTlZroGNkIvVEfn+lCwooFROTtcu29G+y1N7JAtVKs57I7690QQEltL82O2Vw0kvoO1Z2qW7WE/n9lS4oWAk7pScXlFmlonSMZz3uqZ30RH57pSsKKPHfnLjHo7RLvdQuRNZ8XAzoB5Px/Y0uIGBExiVStsjsFR7KGlv4jMMoN+T3N9IhQISFaHxUJpaKEjAcP12OeW6D3Rnf3+hC4kCqKdCyOf2bLEEDFySLgnfmOyipFFDi80zYqVhROkvOFuP5Vx1zuKSeyO+vdEEBpWhyilGbV9pLzgWTgvfHJC6hJ/P7K11QQCmZlBJpM0t7CWN+RykfnywJPZnfX+mCAkqZO4eJBxiK0llqM21KPe4jGfREfn+lCwooFe4c5lS1bsBZchZLR1Aqi57I7690QQGlyr3D4rXJpb2EkVqINbZ7JAY9kd9f6YKClbCHOn97UdqlXsI7fig+xjTrifz2SlcUUHLcQ6y5KGOTXkIrnW0+1kIa9ER+f6ULCiiRsVn9pPKsNIsceEii2L3n9zfSIQ6imp3LutGrhK6jdb7Y2U7EOxipEkDyBjfSZmV00kvQyPzjNq14Z35/pQsKKAV+xVDUZpj2Urt6VLJtO0APeiK/v9IFBZTiseaqgvReae/1qbbN6Ae6Ht9fSGcAUDIx+5o0obOEy/6YjnQMcoWciO9vdCEBpGwwpVabYdpLzrVLJKWt2CLxZH5/pQsKKBXj0c3RWqSz5Gxte2j5MunJ/P5KFxRQqoZyslG5XtJLzkXWyNb7SU/m91e6oGAl7KyeU9bmmPYSXlkxlONGXInX49sbXUHAyHHPMIdlK/s3WQJGKcH5uNqd+f2VLigOJR5i8J+uKr1KrfWJhEUSRz2Z76CkUkCJuGvIzwdlYNJL0KihzeNe9M78/koXFFDy3DusUZtp2kvO8dOnVmq7aEk9md9f6YICSsGkYnm0oSidJYf7lLILecbr8f2NLiBgFE3k7o7WdL9X8NlIDDk7muVEfn8jHQJE2EbMJW2SaS85V40LRGm2E/H9jS4kgMRnwz/ltRbpLDncWlKyD7Ndj+9vdAEBo2KoeOuVcUkvAcPmEPxK9x7f3+gCAkb8eik+azNMe8kR/zZfyjEqEXYyv7/SBQUrJWtsCUG7nNRLeAsrNVCgSU/mt1e6ojiUaomOlDe3XnK4lcunfMydlHoi30FJpYCS48Y3Fm2GaS81De40tlUlZ733/P5KFxRQws4QKS4b3L/JUhvLkgvHvAmpJ/L7K11QQIlPoWRyytWSXsJbGf8y52nWE/n9lS4ooMT95pKrNsO0l/DK4l9GNs56Ir+/0gUFlKLBTf/LRvdvsoTrkDYlf1xVknoiv7/SBQWUEjZc91a5XtJLXOHOY6p51pP5/ZUuKKCU+TjWavNMe6mN1Wrybfd6qSfz+ytdUECp8HnarM0z7SU8Z6Ln3zDryfz+ShcUUKrGYj+/1ehVQAvtYj2mBQo4Ed/fR0U4dGrFFpqqz6uEV1TwjnIa3US6g5AKwUbZ8vgLOyGtRr2E1xMVm1/vbN1O5rdXuqKAkjO5+lCV1qiX8JzxzoY868n8/koXFFAi/m59eddeQm8op5DbrsiDnsjvr3RBASVvYm1rIq5KZwkNNJ48qcx6Ir+/0gUFlIIJNcaitUtnCa+sTHyYMuuJ/P5KFxRQisZX/ea3XsKVyIpJW2XSk/n9lS4ooJT4OJhTqyidJTxn6mtUIux6en+hCwYIcXew5qRsmSxKeL5kHu0f10qEnMzvr3RBAaXCncPis3KtpJeg4awrTtE78/srXVAcSrVW7V7Ts4JXFYf2WCtI2Ml8ByMNAkSV+4b8tNAapLN0dK55tFZGOxnf3+hCgpGwK7u1ISnXSXrJOXbJ6bgJR9iJ+PZGVxAwwu4azmmzS3vJEfFgP+Q428n8/koXFFDi7rN1Wof7rKB1TuV92qS0E/n9jXQIEHHf2VKMSqvdS46SSSEd0wEEXU/vL3ThACLuOFtP2szSXnIUsYtrOpYvl3Qiv7/SBQWUuOvMR9FmlvYSWh7iM7aLnsjvr3RBASXuPHMbHJQrJb3ksHhSSb6WWU/k91e6oIASd55t9EFruM8Smh8XbTwmTUo9kd9f6YICStx9tslqM0t7yXky0RX7ape6nszvr3RBcShVm3B1UVN6lVx2JqVcXq+4rifzHZRUCijhfqMcvDIw6SUXkymZ8vHZpNCT+an05W8Ip07P3z0ZBGflnl8/v/zq2//+6zff/uHr3z6/+eExSX48Asdjnngkb4/HHx/eOuMPXPm9Ip5/Yjmueoj+reLn/+Fxv/yNaw8nn5Plt3Vctgb38kV4l28PKR7Q59fDNz3+0U88Hj4S7pLj/w5nkWwczuvtlQx/pvjZIecH5s/tgenfMP2Cf/xEbo+mw6P5/O7pHtrDyT+FvbaqcyN+j2dQ8UTBrkrPvxxHby8B/vLrV/r4/uHY8AvLJZ9sW0+QO1XBxmfFBF4+3DcfH7/98PjyX3Hp7vnhzw/L3//hPx///vyV/fXzP54ffvf4lw+P3//CAIHH6KnkyK+c4Vnd408AELBMEI5Wsq/5RwCk8BkFuO3wnlyaX9dn/CkEEna/5aMdK2/8cwFH5fMRRJ9wDSLXOBCI+BMQRO7lBBwtVP+jBCp9RoHKz1Bygfwo0ONPIcDDB9eOZj32QPqnBBTT5yMQrXBoW1kQv1G+t+A9+cmNoEsm9j88kAlYXiUmzEA/CPjE+S/A+gZ8+t3jV+7Xzw/fPfi5WEv0kd+76RBCO8kF3C7K/0kJEM9gyHFHoB8E75Y4SmrfzJDf49fYzJ2Pp33/J/bKseQDfi7gY8v2Z3zz8clH+OKrb7/707/9/Y9/+tsPX3z869/+/sPzq/96/v5zPh7tQx7LQ87z8ejJJ3o8uNtoPXeaMr9ZRfF48NMez9Ff4gGJP/4B8aU21P9bHpBjX05rbTgfERF9ooekVhMz/+m1LYzyWR6ScOOHxGPzhorP1M+HpEef6CHhd2XjeDzqLDcP7rM0W/6+zZZL3KjY9inH+ZD06FM9JDyUSxHjyeQpfJaHhP5vekgwWn/86DH1PxutP4bv/5+N1vnPM7ESUL8gdHZNrkSUXuMp7vYRlooYSm+yhBUgbW23pL/Jow35OVrfwOsCBV54f/fYi371Oksez1DPDiOjjHfSuiCBFiZ4hISL/IvWWcLaGe644iOOJdKdrC5AYIVpHmj8FKuzFAs2/IvBT4Qi3knrggRamPCRbNDarbOU+DlUSq1uRJTxTloXJIdWrSnj0r+i9SolMoXD1k+WiCLeS0slgRbmgLR701ets5QDdrrNuYyIMt5J64KEtbLlL4pTmvizwoNie1yuGglFvJHVBQiosG90KfhoZLE6S/zjiVI6+lqdUKQ7WV2IAIu4m4kGW8E6Sxn3a6V276gk7OlOVhcgsMIkkTZ7drU6S/wGWI7FEsWheriT1AUHpDBRhAePCtR7hf9RK5Uy6ol0JymdA1CYK+ISKQOdXsKM0mSPT/4FoEh3sroQARamjBAuc6xYZwkzJrh5cm40lPFOWhck0MLUEW9Ja63OEv+Le+jt7lJxLJHuZHUBAitMIOG3NmWc00uFu+iB+1R+JJTxTloXJIdWtSE4ZZzTSyUZG9KxiaBEFPFeWioJtDAdJzq1636WmIV/lW0Lvo2IZ7yT1gUJaxXuivMPOuX6TC+1eajBuzIhingjrSsSaHFn3KZolaFOL/GTqFJwNUyIIt5J64IEWtwft5ms0m71UuVf5lNqk1HFwWS8k9YFCbQ8v5pytUonvpcqpsWhr7AgnvFOWhck0Arc0yzaM+tVqNhBPlOtE6CId5JSOeAUuZNZ+UCK1Fmq0fBLrbSlBjugCHeSugCBVeI+Zq1VuTbTS/wEyiFSoolQxDtpXZBAKxvruKukjHd66dg5pqT2Abw42pDv5HWBcnhV53xRxjy91HZMDyEkNzuKfC8vFQVexRRHtiiXanoJjVQMlP3IKOOdtC5IoFVNdoTvXLXOUi1YXdb6MiGKeCetCxLW4v5Scj4UZdTTS3jNRWfDcR2wH23IN/K6QoGX4zMOLittVy/BJZeUXV4dz3wnrwsUeJEJLpSs9E97qWYeSIdg/cgo4520Lkig5Y13MWZl5NNLzmFWVsXmT4PikO/kdYECL8y4SpS11ussoYdlGSdPjCLeSeuCBFrROJdqUkZAvdR2y6RQU54VRb6T1wUKvBI/QXJKyiiol/Cqc5XHP7OjzHfyukA5vCp3n5LWep2ltvdxen1OLRhFvJeWSgItfndzVbskeFbwmguO6LjIJQ1FvpOWTgIsbE5bszYtt5ewwU/iYaJ3o6KMd9K6MAFXNYlsiFrLdZawfj036K7thzQwinwnrwuUj49g+feSc3G9giNKxz4ktrS9IwZHke/jdYkCL2cCuaLMzhUlrNaKje7a7jaDo8h38rpAgRcZTxSV+bmihAXJsOxWmy85OIp8J68LFHh5Q5gPsl7DEaX2PHJkX88v4SjynbwuUOCF+5u5s76Og0QJ99BRzMf4UDrKfCevCxR4RYxlkjJRV5TQbwj8lGqrKEpHme/kdYFyeFU+D6+1X2cJ7VQIBVtQTI4i38tLRYFXMoUSWqDV6yy1bQT5iVTC7CjynbwuUOCVTaakXPbqFaz7WnBsNyuKfCctnQRYhXueOZDWeJ0lLG1Wasyljooy3knrwgRclTue7cb9lessYVXBVF2OYWYU+U5eFyjs5fhHiX9OGQv1EtaqdDnktDiKfCOvKxR4Oe54Vm3oeFbwLMI+H5RnRZHvpKWTAIsMeUtuvY4jSuiQ5lBK+1hRKop4J60LE3B54/yxcdHCdZbQplMmbJU+MYp8J68LFHgFY71LTnsxniU8j3wuR7deMvZ4J60LkkOren6yKFdxeqmtvBJ9m841Ib7He2mpJNCKpnhvlQv2otT6V5jaHGZFke/kdYECr2Sy99lqbddZwjsgdxZScbOjyHfyukCBV+ZTCMEq13B6idsq7jz4/OpJdEeZ7+R1gQKvYqKPVpnPK0popkrhkU+dHGW+k9cFCrywW2HMypxeUUL/nVv118tRMPZ4J60LEizDarGwb1BuJBYlvOpi9uW4Yi8UZb6R1xXKsWwtqbcx9oqjaizZQGFSlPlOWjoJsIg7nbkoE3pFyVHgX5ZLm6AkFUW8k9aFCbg89zpLLErL1UvoYYWaQswzo8h38rpAObwqH6Qoo6Bect4Z6wLlPDnKfC8vFQVegbudbQe/1essoZWKZEvMk6PMd/K6QIFXNJnPJmvN11nCeyB2jmyrtEhHme/kdYECL2x+xM2RMhLqJYfVA4tPbZ2IwVHkO3ldoMArY9V+q8zqFSW0Uy4RvV6PwlHkO3ldoMALn69STlr7dZawJm91MXo3O4p8J68LFHhVPhsfknIlp5fg4lKMx0h7dDzznbwuUNiLW2zCohDKWKiXmku2NWTF8T3fyOsKBV7cAoVQktJ+9VLrl1LJx2Vo4SjznbwuUODF5xZijMqVnF7CjmYh4n6g2VHkO3ldoBxeNfDJKOOhXmrvg5HreXKU+V5eKgq8vCkhqR/R9pLzyRC38DHPjiLfyesCBV7B4CN87en1XnH849y6x2P0KBVFvpOWTgKsyD3P4oMyGOolDHrIMYObFEW8k9aFCbgSdzyxxpTCdZZan5RyiX5mFPlOXhco8OJ3N6zxprVdZ4kHPia5UqObHUW+k9cFCryK8dEGr4yFegmtFLYE84ujyHfyukCBVzUUndNm9vYS3gND5SeWnx1FvpPXBQp7BeyL4rg1Wr16CddsKs7Rz44i38jrCgVezthIuLqwep0lh6eU43Y9zI4i38nrAuXwqtGTNrm3l1qfNNS2KdzA2OO9tFQSaJHhzkAlpfXqJefxwWyxuc6KIt/J6wIFXt7kyOMa5UpOL+FVF4ujFGdHke/kdYECr2BSjMqy672CZ1HE9nN1VhT5Tlo6CbCiif9/d3eTI0luQwF436foE4RF/Wtpw8AA3tk+gjELA56VAZ/ffKHsIENJXoC7mceqrI4PkUpJqZBaT2Q1XU8JN9Gc3GfIh6KKI2k5JuDivlPrg4xhtpTQY2g9z3tHqhejyiN5OSjw4qtpo1pre6VEGCwOSuPLUeWRvBwUeOGpTms7Y6lgvDN4dLj79VpR5ZG0bBJgLe5zTiyq+dZ6Suhf4fCyTzdVFFUcScsxYa6WuM+5mrW2V0rEHfnBr1IPRRUH0vJIthb/ZLZW9kqJuHHHM1T3PqAvRZXH8jJR4EXX7GkuYw5HSriNykqD6umo8kheDgq8sFae2jJGQVLCXODkV2jzcNR5JC8HBV58CT1na22vlHAf5V5KbYejziN5OSjwuvuaOE/x2+spEXe20H8fX44qj+TloMCrXZX/9dMYCEmJeGCd1kj3PsYvR5VH8nJQ4NUvzIla63uldLtwfO/Cfjr+yiN5OSjwGtwCNZy/9e31lDCiXn30kQ9HnUfyclDgNbFia1hvx18VtFKtjjrqoajzSFo2CbAWX1qv1uJeKe1GfZT7eKC34hNH0nJMNtfChs5W2/WU0IdPK++lOFpR4lhaJglr9cTdzjGHMYcjpc3Cf6Baip88kJeHAi/+nz6btbJXSlQnvya/7ebpqPJIXg4KvDJ3O1fuRuMlpfs+ovJZCa0dVR7Jy0GBV+Fu51rdmMeREtVxEXE5n44qj+TloMCrXvyb3VrZKyW870aqeZ2MEkfSckig1a4y+A1ldLykhHfdajRmPhVVHsnLQYHXvV9LasY8jpTgwnfUHPPb8ckjeTko8OL2Z+TRjHGQlKguHOzc9mO02lHlkbwcFHjNKw1MXH1z/aqgkWo9JRqnosojadkkG2sNnKxhan1KYKmUxl6o+lJ84lhapgm41jVHnebN9ZTu0fSak9rJqPJIXg4Ke410jdGata5XSriynng4PQ9HnQfy8lDgxVc8unEAu1SILzP11u4Tnl+KKo+kZZMAK19t9FWMYZCUbpZW9/PGb8RPGsnKEQFW4S7n6NaKXilRx3YlqewVvQpR55G8HBR4VWyTV6xFcVLiP3FRzmNPeSlHnUfyclDg1bjLuVI25nCkRK1eo6U+yumo8kheDgq8Onc617DW9EoJVzbwEHY7HVUeyctBgde40rx3T/r2ekr3+26U3L8cVR7Jy0HZXmsSZWMOR0pwWVTS3sv+7fjksbxMFHhhZS5Na1mvlG6XObFA4tvxVx7Jy0GB17rGzM1a1ysl6tyuJ/qcxaEcdR7Jy0Fhr8k/jciYxZEStXVR4Rdfh6POA3l5KPCiq81iLcZ5KnjX9cxXWE9FlUfSskmAlS8eMXdrXa+UqPN/rtX2F2iiqONIWo4JuMpVZivJaLukhDFPq5XKqShxJC2HBFrYmaunZIyEpIS7qHCL1capqPJIXg4KvNpFs3drXa+UCFP0KY+UT0eVR/JyUODVuds5yrIar6eEt11aM9d1Oqo8kpeDsr24r5mWMRKS0t1jqKm0g1HFsbRMEmhh3fccZkfiKaE/mvuouRyKOo/k5aDAa3Knc9VptV5PibAWbpaeTkedR/JyUOC17p+exjyOlKi3K3H3Ye9cohx1HsnLQWGvxX8Xp8ga4yApUS8XpftLoMNR5YG8PBR4EffO+X1ltF9SwqfgoFb3d0LaUeWRvBwUeHF/k//HWgYtJbRTmLnZq8aVo84jeTko8Cr8OnlZK3uldPfiU66UD0edR/JyUOBVL1rFPMRRSmjXV6f0xShxJC2HBFpYy1ytb2ifCmE9yWq0vgxVHknLJtlYa+FbMFPrU9oDakr7xEulqOJYWqYJuPYZn9aqXimhB89/q8wvRpVH8nJQ4DX4f3rtxjyOlKgTznShfXqJdlR5JC8HBV54roB7T8ZISEo0+K/1PNvpqPNIXg4KvBZ3O8e01vVKCfNbrZbP+cbKUeeRvByUP360xL+6JvbTOL1UCZ+CVPtc5XRUeRwvFwVexN3Oldv3SEiVqPVr9JHuJxm1o84jeTko8Mr8OmsZS3tVifCIcR79ft+9HFUeyctBgRf301Pi1tvwekroZ7Xc6Z5B1Y46j+TloMCLu+qJsCbw2+spod8wxr051eGo8kheDgq8uK+OA+AMrl8VwuLn3DrVU1HlkbRsko21UsZDnZbWp4TpwDL4jiqHoopjaZkm4OoYzdTyPYujStS5e8qveD+b92JUeSQvBwVeAw8pkrG6V5XuVUr8Cdi+HFUeyctBgRd31FM1Ju2lQlhc38e8n9bQijqPpGWTAIt76ek+Oulb6yndH4C17eNAlaKOI2k5JsxF/Kt8McbKXlXC+JD4Nfs4GVUeyMtDgRd30lNfxspeVbo/A9dqNE9HlUfyclDgxZ30NLpxaK8qoU3v/HF4fxP7clR5JC8HBV74jnUWMsZBUsJXGVRzvuebX44qj+TloMCrcr+Tf9z4cJQSxodr0mjtdFR5JC8HZXuttIaxZa8q3fPzfY5P+6UcVR7Ly0SBV7smpZqMoZCU8DnILm3S6ajySF4OCrw6j2aIjNW9qoR+FtGs9/mWL0eVR/JyUOCF05OIX8jwekpo1yvlOU5GiSNpOSTQmhd/3Fn31qcAkjJTu58+OASfPJKVCQKpxbdGIWNdryqhPa9lYYvLl6FKI1k5JKyV01WozGXcWVK6Hz7g10xfiioP5OWhwIv/zVSx28G311PCcuc6R9mjRuWo80heDgq8Mn+utWys61UluHDfnSG+HZ88kpeDAq9yJb7sabReUsL7rqZV0umo80heDsr2WtzZnMYsjpTwGThGS/eOQS9HlcfyMlHgxeMYGsVY16tK6I1mnGK8TkeVR/JyUODFdwjdp5h9ez0lzG7N0T/fZmtHlUfyclDg1XEm6hjGLI6UMPswVusln44qj+TloMBrcNcT28gaXk8J+5RQX6WfjjqP5OWgwGteNWOjvG+uXxX+A1cppa5yKqo8kpZNAqx1lZxmN6ZwpIQlSr2PlOqhqOJIWo4Jc+G8cHy9aoyFpIQ+aZu9f/oSilHlgbw8FHgRdpvKxspeVbrvo1xTW6ejyiN5OSjwus/SMA4SkgquKxXaz/m/FFUeScsm2Vgrl96Mll5KaNJpTe6eHooqjqVlmoCLe5q5FmtVr5SwurLnssY4GVUeyctBgVe9RuaRn9V2PSW06WtSvk/vejmqPJKXgwIv3CFtVGMcJCVcWSN8T3Y6qjySl4MCr451udVa1SslXFnp2JDq2/HJI3k5KPAaWPdN1Wq/nhLGOyX3/bTZy1HlkbwcFHhNvmL+QWMeR0rYKTv3nu7Tu7SjziN5OSjwumeSm7WwV0r3zs/Uy16pqhx1HsnLQWGvmrjrie7Bt5eUcB/xkGefk60ddR7Iy0OBF/fP81rFmMeREuEgVezXWA5HnUfyclC2F76nsLl2BSrYkX0Yik8eS8siAVa+ZqFireyV0n0CTu8trbeijiNpOSbgKtcoOVmrJaSEk2YpEQbXB6PKI3k5KPCqfAl5ZGMsJCWcIjF76vsJR+2o8kheDgq82sWDwWqt7JUSMVFrq957Gb8cVR7Jy0GBV78qV6xJeymhVa+J0u57KUedR/JyUOCF7ynqJGMsJCWczFt54NPr4ajzSF4OCrx4NFNas1b2Smm3622fb/lyVHkkLwcFXtjVuedktV9PiceJ16TPgxyaUeJIWg4Ja3H7g8OAkjESkhLuot5m2+t6taLKA3l5KNtrldGtdb1SotKuXPdTCUpR0lhWJgisiLudsySj5ZISVcw+1LKfcFSGOo/k5aDAK3O3c65lzOJIicpiopxXPRx1HsnLQYEXLmF1a3WvlHC2ZS3Ux+mo80heDgq86tUqv4zVdj0lbtov4t7p7tkrR51H8nJQ4MV3SCWrI/FUCF8zps/JqS9FlUfSskmA1bFV3rCW9koJ3Xf+U/XD9SjqOJKWYwKuwX2BXKfVdj0ltOl91s/zVIpR55G8HBR4YWeuQtOYxZESelhEs9d2Oqo8kpeDAq91pVqmtbJXSlRwYGNOeZ6OKo/k5aBsr1Vrs1b2SolKxXLLMtrpqPJYXiYKe/V0zcoDP2MWR0roM3D8eVJIO6o8kJeHAq97B5I1jLGQlOCCLul9Iujh+OSRvBwUeOWr196txb1SutupQnW399pR5ZG8HBR4Fe56DqyV//Z6Svgc/AyvD0eVR/JyUOBV+Ven+RWtlNCu49SgPTehHVUeyctBgVfjq5nWx+NTwZhnjEF7/aVWVHkkLZsEWJ17nqs2q/F6SoTV4jPPPZGjFFUcScsxAde4qCVqxlhISpTxYDaVPXRUjDqP5OWgwGty7zxNa3WvlChjuFj7XjChHVUeyctB2V6r2bfXrwpUiMc7vXwrPnksLYsEWOviC8/VmMeREjpYdbX6cH0UdRxJyzFhrpGu0fKyVvZKifia+2ShfDDqPJCXhwIvunrjf73RdkkJ77qVet9z0NpR5ZG8HBR45au1WooxjyMlyhNPm5VSTkeVR/JyUODFLVBryVrZKyXCJhx4GOHLUeWRvBwUeNWrNP4BYxwkJcKmErOPPW7UjiqP5OWgwKtdmcc2xZjHkRLuIxqLO6Wno8ojeTko8MK3hoMbIsPrKd0jxL5mqYejziN5OSjwGtz1xAH1htdTImyAVnPpdDjqPJKXg7K9FlYgWe3XUyJsRdjnyuV0VHksLxMFXpM7n8taDP1UKK37fMvP3SWKOo+kZZMAiy+4rWUt7ZUSEf+xtCrRW1HHkbQcE+aa/NM9dTLaLikREXcf8CjVyajyQF4eCrzoap0KGW9GKRG2gWai9OWo8kheDgq88lV7TtaCHCnhPkp8tXWcjiqP5OWgwKtcpeeRjOZLSpQqZm9mzoejziN5OSjwwrn0fD3GXI6UKGHMU2k/FaodVR7Jy0GBV7uoV7LW90pptft0PVpvRh1H0nJIoNX52upYVuv1lPg3rzEo7ef25NVeeSQvB2V7rd6qtaGQlBZ2lZiN8ptRx7G0TBJoDTyTTtbaXimthS7p3NOEClHFkbQcEmjdTw1b2/Y+FUrpKmWUvduLvNYrj6RlkwBrYas8/gFD6ykt7MNe14Go00hWjghjLf67fWZrXa+UeEzN77a2lxHKi+k4kJZHAi0e8/F9Yq3qlRKz4EGN/TjVC/GJI2k5JNDK3Nlc1pK4p8IdrM5dhn3gkryUjiNZ2SCgKligVYYx9JESf+4lymmPfJ7X0mkkK0cEWPUi7ltaa3mlxE05BoPtNFTxo/WnP2e+8J/5599+Ykttvib6+dvPP/319//9+1+//+O3v/z8139/iKjS/EPHNCYep0H+nx8//vmjp3aN1w/v5Pw555XPXz9/Vr2M98p///F/+TavKwplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjI3MzQzCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE3IDAgb2JqCjw8IC9MZW5ndGggMzk1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1SS27FQAjb5xRcoNLwm895UlXdvPtva0NSqSq8iTHGMH3KkLnlS10ScYXJt16uWzymfC5bWpl5iLuLjSU+ttyX7iG2XXQusTgdR/ILMp0qRKjNqtGh+EKWhQeQTvChC8J9Of7jL4DB17ANuOE9MkGwJOYpQsZuURmaEkERYeeRFaikUJ9Zwt9R7uv3MgVqb4ylC2Mc9Am0BUJtSMQC6kAAROyUVK2QjmckE78V3WdiHGDn0bIBrhlURJZ77MeIqc6ojLxExD5PTfoolkwtVsZuUxlf/JSM1Hx0BSqpNPKU8tBVs9ALWIl5EvY5/Ej459ZsIYY6btbyieUfM8UyEs5gSzlgoZfjR+DbWXURrh25uM50gR+V1nBMtOt+yPVP/nTbWs11vHIIokDlTUHwuw6uRrHExDI+nY0peqIssBqavEYzwWEQEdb3w8gDGv1yvBA0p2sitFgim7ViRI2KbHM9vQTWTO/FOdbDE8Js753WobIzMyohgtq6hmrrQHazvvNwtp8/M+iibQplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9MZW5ndGggMjQ5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nE1RSYoDMAy75xX6QCFek7ynQ5lD5//Xyg6FOQQJr5KTlphYCw8xhB8sPfiRIXM3/Rt+otm7WXqSydn/mOciU1H4UqguYkJdiBvPoRHwPaFrElmxvfE5LKOZc74HH4W4BDOhAWN9STK5qOaVIRNODHUcDlqkwrhrYsPiWtE8jdxu+0ZmZSaEDY9kQtwYgIgg6wKyGCyUNjYTMlnOA+0NyQ1aYNepG1GLgiuU1gl0olbEqszgs+bWdjdDLfLgqH3x+mhWl2CF0Uv1WHhfhT6YqZl27pJCeuFNOyLMHgqkMjstK7V7xOpugfo/y1Lw/cn3+B2vD838XJwKZW5kc3RyZWFtCmVuZG9iagoxOSAwIG9iago8PCAvTGVuZ3RoIDk0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWNwRHAIAgE/1RBCQoK2k8mk4f2/40QMnxg5w7uhAULtnlGHwWVJl4VWAdKY9xQj0C94XItydwFD3Anf9rQVJyW03dpkUlVKdykEnn/DmcmkKh50WOd9wtj+yM8CmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0Zvcm0gL0JCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9MZW5ndGggMzkKL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic4zI0MFMwNjVVyOUyNzYCs3LALCNzIyALJItgQWQzuNIAFfMKfAplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9MZW5ndGggMzIyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRu23FMAzsNQUXMCB+Jc3jIEiRt3+bO9qpSNO8H1VeMqVcLnXJKllh8qVDdYqmfJ5mpvwO9ZDjmB7ZIbpT1pZ7GBaWiXlKHbGaLPdwCza+AJoScwvx9wjwK4BRwESgbvH3D7pZEkAaFPwU6JqrllhiAg2Lha3ZFeJW3SlYuKv4diS5BwlyMVnoUw5Fiim3wHwZLNmRWpzrclkK/259AhphhTjss4tE4HnAA0wk/mSAbM8+W+zq6kU2doY46dCAi4CbzSQBQVM4qz64Yftqu+bnmSgnODnWr6Ixvg1O5ktS3le5x8+gQd74Mzxnd45QDppQCPTdAiCH3cBGhD61z8AuA7ZJu3djSvmcZCm+BDYK9qhTHcrwYuzMVm/Y/MfoymZRbJCV9dHpDsrcoBNiHm9koVuytvs3D7N9/wFfGXtkCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0xlbmd0aCA4MyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFjLsNwDAIRHumYAR+JvY+UZTC3r8NECVuuCfdPVwdCZkpbjPDQwaeDCyGXXGB9JYwC1xHUI6d7KNh1b7qBI31plLz7w+Unuys4obrAQJCGmYKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvTGVuZ3RoIDMyMCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDM0MCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UjluBDEM6/0KfSCAbtvv2SBIkfy/DanZFANxdFKUO1pUdsuHhVS17HT5tJXaEjfkd2WFxAnJqxLtUoZIqLxWIdXvmTKvtzVnBMhSpcLkpORxyYI/w6WnC8f5trGv5cgdjx5YFSOhRMAyxcToGpbO7rBmW36WacCPeIScK9Ytx1gFUhvdOO2K96F5LbIGiL2ZlooKHVaJFn5B8aBHjX32GFRYINHtHElwjIlQkYB2gdpIDDl7LHZRH/QzKDET6NobRdxBgSWSmDnFunT03/jQsaD+2Iw3vzoq6VtaWWPSPhvtlMYsMul6WPR089bHgws076L859UMEjRljZLGB63aOYaimVFWeLdDkw3NMcch8w6ewxkJSvo8FL+PJRMdlMjfDg2hf18eo4ycNt4C5qI/bRUHDuKzw165gRVKF2uS9wGpTOiB6f+v8bW+19cfHe2AxgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9MZW5ndGggMjUxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nC1RSXIDQQi7zyv0hGan32OXK4fk/9cIygcGDYtAdFrioIyfICxXvOWRq2jD3zMxgt8Fh34r121Y5EBUIEljUDWhdvF69B7YcZgJzJPWsAxmrA/8jCnc6MXhMRlnt9dl1BDsXa89mUHJrFzEJRMXTNVhI2cOP5kyLrRzPTcg50ZYl2GQblYaMxKONIVIIYWqm6TOBEESjK5GjTZyFPulL490hlWNqDHscy1tX89NOGvQ7Fis8uSUHl1xLicXL6wc9PU2AxdRaazyQEjA/W4P9XOyk994S+fOFtPje83J8sJUYMWb125ANtXi37yI4/uMr+fn+fwDX2BbiAplbmRzdHJlYW0KZW5kb2JqCjI2IDAgb2JqCjw8IC9MZW5ndGggMjE1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQk1RUURWK0RlamFWdVNhbnMgL0ZpcnN0Q2hhciAwIC9MYXN0Q2hhciAyNTUKL0ZvbnREZXNjcmlwdG9yIDE0IDAgUiAvU3VidHlwZSAvVHlwZTMgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0KL0NoYXJQcm9jcyAxNiAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNvZGluZwovRGlmZmVyZW5jZXMgWyA0OCAvemVybyAvb25lIC90d28gL3RocmVlIC9mb3VyIC9maXZlIC9zaXggNTYgL2VpZ2h0IC9uaW5lIF0KPj4KL1dpZHRocyAxMyAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRm9udE5hbWUgL0JNUVFEVitEZWphVnVTYW5zIC9GbGFncyAzMgovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Bc2NlbnQgOTI5IC9EZXNjZW50IC0yMzYgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDAgL1N0ZW1WIDAgL01heFdpZHRoIDEzNDIgPj4KZW5kb2JqCjEzIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9laWdodCAxNyAwIFIgL2ZpdmUgMTggMCBSIC9mb3VyIDE5IDAgUiAvbmluZSAyMSAwIFIgL29uZSAyMiAwIFIKL3NpeCAyMyAwIFIgL3RocmVlIDI0IDAgUiAvdHdvIDI1IDAgUiAvemVybyAyNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE1IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMCAvY2EgMSA+PgovQTIgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PgovQTMgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMC41ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1EZWphVnVTYW5zLW1pbnVzIDIwIDAgUiA+PgplbmRvYmoKMiAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0tpZHMgWyAxMSAwIFIgXSAvQ291bnQgMSA+PgplbmRvYmoKMjcgMCBvYmoKPDwgL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuOS4yLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuOS4yKSAvQ3JlYXRpb25EYXRlIChEOjIwMjUwNDAzMTkzMjMwWikKPj4KZW5kb2JqCnhyZWYKMCAyOAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAzMjkyMiAwMDAwMCBuIAowMDAwMDMyNjU5IDAwMDAwIG4gCjAwMDAwMzI2OTEgMDAwMDAgbiAKMDAwMDAzMjgzMSAwMDAwMCBuIAowMDAwMDMyODUyIDAwMDAwIG4gCjAwMDAwMzI4NzMgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzM5IDAwMDAwIG4gCjAwMDAwMjc3NzkgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDI3NzU3IDAwMDAwIG4gCjAwMDAwMzE0NjggMDAwMDAgbiAKMDAwMDAzMTI2MSAwMDAwMCBuIAowMDAwMDMwODk1IDAwMDAwIG4gCjAwMDAwMzI1MjEgMDAwMDAgbiAKMDAwMDAyNzc5OSAwMDAwMCBuIAowMDAwMDI4MjY3IDAwMDAwIG4gCjAwMDAwMjg1ODkgMDAwMDAgbiAKMDAwMDAyODc1NSAwMDAwMCBuIAowMDAwMDI4OTI3IDAwMDAwIG4gCjAwMDAwMjkzMjIgMDAwMDAgbiAKMDAwMDAyOTQ3NyAwMDAwMCBuIAowMDAwMDI5ODcwIDAwMDAwIG4gCjAwMDAwMzAyODMgMDAwMDAgbiAKMDAwMDAzMDYwNyAwMDAwMCBuIAowMDAwMDMyOTgyIDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgMjggL1Jvb3QgMSAwIFIgL0luZm8gMjcgMCBSID4+CnN0YXJ0eHJlZgozMzEzMwolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:32:29.946332\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["fig, ax = plt.subplots(2, 2, figsize=(10, 6))\n", "for i in range(4):\n", " ax_sub = ax[i // 2][i % 2]\n", " ax_sub.bar(np.arange(out.shape[1], dtype=np.int32), out[i + 4, :, 0, 14, 14], **plot_args)\n", " ax_sub.set_yscale(\"log\")\n", " ax_sub.set_xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "b3297f75", "metadata": {"papermill": {"duration": 0.032328, "end_time": "2025-04-03T19:32:31.327189", "exception": false, "start_time": "2025-04-03T19:32:31.294861", "status": "completed"}, "tags": []}, "source": ["Overall we see a very diverse set of distributions, with a usual peak\n", "for 0 and close to 1. However, the distributions in the first row show a\n", "potentially undesirable behavior. For instance, the value 242 has a\n", "1000x lower likelihood than 243 although they are extremely close and\n", "can often not be distinguished. This shows that the model might have not\n", "generalized well over pixel values. The better solution to this problem\n", "is to use discrete logitics mixtures instead of a softmax distribution.\n", "A discrete logistic distribution can be imagined as discretized, binned\n", "Gaussians. Using a mixture of discrete logistics instead of a softmax\n", "introduces an inductive bias to the model to assign close-by values\n", "similar likelihoods. We can visualize a discrete logistic below:"]}, {"cell_type": "code", "execution_count": 28, "id": "50698219", "metadata": {"execution": {"iopub.execute_input": "2025-04-03T19:32:31.392122Z", "iopub.status.busy": "2025-04-03T19:32:31.391934Z", "iopub.status.idle": "2025-04-03T19:32:32.132931Z", "shell.execute_reply": "2025-04-03T19:32:32.131943Z"}, "papermill": {"duration": 0.775447, "end_time": "2025-04-03T19:32:32.134322", "exception": false, "start_time": "2025-04-03T19:32:31.358875", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNDA5LjMyNSAyMzEuNjEwNjI1IF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nNXdTXMcVbIG4H39ilqaBenz/bHElxlHsLqAIu6CmMVgDCPCPQQ2AzP//mZWd0vVlW96HNhaJDMOrBep1O8jqfucVHfV8y9f/37/6vU3L1+s//Pt8vzxrVfvlrj+zH9+WsP6M//5Y43rS/7z0xL4rdNSwqScKv/9zcPfU47UYmj81zf8bjdv/mNZflyef8EHeLcGmrG30kMdQ71RZoizhT7Wt/K5X968w/K+916W2mjwZyuJZqvyeU9LnnwjbrI3D1ni253GJbx87E223ehfV3XYnAuNNbbGtde3r9f/W/+5Pv8iSbe4fsV/fuY/Z7FH04VNe6d+e/vOyc1nXb5dvl5/vR4wUKz8Vbgec3vz5SVdfuWvUFg/D/yfWie5+ZXO4q9Oy4u75flf4xrjevfj9lW7+2H5bn0WQ/hs/dt699Xyl7vl6+1TfbqCMQcK7fA1eAg/Qc0YJ20H+JCm8SmbjkT58MW8Zp+iZ480+ofVTE9YM/EHtnHo+RB+gqIpFYof2jQ/ZdPRb0vy25+iX2/0YeXKE5bLme+i4/G+8Rp+gpo5DRrxA38u61M2VXf4J3yH/yd7jkgfWrPd1Nz/cM9EQe4oJvX9AdLtAf73M77FNPL1n/XZ/b85mvyw1yfTtNbXZ6/frL///c2/Xj+d6Ac+qv7ZH//t4HlQH3WKiQ3KjyJP+I1zOVIbVEoso+xqPmYfWbMVSi322eJo/6VpevKmM8tiKpe0a/qYfWTTwd/aubdaRs/vL1qevKisGfLMKcRd0134kVVjrJRirvz/Ov/LV7U9fdnCH99maPuuD9nHVi2BH/j5x5Qftdv7m46nb9oD5VJmHPuqj+HHduUbFltNOcz83vt5PsLT3y3FWfnfofe+L/sYfmxZOWKJYfBd03x/13T7oCYH+VwOF9u2CZyF+K5yWx2+95GN332mHsblke2teqz75fu/f3//5v63//xJ23xmSHOEPFOsnd8o551jypV3ku26dyzbVpNXaX/Y77/evP83L5cPfE/Zln7O/5HmVPsizhPv/ubtNuLN+3JwnG2b+sKtDGgqMrlTTSOmo4CVYzHPMqjpJpP4KIct2Zv35VDMtQxoKjKp8wNsPf502DkW8yyDmm4yke9ai74/MXMo5loGNBUZfhhqIev7EyvHYp5lUNNNJlLuUd+fmDkUcy0DmopM4IVhUY/Cdo7FPMugpptMoBnAusXMoZhrGdCUZTKv1FvX6xYrN8Qcy8Cmm4x8oF632DkUcy0DmorMkLmCXreYORbzLIOaikyfNIdet5g5FvMsg5puMoX3QXrdYudQzLUMaCoybfI+SK9bzByLeZZBTTeZwkfR6xY7h2KuZUBTkamTApjDmDkW8yyDmm4ymauA9YyZQzHXMqCpyJRBFcxhzByLeZZBTTeZTBnMYewcirmWAU1FJg+KYA5j5ljMswxqusnwW2AOY+dQzLUMaCoyqfM+CKxbrByLeZZBTTeZxB8I1i1mDsVcy4CmIhM7JTCHMXMs5lkGNd1kIk00nzFzKOZaBjQVmdCoozmMlWMxzzKo6SYTqaI5jJlDMdcyoCnLpNkogTmMlRtijmVg000m8j5Ir1vsHIq5lgFNRWbIc1n0usXMsZhnGdR0k+HPCuYwdg7FXMuApiLTK2UwhzFzLOZZBjXdZOSZWnrdYudQzLUMaCoyjW84mMOYORbzLIOaikyd1MAcxsyxmGcZ1HSTKVTAHMbOoZhrGdBUZMrkfRBYt1g5FvMsg5puMpn3QWDdYuZQzLUMaCoy8mIPMIcxcyzmWQY13WQyVTCHsXMo5loGNBWZNCiBOYyZYzHPMqjpJpMpgDmMnUMx1zKgqcjETgPMYcwci3mWQU03mUQVzWfMHIq5lgFNRSZ03geBdYuVYzHPMqjpJpN4HwTWLWYOxVzLgKYsw+/E+yC9brFyQ8yxDGy6yURqYA5j51DMtQxoKjKjUQFzGDPHYp5lUNNNJlIEcxg7h2KuZUBTken1+tatgJVjMc8yqOkmE6iDOYydQzHXMqCpyLTK+yC9bjFzLOZZBjXdZPgoYA5j51DMtQxoKjK18j4IrGesHIt5lkFNRaZMGmAOY+ZYzLMMarrJFKpgDmPnUMy1DGgqMnKSJTCHMXMs5lkGNd1kCkUwh7FzKOZaBjQVGTmZFpjDmDkW8yyDmm4ymfdBYN1i5lDMtQxoKjJx8D4IrFusHIt5lkFNN5nM+yCwbjFzKOZaBjQVmdBpojmMlWMxzzKo6SaTqKM5jJlDMdcyoCnLzE4FjGGM2PBy7IKKCkuiBGYwVgy1PKuAnqwyOgUwfzFijOVYBfUUlch7H71WsWKI5VkF9GQVOU8umLsYMcZyrIJ6ikrkPY9eo1gxxPKsAnqyipyaEcxbjBhjOVZBPUUl0ACzFiuGWJ5VQE9WqZUamLMYMcZyrIJ6igp/GJixWDHE8qwCesoJrCpFMF8xYozlWAX13E7RxHsb8FprHGMsxyqo53Y6A97ZgNdZGzHE8qwCem4vcOOdDXjOC44xlmMV1HN7Mjgl9HokI4ZYnlVAz+3pQRTgc11gjLEcq6Ce269FqKNJihFDLM8qoKfsmamCOQpOMZVjE1BTHoAoo3PDwBRCORYBLU9LRC9HgyFW8usBSp6Wjl63CENo5FcDdORvjoRe4KrThpCaaw9UnUE6ei20TpvJ5FgEdD8t27Uw1GpUpx06ddciqLtcAw2dY0GnEzpN3yKgu1wzDZ2PQ6U5Iqct9SuCurPIQOduUWnGTr4faVD301IyOs+PSktETsX39wjqziIDnRNKpTUhpy11LAK6n5Za0PnDVNomcmq+71lRdxaZBM41p9LZkdP0/eiLup+WVtB5CY9pLwU4nVO/Iqg7i0x0DstjOlICTufUsQjozpu8is53ekwnYpq+H3tR89MyAjoz7iFtsWmkc+jXAzVnj4rOoXxIWw0dKJ1TxyKg+2mZAZ1v+5D2MDTTOfTrgZqzR0NnZj+kI6SslS6pYxHQ/bTEEAmcxv8Yz9qnlrqkfk1ge0Fp6KoPh7inuC08Dge5xp5VQH1WiRFdJeQQ95Z7BliX2LEKqi8qDV1V5hAPbgewrrFnFVBfJtAJXYXoEM8y8tRY19ixCqovKp3AVav2caYQcx/zgLWLPauA+qySE7rK2T6WkxXVgLCusWMVVF9UOroq3j7m+pzHpLGusWcVUJ9VuB24iuI+5r+NmFtWWA+xYxVUX1QGuurmPs7cKMY6FNZD7FkF1GeVmtFVWvdxplr5m6IprIfYsQqqLyoDXdV3H2eaOcSeFNZD7FkF1GeVVtBVoPdx4W+KxAc+YO1ixyqovqhMAlcN38eVas/tvA/aHWQXe1YB9eXJBAVdZX4fd0olpdAOWLvYsQqqLyqTdzZ6vbKLt6cF85bngLWLPauA+qwyCu9n9HrlMa6FcpzjvOJ/PMg+dqyC6rPK5GOA+cpj3CL/qOSZxy3WPnasguqLSqUC5iuPcY/8o9JGzbdY+9izCqh/WlLg99TzlV08Oo2R48Fql/o1geXFpNLU05VdHGPgXWBsudxa3eSeXQAAu8TIOxu1XtnFMcvzlPqc5eC1zx27IABx4RusJyy7ODY5p0NstRy89rlnFwAgT4KMvLtRa5ZdHGeiEFJSXvvcsQsCEBcuoqcsuziFeP396Y3XTe7ZBQCwS+avup6z7OLL9wX//dbrJnfsggDEpVPWk5ZdfL0f2X4HtPfa555dAAC78BtVz1p28eVxZ8x58Nrnjl0QgLh06nrasouv65SRDl773LMLAGCXKr/90uuXx/iyqm23WrvUsQkqLyaD9zl67fIYX7Y/8UC1jz2rgPqs0jJvdPTK5THetsolxHGLtY8dq6D6ojLkPdX9ymN8HqvMUG6x9rFnFVCfVXqmCV7rs4u3Edz5abT7g+xixyqovqhMiuD1Prt4G9fm0g9Yu9izCqgvL4LinwQwbdnF22i/biO4/UF2sWMVVF9UJjUwb9nF26+BWisHrF3sWQXUZ5XJNxtMW3bx9ivDtD2ZZ3+QXexYBdU/LTnwClXPWvbx+dfLfSish9ivCqwvKpW3Nmq9so/PT0XoXWE9xJ5VQH1W4d1M1XOWfbw9bSWVqbAeYscqqL6oVOp6yrKPz09x2oa1t1gPsWcVUJ9VUqSgZyz7+Px0uPNLLW+xrrFjFVRfVBolPWHZx+enTraosa6xZxVQX164Hano+cohnmV7rDkc45I6NkHlxaTxe6rVyiEeOc+oqa6xZxVQn1VK5H2NXq3cxr2V7Ukrh4NcY8cqqL6odN7X6NXKbdz5ExSAdYk9q4D6rFITZT1dOcazbsP840EusWMVVF9UOjU9XTnGI1SEdYk9q4D6rNISDT1dOcZtzgSwLrFjFVRfVAYFcEaVQ9xqbgjrHHtWAfVZpfMeD5xV5RC3lBPCOseOVVB9URm8r9HrlWM8t1MPqYNM52ckgvXlBDyZ9zV6vXKM+3mOcDxI9z5HQPVFZfK+Rq9XjvFFBWN5VgH1WWUWSmC6YsTY0LEK6ikqkwqYrhzjy/fK8SDuv1dQ/dNSQqGupytGbBj6VYE9RWXS1NMVM4ZYnlVAT1aJlaKerlgxxnKsgnqySgq8s9FngjNijOVYBfUUlco7G302OCuGWJ5VQE85aWDgnY16rLVijOVYBfUUlUZBz1fMGGJ5VgE9WaVE+Q2gugMxYozlWAX1FJVGVc9XzBhieVYBPVmlRup6vmLFGMuxCuopKp2Cnq+YMcTyrAJ6skpLvLPRCxMjxliOVVBPUem8s9ELEyuGWJ5VQE9W6Yl3NnphYsQYy7EK6ikqnaYepJgxxPKsAnrK6Z8zRT0bsGKM5VgF9RSVQQXMV6wYYnlWAT1ZZWZqYJBixBjLsQrqKSqDBhikWDHE8qwCep6WGoo8u+34M2HEBpZfFdhTVCbfYPVYa8YQy7MK6MkqUZ53rs+4b8QYy7EK6ikqk7qeDZgxxPKsAnqySqoU9GzAijGWYxXUk1UyH0PPBqwYYzlWQT1FpVLRswEzhlieVUBPubxJoK5nA1aMsRyroJ6iUnlno9crVgyxPKuAnqxSI+9s9MLEiDGWYxXUU1Qa72z0wsSKIZZnFdCTVVqkBq4SZMQYy7EK6ikqXETPBswYYnlWAT1ZpSeKejZgxRjLsQrqKSodXfvWjCGWZxXQk1VGQldItmKM5VgF9RSVjq6kbcYQy7MK6CkXqsvomutWjLEcq6CeojJ4Z6MXJlYMsTyrgJ6npYVMRc8GjNjA8qsCe4rKoK5nA2YMsTyrgJ6sErOcW0Zd3dCIMZZjFdRTVCZFPRswY4jlWQX0ZJVUqOjZgBVjLMcqqKeoTN7ZqMdaM4ZYnlVAT1aRm61nA1aMsRyroJ5ycd3AOxv1WGvFGMuxCuopKpWyng2YMcTyrAJ6skrlzwmuxGzEGMuxCuopKpUGuBqzFUMszyqgJ6u0SEHPBqwYYzlWQT1FpVHSswEzhlieVUBPVumRdzZ6YWLEGMuxCuopKo13NnphYsUQy7MK6MkqQ87dqxcmRoyxHKugnqLSKYL5ihVDLM8qoCerzEQFDFKMGGM5VkE9RaVTA4MUK4ZYnlVAz9PSQ6KhZwNGbGD5VYE9RWVQ1LMBM4ZYnlVAT1aJmXc26rHWijGWYxXUU1QG72zUY60ZQyzPKqAnq6TMOxv1WGvFGMuxCuopKpOCng2YMcTyrAJ6sgrf7KRnA1aMsRyroJ6iMqnq2YAZQyzPKqAnq5RCXc8GrBhjOVZBPUVlXt+4VTFiiOVZBfRklVp5Z6MXJkaMsRyroJ6s0vjD9GzAijGWYxXUU1Qq72z0wsSKIZZnFdCTVXqgoWcDVoyxHKugnqLSKOrZgBlDLM8qoCerjEhZzwasGGM5VkE9RaVRBfMVK4ZYnlVAT1aZkQYYpBgxxnKsgnqKSuedjV6YWDHE8qwCep6WERLvbNRjrREbWH5VYE9R6byzUY+1ZgyxPKuAnqwSE3U9G7BijOVYBfUUlU5TzwbMGGJ5VgE9WSVlSno2YMUYy7EK6ikqg4qeDZgxxPKsAnqySs7U9GzAijGWYxXUU1QG72zUY60ZQyzPKqAnq5TCOxu9XjFijOVYBfUUlck7G70wsWKI5VkF9GSVWqjq2YAVYyzHKqinqEwuohcmVgyxPKuAnqzSKgU9G7BijOVYBfVklc7H0LMBK8ZYjlVQT1GpVPVswIwhlmcV0JNVRuCdjV6YGDHGcqyCeopK5Z2NXphYMcTyrAJ6ssqMvLPRCxMjxlgPKs+/SFI4rV+t/PHSJa4v1+dfvv79/tXrb16+WF+9W5Tc6RrxJ+Sfz8sxl2+XzDfi+L19zY7vC46qP3z/nrsD2EcNXCRQrOtPXORn/vPH+c2Xl3T5dYlrWD8P/J+ivErobMNfjuthXp2WF3fL87/yf0/r3Y98xLDe/bB8tz778v7dq7ef8ack3jFc/lmfvf7t9frml5/u3/12/2r9gf/19v77f/12/8s/P1v/tt59tfzlbvma//f/oufBugplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjQ4NjgKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTcgMCBvYmoKPDwgL0xlbmd0aCAxNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZDBEUMhCETvVrElgIBAPclkcvi//2tAk1xkHWD3qTuBkFGHM8Nn4smD07E0cG8VjGsIryP0CE0Ck8DEwZp4DAsBp2GRYy7fVZZVp5Wumo2e171jQdVplzUNbdqB8q2PP8I13qPwGuweQgexKHRuZVoLmVg8a5w7zKPM535O23c9GK2m1Kw3ctnXPTrL1FBeWvuEzmi0/SfXL7sxXh+FFDkICmVuZHN0cmVhbQplbmRvYmoKMTggMCBvYmoKPDwgL0xlbmd0aCAxNzAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZBLEsMgDEP3nEJHAP+A87TT6YLcf1vLmXSDFGPLL0RXdOyVh8fGlI33aGNPhC1c5XQaTlMZj4u7Zl2gy2Ey02+8mrnAVGGR1eyi+hi8ofOsZoevVTMxhDeZEhpgKndyD/X1pzjt25KQbFdh0J0apLMwzJH8PRBTc9BziJH8I19ya2HQmeYXFy2rGa1lTNHsYapsLQzqjUF3yvXUeq7zMBHv8wPfQT5kCmVuZHN0cmVhbQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvTGVuZ3RoIDI0NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkU1yBSEIhPeeoi/wquRXPc+kUllM7r8NzbwkK1qF5gPTAhNH8BJD7ImVEx8yfC/oMny3MjvwOtmZcE+4blzDZcMzYVvgOyrLO15Dd7ZSP52hqu8aOd4uUjV0ZWSfeqGaC8yQiK4RWXQrl3VA05TuUuEabFuCFPVKrCedoDToEcrwd5RrfHUTT6+x5FTNIVrNrRMairBseEHUySQRtQ2LJ5ZzIVH5qhurOi5gkyXi9IDcoJVmfHpSSREwg3ysyWjMAjbQk7tnF8aaSx5Fjlc0mLA7STXwgPfitr73NnGP8xf4hXff/ysOfdcCPn8AS/5dBgplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9MZW5ndGggMjMyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRSW7EMAy7+xX8wADW7rwnxaCH9v/XUsoUCEAltrglYmMjAi8x+DmI3PiSNaMmfmdyV/wsT4VHwq3gSRSBl+FedoLLG8ZlPw4zH7yXVs6kxpMMyEU2PTwRMtglEDowuwZ12Gbaib4h4bMjUs1GltPXEvTSKgTKU7bf6YISbav6c/usC2372hNOdnvqSeUTiOeWrMBl4xWTxVgGPVG5SzF9kOpsoSehvCifg2w+aohElyhn4InBwSjQDuy57WfiVSFoXd2nbWOoRkrH078NTU2SCPlECWe2NO4W/n/Pvb7X+w9OIVQRCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0xlbmd0aCAyMzEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNU85kgQhDMt5hT4wVRjbQL+np7Y22Pl/upKZTpDwIcnTEx2ZeJkjI7Bmx9taZCBm4FNMxb/2tA8TqvfgHiKUiwthhpFw1qzjbp6OF/92lc9YB+82+IpZXhDYwkzWVxZnLtsFY2mcxDnJboxdE7GNda2nU1hHMKEMhHS2w5Qgc1Sk9MmOMuboOJEnnovv9tssdjl+DusLNo0hFef4KnqCNoOi7HnvAhpyQf9d3fgeRbvoJSAbCRbWUWLunOWEX712dB61KBJzQppBLhMhzekqphCaUKyzo6BSUXCpPqforJ9/5V9cLQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjQ5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1QO45EIQzrOYUv8CTyI3AeRqstZu/frgOaKVBMfrYzJNARgUcMMZSv4yWtoK6Bv4tC8W7i64PCIKtDUiDOeg+IdOymNpETOh2cMz9hN2OOwEUxBpzpdKY9ByY5+8IKhHMbZexWSCeJqiKO6jOOKZ4qe594FiztyDZbJ5I95CDhUlKJyaWflMo/bcqUCjpm0QQsErngZBNNOMu7SVKMGZQy6h6mdiJ9rDzIozroZE3OrCOZ2dNP25n4HHC3X9pkTpXHdB7M+Jy0zoM5Fbr344k2B02N2ujs9xNpKi9Sux1anX51EpXdGOcYEpdnfxnfZP/5B/6HWiIKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDM5NSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UktuxUAI2+cUXKDS8JvPeVJV3bz7b2tDUqkqvIkxxjB9ypC55UtdEnGFybderls8pnwuW1qZeYi7i40lPrbcl+4htl10LrE4HUfyCzKdKkSozarRofhCloUHkE7woQvCfTn+4y+AwdewDbjhPTJBsCTmKULGblEZmhJBEWHnkRWopFCfWcLfUe7r9zIFam+MpQtjHPQJtAVCbUjEAupAAETslFStkI5nJBO/Fd1nYhxg59GyAa4ZVESWe+zHiKnOqIy8RMQ+T036KJZMLVbGblMZX/yUjNR8dAUqqTTylPLQVbPQC1iJeRL2OfxI+OfWbCGGOm7W8onlHzPFMhLOYEs5YKGX40fg21l1Ea4dubjOdIEfldZwTLTrfsj1T/5021rNdbxyCKJA5U1B8LsOrkaxxMQyPp2NKXqiLLAamrxGM8FhEBHW98PIAxr9crwQNKdrIrRYIpu1YkSNimxzPb0E1kzvxTnWwxPCbO+d1qGyMzMqIYLauoZq60B2s77zcLafPzPoom0KZW5kc3RyZWFtCmVuZG9iagoyNSAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNUUmKAzAMu+cV+kAhXpO8p0OZQ+f/18oOhTkECa+Sk5aYWAsPMYQfLD34kSFzN/0bfqLZu1l6ksnZ/5jnIlNR+FKoLmJCXYgbz6ER8D2haxJZsb3xOSyjmXO+Bx+FuAQzoQFjfUkyuajmlSETTgx1HA5apMK4a2LD4lrRPI3cbvtGZmUmhA2PZELcGICIIOsCshgslDY2EzJZzgPtDckNWmDXqRtRi4IrlNYJdKJWxKrM4LPm1nY3Qy3y4Kh98fpoVpdghdFL9Vh4X4U+mKmZdu6SQnrhTTsizB4KpDI7LSu1e8TqboH6P8tS8P3J9/gdrw/N/FycCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0xlbmd0aCA5NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFjcERwCAIBP9UQQkKCtpPJpOH9v+NEDJ8YOcO7oQFC7Z5Rh8FlSZeFVgHSmPcUI9AveFyLcncBQ9wJ3/a0FScltN3aZFJVSncpBJ5/w5nJpCoedFjnfcLY/sjPAplbmRzdHJlYW0KZW5kb2JqCjI3IDAgb2JqCjw8IC9MZW5ndGggMzQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEVSS25EMQjbv1NwgUjhl5DztKq6mN5/W5tM1c3gCWBseMtTpmTKsLklIyTXlE99IkOspvw0ciQipvhJCQV2lY/Ha0usjeyRqBSf2vHjsfRGptkVWvXu0aXNolHNysg5yBChnhW6snvUDtnwelxIuu+UzSEcy/9QgSxl3XIKJUFb0HfsEd8PHa6CK4JhsGsug+1lMtT/+ocWXO9992LHLoAWrOe+wQ4AqKcTtAXIGdruNiloAFW6i0nCo/J6bnaibKNV6fkcADMOMHLAiCVbHb7R3gCWfV3oRY2K/StAUVlA/MjVdsHeMclIcBbmBo69cDzFmXBLOMYCQIq94hh68CXY5i9Xroia8Al1umQvvMKe2ubnQpMId60ADl5kw62ro6iW7ek8gvZnRXJGjNSLODohklrSOYLi0qAeWuNcN7HibSOxuVff7h/hnC9c9usXS+yExAplbmRzdHJlYW0KZW5kb2JqCjI4IDAgb2JqCjw8IC9MZW5ndGggNzIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZcQL6piblCLhdIDMTKAbMMgLQlnIKIZ4CYIG0QxSAWRLGZiRlEHZwBkcvgSgMAJdsWyQplbmRzdHJlYW0KZW5kb2JqCjI5IDAgb2JqCjw8IC9MZW5ndGggNDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZclhBWLhdMLAfMAtGWcAoinsGVBgC5Zw0nCmVuZHN0cmVhbQplbmRvYmoKMzAgMCBvYmoKPDwgL0xlbmd0aCAxNjMgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA7EgMhDEN7TqEj+CMDPs9mMik2929j2GxSwNNYIIO7E4LU2oKJ6IKHtiXdBe+tBGdj/Ok2bjUS5AR1gFak42iUUn25xWmVdPFoNnMrC60THWYOepSjGaAQOhXe7aLkcqbuzvlDcPVf9b9i3TmbiYHJyh0IzepT3Pk2O6K6usn+pMfcrNd+K+xVYWlZS8sJt527ZkAJ3FM52qs9Px8KOvYKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvTGVuZ3RoIDIxOCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9ULmNBDEMy12FGljAeu2pZxaLS6b/9Ej59iLRFkVSKjWZkikvdZQlWVPeOnyWxA55huVuZDYlKkUvk7Al99AK8X2J5hT33dWWs0M0l2g5fgszKqobHdNLNppwKhO6oNzDM/oNbXQDVocesVsg0KRg17YgcscPGAzBmROLIgxKTQb/rnKPn16LGz7D8UMUkZIO5jX/WP3ycw2vU48nkW5vvuJenKkOAxEckpq8I11YsS4SEWk1QU3PwFotgLu3Xv4btCO6DED2icRxmlKOob9rcKXPL+UnU9gKZW5kc3RyZWFtCmVuZG9iagozMiAwIG9iago8PCAvTGVuZ3RoIDgzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWMuw3AMAhEe6ZgBH4m9j5RlMLevw0QJW64J909XB0JmSluM8NDBp4MLIZdcYH0ljALXEdQjp3so2HVvuoEjfWmUvPvD5Se7KzihusBAkIaZgplbmRzdHJlYW0KZW5kb2JqCjMzIDAgb2JqCjw8IC9MZW5ndGggNTEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMza0UDBQMDQwB5JGhkCWkYlCiiEXSADEzOWCCeaAWQZAGqI4B64mhyuDKw0A4bQNmAplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9MZW5ndGggMTYwIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzUgMCBvYmoKPDwgL0xlbmd0aCAzMzQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLVJLcsUgDNtzCl2gM/gH5DzpdLp4vf+2kpNFRg5g9DHlholKfFkgt6PWxLeNzECF4a+rzIXPSNvIOojLkIu4ki2Fe0Qs5DHEPMSC76vxHh75rMzJswfGL9l3Dyv21IRlIePFGdphFcdhFeRYsHUhqnt4U6TDqSTY44v/PsVzLQQtfEbQgF/kn6+O4PmSFmn3mG3TrnqwTDuqpLAcbE9zXiZfWme5Oh7PB8n2rtgRUrsCFIW5M85z4SjTVka0FnY2SGpcbG+O/VhK0IVuXEaKI5CfqSI8oKTJzCYK4o+cHnIqA2Hqmq50chtVcaeezDWbi7czSWbrvkixmcJ5XTiz/gxTZrV5J89yotSpCO+xZ0vQ0Dmunr2WWWh0mxO8pITPxk5PTr5XM+shORUJqWJaV8FpFJliCdsSX1NRU5p6Gf778u7xO37+ASxzfHMKZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvTGVuZ3RoIDMyMCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM5IDAgb2JqCjw8IC9MZW5ndGggMzQwIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSOW4EMQzr/Qp9IIBu2+/ZIEiR/L8NqdkUA3F0UpQ7WlR2y4eFVLXsdPm0ldoSN+R3ZYXECcmrEu1ShkiovFYh1e+ZMq+3NWcEyFKlwuSk5HHJgj/DpacLx/m2sa/lyB2PHlgVI6FEwDLFxOgals7usGZbfpZpwI94hJwr1i3HWAVSG9047Yr3oXktsgaIvZmWigodVokWfkHxoEeNffYYVFgg0e0cSXCMiVCRgHaB2kgMOXssdlEf9DMoMRPo2htF3EGBJZKYOcW6dPTf+NCxoP7YjDe/OirpW1pZY9I+G+2Uxiwy6XpY9HTz1seDCzTvovzn1QwSNGWNksYHrdo5hqKZUVZ4t0OTDc0xxyHzDp7DGQlK+jwUv48lEx2UyN8ODaF/Xx6jjJw23gLmoj9tFQcO4rPDXrmBFUoXa5L3AalM6IHp/6/xtb7X1x8d7YDGCmVuZHN0cmVhbQplbmRvYmoKNDAgMCBvYmoKPDwgL0xlbmd0aCAyNTEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLVFJcgNBCLvPK/SEZqffY5crh+T/1wjKBwYNi0B0WuKgjJ8gLFe85ZGraMPfMzGC3wWHfivXbVjkQFQgSWNQNaF28Xr0HthxmAnMk9awDGasD/yMKdzoxeExGWe312XUEOxdrz2ZQcmsXMQlExdM1WEjZw4/mTIutHM9NyDnRliXYZBuVhozEo40hUghhaqbpM4EQRKMrkaNNnIU+6Uvj3SGVY2oMexzLW1fz004a9DsWKzy5JQeXXEuJxcvrBz09TYDF1FprPJASMD9bg/1c7KT33hL584W0+N7zcnywlRgxZvXbkA21eLfvIjj+4yv5+f5/ANfYFuICmVuZHN0cmVhbQplbmRvYmoKNDEgMCBvYmoKPDwgL0xlbmd0aCAxNzQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTZBJDkMhDEP3nMIXqIQzwOc8v6q6aO+/rUMHdYH85CBwPDzQcSQudGTojI4rmxzjwLMgY+LROP/JuD7EMUHdoi1Yl3bH2cwSc8IyMQK2RsnZPKLAD8dcCBJklx++wCAiXY/5VvNZk/TPtzvdj7q0Zl89osCJ7AjFsAFXgP26x4FLwvle0+SXKiVjE4fygeoiUjY7oRC1VOxyqoqz3ZsrcBX0/NFD7u0FtSM83wplbmRzdHJlYW0KZW5kb2JqCjQyIDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjQzIDAgb2JqCjw8IC9MZW5ndGggODkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNU25EYAwDOs9hUfAj0i8D8dRhP1b7IQ0lk6fEcoHa+QBguGNLyH4oi8ZhLULDyr7SHTYRA1nFSQTw68s8KqcFW1zJRPZWUyjs0HL9K3tb4Meuj/djhwKCmVuZHN0cmVhbQplbmRvYmoKNDQgMCBvYmoKPDwgL0xlbmd0aCAxNDEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY/BDsMwCEPv+Qr/QKTYKaF8T6dqh+7/ryNLuwt6AmOMhdDQG6qaw4Zgm+PF0iVUa/gUxUAlN8iZYA6lpNIdR5F6YjgYXB60G47isej6EbuSZn3QxkK6JWiAe6xTadymcRPEHTUF6inqnKO8ELmfqWfYNJLdNLOSc7gNv3vPU9f/p6u8y/kFvXcu/gplbmRzdHJlYW0KZW5kb2JqCjQ1IDAgb2JqCjw8IC9MZW5ndGggMjE1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQk1RUURWK0RlamFWdVNhbnMgL0ZpcnN0Q2hhciAwIC9MYXN0Q2hhciAyNTUKL0ZvbnREZXNjcmlwdG9yIDE0IDAgUiAvU3VidHlwZSAvVHlwZTMgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0KL0NoYXJQcm9jcyAxNiAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNvZGluZwovRGlmZmVyZW5jZXMgWyAzMiAvc3BhY2UgNDYgL3BlcmlvZCA0OCAvemVybyAvb25lIC90d28gL3RocmVlIC9mb3VyIC9maXZlIC9zaXggNTYKL2VpZ2h0IDY4IC9EIDgwIC9QIDk3IC9hIC9iIC9jIC9kIC9lIDEwMyAvZyAxMDUgL2kgMTA4IC9sIDExMCAvbiAvbyAxMTQgL3IKL3MgL3QgL3UgL3YgMTIwIC94IC95IF0KPj4KL1dpZHRocyAxMyAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRm9udE5hbWUgL0JNUVFEVitEZWphVnVTYW5zIC9GbGFncyAzMgovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Bc2NlbnQgOTI5IC9EZXNjZW50IC0yMzYgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDAgL1N0ZW1WIDAgL01heFdpZHRoIDEzNDIgPj4KZW5kb2JqCjEzIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9EIDE3IDAgUiAvUCAxOCAwIFIgL2EgMTkgMCBSIC9iIDIwIDAgUiAvYyAyMSAwIFIgL2QgMjIgMCBSIC9lIDIzIDAgUgovZWlnaHQgMjQgMCBSIC9maXZlIDI1IDAgUiAvZm91ciAyNiAwIFIgL2cgMjcgMCBSIC9pIDI4IDAgUiAvbCAyOSAwIFIKL24gMzAgMCBSIC9vIDMxIDAgUiAvb25lIDMyIDAgUiAvcGVyaW9kIDMzIDAgUiAvciAzNCAwIFIgL3MgMzUgMCBSCi9zaXggMzYgMCBSIC9zcGFjZSAzNyAwIFIgL3QgMzggMCBSIC90aHJlZSAzOSAwIFIgL3R3byA0MCAwIFIgL3UgNDEgMCBSCi92IDQyIDAgUiAveCA0MyAwIFIgL3kgNDQgMCBSIC96ZXJvIDQ1IDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTUgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAwIC9jYSAxID4+Ci9BMiA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAxID4+Ci9BMyA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAwLjUgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjQ2IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjkuMikgL0NyZWF0aW9uRGF0ZSAoRDoyMDI1MDQwMzE5MzIzMlopCj4+CmVuZG9iagp4cmVmCjAgNDcKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMTUzMDQgMDAwMDAgbiAKMDAwMDAxNTA2OSAwMDAwMCBuIAowMDAwMDE1MTAxIDAwMDAwIG4gCjAwMDAwMTUyNDEgMDAwMDAgbiAKMDAwMDAxNTI2MiAwMDAwMCBuIAowMDAwMDE1MjgzIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0MSAwMDAwMCBuIAowMDAwMDA1MzA1IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwNTI4NCAwMDAwMCBuIAowMDAwMDEzNjcyIDAwMDAwIG4gCjAwMDAwMTM0NjUgMDAwMDAgbiAKMDAwMDAxMjk5NCAwMDAwMCBuIAowMDAwMDE0NzI1IDAwMDAwIG4gCjAwMDAwMDUzMjUgMDAwMDAgbiAKMDAwMDAwNTU2MiAwMDAwMCBuIAowMDAwMDA1ODA1IDAwMDAwIG4gCjAwMDAwMDYxODUgMDAwMDAgbiAKMDAwMDAwNjUwMiAwMDAwMCBuIAowMDAwMDA2ODA3IDAwMDAwIG4gCjAwMDAwMDcxMTEgMDAwMDAgbiAKMDAwMDAwNzQzMyAwMDAwMCBuIAowMDAwMDA3OTAxIDAwMDAwIG4gCjAwMDAwMDgyMjMgMDAwMDAgbiAKMDAwMDAwODM4OSAwMDAwMCBuIAowMDAwMDA4ODAzIDAwMDAwIG4gCjAwMDAwMDg5NDcgMDAwMDAgbiAKMDAwMDAwOTA2NiAwMDAwMCBuIAowMDAwMDA5MzAyIDAwMDAwIG4gCjAwMDAwMDk1OTMgMDAwMDAgbiAKMDAwMDAwOTc0OCAwMDAwMCBuIAowMDAwMDA5ODcxIDAwMDAwIG4gCjAwMDAwMTAxMDQgMDAwMDAgbiAKMDAwMDAxMDUxMSAwMDAwMCBuIAowMDAwMDEwOTA0IDAwMDAwIG4gCjAwMDAwMTA5OTQgMDAwMDAgbiAKMDAwMDAxMTIwMCAwMDAwMCBuIAowMDAwMDExNjEzIDAwMDAwIG4gCjAwMDAwMTE5MzcgMDAwMDAgbiAKMDAwMDAxMjE4NCAwMDAwMCBuIAowMDAwMDEyMzMxIDAwMDAwIG4gCjAwMDAwMTI0OTIgMDAwMDAgbiAKMDAwMDAxMjcwNiAwMDAwMCBuIAowMDAwMDE1MzY0IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgNDcgL1Jvb3QgMSAwIFIgL0luZm8gNDYgMCBSID4+CnN0YXJ0eHJlZgoxNTUxNQolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2025-04-03T19:32:31.632581\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.9.2, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["mu = Tensor([128])\n", "sigma = Tensor([2.0])\n", "\n", "\n", "def discrete_logistic(x, mu, sigma):\n", " return torch.sigmoid((x + 0.5 - mu) / sigma) - torch.sigmoid((x - 0.5 - mu) / sigma)\n", "\n", "\n", "x = torch.arange(256)\n", "p = discrete_logistic(x, mu, sigma)\n", "\n", "# Visualization\n", "plt.figure(figsize=(6, 3))\n", "plt.bar(x.numpy(), p.numpy(), **plot_args)\n", "plt.xlim(96, 160)\n", "plt.title(\"Discrete logistic distribution\")\n", "plt.xlabel(\"Pixel value\")\n", "plt.ylabel(\"Probability\")\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "f281dcfa", "metadata": {"papermill": {"duration": 0.034825, "end_time": "2025-04-03T19:32:32.205789", "exception": false, "start_time": "2025-04-03T19:32:32.170964", "status": "completed"}, "tags": []}, "source": ["Instead of the softmax, the model would output mean and standard\n", "deviations for the $K$ logistics we use in the mixture. This is one of\n", "the improvements in autoregressive models that PixelCNN++ [3] has\n", "introduced compared to the original PixelCNN."]}, {"cell_type": "markdown", "id": "7295a6fd", "metadata": {"papermill": {"duration": 0.034305, "end_time": "2025-04-03T19:32:32.274421", "exception": false, "start_time": "2025-04-03T19:32:32.240116", "status": "completed"}, "tags": []}, "source": ["## Conclusion\n", "\n", "In this tutorial, we have looked at autoregressive image modeling, and\n", "implemented the PixelCNN architecture. With the usage of masked\n", "convolutions, we are able to apply a convolutional network in which a\n", "pixel is only influenced by all its predecessors. Separating the masked\n", "convolution into a horizontal and vertical stack allowed us to remove\n", "the known blind spot on the right upper row of a pixel. In experiments,\n", "autoregressive models outperformed normalizing flows in terms of bits\n", "per dimension, but are much slower to sample from. Improvements, that we\n", "have not implemented ourselves here, are discrete logistic mixtures, a\n", "downsampling architecture, and changing the pixel order in a diagonal\n", "fashion (see PixelSNAIL). Overall, autoregressive models are another,\n", "strong family of generative models, which however are mostly used in\n", "sequence tasks because of their linear scaling in sampling time than\n", "quadratic as on images."]}, {"cell_type": "markdown", "id": "eccc8650", "metadata": {"papermill": {"duration": 0.034399, "end_time": "2025-04-03T19:32:32.343538", "exception": false, "start_time": "2025-04-03T19:32:32.309139", "status": "completed"}, "tags": []}, "source": ["## References\n", "[1] van den Oord, A., et al.\n", "\"Pixel Recurrent Neural Networks.\"\n", "arXiv preprint arXiv:1601.06759 (2016).\n", "[Link](https://arxiv.org/abs/1601.06759)\n", "\n", "[2] van den Oord, A., et al.\n", "\"Conditional Image Generation with PixelCNN Decoders.\"\n", "In Advances in Neural Information Processing Systems 29, pp.\n", "4790\u20134798 (2016).\n", "[Link](http://papers.nips.cc/paper/6527-conditional-image-generation-with-pixelcnn-decoders.pdf)\n", "\n", "[3] Salimans, Tim, et al.\n", "\"PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications.\"\n", "arXiv preprint arXiv:1701.05517 (2017).\n", "[Link](https://arxiv.org/abs/1701.05517)"]}, {"cell_type": "markdown", "id": "469bca31", "metadata": {"papermill": {"duration": 0.034259, "end_time": "2025-04-03T19:32:32.412029", "exception": false, "start_time": "2025-04-03T19:32:32.377770", "status": "completed"}, "tags": []}, "source": ["## Congratulations - Time to Join the Community!\n", "\n", "Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the Lightning\n", "movement, you can do so in the following ways!\n", "\n", "### Star [Lightning](https://github.com/Lightning-AI/lightning) on GitHub\n", "The easiest way to help our community is just by starring the GitHub repos! This helps raise awareness of the cool\n", "tools we're building.\n", "\n", "### Join our [Discord](https://discord.com/invite/tfXFetEZxv)!\n", "The best way to keep up to date on the latest advancements is to join our community! Make sure to introduce yourself\n", "and share your interests in `#general` channel\n", "\n", "\n", "### Contributions !\n", "The best way to contribute to our community is to become a code contributor! At any time you can go to\n", "[Lightning](https://github.com/Lightning-AI/lightning) or [Bolt](https://github.com/Lightning-AI/lightning-bolts)\n", "GitHub Issues page and filter for \"good first issue\".\n", "\n", "* [Lightning good first issue](https://github.com/Lightning-AI/lightning/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* [Bolt good first issue](https://github.com/Lightning-AI/lightning-bolts/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* You can also contribute your own notebooks with useful examples !\n", "\n", "### Great thanks from the entire Pytorch Lightning Team for your interest !\n", "\n", "[![Pytorch Lightning](data:image/png;base64,NDA0OiBOb3QgRm91bmQ=){height=\"60px\" width=\"240px\"}](https://pytorchlightning.ai)"]}, {"cell_type": "raw", "metadata": {"raw_mimetype": "text/restructuredtext"}, "source": [".. customcarditem::\n", " :header: Tutorial 10: Autoregressive Image Modeling\n", " :card_description: In this tutorial, we implement an autoregressive likelihood model for the task of image modeling. Autoregressive models are naturally strong generative models that constitute...\n", " :tags: Image,GPU/TPU,UvA-DL-Course\n", " :image: _static/images/course_UvA-DL/10-autoregressive-image-modeling.jpg"]}], "metadata": {"jupytext": {"cell_metadata_filter": "colab_type,id,colab,-all", "formats": "ipynb,py:percent", "main_language": "python"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12"}, "papermill": {"default_parameters": {}, "duration": 330.326727, "end_time": "2025-04-03T19:32:34.740728", "environment_variables": {}, "exception": null, "input_path": "course_UvA-DL/10-autoregressive-image-modeling/notebook.ipynb", "output_path": ".notebooks/course_UvA-DL/10-autoregressive-image-modeling.ipynb", "parameters": {}, "start_time": "2025-04-03T19:27:04.414001", "version": "2.6.0"}, "widgets": {"application/vnd.jupyter.widget-state+json": {"state": {"04cad21ed6b44dd8804ef33ada016d9c": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "0799de0e124143f0a492bd4b8280f699": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_e67d30881efb45819c7ec97cd84835cf", "placeholder": "\u200b", "style": "IPY_MODEL_fa28c20053154d909e39f63325664407", "tabbable": null, "tooltip": null, "value": "100%"}}, "0edc1039d3734abead3b3ccc305e9a7a": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_04cad21ed6b44dd8804ef33ada016d9c", "placeholder": "\u200b", "style": "IPY_MODEL_1b021482a9d6476f816ff17913045821", "tabbable": null, "tooltip": null, "value": "\u200764/64\u2007[03:40<00:00,\u2007\u20073.48s/it]"}}, "0f224aab42344cac8e7e447ee78fe422": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_20ffd8d6d33c46e8b4beba4b4e609df2", "placeholder": "\u200b", "style": "IPY_MODEL_2642f1332747406b818285a9dc74b427", "tabbable": null, "tooltip": null, "value": "\u200793%"}}, "1897878b66a94bd0b92a23ec9a7d2736": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "1b021482a9d6476f816ff17913045821": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "20ffd8d6d33c46e8b4beba4b4e609df2": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "2642f1332747406b818285a9dc74b427": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "27f2f842b1354ae385e616ed95ef96a3": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_617f716ac0a54d7b8cca51bc243c5f4d", "max": 64.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_3c8f398e8b4b4038b0c82b0bfd7b3765", "tabbable": null, "tooltip": null, "value": 64.0}}, "2d9654cea614441fb82875df3ab2a0ff": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "32d8980a75d240f0966b088728e0489e": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": null}}, "3b2e428ddfa7446d82115389e79505ea": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_a9b03abf6a1d4aa79214d4b1fb6a7c57", "placeholder": "\u200b", "style": "IPY_MODEL_9563c68850de47218db4302e0766fb90", "tabbable": null, "tooltip": null, "value": "\u200793%"}}, "3c8f398e8b4b4038b0c82b0bfd7b3765": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "4276d4c25f73489ca90c67d4afe17b81": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "43ce4daac95244ad86da23c2ae83968d": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_f7df667fae144b6f85be00056e0d8d84", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_63c65f1f24fe48f38bbf8687ebf29188", "tabbable": null, "tooltip": null, "value": 28.0}}, "47f6d2c15c0c45daa463e44d8f729d12": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "4e6fdc20c77f46ccb1659ec4d793c0f1": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": null}}, "55de96dee29b4139b173f55b1978a7c6": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_fa498a95de0843bd85566b16ac5d1751", "IPY_MODEL_43ce4daac95244ad86da23c2ae83968d", "IPY_MODEL_e35bbe730034440083ac1ad78ef38127"], "layout": "IPY_MODEL_32d8980a75d240f0966b088728e0489e", "tabbable": null, "tooltip": null}}, "563b3b343ee648e893899a3be3844e8b": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "5bd943cd155547e09be42bb9f0fa23da": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "617f716ac0a54d7b8cca51bc243c5f4d": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "629f2a6c16a9438daba275a9b2285ca0": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_2d9654cea614441fb82875df3ab2a0ff", "placeholder": "\u200b", "style": "IPY_MODEL_e18160253019442394acc26817ac4a3c", "tabbable": null, "tooltip": null, "value": "100%"}}, "63c65f1f24fe48f38bbf8687ebf29188": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "650187dab5d54cd7bfa341192f023356": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "66fe506ed4f64ae1a60594ddc9763970": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "6cfd5abd1da845c2904d9a83061635ba": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": null}}, "74a0f4fb9234459bb1d762be9325aeb8": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_0f224aab42344cac8e7e447ee78fe422", "IPY_MODEL_74d5b389bb9e42439ffe59885dd0f822", "IPY_MODEL_999237856e14485f8436e13eb4ef11b8"], "layout": "IPY_MODEL_6cfd5abd1da845c2904d9a83061635ba", "tabbable": null, "tooltip": null}}, "74d5b389bb9e42439ffe59885dd0f822": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_d3c36413750e4367a8db2ac531f0154d", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_b3c0fe7b04254341ab90519355f2fd30", "tabbable": null, "tooltip": null, "value": 28.0}}, "892ecd454d164d5ba5500b447609c0d6": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "95083c1395954b82a53ddc37f8cf88f6": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_4276d4c25f73489ca90c67d4afe17b81", "placeholder": "\u200b", "style": "IPY_MODEL_c8a15f6cc1834f4d8281d9bb189fbc60", "tabbable": null, "tooltip": null, "value": "\u200728/28\u2007[01:22<00:00,\u2007\u20072.93s/it]"}}, "9563c68850de47218db4302e0766fb90": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "999237856e14485f8436e13eb4ef11b8": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_563b3b343ee648e893899a3be3844e8b", "placeholder": "\u200b", "style": "IPY_MODEL_b49b0c4abde54a9e919b5d0ed52e514a", "tabbable": null, "tooltip": null, "value": "\u200726/28\u2007[00:01<00:00,\u200714.31it/s]"}}, "9dfb9ce840e94079b6afc17d7f525c1e": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_3b2e428ddfa7446d82115389e79505ea", "IPY_MODEL_b9e589956b08494eb287bba9b5b7c6db", "IPY_MODEL_d2822fc9b98e450fb189f3ae8042799c"], "layout": "IPY_MODEL_a2c71701afcd40f5ae25884a5a2f6715", "tabbable": null, "tooltip": null}}, "a2c71701afcd40f5ae25884a5a2f6715": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": null}}, "a9b03abf6a1d4aa79214d4b1fb6a7c57": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "b37c72cbfe504dea8ff87986a6882dc8": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_650187dab5d54cd7bfa341192f023356", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_5bd943cd155547e09be42bb9f0fa23da", "tabbable": null, "tooltip": null, "value": 28.0}}, "b3c0fe7b04254341ab90519355f2fd30": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "b49b0c4abde54a9e919b5d0ed52e514a": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "b9e589956b08494eb287bba9b5b7c6db": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_ee90b90a994b4627afdba499c98369f5", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_c5d188123e674dfb9bc1dfbcb5f1a5b7", "tabbable": null, "tooltip": null, "value": 28.0}}, "ba5576e57253404485d1be865ce8e4f1": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": null}}, "c5d188123e674dfb9bc1dfbcb5f1a5b7": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "c5fa6d3178fe4940ad9f698e0821d952": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_0799de0e124143f0a492bd4b8280f699", "IPY_MODEL_b37c72cbfe504dea8ff87986a6882dc8", "IPY_MODEL_95083c1395954b82a53ddc37f8cf88f6"], "layout": "IPY_MODEL_ba5576e57253404485d1be865ce8e4f1", "tabbable": null, "tooltip": null}}, "c8a15f6cc1834f4d8281d9bb189fbc60": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "cbe737e7f3674f02a2a179f45e74f14f": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "d2822fc9b98e450fb189f3ae8042799c": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_892ecd454d164d5ba5500b447609c0d6", "placeholder": "\u200b", "style": "IPY_MODEL_47f6d2c15c0c45daa463e44d8f729d12", "tabbable": null, "tooltip": null, "value": "\u200726/28\u2007[00:01<00:00,\u200714.34it/s]"}}, "d3c36413750e4367a8db2ac531f0154d": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "e18160253019442394acc26817ac4a3c": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "e35bbe730034440083ac1ad78ef38127": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_66fe506ed4f64ae1a60594ddc9763970", "placeholder": "\u200b", "style": "IPY_MODEL_e750c2cc5bc0487e87e3eff1e03e1e8a", "tabbable": null, "tooltip": null, "value": "\u200728/28\u2007[00:02<00:00,\u200711.47it/s]"}}, "e67d30881efb45819c7ec97cd84835cf": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "e750c2cc5bc0487e87e3eff1e03e1e8a": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "ee90b90a994b4627afdba499c98369f5": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "ee9dd5ee3bd84912a6cdd48f511d3bc4": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_629f2a6c16a9438daba275a9b2285ca0", "IPY_MODEL_27f2f842b1354ae385e616ed95ef96a3", "IPY_MODEL_0edc1039d3734abead3b3ccc305e9a7a"], "layout": "IPY_MODEL_4e6fdc20c77f46ccb1659ec4d793c0f1", "tabbable": null, "tooltip": null}}, "f7df667fae144b6f85be00056e0d8d84": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "fa28c20053154d909e39f63325664407": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "fa498a95de0843bd85566b16ac5d1751": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_cbe737e7f3674f02a2a179f45e74f14f", "placeholder": "\u200b", "style": "IPY_MODEL_1897878b66a94bd0b92a23ec9a7d2736", "tabbable": null, "tooltip": null, "value": "100%"}}}, "version_major": 2, "version_minor": 0}}}, "nbformat": 4, "nbformat_minor": 5}