{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Multiclass extension\n", "\n", "In the previous notebooks, most metrics only consider binary classification except $\\text{ECE}_{top}$ and $\\text{MCE}_{top}$. To handle multi-class classification, the most common way is to convert it to a series of binary classification problems by using 1-vs-rest approach. Alternativly, if the user only care about whether the predicted class of each sample is having the correct predicted probability, the user transform the data to a top-class binary classification problem. In top-class problem, the class 0 probability is the $1-p_{top}$ and class 1 probability is $p_{top}$ and the label is whether true label is equal to the top-class prediction. In `calzone`, user can transform the data using `data_loader.transform_topclass()` function. Most metrics will still works but the interpretation of the metrics will be different." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from calzone.utils import data_loader,reliability_diagram\n", "from calzone.vis import plot_reliability_diagram\n", "from calzone.metrics import CalibrationMetrics\n", "\n", "dataset = data_loader('../../../example_data/simulated_welldata.csv')\n", "cal_metrics = CalibrationMetrics(class_to_calculate=1)\n", "### Transform to top-class \n", "transformed_data = dataset.transform_topclass()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAIhCAYAAACot7njAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAACd2ElEQVR4nOzdd1hT5/s/8HfCCBsZgogDtWoVXIgDFyKi4N4DFwpV667b2rqt22q14sQtjtb1sY4iVeteFSvuulAElS074/z+8Ee+RkCJBgLk/bourzbPec4598lJwp0nzxAJgiCAiIiIiKiEE2s7ACIiIiKiwsDEl4iIiIh0AhNfIiIiItIJTHyJiIiISCcw8SUiIiIincDEl4iIiIh0AhNfIiIiItIJTHyJiIiISCcw8SUiIiIincDElwqVSCTK17/Tp09rO1QAgJOTE/z9/bUdhsZt2bIFIpEIT58+VZb5+/vDycnps47XsmVLuLi45KuuSCTCrFmzlI9Pnz6d457PmjULIpFIZb81a9Zgy5YtnxWfup4+fQqRSKRyvtxi0jV79uyBs7MzjI2NIRKJEB4eru2QPpum3wOfK7+vfycnJ3To0KFQYkpLS8OsWbMK5HM4t+tVx5e8D3ft2oUVK1Z81r5UcuhrOwDSLRcvXlR5PHfuXJw6dQp//fWXSnnNmjULMywC8OOPP2Ls2LEFfp6LFy+iXLlyH60TGBgIHx8flbI1a9bA1tZWa19EcotJl7x58wYDBgyAj48P1qxZA4lEgmrVqmk7rBJJ26+1tLQ0zJ49G8C7L7Ulxa5duxAREYFx48ZpOxTSIia+VKgaN26s8rh06dIQi8U5ykk9UqkUIpEI+vqf/5auUqWKBiPKW37udbly5T6ZHBc2bcUkl8shk8kgkUgK/dzve/DgAaRSKfr37w8PDw+NHDMtLQ0mJiYaOVZJounXmiY+H4hKCnZ1oCInPj4eI0aMgKOjIwwNDVG5cmVMnz4dmZmZKvVEIhFGjRqFdevWoVq1apBIJKhZsyZ2796dr/NkZmZizpw5qFGjBoyMjGBjYwNPT09cuHAhz30yMjIwYcIE1K1bF5aWlrC2toa7uzsOHTqUo+6+ffvQqFEjWFpawsTEBJUrV8aQIUOU2xUKBebNm4fq1avD2NgYpUqVQu3atbFy5cqPxp39U+H27dsxYcIEODo6QiKR4L///gMAnDx5El5eXrCwsICJiQmaNm2KsLCwTz4fuf3M++uvv6JFixaws7ODqakpatWqhcWLF0MqleZ6jLNnz6Jx48YwNjaGo6MjfvzxR8jlcpU6H3Z1yM2HP2c6OTnh9u3bOHPmjLI7jJOTE1JSUlCqVCkMGzYsxzGePn0KPT09LFmy5KPnevnyJXr16gVzc3NYWlqid+/eiImJ+WRMwLuf/tu0aQMHBwcYGxujRo0amDp1KlJTU3Psv2HDBpXX6a5du3I859ldLBYvXox58+ahUqVKkEgkOHXqlFqvvez3xubNm5WvLzc3N1y6dAmCIGDJkiWoVKkSzMzM0KpVK+VrJy/+/v5o1qwZAKB3794QiUQqLYGHDx+Gu7s7TExMYG5uDm9v7xy/7mQ/f//88w969OgBKyurPL9sJScnQ19fX+XexcbGQiwWw9LSEjKZTFk+ZswYlC5dGoIgKMs+9z2gjuPHj8PLy0v5/q5RowYWLFig3H7t2jX06dMHTk5OMDY2hpOTE/r27Ytnz5598tgf+zn/wIEDqF27NoyMjFC5cmX88ssvKts/9vnw5s0bjBgxAjVr1oSZmRns7OzQqlUrnD17Vrn/06dPUbp0aQDA7Nmzle+3939pefjwIfz8/GBnZweJRIIaNWrg119/zRHrvXv34OPjAxMTE9ja2mL48OF4+/btJ68/2x9//IG6detCIpGgUqVKWLp0aa718vM51bJlS/zxxx949uyZSre6bLNnz0ajRo1gbW0NCwsLuLq6YtOmTSqvKyoZ+PWPipSMjAx4enri0aNHmD17NmrXro2zZ89iwYIFCA8Pxx9//KFS//Dhwzh16hTmzJkDU1NTrFmzBn379oW+vj569OiR53lkMhl8fX1x9uxZjBs3Dq1atYJMJsOlS5cQGRmJJk2a5LpfZmYm4uPjMXHiRDg6OiIrKwsnT55Et27dsHnzZgwcOBDAu5/ze/fujd69e2PWrFkwMjLCs2fPVLp0LF68GLNmzcIPP/yAFi1aQCqV4t69e0hMTMzXczVt2jS4u7tj7dq1EIvFsLOzw44dOzBw4EB07twZW7duhYGBAdatW4e2bdvixIkT8PLyytexsz169Ah+fn6oVKkSDA0NcfPmTcyfPx/37t1DcHCwSt2YmBj06dMHU6dOxZw5c/DHH39g3rx5SEhIwOrVq9U674cOHDiAHj16wNLSEmvWrAEASCQSmJmZYciQIVi/fj0WL14MS0tL5T5r1qyBoaGhypeND6Wnp6N169Z4+fIlFixYgGrVquGPP/5A79698xXXw4cP0a5dO4wbNw6mpqa4d+8eFi1ahCtXrqjc6/Xr12PYsGHo3r07fv75ZyQlJWH27Nk5vsxl++WXX1CtWjUsXboUFhYWqFq1ar5fe9mOHDmCGzduYOHChRCJRJgyZQrat2+PQYMG4fHjx1i9ejWSkpIwfvx4dO/eHeHh4XkmWz/++CMaNmyIkSNH4qeffoKnpycsLCwAvPv5uF+/fmjTpg1CQkKQmZmJxYsXo2XLlggLC1MmzNm6deuGPn36YPjw4bl+QQAACwsLNGjQACdPnsSkSZMAAGFhYZBIJHj79i2uXLmifI+ePHkSrVq1Usau6fdAbjZt2oRvvvkGHh4eWLt2Lezs7PDgwQNEREQo6zx9+hTVq1dHnz59YG1tjejoaAQFBaFBgwa4c+cObG1t1T5veHg4xo0bh1mzZqFMmTLYuXMnxo4di6ysLEycOFGlbm6fD2/evAEAzJw5E2XKlEFKSgoOHDigvFctW7aEg4MDjh8/Dh8fHwQEBCAwMBAAlMnwnTt30KRJE1SoUAHLli1DmTJlcOLECYwZMwaxsbGYOXMmAODVq1fw8PCAgYEB1qxZA3t7e+zcuROjRo3K17WGhYWhc+fOcHd3x+7duyGXy7F48WK8evUqR938fE6tWbMGQ4cOxaNHj3DgwIEcx3j69CmGDRuGChUqAAAuXbqE0aNHIyoqCjNmzMhXzFRMCERaNGjQIMHU1FT5eO3atQIAYe/evSr1Fi1aJAAQ/vzzT2UZAMHY2FiIiYlRlslkMuHrr78Wvvrqq4+ed9u2bQIAYcOGDR+tV7FiRWHQoEF5bpfJZIJUKhUCAgKEevXqKcuXLl0qABASExPz3LdDhw5C3bp1P3r+3Jw6dUoAILRo0UKlPDU1VbC2thY6duyoUi6Xy4U6deoIDRs2VJZt3rxZACA8efJEWTZo0CChYsWKeZ5XLpcLUqlU2LZtm6CnpyfEx8crt3l4eAgAhEOHDqns88033whisVh49uyZsgyAMHPmzBzXc+rUKWXZzJkzhQ8/npydnQUPD48ccT169EgQi8XCzz//rCxLT08XbGxshMGDB+d5PYIgCEFBQXnGDUDYvHnzR2N6n0KhEKRSqXDmzBkBgHDz5k1BEN49b2XKlBEaNWqkUv/Zs2eCgYGBynP+5MkTAYBQpUoVISsr66Ox5/XaE4R3z3GZMmWElJQUZdnBgwcFAELdunUFhUKhLF+xYoUAQPj3338/er7s+7Rv3z5lmVwuF8qWLSvUqlVLkMvlyvK3b98KdnZ2QpMmTZRl2c/fjBkzPnqebD/88INgbGwsZGRkCIIgCIGBgYKPj49Qu3ZtYfbs2YIgCEJUVJQAQFi/fr0gCAX/Hsi+NgsLC6FZs2Yqz+OnyGQyISUlRTA1NRVWrlypLM/v679ixYqCSCQSwsPDVcq9vb0FCwsLITU1VeV4H34+5BWTVCoVvLy8hK5duyrL37x5k+N9mq1t27ZCuXLlhKSkJJXyUaNGCUZGRsrPhSlTpuQZ74fXm5tGjRoJZcuWFdLT05VlycnJgrW19Uffhx/7nGrfvv0n7+/7x5gzZ45gY2Oj1n2moo9dHahI+euvv2BqapqjtTb7Z7YPf6708vKCvb298rGenh569+6N//77Dy9evMjzPMeOHYORkdFHWwPzsm/fPjRt2hRmZmbQ19eHgYEBNm3ahLt37yrrNGjQAADQq1cv7N27F1FRUTmO07BhQ9y8eRMjRozAiRMnkJycrFYc3bt3V3l84cIFxMfHY9CgQZDJZMp/CoUCPj4+uHr1ap4tbHm5ceMGOnXqBBsbG+jp6cHAwAADBw6EXC7HgwcPVOqam5ujU6dOKmV+fn5QKBT4+++/1TqvOipXrowOHTpgzZo1yp8ld+3ahbi4uE+2Lp06dSrPuPPj8ePH8PPzQ5kyZZTPT3b/1+zXw/379xETE4NevXqp7FuhQgU0bdo01+N26tQJBgYGOcrz89rL5unpCVNTU+XjGjVqAAB8fX1VWnazy/PzE/yH7t+/j5cvX2LAgAEQi//vz4mZmRm6d++OS5cuIS0tTWWfD1+3efHy8kJ6erqy69HJkyfh7e2N1q1bIzQ0VFkGAK1btwZQMO+BD124cAHJyckYMWLER2cXSElJwZQpU/DVV19BX18f+vr6MDMzQ2pqaq73Kz+cnZ1Rp04dlTI/Pz8kJyfjn3/+USnP63leu3YtXF1dYWRkpHwNhYWF5SumjIwMhIWFoWvXrjAxMVF5jtu1a4eMjAxcunQJwLv3Vl7xfkpqaiquXr2Kbt26wcjISFlubm6Ojh075qivzudUXv766y+0bt0alpaWymPMmDEDcXFxeP36db6OQcUDE18qUuLi4lCmTJkcf1Ds7Oygr6+PuLg4lfIyZcrkOEZ22Yd13/fmzRuULVtW5Y91fuzfvx+9evWCo6MjduzYgYsXL+Lq1asYMmQIMjIylPVatGiBgwcPQiaTYeDAgShXrhxcXFwQEhKirDNt2jQsXboUly5dgq+vL2xsbODl5YVr167lKxYHBweVx9k/Afbo0QMGBgYq/xYtWgRBEBAfH5/va42MjETz5s0RFRWFlStX4uzZs7h69aqyL196erpK/fe/gGTLz73QhLFjx+Lhw4fKhOjXX3+Fu7s7XF1dP7pfXFzcR+P+mJSUFDRv3hyXL1/GvHnzcPr0aVy9ehX79+8H8H/PT/a153ae3MqAnPcWyP9rL5u1tbXKY0NDw4+W53aMT8m+ttziLVu2LBQKBRISElTKc6ubmyZNmsDExAQnT57Ef//9h6dPnyoT38uXLyMlJQUnT55E5cqVUalSJQCafw/kJru7wKcGn/n5+WH16tUIDAzEiRMncOXKFVy9ehWlS5fO8d7JL3U+73J7npcvX45vv/0WjRo1wu+//45Lly7h6tWr8PHxyVdMcXFxkMlkWLVqVY7nt127dgDe9cXOrvuxeD8mISEBCoUiX/ur+zmVmytXrqBNmzYA3vXFP3/+PK5evYrp06fn+xhUfLCPLxUpNjY2uHz5MgRBUEl+X79+DZlMlqNfXG6DkLLLbGxs8jxP6dKlce7cOSgUCrWS3x07dqBSpUrYs2ePSny59dXs3LkzOnfujMzMTFy6dAkLFiyAn58fnJyc4O7uDn19fYwfPx7jx49HYmIiTp48ie+//x5t27bF8+fPPzna/cMvB9nPzapVq/KcOSGvRCs3Bw8eRGpqKvbv34+KFSsqy/OauzW3vnf5uRea0KpVK7i4uGD16tUwMzPDP//8gx07dnxyPxsbG1y5ciVHeW6vqw/99ddfePnyJU6fPq0yy8GHfbSzr/1jz8+HcmtJVOe1V1iyry06OjrHtpcvX0IsFsPKykqlPL9zsBoaGqJZs2Y4efIkypUrhzJlyqBWrVqoXLkygHeDuMLCwlTmttX0eyA32X1dP/aLUlJSEo4cOYKZM2di6tSpyvLsftqfS53Pu7xeQy1btkRQUJBKeX4HnFlZWUFPTw8DBgzAyJEjc62T/SXExsbmo/F+6jwikShf+6v7OZWb3bt3w8DAAEeOHFFpYT548GC+j0HFB1t8qUjx8vJCSkpKjg+cbdu2Kbe/LywsTCWhkMvl2LNnD6pUqfLRFhlfX19kZGSovSCCSCSCoaGhyh+VmJiYXEfWZ5NIJPDw8MCiRYsAvPtZ7kOlSpVCjx49MHLkSMTHx6tMqp9fTZs2RalSpXDnzh24ubnl+i+7dS8/sq/x/Wm0BEHAhg0bcq3/9u1bHD58WKVs165dEIvFaNGihdrX8yGJRPLRlpcxY8bgjz/+wLRp02Bvb4+ePXt+8pienp55xv0puT0/ALBu3TqVx9WrV0eZMmWwd+9elfLIyMiPziCS2/nUfe0VtOrVq8PR0RG7du1SGf2empqK33//XTnTw+dq3bo1rl+/jt9//13ZncHU1BSNGzfGqlWr8PLlS2U5oPn3QG6aNGkCS0tLrF27Ns8R/yKRCIIg5HhtbNy4MccsJ+q4ffs2bt68qVK2a9cumJubf/LXjey4Pozp33//zTEDR3adD99vJiYm8PT0xI0bN1C7du1cn9/sBNzT0zPPeD/F1NQUDRs2xP79+1V+iXj79i3+97//5bim92MG8v6cyuszJHuqNz09PWVZeno6tm/f/slYqfhhiy8VKQMHDsSvv/6KQYMG4enTp6hVqxbOnTuHn376Ce3atVP5Iwe8a+Fp1aoVfvzxR+WsDvfu3fvklGZ9+/bF5s2bMXz4cNy/fx+enp5QKBS4fPkyatSogT59+uS6X4cOHbB//36MGDECPXr0wPPnzzF37lw4ODjg4cOHynozZszAixcv4OXlhXLlyiExMRErV65U6QPasWNHuLi4wM3NDaVLl8azZ8+wYsUKVKxYEVWrVlX7uTMzM8OqVaswaNAgxMfHo0ePHsqR3Ddv3sSbN29ytPR8jLe3NwwNDdG3b19MnjwZGRkZCAoKyvHTdTYbGxt8++23iIyMRLVq1XD06FFs2LAB3377rXKk9JeoVasWdu/ejT179qBy5cowMjJCrVq1lNv79++PadOm4e+//8YPP/yQrwRn4MCB+PnnnzFw4EDMnz8fVatWxdGjR3HixIlP7tukSRNYWVlh+PDhmDlzJgwMDLBz584cf+jFYjFmz56NYcOGoUePHhgyZAgSExMxe/ZsODg45PsXh/y+9gqTWCzG4sWL0a9fP3To0AHDhg1DZmYmlixZgsTERCxcuPCLju/l5QW5XI6wsDBs3bpVWd66dWvMnDkTIpEIrVq1UpZr+j2QGzMzMyxbtgyBgYFo3bo1vvnmG9jb2+O///7DzZs3sXr1alhYWKBFixZYsmQJbG1t4eTkhDNnzmDTpk0oVarUZ5+7bNmy6NSpE2bNmgUHBwfs2LEDoaGhWLRoUb6+YHTo0AFz587FzJkz4eHhgfv372POnDmoVKmSyhRx5ubmqFixIg4dOgQvLy9YW1srr2PlypVo1qwZmjdvjm+//RZOTk54+/Yt/vvvP/zvf/9TzmYybtw4BAcHo3379pg3b55yVod79+7l61rnzp0LHx8feHt7Y8KECZDL5Vi0aBFMTU1VWs3V+ZyqVasW9u/fj6CgINSvXx9isRhubm5o3749li9fDj8/PwwdOhRxcXFYunSp1ufOpgKitWF1RELOWR0EQRDi4uKE4cOHCw4ODoK+vr5QsWJFYdq0acrR3dkACCNHjhTWrFkjVKlSRTAwMBC+/vprYefOnfk6d3p6ujBjxgyhatWqgqGhoWBjYyO0atVKuHDhgrJObrM6LFy4UHBychIkEolQo0YNYcOGDTlGYR85ckTw9fUVHB0dBUNDQ8HOzk5o166dcPbsWWWdZcuWCU2aNBFsbW0FQ0NDoUKFCkJAQIDw9OnTj8ad2+j69505c0Zo3769YG1tLRgYGAiOjo5C+/btVernd0T7//73P6FOnTqCkZGR4OjoKEyaNEk4duxYjlHZHh4egrOzs3D69GnBzc1NkEgkgoODg/D9998LUqlU5Zj4zFkdnj59KrRp00YwNzcXAOQ6Otvf31/Q19cXXrx4kfuTl4sXL14I3bt3F8zMzARzc3Ohe/fuwoULF/I1q8OFCxcEd3d3wcTERChdurQQGBgo/PPPPzn2FQRBWL9+vfDVV18JhoaGQrVq1YTg4GChc+fOKjMyZM/qsGTJklxjzc9rTxD+773xvryO/anXU37qHTx4UGjUqJFgZGQkmJqaCl5eXsL58+dV6mTH+ebNm4+e530KhUKwtbUVAAhRUVHK8vPnzwsABFdX11z30+R7IC9Hjx4VPDw8BFNTU8HExESoWbOmsGjRIuX27NeVlZWVYG5uLvj4+AgRERE5PlPUmdWhffv2wm+//SY4OzsLhoaGgpOTk7B8+XKVeh+7T5mZmcLEiRMFR0dHwcjISHB1dRUOHjyY63WfPHlSqFevniCRSAQAKjE/efJEGDJkiODo6CgYGBgIpUuXFpo0aSLMmzdP5Rh37twRvL29BSMjI8Ha2loICAgQDh06lK9ZHQRBEA4fPizUrl1b+fm4cOHCXJ+b/H5OxcfHCz169BBKlSoliEQileMEBwcL1atXFyQSiVC5cmVhwYIFwqZNm3K8Rqj4EwkCZ2em4kkkEmHkyJFfPEcslQxZWVlwcnJCs2bNcnQrKIoSExNRrVo1dOnSBevXr9d2OEREOoFdHYioWHvz5g3u37+PzZs349WrVyqDiYqKmJgYzJ8/H56enrCxscGzZ8/w888/4+3btxg7dqy2wyMi0hlMfImoWPvjjz8wePBgODg4YM2aNfka5FPYJBIJnj59ihEjRiA+Ph4mJiZo3Lgx1q5dC2dnZ22HR0SkM9jVgYiIiIh0glanM/v777/RsWNHlC1bFiKRKF9z5p05cwb169eHkZERKleujLVr1xZ8oERERERU7Gk18U1NTUWdOnXyPTjpyZMnaNeuHZo3b44bN27g+++/x5gxY/D7778XcKREREREVNwVma4OIpEIBw4cQJcuXfKsM2XKFBw+fFhlTfHhw4fj5s2bOSbgJiIiIiJ6X7Ea3Hbx4kXletrZ2rZti02bNkEqlcLAwCDHPpmZmSpLeioUCsTHx8PGxibfS2cSERERUeERBAFv375F2bJl873QT34Uq8Q3JiYmxzrr9vb2kMlkiI2NhYODQ459FixYgNmzZxdWiERERESkIc+fP0e5cuU0drxilfgCyNFKm91TI6/W22nTpmH8+PHKx0lJSahQoQIePHgAa2vrgguUigSpVIpTp07B09Mz118EqGTh/dYtvN+6hfe7ZEtKSsKECRPwzTffoFGjRoiPj0e1atVgbm6u0fMUq8S3TJkyiImJUSl7/fo19PX1YWNjk+s+Eokk1/W2ra2t89yHSg6pVAoTExPY2Njwg1IH8H7rFt5v3cL7XXJdvXoVvXv3xpMnT3Dr1i3cu3dPuU3T3VK1OquDutzd3REaGqpS9ueff8LNzY1vAiIiIqJiRBAErFixAk2bNsWTJ0/g5OSEkJAQ6OsXXLusVhPflJQUhIeHIzw8HMC76crCw8MRGRkJ4F03hYEDByrrDx8+HM+ePcP48eNx9+5dBAcHY9OmTZg4caI2wiciIiKizxAfH48uXbrgu+++g1QqRbdu3XDjxg00aNCgQM+r1a4O165dg6enp/Jxdl/cQYMGYcuWLYiOjlYmwQBQqVIlHD16FN999x1+/fVXlC1bFr/88gu6d+9e6LETERERUd4UCgVu376NhIQEWFlZwdnZGWKxGNHR0WjcuDEiIyNhaGiI5cuXY8SIEYUy25ZWE9+WLVviY9MIb9myJUeZh4cH/vnnnwKM6l3Tu0wmg1wuL9DzUMGTSqXQ19dHRkYG7+d79PT0oK+vzyn9iIhII+Lj45GQkAAAWLlyJV69eoX09HT8999/yMrKgqGhIb766isYGxvDzs4ONWrUAPAu13u/EbSgFavBbYUhKysL0dHRSEtL03YopAGCIKBMmTJ4/vw5k7wPmJiYwMHBAYaGhtoOhYiIirnjx48jJCQEAHDz5k08f/5cZfyVqakp/vvvP0ilUpQvXx41a9ZEjRo1EB0dXahxMvF9j0KhwJMnT6Cnp4eyZcvC0NCQyVIxp1AokJKSAjMzM41OgF2cCYKArKwsvHnzBk+ePEHVqlX53BAR0Rfx8fFBo0aNoFAoMGHCBKSkpCAhIQFGRkYQiUR4+/Yt5HI5rKysULduXSxbtgxisRhWVlaFGicT3/dkZWVBoVCgfPnyMDEx0XY4pAEKhQJZWVkwMjJicvceY2NjGBgY4NmzZ8rnh4iI6HNZW1vD2toat27dwvPnz1GnTh08ePAAL1++VNYxMDBAzZo18fz5c2RkZKBWrVqFHicT31wwQSJdwNc5ERFpWkJCAtLT0xEbG6uS9AKAl5cXRCKRSn/gwsbEl4iIiIg0IjY2Fo8fP4ZUKs2x7fHjxyhbtiwkEkmhd3HIxsSXiIiIiL6IIAj49ddfMWHCBJWk19jYGObm5rC0tMSDBw8QHx+PNm3awNnZWStx8rdOHfH06VOIRCLlYiGnT5+GSCRCYmKiVuMiIiKi4u3169do27YtRo8ejaysLGVrrqWlJSQSCWQyGUqVKgUrKyvExsZCJpPhyZMnePToEeLj4ws1Vrb4EhEREdFnOX78OPr27YvExESIxWLUqFEDWVlZMDMzg1wuR0pKCmQyGe7fvw8zMzOYmZkhPDwc48aNAwD07dsXfn5+hRYvE18iIiIiUktGRgYmTZqE1atXAwBEIhHWrl2LVq1aKesoFAo8fPgQSUlJsLS0zHX6TE5nVsQIgqC1xSxMTEzUmkf4+PHjmDdvHiIiIqCnpwd3d3esXLkSVapUKcAoiYiISJdERESgZ8+euHfvnrJsyJAh6N+/P4yNjVXqVq1atbDD+ygmvp+QlpYGMzMzrZw7JSUFpqam+a6fmpqK8ePHo1atWkhNTcWMGTPQtWtXZb9eIiIios+VPYBt/PjxygFsxsbGCA4ORp8+fbQcXf4w8S1BunfvrvJ406ZNsLOzw507d7SWvBMREVHx9/r1awwePBhHjx5VltWqVQu///57kWvV/Rgmvp9gYmKClJQUrZ1bHY8ePcKPP/6IS5cuITY2FgqFAgAQGRmJmjVrFkSIREREVMIdP34c/v7+ePXqFfT09CCXyzFixAgsW7as2K38ycT3E0QikVrdDbSpY8eOKF++PDZs2ICyZctCoVDAxcUFWVlZ2g6NiIiIipmMjAxMmTIFv/zyCwDA2dkZ27ZtQ1xcHLy9vbUc3edh4ltCxMXF4e7du1i3bh2aN28OADh37pyWoyIiIqLiKCIiAn369MHt27cBAN9++y2WLVuWY/BaccPEt4SwsrKCjY0N1q9fDwcHB0RGRmLq1KnaDouIiIiKkfdXYHv/F+O2bdsW+6QX4MptJYZYLMbu3btx/fp1uLi44LvvvsOSJUu0HRYREREVE69fv0bHjh2VK7ABQKlSpXDo0CF07txZy9FpBlt8S5DWrVvjzp07KmWCIOT6/y1btlR5TERERLrr+PHjGDRoEF6/fq0sa9y4MXbv3o2KFStqMTLNYosvERERkY7KyMjA2LFj4evrq5L0Tpo0CX///XeJSnoBtvgSERER6aSIiAj4+fnh1q1bAID+/fvj3LlzWL16Ndq3b6/l6AoGE18iIiIiHZI9gG3ixInIzMxE6dKlsXnzZrRv3x4ymQz6+iU3PSy5V0ZEREREKl6/fo0hQ4bgjz/+UJatWbNG2cJbkpNegIkvERERkU54fwW2bKVLl4aVlZUWoypcHNxGREREVIJlZGRg3Lhx8PX1VUl6PT09cfPmTXh5eWkxusLFxJeIiIiohIqIiEDDhg2xcuVKZZlIJMLMmTMRGhoKBwcHLUZX+NjVgYiIiKiEyR7ANmnSJGRkZMDc3Bxv375FmTJlsHPnTrRq1UrbIWoFE18iIiKiIiI+Ph4JCQlq72dlZQVra2sAOQew+fj4IDg4GCEhIejXrx/s7e01GnNxwsSXPmn9+vWYO3cuoqKisHz5cowbN65AzuPk5IRx48YV2PHVIRKJcODAAXTp0gVPnz5FpUqVcOPGDdStWxenT5+Gp6cnEhISUKpUKW2H+lHvXwcRERV9x48fR0hIiEqZQqHAtWvXAABubm4Qi3P2VO3bty/8/PxUBrCJRCLMnz8fU6ZMgVgsxvjx4wvlGooyJr4lhL+/P7Zu3Qrg3VQk5cuXR7du3TB79myYmpp+9nGTk5MxatQoLF++HN27d4elpeUXx7plyxaMGzcOiYmJX3ysDwmCgA0bNmDTpk24ffs29PX1UalSJQwcOBDDhg2DiYmJ2scsX748oqOjYWtrq/F4c8NklYhId/n4+KBRo0YqZZmZmRg1ahQAYMmSJZBIJDn2MzY2xrhx45R9eUUiEQRBwKNHj3JNlHUVE98SxMfHB5s3b4ZUKsXZs2cRGBiI1NRUBAUFqX0sQRAgl8sRGRkJqVSK9u3bF4sO8AMGDMD+/fvxww8/YPXq1bCxscGlS5ewYcMGVKpU6bOSST09PZQpU+aL4pLL5RCJRPzwISKij7K2tlZ2WciWkZGhbMSqXLkyjIyMVLZHRESga9euyhXYgHd/x319fbFw4cKCD7oY4V/hfEpNTc3zX0ZGRr7rpqen56vu55BIJChTpgzKly8PPz8/9OvXDwcPHgTw7g2wePFiVK5cGcbGxqhTpw5+++035b6nT5+GSCTCiRMn4ObmBolEgu3bt6NWrVoA3r3RRCIRnj59CgD43//+h/r168PIyAiVK1fG7NmzIZPJlMdLTEzE0KFDYW9vDyMjI7i4uODIkSM4ffo0Bg8ejKSkJIhEIohEIsyaNSvHtQwZMgQdOnRQKZPJZChTpgyCg4Nzvf69e/di586dCAkJwffff48GDRrAyckJ7dq1w8mTJ+Hp6QkAuHr1Kry9vWFrawtLS0t4eHjgn3/+yfN5ffr0KUQiEcLDw1XKz58/jzp16sDIyAiNGjVS+cDZsmULSpUqhSNHjqBmzZqQSCR49uzZJ8/t5OQEAOjatStEIpHycX6e84cPH6JFixYwMjJCzZo1ERoamuc1ERFR8ScIAlavXo0GDRrg1q1b0NPTA/CuwWbx4sU4cuRIof1aWVywxTefzMzM8tzWrl07lRVQ7OzskJaWlmtdDw8PnD59WvnYyckJsbGxOeoJgvD5wf5/xsbGkEqlAIAffvgB+/fvR1BQEKpWrYq///4b/fv3R+nSpeHh4aHcZ/LkyVi6dKnyG+XJkyfRunVrXLlyBeXLl0fp0qVx4sQJ9O/fH7/88guaN2+OR48eYejQoQCAmTNnQqFQwNfXF2/fvsWOHTtQpUoV3LlzB3p6emjSpAlWrFiBGTNm4P79+wByf24DAwPRokULREdHK1uajx49ipSUFPTq1SvX6925cyeqV6+Ozp0759gmEomU3TTevn2LQYMG4ZdffgEALFu2DO3atcPDhw9hbm6e7+d30qRJWLlyJcqUKYPvv/8enTp1woMHD2BgYAAASEtLw4IFC7Bx40bY2NjAzs4OT548+ei5r169Cjs7O2zevBk+Pj7KD7H8POfdunWDra0tLl26hOTk5CLRV5qIiL6cQqFAcnIypFIpIiIi4OrqitjYWJUBbCKRCHK5HBUqVMDu3bvh7u6u5aiLKEHHJCUlCQCE2NjYHNvS09OFO3fuCOnp6Tm2AcjzX7t27VTqmpiY5FnXw8NDpa6trW2u9dQ1aNAgoXPnzsrHly9fFmxsbIRevXoJKSkpgpGRkXDhwgWVfQICAoS+ffsKgiAIp06dEgAIBw8eVKlz48YNAYDw5MkTZVnz5s2Fn376SaXe9u3bBQcHB0EQBOHEiROCWCwW7t+/n2usmzdvFiwtLXOUV6xYUfj555+Vj2vWrCksWrRI+bhLly6Cv79/ns9BjRo1hE6dOqmUyeVyISEhQZDL5XnuJ5PJBHNzc+F///ufsgyAcODAAUEQBOHJkycCAOHGjRuCIPzfc7V7925l/bi4OMHY2FjYs2eP8hoBCOHh4XmeNz/nzpaf51xPT094/vy5cvuxY8dyPVa2j73ei6usrCzh4MGDQlZWlrZDoULA+61bdPF+z5w5UwgMDBR69eolWFhYCGZmZkKdOnWEli1bCsbGxgIAwdDQUPjpp5+EihUrCp06dRLi4uK0HbZGxMbGCgCEpKQkjR6XLb75lJKSkue27Fa5bK9fv86z7od9PLO7DmjCkSNHYGZmBplMBqlUis6dO2PVqlW4c+cOMjIy4O3trVI/KysL9erVUylzc3P75HmuX7+Oq1evYv78+coyuVyOjIwMpKWlITw8HOXKlUO1atW+6HoCAwOxfv16TJ48Ga9fv8Yff/yBsLCwPOsLggCRSPTJ475+/RozZszAX3/9hVevXkEulyMtLQ2RkZFqxff+t2lra2tUr14dd+/eVZYZGhqidu3aGjn3p57zu3fvokKFCihXrlyu8RERUfETFRWl/NVQX18fenp6iI2Nxc2bNwEApUqVwpkzZ1C7dm0MGjQIDg4O+fo7qMuY+OaTOjMjFFTdT/H09ERQUBAMDAxQtmxZ5U/uT548AQD88ccfcHR0VNnnw5Gh+YlHoVBg9uzZ6NatW45tRkZGMDY2/txLUDFw4EBMnToVFy9exMWLF+Hk5ITmzZvnWb9atWoqiWde/P398ebNG6xYsQIVK1aERCKBu7s7srKyvjjm9z9wjI2Nc3wAfe65P/WcC7l0jeGHHxFR8aVQKJCamgpbW1vExsbC2NgYWVlZkMvlyjpVqlSBi4sLAKBs2bLaCrVYYeJbgpiamuKrr77KUZ49uCoyMlKlP+/ncnV1xf3793M9FwDUrl0bL168wIMHD3Jt9TU0NFR54+bFxsYGXbp0webNm3Hx4kUMHjz4o/X9/PzQp08fHDp0KEc/X0EQkJSUBEtLS5w9exZr1qxBu3btAADPnz/PtZ/1p1y6dAkVKlQAACQkJODBgwf4+uuvP7pPfs5tYGCQ4/n51HNes2ZNREZG4uXLl8oPv4sXL6p9TUREVDTcvn0bd+/ehYuLC+7evYtXr17lqPP48WPcvn1bORCdPo2Jrw4wNzfHxIkT8d1330GhUKBZs2ZITk7GhQsXYGZmhkGDBql1vBkzZqBDhw4oX748evbsCbFYjH///Re3bt3CvHnz4OHhgRYtWqB79+5Yvnw5vvrqK9y7dw8ikQg+Pj5wcnJCSkoKwsLCUKdOHZiYmOQ5v25gYCA6dOgAuVz+yTh79eqFAwcOoG/fvvjxxx/h7e0NGxsbXL58GRs2bMDo0aPRpUsXfPXVV9i+fTvc3NyQnJyMSZMmfVYr9Zw5c2BjYwN7e3tMnz4dtra2n5wuLT/ndnJyQlhYGJo2bQqJRAIrK6tPPuetW7dG9erVMXDgQCxbtgzJycmYPn262tdERERFQ0JCAtLT0xEbG5sj6RWJRKhRowYEQfisVd50Gacz0xFz587FjBkzsGDBAtSoUQNt27bF//73P1SqVEntY7Vt2xZHjhxBaGgoGjRogMaNG2P58uWoWLGiss7vv/+OBg0aoG/fvqhZsyYmT56sbMVs0qQJhg8fjt69e6N06dJYvHhxnudq3bo1HBwc0LZt20/+jCMSibBr1y4sX74cBw4cgIeHB+rWrYtFixahU6dOaNu2LQAgODgYCQkJqFevHgYMGIAxY8bAzs5O7edh4cKFGDt2LOrXr4/o6GgcPnwYhoaGH90nP+detmwZQkNDUb58eWUf7E8952KxGAcOHEBmZiYaNmyIwMBAlf7ARERUvCQlJeHZs2d48eJFjm0VKlSAnZ2dsnGE8k8k5NY5sARLTk6GpaUlYmNjYWNjo7ItIyMDT548QaVKlXJMDk3akZaWhrJlyyI4ODjX/q2fkj0FjIWFBReP+EBJfL1LpVIcPXoU7dq1U/Zxp5KL91u36NL93rdvH4YMGYKUlBTlCmzAu3E55cqVw6NHj2Bra4s2bdpg+/btJfLvW1xcHGxtbZGUlAQLCwuNHbfkPVNUIigUCrx8+RI//vgjLC0t0alTJ22HREREVKAyMzMxevRo9OrVCykpKbC3t4cgCMougRYWFnB0dFQOeDMxMSmRSW9BYh9fKpIiIyNRqVIllCtXDlu2bIG+Pl+qRERUcj19+hS9evXC1atXAQDffPMNypQpg+joaCQnJ+P48eNIT09HUlIS2rRpAxMTkxwzNdGnMZugIsnJyUkjq9cREREVdUeOHMHAgQORkJAAfX19yGQyXLlyBVeuXIGhoSHS0tLg6+sLqVSKX375Ba6urmzp/Ux81oiIiIi0QCaTYerUqejYsSMSEhJgYGAAmUwGY2NjjB07VtmXWSwWw8LCAjY2NnBxcWHS+wXY4psLtjSSLuDrnIhIe16+fIk+ffrg7NmzAN7NTCSVSlGzZk3s3bsXzs7OWo6wZOJXhvdkf7NKS0vTciREBS/7dV7SR0cTERU1J0+eRN26dXH27Fno6ekBeNcYMXjwYFy5coVJbwFii+979PT0UKpUKbx+/RoAYGJiwmVfizmFQoGsrCxkZGTwp6H/TxAEpKWl4fXr1yhVqpTyQ5eIiAqWXC7H/PnzMWvWLAiCgFq1akFfXx8PHjxAUFAQBgwYoO0QSzwmvh8oU6YMACiTXyreBEFAeno6jI2N+SXmA6VKlVK+3omIqGC9efMG/fv3x59//gkACAgIwKpVq/DmzRukpaUpl7yPj4/PsRpbZmYmUlNTAbxbplgikeQ4vpWVFaytrQv4Koo/Jr4fEIlEcHBwgJ2dHaRSqbbDoS8klUrx999/o0WLFvxJ/z0GBgZs6SUiKiTnzp1Dnz59EBUVBbFYjDZt2mDjxo0A3q3C9r7jx48jJCQkxzFMTU0BAFOmTMn1HH379oWfn5+GIy95mPjmQU9Pj4lBCaCnpweZTAYjIyMmvkREVKgEQcCyZcswdepUyOVy6OnpQS6X4/z583j16hXs7e1z7OPj44NGjRqpfS4uXZw/THyJiIiINCwhIQH+/v44fPiwskwul6Nu3brYu3dvrkkvAFhbW7PLQgHiaB8iIiIiDbp+/Trq16+Pw4cPq4wvGTlyJC5evIiqVatqMTrdxsSXiIiISAMEQcCaNWvQpEkTPHnyBPr6+hAEAZaWlvjtt9+wevVqGBkZaTtMncauDkRERERf6O3btxg6dCh2794NAOjcuTM6dOiA9evXY/fu3ahcubKWIySAiS8RERHRF7l16xZ69uyJ+/fvQ09PD4sXL8Z3330HkUgEf39/6Osz3SoqeCeIiIioRMttbtz8yM/cuFu2bMGIESOQnp4OkUiEUqVKoV+/fsq+vUx6ixbeDSIiIirRcpsbV6FQ4Nq1awAANze3XFf3/NjcuGlpaRg9ejSCg4OVZYIgoGrVqsjKytJg9KRJTHyJiIioRMttbtzMzEyMGjUKALBkyZI8V0PLzYMHD9CjRw/cunVLpXzSpEmYP38+540vwpj4EhERUYmW29y4GRkZytXQKleunO/ZFvbt24eAgAC8ffsWIpEIgiDAxsYGW7duRfv27TUeO2kWpzMjIiIi+oTMzEyMHj0avXr1wtu3b1G6dGkIgoBmzZohPDycSW8xwcSXiIiI6COePn2K5s2bY/Xq1QCAqVOn4v79+1iyZAlOnTqFcuXKaTlCyi92dSAiIiLKw5EjRzBw4EAkJCTA0NAQv//+Ozp06AAAmDhxopajI3WxxZeIiIjoAzKZDFOmTEHHjh2VU6FlZWUhNTVVy5HRl2CLLxEREdF7oqKi0LdvX5w9e1ZZJhKJMGPGDPTo0UOLkdGXYuJLREREOkehUCA5ORlSqRQRERFwdXWFWCzGyZMn4efnhzdv3ijrlilTBjt37kSrVq20GDFpAhNfIiIi0hmzZs1CVFQUkpOTER4eDoVCgcDAQNSoUQMPHjzAP//8o1K/devW2LFjB+zt7bUUMWkSE18iIiLSGVFRUdi4cSNsbGygr68PPT09GBsb4/fff4dUKgUAdOrUCX/++Sd++OEHTJs2LddV3ah4YuJLREREOkGhUCA1NRW2traIjY2FqakpDA0NcfXqVcjlcgBA48aNceDAAbx69QoODg5ajpg0jV9hiIiISCfcvn0bd+/ehYuLC6pWrYrU1FQkJCQok15nZ2dkZGTg9u3bTHpLKCa+REREpBMSEhKQmZkJY2Nj6Onp5dguFouRmZmpnL6MSh4mvkRERKQTrKysIJFIcP/+fdy7d09lW/ny5WFjYwOJRAIrKystRUgFjX18iYiISCc4OztDJpPh8ePHyjJDQ0OUK1cOjx8/Rnp6Otq0aQNnZ2ctRkkFiYkvERER6YTFixcjIiJC+djExASmpqYoV64ckpOTERsbCxMTE87iUIIx8SUiIqIS76effsL06dMBANbW1qhXrx6uXr2K9PR0JCUloU2bNjAxMYGjo6OWI6WCxMSXiIiISqyEhAR06tQJ586dAwDMnTsXP/zwA9LS0uDr6wupVIpffvlFuXIblWxMfImIiKhEunz5Mnx8fJCYmAgAmD9/Pr7//nsA72ZwsLCwAAC4uLgw6dURvMtERERUogiCgGXLlqFJkybKpHfkyJHKpJd0F1t8iYiIqMSIi4uDv78/jhw5oiybNWsWZs6cqcWoqKhg4ktEREQlwvXr19GlSxe8ePFCWbZo0SJMnjxZi1FRUcLEl4iIiEoEGxsbxMbGKh8vXboUEyZM0GJEVNSwjy8REREVW2lpaQDe9evdsGEDMjIyAADLly9n0ks5MPElIiKiYun06dOoWrUqjhw5gu+//x4//fQTAGDFihX47rvvtBwdFUVaT3zXrFmDSpUqwcjICPXr18fZs2c/Wn/nzp2oU6cOTExM4ODggMGDByMuLq6QoiUiIiJtk8vlmDNnDry8vPDy5Ut8++23WLhwIQBg5cqVGDt2rJYjpKJKq3189+zZg3HjxmHNmjVo2rQp1q1bB19fX9y5cwcVKlTIUf/cuXMYOHAgfv75Z3Ts2BFRUVEYPnw4AgMDceDAAS1cARERERWmmJgYDB48GGFhYQAAZ2dn3L59GwCwatUqjBo1Ksc+8fHxSEhIUCnLzMxEamoqAODx48eQSCQ59rOysoK1tbWmL4G0SKuJ7/LlyxEQEIDAwEAA736aOHHiBIKCgrBgwYIc9S9dugQnJyeMGTMGAFCpUiUMGzYMixcvLtS4iYiIqPDdvHkTw4YNw6tXr2BiYoIWLVrg+PHjAIDVq1dj5MiRue53/PhxhISE5Cg3NTUFAEyZMiXX/fr27Qs/Pz8NRU9FgdYS36ysLFy/fh1Tp05VKW/Tpg0uXLiQ6z5NmjTB9OnTcfToUfj6+uL169f47bff0L59+zzPk5mZiczMTOXj5ORkAIBUKoVUKtXAlVBRln2Pea91A++3buH91i3h4eGYNWsWBEGAi4sL6tWrh+3btwN419I7dOjQPF8LXl5ecHV1VfucVlZWfH1pSUE971pLfGNjYyGXy2Fvb69Sbm9vj5iYmFz3adKkCXbu3InevXsjIyMDMpkMnTp1wqpVq/I8z4IFCzB79uwc5adOnYKJicmXXQQVG6GhodoOgQoR77du4f3WHa1btwYA6OvrK5Peb7/9FuXLl8fRo0e1GRppWPZsHZqm9Xl8RSKRymNBEHKUZbtz5w7GjBmDGTNmoG3btoiOjsakSZMwfPhwbNq0Kdd9pk2bhvHjxysfJycno3z58vD09ISNjY3mLoSKJKlUitDQUHh7e8PAwEDb4VAB4/3WLbzfJV9oaCjq1KkDOzs7SKVSyGQyhIWFISgoCAAQFBSEgIAALUdJBaGgJi7QWuJra2sLPT29HK27r1+/ztEKnG3BggVo2rQpJk2aBACoXbs2TE1N0bx5c8ybNw8ODg459pFIJLl2WDcwMOAHpQ7h/dYtvN+6hfe75JFKpfjxxx+xaNEitGnTBseOHYMgCAgODla27G7YsEE5RohKnoJ6T2ttOjNDQ0PUr18/x09UoaGhaNKkSa77pKWlQSxWDVlPTw/Au5ZiIiIiKt6eP3+Oli1bYtGiRQCAr776ClKpFGPHjsXRo0chEomwadMmJr30WbTa1WH8+PEYMGAA3Nzc4O7ujvXr1yMyMhLDhw8H8K6bQlRUFLZt2wYA6NixI7755hsEBQUpuzqMGzcODRs2RNmyZbV5KURERPSFjhw5gkGDBiE+Ph4WFhbYuHEjunfvjlGjRmHt2rUQiURYv349hgwZou1QqZjSauLbu3dvxMXFYc6cOYiOjoaLiwuOHj2KihUrAgCio6MRGRmprO/v74+3b99i9erVmDBhAkqVKoVWrVopvxUSERFR8SOVSjFt2jQsW7YMAFC/fn3s2bMHlSpVwogRI7Bu3TqIRCKMHj0agwYN0nK0VJxpfXDbiBEjMGLEiFy3bdmyJUfZ6NGjMXr06AKOioiIiPIjt8Uh8uP9xSEyMjJw6NAhAMDYsWOxaNEiGBgYYPjw4diwYYOyewMXk6AvpfXEl4iIiIqv3BaHUCgUuHbtGgDAzc0tx/gcQHVxCHNzc+zduxfPnj1Dly5doFAoMGzYMGzcuBFisRhbt25F7969OWUZfTEmvkRERPTZfHx80KhRI5WyzMxM5dLBS5YsyTG7UmZmJlauXInY2Fjlaqz16tVDvXr1oFAo8M033yA4OBhisRjbt2+Hn58fF5IgjWDiS0RERJ/N2to6RxeEjIwM5XLAlStXhpGRkXLbo0ePMHDgQFy/fh0SiQQ9evRQDlCXy+UIDAzEli1bIBaLsWPHDvTt27fwLoZKPK1NZ0ZERES6Zd++fXB1dcX169dhY2OD33//XSXpDQgIwJYtW6Cnp4ddu3Yx6SWNY+JLREREBSojIwMjRoxAr169kJycjKZNmyI8PBzt27cH8C7pHTx4MLZu3apMenv37q3lqKkkYlcHIiIiKjAymQytWrXC1atXAbybo3/OnDnQ13+Xgsjlcvj7+2PHjh3Q09PD7t270aNHD22GTCUYE18iIiIqMPr6+ujRoweePn2K7du3o23btsptMpkMgwYNwq5du6Cvr4/du3eje/fuWoyWSjp2dSAiIiKNSktLQ1pamvLxxIkTERERkSPpHThwoDLp3bt3L5NeKnBMfImIiEhj7t69ixYtWuDKlSuQyWQAALFYDDs7O2UdmUyG/v37IyQkBPr6+ti3bx+6du2qrZBJhzDxJSIiIo3YunUr3NzccPv2bUilUpVW32xSqRR+fn7Ys2cPDAwM8Ntvv6FLly6FHyzpJPbxJSIioi+SmpqKkSNHYuvWrQAAT09P6OnpqczfC/xf0vvbb7/BwMAAv//+Ozp27KiNkElHscWXiIiIPltERAQaNGiArVu3QiwWY86cOfjf//6Xa9Lbt29f/PbbbzA0NMT+/fuZ9FKhY4svERERfbYpU6bg7t27KFu2LHbt2gUPDw+kpaUhOTkZUqkUERERcHFxgZ+fHw4cOABDQ0McOHAA7dq103bopIOY+BIREdFn27hxIyZNmoSff/4Zv/76K3bs2IHk5GSEh4dDoVBgyJAhiI+PR1RUFPT09HDo0CH4+PhoO2zSUUx8iYiIKN/Cw8Nx7NgxTJs2DQDg4OCAHTt2AACioqKwceNG2NjYQF9fH3p6eoiKikJ8fDwAoHXr1kx6SauY+BIREdEnCYKAtWvX4rvvvkNmZiZq1KihMhuDQqFAamoqbG1tERsbC1NTU0ilUmRlZQEALCwsYGNjA4VCAbGYQ4xIO/jKIyIioo9KSkpC7969MWLECGRmZqJDhw5o3ry5Sp3bt2/j7t27cHFxQeXKlZGamqpMesuVKwdXV1fcuXMHt2/f1sYlEAFg4ktEREQfce3aNbi6umLfvn3Q19fHsmXLcPjwYdjY2KjUS0hIQEZGBmJiYhAZGamyrW7dujA2NkZmZiYSEhIKM3wiFezqQERERLnasGEDRo4cCalUiooVK2LPnj1o1KhRjnqCIODq1at4/PixspX3fQ8ePEDZsmUhkUhgZWVVGKET5YqJLxEREeWqdOnSkEql6NKlC4KDg3NNWi9evIgJEybg4sWLAN4tT6xQKGBqagpTU1NYWlriwYMHiI+PR5s2beDs7FzYl0GkxK4OREREpJSamqr8/y5duuD06dPYv39/jqT38ePH6NWrF5o0aYKLFy/CxMQErq6uUCgUsLGxgUQigSAIcHBwUA54MzEx4cA20iq2+BIRERVR8fHxn9Un1srKCtbW1mrtIwgCli9fjmXLluHKlSsoV64cAMDDw0OlXkJCAubPn49Vq1YhKysLIpEIQ4YMwdy5c7Fu3Tq4uroiOTkZx48fR1ZWFpKSktCmTRuYmJjA0dFR7Wsh0iQmvkREREXU8ePHERISolKmUChw7do1AICbm1uuLah9+/aFn59fvs8TFxcHf39/HDlyBACwefNm/Pjjjyp1srKyEBQUhDlz5ijn5fX29sbSpUtRu3ZtAMCsWbMAAGlpafD19YVUKsUvv/wCV1dXtvRSkcDEl4iIqIjy8fHJMZgsMzMTo0aNAgAsWbIEEokkx37qDCA7f/48+vTpgxcvXkAikeDnn3/G8OHDldsFQcCBAwcwZcoU/PfffwAAZ2dnLF26NM/FKMRiMSwsLAAALi4uTHqpyGDiS0REVERZW1vn6LKQkZEBU1NTAEDlypVhZGT0WcdWKBRYvHgxfvjhB8jlclStWhV79+5F3bp1lXWuXLmCCRMm4Ny5cwAAe3t7zJ07F4MHD4a+PlMIKn74qiUiItJBq1atUi477Ofnh7Vr18Lc3BwA8PTpU0ybNg27d+8GABgbG2PixImYNGmSsg5RccTfHoiIiHTQN998gwYNGmDjxo3YsWMHzM3NkZiYiMmTJ6N69erYvXs3RCIR/P398fDhQ8yZM4dJLxV7bPElIiLSAXK5HCEhIfDz84NYLIaJiQkuXboEsVgMqVSKtWvXYvbs2YiLiwMAtGrVCsuWLVPp+kBU3LHFl4iIqISLiYlB27ZtMWDAACxevFhZLhKJcOjQIbi4uGDMmDGIi4tDjRo1cOTIEZw8eZJJL5U4bPElIiIqwcLCwtCvXz+8evVKZS7da9euYeLEiThz5gyAd6u0zZkzB4GBgRy4RiUWX9lEREQlkFwux+zZszFv3jwIggAXFxfs3bsXpqam6N+/P3bu3AkAMDIywvjx4zFlyhTlFGREJRW7OhAREZUwL1++hJeXF+bOnQtBEBAYGIjQ0FBs27YN1apVUya9AwYMwIMHDzB//nwmvaQT2OJLRERUwrx69QoXL16EmZkZ1qxZg5SUFNSuXRtv3rwB8G4Z4mXLlqF+/fpajpSocDHxJSIiKmHq1auHbdu2ITExET/99BPu3bsHAKhevToWL16Mjh07QiQSaTlKosLHrg5ERETF3PPnz+Hl5YVr164BAG7cuIF169Zh+PDhuHfvHmxtbbF69WrcunULnTp1YtJLOostvkRERMWIQqFAcnIypFIpIiIiEB0dDX9/f8THx2Pw4MFwdXXF9u3bIQgCJBIJxo0bh2nTpsHS0lLboRNpHRNfIiKiYmDWrFmIiopCcnIywsPDIZfL4evri9jYWACAmZkZHj58iIiICADvliGeP38+nJycCjSu+Ph4JCQkqJRlZmYiNTUVAPD48WNIJJIc+1lZWcHa2rpAYyP6EBNfIiKiYiAqKgobN26EjY0NxGIxUlNTlcklAKSkpAAAmjVrhmXLlqFhw4aFEtfx48cREhKSo9zU1BQAMGXKlFz369u3L/z8/Ao0NqIPMfElIiIq4hQKBVJTU2Fra6ts4f2QmZkZtmzZgm7duhVqH14fHx80atRI7f2srKwKIBqij2PiS0REVMTdvn0bd+/ehYuLCyIjI/H48WPlNrFYjMqVK8PU1BTVqlUr9IFr1tbW7LJAxQZndSAiIiribt++jfT0dBgaGiIuLk5lm7e3N6pWrYqsrKwcfW2JSBVbfImIiIqwffv2ITAwEIaGhoiNjUVSUpLK9idPnqBs2bKQSCTsPkD0CUx8iYiIiqCMjAyMHz8eQUFBAACZTIbMzEwAgJGRESwsLGBpaYkHDx4gPj4ebdq0gbOzszZDJiry2NWBiIioiHn48CHc3d2VSW+LFi2USa+5uTlMTEwgCAIcHByUA95MTEwgFvPPOtHHsMWXiIioCAkJCcHQoUORkpICW1tbDBw4EMuXLwcAuLu7o3z58jh+/DiysrKQlJSENm3awMTEBI6OjlqOnKjoY+JLRERURCQlJWHs2LFISUlBixYtMHToUPj7+wN4Nx/uwoULkZaWBl9fX0ilUvzyyy9wdXVlSy9RPjHxJSIiKiIsLS2xfft2nDt3Dh07dkSrVq0gk8nQr18//PTTTwDeTV9mYWEBAHBxcWHSS6QGJr5ERERatG3bNpibm6Nr164AgLZt26JatWpwd3dHamoqvLy8EBwczASXSAOY+BIREWlBamoqRo0ahS1btsDS0hINGjRAuXLlEBsbCx8fH7x69Qp16tTB/v37YWhoqO1wiUoEJr5ERESFLCIiAr169cLdu3chFosxYcIEODg4IC0tDZ06dcKDBw9QoUIFHD16VNmtgYi+HBNfIiKiQiIIAoKDgzFq1ChkZGTAwcEBISEh8PDwgFwuR79+/XDx4kWUKlUKx44dQ9myZbUdMlGJwsSXiIioEMjlcgwaNAg7d+4E8K4v77Zt22BnZwdBEDBmzBgcPHgQEokEhw8fRs2aNbUcMVHJw57yREREhUBPTw/W1tbQ09PDggULcPToUdjZ2QEAFi5ciDVr1kAkEmHHjh1o3ry5lqMlKpnY4ktERFRABEFAamoqzMzMAABLlizBgAED0KBBA2Wd7du34/vvvwcArFixAj169NBKrES6gC2+REREBSApKQl9+vRB+/btIZPJAAASiUQl6Q0NDcWQIUMAABMnTsSYMWO0EiuRrmCLLxERkYZdv34dvXv3xqNHj6Cvr49Lly6hWbNmKnVu3LiBbt26QSaToU+fPli0aJGWoiXSHWzxJSIi0hBBELBq1So0adIEjx49QsWKFXH27NkcSe/Tp0/Rrl07pKSkwNPTE1u2bOECFUSFgC2+REREGpCQkICAgAAcOHAAANClSxcEBwfDyspKpV58fDx8fX0RExODWrVq4cCBA5BIJNoImUjn8OslERGRBgwYMAAHDhyAgYEBVq5cif379+dIetPT09GpUyfcu3cP5cqVw9GjR2FpaamliIl0D1t8iYiINGDRokV49uwZNm/eDDc3txzb5XI5+vfvj/Pnz6NUqVI4fvw4ypUrp4VIiXQXE18iIqLPEB8fj9OnT6Nbt24AAGdnZ9y8eTPXvrqCIGDcuHHYv38/DA0NcfDgQTg7O+frHAkJCSplmZmZSE1NBQA8fvw4124SVlZWsLa2/pzLIirRmPgSERGp6cKFC+jTpw9evnyJM2fOoGnTpgCQ5wC1JUuWYPXq1QDezdvr4eGRr/McP34cISEhOcpNTU0BAFOmTMl1v759+8LPzy9f5yDSJUx8iYiI8kmhUGDJkiWYPn065HI5qlatqkxC87Jr1y5lgrp8+XL06tUr3+fz8fFBo0aN1I7zw77FRPSO2onv6dOn0bJlywIIhYiIqOh68+YNBg0ahGPHjgF416q6bt06mJub57lPWFgY/P39AQDjx4/Hd999p9Y5ra2t2WWBSIPUntXBx8cHVapUwbx58/D8+fOCiImIiKhI+fvvv1G3bl0cO3YMRkZG2LBhA3bu3PnRpPfmzZvo2rUrpFIpevfujSVLlhRixESUG7VbfF++fIkdO3Zgy5YtmDVrFry8vBAQEIAuXbrA0NCwIGIkIiLSqNwGjX1MWFgYXr58iapVq+L3339HrVq1Plo/MjIS7dq1w9u3b+Hh4YGtW7dygQqiIkDtxNfa2hpjxozBmDFjEB4ejuDgYIwcORLffvst+vXrh4CAANSpU6cgYiUiItKI3AaNKRQKXLt2DQDg5uYGkUgEkUgE4N2sDM7Ozhg/fvwnk96EhAT4+Pjg5cuXcHZ2xsGDB7lABVER8UWD2+rWrYupU6fC2toaCxcuRHBwMNasWQN3d3esXbs2X1O1EBERFbbcBo1lZmZi1KhRAIDu3bsjKCgIGzZsUOnO8KlBYxkZGejcuTPu3r0LR0dHHDt2DKVKldJ4/ET0eT7rdxepVIrffvsN7dq1Q8WKFXHixAmsXr0ar169wpMnT1C+fHn07NlT07ESERFphLW1NapUqaLyr3LlyjAxMcGLFy8QGBiIa9euYdeuXSp1PjbQTKFQYMCAATh79iwsLCxw7NgxlC9fvhCviog+Re0W39GjRyt/Hurfvz8WL14MFxcX5XZTU1MsXLgQTk5OGguSiIiooL18+RKXLl1CXFwcACAgIACzZs3K176CIGD8+PH47bfflAtUfKpLBBEVPrUT3zt37mDVqlXo3r17noPZypYti1OnTn1xcERERIXhzz//RP/+/REXFwc9PT1s2LABgwcPzvf+y5cvx8qVKwEAW7duhaenZ0GFSkRfQO3ENyws7NMH1dfP96o0RESkeerOWpBNF5e63b59OwYOHAgAsLCwgKurK/r27Zvv/Xfv3o2JEycCAJYuXYo+ffoUSJxE9OXUTnwXLFgAe3t7DBkyRKU8ODgYb968yXP5RCIiKjz5mbUgt+m1dHGpWx8fH5QtWxbt27fHixcvoKenl+99T506pUyax44di/HjxxdUmESkAWonvuvWrcOuXbtylDs7O6NPnz5MfImIioBPzVqwZMmSXKfY0pWlbv/991/Url0bAFC6dGn8+++/MDU1VWtg9q1bt9ClSxdIpVL06NEDy5cvV05/RkRFk9qJb0xMDBwcHHKUly5dGtHR0RoJioiIvkxuS91mZGTA1NQUAFC5cmUYGRlpIzStkkql+P7777F06VJs374d/fv3BwDY2NggIyMj38d5/vw5fH19kZycjObNm2P79u1coIKoGFA78S1fvjzOnz+PSpUqqZSfP38eZcuW1VhgREREmvTs2TP06dMHly5dAgBERER81nESExPh6+uLqKgo1KxZE4cOHdLJLxFExZHaiW9gYCDGjRsHqVSKVq1aAXg34G3y5MmYMGGCxgMkIqLio6gOqjt06BD8/f2RmJgIS0tLBAcHo1u3bmofJzMzE126dMHt27dRtmxZHDt2TGe6hxCVBGonvpMnT0Z8fDxGjBiBrKwsAICRkRGmTJmCadOmaTxAIiIqPoraoLqsrCxMnjxZOdVYgwYNsGfPnhy/WuaHQqHAwIEDcebMGZibm+Po0aOoUKGCpkMmogKkduIrEomwaNEi/Pjjj7h79y6MjY1RtWpVrkNORERFblDdxYsXlUnv+PHjsWDBgjznoP+USZMmYe/evTAwMMCBAwdQp04dTYZKRIVA7cQ3m5mZGRo0aKDJWIiIqJgraoPqPDw8MH/+fNSqVQsdO3b87OP8/PPPWL58OQBgy5Yt8PLy0lSIRFSI1E58U1NTsXDhQoSFheH169dQKBQq2x8/fqyx4IiIiNSRkZGBH374ASNHjlR2Z/j++++/6Jh79+5Vzs+7aNEinZvnmKgk+azBbWfOnMGAAQPg4ODAOQuJiKhIePjwIXr37o0bN27g/PnzOH/+/BdPMZb99w4ARo0ahUmTJmkiVCLSErUT32PHjuGPP/5A06ZNNRLAmjVrsGTJEkRHR8PZ2RkrVqxA8+bN86yfmZmJOXPmYMeOHYiJiUG5cuUwffr0HCvJERGR7ggJCcHQoUORkpICW1tbzJw5U+2kV6FQIDk5GVKpFBEREZBIJOjSpQuysrLQrVs3rFixgo09RMWc2omvJqec2bNnD8aNG4c1a9agadOmWLduHXx9fXHnzp08R8r26tULr169wqZNm/DVV1/h9evXkMlkGomHiIiKl8zMTHz77bfYtGkTAKBFixbYtWsXHB0d832MWbNmISoqCsnJyQgPD1fO3vD06VOkp6ejfPny2LFjh1pLGRNR0aR24jt37lzMmDEDW7duhYmJyRedfPny5QgICEBgYCAAYMWKFThx4gSCgoKwYMGCHPWPHz+OM2fO4PHjx8rk28nJ6YtiICKi4unFixeYPHkynj17BpFIhOnTp2PmzJnQ11fvT1tUVBQ2btwIGxsb6OvrQywW49mzZ0hPTwfwboCcsbFxQVwCERUytRPfZcuW4dGjR7C3t4eTkxMMDAxUtv/zzz/5Ok5WVhauX7+OqVOnqpS3adMGFy5cyHWfw4cPw83NDYsXL8b27dthamqKTp06Ye7cuXl+KGVmZiIzM1P5ODk5GcC7ZSulUmm+YqXiK/se817rBt7vj5NKpcoByVKptNBaMAvqvJaWltDT00Pp0qWxbds2eHl5QRAEte6/QqHA27dvYWNjg7i4OJiYmKj8fbCysoJMJkNmZiaXJNYyvr91S0HdZ7UT3y5dumjkxLGxsZDL5bC3t1cpt7e3R0xMTK77PH78GOfOnYORkREOHDiA2NhYjBgxAvHx8QgODs51nwULFmD27Nk5yk+dOvXFLdZUfISGhmo7BCpEvN+5y8rKwuvXrwG8+wXtc+ez1eZ5MzMzoa+vr0yep0yZAgMDA2RmZuLo0aNqH++///7D33//DT09PRgYGCAtLU25rXTp0rCzs8Ply5exdu1a/sJYRPD9rRvefy9qktqJ78yZMzUawIcDBQRByHPwgEKhgEgkws6dO2FpaQngXXeJHj164Ndff8211XfatGnKaWiAdy2+5cuXh6enJ2xsbDR4JVQUSaVShIaGwtvbO8evE1Ty8H5/XEZGBrZs2QLg3UIThTWfrqbOe/v2bfj5+aFbt26YOXPmZ91vhUKBmzdv4tSpUzh9+jROnz6NjIyMXOs2aNAAcrkcz549Q61atT468JoKHt/fuiUuLq5AjvtZC1gkJibit99+w6NHjzBp0iRYW1vjn3/+gb29fb4HFNja2kJPTy9H6+7r169ztAJnc3BwgKOjozLpBYAaNWpAEAS8ePECVatWzbGPRCLJdZUgAwMDvnF0CO+3buH9zp1cLlf+XF+Yz9GXnlcQBGzevBmjRo1Ceno6tmzZgilTpigT6I8dUxAE/PfffwgLC0NYWBhOnTqV4w+qnp6esiEku2UaeNcaXLZsWRgZGaF06dJ8TRURfH/rhoK6x2onvv/++y9at24NS0tLPH36FN988w2sra1x4MABPHv2DNu2bcvXcQwNDVG/fn2Ehoaia9euyvLQ0FB07tw5132aNm2Kffv2ISUlBWZmZgCABw8eQCwWo1y5cupeChERFXEpKSkYPnw4du7cCeDdOJDt27fD3Nw8zz6A0dHRykQ3LCwMz58/V9luZmYGDw8PeHl5wdPTE4sWLcLJkycRGxsLU1NTmJqawtLSEg8ePEB8fDzatGkDZ2fnAr9WIip4aie+48ePh7+/PxYvXgxzc3Nlua+vr9qr2YwfPx4DBgyAm5sb3N3dsX79ekRGRmL48OEA3nVTiIqKUibTfn5+mDt3LgYPHozZs2cjNjYWkyZNwpAhQzjiloiohLl58yZ69eqlbOCYO3cupk6dmmOQWWJiIs6fP69MdO/evauy3dDQEO7u7vDy8oKXlxcaNGig0ppkZmaG2NhY2NjYQBAECIIABwcHJCQkIDY2FiYmJhzYRlRCqJ34Xr16FevWrctR7ujomOegtLz07t0bcXFxmDNnDqKjo+Hi4oKjR4+iYsWKAN59a4+MjFTWNzMzQ2hoKEaPHg03NzfY2NigV69emDdvnrqXQURERVhKSgpatWqF+Ph4ODo6IiQkRNnHNj09HefPn8eff/6JgwcP4tGjR8pZI4B3Y0dcXV2ViW6zZs0+OpjZ0dERgYGBSE5OxvHjx5GVlYWkpCS0adMGJiYmas0JTERFm9qJr5GRkXJKsPfdv38fpUuXVjuAESNGYMSIEbluyx4M8b6vv/6aIzqJiEo4MzMzLFmyBL///js2bdqEJ0+eYN68eQgLC8OFCxeQlZWlUv/rr7+Gl5cXWrVqhZYtW6q10NKsWbMAvBtF7uvrC6lUil9++QWurq5s6SUqYdROfDt37ow5c+Zg7969AN59s46MjMTUqVPRvXt3jQdIRETF24dLAeeVUP7zzz+QyWRo0KABIiIikJSUBJFIhGrVquHt27cqdcuVKwdPT09YW1tj3LhxGplqTCwWw8LCAgDg4uLCpJeoBFI78V26dCnatWsHOzs7pKenw8PDAzExMXB3d8f8+fMLIkYiIiqGclsKODAwEDVr1oSpqSkcHR0xa9YsCIKAOXPmYN68eTA0NISJiQliY2NVjmVtbQ1PT09l94WqVatCJpPh6NGj7IpARPmmduJrYWGBc+fO4a+//sI///wDhUIBV1dXtG7duiDiIyKiYurDpYD19PRgaWmJ0NBQxMbGomXLlhgwYAD279+vnKxeJpMhLS0NJiYmaN68uTLRrVu3LltgieiLfdY8vgDQqlUrtGrVSpOxEBFRCaFQKJCamgpbW1vlzAgGBgb4999/kZiYCAA4ffq0yj6VKlXCgAED0Lp1azRq1KjQVpYjIt2hduI7Z86cj26fMWPGZwdDRETFl1QqxaNHj3Dv3j2cOXMGJ06cgFQqhVgs/ujyo2XKlMHu3bvh4eFRiNESkS5SO/E9cOCAymOpVIonT55AX18fVapUYeJLRFRE5XeQ2afExcXh3r17uH//Pu7du6f8/0ePHkEul39yf1dXVzx//hxv3rxBixYtcOjQIZQqVeozroiISD1qJ743btzIUZacnAx/f3+VFdiIiKhoyO8gs/fJZDI8fvxYmdy+n+R+uOTv+0xNTfH111/D3t4e//zzD6ytrZGSkqIyJ3tKSgqqVasGIyMjrFq1ikkvERWaz+7j+z4LCwvMmTMHHTp0wIABAzRxSCIi0pBPDTLr1KkTtmzZopLk/vfff3kuCQwAFSpUQPXq1fH1118r//v111+jbNmyEIlEUCgU6N+/v/IcBgYG0NPTQ/ny5VWWAnZxcSnEZ4KIdJ1GEl/g3ZKRSUlJmjocERFpwIeDzAwNDSEWi3HlyhVkZGQAAA4fPozDhw/n2NfExATVqlXLkdxWrVoVpqamHz2vWCyGqampMumVSqWQSqWQSCQqA944UwMRFSa1E99ffvlF5bEgCIiOjsb27dvh4+OjscCIiOjL3b59G3fv3oWLiwsePnyIqKioHHUMDAzg6uqK+vXrK5Pb6tWro1y5cl+UmMpkMpiYmCgHtkkkEojFYi4FTERao3bi+/PPP6s8FovFKF26NAYNGoRp06ZpLDAiIvpyCQkJyMzMhLGxMapVq6aS+DZv3hxGRkZ48eIFFi9ejBYtWmjknAqFAgsWLMC2bdugUChQtWpVWFhYwNDQkEsBE5FWqZ34PnnypCDiICKiAmBlZQWJRIL09PQcrb2vXr1C2bJlIZFIYGVlpbFzDh48GNu2bQMADBgwAMuXL8fgwYMBcClgItIufvoQEZVgzs7OqFGjBiIiIvDw4UOYmprCzs4OVatWxYMHDxAREYGaNWvC2dlZY+ccPHgwzMzMsHnzZmzbtg1mZmYaOzYR0ZdQu8W3a9euEIlE+aq7f/9+tQMiIiLNeX+QmY2NDQRBgCAIcHBwQEJCgkYGmcnlcty+fRu1a9cGALRs2RLPnj2DtbW1pi6DiEgj1E58LS0tceDAAVhaWsLNzQ0AcP36dSQlJaFLly75ToqJiKhwODo6IjAwEMnJyTh+/DiysrKQlJSkkUFm0dHR8PPzw7Vr13D9+nVUq1YNAIp80hsfH4+EhASVsszMTKSmpgIAHj9+DIlEkmM/KyurIn9tRJQ3tRNfe3t79OrVC2vXroWenh6Ad9/2R4wYAQsLCyxZskTjQRIR0efLXpwiLS0Nvr6+kEqlGhlk9ueff6J///548+YNTE1N8eDBA2XiW9QdP34cISEhOcqzp2mbMmVKrvv17dsXfn5+BRobERUctRPf4OBgnDt3Tpn0AoCenh7Gjx+PJk2aMPElIiqixGIxLCwsAHzZIDOZTIaZM2diwYIFEAQBtWvXxp49e/D1119rMtwC5ePjg0aNGqm9nyYHARJR4VM78ZXJZLh79y6qV6+uUn737l0oFAqNBUZEREXPixcv4Ofnh7NnzwIAhg0bhp9//hnGxsZajkw91tbW7LJApIPUTnwHDx6MIUOG4L///kPjxo0BAJcuXcLChQuV09UQEVHJtGHDBpw9exbm5uZYv349+vTpo+2QiIjyTe3Ed+nSpShTpgx+/vlnREdHAwAcHBwwefJkTJgwQeMBEhFR0fHDDz8gOjoakydPxldffaXtcIiI1KJ2By+xWIzJkycjKioKiYmJSExMRFRUFCZPnqzS75eIiIq/yMhIjBw5ElKpFMC75Y3Xr1/PpJeIiqXPGtkgk8lw8uRJhISEKKcve/nyJVJSUjQaHBERac/hw4dRt25drFmzBrNnz9Z2OEREX0ztrg7Pnj2Dj48PIiMjkZmZCW9vb5ibm2Px4sXIyMjA2rVrCyJOIiIqJFlZWZgyZQpWrFgBAGjQoAECAgK0GxQRkQao3eI7duxYuLm5ISEhQWUUb9euXREWFqbR4IiIqHA9efIEzZo1Uya93333Hc6dO4dKlSppNzAiIg1Qu8X33LlzOH/+PAwNDVXKK1asiKioKI0FRkREhSs0NBQ9e/ZEUlISrKyssGXLFnTq1EnbYRERaYzaia9CoYBcLs9R/uLFC5ibm2skKCIiKnxOTk6Qy+Vwd3fH7t27UaFCBbWPwaWAiagoUzvx9fb2xooVK7B+/XoAgEgkQkpKCmbOnIl27dppPEAiIio4SUlJsLS0BABUrVoVZ86cQa1atWBgYPBZx+NSwERUlKmd+C5fvhytWrVCzZo1kZGRAT8/Pzx8+BC2tra5ftgREVHRtHv3bnz77bfYv38/PD09AQCurq5fdEwuBUxERZnaia+joyPCw8Oxe/duXL9+HQqFAgEBAejXr1+xW7KSiEgXyeVyjBo1Cps2bQIABAUFKRPfL8WlgImoKFMr8ZVKpahevTqOHDmCwYMHc4liIqJiJiUlBdevX8fbt28hEonw/fffY9asWdoOi4ioUKiV+BoYGCAzM1O5aAURERUfISEhOHv2LORyOUqXLo2dO3fC29tb22ERERUatefxHT16NBYtWgSZTFYQ8RARUQE4c+YMhgwZArlcDhsbG1y+fJlJLxHpHLX7+F6+fBlhYWH4888/UatWLeVI3Wz79+/XWHBERKQZLVq0gJ+fH65evYpq1arBwcFB2yERERU6tRPfUqVKoXv37gURCxERaYggCAgJCYGPjw+sra0hEomwceNG9OrVS9uhERFpTb4T37/++gstWrTA5s2bCzIeIiL6QikpKRgxYgS2b9+Ozp0748CBAxCJRByfQUQ6L999fL29vREfH6983LhxYy5RTERUxPz7779wc3PD9u3bIRaL0bBhQwiCoO2wiIiKhHy3+H74wXn79m1kZmZqPCAiIlKfIAjYsGEDxo4di4yMDDg6OiIkJATNmzfXdmhEREWG2n18iYioaElOTsawYcOwe/duAICvry+2bdsGW1tbLUdGRFS05Lurw4f9w9hfjIioaJDJZLhw4QL09PSwePFiHDlyhEkvEVEu1Orq4OXlBX39d7ukpaWhY8eOMDQ0VKn3zz//aDZCIiLKIbv7mUgkgrW1Nfbt2we5XA53d3ctR0ZEVHTlO/GdOXOmyuPOnTtrPBgiIvq0xMREBAYGwsfHB4GBgQCAhg0bqtSJj49HQkKCSllmZiZSU1MBAI8fP4ZEIslxbCsrK1hbWxdQ5ERE2vXZiS8RERW+q1evonfv3njy5AlCQ0PRo0cPlCpVKke948ePIyQkJEd59qJDU6ZMyfX4ffv2hZ+fn0ZjJiIqKji4jYioGBAEAStXrsTkyZMhlUrh5OSEPXv25Jr0AoCPjw8aNWqk9nmsrKy+MFIioqKLiS8RUREXHx+PwYMH4/DhwwCAbt26YdOmTXkmvQBgbW3NLgtERB9g4ktEVISlpaXBzc0NT548gaGhIZYvX44RI0ZwVh0ios+Q7+nMiIio8JmYmGDgwIGoUqUKLl68iJEjRzLpJSL6TEx8iYiKmNjYWDx9+lT5+Mcff8SNGzfg6uqqvaCIiEqAfHV1+OWXX/J9wDFjxnx2MEREuu7s2bPo27cvSpcujYsXL8LIyAh6enowNzfXdmhERMVevhLfn3/+OV8HE4lETHyJiD6DQqHAwoULMWPGDMjlcpiamiImJgZOTk7aDo2IqMTIV+L75MmTgo6DiEhnvX79Gv3790doaCgAoH///ggKCoKZmZmWIyMiKlk4qwMRkRadOnUKfn5+iImJgbGxMX799Vf4+/tzABsRUQH4rMT3xYsXOHz4MCIjI5GVlaWybfny5RoJjIiopBMEAT/++CNiYmJQs2ZN7N27F87OztoOi4ioxFI78Q0LC0OnTp1QqVIl3L9/Hy4uLnj69CkEQeCIYyIiNYhEIuzcuRNLly7FwoULlcsJExFRwVB7OrNp06ZhwoQJiIiIgJGREX7//Xc8f/4cHh4e6NmzZ0HESERUYpw8eRILFixQPq5YsSJWrVrFpJeIqBConfjevXsXgwYNAgDo6+sjPT0dZmZmmDNnDhYtWqTxAImISgKZTIYffvgBbdq0wffff4+wsDBth0REpHPUTnxNTU2RmZkJAChbtiwePXqk3BYbG6u5yIiISoioqCh4eXlh/vz5EAQBw4YNQ5MmTbQdFhGRzlG7j2/jxo1x/vx51KxZE+3bt8eECRNw69Yt7N+/H40bNy6IGImIiq1jx45h4MCBiI2Nhbm5OdavX48+ffpoOywiIp2kduK7fPlypKSkAABmzZqFlJQU7NmzB1999VW+F7ogItIFc+bMwcyZMwEA9erVw549e1C1alUtR0VEpLvUTnwrV66s/H8TExOsWbNGowEREZUUX331FQBg5MiRWLp0KYyMjLQcERGRblM78b169SoUCgUaNWqkUn758mXo6enBzc1NY8ERERU3iYmJKFWqFADAz88P1apV4+ciEVERofbgtpEjR+L58+c5yqOiojBy5EiNBEVEVNxkZWVh/PjxcHZ2xuvXr5XlTHqJiIoOtRPfO3fu5LpQRb169XDnzh2NBEVEVJw8efIEzZs3x88//4yXL1/i8OHD2g6JiIhyoXbiK5FI8OrVqxzl0dHR0Nf/rBWQiYiKrf3796NevXq4cuUKrKyscOjQIQQGBmo7LCIiyoXaia+3tzemTZuGpKQkZVliYiK+//57eHt7azQ4IqKiKjMzE6NHj0b37t2RlJSExo0b48aNG+jUqZO2QyMiojyo3US7bNkytGjRAhUrVkS9evUAAOHh4bC3t8f27ds1HiARUVE0b948rF69GgAwadIkzJ8/HwYGBlqOioiIPkbtxNfR0RH//vsvdu7ciZs3b8LY2BiDBw9G3759+aFPRDpj0qRJOHnyJH744Qe0b99e2+EQEVE+fFanXFNTUwwdOlTTsRARFVkZGRnYunUrhg4dCpFIBAsLC1y4cAEikUjboRERUT7lK/E9fPgwfH19YWBg8MnRyuzfRkQlzf3799GrVy/8+++/yMzMxJgxYwCASS8RUTGTr8S3S5cuiImJgZ2dHbp06ZJnPZFIBLlcrqnYiIi0bufOnRg2bBhSU1NhZ2eHGjVqaDskIiL6TPlKfBUKRa7/T0RUUqWlpWHMmDHYtGkTAMDT0xM7d+6Eg4ODliMjIqLPpdZ0ZlKpFJ6ennjw4EFBxUNEpHV37txBw4YNsWnTJohEIsycOROhoaFMeomIijm1BrcZGBggIiKC/dqIqERLTEzEvXv3UKZMGezcuROtWrXSdkhERKQBai9gMXDgQOVPf0REJYUgCMr/b9KkCUJCQhAeHs6kl4ioBFF7OrOsrCxs3LgRoaGhcHNzg6mpqcr25cuXayw4IqLCcOvWLfj7+2Pr1q1wcXEBAPTs2VPLURERkaapnfhGRETA1dUVAHL09WUXCCIqTgRBwMaNGzFmzBhkZGTgu+++Q2hoqLbDIiKiAqJ24nvq1KmCiIOIqFAlJydj2LBh2L17NwDA19cX27Zt03JURERUkNTu40tEVNzduHED9evXx+7du6Gnp4dFixbhyJEjsLW11XZoRERUgPLV4tutWzds2bIFFhYW6Nat20fr7t+/XyOBEREVhCtXrqB58+bIyspC+fLlsXv3bjRp0kTbYRERUSHIV+JraWmp7L9raWlZoAERERWk+vXrw93dHRYWFtiyZQusra21HRIRERWSfCW+mzdvzvX/iYiKg5s3b6J69eowMjKCnp4eDh8+DHNzcw7IJSLSMZ/dx/f169c4e/Yszp07h9evX2syJiIijRAEAStXrkSDBg0wceJEZbmFhQWTXiIiHaR24pucnIwBAwbA0dERHh4eaNGiBRwdHdG/f38kJSWpHcCaNWtQqVIlGBkZoX79+jh79my+9jt//jz09fVRt25dtc9JRCVfQkICunXrhnHjxkEqlSImJgYymUzbYRERkRapnfgGBgbi8uXLOHLkCBITE5GUlIQjR47g2rVr+Oabb9Q61p49ezBu3DhMnz4dN27cQPPmzeHr64vIyMiP7peUlISBAwfCy8tL3fCJSAdcvnwZ9erVw8GDB2FoaIjVq1dj37590NdXewZHIiIqQdROfP/44w8EBwejbdu2sLCwgLm5Odq2bYsNGzbgjz/+UOtYy5cvR0BAAAIDA1GjRg2sWLEC5cuXR1BQ0Ef3GzZsGPz8/ODu7q5u+ERUgikUChw8eBCenp549uwZqlSpgosXL2LkyJHs2kBEROovYGFjY5PrzA6WlpawsrLK93GysrJw/fp1TJ06VaW8TZs2uHDhQp77bd68GY8ePcKOHTswb968T54nMzMTmZmZysfJyckAAKlUCqlUmu94qXjKvse817rhxYsX2LdvH2QyGXr27ImgoCBYWFjw/pdQfH/rFt5v3VJQ91ntxPeHH37A+PHjsW3bNjg4OAAAYmJiMGnSJPz444/5Pk5sbCzkcjns7e1Vyu3t7RETE5PrPg8fPsTUqVNx9uzZfP9kuWDBAsyePTtH+alTp2BiYpLveKl44zK0umPMmDFITExEmzZtcO7cOW2HQ4WA72/dwvutG9LS0grkuGonvkFBQfjvv/9QsWJFVKhQAQAQGRkJiUSCN2/eYN26dcq6//zzzyeP9+HPj4Ig5PqTpFwuh5+fH2bPno1q1arlO95p06Zh/PjxysfJyckoX748PD09YWNjk+/jUPEklUoRGhoKb29vGBgYaDsc0jCFQoElS5agTp068PHxUbYQ8H7rBr6/dQvvt26Ji4srkOOqnfh26dJFIye2tbWFnp5ejtbd169f52gFBoC3b9/i2rVruHHjBkaNGgXg3R89QRCgr6+PP//8E61atcqxn0QigUQiyVFuYGDAN44O4f0ueV6/fo0BAwbgzz//hI2NDe7fvw8LCwsAvN+6hvdbt/B+64aCusdqJ74zZ87UyIkNDQ1Rv359hIaGomvXrsry0NBQdO7cOUd9CwsL3Lp1S6VszZo1+Ouvv/Dbb7+hUqVKGomLiIq+06dPw8/PD9HR0TA2NsbixYthbW3N6cqIiOijvmhun5SUFCgUCpWy7BaX/Bg/fjwGDBgANzc3uLu7Y/369YiMjMTw4cMBvOumEBUVhW3btkEsFsPFxUVlfzs7OxgZGeUoJ6KSSS6XY/78+Zg9ezYUCgVq1qyJvXv3wtnZWduhERFRMaB24vvkyROMGjUKp0+fRkZGhrI8u2+uXC7P97F69+6NuLg4zJkzB9HR0XBxccHRo0dRsWJFAEB0dPQn5/QlIt2QkZGB9u3b46+//gIADB48GKtWrYKpqamWIyMiouJC7cS3X79+AIDg4GDY29t/8dyYI0aMwIgRI3LdtmXLlo/uO2vWLMyaNeuLzk9ExYORkRGcnJxgamqKoKAgDBgwQNshERFRMaN24vvvv//i+vXrqF69ekHEQ0SkJJPJkJqaqpw7fNWqVZgyZYpaM7sQERFlU3vltgYNGuD58+cFEQsRkVJUVBS8vLzQu3dv5VgCExMTJr1ERPTZ1G7x3bhxI4YPH46oqCi4uLjkmG6idu3aGguOiHTT8ePHMWDAAMTGxsLMzAx3797lADYiIvpiaie+b968waNHjzB48GBlmUgk+qzBbURE75NKpfjxxx+xaNEiAEC9evWwZ88eVK1aVcuRERFRSaB24jtkyBDUq1cPISEhGhncRkQEAM+fP0efPn1w4cIFAMDIkSOxdOlSGBkZaTkyIiIqKdROfJ89e4bDhw/jq6++Koh4iEgHCYKAnj174vLly7CwsMCmTZvQo0cPbYdFREQljNqD21q1aoWbN28WRCxEpKNEIhGCgoLQrFkz3Lhxg0kvEREVCLVbfDt27IjvvvsOt27dQq1atXIMbuvUqZPGgiOikuvp06e4du2aMsmtV68e/v77b3afIiKiAqN24pu9nPCcOXNybOPgNiLKjwMHDmDIkCFIS0tDpUqVUL9+fQBg0ktERAVK7a4OCoUiz39MeonoYzIzMzFmzBh069YNiYmJcHV1ha2trbbDIiIiHaF24ktE9DkePXqEpk2bYtWqVQCASZMm4e+//0bFihW1HBkREemKfCe+7dq1Q1JSkvLx/PnzkZiYqHwcFxeHmjVrajQ4IioZ9u3bB1dXV1y/fh02NjY4cuQIFi9enGOMABERUUHKd+J74sQJZGZmKh8vWrQI8fHxyscymQz379/XbHREVCI8evQIycnJaNasGcLDw9G+fXtth0RERDoo34PbBEH46GMiovdlr+YIAJMnT4a9vT0GDBgAfX21x9QSERFpBPv4EpHG7dy5E+7u7khNTQUAiMViDB48mEkvERFpVb4TX5FIlGOqIU49RETvS0tLQ2BgIPr374/Lly9jzZo12g6JiIhISa2uDv7+/pBIJACAjIwMDB8+HKampgCg0v+XiHTP3bt30atXL0REREAkEmHGjBkYP368tsMiIiJSynfiO2jQIJXH/fv3z1Fn4MCBXx4RERU7W7duxYgRI5CWloYyZcpg586daNWqlbbDIiIiUpHvxHfz5s0FGQcRFVNLly7FpEmTAACtW7fGjh07YG9vr+WoiIiIcuLgNiL6In379kWZMmUwb948HD9+nEkvEREVWRxiTURqEQQBly5dgru7OwDA0dERDx48gLm5uZYjIyIi+ji2+BJRvr19+xb9+/dHkyZNsH//fmU5k14iIioO2OJLRPkSHh6OXr164eHDh9DT00NUVJS2QyIiIlILW3yJ6KMEQUBQUBAaN26Mhw8fonz58vj7778xevRobYdGRESkFrb4ElGekpKS8M0332Dfvn0AgI4dO2Lz5s2wsbHRcmRERETqY4svEeXp77//xr59+6Cvr4/ly5fj0KFDTHqJiKjYYosvEeWpY8eOmDdvHry9vdGwYUNth0NERPRF2OJLREoJCQkICAhQGbg2ffp0Jr1ERFQisMWXiAAAly9fRu/evfHs2TM8f/4cf/75p7ZDIiIi0ii2+BLpOEEQsGzZMjRr1gzPnj1DlSpVsGDBAm2HRUREpHFs8SXSYXFxcfD398eRI0cAAL169cL69ethaWmp5ciIiIg0j4kvkY66e/cu2rRpgxcvXkAikWDlypUYOnQoRCKRtkMjIiIqEEx8iXRUhQoVYGFhgWrVqmHv3r2oU6eOtkMiIiIqUEx8iXRIfHw8SpUqBbFYDFNTUxw5cgS2trYwNzfXdmhEREQFjoPbiHTEmTNn4OLigqVLlyrLKlWqxKSXiIh0BhNfohJOLpdj7ty5aNWqFaKjo7Fz505IpVJth0VERFTomPgSlWAxMTFo27YtZsyYAYVCAX9/f1y4cAEGBgbaDo2IiKjQsY8vUQkVFhaGfv364dWrVzAxMUFQUBAGDhyo7bCIiIi0hokvUQn06tUrdOjQARkZGXBxccG+ffvw9ddfazssIiIirWLiS1QC2dvbY/Hixbh16xZWrlwJY2NjbYdERESkdUx8iUqIEydOwM7ODvXq1QMAjBo1iotREBERvYeD24iKOZlMhmnTpsHHxwc9e/ZEcnIyADDpJSIi+gBbfImKsefPn6Nv3744f/48AKBt27YwNDTUclRERERFExNfomLqjz/+wMCBAxEfHw8LCwts3LgRPXv21HZYRERERRa7OhAVMzKZDJMmTUKHDh0QHx8PNzc33Lhxg0kvERHRJzDxJSpmxGIxbt26BQAYO3Yszp07h8qVK2s5KiIioqKPXR2IigmFQgGxWAyxWIxt27bh8uXL6Nixo7bDIiIiKjbY4ktUxGVmZmLMmDEYOnSosszOzo5JLxERkZrY4ktUhD169Ai9e/fG9evXAQAjR45UztNLRERE6mGLL1ERtW/fPri6uuL69euwtrbGkSNHmPQSERF9ASa+REVMRkYGRowYgV69eiE5ORlNmzZFeHg42rdvr+3QiIiIijUmvkRFTKdOnRAUFAQAmDZtGk6fPo3y5ctrOSoiIqLij318iYqY7777Djdv3sS2bdvQtm1bbYdDRERUYjDxJdKytLQ03LlzB25ubgAAX19fPH78GKamplqOjIiIqGRhVwciLbp79y4aNWoEb29vPH36VFnOpJeIiEjzmPgSacnWrVvh5uaGiIgISCQSREdHazskIiKiEo2JL1EhS01Nhb+/P/z9/ZGWlgYvLy+Eh4fD3d1d26ERERGVaEx8iQpRREQEGjRogK1bt0IsFmPu3Lk4ceIEypQpo+3QiIiISjwObiMqRBs3bsTdu3dRtmxZ7Nq1Cx4eHtoOiYiISGcw8SUqRAsXLgQATJ8+HaVLl9ZyNERERLqFXR2IClB4eDgCAgIgl8sBAEZGRlixYgWTXiIiIi1g4ktUAARBQFBQEBo3bozg4GAsW7ZM2yERERHpPHZ1INKwpKQkDB06FHv37gUAdOjQAQEBAVqOioiIiNjiS6RB169fh6urK/bu3Qt9fX0sW7YMhw8fho2NjbZDIyIi0nls8SXSkF27dmHw4MHIyspCxYoVsWfPHjRq1EjbYREREdH/xxZfIg2pXbs29PT00LVrV9y4cYNJLxERURHDFl+iL/D69WvY2dkBAFxcXHDt2jXUqFEDIpFIy5ERERHRh9jiS/QZFAoFli1bBicnJ1y8eFFZXrNmTSa9RERERRQTXyI1xcXFoVOnTpg4cSLS09OxZ88ebYdERERE+cCuDkRqOH/+PPr06YMXL15AIpFgxYoVGDZsmLbDIiIionxgiy9RPigUCixcuBAeHh548eIFqlatikuXLmH48OHs2kBERFRMMPElyoeDBw9i2rRpkMvl8PPzw/Xr11G3bl1th0VERERqYFcHonzo2rUr/Pz84OnpiYCAALbyEhERFUNMfIlyIZfL8euvv8Lf3x8WFhYQiUTYuXOntsMiIiKiL8CuDkQfiImJQdu2bTF27FgMGzYMgiBoOyQiIiLSALb4Er0nLCwM/fr1w6tXr2BiYgIfHx92ayAiIioh2OJLhHddG2bOnAlvb2+8evUKLi4uuHr1KgYNGqTt0IiIiEhD2OJLOi8mJgZ9+vTBmTNnAACBgYFYuXIlTExMtBwZERERaRITX9J5YrEYDx48gJmZGdatWwc/Pz9th0REREQFgIkv6SSFQgGx+F1PHzs7O/z++++wsbFBtWrVtBwZERERFRT28SWd8/z5c7Ro0QK7du1Slrm7uzPpJSIiKuG0nviuWbMGlSpVgpGREerXr4+zZ8/mWXf//v3w9vZG6dKlYWFhAXd3d5w4caIQo6Xi7siRI6hbty7Onz+PyZMnIzMzU9shERERUSHRauK7Z88ejBs3DtOnT8eNGzfQvHlz+Pr6IjIyMtf6f//9N7y9vXH06FFcv34dnp6e6NixI27cuFHIkVNxk5WVhYkTJ6Jjx46Ij49H/fr1cebMGUgkEm2HRkRERIVEq318ly9fjoCAAAQGBgIAVqxYgRMnTiAoKAgLFizIUX/FihUqj3/66SccOnQI//vf/1CvXr3CCJmKoVevXqFVq1a4cuUKAGDMmDFYvHgxk14iIiIdo7XENysrC9evX8fUqVNVytu0aYMLFy7k6xgKhQJv376FtbV1nnUyMzNVfs5OTk4GAEilUkil0s+InIqTmJgYTJw4EW/fvkWpUqWwfv16dOnSBQB4/0ug7HvKe6sbeL91C++3bimo+6y1xDc2NhZyuRz29vYq5fb29oiJicnXMZYtW4bU1FT06tUrzzoLFizA7Nmzc5SfOnWK87TqCC8vL9y+fRsTJ06EoaEhjh49qu2QqICFhoZqOwQqRLzfuoX3WzekpaUVyHG1Pp3Zh8vBCoKQryViQ0JCMGvWLBw6dAh2dnZ51ps2bRrGjx+vfJycnIzy5cvD09MTNjY2nx84FVmPHz+Gvr4+KlSoAKlUCplMBi8vL5iammo7NCpgUqkUoaGh8Pb2hoGBgbbDoQLG+61beL91S1xcXIEcV2uJr62tLfT09HK07r5+/TpHK/CH9uzZg4CAAOzbtw+tW7f+aF2JRJJrX04DAwO+cUqg3377DQEBAahZsyb+/vtvGBgYQF9fH6amprzfOoTvb93C+61beL91Q0HdY63N6mBoaIj69evn+MkiNDQUTZo0yXO/kJAQ+Pv7Y9euXWjfvn1Bh0nFREZGBkaMGIGePXsiOTkZYrEYSUlJ2g6LiIiIihCtdnUYP348BgwYADc3N7i7u2P9+vWIjIzE8OHDAbzrphAVFYVt27YBeJf0Dhw4ECtXrkTjxo2VrcXGxsawtLTU2nWQdj18+BC9evVCeHg4AGDq1KmYM2cODAwMOAiCiIiIlLSa+Pbu3RtxcXGYM2cOoqOj4eLigqNHj6JixYoAgOjoaJU5fdetWweZTIaRI0di5MiRyvJBgwZhy5YthR0+FQEhISEYOnQoUlJSYGtri+3bt8PHx0fbYREREVERpPXBbSNGjMCIESNy3fZhMnv69OmCD4iKDZlMhsWLFyMlJUW5BLGjo6O2wyIiIqIiSuuJL9Hn0tfXx969e7Fr1y5Mnz4d+vp8ORMREVHetLpkMZG6tm3bhkWLFikfV61aFTNnzmTSS0RERJ/EbIGKhdTUVIwaNQpbtmyBSCRCq1at0KBBA22HRURERMUIE18q8iIiItCrVy/cvXsXYrEYs2bNgqurq7bDIiIiomKGiS8VWYIgIDg4GKNHj0Z6ejocHBywa9cutGzZUtuhERERUTHExJeKrGHDhmHDhg0AgLZt22Lbtm0fXZ6aiIiI6GM4uI2KrIYNG0JPTw8LFizA0aNHmfQSERHRF2GLLxUZgiDg9evXsLe3BwAEBASgWbNm+Prrr7UcGREREZUEbPGlIiE5ORl9+vRBw4YNkZCQAAAQiURMeomIiEhjmPiS1l2/fh2urq7Yu3cvXr58ibNnz2o7JCIiIiqBmPiS1giCgFWrVqFJkyZ49OgRKlasiLNnz6JTp07aDo2IiIhKIPbxJa1ISEhAQEAADhw4AADo0qULgoODYWVlpeXIiIiIqKRiiy9pxffff48DBw7AwMAAK1euxP79+5n0EhERUYFiiy9pxfz583H//n0sXrwYbm5u2g6HiIiIdABbfKlQxMfHY8WKFRAEAQBgbW2Nv/76i0kvERERFRq2+FKBu3DhAvr06YPnz5/DzMwMgYGB2g6JiIiIdBBbfKnAKBQKLFq0CC1atMDz589RtWpVtvASERGR1rDFlwrEmzdvMGjQIBw7dgwA0LdvX6xbtw7m5uZajoyIiIh0FRNf0rhz586hd+/eePnyJYyMjLBq1SoEBARAJBJpOzQiIiLSYUx8SeOkUimio6Px9ddfY+/evahVq5a2QyIiIiJi4kuaIZfLoaenBwDw9PTE/v370bp1a5iZmWk5MiIiIqJ3OLiNvlhYWBhq1KiBhw8fKsu6dOnCpJeIiIiKFCa+9NnkcjlmzpwJb29vPHz4EDNnztR2SERERER5YlcH+iwvX75Ev379cPr0aQBAQEAAfvnlF+0GRURERPQRTHxJbSdOnMCAAQPw5s0bmJqaYt26dejXr5+2wyIiIiL6KCa+pJZjx46hXbt2AIA6depg7969qFatmpajIiIiIvo0Jr6kltatW6Nx48aoW7culi9fDmNjY22HRERERJQvTHzpk86cOYMmTZrAwMAABgYG+Ouvv5jwEhERUbHDWR0oT1KpFJMnT0bLli3xww8/KMuZ9BIREVFxxBZfytWzZ8/Qp08fXLp0CQCQmZkJQRC47DAREREVW0x8KYeDBw9i8ODBSExMhKWlJYKDg9GtWzdth0VERET0RdjVgZSysrIwbtw4dO3aFYmJiWjYsCFu3LjBpJeIiIhKBCa+pPT8+XNs3LgRADB+/HicPXsWlSpV0nJURERERJrBrg6kVKVKFWzevBlGRkbo2LGjtsMhIiIi0ii2+OqwjIwMjB49WrnsMAD07NmTSS8RERGVSGzx1VEPHz5E7969cePGDezfvx///fcfpykjIiKiEo0tvjpo9+7dcHV1xY0bN2Bra4uNGzcy6SUiIqISj4mvDklPT8ewYcPQt29fpKSkoHnz5ggPD4evr6+2QyMiIiIqcOzqoCMSExPRokUL3Lp1CyKRCNOnT8fMmTOhr8+XABEREekGZj06wtLSEs7Oznj16hV27NgBb29vbYdEREREVKiY+JZgqampkMlksLS0hEgkwrp165CamgoHBwdth/b/2rvzqKjO+3/g74FhGVGsiiK7uCHY4AIiEP2qqYq4YdWI0SooGKhaFeKCS3BJE1v3pS51Q2tQcUGqUavEuuBGgkK1wBGjIBqxKG4oBFme3x8e5peRQZ1hFmDer3P8Y577zL3vuZ9D8uHy3DtEREREOsc1vvVUeno6vLy8EBwcDCEEAMDS0pJNLxERERksNr71jBACO3bsQLdu3ZCRkYHk5GTcv39f37GIiIiI9I6Nbz3y8uVLjBs3DiEhISguLkb//v2RlpYGBwcHfUcjIiIi0js2vvXEf/7zH3h4eCA2NhbGxsb45ptvcOLECbRo0ULf0YiIiIhqBd7cVg+Ul5dj1KhRyMrKgp2dHfbt24cePXroOxYRERFRrcIrvvWAsbExYmJiEBAQgLS0NDa9REREREqw8a2jrl27hgMHDshf+/r6IiEhAVZWVnpMRURERFR7sfGtY4QQ+Nvf/gYfHx8EBQUhPT1d35GIiIiI6gSu8a1Dnj17hpCQEMTHxwMAhg4dyufyEhEREX0gXvGtI3744Qd06dIF8fHxMDExwZo1a5CQkICmTZvqOxoRERFRncArvnXA2rVrMWvWLJSWlsLZ2RlxcXHo1q2bvmMRERER1Sm84lsHPHnyBKWlpRgxYgSuXbvGppeIiIhIDbziW0uVlZVBKn1TnujoaHz00UcYMWIEJBKJnpMRERER1U284lvLVFRUYNmyZejRowdKSkoAvHlO78iRI9n0EhEREdUAG99a5NGjRxg8eDDmzJmD5ORk7N27V9+RiIiIiOoNNr61xPnz59G5c2ecOHEC5ubm2LJlC4KCgvQdi4iIiKjeYOOrZxUVFfj666/Rp08fPHjwAC4uLkhOTsakSZO4tIGIiIhIg9j46tns2bOxYMECVFRUYNy4cUhJSYG7u7u+YxERERHVO2x89Wzq1KmwtbXFjh07sGvXLjRs2FDfkYiIiIjqJT7OTMfKy8tx5swZ9O3bFwDQqlUr3L59G+bm5npORkRERFS/8YqvDuXl5aFfv37o168fTpw4IR9n00tERESkfWx8dSQxMRGdO3fGmTNnYGFhgcLCQn1HIiIiIjIobHy1rKysDAsWLICfnx/y8/Ph7u6OlJQUjBo1St/RiIiIiAwK1/hq0f379zFmzBgkJSUBAMLCwrB69WrIZDI9JyMiIiIyPGx8tSgpKQlJSUlo1KgRtm7disDAQH1HIiIiIjJYbHy16LPPPkNOTg4+/fRTtG3bVt9xiIiIiAwa1/hqUG5uLkaMGIFHjx7Jx+bOncuml4iIiKgW4BVfDTly5AiCg4Px9OlTSKVSxMXF6TsSEREREf0Kr/jW0OvXrxEREYGAgAA8ffoU3bp1w1/+8hd9xyIiIiKit7DxrYHs7Gz06NEDa9asAQBERETgwoULcHZ21m8wIiIiIqqCSx3UdPnyZfj7++P58+do0qQJdu7ciaFDh+o7FhERERFVg42vmjp27AgrKyu4ublh7969cHJy0nckIiIiInoHNr4q+Pnnn2FrawuJRAJLS0ucPn0atra2MDEx0Xc0IiIiInoPrvH9QHFxcXB1dcWGDRvkY05OTmx6iYiIiOoINr7vUVxcjLCwMIwePRqFhYX45z//CSGEvmMRERERkYrY+L7DzZs34e3tjS1btkAikWD+/Pk4ceIEJBKJvqMRERERkYq4xrca3377LcLDw/Hq1Su0aNEC3377Lfr166fvWERERESkJja+Sty6dQvBwcEoLy9Hnz59EBsbCxsbG33HIiIiIqIaYOOrRLt27bB06VIUFRVhwYIFMDY21nckIiIiIqohNr4AhBDYtWsXunXrho4dOwIAZs2apedURERERKRJBn9z28uXLxEUFIQJEyZg1KhRKCoq0nckIiIiItICvTe+GzduhLOzM8zNzeHh4YGkpKR3zj937hw8PDxgbm6O1q1bY/PmzWof+/r16/D09MTu3bthZGSEsWPHwtzcXO39EREREVHtpdfGNy4uDjNmzMD8+fORmpqKnj17wt/fH7m5uUrnZ2dnY+DAgejZsydSU1Mxb948TJs2DYcOHVL52Lt27YKXlxdu3rwJOzs7nD17FvPmzYORkd5/FyAiIiIiLdBrl7dq1SqEhIQgNDQUrq6uWLNmDRwcHLBp0yal8zdv3gxHR0esWbMGrq6uCA0NxcSJE7FixQqVj/3FF1+gpKQE/v7+SEtLQ8+ePWv6cYiIiIioFtPbzW2vX7/G1atXERUVpTDev39/XLp0Sel7Ll++jP79+yuM+fn5Yfv27SgtLVX69cElJSUoKSmRv37+/DkAQCKRIDo6GlOmTIFEIkFBQUFNPxLVQqWlpSgqKkJBQQG/XtoAsN6GhfU2LKy3YXny5AkAaPzbcvXW+D5+/Bjl5eWwtrZWGLe2tsbDhw+Vvufhw4dK55eVleHx48dKn7W7dOlSLF68uMq4EAKLFy9Wuo2IiIiI9K+goACNGzfW2P70/jizt7/+Vwjxzq8EVjZf2XiluXPnIjIyUv762bNncHJyQm5urkZPJNVOL168gIODA+7duwdLS0t9xyEtY70NC+ttWFhvw/L8+XM4OjqiadOmGt2v3hpfKysrGBsbV7m6m5+fX+WqbqWWLVsqnS+VStGsWTOl7zEzM4OZmVmV8caNG/MHx4BYWlqy3gaE9TYsrLdhYb0Ni6YfOqC3m9tMTU3h4eGBxMREhfHExET4+voqfY+Pj0+V+adOnYKnpyfX+xARERHRO+n1qQ6RkZHYtm0bduzYgczMTERERCA3Nxfh4eEA3ixTGD9+vHx+eHg47t69i8jISGRmZmLHjh3Yvn07Zs6cqa+PQERERER1hF7X+AYGBqKgoABLlixBXl4efvvb3+L48eNwcnICAOTl5Sk809fZ2RnHjx9HREQENmzYAFtbW6xbtw4jRoz44GOamZlh4cKFSpc/UP3DehsW1tuwsN6GhfU2LNqqt0Ro+jkRRERERES1EL+mjIiIiIgMAhtfIiIiIjIIbHyJiIiIyCCw8SUiIiIig1AvG9+NGzfC2dkZ5ubm8PDwQFJS0jvnnzt3Dh4eHjA3N0fr1q2xefNmHSUlTVCl3vHx8ejXrx+aN28OS0tL+Pj44OTJkzpMSzWl6s93pYsXL0IqlaJz587aDUgapWq9S0pKMH/+fDg5OcHMzAxt2rTBjh07dJSWakrVesfGxqJTp05o0KABbGxsMGHCBBQUFOgoLdXE+fPnMWTIENja2kIikSAhIeG979FIvybqmX379gkTExOxdetWkZGRIaZPny4sLCzE3bt3lc6/c+eOaNCggZg+fbrIyMgQW7duFSYmJuLgwYM6Tk7qULXe06dPF3/961/FDz/8ILKyssTcuXOFiYmJuHbtmo6TkzpUrXelZ8+eidatW4v+/fuLTp066SYs1Zg69R46dKjo3r27SExMFNnZ2SI5OVlcvHhRh6lJXarWOykpSRgZGYm1a9eKO3fuiKSkJNGxY0cxbNgwHScndRw/flzMnz9fHDp0SAAQhw8ffud8TfVr9a7x9fLyEuHh4QpjHTp0EFFRUUrnz549W3To0EFhLCwsTHh7e2stI2mOqvVWxs3NTSxevFjT0UgL1K13YGCgWLBggVi4cCEb3zpE1XqfOHFCNG7cWBQUFOgiHmmYqvVevny5aN26tcLYunXrhL29vdYyknZ8SOOrqX6tXi11eP36Na5evYr+/fsrjPfv3x+XLl1S+p7Lly9Xme/n54eUlBSUlpZqLSvVnDr1fltFRQUKCwvRtGlTbUQkDVK33jExMbh9+zYWLlyo7YikQerU+8iRI/D09MSyZctgZ2eH9u3bY+bMmSguLtZFZKoBdert6+uL+/fv4/jx4xBC4H//+x8OHjyIQYMG6SIy6Zim+jW9fnObpj1+/Bjl5eWwtrZWGLe2tsbDhw+Vvufhw4dK55eVleHx48ewsbHRWl6qGXXq/baVK1fi1atXGDVqlDYikgapU+9bt24hKioKSUlJkErr1X/u6j116n3nzh1cuHAB5ubmOHz4MB4/fozJkyfjyZMnXOdby6lTb19fX8TGxiIwMBC//PILysrKMHToUKxfv14XkUnHNNWv1asrvpUkEonCayFElbH3zVc2TrWTqvWutHfvXixatAhxcXFo0aKFtuKRhn1ovcvLyzFmzBgsXrwY7du311U80jBVfr4rKiogkUgQGxsLLy8vDBw4EKtWrcLOnTt51beOUKXeGRkZmDZtGqKjo3H16lX861//QnZ2NsLDw3URlfRAE/1avboEYmVlBWNj4yq/Hebn51f5LaFSy5Ytlc6XSqVo1qyZ1rJSzalT70pxcXEICQnBgQMH0LdvX23GJA1Rtd6FhYVISUlBamoqpk6dCuBNYySEgFQqxalTp/DJJ5/oJDupTp2fbxsbG9jZ2aFx48byMVdXVwghcP/+fbRr106rmUl96tR76dKl+PjjjzFr1iwAgLu7OywsLNCzZ0/8+c9/5l9s6xlN9Wv16oqvqakpPDw8kJiYqDCemJgIX19fpe/x8fGpMv/UqVPw9PSEiYmJ1rJSzalTb+DNld7g4GDs2bOHa8HqEFXrbWlpiRs3biAtLU3+Lzw8HC4uLkhLS0P37t11FZ3UoM7P98cff4wHDx7g5cuX8rGsrCwYGRnB3t5eq3mpZtSpd1FREYyMFNsYY2NjAP//SiDVHxrr11S6Fa4OqHwcyvbt20VGRoaYMWOGsLCwEDk5OUIIIaKiosS4cePk8ysfjxERESEyMjLE9u3b+TizOkTVeu/Zs0dIpVKxYcMGkZeXJ//37NkzfX0EUoGq9X4bn+pQt6ha78LCQmFvby9Gjhwp0tPTxblz50S7du1EaGiovj4CqUDVesfExAipVCo2btwobt++LS5cuCA8PT2Fl5eXvj4CqaCwsFCkpqaK1NRUAUCsWrVKpKamyh9fp61+rd41vkIIsWHDBuHk5CRMTU1F165dxblz5+TbgoKCRK9evRTmnz17VnTp0kWYmpqKVq1aiU2bNuk4MdWEKvXu1auXAFDlX1BQkO6Dk1pU/fn+NTa+dY+q9c7MzBR9+/YVMplM2Nvbi8jISFFUVKTj1KQuVeu9bt064ebmJmQymbCxsRFjx44V9+/f13FqUseZM2fe+f9jbfVrEiH49wAiIiIiqv/q1RpfIiIiIqLqsPElIiIiIoPAxpeIiIiIDAIbXyIiIiIyCGx8iYiIiMggsPElIiIiIoPAxpeIiIiIDAIbXyIiIiIyCGx8iYh05OzZs5BIJHj27BkAYOfOnfjNb36jlyytWrXCmjVrdH5cTX3mD8kvkUiQkJAAAMjJyYFEIkFaWhqAqrUgIsPAxpeItCI4OBgSiQTh4eFVtk2ePBkSiQTBwcG6D/aWnTt3QiKRyP/Z2Nhg1KhRyM7O1vqxAwMDkZWV9cHzddmsLlq0SH5OjI2N4eDggNDQUDx69Egnx9eEvLw8+Pv7K93m6+uLvLw8NG7cGIB+fwkhIt1h40tEWuPg4IB9+/ahuLhYPvbLL79g7969cHR01GMyRZaWlsjLy8ODBw+wZ88epKWlYejQoSgvL68yVwiBsrIyjRxXJpOhRYsWGtmXNnTs2BF5eXnIzc3Fpk2bcPToUYwfP17p3PLyclRUVOg44bu1bNkSZmZmSreZmpqiZcuWkEgkOk5FRPrExpeItKZr165wdHREfHy8fCw+Ph4ODg7o0qWLwlwhBJYtW4bWrVtDJpOhU6dOOHjwoHx7eXk5QkJC4OzsDJlMBhcXF6xdu1ZhH8HBwRg2bBhWrFgBGxsbNGvWDFOmTEFpaek7c0okErRs2RI2Njbo06cPFi5ciP/+97/46aef5H8SP3nyJDw9PWFmZoakpKT35gWA48ePo3379pDJZOjTpw9ycnIUtiu7ynjkyBF4enrC3NwcVlZWGD58OACgd+/euHv3LiIiIuRXYitdunQJ//d//weZTAYHBwdMmzYNr169km/Pz8/HkCFDIJPJ4OzsjNjY2Heej0pSqRQtW7aEnZ0dBg8ejGnTpuHUqVMoLi6WZ//uu+/g5uYGMzMz3L17F0+fPsX48ePRpEkTNGjQAP7+/rh161aVfSckJKB9+/YwNzdHv379cO/ePfm227dvIyAgANbW1mjYsCG6deuG77//vso+CgsLMWbMGDRs2BC2trZYv369wvZfL3V426+XOpw9exYTJkzA8+fP5ed20aJFWLJkCT766KMq7/Xw8EB0dPQHnUMiql3Y+BKRVk2YMAExMTHy1zt27MDEiROrzFuwYAFiYmKwadMmpKenIyIiAn/4wx9w7tw5AEBFRQXs7e2xf/9+ZGRkIDo6GvPmzcP+/fsV9nPmzBncvn0bZ86cwa5du7Bz507s3LlTpcwymQwAFBrm2bNnY+nSpcjMzIS7u/t78967dw/Dhw/HwIEDkZaWhtDQUERFRb3zuMeOHcPw4cMxaNAgpKam4vTp0/D09ATw5hcGe3t7LFmyBHl5ecjLywMA3LhxA35+fhg+fDiuX7+OuLg4XLhwAVOnTpXvNzg4GDk5Ofj3v/+NgwcPYuPGjcjPz1fpnFSel4qKCvkV76KiIixduhTbtm1Deno6WrRogeDgYKSkpODIkSO4fPkyhBAYOHCgwrksKirC119/jV27duHixYt48eIFRo8eLd/+8uVLDBw4EN9//z1SU1Ph5+eHIUOGIDc3VyHP8uXL4e7ujmvXrmHu3LmIiIhAYmKiyp/L19cXa9askV/5z8vLw8yZMzFx4kRkZGTgxx9/lM+9fv06UlNTa8UyHSJSgyAi0oKgoCAREBAgHj16JMzMzER2drbIyckR5ubm4tGjRyIgIEAEBQUJIYR4+fKlMDc3F5cuXVLYR0hIiPjss8+qPcbkyZPFiBEjFI7p5OQkysrK5GOffvqpCAwMrHYfMTExonHjxvLX9+7dE97e3sLe3l6UlJSIM2fOCAAiISFBPudD8s6dO1e4urqKiooK+fY5c+YIAOLp06dKj+3j4yPGjh1bbVYnJyexevVqhbFx48aJzz//XGEsKSlJGBkZieLiYnHz5k0BQFy5ckW+PTMzUwCosq9fW7hwoejUqZPCe9q2bSu8vLzk2QGItLQ0+ZysrCwBQFy8eFE+9vjxYyGTycT+/fsV3qcsT3JycrV53NzcxPr16xXOxYABAxTmBAYGCn9/f/lrAOLw4cNCCCGys7MFAJGamiqEEPK6VleLSv7+/uKPf/yj/PWMGTNE7969q81JRLWbVE/9NhEZCCsrKwwaNAi7du2CEAKDBg2ClZWVwpyMjAz88ssv6Nevn8L469evFZZEbN68Gdu2bcPdu3dRXFyM169fo3Pnzgrv6dixI4yNjeWvbWxscOPGjXdmfP78ORo2bAghBIqKitC1a1fEx8fD1NRUPqfyyuuH5s3MzIS3t7fCkgQfH5935khLS8OkSZPeOedtV69exU8//aSwfEEIgYqKCmRnZyMrKwtSqVQhf4cOHT7oRq4bN26gYcOGKC8vR0lJCXr37o0tW7bIt5uamsLd3V3+OjMzE1KpFN27d5ePNWvWDC4uLsjMzJSPVZcnMzMTXl5eePXqFRYvXozvvvsODx48QFlZGYqLi6tc8X37fPr4+Gj85r9JkyZh4sSJWLVqFYyNjREbG4uVK1dq9BhEpDtsfIlI6yZOnCj/0/uGDRuqbK+8KerYsWOws7NT2FZ5c9L+/fsRERGBlStXwsfHB40aNcLy5cuRnJysMN/ExEThtUQiee9NV40aNcK1a9dgZGQEa2trWFhYVJnz67EPySuEeOcxlalcYqGKiooKhIWFYdq0aVW2OTo64ubNmwCg1k1cLi4uOHLkCIyNjWFra1vlRjGZTKaw3+o+sxCiyvGV5akcmzVrFk6ePIkVK1agbdu2kMlkGDlyJF6/fv3ezJq+WW3IkCEwMzPD4cOHYWZmhpKSEowYMUKjxyAi3WHjS0RaN2DAAHnT4ufnV2V75c1Rubm56NWrl9J9JCUlwdfXF5MnT5aP3b59WyP5jIyM0LZt2w+e/yF53dzcqtxYdeXKlXfu193dHadPn8aECROUbjc1Na3ypImuXbsiPT292vyurq4oKytDSkoKvLy8AAA3b978oOfXmpqaqnxeysrKkJycDF9fXwBAQUEBsrKy4OrqKp9XXZ4OHToAeFPr4OBg/P73vwfwZs3v2zcGAlXP55UrV+T7UJWycwu8uTodFBSEmJgYmJmZYfTo0WjQoIFaxyAi/WPjS0RaZ2xsLP9T96+XIVRq1KgRZs6ciYiICFRUVKBHjx548eIFLl26hIYNGyIoKAht27bFP/7xD5w8eRLOzs7YvXs3fvzxRzg7O+v643xQ3vDwcKxcuRKRkZEICwvD1atX33uT3cKFC/G73/0Obdq0wejRo1FWVoYTJ05g9uzZAN48x/f8+fMYPXo0zMzMYGVlhTlz5sDb2xtTpkzBpEmTYGFhgczMTCQmJmL9+vVwcXHBgAEDMGnSJGzZsgVSqRQzZsxQ6+ry+7Rr1w4BAQGYNGkS/v73v6NRo0aIioqCnZ0dAgIC5PNMTEzwpz/9CevWrYOJiQmmTp0Kb29veSPctm1bxMfHY8iQIZBIJPjyyy+VXrW/ePEili1bhmHDhiExMREHDhzAsWPH1MreqlUrvHz5EqdPn0anTp3QoEEDeYMbGhoqb9wvXryo1v6JqHbgUx2ISCcsLS1haWlZ7favvvoK0dHRWLp0KVxdXeHn54ejR4/KG9vw8HAMHz4cgYGB6N69OwoKChSu/ura+/I6Ojri0KFDOHr0KDp16oTNmzfjm2++eec+e/fujQMHDuDIkSPo3LkzPvnkE4WlHEuWLEFOTg7atGmD5s2bA3hzlfjcuXO4desWevbsiS5duuDLL7+EjY2N/H0xMTFwcHBAr169MHz4cHz++edae35wTEwMPDw8MHjwYPj4+EAIgePHjyssQWnQoAHmzJmDMWPGwMfHBzKZDPv27ZNvX716NZo0aQJfX18MGTIEfn5+6Nq1a5VjffHFF7h69Sq6dOmCr776CitXrlT6F4UP4evri/DwcAQGBqJ58+ZYtmyZfFu7du3g6+sLFxcXhfXLRFT3SIQ6C9GIiIgMhBACHTp0QFhYGCIjI/Udh4hqgEsdiIiIqpGfn4/du3fj559/rnbtNRHVHWx8iYiIqmFtbQ0rKyts2bIFTZo00XccIqohNr5ERETV4GpAovqFN7cRERERkUFg40tEREREBoGNLxEREREZBDa+RERERGQQ2PgSERERkUFg40tEREREBoGNLxEREREZBDa+RERERGQQ/h+YhigHPapGaAAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "### Now the transformed data become a top-class problem when we specify the class-of-interest to be 1\n", "reliability,confindence,bin_edges,bin_counts = reliability_diagram(transformed_data.labels, transformed_data.probs, num_bins=15,class_to_plot=1)\n", "# Plot the reliability diagram\n", "plot_reliability_diagram(reliability,confindence,bin_counts,error_bar=True,title='Top class reliability diagram for well calibrated data')" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'SpiegelhalterZ score': 0.37632691618773545,\n", " 'SpiegelhalterZ p-value': 0.7066738713391101,\n", " 'ECE-H topclass': 0.009608653731328977,\n", " 'ECE-H': 0.009608653731329372,\n", " 'MCE-H topclass': 0.03926468843081976,\n", " 'MCE-H': 0.03926468843081932,\n", " 'HL-H score': 6.029652031234071,\n", " 'HL-H p-value': 0.11017646753618837,\n", " 'ECE-C topclass': 0.009458033653818828,\n", " 'ECE-C': 0.009458033653818974,\n", " 'MCE-C topclass': 0.020515047600205505,\n", " 'MCE-C': 0.020515047600205394,\n", " 'HL-C score': 5.0830845954424,\n", " 'HL-C p-value': 0.7486601568004448,\n", " 'COX coef': 0.9990721119864874,\n", " 'COX intercept': -0.009313116424641145,\n", " 'COX coef lowerci': 0.9097229915817346,\n", " 'COX coef upperci': 1.0884212323912401,\n", " 'COX intercept lowerci': -0.1322110153978738,\n", " 'COX intercept upperci': 0.1135847825485915,\n", " 'COX ICI': 0.0012819521292332472,\n", " 'Loess ICI': 0.0038637011857438034}" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cal_metrics.calculate_metrics(transformed_data.labels, transformed_data.probs,metrics='all')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can see the top-class ECE is the same as regular ECE." ] } ], "metadata": { "kernelspec": { "display_name": "uq", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.4" } }, "nbformat": 4, "nbformat_minor": 2 }