{ "cells": [ { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from math import sqrt\n", "from numpy import concatenate\n", "from matplotlib import pyplot\n", "import pandas as pd\n", "import numpy as np\n", "from sklearn.preprocessing import MinMaxScaler\n", "from sklearn.preprocessing import LabelEncoder\n", "from sklearn.metrics import mean_squared_error\n", "from tensorflow.keras import Sequential\n", "\n", "from tensorflow.keras.layers import Dense\n", "from tensorflow.keras.layers import LSTM\n", "from tensorflow.keras.layers import Dropout\n", "from sklearn.model_selection import train_test_split\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "这段代码是一个函数 time_series_to_supervised,它用于将时间序列数据转换为监督学习问题的数据集。下面是该函数的各个部分的含义:\n", "\n", "data: 输入的时间序列数据,可以是列表或2D NumPy数组。\n", "n_in: 作为输入的滞后观察数,即用多少个时间步的观察值作为输入。默认值为96,表示使用前96个时间步的观察值作为输入。\n", "n_out: 作为输出的观测数量,即预测多少个时间步的观察值。默认值为1,表示预测未来1个时间步的观察值。\n", "dropnan: 布尔值,表示是否删除具有NaN值的行。默认为True,即删除具有NaN值的行。\n", "函数首先检查输入数据的维度,并初始化一些变量。然后,它创建一个新的DataFrame对象 df 来存储输入数据,并保存原始的列名。接着,它创建了两个空列表 cols 和 names,用于存储新的特征列和列名。\n", "\n", "接下来,函数开始构建特征列和对应的列名。首先,它将原始的观察序列添加到 cols 列表中,并将其列名添加到 names 列表中。然后,它依次将滞后的观察序列添加到 cols 列表中,并构建相应的列名,格式为 (原始列名)(t-滞后时间)。这样就创建了输入特征的部分。\n", "\n", "接着,函数开始构建输出特征的部分。它依次将未来的观察序列添加到 cols 列表中,并构建相应的列名,格式为 (原始列名)(t+未来时间)。\n", "\n", "最后,函数将所有的特征列拼接在一起,构成一个新的DataFrame对象 agg。如果 dropnan 参数为True,则删除具有NaN值的行。最后,函数返回处理后的数据集 agg。" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "def time_series_to_supervised(data, n_in=96, n_out=1,dropnan=True):\n", " \"\"\"\n", " :param data:作为列表或2D NumPy数组的观察序列。需要。\n", " :param n_in:作为输入的滞后观察数(X)。值可以在[1..len(数据)]之间可选。默认为1。\n", " :param n_out:作为输出的观测数量(y)。值可以在[0..len(数据)]之间。可选的。默认为1。\n", " :param dropnan:Boolean是否删除具有NaN值的行。可选的。默认为True。\n", " :return:\n", " \"\"\"\n", " n_vars = 1 if type(data) is list else data.shape[1]\n", " df = pd.DataFrame(data)\n", " origNames = df.columns\n", " cols, names = list(), list()\n", " cols.append(df.shift(0))\n", " names += [('%s' % origNames[j]) for j in range(n_vars)]\n", " n_in = max(0, n_in)\n", " for i in range(n_in, 0, -1):\n", " time = '(t-%d)' % i\n", " cols.append(df.shift(i))\n", " names += [('%s%s' % (origNames[j], time)) for j in range(n_vars)]\n", " n_out = max(n_out, 0)\n", " for i in range(1, n_out+1):\n", " time = '(t+%d)' % i\n", " cols.append(df.shift(-i))\n", " names += [('%s%s' % (origNames[j], time)) for j in range(n_vars)]\n", " agg = pd.concat(cols, axis=1)\n", " agg.columns = names\n", " if dropnan:\n", " agg.dropna(inplace=True)\n", " return agg" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " Temp Humidity GHI DHI Rainfall Power\n", "0 19.779453 40.025826 3.232706 1.690531 0.0 0.0\n", "1 19.714937 39.605961 3.194991 1.576346 0.0 0.0\n", "2 19.549330 39.608631 3.070866 1.576157 0.0 0.0\n", "3 19.405870 39.680702 3.038623 1.482489 0.0 0.0\n", "4 19.387363 39.319881 2.656474 1.134153 0.0 0.0\n", "(104256, 6)\n" ] } ], "source": [ "# 加载数据\n", "path1 = r\"D:\\project\\小论文1-基于ICEEMDAN分解的时序高维变化的短期光伏功率预测模型\\CEEMAN-PosConv1dbiLSTM-LSTM\\模型代码流程\\data6.csv\"#数据所在路径\n", "#我的数据是excel表,若是csv文件用pandas的read_csv()函数替换即可。\n", "datas1 = pd.DataFrame(pd.read_csv(path1))\n", "#我只取了data表里的第3、23、16、17、18、19、20、21、27列,如果取全部列的话这一行可以去掉\n", "# data1 = datas1.iloc[:,np.r_[3,23,16:22,27]]\n", "data1=datas1.interpolate()\n", "values1 = data1.values\n", "print(data1.head())\n", "print(data1.shape)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(104256, 6)\n" ] } ], "source": [ "# 使用MinMaxScaler进行归一化\n", "scaler = MinMaxScaler(feature_range=(0, 1))\n", "scaledData1 = scaler.fit_transform(data1)\n", "print(scaledData1.shape)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " 0 1 2 3 4 5 0(t-96) \\\n", "96 0.555631 0.349673 0.190042 0.040558 0.0 0.236302 0.490360 \n", "97 0.564819 0.315350 0.211335 0.044613 0.0 0.258204 0.489088 \n", "98 0.576854 0.288321 0.229657 0.047549 0.0 0.279860 0.485824 \n", "99 0.581973 0.268243 0.247775 0.053347 0.0 0.301336 0.482997 \n", "100 0.586026 0.264586 0.266058 0.057351 0.0 0.322851 0.482632 \n", "\n", " 1(t-96) 2(t-96) 3(t-96) ... 2(t-1) 3(t-1) 4(t-1) 5(t-1) \\\n", "96 0.369105 0.002088 0.002013 ... 0.166009 0.036794 0.0 0.214129 \n", "97 0.364859 0.002061 0.001839 ... 0.190042 0.040558 0.0 0.236302 \n", "98 0.364886 0.001973 0.001839 ... 0.211335 0.044613 0.0 0.258204 \n", "99 0.365615 0.001950 0.001697 ... 0.229657 0.047549 0.0 0.279860 \n", "100 0.361965 0.001679 0.001167 ... 0.247775 0.053347 0.0 0.301336 \n", "\n", " 0(t+1) 1(t+1) 2(t+1) 3(t+1) 4(t+1) 5(t+1) \n", "96 0.564819 0.315350 0.211335 0.044613 0.0 0.258204 \n", "97 0.576854 0.288321 0.229657 0.047549 0.0 0.279860 \n", "98 0.581973 0.268243 0.247775 0.053347 0.0 0.301336 \n", "99 0.586026 0.264586 0.266058 0.057351 0.0 0.322851 \n", "100 0.590772 0.258790 0.282900 0.060958 0.0 0.343360 \n", "\n", "[5 rows x 588 columns]\n" ] } ], "source": [ "n_steps_in =96 #历史时间长度\n", "n_steps_out=1#预测时间长度\n", "processedData1 = time_series_to_supervised(scaledData1,n_steps_in,n_steps_out)\n", "print(processedData1.head())" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "data_x = processedData1.loc[:,'0(t-96)':'5(t-1)']#去除power剩下的做标签列\n", "data_y = processedData1.loc[:,'5']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "冒号\n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(104159, 576)" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_x.shape" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "96 0.236302\n", "97 0.258204\n", "98 0.279860\n", "99 0.301336\n", "100 0.322851\n", " ... \n", "104250 0.000000\n", "104251 0.000000\n", "104252 0.000000\n", "104253 0.000000\n", "104254 0.000000\n", "Name: 5, Length: 104159, dtype: float64" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_y" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(104159,)" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_y.shape" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(93743, 96, 6) (93743,) (8854, 96, 6) (8854,) (1562, 96, 6) (1562,)\n" ] } ], "source": [ "# 计算训练集、验证集和测试集的大小\n", "train_size = int(len(data_x) * 0.90)\n", "test_size = int(len(data_x) * 0.015)\n", "val_size = len(data_x) - train_size - test_size\n", "\n", "# 计算训练集、验证集和测试集的索引范围\n", "train_indices = range(train_size)\n", "val_indices = range(train_size, train_size + val_size)\n", "test_indices = range(train_size + val_size, len(data_x))\n", "\n", "# 根据索引范围划分数据集\n", "train_X1 = data_x.iloc[train_indices].values.reshape((-1, n_steps_in, scaledData1.shape[1]))\n", "val_X1 = data_x.iloc[val_indices].values.reshape((-1, n_steps_in, scaledData1.shape[1]))\n", "test_X1 = data_x.iloc[test_indices].values.reshape((-1, n_steps_in, scaledData1.shape[1]))\n", "train_y = data_y.iloc[train_indices].values\n", "val_y = data_y.iloc[val_indices].values\n", "test_y = data_y.iloc[test_indices].values\n", "\n", "# reshape input to be 3D [samples, timesteps, features]\n", "train_X = train_X1.reshape((train_X1.shape[0], n_steps_in, scaledData1.shape[1]))\n", "val_X = val_X1.reshape((val_X1.shape[0], n_steps_in, scaledData1.shape[1]))\n", "test_X = test_X1.reshape((test_X1.shape[0], n_steps_in, scaledData1.shape[1]))\n", "\n", "print(train_X.shape, train_y.shape, val_X.shape, val_y.shape, test_X.shape, test_y.shape)" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(93743, 96, 6)" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_X1.shape" ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Model: \"functional_4\"\n",
"
\n"
],
"text/plain": [
"\u001b[1mModel: \"functional_4\"\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓\n", "┃ Layer (type) ┃ Output Shape ┃ Param # ┃ Connected to ┃\n", "┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩\n", "│ input_layer_4 │ (None, 96, 6) │ 0 │ - │\n", "│ (InputLayer) │ │ │ │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ conv1d_4 (Conv1D) │ (None, 95, 64) │ 832 │ input_layer_4[0]… │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ max_pooling1d_4 │ (None, 95, 64) │ 0 │ conv1d_4[0][0] │\n", "│ (MaxPooling1D) │ │ │ │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ bidirectional_4 │ (None, 95, 128) │ 49,920 │ max_pooling1d_4[… │\n", "│ (Bidirectional) │ │ │ │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ attention_with_imp… │ [(None, None, │ 66,304 │ bidirectional_4[… │\n", "│ (AttentionWithImpr… │ 128), (None, 8, │ │ bidirectional_4[… │\n", "│ │ None, None)] │ │ bidirectional_4[… │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ global_average_poo… │ (None, 128) │ 0 │ attention_with_i… │\n", "│ (GlobalAveragePool… │ │ │ │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ dense_24 (Dense) │ (None, 1) │ 129 │ global_average_p… │\n", "└─────────────────────┴───────────────────┴────────────┴───────────────────┘\n", "\n" ], "text/plain": [ "┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mConnected to \u001b[0m\u001b[1m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩\n", "│ input_layer_4 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m96\u001b[0m, \u001b[38;5;34m6\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │ - │\n", "│ (\u001b[38;5;33mInputLayer\u001b[0m) │ │ │ │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ conv1d_4 (\u001b[38;5;33mConv1D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m95\u001b[0m, \u001b[38;5;34m64\u001b[0m) │ \u001b[38;5;34m832\u001b[0m │ input_layer_4[\u001b[38;5;34m0\u001b[0m]… │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ max_pooling1d_4 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m95\u001b[0m, \u001b[38;5;34m64\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │ conv1d_4[\u001b[38;5;34m0\u001b[0m][\u001b[38;5;34m0\u001b[0m] │\n", "│ (\u001b[38;5;33mMaxPooling1D\u001b[0m) │ │ │ │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ bidirectional_4 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m95\u001b[0m, \u001b[38;5;34m128\u001b[0m) │ \u001b[38;5;34m49,920\u001b[0m │ max_pooling1d_4[\u001b[38;5;34m…\u001b[0m │\n", "│ (\u001b[38;5;33mBidirectional\u001b[0m) │ │ │ │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ attention_with_imp… │ [(\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m, │ \u001b[38;5;34m66,304\u001b[0m │ bidirectional_4[\u001b[38;5;34m…\u001b[0m │\n", "│ (\u001b[38;5;33mAttentionWithImpr…\u001b[0m │ \u001b[38;5;34m128\u001b[0m), (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m8\u001b[0m, │ │ bidirectional_4[\u001b[38;5;34m…\u001b[0m │\n", "│ │ \u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m)] │ │ bidirectional_4[\u001b[38;5;34m…\u001b[0m │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ global_average_poo… │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m128\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │ attention_with_i… │\n", "│ (\u001b[38;5;33mGlobalAveragePool…\u001b[0m │ │ │ │\n", "├─────────────────────┼───────────────────┼────────────┼───────────────────┤\n", "│ dense_24 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m1\u001b[0m) │ \u001b[38;5;34m129\u001b[0m │ global_average_p… │\n", "└─────────────────────┴───────────────────┴────────────┴───────────────────┘\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Total params: 117,185 (457.75 KB)\n", "\n" ], "text/plain": [ "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m117,185\u001b[0m (457.75 KB)\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Trainable params: 117,185 (457.75 KB)\n", "\n" ], "text/plain": [ "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m117,185\u001b[0m (457.75 KB)\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Non-trainable params: 0 (0.00 B)\n", "\n" ], "text/plain": [ "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import tensorflow as tf\n", "from tensorflow.keras.layers import Input, Conv1D, Bidirectional, GlobalAveragePooling1D, Dense, GRU, MaxPooling1D\n", "from tensorflow.keras.models import Model\n", "from tensorflow.keras.initializers import RandomUniform\n", "class AttentionWithImproveRelativePositionEncoding(tf.keras.layers.Layer):\n", " def __init__(self, d_model, num_heads, max_len=5000):\n", " super(AttentionWithImproveRelativePositionEncoding, self).__init__()\n", " self.num_heads = num_heads\n", " self.d_model = d_model\n", " self.max_len = max_len\n", " self.wq = tf.keras.layers.Dense(d_model)\n", " self.wk = tf.keras.layers.Dense(d_model)\n", " self.wv = tf.keras.layers.Dense(d_model)\n", " self.dense = tf.keras.layers.Dense(d_model)\n", " self.position_encoding = ImproveRelativePositionEncoding(d_model)\n", "\n", " def call(self, v, k, q, mask):\n", " batch_size = tf.shape(q)[0]\n", " q = self.wq(q)\n", " k = self.wk(k)\n", " v = self.wv(v)\n", "\n", " # 添加位置编码\n", " k += self.position_encoding (k)\n", " q += self.position_encoding (q)\n", "\n", " q = self.split_heads(q, batch_size)\n", " k = self.split_heads(k, batch_size)\n", " v = self.split_heads(v, batch_size)\n", "\n", " scaled_attention, attention_weights = self.scaled_dot_product_attention(q, k, v, mask)\n", " scaled_attention = tf.transpose(scaled_attention, perm=[0, 2, 1, 3])\n", " concat_attention = tf.reshape(scaled_attention, (batch_size, -1, self.d_model))\n", " output = self.dense(concat_attention)\n", " return output, attention_weights\n", "\n", " def split_heads(self, x, batch_size):\n", " x = tf.reshape(x, (batch_size, -1, self.num_heads, self.d_model // self.num_heads))\n", " return tf.transpose(x, perm=[0, 2, 1, 3])\n", "\n", " def scaled_dot_product_attention(self, q, k, v, mask):\n", " matmul_qk = tf.matmul(q, k, transpose_b=True)\n", " dk = tf.cast(tf.shape(k)[-1], tf.float32)\n", " scaled_attention_logits = matmul_qk / tf.math.sqrt(dk)\n", "\n", " if mask is not None:\n", " scaled_attention_logits += (mask * -1e9)\n", "\n", " attention_weights = tf.nn.softmax(scaled_attention_logits, axis=-1)\n", " output = tf.matmul(attention_weights, v)\n", " return output, attention_weights\n", "\n", "class ImproveRelativePositionEncoding(tf.keras.layers.Layer):\n", " def __init__(self, d_model, max_len=5000):\n", " super(ImproveRelativePositionEncoding, self).__init__()\n", " self.max_len = max_len\n", " self.d_model = d_model\n", " # 引入可变化的参数u和v进行线性变化\n", " self.u = self.add_weight(shape=(self.d_model,),\n", " initializer=RandomUniform(),\n", " trainable=True)\n", " self.v = self.add_weight(shape=(self.d_model,),\n", " initializer=RandomUniform(),\n", " trainable=True)\n", " def call(self, inputs):\n", " seq_length = inputs.shape[1]\n", " pos_encoding = self.relative_positional_encoding(seq_length, self.d_model)\n", " \n", " # 调整原始的相对位置编码公式,将u和v参数融入其中\n", " pe_with_params = pos_encoding * self.u+ pos_encoding * self.v\n", " return inputs + pe_with_params\n", "\n", " def relative_positional_encoding(self, position, d_model):\n", " pos = tf.range(position, dtype=tf.float32)\n", " i = tf.range(d_model, dtype=tf.float32)\n", " \n", " angles = 1 / tf.pow(10000.0, (2 * (i // 2)) / tf.cast(d_model, tf.float32))\n", " angle_rads = tf.einsum('i,j->ij', pos, angles)\n", " #保留了sinous机制\n", " # Apply sin to even indices; 2i\n", " angle_rads_sin = tf.sin(angle_rads[:, 0::2])\n", " # Apply cos to odd indices; 2i+1\n", " angle_rads_cos = tf.cos(angle_rads[:, 1::2])\n", "\n", " pos_encoding = tf.stack([angle_rads_sin, angle_rads_cos], axis=2)\n", " pos_encoding = tf.reshape(pos_encoding, [1, position, d_model])\n", "\n", " return pos_encoding\n", "\n", "\n", "\n", "def PosConv1biGRUWithSelfAttention(input_shape, gru_units, num_heads):\n", " inputs = Input(shape=input_shape)\n", " # CNN layer\n", " cnn_layer = Conv1D(filters=64, kernel_size=2, activation='relu')(inputs)\n", " cnn_layer = MaxPooling1D(pool_size=1)(cnn_layer)\n", " gru_output = Bidirectional(GRU(gru_units, return_sequences=True))(cnn_layer)\n", " \n", " # Apply Self-Attention\n", " self_attention =AttentionWithImproveRelativePositionEncoding(d_model=gru_units*2, num_heads=num_heads)\n", " gru_output, _ = self_attention(gru_output, gru_output, gru_output, mask=None)\n", " \n", " pool1 = GlobalAveragePooling1D()(gru_output)\n", " output = Dense(1)(pool1)\n", " \n", " return Model(inputs=inputs, outputs=output)\n", "\n", "\n", "input_shape = (96, 6)\n", "gru_units = 64\n", "num_heads = 8\n", "\n", "# Create model\n", "model = PosConv1biGRUWithSelfAttention(input_shape, gru_units, num_heads)\n", "model.compile(optimizer='adam', loss='mse')\n", "model.summary()\n" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m106s\u001b[0m 71ms/step - loss: 0.0198 - val_loss: 0.0016\n", "Epoch 2/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m107s\u001b[0m 73ms/step - loss: 0.0016 - val_loss: 0.0015\n", "Epoch 3/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m108s\u001b[0m 74ms/step - loss: 0.0015 - val_loss: 0.0015\n", "Epoch 4/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m107s\u001b[0m 73ms/step - loss: 0.0015 - val_loss: 0.0014\n", "Epoch 5/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m106s\u001b[0m 73ms/step - loss: 0.0014 - val_loss: 0.0016\n", "Epoch 6/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m105s\u001b[0m 71ms/step - loss: 0.0014 - val_loss: 0.0015\n", "Epoch 7/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m104s\u001b[0m 71ms/step - loss: 0.0014 - val_loss: 0.0014\n", "Epoch 8/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m107s\u001b[0m 73ms/step - loss: 0.0013 - val_loss: 0.0014\n", "Epoch 9/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m107s\u001b[0m 73ms/step - loss: 0.0013 - val_loss: 0.0014\n", "Epoch 10/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m106s\u001b[0m 72ms/step - loss: 0.0013 - val_loss: 0.0015\n", "Epoch 11/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m105s\u001b[0m 71ms/step - loss: 0.0013 - val_loss: 0.0014\n", "Epoch 12/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m105s\u001b[0m 72ms/step - loss: 0.0013 - val_loss: 0.0015\n", "Epoch 13/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m107s\u001b[0m 73ms/step - loss: 0.0013 - val_loss: 0.0014\n", "Epoch 14/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m108s\u001b[0m 74ms/step - loss: 0.0012 - val_loss: 0.0014\n", "Epoch 15/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m107s\u001b[0m 73ms/step - loss: 0.0013 - val_loss: 0.0014\n", "Epoch 16/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m104s\u001b[0m 71ms/step - loss: 0.0013 - val_loss: 0.0013\n", "Epoch 17/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m105s\u001b[0m 72ms/step - loss: 0.0013 - val_loss: 0.0013\n", "Epoch 18/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m105s\u001b[0m 72ms/step - loss: 0.0012 - val_loss: 0.0014\n", "Epoch 19/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m102s\u001b[0m 69ms/step - loss: 0.0012 - val_loss: 0.0013\n", "Epoch 20/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m102s\u001b[0m 70ms/step - loss: 0.0012 - val_loss: 0.0014\n", "Epoch 21/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m103s\u001b[0m 70ms/step - loss: 0.0012 - val_loss: 0.0014\n", "Epoch 22/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m102s\u001b[0m 70ms/step - loss: 0.0011 - val_loss: 0.0014\n", "Epoch 23/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m102s\u001b[0m 69ms/step - loss: 0.0012 - val_loss: 0.0018\n", "Epoch 24/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m101s\u001b[0m 69ms/step - loss: 0.0012 - val_loss: 0.0014\n", "Epoch 25/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m102s\u001b[0m 70ms/step - loss: 0.0011 - val_loss: 0.0014\n", "Epoch 26/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m102s\u001b[0m 70ms/step - loss: 0.0012 - val_loss: 0.0015\n", "Epoch 27/100\n", "\u001b[1m1465/1465\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m97s\u001b[0m 66ms/step - loss: 0.0012 - val_loss: 0.0015\n" ] } ], "source": [ "# Compile and train the model\n", "model.compile(optimizer='adam', loss='mean_squared_error')\n", "from keras.callbacks import EarlyStopping, ModelCheckpoint\n", "\n", "# 定义早停机制\n", "early_stopping = EarlyStopping(monitor='val_loss', min_delta=0, patience=10, verbose=0, mode='min')\n", "\n", "# 拟合模型,并添加早停机制和模型检查点\n", "history = model.fit(train_X, train_y, epochs=100, batch_size=64, validation_data=(val_X, val_y), \n", " callbacks=[early_stopping])\n" ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m49/49\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 16ms/step\n" ] } ], "source": [ "# 预测\n", "lstm_pred = model.predict(test_X)\n", "# 将预测结果的形状修改为与原始数据相同的形状\n" ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [], "source": [ "test_y_pre=test_y" ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAGdCAYAAADqsoKGAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAABKLUlEQVR4nO3de3zT9aE//lfuaZs0UApNWwqtUEDGpQpSy5jorBZlzqpzyJxcfhzQjXnAikwYFHVod0AcopzD8exMtp0hyFeGlzEmq3epRW46pnITLArpBWzThjbXz++PT/JJA2lp2s8nn7a8no9HHkk+eeeTd0JoX31fNYIgCCAiIiLq4bRqV4CIiIhIDgw1RERE1Csw1BAREVGvwFBDREREvQJDDREREfUKDDVERETUKzDUEBERUa/AUENERES9gl7tCsRLIBDA6dOnYbVaodFo1K4OERERdYAgCGhsbERGRga02vbbYi6bUHP69GlkZWWpXQ0iIiLqhFOnTmHgwIHtlrlsQo3VagUgfijJyckq14aIiIg6wul0IisrS/o93p7LJtSEupySk5MZaoiIiHqYjgwd4UBhIiIi6hUYaoiIiKhXYKghIiKiXuGyGVNDRESkFEEQ4PP54Pf71a5Kj6PT6aDX62VZboWhhoiIqAs8Hg/OnDmD8+fPq12VHisxMRHp6ekwGo1dOg9DDRERUScFAgGcOHECOp0OGRkZMBqNXOA1BoIgwOPxoLa2FidOnEBubu4lF9hrD0MNERFRJ3k8HgQCAWRlZSExMVHt6vRICQkJMBgM+Oqrr+DxeGA2mzt9Lg4UJiIi6qKutC6QfJ8f/xWIiIioV2CoISIiol6BoYaIiIi6JDs7G2vXrlW7GhwoTEREdDm6/vrrkZeXJ0sY+fjjj5GUlNT1SnURQ00XHa1uxOaPT6G/1YQHJg9RuzpERESyEAQBfr8fev2lo0L//v3jUKNLY/dTF51uaMH/fnACrx48rXZViIhIZYIg4LzHp8pFEIQO13PWrFl499138eyzz0Kj0UCj0WDjxo3QaDT429/+hnHjxsFkMuGDDz7A8ePHcfvttyMtLQ0WiwXXXHMN/vGPf0Sc78LuJ41Gg9/97ne44447kJiYiNzcXLz22mtyfcxtYktNF1lM4kfY5PaqXBMiIlJbs9ePkaV/V+W1P3uiCInGjv1af/bZZ3HkyBGMGjUKTzzxBADgX//6FwDg0UcfxdNPP40rrrgCffv2xalTp3DrrbfiySefhMlkwh//+EfcdtttOHz4MAYNGtTmazz++ONYtWoVVq9ejeeeew733nsvvvrqK6SkpHT9zbaBLTVdZDUHQ02LT+WaEBERdYzNZoPRaERiYiLsdjvsdjt0Oh0A4IknnsBNN92EIUOGICUlBWPHjsX999+PUaNGITc3F7/+9a8xZMiQS7a8zJo1C9OnT8fQoUPx1FNPoampCXv27FH0fbGlpovCLTVi0x+XxyYiunwlGHT47Iki1V5bDuPHj4+439TUhMceewx//etfcebMGfh8PjQ3N6Oqqqrd84wZM0a6nZSUhOTkZNTU1MhSx7Yw1HRRqKXG6xfg9gVglulLRUREPY9Go+lwF1B3deEspkWLFmHXrl14+umnMXToUCQkJOBHP/oRPB5Pu+cxGAwR9zUaDQKBgOz1ba1nf/LdQFKrL29ji4+hhoiIegSj0Qi/33/Jch9++CFmzZqFO+64A4DYcnPy5EmFa9c5HFPTRVqtJqILioiIqCfIzs5GZWUlTp48ibq6ujZbUXJzc7Ft2zYcPHgQn3zyCX7yk58o3uLSWQw1MpBCDQcLExFRD7Fo0SLodDqMHDkS/fv3b3OMzDPPPIO+ffti4sSJuO2221BUVISrr746zrXtGHY/ycBi1gNOoJHTuomIqIcYNmwYKioqIo7NmjXronLZ2dl46623Io7Nnz8/4v6F3VHR1sypr6/vVD1jwZYaGXBaNxERkfoYamQQ6n5qZKghIiJSDUONDKSWGg4UJiIiUg1DjQw4+4mIiEh9DDUysJjEBYbY/URERKQehhoZhLufOPuJiIhILQw1MgiFGrbUEBERqYehRgZcfI+IiEh9DDUysIRaajhQmIiISDUMNTJgSw0REfU0119/PRYuXCjb+WbNmoXi4mLZztcZDDUysJqDs584UJiIiEg1DDUy4DYJRETUk8yaNQvvvvsunn32WWg0Gmg0Gpw8eRKHDh3CLbfcAovFgrS0NNx3332oq6uTnvf//t//w+jRo5GQkIB+/fqhsLAQLpcLjz32GP7whz/g1Vdflc73zjvvxP19cUNLGbRefE8QBGg0GpVrREREqhAEwHtendc2JAId/P3z7LPP4siRIxg1ahSeeOIJ8ekGAyZMmIB/+7d/w29/+1s0Nzfjl7/8JX784x/jrbfewpkzZzB9+nSsWrUKd9xxBxobG/H+++9DEAQsWrQIn3/+OZxOJ1588UUAQEpKimJvtS0MNTIIDRT2+gW4fQGYDTqVa0RERKrwngeeylDntZeeBoxJHSpqs9lgNBqRmJgIu90OAFi5ciWuuuoqPPXUU1K53//+98jKysKRI0fQ1NQEn8+HO++8E4MHDwYAjB49WiqbkJAAt9stnU8Nnep+Wr9+PbKzs2E2m5Gfn489e/a0W37r1q0YMWIEzGYzRo8ejR07dkQ8LggCSktLkZ6ejoSEBBQWFuLo0aMXneevf/0r8vPzkZCQgL59+6o+ICnEYgxnQ26VQEREPdEnn3yCt99+GxaLRbqMGDECAHD8+HGMHTsWN954I0aPHo27774b//M//4Nvv/1W5VpHirmlZsuWLSgpKcGGDRuQn5+PtWvXoqioCIcPH8aAAQMuKr97925Mnz4dZWVl+MEPfoBNmzahuLgY+/fvx6hRowAAq1atwrp16/CHP/wBOTk5WL58OYqKivDZZ5/BbDYDAF555RXMnTsXTz31FL7//e/D5/Ph0KFDXXz78tBqNbCY9Ghy+9DY4kOqxaR2lYiISA2GRLHFRK3X7oKmpibcdttt+I//+I+LHktPT4dOp8OuXbuwe/duvPnmm3juuefwq1/9CpWVlcjJyenSa8tGiNGECROE+fPnS/f9fr+QkZEhlJWVRS3/4x//WJg6dWrEsfz8fOH+++8XBEEQAoGAYLfbhdWrV0uP19fXCyaTSXjppZcEQRAEr9crZGZmCr/73e9ira6koaFBACA0NDR0+hztyX/yH8LgX74hfHqqXpHzExFR99Pc3Cx89tlnQnNzs9pVidlNN90k/OIXv5DuL126VBg+fLjg9Xo79HyfzydkZmYKa9asEQRBEObOnSv84Ac/6FRd2vscY/n9HVP3k8fjwb59+1BYWCgd02q1KCwsREVFRdTnVFRURJQHgKKiIqn8iRMn4HA4IsrYbDbk5+dLZfbv349vvvkGWq0WV111FdLT03HLLbd0m5YaoPUCfJzWTURE3V92djYqKytx8uRJ1NXVYf78+Th37hymT5+Ojz/+GMePH8ff//53zJ49G36/H5WVlXjqqaewd+9eVFVVYdu2baitrcWVV14pne/TTz/F4cOHUVdXB683/r8PYwo1dXV18Pv9SEtLizielpYGh8MR9TkOh6Pd8qHr9sp8+eWXAIDHHnsMy5YtwxtvvIG+ffvi+uuvx7lz56K+rtvthtPpjLgoiQvwERFRT7Jo0SLodDqMHDkS/fv3h8fjwYcffgi/34+bb74Zo0ePxsKFC9GnTx9otVokJyfjvffew6233ophw4Zh2bJlWLNmDW655RYAwNy5czF8+HCMHz8e/fv3x4cffhj399QjZj8FAgEAwK9+9SvcddddAIAXX3wRAwcOxNatW3H//fdf9JyysjI8/vjjcatjeKduhhoiIur+hg0bFrWXZdu2bVHLX3nlldi5c2eb5+vfvz/efPNN2erXGTG11KSmpkKn06G6ujrieHV1dZtTuOx2e7vlQ9ftlUlPTwcAjBw5UnrcZDLhiiuuQFVVVdTXXbJkCRoaGqTLqVOnOvo2O4U7dRMREakrplBjNBoxbtw4lJeXS8cCgQDKy8tRUFAQ9TkFBQUR5QFg165dUvmcnBzY7faIMk6nE5WVlVKZcePGwWQy4fDhw1IZr9eLkydPSnPlL2QymZCcnBxxUVLrBfiIiIgo/mLufiopKcHMmTMxfvx4TJgwAWvXroXL5cLs2bMBADNmzEBmZibKysoAAAsWLMDkyZOxZs0aTJ06FZs3b8bevXvxwgsvAAA0Gg0WLlyIlStXIjc3V5rSnZGRIa1Dk5ycjAceeAArVqxAVlYWBg8ejNWrVwMA7r77bjk+hy6zmIL7P7GlhoiISBUxh5pp06ahtrYWpaWlcDgcyMvLw86dO6WBvlVVVdBqww1AEydOxKZNm7Bs2TIsXboUubm52L59u7RGDQAsXrwYLpcL8+bNQ319PSZNmoSdO3dKa9QAwOrVq6HX63HfffehubkZ+fn5eOutt9C3b9+uvH/ZWKQxNZz9REREpAaNIAiC2pWIB6fTCZvNhoaGBkW6on73/pdY+dfPcXteBp695yrZz09ERN1PS0sLTpw4gZycnIg/xCk27X2Osfz+5i7dMuGUbiKiy9dl0j6gGLk+P4YamYQX32OoISK6XBgM4njK8+dV2pm7lwh9fqHPs7N6xDo1PQFbaoiILj86nQ59+vRBTU0NACAxMREajUblWvUcgiDg/PnzqKmpQZ8+faDT6bp0PoYamVjNYrrklG4iostLaE21ULCh2PXp06fN9e5iwVAjk/Die5z9RER0OdFoNEhPT8eAAQNU2e+opzMYDF1uoQlhqJFJ68X3BEFg8yMR0WVGp9PJ9suZOocDhWUSGijs9Qtw+wIq14aIiOjyw1AjkyRjuNGL42qIiIjij6FGJjqthjOgiIiIVMRQI6NQqOH+T0RERPHHUCOj8AJ8HP1OREQUbww1MmL3ExERkXoYamRkNYendRMREVF8MdTIiKGGiIhIPQw1MuJAYSIiIvUw1MjIYhL3f2KoISIiij+GGhlZpO4nzn4iIiKKN4YaGVk5+4mIiEg1DDUyCu/UzVBDREQUbww1MgovvsdQQ0REFG8MNTLi4ntERETqYaiREdepISIiUg9DjYysZnFKN0MNERFR/DHUyCi8+J4XgiCoXBsiIqLLC0ONjEIDhb1+AW5fQOXaEBERXV4YamSUZNRLt9kFRUREFF8MNTLSaTVIMuoAcAYUERFRvDHUyIyDhYmIiNTBUCOz0LgaZwv3fyIiIoonhhqZcQE+IiIidTDUyIwL8BEREamDoUZmUksNQw0REVFcMdTIjDt1ExERqYOhRmYWkzj7iaGGiIgovhhqZGaRxtRw9hMREVE8MdTIzMrZT0RERKpgqJEZZz8RERGpg6FGZuHF9xhqiIiI4omhRmZcfI+IiEgdDDUyY/cTERGROhhqZBaa0s1QQ0REFF8MNTKTWmrY/URERBRXDDUyCw0U9vgDaPH6Va4NERHR5YOhRmZJRr10m11QRERE8cNQIzOdVoMkow4Au6CIiIjiiaFGARbOgCIiIoo7hhoFWM3c1JKIiCjeGGoUEFqAr7GFm1oSERHFC0ONArgAHxERUfwx1ChA2iqBoYaIiChuGGoUEGqp4ZgaIiKi+GGoUQC3SiAiIoo/hhoFWMwcKExERBRvnQo169evR3Z2NsxmM/Lz87Fnz552y2/duhUjRoyA2WzG6NGjsWPHjojHBUFAaWkp0tPTkZCQgMLCQhw9ejSiTHZ2NjQaTcTlN7/5TWeqrzirifs/ERERxVvMoWbLli0oKSnBihUrsH//fowdOxZFRUWoqamJWn737t2YPn065syZgwMHDqC4uBjFxcU4dOiQVGbVqlVYt24dNmzYgMrKSiQlJaGoqAgtLS0R53riiSdw5swZ6fLggw/GWv244OJ7RERE8RdzqHnmmWcwd+5czJ49GyNHjsSGDRuQmJiI3//+91HLP/vss5gyZQoeeeQRXHnllfj1r3+Nq6++Gs8//zwAsZVm7dq1WLZsGW6//XaMGTMGf/zjH3H69Gls37494lxWqxV2u126JCUlxf6O44ADhYmIiOIvplDj8Xiwb98+FBYWhk+g1aKwsBAVFRVRn1NRURFRHgCKioqk8idOnIDD4YgoY7PZkJ+ff9E5f/Ob36Bfv3646qqrsHr1avh8bYcGt9sNp9MZcYmX8OJ7DDVERETxor90kbC6ujr4/X6kpaVFHE9LS8MXX3wR9TkOhyNqeYfDIT0eOtZWGQD493//d1x99dVISUnB7t27sWTJEpw5cwbPPPNM1NctKyvD448/Hsvbkw0X3yMiIoq/mEKNmkpKSqTbY8aMgdFoxP3334+ysjKYTKaLyi9ZsiTiOU6nE1lZWXGpK6d0ExERxV9M3U+pqanQ6XSorq6OOF5dXQ273R71OXa7vd3yoetYzgkA+fn58Pl8OHnyZNTHTSYTkpOTIy7xIg0UZvcTERFR3MQUaoxGI8aNG4fy8nLpWCAQQHl5OQoKCqI+p6CgIKI8AOzatUsqn5OTA7vdHlHG6XSisrKyzXMCwMGDB6HVajFgwIBY3kJchLqfPP4A3D6/yrUhIiK6PMTc/VRSUoKZM2di/PjxmDBhAtauXQuXy4XZs2cDAGbMmIHMzEyUlZUBABYsWIDJkydjzZo1mDp1KjZv3oy9e/fihRdeAABoNBosXLgQK1euRG5uLnJycrB8+XJkZGSguLgYgDjYuLKyEjfccAOsVisqKirw0EMP4ac//Sn69u0r00chnyRj+GNtbPHBZNGpWBsiIqLLQ8yhZtq0aaitrUVpaSkcDgfy8vKwc+dOaaBvVVUVtNpwA9DEiROxadMmLFu2DEuXLkVubi62b9+OUaNGSWUWL14Ml8uFefPmob6+HpMmTcLOnTthNpsBiF1JmzdvxmOPPQa3242cnBw89NBDEWNmuhOdVoMkow4ujx9NLT6kWi4e80NERETy0giCIKhdiXhwOp2w2WxoaGiIy/ia/Kf+gWqnG288OAmjMm2Kvx4REVFvFMvvb+79pBCuVUNERBRfDDUKsZo5rZuIiCieGGoUYuVO3URERHHFUKOQUPcTW2qIiIjig6FGIRxTQ0REFF8MNQrhmBoiIqL4YqhRCLdKICIiii+GGoVYTRwoTEREFE8MNQqRWmrY/URERBQXDDUK4UBhIiKi+GKoUYiVLTVERERxxVCjkPDieww1RERE8cBQoxCLiVO6iYiI4omhRiGc0k1ERBRfDDUKCQ0U9vgDcPv8KteGiIio92OoUUgo1ABsrSEiIooHhhqF6LQaJBl1ADhYmIiIKB4YahTEBfiIiIjih6FGQVyAj4iIKH4YahTEnbqJiIjih6FGQeFVhbmpJRERkdIYahTE7iciIqL4YahREEMNERFR/DDUKIizn4iIiOKHoUZB0kBhttQQEREpjqFGQVYTW2qIiIjihaFGQaHup8YWzn4iIiJSGkONgjhQmIiIKH4YahTEgcJERETxw1CjoGSGGiIiorhhqFGQxSTOfmL3ExERkfIYahQkdT8x1BARESmOoUZBoYHCHn8Abp9f5doQERH1bgw1CgqFGoCtNUREREpjqFGQTqtBklEHgIOFiYiIlMZQo7DwAnwMNUREREpiqFEYF+AjIiKKD4YahVlCm1qy+4mIiEhRDDUKCy/Ax/2fiIiIlMRQo7BQ9xNnPxERESmLoUZhoVDjZKghIiJSFEONwripJRERUXww1CjMyu4nIiKiuGCoUZiVs5+IiIjigqFGYeHF9zj7iYiISEkMNQrj4ntERETxwVCjMA4UJiIiig+GGoVJA4UZaoiIiBTFUKMwaaAwu5+IiIgUxVCjMO7STUREFB8MNQoLDRT2+ANw+/wq14aIiKj3YqhRWCjUAOyCIiIiUhJDjcJ0Wg2SjDoAHCxMRESkpE6FmvXr1yM7Oxtmsxn5+fnYs2dPu+W3bt2KESNGwGw2Y/To0dixY0fE44IgoLS0FOnp6UhISEBhYSGOHj0a9Vxutxt5eXnQaDQ4ePBgZ6ofdxxXQ0REpLyYQ82WLVtQUlKCFStWYP/+/Rg7diyKiopQU1MTtfzu3bsxffp0zJkzBwcOHEBxcTGKi4tx6NAhqcyqVauwbt06bNiwAZWVlUhKSkJRURFaWlouOt/ixYuRkZERa7VVxQX4iIiIlBdzqHnmmWcwd+5czJ49GyNHjsSGDRuQmJiI3//+91HLP/vss5gyZQoeeeQRXHnllfj1r3+Nq6++Gs8//zwAsZVm7dq1WLZsGW6//XaMGTMGf/zjH3H69Gls37494lx/+9vf8Oabb+Lpp5+O/Z2qyML9n4iIiBQXU6jxeDzYt28fCgsLwyfQalFYWIiKioqoz6moqIgoDwBFRUVS+RMnTsDhcESUsdlsyM/PjzhndXU15s6diz/96U9ITEy8ZF3dbjecTmfERS3hBfi4/xMREZFSYgo1dXV18Pv9SEtLizielpYGh8MR9TkOh6Pd8qHr9soIgoBZs2bhgQcewPjx4ztU17KyMthsNumSlZXVoecpwRraKoHdT0RERIrpEbOfnnvuOTQ2NmLJkiUdfs6SJUvQ0NAgXU6dOqVgDdsnjalh9xMREZFiYgo1qamp0Ol0qK6ujjheXV0Nu90e9Tl2u73d8qHr9sq89dZbqKiogMlkgl6vx9ChQwEA48ePx8yZM6O+rslkQnJycsRFLZz9REREpLyYQo3RaMS4ceNQXl4uHQsEAigvL0dBQUHU5xQUFESUB4Bdu3ZJ5XNycmC32yPKOJ1OVFZWSmXWrVuHTz75BAcPHsTBgwelKeFbtmzBk08+GctbUIU0poahhoiISDH6SxeJVFJSgpkzZ2L8+PGYMGEC1q5dC5fLhdmzZwMAZsyYgczMTJSVlQEAFixYgMmTJ2PNmjWYOnUqNm/ejL179+KFF14AAGg0GixcuBArV65Ebm4ucnJysHz5cmRkZKC4uBgAMGjQoIg6WCwWAMCQIUMwcODATr/5eAm11HD2ExERkXJiDjXTpk1DbW0tSktL4XA4kJeXh507d0oDfauqqqDVhhuAJk6ciE2bNmHZsmVYunQpcnNzsX37dowaNUoqs3jxYrhcLsybNw/19fWYNGkSdu7cCbPZLMNbVF9op252PxERESlHIwiCoHYl4sHpdMJms6GhoSHu42te/+Q0HnzpAPJzUrDl/ujddERERHSxWH5/94jZTz0du5+IiIiUx1ATB+HF9xhqiIiIlMJQEwehMTWc/URERKQchpo4kNapYUsNERGRYhhq4iC0orDHF4Db51e5NkRERL0TQ00chEINwC4oIiIipTDUxIFOq0GiUQeAg4WJiIiUwlATJ1bu/0RERKQohpo4sXBaNxERkaIYauLEwq0SiIiIFMVQEyfhBfi8KteEiIiod2KoiROp+4ktNURERIpgqIkTKxfgIyIiUhRDTZxIm1qypYaIiEgRDDVxEhpTw4HCREREymCoiROppYbdT0RERIpgqIkTi4lTuomIiJTEUBMnVjOndBMRESmJoSZOLNwmgYiISFEMNXFi5TYJREREimKoiRNO6SYiIlIWQ02cWEN7P7GlhoiISBEMNXES2ibB4wvA7fOrXBsiIqLeh6EmTkKhBmAXFBERkRIYauJEp9Ug0agDwMHCRERESmCoiSMLt0ogIiJSDENNHFm5VQIREZFiGGriyBKcAcUxNURERPJjqIkjaadubpVAREQkO4aaOAqNqWFLDRERkfwYauJI2v+JY2qIiIhkx1ATR1ZulUBERKQYhpo44qaWREREymGoiSOp+4ktNURERLJjqIkjiym4qSVDDRERkewYauIovPgep3QTERHJjaEmjixcUZiIiEgxDDVxZOXeT0RERIphqIkjC6d0ExERKYahJo6kXbrZ/URERCQ7hpo4sgY3tPT4AnD7/CrXhoiIqHdhqImjUEsNALjcDDVERERyYqiJI51Wg0SjDgDQ2MJp3URERHJiqIkzC2dAERERKYKhJs64Vg0REZEyGGriLDRYmNO6iYiI5MVQE2fcqZuIiEgZDDVxFh5Tw4HCREREcmKoibPQmBouwEdERCQvhpo4C7XUcEwNERGRvBhq4iyZs5+IiIgUwVATZ9zUkoiISBkMNXFmMYlTup0MNURERLLqVKhZv349srOzYTabkZ+fjz179rRbfuvWrRgxYgTMZjNGjx6NHTt2RDwuCAJKS0uRnp6OhIQEFBYW4ujRoxFlfvjDH2LQoEEwm81IT0/Hfffdh9OnT3em+qoKL77H2U9ERERyijnUbNmyBSUlJVixYgX279+PsWPHoqioCDU1NVHL7969G9OnT8ecOXNw4MABFBcXo7i4GIcOHZLKrFq1CuvWrcOGDRtQWVmJpKQkFBUVoaWlRSpzww034OWXX8bhw4fxyiuv4Pjx4/jRj37UibesLivH1BARESlCIwiCEMsT8vPzcc011+D5558HAAQCAWRlZeHBBx/Eo48+elH5adOmweVy4Y033pCOXXvttcjLy8OGDRsgCAIyMjLw8MMPY9GiRQCAhoYGpKWlYePGjbjnnnui1uO1115DcXEx3G43DAbDJevtdDphs9nQ0NCA5OTkWN6yrPaePIcfbahAdr9EvPPIDarVg4iIqCeI5fd3TC01Ho8H+/btQ2FhYfgEWi0KCwtRUVER9TkVFRUR5QGgqKhIKn/ixAk4HI6IMjabDfn5+W2e89y5c/jzn/+MiRMnthlo3G43nE5nxKU7kNap4ZgaIiIiWcUUaurq6uD3+5GWlhZxPC0tDQ6HI+pzHA5Hu+VD1x055y9/+UskJSWhX79+qKqqwquvvtpmXcvKymCz2aRLVlZWx96kwqQVhdn9REREJKseNfvpkUcewYEDB/Dmm29Cp9NhxowZaKv3bMmSJWhoaJAup06dinNto7MGZz95fAG4fX6Va0NERNR76GMpnJqaCp1Oh+rq6ojj1dXVsNvtUZ9jt9vbLR+6rq6uRnp6ekSZvLy8i14/NTUVw4YNw5VXXomsrCx89NFHKCgouOh1TSYTTCZTLG8vLkLdTwDgcvth0utUrA0REVHvEVNLjdFoxLhx41BeXi4dCwQCKC8vjxosAKCgoCCiPADs2rVLKp+TkwO73R5Rxul0orKyss1zhl4XEMfO9CQ6rQaJRjHIcAE+IiIi+cTUUgMAJSUlmDlzJsaPH48JEyZg7dq1cLlcmD17NgBgxowZyMzMRFlZGQBgwYIFmDx5MtasWYOpU6di8+bN2Lt3L1544QUAgEajwcKFC7Fy5Urk5uYiJycHy5cvR0ZGBoqLiwEAlZWV+PjjjzFp0iT07dsXx48fx/LlyzFkyJB2g093ZTHpcd7jh5M7dRMREckm5lAzbdo01NbWorS0FA6HA3l5edi5c6c00LeqqgpabbgBaOLEidi0aROWLVuGpUuXIjc3F9u3b8eoUaOkMosXL4bL5cK8efNQX1+PSZMmYefOnTCbzQCAxMREbNu2DStWrIDL5UJ6ejqmTJmCZcuWdcsupkuxmPWoaXRzrRoiIiIZxbxOTU/VXdapAYDbn/8An3zdgN/NGI/CkWmXfgIREdFlSrF1akgeVrM4A4otNURERPJhqFEB16ohIiKSH0ONCsKrCnOgMBERkVwYalQQaqnhlG4iIiL5MNSoIJk7dRMREcmOoUYFoe4nttQQERHJh6FGBZbg/k8cKExERCQfhhoVcKAwERGR/BhqVGA1cUwNERGR3BhqVGDlmBoiIiLZMdSowMLZT0RERLJjqFFBaJ0aJ1tqiIiIZMNQowJrcPaTxxeA2+dXuTZERES9A0ONCpJMOum2y81QQ0REJAeGGhXodVokGsVgw8HCRERE8mCoUUl4p26uVUNERCQHhhqVhBfgY0sNERGRHBhqVGLlTt1ERESyYqhRidUszoDiWjVERETyYKhRSXhMDUMNERGRHBhqVGLhVglERESyYqhRidRSw526iYiIZMFQoxIr938iIiKSFUONSrhTNxERkbwYalRiCe7/xIHCRERE8mCoUUl48T2OqSEiIpIDQ41KpMX32FJDREQkC4YalXBKNxERkbwYalTC2U9ERETyYqhRSXidGoYaIiIiOTDUqMQanP3k9gXg8QVUrg0REVHPx1CjkiSTTrrNLigiIqKuY6hRiV6nRYJBDDYcLExERNR1DDUqCg0WbnRzrRoiIqKuYqhREad1ExERyYehRkVWzoAiIiKSDUONiixcq4aIiEg2DDUqsnJTSyIiItkw1KiIY2qIiIjkw1CjIou0qSVnPxEREXUVQ42KpCndbKkhIiLqMoYaFUktNQw1REREXcZQoyKrmQOFiYiI5MJQoyIOFCYiIpIPQ42KpMX3OFCYiIioyxhqVMSWGiIiIvkw1KgoPKWboYaIiKirGGpUxCndRERE8mGoUVFomwS3LwCPL6BybYiIiHo2hhoVJZl00m12QREREXUNQ42K9DotEgxisOFgYSIioq5hqFGZNK6G07qJiIi6pFOhZv369cjOzobZbEZ+fj727NnTbvmtW7dixIgRMJvNGD16NHbs2BHxuCAIKC0tRXp6OhISElBYWIijR49Kj588eRJz5sxBTk4OEhISMGTIEKxYsQIej6cz1e9WOK2biIhIHjGHmi1btqCkpAQrVqzA/v37MXbsWBQVFaGmpiZq+d27d2P69OmYM2cODhw4gOLiYhQXF+PQoUNSmVWrVmHdunXYsGEDKisrkZSUhKKiIrS0tAAAvvjiCwQCAfz3f/83/vWvf+G3v/0tNmzYgKVLl3bybXcfVk7rJiIikoVGEAQhlifk5+fjmmuuwfPPPw8ACAQCyMrKwoMPPohHH330ovLTpk2Dy+XCG2+8IR279tprkZeXhw0bNkAQBGRkZODhhx/GokWLAAANDQ1IS0vDxo0bcc8990Stx+rVq/Ff//Vf+PLLLztUb6fTCZvNhoaGBiQnJ8fylhV17+8+wofHzmLttDwUX5WpdnWIiIi6lVh+f8fUUuPxeLBv3z4UFhaGT6DVorCwEBUVFVGfU1FREVEeAIqKiqTyJ06cgMPhiChjs9mQn5/f5jkBMfikpKS0+bjb7YbT6Yy4dEcWaasEttQQERF1RUyhpq6uDn6/H2lpaRHH09LS4HA4oj7H4XC0Wz50Hcs5jx07hueeew73339/m3UtKyuDzWaTLllZWe2/OZWEdurmmBoiIqKu6XGzn7755htMmTIFd999N+bOndtmuSVLlqChoUG6nDp1Ko617LjwVgmc/URERNQVMYWa1NRU6HQ6VFdXRxyvrq6G3W6P+hy73d5u+dB1R855+vRp3HDDDZg4cSJeeOGFdutqMpmQnJwccemOrJz9REREJIuYQo3RaMS4ceNQXl4uHQsEAigvL0dBQUHU5xQUFESUB4Bdu3ZJ5XNycmC32yPKOJ1OVFZWRpzzm2++wfXXX49x48bhxRdfhFbb4xqZopLG1DDUEBERdYk+1ieUlJRg5syZGD9+PCZMmIC1a9fC5XJh9uzZAIAZM2YgMzMTZWVlAIAFCxZg8uTJWLNmDaZOnYrNmzdj7969UkuLRqPBwoULsXLlSuTm5iInJwfLly9HRkYGiouLAYQDzeDBg/H000+jtrZWqk9bLUQ9hcXMgcJERERyiDnUTJs2DbW1tSgtLYXD4UBeXh527twpDfStqqqKaEWZOHEiNm3ahGXLlmHp0qXIzc3F9u3bMWrUKKnM4sWL4XK5MG/ePNTX12PSpEnYuXMnzGYzALFl59ixYzh27BgGDhwYUZ8YZ6R3OxwoTEREJI+Y16npqbrrOjVvf1GD2Rs/xuhMG15/cJLa1SEiIupWFFunhuQndT+1cPYTERFRVzDUqMzCbRKIiIhkwVCjMs5+IiIikgdDjcqSgwOF3b4APL6AyrUhIiLquRhqVJZk0km3XeyCIiIi6jSGGpXpdVokGMRgwy4oIqIewPFP4IO1gOe82jWhC8S8Tg3Jz2LWo9nrRyP3fyIi6t6avwX+70dAkwOo+Qy4478BjUbtWlEQW2q6Ae7/RETUQ+xcIgYaAPh0C7D39+rWhyIw1HQDVk7rJiLq/g7vBD55CdBogbE/EY/tfBT4Zr+69SIJQ003EFqAj6GGiKibav4WeGOheLtgPlD8n8CIHwB+D/DyTOD8OVWrRyKGmm4gtFaNk91PRETd099/BTSeAfoNBW74lTiO5vb1QN8coKEK2DYPCHBZDrUx1HQDFhM3tSQi6raO/B04+GcAGuD2/wQMCeLxhD7AtD8BejNwbBfw/ho1a0lgqOkWpIHCnP1ERNS9NNcDry8QbxfMBwblRz5uHw1MDYaZt58Ejr8d1+pRJIaaboCzn4iIuqlQt1PKELHbKZqrfgpcdR8AAXhlDtDwTVyrSGEMNd2AtP8TBwoTEXUfR3cBB/8PgEYcGGxMbLvsravFVpvzZ4GtswCfJ161pFYYarqB0OwnrihMRNRNtDQAr/27ePvanwODrm2/vCEB+PGfAJMN+HoPsKtU+TrSRRhquoFQSw27n4iIuom/LwUaTwMpVwDfX9ax56TkAHdsEG9X/hdwaJty9aOoGGq6gdBO3VynhoioGzj6D+BAsNvp9kt0O11oxK3AdxeKt197EKg9okQNqQ0MNd0AF98jIuomWhqA10PdTj8DBhfEfo7vLweyvwd4moCXZwAel7x1pDYx1HSVxwWc+VRcTVIQOnUKaaBwSy+b0s2FqIiop3lzGeD8RlxU7/vLO3cOnR64638Bix2o/Rx4fWGnfz9QbLhLd1ed+QR48RbxttEC2AYCtqzg9UCgz6DwbWuG+GW/QDjU9IKWGp9H3Bvl/TVAUzUwYqq4R8qQGwCtTu3aERG17dg/gP1/RIdmO12KNQ24+0Vg4w+Af74srm9zzb/JVlWKjqGmqzwuIKk/4KoVmxprvxAv0Wi0YrDp0yr02LKQYk5HruZrnPNZ4Wk8C6PBIAYArR7Q6MTb3X1reynMPA3UV4WPH3pFvFjswJgfA2OnA2kj1asnycPnBj7+X/Hf9qp7gXGzu/93lKg9rWc75d8PDJ7Y9XMOnggUPgbsWi7u7p1xFZA5ruvnpTZpBOHyaBNzOp2w2WxoaGhAcnKy/C/gbRYXXGo4Fbx8LV7qq8Rr5zfixmedFQo3rYNO6H7rY4ZEIPcmIO8nQP/h8r2/tvi9wMFNkWEmaQAw6SFg4DXAP7eKl+ZWm72ljxVbb0b/CEhKVb6OJJ9AQPz3fHtlZHgddgvww+cAS3/16kbUFa/9O7D/D2K3088+BIxJ8pxXEIAtPwW+eENsxb//PSAxRZ5zXyZi+f3NUBMvgQDgqokMOlL4OYX6M1+ij0bmwWQZV4vhZtRd8v8nai/MjJsV2Wzr8wBH3xRbco78HQgExw5p9UDuzWLrzbAiQG+St44kH0EAjpcDux4Dqv8pHrOmA1feBuzbKAb2pP5A8X+JoZqoJzlWDvzfneLtWTuA7O/Ke/6WBuCF64FzXwJDbwJ+8jKg5ZDWjmKoiUL1UHMJS7Z9ipf2VEGHQPDiR7JJi4JsGyYN6YuCnD7ITDYAAT8Q8AFCQLyW7vvF2w1fA5++LIYIwS+eXGsAhk8B8u4FhhYCOkPnKxpLmInGdVbssvhkE3D6QPh4Ql9g1I/EgJN5NbsyohEE8YfiqT1ii0jO5K79W3bUN/uAfzwGnHhPvG+yAZMWAvkPiP/ejn8Cr8wVB0QCwIR5wE1PhDf9I+rOWpzAfxYAzq+BCfcDt65S5nUc/wR+Vwj4WsTtFiYvVuZ1YuFtBqo/AxyfAM7TgNUO2ILjQPtkASar2jUEwFATVXcPNQBQ2+jG+0dr8e6RWrx/tA7nXJHdVVf0T8J1uf0xeXh/XJvTDwnGdgbeNtWK3QSfbBL/M4UkpobHtqSP6Xjl/F6xpeW91ReEmYXieIrODKir+UI856dbxL1VQlKHA2PvAcZMA2yZsZ+3txAEoO4o8NUHwMkPga8+jPycElKAkT8UW+IGf1f+gdhnjwNv/Rr411/E+zqjGFi+9/DFLX/eZjH4VAYXHus/Arjzf2L7jhGp4fUFYmtj32zgZ7vl63aK5sCfgVd/DkAD3LcNGPJ95V7rQs31gONTcbZu6LruSPiP32jMtsiQI02EyRLvJw2IS4sTQ00UPSHUtBYICDh0ugHvHRFDzv6qevgD4X8qo16L/JwUKeTkDrBA01brhuNQMDy8LHaBhaSNEsPNmB8DlgHRn6tEmLnozfqBL98RX+fzNwBfc/ABDXDFZODqGcCI2wC9seuv1Z0FAuIg868+BE5+AHy1O/LfCxCDRcZVYouNqzZ83JIGfOcO4Dt3imOZuvKDpqkGePc/xB/0AR8AjRgyb1gqzuZrz7F/ANt/Ls580xmBG0uBa+ezqZ26p+NvA38qFm/PfAPI+Z7yr/nag+IMq8R+4vga20B5zy8IQKOjVYD5RLyu/yp6+cR+gH0M0Hew+H+/PjgutKX+0q+lMwLJmZEzfdNGiX9syYihJoqeFmou5GzxYvexOrx7pA7vHanFN/XNEY+n28y4Lrc/vjcsFaMzbcjqmwit9oKQ4/eJ4yIObgIO7wgPXNboxG6pvOnigE+DOT5hJpoWJ/DZq+Jrf/Vh+HhiqjjL5uqZQL8hyrx2vAUCQPWhyBDTekA1AOjNYkjJniS2xgwcL3br+H3AyffFrrzPX4/8AWTLEgPOqLvEQdkd7cpzNwK7nwN2Pw94g+O7cm8GblwB2Ed1/H256sQf3Id3iPdzJotLxydndPwcREpzN4rdTg2nxBbIW1fH53W9LcD/3iSGjoHXAD/dJv4fFQLiH3jStT98X/CLPy8uOha8XV8V2QrT+g+e1myDxNZT+5jwdXJG9J8RLU5xgkso5DScCt4OjgdtPCO+9oVyJgMzX5P1I2OoiaKnh5rWBEHA8VoX3j1Si/eO1OKjL8/C7Yv8ciUYdMhNs2BYmhUj7FYMS7NiuN2KAVaT2KLT/K24L8knLwFffxx+otkGjPiB+AsznmEmmnMnxAB24E+R3S5XXC+O3xk+tee03vh9YmBpOAV8VSEGma92X/zXkCERyJoADJ4kDlbMHHfpAdQ+D/Dl22LA+eKv4tICISlDxHAz6i5gwIi2n79vo9g6c75OPJY5Dih8vPN/uQqCeM6/LwW85wFzH+CH64CRt3fufERye30hsO9FoM9gsdvJZInfa587AbwwWRxArASNFkgdFhle7KPlnTDi94rjcKRJL8HQ0384UDBfvtcBQ01UvSnUXKjF68eeE+fwbjDgHK1pgscXJUED6JNoEANOmhXD7GLgGaFzwHp4qzi2xflNuHBSf3EPk/H/X3zDzIX8PuDo34G9L4rdGxDC9bvqp2LrTUpOfOvkcQHnz4qtEufPtrodvO862+p2XdtNuUaLuPvv4O+KrTHpeV0Lat5mcZD4oVfEmWa+lvBjA0YCo+4Uu6j6DRH/8vvXNnHczLcnxTIpQ4DCFcCVP5RnsHbdUeCVfwPOHBTv5/0UuOU33WYAIsWB3ycuUnr2aHjpCZ0huByF4RL3W11Cx8y2ro8fi+h2eh3Iua7LbzNmR94Ets1t42eDRgwmWl2rtcp0YjeuRnvxscTUVi0wY8X/62r+zJYZQ00UvTnUXMgfEPDVWReOVDfiC0cjjlQ34rCjESfqXAi08a9tTzZjeFoiihKP4hpPJUz9r4B23EwM6JcCo74bjYf49iux5Wb/n4AmR/j4kO8HW29ulWdGUCAA1J8Eqv8lzg6o+Zf4i//8OTGk+JovdYYoNGL/deY4sRVm8CTxB1CUVaZl4W4EDv9NbJE79o/wVHpAHJcT8ItN1YDYGnf9o+L4JblnVPk8wDtlwAe/BSCI64Dc+T9A1jXyvg6FeZvFLs2vPxbHPGReDfS/UrnvWmsBvzg54eT7wIn3gaoKwO2U7/warfgHTdIAcRagJU28bxlw8e2ElIvHc7kbgf+cCDRUiSv8Tl0jX91i5fMAfvcFIaUHLLYaZww1UVxOoaYtLV4/jtc2BUNOEw47nDhS3XTR+JwLpVpMSLeZYbeZI6+TE6T7ZkOct0Dwe4EjO8XWm+NvQWq9saQFW29miLMZOsJ1Vgwt1cFLzWfizKzQuJK26IziX0hJ/cSwkpgqLiaYGLwv3Q4eT+ir3lYRzd+Kg7APvQKceDfcF260At9dABT8XNlZH4A4g+sv94vN1BodMPmX4kyqePyivRycPQ4c3QUc2yUGmtatdACgTxD/ms+4Wgw5meOAlCu6/gs0EBCn8594XwwyJz+4uPXBbBMDPDTBZSi84iB0v/eC+77gUhVt3I+VRif+37MMCIagAWJr9In3xIGtP6uIb7cTdQpDTRQMNW1ztnhxtLoJh1u16pxuaMaZhpY2u7Eu1CfRAHtyKPSIYae/1YRkswHJCXpYzQYkm4PXCXqY9DL+cv/2pDib4MD/ibNuAAAasfVm/Gxg2BSx9cHbIs4uqvksMsBIz7mAziT2D6d9R2zOTc0V/woMhRaTtWf+RdVUC3z+qtiFlndvfFd1bq4HdiwSlxsAgKx84I7/jn/3YW8Qao05+qYYZr49Efl48kCxS7PxNHD6YPTWErNNbLULBZ2Mq9seOBoSWmrg5HtiODj5gdjN2prRKu5unf09cVyWfUzXA70giCHo/FlxVmBTrfh/96LbwcuFg+4vNOM1cXYldXsMNVEw1MROEAR8e96LMw3NcDS04ExDS/ja2SzdP+9pZ52DNhj1WjHwmPWwJojXyWYDrGY9khMMsJrE6ySTHgFBgD8gwOcPwBcQ4PML8AUE+AMBeP3iY95AAILXiyHfvodxddsxtDE8+LlB1w9eQxJSWr6GFm2EtL7ZwIDviPtSDRgpBpmUIWxFUMqnLwN/fVj8RWu0AmPuFsdLSD+OgteXut/6mEYjjsnQGcV/N50xeD90MQbHZVyqjFEcnK0zidd6U/CYOVhexe7Y9lpjtAYxSAy9SVzVuf+IcDgJBIBzx8WFFL/ZD5zeL86U8bsvfg1LWmTIybxabOkLdSed/CCy6xcQB7gPujYYYq4Tx4ap/X/H7xVnATXVBK+rw7fT84Cx09StH3UYQ00UDDXKEAQBzhYfqp2h0BMOO3VNbjhbfHA2e9HY4oOzxYsmtw/x+MZlaaoxXfc27ta9g/6a8F+o5wQLDguD8I0xB+f7DIfWPgopOWMwdKAdOalJMOi60fih3u7br8TuqKoKtWsSG61BDDh6Y/TgozeJXXnmPkBCH7E1JOrt4H1jUtstI57zYog4tit6a4wtS1yOIfcmMUzEMgDb7xVbKkMh55sD4v32FmML0ZnEWXo514lBJnNcz5mJSD0OQ00UDDXdQyAgoMnjE0NOKOw0e9Ho9sLZ7ENjixfOluB1sw9Nbh90Wg10Wg0MOg10Wi30Wo140YnH9cFjOp0GBq02eEwDvU4LI3zI/LYSZ10+7HalobLGiLoLVmoOMeg0GNLfIk1/Hx68zuyTcPGaPx18rx5/AG5fAB5fAG6fH/6AgJQkIywmfduLJV5OAn7gk83h2VdAq1/wmtiOCQFx7IU/eAl4xbWY/D7xuvVjfk9wzIbn4vKhwZu+4CVaa4actPpwwAmFnYQ+YlfdVx92vDVGDp7z4uBxKejsF1t4tAZxjaRQiBl4jbieFVEcMNREwVBDIWeb3Dhc3YgjjkYcDo4hOlLdhCZ39IGISUYdctPEcOPxhwOKeC3e9/gDcHsDEY97/W3/10ow6DAg2YQBVhMGWMXxRwOSTUizmoPHzRhgNaFPooHhR22CEAw77uB1ywW3LwhBvhZxraCWBjGYtNS3fbsjg1+70hojh5YGMdT0oinC1LMw1ETBUEPtEQQB39Q343Aw6BxxiNPhj9c2tRtOYmHUa6HVAC3ejg2+BgCjTisFnlAAGmA1wWTQIiCI0/cDAQF+odW1ILYS+S9xXAMNzAYtEgw6mA06JBh1MOm1SDDqYNaL980GLcx6HcwXHAs9x6TXMnR1liCICxO2FXw0WnF11v7De+aAdCKZMNREwVBDneH1B3CyzoUvHI042+SGyaCDUaeFUa+FSS9eh26b9Drxvk4Lk0ErlQsdC/3yP+/xocbpRk2jGzWNLeHbzpbwsUY36s97L1E79Wk0QJJRjySTDkkmPawmPZJMelhCF3Pk/QsfswSfl2jQw2TQMiQR0UVi+f3NqR1E7TDotMhNsyI3Tb4m/0SjHtmpemSntr8ujNvnR21jKPC4UdvYgmqnG7WNbnj9AWi1Gmg1gE6rgVajka7F24BWq4EueD90W6cFNMGygiCuXRS+BNAcuu0LoMXjR4tPvN8cfDxUNtR6JQhAk9sX7LqTZ+yJSa+F2SC2CJn0wdYig9hSZGp1LOKxYIuS2JKkQ0Kw5SnBGLxtCN82G3RIDJbTdWKsFBF1Xww1RN2USa/DwL6JGNi3+41l8PkDaPEF0Ozx43xw4LcrGG5CF5fbh6YWH5rcfjS5vXC5/WiMOB4u52u11LU7OFapoTOLNsfIqNdGhB6zQQeDLjjQPDToXBceeK6XBq1rWw1ejxysrtdqoIEGAsQuP0EQuzcFiN1/AoCAIEQeD94XPwYBgYAYSq1msfXLatbDElzywBpc/sASPG41G7rXqt9EKmKoIaKY6XVaWHRaWEx6AJfYcPMSBEGA1y/A7Qu3Brl9oWvxWLTHLjweujR7/Wj2ii1NzV4xdIVaoZqDx0I8wYHeDc3dv6uvPSa9Vgo4oeAjhh5DOLDptVHHRpkM4Ras1uOlQvdbd50SdXcMNUSkKo1GA6NeA6NeC2scZgkLggC3L4DzwYDT7GkVhjx++AIBaYHHCxd5DC0A6Q8IwWORC0KGHhffF6CBRlwTUCO+z9Ax8T6g1WjECema4LFWj/kCgtSi1Xqdp8bgkgdNLT64ggtfun0BuJs8qGuKvlxBV2g1kAaFX3htauv4BV2EobJGnRYGfWi8mQZGndgyFj6mlcoYdBqYdDoY9BoYgy1jDFd0KQw1RHRZ0Wg0UktET+cPBp9GtzcYdnxococWuxS7+Zq9frilcVHBVqwLWrZavGJXotsXbs0K9QgGBOC8xx9cOVy9Fi2NRhzjlmjUITHY+pRo1Iv3g7cTjDokGXVIaHVcPKYPlg8eM+ilx0JjrbjwZu/AUENE1EPptBrYEg2wJcq7s3qoS7DF5xcHjLfq6ovlOtyNGIDb64fHL8AbXNfJ6w+v8RS67Q0+7g7ej6xTuLuwXoFwZdBppEHkYpedvtVt8To0wDzRqJP2rxMQHjQv3m9VYUQeC5cJPyYO4Ne2Wky09XV4HFfk49qIcsbgUgyhuiYa9DAbL8+uQ4YaIiKK0LpLMNksb2DqKEEQu/S8/gC8PgFuv7jgZYtXbDVyuf1o9vqkVqTzbh/OB7sQzwcHsIcea/b44fL4Wj3mD57HJ7VIef0CvH6xtau30GogBbQEozZ8Ozh2KtGoD65RpUWiUS8NPg+Nx0o2i0svWFsNTO/uyy4w1BARUbej0Yizyww6LWAEAPnDlSCIW5k0S4PKw91vzRHhJ3TMJ5Vz+wLSph2tf8cHR0ldcCz8ni4UCIY3f6uNekPjtiKvxbFeFx73BVu6QnVsveRCQABcHr809koOBp0mIuSEAlBogPpwuxX35g+W7fVixVBDRESXJY1GA5Ne7Erqo3ZlZOT1B9ec8rQKZKHZf61CW+h4qKXL5QmPxWps8QbHaPmka/HcAs65PDjXxh561w3rz1BDRERE8jDotDDo5O06DG1G3NRqQHo4AIUHqGelqLuuFkMNERERtUur1SDZbFBtjFVHcQ4bERER9QoMNURERNQrMNQQERFRr9CpULN+/XpkZ2fDbDYjPz8fe/bsabf81q1bMWLECJjNZowePRo7duyIeFwQBJSWliI9PR0JCQkoLCzE0aNHI8o8+eSTmDhxIhITE9GnT5/OVJuIiIh6sZhDzZYtW1BSUoIVK1Zg//79GDt2LIqKilBTUxO1/O7duzF9+nTMmTMHBw4cQHFxMYqLi3Ho0CGpzKpVq7Bu3Tps2LABlZWVSEpKQlFREVpaWqQyHo8Hd999N372s5914m0SERFRb6cRhFZrOXdAfn4+rrnmGjz//PMAgEAggKysLDz44IN49NFHLyo/bdo0uFwuvPHGG9Kxa6+9Fnl5ediwYQMEQUBGRgYefvhhLFq0CADQ0NCAtLQ0bNy4Effcc0/E+TZu3IiFCxeivr4+pjfqdDphs9nQ0NCA5OTkmJ5LRERE6ojl93dMLTUejwf79u1DYWFh+ARaLQoLC1FRURH1ORUVFRHlAaCoqEgqf+LECTgcjogyNpsN+fn5bZ6zI9xuN5xOZ8SFiIiIeq+YQk1dXR38fj/S0tIijqelpcHhcER9jsPhaLd86DqWc3ZEWVkZbDabdMnKyur0uYiIiKj767Wzn5YsWYKGhgbpcurUKbWrRERERAqKKdSkpqZCp9Ohuro64nh1dTXsdnvU59jt9nbLh65jOWdHmEwmJCcnR1yIiIio94op1BiNRowbNw7l5eXSsUAggPLychQUFER9TkFBQUR5ANi1a5dUPicnB3a7PaKM0+lEZWVlm+ckIiIiulDMez+VlJRg5syZGD9+PCZMmIC1a9fC5XJh9uzZAIAZM2YgMzMTZWVlAIAFCxZg8uTJWLNmDaZOnYrNmzdj7969eOGFFwCIu6QuXLgQK1euRG5uLnJycrB8+XJkZGSguLhYet2qqiqcO3cOVVVV8Pv9OHjwIABg6NChsFgsXfwYiIiIqKeLOdRMmzYNtbW1KC0thcPhQF5eHnbu3CkN9K2qqoJWG24AmjhxIjZt2oRly5Zh6dKlyM3Nxfbt2zFq1CipzOLFi+FyuTBv3jzU19dj0qRJ2LlzJ8xms1SmtLQUf/jDH6T7V111FQDg7bffxvXXXx/zGyciIqLeJeZ1anqqhoYG9OnTB6dOneL4GiIioh7C6XQiKysL9fX1sNls7ZaNuaWmp2psbAQATu0mIiLqgRobGy8Zai6blppAIIDTp0/DarVCo9HIeu5QimQrkLz4uSqHn61y+Nkqh5+tMrr75yoIAhobG5GRkRExvCWay6alRqvVYuDAgYq+BqeOK4Ofq3L42SqHn61y+Nkqozt/rpdqoQnptYvvERER0eWFoYaIiIh6BYYaGZhMJqxYsQImk0ntqvQq/FyVw89WOfxslcPPVhm96XO9bAYKExERUe/GlhoiIiLqFRhqiIiIqFdgqCEiIqJegaGGiIiIegWGmi5av349srOzYTabkZ+fjz179qhdpR7vscceg0ajibiMGDFC7Wr1SO+99x5uu+02ZGRkQKPRYPv27RGPC4KA0tJSpKenIyEhAYWFhTh69Kg6le1hLvXZzpo166Lv8ZQpU9SpbA9SVlaGa665BlarFQMGDEBxcTEOHz4cUaalpQXz589Hv379YLFYcNddd6G6ulqlGvccHflsr7/++ou+tw888IBKNY4dQ00XbNmyBSUlJVixYgX279+PsWPHoqioCDU1NWpXrcf7zne+gzNnzkiXDz74QO0q9Ugulwtjx47F+vXroz6+atUqrFu3Dhs2bEBlZSWSkpJQVFSElpaWONe057nUZwsAU6ZMifgev/TSS3GsYc/07rvvYv78+fjoo4+wa9cueL1e3HzzzXC5XFKZhx56CK+//jq2bt2Kd999F6dPn8add96pYq17ho58tgAwd+7ciO/tqlWrVKpxJwjUaRMmTBDmz58v3ff7/UJGRoZQVlamYq16vhUrVghjx45Vuxq9DgDhL3/5i3Q/EAgIdrtdWL16tXSsvr5eMJlMwksvvaRCDXuuCz9bQRCEmTNnCrfffrsq9elNampqBADCu+++KwiC+B01GAzC1q1bpTKff/65AECoqKhQq5o90oWfrSAIwuTJk4UFCxaoV6kuYktNJ3k8Huzbtw+FhYXSMa1Wi8LCQlRUVKhYs97h6NGjyMjIwBVXXIF7770XVVVValep1zlx4gQcDkfEd9hmsyE/P5/fYZm88847GDBgAIYPH46f/exnOHv2rNpV6nEaGhoAACkpKQCAffv2wev1RnxvR4wYgUGDBvF7G6MLP9uQP//5z0hNTcWoUaOwZMkSnD9/Xo3qdcpls6Gl3Orq6uD3+5GWlhZxPC0tDV988YVKteod8vPzsXHjRgwfPhxnzpzB448/ju9973s4dOgQrFar2tXrNRwOBwBE/Q6HHqPOmzJlCu68807k5OTg+PHjWLp0KW655RZUVFRAp9OpXb0eIRAIYOHChfjud7+LUaNGARC/t0ajEX369Ikoy+9tbKJ9tgDwk5/8BIMHD0ZGRgY+/fRT/PKXv8Thw4exbds2FWvbcQw11O3ccsst0u0xY8YgPz8fgwcPxssvv4w5c+aoWDOijrvnnnuk26NHj8aYMWMwZMgQvPPOO7jxxhtVrFnPMX/+fBw6dIhj6hTQ1mc7b9486fbo0aORnp6OG2+8EcePH8eQIUPiXc2Ysfupk1JTU6HT6S4acV9dXQ273a5SrXqnPn36YNiwYTh27JjaVelVQt9Tfofj44orrkBqaiq/xx30i1/8Am+88QbefvttDBw4UDput9vh8XhQX18fUZ7f245r67ONJj8/HwB6zPeWoaaTjEYjxo0bh/LyculYIBBAeXk5CgoKVKxZ79PU1ITjx48jPT1d7ar0Kjk5ObDb7RHfYafTicrKSn6HFfD111/j7Nmz/B5fgiAI+MUvfoG//OUveOutt5CTkxPx+Lhx42AwGCK+t4cPH0ZVVRW/t5dwqc82moMHDwJAj/nesvupC0pKSjBz5kyMHz8eEyZMwNq1a+FyuTB79my1q9ajLVq0CLfddhsGDx6M06dPY8WKFdDpdJg+fbraVetxmpqaIv7COnHiBA4ePIiUlBQMGjQICxcuxMqVK5Gbm4ucnBwsX74cGRkZKC4uVq/SPUR7n21KSgoef/xx3HXXXbDb7Th+/DgWL16MoUOHoqioSMVad3/z58/Hpk2b8Oqrr8JqtUrjZGw2GxISEmCz2TBnzhyUlJQgJSUFycnJePDBB1FQUIBrr71W5dp3b5f6bI8fP45Nmzbh1ltvRb9+/fDpp5/ioYcewnXXXYcxY8aoXPsOUnv6VU/33HPPCYMGDRKMRqMwYcIE4aOPPlK7Sj3etGnThPT0dMFoNAqZmZnCtGnThGPHjqldrR7p7bffFgBcdJk5c6YgCOK07uXLlwtpaWmCyWQSbrzxRuHw4cPqVrqHaO+zPX/+vHDzzTcL/fv3FwwGgzB48GBh7ty5gsPhULva3V60zxSA8OKLL0plmpubhZ///OdC3759hcTEROGOO+4Qzpw5o16le4hLfbZVVVXCddddJ6SkpAgmk0kYOnSo8MgjjwgNDQ3qVjwGGkEQhHiGKCIiIiIlcEwNERER9QoMNURERNQrMNQQERFRr8BQQ0RERL0CQw0RERH1Cgw1RERE1Csw1BAREVGvwFBDREREvQJDDREREfUKDDVERETUKzDUEBERUa/AUENERES9wv8PRgoqFCH6YgoAAAAASUVORK5CYII=", "text/plain": [ "