深度残差网络+自适应参数化ReLU激活函数（调参记录9）Cifar10~93.71%

Posted 2021-02-19 shisuzanian

本文在调参记录6的基础上，继续调整超参数，测试Adaptively Parametric ReLU（APReLU）激活函数在Cifar10图像集上的效果。

深度残差网络+自适应参数化ReLU激活函数（调参记录6）
https://www.cnblogs.com/shisuzanian/p/12907482.html

自适应参数化ReLU激活函数的基本原理见下图：

在Keras里，Batch Normalization的momentum默认为0.99，现在设置为0.9，这是因为momentum=0.9似乎更常见。原先Batch Normalization默认没有正则化，现在加上L2正则化，来减小过拟合。

Keras程序如下：

  1 #!/usr/bin/env python3
  2 # -*- coding: utf-8 -*-
  3 """
  4 Created on Tue Apr 14 04:17:45 2020
  5 Implemented using TensorFlow 1.0.1 and Keras 2.2.1
  6 
  7 Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,
  8 Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, 
  9 IEEE Transactions on Industrial Electronics, 2020,  DOI: 10.1109/TIE.2020.2972458 
 10 
 11 @author: Minghang Zhao
 12 """
 13 
 14 from __future__ import print_function
 15 import keras
 16 import numpy as np
 17 from keras.datasets import cifar10
 18 from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimum
 19 from keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshape
 20 from keras.regularizers import l2
 21 from keras import backend as K
 22 from keras.models import Model
 23 from keras import optimizers
 24 from keras.preprocessing.image import ImageDataGenerator
 25 from keras.callbacks import LearningRateScheduler
 26 K.set_learning_phase(1)
 27 
 28 # The data, split between train and test sets
 29 (x_train, y_train), (x_test, y_test) = cifar10.load_data()
 30 
 31 # Noised data
 32 x_train = x_train.astype(‘float32‘) / 255.
 33 x_test = x_test.astype(‘float32‘) / 255.
 34 x_test = x_test-np.mean(x_train)
 35 x_train = x_train-np.mean(x_train)
 36 print(‘x_train shape:‘, x_train.shape)
 37 print(x_train.shape[0], ‘train samples‘)
 38 print(x_test.shape[0], ‘test samples‘)
 39 
 40 # convert class vectors to binary class matrices
 41 y_train = keras.utils.to_categorical(y_train, 10)
 42 y_test = keras.utils.to_categorical(y_test, 10)
 43 
 44 # Schedule the learning rate, multiply 0.1 every 300 epoches
 45 def scheduler(epoch):
 46     if epoch % 300 == 0 and epoch != 0:
 47         lr = K.get_value(model.optimizer.lr)
 48         K.set_value(model.optimizer.lr, lr * 0.1)
 49         print("lr changed to {}".format(lr * 0.1))
 50     return K.get_value(model.optimizer.lr)
 51 
 52 # An adaptively parametric rectifier linear unit (APReLU)
 53 def aprelu(inputs):
 54     # get the number of channels
 55     channels = inputs.get_shape().as_list()[-1]
 56     # get a zero feature map
 57     zeros_input = keras.layers.subtract([inputs, inputs])
 58     # get a feature map with only positive features
 59     pos_input = Activation(‘relu‘)(inputs)
 60     # get a feature map with only negative features
 61     neg_input = Minimum()([inputs,zeros_input])
 62     # define a network to obtain the scaling coefficients
 63     scales_p = GlobalAveragePooling2D()(pos_input)
 64     scales_n = GlobalAveragePooling2D()(neg_input)
 65     scales = Concatenate()([scales_n, scales_p])
 66     scales = Dense(channels, activation=‘linear‘, kernel_initializer=‘he_normal‘, kernel_regularizer=l2(1e-4))(scales)
 67     scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
 68     scales = Activation(‘relu‘)(scales)
 69     scales = Dense(channels, activation=‘linear‘, kernel_initializer=‘he_normal‘, kernel_regularizer=l2(1e-4))(scales)
 70     scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
 71     scales = Activation(‘sigmoid‘)(scales)
 72     scales = Reshape((1,1,channels))(scales)
 73     # apply a paramtetric relu
 74     neg_part = keras.layers.multiply([scales, neg_input])
 75     return keras.layers.add([pos_input, neg_part])
 76 
 77 # Residual Block
 78 def residual_block(incoming, nb_blocks, out_channels, downsample=False,
 79                    downsample_strides=2):
 80     
 81     residual = incoming
 82     in_channels = incoming.get_shape().as_list()[-1]
 83     
 84     for i in range(nb_blocks):
 85         
 86         identity = residual
 87         
 88         if not downsample:
 89             downsample_strides = 1
 90         
 91         residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
 92         residual = aprelu(residual)
 93         residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides), 
 94                           padding=‘same‘, kernel_initializer=‘he_normal‘, 
 95                           kernel_regularizer=l2(1e-4))(residual)
 96         
 97         residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
 98         residual = aprelu(residual)
 99         residual = Conv2D(out_channels, 3, padding=‘same‘, kernel_initializer=‘he_normal‘, 
100                           kernel_regularizer=l2(1e-4))(residual)
101         
102         # Downsampling
103         if downsample_strides > 1:
104             identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity)
105             
106         # Zero_padding to match channels
107         if in_channels != out_channels:
108             zeros_identity = keras.layers.subtract([identity, identity])
109             identity = keras.layers.concatenate([identity, zeros_identity])
110             in_channels = out_channels
111         
112         residual = keras.layers.add([residual, identity])
113     
114     return residual
115 
116 
117 # define and train a model
118 inputs = Input(shape=(32, 32, 3))
119 net = Conv2D(16, 3, padding=‘same‘, kernel_initializer=‘he_normal‘, kernel_regularizer=l2(1e-4))(inputs)
120 net = residual_block(net, 9, 16, downsample=False)
121 net = residual_block(net, 1, 32, downsample=True)
122 net = residual_block(net, 8, 32, downsample=False)
123 net = residual_block(net, 1, 64, downsample=True)
124 net = residual_block(net, 8, 64, downsample=False)
125 net = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(net)
126 net = Activation(‘relu‘)(net)
127 net = GlobalAveragePooling2D()(net)
128 outputs = Dense(10, activation=‘softmax‘, kernel_initializer=‘he_normal‘, kernel_regularizer=l2(1e-4))(net)
129 model = Model(inputs=inputs, outputs=outputs)
130 sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)
131 model.compile(loss=‘categorical_crossentropy‘, optimizer=sgd, metrics=[‘accuracy‘])
132 
133 # data augmentation
134 datagen = ImageDataGenerator(
135     # randomly rotate images in the range (deg 0 to 180)
136     rotation_range=30,
137     # randomly flip images
138     horizontal_flip=True,
139     # randomly shift images horizontally
140     width_shift_range=0.125,
141     # randomly shift images vertically
142     height_shift_range=0.125)
143 
144 reduce_lr = LearningRateScheduler(scheduler)
145 # fit the model on the batches generated by datagen.flow().
146 model.fit_generator(datagen.flow(x_train, y_train, batch_size=100),
147                     validation_data=(x_test, y_test), epochs=1000, 
148                     verbose=1, callbacks=[reduce_lr], workers=4)
149 
150 # get results
151 K.set_learning_phase(0)
152 DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0)
153 print(‘Train loss:‘, DRSN_train_score[0])
154 print(‘Train accuracy:‘, DRSN_train_score[1])
155 DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0)
156 print(‘Test loss:‘, DRSN_test_score[0])
157 print(‘Test accuracy:‘, DRSN_test_score[1])

实验结果如下：

   1 x_train shape: (50000, 32, 32, 3)
   2 50000 train samples
   3 10000 test samples
   4 Epoch 1/1000
   5 97s 195ms/step - loss: 3.2344 - acc: 0.4133 - val_loss: 2.7840 - val_acc: 0.5398
   6 Epoch 2/1000
   7 65s 131ms/step - loss: 2.6095 - acc: 0.5574 - val_loss: 2.3084 - val_acc: 0.6296
   8 Epoch 3/1000
   9 65s 131ms/step - loss: 2.2160 - acc: 0.6249 - val_loss: 1.9625 - val_acc: 0.6837
  10 Epoch 4/1000
  11 65s 131ms/step - loss: 1.9251 - acc: 0.6702 - val_loss: 1.7395 - val_acc: 0.7116
  12 Epoch 5/1000
  13 65s 131ms/step - loss: 1.7015 - acc: 0.7016 - val_loss: 1.5316 - val_acc: 0.7429
  14 Epoch 6/1000
  15 65s 131ms/step - loss: 1.5268 - acc: 0.7228 - val_loss: 1.3858 - val_acc: 0.7608
  16 Epoch 7/1000
  17 65s 131ms/step - loss: 1.3979 - acc: 0.7372 - val_loss: 1.2604 - val_acc: 0.7761
  18 Epoch 8/1000
  19 65s 131ms/step - loss: 1.2921 - acc: 0.7483 - val_loss: 1.1713 - val_acc: 0.7798
  20 Epoch 9/1000
  21 66s 131ms/step - loss: 1.2057 - acc: 0.7627 - val_loss: 1.1200 - val_acc: 0.7846
  22 Epoch 10/1000
  23 65s 131ms/step - loss: 1.1358 - acc: 0.7690 - val_loss: 1.0900 - val_acc: 0.7811
  24 Epoch 11/1000
  25 65s 131ms/step - loss: 1.0823 - acc: 0.7741 - val_loss: 0.9822 - val_acc: 0.8058
  26 Epoch 12/1000
  27 65s 131ms/step - loss: 1.0365 - acc: 0.7802 - val_loss: 0.9840 - val_acc: 0.7976
  28 Epoch 13/1000
  29 65s 130ms/step - loss: 1.0040 - acc: 0.7847 - val_loss: 0.9539 - val_acc: 0.7995
  30 Epoch 14/1000
  31 65s 131ms/step - loss: 0.9737 - acc: 0.7870 - val_loss: 0.9181 - val_acc: 0.8093
  32 Epoch 15/1000
  33 65s 131ms/step - loss: 0.9468 - acc: 0.7933 - val_loss: 0.8972 - val_acc: 0.8071
  34 Epoch 16/1000
  35 65s 131ms/step - loss: 0.9210 - acc: 0.7964 - val_loss: 0.9039 - val_acc: 0.8077
  36 Epoch 17/1000
  37 65s 131ms/step - loss: 0.9084 - acc: 0.8008 - val_loss: 0.8491 - val_acc: 0.8200
  38 Epoch 18/1000
  39 65s 131ms/step - loss: 0.8879 - acc: 0.8027 - val_loss: 0.8565 - val_acc: 0.8161
  40 Epoch 19/1000
  41 65s 131ms/step - loss: 0.8770 - acc: 0.8044 - val_loss: 0.8640 - val_acc: 0.8116
  42 Epoch 20/1000
  43 65s 131ms/step - loss: 0.8695 - acc: 0.8066 - val_loss: 0.8369 - val_acc: 0.8187
  44 Epoch 21/1000
  45 65s 131ms/step - loss: 0.8565 - acc: 0.8097 - val_loss: 0.8403 - val_acc: 0.8221
  46 Epoch 22/1000
  47 65s 131ms/step - loss: 0.8516 - acc: 0.8119 - val_loss: 0.8131 - val_acc: 0.8315
  48 Epoch 23/1000
  49 65s 131ms/step - loss: 0.8402 - acc: 0.8156 - val_loss: 0.7879 - val_acc: 0.8397
  50 Epoch 24/1000
  51 65s 131ms/step - loss: 0.8271 - acc: 0.8179 - val_loss: 0.7942 - val_acc: 0.8379
  52 Epoch 25/1000
  53 65s 131ms/step - loss: 0.8282 - acc: 0.8196 - val_loss: 0.8132 - val_acc: 0.8270
  54 Epoch 26/1000
  55 65s 130ms/step - loss: 0.8203 - acc: 0.8203 - val_loss: 0.7870 - val_acc: 0.8354
  56 Epoch 27/1000
  57 65s 131ms/step - loss: 0.8141 - acc: 0.8231 - val_loss: 0.7780 - val_acc: 0.8405
  58 Epoch 28/1000
  59 65s 131ms/step - loss: 0.8075 - acc: 0.8270 - val_loss: 0.7806 - val_acc: 0.8386
  60 Epoch 29/1000
  61 65s 131ms/step - loss: 0.8051 - acc: 0.8260 - val_loss: 0.7865 - val_acc: 0.8309
  62 Epoch 30/1000
  63 65s 131ms/step - loss: 0.8015 - acc: 0.8262 - val_loss: 0.7600 - val_acc: 0.8458
  64 Epoch 31/1000
  65 65s 131ms/step - loss: 0.7948 - acc: 0.8295 - val_loss: 0.7560 - val_acc: 0.8458
  66 Epoch 32/1000
  67 65s 131ms/step - loss: 0.7890 - acc: 0.8323 - val_loss: 0.7760 - val_acc: 0.8407
  68 Epoch 33/1000
  69 65s 131ms/step - loss: 0.7868 - acc: 0.8335 - val_loss: 0.7845 - val_acc: 0.8348
  70 Epoch 34/1000
  71 66s 131ms/step - loss: 0.7845 - acc: 0.8346 - val_loss: 0.7517 - val_acc: 0.8460
  72 Epoch 35/1000
  73 65s 131ms/step - loss: 0.7764 - acc: 0.8377 - val_loss: 0.7683 - val_acc: 0.8432
  74 Epoch 36/1000
  75 65s 131ms/step - loss: 0.7720 - acc: 0.8370 - val_loss: 0.7383 - val_acc: 0.8518
  76 Epoch 37/1000
  77 65s 131ms/step - loss: 0.7738 - acc: 0.8374 - val_loss: 0.7491 - val_acc: 0.8469
  78 Epoch 38/1000
  79 65s 131ms/step - loss: 0.7666 - acc: 0.8405 - val_loss: 0.7591 - val_acc: 0.8437
  80 Epoch 39/1000
  81 65s 131ms/step - loss: 0.7656 - acc: 0.8421 - val_loss: 0.7389 - val_acc: 0.8533
  82 Epoch 40/1000
  83 65s 131ms/step - loss: 0.7619 - acc: 0.8431 - val_loss: 0.7583 - val_acc: 0.8461
  84 Epoch 41/1000
  85 65s 130ms/step - loss: 0.7594 - acc: 0.8433 - val_loss: 0.7199 - val_acc: 0.8576
  86 Epoch 42/1000
  87 65s 131ms/step - loss: 0.7594 - acc: 0.8428 - val_loss: 0.7272 - val_acc: 0.8558
  88 Epoch 43/1000
  89 65s 131ms/step - loss: 0.7559 - acc: 0.8451 - val_loss: 0.7353 - val_acc: 0.8535
  90 Epoch 44/1000
  91 65s 131ms/step - loss: 0.7528 - acc: 0.8454 - val_loss: 0.7492 - val_acc: 0.8487
  92 Epoch 45/1000
  93 65s 131ms/step - loss: 0.7564 - acc: 0.8465 - val_loss: 0.7510 - val_acc: 0.8505
  94 Epoch 46/1000
  95 65s 131ms/step - loss: 0.7494 - acc: 0.8487 - val_loss: 0.7626 - val_acc: 0.8462
  96 Epoch 47/1000
  97 65s 131ms/step - loss: 0.7505 - acc: 0.8491 - val_loss: 0.7417 - val_acc: 0.8561
  98 Epoch 48/1000
  99 65s 131ms/step - loss: 0.7434 - acc: 0.8509 - val_loss: 0.7247 - val_acc: 0.8580
 100 Epoch 49/1000
 101 65s 131ms/step - loss: 0.7426 - acc: 0.8502 - val_loss: 0.7203 - val_acc: 0.8612
 102 Epoch 50/1000
 103 65s 130ms/step - loss: 0.7436 - acc: 0.8503 - val_loss: 0.7190 - val_acc: 0.8621
 104 Epoch 51/1000
 105 65s 130ms/step - loss: 0.7415 - acc: 0.8509 - val_loss: 0.7315 - val_acc: 0.8590
 106 Epoch 52/1000
 107 65s 130ms/step - loss: 0.7342 - acc: 0.8549 - val_loss: 0.7141 - val_acc: 0.8627
 108 Epoch 53/1000
 109 65s 130ms/step - loss: 0.7341 - acc: 0.8525 - val_loss: 0.7209 - val_acc: 0.8582
 110 Epoch 54/1000
 111 65s 130ms/step - loss: 0.7326 - acc: 0.8546 - val_loss: 0.7114 - val_acc: 0.8640
 112 Epoch 55/1000
 113 65s 131ms/step - loss: 0.7338 - acc: 0.8546 - val_loss: 0.7166 - val_acc: 0.8587
 114 Epoch 56/1000
 115 65s 131ms/step - loss: 0.7291 - acc: 0.8564 - val_loss: 0.7109 - val_acc: 0.8642
 116 Epoch 57/1000
 117 65s 131ms/step - loss: 0.7261 - acc: 0.8563 - val_loss: 0.7116 - val_acc: 0.8638
 118 Epoch 58/1000
 119 65s 131ms/step - loss: 0.7270 - acc: 0.8567 - val_loss: 0.7272 - val_acc: 0.8591
 120 Epoch 59/1000
 121 65s 131ms/step - loss: 0.7240 - acc: 0.8577 - val_loss: 0.6949 - val_acc: 0.8730
 122 Epoch 60/1000
 123 65s 130ms/step - loss: 0.7268 - acc: 0.8575 - val_loss: 0.7129 - val_acc: 0.8645
 124 Epoch 61/1000
 125 65s 131ms/step - loss: 0.7222 - acc: 0.8599 - val_loss: 0.7174 - val_acc: 0.8642
 126 Epoch 62/1000
 127 65s 131ms/step - loss: 0.7195 - acc: 0.8611 - val_loss: 0.7178 - val_acc: 0.8608
 128 Epoch 63/1000
 129 65s 131ms/step - loss: 0.7177 - acc: 0.8619 - val_loss: 0.7142 - val_acc: 0.8586
 130 Epoch 64/1000
 131 65s 131ms/step - loss: 0.7146 - acc: 0.8632 - val_loss: 0.7119 - val_acc: 0.8619
 132 Epoch 65/1000
 133 65s 131ms/step - loss: 0.7174 - acc: 0.8599 - val_loss: 0.7174 - val_acc: 0.8640
 134 Epoch 66/1000
 135 65s 131ms/step - loss: 0.7145 - acc: 0.8619 - val_loss: 0.7075 - val_acc: 0.8647
 136 Epoch 67/1000
 137 65s 131ms/step - loss: 0.7116 - acc: 0.8635 - val_loss: 0.7449 - val_acc: 0.8534
 138 Epoch 68/1000
 139 65s 131ms/step - loss: 0.7058 - acc: 0.8632 - val_loss: 0.6978 - val_acc: 0.8713
 140 Epoch 69/1000
 141 65s 131ms/step - loss: 0.7111 - acc: 0.8632 - val_loss: 0.7132 - val_acc: 0.8641
 142 Epoch 70/1000
 143 66s 131ms/step - loss: 0.7046 - acc: 0.8655 - val_loss: 0.6695 - val_acc: 0.8764
 144 Epoch 71/1000
 145 66s 131ms/step - loss: 0.7062 - acc: 0.8640 - val_loss: 0.6967 - val_acc: 0.8704
 146 Epoch 72/1000
 147 66s 131ms/step - loss: 0.7044 - acc: 0.8655 - val_loss: 0.6786 - val_acc: 0.8771
 148 Epoch 73/1000
 149 66s 131ms/step - loss: 0.7018 - acc: 0.8667 - val_loss: 0.7139 - val_acc: 0.8639
 150 Epoch 74/1000
 151 65s 131ms/step - loss: 0.7029 - acc: 0.8667 - val_loss: 0.7264 - val_acc: 0.8565
 152 Epoch 75/1000
 153 65s 131ms/step - loss: 0.6981 - acc: 0.8661 - val_loss: 0.6919 - val_acc: 0.8738
 154 Epoch 76/1000
 155 65s 131ms/step - loss: 0.6997 - acc: 0.8667 - val_loss: 0.7023 - val_acc: 0.8700
 156 Epoch 77/1000
 157 65s 131ms/step - loss: 0.6967 - acc: 0.8685 - val_loss: 0.6810 - val_acc: 0.8769
 158 Epoch 78/1000
 159 65s 131ms/step - loss: 0.6982 - acc: 0.8673 - val_loss: 0.7090 - val_acc: 0.8648
 160 Epoch 79/1000
 161 66s 131ms/step - loss: 0.6989 - acc: 0.8670 - val_loss: 0.7114 - val_acc: 0.8691
 162 Epoch 80/1000
 163 66s 131ms/step - loss: 0.6900 - acc: 0.8704 - val_loss: 0.7039 - val_acc: 0.8707
 164 Epoch 81/1000
 165 66s 131ms/step - loss: 0.6920 - acc: 0.8703 - val_loss: 0.6878 - val_acc: 0.8742
 166 Epoch 82/1000
 167 66s 131ms/step - loss: 0.6904 - acc: 0.8705 - val_loss: 0.6966 - val_acc: 0.8724
 168 Epoch 83/1000
 169 66s 131ms/step - loss: 0.6907 - acc: 0.8694 - val_loss: 0.6880 - val_acc: 0.8725
 170 Epoch 84/1000
 171 65s 131ms/step - loss: 0.6933 - acc: 0.8692 - val_loss: 0.7006 - val_acc: 0.8697
 172 Epoch 85/1000
 173 65s 131ms/step - loss: 0.6934 - acc: 0.8709 - val_loss: 0.7079 - val_acc: 0.8679
 174 Epoch 86/1000
 175 65s 131ms/step - loss: 0.6899 - acc: 0.8710 - val_loss: 0.7029 - val_acc: 0.8661
 176 Epoch 87/1000
 177 66s 131ms/step - loss: 0.6946 - acc: 0.8696 - val_loss: 0.6892 - val_acc: 0.8746
 178 Epoch 88/1000
 179 66s 131ms/step - loss: 0.6925 - acc: 0.8709 - val_loss: 0.6920 - val_acc: 0.8698
 180 Epoch 89/1000
 181 66s 131ms/step - loss: 0.6861 - acc: 0.8703 - val_loss: 0.6857 - val_acc: 0.8762
 182 Epoch 90/1000
 183 66s 131ms/step - loss: 0.6878 - acc: 0.8721 - val_loss: 0.6827 - val_acc: 0.8740
 184 Epoch 91/1000
 185 66s 131ms/step - loss: 0.6845 - acc: 0.8728 - val_loss: 0.6995 - val_acc: 0.8702
 186 Epoch 92/1000
 187 65s 131ms/step - loss: 0.6890 - acc: 0.8719 - val_loss: 0.6769 - val_acc: 0.8767
 188 Epoch 93/1000
 189 66s 131ms/step - loss: 0.6836 - acc: 0.8734 - val_loss: 0.6992 - val_acc: 0.8689
 190 Epoch 94/1000
 191 65s 131ms/step - loss: 0.6809 - acc: 0.8737 - val_loss: 0.7046 - val_acc: 0.8682
 192 Epoch 95/1000
 193 65s 131ms/step - loss: 0.6803 - acc: 0.8727 - val_loss: 0.6755 - val_acc: 0.8793
 194 Epoch 96/1000
 195 65s 131ms/step - loss: 0.6833 - acc: 0.8742 - val_loss: 0.6857 - val_acc: 0.8741
 196 Epoch 97/1000
 197 65s 131ms/step - loss: 0.6837 - acc: 0.8732 - val_loss: 0.6969 - val_acc: 0.8715
 198 Epoch 98/1000
 199 65s 131ms/step - loss: 0.6836 - acc: 0.8738 - val_loss: 0.6762 - val_acc: 0.8763
 200 Epoch 99/1000
 201 65s 131ms/step - loss: 0.6837 - acc: 0.8727 - val_loss: 0.6817 - val_acc: 0.8759
 202 Epoch 100/1000
 203 65s 131ms/step - loss: 0.6809 - acc: 0.8755 - val_loss: 0.6859 - val_acc: 0.8736
 204 Epoch 101/1000
 205 65s 131ms/step - loss: 0.6814 - acc: 0.8745 - val_loss: 0.6695 - val_acc: 0.8816
 206 Epoch 102/1000
 207 65s 131ms/step - loss: 0.6813 - acc: 0.8735 - val_loss: 0.6878 - val_acc: 0.8732
 208 Epoch 103/1000
 209 66s 131ms/step - loss: 0.6852 - acc: 0.8744 - val_loss: 0.6906 - val_acc: 0.8719
 210 Epoch 104/1000
 211 65s 131ms/step - loss: 0.6804 - acc: 0.8753 - val_loss: 0.6803 - val_acc: 0.8779
 212 Epoch 105/1000
 213 65s 131ms/step - loss: 0.6771 - acc: 0.8748 - val_loss: 0.6838 - val_acc: 0.8754
 214 Epoch 106/1000
 215 65s 131ms/step - loss: 0.6741 - acc: 0.8768 - val_loss: 0.7191 - val_acc: 0.8606
 216 Epoch 107/1000
 217 65s 131ms/step - loss: 0.6774 - acc: 0.8751 - val_loss: 0.6901 - val_acc: 0.8725
 218 Epoch 108/1000
 219 65s 131ms/step - loss: 0.6752 - acc: 0.8768 - val_loss: 0.7003 - val_acc: 0.8711
 220 Epoch 109/1000
 221 65s 130ms/step - loss: 0.6772 - acc: 0.8752 - val_loss: 0.6926 - val_acc: 0.8756
 222 Epoch 110/1000
 223 65s 131ms/step - loss: 0.6729 - acc: 0.8775 - val_loss: 0.7088 - val_acc: 0.8647
 224 Epoch 111/1000
 225 65s 131ms/step - loss: 0.6670 - acc: 0.8793 - val_loss: 0.6932 - val_acc: 0.8725
 226 Epoch 112/1000
 227 65s 131ms/step - loss: 0.6724 - acc: 0.8775 - val_loss: 0.6781 - val_acc: 0.8779
 228 Epoch 113/1000
 229 65s 131ms/step - loss: 0.6753 - acc: 0.8771 - val_loss: 0.6676 - val_acc: 0.8815
 230 Epoch 114/1000
 231 65s 131ms/step - loss: 0.6720 - acc: 0.8775 - val_loss: 0.6813 - val_acc: 0.8763
 232 Epoch 115/1000
 233 66s 131ms/step - loss: 0.6754 - acc: 0.8746 - val_loss: 0.6662 - val_acc: 0.8761
 234 Epoch 116/1000
 235 65s 130ms/step - loss: 0.6763 - acc: 0.8758 - val_loss: 0.6668 - val_acc: 0.8798
 236 Epoch 117/1000
 237 65s 131ms/step - loss: 0.6680 - acc: 0.8788 - val_loss: 0.6860 - val_acc: 0.8791
 238 Epoch 118/1000
 239 65s 131ms/step - loss: 0.6737 - acc: 0.8781 - val_loss: 0.6630 - val_acc: 0.8794
 240 Epoch 119/1000
 241 65s 131ms/step - loss: 0.6621 - acc: 0.8812 - val_loss: 0.6847 - val_acc: 0.8772
 242 Epoch 120/1000
 243 65s 131ms/step - loss: 0.6638 - acc: 0.8794 - val_loss: 0.6777 - val_acc: 0.8768
 244 Epoch 121/1000
 245 65s 131ms/step - loss: 0.6682 - acc: 0.8793 - val_loss: 0.7159 - val_acc: 0.8659
 246 Epoch 122/1000
 247 65s 131ms/step - loss: 0.6726 - acc: 0.8762 - val_loss: 0.6771 - val_acc: 0.8803
 248 Epoch 123/1000
 249 65s 131ms/step - loss: 0.6660 - acc: 0.8800 - val_loss: 0.6986 - val_acc: 0.8730
 250 Epoch 124/1000
 251 65s 131ms/step - loss: 0.6697 - acc: 0.8780 - val_loss: 0.6978 - val_acc: 0.8741
 252 Epoch 125/1000
 253 65s 131ms/step - loss: 0.6680 - acc: 0.8803 - val_loss: 0.6767 - val_acc: 0.8787
 254 Epoch 126/1000
 255 65s 131ms/step - loss: 0.6604 - acc: 0.8827 - val_loss: 0.6827 - val_acc: 0.8751
 256 Epoch 127/1000
 257 65s 131ms/step - loss: 0.6647 - acc: 0.8816 - val_loss: 0.7081 - val_acc: 0.8681
 258 Epoch 128/1000
 259 65s 130ms/step - loss: 0.6668 - acc: 0.8808 - val_loss: 0.6697 - val_acc: 0.8780
 260 Epoch 129/1000
 261 65s 131ms/step - loss: 0.6629 - acc: 0.8808 - val_loss: 0.6848 - val_acc: 0.8725
 262 Epoch 130/1000
 263 65s 131ms/step - loss: 0.6634 - acc: 0.8802 - val_loss: 0.6862 - val_acc: 0.8730
 264 Epoch 131/1000
 265 65s 131ms/step - loss: 0.6637 - acc: 0.8797 - val_loss: 0.7044 - val_acc: 0.8704
 266 Epoch 132/1000
 267 65s 131ms/step - loss: 0.6647 - acc: 0.8817 - val_loss: 0.6798 - val_acc: 0.8779
 268 Epoch 133/1000
 269 65s 131ms/step - loss: 0.6604 - acc: 0.8830 - val_loss: 0.6790 - val_acc: 0.8770
 270 Epoch 134/1000
 271 65s 131ms/step - loss: 0.6638 - acc: 0.8821 - val_loss: 0.6786 - val_acc: 0.8777
 272 Epoch 135/1000
 273 65s 131ms/step - loss: 0.6621 - acc: 0.8829 - val_loss: 0.6990 - val_acc: 0.8676
 274 Epoch 136/1000
 275 65s 131ms/step - loss: 0.6650 - acc: 0.8803 - val_loss: 0.6916 - val_acc: 0.8742
 276 Epoch 137/1000
 277 65s 131ms/step - loss: 0.6600 - acc: 0.8814 - val_loss: 0.6645 - val_acc: 0.8822
 278 Epoch 138/1000
 279 65s 131ms/step - loss: 0.6606 - acc: 0.8827 - val_loss: 0.6554 - val_acc: 0.8902
 280 Epoch 139/1000
 281 65s 131ms/step - loss: 0.6575 - acc: 0.8849 - val_loss: 0.6895 - val_acc: 0.8782
 282 Epoch 140/1000
 283 65s 131ms/step - loss: 0.6590 - acc: 0.8824 - val_loss: 0.6689 - val_acc: 0.8830
 284 Epoch 141/1000
 285 65s 131ms/step - loss: 0.6589 - acc: 0.8827 - val_loss: 0.6620 - val_acc: 0.8816
 286 Epoch 142/1000
 287 65s 131ms/step - loss: 0.6580 - acc: 0.8833 - val_loss: 0.6765 - val_acc: 0.8787
 288 Epoch 143/1000
 289 66s 131ms/step - loss: 0.6559 - acc: 0.8830 - val_loss: 0.7018 - val_acc: 0.8691
 290 Epoch 144/1000
 291 65s 131ms/step - loss: 0.6579 - acc: 0.8818 - val_loss: 0.6733 - val_acc: 0.8819
 292 Epoch 145/1000
 293 66s 131ms/step - loss: 0.6559 - acc: 0.8843 - val_loss: 0.6702 - val_acc: 0.8809
 294 Epoch 146/1000
 295 65s 131ms/step - loss: 0.6557 - acc: 0.8826 - val_loss: 0.6474 - val_acc: 0.8871
 296 Epoch 147/1000
 297 65s 131ms/step - loss: 0.6552 - acc: 0.8844 - val_loss: 0.6815 - val_acc: 0.8769
 298 Epoch 148/1000
 299 65s 131ms/step - loss: 0.6565 - acc: 0.8830 - val_loss: 0.6770 - val_acc: 0.8818
 300 Epoch 149/1000
 301 65s 131ms/step - loss: 0.6501 - acc: 0.8852 - val_loss: 0.6885 - val_acc: 0.8764
 302 Epoch 150/1000
 303 65s 131ms/step - loss: 0.6566 - acc: 0.8832 - val_loss: 0.6701 - val_acc: 0.8815
 304 Epoch 151/1000
 305 65s 131ms/step - loss: 0.6521 - acc: 0.8861 - val_loss: 0.6785 - val_acc: 0.8785
 306 Epoch 152/1000
 307 65s 131ms/step - loss: 0.6539 - acc: 0.8851 - val_loss: 0.6681 - val_acc: 0.8841
 308 Epoch 153/1000
 309 65s 131ms/step - loss: 0.6514 - acc: 0.8849 - val_loss: 0.6773 - val_acc: 0.8785
 310 Epoch 154/1000
 311 65s 131ms/step - loss: 0.6561 - acc: 0.8836 - val_loss: 0.6747 - val_acc: 0.8803
 312 Epoch 155/1000
 313 65s 131ms/step - loss: 0.6524 - acc: 0.8852 - val_loss: 0.6545 - val_acc: 0.8854
 314 Epoch 156/1000
 315 65s 131ms/step - loss: 0.6587 - acc: 0.8828 - val_loss: 0.7070 - val_acc: 0.8692
 316 Epoch 157/1000
 317 65s 131ms/step - loss: 0.6558 - acc: 0.8838 - val_loss: 0.6618 - val_acc: 0.8843
 318 Epoch 158/1000
 319 65s 131ms/step - loss: 0.6514 - acc: 0.8873 - val_loss: 0.6874 - val_acc: 0.8763
 320 Epoch 159/1000
 321 65s 131ms/step - loss: 0.6564 - acc: 0.8848 - val_loss: 0.6804 - val_acc: 0.8805
 322 Epoch 160/1000
 323 65s 131ms/step - loss: 0.6450 - acc: 0.8868 - val_loss: 0.6752 - val_acc: 0.8800
 324 Epoch 161/1000
 325 65s 131ms/step - loss: 0.6555 - acc: 0.8847 - val_loss: 0.6589 - val_acc: 0.8857
 326 Epoch 162/1000
 327 65s 131ms/step - loss: 0.6492 - acc: 0.8860 - val_loss: 0.6544 - val_acc: 0.8862
 328 Epoch 163/1000
 329 65s 131ms/step - loss: 0.6544 - acc: 0.8844 - val_loss: 0.6807 - val_acc: 0.8775
 330 Epoch 164/1000
 331 65s 131ms/step - loss: 0.6504 - acc: 0.8850 - val_loss: 0.6861 - val_acc: 0.8761
 332 Epoch 165/1000
 333 65s 131ms/step - loss: 0.6538 - acc: 0.8832 - val_loss: 0.6653 - val_acc: 0.8842
 334 Epoch 166/1000
 335 65s 131ms/step - loss: 0.6520 - acc: 0.8866 - val_loss: 0.6685 - val_acc: 0.8823
 336 Epoch 167/1000
 337 65s 131ms/step - loss: 0.6483 - acc: 0.8869 - val_loss: 0.6916 - val_acc: 0.8719
 338 Epoch 168/1000
 339 65s 131ms/step - loss: 0.6501 - acc: 0.8855 - val_loss: 0.6789 - val_acc: 0.8785
 340 Epoch 169/1000
 341 65s 131ms/step - loss: 0.6484 - acc: 0.8863 - val_loss: 0.6853 - val_acc: 0.8740
 342 Epoch 170/1000
 343 65s 131ms/step - loss: 0.6485 - acc: 0.8863 - val_loss: 0.6654 - val_acc: 0.8808
 344 Epoch 171/1000
 345 65s 131ms/step - loss: 0.6474 - acc: 0.8863 - val_loss: 0.6636 - val_acc: 0.8858
 346 Epoch 172/1000
 347 65s 131ms/step - loss: 0.6469 - acc: 0.8863 - val_loss: 0.6752 - val_acc: 0.8793
 348 Epoch 173/1000
 349 65s 131ms/step - loss: 0.6411 - acc: 0.8886 - val_loss: 0.6869 - val_acc: 0.8769
 350 Epoch 174/1000
 351 65s 130ms/step - loss: 0.6456 - acc: 0.8873 - val_loss: 0.6714 - val_acc: 0.8808
 352 Epoch 175/1000
 353 65s 130ms/step - loss: 0.6536 - acc: 0.8853 - val_loss: 0.6580 - val_acc: 0.8885
 354 Epoch 176/1000
 355 65s 130ms/step - loss: 0.6491 - acc: 0.8857 - val_loss: 0.6743 - val_acc: 0.8816
 356 Epoch 177/1000
 357 65s 130ms/step - loss: 0.6492 - acc: 0.8851 - val_loss: 0.6625 - val_acc: 0.8897
 358 Epoch 178/1000
 359 65s 130ms/step - loss: 0.6481 - acc: 0.8845 - val_loss: 0.6671 - val_acc: 0.8826
 360 Epoch 179/1000
 361 65s 131ms/step - loss: 0.6495 - acc: 0.8854 - val_loss: 0.6968 - val_acc: 0.8724
 362 Epoch 180/1000
 363 65s 131ms/step - loss: 0.6474 - acc: 0.8879 - val_loss: 0.6602 - val_acc: 0.8860
 364 Epoch 181/1000
 365 65s 131ms/step - loss: 0.6449 - acc: 0.8869 - val_loss: 0.6648 - val_acc: 0.8849
 366 Epoch 182/1000
 367 65s 131ms/step - loss: 0.6515 - acc: 0.8849 - val_loss: 0.6675 - val_acc: 0.8812
 368 Epoch 183/1000
 369 65s 131ms/step - loss: 0.6489 - acc: 0.8861 - val_loss: 0.6561 - val_acc: 0.8863
 370 Epoch 184/1000
 371 65s 131ms/step - loss: 0.6435 - acc: 0.8892 - val_loss: 0.6526 - val_acc: 0.8894
 372 Epoch 185/1000
 373 65s 131ms/step - loss: 0.6471 - acc: 0.8868 - val_loss: 0.6856 - val_acc: 0.8758
 374 Epoch 186/1000
 375 65s 131ms/step - loss: 0.6525 - acc: 0.8854 - val_loss: 0.6785 - val_acc: 0.8781
 376 Epoch 187/1000
 377 65s 131ms/step - loss: 0.6489 - acc: 0.8850 - val_loss: 0.6638 - val_acc: 0.8832
 378 Epoch 188/1000
 379 65s 131ms/step - loss: 0.6454 - acc: 0.8872 - val_loss: 0.6673 - val_acc: 0.8841
 380 Epoch 189/1000
 381 65s 131ms/step - loss: 0.6491 - acc: 0.8868 - val_loss: 0.6410 - val_acc: 0.8893
 382 Epoch 190/1000
 383 65s 131ms/step - loss: 0.6428 - acc: 0.8884 - val_loss: 0.6678 - val_acc: 0.8835
 384 Epoch 191/1000
 385 65s 131ms/step - loss: 0.6463 - acc: 0.8871 - val_loss: 0.6676 - val_acc: 0.8854
 386 Epoch 192/1000
 387 65s 131ms/step - loss: 0.6435 - acc: 0.8892 - val_loss: 0.6869 - val_acc: 0.8764
 388 Epoch 193/1000
 389 65s 131ms/step - loss: 0.6465 - acc: 0.8877 - val_loss: 0.6578 - val_acc: 0.8849
 390 Epoch 194/1000
 391 65s 131ms/step - loss: 0.6446 - acc: 0.8879 - val_loss: 0.6819 - val_acc: 0.8825
 392 Epoch 195/1000
 393 65s 131ms/step - loss: 0.6464 - acc: 0.8868 - val_loss: 0.6682 - val_acc: 0.8831
 394 Epoch 196/1000
 395 65s 131ms/step - loss: 0.6455 - acc: 0.8888 - val_loss: 0.6580 - val_acc: 0.8863
 396 Epoch 197/1000
 397 65s 131ms/step - loss: 0.6408 - acc: 0.8883 - val_loss: 0.6818 - val_acc: 0.8778
 398 Epoch 198/1000
 399 65s 131ms/step - loss: 0.6415 - acc: 0.8887 - val_loss: 0.6616 - val_acc: 0.8856
 400 Epoch 199/1000
 401 65s 131ms/step - loss: 0.6429 - acc: 0.8897 - val_loss: 0.6876 - val_acc: 0.8769
 402 Epoch 200/1000
 403 66s 131ms/step - loss: 0.6490 - acc: 0.8857 - val_loss: 0.6679 - val_acc: 0.8827
 404 Epoch 201/1000
 405 65s 131ms/step - loss: 0.6403 - acc: 0.8905 - val_loss: 0.6663 - val_acc: 0.8818
 406 Epoch 202/1000
 407 66s 131ms/step - loss: 0.6407 - acc: 0.8900 - val_loss: 0.6714 - val_acc: 0.8789
 408 Epoch 203/1000
 409 66s 131ms/step - loss: 0.6380 - acc: 0.8906 - val_loss: 0.6718 - val_acc: 0.8799
 410 Epoch 204/1000
 411 65s 131ms/step - loss: 0.6422 - acc: 0.8882 - val_loss: 0.6778 - val_acc: 0.8770
 412 Epoch 205/1000
 413 65s 129ms/step - loss: 0.6392 - acc: 0.8894 - val_loss: 0.6697 - val_acc: 0.8805
 414 Epoch 206/1000
 415 65s 129ms/step - loss: 0.6467 - acc: 0.8882 - val_loss: 0.6956 - val_acc: 0.8737
 416 Epoch 207/1000
 417 65s 131ms/step - loss: 0.6391 - acc: 0.8902 - val_loss: 0.6641 - val_acc: 0.8849
 418 Epoch 208/1000
 419 65s 131ms/step - loss: 0.6378 - acc: 0.8900 - val_loss: 0.6890 - val_acc: 0.8733
 420 Epoch 209/1000
 421 65s 131ms/step - loss: 0.6421 - acc: 0.8897 - val_loss: 0.6654 - val_acc: 0.8824
 422 Epoch 210/1000
 423 65s 131ms/step - loss: 0.6405 - acc: 0.8892 - val_loss: 0.6685 - val_acc: 0.8793
 424 Epoch 211/1000
 425 65s 131ms/step - loss: 0.6381 - acc: 0.8893 - val_loss: 0.6581 - val_acc: 0.8855
 426 Epoch 212/1000
 427 65s 131ms/step - loss: 0.6379 - acc: 0.8915 - val_loss: 0.6626 - val_acc: 0.8893
 428 Epoch 213/1000
 429 65s 131ms/step - loss: 0.6405 - acc: 0.8892 - val_loss: 0.6688 - val_acc: 0.8803
 430 Epoch 214/1000
 431 65s 131ms/step - loss: 0.6369 - acc: 0.8896 - val_loss: 0.6827 - val_acc: 0.8770
 432 Epoch 215/1000
 433 65s 131ms/step - loss: 0.6412 - acc: 0.8892 - val_loss: 0.6545 - val_acc: 0.8849
 434 Epoch 216/1000
 435 65s 131ms/step - loss: 0.6383 - acc: 0.8901 - val_loss: 0.6683 - val_acc: 0.8836
 436 Epoch 217/1000
 437 65s 131ms/step - loss: 0.6369 - acc: 0.8901 - val_loss: 0.6657 - val_acc: 0.8854
 438 Epoch 218/1000
 439 65s 131ms/step - loss: 0.6408 - acc: 0.8896 - val_loss: 0.6496 - val_acc: 0.8864
 440 Epoch 219/1000
 441 65s 131ms/step - loss: 0.6391 - acc: 0.8900 - val_loss: 0.6728 - val_acc: 0.8818
 442 Epoch 220/1000
 443 65s 131ms/step - loss: 0.6352 - acc: 0.8905 - val_loss: 0.6821 - val_acc: 0.8817
 444 Epoch 221/1000
 445 65s 131ms/step - loss: 0.6365 - acc: 0.8919 - val_loss: 0.6650 - val_acc: 0.8845
 446 Epoch 222/1000
 447 65s 131ms/step - loss: 0.6389 - acc: 0.8907 - val_loss: 0.6509 - val_acc: 0.8870
 448 Epoch 223/1000
 449 65s 131ms/step - loss: 0.6364 - acc: 0.8911 - val_loss: 0.6672 - val_acc: 0.8853
 450 Epoch 224/1000
 451 65s 131ms/step - loss: 0.6329 - acc: 0.8909 - val_loss: 0.6668 - val_acc: 0.8819
 452 Epoch 225/1000
 453 65s 131ms/step - loss: 0.6349 - acc: 0.8918 - val_loss: 0.6517 - val_acc: 0.8890
 454 Epoch 226/1000
 455 65s 131ms/step - loss: 0.6383 - acc: 0.8901 - val_loss: 0.6778 - val_acc: 0.8791
 456 Epoch 227/1000
 457 65s 131ms/step - loss: 0.6375 - acc: 0.8907 - val_loss: 0.6692 - val_acc: 0.8836
 458 Epoch 228/1000
 459 65s 131ms/step - loss: 0.6354 - acc: 0.8914 - val_loss: 0.6800 - val_acc: 0.8805
 460 Epoch 229/1000
 461 65s 131ms/step - loss: 0.6373 - acc: 0.8915 - val_loss: 0.6575 - val_acc: 0.8852
 462 Epoch 230/1000
 463 65s 131ms/step - loss: 0.6388 - acc: 0.8894 - val_loss: 0.6676 - val_acc: 0.8846
 464 Epoch 231/1000
 465 65s 131ms/step - loss: 0.6374 - acc: 0.8916 - val_loss: 0.6638 - val_acc: 0.8841
 466 Epoch 232/1000
 467 66s 132ms/step - loss: 0.6367 - acc: 0.8925 - val_loss: 0.6715 - val_acc: 0.8851
 468 Epoch 233/1000
 469 65s 131ms/step - loss: 0.6407 - acc: 0.8894 - val_loss: 0.6633 - val_acc: 0.8862
 470 Epoch 234/1000
 471 66s 131ms/step - loss: 0.6320 - acc: 0.8936 - val_loss: 0.6821 - val_acc: 0.8789
 472 Epoch 235/1000
 473 65s 131ms/step - loss: 0.6376 - acc: 0.8914 - val_loss: 0.6735 - val_acc: 0.8812
 474 Epoch 236/1000
 475 65s 131ms/step - loss: 0.6353 - acc: 0.8904 - val_loss: 0.6680 - val_acc: 0.8871
 476 Epoch 237/1000
 477 65s 131ms/step - loss: 0.6357 - acc: 0.8913 - val_loss: 0.6624 - val_acc: 0.8864
 478 Epoch 238/1000
 479 65s 131ms/step - loss: 0.6310 - acc: 0.8936 - val_loss: 0.6616 - val_acc: 0.8832
 480 Epoch 239/1000
 481 65s 131ms/step - loss: 0.6383 - acc: 0.8902 - val_loss: 0.6663 - val_acc: 0.8842
 482 Epoch 240/1000
 483 65s 131ms/step - loss: 0.6337 - acc: 0.8932 - val_loss: 0.6471 - val_acc: 0.8892
 484 Epoch 241/1000
 485 65s 131ms/step - loss: 0.6311 - acc: 0.8921 - val_loss: 0.6608 - val_acc: 0.8853
 486 Epoch 242/1000
 487 65s 131ms/step - loss: 0.6373 - acc: 0.8899 - val_loss: 0.6988 - val_acc: 0.8710
 488 Epoch 243/1000
 489 65s 131ms/step - loss: 0.6369 - acc: 0.8905 - val_loss: 0.6644 - val_acc: 0.8843
 490 Epoch 244/1000
 491 65s 130ms/step - loss: 0.6317 - acc: 0.8927 - val_loss: 0.6922 - val_acc: 0.8721
 492 Epoch 245/1000
 493 65s 131ms/step - loss: 0.6304 - acc: 0.8929 - val_loss: 0.6733 - val_acc: 0.8798
 494 Epoch 246/1000
 495 65s 131ms/step - loss: 0.6328 - acc: 0.8912 - val_loss: 0.6564 - val_acc: 0.8860
 496 Epoch 247/1000
 497 65s 131ms/step - loss: 0.6400 - acc: 0.8896 - val_loss: 0.6664 - val_acc: 0.8794
 498 Epoch 248/1000
 499 65s 131ms/step - loss: 0.6361 - acc: 0.8898 - val_loss: 0.6896 - val_acc: 0.8777
 500 Epoch 249/1000
 501 65s 131ms/step - loss: 0.6332 - acc: 0.8914 - val_loss: 0.6707 - val_acc: 0.8829
 502 Epoch 250/1000
 503 65s 131ms/step - loss: 0.6348 - acc: 0.8901 - val_loss: 0.6581 - val_acc: 0.8850
 504 Epoch 251/1000
 505 65s 131ms/step - loss: 0.6325 - acc: 0.8918 - val_loss: 0.6623 - val_acc: 0.8870
 506 Epoch 252/1000
 507 65s 131ms/step - loss: 0.6337 - acc: 0.8915 - val_loss: 0.6795 - val_acc: 0.8806
 508 Epoch 253/1000
 509 65s 131ms/step - loss: 0.6339 - acc: 0.8909 - val_loss: 0.6760 - val_acc: 0.8788
 510 Epoch 254/1000
 511 65s 131ms/step - loss: 0.6350 - acc: 0.8907 - val_loss: 0.6667 - val_acc: 0.8863
 512 Epoch 255/1000
 513 65s 131ms/step - loss: 0.6312 - acc: 0.8927 - val_loss: 0.6825 - val_acc: 0.8775
 514 Epoch 256/1000
 515 65s 131ms/step - loss: 0.6304 - acc: 0.8920 - val_loss: 0.6648 - val_acc: 0.8839
 516 Epoch 257/1000
 517 65s 131ms/step - loss: 0.6317 - acc: 0.8917 - val_loss: 0.6624 - val_acc: 0.8830
 518 Epoch 258/1000
 519 65s 131ms/step - loss: 0.6335 - acc: 0.8914 - val_loss: 0.6547 - val_acc: 0.8877
 520 Epoch 259/1000
 521 65s 131ms/step - loss: 0.6346 - acc: 0.8903 - val_loss: 0.6671 - val_acc: 0.8863
 522 Epoch 260/1000
 523 65s 131ms/step - loss: 0.6303 - acc: 0.8909 - val_loss: 0.6491 - val_acc: 0.8862
 524 Epoch 261/1000
 525 65s 131ms/step - loss: 0.6348 - acc: 0.8902 - val_loss: 0.6778 - val_acc: 0.8781
 526 Epoch 262/1000
 527 65s 131ms/step - loss: 0.6325 - acc: 0.8928 - val_loss: 0.6651 - val_acc: 0.8800
 528 Epoch 263/1000
 529 65s 131ms/step - loss: 0.6377 - acc: 0.8895 - val_loss: 0.6474 - val_acc: 0.8908
 530 Epoch 264/1000
 531 65s 131ms/step - loss: 0.6293 - acc: 0.8927 - val_loss: 0.6707 - val_acc: 0.8821
 532 Epoch 265/1000
 533 65s 131ms/step - loss: 0.6321 - acc: 0.8915 - val_loss: 0.6679 - val_acc: 0.8820
 534 Epoch 266/1000
 535 65s 131ms/step - loss: 0.6323 - acc: 0.8936 - val_loss: 0.6647 - val_acc: 0.8851
 536 Epoch 267/1000
 537 65s 131ms/step - loss: 0.6311 - acc: 0.8926 - val_loss: 0.6748 - val_acc: 0.8786
 538 Epoch 268/1000
 539 65s 131ms/step - loss: 0.6344 - acc: 0.8920 - val_loss: 0.6851 - val_acc: 0.8826
 540 Epoch 269/1000
 541 65s 131ms/step - loss: 0.6296 - acc: 0.8943 - val_loss: 0.6626 - val_acc: 0.8854
 542 Epoch 270/1000
 543 65s 131ms/step - loss: 0.6323 - acc: 0.8931 - val_loss: 0.6555 - val_acc: 0.8864
 544 Epoch 271/1000
 545 65s 131ms/step - loss: 0.6285 - acc: 0.8933 - val_loss: 0.6781 - val_acc: 0.8817
 546 Epoch 272/1000
 547 65s 131ms/step - loss: 0.6316 - acc: 0.8921 - val_loss: 0.6630 - val_acc: 0.8870
 548 Epoch 273/1000
 549 65s 131ms/step - loss: 0.6296 - acc: 0.8943 - val_loss: 0.6682 - val_acc: 0.8824
 550 Epoch 274/1000
 551 65s 131ms/step - loss: 0.6221 - acc: 0.8957 - val_loss: 0.6788 - val_acc: 0.8791
 552 Epoch 275/1000
 553 65s 131ms/step - loss: 0.6317 - acc: 0.8918 - val_loss: 0.6434 - val_acc: 0.8917
 554 Epoch 276/1000
 555 65s 130ms/step - loss: 0.6290 - acc: 0.8927 - val_loss: 0.6572 - val_acc: 0.8868
 556 Epoch 277/1000
 557 65s 131ms/step - loss: 0.6279 - acc: 0.8931 - val_loss: 0.6877 - val_acc: 0.8757
 558 Epoch 278/1000
 559 65s 131ms/step - loss: 0.6301 - acc: 0.8923 - val_loss: 0.6746 - val_acc: 0.8770
 560 Epoch 279/1000
 561 65s 131ms/step - loss: 0.6334 - acc: 0.8919 - val_loss: 0.6553 - val_acc: 0.8863
 562 Epoch 280/1000
 563 65s 131ms/step - loss: 0.6320 - acc: 0.8927 - val_loss: 0.6727 - val_acc: 0.8812
 564 Epoch 281/1000
 565 65s 131ms/step - loss: 0.6290 - acc: 0.8944 - val_loss: 0.6784 - val_acc: 0.8765
 566 Epoch 282/1000
 567 65s 131ms/step - loss: 0.6290 - acc: 0.8937 - val_loss: 0.6466 - val_acc: 0.8924
 568 Epoch 283/1000
 569 65s 131ms/step - loss: 0.6297 - acc: 0.8940 - val_loss: 0.6622 - val_acc: 0.8853
 570 Epoch 284/1000
 571 65s 131ms/step - loss: 0.6267 - acc: 0.8940 - val_loss: 0.6592 - val_acc: 0.8860
 572 Epoch 285/1000
 573 65s 131ms/step - loss: 0.6319 - acc: 0.8926 - val_loss: 0.6628 - val_acc: 0.8849
 574 Epoch 286/1000
 575 65s 131ms/step - loss: 0.6314 - acc: 0.8935 - val_loss: 0.6617 - val_acc: 0.8855
 576 Epoch 287/1000
 577 65s 131ms/step - loss: 0.6251 - acc: 0.8949 - val_loss: 0.6846 - val_acc: 0.8761
 578 Epoch 288/1000
 579 65s 131ms/step - loss: 0.6311 - acc: 0.8923 - val_loss: 0.6675 - val_acc: 0.8826
 580 Epoch 289/1000
 581 65s 131ms/step - loss: 0.6282 - acc: 0.8938 - val_loss: 0.6756 - val_acc: 0.8799
 582 Epoch 290/1000
 583 65s 131ms/step - loss: 0.6289 - acc: 0.8938 - val_loss: 0.6717 - val_acc: 0.8831
 584 Epoch 291/1000
 585 65s 131ms/step - loss: 0.6288 - acc: 0.8926 - val_loss: 0.6444 - val_acc: 0.8908
 586 Epoch 292/1000
 587 65s 131ms/step - loss: 0.6257 - acc: 0.8943 - val_loss: 0.6434 - val_acc: 0.8882
 588 Epoch 293/1000
 589 65s 131ms/step - loss: 0.6269 - acc: 0.8926 - val_loss: 0.6450 - val_acc: 0.8896
 590 Epoch 294/1000
 591 65s 131ms/step - loss: 0.6284 - acc: 0.8929 - val_loss: 0.6520 - val_acc: 0.8855
 592 Epoch 295/1000
 593 65s 131ms/step - loss: 0.6234 - acc: 0.8941 - val_loss: 0.6519 - val_acc: 0.8899
 594 Epoch 296/1000
 595 66s 131ms/step - loss: 0.6284 - acc: 0.8935 - val_loss: 0.6571 - val_acc: 0.8827
 596 Epoch 297/1000
 597 65s 131ms/step - loss: 0.6265 - acc: 0.8940 - val_loss: 0.6566 - val_acc: 0.8857
 598 Epoch 298/1000
 599 65s 131ms/step - loss: 0.6287 - acc: 0.8936 - val_loss: 0.6573 - val_acc: 0.8841
 600 Epoch 299/1000
 601 65s 131ms/step - loss: 0.6237 - acc: 0.8954 - val_loss: 0.6371 - val_acc: 0.8937
 602 Epoch 300/1000
 603 65s 131ms/step - loss: 0.6263 - acc: 0.8943 - val_loss: 0.6537 - val_acc: 0.8884
 604 Epoch 301/1000
 605 lr changed to 0.010000000149011612
 606 65s 131ms/step - loss: 0.5256 - acc: 0.9298 - val_loss: 0.5518 - val_acc: 0.9215
 607 Epoch 302/1000
 608 66s 131ms/step - loss: 0.4681 - acc: 0.9470 - val_loss: 0.5407 - val_acc: 0.9233
 609 Epoch 303/1000
 610 66s 131ms/step - loss: 0.4455 - acc: 0.9532 - val_loss: 0.5319 - val_acc: 0.9258
 611 Epoch 304/1000
 612 65s 131ms/step - loss: 0.4308 - acc: 0.9559 - val_loss: 0.5251 - val_acc: 0.9277
 613 Epoch 305/1000
 614 65s 131ms/step - loss: 0.4180 - acc: 0.9595 - val_loss: 0.5182 - val_acc: 0.9290
 615 Epoch 306/1000
 616 65s 131ms/step - loss: 0.4088 - acc: 0.9609 - val_loss: 0.5124 - val_acc: 0.9300
 617 Epoch 307/1000
 618 65s 131ms/step - loss: 0.3970 - acc: 0.9628 - val_loss: 0.5158 - val_acc: 0.9277
 619 Epoch 308/1000
 620 65s 131ms/step - loss: 0.3877 - acc: 0.9653 - val_loss: 0.5093 - val_acc: 0.9298
 621 Epoch 309/1000
 622 65s 131ms/step - loss: 0.3794 - acc: 0.9664 - val_loss: 0.5062 - val_acc: 0.9281
 623 Epoch 310/1000
 624 65s 131ms/step - loss: 0.3736 - acc: 0.9666 - val_loss: 0.5056 - val_acc: 0.9267
 625 Epoch 311/1000
 626 65s 131ms/step - loss: 0.3675 - acc: 0.9669 - val_loss: 0.4959 - val_acc: 0.9295
 627 Epoch 312/1000
 628 65s 131ms/step - loss: 0.3631 - acc: 0.9670 - val_loss: 0.4913 - val_acc: 0.9313
 629 Epoch 313/1000
 630 65s 131ms/step - loss: 0.3538 - acc: 0.9686 - val_loss: 0.4924 - val_acc: 0.9299
 631 Epoch 314/1000
 632 65s 131ms/step - loss: 0.3432 - acc: 0.9716 - val_loss: 0.4920 - val_acc: 0.9296
 633 Epoch 315/1000
 634 65s 131ms/step - loss: 0.3434 - acc: 0.9701 - val_loss: 0.4838 - val_acc: 0.9277
 635 Epoch 316/1000
 636 65s 131ms/step - loss: 0.3325 - acc: 0.9719 - val_loss: 0.4822 - val_acc: 0.9301
 637 Epoch 317/1000
 638 65s 130ms/step - loss: 0.3283 - acc: 0.9724 - val_loss: 0.4882 - val_acc: 0.9270
 639 Epoch 318/1000
 640 65s 129ms/step - loss: 0.3259 - acc: 0.9727 - val_loss: 0.4866 - val_acc: 0.9263
 641 Epoch 319/1000
 642 65s 130ms/step - loss: 0.3200 - acc: 0.9728 - val_loss: 0.4780 - val_acc: 0.9279
 643 Epoch 320/1000
 644 65s 131ms/step - loss: 0.3156 - acc: 0.9733 - val_loss: 0.4768 - val_acc: 0.9256
 645 Epoch 321/1000
 646 65s 131ms/step - loss: 0.3109 - acc: 0.9738 - val_loss: 0.4662 - val_acc: 0.9274
 647 Epoch 322/1000
 648 65s 131ms/step - loss: 0.3070 - acc: 0.9743 - val_loss: 0.4666 - val_acc: 0.9266
 649 Epoch 323/1000
 650 65s 131ms/step - loss: 0.3008 - acc: 0.9754 - val_loss: 0.4734 - val_acc: 0.9244
 651 Epoch 324/1000
 652 65s 131ms/step - loss: 0.3005 - acc: 0.9739 - val_loss: 0.4770 - val_acc: 0.9276
 653 Epoch 325/1000
 654 65s 131ms/step - loss: 0.2967 - acc: 0.9736 - val_loss: 0.4575 - val_acc: 0.9289
 655 Epoch 326/1000
 656 65s 131ms/step - loss: 0.2945 - acc: 0.9742 - val_loss: 0.4677 - val_acc: 0.9247
 657 Epoch 327/1000
 658 65s 131ms/step - loss: 0.2862 - acc: 0.9760 - val_loss: 0.4682 - val_acc: 0.9263
 659 Epoch 328/1000
 660 65s 131ms/step - loss: 0.2850 - acc: 0.9762 - val_loss: 0.4657 - val_acc: 0.9247
 661 Epoch 329/1000
 662 65s 131ms/step - loss: 0.2816 - acc: 0.9757 - val_loss: 0.4617 - val_acc: 0.9265
 663 Epoch 330/1000
 664 65s 131ms/step - loss: 0.2812 - acc: 0.9744 - val_loss: 0.4649 - val_acc: 0.9226
 665 Epoch 331/1000
 666 65s 131ms/step - loss: 0.2791 - acc: 0.9744 - val_loss: 0.4484 - val_acc: 0.9282
 667 Epoch 332/1000
 668 65s 131ms/step - loss: 0.2743 - acc: 0.9757 - val_loss: 0.4503 - val_acc: 0.9242
 669 Epoch 333/1000
 670 65s 131ms/step - loss: 0.2706 - acc: 0.9767 - val_loss: 0.4464 - val_acc: 0.9295
 671 Epoch 334/1000
 672 65s 131ms/step - loss: 0.2690 - acc: 0.9757 - val_loss: 0.4507 - val_acc: 0.9272
 673 Epoch 335/1000
 674 65s 131ms/step - loss: 0.2649 - acc: 0.9762 - val_loss: 0.4510 - val_acc: 0.9246
 675 Epoch 336/1000
 676 66s 131ms/step - loss: 0.2626 - acc: 0.9776 - val_loss: 0.4529 - val_acc: 0.9226
 677 Epoch 337/1000
 678 66s 131ms/step - loss: 0.2615 - acc: 0.9772 - val_loss: 0.4453 - val_acc: 0.9270
 679 Epoch 338/1000
 680 66s 131ms/step - loss: 0.2597 - acc: 0.9763 - val_loss: 0.4571 - val_acc: 0.9232
 681 Epoch 339/1000
 682 65s 131ms/step - loss: 0.2555 - acc: 0.9776 - val_loss: 0.4449 - val_acc: 0.9247
 683 ...
 684 Epoch 755/1000
 685 65s 130ms/step - loss: 0.1093 - acc: 0.9992 - val_loss: 0.3584 - val_acc: 0.9337
 686 Epoch 756/1000
 687 65s 130ms/step - loss: 0.1093 - acc: 0.9990 - val_loss: 0.3583 - val_acc: 0.9346
 688 Epoch 757/1000
 689 65s 130ms/step - loss: 0.1095 - acc: 0.9991 - val_loss: 0.3560 - val_acc: 0.9346
 690 Epoch 758/1000
 691 65s 130ms/step - loss: 0.1090 - acc: 0.9991 - val_loss: 0.3587 - val_acc: 0.9346
 692 Epoch 759/1000
 693 65s 130ms/step - loss: 0.1092 - acc: 0.9989 - val_loss: 0.3594 - val_acc: 0.9346
 694 Epoch 760/1000
 695 65s 130ms/step - loss: 0.1086 - acc: 0.9992 - val_loss: 0.3560 - val_acc: 0.9345
 696 Epoch 761/1000
 697 65s 130ms/step - loss: 0.1081 - acc: 0.9993 - val_loss: 0.3573 - val_acc: 0.9346
 698 Epoch 762/1000
 699 65s 129ms/step - loss: 0.1083 - acc: 0.9992 - val_loss: 0.3598 - val_acc: 0.9343
 700 Epoch 763/1000
 701 65s 130ms/step - loss: 0.1080 - acc: 0.9991 - val_loss: 0.3590 - val_acc: 0.9341
 702 Epoch 764/1000
 703 65s 130ms/step - loss: 0.1076 - acc: 0.9993 - val_loss: 0.3567 - val_acc: 0.9336
 704 Epoch 765/1000
 705 65s 130ms/step - loss: 0.1077 - acc: 0.9991 - val_loss: 0.3556 - val_acc: 0.9375
 706 Epoch 766/1000
 707 65s 130ms/step - loss: 0.1072 - acc: 0.9993 - val_loss: 0.3562 - val_acc: 0.9349
 708 Epoch 767/1000
 709 65s 130ms/step - loss: 0.1075 - acc: 0.9992 - val_loss: 0.3538 - val_acc: 0.9364
 710 Epoch 768/1000
 711 65s 130ms/step - loss: 0.1071 - acc: 0.9991 - val_loss: 0.3607 - val_acc: 0.9347
 712 Epoch 769/1000
 713 65s 130ms/step - loss: 0.1067 - acc: 0.9994 - val_loss: 0.3626 - val_acc: 0.9348
 714 Epoch 770/1000
 715 65s 130ms/step - loss: 0.1070 - acc: 0.9991 - val_loss: 0.3595 - val_acc: 0.9364
 716 Epoch 771/1000
 717 65s 130ms/step - loss: 0.1067 - acc: 0.9991 - val_loss: 0.3578 - val_acc: 0.9353
 718 Epoch 772/1000
 719 65s 130ms/step - loss: 0.1066 - acc: 0.9991 - val_loss: 0.3561 - val_acc: 0.9357
 720 Epoch 773/1000
 721 65s 130ms/step - loss: 0.1062 - acc: 0.9992 - val_loss: 0.3555 - val_acc: 0.9357
 722 Epoch 774/1000
 723 65s 130ms/step - loss: 0.1062 - acc: 0.9992 - val_loss: 0.3546 - val_acc: 0.9367
 724 Epoch 775/1000
 725 65s 130ms/step - loss: 0.1059 - acc: 0.9992 - val_loss: 0.3570 - val_acc: 0.9367
 726 Epoch 776/1000
 727 65s 130ms/step - loss: 0.1061 - acc: 0.9990 - val_loss: 0.3570 - val_acc: 0.9355
 728 Epoch 777/1000
 729 65s 129ms/step - loss: 0.1065 - acc: 0.9988 - val_loss: 0.3569 - val_acc: 0.9361
 730 Epoch 778/1000
 731 65s 129ms/step - loss: 0.1055 - acc: 0.9991 - val_loss: 0.3592 - val_acc: 0.9347
 732 Epoch 779/1000
 733 65s 129ms/step - loss: 0.1053 - acc: 0.9991 - val_loss: 0.3578 - val_acc: 0.9345
 734 Epoch 780/1000
 735 65s 130ms/step - loss: 0.1057 - acc: 0.9990 - val_loss: 0.3550 - val_acc: 0.9361
 736 Epoch 781/1000
 737 65s 130ms/step - loss: 0.1054 - acc: 0.9988 - val_loss: 0.3598 - val_acc: 0.9359
 738 Epoch 782/1000
 739 65s 130ms/step - loss: 0.1053 - acc: 0.9988 - val_loss: 0.3548 - val_acc: 0.9349
 740 Epoch 783/1000
 741 65s 129ms/step - loss: 0.1047 - acc: 0.9992 - val_loss: 0.3541 - val_acc: 0.9366
 742 Epoch 784/1000
 743 65s 130ms/step - loss: 0.1048 - acc: 0.9990 - val_loss: 0.3540 - val_acc: 0.9346
 744 Epoch 785/1000
 745 65s 130ms/step - loss: 0.1046 - acc: 0.9991 - val_loss: 0.3534 - val_acc: 0.9350
 746 Epoch 786/1000
 747 65s 130ms/step - loss: 0.1041 - acc: 0.9992 - val_loss: 0.3559 - val_acc: 0.9349
 748 Epoch 787/1000
 749 65s 130ms/step - loss: 0.1042 - acc: 0.9992 - val_loss: 0.3547 - val_acc: 0.9336
 750 Epoch 788/1000
 751 65s 130ms/step - loss: 0.1039 - acc: 0.9992 - val_loss: 0.3523 - val_acc: 0.9347
 752 Epoch 789/1000
 753 65s 130ms/step - loss: 0.1037 - acc: 0.9991 - val_loss: 0.3487 - val_acc: 0.9375
 754 Epoch 790/1000
 755 65s 130ms/step - loss: 0.1034 - acc: 0.9992 - val_loss: 0.3481 - val_acc: 0.9365
 756 Epoch 791/1000
 757 65s 130ms/step - loss: 0.1034 - acc: 0.9992 - val_loss: 0.3514 - val_acc: 0.9370
 758 Epoch 792/1000
 759 65s 130ms/step - loss: 0.1034 - acc: 0.9991 - val_loss: 0.3507 - val_acc: 0.9363
 760 Epoch 793/1000
 761 65s 130ms/step - loss: 0.1029 - acc: 0.9992 - val_loss: 0.3531 - val_acc: 0.9358
 762 Epoch 794/1000
 763 65s 129ms/step - loss: 0.1032 - acc: 0.9990 - val_loss: 0.3563 - val_acc: 0.9351
 764 Epoch 795/1000
 765 65s 129ms/step - loss: 0.1026 - acc: 0.9992 - val_loss: 0.3529 - val_acc: 0.9362
 766 Epoch 796/1000
 767 65s 130ms/step - loss: 0.1024 - acc: 0.9992 - val_loss: 0.3511 - val_acc: 0.9360
 768 Epoch 797/1000
 769 65s 130ms/step - loss: 0.1023 - acc: 0.9990 - val_loss: 0.3520 - val_acc: 0.9358
 770 Epoch 798/1000
 771 65s 130ms/step - loss: 0.1023 - acc: 0.9990 - val_loss: 0.3524 - val_acc: 0.9354
 772 Epoch 799/1000
 773 65s 130ms/step - loss: 0.1022 - acc: 0.9991 - val_loss: 0.3547 - val_acc: 0.9349
 774 Epoch 800/1000
 775 65s 130ms/step - loss: 0.1020 - acc: 0.9991 - val_loss: 0.3548 - val_acc: 0.9356
 776 Epoch 801/1000
 777 65s 129ms/step - loss: 0.1016 - acc: 0.9993 - val_loss: 0.3524 - val_acc: 0.9356
 778 Epoch 802/1000
 779 65s 130ms/step - loss: 0.1016 - acc: 0.9992 - val_loss: 0.3516 - val_acc: 0.9360
 780 Epoch 803/1000
 781 65s 130ms/step - loss: 0.1015 - acc: 0.9991 - val_loss: 0.3497 - val_acc: 0.9353
 782 Epoch 804/1000
 783 65s 129ms/step - loss: 0.1012 - acc: 0.9992 - val_loss: 0.3520 - val_acc: 0.9355
 784 Epoch 805/1000
 785 65s 130ms/step - loss: 0.1014 - acc: 0.9991 - val_loss: 0.3539 - val_acc: 0.9354
 786 Epoch 806/1000
 787 65s 130ms/step - loss: 0.1010 - acc: 0.9990 - val_loss: 0.3580 - val_acc: 0.9352
 788 Epoch 807/1000
 789 65s 130ms/step - loss: 0.1011 - acc: 0.9990 - val_loss: 0.3513 - val_acc: 0.9349
 790 Epoch 808/1000
 791 65s 130ms/step - loss: 0.1006 - acc: 0.9992 - val_loss: 0.3521 - val_acc: 0.9367
 792 Epoch 809/1000
 793 65s 130ms/step - loss: 0.1005 - acc: 0.9991 - val_loss: 0.3495 - val_acc: 0.9368
 794 Epoch 810/1000
 795 65s 129ms/step - loss: 0.1008 - acc: 0.9988 - val_loss: 0.3529 - val_acc: 0.9350
 796 Epoch 811/1000
 797 65s 129ms/step - loss: 0.1001 - acc: 0.9992 - val_loss: 0.3569 - val_acc: 0.9358
 798 Epoch 812/1000
 799 65s 130ms/step - loss: 0.0998 - acc: 0.9991 - val_loss: 0.3532 - val_acc: 0.9355
 800 Epoch 813/1000
 801 65s 129ms/step - loss: 0.0996 - acc: 0.9992 - val_loss: 0.3559 - val_acc: 0.9347
 802 Epoch 814/1000
 803 65s 130ms/step - loss: 0.0997 - acc: 0.9992 - val_loss: 0.3532 - val_acc: 0.9345
 804 Epoch 815/1000
 805 65s 130ms/step - loss: 0.0996 - acc: 0.9991 - val_loss: 0.3544 - val_acc: 0.9340
 806 Epoch 816/1000
 807 65s 130ms/step - loss: 0.0991 - acc: 0.9991 - val_loss: 0.3529 - val_acc: 0.9358
 808 Epoch 817/1000
 809 65s 130ms/step - loss: 0.0984 - acc: 0.9995 - val_loss: 0.3508 - val_acc: 0.9365
 810 Epoch 818/1000
 811 65s 130ms/step - loss: 0.0994 - acc: 0.9989 - val_loss: 0.3533 - val_acc: 0.9362
 812 Epoch 819/1000
 813 65s 129ms/step - loss: 0.0987 - acc: 0.9993 - val_loss: 0.3519 - val_acc: 0.9351
 814 Epoch 820/1000
 815 65s 130ms/step - loss: 0.0988 - acc: 0.9991 - val_loss: 0.3528 - val_acc: 0.9352
 816 Epoch 821/1000
 817 65s 130ms/step - loss: 0.0983 - acc: 0.9992 - val_loss: 0.3479 - val_acc: 0.9354
 818 Epoch 822/1000
 819 65s 130ms/step - loss: 0.0984 - acc: 0.9991 - val_loss: 0.3485 - val_acc: 0.9367
 820 Epoch 823/1000
 821 65s 130ms/step - loss: 0.0985 - acc: 0.9990 - val_loss: 0.3530 - val_acc: 0.9358
 822 Epoch 824/1000
 823 65s 130ms/step - loss: 0.0981 - acc: 0.9992 - val_loss: 0.3464 - val_acc: 0.9377
 824 Epoch 825/1000
 825 65s 130ms/step - loss: 0.0978 - acc: 0.9993 - val_loss: 0.3477 - val_acc: 0.9358
 826 Epoch 826/1000
 827 65s 130ms/step - loss: 0.0973 - acc: 0.9992 - val_loss: 0.3468 - val_acc: 0.9364
 828 Epoch 827/1000
 829 65s 130ms/step - loss: 0.0979 - acc: 0.9991 - val_loss: 0.3502 - val_acc: 0.9358
 830 Epoch 828/1000
 831 65s 130ms/step - loss: 0.0974 - acc: 0.9991 - val_loss: 0.3470 - val_acc: 0.9356
 832 Epoch 829/1000
 833 65s 130ms/step - loss: 0.0969 - acc: 0.9994 - val_loss: 0.3459 - val_acc: 0.9351
 834 Epoch 830/1000
 835 65s 130ms/step - loss: 0.0970 - acc: 0.9990 - val_loss: 0.3528 - val_acc: 0.9347
 836 Epoch 831/1000
 837 65s 130ms/step - loss: 0.0969 - acc: 0.9992 - val_loss: 0.3484 - val_acc: 0.9360
 838 Epoch 832/1000
 839 65s 129ms/step - loss: 0.0970 - acc: 0.9992 - val_loss: 0.3542 - val_acc: 0.9353
 840 Epoch 833/1000
 841 65s 130ms/step - loss: 0.0969 - acc: 0.9990 - val_loss: 0.3496 - val_acc: 0.9345
 842 Epoch 834/1000
 843 65s 130ms/step - loss: 0.0970 - acc: 0.9990 - val_loss: 0.3460 - val_acc: 0.9372
 844 Epoch 835/1000
 845 65s 129ms/step - loss: 0.0960 - acc: 0.9993 - val_loss: 0.3514 - val_acc: 0.9349
 846 Epoch 836/1000
 847 65s 130ms/step - loss: 0.0962 - acc: 0.9994 - val_loss: 0.3420 - val_acc: 0.9376
 848 Epoch 837/1000
 849 65s 130ms/step - loss: 0.0960 - acc: 0.9992 - val_loss: 0.3441 - val_acc: 0.9358
 850 Epoch 838/1000
 851 65s 130ms/step - loss: 0.0957 - acc: 0.9993 - val_loss: 0.3474 - val_acc: 0.9368
 852 Epoch 839/1000
 853 65s 130ms/step - loss: 0.0955 - acc: 0.9993 - val_loss: 0.3447 - val_acc: 0.9355
 854 Epoch 840/1000
 855 65s 129ms/step - loss: 0.0951 - acc: 0.9995 - val_loss: 0.3508 - val_acc: 0.9355
 856 Epoch 841/1000
 857 65s 130ms/step - loss: 0.0951 - acc: 0.9993 - val_loss: 0.3488 - val_acc: 0.9366
 858 Epoch 842/1000
 859 65s 130ms/step - loss: 0.0952 - acc: 0.9992 - val_loss: 0.3500 - val_acc: 0.9368
 860 Epoch 843/1000
 861 65s 129ms/step - loss: 0.0952 - acc: 0.9991 - val_loss: 0.3464 - val_acc: 0.9359
 862 Epoch 844/1000
 863 65s 129ms/step - loss: 0.0947 - acc: 0.9993 - val_loss: 0.3470 - val_acc: 0.9365
 864 Epoch 845/1000
 865 65s 129ms/step - loss: 0.0947 - acc: 0.9993 - val_loss: 0.3478 - val_acc: 0.9353
 866 Epoch 846/1000
 867 65s 130ms/step - loss: 0.0952 - acc: 0.9990 - val_loss: 0.3501 - val_acc: 0.9355
 868 Epoch 847/1000
 869 65s 130ms/step - loss: 0.0944 - acc: 0.9993 - val_loss: 0.3463 - val_acc: 0.9354
 870 Epoch 848/1000
 871 65s 130ms/step - loss: 0.0944 - acc: 0.9993 - val_loss: 0.3504 - val_acc: 0.9351
 872 Epoch 849/1000
 873 65s 130ms/step - loss: 0.0941 - acc: 0.9993 - val_loss: 0.3468 - val_acc: 0.9373
 874 Epoch 850/1000
 875 65s 129ms/step - loss: 0.0947 - acc: 0.9988 - val_loss: 0.3432 - val_acc: 0.9378
 876 Epoch 851/1000
 877 65s 129ms/step - loss: 0.0943 - acc: 0.9989 - val_loss: 0.3456 - val_acc: 0.9369
 878 Epoch 852/1000
 879 65s 129ms/step - loss: 0.0943 - acc: 0.9990 - val_loss: 0.3471 - val_acc: 0.9365
 880 Epoch 853/1000
 881 65s 130ms/step - loss: 0.0940 - acc: 0.9990 - val_loss: 0.3506 - val_acc: 0.9356
 882 Epoch 854/1000
 883 65s 130ms/step - loss: 0.0936 - acc: 0.9992 - val_loss: 0.3498 - val_acc: 0.9358
 884 Epoch 855/1000
 885 65s 130ms/step - loss: 0.0934 - acc: 0.9992 - val_loss: 0.3469 - val_acc: 0.9361
 886 Epoch 856/1000
 887 65s 130ms/step - loss: 0.0931 - acc: 0.9993 - val_loss: 0.3483 - val_acc: 0.9361
 888 Epoch 857/1000
 889 65s 130ms/step - loss: 0.0930 - acc: 0.9993 - val_loss: 0.3440 - val_acc: 0.9350
 890 Epoch 858/1000
 891 65s 129ms/step - loss: 0.0930 - acc: 0.9991 - val_loss: 0.3445 - val_acc: 0.9365
 892 Epoch 859/1000
 893 65s 130ms/step - loss: 0.0928 - acc: 0.9992 - val_loss: 0.3465 - val_acc: 0.9366
 894 Epoch 860/1000
 895 65s 130ms/step - loss: 0.0928 - acc: 0.9990 - val_loss: 0.3527 - val_acc: 0.9345
 896 Epoch 861/1000
 897 65s 129ms/step - loss: 0.0924 - acc: 0.9992 - val_loss: 0.3465 - val_acc: 0.9369
 898 Epoch 862/1000
 899 65s 130ms/step - loss: 0.0923 - acc: 0.9992 - val_loss: 0.3445 - val_acc: 0.9366
 900 Epoch 863/1000
 901 65s 130ms/step - loss: 0.0923 - acc: 0.9992 - val_loss: 0.3476 - val_acc: 0.9362
 902 Epoch 864/1000
 903 65s 130ms/step - loss: 0.0920 - acc: 0.9993 - val_loss: 0.3454 - val_acc: 0.9369
 904 Epoch 865/1000
 905 65s 130ms/step - loss: 0.0922 - acc: 0.9990 - val_loss: 0.3486 - val_acc: 0.9337
 906 Epoch 866/1000
 907 65s 130ms/step - loss: 0.0914 - acc: 0.9994 - val_loss: 0.3489 - val_acc: 0.9355
 908 Epoch 867/1000
 909 65s 129ms/step - loss: 0.0918 - acc: 0.9991 - val_loss: 0.3467 - val_acc: 0.9359
 910 Epoch 868/1000
 911 65s 130ms/step - loss: 0.0918 - acc: 0.9992 - val_loss: 0.3486 - val_acc: 0.9348
 912 Epoch 869/1000
 913 65s 130ms/step - loss: 0.0913 - acc: 0.9992 - val_loss: 0.3437 - val_acc: 0.9364
 914 Epoch 870/1000
 915 65s 130ms/step - loss: 0.0914 - acc: 0.9992 - val_loss: 0.3488 - val_acc: 0.9350
 916 Epoch 871/1000
 917 65s 130ms/step - loss: 0.0913 - acc: 0.9991 - val_loss: 0.3473 - val_acc: 0.9367
 918 Epoch 872/1000
 919 65s 130ms/step - loss: 0.0911 - acc: 0.9992 - val_loss: 0.3448 - val_acc: 0.9380
 920 Epoch 873/1000
 921 65s 130ms/step - loss: 0.0907 - acc: 0.9993 - val_loss: 0.3439 - val_acc: 0.9373
 922 Epoch 874/1000
 923 65s 130ms/step - loss: 0.0911 - acc: 0.9988 - val_loss: 0.3421 - val_acc: 0.9384
 924 Epoch 875/1000
 925 65s 130ms/step - loss: 0.0904 - acc: 0.9992 - val_loss: 0.3430 - val_acc: 0.9365
 926 Epoch 876/1000
 927 65s 130ms/step - loss: 0.0908 - acc: 0.9990 - val_loss: 0.3471 - val_acc: 0.9355
 928 Epoch 877/1000
 929 65s 129ms/step - loss: 0.0905 - acc: 0.9991 - val_loss: 0.3452 - val_acc: 0.9359
 930 Epoch 878/1000
 931 65s 130ms/step - loss: 0.0905 - acc: 0.9990 - val_loss: 0.3379 - val_acc: 0.9372
 932 Epoch 879/1000
 933 65s 130ms/step - loss: 0.0906 - acc: 0.9989 - val_loss: 0.3442 - val_acc: 0.9369
 934 Epoch 880/1000
 935 65s 130ms/step - loss: 0.0903 - acc: 0.9990 - val_loss: 0.3413 - val_acc: 0.9363
 936 Epoch 881/1000
 937 65s 130ms/step - loss: 0.0898 - acc: 0.9992 - val_loss: 0.3437 - val_acc: 0.9354
 938 Epoch 882/1000
 939 65s 129ms/step - loss: 0.0898 - acc: 0.9992 - val_loss: 0.3421 - val_acc: 0.9371
 940 Epoch 883/1000
 941 65s 130ms/step - loss: 0.0897 - acc: 0.9991 - val_loss: 0.3442 - val_acc: 0.9363
 942 Epoch 884/1000
 943 65s 130ms/step - loss: 0.0900 - acc: 0.9990 - val_loss: 0.3471 - val_acc: 0.9366
 944 Epoch 885/1000
 945 65s 130ms/step - loss: 0.0897 - acc: 0.9991 - val_loss: 0.3443 - val_acc: 0.9361
 946 Epoch 886/1000
 947 65s 130ms/step - loss: 0.0892 - acc: 0.9990 - val_loss: 0.3434 - val_acc: 0.9355
 948 Epoch 887/1000
 949 65s 130ms/step - loss: 0.0890 - acc: 0.9991 - val_loss: 0.3411 - val_acc: 0.9367
 950 Epoch 888/1000
 951 65s 130ms/step - loss: 0.0889 - acc: 0.9992 - val_loss: 0.3478 - val_acc: 0.9338
 952 Epoch 889/1000
 953 65s 130ms/step - loss: 0.0889 - acc: 0.9991 - val_loss: 0.3404 - val_acc: 0.9366
 954 Epoch 890/1000
 955 65s 130ms/step - loss: 0.0889 - acc: 0.9991 - val_loss: 0.3356 - val_acc: 0.9373
 956 Epoch 891/1000
 957 65s 130ms/step - loss: 0.0886 - acc: 0.9992 - val_loss: 0.3358 - val_acc: 0.9362
 958 Epoch 892/1000
 959 65s 130ms/step - loss: 0.0883 - acc: 0.9992 - val_loss: 0.3380 - val_acc: 0.9368
 960 Epoch 893/1000
 961 65s 129ms/step - loss: 0.0886 - acc: 0.9991 - val_loss: 0.3369 - val_acc: 0.9374
 962 Epoch 894/1000
 963 65s 130ms/step - loss: 0.0881 - acc: 0.9993 - val_loss: 0.3397 - val_acc: 0.9386
 964 Epoch 895/1000
 965 65s 130ms/step - loss: 0.0885 - acc: 0.9991 - val_loss: 0.3400 - val_acc: 0.9365
 966 Epoch 896/1000
 967 65s 130ms/step - loss: 0.0883 - acc: 0.9989 - val_loss: 0.3367 - val_acc: 0.9355
 968 Epoch 897/1000
 969 65s 130ms/step - loss: 0.0886 - acc: 0.9986 - val_loss: 0.3375 - val_acc: 0.9361
 970 Epoch 898/1000
 971 65s 130ms/step - loss: 0.0878 - acc: 0.9989 - val_loss: 0.3444 - val_acc: 0.9354
 972 Epoch 899/1000
 973 65s 130ms/step - loss: 0.0875 - acc: 0.9992 - val_loss: 0.3444 - val_acc: 0.9367
 974 Epoch 900/1000
 975 65s 130ms/step - loss: 0.0877 - acc: 0.9990 - val_loss: 0.3457 - val_acc: 0.9353
 976 Epoch 901/1000
 977 lr changed to 9.999999310821295e-05
 978 66s 132ms/step - loss: 0.0873 - acc: 0.9992 - val_loss: 0.3442 - val_acc: 0.9350
 979 Epoch 902/1000
 980 66s 133ms/step - loss: 0.0867 - acc: 0.9994 - val_loss: 0.3425 - val_acc: 0.9361
 981 Epoch 903/1000
 982 66s 132ms/step - loss: 0.0874 - acc: 0.9991 - val_loss: 0.3432 - val_acc: 0.9358
 983 Epoch 904/1000
 984 66s 131ms/step - loss: 0.0872 - acc: 0.9992 - val_loss: 0.3431 - val_acc: 0.9360
 985 Epoch 905/1000
 986 66s 131ms/step - loss: 0.0871 - acc: 0.9991 - val_loss: 0.3426 - val_acc: 0.9371
 987 Epoch 906/1000
 988 66s 132ms/step - loss: 0.0868 - acc: 0.9991 - val_loss: 0.3422 - val_acc: 0.9371
 989 Epoch 907/1000
 990 66s 132ms/step - loss: 0.0869 - acc: 0.9993 - val_loss: 0.3418 - val_acc: 0.9368
 991 Epoch 908/1000
 992 66s 132ms/step - loss: 0.0870 - acc: 0.9991 - val_loss: 0.3415 - val_acc: 0.9366
 993 Epoch 909/1000
 994 66s 131ms/step - loss: 0.0863 - acc: 0.9995 - val_loss: 0.3410 - val_acc: 0.9371
 995 Epoch 910/1000
 996 66s 131ms/step - loss: 0.0870 - acc: 0.9991 - val_loss: 0.3405 - val_acc: 0.9363
 997 Epoch 911/1000
 998 66s 132ms/step - loss: 0.0864 - acc: 0.9995 - val_loss: 0.3412 - val_acc: 0.9367
 999 Epoch 912/1000
1000 66s 132ms/step - loss: 0.0862 - acc: 0.9995 - val_loss: 0.3403 - val_acc: 0.9370
1001 Epoch 913/1000
1002 78s 155ms/step - loss: 0.0862 - acc: 0.9995 - val_loss: 0.3399 - val_acc: 0.9368
1003 Epoch 914/1000
1004 84s 168ms/step - loss: 0.0860 - acc: 0.9997 - val_loss: 0.3402 - val_acc: 0.9373
1005 Epoch 915/1000
1006 65s 130ms/step - loss: 0.0865 - acc: 0.9994 - val_loss: 0.3403 - val_acc: 0.9371
1007 Epoch 916/1000
1008 65s 130ms/step - loss: 0.0866 - acc: 0.9993 - val_loss: 0.3399 - val_acc: 0.9369
1009 Epoch 917/1000
1010 65s 130ms/step - loss: 0.0868 - acc: 0.9992 - val_loss: 0.3385 - val_acc: 0.9378
1011 Epoch 918/1000
1012 65s 129ms/step - loss: 0.0865 - acc: 0.9993 - val_loss: 0.3374 - val_acc: 0.9376
1013 Epoch 919/1000
1014 65s 130ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3378 - val_acc: 0.9373
1015 Epoch 920/1000
1016 65s 130ms/step - loss: 0.0864 - acc: 0.9993 - val_loss: 0.3373 - val_acc: 0.9380
1017 Epoch 921/1000
1018 65s 130ms/step - loss: 0.0863 - acc: 0.9995 - val_loss: 0.3374 - val_acc: 0.9375
1019 Epoch 922/1000
1020 65s 130ms/step - loss: 0.0863 - acc: 0.9993 - val_loss: 0.3371 - val_acc: 0.9376
1021 Epoch 923/1000
1022 65s 129ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3372 - val_acc: 0.9370
1023 Epoch 924/1000
1024 65s 130ms/step - loss: 0.0860 - acc: 0.9994 - val_loss: 0.3374 - val_acc: 0.9369
1025 Epoch 925/1000
1026 65s 130ms/step - loss: 0.0860 - acc: 0.9996 - val_loss: 0.3375 - val_acc: 0.9368
1027 Epoch 926/1000
1028 65s 130ms/step - loss: 0.0862 - acc: 0.9994 - val_loss: 0.3378 - val_acc: 0.9373
1029 Epoch 927/1000
1030 65s 130ms/step - loss: 0.0864 - acc: 0.9992 - val_loss: 0.3384 - val_acc: 0.9371
1031 Epoch 928/1000
1032 65s 130ms/step - loss: 0.0863 - acc: 0.9993 - val_loss: 0.3386 - val_acc: 0.9367
1033 Epoch 929/1000
1034 65s 130ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3392 - val_acc: 0.9365
1035 Epoch 930/1000
1036 65s 130ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3386 - val_acc: 0.9368
1037 Epoch 931/1000
1038 65s 130ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3384 - val_acc: 0.9375
1039 Epoch 932/1000
1040 65s 130ms/step - loss: 0.0856 - acc: 0.9996 - val_loss: 0.3388 - val_acc: 0.9376
1041 Epoch 933/1000
1042 65s 130ms/step - loss: 0.0859 - acc: 0.9995 - val_loss: 0.3390 - val_acc: 0.9376
1043 Epoch 934/1000
1044 65s 130ms/step - loss: 0.0861 - acc: 0.9995 - val_loss: 0.3389 - val_acc: 0.9375
1045 Epoch 935/1000
1046 65s 130ms/step - loss: 0.0859 - acc: 0.9994 - val_loss: 0.3390 - val_acc: 0.9376
1047 Epoch 936/1000
1048 65s 130ms/step - loss: 0.0859 - acc: 0.9994 - val_loss: 0.3397 - val_acc: 0.9373
1049 Epoch 937/1000
1050 65s 130ms/step - loss: 0.0858 - acc: 0.9995 - val_loss: 0.3396 - val_acc: 0.9371
1051 Epoch 938/1000
1052 65s 130ms/step - loss: 0.0860 - acc: 0.9994 - val_loss: 0.3390 - val_acc: 0.9379
1053 Epoch 939/1000
1054 65s 130ms/step - loss: 0.0859 - acc: 0.9993 - val_loss: 0.3393 - val_acc: 0.9382
1055 Epoch 940/1000
1056 65s 130ms/step - loss: 0.0858 - acc: 0.9994 - val_loss: 0.3391 - val_acc: 0.9379
1057 Epoch 941/1000
1058 65s 130ms/step - loss: 0.0858 - acc: 0.9995 - val_loss: 0.3392 - val_acc: 0.9378
1059 Epoch 942/1000
1060 65s 130ms/step - loss: 0.0857 - acc: 0.9995 - val_loss: 0.3396 - val_acc: 0.9382
1061 Epoch 943/1000
1062 65s 130ms/step - loss: 0.0858 - acc: 0.9995 - val_loss: 0.3403 - val_acc: 0.9376
1063 Epoch 944/1000
1064 65s 130ms/step - loss: 0.0859 - acc: 0.9993 - val_loss: 0.3405 - val_acc: 0.9374
1065 Epoch 945/1000
1066 65s 130ms/step - loss: 0.0854 - acc: 0.9996 - val_loss: 0.3402 - val_acc: 0.9371
1067 Epoch 946/1000
1068 65s 130ms/step - loss: 0.0859 - acc: 0.9994 - val_loss: 0.3398 - val_acc: 0.9376
1069 Epoch 947/1000
1070 65s 130ms/step - loss: 0.0857 - acc: 0.9994 - val_loss: 0.3397 - val_acc: 0.9371
1071 Epoch 948/1000
1072 65s 129ms/step - loss: 0.0855 - acc: 0.9996 - val_loss: 0.3396 - val_acc: 0.9375
1073 Epoch 949/1000
1074 65s 130ms/step - loss: 0.0853 - acc: 0.9996 - val_loss: 0.3398 - val_acc: 0.9376
1075 Epoch 950/1000
1076 65s 130ms/step - loss: 0.0856 - acc: 0.9996 - val_loss: 0.3397 - val_acc: 0.9378
1077 Epoch 951/1000
1078 65s 130ms/step - loss: 0.0856 - acc: 0.9994 - val_loss: 0.3393 - val_acc: 0.9375
1079 Epoch 952/1000
1080 65s 130ms/step - loss: 0.0857 - acc: 0.9996 - val_loss: 0.3397 - val_acc: 0.9374
1081 Epoch 953/1000
1082 65s 130ms/step - loss: 0.0854 - acc: 0.9995 - val_loss: 0.3400 - val_acc: 0.9378
1083 Epoch 954/1000
1084 65s 130ms/step - loss: 0.0856 - acc: 0.9995 - val_loss: 0.3401 - val_acc: 0.9368
1085 Epoch 955/1000
1086 65s 129ms/step - loss: 0.0855 - acc: 0.9994 - val_loss: 0.3403 - val_acc: 0.9370
1087 Epoch 956/1000
1088 65s 130ms/step - loss: 0.0856 - acc: 0.9994 - val_loss: 0.3405 - val_acc: 0.9371
1089 Epoch 957/1000
1090 65s 130ms/step - loss: 0.0857 - acc: 0.9994 - val_loss: 0.3408 - val_acc: 0.9375
1091 Epoch 958/1000
1092 65s 130ms/step - loss: 0.0854 - acc: 0.9994 - val_loss: 0.3405 - val_acc: 0.9374
1093 Epoch 959/1000
1094 65s 130ms/step - loss: 0.0856 - acc: 0.9993 - val_loss: 0.3408 - val_acc: 0.9375
1095 Epoch 960/1000
1096 65s 130ms/step - loss: 0.0856 - acc: 0.9995 - val_loss: 0.3407 - val_acc: 0.9369
1097 Epoch 961/1000
1098 65s 130ms/step - loss: 0.0853 - acc: 0.9996 - val_loss: 0.3402 - val_acc: 0.9371
1099 Epoch 962/1000
1100 65s 129ms/step - loss: 0.0855 - acc: 0.9994 - val_loss: 0.3399 - val_acc: 0.9371
1101 Epoch 963/1000
1102 65s 129ms/step - loss: 0.0852 - acc: 0.9996 - val_loss: 0.3400 - val_acc: 0.9378
1103 Epoch 964/1000
1104 65s 129ms/step - loss: 0.0856 - acc: 0.9994 - val_loss: 0.3399 - val_acc: 0.9375
1105 Epoch 965/1000
1106 65s 130ms/step - loss: 0.0854 - acc: 0.9994 - val_loss: 0.3396 - val_acc: 0.9375
1107 Epoch 966/1000
1108 65s 130ms/step - loss: 0.0852 - acc: 0.9995 - val_loss: 0.3391 - val_acc: 0.9368
1109 Epoch 967/1000
1110 65s 130ms/step - loss: 0.0854 - acc: 0.9994 - val_loss: 0.3383 - val_acc: 0.9374
1111 Epoch 968/1000
1112 65s 130ms/step - loss: 0.0854 - acc: 0.9994 - val_loss: 0.3384 - val_acc: 0.9375
1113 Epoch 969/1000
1114 65s 130ms/step - loss: 0.0851 - acc: 0.9997 - val_loss: 0.3383 - val_acc: 0.9375
1115 Epoch 970/1000
1116 65s 129ms/step - loss: 0.0853 - acc: 0.9995 - val_loss: 0.3388 - val_acc: 0.9365
1117 Epoch 971/1000
1118 65s 130ms/step - loss: 0.0851 - acc: 0.9996 - val_loss: 0.3381 - val_acc: 0.9356
1119 Epoch 972/1000
1120 65s 130ms/step - loss: 0.0855 - acc: 0.9994 - val_loss: 0.3387 - val_acc: 0.9362
1121 Epoch 973/1000
1122 65s 130ms/step - loss: 0.0857 - acc: 0.9994 - val_loss: 0.3385 - val_acc: 0.9372
1123 Epoch 974/1000
1124 65s 130ms/step - loss: 0.0854 - acc: 0.9995 - val_loss: 0.3385 - val_acc: 0.9373
1125 Epoch 975/1000
1126 65s 130ms/step - loss: 0.0851 - acc: 0.9996 - val_loss: 0.3380 - val_acc: 0.9375
1127 Epoch 976/1000
1128 65s 130ms/step - loss: 0.0853 - acc: 0.9994 - val_loss: 0.3380 - val_acc: 0.9379
1129 Epoch 977/1000
1130 65s 130ms/step - loss: 0.0854 - acc: 0.9994 - val_loss: 0.3374 - val_acc: 0.9376
1131 Epoch 978/1000
1132 65s 130ms/step - loss: 0.0851 - acc: 0.9996 - val_loss: 0.3376 - val_acc: 0.9379
1133 Epoch 979/1000
1134 65s 130ms/step - loss: 0.0853 - acc: 0.9995 - val_loss: 0.3380 - val_acc: 0.9378
1135 Epoch 980/1000
1136 65s 130ms/step - loss: 0.0852 - acc: 0.9995 - val_loss: 0.3376 - val_acc: 0.9381
1137 Epoch 981/1000
1138 65s 130ms/step - loss: 0.0854 - acc: 0.9995 - val_loss: 0.3377 - val_acc: 0.9381
1139 Epoch 982/1000
1140 65s 129ms/step - loss: 0.0849 - acc: 0.9996 - val_loss: 0.3373 - val_acc: 0.9384
1141 Epoch 983/1000
1142 65s 130ms/step - loss: 0.0852 - acc: 0.9993 - val_loss: 0.3372 - val_acc: 0.9379
1143 Epoch 984/1000
1144 65s 130ms/step - loss: 0.0848 - acc: 0.9997 - val_loss: 0.3368 - val_acc: 0.9381
1145 Epoch 985/1000
1146 65s 130ms/step - loss: 0.0852 - acc: 0.9994 - val_loss: 0.3373 - val_acc: 0.9382
1147 Epoch 986/1000
1148 65s 130ms/step - loss: 0.0847 - acc: 0.9997 - val_loss: 0.3372 - val_acc: 0.9380
1149 Epoch 987/1000
1150 65s 130ms/step - loss: 0.0848 - acc: 0.9996 - val_loss: 0.3371 - val_acc: 0.9387
1151 Epoch 988/1000
1152 65s 130ms/step - loss: 0.0853 - acc: 0.9993 - val_loss: 0.3377 - val_acc: 0.9380
1153 Epoch 989/1000
1154 65s 130ms/step - loss: 0.0851 - acc: 0.9995 - val_loss: 0.3371 - val_acc: 0.9385
1155 Epoch 990/1000
1156 65s 130ms/step - loss: 0.0848 - acc: 0.9996 - val_loss: 0.3379 - val_acc: 0.9384
1157 Epoch 991/1000
1158 65s 130ms/step - loss: 0.0849 - acc: 0.9994 - val_loss: 0.3377 - val_acc: 0.9380
1159 Epoch 992/1000
1160 65s 130ms/step - loss: 0.0852 - acc: 0.9993 - val_loss: 0.3370 - val_acc: 0.9381
1161 Epoch 993/1000
1162 65s 130ms/step - loss: 0.0851 - acc: 0.9994 - val_loss: 0.3371 - val_acc: 0.9380
1163 Epoch 994/1000
1164 65s 130ms/step - loss: 0.0848 - acc: 0.9996 - val_loss: 0.3371 - val_acc: 0.9381
1165 Epoch 995/1000
1166 65s 130ms/step - loss: 0.0849 - acc: 0.9994 - val_loss: 0.3381 - val_acc: 0.9381
1167 Epoch 996/1000
1168 65s 130ms/step - loss: 0.0849 - acc: 0.9996 - val_loss: 0.3379 - val_acc: 0.9379
1169 Epoch 997/1000
1170 65s 130ms/step - loss: 0.0853 - acc: 0.9993 - val_loss: 0.3384 - val_acc: 0.9377
1171 Epoch 998/1000
1172 65s 129ms/step - loss: 0.0849 - acc: 0.9995 - val_loss: 0.3393 - val_acc: 0.9369
1173 Epoch 999/1000
1174 65s 130ms/step - loss: 0.0849 - acc: 0.9994 - val_loss: 0.3395 - val_acc: 0.9369
1175 Epoch 1000/1000
1176 65s 130ms/step - loss: 0.0847 - acc: 0.9996 - val_loss: 0.3389 - val_acc: 0.9371
1177 Train loss: 0.08910960255563259
1178 Train accuracy: 0.9977200021743774
1179 Test loss: 0.3388938118517399
1180 Test accuracy: 0.9371000009775162

测试准确率到了93.71%，比之前的都高一点。

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458

https://ieeexplore.ieee.org/document/8998530