deeplearn@ML-RefVm-967342:~$ nvidia-smi Sat Oct 1 16:03:16 2022 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 470.141.03 Driver Version: 470.141.03 CUDA Version: 11.4 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla K80 On | 00000001:00:00.0 Off | 0 | | N/A 41C P8 27W / 149W | 4MiB / 11441MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 1015 G /usr/lib/xorg/Xorg 3MiB | +-----------------------------------------------------------------------------+ deeplearn@ML-RefVm-967342:~$ python -c 'import tensorflow; print(tensorflow.__version__)' /anaconda/envs/py38_default/lib/python3.8/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.1 warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}" 2.9.1 deeplearn@ML-RefVm-967342:~$ pip install --upgrade scipy Requirement already satisfied: scipy in /anaconda/envs/py38_default/lib/python3.8/site-packages (1.7.3) Collecting scipy Downloading scipy-1.9.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.4/43.4 MB 14.5 MB/s eta 0:00:00 Requirement already satisfied: numpy<1.25.0,>=1.18.5 in /anaconda/envs/py38_default/lib/python3.8/site-packages (from scipy) (1.23.1) Installing collected packages: scipy Attempting uninstall: scipy Found existing installation: scipy 1.7.3 Uninstalling scipy-1.7.3: Successfully uninstalled scipy-1.7.3 Successfully installed scipy-1.9.1 deeplearn@ML-RefVm-967342:~$ python -c 'import tensorflow; print(tensorflow.__version__)' 2.9.1 deeplearn@ML-RefVm-967342:~$ mkdir mnist deeplearn@ML-RefVm-967342:~$ cd mnist deeplearn@ML-RefVm-967342:~/mnist$ kaggle competitions download -c ml530-2022-fall-mnist Warning: Your Kaggle API key is readable by other users on this system! To fix this, you can run 'chmod 600 /home/deeplearn/.kaggle/kaggle.json' Downloading ml530-2022-fall-mnist.zip to /home/deeplearn/mnist 51%|████████████████████████████████████████████████ | 16.0M/31.3M [00:00<00:00, 162MB/s] 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 31.3M/31.3M [00:00<00:00, 160MB/s] deeplearn@ML-RefVm-967342:~/mnist$ wget https://cross-entropy.net/ML530/mnist-search.py.txt --2022-10-01 16:05:10-- https://cross-entropy.net/ML530/mnist-search.py.txt Resolving cross-entropy.net (cross-entropy.net)... 107.180.57.14 Connecting to cross-entropy.net (cross-entropy.net)|107.180.57.14|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 4920 (4.8K) [text/plain] Saving to: ‘mnist-search.py.txt’ mnist-search.py.txt 100%[============================================================>] 4.80K --.-KB/s in 0s 2022-10-01 16:05:10 (2.55 GB/s) - ‘mnist-search.py.txt’ saved [4920/4920] deeplearn@ML-RefVm-967342:~/mnist$ time python mnist-search.py.txt 2022-10-01 16:05:41.511983: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-10-01 16:05:51.905574: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 10790 MB memory: -> device: 0, name: Tesla K80, pci bus id: 0001:00:00.0, compute capability: 3.7 Search space summary Default search space size: 8 depth (Int) {'default': None, 'conditions': [], 'min_value': 0, 'max_value': 4, 'step': 1, 'sampling': None} width (Choice) {'default': 64, 'conditions': [], 'values': [64, 128, 256, 512], 'ordered': True} activation (Choice) {'default': 'linear', 'conditions': [], 'values': ['linear', 'relu', 'sigmoid', 'swish', 'tanh'], 'ordered': False} regularization_function (Choice) {'default': 'none', 'conditions': [], 'values': ['none', 'l1', 'l2', 'l1_l2'], 'ordered': False} regularization_penalty (Choice) {'default': 0.0001, 'conditions': [], 'values': [0.0001, 1e-05], 'ordered': True} dropout (Float) {'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.5, 'step': 0.1, 'sampling': None} optimizer (Choice) {'default': 'adam', 'conditions': [], 'values': ['adam', 'rmsprop', 'sgd'], 'ordered': False} learning_rate (Choice) {'default': 0.01, 'conditions': [], 'values': [0.01, 0.001, 0.0001], 'ordered': True} Search: Running Trial #1 Value |Best Value So Far |Hyperparameter 0 |? |depth 64 |? |width tanh |? |activation l2 |? |regularization_function 0.0001 |? |regularization_penalty 0.1 |? |dropout sgd |? |optimizer 0.01 |? |learning_rate 2 |? |tuner/epochs 0 |? |tuner/initial_epoch 3 |? |tuner/bracket 0 |? |tuner/round Epoch 1/2 1688/1688 [==============================] - 18s 2ms/step - loss: 0.3704 - accuracy: 0.8971 - val_loss: 0.3741 - val_accuracy: 0.9005 - lr: 0.0100 Epoch 2/2 1688/1688 [==============================] - 3s 2ms/step - loss: 0.3360 - accuracy: 0.9104 - val_loss: 0.3812 - val_accuracy: 0.9032 - lr: 0.0100 Trial 1 Complete [00h 00m 23s] val_accuracy: 0.903166651725769 Best val_accuracy So Far: 0.903166651725769 Total elapsed time: 00h 00m 23s Search: Running Trial #2 Value |Best Value So Far |Hyperparameter 3 |0 |depth 256 |64 |width tanh |tanh |activation l1 |l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.3 |0.1 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 32 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 1688/1688 [==============================] - 6s 3ms/step - loss: 1.2091 - accuracy: 0.7851 - val_loss: 1.2379 - val_accuracy: 0.8327 - lr: 0.0100 Epoch 2/2 1688/1688 [==============================] - 5s 3ms/step - loss: 1.5754 - accuracy: 0.7788 - val_loss: 1.2891 - val_accuracy: 0.8777 - lr: 0.0100 Trial 2 Complete [00h 00m 12s] val_accuracy: 0.8776666522026062 Best val_accuracy So Far: 0.903166651725769 Total elapsed time: 00h 00m 35s Search: Running Trial #3 Value |Best Value So Far |Hyperparameter 3 |0 |depth 512 |64 |width swish |tanh |activation l2 |l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.2 |0.1 |dropout sgd |sgd |optimizer 0.0001 |0.01 |learning_rate 512 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 106/106 [==============================] - 2s 9ms/step - loss: 1.3890 - accuracy: 0.7159 - val_loss: 0.5960 - val_accuracy: 0.8780 - lr: 1.0000e-04 Epoch 2/2 106/106 [==============================] - 1s 6ms/step - loss: 0.5748 - accuracy: 0.8821 - val_loss: 0.4774 - val_accuracy: 0.9092 - lr: 1.0000e-04 Trial 3 Complete [00h 00m 03s] val_accuracy: 0.909166693687439 Best val_accuracy So Far: 0.909166693687439 Total elapsed time: 00h 00m 37s Search: Running Trial #4 Value |Best Value So Far |Hyperparameter 4 |3 |depth 128 |512 |width tanh |swish |activation l1_l2 |l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.2 |0.2 |dropout sgd |sgd |optimizer 0.0001 |0.0001 |learning_rate 256 |512 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 2s 4ms/step - loss: 1.3072 - accuracy: 0.6346 - val_loss: 0.6114 - val_accuracy: 0.8642 - lr: 1.0000e-04 Epoch 2/2 211/211 [==============================] - 1s 3ms/step - loss: 0.6079 - accuracy: 0.8504 - val_loss: 0.4401 - val_accuracy: 0.8982 - lr: 1.0000e-04 Trial 4 Complete [00h 00m 03s] val_accuracy: 0.8981666564941406 Best val_accuracy So Far: 0.909166693687439 Total elapsed time: 00h 00m 40s Search: Running Trial #5 Value |Best Value So Far |Hyperparameter 4 |3 |depth 512 |512 |width swish |swish |activation none |l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.5 |0.2 |dropout adam |sgd |optimizer 0.01 |0.0001 |learning_rate 256 |512 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 2s 6ms/step - loss: 0.7755 - accuracy: 0.8083 - val_loss: 0.2501 - val_accuracy: 0.9355 - lr: 0.0100 Epoch 2/2 211/211 [==============================] - 1s 5ms/step - loss: 0.5245 - accuracy: 0.8710 - val_loss: 0.2521 - val_accuracy: 0.9330 - lr: 0.0100 Trial 5 Complete [00h 00m 03s] val_accuracy: 0.9355000257492065 Best val_accuracy So Far: 0.9355000257492065 Total elapsed time: 00h 00m 43s Search: Running Trial #6 Value |Best Value So Far |Hyperparameter 1 |4 |depth 64 |512 |width sigmoid |swish |activation l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.2 |0.5 |dropout sgd |adam |optimizer 0.001 |0.01 |learning_rate 64 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 2s 2ms/step - loss: 0.7172 - accuracy: 0.8216 - val_loss: 0.3763 - val_accuracy: 0.9085 - lr: 0.0010 Epoch 2/2 844/844 [==============================] - 2s 2ms/step - loss: 0.3823 - accuracy: 0.9042 - val_loss: 0.3062 - val_accuracy: 0.9268 - lr: 0.0010 Trial 6 Complete [00h 00m 04s] val_accuracy: 0.9268333315849304 Best val_accuracy So Far: 0.9355000257492065 Total elapsed time: 00h 00m 48s Search: Running Trial #7 Value |Best Value So Far |Hyperparameter 3 |4 |depth 256 |512 |width tanh |swish |activation l1_l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.5 |dropout adam |adam |optimizer 0.001 |0.01 |learning_rate 64 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 3s 3ms/step - loss: 1.3487 - accuracy: 0.8782 - val_loss: 0.9455 - val_accuracy: 0.9177 - lr: 0.0010 Epoch 2/2 844/844 [==============================] - 2s 3ms/step - loss: 0.8121 - accuracy: 0.9118 - val_loss: 0.6499 - val_accuracy: 0.9285 - lr: 0.0010 Trial 7 Complete [00h 00m 06s] val_accuracy: 0.9284999966621399 Best val_accuracy So Far: 0.9355000257492065 Total elapsed time: 00h 00m 54s Search: Running Trial #8 Value |Best Value So Far |Hyperparameter 3 |4 |depth 256 |512 |width sigmoid |swish |activation l1 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.5 |dropout sgd |adam |optimizer 0.0001 |0.01 |learning_rate 1024 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 53/53 [==============================] - 1s 9ms/step - loss: 2.5489 - accuracy: 0.1080 - val_loss: 2.4235 - val_accuracy: 0.1050 - lr: 1.0000e-04 Epoch 2/2 53/53 [==============================] - 0s 6ms/step - loss: 2.4296 - accuracy: 0.1327 - val_loss: 2.3822 - val_accuracy: 0.2847 - lr: 1.0000e-04 Trial 8 Complete [00h 00m 02s] val_accuracy: 0.28466665744781494 Best val_accuracy So Far: 0.9355000257492065 Total elapsed time: 00h 00m 56s Search: Running Trial #9 Value |Best Value So Far |Hyperparameter 3 |4 |depth 64 |512 |width tanh |swish |activation l1_l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.5 |dropout rmsprop |adam |optimizer 0.001 |0.01 |learning_rate 32 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 1688/1688 [==============================] - 5s 3ms/step - loss: 0.3839 - accuracy: 0.8952 - val_loss: 0.2445 - val_accuracy: 0.9363 - lr: 0.0010 Epoch 2/2 1688/1688 [==============================] - 4s 3ms/step - loss: 0.2359 - accuracy: 0.9383 - val_loss: 0.1853 - val_accuracy: 0.9535 - lr: 0.0010 Trial 9 Complete [00h 00m 10s] val_accuracy: 0.953499972820282 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 06s Search: Running Trial #10 Value |Best Value So Far |Hyperparameter 3 |3 |depth 256 |64 |width sigmoid |tanh |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.1 |dropout sgd |rmsprop |optimizer 0.001 |0.001 |learning_rate 128 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 422/422 [==============================] - 2s 3ms/step - loss: 1.9515 - accuracy: 0.5882 - val_loss: 1.1939 - val_accuracy: 0.8448 - lr: 0.0010 Epoch 2/2 422/422 [==============================] - 1s 3ms/step - loss: 1.2120 - accuracy: 0.8087 - val_loss: 0.9840 - val_accuracy: 0.8808 - lr: 0.0010 Trial 10 Complete [00h 00m 04s] val_accuracy: 0.8808333277702332 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 09s Search: Running Trial #11 Value |Best Value So Far |Hyperparameter 4 |3 |depth 64 |64 |width linear |tanh |activation none |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.1 |dropout adam |rmsprop |optimizer 0.01 |0.001 |learning_rate 32 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 1688/1688 [==============================] - 5s 2ms/step - loss: 1.1804 - accuracy: 0.6674 - val_loss: 0.7649 - val_accuracy: 0.7830 - lr: 0.0100 Epoch 2/2 1688/1688 [==============================] - 4s 2ms/step - loss: 1.2423 - accuracy: 0.6550 - val_loss: 0.7476 - val_accuracy: 0.7878 - lr: 0.0100 Trial 11 Complete [00h 00m 09s] val_accuracy: 0.7878333330154419 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 19s Search: Running Trial #12 Value |Best Value So Far |Hyperparameter 1 |3 |depth 512 |64 |width linear |tanh |activation l1_l2 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.1 |dropout adam |rmsprop |optimizer 0.01 |0.001 |learning_rate 512 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 106/106 [==============================] - 1s 5ms/step - loss: 1.3635 - accuracy: 0.8479 - val_loss: 0.7116 - val_accuracy: 0.8980 - lr: 0.0100 Epoch 2/2 106/106 [==============================] - 0s 3ms/step - loss: 0.7058 - accuracy: 0.8858 - val_loss: 0.6985 - val_accuracy: 0.8873 - lr: 0.0100 Trial 12 Complete [00h 00m 02s] val_accuracy: 0.8980000019073486 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 20s Search: Running Trial #13 Value |Best Value So Far |Hyperparameter 4 |3 |depth 64 |64 |width linear |tanh |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.1 |dropout adam |rmsprop |optimizer 0.0001 |0.001 |learning_rate 32 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 1688/1688 [==============================] - 5s 3ms/step - loss: 1.3922 - accuracy: 0.6334 - val_loss: 0.7040 - val_accuracy: 0.8828 - lr: 1.0000e-04 Epoch 2/2 1688/1688 [==============================] - 5s 3ms/step - loss: 0.9282 - accuracy: 0.7949 - val_loss: 0.6343 - val_accuracy: 0.8940 - lr: 1.0000e-04 Trial 13 Complete [00h 00m 11s] val_accuracy: 0.8939999938011169 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 31s Search: Running Trial #14 Value |Best Value So Far |Hyperparameter 0 |3 |depth 128 |64 |width relu |tanh |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.1 |dropout sgd |rmsprop |optimizer 0.01 |0.001 |learning_rate 64 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 2s 2ms/step - loss: 0.3600 - accuracy: 0.8980 - val_loss: 0.3124 - val_accuracy: 0.9167 - lr: 0.0100 Epoch 2/2 844/844 [==============================] - 2s 2ms/step - loss: 0.3052 - accuracy: 0.9154 - val_loss: 0.3277 - val_accuracy: 0.9117 - lr: 0.0100 Trial 14 Complete [00h 00m 04s] val_accuracy: 0.9166666865348816 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 35s Search: Running Trial #15 Value |Best Value So Far |Hyperparameter 2 |3 |depth 512 |64 |width tanh |tanh |activation l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.1 |dropout adam |rmsprop |optimizer 0.0001 |0.001 |learning_rate 128 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 422/422 [==============================] - 2s 4ms/step - loss: 0.5455 - accuracy: 0.8554 - val_loss: 0.3339 - val_accuracy: 0.9070 - lr: 1.0000e-04 Epoch 2/2 422/422 [==============================] - 2s 4ms/step - loss: 0.3077 - accuracy: 0.9144 - val_loss: 0.2963 - val_accuracy: 0.9193 - lr: 1.0000e-04 Trial 15 Complete [00h 00m 04s] val_accuracy: 0.9193333387374878 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 39s Search: Running Trial #16 Value |Best Value So Far |Hyperparameter 3 |3 |depth 256 |64 |width relu |tanh |activation l2 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.1 |dropout sgd |rmsprop |optimizer 0.001 |0.001 |learning_rate 1024 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 53/53 [==============================] - 1s 11ms/step - loss: 0.9071 - accuracy: 0.7393 - val_loss: 0.3319 - val_accuracy: 0.9243 - lr: 0.0010 Epoch 2/2 53/53 [==============================] - 0s 7ms/step - loss: 0.3524 - accuracy: 0.9196 - val_loss: 0.2450 - val_accuracy: 0.9507 - lr: 0.0010 Trial 16 Complete [00h 00m 02s] val_accuracy: 0.9506666660308838 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 41s Search: Running Trial #17 Value |Best Value So Far |Hyperparameter 1 |3 |depth 128 |64 |width tanh |tanh |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.1 |dropout rmsprop |rmsprop |optimizer 0.01 |0.001 |learning_rate 256 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 1s 3ms/step - loss: 0.6959 - accuracy: 0.8711 - val_loss: 0.5243 - val_accuracy: 0.9250 - lr: 0.0100 Epoch 2/2 211/211 [==============================] - 1s 3ms/step - loss: 0.5953 - accuracy: 0.9006 - val_loss: 0.5004 - val_accuracy: 0.9337 - lr: 0.0100 Trial 17 Complete [00h 00m 02s] val_accuracy: 0.9336666464805603 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 43s Search: Running Trial #18 Value |Best Value So Far |Hyperparameter 1 |3 |depth 64 |64 |width swish |tanh |activation l1_l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout adam |rmsprop |optimizer 0.01 |0.001 |learning_rate 64 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 2s 2ms/step - loss: 0.4384 - accuracy: 0.8830 - val_loss: 0.2471 - val_accuracy: 0.9465 - lr: 0.0100 Epoch 2/2 844/844 [==============================] - 2s 2ms/step - loss: 0.3833 - accuracy: 0.9066 - val_loss: 0.2617 - val_accuracy: 0.9522 - lr: 0.0100 Trial 18 Complete [00h 00m 05s] val_accuracy: 0.9521666765213013 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 48s Search: Running Trial #19 Value |Best Value So Far |Hyperparameter 3 |3 |depth 128 |64 |width swish |tanh |activation none |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout adam |rmsprop |optimizer 0.001 |0.001 |learning_rate 128 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 422/422 [==============================] - 2s 3ms/step - loss: 0.5463 - accuracy: 0.8325 - val_loss: 0.2200 - val_accuracy: 0.9348 - lr: 0.0010 Epoch 2/2 422/422 [==============================] - 1s 3ms/step - loss: 0.2621 - accuracy: 0.9217 - val_loss: 0.1502 - val_accuracy: 0.9520 - lr: 0.0010 Trial 19 Complete [00h 00m 03s] val_accuracy: 0.9520000219345093 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 52s Search: Running Trial #20 Value |Best Value So Far |Hyperparameter 4 |3 |depth 128 |64 |width swish |tanh |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.1 |dropout rmsprop |rmsprop |optimizer 0.0001 |0.001 |learning_rate 1024 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 53/53 [==============================] - 1s 10ms/step - loss: 3.0222 - accuracy: 0.1744 - val_loss: 2.9235 - val_accuracy: 0.5942 - lr: 1.0000e-04 Epoch 2/2 53/53 [==============================] - 0s 6ms/step - loss: 2.7943 - accuracy: 0.3334 - val_loss: 2.5085 - val_accuracy: 0.5928 - lr: 1.0000e-04 Trial 20 Complete [00h 00m 02s] val_accuracy: 0.5941666960716248 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 54s Search: Running Trial #21 Value |Best Value So Far |Hyperparameter 3 |3 |depth 128 |64 |width tanh |tanh |activation none |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.2 |0.1 |dropout sgd |rmsprop |optimizer 0.0001 |0.001 |learning_rate 128 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 422/422 [==============================] - 2s 3ms/step - loss: 0.9317 - accuracy: 0.7325 - val_loss: 0.4157 - val_accuracy: 0.8853 - lr: 1.0000e-04 Epoch 2/2 422/422 [==============================] - 1s 3ms/step - loss: 0.4189 - accuracy: 0.8787 - val_loss: 0.3235 - val_accuracy: 0.9065 - lr: 1.0000e-04 Trial 21 Complete [00h 00m 03s] val_accuracy: 0.906499981880188 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 57s Search: Running Trial #22 Value |Best Value So Far |Hyperparameter 2 |3 |depth 512 |64 |width tanh |tanh |activation none |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.1 |dropout rmsprop |rmsprop |optimizer 0.001 |0.001 |learning_rate 512 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 106/106 [==============================] - 1s 6ms/step - loss: 0.3964 - accuracy: 0.8829 - val_loss: 0.2731 - val_accuracy: 0.9223 - lr: 0.0010 Epoch 2/2 106/106 [==============================] - 1s 6ms/step - loss: 0.2222 - accuracy: 0.9349 - val_loss: 0.2102 - val_accuracy: 0.9395 - lr: 0.0010 Trial 22 Complete [00h 00m 02s] val_accuracy: 0.9394999742507935 Best val_accuracy So Far: 0.953499972820282 Total elapsed time: 00h 01m 59s Search: Running Trial #23 Value |Best Value So Far |Hyperparameter 1 |3 |depth 128 |64 |width swish |tanh |activation l1_l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout adam |rmsprop |optimizer 0.01 |0.001 |learning_rate 64 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 3s 2ms/step - loss: 0.4368 - accuracy: 0.8927 - val_loss: 0.2873 - val_accuracy: 0.9473 - lr: 0.0100 Epoch 2/2 844/844 [==============================] - 2s 2ms/step - loss: 0.4040 - accuracy: 0.9143 - val_loss: 0.2796 - val_accuracy: 0.9547 - lr: 0.0100 Trial 23 Complete [00h 00m 05s] val_accuracy: 0.9546666741371155 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 04s Search: Running Trial #24 Value |Best Value So Far |Hyperparameter 1 |1 |depth 512 |128 |width sigmoid |swish |activation none |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.4 |dropout adam |adam |optimizer 0.0001 |0.01 |learning_rate 64 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 2s 2ms/step - loss: 1.1587 - accuracy: 0.6616 - val_loss: 0.5935 - val_accuracy: 0.8650 - lr: 1.0000e-04 Epoch 2/2 844/844 [==============================] - 2s 2ms/step - loss: 0.5577 - accuracy: 0.8464 - val_loss: 0.4226 - val_accuracy: 0.8907 - lr: 1.0000e-04 Trial 24 Complete [00h 00m 05s] val_accuracy: 0.890666663646698 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 09s Search: Running Trial #25 Value |Best Value So Far |Hyperparameter 3 |1 |depth 64 |128 |width swish |swish |activation l2 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.4 |dropout rmsprop |adam |optimizer 0.0001 |0.01 |learning_rate 128 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 422/422 [==============================] - 2s 3ms/step - loss: 1.6741 - accuracy: 0.4804 - val_loss: 0.8080 - val_accuracy: 0.7932 - lr: 1.0000e-04 Epoch 2/2 422/422 [==============================] - 1s 3ms/step - loss: 0.8685 - accuracy: 0.7350 - val_loss: 0.5302 - val_accuracy: 0.8608 - lr: 1.0000e-04 Trial 25 Complete [00h 00m 04s] val_accuracy: 0.8608333468437195 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 12s Search: Running Trial #26 Value |Best Value So Far |Hyperparameter 3 |1 |depth 128 |128 |width linear |swish |activation none |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.2 |0.4 |dropout adam |adam |optimizer 0.0001 |0.01 |learning_rate 128 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 422/422 [==============================] - 2s 3ms/step - loss: 0.8912 - accuracy: 0.7229 - val_loss: 0.4039 - val_accuracy: 0.8892 - lr: 1.0000e-04 Epoch 2/2 422/422 [==============================] - 1s 3ms/step - loss: 0.4417 - accuracy: 0.8670 - val_loss: 0.3378 - val_accuracy: 0.9030 - lr: 1.0000e-04 Trial 26 Complete [00h 00m 03s] val_accuracy: 0.902999997138977 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 16s Search: Running Trial #27 Value |Best Value So Far |Hyperparameter 3 |1 |depth 128 |128 |width swish |swish |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.4 |dropout sgd |adam |optimizer 0.0001 |0.01 |learning_rate 1024 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 53/53 [==============================] - 1s 9ms/step - loss: 2.8285 - accuracy: 0.2636 - val_loss: 2.6504 - val_accuracy: 0.6590 - lr: 1.0000e-04 Epoch 2/2 53/53 [==============================] - 0s 6ms/step - loss: 2.4295 - accuracy: 0.5070 - val_loss: 2.0203 - val_accuracy: 0.7550 - lr: 1.0000e-04 Trial 27 Complete [00h 00m 02s] val_accuracy: 0.7549999952316284 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 18s Search: Running Trial #28 Value |Best Value So Far |Hyperparameter 1 |1 |depth 256 |128 |width relu |swish |activation l1_l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout sgd |adam |optimizer 0.0001 |0.01 |learning_rate 256 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 1s 4ms/step - loss: 1.2446 - accuracy: 0.7053 - val_loss: 0.6705 - val_accuracy: 0.8668 - lr: 1.0000e-04 Epoch 2/2 211/211 [==============================] - 1s 3ms/step - loss: 0.5654 - accuracy: 0.8754 - val_loss: 0.4681 - val_accuracy: 0.8948 - lr: 1.0000e-04 Trial 28 Complete [00h 00m 02s] val_accuracy: 0.8948333263397217 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 20s Search: Running Trial #29 Value |Best Value So Far |Hyperparameter 2 |1 |depth 64 |128 |width sigmoid |swish |activation l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.4 |dropout adam |adam |optimizer 0.0001 |0.01 |learning_rate 64 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 3s 3ms/step - loss: 2.1279 - accuracy: 0.2777 - val_loss: 1.7553 - val_accuracy: 0.6807 - lr: 1.0000e-04 Epoch 2/2 844/844 [==============================] - 2s 3ms/step - loss: 1.5354 - accuracy: 0.5975 - val_loss: 1.1941 - val_accuracy: 0.7653 - lr: 1.0000e-04 Trial 29 Complete [00h 00m 05s] val_accuracy: 0.765333354473114 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 25s Search: Running Trial #30 Value |Best Value So Far |Hyperparameter 1 |1 |depth 512 |128 |width relu |swish |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.4 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 64 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 3s 3ms/step - loss: 1.0759 - accuracy: 0.8618 - val_loss: 0.7818 - val_accuracy: 0.9395 - lr: 0.0100 Epoch 2/2 844/844 [==============================] - 2s 3ms/step - loss: 0.9452 - accuracy: 0.8821 - val_loss: 0.7403 - val_accuracy: 0.9427 - lr: 0.0100 Trial 30 Complete [00h 00m 05s] val_accuracy: 0.9426666498184204 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 31s Search: Running Trial #31 Value |Best Value So Far |Hyperparameter 3 |1 |depth 256 |128 |width sigmoid |swish |activation l1 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout adam |adam |optimizer 0.001 |0.01 |learning_rate 32 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 1688/1688 [==============================] - 6s 3ms/step - loss: 0.6283 - accuracy: 0.8521 - val_loss: 0.3907 - val_accuracy: 0.9315 - lr: 0.0010 Epoch 2/2 1688/1688 [==============================] - 5s 3ms/step - loss: 0.3636 - accuracy: 0.9378 - val_loss: 0.3105 - val_accuracy: 0.9535 - lr: 0.0010 Trial 31 Complete [00h 00m 11s] val_accuracy: 0.953499972820282 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 42s Search: Running Trial #32 Value |Best Value So Far |Hyperparameter 2 |1 |depth 256 |128 |width sigmoid |swish |activation l1_l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.4 |dropout sgd |adam |optimizer 0.0001 |0.01 |learning_rate 256 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 1s 4ms/step - loss: 2.2883 - accuracy: 0.2441 - val_loss: 1.9428 - val_accuracy: 0.7093 - lr: 1.0000e-04 Epoch 2/2 211/211 [==============================] - 1s 3ms/step - loss: 1.6489 - accuracy: 0.5899 - val_loss: 1.2577 - val_accuracy: 0.7797 - lr: 1.0000e-04 Trial 32 Complete [00h 00m 03s] val_accuracy: 0.7796666622161865 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 45s Search: Running Trial #33 Value |Best Value So Far |Hyperparameter 4 |1 |depth 128 |128 |width relu |swish |activation none |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.4 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 1024 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 53/53 [==============================] - 1s 9ms/step - loss: 0.8567 - accuracy: 0.7170 - val_loss: 0.2773 - val_accuracy: 0.9208 - lr: 0.0100 Epoch 2/2 53/53 [==============================] - 0s 6ms/step - loss: 0.3792 - accuracy: 0.8975 - val_loss: 0.2239 - val_accuracy: 0.9378 - lr: 0.0100 Trial 33 Complete [00h 00m 02s] val_accuracy: 0.937833309173584 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 46s Search: Running Trial #34 Value |Best Value So Far |Hyperparameter 3 |1 |depth 512 |128 |width relu |swish |activation l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 128 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 422/422 [==============================] - 2s 5ms/step - loss: 0.3851 - accuracy: 0.9037 - val_loss: 0.2096 - val_accuracy: 0.9528 - lr: 0.0100 Epoch 2/2 422/422 [==============================] - 2s 4ms/step - loss: 0.2689 - accuracy: 0.9415 - val_loss: 0.2383 - val_accuracy: 0.9475 - lr: 0.0100 Trial 34 Complete [00h 00m 05s] val_accuracy: 0.952833354473114 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 51s Search: Running Trial #35 Value |Best Value So Far |Hyperparameter 1 |1 |depth 128 |128 |width swish |swish |activation l1_l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.4 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 64 |64 |batch_size 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0022 |None |tuner/trial_id Epoch 3/4 844/844 [==============================] - 2s 2ms/step - loss: 0.4424 - accuracy: 0.8899 - val_loss: 0.2805 - val_accuracy: 0.9482 - lr: 0.0100 Epoch 4/4 844/844 [==============================] - 2s 2ms/step - loss: 0.4012 - accuracy: 0.9178 - val_loss: 0.2986 - val_accuracy: 0.9507 - lr: 0.0100 Trial 35 Complete [00h 00m 05s] val_accuracy: 0.9506666660308838 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 02m 56s Search: Running Trial #36 Value |Best Value So Far |Hyperparameter 3 |1 |depth 64 |128 |width tanh |swish |activation l1_l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout rmsprop |adam |optimizer 0.001 |0.01 |learning_rate 32 |64 |batch_size 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0008 |None |tuner/trial_id Epoch 3/4 1688/1688 [==============================] - 5s 3ms/step - loss: 0.3809 - accuracy: 0.8956 - val_loss: 0.2285 - val_accuracy: 0.9400 - lr: 0.0010 Epoch 4/4 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2311 - accuracy: 0.9396 - val_loss: 0.1874 - val_accuracy: 0.9542 - lr: 0.0010 Trial 36 Complete [00h 00m 11s] val_accuracy: 0.9541666507720947 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 03m 06s Search: Running Trial #37 Value |Best Value So Far |Hyperparameter 3 |1 |depth 256 |128 |width sigmoid |swish |activation l1 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout adam |adam |optimizer 0.001 |0.01 |learning_rate 32 |64 |batch_size 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0030 |None |tuner/trial_id Epoch 3/4 1688/1688 [==============================] - 6s 3ms/step - loss: 0.6311 - accuracy: 0.8509 - val_loss: 0.3694 - val_accuracy: 0.9360 - lr: 0.0010 Epoch 4/4 1688/1688 [==============================] - 5s 3ms/step - loss: 0.3591 - accuracy: 0.9399 - val_loss: 0.3102 - val_accuracy: 0.9528 - lr: 0.0010 Trial 37 Complete [00h 00m 11s] val_accuracy: 0.952833354473114 Best val_accuracy So Far: 0.9546666741371155 Total elapsed time: 00h 03m 18s Search: Running Trial #38 Value |Best Value So Far |Hyperparameter 3 |1 |depth 512 |128 |width relu |swish |activation l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 128 |64 |batch_size 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0033 |None |tuner/trial_id Epoch 3/4 422/422 [==============================] - 3s 5ms/step - loss: 0.4113 - accuracy: 0.8959 - val_loss: 0.2396 - val_accuracy: 0.9460 - lr: 0.0100 Epoch 4/4 422/422 [==============================] - 2s 5ms/step - loss: 0.2641 - accuracy: 0.9393 - val_loss: 0.2084 - val_accuracy: 0.9573 - lr: 0.0100 Trial 38 Complete [00h 00m 05s] val_accuracy: 0.9573333263397217 Best val_accuracy So Far: 0.9573333263397217 Total elapsed time: 00h 03m 23s Search: Running Trial #39 Value |Best Value So Far |Hyperparameter 1 |3 |depth 64 |512 |width swish |relu |activation l1_l2 |l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 64 |128 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0017 |0033 |tuner/trial_id Epoch 3/4 844/844 [==============================] - 3s 2ms/step - loss: 0.4436 - accuracy: 0.8803 - val_loss: 0.2780 - val_accuracy: 0.9387 - lr: 0.0100 Epoch 4/4 844/844 [==============================] - 2s 2ms/step - loss: 0.3827 - accuracy: 0.9077 - val_loss: 0.2586 - val_accuracy: 0.9490 - lr: 0.0100 Trial 39 Complete [00h 00m 05s] val_accuracy: 0.9490000009536743 Best val_accuracy So Far: 0.9573333263397217 Total elapsed time: 00h 03m 27s Search: Running Trial #40 Value |Best Value So Far |Hyperparameter 3 |3 |depth 128 |512 |width swish |relu |activation none |l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout adam |adam |optimizer 0.001 |0.01 |learning_rate 128 |128 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0018 |0033 |tuner/trial_id Epoch 3/4 422/422 [==============================] - 2s 3ms/step - loss: 0.5546 - accuracy: 0.8294 - val_loss: 0.2170 - val_accuracy: 0.9347 - lr: 0.0010 Epoch 4/4 422/422 [==============================] - 1s 3ms/step - loss: 0.2598 - accuracy: 0.9235 - val_loss: 0.1528 - val_accuracy: 0.9533 - lr: 0.0010 Trial 40 Complete [00h 00m 04s] val_accuracy: 0.95333331823349 Best val_accuracy So Far: 0.9573333263397217 Total elapsed time: 00h 03m 31s Search: Running Trial #41 Value |Best Value So Far |Hyperparameter 3 |3 |depth 256 |512 |width relu |relu |activation l2 |l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.1 |dropout sgd |adam |optimizer 0.001 |0.01 |learning_rate 1024 |128 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0015 |0033 |tuner/trial_id Epoch 3/4 53/53 [==============================] - 1s 9ms/step - loss: 0.8927 - accuracy: 0.7477 - val_loss: 0.3400 - val_accuracy: 0.9207 - lr: 0.0010 Epoch 4/4 53/53 [==============================] - 0s 6ms/step - loss: 0.3543 - accuracy: 0.9189 - val_loss: 0.2532 - val_accuracy: 0.9467 - lr: 0.0010 Trial 41 Complete [00h 00m 02s] val_accuracy: 0.9466666579246521 Best val_accuracy So Far: 0.9573333263397217 Total elapsed time: 00h 03m 33s Search: Running Trial #42 Value |Best Value So Far |Hyperparameter 1 |3 |depth 512 |512 |width relu |relu |activation l1 |l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.1 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 64 |128 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0029 |0033 |tuner/trial_id Epoch 3/4 844/844 [==============================] - 3s 3ms/step - loss: 1.0914 - accuracy: 0.8621 - val_loss: 0.8070 - val_accuracy: 0.9398 - lr: 0.0100 Epoch 4/4 844/844 [==============================] - 2s 2ms/step - loss: 0.9624 - accuracy: 0.8818 - val_loss: 0.7482 - val_accuracy: 0.9412 - lr: 0.0100 Trial 42 Complete [00h 00m 05s] val_accuracy: 0.9411666393280029 Best val_accuracy So Far: 0.9573333263397217 Total elapsed time: 00h 03m 38s Search: Running Trial #43 Value |Best Value So Far |Hyperparameter 2 |3 |depth 512 |512 |width tanh |relu |activation none |l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.1 |dropout rmsprop |adam |optimizer 0.001 |0.01 |learning_rate 512 |128 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0021 |0033 |tuner/trial_id Epoch 3/4 106/106 [==============================] - 1s 6ms/step - loss: 0.4002 - accuracy: 0.8808 - val_loss: 0.2642 - val_accuracy: 0.9252 - lr: 0.0010 Epoch 4/4 106/106 [==============================] - 1s 6ms/step - loss: 0.2235 - accuracy: 0.9341 - val_loss: 0.1945 - val_accuracy: 0.9433 - lr: 0.0010 Trial 43 Complete [00h 00m 02s] val_accuracy: 0.9433333277702332 Best val_accuracy So Far: 0.9573333263397217 Total elapsed time: 00h 03m 40s Search: Running Trial #44 Value |Best Value So Far |Hyperparameter 4 |3 |depth 128 |512 |width relu |relu |activation none |l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 1024 |128 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0032 |0033 |tuner/trial_id Epoch 3/4 53/53 [==============================] - 1s 8ms/step - loss: 0.8061 - accuracy: 0.7355 - val_loss: 0.2545 - val_accuracy: 0.9293 - lr: 0.0100 Epoch 4/4 53/53 [==============================] - 0s 5ms/step - loss: 0.3734 - accuracy: 0.8985 - val_loss: 0.2002 - val_accuracy: 0.9422 - lr: 0.0100 Trial 44 Complete [00h 00m 02s] val_accuracy: 0.9421666860580444 Best val_accuracy So Far: 0.9573333263397217 Total elapsed time: 00h 03m 41s Search: Running Trial #45 Value |Best Value So Far |Hyperparameter 4 |3 |depth 512 |512 |width swish |relu |activation none |l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.5 |0.1 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 256 |128 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0004 |0033 |tuner/trial_id Epoch 3/4 211/211 [==============================] - 2s 6ms/step - loss: 0.7935 - accuracy: 0.8028 - val_loss: 0.2305 - val_accuracy: 0.9377 - lr: 0.0100 Epoch 4/4 211/211 [==============================] - 1s 5ms/step - loss: 0.5706 - accuracy: 0.8601 - val_loss: 0.2794 - val_accuracy: 0.9333 - lr: 0.0100 Trial 45 Complete [00h 00m 03s] val_accuracy: 0.937666654586792 Best val_accuracy So Far: 0.9573333263397217 Total elapsed time: 00h 03m 45s Search: Running Trial #46 Value |Best Value So Far |Hyperparameter 1 |3 |depth 128 |512 |width tanh |relu |activation l1 |l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.1 |dropout rmsprop |adam |optimizer 0.01 |0.01 |learning_rate 256 |128 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0016 |0033 |tuner/trial_id Epoch 3/4 211/211 [==============================] - 1s 3ms/step - loss: 0.6962 - accuracy: 0.8710 - val_loss: 0.5211 - val_accuracy: 0.9215 - lr: 0.0100 Epoch 4/4 211/211 [==============================] - 1s 2ms/step - loss: 0.5989 - accuracy: 0.8986 - val_loss: 0.4918 - val_accuracy: 0.9338 - lr: 0.0100 Trial 46 Complete [00h 00m 02s] val_accuracy: 0.9338333606719971 Best val_accuracy So Far: 0.9573333263397217 Total elapsed time: 00h 03m 47s Search: Running Trial #47 Value |Best Value So Far |Hyperparameter 3 |3 |depth 512 |512 |width relu |relu |activation l2 |l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.1 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 128 |128 |batch_size 11 |4 |tuner/epochs 4 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |1 |tuner/round 0037 |0033 |tuner/trial_id Epoch 5/11 422/422 [==============================] - 2s 4ms/step - loss: 0.4489 - accuracy: 0.8847 - val_loss: 0.2148 - val_accuracy: 0.9513 - lr: 0.0100 Epoch 6/11 422/422 [==============================] - 2s 4ms/step - loss: 0.2636 - accuracy: 0.9370 - val_loss: 0.2213 - val_accuracy: 0.9508 - lr: 0.0100 Epoch 7/11 422/422 [==============================] - 2s 4ms/step - loss: 0.2482 - accuracy: 0.9452 - val_loss: 0.2077 - val_accuracy: 0.9593 - lr: 0.0100 Epoch 8/11 422/422 [==============================] - 2s 4ms/step - loss: 0.2476 - accuracy: 0.9468 - val_loss: 0.2119 - val_accuracy: 0.9613 - lr: 0.0100 Epoch 9/11 422/422 [==============================] - 2s 4ms/step - loss: 0.2421 - accuracy: 0.9506 - val_loss: 0.2121 - val_accuracy: 0.9600 - lr: 0.0100 Epoch 10/11 422/422 [==============================] - 2s 4ms/step - loss: 0.2434 - accuracy: 0.9526 - val_loss: 0.2156 - val_accuracy: 0.9657 - lr: 0.0100 Epoch 11/11 422/422 [==============================] - 2s 4ms/step - loss: 0.2398 - accuracy: 0.9533 - val_loss: 0.2166 - val_accuracy: 0.9638 - lr: 0.0100 Trial 47 Complete [00h 00m 13s] val_accuracy: 0.965666651725769 Best val_accuracy So Far: 0.965666651725769 Total elapsed time: 00h 03m 59s Search: Running Trial #48 Value |Best Value So Far |Hyperparameter 3 |3 |depth 64 |512 |width tanh |relu |activation l1_l2 |l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.1 |dropout rmsprop |adam |optimizer 0.001 |0.01 |learning_rate 32 |128 |batch_size 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |2 |tuner/round 0035 |0037 |tuner/trial_id Epoch 5/11 1688/1688 [==============================] - 6s 3ms/step - loss: 0.3867 - accuracy: 0.8955 - val_loss: 0.2313 - val_accuracy: 0.9408 - lr: 0.0010 Epoch 6/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2327 - accuracy: 0.9393 - val_loss: 0.2029 - val_accuracy: 0.9480 - lr: 0.0010 Epoch 7/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1964 - accuracy: 0.9512 - val_loss: 0.1736 - val_accuracy: 0.9582 - lr: 0.0010 Epoch 8/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1773 - accuracy: 0.9579 - val_loss: 0.1591 - val_accuracy: 0.9615 - lr: 0.0010 Epoch 9/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1619 - accuracy: 0.9619 - val_loss: 0.1552 - val_accuracy: 0.9657 - lr: 0.0010 Epoch 10/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1563 - accuracy: 0.9644 - val_loss: 0.1578 - val_accuracy: 0.9618 - lr: 0.0010 Epoch 11/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1500 - accuracy: 0.9671 - val_loss: 0.1438 - val_accuracy: 0.9693 - lr: 0.0010 Trial 48 Complete [00h 00m 35s] val_accuracy: 0.9693333506584167 Best val_accuracy So Far: 0.9693333506584167 Total elapsed time: 00h 04m 35s Search: Running Trial #49 Value |Best Value So Far |Hyperparameter 3 |3 |depth 128 |64 |width swish |tanh |activation none |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout adam |rmsprop |optimizer 0.001 |0.001 |learning_rate 128 |32 |batch_size 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |2 |tuner/round 0039 |0035 |tuner/trial_id Epoch 5/11 422/422 [==============================] - 2s 3ms/step - loss: 0.5448 - accuracy: 0.8332 - val_loss: 0.2153 - val_accuracy: 0.9342 - lr: 0.0010 Epoch 6/11 422/422 [==============================] - 1s 3ms/step - loss: 0.2628 - accuracy: 0.9226 - val_loss: 0.1613 - val_accuracy: 0.9497 - lr: 0.0010 Epoch 7/11 422/422 [==============================] - 1s 3ms/step - loss: 0.2073 - accuracy: 0.9381 - val_loss: 0.1304 - val_accuracy: 0.9605 - lr: 0.0010 Epoch 8/11 422/422 [==============================] - 1s 3ms/step - loss: 0.1736 - accuracy: 0.9480 - val_loss: 0.1184 - val_accuracy: 0.9633 - lr: 0.0010 Epoch 9/11 422/422 [==============================] - 1s 3ms/step - loss: 0.1561 - accuracy: 0.9530 - val_loss: 0.1040 - val_accuracy: 0.9677 - lr: 0.0010 Epoch 10/11 422/422 [==============================] - 1s 3ms/step - loss: 0.1419 - accuracy: 0.9568 - val_loss: 0.0990 - val_accuracy: 0.9693 - lr: 0.0010 Epoch 11/11 422/422 [==============================] - 1s 3ms/step - loss: 0.1284 - accuracy: 0.9607 - val_loss: 0.0927 - val_accuracy: 0.9712 - lr: 0.0010 Trial 49 Complete [00h 00m 10s] val_accuracy: 0.9711666703224182 Best val_accuracy So Far: 0.9711666703224182 Total elapsed time: 00h 04m 45s Search: Running Trial #50 Value |Best Value So Far |Hyperparameter 3 |3 |depth 256 |128 |width sigmoid |swish |activation l1 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout adam |adam |optimizer 0.001 |0.001 |learning_rate 32 |128 |batch_size 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |2 |tuner/round 0036 |0039 |tuner/trial_id Epoch 5/11 1688/1688 [==============================] - 6s 3ms/step - loss: 0.6263 - accuracy: 0.8531 - val_loss: 0.3726 - val_accuracy: 0.9335 - lr: 0.0010 Epoch 6/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.3587 - accuracy: 0.9393 - val_loss: 0.3167 - val_accuracy: 0.9493 - lr: 0.0010 Epoch 7/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.3078 - accuracy: 0.9524 - val_loss: 0.2838 - val_accuracy: 0.9597 - lr: 0.0010 Epoch 8/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2775 - accuracy: 0.9611 - val_loss: 0.2690 - val_accuracy: 0.9610 - lr: 0.0010 Epoch 9/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2563 - accuracy: 0.9665 - val_loss: 0.2445 - val_accuracy: 0.9672 - lr: 0.0010 Epoch 10/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2428 - accuracy: 0.9682 - val_loss: 0.2398 - val_accuracy: 0.9673 - lr: 0.0010 Epoch 11/11 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2303 - accuracy: 0.9709 - val_loss: 0.2163 - val_accuracy: 0.9733 - lr: 0.0010 Trial 50 Complete [00h 00m 38s] val_accuracy: 0.9733333587646484 Best val_accuracy So Far: 0.9733333587646484 Total elapsed time: 00h 05m 22s Search: Running Trial #51 Value |Best Value So Far |Hyperparameter 3 |3 |depth 256 |256 |width sigmoid |sigmoid |activation l1 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.1 |dropout adam |adam |optimizer 0.001 |0.001 |learning_rate 32 |32 |batch_size 32 |11 |tuner/epochs 11 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 3 |2 |tuner/round 0049 |0036 |tuner/trial_id Epoch 12/32 1688/1688 [==============================] - 6s 3ms/step - loss: 0.6320 - accuracy: 0.8500 - val_loss: 0.3747 - val_accuracy: 0.9365 - lr: 0.0010 Epoch 13/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.3608 - accuracy: 0.9383 - val_loss: 0.3185 - val_accuracy: 0.9517 - lr: 0.0010 Epoch 14/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.3062 - accuracy: 0.9539 - val_loss: 0.2720 - val_accuracy: 0.9633 - lr: 0.0010 Epoch 15/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2775 - accuracy: 0.9613 - val_loss: 0.2603 - val_accuracy: 0.9640 - lr: 0.0010 Epoch 16/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2580 - accuracy: 0.9656 - val_loss: 0.2525 - val_accuracy: 0.9652 - lr: 0.0010 Epoch 17/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2383 - accuracy: 0.9693 - val_loss: 0.2334 - val_accuracy: 0.9703 - lr: 0.0010 Epoch 18/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2300 - accuracy: 0.9709 - val_loss: 0.2237 - val_accuracy: 0.9735 - lr: 0.0010 Epoch 19/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2189 - accuracy: 0.9736 - val_loss: 0.2189 - val_accuracy: 0.9725 - lr: 0.0010 Epoch 20/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2118 - accuracy: 0.9745 - val_loss: 0.2289 - val_accuracy: 0.9702 - lr: 0.0010 Epoch 21/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1813 - accuracy: 0.9831 - val_loss: 0.1869 - val_accuracy: 0.9797 - lr: 1.0000e-04 Epoch 22/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1719 - accuracy: 0.9840 - val_loss: 0.1850 - val_accuracy: 0.9793 - lr: 1.0000e-04 Epoch 23/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1708 - accuracy: 0.9833 - val_loss: 0.1809 - val_accuracy: 0.9795 - lr: 1.0000e-04 Epoch 24/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1648 - accuracy: 0.9854 - val_loss: 0.1799 - val_accuracy: 0.9808 - lr: 1.0000e-05 Epoch 25/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1655 - accuracy: 0.9844 - val_loss: 0.1796 - val_accuracy: 0.9803 - lr: 1.0000e-05 Epoch 26/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1631 - accuracy: 0.9853 - val_loss: 0.1793 - val_accuracy: 0.9802 - lr: 1.0000e-05 Epoch 27/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1644 - accuracy: 0.9851 - val_loss: 0.1791 - val_accuracy: 0.9805 - lr: 1.0000e-06 Epoch 28/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1624 - accuracy: 0.9854 - val_loss: 0.1791 - val_accuracy: 0.9805 - lr: 1.0000e-06 Epoch 29/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1632 - accuracy: 0.9853 - val_loss: 0.1791 - val_accuracy: 0.9805 - lr: 1.0000e-07 Epoch 30/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1636 - accuracy: 0.9848 - val_loss: 0.1791 - val_accuracy: 0.9805 - lr: 1.0000e-07 Epoch 31/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1633 - accuracy: 0.9854 - val_loss: 0.1791 - val_accuracy: 0.9805 - lr: 1.0000e-08 Epoch 32/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1639 - accuracy: 0.9845 - val_loss: 0.1791 - val_accuracy: 0.9805 - lr: 1.0000e-08 Trial 51 Complete [00h 01m 49s] val_accuracy: 0.9808333516120911 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 07m 11s Search: Running Trial #52 Value |Best Value So Far |Hyperparameter 3 |3 |depth 128 |256 |width swish |sigmoid |activation none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout adam |adam |optimizer 0.001 |0.001 |learning_rate 128 |32 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 3 |3 |tuner/bracket 3 |3 |tuner/round 0048 |0049 |tuner/trial_id Epoch 12/32 422/422 [==============================] - 2s 3ms/step - loss: 0.5566 - accuracy: 0.8293 - val_loss: 0.2145 - val_accuracy: 0.9395 - lr: 0.0010 Epoch 13/32 422/422 [==============================] - 1s 3ms/step - loss: 0.2557 - accuracy: 0.9236 - val_loss: 0.1662 - val_accuracy: 0.9468 - lr: 0.0010 Epoch 14/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1997 - accuracy: 0.9405 - val_loss: 0.1324 - val_accuracy: 0.9597 - lr: 0.0010 Epoch 15/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1703 - accuracy: 0.9495 - val_loss: 0.1150 - val_accuracy: 0.9650 - lr: 0.0010 Epoch 16/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1512 - accuracy: 0.9552 - val_loss: 0.1113 - val_accuracy: 0.9660 - lr: 0.0010 Epoch 17/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1371 - accuracy: 0.9588 - val_loss: 0.0971 - val_accuracy: 0.9692 - lr: 0.0010 Epoch 18/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1256 - accuracy: 0.9621 - val_loss: 0.0939 - val_accuracy: 0.9707 - lr: 0.0010 Epoch 19/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1172 - accuracy: 0.9649 - val_loss: 0.0925 - val_accuracy: 0.9718 - lr: 0.0010 Epoch 20/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1110 - accuracy: 0.9665 - val_loss: 0.0948 - val_accuracy: 0.9712 - lr: 0.0010 Epoch 21/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1046 - accuracy: 0.9675 - val_loss: 0.0799 - val_accuracy: 0.9757 - lr: 0.0010 Epoch 22/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0975 - accuracy: 0.9705 - val_loss: 0.0791 - val_accuracy: 0.9750 - lr: 0.0010 Epoch 23/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0940 - accuracy: 0.9704 - val_loss: 0.0829 - val_accuracy: 0.9743 - lr: 0.0010 Epoch 24/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0776 - accuracy: 0.9755 - val_loss: 0.0759 - val_accuracy: 0.9768 - lr: 1.0000e-04 Epoch 25/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0709 - accuracy: 0.9779 - val_loss: 0.0756 - val_accuracy: 0.9768 - lr: 1.0000e-04 Epoch 26/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0723 - accuracy: 0.9774 - val_loss: 0.0741 - val_accuracy: 0.9772 - lr: 1.0000e-04 Epoch 27/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0684 - accuracy: 0.9787 - val_loss: 0.0744 - val_accuracy: 0.9775 - lr: 1.0000e-04 Epoch 28/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0689 - accuracy: 0.9789 - val_loss: 0.0749 - val_accuracy: 0.9778 - lr: 1.0000e-04 Epoch 29/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0679 - accuracy: 0.9788 - val_loss: 0.0748 - val_accuracy: 0.9782 - lr: 1.0000e-04 Epoch 30/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0690 - accuracy: 0.9784 - val_loss: 0.0733 - val_accuracy: 0.9778 - lr: 1.0000e-04 Epoch 31/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0669 - accuracy: 0.9792 - val_loss: 0.0734 - val_accuracy: 0.9778 - lr: 1.0000e-04 Epoch 32/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0646 - accuracy: 0.9797 - val_loss: 0.0731 - val_accuracy: 0.9780 - lr: 1.0000e-05 Trial 52 Complete [00h 00m 28s] val_accuracy: 0.9781666398048401 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 07m 39s Search: Running Trial #53 Value |Best Value So Far |Hyperparameter 2 |3 |depth 256 |256 |width relu |sigmoid |activation none |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.1 |dropout rmsprop |adam |optimizer 0.001 |0.001 |learning_rate 512 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 106/106 [==============================] - 1s 5ms/step - loss: 0.6336 - accuracy: 0.8033 - val_loss: 0.2275 - val_accuracy: 0.9317 - lr: 0.0010 Epoch 2/4 106/106 [==============================] - 1s 5ms/step - loss: 0.2520 - accuracy: 0.9250 - val_loss: 0.1605 - val_accuracy: 0.9508 - lr: 0.0010 Epoch 3/4 106/106 [==============================] - 0s 4ms/step - loss: 0.1897 - accuracy: 0.9434 - val_loss: 0.1319 - val_accuracy: 0.9595 - lr: 0.0010 Epoch 4/4 106/106 [==============================] - 0s 4ms/step - loss: 0.1524 - accuracy: 0.9542 - val_loss: 0.1133 - val_accuracy: 0.9637 - lr: 0.0010 Trial 53 Complete [00h 00m 03s] val_accuracy: 0.9636666774749756 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 07m 42s Search: Running Trial #54 Value |Best Value So Far |Hyperparameter 4 |3 |depth 256 |256 |width sigmoid |sigmoid |activation l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.1 |dropout rmsprop |adam |optimizer 0.01 |0.001 |learning_rate 32 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 1688/1688 [==============================] - 7s 4ms/step - loss: 1.0962 - accuracy: 0.8078 - val_loss: 0.8457 - val_accuracy: 0.9037 - lr: 0.0100 Epoch 2/4 1688/1688 [==============================] - 6s 3ms/step - loss: 0.9007 - accuracy: 0.8869 - val_loss: 0.7469 - val_accuracy: 0.9265 - lr: 0.0100 Epoch 3/4 1688/1688 [==============================] - 6s 3ms/step - loss: 0.8449 - accuracy: 0.8934 - val_loss: 0.7588 - val_accuracy: 0.9093 - lr: 0.0100 Epoch 4/4 1688/1688 [==============================] - 6s 3ms/step - loss: 0.8221 - accuracy: 0.8945 - val_loss: 0.6737 - val_accuracy: 0.9322 - lr: 0.0100 Trial 54 Complete [00h 00m 25s] val_accuracy: 0.9321666955947876 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 08m 07s Search: Running Trial #55 Value |Best Value So Far |Hyperparameter 3 |3 |depth 64 |256 |width swish |sigmoid |activation l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.1 |dropout rmsprop |adam |optimizer 0.0001 |0.001 |learning_rate 1024 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 53/53 [==============================] - 1s 9ms/step - loss: 2.5540 - accuracy: 0.2338 - val_loss: 2.4544 - val_accuracy: 0.5287 - lr: 1.0000e-04 Epoch 2/4 53/53 [==============================] - 0s 6ms/step - loss: 2.3179 - accuracy: 0.4105 - val_loss: 2.0841 - val_accuracy: 0.5817 - lr: 1.0000e-04 Epoch 3/4 53/53 [==============================] - 0s 5ms/step - loss: 1.9601 - accuracy: 0.4984 - val_loss: 1.6354 - val_accuracy: 0.6653 - lr: 1.0000e-04 Epoch 4/4 53/53 [==============================] - 0s 5ms/step - loss: 1.6533 - accuracy: 0.5676 - val_loss: 1.3157 - val_accuracy: 0.7323 - lr: 1.0000e-04 Trial 55 Complete [00h 00m 02s] val_accuracy: 0.7323333621025085 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 08m 09s Search: Running Trial #56 Value |Best Value So Far |Hyperparameter 2 |3 |depth 64 |256 |width tanh |sigmoid |activation none |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.1 |dropout sgd |adam |optimizer 0.01 |0.001 |learning_rate 512 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 106/106 [==============================] - 1s 5ms/step - loss: 0.5333 - accuracy: 0.8366 - val_loss: 0.2403 - val_accuracy: 0.9265 - lr: 0.0100 Epoch 2/4 106/106 [==============================] - 0s 4ms/step - loss: 0.3311 - accuracy: 0.9031 - val_loss: 0.2064 - val_accuracy: 0.9392 - lr: 0.0100 Epoch 3/4 106/106 [==============================] - 0s 3ms/step - loss: 0.2973 - accuracy: 0.9132 - val_loss: 0.1968 - val_accuracy: 0.9433 - lr: 0.0100 Epoch 4/4 106/106 [==============================] - 0s 3ms/step - loss: 0.2750 - accuracy: 0.9195 - val_loss: 0.1898 - val_accuracy: 0.9442 - lr: 0.0100 Trial 56 Complete [00h 00m 02s] val_accuracy: 0.9441666603088379 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 08m 12s Search: Running Trial #57 Value |Best Value So Far |Hyperparameter 4 |3 |depth 512 |256 |width relu |sigmoid |activation none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout sgd |adam |optimizer 0.0001 |0.001 |learning_rate 32 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 1688/1688 [==============================] - 7s 4ms/step - loss: 0.6878 - accuracy: 0.7771 - val_loss: 0.2107 - val_accuracy: 0.9337 - lr: 1.0000e-04 Epoch 2/4 1688/1688 [==============================] - 7s 4ms/step - loss: 0.2493 - accuracy: 0.9250 - val_loss: 0.1463 - val_accuracy: 0.9565 - lr: 1.0000e-04 Epoch 3/4 1688/1688 [==============================] - 7s 4ms/step - loss: 0.1836 - accuracy: 0.9458 - val_loss: 0.1186 - val_accuracy: 0.9635 - lr: 1.0000e-04 Epoch 4/4 1688/1688 [==============================] - 7s 4ms/step - loss: 0.1455 - accuracy: 0.9561 - val_loss: 0.1049 - val_accuracy: 0.9668 - lr: 1.0000e-04 Trial 57 Complete [00h 00m 28s] val_accuracy: 0.9668333530426025 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 08m 39s Search: Running Trial #58 Value |Best Value So Far |Hyperparameter 1 |3 |depth 64 |256 |width linear |sigmoid |activation none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.5 |0.1 |dropout adam |adam |optimizer 0.01 |0.001 |learning_rate 64 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 844/844 [==============================] - 2s 2ms/step - loss: 0.5836 - accuracy: 0.8343 - val_loss: 0.3853 - val_accuracy: 0.8933 - lr: 0.0100 Epoch 2/4 844/844 [==============================] - 2s 2ms/step - loss: 0.5479 - accuracy: 0.8529 - val_loss: 0.4349 - val_accuracy: 0.8870 - lr: 0.0100 Epoch 3/4 844/844 [==============================] - 2s 2ms/step - loss: 0.5585 - accuracy: 0.8519 - val_loss: 0.4082 - val_accuracy: 0.8958 - lr: 0.0100 Epoch 4/4 844/844 [==============================] - 2s 2ms/step - loss: 0.5590 - accuracy: 0.8539 - val_loss: 0.4137 - val_accuracy: 0.8913 - lr: 0.0100 Trial 58 Complete [00h 00m 08s] val_accuracy: 0.8958333134651184 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 08m 48s Search: Running Trial #59 Value |Best Value So Far |Hyperparameter 1 |3 |depth 64 |256 |width relu |sigmoid |activation l1_l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout sgd |adam |optimizer 0.0001 |0.001 |learning_rate 256 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 211/211 [==============================] - 1s 3ms/step - loss: 1.7676 - accuracy: 0.4468 - val_loss: 1.1194 - val_accuracy: 0.7903 - lr: 1.0000e-04 Epoch 2/4 211/211 [==============================] - 1s 3ms/step - loss: 1.0224 - accuracy: 0.7264 - val_loss: 0.7156 - val_accuracy: 0.8532 - lr: 1.0000e-04 Epoch 3/4 211/211 [==============================] - 1s 3ms/step - loss: 0.7806 - accuracy: 0.7896 - val_loss: 0.5630 - val_accuracy: 0.8745 - lr: 1.0000e-04 Epoch 4/4 211/211 [==============================] - 1s 3ms/step - loss: 0.6622 - accuracy: 0.8206 - val_loss: 0.4864 - val_accuracy: 0.8850 - lr: 1.0000e-04 Trial 59 Complete [00h 00m 03s] val_accuracy: 0.8849999904632568 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 08m 51s Search: Running Trial #60 Value |Best Value So Far |Hyperparameter 0 |3 |depth 256 |256 |width linear |sigmoid |activation l1 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.1 |dropout sgd |adam |optimizer 0.0001 |0.001 |learning_rate 128 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 422/422 [==============================] - 1s 2ms/step - loss: 1.6825 - accuracy: 0.5567 - val_loss: 1.2406 - val_accuracy: 0.7685 - lr: 1.0000e-04 Epoch 2/4 422/422 [==============================] - 1s 2ms/step - loss: 1.0258 - accuracy: 0.8024 - val_loss: 0.8614 - val_accuracy: 0.8242 - lr: 1.0000e-04 Epoch 3/4 422/422 [==============================] - 1s 2ms/step - loss: 0.7689 - accuracy: 0.8400 - val_loss: 0.6898 - val_accuracy: 0.8495 - lr: 1.0000e-04 Epoch 4/4 422/422 [==============================] - 1s 2ms/step - loss: 0.6387 - accuracy: 0.8574 - val_loss: 0.5926 - val_accuracy: 0.8627 - lr: 1.0000e-04 Trial 60 Complete [00h 00m 04s] val_accuracy: 0.862666666507721 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 08m 55s Search: Running Trial #61 Value |Best Value So Far |Hyperparameter 2 |3 |depth 64 |256 |width relu |sigmoid |activation l1_l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.1 |dropout sgd |adam |optimizer 0.0001 |0.001 |learning_rate 256 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 211/211 [==============================] - 1s 4ms/step - loss: 1.9401 - accuracy: 0.3662 - val_loss: 1.2764 - val_accuracy: 0.7352 - lr: 1.0000e-04 Epoch 2/4 211/211 [==============================] - 1s 3ms/step - loss: 1.1812 - accuracy: 0.6486 - val_loss: 0.7232 - val_accuracy: 0.8370 - lr: 1.0000e-04 Epoch 3/4 211/211 [==============================] - 1s 3ms/step - loss: 0.8601 - accuracy: 0.7444 - val_loss: 0.5334 - val_accuracy: 0.8707 - lr: 1.0000e-04 Epoch 4/4 211/211 [==============================] - 1s 3ms/step - loss: 0.7139 - accuracy: 0.7905 - val_loss: 0.4483 - val_accuracy: 0.8883 - lr: 1.0000e-04 Trial 61 Complete [00h 00m 04s] val_accuracy: 0.8883333206176758 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 08m 59s Search: Running Trial #62 Value |Best Value So Far |Hyperparameter 3 |3 |depth 64 |256 |width sigmoid |sigmoid |activation l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.1 |dropout rmsprop |adam |optimizer 0.0001 |0.001 |learning_rate 512 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 106/106 [==============================] - 1s 5ms/step - loss: 2.5144 - accuracy: 0.0998 - val_loss: 2.3559 - val_accuracy: 0.0937 - lr: 1.0000e-04 Epoch 2/4 106/106 [==============================] - 0s 4ms/step - loss: 2.3348 - accuracy: 0.1061 - val_loss: 2.2868 - val_accuracy: 0.0937 - lr: 1.0000e-04 Epoch 3/4 106/106 [==============================] - 0s 4ms/step - loss: 2.2917 - accuracy: 0.1329 - val_loss: 2.2537 - val_accuracy: 0.1852 - lr: 1.0000e-04 Epoch 4/4 106/106 [==============================] - 0s 4ms/step - loss: 2.2581 - accuracy: 0.1742 - val_loss: 2.2161 - val_accuracy: 0.3197 - lr: 1.0000e-04 Trial 62 Complete [00h 00m 03s] val_accuracy: 0.31966665387153625 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 09m 02s Search: Running Trial #63 Value |Best Value So Far |Hyperparameter 0 |3 |depth 64 |256 |width relu |sigmoid |activation l1 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.1 |dropout adam |adam |optimizer 0.01 |0.001 |learning_rate 1024 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 53/53 [==============================] - 1s 6ms/step - loss: 0.5969 - accuracy: 0.8275 - val_loss: 0.3519 - val_accuracy: 0.9020 - lr: 0.0100 Epoch 2/4 53/53 [==============================] - 0s 4ms/step - loss: 0.3231 - accuracy: 0.9094 - val_loss: 0.3172 - val_accuracy: 0.9122 - lr: 0.0100 Epoch 3/4 53/53 [==============================] - 0s 4ms/step - loss: 0.2954 - accuracy: 0.9174 - val_loss: 0.3026 - val_accuracy: 0.9157 - lr: 0.0100 Epoch 4/4 53/53 [==============================] - 0s 4ms/step - loss: 0.2794 - accuracy: 0.9222 - val_loss: 0.2954 - val_accuracy: 0.9178 - lr: 0.0100 Trial 63 Complete [00h 00m 02s] val_accuracy: 0.9178333282470703 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 09m 03s Search: Running Trial #64 Value |Best Value So Far |Hyperparameter 0 |3 |depth 64 |256 |width tanh |sigmoid |activation l1 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout sgd |adam |optimizer 0.001 |0.001 |learning_rate 1024 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 53/53 [==============================] - 1s 6ms/step - loss: 1.5251 - accuracy: 0.6065 - val_loss: 0.9796 - val_accuracy: 0.7960 - lr: 0.0010 Epoch 2/4 53/53 [==============================] - 0s 4ms/step - loss: 0.7996 - accuracy: 0.8259 - val_loss: 0.6672 - val_accuracy: 0.8488 - lr: 0.0010 Epoch 3/4 53/53 [==============================] - 0s 6ms/step - loss: 0.6031 - accuracy: 0.8608 - val_loss: 0.5466 - val_accuracy: 0.8687 - lr: 0.0010 Epoch 4/4 53/53 [==============================] - 0s 4ms/step - loss: 0.5125 - accuracy: 0.8758 - val_loss: 0.4820 - val_accuracy: 0.8812 - lr: 0.0010 Trial 64 Complete [00h 00m 02s] val_accuracy: 0.8811666369438171 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 09m 05s Search: Running Trial #65 Value |Best Value So Far |Hyperparameter 0 |3 |depth 256 |256 |width sigmoid |sigmoid |activation l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.1 |dropout adam |adam |optimizer 0.0001 |0.001 |learning_rate 32 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 1688/1688 [==============================] - 4s 2ms/step - loss: 1.1602 - accuracy: 0.7366 - val_loss: 0.6983 - val_accuracy: 0.8467 - lr: 1.0000e-04 Epoch 2/4 1688/1688 [==============================] - 3s 2ms/step - loss: 0.5791 - accuracy: 0.8657 - val_loss: 0.4975 - val_accuracy: 0.8793 - lr: 1.0000e-04 Epoch 3/4 1688/1688 [==============================] - 3s 2ms/step - loss: 0.4554 - accuracy: 0.8843 - val_loss: 0.4242 - val_accuracy: 0.8940 - lr: 1.0000e-04 Epoch 4/4 1688/1688 [==============================] - 3s 2ms/step - loss: 0.4013 - accuracy: 0.8949 - val_loss: 0.3865 - val_accuracy: 0.9003 - lr: 1.0000e-04 Trial 65 Complete [00h 00m 14s] val_accuracy: 0.9003333449363708 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 09m 19s Search: Running Trial #66 Value |Best Value So Far |Hyperparameter 0 |3 |depth 128 |256 |width linear |sigmoid |activation l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.5 |0.1 |dropout sgd |adam |optimizer 0.01 |0.001 |learning_rate 64 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 844/844 [==============================] - 2s 2ms/step - loss: 0.3638 - accuracy: 0.8956 - val_loss: 0.3485 - val_accuracy: 0.9082 - lr: 0.0100 Epoch 2/4 844/844 [==============================] - 2s 2ms/step - loss: 0.3035 - accuracy: 0.9156 - val_loss: 0.3166 - val_accuracy: 0.9175 - lr: 0.0100 Epoch 3/4 844/844 [==============================] - 2s 2ms/step - loss: 0.2966 - accuracy: 0.9177 - val_loss: 0.3390 - val_accuracy: 0.9133 - lr: 0.0100 Epoch 4/4 844/844 [==============================] - 2s 2ms/step - loss: 0.2916 - accuracy: 0.9206 - val_loss: 0.3603 - val_accuracy: 0.9037 - lr: 0.0100 Trial 66 Complete [00h 00m 08s] val_accuracy: 0.9175000190734863 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 09m 27s Search: Running Trial #67 Value |Best Value So Far |Hyperparameter 1 |3 |depth 64 |256 |width sigmoid |sigmoid |activation l1 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.1 |dropout rmsprop |adam |optimizer 0.0001 |0.001 |learning_rate 1024 |32 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 53/53 [==============================] - 1s 7ms/step - loss: 2.3471 - accuracy: 0.1807 - val_loss: 2.1902 - val_accuracy: 0.2598 - lr: 1.0000e-04 Epoch 2/4 53/53 [==============================] - 0s 4ms/step - loss: 2.1120 - accuracy: 0.3162 - val_loss: 1.9985 - val_accuracy: 0.5002 - lr: 1.0000e-04 Epoch 3/4 53/53 [==============================] - 0s 4ms/step - loss: 1.9425 - accuracy: 0.4776 - val_loss: 1.8454 - val_accuracy: 0.6155 - lr: 1.0000e-04 Epoch 4/4 53/53 [==============================] - 0s 4ms/step - loss: 1.8003 - accuracy: 0.5748 - val_loss: 1.7119 - val_accuracy: 0.6737 - lr: 1.0000e-04 Trial 67 Complete [00h 00m 02s] val_accuracy: 0.6736666560173035 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 09m 29s Search: Running Trial #68 Value |Best Value So Far |Hyperparameter 4 |3 |depth 512 |256 |width relu |sigmoid |activation none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout sgd |adam |optimizer 0.0001 |0.001 |learning_rate 32 |32 |batch_size 11 |32 |tuner/epochs 4 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 1 |3 |tuner/round 0056 |0049 |tuner/trial_id Epoch 5/11 1688/1688 [==============================] - 7s 4ms/step - loss: 0.6944 - accuracy: 0.7726 - val_loss: 0.2130 - val_accuracy: 0.9340 - lr: 1.0000e-04 Epoch 6/11 1688/1688 [==============================] - 7s 4ms/step - loss: 0.2541 - accuracy: 0.9235 - val_loss: 0.1505 - val_accuracy: 0.9535 - lr: 1.0000e-04 Epoch 7/11 1688/1688 [==============================] - 7s 4ms/step - loss: 0.1847 - accuracy: 0.9442 - val_loss: 0.1206 - val_accuracy: 0.9647 - lr: 1.0000e-04 Epoch 8/11 1688/1688 [==============================] - 7s 4ms/step - loss: 0.1499 - accuracy: 0.9548 - val_loss: 0.0988 - val_accuracy: 0.9698 - lr: 1.0000e-04 Epoch 9/11 1688/1688 [==============================] - 7s 4ms/step - loss: 0.1233 - accuracy: 0.9621 - val_loss: 0.0956 - val_accuracy: 0.9723 - lr: 1.0000e-04 Epoch 10/11 1688/1688 [==============================] - 7s 4ms/step - loss: 0.1096 - accuracy: 0.9665 - val_loss: 0.0945 - val_accuracy: 0.9723 - lr: 1.0000e-04 Epoch 11/11 1688/1688 [==============================] - 7s 4ms/step - loss: 0.0939 - accuracy: 0.9720 - val_loss: 0.0824 - val_accuracy: 0.9750 - lr: 1.0000e-04 Trial 68 Complete [00h 00m 47s] val_accuracy: 0.9750000238418579 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 10m 16s Search: Running Trial #69 Value |Best Value So Far |Hyperparameter 2 |3 |depth 256 |256 |width relu |sigmoid |activation none |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.1 |dropout rmsprop |adam |optimizer 0.001 |0.001 |learning_rate 512 |32 |batch_size 11 |32 |tuner/epochs 4 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 1 |3 |tuner/round 0052 |0049 |tuner/trial_id Epoch 5/11 106/106 [==============================] - 1s 5ms/step - loss: 0.6300 - accuracy: 0.8036 - val_loss: 0.2224 - val_accuracy: 0.9335 - lr: 0.0010 Epoch 6/11 106/106 [==============================] - 0s 4ms/step - loss: 0.2467 - accuracy: 0.9263 - val_loss: 0.1576 - val_accuracy: 0.9537 - lr: 0.0010 Epoch 7/11 106/106 [==============================] - 0s 4ms/step - loss: 0.1837 - accuracy: 0.9452 - val_loss: 0.1265 - val_accuracy: 0.9623 - lr: 0.0010 Epoch 8/11 106/106 [==============================] - 0s 4ms/step - loss: 0.1514 - accuracy: 0.9547 - val_loss: 0.1114 - val_accuracy: 0.9663 - lr: 0.0010 Epoch 9/11 106/106 [==============================] - 0s 4ms/step - loss: 0.1296 - accuracy: 0.9611 - val_loss: 0.0991 - val_accuracy: 0.9712 - lr: 0.0010 Epoch 10/11 106/106 [==============================] - 0s 4ms/step - loss: 0.1144 - accuracy: 0.9653 - val_loss: 0.0916 - val_accuracy: 0.9713 - lr: 0.0010 Epoch 11/11 106/106 [==============================] - 0s 4ms/step - loss: 0.1013 - accuracy: 0.9693 - val_loss: 0.0848 - val_accuracy: 0.9740 - lr: 0.0010 Trial 69 Complete [00h 00m 04s] val_accuracy: 0.9739999771118164 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 10m 20s Search: Running Trial #70 Value |Best Value So Far |Hyperparameter 2 |3 |depth 64 |256 |width tanh |sigmoid |activation none |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.1 |dropout sgd |adam |optimizer 0.01 |0.001 |learning_rate 512 |32 |batch_size 11 |32 |tuner/epochs 4 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 1 |3 |tuner/round 0055 |0049 |tuner/trial_id Epoch 5/11 106/106 [==============================] - 1s 5ms/step - loss: 0.5486 - accuracy: 0.8301 - val_loss: 0.2650 - val_accuracy: 0.9238 - lr: 0.0100 Epoch 6/11 106/106 [==============================] - 0s 3ms/step - loss: 0.3317 - accuracy: 0.9029 - val_loss: 0.2176 - val_accuracy: 0.9357 - lr: 0.0100 Epoch 7/11 106/106 [==============================] - 0s 3ms/step - loss: 0.2900 - accuracy: 0.9157 - val_loss: 0.2079 - val_accuracy: 0.9418 - lr: 0.0100 Epoch 8/11 106/106 [==============================] - 0s 3ms/step - loss: 0.2721 - accuracy: 0.9206 - val_loss: 0.1928 - val_accuracy: 0.9448 - lr: 0.0100 Epoch 9/11 106/106 [==============================] - 0s 3ms/step - loss: 0.2547 - accuracy: 0.9249 - val_loss: 0.1865 - val_accuracy: 0.9455 - lr: 0.0100 Epoch 10/11 106/106 [==============================] - 0s 3ms/step - loss: 0.2447 - accuracy: 0.9286 - val_loss: 0.1756 - val_accuracy: 0.9490 - lr: 0.0100 Epoch 11/11 106/106 [==============================] - 0s 3ms/step - loss: 0.2370 - accuracy: 0.9295 - val_loss: 0.1764 - val_accuracy: 0.9483 - lr: 0.0100 Trial 70 Complete [00h 00m 04s] val_accuracy: 0.9490000009536743 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 10m 24s Search: Running Trial #71 Value |Best Value So Far |Hyperparameter 4 |3 |depth 256 |256 |width sigmoid |sigmoid |activation l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.1 |dropout rmsprop |adam |optimizer 0.01 |0.001 |learning_rate 32 |32 |batch_size 11 |32 |tuner/epochs 4 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 1 |3 |tuner/round 0053 |0049 |tuner/trial_id Epoch 5/11 1688/1688 [==============================] - 7s 4ms/step - loss: 1.2232 - accuracy: 0.7466 - val_loss: 0.7340 - val_accuracy: 0.9348 - lr: 0.0100 Epoch 6/11 1688/1688 [==============================] - 6s 3ms/step - loss: 0.9098 - accuracy: 0.8810 - val_loss: 0.7804 - val_accuracy: 0.9120 - lr: 0.0100 Epoch 7/11 1688/1688 [==============================] - 6s 3ms/step - loss: 0.8489 - accuracy: 0.8905 - val_loss: 0.6859 - val_accuracy: 0.9375 - lr: 0.0100 Epoch 8/11 1688/1688 [==============================] - 6s 3ms/step - loss: 0.8137 - accuracy: 0.8935 - val_loss: 0.7352 - val_accuracy: 0.9113 - lr: 0.0100 Epoch 9/11 1688/1688 [==============================] - 6s 3ms/step - loss: 0.7918 - accuracy: 0.8967 - val_loss: 0.6015 - val_accuracy: 0.9410 - lr: 0.0100 Epoch 10/11 1688/1688 [==============================] - 6s 3ms/step - loss: 0.7767 - accuracy: 0.8963 - val_loss: 0.6363 - val_accuracy: 0.9292 - lr: 0.0100 Epoch 11/11 1688/1688 [==============================] - 6s 3ms/step - loss: 0.7712 - accuracy: 0.8947 - val_loss: 0.6244 - val_accuracy: 0.9382 - lr: 0.0100 Trial 71 Complete [00h 00m 41s] val_accuracy: 0.9409999847412109 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 11m 05s Search: Running Trial #72 Value |Best Value So Far |Hyperparameter 0 |3 |depth 64 |256 |width relu |sigmoid |activation l1 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.1 |dropout adam |adam |optimizer 0.01 |0.001 |learning_rate 1024 |32 |batch_size 11 |32 |tuner/epochs 4 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 1 |3 |tuner/round 0062 |0049 |tuner/trial_id Epoch 5/11 53/53 [==============================] - 1s 6ms/step - loss: 0.5790 - accuracy: 0.8357 - val_loss: 0.3509 - val_accuracy: 0.9007 - lr: 0.0100 Epoch 6/11 53/53 [==============================] - 0s 4ms/step - loss: 0.3198 - accuracy: 0.9102 - val_loss: 0.3168 - val_accuracy: 0.9127 - lr: 0.0100 Epoch 7/11 53/53 [==============================] - 0s 4ms/step - loss: 0.2940 - accuracy: 0.9173 - val_loss: 0.3015 - val_accuracy: 0.9165 - lr: 0.0100 Epoch 8/11 53/53 [==============================] - 0s 4ms/step - loss: 0.2794 - accuracy: 0.9223 - val_loss: 0.2977 - val_accuracy: 0.9177 - lr: 0.0100 Epoch 9/11 53/53 [==============================] - 0s 4ms/step - loss: 0.2710 - accuracy: 0.9245 - val_loss: 0.2957 - val_accuracy: 0.9198 - lr: 0.0100 Epoch 10/11 53/53 [==============================] - 0s 4ms/step - loss: 0.2654 - accuracy: 0.9255 - val_loss: 0.2931 - val_accuracy: 0.9210 - lr: 0.0100 Epoch 11/11 53/53 [==============================] - 0s 4ms/step - loss: 0.2589 - accuracy: 0.9282 - val_loss: 0.2999 - val_accuracy: 0.9185 - lr: 0.0100 Trial 72 Complete [00h 00m 02s] val_accuracy: 0.9210000038146973 Best val_accuracy So Far: 0.9808333516120911 Total elapsed time: 00h 11m 07s Search: Running Trial #73 Value |Best Value So Far |Hyperparameter 4 |3 |depth 512 |256 |width relu |sigmoid |activation none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout sgd |adam |optimizer 0.0001 |0.001 |learning_rate 32 |32 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 2 |3 |tuner/round 0067 |0049 |tuner/trial_id Epoch 12/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.6983 - accuracy: 0.7726 - val_loss: 0.2218 - val_accuracy: 0.9320 - lr: 1.0000e-04 Epoch 13/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.2555 - accuracy: 0.9236 - val_loss: 0.1491 - val_accuracy: 0.9528 - lr: 1.0000e-04 Epoch 14/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.1856 - accuracy: 0.9436 - val_loss: 0.1190 - val_accuracy: 0.9635 - lr: 1.0000e-04 Epoch 15/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.1495 - accuracy: 0.9550 - val_loss: 0.1072 - val_accuracy: 0.9668 - lr: 1.0000e-04 Epoch 16/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.1224 - accuracy: 0.9624 - val_loss: 0.0961 - val_accuracy: 0.9693 - lr: 1.0000e-04 Epoch 17/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.1060 - accuracy: 0.9680 - val_loss: 0.0845 - val_accuracy: 0.9743 - lr: 1.0000e-04 Epoch 18/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.0952 - accuracy: 0.9717 - val_loss: 0.0822 - val_accuracy: 0.9755 - lr: 1.0000e-04 Epoch 19/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.0816 - accuracy: 0.9752 - val_loss: 0.0822 - val_accuracy: 0.9758 - lr: 1.0000e-04 Epoch 20/32 1688/1688 [==============================] - 6s 4ms/step - loss: 0.0753 - accuracy: 0.9770 - val_loss: 0.0746 - val_accuracy: 0.9782 - lr: 1.0000e-04 Epoch 21/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.0647 - accuracy: 0.9802 - val_loss: 0.0755 - val_accuracy: 0.9788 - lr: 1.0000e-04 Epoch 22/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.0633 - accuracy: 0.9804 - val_loss: 0.0745 - val_accuracy: 0.9802 - lr: 1.0000e-04 Epoch 23/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.0588 - accuracy: 0.9813 - val_loss: 0.0732 - val_accuracy: 0.9803 - lr: 1.0000e-04 Epoch 24/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.0532 - accuracy: 0.9838 - val_loss: 0.0759 - val_accuracy: 0.9803 - lr: 1.0000e-04 Epoch 25/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.0485 - accuracy: 0.9842 - val_loss: 0.0796 - val_accuracy: 0.9808 - lr: 1.0000e-04 Epoch 26/32 1688/1688 [==============================] - 6s 4ms/step - loss: 0.0458 - accuracy: 0.9851 - val_loss: 0.0785 - val_accuracy: 0.9798 - lr: 1.0000e-04 Epoch 27/32 1688/1688 [==============================] - 6s 4ms/step - loss: 0.0418 - accuracy: 0.9869 - val_loss: 0.0915 - val_accuracy: 0.9782 - lr: 1.0000e-04 Epoch 28/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.0349 - accuracy: 0.9888 - val_loss: 0.0770 - val_accuracy: 0.9827 - lr: 1.0000e-05 Epoch 29/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.0310 - accuracy: 0.9901 - val_loss: 0.0752 - val_accuracy: 0.9830 - lr: 1.0000e-05 Epoch 30/32 1688/1688 [==============================] - 7s 4ms/step - loss: 0.0268 - accuracy: 0.9918 - val_loss: 0.0749 - val_accuracy: 0.9833 - lr: 1.0000e-05 Epoch 31/32 1688/1688 [==============================] - 6s 4ms/step - loss: 0.0263 - accuracy: 0.9917 - val_loss: 0.0763 - val_accuracy: 0.9832 - lr: 1.0000e-05 Epoch 32/32 1688/1688 [==============================] - 6s 4ms/step - loss: 0.0264 - accuracy: 0.9915 - val_loss: 0.0766 - val_accuracy: 0.9830 - lr: 1.0000e-05 Trial 73 Complete [00h 02m 19s] val_accuracy: 0.9833333492279053 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 13m 26s Search: Running Trial #74 Value |Best Value So Far |Hyperparameter 2 |4 |depth 256 |512 |width relu |relu |activation none |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.4 |dropout rmsprop |sgd |optimizer 0.001 |0.0001 |learning_rate 512 |32 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 2 |2 |tuner/bracket 2 |2 |tuner/round 0068 |0067 |tuner/trial_id Epoch 12/32 106/106 [==============================] - 1s 6ms/step - loss: 0.6322 - accuracy: 0.8063 - val_loss: 0.2258 - val_accuracy: 0.9322 - lr: 0.0010 Epoch 13/32 106/106 [==============================] - 0s 4ms/step - loss: 0.2483 - accuracy: 0.9259 - val_loss: 0.1575 - val_accuracy: 0.9532 - lr: 0.0010 Epoch 14/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1851 - accuracy: 0.9449 - val_loss: 0.1248 - val_accuracy: 0.9633 - lr: 0.0010 Epoch 15/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1509 - accuracy: 0.9548 - val_loss: 0.1077 - val_accuracy: 0.9672 - lr: 0.0010 Epoch 16/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1339 - accuracy: 0.9611 - val_loss: 0.0990 - val_accuracy: 0.9707 - lr: 0.0010 Epoch 17/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1145 - accuracy: 0.9649 - val_loss: 0.0904 - val_accuracy: 0.9735 - lr: 0.0010 Epoch 18/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1014 - accuracy: 0.9686 - val_loss: 0.0826 - val_accuracy: 0.9750 - lr: 0.0010 Epoch 19/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0922 - accuracy: 0.9717 - val_loss: 0.0819 - val_accuracy: 0.9745 - lr: 0.0010 Epoch 20/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0841 - accuracy: 0.9731 - val_loss: 0.0774 - val_accuracy: 0.9762 - lr: 0.0010 Epoch 21/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0759 - accuracy: 0.9761 - val_loss: 0.0750 - val_accuracy: 0.9758 - lr: 0.0010 Epoch 22/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0749 - accuracy: 0.9764 - val_loss: 0.0764 - val_accuracy: 0.9780 - lr: 0.0010 Epoch 23/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0681 - accuracy: 0.9787 - val_loss: 0.0726 - val_accuracy: 0.9785 - lr: 0.0010 Epoch 24/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0606 - accuracy: 0.9807 - val_loss: 0.0707 - val_accuracy: 0.9793 - lr: 0.0010 Epoch 25/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0567 - accuracy: 0.9814 - val_loss: 0.0720 - val_accuracy: 0.9795 - lr: 0.0010 Epoch 26/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0563 - accuracy: 0.9820 - val_loss: 0.0717 - val_accuracy: 0.9808 - lr: 0.0010 Epoch 27/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0532 - accuracy: 0.9832 - val_loss: 0.0713 - val_accuracy: 0.9805 - lr: 0.0010 Epoch 28/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0482 - accuracy: 0.9839 - val_loss: 0.0727 - val_accuracy: 0.9803 - lr: 0.0010 Epoch 29/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0395 - accuracy: 0.9872 - val_loss: 0.0698 - val_accuracy: 0.9812 - lr: 1.0000e-04 Epoch 30/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0387 - accuracy: 0.9875 - val_loss: 0.0682 - val_accuracy: 0.9813 - lr: 1.0000e-04 Epoch 31/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0369 - accuracy: 0.9882 - val_loss: 0.0687 - val_accuracy: 0.9812 - lr: 1.0000e-04 Epoch 32/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0350 - accuracy: 0.9889 - val_loss: 0.0677 - val_accuracy: 0.9820 - lr: 1.0000e-04 Trial 74 Complete [00h 00m 10s] val_accuracy: 0.9819999933242798 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 13m 36s Search: Running Trial #75 Value |Best Value So Far |Hyperparameter 0 |4 |depth 512 |512 |width swish |relu |activation l1 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.4 |dropout rmsprop |sgd |optimizer 0.0001 |0.0001 |learning_rate 128 |32 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 422/422 [==============================] - 1s 2ms/step - loss: 1.6544 - accuracy: 0.5691 - val_loss: 1.2286 - val_accuracy: 0.7607 - lr: 1.0000e-04 Epoch 2/11 422/422 [==============================] - 1s 2ms/step - loss: 1.0175 - accuracy: 0.8009 - val_loss: 0.8578 - val_accuracy: 0.8225 - lr: 1.0000e-04 Epoch 3/11 422/422 [==============================] - 1s 2ms/step - loss: 0.7647 - accuracy: 0.8386 - val_loss: 0.6878 - val_accuracy: 0.8472 - lr: 1.0000e-04 Epoch 4/11 422/422 [==============================] - 1s 2ms/step - loss: 0.6359 - accuracy: 0.8575 - val_loss: 0.5919 - val_accuracy: 0.8617 - lr: 1.0000e-04 Epoch 5/11 422/422 [==============================] - 1s 2ms/step - loss: 0.5580 - accuracy: 0.8692 - val_loss: 0.5306 - val_accuracy: 0.8720 - lr: 1.0000e-04 Epoch 6/11 422/422 [==============================] - 1s 2ms/step - loss: 0.5058 - accuracy: 0.8767 - val_loss: 0.4876 - val_accuracy: 0.8797 - lr: 1.0000e-04 Epoch 7/11 422/422 [==============================] - 1s 2ms/step - loss: 0.4682 - accuracy: 0.8834 - val_loss: 0.4562 - val_accuracy: 0.8860 - lr: 1.0000e-04 Epoch 8/11 422/422 [==============================] - 1s 2ms/step - loss: 0.4400 - accuracy: 0.8879 - val_loss: 0.4321 - val_accuracy: 0.8912 - lr: 1.0000e-04 Epoch 9/11 422/422 [==============================] - 1s 2ms/step - loss: 0.4179 - accuracy: 0.8913 - val_loss: 0.4133 - val_accuracy: 0.8947 - lr: 1.0000e-04 Epoch 10/11 422/422 [==============================] - 1s 2ms/step - loss: 0.4001 - accuracy: 0.8946 - val_loss: 0.3980 - val_accuracy: 0.8962 - lr: 1.0000e-04 Epoch 11/11 422/422 [==============================] - 1s 2ms/step - loss: 0.3856 - accuracy: 0.8974 - val_loss: 0.3856 - val_accuracy: 0.8990 - lr: 1.0000e-04 Trial 75 Complete [00h 00m 11s] val_accuracy: 0.8989999890327454 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 13m 47s Search: Running Trial #76 Value |Best Value So Far |Hyperparameter 1 |4 |depth 64 |512 |width sigmoid |relu |activation l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.2 |0.4 |dropout rmsprop |sgd |optimizer 0.0001 |0.0001 |learning_rate 32 |32 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 1688/1688 [==============================] - 4s 2ms/step - loss: 1.4337 - accuracy: 0.6316 - val_loss: 0.8941 - val_accuracy: 0.8385 - lr: 1.0000e-04 Epoch 2/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.7780 - accuracy: 0.8233 - val_loss: 0.5968 - val_accuracy: 0.8810 - lr: 1.0000e-04 Epoch 3/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.5863 - accuracy: 0.8579 - val_loss: 0.4786 - val_accuracy: 0.8968 - lr: 1.0000e-04 Epoch 4/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.4994 - accuracy: 0.8772 - val_loss: 0.4181 - val_accuracy: 0.9033 - lr: 1.0000e-04 Epoch 5/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.4523 - accuracy: 0.8856 - val_loss: 0.3839 - val_accuracy: 0.9082 - lr: 1.0000e-04 Epoch 6/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.4183 - accuracy: 0.8940 - val_loss: 0.3602 - val_accuracy: 0.9130 - lr: 1.0000e-04 Epoch 7/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3980 - accuracy: 0.8984 - val_loss: 0.3444 - val_accuracy: 0.9157 - lr: 1.0000e-04 Epoch 8/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3821 - accuracy: 0.9022 - val_loss: 0.3313 - val_accuracy: 0.9190 - lr: 1.0000e-04 Epoch 9/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3683 - accuracy: 0.9068 - val_loss: 0.3209 - val_accuracy: 0.9200 - lr: 1.0000e-04 Epoch 10/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3560 - accuracy: 0.9096 - val_loss: 0.3119 - val_accuracy: 0.9217 - lr: 1.0000e-04 Epoch 11/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3480 - accuracy: 0.9123 - val_loss: 0.3042 - val_accuracy: 0.9248 - lr: 1.0000e-04 Trial 76 Complete [00h 00m 42s] val_accuracy: 0.924833357334137 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 14m 29s Search: Running Trial #77 Value |Best Value So Far |Hyperparameter 2 |4 |depth 256 |512 |width linear |relu |activation l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.5 |0.4 |dropout rmsprop |sgd |optimizer 0.0001 |0.0001 |learning_rate 1024 |32 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 53/53 [==============================] - 1s 9ms/step - loss: 1.8807 - accuracy: 0.3620 - val_loss: 1.0023 - val_accuracy: 0.7767 - lr: 1.0000e-04 Epoch 2/11 53/53 [==============================] - 0s 6ms/step - loss: 1.0033 - accuracy: 0.6881 - val_loss: 0.6386 - val_accuracy: 0.8433 - lr: 1.0000e-04 Epoch 3/11 53/53 [==============================] - 0s 6ms/step - loss: 0.7408 - accuracy: 0.7705 - val_loss: 0.5071 - val_accuracy: 0.8667 - lr: 1.0000e-04 Epoch 4/11 53/53 [==============================] - 0s 6ms/step - loss: 0.6210 - accuracy: 0.8119 - val_loss: 0.4416 - val_accuracy: 0.8822 - lr: 1.0000e-04 Epoch 5/11 53/53 [==============================] - 0s 6ms/step - loss: 0.5554 - accuracy: 0.8321 - val_loss: 0.4046 - val_accuracy: 0.8908 - lr: 1.0000e-04 Epoch 6/11 53/53 [==============================] - 0s 6ms/step - loss: 0.5168 - accuracy: 0.8459 - val_loss: 0.3811 - val_accuracy: 0.8975 - lr: 1.0000e-04 Epoch 7/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4848 - accuracy: 0.8561 - val_loss: 0.3639 - val_accuracy: 0.9018 - lr: 1.0000e-04 Epoch 8/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4719 - accuracy: 0.8600 - val_loss: 0.3520 - val_accuracy: 0.9042 - lr: 1.0000e-04 Epoch 9/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4517 - accuracy: 0.8663 - val_loss: 0.3443 - val_accuracy: 0.9045 - lr: 1.0000e-04 Epoch 10/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4360 - accuracy: 0.8725 - val_loss: 0.3373 - val_accuracy: 0.9072 - lr: 1.0000e-04 Epoch 11/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4316 - accuracy: 0.8744 - val_loss: 0.3317 - val_accuracy: 0.9092 - lr: 1.0000e-04 Trial 77 Complete [00h 00m 04s] val_accuracy: 0.909166693687439 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 14m 33s Search: Running Trial #78 Value |Best Value So Far |Hyperparameter 2 |4 |depth 64 |512 |width linear |relu |activation l1_l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.4 |dropout adam |sgd |optimizer 0.0001 |0.0001 |learning_rate 64 |32 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 844/844 [==============================] - 3s 3ms/step - loss: 1.2358 - accuracy: 0.6081 - val_loss: 0.5487 - val_accuracy: 0.8638 - lr: 1.0000e-04 Epoch 2/11 844/844 [==============================] - 2s 3ms/step - loss: 0.6552 - accuracy: 0.8047 - val_loss: 0.4195 - val_accuracy: 0.8885 - lr: 1.0000e-04 Epoch 3/11 844/844 [==============================] - 2s 3ms/step - loss: 0.5589 - accuracy: 0.8372 - val_loss: 0.3822 - val_accuracy: 0.8967 - lr: 1.0000e-04 Epoch 4/11 844/844 [==============================] - 2s 3ms/step - loss: 0.5146 - accuracy: 0.8522 - val_loss: 0.3635 - val_accuracy: 0.9012 - lr: 1.0000e-04 Epoch 5/11 844/844 [==============================] - 2s 3ms/step - loss: 0.4959 - accuracy: 0.8603 - val_loss: 0.3563 - val_accuracy: 0.9033 - lr: 1.0000e-04 Epoch 6/11 844/844 [==============================] - 2s 3ms/step - loss: 0.4704 - accuracy: 0.8688 - val_loss: 0.3491 - val_accuracy: 0.9063 - lr: 1.0000e-04 Epoch 7/11 844/844 [==============================] - 2s 3ms/step - loss: 0.4584 - accuracy: 0.8722 - val_loss: 0.3411 - val_accuracy: 0.9077 - lr: 1.0000e-04 Epoch 8/11 844/844 [==============================] - 2s 3ms/step - loss: 0.4525 - accuracy: 0.8755 - val_loss: 0.3405 - val_accuracy: 0.9088 - lr: 1.0000e-04 Epoch 9/11 844/844 [==============================] - 2s 3ms/step - loss: 0.4425 - accuracy: 0.8771 - val_loss: 0.3363 - val_accuracy: 0.9092 - lr: 1.0000e-04 Epoch 10/11 844/844 [==============================] - 2s 3ms/step - loss: 0.4337 - accuracy: 0.8815 - val_loss: 0.3338 - val_accuracy: 0.9122 - lr: 1.0000e-04 Epoch 11/11 844/844 [==============================] - 2s 3ms/step - loss: 0.4291 - accuracy: 0.8832 - val_loss: 0.3328 - val_accuracy: 0.9127 - lr: 1.0000e-04 Trial 78 Complete [00h 00m 25s] val_accuracy: 0.9126666784286499 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 14m 58s Search: Running Trial #79 Value |Best Value So Far |Hyperparameter 1 |4 |depth 128 |512 |width linear |relu |activation l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.4 |dropout sgd |sgd |optimizer 0.001 |0.0001 |learning_rate 1024 |32 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 53/53 [==============================] - 1s 7ms/step - loss: 0.8258 - accuracy: 0.7554 - val_loss: 0.4067 - val_accuracy: 0.8875 - lr: 0.0010 Epoch 2/11 53/53 [==============================] - 0s 5ms/step - loss: 0.3907 - accuracy: 0.8891 - val_loss: 0.3388 - val_accuracy: 0.9053 - lr: 0.0010 Epoch 3/11 53/53 [==============================] - 0s 4ms/step - loss: 0.3429 - accuracy: 0.9012 - val_loss: 0.3175 - val_accuracy: 0.9117 - lr: 0.0010 Epoch 4/11 53/53 [==============================] - 0s 4ms/step - loss: 0.3219 - accuracy: 0.9080 - val_loss: 0.3082 - val_accuracy: 0.9150 - lr: 0.0010 Epoch 5/11 53/53 [==============================] - 0s 5ms/step - loss: 0.3099 - accuracy: 0.9129 - val_loss: 0.3010 - val_accuracy: 0.9173 - lr: 0.0010 Epoch 6/11 53/53 [==============================] - 0s 5ms/step - loss: 0.3034 - accuracy: 0.9145 - val_loss: 0.2983 - val_accuracy: 0.9202 - lr: 0.0010 Epoch 7/11 53/53 [==============================] - 0s 4ms/step - loss: 0.2971 - accuracy: 0.9168 - val_loss: 0.2976 - val_accuracy: 0.9173 - lr: 0.0010 Epoch 8/11 53/53 [==============================] - 0s 4ms/step - loss: 0.2919 - accuracy: 0.9184 - val_loss: 0.2945 - val_accuracy: 0.9195 - lr: 0.0010 Epoch 9/11 53/53 [==============================] - 0s 5ms/step - loss: 0.2837 - accuracy: 0.9200 - val_loss: 0.2929 - val_accuracy: 0.9200 - lr: 1.0000e-04 Epoch 10/11 53/53 [==============================] - 0s 4ms/step - loss: 0.2832 - accuracy: 0.9218 - val_loss: 0.2929 - val_accuracy: 0.9220 - lr: 1.0000e-04 Epoch 11/11 53/53 [==============================] - 0s 5ms/step - loss: 0.2826 - accuracy: 0.9214 - val_loss: 0.2921 - val_accuracy: 0.9207 - lr: 1.0000e-04 Trial 79 Complete [00h 00m 04s] val_accuracy: 0.921999990940094 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 15m 02s Search: Running Trial #80 Value |Best Value So Far |Hyperparameter 3 |4 |depth 512 |512 |width linear |relu |activation l1 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.2 |0.4 |dropout rmsprop |sgd |optimizer 0.01 |0.0001 |learning_rate 256 |32 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 211/211 [==============================] - 2s 5ms/step - loss: 9.5651 - accuracy: 0.8112 - val_loss: 2.9829 - val_accuracy: 0.9038 - lr: 0.0100 Epoch 2/11 211/211 [==============================] - 1s 4ms/step - loss: 2.5454 - accuracy: 0.8970 - val_loss: 2.1123 - val_accuracy: 0.9062 - lr: 0.0100 Epoch 3/11 211/211 [==============================] - 1s 4ms/step - loss: 1.7799 - accuracy: 0.9039 - val_loss: 1.4701 - val_accuracy: 0.9050 - lr: 0.0100 Epoch 4/11 211/211 [==============================] - 1s 4ms/step - loss: 1.2425 - accuracy: 0.9027 - val_loss: 1.0415 - val_accuracy: 0.9085 - lr: 0.0100 Epoch 5/11 211/211 [==============================] - 1s 4ms/step - loss: 0.9297 - accuracy: 0.9003 - val_loss: 0.8390 - val_accuracy: 0.8927 - lr: 0.0100 Epoch 6/11 211/211 [==============================] - 1s 4ms/step - loss: 0.7741 - accuracy: 0.8947 - val_loss: 0.6983 - val_accuracy: 0.9007 - lr: 0.0100 Epoch 7/11 211/211 [==============================] - 1s 5ms/step - loss: 0.6190 - accuracy: 0.9126 - val_loss: 0.5870 - val_accuracy: 0.9163 - lr: 1.0000e-03 Epoch 8/11 211/211 [==============================] - 1s 4ms/step - loss: 0.5856 - accuracy: 0.9143 - val_loss: 0.5683 - val_accuracy: 0.9180 - lr: 1.0000e-03 Epoch 9/11 211/211 [==============================] - 1s 4ms/step - loss: 0.5720 - accuracy: 0.9135 - val_loss: 0.5573 - val_accuracy: 0.9170 - lr: 1.0000e-03 Epoch 10/11 211/211 [==============================] - 1s 4ms/step - loss: 0.5601 - accuracy: 0.9140 - val_loss: 0.5451 - val_accuracy: 0.9128 - lr: 1.0000e-03 Epoch 11/11 211/211 [==============================] - 1s 4ms/step - loss: 0.5389 - accuracy: 0.9160 - val_loss: 0.5288 - val_accuracy: 0.9197 - lr: 1.0000e-04 Trial 80 Complete [00h 00m 11s] val_accuracy: 0.9196666479110718 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 15m 13s Search: Running Trial #81 Value |Best Value So Far |Hyperparameter 4 |4 |depth 512 |512 |width linear |relu |activation l1 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.2 |0.4 |dropout sgd |sgd |optimizer 0.01 |0.0001 |learning_rate 1024 |32 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 53/53 [==============================] - 1s 13ms/step - loss: 134.8654 - accuracy: 0.5381 - val_loss: 13.3427 - val_accuracy: 0.8350 - lr: 0.0100 Epoch 2/11 53/53 [==============================] - 1s 10ms/step - loss: 8.7736 - accuracy: 0.8398 - val_loss: 5.7989 - val_accuracy: 0.8952 - lr: 0.0100 Epoch 3/11 53/53 [==============================] - 1s 10ms/step - loss: 5.6389 - accuracy: 0.8567 - val_loss: 4.9619 - val_accuracy: 0.9032 - lr: 0.0100 Epoch 4/11 53/53 [==============================] - 1s 10ms/step - loss: 4.9949 - accuracy: 0.8610 - val_loss: 4.6277 - val_accuracy: 0.9053 - lr: 0.0100 Epoch 5/11 53/53 [==============================] - 1s 10ms/step - loss: 4.6628 - accuracy: 0.8689 - val_loss: 4.4144 - val_accuracy: 0.9078 - lr: 0.0100 Epoch 6/11 53/53 [==============================] - 1s 10ms/step - loss: 4.4491 - accuracy: 0.8783 - val_loss: 4.2626 - val_accuracy: 0.9100 - lr: 0.0100 Epoch 7/11 53/53 [==============================] - 0s 9ms/step - loss: 4.2829 - accuracy: 0.8819 - val_loss: 4.1377 - val_accuracy: 0.9093 - lr: 0.0100 Epoch 8/11 53/53 [==============================] - 1s 10ms/step - loss: 4.1461 - accuracy: 0.8885 - val_loss: 4.0145 - val_accuracy: 0.9130 - lr: 0.0100 Epoch 9/11 53/53 [==============================] - 0s 9ms/step - loss: 4.0194 - accuracy: 0.8935 - val_loss: 3.9046 - val_accuracy: 0.9125 - lr: 0.0100 Epoch 10/11 53/53 [==============================] - 0s 9ms/step - loss: 3.8993 - accuracy: 0.8967 - val_loss: 3.8020 - val_accuracy: 0.9130 - lr: 0.0100 Epoch 11/11 53/53 [==============================] - 1s 10ms/step - loss: 3.8196 - accuracy: 0.9011 - val_loss: 3.7707 - val_accuracy: 0.9158 - lr: 1.0000e-03 Trial 81 Complete [00h 00m 07s] val_accuracy: 0.9158333539962769 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 15m 20s Search: Running Trial #82 Value |Best Value So Far |Hyperparameter 3 |4 |depth 128 |512 |width tanh |relu |activation l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.4 |dropout rmsprop |sgd |optimizer 0.001 |0.0001 |learning_rate 1024 |32 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 53/53 [==============================] - 1s 9ms/step - loss: 0.7267 - accuracy: 0.8067 - val_loss: 0.3612 - val_accuracy: 0.9075 - lr: 0.0010 Epoch 2/11 53/53 [==============================] - 0s 5ms/step - loss: 0.3421 - accuracy: 0.9112 - val_loss: 0.2998 - val_accuracy: 0.9268 - lr: 0.0010 Epoch 3/11 53/53 [==============================] - 0s 5ms/step - loss: 0.2948 - accuracy: 0.9264 - val_loss: 0.2696 - val_accuracy: 0.9327 - lr: 0.0010 Epoch 4/11 53/53 [==============================] - 0s 5ms/step - loss: 0.2613 - accuracy: 0.9356 - val_loss: 0.2493 - val_accuracy: 0.9405 - lr: 0.0010 Epoch 5/11 53/53 [==============================] - 0s 5ms/step - loss: 0.2355 - accuracy: 0.9441 - val_loss: 0.2265 - val_accuracy: 0.9453 - lr: 0.0010 Epoch 6/11 53/53 [==============================] - 0s 5ms/step - loss: 0.2175 - accuracy: 0.9489 - val_loss: 0.2124 - val_accuracy: 0.9490 - lr: 0.0010 Epoch 7/11 53/53 [==============================] - 0s 5ms/step - loss: 0.2013 - accuracy: 0.9532 - val_loss: 0.2001 - val_accuracy: 0.9537 - lr: 0.0010 Epoch 8/11 53/53 [==============================] - 0s 5ms/step - loss: 0.1880 - accuracy: 0.9575 - val_loss: 0.1837 - val_accuracy: 0.9585 - lr: 0.0010 Epoch 9/11 53/53 [==============================] - 0s 5ms/step - loss: 0.1791 - accuracy: 0.9594 - val_loss: 0.1815 - val_accuracy: 0.9595 - lr: 0.0010 Epoch 10/11 53/53 [==============================] - 0s 5ms/step - loss: 0.1698 - accuracy: 0.9621 - val_loss: 0.1731 - val_accuracy: 0.9612 - lr: 0.0010 Epoch 11/11 53/53 [==============================] - 0s 5ms/step - loss: 0.1608 - accuracy: 0.9648 - val_loss: 0.1638 - val_accuracy: 0.9647 - lr: 0.0010 Trial 82 Complete [00h 00m 04s] val_accuracy: 0.9646666646003723 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 15m 24s Search: Running Trial #83 Value |Best Value So Far |Hyperparameter 3 |4 |depth 128 |512 |width tanh |relu |activation l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.4 |dropout rmsprop |sgd |optimizer 0.001 |0.0001 |learning_rate 1024 |32 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 1 |2 |tuner/round 0081 |0067 |tuner/trial_id Epoch 12/32 53/53 [==============================] - 1s 8ms/step - loss: 0.7728 - accuracy: 0.7895 - val_loss: 0.3654 - val_accuracy: 0.9090 - lr: 0.0010 Epoch 13/32 53/53 [==============================] - 0s 5ms/step - loss: 0.3470 - accuracy: 0.9100 - val_loss: 0.3129 - val_accuracy: 0.9225 - lr: 0.0010 Epoch 14/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2992 - accuracy: 0.9250 - val_loss: 0.2743 - val_accuracy: 0.9322 - lr: 0.0010 Epoch 15/32 53/53 [==============================] - 0s 6ms/step - loss: 0.2665 - accuracy: 0.9347 - val_loss: 0.2494 - val_accuracy: 0.9402 - lr: 0.0010 Epoch 16/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2403 - accuracy: 0.9425 - val_loss: 0.2302 - val_accuracy: 0.9455 - lr: 0.0010 Epoch 17/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2216 - accuracy: 0.9469 - val_loss: 0.2153 - val_accuracy: 0.9490 - lr: 0.0010 Epoch 18/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2068 - accuracy: 0.9513 - val_loss: 0.2005 - val_accuracy: 0.9542 - lr: 0.0010 Epoch 19/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1906 - accuracy: 0.9551 - val_loss: 0.1904 - val_accuracy: 0.9543 - lr: 0.0010 Epoch 20/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1830 - accuracy: 0.9581 - val_loss: 0.1848 - val_accuracy: 0.9592 - lr: 0.0010 Epoch 21/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1721 - accuracy: 0.9614 - val_loss: 0.1723 - val_accuracy: 0.9620 - lr: 0.0010 Epoch 22/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1637 - accuracy: 0.9639 - val_loss: 0.1722 - val_accuracy: 0.9607 - lr: 0.0010 Epoch 23/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1558 - accuracy: 0.9659 - val_loss: 0.1646 - val_accuracy: 0.9663 - lr: 0.0010 Epoch 24/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1525 - accuracy: 0.9671 - val_loss: 0.1598 - val_accuracy: 0.9657 - lr: 0.0010 Epoch 25/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1464 - accuracy: 0.9696 - val_loss: 0.1573 - val_accuracy: 0.9672 - lr: 0.0010 Epoch 26/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1427 - accuracy: 0.9701 - val_loss: 0.1552 - val_accuracy: 0.9670 - lr: 0.0010 Epoch 27/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1373 - accuracy: 0.9716 - val_loss: 0.1490 - val_accuracy: 0.9700 - lr: 0.0010 Epoch 28/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1315 - accuracy: 0.9734 - val_loss: 0.1488 - val_accuracy: 0.9692 - lr: 0.0010 Epoch 29/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1285 - accuracy: 0.9743 - val_loss: 0.1446 - val_accuracy: 0.9693 - lr: 0.0010 Epoch 30/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1198 - accuracy: 0.9765 - val_loss: 0.1413 - val_accuracy: 0.9718 - lr: 1.0000e-04 Epoch 31/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1172 - accuracy: 0.9780 - val_loss: 0.1398 - val_accuracy: 0.9718 - lr: 1.0000e-04 Epoch 32/32 53/53 [==============================] - 0s 5ms/step - loss: 0.1171 - accuracy: 0.9778 - val_loss: 0.1403 - val_accuracy: 0.9720 - lr: 1.0000e-04 Trial 83 Complete [00h 00m 07s] val_accuracy: 0.972000002861023 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 15m 31s Search: Running Trial #84 Value |Best Value So Far |Hyperparameter 1 |4 |depth 64 |512 |width sigmoid |relu |activation l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.2 |0.4 |dropout rmsprop |sgd |optimizer 0.0001 |0.0001 |learning_rate 32 |32 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 1 |2 |tuner/round 0075 |0067 |tuner/trial_id Epoch 12/32 1688/1688 [==============================] - 4s 2ms/step - loss: 1.4311 - accuracy: 0.6306 - val_loss: 0.8962 - val_accuracy: 0.8363 - lr: 1.0000e-04 Epoch 13/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.7825 - accuracy: 0.8232 - val_loss: 0.5935 - val_accuracy: 0.8772 - lr: 1.0000e-04 Epoch 14/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.5896 - accuracy: 0.8592 - val_loss: 0.4749 - val_accuracy: 0.8958 - lr: 1.0000e-04 Epoch 15/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.4989 - accuracy: 0.8763 - val_loss: 0.4156 - val_accuracy: 0.9023 - lr: 1.0000e-04 Epoch 16/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.4498 - accuracy: 0.8859 - val_loss: 0.3813 - val_accuracy: 0.9085 - lr: 1.0000e-04 Epoch 17/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.4207 - accuracy: 0.8931 - val_loss: 0.3596 - val_accuracy: 0.9100 - lr: 1.0000e-04 Epoch 18/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3974 - accuracy: 0.8977 - val_loss: 0.3431 - val_accuracy: 0.9142 - lr: 1.0000e-04 Epoch 19/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3823 - accuracy: 0.9027 - val_loss: 0.3313 - val_accuracy: 0.9173 - lr: 1.0000e-04 Epoch 20/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3706 - accuracy: 0.9049 - val_loss: 0.3212 - val_accuracy: 0.9185 - lr: 1.0000e-04 Epoch 21/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3595 - accuracy: 0.9081 - val_loss: 0.3125 - val_accuracy: 0.9208 - lr: 1.0000e-04 Epoch 22/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3508 - accuracy: 0.9103 - val_loss: 0.3053 - val_accuracy: 0.9238 - lr: 1.0000e-04 Epoch 23/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3416 - accuracy: 0.9136 - val_loss: 0.2989 - val_accuracy: 0.9243 - lr: 1.0000e-04 Epoch 24/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3332 - accuracy: 0.9156 - val_loss: 0.2927 - val_accuracy: 0.9275 - lr: 1.0000e-04 Epoch 25/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3265 - accuracy: 0.9177 - val_loss: 0.2876 - val_accuracy: 0.9283 - lr: 1.0000e-04 Epoch 26/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3216 - accuracy: 0.9199 - val_loss: 0.2829 - val_accuracy: 0.9293 - lr: 1.0000e-04 Epoch 27/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3146 - accuracy: 0.9214 - val_loss: 0.2771 - val_accuracy: 0.9323 - lr: 1.0000e-04 Epoch 28/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3102 - accuracy: 0.9230 - val_loss: 0.2731 - val_accuracy: 0.9327 - lr: 1.0000e-04 Epoch 29/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3057 - accuracy: 0.9240 - val_loss: 0.2688 - val_accuracy: 0.9348 - lr: 1.0000e-04 Epoch 30/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3020 - accuracy: 0.9252 - val_loss: 0.2655 - val_accuracy: 0.9365 - lr: 1.0000e-04 Epoch 31/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.2958 - accuracy: 0.9268 - val_loss: 0.2622 - val_accuracy: 0.9383 - lr: 1.0000e-04 Epoch 32/32 1688/1688 [==============================] - 4s 2ms/step - loss: 0.2924 - accuracy: 0.9273 - val_loss: 0.2583 - val_accuracy: 0.9387 - lr: 1.0000e-04 Trial 84 Complete [00h 01m 17s] val_accuracy: 0.9386666417121887 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 16m 48s Search: Running Trial #85 Value |Best Value So Far |Hyperparameter 1 |4 |depth 128 |512 |width linear |relu |activation l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.4 |dropout sgd |sgd |optimizer 0.001 |0.0001 |learning_rate 1024 |32 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 1 |2 |tuner/round 0078 |0067 |tuner/trial_id Epoch 12/32 53/53 [==============================] - 1s 7ms/step - loss: 0.8010 - accuracy: 0.7661 - val_loss: 0.4020 - val_accuracy: 0.8910 - lr: 0.0010 Epoch 13/32 53/53 [==============================] - 0s 4ms/step - loss: 0.3887 - accuracy: 0.8876 - val_loss: 0.3380 - val_accuracy: 0.9087 - lr: 0.0010 Epoch 14/32 53/53 [==============================] - 0s 4ms/step - loss: 0.3444 - accuracy: 0.9024 - val_loss: 0.3159 - val_accuracy: 0.9125 - lr: 0.0010 Epoch 15/32 53/53 [==============================] - 0s 4ms/step - loss: 0.3244 - accuracy: 0.9088 - val_loss: 0.3086 - val_accuracy: 0.9135 - lr: 0.0010 Epoch 16/32 53/53 [==============================] - 0s 4ms/step - loss: 0.3105 - accuracy: 0.9125 - val_loss: 0.3003 - val_accuracy: 0.9180 - lr: 0.0010 Epoch 17/32 53/53 [==============================] - 0s 4ms/step - loss: 0.3047 - accuracy: 0.9142 - val_loss: 0.2973 - val_accuracy: 0.9173 - lr: 0.0010 Epoch 18/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2979 - accuracy: 0.9163 - val_loss: 0.2960 - val_accuracy: 0.9188 - lr: 0.0010 Epoch 19/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2937 - accuracy: 0.9182 - val_loss: 0.2956 - val_accuracy: 0.9202 - lr: 0.0010 Epoch 20/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2905 - accuracy: 0.9181 - val_loss: 0.2937 - val_accuracy: 0.9213 - lr: 0.0010 Epoch 21/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2855 - accuracy: 0.9221 - val_loss: 0.2939 - val_accuracy: 0.9220 - lr: 0.0010 Epoch 22/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2831 - accuracy: 0.9216 - val_loss: 0.2953 - val_accuracy: 0.9217 - lr: 0.0010 Epoch 23/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2825 - accuracy: 0.9221 - val_loss: 0.2929 - val_accuracy: 0.9223 - lr: 0.0010 Epoch 24/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2795 - accuracy: 0.9220 - val_loss: 0.2930 - val_accuracy: 0.9230 - lr: 0.0010 Epoch 25/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2750 - accuracy: 0.9244 - val_loss: 0.2932 - val_accuracy: 0.9203 - lr: 0.0010 Epoch 26/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2742 - accuracy: 0.9254 - val_loss: 0.2931 - val_accuracy: 0.9215 - lr: 0.0010 Epoch 27/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2674 - accuracy: 0.9266 - val_loss: 0.2897 - val_accuracy: 0.9223 - lr: 1.0000e-04 Epoch 28/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2665 - accuracy: 0.9265 - val_loss: 0.2895 - val_accuracy: 0.9212 - lr: 1.0000e-04 Epoch 29/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2652 - accuracy: 0.9274 - val_loss: 0.2893 - val_accuracy: 0.9217 - lr: 1.0000e-05 Epoch 30/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2650 - accuracy: 0.9273 - val_loss: 0.2891 - val_accuracy: 0.9215 - lr: 1.0000e-05 Epoch 31/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2650 - accuracy: 0.9278 - val_loss: 0.2891 - val_accuracy: 0.9217 - lr: 1.0000e-06 Epoch 32/32 53/53 [==============================] - 0s 4ms/step - loss: 0.2655 - accuracy: 0.9271 - val_loss: 0.2891 - val_accuracy: 0.9215 - lr: 1.0000e-06 Trial 85 Complete [00h 00m 06s] val_accuracy: 0.9229999780654907 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 16m 54s Search: Running Trial #86 Value |Best Value So Far |Hyperparameter 2 |4 |depth 128 |512 |width swish |relu |activation none |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.4 |dropout sgd |sgd |optimizer 0.01 |0.0001 |learning_rate 128 |32 |batch_size 32 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/32 422/422 [==============================] - 2s 3ms/step - loss: 0.2576 - accuracy: 0.9203 - val_loss: 0.1578 - val_accuracy: 0.9528 - lr: 0.0100 Epoch 2/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1729 - accuracy: 0.9499 - val_loss: 0.1616 - val_accuracy: 0.9512 - lr: 0.0100 Epoch 3/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1486 - accuracy: 0.9573 - val_loss: 0.1459 - val_accuracy: 0.9638 - lr: 0.0100 Epoch 4/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1388 - accuracy: 0.9611 - val_loss: 0.1682 - val_accuracy: 0.9598 - lr: 0.0100 Epoch 5/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1264 - accuracy: 0.9655 - val_loss: 0.1343 - val_accuracy: 0.9657 - lr: 0.0100 Epoch 6/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1217 - accuracy: 0.9660 - val_loss: 0.1503 - val_accuracy: 0.9615 - lr: 0.0100 Epoch 7/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1244 - accuracy: 0.9672 - val_loss: 0.1256 - val_accuracy: 0.9670 - lr: 0.0100 Epoch 8/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1155 - accuracy: 0.9694 - val_loss: 0.1502 - val_accuracy: 0.9632 - lr: 0.0100 Epoch 9/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1161 - accuracy: 0.9692 - val_loss: 0.1275 - val_accuracy: 0.9662 - lr: 0.0100 Epoch 10/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0620 - accuracy: 0.9818 - val_loss: 0.1063 - val_accuracy: 0.9762 - lr: 1.0000e-03 Epoch 11/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0474 - accuracy: 0.9860 - val_loss: 0.1043 - val_accuracy: 0.9777 - lr: 1.0000e-03 Epoch 12/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0399 - accuracy: 0.9876 - val_loss: 0.1071 - val_accuracy: 0.9767 - lr: 1.0000e-03 Epoch 13/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0357 - accuracy: 0.9890 - val_loss: 0.1095 - val_accuracy: 0.9775 - lr: 1.0000e-03 Epoch 14/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0326 - accuracy: 0.9900 - val_loss: 0.1083 - val_accuracy: 0.9778 - lr: 1.0000e-04 Epoch 15/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0307 - accuracy: 0.9906 - val_loss: 0.1076 - val_accuracy: 0.9777 - lr: 1.0000e-04 Epoch 16/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0304 - accuracy: 0.9903 - val_loss: 0.1071 - val_accuracy: 0.9780 - lr: 1.0000e-04 Epoch 17/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0295 - accuracy: 0.9906 - val_loss: 0.1073 - val_accuracy: 0.9778 - lr: 1.0000e-04 Epoch 18/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0284 - accuracy: 0.9911 - val_loss: 0.1065 - val_accuracy: 0.9785 - lr: 1.0000e-04 Epoch 19/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0293 - accuracy: 0.9912 - val_loss: 0.1065 - val_accuracy: 0.9783 - lr: 1.0000e-04 Epoch 20/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0262 - accuracy: 0.9917 - val_loss: 0.1070 - val_accuracy: 0.9783 - lr: 1.0000e-04 Epoch 21/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0281 - accuracy: 0.9911 - val_loss: 0.1071 - val_accuracy: 0.9780 - lr: 1.0000e-05 Epoch 22/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0270 - accuracy: 0.9916 - val_loss: 0.1070 - val_accuracy: 0.9782 - lr: 1.0000e-05 Epoch 23/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0271 - accuracy: 0.9916 - val_loss: 0.1071 - val_accuracy: 0.9782 - lr: 1.0000e-06 Epoch 24/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0292 - accuracy: 0.9911 - val_loss: 0.1070 - val_accuracy: 0.9782 - lr: 1.0000e-06 Epoch 25/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0276 - accuracy: 0.9913 - val_loss: 0.1070 - val_accuracy: 0.9782 - lr: 1.0000e-07 Epoch 26/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0280 - accuracy: 0.9913 - val_loss: 0.1070 - val_accuracy: 0.9782 - lr: 1.0000e-07 Trial 86 Complete [00h 00m 30s] val_accuracy: 0.9785000085830688 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 17m 24s Search: Running Trial #87 Value |Best Value So Far |Hyperparameter 1 |4 |depth 64 |512 |width relu |relu |activation l1_l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout rmsprop |sgd |optimizer 0.001 |0.0001 |learning_rate 128 |32 |batch_size 32 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/32 422/422 [==============================] - 2s 3ms/step - loss: 0.5012 - accuracy: 0.8646 - val_loss: 0.2749 - val_accuracy: 0.9293 - lr: 0.0010 Epoch 2/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2662 - accuracy: 0.9293 - val_loss: 0.2220 - val_accuracy: 0.9418 - lr: 0.0010 Epoch 3/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2165 - accuracy: 0.9434 - val_loss: 0.1928 - val_accuracy: 0.9502 - lr: 0.0010 Epoch 4/32 422/422 [==============================] - 1s 2ms/step - loss: 0.1876 - accuracy: 0.9525 - val_loss: 0.1749 - val_accuracy: 0.9552 - lr: 0.0010 Epoch 5/32 422/422 [==============================] - 1s 2ms/step - loss: 0.1661 - accuracy: 0.9588 - val_loss: 0.1567 - val_accuracy: 0.9602 - lr: 0.0010 Epoch 6/32 422/422 [==============================] - 1s 2ms/step - loss: 0.1507 - accuracy: 0.9626 - val_loss: 0.1497 - val_accuracy: 0.9617 - lr: 0.0010 Epoch 7/32 422/422 [==============================] - 1s 2ms/step - loss: 0.1404 - accuracy: 0.9659 - val_loss: 0.1385 - val_accuracy: 0.9655 - lr: 0.0010 Epoch 8/32 422/422 [==============================] - 1s 2ms/step - loss: 0.1317 - accuracy: 0.9679 - val_loss: 0.1316 - val_accuracy: 0.9673 - lr: 0.0010 Epoch 9/32 422/422 [==============================] - 1s 2ms/step - loss: 0.1223 - accuracy: 0.9710 - val_loss: 0.1283 - val_accuracy: 0.9702 - lr: 0.0010 Epoch 10/32 422/422 [==============================] - 1s 2ms/step - loss: 0.1143 - accuracy: 0.9735 - val_loss: 0.1289 - val_accuracy: 0.9690 - lr: 0.0010 Epoch 11/32 422/422 [==============================] - 1s 2ms/step - loss: 0.1110 - accuracy: 0.9747 - val_loss: 0.1211 - val_accuracy: 0.9698 - lr: 0.0010 Epoch 12/32 422/422 [==============================] - 1s 2ms/step - loss: 0.0991 - accuracy: 0.9781 - val_loss: 0.1188 - val_accuracy: 0.9717 - lr: 1.0000e-04 Epoch 13/32 422/422 [==============================] - 1s 2ms/step - loss: 0.0961 - accuracy: 0.9795 - val_loss: 0.1179 - val_accuracy: 0.9715 - lr: 1.0000e-04 Epoch 14/32 422/422 [==============================] - 1s 2ms/step - loss: 0.0956 - accuracy: 0.9800 - val_loss: 0.1171 - val_accuracy: 0.9705 - lr: 1.0000e-04 Epoch 15/32 422/422 [==============================] - 1s 2ms/step - loss: 0.0935 - accuracy: 0.9802 - val_loss: 0.1169 - val_accuracy: 0.9708 - lr: 1.0000e-05 Epoch 16/32 422/422 [==============================] - 1s 2ms/step - loss: 0.0937 - accuracy: 0.9802 - val_loss: 0.1168 - val_accuracy: 0.9712 - lr: 1.0000e-05 Epoch 17/32 422/422 [==============================] - 1s 2ms/step - loss: 0.0930 - accuracy: 0.9801 - val_loss: 0.1168 - val_accuracy: 0.9712 - lr: 1.0000e-06 Epoch 18/32 422/422 [==============================] - 1s 2ms/step - loss: 0.0925 - accuracy: 0.9806 - val_loss: 0.1168 - val_accuracy: 0.9712 - lr: 1.0000e-06 Epoch 19/32 422/422 [==============================] - 1s 2ms/step - loss: 0.0934 - accuracy: 0.9799 - val_loss: 0.1168 - val_accuracy: 0.9712 - lr: 1.0000e-07 Epoch 20/32 422/422 [==============================] - 1s 2ms/step - loss: 0.0929 - accuracy: 0.9808 - val_loss: 0.1168 - val_accuracy: 0.9712 - lr: 1.0000e-07 Trial 87 Complete [00h 00m 21s] val_accuracy: 0.971666693687439 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 17m 44s Search: Running Trial #88 Value |Best Value So Far |Hyperparameter 3 |4 |depth 128 |512 |width swish |relu |activation none |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.4 |dropout sgd |sgd |optimizer 0.01 |0.0001 |learning_rate 512 |32 |batch_size 32 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/32 106/106 [==============================] - 1s 6ms/step - loss: 0.3897 - accuracy: 0.8796 - val_loss: 0.1507 - val_accuracy: 0.9522 - lr: 0.0100 Epoch 2/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1945 - accuracy: 0.9437 - val_loss: 0.1341 - val_accuracy: 0.9612 - lr: 0.0100 Epoch 3/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1667 - accuracy: 0.9512 - val_loss: 0.1114 - val_accuracy: 0.9677 - lr: 0.0100 Epoch 4/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1571 - accuracy: 0.9555 - val_loss: 0.1039 - val_accuracy: 0.9703 - lr: 0.0100 Epoch 5/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1476 - accuracy: 0.9574 - val_loss: 0.1041 - val_accuracy: 0.9715 - lr: 0.0100 Epoch 6/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1441 - accuracy: 0.9599 - val_loss: 0.1099 - val_accuracy: 0.9690 - lr: 0.0100 Epoch 7/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1352 - accuracy: 0.9621 - val_loss: 0.1077 - val_accuracy: 0.9717 - lr: 0.0100 Epoch 8/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1272 - accuracy: 0.9642 - val_loss: 0.1066 - val_accuracy: 0.9733 - lr: 0.0100 Epoch 9/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1309 - accuracy: 0.9646 - val_loss: 0.1093 - val_accuracy: 0.9723 - lr: 0.0100 Epoch 10/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1334 - accuracy: 0.9636 - val_loss: 0.1160 - val_accuracy: 0.9728 - lr: 0.0100 Epoch 11/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0955 - accuracy: 0.9737 - val_loss: 0.0973 - val_accuracy: 0.9750 - lr: 1.0000e-03 Epoch 12/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0723 - accuracy: 0.9798 - val_loss: 0.0946 - val_accuracy: 0.9762 - lr: 1.0000e-03 Epoch 13/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0648 - accuracy: 0.9806 - val_loss: 0.0941 - val_accuracy: 0.9777 - lr: 1.0000e-03 Epoch 14/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0593 - accuracy: 0.9829 - val_loss: 0.0899 - val_accuracy: 0.9787 - lr: 1.0000e-03 Epoch 15/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0584 - accuracy: 0.9828 - val_loss: 0.0908 - val_accuracy: 0.9785 - lr: 1.0000e-03 Epoch 16/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0539 - accuracy: 0.9846 - val_loss: 0.0929 - val_accuracy: 0.9787 - lr: 1.0000e-03 Epoch 17/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0513 - accuracy: 0.9846 - val_loss: 0.0916 - val_accuracy: 0.9790 - lr: 1.0000e-04 Epoch 18/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0516 - accuracy: 0.9851 - val_loss: 0.0913 - val_accuracy: 0.9790 - lr: 1.0000e-04 Epoch 19/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0502 - accuracy: 0.9853 - val_loss: 0.0907 - val_accuracy: 0.9795 - lr: 1.0000e-04 Epoch 20/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0498 - accuracy: 0.9854 - val_loss: 0.0900 - val_accuracy: 0.9795 - lr: 1.0000e-04 Epoch 21/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0493 - accuracy: 0.9852 - val_loss: 0.0896 - val_accuracy: 0.9795 - lr: 1.0000e-04 Epoch 22/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0496 - accuracy: 0.9854 - val_loss: 0.0896 - val_accuracy: 0.9795 - lr: 1.0000e-05 Epoch 23/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0488 - accuracy: 0.9849 - val_loss: 0.0896 - val_accuracy: 0.9795 - lr: 1.0000e-05 Epoch 24/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0472 - accuracy: 0.9857 - val_loss: 0.0896 - val_accuracy: 0.9795 - lr: 1.0000e-06 Epoch 25/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0493 - accuracy: 0.9853 - val_loss: 0.0896 - val_accuracy: 0.9795 - lr: 1.0000e-06 Epoch 26/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0483 - accuracy: 0.9858 - val_loss: 0.0896 - val_accuracy: 0.9795 - lr: 1.0000e-07 Epoch 27/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0504 - accuracy: 0.9853 - val_loss: 0.0896 - val_accuracy: 0.9795 - lr: 1.0000e-07 Trial 88 Complete [00h 00m 13s] val_accuracy: 0.9794999957084656 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 17m 57s Search: Running Trial #89 Value |Best Value So Far |Hyperparameter 1 |4 |depth 256 |512 |width swish |relu |activation l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0 |0.4 |dropout adam |sgd |optimizer 0.001 |0.0001 |learning_rate 512 |32 |batch_size 32 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/32 106/106 [==============================] - 1s 5ms/step - loss: 0.5740 - accuracy: 0.8566 - val_loss: 0.3245 - val_accuracy: 0.9178 - lr: 0.0010 Epoch 2/32 106/106 [==============================] - 0s 3ms/step - loss: 0.2906 - accuracy: 0.9279 - val_loss: 0.2691 - val_accuracy: 0.9318 - lr: 0.0010 Epoch 3/32 106/106 [==============================] - 0s 3ms/step - loss: 0.2355 - accuracy: 0.9427 - val_loss: 0.2237 - val_accuracy: 0.9470 - lr: 0.0010 Epoch 4/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1981 - accuracy: 0.9536 - val_loss: 0.1995 - val_accuracy: 0.9515 - lr: 0.0010 Epoch 5/32 106/106 [==============================] - 0s 3ms/step - loss: 0.1705 - accuracy: 0.9619 - val_loss: 0.1783 - val_accuracy: 0.9585 - lr: 0.0010 Epoch 6/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1517 - accuracy: 0.9671 - val_loss: 0.1646 - val_accuracy: 0.9618 - lr: 0.0010 Epoch 7/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1379 - accuracy: 0.9716 - val_loss: 0.1535 - val_accuracy: 0.9643 - lr: 0.0010 Epoch 8/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1259 - accuracy: 0.9751 - val_loss: 0.1443 - val_accuracy: 0.9673 - lr: 0.0010 Epoch 9/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1169 - accuracy: 0.9777 - val_loss: 0.1369 - val_accuracy: 0.9682 - lr: 0.0010 Epoch 10/32 106/106 [==============================] - 0s 4ms/step - loss: 0.1087 - accuracy: 0.9797 - val_loss: 0.1317 - val_accuracy: 0.9715 - lr: 0.0010 Epoch 11/32 106/106 [==============================] - 0s 3ms/step - loss: 0.1026 - accuracy: 0.9818 - val_loss: 0.1301 - val_accuracy: 0.9707 - lr: 0.0010 Epoch 12/32 106/106 [==============================] - 0s 3ms/step - loss: 0.0979 - accuracy: 0.9831 - val_loss: 0.1253 - val_accuracy: 0.9715 - lr: 0.0010 Epoch 13/32 106/106 [==============================] - 0s 3ms/step - loss: 0.0880 - accuracy: 0.9867 - val_loss: 0.1216 - val_accuracy: 0.9737 - lr: 1.0000e-04 Epoch 14/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0864 - accuracy: 0.9870 - val_loss: 0.1214 - val_accuracy: 0.9738 - lr: 1.0000e-04 Epoch 15/32 106/106 [==============================] - 0s 4ms/step - loss: 0.0857 - accuracy: 0.9874 - val_loss: 0.1210 - val_accuracy: 0.9742 - lr: 1.0000e-04 Epoch 16/32 106/106 [==============================] - 0s 3ms/step - loss: 0.0851 - accuracy: 0.9875 - val_loss: 0.1208 - val_accuracy: 0.9740 - lr: 1.0000e-04 Epoch 17/32 106/106 [==============================] - 0s 3ms/step - loss: 0.0845 - accuracy: 0.9876 - val_loss: 0.1201 - val_accuracy: 0.9737 - lr: 1.0000e-04 Epoch 18/32 106/106 [==============================] - 0s 3ms/step - loss: 0.0834 - accuracy: 0.9883 - val_loss: 0.1201 - val_accuracy: 0.9737 - lr: 1.0000e-05 Epoch 19/32 106/106 [==============================] - 0s 3ms/step - loss: 0.0833 - accuracy: 0.9882 - val_loss: 0.1201 - val_accuracy: 0.9738 - lr: 1.0000e-05 Epoch 20/32 106/106 [==============================] - 0s 3ms/step - loss: 0.0832 - accuracy: 0.9882 - val_loss: 0.1201 - val_accuracy: 0.9737 - lr: 1.0000e-06 Epoch 21/32 106/106 [==============================] - 0s 3ms/step - loss: 0.0832 - accuracy: 0.9882 - val_loss: 0.1201 - val_accuracy: 0.9737 - lr: 1.0000e-06 Epoch 22/32 106/106 [==============================] - 0s 3ms/step - loss: 0.0832 - accuracy: 0.9882 - val_loss: 0.1201 - val_accuracy: 0.9737 - lr: 1.0000e-07 Epoch 23/32 106/106 [==============================] - 0s 3ms/step - loss: 0.0832 - accuracy: 0.9882 - val_loss: 0.1201 - val_accuracy: 0.9737 - lr: 1.0000e-07 Trial 89 Complete [00h 00m 09s] val_accuracy: 0.9741666913032532 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 18m 07s Search: Running Trial #90 Value |Best Value So Far |Hyperparameter 2 |4 |depth 256 |512 |width tanh |relu |activation l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout adam |sgd |optimizer 0.001 |0.0001 |learning_rate 64 |32 |batch_size 32 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/32 844/844 [==============================] - 3s 3ms/step - loss: 0.3003 - accuracy: 0.9113 - val_loss: 0.1909 - val_accuracy: 0.9470 - lr: 0.0010 Epoch 2/32 844/844 [==============================] - 2s 3ms/step - loss: 0.1575 - accuracy: 0.9546 - val_loss: 0.1410 - val_accuracy: 0.9600 - lr: 0.0010 Epoch 3/32 844/844 [==============================] - 2s 3ms/step - loss: 0.1159 - accuracy: 0.9680 - val_loss: 0.1140 - val_accuracy: 0.9680 - lr: 0.0010 Epoch 4/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0940 - accuracy: 0.9740 - val_loss: 0.1182 - val_accuracy: 0.9677 - lr: 0.0010 Epoch 5/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0800 - accuracy: 0.9779 - val_loss: 0.1042 - val_accuracy: 0.9705 - lr: 0.0010 Epoch 6/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0699 - accuracy: 0.9816 - val_loss: 0.0980 - val_accuracy: 0.9748 - lr: 0.0010 Epoch 7/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0637 - accuracy: 0.9838 - val_loss: 0.0984 - val_accuracy: 0.9752 - lr: 0.0010 Epoch 8/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0608 - accuracy: 0.9851 - val_loss: 0.0944 - val_accuracy: 0.9768 - lr: 0.0010 Epoch 9/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0538 - accuracy: 0.9877 - val_loss: 0.0898 - val_accuracy: 0.9798 - lr: 0.0010 Epoch 10/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0523 - accuracy: 0.9883 - val_loss: 0.1012 - val_accuracy: 0.9755 - lr: 0.0010 Epoch 11/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0500 - accuracy: 0.9893 - val_loss: 0.1005 - val_accuracy: 0.9788 - lr: 0.0010 Epoch 12/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0338 - accuracy: 0.9959 - val_loss: 0.0859 - val_accuracy: 0.9803 - lr: 1.0000e-04 Epoch 13/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0300 - accuracy: 0.9971 - val_loss: 0.0847 - val_accuracy: 0.9807 - lr: 1.0000e-04 Epoch 14/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0289 - accuracy: 0.9974 - val_loss: 0.0833 - val_accuracy: 0.9813 - lr: 1.0000e-04 Epoch 15/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0275 - accuracy: 0.9976 - val_loss: 0.0859 - val_accuracy: 0.9813 - lr: 1.0000e-04 Epoch 16/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0267 - accuracy: 0.9979 - val_loss: 0.0832 - val_accuracy: 0.9828 - lr: 1.0000e-04 Epoch 17/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0258 - accuracy: 0.9982 - val_loss: 0.0833 - val_accuracy: 0.9822 - lr: 1.0000e-04 Epoch 18/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0248 - accuracy: 0.9986 - val_loss: 0.0841 - val_accuracy: 0.9815 - lr: 1.0000e-04 Epoch 19/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0239 - accuracy: 0.9990 - val_loss: 0.0830 - val_accuracy: 0.9825 - lr: 1.0000e-05 Epoch 20/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0240 - accuracy: 0.9988 - val_loss: 0.0829 - val_accuracy: 0.9825 - lr: 1.0000e-05 Epoch 21/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0238 - accuracy: 0.9990 - val_loss: 0.0829 - val_accuracy: 0.9825 - lr: 1.0000e-06 Epoch 22/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0239 - accuracy: 0.9989 - val_loss: 0.0829 - val_accuracy: 0.9825 - lr: 1.0000e-06 Epoch 23/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0238 - accuracy: 0.9991 - val_loss: 0.0829 - val_accuracy: 0.9825 - lr: 1.0000e-07 Epoch 24/32 844/844 [==============================] - 2s 3ms/step - loss: 0.0241 - accuracy: 0.9986 - val_loss: 0.0829 - val_accuracy: 0.9825 - lr: 1.0000e-07 Trial 90 Complete [00h 00m 56s] val_accuracy: 0.9828333258628845 Best val_accuracy So Far: 0.9833333492279053 Total elapsed time: 00h 19m 03s Results summary Results in tuning/bandit Showing 10 best trials Trial summary Hyperparameters: depth: 4 width: 512 activation: relu regularization_function: none regularization_penalty: 1e-05 dropout: 0.4 optimizer: sgd learning_rate: 0.0001 batch_size: 32 tuner/epochs: 32 tuner/initial_epoch: 11 tuner/bracket: 2 tuner/round: 2 tuner/trial_id: 0067 Score: 0.9833333492279053 Trial summary Hyperparameters: depth: 2 width: 256 activation: tanh regularization_function: l2 regularization_penalty: 1e-05 dropout: 0.1 optimizer: adam learning_rate: 0.001 batch_size: 64 tuner/epochs: 32 tuner/initial_epoch: 0 tuner/bracket: 0 tuner/round: 0 Score: 0.9828333258628845 Trial summary Hyperparameters: depth: 2 width: 256 activation: relu regularization_function: none regularization_penalty: 0.0001 dropout: 0.4 optimizer: rmsprop learning_rate: 0.001 batch_size: 512 tuner/epochs: 32 tuner/initial_epoch: 11 tuner/bracket: 2 tuner/round: 2 tuner/trial_id: 0068 Score: 0.9819999933242798 Trial summary Hyperparameters: depth: 3 width: 256 activation: sigmoid regularization_function: l1 regularization_penalty: 1e-05 dropout: 0.1 optimizer: adam learning_rate: 0.001 batch_size: 32 tuner/epochs: 32 tuner/initial_epoch: 11 tuner/bracket: 3 tuner/round: 3 tuner/trial_id: 0049 Score: 0.9808333516120911 Trial summary Hyperparameters: depth: 3 width: 128 activation: swish regularization_function: none regularization_penalty: 0.0001 dropout: 0.30000000000000004 optimizer: sgd learning_rate: 0.01 batch_size: 512 tuner/epochs: 32 tuner/initial_epoch: 0 tuner/bracket: 0 tuner/round: 0 Score: 0.9794999957084656 Trial summary Hyperparameters: depth: 2 width: 128 activation: swish regularization_function: none regularization_penalty: 0.0001 dropout: 0.1 optimizer: sgd learning_rate: 0.01 batch_size: 128 tuner/epochs: 32 tuner/initial_epoch: 0 tuner/bracket: 0 tuner/round: 0 Score: 0.9785000085830688 Trial summary Hyperparameters: depth: 3 width: 128 activation: swish regularization_function: none regularization_penalty: 1e-05 dropout: 0.4 optimizer: adam learning_rate: 0.001 batch_size: 128 tuner/epochs: 32 tuner/initial_epoch: 11 tuner/bracket: 3 tuner/round: 3 tuner/trial_id: 0048 Score: 0.9781666398048401 Trial summary Hyperparameters: depth: 4 width: 512 activation: relu regularization_function: none regularization_penalty: 1e-05 dropout: 0.4 optimizer: sgd learning_rate: 0.0001 batch_size: 32 tuner/epochs: 11 tuner/initial_epoch: 4 tuner/bracket: 2 tuner/round: 1 tuner/trial_id: 0056 Score: 0.9750000238418579 Trial summary Hyperparameters: depth: 1 width: 256 activation: swish regularization_function: l2 regularization_penalty: 0.0001 dropout: 0.0 optimizer: adam learning_rate: 0.001 batch_size: 512 tuner/epochs: 32 tuner/initial_epoch: 0 tuner/bracket: 0 tuner/round: 0 Score: 0.9741666913032532 Trial summary Hyperparameters: depth: 2 width: 256 activation: relu regularization_function: none regularization_penalty: 0.0001 dropout: 0.4 optimizer: rmsprop learning_rate: 0.001 batch_size: 512 tuner/epochs: 11 tuner/initial_epoch: 4 tuner/bracket: 2 tuner/round: 1 tuner/trial_id: 0052 Score: 0.9739999771118164 {'depth': 4, 'width': 512, 'activation': 'relu', 'regularization_function': 'none', 'regularization_penalty': 1e-05, 'dropout': 0.4, 'optimizer': 'sgd', 'learning_rate': 0.0001, 'batch_size': 32, 'tuner/epochs': 32, 'tuner/initial_epoch': 11, 'tuner/bracket': 2, 'tuner/round': 2, 'tuner/trial_id': '0067'} 313/313 [==============================] - 1s 1ms/step Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense (Dense) (None, 512) 401920 dropout (Dropout) (None, 512) 0 dense_1 (Dense) (None, 512) 262656 dropout_1 (Dropout) (None, 512) 0 dense_2 (Dense) (None, 512) 262656 dropout_2 (Dropout) (None, 512) 0 dense_3 (Dense) (None, 512) 262656 dropout_3 (Dropout) (None, 512) 0 dense_4 (Dense) (None, 10) 5130 ================================================================= Total params: 1,195,018 Trainable params: 1,195,018 Non-trainable params: 0 _________________________________________________________________ real 19m37.838s user 23m33.825s sys 3m15.511s deeplearn@ML-RefVm-967342:~/mnist$ kaggle competitions submit -c ml530-2022-fall-mnist -f predictions.csv -m "'depth': 4" Warning: Your Kaggle API key is readable by other users on this system! To fix this, you can run 'chmod 600 /home/deeplearn/.kaggle/kaggle.json' 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 78.1k/78.1k [00:00<00:00, 102kB/s] Successfully submitted to ml530-2022-fall-mnistdeeplearn@ML-RefVm-967342:~/mnist$ rm -rf tuning/bandit/ deeplearn@ML-RefVm-967342:~/mnist$ time python mnist-search.py.txt 2022-10-01 16:27:16.208677: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-10-01 16:27:16.772296: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 10790 MB memory: -> device: 0, name: Tesla K80, pci bus id: 0001:00:00.0, compute capability: 3.7 Search space summary Default search space size: 8 depth (Int) {'default': None, 'conditions': [], 'min_value': 0, 'max_value': 4, 'step': 1, 'sampling': None} width (Choice) {'default': 64, 'conditions': [], 'values': [64, 128, 256, 512], 'ordered': True} activation (Choice) {'default': 'linear', 'conditions': [], 'values': ['linear', 'relu', 'sigmoid', 'swish', 'tanh'], 'ordered': False} regularization_function (Choice) {'default': 'none', 'conditions': [], 'values': ['none', 'l1', 'l2', 'l1_l2'], 'ordered': False} regularization_penalty (Choice) {'default': 0.0001, 'conditions': [], 'values': [0.0001, 1e-05], 'ordered': True} dropout (Float) {'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.5, 'step': 0.1, 'sampling': None} optimizer (Choice) {'default': 'adam', 'conditions': [], 'values': ['adam', 'rmsprop', 'sgd'], 'ordered': False} learning_rate (Choice) {'default': 0.01, 'conditions': [], 'values': [0.01, 0.001, 0.0001], 'ordered': True} Search: Running Trial #1 Value |Best Value So Far |Hyperparameter 2 |? |depth 512 |? |width linear |? |activation l2 |? |regularization_function 1e-05 |? |regularization_penalty 0.2 |? |dropout sgd |? |optimizer 0.0001 |? |learning_rate 2 |? |tuner/epochs 0 |? |tuner/initial_epoch 3 |? |tuner/bracket 0 |? |tuner/round Epoch 1/2 1688/1688 [==============================] - 6s 3ms/step - loss: 0.4402 - accuracy: 0.8772 - val_loss: 0.3151 - val_accuracy: 0.9143 - lr: 1.0000e-04 Epoch 2/2 1688/1688 [==============================] - 5s 3ms/step - loss: 0.3245 - accuracy: 0.9101 - val_loss: 0.3185 - val_accuracy: 0.9125 - lr: 1.0000e-04 Trial 1 Complete [00h 00m 12s] val_accuracy: 0.9143333435058594 Best val_accuracy So Far: 0.9143333435058594 Total elapsed time: 00h 00m 12s Search: Running Trial #2 Value |Best Value So Far |Hyperparameter 2 |2 |depth 128 |512 |width sigmoid |linear |activation none |l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.2 |dropout adam |sgd |optimizer 0.01 |0.0001 |learning_rate 64 |32 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 3s 3ms/step - loss: 0.3979 - accuracy: 0.8778 - val_loss: 0.1744 - val_accuracy: 0.9448 - lr: 0.0100 Epoch 2/2 844/844 [==============================] - 2s 2ms/step - loss: 0.2324 - accuracy: 0.9307 - val_loss: 0.1400 - val_accuracy: 0.9577 - lr: 0.0100 Trial 2 Complete [00h 00m 05s] val_accuracy: 0.9576666951179504 Best val_accuracy So Far: 0.9576666951179504 Total elapsed time: 00h 00m 17s Search: Running Trial #3 Value |Best Value So Far |Hyperparameter 1 |2 |depth 256 |128 |width swish |sigmoid |activation l1 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.4 |dropout sgd |adam |optimizer 0.01 |0.01 |learning_rate 128 |64 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 422/422 [==============================] - 2s 3ms/step - loss: 0.3939 - accuracy: 0.9092 - val_loss: 0.2698 - val_accuracy: 0.9560 - lr: 0.0100 Epoch 2/2 422/422 [==============================] - 1s 3ms/step - loss: 0.3496 - accuracy: 0.9354 - val_loss: 0.2912 - val_accuracy: 0.9583 - lr: 0.0100 Trial 3 Complete [00h 00m 03s] val_accuracy: 0.9583333134651184 Best val_accuracy So Far: 0.9583333134651184 Total elapsed time: 00h 00m 20s Search: Running Trial #4 Value |Best Value So Far |Hyperparameter 1 |1 |depth 512 |256 |width linear |swish |activation l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.4 |dropout rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate 1024 |128 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 53/53 [==============================] - 1s 9ms/step - loss: 3.0002 - accuracy: 0.5043 - val_loss: 2.3159 - val_accuracy: 0.7957 - lr: 1.0000e-04 Epoch 2/2 53/53 [==============================] - 0s 6ms/step - loss: 2.1305 - accuracy: 0.7943 - val_loss: 1.9034 - val_accuracy: 0.8472 - lr: 1.0000e-04 Trial 4 Complete [00h 00m 02s] val_accuracy: 0.8471666574478149 Best val_accuracy So Far: 0.9583333134651184 Total elapsed time: 00h 00m 22s Search: Running Trial #5 Value |Best Value So Far |Hyperparameter 3 |1 |depth 64 |256 |width relu |swish |activation l1 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.4 |dropout rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate 32 |128 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 1688/1688 [==============================] - 5s 3ms/step - loss: 1.4815 - accuracy: 0.6025 - val_loss: 0.9184 - val_accuracy: 0.8452 - lr: 0.0100 Epoch 2/2 1688/1688 [==============================] - 5s 3ms/step - loss: 1.4701 - accuracy: 0.6329 - val_loss: 1.0795 - val_accuracy: 0.7832 - lr: 0.0100 Trial 5 Complete [00h 00m 10s] val_accuracy: 0.8451666831970215 Best val_accuracy So Far: 0.9583333134651184 Total elapsed time: 00h 00m 32s Search: Running Trial #6 Value |Best Value So Far |Hyperparameter 1 |1 |depth 512 |256 |width relu |swish |activation none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 256 |128 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 1s 4ms/step - loss: 0.2480 - accuracy: 0.9258 - val_loss: 0.1169 - val_accuracy: 0.9658 - lr: 0.0100 Epoch 2/2 211/211 [==============================] - 1s 3ms/step - loss: 0.1062 - accuracy: 0.9664 - val_loss: 0.1156 - val_accuracy: 0.9670 - lr: 0.0100 Trial 6 Complete [00h 00m 02s] val_accuracy: 0.9670000076293945 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 00m 34s Search: Running Trial #7 Value |Best Value So Far |Hyperparameter 1 |1 |depth 128 |512 |width relu |relu |activation none |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.1 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 32 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 1688/1688 [==============================] - 4s 2ms/step - loss: 0.2786 - accuracy: 0.9169 - val_loss: 0.1928 - val_accuracy: 0.9472 - lr: 0.0100 Epoch 2/2 1688/1688 [==============================] - 4s 2ms/step - loss: 0.2029 - accuracy: 0.9427 - val_loss: 0.1840 - val_accuracy: 0.9477 - lr: 0.0100 Trial 7 Complete [00h 00m 08s] val_accuracy: 0.9476666450500488 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 00m 43s Search: Running Trial #8 Value |Best Value So Far |Hyperparameter 4 |1 |depth 128 |512 |width sigmoid |relu |activation l1_l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.1 |dropout adam |sgd |optimizer 0.001 |0.01 |learning_rate 512 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 106/106 [==============================] - 1s 7ms/step - loss: 2.2932 - accuracy: 0.1797 - val_loss: 1.6628 - val_accuracy: 0.4633 - lr: 0.0010 Epoch 2/2 106/106 [==============================] - 0s 5ms/step - loss: 1.2970 - accuracy: 0.5635 - val_loss: 0.8556 - val_accuracy: 0.7802 - lr: 0.0010 Trial 8 Complete [00h 00m 02s] val_accuracy: 0.7801666855812073 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 00m 45s Search: Running Trial #9 Value |Best Value So Far |Hyperparameter 3 |1 |depth 128 |512 |width linear |relu |activation l1_l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.1 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 512 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 106/106 [==============================] - 1s 6ms/step - loss: 0.7458 - accuracy: 0.8337 - val_loss: 0.4101 - val_accuracy: 0.9035 - lr: 0.0100 Epoch 2/2 106/106 [==============================] - 0s 4ms/step - loss: 0.4135 - accuracy: 0.9006 - val_loss: 0.4395 - val_accuracy: 0.8963 - lr: 0.0100 Trial 9 Complete [00h 00m 02s] val_accuracy: 0.9035000205039978 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 00m 47s Search: Running Trial #10 Value |Best Value So Far |Hyperparameter 0 |1 |depth 128 |512 |width sigmoid |relu |activation l1_l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.1 |dropout sgd |sgd |optimizer 0.0001 |0.01 |learning_rate 32 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 1688/1688 [==============================] - 4s 2ms/step - loss: 1.2084 - accuracy: 0.7013 - val_loss: 0.7105 - val_accuracy: 0.8425 - lr: 1.0000e-04 Epoch 2/2 1688/1688 [==============================] - 3s 2ms/step - loss: 0.5884 - accuracy: 0.8629 - val_loss: 0.4984 - val_accuracy: 0.8777 - lr: 1.0000e-04 Trial 10 Complete [00h 00m 08s] val_accuracy: 0.8776666522026062 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 00m 55s Search: Running Trial #11 Value |Best Value So Far |Hyperparameter 4 |1 |depth 64 |512 |width swish |relu |activation none |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.1 |dropout adam |sgd |optimizer 0.0001 |0.01 |learning_rate 64 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 3s 3ms/step - loss: 1.4476 - accuracy: 0.5413 - val_loss: 0.6224 - val_accuracy: 0.8402 - lr: 1.0000e-04 Epoch 2/2 844/844 [==============================] - 3s 3ms/step - loss: 0.7421 - accuracy: 0.7731 - val_loss: 0.4181 - val_accuracy: 0.8832 - lr: 1.0000e-04 Trial 11 Complete [00h 00m 06s] val_accuracy: 0.8831666707992554 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 01s Search: Running Trial #12 Value |Best Value So Far |Hyperparameter 1 |1 |depth 128 |512 |width linear |relu |activation l1_l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0 |0.1 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 256 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 1s 3ms/step - loss: 0.4956 - accuracy: 0.8757 - val_loss: 0.3824 - val_accuracy: 0.9087 - lr: 0.0100 Epoch 2/2 211/211 [==============================] - 1s 3ms/step - loss: 0.3613 - accuracy: 0.9109 - val_loss: 0.4011 - val_accuracy: 0.9025 - lr: 0.0100 Trial 12 Complete [00h 00m 02s] val_accuracy: 0.9086666703224182 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 03s Search: Running Trial #13 Value |Best Value So Far |Hyperparameter 0 |1 |depth 128 |512 |width relu |relu |activation l1_l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0 |0.1 |dropout rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate 512 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 106/106 [==============================] - 1s 4ms/step - loss: 2.1844 - accuracy: 0.2155 - val_loss: 1.9558 - val_accuracy: 0.3883 - lr: 1.0000e-04 Epoch 2/2 106/106 [==============================] - 0s 3ms/step - loss: 1.7906 - accuracy: 0.5227 - val_loss: 1.6231 - val_accuracy: 0.6385 - lr: 1.0000e-04 Trial 13 Complete [00h 00m 01s] val_accuracy: 0.6384999752044678 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 04s Search: Running Trial #14 Value |Best Value So Far |Hyperparameter 3 |1 |depth 64 |512 |width swish |relu |activation l1 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.1 |dropout adam |sgd |optimizer 0.0001 |0.01 |learning_rate 1024 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 53/53 [==============================] - 1s 9ms/step - loss: 2.5408 - accuracy: 0.1858 - val_loss: 2.4433 - val_accuracy: 0.5220 - lr: 1.0000e-04 Epoch 2/2 53/53 [==============================] - 0s 5ms/step - loss: 2.3521 - accuracy: 0.3380 - val_loss: 2.1601 - val_accuracy: 0.6478 - lr: 1.0000e-04 Trial 14 Complete [00h 00m 02s] val_accuracy: 0.6478333473205566 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 06s Search: Running Trial #15 Value |Best Value So Far |Hyperparameter 2 |1 |depth 128 |512 |width relu |relu |activation l1_l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0 |0.1 |dropout rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate 512 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 106/106 [==============================] - 1s 5ms/step - loss: 2.2826 - accuracy: 0.5669 - val_loss: 1.6211 - val_accuracy: 0.7858 - lr: 1.0000e-04 Epoch 2/2 106/106 [==============================] - 0s 4ms/step - loss: 1.2650 - accuracy: 0.8388 - val_loss: 1.0351 - val_accuracy: 0.8647 - lr: 1.0000e-04 Trial 15 Complete [00h 00m 02s] val_accuracy: 0.8646666407585144 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 08s Search: Running Trial #16 Value |Best Value So Far |Hyperparameter 1 |1 |depth 128 |512 |width linear |relu |activation l1 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.1 |dropout rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate 1024 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 53/53 [==============================] - 1s 7ms/step - loss: 0.8896 - accuracy: 0.8291 - val_loss: 0.5484 - val_accuracy: 0.9102 - lr: 0.0100 Epoch 2/2 53/53 [==============================] - 0s 4ms/step - loss: 0.5450 - accuracy: 0.8974 - val_loss: 0.4803 - val_accuracy: 0.9157 - lr: 0.0100 Trial 16 Complete [00h 00m 01s] val_accuracy: 0.9156666398048401 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 09s Search: Running Trial #17 Value |Best Value So Far |Hyperparameter 4 |1 |depth 128 |512 |width sigmoid |relu |activation l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.1 |dropout rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate 256 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 2s 5ms/step - loss: 0.8302 - accuracy: 0.7290 - val_loss: 0.2812 - val_accuracy: 0.9392 - lr: 0.0100 Epoch 2/2 211/211 [==============================] - 1s 4ms/step - loss: 0.2869 - accuracy: 0.9364 - val_loss: 0.2117 - val_accuracy: 0.9588 - lr: 0.0100 Trial 17 Complete [00h 00m 03s] val_accuracy: 0.9588333368301392 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 12s Search: Running Trial #18 Value |Best Value So Far |Hyperparameter 4 |1 |depth 64 |512 |width tanh |relu |activation l1_l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.5 |0.1 |dropout sgd |sgd |optimizer 0.0001 |0.01 |learning_rate 256 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 2s 5ms/step - loss: 2.3050 - accuracy: 0.1964 - val_loss: 1.4547 - val_accuracy: 0.6268 - lr: 1.0000e-04 Epoch 2/2 211/211 [==============================] - 1s 4ms/step - loss: 1.7673 - accuracy: 0.3836 - val_loss: 1.0120 - val_accuracy: 0.7105 - lr: 1.0000e-04 Trial 18 Complete [00h 00m 03s] val_accuracy: 0.7105000019073486 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 15s Search: Running Trial #19 Value |Best Value So Far |Hyperparameter 4 |1 |depth 64 |512 |width linear |relu |activation l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0 |0.1 |dropout rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate 64 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 3s 3ms/step - loss: 0.6764 - accuracy: 0.8179 - val_loss: 0.3778 - val_accuracy: 0.9013 - lr: 1.0000e-04 Epoch 2/2 844/844 [==============================] - 2s 3ms/step - loss: 0.3610 - accuracy: 0.9050 - val_loss: 0.3447 - val_accuracy: 0.9110 - lr: 1.0000e-04 Trial 19 Complete [00h 00m 06s] val_accuracy: 0.9110000133514404 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 21s Search: Running Trial #20 Value |Best Value So Far |Hyperparameter 1 |1 |depth 256 |512 |width relu |relu |activation l1 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.1 |dropout sgd |sgd |optimizer 0.001 |0.01 |learning_rate 32 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 1688/1688 [==============================] - 4s 2ms/step - loss: 0.6626 - accuracy: 0.8967 - val_loss: 0.4262 - val_accuracy: 0.9450 - lr: 0.0010 Epoch 2/2 1688/1688 [==============================] - 4s 2ms/step - loss: 0.4516 - accuracy: 0.9346 - val_loss: 0.3666 - val_accuracy: 0.9583 - lr: 0.0010 Trial 20 Complete [00h 00m 09s] val_accuracy: 0.9583333134651184 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 29s Search: Running Trial #21 Value |Best Value So Far |Hyperparameter 2 |1 |depth 128 |512 |width relu |relu |activation l1 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0 |0.1 |dropout adam |sgd |optimizer 0.001 |0.01 |learning_rate 128 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 422/422 [==============================] - 2s 3ms/step - loss: 0.6660 - accuracy: 0.9020 - val_loss: 0.4585 - val_accuracy: 0.9447 - lr: 0.0010 Epoch 2/2 422/422 [==============================] - 1s 3ms/step - loss: 0.3920 - accuracy: 0.9561 - val_loss: 0.3584 - val_accuracy: 0.9603 - lr: 0.0010 Trial 21 Complete [00h 00m 03s] val_accuracy: 0.9603333473205566 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 33s Search: Running Trial #22 Value |Best Value So Far |Hyperparameter 2 |1 |depth 128 |512 |width sigmoid |relu |activation l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.1 |dropout rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate 64 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 3s 3ms/step - loss: 1.9994 - accuracy: 0.3322 - val_loss: 1.4160 - val_accuracy: 0.7223 - lr: 1.0000e-04 Epoch 2/2 844/844 [==============================] - 2s 3ms/step - loss: 1.1987 - accuracy: 0.6613 - val_loss: 0.8317 - val_accuracy: 0.8228 - lr: 1.0000e-04 Trial 22 Complete [00h 00m 05s] val_accuracy: 0.8228333592414856 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 38s Search: Running Trial #23 Value |Best Value So Far |Hyperparameter 0 |1 |depth 512 |512 |width relu |relu |activation none |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.1 |dropout rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate 1024 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 53/53 [==============================] - 1s 6ms/step - loss: 2.2841 - accuracy: 0.1527 - val_loss: 2.1563 - val_accuracy: 0.2172 - lr: 1.0000e-04 Epoch 2/2 53/53 [==============================] - 0s 4ms/step - loss: 2.0437 - accuracy: 0.3186 - val_loss: 1.9356 - val_accuracy: 0.4192 - lr: 1.0000e-04 Trial 23 Complete [00h 00m 01s] val_accuracy: 0.4191666543483734 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 39s Search: Running Trial #24 Value |Best Value So Far |Hyperparameter 4 |1 |depth 256 |512 |width linear |relu |activation l1_l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.5 |0.1 |dropout sgd |sgd |optimizer 0.001 |0.01 |learning_rate 64 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 4s 4ms/step - loss: 1.0418 - accuracy: 0.7577 - val_loss: 0.5455 - val_accuracy: 0.8970 - lr: 0.0010 Epoch 2/2 844/844 [==============================] - 3s 4ms/step - loss: 0.7273 - accuracy: 0.8354 - val_loss: 0.5421 - val_accuracy: 0.8878 - lr: 0.0010 Trial 24 Complete [00h 00m 08s] val_accuracy: 0.8970000147819519 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 47s Search: Running Trial #25 Value |Best Value So Far |Hyperparameter 2 |1 |depth 128 |512 |width sigmoid |relu |activation none |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.1 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 256 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 1s 4ms/step - loss: 0.4467 - accuracy: 0.8619 - val_loss: 0.1779 - val_accuracy: 0.9470 - lr: 0.0100 Epoch 2/2 211/211 [==============================] - 1s 3ms/step - loss: 0.1701 - accuracy: 0.9483 - val_loss: 0.1286 - val_accuracy: 0.9588 - lr: 0.0100 Trial 25 Complete [00h 00m 02s] val_accuracy: 0.9588333368301392 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 49s Search: Running Trial #26 Value |Best Value So Far |Hyperparameter 2 |1 |depth 256 |512 |width linear |relu |activation l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.1 |dropout sgd |sgd |optimizer 0.0001 |0.01 |learning_rate 512 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 106/106 [==============================] - 1s 6ms/step - loss: 1.1931 - accuracy: 0.6825 - val_loss: 0.6309 - val_accuracy: 0.8582 - lr: 1.0000e-04 Epoch 2/2 106/106 [==============================] - 0s 4ms/step - loss: 0.5496 - accuracy: 0.8705 - val_loss: 0.4618 - val_accuracy: 0.8893 - lr: 1.0000e-04 Trial 26 Complete [00h 00m 02s] val_accuracy: 0.8893333077430725 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 51s Search: Running Trial #27 Value |Best Value So Far |Hyperparameter 1 |1 |depth 512 |512 |width linear |relu |activation l1 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.1 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 256 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 1s 4ms/step - loss: 0.6783 - accuracy: 0.8659 - val_loss: 0.4868 - val_accuracy: 0.8982 - lr: 0.0100 Epoch 2/2 211/211 [==============================] - 1s 3ms/step - loss: 0.5407 - accuracy: 0.8831 - val_loss: 0.5806 - val_accuracy: 0.8770 - lr: 0.0100 Trial 27 Complete [00h 00m 02s] val_accuracy: 0.8981666564941406 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 01m 54s Search: Running Trial #28 Value |Best Value So Far |Hyperparameter 1 |1 |depth 64 |512 |width swish |relu |activation l1 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.1 |dropout rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate 32 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 1688/1688 [==============================] - 4s 2ms/step - loss: 0.5647 - accuracy: 0.9060 - val_loss: 0.4457 - val_accuracy: 0.9480 - lr: 0.0100 Epoch 2/2 1688/1688 [==============================] - 4s 2ms/step - loss: 0.5183 - accuracy: 0.9302 - val_loss: 0.4498 - val_accuracy: 0.9502 - lr: 0.0100 Trial 28 Complete [00h 00m 09s] val_accuracy: 0.950166642665863 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 02m 02s Search: Running Trial #29 Value |Best Value So Far |Hyperparameter 2 |1 |depth 64 |512 |width tanh |relu |activation l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.2 |0.1 |dropout adam |sgd |optimizer 0.001 |0.01 |learning_rate 32 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 1688/1688 [==============================] - 5s 3ms/step - loss: 0.4018 - accuracy: 0.8884 - val_loss: 0.2402 - val_accuracy: 0.9352 - lr: 0.0010 Epoch 2/2 1688/1688 [==============================] - 4s 3ms/step - loss: 0.2600 - accuracy: 0.9302 - val_loss: 0.2025 - val_accuracy: 0.9488 - lr: 0.0010 Trial 29 Complete [00h 00m 10s] val_accuracy: 0.9488333463668823 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 02m 12s Search: Running Trial #30 Value |Best Value So Far |Hyperparameter 1 |1 |depth 512 |512 |width relu |relu |activation l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout sgd |sgd |optimizer 0.001 |0.01 |learning_rate 512 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 106/106 [==============================] - 1s 6ms/step - loss: 0.4931 - accuracy: 0.8601 - val_loss: 0.2380 - val_accuracy: 0.9347 - lr: 0.0010 Epoch 2/2 106/106 [==============================] - 1s 5ms/step - loss: 0.2236 - accuracy: 0.9379 - val_loss: 0.1729 - val_accuracy: 0.9517 - lr: 0.0010 Trial 30 Complete [00h 00m 02s] val_accuracy: 0.9516666531562805 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 02m 14s Search: Running Trial #31 Value |Best Value So Far |Hyperparameter 1 |1 |depth 64 |512 |width swish |relu |activation l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.5 |0.1 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 1024 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 53/53 [==============================] - 1s 7ms/step - loss: 0.6071 - accuracy: 0.8141 - val_loss: 0.2449 - val_accuracy: 0.9295 - lr: 0.0100 Epoch 2/2 53/53 [==============================] - 0s 4ms/step - loss: 0.3094 - accuracy: 0.9109 - val_loss: 0.1966 - val_accuracy: 0.9448 - lr: 0.0100 Trial 31 Complete [00h 00m 01s] val_accuracy: 0.9448333382606506 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 02m 15s Search: Running Trial #32 Value |Best Value So Far |Hyperparameter 4 |1 |depth 256 |512 |width sigmoid |relu |activation l1_l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0 |0.1 |dropout rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate 256 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 2s 5ms/step - loss: 2.5956 - accuracy: 0.1022 - val_loss: 2.3968 - val_accuracy: 0.1055 - lr: 0.0100 Epoch 2/2 211/211 [==============================] - 1s 4ms/step - loss: 2.3946 - accuracy: 0.1002 - val_loss: 2.4026 - val_accuracy: 0.0997 - lr: 0.0100 Trial 32 Complete [00h 00m 03s] val_accuracy: 0.1054999977350235 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 02m 18s Search: Running Trial #33 Value |Best Value So Far |Hyperparameter 0 |1 |depth 256 |512 |width relu |relu |activation l1_l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.1 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 256 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 211/211 [==============================] - 1s 3ms/step - loss: 0.4034 - accuracy: 0.8830 - val_loss: 0.3129 - val_accuracy: 0.9132 - lr: 0.0100 Epoch 2/2 211/211 [==============================] - 1s 2ms/step - loss: 0.2896 - accuracy: 0.9185 - val_loss: 0.3009 - val_accuracy: 0.9178 - lr: 0.0100 Trial 33 Complete [00h 00m 02s] val_accuracy: 0.9178333282470703 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 02m 20s Search: Running Trial #34 Value |Best Value So Far |Hyperparameter 3 |1 |depth 128 |512 |width relu |relu |activation none |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.1 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 64 |256 |batch_size 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 844/844 [==============================] - 3s 3ms/step - loss: 0.4432 - accuracy: 0.8751 - val_loss: 0.2341 - val_accuracy: 0.9368 - lr: 0.0100 Epoch 2/2 844/844 [==============================] - 2s 3ms/step - loss: 0.3442 - accuracy: 0.9108 - val_loss: 0.2245 - val_accuracy: 0.9450 - lr: 0.0100 Trial 34 Complete [00h 00m 06s] val_accuracy: 0.9449999928474426 Best val_accuracy So Far: 0.9670000076293945 Total elapsed time: 00h 02m 26s Search: Running Trial #35 Value |Best Value So Far |Hyperparameter 1 |1 |depth 512 |512 |width relu |relu |activation none |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.1 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 256 |256 |batch_size 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0005 |None |tuner/trial_id Epoch 3/4 211/211 [==============================] - 1s 4ms/step - loss: 0.2509 - accuracy: 0.9239 - val_loss: 0.1221 - val_accuracy: 0.9627 - lr: 0.0100 Epoch 4/4 211/211 [==============================] - 1s 4ms/step - loss: 0.1081 - accuracy: 0.9672 - val_loss: 0.1012 - val_accuracy: 0.9698 - lr: 0.0100 Trial 35 Complete [00h 00m 02s] val_accuracy: 0.9698333144187927 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 02m 28s Search: Running Trial #36 Value |Best Value So Far |Hyperparameter 2 |1 |depth 128 |512 |width relu |relu |activation l1 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0 |0.1 |dropout adam |sgd |optimizer 0.001 |0.01 |learning_rate 128 |256 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0020 |0005 |tuner/trial_id Epoch 3/4 422/422 [==============================] - 2s 3ms/step - loss: 0.6622 - accuracy: 0.9028 - val_loss: 0.4467 - val_accuracy: 0.9443 - lr: 0.0010 Epoch 4/4 422/422 [==============================] - 1s 3ms/step - loss: 0.3914 - accuracy: 0.9552 - val_loss: 0.3575 - val_accuracy: 0.9595 - lr: 0.0010 Trial 36 Complete [00h 00m 03s] val_accuracy: 0.9595000147819519 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 02m 32s Search: Running Trial #37 Value |Best Value So Far |Hyperparameter 4 |1 |depth 128 |512 |width sigmoid |relu |activation l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.1 |dropout rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate 256 |256 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0016 |0005 |tuner/trial_id Epoch 3/4 211/211 [==============================] - 2s 5ms/step - loss: 0.6597 - accuracy: 0.8014 - val_loss: 0.2454 - val_accuracy: 0.9408 - lr: 0.0100 Epoch 4/4 211/211 [==============================] - 1s 4ms/step - loss: 0.2753 - accuracy: 0.9385 - val_loss: 0.2173 - val_accuracy: 0.9558 - lr: 0.0100 Trial 37 Complete [00h 00m 03s] val_accuracy: 0.9558333158493042 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 02m 34s Search: Running Trial #38 Value |Best Value So Far |Hyperparameter 2 |1 |depth 128 |512 |width sigmoid |relu |activation none |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.1 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 256 |256 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0024 |0005 |tuner/trial_id Epoch 3/4 211/211 [==============================] - 1s 4ms/step - loss: 0.4400 - accuracy: 0.8668 - val_loss: 0.1855 - val_accuracy: 0.9450 - lr: 0.0100 Epoch 4/4 211/211 [==============================] - 1s 3ms/step - loss: 0.1717 - accuracy: 0.9481 - val_loss: 0.1231 - val_accuracy: 0.9617 - lr: 0.0100 Trial 38 Complete [00h 00m 02s] val_accuracy: 0.9616666436195374 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 02m 37s Search: Running Trial #39 Value |Best Value So Far |Hyperparameter 1 |1 |depth 256 |512 |width swish |relu |activation l1 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 128 |256 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0002 |0005 |tuner/trial_id Epoch 3/4 422/422 [==============================] - 2s 3ms/step - loss: 0.3995 - accuracy: 0.9088 - val_loss: 0.2842 - val_accuracy: 0.9542 - lr: 0.0100 Epoch 4/4 422/422 [==============================] - 1s 3ms/step - loss: 0.3536 - accuracy: 0.9349 - val_loss: 0.2742 - val_accuracy: 0.9608 - lr: 0.0100 Trial 39 Complete [00h 00m 03s] val_accuracy: 0.9608333110809326 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 02m 40s Search: Running Trial #40 Value |Best Value So Far |Hyperparameter 1 |1 |depth 256 |512 |width relu |relu |activation l1 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.1 |dropout sgd |sgd |optimizer 0.001 |0.01 |learning_rate 32 |256 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0019 |0005 |tuner/trial_id Epoch 3/4 1688/1688 [==============================] - 4s 2ms/step - loss: 0.6621 - accuracy: 0.8952 - val_loss: 0.4308 - val_accuracy: 0.9472 - lr: 0.0010 Epoch 4/4 1688/1688 [==============================] - 4s 2ms/step - loss: 0.4474 - accuracy: 0.9359 - val_loss: 0.3633 - val_accuracy: 0.9602 - lr: 0.0010 Trial 40 Complete [00h 00m 09s] val_accuracy: 0.9601666927337646 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 02m 49s Search: Running Trial #41 Value |Best Value So Far |Hyperparameter 2 |1 |depth 128 |512 |width sigmoid |relu |activation none |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 64 |256 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0001 |0005 |tuner/trial_id Epoch 3/4 844/844 [==============================] - 3s 3ms/step - loss: 0.4001 - accuracy: 0.8765 - val_loss: 0.1746 - val_accuracy: 0.9425 - lr: 0.0100 Epoch 4/4 844/844 [==============================] - 2s 3ms/step - loss: 0.2313 - accuracy: 0.9324 - val_loss: 0.1410 - val_accuracy: 0.9543 - lr: 0.0100 Trial 41 Complete [00h 00m 05s] val_accuracy: 0.9543333053588867 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 02m 54s Search: Running Trial #42 Value |Best Value So Far |Hyperparameter 1 |1 |depth 512 |512 |width relu |relu |activation l2 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout sgd |sgd |optimizer 0.001 |0.01 |learning_rate 512 |256 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0029 |0005 |tuner/trial_id Epoch 3/4 106/106 [==============================] - 1s 6ms/step - loss: 0.4965 - accuracy: 0.8600 - val_loss: 0.2465 - val_accuracy: 0.9345 - lr: 0.0010 Epoch 4/4 106/106 [==============================] - 0s 4ms/step - loss: 0.2254 - accuracy: 0.9373 - val_loss: 0.1803 - val_accuracy: 0.9477 - lr: 0.0010 Trial 42 Complete [00h 00m 02s] val_accuracy: 0.9476666450500488 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 02m 56s Search: Running Trial #43 Value |Best Value So Far |Hyperparameter 1 |1 |depth 64 |512 |width swish |relu |activation l1 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.1 |dropout rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate 32 |256 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0027 |0005 |tuner/trial_id Epoch 3/4 1688/1688 [==============================] - 4s 2ms/step - loss: 0.5593 - accuracy: 0.9079 - val_loss: 0.4575 - val_accuracy: 0.9500 - lr: 0.0100 Epoch 4/4 1688/1688 [==============================] - 4s 2ms/step - loss: 0.5244 - accuracy: 0.9300 - val_loss: 0.4564 - val_accuracy: 0.9495 - lr: 0.0100 Trial 43 Complete [00h 00m 09s] val_accuracy: 0.949999988079071 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 03m 05s Search: Running Trial #44 Value |Best Value So Far |Hyperparameter 2 |1 |depth 64 |512 |width tanh |relu |activation l2 |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.2 |0.1 |dropout adam |sgd |optimizer 0.001 |0.01 |learning_rate 32 |256 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0028 |0005 |tuner/trial_id Epoch 3/4 1688/1688 [==============================] - 5s 3ms/step - loss: 0.4118 - accuracy: 0.8847 - val_loss: 0.2804 - val_accuracy: 0.9245 - lr: 0.0010 Epoch 4/4 1688/1688 [==============================] - 4s 2ms/step - loss: 0.2710 - accuracy: 0.9277 - val_loss: 0.2222 - val_accuracy: 0.9455 - lr: 0.0010 Trial 44 Complete [00h 00m 10s] val_accuracy: 0.9455000162124634 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 03m 14s Search: Running Trial #45 Value |Best Value So Far |Hyperparameter 1 |1 |depth 128 |512 |width relu |relu |activation none |none |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.1 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 32 |256 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0006 |0005 |tuner/trial_id Epoch 3/4 1688/1688 [==============================] - 4s 2ms/step - loss: 0.2836 - accuracy: 0.9153 - val_loss: 0.2265 - val_accuracy: 0.9412 - lr: 0.0100 Epoch 4/4 1688/1688 [==============================] - 4s 2ms/step - loss: 0.2073 - accuracy: 0.9425 - val_loss: 0.2126 - val_accuracy: 0.9453 - lr: 0.0100 Trial 45 Complete [00h 00m 09s] val_accuracy: 0.9453333616256714 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 03m 23s Search: Running Trial #46 Value |Best Value So Far |Hyperparameter 3 |1 |depth 128 |512 |width relu |relu |activation none |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.1 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 64 |256 |batch_size 4 |4 |tuner/epochs 2 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |1 |tuner/round 0033 |0005 |tuner/trial_id Epoch 3/4 844/844 [==============================] - 3s 3ms/step - loss: 0.4402 - accuracy: 0.8759 - val_loss: 0.2814 - val_accuracy: 0.9337 - lr: 0.0100 Epoch 4/4 844/844 [==============================] - 2s 3ms/step - loss: 0.3448 - accuracy: 0.9105 - val_loss: 0.2167 - val_accuracy: 0.9453 - lr: 0.0100 Trial 46 Complete [00h 00m 06s] val_accuracy: 0.9453333616256714 Best val_accuracy So Far: 0.9698333144187927 Total elapsed time: 00h 03m 28s Search: Running Trial #47 Value |Best Value So Far |Hyperparameter 1 |1 |depth 512 |512 |width relu |relu |activation none |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.1 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 256 |256 |batch_size 11 |4 |tuner/epochs 4 |2 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |1 |tuner/round 0034 |0005 |tuner/trial_id Epoch 5/11 211/211 [==============================] - 1s 4ms/step - loss: 0.2365 - accuracy: 0.9273 - val_loss: 0.1457 - val_accuracy: 0.9562 - lr: 0.0100 Epoch 6/11 211/211 [==============================] - 1s 3ms/step - loss: 0.1062 - accuracy: 0.9681 - val_loss: 0.1092 - val_accuracy: 0.9677 - lr: 0.0100 Epoch 7/11 211/211 [==============================] - 1s 3ms/step - loss: 0.0805 - accuracy: 0.9748 - val_loss: 0.1045 - val_accuracy: 0.9702 - lr: 0.0100 Epoch 8/11 211/211 [==============================] - 1s 3ms/step - loss: 0.0752 - accuracy: 0.9770 - val_loss: 0.1113 - val_accuracy: 0.9700 - lr: 0.0100 Epoch 9/11 211/211 [==============================] - 1s 3ms/step - loss: 0.0643 - accuracy: 0.9799 - val_loss: 0.1168 - val_accuracy: 0.9725 - lr: 0.0100 Epoch 10/11 211/211 [==============================] - 1s 3ms/step - loss: 0.0626 - accuracy: 0.9809 - val_loss: 0.1164 - val_accuracy: 0.9718 - lr: 0.0100 Epoch 11/11 211/211 [==============================] - 1s 3ms/step - loss: 0.0581 - accuracy: 0.9823 - val_loss: 0.1168 - val_accuracy: 0.9730 - lr: 0.0100 Trial 47 Complete [00h 00m 06s] val_accuracy: 0.9729999899864197 Best val_accuracy So Far: 0.9729999899864197 Total elapsed time: 00h 03m 34s Search: Running Trial #48 Value |Best Value So Far |Hyperparameter 2 |1 |depth 128 |512 |width sigmoid |relu |activation none |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.1 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 256 |256 |batch_size 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |2 |tuner/round 0037 |0034 |tuner/trial_id Epoch 5/11 211/211 [==============================] - 1s 4ms/step - loss: 0.4340 - accuracy: 0.8676 - val_loss: 0.1810 - val_accuracy: 0.9427 - lr: 0.0100 Epoch 6/11 211/211 [==============================] - 1s 3ms/step - loss: 0.1730 - accuracy: 0.9477 - val_loss: 0.1288 - val_accuracy: 0.9628 - lr: 0.0100 Epoch 7/11 211/211 [==============================] - 1s 3ms/step - loss: 0.1301 - accuracy: 0.9604 - val_loss: 0.0995 - val_accuracy: 0.9695 - lr: 0.0100 Epoch 8/11 211/211 [==============================] - 1s 3ms/step - loss: 0.1073 - accuracy: 0.9665 - val_loss: 0.0949 - val_accuracy: 0.9715 - lr: 0.0100 Epoch 9/11 211/211 [==============================] - 1s 3ms/step - loss: 0.0942 - accuracy: 0.9704 - val_loss: 0.0907 - val_accuracy: 0.9720 - lr: 0.0100 Epoch 10/11 211/211 [==============================] - 1s 3ms/step - loss: 0.0813 - accuracy: 0.9740 - val_loss: 0.0880 - val_accuracy: 0.9737 - lr: 0.0100 Epoch 11/11 211/211 [==============================] - 1s 3ms/step - loss: 0.0735 - accuracy: 0.9767 - val_loss: 0.0852 - val_accuracy: 0.9745 - lr: 0.0100 Trial 48 Complete [00h 00m 05s] val_accuracy: 0.9745000004768372 Best val_accuracy So Far: 0.9745000004768372 Total elapsed time: 00h 03m 39s Search: Running Trial #49 Value |Best Value So Far |Hyperparameter 1 |2 |depth 256 |128 |width swish |sigmoid |activation l1 |none |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.2 |dropout sgd |adam |optimizer 0.01 |0.01 |learning_rate 128 |256 |batch_size 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |2 |tuner/round 0038 |0037 |tuner/trial_id Epoch 5/11 422/422 [==============================] - 2s 3ms/step - loss: 0.3959 - accuracy: 0.9083 - val_loss: 0.2938 - val_accuracy: 0.9492 - lr: 0.0100 Epoch 6/11 422/422 [==============================] - 1s 2ms/step - loss: 0.3588 - accuracy: 0.9333 - val_loss: 0.2848 - val_accuracy: 0.9590 - lr: 0.0100 Epoch 7/11 422/422 [==============================] - 1s 2ms/step - loss: 0.3497 - accuracy: 0.9401 - val_loss: 0.2797 - val_accuracy: 0.9670 - lr: 0.0100 Epoch 8/11 422/422 [==============================] - 1s 3ms/step - loss: 0.3518 - accuracy: 0.9431 - val_loss: 0.2801 - val_accuracy: 0.9685 - lr: 0.0100 Epoch 9/11 422/422 [==============================] - 1s 2ms/step - loss: 0.3556 - accuracy: 0.9439 - val_loss: 0.2955 - val_accuracy: 0.9645 - lr: 0.0100 Epoch 10/11 422/422 [==============================] - 1s 2ms/step - loss: 0.3421 - accuracy: 0.9491 - val_loss: 0.2869 - val_accuracy: 0.9655 - lr: 0.0100 Epoch 11/11 422/422 [==============================] - 1s 3ms/step - loss: 0.2683 - accuracy: 0.9674 - val_loss: 0.2429 - val_accuracy: 0.9750 - lr: 1.0000e-03 Trial 49 Complete [00h 00m 08s] val_accuracy: 0.9750000238418579 Best val_accuracy So Far: 0.9750000238418579 Total elapsed time: 00h 03m 48s Search: Running Trial #50 Value |Best Value So Far |Hyperparameter 1 |1 |depth 256 |256 |width relu |swish |activation l1 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.4 |dropout sgd |sgd |optimizer 0.001 |0.01 |learning_rate 32 |128 |batch_size 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |2 |tuner/round 0039 |0038 |tuner/trial_id Epoch 5/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.6656 - accuracy: 0.8961 - val_loss: 0.4233 - val_accuracy: 0.9507 - lr: 0.0010 Epoch 6/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.4508 - accuracy: 0.9357 - val_loss: 0.3692 - val_accuracy: 0.9578 - lr: 0.0010 Epoch 7/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.4145 - accuracy: 0.9430 - val_loss: 0.3566 - val_accuracy: 0.9635 - lr: 0.0010 Epoch 8/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3949 - accuracy: 0.9477 - val_loss: 0.3406 - val_accuracy: 0.9623 - lr: 0.0010 Epoch 9/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3830 - accuracy: 0.9495 - val_loss: 0.3245 - val_accuracy: 0.9682 - lr: 0.0010 Epoch 10/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3769 - accuracy: 0.9521 - val_loss: 0.3261 - val_accuracy: 0.9652 - lr: 0.0010 Epoch 11/11 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3732 - accuracy: 0.9528 - val_loss: 0.3158 - val_accuracy: 0.9710 - lr: 0.0010 Trial 50 Complete [00h 00m 28s] val_accuracy: 0.9710000157356262 Best val_accuracy So Far: 0.9750000238418579 Total elapsed time: 00h 04m 16s Search: Running Trial #51 Value |Best Value So Far |Hyperparameter 1 |1 |depth 256 |256 |width swish |swish |activation l1 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.4 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 128 |128 |batch_size 32 |11 |tuner/epochs 11 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 3 |2 |tuner/round 0048 |0038 |tuner/trial_id Epoch 12/32 422/422 [==============================] - 2s 3ms/step - loss: 0.3958 - accuracy: 0.9100 - val_loss: 0.2785 - val_accuracy: 0.9525 - lr: 0.0100 Epoch 13/32 422/422 [==============================] - 1s 2ms/step - loss: 0.3517 - accuracy: 0.9359 - val_loss: 0.2837 - val_accuracy: 0.9628 - lr: 0.0100 Epoch 14/32 422/422 [==============================] - 1s 3ms/step - loss: 0.3504 - accuracy: 0.9402 - val_loss: 0.2828 - val_accuracy: 0.9657 - lr: 0.0100 Epoch 15/32 422/422 [==============================] - 1s 3ms/step - loss: 0.3445 - accuracy: 0.9454 - val_loss: 0.2786 - val_accuracy: 0.9673 - lr: 0.0100 Epoch 16/32 422/422 [==============================] - 1s 3ms/step - loss: 0.3427 - accuracy: 0.9470 - val_loss: 0.3110 - val_accuracy: 0.9638 - lr: 0.0100 Epoch 17/32 422/422 [==============================] - 1s 3ms/step - loss: 0.3571 - accuracy: 0.9458 - val_loss: 0.3137 - val_accuracy: 0.9683 - lr: 0.0100 Epoch 18/32 422/422 [==============================] - 1s 2ms/step - loss: 0.3543 - accuracy: 0.9499 - val_loss: 0.2933 - val_accuracy: 0.9657 - lr: 0.0100 Epoch 19/32 422/422 [==============================] - 1s 3ms/step - loss: 0.3505 - accuracy: 0.9497 - val_loss: 0.3018 - val_accuracy: 0.9700 - lr: 0.0100 Epoch 20/32 422/422 [==============================] - 1s 3ms/step - loss: 0.3494 - accuracy: 0.9520 - val_loss: 0.3092 - val_accuracy: 0.9688 - lr: 0.0100 Epoch 21/32 422/422 [==============================] - 1s 3ms/step - loss: 0.3632 - accuracy: 0.9475 - val_loss: 0.2967 - val_accuracy: 0.9708 - lr: 0.0100 Epoch 22/32 422/422 [==============================] - 1s 3ms/step - loss: 0.3429 - accuracy: 0.9535 - val_loss: 0.2986 - val_accuracy: 0.9690 - lr: 0.0100 Epoch 23/32 422/422 [==============================] - 1s 2ms/step - loss: 0.3558 - accuracy: 0.9512 - val_loss: 0.3081 - val_accuracy: 0.9693 - lr: 0.0100 Epoch 24/32 422/422 [==============================] - 1s 3ms/step - loss: 0.2841 - accuracy: 0.9681 - val_loss: 0.2670 - val_accuracy: 0.9763 - lr: 1.0000e-03 Epoch 25/32 422/422 [==============================] - 1s 3ms/step - loss: 0.2493 - accuracy: 0.9753 - val_loss: 0.2520 - val_accuracy: 0.9777 - lr: 1.0000e-03 Epoch 26/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2343 - accuracy: 0.9768 - val_loss: 0.2396 - val_accuracy: 0.9767 - lr: 1.0000e-03 Epoch 27/32 422/422 [==============================] - 1s 3ms/step - loss: 0.2172 - accuracy: 0.9788 - val_loss: 0.2299 - val_accuracy: 0.9793 - lr: 1.0000e-03 Epoch 28/32 422/422 [==============================] - 1s 3ms/step - loss: 0.2052 - accuracy: 0.9808 - val_loss: 0.2224 - val_accuracy: 0.9782 - lr: 1.0000e-03 Epoch 29/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1945 - accuracy: 0.9813 - val_loss: 0.2136 - val_accuracy: 0.9793 - lr: 1.0000e-03 Epoch 30/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1867 - accuracy: 0.9828 - val_loss: 0.2113 - val_accuracy: 0.9793 - lr: 1.0000e-04 Epoch 31/32 422/422 [==============================] - 1s 2ms/step - loss: 0.1824 - accuracy: 0.9834 - val_loss: 0.2103 - val_accuracy: 0.9788 - lr: 1.0000e-04 Epoch 32/32 422/422 [==============================] - 1s 2ms/step - loss: 0.1803 - accuracy: 0.9842 - val_loss: 0.2101 - val_accuracy: 0.9790 - lr: 1.0000e-05 Trial 51 Complete [00h 00m 23s] val_accuracy: 0.9793333411216736 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 04m 39s Search: Running Trial #52 Value |Best Value So Far |Hyperparameter 2 |1 |depth 128 |256 |width sigmoid |swish |activation none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.4 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 256 |128 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 3 |3 |tuner/bracket 3 |3 |tuner/round 0047 |0048 |tuner/trial_id Epoch 12/32 211/211 [==============================] - 1s 4ms/step - loss: 0.4343 - accuracy: 0.8679 - val_loss: 0.1846 - val_accuracy: 0.9468 - lr: 0.0100 Epoch 13/32 211/211 [==============================] - 1s 3ms/step - loss: 0.1730 - accuracy: 0.9486 - val_loss: 0.1370 - val_accuracy: 0.9577 - lr: 0.0100 Epoch 14/32 211/211 [==============================] - 1s 3ms/step - loss: 0.1285 - accuracy: 0.9611 - val_loss: 0.1110 - val_accuracy: 0.9653 - lr: 0.0100 Epoch 15/32 211/211 [==============================] - 1s 3ms/step - loss: 0.1055 - accuracy: 0.9675 - val_loss: 0.1033 - val_accuracy: 0.9667 - lr: 0.0100 Epoch 16/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0956 - accuracy: 0.9697 - val_loss: 0.0920 - val_accuracy: 0.9715 - lr: 0.0100 Epoch 17/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0822 - accuracy: 0.9736 - val_loss: 0.0956 - val_accuracy: 0.9707 - lr: 0.0100 Epoch 18/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0745 - accuracy: 0.9769 - val_loss: 0.0954 - val_accuracy: 0.9708 - lr: 0.0100 Epoch 19/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0528 - accuracy: 0.9829 - val_loss: 0.0769 - val_accuracy: 0.9767 - lr: 1.0000e-03 Epoch 20/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0462 - accuracy: 0.9859 - val_loss: 0.0774 - val_accuracy: 0.9768 - lr: 1.0000e-03 Epoch 21/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0429 - accuracy: 0.9864 - val_loss: 0.0763 - val_accuracy: 0.9770 - lr: 1.0000e-03 Epoch 22/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0417 - accuracy: 0.9869 - val_loss: 0.0758 - val_accuracy: 0.9765 - lr: 1.0000e-03 Epoch 23/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0392 - accuracy: 0.9870 - val_loss: 0.0776 - val_accuracy: 0.9772 - lr: 1.0000e-03 Epoch 24/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0377 - accuracy: 0.9884 - val_loss: 0.0761 - val_accuracy: 0.9763 - lr: 1.0000e-03 Epoch 25/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0379 - accuracy: 0.9876 - val_loss: 0.0759 - val_accuracy: 0.9770 - lr: 1.0000e-03 Epoch 26/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0348 - accuracy: 0.9890 - val_loss: 0.0751 - val_accuracy: 0.9768 - lr: 1.0000e-04 Epoch 27/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0328 - accuracy: 0.9901 - val_loss: 0.0746 - val_accuracy: 0.9763 - lr: 1.0000e-04 Epoch 28/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0329 - accuracy: 0.9894 - val_loss: 0.0747 - val_accuracy: 0.9763 - lr: 1.0000e-05 Epoch 29/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0336 - accuracy: 0.9893 - val_loss: 0.0747 - val_accuracy: 0.9767 - lr: 1.0000e-05 Epoch 30/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0346 - accuracy: 0.9886 - val_loss: 0.0747 - val_accuracy: 0.9765 - lr: 1.0000e-06 Epoch 31/32 211/211 [==============================] - 1s 3ms/step - loss: 0.0328 - accuracy: 0.9899 - val_loss: 0.0747 - val_accuracy: 0.9765 - lr: 1.0000e-06 Trial 52 Complete [00h 00m 13s] val_accuracy: 0.9771666526794434 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 04m 52s Search: Running Trial #53 Value |Best Value So Far |Hyperparameter 1 |1 |depth 512 |256 |width swish |swish |activation l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.4 |dropout rmsprop |sgd |optimizer 0.001 |0.01 |learning_rate 1024 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 53/53 [==============================] - 1s 9ms/step - loss: 1.7189 - accuracy: 0.8086 - val_loss: 1.0401 - val_accuracy: 0.9032 - lr: 0.0010 Epoch 2/4 53/53 [==============================] - 0s 6ms/step - loss: 0.8971 - accuracy: 0.9015 - val_loss: 0.7500 - val_accuracy: 0.9152 - lr: 0.0010 Epoch 3/4 53/53 [==============================] - 0s 6ms/step - loss: 0.7029 - accuracy: 0.9102 - val_loss: 0.6223 - val_accuracy: 0.9182 - lr: 0.0010 Epoch 4/4 53/53 [==============================] - 0s 6ms/step - loss: 0.6019 - accuracy: 0.9149 - val_loss: 0.5423 - val_accuracy: 0.9230 - lr: 0.0010 Trial 53 Complete [00h 00m 02s] val_accuracy: 0.9229999780654907 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 04m 54s Search: Running Trial #54 Value |Best Value So Far |Hyperparameter 3 |1 |depth 128 |256 |width sigmoid |swish |activation none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.4 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 128 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 422/422 [==============================] - 2s 3ms/step - loss: 0.4385 - accuracy: 0.8641 - val_loss: 0.1633 - val_accuracy: 0.9528 - lr: 0.0100 Epoch 2/4 422/422 [==============================] - 1s 3ms/step - loss: 0.1729 - accuracy: 0.9493 - val_loss: 0.1209 - val_accuracy: 0.9652 - lr: 0.0100 Epoch 3/4 422/422 [==============================] - 1s 3ms/step - loss: 0.1400 - accuracy: 0.9588 - val_loss: 0.1132 - val_accuracy: 0.9682 - lr: 0.0100 Epoch 4/4 422/422 [==============================] - 1s 3ms/step - loss: 0.1240 - accuracy: 0.9635 - val_loss: 0.1025 - val_accuracy: 0.9697 - lr: 0.0100 Trial 54 Complete [00h 00m 06s] val_accuracy: 0.9696666598320007 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 05m 00s Search: Running Trial #55 Value |Best Value So Far |Hyperparameter 3 |1 |depth 128 |256 |width linear |swish |activation none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout rmsprop |sgd |optimizer 0.001 |0.01 |learning_rate 128 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 422/422 [==============================] - 2s 3ms/step - loss: 0.4263 - accuracy: 0.8743 - val_loss: 0.3304 - val_accuracy: 0.9105 - lr: 0.0010 Epoch 2/4 422/422 [==============================] - 1s 3ms/step - loss: 0.3369 - accuracy: 0.9034 - val_loss: 0.3225 - val_accuracy: 0.9108 - lr: 0.0010 Epoch 3/4 422/422 [==============================] - 1s 3ms/step - loss: 0.3235 - accuracy: 0.9072 - val_loss: 0.3173 - val_accuracy: 0.9168 - lr: 0.0010 Epoch 4/4 422/422 [==============================] - 1s 3ms/step - loss: 0.3165 - accuracy: 0.9107 - val_loss: 0.3075 - val_accuracy: 0.9185 - lr: 0.0010 Trial 55 Complete [00h 00m 06s] val_accuracy: 0.9185000061988831 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 05m 06s Search: Running Trial #56 Value |Best Value So Far |Hyperparameter 1 |1 |depth 128 |256 |width linear |swish |activation l1 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.4 |dropout rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate 1024 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 53/53 [==============================] - 1s 7ms/step - loss: 1.9725 - accuracy: 0.3509 - val_loss: 1.4923 - val_accuracy: 0.6685 - lr: 1.0000e-04 Epoch 2/4 53/53 [==============================] - 0s 4ms/step - loss: 1.2802 - accuracy: 0.6924 - val_loss: 1.0434 - val_accuracy: 0.7858 - lr: 1.0000e-04 Epoch 3/4 53/53 [==============================] - 0s 4ms/step - loss: 0.9585 - accuracy: 0.7805 - val_loss: 0.8231 - val_accuracy: 0.8247 - lr: 1.0000e-04 Epoch 4/4 53/53 [==============================] - 0s 4ms/step - loss: 0.7852 - accuracy: 0.8187 - val_loss: 0.6976 - val_accuracy: 0.8473 - lr: 1.0000e-04 Trial 56 Complete [00h 00m 02s] val_accuracy: 0.8473333120346069 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 05m 08s Search: Running Trial #57 Value |Best Value So Far |Hyperparameter 1 |1 |depth 128 |256 |width relu |swish |activation none |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.4 |dropout adam |sgd |optimizer 0.001 |0.01 |learning_rate 128 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 422/422 [==============================] - 1s 3ms/step - loss: 0.4825 - accuracy: 0.8581 - val_loss: 0.2305 - val_accuracy: 0.9370 - lr: 0.0010 Epoch 2/4 422/422 [==============================] - 1s 2ms/step - loss: 0.2448 - accuracy: 0.9293 - val_loss: 0.1742 - val_accuracy: 0.9507 - lr: 0.0010 Epoch 3/4 422/422 [==============================] - 1s 2ms/step - loss: 0.1936 - accuracy: 0.9441 - val_loss: 0.1426 - val_accuracy: 0.9592 - lr: 0.0010 Epoch 4/4 422/422 [==============================] - 1s 2ms/step - loss: 0.1646 - accuracy: 0.9522 - val_loss: 0.1263 - val_accuracy: 0.9627 - lr: 0.0010 Trial 57 Complete [00h 00m 05s] val_accuracy: 0.9626666903495789 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 05m 13s Search: Running Trial #58 Value |Best Value So Far |Hyperparameter 0 |1 |depth 512 |256 |width linear |swish |activation l1 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.4 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 512 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 106/106 [==============================] - 1s 4ms/step - loss: 0.4743 - accuracy: 0.8627 - val_loss: 0.3238 - val_accuracy: 0.9105 - lr: 0.0100 Epoch 2/4 106/106 [==============================] - 0s 3ms/step - loss: 0.2996 - accuracy: 0.9163 - val_loss: 0.3020 - val_accuracy: 0.9155 - lr: 0.0100 Epoch 3/4 106/106 [==============================] - 0s 3ms/step - loss: 0.2798 - accuracy: 0.9216 - val_loss: 0.2938 - val_accuracy: 0.9170 - lr: 0.0100 Epoch 4/4 106/106 [==============================] - 0s 3ms/step - loss: 0.2698 - accuracy: 0.9253 - val_loss: 0.2944 - val_accuracy: 0.9180 - lr: 0.0100 Trial 58 Complete [00h 00m 02s] val_accuracy: 0.9179999828338623 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 05m 15s Search: Running Trial #59 Value |Best Value So Far |Hyperparameter 0 |1 |depth 128 |256 |width linear |swish |activation l1_l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.4 |dropout sgd |sgd |optimizer 0.0001 |0.01 |learning_rate 512 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 106/106 [==============================] - 1s 4ms/step - loss: 2.1838 - accuracy: 0.1930 - val_loss: 1.9579 - val_accuracy: 0.3852 - lr: 1.0000e-04 Epoch 2/4 106/106 [==============================] - 0s 3ms/step - loss: 1.7806 - accuracy: 0.5298 - val_loss: 1.6228 - val_accuracy: 0.6257 - lr: 1.0000e-04 Epoch 3/4 106/106 [==============================] - 0s 3ms/step - loss: 1.4935 - accuracy: 0.6798 - val_loss: 1.3776 - val_accuracy: 0.7152 - lr: 1.0000e-04 Epoch 4/4 106/106 [==============================] - 0s 3ms/step - loss: 1.2827 - accuracy: 0.7438 - val_loss: 1.1965 - val_accuracy: 0.7595 - lr: 1.0000e-04 Trial 59 Complete [00h 00m 02s] val_accuracy: 0.7595000267028809 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 05m 17s Search: Running Trial #60 Value |Best Value So Far |Hyperparameter 3 |1 |depth 512 |256 |width swish |swish |activation l1_l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.4 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 512 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 106/106 [==============================] - 2s 9ms/step - loss: 1.0695 - accuracy: 0.8449 - val_loss: 0.5511 - val_accuracy: 0.9518 - lr: 0.0100 Epoch 2/4 106/106 [==============================] - 1s 7ms/step - loss: 0.5146 - accuracy: 0.9547 - val_loss: 0.4410 - val_accuracy: 0.9655 - lr: 0.0100 Epoch 3/4 106/106 [==============================] - 1s 6ms/step - loss: 0.4241 - accuracy: 0.9637 - val_loss: 0.4060 - val_accuracy: 0.9652 - lr: 0.0100 Epoch 4/4 106/106 [==============================] - 1s 6ms/step - loss: 0.3770 - accuracy: 0.9655 - val_loss: 0.3479 - val_accuracy: 0.9705 - lr: 0.0100 Trial 60 Complete [00h 00m 04s] val_accuracy: 0.9704999923706055 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 05m 21s Search: Running Trial #61 Value |Best Value So Far |Hyperparameter 2 |1 |depth 128 |256 |width linear |swish |activation l1 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.4 |dropout rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate 64 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 844/844 [==============================] - 3s 3ms/step - loss: 1.2295 - accuracy: 0.7655 - val_loss: 1.4307 - val_accuracy: 0.7970 - lr: 0.0100 Epoch 2/4 844/844 [==============================] - 2s 2ms/step - loss: 1.5945 - accuracy: 0.7805 - val_loss: 0.6511 - val_accuracy: 0.8868 - lr: 0.0100 Epoch 3/4 844/844 [==============================] - 2s 2ms/step - loss: 0.9880 - accuracy: 0.8047 - val_loss: 0.8767 - val_accuracy: 0.8515 - lr: 0.0100 Epoch 4/4 844/844 [==============================] - 2s 2ms/step - loss: 2.4502 - accuracy: 0.7604 - val_loss: 2.2386 - val_accuracy: 0.8277 - lr: 0.0100 Trial 61 Complete [00h 00m 09s] val_accuracy: 0.8868333101272583 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 05m 30s Search: Running Trial #62 Value |Best Value So Far |Hyperparameter 0 |1 |depth 64 |256 |width sigmoid |swish |activation l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.4 |dropout sgd |sgd |optimizer 0.0001 |0.01 |learning_rate 1024 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 53/53 [==============================] - 1s 6ms/step - loss: 2.2443 - accuracy: 0.1484 - val_loss: 2.1107 - val_accuracy: 0.2570 - lr: 1.0000e-04 Epoch 2/4 53/53 [==============================] - 0s 4ms/step - loss: 2.0009 - accuracy: 0.3652 - val_loss: 1.8945 - val_accuracy: 0.4732 - lr: 1.0000e-04 Epoch 3/4 53/53 [==============================] - 0s 4ms/step - loss: 1.8030 - accuracy: 0.5374 - val_loss: 1.7131 - val_accuracy: 0.6027 - lr: 1.0000e-04 Epoch 4/4 53/53 [==============================] - 0s 4ms/step - loss: 1.6360 - accuracy: 0.6421 - val_loss: 1.5589 - val_accuracy: 0.6822 - lr: 1.0000e-04 Trial 62 Complete [00h 00m 02s] val_accuracy: 0.6821666955947876 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 05m 32s Search: Running Trial #63 Value |Best Value So Far |Hyperparameter 4 |1 |depth 64 |256 |width relu |swish |activation l1 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.4 |dropout rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate 32 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 1688/1688 [==============================] - 5s 3ms/step - loss: 1.1781 - accuracy: 0.7603 - val_loss: 0.7978 - val_accuracy: 0.8993 - lr: 0.0100 Epoch 2/4 1688/1688 [==============================] - 5s 3ms/step - loss: 1.1378 - accuracy: 0.8135 - val_loss: 0.8439 - val_accuracy: 0.9010 - lr: 0.0100 Epoch 3/4 1688/1688 [==============================] - 5s 3ms/step - loss: 1.1710 - accuracy: 0.8108 - val_loss: 0.8520 - val_accuracy: 0.9052 - lr: 0.0100 Epoch 4/4 1688/1688 [==============================] - 5s 3ms/step - loss: 1.1510 - accuracy: 0.8141 - val_loss: 0.8714 - val_accuracy: 0.8787 - lr: 0.0100 Trial 63 Complete [00h 00m 20s] val_accuracy: 0.9051666855812073 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 05m 52s Search: Running Trial #64 Value |Best Value So Far |Hyperparameter 0 |1 |depth 512 |256 |width tanh |swish |activation l1_l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.4 |dropout adam |sgd |optimizer 0.0001 |0.01 |learning_rate 32 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 1688/1688 [==============================] - 4s 2ms/step - loss: 1.1910 - accuracy: 0.7131 - val_loss: 0.7070 - val_accuracy: 0.8470 - lr: 1.0000e-04 Epoch 2/4 1688/1688 [==============================] - 3s 2ms/step - loss: 0.5858 - accuracy: 0.8632 - val_loss: 0.5009 - val_accuracy: 0.8803 - lr: 1.0000e-04 Epoch 3/4 1688/1688 [==============================] - 3s 2ms/step - loss: 0.4584 - accuracy: 0.8843 - val_loss: 0.4257 - val_accuracy: 0.8927 - lr: 1.0000e-04 Epoch 4/4 1688/1688 [==============================] - 3s 2ms/step - loss: 0.4025 - accuracy: 0.8942 - val_loss: 0.3872 - val_accuracy: 0.9008 - lr: 1.0000e-04 Trial 64 Complete [00h 00m 15s] val_accuracy: 0.9008333086967468 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 06m 07s Search: Running Trial #65 Value |Best Value So Far |Hyperparameter 0 |1 |depth 256 |256 |width swish |swish |activation l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.4 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 32 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3732 - accuracy: 0.8954 - val_loss: 0.3548 - val_accuracy: 0.9088 - lr: 0.0100 Epoch 2/4 1688/1688 [==============================] - 3s 2ms/step - loss: 0.3415 - accuracy: 0.9081 - val_loss: 0.3719 - val_accuracy: 0.9053 - lr: 0.0100 Epoch 3/4 1688/1688 [==============================] - 3s 2ms/step - loss: 0.3290 - accuracy: 0.9123 - val_loss: 0.3706 - val_accuracy: 0.9120 - lr: 0.0100 Epoch 4/4 1688/1688 [==============================] - 3s 2ms/step - loss: 0.3243 - accuracy: 0.9134 - val_loss: 0.3919 - val_accuracy: 0.9050 - lr: 0.0100 Trial 65 Complete [00h 00m 15s] val_accuracy: 0.9120000004768372 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 06m 21s Search: Running Trial #66 Value |Best Value So Far |Hyperparameter 4 |1 |depth 128 |256 |width sigmoid |swish |activation l1 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.4 |dropout adam |sgd |optimizer 0.0001 |0.01 |learning_rate 32 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 1688/1688 [==============================] - 6s 3ms/step - loss: 2.1195 - accuracy: 0.2607 - val_loss: 1.5155 - val_accuracy: 0.5533 - lr: 1.0000e-04 Epoch 2/4 1688/1688 [==============================] - 5s 3ms/step - loss: 1.1691 - accuracy: 0.6409 - val_loss: 0.8835 - val_accuracy: 0.7517 - lr: 1.0000e-04 Epoch 3/4 1688/1688 [==============================] - 5s 3ms/step - loss: 0.8185 - accuracy: 0.7759 - val_loss: 0.6531 - val_accuracy: 0.8430 - lr: 1.0000e-04 Epoch 4/4 1688/1688 [==============================] - 5s 3ms/step - loss: 0.6481 - accuracy: 0.8352 - val_loss: 0.5291 - val_accuracy: 0.8773 - lr: 1.0000e-04 Trial 66 Complete [00h 00m 22s] val_accuracy: 0.8773333430290222 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 06m 44s Search: Running Trial #67 Value |Best Value So Far |Hyperparameter 0 |1 |depth 256 |256 |width swish |swish |activation l1 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.4 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 256 |128 |batch_size 4 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |3 |tuner/round Epoch 1/4 211/211 [==============================] - 1s 3ms/step - loss: 0.4000 - accuracy: 0.8853 - val_loss: 0.3109 - val_accuracy: 0.9122 - lr: 0.0100 Epoch 2/4 211/211 [==============================] - 1s 2ms/step - loss: 0.2895 - accuracy: 0.9184 - val_loss: 0.3083 - val_accuracy: 0.9180 - lr: 0.0100 Epoch 3/4 211/211 [==============================] - 0s 2ms/step - loss: 0.2739 - accuracy: 0.9225 - val_loss: 0.3079 - val_accuracy: 0.9175 - lr: 0.0100 Epoch 4/4 211/211 [==============================] - 1s 2ms/step - loss: 0.2680 - accuracy: 0.9255 - val_loss: 0.3021 - val_accuracy: 0.9200 - lr: 0.0100 Trial 67 Complete [00h 00m 03s] val_accuracy: 0.9200000166893005 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 06m 47s Search: Running Trial #68 Value |Best Value So Far |Hyperparameter 3 |1 |depth 512 |256 |width swish |swish |activation l1_l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.4 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 512 |128 |batch_size 11 |32 |tuner/epochs 4 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 1 |3 |tuner/round 0059 |0048 |tuner/trial_id Epoch 5/11 106/106 [==============================] - 2s 9ms/step - loss: 1.1842 - accuracy: 0.8429 - val_loss: 0.5744 - val_accuracy: 0.9500 - lr: 0.0100 Epoch 6/11 106/106 [==============================] - 1s 7ms/step - loss: 0.5356 - accuracy: 0.9533 - val_loss: 0.4727 - val_accuracy: 0.9607 - lr: 0.0100 Epoch 7/11 106/106 [==============================] - 1s 7ms/step - loss: 0.4512 - accuracy: 0.9614 - val_loss: 0.4084 - val_accuracy: 0.9667 - lr: 0.0100 Epoch 8/11 106/106 [==============================] - 1s 7ms/step - loss: 0.3873 - accuracy: 0.9670 - val_loss: 0.3698 - val_accuracy: 0.9693 - lr: 0.0100 Epoch 9/11 106/106 [==============================] - 1s 7ms/step - loss: 0.3431 - accuracy: 0.9700 - val_loss: 0.3325 - val_accuracy: 0.9728 - lr: 0.0100 Epoch 10/11 106/106 [==============================] - 1s 6ms/step - loss: 0.3164 - accuracy: 0.9719 - val_loss: 0.3076 - val_accuracy: 0.9750 - lr: 0.0100 Epoch 11/11 106/106 [==============================] - 1s 7ms/step - loss: 0.2919 - accuracy: 0.9737 - val_loss: 0.2845 - val_accuracy: 0.9757 - lr: 0.0100 Trial 68 Complete [00h 00m 06s] val_accuracy: 0.9756666421890259 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 06m 53s Search: Running Trial #69 Value |Best Value So Far |Hyperparameter 3 |1 |depth 128 |256 |width sigmoid |swish |activation none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.4 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 128 |128 |batch_size 11 |32 |tuner/epochs 4 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 1 |3 |tuner/round 0053 |0048 |tuner/trial_id Epoch 5/11 422/422 [==============================] - 2s 3ms/step - loss: 0.4487 - accuracy: 0.8581 - val_loss: 0.1615 - val_accuracy: 0.9503 - lr: 0.0100 Epoch 6/11 422/422 [==============================] - 1s 3ms/step - loss: 0.1767 - accuracy: 0.9475 - val_loss: 0.1305 - val_accuracy: 0.9622 - lr: 0.0100 Epoch 7/11 422/422 [==============================] - 1s 3ms/step - loss: 0.1442 - accuracy: 0.9576 - val_loss: 0.1076 - val_accuracy: 0.9678 - lr: 0.0100 Epoch 8/11 422/422 [==============================] - 1s 3ms/step - loss: 0.1228 - accuracy: 0.9639 - val_loss: 0.1008 - val_accuracy: 0.9698 - lr: 0.0100 Epoch 9/11 422/422 [==============================] - 1s 3ms/step - loss: 0.1098 - accuracy: 0.9664 - val_loss: 0.0988 - val_accuracy: 0.9682 - lr: 0.0100 Epoch 10/11 422/422 [==============================] - 1s 3ms/step - loss: 0.0997 - accuracy: 0.9708 - val_loss: 0.1006 - val_accuracy: 0.9675 - lr: 0.0100 Epoch 11/11 422/422 [==============================] - 1s 3ms/step - loss: 0.0697 - accuracy: 0.9793 - val_loss: 0.0798 - val_accuracy: 0.9777 - lr: 1.0000e-03 Trial 69 Complete [00h 00m 09s] val_accuracy: 0.9776666760444641 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 07m 02s Search: Running Trial #70 Value |Best Value So Far |Hyperparameter 1 |1 |depth 128 |256 |width relu |swish |activation none |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.4 |dropout adam |sgd |optimizer 0.001 |0.01 |learning_rate 128 |128 |batch_size 11 |32 |tuner/epochs 4 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 1 |3 |tuner/round 0056 |0048 |tuner/trial_id Epoch 5/11 422/422 [==============================] - 2s 3ms/step - loss: 0.4779 - accuracy: 0.8588 - val_loss: 0.2287 - val_accuracy: 0.9382 - lr: 0.0010 Epoch 6/11 422/422 [==============================] - 1s 2ms/step - loss: 0.2447 - accuracy: 0.9293 - val_loss: 0.1709 - val_accuracy: 0.9515 - lr: 0.0010 Epoch 7/11 422/422 [==============================] - 1s 2ms/step - loss: 0.1947 - accuracy: 0.9431 - val_loss: 0.1458 - val_accuracy: 0.9563 - lr: 0.0010 Epoch 8/11 422/422 [==============================] - 1s 3ms/step - loss: 0.1666 - accuracy: 0.9507 - val_loss: 0.1267 - val_accuracy: 0.9617 - lr: 0.0010 Epoch 9/11 422/422 [==============================] - 1s 2ms/step - loss: 0.1450 - accuracy: 0.9576 - val_loss: 0.1141 - val_accuracy: 0.9662 - lr: 0.0010 Epoch 10/11 422/422 [==============================] - 1s 2ms/step - loss: 0.1297 - accuracy: 0.9612 - val_loss: 0.1057 - val_accuracy: 0.9692 - lr: 0.0010 Epoch 11/11 422/422 [==============================] - 1s 2ms/step - loss: 0.1199 - accuracy: 0.9640 - val_loss: 0.1006 - val_accuracy: 0.9695 - lr: 0.0010 Trial 70 Complete [00h 00m 08s] val_accuracy: 0.9695000052452087 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 07m 11s Search: Running Trial #71 Value |Best Value So Far |Hyperparameter 1 |1 |depth 512 |256 |width swish |swish |activation l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.4 |dropout rmsprop |sgd |optimizer 0.001 |0.01 |learning_rate 1024 |128 |batch_size 11 |32 |tuner/epochs 4 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 1 |3 |tuner/round 0052 |0048 |tuner/trial_id Epoch 5/11 53/53 [==============================] - 1s 9ms/step - loss: 1.7257 - accuracy: 0.8085 - val_loss: 1.0447 - val_accuracy: 0.9038 - lr: 0.0010 Epoch 6/11 53/53 [==============================] - 0s 6ms/step - loss: 0.9003 - accuracy: 0.9027 - val_loss: 0.7559 - val_accuracy: 0.9143 - lr: 0.0010 Epoch 7/11 53/53 [==============================] - 0s 6ms/step - loss: 0.7070 - accuracy: 0.9106 - val_loss: 0.6233 - val_accuracy: 0.9200 - lr: 0.0010 Epoch 8/11 53/53 [==============================] - 0s 6ms/step - loss: 0.6027 - accuracy: 0.9160 - val_loss: 0.5453 - val_accuracy: 0.9260 - lr: 0.0010 Epoch 9/11 53/53 [==============================] - 0s 6ms/step - loss: 0.5388 - accuracy: 0.9196 - val_loss: 0.5040 - val_accuracy: 0.9245 - lr: 0.0010 Epoch 10/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4958 - accuracy: 0.9239 - val_loss: 0.4594 - val_accuracy: 0.9343 - lr: 0.0010 Epoch 11/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4647 - accuracy: 0.9287 - val_loss: 0.4292 - val_accuracy: 0.9372 - lr: 0.0010 Trial 71 Complete [00h 00m 03s] val_accuracy: 0.937166690826416 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 07m 14s Search: Running Trial #72 Value |Best Value So Far |Hyperparameter 0 |1 |depth 256 |256 |width swish |swish |activation l1 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.4 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 256 |128 |batch_size 11 |32 |tuner/epochs 4 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 1 |3 |tuner/round 0066 |0048 |tuner/trial_id Epoch 5/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3997 - accuracy: 0.8843 - val_loss: 0.3133 - val_accuracy: 0.9150 - lr: 0.0100 Epoch 6/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2905 - accuracy: 0.9186 - val_loss: 0.3018 - val_accuracy: 0.9158 - lr: 0.0100 Epoch 7/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2751 - accuracy: 0.9234 - val_loss: 0.3151 - val_accuracy: 0.9148 - lr: 0.0100 Epoch 8/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2695 - accuracy: 0.9246 - val_loss: 0.3013 - val_accuracy: 0.9178 - lr: 0.0100 Epoch 9/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2640 - accuracy: 0.9263 - val_loss: 0.3071 - val_accuracy: 0.9153 - lr: 0.0100 Epoch 10/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2611 - accuracy: 0.9267 - val_loss: 0.2982 - val_accuracy: 0.9213 - lr: 0.0100 Epoch 11/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2555 - accuracy: 0.9295 - val_loss: 0.3030 - val_accuracy: 0.9192 - lr: 0.0100 Trial 72 Complete [00h 00m 04s] val_accuracy: 0.9213333129882812 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 07m 18s Search: Running Trial #73 Value |Best Value So Far |Hyperparameter 3 |1 |depth 128 |256 |width sigmoid |swish |activation none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.4 |dropout sgd |sgd |optimizer 0.01 |0.01 |learning_rate 128 |128 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 2 |3 |tuner/round 0068 |0048 |tuner/trial_id Epoch 12/32 422/422 [==============================] - 2s 3ms/step - loss: 0.4130 - accuracy: 0.8726 - val_loss: 0.1643 - val_accuracy: 0.9510 - lr: 0.0100 Epoch 13/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1724 - accuracy: 0.9491 - val_loss: 0.1527 - val_accuracy: 0.9562 - lr: 0.0100 Epoch 14/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1400 - accuracy: 0.9594 - val_loss: 0.1098 - val_accuracy: 0.9677 - lr: 0.0100 Epoch 15/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1225 - accuracy: 0.9640 - val_loss: 0.1004 - val_accuracy: 0.9710 - lr: 0.0100 Epoch 16/32 422/422 [==============================] - 1s 3ms/step - loss: 0.1096 - accuracy: 0.9675 - val_loss: 0.1014 - val_accuracy: 0.9715 - lr: 0.0100 Epoch 17/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0973 - accuracy: 0.9708 - val_loss: 0.0933 - val_accuracy: 0.9733 - lr: 0.0100 Epoch 18/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0940 - accuracy: 0.9721 - val_loss: 0.1004 - val_accuracy: 0.9737 - lr: 0.0100 Epoch 19/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0863 - accuracy: 0.9735 - val_loss: 0.1011 - val_accuracy: 0.9717 - lr: 0.0100 Epoch 20/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0853 - accuracy: 0.9744 - val_loss: 0.0949 - val_accuracy: 0.9753 - lr: 0.0100 Epoch 21/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0830 - accuracy: 0.9743 - val_loss: 0.1156 - val_accuracy: 0.9680 - lr: 0.0100 Epoch 22/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0800 - accuracy: 0.9752 - val_loss: 0.0887 - val_accuracy: 0.9720 - lr: 0.0100 Epoch 23/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0556 - accuracy: 0.9832 - val_loss: 0.0823 - val_accuracy: 0.9777 - lr: 1.0000e-03 Epoch 24/32 422/422 [==============================] - 1s 2ms/step - loss: 0.0431 - accuracy: 0.9861 - val_loss: 0.0826 - val_accuracy: 0.9772 - lr: 1.0000e-03 Epoch 25/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0406 - accuracy: 0.9871 - val_loss: 0.0848 - val_accuracy: 0.9790 - lr: 1.0000e-03 Epoch 26/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0372 - accuracy: 0.9882 - val_loss: 0.0823 - val_accuracy: 0.9790 - lr: 1.0000e-03 Epoch 27/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0344 - accuracy: 0.9894 - val_loss: 0.0871 - val_accuracy: 0.9793 - lr: 1.0000e-03 Epoch 28/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0336 - accuracy: 0.9895 - val_loss: 0.0837 - val_accuracy: 0.9787 - lr: 1.0000e-03 Epoch 29/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0323 - accuracy: 0.9898 - val_loss: 0.0853 - val_accuracy: 0.9777 - lr: 1.0000e-03 Epoch 30/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0292 - accuracy: 0.9911 - val_loss: 0.0850 - val_accuracy: 0.9782 - lr: 1.0000e-04 Epoch 31/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0275 - accuracy: 0.9914 - val_loss: 0.0855 - val_accuracy: 0.9782 - lr: 1.0000e-04 Epoch 32/32 422/422 [==============================] - 1s 3ms/step - loss: 0.0286 - accuracy: 0.9911 - val_loss: 0.0855 - val_accuracy: 0.9782 - lr: 1.0000e-05 Trial 73 Complete [00h 00m 24s] val_accuracy: 0.9793333411216736 Best val_accuracy So Far: 0.9793333411216736 Total elapsed time: 00h 07m 42s Search: Running Trial #74 Value |Best Value So Far |Hyperparameter 3 |1 |depth 512 |256 |width swish |swish |activation l1_l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.4 |dropout adam |sgd |optimizer 0.01 |0.01 |learning_rate 512 |128 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 2 |3 |tuner/bracket 2 |3 |tuner/round 0067 |0048 |tuner/trial_id Epoch 12/32 106/106 [==============================] - 2s 8ms/step - loss: 1.3544 - accuracy: 0.8244 - val_loss: 0.5954 - val_accuracy: 0.9467 - lr: 0.0100 Epoch 13/32 106/106 [==============================] - 1s 6ms/step - loss: 0.5543 - accuracy: 0.9503 - val_loss: 0.4877 - val_accuracy: 0.9618 - lr: 0.0100 Epoch 14/32 106/106 [==============================] - 1s 6ms/step - loss: 0.4616 - accuracy: 0.9628 - val_loss: 0.4205 - val_accuracy: 0.9675 - lr: 0.0100 Epoch 15/32 106/106 [==============================] - 1s 6ms/step - loss: 0.4037 - accuracy: 0.9666 - val_loss: 0.3963 - val_accuracy: 0.9623 - lr: 0.0100 Epoch 16/32 106/106 [==============================] - 1s 6ms/step - loss: 0.3602 - accuracy: 0.9699 - val_loss: 0.3321 - val_accuracy: 0.9720 - lr: 0.0100 Epoch 17/32 106/106 [==============================] - 1s 6ms/step - loss: 0.3231 - accuracy: 0.9717 - val_loss: 0.3396 - val_accuracy: 0.9655 - lr: 0.0100 Epoch 18/32 106/106 [==============================] - 1s 6ms/step - loss: 0.3046 - accuracy: 0.9722 - val_loss: 0.3109 - val_accuracy: 0.9712 - lr: 0.0100 Epoch 19/32 106/106 [==============================] - 1s 7ms/step - loss: 0.2496 - accuracy: 0.9852 - val_loss: 0.2551 - val_accuracy: 0.9833 - lr: 1.0000e-03 Epoch 20/32 106/106 [==============================] - 1s 6ms/step - loss: 0.2278 - accuracy: 0.9878 - val_loss: 0.2438 - val_accuracy: 0.9845 - lr: 1.0000e-03 Epoch 21/32 106/106 [==============================] - 1s 6ms/step - loss: 0.2159 - accuracy: 0.9894 - val_loss: 0.2362 - val_accuracy: 0.9842 - lr: 1.0000e-03 Epoch 22/32 106/106 [==============================] - 1s 6ms/step - loss: 0.2079 - accuracy: 0.9898 - val_loss: 0.2305 - val_accuracy: 0.9835 - lr: 1.0000e-03 Epoch 23/32 106/106 [==============================] - 1s 6ms/step - loss: 0.2001 - accuracy: 0.9912 - val_loss: 0.2273 - val_accuracy: 0.9842 - lr: 1.0000e-04 Epoch 24/32 106/106 [==============================] - 1s 7ms/step - loss: 0.1978 - accuracy: 0.9917 - val_loss: 0.2259 - val_accuracy: 0.9847 - lr: 1.0000e-04 Epoch 25/32 106/106 [==============================] - 1s 6ms/step - loss: 0.1971 - accuracy: 0.9915 - val_loss: 0.2254 - val_accuracy: 0.9850 - lr: 1.0000e-04 Epoch 26/32 106/106 [==============================] - 1s 6ms/step - loss: 0.1964 - accuracy: 0.9911 - val_loss: 0.2244 - val_accuracy: 0.9847 - lr: 1.0000e-04 Epoch 27/32 106/106 [==============================] - 1s 6ms/step - loss: 0.1956 - accuracy: 0.9915 - val_loss: 0.2238 - val_accuracy: 0.9847 - lr: 1.0000e-04 Epoch 28/32 106/106 [==============================] - 1s 6ms/step - loss: 0.1935 - accuracy: 0.9920 - val_loss: 0.2236 - val_accuracy: 0.9847 - lr: 1.0000e-05 Epoch 29/32 106/106 [==============================] - 1s 6ms/step - loss: 0.1947 - accuracy: 0.9913 - val_loss: 0.2233 - val_accuracy: 0.9847 - lr: 1.0000e-05 Epoch 30/32 106/106 [==============================] - 1s 6ms/step - loss: 0.1944 - accuracy: 0.9921 - val_loss: 0.2233 - val_accuracy: 0.9847 - lr: 1.0000e-06 Epoch 31/32 106/106 [==============================] - 1s 6ms/step - loss: 0.1937 - accuracy: 0.9917 - val_loss: 0.2232 - val_accuracy: 0.9847 - lr: 1.0000e-06 Epoch 32/32 106/106 [==============================] - 1s 6ms/step - loss: 0.1946 - accuracy: 0.9920 - val_loss: 0.2232 - val_accuracy: 0.9847 - lr: 1.0000e-07 Trial 74 Complete [00h 00m 15s] val_accuracy: 0.9850000143051147 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 07m 56s Search: Running Trial #75 Value |Best Value So Far |Hyperparameter 2 |3 |depth 512 |512 |width linear |swish |activation l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.2 |dropout adam |adam |optimizer 0.001 |0.01 |learning_rate 64 |512 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 844/844 [==============================] - 3s 3ms/step - loss: 0.4619 - accuracy: 0.8718 - val_loss: 0.3707 - val_accuracy: 0.9023 - lr: 0.0010 Epoch 2/11 844/844 [==============================] - 2s 3ms/step - loss: 0.3847 - accuracy: 0.8954 - val_loss: 0.3905 - val_accuracy: 0.8940 - lr: 0.0010 Epoch 3/11 844/844 [==============================] - 3s 3ms/step - loss: 0.3743 - accuracy: 0.8973 - val_loss: 0.3585 - val_accuracy: 0.9075 - lr: 0.0010 Epoch 4/11 844/844 [==============================] - 3s 3ms/step - loss: 0.3654 - accuracy: 0.9015 - val_loss: 0.3566 - val_accuracy: 0.9087 - lr: 0.0010 Epoch 5/11 844/844 [==============================] - 3s 3ms/step - loss: 0.3649 - accuracy: 0.9008 - val_loss: 0.3821 - val_accuracy: 0.8988 - lr: 0.0010 Epoch 6/11 844/844 [==============================] - 3s 3ms/step - loss: 0.3657 - accuracy: 0.9004 - val_loss: 0.3665 - val_accuracy: 0.9015 - lr: 0.0010 Epoch 7/11 844/844 [==============================] - 3s 3ms/step - loss: 0.2938 - accuracy: 0.9219 - val_loss: 0.3084 - val_accuracy: 0.9207 - lr: 1.0000e-04 Epoch 8/11 844/844 [==============================] - 3s 3ms/step - loss: 0.2837 - accuracy: 0.9250 - val_loss: 0.3068 - val_accuracy: 0.9192 - lr: 1.0000e-04 Epoch 9/11 844/844 [==============================] - 3s 3ms/step - loss: 0.2816 - accuracy: 0.9256 - val_loss: 0.3072 - val_accuracy: 0.9208 - lr: 1.0000e-04 Epoch 10/11 844/844 [==============================] - 3s 3ms/step - loss: 0.2786 - accuracy: 0.9255 - val_loss: 0.3084 - val_accuracy: 0.9212 - lr: 1.0000e-04 Epoch 11/11 844/844 [==============================] - 3s 3ms/step - loss: 0.2767 - accuracy: 0.9271 - val_loss: 0.3075 - val_accuracy: 0.9203 - lr: 1.0000e-04 Trial 75 Complete [00h 00m 30s] val_accuracy: 0.9211666584014893 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 08m 26s Search: Running Trial #76 Value |Best Value So Far |Hyperparameter 3 |3 |depth 64 |512 |width linear |swish |activation l2 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.2 |dropout rmsprop |adam |optimizer 0.001 |0.01 |learning_rate 256 |512 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 211/211 [==============================] - 1s 4ms/step - loss: 1.1871 - accuracy: 0.6175 - val_loss: 0.4250 - val_accuracy: 0.8875 - lr: 0.0010 Epoch 2/11 211/211 [==============================] - 1s 3ms/step - loss: 0.7439 - accuracy: 0.7791 - val_loss: 0.3881 - val_accuracy: 0.8980 - lr: 0.0010 Epoch 3/11 211/211 [==============================] - 1s 3ms/step - loss: 0.6636 - accuracy: 0.8072 - val_loss: 0.3660 - val_accuracy: 0.9038 - lr: 0.0010 Epoch 4/11 211/211 [==============================] - 1s 3ms/step - loss: 0.6245 - accuracy: 0.8193 - val_loss: 0.3643 - val_accuracy: 0.9038 - lr: 0.0010 Epoch 5/11 211/211 [==============================] - 1s 3ms/step - loss: 0.6038 - accuracy: 0.8278 - val_loss: 0.3601 - val_accuracy: 0.9065 - lr: 0.0010 Epoch 6/11 211/211 [==============================] - 1s 3ms/step - loss: 0.5796 - accuracy: 0.8378 - val_loss: 0.3568 - val_accuracy: 0.9047 - lr: 0.0010 Epoch 7/11 211/211 [==============================] - 1s 3ms/step - loss: 0.5677 - accuracy: 0.8396 - val_loss: 0.3529 - val_accuracy: 0.9068 - lr: 0.0010 Epoch 8/11 211/211 [==============================] - 1s 3ms/step - loss: 0.5576 - accuracy: 0.8454 - val_loss: 0.3599 - val_accuracy: 0.9043 - lr: 0.0010 Epoch 9/11 211/211 [==============================] - 1s 3ms/step - loss: 0.5411 - accuracy: 0.8523 - val_loss: 0.3446 - val_accuracy: 0.9115 - lr: 0.0010 Epoch 10/11 211/211 [==============================] - 1s 3ms/step - loss: 0.5340 - accuracy: 0.8528 - val_loss: 0.3473 - val_accuracy: 0.9112 - lr: 0.0010 Epoch 11/11 211/211 [==============================] - 1s 3ms/step - loss: 0.5353 - accuracy: 0.8536 - val_loss: 0.3433 - val_accuracy: 0.9083 - lr: 0.0010 Trial 76 Complete [00h 00m 08s] val_accuracy: 0.9114999771118164 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 08m 35s Search: Running Trial #77 Value |Best Value So Far |Hyperparameter 0 |3 |depth 128 |512 |width sigmoid |swish |activation l1_l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.2 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 128 |512 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 422/422 [==============================] - 1s 2ms/step - loss: 0.3693 - accuracy: 0.8946 - val_loss: 0.3100 - val_accuracy: 0.9152 - lr: 0.0100 Epoch 2/11 422/422 [==============================] - 1s 2ms/step - loss: 0.2944 - accuracy: 0.9179 - val_loss: 0.3050 - val_accuracy: 0.9178 - lr: 0.0100 Epoch 3/11 422/422 [==============================] - 1s 2ms/step - loss: 0.2797 - accuracy: 0.9222 - val_loss: 0.3256 - val_accuracy: 0.9148 - lr: 0.0100 Epoch 4/11 422/422 [==============================] - 1s 2ms/step - loss: 0.2753 - accuracy: 0.9247 - val_loss: 0.3296 - val_accuracy: 0.9122 - lr: 0.0100 Epoch 5/11 422/422 [==============================] - 1s 2ms/step - loss: 0.2413 - accuracy: 0.9339 - val_loss: 0.2946 - val_accuracy: 0.9207 - lr: 1.0000e-03 Epoch 6/11 422/422 [==============================] - 1s 2ms/step - loss: 0.2352 - accuracy: 0.9360 - val_loss: 0.2936 - val_accuracy: 0.9225 - lr: 1.0000e-03 Epoch 7/11 422/422 [==============================] - 1s 2ms/step - loss: 0.2343 - accuracy: 0.9363 - val_loss: 0.2927 - val_accuracy: 0.9227 - lr: 1.0000e-03 Epoch 8/11 422/422 [==============================] - 1s 2ms/step - loss: 0.2333 - accuracy: 0.9359 - val_loss: 0.2916 - val_accuracy: 0.9218 - lr: 1.0000e-03 Epoch 9/11 422/422 [==============================] - 1s 2ms/step - loss: 0.2327 - accuracy: 0.9368 - val_loss: 0.2910 - val_accuracy: 0.9245 - lr: 1.0000e-03 Epoch 10/11 422/422 [==============================] - 1s 2ms/step - loss: 0.2323 - accuracy: 0.9371 - val_loss: 0.2928 - val_accuracy: 0.9225 - lr: 1.0000e-03 Epoch 11/11 422/422 [==============================] - 1s 2ms/step - loss: 0.2317 - accuracy: 0.9365 - val_loss: 0.2914 - val_accuracy: 0.9220 - lr: 1.0000e-03 Trial 77 Complete [00h 00m 11s] val_accuracy: 0.9244999885559082 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 08m 46s Search: Running Trial #78 Value |Best Value So Far |Hyperparameter 2 |3 |depth 256 |512 |width swish |swish |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.2 |dropout adam |adam |optimizer 0.0001 |0.01 |learning_rate 128 |512 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 422/422 [==============================] - 2s 3ms/step - loss: 2.0032 - accuracy: 0.7151 - val_loss: 1.2925 - val_accuracy: 0.8878 - lr: 1.0000e-04 Epoch 2/11 422/422 [==============================] - 1s 3ms/step - loss: 1.2501 - accuracy: 0.8645 - val_loss: 1.0795 - val_accuracy: 0.9057 - lr: 1.0000e-04 Epoch 3/11 422/422 [==============================] - 1s 3ms/step - loss: 1.0970 - accuracy: 0.8873 - val_loss: 0.9890 - val_accuracy: 0.9158 - lr: 1.0000e-04 Epoch 4/11 422/422 [==============================] - 1s 3ms/step - loss: 1.0062 - accuracy: 0.8986 - val_loss: 0.9185 - val_accuracy: 0.9205 - lr: 1.0000e-04 Epoch 5/11 422/422 [==============================] - 1s 3ms/step - loss: 0.9386 - accuracy: 0.9061 - val_loss: 0.8621 - val_accuracy: 0.9247 - lr: 1.0000e-04 Epoch 6/11 422/422 [==============================] - 1s 3ms/step - loss: 0.8832 - accuracy: 0.9127 - val_loss: 0.8113 - val_accuracy: 0.9285 - lr: 1.0000e-04 Epoch 7/11 422/422 [==============================] - 1s 3ms/step - loss: 0.8335 - accuracy: 0.9157 - val_loss: 0.7661 - val_accuracy: 0.9308 - lr: 1.0000e-04 Epoch 8/11 422/422 [==============================] - 1s 3ms/step - loss: 0.7834 - accuracy: 0.9222 - val_loss: 0.7253 - val_accuracy: 0.9363 - lr: 1.0000e-04 Epoch 9/11 422/422 [==============================] - 1s 3ms/step - loss: 0.7449 - accuracy: 0.9236 - val_loss: 0.6874 - val_accuracy: 0.9385 - lr: 1.0000e-04 Epoch 10/11 422/422 [==============================] - 1s 3ms/step - loss: 0.7095 - accuracy: 0.9255 - val_loss: 0.6539 - val_accuracy: 0.9387 - lr: 1.0000e-04 Epoch 11/11 422/422 [==============================] - 1s 3ms/step - loss: 0.6756 - accuracy: 0.9306 - val_loss: 0.6226 - val_accuracy: 0.9433 - lr: 1.0000e-04 Trial 78 Complete [00h 00m 15s] val_accuracy: 0.9433333277702332 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 09m 01s Search: Running Trial #79 Value |Best Value So Far |Hyperparameter 1 |3 |depth 64 |512 |width sigmoid |swish |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.2 |dropout sgd |adam |optimizer 0.001 |0.01 |learning_rate 256 |512 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 211/211 [==============================] - 1s 3ms/step - loss: 1.2356 - accuracy: 0.7426 - val_loss: 0.7345 - val_accuracy: 0.8775 - lr: 0.0010 Epoch 2/11 211/211 [==============================] - 1s 3ms/step - loss: 0.6638 - accuracy: 0.8766 - val_loss: 0.5619 - val_accuracy: 0.8978 - lr: 0.0010 Epoch 3/11 211/211 [==============================] - 1s 3ms/step - loss: 0.5562 - accuracy: 0.8926 - val_loss: 0.4980 - val_accuracy: 0.9077 - lr: 0.0010 Epoch 4/11 211/211 [==============================] - 1s 3ms/step - loss: 0.5078 - accuracy: 0.9023 - val_loss: 0.4626 - val_accuracy: 0.9135 - lr: 0.0010 Epoch 5/11 211/211 [==============================] - 1s 3ms/step - loss: 0.4782 - accuracy: 0.9082 - val_loss: 0.4388 - val_accuracy: 0.9185 - lr: 0.0010 Epoch 6/11 211/211 [==============================] - 1s 3ms/step - loss: 0.4567 - accuracy: 0.9120 - val_loss: 0.4205 - val_accuracy: 0.9215 - lr: 0.0010 Epoch 7/11 211/211 [==============================] - 1s 3ms/step - loss: 0.4389 - accuracy: 0.9154 - val_loss: 0.4070 - val_accuracy: 0.9267 - lr: 0.0010 Epoch 8/11 211/211 [==============================] - 1s 3ms/step - loss: 0.4240 - accuracy: 0.9194 - val_loss: 0.3913 - val_accuracy: 0.9270 - lr: 0.0010 Epoch 9/11 211/211 [==============================] - 1s 3ms/step - loss: 0.4093 - accuracy: 0.9226 - val_loss: 0.3824 - val_accuracy: 0.9315 - lr: 0.0010 Epoch 10/11 211/211 [==============================] - 1s 3ms/step - loss: 0.4010 - accuracy: 0.9237 - val_loss: 0.3682 - val_accuracy: 0.9335 - lr: 0.0010 Epoch 11/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3881 - accuracy: 0.9273 - val_loss: 0.3627 - val_accuracy: 0.9350 - lr: 0.0010 Trial 79 Complete [00h 00m 07s] val_accuracy: 0.9350000023841858 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 09m 08s Search: Running Trial #80 Value |Best Value So Far |Hyperparameter 3 |3 |depth 128 |512 |width linear |swish |activation l1 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.2 |dropout sgd |adam |optimizer 0.001 |0.01 |learning_rate 256 |512 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 211/211 [==============================] - 1s 4ms/step - loss: 0.5091 - accuracy: 0.8666 - val_loss: 0.3846 - val_accuracy: 0.9108 - lr: 0.0010 Epoch 2/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3887 - accuracy: 0.9051 - val_loss: 0.3612 - val_accuracy: 0.9168 - lr: 0.0010 Epoch 3/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3710 - accuracy: 0.9091 - val_loss: 0.3609 - val_accuracy: 0.9200 - lr: 0.0010 Epoch 4/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3610 - accuracy: 0.9119 - val_loss: 0.3595 - val_accuracy: 0.9138 - lr: 0.0010 Epoch 5/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3530 - accuracy: 0.9154 - val_loss: 0.3527 - val_accuracy: 0.9178 - lr: 0.0010 Epoch 6/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3264 - accuracy: 0.9231 - val_loss: 0.3382 - val_accuracy: 0.9207 - lr: 1.0000e-04 Epoch 7/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3225 - accuracy: 0.9248 - val_loss: 0.3370 - val_accuracy: 0.9220 - lr: 1.0000e-04 Epoch 8/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3193 - accuracy: 0.9235 - val_loss: 0.3377 - val_accuracy: 0.9212 - lr: 1.0000e-04 Epoch 9/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3177 - accuracy: 0.9254 - val_loss: 0.3393 - val_accuracy: 0.9212 - lr: 1.0000e-04 Epoch 10/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3135 - accuracy: 0.9267 - val_loss: 0.3377 - val_accuracy: 0.9215 - lr: 1.0000e-05 Epoch 11/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3129 - accuracy: 0.9271 - val_loss: 0.3374 - val_accuracy: 0.9213 - lr: 1.0000e-05 Trial 80 Complete [00h 00m 09s] val_accuracy: 0.921999990940094 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 09m 16s Search: Running Trial #81 Value |Best Value So Far |Hyperparameter 4 |3 |depth 128 |512 |width relu |swish |activation l1_l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.5 |0.2 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 1024 |512 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 53/53 [==============================] - 1s 10ms/step - loss: 1.1964 - accuracy: 0.6220 - val_loss: 0.4234 - val_accuracy: 0.9145 - lr: 0.0100 Epoch 2/11 53/53 [==============================] - 0s 6ms/step - loss: 0.6469 - accuracy: 0.8480 - val_loss: 0.3567 - val_accuracy: 0.9273 - lr: 0.0100 Epoch 3/11 53/53 [==============================] - 0s 6ms/step - loss: 0.5589 - accuracy: 0.8753 - val_loss: 0.3263 - val_accuracy: 0.9370 - lr: 0.0100 Epoch 4/11 53/53 [==============================] - 0s 6ms/step - loss: 0.5305 - accuracy: 0.8854 - val_loss: 0.3290 - val_accuracy: 0.9387 - lr: 0.0100 Epoch 5/11 53/53 [==============================] - 0s 6ms/step - loss: 0.5041 - accuracy: 0.8937 - val_loss: 0.3241 - val_accuracy: 0.9387 - lr: 0.0100 Epoch 6/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4994 - accuracy: 0.8967 - val_loss: 0.3092 - val_accuracy: 0.9438 - lr: 0.0100 Epoch 7/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4941 - accuracy: 0.8995 - val_loss: 0.3092 - val_accuracy: 0.9478 - lr: 0.0100 Epoch 8/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4869 - accuracy: 0.9027 - val_loss: 0.3044 - val_accuracy: 0.9452 - lr: 0.0100 Epoch 9/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4928 - accuracy: 0.9024 - val_loss: 0.3192 - val_accuracy: 0.9462 - lr: 0.0100 Epoch 10/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4582 - accuracy: 0.9141 - val_loss: 0.2936 - val_accuracy: 0.9508 - lr: 1.0000e-03 Epoch 11/11 53/53 [==============================] - 0s 6ms/step - loss: 0.4221 - accuracy: 0.9205 - val_loss: 0.2893 - val_accuracy: 0.9515 - lr: 1.0000e-03 Trial 81 Complete [00h 00m 05s] val_accuracy: 0.9514999985694885 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 09m 21s Search: Running Trial #82 Value |Best Value So Far |Hyperparameter 0 |3 |depth 64 |512 |width tanh |swish |activation l1_l2 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0 |0.2 |dropout sgd |adam |optimizer 0.01 |0.01 |learning_rate 256 |512 |batch_size 11 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/11 211/211 [==============================] - 1s 3ms/step - loss: 0.3998 - accuracy: 0.8856 - val_loss: 0.3192 - val_accuracy: 0.9122 - lr: 0.0100 Epoch 2/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2886 - accuracy: 0.9184 - val_loss: 0.3069 - val_accuracy: 0.9145 - lr: 0.0100 Epoch 3/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2770 - accuracy: 0.9219 - val_loss: 0.3003 - val_accuracy: 0.9162 - lr: 0.0100 Epoch 4/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2677 - accuracy: 0.9255 - val_loss: 0.3132 - val_accuracy: 0.9177 - lr: 0.0100 Epoch 5/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2627 - accuracy: 0.9282 - val_loss: 0.3019 - val_accuracy: 0.9195 - lr: 0.0100 Epoch 6/11 211/211 [==============================] - 1s 2ms/step - loss: 0.2604 - accuracy: 0.9270 - val_loss: 0.2995 - val_accuracy: 0.9212 - lr: 0.0100 Epoch 7/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2566 - accuracy: 0.9286 - val_loss: 0.3028 - val_accuracy: 0.9203 - lr: 0.0100 Epoch 8/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2556 - accuracy: 0.9291 - val_loss: 0.3005 - val_accuracy: 0.9172 - lr: 0.0100 Epoch 9/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2311 - accuracy: 0.9365 - val_loss: 0.2934 - val_accuracy: 0.9218 - lr: 1.0000e-03 Epoch 10/11 211/211 [==============================] - 1s 2ms/step - loss: 0.2282 - accuracy: 0.9378 - val_loss: 0.2918 - val_accuracy: 0.9220 - lr: 1.0000e-03 Epoch 11/11 211/211 [==============================] - 0s 2ms/step - loss: 0.2273 - accuracy: 0.9378 - val_loss: 0.2928 - val_accuracy: 0.9207 - lr: 1.0000e-03 Trial 82 Complete [00h 00m 06s] val_accuracy: 0.921999990940094 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 09m 28s Search: Running Trial #83 Value |Best Value So Far |Hyperparameter 4 |3 |depth 128 |512 |width relu |swish |activation l1_l2 |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.5 |0.2 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 1024 |512 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 1 |2 |tuner/round 0080 |0067 |tuner/trial_id Epoch 12/32 53/53 [==============================] - 1s 10ms/step - loss: 1.3158 - accuracy: 0.5698 - val_loss: 0.4636 - val_accuracy: 0.8890 - lr: 0.0100 Epoch 13/32 53/53 [==============================] - 0s 6ms/step - loss: 0.6910 - accuracy: 0.8284 - val_loss: 0.3717 - val_accuracy: 0.9203 - lr: 0.0100 Epoch 14/32 53/53 [==============================] - 0s 6ms/step - loss: 0.5984 - accuracy: 0.8606 - val_loss: 0.3350 - val_accuracy: 0.9303 - lr: 0.0100 Epoch 15/32 53/53 [==============================] - 0s 6ms/step - loss: 0.5666 - accuracy: 0.8719 - val_loss: 0.3388 - val_accuracy: 0.9347 - lr: 0.0100 Epoch 16/32 53/53 [==============================] - 0s 6ms/step - loss: 0.5381 - accuracy: 0.8811 - val_loss: 0.3146 - val_accuracy: 0.9363 - lr: 0.0100 Epoch 17/32 53/53 [==============================] - 0s 6ms/step - loss: 0.5329 - accuracy: 0.8843 - val_loss: 0.3236 - val_accuracy: 0.9395 - lr: 0.0100 Epoch 18/32 53/53 [==============================] - 0s 5ms/step - loss: 0.5107 - accuracy: 0.8907 - val_loss: 0.3318 - val_accuracy: 0.9360 - lr: 0.0100 Epoch 19/32 53/53 [==============================] - 0s 6ms/step - loss: 0.5153 - accuracy: 0.8901 - val_loss: 0.3153 - val_accuracy: 0.9417 - lr: 0.0100 Epoch 20/32 53/53 [==============================] - 0s 6ms/step - loss: 0.5154 - accuracy: 0.8926 - val_loss: 0.3104 - val_accuracy: 0.9465 - lr: 0.0100 Epoch 21/32 53/53 [==============================] - 0s 6ms/step - loss: 0.5008 - accuracy: 0.8961 - val_loss: 0.3178 - val_accuracy: 0.9442 - lr: 0.0100 Epoch 22/32 53/53 [==============================] - 0s 5ms/step - loss: 0.5099 - accuracy: 0.8940 - val_loss: 0.3182 - val_accuracy: 0.9412 - lr: 0.0100 Epoch 23/32 53/53 [==============================] - 0s 6ms/step - loss: 0.4668 - accuracy: 0.9069 - val_loss: 0.2969 - val_accuracy: 0.9477 - lr: 1.0000e-03 Epoch 24/32 53/53 [==============================] - 0s 5ms/step - loss: 0.4430 - accuracy: 0.9116 - val_loss: 0.2921 - val_accuracy: 0.9473 - lr: 1.0000e-03 Epoch 25/32 53/53 [==============================] - 0s 6ms/step - loss: 0.4287 - accuracy: 0.9162 - val_loss: 0.2909 - val_accuracy: 0.9483 - lr: 1.0000e-03 Epoch 26/32 53/53 [==============================] - 0s 6ms/step - loss: 0.4210 - accuracy: 0.9170 - val_loss: 0.2863 - val_accuracy: 0.9490 - lr: 1.0000e-03 Epoch 27/32 53/53 [==============================] - 0s 5ms/step - loss: 0.4189 - accuracy: 0.9175 - val_loss: 0.2857 - val_accuracy: 0.9488 - lr: 1.0000e-03 Epoch 28/32 53/53 [==============================] - 0s 5ms/step - loss: 0.4131 - accuracy: 0.9174 - val_loss: 0.2845 - val_accuracy: 0.9488 - lr: 1.0000e-03 Epoch 29/32 53/53 [==============================] - 0s 6ms/step - loss: 0.4046 - accuracy: 0.9209 - val_loss: 0.2838 - val_accuracy: 0.9495 - lr: 1.0000e-04 Epoch 30/32 53/53 [==============================] - 0s 5ms/step - loss: 0.4036 - accuracy: 0.9216 - val_loss: 0.2839 - val_accuracy: 0.9493 - lr: 1.0000e-04 Epoch 31/32 53/53 [==============================] - 0s 5ms/step - loss: 0.4041 - accuracy: 0.9202 - val_loss: 0.2839 - val_accuracy: 0.9493 - lr: 1.0000e-04 Epoch 32/32 53/53 [==============================] - 0s 5ms/step - loss: 0.4021 - accuracy: 0.9210 - val_loss: 0.2838 - val_accuracy: 0.9492 - lr: 1.0000e-05 Trial 83 Complete [00h 00m 08s] val_accuracy: 0.9495000243186951 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 09m 35s Search: Running Trial #84 Value |Best Value So Far |Hyperparameter 2 |3 |depth 256 |512 |width swish |swish |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.2 |dropout adam |adam |optimizer 0.0001 |0.01 |learning_rate 128 |512 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 1 |2 |tuner/round 0077 |0067 |tuner/trial_id Epoch 12/32 422/422 [==============================] - 2s 3ms/step - loss: 1.9935 - accuracy: 0.7124 - val_loss: 1.2945 - val_accuracy: 0.8835 - lr: 1.0000e-04 Epoch 13/32 422/422 [==============================] - 1s 3ms/step - loss: 1.2551 - accuracy: 0.8644 - val_loss: 1.0796 - val_accuracy: 0.9047 - lr: 1.0000e-04 Epoch 14/32 422/422 [==============================] - 1s 3ms/step - loss: 1.0928 - accuracy: 0.8898 - val_loss: 0.9849 - val_accuracy: 0.9137 - lr: 1.0000e-04 Epoch 15/32 422/422 [==============================] - 1s 3ms/step - loss: 1.0065 - accuracy: 0.8991 - val_loss: 0.9157 - val_accuracy: 0.9205 - lr: 1.0000e-04 Epoch 16/32 422/422 [==============================] - 1s 3ms/step - loss: 0.9380 - accuracy: 0.9060 - val_loss: 0.8580 - val_accuracy: 0.9248 - lr: 1.0000e-04 Epoch 17/32 422/422 [==============================] - 1s 3ms/step - loss: 0.8802 - accuracy: 0.9109 - val_loss: 0.8096 - val_accuracy: 0.9260 - lr: 1.0000e-04 Epoch 18/32 422/422 [==============================] - 1s 3ms/step - loss: 0.8320 - accuracy: 0.9159 - val_loss: 0.7630 - val_accuracy: 0.9297 - lr: 1.0000e-04 Epoch 19/32 422/422 [==============================] - 1s 3ms/step - loss: 0.7867 - accuracy: 0.9202 - val_loss: 0.7227 - val_accuracy: 0.9343 - lr: 1.0000e-04 Epoch 20/32 422/422 [==============================] - 1s 3ms/step - loss: 0.7473 - accuracy: 0.9221 - val_loss: 0.6870 - val_accuracy: 0.9365 - lr: 1.0000e-04 Epoch 21/32 422/422 [==============================] - 1s 3ms/step - loss: 0.7128 - accuracy: 0.9259 - val_loss: 0.6533 - val_accuracy: 0.9385 - lr: 1.0000e-04 Epoch 22/32 422/422 [==============================] - 1s 3ms/step - loss: 0.6786 - accuracy: 0.9279 - val_loss: 0.6241 - val_accuracy: 0.9405 - lr: 1.0000e-04 Epoch 23/32 422/422 [==============================] - 1s 3ms/step - loss: 0.6515 - accuracy: 0.9305 - val_loss: 0.5966 - val_accuracy: 0.9422 - lr: 1.0000e-04 Epoch 24/32 422/422 [==============================] - 1s 3ms/step - loss: 0.6262 - accuracy: 0.9311 - val_loss: 0.5729 - val_accuracy: 0.9448 - lr: 1.0000e-04 Epoch 25/32 422/422 [==============================] - 1s 3ms/step - loss: 0.6027 - accuracy: 0.9318 - val_loss: 0.5527 - val_accuracy: 0.9432 - lr: 1.0000e-04 Epoch 26/32 422/422 [==============================] - 1s 3ms/step - loss: 0.5822 - accuracy: 0.9349 - val_loss: 0.5335 - val_accuracy: 0.9447 - lr: 1.0000e-04 Epoch 27/32 422/422 [==============================] - 1s 3ms/step - loss: 0.5687 - accuracy: 0.9351 - val_loss: 0.5280 - val_accuracy: 0.9448 - lr: 1.0000e-05 Epoch 28/32 422/422 [==============================] - 1s 3ms/step - loss: 0.5639 - accuracy: 0.9358 - val_loss: 0.5251 - val_accuracy: 0.9462 - lr: 1.0000e-05 Epoch 29/32 422/422 [==============================] - 1s 3ms/step - loss: 0.5598 - accuracy: 0.9361 - val_loss: 0.5229 - val_accuracy: 0.9460 - lr: 1.0000e-05 Epoch 30/32 422/422 [==============================] - 1s 3ms/step - loss: 0.5583 - accuracy: 0.9359 - val_loss: 0.5205 - val_accuracy: 0.9462 - lr: 1.0000e-05 Epoch 31/32 422/422 [==============================] - 1s 3ms/step - loss: 0.5589 - accuracy: 0.9353 - val_loss: 0.5202 - val_accuracy: 0.9463 - lr: 1.0000e-06 Epoch 32/32 422/422 [==============================] - 1s 3ms/step - loss: 0.5595 - accuracy: 0.9366 - val_loss: 0.5200 - val_accuracy: 0.9465 - lr: 1.0000e-06 Trial 84 Complete [00h 00m 26s] val_accuracy: 0.9465000033378601 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 10m 02s Search: Running Trial #85 Value |Best Value So Far |Hyperparameter 1 |3 |depth 64 |512 |width sigmoid |swish |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.2 |dropout sgd |adam |optimizer 0.001 |0.01 |learning_rate 256 |512 |batch_size 32 |32 |tuner/epochs 11 |11 |tuner/initial_epoch 1 |2 |tuner/bracket 1 |2 |tuner/round 0078 |0067 |tuner/trial_id Epoch 12/32 211/211 [==============================] - 1s 3ms/step - loss: 1.2387 - accuracy: 0.7541 - val_loss: 0.7307 - val_accuracy: 0.8803 - lr: 0.0010 Epoch 13/32 211/211 [==============================] - 1s 3ms/step - loss: 0.6617 - accuracy: 0.8778 - val_loss: 0.5616 - val_accuracy: 0.9030 - lr: 0.0010 Epoch 14/32 211/211 [==============================] - 1s 3ms/step - loss: 0.5584 - accuracy: 0.8939 - val_loss: 0.4983 - val_accuracy: 0.9073 - lr: 0.0010 Epoch 15/32 211/211 [==============================] - 1s 3ms/step - loss: 0.5091 - accuracy: 0.9018 - val_loss: 0.4633 - val_accuracy: 0.9148 - lr: 0.0010 Epoch 16/32 211/211 [==============================] - 1s 3ms/step - loss: 0.4779 - accuracy: 0.9094 - val_loss: 0.4383 - val_accuracy: 0.9183 - lr: 0.0010 Epoch 17/32 211/211 [==============================] - 1s 3ms/step - loss: 0.4561 - accuracy: 0.9122 - val_loss: 0.4195 - val_accuracy: 0.9220 - lr: 0.0010 Epoch 18/32 211/211 [==============================] - 1s 3ms/step - loss: 0.4361 - accuracy: 0.9170 - val_loss: 0.4038 - val_accuracy: 0.9263 - lr: 0.0010 Epoch 19/32 211/211 [==============================] - 1s 3ms/step - loss: 0.4225 - accuracy: 0.9195 - val_loss: 0.3898 - val_accuracy: 0.9277 - lr: 0.0010 Epoch 20/32 211/211 [==============================] - 1s 3ms/step - loss: 0.4080 - accuracy: 0.9227 - val_loss: 0.3792 - val_accuracy: 0.9328 - lr: 0.0010 Epoch 21/32 211/211 [==============================] - 1s 3ms/step - loss: 0.3990 - accuracy: 0.9251 - val_loss: 0.3701 - val_accuracy: 0.9340 - lr: 0.0010 Epoch 22/32 211/211 [==============================] - 1s 3ms/step - loss: 0.3875 - accuracy: 0.9271 - val_loss: 0.3585 - val_accuracy: 0.9382 - lr: 0.0010 Epoch 23/32 211/211 [==============================] - 1s 3ms/step - loss: 0.3793 - accuracy: 0.9297 - val_loss: 0.3499 - val_accuracy: 0.9392 - lr: 0.0010 Epoch 24/32 211/211 [==============================] - 1s 3ms/step - loss: 0.3704 - accuracy: 0.9316 - val_loss: 0.3439 - val_accuracy: 0.9408 - lr: 0.0010 Epoch 25/32 211/211 [==============================] - 1s 2ms/step - loss: 0.3630 - accuracy: 0.9331 - val_loss: 0.3357 - val_accuracy: 0.9407 - lr: 0.0010 Epoch 26/32 211/211 [==============================] - 1s 3ms/step - loss: 0.3541 - accuracy: 0.9349 - val_loss: 0.3298 - val_accuracy: 0.9440 - lr: 0.0010 Epoch 27/32 211/211 [==============================] - 1s 3ms/step - loss: 0.3475 - accuracy: 0.9360 - val_loss: 0.3225 - val_accuracy: 0.9445 - lr: 0.0010 Epoch 28/32 211/211 [==============================] - 1s 3ms/step - loss: 0.3410 - accuracy: 0.9382 - val_loss: 0.3166 - val_accuracy: 0.9455 - lr: 0.0010 Epoch 29/32 211/211 [==============================] - 1s 3ms/step - loss: 0.3365 - accuracy: 0.9392 - val_loss: 0.3125 - val_accuracy: 0.9467 - lr: 0.0010 Epoch 30/32 211/211 [==============================] - 1s 3ms/step - loss: 0.3311 - accuracy: 0.9408 - val_loss: 0.3043 - val_accuracy: 0.9493 - lr: 0.0010 Epoch 31/32 211/211 [==============================] - 1s 3ms/step - loss: 0.3268 - accuracy: 0.9406 - val_loss: 0.3000 - val_accuracy: 0.9487 - lr: 0.0010 Epoch 32/32 211/211 [==============================] - 1s 3ms/step - loss: 0.3201 - accuracy: 0.9428 - val_loss: 0.2953 - val_accuracy: 0.9510 - lr: 0.0010 Trial 85 Complete [00h 00m 12s] val_accuracy: 0.9509999752044678 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 10m 14s Search: Running Trial #86 Value |Best Value So Far |Hyperparameter 0 |3 |depth 512 |512 |width linear |swish |activation l1_l2 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.2 |dropout adam |adam |optimizer 0.01 |0.01 |learning_rate 128 |512 |batch_size 32 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/32 422/422 [==============================] - 1s 2ms/step - loss: 0.3725 - accuracy: 0.8929 - val_loss: 0.3175 - val_accuracy: 0.9123 - lr: 0.0100 Epoch 2/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2944 - accuracy: 0.9169 - val_loss: 0.3057 - val_accuracy: 0.9168 - lr: 0.0100 Epoch 3/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2819 - accuracy: 0.9215 - val_loss: 0.3251 - val_accuracy: 0.9133 - lr: 0.0100 Epoch 4/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2755 - accuracy: 0.9233 - val_loss: 0.3274 - val_accuracy: 0.9128 - lr: 0.0100 Epoch 5/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2407 - accuracy: 0.9343 - val_loss: 0.2957 - val_accuracy: 0.9198 - lr: 1.0000e-03 Epoch 6/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2355 - accuracy: 0.9354 - val_loss: 0.2934 - val_accuracy: 0.9213 - lr: 1.0000e-03 Epoch 7/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2342 - accuracy: 0.9365 - val_loss: 0.2913 - val_accuracy: 0.9227 - lr: 1.0000e-03 Epoch 8/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2334 - accuracy: 0.9362 - val_loss: 0.2919 - val_accuracy: 0.9208 - lr: 1.0000e-03 Epoch 9/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2327 - accuracy: 0.9365 - val_loss: 0.2935 - val_accuracy: 0.9200 - lr: 1.0000e-03 Epoch 10/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2285 - accuracy: 0.9381 - val_loss: 0.2903 - val_accuracy: 0.9225 - lr: 1.0000e-04 Epoch 11/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2279 - accuracy: 0.9380 - val_loss: 0.2902 - val_accuracy: 0.9232 - lr: 1.0000e-04 Epoch 12/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2278 - accuracy: 0.9384 - val_loss: 0.2901 - val_accuracy: 0.9222 - lr: 1.0000e-04 Epoch 13/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2277 - accuracy: 0.9380 - val_loss: 0.2899 - val_accuracy: 0.9232 - lr: 1.0000e-04 Epoch 14/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2271 - accuracy: 0.9384 - val_loss: 0.2899 - val_accuracy: 0.9233 - lr: 1.0000e-05 Epoch 15/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2271 - accuracy: 0.9384 - val_loss: 0.2899 - val_accuracy: 0.9235 - lr: 1.0000e-05 Epoch 16/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2271 - accuracy: 0.9383 - val_loss: 0.2899 - val_accuracy: 0.9232 - lr: 1.0000e-05 Epoch 17/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2271 - accuracy: 0.9383 - val_loss: 0.2899 - val_accuracy: 0.9232 - lr: 1.0000e-05 Epoch 18/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2270 - accuracy: 0.9384 - val_loss: 0.2899 - val_accuracy: 0.9232 - lr: 1.0000e-06 Epoch 19/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2270 - accuracy: 0.9384 - val_loss: 0.2899 - val_accuracy: 0.9232 - lr: 1.0000e-06 Epoch 20/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2270 - accuracy: 0.9384 - val_loss: 0.2899 - val_accuracy: 0.9232 - lr: 1.0000e-07 Epoch 21/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2270 - accuracy: 0.9384 - val_loss: 0.2899 - val_accuracy: 0.9232 - lr: 1.0000e-07 Epoch 22/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2270 - accuracy: 0.9384 - val_loss: 0.2899 - val_accuracy: 0.9232 - lr: 1.0000e-08 Epoch 23/32 422/422 [==============================] - 1s 2ms/step - loss: 0.2270 - accuracy: 0.9384 - val_loss: 0.2899 - val_accuracy: 0.9232 - lr: 1.0000e-08 Trial 86 Complete [00h 00m 22s] val_accuracy: 0.9235000014305115 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 10m 36s Search: Running Trial #87 Value |Best Value So Far |Hyperparameter 4 |3 |depth 128 |512 |width sigmoid |swish |activation none |l1_l2 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.5 |0.2 |dropout adam |adam |optimizer 0.001 |0.01 |learning_rate 32 |512 |batch_size 32 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/32 1688/1688 [==============================] - 6s 3ms/step - loss: 1.3671 - accuracy: 0.4858 - val_loss: 0.5711 - val_accuracy: 0.8453 - lr: 0.0010 Epoch 2/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.6508 - accuracy: 0.8077 - val_loss: 0.3636 - val_accuracy: 0.9013 - lr: 0.0010 Epoch 3/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.4829 - accuracy: 0.8718 - val_loss: 0.2816 - val_accuracy: 0.9233 - lr: 0.0010 Epoch 4/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.4010 - accuracy: 0.8955 - val_loss: 0.2329 - val_accuracy: 0.9352 - lr: 0.0010 Epoch 5/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.3457 - accuracy: 0.9092 - val_loss: 0.2134 - val_accuracy: 0.9397 - lr: 0.0010 Epoch 6/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.3183 - accuracy: 0.9162 - val_loss: 0.1969 - val_accuracy: 0.9440 - lr: 0.0010 Epoch 7/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2942 - accuracy: 0.9226 - val_loss: 0.1852 - val_accuracy: 0.9492 - lr: 0.0010 Epoch 8/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2740 - accuracy: 0.9274 - val_loss: 0.1703 - val_accuracy: 0.9505 - lr: 0.0010 Epoch 9/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2552 - accuracy: 0.9318 - val_loss: 0.1666 - val_accuracy: 0.9532 - lr: 0.0010 Epoch 10/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2431 - accuracy: 0.9355 - val_loss: 0.1608 - val_accuracy: 0.9555 - lr: 0.0010 Epoch 11/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2325 - accuracy: 0.9387 - val_loss: 0.1476 - val_accuracy: 0.9558 - lr: 0.0010 Epoch 12/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2248 - accuracy: 0.9408 - val_loss: 0.1520 - val_accuracy: 0.9552 - lr: 0.0010 Epoch 13/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2163 - accuracy: 0.9418 - val_loss: 0.1465 - val_accuracy: 0.9580 - lr: 0.0010 Epoch 14/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.2084 - accuracy: 0.9446 - val_loss: 0.1356 - val_accuracy: 0.9607 - lr: 0.0010 Epoch 15/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1982 - accuracy: 0.9473 - val_loss: 0.1351 - val_accuracy: 0.9600 - lr: 0.0010 Epoch 16/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1992 - accuracy: 0.9469 - val_loss: 0.1353 - val_accuracy: 0.9585 - lr: 0.0010 Epoch 17/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1830 - accuracy: 0.9518 - val_loss: 0.1322 - val_accuracy: 0.9613 - lr: 1.0000e-04 Epoch 18/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1781 - accuracy: 0.9514 - val_loss: 0.1297 - val_accuracy: 0.9615 - lr: 1.0000e-04 Epoch 19/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1761 - accuracy: 0.9521 - val_loss: 0.1297 - val_accuracy: 0.9613 - lr: 1.0000e-04 Epoch 20/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1719 - accuracy: 0.9540 - val_loss: 0.1317 - val_accuracy: 0.9613 - lr: 1.0000e-04 Epoch 21/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1692 - accuracy: 0.9547 - val_loss: 0.1315 - val_accuracy: 0.9612 - lr: 1.0000e-05 Epoch 22/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1724 - accuracy: 0.9548 - val_loss: 0.1315 - val_accuracy: 0.9615 - lr: 1.0000e-05 Epoch 23/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1734 - accuracy: 0.9529 - val_loss: 0.1314 - val_accuracy: 0.9618 - lr: 1.0000e-06 Epoch 24/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1730 - accuracy: 0.9541 - val_loss: 0.1314 - val_accuracy: 0.9618 - lr: 1.0000e-06 Epoch 25/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1697 - accuracy: 0.9542 - val_loss: 0.1314 - val_accuracy: 0.9618 - lr: 1.0000e-06 Epoch 26/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1718 - accuracy: 0.9538 - val_loss: 0.1314 - val_accuracy: 0.9618 - lr: 1.0000e-07 Epoch 27/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1716 - accuracy: 0.9540 - val_loss: 0.1314 - val_accuracy: 0.9618 - lr: 1.0000e-07 Epoch 28/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1712 - accuracy: 0.9539 - val_loss: 0.1314 - val_accuracy: 0.9618 - lr: 1.0000e-08 Epoch 29/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1743 - accuracy: 0.9539 - val_loss: 0.1314 - val_accuracy: 0.9618 - lr: 1.0000e-08 Epoch 30/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1697 - accuracy: 0.9541 - val_loss: 0.1314 - val_accuracy: 0.9618 - lr: 1.0000e-09 Epoch 31/32 1688/1688 [==============================] - 5s 3ms/step - loss: 0.1718 - accuracy: 0.9533 - val_loss: 0.1314 - val_accuracy: 0.9618 - lr: 1.0000e-09 Trial 87 Complete [00h 02m 32s] val_accuracy: 0.9618333578109741 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 13m 08s Search: Running Trial #88 Value |Best Value So Far |Hyperparameter 3 |3 |depth 512 |512 |width swish |swish |activation l2 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.2 |dropout sgd |adam |optimizer 0.01 |0.01 |learning_rate 64 |512 |batch_size 32 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/32 844/844 [==============================] - 4s 4ms/step - loss: 1.1430 - accuracy: 0.8444 - val_loss: 1.0467 - val_accuracy: 0.9262 - lr: 0.0100 Epoch 2/32 844/844 [==============================] - 4s 4ms/step - loss: 1.3585 - accuracy: 0.8603 - val_loss: 1.0216 - val_accuracy: 0.9295 - lr: 0.0100 Epoch 3/32 844/844 [==============================] - 4s 4ms/step - loss: 1.6084 - accuracy: 0.8471 - val_loss: 1.5711 - val_accuracy: 0.8885 - lr: 0.0100 Epoch 4/32 844/844 [==============================] - 4s 4ms/step - loss: 1.3795 - accuracy: 0.8700 - val_loss: 0.8905 - val_accuracy: 0.9363 - lr: 0.0100 Epoch 5/32 844/844 [==============================] - 4s 4ms/step - loss: 1.2218 - accuracy: 0.8680 - val_loss: 1.0429 - val_accuracy: 0.9257 - lr: 0.0100 Epoch 6/32 844/844 [==============================] - 4s 4ms/step - loss: 1.2312 - accuracy: 0.8667 - val_loss: 0.8810 - val_accuracy: 0.9262 - lr: 0.0100 Epoch 7/32 844/844 [==============================] - 4s 4ms/step - loss: 0.8283 - accuracy: 0.9221 - val_loss: 0.6089 - val_accuracy: 0.9560 - lr: 1.0000e-03 Epoch 8/32 844/844 [==============================] - 4s 4ms/step - loss: 0.6428 - accuracy: 0.9370 - val_loss: 0.5022 - val_accuracy: 0.9593 - lr: 1.0000e-03 Epoch 9/32 844/844 [==============================] - 4s 4ms/step - loss: 0.5288 - accuracy: 0.9445 - val_loss: 0.4216 - val_accuracy: 0.9628 - lr: 1.0000e-03 Epoch 10/32 844/844 [==============================] - 4s 4ms/step - loss: 0.4455 - accuracy: 0.9507 - val_loss: 0.3597 - val_accuracy: 0.9652 - lr: 1.0000e-03 Epoch 11/32 844/844 [==============================] - 4s 4ms/step - loss: 0.3827 - accuracy: 0.9536 - val_loss: 0.3116 - val_accuracy: 0.9685 - lr: 1.0000e-03 Epoch 12/32 844/844 [==============================] - 4s 4ms/step - loss: 0.3377 - accuracy: 0.9566 - val_loss: 0.2908 - val_accuracy: 0.9647 - lr: 1.0000e-03 Epoch 13/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2996 - accuracy: 0.9603 - val_loss: 0.2651 - val_accuracy: 0.9672 - lr: 1.0000e-03 Epoch 14/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2596 - accuracy: 0.9670 - val_loss: 0.2398 - val_accuracy: 0.9737 - lr: 1.0000e-04 Epoch 15/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2485 - accuracy: 0.9687 - val_loss: 0.2344 - val_accuracy: 0.9720 - lr: 1.0000e-04 Epoch 16/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2394 - accuracy: 0.9697 - val_loss: 0.2285 - val_accuracy: 0.9735 - lr: 1.0000e-04 Epoch 17/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2352 - accuracy: 0.9706 - val_loss: 0.2275 - val_accuracy: 0.9740 - lr: 1.0000e-05 Epoch 18/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2327 - accuracy: 0.9716 - val_loss: 0.2269 - val_accuracy: 0.9738 - lr: 1.0000e-05 Epoch 19/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2304 - accuracy: 0.9716 - val_loss: 0.2265 - val_accuracy: 0.9738 - lr: 1.0000e-05 Epoch 20/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2308 - accuracy: 0.9722 - val_loss: 0.2264 - val_accuracy: 0.9738 - lr: 1.0000e-06 Epoch 21/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2298 - accuracy: 0.9729 - val_loss: 0.2263 - val_accuracy: 0.9738 - lr: 1.0000e-06 Epoch 22/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2302 - accuracy: 0.9723 - val_loss: 0.2263 - val_accuracy: 0.9738 - lr: 1.0000e-07 Epoch 23/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2312 - accuracy: 0.9719 - val_loss: 0.2263 - val_accuracy: 0.9738 - lr: 1.0000e-07 Epoch 24/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2301 - accuracy: 0.9720 - val_loss: 0.2263 - val_accuracy: 0.9738 - lr: 1.0000e-08 Epoch 25/32 844/844 [==============================] - 4s 4ms/step - loss: 0.2304 - accuracy: 0.9716 - val_loss: 0.2263 - val_accuracy: 0.9738 - lr: 1.0000e-08 Trial 88 Complete [00h 01m 30s] val_accuracy: 0.9739999771118164 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 14m 38s Search: Running Trial #89 Value |Best Value So Far |Hyperparameter 2 |3 |depth 128 |512 |width swish |swish |activation l1 |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0 |0.2 |dropout adam |adam |optimizer 0.001 |0.01 |learning_rate 1024 |512 |batch_size 32 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/32 53/53 [==============================] - 1s 8ms/step - loss: 1.2994 - accuracy: 0.7878 - val_loss: 0.7188 - val_accuracy: 0.9048 - lr: 0.0010 Epoch 2/32 53/53 [==============================] - 0s 5ms/step - loss: 0.6443 - accuracy: 0.9157 - val_loss: 0.5919 - val_accuracy: 0.9240 - lr: 0.0010 Epoch 3/32 53/53 [==============================] - 0s 5ms/step - loss: 0.5544 - accuracy: 0.9308 - val_loss: 0.5308 - val_accuracy: 0.9370 - lr: 0.0010 Epoch 4/32 53/53 [==============================] - 0s 5ms/step - loss: 0.4980 - accuracy: 0.9405 - val_loss: 0.4838 - val_accuracy: 0.9410 - lr: 0.0010 Epoch 5/32 53/53 [==============================] - 0s 5ms/step - loss: 0.4565 - accuracy: 0.9478 - val_loss: 0.4462 - val_accuracy: 0.9467 - lr: 0.0010 Epoch 6/32 53/53 [==============================] - 0s 5ms/step - loss: 0.4233 - accuracy: 0.9534 - val_loss: 0.4219 - val_accuracy: 0.9502 - lr: 0.0010 Epoch 7/32 53/53 [==============================] - 0s 5ms/step - loss: 0.3964 - accuracy: 0.9580 - val_loss: 0.3977 - val_accuracy: 0.9533 - lr: 0.0010 Epoch 8/32 53/53 [==============================] - 0s 5ms/step - loss: 0.3763 - accuracy: 0.9606 - val_loss: 0.3866 - val_accuracy: 0.9560 - lr: 0.0010 Epoch 9/32 53/53 [==============================] - 0s 5ms/step - loss: 0.3566 - accuracy: 0.9634 - val_loss: 0.3625 - val_accuracy: 0.9580 - lr: 0.0010 Epoch 10/32 53/53 [==============================] - 0s 5ms/step - loss: 0.3411 - accuracy: 0.9659 - val_loss: 0.3551 - val_accuracy: 0.9573 - lr: 0.0010 Epoch 11/32 53/53 [==============================] - 0s 5ms/step - loss: 0.3288 - accuracy: 0.9666 - val_loss: 0.3450 - val_accuracy: 0.9583 - lr: 0.0010 Epoch 12/32 53/53 [==============================] - 0s 5ms/step - loss: 0.3157 - accuracy: 0.9692 - val_loss: 0.3320 - val_accuracy: 0.9617 - lr: 0.0010 Epoch 13/32 53/53 [==============================] - 0s 5ms/step - loss: 0.3036 - accuracy: 0.9707 - val_loss: 0.3194 - val_accuracy: 0.9622 - lr: 0.0010 Epoch 14/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2986 - accuracy: 0.9701 - val_loss: 0.3145 - val_accuracy: 0.9640 - lr: 0.0010 Epoch 15/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2883 - accuracy: 0.9725 - val_loss: 0.3038 - val_accuracy: 0.9663 - lr: 0.0010 Epoch 16/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2780 - accuracy: 0.9742 - val_loss: 0.2960 - val_accuracy: 0.9657 - lr: 0.0010 Epoch 17/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2701 - accuracy: 0.9750 - val_loss: 0.2964 - val_accuracy: 0.9643 - lr: 0.0010 Epoch 18/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2588 - accuracy: 0.9776 - val_loss: 0.2826 - val_accuracy: 0.9672 - lr: 1.0000e-04 Epoch 19/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2548 - accuracy: 0.9783 - val_loss: 0.2811 - val_accuracy: 0.9677 - lr: 1.0000e-04 Epoch 20/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2535 - accuracy: 0.9788 - val_loss: 0.2805 - val_accuracy: 0.9675 - lr: 1.0000e-04 Epoch 21/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2526 - accuracy: 0.9786 - val_loss: 0.2794 - val_accuracy: 0.9680 - lr: 1.0000e-04 Epoch 22/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2518 - accuracy: 0.9787 - val_loss: 0.2791 - val_accuracy: 0.9678 - lr: 1.0000e-04 Epoch 23/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2509 - accuracy: 0.9785 - val_loss: 0.2781 - val_accuracy: 0.9680 - lr: 1.0000e-04 Epoch 24/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2495 - accuracy: 0.9790 - val_loss: 0.2774 - val_accuracy: 0.9682 - lr: 1.0000e-05 Epoch 25/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2492 - accuracy: 0.9791 - val_loss: 0.2772 - val_accuracy: 0.9685 - lr: 1.0000e-05 Epoch 26/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2491 - accuracy: 0.9792 - val_loss: 0.2772 - val_accuracy: 0.9687 - lr: 1.0000e-05 Epoch 27/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2490 - accuracy: 0.9792 - val_loss: 0.2771 - val_accuracy: 0.9683 - lr: 1.0000e-05 Epoch 28/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2489 - accuracy: 0.9792 - val_loss: 0.2770 - val_accuracy: 0.9682 - lr: 1.0000e-05 Epoch 29/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2487 - accuracy: 0.9793 - val_loss: 0.2770 - val_accuracy: 0.9682 - lr: 1.0000e-06 Epoch 30/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2487 - accuracy: 0.9793 - val_loss: 0.2769 - val_accuracy: 0.9682 - lr: 1.0000e-06 Epoch 31/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2487 - accuracy: 0.9793 - val_loss: 0.2769 - val_accuracy: 0.9682 - lr: 1.0000e-07 Epoch 32/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2487 - accuracy: 0.9793 - val_loss: 0.2769 - val_accuracy: 0.9682 - lr: 1.0000e-07 Trial 89 Complete [00h 00m 09s] val_accuracy: 0.968666672706604 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 14m 48s Search: Running Trial #90 Value |Best Value So Far |Hyperparameter 2 |3 |depth 256 |512 |width linear |swish |activation none |l1_l2 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.3 |0.2 |dropout rmsprop |adam |optimizer 0.01 |0.01 |learning_rate 1024 |512 |batch_size 32 |32 |tuner/epochs 0 |11 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |2 |tuner/round Epoch 1/32 53/53 [==============================] - 1s 9ms/step - loss: 0.9767 - accuracy: 0.8035 - val_loss: 0.3363 - val_accuracy: 0.9085 - lr: 0.0100 Epoch 2/32 53/53 [==============================] - 0s 6ms/step - loss: 0.3496 - accuracy: 0.9010 - val_loss: 0.3077 - val_accuracy: 0.9180 - lr: 0.0100 Epoch 3/32 53/53 [==============================] - 0s 5ms/step - loss: 0.3275 - accuracy: 0.9077 - val_loss: 0.3191 - val_accuracy: 0.9137 - lr: 0.0100 Epoch 4/32 53/53 [==============================] - 0s 5ms/step - loss: 0.3320 - accuracy: 0.9069 - val_loss: 0.3345 - val_accuracy: 0.9083 - lr: 0.0100 Epoch 5/32 53/53 [==============================] - 0s 6ms/step - loss: 0.2917 - accuracy: 0.9173 - val_loss: 0.2932 - val_accuracy: 0.9200 - lr: 1.0000e-03 Epoch 6/32 53/53 [==============================] - 0s 6ms/step - loss: 0.2776 - accuracy: 0.9222 - val_loss: 0.2890 - val_accuracy: 0.9223 - lr: 1.0000e-03 Epoch 7/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2767 - accuracy: 0.9225 - val_loss: 0.2895 - val_accuracy: 0.9215 - lr: 1.0000e-03 Epoch 8/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2744 - accuracy: 0.9236 - val_loss: 0.2904 - val_accuracy: 0.9217 - lr: 1.0000e-03 Epoch 9/32 53/53 [==============================] - 0s 6ms/step - loss: 0.2714 - accuracy: 0.9233 - val_loss: 0.2887 - val_accuracy: 0.9230 - lr: 1.0000e-04 Epoch 10/32 53/53 [==============================] - 0s 6ms/step - loss: 0.2679 - accuracy: 0.9256 - val_loss: 0.2887 - val_accuracy: 0.9237 - lr: 1.0000e-04 Epoch 11/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2693 - accuracy: 0.9242 - val_loss: 0.2886 - val_accuracy: 0.9223 - lr: 1.0000e-04 Epoch 12/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2682 - accuracy: 0.9254 - val_loss: 0.2886 - val_accuracy: 0.9235 - lr: 1.0000e-04 Epoch 13/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2673 - accuracy: 0.9247 - val_loss: 0.2885 - val_accuracy: 0.9233 - lr: 1.0000e-05 Epoch 14/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2682 - accuracy: 0.9259 - val_loss: 0.2884 - val_accuracy: 0.9230 - lr: 1.0000e-05 Epoch 15/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2665 - accuracy: 0.9253 - val_loss: 0.2884 - val_accuracy: 0.9230 - lr: 1.0000e-06 Epoch 16/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2679 - accuracy: 0.9255 - val_loss: 0.2884 - val_accuracy: 0.9230 - lr: 1.0000e-06 Epoch 17/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2655 - accuracy: 0.9259 - val_loss: 0.2884 - val_accuracy: 0.9230 - lr: 1.0000e-07 Epoch 18/32 53/53 [==============================] - 0s 5ms/step - loss: 0.2681 - accuracy: 0.9254 - val_loss: 0.2884 - val_accuracy: 0.9230 - lr: 1.0000e-07 Trial 90 Complete [00h 00m 06s] val_accuracy: 0.9236666560173035 Best val_accuracy So Far: 0.9850000143051147 Total elapsed time: 00h 14m 54s Results summary Results in tuning/bandit Showing 10 best trials Trial summary Hyperparameters: depth: 3 width: 512 activation: swish regularization_function: l1_l2 regularization_penalty: 1e-05 dropout: 0.2 optimizer: adam learning_rate: 0.01 batch_size: 512 tuner/epochs: 32 tuner/initial_epoch: 11 tuner/bracket: 2 tuner/round: 2 tuner/trial_id: 0067 Score: 0.9850000143051147 Trial summary Hyperparameters: depth: 1 width: 256 activation: swish regularization_function: l1 regularization_penalty: 1e-05 dropout: 0.4 optimizer: sgd learning_rate: 0.01 batch_size: 128 tuner/epochs: 32 tuner/initial_epoch: 11 tuner/bracket: 3 tuner/round: 3 tuner/trial_id: 0048 Score: 0.9793333411216736 Trial summary Hyperparameters: depth: 3 width: 128 activation: sigmoid regularization_function: none regularization_penalty: 1e-05 dropout: 0.2 optimizer: sgd learning_rate: 0.01 batch_size: 128 tuner/epochs: 32 tuner/initial_epoch: 11 tuner/bracket: 2 tuner/round: 2 tuner/trial_id: 0068 Score: 0.9793333411216736 Trial summary Hyperparameters: depth: 3 width: 128 activation: sigmoid regularization_function: none regularization_penalty: 1e-05 dropout: 0.2 optimizer: sgd learning_rate: 0.01 batch_size: 128 tuner/epochs: 11 tuner/initial_epoch: 4 tuner/bracket: 2 tuner/round: 1 tuner/trial_id: 0053 Score: 0.9776666760444641 Trial summary Hyperparameters: depth: 2 width: 128 activation: sigmoid regularization_function: none regularization_penalty: 1e-05 dropout: 0.2 optimizer: adam learning_rate: 0.01 batch_size: 256 tuner/epochs: 32 tuner/initial_epoch: 11 tuner/bracket: 3 tuner/round: 3 tuner/trial_id: 0047 Score: 0.9771666526794434 Trial summary Hyperparameters: depth: 3 width: 512 activation: swish regularization_function: l1_l2 regularization_penalty: 1e-05 dropout: 0.2 optimizer: adam learning_rate: 0.01 batch_size: 512 tuner/epochs: 11 tuner/initial_epoch: 4 tuner/bracket: 2 tuner/round: 1 tuner/trial_id: 0059 Score: 0.9756666421890259 Trial summary Hyperparameters: depth: 1 width: 256 activation: swish regularization_function: l1 regularization_penalty: 1e-05 dropout: 0.4 optimizer: sgd learning_rate: 0.01 batch_size: 128 tuner/epochs: 11 tuner/initial_epoch: 4 tuner/bracket: 3 tuner/round: 2 tuner/trial_id: 0038 Score: 0.9750000238418579 Trial summary Hyperparameters: depth: 2 width: 128 activation: sigmoid regularization_function: none regularization_penalty: 1e-05 dropout: 0.2 optimizer: adam learning_rate: 0.01 batch_size: 256 tuner/epochs: 11 tuner/initial_epoch: 4 tuner/bracket: 3 tuner/round: 2 tuner/trial_id: 0037 Score: 0.9745000004768372 Trial summary Hyperparameters: depth: 3 width: 512 activation: swish regularization_function: l2 regularization_penalty: 0.0001 dropout: 0.30000000000000004 optimizer: sgd learning_rate: 0.01 batch_size: 64 tuner/epochs: 32 tuner/initial_epoch: 0 tuner/bracket: 0 tuner/round: 0 Score: 0.9739999771118164 Trial summary Hyperparameters: depth: 1 width: 512 activation: relu regularization_function: none regularization_penalty: 1e-05 dropout: 0.1 optimizer: sgd learning_rate: 0.01 batch_size: 256 tuner/epochs: 11 tuner/initial_epoch: 4 tuner/bracket: 3 tuner/round: 2 tuner/trial_id: 0034 Score: 0.9729999899864197 {'depth': 3, 'width': 512, 'activation': 'swish', 'regularization_function': 'l1_l2', 'regularization_penalty': 1e-05, 'dropout': 0.2, 'optimizer': 'adam', 'learning_rate': 0.01, 'batch_size': 512, 'tuner/epochs': 32, 'tuner/initial_epoch': 11, 'tuner/bracket': 2, 'tuner/round': 2, 'tuner/trial_id': '0067'} 313/313 [==============================] - 1s 1ms/step Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense (Dense) (None, 512) 401920 dropout (Dropout) (None, 512) 0 dense_1 (Dense) (None, 512) 262656 dropout_1 (Dropout) (None, 512) 0 dense_2 (Dense) (None, 512) 262656 dropout_2 (Dropout) (None, 512) 0 dense_3 (Dense) (None, 10) 5130 ================================================================= Total params: 932,362 Trainable params: 932,362 Non-trainable params: 0 _________________________________________________________________ real 15m11.864s user 18m56.489s sys 2m29.174s deeplearn@ML-RefVm-967342:~/mnist$ kaggle competitions submit -c ml530-2022-fall-mnist -f predictions.csv -m "'depth': 3" Warning: Your Kaggle API key is readable by other users on this system! To fix this, you can run 'chmod 600 /home/deeplearn/.kaggle/kaggle.json' 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 78.1k/78.1k [00:00<00:00, 144kB/s] Successfully submitted to ml530-2022-fall-mnistdeeplearn@ML-RefVm-967342:~/mnist$