deeplearn@ML-RefVm-967342:~/ptb/ptb-data$ wget https://cross-entropy.net/ML530/ptb-vocabulary.py.txt --2022-11-23 02:05:40-- https://cross-entropy.net/ML530/ptb-vocabulary.py.txt Resolving cross-entropy.net (cross-entropy.net)... 107.180.57.14 Connecting to cross-entropy.net (cross-entropy.net)|107.180.57.14|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1764 (1.7K) [text/plain] Saving to: ‘ptb-vocabulary.py.txt’ ptb-vocabulary.py.txt 100%[============================================================>] 1.72K --.-KB/s in 0s 2022-11-23 02:05:40 (1.37 GB/s) - ‘ptb-vocabulary.py.txt’ saved [1764/1764] deeplearn@ML-RefVm-967342:~/ptb/ptb-data$ time python ptb-vocabulary.py.txt real 0m25.121s user 0m23.776s sys 0m1.873s deeplearn@ML-RefVm-967342:~/ptb/ptb-data$ wget https://cross-entropy.net/ML530/ptb-sentences.py.txt --2022-11-23 02:06:19-- https://cross-entropy.net/ML530/ptb-sentences.py.txt Resolving cross-entropy.net (cross-entropy.net)... 107.180.57.14 Connecting to cross-entropy.net (cross-entropy.net)|107.180.57.14|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1494 (1.5K) [text/plain] Saving to: ‘ptb-sentences.py.txt’ ptb-sentences.py.txt 100%[============================================================>] 1.46K --.-KB/s in 0s 2022-11-23 02:06:19 (1.26 GB/s) - ‘ptb-sentences.py.txt’ saved [1494/1494] deeplearn@ML-RefVm-967342:~/ptb/ptb-data$ time python ptb-sentences.py.txt real 0m27.803s user 0m26.229s sys 0m2.202s deeplearn@ML-RefVm-967342:~/ptb/ptb-data$ wget https://cross-entropy.net/ML530/ptb-tensors.py.txt --2022-11-23 02:07:00-- https://cross-entropy.net/ML530/ptb-tensors.py.txt Resolving cross-entropy.net (cross-entropy.net)... 107.180.57.14 Connecting to cross-entropy.net (cross-entropy.net)|107.180.57.14|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1103 (1.1K) [text/plain] Saving to: ‘ptb-tensors.py.txt’ ptb-tensors.py.txt 100%[============================================================>] 1.08K --.-KB/s in 0s 2022-11-23 02:07:00 (877 MB/s) - ‘ptb-tensors.py.txt’ saved [1103/1103] deeplearn@ML-RefVm-967342:~/ptb/ptb-data$ time python ptb-tensors.py.txt real 1m26.345s user 1m25.233s sys 0m1.721s deeplearn@ML-RefVm-967342:~/ptb/ptb-data$ wget https://cross-entropy.net/ML530/ptb-train.py.txt --2022-11-23 02:08:44-- https://cross-entropy.net/ML530/ptb-train.py.txt Resolving cross-entropy.net (cross-entropy.net)... 107.180.57.14 Connecting to cross-entropy.net (cross-entropy.net)|107.180.57.14|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1902 (1.9K) [text/plain] Saving to: ‘ptb-train.py.txt’ ptb-train.py.txt 100%[============================================================>] 1.86K --.-KB/s in 0s 2022-11-23 02:08:44 (1.04 GB/s) - ‘ptb-train.py.txt’ saved [1902/1902] deeplearn@ML-RefVm-967342:~/ptb/ptb-data$ time python ptb-train.py.txt 2022-11-23 02:08:56.352812: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-11-23 02:08:56.970413: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 10794 MB memory: -> device: 0, name: Tesla K80, pci bus id: 0001:00:00.0, compute capability: 3.7 Model: "model" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) [(None, 250)] 0 embedding (Embedding) (None, 250, 1024) 10240000 lstm (LSTM) (None, 1024) 8392704 tied_embedding (TiedEmbeddi (None, 10000) 10240000 ng) ================================================================= Total params: 18,632,704 Trainable params: 18,632,704 Non-trainable params: 0 _________________________________________________________________ Epoch 1/4 2022-11-23 02:09:04.006203: W tensorflow/core/common_runtime/forward_type_inference.cc:231] Type inference failed. This indicates an invalid graph that escaped type checking. Error message: INVALID_ARGUMENT: expected compatible input types, but input 1: type_id: TFT_OPTIONAL args { type_id: TFT_PRODUCT args { type_id: TFT_TENSOR args { type_id: TFT_LEGACY_VARIANT } } } is neither a subtype nor a supertype of the combined inputs preceding it: type_id: TFT_OPTIONAL args { type_id: TFT_PRODUCT args { type_id: TFT_TENSOR args { type_id: TFT_INT8 } } } while inferring type of node 'cond_40/output/_26' 2022-11-23 02:09:04.677576: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8500 3805/3805 [==============================] - 1180s 309ms/step - loss: 5.1800 - sparse_categorical_crossentropy: 4.9127 - val_loss: 4.7807 - val_sparse_categorical_crossentropy: 4.4809 Epoch 2/4 3805/3805 [==============================] - 1174s 309ms/step - loss: 4.5673 - sparse_categorical_crossentropy: 4.2310 - val_loss: 4.6369 - val_sparse_categorical_crossentropy: 4.2781 Epoch 3/4 3805/3805 [==============================] - 1175s 309ms/step - loss: 4.3332 - sparse_categorical_crossentropy: 3.9393 - val_loss: 4.6293 - val_sparse_categorical_crossentropy: 4.2175 Epoch 4/4 3805/3805 [==============================] - 1175s 309ms/step - loss: 4.1724 - sparse_categorical_crossentropy: 3.7274 - val_loss: 4.6678 - val_sparse_categorical_crossentropy: 4.2059 3723/3723 [==============================] - 75s 20ms/step real 79m54.537s user 50m31.982s sys 1m29.544s deeplearn@ML-RefVm-967342:~/ptb/ptb-data$ kaggle competitions submit -c ml530-2022-fall-ptb -f predictions.csv -m "1:19:55" 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 2.14M/2.14M [00:00<00:00, 2.88MB/s] Successfully submitted to ml530-2022-fall-ptbdeeplearn@ML-RefVm-967342:~/ptb/ptb-data$ deeplearn@ML-RefVm-967342:~/ptb/ptb-data$