2020-09-04 11:38:20.802072: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1 2020-09-04 11:38:23.400467: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1 2020-09-04 11:38:23.457175: E tensorflow/stream_executor/cuda/cuda_driver.cc:314] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected 2020-09-04 11:38:23.457270: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (9badf6d81adc): /proc/driver/nvidia/version does not exist 2020-09-04 11:38:23.486693: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2200000000 Hz 2020-09-04 11:38:23.486990: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x204ca00 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-09-04 11:38:23.487025: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/inception_v3/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5 87916544/87910968 [==============================] - 2s 0us/step Traceback (most recent call last): File "Predictor.py", line 96, in <module> main() File "Predictor.py", line 41, in main result = predictor(args.input, encoder=encoder, decoder=decoder, optimizer=optimizer, tokenizer=tokenizer) File "Predictor.py", line 61, in predictor result = evaluate(path, encoder=encoder, decoder=decoder, tokenizer=tokenizer) File "Predictor.py", line 82, in evaluate predictions, hidden, attention_weights = decoder(dec_input, features, hidden) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py", line 985, in __call__ outputs = call_fn(inputs, *args, **kwargs) File "/content/MyDrive/My Drive/setup/Predictions/I2S_Model.py", line 46, in call x = self.embedding(x) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py", line 982, in __call__ self._maybe_build(inputs) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py", line 2643, in _maybe_build self.build(input_shapes) # pylint:disable=not-callable File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/tf_utils.py", line 323, in wrapper output_shape = fn(instance, input_shape) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/embeddings.py", line 149, in build constraint=self.embeddings_constraint) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py", line 614, in add_weight caching_device=caching_device) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/base.py", line 731, in _add_variable_with_custom_getter name=name, shape=shape) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/base.py", line 798, in _preload_simple_restoration checkpoint_position=checkpoint_position, shape=shape) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/base.py", line 75, in __init__ self.wrapped_value.set_shape(shape) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 1209, in set_shape (self.shape, shape)) ValueError: Tensor's shape (10005, 600) is not compatible with supplied shape (69, 600) WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.embedding.embeddings WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.gru.state_spec WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.fc1.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.fc1.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.fc2.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.fc2.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.gru.cell.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.gru.cell.recurrent_kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.gru.cell.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.embedding.embeddings WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.fc1.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.fc1.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.fc2.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.fc2.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.gru.cell.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.gru.cell.recurrent_kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.gru.cell.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.embedding.embeddings WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.fc1.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.fc1.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.fc2.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.fc2.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.gru.cell.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.gru.cell.recurrent_kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.gru.cell.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
자료가 날라갔는데.. 다시 정리해보면..
smiles를 벡터화 시키지 못했다.
smiles를 어떻게 token화 시킬 것인가?
https://pypi.org/project/SmilesPE/
SmilesPE
Tokenize SMILES with substructure units
pypi.org
https://github.com/EBjerrum/SMILES-enumeration
EBjerrum/SMILES-enumeration
SMILES enumeration for QSAR modelling using LSTM recurrent neural networks - EBjerrum/SMILES-enumeration
github.com
둘중에 어떤것을 써 볼것인가?
첫번째것을 일단 써보자.
그리고 논문에서는 어떻게 smiles로 translate back했는지 알아보자.
smilesPE를 이용해서 smile을 token화 하는 과정을 추가하고 기존 코드 그대로 keras의 Tokenizer를 그대로 이용하였다.
spe_vob= codecs.open('/content/MyDrive/My Drive/setup/Image2SMILES/SPE_ChEMBL.txt')
spe = SPE_Tokenizer(spe_vob)
tkd_train_captions = []
for smile in train_captions :
smile_tkd = spe.tokenize(smile)
tkd_train_captions.append('<start> ' + smile_tkd +' <end>')
max_voc = 10000
#Creating tokenizer with defined characters
tokenizer = tf.keras.preprocessing.text.Tokenizer(num_words=max_voc,oov_token="<unk>",filters='!"$&:;?^`{}~ ',lower=False)
tokenizer.fit_on_texts(train_captions)
train_seqs = tokenizer.texts_to_sequences(train_captions)
tokenizer.word_index['<pad>'] = 0
tokenizer.index_word[0] = '<pad>'
cap_vector = tf.keras.preprocessing.sequence.pad_sequences(train_seqs, padding='post')
max_length = calc_max_length(train_seqs)
과연 학습이 잘 되고. 에러가 뜨지 않을지 궁금하다. 지금 학습중..
tokenizer가 뭔지 알아야 했으므로 다음 사이트들을 참고하였다.
https://subinium.github.io/Keras-6-1/
알고보니 인코딩이 또 이렇게 되어있었다...왜지?
train_captions = tkd_train_captions --> 이렇게 바꿔줬어야하는데 바꿔주질 않아서..
실행을 했는데 왜 되다 마는 것일까...
아니!! 에러가 뜨는것도 아니고 왜 실행이 안되는거냐고!
왜 두개가 차이가 나는 것인가?
#Splitting the text file and saving SMILES as Captions
for line in smiles.split('\n'):
tokens = line.split(',')
image_id = str(tokens[0])+'.png'
try:
caption = '<start> ' + str(tokens[1].rstrip()) + ' <end>'
except IndexError as e:
print (e,flush=True)
full_image_path = PATH + image_id
all_img_name.append(full_image_path)
all_smiles.append(caption)이부분에서 잘못 되었다. deepsmiles에서 <strat> <end>를 우선 빼주고 token으로 나눈 다음에 추가 했어야한다.
깨달았다!! 이미 5 epoch을 한 상태였음. 그래서 안되었던 것임.
start_epoch = 0
#Loading checkpoint to last saved
if ckpt_manager.latest_checkpoint:
ckpt.restore(tf.train.latest_checkpoint(checkpoint_path))
start_epoch = int(ckpt_manager.latest_checkpoint.split('-')[-1])tensorflow.python.framework.errors_impl.UnknownError: OSError: [Errno 5] Input/output error: '/content/MyDrive/My Drive/setup/Data/train_npy/train_339.png.npy' Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/script_ops.py", line 244, in __call__ ret = func(*args) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/autograph/impl/api.py", line 302, in wrapper return func(*args, **kwargs) File "./Image2SMILES/Trainer_Image2Smiles.py", line 63, in map_func img_tensor = np.load(img_name.decode('utf-8').replace('/Train_Images', '/train_npy/')+'.npy') File "/usr/local/lib/python3.6/dist-packages/numpy/lib/npyio.py", line 428, in load fid = open(os_fspath(file), "rb") OSError: [Errno 5] Input/output error: '/content/MyDrive/My Drive/setup/Data/train_npy/train_339.png.npy' [[{{node PyFunc}}]] ^
또 에러가 떳나.. 그래서 그냥 ipynb로 하나하나 실행했더니 학습이 잘 되고 있다.
ValueError: Tensor's shape (399, 600) is not compatible with supplied shape (69, 600)
또 안된다. ㅜㅜ
--> tokenizer일 때문일 가능성.
--> tokenizer를 불러와서 학습해 봐야겠다.
저장되있던 tokenizer를 확인 해 보아야 겠다. -->코드 상에 있는 것과 똑같은 tokenizer이다..
--> tokenizer를 저장한 이유는. predictor에서 쓰기 위함.