Tensorflowhed-tutorial-for-document-scanning

Posted 2022-10-17 Taily老段

hed-tutorial-for-document-scanning

(py27tf) bash-3.2$ ls
README.md
checkpoint
const.py
const.pyc
dataset
evaluate_hed.py
freeze_model.py
generate_training_dataset
hed_net.py
hed_net.pyc
how_to_build_tensorflow_and_change_namespace_of_protobuf.txt
input_pipeline.py
input_pipeline.pyc
ios_demo
mobilenet.py
mobilenet.pyc
preprocess_generate_training_dataset.py
run_preprocess.sh
run_train.sh
sample_images
test_image
train_hed.py
util.py
util.pyc
(py27tf) bash-3.2$ pwd
/Users/taily/githubproj/hed-tutorial-for-document-scanning
(py27tf) bash-3.2$ gshuf ./dataset/generate_sample_by_ios_image_size_256_256_thickness_0.2.csv > ./dataset/temp.txt
(py27tf) bash-3.2$ gshuf ./dataset/temp.txt > ./dataset/generate_sample_by_ios_image_size_256_256_thickness_0.2.csv
(py27tf) bash-3.2$ sh run_train.sh
TensorFlow Version: 1.9.0
###########################################
###########################################
dataset_root_dir is: dataset
os.path.join(FLAGS.dataset_root_dir, '') is: dataset/
csv_path is: dataset/generate_sample_by_ios_image_size_256_256_thickness_0.2.csv
checkpoint_dir is: ./checkpoint
debug_image_dir is: ./debug_output_image
###########################################
###########################################
image_tensor shape is: (4, 256, 256, 3)
dsn_fuse shape is: (4, 256, 256, 1)
cost shape is: ()



############################################################
2019-01-22 21:46:42.382968: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2019-01-22 21:46:42.383826: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 8. Tune using inter_op_parallelism_threads for best performance.
restore from latest checkpoint file : ./checkpoint/hed.ckpt-130

Start train...
batch_size is: 4
iterations is: 90000000
display-step is: 5
learning-rate is: 0.0005
++ use batch norm
2019-01-22 21:46:50.439666: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at whole_file_read_ops.cc:114 : Not found: dataset/Documents/j0AHxUSqtL_random_size_118_101_1_annotation_thresh_gray.png; No such file or directory
2019-01-22 21:46:50.439669: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at whole_file_read_ops.cc:114 : Not found: dataset/Documents/j0AHxUSqtL_random_size_118_101_1_color.jpg; No such file or directory
Traceback (most recent call last):
  File "train_hed.py", line 205, in <module>
    feed_dict=feed_dict_to_use)
  File "/anaconda2/envs/py27tf/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 900, in run
    run_metadata_ptr)
  File "/anaconda2/envs/py27tf/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1135, in _run
    feed_dict_tensor, options, run_metadata)
  File "/anaconda2/envs/py27tf/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1316, in _do_run
    run_metadata)
  File "/anaconda2/envs/py27tf/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1335, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 4, current size 0)
	 [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

Caused by op u'shuffle_batch', defined at:
  File "train_hed.py", line 98, in <module>
    FLAGS.batch_size)
  File "/Users/taily/githubproj/hed-tutorial-for-document-scanning/input_pipeline.py", line 71, in fix_size_image_pipeline
    min_after_dequeue = min_after_dequeue)
  File "/anaconda2/envs/py27tf/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 1300, in shuffle_batch
    name=name)
  File "/anaconda2/envs/py27tf/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 846, in _shuffle_batch
    dequeued = queue.dequeue_many(batch_size, name=name)
  File "/anaconda2/envs/py27tf/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 483, in dequeue_many
    self._queue_ref, n=n, component_types=self._dtypes, name=name)
  File "/anaconda2/envs/py27tf/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3480, in queue_dequeue_many_v2
    component_types=component_types, timeout_ms=timeout_ms, name=name)
  File "/anaconda2/envs/py27tf/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/anaconda2/envs/py27tf/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3414, in create_op
    op_def=op_def)
  File "/anaconda2/envs/py27tf/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1740, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

OutOfRangeError (see above for traceback): RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 4, current size 0)
	 [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

(py27tf) bash-3.2$ 

(py27tf) bash-3.2$ gshuf ./dataset/Documents.csv > ./dataset/temp.txt
(py27tf) bash-3.2$ gshuf ./dataset/temp.txt > ./dataset/Documents.csv
(py27tf) bash-3.2$ sh run_train.sh
TensorFlow Version: 1.9.0
###########################################
###########################################
dataset_root_dir is: dataset
os.path.join(FLAGS.dataset_root_dir, '') is: dataset/
csv_path is: dataset/Documents.csv
checkpoint_dir is: ./checkpoint
debug_image_dir is: ./debug_output_image
###########################################
###########################################
image_tensor shape is: (4, 256, 256, 3)
dsn_fuse shape is: (4, 256, 256, 1)
cost shape is: ()



############################################################
2019-01-22 21:58:17.180934: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2019-01-22 21:58:17.181668: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 8. Tune using inter_op_parallelism_threads for best performance.
restore from latest checkpoint file : ./checkpoint/hed.ckpt-130

Start train...
batch_size is: 4
iterations is: 90000000
display-step is: 5
learning-rate is: 0.0005
++ use batch norm
Step: 0, Current Mean Loss: 0.00144546641968
Step: 5, Current Mean Loss: 0.00107343564741
Step: 10, Current Mean Loss: 0.00129083264619
Step: 15, Current Mean Loss: 0.00104159826878
Step: 20, Current Mean Loss: 0.00137554691173
Step: 25, Current Mean Loss: 0.00208519585431
Step: 30, Current Mean Loss: 0.00133184040897
Step: 35, Current Mean Loss: 0.00169059564359
Step: 40, Current Mean Loss: 0.00172041752376
Step: 45, Current Mean Loss: 0.0021527602803
Step: 50, Current Mean Loss: 0.00364822335541
Step: 55, Current Mean Loss: 0.00226333946921
Step: 60, Current Mean Loss: 0.00129697239026
Step: 65, Current Mean Loss: 0.0053047221154
Step: 70, Current Mean Loss: 0.00146268832032
Step: 75, Current Mean Loss: 0.00175709614996
Step: 80, Current Mean Loss: 0.00176999950781
Step: 85, Current Mean Loss: 0.00185493286699
Step: 90, Current Mean Loss: 0.0024897470139
Step: 95, Current Mean Loss: 0.00242765760049
Step: 100, Current Mean Loss: 0.00135294231586
Step: 105, Current Mean Loss: 0.0012677735649
Step: 110, Current Mean Loss: 0.00466330628842
Step: 115, Current Mean Loss: 0.00320365303196
Step: 120, Current Mean Loss: 0.00117148994468
Step: 125, Current Mean Loss: 0.00102473050356
Step: 130, Current Mean Loss: 0.00296807009727
Step: 135, Current Mean Loss: 0.00162225530948
Step: 140, Current Mean Loss: 0.00163191650063
Step: 145, Current Mean Loss: 0.00130305998027
Step: 150, Current Mean Loss: 0.00278477417305
Step: 155, Current Mean Loss: 0.00198156945407
Step: 160, Current Mean Loss: 0.00142371770926
Step: 165, Current Mean Loss: 0.00126392161474
Step: 170, Current Mean Loss: 0.00267102662474