码迷,mamicode.com
首页 > 其他好文 > 详细

Operation was explicitly assigned to /job:ps/task:0/device:CPU:0 but available devices are [ /job:localhost/replica:0/task:0/cpu:0 ]

时间:2018-10-14 19:09:11      阅读:1212      评论:0      收藏:0      [点我收藏+]

标签:assigned   ida   lidar   flag   原因   names   ted   exp   error:   


训练启动时报错:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device for operation ‘save/RestoreV2_10‘: Operation was explicitly assigned to /job:ps/task:0/device:CPU:0 
but available devices are [ /job:localhost/replica:0/task:0/cpu:0 ]. Make sure the device specification refers to a valid device.
[[Node: save/RestoreV2_10 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:ps/task:0/device:CPU:0"](save/Const, save/RestoreV2_10/tensor_names, save/RestoreV2_10/shape_and_slices)]]

原因:available devices are [ /job:localhost/replica:0/task:0/cpu:0 ]表示tf.Session没有连接到tf.train.Server,In particular, it seems to be a local (or "direct") session that can only access devices in the local process.

解决办法要解决这个问题,需要在创建session时添加server.target。例如:

# Creating a session explicitly.
with tf.Session(server.target) as sess:
# ...

# Using a `tf.train.Supervisor` called `sv`.
with sv.managed_session(server.target):
# ...

# Using a `tf.train.MonitoredTrainingSession`.
with tf.train.MonitoredTrainingSession(server.target):
# ...

 

我们的代码

with tf.train.MonitoredTrainingSession(
  # is_chief=is_chief,
  checkpoint_dir=checkpoint_dir,
  save_checkpoint_secs=FLAGS.save_interval_secs,
  save_summaries_steps=100,
  save_summaries_secs=None,
  config=sess_config,
  hooks=hooks) as sess:

 

MonitoredTrainingSession中没有指定:

master=server.target

最终代码是:

with tf.train.MonitoredTrainingSession(
  master=server.target,
  is_chief=is_chief,
  checkpoint_dir=checkpoint_dir,
  save_checkpoint_secs=FLAGS.save_interval_secs,
  save_summaries_steps=100,
  save_summaries_secs=None,
  config=sess_config,
  hooks=hooks) as sess:

  

 

参考: https://stackoverflow.com/questions/42397370/distributed-tensorflow-save-fails-no-device

 

Operation was explicitly assigned to /job:ps/task:0/device:CPU:0 but available devices are [ /job:localhost/replica:0/task:0/cpu:0 ]

标签:assigned   ida   lidar   flag   原因   names   ted   exp   error:   

原文地址:https://www.cnblogs.com/lixiaolun/p/9787051.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!