代码之家  ›  专栏  ›  技术社区  ›  lppier

向LSTMCell馈送初始状态

  •  0
  • lppier  · 技术社区  · 8 年前

    我在这里引用代码 https://github.com/martin-gorner/tensorflow-rnn-shakespeare/blob/master/rnn_train.py 我正在尝试将单元格从GRUCell转换为LSTMCell。下面是代码的摘录。

    # input state
    Hin = tf.placeholder(tf.float32, [None, INTERNALSIZE * NLAYERS], name='Hin')  # [ BATCHSIZE, INTERNALSIZE * NLAYERS]
    
    # using a NLAYERS=3 layers of GRU cells, unrolled SEQLEN=30 times
    # dynamic_rnn infers SEQLEN from the size of the inputs Xo
    
    # How to properly apply dropout in RNNs: see README.md
    cells = [rnn.GRUCell(INTERNALSIZE) for _ in range(NLAYERS)]
    
    # "naive dropout" implementation
    dropcells = [rnn.DropoutWrapper(cell, input_keep_prob=pkeep) for cell in cells]
    multicell = rnn.MultiRNNCell(dropcells, state_is_tuple=False)
    multicell = rnn.DropoutWrapper(multicell, output_keep_prob=pkeep)  # dropout for the softmax layer
    
    Yr, H = tf.nn.dynamic_rnn(multicell, Xo, dtype=tf.float32, initial_state=Hin)
    # Yr: [ BATCHSIZE, SEQLEN, INTERNALSIZE ]
    # H:  [ BATCHSIZE, INTERNALSIZE*NLAYERS ] # this is the last state in the sequence
    
    H = tf.identity(H, name='H')  # just to give it a name
    

    我知道LSTMCell有两种状态,单元状态C和输出状态H。我想做的是用两种状态的元组来填充初始的\u状态。 我怎样才能以正确的方式做到这一点?我尝试了各种方法,但总是遇到tensorflow错误。

    编辑:这是尝试之一:

    # inputs
    X = tf.placeholder(tf.uint8, [None, None], name='X')  # [ BATCHSIZE, SEQLEN ]
    Xo = tf.one_hot(X, ALPHASIZE, 1.0, 0.0)  # [ BATCHSIZE, SEQLEN, ALPHASIZE ]
    # expected outputs = same sequence shifted by 1 since we are trying to predict the next character
    Y_ = tf.placeholder(tf.uint8, [None, None], name='Y_')  # [ BATCHSIZE, SEQLEN ]
    Yo_ = tf.one_hot(Y_, ALPHASIZE, 1.0, 0.0)  # [ BATCHSIZE, SEQLEN, ALPHASIZE ]
    # input state
    Hin = tf.placeholder(tf.float32, [None, INTERNALSIZE * NLAYERS], name='Hin')  # [ BATCHSIZE, INTERNALSIZE * NLAYERS]
    Cin = tf.placeholder(tf.float32, [None, INTERNALSIZE * NLAYERS], name='Cin')
    initial_state = tf.nn.rnn_cell.LSTMStateTuple(Cin, Hin)
    # using a NLAYERS=3 layers of GRU cells, unrolled SEQLEN=30 times
    # dynamic_rnn infers SEQLEN from the size of the inputs Xo
    
    # How to properly apply dropout in RNNs: see README.md
    cells = [rnn.LSTMCell(INTERNALSIZE) for _ in range(NLAYERS)]
    
    # "naive dropout" implementation
    dropcells = [rnn.DropoutWrapper(cell, input_keep_prob=pkeep) for cell in cells]
    multicell = rnn.MultiRNNCell(dropcells, state_is_tuple=True)
    multicell = rnn.DropoutWrapper(multicell, output_keep_prob=pkeep)  # dropout for the softmax layer
    
    Yr, H = tf.nn.dynamic_rnn(multicell, Xo, dtype=tf.float32, initial_state=initial_state)
    

    它说“TypeError:‘Tensor’对象不可编辑。”

    谢谢

    1 回复  |  直到 8 年前
        1
  •  2
  •   parsethis    8 年前

    发生此错误的原因是您必须为每一层提供一个元组(占位符) 分别地 构建图形时,在进行培训时,必须提供第一层的状态。

    错误是:我需要迭代 元组 因为你有多个单元格,我需要初始化它们的所有状态,但我看到的只是张量,我无法迭代。

    此代码段显示如何在构建图形时设置占位符:

    state_size = 10
    num_layers = 3
    
    X = tf.placeholder(tf.float32, [None, 100, 10])
    
    # the second dimension is size 2 and represents
    # c, m ( the cell and hidden state ) 
    # set the batch_size to None
    state_placeholder = tf.placeholder(tf.float32, [num_layers, 2, 
                                        None, state_size])
    # l is number of layers placeholders 
    l = tf.unstack(state_placeholder, axis=0)
    
    then we create a tuple of LSTMStateTuple for each layer
    rnn_tuple_state = tuple(
             [rnn.LSTMStateTuple(l[idx][0],l[idx][1])
              for idx in range(num_layers)]
    )
    
    # I had to set resuse = True here : tf.__version__ 1.7.0
    cells  = [rnn.LSTMCell(10, reuse=True)] * num_layers
    mc = rnn.MultiRNNCell(cells, state_is_tuple=True)
    
    outputs, state = tf.nn.dynamic_rnn(cell=mc,
                                       inputs=X,
                                       initial_state=rnn_tuple_state,
                                       dtype=tf.float32)
    

    以下是 docs :

    initial\u state:(可选)RNN的初始状态。如果 单间牢房state\u size是一个整数,它必须是适当的张量 类型和形状[batch\u size,cell.state\u size]。

    因此,我们最终为每个单元格(层)创建了一个具有必要大小的占位符元组。(batch\u size,state\u size),其中batch\u size=无。 我对此进行了阐述 answer

    推荐文章