代码之家 › 专栏 › 技术社区 › ytrewq

PyTorch:传递numpy数组进行权重初始化

rnn pytorch initialization numpy python

ytrewq · 技术社区 · 7 年前

我想用np数组初始化RNN的参数。

在下面的例子中,我想通过 w 参数 rnn . 我知道pytorch提供了很多初始化方法,比如Xavier、uniform等,但是有没有办法通过传递numpy数组来初始化参数呢?

import numpy as np
import torch as nn
rng = np.random.RandomState(313)
w = rng.randn(input_size, hidden_size).astype(np.float32)

rnn = nn.RNN(input_size, hidden_size, num_layers)

2 回复 | 直到 7 年前

benjaminplanche 7 年前

首先,让我们注意到 nn.RNN 有多个权重变量,c.f.the documentation :

变量:

weight_ih_l[k] 的可学习输入隐藏权重 k -第层,形状 (hidden_size * input_size) 对于 k = 0 . 否则, 形状是 (hidden_size * hidden_size)

weight_hh_l[k] 可学习的隐藏权重 千 -第层,形状 (隐藏大小*隐藏大小)

bias_ih_l[k] 可学习的输入隐藏了 千 -第层,形状 (hidden_size)

bias_hh_l[k] 可学习的隐藏偏见 千 -第层,形状 (隐藏大小)

现在,每个变量( Parameter 实例)是 nn.RNN公司 实例。您可以通过两种方式访问和编辑它们,如下所示:

解决方案1:访问所有RNN 参数 按名称列出的属性( rnn.weight_hh_lK , rnn.weight_ih_lK 等):

import torch
from torch import nn
import numpy as np

input_size, hidden_size, num_layers = 3, 4, 2
use_bias = True
rng = np.random.RandomState(313)

rnn = nn.RNN(input_size, hidden_size, num_layers, bias=use_bias)

def set_nn_parameter_data(layer, parameter_name, new_data):
    param = getattr(layer, parameter_name)
    param.data = new_data

for i in range(num_layers):
    weights_hh_layer_i = rng.randn(hidden_size, hidden_size).astype(np.float32)
    weights_ih_layer_i = rng.randn(hidden_size, hidden_size).astype(np.float32)
    set_nn_parameter_data(rnn, "weight_hh_l{}".format(i), 
                          torch.from_numpy(weights_hh_layer_i))
    set_nn_parameter_data(rnn, "weight_ih_l{}".format(i), 
                          torch.from_numpy(weights_ih_layer_i))

    if use_bias:
        bias_hh_layer_i = rng.randn(hidden_size).astype(np.float32)
        bias_ih_layer_i = rng.randn(hidden_size).astype(np.float32)
        set_nn_parameter_data(rnn, "bias_hh_l{}".format(i), 
                              torch.from_numpy(bias_hh_layer_i))
        set_nn_parameter_data(rnn, "bias_ih_l{}".format(i), 
                              torch.from_numpy(bias_ih_layer_i))

解决方案2:访问所有RNN 参数 属性通过 rnn.all_weights 列表属性:

import torch
from torch import nn
import numpy as np

input_size, hidden_size, num_layers = 3, 4, 2
use_bias = True
rng = np.random.RandomState(313)

rnn = nn.RNN(input_size, hidden_size, num_layers, bias=use_bias)

for i in range(num_layers):
    weights_hh_layer_i = rng.randn(hidden_size, hidden_size).astype(np.float32)
    weights_ih_layer_i = rng.randn(hidden_size, hidden_size).astype(np.float32)
    rnn.all_weights[i][0].data = torch.from_numpy(weights_ih_layer_i)
    rnn.all_weights[i][1].data = torch.from_numpy(weights_hh_layer_i)

    if use_bias:
        bias_hh_layer_i = rng.randn(hidden_size).astype(np.float32)
        bias_ih_layer_i = rng.randn(hidden_size).astype(np.float32)
        rnn.all_weights[i][2].data = torch.from_numpy(bias_ih_layer_i)
        rnn.all_weights[i][3].data = torch.from_numpy(bias_hh_layer_i)

Wasi Ahmad 7 年前

作为一个详细的答案,我只是补充一句话。一个 nn.Module 是张量(以前,它是自传变量, which is deperecated in Pytorch 0.4 ). 所以,实际上你需要使用 torch.from_numpy() 方法将Numpy数组转换为张量,然后使用它们初始化 nn.模块 参数。