代码之家  ›  专栏  ›  技术社区  ›  blue-sky

用pytorch初始化权重和偏差-如何更正尺寸?

  •  0
  • blue-sky  · 技术社区  · 7 年前

    使用这个模型,我试图用我预先定义的权重和偏差初始化我的网络:

    dimensions_input = 10
    hidden_layer_nodes = 5
    output_dimension = 10
    
    class Model(torch.nn.Module):
        def __init__(self):
            super(Model, self).__init__()
            self.linear = torch.nn.Linear(dimensions_input,hidden_layer_nodes)
            self.linear2 = torch.nn.Linear(hidden_layer_nodes,output_dimension)
    
            self.linear.weight = torch.nn.Parameter(torch.zeros(dimensions_input,hidden_layer_nodes))
            self.linear.bias = torch.nn.Parameter(torch.ones(hidden_layer_nodes))
    
            self.linear2.weight = torch.nn.Parameter(torch.zeros(dimensions_input,hidden_layer_nodes))
            self.linear2.bias = torch.nn.Parameter(torch.ones(hidden_layer_nodes))
    
        def forward(self, x):
            l_out1 = self.linear(x)
            y_pred = self.linear2(l_out1)
            return y_pred
    
    model = Model()
    
    criterion = torch.nn.MSELoss(size_average = False)
    optim = torch.optim.SGD(model.parameters(), lr = 0.00001)
    
    def train_model():
        y_data = x_data.clone()
        for i in range(10000):
            y_pred = model(x_data)
            loss = criterion(y_pred, y_data)
    
            if i % 5000 == 0:
                print(loss)
            optim.zero_grad()
    
            loss.backward()
            optim.step()
    

    运行时错误:

    张量(10)的展开尺寸必须与现有尺寸(5)匹配。 非单件尺寸1

    我的尺寸看起来正确,因为它们与相应的线性层匹配?

    1 回复  |  直到 7 年前
        1
  •  1
  •   djd    7 年前

    由于以下事实,提供的代码无法运行 x_data 没有定义,所以我不能确定这是问题所在,但有一件事让我感到震惊,那就是你应该更换

    self.linear2.weight = torch.nn.Parameter(torch.zeros(dimensions_input,hidden_layer_nodes))
    self.linear2.bias = torch.nn.Parameter(torch.ones(hidden_layer_nodes))
    

    具有

    self.linear2.weight = torch.nn.Parameter(torch.zeros(hidden_layer_nodes, output_dimension))
    self.linear2.bias = torch.nn.Parameter(torch.ones(output_dimension))