为啥损失减少而准确率却没有增加？ PyTorch

Posted 2023-02-23

【中文标题】为啥损失减少而准确率却没有增加？ PyTorch

【英文标题】：Why does the loss decreases and the accuracy dosen't increases? PyTorch为什么损失减少而准确率却没有增加？ PyTorch

【发布时间】：2022-01-22 11:17:07

【问题描述】：

我正在 Pytorch 中创建一个 CNN，但我认为训练功能有问题。

对于每个 epoch，损失都会减少。但准确度保持不变，不会改变。 训练函数的输出是这样的：

Epoch: 1
correct: 234, N_test: 468 ------>  loss: 58.2041027, accuracy_val: %50.0
Epoch: 2
correct: 234, N_test: 468 ------>  loss: 51.47981386, accuracy_val: %50.0
Epoch: 3
correct: 234, N_test: 468 ------>  loss: 51.57150275, accuracy_val: %50.0
Epoch: 4
correct: 234, N_test: 468 ------>  loss: 39.14232715, accuracy_val: %50.0
Epoch: 5
correct: 234, N_test: 468 ------>  loss: 32.23730827, accuracy_val: %50.0

我知道虽然它们是相关的，但损失和准确性有其复杂性，但我相信代码可能有问题，我无法确定是什么。

这是神经网络：

class CNN(nn.Module):

# Contructor
def __init__(self):
    super(CNN, self).__init__()
    # Conv1
    self.cnn1 = nn.Conv2d(in_channels=1, out_channels=64, kernel_size=5, stride=1, padding=0)
    self.conv1_bn = nn.BatchNorm2d(64)
    self.maxpool1=nn.MaxPool2d(kernel_size=2, stride=2)
    
    # Conv2
    self.cnn2 = nn.Conv2d(in_channels=64, out_channels=64, kernel_size=5,stride=1, padding=0)
    self.conv2_bn = nn.BatchNorm2d(64)
    self.maxpool2=nn.MaxPool2d(kernel_size=2, stride=2)  
    
    # Conv3
    self.cnn3 = nn.Conv2d(in_channels=64, out_channels=128, kernel_size=5,stride=1, padding=0)
    self.conv3_bn = nn.BatchNorm2d(128)
    self.maxpool3=nn.MaxPool2d(kernel_size=2, stride=2)
    
    
    # FCL 1
    self.fc1 = nn.Linear(in_features=128 * 27 * 27, out_features=500)
    self.bn_fc1 = nn.BatchNorm1d(500)
    
    # FCL 2
    self.fc2 = nn.Linear(in_features=500, out_features=500)
    self.bn_fc2 = nn.BatchNorm1d(500)
    
    # FCL3
    self.fc3 = nn.Linear(in_features=500, out_features=1)
    

    

# Prediction
def forward(self, x):
    # conv1
    x = self.cnn1(x)
    x = self.conv1_bn(x)
    x = torch.relu(x)
    x = self.maxpool1(x)
    # conv2
    x = self.cnn2(x)
    x = self.conv2_bn(x)
    x = torch.relu(x)
    x = self.maxpool2(x)
    # conv3
    x = self.cnn3(x)
    x = self.conv3_bn(x)
    x = torch.relu(x)
    x = self.maxpool3(x)
    
    # Fcl1
    x = x.view(x.size(0), -1)
    x = self.fc1(x)
    x = self.bn_fc1(x)
    x = torch.relu(x)
    # Fcl2
    x = self.fc2(x)
    x = self.bn_fc2(x)
    x = torch.relu(x)
    # final fcl
    x = self.fc3(x)
    x = torch.sigmoid(x)
   
    return x

训练函数：

def train_model(model,train_loader,test_loader,optimizer,n_epochs=5):

#global variable 
N_test=len(dataset_val)
accuracy_list=[]

loss_list=[]
for epoch in range(n_epochs):
    cost = 0
    model.train()
    print(f"Epoch: epoch + 1")
    for x, y in train_loader:
        x, y = x.to(device), y.to(device)
        optimizer.zero_grad()
        z = model(x)
        y = y.unsqueeze(-1)
        y = y.float()
        loss = criterion(z, y)
        loss.backward()
        optimizer.step()
        cost+=loss.item()

    correct=0
    model.eval()
    #perform a prediction on the validation  data  
    for x_test, y_test in test_loader:
        x_test, y_test = x_test.to(device), y_test.to(device)
        
        z = model(x_test)
        _, yhat = torch.max(z.data, 1)
        correct += (yhat == y_test).sum().item()

    accuracy = correct / N_test
    accuracy_list.append(accuracy)
    loss_list.append(cost)
    print(f"------>  loss: round(cost, 8), accuracy_val: %accuracy * 100")

 
return accuracy_list, loss_lis

剧情是这样的：

Plot with accuracy and loss

【参考方案1】：

我从最后一层移除了 sigmoid 函数并将BCELoss() 替换为CrossEntropyLoss() 并且工作正常！

另外，正如@jhso 所说，要进行二元分类，需要一个阈值，我们必须将数据分成两个类别，或者有 2 个输出（在这种情况下可能更容易）。

【参考方案2】：

你的输出都将是 1，因为你有 1 个输出并且你在第二维取最大值：

    _, yhat = torch.max(z.data, 1)
    correct += (yhat == y_test).sum().item()

要进行二元分类，您需要选择一个阈值，然后将您的数据分成两个类别，或者有 2 个输出（在这种情况下可能更容易）。

【讨论】：

谢谢你！我也找到了另一个解决方案。我从最后一层删除了 sigmoid 函数，并将 BCELoss() 替换为 CrossEntropyLoss() 但你也说得对，谢谢！