在 PyTorch 中计算每个 epoch 的准确率
Posted
技术标签:
【中文标题】在 PyTorch 中计算每个 epoch 的准确率【英文标题】：Calculate the accuracy every epoch in PyTorch 【发布时间】：2019-01-01 09:14:08 【问题描述】：我正在研究神经网络问题，将数据分类为 1 或 0。我正在使用二元交叉熵损失来执行此操作。损失很好，但是，准确性非常低并且没有提高。我假设我在准确性计算中犯了一个错误。在每个时期之后，我在对输出进行阈值处理后计算正确的预测，并将该数字除以数据集的总数。我在精度计算中做错了什么吗？为什么它没有改善，反而变得更糟？ 这是我的代码：
net = Model()
criterion = torch.nn.BCELoss(size_average=True)
optimizer = torch.optim.SGD(net.parameters(), lr=0.1)
num_epochs = 100
for epoch in range(num_epochs):
for i, (inputs,labels) in enumerate (train_loader):
inputs = Variable(inputs.float())
labels = Variable(labels.float())
output = net(inputs)
optimizer.zero_grad()
loss = criterion(output, labels)
loss.backward()
optimizer.step()
#Accuracy
output = (output>0.5).float()
correct = (output == labels).float().sum()
print("Epoch /, Loss: :.3f, Accuracy: :.3f".format(epoch+1,num_epochs, loss.data[0], correct/x.shape[0]))
这是我得到的奇怪输出：
Epoch 1/100, Loss: 0.389, Accuracy: 0.035
Epoch 2/100, Loss: 0.370, Accuracy: 0.036
Epoch 3/100, Loss: 0.514, Accuracy: 0.030
Epoch 4/100, Loss: 0.539, Accuracy: 0.030
Epoch 5/100, Loss: 0.583, Accuracy: 0.029
Epoch 6/100, Loss: 0.439, Accuracy: 0.031
Epoch 7/100, Loss: 0.429, Accuracy: 0.034
Epoch 8/100, Loss: 0.408, Accuracy: 0.035
Epoch 9/100, Loss: 0.316, Accuracy: 0.035
Epoch 10/100, Loss: 0.436, Accuracy: 0.035
Epoch 11/100, Loss: 0.365, Accuracy: 0.034
Epoch 12/100, Loss: 0.485, Accuracy: 0.031
Epoch 13/100, Loss: 0.392, Accuracy: 0.033
Epoch 14/100, Loss: 0.494, Accuracy: 0.030
Epoch 15/100, Loss: 0.369, Accuracy: 0.035
Epoch 16/100, Loss: 0.495, Accuracy: 0.029
Epoch 17/100, Loss: 0.415, Accuracy: 0.034
Epoch 18/100, Loss: 0.410, Accuracy: 0.035
Epoch 19/100, Loss: 0.282, Accuracy: 0.038
Epoch 20/100, Loss: 0.499, Accuracy: 0.031
Epoch 21/100, Loss: 0.446, Accuracy: 0.030
Epoch 22/100, Loss: 0.585, Accuracy: 0.026
Epoch 23/100, Loss: 0.419, Accuracy: 0.035
Epoch 24/100, Loss: 0.492, Accuracy: 0.031
Epoch 25/100, Loss: 0.537, Accuracy: 0.031
Epoch 26/100, Loss: 0.439, Accuracy: 0.033
Epoch 27/100, Loss: 0.421, Accuracy: 0.035
Epoch 28/100, Loss: 0.532, Accuracy: 0.034
Epoch 29/100, Loss: 0.234, Accuracy: 0.038
Epoch 30/100, Loss: 0.492, Accuracy: 0.027
Epoch 31/100, Loss: 0.407, Accuracy: 0.035
Epoch 32/100, Loss: 0.305, Accuracy: 0.038
Epoch 33/100, Loss: 0.663, Accuracy: 0.025
Epoch 34/100, Loss: 0.588, Accuracy: 0.031
Epoch 35/100, Loss: 0.329, Accuracy: 0.035
Epoch 36/100, Loss: 0.474, Accuracy: 0.033
Epoch 37/100, Loss: 0.535, Accuracy: 0.031
Epoch 38/100, Loss: 0.406, Accuracy: 0.033
Epoch 39/100, Loss: 0.513, Accuracy: 0.030
Epoch 40/100, Loss: 0.593, Accuracy: 0.030
Epoch 41/100, Loss: 0.265, Accuracy: 0.036
Epoch 42/100, Loss: 0.576, Accuracy: 0.031
Epoch 43/100, Loss: 0.565, Accuracy: 0.027
Epoch 44/100, Loss: 0.576, Accuracy: 0.030
Epoch 45/100, Loss: 0.396, Accuracy: 0.035
Epoch 46/100, Loss: 0.423, Accuracy: 0.034
Epoch 47/100, Loss: 0.489, Accuracy: 0.033
Epoch 48/100, Loss: 0.591, Accuracy: 0.029
Epoch 49/100, Loss: 0.415, Accuracy: 0.034
Epoch 50/100, Loss: 0.291, Accuracy: 0.039
Epoch 51/100, Loss: 0.395, Accuracy: 0.033
Epoch 52/100, Loss: 0.540, Accuracy: 0.026
Epoch 53/100, Loss: 0.436, Accuracy: 0.033
Epoch 54/100, Loss: 0.346, Accuracy: 0.036
Epoch 55/100, Loss: 0.519, Accuracy: 0.029
Epoch 56/100, Loss: 0.456, Accuracy: 0.031
Epoch 57/100, Loss: 0.425, Accuracy: 0.035
Epoch 58/100, Loss: 0.311, Accuracy: 0.039
Epoch 59/100, Loss: 0.406, Accuracy: 0.034
Epoch 60/100, Loss: 0.360, Accuracy: 0.035
Epoch 61/100, Loss: 0.476, Accuracy: 0.030
Epoch 62/100, Loss: 0.404, Accuracy: 0.034
Epoch 63/100, Loss: 0.382, Accuracy: 0.036
Epoch 64/100, Loss: 0.538, Accuracy: 0.031
Epoch 65/100, Loss: 0.392, Accuracy: 0.034
Epoch 66/100, Loss: 0.434, Accuracy: 0.033
Epoch 67/100, Loss: 0.479, Accuracy: 0.031
Epoch 68/100, Loss: 0.494, Accuracy: 0.031
Epoch 69/100, Loss: 0.415, Accuracy: 0.034
Epoch 70/100, Loss: 0.390, Accuracy: 0.036
Epoch 71/100, Loss: 0.330, Accuracy: 0.038
Epoch 72/100, Loss: 0.449, Accuracy: 0.030
Epoch 73/100, Loss: 0.315, Accuracy: 0.039
Epoch 74/100, Loss: 0.450, Accuracy: 0.031
Epoch 75/100, Loss: 0.562, Accuracy: 0.030
Epoch 76/100, Loss: 0.447, Accuracy: 0.031
Epoch 77/100, Loss: 0.408, Accuracy: 0.038
Epoch 78/100, Loss: 0.359, Accuracy: 0.034
Epoch 79/100, Loss: 0.372, Accuracy: 0.035
Epoch 80/100, Loss: 0.452, Accuracy: 0.034
Epoch 81/100, Loss: 0.360, Accuracy: 0.035
Epoch 82/100, Loss: 0.453, Accuracy: 0.031
Epoch 83/100, Loss: 0.578, Accuracy: 0.030
Epoch 84/100, Loss: 0.537, Accuracy: 0.030
Epoch 85/100, Loss: 0.483, Accuracy: 0.035
Epoch 86/100, Loss: 0.343, Accuracy: 0.036
Epoch 87/100, Loss: 0.439, Accuracy: 0.034
Epoch 88/100, Loss: 0.686, Accuracy: 0.023
Epoch 89/100, Loss: 0.265, Accuracy: 0.039
Epoch 90/100, Loss: 0.369, Accuracy: 0.035
Epoch 91/100, Loss: 0.521, Accuracy: 0.027
Epoch 92/100, Loss: 0.662, Accuracy: 0.027
Epoch 93/100, Loss: 0.581, Accuracy: 0.029
Epoch 94/100, Loss: 0.322, Accuracy: 0.034
Epoch 95/100, Loss: 0.375, Accuracy: 0.035
Epoch 96/100, Loss: 0.575, Accuracy: 0.031
Epoch 97/100, Loss: 0.489, Accuracy: 0.030
Epoch 98/100, Loss: 0.435, Accuracy: 0.033
Epoch 99/100, Loss: 0.440, Accuracy: 0.031
Epoch 100/100, Loss: 0.444, Accuracy: 0.033
【问题讨论】：
您能否发布更多代码以便更好地理解？ 您的准确度公式对我来说很合适，请提供更多代码 你能把x.shape[0] 中的x 贴出来吗？
您能澄清一下i 在这一行中是什么：for i, (inputs,labels) in enumerate (train_loader): 吗？这个指数是训练次数吗？
一个行来获得准确度acc == (true == mdl(x).max(1).item() / true.size(0) 假设第 0 维是批量大小，第 1 维保存分类标签的 logits/raw 值。
【参考方案1】：
x 是整个输入数据集吗？如果是这样，您可能会除以 correct/x.shape[0] 中整个输入数据集的大小（而不是小批量的大小）。尝试将其更改为correct/output.shape[0]
【讨论】：
我将它除以数据集的总数，因为我已经完成了一个 epoch。可以看到print语句在epoch循环里面，不是batch循环里面。 是的，我看到了。然而，“正确”仍然只有一个小批量 是的。我想你是对的。输出在这种情况下是最后一个小批量输出，我们将在其中验证每个 epoch。所以我们应该划分 epoch 的最后一次迭代的 mini-batch 大小。感谢您的回答【参考方案2】：更好的方法是在优化步骤之后立即计算正确
for epoch in range(num_epochs):
correct = 0
for i, (inputs,labels) in enumerate (train_loader):
...
output = net(inputs)
...
optimizer.step()
correct += (output == labels).float().sum()
accuracy = 100 * correct / len(trainset)
# trainset, not train_loader
# probably x in your case
print("Accuracy = ".format(accuracy))
【讨论】：
也许使用.item() 会比.float() 更好
您作为伪代码/注释编写的代码是其中最棘手的部分，我正在寻求解释：max_vals, max_indices = torch.max(mdl(X),1)
@CharlieParker .item() 在张量中恰好有 1 个值时起作用。否则会报错。 (output == labels) 是一个具有许多值的布尔张量，通过将其转换为浮点数，False 被强制转换为 0，Trues 被强制转换为 1。然后我们将 True 的数量相加（.sum() 本身可能就足够了它应该做铸造的东西）【参考方案3】：
这是我的解决方案：
def evaluate(model, validation_loader, use_cuda=True):
model.eval()
with torch.no_grad():
acc = .0
for i, data in enumerate(validation_loader):
X = data[0]
y = data[1]
if use_cuda:
X = X.cuda()
y = y.cuda()
predicted = model(X)
acc+=(predicted.round() == y).sum()/float(predicted.shape[0])
model.train()
return (acc/(i+1)).detach().item()
注意1：在验证时将模型设置为eval模式，然后再返回train模式。
注意 2： 我不确定是否需要禁用 autograd。这是thread on it
对于 one-hot 结果，可以使用torch.max。 Example:
correct = 0
total = 0
with torch.no_grad():
for data in testloader:
images, labels = data
outputs = net(images)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print('Accuracy of the network on the 10000 test images: %d %%' % (
100 * correct / total))
【讨论】：
我通常更喜欢在我的实验脚本的顶部调用它device = torch.device("cuda" if torch.cuda.is_available() else "cpu") 然后使用 mdl.to(device) 或 tensor.to(device) 使其更短（并且也与模型无关）
另外，如果你已经在使用.item()，我认为你不需要.detach()。
你为什么不做acc+=(predicted.round() == y).sum().item() / predicted.shape[0].item()或者更短的？看来您的代码有很多随机冗余......
为什么：_, predicted = torch.max(outputs.data, 1) 正确？【参考方案4】：
只需阅读此答案：
https://***.com/a/63271002/1601580
旧
我认为最简单的答案是来自the cifar10 tutorial：
total = 0
with torch.no_grad():
net.eval()
for data in testloader:
images, labels = data
outputs = net(images)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print('Accuracy of the network on the 10000 test images: %d %%' % (
100 * correct / total))
所以：
acc = (true == pred).sum().item()
如果您有计数器，请不要忘记最终除以数据集或类似值的大小。
我用过：
N = data.size(0) # since usually it's size (batch_size, D1, D2, ...)
correct += (1/N) * correct
自包含代码：
# testing accuracy function
# https://discuss.pytorch.org/t/calculating-accuracy-of-the-current-minibatch/4308/11
# https://***.com/questions/51503851/calculate-the-accuracy-every-epoch-in-pytorch
import torch
import torch.nn as nn
D = 1
true = torch.tensor([0,1,0,1,1]).reshape(5,1)
print(f'true.size() = true.size()')
batch_size = true.size(0)
print(f'batch_size = batch_size')
x = torch.randn(batch_size,D)
print(f'x = x')
print(f'x.size() = x.size()')
mdl = nn.Linear(D,1)
logit = mdl(x)
_, pred = torch.max(logit.data, 1)
print(f'logit = logit')
print(f'pred = pred')
print(f'true = true')
acc = (true == pred).sum().item()
print(f'acc = acc')
另外，我觉得这段代码是很好的参考：
def calc_accuracy(mdl, X, Y):
# reduce/collapse the classification dimension according to max op
# resulting in most likely label
max_vals, max_indices = mdl(X).max(1)
# assumes the first dimension is batch size
n = max_indices.size(0) # index 0 for extracting the # of elements
# calulate acc (note .item() to do float division)
acc = (max_indices == Y).sum().item() / n
return acc
解释pred = mdl(x).max(1)看这个https://discuss.pytorch.org/t/how-does-one-get-the-predicted-classification-label-from-a-pytorch-model/91649
主要是你必须减少/折叠分类原始值/logit 最大的维度，然后使用.indices 选择它。通常这是尺寸1，因为 dim 0 具有批量大小，例如[batch_size,D_classification] 原始数据大小可能为[batch_size,C,H,W]
一维原始数据的合成示例如下：
import torch
import torch.nn as nn
# data dimension [batch-size, D]
D, Dout = 1, 5
batch_size = 16
x = torch.randn(batch_size, D)
y = torch.randint(low=0,high=Dout,size=(batch_size,))
mdl = nn.Linear(D, Dout)
logits = mdl(x)
print(f'y.size() = y.size()')
# removes the 1th dimension with a max, which is the classification layer
# which means it returns the most likely label. Also, note you need to choose .indices since you want to return the
# position of where the most likely label is (not it's raw logit value)
pred = logits.max(1).indices
print(pred)
print('--- preds vs truth ---')
print(f'predictions = pred')
print(f'y = y')
acc = (pred == y).sum().item() / pred.size(0)
print(acc)
输出：
y.size() = torch.Size([16])
tensor([3, 1, 1, 3, 4, 1, 4, 3, 1, 1, 4, 4, 4, 4, 3, 1])
--- preds vs truth ---
predictions = tensor([3, 1, 1, 3, 4, 1, 4, 3, 1, 1, 4, 4, 4, 4, 3, 1])
y = tensor([3, 3, 3, 0, 3, 4, 0, 1, 1, 2, 1, 4, 4, 2, 0, 0])
0.25
参考：
https://discuss.pytorch.org/t/calculating-accuracy-of-the-current-minibatch/4308/5 https://discuss.pytorch.org/t/how-does-one-get-the-predicted-classification-label-from-a-pytorch-model/91649/3 所以：Calculate the accuracy every epoch in PyTorch【讨论】：
一个行来获得准确度acc == (true == mdl(x).max(1).item() / true.size(0) 假设第 0 维是批量大小，第 1 维保存分类标签的 logits/raw 值。【参考方案5】：
让我们看看基础知识：
Accuracy = Total Correct Observations / Total Observations
在您的代码中，当您计算准确度时，您将 Total Correct Observations 在一个时期内除以不正确的总观察值
correct/x.shape[0]
相反，您应该将其除以每个时期的观察次数，即批量大小。假设您的批量大小 = batch_size
Solution 1. Accuracy = correct/batch_size
Solution 2. Accuracy = correct/len(labels)
Solution 3. Accuracy = correct/len(input)
理想情况下，在每个时期，您的批量大小、输入长度（行数）和标签长度应该相同。
【讨论】：
【参考方案6】：一个班轮获得准确性
acc == (true == mdl(x).max(1).item() / true.size(0)
假设第 0 维是批量大小，第 1 维保存分类标签的 logits/raw 值。
更多细节：
def calc_error(mdl: torch.nn.Module, X: torch.Tensor, Y):
# acc == (true != mdl(x).max(1).item() / true.size(0)
train_acc = calc_accuracy(mdl, X, Y)
train_err = 1.0 - train_acc
return train_err
def calc_accuracy(mdl: torch.nn.Module, X: torch.Tensor, Y: torch.Tensor) -> float:
"""
Get the accuracy with respect to the most likely label
:param mdl:
:param X:
:param Y:
:return:
"""
# get the scores for each class (or logits)
y_logits = mdl(X) # unnormalized probs
# return the values & indices with the largest value in the dimension where the scores for each class is
# get the scores with largest values & their corresponding idx (so the class that is most likely)
max_scores, max_idx_class = mdl(X).max(dim=1) # [B, n_classes] -> [B], # get values & indices with the max vals in the dim with scores for each class/label
# usually 0th coordinate is batch size
n = X.size(0)
assert( n == max_idx_class.size(0))
# calulate acc (note .item() to do float division)
acc = (max_idx_class == Y).sum().item() / n
return acc
【讨论】：
【参考方案7】：在这里检查这些定义：
def train(model, train_loader):
model.train()
train_acc, correct_train, train_loss, target_count = 0, 0, 0, 0
for i, (input, target) in enumerate(train_loader):
target = target.cuda()
input_var = Variable(input)
target_var = Variable(target)
optimizer.zero_grad()
output = model(input_var)
loss = criterion(output, target_var)
loss.backward()
optimizer.step()
# accuracy
_, predicted = torch.max(output.data, 1)
target_count += target_var.size(0)
correct_train += (target_var == predicted).sum().item()
train_acc = (100 * correct_train) / target_count
return train_acc, train_loss / target_count
def validate(model, val_loader):
model.eval()
val_acc, correct_val, val_loss, target_count = 0, 0, 0, 0
for i, (input, target) in enumerate(val_loader):
target = target.cuda()
input_var = Variable(input, volatile=True)
target_var = Variable(target, volatile=True)
output = model(input_var)
loss = criterion(output, target_var)
val_loss += loss.item()
# accuracy
_, predicted = torch.max(output.data, 1)
target_count += target_var.size(0)
correct_val += (target_var == predicted).sum().item()
val_acc = 100 * correct_val / target_count
return (val_acc * 100) / target_count, val_loss / target_count
for epoch in range(0, n_epoch):
train_acc, train_loss = train(model, train_loader)
val_loss = validate(model, val_loader)
print("Epoch 0: train_acc 1 \t train_loss 2 \t val_acc 3 \t val_loss 4".format(epoch, train_acc, train_loss, val_acc, val_loss))
【讨论】：
【参考方案8】：只需阅读此答案：
https://***.com/a/63271002/1601580分步示例
以下是一步一步的解释，以自包含代码为例：
#%%
# refs:
# https://***.com/questions/51503851/calculate-the-accuracy-every-epoch-in-pytorch
# https://discuss.pytorch.org/t/how-to-calculate-accuracy-in-pytorch/80476/5
# https://discuss.pytorch.org/t/how-does-one-get-the-predicted-classification-label-from-a-pytorch-model/91649
# how to get the class prediction
batch_size = 4
n_classes = 2
y_logits = torch.randn(batch_size, n_classes) # usually the scores
print('scores (logits) for each class for each example in batch (how likely a class is unnormalized)')
print(y_logits)
print('the max over entire tensor (not usually what we want)')
print(y_logits.max())
print('the max over the n_classes dim. For each example in batch returns: '
'1) the highest score for each class (most likely class)\n, and '
'2) the idx (=class) with that highest score')
print(y_logits.max(1))
print('-- calculate accuracy --')
# computing accuracy in pytorch
"""
random.choice(a, size=None, replace=True, p=None)
Generates a random sample from a given 1-D array
for pytorch random choice https://***.com/questions/59461811/random-choice-with-pytorch
"""
import torch
import torch.nn as nn
in_features = 1
n_classes = 10
batch_size = n_classes
mdl = nn.Linear(in_features=in_features, out_features=n_classes)
x = torch.randn(batch_size, in_features)
y_logits = mdl(x) # scores/logits for each example in batch [B, n_classes]
# get for each example in batch the label/idx most likely according to score
# y_max_idx[b] = y_pred[b] = argmax_idx \in [n_classes] y_logit[idx]
y_max_scores, y_max_idx = y_logits.max(dim=1)
y_pred = y_max_idx # predictions are really the inx \in [n_classes] with the highest scores
y = torch.randint(high=n_classes, size=(batch_size,))
# accuracy for 1 batch
assert (y.size(0) == batch_size)
acc = (y == y_pred).sum() / y.size(0)
acc = acc.item()
print(y)
print(y_pred)
print(acc)
输出：
scores (logits) for each class for each example in batch (how likely a class is unnormalized)
tensor([[ 0.4912, 1.5143],
[ 1.2378, 0.3172],
[-1.0164, -1.2786],
[-1.6685, -0.6693]])
the max over entire tensor (not usually what we want)
tensor(1.5143)
the max over the n_classes dim. For each example in batch returns: 1) the highest score for each class (most likely class)
, and 2) the idx (=class) with that highest score
torch.return_types.max(
values=tensor([ 1.5143, 1.2378, -1.0164, -0.6693]),
indices=tensor([1, 0, 0, 1]))
-- calculate accuracy --
tensor([6, 1, 3, 5, 3, 9, 6, 5, 6, 6])
tensor([5, 5, 5, 5, 5, 7, 7, 5, 5, 7])
0.20000000298023224
【讨论】：
以上是关于在 PyTorch 中计算每个 epoch 的准确率的主要内容，如果未能解决你的问题，请参考以下文章
【Pytorch+torchvision】MNIST手写数字识别（代码附最详细注释）