计算精度问题

Posted 2023-03-27

【中文标题】计算精度问题

【英文标题】：issue with calculating accuracy

【发布时间】：2021-07-27 07:16:42

【问题描述】：

我正在使用 Torch Metrics 来尝试计算我的模型的准确性。但我收到了这个错误。我尝试使用.to(device="cuda:0")，但出现 cuda 初始化错误。我也尝试过使用.cuda()，但这也没有用。我正在使用带有 Titan Xp GPU 的 PyTorch 闪电。我在电影镜头数据集上使用了一个 mish 激活函数。

代码：

# %% [markdown]
# # Data Preprocessing
# 
# Before we start building and training our model, let's do some preprocessing to get the data in the required format.

# %% [code] "_kg_hide-input":true,"_kg_hide-output":true
import pandas as pd
import numpy as np
from tqdm.notebook import tqdm

import torch
import torch.nn as nn
from torch.utils.data import Dataset, DataLoader
import pytorch_lightning as pl
import torch.nn.functional as F
from pytorch_lightning.callbacks import EarlyStopping
import wandb
import torchmetrics

wandb.init(project="Mocean-Recommendor",entity="maxall4")

config = wandb.config


def mish(x):
    return (x*torch.tanh(F.softplus(x)))


np.random.seed(123)

# %% [markdown]
# First, we import the ratings dataset.

# %% [code]
ratings = pd.read_csv('rating.csv', 
                      parse_dates=['timestamp'])

# %% [markdown]
# In order to keep memory usage manageable within Kaggle's kernel, we will only use data from 30% of the users in this dataset. Let's randomly select 30% of the users and only use data from the selected users.

# %% [code]
rand_userIds = np.random.choice(ratings['userId'].unique(), 
                                size=int(len(ratings['userId'].unique())*0.3), 
                                replace=False)

ratings = ratings.loc[ratings['userId'].isin(rand_userIds)]

print('There are  rows of data from  users'.format(len(ratings), len(rand_userIds)))

# %% [code]
ratings.sample(5)

# %% [code]
ratings['rank_latest'] = ratings.groupby(['userId'])['timestamp'] \
                                .rank(method='first', ascending=False)

train_ratings = ratings[ratings['rank_latest'] != 1]
test_ratings = ratings[ratings['rank_latest'] == 1]

# drop columns that we no longer need
train_ratings = train_ratings[['userId', 'movieId', 'rating']]
test_ratings = test_ratings[['userId', 'movieId', 'rating']]

# %% [markdown]
# ### Converting the dataset into an implicit feedback dataset


# %% [code]
train_ratings.loc[:, 'rating'] = 1

train_ratings.sample(5)

# %% [markdown]
# The code below generates 4 negative samples for each row of data. In other words, the ratio of negative to positive samples is 4:1. This ratio is chosen arbitrarily but I found that it works rather well (feel free to find the best ratio yourself!)

# %% [code]
# Get a list of all movie IDs
all_movieIds = ratings['movieId'].unique()

# Placeholders that will hold the training data
users, items, labels = [], [], []

# This is the set of items that each user has interaction with
user_item_set = set(zip(train_ratings['userId'], train_ratings['movieId']))

# 4:1 ratio of negative to positive samples
num_negatives = 4

for (u, i) in tqdm(user_item_set):
    users.append(u)
    items.append(i)
    labels.append(1) # items that the user has interacted with are positive
    for _ in range(num_negatives):
        # randomly select an item
        negative_item = np.random.choice(all_movieIds) 
        # check that the user has not interacted with this item
        while (u, negative_item) in user_item_set:
            negative_item = np.random.choice(all_movieIds)
        users.append(u)
        items.append(negative_item)
        labels.append(0) # items not interacted with are negative


# %% [code]
class MovieLensTrainDataset(Dataset):
    """MovieLens PyTorch Dataset for Training
    
    Args:
        ratings (pd.DataFrame): Dataframe containing the movie ratings
        all_movieIds (list): List containing all movieIds
    
    """

    def __init__(self, ratings, all_movieIds):
        self.users, self.items, self.labels = self.get_dataset(ratings, all_movieIds)

    def __len__(self):
        return len(self.users)
  
    def __getitem__(self, idx):
        return self.users[idx], self.items[idx], self.labels[idx]

    def get_dataset(self, ratings, all_movieIds):
        users, items, labels = [], [], []
        user_item_set = set(zip(ratings['userId'], ratings['movieId']))

        num_negatives = 4
        for u, i in user_item_set:
            users.append(u)
            items.append(i)
            labels.append(1)
            for _ in range(num_negatives):
                negative_item = np.random.choice(all_movieIds)
                while (u, negative_item) in user_item_set:
                    negative_item = np.random.choice(all_movieIds)
                users.append(u)
                items.append(negative_item)
                labels.append(0)

        return torch.tensor(users), torch.tensor(items), torch.tensor(labels)


# %% [code]
acc_metric = torchmetrics.Accuracy()
class NCF(pl.LightningModule):
    """ Neural Collaborative Filtering (NCF)
    
        Args:
            num_users (int): Number of unique users
            num_items (int): Number of unique items
            ratings (pd.DataFrame): Dataframe containing the movie ratings for training
            all_movieIds (list): List containing all movieIds (train + test)
    """
    
    def __init__(self, num_users, num_items, ratings, all_movieIds):
        super().__init__()
        self.user_embedding = nn.Embedding(num_embeddings=num_users, embedding_dim=8)
        self.item_embedding = nn.Embedding(num_embeddings=num_items, embedding_dim=8)
        self.fc1 = nn.Linear(in_features=16, out_features=64)
        self.fc2 = nn.Linear(in_features=64, out_features=32)
        self.output = nn.Linear(in_features=32, out_features=1)
        self.ratings = ratings
        self.all_movieIds = all_movieIds
    
    def on_validation_end(self,outputs):
        loss = torch.stack([x['val_loss'] for x in outputs]).mean()
        return  'loss' : loss     
    def forward(self, user_input, item_input):
        
        # Pass through embedding layers
        user_embedded = self.user_embedding(user_input)
        item_embedded = self.item_embedding(item_input)

        # Concat the two embedding layers
        vector = torch.cat([user_embedded, item_embedded], dim=-1)

        # Pass through dense layer
        vector = mish(self.fc1(vector))
        vector = mish(self.fc2(vector))

        # Output layer
        pred = nn.Sigmoid()(self.output(vector))

        return pred
    
    def training_step(self, batch, batch_idx):
        user_input, item_input, labels = batch
        predicted_labels = self(user_input, item_input)
        loss = nn.BCELoss()(predicted_labels, labels.view(-1, 1).float())

        acc = acc_metric(predicted_labels,labels)

        wandb.log("loss": loss,"acc":acc)
        return loss

    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters())

    def train_dataloader(self):
        return DataLoader(MovieLensTrainDataset(self.ratings, self.all_movieIds),
                          batch_size=512, num_workers=4)

# %% [markdown]
# We instantiate the NCF model using the class that we have defined above.

# %% [code]
num_users = ratings['userId'].max()+1
num_items = ratings['movieId'].max()+1

all_movieIds = ratings['movieId'].unique()

model = NCF(num_users, num_items, train_ratings, all_movieIds)


# %% [code]
wandb.watch(model)
early_stopping = EarlyStopping(
        monitor='loss',
        min_delta=0.00,
        patience=3,
        verbose=False,
        mode='min',
    )

trainer = pl.Trainer(max_epochs=100, gpus=1, reload_dataloaders_every_epoch=True,
                     progress_bar_refresh_rate=50, logger=False, checkpoint_callback=True,callbacks=[early_stopping])

trainer.fit(model)


# %% [markdown]
# ### Hit Ratio @ 10 

# %% [code]
# User-item pairs for testing
test_user_item_set = set(zip(test_ratings['userId'], test_ratings['movieId']))

# Dict of all items that are interacted with by each user
user_interacted_items = ratings.groupby('userId')['movieId'].apply(list).to_dict()

hits = []
for (u,i) in tqdm(test_user_item_set):
    interacted_items = user_interacted_items[u]
    not_interacted_items = set(all_movieIds) - set(interacted_items)
    selected_not_interacted = list(np.random.choice(list(not_interacted_items), 99))
    test_items = selected_not_interacted + [i]
    
    predicted_labels = np.squeeze(model(torch.tensor([u]*100), 
                                        torch.tensor(test_items)).detach().numpy())
    
    top10_items = [test_items[i] for i in np.argsort(predicted_labels)[::-1][0:10].tolist()]
    
    if i in top10_items:
        hits.append(1)
    else:
        hits.append(0)
        
print("The Hit Ratio @ 10 is :.2f".format(np.average(hits)))
wandb.log("hit ratio": np.average(hits))

错误：

Traceback (most recent call last):
  File "main.py", line 359, in <module>
    trainer = pl.Trainer(max_epochs=100, gpus=1, reload_dataloaders_every_epoch=True,
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 499, in fit
    self.dispatch()
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 546, in dispatch
    self.accelerator.start_training(self)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 73, in start_training
    self.training_type_plugin.start_training(trainer)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 114, in start_training
    self._results = trainer.run_train()
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 637, in run_train
    self.train_loop.run_training_epoch()
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 492, in run_training_epoch
    batch_output = self.run_training_batch(batch, batch_idx, dataloader_idx)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 654, in run_training_batch
    self.optimizer_step(optimizer, opt_idx, batch_idx, train_step_and_backward_closure)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 425, in optimizer_step
    model_ref.optimizer_step(
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/core/lightning.py", line 1390, in optimizer_step
    optimizer.step(closure=optimizer_closure)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/core/optimizer.py", line 214, in step
    self.__optimizer_step(*args, closure=closure, profiler_name=profiler_name, **kwargs)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/core/optimizer.py", line 134, in __optimizer_step
    trainer.accelerator.optimizer_step(optimizer, self._optimizer_idx, lambda_closure=closure, **kwargs)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 277, in optimizer_step
    self.run_optimizer_step(optimizer, opt_idx, lambda_closure, **kwargs)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 282, in run_optimizer_step
    self.training_type_plugin.optimizer_step(optimizer, lambda_closure=lambda_closure, **kwargs)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 163, in optimizer_step
    optimizer.step(closure=lambda_closure, **kwargs)
  File "/home/max/.local/lib/python3.8/site-packages/torch/optim/optimizer.py", line 89, in wrapper
    return func(*args, **kwargs)
  File "/home/max/.local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/max/.local/lib/python3.8/site-packages/torch/optim/adam.py", line 66, in step
    loss = closure()
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 648, in train_step_and_backward_closure
    result = self.training_step_and_backward(
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 742, in training_step_and_backward
    result = self.training_step(split_batch, batch_idx, opt_idx, hiddens)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 293, in training_step
    training_step_output = self.trainer.accelerator.training_step(args)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 156, in training_step
    return self.training_type_plugin.training_step(*args)
  File "/home/max/.local/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 125, in training_step
    return self.lightning_module.training_step(*args, **kwargs)
  File "main.py", line 318, in training_step
    print(type(labels))
  File "/home/max/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/max/.local/lib/python3.8/site-packages/torchmetrics/metric.py", line 152, in forward
    self.update(*args, **kwargs)
  File "/home/max/.local/lib/python3.8/site-packages/torchmetrics/metric.py", line 199, in wrapped_func
    return update(*args, **kwargs)
  File "/home/max/.local/lib/python3.8/site-packages/torchmetrics/classification/accuracy.py", line 142, in update
    self.correct += correct
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

【问题讨论】：

尝试打印这个：print(next(model.parameters()).device)

@KnowledgeGainer 我得到了cpu

在这一行中尝试将 GPUs 设置为 None 仅用于测试，并判断它是否工作，trainer = pl.Trainer(max_epochs=100, gpus=None, reload_dataloaders_every_epoch=True, progress_bar_refresh_rate= 50, logger=False, checkpoint_callback=True,callbacks=[early_stopping])

@KnowledgeGainer 将 GPU 设置为 None 修复了它。

这有好有坏，好的是你的程序可以运行，但不好的是你的程序只能在CPU上运行，不会使用GPU。你还好吗？

【参考方案1】：

我在这里解释一下，

这个命令：

print(next(model.parameters()).device)

它将打印加载模型参数的设备。

要检查它们是否加载到 GPU 上，您可以这样做：

print(next(model.parameters()).is_cuda)

它会返回一个布尔值，

看到您的代码后，正如您提到的，它在打印时返回“CPU”： 下一步(model.parameters()).device

表示你的模型的参数是加载到 CPU 上的，但是这一行

trainer = pl.Trainer(max_epochs=100, gpus=1, reload_dataloaders_every_epoch=True,
                 progress_bar_refresh_rate=50, logger=False, checkpoint_callback=True,callbacks=[early_stopping])

这里 gpus=1 表示它将设置要训练的gpus数量， 由于默认情况下会加载所有张量 CPU，因此您遇到了该错误。

当你设置了 gpus=None 后，就不再使用 gpus 进行训练了。

在 GPU 上运行：

您必须将张量从 CPU 移动到 GPU，

例如：

ex_tensor=torch.zeros((7,7))
ex_tensor = ex_tensor.cuda()

还有你的模型参数，

model = model.cuda()

【讨论】：

我将张量和模型参数移动到了 gpu。但我得到一个 Cuda 初始化错误。

@maxall4 在这个函数中： def train_dataloader(self): return DataLoader(MovieLensTrainDataset(self.ratings, self.all_movieIds), batch_size=512, num_workers=4) 你能设置 num_workers=0, 是将其设置为 0 后它可以工作。

当我设置 num_workers=0 我得到这个错误：RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

这是我当前的代码：pastebin.com/mcEeMz5M

能否改成：model = model.cuda()