paddleNLP-BUG和一些细节记录

Posted 2022-06-08 汀、

1.TypeError: isfile() takes 1 positional argument but 2 were given

File "/root/miniconda3/envs/paddle_nlp/lib/python3.7/site-packages/paddlenlp/taskflow/information_extraction.py", line 213, in _construct_tokenizer
    self._tokenizer = AutoTokenizer.from_pretrained(self._task_path)
  File "/root/miniconda3/envs/paddle_nlp/lib/python3.7/site-packages/paddlenlp/transformers/auto/tokenizer.py", line 234, in from_pretrained
    pretrained_model_name_or_path, *model_args, **kwargs)
  File "/root/miniconda3/envs/paddle_nlp/lib/python3.7/site-packages/paddlenlp/transformers/tokenizer_utils_base.py", line 1623, in from_pretrained
    '') and os.path.isfile(file_path):
TypeError: isfile() takes 1 positional argument but 2 were given

把源码这个位置的 '' 参数删掉就可以了

2. 只保存最优训练模型

注释掉：

save_dir = os.path.join(args.save_dir, "model_%d" % global_step)
                if not os.path.exists(save_dir):
                    os.makedirs(save_dir)
                model_to_save = model._layers if isinstance(
                    model, paddle.DataParallel) else model
                model_to_save.save_pretrained(save_dir)
                tokenizer.save_pretrained(save_dir)