类 InstagramSpider(scrapy.Spider): AttributeError: 'module' 对象没有属性 'Spider'

Posted 2023-03-31

【中文标题】类 InstagramSpider(scrapy.Spider): AttributeError: \'module\' 对象没有属性 \'Spider\'

【英文标题】：class InstagramSpider(scrapy.Spider): AttributeError: 'module' object has no attribute 'Spider'类 InstagramSpider(scrapy.Spider): AttributeError: 'module' 对象没有属性 'Spider'

【发布时间】：2017-03-01 23:28:07

【问题描述】：

我应该如何解决这个错误？

mona@pascal:~/computer_vision/instagram/instagram$ scrapy crawl instagramspider
2017-03-01 17:24:03-0600 [scrapy] INFO: Scrapy 0.14.4 started (bot: instagram)
2017-03-01 17:24:04-0600 [scrapy] DEBUG: Enabled extensions: LogStats, TelnetConsole, CloseSpider, WebService, CoreStats, MemoryUsage, SpiderState
Traceback (most recent call last):
  File "/usr/bin/scrapy", line 4, in <module>
    execute()
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 132, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 97, in _run_print_help
    func(*a, **kw)
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 139, in _run_command
    cmd.run(args, opts)
  File "/usr/lib/python2.7/dist-packages/scrapy/commands/crawl.py", line 43, in run
    spider = self.crawler.spiders.create(spname, **opts.spargs)
  File "/usr/lib/python2.7/dist-packages/scrapy/command.py", line 34, in crawler
    self._crawler.configure()
  File "/usr/lib/python2.7/dist-packages/scrapy/crawler.py", line 36, in configure
    self.spiders = spman_cls.from_crawler(self)
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 37, in from_crawler
    return cls.from_settings(crawler.settings)
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 33, in from_settings
    return cls(settings.getlist('SPIDER_MODULES'))
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 23, in __init__
    for module in walk_modules(name):
  File "/usr/lib/python2.7/dist-packages/scrapy/utils/misc.py", line 65, in walk_modules
    submod = __import__(fullpath, , , [''])
  File "/home/mona/computer_vision/instagram/instagram/instagram/spiders/spider.py", line 7, in <module>
    class InstagramSpider(scrapy.Spider):
AttributeError: 'module' object has no attribute 'Spider'

更多信息可以在这里找到： From scraper_user.items import UserItem ImportError: No module named scraper_user.items

Python 2.7.6 (default, Oct 26 2016, 20:30:19) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import scrapy
>>> print scrapy.__file__
/usr/lib/python2.7/dist-packages/scrapy/__init__.pyc
>>> print(dir(scrapy))
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__', 'boto', 'optional_features', 'os', 'sys', 'twisted_250_monkeypatches', 'urlparse_monkeypatches', 'version_info', 'warnings', 'xlib']
>>> print(scrapy.spider)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'spider'

【问题讨论】：

我建议你使用 Instagram API 而不是使用 scrapy。使用 API 解析和爬取页面，您不会感到痛苦

【参考方案1】：

根据输出，您使用的是非常非常旧的 Scrapy 0.14.4，它没有将 Spider 暴露到 scrapy 命名空间中。

任一更新：

pip install scrapy --upgrade

或者，将您的导入更改为（不确定这是否可以正常工作）：

from scrapy.spider import BaseSpider as Spider

【讨论】：

确认从 scrapy.spider import BaseSpider as Spider 工作但不是升级。 pastebin.com/6YB0TZRs 谢谢！

@MonaJalal 是的，由于某种原因，您基本上无法安装最新的scrapy 版本。它报告它正在安装，但scrapy 命令仍然“绑定”到 0.14.4。很高兴解决方法奏效了，尽管我会努力使升级成为可能 - 0.14.4 真的非常老了 - 从那时起很多事情都发生了变化。

你能看看这个问题吗？ ***.com/questions/42545020/…

【参考方案2】：

你可能需要通过你的包管理器升级scrapy，看来你使用的是基于Linux的操作系统。

如果您有 Python 2 和 3，请尝试同时使用 pip2 和 pip3 进行升级