python爬虫：scrapy自定义item

Posted 2020-11-29 HuaBro

items.py

class LianhezaobaospyderItem(scrapy.Item):
    # define the fields for your item here like:
    # name = scrapy.Field()
    # pass
    body=scrapy.Field()
    link=scrapy.Field()

爬虫.py

from .. import items

def parse_news(self,response):
    item=items.LianhezaobaospyderItem()                
    item[‘body‘]=response.xpath("//div[@class=‘xx‘]/text()").get()
    item[‘link‘]=response.url
    yield item    

item和字典类似，数据量大时，字典可能键值对错误