刮亚马逊品牌页面

Posted

技术标签:

【中文标题】刮亚马逊品牌页面【英文标题】:Scraping Amazon Brand Page 【发布时间】:2020-07-02 13:55:46 【问题描述】:

出于培训目的,试图抓取下一页的价格

https://www.amazon.fr/s?i=watches&bbn=60649031&rh=n%3A60649031%2Cp_89%3AICE-Watch&dc&fst=as%3Aoff&qid=1584806397&rnid=1680780031&ref=sr_pg_1

使用Puppeteer,这里是evaluate里面的选择器代码部分:

        const result = await page.evaluate(() =>
        return [...document.querySelectorAll('.sg-col-inner')].map(o => 
            return 
              brand: o.querySelector('.a-size-base-plus.a-color-base').innerText,
              title: o.querySelector('.a-size-base-plus.a-color-base.a-text-normal').innerText,
              price: o.querySelector('.a-price-whole').innerText,
              url: o.querySelector('.a-link-normal.a-text-normal').href,
            ;
          );
      );

brand、title 和 url 工作正常并返回数据,而 price 仍未定义。

错误信息:

error happened Error: Evaluation failed: TypeError: Cannot read property 'innerText' of null

有什么想法吗?

【问题讨论】:

【参考方案1】:

看来:

    并非所有'.sg-col-inner' 元素都有效,也许'.s-result-list.s-search-results .sg-col-inner' 是更好的选择器。

    并非所有元素在链接中都有价格。也许像这样的选择器更好:'.a-price-whole, br + .a-color-base'

例子:

const puppeteer = require('puppeteer');

(async function main() 
  try 
    const browser = await puppeteer.launch();
    const [page] = await browser.pages();

    await page.goto('https://www.amazon.fr/s?i=watches&bbn=60649031&rh=n%3A60649031%2Cp_89%3AICE-Watch&dc&fst=as%3Aoff&qid=1584806397&rnid=1680780031&ref=sr_pg_1');

    const data = await page.evaluate(() => 
      return [...document.querySelectorAll('.s-result-list.s-search-results .sg-col-inner')].map(o => 
        return 
          brand: o.querySelector('.a-size-base-plus.a-color-base').innerText,
          title: o.querySelector('.a-size-base-plus.a-color-base.a-text-normal').innerText,
          price: o.querySelector('.a-price-whole, br + .a-color-base').innerText,
          url: o.querySelector('.a-link-normal.a-text-normal').href,
        ;
      );
    );

    console.log(data);

    await browser.close();
   catch (err) 
    console.error(err);
  
)();

结果:

[
  
    brand: 'ICE-Watch',
    title: 'Ice Steel Montre Homme Analogique Quartz avec Bracelet Silicone IC017325',
    price: '132,00 €',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Silicone-IC017325/dp/B081S1WZRM/ref=sr_1_1?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-1'
  ,
  
    brand: 'ICE-Watch',
    title: 'IC016982 Montre à Bracelet pour Femme Neuve et Original',
    price: '95,95',
    url: 'https://www.amazon.fr/Ice-Watch-IC016982-Montre-Bracelet-Original/dp/B07VRZF1P8/ref=sr_1_2?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-2'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Flower Montre Femme Analogique Quartz avec Bracelet Silicone IC016665',
    price: '95,95',
    url: 'https://www.amazon.fr/Flower-Analogique-Bracelet-Silicone-IC016665/dp/B07NYZBT3Q/ref=sr_1_3?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-3'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Steel Montre Femme Analogique Quartz avec Bracelet Acier Inoxydable IC017321',
    price: '165,00 €',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Inoxydable-IC017321/dp/B081S2LV53/ref=sr_1_4?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-4'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Steel Montre Femme Analogique Quartz avec Bracelet Acier Inoxydable IC017319',
    price: '165,00 €',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Inoxydable-IC017319/dp/B081S2K22C/ref=sr_1_5?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-5'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Steel Montre Femme Analogique Quartz avec Bracelet Acier Inoxydable IC017322',
    price: '165,00 €',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Inoxydable-IC017322/dp/B081S1P3SX/ref=sr_1_6?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-6'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Change Montre Mixte Analogique Quartz avec Bracelet Silicone IC016052',
    price: '86,95',
    url: 'https://www.amazon.fr/Change-Analogique-Bracelet-Silicone-IC016052/dp/B081RYZM94/ref=sr_1_7?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-7'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Tintin Montre Mixte Analogique Quartz avec Bracelet Silicone IC015306',
    price: '99,00',
    url: 'https://www.amazon.fr/Tintin-Analogique-Bracelet-Silicone-IC015306/dp/B07VSGY8JG/ref=sr_1_8?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-8'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice-Glam Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC017057',
    price: '99,00 €',
    url: 'https://www.amazon.fr/Ice-Glam-Analogique-Bracelet-Caoutchouc-IC017057/dp/B07VS1Z8KX/ref=sr_1_9?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-9'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Duo Chic Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC016979',
    price: '110,00 €',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC016979/dp/B07VRYBQHH/ref=sr_1_10?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-10'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Duo Chic Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC016985',
    price: '110,00 €',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC016985/dp/B07VPV5P56/ref=sr_1_11?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-11'
  ,
  
    brand: 'ICE-Watch',
    title: 'IC017058 Montre à Bracelet pour Femme Neuve et Original',
    price: '134,90',
    url: 'https://www.amazon.fr/Ice-Watch-IC017058-Montre-Bracelet-Original/dp/B07VPSLVXH/ref=sr_1_12?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-12'
  ,
  
    brand: 'ICE-Watch',
    title: 'IC016984 Montre à Bracelet pour Femme Neuve et Original',
    price: '99,00',
    url: 'https://www.amazon.fr/Ice-Watch-IC016984-Montre-Bracelet-Original/dp/B07VLM2L85/ref=sr_1_13?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-13'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice lo Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC015606',
    price: '99,00',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC015606/dp/B07TJZ3XMW/ref=sr_1_14?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-14'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice lo Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC016904',
    price: '110,00 €',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC016904/dp/B07TJXDKGD/ref=sr_1_15?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-15'
  ,
  
    brand: 'ICE-Watch',
    title: 'ICE Watch Watch IC016769 Femmes Argent/Acier Gris',
    price: '162,40',
    url: 'https://www.amazon.fr/Watch-IC016769-Femmes-Argent-Acier/dp/B07SXRLHPC/ref=sr_1_16?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-16'
  ,
  
    brand: 'ICE-Watch',
    title: 'ICE Watch IC016763 Montre Unisexe Or Rose Acier',
    price: '164,60',
    url: 'https://www.amazon.fr/Watch-IC016763-Montre-Unisexe-Acier/dp/B07SXR4WHX/ref=sr_1_17?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-17'
  ,
  
    brand: 'ICE-Watch',
    title: 'ICE Watch Watch IC016777 Femmes Or Acier',
    price: '146,80',
    url: 'https://www.amazon.fr/ICE-Watch-IC016777-Femmes-Acier/dp/B07STJ7FDZ/ref=sr_1_18?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-18'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice lo Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC015604',
    price: '95,95',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC015604/dp/B07STF8T72/ref=sr_1_19?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-19'
  ,
  
    brand: 'ICE-Watch',
    title: 'ICE Watch IC016771 Montre en Silicone Argent/Gris pour Homme',
    price: '116,90',
    url: 'https://www.amazon.fr/Watch-IC016771-Montre-Silicone-Argent/dp/B07SSH93YX/ref=sr_1_20?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-20'
  ,
  
    brand: 'ICE-Watch',
    title: 'ICE Watch Watch IC016891 Femmes Argent/Acier Gris',
    price: '144,60',
    url: 'https://www.amazon.fr/Watch-IC016891-Femmes-Argent-Acier/dp/B07SRGQGVV/ref=sr_1_21?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-21'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice lo Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC015605',
    price: '99,00',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC015605/dp/B07SRCM67N/ref=sr_1_22?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-22'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice lo Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC013990',
    price: '97,94',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC013990/dp/B07SQ9JLSV/ref=sr_1_23?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-23'
  ,
  
    brand: 'ICE-Watch',
    title: 'ICE Watch IC016761 Montre Unisexe Or Acier',
    price: '163,50',
    url: 'https://www.amazon.fr/Watch-IC016761-Montre-Unisexe-Acier/dp/B07RL3DD8Q/ref=sr_1_24?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-24'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice-Sunshine - Sun.NYW.S.S.13 : Montre Mixte Résine & Silicone - Mouvement Quartz - Cadran Jaune',
    price: '46,46',
    url: 'https://www.amazon.fr/Ice-Watch-Ice-Sunshine-Sun-NYW-S-S-13-Silicone-Mouvement/dp/B07QR42J5M/ref=sr_1_25?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-25'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Urban Montre Homme Analogique Quartz avec Bracelet Silicone IC016307',
    price: '143,00 €',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Silicone-IC016307/dp/B07PQ7FWFM/ref=sr_1_26?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-26'
  ,
  
    brand: 'ICE-Watch',
    title: '015765 Montre Femme',
    price: '82,28',
    url: 'https://www.amazon.fr/Ice-Watch-015765-Montre-Pastel-Femme/dp/B07NJRMCF7/ref=sr_1_27?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-27'
  ,
  
    brand: 'ICE-Watch',
    title: '015761 Montre Femme',
    price: '82,28',
    url: 'https://www.amazon.fr/Ice-Watch-015761-Montre-Femme-Pastel/dp/B07NJRCBL2/ref=sr_1_28?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-28'
  ,
  
    brand: 'ICE-Watch',
    title: '015758 Montre Femme',
    price: '82,28',
    url: 'https://www.amazon.fr/Ice-Watch-015758-Montre-Pastel-Femme/dp/B07NJR6Q7S/ref=sr_1_29?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-29'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Ola Montre Mixte Analogique Quartz avec Bracelet Silicone Ice.PE.U.S.15',
    price: '79,00',
    url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Silicone-Ice-PE-U-S-15/dp/B07NF47C47/ref=sr_1_30?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-30'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Watch Loveheart Montre analogique',
    price: '50,40',
    url: 'https://www.amazon.fr/ICE-Watch-Montre-013730/dp/B074N8LD1R/ref=sr_1_31?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-31'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice Safari Montre Mixte Analogique Quartz avec Bracelet Silicone SP.Ice.CHA.U.S.15',
    price: '89,00',
    url: 'https://www.amazon.fr/Safari-Analogique-Bracelet-Silicone-SP-Ice-CHA-U-S-15/dp/B017DGW3VW/ref=sr_1_32?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-32'
  ,
  
    brand: 'ICE-Watch',
    title: 'Ice-Chocolate Montre Mixte Analogique Quartz avec Bracelet Silicone CT.CA.U.S.10',
    price: '89,95',
    url: 'https://www.amazon.fr/Ice-Chocolate-Analogique-Bracelet-Silicone-CT-CA-U-S-10/dp/B00CFP0M5Q/ref=sr_1_33?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-33'
  ,
  
    brand: 'ICE-Watch',
    title: 'Stone Multifunction Montre Femme Unisexe MF. WS. U.S.10, Bracelet Plastique, Analogique, Quartz, 5 Bar, Verre minéral One Size, Blanc, Blanc',
    price: '159,90',
    url: 'https://www.amazon.fr/Multifunction-unisexe-Bracelet-Plastique-Analogique/dp/B004N4Z9UA/ref=sr_1_34?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-34'
  
]

【讨论】:

您能否提供有关该错误的任何信息?我添加了一个代码示例及其结果,它对我有用。 请注意,虽然有些价格以' €' 结尾。

以上是关于刮亚马逊品牌页面的主要内容,如果未能解决你的问题,请参考以下文章

刮亚马逊所有交易php curl?

劲爆消息!亚马逊品牌备案开放了(内附详细流程图)

使用 python 从亚马逊页面获取 ASIN 编号

亚马逊实践领域驱动设计之道

亚马逊 MWS - 内嵌图片

亚马逊美国Lightning_Deals爬虫