刮亚马逊品牌页面
Posted
技术标签:
【中文标题】刮亚马逊品牌页面【英文标题】：Scraping Amazon Brand Page 【发布时间】：2020-07-02 13:55:46 【问题描述】：出于培训目的，试图抓取下一页的价格
https://www.amazon.fr/s?i=watches&bbn=60649031&rh=n%3A60649031%2Cp_89%3AICE-Watch&dc&fst=as%3Aoff&qid=1584806397&rnid=1680780031&ref=sr_pg_1
使用Puppeteer，这里是evaluate里面的选择器代码部分：
const result = await page.evaluate(() =>
return [...document.querySelectorAll('.sg-col-inner')].map(o =>
return
brand: o.querySelector('.a-size-base-plus.a-color-base').innerText,
title: o.querySelector('.a-size-base-plus.a-color-base.a-text-normal').innerText,
price: o.querySelector('.a-price-whole').innerText,
url: o.querySelector('.a-link-normal.a-text-normal').href,
;
);
);
brand、title 和 url 工作正常并返回数据，而 price 仍未定义。
错误信息：
error happened Error: Evaluation failed: TypeError: Cannot read property 'innerText' of null
有什么想法吗？
【问题讨论】：
【参考方案1】：看来：
并非所有'.sg-col-inner' 元素都有效，也许'.s-result-list.s-search-results .sg-col-inner' 是更好的选择器。
并非所有元素在链接中都有价格。也许像这样的选择器更好：'.a-price-whole, br + .a-color-base'。
例子：
const puppeteer = require('puppeteer');
(async function main()
try
const browser = await puppeteer.launch();
const [page] = await browser.pages();
await page.goto('https://www.amazon.fr/s?i=watches&bbn=60649031&rh=n%3A60649031%2Cp_89%3AICE-Watch&dc&fst=as%3Aoff&qid=1584806397&rnid=1680780031&ref=sr_pg_1');
const data = await page.evaluate(() =>
return [...document.querySelectorAll('.s-result-list.s-search-results .sg-col-inner')].map(o =>
return
brand: o.querySelector('.a-size-base-plus.a-color-base').innerText,
title: o.querySelector('.a-size-base-plus.a-color-base.a-text-normal').innerText,
price: o.querySelector('.a-price-whole, br + .a-color-base').innerText,
url: o.querySelector('.a-link-normal.a-text-normal').href,
;
);
);
console.log(data);
await browser.close();
catch (err)
console.error(err);
)();
结果：
[
brand: 'ICE-Watch',
title: 'Ice Steel Montre Homme Analogique Quartz avec Bracelet Silicone IC017325',
price: '132,00 €',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Silicone-IC017325/dp/B081S1WZRM/ref=sr_1_1?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-1'
,
brand: 'ICE-Watch',
title: 'IC016982 Montre à Bracelet pour Femme Neuve et Original',
price: '95,95',
url: 'https://www.amazon.fr/Ice-Watch-IC016982-Montre-Bracelet-Original/dp/B07VRZF1P8/ref=sr_1_2?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-2'
,
brand: 'ICE-Watch',
title: 'Ice Flower Montre Femme Analogique Quartz avec Bracelet Silicone IC016665',
price: '95,95',
url: 'https://www.amazon.fr/Flower-Analogique-Bracelet-Silicone-IC016665/dp/B07NYZBT3Q/ref=sr_1_3?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-3'
,
brand: 'ICE-Watch',
title: 'Ice Steel Montre Femme Analogique Quartz avec Bracelet Acier Inoxydable IC017321',
price: '165,00 €',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Inoxydable-IC017321/dp/B081S2LV53/ref=sr_1_4?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-4'
,
brand: 'ICE-Watch',
title: 'Ice Steel Montre Femme Analogique Quartz avec Bracelet Acier Inoxydable IC017319',
price: '165,00 €',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Inoxydable-IC017319/dp/B081S2K22C/ref=sr_1_5?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-5'
,
brand: 'ICE-Watch',
title: 'Ice Steel Montre Femme Analogique Quartz avec Bracelet Acier Inoxydable IC017322',
price: '165,00 €',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Inoxydable-IC017322/dp/B081S1P3SX/ref=sr_1_6?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-6'
,
brand: 'ICE-Watch',
title: 'Ice Change Montre Mixte Analogique Quartz avec Bracelet Silicone IC016052',
price: '86,95',
url: 'https://www.amazon.fr/Change-Analogique-Bracelet-Silicone-IC016052/dp/B081RYZM94/ref=sr_1_7?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-7'
,
brand: 'ICE-Watch',
title: 'Ice Tintin Montre Mixte Analogique Quartz avec Bracelet Silicone IC015306',
price: '99,00',
url: 'https://www.amazon.fr/Tintin-Analogique-Bracelet-Silicone-IC015306/dp/B07VSGY8JG/ref=sr_1_8?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-8'
,
brand: 'ICE-Watch',
title: 'Ice-Glam Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC017057',
price: '99,00 €',
url: 'https://www.amazon.fr/Ice-Glam-Analogique-Bracelet-Caoutchouc-IC017057/dp/B07VS1Z8KX/ref=sr_1_9?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-9'
,
brand: 'ICE-Watch',
title: 'Ice Duo Chic Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC016979',
price: '110,00 €',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC016979/dp/B07VRYBQHH/ref=sr_1_10?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-10'
,
brand: 'ICE-Watch',
title: 'Ice Duo Chic Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC016985',
price: '110,00 €',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC016985/dp/B07VPV5P56/ref=sr_1_11?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-11'
,
brand: 'ICE-Watch',
title: 'IC017058 Montre à Bracelet pour Femme Neuve et Original',
price: '134,90',
url: 'https://www.amazon.fr/Ice-Watch-IC017058-Montre-Bracelet-Original/dp/B07VPSLVXH/ref=sr_1_12?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-12'
,
brand: 'ICE-Watch',
title: 'IC016984 Montre à Bracelet pour Femme Neuve et Original',
price: '99,00',
url: 'https://www.amazon.fr/Ice-Watch-IC016984-Montre-Bracelet-Original/dp/B07VLM2L85/ref=sr_1_13?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-13'
,
brand: 'ICE-Watch',
title: 'Ice lo Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC015606',
price: '99,00',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC015606/dp/B07TJZ3XMW/ref=sr_1_14?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-14'
,
brand: 'ICE-Watch',
title: 'Ice lo Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC016904',
price: '110,00 €',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC016904/dp/B07TJXDKGD/ref=sr_1_15?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-15'
,
brand: 'ICE-Watch',
title: 'ICE Watch Watch IC016769 Femmes Argent/Acier Gris',
price: '162,40',
url: 'https://www.amazon.fr/Watch-IC016769-Femmes-Argent-Acier/dp/B07SXRLHPC/ref=sr_1_16?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-16'
,
brand: 'ICE-Watch',
title: 'ICE Watch IC016763 Montre Unisexe Or Rose Acier',
price: '164,60',
url: 'https://www.amazon.fr/Watch-IC016763-Montre-Unisexe-Acier/dp/B07SXR4WHX/ref=sr_1_17?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-17'
,
brand: 'ICE-Watch',
title: 'ICE Watch Watch IC016777 Femmes Or Acier',
price: '146,80',
url: 'https://www.amazon.fr/ICE-Watch-IC016777-Femmes-Acier/dp/B07STJ7FDZ/ref=sr_1_18?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-18'
,
brand: 'ICE-Watch',
title: 'Ice lo Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC015604',
price: '95,95',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC015604/dp/B07STF8T72/ref=sr_1_19?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-19'
,
brand: 'ICE-Watch',
title: 'ICE Watch IC016771 Montre en Silicone Argent/Gris pour Homme',
price: '116,90',
url: 'https://www.amazon.fr/Watch-IC016771-Montre-Silicone-Argent/dp/B07SSH93YX/ref=sr_1_20?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-20'
,
brand: 'ICE-Watch',
title: 'ICE Watch Watch IC016891 Femmes Argent/Acier Gris',
price: '144,60',
url: 'https://www.amazon.fr/Watch-IC016891-Femmes-Argent-Acier/dp/B07SRGQGVV/ref=sr_1_21?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-21'
,
brand: 'ICE-Watch',
title: 'Ice lo Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC015605',
price: '99,00',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC015605/dp/B07SRCM67N/ref=sr_1_22?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-22'
,
brand: 'ICE-Watch',
title: 'Ice lo Montre Femme Analogique Quartz avec Bracelet Caoutchouc IC013990',
price: '97,94',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Caoutchouc-IC013990/dp/B07SQ9JLSV/ref=sr_1_23?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-23'
,
brand: 'ICE-Watch',
title: 'ICE Watch IC016761 Montre Unisexe Or Acier',
price: '163,50',
url: 'https://www.amazon.fr/Watch-IC016761-Montre-Unisexe-Acier/dp/B07RL3DD8Q/ref=sr_1_24?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-24'
,
brand: 'ICE-Watch',
title: 'Ice-Sunshine - Sun.NYW.S.S.13 : Montre Mixte Résine & Silicone - Mouvement Quartz - Cadran Jaune',
price: '46,46',
url: 'https://www.amazon.fr/Ice-Watch-Ice-Sunshine-Sun-NYW-S-S-13-Silicone-Mouvement/dp/B07QR42J5M/ref=sr_1_25?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-25'
,
brand: 'ICE-Watch',
title: 'Ice Urban Montre Homme Analogique Quartz avec Bracelet Silicone IC016307',
price: '143,00 €',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Silicone-IC016307/dp/B07PQ7FWFM/ref=sr_1_26?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-26'
,
brand: 'ICE-Watch',
title: '015765 Montre Femme',
price: '82,28',
url: 'https://www.amazon.fr/Ice-Watch-015765-Montre-Pastel-Femme/dp/B07NJRMCF7/ref=sr_1_27?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-27'
,
brand: 'ICE-Watch',
title: '015761 Montre Femme',
price: '82,28',
url: 'https://www.amazon.fr/Ice-Watch-015761-Montre-Femme-Pastel/dp/B07NJRCBL2/ref=sr_1_28?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-28'
,
brand: 'ICE-Watch',
title: '015758 Montre Femme',
price: '82,28',
url: 'https://www.amazon.fr/Ice-Watch-015758-Montre-Pastel-Femme/dp/B07NJR6Q7S/ref=sr_1_29?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-29'
,
brand: 'ICE-Watch',
title: 'Ice Ola Montre Mixte Analogique Quartz avec Bracelet Silicone Ice.PE.U.S.15',
price: '79,00',
url: 'https://www.amazon.fr/Montre-Analogique-Bracelet-Silicone-Ice-PE-U-S-15/dp/B07NF47C47/ref=sr_1_30?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-30'
,
brand: 'ICE-Watch',
title: 'Ice Watch Loveheart Montre analogique',
price: '50,40',
url: 'https://www.amazon.fr/ICE-Watch-Montre-013730/dp/B074N8LD1R/ref=sr_1_31?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-31'
,
brand: 'ICE-Watch',
title: 'Ice Safari Montre Mixte Analogique Quartz avec Bracelet Silicone SP.Ice.CHA.U.S.15',
price: '89,00',
url: 'https://www.amazon.fr/Safari-Analogique-Bracelet-Silicone-SP-Ice-CHA-U-S-15/dp/B017DGW3VW/ref=sr_1_32?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-32'
,
brand: 'ICE-Watch',
title: 'Ice-Chocolate Montre Mixte Analogique Quartz avec Bracelet Silicone CT.CA.U.S.10',
price: '89,95',
url: 'https://www.amazon.fr/Ice-Chocolate-Analogique-Bracelet-Silicone-CT-CA-U-S-10/dp/B00CFP0M5Q/ref=sr_1_33?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-33'
,
brand: 'ICE-Watch',
title: 'Stone Multifunction Montre Femme Unisexe MF. WS. U.S.10, Bracelet Plastique, Analogique, Quartz, 5 Bar, Verre minéral One Size, Blanc, Blanc',
price: '159,90',
url: 'https://www.amazon.fr/Multifunction-unisexe-Bracelet-Plastique-Analogique/dp/B004N4Z9UA/ref=sr_1_34?fst=as%3Aoff&qid=1584870152&refinements=p_89%3AICE-Watch&rnid=1680780031&s=watch&sr=1-34'
]
【讨论】：
您能否提供有关该错误的任何信息？我添加了一个代码示例及其结果，它对我有用。 请注意，虽然有些价格以' €' 结尾。以上是关于刮亚马逊品牌页面的主要内容，如果未能解决你的问题，请参考以下文章