来自 *** 公司的网络抓取公司描述

Posted 2023-02-23

【中文标题】来自 *** 公司的网络抓取公司描述

【英文标题】：Web scraping company description from *** companies

【发布时间】：2021-02-20 01:27:30

【问题描述】：

我正在尝试使用 beautifulsoup 从https://***.com/jobs/companies 获取所有公司（所有页面）的描述，并且无法确定是否以及如何使用“mt8 mb0 fs-body1 fc-black-700”类

from bs4 import BeautifulSoup
import requests

page = requests.get('https://***.com/jobs/companies')

soup = BeautifulSoup(page.text, "html.parser")


descriptions = soup.find_all('p', class_ = 'mt8 mb0 fs-body1 fc-black-700')

for description in descriptions:
    print(description)
    print()

【问题讨论】：

你好 :) 你能告诉我们你到目前为止做了什么吗？

您好，我添加了我尝试过的内容。我得到了描述和类名作为输出，但现在有了下面的答案，它工作得很好！我是编码新手，尤其是网络报废。现在我将尝试制作一个循环，以便我可以从所有页面中获取描述:) @Shamhirsaz.Navid

【参考方案1】：

是的。您可以使用mt8 mb0 fs-body1 fc-black-700 作为class name。你的最终代码应该是这样的：

from bs4 import BeautifulSoup
import requests

r = requests.get('https://***.com/jobs/companies').text

soup = BeautifulSoup(r,'html5lib')

descriptions = soup.find_all('p', class_ = 'mt8 mb0 fs-body1 fc-black-700')

for desc in descriptions:
    print(desc.text)

输出：

Verys was founded in 2012 by Mike Zerkel and Chris Antonius to help bring cutting edge technology innovation back to the United States – p...
We are a global team with plenty ofenterpriseand startup experience. Mix of creative, strategists, technologists, and engineer...
You’ll work with a talented and highly-motivated team that is friendly, fast-moving, self-managing, and highly capable with a sense of hum...
We take the long view of software development and maintenance by
About OutMatch
The OutMatch Talent Intelligence Platform brings clarity to talent decisions by gat...
We do way more than the usual software full stack development. We are also mechanical, electronics hardware, and firmware engineers: Angular.js...
At Citizens, we’re more than a network of brick and mortar branches. We’re a digital company that happens to be a bank.
We h...
Kaliber Labs is a full stack artificial intelligence company focused on bringing objectivity and consistency to the surgical setting with...
Our technologists work on a diverse range of solutions that include strategic technology initiatives, big data, mobile, electronic payments, mac...
We believe in cross-functional, autonomous teams owning strategic parts of the product, end-to-end.
A career at Paige is deeply mission-...