Python爬虫

2020/07/16 Python 共 574 字,约 2 分钟

按照一定的规则获取数据

BeautifulSoup4

from urllib.request import urlopen
from bs4 import BeautifulSoup

# 爬取imdb评分榜
html = urlopen("https://www.imdb.com/chart/top").read()
soup = BeautifulSoup(html, features="lxml")  # 解析方式

# 也可使用soup.select('标签.属性1.属性2')
tbody = soup.find('tbody', attrs={'class': 'lister-list'})
tr = tbody.find_all('tr')

for item in tr:
    title = item.find('td', attrs={'class': 'titleColumn'}).find('a').get_text()
    href = item.find('td', attrs={'class': 'titleColumn'}).find('a').get('href')
    rating = item.find('td', attrs={'class': 'imdbRating'}).find('strong').get_text()
    print(title, rating, href)

Scrapy

待更新

Selenium

待更新

文档信息

Search

    Table of Contents