关于python 2.7：BeautifulSoup：获取特定类别的所有产品链接


BeautifulSoup: Get all product links from specific category

我想通过在python中使用beautifulsoup来获取特定类别的所有产品链接。

我尝试了以下操作，但没有得到结果：

1
2
3
4
5
6
7
8
9
import lxml
import urllib2
from bs4 import BeautifulSoup
html=urllib2.urlopen("http://www.bedbathandbeyond.com/store/category/bedding/bedding/quilts-coverlets/12018/1-96?pagSortOpt=DEFAULT-0&view=grid")

br= BeautifulSoup(html.read(),'lxml')

for links in br.findAll('a', class_='prodImg'):
    print links['href']
相关讨论

你用urllib2错了。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import lxml
import urllib2
from bs4 import BeautifulSoup

#create a http request
req=urllib2.Request("http://www.bedbathandbeyond.com/store/category/bedding/bedding/quilts-coverlets/12018/1-96?pagSortOpt=DEFAULT-0&view=grid")
# send the request
response = urllib2.urlopen(req)
# read the content of the response
html = response.read()
br= BeautifulSoup(html,'lxml')

for links in br.findAll('a', class_='prodImg'):
    print links['href']
相关讨论

1
2
3
4
5
6
7
8
9
10
from bs4 import BeautifulSoup
import requests

html=requests.get("http://www.bedbathandbeyond.com/store/category/bedding/bedding/quilts-coverlets/12018/1-96?pagSortOpt=DEFAULT-0&view=grid")

br= BeautifulSoup(html.content,"lxml")

data=br.findAll('div',attrs={'class':'productShadow'})
for div in br.find_all('a'):
        print div.get('href')

号

尝试此代码