代码之家 › 专栏 › 技术社区 › Minial

使用beautifulsoup从<a>href链接获取文本

beautifulsoup

Minial · 技术社区 · 7 年前

他们已经成功了,但现在我想从文本中收集数据;而不是标题。等: “小米70Mai Pro”

我试着引用这两个来源;但我还是不确定。。。

WebScraper-Sample

Parse HTML Table for URL, Place into List

 links = 'SampleLink... with table cell'

 def getURLData(url):  # scrap data from Link
 try:
     page = requests.get(url)
     content = page.content
     soup = BeautifulSoup(content, "html.parser")
     return soup
 except Exception as e:
     print('Error.getURLData:', e)
     return None


 inputLink = getURLData(links)


 def tableCheck():  # if there's a table cell;
 data = []
 for table_tag in inputLink.find_all('td', {'class': 'row1'}):
     topic_title = table_tag.find('a', href=True)
     if topic_title:
         datum = {'topic_title': topic_title['title']}
         data.append(datum)
 return data


 print(tableCheck())

这就是输出

  {'topic_title': 'This topic was started: Dec 6 2018, 12:20 PM'}, 
  {'topic_title': 'This topic was started: Nov 19 2018, 10:30 AM'}, 
  {'topic_title': 'This topic was started: Nov 28 2018, 09:16 PM'},
  {'topic_title': 'This topic was started: Oct 3 2018, 11:10 AM'},

这就是我试图从中提取数据的单元;我已经试着使用 topic_title = table_tag.find('a', href=True).text

<td class = "row1" valign = "middle" >
   <div >
        <div style = "float:left" >
           <a href = "/topic/4667583" title = "This topic was started: Oct 3 2018, 
            11:10 AM" > 
           Xiaomi 70Mai Pro < /a >
        </div >
        <br >
    </div >
    </td

1 回复 | 直到 5 年前

alecxe 7 年前

要添加到现有答案中,您需要做的唯一修改是 :

topic_title = table_tag.find('a', href=True)
if topic_title:
    datum = {
        'topic_title': topic_title['title'], 
        'topic_text': topic_title.text
    }
    data.append(datum)

推荐文章

Dinosaur · 使用BeautifulSoup点击div标签后抓取html页面

1 年前

Stackie · 无法使用Selenium访问废料数据的链接

1 年前

Avraham · 如何在JS中将beautifulsoup中的文本设置为.innerText而非.textContent

1 年前