Web scraping with python how to get to the text

I’m trying to get the text from a website but can’t find a way do to it. How do I need to write it?

link="https://www.ynet.co.il/articles/0,7340,L-5553905,00.html"
response = requests.get(link)

soup = BeautifulSoup(response.text,'html.parser')
info = soup.find('div', attrs={'class':'text14'})
name = info.text.strip()
print(name)

This is how it looks: enter image description here

i’m getting none everytime

Answer

import requests
from bs4 import BeautifulSoup
import json
link="https://www.ynet.co.il/articles/0,7340,L-5553905,00.html" 
response = requests.get(link)
soup = BeautifulSoup(response.text,'html.parser') 
info = soup.findAll('script',attrs={'type':"application/ld+json"})[0].text.strip()
jsonDict = json.loads(info)
print(jsonDict['articleBody'])

The page seems to store all the article data in json in the <script> tag so try this code.

Leave a Reply

Your email address will not be published. Required fields are marked *