Web scraping python

Python scraping

#Scrapes Python's URL, version number and logo from its Wikipedia page:

# $ pip3 install requests beautifulsoup4
import requests, bs4, os, sys

URL = 'https://en.wikipedia.org/wiki/Python_(programming_language)'
try:
    html       = requests.get(URL).text
    document   = bs4.BeautifulSoup(html, 'html.parser')
    table      = document.find('table', class_='infobox vevent')
    python_url = table.find('th', text='Website').next_sibling.a['href']
    version    = table.find('th', text='Stable release').next_sibling.strings.__next__()
    logo_url   = table.find('img')['src']
    logo       = requests.get(f'https:{logo_url}').content
    filename   = os.path.basename(logo_url)
    with open(filename, 'wb') as file:
        file.write(logo)
    print(f'{python_url}, {version}, file://{os.path.abspath(filename)}')
except requests.exceptions.ConnectionError:
    print("You've got problems with connection.", file=sys.stderr)

Source: gto76.github.io

Web scraping using python code

>>> from bs4 import BeautifulSoup
>>> raw_html = open('contrived.html').read()
>>> html = BeautifulSoup(raw_html, 'html.parser')
>>> for p in html.select('p'):
...     if p['id'] == 'walrus':
...         print(p.text)

'I am the walrus'


import requests
from bs4 import BeautifulSoup

URL = 'https://www.monster.com/jobs/search/?q=Software-Developer&where=Australia'
page = requests.get(URL)

soup = BeautifulSoup(page.content, 'html.parser')

Source: grepper.com

Python web scraping

	#Run request
	url = "https://nanolooker.com/account/" + address
    response = requests.get(url)

    if response.ok:
        # Make some soup
    	soup = BeautifulSoup(response.text, "lxml")

        #Show the title
    	print("The title is: " + str(soup.title.string))

Source: grepper.com

Similar codes

2 votes161 views2 answers
Create pdf from images python
Python
0 vote223 views1 answer
Django create token for user
Python Django
2 votes180 views2 answers
Np float to int
Python
7 votes178 views4 answers
How to create 2d list in python
Python
0 vote175 views1 answer
Python save .mat
Python
0 vote246 views1 answer
Python group groupe of 2
Python
0 vote168 views1 answer
Check remote port is open or not using python
Python
11 votes189 views2 answers
Sending whatsapp message using python
Python
13 votes219 views4 answers
Python write list to text file
Python
2 votes216 views1 answer
How to install from url in python
Python