Python web crawler

import scrapy

class BlogSpider(scrapy.Spider):
    name = 'blogspider'
    start_urls = ['https://blog.scrapinghub.com']

    def parse(self, response):
        for title in response.css('.post-header>h2'):
            yield {'title': title.css('a ::text').get()}

        for next_page in response.css('a.next-posts-link'):
            yield response.follow(next_page, self.parse)

Source: scrapy.org

Similar codes

0 vote138 views1 answer
Make_interp_spline
Python
0 vote156 views1 answer
Merge dataframe using pandas
Python
0 vote149 views2 answers
Allow x_frame_options django
Python
53 votes162 views2 answers
Add values to dictionary key python
Python
0 vote142 views1 answer
Converting double spaces int single spaces python
Python
0 vote136 views1 answer
Updating many to many field in django using pk
Python
1 vote148 views1 answer
How to open dialog box to select files in python
Python
3 votes144 views3 answers
Python zip()
Python
7 votes102 views1 answer
Telegram markdown
Python
0 vote141 views1 answer
Cbind arrays python
Python