Python web crawler

import scrapy

class BlogSpider(scrapy.Spider):
    name = 'blogspider'
    start_urls = ['https://blog.scrapinghub.com']

    def parse(self, response):
        for title in response.css('.post-header>h2'):
            yield {'title': title.css('a ::text').get()}

        for next_page in response.css('a.next-posts-link'):
            yield response.follow(next_page, self.parse)

Source: scrapy.org

Similar codes

0 vote138 views1 answer
Django no such table
Python
1 vote134 views2 answers
Change base python
Python
0 vote143 views1 answer
How to find outliers in python
Python
0 vote132 views2 answers
How to make SAPI.SpVoice say a variable in python
Python
69 votes134 views3 answers
Puthon for loop
Python
0 vote125 views1 answer
Remove dot from number python
Python
0 vote172 views1 answer
Check for nat entries pandas
Python
7 votes135 views2 answers
How to get index of pandas dataframe python
Python
0 vote129 views1 answer
Modulenotfounderror no module named 'config' python
Python
0 vote88 views1 answer
Python ansi color codes
Python