python-运行scrapy Web搜寻器时出错

内容导读

互联网集市收集整理的这篇技术教程文章主要介绍了python-运行scrapy Web搜寻器时出错，小编现在分享给大家，供广大互联网技能从业者学习和参考。文章包含2335字，纯文字阅读大概需要4分钟。

内容图文

import scrapy

class ExampleSpider(scrapy.Spider):
    name = "example"
    allowed_domains = ["dmoz.org"]
    start_urls = [
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
    ]

    def parse(self, response):
        for sel in response.xpath('//ul/li'):
            title = sel.xpath('a/text()').extract()
            link = sel.xpath('a/@href').extract()
            desc = sel.xpath('text()').extract()
            print title, link, desc

但是,当我尝试调用蜘蛛程序时,出现以下错误消息：

[example] ERROR: Spider error processing <GET http://www.dmoz.org/Computers/Programming/Languages/Python/Books/>
    Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/internet/base.py", line 1178, in mainLoop
        self.runUntilCurrent()
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/internet/base.py", line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/internet/defer.py", line 368, in callback
        self._startRunCallbacks(result)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/internet/defer.py", line 464, in _startRunCallbacks
        self._runCallbacks()
    --- <exception caught here> ---
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/internet/defer.py", line 551, in _runCallbacks
        current.result = callback(current.result, *args, **kw)
      File "/Users/andy2/Documents/Python/tutorial/tutorial/spiders/example.py", line 18, in parse
        print title, link, desc
    exceptions.NameError: global name 'link' is not defined

我有什么办法可以使此代码正常工作？

谁能帮我？

谢谢！！！

解决方法:

您需要实例化一个Selector并将响应作为参数传递.另外,您的进口商品不正确.这是蜘蛛的固定版本：

from scrapy.selector import Selector
from scrapy.spider import Spider


class ExampleSpider(Spider):
    name = "example"
    allowed_domains = ["dmoz.org"]
    start_urls = [
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
    ]

    def parse(self, response):
        sel = Selector(response)
        for li in sel.xpath('//ul/li'):
            title = li.xpath('a/text()').extract()
            link = li.xpath('a/@href').extract()
            desc = li.xpath('text()').extract()
            print title, link, desc

内容总结

以上是互联网集市为您收集整理的python-运行scrapy Web搜寻器时出错全部内容，希望文章能够帮你解决python-运行scrapy Web搜寻器时出错所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错，欢迎将互联网集市网站推荐给程序员好友。

内容备注

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 gblab@vip.qq.com 举报，一经查实，本站将立刻删除。

内容手机端