Пытаюсь разобраться scrapy написал тестового паука который должен парсить сайт с https. Запускаю получаю вот такую ошибку
2019-07-25 14:11:07 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: tutorial) 2019-07-25 14:11:07 [scrapy.utils.log] INFO: Versions: lxml 4.3.3.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.0, Python 2.7.5 (default, Apr 9 2019, 14:30:50) - [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)], pyOpenSSL 0.13.1 (OpenSSL 1.0.2k-fips 26 Jan 2017), cryptography 1.7.2, Platform Linux-3.10.0-957.1.3.el7.x86_64-x86_64-with-centos-7.6.1810-Core 2019-07-25 14:11:07 [scrapy.crawler] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tutorial.spiders', 'SPIDER_MODULES': ['tutorial.spiders'], 'ROBOTSTXT_OBEY': True, 'BOT_NAME': 'tutorial'} Traceback (most recent call last): File "/usr/bin/scrapy", line 10, in <module> sys.exit(execute()) File "/usr/lib/python2.7/site-packages/scrapy/cmdline.py", line 150, in execute _run_print_help(parser, _run_command, cmd, args, opts) File "/usr/lib/python2.7/site-packages/scrapy/cmdline.py", line 90, in _run_print_help func(*a, **kw) File "/usr/lib/python2.7/site-packages/scrapy/cmdline.py", line 157, in _run_command cmd.run(args, opts) File "/usr/lib/python2.7/site-packages/scrapy/commands/crawl.py", line 57, in run self.crawler_process.crawl(spname, **opts.spargs) File "/usr/lib/python2.7/site-packages/scrapy/crawler.py", line 171, in crawl crawler = self.create_crawler(crawler_or_spidercls) File "/usr/lib/python2.7/site-packages/scrapy/crawler.py", line 200, in create_crawler return self._create_crawler(crawler_or_spidercls) File "/usr/lib/python2.7/site-packages/scrapy/crawler.py", line 205, in _create_crawler return Crawler(spidercls, self.settings) File "/usr/lib/python2.7/site-packages/scrapy/crawler.py", line 55, in __init__ self.extensions = ExtensionManager.from_crawler(self) File "/usr/lib/python2.7/site-packages/scrapy/middleware.py", line 53, in from_crawler return cls.from_settings(crawler.settings, crawler) File "/usr/lib/python2.7/site-packages/scrapy/middleware.py", line 34, in from_settings mwcls = load_object(clspath) File "/usr/lib/python2.7/site-packages/scrapy/utils/misc.py", line 44, in load_object mod = import_module(module) File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) File "/usr/lib/python2.7/site-packages/scrapy/extensions/memusage.py", line 16, in <module> from scrapy.mail import MailSender File "/usr/lib/python2.7/site-packages/scrapy/mail.py", line 25, in <module> from twisted.internet import defer, reactor, ssl File "/usr/lib64/python2.7/site-packages/twisted/internet/ssl.py", line 230, in <module> from twisted.internet._sslverify import ( File "/usr/lib64/python2.7/site-packages/twisted/internet/_sslverify.py", line 14, in <module> from OpenSSL._util import lib as pyOpenSSLlib
ip install pyopenssl --user --upgrade
Collecting pyopenssl Cache entry deserialization failed, entry ignored Cache entry deserialization failed, entry ignored Downloading https://files.pythonhosted.org/packages/01/c8/ceb170d81bd3941cbeb9940fc6cc2ef2ca4288d0ca8929ea4db5905d904d/pyOpenSSL-19.0.0-py2.py3-none-any.whl (53kB) 100% |████████████████████████████████| 61kB 6.9MB/s Collecting cryptography>=2.3 (from pyopenssl) Cache entry deserialization failed, entry ignored Cache entry deserialization failed, entry ignored Downloading https://files.pythonhosted.org/packages/97/18/c6557f63a6abde34707196fb2cad1c6dc0dbff25a200d5044922496668a4/cryptography-2.7-cp34-abi3-manylinux1_x86_64.whl (2.3MB) 100% |████████████████████████████████| 2.3MB 191kB/s Collecting six>=1.5.2 (from pyopenssl) Cache entry deserialization failed, entry ignored Cache entry deserialization failed, entry ignored Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl Collecting cffi!=1.11.3,>=1.8 (from cryptography>=2.3->pyopenssl) Cache entry deserialization failed, entry ignored Downloading https://files.pythonhosted.org/packages/d5/a1/7e366f126fcd72cbc39a96166fa2b39d52808a233c73bf6405714277a852/cffi-1.12.3-cp34-cp34m-manylinux1_x86_64.whl (429kB) 100% |████████████████████████████████| 430kB 2.6MB/s Collecting asn1crypto>=0.21.0 (from cryptography>=2.3->pyopenssl) Cache entry deserialization failed, entry ignored Cache entry deserialization failed, entry ignored Downloading https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB) 100% |████████████████████████████████| 102kB 8.3MB/s Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography>=2.3->pyopenssl) Cache entry deserialization failed, entry ignored Installing collected packages: six, pycparser, cffi, asn1crypto, cryptography, pyopenssl Successfully installed asn1crypto-0.24.0 cffi-1.12.3 cryptography-2.7 pycparser-2.19 pyopenssl-19.0.0 six-1.12.0 You are using pip version 9.0.1, however version 19.2.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.
Что меня смущает
сторочка You are using pip version 9.0.1, however version 19.2.1 is available. хотя есть набираю pip -V
то получаю
pip 19.1.1 from /var/www/test/test/lib64/python3.4/site-packages/pip (python 3.4)
ну и что собственно к
python -V
Python 3.4.10
А когда запускал паука в разделе INFO фигурировала строчка Python 2.7.5 (default, Apr 9 2019, 14:30:50)
что это и как с этим быть?