码迷,mamicode.com
首页 > 编程语言 > 详细

python3.7 Scrapy在Windows平台的安装

时间:2019-02-16 11:48:23      阅读:242      评论:0      收藏:0      [点我收藏+]

标签:efi   more   cto   uil   pre   解决   open   val   attr   

1.安装python并配置好环境变量

2.更新pip ,这步操作时遇到了问题,更新超时了,所以加上--default-timeout来解决超时问题:python -m pip --default-timeout=100 install --upgrade pip

3.安装lxml :pip intsall lxml

4.安装wheel: pip install wheel

5.在python lib网站上以下安装包,然后使用pip install xxx 安装:

Twisted-18.9.0-cp37-cp37m-win_amd64.whl

pywin32-224-cp37-cp37m-win_amd64.whl

Scrapy-1.6.0-py2.py3-none-any.whl

Python Lib网站:https://www.lfd.uci.edu/~gohlke/pythonlibs/

6.Scrapy调试,验证是否安装成功:

C:\Users\admin>scrapy
Scrapy 1.6.0 - no active project

Usage:
scrapy <command> [options] [args]

Available commands:
bench Run quick benchmark test
fetch Fetch a URL using the Scrapy downloader
genspider Generate new spider using pre-defined templates
runspider Run a self-contained spider (without creating a project)
settings Get settings values
shell Interactive scraping console
startproject Create new project
version Print Scrapy version
view Open URL in browser, as seen by Scrapy

[ more ] More commands available when run from project directory

Use "scrapy <command> -h" to see more info about a command

部分安装日志如下:

C:\Users\admin>pip install pywin32-224-cp37-cp37m-win_amd64.whl
Processing c:\users\admin\pywin32-224-cp37-cp37m-win_amd64.whl
Installing collected packages: pywin32
Successfully installed pywin32-224

C:\Users\admin>pip install Scrapy-1.6.0-py2.py3-none-any.whl
Processing c:\users\admin\scrapy-1.6.0-py2.py3-none-any.whl
Collecting PyDispatcher>=2.0.5 (from Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/cd/37/39aca520918ce1935bea9c356bcbb7ed7e52ad4e31bff9b943dfc8e7115b/PyDispatcher-2.0.5.tar.gz
Collecting pyOpenSSL (from Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/01/c8/ceb170d81bd3941cbeb9940fc6cc2ef2ca4288d0ca8929ea4db5905d904d/pyOpenSSL-19.0.0-py2.py3-none-any.whl (53kB)
100% |████████████████████████████████| 61kB 64kB/s
Collecting cssselect>=0.9 (from Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/7b/44/25b7283e50585f0b4156960691d951b05d061abf4a714078393e51929b30/cssselect-1.0.3-py2.py3-none-any.whl
Collecting w3lib>=1.17.0 (from Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/81/43/9dcf92a77f5f0afe4f4df2407d7289eea01368a08b64bda00dd318ca62a6/w3lib-1.20.0-py2.py3-none-any.whl
Requirement already satisfied: Twisted>=13.1.0 in d:\python37\lib\site-packages (from Scrapy==1.6.0) (18.9.0)
Collecting service-identity (from Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/e9/7c/2195b890023e098f9618d43ebc337d83c8b38d414326685339eb024db2f6/service_identity-18.1.0-py2.py3-none-any.whl
Collecting queuelib (from Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/4c/85/ae64e9145f39dd6d14f8af3fa809a270ef3729f3b90b3c0cf5aa242ab0d4/queuelib-1.5.0-py2.py3-none-any.whl
Requirement already satisfied: six>=1.5.2 in d:\python37\lib\site-packages (from Scrapy==1.6.0) (1.12.0)
Requirement already satisfied: lxml in d:\python37\lib\site-packages (from Scrapy==1.6.0) (4.3.1)
Collecting parsel>=1.5 (from Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/96/69/d1d5dba5e4fecd41ffd71345863ed36a45975812c06ba77798fc15db6a64/parsel-1.5.1-py2.py3-none-any.whl
Collecting cryptography>=2.3 (from pyOpenSSL->Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/cd/ab/06deda5e26c4290d9973bbf2d53d09d09898f6f972576807c8e76ff13e76/cryptography-2.5-cp37-cp37m-win_amd64.whl (1.5MB)
100% |████████████████████████████████| 1.5MB 17kB/s
Requirement already satisfied: attrs>=17.4.0 in d:\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.6.0) (18.2.0)
Requirement already satisfied: hyperlink>=17.1.1 in d:\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.6.0) (18.0.0)
Requirement already satisfied: zope.interface>=4.4.2 in d:\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.6.0) (4.6.0)
Requirement already satisfied: constantly>=15.1 in d:\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.6.0) (15.1.0)
Requirement already satisfied: incremental>=16.10.1 in d:\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.6.0) (17.5.0)
Requirement already satisfied: Automat>=0.3.0 in d:\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.6.0) (0.7.0)
Requirement already satisfied: PyHamcrest>=1.9.0 in d:\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.6.0) (1.9.0)
Collecting pyasn1-modules (from service-identity->Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/da/98/8ddd9fa4d84065926832bcf2255a2b69f1d03330aa4d1c49cc7317ac888e/pyasn1_modules-0.2.4-py2.py3-none-any.whl (66kB)
100% |████████████████████████████████| 71kB 18kB/s
Collecting pyasn1 (from service-identity->Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/7b/7c/c9386b82a25115cccf1903441bba3cbadcfae7b678a20167347fa8ded34c/pyasn1-0.4.5-py2.py3-none-any.whl (73kB)
100% |████████████████████████████████| 81kB 13kB/s
Collecting cffi!=1.11.3,>=1.8 (from cryptography>=2.3->pyOpenSSL->Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/6b/a4/4d9ba87108405ef0612907c1da4595a2cb4fba91a18105e68c1b7751bfe3/cffi-1.12.0-cp37-cp37m-win_amd64.whl (170kB)
100% |████████████████████████████████| 174kB 11kB/s
Collecting asn1crypto>=0.21.0 (from cryptography>=2.3->pyOpenSSL->Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB)
100% |████████████████████████████████| 102kB 18kB/s
Requirement already satisfied: idna>=2.5 in d:\python37\lib\site-packages (from hyperlink>=17.1.1->Twisted>=13.1.0->Scrapy==1.6.0) (2.8)
Requirement already satisfied: setuptools in d:\python37\lib\site-packages (from zope.interface>=4.4.2->Twisted>=13.1.0->Scrapy==1.6.0) (39.0.1)
Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography>=2.3->pyOpenSSL->Scrapy==1.6.0)
Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
100% |████████████████████████████████| 163kB 16kB/s
Building wheels for collected packages: PyDispatcher, pycparser
Building wheel for PyDispatcher (setup.py) ... done
Stored in directory: C:\Users\admin\AppData\Local\pip\Cache\wheels\88\99\96\cfef6665f9cb1522ee6757ae5955feedf2fe25f1737f91fa7f
Building wheel for pycparser (setup.py) ... done
Stored in directory: C:\Users\admin\AppData\Local\pip\Cache\wheels\f2\9a\90\de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511
Successfully built PyDispatcher pycparser
Installing collected packages: PyDispatcher, pycparser, cffi, asn1crypto, cryptography, pyOpenSSL, cssselect, w3lib, pyasn1, pyasn1-modules, service-identity, queuelib, parsel, Scrapy
Successfully installed PyDispatcher-2.0.5 Scrapy-1.6.0 asn1crypto-0.24.0 cffi-1.12.0 cryptography-2.5 cssselect-1.0.3 parsel-1.5.1 pyOpenSSL-19.0.0 pyasn1-0.4.5 pyasn1-modules-0.2.4 pycparser-2.19 queuelib-1.5.0 service-identity-18.1.0 w3lib-1.20.0

 

python3.7 Scrapy在Windows平台的安装

标签:efi   more   cto   uil   pre   解决   open   val   attr   

原文地址:https://www.cnblogs.com/jwr810/p/10387151.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!