豆瓣图书爬虫程序
This topic created in 3629 days ago, the information mentioned may be changed or developed.
Supplement 1 · Jul 2, 2016
我换 linux 环境现在可以运行了，运行$ python doubanSpider.py 后一直在下载，是什么意思额？
/usr/local/lib/python2.7/dist-packages/bs4/__init__.py:166: UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html.parser"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.
To get rid of this warning, change this:
BeautifulSoup([your markup])
to this:
BeautifulSoup([your markup], "html.parser")
markup_type=markup_type))
Downloading Information From Page 1
Downloading Information From Page 2
Downloading Information From Page 3
Downloading Information From Page 4
Downloading Information From Page 5
Downloading Information From Page 6
WARNING:root:Some characters could not be decoded, and were replaced with REPLACEMENT CHARACTER.
Downloading Information From Page 7
Downloading Information From Page 8
Downloading Information From Page 9
Downloading Information From Page 10
Downloading Information From Page 11
Downloading Information From Page 12
Downloading Information From Page 13
Downloading Information From Page 14
Downloading Information From Page 15
Downloading Information From Page 16
Downloading Information From Page 17
Downloading Information From Page 18
Downloading Information From Page 19
Downloading Information From Page 20
Downloading Information From Page 21
Downloading Information From Page 22
Downloading Information From Page 23
Downloading Information From Page 24
/usr/local/lib/python2.7/dist-packages/bs4/__init__.py:166: UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html.parser"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.
To get rid of this warning, change this:
BeautifulSoup([your markup])
to this:
BeautifulSoup([your markup], "html.parser")
markup_type=markup_type))
Downloading Information From Page 1
Downloading Information From Page 2
Downloading Information From Page 3
Downloading Information From Page 4
Downloading Information From Page 5
Downloading Information From Page 6
WARNING:root:Some characters could not be decoded, and were replaced with REPLACEMENT CHARACTER.
Downloading Information From Page 7
Downloading Information From Page 8
Downloading Information From Page 9
Downloading Information From Page 10
Downloading Information From Page 11
Downloading Information From Page 12
Downloading Information From Page 13
Downloading Information From Page 14
Downloading Information From Page 15
Downloading Information From Page 16
Downloading Information From Page 17
Downloading Information From Page 18
Downloading Information From Page 19
Downloading Information From Page 20
Downloading Information From Page 21
Downloading Information From Page 22
Downloading Information From Page 23
Downloading Information From Page 24
 |
1
upczww Jul 1, 2016 via Smartisan T1
没有代码别人怎么帮你？
|
 |
2
grey5659 OP @upczww 就是这个 http://blog.csdn.net/lanbing510/article/details/45887075 .py 文件如何访问？
|
 |
3
AnonymousID Jul 1, 2016 via Android
难道不是直接运行那个 py 文件？
|
|
4
grey5659 OP @AnonymousID 不会吧？
|
|
5
AnonymousID Jul 1, 2016 via Android
@grey5659 明明就是啊，不就那一个文件可以执吗
|
|
6
AnonymousID Jul 1, 2016 via Android
楼上漏了个 行 字
|
 |
7
niboy Jul 1, 2016
首先要安装 python ，到 python.org 上下载安装，然后双击运行 python 文件或者 python ***.py
|
|
8
grey5659 OP @niboy 安装了 双击一闪而过，改名成 doubanSpider.pyw 用 IDLE 打开后，运行 run module 提示 Traceback (most recent call last):
File "C:\Users\lenovo\Desktop\DouBanSpider-master\doubanSpider.pyw", line 7, in <module> import requests ImportError: No module named requests |
|
9
niboy Jul 1, 2016
@grey5659
你缺少 requests 依赖包。。 http://blog.csdn.net/alpha5/article/details/24964009 其他的，你自己想办法解决吧，比如下面这些。。 import numpy as np from bs4 import BeautifulSoup from openpyxl import Workbook |
|
10
upczww Jul 1, 2016
就一个文件，直接运行就好了
|
 |
11
ksupertu Jul 1, 2016
安装 python2.7 然后 cmd 运行 pip install requests ，省事的话就虚拟个 ubuntu 去运行，不然 windows 各种 bug 让你抓狂
|
 |
12
luyuncheng Jul 1, 2016
你不应该先学学 python 入门？
|
|
13
grey5659 OP @luyuncheng 只是当工具用 用
|
|
14
grey5659 OP @niboy @ksupertu
我换 linux 环境现在可以运行了，运行$ python doubanSpider.py 后一直在下载，是什么意思额？ /usr/local/lib/python2.7/dist-packages/bs4/__init__.py:166: UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html.parser"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently. To get rid of this warning, change this: BeautifulSoup([your markup]) to this: BeautifulSoup([your markup], "html.parser") markup_type=markup_type)) Downloading Information From Page 1 Downloading Information From Page 2 Downloading Information From Page 3 Downloading Information From Page 4 Downloading Information From Page 5 Downloading Information From Page 6 WARNING:root:Some characters could not be decoded, and were replaced with REPLACEMENT CHARACTER. Downloading Information From Page 7 Downloading Information From Page 8 Downloading Information From Page 9 Downloading Information From Page 10 Downloading Information From Page 11 Downloading Information From Page 12 Downloading Information From Page 13 Downloading Information From Page 14 Downloading Information From Page 15 Downloading Information From Page 16 Downloading Information From Page 17 Downloading Information From Page 18 Downloading Information From Page 19 Downloading Information From Page 20 Downloading Information From Page 21 Downloading Information From Page 22 Downloading Information From Page 23 Downloading Information From Page 24 |
|
15
ksupertu Jul 3, 2016 via Android
没什么大问题，就是 beautifulsoup 这个库报了个警告，因为没显式指定 html 解析器，爬虫已经在工作了
|
Select Language