2 QQ 32236581 Qq_322365812016.01.31 00:24 questions

Python crawler can not read the website code problem

Import urllib
Import urllib2
Url='Http://www.qiushibaike.com/hot/page/'+str (page)
User_agent ='Mozilla/4.0 (compatible; MSIE 5.5; Windows NT).
Headers={'User_Agent': user_agent}
Request=urllib2.Request (URL, headers=headers)
Response=urllib2.urlopen (request)
Print (response.read)
Except urllib2.URLError, e:
If hasattr (E, code):
Print e.code
If hasattr (E, reason):
Print e.reason

Use a proxy or can not read the page code
Raise BadStatusLine (line)

BadStatusLine: ''
This is an error

The 2 answer

Caozhy   Ds   Rxr 2016.01.31 07:00

Your user agent is very old, no additional cookie, referer and so on, your agent is a transparent proxy blocked your IP and so on are possible.

First use the browser to access, to ensure success with Fiddler capture, write your program control.

Oyljerry   Ds   Rxr 2016.01.31 09:26

This error is generally not to accept your request return directly to the server error

Csdn user default icon
Upload photo
Insert a picture