Python语言技术文档

微信小程序技术文档

php语言技术文档

jsp语言技术文档

asp语言技术文档

C#/.NET语言技术文档

html5/css技术文档

javascript

点击排行

您现在的位置:首页 > 技术文档 > Python网络爬虫

利用Python检测URL状态

来源:中文源码网    浏览:375 次    日期:2024-04-26 00:58:20
【下载文档:  利用Python检测URL状态.txt 】


利用Python检测URL状态
需求:Python检测URL状态,并追加保存200的URL
代码一:
#! /usr/bin/env python
#coding=utf-8
import sys
import requests
def getHttpStatusCode(url):
try:
request = requests.get(url)
httpStatusCode = request.status_code
return httpStatusCode
except requests.exceptions.HTTPError as e:
return e
if __name__ == "__main__":
with open('1.txt', 'r') as f:
for line in f:
try:
status = getHttpStatusCode(line.strip('\n'))#换行符
if status == 200:
with open('200.txt','a') as f:
f.write(line + '\n')
print line
else:
print 'no 200 code'
except Exception as e:
print e
代码二:
#! /usr/bin/env python
# -*--coding:utf-8*-
import requests
def request_status(line):
conn = requests.get(line)
if conn.status_code == 200:
with open('url_200.txt', 'a') as f:
f.write(line + '\n')
return line13 else:
return None
if __name__ == '__main__':
with open('/1.txt', 'rb') as f:
for line in f:
try:
purge_url = request_status(line.strip('\n'))
except Exception as e:
pass
代码三:
#! /usr/bin/env python
#coding:utf-8
import os,urllib,linecache
import sys
result = list()
for x in linecache.updatecache(r'1.txt'):
try:
a = urllib.urlopen(x.replace('/n','')).getcode()
#print x,a
except Exception,e:
print e
if a == 200:
#result.append(x) #保存
#result.sort() #排序结果
#open('2.txt', 'w').write('%s' % '\n'.join(result)) #保存入结果文件
with open ('200urllib.txt','a') as f: ## r只读,w可写,a追加
f.write(x + '\n')
else:
print 'error'
总结
以上所述是小编给大家介绍的python 检测url 状态,希望对大家有所帮助,如果大家有任何疑问请给我留言,小编会及时回复大家的。在此也非常感谢大家对中文源码网网站的支持!
如果你觉得本文对你有帮助,欢迎转载,烦请注明出处,谢谢!

相关内容