Python多线程


关于多线程
多线程主要用途就是提高程序和 CPU 的使用率,从而达到高效运行,看起来像同一时间能完成多件事情。
下面举个例子对比下
以爬取豆瓣电影top250为例:

不使用多线程时

if __name__ == '__main__':
    start_time = time.time()
    for i in range(0,10):
        main(i)
    end_time = time.time()
    dtime = end_time - start_time
    print("程序运行时间:%.8s s" % dtime)

看下结果

一共用时3.4s

使用多线程时

if __name__ == '__main__':
    start_time = time.time()
    page = [i for i in range(0,10)]
    with concurrent.futures.ThreadPoolExecutor(max_workers = 10) as executor:
        executor.map(main,page)
    end_time = time.time()
    dtime = end_time - start_time
    print("程序运行时间:%.8s s" % dtime)

结果:

耗时0.36s

对比

  • 单线程用时:3.45s
  • 多线程用时:0.36s

一件不幸的事
在测试完多线程效果之后::(滑稽)::(滑稽)::(滑稽)

对的,你没看错::(泪)::(泪)

以下是代码

import requests
from lxml import etree
import time
import concurrent.futures

headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"}

def request_douban(url):
    try:
        response = requests.get(url, headers = headers)
        html = response.text
        data = etree.HTML(html)
        movie_name = data.xpath('//*[@id="content"]/div/div[1]/ol/li/div/div[2]/div[1]/a/span[1]/text()')
        return movie_name
    except:
        return None

def main(page):
    url = "https://movie.douban.com/top250?start=" + str(page * 25) + "&filter="
    name_list = request_douban(url)
    print(name_list)
    for name in name_list:
        print(name, end='\n')


# if __name__ == '__main__':

# 使用多线程
#     start_time = time.time()
#     page = [i for i in range(0,10)]
#     with concurrent.futures.ThreadPoolExecutor(max_workers = 20) as executor:
#         executor.map(main,page)
#     end_time = time.time()
#     dtime = end_time - start_time
#     print("程序运行时间:%.8s s" % dtime)


# 不使用多线程
#     start_time = time.time()
#     for i in range(0,10):
#         main(i)
#     end_time = time.time()
#     dtime = end_time - start_time
#     print("程序运行时间:%.8s s" % dtime)
  文章目录
https://file.ztongyang.cn/yang/picttures/QQqr.jpg https://file.ztongyang.cn/yang/picttures/wechatqr.png https://metu.ztongyang.cn/a/avatar.jpg avatar

南玖

生命不息,折腾不止

  网站咨询
  •   当前在线1人
  •   加载耗时29 ms
  •   文章数目40篇
  •   分类总数10个
  •   评论总数58条
  •   站点字数3.84 W
  •   运行时间90天
  访问信息
  •   
  •   2020-10-23 14:02:20 Fri
  •   ?浏览器
  •   操作系统
  热门文章