D10——C语言基础学PYTHON
Posted m1racle
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了D10——C语言基础学PYTHON相关的知识，希望对你有一定的参考价值。
C语言基础学习PYTHON——基础学习D10
20180906内容纲要：
1、协程
（1）yield
（2）greenlet
（3）gevent
（4）gevent实现单线程下socket多并发
2、简单爬虫
3、select
4、IO多路复用
5、小结
6、练习
1 协程
协程
又叫微线程，纤程。协程是一种用户态的轻量级线程。
协程有自己的寄存器上下文和栈。协程调度切换时，将寄存器上下文、和栈保存到其他地方，在切换回来的时候回复先前保存的寄存器上下文和栈。
协程能保存上一次调用时的状态。一个cpu支持上万个协程都不是问题，很适合用于高并发处理。
协程的本质就是单线程。无法利用多核资源。
实现协程主要有以下几种方式：
（1）yield
1 #Author：ZhangKanghui 2 3 4 def Producer(): 5 r = con1.__next__() 6 r = con2.__next__() 7 n = 0 8 while n < 5: 9 n += 1 10 con1.send(n) 11 con2.send(n) 12 print("\\033[32;1m[Producer]\\033[0m is making baozi %s"%n) 13 14 def Consumer(name): 15 print("-->>start---") 16 while True: 17 new_balozi = yield 18 print("[%s] is eating baozi %s"%(name,new_balozi)) 19 #time.sleep(1) 20 21 if __name__ == \'__main__\': 22 con1 = Consumer("c1") #函数中存在yield只是生成器，并没有真正生成，所以需要necxt 23 con2 = Consumer("c2") 24 p = Producer()
通过这个例子协程到底是什么呢？
- 必须在只有一个单线程里实现并发
- 修改共享数据不需加锁
- 用户程序里自己保存多个控制流的上下文栈
- 一个协程遇到IO操作自动切换到其它协程
当然，还有其他实现方式。
（2）greenlet
greenlet是一个用C实现的协程模块。
1 #Author：ZhangKanghui 2 3 from greenlet import greenlet 4 def func1(): 5 print(12) 6 gr2.switch() 7 print(34) 8 gr2.switch() 9 10 def func2(): 11 print(56) 12 gr1.switch() 13 print(78) 14 15 gr1 = greenlet(func1) #启动协程 16 gr2 = greenlet(func2) 17 gr1.switch()
（3）gevent
Gevent 是一个第三方库，可以轻松通过gevent实现并发同步或异步编程。
在gevent中用到的主要模式是Greenlet, 它是以C扩展模块形式接入Python的轻量级协程。
Greenlet全部运行在主程序操作系统进程的内部，但它们被协作式地调度。
1 #Author：ZhangKanghui 2 3 import gevent 4 5 def Foo(): 6 print("Running in Foo") 7 gevent.sleep(2) 8 print("Explicit context switch to Foo again") 9 10 def bar(): 11 print("Explicit context to bar") 12 gevent.sleep(1) 13 print("Implicit context switch back to bar") 14 15 def func(): 16 print("running in func") 17 gevent.sleep(0) 18 print("running in func again") 19 gevent.joinall( 20 [gevent.spawn(Foo), 21 gevent.spawn(bar), 22 gevent.spawn(func) 23 ])
（4）gevent实现单线程下socket多并发
1 #Author：ZhangKanghui 2 3 import sys 4 import socket 5 import time 6 import gevent 7 from gevent import socket,monkey 8 9 monkey.patch_all() 10 11 def server(port): 12 s = socket.socket() 13 s.bind((\'localhost\',port)) 14 s.listen(500) 15 while True: 16 cli,addr = s.accept() 17 gevent.spawn(handle_request,cli) 18 19 def handle_request(conn): 20 try: 21 while True: 22 data = conn.recv(10214) 23 print("recv:",data) 24 conn.send(data) 25 if not data: 26 conn.shutdown(socket.SHUT_WR) 27 except Exception as e: 28 print(e) 29 finally: 30 conn.close() 31 32 if __name__ == \'__main__\': 33 server(8001)
1 #Author：ZhangKanghui 2 3 import socket 4 5 HOST = \'localhost\' 6 PORT = 8001 7 s = socket.socket(socket.AF_INET,socket.SOCK_STREAM) 8 s.connect((HOST,PORT)) 9 while True: 10 msg = bytes(input(">>:"),encoding="utf-8") 11 s.sendall(msg) 12 data = s.recv(1024) 13 14 #print("Received",repr(data)) 15 print("Received",data) 16 17 s.close()
2 简单爬虫
网络爬虫（又被称为网页蜘蛛，网络机器人，在FOAF社区中间，更经常被称为网页追逐者），是一种按照一定的规则，自动的抓取万维网信息的程序或者脚本，已被广泛应用于互联网领域。搜索引擎使用网络爬虫抓取Web网页、文档甚至图片、音频、视频等资源，通过相应的索引技术组织这些信息，提供给搜索用户进行查询。网络爬虫也为中小站点的推广提供了有效的途径。
那么我们就简单来看看以下爬虫吧：
1 #Author：ZhangKanghui 2 3 from urllib import request 4 5 def f(url): 6 print("GET:%s"%url) 7 resp = request.urlopen(url) 8 data = resp.read() 9 f = open("url.html",\'wb\') 10 f.write(data) 11 f.close() 12 print("%d bytes received from %s."%(len(data),url)) 13 14 f("https://www.cnblogs.com/zhangkanghui/p/9577714.html")
1 <!DOCTYPE html> 2 <html lang="zh-cn"> 3 <head> 4 <meta charset="utf-8"/> 5 <meta name="viewport" content="width=device-width, initial-scale=1" /> 6 <meta name="referrer" content="origin" /> 7 <title>D09——C语言基础学PYTHON - m1racle - 博客园</title> 8 <meta property="og:description" content="C语言基础学习PYTHON——基础学习D09 20180903内容纲要： 线程、进程 1、paramiko 2、线程、进程初识 3、线程 （1）线程的调用方式 （2）join （3）线程锁、递归锁、信" /> 9 <link type="text/css" rel="stylesheet" href="/bundles/blog-common.css?v=D7Le-lOZiZVAXQkZQuNwdTWqjabXaVBE_2YAWzY_YZs1"/> 10 <link id="MainCss" type="text/css" rel="stylesheet" href="/skins/red_autumnal_leaves/bundle-red_autumnal_leaves.css?v=EDEp2d1uMe8iyN6qDoW8MQgYb-JCFIeiYP0oX3XiiRM1"/> 11 <link id="mobile-style" media="only screen and (max-width: 767px)" type="text/css" rel="stylesheet" href="/skins/red_autumnal_leaves/bundle-red_autumnal_leaves-mobile.css?v=d9LctKHRIQp9rreugMcQ1-UJuq_j1fo0GZXTXj8Bqrk1"/> 12 <link title="RSS" type="application/rss+xml" rel="alternate" href="https://www.cnblogs.com/zhangkanghui/rss"/> 13 <link title="RSD" type="application/rsd+xml" rel="EditURI" href="https://www.cnblogs.com/zhangkanghui/rsd.xml"/> 14 <link type="application/wlwmanifest+xml" rel="wlwmanifest" href="https://www.cnblogs.com/zhangkanghui/wlwmanifest.xml"/> 15 <script src="//common.cnblogs.com/scripts/jquery-2.2.0.min.js"></script> 16 <script type="text/javascript">var currentBlogApp = \'zhangkanghui\', cb_enable_mathjax=false;var isLogined=false;</script> 17 <script src="/bundles/blog-common.js?v=yRkjgN2sBQkB4hX-wirHxPomeBT9sB5dawr6ob7KIvg1" type="text/javascript"></script> 18 </head> 19 <body> 20 <a name="top"></a> 21 22 <!--done--> 23 <div id="home"> 24 <div id="header"> 25 <div id="blogTitle"> 26 <a id="lnkBlogLogo" href="https://www.cnblogs.com/zhangkanghui/"><img id="blogLogo" src="/Skins/custom/images/logo.gif" alt="返回主页" /></a> 27 28 <!--done--> 29 <h1><a id="Header1_HeaderTitle" class="headermaintitle" href="https://www.cnblogs.com/zhangkanghui/">m1racle</a></h1> 30 <h2></h2> 31 32 33 34 35 </div><!--end: blogTitle 博客的标题和副标题 --> 36 <div id="navigator"> 37 38 <ul id="navList"> 39 <li><a id="blog_nav_sitehome" class="menu" href="https://www.cnblogs.com/">博客园</a></li> 40 <li><a id="blog_nav_myhome" class="menu" href="https://www.cnblogs.com/zhangkanghui/">首页</a></li> 41 <li><a id="blog_nav_newpost" class="menu" rel="nofollow" href="https://i.cnblogs.com/EditPosts.aspx?opt=1">新随笔</a></li> 42 <li><a id="blog_nav_contact" class="menu" rel="nofollow" href="https://msg.cnblogs.com/send/m1racle">联系</a></li> 43 <li><a id="blog_nav_rss" class="menu" href="https://www.cnblogs.com/zhangkanghui/rss">订阅</a> 44 <!--<a id="blog_nav_rss_image" class="aHeaderXML" href="https://www.cnblogs.com/zhangkanghui/rss"><img src="//www.cnblogs.com/images/xml.gif" alt="订阅" /></a>--></li> 45 <li><a id="blog_nav_admin" class="menu" rel="nofollow" href="https://i.cnblogs.com/">管理</a></li> 46 </ul> 47 <div class="blogStats"> 48 49 <div id="blog_stats"> 50 <span id="stats_post_count">随笔 - 8 </span> 51 <span id="stats_article_count">文章 - 0 </span> 52 <span id="stats-comment_count">评论 - 1</span> 53 </div> 54 55 </div><!--end: blogStats --> 56 </div><!--end: navigator 博客导航栏 --> 57 </div><!--end: header 头部 --> 58 59 <div id="main"> 60 <div id="mainContent"> 61 <div class="forFlow"> 62 63 <div id="post_detail"> 64 <!--done--> 65 <div id="topics"> 66 <div class = "post"> 67 <h1 class = "postTitle"> 68 <a id="cb_post_title_url" class="postTitle2" href="https://www.cnblogs.com/zhangkanghui/p/9577714.html">D09——C语言基础学PYTHON</a> 69 </h1> 70 <div class="clear"></div> 71 <div class="postBody"> 72 <div id="cnblogs_post_body" class="blogpost-body"><p><span style="font-size: 18pt; background-color: #ffff00"><strong>C语言基础学习PYTHON——基础学习D09</strong></span></p> 73 <blockquote> 74 <p><strong><span style="font-size: 18px">20180903内容纲要：</span></strong></p> 75 </blockquote> 76 <p> </p> 77 <p><strong><strong> 线程、进程</strong></strong></p> 78 <p> </p> 79 <p><strong><strong> 1、paramiko</strong></strong></p> 80 <p><strong><strong> 2、线程、进程初识</strong></strong></p> 81 <p><strong><strong> 3、线程</strong></strong></p> 82 <p><strong><strong> （1）线程的调用方式</strong></strong></p> 83 <p><strong><strong> （2）join</strong></strong></p> 84 <p><strong><strong> （3）线程锁、递归锁、信号量</strong></strong></p> 85 <p><strong> （4）Timer</strong></p> 86 <p><strong> （5）Event</strong></p> 87 <p><strong> （6）Queue队列</strong></p> 88 <p><strong> 4、小结</strong></p> 89 <p><strong> 5、练习：简单主机批量管理工具</strong></p> 90 <p><strong> </strong></p> 91 <blockquote> 92 <p><span style="font-size: 14pt"><strong>1、paramiko</strong></span></p> 93 </blockquote> 94 <p>paramiko模块提供了ssh及sft进行远程登录服务器执行命令和上传下载文件的功能。这是一个第三方的软件包，使用之前需要安装。</p> 95 <p>我个人觉得，这个在Windows上不太好用。在windows上python3需要安装vs2010，但我装了之后还是不行，可能是我自己的原因，在python2.7上好像可以，没试过。</p> 96 <p>这有链接：<a href="https://blog.csdn.net/songfreeman/article/details/50920767" target="_blank">https://blog.csdn.net/songfreeman/article/details/50920767<br></a></p> 97 <p>很详细！</p> 98 <p> </p> 99 <blockquote> 100 <p><span style="font-size: 14pt"><strong>2 线程、进程初识</strong></span></p> 101 102 103 104 105 106 107 </blockquote> 108 <h3>什么是线程(thread)？</h3> 109 <p>线程是操作系统能够进行运算调度的最小单位。它被包含在进程之中，是进程中的实际运作单位。</p> 110 <p>一条线程指的是进程中一个单一顺序的控制流，一个进程中可以并发多个线程，每条线程并行执行不同的任务</p> 111 <p>A thread is an execution context, which is all the information a CPU needs to execute a stream of instructions.</p> 112 <div class="cnblogs_code"><img id="code_img_closed_f52f3cbd-a865-4c8a-bd82-2732e6fb8022" class="code_img_closed" src="https://image.cha138.com/20210602/caa448d678704b34a52d499cd9c7b09f.jpg" alt=""><img id="code_img_opened_f52f3cbd-a865-4c8a-bd82-2732e6fb8022" class="code_img_opened" style="display: none" src="https://image.cha138.com/20210602/850d07614bbe4202b0f99956e4038021.jpg" alt=""> 113 <div id="cnblogs_code_open_f52f3cbd-a865-4c8a-bd82-2732e6fb8022" class="cnblogs_code_hide"> 114 <pre>Suppose you<span style="color: #800000">\'</span><span style="color: #800000">re reading a book, and you want to take a break right now, but you want to be able to come back and resume reading from the exact point where you stopped. One way to achieve that is by jotting down the page number, line number, and word number. So your execution context for reading a book is these 3 numbers.</span> 115 <span style="color: #000000"> 116 If you have a roommate, </span><span style="color: #0000ff">and</span> she<span style="color: #800000">\'</span><span style="color: #800000">s using the same technique, she can take the book while you</span><span style="color: #800000">\'</span>re <span style="color: #0000ff">not</span> using it, <span style="color: #0000ff">and</span> resume reading <span style="color: #0000ff">from</span> where she stopped. Then you can take it back, <span style="color: #0000ff">and</span> resume it <span style="color: #0000ff">from</span><span style="color: #000000"> where you were. 117 118 Threads work </span><span style="color: #0000ff">in</span> the same way. A CPU <span style="color: #0000ff">is</span> giving you the illusion that it<span style="color: #800000">\'</span><span style="color: #800000">s doing multiple computations at the same time. It does that by spending a bit of time on each computation. It can do that because it has an execution context for each computation. Just like you can share a book with your friend, many tasks can share a CPU.</span> 119 <span style="color: #000000"> 120 On a more technical level, an execution context (therefore a thread) consists of the values of the CPU</span><span style="color: #800000">\'</span><span style="color: #800000">s registers.</span> 121 <span style="color: #000000"> 122 Last: threads are different </span><span style="color: #0000ff">from</span> processes. A thread <span style="color: #0000ff">is</span> a context of execution, <span style="color: #0000ff">while</span> a process <span style="colo以上是关于D10——C语言基础学PYTHON的主要内容，如果未能解决你的问题，请参考以下文章