Python Socket编程基础篇

Posted Jason_wang_2016

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Python Socket编程基础篇相关的知识，希望对你有一定的参考价值。

Socket网络编程

socket通常也称作"套接字"，用于描述IP地址和端口，是一个通信链的句柄，应用程序通常通过"套接字"向网络发出请求或者应答网络请求。

socket起源于Unix，而Unix/Linux基本哲学之一就是“一切皆文件”，对于文件用打开、读写、关闭模式来操作。socket就是该模式的一个实现，socket即是一种特殊的文件，一些socket函数就是对其进行的操作（读/写IO、打开、关闭）

socket和file的区别：

file模块是针对某个指定文件进行打开、读写、关闭

socket模块是针对 服务器端 和 客户端Socket 进行打开、读写、关闭

Socket的英文原义是“孔”或“插座”。Socket正如其英文原意那样，像一个多孔插座。一台主机犹如布满各种插座的房间，每个插座有一个编号，有的插座提供220伏交流电， 有的提供110伏交流电，有的则提供有线电视节目。 客户软件将插头插到不同编号的插座，就可以得到不同的服务。

我知道上面那段话很枯燥，那就忘记吧！我们只需要记得：两个程序通过“网络”交互数据就使用socket，它只负责两件事：建立连接，传递数据。不能再简练了！！！

一个完整的socket通信流程大致如下图所示

实际socket服务器和和客户端通信的过程有些类似日常生活中拨打电话的过程，需要有手机，有电话卡，开机，拨号，建立连接，发送消息一直到通话结束，中间也有其他人打进电话的情况出现，

这个跟socket客户端连接服务器端，服务器端正在连接客户端时会将新进来的连接放到连接池中处于等待状态，下面看下代码怎么实现的：

下面来实现最简单的一对一拨打电话过程

Client端

建立连接，输入交互命令

Server端

socket的实现就像是打电话的过程，接收电话的是server端，拨打电话是client端，send和recv中的内容就是我们要发送和接受的东西，那么准备好了要发送的东西，我们要告诉手机打电话的目标电话，看server中的sock.bind方法，这里面传了一个元组(\'ip\',\'port\')给bind方法，就是把自己的电话开机处于接听状态；而client端的obj_client.connect方法是告诉自己手机终端，去找哪个号码建立通信。所以这个元组中的ip和port内容必须一致，是接听和拨打电话双方约定好的。我send的内容无所谓是什么，我send什么，客户端就recv什么，因为我发送的什么，接收到的就是什么，他没有权利

左右我发送什么东西。

#!/usr/bin/env python
# -*- coding:utf-8 -*-
#  Author: Jason Wang

import socket
import subprocess #导入执行命令模块
ip_port=(\'127.0.0.1\',9999) #定义元祖
#买手机
s=socket.socket()  #绑定协议，生成套接字
s.bind(ip_port)    #绑定ip+协议+端口：用来唯一标识一个进程，ip_port必须是元组格式
s.listen(5)        #定义最大可以挂起链接数
#等待电话
while True:  #用来重复接收新的链接
    conn,addr=s.accept()   #接收客户端胡的接请求，返回conn（相当于一个特定胡链接），addr是客户端ip+port
    conn.sendall(bytes(\'欢迎致电 10086，请输入1xxx,0转人工服务.\',encoding=\'utf-8\'))
    #收消息
    while True: #用来基于一个链接重复收发消息
            try: #捕捉客户端异常关闭（ctrl+c）
                recv_data=conn.recv(1024) #收消息，阻塞
                if len(recv_data) == 0:break #客户端如果退出，服务端将收到空消息，退出

                #发消息
                p=subprocess.Popen(str(recv_data,encoding=\'utf8\'),shell=True,stdout=subprocess.PIPE) #执行系统命令，windows平
                                                                                                      # 台命令的标准输出是gbk编码，需要转换
                res=p.stdout.read()   #获取标准输出
                if len(res) == 0:   #执行错误命令，标准输出为空，
                    send_data=\'cmd err\'
                # else:
                #     send_data=str(res,encoding=\'gbk\')  #命令执行ok，字节gbk---->str---->字节utf-8
                print(str)
                send_data=bytes(str(res,encoding=\'utf8\'),encoding=\'utf8\')


                #解决粘包问题
                ready_tag=\'Ready|%s\' %len(send_data)
                conn.send(bytes(ready_tag,encoding=\'utf8\')) #发送数据长度
                feedback=conn.recv(1024)  #接收确认信息
                feedback=str(feedback,encoding=\'utf8\')

                if feedback.startswith(\'Start\'):
                    conn.send(send_data)  #发送命令的执行结果
            except Exception:
                break
    #挂电话
    conn.close()
socket_server_demo

 

#!/usr/bin/env python
# -*- coding:utf-8 -*-
#  Author: Jason Wang

import socket
ip_port=(\'127.0.0.1\',9999)
#买手机
s=socket.socket()
#拨号
s.connect(ip_port)  #链接服务端，如果服务已经存在一个好的连接，那么挂起

welcom_msg = s.recv(200).decode()#获取服务端欢迎消息
print(welcom_msg)
while True:        #基于connect建立的连接来循环发送消息
    send_data=input(">>: ").strip()
    if send_data == \'exit\':break
    if len(send_data) == 0:continue
    s.send(bytes(send_data,encoding=\'utf8\'))

    #解决粘包问题
    ready_tag=s.recv(1024) #收取带数据长度的字节：Ready|9998
    ready_tag=str(ready_tag,encoding=\'utf8\')
    if ready_tag.startswith(\'Ready\'):#Ready|9998
        msg_size=int(ready_tag.split(\'|\')[-1])  #获取待接收数据长度
    start_tag=\'Start\'
    s.send(bytes(start_tag,encoding=\'utf8\')) #发送确认信息

    #基于已经收到的待接收数据长度，循环接收数据
    recv_size=0
    recv_msg=b\'\'
    while recv_size < msg_size:
        recv_data=s.recv(1024)
        recv_msg+=recv_data
        recv_size+=len(recv_data)
        print(\'MSG SIZE %s RECE SIZE %s\' %(msg_size,recv_size))

    print(str(recv_msg,encoding=\'utf8\'))
    #挂电话
s.close()
socket_client_demo

 

WEB服务应用：

#!/usr/bin/env python
#coding:utf-8
import socket
 
def handle_request(client):
    buf = client.recv(1024)
    client.send("HTTP/1.1 200 OK\\r\\n\\r\\n")
    client.send("Hello, World")
 
def main():
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.bind((\'localhost\',8080))
    sock.listen(5)
 
    while True:
        connection, address = sock.accept()
        handle_request(connection)
        connection.close()
 
if __name__ == \'__main__\':
  main()

更多功能

sk = socket.socket(socket.AF_INET,socket.SOCK_STREAM,0)

参数一：地址簇

　　socket.AF_INET IPv4（默认）
　　socket.AF_INET6 IPv6

　　socket.AF_UNIX 只能够用于单一的Unix系统进程间通信

参数二：类型

　　socket.SOCK_STREAM　　流式socket , for TCP （默认）
　　socket.SOCK_DGRAM　　 数据报式socket , for UDP

　　socket.SOCK_RAW 原始套接字，普通的套接字无法处理ICMP、IGMP等网络报文，而SOCK_RAW可以；其次，SOCK_RAW也可以处理特殊的IPv4报文；此外，利用原始套接字，可以通过IP_HDRINCL套接字选项由用户构造IP头。
　　socket.SOCK_RDM 是一种可靠的UDP形式，即保证交付数据报但不保证顺序。SOCK_RAM用来提供对原始协议的低级访问，在需要执行某些特殊操作时使用，如发送ICMP报文。SOCK_RAM通常仅限于高级用户或管理员运行的程序使用。
　　socket.SOCK_SEQPACKET 可靠的连续数据包服务

参数三：协议

　　0　　（默认）与特定的地址家族相关的协议,如果是 0 ，则系统就会根据地址格式和套接类别,自动选择一个合适的协议
import socket
ip_port = (\'127.0.0.1\',9999)
sk = socket.socket(socket.AF_INET,socket.SOCK_DGRAM,0)
sk.bind(ip_port)

while True:
    data = sk.recv(1024)
    print data




import socket
ip_port = (\'127.0.0.1\',9999)

sk = socket.socket(socket.AF_INET,socket.SOCK_DGRAM,0)
while True:
    inp = raw_input(\'数据：\').strip()
    if inp == \'exit\':
        break
    sk.sendto(inp,ip_port)

sk.close()
UDP

 

sk.bind(address)

　　s.bind(address) 将套接字绑定到地址。address地址的格式取决于地址族。在AF_INET下，以元组（host,port）的形式表示地址。

sk.listen(backlog)

　　开始监听传入连接。backlog指定在拒绝连接之前，可以挂起的最大连接数量。

      backlog等于5，表示内核已经接到了连接请求，但服务器还没有调用accept进行处理的连接个数最大为5
      这个值不能无限大，因为要在内核中维护连接队列

sk.setblocking(bool)

　　是否阻塞（默认True），如果设置False，那么accept和recv时一旦无数据，则报错。

sk.accept()

　　接受连接并返回（conn,address）,其中conn是新的套接字对象，可以用来接收和发送数据。address是连接客户端的地址。

　　接收TCP 客户的连接（阻塞式）等待连接的到来

sk.connect(address)

　　连接到address处的套接字。一般，address的格式为元组（hostname,port）,如果连接出错，返回socket.error错误。

sk.connect_ex(address)

　　同上，只不过会有返回值，连接成功时返回 0 ，连接失败时候返回编码，例如：10061

sk.close()

　　关闭套接字

sk.recv(bufsize[,flag])

　　接受套接字的数据。数据以字符串形式返回，bufsize指定最多可以接收的数量。flag提供有关消息的其他信息，通常可以忽略。

sk.recvfrom(bufsize[.flag])

　　与recv()类似，但返回值是（data,address）。其中data是包含接收数据的字符串，address是发送数据的套接字地址。

sk.send(string[,flag])

　　#Python 2将string中的数据发送到连接的套接字。返回值是要发送的字节数量，该数量可能小于string的字节大小。即：可能未将指定内容全部发送。

　　#Python 3必须转换为字节形式bytes发送

sk.sendall(string[,flag])

　　#将string中的数据发送到连接的套接字，但在返回之前会尝试发送所有数据。成功返回None，失败则抛出异常。

　　#Python 3必须转换为字节形式bytes发送

      内部通过递归调用send，将所有内容发送出去。

sk.sendto(string[,flag],address)

　　将数据发送到套接字，address是形式为（ipaddr，port）的元组，指定远程地址。返回值是发送的字节数。该函数主要用于UDP协议。

sk.settimeout(timeout)

　　设置套接字操作的超时期，timeout是一个浮点数，单位是秒。值为None表示没有超时期。一般，超时期应该在刚创建套接字时设置，因为它们可能用于连接的操作（如 client 连接最多等待5s ）

sk.getpeername()

　　返回连接套接字的远程地址。返回值通常是元组（ipaddr,port）。

sk.getsockname()

　　返回套接字自己的地址。通常是一个元组(ipaddr,port)

sk.fileno()

　　套接字的文件描述符

 

SocketServer模块

SocketServer内部使用 IO多路复用 以及 “多线程” 和 “多进程” ，从而实现并发处理多个客户端请求的Socket服务端。即：每个客户端请求连接到服务器时，Socket服务端都会在服务器是创建一个“线程”或者“进程” 专门负责处理当前客户端的所有请求。

ThreadingTCPServer

ThreadingTCPServer实现的Soket服务器内部会为每个client创建一个 “线程”，该线程用来和客户端进行交互。

1、ThreadingTCPServer基础

使用ThreadingTCPServer:

  • 创建一个继承自 SocketServer.BaseRequestHandler 的类
  • 类中必须定义一个名称为 handle 的方法
  • 启动ThreadingTCPServer
#!/usr/bin/env python
# -*- coding:utf-8 -*-
import SocketServer

class MyServer(SocketServer.BaseRequestHandler):

    def handle(self):
        # print self.request,self.client_address,self.server
        conn = self.request
        conn.sendall(\'欢迎致电 10086，请输入1xxx,0转人工服务.\')
        Flag = True
        while Flag:
            data = conn.recv(1024)
            if data == \'exit\':
                Flag = False
            elif data == \'0\':
                conn.sendall(\'通过可能会被录音.balabala一大推\')
            else:
                conn.sendall(\'请重新输入.\')


if __name__ == \'__main__\':
    server = SocketServer.ThreadingTCPServer((\'127.0.0.1\',8009),MyServer)
    server.serve_forever()
SocketServer实现服务器
#!/usr/bin/env python
# -*- coding:utf-8 -*-

import socket


ip_port = (\'127.0.0.1\',8009)
sk = socket.socket()
sk.connect(ip_port)
sk.settimeout(5)

while True:
    data = sk.recv(1024)
    print \'receive:\',data
    inp = raw_input(\'please input:\')
    sk.sendall(inp)
    if inp == \'exit\':
        break

sk.close()
客户端

2、ThreadingTCPServer源码剖析

内部调用流程为：

  • 启动服务端程序
  • 执行 TCPServer.__init__ 方法，创建服务端Socket对象并绑定 IP 和 端口
  • 执行 BaseServer.__init__ 方法，将自定义的继承自SocketServer.BaseRequestHandler 的类 MyRequestHandle赋值给self.RequestHandlerClass
  • 执行 BaseServer.server_forever 方法，While 循环一直监听是否有客户端请求到达 ...
  • 当客户端连接到达服务器
  • 执行 ThreadingMixIn.process_request 方法，创建一个 “线程” 用来处理请求
  • 执行 ThreadingMixIn.process_request_thread 方法
  • 执行 BaseServer.finish_request 方法，执行 self.RequestHandlerClass()  即：执行 自定义 MyRequestHandler 的构造方法（自动调用基类BaseRequestHandler的构造方法，在该构造方法中又会调用 MyRequestHandler的handle方法）

ThreadingTCPServer相关源码：

class BaseServer:

    """Base class for server classes.

    Methods for the caller:

    - __init__(server_address, RequestHandlerClass)
    - serve_forever(poll_interval=0.5)
    - shutdown()
    - handle_request()  # if you do not use serve_forever()
    - fileno() -> int   # for select()

    Methods that may be overridden:

    - server_bind()
    - server_activate()
    - get_request() -> request, client_address
    - handle_timeout()
    - verify_request(request, client_address)
    - server_close()
    - process_request(request, client_address)
    - shutdown_request(request)
    - close_request(request)
    - handle_error()

    Methods for derived classes:

    - finish_request(request, client_address)

    Class variables that may be overridden by derived classes or
    instances:

    - timeout
    - address_family
    - socket_type
    - allow_reuse_address

    Instance variables:

    - RequestHandlerClass
    - socket

    """

    timeout = None

    def __init__(self, server_address, RequestHandlerClass):
        """Constructor.  May be extended, do not override."""
        self.server_address = server_address
        self.RequestHandlerClass = RequestHandlerClass
        self.__is_shut_down = threading.Event()
        self.__shutdown_request = False

    def server_activate(self):
        """Called by constructor to activate the server.

        May be overridden.

        """
        pass

    def serve_forever(self, poll_interval=0.5):
        """Handle one request at a time until shutdown.

        Polls for shutdown every poll_interval seconds. Ignores
        self.timeout. If you need to do periodic tasks, do them in
        another thread.
        """
        self.__is_shut_down.clear()
        try:
            while not self.__shutdown_request:
                # XXX: Consider using another file descriptor or
                # connecting to the socket to wake this up instead of
                # polling. Polling reduces our responsiveness to a
                # shutdown request and wastes cpu at all other times.
                r, w, e = _eintr_retry(select.select, [self], [], [],
                                       poll_interval)
                if self in r:
                    self._handle_request_noblock()
        finally:
            self.__shutdown_request = False
            self.__is_shut_down.set()

    def shutdown(self):
        """Stops the serve_forever loop.

        Blocks until the loop has finished. This must be called while
        serve_forever() is running in another thread, or it will
        deadlock.
        """
        self.__shutdown_request = True
        self.__is_shut_down.wait()

    # The distinction between handling, getting, processing and
    # finishing a request is fairly arbitrary.  Remember:
    #
    # - handle_request() is the top-level call.  It calls
    #   select, get_request(), verify_request() and process_request()
    # - get_request() is different for stream or datagram sockets
    # - process_request() is the place that may fork a new process
    #   or create a new thread to finish the request
    # - finish_request() instantiates the request handler class;
    #   this constructor will handle the request all by itself

    def handle_request(self):
        """Handle one request, possibly blocking.

        Respects self.timeout.
        """
        # Support people who used socket.settimeout() to escape
        # handle_request before self.timeout was available.
        timeout = self.socket.gettimeout()
        if timeout is None:
            timeout = self.timeout
        elif self.timeout is not None:
            timeout = min(timeout, self.timeout)
        fd_sets = _eintr_retry(select.select, [self], [], [], timeout)
        if not fd_sets[0]:
            self.handle_timeout()
            return
        self._handle_request_noblock()

    def _handle_request_noblock(self):
        """Handle one request, without blocking.

        I assume that select.select has returned that the socket is
        readable before this function was called, so there should be
        no risk of blocking in get_request().
        """
        try:
            request, client_address = self.get_request()
        except socket.error:
            return
        if self.verify_request(request, client_address):
            try:
                self.process_request(request, client_address)
            except:
                self.handle_error(request, client_address)
                self.shutdown_request(request)

    def handle_timeout(self):
        """Called if no new request arrives within self.timeout.

        Overridden by ForkingMixIn.
        """
        pass

    def verify_request(self, request, client_address):
        """Verify the request.  May be overridden.

        Return True if we should proceed with this request.

        """
        return True

    def process_request(self, request, client_address):
        """Call finish_request.

        Overridden by ForkingMixIn and ThreadingMixIn.

        """
        self.finish_request(request, client_address)
        self.shutdown_request(request)

    def server_close(self):
        """Called to clean-up the server.

        May be overridden.

        """
        pass

    def finish_request(self, request, client_address):
        """Finish one request by instantiating RequestHandlerClass."""
        self.RequestHandlerClass(request, client_address, self)

    def shutdown_request(self, request):
        """Called to shutdown and close an individual request."""
        self.close_request(request)

    def close_request(self, request):
        """Called to clean up an individual request."""
        pass

    def handle_error(self, request, client_address):
        """Handle an error gracefully.  May be overridden.

        The default is to print a traceback and continue.

        """
        print \'-\'*40
        print \'Exception happened during processing of request from\',
        print client_address
        import traceback
        traceback.print_exc() # XXX But this goes to stderr!
        print \'-\'*40
BaseServer
class TCPServer(BaseServer):

    """Base class for various socket-based server classes.

    Defaults to synchronous IP stream (i.e., TCP).

    Methods for the caller:

    - __init__(server_address, RequestHandlerClass, bind_and_activate=True)
    - serve_forever(poll_interval=0.5)
    - shutdown()
    - handle_request()  # if you don\'t use serve_forever()
    - fileno() -> int   # for select()

    Methods that may be overridden:

    - server_bind()
    - server_activate()
    - get_request() -> request, client_address
    - handle_timeout()
    - verify_request(request, client_address)
    - process_request(request, client_address)
    - shutdown_request(request)
    - close_request(request)
    - handle_error()

    Methods for derived classes:

    - finish_request(request, client_address)

    Class variables that may be overridden by derived classes or
    instances:

    - timeout
    - address_family
    - socket_type
    - request_queue_size (only for stream sockets)
    - allow_reuse_address

    Instance variables:

    - server_address
    - RequestHandlerClass
    - socket

    """

    address_family = socket.AF_INET

    socket_type = socket.SOCK_STREAM

    request_queue_size = 5

    allow_reuse_address = False

    def __init__(self, server_address, RequestHandlerClass, bind_and_activate=True):
        """Constructor.  May be extended, do not override."""
        BaseServer.__init__(self, server_address, RequestHandlerClass)
        self.socket = socket.socket(self.address_family,
                                    self.socket_type)
        if bind_and_activate:
            try:
                self.server_bind()
                self.server_activate()
            except:
                self.server_close()
                raise

    def server_bind(self):
        """Called by constructor to bind the socket.

        May be overridden.

        """
        if self.allow_reuse_address:
            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.socket.bind(self.server_address)
        self.server_address = self.socket.getsockname()

    def server_activate(self):
        """Called by constructor to activate the server.

        May be overridden.

        """
        self.socket.listen(self.request_queue_size)

    def server_close(self):
        """Called to clean-up the server.

        May be overridden.

        """
        self.socket.close()

    def fileno(self):
        """Return socket file number.

        Interface required by select().

        """
        return self.socket.fileno()

    def get_request(self):
        """Get the request and client address from the socket.

        May be overridden.

        """
        return self.socket.accept()

    def shutdown_request(self, request):
        """Called to shutdown and close an individual request."""
        try:
            #explicitly shutdown.  socket.close() merely releases
            #the socket and waits for GC to perform the actual close.
            request.shutdown(socket.SHUT_WR)
        except socket.error:
            pass #some platforms may raise ENOTCONN here
        self.close_request(request)

    def close_request(self, request):
        """Called to clean up an individual request."""
        request.close()
TCPServer
class ThreadingMixIn:
    """Mix-in class to handle each request in a new thread."""

    # Decides how threads will act upon termination of the
    # main process
    daemon_threads = False

    def process_request_thread(self, request, client_address):
        """Same as in BaseServer but as a thread.

        In addition, exception handling is done here.

        """
        try:
            self.finish_request(request, client_address)
            self.shutdown_request(request)
        except:
            self.handle_error(request, client_address)
            self.shutdown_request(request)

    def process_request(self, request, client_address):
        """Start a new thread to process the request."""
        t = threading.Thread(target = self.process_request_thread,
                             args = (request, client_address))
        t.daemon = self.daemon_threads
        t.start()
ThreadingMixIn
class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass
ThreadingTCPServer

RequestHandler相关源码

lass BaseRequestHandler:

    """Base class for request handler classes.

    This class is instantiated for each request to be handled.  The
    constructor sets the instance variables request, client_address
    and server, and then calls the handle() method.  To implement a
    specific service, all you need to do is to derive a class which
    defines a handle() method.

    The handle() method can find the request as self.request, the
    client address as self.client_address, and the server (in case it
    needs access to per-server information) as self.server.  Since a
    separate instance is created for each request, the handle() method
    can define arbitrary other instance variariables.

    """

    def __init__(self, request, client_address, server):
        self.request = request
        self.client_address = client_address
        self.server = server
        self.setup()
        try:
            self.handle()
        finally:
            self.finish()

    def setup(self):
        pass

    def handle(self):
        pass

    def finish(self):
        pass
SocketServer.BaseRequestHandler

实例

#!/usr/bin/env python
# -*- coding:utf-8 -*-
#  Author: Jason Wang

import socketserver
import subprocess

class MyServer(socketserver.BaseRequestHandler):
    def handle(self):

        self.request.sendall(bytes(\'欢迎致电10086，请输入1xx,0转入人工服务\',encoding=\'utf-8\'))
        whilePython学习笔记——基础篇第七周———FTP作业（面向对象编程进阶 & Socket编程基础）


 python全栈开发基础第十八篇网络编程（socket）


 Python成长笔记 - 基础篇 


 Python基础：网络编程socketserver高级篇


 python 网络编程篇


 Java Socket编程基础篇