[Python]通过python-jenkins操作jenkins slave启动job | 通过python-jenkins实现ios自动化打包接口

Posted 2020-11-26 wangju003

 环境说明:

我在master-jenkins配置了2个节点,分别是mac_10.1.5.94和mac_10.1.71.51 

关于jenkins集群管理(节点管理),参照:[Jenkins]集群 节点管理| |分布式打包

 

python-jenkins官方文档

下面是几个python-jenkins提供的有关节点的方法, 代码示例 及 返回值:  

get_nodes(depth=0) 

Get a list of nodes connected to the Master 

Each node is a dict with keys ‘name’ and ‘offline’

 

Returns:	List of nodes, [ { str: str, str: bool} ]

代码示例: 

aa= server.get_nodes()

 返回值: 

[
    {
        "name": "master",
        "offline": false
    },
    {
        "name": "mac_10.1.5.94",
        "offline": false
    },
    {
        "name": "mac_10.1.71.51",
        "offline": false
    }
]

 get_node_info(name, depth=0) 

Get node information dictionary

 

Parameters:	name – Node name, str depth – JSON depth, int
Returns:	Dictionary of node info, dict

 

代码示例:

aa= server.get_node_info("mac_10.1.5.94")

返回值: 

{
    "_class": "hudson.slaves.SlaveComputer",
    "actions": [],
    "assignedLabels": [
        {
            "name": "mac_10.1.5.94"
        },
        {
            "name": "mac_imac_slave"
        }
    ],
    "description": "IMAC",
    "displayName": "mac_10.1.5.94",
    "executors": [
        {},
        {}
    ],
    "icon": "computer.png",
    "iconClassName": "icon-computer",
    "idle": true,
    "jnlpAgent": true,
    "launchSupported": false,
    "loadStatistics": {
        "_class": "hudson.model.Label$1"
    },
    "manualLaunchAllowed": true,
    "monitorData": {
        "hudson.node_monitors.SwapSpaceMonitor": {
            "_class": "hudson.node_monitors.SwapSpaceMonitor$MemoryUsage2",
            "availablePhysicalMemory": -1,
            "availableSwapSpace": 1010827264,
            "totalPhysicalMemory": -1,
            "totalSwapSpace": 1073741824
        },
        "hudson.node_monitors.TemporarySpaceMonitor": {
            "_class": "hudson.node_monitors.DiskSpaceMonitorDescriptor$DiskSpace",
            "timestamp": 1577348794553,
            "path": "/private/var/folders/m2/8_kj4c7s1jbdcc1brnnjs44c0000gn/T",
            "size": 79254544384
        },
        "hudson.node_monitors.DiskSpaceMonitor": {
            "_class": "hudson.node_monitors.DiskSpaceMonitorDescriptor$DiskSpace",
            "timestamp": 1577348794517,
            "path": "/Users/chenpeisong/agent_jenkins",
            "size": 79254544384
        },
        "hudson.node_monitors.ArchitectureMonitor": "Mac OS X (x86_64)",
        "hudson.node_monitors.ResponseTimeMonitor": {
            "_class": "hudson.node_monitors.ResponseTimeMonitor$Data",
            "timestamp": 1577348794517,
            "average": 376
        },
        "hudson.node_monitors.ClockMonitor": {
            "_class": "hudson.util.ClockDifference",
            "diff": -1
        }
    },
    "numExecutors": 2,
    "offline": false,
    "offlineCause": null,
    "offlineCauseReason": "",
    "oneOffExecutors": [],
    "temporarilyOffline": false,
    "absoluteRemotePath": "/Users/chenpeisong/agent_jenkins"
}

 这里注意一下这   "idle"(空闲)这个元素,当前slave机器 配置了同时构建2个job,只要slave已经在构建job了(无论正在构建1个还是2个),idle=flase

 只有当slave没有正在构建的job时,idle=true 

node_exists(name)
Check whether a node exists

 

Parameters:	name – Name of Jenkins node, str
Returns:	True if Jenkins node exists

 代码示例:

aa= server.node_exists("mac_10.1.5.94")

返回值: 

true

 assert_node_exists(name, exception_message=\'node[%s] does not exist\') 

Raise an exception if a node does not exist

 

Parameters:	name – Name of Jenkins node, str exception_message – Message to use for the exception. Formatted with name
Throws:	JenkinsException whenever the node does not exist

 代码示例:

aa= server.assert_node_exists("mac_10.1.5.1",exception_message=\'node[%s] does not exist\')
print(json.dumps(aa,ensure_ascii=False,indent=4))

当节点不存在时,报出指定的错误,如果节点存在,则返回null

返回值: 

Traceback (most recent call last):
  File "/home/wangju/PycharmProjects/wjTest/jenkinsTest/start_slave_job.py", line 16, in <module>
    aa= server.assert_node_exists("mac_10.1.5.1",exception_message=\'node[%s] does not exist\')
  File "/home/wangju/.virtualenvs/wjTest-c3XANhw-/lib/python3.6/site-packages/jenkins/__init__.py", line 1539, in assert_node_exists
    raise JenkinsException(exception_message % name)
jenkins.JenkinsException: node[mac_10.1.5.1] does not exist

 get_node_config(name) 

Get the configuration for a node. 

Parameters:	name – Jenkins node name, str

代码示例:

aa= server.get_node_config("mac_10.1.5.94")

返回值:
是一个xml格式的配置文件 

"<?xml version=\\"1.1\\" encoding=\\"UTF-8\\"?>\\n<slave>\\n  <name>mac_10.1.5.94</name>\\n  <description>IMAC</description>\\n  <remoteFS>/Users/chenpeisong/agent_jenkins</remoteFS>\\n  <numExecutors>2</numExecutors>\\n  <mode>EXCLUSIVE</mode>\\n  <retentionStrategy class=\\"hudson.slaves.RetentionStrategy$Always\\"/>\\n  <launcher class=\\"hudson.slaves.JNLPLauncher\\">\\n    <workDirSettings>\\n      <disabled>false</disabled>\\n      <internalDir>remoting</internalDir>\\n      <failIfWorkDirIsMissing>false</failIfWorkDirIsMissing>\\n    </workDirSettings>\\n  </launcher>\\n  <label>mac_imac_slave</label>\\n  <nodeProperties>\\n    <hudson.slaves.EnvironmentVariablesNodeProperty>\\n      <envVars serialization=\\"custom\\">\\n        <unserializable-parents/>\\n        <tree-map>\\n          <default>\\n            <comparator class=\\"hudson.util.CaseInsensitiveComparator\\"/>\\n          </default>\\n          <int>2</int>\\n          <string></string>\\n          <string></string>\\n          <string>PATH</string>\\n          <string>/Library/Java/JavaVirtualMachines/jdk1.8.0_211.jdk/Contents/Home/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/rvm/bin:.:/Users/chenpeisong/Library/Android/sdk/platform-tools:/Users/chenpeisong/Library/Android/sdk/tools</string>\\n        </tree-map>\\n      </envVars>\\n    </hudson.slaves.EnvironmentVariablesNodeProperty>\\n  </nodeProperties>\\n</slave>"

 

我的应用: 

我的需求:实现ios分布式打包.因ios打包时间较长,且为了不影响打包性能,jenkins最多能支持构建2个job,这样在发版周,就需要经常排队打包.

为了提升打包的效率,现在使用2台mac打包.

为了更减轻测试打包的负担,现在为自动化打包做了接口支持,如何通过接口实现选择不同的slave打包呢?这就是我需要解决的问题.

我的配置:

slave1:

node_name:mac_10.1.71.51

配置在该slave的job名:ios_official_51

 

slave2:

node_name:mac_10.1.5.94

配置在该slave的job名:ios_official_94

slave1的配置高于slave2,所以我的策略是:只有当slave1有构建任务时,才使用slave2打包

 

无论要构建的job是不是位于slave,通过接口操作jenkins构建job,分为3步:

1.获取打包job名称

2.启动打包,并获得build_number

3.根据build_number获取打包结果

 

我的思考:

如果要启动的job位于slave,与启动普通的job相比,只是会增加1个步骤,判断slave是否在线.

如果slave都在线,选择哪个slave? 如果slave有的在线,有的不在线,如何选择?解决了这个问题,其它的操作步骤与构建普通的job一样.

我使用的策略: 

因为目前我一共配置了2台slave,所以

分3种情况:

• 2台slave都在线 返回True
• 2台slave都不在线 返回False
• 1台salve在线,1台slave不在线 :返回在线的salve名称

这个函数中的核心是使用python-jenkins提供的方法server.get_nodes()获取salve机器的在线状态:

如果slave是离线状态,则offline的值为true

 

代码实现: 

首先创建jenkins server实例: 

创建jenkins实例(先通过pip下载python-jenkins): 

#导入依赖
import jenkins
import time
import json #从文件中读取json格式的测试结果

#定义远程的jenkins master server 的url以及Port
jenkins_server_url = \'http://10.2.1.92:8080/jenkins/\'
#定义用户的userid 和 apitoken(在jenkins中生成)
user_id = \'admin\'
api_token = \'11d8c79994b1e6d554c857b1d96fcf4dfe\' #测试服务器

#实例化jenkins对象，连接远程的jenkins master server
server = jenkins.Jenkins(jenkins_server_url,username=user_id,password=api_token)


aa= server.get_nodes()

print(json.dumps(aa,ensure_ascii=False,indent=4))
 

 

 如果只有1个slave在线,返回其名称,都在线true,都不在线flase

def get_node_offline_state_or_line_node_name(server):
    \'\'\'
    获取node在线状态
    :param:jenkins_server
    :return: 都在线 true,都不在线 flase,1个在线:在线node名称
    \'\'\'
    node_offline_list = server.get_nodes()[1:]
    node_imac_offline_state = node_offline_list[0][\'offline\']
    node_mini_offline_state = node_offline_list[1][\'offline\']

    if node_mini_offline_state and node_imac_offline_state:
        # 都不在线
        return False
    elif not node_mini_offline_state and not node_imac_offline_state:
        # 都在线
        return True
    elif not node_mini_offline_state or not node_imac_offline_state:
        # 1台机器在线,返回在线的node名
        if not node_mini_offline_state == True:
            return \'mac_10.1.71.51\'
        else:
            return \'mac_10.1.5.94\'

根据在线的slave,选择要构建的job名字

下面的函数中使用到了python-jenkins提供的方法get_node_info(node_name),它的返回值中有一个idle字段,可以判断slave是否有正在构建的job

注意一下:只要slave有1个job是正在构建的状态,idle的状态就是false

 node_offline_state_or_line_node_name = get_node_offline_state_or_line_node_name(server)
        if not node_offline_state_or_line_node_name:
            # 都不在线
            job_name = None
            code = 204
            msg = \'slave离线,请联系管理员!\'
        elif node_offline_state_or_line_node_name is True:
            # 都在线
            mini_idle_state = server.get_node_info(\'mac_10.1.71.51\')[\'idle\']
            imac_idle_state = server.get_node_info(\'mac_10.1.5.94\')[\'idle\']
            if not mini_idle_state and imac_idle_state:
                # 只有mini不空闲时,选择imac打包
                job_name = \'ios_official_94\'
            else:
                # mini 空闲,imace空闲,选择nimi打包
                # mini 和 imac都不空闲,选择mini打包
                job_name = \'ios_official_51\'
        elif node_offline_state_or_line_node_name == \'mac_10.1.71.51\':
            # nini在线
            # return \'mac_10.1.71.51\'
            job_name = \'ios_official_51\'
        elif node_offline_state_or_line_node_name == \'mac_10.1.5.94\':
            # imac在线
            # return \'mac_10.1.5.94\'
            job_name = \'ios_official_94\'