运维调试记录：Ubuntu14.04下部署Opendaylight Nitrogen集群
Posted KiteRunner24
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了运维调试记录：Ubuntu14.04下部署Opendaylight Nitrogen集群相关的知识，希望对你有一定的参考价值。
建议参考官网教程：Setting Up Clustering
一、实验环境
- 主机节点系统版本： Ubuntu 14.04 (64bit)
odl@mpodl:~$ uname -a
Linux mpodl 4.2.0-27-generic #32~14.04.1-Ubuntu SMP Fri Jan 22 15:32:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux- 主机节点硬件配置：单核CPU+4GB内存+50GB硬盘
odl@mpodl:~$ sudo lshw
description: Computer
product: Standard PC (i440FX + PIIX, 1996) ()
vendor: QEMU
version: pc-i440fx-trusty
width: 64 bits
capabilities: smbios-2.4 dmi-2.4 vsyscall32
configuration: boot=normal uuid=053D43B6-2E3C-CEA4-4C52-833DDD1749BE
*-core
description: Motherboard
physical id: 0
# CPU信息
*-cpu
description: CPU
product: QEMU Virtual CPU version 2.0.0
vendor: Intel Corp.
physical id: 401
bus info: cpu@0
slot: CPU 1
size: 2GHz
capacity: 2GHz
width: 64 bits
# 内存信息
*-memory
description: System Memory
physical id: 1000
size: 4GiB
# 硬盘信息
*-disk
description: ATA Disk
product: QEMU HARDDISK
physical id: 0.0.0
bus info: scsi@0:0.0.0
logical name: /dev/sda
version: 0
serial: QM00001
size: 50GiB (53GB)
*-volume:0
description: EXT4 volume
vendor: Linux
physical id: 1
bus info: scsi@0:0.0.0,1
logical name: /dev/sda1
logical name: /
version: 1.0
serial: 65d80188-ddfb-4f08-8018-a0c2e1da8af3
size: 46GiB
capacity: 46GiB
*-volume:1
description: Extended partition
physical id: 2
bus info: scsi@0:0.0.0,2
logical name: /dev/sda2
size: 4093MiB
capacity: 4093MiB- 集群环境：3台主机节点
Cluster_Node1: Ubuntu 14.04 -- [IP_Addr]=192.168.1.124
Cluster_Node2: Ubuntu 14.04 -- [IP_Addr]=192.168.1.125
Cluster_Node3: Ubuntu 14.04 -- [IP_Addr]=192.168.1.104所有主机节点都连接到同一台OpenvSwitch交换机上，且能互相PING通！！！
每台主机节点都安装配置有JDK 1.8，安装方法见：Ubuntu下通过PPA方式安装Java 8并自动配置环境变量
二、部署方法
1. 下载Opendaylight Nitrogen
在每台主机节点上执行如下命令，下载tar.gz格式的压缩包到用户目录下的ODL_N子目录：
odl@mpodl:~/ODL_N$ wget -P . https://nexus.opendaylight.org/content/repositories/public/org/opendaylight/integration/karaf/0.7.2/karaf-0.7.2.tar.gz或在官网下载，然后通过xftp软件从本地上传到服务器的Opendaylight运行环境中。
2. 安装Opendaylight Nitrogen
在每台主机节点上执行如下命令，解压压缩包到ODL_N子目录：
odl@mpodl:~/ODL_N$ tar -zxvf karaf-0.7.2.tar.gz解压完成，即认为Opendaylight Nitrogen安装成功。
3. 配置Opendaylight Nitrogen集群
在每台主机节点下执行如下命令，完成集群脚本配置工作：
命令格式：sudo bash ./karaf-0.7.2/bin/configure_cluster.sh [index] [seed_node_list]
其中，[index]为正整数，表示在[seed_node_list]中对应哪个主机节点上配置集群脚本。另外，[seed_node_list]是组成集群的各个主机节点对应的IP地址列表，以空格或逗号隔开。
因此，具体执行命令如下：
# Cluster_Node1: IP=192.168.1.124
odl@mpodl:~/ODL_N$ sudo bash ./karaf-0.7.2/bin/configure_cluster.sh 1 192.168.1.124 192.168.1.125 192.168.1.104
# Cluster_Node2: IP=192.168.1.125
odl@mpodl:~/ODL_N$ sudo bash ./karaf-0.7.2/bin/configure_cluster.sh 2 192.168.1.124 192.168.1.125 192.168.1.104
# Cluster_Node3: IP=192.168.1.104
odl@mpodl:~/ODL_N$ sudo bash ./karaf-0.7.2/bin/configure_cluster.sh 3 192.168.1.124 192.168.1.125 192.168.1.104当执行完之后，结果如下所示：
# Cluster_Node1: IP=192.168.1.124
odl@mpodl:~/ODL_N$ sudo bash ./karaf-0.7.2/bin/configure_cluster.sh 1 192.168.1.124 192.168.1.125 192.168.1.104
################################################
## Configure Cluster ##
################################################
Configuring unique name in akka.conf
Configuring hostname in akka.conf
Configuring data and rpc seed nodes in akka.conf
modules = [
name = "inventory"
namespace = "urn:opendaylight:inventory"
shard-strategy = "module"
,
name = "topology"
namespace = "urn:TBD:params:xml:ns:yang:network-topology"
shard-strategy = "module"
,
name = "toaster"
namespace = "http://netconfcentral.org/ns/toaster"
shard-strategy = "module"
]
Configuring replication type in module-shards.conf
################################################
## NOTE: Manually restart controller to ##
## apply configuration. ##
################################################备注：
（1）执行如上命令后，会在karaf-0.7.2/configuration目录下生成initial子目录，结果如下所示：
odl@mpodl:~/ODL_N$ ls ./karaf-0.7.2/configuration/
context.xml factory initial logback.xml tomcat-logging.properties tomcat-server.xml
odl@mpodl:~/ODL_N$ ls ./karaf-0.7.2/configuration/initial/
akka.conf modules.conf module-shards.conf可见，集群配置脚本在initial子目录下生成了akka.conf、modules.conf和module-shards.conf三个配置文件。
（2）查看第一台Ubuntu主机节点的akka.conf文件内容，具体如下所示：
odl@mpodl:~/ODL_N$ cat ./karaf-0.7.2/configuration/initial/akka.conf
odl-cluster-data
akka
remote
artery
enabled = off
canonical.hostname = "192.168.1.124" # 本机IP地址
canonical.port = 2550
netty.tcp
hostname = "192.168.1.124" # 本机IP地址
port = 2550
# when under load we might trip a false positive on the failure detector
# transport-failure-detector
# heartbeat-interval = 4 s
# acceptable-heartbeat-pause = 16s
#
cluster
# Remove ".tcp" when using artery.
# 集群节点列表
seed-nodes = ["akka.tcp://opendaylight-cluster-data@192.168.1.124:2550",
"akka.tcp://opendaylight-cluster-data@192.168.1.125:2550",
"akka.tcp://opendaylight-cluster-data@192.168.1.104:2550"]
roles = ["member-1"]
persistence
# By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
# modifying the following two properties. The directory location specified may be a relative or absolute path.
# The relative path is always relative to KARAF_HOME.
# snapshot-store.local.dir = "target/snapshots"
# journal.leveldb.dir = "target/journal"
journal
leveldb
# Set native = off to use a Java-only implementation of leveldb.
# Note that the Java-only version is not currently considered by Akka to be production quality.
# native = off
同样地，第二台和第三台Ubuntu主机节点的akka.conf文件内容相似，只是对应的IP地址不同。
（3）查看第一台Ubuntu主机节点的module-shards.conf文件内容，具体如下所示：
odl@mpodl:~/ODL_N$ cat ./karaf-0.7.2/configuration/initial/module-shards.conf
module-shards = [
name = "default"
shards = [
name = "default"
replicas = ["member-1",
"member-2",
"member-3"]
]
,
name = "inventory"
shards = [
name="inventory"
replicas = ["member-1",
"member-2",
"member-3"]
]
,
name = "topology"
shards = [
name="topology"
replicas = ["member-1",
"member-2",
"member-3"]
]
,
name = "toaster"
shards = [
name="toaster"
replicas = ["member-1",
"member-2",
"member-3"]
]
]同样地，第二台和第三台Ubuntu主机节点的module-shards.conf文件内容完全相同。
4. 启动Opendaylight Nitrogen集群
在每台主机节点下执行如下命令，完成Opendaylight节点启动工作：
odl@mpodl:~/ODL_N$ ./karaf-0.7.2/bin/karaf
karaf: JAVA_HOME not set; results may vary
Apache Karaf starting up. Press Enter to open the shell now...
100% [========================================================================]
Karaf started in 8s. Bundle stats: 208 active, 209 total
________ ________ .__ .__ .__ __
\\_____ \\ ______ ____ ____ \\______ \\ _____ ___.__.| | |__| ____ | |___/ |_
/ | \\\\____ \\_/ __ \\ / \\ | | \\\\__ \\< | || | | |/ ___\\| | \\ __\\
/ | \\ |_> > ___/| | \\| ` \\/ __ \\\\___ || |_| / /_/ > Y \\ |
\\_______ / __/ \\___ >___| /_______ (____ / ____||____/__\\___ /|___| /__|
\\/|__| \\/ \\/ \\/ \\/\\/ /_____/ \\/
Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown OpenDaylight.
opendaylight-user@root>然后，执行命令feature:list -i检查odl-mdsal-clustering是否处于已安装状态。如果没有安装，则执行命令feature:install odl-mdsal-clustering完成对应Feature的安装。
opendaylight-user@root>feature:list -i
Name | Version | Required | State | Repository | Description
----------------------------------------------------------------------------------------------------------------------------------------------------
odl-mdsal-broker | 1.6.2 | | Started | odl-mdsal-1.6.2 | odl-mdsal-broker
odl-mdsal-clustering | 1.6.2 | x | Started | odl-mdsal-clustering | odl-mdsal-clustering5. 检查Opendaylight Nitrogen集群是否建立
在每台主机节点下执行如下命令，获取主机节点的角色信息（Leader/Follower）：
opendaylight-user@root> ld | grep clustering于是，在主机节点一，输出如下：
2018-03-21 13:22:45,688 | INFO | d-dispatcher-125 | Shard | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-prefix-configuration-shard-config (Candidate): Starting new election term 21
2018-03-21 13:22:45,741 | INFO | d-dispatcher-125 | Shard | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-prefix-configuration-shard-config (Candidate) :- Switching from behavior Candidate to Leader, election term: 21
2018-03-21 13:22:45,742 | INFO | ult-dispatcher-5 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-1-shard-prefix-configuration-shard-config , received role change from Candidate to Leader
2018-03-21 13:22:45,821 | INFO | d-dispatcher-121 | Shard | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-prefix-configuration-shard-operational (Candidate): Starting new election term 21
2018-03-21 13:22:45,858 | INFO | d-dispatcher-125 | Shard | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-prefix-configuration-shard-operational (Candidate) :- Switching from behavior Candidate to Leader, election term: 21
2018-03-21 13:22:45,858 | INFO | ult-dispatcher-5 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-1-shard-prefix-configuration-shard-operational , received role change from Candidate to Leader
2018-03-21 13:22:45,872 | INFO | d-dispatcher-125 | EntityOwnershipShard | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-entity-ownership-operational (Candidate): Starting new election term 21
2018-03-21 13:22:45,912 | INFO | d-dispatcher-145 | EntityOwnershipShard | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-entity-ownership-operational (Candidate) :- Switching from behavior Candidate to Leader, election term: 21
2018-03-21 13:22:45,921 | INFO | lt-dispatcher-21 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-1-shard-entity-ownership-operational , received role change from Candidate to Leader在主机节点二，输出如下：
2018-03-21 13:22:36,286 | INFO | ult-dispatcher-4 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-2-shard-prefix-configuration-shard-config , received role change from null to Follower
2018-03-21 13:22:36,287 | INFO | ult-dispatcher-4 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-2-shard-prefix-configuration-shard-operational , received role change from null to Follower
2018-03-21 13:22:36,287 | INFO | ult-dispatcher-4 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-2-shard-prefix-configuration-shard-config , registered listener akka://opendaylight-cluster-data/user/shardmanager-config
2018-03-21 13:22:36,287 | INFO | ult-dispatcher-4 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-2-shard-prefix-configuration-shard-operational , registered listener akka://opendaylight-cluster-data/user/shardmanager-operational
2018-03-21 13:22:36,305 | INFO | rd-dispatcher-32 | EntityOwnershipShard | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | Starting recovery for member-2-shard-entity-ownership-operational with journal batch size 1
2018-03-21 13:22:36,313 | INFO | rd-dispatcher-38 | EntityOwnershipShard | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | Recovery completed - Switching actor to Follower - Persistence Id = member-2-shard-entity-ownership-operational Last index in log = -1, snapshotIndex = -1, snapshotTerm = -1, journal-size = 0
2018-03-21 13:22:36,317 | INFO | ult-dispatcher-2 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-2-shard-entity-ownership-operational , received role change from null to Follower在主机节点三，输出如下：
2018-03-21 13:22:39,418 | INFO | ult-dispatcher-6 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-3-shard-prefix-configuration-shard-operational , received role change from null to Follower
2018-03-21 13:22:39,418 | INFO | ult-dispatcher-6 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-3-shard-prefix-configuration-shard-config , received role change from null to Follower
2018-03-21 13:22:39,418 | INFO | ult-dispatcher-6 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-3-shard-prefix-configuration-shard-operational , registered listener akka://opendaylight-cluster-data/user/shardmanager-operational
2018-03-21 13:22:39,418 | INFO | ult-dispatcher-6 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-3-shard-prefix-configuration-shard-config , registered listener akka://opendaylight-cluster-data/user/shardmanager-config
2018-03-21 13:22:39,466 | INFO | rd-dispatcher-23 | EntityOwnershipShard | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | Starting recovery for member-3-shard-entity-ownership-operational with journal batch size 1
2018-03-21 13:22:39,470 | INFO | rd-dispatcher-23 | EntityOwnershipShard | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | Recovery completed - Switching actor to Follower - Persistence Id = member-3-shard-entity-ownership-operational Last index in log = -1, snapshotIndex = -1, snapshotTerm = -1, journal-size = 0
2018-03-21 13:22:39,473 | INFO | lt-dispatcher-31 | RoleChangeNotifier | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-3-shard-entity-ownership-operational , received role change from null to Follower可以看出，第一台Ubuntu主机节点成为Leader，其它两台Ubuntu主机节点成为 Follower ，集群配置成功。但是，从日志也可以看出，对于不同的Shard，存在不同的集群关系。
三、小结
本文详细介绍了Opendaylight Nitrogen集群的搭建指南，后续碰到问题将继续补充。
以上是关于运维调试记录：Ubuntu14.04下部署Opendaylight Nitrogen集群的主要内容，如果未能解决你的问题，请参考以下文章
使用openshit在ubuntu14.04下一键部署openstack(juno版本)
ubuntu14.04下安装cudnn5.1.3,opencv3.0,编译caffe及matlab和python接口过程记录