如何将第三个节点添加到 2 节点 Cassandra 集群?

Posted

技术标签:

【中文标题】如何将第三个节点添加到 2 节点 Cassandra 集群?【英文标题】:How do I add a third node to a 2-node Cassandra Cluster? 【发布时间】:2021-06-21 10:38:42 【问题描述】:

我正在使用 Cassandra v3.9。我更新了配置文件,如下所示,但 25.80.44.50 不会加入集群(它与 25.80.44.51 一起正式成为 2 节点集群的一部分)。

当我恢复到 2 节点集群时,数据库已损坏,我必须从备份中恢复。

节点 1

配置:

[root@program-node01-vm conf]# egrep -v "^$|^#|^ *#" /var/lib/cassandra/conf/cassandra.yaml
cluster_name: 'My Cluster'
num_tokens: 256
hinted_handoff_enabled: true
max_hint_window_in_ms: 10800000 # 3 hours
hinted_handoff_throttle_in_kb: 1024
max_hints_delivery_threads: 2
hints_flush_period_in_ms: 10000
max_hints_file_size_in_mb: 128
batchlog_replay_throttle_in_kb: 1024
authenticator: AllowAllAuthenticator
authorizer: AllowAllAuthorizer
role_manager: CassandraRoleManager
roles_validity_in_ms: 2000
permissions_validity_in_ms: 2000
credentials_validity_in_ms: 2000
partitioner: org.apache.cassandra.dht.Murmur3Partitioner
data_file_directories:
  - /var/lib/cassandra/data
commitlog_directory: /var/lib/cassandra/commitlog
cdc_enabled: false
disk_failure_policy: stop
commit_failure_policy: stop
prepared_statements_cache_size_mb:
thrift_prepared_statements_cache_size_mb:
key_cache_size_in_mb:
key_cache_save_period: 14400
row_cache_size_in_mb: 0
row_cache_save_period: 0
counter_cache_size_in_mb:
counter_cache_save_period: 7200
commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000
commitlog_segment_size_in_mb: 32
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          - seeds: "25.80.44.51,25.80.44.52"
concurrent_reads: 32
concurrent_writes: 32
concurrent_counter_writes: 32
concurrent_materialized_view_writes: 32
memtable_allocation_type: heap_buffers
index_summary_capacity_in_mb:
index_summary_resize_interval_in_minutes: 60
trickle_fsync: false
trickle_fsync_interval_in_kb: 10240
storage_port: 7000
ssl_storage_port: 7001
listen_address: 25.80.44.50
start_native_transport: true
native_transport_port: 9042
start_rpc: false
rpc_address: 25.80.44.50
rpc_port: 9160
rpc_keepalive: true
rpc_server_type: sync
thrift_framed_transport_size_in_mb: 15
incremental_backups: false
snapshot_before_compaction: false
auto_snapshot: true
column_index_size_in_kb: 64
column_index_cache_size_in_kb: 2
compaction_throughput_mb_per_sec: 16
sstable_preemptive_open_interval_in_mb: 50
read_request_timeout_in_ms: 5000
range_request_timeout_in_ms: 10000
write_request_timeout_in_ms: 2000
counter_write_request_timeout_in_ms: 5000
cas_contention_timeout_in_ms: 1000
truncate_request_timeout_in_ms: 60000
request_timeout_in_ms: 10000
cross_node_timeout: false
endpoint_snitch: SimpleSnitch
dynamic_snitch_update_interval_in_ms: 100
dynamic_snitch_reset_interval_in_ms: 600000
dynamic_snitch_badness_threshold: 0.1
request_scheduler: org.apache.cassandra.scheduler.NoScheduler
server_encryption_options:
    internode_encryption: none
    require_client_auth: false
    require_endpoint_verification: false
client_encryption_options:
    enabled: false
    require_client_auth: false
internode_compression: none
inter_dc_tcp_nodelay: false
tracetype_query_ttl: 86400
tracetype_repair_ttl: 604800
enable_user_defined_functions: false
enable_scripted_user_defined_functions: false
windows_timer_interval: 1
transparent_data_encryption_options:
    enabled: false
    chunk_length_kb: 64
    cipher: AES/CBC/PKCS5Padding
    key_alias: testing:1
    key_provider:
      - class_name: org.apache.cassandra.security.JKSKeyProvider
        parameters:
          - keystore: conf/.keystore
            keystore_password: cassandra
            store_type: JCEKS
            key_password: cassandra
tombstone_warn_threshold: 1000
tombstone_failure_threshold: 100000
batch_size_warn_threshold_in_kb: 5
batch_size_fail_threshold_in_kb: 50
unlogged_batch_across_partitions_warn_threshold: 10
compaction_large_partition_warning_threshold_mb: 100
gc_warn_threshold_in_ms: 1000

LSOF:

[root@program-node01 conf]# lsof -P -n -p 13328 | grep -i tcp
java    13328 cassandra   93u     IPv4            7854361       0t0        TCP 127.0.0.1:46464 (LISTEN)
java    13328 cassandra   94u     IPv4            7854362       0t0        TCP *:7199 (LISTEN)
java    13328 cassandra   97u     IPv4            7803257       0t0        TCP 25.80.44.50:7000->25.80.44.51:53828 (ESTABLISHED)
java    13328 cassandra   99u     IPv4            7866406       0t0        TCP 25.80.44.50:37347->25.80.44.51:7000 (ESTABLISHED)
java    13328 cassandra  101u     IPv4            7828918       0t0        TCP 25.80.44.50:51688->25.80.44.52:7000 (ESTABLISHED)
java    13328 cassandra  104u     IPv4            7866413       0t0        TCP 25.80.44.50:7000->25.80.44.51:41813 (ESTABLISHED)
java    13328 cassandra  105u     IPv4            7752003       0t0        TCP 25.80.44.50:7000->25.80.44.52:50307 (ESTABLISHED)
java    13328 cassandra  113u     IPv4            7869505       0t0        TCP 25.80.44.50:34373->25.80.44.52:7000 (ESTABLISHED)
java    13328 cassandra  194u     IPv4            7826676       0t0        TCP 25.80.44.50:60348->25.80.44.51:7000 (ESTABLISHED)
java    13328 cassandra  220u     IPv4            7803243       0t0        TCP 25.80.44.50:7000 (LISTEN)
java    13328 cassandra  336u     IPv4            7803259       0t0        TCP 25.80.44.50:9042 (LISTEN)
java    13328 cassandra  337u     IPv4            7752004       0t0        TCP 25.80.44.50:7000->25.80.44.50:37007 (ESTABLISHED)
java    13328 cassandra  340u     IPv4            7884819       0t0        TCP 25.80.44.50:37007->25.80.44.50:7000 (ESTABLISHED)

这证明该节点 (25.80.44.50) 已连接(通过 tcp)到其他两个节点(25.80.44.51 和 25.80.44.52)。但是,我没有看到类似于 7000 的端口 9042 的连接。我认为 9042 是节点-节点通信端口。

网络统计

[root@program-node01 conf]# netstat -anp | grep 7000
tcp        0      0 25.80.44.50:7000        0.0.0.0:*               LISTEN      13328/java
tcp        0      0 25.80.44.50:7000        25.80.44.51:53828       ESTABLISHED 13328/java
tcp        0      0 25.80.44.50:7000        25.80.44.50:37007       ESTABLISHED 13328/java
tcp        0      0 25.80.44.50:51688       25.80.44.52:7000        ESTABLISHED 13328/java
tcp        0      0 25.80.44.50:7000        25.80.44.52:50307       ESTABLISHED 13328/java
tcp        0      0 25.80.44.50:37007       25.80.44.50:7000        ESTABLISHED 13328/java
tcp        0      0 25.80.44.50:7000        25.80.44.51:41813       ESTABLISHED 13328/java
tcp        0      0 25.80.44.50:34373       25.80.44.52:7000        ESTABLISHED 13328/java
tcp        0      0 25.80.44.50:37347       25.80.44.51:7000        ESTABLISHED 13328/java
tcp        0      0 25.80.44.50:60348       25.80.44.51:7000        ESTABLISHED 13328/java
[root@program-node01 conf]# netstat -anp | grep 9042
tcp        0      0 25.80.44.50:9042        0.0.0.0:*               LISTEN      13328/java
[root@program-node01 conf]#

节点工具信息:

[root@program-node01 conf]# /var/lib/cassandra/bin/nodetool info
ID                     : 5db50d6d-3eae-4a47-b7c9-4743cd80ec99
Gossip active          : true
Thrift active          : false
Native Transport active: true
Load                   : 50.69 MiB
Generation No          : 1616531552
Uptime (seconds)       : 5663
Heap Memory (MB)       : 80.06 / 7987.25
Off Heap Memory (MB)   : 0.07
Data Center            : datacenter1
Rack                   : rack1
Exceptions             : 0
Key Cache              : entries 132, size 24.41 KiB, capacity 100 MiB, 94 hits, 161 requests, 0.584 recent hit rate, 14400 save period in seconds
Row Cache              : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 requests, NaN recent hit rate, 0 save period in seconds
Counter Cache          : entries 10, size 1.17 KiB, capacity 50 MiB, 0 hits, 0 requests, NaN recent hit rate, 7200 save period in seconds
Chunk Cache            : entries 14, size 896 KiB, capacity 480 MiB, 70 misses, 267 requests, 0.738 recent hit rate, NaN microseconds miss latency
Percent Repaired       : 91.24266615492786%
Token                  : (node is not joined to the cluster)
[root@program-node01 conf]#

这就是我认为的问题所在!Token:(node is not joined to the cluster)

节点 2

配置:

[root@program-node02-vm conf]# egrep -v "^$|^#|^ *#" /var/lib/cassandra/conf/cassandra.yaml
cluster_name: 'My Cluster'
num_tokens: 256
hinted_handoff_enabled: true
max_hint_window_in_ms: 10800000 # 3 hours
hinted_handoff_throttle_in_kb: 1024
max_hints_delivery_threads: 2
hints_flush_period_in_ms: 10000
max_hints_file_size_in_mb: 128
batchlog_replay_throttle_in_kb: 1024
authenticator: AllowAllAuthenticator
authorizer: AllowAllAuthorizer
role_manager: CassandraRoleManager
roles_validity_in_ms: 2000
permissions_validity_in_ms: 2000
credentials_validity_in_ms: 2000
partitioner: org.apache.cassandra.dht.Murmur3Partitioner
data_file_directories:
  - /var/lib/cassandra/data
commitlog_directory: /var/lib/cassandra/commitlog
cdc_enabled: false
disk_failure_policy: stop
commit_failure_policy: stop
prepared_statements_cache_size_mb:
thrift_prepared_statements_cache_size_mb:
key_cache_size_in_mb:
key_cache_save_period: 14400
row_cache_size_in_mb: 0
row_cache_save_period: 0
counter_cache_size_in_mb:
counter_cache_save_period: 7200
commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000
commitlog_segment_size_in_mb: 32
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          - seeds: "25.80.44.50,25.80.44.52"
concurrent_reads: 32
concurrent_writes: 32
concurrent_counter_writes: 32
concurrent_materialized_view_writes: 32
memtable_allocation_type: heap_buffers
index_summary_capacity_in_mb:
index_summary_resize_interval_in_minutes: 60
trickle_fsync: false
trickle_fsync_interval_in_kb: 10240
storage_port: 7000
ssl_storage_port: 7001
listen_address: 25.80.44.51
start_native_transport: true
native_transport_port: 9042
start_rpc: false
rpc_address: 25.80.44.51
rpc_port: 9160
rpc_keepalive: true
rpc_server_type: sync
thrift_framed_transport_size_in_mb: 15
incremental_backups: false
snapshot_before_compaction: false
auto_snapshot: true
column_index_size_in_kb: 64
column_index_cache_size_in_kb: 2
compaction_throughput_mb_per_sec: 16
sstable_preemptive_open_interval_in_mb: 50
read_request_timeout_in_ms: 5000
range_request_timeout_in_ms: 10000
write_request_timeout_in_ms: 2000
counter_write_request_timeout_in_ms: 5000
cas_contention_timeout_in_ms: 1000
truncate_request_timeout_in_ms: 60000
request_timeout_in_ms: 10000
cross_node_timeout: false
endpoint_snitch: SimpleSnitch
dynamic_snitch_update_interval_in_ms: 100
dynamic_snitch_reset_interval_in_ms: 600000
dynamic_snitch_badness_threshold: 0.1
request_scheduler: org.apache.cassandra.scheduler.NoScheduler
server_encryption_options:
    internode_encryption: none
    require_client_auth: false
    require_endpoint_verification: false
client_encryption_options:
    enabled: false
    require_client_auth: false
internode_compression: none
inter_dc_tcp_nodelay: false
tracetype_query_ttl: 86400
tracetype_repair_ttl: 604800
enable_user_defined_functions: false
enable_scripted_user_defined_functions: false
windows_timer_interval: 1
transparent_data_encryption_options:
    enabled: false
    chunk_length_kb: 64
    cipher: AES/CBC/PKCS5Padding
    key_alias: testing:1
    key_provider:
      - class_name: org.apache.cassandra.security.JKSKeyProvider
        parameters:
          - keystore: conf/.keystore
            keystore_password: cassandra
            store_type: JCEKS
            key_password: cassandra
tombstone_warn_threshold: 1000
tombstone_failure_threshold: 100000
batch_size_warn_threshold_in_kb: 5
batch_size_fail_threshold_in_kb: 50
unlogged_batch_across_partitions_warn_threshold: 10
compaction_large_partition_warning_threshold_mb: 100
gc_warn_threshold_in_ms: 1000

LSOF:

[root@program-node02 conf]# lsof -P -n -p 12100 | grep -i tcp
java    12100 cassandra   72u     IPv4            7457886       0t0        TCP 25.80.44.51:7000->25.80.44.52:39067 (ESTABLISHED)
java    12100 cassandra   73u     IPv4            7441813       0t0        TCP 25.80.44.51:55223->25.80.44.52:7000 (ESTABLISHED)
java    12100 cassandra   76u     IPv4            7452718       0t0        TCP 25.80.44.51:41813->25.80.44.50:7000 (ESTABLISHED)
java    12100 cassandra   89u     IPv4            7475374       0t0        TCP 127.0.0.1:39020 (LISTEN)
java    12100 cassandra   90u     IPv4            7475375       0t0        TCP *:7199 (LISTEN)
java    12100 cassandra   92u     IPv4            7429183       0t0        TCP 25.80.44.51:44473->25.80.44.52:7000 (ESTABLISHED)
java    12100 cassandra   93u     IPv4            6972054       0t0        TCP 25.80.44.51:33449->25.80.44.52:7000 (ESTABLISHED)
java    12100 cassandra   94u     IPv4            7451798       0t0        TCP 25.80.44.51:7000->25.80.44.50:60348 (ESTABLISHED)
java    12100 cassandra   97u     IPv4            7441814       0t0        TCP 25.80.44.51:7000->25.80.44.51:33562 (ESTABLISHED)
java    12100 cassandra  235u     IPv4            7464346       0t0        TCP 25.80.44.51:7000 (LISTEN)
java    12100 cassandra  236u     IPv4            7291404       0t0        TCP 25.80.44.51:7000->25.80.44.50:37347 (ESTABLISHED)
java    12100 cassandra  237u     IPv4            7451793       0t0        TCP 25.80.44.51:7000->25.80.44.52:59547 (ESTABLISHED)
java    12100 cassandra  325u     IPv4            7429185       0t0        TCP 25.80.44.51:9042 (LISTEN)
java    12100 cassandra  326u     IPv4            6972044       0t0        TCP 25.80.44.51:53828->25.80.44.50:7000 (ESTABLISHED)
java    12100 cassandra  329u     IPv4            7291409       0t0        TCP 25.80.44.51:33562->25.80.44.51:7000 (ESTABLISHED)

网络统计:

[root@program-node02 conf]# netstat -anp | grep 7000
tcp        0      0 25.80.44.51:7000        0.0.0.0:*               LISTEN      12100/java
tcp        0      0 25.80.44.51:33562       25.80.44.51:7000        ESTABLISHED 12100/java
tcp        0      0 25.80.44.51:7000        25.80.44.50:37347       ESTABLISHED 12100/java
tcp        0      0 25.80.44.51:7000        25.80.44.51:33562       ESTABLISHED 12100/java
tcp        0      0 25.80.44.51:7000        25.80.44.52:59547       ESTABLISHED 12100/java
tcp        0      0 25.80.44.51:7000        25.80.44.52:39067       ESTABLISHED 12100/java
tcp        0      0 25.80.44.51:41813       25.80.44.50:7000        ESTABLISHED 12100/java
tcp        0      0 25.80.44.51:53828       25.80.44.50:7000        ESTABLISHED 12100/java
tcp        0      0 25.80.44.51:7000        25.80.44.50:60348       ESTABLISHED 12100/java
tcp        0      0 25.80.44.51:44473       25.80.44.52:7000        ESTABLISHED 12100/java
tcp        0      0 25.80.44.51:55223       25.80.44.52:7000        ESTABLISHED 12100/java
[root@program-node02 conf]# netstat -anp | grep 9042
tcp        0      0 25.80.44.51:9042        0.0.0.0:*               LISTEN      12100/java
[root@program-node02 conf]#

节点工具信息:

[root@program-node02 conf]# /var/lib/cassandra/bin/nodetool info
ID                     : fa92ebd8-32f8-4f4f-bc85-37afd86a20c2
Gossip active          : true
Thrift active          : false
Native Transport active: true
Load                   : 50.71 MiB
Generation No          : 1616531553
Uptime (seconds)       : 5673
Heap Memory (MB)       : 1122.51 / 7987.25
Off Heap Memory (MB)   : 0.07
Data Center            : datacenter1
Rack                   : rack1
Exceptions             : 0
Key Cache              : entries 110, size 22.55 KiB, capacity 100 MiB, 108 hits, 173 requests, 0.624 recent hit rate, 14400 save period in seconds
Row Cache              : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 requests, NaN recent hit rate, 0 save period in seconds
Counter Cache          : entries 4, size 480 bytes, capacity 50 MiB, 0 hits, 0 requests, NaN recent hit rate, 7200 save period in seconds
Chunk Cache            : entries 493, size 30.81 MiB, capacity 480 MiB, 567 misses, 796 requests, 0.288 recent hit rate, NaN microseconds miss latency
Percent Repaired       : 100.0%
Token                  : (invoke with -T/--tokens to see all 256 tokens)
[root@program-node02 conf]#

节点 3

配置:

[root@program-node03 conf]# egrep -v "^$|^#|^ *#" /var/lib/cassandra/conf/cassandra.yaml
cluster_name: 'My Cluster'
num_tokens: 256
hinted_handoff_enabled: true
max_hint_window_in_ms: 10800000 # 3 hours
hinted_handoff_throttle_in_kb: 1024
max_hints_delivery_threads: 2
hints_flush_period_in_ms: 10000
max_hints_file_size_in_mb: 128
batchlog_replay_throttle_in_kb: 1024
authenticator: AllowAllAuthenticator
authorizer: AllowAllAuthorizer
role_manager: CassandraRoleManager
roles_validity_in_ms: 2000
permissions_validity_in_ms: 2000
credentials_validity_in_ms: 2000
partitioner: org.apache.cassandra.dht.Murmur3Partitioner
data_file_directories:
  - /var/lib/cassandra/data
commitlog_directory: /var/lib/cassandra/commitlog
cdc_enabled: false
disk_failure_policy: stop
commit_failure_policy: stop
prepared_statements_cache_size_mb:
thrift_prepared_statements_cache_size_mb:
key_cache_size_in_mb:
key_cache_save_period: 14400
row_cache_size_in_mb: 0
row_cache_save_period: 0
counter_cache_size_in_mb:
counter_cache_save_period: 7200
commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000
commitlog_segment_size_in_mb: 32
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          - seeds: "25.80.44.50,25.80.44.51"
concurrent_reads: 32
concurrent_writes: 32
concurrent_counter_writes: 32
concurrent_materialized_view_writes: 32
memtable_allocation_type: heap_buffers
index_summary_capacity_in_mb:
index_summary_resize_interval_in_minutes: 60
trickle_fsync: false
trickle_fsync_interval_in_kb: 10240
storage_port: 7000
ssl_storage_port: 7001
listen_address: 25.80.44.52
start_native_transport: true
native_transport_port: 9042
start_rpc: false
rpc_address: 25.80.44.52
rpc_port: 9160
rpc_keepalive: true
rpc_server_type: sync
thrift_framed_transport_size_in_mb: 15
incremental_backups: false
snapshot_before_compaction: false
auto_snapshot: true
column_index_size_in_kb: 64
column_index_cache_size_in_kb: 2
compaction_throughput_mb_per_sec: 16
sstable_preemptive_open_interval_in_mb: 50
read_request_timeout_in_ms: 5000
range_request_timeout_in_ms: 10000
write_request_timeout_in_ms: 2000
counter_write_request_timeout_in_ms: 5000
cas_contention_timeout_in_ms: 1000
truncate_request_timeout_in_ms: 60000
request_timeout_in_ms: 10000
cross_node_timeout: false
endpoint_snitch: SimpleSnitch
dynamic_snitch_update_interval_in_ms: 100
dynamic_snitch_reset_interval_in_ms: 600000
dynamic_snitch_badness_threshold: 0.1
request_scheduler: org.apache.cassandra.scheduler.NoScheduler
server_encryption_options:
    internode_encryption: none
    require_client_auth: false
    require_endpoint_verification: false
client_encryption_options:
    enabled: false
    require_client_auth: false
internode_compression: none
inter_dc_tcp_nodelay: false
tracetype_query_ttl: 86400
tracetype_repair_ttl: 604800
enable_user_defined_functions: false
enable_scripted_user_defined_functions: false
windows_timer_interval: 1
transparent_data_encryption_options:
    enabled: false
    chunk_length_kb: 64
    cipher: AES/CBC/PKCS5Padding
    key_alias: testing:1
    key_provider:
      - class_name: org.apache.cassandra.security.JKSKeyProvider
        parameters:
          - keystore: conf/.keystore
            keystore_password: cassandra
            store_type: JCEKS
            key_password: cassandra
tombstone_warn_threshold: 1000
tombstone_failure_threshold: 100000
batch_size_warn_threshold_in_kb: 5
batch_size_fail_threshold_in_kb: 50
unlogged_batch_across_partitions_warn_threshold: 10
compaction_large_partition_warning_threshold_mb: 100

LSOF:

[root@program-node03 conf]#  lsof -P -n -p 30557 | grep -i tcp
java    30557 cassandra   79u     IPv4           32883385       0t0        TCP 25.80.44.52:7000->25.80.44.52:35735 (ESTABLISHED)
java    30557 cassandra   80u     IPv4           32848102       0t0        TCP 25.80.44.52:50307->25.80.44.50:7000 (ESTABLISHED)
java    30557 cassandra   84u     IPv4           32833187       0t0        TCP 25.80.44.52:7000->25.80.44.51:44473 (ESTABLISHED)
java    30557 cassandra   87u     IPv4           32848098       0t0        TCP 25.80.44.52:7000->25.80.44.50:51688 (ESTABLISHED)
java    30557 cassandra   89u     IPv4           32848099       0t0        TCP 25.80.44.52:39067->25.80.44.51:7000 (ESTABLISHED)
java    30557 cassandra   95u     IPv4           32844289       0t0        TCP 127.0.0.1:33308 (LISTEN)
java    30557 cassandra   96u     IPv4           32844290       0t0        TCP *:7199 (LISTEN)
java    30557 cassandra   97u     IPv4           32848100       0t0        TCP 25.80.44.52:7000->25.80.44.50:34373 (ESTABLISHED)
java    30557 cassandra   98u     IPv4           32887275       0t0        TCP 25.80.44.52:35735->25.80.44.52:7000 (ESTABLISHED)
java    30557 cassandra  100u     IPv4           32848101       0t0        TCP 25.80.44.52:7000->25.80.44.52:37285 (ESTABLISHED)
java    30557 cassandra  212u     IPv4           32833190       0t0        TCP 25.80.44.52:9042 (LISTEN)
java    30557 cassandra  215u     IPv4           32853056       0t0        TCP 25.80.44.52:37285->25.80.44.52:7000 (ESTABLISHED)
java    30557 cassandra  216u     IPv4           32851005       0t0        TCP 25.80.44.52:7000->25.80.44.51:33449 (ESTABLISHED)
java    30557 cassandra  226u     IPv4           32833179       0t0        TCP 25.80.44.52:7000 (LISTEN)
java    30557 cassandra  228u     IPv4           32833185       0t0        TCP 25.80.44.52:59547->25.80.44.51:7000 (ESTABLISHED)
java    30557 cassandra  229u     IPv4           32833186       0t0        TCP 25.80.44.52:7000->25.80.44.51:55223 (ESTABLISHED)

网络统计:

[root@program-node03 conf]# netstat -anp | grep 7000
tcp        0      0 25.80.44.52:7000        0.0.0.0:*               LISTEN      30557/java
tcp        0      0 25.80.44.52:7000        25.80.44.51:44473       ESTABLISHED 30557/java
tcp        0      0 25.80.44.52:37285       25.80.44.52:7000        ESTABLISHED 30557/java
tcp        0      0 25.80.44.52:7000        25.80.44.50:34373       ESTABLISHED 30557/java
tcp        0      0 25.80.44.52:7000        25.80.44.50:51688       ESTABLISHED 30557/java
tcp        0      0 25.80.44.52:50307       25.80.44.50:7000        ESTABLISHED 30557/java
tcp        0      0 25.80.44.52:59547       25.80.44.51:7000        ESTABLISHED 30557/java
tcp        0      0 25.80.44.52:7000        25.80.44.52:37285       ESTABLISHED 30557/java
tcp        0      0 25.80.44.52:39067       25.80.44.51:7000        ESTABLISHED 30557/java
tcp        0      0 25.80.44.52:7000        25.80.44.51:55223       ESTABLISHED 30557/java
[root@program-node03 conf]# netstat -anp | grep 9042
tcp        0      0 25.80.44.52:9042        0.0.0.0:*               LISTEN      30557/java
[root@program-node03 conf]#

节点工具信息:

[root@program-node03 conf]# /var/lib/cassandra/bin/nodetool info
ID                     : 5db50d6d-3eae-4a47-b7c9-4743cd80ec99
Gossip active          : true
Thrift active          : false
Native Transport active: true
Load                   : 47.3 MiB
Generation No          : 1616531553
Uptime (seconds)       : 5674
Heap Memory (MB)       : 81.91 / 1916.00
Off Heap Memory (MB)   : 0.07
Data Center            : datacenter1
Rack                   : rack1
Exceptions             : 0
Key Cache              : entries 93, size 12.27 KiB, capacity 95 MiB, 164 hits, 221 requests, 0.742 recent hit rate, 14400 save period in seconds
Row Cache              : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 requests, NaN recent hit rate, 0 save period in seconds
Counter Cache          : entries 8, size 960 bytes, capacity 47 MiB, 0 hits, 0 requests, NaN recent hit rate, 7200 save period in seconds
Chunk Cache            : entries 420, size 26.25 MiB, capacity 447 MiB, 559 misses, 866 requests, 0.355 recent hit rate, NaN microseconds miss latency
Percent Repaired       : 100.0%
Token                  : (invoke with -T/--tokens to see all 256 tokens)
[root@program-node03 conf]#

【问题讨论】:

注意:我指出我在节点 1 的“Nodetool 信息”段落底部看到的问题。 一开始就使用AllowAll auth 类来构建集群可能不是一个好主意。在列出您的 IP 地址的同时,绝对这样做不是一个好主意。 【参考方案1】:

已解决: 问题出在数据库数据中。使用上述配置,我:

    已停止 Cassandra 的所有 3 个节点。 从节点 2 和 3 中删除数据库 从我在更改任何内容之前所做的备份中恢复了节点 1 数据库 在节点 1 上启动 Cassandra(并让它“稳定”) 在节点 2 上启动 Cassandra(并让它将所有数据复制到其中) 在节点 3 上启动 Cassandra(并将所有数据复制到其中)

我唯一剩下的问题是每个节点只拥有 68%(ish)的数据。由于我正在寻找容错能力,我认为我需要将架构中所有数据的 replication_factor 更改为 3。

[root@program-node01 ~]# /var/lib/cassandra/bin/nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address      Load       Tokens       Owns (effective)  Host ID                               Rack
UN  25.80.44.52  45.77 MiB  256          67.2%             bcee7571-a2eb-402e-b86f-c7e3327a8b7d  rack1
UN  25.80.44.50  51.93 MiB  256          68.0%             cc96b4c7-d55e-4b32-80aa-a114bf8af7d1  rack1
UN  25.80.44.51  51.89 MiB  256          64.8%             fa92ebd8-32f8-4f4f-bc85-37afd86a20c2  rack1

【讨论】:

干得好,比尔。很高兴你把它修好了! 我确实更新了种子以反映所有配置的 node1 和 node2。 感谢大家的cmets和帮助【参考方案2】:

尝试将您的 seeds 列表设置为在所有 3 个节点上都相同。

现在,每个节点都不同...仅当种子列表中的一个节点已经运行时才有效。集群中的第一个节点需要在该种子列表中找到自己。

在像这样的小型集群中,我会从您的第一个节点获取 seeds 设置,并确保所有其他节点的 seeds 属性匹配它:

- seeds: "25.80.44.51,25.80.44.52"

这将是一个好的开始。

【讨论】:

以上是关于如何将第三个节点添加到 2 节点 Cassandra 集群?的主要内容,如果未能解决你的问题,请参考以下文章

将节点添加到 Cassandra 集群

添加 3 个节点时,Storm Supervisor 未启动 [关闭]

如何将第三个按钮添加到 Android 警报对话框?

(单链表)单链表的整体逆序和局部逆序

11g两节点RAC添加第三个节点

[ZJOI2016]大森林