无法连接：通信链路故障
Posted
技术标签:
【中文标题】无法连接：通信链路故障【英文标题】：Unable to connect: Communications link failure 【发布时间】：2021-11-21 15:45:53 【问题描述】：我正在尝试遵循教程Deploying Debezium using the new KafkaConnector resource。 根据教程，我也在使用 minikube，但使用了 docker 驱动程序。基本上只是一步一步地完全按照。
但是，对于步骤“创建连接器”，在创建连接器之后
cat <<EOF | kubectl -n kafka apply -f -
apiVersion: "kafka.strimzi.io/v1alpha1"
kind: "KafkaConnector"
metadata:
name: "inventory-connector"
labels:
strimzi.io/cluster: my-connect-cluster
spec:
class: io.debezium.connector.mysql.MySqlConnector
tasksMax: 1
config:
database.hostname: 192.168.99.1
database.port: "3306"
database.user: "$file:/opt/kafka/external-configuration/connector-config/debezium-mysql-credentials.properties:mysql_username"
database.password: "$file:/opt/kafka/external-configuration/connector-config/debezium-mysql-credentials.properties:mysql_password"
database.server.id: "184054"
database.server.name: "dbserver1"
database.whitelist: "inventory"
database.history.kafka.bootstrap.servers: "my-cluster-kafka-bootstrap:9092"
database.history.kafka.topic: "schema-changes.inventory"
include.schema.changes: "true"
EOF
然后检查
kubectl -n kafka get kctr inventory-connector -o yaml
我有错误
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaConnector
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
"apiVersion":"kafka.strimzi.io/v1alpha1","kind":"KafkaConnector","metadata":"annotations":,"labels":"strimzi.io/cluster":"my-connect-cluster","name":"inventory-connector","namespace":"kafka","spec":"class":"io.debezium.connector.mysql.MySqlConnector","config":"database.history.kafka.bootstrap.servers":"my-cluster-kafka-bootstrap:9092","database.history.kafka.topic":"schema-changes.inventory","database.hostname":"192.168.49.2","database.password":"","database.port":"3306","database.server.id":"184054","database.server.name":"dbserver1","database.user":"","database.whitelist":"inventory","include.schema.changes":"true","tasksMax":1
creationTimestamp: "2021-09-29T18:20:11Z"
generation: 1
labels:
strimzi.io/cluster: my-connect-cluster
name: inventory-connector
namespace: kafka
resourceVersion: "12777"
uid: 083df9a3-83ce-4170-a9bc-9573dafdb286
spec:
class: io.debezium.connector.mysql.MySqlConnector
config:
database.history.kafka.bootstrap.servers: my-cluster-kafka-bootstrap:9092
database.history.kafka.topic: schema-changes.inventory
database.hostname: 192.168.49.2
database.password: ""
database.port: "3306"
database.server.id: "184054"
database.server.name: dbserver1
database.user: ""
database.whitelist: inventory
include.schema.changes: "true"
tasksMax: 1
status:
conditions:
- lastTransitionTime: "2021-09-29T18:20:11.548Z"
message: |-
PUT /connectors/inventory-connector/config returned 400 (Bad Request): Connector configuration is invalid and contains the following 1 error(s):
A value is required
You can also find the above list of errors at the endpoint `/connectorType/config/validate`
reason: ConnectRestException
status: "True"
type: NotReady
observedGeneration: 1
我试图改变
database.user: "$file:/opt/kafka/external-configuration/connector-config/debezium-mysql-credentials.properties:mysql_username"
database.password: "$file:/opt/kafka/external-configuration/connector-config/debezium-mysql-credentials.properties:mysql_password"
到
database.user: "debezium"
database.password: "dbz"
根据“保护数据库凭据”步骤中的用户和密码信息直接重新申请。
另外，根据教程中的描述
我使用
database.hostname: 192.168.99.1作为连接 MySQL 的 IP 地址，因为我正在使用带有 virtualbox VM 驱动程序的 minikube 如果您使用带有minikube的不同 VM 驱动程序，您可能需要不同的 IP 地址。
我实际上对上面的描述有点困惑。演示中的 MySQL 部署在 Docker 中，而 Kafka 等其余部分部署在 minikube 中。为什么database.hostname 的描述是 minikube 而不是 Docker？
无论如何，当我运行minikube ip 时，我得到了192.168.49.2。但是，在我将database.hostname 更改为192.168.49.2 并运行kubectl get kctr inventory-connector -o yaml -n kafka 后，我得到了
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaConnector
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
"apiVersion":"kafka.strimzi.io/v1alpha1","kind":"KafkaConnector","metadata":"annotations":,"labels":"strimzi.io/cluster":"my-connect-cluster","name":"inventory-connector","namespace":"kafka","spec":"class":"io.debezium.connector.mysql.MySqlConnector","config":"database.history.kafka.bootstrap.servers":"my-cluster-kafka-bootstrap:9092","database.history.kafka.topic":"schema-changes.inventory","database.hostname":"192.168.49.2","database.password":"","database.port":"3306","database.server.id":"184054","database.server.name":"dbserver1","database.user":"","database.whitelist":"inventory","include.schema.changes":"true","tasksMax":1
creationTimestamp: "2021-09-29T18:20:11Z"
generation: 1
labels:
strimzi.io/cluster: my-connect-cluster
name: inventory-connector
namespace: kafka
resourceVersion: "12777"
uid: 083df9a3-83ce-4170-a9bc-9573dafdb286
spec:
class: io.debezium.connector.mysql.MySqlConnector
config:
database.history.kafka.bootstrap.servers: my-cluster-kafka-bootstrap:9092
database.history.kafka.topic: schema-changes.inventory
database.hostname: 192.168.49.2
database.password: ""
database.port: "3306"
database.server.id: "184054"
database.server.name: dbserver1
database.user: ""
database.whitelist: inventory
include.schema.changes: "true"
tasksMax: 1
status:
conditions:
- lastTransitionTime: "2021-09-29T18:20:11.548Z"
message: |-
PUT /connectors/inventory-connector/config returned 400 (Bad Request): Connector configuration is invalid and contains the following 1 error(s):
A value is required
You can also find the above list of errors at the endpoint `/connectorType/config/validate`
reason: ConnectRestException
status: "True"
type: NotReady
observedGeneration: 1
我可以通过localhost 访问 MySQL，因为它托管在 Docker 中。
但是，当我将database.hostname 更改为localhost 时，我仍然出现同样的错误。
有什么想法吗？谢谢！
【问题讨论】：
错误似乎是 400 Bad Request，所以它可能已连接到主机，但您的输入可能无效。您可以尝试验证您的配置吗？ 在数据库主机名和其他字段中使用双引号“”有帮助吗？ 感谢@JavaTechnical 在添加了database.hostname: "192.168.49.2" 之类的引用后，还是一样。
您是否为所有人添加了“”？还可以尝试将connector.class 添加到配置并检查
@JavaTechnical 嗯，你是什么意思？在规范 -> 配置中，只有 database.hostname 值没有 ""。
【参考方案1】：
该问题与 minikube 中的服务无法与 docker 中的 MySQL 通信有关。
关于如何从Kubernetes集群内部访问主机的localhost，我找到了How to access host's localhost from inside kubernetes cluster
但是，我最终在 Kubernetes 方向上部署 MySQL
kubectl apply -f https://k8s.io/examples/application/mysql/mysql-pv.yaml
kubectl apply -f https://k8s.io/examples/application/mysql/mysql-deployment.yaml
（复制自https://kubernetes.io/docs/tasks/run-application/run-single-instance-stateful-application/）
与
database.hostname: "mysql.default" # service `mysql` in namespace `default`
database.port: "3306"
database.user: "root"
database.password: "password"
现在当我跑步时
kubectl -n kafka get kctr inventory-connector -o yaml
我得到一个新的错误，说 MySQL 没有启用行级 binlog，但是，这意味着它现在可以连接 MySQL。
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaConnector
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
"apiVersion":"kafka.strimzi.io/v1alpha1","kind":"KafkaConnector","metadata":"annotations":,"labels":"strimzi.io/cluster":"my-connect-cluster","name":"inventory-connector","namespace":"kafka","spec":"class":"io.debezium.connector.mysql.MySqlConnector","config":"database.history.kafka.bootstrap.servers":"my-cluster-kafka-bootstrap:9092","database.history.kafka.topic":"schema-changes.inventory","database.hostname":"mysql.default","database.password":"password","database.port":"3306","database.server.id":"184054","database.server.name":"dbserver1","database.user":"root","database.whitelist":"inventory","include.schema.changes":"true","tasksMax":1
creationTimestamp: "2021-09-29T19:36:52Z"
generation: 1
labels:
strimzi.io/cluster: my-connect-cluster
name: inventory-connector
namespace: kafka
resourceVersion: "2918"
uid: 48bb46e1-42bb-4574-a3dc-221ae7d6a803
spec:
class: io.debezium.connector.mysql.MySqlConnector
config:
database.history.kafka.bootstrap.servers: my-cluster-kafka-bootstrap:9092
database.history.kafka.topic: schema-changes.inventory
database.hostname: mysql.default
database.password: password
database.port: "3306"
database.server.id: "184054"
database.server.name: dbserver1
database.user: root
database.whitelist: inventory
include.schema.changes: "true"
tasksMax: 1
status:
conditions:
- lastTransitionTime: "2021-09-29T19:36:53.605Z"
status: "True"
type: Ready
connectorStatus:
connector:
state: UNASSIGNED
worker_id: 172.17.0.8:8083
name: inventory-connector
tasks:
- id: 0
state: FAILED
trace: "org.apache.kafka.connect.errors.ConnectException: The MySQL server is
not configured to use a row-level binlog, which is required for this connector
to work properly. Change the MySQL configuration to use a row-level binlog
and restart the connector.\n\tat io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:207)\n\tat
io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:49)\n\tat
org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:208)\n\tat
org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)\n\tat
org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)\n\tat
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat
java.lang.Thread.run(Thread.java:748)\n"
worker_id: 172.17.0.8:8083
type: source
observedGeneration: 1
【讨论】：
推荐你把数据库密码移到k8s secrets 感谢@OneCricketeer 的提醒！ ? 只是从kubernetes.io/docs/tasks/run-application/…复制的demo以上是关于无法连接：通信链路故障的主要内容，如果未能解决你的问题，请参考以下文章