使用 TcpDiscoveryKubernetesIpFinder 在 Kubernetes 集群中无法发现 Ignite
Posted
技术标签:
【中文标题】使用 TcpDiscoveryKubernetesIpFinder 在 Kubernetes 集群中无法发现 Ignite【英文标题】:Ignite not discoverable in kubernetes cluster with TcpDiscoveryKubernetesIpFinder 【发布时间】:2019-02-01 14:23:40 【问题描述】:我正在尝试使用TcpDiscoveryKubernetesIpFinder
使部署在 k8s 中的 ignite 可被发现。我还使用了 apache ignite 文档中推荐的所有部署配置,以使其可被发现。点燃版本是v2.6
。当我尝试从集群(和命名空间)内的另一个服务访问 ignite 时,它无法给出以下错误。
。 . instance-14292nccv10-74997cfdff-kqdqh] 引起: java.io.IOException:服务器返回 HTTP 响应代码:403 用于 URL: https://kubernetes.default.svc.cluster.local:443/api/v1/namespaces/my-namespace/endpoints/ignite-service [实例-14292nccv10-74997cfdff-kqdqh] 在 sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894) ~[na:1.8.0_151] [instance-14292nccv10-74997cfdff-kqdqh] 在 sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) ~[na:1.8.0_151] [instance-14292nccv10-74997cfdff-kqdqh] 在 sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263) ~[na:1.8.0_151] [instance-14292nccv10-74997cfdff-kqdqh]。 .
我的 ignite 配置使其可被发现如下,
apiVersion: v1
kind: ServiceAccount
metadata:
name: ignite-service
namespace: my-namespace
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: ignite-service
namespace: my-namespace
rules:
- apiGroups:
- ""
resources:
- pods
- endpoints
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: ignite-service
roleRef:
kind: ClusterRole
name: ignite-service
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: ignite-service
namespace: my-namespace
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: ignite-service-volume-claim-blr3
namespace: my-namespace
spec:
storageClassName: ssd
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: Secret
metadata:
name: ignite-files
namespace: my-namespace
data:
ignite-config.xml: PGJlYW5zIHhtbG5zID0gImh0dHA6Ly93d3cuc3ByaW5nZnJhbWV3b3JrLm9yZy9zY2hlbWEvYmVhbnMiCiAgICAgICB4bWxuczp4c2kgPSAiaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiCiAgICAgICB4bWxuczp1dGlsID0gImh0dHA6Ly93d3cuc3ByaW5nZnJhbWV3b3JrLm9yZy9zY2hlbWEvdXRpbCIKICAgICAgIHhzaTpzY2hlbWFMb2NhdGlvbiA9ICIKICAgICAgIGh0dHA6Ly93d3cuc3ByaW5nZnJhbWV3b3JrLm9yZy9zY2hlbWEvYmVhbnMKICAgICAgIGh0dHA6Ly93d3cuc3ByaW5nZnJhbWV3b3JrLm9yZy9zY2hlbWEvYmVhbnMvc3ByaW5nLWJlYW5zLnhzZAogICAgICAgaHR0cDovL3d3dy5zcHJpbmdmcmFtZXdvcmsub3JnL3NjaGVtYS91dGlsCiAgICAgICBodHRwOi8vd3d3LnNwcmluZ2ZyYW1ld29yay5vcmcvc2NoZW1hL3V0aWwvc3ByaW5nLXV0aWwueHNkIj4KCiAgICA8YmVhbiBjbGFzcyA9ICJvcmcuYXBhY2hlLmlnbml0ZS5jb25maWd1cmF0aW9uLklnbml0ZUNvbmZpZ3VyYXRpb24iPgogICAgICAgIDxwcm9wZXJ0eSBuYW1lID0gImRpc2NvdmVyeVNwaSI+CiAgICAgICAgICAgIDxiZWFuIGNsYXNzID0gIm9yZy5hcGFjaGUuaWduaXRlLnNwaS5kaXNjb3ZlcnkudGNwLlRjcERpc2NvdmVyeVNwaSI+CiAgICAgICAgICAgICAgICA8cHJvcGVydHkgbmFtZSA9ICJpcEZpbmRlciI+CiAgICAgICAgICAgICAgICAgICAgPGJlYW4gY2xhc3MgPSAib3JnLmFwYWNoZS5pZ25pdGUuc3BpLmRpc2NvdmVyeS50Y3AuaXBmaW5kZXIua3ViZXJuZXRlcy5UY3BEaXNjb3ZlcnlLdWJlcm5ldGVzSXBGaW5kZXIiPgogICAgICAgICAgICAgICAgICAgICAgICA8cHJvcGVydHkgbmFtZT0ibmFtZXNwYWNlIiB2YWx1ZT0ibXktbmFtZXNwYWNlIi8+CiAgICAgICAgICAgICAgICAgICAgICAgIDxwcm9wZXJ0eSBuYW1lPSJzZXJ2aWNlTmFtZSIgdmFsdWU9Imlnbml0ZS1zZXJ2aWNlIi8+CiAgICAgICAgICAgICAgICAgICAgPC9iZWFuPgogICAgICAgICAgICAgICAgPC9wcm9wZXJ0eT4KICAgICAgICAgICAgPC9iZWFuPgogICAgICAgIDwvcHJvcGVydHk+CiAgICAgICAgPCEtLSBFbmFibGluZyBBcGFjaGUgSWduaXRlIG5hdGl2ZSBwZXJzaXN0ZW5jZS4gLS0+CiAgICAgICAgPHByb3BlcnR5IG5hbWUgPSAiZGF0YVN0b3JhZ2VDb25maWd1cmF0aW9uIj4KICAgICAgICAgICAgPGJlYW4gY2xhc3MgPSAib3JnLmFwYWNoZS5pZ25pdGUuY29uZmlndXJhdGlvbi5EYXRhU3RvcmFnZUNvbmZpZ3VyYXRpb24iPgogICAgICAgICAgICAgICAgPHByb3BlcnR5IG5hbWUgPSAiZGVmYXVsdERhdGFSZWdpb25Db25maWd1cmF0aW9uIj4KICAgICAgICAgICAgICAgICAgICA8YmVhbiBjbGFzcyA9ICJvcmcuYXBhY2hlLmlnbml0ZS5jb25maWd1cmF0aW9uLkRhdGFSZWdpb25Db25maWd1cmF0aW9uIj4KICAgICAgICAgICAgICAgICAgICAgICAgPHByb3BlcnR5IG5hbWUgPSAicGVyc2lzdGVuY2VFbmFibGVkIiB2YWx1ZSA9ICJ0cnVlIi8+CiAgICAgICAgICAgICAgICAgICAgPC9iZWFuPgogICAgICAgICAgICAgICAgPC9wcm9wZXJ0eT4KICAgICAgICAgICAgICAgIDxwcm9wZXJ0eSBuYW1lID0gInN0b3JhZ2VQYXRoIiB2YWx1ZSA9ICIvZGF0YS9pZ25pdGUvc3RvcmFnZSIvPgogICAgICAgICAgICAgICAgPHByb3BlcnR5IG5hbWUgPSAid2FsUGF0aCIgdmFsdWUgPSAiL2RhdGEvaWduaXRlL2RiL3dhbCIvPgogICAgICAgICAgICAgICAgPHByb3BlcnR5IG5hbWUgPSAid2FsQXJjaGl2ZVBhdGgiIHZhbHVlID0gIi9kYXRhL2lnbml0ZS9kYi93YWwvYXJjaGl2ZSIvPgogICAgICAgICAgICA8L2JlYW4+CiAgICAgICAgPC9wcm9wZXJ0eT4KICAgIDwvYmVhbj4KPC9iZWFucz4=
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
# Name of Ignite Service used by Kubernetes IP finder.
# The name must be equal to TcpDiscoveryKubernetesIpFinder.serviceName.
name: ignite-service
namespace: my-namespace
spec:
clusterIP: None # custom value.
ports:
- port: 9042 # custom value.
selector:
# Must be equal to one of the labels set in Ignite pods'
# deployement configuration.
app: ignite-service
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
# Custom Ignite cluster's name.
name: ignite-service
namespace: my-namespace
spec:
# A number of Ignite pods to be started by Kubernetes initially.
replicas: 1
template:
metadata:
labels:
# This label has to be added to the selector's section of
# ignite-service.yaml so that the Kubernetes Ignite lookup service
# can easily track all Ignite pods available deployed so far.
app: ignite-service
spec:
serviceAccountName: ignite-service
volumes:
# Custom name for the storage that holds Ignite's configuration
# which is example-kube.xml.
- name: ignite-storage
persistentVolumeClaim:
# Must be equal to the PersistentVolumeClaim created before.
claimName: ignite-service-volume-claim-blr3
- name: ignite-files
secret:
secretName: ignite-files
containers:
# Custom Ignite pod name.
- name: ignite-node
# Ignite Docker image. Kubernetes IP finder is supported starting from
# Apache Ignite 2.6.0
image: apacheignite/ignite:2.6.0
lifecycle:
postStart:
exec:
command: ['/bin/sh', '/opt/ignite/apache-ignite-fabric/bin/control.sh', '--activate']
env:
# Ignite's Docker image parameter. Adding the jar file that
# contain TcpDiscoveryKubernetesIpFinder implementation.
- name: OPTION_LIBS
value: ignite-kubernetes
# Ignite's Docker image parameter. Passing the Ignite configuration
# to use for an Ignite pod.
- name: CONFIG_URI
value: file:///etc/ignite-files/ignite-config.xml
- name: ENV
value: my-namespace
ports:
# Ports to open.
# Might be optional depending on your Kubernetes environment.
- containerPort: 11211 # REST port number.
- containerPort: 47100 # communication SPI port number.
- containerPort: 47500 # discovery SPI port number.
- containerPort: 49112 # JMX port number.
- containerPort: 10800 # SQL port number.
volumeMounts:
# Mounting the storage with the Ignite configuration.
- mountPath: "/data/ignite"
name: ignite-storage
- name: ignite-files
mountPath: "/etc/ignite-files"
我在 *** 中看到了一些具有类似问题的链接,遵循了建议的解决方案,但这也不起作用。对此的任何指示都会有很大帮助!
【问题讨论】:
你能分享你的点燃配置吗 更新了整个 deployment.yml 以进行点燃 你能检查kubectl -n my-namespace get svc
和kubectl -n my-namespace get ep
,它是否列出了名称为ignite-service
的服务
这两个命令都列出了我的服务。第一个命令列出了 cluster-Ip 为 None 的命令。另一个命令以 ip:port 格式显示端点。
【参考方案1】:
根据 URL,IP 查找器尝试使用名为 ignite
的服务,而您使用名称 ignite-service
创建它。
您应该在 IP finder 配置中同时提供命名空间和服务名称:
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
<property name="namespace" value="my-namespace"/>
<property name="serviceName" value="ignite-service"/>
</bean>
【讨论】:
错误现在变成了,java.io.IOException: Server returned HTTP response code: 403 for URL: kubernetes.default.svc.cluster.local:443/api /v1/namespaces/my-namespace/endpoints/ignite-service跨度> @Kaps,看来你还需要在部署文件中指定serviceAccountName: ignite-service
(比如这里apacheignite.readme.io/docs/…)
@Kaps 你修改了配置,但还是不行?如果是这样,请使用最新配置更新您的问题。
@ValentinKulichenko,更新了问题中的最新部署文件配置。
@ValentinKulichenko,对此有何想法?【参考方案2】:
您需要确保已锁定并处理了以下内容。
在 Kubernetes 中创建命名空间 在 Kubernetes 中创建您的服务帐户 在集群的命名空间中为您的服务帐户设置的权限。服务帐号权限
https://kubernetes.io/docs/reference/access-authn-authz/rbac/#service-account-permissions
【讨论】:
以上是关于使用 TcpDiscoveryKubernetesIpFinder 在 Kubernetes 集群中无法发现 Ignite的主要内容,如果未能解决你的问题,请参考以下文章
在使用加载数据流步骤的猪中,使用(使用 PigStorage)和不使用它有啥区别?
Qt静态编译时使用OpenSSL有三种方式(不使用,动态使用,静态使用,默认是动态使用)