在k8s上部署嵌入式模式的hazelcast
将带有嵌入式Hazelcast的应用程序部署到Kubernetes集群中。
来自每个应用程序副本的Hazelcast实例都将自动发现它们自己，并形成一个一致的Hazelcast集群。得益于 Hazelcast Kubernetes自动发现插件，不需要静态地写死配置。
准备
- Docker (Docker for Desktop is good enough)
- Kubernetes cluster (Docker for Desktop or Minikube is good enough)
- Git
- JDK 1.8+
- Apache Maven 3.2+
示例
我们可以将 Hazelcast 嵌入到任何基于jvm的应用程序中，并使用任何您想要的web框架。 作为本教程的一个示例，我们使用 Spring Boot 入门教程中的应用程序。我执行以下命令来下载：
git clone https://github.com/hazelcast-guides/hazelcast-embedded-springboot.git
配置
Hazelcast提供了专用的Hazelcast Kubernetes插件，允许在Kubernetes环境中自动形成Hazelcast集群。我们使用以下Hazelcast配置来启动这个功能。
hazelcast:
network:
join:
multicast:
enabled: false # 反使能多播模式
kubernetes:
enabled: true # 使能kubernetes模式要将此文件包含在Spring Boot项目中，将其复制到hazelcast-embedded-springboot/src/resources/中。
cp hazelcast.yaml hazelcast-embedded-springboot/src/main/resources/
现在，我们可以使用以下命令构建项目。
mvn package -f hazelcast-embedded-springboot/pom.xml
作为输出，我们的应用程序的JAR文件应该在hazelcast-embedded-springboot/target/*. JAR中创建。
将应用程序容器化
要将应用程序容器化，需要安装Docker，我们可以使用以下Dockerfile
FROM openjdk:8-jre-alpine
COPY hazelcast-embedded-springboot/target/*.jar app.jar
ENTRYPOINT ["java","-jar","app.jar"]
编译成镜像
docker build -t hazelcastguides/hazelcast-embedded-kubernetes .
推送到远程
docker push hazelcastguides/hazelcast-embedded-kubernetes配置Configure RBAC
Hazelcast Kubernetes发现插件通过调用Kubernetes API来提供自动成员发现。因此，它需要授予特定的ClusterRole规则。我们可以使用以下命令应用最小的RBAC配置(对于默认名称空间中的默认服务帐户)。
kubectl apply -f https://raw.githubusercontent.com/hazelcast/hazelcast/master/kubernetes-rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: hazelcast-cluster-role
rules:
- apiGroups:
- ""
# Access to apps API is only required to support automatic cluster state management
# when persistence (hot-restart) is enabled.
- apps
resources:
- endpoints
- pods
- nodes
- services
# Access to statefulsets resource is only required to support automatic cluster state management
# when persistence (hot-restart) is enabled.
- statefulsets
verbs:
- get
- list
# Watching resources is only required to support automatic cluster state management
# when persistence (hot-restart) is enabled.
- watch
- apiGroups:
- "discovery.k8s.io"
resources:
- endpointslices
verbs:
- get
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: hazelcast-cluster-role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: hazelcast-cluster-role
subjects:
- kind: ServiceAccount
name: default
namespace: default如果您使用非默认的服务帐户或非默认的命名空间，则需要修改 https://raw.githubusercontent.com/hazelcast/hazelcast/master/kubernetes-rbac.yaml 如果您的Kubernetes集群不使用RBAC，则可以跳过整个“配置RBAC”步骤
将应用程序部署到Kubernetes
可以直接 apply 以下的部署的yaml文件
apiVersion: apps/v1
kind: Deployment
metadata:
name: hazelcast-test
spec:
replicas: 2
selector:
matchLabels:
app: hazelcast-test
template:
metadata:
labels:
app: hazelcast-test
app.hazelcast/name: iidp
spec:
containers:
- name: hazelcast-test
image: lizhanbin/hazelcast-test
imagePullPolicy: Always
resources:
requests:
memory: 100Mi
---
apiVersion: v1
kind: Service
metadata:
name: my-app-svc
labels:
app: my-app-svc
spec:
type: ClusterIP
ports:
- port: 8080
selector:
app: my-app
然后我们直接查看组网情况
$ kubectl logs pod/my-app-86df8b785f-4x9pj
...
Members {size:2, ver:2} [
Member [10.24.1.10]:5701 - a7eb36b6-6d86-4d26-8eb6-47986e46d055 this
Member [10.24.2.6]:5701 - 9994d6c6-d271-4ddd-9aa9-1ac4767c1a73
]测试应用程序
直接执行命令请求服务
$ curl --data "key=key1&value=hazelcast" "localhost:8080/put"
{"value":"hazelcast"}
$ curl "localhost:8080/get?key=key1"
{"value":"hazelcast"}
停止集群
kubectl delete deployment/my-app service/my-app
kubectl delete -f https://raw.githubusercontent.com/hazelcast/hazelcast/master/kubernetes-rbac.yaml
sidecar形式部署
hazelcast支持以sidecar形式进行部署，从而可以跨语言来进行通信，而不仅仅是在java中使用，我们以golang为例，来实现一个集群，且与java版的hazelcast集群正常通信和组网。
我们编写 main.go
package main
import (
"context"
"fmt"
"github.com/gin-gonic/gin"
"github.com/hazelcast/hazelcast-go-client"
)
var (
hz *hazelcast.Client
mp *hazelcast.Map
)
func main() {
r := gin.New()
r.GET("/ping", func(c *gin.Context) {
c.JSON(200, gin.H{
"message": "pong",
"code": 200,
})
})
r.POST("/set", func(c *gin.Context) {
// get kv
// get from hazelcast
err := mp.Set(c, "foo", "bar")
if err != nil {
c.JSON(500, gin.H{
"message": "get error",
"code": 500,
})
return
}
c.JSON(200, gin.H{
"message": "ok",
"code": 200,
})
})
r.GET("/get", func(c *gin.Context) {
// get kv
// get from hazelcast
resp, err := mp.Get(c, "foo")
if err != nil {
c.JSON(500, gin.H{
"message": "get error",
"code": 500,
})
return
}
c.JSON(200, gin.H{
"message": "ok",
"code": 200,
})
})
r.GET("/get", func(c *gin.Context) {
// get kv
// get from hazelcast
resp, err := mp.Get(c, "foo")
if err != nil {
c.JSON(500, gin.H{
"message": "get error",
"code": 500,
})
return
}
c.JSON(200, gin.H{
"message": resp,
"code": 200,
})
})
r.Run(":8888")
}
func init() {
ctx := context.TODO()
cfg := hazelcast.Config{}
cfg.Cluster.Name = "hazelcast-benchmark"
// cfg.Cluster.Network.Addresses = []string{"192.168.168.176:5701"}
var err error
hz, err = hazelcast.StartNewClientWithConfig(ctx, cfg)
if err != nil {
panic(fmt.Errorf("starting the client with config: %w", err))
}
mp, err = hz.GetMap(ctx, "my-distributed-map")
if err != nil {
panic(fmt.Errorf("trying to get a map: %w", err))
}
}
同样地，我们编写yaml文件来部署这个golang程序，并携带上sidecar，如下所示。得益于k8s容器机制，如果是在同一个pod下的容器，则天然是共享同一个网络的，那么就可以直接基于localhost来进行通信。
apiVersion: apps/v1
kind: Deployment
metadata:
name: sidecar-test
spec:
replicas: 3
selector:
matchLabels:
app: test
template:
metadata:
labels:
app: test
spec:
containers:
- name: test
image: lizhanbin/sidecar-test
imagePullPolicy: Always
ports:
- containerPort: 8888
- name: hazelcast
image: hazelcast/hazelcast:5.3.6
lifecycle:
type: Sidecar
env:
- name: HZ_CLUSTERNAME
value: hazelcast-benchmark
ports:
- name: hazelcast
containerPort: 5701
- name: mc
image: hazelcast/management-center:latest-snapshot
ports:
- name: mc
containerPort: 8080