A ConfigMap is an API object used to store non-confidential data in key-value pairs. Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume.
configmap可以把配置映射到 环境变量中，也可以写到文件中供程序读取: 这样以来，就部署迁移到k8s,对于程序来说是无感知的
把app部署到k8s运行，几个关键点:
- 打包镜像,处理daemon process
- 配置环境变量化(加少量加载配置代码),也可以把configMap配置到文件中(无需改动程序加载配置代码)
- 目录volumn配置映射
- helm 配置化
把app部署到k8s时，考虑最小化改动程序代码，这里把原来app运行时用到的config.yaml的内容设置到一个环境变量如这里的talkingbot_yaml中, app启动代码优先读这个环境变量，如果不为空则取改变量值作为替换为原来config.yaml的内容,如果为空则说明按照原有的文件路径的方式取config.yaml
部署app的演进历程 #
running one app:raw #
- app -c config.yaml
running one app:docker #
- DockerFile: entrypoint e.g. talkingbot:1.0
- build images, e.g. talkingbot:1.0
- run container: mount -v /data:/home/app (path host:/data/config.yaml container: /home/app/config.yaml) or set config to env on docker run
- app -c config.yaml
running one app:k8s #
- base on image
- deployment: pvc, pc,configmap,deployment(pod replica:containers that running your app), service
- pvc: ask for storage
- pc: genearte storage for pvc
- service: auto dns for pods, round robin requst to pods; demo using helm
- configmap: provide to app by setting config in env
running one app:k8s-helm #
what is helm?
- helm on k8s like apt on ubuntu, yum on centos
show me the code 实操: 以部署stateless app的部署为例子 #
Dokcerfile #
FROM alpine
RUN apk add --no-cache ca-certificates su-exec
VOLUME ["/data/talkingbot"]
COPY ./TalkingBot /bin/TalkingBot
COPY ./deployment/dumb-init /bin/dumb-init
RUN chmod +x /bin/dumb-init
EXPOSE 8086
ENTRYPOINT ["/bin/dumb-init", "/bin/TalkingBot","-c","/data/talkingbot/config.yaml"]
假设开始的app为TalkingBot,http监听端口为8086,这里使用dumb-init来防止一些daemon process issue
生成镜像
docker build -t talkingbot:1.0 .
#docker run --rm --name TalkingBot -p 8086:8086 -v /data/talkingbot:/data/talkingbot talkingbot:1.0
k8s Deployment all in one yaml file #
部署app用的k8s配置k8s-all-in-one.yaml, 文件里面包含了k8s运行app所用到的
deployment,service,configmap,pc,pvc
apiVersion: apps/v1
kind: Deployment
metadata:
name: talkingbot
namespace: kube-public
spec:
replicas: 2
selector:
matchLabels:
app: talkingbot
template:
metadata:
labels:
app: talkingbot
spec:
containers:
- name: talkingbot
image: 'talkingbot:1.0'
env:
- name: talkingbot_yaml
valueFrom:
configMapKeyRef:
name: talkingbot-config
key: configmapkey-talkingbot.yaml
ports:
- containerPort: 8086
volumeMounts:
- name: talkingbot-volume
mountPath: /data/talkingbot
volumes:
- name: talkingbot-volume
emptyDir: {}
# hostPath:
# directory location on physical host
# path: /data/go/src/talkingbot/config
---
apiVersion: v1
kind: Service
metadata:
name: talkingbot-svc
namespace: kube-public
labels:
app: talkingbot-svc
spec:
type: NodePort
ports:
- port: 8086 #port of service running inner pod, must exist
name: talkingbot-port
nodePort: 30081 # request by external load balance, must exist when `type: NodePort`
# targetPort: 8170 #port of service running inner pod?, can be ignored ; By default the targetPort will be set to the same value as the port
# externalIPs:
# - 192.168.1.125
selector:
app: talkingbot
---
apiVersion: v1
data:
configmapkey-talkingbot.yaml: |
appName: 'TalkingBot' #应用名称
serverPort: 8086 #端口
devMode: True #开发调试模式开关
flowDir: '/data/talkingbot' #已载入流程存放路径，if empty ,it will use ./
audioDir: "/data/talkingbot/audioDir/"
httpTimeoutSecond: 1
delayResponceBeginSecond: 1 #开始延迟回复,秒
httpRateLimitMax: 100 #请求model时最大迸发数，防止model负载过高503,默认100/s
redisDeployKind: '1' #1:单例(默认), 2:哨兵主从, 3:官方redis-cluster
redisSentinelMasterName: 'master' #哨兵模式下 master key name, 默认master
redisMain: 'my-redis-master.default.svc.cluster.local:6379'
redisPassword: 'LEt3bEzn4g'
kind: ConfigMap
metadata:
name: talkingbot-config
namespace: kube-public
操作记录
jialin in DESKTOP-F6Q50AM in TalkingBot on k8s [$?] via 🐹 v1.18
❯ kubectl apply -f k8s-all-in-one.yaml
deployment.apps/talkingbot created
service/talkingbot-svc created
configmap/talkingbot-config created
jialin in DESKTOP-F6Q50AM in TalkingBot on k8s [$?] via 🐹 v1.18
❯ kubectl get pod -n kube-public
NAME READY STATUS RESTARTS AGE
talkingbot-57b9948868-5tzxr 0/1 ContainerCreating 0 2s
talkingbot-57b9948868-lrw77 0/1 ContainerCreating 0 2s
jialin in DESKTOP-F6Q50AM in TalkingBot on k8s [$?] via 🐹 v1.18
❯ kubectl get pod -n kube-public
NAME READY STATUS RESTARTS AGE
talkingbot-57b9948868-5tzxr 1/1 Running 0 12s
talkingbot-57b9948868-lrw77 1/1 Running 0 12s
k8s Deployment by helm #
helm 把上面的 # 做成了 模板配置化的形式，把很可能需要变更的配置项放到了values.yaml中,部署时在values.yaml 进行配置即可
helm的结构如下
#tree helm-talkingbot
helm-talkingbot
├── charts
├── Chart.yaml
├── templates
│ ├── configmap.yaml
│ ├── deployment.yaml
│ ├── _helpers.tpl
│ ├── hpa.yaml
│ ├── ingress.yaml
│ ├── NOTES.txt
│ ├── pvc.yaml
│ ├── serviceaccount.yaml
│ ├── service.yaml
│ └── tests
│ └── test-connection.yaml
└── values.yaml
文件太多，打包放这里k8s-helm-talkingbot.tar.gz, helm部署配置在目录helm-talkingbot中
操作记录
helm install my-talkingbot ./helm-talkingbot --set volumePermissions.enabled=true &&kubectl apply -f mytalkingbotpv.yaml
NAME: my-talkingbot
LAST DEPLOYED: Sat May 14 10:17:54 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
talkingbot; can be accessed via port 8086 on the following DNS name from within your cluster:
my-talkingbot-helm-talkingbot.default.svc.cluster.local
persistentvolume/mytalkingbotpv1 created
jialin in DESKTOP-F6Q50AM in TalkingBot/deployment on k8s [$✘!?]
❯ kubectl get pod -n default
NAME READY STATUS RESTARTS AGE
dev212-dd6b559dc-qx7j6 1/1 Running 2 (14d ago) 25d
my-minio-7785f7576f-fzgxr 1/1 Running 0 8d
my-redis-master-0 1/1 Running 1 (14d ago) 18d
my-redis-replicas-0 1/1 Running 0 8d
my-redis-replicas-1 1/1 Running 0 8d
my-redis-replicas-2 1/1 Running 0 8d
my-talkingbot-helm-talkingbot-67d7c8fc95-8k8j6 0/1 ContainerCreating 0 11s
jialin in DESKTOP-F6Q50AM in TalkingBot/deployment on k8s [$✘!?]
❯ kubectl get pod -n default
NAME READY STATUS RESTARTS AGE
dev212-dd6b559dc-qx7j6 1/1 Running 2 (14d ago) 25d
my-minio-7785f7576f-fzgxr 1/1 Running 0 8d
my-redis-master-0 1/1 Running 1 (14d ago) 18d
my-redis-replicas-0 1/1 Running 0 8d
my-redis-replicas-1 1/1 Running 0 8d
my-redis-replicas-2 1/1 Running 0 8d
my-talkingbot-helm-talkingbot-67d7c8fc95-8k8j6 1/1 Running 0 15s
k8s集群环境内可以通过这个my-talkingbot-helm-talkingbot.default.svc.cluster.local:8086地址访问talkingbot的服务
如果要调试可以用我在之前的文章中Dev in K8s Env提过的 运行一个dev pod要访问k8s中的资源
参考资料：
k8s configmap
why helma and what is helm?
k8s serivce
k8s workloads
k8s architecture
k8s load balancer
twelve-factor methodology
12 Factor App meets Kubernetes: Benefits for cloud-native apps