小编给大家分享一下docker中ingress资源和ingress controller的示例分析,相信大部分人都还不怎么了解,因此分享这篇文章给大家参考一下,希望大家阅读完这篇文章后大有收获,下面让我们一起去了解一下吧!
Ingress:就是能利用 Nginx(不常用)、Haproxy(不常用)、Traefik(常用)、Envoy(常用) 啥的负载均衡器暴露集群内服务的工具。
Ingress为您提供七层负载均衡能力,您可以通过 Ingress 配置提供外部可访问的 URL、负载均衡、SSL、基于名称的虚拟主机等。作为集群流量接入层,Ingress 的高可靠性显得尤为重要。
小知识:我们把k8s里面的pod服务发布到集群外部,可以用ingress,也可以用NodePort。
externalLB:外部的负载均衡器
service site:只是用来给pod分组归类的。
[root@master manifests]# kubectl explain ingress
创建名称空间:
[root@master manifests]# kubectl create namespace ingress-nginxnamespace/dev created[root@master manifests]# kubectl get nsNAME STATUS AGEdefault Active 17dingress-nginx Active 8skube-public Active 17dkube-system Active 17d
访问https://github.com/kubernetes/ingress-nginx,进入deploy目录,里面就有我们要用的yaml文件。
各文件的作用:configmap.yaml:提供configmap可以在线更行nginx的配置default-backend.yaml:提供一个缺省的后台错误页面 404namespace.yaml:创建一个独立的命名空间 ingress-nginxrbac.yaml:创建对应的role rolebinding 用于rbactcp-services-configmap.yaml:修改L4负载均衡配置的configmapudp-services-configmap.yaml:修改L4负载均衡配置的configmapwith-rbac.yaml:有应用rbac的nginx-ingress-controller组件
访问https://kubernetes.github.io/ingress-nginx/deploy/#generice-deployment,里面是ingress的部署文档
[root@master ~]# mkdir ingress-nginx[root@master ~]# cd ingress-nginx
部署ingress方法一(分步部署):
下载如下配置文件:
[root@master ingress-nginx]# for file in namespace.yaml configmap.yaml rbac.yaml tcp-services-configmap.yaml with-rbac.yaml udp-services-configmap.yaml; do wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/$file; done
[root@master ingress-nginx]# lsconfigmap.yaml namespace.yaml rbac.yaml tcp-services-configmap.yaml udp-services-configmap.yaml with-rbac.yaml
创建名称空间:
[root@master ingress-nginx]# kubectl apply -f namespace.yaml namespace/ingress-nginx configured
把剩下的ymal文件全应用
[root@master ingress-nginx]# kubectl apply -f ./configmap/nginx-configuration creatednamespace/ingress-nginx configuredserviceaccount/nginx-ingress-serviceaccount createdclusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole createdrole.rbac.authorization.k8s.io/nginx-ingress-role createdrolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding createdclusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding createdconfigmap/tcp-services createdconfigmap/udp-services createddeployment.extensions/nginx-ingress-controller created
[root@master ingress-nginx]# kubectl get pods -n ingress-nginx -wNAME READY STATUS RESTARTS AGEdefault-http-backend-6586bc58b6-qd9fk 0/1 running 0 4mnginx-ingress-controller-6bd7c597cb-zcbbz 0/1 running 0 1m
可以看到ingress-nginx名称空间里面有两个pod都处于running状态
部署ingress方法二(一键部署):
只下载mandatory.yaml文件,因为这个文件里面包含了上面所有yaml文件里面的内容。这是一键部署。
[root@master ingress-nginx]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
[root@master ingress-nginx]#kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
[root@master ~]# kubectl get pods -n ingress-nginx -w -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODEdefault-http-backend-6586bc58b6-qd9fk 1/1 Running 0 11h 10.244.1.95 node1 <none>nginx-ingress-controller-6bd7c597cb-jlqzp 1/1 Running 3 11h 10.244.1.96 node1 <none>
可以看到ingress-nginx名称空间里面有两个pod都处于running状态
安装service-nodeport
上面我们把ingress-nginx部署到了1号node上。接下来我们还需要部署一个service-nodeport服务,才能实现把集群外部流量接入到集群中来。
[root@master ingress]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml
我们为了不让service nodeport自动分配端口,我们自己指定一下nodeport,修改文件中加两个nodePort参数,最终如下:
[root@master ingress]# cat service-nodeport.yaml apiVersion: v1kind: Servicemetadata: name: ingress-nginx namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginxspec: type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30080 - name: https port: 443 targetPort: 443 protocol: TCP nodePort: 30443 selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx
[root@master ingress]# kubectl apply -f service-nodeport.yaml service/ingress-nginx created
[root@master ingress]# kubectl get svc -n ingress-nginxNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEdefault-http-backend ClusterIP 10.110.74.183 <none> 80/TCP 12hingress-nginx NodePort 10.102.78.188 <none> 80:30080/TCP,443:30443/TCP 2m
上面我看到80对应30080,,43对应30443
我们直接通过node1节点的ip就可以访问到应用:
[root@master ingress]# curl http://172.16.1.101:30080default backend - 404
定义myapp service
[root@master manifests]# mkdir /root/manifests/ingress
[root@master ~]# kubectl explain service.spec.ports
[root@master ingress]# cat deploy-demo.yaml apiVersion: v1kind: Service#必须设置为无头servicemetadata: name: myapp namespace: defaultspec: selector: app: myapp release: canary ports: - name: http targetPort: 80 #这是容器port port: 80 #这是service port---apiVersion: apps/v1kind: Deploymentmetadata: name: myapp-deploy namespace: defaultspec: replicas: 2 selector: #标签选择器 matchLabels: #匹配的标签为 app: myapp release: canary template: metadata: labels: app: myapp #和上面的myapp要匹配 release: canary spec: containers: - name: myapp image: ikubernetes/myapp:v1 ports: - name: http containerPort: 80
[root@master ingress]# kubectl apply -f deploy-demo.yaml service/myapp createddeployment.apps/myapp-deploy unchanged
[root@master ingress]# kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEkubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17dmyapp ClusterIP 10.108.177.62 <none> 80/TCP 1m
[root@master ingress]# kubectl get podsNAME READY STATUS RESTARTS AGEmyapp-deploy-69b47bc96d-79fqh 1/1 Running 0 1dmyapp-deploy-69b47bc96d-tc54k 1/1 Running 0 1d
把myapp service通过ingress发布出去
下面我们再定义一个清单文件,把myapp应用通过Ingress(相当于nginx的反向代理功能)发布出去:
[root@master ingress]# cat ingress-myapp.yaml apiVersion: extensions/v1beta1 kind: Ingressmetadata: name: ingress-myapp namespace: default #要和deployment和要发布的service处于同一个名称空间 annotations: #这个注解说明我们要用到的ingress-controller是nginx,而不是traefic,enjoy kubernetes.io/ingress.class: "nginx"spec: rules: - host: myapp.zhixin.com #表示访问这个域名,就会转发到后端myapp管理的pod上的服务: http: paths: - path: backend: serviceName: myapp servicePort: 80
[root@master ingress]# kubectl apply -f ingress-myapp.yaml ingress.extensions/ingress-myapp created
[root@master ingress]# kubectl get ingressNAME HOSTS ADDRESS PORTS AGEingress-myapp myapp.zhixin.com 80 8m
[root@master ingress]# kubectl describe ingress
[root@master ingress]# kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGEdefault-http-backend-6586bc58b6-qd9fk 1/1 Running 0 12hnginx-ingress-controller-6bd7c597cb-jlqzp 1/1 Running 3 12h
进入ingress-controller交互式命令行里面,可以清晰的看到nginx是怎么反向代理我们myapp.zhixin.com的:
[root@master ingress]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-6bd7c597cb-jlqzp -- /bin/sh$ cat nginx.conf## start server myapp.zhixin.comserver {server_name myapp.zhixin.com ;listen 80;set $proxy_upstream_name "-";location / {set $namespace "default";set $ingress_name "ingress-myapp";set $service_name "myapp";set $service_port "80";........
测试,下面我们把myapp.zhixin.com域名解析到node1 ip 172.16.1.101上。
[root@master ingress]# curl myapp.zhixin.com:30080Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
把tomcat service通过ingress发布出去(新例子)
[root@master ingress]# cat tomcat-demo.yaml apiVersion: v1kind: Service#必须设置为无头servicemetadata: name: tomcat namespace: defaultspec: selector: app: tomcat release: canary ports: - name: http targetPort: 8080 #这是容器port port: 8080 #这是service port - name: ajp targetPort: 8009 port: 8009---apiVersion: apps/v1kind: Deploymentmetadata: name: tomcat-deploy namespace: defaultspec: replicas: 2 selector: #标签选择器 matchLabels: #匹配的标签为 app: tomcat release: canary template: metadata: labels: app: tomcat #和上面的myapp要匹配 release: canary spec: containers: - name: tomcat image: tomcat:8.5.34-jre8-alpine #在https://hub.docker.com/r/library/tomcat/tags/上面找 ports: - name: http containerPort: 8080 - name: ajp containerPort: 8009
[root@master ingress]# kubectl apply -f tomcat-demo.yaml service/tomcat createddeployment.apps/tomcat-deploy created
[root@master ingress]# kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEkubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17dtomcat ClusterIP 10.109.76.87 <none> 8080/TCP,8009/TCP 1m
[root@master ingress]# kubectl get podsNAME READY STATUS RESTARTS AGEtomcat-deploy-64c4d54df4-68sk8 1/1 Running 0 51stomcat-deploy-64c4d54df4-7b58g 1/1 Running 0 51s
[root@master ingress]# cat ingress-tomcat.yaml apiVersion: extensions/v1beta1 kind: Ingressmetadata: name: ingress-tomcat namespace: default #要和deployment和要发布的service处于同一个名称空间 annotations: #这个注解说明我们要用到的ingress-controller是nginx,而不是traefic,enjoy kubernetes.io/ingress.class: "nginx"spec: rules: - host: tomcat.zhixin.com #表示访问这个域名,就会转发到后端myapp管理的pod上的服务: http: paths: - path: backend: serviceName: tomcat servicePort: 8080
[root@master ingress]# kubectl apply -f ingress-tomcat.yaml ingress.extensions/ingress-myapp configured
[root@master ingress]# kubectl get ingressNAME HOSTS ADDRESS PORTS AGEingress-tomcat tomcat.zhixin.com 80 11s
[root@master ingress]# kubectl describe ingress ingress-tomcatName: ingress-tomcatNamespace: defaultAddress: Default backend: default-http-backend:80 (<none>)Rules: Host Path Backends ---- ---- -------- tomcat.zhixin.com tomcat:8080 (<none>)Annotations: kubernetes.io/ingress.class: nginxEvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 1m nginx-ingress-controller Ingress default/ingress-tomcat
把tomcat.zhixin.com解析到node1上节点物理ip(我的是172.16.1.101)
测试,可以看到tomcat欢迎界面:
[root@master ingress]# curl tomcat.zhixin.com:30080
使用https访问(新例子)
先做个自签的证书(我们这里不演示CA的例子)
[root@master ingress]# openssl genrsa -out tls.key 2048
[root@master ingress]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/O=DevOps/CN=tomcat.zhixin.com
通过secret把证书注入到pod中。
[root@master ingress]# kubectl create secret tls tomcat-infress-secret --cert=tls.crt --key=tls.key secret/tomcat-infress-secret created
[root@master ingress]# kubectl get secretNAME TYPE DATA AGEdefault-token-5r85r kubernetes.io/service-account-token 3 17dtomcat-ingress-secret kubernetes.io/tls 2 41s
[root@master ingress]# kubectl describe secret tomcat-ingress-secret Name: tomcat-ingress-secretNamespace: defaultLabels: <none>Annotations: <none>Type: kubernetes.io/tlsData====tls.crt: 1245 bytestls.key: 1679 bytes
配置ingress为tls方式
[root@master ingress]# cat ingress-tomcat-tls.yaml apiVersion: extensions/v1beta1 kind: Ingressmetadata: name: ingress-tomcat-tls namespace: default #要和deployment和要发布的service处于同一个名称空间 annotations: #这个注解说明我们要用到的ingress-controller是nginx,而不是traefic,enjoy kubernetes.io/ingress.class: "nginx"spec: tls: - hosts: - tomcat.zhixin.com secretName: tomcat-ingress-secret #kubectl get secret命令查到的名字 rules: - host: tomcat.zhixin.com #表示访问这个域名,就会转发到后端myapp管理的pod上的服务: http: paths: - path: backend: serviceName: tomcat servicePort: 8080
[root@master ingress]# kubectl get ingressNAME HOSTS ADDRESS PORTS AGEingress-tomcat tomcat.zhixin.com 80 2hingress-tomcat-tls tomcat.zhixin.com 80, 443 3m
[root@master ingress]# kubectl describe ingress ingress-tomcat-tls Name: ingress-tomcat-tlsNamespace: defaultAddress: Default backend: default-http-backend:80 (<none>)TLS: tomcat-ingress-secret terminates tomcat.zhixin.comRules: Host Path Backends ---- ---- -------- tomcat.zhixin.com tomcat:8080 (<none>)Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-tomcat-tls","namespace":"default"},"spec":{"rules":[{"host":"tomcat.zhixin.com","http":{"paths":[{"backend":{"serviceName":"tomcat","servicePort":8080},"path":null}]}}],"tls":[{"hosts":["tomcat.zhixin.com"],"secretName":"tomcat-ingress-secret"}]}} kubernetes.io/ingress.class: nginxEvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 4m nginx-ingress-controller Ingress default/ingress-tomcat-tls
连如ingress-controller查看nginx.conf的配置
[root@master ingress]# kubectl get pods -n ingress-nginxNAME READY STATUS RESTARTS AGEdefault-http-backend-6586bc58b6-qd9fk 1/1 Running 0 16hnginx-ingress-controller-6bd7c597cb-jlqzp 1/1 Running 3 16h
[root@master ingress]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-6bd7c597cb-jlqzp -- /bin/sh$ $ cat nginx.conf## start server tomcat.zhixin.comserver {server_name tomcat.zhixin.com ;listen 80;set $proxy_upstream_name "-";listen 443 ssl http2;
看到有listen 443了。
测试https
[root@master ingress]# curl https://tomcat.zhixin.com:30443
部署方法三(利用ingress 的80端口)
前面两种部署方法,是用node ip + 非80端口,访问k8s集群内部的服务。可是,我们实际生产中更希望的是node ip + 80端口的方式,访问k8s集群内的服务。我感觉这个方法最好,下面就就介绍这个方法。
这部分内容参考的是博文http://blog.51cto.com/devingeng/2149377
下载地址
https://github.com/kubernetes/ingress-nginx/archive/nginx-0.11.0.tar.gz
ingress-nginx文件位于deploy目录下,各文件的作用:configmap.yaml:提供configmap可以在线更行nginx的配置default-backend.yaml:提供一个缺省的后台错误页面 404namespace.yaml:创建一个独立的命名空间 ingress-nginxrbac.yaml:创建对应的role rolebinding 用于rbactcp-services-configmap.yaml:修改L4负载均衡配置的configmapudp-services-configmap.yaml:修改L4负载均衡配置的configmapwith-rbac.yaml:有应用rbac的nginx-ingress-controller组件
修改with-rbac.yaml
apiVersion: extensions/v1beta1kind: Daemonsetmetadata: name: nginx-ingress-controller namespace: ingress-nginx spec: selector: matchLabels: app: ingress-nginx template: metadata: labels: app: ingress-nginx annotations: prometheus.io/port: '10254' prometheus.io/scrape: 'true' spec: serviceAccountName: nginx-ingress-serviceaccount hostNetwork: true containers: - name: nginx-ingress-controller image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.11.0 args: - /nginx-ingress-controller - --default-backend-service=$(POD_NAMESPACE)/default-http-backend - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --annotations-prefix=nginx.ingress.kubernetes.io env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - name: http containerPort: 80 - name: https containerPort: 443 livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 nodeSelector: custom/ingress-controller-ready: "true"
需要修改的地方:
kind: DaemonSet:官方原始文件使用的是deployment,replicate 为 1,这样将会在某一台节点上启动对应的nginx-ingress-controller pod。外部流量访问至该节点,由该节点负载分担至内部的service。测试环境考虑防止单点故障,改为DaemonSet然后删掉replicate ,配合亲和性部署在制定节点上启动nginx-ingress-controller pod,确保有多个节点启动nginx-ingress-controller pod,后续将这些节点加入到外部硬件负载均衡组实现高可用性。
hostNetwork: true:添加该字段,暴露nginx-ingress-controller pod的服务端口(80)
nodeSelector: 增加亲和性部署,有custom/ingress-controller-ready 标签的节点才会部署该DaemonSet
为需要部署nginx-ingress-controller的节点设置lable
kubectl label nodes vmnode2 custom/ingress-controller-ready=truekubectl label nodes vmnode3 custom/ingress-controller-ready=truekubectl label nodes vmnode4 custom/ingress-controller-ready=true
加载yaml文件
kubectl apply -f namespace.yamlkubectl apply -f default-backend.yamlkubectl apply -f configmap.yamlkubectl apply -f tcp-services-configmap.yamlkubectl apply -f udp-services-configmap.yamlkubectl apply -f rbac.yamlkubectl apply -f with-rbac.yaml
查看pod是否正常创建
##下载镜像可能会比较慢,等待一会所有pod都是Running状态,按Ctrl + c 退出
[root@vmnode1 deploy]# kubectl get pods --namespace=ingress-nginx --watchNAME READY STATUS RESTARTS AGEdefault-http-backend-6c59748b9b-hc8q9 1/1 Running 0 6mnginx-ingress-controller-7fmlp 1/1 Running 1 13dnginx-ingress-controller-j95fb 1/1 Running 1 13dnginx-ingress-controller-ld2jw 1/1 Running 1 13d
测试ingress
创建一个tomcat的Service
[root@k8s-master1 test]# cat mytomcat.yaml apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: mytomcatspec: replicas: 2 template: metadata: labels: run: mytomcat spec: containers: - name: mytomcat image: tomcat ports: - containerPort: 8080---apiVersion: v1kind: Servicemetadata: name: mytomcat labels: run: mytomcatspec: type: NodePort ports: - port: 8080 targetPort: 8080 selector: run: mytomcat
[root@k8s-master1 test]# kubectl apply -f mytomcat.yaml
配置ingress转发文件:
[root@k8s-master1 test]# cat test-ingress.yaml apiVersion: extensions/v1beta1kind: Ingressmetadata: name: test-ingress namespace: defaultspec: rules: - host: test.zhixin.com http: paths: - path: / backend: serviceName: mytomcat servicePort: 8080
host: 对应的域名
path: url上下文
backend:后向转发 到对应的 serviceName: servicePort:
[root@k8s-master1 test]# kubectl apply -f test-ingress.yaml ingress.extensions/test-ingress created
nginx-ingress-controller运行在node1和nod2两个节点上。如果网络中有dns服务器,在dns中把这两个域名映射到nginx-ingress-controller运行的任意一个节点上,如果没有dns服务器只能修改host文件了。
正规的做法是在node1和node2这两个节点上安装keepalive,生成一个vip。在dns上把域名和vip做映射。
我这里直接在node1节点上操作了:
我这里node1节点的ip是172.16.22.201;node2节点的ip是172.16.22.202
[root@k8s-master1 test]# echo " 172.16.22.201 test.zhixin.com" >> /etc/hosts[root@k8s-master1 test]# echo "172.16.22.202 test.zhixin.com" >> /etc/hosts
然后访问测试:
看到,我们把域名test.zhixin.com绑定到Node节点的ip补上,然后我们直接访问http://test.zhixin.com,就能访问到k8s集群里面的pod服务。
以上是“docker中ingress资源和ingress controller的示例分析”这篇文章的所有内容,感谢各位的阅读!相信大家都有了一定的了解,希望分享的内容对大家有所帮助,如果还想学习更多知识,欢迎关注编程网行业资讯频道!