Kubernetes sur K5 – Simplifié

2017-05-26

Machine-translated from English. Read the English original

Voilà presque 2 semaines que je suis de retour du Boston OpenStack Summit – quelle semaine formidable pour un geek. Manger, boire et dormir de la technologie… J’adore. C’était un plaisir de rencontrer tant de geeks passionnés et de découvrir ce que les clients explorent dans leurs différents parcours cloud. Si vous souhaitez revoir ma présentation sur le Fujitsu Cloud Service K5, vous pouvez la consulter sur YouTube ici – (je dois absolument perdre quelques livres)

Contenu intégré

Un thème récurrent de ce sommet OpenStack était la manière dont Kubernetes et OpenStack sont de bons compagnons, tout comme la confiture de fraises et la crème – qui veut un thé à la crème… l’été approche.

Cependant, le grand débat britannique sur le thé à la crème se poursuit…

straeberriesandcream

Certaines personnes souhaitent utiliser Kubernetes pour déployer OpenStack en tant qu’application ‘kube’, tandis que d’autres voient Kubernetes comme la couche d’orchestration de conteneurs au-dessus d’OpenStack. Je vois des mérites dans les deux approches, mais je resterai neutre jusqu’à ce que la communauté décide s’il y aura un gagnant définitif. Dans tous les cas, il s’agira de solutions complexes à déboguer en cas de problème – à mon avis, celui qui disposera des outils d’auto-guérison les plus simples remportera cette course.

Infrastructure as Code – Templates OpenStack Heat pour déployer Kubernetes

Dans l’exemple suivant, j’ai créé une pile Heat qui déploie Kubernetes au-dessus de l’IaaS Fujitsu Cloud Service K5. La pile Heat provisionne l’infrastructure prérequis dans K5, puis l’un des serveurs déployés utilise le Projet Kubernetes Kargo qui utilise un playbook ansible pour installer et configurer la couche PaaS Kubernetes.

Je vous propose le choix entre 2 templates :

Déploiement simple de 3+1 serveurs, non-HA. Adapté pour évaluer Kubernetes et déployer des applications.

Ce fichier contient des caractères Unicode cachés ou bidirectionnels qui peuvent être interprétés ou compilés différemment de ce qui est affiché ci-dessous. Pour les examiner, ouvrez le fichier dans un éditeur qui révèle les caractères Unicode cachés.
En savoir plus sur les caractères Unicode bidirectionnels

Afficher les caractères cachés


	heat_template_version: 2013-05-23
	# Author: Graham Land
	# Date: 26/05/2017
	# Purpose: Deploy Kubernetes on Fujitsu's K5 IaaS Platform
	#
	# Twitter: @allthingsclowd
	# Blog: https://allthingscloud.eu
	# Gist: https://gist.github.com/allthingsclowd/4814fb3895ef5842befbb90cdf93244e
	#

	description: Simple Kubernetes Non HA Deployment

	# Input parameters
	parameters:
	k5_image:
	type: string
	label: Image name or ID
	description: Image to be used for compute instance
	default: "CentOS 7.2 64bit (English) 02"
	flavor:
	type: string
	label: Flavor
	description: Type of instance (flavor) to be used
	default: "S-1"
	key_az_1:
	type: string
	label: Key name
	description: Name of key-pair to be used for compute instance
	default: "KUBE-KP-AZ1"
	public_net_az1:
	type: string
	label: external network ID
	description: Public network
	default: "73598426-3678-4eb6-8131-9007770d3ccd"
	az1:
	type: string
	label: Availability Zone
	description: Region AZ to use
	default: "de-1a"
	az1_dns:
	type: string
	label: Availability Zone DNS Servers
	description: Availability Zone DNS Servers to use
	default: ["185.149.225.9", "185.149.225.10"]
	my_ip:
	type: string
	label: External management IP
	description: IP address allowed for access to jump server on SSH
	default: "165.120.203.180/32"
	kube_router:
	type: string
	label: Kubernetes Router Used to Provide External Access to VMs
	description: Kubernetes Router Used to Provide External Access to VMs
	default: "020d1ecc-0fd5-40ec-aa69-cc49e7eb98b1"
	default_security_group:
	type: string
	label: Kubernetes Router Used to Provide External Access to VMs
	description: Kubernetes Router Used to Provide External Access to VMs
	default: "0b816413-cba5-4163-b809-73faf8d598c7"
	kube_service_network:
	type: string
	label: Kube services network
	description: Kubernetes Services Network
	default: "10.233.0.0/18"
	kube_pod_network:
	type: string
	label: Kube pods network
	description: Kubernetes Pods Network
	default: "10.233.64.0/18"

	# K5 Infrastructure resources to be built
	resources:

	# Create a private network in availability zone 1
	kube_private_net_az1:
	type: OS::Neutron::Net
	properties:
	availability_zone: { get_param: az1 }
	name: "kube_private_net1"

	# Create a new subnet on the private network
	kube_private_subnet_az1:
	type: OS::Neutron::Subnet
	depends_on: kube_private_net_az1
	properties:
	availability_zone: { get_param: az1 }
	name: "Kube_private_subnet_az1"
	network_id: { get_resource: kube_private_net_az1 }
	cidr: "192.168.200.0/24"
	gateway_ip: "192.168.200.254"
	allocation_pools:
	– start: "192.168.200.100"
	end: "192.168.200.150"
	dns_nameservers: ["185.149.225.9", "185.149.225.10"]

	# Connect an interface on the web tier network's subnet to the router
	az1_router_interface:
	type: OS::Neutron::RouterInterface
	depends_on: [kube_private_subnet_az1]
	properties:
	router_id: { get_param: kube_router }
	subnet_id: { get_resource: kube_private_subnet_az1 }

	# Create jump server security group
	jump_security_group:
	type: OS::Neutron::SecurityGroup
	properties:
	description: kube stack jump server security group
	name: kube-jumpbox
	rules:
	# allow inbound ssh from my ip
	– remote_ip_prefix: { get_param: my_ip }
	protocol: tcp
	port_range_min: 22
	port_range_max: 22
	– remote_ip_prefix: { get_param: my_ip }
	protocol: icmp

	# Create lbaas server security group
	app_frontend_security_group:
	type: OS::Neutron::SecurityGroup
	properties:
	description: kube stack network connector security group
	name: sock-shop-frontend
	rules:
	# allow http traffic to etcd sock-shop service
	– remote_ip_prefix: "0.0.0.0/0"
	protocol: tcp
	port_range_min: 30001
	port_range_max: 30001


	################### JumpBox – AZ 1 ######################################################################

	# Create a new port for the server interface, assign an ip address and security group
	jump-server-port:
	type: OS::Neutron::Port
	depends_on: [ jump_security_group ]
	properties:
	availability_zone: { get_param: az1 }
	network_id: { get_resource: kube_private_net_az1 }
	security_groups: [{ get_resource: jump_security_group }, { get_param: default_security_group }]

	# Allocate a floating/global ip address
	jump-server-floating-ip:
	type: OS::Neutron::FloatingIP
	properties:
	availability_zone: { get_param: az1 }
	floating_network_id: { get_param: public_net_az1 }

	# Assign a floating/global ip address to the fixed server ip address
	jump-server-floating-ip-association:
	type: OS::Neutron::FloatingIPAssociation
	depends_on: jump-server-floating-ip
	properties:
	floatingip_id: { get_resource: jump-server-floating-ip }
	port_id: { get_resource: jump-server-port }

	# Create a system volume for use with the server
	jump-sys-vol:
	type: OS::Cinder::Volume
	properties:
	availability_zone: { get_param: az1 }
	name: "jump-box-vol"
	size: 30
	volume_type: "M1"
	image : { get_param: k5_image }

	# Build a server using the system volume defined above
	jump-server:
	type: OS::Nova::Server
	depends_on: [ etcd1-server, worker1-server, worker2-server, jump-server-port ]
	properties:
	key_name: { get_param: key_az_1 }
	image: { get_param: k5_image }
	flavor: { get_param: flavor }
	admin_user: k5user
	metadata: { "fcx.autofailover": True }
	block_device_mapping: [{"volume_size": "30", "volume_id": {get_resource: jump-sys-vol}, "delete_on_termination": True, "device_name": "/dev/vda"}]
	# Create private key file to enable SSH to remaining servers in the environment
	user_data_format: RAW
	user_data:
	str_replace:
	template:
	#cloud-config
	write_files:
	– path: /root/.ssh/id_rsa
	content:
	—–BEGIN RSA PRIVATE KEY—–
	MIIEoQIBAAKCAQEA4MBSCRr6XOdLBsVSF3x3X/Y+jSoCP4walw5x/8A9g+kF9vuC
	lBU32TlQD/k1B4mxODhxTi1pzmw7Se/ccu3dlGivRgKuQzplVMeCJKAd15MxTr8i
	9ZKlwEj02xxBIzl58pVlHGIGVLwzxyfKtzOSFkEVdUYmJUvifoTYM5R5aWPI2Rhh
	hvkxtmz+WHC67dePcrzAaE0KizRWF4xRP/ZFUXUwcB/Z2fbF5RPocARBjbZgKDb4
	ufC0v9eKdn6mDQDu/Vq7VY8058efitNcfqxhx5d+t0EpwZCha6J7/3KJa9rXABzH
	WrVt1Tk/1F/Z8h/VNyo0k1KLMlBlsHZH5BSMgQIBIwKCAQBmvlFjP4hk/AUKaNUS
	DQNfEXuuPxb4eo/QBpqDixTOlmkgcvndS4c+ux1JIXdT5yxxeONW8DBeXVzumYlZ
	HEgJVG1h1Vb6KVLk7X1SklbI8tS2SL+GNGkHcc8FFDsIydFY89ZzYALk6FIvKCIn
	3Q+VJREfqwofraleaJ1ZaHIESgPzzPHoVc9//51rtqc2pjmmCu3BsJkR8isAiiyL
	LiegC9FIcXXGcgeMWqFBPqHUELcZXTaUYIu/Ie8FBvDXiZMWOQFbemQJczF8BS3j
	YFoGXoY6c8y052ekh8FGziJgy35gHf3ygUMnaVLirJpG0i3SH3M0ZHkVeP8fcw5x
	uFFLAoGBAPqx+ubMIM/tH4X+zUD7mgaX8+EByd8IUKLwVTXGS7XzuAnkVTBr58bC
	AOCtKX9oLcR40J3Y5N0rzZRmzs7+YNXnK2VQTYbZFPWfD6giKuy+JFc+5Ej+Men/
	dOviiF74V8dsHtSwjX/CStzLYdEy6+xyA3Q0USPeVAQKvXDlOe27AoGBAOWBzSmO
	nKvhTk/+A+wqseq/kPdO0NyrofUWCNoJNZivo0y9UnfL9Y6gDMcEIcAlpAsn6nN8
	eKXh77s0sKg3IydVkqMNJfM6Bq2seF/7HrAbhGHYf5D3pIS0RJlSBz8/xeshvFcn
	rSsst/25JHdESCSJgfRcZZ51A7OQ7SdmF2zzAoGBAKS+Gegfvcp3QJnh9Jhq0u5j
	2scPzc0UF7uH/Xsb6Jwj0LYLE2j2c70gZvoLZGmq3D9IFA/07irw5i5SMCGf2Twb
	kYu4bXXfHGbk3mcsZVm+x2yIaiFWldRIyS1LuLNv/ytHDPIq4KRpvCqxiWw3Z9Xk
	he1G8334gFptdSzuZ+VdAoGAVT7XLLFQIpWDfMwQFeP48MrPcczRPAU8J9T7+Tae
	4PDHoCkQAJw2orB5xkNc/jnd2EIGn+xnU45u/GQHGeiYB0urfmP4JyQuXcO3rp8a
	BuWmMvikaQuGP+svbCXILW91ZfaWaYPD+hfskXCnJPway74Lsoi4B6fOLMDNHUMt
	RbkCgYAi4KlL+fT5xJ9X3EbYxQ4P2TAEUAwKh7yvpoFVH/OuKZedIi2sHNOkb895
	qlcmVtbD96T0m4sZ2UVk0oDURsjp7tK8pIdCqXkxx+Y/cSEc+1R/3J0GcUHt2cbc
	YmDPfr1iqizT0n9em0B51SQb3UH88Egf56GHXiVaEZYc7/o0DQ==
	—–END RSA PRIVATE KEY—–
	– path: /home/k5user/kubernetes.cfg
	content:
	[etcd]
	$etcd1_az1_ip

	[kube-master]
	$etcd1_az1_ip

	[kube-node]
	$worker1_az1_ip
	$worker2_az1_ip

	[k8s-cluster:children]
	kube-node
	kube-master
	runcmd:
	– chmod 0600 /root/.ssh/id_rsa
	– yum makecache fast
	– yum install -y git
	– yum install -y epel-release
	– yum install -y ansible
	– yum install -y python-netaddr
	– yum install -y python-pip
	– pip install –upgrade pip
	– pip install –upgrade jinja2
	– mkdir -p /home/k5user/kubernetes/kargo
	– git clone https://github.com/kubernetes-incubator/kargo.git /home/k5user/kubernetes/kargo
	– cd /home/k5user/kubernetes/kargo
	– git reset –hard 9e64267
	– cp /home/k5user/kubernetes.cfg /home/k5user/kubernetes/kargo/inventory/inventory.cfg
	– ansible-playbook -u k5user -b -i inventory/inventory.cfg cluster.yml
	– ssh k5user@$etcd1_az1_ip 'kubectl create -f "https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml?raw=true"'
	– ssh k5user@$etcd1_az1_ip 'kubectl version && kubectl get nodes && kubectl get pods –all-namespaces && sudo docker ps'

	params:
	$etcd1_az1_ip: { get_attr: [ etcd1-server, first_address ] }
	$worker1_az1_ip: { get_attr: [ worker1-server, first_address ] }
	$worker2_az1_ip: { get_attr: [ worker2-server, first_address ] }
	message: "Installation Complete"
	name: "kube-jumpbox-az1"
	networks:
	– port: { get_resource: jump-server-port }

	##########################################################################################################

	################### Kube etcd master 1 – Net 1 – AZ 1 ##############################################################################

	# Create a new port for the server interface, assign an ip address and security group
	etcd1-server-port:
	type: OS::Neutron::Port
	properties:
	availability_zone: { get_param: az1 }
	network_id: { get_resource: kube_private_net_az1 }
	security_groups: [{ get_resource: app_frontend_security_group }, { get_param: default_security_group }]
	allowed_address_pairs:
	– ip_address: { get_param: kube_service_network }
	– ip_address: { get_param: kube_pod_network }

	# Allocate a floating/global ip address
	etcd1-server-floating-ip:
	type: OS::Neutron::FloatingIP
	properties:
	availability_zone: { get_param: az1 }
	floating_network_id: { get_param: public_net_az1 }

	# Assign a floating/global ip address to the fixed server ip address
	etcd1-server-floating-ip-association:
	type: OS::Neutron::FloatingIPAssociation
	depends_on: jump-server-floating-ip
	properties:
	floatingip_id: { get_resource: etcd1-server-floating-ip }
	port_id: { get_resource: etcd1-server-port }

	# Create a system volume for use with the server
	etcd1-sys-vol:
	type: OS::Cinder::Volume
	properties:
	availability_zone: { get_param: az1 }
	name: "etcd1-boot-vol"
	size: 30
	volume_type: "M1"
	image : { get_param: k5_image }

	# Build a server using the system volume defined above
	etcd1-server:
	type: OS::Nova::Server
	# depends_on: [ jump-server,etcd3-server,etcd1-server-port,etcd1-sys-vol ]
	properties:
	key_name: { get_param: key_az_1 }
	image: { get_param: k5_image }
	flavor: { get_param: flavor }
	admin_user: k5user
	metadata: { "fcx.autofailover": True }
	block_device_mapping: [{"volume_size": "30", "volume_id": {get_resource: etcd1-sys-vol}, "delete_on_termination": True, "device_name": "/dev/vda"}]
	name: "etcd1_az1"
	networks:
	– port: { get_resource: etcd1-server-port }

	###########################################################################################################

	################### worker 1 – Net 1 – AZ 1 ##############################################################################

	# Create a new port for the server interface, assign an ip address and security group
	worker1-server-port:
	type: OS::Neutron::Port
	properties:
	availability_zone: { get_param: az1 }
	network_id: { get_resource: kube_private_net_az1 }
	# security_groups: [{ get_resource: web_security_group }]
	allowed_address_pairs:
	– ip_address: { get_param: kube_service_network }
	– ip_address: { get_param: kube_pod_network }

	# Create a system volume for use with the server
	worker1-sys-vol:
	type: OS::Cinder::Volume
	properties:
	availability_zone: { get_param: az1 }
	name: "worker1-boot-vol"
	size: 30
	volume_type: "M1"
	image : { get_param: k5_image }

	# Build a server using the system volume defined above
	worker1-server:
	type: OS::Nova::Server
	# depends_on: [ jump-server ]
	properties:
	key_name: { get_param: key_az_1 }
	image: { get_param: k5_image }
	flavor: { get_param: flavor }
	admin_user: k5user
	metadata: { "fcx.autofailover": True }
	block_device_mapping: [{"volume_size": "30", "volume_id": {get_resource: worker1-sys-vol}, "delete_on_termination": True, "device_name": "/dev/vda"}]
	name: "worker1_az1"
	networks:
	– port: { get_resource: worker1-server-port }

	###########################################################################################################

	################### worker 2 – Net 1 – AZ 1 ##############################################################################

	# Create a new port for the server interface, assign an ip address and security group
	worker2-server-port:
	type: OS::Neutron::Port
	properties:
	availability_zone: { get_param: az1 }
	network_id: { get_resource: kube_private_net_az1 }
	# security_groups: [{ get_resource: web_security_group }]
	allowed_address_pairs:
	– ip_address: { get_param: kube_service_network }
	– ip_address: { get_param: kube_pod_network }

	# Create a system volume for use with the server
	worker2-sys-vol:
	type: OS::Cinder::Volume
	properties:
	availability_zone: { get_param: az1 }
	name: "worker2-boot-vol"
	size: 30
	volume_type: "M1"
	image : { get_param: k5_image }

	# Build a server using the system volume defined above
	worker2-server:
	type: OS::Nova::Server
	# depends_on: [ jump-server ]
	properties:
	key_name: { get_param: key_az_1 }
	image: { get_param: k5_image }
	flavor: { get_param: flavor }
	admin_user: k5user
	metadata: { "fcx.autofailover": True }
	block_device_mapping: [{"volume_size": "30", "volume_id": {get_resource: worker2-sys-vol}, "delete_on_termination": True, "device_name": "/dev/vda"}]
	name: "worker2_az1"
	networks:
	– port: { get_resource: worker2-server-port }

	###########################################################################################################

voir le code brut
K5-Kubernetes-Non-HA.yml
hébergé avec ❤ par GitHub

Déploiement HA haute disponibilité, multi-AZ K5 de 6+1 serveurs. Adapté pour évaluer un Kubernetes avec etcd en cluster et déployer des applications.

Afficher les caractères cachés


	heat_template_version: 2013-05-23
	# Author: Graham Land
	# Date: 26/05/2017
	# Purpose: Deploy Kubernetes on Fujitsu's K5 IaaS Platform
	#
	# Twitter: @allthingsclowd
	# Blog: https://allthingscloud.eu
	#
	#

	description: Highly Available (multiple etcd & minions across AZs) Kubernetes Cluster

	# Input parameters
	parameters:
	k5_image:
	type: string
	label: Image name or ID
	description: Image to be used for compute instance
	default: "CentOS 7.2 64bit (English) 02"
	flavor:
	type: string
	label: Flavor
	description: Type of instance (flavor) to be used
	default: "S-1"
	key_az_1:
	type: string
	label: Key name
	description: Name of key-pair to be used for compute instance
	default: "KUBE-KP-AZ1"
	key_az_2:
	type: string
	label: Key name
	description: Name of key-pair to be used for compute instance
	default: "KUBE-KP-AZ2"
	public_net_az1:
	type: string
	label: external network ID
	description: Public network
	default: "73598426-3678-4eb6-8131-9007770d3ccd"
	public_net_az2:
	type: string
	label: external network ID
	description: Public network
	default: "58e863e5-7d31-4f32-9178-370a3288db42"
	az1:
	type: string
	label: Availability Zone
	description: Region AZ to use
	default: "de-1a"
	az2:
	type: string
	label: Availability Zone
	description: Region AZ to use
	default: "de-1b"
	my_ip:
	type: string
	label: External management IP
	description: IP address allowed for access to jump server on SSH
	default: "86.187.115.18/32"
	kube_router:
	type: string
	label: Kubernetes Router Used to Provide External Access to VMs
	description: Kubernetes Router Used to Provide External Access to VMs
	default: "020d1ecc-0fd5-40ec-aa69-cc49e7eb98b1"
	kube_router_az2:
	type: string
	label: Kubernetes Router Used to Provide External Access to VMs
	description: Kubernetes Router Used to Provide External Access to VMs
	default: "b48771b7-3fcc-4da1-b925-d3cbfe641381"
	default_security_group:
	type: string
	label: Kubernetes Router Used to Provide External Access to VMs
	description: Kubernetes Router Used to Provide External Access to VMs
	default: "0b816413-cba5-4163-b809-73faf8d598c7"
	kube_service_network:
	type: string
	label: Kube services network
	description: Kubernetes Services Network
	default: "10.233.0.0/18"
	kube_pod_network:
	type: string
	label: Kube pods network
	description: Kubernetes Pods Network
	default: "10.233.64.0/18"

	# K5 Infrastructure resources to be built
	resources:

	# Create a private network in availability zone 1
	kube_private_net_az1:
	type: OS::Neutron::Net
	properties:
	availability_zone: { get_param: az1 }
	name: "kube_private_net1"

	# Create a new subnet on the private network
	kube_private_subnet_az1:
	type: OS::Neutron::Subnet
	depends_on: kube_private_net_az1
	properties:
	availability_zone: { get_param: az1 }
	name: "Kube_private_subnet_az1"
	network_id: { get_resource: kube_private_net_az1 }
	cidr: "192.168.200.0/24"
	gateway_ip: "192.168.200.254"
	allocation_pools:
	– start: "192.168.200.100"
	end: "192.168.200.150"
	dns_nameservers: ["185.149.225.9", "185.149.225.10"]
	host_routes: [{"nexthop": "192.168.200.253", "destination": "10.11.200.0/24"}]

	# Connect an interface on the web tier network's subnet to the router
	az1_router_interface:
	type: OS::Neutron::RouterInterface
	depends_on: [kube_private_subnet_az1]
	properties:
	router_id: { get_param: kube_router }
	subnet_id: { get_resource: kube_private_subnet_az1 }

	# Create another private network in availability zone 1
	kube_private_net_az2:
	type: OS::Neutron::Net
	properties:
	availability_zone: { get_param: az2 }
	name: "kube_private_net2"

	# Create a new subnet on the private network
	kube_private_subnet_az2:
	type: OS::Neutron::Subnet
	depends_on: kube_private_net_az2
	properties:
	availability_zone: { get_param: az2 }
	name: "kube_private_subnet_az2"
	network_id: { get_resource: kube_private_net_az2 }
	cidr: "10.11.200.0/24"
	gateway_ip: "10.11.200.254"
	allocation_pools:
	– start: "10.11.200.100"
	end: "10.11.200.150"
	dns_nameservers: ["185.149.227.9", "185.149.227.10"]
	host_routes: [{"nexthop": "10.11.200.253", "destination": "192.168.200.0/24"}]


	# Connect an interface on the web tier network's subnet to the router
	az2_router_interface:
	type: OS::Neutron::RouterInterface
	depends_on: [kube_private_subnet_az2]
	properties:
	router_id: { get_param: kube_router_az2 }
	subnet_id: { get_resource: kube_private_subnet_az2 }



	# Create jump server security group
	jump_security_group:
	type: OS::Neutron::SecurityGroup
	properties:
	description: kube stack jump server security group
	name: kube-jumpbox
	rules:
	# allow inbound ssh from my ip
	– remote_ip_prefix: { get_param: my_ip }
	protocol: tcp
	port_range_min: 22
	port_range_max: 22
	– remote_ip_prefix: { get_param: my_ip }
	protocol: icmp

	# Create network connector server security group
	nc_security_group:
	type: OS::Neutron::SecurityGroup
	properties:
	description: kube stack network connector security group
	name: kube-network-connector
	rules:
	# allow full tcp and udp communication internally
	– remote_ip_prefix: "0.0.0.0/0"
	protocol: tcp
	– remote_ip_prefix: "0.0.0.0/0"
	protocol: udp
	– remote_ip_prefix: "0.0.0.0/0"
	protocol: icmp

	# Create lbaas server security group
	lbaas_security_group:
	type: OS::Neutron::SecurityGroup
	properties:
	description: kube stack network connector security group
	name: kube-load-balancer
	rules:
	# allow http traffic to etcd sock-shop service
	– remote_ip_prefix: "0.0.0.0/0"
	protocol: tcp
	port_range_min: 30001
	port_range_max: 30001
	– remote_ip_prefix: "0.0.0.0/0"
	protocol: tcp
	port_range_min: 80
	port_range_max: 80

	############################### AZ Interconnect ###########################################################

	az1_nc_port:
	type: OS::Neutron::Port
	depends_on: [ nc_security_group ]
	properties:
	availability_zone: { get_param: az1 }
	network_id: { get_resource: kube_private_net_az1 }
	security_groups: [{ get_param: default_security_group }]
	fixed_ips:
	– subnet_id: { get_resource: kube_private_subnet_az1 }
	ip_address: '192.168.200.253'
	allowed_address_pairs:
	– ip_address: { get_param: kube_service_network }
	– ip_address: { get_param: kube_pod_network }

	az2_nc_port:
	type: OS::Neutron::Port
	depends_on: [ nc_security_group ]
	properties:
	availability_zone: { get_param: az2 }
	network_id: { get_resource: kube_private_net_az2 }
	security_groups: [{ get_param: default_security_group }]
	fixed_ips:
	– subnet_id: { get_resource: kube_private_subnet_az2 }
	ip_address: '10.11.200.253'
	allowed_address_pairs:
	– ip_address: { get_param: kube_service_network }
	– ip_address: { get_param: kube_pod_network }

	az1_network_connector:
	type: FCX::Neutron::NetworkConnector
	depends_on: [ az1_nc_port,az2_nc_port ]
	properties:
	name: "kube_connector"

	az1_nc_endpoint:
	type: FCX::Neutron::NetworkConnectorEndpoint
	depends_on: [ az1_nc_port,az2_nc_port ]
	properties:
	endpoint_type: "availability_zone"
	name: "kube_ep_az1"
	network_connector_id: { get_resource: az1_network_connector }
	location: { get_param: az1 }

	az2_nc_endpoint:
	type: FCX::Neutron::NetworkConnectorEndpoint
	depends_on: [ az1_nc_port,az2_nc_port ]
	properties:
	endpoint_type: "availability_zone"
	name: "kube_ep_az2"
	network_connector_id: { get_resource: az1_network_connector }
	location: { get_param: az2 }

	az1_endpoint_connection:
	type: FCX::Neutron::NetworkConnectorEndpointConnection
	depends_on: [ az1_nc_port,az2_nc_port ]
	properties:
	port_id: { get_resource: az1_nc_port }
	network_connector_endpoint_id: { get_resource: az1_nc_endpoint }

	az2_endpoint_connection:
	type: FCX::Neutron::NetworkConnectorEndpointConnection
	depends_on: [ az1_nc_port,az2_nc_port ]
	properties:
	port_id: { get_resource: az2_nc_port }
	network_connector_endpoint_id: { get_resource: az2_nc_endpoint }

	#########################################################################################################

	# AZ Load Balancer
	Kube_LBaaS:
	type: FCX::ExpandableLoadBalancer::LoadBalancer
	depends_on: [ worker1-server, worker2-server, worker3-server ]
	properties:
	Subnets: [{get_resource: kube_private_subnet_az1}, {get_resource: kube_private_subnet_az2}]
	Instances: [{get_resource: worker1-server}, {get_resource: worker2-server}, {get_resource: worker3-server}]
	Listeners:
	– {LoadBalancerPort: '80',
	InstancePort: '30001',
	Protocol: 'HTTP',
	InstanceProtocol: 'HTTP' }
	HealthCheck: {Target: 'HTTP:30001/index.html',
	HealthyThreshold: '3',
	UnhealthyThreshold: '5',
	Interval: '30',
	Timeout: '5'}
	Version: 2014-09-30
	Scheme: public
	LoadBalancerName: kubelbaas
	SecurityGroups: [ {get_resource: lbaas_security_group} ]

	# ################### JumpBox – AZ 1 ######################################################################


	################### JumpBox – AZ 1 ######################################################################

	# Create a new port for the server interface, assign an ip address and security group
	jump-server-port:
	type: OS::Neutron::Port
	depends_on: [ jump_security_group ]
	properties:
	availability_zone: { get_param: az1 }
	network_id: { get_resource: kube_private_net_az1 }
	security_groups: [{ get_resource: jump_security_group }, { get_param: default_security_group }]

	# Allocate a floating/global ip address
	jump-server-floating-ip:
	type: OS::Neutron::FloatingIP
	properties:
	availability_zone: { get_param: az1 }
	floating_network_id: { get_param: public_net_az1 }

	# Assign a floating/global ip address to the fixed server ip address
	jump-server-floating-ip-association:
	type: OS::Neutron::FloatingIPAssociation
	depends_on: jump-server-floating-ip
	properties:
	floatingip_id: { get_resource: jump-server-floating-ip }
	port_id: { get_resource: jump-server-port }

	# Create a system volume for use with the server
	jump-sys-vol:
	type: OS::Cinder::Volume
	properties:
	availability_zone: { get_param: az1 }
	name: "Kube-JumpBox"
	size: 30
	volume_type: "M1"
	image : { get_param: k5_image }

	# Build a server using the system volume defined above
	jump-server:
	type: OS::Nova::Server
	depends_on: [ etcd1-server, etcd2-server, etcd3-server, worker1-server, worker2-server, worker3-server, jump-server-port ]
	properties:
	key_name: { get_param: key_az_1 }
	image: { get_param: k5_image }
	flavor: { get_param: flavor }
	admin_user: k5user
	metadata: { "fcx.autofailover": True }
	block_device_mapping: [{"volume_size": "30", "volume_id": {get_resource: jump-sys-vol}, "delete_on_termination": True, "device_name": "/dev/vda"}]
	# Create private key file to enable SSH to remaining servers in the environment
	user_data_format: RAW
	user_data:
	str_replace:
	template:
	#cloud-config
	write_files:
	– path: /root/.ssh/id_rsa
	content:
	—–BEGIN RSA PRIVATE KEY—–
	MIIEoQIBAAKCAQEA4MBSCRr6XOdLBsVSF3x3X/Y+jSoCP4walw5x/8A9g+kF9vuC
	lBU32TlQD/k1B4mxODhxTi1pzmw7Se/ccu3dlGivRgKuQzplVMeCJKAd15MxTr8i
	9ZKlwEj02xxBIzl58pVlHGIGVLwzxyfKtzOSFkEVdUYmJUvifoTYM5R5aWPI2Rhh
	hvkxtmz+WHC67dePcrzAaE0KizRWF4xRP/ZFUXUwcB/Z2fbF5RPocARBjbZgKDb4
	ufC0v9eKdn6mDQDu/Vq7VY8058efitNcfqxhx5d+t0EpwZCha6J7/3KJa9rXABzH
	WrVt1Tk/1F/Z8h/VNyo0k1KLMlBlsHZH5BSMgQIBIwKCAQBmvlFjP4hk/AUKaNUS
	DQNfEXuuPxb4eo/QBpqDixTOlmkgcvndS4c+ux1JIXdT5yxxeONW8DBeXVzumYlZ
	HEgJVG1h1Vb6KVLk7X1SklbI8tS2SL+GNGkHcc8FFDsIydFY89ZzYALk6FIvKCIn
	3Q+VJREfqwofraleaJ1ZaHIESgPzzPHoVc9//51rtqc2pjmmCu3BsJkR8isAiiyL
	LiegC9FIcXXGcgeMWqFBPqHUELcZXTaUYIu/Ie8FBvDXiZMWOQFbemQJczF8BS3j
	YFoGXoY6c8y052ekh8FGziJgy35gHf3ygUMnaVLirJpG0i3SH3M0ZHkVeP8fcw5x
	uFFLAoGBAPqx+ubMIM/tH4X+zUD7mgaX8+EByd8IUKLwVTXGS7XzuAnkVTBr58bC
	AOCtKX9oLcR40J3Y5N0rzZRmzs7+YNXnK2VQTYbZFPWfD6giKuy+JFc+5Ej+Men/
	dOviiF74V8dsHtSwjX/CStzLYdEy6+xyA3Q0USPeVAQKvXDlOe27AoGBAOWBzSmO
	nKvhTk/+A+wqseq/kPdO0NyrofUWCNoJNZivo0y9UnfL9Y6gDMcEIcAlpAsn6nN8
	eKXh77s0sKg3IydVkqMNJfM6Bq2seF/7HrAbhGHYf5D3pIS0RJlSBz8/xeshvFcn
	rSsst/25JHdESCSJgfRcZZ51A7OQ7SdmF2zzAoGBAKS+Gegfvcp3QJnh9Jhq0u5j
	2scPzc0UF7uH/Xsb6Jwj0LYLE2j2c70gZvoLZGmq3D9IFA/07irw5i5SMCGf2Twb
	kYu4bXXfHGbk3mcsZVm+x2yIaiFWldRIyS1LuLNv/ytHDPIq4KRpvCqxiWw3Z9Xk
	he1G8334gFptdSzuZ+VdAoGAVT7XLLFQIpWDfMwQFeP48MrPcczRPAU8J9T7+Tae
	4PDHoCkQAJw2orB5xkNc/jnd2EIGn+xnU45u/GQHGeiYB0urfmP4JyQuXcO3rp8a
	BuWmMvikaQuGP+svbCXILW91ZfaWaYPD+hfskXCnJPway74Lsoi4B6fOLMDNHUMt
	RbkCgYAi4KlL+fT5xJ9X3EbYxQ4P2TAEUAwKh7yvpoFVH/OuKZedIi2sHNOkb895
	qlcmVtbD96T0m4sZ2UVk0oDURsjp7tK8pIdCqXkxx+Y/cSEc+1R/3J0GcUHt2cbc
	YmDPfr1iqizT0n9em0B51SQb3UH88Egf56GHXiVaEZYc7/o0DQ==
	—–END RSA PRIVATE KEY—–
	– path: /home/k5user/kubernetes.cfg
	content:
	[etcd]
	$etcd1_az1_ip
	$etcd2_az1_ip
	$etcd3_az2_ip

	[kube-master]
	$etcd1_az1_ip

	[kube-node]
	$worker1_az1_ip
	$worker2_az1_ip
	$worker3_az2_ip

	[k8s-cluster:children]
	kube-node
	kube-master
	runcmd:
	– chmod 0600 /root/.ssh/id_rsa
	– yum makecache fast
	– yum install -y git
	– yum install -y epel-release
	– yum install -y ansible
	– yum install -y python-netaddr
	– yum install -y python-pip
	– pip install –upgrade pip
	– pip install –upgrade jinja2
	– mkdir -p /home/k5user/kubernetes/kargo
	– git clone https://github.com/kubernetes-incubator/kargo.git /home/k5user/kubernetes/kargo
	– cp /home/k5user/kubernetes.cfg /home/k5user/kubernetes/kargo/inventory/inventory.cfg
	– cd /home/k5user/kubernetes/kargo
	– ansible-playbook -u k5user -b -i inventory/inventory.cfg cluster.yml
	– sleep 180
	– ssh k5user@$etcd1_az1_ip 'kubectl create -f "https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml?raw=true"'
	– sleep 180
	– ssh k5user@$etcd1_az1_ip 'kubectl get cs && kubectl cluster-info dump && kubectl version && kubectl get nodes && kubectl get pods –all-namespaces && sudo docker ps'
	– ssh k5user@$etcd3_az2_ip 'kubectl get cs && kubectl cluster-info dump && kubectl version && kubectl get nodes && kubectl get pods –all-namespaces && sudo docker ps'
	– ssh k5user@$etcd2_az1_ip 'kubectl get cs && kubectl cluster-info dump && kubectl version && kubectl get nodes && kubectl get pods –all-namespaces && sudo docker ps'

	params:
	$etcd1_az1_ip: { get_attr: [ etcd1-server, first_address ] }
	$etcd2_az1_ip: { get_attr: [ etcd2-server, first_address ] }
	$etcd3_az2_ip: { get_attr: [ etcd3-server, first_address ] }
	$worker1_az1_ip: { get_attr: [ worker1-server, first_address ] }
	$worker2_az1_ip: { get_attr: [ worker2-server, first_address ] }
	$worker3_az2_ip: { get_attr: [ worker3-server, first_address ] }
	message: "Installation Complete"
	name: "kube-jumpbox-az1"
	networks:
	– port: { get_resource: jump-server-port }

	##########################################################################################################

	################### Kube etcd master 1 – Net 1 – AZ 1 ##############################################################################

	# Create a new port for the server interface, assign an ip address and security group
	etcd1-server-port:
	type: OS::Neutron::Port
	properties:
	availability_zone: { get_param: az1 }
	network_id: { get_resource: kube_private_net_az1 }
	# security_groups: [{ get_param: default_security_group }]
	allowed_address_pairs:
	– ip_address: { get_param: kube_service_network }
	– ip_address: { get_param: kube_pod_network }

	# Create a system volume for use with the server
	etcd1-sys-vol:
	type: OS::Cinder::Volume
	properties:
	availability_zone: { get_param: az1 }
	name: "etcd1-boot-vol"
	size: 30
	volume_type: "M1"
	image : { get_param: k5_image }

	# Build a server using the system volume defined above
	etcd1-server:
	type: OS::Nova::Server
	# depends_on: [ jump-server,etcd3-server,etcd1-server-port,etcd1-sys-vol ]
	properties:
	key_name: { get_param: key_az_1 }
	image: { get_param: k5_image }
	flavor: { get_param: flavor }
	admin_user: k5user
	metadata: { "fcx.autofailover": True }
	block_device_mapping: [{"volume_size": "30", "volume_id": {get_resource: etcd1-sys-vol}, "delete_on_termination": True, "device_name": "/dev/vda"}]
	name: "etcd1_az1"
	networks:
	– port: { get_resource: etcd1-server-port }

	###########################################################################################################

	################### Kube etcd master 2 – Net 1 – AZ 1 ##############################################################################

	# Create a new port for the server interface, assign an ip address and security group
	etcd2-server-port:
	type: OS::Neutron::Port
	properties:
	availability_zone: { get_param: az1 }
	network_id: { get_resource: kube_private_net_az1 }
	# security_groups: [{ get_resource: web_security_group }]
	allowed_address_pairs:
	– ip_address: { get_param: kube_service_network }
	– ip_address: { get_param: kube_pod_network }

	# Create a system volume for use with the server
	etcd2-sys-vol:
	type: OS::Cinder::Volume
	properties:
	availability_zone: { get_param: az1 }
	name: "etcd2-boot-vol"
	size: 30
	volume_type: "M1"
	image : { get_param: k5_image }

	# Build a server using the system volume defined above
	etcd2-server:
	type: OS::Nova::Server
	# depends_on: [ jump-server,etcd3-server,etcd1-server-port,etcd1-sys-vol ]
	properties:
	key_name: { get_param: key_az_1 }
	image: { get_param: k5_image }
	flavor: { get_param: flavor }
	admin_user: k5user
	metadata: { "fcx.autofailover": True }
	block_device_mapping: [{"volume_size": "30", "volume_id": {get_resource: etcd2-sys-vol}, "delete_on_termination": True, "device_name": "/dev/vda"}]
	name: "etcd2_az1"
	networks:
	– port: { get_resource: etcd2-server-port }

	###########################################################################################################
	################### Kube etcd master 3 – Net 1 – AZ2 #####################################################

	etcd3-server-port:
	type: OS::Neutron::Port
	properties:
	availability_zone: { get_param: az2 }
	network_id: { get_resource: kube_private_net_az2 }
	#security_groups: [{ get_param: default_security_group }]
	allowed_address_pairs:
	– ip_address: { get_param: kube_service_network }
	– ip_address: { get_param: kube_pod_network }

	# Create a system volume for use with the server
	etcd3-sys-vol:
	type: OS::Cinder::Volume
	properties:
	availability_zone: { get_param: az2 }
	name: "etcd3-boot-vol"
	size: 30
	volume_type: "M1"
	image : { get_param: k5_image }

	# Build a server using the system volume defined above
	etcd3-server:
	type: OS::Nova::Server
	# depends_on: [ jump-server ]
	properties:
	key_name: { get_param: key_az_2 }
	image: { get_param: k5_image }
	flavor: { get_param: flavor }
	admin_user: k5user
	metadata: { "fcx.autofailover": True }
	block_device_mapping: [{"volume_size": "30", "volume_id": {get_resource: etcd3-sys-vol}, "delete_on_termination": True, "device_name": "/dev/vda"}]
	name: "etcd3_az2"
	networks:
	– port: { get_resource: etcd3-server-port }

	###########################################################################################################

	################### worker 1 – Net 1 – AZ 1 ##############################################################################

	# Create a new port for the server interface, assign an ip address and security group
	worker1-server-port:
	type: OS::Neutron::Port
	properties:
	availability_zone: { get_param: az1 }
	network_id: { get_resource: kube_private_net_az1 }
	security_groups: [{ get_resource: lbaas_security_group }, { get_param: default_security_group }]
	allowed_address_pairs:
	– ip_address: { get_param: kube_service_network }
	– ip_address: { get_param: kube_pod_network }

	# Create a system volume for use with the server
	worker1-sys-vol:
	type: OS::Cinder::Volume
	properties:
	availability_zone: { get_param: az1 }
	name: "worker1-boot-vol"
	size: 30
	volume_type: "M1"
	image : { get_param: k5_image }

	# Build a server using the system volume defined above
	worker1-server:
	type: OS::Nova::Server
	# depends_on: [ jump-server ]
	properties:
	key_name: { get_param: key_az_1 }
	image: { get_param: k5_image }
	flavor: { get_param: flavor }
	admin_user: k5user
	metadata: { "fcx.autofailover": True }
	block_device_mapping: [{"volume_size": "30", "volume_id": {get_resource: worker1-sys-vol}, "delete_on_termination": True, "device_name": "/dev/vda"}]
	name: "worker1_az1"
	networks:
	– port: { get_resource: worker1-server-port }

	###########################################################################################################

	################### worker 2 – Net 1 – AZ 1 ##############################################################################

	# Create a new port for the server interface, assign an ip address and security group
	worker2-server-port:
	type: OS::Neutron::Port
	properties:
	availability_zone: { get_param: az1 }
	network_id: { get_resource: kube_private_net_az1 }
	security_groups: [{ get_resource: lbaas_security_group }, { get_param: default_security_group }]
	allowed_address_pairs:
	– ip_address: { get_param: kube_service_network }
	– ip_address: { get_param: kube_pod_network }

	# Create a system volume for use with the server
	worker2-sys-vol:
	type: OS::Cinder::Volume
	properties:
	availability_zone: { get_param: az1 }
	name: "worker2-boot-vol"
	size: 30
	volume_type: "M1"
	image : { get_param: k5_image }

	# Build a server using the system volume defined above
	worker2-server:
	type: OS::Nova::Server
	# depends_on: [ jump-server ]
	properties:
	key_name: { get_param: key_az_1 }
	image: { get_param: k5_image }
	flavor: { get_param: flavor }
	admin_user: k5user
	metadata: { "fcx.autofailover": True }
	block_device_mapping: [{"volume_size": "30", "volume_id": {get_resource: worker2-sys-vol}, "delete_on_termination": True, "device_name": "/dev/vda"}]
	name: "worker2_az1"
	networks:
	– port: { get_resource: worker2-server-port }

	##########################################################################################################

	################### worker 3 – Net 1 – AZ 2 ################################################################

	# Create a new port for the server interface, assign an ip address and security group
	worker3-server-port:
	type: OS::Neutron::Port
	properties:
	availability_zone: { get_param: az2 }
	network_id: { get_resource: kube_private_net_az2 }
	security_groups: [{ get_resource: lbaas_security_group }, { get_param: default_security_group }]
	allowed_address_pairs:
	– ip_address: { get_param: kube_service_network }
	– ip_address: { get_param: kube_pod_network }

	# Create a system volume for use with the server
	worker3-sys-vol:
	type: OS::Cinder::Volume
	properties:
	availability_zone: { get_param: az2 }
	name: "worker3-boot-vol"
	size: 30
	volume_type: "M1"
	image : { get_param: k5_image }

	# Build a server using the system volume defined above
	worker3-server:
	type: OS::Nova::Server
	# depends_on: [ jump-server ]
	properties:
	key_name: { get_param: key_az_2 }
	image: { get_param: k5_image }
	flavor: { get_param: flavor }
	admin_user: k5user
	metadata: { "fcx.autofailover": True }
	block_device_mapping: [{"volume_size": "30", "volume_id": {get_resource: worker3-sys-vol}, "delete_on_termination": True, "device_name": "/dev/vda"}]
	name: "worker3_az2"
	networks:
	– port: { get_resource: worker3-server-port }

	###########################################################################################################

voir le code brut
K5_HA_Kubernetes.yml
hébergé avec ❤ par GitHub

Notes : Je ne suis pas un guru de Kubernetes – il s’agit d’exemples de templates Heat qui vous permettront de mettre Kubernetes en place rapidement sur Fujitsu K5 à des fins d’évaluation. Ils n’ont pas été conçus pour des charges de production. Kubernetes est encore un produit relativement jeune qui nécessite des améliorations de ses composants de multi-tenancy et de sécurité avant d’être prêt pour le marché de la production entreprise général – à moins que vous ayez une équipe de développeurs impatients de coder – c’est la beauté des initiatives open source.

Le TYPE LoadBalancer de Kubernetes est API compatible avec GCE, surprise, surprise, et également avec AWS. Bien que j’aie tenté de placer un LoadBalancer K5 devant les minions, il semble qu’il nécessite plus de débogage car je n’obtiens que des retours de page partiels et j’ai manqué de temps pour déboguer davantage…. Je joue avec Kubernetes uniquement le soir pour le moment. Comme nous n’utilisons pas de LBaaS v1 ou v2 natif d’OpenStack, ceux-ci n’étaient pas adaptés au moment du déploiement de K5, malheureusement je n’ai pas pu non plus exploiter leur solution LBaaS Kubernetes.

Si vous avez besoin d’une solution de qualité production sur K5, Kubernetes a été délibérément architecturé pour faciliter les plugins, il ne serait donc pas si difficile de ‘développer votre propre’ API LBaaS. Et grâce à l’esprit communautaire, Concur a déjà fait la plus grande partie du travail difficile pour nous en fournissant un template d’aide ici – https://github.com/concur/kubegowatcher.

Prérequis de déploiement

Veuillez prêter une attention particulière aux paramètres d’entrée et assurez-vous de les configurer pour qu’ils correspondent à votre environnement avant le déploiement.

Une fois déployé, connectez-vous au JumpBox et depuis celui-ci, vous pouvez accéder à tous les nœuds kube.

Commandes utiles :

Afficher les caractères cachés


	# check version
	kubectl version

	# list kube nodes
	kubectl get nodes

	# list pods
	kubectl get pods –all-namespaces

	# check docker
	sudo docker ps

	# deploy demo application
	kubectl create -f "https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml?raw=true"

	# check application status
	kubectl get pod –namespace=sock-shop

	# view the application service
	kubectl describe svc front-end -n sock-shop

	# delete demo application
	kubectl delete -f "https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml?raw=true"

voir le code brut
Kubectl.txt
hébergé avec ❤ par GitHub

Bonne création de piles !

withk5youcan

Originally published on allthingscloud.eu (2017-05-26).