OpenShift - Administration
In this chapter, we will cover topics such as how to manage a node, configure a service account, etc.
Master and Node Configuration
In OpenShift, we need to use the start command along with OC to boot up a new server. While launching a new master, we need to use the master along with the start command, whereas while starting the new node we need to use the node along with the start command. In order to do this, we need to create configuration files for the master as well as for the nodes. We can create a basic configuration file for the master and the node using the following command.
For master configuration file
$ openshift start master --write-config = /openshift.local.config/master
For node configuration file
$ oadm create-node-config --node-dir = /openshift.local.config/node-<node_hostname> --node = <node_hostname> --hostnames = <hostname>,<ip_address>
Once we run the following commands, we will get the base configuration files that can be used as the starting point for configuration. Later, we can have the same file to boot the new servers.
apiLevels:
- v1beta3
- v1
apiVersion: v1
assetConfig:
logoutURL: ""
masterPublicURL: https://172.10.12.1:7449
publicURL: https://172.10.2.2:7449/console/
servingInfo:
bindAddress: 0.0.0.0:7449
certFile: master.server.crt
clientCA: ""
keyFile: master.server.key
maxRequestsInFlight: 0
requestTimeoutSeconds: 0
controllers: '*'
corsAllowedOrigins:
- 172.10.2.2:7449
- 127.0.0.1
- localhost
dnsConfig:
bindAddress: 0.0.0.0:53
etcdClientInfo:
ca: ca.crt
certFile: master.etcd-client.crt
keyFile: master.etcd-client.key
urls:
- https://10.0.2.15:4001
etcdConfig:
address: 10.0.2.15:4001
peerAddress: 10.0.2.15:7001
peerServingInfo:
bindAddress: 0.0.0.0:7001
certFile: etcd.server.crt
clientCA: ca.crt
keyFile: etcd.server.key
servingInfo:
bindAddress: 0.0.0.0:4001
certFile: etcd.server.crt
clientCA: ca.crt
keyFile: etcd.server.key
storageDirectory: /root/openshift.local.etcd
etcdStorageConfig:
kubernetesStoragePrefix: kubernetes.io
kubernetesStorageVersion: v1
openShiftStoragePrefix: openshift.io
openShiftStorageVersion: v1
imageConfig:
format: openshift/origin-${component}:${version}
latest: false
kind: MasterConfig
kubeletClientInfo:
ca: ca.crt
certFile: master.kubelet-client.crt
keyFile: master.kubelet-client.key
port: 10250
kubernetesMasterConfig:
apiLevels:
- v1beta3
- v1
apiServerArguments: null
controllerArguments: null
masterCount: 1
masterIP: 10.0.2.15
podEvictionTimeout: 5m
schedulerConfigFile: ""
servicesNodePortRange: 30000-32767
servicesSubnet: 172.30.0.0/16
staticNodeNames: []
masterClients:
externalKubernetesKubeConfig: ""
openshiftLoopbackKubeConfig: openshift-master.kubeconfig
masterPublicURL: https://172.10.2.2:7449
networkConfig:
clusterNetworkCIDR: 10.1.0.0/16
hostSubnetLength: 8
networkPluginName: ""
serviceNetworkCIDR: 172.30.0.0/16
oauthConfig:
assetPublicURL: https://172.10.2.2:7449/console/
grantConfig:
method: auto
identityProviders:
- challenge: true
login: true
name: anypassword
provider:
apiVersion: v1
kind: AllowAllPasswordIdentityProvider
masterPublicURL: https://172.10.2.2:7449/
masterURL: https://172.10.2.2:7449/
sessionConfig:
sessionMaxAgeSeconds: 300
sessionName: ssn
sessionSecretsFile: ""
tokenConfig:
accessTokenMaxAgeSeconds: 86400
authorizeTokenMaxAgeSeconds: 300
policyConfig:
bootstrapPolicyFile: policy.json
openshiftInfrastructureNamespace: openshift-infra
openshiftSharedResourcesNamespace: openshift
projectConfig:
defaultNodeSelector: ""
projectRequestMessage: ""
projectRequestTemplate: ""
securityAllocator:
mcsAllocatorRange: s0:/2
mcsLabelsPerProject: 5
uidAllocatorRange: 1000000000-1999999999/10000
routingConfig:
subdomain: router.default.svc.cluster.local
serviceAccountConfig:
managedNames:
- default
- builder
- deployer
masterCA: ca.crt
privateKeyFile: serviceaccounts.private.key
privateKeyFile: serviceaccounts.private.key
publicKeyFiles:
- serviceaccounts.public.key
servingInfo:
bindAddress: 0.0.0.0:8443
certFile: master.server.crt
clientCA: ca.crt
keyFile: master.server.key
maxRequestsInFlight: 0
requestTimeoutSeconds: 3600
Node configuration files
allowDisabledDocker: true
apiVersion: v1
dnsDomain: cluster.local
dnsIP: 172.10.2.2
dockerConfig:
execHandlerName: native
imageConfig:
format: openshift/origin-${component}:${version}
latest: false
kind: NodeConfig
masterKubeConfig: node.kubeconfig
networkConfig:
mtu: 1450
networkPluginName: ""
nodeIP: ""
nodeName: node1.example.com
podManifestConfig:
path: "/path/to/pod-manifest-file"
fileCheckIntervalSeconds: 30
servingInfo:
bindAddress: 0.0.0.0:10250
certFile: server.crt
clientCA: node-client-ca.crt
keyFile: server.key
volumeDirectory: /root/openshift.local.volumes
This is how the node configuration files look like. Once we have these configuration files in place, we can run the following command to create master and node server.
$ openshift start --master-config = /openshift.local.config/master/master-
config.yaml --node-config = /openshift.local.config/node-<node_hostname>/node-
config.yaml
Managing Nodes
In OpenShift, we have OC command line utility which is mostly used for carrying out all the operations in OpenShift. We can use the following commands to manage the nodes.
For listing a node
$ oc get nodes
NAME LABELS
node1.example.com kubernetes.io/hostname = vklnld1446.int.example.com
node2.example.com kubernetes.io/hostname = vklnld1447.int.example.com
Describing details about a node
$ oc describe node <node name>
Deleting a node
$ oc delete node <node name>
Listing pods on a node
$ oadm manage-node <node1> <node2> --list-pods [--pod-selector=<pod_selector>] [-o json|yaml]
Evaluating pods on a node
$ oadm manage-node <node1> <node2> --evacuate --dry-run [--pod-selector=<pod_selector>]
Configuration Authentication
In OpenShift master, there is a built-in OAuth server, which can be used for managing authentication. All OpenShift users get the token from this server, which helps them communicate to OpenShift API.
There are different kinds of authentication level in OpenShift, which can be configured along with the main configuration file.
- Allow all
- Deny all
- HTPasswd
- LDAP
- Basic authentication
- Request header
While defining the master configuration, we can define the identification policy where we can define the type of policy that we wish to use.
Allow All
Allow All
oauthConfig:
...
identityProviders:
- name: Allow_Authontication
challenge: true
login: true
provider:
apiVersion: v1
kind: AllowAllPasswordIdentityProvider
Deny All
This will deny access to all usernames and passwords.
oauthConfig:
...
identityProviders:
- name: deny_Authontication
challenge: true
login: true
provider:
apiVersion: v1
kind: DenyAllPasswordIdentityProvider
HTPasswd
HTPasswd is used to validate the username and password against an encrypted file password.
For generating an encrypted file, following is the command.
$ htpasswd </path/to/users.htpasswd> <user_name>
Using the encrypted file.
oauthConfig:
...
identityProviders:
- name: htpasswd_authontication
challenge: true
login: true
provider:
apiVersion: v1
kind: HTPasswdPasswordIdentityProvider
file: /path/to/users.htpasswd
LDAP Identity Provider
This is used for LDAP authentication wherein LDAP server plays a key role in authentication.
oauthConfig:
...
identityProviders:
- name: "ldap_authontication"
challenge: true
login: true
provider:
apiVersion: v1
kind: LDAPPasswordIdentityProvider
attributes:
id:
- dn
email:
- mail
name:
- cn
preferredUsername:
- uid
bindDN: ""
bindPassword: ""
ca: my-ldap-ca-bundle.crt
insecure: false
url: "ldap://ldap.example.com/ou=users,dc=acme,dc=com?uid"
Basic Authentication
This is used when the validation of username and password is done against a server-to-server authentication. The authentication is protected in the base URL and is presented in JSON format.
oauthConfig:
...
identityProviders:
- name: my_remote_basic_auth_provider
challenge: true
login: true
provider:
apiVersion: v1
kind: BasicAuthPasswordIdentityProvider
url: https://www.vklnld908.int.example.com/remote-idp
ca: /path/to/ca.file
certFile: /path/to/client.crt
keyFile: /path/to/client.key
Configuring a Service Account
Service accounts provide a flexible way of accessing OpenShift API exposing the username and password for authentication.
Enabling a Service Account
Service account uses a key pair of public and private key for authentication. Authentication to API is done using a private key and validating it against a public key.
ServiceAccountConfig:
...
masterCA: ca.crt
privateKeyFile: serviceaccounts.private.key
publicKeyFiles:
- serviceaccounts.public.key
- ...
Creating a Service Account
Use the following command to create a service account
$ Openshift cli create service account <name of server account>
Working with HTTP Proxy
In most of the production environment, direct access to Internet is restricted. They are either not exposed to Internet or they are exposed via a HTTP or HTTPS proxy. In an OpenShift environment, this proxy machine definition is set as an environment variable.
This can be done by adding a proxy definition on the master and node files located under /etc/sysconfig. This is similar as we do for any other application.
Master Machine
/etc/sysconfig/openshift-master
HTTP_PROXY=http://USERNAME:PASSWORD@172.10.10.1:8080/
HTTPS_PROXY=https://USERNAME:PASSWORD@172.10.10.1:8080/
NO_PROXY=master.vklnld908.int.example.com
Node Machine
/etc/sysconfig/openshift-node
HTTP_PROXY=http://USERNAME:PASSWORD@172.10.10.1:8080/
HTTPS_PROXY=https://USERNAME:PASSWORD@172.10.10.1:8080/
NO_PROXY=master.vklnld908.int.example.com
Once done, we need to restart the master and node machines.
For Docker Pull
/etc/sysconfig/docker
HTTP_PROXY = http://USERNAME:PASSWORD@172.10.10.1:8080/
HTTPS_PROXY = https://USERNAME:PASSWORD@172.10.10.1:8080/
NO_PROXY = master.vklnld1446.int.example.com
In order to make a pod run in a proxy environment, it can be done using −
containers:
- env:
- name: "HTTP_PROXY"
value: "http://USER:PASSWORD@:10.0.1.1:8080"
OC environment command can be used to update the existing env.
OpenShift Storage with NFS
In OpenShift, the concept of persistent volume and persistent volume claims forms persistent storage. This is one of the key concepts in which first persistent volume is created and later that same volume is claimed. For this, we need to have enough capacity and disk space on the underlying hardware.
apiVersion: v1
kind: PersistentVolume
metadata:
name: storage-unit1
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
nfs:
path: /opt
server: 10.12.2.2
persistentVolumeReclaimPolicy: Recycle
Next, using OC create command create Persistent Volume.
$ oc create -f storage-unit1.yaml
persistentvolume " storage-unit1 " created
Claiming the created volume.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: Storage-clame1
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
Create the claim.
$ oc create -f Storage-claim1.yaml
persistentvolume " Storage-clame1 " created
User and Role Management
User and role administration is used to manage users, their access and controls on different projects.
Creating a User
Predefined templates can be used to create new users in OpenShift.
kind: "Template"
apiVersion: "v1"
parameters:
- name: vipin
required: true
objects:
- kind: "User"
apiVersion: "v1"
metadata:
name: "${email}"
- kind: "Identity"
apiVersion: "v1"
metadata:
name: "vipin:${email}"
providerName: "SAML"
providerUserName: "${email}"
- kind: "UserIdentityMapping"
apiVersion: "v1"
identity:
name: "vipin:${email}"
user:
name: "${email}"
Use oc create –f <file name> to create users.
$ oc create –f vipin.yaml
Use the following command to delete a user in OpenShift.
$ oc delete user <user name>
Limiting User Access
ResourceQuotas and LimitRanges are used for limiting user access levels. They are used for limiting the pods and containers on the cluster.
apiVersion: v1
kind: ResourceQuota
metadata:
name: resources-utilization
spec:
hard:
pods: "10"
Creating the quote using the above configuration
$ oc create -f resource-quota.yaml –n –Openshift-sample
Describing the resource quote
$ oc describe quota resource-quota -n Openshift-sample
Name: resource-quota
Namespace: Openshift-sample
Resource Used Hard
-------- ---- ----
pods 3 10
Defining the container limits can be used for limiting the resources which are going to be used by deployed containers. They are used to define the maximum and minimum limitations of certain objects.
User project limitations
This is basically used for the number of projects a user can have at any point of time. They are basically done by defining the user levels in categories of bronze, silver, and gold.
We need to first define an object which holds the value of how many projects a bronze, silver, and gold category can have. These need to be done in the master-confif.yaml file.
admissionConfig:
pluginConfig:
ProjectRequestLimit:
configuration:
apiVersion: v1
kind: ProjectRequestLimitConfig
limits:
- selector:
level: platinum
- selector:
level: gold
maxProjects: 15
- selector:
level: silver
maxProjects: 10
- selector:
level: bronze
maxProjects: 5
Restart the master server.
Assigning a user to a particular level.
$ oc label user vipin level = gold
Moving the user out of the label, if required.
$ oc label user <user_name> level-
Adding roles to a user.
$ oadm policy add-role-to-user <user_name>
Removing the role from a user.
$ oadm policy remove-role-from-user <user_name>
Adding a cluster role to a user.
$ oadm policy add-cluster-role-to-user <user_name>
Removing a cluster role from a user.
$ oadm policy remove-cluster-role-from-user <user_name>
Adding a role to a group.
$ oadm policy add-role-to-user <user_name>
Removing a role from a group.
$ oadm policy remove-cluster-role-from-user <user_name>
Adding a cluster role to a group.
$ oadm policy add-cluster-role-to-group <groupname>
Removing a cluster role from a group.
$ oadm policy remove-cluster-role-from-group <role> <groupname>
User for cluster administration
This is one of the most powerful roles where the user has the capability to manage a complete cluster starting from creation till deletion of a cluster.
$ oadm policy add-role-to-user admin <user_name> -n <project_name>
User with ultimate power
$ oadm policy add-cluster-role-to-user cluster-admin <user_name>
