How to create a custom cluster role in Rancher v2.x to grant permission on the metrics endpoint of the kube-apiserver in a Rancher managed cluster

• A Rancher v2.x instance
• A Rancher-managed Kubernetes cluster, with Metrics Server deployed. This is deployed by default in Rancher-provisioned clusters.

This article details how to create a cluster role to grant users access to the /metrics endpoint of the Kubernetes API Server, in Rancher-managed Kubernetes clusters.

• Rancher v2.4.x - v2.7.x: creating the role is only possible using the Rancher API.
• Rancher 2.8.x and above: it is possible to create the role using the Rancher API or the Rancher UI.

1. The first step is to create a custom cluster role within Rancher, that grants get permission on the non-resource URL /metrics endpoint.

   As an admin user, generate an un-scoped Rancher API token, and execute the following API request via cURL, to create the required role. You will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY and RANCHER_URL to reflect the generated API token and your Rancher URL. You can also edit the role name, as desired, which is set to kube-api metrics in this example.

   export CATTLE_ACCESS_KEY=token-8jn92
   export CATTLE_SECRET_KEY=l2r4nq9sx6pdhpm4bgwntvgk49qn6rvvmtsvlvkmjk9rjsfd7n65fz
   export RANCHER_URL=rancher.example.com
   curl -k -u "${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}" \
   -X POST \
   -H 'Accept: application/json' \
   -H 'Content-Type: application/json' \
   -d '{"context":"cluster","clusterCreatorDefault":false," projectCreatorDefault":false,"name":"kube-api metrics","rules":[{"nonResourceURLs":["/metrics"],"type":"/v3/schemas/policyRule","verbs":["get"]}]}' \
   "https://${RANCHER_URL}/v3/roletemplates"

2. After creating the cluster role, you can then grant this for a user or group. To do so, follow the steps in the Rancher documentation on assigning a cluster role to a user or group.

3. Once the role is granted to a user, they will be able to test their access to the /metrics endpoint.

   The user can access the endpoint, with the applicable cluster ID, via the Rancher proxied Kubernetes API Server endpoint, by generating a cluster-scoped or un-scoped API token. The user will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY, RANCHER_URL and CLUSTER_ID to reflect the generated API token, Rancher URL and cluster ID.

   export CATTLE_ACCESS_KEY=token-8jn92
   export CATTLE_SECRET_KEY=l2r4nq9sx6pdhpm4bgwntvgk49qn6rvvmtsvlvkmjk9rjsfd7n65fz
   export RANCHER_URL=rancher.example.com
   export CLUSTER_ID=c-wwdjc
   curl -k https://${RANCHER_URL}/k8s/clusters/${CLUSTER_ID}/metrics \
   -H "Authorization: Bearer ${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}"

   For Rancher-provisioned Kubernetes clusters with Authorized Cluster Endpoint enabled, the user can also query the endpoint by connecting to the Kubernetes API Server on the cluster's control plane nodes directly, using a cluster-scoped API token. The user will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY and AUTHORIZED_ENDPOINT_ADDRESS to reflect the generated API token, and the authorized endpoint address.

   export CATTLE_ACCESS_KEY=token-d6cls
   export CATTLE_SECRET_KEY=b6gk6lmgrhsb4rjccktzkwxn5df7tm87msggq87lpmls2pkbpc5t5r
   export AUTHORIZED_ENDPOINT_ADDRESS=controlplane-01.example.com
   curl -k https://${AUTHORIZED_ENDPOINT_ADDRESS}:6443/metrics \
   -H "Authorization: Bearer ${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}"

Further reading

• Rancher documentation on custom roles