How to send logs to Amazon Web Services (AWS) CloudWatch with the new logging services available on Rancher v2.5.x
This document (000020008) is provided subject to the disclaimer at the end of this document.
Situation
Task
New logging service introduced in Rancher v2.5.x allows users to send logs to Amazon Web Services (AWS) Cloudwatch. This article details how to send logs to AWS CloudWatch with the new logging services available on Rancher v2.5.x, in Rancher Kubernetes Engine (RKE) CLI or Rancher v2.5.x provisioned Kubernetes clusters.
Pre-requisites
- A Rancher Kubernetes Engine (RKE) CLI or Rancher v2.x provisioned Kubernetes cluster with Logging in Rancher 2.5 enabled
- Rancher v2.5.x
- AWS IAM policy with at least the following permissions, the policy is attached to either an IAM user with credentials, or an EC2 instance profile which is attached to the nodes in the cluster
{ "Version": "2012-10-17", "Statement": [ { "Action": [ "logs:PutLogEvents", "logs:CreateLogGroup", "logs:PutRetentionPolicy", "logs:CreateLogStream", "logs:DescribeLogGroups", "logs:DescribeLogStreams" ], "Effect": "Allow", "Resource": "*" } ] }
Steps
- Ensure the Rancher v2.5 logging is enabled on the cluster; visit and follow the Logging section in the Rancher docs if it is not already enabled.
-
Optional Create a secret containing the AWS Access key ID and Secret access key in
cattle-logging-system
namespace:
> Note, this step is not required if using an EC2 instance profilecat <<EOF | kubectl apply -f - apiVersion: v1 data: id: <AWS Access key ID> secret: <AWS Secret access key> kind: Secret metadata: name: aws namespace: cattle-logging-system type: Opaque EOF
- Create the ClusterOutput and ClusterFlow to forward the logs to the CloudWatch.
> Note, thecat <<EOF | kubectl apply -f - apiVersion: logging.banzaicloud.io/v1beta1 kind: ClusterOutput metadata: name: cloudwatch namespace: cattle-logging-system spec: cloudwatch: auto_create_stream: true #Set to false to disable automatically create Log Stream under the Log Group aws_key_id: valueFrom: secretKeyRef: key: id name: aws aws_sec_key: valueFrom: secretKeyRef: key: secret name: aws buffer: timekey: 30s timekey_use_utc: true timekey_wait: 30s log_group_name: <LOG GROUP NAME ON THE CLOUDWATCH> log_stream_name: <LOG STREAM NAME UNDER THE LOG GROUP> region: <AWS REGION> --- apiVersion: logging.banzaicloud.io/v1beta1 kind: ClusterFlow metadata: name: logging namespace: cattle-logging-system spec: globalOutputRefs: - cloudwatch EOF
aws_key_id
andaws_sec_key
should be removed if using an EC2 instance profile - The logs will start sending to the CloudWatch once the ClusterOuput and ClusterFlow are created. You may visit the docs in the below to explore all the available configurations for the Rancher v2.5 logging.
Further reading
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000020008
- Creation Date: 06-May-2021
- Modified Date:06-May-2021
-
- SUSE Rancher
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com