Monitor Memory and Disk Utilization on EC2 instances with Cloudwatch Custom metrics



As you know, in AWS we cannot monitor Hard Disk (Volume) and Memory (RAM) through CloudWatch but with few steps, we can monitor and set alarm on it according to our needs. So, in this post we will see, how can we accomplish that.

Step1.

Installation of dependencies (packages and binaries) in order to setup Memory and Disk utilization

RHEL 7
sudo yum install perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA --enablerepo="rhui-REGION-rhel-server-optional" -y
sudo yum install zip unzip -y
Amazon Linux 2 and Amazon Linux AMI
sudo yum install perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA.x86_64 -y 
sudo yum install zip unzip -y
SUSE Linux Enterprise Server
sudo zypper install perl-Switch perl-DateTime
sudo zypper install –y "perl(LWP::Protocol::https)"
Debian Based Systems 
sudo apt-get update
sudo apt-get install unzip libwww-perl libdatetime-perl


Step2.

Download and configure the "Memory" and "Disk/Volume" script

Create a folder cloudwatch
mkdir cloudwatch
Download the package into the cloudwatch directory
wget https://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.2.zip
Unzip the downloaded script
unzip CloudWatchMonitoringScripts-1.2.2.zip
Remove the download zip script compress file
rm CloudWatchMonitoringScripts-1.2.2.zip
Now go inside the extracted script directory
cd aws-scripts-mon
Create a copy of awscreds.template to awscreds.conf
cp awscreds.template awscreds.conf
Now edit the copy file and put AWSAccessId and AWSSecretKey
AWSAccessId=

AWSSecretKey=



Step 3.

Create a user cloudwatch user with Programmatic access

Go to IAM and create a user with following details -
Username - cloudwatchuser 
Access type - Programmatic access 
Permission - CloudWatchFullAccess
Copy the AWS Access ID and AWS Secret Key for next step to configure "awscred.conf"

Now, Paste here the AWSAccessId and AWSSecretKey copy in the last step into the awscreds.conf file

Varify Command
./mon-put-instance-data.pl --mem-util --verify --verbose
Original command
./mon-put-instance-data.pl --mem-used-incl-cache-buff --mem-util --mem-used --mem-avail
The memory commands All in One (this command is same as above command
./mon-put-instance-data.pl --mem-util --mem-used --mem-avail

now run the following command to send data to cloudwatch (Disk data)

Disk Utility
./mon-put-instance-data.pl --disk-space-util --disk-path=/dev/xvda1
Disk Space Available
./mon-put-instance-data.pl --disk-space-avail --disk-path=/dev/xvda1
Disk Space Used
./mon-put-instance-data.pl --disk-space-used --disk-path=/dev/xvda1
Disk All in One command
./mon-put-instance-data.pl --disk-space-util --disk-space-used --disk-space-avail --disk-path=/dev/xvda1

To see the all information related CPU, Memory and Disk details in the last 24 hours
./mon-get-instance-stats.pl --recent-hours=24


Step 4.

For help related to cronjob and other commands
./mon-put-instance-data.pl --help


Step 5.
To continuously send the data to CloudWatch. We will set a cronjob
*/5 * * * * /home/gautam/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl --mem-util --disk-space-util --disk-path=/dev/xvda1 --from-cron

*/5 * * * * /home/gautam/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl --mem-used --disk-space-used --disk-path=/dev/xvda1 --from-cron

*/5 * * * * /home/gautam/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl --mem-avail --disk-space-avail --disk-path=/dev/xvda1 --from-cron



Step 6.
Now we are setup alarm for disk usage

Copy your instance id, Linux System > Filesystem, instanceId, MountPath => selct DiskSpaceUsed

Popular posts from this blog

How to Convert PEM KeyFile into PPK KeyFile and vice versa?

Image
Hi, In this post, we will learn that how can we convert PEM KeyFile into PPK KeyFile and vice versa including the installation of the tool which we will use to accomplish this. For the Windows Operating System In Windows system is easy and straight forward. You just need a tool called "puttygen.exe" from the URL -  https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html Choose the appropriate version as per your OS. Convert PEM into PPK in Windows After downloading the PuTTYgen and open it, load your file (select All Files (*.*) in the bottom dropdown to select your PEM key file).  You will get a message Successfully imported foreign key, Just click OK, Now click on Save private key button to save that key in the location where do you want to save it. Convert PPK into PEM in Windows Now we will convert PPK file into PEM using PuTTYgen in Windows PC Open the PuTTYgen tool, now load your PPK file into it, and finally, Go to Conversions Menu and Click on Export OpenSSH k
Read more

How to transfer Google Cloud Storage One account data to Google Cloud another account?

Image
                                                          Image Source: infiflex.com Step 1: Go to the account where storage bucket are resides. Select the Select the Bucket, you want to transfer data and Click on SHOW INFO PANEL on the Top Right. Step 2: Under PERMISSIONS click on Add members. Now type the email id of the another account where you want to transfer all of your files in New Members Box. Select Role Storage > Storage Admin. Step 3: Now, Go to second account, Go to your storage bucket and select and repeat the last steps but this time email id will be the 1st account email id. All Set. Step 4: Finally Go to First account open the Google Cloud Shell and run the following command - gsutil -m cp -r gs://SOURCE-BUCKET-NAME /* gs://DESTINATION-BUCKET-NAME /
Read more

MySQL Commands, Administration, Backup, Restore

Image
  SOURCE : STACKOVERFLOW  To export If it’s an entire DB, then:  mysqldump -u [uname] -p[pass] db_name > db_backup.sql If it’s all DBs, then:  mysqldump -u [uname] -p[pass] --all-databases > all_db_backup.sql If it’s specific tables within a DB, then:  mysqldump -u [uname] -p[pass] db_name table1 table2 > table_backup.sql You can even go as far as auto-compressing the output using gzip (if your DB is very big):  mysqldump -u [uname] -p[pass] db_name | gzip > db_backup.sql.gz If you want to do this remotely and you have the access to the server in question, then the following would work (presuming the MySQL server is on port 3306):  mysqldump -P 3306 -h [ip_address] -u [uname] -p[pass] db_name > db_backup.sql To import Type the following command to import sql data file:  mysql -u username -p -h localhost DATA-BASE-NAME < data.sql In this example, import ‘data.sql’ file into ‘blog’ database using sat as username:  mysql -u sat -p -h localhost blog < data.sql If you h
Read more