How to mount Amazon S3 Bucket into AWS EC2 instance





Step 1
Update all the packages if required
(RHEL/CentOS)
yum update

(Debian/Ubuntu)
apt-get update

Step 2
Install all the dependencies require to configure
(RHEL/CentOS)
sudo yum install automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel

(Debian/Ubuntu)
sudo apt-get install automake autotools-dev fuse g++ git libcurl4-gnutls-dev libfuse-dev libssl-dev libxml2-dev make pkg-config

Step 3
Clone the s3fs repository from the GitHub
(RHEL Family and Debian Family)
git clone https://github.com/s3fs-fuse/s3fs-fuse.git

Step 4
Now change directory to just clone from the GitHub and compile and setup the s3fs-fuse
cd s3fs-fuse

./autogen.sh

./configure --prefix=/usr --with-openssl

make

sudo make install


Step 5
Run the following command to check if every set up correctly
which s3fs
Output
/usr/bin/s3fs

Step 6
Now, Go to IAM and create a user name "s3user" and give him Programmatic Access and attached the Policy "AmazonS3FullAccess".

While creating a user, it will provide you "Access key ID" and "Secret access key". Note it down into a notepad or download the CSV file. We will need this for configuring the configuration file for s3fs.

Step 7
Now create a configuration file name "passwd-s3fs" under etc folder
vi /etc/passwd-s3fs

and enter the following into the file and save and exit
Access-key-ID:Secret-access-key
eg. AKIATJAADJMXQVLQIOSR:gUSMAUjXPJy4sAPa00+nntGpIjR0eWnL96M9AHOx

Step 8
Change the permission of the file
chmod 640 /etc/passwd-s3fs
Step 9
Now, create a directory into your ec2 instance where you want to mount the S3 Bucket
mkdir /gautam-bucket

Step 10
Now mount the S3 Bucket into created directory name "gautam-bucket"
sudo s3fs gautam-bucket /gautam-bucket -o passwd_file=/etc/passwd-s3fs

We have mounted the gautam-bucket into /gautam-bucket folder successfully. But But But this is not the secure way to access the bucket into ec2 instance because you have paste the Access key ID and Secret access key into the instance.

In-case your system compromise, hacker can easily steal your data. To overcome this situation we use roles.

Now we will umount the S3 Bucket and remove the passwd-s3fs file. We will now use the Roles to achieve this.

Step 11
Now we will create a role for ec2 to give permission "AmazonS3FullAccess"

Step 12
Go to ec2 instance dashboard select the instance, go to instance setting and attached the s3role with the instance.

Step 13
Now, mount the S3 Bucket again with IAM Role
s3fs -o iam_role="s3role" gautam-bucket /gautam-bucket
Congratulations. You have setup the S3 Bucket privately from your EC2 instance. 

Popular posts from this blog

How to Convert PEM KeyFile into PPK KeyFile and vice versa?

Image
Hi, In this post, we will learn that how can we convert PEM KeyFile into PPK KeyFile and vice versa including the installation of the tool which we will use to accomplish this. For the Windows Operating System In Windows system is easy and straight forward. You just need a tool called "puttygen.exe" from the URL -  https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html Choose the appropriate version as per your OS. Convert PEM into PPK in Windows After downloading the PuTTYgen and open it, load your file (select All Files (*.*) in the bottom dropdown to select your PEM key file).  You will get a message Successfully imported foreign key, Just click OK, Now click on Save private key button to save that key in the location where do you want to save it. Convert PPK into PEM in Windows Now we will convert PPK file into PEM using PuTTYgen in Windows PC Open the PuTTYgen tool, now load your PPK file into it, and finally, Go to Conversions Menu and Click on Export OpenSSH k
Read more

How to transfer Google Cloud Storage One account data to Google Cloud another account?

Image
                                                          Image Source: infiflex.com Step 1: Go to the account where storage bucket are resides. Select the Select the Bucket, you want to transfer data and Click on SHOW INFO PANEL on the Top Right. Step 2: Under PERMISSIONS click on Add members. Now type the email id of the another account where you want to transfer all of your files in New Members Box. Select Role Storage > Storage Admin. Step 3: Now, Go to second account, Go to your storage bucket and select and repeat the last steps but this time email id will be the 1st account email id. All Set. Step 4: Finally Go to First account open the Google Cloud Shell and run the following command - gsutil -m cp -r gs://SOURCE-BUCKET-NAME /* gs://DESTINATION-BUCKET-NAME /
Read more

MySQL Commands, Administration, Backup, Restore

Image
  SOURCE : STACKOVERFLOW  To export If it’s an entire DB, then:  mysqldump -u [uname] -p[pass] db_name > db_backup.sql If it’s all DBs, then:  mysqldump -u [uname] -p[pass] --all-databases > all_db_backup.sql If it’s specific tables within a DB, then:  mysqldump -u [uname] -p[pass] db_name table1 table2 > table_backup.sql You can even go as far as auto-compressing the output using gzip (if your DB is very big):  mysqldump -u [uname] -p[pass] db_name | gzip > db_backup.sql.gz If you want to do this remotely and you have the access to the server in question, then the following would work (presuming the MySQL server is on port 3306):  mysqldump -P 3306 -h [ip_address] -u [uname] -p[pass] db_name > db_backup.sql To import Type the following command to import sql data file:  mysql -u username -p -h localhost DATA-BASE-NAME < data.sql In this example, import ‘data.sql’ file into ‘blog’ database using sat as username:  mysql -u sat -p -h localhost blog < data.sql If you h
Read more