Left4Code

sha*sum for hashing files from the command line

To clear up any confusion, when I refer to sha*sum, I'm referring to most of the command line hashing programs that come with most linux distributions by default. (md5sum, sha512sum, sha256sum) So just replace sha*sum with the cli hash utility you're currently using. If you want to see what hashing utilities you have on your system, you can have a look in /bin to see what you've got! This is a quick and dirty way to see.

ls /bin | grep sum

What you need to know (To get the most out of this!)

Basic understanding of the Linux command line (bash). Specifically, do you understand output and input redirection and pipes ('>', '<', and '|') Some determination to read, mentally digest, practice, and learn for yourself. How to use the manpages (run "man man" without the double quotes in your terminal if you don't know) this is so you can always use the manpages if this course doesn't get completed or updated. I want to teach you to fish, not give you fish. Whenever I put carat symbols outside of something, don't add them to the command in your actual terminal, ex: <yourfile> should be typed in your terminal as yourfile, or whatever you want to name it, it's just a placeholder, you get it.

What this page covers (To Not Waste Your Time!)

Quick introduction to hashing things with the command line What can be hashed Some techniques to make hashing more effective Saving hash output to a file Binary mode Taking a hash from a file and comparing it against a file to be hashed Closing notes

A Quick Introduction to Hashing Using the Command Line!

If you've already read the gtkhash section of the course, you'll know the basics of how hashing works from a high level, and why it's useful for a forensic investigation, to avoid wasting your time, I will quickly explain how to use 99% of command line hashing tools and then go into further detail about some things that are a little more advanced. (moar content!)

Starting off with the basics of the basics, if you want to hash a file using the command line, type the following (obviously remember to change sha*sum to your preferred hash program!)

sha*sum <file_you_want_to_hash>

So if all you wanted to know was how to get the hash of a file, that should do it. It will just print the hash of the corresponding file to standard output in the terminal. Now if you want to learn some more things you can do involving bash and these utilities, stick around.

What can be hashed?

Something you might commonly hear after you've used Linux enough is "in Linux, everything is a file". So if that's the case, then technically we could hash anything on the system, couldn't we? Let's see some common examples of things that can be hashed! A fun little list I've cobbled together shows you some of the fun things you can use sha*sum on.

1 — Output from other programs (this will come in handy later!)

echo "Hello" | sha*sum

2 — /dev/null !

sha*sum /dev/null

3 — File Metadata!

mat2 -s <your_file> | sha*sum

4 — The Git Repo for this course!

wget https://git.i2pd.xyz/Left4Code/L4C_Forensics_CTF/ -O h1.html && sha*sum $_

Basically, you can hash whatever your heart desires if you're thinking hard enough. I'll manipulate some of the above examples to instead be forensics-oriented in the next section.

Some Techniques to Make Hashing Effective for Forensics

Take this scenario for example. You're a forensic investigator and need to always be completely sure that the content given to you by someone (let's say a laptop hard-drive) will keep it's integrity and it can always be verified that nobody has modified it. How would we do that? Well.. With that new knowledge about hashing you just learned, you know that we can use it to hash the files on the drive. But let's go a step further, remember that saying "In Linux, everything is a file"? This includes drives. So instead of hashing out every single file on the drive, just hash the drive file! If anything on the drive changes, the hash will change when verified again and then you can restore from a backup or take the necessary action based on your hashing precaution. To hash a drive, it's pretty simple, you can first use another command line utility (dd) to generate a drive image, then hash it! (In the example below, sdX will need to be changed to the drive you actually want to hash.)

Quick word of WARNING, the command below this message will create a complete disk image clone to the actual size of your drive, running this command will effectively fill up all space on your drive. If you still want to run this, maybe get a small usb drive, put some files on it, and create a disk image and hash from that instead to get comfortable with the process.

sudo dd if=/dev/sdX of=/<your_dir>/<drive_dump> bs=4M status=progress && sha*sum <drive_dump>

Once we generate the .dd file for the target drive and generate the hash for it, we would theoretically be able to pass this to another investigator without the fear of it being modified and nobody knowing about it.

If you don't have the disk space to copy your entire drive to another one. Then you can run this command which will directly generate a hash from your drive and only read from it and not write to it.

sha*sum /dev/sdX

If you want to check what drives you have available to be hashed on the system, you can use the following command to check:

lsblk

This would only be for the cases where you can't use dc3dd, because it has the ability to hash the .dd file immediately after and this is not necessary. However, using sha*sum on files can still be useful for things like creating hash databases, getting known hashes and inputting them into something like autopsy or sleuthkit to automatically scan for them when looking through a drive, and hashing a live linux system.

Saving the output from sha*sum to a text file

When you normally work with sha*sum, you will not be able to save the output of the hash you generate to a file, there's no -o option and it just prints to standard output so you'll have to use the shell to save the output to a file.

sha*sum <file_youre_hashing> > <output_file>

Reading files in binary mode

When specifying sha*sum to read in binary mode with the -b flag, this is specifically used so that binary and other files which need very careful attention to detail are read properly, sha*sum does this by reading the input file byte by byte instead of text character by text character, it is very rare that you will ever use this, but it's good to know that it exists if you need to use it for very specific circumstances where a file is presenting two different hashes depending on the mode specified.

sha*sum -b <file_youre_hashing> > <output_file>

Comparing hash files to generated hashes with sha*sum

This is honestly useful even without the forensic context, it's important to verify the hashes of software you're downloading to ensure that the software is coming from the developers and has not been modified by a third party. The complete version of this involves using pgp keys in combination with the hashes, but to keep this simple (and also because I have no idea how to do it, when I figure it out I will update this) I will just show the check functionality for sha*sum

Let's use this scenario: I'm the developer, and I want to prove to the user that the executable they are downloading comes from me and has not been tampered with, I would first hash the executable to a file like so:

The Developer:

sha*sum --tag <the_executable> > <the_hash_file>

the "--tag" makes it so sha*sum won't throw a beginning error when you check the hash file against the file you're running sha*sum on and sha*sum will add a little more content to the ouput file showing the correlation of the file and the hash, not putting --tag does not negatively impact sha*sum's ability to check the hash file compared to what it is being ran against. I would then include the hash file and the executable file together so that the user can download both, then, as the user, I would download both and then run the following command:

The User:

sha*sum -c <the_executable> <the_hash_file>

The output of this command should say OK somewhere in the terminal, if it does not, and says FAILED: checksum did not match, then you know someone's up to some funny business and you probably shouldn't install that piece of software.

Conclusion

This covers most of the functionality of the sha*sum utilities and the md*sum utilities. With this you should be able to hash basically anything you want and be able to check and verify that hashes you receive are correct and actually coming from a valid source.