This collection of public utilities will be useful for the exploration of binary data.
Binwalk is commonly used for firmware analysis. With a diverse set of signatures built-in to recognize compressed stream, executable code, cryptographic markers and so on, you can use Binwalk to scan arbitrary binaries.
Visual analysis of binary files
Binvis.io is an interactive online utility for the visual exploration of binary data.
Strings scans the input binary for ASCII and Unicode texts.
Deepmage is a hex editor that can handle data in units of arbitrary bit width.
Hexyl is hex viewer that produces a hex dump in colors to distinguish different categories of bytes.
Multidiff compares multiple binary files.
FV program is a well-known utility in the compression development industry. It is used to visualize the matches in the data. There is a reference to FV in the Data Compression Explained book.
DataSmoke aims to distinguish different data types (in order to choose the best compression method). It has multiple short algorithms and some are based on entropy calculation.
ent – A Pseudorandom Number Sequence Test Program
Ent tests the randomness of the content of a file. It uses various algorithms including entropy, chi-squared test, arithmetic mean, correlation coefficient, and so on.
Dump2Picture adds a BMP header to arbitrary file. The result can be viewed as an image.
QuickBMS allows to extract content from compressed and encryped file formats.
Signsrch scans files to recognize compressed stream, encryption, checksum, and so on. It uses an external signature file.
NIST Statistical Test Suite
This research project can be useful to learn about approaches to analyze binary data.
Feel free to send an email about other valuable utilities on the topic.