std.digest
Category | Functions |
---|---|
Template API | isDigest DigestType hasPeek hasBlockSize ExampleDigest digest hexDigest makeDigest |
OOP API | Digest |
Helper functions | toHexString secureEqual |
Implementation helpers | digestLength WrapperDigest |
APIs There are two APIs for digests: The template API and the OOP API. The template API uses structs and template helpers like isDigest. The OOP API implements digests as classes inheriting the Digest interface. All digests are named so that the template API struct is called "x" and the OOP API class is called "xDigest". For example we have MD5 <--> MD5Digest, CRC32 <--> CRC32Digest, etc.
The template API is slightly more efficient. It does not have to allocate memory dynamically, all memory is allocated on the stack. The OOP API has to allocate in the finish method if no buffer was provided. If you provide a buffer to the OOP APIs finish function, it doesn't allocate, but the Digest classes still have to be created using new which allocates them using the GC. The OOP API is useful to change the digest function and/or digest backend at 'runtime'. The benefit here is that switching e.g. Phobos MD5Digest and an OpenSSLMD5Digest implementation is ABI compatible. If just one specific digest type and backend is needed, the template API is usually a good fit. In this simplest case, the template API can even be used without templates: Just use the "x" structs directly.Source std/digest/package.d
CTFE Digests do not work in CTFE
TODO Digesting single bits (as opposed to bytes) is not implemented. This will be done as another template constraint helper (hasBitDigesting!T) and an additional interface (BitDigest)
import std.digest.crc; //Simple example char[8] hexHash = hexDigest!CRC32("The quick brown fox jumps over the lazy dog"); writeln(hexHash); // "39A34F41" //Simple example, using the API manually CRC32 context = makeDigest!CRC32(); context.put(cast(ubyte[])"The quick brown fox jumps over the lazy dog"); ubyte[4] hash = context.finish(); writeln(toHexString(hash)); // "39A34F41"
//Generating the hashes of a file, idiomatic D way import std.digest.crc, std.digest.md, std.digest.sha; import std.stdio; // Digests a file and prints the result. void digestFile(Hash)(string filename) if (isDigest!Hash) { auto file = File(filename); auto result = digest!Hash(file.byChunk(4096 * 1024)); writefln("%s (%s) = %s", Hash.stringof, filename, toHexString(result)); } void main(string[] args) { foreach (name; args[1 .. $]) { digestFile!MD5(name); digestFile!SHA1(name); digestFile!CRC32(name); } }
//Generating the hashes of a file using the template API import std.digest.crc, std.digest.md, std.digest.sha; import std.stdio; // Digests a file and prints the result. void digestFile(Hash)(ref Hash hash, string filename) if (isDigest!Hash) { File file = File(filename); //As digests imlement OutputRange, we could use std.algorithm.copy //Let's do it manually for now foreach (buffer; file.byChunk(4096 * 1024)) hash.put(buffer); auto result = hash.finish(); writefln("%s (%s) = %s", Hash.stringof, filename, toHexString(result)); } void uMain(string[] args) { MD5 md5; SHA1 sha1; CRC32 crc32; md5.start(); sha1.start(); crc32.start(); foreach (arg; args[1 .. $]) { digestFile(md5, arg); digestFile(sha1, arg); digestFile(crc32, arg); } }
import std.digest.crc, std.digest.md, std.digest.sha; import std.stdio; // Digests a file and prints the result. void digestFile(Digest hash, string filename) { File file = File(filename); //As digests implement OutputRange, we could use std.algorithm.copy //Let's do it manually for now foreach (buffer; file.byChunk(4096 * 1024)) hash.put(buffer); ubyte[] result = hash.finish(); writefln("%s (%s) = %s", typeid(hash).toString(), filename, toHexString(result)); } void umain(string[] args) { auto md5 = new MD5Digest(); auto sha1 = new SHA1Digest(); auto crc32 = new CRC32Digest(); foreach (arg; args[1 .. $]) { digestFile(md5, arg); digestFile(sha1, arg); digestFile(crc32, arg); } }
- struct
ExampleDigest
; - This documents the general structure of a Digest in the template API. All digest implementations should implement the following members and therefore pass the isDigest test.
Note
Examples://Using the OutputRange feature import std.algorithm.mutation : copy; import std.digest.md; import std.range : repeat; auto oneMillionRange = repeat!ubyte(cast(ubyte)'a', 1000000); auto ctx = makeDigest!MD5(); copy(oneMillionRange, &ctx); //Note: You must pass a pointer to copy! writeln(ctx.finish().toHexString()); // "7707D6AE4E027C70EEA2A935C2296F21"
- @trusted void
put
(scope const(ubyte)[]data
...); - Use this to feed the digest with data. Also implements the std.range.primitives.isOutputRange interface for ubyte and const(ubyte)[]. The following usages of
put
must work for any type which passes isDigest:Example
ExampleDigest dig; dig.put(cast(ubyte) 0); //single ubyte dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic ubyte[10] buf; dig.put(buf); //buffer
- @trusted void
start
(); - This function is used to (re)initialize the digest. It must be called before using the digest and it also works as a 'reset' function if the digest has already processed data.
- @trusted ubyte[16]
finish
(); - The finish function returns the final hash sum and resets the Digest.
Note The actual type returned by finish depends on the digest implementation. ubyte[16] is just used as an example. It is guaranteed that the type is a static array of ubytes.
- Use DigestType to obtain the actual return type.
- Use digestLength to obtain the length of the ubyte array.
- enum bool
isDigest
(T); - Use this to check if a type is a digest. See ExampleDigest to see what a type must provide to pass this check.
Note This is very useful as a template constraint (see examples)
Bugs:- Does not yet verify that put takes scope parameters.
- Should check that finish() returns a ubyte[num] array
Examples:import std.digest.crc; static assert(isDigest!CRC32);
Examples:import std.digest.crc; void myFunction(T)() if (isDigest!T) { T dig; dig.start(); auto result = dig.finish(); } myFunction!CRC32();
- template
DigestType
(T) - Use this template to get the type which is returned by a digest's finish method.Examples:
import std.digest.crc; assert(is(DigestType!(CRC32) == ubyte[4]));
Examples:import std.digest.crc; CRC32 dig; dig.start(); DigestType!CRC32 result = dig.finish();
- enum bool
hasPeek
(T); - Used to check if a digest supports the peek method. Peek has exactly the same function signatures as finish, but it doesn't reset the digest's internal state.
Note
- This is very useful as a template constraint (see examples)
- This also checks if T passes isDigest
Examples:import std.digest.crc, std.digest.md; assert(!hasPeek!(MD5)); assert(hasPeek!CRC32);
Examples:import std.digest.crc; void myFunction(T)() if (hasPeek!T) { T dig; dig.start(); auto result = dig.peek(); } myFunction!CRC32();
- template
hasBlockSize
(T) if (isDigest!T) - Checks whether the digest has a blockSize member, which contains the digest's internal block size in bits. It is primarily used by std.digest.hmac.HMAC.Examples:
import std.digest.hmac, std.digest.md; static assert(hasBlockSize!MD5 && MD5.blockSize == 512); static assert(hasBlockSize!(HMAC!MD5) && HMAC!MD5.blockSize == 512);
- DigestType!Hash
digest
(Hash, Range)(auto ref Rangerange
)
if (!isArray!Range && isDigestibleRange!Range); - This is a convenience function to calculate a hash using the template API. Every digest passing the isDigest test can be used with this function.Parameters:
Range range
an InputRange with ElementType ubyte, ubyte[] or ubyte[num] Examples:import std.digest.md; import std.range : repeat; auto testRange = repeat!ubyte(cast(ubyte)'a', 100); auto md5 = digest!MD5(testRange);
- DigestType!Hash
digest
(Hash, T...)(scope const Tdata
)
if (allSatisfy!(isArray, typeof(data
))); - This overload of the digest function handles arrays.Parameters:
T data
one or more arrays of any type Examples:import std.digest.crc, std.digest.md, std.digest.sha; auto md5 = digest!MD5( "The quick brown fox jumps over the lazy dog"); auto sha1 = digest!SHA1( "The quick brown fox jumps over the lazy dog"); auto crc32 = digest!CRC32("The quick brown fox jumps over the lazy dog"); writeln(toHexString(crc32)); // "39A34F41"
Examples:import std.digest.crc; auto crc32 = digest!CRC32("The quick ", "brown ", "fox jumps over the lazy dog"); writeln(toHexString(crc32)); // "39A34F41"
- char[digestLength!Hash * 2]
hexDigest
(Hash, Order order = Order.increasing, Range)(ref Rangerange
)
if (!isArray!Range && isDigestibleRange!Range); - This is a convenience function similar to digest, but it returns the string representation of the hash. Every digest passing the isDigest test can be used with this function.Parameters:
order the order in which the bytes are processed (see toHexString) Range range
an InputRange with ElementType ubyte, ubyte[] or ubyte[num] Examples:import std.digest.md; import std.range : repeat; auto testRange = repeat!ubyte(cast(ubyte)'a', 100); writeln(hexDigest!MD5(testRange)); // "36A92CC94A9E0FA21F625F8BFB007ADF"
- char[digestLength!Hash * 2]
hexDigest
(Hash, Order order = Order.increasing, T...)(scope const Tdata
)
if (allSatisfy!(isArray, typeof(data
))); - This overload of the hexDigest function handles arrays.Parameters:
order the order in which the bytes are processed (see toHexString) T data
one or more arrays of any type Examples:import std.digest.crc; // "414FA339" writeln(hexDigest!(CRC32, Order.decreasing)("The quick brown fox jumps over the lazy dog"));
Examples:import std.digest.crc; // "414FA339" writeln(hexDigest!(CRC32, Order.decreasing)("The quick ", "brown ", "fox jumps over the lazy dog"));
- Hash
makeDigest
(Hash)(); - This is a convenience function which returns an initialized digest, so it's not necessary to call start manually.Examples:
import std.digest.md; auto md5 = makeDigest!MD5(); md5.put(0); writeln(toHexString(md5.finish())); // "93B885ADFE0DA089CDF634904FD59F71"
- interface
Digest
; - This describes the OOP API. To understand when to use the template API and when to use the OOP API, see the module documentation at the top of this page.The Digest interface is the base interface which is implemented by all digests.
Note A Digest implementation is always an OutputRange
Examples://Using the OutputRange feature import std.algorithm.mutation : copy; import std.digest.md; import std.range : repeat; auto oneMillionRange = repeat!ubyte(cast(ubyte)'a', 1000000); auto ctx = new MD5Digest(); copy(oneMillionRange, ctx); writeln(ctx.finish().toHexString()); // "7707D6AE4E027C70EEA2A935C2296F21"
Examples:import std.digest.crc, std.digest.md, std.digest.sha; ubyte[] md5 = (new MD5Digest()).digest("The quick brown fox jumps over the lazy dog"); ubyte[] sha1 = (new SHA1Digest()).digest("The quick brown fox jumps over the lazy dog"); ubyte[] crc32 = (new CRC32Digest()).digest("The quick brown fox jumps over the lazy dog"); writeln(crcHexString(crc32)); // "414FA339"
Examples:import std.digest.crc; ubyte[] crc32 = (new CRC32Digest()).digest("The quick ", "brown ", "fox jumps over the lazy dog"); writeln(crcHexString(crc32)); // "414FA339"
Examples:void test(Digest dig) { dig.put(cast(ubyte) 0); //single ubyte dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic ubyte[10] buf; dig.put(buf); //buffer }
- abstract nothrow @trusted void
put
(scope const(ubyte)[]data
...); - Use this to feed the digest with data. Also implements the std.range.primitives.isOutputRange interface for ubyte and const(ubyte)[].
Example
void test(Digest dig) { dig.put(cast(ubyte) 0); //single ubyte dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic ubyte[10] buf; dig.put(buf); //buffer }
- abstract nothrow @trusted void
reset
(); - Resets the internal state of the digest.
- abstract const nothrow @property @trusted size_t
length
(); - abstract nothrow @trusted ubyte[]
finish
();
abstract nothrow ubyte[]finish
(ubyte[]buf
); - The finish function returns the hash value. It takes an optional buffer to copy the data into. If a buffer is passed, it must be at least length bytes big.
- final nothrow @trusted ubyte[]
digest
(scope const(void[])[]data
...); - This is a convenience function to calculate the hash of a value using the OOP API.
- enum
Order
: bool; - See toHexStringExamples:
import std.digest.crc : CRC32; auto crc32 = digest!CRC32("The quick ", "brown ", "fox jumps over the lazy dog"); writeln(crc32.toHexString!(Order.decreasing)); // "414FA339" writeln(crc32.toHexString!(LetterCase.lower, Order.decreasing)); // "414fa339"
increasing
decreasing
- char[num * 2]
toHexString
(Order order = Order.increasing, size_t num, LetterCase letterCase = LetterCase.upper)(const ubyte[num]digest
);
char[num * 2]toHexString
(LetterCase letterCase, Order order = Order.increasing, size_t num)(in ubyte[num]digest
);
stringtoHexString
(Order order = Order.increasing, LetterCase letterCase = LetterCase.upper)(in ubyte[]digest
);
stringtoHexString
(LetterCase letterCase, Order order = Order.increasing)(in ubyte[]digest
); - Used to convert a hash value (a static or dynamic array of ubytes) to a string. Can be used with the OOP and with the template API.The additional order parameter can be used to specify the order of the input data. By default the data is processed in increasing order, starting at index 0. To process it in the opposite order, pass Order.decreasing as a parameter. The additional letterCase parameter can be used to specify the case of the output data. By default the output is in upper case. To change it to the lower case pass LetterCase.lower as a parameter.
Note The function overloads returning a string allocate their return values using the GC. The versions returning static arrays use pass-by-value for the return value, effectively avoiding dynamic allocation.
Examples:import std.digest.crc; //Test with template API: auto crc32 = digest!CRC32("The quick ", "brown ", "fox jumps over the lazy dog"); //Lower case variant: writeln(toHexString!(LetterCase.lower)(crc32)); // "39a34f41" //Usually CRCs are printed in this order, though: writeln(toHexString!(Order.decreasing)(crc32)); // "414FA339" writeln(toHexString!(LetterCase.lower, Order.decreasing)(crc32)); // "414fa339"
Examples:import std.digest.crc; // With OOP API auto crc32 = (new CRC32Digest()).digest("The quick ", "brown ", "fox jumps over the lazy dog"); //Usually CRCs are printed in this order, though: writeln(toHexString!(Order.decreasing)(crc32)); // "414FA339"
- class
WrapperDigest
(T) if (isDigest!T): Digest; - Wraps a template API hash struct into a Digest interface. Modules providing digest implementations will usually provide an alias for this template (e.g. MD5Digest, SHA1Digest, ...).Examples:
import std.digest.md; //Simple example auto hash = new WrapperDigest!MD5(); hash.put(cast(ubyte) 0); auto result = hash.finish();
Examples://using a supplied buffer import std.digest.md; ubyte[16] buf; auto hash = new WrapperDigest!MD5(); hash.put(cast(ubyte) 0); auto result = hash.finish(buf[]); //The result is now in result (and in buf). If you pass a buffer which is bigger than //necessary, result will have the correct length, but buf will still have it's original //length
- this();
- Initializes the digest.
- nothrow @trusted void
put
(scope const(ubyte)[]data
...); - Use this to feed the digest with data. Also implements the std.range.primitives.isOutputRange interface for ubyte and const(ubyte)[].
- nothrow @trusted void
reset
(); - Resets the internal state of the digest.
- const pure nothrow @property @trusted size_t
length
(); - nothrow ubyte[]
finish
(ubyte[]buf
);
nothrow @trusted ubyte[]finish
(); - The finish function returns the hash value. It takes an optional buffer to copy the data into. If a buffer is passed, it must have a length at least length bytes.
Example
import std.digest.md; ubyte[16] buf; auto hash = new WrapperDigest!MD5(); hash.put(cast(ubyte) 0); auto result = hash.finish(buf[]); //The result is now in result (and in buf). If you pass a buffer which is bigger than //necessary, result will have the correct length, but buf will still have it's original //length
- const @trusted ubyte[]
peek
(ubyte[]buf
);
const @trusted ubyte[]peek
(); - Works like finish but does not reset the internal state, so it's possible to continue putting data into this WrapperDigest after a call to peek.These functions are only available if hasPeek!T is true.
- bool
secureEqual
(R1, R2)(R1r1
, R2r2
)
if (isInputRange!R1 && isInputRange!R2 && !isInfinite!R1 && !isInfinite!R2 && (isIntegral!(ElementEncodingType!R1) || isSomeChar!(ElementEncodingType!R1)) && !is(CommonType!(ElementEncodingType!R1, ElementEncodingType!R2) == void)); - Securely compares two digest representations while protecting against timing attacks. Do not use == to compare digest representations.The attack happens as follows:
- An attacker wants to send harmful data to your server, which requires a integrity HMAC SHA1 token signed with a secret.
- The length of the token is known to be 40 characters long due to its format, so the attacker first sends "0000000000000000000000000000000000000000", then "1000000000000000000000000000000000000000", and so on.
- The given HMAC token is compared with the expected token using the == string comparison, which returns false as soon as the first wrong element is found. If a wrong element is found, then a rejection is sent back to the sender.
- Eventually, the attacker is able to determine the first character in the correct token because the sever takes slightly longer to return a rejection. This is due to the comparison moving on to second item in the two arrays, seeing they are different, and then sending the rejection.
- It may seem like too small of a difference in time for the attacker to notice, but security researchers have shown that differences as small as 20µs can be reliably distinguished even with network inconsistencies.
- Repeat the process for each character until the attacker has the whole correct token and the server accepts the harmful data. This can be done in a week with the attacker pacing the attack to 10 requests per second with only one client.
Parameters:R1 r1
A digest representation R2 r2
A digest representation Returns:true if both representations are equal, false otherwiseSee Also:Examples:import std.digest.hmac : hmac; import std.digest.sha : SHA1; import std.string : representation; // a typical HMAC data integrity verification auto secret = "A7GZIP6TAQA6OHM7KZ42KB9303CEY0MOV5DD6NTV".representation; auto data = "data".representation; auto hex1 = data.hmac!SHA1(secret).toHexString; auto hex2 = data.hmac!SHA1(secret).toHexString; auto hex3 = "data1".representation.hmac!SHA1(secret).toHexString; assert( secureEqual(hex1[], hex2[])); assert(!secureEqual(hex1[], hex3[]));