MI-80

National Institute of Standards and Technology (NIST) is at it again. After the last competition to develop a new encryption standard to replace the aging Data Encryption Standard (DES), NIST's in full swing hosting the forum to develop a next-generation hash function.

Hash functions form the backbone of modern information security by letting you create unique digital fingerprints of your data. Encryption ensures no one else can read your data, but hashes allow you to validate who is on the other side of the conversation and to tell if someone alters your data. However, amazingly, hashes have never undergone the scrutiny that encryption algorithms have. The most common hash functions today were developed by private companies or by government agencies like the National Security Agency developed SHA family of functions. A drawn out, public competition has never been held for hashes.

This began to change in November when NIST began reviewing entries for SHA-3 to replace the current family of SHA functions. SHA-1 is in common use today due to security issues with the original SHA-0, and several offshoots have developed to address rising weaknesses in SHA-1, named simply for their extended bit length, such as SHA-192 and SHA-256. While SHA-1 is still secure for the near future, major breaks in another popular hash function, MD5, including a well-publicized flaw that permitted the falsifying of web site SSL certificates, has brought the need for a strong, peer-reviewed, hash function into focus.

The goal of this competition is simple: accept entries from all around the world and spend the next several years allowing the entire cryptographic community to attack them and weed out the weakest, culminating with one "winner" to become the new international standard. Many expect the contest to heat up as everything from secure password storage to ensuring safe delivery of updates from operating system and anti-virus vendors will utilize the winning algorithm for decades to come.

The competition is already drawing an amazing number of entries. While NIST’s earlier encryption competition brought in 15 candidates, as of the October 31 deadline 64 submissions were presented for this new hash challenge. Of those, 28 have been opened to the public and six have already been broken outright. Fortify Software also just announced their results showing several others to suffer from security, performance, and stability issues in their reference implementations.

Initial hopes were to eliminate about a dozen entries during this first round of competition, but analysis has already narrowed the field down by 22. The community will cryptanalyze the remaining functions throughout 2009, permitting another major cut sometime in 2010, though some would like to see the field down to 15 or 20 entries by as early as this summer. After another year of analysis NIST will chose a winner in 2011, with them formally finalizing the standard for adoption in 2012.

One thing I learned from the competition a decade ago is to not bet jump the gun and use anything before NIST finalizes the standard. The winner of the Advanced Encryption Standard (AES) competition, Rijndael, was actually an underdog in many analysts’ eyes. NIST chose arguably what was not the most secure algorithm, but the best combination of security, performance, and other factors. Major attention was paid to Rijndael’s speed and ease of implementation in hardware, and this will likely hold true with the new hash standard, especially since multi-core processors open so many new doors to designers. Not only do you not want to be left with an abandoned standard, but choosing to implement one too early may only end up as an issue down the line when someone discovers a major security flaw. So in short, wait for the final standard and stick with the higher-level SHA algorithms in the mean time. Despite breaks showing that some hashes are broken and some are weaker than once thought, there is no reason to panic. If you are still using MD5 for any reason, like some lingering VPN tunnels or elsewhere, migrate those to the SHA family, where they are (likely) safe while we wait on the next generation hash function.