C# Cautionary Tale — The dangers of SHA256 reuse
Update: Based on feedback by Thibaut Le Guilly, it seems my answer can be simplified using the ThreadStatic attribute. See here.
Original: I’ve been working on a spare-time project that hashes some data using SHA256; nothing exciting or particularly groundbreaking. However, during the course of this project I noticed that every now and then, the results from my hashing algorithm were not what I expected.
The software I am writing is really simple. It stores a record of data in Azure Table Storage. For every row inside Table Storage you need a Partition Key and a Row key. In my implementation I’m hashing an input value and using the resulting hash as my Partition Key. So far so good.
Recently I noticed that sometimes the software would warn that some records couldn’t be found but after a refresh, they re-appeared. Odd.
A bit of logging later (and using some carefully crafted telemetry statements) I could see that in some circumstances the hashing algorithm I was using produced slightly different outputs!
The code I was using for my hasher can be found here.
As you can see, I use the standard Dot Net Core SHA256 algorithm but when coding I decided that by storing my SHA256 object for reuse I would somehow save some CPU cycles by not recreating it over and over again.
It turns out, I was wrong.
I experimented with a simple console app and confirmed my suspicions. By reusing the engine I was corrupting the state; so when used concurrently, the results would be different.
In my sample app I tried 2 things.
- Using parallel tasks, try 10 hashes (using a shared engine)
- Using parallel tasks, try 10 hashes (using a new engine every time)
The results were as follows. Of the 10 attempts using the shared engine, 7 attempts produced different hash values. When using the new engine instance, all of the hashes for successful.
As usual, a few lines of code and a major problem is fixed. For reference here is my new hashing code.
So the moral of this tale, use a new engine every time or it’ll bite you.
