SecurEntity integrity verification failed

Jan 31 at 9:58 PM
Edited Feb 1 at 1:56 PM
I am running into a "SecurEntity integrity verification failed" exception, from Decrypt() => ch.Verify(). This happens when I move a database from one machine to another. I am using the exact same .pfx on both machines, both with the SecurEntity friendly name.

I looked at the code and I don't see anything that would be machine-dependent. The HMAC is generated using the secret key, which should be identical on both machines since I'm using the same .pfx.

Is there something machine-dependent that I'm missing here?

[ EDIT ]

After further investigation I have determined the secretKey is different on each machine even though the identical certificate is being used. Examining the code in SecretKeyStorage.cs, it appears that the only "variable" used in generation of the secret key is KeyContainerName in the cspParams:
CspParameters cspParams = new CspParameters(
    24,
    "Microsoft Enhanced RSA and AES Cryptographic Provider",
    rsa.CspKeyContainerInfo.KeyContainerName);
Just for kicks, I decided to set KeyContainerName to a fixed string, expecting that the resulting secret key would be the same on both machines. However, this was not the case, and the secret key was again different:
Local Machine:
Certificate found: SecurEntity
sha.Hash:2916xxxxx39B9
rsa.CspKeyContainerInfo.KeyContainerName: {4E1xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx75D}
secretKey: 6C4ExxxxxB173

Server Machine:
Certificate found: SecurEntity
sha.Hash: 2916xxxxx39B9
rsa.CspKeyContainerInfo.KeyContainerName: {4E1xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx75D}
secretKey: 5CCAxxxxxF777
I have also verified that the certificates on each machine are identical:

Image


I'm at a loss... there must be something else I'm overlooking. However, my ultimate question is: is it possible generate the secret key deterministically across machines?

Thanks.
Coordinator
Feb 4 at 2:09 AM
Are the two host servers running the exact same version of Windows, as well as of the .NET Framework?
Feb 4 at 2:19 AM
Edited Feb 4 at 3:01 AM
Both on .NET 4.5, however one machine is Windows Server 2008 and the other is Windows 8.

Also, I don't know if it's related, but I'm seeing some strange behavior on my production server related to Verify() (my dev environment works fine). On the production server Verify() fails when the application runs as the IIS user (IIS 7), but works fine when running under my account (via impersonation). So far I've determined that the secretKey generated by
secretKey = rsa.SignData(sha.Hash, "SHA512");
is different when the application runs as the IIS user vs under my account, even though the sha.Hash input is identical.

I don't get it... seems like the IIS user may not have permission to the underlying Crypto API, but I'm not getting any exceptions, just an incorrect secretKey. I've confirmed that the IIS user has access to the certificate and the keys are being read correctly.

Thanks.
Coordinator
Feb 4 at 8:16 PM
Edited Feb 4 at 8:52 PM
On the host where you're running the lookup as IIS user, into what certificate store did you import your SecurEntity certificate? As you know, by default, SecurEntity looks for it in the current user MY store, but that approach can be less natural when you're using the default app pool account, in which case it can make more sense to use the computer store.

Similarly for the host where you're running the lookup under your account - you used the same PFX, per above, and you imported it into your user MY store?

Ah - also, what's the RSA key size of your SecurEntity certificate?

(For what it's worth, per section 8.1 in http://tools.ietf.org/html/rfc2313, for a private key operation - such as signing, which is what we're doing to derive the master key - the resulting encryption block should be constant for constant input data. That's the assumption being made by our master key derivation scheme, and it seems be bear out for a single key holder. Multiple key holder appears to be causing something weird.)
Feb 4 at 10:37 PM
The SecurEntity certificate is in the computer (LocalMachine) store, for the very reason that you stated. It is on that same production machine where I use the certificate under my account, and it's the exact same pfx. I have also looked at the permissions on the file itself (C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys) and verified that the IIS user has permissions (in fact, I gave it Full Control permissions instead of Read Only, just to make sure it could access the cert).

I have tried various certificates. I created a 2048-bit key using openssl, and also followed Using Makecert to Create Certificates for Development instructions to create a certificate and a CA certificate). Still, no luck :(

I've done quite a bit of debugging on this and it seems to come down to the rsa.SignData() resulting in different secret keys, yielding unequal hmacs and causing Verify() to fail. I believe the certificate is being read correctly under both mine and the IIS user accounts. Here is a comparison of my account vs. the IIS user account. As you can see it's all identical up to the SHA1 Hash:
MY USER ACCOUNT
===============
Certificate:
    Content Type:          Cert
    Friendly Name:         SecurEntity
    Certificate Verified?: False
    Simple Name:           SecurEntity
    Signature Algorithm:   sha1RSA
    Certificate Archived?: False
    Length of Raw Data:    828

Private Key:
    <RSAKeyValue><Modulus>7jypT19ztiJ+eINd6tTS89l7dU0kLSr0sLLN9MG0jR+cxe06N4ujYegIFo
    JPoY01gD1xVIs1znJCf50xod9roycx797khTau0nAR1HF/b++L5svvxvL+5cuZQebSPhZGaWCdByTM0m
    [deleted for brevity]
    eZsiB0zguysAY9ylgecHoBM4aGfvoA1whIzdoWdBzV8xcZBrtP09QuiYaiQ==</D></RSAKeyValue>

Public Key:
    <RSAKeyValue><Modulus>7jypT19ztiJ+eINd6tTS89l7dU0kLSr0sLLN9MG0jR+cxe06N4ujYegIFoJPoY
    01gD1xVIs1znJCf50xod9roycx797khTau0nAR1HF/b++L5svvxvL+5cuZQebSPhZGaWCdByTM0mLxa+Lnnk
    lL6x8apkbpZEkY9dGVkgiinlp6JcMo7l4bEk1ONVIAaTTuPeMkMKmcypr6BJpBg6h0opie0k9vxfO3wKc07m
    Vr21qkcwLiW+TdV1Bvmqftc9PM4DVRdsmFsMBr3UCT7JO3tZav+OPY7DSuliZDbLuoN0oPke058WgW8AP+iI
    NqrejKiUQOmUsx7Mu60PnbI39Gbw==</Modulus><Exponent>AQAB</Exponent></RSAKeyValue>

CspKeyContainerInfo:   
    Accessible property: True   
    Exportable property: True   
    HardwareDevice property: False   
    KeyContainerName property: {5AFEE6A5-5005-426C-91EA-06F8F481B89F}   
    KeyNumber property: Exchange   
    MachineKeyStore property: False   
    Protected property: False   
    ProviderName property: Microsoft Enhanced RSA and AES Cryptographic Provider   
    ProviderType property: 24  
    RandomlyGenerated property: False   
    Removable property: False   
    UniqueKeyContainerName property: 90c1da97fce189cd2a3d3a93891c45d1_5337ca56-8505-431d-a6aa-a11897f559fc

SHA Hash:
    29163BBEC2FD47A43DA91E45772B7ECB321AEDB43A69A8437D00BCBD9C0A22FD4A95F7847E95CCF5BCA06F30EC5A5095649BAEFAB23F7007ACD50A91A0A239B9

Secret Key:
    C905F4D85002E972D6C4FCE9F723D7C89CB0D2FF6FA6A7F4111FCDAAC888CB8246711A323ED981C4
    7B37733076C2E4A67E403533147A95DEEDB3D3AC3B6A87617592E258A88EAFEF73BA0765E1B0FC39
    6BF124A1A2D35388DDFF2E4AA87A70E6D76CDA5BBA38D7B793770FBBDDED57F59B6C4ABFB28FA657
    83DCFC4DADEEA88B


IIS USER
========
Certificate:
    Content Type:          Cert  
    Friendly Name:         SecurEntity  
    Certificate Verified?: False  
    Simple Name:           SecurEntity  
    Signature Algorithm:   sha1RSA  
    Certificate Archived?: False  
    Length of Raw Data:    828  

Private Key:            
    <RSAKeyValue><Modulus>7jypT19ztiJ+eINd6tTS89l7dU0kLSr0sLLN9MG0jR+cxe06N4ujYegIFo
    JPoY01gD1xVIs1znJCf50xod9roycx797khTau0nAR1HF/b++L5svvxvL+5cuZQebSPhZGaWCdByTM0m
    [deleted for brevity]
    eZsiB0zguysAY9ylgecHoBM4aGfvoA1whIzdoWdBzV8xcZBrtP09QuiYaiQ==</D>
    </RSAKeyValue>  

Public Key:             
    <RSAKeyValue><Modulus>7jypT19ztiJ+eINd6tTS89l7dU0kLSr0sLLN9MG0jR+cxe06N4ujYegIFoJPoY
    01gD1xVIs1znJCf50xod9roycx797khTau0nAR1HF/b++L5svvxvL+5cuZQebSPhZGaWCdByTM0mLxa+Lnnk
    lL6x8apkbpZEkY9dGVkgiinlp6JcMo7l4bEk1ONVIAaTTuPeMkMKmcypr6BJpBg6h0opie0k9vxfO3wKc07m
    Vr21qkcwLiW+TdV1Bvmqftc9PM4DVRdsmFsMBr3UCT7JO3tZav+OPY7DSuliZDbLuoN0oPke058WgW8AP+iI
    NqrejKiUQOmUsx7Mu60PnbI39Gbw==</Modulus><Exponent>AQAB</Exponent></RSAKeyValue>  

CspKeyContainerInfo:   
    Accessible property: True   
    Exportable property: True   
    HardwareDevice property: False   
    KeyContainerName property: {5AFEE6A5-5005-426C-91EA-06F8F481B89F}   
    KeyNumber property: Exchange   
    MachineKeyStore property: False   
    Protected property: False   
    ProviderName property: Microsoft Enhanced RSA and AES Cryptographic Provider   
    ProviderType property: 24   
    RandomlyGenerated property: False   
    Removable property: False   
    UniqueKeyContainerName property: 90c1da97fce189cd2a3d3a93891c45d1_5337ca56-8505-431d-a6aa-a11897f559fc
    
SHA Hash:
    29163BBEC2FD47A43DA91E45772B7ECB321AEDB43A69A8437D00BCBD9C0A22FD4A95F7847E95CCF5BCA06F30EC5A5095649BAEFAB23F7007ACD50A91A0A239B9    

Secret Key:                         
    750EC87A066EDBAB02701132F86178623BEB860EA6AC501E136313DB40D4D2473E483325529F6
    41D45677EC7068129C42B5C42E520233ACFC8CCF7E45045DEC81A83948645F9CFDFFFCE904D30
    E6E07B9C1692B89F37D6A741CA6BB99147749D5776A230EDB12B847F3AA057810115F312301EB
    26864D75BE26B93CDCF5FABC8  

hmacResult=   72C28CFDF73EA631C4CE233A6CE86223763F330DE9F8613EA771ABAB01239D2773EC321600876913D891F1FBB734AC070CD20609BCFE53AFF4F06A55E5C95449 
hmac512.Hash= 6AFC1A5D27920FCB5B591A97F7A0573590159F4721F682BD507D652B114883223272A9883C4E91ABEE6E3B00040E74B6DDE511E9D4481333D2D779AFB58A2DC6  
    
My server admin and I have been working on this for 3 days... we don't know where to go from here. By everything we have read and tried, it should be working.

Dan -- thanks for your help, I greatly appreciate it.
Coordinator
Feb 6 at 3:08 AM
This is fixed in 1.6.
Feb 6 at 11:35 AM
Edited Feb 6 at 1:24 PM
Great, works much better -- same key no matter the user or machine as long as the same certificate is used.

However... now I'm getting an exception when I change a previously-encrypted entity (e.g. edit a name or email address). When trying to decrypt this re-encrypted entity the exception is raised. This is in the dev environment, and was working prior to v1.6:

Image

After some Google research I thought this was because the BlockSize was not being set, so I added that to _DeriveCipher():
        private void _DeriveCipher()
        {
            SHA512Managed sha = new SHA512Managed();
            sha.TransformBlock(secretKey, 0, secretKey.Length, secretKey, 0);
            sha.TransformFinalBlock(cipherSalt, 0, cipherSalt.Length);

            aes = new AesManaged();
            aes.KeySize = (int)aesKeyBits;
            aes.BlockSize = (int)cipherBlockLength * 8;
            aes.IV = IV;
            aes.Key = new byte[aesKeyBits / 8];
            Array.Copy(sha.Hash, aes.Key, aes.Key.Length);
            aes.Mode = CipherMode.CBC;
        }
...but I stll got the exception. I also noticed that aes.Key always contains 0's after the Array copy.

Any idea what's going on? Again, this is only when changing a previously-encrypted entity... the initial encryption is fine.

Thanks.
Coordinator
Feb 6 at 3:58 PM
Can you try switching to SHA-1 temporarily in the code above? The SHA512Managed class seems to be doing some undocumented things.

Note that in the Array.Copy you may need to reuse some of the bytes of the SHA-1 result in order to fill the whole Key buffer.
Feb 6 at 5:15 PM
Same exception as above. Here is my code so you can verify I made the change and copied the buffer correctly:
        private void _DeriveCipher()
        {
            SHA1Managed sha = new SHA1Managed();
            sha.TransformBlock(secretKey, 0, secretKey.Length, secretKey, 0);
            sha.TransformFinalBlock(cipherSalt, 0, cipherSalt.Length);

            aes = new AesManaged();
            aes.KeySize = (int)aesKeyBits;
            aes.BlockSize = (int)cipherBlockLength * 8;
            aes.IV = IV;
            aes.Key = new byte[aesKeyBits / 8];
            //Array.Copy(sha.Hash, aes.Key, aes.Key.Length);

            int quot = aes.Key.Length / sha.Hash.Length;
            int rem = aes.Key.Length % sha.Hash.Length;
            int round;
            for (round = 0; round < quot; round++)
            {
                Array.Copy(sha.Hash, 0, aes.Key, round * sha.Hash.Length, sha.Hash.Length);
            }
            Array.Copy(sha.Hash, 0, aes.Key, round * sha.Hash.Length, rem);

            aes.Mode = CipherMode.CBC;
        }