Steve Bellovin has a post The Uses and Abuses of Cryptography in which he comments on the recent Anthem data breach. At Anthem, supposedly, the database of important stuff like customer addresses and social security numbers was not encrypted, because it was in use all the time.Steve says, “If your OS is secure, you don’t need the crypto; if it’s not, the crypto won’t protect your data.”
His point is that the decryption keys have to be in RAM somewhere for the system to work, so if the OS is insecure, the keys can be stolen, and the encrypted database decrypted anyway. This is not necessarily true. IBM (see, for example IBM PCIe Cryptographic Coprocessor) http://www-03.ibm.com/security/cryptocards/pciecc/overview.shtml) and others make hardware units that provide encryption and decryption, and store the master keys. With appropriate hardware, the keys are NOT in RAM and can’t be stolen. This still isn’t enough, because a compromised host system can still command the crypto box to decrypt the data. To go further, you have to have velocity checks inside the trusted part of the system, to alarm on and halt unexpected volumes of traffic.
The data itself also has to be carefully organized. Each record has to have its own key. Record keys are stored in the database encyrpted by a master key, which is only stored in the cryptographic hardware module.
It is probably going to be impossible to prevent theft of individual records. An insider can always photograph data off the screen in a call center. I think we can do much better about technical means to prevent bulk data breaches.
There is a whole new area of research on how to make cloud computing trustworthy. How can you get anything done when your code is running on potentially compromised hardware or on a virtual machine pwned by the bad guys? It might be possible! Homomorphic encryption makes it possible to perform computations on encrypted data, and perhaps cloud servers will at least come with cryptographic modules that at least can limit the rate at which your data can be stolen.
Update: February 24, 2015
Steve points out via email that many kinds of tasks, such as a batch job generating annual statements, have to touch all records, so rate-limiting (velocity checks) might not be effective.