Using Cryptlib - Encryption Based On Solid Foundations
After a decent number of weeks of intense coding and testing, I have finally finished my latest program clAES (sig). This work is the result of my attempt to use Cryptlib as the foundation for file encryption (coded in Python3) which does one thing and only that, safely encrypt or decrypt a file with password-based AES.
And nothing more.
File encryption is long solved. Why clAES?
Talking about my project while still in progress, I had been confronted with the obvious question "why not simply use gpg" and the advice "use openssl directly, if you are brave". These objections to even start my now finished project are legitimate and demand a convincing reply.
Both GnuPG and OpenSSL basically rely on the use of two different shared libraries that provide the basic functions for encryption and decryption of data. As "libgcrypt" for gpg and "libcrypto" for OpenSSL both comprise of a vast amount of functions with great complexity, the correct use and safe handling of these functions fall into the hands of the C-programmer, who often does not have the cryptographic expertise nor the experience needed to safely apply the library functions to their own code without the risk of coding something that ends in disaster. It is very likely that shooting oneself in the foot unintentionally will be the result of trying to add security to a project.
More often than not, the prospect to ruin the objective of safeguarding his own code by using cryptography may put him off using cryptography at all. As a result, the code is unsafe to use and could be much more reliable and privacy-protecting if there was a way to use encryption without the risk of doing something nasty unintentionally.
Cryptlib - The Well-designed Security Architecture
Obviously the last thing that is needed is another collection of crypto-functions. And Cryptlib, which is developed and maintained by Peter Gutmann, is so much more than just another library of functions. Cryptlib is designed to be integrated into any program that needs security features in a way that it makes unsafe use of cryptography almost impossible.
More than 20 years ago, Peter Gutmann described a security architecture in his PhD thesis at the University of Auckland, which had become the foundation for Cryptlib as it is available in different programming languages (including Python) today.
It is quite difficult to describe Cryptlib in a few sentences. But what sets it apart from a collection of functions is its design. In layman's words, the core of the software is a security kernel which is completely isolated from the code that initiates the desired cryptographic actions. The security kernel communicates with the code that uses the library in a way that sensitive information does never leave the kernel until the task is completed and only safe access to processes in the security kernel will be allowed from the outside. The objects inside the security kernel interact with one another based on principles that can be verified to ensure safe operation. Secure defaults are used at all times without the need to produce certain parameters by the programmer who uses Cryptlib. All internal methods are safeguarded by strict parameter checking so that no malformed input can jeopardize the functionality and secure operation inside the architecture.
Such a well-designed system also provides methods of access to its functionality that can be used by inexperienced programmers. This ensures that every project in need of reliable cryptography can be extended by the use of Cryptlib.
With this extraordinary foundation at hand, I started to code a message encryption program that works reliably and is able to interact with GnuPG and OpenSSL. If you use only one of them, the resulting encrypted cipher texts are stored in a format that cannot be interchanged with the other. GnuPG aims to implement the OpenPGP standard with various degrees of success in a format that consists of packets described in RFC-4880, while OpenSSL (accessed via the command line tools) can produce cipher text in CMS-format or in its own format. As it is seen so often, both worlds (although using AES) cannot talk to each other easily.
But clAES is made to talk to both, as it can produce OpenPGP messages (as the default) and additionally switch to CMS or OpenSSL message formats if need be. In any case the input data is encrypted with a passphrase that the user provides to the program and AES will be used as the default encryption method.
Facing Unexpected Obstacles
Although it is possible to use Cryptlib safely with very few lines of Python code, actually I encountered a number of hurdles standing in the way while finishing my program. Most of them I didn't expect.
The lessons I learned while coding clAES seem to me worth describing, so I hope my experience with Cryptlib-coding in Python may pave the way for those of you who have developed an appetite for using Cryptlib in your project by now.
Starting with a test program of only a few dozen lines I ended up with a finished clAES of some 1083 lines of Python code. Some of it of course were necessary to handle options selecting between the default OpenPGP message format and the other modes of operation. And a great part of the code base was due to handle safe input and output of text being read from the file system or standard input.
Then the input, which is always expected as ascii-armoured (or base64 encoded) data has a different structure in GnuPG and in OpenSSL. These structures had to be parsed and separated into blocks containing the naked plain texts or cipher texts.
Before I could start to encrypt or decrypt the data blocks, I had to code a checksum function for GnuPG messages (crc24) and a function to derive an AES sessionkey from the user's passphrase for OpenSSL (pbkdf2).
Finally, the rest of the program, some 400 lines, would deal with using Cryptlib (from line 624 to the end) for conventional encryption.
The definitive source of documentation
Before I draw you into the details of my program, I have to mention the excellent documentation of Cryptlib. In the 358 pages of this manual you will find explanations of everything that can be achieved with Cryptlib which goes way beyond mere conventional encryption. You don't have to digest everything in the manual before you can start using Cryptlib, but you'll find it a valuable source of information (including code examples) to come back to from time to time.By the way, there is a unix-style man page for clAES as well.
Starting to use Cryptlib Envelopes (High-Level)
Let's start with some peculiarities that are specific to the Python language.Everything that relates to OpenPGP messages can be done with envelopes. Envelopes are the Cryptlib objects that do not only initiate encryption but also produce output in the OpenPGP message format. The first step is to create an envelope of type CRYPT_FORMAT_PGP.
Envelope_object = cryptCreateEnvelope( cryptUser, CRYPT_FORMAT_PGP )
Envelope = int( Envelope_object )
One would expect that the first line would suffice. But in every function that subsequently uses this envelope object an integer value is expected as the first parameter of many functions (a handle to the envelope). So casting the object into an integer for further use (including destruction of the envelope object) is necessary in the Python language.
Try-Except needs to be used instead of status codes
The following lines will add a user-provided passphrase to the PGP-envelope by setting a string attribute of the envelope named CRYPT_ENVINFO_PASSWORD. Coding in the C language you would store the return code of the cryptlib function "cryptSetAttributeString" and check if the operation has succeeded by checking for (status == 0) or (status == CRYPT_OK). The program would proceed once the status is 0.In Python most functions don't return a status code, so it is essential that you check the success of each cryptlib function with a try-except block in which you can extract the status (as an integer) as well as a message (as a real string) for debugging purposes. Let me stress this important point: before exiting the program as a consequence of an unrecoverable error, you need to clean up the memory. Don't just exit the code without taking care that sensible information in memory is being randomized reliably.
try:
# add the encryption password to the envelope
cryptSetAttributeString( Envelope, CRYPT_ENVINFO_PASSWORD, password )
except CryptException as e :
status, message = e.args
if (status == CRYPT_ERROR_WRONGKEY) :
print("Error: " + message)
clean_envelope()
exit( ERR_WRONGKEY )
Strings are always bytearrays
Some attributes of envelopes are numbers for instance the maximum buffer size CRYPT_ATTRIBUTE_BUFFERSIZE. Other attributes like passwords are strings. Only that strings are not actually strings, because a string in Python has an encoding method, but in Cryptlib all strings must be interpreted as literal bytes. So wherever a string is needed, you have to use a bytearray() instead. Bytearrays can be extended with actual strings or they can be appended by adding a byte at a time. The len() function also works similar to strings as does the indexing.
Text input to the envelope has to be a bytearray() too, of course.
When the passphrase is added to the envelope and CRYPT_ENVINFO_DATASIZE is set to the length of the input data (which is only necessary with PGP-envelopes), the actual encryption is done by:
bytesCopied = cryptPushData( Envelope, Data )
cryptFlushData( Envelope )
bytesCopied = cryptPopData( Envelope, envelopedData, DataBufferSize )
Buffer = envelopedData[:bytesCopied]
# Buffer holds the encrypted data
I have deliberately omitted the try-except blocks and every safeguard I talked about above to highlight the essential function calls. The bytearray "Buffer" can be written to the file system as a byte stream after being base64 encoded and being decorated with the usual "-----Begin ..." lines. (And uhh, adding the CRC24 checksum, I forgot to mention)
s2k-Overkill
When OpenPGP messages are to be decrypted inside a PGP-envelope the encryption algorithm that had been used must be detected by Cryptlib. In addition to that, it has to be determined how often the envelope has to hash the passphrase internally (using the s2k function) to derive an AES session key from the user provided passphrase.In fact, messages that are produced by gpg2 demand a hilariously large number of hash iterations, so that Cryptlib has to put an end to this nonsense by allowing only some eight million iterations. This can cause the decryption of some gpg messages to fail, because enough is enough and from a security point of view it is not prudent to force the maximum allowed number of iterations on every software that attempts to decrypt gpg messages. Maybe in future Cryptlib will accept hilariously large s2k counts but at the moment there is a generous limit, that should not be exceeded by gpg2.
Using Crypt-Contexts (Low-Level)
I found that switching to OpenSSL messages, envelopes are not enough. I had to do something which, although it is possible, under normal circumstances has to be avoided as much as possible. I'm talking about resorting to low-level use of Cryptlib. To manipulate the decryption process at a low level, Cryptlib provides crypt-contexts that can be used as an alternative to envelopes.Obviously, crypt-contexts are more error-prone than envelopes, because the programmer has to use them in the cryptographically correct way, which is not easy as my code will show you.
crypt_object = cryptCreateContext( cryptUser , CRYPT_ALGO_AES )
AESContext = int ( crypt_object )Insted of an envelope you will now create a context that is prepared for applying the AES cipher to its input. Similar to envelopes, the AEScontext has a number of attributes that have to be set correctly in order to ensure proper encryption.
First of all, the session key (128 bits or 256 bits) as well as the initialisation vector (128 bits) need to be derived from the passphrase in advance. I had to code the function pbkdf2 for this task. But this function expects a salt value of 8 bytes that is really random and does not repeat. This in itself is a very complex issue, so I decided to use the method of getting real random numbers that Peter Gutmann describes in his manual on page 287.
Once everything is set and done, all necessary attributes can be set inside the AEScontext.
cryptSetAttribute( AESContext, CRYPT_CTXINFO_MODE, CRYPT_MODE_CBC )
cryptSetAttribute( AESContext, CRYPT_CTXINFO_KEYSIZE, AESblocksize )
cryptSetAttributeString( AESContext, CRYPT_CTXINFO_KEY, sessionkey )
cryptSetAttributeString( AESContext, CRYPT_CTXINFO_IV, iv )
# encrypt Data in the context
try:
status = cryptEncrypt( AESContext, Data )
except CryptException as e :
status, message = e.args
print_debug( "Encryption error while encrypting data ...")
...If the cryptEncrypt function is successful, the cipher text can be taken from the bytearray() "Data" that originally provided the clear text, because crypt-contexts always change data "in-place". No pop-function is necessary in crypt-contexts to extract the cipher text (or the plain text) as it would be used with envelopes.
The need for padding the clear text
But if you think that is it, you're wrong. Because the AES cipher is designed to work on 128 bit blocks of data (16 bytes at once) the input data, a bytearray, must have a specific length. The input needs to be a multiple of the block size, otherwise AES cannot work.To achieve this, all input has to be extended at the end to the exact number of bytes that are needed. Even input that already has a length of a multiple of the blocksize needs to be padded. There is a method that openssl uses to pad all messages (PKCS#7 padding) that appends a certain number of bytes to every input so that it reaches the desired length. For instance, if the input lacks seven bytes to be a multiple of 16 bytes, then the padding adds seven bytes of the value seven to the end. If it is nine, then nine bytes of value nine are added and if the length is already ok, sixteen bytes of value sixteen are added.
With this clever method every decrypted message can be chopped to its original length by reading the last byte of the decrypted message and chopping as many bytes off the plain text as the value in the last byte demands. This guarantees that the padding is removed reliably from the decrypted data after decryption.
Randomize all sensitive data before exiting
Errors, even unrecoverable errors can happen. Just think of the fact, that the provided passphrase is empty or does not produce the AES session key which had been used to encrypt the message. My code base for clAES shows 35 points where the program cannot continue and has to be aborted. In all these cases it would be irresponsible to just throw an exit() and be done. Sensible data like the password bytearray or clear text buffers are still present in memory. So the least you can do is to clear the buffers by filling them with random data.This is the task of the function "clean_envelope()" and "clean_context()" in addition to destroying the envelope or crypt-context.
Get Your Hands on Cryptlib
I hope you could gain some benefit out of my explanations. Most difficulties, I ran into, are owed to the fact that I tried to connect to two very different worlds of practical file encryption, GnuPG and OpenSSL. But for every project that uses cryptography it is essential to determine, who will get the final results of this project. And that involves hard thinking about the formats in which encrypted data is presented to ordinary users. No amount of scrutiny and invention is wasted on this issue.So let me finally tell you how you can get and use Cryptlib.
The direct way is visiting Peter Gutmann's Cryptlib download page. Here you'll find the original zipped archive of all files Peter releases from time to time. Once you have unzipped the archive, you are only a "make shared" away from building the shared library on your own OS yourself.
Another path is to use my Cryptlib Hub to download a recent version of the library together with bindings for Python, Perl and Java in a single RPM package or DEB package, that can be installed on many OS, that handle these software bundles. (Fedora, Centos, Ubuntu, etc.)
These packages will eventually contain all cryptlib-tools that I develop, including clAES of course.
If you happen to use the Fedora OS, everything mentioned above is available from the main repository.
As a last remark, if you have experience with using Cryptlib now or in future, I'd really like to hear from you.
Trying To Solve The Hen-And-Egg Problem
We're living in an online world that is severely harmed by universal surveillance.
Our every-day lives in this world depend on complex and pretty obscure services run by companies, that leave the ordinary internet user with little alternative to giving up their privacy.
In doing so, they also have substantially reduced our freedom, the freedom to share thoughts with other people confidentially.
It's time to take back this freedom and to have secure communications without the fear of surreptitious intrusions into our privacy. Secure communication is a right, not a generous concession. And it will only work, if you're going to take control over what happens yourself.
Why is it so hard to find a useful solution?
Ordinary users are no encryption experts, and they don't want to become an expert. Secure communication should be a matter of course for everyone.
On the other hand, there is no simple solution for secure communications ready to go. And the reason is simple: a solution is useful only, if it is secure, if it can defend against the numerous threats lurking around in the online world. To design such a secure solution is hard, because the threats are complex and constantly changing.
This fact makes people believe that they have to trust the internet giants to deliver and run secure communication for them, which they don't control nor understand or worse, that they have to send their messages unprotected. And many decide to do exactly this.
A well-designed encryption tool under your control
The Crypto Bone is here to change that.
People who have tried to use encryption often find themselves in the desperate situation of managing all the different and confusing encryption keys. Key management is a really hard job and should also better be done by a separate device, especially designed for that task.
We all know that computers we use today are complex things and can be attacked in may ways.
So it is prudent to delegate all the hard work of message encryption to a separate device - the Crypto Bone - that is prepared to work reliably as an isolated, well-designed tool with minimal complexity.
It is much easier to ensure that such a device encrypts your messages securely than your complex computer.
These were the two main reasons for me to develop the Crypto Bone.
In short, the Crypto Bone will make your life easier, because it offers you a choice, you didn't have before: to escape the universal surveillance in a way that you control yourself.
Encryption has to be both: Secure and Usable
This new approach to key management is what makes the Crypto Bone really easy-to-use. For any contact you wish to share messages in private, you'll need to provide an initial secret to the Crypto Bone only once.
After that the messages are automatically encrypted and sent to your correspondent. You won't have to bother about message keys from now on and you can send your messages safely through the Crypto Bone, which takes care of all the complex tasks, you won't spend a second thought about.
But how would you be sure that the Crypto Bone is really secure in its operation?
Well, the only answer is peer-review. The Crypto Bone has been designed to be as auditable as possible, it uses the OpenBSD operating system and one single high-quality crypto library (CryptLib, maintained by Peter Gutmann) as the core crypto engine. And I have invited security experts from day one of the development to scrutinize the source code.
The Hen-And-Egg Problem
But as far as I know, this peer-review has not happened, yet.
And despite the fact that I have taken any possible precaution to make sure the code does not have any security problems, I cannot say the Crypto Bone is secure to use.
But on the other hand, without a relevant user base, the incentive to take the time and effort for a thorough code review is virtually non-existent.
This is a special version of the hen-and-egg problem, that has to be solved one way or the other.
The Crypto Bone ALL-IN-ONE
Well, not everyone has a Beagle Bone.
And, if you're using the Crypto Bone, your encrypted messages will reach other people who might not have a Beagle Bone. And that's why very few people will use this kind of secure communication, yet.
On the other hand, checking the software - and even the core software - for security pays off, only if many users rely on this software day-by-day. Only if the Crypto Bone's user base has become large enough, security experts will take the time to poke inside the software for bugs and security problems.
I try to approach this dilemma by making it easy for everyone to run a Crypto Bone, even if you don't possess the extra hardware. I have developed a software-only version of the Crypto Bone that can be installed on every Linux computer, as RPM and DEB packages are readily available.
After installation of the RPM or DEB package, anyone will be able to use a Crypto Bone within seconds. Think of it as a virtual Crypto Bone, made available on your Linux computer instead of using an external, separate hardware.
Of course, the question has to be answered, if the software-based Crypto Bone is secure to use. Would the software-based version not destroy the main benefit of a separate encryption device, its isolation and independence from all the numerous attacks on your main computer?
Attacks on the Crypto Bone
In a way, yes that's true, a separate system is always better protected than the virtual Crypto Bone.
There are a number of possible attacks on the Crypto Bone most of which can be defeated with the current design of the Crypto Bone. If the protected message key database is now stored on your computer instead of the separate device, the virtual Crypto Bone has to make sure, that someone who wants to attack these secrets, would need root permission to do this. Of course, its much more likely that an attacker will succeed to gain root permission on your local computer, because of the sheer number of additional software installed, but if someone has got root permission, well, essentialy the game is over already.
In this sense, using the ALL-IN-ONE virtual Crypto Bone is pretty safe if you guard your root account and you can switch to using a real Crypto Bone while you are using your GUI program that controls your virtual or real Crypto Bone.
So why not check out the ALL-IN-ONE Crypto Bone, and tell me what you think.
The Crypto Pi
This year has had some devastating news about the state of the internet infrastructure in store and came with some disillusion about the vulnerability of tools we use day by day. And BTW it had turned out, that Johnny still can't encrypt.
To change that, I've put some effort into developing the crypto pi.
While running on a minimalist Linux OS with only the necessary tools installed, the crypto pi is different both from internet servers and insecure endpoints, and helps Johnny to establish secure communication that is always encrypted, without burdening him with complex tasks that would only make him avoid using secure communication.
The crypto pi is in its early stages of its infant life and although it's working and looking great, it has not got the most important ingredient for a happy life (yet), peer-review.
So I'd like to ask all of you, who think a well-designed, isolated crypto box under the sole control of its user, capable of doing message encryption reliably, may improve the situation we're facing today, to give a hand and scrutinize the design and implementation of the crypto pi. Let's make the crypto pi a success in 2015, together.
The concept of secure communication using the crypto pi relies on several assumptions, not everyone will agree to:
- Johnny will be able to communicate securely with people he knows, if he had been able to exchange an initial secret information (on a piece of paper, via telephone, or some other way)
- Johnny's endpoint device is not trustworthy, as it runs all kinds of complex programs that are prone to attack the secrets on his device without notice in unforeseeable ways.
- Apart from feeding the crypto pi with the initial secret and an email address of the recipient (by filling out a form) Johnny has nothing to do with key management, but will be able to verify that message encryption has been performed.
- All secrets are stored on the crypto pi and messages leave the crypto pi AES-encrypted with a strong randomly generated key. The crypto pi does not use public key cryptography, there is no PKI nor CAs involved.
- Johnny uses one single secret that he alone knows to establish an encrypted tunnel to the crypto pi over which he interacts with the web server on the crypto pi to read and write messages.
- The local network in which the crypto pi works is not trustworthy, so all information that originates from the crypto pi is encrypted and only encrypted information that enters the crypto pi will be processed inside.
- Although desirable, ensuring anonymity is not a pre-requisite of the crypto pi's design (at the moment).
- All source code is licensed under GPL.
Fortunately, the crypto pi has a home (crypto-pi.com) where you can get more detailed information about its fundamental concepts and implementation. Make sure, your criticism and constructive suggestions will be used to improve this project.
Best wishes for 2015
Ralf Senderek
How To Keep A Secret
Obviously, the real-world advice "Don't tell anyone!" wouldn't work in the online world, because you have to use your secret to log into a web service and to access private information after login. Using your secret often and still keeping it private, so only you can use it, that is the challenge all web applications must live up to.
And many fail in this regard.
Let's assume you will be using an online application to read personal information, that is meant for your eyes only, then your secret will be used at some point to encrypt the private information in order to protect it.
Let's assume, there is a reliable standard solution in place (SSL/TLS) that protects your secret while it travels safely from your browser to the web server, it is the secret's life after having arrived at the server that is really difficult to secure. But I'll show you exactly what it takes to protect your secret while it is used by an online application.
What's The Problem?
Well, you've already provided your secret to log in, why can it be used to grant you exclusive access to some private information ten minutes later?The answer is, because it is stored somewhere, and that's where things get pretty tricky.
Your secret must be stored in the server's memory, otherwise it would be impossible to decrypt protected information you want to read. The safest way to keep your secret private is to make the server forget it after it has been used. This would turn out to be a major nuisance for you, as the server has to ask you again and again to resubmit your secret if you decide to read a different piece of information that has to be decrypted.
To save you the hassle of entering your secret again and again web applications tend to introduce a mechanism to store your secret for a while to make your life easier. Nice idea, isn't it? But the place where your secret is being stored, as well as the length of this while makes quite a big difference when it comes to protecting your secret.
The Bad Idea
You might think that storing your secret unprotected in the server's file system is the worst idea a web application could possibly come up with? Actually, it's quite easy to let your secret end up in files that are not being deleted properly. In particular, when a web application adds your secret to the PHP $_SESSION variable, it will be stored in the file system, inevitably. How could anyone come up with such a hazardous security design choice? Well. ask your favourite web application developer and listen carefully to what his justification will be, in case you get an answer at all.
How To Protect Your Secret Online
By now, I hope I've made it clear that instead of storing your secret on the server, something has to move from your browser to the server to enable the decryption of your protected information, when decryption is actually required.
This "something" must be derived from your login secret and its lifetime must be limited, and under no circumstances must this something end up anywhere, except in memory when the process of decryption needs it.
As I said previously, you will rely on the existence of a safe tunnel between your browser and the server, because this "something" must be resubmitted many times during an online session. If you wonder if there is such a reliable encrypted tunnel, I have discussed issues with the HTTPS protocol in a separate posting here.
Before you actually start to pick your brain on how to establish such a secret-protecting mechanism for your own web application, let me tell you that you can see all the details in a demonstration web application, I have developed to show how protecting a secret is done properly.
Getting Into The Details
I will briefly outline the details of my secure web application demo. Within the demo, you can enter text and encrypt it to be stored in a file, and you can also read the plain text content after decryption. The demo's main purpose is to demonstrate that your login secret can be used to create a cookie in your browser that enables the use of a RSA private key which is stored in encrypted form in the server's file system.
Without the cookie, provided by the browser, the server will not be able to use the RSA private key, as it's stored encrypted. In case an attacker manages to read files on the server, your private information is still secure, because the crucial part, a secret that comes from outside the server is still missing, and the attacker's attempts to decrypt your files will fail.
So anyone who thinks about storing sensitive information in the PHP $_SESSION variable is advised to look at the code and see how the secret is being handled in the secure web application demo. The code is designed to make sure, that the secret information is being encrypted before it is added to the $_SESSION variable, and most importantly, that the key needed to use this information always comes from outside the server.
It's the only way to keep an online secret, over time.
If you wish to use this code to secure your own web application, that's fine, the code is licensed under the GPL. Just learn from the code and make sure you include a decent authentication mechanism, as this is only a demonstration for one single user.
In short, this is what the demonstration application does:
First, it checks whether or not HTTPS is being used. If not, the app dies immediately, there is no way to keep a secret without using HTTPS.
Secondly, if you're not logged in, the app asks for your secret and uses it to decrypt the RSA private key, preparing the RSA private key to be used in memory. The RSA private key will be destroyed a split second later. Now, a fresh randomly chosen 128 bit session key is created and will be used to AES-encrypt the RSA private key again. This protected key can now be added to the $_SESSION variable. It wouldn't be of any use without the session key, so it doesn't matter if it ends up in the file system.
After wiping the RSA private key from memory, the web app makes sure that the session key leaves the server as soon as possible. Thus the session key (the "something") will be transferred via HTTPS to the user's browser and stored in a cookie. At this point the encrypted RSA key (in $_SESSION) and the cookie are separated and the server as well as the user possess only half of the information necessary to use the RSA key later again.
Every time a decryption of personal data takes place on the server, the combination of the user's cookie and the server's $_SESSION information is necessary to recover the RSA private key in memory to make decryption possible.
For demonstration purposes the decrypted personal information will replace the encrypted information in the file to emulate a mini cloud application. It might depend on the purpose of your own web application, whether it's safer to display the decrypted information to the user in a separate browser window instead.
At the bottom of the demo's screen there are buttons that allow you to inspect the content of the PHP $_SESSION variable and the contents of session files stored by PHP directly as well as the source code of the secure web application.
Have a look, and tell me what you think.
Why Secure Boot Is A Good Idea
Despite the fact that Microsoft have abused the term "secure boot" for a long time to regain exclusive control over a computer's boot process, there are some very good reasons for improving the Linux boot sequence to meet privacy concerns. I will insist to call one of these improvements secure boot.I've developed a secureboot Linux package that basically makes an encrypted container available during the boot process, so that confidential user data can be locked inside this container in one big encrypted file. This package is available as RPM for RedHat-like Linux systems and as a DEB package for all the others as well.
You might think that there is already such an option, as many Linux distributions offer "encrypt my files" or allow to encrypt entire disk partitions. Of course there is nothing wrong with these tools, it might just be what you are looking for.
In my opinion there is still a need to handle encrypted containers separately, because encrypting a whole partition does not distinguish between sensitive data on the one hand and a whole bunch of publicly known files that don't need protection on the other hand. After a while sensitive data tends to spread over the whole file system and it is very important to confine valuable data into one spot, the one single, encrypted container.
One Encrypted Container For Your Valuable Data
And it helps to make your backups much simpler, as you don't have to worry about your essential data being lost. All you need to recover a broken system lives inside your big encrypted file. Once you have a copy of your container - on a portable hard disk, your smartphone or in the cloud - with a fresh, clean Linux system on new hardware you'll be up and running in no time when you are able to mount your container into your new file system and everything is in place again. If you're traveling a lot, the benefits of a single encrypted container are obvious. You can even travel without your files and your own hardware knowing that you'll be able to download a smaller version of your big file containing the essential part of your data you'll need at your destination from the cloud.
No more worries about your files being seized or stolen on your journey or being concerned about accidental damage to your laptop by the airport x-ray or bumping of your hardware in flight.
For a time there was only speculation how backdoors are being implanted to weaken encryption and to subvert trust in secure online communication. This has all changed with the latest revelations about NSA's advanced spying capabilities, a catalogue of ready-made backdoor tools for almost everything.
There are a number of requirements a good backdoor has to meet to become an efficient surveillance tool for the NSA and other like-minded organisations.
First, and most importantly, it has to work quietly and must not be easily detectable.
This excludes everything with a noticeable impact on the usual functionality of a computer system. Side-channels are perfect for this purpose, as they leak sensitive information about encryption keys to an attacker mainly unnoticed. Anyone looking for signs of huge impact is looking in the wrong place.
Then, backdoors must hide as innocently as possible. For someone looking for vulnerabilities in computer systems a deliberate backdoor must look like an innocent mistake and the knowledge about its existence need to be kept secret by those few people that need to exploit the backdoor.
Fortunately, this is what has gone fundamentally wrong with the catalogue of backdoor tools. A backdoor known to the public, is of far less use, as people can and hopefully will make informed decisions and try to avoid risks, if they have a choice, of course.
How To Weaken A Cryptography Standard
An interesting example how a crypto standard can be weakened deliberately came with the DUAL-EC-DRBG random number generator. Random numbers are essential for the security of every crypto system, so if the output of a random number generator is predictable, the system is essentially broken.
There is a very knowledgeable video in which Professor Edward Frenkel explains how influencing a standard can put someone with intimate knowledge into the pole position when it comes to subverting an encryption process. What the NSA did was persuading the crypto community through NIST to use two large numbers that look totally random to everyone except the NSA to produce secret keys for email encryption. So they didn't actually break the encryption algorithm but they could figure out which key was being used after having observed some output of the random number generator they had backdoored.
For years this backdoor remained undetected because it was a perfect example for an innocent, helpful act of benevolence to provide the two numbers through an approved standard.
Looking At Not-So-Innocent-Looking Backdoors
With the publication of the backdoor tool catalogue there is evidence that for a number of years the NSA and others have implanted backdoors in almost every device they could get their hands on.
One of the most prominent targets have been Dell PowerEdge and HP Proliant servers, whose BIOS code has been abused to load spy-software into those machines. The BIOS is a pretty nice place to hook the implant as it will periodically be activated and thus will survive any operating system upgrade. The NSA calls this a "Persistent Backdoor" (PBD) which has been inserted into Huawei and Juniper firewalls and routers at the BIOS level as well.
As a consequence, most of the internet infrastructure can be seen as potentially compromised.
But there are similar attacks on hard drives, that provide BIOS-like firmware that can be modified to insert a backdoor. Infected hard drives will first execute spy-software before resuming with the original operating system boot process, as the malicious code has been loaded into the hard drive's "Host Protected Area".
The list continues with mobile phone BIOS exploits and even a GSM tower that is used to provide a "GSM Telephone Tripwire" to the NSA. The list goes on.
The Value Of Trust In The Internet Age
By now it's evident that these activities have seriously damaged the internet infrastructure and have irreversibly subverted the trust in that infrastructure. The damage done is that this surveillance and subversion has made the internet less secure for all of us.
To change the whole situation, I won't put much hope in the inevitable efforts to harness the power of those organisations gone mad, because if proper and effective oversight will ever happen, it won't undo the damage, making exploitation easier for others who won't be impressed by NSA oversight.
Likewise, I don't like the conclusion that, given the current situation, we have to go on without any trust at all. Accepting that there will never be any trust in the internet smells too much like surrendering important values, which we shouldn't do as a society, because trust is the fabric on which progressive developments are founded.
And surveillance in itself is not the final purpose, it simply helps to manipulate people and finally leads to controlling them in a subtle way. Implanting a backdoor does not stop at snooping, after all it is the attempt to control my computer and to coerce me as a person to - unknowingly - do something I don't want to do. It is an attempt to own my will, to take away my freedom. And this is not acceptable.
Life without encryption is possible, but it makes no sense.
The Way Forward
It's not enough to demand an end to this blatant attack on our online security. We need to look for solutions that work against these threats. This won't be an easy task, because we have do find out practically which systems and procedures can be trusted and which cannot.
There's clearly a need for more transparency and proven methods to make sure our software works reliably the way we expect it to work. I think the justifications for using proprietary security solutions are gone. Even though it is not enough, demanding that security solutions always be open source is a necessary requirement.
And we as a society have to spend considerably more effort to scrutinize and validate the security mechanisms we have found to work for us. We have to take a stand for the position that secure solutions matter, no matter how indifferent or ignorant other people around us go ahead with their online lives.
Is HTTPS A Lost Cause?
Following the allegations that the NSA is capable of reading encrypted communications many people wonder if using HTTPS is beyond help. Is any trust left in HTTPS at all?And we're not talking about forged certificates. The question is whether or not the encryption built into HTTPS can be circumvented and if so, whether or not there is a cure.
The NSA's Surveillance and Attack Workbench
According to Bruce Schneier who has helped the Guardian journalists to make sense of the documents Edward Snowden handed them, the NSA has acquired a number of capabilities to turn the internet into a global surveillance tool:
As to Bruce Schneier's interpretation of the Snowden documents there is little doubt that
All this sounds scary enough to fuel severe concerns about the online security of every single individual using the internet. But let me focus on the security of HTTPS for a moment.
- the NSA monitors the internet traffic to an extend, that interesting activity of individual users can be identified. By storing fingerprints of interesting HTTPS requests in large databases the NSA can trigger their activities by those fingerprints and is able to react to individual user's actions.
- The NSA can redirect HTTPS traffic to a set of their own (quantum) servers, which are located at the core backbone of the internet and they can use their quantum servers to impersonate ordinary websites to launch an infection attack on a user's browser and computer system.
This is possible, because NSA's quantum servers are based at a privileged position on the internet backbone, so they can react much faster to a HTTPS request than the legitimate website does. This speed advantage is being used to prevent the browser to build a connection with the original, legitimate website. Think of this as a better version of DNS spoofing without altering the IP.
- Once the traffic has been redirected to quantum servers, these servers launch a variety of pre-arranged attacks against the user's computer system exploiting vulnerabilities of the OS and installed applications to download more pre-fabricated, NSA-prepared payloads of infecting code that help to compromise the computer system long-term. These activities read pretty much like a sophisticated version of what you'll expect from the ordinary, criminal malware industry that bothers internet users for the last couple of years.
How would someone who has this kind of abilities use it to make HTTPS traffic insecure?
I'll try to find an answer to this question, because if HTTPS is beyond help, we've lost one of the foundations on which trust in the internet age is being build.
It Starts with the Beginning
The trust HTTPS helps to establish relies on two major tasks:
Using HTTPS should make sure that the browser we use is actually connected to the website we intend to visit and after connecting to the intended server, a reliably encrypted connection should make the transfer of information between the browser and the server inaccessible for everyone, except to the endpoints of the connection.
The two tools to achieve this are digital signatures and strong symmetric ciphers which are built into the HTTPS protocol.
In order to find out what a gifted agency can do to harm the HTTPS data transmission, I'll take you to follow a process of informed guessing on what's going on in the first few milliseconds of an HTTPS connection as described by Jeff Moser in the year 2009.
The Handshake
When a browser contacts say Google via https, NSA's Quantum computers will use their speed advantage to modify the HTTPS handshake process which is performed in the clear before any kind of cryptography is being applied. Essentially, there are two pieces of information that have to be sent back to the victim's browser in the server hello packet, which can make a difference if modified. First, the server picks the encryption method that will eventually be used, and at this point the NSA can pick one of the weaker algorithms (ie. RC4-MD5) and secondly, the server certificate has to be sent to the browser. While choosing a weak encryption algorithm is easy, the certificate has to stand the test of the browser.These server certificates were introduced to prevent anyone to sneak into the connection by attaching digital signatures to the information being exchanged. But checking the validity of such a digital signature relies entirely on the trustworthiness of the server certificate that's presented in the handshake dance between the browser and the quantum server dressed as Google. All the browser can do to check the server certificate is to use another public key from its built-in database, which is maintained by Mozilla. Roughly 20 percent of the institutions (CAs) in this database use a public key of only 1024 Bits. I found some 22 of the 116 CA public keys on a CentOS installation (2010) and 20 of 131 on Fedora (2012).
This raises the question, if one of these CAs might have co-operated with the NSA as far as to issue a sub-CA certificate signed with one of these minimal-length RSA keys, which would enable the agency to fabricate fake server certificates for any server they like, because this would extend the trust-chain of SSL certificates right into the proper NSA department. On the other hand, they might have been successful in factoring one of these 20 public key moduli to use the secret key directly, 1024 bit is not out of reach for a committed attempt to gain the ability to sign server certificates.
Attacks are selective
Forging a Google server certificate might sound risky, and you can be sure that the NSA will do its very best to avoid all risks of being caught in the act. But bear in mind that such a forged server cert will only be presented to one victim's browser at a time. All the rest of the internet community will get the real thing from Google, because the quantum servers only kick in when a trigger indicates that an interesting target is building up a connection. The victim might be able to reproduce signs of a forgery, the internet community as a whole will not notice anything strange.Anyway, without a success at this step, no-one would be able to read the pre-master secret the victim's browser will send encrypted with the server's public key from the certificate. If the agency cannot modify certificates that stand the browser test, they will only be able to act as a kind of relay proxy to the real Google server, being able to record the encrypted traffic for later scrutiny.
Given that there has not been any certificate modification, what can be gained by selecting a weak encryption method in the HTTPS handshake?
Well, maybe the most important achievement is switching off the Diffie-Hellman Key Exchange which would make things more difficult for the agency. But then there is the problem of attacking RC4. How difficult is that?
Certainly, RC4 has its weaknesses, but its hard to say if there is a feasible attack under real-world conditions, when RC4 is used as it should be. Known attacks on RC4 today usually rely on a very large number of encryptions of the same plain text, a condition that isn't met by real-world use of RC4 in encrypted internet traffic. But it's certainly much easier to tackle RC4 than any of the strong encryption methods like AES with 256 bit keys.
Switching on Strong Cryptography
As we've seen the server hello packet determines the final selection of the encryption method used by HTTPS. When you look at your browser's list of supported ciphers you'll see 48 different encryption methods from DHE-DSS-AES-256-SHA down to RSA-RC4-40-MD5.
Some of them are disabled (set to false) because they use desperately short key lengths from 56 bits to 40 bits or no encryption at all.
But RSA with RC4 using 128 bit keys and the outdated MD5 hash function is still a choice the Firefox browser will be happy to play with.
You can use the right mouse button to toggle the boolean value to false to tell your browser not to use this encryption suite any longer, but I doubt this user setting will survive a browser update.
So, what can the web server actually do to improve HTTPS security, as it's the web server's choice that eventually determines the encryption mechanisms that will be used.
Endpoint Security Matters
In his famous quote about the quality of encryption Edward Snowdon highlighted the importance of the endpoints (the web server and the user's computer) being secure.
Encryption works. Properly implemented strong crypto systems are one of the few things that you can rely on. Unfortunately, endpoint security is so terrifically weak that NSA can frequently find ways around it.In view of the NSA's attack capabilities one can imagine that maintaining endpoint security on the user's side is the hardest job. For that reason it might be prudent to focus on how to configure the web server to use only strong cryptography and to get rid of the weaker encryption methods.Unfortunately there is a default setting in apache that might be the norm when SSL-enabled web server are being set up
SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXPI think, the majority of web servers are currently using this default setting, provided within the apache package. What does all this mean?
According to the brilliant but complex apache manual page this default is composed to be the most liberal view of accepting ciphers arranging them in the following way:
first, remove from consideration any ciphers that do not authenticate, i.e. for SSL only the Anonymous Diffie-Hellman ciphers. Next, use ciphers using RC4 and RSA. Next include the high, medium and then the low security ciphers. Finally pull all SSLv2 and export ciphers to the end of the list.The default will generate a list of all 50 different encryption suites, with the weakest export ciphers at the end. Because RSA-RC4 is top of the list, it gets automatically selected during the handshake process.A much more restrictive set up would be:
SSLCipherSuite !ADH:!SSLv2:!SSLv3:AES:+HIGHThis will "only" enable the following thirtytwo ciphers :
openssl ciphers -v '!ADH:!SSLv2:!SSLv3:AES:+HIGH' ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(256) Mac=AEAD ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA384 ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA384 DHE-DSS-AES256-GCM-SHA384 TLSv1.2 Kx=DH Au=DSS Enc=AESGCM(256) Mac=AEAD DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(256) Mac=AEAD DHE-RSA-AES256-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AES(256) Mac=SHA256 DHE-DSS-AES256-SHA256 TLSv1.2 Kx=DH Au=DSS Enc=AES(256) Mac=SHA256 ADH-AES256-GCM-SHA384 TLSv1.2 Kx=DH Au=None Enc=AESGCM(256) Mac=AEAD ADH-AES256-SHA256 TLSv1.2 Kx=DH Au=None Enc=AES(256) Mac=SHA256 ECDH-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AESGCM(256) Mac=AEAD ECDH-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AESGCM(256) Mac=AEAD ECDH-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AES(256) Mac=SHA384 ECDH-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AES(256) Mac=SHA384 AES256-GCM-SHA384 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(256) Mac=AEAD AES256-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA256 ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(128) Mac=AEAD ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA256 ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA256 DHE-DSS-AES128-GCM-SHA256 TLSv1.2 Kx=DH Au=DSS Enc=AESGCM(128) Mac=AEAD DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(128) Mac=AEAD DHE-RSA-AES128-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AES(128) Mac=SHA256 DHE-DSS-AES128-SHA256 TLSv1.2 Kx=DH Au=DSS Enc=AES(128) Mac=SHA256 ADH-AES128-GCM-SHA256 TLSv1.2 Kx=DH Au=None Enc=AESGCM(128) Mac=AEAD ADH-AES128-SHA256 TLSv1.2 Kx=DH Au=None Enc=AES(128) Mac=SHA256 ECDH-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AESGCM(128) Mac=AEAD ECDH-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AESGCM(128) Mac=AEAD ECDH-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AES(128) Mac=SHA256 ECDH-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AES(128) Mac=SHA256 AES128-GCM-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(128) Mac=AEAD AES128-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA256I wonder how many browsers would play nicely with such a setting, but in fairness, it doesn't seem unreasonable to me to request using strong cryptography from a browser, in light of the prevailing attempts to undermine our online security on a global scale.
