Sunday, March 23, 2014

All about RSA key formats

I've spent the past few weeks building up the PCLCrypto library which targets .NET Framework, Windows Store (WinRT), Windows Phone (WP8), Silverlight (SL), Xamarin.Android (XA) and Xamarin.iOS (XI). What started out pretty easy turned into a pretty rough ride. The short version is: everything I've learned is now included in the PCLCrypto library so all you have to do is use its simple WinRT-like API from your app or library and you'll get native performance on each platform with no additional work on your part.

Getting to that point was, as I said: a rough ride, mostly in the RSA space. There is no RSA available at all for Silverlight (from the platform, anyway). WinRT has its own unique API but it's a good one and supports a variety of key formats. The rest of the platforms (.NET, WP8, XA, XI) all share an RSACryptoServiceProvider class, which at first made it look easy. I had all my tests passing in very little time because I just had to write code for WinRT and .NET.
The problem started when I realized how incredibly slow RSA operations (especially key generation) were on Xamarin mobile apps. I discovered that while Microsoft implements RSACryptoServiceProvider by simply P/Invoking into the Win32 CryptoAPI, Xamarin/mono implements it in all managed code. Managed implementations of RSA are slow. RSA requires a lot of BigInteger math so optimizations even at the assembler level really make a huge difference, so managed code can't touch that perf.

So the problem to tackle became how to implement crypto for each platform. Each platform of course has its documentation and samples for how to do crypto, but that's about where the similarity ended. Their preferred languages were different: iOS uses Objective-C and Android uses Java. I had to write everything in C# for the PCL library. Fortunately Xamarin made (most of) the APIs available to my C# library. Translating the samples from Java to C# was straightforward, but translating from Objective-C was a lesson in a very foreign language syntax for me. I can now say that I can read the language reasonably well, although I certainly couldn't write it by myself yet.

And it just got harder from there. I had to support exporting private keys generated from each platform. For .NET and WinRT this was a piece of cake. Each key object has an Export method, possibly with options of different key formats, and that's it.
Android makes discovering the private keys more difficult to export, involving downcasts and gotchas like having to do it immediately after key generation or you lose the opportunity to get at some of the RSA parameters.
iOS was the worst: they have no supported way to export the private key. There is an unsupported way to do it that has worked since around 2010 (from what I could figure out from the forums), but the Apple folks strongly discourage its use, threatening that any future version of iOS could break the 'backdoor' for obtaining private keys. Nevertheless it hasn't been closed off yet, and there is no other way to get it done, so I had to take that route. Personally, I think it's a perfectly reasonable approach and one I think Apple should maintain and support.

Then there were the differences in native formats of the keys themselves after I got them exported. RSACryptoServiceProvider's native key format is called CAPI (which took some work to discover because their method calls it a "CspBlob". Not only is CAPI the format you get an exported key in, but it's the format used internally during computations (which will become interesting later).
Android's native private key format is PKCS#8 PrivateKeyInfo. The native format in iOS is PKCS#1 rsaPrivateKey. For these systems to be able to interoperate, I had to be able to freely convert each format to every other format. For WinRT this was easy, since it can import/export all these formats in the platform. For all the other platforms this required that I study several new RSA key specs, learn ASN.1, DER, and BER encoding rules, and implement all these in a PCL-compatible way so I could leverage the code to help all the platforms interoperate. What made this particularly difficult was putting together the information I could find on the Internet (between RFCs, forum posts, docs, samples, etc.) since all of them seemed to be focused on only a very tiny bit of my overall problem, provide old or incorrect information, hard-code a bunch of formats without reasonable abstractions (making discovering how ASN.1 really works difficult), or assumed a certain level of familiarity with all the preceding RFCs which I didn't have at the time.

Then there was the iOS P and Q problem. It turns out that when iOS generates an RSA keypair, the P parameter is longer than the Q parameter. For Windows and Android platforms these parameters tend to be the same length. In fact this tendency in Windows is so strong that the CAPI key format actually depends on it, because none of the parameter lengths are actually included in the file format -- all the lengths are assumed. If you have a private key with P and Q of different lengths, it's fundamentally incompatible with any CAPI-based RSA operations. That took a very long time to finally figure out. And to date I still haven't found a way to get iOS to generate keypairs with equal length P and Q.
On most platforms this isn't a problem. Almost every platform offers a way to use non-CAPI crypto. In fact CAPI crypto was deprecated with Windows Vista -- but .NET Framework still offers the old CAPI stuff. Using the newer CNG API requires either P/Invoking from .NET, or using a Security.Cryptography.dll library Microsoft dropped on CodePlex back in 2010 and hasn't touched since, and it comes with its own limitations.
The requirement I was working toward of course was to get a private key generated on iOS to work in PCLCrypto on every other platform. Mono.Security.dll offered a way: it has a managed RSA implementation that doesn't depend on the CAPI file format. So when targeting the .NET Framework, PCLCrypto would first try using the platform (CAPI) and if the RSA key isn't compatible with it, it falls back to the mono managed implementation of RSA. Why not use CNG? Well, I tried, but it doesn't let you import keypairs. Go figure.
That just left Windows Phone. How was I going to get these iOS-generated private keys working on Windows Phone 8.0, where P/Invoke is not allowed and the only RSA implementation is CAPI-based? CNG isn't an option. Mono.Security.dll doesn't target WP8, and while it target Silverlight and its RSA implementation seems to work on WP8, it lacks OAEP padding support which is a requirement. That's actually where I'm at with WP8: it can't import private keys generated with iOS today.

To repeat the summary at the top: all this study and sweat over the past few weeks has gone into PCLCrypto so that you can use the very simple API for many kinds of crypto including exchanging private keys in a variety of formats and between most platforms. You can reference it from your PCL library or your desktop, store, or mobile apps. It's also free and open source, so please check it out, send feedback and pull requests.

Would you like to donate to express thanks for the project?
Donate Bitcoins

Why am I doing all this? Two reasons: I enjoy contributing whatever I can to the community, and it helps me deliver Dart, my mobile app that provides more privacy in your communication than any other app.

Sunday, January 05, 2014

Update on the secure messaging IronPigeon protocol and the Dart application

In my last post I mentioned I was focusing my attention on the IronPigeon protocol. Since then, the IronPigeon project has matured and added support for both iOS and Android platforms in addition to its original Windows platforms it supported before.

As just a protocol, IronPigeon doesn't directly help end users exchange messages securely. Dart for Windows Phone 8 has been available for several months to allow Windows Phone users to exchange messages using the IronPigeon protocol and all of its security benefits.

But as most folks don't use Windows Phone, I'm pleased to announce that Dart for Android devices is now available. It uses the same protocol and offers nearly the same functionality as the Windows Phone app, and of course users can expect to exchange messages with others regardless of which type of phone they happen to be using.

Support for Apple's iOS platform (e.g. iPad, iPhone) is coming in the near future. As well as more features within Dart across all platforms.

I look forward to hearing the feedback. And if you like the app or feel strongly about user privacy, please help evangelize the Dart app with your friends and social circles.

Wednesday, June 12, 2013

Moving on… DotNetOpenAuth in search for new project leaders

Disclosure and disclaimer: I am a software engineer at Microsoft, but the following post (just like all other posts on this blog) is my own and in no way represents the views of Microsoft. All statements regarding Microsoft expressed here are my opinions based on publicly available information.

When I started working on DotNetOpenAuth (by its former name of DotNetOpenId) in 2007 my primary motivation was to act as a kind of "Internet consumer privacy advocate." End users on the Internet are pawns in an ocean of big companies that want to collect as much of your personal information as they can. Their privacy policies may change more frequently than women’s fashion, but much more subtly as most users don’t read them and couldn’t understand their ramifications even if they did read them. It is apparent to me that no for-profit business is going to truly be on the customer’s side to protect their interests really. Sure, they’ll claim to champion privacy while using that assurance to woo away the private information from their customers and store on servers to be analyzed later either by themselves or by government subpoenas. I endeavored to truly represent the end user’s interests as I participated in identity-centric mailing lists, attended OpenID conferences, etc.

Passwords are horribly implemented and overall just a bad idea compared with modern alternatives. Back when I began working on DotNetOpenId, “Microsoft Passport” was just failing as an attempt to make it easier for users and sites to authenticate, but at the ridiculous cost of $10,000 per relying party just to sign up, and then you have the central controlling entity problem. OpenID promised a decentralized way to solve the problem that really appealed to me. It still does.

Information Cards (a.k.a. Windows CardSpace) was another Microsoft attempt to solve the authentication problem. Two flavors existed: m-cards and p-cards as they were known. Managed cards relied on a business having issued them, while personal cards were self-issued. I really believed in p-cards because they truly gave users control of their own identity and actually presented it in a user-friendly way. Unlike (most) X.509 certificates or m-cards, p-cards were self-issued and self-asserting, so that only your local computer could authenticate you. Weaknesses in implementation that gave CardSpace bad startup perf, and abysmal marketing of the technology were the two factors which (IMO) led to its ultimate demise and removal from later versions of Windows.

I had preferred Information Cards over OpenID because of users having ultimate control instead of some server (usually a big company) that actually owned the identity of the customer. But I really appreciated and helped drive OpenID adoption with DotNetOpenAuth because as p-cards were not being adopted, I saw OpenID as the greatest practical hope end users had. And while OpenID isn’t perfect, getting everyone to adopt it would not only educate web developers and users that there is a better way, if successful, it would force the re-thinking of user identity so that a superior successor to OpenID could eventually come in to replace it with much less friction than the earlier password-to-OpenID transition.

DotNetOpenAuth has been my contribution to help in this effort, because I didn’t see Microsoft, or anyone else, helping the .NET community to write better authentication in ASP.NET applications. I saw a gap and wanted to fill it. OAuth 1 and OAuth 2 came along and as it helped put the user a bit more in control of their data (by removing the need for users to disclose their passwords with arbitrary 3rd parties) it made sense to add support for authorization protocols into DotNetOpenAuth.

It’s been several years, and times change. Google, Yahoo, AOL, and Microsoft have all taken identity and web authentication for ordinary end users more seriously. They still grab increasing amounts of customer data, but authentication to their services and to 3rd party web sites that access data with these big companies has really improved over the years. The gap between what was offered and what end users deserved, that originally attracted me to this space, has largely been closed. Most particularly for me, the ASP.NET team has released libraries that build on and beside DotNetOpenAuth to make it even easer to build web sites that offer secure authentication options. With corporations now paying engineers to do what I was doing without pay, I’m happy to refocus my efforts elsewhere.

So as the title of this post suggests, my limited time for spending on projects like these requires that I move my focus away from DotNetOpenAuth. I plan to step down as leader of the DotNetOpenAuth project by June 30, 2013. I hope by then someone I can endorse will step up to take the baton and keep supporting the large user base we currently have. DotNetOpenAuth has probably amassed more than 1 million downloads in total over its lifetime, which isn’t huge for popular open source projects but does make it in one of the top tiers, I think. It’s user base is loyal and I do not want to leave them stranded. I appreciate all the support they’ve shown. I will continue to provide some minimal maintenance support for as long as necessary (and my resources allow) for security patches, etc. If you’ve been at least semi-active in the DotNetOpenAuth community and are interested in filling in as (one of) the project lead(s), please email me. You’ll know how to reach me. :) We may even set up some kind of community voting if we get several good volunteers, as this certainly isn’t something I need to control.

This decision has not been an easy one. In fact I’ve struggled to make it for the last year or so. With the news coming in the last couple years of massive user data leaks (whether intentional or otherwise), I’ve finally been pushed over the edge to change my focus.

I should call out a big thanks to David Christiansen, who has collaborated with me on DotNetOpenAuth for years, and found us free web hosting, created or helped with several of our web sites, etc. He has volunteered to help with the transition to a new project lead.

There is a major release of DotNetOpenAuth that has been in the making for several months. DotNetOpenAuth 5.0 is a huge update to use the latest that the .NET 4.5 platform has to offer. I hope to either wrap up that release and publish it before I step down, or have it close enough that some successor(s) to the project can complete the work and release it soon.

My passion for Internet consumer privacy has led me to a new gap – one that I think corporations will be much slower to try to fill and thus makes it more important for independent folks like myself to work to fill it. I now work (when I’m not working at my day job) to protect the authenticity and confidentiality of Internet communications and private data. With the advent of moving everything into the cloud from files (SkyDrive, Google Drive, DropBox) to communications (Gmail, Hotmail, Skype, Facebook, Google+), customer data may have never been more at the whim of big company policies – or technical foul-ups. These things can be and are monitored both by the companies and by governments. And I don’t care how much you trust your current government, there’s no telling how well you’ll trust government when an opposing party is in control (assuming you live in a democracy or a republic such as the U.S.). I’ll stop there and save more feelings on this topic for another post. 

My next focus then is to provide an authenticated and confidential alternative to email. The SMTP protocol for emails is one of the oldest protocols on the Internet, and also one of the least secure. Users deserve the privacy they ignorantly assume they have when sending emails to others. Yes, there are already many technologies and services out there that claim to provide confidentiality and authenticity assurances for email (PGP is noteworthy because it’s actually a decentralized solution rather than depending upon and trusting a company). But as far as I have found, they all fall short.

IronPigeon is my new project. It is a general message passing protocol upon which email-like experiences can be built that provides ultimate confidentiality and authenticity assurances. And it is very carefully designed to be totally decentralized. No one need rely on or trust a server in the cloud, and no malfunction or loss of a central service will cripple the network. The protocol has the ability to route messages between users without knowing who is sending or receiving the message, how large the message is, or what any of its content is (headers included). Perhaps I could patent and sell such a protocol, but as my motivation is to help end users, my goal is to maximize adoption and thus I make it available for free. If there is sufficient interest, maybe we’ll kick up a WG to standardize the protocol.

Of course a protocol by itself is of no use to end users. We need user-friendly apps written that support the protocol available on all platforms and devices. I’m working on a couple of these apps myself but have limited time, resources, and UX talent to do a thorough job everywhere. So I am really hoping the community picks up on this to write apps based on it.

To those of you who are only interested in my work because of authentication-related topics, thank you for your following, and I wish you the best. For those whose interests span into the more general topics of user privacy, I hope to see much more of you (and from you) as we embark to create an open and individual user-controlled ecosystem for our world community.