(Wow, what an utterly unappealing post title…)
I have run into an annoying issue with the DER output of RSA public keys in OpenSSL. The basic problem is that OpenSSL adds an OID header to its ASN.1 DER output, but other tools are not expecting this (be it iOS, a keystore in Java, the iPhone keystore, or Erlang’s public_key module).
I first noticed this when experiencing decode failures in Erlang trying to use the keys output by an openssl command sequence like:
openssl genpkey \
-algorithm rsa \
-out $keyfile \
-outform DER \
-pkeyopt rsa_keygen_bits:8192
openssl rsa \
-inform DER \
-in $keyfile \
-outform DER \
-pubout \
-out $pubfile
Erlang’s public_key:der_decode(‘RSAPrivateKey’, KeyBin) would give me the correct #’RSAPrivateKey’ record but public_key:der_decode(‘RSAPublicKey’, PubBin) would choke and give me an asn1 parse error (ML thread I posted about this). Of course, OpenSSL is expecting the OID header, so it works fine there, just not anywhere else.
Most folks probably don’t notice this, though, because the primary use case for most folks is either to use OpenSSL directly to generate and then use keys, use a tool that calls OpenSSL through a shell to do the same, or use a low-level tool to generate the keys and then use the same library to use the keys. Heaven forbid you try to use an OpenSSL-generated public key in DER format somewhere other than OpenSSL, though! (Another reason this sort of thing usually doesn’t cause problems is that almost nobody fools around with crypto stuff in their tools to begin with, leaving this sort of thing as an issue far to the outside of mainstream hackerdom…)
Of course, the solution could be to chop off the header, which happens to be 24-bits long:
1> {ok, OpenSSLBin} = file:read_file("rsa3.pub.der.openssl").
{ok,<<48,130,4,34,48,13,6,9,42,134,72,134,247,13,1,1,1,5,
0,3,130,4,15,0,48,130,4,...>>}
2> {ok, ErlangBin} = file:read_file("rsa3.pub.der.erlang").
{ok,<<48,130,4,10,2,130,4,1,0,202,167,130,153,242,77,196,
252,167,142,159,17,13,69,148,41,161,50,...>>}
3> <<_:24/binary, ChoppedBin/binary>> = OpenSSLBin.
<<48,130,4,34,48,13,6,9,42,134,72,134,247,13,1,1,1,5,0,3,
130,4,15,0,48,130,4,10,2,...>>
4> ChoppedBin = ErlangBin.
<<48,130,4,10,2,130,4,1,0,202,167,130,153,242,77,196,252,
167,142,159,17,13,69,148,41,161,50,44,138,...>
But… that’s pretty darn arbitrary. It turns out the header is always:
<<48,130,4,34,48,13,6,9,42,134,72,134,247,13,1,1,1,5,0,3,130,4,15,0>>
I could match on that, but it still feels weird because that particular binary string just doesn’t mean anything to me, so matching on it is still the same hack. I could, of course, go around that by writing just the public key as PEM, loading it, encoding it to DER, and saving that as the DER file from within Erlang (thus creating a same-tool-to-same-tool situation). But that’s goofy too: PEM is just a base64 encoded DER binary wrapped in a text header/footer! Its completely arbitrary that PEM should work but DER shouldn’t!
-module(keygen).
-export([start/1]).
start([Prefix]) ->
PemFile = string:concat(Prefix, ".pub.pem"),
KeyFile = string:concat(Prefix, ".key.der"),
PubFile = string:concat(Prefix, ".pub.der"),
{ok, PemBin} = file:read_file(PemFile),
[PemData] = public_key:pem_decode(PemBin),
Pub = public_key:pem_entry_decode(PemData),
PubDer = public_key:der_encode('RSAPublicKey', Pub),
ok = file:write_file(PubFile, PubDer),
io:format("Wrote private key to: ~ts.~n", [KeyFile]),
io:format("Wrote public key to:Â ~ts.~n", [PubFile]),
case check(KeyFile, PubFile) of
true ->
ok = file:delete(PemFile),
io:format("~ts and ~ts agree~n", [KeyFile, PubFile]),
init:stop();
false ->
io:format("Something has gone wrong.~n"),
init:stop(1)
end.
check(KeyFile, PubFile) ->
{ok, KeyBin} = file:read_file(KeyFile),
{ok, PubBin} = file:read_file(PubFile),
Key = public_key:der_decode('RSAPrivateKey', KeyBin),
Pub = public_key:der_decode('RSAPublicKey', PubBin),
TestMessage = <<"Some test data to sign.">>,
Signature = public_key:sign(TestMessage, sha512, Key),
public_key:verify(TestMessage, sha512, Signature, Pub).
This is so silly its maddening — and apparently folks from the iOS, PHP and Java worlds have essentially built themselves hacks to handle DER keys that deal directly with this.
I finally found a way that is both semantically meaningful (well, somewhat) and is sure to either generate a proper PKCS#1 DER public RSA key or fail telling me that its looking at badly-formed ASN.1 (or binary trash): the magical command “openssl asn1parse -strparse [offset]”.
A key generation script therefore looks something like this:
#! /bin/bash
prefix=${1:?"Cannot proceed without a file prefix."}
keyfile="$prefix"".key.der"
pubfile="$prefix"".pub.der"
# Generate 8192 RSA key
openssl genpkey \
-algorithm rsa \
-out $keyfile \
-outform DER \
-pkeyopt rsa_keygen_bits:8192
# OpenSSL's PKCS#1 ASN.1 output adds a 24-byte header that
# other tools (Erlang, iOS, Java, etc.) choke on, so we clip
# the OID header off with asn1parse.
openssl rsa \
-inform DER \
-in $keyfile \
-outform DER \
-pubout \
| openssl asn1parse \
-strparse 24 \
-inform DER \
-out $pubfile
The output on stdout will look something like:
$ openssl asn1parse -strparse 24 -inform DER -in rsa3.pub.der.openssl
0:d=0Â hl=4 l=1034 cons: SEQUENCE
4:d=1Â hl=4 l=1025 prim: INTEGER
:CAA78299F24DC4FCA78E9F110D459429A1322C8AF0BF558B7E49B21A30D5EFDF0FFF512C15E5292CD85209EBB21861235250ABA3FBD48F0830CC3F355DDCE5BA467EA03F58F91E12985E54336309D8F954525802949FA68A5A742B4933A5F26F148B912CF60D5A140C52C2E72D1431FA2D40A3E3BAFA8C166AF13EBAD774E7FEB17CAE94EB12BE97F3CABD668833691D2A2FB3001BF333725E65081F091E46597028C3D50ABAFAF96FA2DF16E6AEE2AE4B8EBF4360FBF84C1312001A10056AF135504E02247BD16C5A03C5258139A76D3186753A98B7F8E7022E76F830863536FA58F08219C798229A23D0DDB1A0B57F1A4CCA40FE2E5EF67731CA094B03CA1ADF3115D14D155859E4D8CD44F043A2481168AA7E95CCC599A739FF0BB037969C584555BB42021BDB65C0B7EF4C6640CCE89A860D93FD843FE77974607F194206462E15B141A54781A5867557D8A611D648F546B72501E5EAC13A4E01E8715DFE0B8211656CFA67F956BC93EA7FB70D4C4BCFEC764128EAE8314F15F2473FCF4C6EEA29299D0B13C2CCC11A73BF58209A056CB44985C81504013529C75B6563CED3CC27988C70733080B01F78726A3ABBCD8765538D515D2282EF79F4818A807A9173CC751E46BDB930E87F8DA64C6ABDD8127900D94A20EE87F435716F4871463854AD0B7443243BDA0682F999D066EA213952F296089DA4577B7B313D536185E7C37E68F87D43A53B37F1E0FB7969760F31C291FDB99F63F6010E966EC418A10EFE7D4EBD4B494693965412F0E78150150B61A4FF84AB26355FC607697B86BFB014E6450793D6BF5EA0565C63161D670CD64E99FFD9AE9CCF90C6F77949A137E562E08787D870604825EF955B3A26F078CBA5E889F9AFEEF48FE7F36A51D537E9ADF0746EA81438A91DF088C6C048622AC372AEEBD11A2427DFCBC2FE163FD62BE3DCBDC4961ECD685CBAD55898B22E07A8C7E3FB4AEA8AC212F9CA6D6FE522EC788FEDD08E403BD70D1ABEDE03B760057920B27B70CBDCB3C1977A7B21B5CD5AF3B7D57A0B2003864C4A56686592CBFC1BBAF746E35937A3587664A965B806C06C94EC78119A8F3BB18BCFCEB1E5193C0EFD025A86AD6C2AE81551475A14B81D5A153E993612E1D7AED44AB38E9A4915D4A9B3A3B4B2DF2C05C394FC65B46E3983EA0F951B04B7558099DB04E73F550D08A453020EFD7E1A4F2383C90E8F92A2A44CC03A5E0B4F024656741464D86E89EA66526A11193435EB51AED58438AA73A4A74ABF3CDE1DE645D55A09E33F3408AD340890A72D2C6E4958669C54B87B7C146611BFD6594519CDC9D9D76DF4D5F9D391F4A8D851FF4DAE5207585C0A544A817801868AA5106869B995C66FB45D3C27EE27C078084F58476A7286E9EE5C1EF76A6CF17F8CDD829EFBB0C5CED4AD3D1C2F101AB90DC68CEA93F7B891B217
1033:d=1 hl=2 l=3 prim: INTEGER :010001
And if we screw it up and don’t actually align with the ASN.1 header properly things explode:
$ openssl asn1parse -strparse 32 -inform DER -in rsa3.pub.der.openssl Error parsing structure 140488074528416:error:0D07207B:asn1 encoding routines:ASN1_get_object:header too long:asn1_lib.c:150: 140488074528416:error:0D068066:asn1 encoding routines:ASN1_CHECK_TLEN:bad object header:tasn_dec.c:1306: 140488074528416:error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:814:
Now my real question is… why isn’t any of this documented? This was a particularly annoying issue to work my way around, has obviously affected others, and yet is obscure enough that almost no mention of it can be found in documentation anywhere. GHAH!