Root Zone KSK Rollover and HSM Update
In April we announced that the manufacturer of our hardware security modules (HSMs) will cease production of the devices https://mm.icann.org/pipermail/root-dnssec-announce/2023/000157.html. As noted in that communication, we continued with our previously announced plans to begin the first phases of a KSK rollover. We generated a new KSK at the KSK Ceremony 49 in April, and plan to replicate the KSK to the second facility in the upcoming KSK Ceremony 50 this week. In the past few months we've procured Keyper HSMs to both meet our replacement schedule and provide additional spare units. We've been engaging HSM manufacturers to identify a new vendor and collaborating with our root zone management partner, Verisign, who is also impacted in relation to management of the root zone ZSK. The operational considerations for the ZSK differ from the KSK, particularly given the need for online day-to-day signing, but the security of the root zone relies on the robustness of all of these parts. In light of the uncertainty surrounding the future configuration of the HSMs, we have decided to not immediately update the root zone trust anchor files with the digest of KSK-2023 immediately following Ceremony 50. There is a strong likelihood we will seek to generate a new KSK on a new HSM platform once operationalized, which will cause us to abandon the recently generated KSK. We will however retain the recently generated KSK for now should those plans not pan out in a suitable time frame. Potential options are being actively evaluated, and we expect to have developed a preferred remediation approach in the coming months. While we don't have all the answers at this time, we encourage questions and feedback from trusted community representatives and other interested observers. This input will help inform our future planning. James Mitchell Director, IANA Technical Services
I wonder if the functionality we rely on is standardized enough to procure from different vendors, that would reduce the risk in single vendor vulnerabilities, like a batch of faulty condensers, or other on pront hardware. — Olaf On 19 Jul 2023, at 0:29, James Mitchell via ksk-rollover wrote:
In April we announced that the manufacturer of our hardware security modules (HSMs) will cease production of the devices https://mm.icann.org/pipermail/root-dnssec-announce/2023/000157.html.
As noted in that communication, we continued with our previously announced plans to begin the first phases of a KSK rollover. We generated a new KSK at the KSK Ceremony 49 in April, and plan to replicate the KSK to the second facility in the upcoming KSK Ceremony 50 this week.
In the past few months we've procured Keyper HSMs to both meet our replacement schedule and provide additional spare units. We've been engaging HSM manufacturers to identify a new vendor and collaborating with our root zone management partner, Verisign, who is also impacted in relation to management of the root zone ZSK. The operational considerations for the ZSK differ from the KSK, particularly given the need for online day-to-day signing, but the security of the root zone relies on the robustness of all of these parts.
In light of the uncertainty surrounding the future configuration of the HSMs, we have decided to not immediately update the root zone trust anchor files with the digest of KSK-2023 immediately following Ceremony 50. There is a strong likelihood we will seek to generate a new KSK on a new HSM platform once operationalized, which will cause us to abandon the recently generated KSK. We will however retain the recently generated KSK for now should those plans not pan out in a suitable time frame.
Potential options are being actively evaluated, and we expect to have developed a preferred remediation approach in the coming months. While we don't have all the answers at this time, we encourage questions and feedback from trusted community representatives and other interested observers. This input will help inform our future planning.
James Mitchell Director, IANA Technical Services
_______________________________________________ ksk-rollover mailing list ksk-rollover@icann.org https://mm.icann.org/mailman/listinfo/ksk-rollover
_______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Olaf M. Kolkman Principal, Internet Society https://www.internetsociety.org Used to tweet as: @kolkman, Toots from: social.secret-wg.org/@olaf Talk to me if you or your organization wants to support our cause. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
On Mon, Jul 31, 2023 at 02:03:59PM +0200, Olaf Kolkman via ksk-rollover wrote:
I wonder if the functionality we rely on is standardized enough to procure from different vendors, that would reduce the risk in single vendor vulnerabilities, like a batch of faulty condensers, or other on pront hardware.
From our experience besides admin interfaces, standard APIs for regular operations, generating keys, sign, verify etc... are available (PKCS#11/KMIP) from multiple vendors. But exporting/importing a key, specially with the no-export attribute set, among vendors is not available.
So to switch vendors you'll need to do a keyroll still having access to the old HSMs.
— Olaf
Fred
On 19 Jul 2023, at 0:29, James Mitchell via ksk-rollover wrote:
In April we announced that the manufacturer of our hardware security modules (HSMs) will cease production of the devices https://mm.icann.org/pipermail/root-dnssec-announce/2023/000157.html.
As noted in that communication, we continued with our previously announced plans to begin the first phases of a KSK rollover. We generated a new KSK at the KSK Ceremony 49 in April, and plan to replicate the KSK to the second facility in the upcoming KSK Ceremony 50 this week.
In the past few months we've procured Keyper HSMs to both meet our replacement schedule and provide additional spare units. We've been engaging HSM manufacturers to identify a new vendor and collaborating with our root zone management partner, Verisign, who is also impacted in relation to management of the root zone ZSK. The operational considerations for the ZSK differ from the KSK, particularly given the need for online day-to-day signing, but the security of the root zone relies on the robustness of all of these parts.
In light of the uncertainty surrounding the future configuration of the HSMs, we have decided to not immediately update the root zone trust anchor files with the digest of KSK-2023 immediately following Ceremony 50. There is a strong likelihood we will seek to generate a new KSK on a new HSM platform once operationalized, which will cause us to abandon the recently generated KSK. We will however retain the recently generated KSK for now should those plans not pan out in a suitable time frame.
Potential options are being actively evaluated, and we expect to have developed a preferred remediation approach in the coming months. While we don't have all the answers at this time, we encourage questions and feedback from trusted community representatives and other interested observers. This input will help inform our future planning.
James Mitchell Director, IANA Technical Services
On 2023-07-31 at 14:53, Frederico A C Neves via ksk-rollover wrote:
From our experience besides admin interfaces, standard APIs for regular operations, generating keys, sign, verify etc... are available (PKCS#11/KMIP) from multiple vendors. But exporting/importing a key, specially with the no-export attribute set, among vendors is not available.
I concur; moving keys not marked as CKA_EXTRACTABLE (at time of generation) is generally not supported (due to FIPS requirements). jakob -- Jakob Schlyter Kirei AB - www.kirei.se
There is not much you can do with the existing keys but still, KMIP is something to consider going forward if one is concerned about vendor lock-ins. Needless to say, like anything else, there is a tradeoff. Cheers! T. On Mon, Jul 31, 2023 at 11:23 PM Jakob Schlyter via ksk-rollover < ksk-rollover@icann.org> wrote:
On 2023-07-31 at 14:53, Frederico A C Neves via ksk-rollover wrote:
From our experience besides admin interfaces, standard APIs for regular operations, generating keys, sign, verify etc... are available (PKCS#11/KMIP) from multiple vendors. But exporting/importing a key, specially with the no-export attribute set, among vendors is not available.
I concur; moving keys not marked as CKA_EXTRACTABLE (at time of generation) is generally not supported (due to FIPS requirements).
jakob
-- Jakob Schlyter Kirei AB - www.kirei.se _______________________________________________ ksk-rollover mailing list ksk-rollover@icann.org https://mm.icann.org/mailman/listinfo/ksk-rollover
_______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
Hi Tomofumi - KMIP is probably not relevant to this problem. The problem I think you're trying to solve here is not one of interface (how to talk to the keys), but of key protection. Mike On 8/2/2023 2:35 AM, Tomofumi Okubo via ksk-rollover wrote:
There is not much you can do with the existing keys but still, KMIP is something to consider going forward if one is concerned about vendor lock-ins. Needless to say, like anything else, there is a tradeoff.
Cheers! T.
On Mon, Jul 31, 2023 at 11:23 PM Jakob Schlyter via ksk-rollover <ksk-rollover@icann.org> wrote:
On 2023-07-31 at 14:53, Frederico A C Neves via ksk-rollover wrote:
> From our experience besides admin interfaces, standard APIs for > regular operations, generating keys, sign, verify etc... are available > (PKCS#11/KMIP) from multiple vendors. But exporting/importing a key, > specially with the no-export attribute set, among vendors is not > available.
I concur; moving keys not marked as CKA_EXTRACTABLE (at time of generation) is generally not supported (due to FIPS requirements).
jakob
-- Jakob Schlyter Kirei AB - www.kirei.se <http://www.kirei.se> _______________________________________________ ksk-rollover mailing list ksk-rollover@icann.org https://mm.icann.org/mailman/listinfo/ksk-rollover
_______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
_______________________________________________ ksk-rollover mailing list ksk-rollover@icann.org https://mm.icann.org/mailman/listinfo/ksk-rollover
_______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
Hello Mike, Yes, I understand it won’t help with the existing/current keys on Keyper which are not exportable. As KMIP allows us to move keys from one HSM to another (different vendor) I thought it would help with vendor lock-ins going forward. “Going forward” is when a different HSM is being selected. Although, it seems like a good number of Keypers are secured before the EoL so I suppose we are good for quite a while. Plenty of time to think what to do. Cheers! Tomofumi From: ksk-rollover <ksk-rollover-bounces@icann.org> On Behalf Of Michael StJohns via ksk-rollover Sent: Friday, August 4, 2023 3:57 AM To: ksk-rollover@icann.org Subject: Re: [ksk-rollover] Root Zone KSK Rollover and HSM Update Hi Tomofumi - KMIP is probably not relevant to this problem. The problem I think you're trying to solve here is not one of interface (how to talk to the keys), but of key protection. Mike On 8/2/2023 2:35 AM, Tomofumi Okubo via ksk-rollover wrote: There is not much you can do with the existing keys but still, KMIP is something to consider going forward if one is concerned about vendor lock-ins. Needless to say, like anything else, there is a tradeoff. Cheers! T. On Mon, Jul 31, 2023 at 11:23 PM Jakob Schlyter via ksk-rollover <ksk-rollover@icann.org<mailto:ksk-rollover@icann.org>> wrote: On 2023-07-31 at 14:53, Frederico A C Neves via ksk-rollover wrote:
From our experience besides admin interfaces, standard APIs for regular operations, generating keys, sign, verify etc... are available (PKCS#11/KMIP) from multiple vendors. But exporting/importing a key, specially with the no-export attribute set, among vendors is not available.
I concur; moving keys not marked as CKA_EXTRACTABLE (at time of generation) is generally not supported (due to FIPS requirements). jakob -- Jakob Schlyter Kirei AB - www.kirei.se<http://www.kirei.se> _______________________________________________ ksk-rollover mailing list ksk-rollover@icann.org<mailto:ksk-rollover@icann.org> https://mm.icann.org/mailman/listinfo/ksk-rollover _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. _______________________________________________ ksk-rollover mailing list ksk-rollover@icann.org<mailto:ksk-rollover@icann.org> https://mm.icann.org/mailman/listinfo/ksk-rollover _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
On 7/31/2023 10:23 AM, Jakob Schlyter via ksk-rollover wrote:
On 2023-07-31 at 14:53, Frederico A C Neves via ksk-rollover wrote:
From our experience besides admin interfaces, standard APIs for regular operations, generating keys, sign, verify etc... are available (PKCS#11/KMIP) from multiple vendors. But exporting/importing a key, specially with the no-export attribute set, among vendors is not available. I concur; moving keys not marked as CKA_EXTRACTABLE (at time of generation) is generally not supported (due to FIPS requirements).
jakob
A long while back (~15 years), I approached the CTO of a crypto token company about this specific problem. I had a need to be able to use the same root key pair for 20+ years over many different technologies/device types. He made a simple suggestion - generate the key pair outside the HSM, generate an encryption key, encrypt the private key, split the encryption key using Shamirs Secret Sharing. Print everything to archival paper. Lock up the printed shares in different places that were set up for physical escrow of intellectual property. When you need to load the stuff on a new device or new technology, combine the shares, decrypt the private key and load the HSM from the reconstituted keys. We ended up building a set of tools to do this - the key generation tool to print the shares, and the key combination tool to decrypt the private key. Every few years we send a few people out to a randomly selected PC parts place to randomly select a motherboard and all the parts we need to build a disk-less PC. We built a Linux live DVD (read-only) to run it. As part of all of this, we selected a good HSM that had both a rack mount format and a compatible PCIe card - the latter for use on the kiosk PC. Using the backup/restore functions we could load the kiosk HSM with the master keys the other HSM, create a PKCS11 token and key set, and back up that token. We could then secure wipe the kiosk HSM, and load the back up into the production HSM. The hardest part about the above is making sure data doesn't exfiltrate to things like printers (which sometimes sneakily have NV storage) via the USB connections. We actually considered shredding the printer after share creation, but believed we had locked things down enough that we shouldn't have an issue if we followed the correct steps and protected the tools when not in use (e.g. tamper bags, serialize tamper packaging) Our first technology was a PCMCIA card, our second was big iron HSMs with PKCS11 support. We've moved technology generations once within our current vendor and don't expect to have issues doing it again. There are quite a number of "rituals" needed to support something like this, and ours have evolved over many years. We use tamper bags and envelopes for the printed shares. We've got safe deposit boxes that require two party access to get to for most shares. Etc, etc. ---------------------------------- The above is one thought. ---------------------------------- Another approach is to use RSA threshold signatures based on Shoup's Practical Threshold Signatures paper. An implementation of this is here https://github.com/sweis/threshsig. A long while back (~2005) I wrote a DNSSEC signing tool using that as a core. This approach has a neat attribute - all the partial signature blobs are public and can be combined by anyone without the need for them to have a copy of any secrets. The KSK signing authority would send out something to be signed to the N key share holders and would get back ~N partial signatures over that data. Those partial signatures would be published, and the KSK signing authority would then publish a signature derived from M of those partial signatures to the DNS once verified. The output of the combining operation is verifiable using normal RSA math. This would obviate the need for a central HSM and people actually showing up to the facility to do the work. There would still be a need for a central facility to generate the individual key shares and provide them to the key share holders. All of my implementations were soft-keys, but it shouldn't be all that hard to build a smart card applet that does the partial signature stuff. I can think of a few ways to further secure these so it would be difficult or impossible to extract information from them without the collaboration of a few other key share cards. The way the math works, possession of less than N key shares provides an attacker with no information. The FROST2 threshold signature stuff currently wending its way through the CFRG does similar things for EC signatures - unfortunately not ECDSA compatible. That would require publication of a different EC signature algorithm with the associated time to implement and deploy. ---------------------------------- The above is a long way of saying - don't get hung up on a set of HSM requirements that may have made sense originally, but might be problematic now. It's probably time to reassess the HSM requirements in light of small number of big iron HSMs vendors left. Mike
participants (7)
-
Frederico A C Neves -
Jakob Schlyter -
James Mitchell -
Michael StJohns -
Olaf Kolkman -
Tomofumi Okubo -
Tomofumi Okubo