Home → Stable →

cannot upload existing public keys to hockeypuck key servers

• Edit

gpg_dude

07 Jul, 2022 09:49 PM

Which of our tools is giving you problems?

GPG Keychain

Attach a screenshot of the version info for all installed components (how to: https://gpgtools.tenderapp.com/kb/faq/where-can-i-find-version-info...):

Attached Screen Shot 2022-07-07 at 14.24.01.png

Describe your problem. Add as much detail as possible.

In the process of testing a cut over from the older SKS key server software to the newer Hockeypuck key server software, I was having issues uploading keys to the Hockey puck keyserver. GPG Keychain throws the error stating "Public key not allowed on key servers. The following keys do not reside on the key servers:"

However, the public key in question did already reside on the key server. The same key could be uploaded to the older SKS key servers without issue. I switched from hkps to hkp and watched the network traffic on the wire and found the following difference between SKS & Hockeypuck in response to the GET query GPG Keychain appears to make to determine whether or not the public key is already present on the server (e.g. GET /pks/lookup?op=index&options=mr&search=0x40_CHARACTER_KEY_FINGERPRINT)

SKS:

info:1:1
pub:0x40_CHARACTER_KEY_FINGERPRINT:1:4096:timestamp1:timestamp2:
uid:First1 Last1 <[email blocked]>:timestamp1::
uid:First2 Last2 <[email blocked]>:timestamp2::

HOCKEYPUCK:

info:1:1
pub:16_CHARACTER_KEY_ID:1:4096:timestamp1::
uid:First2 Last2 <[email blocked]>:timestamp1:timestamp2:
uid:First1 Last1 <[email blocked]>:timestamp1:timestamp2:

Based on the above responses, GPG Keychain refused to continue with the upload after it received the response from Hockeypuck but was fine to proceed with the upload after it received the response from SKS. I suspect the issue is the 16_CHARACTER_KEY_ID in Hockeypuck vs. the 40_CHARACTER_KEY_FINGERPRINT in SKS.

I also found that after switching key servers a number of times to test this I was able to successfully upload the key to a hockeypuck key server - specifically keyserver.ubuntu.com. When I went to check network traffic on the wire again to find what was different I found GPG Keychain had immediately uploaded the public key via a POST without even checking if the public key already existed on the key server via a GET. I switched back to one of my Hockeypuck key servers and observed the same change in behavior. I later switched key servers a few more times (after leaving things idle for a bit to start writing up this report) and now it is back to checking for the public key's existence first and then failing to upload to any/all Hockeypuck-based key servers (including keyserver.ubuntu.com), claiming "The following keys do not reside on the key servers:"

What did you expect instead

I expect GPG Keychain to be able to upload public keys to Hockeypuck-based key servers. I can inquire with the Hockeypuck developers to see if they can change the server to respond with the 40_CHARACTER_KEY_FINGERPRINT instead of the 16_CHARACTER_KEY_ID, but am also logging the issue here for the GPG Tools team to look into.

I would also expect the behavior to be consistent. If GPG Keychain is supposed to check for the public key's existence on the key server prior to uploading I would expect this to happen 100% of the time. I cannot explain how/why that behavior seemed to change for me midstream (when it stopped checking & just uploaded the key) and then later change back (when it resumed checking and refused to upload the key).

Describe steps leading to the problem.

See above

Are you using any other Mail.app plugins?

n/a - not in Mail.app at all in this case

• Screen_Shot_2022-07-07_at_14.24.01.png 101 KB

1. 1 Posted by andrewg on 14 Jul, 2022 07:44 PM

   

   This is probably because SKS returns a fingerprint in this field, whereas hockeypuck returns a long-id.

   According to the draft HKP spec, it is compliant to return any of the fingerprint, the short-id or the long-id in a machine-readable [v]index query:

      The key listings themselves are made up of several lines per key.
      The first line specifies the primary key:
   
      pub:<keyid>:<algo>:<keylen>:<creationdate>:<expirationdate>:<flags>
   
         <keyid> = this is either the fingerprint or the key ID of the
                   key.  Either the 16-digit or 8-digit key IDs are
                   acceptable, but obviously the fingerprint is best.  A
                   keyserver should use the most specific of the key IDs
                   that it has available.  Since it is not possible to
                   calculate the key ID from a V3 key fingerprint, for V3
                   keys this should be either the 16-digit or 8-digit
                   key ID only.
   

   https://datatracker.ietf.org/doc/html/draft-shaw-openpgp-hkp-00#sec...

   So GPGTools should accept userIDs as well as fingerprints in this field.

   • Edit

2. 2 Posted by andrewg on 14 Jul, 2022 08:36 PM

   

   One possible workaround would be to append &fingerprint=on to the call to /pks/lookup?op=vindex

   • Edit

3. 3 Posted by gpg_dude on 14 Jul, 2022 09:34 PM

   

   Thanks for the updates, Andrew. I've edited the original report to more accurately describe the difference being between the 40 character key fingerprint vs. the 16 character key ID. I'll also look at modifying the proxy server to automatically add that argument onto incoming queries as a workaround until hockeypuck and/or GPG Tools behavior changes.

   • Edit

4. 4 Posted by gpg_dude on 15 Jul, 2022 12:29 AM

   

   The workaround is successful. Adding this to the proxy server's location /pks block does the trick:

   rewrite ^/pks/lookup(.*) $uri?fingerprint=on break;

   So while it will still be nice to get this fixed, it's not as urgent. The hkps fallback keyserver would take priority and could really use fixing given how long it's been broken now and how isolated the fix would seem to be.

   • Edit

5. Support Staff 5 Posted by Steve on 01 Oct, 2022 01:35 PM

   

   Hi gpg_dude,

   please excuse the long silence and late feedback. I had hoped that by now we would have gotten around to fixing the fallback key server situation.

   Sadly that is still not the case. So for now I connected this discussion with the open ticket about that problem.

   Sorry we don't have better news.

   Best,
   Steve

   • Edit

6. Steve closed this discussion on 21 Oct, 2022 05:07 PM.

7. gpg_dude re-opened this discussion on 21 Oct, 2022 07:11 PM

8. 6 Posted by gpg_dude on 21 Oct, 2022 07:11 PM

   

   Just making sure that the original problem report here of GPG tools not being able to upload existing keys if the key server does not respond with a full finger print vs long key id is not getting lost in the mention of the fall back key server which is being trakced in another post.

   • Edit

9. Support Staff 7 Posted by Steve on 21 Oct, 2022 08:55 PM

   

   This should be fixed by MacGPG 2.2.40 which changes how key servers respond to requests.

   Could you please recheck, once the next release is out.

   • Edit

10. Support Staff 8 Posted by Steve on 04 Apr, 2023 12:37 PM

    

    Were you able to re-test this in GPG Suite 2023.2? Is the issue resolved for you?

    • Edit

11. 9 Posted by gpg_dude on 04 Apr, 2023 10:35 PM

    

    Hi Steve,
    I'm not sure how/where I could test this since I already modified my Hockeypuck key server to provide the response GPG Keychain was expecting. I also am not being offered 2023.2 as an update - it says 2023.1 is the latest when I use the built-in check for updates function

    • Edit

12. Support Staff 10 Posted by Steve on 05 Apr, 2023 07:08 PM

    

    That is likely because 2023.2 is ever only shown to macOS Ventura users, since the only change was the UUID change for macOS 13.3. Let's assume this fixed with 2.2.40.

    • Edit

13. Steve closed this discussion on 05 Apr, 2023 07:08 PM.