Home → Stable →

GPGMail not recognizing public key changes made via CLI

• Edit

gpg_dude

02 Feb, 2019 02:07 AM

I had some people tell me they weren't able to encrypt messages to someone whose key had recently expired. I wrote it off to them not having updated said public key in their keyring(s). Until today, I went to email this same person today and found the lock icon for encryption was greyed out and hovering over it I was told GPG could not find a usable key for them. I looked in my GPG keyring and the key was present, valid (not expired or revoked), with 3 green bars as it is signed by a key I trust. I was puzzled by this and first tried "update from key server" and that seemed to resolve the issue and now the lock icon in mail was no longer greyed out and I could encrypt the message to this key and send it.

I started doing some debugging and believe I found the cause. I run a bash script to manage certain public keys in my keyring, including this one. I am able to reliably reproduce the bad behavior using GPG 2018.5 on macOS 10.12 Sierra, 10.13 High Sierra, & 10.14 Mojave using the following steps:

• Open mail.app & compose a message to an email address that you do not have a valid key for (the key can be expired or just not in your local keyring)
• Confirm that the lock icon is greyed out due to the invalid/missing key
• Open terminal and use the CLI to import the public key for this email address. You can either "gpg --search" or the key or "gpg --import" the key if you already have it downloaded
• Go back into mail and confirm that the lock icon continues to be greyed out

I found two ways to recover from this scenario:

1) Restart mail.app
2) In GPG Keychain, select the key and do "update from keyserver"

After doing one of the above, the lock icon becomes clickable and the message can be encrypted and sent.

1. Support Staff 1 Posted by Steve on 04 Feb, 2019 08:53 PM

   

   Hi there,

   this is indeed expected behavior. Changes to gpg or your key setup require a restart of Mail.app to be reflected.

   I'll ping Luke about this behavior, but I am pretty sure there's no trivial fix to make GPG Mail reflect such changes without having to restart.

   There are some cases where no restart is necessary. I just tried this

   1. remove [email blocked] public key from GPG Keychain
      
   2. Create new draft to team email (lock button is greyed out: expected)
      
   3. While composing the draft, switch to GPG Keychain and download public key
      
   4. go back to open draft

   The encrypt button is enabled (as I use the default to encrypt emails). So at least on macOS 10.14.3 with latest GPG Suite it's not necessary to restart Mail.app in all cases. But if changes are not picked up, a restart will do the trick.

   All the best,
   Steve

   • Edit

2. 2 Posted by gpg_dude on 04 Feb, 2019 08:58 PM

   

   Perhaps Luke can shed some light as to the reason why a restart is required? Or perhaps what it is GPG Keychain is doing that the CLI command is not that allows GPGMail to see the update?

   This behavior is causing problems for those of us who have scripted some levels of key management as the user can't use the updated key(s) without restarting Mail.app which is confusing since they likely won't be aware something was updated in the background while they had mail.app open.

   • Edit

3. Support Staff 3 Posted by Steve on 05 Feb, 2019 08:21 AM

   

   Are those scripts about automatic key updates in the background? Could you share a bit what use-cases they cover?

   • Edit

4. 4 Posted by gpg_dude on 07 Feb, 2019 04:19 AM

   

   i guess you could consider them in the background ... they happen via an app that runs a shell script on a xhecule. The use case is to try to automate the lion share of key management across a number of interconnected organizations so users don't need to search/download/refresh keys provided they are signed by an appropriate authority. So the app does all that heavy lifting behind the scenes:

   • searching for & downloading keys within a domain to a temp keyring

   • checking signatures and revocations on said keys and decideding what action to take (export from temp keyring and import to user's keyring or removing it from the user's keyring if it's deemed invalid)

   • Edit

5. Support Staff 5 Posted by Luke Le on 13 Feb, 2019 06:10 PM

   

   Hi gpg_dude,

   thank you for the detailed description of your workflow. Libmacgpg (the framework communicating with GnuPG) takes a best effort to inform all of our apps of keyring changes, whether they happen in GPG Keychain and GPG Mail has to be informed, or they happen directly via command line scripts and GPG Keychain as well as GPG Mail have to be informed. Unfortunately these notifications are not always reliable and are not even always sent by the system. We have already much improved these notifications in the past, but unfortunately there are still times were a restart will be necessary.

   • Edit

6. 6 Posted by gpg_dude on 13 Feb, 2019 10:01 PM

   

   Is there any way I could run a command to trigger the necessary notifications myself?

   • Edit

7. 7 Posted by gpg_dude on 27 Feb, 2019 09:48 PM

   

   ^Bump

   • Edit

8. 8 Posted by Mento on 01 Mar, 2019 09:06 AM

   

   Hi,

   you can send the required notification with a single line of Objective-C code:
   

   [[NSDistributedNotificationCenter defaultCenter] postNotificationName:@"GPGKeysChangedNotification" object:nil userInfo:nil deliverImmediately:YES];
   

   Regards, Mento

   • Edit

9. 9 Posted by gpg_dude on 03 Mar, 2019 10:15 PM

   

   @Mento - could you provide a bit more context/code for me as I'm not versed in Objective-C code. Ideally I'd have a simple program that could be compiled & called from within the shell script to notify all the GPGTools UI apps that the public keyring has changed and so they will see the updated data.

   • Edit

10. Support Staff 10 Posted by Luke Le on 03 Mar, 2019 10:42 PM

    

    Hi,

    we have identified a bug which might in fact affect you. Not all keyring related files were checked for updates. While usually pubring.gpg is updated even on newer Installations of GnuPG, it can happen that only pubring.kbx is updated, in which case our tools were not properly notified of that update.

    We believe to have fixed this issue in our latest hotfix: https://releases.gpgtools.org/nightlies/GPG_Suite-2404n.dmg

    • Edit

11. 11 Posted by gpg_dude on 04 Mar, 2019 03:28 AM

    

    @Luke - that version is no good for me. GPG Keychain crashes on startup. See attached.

    • gpg-crash.txt 49.5 KB
    • Edit

12. Support Staff 12 Posted by Luke Le on 04 Mar, 2019 10:38 AM

    

    That is interesting. We will have a look and report back once we know more. Thanks for the update!

    • Edit

13. Support Staff 13 Posted by Steve on 04 Mar, 2019 09:28 PM

    

    We are tracking this crash in a ticket which I connected to this discussion. We will fix this problem this or the coming week and report back. Until then please revert to the stable release to use GPG Keychain.

    Sorry for the inconvenience and thanks for your patience.

    • Edit

14. Support Staff 14 Posted by Steve on 06 Mar, 2019 01:26 PM

    

    Could you please try with GPG Suite 2405n.

    • Edit

15. 15 Posted by gpg_dude on 06 Mar, 2019 07:16 PM

    

    @Steve - that looks like it did the trick. I imported an expired key. They updated it via CLI (both with --refresh-keys & with --export --import from a temporary keyring that had the updated key) ... with GPG Keychain & Mail.app open ... and was able to verify both apps did see the update. Thanks!

    Can you confirm if this fix will make it into the next stable release and roughly when to expect it?

    • Edit

16. 16 Posted by gpg_dude on 06 Mar, 2019 07:16 PM

    

    Also FYI, lately I've been getting errors whenever I submit comments

    We're sorry, but something went wrong We've been notified about this issue and we'll take a look at it shortly.

    but when I refresh the page I find my comment is always saved.

    • Edit

17. Support Staff 17 Posted by Steve on 06 Mar, 2019 07:23 PM

    

    Fix will surely be in the next release. ETAs we don't give. There's an unsolved problem in regards to the proprietary and not very well documented OpenPGP implementation Symantec uses in its PGP Desktop software. Our plan is to wait for that problem to be solved. It's been painful to solve and used up a lot of development time already. Sadly we are not there yet.

    Once that is done, the next release should not be too far away.

    About the problems you are seeing when commenting: Are you posting from your browser? And if yes, which browser and version are you using? I am not seeing such errors and we've not yet heard complaints from other users.

    If this persists, we should probably file a discussion with the maker of this support platform: https://help.tenderapp.com/

    All the best,
    Steve

    • Edit

18. 18 Posted by gpg_dude on 15 Mar, 2019 09:25 PM

    

    FYI - while running this nightly I had a very strange issue come and go. I asked GPG Keychain to do a search and it hung for a long time. Even after cancelling it, since I was querying my own key server I found my client was repeatedly querying for the same string over and over and over again. I had to force kill dirmngr to make it stop. I just tried it again and it's now not doing it (in favor of returning "no keys found" as expected almost immediately). Not sure you can do much with that report, but wanted to mention it in case it lines up with anything else you've heard.

    Understood about the ETA, just a habit to ask.

    I am posting from the latest version of Chrome on Mac (currently 73.0.3683.75, though I think that's a recent update from 72.x). I'll see if it does it now when I try to submit this comment. I also run UBlock Origin & HTTPS everywhere extensions.

    • Edit

19. 19 Posted by gpg_dude on 15 Mar, 2019 09:26 PM

    

    Of course this time it submitted OK without the error. ¯\_(ツ)_/¯

    • Edit

20. Support Staff 20 Posted by Steve on 15 Mar, 2019 10:39 PM

    

    Thanks for your feedback on the nightly build.

    Sadly both issues seem non trivial to resolve. https everywhere should be fine as should uBlock Origin. Using the latter here as well.

    Can you keep an eye on both issues you mentioned (forum posting problems and the key server lookup glitch) and do ping us if either reoccurs.

    That would be helpful.

    Thanks and have a great weekend,
    Steve

    • Edit

21. 21 Posted by gpg_dude on 16 Sep, 2019 05:21 PM

    

    I think this can be closed unless there is something still pending. The problem originally reported seems to have been fixed in a subsequent release.

    • Edit

22. gpg_dude closed this discussion on 16 Sep, 2019 05:22 PM.