Patch gmplib to not use reserved x18 registers on darwin (Apple Silicon)

brandonfriess's Avatar

brandonfriess

23 Nov, 2022 03:15 PM

Which of our tools is giving you problems?

GPG Suite, but more specifically, the gmplib.

Attach a screenshot of the version info for all installed components (how to: https://gpgtools.tenderapp.com/kb/faq/where-can-i-find-version-info...):

Attached below

Describe your problem. Add as much detail as possible.

We are seeing the gpg client crash when trying to send keys and search keys on Apple Silicon machines running macOS Ventura (snippet of crash below):

Thread 1 Crashed:
0   libgmp.10.dylib                        0x10140baf0 __gmpn_sub_n + 136
1   libhogweed.6.1.dylib                   0x101146e0c _nettle_ecc_mod_random + 296
2   libhogweed.6.1.dylib                   0x101147aac nettle_ecdsa_generate_keypair + 144
3   ???                                 0x650167415a24 ???

After some investigation, per https://developer.apple.com/documentation/xcode/writing-arm64-code-..., Apple is reserving register x18 and explicitly says "don’t use this register".

We've also seen this (and similar) issues in other repos:

It appears the ECC crypto operations are using a reserved register that wasn’t previously in use, but now that Apple is using it, it's showing up as a segfault.

It looks like gmplib fixed this upstream in https://gmplib.org/repo/gmp/rev/5f32dbc41afc but GPG Suite 2022.2 doesn't have this?

gmplib version on my machine:

❯ strings /usr/local/MacGPG2/lib/libgmp.10.dylib | grep 6.2
6.2.1
6.2.1

What did you expect instead

I expect gpg --send-key and gpg --search-keys to not crash

Describe steps leading to the problem.

Commands that trigger the crash:

❯ gpg --send-keys FCBF72EF81BD9F1D0D86C1C50E5BB12345678910
gpg: sending key 0E5BB68982375825 to hkps://keymaster.company.com
gpg: keyserver send failed: End of file
gpg: keyserver send failed: End of file
❯ gpg --search-keys 7884711ADFA0E21D473C15F5EDD012345678910
gpg: error searching keyserver: End of file
gpg: keyserver search failed: End of file

Are you using any other Mail.app plugins?

No, we aren't using Mail.app plugins.

  1. Support Staff 1 Posted by Steve on 23 Nov, 2022 05:51 PM

    Steve's Avatar

    Hi Brandon,

    thanks for getting in touch and taking the time to report this crash. Your analysis is spot on.

    The crash is caused by an error in the bignum library libgmp used by gnutls, which is used for tls connections in dirmngr. The error only happens on M1/M2 and only when a certain register is written to which is reserved by M1/M2.

    Currently you can workaround this by retrying to show the email or to import the key when a key lookup caused the crash (depending on what triggered the error in the first place).

    We are looking into a solution to this problem and will inform you as soon as we have a fix available.

    Best,
    Steve

  2. 2 Posted by brandonfriess on 23 Nov, 2022 06:39 PM

    brandonfriess's Avatar

    Hi Steve,

    Thanks for the response and confirmation. Unfortunately, retrying the raw gpg cli commands hasn't been successful for us. If you know of any other work arounds or additional steps, do let us know!

    Thanks,
    Brandon

  3. Support Staff 3 Posted by Luke Le on 23 Nov, 2022 06:43 PM

    Luke Le's Avatar

    Hi Brandon,

    what else might help is killing the dirmngr process.

    Lukas

  4. 4 Posted by brandonfriess on 23 Nov, 2022 07:43 PM

    brandonfriess's Avatar

    Hi Lukas,

    Thanks for help!

    I've tried killing the drmngr process via kill -9 and gpgconf --kill dirmngr but each time I try to run gpg --search-keys it ends up spawning dirmngr and subsequently crashes.

    Process:               dirmngr [80516]
    Path:                  /usr/local/MacGPG2/bin/dirmngr
    Identifier:            dirmngr
    Version:               ???
    Code Type:             ARM-64 (Native)
    Parent Process:        launchd [1]
    Responsible:           iTerm2 [61083]
    User ID:               501
    
    Date/Time:             2022-11-23 13:38:52.5269 -0600
    OS Version:            macOS 13.0.1 (22A400)
    Report Version:        12
    Anonymous UUID:        E8DA786F-C425-6ABC-06A2-0B4DD910E50E
    
    Sleep/Wake UUID:       38968505-E6C8-4F0F-B798-47590D29C1AF
    
    Time Awake Since Boot: 640000 seconds
    Time Since Wake:       852 seconds
    
    System Integrity Protection: enabled
    
    Crashed Thread:        1
    
    Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
    Exception Codes:       KERN_INVALID_ADDRESS at 0x000000016dc7c000
    Exception Codes:       0x0000000000000001, 0x000000016dc7c000
    
    Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
    Terminating Process:   exc handler [80516]
    
    VM Region Info: 0x16dc7c000 is not in any region.  Bytes after previous region: 1  Bytes before following region: 567820288
          REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
          Stack                       16dbf4000-16dc7c000    [  544K] rw-/rwx SM=PRV  thread 1
    --->  GAP OF 0x21d84000 BYTES
          unused __TEXT               18fa00000-18fa58000    [  352K] r-x/r-x SM=COW  ...ed lib __TEXT
    
    Application Specific Information:
    *** multi-threaded process forked ***
    crashed on child side of fork pre-exec
    
    
    Thread 0::  Dispatch queue: com.apple.main-thread
    0   libsystem_kernel.dylib                 0x18fd95024 __pselect + 8
    1   libsystem_kernel.dylib                 0x18fda7b34 pselect + 112
    2   libnpth.0.dylib                        0x1025976ac npth_pselect + 96
    3   dirmngr                                0x102446544 handle_connections + 504
    4   dirmngr                                0x102444c1c main + 2736
    5   dyld                                   0x18faa3e50 start + 2544
    
    Thread 1 Crashed:
    0   libgmp.10.dylib                        0x1027b29f0 __gmpn_sub_n + 136
    1   libhogweed.6.4.dylib                   0x1026befac _nettle_ecc_mod_random + 340
    2   libhogweed.6.4.dylib                   0x1026bfc64 nettle_ecdsa_generate_keypair + 140
    3   ???                                 0x650167415a24 ???
    

    Cheers,
    Brandon

  5. Support Staff 5 Posted by Steve on 29 Nov, 2022 12:03 PM

    Steve's Avatar

    Hi Brandon,

    could you please download and install our latest hotfix GPG Suite and see if that solves your problem.

    All the best,
    Steve

    Disclaimer: Hotfixes are GPG Suite builds containing our latest source code, so bugs and crashes may occur.

  6. 6 Posted by brandonfriess on 29 Nov, 2022 03:52 PM

    brandonfriess's Avatar

    Hi Steve,

    Good news, the latest hotfix is working for me!

    Both the gpg --send-keys and gpg --search-keys commands that were previously failing are working for me.

  7. Support Staff 7 Posted by Steve on 01 Dec, 2022 11:11 PM

    Steve's Avatar

    Glad this is solved for you. I'm closing this discussion. Should you need further assistance or have questions you can re-open this discussion here or open a new one any time.

    Best,
    Steve

  8. Steve closed this discussion on 01 Dec, 2022 11:11 PM.

  9. brandonfriess re-opened this discussion on 02 Dec, 2022 02:37 PM

  10. 8 Posted by brandonfriess on 02 Dec, 2022 02:37 PM

    brandonfriess's Avatar

    Thanks for all the help!

    Out of curiosity, about how long until this fix lands into a stable release?

    Cheers,
    Brandon

  11. Support Staff 9 Posted by Steve on 04 Dec, 2022 07:39 PM

    Steve's Avatar

    We don't give ETAs but I don't see a release happening in the next few weeks. We are currently working on some internal changes on our infrastructure and are still investigating some problems with GPG Mail that should be part of the next release.

    Are you using GPG Mail more broadly at stripe and did you get many reports about this specific issue?

  12. 10 Posted by Mark Gardner on 04 Dec, 2022 11:45 PM

    Mark Gardner's Avatar

    FWIW I’ve been monitoring this issue after having the same problem, so it’s not just Brandon.

  13. 11 Posted by brandonfriess on 05 Dec, 2022 07:25 PM

    brandonfriess's Avatar

    Hey Steve,

    We don't use GPG Mail. Rather, we have a number of internal tools that make calls to the gpg binary. As we've rolled out more and more Apple Silicon machines, I've started to get reports of the errors outlined in this thread.

    Understandable on not providing ETAs. My motivation for asking was so that I could level set my expectations and plan internally.

    Thanks,
    Brandon

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Already uploaded files

  • Screenshot_2022-11-23_at_8.54.14_AM.png 929 KB

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac