[Bug gdb/20981] New: Dynamic library debugging on MacOS X 10.12 (Sierra) and dyld 15 is broken

[stefan.mb at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

Bug ID: 20981
Summary: Dynamic library debugging on MacOS X 10.12 (Sierra)
and dyld 15 is broken
Product: gdb
Version: HEAD
Status: UNCONFIRMED
Severity: normal
Priority: P2
Component: gdb
Assignee: unassigned at sourceware dot org
Reporter: stefan.mb at gmail dot com
Target Milestone: ---

I am running Mac OS X 10.12.2 (Sierra) and I've noticed gdb's support for
debugging dynamic libraries is broken. The following message is printed on
startup:

"warning: unhandled dyld version (15)"

It appears the latest dyld on MacOS Sierra has undergone major changes and the
version was bumped from 14 to 15.
(https://opensource.apple.com/source/dyld/dyld-421.2/)

Last time a version bump happened the fix was to simply modify the
DYLD_VERSION_MAX define (see
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=d4ccb5e05c99c4006fe43ab08ebe13b7a74fc111).

I tried doing this and was met with a new error:

"Cannot insert breakpoint -1
Cannot access memory at address 0xe85d"

The address 0xe85d is the offset of the "gdb_image_notifier" function in the
/usr/lib/dyld executable (see
https://opensource.apple.com/source/dyld/dyld-421.2/src/dyld_gdb.cpp.auto.html).

It appears Apple has changed something fundamental about how dyld is built
and/or loaded (possibly related to PIE/ASLR), here are before and after
examples:

Mac OS X 10.11.6:

otool -l /usr/lib/dyld
Mach header
magic cputype cpusubtype caps filetype ncmds sizeofcmds flags
0xfeedfacf 16777223 3 0x00 7 14 1696 0x00000085
Load command 0
cmd LC_SEGMENT_64
cmdsize 552
segname __TEXT
vmaddr 0x00007fff5fc00000
vmsize 0x0000000000038000
fileoff 0
filesize 229376
maxprot 0x00000007
initprot 0x00000005
nsects 6
flags 0x0

Mac OS X 10.12.2:

otool -l /usr/lib/dyld
/usr/lib/dyld:
Mach header
magic cputype cpusubtype caps filetype ncmds sizeofcmds flags
0xfeedfacf 16777223 3 0x00 7 14 1696 0x00000085
Load command 0
cmd LC_SEGMENT_64
cmdsize 552
segname __TEXT
vmaddr 0x0000000000000000
vmsize 0x000000000003e000
fileoff 0
filesize 253952
maxprot 0x00000007
initprot 0x00000005
nsects 6
flags 0x0

Notice that vmaddr is now 0x0. I wrote a hack to add the base address of the
TEXT segment of dyld to the notifier address and this gets gdb to start
successfully:
https://github.com/stefanmb/binutils-gdb/commit/ee5bfeff7174f6e0b515f87d8ba2214ce45a02aa

With the patch "info shared" appears to work correctly.

I am now encountering further issues related to shared libraries and forking
(children are seeing SIGSEGV after my patch), but I have not had a chance to
investigate further. I will update this issue if I find more relevant info.

This behaviour reproduces on the latest gdb (7.12.50.20161217-git).

Does anyone know what has to be done to fix this issue properly?

I am available to test potential fixes, but I require some support to resolve
this problem properly.

Thank you very much for your help!

[stefan.mb at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

Stefan Budeanu <stefan.mb at gmail dot com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |stefan.mb at gmail dot com

[simon at pushface dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

Simon Wright <simon at pushface dot org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |simon at pushface dot org

[mephi42 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

mephi42 at gmail dot com changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |mephi42 at gmail dot com

[jothiram.s at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

Joe <jothiram.s at gmail dot com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |jothiram.s at gmail dot com

[jothiram.s at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

--- Comment #2 from Joe <jothiram.s at gmail dot com> ---
In my case, the gdb is starting the original program successfully, but

gdb) info shared

does not show any information. Also, the break points that I had set in the
shared library are not detected and the execution does not stop there.

[shahzeb001 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

Shahzeb <shahzeb001 at gmail dot com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |shahzeb001 at gmail dot com

--- Comment #3 from Shahzeb <shahzeb001 at gmail dot com> ---
Any update on this?

I've uninstalled and then installed gdb twice (via brew). I've also codesigned
it. Still get "No shared libraries loaded at this time.". Really frustrating.

macOS: 10.12.4

[chrisfriedt at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

Chris Friedt <chrisfriedt at gmail dot com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |chrisfriedt at gmail dot com

--- Comment #4 from Chris Friedt <chrisfriedt at gmail dot com> ---
I'm also encountering this on 10.12.6 ... sigh..

[evan.torrie at yahoo dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

evan.torrie at yahoo dot com changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |evan.torrie at yahoo dot com

--- Comment #5 from evan.torrie at yahoo dot com ---
This also occurs on Mac OS X 10.13 (High Sierra)

[shahzeb001 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

--- Comment #6 from Shahzeb <shahzeb001 at gmail dot com> ---
This problem still isn't resolved, but I came up with a solution: Simply run a
docker container I've built which has GDB within it. Instructions on my github:
https://github.com/shahzeb1/cpp-gdb-docker-image

[question123446 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

Andy Law <question123446 at gmail dot com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |question123446 at gmail dot com

--- Comment #7 from Andy Law <question123446 at gmail dot com> ---
Gdb 8.0.1 doesn't have this problem as I see. However, the dynamic library
still can't debug at all. It can't collect debug symbols...

[wsimmons at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

Wade Simmons <wsimmons at gmail dot com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |wsimmons at gmail dot com

[chrisfriedt at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

--- Comment #8 from Chris Friedt <chrisfriedt at gmail dot com> ---
#sadface :(

[palves at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

Pedro Alves <palves at redhat dot com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |palves at redhat dot com

--- Comment #9 from Pedro Alves <palves at redhat dot com> ---
The simple hard fact is that unfortunately the macOS/Darwin port isn't really
getting much attention. Most gdb devs tend to run/focus on GNU/Linux.
We need someone motivated to actually dig in to the sources, study the issue in
detail, and fix the issue. Who knows, maybe end up becoming port maintainer.
:-)

This could be you!

[benconvey at hotmail dot co.uk]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

ben convey <benconvey at hotmail dot co.uk> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |benconvey at hotmail dot co.uk

--- Comment #10 from ben convey <benconvey at hotmail dot co.uk> ---
same issue on high sierra ( 10.13.2 ) :(

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

Tom Tromey <tromey at sourceware dot org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Target| |*-*-darwin*
CC| |tromey at sourceware dot org

[chrisfriedt at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

--- Comment #11 from Chris Friedt <chrisfriedt at gmail dot com> ---
These days, things are also just being run as Linux binaries inside of a Docker
container on Mac OS X.. except those rare occasions when it *really* needs to
be MachO.

Maybe one of these days I'll get around to it.

[cvs-commit at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

--- Comment #12 from cvs-commit at gcc dot gnu.org <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Tom Tromey <***@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=79b32f4a3a109321cb9a99014293b0ade57535ab

commit 79b32f4a3a109321cb9a99014293b0ade57535ab
Author: Xavier Roirand <***@adacore.com>
Date: Wed Aug 22 12:11:14 2018 +0200

Darwin: Handle unrelocated dyld.

On Darwin, debugging an helloworld program with GDB does
not work and ends with:

(gdb) set startup-with-shell off
(gdb) start
Temporary breakpoint 1 at 0x100000fb4: file /tmp/helloworld.c, line 1.
Starting program: /private/tmp/helloworld
[New Thread 0x2703 of process 18906]
[New Thread 0x2603 of process 18906]

[1]+ Stopped ./gdb/gdb /tmp/helloworld

When debugging with lldb, instead of having the STOP signal, we can
see that a breakpoint is not set to a proper location:

Warning:
Cannot insert breakpoint -1.
Cannot access memory at address 0xf726

Command aborted.

The inserted breakpoint is the one used when GDB has to stop the target
when a shared library is loaded or unloaded. The notifier address used
for adding the breakpoint is wrong thus the above failure.
This notifier address is an offset relative to dyld base address, so
the value calculation has to be updated to reflect this.

This was tested on High Sierra by trying to run a simple "hello world"
program.

gdb/ChangeLog:

PR gdb/20981:
* solib-darwin.c (darwin_get_dyld_bfd): New function.
(darwin_solib_get_all_image_info_addr_at_init): Update call.
(darwin_handle_solib_event): New function.
(darwin_solib_create_inferior_hook): Handle unrelocated dyld.

Change-Id: I7dde5008c9158f17b78dc89bd7f4bd8a12d4a6e1

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=20981

Tom Tromey <tromey at sourceware dot org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |RESOLVED
Resolution|--- |FIXED
Target Milestone|--- |8.3

--- Comment #13 from Tom Tromey <tromey at sourceware dot org> ---
I've pushed a variant of Xavier's patch which fixes the problem
for me on High Sierra. I think this is fixed.

However, I could only test High Sierra. So, if you can check
out git master gdb and try it on other versions, that would be nice.

I am going to close this bug; but feel free to report back here
(I'll still be listening) and/or reopen if there are similar problems.