[Bug remote/23801] New: remote target code slowed by unnecessary remote_check_symbols() calls

[jburgess777 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=23801

Bug ID: 23801
Summary: remote target code slowed by unnecessary
remote_check_symbols() calls
Product: gdb
Version: unknown
Status: UNCONFIRMED
Severity: normal
Priority: P2
Component: remote
Assignee: unassigned at sourceware dot org
Reporter: jburgess777 at gmail dot com
Target Milestone: ---

Created attachment 11348
--> https://sourceware.org/bugzilla/attachment.cgi?id=11348&action=edit
Proof of concept which defers the remote_check_symbols()

When debugging remote target over a high latency link I noticed that gdb was
sending $qSymbol:: messages for every shared library loaded during the
execution of the 'target remote' command. The remote target responded by
attempting symbol lookup calls multiple times until the library it was looking
for was eventually loaded and these finally succeeded. In the case of separate
debug info we see two calls per library:

$ egrep '(Reading|\$qSymbol::#5b)' /tmp/old
Reading symbols from /lib64/libpthread.so.0...Reading symbols from
/usr/lib/debug/usr/lib64/libpthread-2.27.so.debug...done.
Sending packet: $qSymbol::#5b...Packet received:
qSymbol:6764625f6167656e745f6764625f74705f686561705f627566666572
Sending packet: $qSymbol::#5b...Packet received:
qSymbol:6764625f6167656e745f6764625f74705f686561705f627566666572
Reading symbols from /lib64/libinproctrace.so...Reading symbols from
/usr/lib/debug/usr/lib64/libinproctrace.so-8.1.1-3.fc28.x86_64.debug...done.
Sending packet: $qSymbol::#5b...Packet received:
qSymbol:6764625f6167656e745f6764625f74705f686561705f627566666572
Sending packet: $qSymbol::#5b...Packet received: OK
Reading symbols from /lib64/libc.so.6...Reading symbols from
/usr/lib/debug/usr/lib64/libc-2.27.so.debug...done.
Sending packet: $qSymbol::#5b...Packet received: OK
Sending packet: $qSymbol::#5b...Packet received: OK
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from
/usr/lib/debug/usr/lib64/ld-2.27.so.debug...done.
Sending packet: $qSymbol::#5b...Packet received: OK
Sending packet: $qSymbol::#5b...Packet received: OK
Reading symbols from /lib64/libdl.so.2...Reading symbols from
/usr/lib/debug/usr/lib64/libdl-2.27.so.debug...done.
Sending packet: $qSymbol::#5b...Packet received: OK
Sending packet: $qSymbol::#5b...Packet received: OK
Reading symbols from /lib64/libstdc++.so.6...Reading symbols from
/usr/lib/debug/usr/lib64/libstdc++.so.6.0.25-8.1.1-5.fc28.x86_64.debug...done.
Sending packet: $qSymbol::#5b...Packet received: OK
Sending packet: $qSymbol::#5b...Packet received: OK
Reading symbols from /lib64/libm.so.6...Reading symbols from
/usr/lib/debug/usr/lib64/libm-2.27.so.debug...done.
Sending packet: $qSymbol::#5b...Packet received: OK
Sending packet: $qSymbol::#5b...Packet received: OK
Reading symbols from /lib64/libgcc_s.so.1...Reading symbols from
/usr/lib/debug/usr/lib64/libgcc_s-8-20180712.so.1-8.1.1-5.fc28.x86_64.debug...done.
Sending packet: $qSymbol::#5b...Packet received: OK
Sending packet: $qSymbol::#5b...Packet received: OK
Sending packet: $qSymbol::#5b...Packet received: OK
Sending packet: $qSymbol::#5b...Packet received: OK

Performing this round trip for every library seems unnecessary. I was wondering
whether gdb could defer the remote_check_symbols() calls until after all the
initial libraries have been loaded.

The attached patch is a proof of concept which defers the
remote_check_symbols() call until the end of the 'target remote' call and this
dramatically reduces the number of qSymbol:: round trips with the remote target
while still allowing it to resolve all the symbols it asked for:

$ egrep '(Reading|\$qSymbol::#5b)' /tmp/new
Reading symbols from /lib64/libpthread.so.0...Reading symbols from
/usr/lib/debug/usr/lib64/libpthread-2.27.so.debug...done.
Reading symbols from /lib64/libinproctrace.so...Reading symbols from
/usr/lib/debug/usr/lib64/libinproctrace.so-8.1.1-3.fc28.x86_64.debug...done.
Reading symbols from /lib64/libc.so.6...Reading symbols from
/usr/lib/debug/usr/lib64/libc-2.27.so.debug...done.
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from
/usr/lib/debug/usr/lib64/ld-2.27.so.debug...done.
Reading symbols from /lib64/libdl.so.2...Reading symbols from
/usr/lib/debug/usr/lib64/libdl-2.27.so.debug...done.
Reading symbols from /lib64/libstdc++.so.6...Reading symbols from
/usr/lib/debug/usr/lib64/libstdc++.so.6.0.25-8.1.1-5.fc28.x86_64.debug...done.
Reading symbols from /lib64/libm.so.6...Reading symbols from
/usr/lib/debug/usr/lib64/libm-2.27.so.debug...done.
Reading symbols from /lib64/libgcc_s.so.1...Reading symbols from
/usr/lib/debug/usr/lib64/libgcc_s-8-20180712.so.1-8.1.1-5.fc28.x86_64.debug...done.
Sending packet: $qSymbol::#5b...Packet received:
qSymbol:6764625f6167656e745f6764625f74705f686561705f627566666572
Sending packet: $qSymbol::#5b...Packet received: OK

The output above is a patched version of Fdeora 28 GDB but the attached patch
is against binutils-gdb master.

In this simple example it removes about 10% of the round trips with the remote:

$ grep -c 'Sending packet:' /tmp/old
185
$ grep -c 'Sending packet:' /tmp/new
167

With more shared libraries the saving would be greater.

[simon.marchi at ericsson dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=23801

Simon Marchi <simon.marchi at ericsson dot com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |simon.marchi at ericsson dot com

--- Comment #1 from Simon Marchi <simon.marchi at ericsson dot com> ---
That would make sense, I believe.

It would be good to give it a test run to make sure this introduces no
regression, with

make check RUNTESTFLAGS="--target_board=native-gdbserver"

and

make check RUNTESTFLAGS="--target_board=native-extended-gdbserver"

It is expected (but not desired) that there are many failures when running the
testsuite, so we usually run the tests without and with the patch, and then
compare the two resulting testsuite/gdb.sum files. More details here:

https://sourceware.org/gdb/wiki/TestingGDB

If this works well, here are the instructions for submitting a patch:

https://sourceware.org/gdb/wiki/ContributionChecklist