mingwei.zhang at intel dot com
2018-05-16 20:22:01 UTC
https://sourceware.org/bugzilla/show_bug.cgi?id=23191
Bug ID: 23191
Summary: GDB crashes when calling a (dummy) user function at
breakpoint in a multi-thread program. Use
scheduler-locking does not help.
Product: gdb
Version: 7.12
Status: UNCONFIRMED
Severity: normal
Priority: P2
Component: python
Assignee: unassigned at sourceware dot org
Reporter: mingwei.zhang at intel dot com
Target Milestone: ---
Created attachment 11022
--> https://sourceware.org/bugzilla/attachment.cgi?id=11022&action=edit
code to reproduce the bug
GDB version: 7.12.50
OS: Ubuntu 17.04
Kernel: 4.10.0-21-generic
python: v3.5
Bug Description
======================================================================
I try to test a simple multi-thread program in C. Before the program launches,
I add a break point using GDB python API in the function that would be called
by all worker threads. Inside the break point, I add some code which would call
a dummy function "bar()" in the target program. To prevent concurrency, I use
"set scheduler-locking on/off" to wrap my function call.
The problem is that at some point, when a new thread is created, the
scheduler-locking is broken and 1st thread entering "bar()" would hang there
and when the 2nd thread gets into "bar()", gdb crashes. After checking gdb
database, this bug seems related with Bug 20291.
I put the code to help reproduce the problem.
Code
======================================================================
C code:
----------------------------------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
struct thread_info { /* Used as argument to thread_start() */
pthread_t thread_id; /* ID returned by pthread_create() */
int thread_num; /* Application-defined thread # */
char *argv_string; /* From command-line argument */
};
/* some function that should report something to gdb */
int bar()
{
/* do nothing */
return 0;
}
void execute_task(int id)
{
printf("worker %d, execute task\n", id);
}
void *worker(void *arg)
{
int idx = 0;
int rounds = 10;
struct thread_info *tinfo = (struct thread_info *)arg;
for (idx = 0; idx < rounds; idx++) {
execute_task(tinfo->thread_num);
}
}
void main()
{
int tnum = 3;
int num_threads = tnum;
void *res;
struct thread_info *tinfo = calloc(num_threads,
sizeof(struct thread_info));
for (tnum = 0; tnum < num_threads; tnum++) {
tinfo[tnum].thread_num = tnum + 1;
pthread_create(&tinfo[tnum].thread_id, NULL,
&worker, &tinfo[tnum]);
}
for (tnum = 0; tnum < num_threads; tnum++) {
pthread_join(tinfo[tnum].thread_id, &res);
}
}
Note: this is a simple C program that create 3 worker thread, each of which is
basically print and done.
gdb python code:
----------------------------------------------------------------------
class IgnoreErrorsCommand (gdb.Command):
"""Execute a single command, ignoring all errors.
Only one-line commands are supported.
This is primarily useful in scripts."""
def __init__ (self):
super (IgnoreErrorsCommand, self).__init__ ("ignore-errors",
gdb.COMMAND_OBSCURE,
# FIXME...
gdb.COMPLETE_COMMAND)
def invoke (self, arg, from_tty):
try:
gdb.execute (arg, from_tty)
except:
pass
IgnoreErrorsCommand ()
class ReturnBreakpoint(gdb.Breakpoint):
def stop (self):
print("come to returnbreakpoint")
tid = gdb.selected_thread().num
print("thread id: %d"%tid)
gdb.execute("call $myfunc(0)")
gdb.write("done calling function\n")
return False
ReturnBreakpoint("*execute_task")
class MyCmd (gdb.Function):
"""JIT Code Memory Permission Controller using Pkey."""
def __init__ (self):
super (MyCmd, self).__init__ ("myfunc")
def invoke (self, arg):
print("++++++++++++++++++command execution begin+++++++++++++++")
tid = gdb.selected_thread().num
try:
gdb.execute("set scheduler-locking on")
gdb.execute("call bar()")
gdb.execute("set scheduler-locking off")
except:
print("*****exception*****")
print("++++++++++++++++++command execution done++++++++++++++++")
return "done"
MyCmd()
gdbinit file:
----------------------------------------------------------------------
set logging on
set pagination off
set target-async 1
set non-stop on
set breakpoint pending on
set print thread-events on
set confirm off
handle 11 nopass
source ./python_code.py
b *0x0
ignore-errors r
del 2
r
gdb launch command:
----------------------------------------------------------------------
gdb -x gdbinit ../test
Result:
======================================================================
GDB response:
Reading symbols from ../test...done.
Breakpoint 1 at 0x80b: file main.c, line 21.
Breakpoint 2 at 0x0
No unwaited-for children left.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff77f1700 (LWP 17090)]
[New Thread 0x7ffff6ff0700 (LWP 17091)]
come to returnbreakpoint
thread id: 2
++++++++++++++++++command execution begin+++++++++++++++
[New Thread 0x7ffff67ef700 (LWP 17092)]
$1 = 0
++++++++++++++++++command execution done++++++++++++++++
$2 = "done"
done calling function
come to returnbreakpoint
thread id: 3
++++++++++++++++++command execution begin+++++++++++++++
worker 1, execute task
come to returnbreakpoint
thread id: 4
++++++++++++++++++command execution begin+++++++++++++++
come to returnbreakpoint
thread id: 2
++++++++++++++++++command execution begin+++++++++++++++
/build/gdb-sBS5Fz/gdb-7.12.50.20170314/gdb/infcall.c:1372: internal-error:
value* call_function_by_hand_dummy(value*, int, value**, void (*)(void*, int),
void*): ... should not be here
A problem internal to GDB has been detected,
further debugging may prove unreliable.
This is a bug, please report it. For instructions, see:
<http://www.gnu.org/software/gdb/bugs/>.
./debug.sh: line 8: 17083 Aborted (core dumped) gdb -x $1
--args ../test
GDB core stacktrace:
#0 __GI_raise (sig=***@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:58
58 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=***@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:58
#1 0x00007f53de7e337a in __GI_abort () at abort.c:89
#2 0x00005575fc78bf66 in dump_core () at ./gdb/utils.c:465
#3 0x00005575fc78e70a in internal_vproblem(internal_problem *, const char *,
int, const char *, typedef __va_list_tag __va_list_tag *) (
problem=***@entry=0x5575fce031e0 <internal_error_problem>,
file=<optimized out>, line=<optimized out>, fmt=<optimized out>,
ap=***@entry=0x7ffe6df0ae10) at ./gdb/utils.c:676
#4 0x00005575fc78e7db in internal_verror (file=<optimized out>,
line=<optimized out>, fmt=<optimized out>, ap=***@entry=0x7ffe6df0ae10) at
./gdb/utils.c:702
#5 0x00005575fc69ce1f in internal_error (file=***@entry=0x5575fc990e98
"/build/gdb-sBS5Fz/gdb-7.12.50.20170314/gdb/infcall.c", line=***@entry=1372,
fmt=***@entry=0x5575fc96f843 "%s: %s")
at ./gdb/common/errors.c:55
#6 0x00005575fc6d4239 in call_function_by_hand_dummy at ./gdb/infcall.c:1372
#7 0x00005575fc6d457a in call_function_by_hand (......) at ./gdb/infcall.c:677
#8 0x00005575fc6a158e in evaluate_subexp_standard (......) at
./gdb/eval.c:1750
#9 0x00005575fc64161d in evaluate_subexp_c (......) at ./gdb/c-lang.c:713
#10 0x00005575fc69d6dd in evaluate_expression (exp=***@entry=0x5575fd66a100) at
./gdb/eval.c:143
#11 0x00005575fc723793 in print_command_1 (exp=<optimized out>, voidprint=0) at
./gdb/printcmd.c:1258
#12 0x00005575fc58c8f5 in cmd_func (cmd=0x5575fd06e150, args=0x7ffe6df0b525
"bar()", from_tty=0) at ./gdb/cli/cli-decode.c:1888
#13 0x00005575fc7860a3 in execute_command (p=<optimized out>,
from_tty=***@entry=0) at ./gdb/top.c:674
#14 0x00005575fc5cbfa0 in execute_gdb_command (self=<optimized out>,
args=<optimized out>, kw=<optimized out>) at ./gdb/python/python.c:621
#15 0x00007f53df9b9299 in PyCFunction_Call () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#16 0x00007f53dfaea6f9 in PyEval_EvalFrameEx () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#17 0x00007f53dfbac084 in ?? () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#18 0x00007f53dfbac163 in PyEval_EvalCodeEx () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#19 0x00007f53dfa3f528 in ?? () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#20 0x00007f53dfb1b9f7 in PyObject_Call () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#21 0x00007f53dfb6792c in ?? () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#22 0x00007f53dfb1b9f7 in PyObject_Call () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#23 0x00005575fc5b738b in fnpy_call (gdbarch=<optimized out>,
language=<optimized out>, cookie=0x7f53dd3a7048, argc=<optimized out>,
argv=0x7ffe6df0b9d8) at ./gdb/python/py-function.c:82
#24 0x00005575fc6a15b3 in evaluate_subexp_standard
(expect_type=***@entry=0x0, exp=***@entry=0x5575fd6493f0,
pos=<optimized out>, noside=***@entry=EVAL_NORMAL) at ./gdb/eval.c:1746
#25 0x00005575fc64161d in evaluate_subexp_c (expect_type=0x0,
exp=0x5575fd6493f0, pos=0x7ffe6df0bc94, noside=EVAL_NORMAL) at
./gdb/c-lang.c:713
#26 0x00005575fc69d6dd in evaluate_expression (exp=***@entry=0x5575fd6493f0) at
./gdb/eval.c:143
#27 0x00005575fc723793 in print_command_1 (exp=<optimized out>, voidprint=0) at
./gdb/printcmd.c:1258
#28 0x00005575fc58c8f5 in cmd_func (cmd=0x5575fd06e150, args=0x5575fd493c95
"$jitmemctrl(0, 1, 1)", from_tty=0) at ./gdb/cli/cli-decode.c:1888
#29 0x00005575fc7860a3 in execute_command (p=<optimized out>,
from_tty=***@entry=0) at ./gdb/top.c:674
#30 0x00005575fc5cbfa0 in execute_gdb_command (self=<optimized out>,
args=<optimized out>, kw=<optimized out>) at ./gdb/python/python.c:621
#31 0x00007f53df9b9299 in PyCFunction_Call () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
Bug ID: 23191
Summary: GDB crashes when calling a (dummy) user function at
breakpoint in a multi-thread program. Use
scheduler-locking does not help.
Product: gdb
Version: 7.12
Status: UNCONFIRMED
Severity: normal
Priority: P2
Component: python
Assignee: unassigned at sourceware dot org
Reporter: mingwei.zhang at intel dot com
Target Milestone: ---
Created attachment 11022
--> https://sourceware.org/bugzilla/attachment.cgi?id=11022&action=edit
code to reproduce the bug
GDB version: 7.12.50
OS: Ubuntu 17.04
Kernel: 4.10.0-21-generic
python: v3.5
Bug Description
======================================================================
I try to test a simple multi-thread program in C. Before the program launches,
I add a break point using GDB python API in the function that would be called
by all worker threads. Inside the break point, I add some code which would call
a dummy function "bar()" in the target program. To prevent concurrency, I use
"set scheduler-locking on/off" to wrap my function call.
The problem is that at some point, when a new thread is created, the
scheduler-locking is broken and 1st thread entering "bar()" would hang there
and when the 2nd thread gets into "bar()", gdb crashes. After checking gdb
database, this bug seems related with Bug 20291.
I put the code to help reproduce the problem.
Code
======================================================================
C code:
----------------------------------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
struct thread_info { /* Used as argument to thread_start() */
pthread_t thread_id; /* ID returned by pthread_create() */
int thread_num; /* Application-defined thread # */
char *argv_string; /* From command-line argument */
};
/* some function that should report something to gdb */
int bar()
{
/* do nothing */
return 0;
}
void execute_task(int id)
{
printf("worker %d, execute task\n", id);
}
void *worker(void *arg)
{
int idx = 0;
int rounds = 10;
struct thread_info *tinfo = (struct thread_info *)arg;
for (idx = 0; idx < rounds; idx++) {
execute_task(tinfo->thread_num);
}
}
void main()
{
int tnum = 3;
int num_threads = tnum;
void *res;
struct thread_info *tinfo = calloc(num_threads,
sizeof(struct thread_info));
for (tnum = 0; tnum < num_threads; tnum++) {
tinfo[tnum].thread_num = tnum + 1;
pthread_create(&tinfo[tnum].thread_id, NULL,
&worker, &tinfo[tnum]);
}
for (tnum = 0; tnum < num_threads; tnum++) {
pthread_join(tinfo[tnum].thread_id, &res);
}
}
Note: this is a simple C program that create 3 worker thread, each of which is
basically print and done.
gdb python code:
----------------------------------------------------------------------
class IgnoreErrorsCommand (gdb.Command):
"""Execute a single command, ignoring all errors.
Only one-line commands are supported.
This is primarily useful in scripts."""
def __init__ (self):
super (IgnoreErrorsCommand, self).__init__ ("ignore-errors",
gdb.COMMAND_OBSCURE,
# FIXME...
gdb.COMPLETE_COMMAND)
def invoke (self, arg, from_tty):
try:
gdb.execute (arg, from_tty)
except:
pass
IgnoreErrorsCommand ()
class ReturnBreakpoint(gdb.Breakpoint):
def stop (self):
print("come to returnbreakpoint")
tid = gdb.selected_thread().num
print("thread id: %d"%tid)
gdb.execute("call $myfunc(0)")
gdb.write("done calling function\n")
return False
ReturnBreakpoint("*execute_task")
class MyCmd (gdb.Function):
"""JIT Code Memory Permission Controller using Pkey."""
def __init__ (self):
super (MyCmd, self).__init__ ("myfunc")
def invoke (self, arg):
print("++++++++++++++++++command execution begin+++++++++++++++")
tid = gdb.selected_thread().num
try:
gdb.execute("set scheduler-locking on")
gdb.execute("call bar()")
gdb.execute("set scheduler-locking off")
except:
print("*****exception*****")
print("++++++++++++++++++command execution done++++++++++++++++")
return "done"
MyCmd()
gdbinit file:
----------------------------------------------------------------------
set logging on
set pagination off
set target-async 1
set non-stop on
set breakpoint pending on
set print thread-events on
set confirm off
handle 11 nopass
source ./python_code.py
b *0x0
ignore-errors r
del 2
r
gdb launch command:
----------------------------------------------------------------------
gdb -x gdbinit ../test
Result:
======================================================================
GDB response:
Reading symbols from ../test...done.
Breakpoint 1 at 0x80b: file main.c, line 21.
Breakpoint 2 at 0x0
No unwaited-for children left.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff77f1700 (LWP 17090)]
[New Thread 0x7ffff6ff0700 (LWP 17091)]
come to returnbreakpoint
thread id: 2
++++++++++++++++++command execution begin+++++++++++++++
[New Thread 0x7ffff67ef700 (LWP 17092)]
$1 = 0
++++++++++++++++++command execution done++++++++++++++++
$2 = "done"
done calling function
come to returnbreakpoint
thread id: 3
++++++++++++++++++command execution begin+++++++++++++++
worker 1, execute task
come to returnbreakpoint
thread id: 4
++++++++++++++++++command execution begin+++++++++++++++
come to returnbreakpoint
thread id: 2
++++++++++++++++++command execution begin+++++++++++++++
/build/gdb-sBS5Fz/gdb-7.12.50.20170314/gdb/infcall.c:1372: internal-error:
value* call_function_by_hand_dummy(value*, int, value**, void (*)(void*, int),
void*): ... should not be here
A problem internal to GDB has been detected,
further debugging may prove unreliable.
This is a bug, please report it. For instructions, see:
<http://www.gnu.org/software/gdb/bugs/>.
./debug.sh: line 8: 17083 Aborted (core dumped) gdb -x $1
--args ../test
GDB core stacktrace:
#0 __GI_raise (sig=***@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:58
58 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=***@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:58
#1 0x00007f53de7e337a in __GI_abort () at abort.c:89
#2 0x00005575fc78bf66 in dump_core () at ./gdb/utils.c:465
#3 0x00005575fc78e70a in internal_vproblem(internal_problem *, const char *,
int, const char *, typedef __va_list_tag __va_list_tag *) (
problem=***@entry=0x5575fce031e0 <internal_error_problem>,
file=<optimized out>, line=<optimized out>, fmt=<optimized out>,
ap=***@entry=0x7ffe6df0ae10) at ./gdb/utils.c:676
#4 0x00005575fc78e7db in internal_verror (file=<optimized out>,
line=<optimized out>, fmt=<optimized out>, ap=***@entry=0x7ffe6df0ae10) at
./gdb/utils.c:702
#5 0x00005575fc69ce1f in internal_error (file=***@entry=0x5575fc990e98
"/build/gdb-sBS5Fz/gdb-7.12.50.20170314/gdb/infcall.c", line=***@entry=1372,
fmt=***@entry=0x5575fc96f843 "%s: %s")
at ./gdb/common/errors.c:55
#6 0x00005575fc6d4239 in call_function_by_hand_dummy at ./gdb/infcall.c:1372
#7 0x00005575fc6d457a in call_function_by_hand (......) at ./gdb/infcall.c:677
#8 0x00005575fc6a158e in evaluate_subexp_standard (......) at
./gdb/eval.c:1750
#9 0x00005575fc64161d in evaluate_subexp_c (......) at ./gdb/c-lang.c:713
#10 0x00005575fc69d6dd in evaluate_expression (exp=***@entry=0x5575fd66a100) at
./gdb/eval.c:143
#11 0x00005575fc723793 in print_command_1 (exp=<optimized out>, voidprint=0) at
./gdb/printcmd.c:1258
#12 0x00005575fc58c8f5 in cmd_func (cmd=0x5575fd06e150, args=0x7ffe6df0b525
"bar()", from_tty=0) at ./gdb/cli/cli-decode.c:1888
#13 0x00005575fc7860a3 in execute_command (p=<optimized out>,
from_tty=***@entry=0) at ./gdb/top.c:674
#14 0x00005575fc5cbfa0 in execute_gdb_command (self=<optimized out>,
args=<optimized out>, kw=<optimized out>) at ./gdb/python/python.c:621
#15 0x00007f53df9b9299 in PyCFunction_Call () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#16 0x00007f53dfaea6f9 in PyEval_EvalFrameEx () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#17 0x00007f53dfbac084 in ?? () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#18 0x00007f53dfbac163 in PyEval_EvalCodeEx () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#19 0x00007f53dfa3f528 in ?? () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#20 0x00007f53dfb1b9f7 in PyObject_Call () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#21 0x00007f53dfb6792c in ?? () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#22 0x00007f53dfb1b9f7 in PyObject_Call () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#23 0x00005575fc5b738b in fnpy_call (gdbarch=<optimized out>,
language=<optimized out>, cookie=0x7f53dd3a7048, argc=<optimized out>,
argv=0x7ffe6df0b9d8) at ./gdb/python/py-function.c:82
#24 0x00005575fc6a15b3 in evaluate_subexp_standard
(expect_type=***@entry=0x0, exp=***@entry=0x5575fd6493f0,
pos=<optimized out>, noside=***@entry=EVAL_NORMAL) at ./gdb/eval.c:1746
#25 0x00005575fc64161d in evaluate_subexp_c (expect_type=0x0,
exp=0x5575fd6493f0, pos=0x7ffe6df0bc94, noside=EVAL_NORMAL) at
./gdb/c-lang.c:713
#26 0x00005575fc69d6dd in evaluate_expression (exp=***@entry=0x5575fd6493f0) at
./gdb/eval.c:143
#27 0x00005575fc723793 in print_command_1 (exp=<optimized out>, voidprint=0) at
./gdb/printcmd.c:1258
#28 0x00005575fc58c8f5 in cmd_func (cmd=0x5575fd06e150, args=0x5575fd493c95
"$jitmemctrl(0, 1, 1)", from_tty=0) at ./gdb/cli/cli-decode.c:1888
#29 0x00005575fc7860a3 in execute_command (p=<optimized out>,
from_tty=***@entry=0) at ./gdb/top.c:674
#30 0x00005575fc5cbfa0 in execute_gdb_command (self=<optimized out>,
args=<optimized out>, kw=<optimized out>) at ./gdb/python/python.c:621
#31 0x00007f53df9b9299 in PyCFunction_Call () from
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
--
You are receiving this mail because:
You are on the CC list for the bug.
You are receiving this mail because:
You are on the CC list for the bug.