Commit
c6a3f0298ebf0ec1cb1c455320876da5b4a0b07b
by yonghong-songRemove namespace code from libbpf to fix USDT
This removes the namespace code from libbpf, as they are no longer necessary since #2324 was merged, and have caused regressions on older Kernels that do not use the new API for creating probes. This also deletes the dead code for namespace handling in libbpf, as this was the last use of it.
This also introduces regression tests to ensure that processes in containers can be USDT probed, by adding tests that unshare the mount and process namespaces. (commit: c6a3f02) |
| tests/cc/test_usdt_probes.cc (diff) |
| src/cc/libbpf.c (diff) |
Commit
dbfb18851866254a7b127146c8a9c2d76260ee78
by yonghong-songfix a bug in networking/simulation.py
test test_brb.py failed on fc31 with the following error messages: Traceback (most recent call last): File "./test_brb.py", line 162, in test_brb disable_ipv6=True) File "/home/yhs/work/bcc/tests/python/simulation.py", line 94, in _create_ns disable_ipv6) File "/home/yhs/work/bcc/tests/python/simulation.py", line 68, in _ns_add_ifc ns_ipdb.interfaces.lo.up().commit() File "/usr/local/lib/python3.7/site-packages/pyroute2/ipdb/interfaces.py", line 1078, in commit raise error File "/usr/local/lib/python3.7/site-packages/pyroute2/ipdb/interfaces.py", line 859, in commit transaction.wait_all_targets() File "/usr/local/lib/python3.7/site-packages/pyroute2/ipdb/transactional.py", line 507, in wait_all_targets raise CommitException('target %s is not set' % key) pyroute2.ipdb.exceptions.CommitException: target state is not set
The reason is in networking/simulation.py, if the interface 'lo' inside the namespace is already up and it is tried to commit to 'up' state again, the pyroute2 library will cause an exception.
The fix is to avoid to 'up' interface 'lo' again if the interface is already in 'up' state.
Signed-off-by: Yonghong Song <yhs@fb.com> (commit: dbfb188) |
| examples/networking/simulation.py (diff) |
Commit
0cafe5571865d4dcab8ff2bd398b41dbc10b42f4
by yonghong-songfix line TypeError
./profile.py -adf -p `pgrep -n main` 5 Traceback (most recent call last): File "./profile.py", line 342, in <module> print("%s %d" % (b";".join(line).decode('utf-8', 'replace'), v.value)) TypeError: sequence item 5: expected a bytes-like object, str found (commit: 0cafe55) |
| tools/profile.py (diff) |
|
| src/cc/libbpf.c (diff) |
| introspection/bps.c (diff) |
| src/cc/libbpf (diff) |
| src/cc/compat/linux/virtual_bpf.h (diff) |
| docs/kernel-versions.md (diff) |
Commit
a47c44fa0d570b64d8cb06449052db4f363e80a4
by yonghong-songRun BCC test suite with github actions
With this commit, pushes to branches can trigger tests directly on Github Actions.
Here it will be able to test against kernel 4.14 and 5.0.0, which correspond to the latest Ubuntu LTS kernel releases.
Tests are run for both Debug and Release modes.
Tests run inside docker, in an ubuntu 19.04 container base.
For the github workflow:
- The test_libbcc suite is run first, to potentially fail fast on these faster unit tests. - The Python test suite is run
Some of these tests are allowed to fail, but the failure is still reported:
- In catch2 tests, using the [!mayfail] tag, where this will be displayed in the test summary - In Python unittests, using the `@mayFail("Reason")` decorator, which is introduce for consistency with catch2. These will report the reason for the failure, and log this to an artifact of the test. (commit: a47c44f) |
| tests/cc/test_usdt_probes.cc (diff) |
| examples/networking/simulation.py (diff) |
| tests/python/test_brb2.py (diff) |
| tests/python/test_usdt3.py (diff) |
| .gitignore (diff) |
| tests/python/utils.py (diff) |
| tests/python/test_debuginfo.py (diff) |
| tests/python/test_tools_smoke.py (diff) |
| tests/python/test_usdt.py (diff) |
| tests/wrapper.sh.in (diff) |
| .github/workflows/bcc-test.yml |
| tests/cc/test_c_api.cc (diff) |
| tests/python/test_stackid.py (diff) |
| tests/python/test_trace3.py (diff) |
| Dockerfile.tests |
| tests/python/test_usdt2.py (diff) |
|
| src/cc/api/BPF.h (diff) |
| tests/python/test_usdt.py (diff) |
| tests/python/test_usdt2.py (diff) |
| src/cc/usdt.h (diff) |
| src/cc/api/BPF.cc (diff) |
| tests/python/test_usdt3.py (diff) |
|
| README.md (diff) |
Commit
90b2382b2068031930704d42205829b356f41ffb
by yonghong-songtools/klockstat.py: Do not display symbol twice for stack
Currently we display the caller symbol in stack, which ends up with output below when we enable stack:
Caller Avg Hold Count Max hold Total hold b'flush_to_ldisc+0x22' 56112 2 103914 112225 b'flush_to_ldisc+0x22' b'process_one_work+0x1b0' b'worker_thread+0x50' b'kthread+0xfb'
Skipping one more symbol in stack to fix that:
Caller Avg Hold Count Max hold Total hold b'flush_to_ldisc+0x22' 1893 2 2765 3787 b'process_one_work+0x1b0' b'worker_thread+0x50' b'kthread+0xfb'
Signed-off-by: Jiri Olsa <jolsa@kernel.org> (commit: 90b2382) |
| tools/klockstat.py (diff) |
Commit
0e63c5c1ad267281f8d0b652eaf87dd494ddba04
by yonghong-songfix compilation error due to latest llvm change
llvm 11.0 required explicit conversion from StringRef to std::string. The patch is https://github.com/llvm/llvm-project/commit/777180a32b61070a10dd330b4f038bf24e916af1
This patch made a compatible change which works for old llvm as well.
Signed-off-by: Yonghong Song <yhs@fb.com> (commit: 0e63c5c) |
| src/cc/frontends/clang/b_frontend_action.cc (diff) |
|
| src/cc/api/BPFTable.cc (diff) |
| docs/reference_guide.md (diff) |
| src/cc/api/BPFTable.h (diff) |
| src/cc/frontends/clang/b_frontend_action.cc (diff) |
| src/cc/export/helpers.h (diff) |
| src/python/bcc/table.py (diff) |
| src/cc/api/BPF.cc (diff) |
| src/cc/api/BPF.h (diff) |
Commit
f727a00fb4a388f30ab0f637b43f27345b0f6da7
by yonghong-songbcc.BPF.cleanup(): Ensure self.funcs items get deleted during cleanup
Since commit 115b959d86 ("Fix a file descriptor leak when module is deleted (#2530)"), we observe the following exceptions during python exit:
Error in atexit._run_exitfuncs: Traceback (most recent call last): File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 1366, in cleanup os.close(fn.fd) OSError: [Errno 9] Bad file descriptor
which occurs for python programs issuing a call to 'cleanup()', or using the 'with bcc.BPF(...)' code pattern.
BPF's 'cleanup' is registered to be invoked atexit. Alas, commit 115b959d86 introduced an 'os.close(fn.fd)' call for each func loaded (to prevent accidental FD leakage).
Problem is, that the 'self.funcs' dict entries are NOT deleted, making subsequent calls to 'cleanup' to attempt closing the same 'fn.fd' again and again.
It is expected from 'cleanup' to operate correctly when called repeatedly; Therefore, it should "nullify" references to closed resources.
Fix, by deleting the reference to each unloaded function from the 'self.func' dictionary.
Fixes: 115b959d86 ("Fix a file descriptor leak when module is deleted (#2530)") Reported-by: Dana Rubin <drubin@metanetworks.com> Signed-off-by: Shmulik Ladkani <shmulik.ladkani@gmail.com> (commit: f727a00) |
| src/python/bcc/__init__.py (diff) |
|
| src/cc/export/helpers.h (diff) |
Commit
685ec23ecaa9c55409a21b5cd3c919fe1206fb20
by yonghong-songsnap: update and cleanup snapcraft.yaml
A couple of issues fixed:
- put snapcraft.yaml in the snap directory so it is picked up by snapcraft when run from the project root. - use snap/local for local snapcraft assets. - setup.py.in should not add root for sdist targets - cleanup snap/README.md. - re-arrange parts in snapcraft.yaml and move to python3.
Signed-off-by: Sergio Schvezov <sergio.schvezov@canonical.com> (commit: 685ec23) |
| snapcraft/snapcraft.yaml |
| snap/local/bcc-wrapper |
| snap/README.md |
| snap/snapcraft.yaml |
| snapcraft/bcc-wrapper |
| src/python/setup.py.in (diff) |
| snapcraft/Makefile |
| snapcraft/README.md |
Commit
a4834a6cacd14f251e72695f08766a16216642b2
by yonghong-songprevent rewriting for array element type
Fix issue #2352. Corresponding llvm bug: https://bugs.llvm.org/show_bug.cgi?id=41918.
If the element type is array type, the rewriter may generate code like addr = ({ typeof(__u8 [16]) _val; __builtin_memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), (u64)&daddr->s6_addr); _val; }) for something like addr = daddr->s6_addr; where s6_addr is an array.
The above code has an issue then as addr is pointing to some data which is out of scope, which meaning compiler is free to use the space.
Let us disable such transformation until we find a good solution.
Signed-off-by: Yonghong Song <yhs@fb.com> (commit: a4834a6) |
| src/cc/frontends/clang/b_frontend_action.cc (diff) |
| tools/tcptop.py (diff) |
Commit
f149ca5d774aa81d8b78cfcd5be6f5cf58cbdab7
by yonghong-songDo not rewrite array subscripts if invalid sourceloc range
Fix #2739.
The issue exposed an issue that to rewrite CODE1: __u8 byte = daddr->s6_addr[4]; will segfault and to rewrite CODE2: __u8 byte = (daddr->s6_addr)[4]; will be okay.
For CODE1, the clang did not give enough information to find the text which contains the left bracket "[", given base "daddr->s6_addr" and subscript "4". For CODE2, the clang is able to get the information successfuly.
I think if we really go inside the base "daddr->s6_addr" and gets to its member field "s6_addr", we can find the needed information for the text range containing "[". Let us fix the segfault first and if really desirable, we can try to enhance later for CODE1 patterns.
Signed-off-by: Yonghong Song <yhs@fb.com> (commit: f149ca5) |
| src/cc/frontends/clang/b_frontend_action.cc (diff) |
|
| tools/offwaketime.py (diff) |
Commit
6cacc41462bae12b1c38aabc7a6b85f40783da9a
by yonghong-songsupport sockmap/sockhash/cgroup_local_storage maps
This patch supports sockmap, sockhash and cgroup_local_storage maps. Two C++ APIs, attach_func and detach_func are also added to allow to attach program to cgroups. So this makes using C++ APIs for cgroup based networking applications easier to integrate with bpf programs.
The unit testing is rough as it needs some work to set up cgroups and establish TCP connections to really test the result of map operations. But still all supported map operations in kernel and in C++ APIs are tested syntacically.
Signed-off-by: Yonghong Song <yhs@fb.com> (commit: 6cacc41) |
| src/cc/bpf_module.h (diff) |
| src/cc/api/BPF.cc (diff) |
| src/cc/api/BPFTable.h (diff) |
| tests/cc/test_cg_storage.cc |
| src/cc/api/BPF.h (diff) |
| src/cc/frontends/clang/b_frontend_action.cc (diff) |
| src/cc/bpf_module.cc (diff) |
| src/cc/export/helpers.h (diff) |
| src/cc/api/BPFTable.cc (diff) |
| tests/cc/CMakeLists.txt (diff) |
| tests/cc/test_sock_table.cc |
|
| debian/control (diff) |
|
| INSTALL.md (diff) |
|
| snap/snapcraft.yaml (diff) |
|
| libbpf-tools/bin/bpftool |
|
| libbpf-tools/vmlinux_505.h |
| libbpf-tools/vmlinux.h |
|
| CMakeLists.txt (diff) |
Commit
e1496e15a749fa700315f759dfca4ac79f3f6428
by andriintools: add libbpf-based tools to BCC w/ runqslower as first converted tool
Add runqslower as a first tool converted from BCC to libbpf, utilizing BPF CO-RE (Compile Once - Run Everywhere) approach and BPF skeleton for interfacing with BPF programs. Current set up is Makefile based and is set up in such a way as to enable easy addition for more tools, based on a simple and convenient naming pattern. General build infrastructure takes case of BPF skeleton generation, tracking dependencies, clean up, etc.
Signed-off-by: Andrii Nakryiko <andriin@fb.com> (commit: e1496e1) |
| libbpf-tools/Makefile |
| libbpf-tools/runqslower.h |
| libbpf-tools/.gitignore |
| man/man8/runqslower.8 (diff) |
| libbpf-tools/runqslower.c |
| libbpf-tools/runqslower_example.txt |
| libbpf-tools/runqslower.bpf.c |
| libbpf-tools/README.md |
Commit
a28ad059edf6f74acc124fde4b4934a2fc410a35
by yonghong-songfix pid filter bug
1. Wirte a simple go, just print pid and get some website. func main() { pid := os.Getpid() fmt.Println(pid) response, err := http.Get("http://www.baidu.com") .... }
./main 3581
2. But when run `tcpconnlat`, we just go tid 3585, (not pid 3581) #./tcpconnlat PID COMM IP SADDR DADDR DPORT LAT(ms) 3585 main 4 10.0.2.15 180.101.49.11 80 60.68
3. So run `./tcpconnlat -p 3581` not work I have tested this situation under kernel 3.10 and 5.0.9. (commit: a28ad05) |
| tools/tcpconnlat.py (diff) |
Commit
9b6a0e357c30b52e115f803bb6babb3ffc4fdabf
by yonghong-songtools/stackcount: Fix address resolution for user-space stack
This patch fixes issue #2748. The bug was that address-to-symbol resolution didn't work for user-space stacks without the `-P` (per-pid) flag when tracing single, isolated processes. The current documentation of the `-P` option indicates that it's used to "display stacks separately for each process", and this doesn't match with the intended usage.
This patch has two parts:
- Fix `tools/stackcount.py` to explicitly set perpid to True if `-p <pid>` is used - Remove the `-P` option from the example of tracing single, isolated process in `tools/stackcount_example.txt`, since the usage of the option can be confusing (and unnecessary after the current change) (commit: 9b6a0e3) |
| tools/stackcount_example.txt (diff) |
| tools/stackcount.py (diff) |
|
| examples/tracing/dddos.py (diff) |
|
| tests/python/test_tools_smoke.py (diff) |
| README.md (diff) |
| tools/bindsnoop.py |
| tools/bindsnoop_example.txt |
| man/man8/bindsnoop.8 |
|
| src/cc/libbpf (diff) |
|
| tools/profile.py (diff) |
|
| tools/execsnoop.py (diff) |
Commit
942227484d3207f6a42103674001ef01fb5335a0
by yonghong-songdebian changelog for v0.13.0 tag
* Support for kernel up to 5.5 * bindsnoop tool to track tcp/udp bind information * added compile-once run-everywhere based libbpf-tools, currently only runqslower is implemented. * new map support: sockhash, sockmap, sk_storage, cgroup_storage * enable to run github actions on the diff * cgroupmap based cgroup filtering for opensnoop, execsnoop and bindsnoop. * lots of bug fixes.
Signed-off-by: Yonghong Song <yhs@fb.com> (commit: 9422274) |
| debian/changelog (diff) |
|
| tools/tcptracer.py (diff) |
| docs/filtering_by_cgroups.md (diff) |
| tools/tcptracer_example.txt (diff) |
| tools/tcpconnect.py (diff) |
| tools/tcpconnect_example.txt (diff) |
| tools/tcpaccept_example.txt (diff) |
| tools/tcpaccept.py (diff) |
|
| man/man8/tcpconnect.8 (diff) |
| man/man8/execsnoop.8 (diff) |
| man/man8/tcptracer.8 (diff) |
| man/man8/tcpaccept.8 (diff) |
| man/man8/opensnoop.8 (diff) |
|
| src/cc/libbpf.c (diff) |
| src/cc/libbpf.h (diff) |
| src/cc/api/BPF.h (diff) |
| src/cc/api/BPF.cc (diff) |
|
| examples/cpp/RandomRead.cc (diff) |
Commit
7ad17638acd9c1d82264451c9b0800ecf5ce4f59
by andriinlibbpf-tools: add links to BPF CO-RE posts
Add links to BPF CO-RE blog posts, explainint what it is, how to use it, and giving practical recommendations on how to convert BCC BPF program to libbpf-based one.
Signed-off-by: Andrii Nakryiko <andriin@fb.com> (commit: 7ad1763) |
| libbpf-tools/README.md (diff) |
|
| man/man8/execsnoop.8 (diff) |
| tools/execsnoop_example.txt (diff) |
| tools/execsnoop.py (diff) |
|
| tools/oomkill.py (diff) |
|
| examples/networking/http_filter/http-parse-complete.py (diff) |
|
| docs/reference_guide.md (diff) |
|
| tests/cc/CMakeLists.txt (diff) |
Commit
1e7862fdfd8b428e6b2e3ea23f63a1469c3fabf3
by yonghong-songUnite libbpf includes
Currently we include headers from local libbpf subpackage, which does not work if user specify LIBBPF_INCLUDE_DIR.
Adding HAVE_EXTERNAL_LIBBPF macro, that gets defined when user specifies libbpf header path. This macro is checked in bcc_libbpf_inc.h and proper files are included.
Using bcc_libbpf_inc.h in place of libbpf includes.
Signed-off-by: Jiri Olsa <jolsa@kernel.org> (commit: 1e7862f) |
| src/cc/bpf_module.cc (diff) |
| src/cc/bcc_libbpf_inc.h |
| src/cc/bcc_btf.cc (diff) |
| src/cc/CMakeLists.txt (diff) |
| src/cc/frontends/clang/b_frontend_action.cc (diff) |
| src/cc/libbpf.c (diff) |
Commit
550706a220da1636fafd322f0366cfde2a188d16
by yonghong-songDo not initialize kern_version for TRACING/EXT programs
The TRACING/EXT programs use attach_btf_id and attach_prog_fd fields from struct bpf_load_program_attr.
The attach_prog_fd field shares space with kern_version, so by setting kern_version unconditionally we also set attach_prog_fd to bogus value and kernel fails the load because it tries to look it up.
Setting kern_version only for programs other than TRACING/EXT type.
Signed-off-by: Jiri Olsa <jolsa@kernel.org> (commit: 550706a) |
| src/cc/bpf_module.cc (diff) |
| src/cc/libbpf.c (diff) |
Commit
572478b2083651f25c4e278f8c34ff3d16e3be6a
by yonghong-songIntroduce {attach|detach}_kfunc API
Kernel added new probe called trampoline allowing to probe almost any kernel function when BTF info is available in the system.
Adding the interface to define trampoline function for given kernel function via BPF_PROG macro, like:
KFUNC_PROBE(do_sys_open, int dfd, const char *filename, int flags, int mode) { ... }
which defines trampoline function with the 'kfunc__do_sys_open' name, that instruments do_sys_open kernel function before the function is executed.
or:
KRETFUNC_PROBE(do_sys_open, int dfd, const char *filename, int flags, int mode, int ret) { ... }
which defines trampoline function with the 'kfunc__do_sys_open' name, that instruments do_sys_open kernel function after the function is executed.
The main benefit is really lower overhead for trampolines (please see following commit for klockstat.py with perf comparison).
Another benefit is the ability of kretfunc probe to access function arguments, so some tools might need only one program instead of entry/exit ones (please see following commit for opensnoop.py changes).
Currently the interface does not allow to define function of different name than: kfunc__<function_name> or kretfunc__<function_name>
which is sufficient by now, and can be easily changed in future if needed.
Signed-off-by: Jiri Olsa <jolsa@kernel.org> (commit: 572478b) |
| src/cc/export/helpers.h (diff) |
| src/cc/libbpf.c (diff) |
| src/cc/libbpf.h (diff) |
| src/python/bcc/__init__.py (diff) |
| docs/reference_guide.md (diff) |
|
| src/python/bcc/__init__.py (diff) |
| src/cc/libbpf.h (diff) |
| src/cc/libbpf.c (diff) |
Commit
c347fe6c9f75d6740cf45f80312009daba275f76
by yonghong-songSupport kfunc in opensnoop.py
Adding kfunc return trampoline probe if available instead of kprobe/kretprobe probes.
The return trampoline has access to function entry arguments, so we are good with just single eBPF program.
The kfunc trampolines are also faster - less intrusive.
Below are stats for compiling linux kernel while running opensnoop.py on the background for kprobes and kfuncs.
Without opensnoop.py:
Performance counter stats for 'system wide':
849,741,782,765 cycles:k
162.235646336 seconds time elapsed
With opensnoop.py using kprobes:
Performance counter stats for 'system wide':
941,615,199,769 cycles:k
164.355032879 seconds time elapsed
With opensnoop.py using trampolines:
Performance counter stats for 'system wide':
913,437,005,488 cycles:k
163.742914795 seconds time elapsed
Signed-off-by: Jiri Olsa <jolsa@kernel.org> (commit: c347fe6) |
| tools/opensnoop.py (diff) |
Commit
da7cac733c931f06801cc3ce118140892fcb8066
by yonghong-songSupport kfunc in klockstat.py
Adding kfunc return trampoline probe if available instead of kprobe/kretprobe probes.
The kfunc trampolines are faster - less intrusive. The benchmark without klockstat.py script on background:
$ perf bench sched messaging -l 50000 # Running 'sched/messaging' benchmark: # 20 sender and receiver processes per group # 10 groups == 400 processes run
Total time: 18.571 [sec]
With kprobe tracing: $ perf bench sched messaging -l 50000 # Running 'sched/messaging' benchmark: # 20 sender and receiver processes per group # 10 groups == 400 processes run
Total time: 183.395 [sec]
With kfunc tracing: $ perf bench sched messaging -l 50000 # Running 'sched/messaging' benchmark: # 20 sender and receiver processes per group # 10 groups == 400 processes run
Total time: 39.773 [sec]
Signed-off-by: Jiri Olsa <jolsa@kernel.org> (commit: da7cac7) |
| tools/klockstat.py (diff) |
Commit
2fa54c0bd388898fdda58f30dcfe5a68d6715efc
by yonghong-songSupport kfunc in vfsstat.py
Adding kfunc return trampoline probe if available instead of kprobe/kretprobe probes.
The kfunc trampolines are faster - less intrusive.
Below are stats for running perf bench sched pipe benchamark while running vfsstat.py on the background for kprobes and kfuncs.
With kprobes: Performance counter stats for './perf bench sched pipe -l 5000000' (3 runs):
112,520,853,574 cycles:k
48.674 +- 0.672 seconds time elapsed ( +- 1.38% )
With kfuncs: Performance counter stats for './perf bench sched pipe -l 5000000' (3 runs):
106,304,165,424 cycles:k
46.820 +- 0.197 seconds time elapsed ( +- 0.42% )
Signed-off-by: Jiri Olsa <jolsa@kernel.org> (commit: 2fa54c0) |
| tools/vfsstat.py (diff) |
|
| src/cc/bcc_exception.h (diff) |
|
| INSTALL.md (diff) |
|
| src/python/bcc/usdt.py (diff) |
|
| libbpf-tools/bin/bpftool (diff) |
|
| man/man8/drsnoop.8 (diff) |
| tools/drsnoop.py (diff) |
| man/man8/compactsnoop.8 (diff) |
| libbpf-tools/drsnoop_example.txt |
| libbpf-tools/trace_helpers.c |
| libbpf-tools/Makefile (diff) |
| libbpf-tools/drsnoop.bpf.c |
| libbpf-tools/drsnoop.c |
| libbpf-tools/drsnoop.h |
| libbpf-tools/trace_helpers.h |
| snap/snapcraft.yaml (diff) |
|
| src/python/bcc/usdt.py (diff) |
Commit
5011f992b3167c162b5a0ad57fd3fa23c9ea76b3
by yonghong-songusdt: fix parsing sp register in arguments on AArch64
One of the USDT probes for OpenJDK on AArch64 has an argument as an offset from the stack pointer register like "8@[sp, 112]". This causes the argument parser to fail:
Parse error: 8@x22 8@x20 8@x23 8@x0 8@x26 8@x27 8@[sp, 112] 8@[sp, 120] ------------------------------------------^
The error message then repeats forever.
Changed ArgumentParser_aarch64::parse_register so it accepts either "xNN" or "sp" and outputs the register name rather than the register number. The stack pointer is in a separate field `sp` in `struct pt_regs` rather than in the `regs[]` array [1].
Note that the parser currently accepts "x31" and converts that into a reference to `regs[31]' but that array only has 31 elements. Made x31 an alias for `sp` to avoid undefined behaviour from reading past the end of the array.
[1]: https://elixir.bootlin.com/linux/latest/source/arch/arm64/include/asm/ptrace.h#L160
Change-Id: I88b6ff741914b5d06ad5798a55bd21ea03f69825 Signed-off-by: Nick Gasson <nick.gasson@arm.com> (commit: 5011f99) |
| src/cc/usdt/usdt_args.cc (diff) |
| tests/cc/test_usdt_args.cc (diff) |
| src/cc/usdt.h (diff) |
|
| src/cc/api/BPF.cc (diff) |
| src/cc/api/BPFTable.cc (diff) |
| src/cc/frontends/b/codegen_llvm.cc (diff) |
| src/cc/frontends/b/printer.cc (diff) |
| src/cc/bpf_module_rw_engine.cc (diff) |
| src/cc/api/BPFTable.h (diff) |
| src/cc/frontends/b/type_check.cc (diff) |
|
| src/python/bcc/table.py (diff) |
Commit
cf20a499655583a985eb959f98fae9c2e7ecf189
by yonghong-songGitHub Actions: optionally publish container image
This is optional: this GitHub Action will only attempt to publish the container image if the GitHub repository has been configured with GitHub secrets (e.g. https://github.com/iovisor/bcc/settings/secrets).
The GitHub secrets can be configured as follows: - DOCKER_PUBLISH = 1 - DOCKER_IMAGE = docker.io/myorg/bcc - DOCKER_USERNAME = username - DOCKER_PASSWORD = password
This is intended to make it easy for anyone to fork iovisor/bcc on GitHub and publish custom container images with bcc. (commit: cf20a49) |
| .github/workflows/bcc-test.yml (diff) |
| Dockerfile.ubuntu (diff) |
Commit
b7541d0b9497e5bca4de28d894d18cc8578ccdc4
by yonghong-songlibbpf-tools: optimize ksyms cache
Re-implement internals of ksyms cache to minimize memory overhead and allocations. Addr is also changed to be unsigned long.
fscanf() can be further optimized into manual parsing, but it's low enough overhead right now that I felt like readibility is more important.
Benchmarking ksyms loading/parsing parts:
BEFORE: $ /usr/bin/time ./test 0.03user 0.04system 0:00.08elapsed 98%CPU (0avgtext+0avgdata 6968maxresident)k 0inputs+0outputs (0major+1512minor)pagefaults 0swaps
AFTER: $ /usr/bin/time ./test 0.02user 0.03system 0:00.06elapsed 100%CPU (0avgtext+0avgdata 9508maxresident)k 0inputs+0outputs (0major+2106minor)pagefaults 0swaps
RSS goes down from 9.5MB to <7MB, while CPU time went up about 20ms.
Signed-off-by: Andrii Nakryiko <andriin@fb.com> (commit: b7541d0) |
| libbpf-tools/Makefile (diff) |
| libbpf-tools/trace_helpers.c (diff) |
| libbpf-tools/.gitignore (diff) |
| libbpf-tools/trace_helpers.h (diff) |
|
| tests/python/test_usdt3.py (diff) |
| src/python/bcc/libbcc.py (diff) |
| src/python/bcc/usdt.py (diff) |
|
| src/cc/bcc_usdt.h (diff) |
| tools/trace.py (diff) |
| src/python/bcc/usdt.py (diff) |
| src/cc/usdt/usdt.cc (diff) |
| src/python/bcc/libbcc.py (diff) |
Commit
9465f8cf2a19d28dcc8bce5845b5e8017ee9feb8
by edwardwusoftirqs: Combined CPU as part of the key is necessary to avoid amiss value.
In the environment of massive software interrupts. <idle>-0 [003] ..s1 106.421020: softirq_entry: vec=6 [action=TASKLET] <idle>-0 [000] ..s1 106.421063: softirq_entry: vec=3 [action=NET_RX] <idle>-0 [003] ..s1 106.421083: softirq_exit: vec=6 [action=TASKLET]
Follow the above ftrace logs, we know the correct vec-6 start timestamp is replaced with incorrect vec-3. Because PID is idle-0. It will produce the wrong result after calculating delta. (commit: 9465f8c) |
| tools/softirqs.py (diff) |
|
| tools/capable.py (diff) |
| tools/capable_example.txt (diff) |
| man/man8/capable.8 (diff) |
Commit
9d7feeed87c5e2977e0e996cc097dcd9d3001c0d
by yonghong-songtools: add option --unique to capable.py
Some processes can do a lot of security capability checks, generating a lot of ouput. In this case, the --unique option is useful to only print once the same set of capability, pid (or cgroup if --cgroupmap is used) and kernel/user stacks (if -K or -U are used).
# ./capable.py -K -U --unique
Documentation (man page and example text) updated. (commit: 9d7feee) |
| man/man8/capable.8 (diff) |
| tools/capable.py (diff) |
| tools/capable_example.txt (diff) |
|
| Dockerfile.ubuntu (diff) |
|
| libbpf-tools/opensnoop.bpf.c |
| libbpf-tools/opensnoop.h |
| libbpf-tools/opensnoop.c |
| libbpf-tools/Makefile (diff) |
|
| tools/tcptop_example.txt (diff) |
| man/man8/tcptop.8 (diff) |
| tools/tcptop.py (diff) |
Commit
8b6a7db9c8eeb5c513a162c90a7dd8f0bd7620dd
by yonghong-songlibbpf-tools: update vmlinux.h with BPF helper flags generated from UAPI
Update vmlinux.h to a version generated from same v5.5 tag and default config with cherry-picked 1aae4bdd7879 ("bpf: Switch BPF UAPI #define constants used from BPF program side to enums") on top of it. This adds lots of BPF helper flags often useful from BPF program side.
Signed-off-by: Andrii Nakryiko <andriin@fb.com> (commit: 8b6a7db) |
| libbpf-tools/vmlinux_505.h (diff) |
|
| src/cc/libbpf (diff) |
Commit
454b138e6b75a47d4070a4f99c8f2372b383f71e
by yonghong-songlibbpf-tools: small clean ups across all tools
Remove BPF_F_CURRENT_CPU definitions, which are now provided by vmlinux.h after 1aae4bdd7879 ("bpf: Switch BPF UAPI #define constants used from BPF program side to enums") commit in kernel.
Fix potential uninitialized read warning in opensnoop.
Also add opensnoop to .gitignore.
Signed-off-by: Andrii Nakryiko <andriin@fb.com> (commit: 454b138) |
| libbpf-tools/drsnoop.bpf.c (diff) |
| libbpf-tools/.gitignore (diff) |
| libbpf-tools/opensnoop.bpf.c (diff) |
| libbpf-tools/runqslower.bpf.c (diff) |
| libbpf-tools/opensnoop.c (diff) |
Commit
2b5fcc6ec332f8067dab50da833ee870fd690579
by yonghong-songlibbpf-tools: determine target host architecture
BPF code is compiled with -target bpf, but for PT_REGS_PARM macro (and by induction for BPF_KPROBE/BPF_KRETPROBE macros as well), it's important to know what's the target host original architecture was, to use correct definition of struct pt_regs. Determine that based on output of `uname -m` (taking into account that both x86_64 and x86 are defined as x86 internally for kernel).
Signed-off-by: Andrii Nakryiko <andriin@fb.com> (commit: 2b5fcc6) |
| libbpf-tools/Makefile (diff) |
|
| tools/tcpstates.py (diff) |
Commit
cd43be4c6a52540a8e272ed3dcea48211c198073
by yonghong-songlibbpf-tools: adjust Kconfig and re-build vmlinux.h
Default v5.5 kernel config doesn't have most of BPF-related functionality enabled, which leads to vmlinux.h not containing a lot of useful constants. This patch contains re-generated vmlinux.h from kernel built with default config plus minimal changes to enable most (all?) BPF-relevant parts of kernel. Here's a list of added options:
CONFIG_BPF_EVENTS=y CONFIG_BPFILTER_UMH=m CONFIG_BPFILTER=y CONFIG_BPF_JIT=y CONFIG_BPF_KPROBE_OVERRIDE=y CONFIG_BPF_STREAM_PARSER=y CONFIG_BPF_SYSCALL=y CONFIG_CC_HAS_ASM_INLINE=y CONFIG_CC_HAS_KASAN_GENERIC=y CONFIG_CC_HAS_SANCOV_TRACE_PC=y CONFIG_CGROUP_BPF=y CONFIG_GCC_VERSION=70300 CONFIG_IPV6_MULTIPLE_TABLES=y CONFIG_IPV6_SEG6_BPF=y CONFIG_IPV6_SEG6_LWTUNNEL=y CONFIG_LIBCRC32C=y CONFIG_LWTUNNEL_BPF=y CONFIG_LWTUNNEL=y CONFIG_NET_ACT_BPF=y CONFIG_NET_CLS_BPF=y CONFIG_NETFILTER_ADVANCED=y CONFIG_NETFILTER_XT_MATCH_BPF=y CONFIG_NET_SOCK_MSG=y CONFIG_NF_CT_PROTO_DCCP=y CONFIG_NF_CT_PROTO_SCTP=y CONFIG_NF_CT_PROTO_UDPLITE=y CONFIG_SOCK_CGROUP_DATA=y CONFIG_STREAM_PARSER=y CONFIG_XDP_SOCKETS_DIAG=y CONFIG_XDP_SOCKETS=y
To make this vmlinux.h generation process easier for future adjustments (e.g., if some of the tools would need types that default config compiles out), check in used Kconfig along the vmlinux.h itself.
Signed-off-by: Andrii Nakryiko <andriin@fb.com> (commit: cd43be4) |
| libbpf-tools/vmlinux_505.h (diff) |
| libbpf-tools/kernel.config |
Commit
7fd2fa68b52e0f38dd4915bf786d67445941eedd
by yonghong-songFactor out ebpf::parse_tracepoint function
And moving it to common.cc in order to be able to make automated tests for it. Following patches are adding automated test for this function and it seems too much to link in all the clang/llvm stuff to the test binary just for single function test.
Adding ebpf::parse_tracepoint that takes istream of the tracepoint format data and returns tracepoint struct as std::string.
No functional change is intended or expected.
Signed-off-by: Jiri Olsa <jolsa@kernel.org> (commit: 7fd2fa6) |
| src/cc/frontends/clang/tp_frontend_action.cc (diff) |
| src/cc/common.h (diff) |
| src/cc/common.cc (diff) |
|
| tests/cc/CMakeLists.txt (diff) |
| tests/cc/test_parse_tracepoint.cc |
Commit
b8423e66ce450418c6142aab61ee9330e1fcefcc
by yonghong-songStrengthen tracepoint format parsing
There's issue in current RHEL real time kernel with tracepoint format, which makes bcc-tools to return wrong data.
Two new 'common_' fields were added and it causes 2 issues for tracepoint format parser.
First issue - is the gap between common fields and other fields, which is not picked up by the parser, so the resulted struct is not aligned with the data.
Second issue - is the fact that current parser covers common fields with: u64 __do_not_use__ so the new common fields are not accounted for.
This issue is solved in the following patch. I kept both issues and fixes separated to make the change readable.
There's a 'not described gap' in the sched_wakeup's format file and probably in other formats as well:
Having: # cat /sys/kernel/debug/tracing/events/sched/sched_wakeup/format name: sched_wakeup ID: 310 format: field:unsigned short common_type; offset:0; size:2; signed:0; field:unsigned char common_flags; offset:2; size:1; signed:0; field:unsigned char common_preempt_count; offset:3; size:1; signed:0; field:int common_pid; offset:4; size:4; signed:1; field:unsigned char common_migrate_disable; offset:8; size:1; signed:0; field:unsigned char common_preempt_lazy_count; offset:9; size:1; signed:0;
field:char comm[16]; offset:12; size:16; signed:1; field:pid_t pid; offset:28; size:4; signed:1; field:int prio; offset:32; size:4; signed:1; field:int success; offset:36; size:4; signed:1; field:int target_cpu; offset:40; size:4; signed:1;
There's "common_preempt_lazy_count" field on offset 9 with size 1: common_preempt_lazy_count; offset:9; size:1;
and it's followed by "comm" field on offset 12: field:char comm[16]; offset:12; size:16; signed:1;
which makes 2 bytes gap in between, that might confuse some applications like bpftrace or bcc-tools library.
The tracepoint parser makes struct out of the field descriptions, but does not account for such gaps.
I posted patch to fix this [1] in RT kernel, but that might take a while, and we could easily fix our tracepoint parser to workaround this issue.
Adding code to detect this gaps and add 1 byte __pad_X fields, where X is the offset number.
[1] https://lore.kernel.org/linux-rt-users/20200221153541.681468-1-jolsa@kernel.org/ Signed-off-by: Jiri Olsa <jolsa@kernel.org> (commit: b8423e6) |
| tests/cc/test_parse_tracepoint.cc (diff) |
| src/cc/common.cc (diff) |
Commit
0a9d6db96c6410d2ef741507255f5258d6406032
by yonghong-songAdd support for new common fields
Current parser covers common fields with: u64 __do_not_use__ so the new common fields are not accounted for.
Keeping the 'u64 __do_not_use__' field for backward compatibility (who knows who's actualy using it) and adding new fields, like: char __do_not_use__X
for each byte of extra common fields, where X is the offset of the field.
With this fix the bcc-tools correctly parses tracepoints on RT kernel and it's usable again.
Signed-off-by: Jiri Olsa <jolsa@kernel.org> (commit: 0a9d6db) |
| src/cc/common.cc (diff) |
| tests/cc/test_parse_tracepoint.cc (diff) |
|
| src/cc/libbpf (diff) |
|
| examples/networking/net_monitor.py |
|
| CMakeLists.txt (diff) |
| INSTALL.md (diff) |
|
| tools/profile_example.txt (diff) |
| man/man8/profile.8 (diff) |
| tools/profile.py (diff) |
|
| src/cc/bcc_syms.cc (diff) |
|
| tests/cc/test_usdt_probes.cc (diff) |
|
| libbpf-tools/xfsslower.c |
| libbpf-tools/Makefile (diff) |
| libbpf-tools/.gitignore (diff) |
| libbpf-tools/xfsslower.h |
| libbpf-tools/xfsslower.bpf.c |
Commit
510fc7425ccd0bc53d53a56ae847d786eb15e1ce
by yonghong-songopensnoop: fix --cgroupmap with kfunc
Commit c347fe6c9f75 ("Support kfunc in opensnoop.py") introduces an alternative probe on do_sys_open() with kfuncs instead of kprobes. This new implementation is used if the kernel supports it. But it removed the --cgroupmap filter added in commit b2aa29fa3269 ("tools: cgroup filtering in execsnoop/opensnoop").
This patch adds the --cgroupmap filter in the kfunc implementation. (commit: 510fc74) |
| tools/opensnoop.py (diff) |
|
| INSTALL.md (diff) |
Commit
6b8a89673dbba9b9203a6555735e451dd9477836
by yonghong-songtools: fix alignment of ipv6_key_t in tcptop
Fixes the following error on aarch64:
bpf: Failed to load program: Permission denied ; struct sock *sk = ctx->regs[0]; int copied = ctx->regs[1]; 0: (79) r8 = *(u64 *)(r1 +8) ... ; struct ipv6_key_t ipv6_key = {.pid = pid}; 79: (63) *(u32 *)(r10 -48) = r7 ; struct ipv6_key_t ipv6_key = {.pid = pid}; 80: (7b) *(u64 *)(r10 +8) = r9 invalid stack off=8 size=8 processed 96 insns (limit 1000000) max_states_per_insn 0 total_states 7 peak_states 7 mark_read 4 (commit: 6b8a896) |
| tools/tcptop.py (diff) |
|
| tools/cachetop.py (diff) |
|
| tools/capable.py (diff) |
|
| libbpf-tools/filelife.bpf.c |
| libbpf-tools/filelife.c |
| libbpf-tools/filelife.h |
| libbpf-tools/.gitignore (diff) |
| libbpf-tools/Makefile (diff) |
|
| introspection/CMakeLists.txt (diff) |
|
| examples/networking/http_filter/http-parse-complete.py (diff) |
|
| examples/tracing/mallocstacks.py (diff) |
|
| examples/networking/http_filter/http-parse-simple.py (diff) |
|
| libbpf-tools/opensnoop.bpf.c (diff) |
| libbpf-tools/opensnoop.c (diff) |
| libbpf-tools/opensnoop.h (diff) |
|
| tests/cc/test_usdt_probes.cc (diff) |
| src/cc/api/BPF.cc (diff) |
| src/cc/api/BPF.h (diff) |
| examples/cpp/FollyRequestContextSwitch.cc (diff) |
Commit
495a4a3403371b3fb1f7049a8cefc996621e61aa
by yonghong-songexamples/tracing: add nflatency - netfilter hook metrics
This adds a kprobe / kretprobe for chasing down slow netfilter hooks. Since, often, the slowest hooks are on connection initialization and teardown, it is bucketed by TCP flags. (rather than dive deep in to conntrack)
Signed-off-by: Casey Callendrello <cdc@redhat.com> (commit: 495a4a3) |
| examples/tracing/nflatency.py |
Commit
533391eaa94c4b3a148bf801b8aea5bb803c1e9d
by yonghong-songadd test_lpm_trie.py test
test_lpm_trie.py has been in the repo for quite some time, but is not included in the unit test.
The issue https://github.com/iovisor/bcc/issues/2860 exposed an issue involved in using together with BTF, which requires the key type to be a structure.
Let add it as a unit test so we can be sure lpm_trie map is supported properly by bcc.
Signed-off-by: Yonghong Song <yhs@fb.com> (commit: 533391e) |
| tests/python/CMakeLists.txt (diff) |
| tests/python/test_lpm_trie.py (diff) |
Commit
f35dae07bbe48f6233bf50f05ddbf835cc18c86e
by yonghong-songadjust layout string in JIT with llvm11 128bit spec support
To fix the issue (https://github.com/iovisor/bcc/issues/2827) which exposed a problem with aarch64 frontend and bpf backend regarding __int128 type, the following llvm patch https://reviews.llvm.org/D76587 landed to explicitly support i128 type in bpf layout spec.
Adjust the layout string in bpf_module JIT compilation accordingly.
Signed-off-by: Yonghong Song <yhs@fb.com> (commit: f35dae0) |
| src/cc/bpf_module.cc (diff) |
Commit
007ee4930798f854da47f5838097193d75c67ad1
by yonghong-songTranslate virtual addresses to binary addresses for shared libs as well.
Quoting https://refspecs.linuxbase.org/elf/elf.pdf: ``` Symbol table entries for different object file types have slightly different interpretations for the st_value member. - In relocatable files, st_value holds alignment constraints for a symbol whose section index is SHN_COMMON. - In relocatable files, st_value holds a section offset for a defined symbol. That is, st_value is an offset from the beginning of the section that st_shndx identifies. - In executable and shared object files, st_value holds a virtual address. To make these files' symbols more useful for the dynamic linker, the section offset (file interpretation) gives way to a virtual address (memory interpretation) for which the section number is irrelevant. ```
This is a problem in practice as well. I run into this while tracing shared libraries on Android with bpftrace. Some of them have text sections mmapped at different offset than file offset which results in probes being placed at wrong offsets. (commit: 007ee49) |
| tests/python/CMakeLists.txt (diff) |
| src/cc/bcc_syms.cc (diff) |
| tests/python/test_uprobes2.py |
|
| examples/tracing/io_latencies.py |
|
| tools/filelife.py (diff) |
|
| docs/reference_guide.md (diff) |
|
| examples/tracing/io_latencies.py |
| tools/biolatpcts.py |
|
| tools/biolatpcts.py (diff) |
Commit
7214903a9c42c3501e50a47d3698b77547036f70
by tjtools/biolatpcts: --pcts handling improvements
* Multiple arguments parsing conflicts with positional argument and can become confusing. Use a single comma separated list instead.
* Instead of converting to floats while parsing arguments, keep the strings verbatim and use them when outputting results. This allows matching the same target percentile string in the output and is helpful when it's consumed by other programs. (commit: 7214903) |
| tools/biolatpcts.py (diff) |
|
| tools/biolatpcts.py (diff) |
|
| man/man8/biolatpcts.8 |
| tools/biolatpcts_example.txt |
|
| examples/tracing/biolatpcts_example.txt |
| examples/tracing/biolatpcts.py |
|
| src/cc/api/BPF.h (diff) |
| tests/cc/test_usdt_probes.cc (diff) |
Commit
1599c2ef8206988d5df7eeadc3c5138c006ac245
by yonghong-songbuild: fix clang 10 build
some distros already packaging clang 10 (checked fedora and arch) no longer ship all the individual libclang*.so component libraries. Instead, clang from 9.0 onwards provides a new lib, libclang-cpp.so, which includes everything we need.
Tell cmake to use it if the individual libraries are no longer found. (Build-wise, if both are present it is more efficient to use the individual components so keep these first) (commit: 1599c2e) |
| CMakeLists.txt (diff) |
|
| libbpf-tools/README.md (diff) |
|
| examples/cpp/HelloWorld.cc (diff) |
| src/cc/CMakeLists.txt (diff) |
| examples/cpp/CMakeLists.txt (diff) |
| src/cc/bcc_version.h.in |
|
| INSTALL.md (diff) |
Commit
45e63f2b316cdce2d8cc925f6f14a8726ade9ff6
by yonghong-songfix llvm 11 compilation issues
The llvm CreateCall used in bcc is deprecated in llvm 11: https://reviews.llvm.org/D76269 The llvm CreateMemCpy is changed in llvm 11 as well: https://reviews.llvm.org/D71473
This caused bcc compilation error.
/home/yhs/work/bcc/src/cc/frontends/b/codegen_llvm.cc: In member function ‘ebpf::StatusTuple ebpf::cc::CodegenLLVM::emit_log(ebpf::cc::Method CallExprNode*)’: /home/yhs/work/bcc/src/cc/frontends/b/codegen_llvm.cc:691:39: error: no matching function for call to ‘llvm::IRBuilder<>::CreateCall(llvm::Value*&, std::vector<llvm::Value*, std::allocator<llvm::Value*> >&)’ expr_ = B.CreateCall(printk_fn, args); ^ ...
/home/yhs/work/bcc/src/cc/frontends/b/codegen_llvm.cc: In member function ‘virtual ebpf::StatusTuple ebpf::cc::CodegenLLVM::visit_string_exp_node(ebpf::cc::StringExprNode*)’: /home/yhs/work/bcc/src/cc/frontends/b/codegen_llvm.cc:440:55: error: no matching function for call to ‘llvm::IRBuilder<>::CreateMemCpy(llvm:Value*&, int, llvm::Value*&, int, std::__cxx11::basic_string<char>::size_type)’ B.CreateMemCpy(ptr, 1, global, 1, n->val_.size() + 1); ^ ...
This patch fixed the compilation issue. (commit: 45e63f2) |
| src/cc/frontends/b/codegen_llvm.h (diff) |
| src/cc/frontends/b/codegen_llvm.cc (diff) |
|
| introspection/bps.c (diff) |
| src/cc/libbpf (diff) |
| src/cc/compat/linux/virtual_bpf.h (diff) |
| src/cc/export/helpers.h (diff) |
| docs/kernel-versions.md (diff) |
| src/cc/libbpf.c (diff) |
|
| debian/changelog (diff) |