{ "_class" : "hudson.matrix.MatrixBuild", "actions" : [ { "_class" : "hudson.model.CauseAction", "causes" : [ { "_class" : "org.jenkinsci.plugins.ghprb.GhprbCause", "shortDescription" : "GitHub pull request #2710 of commit 4615a23c7ec488c83d9880f1cc2c48afa6a8193e, no merge conflicts." } ] }, { "_class" : "org.jenkinsci.plugins.ghprb.GhprbParametersAction", "parameters" : [ { "_class" : "hudson.model.StringParameterValue", "name" : "sha1", "value" : "origin/pr/2710/merge" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbActualCommit", "value" : "4615a23c7ec488c83d9880f1cc2c48afa6a8193e" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbActualCommitAuthor", "value" : "Dale Hamel" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbActualCommitAuthorEmail", "value" : "dale.hamel@shopify.com" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbAuthorRepoGitUrl", "value" : "https://github.com/dalehamel/bcc.git" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbTriggerAuthor", "value" : "" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbTriggerAuthorEmail", "value" : "" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbTriggerAuthorLogin", "value" : "yonghong-song" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbTriggerAuthorLoginMention", "value" : "@yonghong-song" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbPullId", "value" : "2710" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbTargetBranch", "value" : "master" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbSourceBranch", "value" : "legacy-event-ns-fix" }, { "_class" : "hudson.model.StringParameterValue", "name" : "GIT_BRANCH", "value" : "legacy-event-ns-fix" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbPullAuthorEmail", "value" : "" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbPullAuthorLogin", "value" : "dalehamel" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbPullAuthorLoginMention", "value" : "@dalehamel" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbPullDescription", "value" : "GitHub pull request #2710 of commit 4615a23c7ec488c83d9880f1cc2c48afa6a8193e, no merge conflicts." }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbPullTitle", "value" : "Don't enter target process mount NS for uprobes" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbPullLink", "value" : "https://github.com/iovisor/bcc/pull/2710" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbPullLongDescription", "value" : "On the legacy path, for kernels that don't support creating uprobes via\\r\\nthe perf_event_open (kernels prior to e12f03d), the path to the probe is\\r\\nalready absolute, including /proc/PID/root prefix. Unless the target process\\r\\nis running within the host PID namespace, this path will be invalid.\\r\\n\\r\\nThis should fix #2558 for older kernels\\r\\n\\r\\n# Description of issue\\r\\n\\r\\nOlder kernels (in my case, 4.14 on Chromium-based OS) use this branch to initialize probes:\\r\\n\\r\\nhttps://github.com/iovisor/bcc/blob/ae5a169a0e9e47b57b7e3a8cc2c514e7ad4f3813/src/cc/libbpf.c#L1023-L1028\\r\\n\\r\\nRather than the newer (default) approach:\\r\\n\\r\\nhttps://github.com/iovisor/bcc/blob/ae5a169a0e9e47b57b7e3a8cc2c514e7ad4f3813/src/cc/libbpf.c#L1020-L1021\\r\\n\\r\\nThe path specified by `config1` was modified in https://github.com/iovisor/bcc/pull/2324 to prepend `/proc/PID/root`, to make sure the path is relative to the target PID's mount namespace. \\r\\n\\r\\nHowever, in this legacy code path, there is already a call to enter the target namespace:\\r\\n\\r\\nhttps://github.com/iovisor/bcc/blob/071eef6b1571f23be6f0fc03787523cb06b70659/src/cc/libbpf.c#L990\\r\\n\\r\\nWhen the code tries to initialize the probe by writing to the kernel file descriptor:\\r\\n\\r\\n\\r\\nhttps://github.com/iovisor/bcc/blob/071eef6b1571f23be6f0fc03787523cb06b70659/src/cc/libbpf.c#L993-L995\\r\\n\\r\\nThe writes on the failing host look like:\\r\\n\\r\\n```\\r\\n...\\r\\nwrite(13, \\\"p:uprobes/p__proc_3978339_root_usr_local_bin_memcached_7a66_1_bcc_296056 /proc/3978339/root/usr/local/bin/memcached:0x7a66\\\", 122) = -1 ENOENT (No such file or directory)\\r\\nwrite(2, \\\"cannot attach uprobe, probe entry may not exist\\n\\\", 48cannot attach uprobe, probe entry may not exist\\r\\n...\\r\\n```\\r\\n\\r\\nI reverted BCC to https://github.com/iovisor/bcc/commit/06e30cca8342aed6a1cd83de65d17945499ab6bb and was able to get it to succeed, and the write looks like:\\r\\n\\r\\n```\\r\\n...\\r\\nwrite(16, \\\"p:uprobes/p__usr_local_bin_memcached_7a66_1_bcc_293830 /usr/local/bin/memcached:0x7a66\\\", 86) = 86\\r\\n...\\r\\n```\\r\\n\\r\\nBut the commit from #2324 (https://github.com/iovisor/bcc/commit/01ee0b61a854e06cb62272ddfbb4eeaf16e9ad7c#diff-918ef818c481c73f92c4ce607b1ec386R678) changes this to prepend the namespace.\\r\\n\\r\\n# Summary of change\\r\\n\\r\\nThe writes can still succeed if the target process is in the host mount namespace, but if the target process is containerized it is very likely not, and the path that is given will be invalid in its mount namespace (it will appear to see pids starting from lower numbers, and not even know its own system PID).\\r\\n\\r\\nThese are the cases where this code fails. The solution is to simply not enter the namespace, as thanks to #2324 we don't need to. This makes the path relative to the host PID namespace.\\r\\n\\r\\n# Alternatives\\r\\n\\r\\nAn alternative would be to strip the prepended string. This may actually be desirable, as the tracing process (in this case bpftrace) may not be running in the host mount namespace. This would allow for the path resolution to always be accurate, and I'm beginning to convince myself it would be a better approach, with something like this pseudo-ish code:\\r\\n\\r\\n```c\\r\\n\\r\\nchar* probe_path = config1;\\r\\nif (strncmp(config1, \\\"/proc\\\", 5)\\r\\n{\\r\\n char* path_component = strtok(probe_path, \\\"/\\\"); // /proc/PID/root/PATH\\r\\n path_component = strtok(NULL, \\\"/\\\"); // /PID/root/PATH\\r\\n path_component = strtok(NULL, \\\"/\\\"); // /root/PATH\\r\\n\\r\\n if(strncmp(config1, \\\"/root\\\")\\r\\n {\\r\\n path_component = strtok(NULL, \\\"/\\\");// /PATH\\r\\n probe_path = path_component;\\r\\n }\\r\\n}\\r\\n \\r\\n\\r\\n```\\r\\n\\r\\n\\r\\n\\r\\n" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbCommentBody", "value" : "null" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbGhRepository", "value" : "iovisor/bcc" }, { "_class" : "hudson.model.StringParameterValue", "name" : "ghprbCredentialsId", "value" : "6d3daf13-69b8-48b1-9c8f-ec5353264113" } ] }, { "_class" : "hudson.plugins.git.GitTagAction" }, { }, { }, { "_class" : "org.jenkinsci.plugins.displayurlapi.actions.RunDisplayAction" } ], "artifacts" : [ ], "building" : False, "description" : "PR #2710: Don't enter target process ...", "displayName" : "#252", "duration" : 1093794, "estimatedDuration" : 7460664, "executor" : None, "fullDisplayName" : "bcc-pr #252", "id" : "252", "keepLog" : False, "number" : 252, "queueId" : 2031, "result" : "FAILURE", "timestamp" : 1579544297958, "url" : "https://buildbot.iovisor.org/jenkins/job/bcc-pr/252/", "builtOn" : "", "changeSet" : { "_class" : "hudson.plugins.git.GitChangeSetList", "items" : [ ], "kind" : "git" }, "culprits" : [ ], "runs" : [ { "number" : 252, "url" : "https://buildbot.iovisor.org/jenkins/job/bcc-pr/label=fc25/252/" }, { "number" : 252, "url" : "https://buildbot.iovisor.org/jenkins/job/bcc-pr/label=fc26/252/" }, { "number" : 252, "url" : "https://buildbot.iovisor.org/jenkins/job/bcc-pr/label=fc27/252/" }, { "number" : 252, "url" : "https://buildbot.iovisor.org/jenkins/job/bcc-pr/label=fc28/252/" }, { "number" : 252, "url" : "https://buildbot.iovisor.org/jenkins/job/bcc-pr/label=ubuntu1604/252/" }, { "number" : 252, "url" : "https://buildbot.iovisor.org/jenkins/job/bcc-pr/label=ubuntu1710/252/" }, { "number" : 252, "url" : "https://buildbot.iovisor.org/jenkins/job/bcc-pr/label=ubuntu1804/252/" } ] }