From 56b03f3c4d641dbdbce2e52a2969712e85b0e030 Mon Sep 17 00:00:00 2001 From: Arnaldo Carvalho de Melo Date: Tue, 5 Jan 2010 16:50:31 -0200 Subject: perf tools: Handle relocatable kernels MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DSOs don't have this problem because the kernel emits a PERF_MMAP for each new executable mapping it performs on monitored threads. To fix the kernel case we simulate the same behaviour, by having 'perf record' to synthesize a PERF_MMAP for the kernel, encoded like this: [root@doppio ~]# perf record -a -f sleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.344 MB perf.data (~15038 samples) ] [root@doppio ~]# perf report -D | head -10 0xd0 [0x40]: event: 1 . . ... raw event: size 64 bytes . 0000: 01 00 00 00 00 00 40 00 00 00 00 00 00 00 00 00 ......@........ . 0010: 00 00 00 81 ff ff ff ff 00 00 00 00 00 00 00 00 ............... . 0020: 00 00 00 00 00 00 00 00 5b 6b 65 72 6e 65 6c 2e ........ [kernel . 0030: 6b 61 6c 6c 73 79 6d 73 2e 5f 74 65 78 74 5d 00 kallsyms._text] . 0xd0 [0x40]: PERF_RECORD_MMAP 0/0: [0xffffffff81000000((nil)) @ (nil)]: [kernel.kallsyms._text] I.e. we identify such event as having: .pid = 0 .filename = [kernel.kallsyms.REFNAME] .start = REFNAME addr in /proc/kallsyms at 'perf record' time and use now a hardcoded value of '.text' for REFNAME. Then, later, in 'perf report', if there are any kernel hits and thus we need to resolve kernel symbols, we search for REFNAME and if its address changed, relocation happened and we thus must change the kernel mapping routines to one that uses .pgoff as the relocation to apply. This way we use the same mechanism used for the other DSOs and don't have to do a two pass in all the kernel symbols. Reported-by: Xiao Guangrong Signed-off-by: Arnaldo Carvalho de Melo Cc: Frédéric Weisbecker Cc: "H. Peter Anvin" Cc: Mike Galbraith Cc: Peter Zijlstra Cc: Paul Mackerras Cc: Xiao Guangrong LKML-Reference: <1262717431-1246-1-git-send-email-acme@infradead.org> Signed-off-by: Ingo Molnar --- tools/perf/builtin-record.c | 7 +++++++ 1 file changed, 7 insertions(+) (limited to 'tools/perf/builtin-record.c') diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index 265425322734..8f88420e066b 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -551,6 +551,13 @@ static int __cmd_record(int argc, const char **argv) return err; } + err = event__synthesize_kernel_mmap(process_synthesized_event, + session, "_text"); + if (err < 0) { + pr_err("Couldn't record kernel reference relocation symbol.\n"); + return err; + } + if (!system_wide && profile_cpu == -1) event__synthesize_thread(pid, process_synthesized_event, session); -- cgit v1.2.3 From b7cece76783c68fb391f9882235b4b0c9c300c46 Mon Sep 17 00:00:00 2001 From: Arnaldo Carvalho de Melo Date: Wed, 13 Jan 2010 13:22:17 -0200 Subject: perf tools: Encode kernel module mappings in perf.data MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We were always looking at the running machine /proc/modules, even when processing a perf.data file, which only makes sense when we're doing 'perf record' and 'perf report' on the same machine, and in close sucession, or if we don't use modules at all, right Peter? ;-) Now, at 'perf record' time we read /proc/modules, find the long path for modules, and put them as PERF_MMAP events, just like we did to encode the reloc reference symbol for vmlinux. Talking about that now it is encoded in .pgoff, so that we can use .{start,len} to store the address boundaries for the kernel so that when we reconstruct the kmaps tree we can do lookups right away, without having to fixup the end of the kernel maps like we did in the past (and now only in perf record). One more step in the 'perf archive' direction when we'll finally be able to collect data in one machine and analyse in another. Signed-off-by: Arnaldo Carvalho de Melo Cc: Frédéric Weisbecker Cc: Mike Galbraith Cc: Peter Zijlstra Cc: Paul Mackerras LKML-Reference: <1263396139-4798-1-git-send-email-acme@infradead.org> Signed-off-by: Ingo Molnar --- tools/perf/builtin-record.c | 11 +++++++++++ 1 file changed, 11 insertions(+) (limited to 'tools/perf/builtin-record.c') diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index 8f88420e066b..c130df2676f1 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -465,6 +465,11 @@ static int __cmd_record(int argc, const char **argv) return -1; } + if (perf_session__create_kernel_maps(session) < 0) { + pr_err("Problems creating kernel maps\n"); + return -1; + } + if (!file_new) { err = perf_header__read(&session->header, output); if (err < 0) @@ -558,6 +563,12 @@ static int __cmd_record(int argc, const char **argv) return err; } + err = event__synthesize_modules(process_synthesized_event, session); + if (err < 0) { + pr_err("Couldn't record kernel reference relocation symbol.\n"); + return err; + } + if (!system_wide && profile_cpu == -1) event__synthesize_thread(pid, process_synthesized_event, session); -- cgit v1.2.3 From 18c3daa4961b9fa1f2db0711d93c0acf0c39fd12 Mon Sep 17 00:00:00 2001 From: Arnaldo Carvalho de Melo Date: Thu, 14 Jan 2010 23:45:28 -0200 Subject: perf record: Encode the domain while synthesizing MMAP events MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In the past 'perf record' had to process only userspace MMAP events, the ones generated in the kernel, but after we reused the MMAP events to encode the module mapings we ended up adding them first to the list of userspace DSOs (dsos__user) and to the kernel one (dsos__kernel). Fix this by encoding the header.misc field and then using it, like other parts to decide the right DSOs list to insert/find. The gotcha here is that since the kernel puts zero in .misc, which isn't PERF_RECORD_MISC_KERNEL (1 << 1), to differentiate, we put 1 in .misc. Signed-off-by: Arnaldo Carvalho de Melo Cc: Frédéric Weisbecker Cc: Mike Galbraith Cc: Peter Zijlstra Cc: Paul Mackerras LKML-Reference: <1263519930-22803-2-git-send-email-acme@infradead.org> Signed-off-by: Ingo Molnar --- tools/perf/builtin-record.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) (limited to 'tools/perf/builtin-record.c') diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index c130df2676f1..614fa9a4c67c 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -117,8 +117,12 @@ static void write_event(event_t *buf, size_t size) * Add it to the list of DSOs, so that when we finish this * record session we can pick the available build-ids. */ - if (buf->header.type == PERF_RECORD_MMAP) - dsos__findnew(buf->mmap.filename); + if (buf->header.type == PERF_RECORD_MMAP) { + struct list_head *head = &dsos__user; + if (buf->mmap.header.misc == 1) + head = &dsos__kernel; + __dsos__findnew(head, buf->mmap.filename); + } write_output(buf, size); } -- cgit v1.2.3 From f5a2c3dce03621b55f84496f58adc2d1a87ca16f Mon Sep 17 00:00:00 2001 From: Arnaldo Carvalho de Melo Date: Fri, 15 Jan 2010 18:08:26 -0200 Subject: perf record: Intercept all events MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The event interception we need to do in 'perf record' to create a list of all DSOs in PERF_RECORD_MMAP events wasn't seeing all events, make sure that happens by checking size agains event_t->header.size. Signed-off-by: Arnaldo Carvalho de Melo Cc: Frédéric Weisbecker Cc: Mike Galbraith Cc: Peter Zijlstra Cc: Paul Mackerras LKML-Reference: <1263586107-1756-1-git-send-email-acme@infradead.org> Signed-off-by: Ingo Molnar --- tools/perf/builtin-record.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) (limited to 'tools/perf/builtin-record.c') diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index 614fa9a4c67c..7bb9ca1b30fa 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -113,16 +113,24 @@ static void write_output(void *buf, size_t size) static void write_event(event_t *buf, size_t size) { - /* - * Add it to the list of DSOs, so that when we finish this - * record session we can pick the available build-ids. - */ - if (buf->header.type == PERF_RECORD_MMAP) { - struct list_head *head = &dsos__user; - if (buf->mmap.header.misc == 1) - head = &dsos__kernel; - __dsos__findnew(head, buf->mmap.filename); - } + size_t processed_size = buf->header.size; + event_t *ev = buf; + + do { + /* + * Add it to the list of DSOs, so that when we finish this + * record session we can pick the available build-ids. + */ + if (ev->header.type == PERF_RECORD_MMAP) { + struct list_head *head = &dsos__user; + if (ev->header.misc == 1) + head = &dsos__kernel; + __dsos__findnew(head, ev->mmap.filename); + } + + ev = ((void *)ev) + ev->header.size; + processed_size += ev->header.size; + } while (processed_size < size); write_output(buf, size); } -- cgit v1.2.3 From 64abebf731df87e6f4ae7d9ffc340bdf0c033e44 Mon Sep 17 00:00:00 2001 From: Arnaldo Carvalho de Melo Date: Wed, 27 Jan 2010 21:05:52 -0200 Subject: perf session: Create kernel maps in the constructor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Removing one extra step needed in the tools that need this, fixing a bug in 'perf probe' where this was not being done. Signed-off-by: Arnaldo Carvalho de Melo Cc: Masami Hiramatsu Cc: Frédéric Weisbecker Cc: Mike Galbraith Cc: Peter Zijlstra Cc: Paul Mackerras LKML-Reference: <1264633557-17597-4-git-send-email-acme@infradead.org> Signed-off-by: Ingo Molnar --- tools/perf/builtin-record.c | 5 ----- 1 file changed, 5 deletions(-) (limited to 'tools/perf/builtin-record.c') diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index 7bb9ca1b30fa..90345223908c 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -477,11 +477,6 @@ static int __cmd_record(int argc, const char **argv) return -1; } - if (perf_session__create_kernel_maps(session) < 0) { - pr_err("Problems creating kernel maps\n"); - return -1; - } - if (!file_new) { err = perf_header__read(&session->header, output); if (err < 0) -- cgit v1.2.3 From a8e6f734ce9a79d44ebb296f2a341f435227b34e Mon Sep 17 00:00:00 2001 From: Hitoshi Mitake Date: Sat, 30 Jan 2010 20:55:41 +0900 Subject: Revert "perf record: Intercept all events" This reverts commit f5a2c3dce03621b55f84496f58adc2d1a87ca16f. This patch is required for making "perf lock rec" work. The commit f5a2c3dce0 changes write_event() of builtin-record.c . And changed write_event() sometimes doesn't stop with perf lock rec. Cc: Peter Zijlstra Cc: Mike Galbraith Cc: Paul Mackerras Cc: Arnaldo Carvalho de Melo Cc: Frederic Weisbecker LKML-Reference: [ that commit also causes perf record to not be Ctrl-C-able, and it's concetually wrong to parse the data at record time (unconditionally - even when not needed), as we eventually want to be able to do zero-copy recording, at least for non-archive recordings. ] Signed-off-by: Ingo Molnar --- tools/perf/builtin-record.c | 28 ++++++++++------------------ 1 file changed, 10 insertions(+), 18 deletions(-) (limited to 'tools/perf/builtin-record.c') diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index 90345223908c..eea56910b91c 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -113,24 +113,16 @@ static void write_output(void *buf, size_t size) static void write_event(event_t *buf, size_t size) { - size_t processed_size = buf->header.size; - event_t *ev = buf; - - do { - /* - * Add it to the list of DSOs, so that when we finish this - * record session we can pick the available build-ids. - */ - if (ev->header.type == PERF_RECORD_MMAP) { - struct list_head *head = &dsos__user; - if (ev->header.misc == 1) - head = &dsos__kernel; - __dsos__findnew(head, ev->mmap.filename); - } - - ev = ((void *)ev) + ev->header.size; - processed_size += ev->header.size; - } while (processed_size < size); + /* + * Add it to the list of DSOs, so that when we finish this + * record session we can pick the available build-ids. + */ + if (buf->header.type == PERF_RECORD_MMAP) { + struct list_head *head = &dsos__user; + if (buf->mmap.header.misc == 1) + head = &dsos__kernel; + __dsos__findnew(head, buf->mmap.filename); + } write_output(buf, size); } -- cgit v1.2.3 From b8f46c5a34fa64fd456295388d18f50ae69d9f37 Mon Sep 17 00:00:00 2001 From: Xiao Guangrong Date: Wed, 3 Feb 2010 11:53:14 +0800 Subject: perf tools: Use O_LARGEFILE to open perf data file Open perf data file with O_LARGEFILE flag since its size is easily larger that 2G. For example: # rm -rf perf.data # ./perf kmem record sleep 300 [ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 3142.147 MB perf.data (~137282513 samples) ] # ll -h perf.data -rw------- 1 root root 3.1G ..... Signed-off-by: Xiao Guangrong Cc: Frederic Weisbecker Cc: Steven Rostedt Cc: Paul Mackerras Cc: Peter Zijlstra LKML-Reference: <4B68F32A.9040203@cn.fujitsu.com> Signed-off-by: Ingo Molnar --- tools/perf/builtin-record.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) (limited to 'tools/perf/builtin-record.c') diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index eea56910b91c..949167efa1ed 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -5,6 +5,9 @@ * (or a CPU, or a PID) into the perf.data output file - for * later analysis via perf report. */ +#define _LARGEFILE64_SOURCE +#define _FILE_OFFSET_BITS 64 + #include "builtin.h" #include "perf.h" @@ -451,7 +454,7 @@ static int __cmd_record(int argc, const char **argv) append_file = 0; } - flags = O_CREAT|O_RDWR; + flags = O_CREAT|O_RDWR|O_LARGEFILE; if (append_file) file_new = 0; else -- cgit v1.2.3 From 6122e4e4f5d0913e319ef8a4dc60a47afe4abc0a Mon Sep 17 00:00:00 2001 From: Arnaldo Carvalho de Melo Date: Wed, 3 Feb 2010 16:52:05 -0200 Subject: perf record: Stop intercepting events, use postprocessing to get build-ids MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We want to stream events as fast as possible to perf.data, and also in the future we want to have splice working, when no interception will be possible. Using build_id__mark_dso_hit_ops to create the list of DSOs that back MMAPs we also optimize disk usage in the build-id cache by only caching DSOs that had hits. Suggested-by: Peter Zijlstra Signed-off-by: Arnaldo Carvalho de Melo Cc: Xiao Guangrong Cc: Frédéric Weisbecker Cc: Mike Galbraith Cc: Paul Mackerras LKML-Reference: <1265223128-11786-6-git-send-email-acme@infradead.org> Signed-off-by: Ingo Molnar --- tools/perf/builtin-record.c | 37 ++++++++++++++++++------------------- 1 file changed, 18 insertions(+), 19 deletions(-) (limited to 'tools/perf/builtin-record.c') diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index 949167efa1ed..706f00196b87 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -12,6 +12,7 @@ #include "perf.h" +#include "util/build-id.h" #include "util/util.h" #include "util/parse-options.h" #include "util/parse-events.h" @@ -65,6 +66,7 @@ static int nr_poll = 0; static int nr_cpu = 0; static int file_new = 1; +static off_t post_processing_offset; static struct perf_session *session; @@ -114,26 +116,10 @@ static void write_output(void *buf, size_t size) } } -static void write_event(event_t *buf, size_t size) -{ - /* - * Add it to the list of DSOs, so that when we finish this - * record session we can pick the available build-ids. - */ - if (buf->header.type == PERF_RECORD_MMAP) { - struct list_head *head = &dsos__user; - if (buf->mmap.header.misc == 1) - head = &dsos__kernel; - __dsos__findnew(head, buf->mmap.filename); - } - - write_output(buf, size); -} - static int process_synthesized_event(event_t *event, struct perf_session *self __used) { - write_event(event, event->header.size); + write_output(event, event->header.size); return 0; } @@ -185,14 +171,14 @@ static void mmap_read(struct mmap_data *md) size = md->mask + 1 - (old & md->mask); old += size; - write_event(buf, size); + write_output(buf, size); } buf = &data[old & md->mask]; size = head - old; old += size; - write_event(buf, size); + write_output(buf, size); md->prev = old; mmap_write_tail(md, old); @@ -402,10 +388,21 @@ static void open_counters(int cpu, pid_t pid) nr_cpu++; } +static int process_buildids(void) +{ + u64 size = lseek(output, 0, SEEK_CUR); + + session->fd = output; + return __perf_session__process_events(session, post_processing_offset, + size - post_processing_offset, + size, &build_id__mark_dso_hit_ops); +} + static void atexit_header(void) { session->header.data_size += bytes_written; + process_buildids(); perf_header__write(&session->header, output, true); } @@ -558,6 +555,8 @@ static int __cmd_record(int argc, const char **argv) return err; } + post_processing_offset = lseek(output, 0, SEEK_CUR); + err = event__synthesize_kernel_mmap(process_synthesized_event, session, "_text"); if (err < 0) { -- cgit v1.2.3 From f887f3019e56389a73617f4e70f512e82cc89adb Mon Sep 17 00:00:00 2001 From: Xiao Guangrong Date: Thu, 4 Feb 2010 16:46:42 +0800 Subject: perf tools: Clean up O_LARGEFILE et al usage Setting _FILE_OFFSET_BITS and using O_LARGEFILE, lseek64, etc, is redundant. Thanks H. Peter Anvin for pointing it out. So, this patch removes O_LARGEFILE, lseek64, etc. Suggested-by: "H. Peter Anvin" Signed-off-by: Xiao Guangrong Cc: Frederic Weisbecker Cc: Steven Rostedt Cc: Paul Mackerras Cc: Peter Zijlstra LKML-Reference: <4B6A8972.3070605@cn.fujitsu.com> Signed-off-by: Ingo Molnar --- tools/perf/builtin-record.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'tools/perf/builtin-record.c') diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index 706f00196b87..3ad599b12c91 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -5,7 +5,6 @@ * (or a CPU, or a PID) into the perf.data output file - for * later analysis via perf report. */ -#define _LARGEFILE64_SOURCE #define _FILE_OFFSET_BITS 64 #include "builtin.h" @@ -451,7 +450,7 @@ static int __cmd_record(int argc, const char **argv) append_file = 0; } - flags = O_CREAT|O_RDWR|O_LARGEFILE; + flags = O_CREAT|O_RDWR; if (append_file) file_new = 0; else -- cgit v1.2.3 From f7e7ee36757f68778700cde1aaed89e1d23e59fd Mon Sep 17 00:00:00 2001 From: "austin_zhang@linux.intel.com" Date: Fri, 5 Feb 2010 09:02:42 -0800 Subject: perf record: Fix existing process callgraph symbol When 'perf record -g' a existing process, even with debuginfo packages, still cannnot get symbol from 'perf report'. try: perf record -g -p `pidof xxx` -f perf report 68.26% :1181 b74870f2 [.] 0x000000b74870f2 | |--32.09%-- 0xb73b5b44 | 0xb7487102 | 0xb748a4e2 | 0xb748633d | 0xb73b41cd | 0xb73b4467 | 0xb747d531 The reason is: for existing process, in __cmd_record(), the pid is 0 rather than the existing process id. Signed-off-by: Austin Zhang Acked-by: Peter Zijlstra Cc: Mike Galbraith Cc: Paul Mackerras Cc: Arnaldo Carvalho de Melo Cc: Frederic Weisbecker LKML-Reference: <4710.10.255.24.35.1265389362.squirrel@linux.intel.com> Signed-off-by: Ingo Molnar --- tools/perf/builtin-record.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'tools/perf/builtin-record.c') diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index 3ad599b12c91..771533ced6a8 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -570,7 +570,7 @@ static int __cmd_record(int argc, const char **argv) } if (!system_wide && profile_cpu == -1) - event__synthesize_thread(pid, process_synthesized_event, + event__synthesize_thread(target_pid, process_synthesized_event, session); else event__synthesize_threads(process_synthesized_event, session); -- cgit v1.2.3