Features

This tool helps kernel and operating developers optimize system suspend and resume time. Using a kernel image built with power management, ftrace, and with kprobes enabled (which have no impact on performance when not in use), the tool runs a suspend. During this time, it captures dmesg and ftrace data from suspend start to resume completion. This data is transformed into a timeline or call graph. It gives a quick and detailed view of which devices and kernel processes take the most time in suspend and resume.
 

The following image shows the basic timeline for the S3 suspend-to-mem power mode. In Linux*, suspend occurs in 10 internal phases (shown in the image timeline). Kernel devices can register callbacks in any or all phases. They run in order during a suspend and resume. The timeline shows all registered callbacks and the time taken for completion. It also includes some general task blocks to fill out the timeline with contiguous information. For example: The file system synchronization in prepare and the thaw_processes call in resume_complete.

Figure 1

The zoom buttons allow you to zoom in and out on the timeline for finer detail. The keyboard shortcuts for zoom in, out, and reset are +, -, and *. You can also select the timeline contents and drag them left or right. Whenever you hover over a device, all callbacks highlight in all phases.
 

Selecting a device produces a device Detail Pane that follows the timeline. It shows how much time was spent in each phase. The Device Detail pane gives the full name of the device and the total time all of its callbacks took to complete (suspend or resume). Each phase callback is scaled to its relative length, which shows where most of the time was spent at a glance. To see a tabular list of all the devices and their callback times, on the left side, select Device Detail. This is also useful in determining which devices have parent/child relationships.
 

If you used the -f option in the test run, the page includes a detailed call trace of all the registered device callbacks.

Note There is a huge amount of data (hundreds of megabytes) in a callback trace that may tax your browser. Your outputs may take time to load on screen. The call graph data displays beneath the detail pane and filters based on the device you selected.

Figure 2

The data in the image is taken from ftrace. Each item is a collapsible tree view of all the calls measured in each device callback. This is the lowest level output that can point a developer to specific functions needing optimization.
 

The tool also supports advanced options. One option is the dev mode option that allows you to graph a collection of function calls through kprobes. Dev mode focuses on delay calls: msleep, schedule_timeout, and udela. This is useful in determining how much of the wait time is hard-coded into the device driver. It also includes a suite of callbacks that are important to subsystems in the kernel, such as ata and i915. The following is a dev mode output.

Figure 3

You can also run the tool with the -x2 option, which issues two back to back suspend and resumes. This is useful to:

• Determine when asynchronous resume processes are fully complete. (Many devices spawn asynchronous processes that continue after the resume completes.)
  
• Monitor which user processes are running before and after the suspend.
  

The -proc option enables the tool to collect process information in addition to kernel info. The following timeline was run with both -x2 and -proc.

Figure 4