How to use linux perf tools and save dollars - Part 2

Flame Graph - Introduction

In our previous discussion (I strongly recommend to go through the linked blog), we have seen how in real world, linux perf tool would be helpful in writing a very well optimized code. We have seen an example of how we could save loads of CPU cycles in the example linked. 

Now we will see how to visualize the output which would help us to find the bottle-neck super fast!!!

What is flame graph?

Flame graph is a tool which converts the binary data file from perf tool into a "*.svg" a more comfortable visualized format. In fact, to put it more generically flame graph is a tool which helps to represent different profile data - like perf tool, Dtrace etc in a visual format.

Ok... Tell me how to use it

Step - 1

Install the flame and generate perf data dump. For this, we are using the example in previous discussion - and generate the perf data dump. Check the below GIF which provides the steps executed - 

Install Flame Graph and using perf generate the perf dump

Step - 2

Follow the steps shown in the below GIF to convert the dump into a "*.svg" file which can be opened in any browser and analyzed.

Convert perf data to flame graph

Step - 3

Analysis of the graph

• In the GIF, you can see that the graph generated most of the stack (represents CPU cycles) is related to usleep system call.
• Also, the usleep is triggered by consumer call-back function of the consumer thread.
• Compare the total area of the consumer and producer we would be clearly identifying the total CPU consumed by consumer is far more than producer.

This immediately helps us concluded that - consumer slow is excessively slow and is due to the function usleep in the consumer call back function.

Óyela.... we will get all required answer!!!!

Conclusion

• Use the flame graph tool very effectively, (even in multi-threaded applications also) to visualize your application bottle-neck and fix them.
• Both the long term costs saved, and time to debug will be sufficiently reduced with this great tool.

Keeping looking into this space, you would get much more information on this tool.

Would be happy to hear you out and help you... Please leave your opinion in the comment section.

And a closing thought...