The Magento & PHP Code is the first thing you should be looking at.

From an online shop's servers to the customer, many things can go wrong when it comes to displaying web content in the blink of an eye.

You should definitely care about the front-end performance, use CDNs and caching strategies. But this is only hiding dust under the carpet if the back-end code is slow.

Blackfire is a Profiler. It is a tool for back-end developers to measure code performance, and find fixes & optimizations.