Over on the BruteProtect blog they have a look at the Jetpack Bloat Myth, and find that counter-intuitively even though Jetpack has more comprehensive functionality it’s faster than using individual plugins to do the same things. There are economies of scale to Jetpack’s approach, and it doesn’t even include the impact of doing things more advanced and complex like Related Posts. There’s a reason why some web hosts like WP Engine ban most related post plugins but encourage the use of Jetpack.
The performance of the plugin code, though still faster, is still a small difference when compared to the benefit of offloading certain tasks like image resizing, related posts, stats, video transcoding, and more in the future to the WordPress.com cloud (which is now across 11 datacenters worldwide).
Of course if you don’t need the functionality at all it’s always faster to have nothing, but that’s a shrinking minority. There are still more optimizations to be had, and in line with a performance focus in 2015 look for more improvements to come in the future. In the meantime, check out the Jetpack benchmarks.