Amdahl's law

If part of the work must run sequentially, that part sets the ceiling on how much faster the whole job can get.

This applies to computers as much as people. Adding more compute to a process that must run sequentially (or adding more people to a project where there’s some need for work to happen in order) won’t necessarily speed it up. The limit is based on that piece that can’t be done in parallel.

I love rules of thumb. They’re especially useful here when you want to figure out if a system can improve through parallelizing parts of it. Jeff Dean (Chief Scientist, Google) wrote a deck where he treats back-of-the-envelope performance estimates as a core skill and contrasts parallel and serial system designs to show how the slow stage dominates wall time.

Important skill: ability to estimate performance of a system design - without actually having to build it!

For product and architecture, the law is a budget question before it is a hardware question.