How do you guys set up Smart Groups?

You are not going to get a consensus on this, everyone will be doing it different based on their needs, and policies within their organisation. Instead of asking the world in general, look to your organisation, and other services within that rely on group names for things.

Jamf Pro is basically just who, what, when. A Smart Group can be a who - a group of devices or users that meet one or more criteria at any given time. So instead of just thinking static inventory, think about all the variables that could change on a device, like the installed version of the OS or an app, or its available storage space, or if its connected to network B instead of network A, or if its missing a necessary configuration profile.. keep it modular and granular. 

It would really depend on what data I wanted from the smart group... If I wanted to know how many machines in xxx building were running yyy software, I'd use that criteria. But normally, I set up Smart Groups to scope update-type things, so it'd be like "any machine with xxx software installed that is not yyy version"

As @wakco11 said, there's no one way to do those.