By default you can't. Jamf has stated they're not planning to impliment anything around this. Statements I have seen indicate Jamf believes sites (or lack thereof) should not be utilized for scoping purposes, and is not going to build out this functionality, despite there being many feature requests for it.
However, you can accomplish what you want a few different ways. I use a custom extension attribute that I detail here: https://community.jamf.com/t5/jamf-pro/custom-extension-attribute-based-on-site/m-p/228424#M216665
I have matured this process a bit and run a similar script from a secure machine that looks at a smart group with missing EA info, and then evaluates the site the device is in and then writes that data to the EA. This way nothing needs to be run on the local computers to set this data.
That is a similar approach taken in this thread as well: https://community.jamf.com/t5/jamf-pro/find-macs-not-assigned-to-a-site/m-p/269981#M246705
Basically you have to create a custom extension attribute where you can write the Site the device is actually in, and then script a method to do that writing.
You basically have to search for the negative. What I mean by that is creating smart groups with the only criteria is setting the site (say "All Computers in Site A", "... B", etc). Then make another smart group with the criteria Computer group > "not a member of" with each of those smart groups and that should give you the remaining devices.
Let me make this simple for you.
You can achieve this by creating a custom EA (Extention Attributes) because by deafult "Site" criteria is not avaiable in the Smart Group also once you have created the EA then you use this by following method.
Once you've created the smart group, it will automatically populate with devices that have no site assigned.
Hope this is clear to you.