I've tested with empty packages and our Apps Package, there are no errors, there are also no references to it downloading in the install.log. It will then install during a policy trigger, but when done via prestage enrollment, it just skips over the package.
I've run out of ideas.
Are you installing it from an on-prem or cloud distribution point? PreStage package installs require either HTTPS or CDP to install successfully. If you have selected 'None' under "Enrollment Packages" in your PreStage it won't attempt to install at all.
When you use productbuild to create a distribution pkg on Big Sur (Intel and Apple silicon) both arm64 and x86_64 will be added to the configuration by default.