Yes, if the policy has already run on those machines and successfully cached the app, you'd need another policy to remove it. There's no "undo" for policies in Casper. Though to be fair there are few management products that would be able to do that.
If this cached app is the only item you've deployed to be cached, it could be as simple as a one line script in the run command field for the policy to rm the contents of the /Library/Application Support/JAMF/Waiting Room/ directory, since that's where the dmg/pkg and an xml file that rides along with it get stored until they are installed later. Something like-
rm -Rfd /Library/Application Support/JAMF/Waiting Room/*
should do it.
If you've got other cached packages that you'd rather not remove, then you could still script it, but it would need to specifically target that cached package, not just anything in that folder. Instead of the asterisk in the above command, you'd need to put in the package or disk image name, but with an asterisk at the end as well so it also gets the xml file.
Also, you can create a Smart Group for the Macs that have cached that package by using the Packages Cached by Casper criteria in the Smart Group and entering the package name. And then use that group as the target. That way your removal policy is only running on Macs that already ran the original one.
Hope all the above helps.
