Start with this. See what questions it raises for you.

https://learn.jamf.com/bundle/technical-articles/page/Migrating_to_Another_Server.html

I would very strongly recommend performing an in-place upgrade on the Windows Server. If that is not possible, you can move the database, and other necessary components to a new server with the same version of Jamf on it and recover the database. Updating DNS, CName redirects and all that good stuff so Macs will keep talking to the new server. Moving from Server A to Server B is not for the faint of heart, but totally doable, and you can fail back to the old server simply enough.