JSS on Windows, replicating Distribution Points

[RobertHammen]

If you're running your JSS on Windows, or are using Windows-based distribution points, what are you using to replicate them? Rsync for Windows? DeltaCopy? Syncrify? Something else? Need to know what's working in the real world...

[jarednichols]

I'm not using it, but Robocopy is a great tool

[trherman]

we are planning on trying to move to windows and was wondering if you have figured out a way to do imaging and software updates from windows servers?

[perkins]

We are also interested in setting up Casper on Windows servers.

This article refers to "master distribution point" discussing issues with SMB and Adobe AAMEE build packages.

https://jamfnation.jamfsoftware.com/article.html?id=161

Are there other situations where an SMB share breaks the Casper Suite workflow?

[talkingmoose]

In all fairness, SMB is not the problem. Adobe is using aliases/symlinks in its installers (a no-no in my book) and they are breaking when used with SMB. Not positive if these aliases/symlinks are within the original .app installer or just the AAMEE builds but I believe just the AAMEE builds, which means we may get to see this resolved sooner rather than later.

I'm not sure why the aliases/symlinks are breaking but that's the explanation that Jody Rodgers with Adobe provided me. The .dmg method described in the link you provided is the only known workaround he has at the moment.

[donmontalvo]

@trherman, JAMF released a NetBoot/SUS appliance, but it's on Ubuntu. Not sure they plan to release a Windows solution.

@RobertHammen, did you come up with a Windows replication solution? If so, any chance you can post your scripts? We're testing now for eventual migration from Apple to Windows on the server side. Replication is something we're not sure how we will handle...Robocopy seems to be the staple, but if that's so, we'd love to see examples of scripts that are being used in production (read: proven to be stable and reliable). :)

Don

[CasperSally]

Has anyone tested compiling on windows distribution points? We tested it awhile ago with some very strange results compiling (randomly just not working almost all of the time). Same configs compiled fine on OSX server distribution points. It's what has us looking at buying mac minis for distribution points next year.

[jarednichols]

Anyone solve this? Standing up Windows 2008 R2 JSS and DPs

[donmontalvo]

Our ace datacenter team are looking into setting up DFS to keep the Master replicated out to the Replicas. Hoping to have some notes to share when we're done.

@jarednichols How did the piggy-back-off-SCCM-distribution-points idea work out?

[jarednichols]

@don
Haven't tried it yet. Need to find a server to do it on first. I don't see it being a problem, honestly. Most packages will be chucked out via IIS (which I got wrangled yesterday as well). The biggest bugaboo is the DP replication.

[Jason_Lieble]

I'm currently testing out some robocopy methods, I'll respond to this thread if I get some concrete and acceptable workflows.

[nkalister]

i tried replicating from my current AFP DP to my new SMB DP yesterday using robocopy, and it was not able to copy any of my .mpkg files.
I tried again using rsync (via Synkron, which was mentioned in another JAMFNation thread), and that worked, but I'd rather use robocopy . . . Anyone else run into .mpkg problems with robocopy? I did some googling on that issue, didn't find anything.

[jarednichols]

I've gotten Robocopy setup in a dev environment. Both sides are Windows 2k8 R2 in a domain. Basically, the Robocopy action runs as a Scheduled Task as the domain user that has full read/write on the target box. I'm simply using the /MIR switch and haven't seen any issues copying mpkgs.

[nkalister]

yeah, I just used /mir, too, but I'm going from a Mac server to a Windows 2008r2 destination, so that's different.

[Kumarasinghe]

Jared, would you mind sharing your script?

[jarednichols]

@Kumarasinghe
It's not a script, just a scheduled task. The action is "Start a program" with c:WindowsSystem32Robocopy.exe selected and in the additional arguments field I put the source share, destination share (UNC path) and the /MIR flag.
I have the whole scheduled task being run as the domain service account that has read/write access over the destination share.

[donmontalvo]

FYI, we tested Robocopy and it seemed to work, albiet very slowly on PKG/MPKG files as it traversed every directory during replication. Handles flat PKG and DMG files nicely. :)

The problem we're faced with is we are given a very strict window for replication, so anything that is copying at the end of the window gets corrupted. Robocopy doesn't appear to be able to intelligently complete whatever file(s) are being replicated when the window closes.

We are still waiting for the Wintel group to free up some time to look at setting up DFS, since it's a freebeee I can't rush 'em. :(

Don

[Kumarasinghe]

Thanks Jared. Really appreciate it.

[jarednichols]

Don-

How small is your window? Beyond the initial replication, the replication should be fairly quick. Are we talking minutes here??

[nkalister]

don, are you using the /z flag? that's supposed to make robocopy use it's resume capable mode.
using /z will slow down the throughput, though.

[donmontalvo]

@jarednichols The DP has roughly 200 GB. The windows we got are:

US > UK
Sat 1600 PDT -> Sun 0000 PDT = 8 hrs

US > HK
Sat 0600 PDT -> Sun 0000 PDT = 18 hrs

So we're being asked to do the initial replication to UK and HK on a weekend.

Then going forward we can replicate every day using the same windows.

@kalister Here is the command we tested...the Wintel tech triggered it in the command prompt and I watched as the PKG in the testfolder folder that measured 120 MB took a full 26 minutes to copy:

robocopy.exe \master.domain.comCasperShare$	estfolder \
eplica.domain.comCasperShare$	estfolder /MIR /R:1 /W:1 /Z

Don

[perkins]

Hi Don,

I am wondering what your link speed is to HK and UK.

I am interested in hearing more about how robocopy is working for you on moving data around to Windows SMB shares.

Perkins

[donmontalvo]

@perkins Link speeds below, of course this is shared bandwidth. So consider off hour traffic from the existing infrastructure that supports 10,000+ users and 1,000+ servers, with backups, replication, etc., happening on other systems during the same off hour windows...

US MPLS 200M
UK MPLS 150M
HK MPLS 80M

Robocopy isn't looking promising at the moment, at least not the string we were provided. Hoping what we were told about DFS handling replication was more than speculation. :)

Don

[nkalister]

doing more testing with robocopy today . . . only using the /mir switch. The Microsoft Communicator 13.1.3 mpkg, which is 28 MB, has kept robocopy occupied for the last 3 hours. No data appears to be transferring between the servers.

Jared or anyone else syncing this particular package successfully?

[donmontalvo]

@nkalister That's the problem we ran into with PKG an MPKG packages, it seemed Robocopy slowed to a crawl when it had to traverse the directory structure. We swapped out different PKG and MPKG packages but got the same results. When we tested with flat-PKG or DMG format, it was incredibly fast.

[Jason_Lieble]

Sorry for the delay, but I did multiple tests with robocopy and this workflow provided the best results:

---------------------------------------------------------------
REM For creating a network user for privileges (u:’username’ ‘password’)

NET USE 10.1.1.1IPC$ /u:username paswword

REM The actual robocopy command which replicates from SOURCE to DESTINATION

robocopy 10.1.1.1CasperShare 10.1.1.2CasperShare /MIR /FFT /Z /XA:H /W:5 /log+:robocopy_log.txt

REM Delete the network user

NET USE 10.1.1.1IPC$ /D

REM (optional) keeps window open to show progress

pause

---------------------------------------------------------------

The flags after the robocopy command are as follows:
/MIR specifies that Robocopy should mirror the source directory and the destination directory. Note that this will delete files at the destination if they were deleted at the source.

/FFT uses fat file timing instead of NTFS. This means the granularity is a bit less precise. For across-network share operations this seems to be much more reliable - just don't rely on the file timings to be completely precise to the second.

/Z ensures Robocopy can resume the transfer of a large file in mid-file instead of restarting.

/XA:H makes Robocopy ignore hidden files, usually these will be system files that we're not interested in.

/W:5 reduces the wait time between failures to 5 seconds instead of the 30 second default.

/log+:robocopy_log.txt outputs a log file to wherever the .bat file resides

*Use the /MIR option with caution - it has the ability to delete a file from both the source and destination under certain conditions. This typically occurs if a file/folder in the destination has been deleted, causing ROBOCOPY to mirror the source to the destination. The result is that the same files in the source folder are also deleted. To avoid this situation, never delete any files/folders from the destination - delete them from the source, and then run the backup to mirror the destination to the source. If in doubt, do not use the /MIR option, but be aware that backups will take longer**

[donmontalvo]

@Jason.Lieble Thanks for the summary. Has this been tested to ensure it transfers PKG/MPKG packages without the reported performance issues?

[nkalister]

update for the thread- my robocopy issues with copying .mpkg files appear to be caused by the SMB server in OS X. I can copy mpkg's just fine using robocopy if both the source and destination are windows servers- this confirms what Jared has been reporting. If the source is an SMB share on a Mac, though, then the problem occurs and the mpkg's cause the transfer to stall.