Mac hardware refresh - DEP and user data migration?

[michael_k]

Can anyone provide feedback on how you are migrating user data from existing Macs for hardware replacements? Particularly when it comes to DEP workflows. I know how we have done it for years and while it has worked for us, there probably are better ways we could be doing hardware refreshes. More importantly, Mojave has presented additional challenges that mean steps we took in the past either shouldn't be done or flat out don't work.

Until mid-2017, we were still firmly encamped in imaging new hardware with a monolithic image. When it came time to upgrade hardware, we would just connect up a Thunderbolt cable and drag over the contents of the user's directory and apply the permissions. Worked fine for years.

Toward the end of 2017, we decided to move to DEP with the advent of High Sierra. Our user and management accounts were created during enrollment. Again, we would just log into the management account on the new machine, move the user's data to the user account and apply permissions. That seemed to work fine until late last year.

Moving to Mojave has presented us with problems, primarily with regard to permissions. Some folders inside the user's directory apparently can't have their permissions changed even while operating as su. Should we be using Migration Assistant now to pull in user data? I'm sure users here can point out flaws in our process and can suggest a better way for us to be doing this. I'm open to all advice. Thanks!

[jared_f]

I created a policy that mounted the users home folder and created a backup folder inside it and made it available in Self Service. Then rSynced Documents, Desktop, Pictures, Music, Etc to that folder.

Then the user could drag down whatever they needed. You could make a script to sync everything backdown, but I didn’t.

I am a big Google Drive person and I am finding more and more people are going cloud so this is no longer becoming an issue.

[michael_k]

Thanks @jared_f . How would you deal with transferring data from the user Library though, like preferences, MS Office settings and keychains?

[daniel_behan]

I've been either using Migration Assistant or CrashPlan backups. I'll perform the migration prior to enrolling, then use "sudo profiles renew -type enrollment" to trigger DEP. I'm using SplashBuddy for my enrollment workflow and the policy that handles the user template has a pre-flight script to check to see if Outlook has ever been launched. If it has, then the default user template won't overwrite migrated data.

[michael_k]

Interesting. Do you have prestage enrollment configured? If so, do you just prevent it from kicking off by keeping the machine off net? We do have CrashPlan in our environment, but have found restores of any significant size from their cloud service to be horribly slow.

[kowsar_ahmed]

Why can't you TDM and transfer the contents of the folders across? No need to change permissions so long as you copy the data within the folders... So don't copy the desktop and overwrite the desktop folder on the Mojave Mac. Copy the entire contents within desktop into the desktop folder, that should make sure it inherits permissions from it's parent folder... Best to do this logged in as the same user.

[michael_k]

@kowsar.ahmed , we can do that for every directory except ~/Library. Unless I was shortsighted over the years, you can't just replace preferences, containers, keychains, etc while logged in as the user.

[daniel_behan]

@michael_k Our prestage is configured. The smart groups are set for computers with default names containing MacBook, or iMac, Mac Mini and don't contain Microsoft Word. Our standard naming convention is the device serial number. For the sake of data migration, we have kicked off the migration with the machine off net and use the profiles command to enroll after the data has been migrated. For some legacy machines that are not in DEP, we simply use the /enroll URL and splashbuddy will still kick off with all the same smart group criteria. Cloud restores can take a bit, so they're used as a last resort.

[mpermann]

@michael_k I wrote a small bash script to do an rsync of their home directory from a Time Machine backup to their newly setup computer. I run the script from my Jamf Admin account on their new computer. This preserves their Keychain as long as the login password of their account on the new computer is the same as on their old computer. This has worked reasonably well. The computers are setup through DEP using a DEPNotify workflow to get all the apps installed and their user account setup on the new computer.

[kowsar_ahmed]

@michael_k I’ve always preferred not to migrate prefs over when moving to new hardware/os etc just so you can start ‘clean’ again. The dock and some
User prefs/adobe was the maximum I’d do. You should be able to copy in even when logged in just won’t take affect until a restart of Mac or services (eg, finder)

If you still wanted to move everything over then you can still copy everything I guess via terminal with elevated privileges...

[cpresnall]

With all of our systems enrolled in DEP, we found that using Migration Assistant was the only way to ensure that all Mojave data transferred without interference from the secured folder permissions. This becomes even more important when moving from one T2 chipset device to another.

We do occasionally run into the keychain permissions issue, but there is a script here on JN that allows the user to self-resolve that issue.

[Chris_Hafner]

@michael_k While I completely agree with those here who prefer NOT to move the Library folder completely, it is easily doable. Our help desk generally migrates users, onto loaner machines using TDM. Simply log into an IT account on the "loaner" or a new machine, set the old/broken unit to TDM, drag the user home directory from one /Users directory to the new /Users directory. Then you create the user in System Preferences, using the same short name (and ideally the same user password). When you create it, select "User existing home folder" and voila, the whole user has moved, settings and all (including the Keychain).

That said, we generally only do this at the help desk when a user simply needs everything on a different computer as quickly as possible. Otherwise, I prefer to transfer as little as I can out fo the ~/Library directory.

[lmatthews]

We migrate data by either rsync-ing the whole user folder or just user data (Desktop, Documents, etc...). If a full transfer is done it can create a nice user experience of the environment looking identical to the old machine but some users don't want that. We either use a external drive or, recently, been using target disk mode over thunderbolt.

[bartlomiejsojka]

@michael_k, I believe You should be able to copy everything including the ~/Library folder — keeping your current workflow — as long as you'll manually add Terminal or Finder app (whichever you're using) to Full Disk Access in Privacy tab of the „Security & Privacy” pref–pane on the Mac you're trying to migrate from. Or at least it will allow you to change permissions, so you can copy.

[k3vmo]

I too like the idea of not moving ~/Library - however, our Exchange environment only keeps 90 days worth of mail. Many [unfortunately] rely on mail for an archive.

How would you transfer Local saved email from one system to another? Outlook mail would be the only thing I'd need

Thoughts?

[andrewstandifor]

I have this issue as well. Items are stored locally in Outlook and our email server environment doesn't allow people to store mail. Ideally I'd like to be able to migrate their outlook mail to a new system without resorting to the system Migration Assistant. Is there a viable solution to this problem?

[ira_friedwald]

I had a great deal of trouble last night trying to migrate about 45 GB of data (including ~/Library) from an older 13" MBP to a brand new one. Since I can't use disk images anymore (a real bummer, Apple) I used the migration assistant to automate the process. For the life of me, I could not get the Library directory to copy over intact. Every time I tried log in as the user on the new laptop it would crash after an immovable dialog box asking for a password for the Library directory. Very frustrating. I finally did an AirDrop of the Library folder from the old laptop to the new one, AFTER I changed all the permissions on the old Library first. Even still, there were maybe 20 or 30 folders that didn't transfer well and were still locked. I deleted the Library directory that wasn't working and dragged the new Library in. All this was done from the admin account on the new machine. Finally worked. However, I discovered that there are still a bunch of folders in the Downloads and Documents directories that need their permissions set right by hand. Did that today.
Next time, I'm going to run all the system updaters on the old machine first, then try a third party copy mechanism, perhaps Carbon Copy Cloner to move the entire User directory over first as suggested above by chris_hafner.

[adamcodega]

@jared_f Could you share your policies and scripts on GitHub, blog post, or screenshots? This sounds like a decent method if someone can use it themselves through Self Service.

[john_bio]

You really shouldn't be migrating the ~/Library folder over onto other computers.

The sheer number of files that can occasionally end up in there will take a long time to copy over and you'll most likely bring any problems the user had over to the new machine. (corrupt preferences, etc)

[jared_f]

@adamcodega Sorry for the late response. It is actually pretty simple for us as we are creating a backup to Google Drive. Quite honestly, this script could be far more complex and I could have dozens of checks, etc, but since this is temporary as I am now mandating Google's Backup and Sync to be installed and configured it is working for me as I am sitting there while it is going on.

If you wanted to, you could use this with an SMB share and just have it mount in the beginning and adjust the paths in the script. I scope on a smart group which has the set criteria: Has application Backup and Sync.app and is in a static group "Needs Backup".

Backup Data:

#!/bin/sh
# Script to sync user data to Google Drive. Requires Google Drive Backup & Sync to 
# be installed and configured.

# Make backup directory in users Google Drive folder
mkdir ~/Google Drive/Backup

# Sync user data to backup backup directory.

rsync -a ~/Desktop ~/Google Drive/Backup
rsync -a ~/Documents ~/Google Drive/Backup
rsync -a ~/Downloads ~/Google Drive/Backup

exit 0

I am working on a script now to re-sync the data, using rsync again but reversed.

[k3vmo]

I need to have users data copied to a network drive however, they're frequently switching between WiFi and Ethernet. Can anyone think of a way to do this only when Ethernet is active? Possibly show them some sort of status? I can't figure out how to show an Apple Event [popup window] from a script or terminal process

[jared_f]

@k3vmo You could scope to a network segment in Jamf Pro if your wireless and wired networks are different subnets.

[ryan_ball]

Within the policy you can configure client-side limitations, one being when on ethernet. So when configured this way, the policy will only show up in Self Service when the user is on ethernet.

You can also script something to check a Mac is connected to ethernet as well.

#!/bin/bash

# Detect to see if we are connected via wire
function get_adapters () {
    ethernet=$(/usr/sbin/networksetup -listallhardwareports | grep -A 2 "Hardware Port: Ethernet" | grep "Device:" | awk '{print $2}')
    ethernetIP=$(/sbin/ifconfig "$ethernet" 2> /dev/null | grep "inet" | awk '{print $2}')
    thunderbolt=$(/usr/sbin/networksetup -listallhardwareports | grep -A 2 "Hardware Port: Thunderbolt Ethernet" | grep "Device:" | awk '{print $2}')
    thunderboltIP=$(/sbin/ifconfig "$thunderbolt" 2> /dev/null | grep "inet" | awk '{print $2}')
}

get_adapters

while [[ "$ethernetIP" == "" ]] && [[ "$thunderboltIP" == "" ]]; do
    echo "No active wired adapter found"
    sleep 5
    get_adapters
done

# Any code beyond this point will only run once an ethernet connected is detected
echo "Ethernet is detected; continuing."

exit 0

[ryan_ball]

Here are the scripts that I use to backup to/restore from a network share.

[mattrobb]

You really shouldn't be migrating the ~/Library folder over onto other computers. The sheer number of files that can occasionally end up in there will take a long time to copy over and you'll most likely bring any problems the user had over to the new machine. (corrupt preferences, etc)

I really don't get this argument. If preferences are corrupt, fine, leave them out. Maybe in the education market if a student were to lose MS Office templates, Internet bookmarks, keychain information then big deal. But in a corporate environment where machines are often used for years, items in ~/Library are often pretty important to end users.

[Garci4]

Reviving this thread, would anyone be willing to share a detail workflow of how they do this? Interested in rsync but I have no experience with it. Would this Jamf Now support article still be good in the age of Catalina or anything to be aware of with the split of data & system folders?

Thanks in advance experienced Mac admins!

[naschenbrenner]

@ryan.ball our users network drives are housed as such Server/share/$loggedInUser, when trying your script it's not mounting the drives, and the log shows

mkdir: /Volumes/share no such file or directory mount_smbfs: could not find the mount point /Volumes/share: no such file or directory

When manually mounting our drives they appear as /Volumes/$loggedInUser (aka their username). Any ideas how to get them mounted properly since we have an extra level in our share?