Creating & Restoring Deployment Images in High Sierra

[d_valentine]

Basically my goal is to achieve a monolithic system imaging in High Sierra.

Previously, I was able to do this as per the link below (scroll all the way down to the Advanced section)
https://support.apple.com/en-bn/HT202841

Here's what I did:
I upgraded a MacBook from Sierra to High Sierra OS. I then booted it to recovery mode and entered the following commands in Terminal to create the image.

*hdiutil create -srcFolder /Volumes/Macintosh HD/ -size XX -format UDRW -layout GPTSPUD /Volumes/ExternalHDD/tempimage

hdiutil convert -format UDZO -o /Volumes/Macintosh HD/FinalImage /Volumes/ExternalHDD/tempimage.dmg

asr imagescan --source /Volumes/Macintosh HD/FinalImage.dmg*

This successfully worked and I then copied the image to an external hard drive.

However, the problem I'm having is restoring the image created, either on the same machine that I created the image or on another Mac. Note: Both Mac disks are already formatted to AFPS and running High Sierra.

When I enter the below commands, I get an error stating "APFS inverter failed to invert the volume -invalid argument."

asr restore --source /Volumes/ExternalHDD/FinalImage.dmg --target /Volumes/Macintosh HD/ --erase

I have a feeling I may be entering the incorrect target location when trying to do the restore but I may be completely wrong altogether.

Any ideas?

[Chris_Hafner]

That's an interesting goal... one that most of us have been trying to move away from for some time. However, it's now back in Apple's guidance (Despite several engineers and other folks claiming that "imaging is dead", but I digress).

Let me ask this question... why do this from the recovery partition/CLI? Composer has the ability to capture your whole image as well as the recovery partition for deployment. Have you tried that? Simply get a machine setup the way you want, set it to target disk mode, connect it to a computer with Composer, and then walk through the "Build OS Package" wizard. Easy as pie.

[alexjdale]

Apple likes to claim that inconvenient processes are "dead" so they can make changes without consideration for them. They told us that MDM was a requirement to scale out our macOS deployments when we complained about SKEL requiring it. I found that amusing since we had scaled up to 50% more than our current Mac count (our company has since split), so I know we don't "need" MDM.

That said, we have to have a method of wiping a system and reinstalling the OS to factory settings, since we aren't going to throw away computers when a person leaves the company. Laying down a base image in a minute through an automated process is significantly better than having technicians perform a full OS install manually (which we then have to patch if they aren't keeping it current). So even if I just use an empty base image to start with, I need a "monolithic image" process to do that.

I don't care how I get that base image, but having technicians manually install the OS each time is ridiculous. I need automation and consistency to manage a modern enterprise Mac population.

[easyedc]

Getting in on this thread since I'm in the same boat. what I believe you ned to be running is identified by diskutil.

MacBookPro:~ User$ diskutil list
/dev/disk0 (internal):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                         2.0 TB     disk0
   1:                        EFI EFI                     314.6 MB   disk0s1
   2:                 Apple_APFS Container disk1         2.0 TB     disk0s2

/dev/disk1 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +2.0 TB     disk1
                                 Physical Store disk0s2
   1:                APFS Volume MacBookPro          1.2 TB     disk1s1
   2:                APFS Volume Preboot                 19.0 MB    disk1s2
   3:                APFS Volume Recovery                520.8 MB   disk1s3
   4:                APFS Volume VM                      1.1 GB     disk1s4

and then your image would be applied with

asr restore -s /Users/Ed/Desktop/10.13.0 Base Image.dmg -t /dev/disk1 --erase

In this case, it's

/dev/disk1

since that's the "synthesized" APFS share that would get imaging. Generally for me it's usually

disk3

as the first attached external drive. But that could vary based on any other attached bootable volume connected.

The other gotchya is the firmware part. For reimages of workstations that Have already had 10.13 installed, this seems to go ok. For new upgrades, this is problematic. I believe there is a thread out there (either here on JAMF or google) about pulling out the firmware update tool from the installer and making it a pre-run pkg first.

As for building the OS base image, my vote is for AutoDMG. It has worked for me so far, albeit some of the packages I've tried to install during that build have failed, the OS works.

[alexjdale]

Question on ASR since I haven't used it: if we boot to a USB environment, can ASR restore an APFS image created by AutoDMG to the internal drive? It sounds like it can, but I want to confirm.

And if it can, I wonder if Casper Imaging.app will be able to do this to automate the workflow like we do now (assuming JAMF builds it into the app, of course).

[easyedc]

@alexjdale Yes, that's how I've done it (from an OS running on an external). What I've run into are issues getting the formatting to occur through disk utility to create the APFS container. And the issue about firmware. Assuming the APFS containers are created properly, though, the imaging has taken only a matter of 2-3 minutes to block-copy at that point.

[monaronyc]

Hi @Chris_Hafner trying to follow your instructions. We were asked to start building Mac Book Pro and Mac Book Air High Sierra images and we're having a gang of trouble! When we get to the screen to select the disk in Composer, it's grayed out. Any ideas?

[Chris_Hafner]

Are you using FV2? Composer usually looks like that when dealing with an encrypted drive. That said, I have to get back around to testing this since I did shift away from using Composer for this over the past few years.

[monaronyc]

Thanks @Chris_Hafner! No unfortunately. This Mac Book Air hasn't been FV'd yet. I'm wondering if its the version of Casper we're on. 9.1.00.

[msmith80]

To capture the image you have to be booted into another device I believe. Target disk mode the device you want to capture and open composer on the second Mac.

[monaronyc]

Thanks @msmith80! Yes, that's what that screenshot i attached is. The firewire looking disk is at the top is mounted on the desktop of the computer where the Composer app is launched from.

[alexmcclements]

I've been using a NetInstall image with a script that creates a local admin account and then JAMF recon enrolls the machine with JAMF and policies do the rest.

[TayjayP]

Problem summary:
While upgrading to a bigger SSD (960GB from Transcend- TA960GJDM500) in my MacBook Air mid-2011 (11”) (running macOS 10.13.6) the standard installation instructions from Transcend did not work. The Disk Utility process failed at the last stage due to a reported ‘failure to ‘invert’ the new drive or partition’, due to an invalid argument. Apple support recommended formatting in the older format, not APFS. I have not tried this.

This seems to be a common issue with the new APFS file format. However, I can share a work-around:

I have side-stepped the issue by loading a fresh OS onto the SSD and migrating data from a Time Machine backup and the original drive, both connected externally via a thunderbolt hub giving USB 3.0 ports, as follows: First I updated time machine backup on an external USB drive. Then:
1. Installed the new SSD in my MacBook 2. Attached the old SSD to a built-in USB via the caddy supplied - must be built-in, not via a hub. 3. Booted with cmd+R to ‘restore’ facility (provided by the old disk)
4. Installed a new copy of Mac OS High Sierra onto the new drive internally fitted. Before it invited me to set up;
5. Attached a thunderbolt hub with USB 3.0 ports to cut the time down. (It works with native USB 2.0, a lot slower!)
6. Attached my backup drive with time machine files to a USB 3.0 port.
7. During setup, I imported my data and accounts etc from the backup drive. A Warning on iTunes:
I was invited to update iTunes during the data import and the process crashed. Second time round I ‘skipped’ the update, completed the importation and THEN updated iTunes successfully. I have seen so many people stuck in this problem I hope this workaround helps someone out. I am now back to normal with the new drive running. Good luck.

[MathUser]

It might be late to have this, with new version of OS coming out, but this might be useful if it works for others:

Three problems with imaging APFS or with using asr that I see now:

1) New T2-chip computers encrypt SSD contents, so images taken from these computers will not work on other computers

2) asr utility or disk utility give 'apfs inverter failed to invert the volume' or similar when trying to restore an APFS volume

3) Disk utility or asr gives an error about "contains more than one APFS volume" or "contains an APFS volume" some such

For 1), simply create the OS base from which the image will be taken on an external disk. The internal disks are encrypted at rest via T2 chip, external disks are not.

For 3), it seems this has to do with how diskutility and asr handle dmg files. "contains an APFS volume" is an error you get if you try to apply an APFS container image to the "whole disk" (you have the DISK highlighted in Disk Utility, or you attempt asr to the Physical Disk device node in asr). One should always restore to the synthesized disk instead of the physical disk. So, have "Container Disk X" highlighted in disk utility or use synthesized disk device node in asr.
"contains more than one APFS volume" seems to be relating to specifying a .dmg file with an APFS container containing more than one volume. For some reason, this seems to always be trouble. Neither disk utility nor asr can handle directly specifying the .dmg file when more than one volume exists in the APFS container. They can indeed deal with this just fine, so it seems to be more a logical bug than a technical limitation.

So, to fix this, I find that mounting the .dmg image and sourcing restore from that is much, much more robust than attempting a restore directly specifying the dmg. Both the asr utility and Disk Utility will do the right thing when specifying a volume or container disk for a restore, restoring not just the one data volume in the APFS container, but also the 3 supporting volumes of Preboot, Recovery, and VM.

Consider that the system is booted in internet recovery or some other external boot, and that the .dmg file to restore from resides on a connected flash drive called "MyImages" , and that the image is called "TheImage.dmg" which contains an APFS disk image with the appropriate 4-volumes (data, preboot, recovery, VM). Say that the internal disk is /dev/disk0 So:
Format internal disk as APFS. Phyiscal disk is /dev/disk0 and synthesized container becomes /dev/disk1
Connect flash drive, flash drive becomes /dev/disk2
mount dmg file which is on flash drive. The .dmg file "disk" becomes /dev/disk3 and the .dmg file "container" inside the "disk" becomes /dev/disk4:

hdiutil attach -readonly /Volumes/MyImages/TheImage.dmg -noverify -noautoopen

Ensure that the disk to be restored to is not mounted. Physical disk should be unmounted, container should be unmounted and ejected:

diskutil unmountdisk force /dev/disk1
diskutil unmountdisk force /dev/disk0
diskutil eject /dev/disk1

Restore from the mounted disk image:

asr restore --source /dev/disk4 --target /dev/disk1 --erase --noprompt

For 2), this seems to be caused after 10.13.3. After some research, it seems just like a stupid error.. some files are missing from the APFS filesystem tools directory on apple recovery os images after 10.13.3. If one simply puts these files back, then this error stops happening.

For instance, on OS X recovery after 10.13.3, here is the contents of /System/Library/Filesystems/apfs.fs/Contents/Resources:

drwxr-xr-x 8 merlin staff 272 Jul 20 22:37 . drwxr-xr-x 6 merlin staff 204 Jun 9 22:47 .. -rwxr-xr-x 1 merlin staff 552432 Jul 20 22:37 apfs.util -rwxr-xr-x 1 merlin staff 29456 Jul 20 22:37 apfs_stats -rwxr-xr-x 1 merlin staff 736528 Jul 20 22:37 fsck_apfs -rwxr-xr-x 1 merlin staff 1303440 Jul 20 22:37 hfs_convert -rwxr-xr-x 1 merlin staff 34672 Jul 20 22:37 mount_apfs -rwxr-xr-x 1 merlin staff 584272 Jul 20 22:37 newfs_apfs

For same directory on 10.13.3 or before:

drwxr-xr-x 10 merlin staff 320 Sep 28 11:07 . drwxr-xr-x 7 merlin staff 224 Sep 28 13:22 .. -rwxr-xr-x 1 merlin staff 552432 Jul 20 22:37 apfs.util -rwxr-xr-x@ 1 merlin staff 559552 Jul 20 22:37 apfs_invert -rwxr-xr-x 1 merlin staff 29456 Jul 20 22:37 apfs_stats -rwxr-xr-x 1 merlin staff 736528 Jul 20 22:37 fsck_apfs -rwxr-xr-x 1 merlin staff 1303440 Jul 20 22:37 hfs_convert -rwxr-xr-x 1 merlin staff 34672 Jul 20 22:37 mount_apfs -rwxr-xr-x 1 merlin staff 584272 Jul 20 22:37 newfs_apfs -rwxr-xr-x@ 1 merlin staff 545936 Jul 20 22:37 slurpAPFSMeta

I solved this for myself by simply mounting over this directory with another .dmg that has the right contents. So, I coped the contents of /System/Library/Filesystems/apfs.fs/Contents/Resources to a folder, then created a disk image of that folder in Disk Utility, Scanned it fore Restore, opened once to verify, then put it on the same storage location as the image to restore. Before restoring the image with asr or diskutility, I do:
hdiutil attach MyImages Resources.dmg -mountpoint /System/Library/Filesystems/apfs.fs/Contents/Resources

This mounts over this location on the booted recovery OS with a version of the directory that doesn't have missing files. Now, the restores all work.

[jpfu]

This might help, has a script to create OS dmg...

https://github.com/munki/installr

[Nix4Life]

That does not, It makes a .dmg of the installer application and any packages that you wish to install. That is not an OS.dmg as pertaining to this discussion.

[Machead]

I solved this for myself by simply mounting over this directory with another .dmg that has the right contents. So, I copied the contents of /System/Library/Filesystems/apfs.fs/Contents/Resources to a folder, then created a disk image of that folder in Disk Utility, Scanned it for Restore, opened once to verify, then put it on the same storage location as the image to restore. Before restoring the image with asr or diskutility, I do: hdiutil attach MyImages/Resources.dmg -mountpoint /System/Library/Filesystems/apfs.fs/Contents/Resources This mounts over this location on the booted recovery OS with a version of the directory that doesn't have missing files. Now, the restores all work.

I love everything about this post. I was getting the "failed to invert" error while trying to restore an AutoDMG-created 17G2208 image to a 2018 MBP, and the information quoted above was spot on, and extremely helpful to me.

FWIW, I was able to successfully restore my 17G2208 image by ⌘-option-R booting into Mojave internet recovery, which appears to have a complete/correct set of files in its /System/Library/Filesystems/apfs.fs/Contents/Resources folder.

You'd better believe I'm keeping that "hdiutil attach /Volumes/myShare/myFixes.dmg -mountpoint /System/Some/Broken/Recovery/Stuff" trick in my pocket, though. :) Thanks, MathUser!

[dmaestre]

@Machead What's the size of your bootable apfs volume you're using?