HDD Continues to fill up for no reason

[Fox]

Afternoon!

I reimaged around 30 iMacs 2017s and erased and reset up the HDDs as APFS and everything seemed good to go. However, I noticed that the Hard drives are filling up fast (school isn't in session so I'm the only one who has logged on) and are now maxed out. I originally thought that I must have spaced out and just sloppily messed up*. So, I re-imaged them again (I paid special attention to what I was doing this time around) and the same thing happened.

 They are Smart verified and I also ran Disk Inventory X which shows only about 80 GBs of real usage (this sounds about right). 

I'm wondering if the age of the HDDs could be doing this since they are used A LOT and constantly on. Any thoughts or articles would be great because the college won't want to buy new iMacs (but maybe new SSDs).

*I've tackled setting up maybe a few thousand (if not many thousand) and this is the first time that I have encountered this.

Thanks in advance!

[Qwheel · Accepted solution]

I’ve seen this before.

Have some of your devices stopped booting into MacOS completely. In that instance, you can only fix it by going into recovery -> terminal and locating the file that’s some 800gbs in size.

You can run ‘find’ and search for files bigger than 50GBs and you’ll find the data dump showing up (maybe with a different name) in the same location on each device.

im not sure what was causing it but I did some serious manual analysis and clearing up (asking myself “should this do this”, “is this required”).

It’s terrifying though.

If the situation is like mine, even deleting the file would only restart the dumping process. Gotta work quickly good luck!

check policies with recurring check ins and start smart grouping PPPCs so they only apply to devices with the apps installed. That, and bolt recons onto the end of policies using the ‘file and processes’ payload… so after installed the PPPC comes down as required.

also… I realised I dont trust the built in disk report in JAMF. We have a second extension attribute for “space remaining”, I’ve found it offers much better reporting IMO.

[sdagley]

@Fox Do you know what file(s) are filling up the drives? It's not unusual to find apps improperly spewing large amounts of data to log files. I'm not familiar with Disk Inventory X, but Omni's OmniDiskSweeper (https://www.omnigroup.com/more ) will give you a nice view of what's taking space on a drive so you can quickly determine if that's your problem.

[Qwheel · Accepted solution]

I’ve seen this before.

Have some of your devices stopped booting into MacOS completely. In that instance, you can only fix it by going into recovery -> terminal and locating the file that’s some 800gbs in size.

You can run ‘find’ and search for files bigger than 50GBs and you’ll find the data dump showing up (maybe with a different name) in the same location on each device.

im not sure what was causing it but I did some serious manual analysis and clearing up (asking myself “should this do this”, “is this required”).

It’s terrifying though.

If the situation is like mine, even deleting the file would only restart the dumping process. Gotta work quickly good luck!

check policies with recurring check ins and start smart grouping PPPCs so they only apply to devices with the apps installed. That, and bolt recons onto the end of policies using the ‘file and processes’ payload… so after installed the PPPC comes down as required.

also… I realised I dont trust the built in disk report in JAMF. We have a second extension attribute for “space remaining”, I’ve found it offers much better reporting IMO.

[Fox]

I will look deeper into the policies and see if that's what is happening.

-I took over this school less than two months ago and I am finding a lot of duplicate policies and a lot are pushing out bad/out dated policies--there's around 500 policies just for the mac side (there's around 1000 macs total so it's not huge). 

-I've been looking at the policy history and I am not seeing anything too crazy anymore on these machines....

[Fox]

---

check policies with recurring check ins and start smart grouping PPPCs so they only apply to devices with the apps installed. That, and bolt recons onto the end of policies using the ‘file and processes’ payload… so after installed the PPPC comes down as required.

It was something in the polices or something scoped to that specific group. I reimaged a couple in the same Static Group and they failed. I created a new Static group and applied the polices that I needed (as well as apps) slowly.

-I will have to run a sandbox on one and try and find out what was corrupting the HD (good news is that I got the lab up and running, well 85% up and running, before the first class started).

Thanks for sending me down this path! 

[Fox]

Sorry, I should have included photos in my original post to be more clear (sorry, one was snaped with my phone so it's a bit blurry).

The blurry pic is of Disk utility and it shows that 975Gb are being used in Mad HD-Data (this was zero before I reimaged it).

The other pic is of Disk Inventory X that shows what is being used--it shows only 80 GBs are actually on the machine...disk utility is telling a different story.

[sdagley]

@Fox Any chance you've got something being duplicated a _lot_? With APFS a file that's duplicated won't see a the cop(y/ies) take up any space until they're modified, but Disk Utility might not see it that way. Looking at the graphic you posted from Disk Inventory X I'd expect it to show any duplicated files though.

[EThom]

I have the same problem at one of my schools.  Were you able to find a solution?

[Fox · Accepted solution]

Yes!

It was a policy pushing a script (AD Bind script) as ongoing and it was triggered to run countless times a day (I inherited this issue--I didn't set this up).  It wasn't the script that was piling up so I highly assume that it was the log being created that would fill it up.

Anyways, changing the policy trigger flow resolved the issue.

[Qwheel]

I never worked out what the problem was, but I fixed it by re-assessing our whole JAMF instance - assuring EA, Config Profiles, Policies, Scripts all played nice.

In our area it was only happening to shared devices, so that helped narrowing it all down.

It's very frustrating though.

It'll likely be just one file thats filling it up.

Maybe try running this on a device that is midway through 'filling'. Or run it from Terminal in recovery mode... just make sure you're when in recovery it's running from the right folder... and not just the recovery partition.

Apple Remote Desktop is useful, it can fire out Unix commands like this to lab devices.

sudo find -x / -type f -size +10G -ls 2>/dev/null

[EThom]

I will.  Thank you for your quick response!