Large pkg upload failures

[rpayne]

Has anyone else encountered failures when uploading large files? I have been attempting to upload a pkg containing the installer for Catalina to our instance and the upload keeps failing.

We have a cloud instance. I have renamed the file multiple times. I have tried in Jamf admin AND directly through the Jamf pro Webb interface. I've tried it remote from my home office (gigabit fiber) both wired/wireless, and from both wired/wireless in office. I have had this happen in the past and it was never resolved. The pkg in question is 7.7gb. It has done the same with 2+gb files as well (office pkgs).

[CSCC-JS]

Are you using Composer to capture the "Install....." in the Applications folder, then uploading it via jamf admin or trying to drag the installer as is in to jamf admin?

[rpayne]

The pkg is from the installer file in the applications folder. When run, the pkg just places it back in that same folder. Yes though, we use composer to capture it in the folder.

[_gsm]

I had issues with an Adobe CC package. I switched the File Share Distribution Point to a local server, uploaded the package with Jamf Admin, and then replicated to the cloud. It's not convenient but it works.

[alexmcclements]

same issue here.

[ithangdang]

You may need to reach out to your Jamf buddy to have an engineer look into your instance. We run Jamf on-prem on AWS and ran into these same issues when the web server was not allocated with enough space as it has to cache the file there before it gets placed on your Distribution Point.

[rpayne]

Reaching out right now

[RWynner]

@rpayne Did you get this resolved? I am having the same issue.

[tomt]

Also seeing the same issue today with small packages.

[hepvd]

Same but luckily I'm on Premise and I can drag them on the SMB share (plus its waaaaaaaaay faster)

[tomt]

Supposedly my issue was sent to Engineering but I have not heard anything at all since yesterday and my replications are still failing.

[erichughes]

Was having the same issue on 6/16 for packages of any size. Some would upload properly after the second or third try or after renaming. Are there limitations on name length or other things. I would guess no special characters. I try to limit those to . _ but sometimes a package may have a long version number and contain multiple periods like Chrome usually does.

[tomt]

I've heard back from the escalated support, Engineering is supposedly looking at the issue. We'll see how that goes.

[tomt]

Just heard back again (sorry for spamming this post) and they have identified the failure in the logs.

The evidence in the logs is consistent with the issue now on the rise where we see the FileRecombinationJobCoordinator showing “0% complete” over and over again until we see Recombine task failed for file [<package>] in bucket.

Engineering is on it, I'll update with any further developments.

[erichughes]

Just uploaded The Office 16.38 updates out of 5 uploads 2 hang at availibity pending for a long time and then fail. OneNote and Word are the ones that fail and they were the 2nd and 5th upload. So its pretty hit or miss on size and timing.

[erichughes]

Magically working today for the same exact packages that failed before. Did not re-download or repackage anything.

[tomt]

I'm still getting the same errors that I have been seeing since Tuesday. @MollySchmidt

[jmuchow]

Same here. Uploads to the Jamf Cloud Distribution Point have been failing intermittently over the past several weeks but are failing consistently now. This is the case when attempting to upload either via Jamf Admin or the Jamf Pro Console. Tried renaming the packages but that did not help. The weird thing is, the uploads do not error out and appear to complete successfully. The only way to know if the upload was really successful or not, is to display the individual package in the Jamf Pro console, which is tedious. I wish Jamf would fix this to display a green or red dot next to each package in the list so the upload status could be seen at a glance and failures would stand out. We have reported the problem to Tech Support and are waiting to hear back.

[erichughes]

I got a response from tech support saying they are working on it, and my ticket has been attached to a "product issue". Maybe it will be fixed with the upgrade roll out over the weekend.

[tomt]

@erichughes That would be nice. Unfortunately I'm a pessimist and expect this weekend's upgrade to add new bugs. LOLOL

[tomt]

Still no change on replication. I have not tried to upload anything new but both issues appear to be related.

It seems my last post was rather prophetic, they pulled the update due to a massive security bug.

[jwojda]

I had similar issues with the Office update package. Renamed a couple times, tried admin and the JSS console, neither worked. Tried zipping (per supports recommendation) and uploading that - which failed. Finally renamed again (added a '1' to the end of the filename) and that uploaded and synced fine.

weird.

[hrhnick]

I had a few failing last week, and all are failing this morning. No change on the Status page though: https://status.jamf.com

[tomt]

Just received this in a response to my open ticket.

"After the surge in cases attached to this issue last week, our Engineering teams will be meeting this week to determine next steps. It is going to take time for them to find a code level resolution to the issues with the file uploader."

So it sounds like nothing at all has been done and they are thinking about maybe doing something sometime in the future but have to have meetings about it. I really can't stand the amount of corporate BS that has become the standard when dealing with Jamf.

[ega]

Folks,
This "planned" feature gets at the heart of most of these issues
https://www.jamf.com/jamf-nation/feature-requests/6665/facilitate-uploads-to-cloud-distribution-point-via-api-or-other-non-manual-method
Vote it up and ask your support person why it's been "planned" since 2017...

[Exploding-Toads]

I am having the issue on any size package. I am getting around the problem by first deleting the existing package on Jamf Cloud and recreating a new instance. I also then have to add the new package to any policies that used the previous package.

[tomt]

This needs to be kept at the top so Jamf will actually give it any type of priority.

Anyone else having this issue PLEASE open a ticket.

[john-hsu]

@tomt I've been trying to do an initial sync from my on prem distribution point to the Jamf Cloud Distribution point when I started encountering problems on Thursday. I initially started it on Tuesday, but then started having on Thursday with files trying 3 to 5 times to upload - even if they are small files (such as 4 MB). I'm going to query the person I'm working with on my support ticket. I'm glad I found this thread.

[Polargeek]

Support told me today it's related to PI-005548 but given that, that appears to be a 2 year old issue I'm quite unimpressed

[tomt]

Except that SOMETHING changed in the last week that seems to have exacerbated whatever issue that PI is caused by.

[mistacabbage]

My uploads are failing around 80% of the time!

[emily]

We're seeing a lot of failures recently, no matter if the packages are uploaded through the web interface, via jssimporter, or through Jamf Admin. We finally sent in a note to support today. It's getting a bit ridiculous.

[hrhnick]

Another day of failed uploads for me 🙁

[tomt]

I was told that we should get word in this thread from someone on the Product Team.

[john-hsu]

Thanks for the update,@tomt ! Was a time frame given to you on when the Jamf Product Team might have a resolution?

[tomt]

No timeframe yet. I have been trying to get valid information out of Jamf for two days now. I've escalated this through both front and back channels (I know a few people) to get it this far.

"A Product Issue was re-opened to address the issue, PI-005548"

I don't want to copy/paste the entire email because that doesn't seem right. I will say that it was full of corporate buzzwords like Product Owner, Cadences, Efforts, etc. which does not give me a warm and fuzzy feeling. I hear them every day and usually they mean almost nothing.

That's why I prefer dealing with Engineers, they don't have time for bullshit.

[CGundersen]

Yeah, even 100 MB pkg uploads failing these days. Jamf Cloud Team ... PLEASE get some traction on this issue as I'm unable to get stuff done under these circumstances.

[mscottblake]

I got this note back on Monday:

I believe you have encountered an issue we are tracking currently here at Jamf regard to issue (PI-006754) uploading packages to Jamf Cloud. As far with our testing, even smallest package failed to upload to cloud distribution point and deleting package objects allow you to upload again. We also notice if you rename the package and deleting the object package titles also allow you to upload packages too.

[tomt]

@mscottblake That USED to work and seems to be the new standard boilerplate response from Support.

[mscottblake]

I only thought it noteworthy because I got a newer PI number.