Posted on 09-24-2019 12:04 PM
Is there a way to do this? I cannot figure it out. I have tried to create multiple 500MB ZIP files to then be extracted when they are all on a machine, but it keeps failing when i go to extract it. "invalid compressed data to inflate"
while replicating, due to some of the slow network speeds in our environment, it is best to split them in case the download fails, at least the progress of each zip has been deployed to save time.
Posted on 09-24-2019 12:10 PM
If you zip the whole installer is it then small enough to send? Not sure if you can break it up.
Posted on 09-24-2019 12:18 PM
If your network speeds are that bad, you should probably just make a USB installer. The other thing I've done is Airdrop the installer to other devices.
Posted on 09-24-2019 12:18 PM
@hdsreid On the off chance that @Rosko's pkgChunker could help …
Posted on 09-24-2019 04:24 PM
@dan-snelson very interesting and seems like it will work, but I think I got it working finally...
the problem was using the built in zip command from terminal. i split it using Keka instead, joined all the parts back together using cat, and the unzip terminal command handled it without a problem. this time installer didn't crash after 10 seconds with an error.
Posted on 09-24-2019 09:27 PM
Wow, a thread where I can actually mention this and it has a use case. BIt-Torrent-like technology is exactly designed for this. It is designed to distribute content in small chunks over a distributed network to help reduce bandwidth. Now, getting someone to agree to use Bit Torrent is like moving mountains because of the software's bad reputation, but it can be used for good! I tried for years to get people to let me try this as a distribution model, but no one would listen. You could setup private trackers, chunk your files, hash them, and have a private bit torrent tracker distribute packages across many seeds, and you can enable other systems to help seed the software.
There are commercial tools too, that use BT-like technology that are really just BT essentially under the hood that might help you market it better.
Posted on 09-26-2019 06:51 AM
@tlarkin LOL exactly, but you can only imagine the looks the infosec team would give if i mentioned that.....
the way we do it now isn't the most simple/straight forward, but its one of those things where its been done this way for so long that its routine to do.
Posted on 09-26-2019 09:24 AM
@hdsreid Oh trust me, I have been trying to preach BT for years, and people laughed at me. However, the few Orgs I worked with that did implement it, we saw amazing results. One example was at one site, we were only getting around 1-2 MB throughput to their China based sites. After building out a BT-like tool, we were able to change that to ~50MB throughput. Just by adding private trackers and allowing some other servers to seed.
It works, I have seen it work. It is also more secure as you can hash and chunk the files.