Uploading Apple App Store apps into JSS as an In-House app - Instructions

[bensim123]

While attending the JAMF Regional User Conference, last week, one of the discussions was related to distributing apps. I was asked to provide the workflow that we use to distribute App Store apps, as In-House apps.

As you’re probably well aware, Apple does not provide an easy way to distribute App Store apps to your iOS devices without it either A) Being free, or B) Using all of your redemption codes that have been purchased via VPP. This is troublesome when you want to regulate app updates for your devices. This is even worse when you have multitudes of devices (in our case it’s around 1500). If you’re in a boat similar to ours, you may have found yourself wanting to have a little more control over the whole process. The following steps will allow you to upload App Store apps as In-House apps in JSS.

App Prep

1. Make sure you have the latest version of Xcode installed.

2. If you want to maintain full control, each device will need to be initially set up under the same iTunes account (for this example will say user@domain.com).

3. For proper licensing reasons, you will need to be enrolled into Apple’s VPP program. Purchase the number of licenses you may need. Once purchased, you’ll be provided with your list of redemption codes.

4. Back in iTunes, using the user@domain.com account, redeem one code for each app you purchased. If there any free apps to purchase, download those as well. What we’re trying to do here is make sure that account being used is the same on the device and for the apps being purchased.

5. You should now have a few (or many) apps downloaded at located at ~/Music/iTunes/Mobile Applications. From here, I ended up copying the Mobile Applications folder, to my desktop, just in case something goes wrong in the process. It’s up to you as to which folder you want to work from, but I worked from the copied folder. To accelerated part of the process, I utilized an Automator workflow. a. Ask for Finder Items i. Prompt - Leave empty ii. Start at – Since my folder is on the desktop, Desktop was chosen iii. Type – Files, and Allow Multiple Selections is checked b. Replace Text i. Drop Down Menu – Replace Text ii. Find - .ipa in full name, and ignore case iii. Replace - .zip c. Run the workflow and select all of the apps. This will change the file extension for ipa to zip.

6. Open your folder, select all of the zipped files, and press Command + O to open all of the files. This will extract the data and create a folder for each.

7. Run the Automator workflow again, but swap .zip and .ipa. This will change the extension back on the apps.

8. Back in the folder, select all the items, and expand the folders. You’ll understand why in a second.

9. Edit the Automator workflow once again. This time, Find iTunesArtwork, and replace it with iTunesArtwork.jpg. When you run and it asks you where to look, open the folder as you have before, but since all of the folders are expanded, press Command + A to select all. This will save time instead of needing to click each individual one. It will then add the extension to the files. The iTunes Artwork file is the image file used for the icon you see on your iOS device.

10. Go to developer.apple.com. You’ll need to do these steps for each app you’ll be uploading into JSS. a. Go to the iOS provisioning portal b. Create a new App ID i. Provide the name of the app you’ll be uploading to JSS (for example Office2 HD) ii. Use Team ID iii. Enter a bundle identifier com.organization.appname c. Go to the Provisioning Section, then Distribution, and create a New Profile. i. Distribution Method – In House ii. Profile Name – Name of App iii. Check the box for Distribution Certificate iv. Select the App ID that corresponds with the on you created in step b. v. Submit, refresh, and download the provisioning profile.

JSS

1. Log into the JSS

2. Click Management, then Mobile Device Profiles. a. Upload your provisioning profiles. You’ll have to do each one individually.

3. With the profile(s) uploaded, go back to Management, and select Mobile Device App Catalog

4. Click Add App, select In-House App, and click Continue

5. In the folder that contains your apps, open the corresponding folder of the app you wish to upload. From there you should find a file titled iTunesMetadata.plist. Double click this file; it should automatically open in Xcode.

6. Enter the App name exactly as it is provided in iTunes

7. With the iTunesMetadata file open, find the line item titled softwareVersionBundleID. Copy the string information and paste it into the Bundle ID section.

8. Lood for the line item titled bundleVersion. Copy and paste this information in the Version section.

9. Deployment – This can be set to your own discretion. We set it to Make Available in Self Service.

10. Deploy as managed app, remove app when MDM profile is removed, and Prevent backup – Again, set these options to your own discretion.

11. Description – In our environment we have two iTunes accounts, one for staff devices, one for student. Since some of the apps are the same, but use different iTunes accounts, we used the Description section to help us identify which was which.

12. Icon – Upload Icon. You will need to Browse… to the folder related to the app, and select the iTunesArtwork.jpg. Click Upload Selected File

13. Click Upload App Archive – Browse and select the app you to upload, and click Upload Selected File…

14. Select the provisioning profile that you uploaded to the app.

15. Set up your Scope. Click Save

16. From here, you can go to an enrolled device and open Self Service. If the app has not been installed on the device, it will be available in the In-House Apps section. If it has already been installed, an it is an update, you’ll find it in the Updates section.

17. When it comes to updating the apps in JSS, the two items you’ll need to change are the Bundle Version, and the App Archive File. The rest can stay the same.

Some things to Note:
1. The steps for the provisioning profiles may change. I’m still doing some testing to see if one provisioning profile can be used for all apps, instead of one for each.
2. Some of my Windows skills are a tad rusty, and I’m aware that some of the steps won’t work for you if you’re doing this from a Windows machine. The majority will work, but the location of your mobile apps “should be”, My Documents/Music/iTunes/Mobile Applications. If someone has a process that works like the Automator tasks, let me know.
3. When uploading app archives to JSS, the system does not properly recognize file names such as, Office² HD.ipa. You’ll need to change the file name so that it reflects as Office2 HD.ipa. The file name won’t change what the devices see.
4. Make sure to keep track of the apps you've purchased, and how many devices you've installed them on. You should only install for the number of codes you have.

I know this is a lot of information. Hopefully you find it useful. If you have questions, don't hesitate to ask, and I'll answer them as best I can.

[john_miller]

Hey all. Just wanted to chime in on this conversation as well. There are a few considerations and caveats to the workflow that I wanted to post, hopefully to avoid any potential headaches.

First, we'll want to do some investigation into the strength, bandwidth, and "robustness" of the wireless network when deploying apps in this method. In-House apps distribute straight from the JSS side (not through the App Store) so when we click "Save," it will immediately begin installing the apps to all devices in Scope. Depending on the number of devices in scope, this could put a heavy load on our WiFi infrastructure.

Also, because we're saving these files in the database, there is potential to fill the bandwidth from the MySQL side. Again, this depends on the number of Apps that are being deployed, but we may want to make performance tweaks in order to handle the new load and distribution. 300 iOS devices all deploying 100MB of applications could take some time to distribute from the database if we haven't tweaked the database to handle that kind of bandwidth. The default install of MySQL is optimized for multiple low bandwidth connections vs a few high bandwidth connections.

We'll also want to think about support processes and how we will handle upgrading. In this method, any updates to applications associated with the organization ID will need to authenticate with the org ID credentials. If those devices are ever plugged into an OSX machine to sync, backup, or restore with iTunes, we will need those organization ID credentials. This could present some problems, depending on the deployment scenario, if end users are allowed to use the devices as personal devices and are responsible for backing up/restoring their own devices. Without those organization credentials, iTunes will wipe all the data associated with that org ID. We, as admins, would then have to get the devices, and reload that content.

Lastly, this isn't a workflow that is supported by Apple. Just something to know.

As always, post any questions to this Discussion, or contact your Account Manager with any questions.

[FastGM3]

Thanks Ben for taking the time to document this. It seems some steps have changed or been added since the demo you gave at our site. I've been having some inconsistencies with the deployment process, so hopefully some of these new additions in the steps will improve these inconsistencies.

In particular, I don't think the process you demonstrated to us included creating the Automator Workflows or going to the iOS provisioning portal.

As I'm following the App Prep steps you documented Step 5.) Building the workflow includes b. "Replace Text". Why am I not finding this specific action in the Automator "Library"? The closest match I could find was find and replace text in a word document. I don't use Automator very much, so if you can help me figure this action out, it would be gratefully appreciated. Perhaps it's an action included in a third party "Action Pack"?

Thanks,
Chuck Taylor
San Juan Unified School District

[rmanly]

@chuck The "Rename Finder Items" action will change its name based on what you choose to do in its dropdown.

Drop it in there and tell Automator you don't need to add the "Copy..." action.

Then change the dropdown at the top to "Replace Text"

[rmanly]

@john.miller

I actually read this and thought it was pretty cool. I was thinking of making a bash script to take care of all the Automator bits but....are we sure this is ok?

It seems like if it isn't against the EULA that it *should be* or will be soon...

I hate myself for thinking this way but unfortunately it seems we live in an era when we don't actually own things we are just allowed to use them from time to time.

[FastGM3]

@rmanly - Thanks for the clarification on Rename Finder Items.

@Ben and Josh,

I followed the newer procedures and unfortunately still got mixed results. On some iPads the application would work and on other iPads the application works fine. The application was purchased with a different apple id, I made sure that id was logged into the iPad before I did the push. With the ones that don't work when you click on the application, the screen just does a quick flash and then your back to the iPads page with application icons.

It appears the only thing different in your procedures, than what you showed me when you were here, is the automator stuff and the creation of the provisioning profile through iOS provisioning profile portal.

I should probably note that I am having the same mixed results without the creation of the provisioning profile. Sometimes it works and sometimes it don't.

Chuck

[bensim123]

@Chuck

So far, I've come across two solutions based on the app crashing.

1. Apps uploaded into JSS must use the same Apple ID as the Apple ID that was used when the device was originally set up. If the ID's don't match up, the app will immediately crash.

2. I can't remember the exact name that was referenced at the conference, but I believe it related to a type of time-out that can occur. If you've already verified that the ID's are the same, then you can force an install of an new app (or update an app) from the App store on the device, and the app you've installed via JSS, should start running again.

We've come across this recently on our Staff iPads vs. Student iPads. On each device, we decided to sign out of our account on the device, once it was set up. This allowed the users to sign in with their own, but we still are able to maintain control over the apps we own. From what I've found, the Staff aren't very quick in using their devices. They don't routinely download their own apps. The students, on the other hand, do. When pushing out an app to the staff, for those that have very rarely used it, the app will crash. For the staff and students that regularly using their devices (downloading new apps, or updating apps they've purchased on their own account), apps that we push out from JSS do not crash.

@John - Currently we have a little over 30 apps uploaded to JSS. Thanks for the reminder on the adjustment of the settings. For our MySQL settings, we currently have our max packet size set to 305MB and max database connections set to 200. Our Tomcat settins are currently set to:
Min Memory - 256MB
Max Memory - 4GB
Min PermGen - 64MB
Max PermGen - 128MB

Since adjusting the MySQL and Tomcat settings, we've noticed no slow down at the moment. We are currently scoping the apps for 30 users added at a time. We're looking at opening up for all devices in perhaps the next month. We do not use automatic deployment as we only have a few devices running iOS 5. We don't plan on moving to 5 until this summer. I've only done automatic deployment on 24 devices. There were slowdowns due to our current wireless infrastructure. We've decided to make the apps available via JSS, but not to auto deploy, even after we move to iOS 5, and to have all of the user manually update their apps. They are not restricted to a specific IP range to be able to update, so they could install, or update, while at home. We'll definitely be watching once we open for all of our users, as to what our server does. As it stands right now, we're running a 2.66GHz Core2Duo XServ with 8GB of RAM, running 10.6.8.

[bmollica]

Thank you for taking the time to document this!

I ran down all the steps and get stuck on the part where I need to upload an App Archive. When I point to the origional .ipa file and try to upload JSS says "Unable to upload the specified file." I tried this with Calculator HD and Temple Run.

Here is a screen shot:
http://i41.tinypic.com/9apu05.jpg

Any advice?

[FastGM3]

The issue may be related to your MySQL and Tomcat default settings. When Ben came out to our site to demo this he needed to adjust our server settings. Here are their recommended settings

For our MySQL settings, we currently have our max packet size set to 305MB and max database connections set to 200. Our Tomcat settins are currently set to:
Min Memory - 256MB
Max Memory - 4GB
Min PermGen - 64MB
Max PermGen - 128MB

Make these changes using the JSS Backup Utility application found in your server's LibraryJSSin folder then double click on JSSDatabaseUtil.jar the settings can be changed from the file menu under "Utilities"

Once you change these settings try uploading your .ipa files again, it should work.

Chuck

[bmollica]

Thank you for that info Chuck. While using the JSSDatabaseUtil.jar, my "Change MySQL Settings.." feature was greyed out. To fix I navigated to /etc and created a my.cnf file using TextWrangler and added the following code

# Example MySQL config file for very large systems.
#
# This is for a large system with memory of 1G-2G where the system runs mainly
# MySQL.
#
# MySQL programs look for option files in a set of
# locations which depend on the deployment platform.
# You can copy this option file to one of those
# locations. For information about these locations, see:
# http://dev.mysql.com/doc/mysql/en/option-files.html
#
# In this file, you can use all long options that a program supports.
# If you want to know which options a program supports, run the program
# with the "--help" option.

# The following options will be passed to all MySQL clients
[client]
#password   = your_password
port        = 3306
socket      = /tmp/mysql.sock

# Here follows entries for some specific programs

# The MySQL server
[mysqld]
max_connections=200 # Updated by JSS Database Utility
port        = 3306
socket      = /tmp/mysql.sock
skip-external-locking
key_buffer_size = 384M
max_allowed_packet=1024M # Updated by JSS Database Utility
table_open_cache = 512
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size = 32M
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 8

# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (via the "enable-named-pipe" option) will render mysqld useless!
#
#skip-networking

# Replication Master Server (default)
# binary logging is required for replication
#log-bin=mysql-bin

# required unique id between 1 and 2^32 - 1
# defaults to 1 if master-host is not set
# but will not function as a master if omitted
server-id   = 1

# Replication Slave (comment out master section to use this)
#
# To configure this host as a replication slave, you can choose between
# two methods :
#
# 1) Use the CHANGE MASTER TO command (fully described in our manual) -
#    the syntax is:
#
#    CHANGE MASTER TO MASTER_HOST=<host>, MASTER_PORT=<port>,
#    MASTER_USER=<user>, MASTER_PASSWORD=<password> ;
#
#    where you replace <host>, <user>, <password> by quoted strings and
#    <port> by the master's port number (3306 by default).
#
#    Example:
#
#    CHANGE MASTER TO MASTER_HOST='125.564.12.1', MASTER_PORT=3306,
#    MASTER_USER='joe', MASTER_PASSWORD='secret';
#
# OR
#
# 2) Set the variables below. However, in case you choose this method, then
#    start replication for the first time (even unsuccessfully, for example
#    if you mistyped the password in master-password and the slave fails to
#    connect), the slave will create a master.info file, and any later
#    change in this file to the variables' values below will be ignored and
#    overridden by the content of the master.info file, unless you shutdown
#    the slave server, delete master.info and restart the slaver server.
#    For that reason, you may want to leave the lines below untouched
#    (commented) and instead use CHANGE MASTER TO (see above)
#
# required unique id between 2 and 2^32 - 1
# (and different from the master)
# defaults to 2 if master-host is set
# but will not function as a slave if omitted
#server-id       = 2
#
# The replication master for this slave - required
#master-host     =   <hostname>
#
# The username the slave will use for authentication when connecting
# to the master - required
#master-user     =   <username>
#
# The password the slave will authenticate with when connecting to
# the master - required
#master-password =   <password>
#
# The port the master is listening on.
# optional - defaults to 3306
#master-port     =  <port>
#
# binary logging - not required for slaves, but recommended
#log-bin=mysql-bin
#
# binary logging format - mixed recommended
#binlog_format=mixed

# Uncomment the following if you are using InnoDB tables
#innodb_data_home_dir = /usr/local/mysql/data
#innodb_data_file_path = ibdata1:2000M;ibdata2:10M:autoextend
#innodb_log_group_home_dir = /usr/local/mysql/data
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
#innodb_buffer_pool_size = 384M
#innodb_additional_mem_pool_size = 20M
# Set .._log_file_size to 25 % of buffer pool size
#innodb_log_file_size = 100M
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 50

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[myisamchk]
key_buffer_size = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

Now that that file is there I am able to go into and edit MySql Settings as I need.

The Tomcat settings were straigt forward with no issues. I WAS able to get the app loaded into self service and no password is required when installing in-house apps. Awesome. Sorry for the lengthy post, but I figure it might help anyone who runs into the same problem I did.

[bensim123]

Here's a bit of an update on my end of things.

1. I've discovered that it is NOT necessary to use a provisioning profile with App Store apps uploaded in to JSS. This was found out in a round-a-bout way when I noticed that every time I made a change to an app (newly added, uploaded new version, changed the scope, etc.), JSS would do a call-out to all of our devices, remove all of the provisioning profiles from them, and then re-install the profiles. Needless to say, this can put quite a strain on the server. Through some testing, it was discovered that the provisioning profile is not needed, so I've now removed all provisioning profiles from the system. It does seem strange that this work. Myself, and others I've talked to, have always been under the impression that all in-house apps needed to have a provisioning profile associated with it. Whether it is something that has been changed by Apple, or something else, no one is entirely sure, but either way, it is what it is.

2. We had an issue with our teacher accounts where the apps would crash upon updating. To get the apps to run again, we had to go in to the app store and do a manual update of an app that was associated with our staff iTunes account. After removing the provisions profiles (as explained in #1), this is no longer an issue.

3. A bit of a warning at this point: We are in a situation that we have hit a bit of a limit of the capabilities of the hardware of our server. We are currently at a little over 100 apps uploaded in JSS and are managing about 1500 devices. Our poor little server is overtaxed right now.

So while, yes, the process I've listed out does indeed work, you may be limited by the hardware and potentially the limitations of your SQL database. The reason I mention the SQL database is due to that the in-house apps are stored in the database. Since we have so many apps, the amount of resources used by mysql, is quite large. Things were running fairly smoothly when we had uploaded only about 40 apps and had it scoped to only allow about 200 devices to be able to access them. I'm currently working with the good folks at JAMF Software to figure out a solution to stabilize our server.

[john_miller]

Wanted to chime in with this with some more thoughts on the workflow, and hopefully help with some ideas in planning.

What this workflow gains admins is the ability to install an application without authenticating to an Apple ID. This is done through some of the technology on the iOS side for how the Apple ID is saved on the devices themselves, but at the expense of the resources that are allocated for other functionality.

There are some significant things to think about and be aware of when considering this workflow. There's been the good discussion so far on the technical side and what needs to happen, technically, to make this happen. We also want to think through the more "soft" processes, like support, admin time, long term planning, the implications of usage to our end users with this method, etc.

At the end of the day, this is not a supported workflow from a JAMF perspective, is not supported from an Apple perspective (EULA or otherwise), and can cause some serious headaches long term.

This still leaves the workflow problems for how to get apps installed to devices, and what potential workflows are available to make this process easier. Please reach out to your Account Team to go over these workflows. Configurator provides a huge benefit in taking care of this, and we have a lot of documentation with 8.5 that walks through the process. We're happy to help walk through it.

[nsdjoe]

I have been successful uploading most Apple App Store .ipa files into the JSS (step 13 under "JSS" above). However, there are a handful of apps where I get an error when trying to upload the app archive file. The error reads:

"Unable to upload the specified file."

Has anyone else had this problem? Anyone found a way around it?

Thanks!!
~Joe

[khurram]

Hi @bensim123,
What I understood from this scenario (correct me if I am wrong) is that you are using xcode to convert the App Sotre apps to in-house apps. Also what would be the way of distributing these with VPP codes. To me it seems that even if you managed to deploy the App store apps in-house then the VPP codes are still not associated with apps.