Script to fix OneDrive file, folder, path errors

DougE
New Contributor III

We are working on a script that will fix file naming issues for users new to OneDrive. We have some users who have all kinds of MS illegal characters in their file names and folder names. One user has 800+ such names to fix.

Rather than have them do this manually, or buy third party software we decided to write some scripts that will find all these illegal characters/names and change them. This will be a policy in Self Service that will:
1. Run a script that looks over all the file names and generates a report. 2. Make the changes. 3. Create two reports. One that lists changes made, and a second that lists changes that could not be made and why.

Some questions for those more experienced than we are (we are new to Jamf as of July):

  • Has this been done/attempted before? I don't see anything, but want to make sure we are not re-inventing the wheel or going down a rabbit hole that will ultimately not work for some reason we are missing.
  • We could share this with others, would there be interest?
  • Any suggestions on how to handle the find/replace logic? We can use legal characters, but need to be mindful that we could change an illegal name to a legal, but existing name. For example if "/" was replaced with "-" we could have example/1.doc changed to example-1.doc which may already exist?

With OneDrive being so new for Mac users I can see how many will have this issue. HOpefully this works out and could be helpful to the larger Jamf community.

27 REPLIES 27

blackholemac
Valued Contributor III

While I don't have a script, I have a wonderful Mac app that helped me to weed out the irregular characters in a user's home folder prior to the migration to OneDrive. It's called Name Mangler <https://itunes.apple.com/us/app/name-mangler-3/id603637384?mt=12>.

Check it out...if you have any questions how we used it to solve this problem feel free to email me...it's not automated, but cuts the time on character correction down to mere minutes instead of hours if one was to do it by hand.

DougE
New Contributor III

$18.99 each, thats the deal breaker with that. As we scale that gets expensive.

blackholemac
Valued Contributor III

Two things on that... we only bought two copies for the whole district to give to the guys in the field that were migrating end users.. if you did want to buy it for more than that it's half price at 20 copies if I recall correctly . There is no need to give the software to all the end-users though

blackholemac
Valued Contributor III

In Fact if you wanted to get really creative, you could map to home folder share for your entire user base on one machine run this utility on it ... it Will give you a report and then you could decide what to do .... if I had our migration to do over again I would've done that .

DougE
New Contributor III

The script is basically done now. Works great. If there is an illegal character it will remove it. If the resulting file name or folder name already exists (at that level or anywhere in the OneDrive folder I'm not sure) it will not make a change.

Two text files are created in a folder placed on the Desktop:
-a log of all errors found and the before and after name
-a log of all errors found that could not be fixed (ie if the name already exists) so the user can follow these up manually

We'll see how it works once we test it a bit more and then run it for the guy with 800+ illegal names. For that I'll copy his OneDrive folder to our test machine so we don't risk his data and can repeat the testing as needed.

BOBW
Contributor II

Hi @DougE

Any chance you want to share your script?

I have been playing round with getting this working and thought I might not have to re-invent the wheel

mark_mahabir
Valued Contributor

We're about to roll out OneDrive for Business here, and I would also love to see some example scripts if at all possible!

blackholemac
Valued Contributor III

I don't have scripts to share as I wasn't the lead engineer on this in our district, but I do have some advice that's worthwhile. If you're dealing with a mix of platforms, you will find the Mac users, especially the older ones, tend to be loosey-goosey in their file naming conventions. They tend to use special characters in their names specifically. OneDrive for Business doesn't like that and barfs when trying to sync such files.

We actually did not try to fully automate the transition but worked with each building site one by one (we have 14).

What I did for some of our more problematic Mac users with file name issues just download a product called Name Mangler to my laptop, mounted their share volume and did pattern searching/replacement of all their file names to fix issues before we migrated that user to OneDrive. You could probably do that on a more mass scale for the whole home folder share if you want to deal with a large group of users.

blackholemac
Valued Contributor III

I don't have scripts to share as I wasn't the lead engineer on this in our district, but I do have some advice that's worthwhile. If you're dealing with a mix of platforms, you will find the Mac users, especially the older ones, tend to be loosey-goosey in their file naming conventions. They tend to use special characters in their names specifically. OneDrive for Business doesn't like that and barfs when trying to sync such files.

We actually did not try to fully automate the transition but worked with each building site one by one (we have 14).

What I did for some of our more problematic Mac users with file name issues just download a product called Name Mangler to my laptop, mounted their share volume and did pattern searching/replacement of all their file names to fix issues before we migrated that user to OneDrive. You could probably do that on a more mass scale for the whole home folder share if you want to deal with a large group of users.

EricClarcks
New Contributor

"Long path tool is the very good program for error, unlock solution. Try it and solve your problem."

Cornoir
Contributor II

I have made an Applescript app that will find the illegal characters in a user selected folder and output the files/folders with illegal characters to a text file on the user's desktop. There are a few scripts on the web that make the changes but I wanted the end user to make the changes themselves.

property searchCharacters : "'[<>:|?]*'"
set folderpath to POSIX path of (choose folder with prompt "Chose a Folder")
do shell script "find " & folderpath & " -name " & searchCharacters & " > " & quoted form of POSIX path of (path to desktop as text) & "IllegalCharactersList.txt"

sanbornc
New Contributor III

Hey Doug-

Would you be willing to share your script? Im having to do the same thing. Let me know!

Thanks!

dsavageED
Contributor II

I've got a shell script which will fix illegal characters by turning them into dashes, check for trailing spaces and find long file names. There may be a better way of doing this, but the script has worked for us for a while...

You can have a look here: https://github.com/UoE-macOS/jss/blob/master/utilities-fix-file-names.sh

KyleEricson
Valued Contributor

@DougE Will you please share your script? @dsavageED I want to scan the more folders than Desktop and Downloads is that easy to add to this script?

JoshRouthier
Contributor

I believe Microsoft is/has been working on this problem: https://support.office.com/en-us/article/renaming-items-with-invalid-characters-99333564-c2ed-4e78-8.... I have not tested this yet, so YMMV.

samspade21
New Contributor

Thank you for your script examples, we have incorporated some of these suggestions into our MacOS OneDrive KFM tool: https://github.com/synapsepd/MacOS-OneDrive-KFM

teodle
Contributor

Was playing around with this earlier. Microsoft OneDrive Sync Client for Mac doesn't appear to be able to fix filenames yet. Instead it simply errors out and forces the user to find and fix all their own illegal filenames. Remember that total path length of deeply nested files might be problematic as well. I've tested several scripts that were posted here and none of them worked very well. What did work (for file and folder names with illegal characters), is the miracle of zmv. Run this at the top level of your user's OneDrive Folder and it will fix illegal characters. I have it set up to replace them with underscores. This is such a powerful command. Use it at your own risk. First time I botched the regular expression and forgot to "allow" the "." character, so the command replaced all dots with underscores (xx.jpeg became xx_jpeg) and destroyed Finder file associations....you don't want that!!
Here's the example. First you have to switch to zsh, then load zmv. When you run the command from the top level of a user's OneDrive Sync folder it will recursively remove every single illegal character and replace those characters with underscore. It works. And it works fast. I used this command for a case where user has three four, even 5 levels deep with thousands of instances of illegals. OneDrive tells them the names are illegal, but it doesn't really do a very good job of telling them where the files are past the top level. After running this command, sync errors disappeared. Now we just need wrap it up into a script to make it easy for end users to run in Self-Service and test it thoroughly. All I can say is zmv is pretty cool.

zsh

autoload zmv

4a1c9e16536944b3b82da504d21ad9a1

What is up with display text encoding on the forums? I included the screengrab because if you copy paste the snipped below, it will most certainly return a bad pattern error from zmv

```

zmv '(*/)()' '$1${2//[^A-Za-z0-9.]/_}'
```

jasonhollis
New Contributor

This is awesome, and to your point ran across 22,000 files pretty quickly on my SSD, but it replaces all of my spaces with underscores and spaces are not illegal. How do I leave the spaces there and just replace the illegal characters: " * : < > ? / | with a space?

AdamCraig
Contributor III

posted twice. deleted

AdamCraig
Contributor III

@jasonhollis

Not super familiar with zsh, and haven't tested this yet but...

#!/bin/zsh
loggedInUser=$( echo "show State:/Users/ConsoleUser" | scutil | awk '/Name :/ && ! /loginwindow/ { print $3 }' )

if [[ -z "$loggedInUser" ]] || [[  "$loggedInUser" == 'root' ]] || [[ "$loggedInUser" == "loginwindow" ]] ; then
    loggedInUser="$3"
fi
autoload zmv

oneDriveFolder="/Users/$loggedInUser/OneDrive - CompanyName"

if [[ -d "${oneDriveFolder}" ]] ; then

cd "${oneDriveFolder}"

zmv '(**/)(*)' '$1${2//[^A-Za-z0-9. ]/_}'

else

echo "OneDrive Folder not Found"

fi

nexus0000
New Contributor II

This script woks well but it doesn't remove the leading or trialing whitespace from files.

I have been working on this for a while but haven't been able to workout how to remove the white space only from the start and end of filenames.

I will update here if I work out how to do it.

Any idea would be great 🙂

nexus0000
New Contributor II

so I worked out the leading space....

zmv '(/)()' '$1${2//[^A-Za-z0-9. ]/-}' zmv '(/) ()' '$1$2'

Tailing space is still proving difficult

teodle
Contributor

I've been off jamfnation for a while, and was surprised to see so much activity on this thread. It certainly is an issue that a lot of users who want to start synching legacy data with illegal characters (well, illegal to OneDrive anyway) in files/foldernames, etc. Working within the limitations of OneDrive has been a challenge for these users.

elijahfrankinso
New Contributor

Use LongPathTool it will solve your problem of deleting and copying long path files

rcorbin
Contributor II

@strayer your script works perfectly. Thanks for that ! It would be just great if it could also do leading or ending spaces. Did you or @Heading ever figure that out ?

HenryOzsoy
New Contributor III

her you go champ

zmv '(**/)(*)' '$1${${2##[[:space:]]#}%%[[:space:]]#}'

HenryOzsoy
New Contributor III
#!/bin/zsh
###################################################################
#
# Script to check user's OneDrive folders and files
# for any illegal characters, leading or trailing spaces and
# overlong path names and to correct them to help allow smooth
# synching in OneDrive.
#
# Date: Wed 28 Jul 2021 12:08:33 AEST
# Version: 1
#
##################################################################

loggedInUser=$( ls -l /dev/console | awk '{print $3}' )
	
	if [[ -z "$loggedInUser" ]] || [[  "$loggedInUser" == 'root' ]] || [[ "$loggedInUser" == "loginwindow" ]] ; then
		loggedInUser="$3"
	fi
	autoload zmv
	
	oneDriveFolder=/Users/$loggedInUser/OneDrive\ -\ Oganisation\ Name
	
	if [[ -d "${oneDriveFolder}" ]] ; then
		
		cd "${oneDriveFolder}"
		
		zmv '(**/)(*)' '$1${2//[^A-Za-z0-9. ]/_}'
		zmv '(**/)(*)' '$1${${2##[[:space:]]#}%%[[:space:]]#}'
		
	else
		
		echo "OneDrive Folder not Found"
		
	fi
exit 0