Exporting Firefox bookmarks via script

[AVmcclint]

Does anyone know if the Firefox bookmarks can be exported by running a shell script? I found the manual instructions for doing it via the GUI. We've been having some troubles with version 54 and 55 when dealing with root certs, and I've found that the easiest way to fix it is to delete the user's profile, then delete and reinstall Firefox. Unfortunately I had an unfortunate incident where someone's bookmarks were lost. My goal is a Self Service icon users can use to either just backup their bookmarks or a combo backup their bookmarks with a total delete & reinstall of Firefox.

[AVmcclint]

I just realized that the "Backup" option only saves as a .json file. I'd prefer that the file be saved as the Export option of HTML.

[brock_walters]

Hey There!

The plutil binary should allow you to convert properly formatted json to xml & back even if they are non-Apple files.

I just pulled this random json example from the interwebz and copied it into a text file:

{
    "glossary": {
        "title": "example glossary",
        "GlossDiv": {
            "title": "S",
            "GlossList": {
                "GlossEntry": {
                    "ID": "SGML",
                    "SortAs": "SGML",
                    "GlossTerm": "Standard Generalized Markup Language",
                    "Acronym": "SGML",
                    "Abbrev": "ISO 8879:1986",
                    "GlossDef": {
                        "para": "A meta-markup language, used to create markup languages such as DocBook.",
                        "GlossSeeAlso": ["GML", "XML"]
                    },
                    "GlossSee": "markup"
                }
            }
        }
    }
}

then converted to xml with plutil:

plutil -convert xml1 ~/Desktop/file.txt

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>glossary</key>
    <dict>
        <key>GlossDiv</key>
        <dict>
            <key>GlossList</key>
            <dict>
                <key>GlossEntry</key>
                <dict>
                    <key>Abbrev</key>
                    <string>ISO 8879:1986</string>
                    <key>Acronym</key>
                    <string>SGML</string>
                    <key>GlossDef</key>
                    <dict>
                        <key>GlossSeeAlso</key>
                        <array>
                            <string>GML</string>
                            <string>XML</string>
                        </array>
                        <key>para</key>
                        <string>A meta-markup language, used to create markup languages such as DocBook.</string>
                    </dict>
                    <key>GlossSee</key>
                    <string>markup</string>
                    <key>GlossTerm</key>
                    <string>Standard Generalized Markup Language</string>
                    <key>ID</key>
                    <string>SGML</string>
                    <key>SortAs</key>
                    <string>SGML</string>
                </dict>
            </dict>
            <key>title</key>
            <string>S</string>
        </dict>
        <key>title</key>
        <string>example glossary</string>
    </dict>
</dict>
</plist>

It is possible that the plutil conversion won't work for what you are trying to do, but, if it doesn't, there are a bunch of other linter / conversion type tools out there for doing this type of thing.

Good luck!

[AVmcclint]

If I understand your script, it just converts existing files. The main issue i have is that I need to export the bookmarks out of Firefox via script in the first place. :-

[mm2270]

As I understand from your post, the issue isn't re-installing Firefox, but blowing away the FF user profile, which contains all kinds of stuff that doesn't get repopulated after a re-install.

If so, I took a quick look and it seems the Firefox bookmarks are stored within the user profile in a sqlite file called "places.sqlite" In my quick, informal test, it was enough to simply back up that file to another location before deleting Firefox and re-installing it. Then placing that "places.sqlite" file back into a newly created profile directory.

The one issue I'm certain of is that you have to launch Firefox to create a new profile, since they get randomly named, and then script placing the places file back into it, and then quitting, relaunching Firefox. It should be possible to do all of the above in a Self Service policy. Like, have a Before script run that checks if Firefox is running and quit it if it is, then locate the profile folder for the user running the policy and backup that places.sqlite file somewhere secure on disk, then finally delete Firefox. Then have the main portion of the policy run that re-installs the app, and finally an After script run that opens Firefox in the background just to create the new profile folder, quit it, copy the places.sqlite file back in and then launch Firefox again (if desired)

[AVmcclint]

The only part of the process I can't figure out is scripting the exporting of bookmarks. I have not had any luck with just copying the places.sqlite file into the newly created profile. It's like it doesn't read any data in the file. Yes i've checked permissions, yes i've verified that the bookmarks actually DO exist before copying the file. Manually exporting as HTML and then re-importing that after the new profile is created works every time. Scripting that bit would just make my life easier. And having an HTML backup is more versatile. If it can't be done, then it can't be done and I can live with that.

Just as an FYI, I do have to reinstall the FF app because we have to include extra bits into the .app itself as well as into the profile to placate our InfoSec. It is an unnecessarily complicated action, but one that I am compelled to adhere to. But none of that has any impact on the need to export the bookmarks.

[mm2270]

@AVmcclint What version of Firefox? I'm running the latest version and as I mentioned, copying the backed up places.sqlite file back into the new profile folder and quitting/relaunching Firefox is working fine for me. I verified on first launch that my bookmarks were not in the bookmarks menu, then quit FF, copied the places.sqlite file in, launched FF again and my bookmarks were there under the menu.

I'm not sure why that wouldn't be working for you. The only other thing I did was also delete the "profiles.ini" file that sits at the top of the ~/Library/Application Support/Firefox/ directory, since it appears that ini tells Firefox what profile folder to look in. Maybe the old one is still there and the app is getting confused?

As for scripting copying those bookmarks out, given it's contained in an sqlite file, I'm not sure how much success you will have with that, Maybe an sqlite command can read and copy it's contents out. I had a look at the file with SQLLite Browser and it's not the easiest thing to navigate to find those bookmarks. I don't know enough sql syntax to tell you what to even try with that.

[brock_walters]

Hi guys -

So I made a little progress on this. I don't have a script to share but if I get some time (which I don't really have...) :) I will post something. The good news: I have 2 POC ideas for exporting Firefox Bookmarks in HTML. The 1st is kind of hard but would be worth the effort if you needed to customize them, the 2nd is is pretty straightforward & easy.

1) You can read the tables in the places.sqlite database with a command like the following:

sqlite3 -header -column ~/Library/Application Support/Firefox/Profiles/########.default/places.sqlite 'select * from moz_bookmarks'

The ######## will have to be replaced with the name of the Firefox Profile directory.

What you should see if you enter the command correctly is a nicely columnized version of the Bookmarks data. What you could do, if you exported all the Bookmarks data from the different tables (Folders, Tags, History, URL, etc...) is combine it into whatever format was needed. If what you want is some sort of custom HTML file an .xslt / Xpath style sheet would probably be the way to go.

2) But there's an easier way:

Firefox.app has a preferences file that can be modified by typing "about:config" into the Firefox search bar.

Double-clicking on the 'browser.bookmarks.autoExportHTML' line enables the creation of an automatic HTML backup file of all bookmarks every time Firefox shuts down:

~/Library/Application Support/Firefox/Profiles/########.default/bookmarks.html

Instead of doing this in the GUI you can automatically enable HTML backup file creation by simply modifying the file where Firefox stores these preferences:

~/Library/Application Support/Firefox/Profiles/########.default/prefs.js

Add the following line to the prefs.js file:

user_pref("browser.bookmarks.autoExportHTML", true);

In a script written to automatically generate & grab the HTML file I believe you would have to kill the Firefox.app process because that is what prompts the HTML file to be created.

@AVmcclint Hope this helps & let me know if you have questions. I will try to come back to this one!

[mm2270]

@brock.walters Cool. Option 2 seems like the most logical approach to this. While the sqlite command is neat and good to know, the output isn't terribly useful. I see my bookmark titles, but the URLs themselves appear to be obscured in some encoded fashion. I assume it's the "guid" column since most of the others look like date strings or some other flags. So it might have limited usefulness.

[brock_walters]

The URL are actually in a separate table from the example i gave. You will definitely have to do a lot of pushing & pulling to get the data out of sqlite in a useful state. Adding bookmarks to the places.sqlite database though is actually very easy & might even obviate the need to recreate a user - could just use scripts or policies to inject data into the database directly.