Self-Hosting my Zotero Library

After the big Pi Server V3 upgrade, I’m now comfortable enough to migrate my Zotero library to my own server and ditch another third-party solution in favour of complete control and essentially limitless storage. You can of course support the great open source Zotero project by buying storage directly from them, which I would wholeheartedly recommend, but I have my own server and like to noodle around, so… I noodled.

Seeing as I use Cloudflare’s tunnelling system to direct site visitors to my server, I can also use tunnelling to point Zotero’s library sync feature to my own storage using the WebDav protocol. With my new found love of Docker I was able to set up a container for WebDav and allocate a folder as a dedicated Zotero cloud repository, enabling me to sync my library to my heart’s content.

The one, very minor downside is the Cloudflare limits file uploads to 100MBs each, which almost never going to be an issue for a Zotero library, unless you’re a sicko like me and store video references in your library complete with the full video as an attachment. I had a single video at 114MB that refused to sync, a super rare occurrence, but one I wanted to solve to make the library complete. Turns out this a pretty easy fix.

Temporarily syncing Zotero through a local IP address for unlimited upload size.

Once the offending attachment is synced, this problem goes away because there’s only a limit on uploads. And to do this I can simply change the file sync URL from domain to a local IP address plus assigned port, something like 192.168.1.123:XXXX (where XXXX is the port). As soon as I do this at home, all syncing flows directly to my server, without the need to use Cloudflare tunnelling. This is because I’m already on the same network as the server, and don’t need Cloudflare to tell the browser where my server is, thereby connecting me directly to my server and bypassing any arbitrary file size limits. The added bonus is that file transfers are lightning fast, which is ideal for the large file I’d be using this method for. Once my files are uploaded, I can simply change the URL back to my domain in Zotero and sync to my heart’s content from any location I wish.

This trick works because downloading files is a server to client activity, meaning Cloudflare doesn’t get involved in the conversation and therefore doesn’t impose a 100MB file limit. So I can download the video whenever I want via Zotero, on any device. I’m not gong to be modifying the video in the same way as I might annotate a PDF or book, so syncing changes to the video never happens. I just need to get the video onto my server in the first instance and it’s problem solved.

Considering I’ve only had one file that required this workaround in several years worth of references, it’s a very minor inconvenience, if that, and well worth the effort to be able to self-host my files and avoid any future costs and support issues. It’s also nice to reclaim even more of my online presence, like I have through Self-Hosting my Social Media Profile using WordPress and ActivityPub.

Next up will be my Frugal.Education site. 🤘