I Built My Own Little Search Engine for Images
A few days ago, I wrote about collecting images. Not necessarily photographs we take ourselves, but all those pictures, screenshots, illustrations, diagrams and other visual fragments we come across while wandering around the internet. Some end up in moodboards, others in carefully organised folders, and quite a few simply disappear somewhere on our computers. In...
few days ago, I wrote about collecting images. Not necessarily photographs we take ourselves, but all those pictures, screenshots, illustrations, diagrams and other visual fragments we come across while wandering around the internet. Some end up in moodboards, others in carefully organised folders, and quite a few simply disappear somewhere on our computers.
In that earlier post, I explored the relationship between collecting images and making zines, and how screenshots have become an interesting way of preserving little pieces of the digital world. I also mentioned Screenshot Conf and some of the tools people have developed to work with screenshots. The more I thought about all this, the more I realised that collecting images is only half the story. The other half is finding them again.
I have accumulated a considerable number of images over the years. Some are photographs, others are illustrations or screenshots. Many are related to my work as a technology journalist, while others were simply interesting enough to save. They are stored on my own SSD, spread across different folders. Some are reasonably well organised, but others have filenames that tell me very little about their contents.
Of course, I can search for filenames. But a photograph called IMG_4837.jpg could contain a server room, a street in Berlin or a particularly interesting piece of industrial machinery. What I really wanted was something different. I wanted to type a person standing in a data centre and see photographs that might match that description. Or perhaps select an existing photograph and ask my computer to find other images that look similar. Preferably without uploading my entire collection to somebody else’s cloud.
An interesting project, but the wrong operating system
While exploring possible solutions, I came across an open-source project called FrameScout (https://github.com/bobgsning/FrameScout). It uses AI to search visual content by meaning rather than relying exclusively on filenames and metadata. It was close to what I had in mind, but there was one small problem: its available releases were intended for Windows, while my everyday laptop runs Linux Mint.
I could have tried to compile FrameScout for Linux. Its underlying technologies are not inherently Windows-specific, so that would have been a perfectly reasonable experiment. But I wondered whether it would be easier to build something similar specifically for my own computer. I had no intention of turning this into a commercial product, and I certainly did not need an elaborate desktop application with installers and support for different operating systems. A simple tool that could search my own files would be quite enough.
So I asked Claude whether we could build one. We decided to use FrameScout as inspiration, but create a considerably simpler application using Python and a local web interface. The result is called Beeldvinder, which is Dutch for something like Image Finder.
Meet Beeldvinder
Beeldvinder runs on my Linux Mint laptop and uses a web interface that I open in my normal browser. I visit http://127.0.0.1:5810, and there is my personal image search engine. Although the interface looks like a website, everything runs on my own machine. The browser is simply a convenient way of interacting with the application.
The first search method uses words. I can enter a description such as a person working in a server room, and Beeldvinder searches for images that appear to match that description. This is different from searching for a filename containing the words server or room. The application tries to identify visual content that corresponds to what I am looking for, even when the filename contains no useful information.
The second method is perhaps even more interesting. I can provide an existing image and ask Beeldvinder to find visually similar pictures in my collection. This could be useful when looking for a particular composition, type of illustration, subject or visual atmosphere. Imagine having saved a photograph of an unusual industrial building and remembering that you collected several similar images over the years. Instead of opening folders one by one, you give Beeldvinder the photograph you already have and let it look for possible matches.
The application also includes several additional capabilities. It can recognise text inside images using OCR, which is particularly useful for screenshots. I can attach searchable notes to photographs, and Beeldvinder can examine frames from video files as well. There is also a folder-monitoring function that notices newly added images and incorporates them into the index.
These features make Beeldvinder more than a search box for a directory of photographs. It is becoming a way to explore a personal collection of visual material without having to organise every image in advance.
How does it recognise images?
Under the surface, Beeldvinder uses an AI model called SigLIP 2. This model can represent images and text as numerical vectors, often referred to as embeddings. Images and descriptions with related meanings can be represented in ways that make them comparable, even though one consists of pixels and the other of words.
When Beeldvinder indexes an image, it calculates such a representation and stores it locally. When I enter a description, the application converts my search into a vector as well. It then compares that description with the indexed images and ranks possible matches. Searching with an example photograph works in a similar way: the application compares the representation of the example with those of the images already in its database.
This does not mean that Beeldvinder understands a photograph in the way a person does. Nor does it guarantee that every result will be relevant. The system calculates similarities, and its accuracy depends on the model, the available images and the wording of the search. Nevertheless, it makes searching possible without relying entirely on filenames, manually assigned tags or carefully maintained folder structures.
The technical setup is relatively modest. Beeldvinder uses Python, Flask, SQLite, NumPy and ONNX Runtime, together with several supporting libraries. The AI model runs locally, so there is no need to send the images to an external AI service for analysis. The initial model download is substantial, and indexing thousands of images on a laptop without a dedicated GPU takes time. Once the collection has been indexed, however, searches can be performed much more quickly.
My first search was not particularly impressive
Naturally, I wanted to try Beeldvinder immediately. One of my first searches involved people in datacenters. Since I write quite a lot about datacenters, I have collected numerous relevant photographs over the years. I entered a description along the lines of man or woman in a data center and waited for the results.
They were not exactly what I expected. Instead of suitable photographs, Beeldvinder showed several rather unrelated images, including pictures of robots. I showed Claude a screenshot and asked what was going wrong. It turned out that several things were happening at once.
First, I had been a little impatient. Beeldvinder was still indexing my image collection. At that point, it had processed only 161 of approximately 26,500 files. The search engine was trying to find the best matches in a tiny fraction of the available material. With so few images to choose from, the supposedly best results were not necessarily relevant at all.
There was also a bug in the search code. Beeldvinder gave additional weight to filenames containing words from my search query. Unfortunately, it was matching short fragments of words rather than complete words. That meant ordinary pieces of filenames could receive a surprisingly large relevance bonus. The word in, for example, could match part of LinkedIn or Afbeeldingen (the Dutch word for images). An unrelated photograph could therefore appear highly relevant for entirely the wrong reason.
Claude corrected the matching logic, excluded common short words and added a test based on my actual search. It also suggested something I had not considered: the AI model might respond better to a simpler description, preferably in English. Instead of asking for a man or woman in a data center, I could search separately for a man in a data center, a woman in a data center or a person in a server room.
The combination of incomplete indexing, an unfortunate scoring bug and a rather imprecise search query explained why my first experiment had been so disappointing. Fortunately, the problems were relatively straightforward to address.
The application that started searching itself
Then something rather amusing happened. I had initially told Beeldvinder to index my entire home directory. That seemed convenient. Why bother selecting individual folders when the application could simply search everything?
Unfortunately, Beeldvinder also stored its own thumbnails and database files somewhere inside that home directory. Every time the application discovered an image, it created a small thumbnail to display in the search results. Its folder-monitoring system then noticed that thumbnail and treated it as another image to index. This produced another thumbnail, which was subsequently discovered and indexed again.
In other words, my image search engine had started feeding itself its own images. The process continued until the Linux file-monitoring system complained because its event queue was overflowing. I sent the error messages to Claude, which identified the loop and modified the application to exclude its own data directory. It also adjusted the folder watcher to ignore hidden directories and remove previously indexed thumbnails created by Beeldvinder itself.
There is something wonderfully absurd about a tool designed to help me explore my visual collection becoming so enthusiastic that it starts collecting its own output. It was also a good illustration of the difference between testing software with a small collection of example images and letting it loose on tens of thousands of real files.
After that little adventure, I decided that pointing Beeldvinder at my entire home directory might not have been the cleverest idea. Selecting the folders that actually contain my image collections is a much more sensible approach.
Why I prefer keeping this local
One of my requirements was that Beeldvinder should work with the images already stored on my SSD. I did not want to upload thousands of photographs, screenshots and illustrations to an online service simply to make them searchable.
Some images relate to professional projects. Others are personal. And many are simply things I have collected over the years without giving much thought to where they might eventually end up. I would rather not send all of that material to a remote service when the same basic functionality can run on my own computer.
Using a local AI model means that the indexing and search process can take place entirely on my laptop. Once the software and models have been downloaded, Beeldvinder does not need a remote AI service to perform its basic functions. My images remain where they already are, and I can continue using my existing folders rather than moving everything into another platform.
Naturally, there are trade-offs. Local models consume memory and processing power, and indexing a large collection can be slow. A laptop CPU is not going to match the performance of a powerful workstation with a dedicated GPU. The quality of the search results also depends on the capabilities of the model being used.
But for a personal application, those limitations are acceptable. I am not trying to build the world’s best image search engine. I am trying to make my own collection more useful. And to be honest, given my laptop’s rather modest specifications, Beeldvinder is remarkably fast at finding relevant images.
From collecting images to rediscovering them
What I find most interesting about Beeldvinder is how naturally it follows from my earlier experiments with collecting digital material. A moodboard is usually a deliberate selection of images that belong together in some way. Perhaps they share a colour, a subject, a style or simply a feeling. A personal image archive is often much less organised. It grows gradually, sometimes over many years, as we save things that catch our attention without necessarily knowing what we will eventually do with them.
Traditional folders force us to make organisational decisions in advance. Is an image about architecture, technology, photography or design? What happens when it belongs in several categories? Tags can help, but assigning them takes time, and the categories we invent today may not be the ones we find useful five years from now.
Semantic search offers another possibility. Instead of deciding exactly where every image belongs, we can let the collection remain somewhat untidy and search it through descriptions and examples. That does not eliminate the usefulness of folders, tags or notes, but it gives us another way of navigating our own material.
I particularly like the idea of using an existing photograph as a starting point. Not necessarily because I want to find an identical image, but because I want to discover what else is hiding in my collection. Perhaps a screenshot saved three years ago has something in common with an illustration I downloaded last week. Perhaps two photographs from completely different projects share a visual quality I had never noticed.
Whether Beeldvinder will actually reveal those unexpected connections remains to be seen. That is one of the things I want to experiment with as the collection becomes fully indexed. But the possibility is intriguing. An image archive might become something more than a place where photographs wait to be retrieved. It could become a collection that invites exploration.
A small tool for a very personal problem
Beeldvinder is still an experiment, and I expect to discover more things that need improving as I use it. But I already like the basic idea. I came across an interesting open-source application, realised it did not quite fit my environment and decided to build something that did.
The result is a local image search application that works with my existing files, understands descriptive searches, accepts example photographs and can even recognise text in screenshots. It does not require a cloud subscription, and it does not oblige me to reorganise my entire collection before I can use it.
Perhaps the most enjoyable part is that Beeldvinder addresses a problem I had never considered worth solving with a dedicated software project. I had simply accepted that finding old images was sometimes difficult. That was part of having a large collection of photographs and screenshots.
Did you enjoy this article?
Recommend it — Standard Reader surfaces well-loved writing to more readers across the network.