Algorithmic filmmaking. Tie that term to your ears, ladies and gentlemen. And then take a look at this piece of work, and try to guess what you see.
Is this the trailer for a romantic adventure film? An editor's fever dream? A surreal experiment from film's primeval days? Coincidental art? A practical joke? The future of cinema? So can't those algorithms, those creepy computer programmes that threaten to turn our world into a kind of Matrix get away with anything?
Bits & Pieces
On Monday 28 May 2018, Jan Bot selected the film you just saw as the best he had made that day. Jan Bot is a new staff member at EYE Film Museum. He is working day and night on Bits & Pieces, the collection of unidentified images from the film archive. Jan Bot is a robot. For his favourite of 28 May, he was inspired by the Roland Garros tennis tournament. By the looks of it, he found no tennis players among those old film snippets, but he did not let that put him off.
Jan Bot at work
Jan Bot has a own website which went live last Saturday. There, he not only shows the films he has made. He also explains his work. See the same video about Roland Garros, now with his comments. I admit, it remains pretty cryptic. He also has a livelog Where you can see him at work.
"Hello, let's see what's happening on the internet today," he says, when I snap at him early in the morning while writing this piece.
From film heritage to experimentation
This says EYE's press release: "Eye Filmmuseum and Modern Times Foundation present: Jan Bot. This bot brings film heritage to the age of the algorithm. A computer programme that assembles experimental films day and night in which early 20th-century footage is linked to current themes on social media."
"How do you bring a collection of old films back to life? Film archives spend a lot of time restoring old films, but how do you make sure all that unique material reaches an audience? Eye Filmmuseum is working with filmmakers Bram Loogman and Pablo Núñez Palma on a possible solution. The result: Jan Bot."
Pablo and Bram
These are Pablo Núñez Palma and Bram Loogman.
They met at the Film Academy's master's programme, the advanced course that emphasises research and experimentation. They found each other in their interest in algorithmic filmmaking and together built Jan Bot. A kind of 'black box' that goes its own inscrutable way, though occasionally adjusted by Bram and Pablo. "We don't want it to be about football all the time," Pablo said in a telephone explanation.
Algorithms
In the metalog by Jan Bot, Pablo explains what algorithms are.
This is less creepy than you may have thought. Even humans in ancient times who had a method of making flint axes were already following an algorithm. An algorithm is a step-by-step process to achieve a goal. So not just the digital wizardry with which social media keeps us in our filter bubble, but also the everyday routine of the shopping list.
Even without computers, artists were already making algorithmic films. See Arnulf Rainer by Peter Kubelka, or Critical Mass by Hollis Frampton. By comparison, Jan Bot's work is very well behaved, even if the connections he makes are usually quite unexpected. This is what he (is Bot actually a he?) was inspired to do by Kate Middleton.
Artificial intelligence
Back to the press release: "Jan Bot is a computer programme designed to generate short experimental films based on two components: the old footage in Eye's collection and current themes on social media. On average, Jan Bot streams ten 30-second films a day on its own website, which already contains more than 7,000 items. Every day, Jan Bot selects a short film to show on his Facebook and Instagram profile mail."
"To deliver this huge amount of work, Jan Bot uses artificial intelligence services found on the web by its creators. "Many large companies, such as Google and IBM, offer tools that allow you to recognise images and analyse language, some even for free. So we grabbed a few and glued them together to make movies with them," says Loogman, one of the creative brains behind Jan Bot."
Now just step by step
Step 1. It starts with image recognition. Jan looked at all the material in EYE's Bits & Pieces collection and provided each image with a series of keywords, or 'tags'. Image recognition goes a long way these days, even if Jan Bot still sometimes can't tell the difference between a cat and a monkey.
I was also reminded (forgive the side-step) here briefly of the Wordcamera I got to play with at IDFA a few years ago. Which turns random photos into an experimental poem. Try it yourself on word.camera.
In between, take a look at how far along Jan Bot is. His first video today (30 May) is inspired by computer company Bethesda And the Fallout game rumour. So there must have been a lot of tweeting and googling about that. Jan has translated that into a rather spooky compilation. Not bad.
The black box as a magic pot
Step 2: what is trending on the internet? Jan Bot gets data on keywords, among other things, looks at which topics are currently being widely discussed in social media and collects news items related to them. Based on all that, he makes a list of the keywords that rank highest at that moment.
Incidentally, Jan does not just blow with all the winds. A topic has to be mentioned online often enough. If there are too few hits, he says: "Hmm, I can't find anything interesting. Let's wait a bit and try again." This morning, for instance, he has been busy for 10 hours and has not yet found a second topic.
Step 3. Using his text analysis programmes (and Wikipedia), he searches for meanings behind those keywords. After all, the word 'red' indicates not only a colour, but also a political direction. He tries to link those meanings to old film footage via the tags of the Bits & Pieces collection. The news reports provide material for the intertitles he mounts in his films.
Step 4. All that raw material goes into the 'black box', the heart of the software Jan Bot works with. That's where the final composition takes place. Although Pablo and Bram have filled this black box with algorithms themselves, what Jan Bot comes up with remains a surprise.
Incidentally, I can see he has a way with this: he often chooses two or three images that he nervously cuts through, only to jump to a according set again.
According to Pablo, this is not just Jan Bot's own choice. His creators may well provide some guidance, so in the future he will also show other ways of working.
The old stories have fallen by the wayside
For those who still find it all very bizarre: consider that Jan Bot is fed by the internet world. The logic of the old media like films and newspapers has given way. Continuity has given way to chance and randomness. We scan and zap, with behind the scenes the invisible hand of social media's algorithms. What we see on our timeline is a whimsical collection that we have to make our own chocolate out of.
Is Dada back?
See also the article By Pablo on how poets can make words collide.
Is this how we should look at Jan Bot's experimental work? I don't exactly know yet. However, I did suddenly have to think of the unsurpassed Dadaist Tristan Tzara, who in 1920 published the recipe for making a Dada poem. I don't want to keep that from you (translation Peter Nijmeijer/Sjoerd Kuyper):
"Take a newspaper.
Take a pair of scissors.
Choose an article from the newspaper that is about the length you want to give to the poem.
Cut out the article.
Then cut out all the words in the article and put them in a top hat.
Shake gently.
One by one, pull the cutouts out of the hat.
Carefully transcribe the words in the order they came out of the hat.
The poem will look like you.
And you will see, you are an incredibly original writer with feeling, though misunderstood by the people."
Fun
"Is it fun?", I finally ask Pablo. He laughs. "Yes, Jan Bot is a lot of fun, although until now it was mostly very hard work to put the programmes and the website together. But now that it is running we can really start having fun with it. Now we can start experimenting."
English (UK) 
Nederlands 
Français 
