The Journey of Creating “the Creativity Machine” and the Next Step.
The Chinese version of this post is here.
Table of contents
Background: a personal story
If you are not interested in hearing my personal rumbling about my complicated feeling towards Swiss Design, and just want to see some technical breakdown of the project, you can skip to the next section.
As a college-trained graphic designer practicing design for years, I have grown some very complicated feelings towards one specific design style, or design aesthetics — the International Typographic Style. You might not be familiar with the term, but you must know the famous Helvetica, and the term Swiss Design. On paper, it is just one popular design style, but in reality, its influence has long overflown out of the life of professional designers and becoming a cultural symbol. As an article from The Designest said
“…A model of artistic honesty, pure good taste…“.
First, I have to admit, Swiss Design is truly beautiful, I myself enjoy it very much, sometimes I even wish the world can have more of it. But, I often witness the elitism leaking out of the designers who have gone to a design school or even gotten their expensive MFA, who can execute a perfect Swiss Design. On the other side, it is the hyped idolization towards Swiss Design from the general public. In the years of being a designer, this bothers me, a lot. I didn’t do anything about it though, till I started to code.
Generating something algorithmically is the process of deconstructing it. It is a brutal process that dispels the magic on it. There is nothing better for me than to deconstruct the synonym of “pure good taste” – the Swiss Design.
I am always iterating the project to make it better, till I got over my childish feelings about all those, this series will not end.
Preparation
The first project in the series “The Creativity Machine” was done before the rise of TensorFlow. I hand-wrote an algorithm to generate Swiss design posters using Processing and made it into an installation.
Writing the core code of “The Creativity Machine” took me about two weeks. It has more than 14 classes(which is a lot for a Processing project), with hundreds of if statements (yes, but that is the way I went.) But the process was a lot of fun — it is a constant balance act between complex code architecture and graphic design principles.
Before opening the code editor, the first step is to observe and abstract, like a typical designer. I picked some textbook Swiss design posters online, printed them out, pin them on the wall of my living room. I dragged ZZ to sit down with me and tried to discover some universal patterns that could be programmed. Turned out, it is not difficult at all. Of course, there are some outcasts, well, we got to start from somewhere.
Algorithm Breakdown
Here is the basic code structure. Let me break it down and introduce each section(class) because it is a very, very interesting project.
ColorDesigner
Swiss design often have a very distinguished color scheme, since it is highly geometrical and flat, we don’t need a lot of colors on one poster. While I was getting ready to manually scrap colors from some sample design, i found a site that does exact this.
I got the data from this site and input them into a class ColorScheme. This class picks a set of color from all the sets and hands it to ColorDesigner at the beginning of each design process. Then the ColorDesigner will decide which color will be the foreground and which color will be the background. There are some simple rules to make sure the text will always be more pronounced than the graphics.
TypeDesigner
The TypeDesigner has three tasks: it needs to generate some content for the text — we cannot just display gibberish; it needs to decide which font or font combination to use, and it needs to decide how to arrange the text, facilitate the positioning of headlines and body text.
We have two types of content, the headline and the body text.
Content Generator
Body Text
For the body text, I found a brilliant tool called the Postmodernism Generator.
The Postmodernism Generator is a computer program that automatically produces imitations of postmodernist writing. It was written in 1996 by Andrew C. Bulhak of Monash University using the Dada Engine, a system for generating random text from recursive grammars. A free version is also hosted online. The essays are produced from a formal grammar defined by a recursive transition network. It was mentioned by biologist Richard Dawkins in the conclusion to his article “Postmodernism Disrobed”1 (1998) for the scientific journal Nature, reprinted in his book A Devil’s Chaplain (2004).2
The generator produces fantastic output and it really captures the essence of our little satirical project. Here is a small sample of its creation. (it even contains citations.)
I wrote an algorithm that automatically fetches some content from its generated essay, every-time it will return us some different, equally marvelous fake writings.
Headline
The headline is an entirely different animal. It needs to be relatively short, and punchy. It needs to be able to invoke something, imply something. The long terms from the essay generator won’t work as a good headline. After some exploration, I suddenly recalled the dark days of me sitting in my bedroom 24/7 memorizing TOEFL vocabulary. TOEFL words sit in the perfect space for our headlines — they are not too colloquial, yet not too obscure.
Font Picker
The choice of font is rather simple, I picked a few popular fonts in Swiss Design, and just hand-coded a simple array, and gave each of them a probability. Here are some of my choices.
GraphicDesigner
Here comes the graphics behind the typography. After studying many samples of Swiss Design, we came up with 2 types of shapes with 3 types of arrangements.
If you look closer, you will find out that we can further reduce the number of types in Arrangement. For example, the “offset repeat overlay” and “center overlay” can be combined as just “overlay”, and use their offset value as a parameter – when offset is 0, we get “center overlay”, when offset is none-zero, we can offset repeat overlay. The number of shapes can also be a single variable when the number is a lot – we get the “x, y-axis grid repeat” pattern, when the number is smaller than 5, we get the “overlay” pattern.
There are many different ways to turn this result of observation into well structured code, and we will not go into the details here.
Final Output
There are other parts/classes in the final version, such as PrintManager who manages to send the generated graphics to our printer, and StageManager who oversees the content flow from stage to stage. But really what we covered are the core logic of the program. So, lets see some of the generated posters.
Next Step — a combination of GAN and hand-written algorithm
After all the technical breakdown, you probably can easily spot the internal mechanism. Nonetheless, they do look pretty nice. Now, in the time that literally every single person is playing with Machine Learning, I am trying to explore the possibility of integrating GAN (Generative Adversarial Network) to create a new model that is capable of producing more diverse and even “creative” outcomes.
From last year, a guy called Cristóbal Valenzuela founded a company and created a tool called RunwayML. It is dubbed as “machine learning for the artist”. As it claims, it brings machine learning in the hands of the creative — you don’t need to write a single line of code. The features are very limited, but a very good place to start playing with Machine learning and see what it can do. Last month, it just rolled out a new feature that allows you to train your own model. So I went on Google Image, scraped about 300 Swiss Design pictures.
I inputed the pictures, RunwayML told me the 5000 epoch training will take 3-5 hours. So I went to sleep.
The next morning, training is finished. RunwayML even generated a GIF for me.
and here are some samples generated:
The result is not surprising, it preserved the color palette pretty well. But we lost all the sharp-edged geometries and the nicely placed typography. It seems like the machine just treated everything as graphics, including the typography. Nonetheless, it looks very nice. We got to give it credit.
The obvious next step is to use a hybrid approach, some information used for training (e.g. text content and text’s positionality) will need to be broken down to data instead of images.
It will be some very interesting challenges. (If I can get funding for it.)
end.