Now, over the last year or so I’ve been asking myself: ytf am I doing this? When I code a poetry generator, what exactly am I exploring? How can I formalize, track, measure what I’m working on? There are no pre-existing representations. But I figured I’d use Jim Andrew’s comment as a starting point for thinking about charNG, the most recent generator I’ve developed.

charNG is a character n-gram generator in the tradition of Dissociated Press and Travesty. I coded it in my spare time over the past couple of months, posting various output on Gnoetry Daily, including for a chapbook of poems remixing the text of HP Lovecraft. (advert: if you enjoy surreal gothic arcana, you should take a look: Lovecraft Remixed.)

Anyway, after the cut I’ll trace the development of charNG, focusing on which aspects of the character n-gram generation I implemented, which I parameterized, which I made accessible in the graphic user interface, and why. See below for more.

Intro

The reason I need to think this through is because I do poetry generation the same way a cat stalks the room. I basically do whatever I’m interested in at the moment without much long-term planning. In part this is because my funded research is by nature fairly structured. So I figure if I keep my poetry generation as unstructured as possible, I’ll open myself to research and technologies I might not otherwise have a chance to work with. And since I’m doing poetry generation for myself, I might as well enjoy it.

Poetry generation is process; generators and outputs trace explorations.

Before I started charNG, I knew what character n-grams are, and I had a fairly good idea about how Dissociated Press and Travesty are related, but I wanted to do some coding on it to get a really good feel for the algorithms. I want to emphasize that what’s important here are the algorithms involved, not the code itself. Before too long the platform will be obsolete and the code will be usable only through emulators or antiques. But the algorithms will be the same as they were for Andrey Markov in 1906, as they were for Ausonius in the 4th Century AD, as they were for everybody before and in-between.

Output is subjective and software becomes obsolete, but output sets are infinite and methods and algorithms are eternal.

So: one reason to develop poetry generators it to acquire an intimate understanding of computional poetry algorithms. Once you have this understanding, you will have a stronger mastery of your tools.

Early Versions

First, what is a character n-gram? If you wanna know the details in as unmathematical way as possible, see the explanation I wrote up for charNG. A summary is: it’s a sequence of characters in a text.

I started with a simple interface, built in JavaScript. I chose client-side JavaScript because I wanted something that could be easily saved by anyone who found it interesting. Part of the reason I developed charNG was because there used to be a server-side Travesty generator on a url including eskimo/rstarr/poormfa, but that generator disappeared a couple months ago. But a client-side generator can be saved by anyone, and either re-used personally or posted publically if it’s got a free-software license (as charNG does).

The top textarea contains the corpus, from which the n-gram model is built. I’ve used Shakespeare as a “hello world” ever since I read The American Council of Trustees and Alumni’s report on “The Vanishing Shakespeare“, which talks about how Tha Bard is no longer required in many courses.

The next (grayed-out) textarea is a log of the computations so I can see what’s happening. Even when I’m using a good IDE I like to include logs as well as use debuggers, so if something wacky happens in a run I can see why it happened.

The two buttons show charNG’s first two capabilities: first was “analyze” to show the n-gram model being built; next was “Write”. At first charNG only output character bigrams.

Finally, the lowermost textarea shows the output of the generation.

So the first thing that charNG did was take a corpus (the topmost textarea), count all the different sequences of characters in it, and print it out in the greyed-out area. For example, if the corpus begins with “From fairest creatures we desire increase,”, and we’re building a 4-gram language model, then our model will include “From”, “rom “, “om f”, “m fa”, ” fai”, “fair”, and so on. Some of these may seem kind of strange, but as it turns out the 4-gram “om f” occurs 7 times in Tha Sonnets, in places like “whom fortune”, and “random from”.

The next thing it did was to determine how to use these n-gram models to write poetry. Well, if you’ve come up with the string “cre” so far, and your 4-gram model tells you (among other things) that these are the 4-grams it’s seen beginning with “cre”:

crea 13,
cred 3,
cree 2,
crep 1,
cres 1,
cret 1,

then you may want to just randomly pick a number between 1 and 21; if it’s 13 or less you write “a” to make “crea”; if it’s 14-16 you write “d” to make “cred”; if it’s 17 or 18 you write “e” to make “cree”; if it’s 19 you write “p” to make “crep”; if it’s 20 you pick “s” to make “cres”; if it’s 21 you write “t” to make “cret”. This sort of thing was developed by the engineer Claude Shannon around the 1940s, based on ideas by mathematician Andrey Markov from the early 1900s.

In charNG’s as shown above, the only options are “Write” (to generate text) and “Analyze” (to show the n-gram models in the corpus.) The n-gram model being used (bigram) is hard-coded, along with everything else except the choice of corpus.

Now as a tool, charNG by this point is not very flexible: it only can output something like this:

kn ich flomalveeane thin,
Whe pay s ld tharde whenl,
Wheecheluselauset’dothallseinkn’sselothrkn eav

Which sounds kinda like getting your teeth pulled out. But, it has made me familiar with character n-gram generation. In particular, I was curious about the extent to which you had to represent an n-gram model in memory (i.e. have a data structure that contained a string like “crea” paired to a number like “13″) or whether you could just generate by looking at the data model. The way I did it by just looking at the data model was inspired by how Dissociated Press, one of the earliest character n-gram generators, does it: every time you want to generate a number, you pick the string you’re looking for (like “cre”), you look through the corpus to see how many times “cre” exists, you pick one of those, and you print out the next character you want. It’s equivalent to explicitly building a model for generation, because every instance of an n-gram built at model-building time also exists at Dissociated Press-style-generation time. You’re basically just doing a run-time tradeoff: either you can have an initial model that takes up memory and initializiation time to build, or you can generate a bit slower. Why would anyone want to explore this question? That’s the joy of unfunded efforts: as long as what you’re doing is more fun than TV, you WIN.

When you generate poetry as described above, you have no funders to report to, no program managers to satisfy, no auditors to review your code, and no audience to concern you. Your knowledge and abilities are constrained only by your will. This in itself is poetry.

Mid-development

Anyways, at first charNG only output character bigrams, and the code was not that clean: I had to re-write it to generalize beyond bigrams. After I’d convinced myself that generating from corpus was the same as generating from models (I could have just written out a mathematical proof but this was more fun!) I had to decide whether to continue working on this. I figured I would, and I generalized the n from which n-grams could be built as a parameter in code:

Note that I’ve de-emphasized the corpus window at the expense of the analysis and the generation output. From my time working with designers in a web shop I’ve come to appreciate that I know nothing about design, so in general I just focus completely on functionality: an unadorned interface focusing on the view I need of the algorithms I’m exploring.

Since I’m working on this in my spare time, I don’t want to waste too much time recalling context (i.e. remembering how everything works, what I was working on, and what I thought might be fun to work on next) so I usually end a programming session by noting somewhere what I thought would be interesting next. In this case, I did so by listing the parameters and I was frequently changing on the GUI but not yet making them GUI-editable. (in part because, for example, verbose detail printing was not completely implemented.)

At this point in the development, I was thinking of ways to add newlines and spaces to make the output look more like 20th century free verse. I got the idea from the generator JanusNode, which has an “eecummingsify” button that uses rules to add spaces given a configuration file of text to match for. I think that’s brilliant, but I wanted something simpler. At this point I was just adding newlines every once in a while and generating from higher-order n-grams. But I was already getting more interesting results:

But I worse cold differite,
And han steep in Grece crossed inher’d,
And to despacious cons my memoving,
And did moan walks;
And be.
And this,
And cleasure brange canot me, death of look,
And besmell’ to sic plague-tiend’ring paid dear;
They he can all a faults, of forbeauty, new-fould would ashinks bles up his shal, do you shalf wouldst you no me
When love’s not gones on sakes brain worth to bide,
That shall is thiefs his in our befor ther or time tendom leaven by mortunearseth showers gavength hell?

The above is a contiguous selection from unsupervised generation (i.e. generating a bunch of lines and selecting several adjacent lines without breaking them up or editing them) from Tha Sonnets. The repeated “And” in the selection gives it a bit of coherence, and it has some nice phrases. 4-grams are great for portmanteaux: you’re looking at just enough context to make interesting new words, without getting words that are unnaturally long. The created words sound like Old English or something, and for me the feeling is of a meaning that is possibly comprehensible but just out of reach.

Near-complete

By the time I announced the generator on Gnoetry Daily a couple weeks ago, I had explored a number of new features:

I’d implemented GUI versions of the parameters I’d been playing around with, such as type of n-grams model, number of characters to generate, and verbosity of generation details. I’d implemented free-style transformations by adding a percent chance of inserting a newline after any word, or initial spaces before a line. I’d messed around with these after using them at 0% chance each, which only uses newlines and initial spaces that are seen in the corpus.

I’d added a feature to show the portmanteaux in the output. Basically, this looks at the words in the output, and prints any that it doesn’t see in the words in the corpus. Sometimes it lists words that are in the larger set of English language words; in the image above, charNG would list the word “infect” as a portmanteau, because the corpus only contains “infection” (“infect” is created from the output element “infect,” which is created from the 5-gram “fect,” (note the comma) which is derived from such corpus elements as ” defect,”) One of the ways I like to use charNG is to just paste in a corpus, generate 4-grams, and look at the portmanteaux

Of course, just generating a bunch of text and picking out what you like is fun too. I posted some of this on Gnoetry Daily:

• Crowning the blood
• Pain of Manhood
• you see me (sucka for love)

I especially like how 4-grams worked when processed with codework parenthetical insertions:

all the reployal :f.u[n]tually she lawsui:t.er[ror] to had objections who aborited minationstar mand
his some, assed and, in Alliarace on the :w.ith[ered]eir relights
be:f.or[get]m, thearenticational to the
1.2 bi:s.ho[ut]page which on :l.us[ts]iona, while rule as Howed
t:h.or[ror]res:t.ri[ck]al familigate.]

 
– from be:g.al[lows] US govergned

There are many features that could still be added, but sometimes I think it’s best to keep the generator simple, so I don’t get caught up on one approach for too long. There are so many possibilities out there.

A clockwork toy rattles on the tabletop;
listen to it
and build another.

One of the most interesting things in developing charNG was varying the chaining approach during generation. I tried choosing between “Markov”, “one-character overlap”, or “cento/cut-up”, as described in the documentation I wrote for charNG. Basically, one-char overlap only considers the last character when deciding which n-gram to choose next. And cento/cut-up doesn’t consider any context when deciding what n-gram to pick next; it basically picks a random set of cut-ups.

This gave me the opportunity to read up on Centos a bit. Basically they’re supervised cut-ups, traditionally using classical poetry as source text; the practice is from Greek poets from before 400 BC and Roman poets thereafter. In particular, I tracked down Ausonius’ “Cento Nuptialis”, which was composed to amuse the Emperor Valentinian in the 4th century AD. On pg 373- of a Loeb Classical Library translation that’s been scanned online, Ausonius introduces his cento by saying:

This book, then hurriedly composed in a single day with some lamp-lit hours thrown in, I lately found among my rough drafts; and so great is my confidence in your sincerity and affection, that for all your gravity I could not withhold even a ludicrous production. So take a little work, continuous, though made of disjointed tags; one, though of various scraps; absurd, though of grave materials; mine, though the elements are another’s…

And if you will suffer me, who need instruction myself, to instruct you, I will expound what a cento is. It is a poem compactly built out of a variety of passages and different meanings, in such a way that either two half-lines are joined together to form one, or of one line and the following half with another half. For to place two (whole) lines side by side is weak, and three in succession is mere trifling.

Decimius Magnus Ausonius, consul, soldier, and mediocre poet, from the 21st century I your peer salute you.

DREAMING METHODS–OPEN SOURCE PROJECT
by Andy Campbell
Dreaming Methods has three new projects available to experience – each one created without the use of Flash or any other browser plugin.

MAINLY THE MYSTERIES
by Gregory Whitehead
One of the great audio artists of our time was asked to write about what he still believes in, through all the riptides of the past decades.

THE CLUB
by Jim Andrews
Selected North American politicians, business men, and pyschopaths dbCinemized.

UNDERBELLY & SISTER STONE CARVER
by Christine Wilks
Christine on the relation of her piece Underbelly to where she’s from, and her sister’s stone carving.

UNICODE BY JORG PIRINGER
by Jim Andrews
A review of Jorg’s piece posted on netpoetic.

CITY BIRD BY MILLIE NISS
by Jim Andrews
A short review of the late Millie Niss’s new book of poems.

ANDREW TOPEL
by Jim Andrews
Some outstanding visual poetry by Andrew Topel.

IN THE SOUP WITH THE DIGITAL BOOK
by Paul Green
New contributor Paul Green, audio artist extrordinaire, wonders about the digital book.

RADIAUTEUR–NEW WEBZINE FOR RADIO ART
by Gregory Whitehead
Description of and call for works concerning a new online webzine for radio art.

MARIA ENGBERG REVIEWS FUNKHOUSER AND DRUCKER
by Jim Andrews
Thoughts on Engberg’s article recently published in EBR.

MOM’S MUSIC
by Jim Andrews
A last experience together of music.

SLIDVID 3.0
by Jim Andrews
A software project by Andrews.

THE CURSE OF THE ‘CODE BLUE’ MOTTO
by Jim Andrews
The motto of the Canadian national junior hockey team in 2011 was “Code Blue”. Who knew that their motto would prove ironic? After the final game, did whoever thought up the motto for the Canadian juniors glimpse, in a kind of literary horror, the final meaning the motto would have in history?

it’s always on. no need to switch it on and wait for the computer to start.

it needs no power. you can go hiking with it for weeks.

it’s a musical instrument.

it’s exercise for your fingers.

you cannot delete anything. that helps from time to time.

it looks nice.

you can get multicolored ink-ribbons.

you can sell the sheets of paper as unique artworks.

it’s good for showing in a museum when you are famous.

my typewriter an olivetti LETTERA 82 (travel typewriter)

There’s much to say about the project, but we do have a statement, and instructions, published along with the piece. I’ll just mention one aspect of the project that is not very visible: The file seaspar.js explains that Sea and Spar Between is licensed under a free software license. As the license says, anyone may copy it, modify it, or make use of it in some other way in creating another project – if that level of engagement interests you, please, go for it. In any case, I hope that you’ll take a look at Sea and Spar Between and that you’ll let us know what you think.

Look below to read all the information, click to enlarge any image, and feel free to provide feedback or pointers to useful systems!

1. Collaborative Generation

These are tools that allow humans to interact with them in some structured way while generating poetry.

1a. Gnoetry

Gnoetry is an interactive n-gram generator that allows humans to select words for it to re-generate in the context of a poetic form.

The human begins by selecting a poetic form.

The human then selects the source texts from which to build a language model…

…and decides how much each source texts should be weighed during word generation.

Gnoetry then generates a poem. After this, the human selects words by mousing over them, and clicks ‘Regenerate’ to have Gnoetry replace the selected words. In this way, the human and the tool collaboratively build a poem:

Gnoetry was written by Jon Trowbridge with input from Eric Elshtain and other poets.

• YouTube video of Gnoetry in motion
• More information about Gnoetry including building language models by adding source texts
• Installation on Ubuntu (including setting up a partition on Windows or Mac) or with an Ubuntu LiveCD
• Gnoetry-authored chapbooks and more information at Beard of Bees Press.

1b. jGnoetry

A Javascript implementation of Gnoetry. It allows you to markov-generate a poem from a bigram model built on a weighed corpus. Then you select which words of the poem you want to keep, and which you want to re-generate.

jGnoetry provides you with several options:

More info about jGnoetry:

• Access the jGnoetry JavaScript web page
• More information about jGnoetry’s development

1c. eGnoetry

The full version of Gnoetry can take a while to install and start using; hence “e”Gnoetry, which is a subset/variant of the full Gnoetry. It’s a Java applet on a web page, so it’s easier to access. It was developed with Processing and the RiTa + g4p libraries and includes source code so you can modify and extend it if you wish.

When you first access the web page, a poem is automatically generated:

Here’s how you, the human, can use eGnoetry to write poetry interactively:

• You can edit the currently-generated poem by clicking on the word buttons in blue. Doing so will replace that word with a new word consistent with a bigram model built from a given text
• If you want to start over you can click “regenerate verse” to replace the entire poem
• Clicking “Form…” cycles through the poem’s form (free verse, eHaiku, quatrain).
• Language models for three default texts are provided (Alice in Wonderland, Shakespeare’s Sonnets, and Heart of Darkness) and they can be cycled through by clicking on the “Model:…” button
• You can build new models by copying texts to your computer’s clipboard, then pasting them into the ‘New Model Text’ area and pressing the “Import Model” button
• Clicking the “Export Verse” button attempts to open a new web page containing the poem for printing or copying

More info about eGnoetry:

• Access the eGnoetry Java applet
• More information about eGnoetry’s development

1d. WpN

A generalization of the Oulipo w ± n algorithm. Also enables erasures and generation from templates.

• WpN is a web-based program (Javascript)

1e. Icon Poet

Icon Poet lets you click buttons that represent part-of-speech classes, which cycle through the words that are displayed in the edit area. A GUI interface lets you add members to the parts of speech.

• You can download Icon Poet (a Windows program) from iconpoet.com.

2. Unsupervised Generation

Unsupervised Generation systems primarily generate poetry without direct human involvement. These systems are interactive in allowing humans to write generation rules, set parameters, select input texts, and determine which sequence of generations or transformations the tools should make. Also, humans often use the systems and then select from the output to build their own poems.

2a. JanusNode

JanusNode contains a number of tools to write poetry. First, by selecting from a drop-down menu you can generate various types of text and poetry, such as Haiku in the example below. A number of different types of text can be generated by default, such as Blues lyrics, Insults, and Bureaucratese. The human can write and add new rules to produce new types of generators.

The human can also generate from word-based or character-based bi- and tri-gram models, either using some of the pre-build models, or creating new models.

After some text is generated, it can be made to look more “poetic” by eecummingsification, which introduces linebreaks and spaces in the text being authored. The way JanusNode eecummingsifies can be modified by editing configuration files. Similarly, Dadafying a text rearranges the words and adds spaces between them.

Also, JanusNode can do Text Mapping, which replaces the characters in the text. Below is an example of a Shakespearean Sonnet mapped to “LeetSpeak”. New Text Mappings can be authored and added.

JanusNode is a labor of love, born of two decades’ of part-time development, reflecting the interests of its creator, Chris Westbury.

“The world cannot tell you itself. You have to help it to tell itself to you.” – JanusNode documentation

• official web page with latest versions for Windows and MacOS
• Usage notes on writing generation rules, more advanced rules, automatic rule generation, and n-gram generation.

2b. ePoGeeS

ePoGeeS is a Poetry Generation Sketchbook that can do things like generate several lines of verse, generate individual lines, generate words, and generate rhymed words. Generation is done from a bigram or class-based language model, and while you’re working on a poem you can examine a list of all possible next or previous words in the language model.

Several verse generation variables can be changed, such as number of lines and rhyme scheme. Each individual line is actually generated from a number of possible candidates which are then evaluated and ranked; you can select random sampling or stochastic beam search to manage the evaluation.

Evaluation of candidate lines is done by analyzing the phonemes that make up the sounds of the words in the candidate line. The human user can set the sound types that are more likely to end up in the final candidate line selected.

The default (“hello world”) language model is Shakespeare’s Sonnets. The human can create new word-based language models by pasting in text and clicking ‘build’. The human can also examine various aspects of the language, phoneme, and rhyme models.

ePoGeeS was developed by eddeaddad (i.e. me) for his own poetry generation. It’s a little idiosyncratic because he just added whatever he needed at the time and ignored any details he wasn’t interested in, but you’re welcome to try it out!

• it’s a web-based program (Java applet)
• see online documentation with more screen-captures

2c. Infinite Monkeys

Infinite Monkeys allows you to define templates to generate poetry. Once the poem is generated, a text-to-speech interface will allow you to have your computer read your poem to you.

Language resources include a set of predefined parts-of-speech, which you can add to.

• Infinite Monkeys is written in FreeBASIC for Windows (and possibly Linux and FreeBSD?)
• The Downloads page includes the application as well as a pdf of poetry written with it.

2d. Rhyming Robot (seuss)

Rhyming robot, also known as seuss, is a codeset that generates markov text that is arranged into rhyming lines.

• You can use it on the web
• It is written in python; it can run via command line, web interface, or IRC interface.
• Also see development notes.

2e. charNG

charNG is a character n-gram generator with Markov, cento/cut-up, and one-char overlap modes.

• charNG is a web-based application written in JavaScript
• also see release notes and development notes

2f. Mchain

Mchain is a tool that uses n-grams to generate an arbitrary quantity of text, which can be output to a text file. The human then uses Mchain’s output in some way to create a poem; by selecting interesting phrases from the text, for example. Mchain a command-line application:

Mchain was written by Adam Scovel.

• Mchain 0.3 info and screenshots (at the bottom half of the page)
• Mchain 0.2 (Python script) download and installation information
• Mchain 0.3 (C++ program) download and compilation information

2g. Dissociated Press

Developed in the early 70s, Dissociated Press is possibly the earliest n-gram generator (character- and word-based) that produces poetic-looking output. It is still available in the standard GNU Emacs distribution via “M-x dissociated-press”.

The Emacs version of Dissociated Press was developed by the Free Software Foundation.

• You get it as part of Emacs text editor
• Documentation and usage notes

2h. Other Systems

The web is full of “mad lib”-style slot-filling poetry generators and other simple programs for generating poetry, which may be of varying use to poets. Below are some examples:

• Diastic Reading – a web implementation of an approach used by Jackson Mac Low that creates a poem from a text given a seed string. (an older implementation seems to be offline)
• CAP – Computer Aided Poetry by Eugenio Tisselli – given a seed text, replaces words with semantically-related words. Also see usage notes.
• Weltanschauung, a perl command-line application that generates cut-ups from text. Also see an explanatory presentation on youtube.
• Thinkzone Poem Generator – a slot-filling generator that lets you edit the templates as well as the words in each slot, and provides Javascript code.
• Darwinian Poetry, in which you get to help poems evolve.
• Oulipoems: The Electronic Muse, a grammar-based tool in which the user generates and re-arranges lines of poetry in a variety of styles, with the ability to add words
• ppg256 – simple poetry generators developed in only 256 characters of Perl
• Poetry Idea Engine – an animated Flash program from Scholastic to teach kids poetic forms using slot-filling.
• Wave Books’ Erasures tool – which allows you to remove words from an existing poem
• ALAMO’s Littéraciels – including Langage Algorithmique pour la Production Assistée de Littérature
• Similarly, an Oulipo n+7 generator is always fun

.

3. General Tools

These are tools in which the human takes the initiative in developing the poem, and is more of an essential component of the process than in systems discussed so far.

3a. GTR Language Workbench

GTR Language Workbench was developed a while back, went offline, and is now being distributed by the Newark Review. The web page calls it “a digital studio for language which allows for any number of literary and aesthetic modifications to texts, similar to the way current graphic design software like Photoshop and audio software like Sound Forge permit artists to create, modify and combine different visual and sound pieces.”

• It’s a program for PCs and Macs, you can download an installer from the Newark Review

3b. The Muse

The Muse is made up of four web-based programs that were developed to help songwriters during the February Album Writing Month challenge. The first two programs are oriented towards music. Plot Spline suggests constraints for lyrics, and Struxxure suggests musical structures. Both are random generators.

LyriCloud is the third program, and focuses on helping write song lyrics. It generates a “cloud” of words which are related in some way: semantically, or through corpus adjacency. The human clicks on one of the words, which produces another cloud centered on the word that had been clicked on. In this way, the human generates a sequence of words which can be used to build lyrics around.

Titular is the fourth program, designed to help the songwriter with song titles. It works by filling part-of-speech templates that were built from a corpus of actual song titles.

LyriCloud and Titular are the most obviously suited for text poetry, but all four may be useful, depending on the poet’s needs. The Muse was developed by Burr Settles.

• Try out The Muse at FAWM.org
• A research paper (pdf) describing and evaluating LyriCloud and Titular

3c. RiTa + Processing

Processing is a tool resembling a simplified Java IDE. It is popular for writing animated and installation poetry (as opposed to poetry that focuses on texts and algorithms.) RiTa is a Java class library that was developed for use with Processing and includes objects like an n-gram model builder and generator, a grammar builder, and part-of-speech tagger, and much more. Using RiTa and Processing requires more human development effort than the previous programs, but they can be a powerful set of tools – for example, eGnoetry (above) was written using this combination.

RiTa and Processing are well-documented and include a number of example programs with source code, which makes programming almost as easy as writing HTML. For example, a simple n-gram generator can be written in a few lines of code:

Processing was developed by Ben Fry, Casey Reas, and a number of volunteers. RiTa was developed by Daniel C. Howe.

• Processing – a tool and set of libraries for writing Java-based animated and installation poems
• RiTa – a library developed for text generation with Processing

4. Historical Systems

Humans have been developing poetry generators since the dawn of (computer) history! Many of them are no longer available, but here is some information about several of them, with an emphasis on systems with versions that might still be available or in use. (for the full story on historical systems, check with Chris Funkhouser)

• Christopher Stratchey’s Love Letter generator. Are love letters poems? Maybe, maybe not… but this algorithm dates from 1952, and the developer was pals with Alan Turing!
• If love letters aren’t poems, then maybe Theo Lutz’s Stochastic Texts are! Check out this German emulator and the relevant paper.
• Mark V. Shaney (information) – a word n-gram generator from the early 1980s, used on the Tao Te Ching and for Usenet posts. Also see this Markov Text Synthesizer and this Markov Perl script.
• The character n-gram generator Travesty is from the same era.
• Ray Kurzweil’s Cybernetic Poet – the poetry generator that will help bring on The Singularity!
• Thunder Thought and Versifier – classic DOS poetry generators by Rosemary West
• Charles Hartman is an early programmer of poetry tools.
• Verbasizer – a lyrics generation tool used by David Bowie

That’s all for now! If I’ve missed anything important, please let me know!

1. Unigrams

I’ll use Shakespeare as an example; either you’ll like it, or you’ll like that I’m shredding it.  So think of Shakespeare’s line:
“When I do count the clock that tells the time.”
It contains several words:
1. when
2. I
3. do
4. count
5. the
6. clock
7. that
8. tells
9. the
10. time
Computational linguists might call each word a unigram, as part of a language model.

I could generate my own poetry by rolling a 10-sided dice:
- if a ’3′ came up, I’d write ‘do’.
- if a ’9′ came up I’d write ‘the’.
- if a ’1′ came up I’d write ‘when’.
and so on.  After rolling the dice a couple times I might get something like:

do the when clock the when when that

Which sounds pretty incoherent, but this is just the first step!

2. Bigrams from a single line

Think of that line again:
“When I do count the clock that tells the time.”
It contains several adjacent pairs of words:
1. when I
2. I do
3. do count
4. count the
5. the clock
6. clock that
7. that tells
8. tells the
9. the time
Computational linguists might call these bigrams.

If I found a 9-sided dice, I could generate my own poetry again.
- For example, if an ’8′ came up, I’d write ‘tells the’
- The previous bigram ends with the word ‘the’.  So for the next word I’d look at those bigrams that begin with ‘the’.  There are two of these:
5. (‘the clock’)
9. (‘the time’).
So I flip a coin, and I decide on 5.
So far my poem is:

tells the clock

- Then for the next word I’d look at those bigrams that begin with ‘clock’. Unfortunately there is only one, ‘clock that,’ so I need to choose that.
- And after that, there is only one bigram that begins with ‘that’ (‘that tells’) so I need to choose that!
- And after that there is only one bigram that begins with ‘tells’ (‘tells the’) so I need to choose that.
- Luckily there are again two bigrams that begin with ‘the’ so I flip a coin and pick ‘the clock’.

So thus far my poem is:

tells the clock that tells the clock

which is bad, but at least makes a bit more sense than the unigram-generated poem.

But the sequence ‘the clock that tells the’ is lifted straight from the original line of poetry (a.k.a. the training data) because there weren’t a whole lot of options available.  This is the danger of using a small data set!  So let’s look at a bit more data.  At this point, using a computer programs helps.

3. Bigrams from Shakespeare’s Sonnets

Let’s take the entire set of Shakespeare’s sonnets from Project Gutenberg, and write a program to read the text file and break each line of poetry into bigrams as above.  There are a couple decisions we need to make when tokenizing (splitting the text into words) such as whether we should split things like “self-substantial” into two words or not, but the programming itself is pretty straightforward.

It turns out there are 11817 unique bigrams in Shakespeare’s sonnets which occur a total of 17534 times.  For example:
- “eyes are” is a unique bigram which occurs 2 times
- “fragrant rose” is a unique bigram which occurs 1 time
- “even to” is a unique bigram which occurs 3 times
and so on.

So let’s generate a poem!  How do we know where to start?  We keep a separate list of those words that begin a line of poetry!  Here is a list of the tokens that start lines in the Sonnets, along with the number of times they occur:

I 37, a 17, above 1, accuse 1, admit 1, advantage 1, after 2, against 10, ah 5, alack 1, alas 2, all 10, although 5, am 1, among 1, an 1, and 242, angry 1, anon 1, another 1, applying 1, are 3, art 1, as 40, askance 1, at 5, attending 1, authorizing 1, awakes 1, ay 2, bare 1, be 9, bear 1, bearing 2, beated 1, beauteous 1, beauty 2, beauty’s 1, because 2, before 3, beggar’d 1, being 4, beshrew 1, better 1, betwixt 1, beyond 1, blessed 1, book 1, borne 1, both 3, bound 1, breathed 1, bring 1, but 89, buy 1, by 14, call’d 1, calls 1, came 2, can 3, cannot 1, canst 1, cheered 1, chiding 1, clouds 1, come 2, comes 1, commanded 1, commit 1, compar’d 1, compare 1, consum’d 1, coral 1, could 1, counting 1, crawls 1, creating 1, creep 1, cries 1, crooked 1, crowning 1, cupid 1, darkening 1, dear 1, death’s 1, delights 1, describe 1, deserves 1, desire 1, desiring 1, despite 1, devouring 1, die 2, disdains 1, dissuade 1, distill’d 1, divert 1, do 4, doing 1, dost 3, doth 6, doubting 1, drawn 1, drink 1, drugs 1, dulling 1, duty 1, each 2, eat 1, either 1, enjoy’d 1, entitled 1, ere 2, eternal 1, even 10, exceeded 1, excuse 1, excusing 1, fair 3, fairing 1, farewell 1, featur’d 1, feed’st 1, feeding 1, feeds 1, find 1, finding 2, flatter 1, for 73, forgot 1, from 14, full 2, gainst 1, gentle 1, gilding 2, give 5, giving 1, gor’d 1, grant 1, great 1, growing 1, grows 1, had 2, hang 1, haply 1, happy 1, harsh 1, hast 1, hate 1, hath 6, have 6, he 7, hearing 1, hence 1, her 4, herein 1, hers 1, hiding 1, him 2, his 3, holds 1, how 21, hung 1, if 35, in 36, incapable 1, incertainties 1, increasing 1, injurious 1, intend 1, is 10, is’t 1, it 6, join 1, just 1, kill 2, kind 1, kissing 1, knowing 2, laid 1, lascivious 1, lean 1, leaving 1, leese 1, lest 6, let 10, lifts 1, like 8, lilies 1, lo 3, look 6, looking 2, lord 1, lose 1, love 4, love’s 3, loving 1, mad 2, made 2, make 6, makes 2, making 7, mark 2, may 3, me 2, methinks 1, might 1, mine 10, more 5, most 2, much 1, music 1, my 30, myself 2, naming 1, nativity 1, nature’s 1, nay 2, needs 1, neither 1, never 1, no 17, none 1, nor 24, not 10, nothing 1, now 9, o 47, o’er 1, o’ercharg’d 1, oaths 1, of 20, on 6, one 5, only 1, or 38, others 1, our 2, painting 2, past 3, perforce 1, pitiful 1, pity 3, plods 1, pluck 1, pointing 1, points 1, poor 1, possessing 1, potions 1, praising 1, presents 1, presume 1, prison 1, profitless 1, proving 1, receiving 1, resembling 2, reserve 2, return 2, revenge 1, richer 1, rise 1, robb’d 1, robbing 1, root 1, roses 2, rough 1, ruin 1, sap 1, savage 1, save 7, say 1, seeking 1, seems 1, self 1, serving 1, sets 2, shall 10, she 3, shifts 1, show 1, showing 1, simply 1, sin 1, since 15, sing 1, sings 1, sinks 1, sland’ring 1, so 47, some 7, sometime 2, speak 2, speaking 1, spend’st 1, spending 1, steal 1, stealing 2, still 2, stirr’d 1, straight 1, strikes 1, such 5, suffering 1, suns 1, supposed 1, suspect 1, swear 1, sweet 4, sweets 1, take 3, tan 1, tell 1, tempteth 1, ten 1, than 13, that 83, that’s 1, the 78, thee 1, their 1, theirs 1, then 33, thence 1, there 2, therefore 8, these 5, they 11, thine 4, think 1, this 8, those 7, thou 25, though 10, three 3, through 1, thus 8, thy 29, till 6, time 1, time’s 1, tir’d 1, tired 1, tis 3, to 78, to-morrow 2, too 1, towards 1, triumph 1, truth 1, two 1, under 2, unlearned 1, unless 4, unlook’d 2, unmoved 1, unthrifty 1, until 1, upon 7, use 1, uttering 1, vaunt 1, want 1, was 4, we 1, weary 1, weeds 1, weighs 1, were 4, were’t 1, wh’r 1, what 14, what’s 2, whate’er 1, when 57, whence 1, where 11, wherein 2, whereon 1, whereto 3, which 47, while 4, whilst 11, who 18, whoe’er 1, whoever 1, whom 1, whose 6, why 12, will 4, wilt 1, wishing 1, with 18, within 5, without 5, wooing 1, worthy 1, wound 1, wretched 1, yet 20, you 7, your 5, yourself 2

So in Shakespeare’s sonnets, the token “I” begins a line of verse 37 times; the token “how” begins a line of verse 21 times; the token “why” begins a line of verse 12 times, and so on.

Now let’s say we get a lottery machine, and wrote “I” on 37 of those little ping-pong balls, and wrote “how” on 21 of the ping-pong balls, and “why” on 12 of the ping-pong balls, and so on.  Then we draw out one of the ping-pong balls, look at the token on it, and put it back.  Let’s say the ping-pong ball we drew out said: “that”.  So thus far we have a poem that goes:

that

So now we can have our program look at the tokens that follow “that” in our corpus.  Here is a list of the tokens that follow “that” in Shakespeare’s Sonnets:
I 28,  a 1,  able 1,  affable 1,  all 1,  am 1,  art 1,  audit 1,  barren 1,  be 1,  bears 2, beauteous 1,  beauty 3,  before 1,  besiege 1,  best 1,  better 1,  blessed 1,  bond 1,  bosom 1, brightness 1,  by 1,  calls 1,  can 1,  cannot 1,  censures 1,  churl 1,  copy 1,  deep 1, did 2,  do 2,  doth 3,  due 3,  ease 1,  enfeebled 1,  eternal 1,  ever 1,  every 2,  eyes 1, face 1,  fair 1,  fears 1,  feeds 1,  fell 1,  fester 1,  fire 1,  flies 1,  flower 1,  followed 1, for 2,  fresh 1,  from 1,  full 1,  gainst 1,  gave 1,  give 1,  glory 1,  god 1,  ground 1, grows 1,  guides 1,  harvest 1,  hath 1,  have 2,  having 1,  he 3,  heals 1,  heart 1,  heaven’s 1, heavy 1,  her 1,  heretic 1,  hidden 1,  him 1,  his 1,  honour 1,  idle 1,  in 7,  ink 1, is 9,  it 2,  keeps 2,  languish’d 1,  leads 1,  leaves 1,  level 1,  liberty 1,  life 1, like 1,  long 1,  looks 1,  loss 1,  love 3,  love’s 1,  loves 1,  made 1,  makes 2,  man’s 1, may 1,  men 1,  millions 1,  mine 3,  more 1,  muse 1,  music 1,  my 6,  myself 1,  nimble 1, nothing 1,  offence 1,  on 2,  one 1,  other 1,  our 1,  over-goes 1,  pay 1,  pen 1,  pine 1, plea 1,  poor 1,  pour’st 1,  purpose 1,  putt’st 1,  receive 1,  record 1,  repose 1,  riches 1, right 1,  said 2,  same 1,  says 1,  seals 1,  shall 1,  she 5,  sin 1,  smells 1,  so 2, sometimes 1,  sorrow 1,  still 1,  struck 1,  sun 1,  sweet 3,  taught 1,  tells 2,  the 4, thee 1,  then 1,  there 1,  thereby 1,  they 2,  this 2,  thou 18,  through 1,  thy 2,  time 7, to 5,  tongue 3,  touches 1,  travels 1,  unfair 1,  use 1,  ushers 1,  vex’d 1,  vow’d 1, we 2,  wear 1,  weight 1,  well 1,  were 1,  when 2,  which 19,  wild 1,  with 1,  word 1, writ 1,  writes 1,  you 9,  your 3, yourself 1

So in our corpus, “that I” occurs 28 times, “that music” occurs once, “that word” occurs once, and so on.  So let’s say we get our lottery machine to pick one of these words, and let’s say it picked “loss.”  So thus far we have a poem that goes:

that loss

So now we can have our program look at the words that follow “loss” in our corpus, and we use our lottery machine to pick one of them. Let’s say it picks “in” so we have:

that loss in

So now we can look at the words that follow “in”, and pick one of them. And so on, until we might get a line like:

that loss in youth doth appear and straight

which doesn’t make a lot of sense altogether.  But the first part does.  And since we’re humans, we’re in charge, so we can fix it!  First, let’s cut out the last three words, leaving us with:

that loss in youth doth

Then let’s re-generate the poem, starting with picking a word that follows “doth,” then a word that follows that, and so on.  This time around, we might get:

that loss in youth doth bear all things remov’d

which is better than before.

4. Conclusions

So to sum up:

• n-gram language models are built with things like bigrams and unigrams. They are built on training data.  They collect information about what words are seen next to each other, and how frequently those sequences of words are seen in the data.
• you can generate poetry by having a program select words from the n-gram language model.  You can use the frequency counts to make it more likely that a given word will be picked.
• if the line generated by sampling from a bigram model is coherent, it’s because each pair of words is coherent.  (i.e. the language model does not contain bigrams like ‘when when’)  However, this may not be enough to generate poetry completely algorithmically.  You may need to have a human author intervene.

There’s a lot more to n-grams, of course.  Wikipedia is a good place to start reading on it.  Otherwise, check out a copy of Jurafsky and Martin’s “Speech and Language Processing.”

Hope that was interesting!  later.

From 'Klee Flowers'

I’ve been working on a new Javascript app to display images on the net. If you’ve seen any of the previous dbCinema slideshows, you may recall they didn’t have fade in/out of the images. I was finally motivated to make that sort of app. The motivation was not for the dbCinema images, but for some others which aren’t up yet, which I’ll show you when that project is finished.

However, I have developed the app with the dbCinema images. All of the dbCinema slideshows at http://vispo.com/dbcinema now have that feature. Fade in/out enhances the experience of these images because there is often continuity between the images, and the fading in of one over another reveals the nature of the continuity in a way that both shows how the thing is growing/changing and contributes to the ‘cinema’ of dbCinema.

It also provides ‘action’ while the app is busy downloading the next image, which is a consideration in streaming net art. Usually the images are 1280×1024 in size and vary from around 75kb to about 400kb in file size, so the downloading of the images is usually not instantaneous.

There are lots of slideshow apps on the net. Why not use one of those? Well, the ones I saw were usually suitable for smaller images. I wanted this to take up the entire screen for an art experience, not a showing of photos. I also wanted to have categories of links that usually weren’t available with the ones on the net. And I wanted the interface simple and integrated into the viewing area, rather than having one area for the graphics and one for the controls, so the full screen could be used to show the graphics. But, also, I’ve been kind of interested in the architecture of a slideshow app. More on that a bit later.

The app has two ‘pages’ or screens. The first displays thumbails of some or all of the images. Some of the series have hundreds of images; in that case, the thumbnails show every second or third image. This page or screen also usually contains a link to information about the series. And links to the previous and next series via the ‘<<’ and ‘>>’ buttons.

When you either click a thumbnail image or the ‘[...]‘ button, you enter the second page or screen which displays the slideshow. You can adjust the amount of time images are displayed and the amount of time images fade by clicking the ‘Options’ button. You can also choose whether images fade in/out or just fade in. When the images fade in/out, there’s a dimming and brightening. When they just fade in, that dimming/brightening is not present. Sometimes the dimming/brightening is nice, such as in the Silvia Saint series. It gives it a sort of breath in and out.

But, all that said, the main thing I’ve learned in making this app is how to make a better one that I haven’t made yet. And therein lies an interesting lesson for me.

The architecture of the current app involves two images. One image fades in while the other fades out. Or one fades in while the other, which is already faded in, stays static, and disappears once the other is faded in.

A more general architecture would be as follows. It would involve going OOP (object oriented programming) with the app. The idea would be to program an image object. When it’s time to display a particular image, an image object would be dynamically created. The data of this object would include the image itself, it’s name and so forth. The methods or behaviors would include the mouse event functions, the fade in/out functions, the destructor function, and so on. When the object is no longer required, it’s deleted.

The advantage of this dynamic creation/destruction of image objects is that rather than being limited to two images, you can have several images onscreen at once. So, for instance, when the user quickly presses the ‘>’ button repeatedly, the response could be more as one would like.

But, more importantly, it would free up formal possibilities. For instance, one could program keyboard functionality so that when a particular key is pressed, a particular image could fade in/out or do that and grow/shrink.

Getting away from the notion of a slideshow into a more general form.

And that is of artistic interest. Programming should suggest and open up formal possibilities. Going OOP, properly understood, does that. So next up for me when I approach another Javascript project will be getting a handle on OOP in Javascript. There’s an interesting article on the subject at http://msdn.microsoft.com/en-us/magazine/cc163419.aspx

Part of the idea of dbCinema is to explore the possibilities of cinema between the still, the slideshow, and the sort of dbCinema at http://vispo.com/dbcinema/londonhypotrochoid which is further yet from slideshow but is, nonetheless, based on 100 pictures downloaded from the net via a Google image search of “London”. This latter piece does not use the new slideshow app but dbCinema itself.

ORK

short for Objects R Kool. ORK is an object oriented language with a very verbose syntax. Unlike most of the esoteric programming languages you have to write a lot of code even to get the most simple task done.

a simple hello world-program would be written like this:

When this program starts:
There is a scribe called Writer.
Writer is to write "Hello, world!"

or a script that outputs the input could be like:

There is such a thing as a mouse.
A mouse can be_eaten.
A mouse has a status which is a word.
A mouse has a voice which is a scribe.
A mouse has an input which is an inputter.

When a mouse is to be_eaten:
There is a word called squeaky sound.
input is to readOne squeaky sound.
If input says it's done then status is "eaten".
voice is to write squeaky sound.

There is such a thing as a cat.
A cat can eat a mouse.
A cat has a Lingo which is a linguist

When a cat is to eat a mouse:
The mouse is to be_eaten.
Lingo's first operand is the mouse's status.
Lingo's second operand is "eaten".
Lingo is to compare.
If Lingo says it's not equal then I am to loop.

When this program starts:
There is a cat called Lucifer Sam.
There is a mouse called Gerald.
Gerald's status is "alive".
Lucifer Sam is to eat Gerald.

more info:
http://codu.org/eso/#ORK

and one of the possibly most verbose languages is called

English

i rather quote the Esoteric Programming Languages-wiki than write about English myself:

English is a declarative programming language. Many people are familiar with it even if they don’t know any other programming language.
[…]
A compiler of English (usually to some other high-level language) is called a ‘programmer’. They are usually humans and they err.

a hello world example program:

This program writes "Hello World" (without quotes) to the output.

a fibonacci sequence program:

Read an article about the Fibonacci sequence. Now write a program that computes it.

more info:
http://www.askoxford.com/

there’s a huge number of similar languages (around 6500 that are still in use).

the first of the two languages is called

AlphaBeta

It uses the letters of the alphabet as commands. Upper- and lowercase letters are different commands.

A quote from the description page: AlphaBeta uses 5 registers as a way to store memory, 2 are changeable and hold an integer, 1 is a result register and cannot be changed, 1 is used for looping and the other is used for memory ( like brainfuck‘s > and < ). There is no way of comments. Also, AlphaBeta has 1 kb of ram.

a hello world programm in AlphaBeta:

cccCISccccCIScccCIYx
SGSHaaCLgDLihhhDLDLgggDL
TTGaaCL
SGccbbbCLDLgggDLjggggDLSHDL
TTGaaaCL

a program that copies its input to its output:

JCLigggO

more info:
http://esoteric.voxelperfect.net/wiki/AlphaBeta

the second language is

Capuirequiem

Capuirequiem uses the ASCII characters with the number codes 33 to 126 as commands or literals of the language.

a hello world program:

[Hello World!]O

a program that prints out the fibonacci sequence:

[This function output a number in decimal format]Z
[
[]D/V
D /01!\$ 48,_KW\C/
!SB DL Z\O
]"n{

[Print out the fibonacci sequence]Z
[1.]O 0"a{1"b{ [
"a}"b}D"a{_KD"b{
"n}X [.]O
1L
]X

The language is known to be Turing complete.

more info:
http://esoteric.voxelperfect.net/wiki/Capuirequiem

LOLCODE is inspired by the infamous lolcat internet meme. Lolcats are images distributed via the net with cats and their written “thoughts” on it. The language they speak is called lolspeak (lol is the net-acronym for “loughing out loud”) an english dialect. An example of lolspeak would be the question: “I Can Has Cheezburger?” (it’s also the name of the biggest lolcat-blog).

LOLCODE’s commands are modeled after lolspeak words and grammar. And so a typical “hello world”-program would look like this:

HAI
CAN HAS STDIO?
VISIBLE "HAI WORLD!"
KTHXBYE

or a program that counts to ten:

HAI
CAN HAS STDIO?
I HAS A VAR
IM IN YR LOOP
   UP VAR!!1
   VISIBLE VAR
   IZ VAR BIGGER THAN 10? KTHXBYE
IM OUTTA YR LOOP
KTHXBYE

unlike code poetry in the almost esoteric programming language perl, poems written in LOLCODE can also be understood or at least read by non-programmers.

(picture by kn0thing)

if you’d like to try it out yourself an online javascript translator and interpreter can be found here:
http://fullvolume.co.uk/static/lolcode/

more info:
http://lolcode.com/
http://www.icanhascheezburger.com