My Computer Illustrates Love Poems
Poems (by Humans) with Visuals (by Machine)
{{ Give me a new love carousel }}
For my first love poem carousels, I asked my computer to match each love poem to an image from the Metropolitan Museum of Art. For version 2.0, I asked my computer to illustrate each poem, all by itself.
The images were created with a text-to-image generator called AttnGAN, and I used Spacy, a natural language proccessing tool to help choose which part of the poem to use. For this project, I chose to generate an image based on the longest "noun chunk" in each poem. I hoped the long noun chunk would capture a well-described object, and sometimes, it did. For Edgar Lee Masters's My Light with Yours, the noun chunk my computer identified was "the desert sand." More often, the noun chunks were less concrete. For Frances Ellen Watkins Harper's poem Advice to Girls the noun chunk was "a friendly word." For Rita Dove's American Smooth, "swift and serene magnificence." For Edna St. Vincent Millay's I think I should have loved you presently, "all my pretty follies."
I came across a number of text-to-image generators (text2scene, SG2IM, stackGAN, objGAN, HDGan), all of which I hope to try out eventually. I chose AttnGAN for this project because I discovered a hosted version on RunwayML, which is dedicated to bringing machine learning to creators via an "intuitive and simple visual interface." The model was indeed easy to use. For anyone interested in learning more, details below. I hope you enjoy the new carousel!
Here’s What Happened:
Generated by computer with AttnGAN
Not long ago, a PetaPixel story appeared in my news feed: This AI Generates Photos Using Only Text Captions as a Guide. Naturally, I clicked the link and learned more about Allen Institute for Artificial Intelligence's new machine learning algorithm, LXMERT, which has a fun online demo here.
RunwayML has a web interface, which permitted me to try out the model without doing much more than typing in a caption and hitting run. I just needed to decide which text to use for each poem. The title? First line? First twenty words? Last twenty words? I looked at the exmples in the AttnGAN paper and decided that I'd try to keep things simple.
I used Spacy to parse the text of each poem and identify the noun chunks. The Spacy docs describe noun chunks as "flat phrases that have a noun as their head. You can think of noun chunks as a noun plus the words describing the noun – for example, “the lavish green grass” or “the world’s largest tech fund”)." I then selected the longest chunk from each poem, with hopes that the longer the chunk, the more descriptive.
Once I had the noun chunks, I ran them by the AttnGAN model and generated a picture for each. I chose to use a hosted model and followed the example Frederik De Bleser describes in his six-minute video tutorial to access the results using JavaScript.
Here are a few of my favorites:
North Wind by Lola Ridge
Sink Your Fingers into the Darkness of my Fur by Ellen Bass
The Picnic by the Ocean by Grace Cavalieri
This Poem is Asking For Your Love by Grace Cavalieri
Simaetha by H.D.
Circe by H.D.
The Rainbow by Effie Waller Smith
The Rats by Keetje Kuipers
be specific by Fred L. Joiner
On the Hill-Side by Radclyffe Hall
Thera by Brian Turner
Evangeline: A Tale of Acadie by Henry Wadsworth Longfellow
from "Like Nebraska" by Sophie Klahr
Among the Multitude by Walt Whitman
Unfolded Out of the Folds by Walt Whitman
Songs (VIII) by E. E. Cummings
Want more? See another computer-generated-image love carousel, a sentiment carousel, a category carousel and more here.
Tweet