GPoeTry
Github repository here.
This is an ongoing project, which is mostly based around a bet between my brother and I: he claims that AI written poetry will never be on par with human written poetry. I think that might be true today, but not over the long run. This project is a simple test. I used public domain poems and OpenAI’s GPT3 API to generate and curate a sample of AI poems, and then built a quick site to test whether people could tell them apart. At the moment, the site isn’t public - OpenAI’s usage terms require me to get approval first (because the site serves static content rather than offer API access, it should be okay), and I’ll hopefully update this with a visible project soon!
Working With GPT3
The most interesting part of the project was experimenting with getting GPT3 to write novel poetry.
GPT3 is willing to offer up poems super easily! If you give it a prompt like “Below is a poem written by GPT3.\nTitle:” GPT3 will happily run with the input and write something up. However, I found this often produced exact copies of existing poems. This was especially true when adding in an author as a hint - GPT3 is constantly predicting the next token, and it makes sense that real poems would have a high probability of matching the input at any point. I tried prompting GPT3 with keywords like “novel” and “new”, but didn’t have too much luck. What consistently got GPT3 to write something that a quick search couldn’t identify as pre-existing was a novel poem title. If you gave it that prompt, it would happily write away! In order to keep tone consistent, I fed GPT3 a prompt starting with “Below is a list of poems”, one real example poem, and then a fake title: under that setting, GPT3 tends to write poems matching the style of the example poem, but with novel content. This left one challenge: generating titles for poems. I’m not sure my solution is perfect, but what I did in the end was straightforward; I fed GPT3 poem titles by an author as a list, and took the next one it completed as my input. I was worried this might turn up real titles, but didn’t observe that in practice.
Ultimately, I think there’s something to Gwern’s argument about prompt programming; even though GPT3 does a massive amount of work for you, finding prompts for effective output still requires practice and thought!
Disclaimer from the OpenAI usage regulations: I generated all AI poem output of this project in part with GPT-3, OpenAI’s large-scale language-generation model. Upon generating draft language, I reviewed, edited, and revised the language to my own liking and take ultimate responsibility for the poem content.