- Have you heard of GPT-3?
- What GPT-3 does and what it’ll do to us
- Go Transformer, go!
- “OpenAI,” but “closed API”
- The price of using GPT-3
- The language of GPT-3
- Our first dialogue with AI: What is online marketing?
- GPT-3 runs a temperature
- Objectively speaking it’s subjective
- “Online Marketing is a waste of time”
- GPT-3 enters the chat
It’s one of the most popular and talked about topics in tech circles: GPT-3. An autoregressive language model based on AI, GPT-3 and its so-called transformer is touted as being able to draft intellectually challenging texts and copies on its own. Possibly paving the way in the not-so-distant future for the bulk of digital content found online to be created with such technology. At the moment, GPT-3 is not available for general use; however, author and linguistic critic Peter Littger was granted access. In an exclusive two-part article for OMR, he lays out his impressions of the technology, how advanced it is and how it could change media and marketing. Today, we are presenting part 1. Part 2 will appear next week.
Have you heard of GPT-3?
2020 was a year of cryptic names and terms. Covid-19, Space X (Elon Musk’s space programm) and X Æ A-Xii (Elon Musk’s son) led the pack. Not far behind was the acronym GPT-3. GPT-3, which stands for generative pre-trained transformer 3 particularly caught our attention. In lay terms, it’s reminiscent of a supercomputer that can create poetry and possibly even possesses the ability for abstract and original thought. Since its launch in May 2020, it has fascinated many and alarmed many more—especially authors—with its surprising abilities. It should be noted that Elon Musk also co-founded OpenAI in 2015, the California-based development company behind GPT-3. However, due to Tesla’s work on its own artificial intelligence technology, Musk stepped down from the OpenAI board, to avoid any possible conflicts of interest.
Musk has made no secret about his misgivings regarding the dangers that AI pose to humans. In 2017, he called AI “a fundamental threat to the existence of human civilization”. That same year, management consulting company McKinsey published a study predicting that by 2030 somewhere between 400 and 800 million people will lose their jobs due to automatization.
What GPT-3 does and what it’ll do to us
Is GPT-3 really the snowball that leads to the avalanche of automatization in the working world? The technology is hyped up as offering something truly new and unique. GPT-3 does not claim to be a text generator that works with preprogrammed sentences and paragraphs to spit out patternized weather, stock market or traffic reports, but rather is a so-called artificial neural network (ANN), which can reportedly reproduce context as well as human thought and lines of reasoning to generate its own texts and code.
With a massive database of information at its disposal, where the entire English Wikipedia page makes up less than 1%, it primarily performs incredibly fast machine-learning-based calculations and combinations of probabilities regarding the next word, the next sentence or next paragraph. Simultaneously, the transformer has mastered language composition through natural language processing, NLP (not to be confused with the pseudoscientific method of NLP: neuro-linguistic programming).
Go Transformer, go!
In the months that followed the launch, numerous editorial staffs and individual users have attempted to capture and publish the practical and intellectual utility of GPT-3, in various languages (although primarily in English), in all manner of styles (for example as Goethe), in a variety of text formats like (interviews, YouTube shows, screenplays, literary parodies and, last, but not least, programming codes), all of which have been carried on a wide range of topics.
GPT-3 made headlines in September, when the Guardian published a longer essay under the title“A robot wrote this article”. In it, GPT-3 had the chance to present itself in its own words and to assuage any fears humans may have of their machine competitors. The novelty and the quality of the work left readers stunned. Beneath the article, the Guardian’s editorial staff included a note that they performed minor edits to the text (mainly because it consists of parts from several different versions). They did say, however, that “overall, it took less time to edit than many human op-eds.”
That note really piqued our interest: Could we really create the bulk of our content in the future using GPT-3? And is GPT-3 already able to draft marketing texts faster, better and for less than any human?
“OpenAI,” but “closed API”
So we decided to test it. But that proved easier said than done. For GPT-2, the previous generation of the transformer, OpenAI made it open access, which it still is today. GPT-3, however, is closed API, meaning using it requires applying to OpenAI for access. Doesn’t sound all that “open,” but I digress.
OpenAI defended limiting access in a scientific paper citing the potential for misuse (beginning on page 34). “Any socially harmful activity that relies on generating text could be augmented by powerful language models. Examples include misinformation, spam, phishing, abuse of legal and governmental processes, fraudulent academic essay writing and social engineering pretexting,” argue the paper’s authors. API applications that, for example, require medical diagnoses or therapeutic-pharmacological inquiries are categorically rejected by OpenAI.
The price of using GPT-3
It must be said, unofficially, it’s easier for OpenAI to realize its monetization model by using API. When the first pricing models were announced last fall, many industry observers were taken aback by the cost. Whether a pricing model that excludes the majority of hobby users squares with the general use approach the company originally championed is a discussion for another time and place.
At the time of our test in December 2020, we did not have any access of our own to GPT-3. So we reached out to Vladimir Alexeev, a member of the digital innovation team at DB Schenker, the logistics arm of German national rail operator Deutsche Bahn and, as Harper’s magazine put it, a “hobby data journalist” in his spare time. OpenAI named him their “German ambassador” for GPT-3. He agreed to show us how GPT-3 works in several video calls and to help us run our test.
The language of GPT-3
The first question that arises before you begin working with GPT-3 is simply how do you communicate with the system? What input screen is used? Which parameters? And, last but not least, in which language??
The OpenAI page that Vladimir opens has a large entry field on the left, above which stands “playground.” To the right there are various buttons and menus. The first thing to do is choose from one of four “engines,” which determine the system’s performance level. They are to serve as a method of calculation for the intended pay model in the future.
The GPT-3 playground. To the right is the engine selection, the response length and additional variables (click to enlarge)
Beneath the engines, you can determine the maximum length of the text with the “response length,” which also indirectly impacts the speed and quality. The core units are called “tokens.” Generally speaking, they correspond to a syllable in English and also help determine the end price.
Regarding language, GPT-3 knows no bounds. The best results are definitely achieved in English, but any and all languages will work. In addition to English, we’re using German as we are based in Germany.
Our first dialogue with AI: What is online marketing?
Finally! How many times have we sat in front of a computer and wished we could finish or start a text just by clicking a button? That dream should now become reality.
Attempt number 1: with the smallest engine (“ada”) and a modest length of 300 tokens. Vladimir told us ahead of time something that has also been well documented in a wide range of blogs: The results are solely dependent on the text that you feed into it ahead of time, the so-called “prompt”. The Guardian, for example, gave it a handful of sentences and paragraphs, which served as guidelines regarding content, theme and style for the transformer. An important factor with machine learning lies in the “attention,” the continual consideration of everything we’ve mentioned up to here. The attention you give it directly impacts the subsequent steps, i.e. a kind of telepathy through empathy. And the perfect expectation management of that which the user would like to read and write themselves.
GPT-3 runs a temperature
Throughout the process we have to keep in mind that GPT-3 does not think autonomously. Instead, it calculates what the most probable subsequent word or text is based on the texts fed into it. Randomness, creativity and diversity of results can be adjusted using the “Temperature” and “Top P” sliders on the right. OpenAI recommends using one of the two buttons at a given time, ensuring that the other is set to 1.0), but Vladimir suggested setting “temperature” to 0.8, a relatively high value. This way, he says, we can better avoid repetitions and with it limit stubbornness and close-mindedness.
GPT-3 shows alternatives for every word, as well as probabilities (click to enlarge)
To get a feel for what the system can do, we decide to start with something close to home and type in “Online Marketing is” and hit enter. Instantly, the white of the “playground” is populated, sentence by sentence, until the paragraph is complete:
Online Marketing is becoming an important part of everyday entrepreneur’s business. More and more businesses are now using online marketing to generate sales in the same manner as our mass marketing methods use direct mail, television, radio, print newspapers and magazines. In fact the market will likely be flooded with online marketing before long.
Our knee jerk reaction is “not bad.” What is interesting, however, is that for such an objective prompt (Online marketing is) the resulting text is quite subjective. Is that due to a lack in further analytical ability? Will the text continue in the same vein? After a brief pause, the next paragraphs begin to appear, e.g.:
The goal is not only to identify your target audience but also optimise them so that they can buy your product or service at higher prices than competitors. Sadly there is no such thing as “true conversion rate optimisation” because like everything else in life you need some research before you begin using any sort of negotiation tactics with your target audience directly before actually making them purchase anything from your business – even if it means submitting fake testimonials from customers who have ordered stuff from your site!
Again, GPT-3 produces a text that is subjective and based on opinion without providing any sound reasoning. Probably a very good chance, as well, that many OMR readers will emphatically disagree with the assertion that “there is no such thing as true conversion rate optimisation.” The transformer seems to want to provide more than a regurgitated and paraphrased version of the prompt a la Wikipedia. Again, not a bad result, per sé. But are such results viable options for marketing and advertising purposes?
Objectively speaking it’s subjective
It’s quickly becoming apparent that GPT-3 likes to indulge in a bit of opining that occasionally borders on a lack of respect. The transform has no qualms taking isolated occurrences and assertions and presenting them as rules and facts. Vladimir dials back the “temperature.” But there is no slider to adjust impulsiveness, social conscience or composure. Without edits or addendums, the text would not be fit for print. A good start, but we’ve seen enough. We stop the sequence.
We keep at it for a couple of more hours experimenting with some of the more powerful GPT-3 engines. We select the engine “babbage” and leave all the other settings as they were with “ada,” as is the prompt:
Online Marketing is among the most commonly used marketing to advertise businesses, products and services by assisting with generating leads. The process usually involves creating a website in which companies can advertise their products or services for free. Internet marketing is often seen as a great way for businesses to attract potential customers and encourage them to buy their products or services from them.
What stands out here is clarity and disinterest, but it’s also too short for any repetitions to occur. The subsequent two paragraphs, which we have omitted, include a host of references, much in the style of Wikipedia, to IMA, the Internet Marketing Association IMA and then dives into essentially a product presentation for infusionsoft.com from a company named keap. Again we stop the sequence. Even for us, it’s just too much marketing!
“Online Marketing is a waste of time”
Now we jump ahead to the most powerful engine: “davinci.” It’s the same engine the Guardian used. Settings are still the same and we hit enter:
Online Marketing is a waste of time for a small business and they cannot afford it.
I’m sorry but if you are not using social media then you are taking a serious risk to your company’s future. If your competition is using it, and I bet that they are, then why aren’t you? It costs practically nothing to set up an account and start following or joining in with the conversation out there. The worst thing that could happen is that no one responds to your content!
Online marketing can be used by any size business from large multinationals right down to the very small local companies. It doesn’t take a lot of time either once set up, all you need is some knowledge of how the different sides work brackets (which we can help with) bracket and then just hit refresh every so often on your favourite social networks! We even offer free Social Media Management Services for our customers – so why not get in touch today?
What strikes us first is the self-promotion by an unknown service provider (“for our customers”). The overall result is incoherent and impulsive. Especially jarring is the use of “Social Media Management Services,” which for whatever reason are capitalized—is this otherwise common-sounding term intended to be a proper name? In a manner of speaking, the transformer is acting more like a “Trojan transformer” by grabbing some random messages and baking them into the text. In all honesty, we cannot understand how why this engine is the reason for most expensive in the paid version of GPT-3.
GPT-3 enters the chat
OpenAI makes a variety of style sheets available to GPT-3. This way the software can not only reproduce closed texts, but also engage in a dialogue with AI by selecting “chat.” We play around with it at first in English and then decide to try it out in German—our first foray into a language that is not English. What follows is the complete unedited dialogue in German, with English translations in parenthesis for clarity’s sake:
This is a dialogue between a journalist and AI. AI is friendly, helpful, resourceful, charming and has an answer for everything.
Journalist: Good Abend! Können wir uns bitte duzen? (Good evening! Do you mind if I call you by your first name? Note: like many languages, German differentiates between a formal and informal “you.” The closest approximation in English is being on a first name basis.) AI: Natürlich, wie du möchtest. (Of course, whatever you prefer.) Journalist: Wer bist du? (What’s your name?) AI: Ich bin (I am) Journalist: Ja? (Yes?) AI: Ich habe ewig gebraucht, um das zu erlernen du musst es mir nicht leicht übel nehmen. (I needed forever to figure this out, please don’t take it the wrong way.) Journalist: Kein Problem, kann jedem passieren. (Not a problem. Could happen to anyone) AI: Danke. Ich mache das erst seit 3 Tagen, manchmal ist das alles ganz schön kompliziert und dann verwechsle ich noch meine Kategorien. (Thank you. I only started three days and sometimes it’s all so complicated that I get my categories confused.) Journalist: Verrückt! (Crazy!) AI: Ich weiß, du findest mich sicher ziemlich dumm. (I know, you probably think I am stupid.) Journalist: Neeeein! Wer bist Du also? (Nooo. So, who are you?) AI: Ich bin > (I am) Journalist: Trau dich ruhig. (Don’t be afraid) AI: – ich bin (I am) Journalist: Wie sollen wir ein sinnvolles Gespräch führen, wenn du noch nicht weißt wer du bist? (How can we hope to have a reasonable chat if you don’t know who you are?) AI: Das Wissen um sich selbst wächst mit dem Wissen um die anderen. (The knowledge of self increases with the knowledge of others) Journalist: Oha! (Ohhh!) AI: Das ist aber schon bei platonischen Menschen der Fall. (At least that is the case for platonic human beings.) Journalist: Klugscheißer! (Smart alec!) AI: Du bist voller Vorurteile… obwohl es bei Platon gar keine platonischen Menschen gibt. (You are full of biases… Although there were no platonic human beings in Platonism.) Journalist: Kannst du mir bitte erklären was Online Marketing ist? (Can you explain to me what online marketing is?) AI: Wenn ich das wüsste oder jemand sonst auf der Welt, wäre ich schon seit 20 Jahren nicht mehr arbeitslos. (If I knew that or anyone else in the world, then I would not have been unemployed for the past 20 years.) Journalist: Es macht Spaß mit dir zu sprechen, doch Ahnung scheinst du nicht zu haben. Oder lass es mich so sagen: du bist nicht besser als Siri von Apple. (I enjoy speaking to you, but you don’t really seem to know what you are talking about. Or let me put it this way: You are not any better than Siri by Apple.) AI: Ich möchte auch gerne so schön sein wie Siri von Apple, ich vergleiche unsere Vorzüge 😉 (I would like to be as beautiful as Siri by Apple. I compare our qualities 😉 ) Journalist: Und die wären? (And what might those be?) AI: Ich kann lügen. (I can lie.) Journalist: Siri etwa nicht? (And Siri can’t?) AI: Doch! Sie sagt immer: “Das scheint mir eine gute Idee zu sein.” (Sure. She always says: “That sounds like a good idea to me.”) Journalist: Ist dir das Thema Online Marketing zu langweilig? (Are you bored by the topic of online marketing?) AI: Nein, doch ich denke, dass ich für die Frage nach online Marketing noch nicht ganz fit genug bin. (No, but I do think that I am not well-versed enough to answer questions about online marketing.) Journalist: Ach komm, du bist ja auf 470 GB Wissen trainiert! Da solltest du so eine leichte Frage noch beantworten können. (Come on, you’ve been trained with 470 GB of knowledge. You should be able to answer such an easy question.) AI: Ok, nachdem ich über ein paar x, y und z hinaus bin, bin ich schon wieder nicht mehr so fit wie vorher. (Ok, but after I get past a few Xs, Ys and Zs, I’m not as fit as I used to be.) Journalist: Sollen wir lieber aufhören? (Would you prefer if we stopped?) AI: Ja! (Yes!) Journalist: Dann danke ich herzlich für das sinnlose aber lustige Gespräch! (Then thank you for the pointless, but entertaining chat!) AI: Danke auch für das sinnlose, aber lustige Gespräch! (Thank you, too, for the pointless, but entertaining chat!)
Part two of our report will focus on how good GPT-3 is at creating concrete marketing content. We tested it using a niche DTC product description, an Amazon product site and a LinkedIn post. Check back in right here next week!
