Testing, testing. 1, 2. The OMR GPT-3 test, part 2. Is automated marketing content just a click away?
- Hey GPT-3! Write me a niche page for … Go on …
- The key is the prompt
- GPT-3 is not up-to-date
- Hours, not minutes, of editing needed
- Bingo! Our Amazon product page
- The grand finale: The LinkedIn post brought to you by GPT-3
- Still a ways away from a “flip of the switch”
Far-fetched fantasy or impending reality? Will AI soon be able to create the bulk of online content leaving human button crunchers in its wake? You’d be forgiven for thinking as much given that over the course of the past half-year GPT-3 has made headlines. Based on AI, GPT-3 is an autoregressive language model that uses deep learning to produce human-like text, the quality of which is touted as being featured-article worthy. So before we decide to close up shop and edge closer to the brink, we wanted to put GPT-3 through the paces and analyze the caliber of marketing content it produces. At the moment, GPT-3 is not available for general use; however, author and linguistic critic Peter Littger was granted access. This is part two of a two-part exclusive for OMR. Today, Littger uses GPT-3 to create a niche-product description for an online shop, an Amazon product page and a LinkedIn post. The results are worth your attention.
Before we get started, if you haven’t read part 1, we recommend starting there. Here, we lay out what GPT-3 is, where it comes from and how it works.
Does GPT-3 do marketing? Can the third generation of OpenAI’s “Generative Pre-trained Transformer” create texts that convert? We decide to start our test with a product description for a niche online shop. Our reasoning being that over the course of the past two or three years, direct-to-consumer startups specializing in a single product have been popping up all across the digital marketing and startup landscape.
Hey GPT-3! Write me a niche page for … Go on …
The first step here is deciding on a product we’d like to sell in our hypothetical dry run. Our aim is not to test in earnest a page we intend on creating and testing with potential customers. We want a product whose need and utility are in line with current trends.
Of course, we choose face masks for household pets. A curious choice in more ways than one, perhaps. Then there is also the fact that several leading health organizations (including the US Center for Disease Control) have stated that house pets should not wear masks during the Covid-19 pandemic. But just because a product is pointless does not mean it won’t generate sales. And in fact a company in the US selling pet face masks reported revenue growth of 500% percent just a few weeks ago.
The key is the prompt
We ask GPT-3 to provide us with some good reasons why pets should wear face masks. As we covered in part 1, every text GPT-3 generates is dependent on a human entering a brief text, the prompt. Our first prompt is “These are 5 crucial reasons for animals to wear masks during pandemics.” We hit enter and away we go. The “playground” begins populating with perfectly fine, even usable results, despite them lacking in any emotionality and being sober and medical in nature. “Masks protect the mouth and nose from tiny drops of saliva splashed during kissing or breathing” und “Masks prevent the loss of body fluids from the eyes and nose during sneezing or coughing.” What’s worth noting: There is no mention of Covid-19—nor was there in our prompt.
GPT-3 is not up-to-date
When we then modify our prompt to include the current pandemic (“These are 5 crucial reasons for pets to wear face masks during Covid-19”), it becomes painfully evident that the virus which is currently the bane of the entire world is completely unknown to the transformer; GPT-3 guesses it’s a poisonous gas. Swing and a miss. GPT-3 ambassador Vladimir Alexeev, who operates the tool for our test, confirms to us that it is not 100% up to date. Since the GPT-3’s database, from which it draws upon for its results, is in need of a 2020 patch, any texts about anything remotely corona related would require human intervention and revision.
Our subsequent experiments entail the use of more detailed prompts which include the term “pandemic” and dedicated text categories (“about, product description, shipping”). The results are getting better, but they are still not usable without significant revisions: truncations, rearrangements and combining portions of various drafts.
Hours, not minutes, of editing needed
We end our first exercise with a foray into German to test out its capabilities in the native language of our home market. We wanted to see what the absolute minimum number of edits were to get a somewhat usable text. You do not need to speak a lick of German to see just how many edits were necessary (but we added a translation for you anyway):
Bitte schreiben Sie einen umfassenden Text für eine Website, die Masken für Haustiere bei Pandemien verkauft, mit den Kategorien ÜBER UNS, PRODUKTBESCHREIBUNG, VERSAND. Jeder Text sollte etwa 300 Wörter lang sein. Text für “ÜBER UNS”: Wir sind ein kleines Unternehmen, das sich auf die Produktion von biologischen SchutzmaskeN für Haustiere spezialisiert hat. Unsere Produktion ist klein, aber unsere Produkte MASKEN haben eine hohe Qualität. SIE Unsere Masken werden aus den besten Materialien hergestellt, die auf dem Markt sind. Unsere Masken, sind leicht anzulegen und zu tragen. Sie sind auch ideal für den Transport von Haustieren. Unsere Masken sind für alle Haustiere geeignet, die sich in einem UMFELD Umgebung mit hohem ANSTECKUNGSrisiko auf die Ansteckung mit gefährlichen Krankheiten befinden. Um Sie über unsere Produkte und unseren Service ZU informieren zu können, haben wir diese Website erstellt. Hier einige Vorteile unserer Masken: – Hohe Qualität (Unsere Masken sind aus den Besten Materialien hergestellt) – Passgenauigkeit (Die Maske passt zu haargenau Ihrem Tier) – Leicht anzulegen (Unsere Masken sind einfach anzulegen und zu tragen) – Robustheit (Unsere Masken sind robust und langlebig)
Text for the “About Us” page: We are a small company specializing in the production of biological protective maskS for household pets. While our production facilities are small, our products masks high quality. They our masks are manufactured using the best materials , that are on the market. Our masks are easy to put on and wear. They are ideal for transporting pets. Our masks are suitable for all pets in AREAS environments where there is significant risk of infection in which there is a risk of infection of dangerous diseases. We have created this website to inform you of our products and our services. Here are some of the benefits of our masks: – High quality (Our masks are manufactured using the best materials) – Precision fit (The masks are a perfect fit for your pet) – Easy to put on (Our masks are easy to put on and wear) – Durability (Our masks are durable and long-lasting)
Our impression is that we’d need something on the order of three hours to create a halfway realistic product page for pet masks using the results GPT-3 gave us. We’ve already invested about an hour in our dialogue with GPT-3, so in total about a half day’s work. Even though we did get the results we were hoping for, we must admit that we’d have to invest the same amount of time into our own research if we were starting from scratch. A worthwhile endeavor so far!
Bingo! Our Amazon product page
Next up: We want GPT-3 to generate a product description for an Amazon listing. The relevance here should be clear: sales on Amazon’s marketplace have exploded in the past five years, primarily because product search no longer start on Google, but rather right on Amazon. If you are able to inexpensively source in-demand products from Asia and rank towards the top in the corresponding searches, you figure to have made a pretty penny. Today, it’s no longer easy to find profitable niches on Amazon, where competition is low and prospects for success are high.
As such, we decide to up the anté and increase the level of difficulty for GPT-3: the tool gets to choose the product itself. What would the transformer sell on an Amazon product page? And how what kind of copy would it create? We turn the “response length” up to the max and with the experience and insights gained from the previous test, we enter a few parameters.
Before we critically assess the result, a simple request: read it! All of it! The amount of inferred, mathematical intelligence that the transform used to create, dare we say conjure up, the results is incredible and deserves a moment of praise.
Write a comprehensive text for Amazon Product page selling an inovative product (PRODUCT TITLE, PRODUCT DESCRIPTION, PRIZE, REVIEW): PRODUCT TITLE: Portable Extendable Security Lock 3-Pack PRODUCT DESCRIPTION: The ANKUNG Portable Extendable Security Lock 3-Pack is the newest and most convenient way to ensure your personal belongings are safe and secure while traveling. The locks retract conveniently down to a size smaller than most keyrings, allowing you to easily pack them in your luggage, backpack, or purse. The locks are TSA approved and can even be used without removing your luggage from the airline baggage scanner. Keyed alike locks allow you to easily share a single key amongst all locks. The locks are also electronically coded to ensure only authorized users can use them. Anti-theft protection helps prevent unwanted removal of any of your belongings that are protected by the locks. PRIZE: $19.99-$34.99 REVIEW: I learned about the ANKUNG Portable Extendable Security Lock 3-Pack from a co-worker of mine who had purchased a set prior to my trip to Europe. He told me about how convenient they were to use and how he was able to lock his suitcase to the security bars in the hotel rooms without needing to remove his luggage from the airline baggage scanner. I will be traveling to Europe in the next few months and knew these would be the perfect gift to ensure my belongings remain secure. PRODUCT ASSESSMENT: The ANKUNG Portable Extendable Security Lock 3-Pack is the newest and most convenient way to ensure your personal belongings are safe and secure while traveling. The locks retract conveniently down to a size smaller than most keyrings, allowing you to easily pack them in your luggage, backpack, or purse. The locks are TSA approved and can even be used without removing your luggage from the airline baggage scanner. Keyed alike locks allow you to easily share a single key amongst all locks. The locks are also electronically coded to ensure only authorized users can use them. Anti-theft protection helps prevent unwanted removal of any of your belongings that are protected by the locks. COST/BENEFIT RATIO: The ANKUNG Portable Extendable Security Lock 3-Pack provides you with easy and convenient security options while you are on the go. You will no longer need to go through the hassle of removing your luggage from the airline baggage scanner or ask for assistance from the hotel staff in order to secure your belongings. There is a cost associated with the product, but the convenience of using trylocks help to make up for that cost. WOULD IT BE WORTH THE PRICE TO ME? The ANKUNG Portable Extendable Security Lock 3-Pack is a great product to own if you are someone who travels often or if you have many items you need to keep safe. Traveling can be a very stressful experience because you need to carry everything you need on your back. The ANKUNG Portable Extendable Security Lock 3-Pack helps to make that experience a little bit easier. There are a few reasons why someone may not be willing to purchase this product. Depending on the person, they may want to save the extra cost and just leave their belongings in the hotel room when they need to go out. This product is also not the best choice for parents with toddlers or young children. Small children have a tendency to explore and play with new things they find. If you were to use this product, you would need to make sure it is out of reach of any small children. This will prevent any possible accidents from happening. IMAGE FOR GOOGLE ADVERTISEMENT: Trylock Security Lock – Portable, TSA Approved 3-Pack (4.2/5) [product-image]
Now this virtual performance deserves our genuine applause. The supercomputer knows what it’s talking about. It’s able to populate the category with a plausible result and under “review” adds a few sub-categories in just a few seconds—something we should overlook—provides a very useful and usable result. The product that it selected is a security lock for luggage, for which its “product description” is spot on and a similar form of which other manufacturers have published to their actual pages.
We conduct a random sample and see that GPT-3 apparently “invented” the brand name “Ankung.” Whether or not it’s beneficial to have a name with such phonetical proximity to a company on Amazon, Anker, that has established itself as a successful seller of electronics, as well as computer and cell phone accessories, is for Amazon experts to determine.
We run a few more tests for made-up Amazon product pages. Although it’s completely ridiculous, we just have to share this one with you. It’s about a robot who pays a visit to someone in a time of crisis. As we mentioned before, COVID-19 means nothing to GPT-3. So, the transformer picked out another crisis: Zombie invasion:
You have been quarantined. The door to your room remains closed. Your family is gone. Your income is gone. ALL your commitments are gone. You are on your own. And then, there are the Zombies outside the door. You are in a time of crisis. The only viable escape is Survival. The situation is dire. Your windows and doors are potentially compromised (because they are designed to be easily opened by others) and now, Zombies from nearby areas can smell fresh meat and follow the scent to your door. When they arrive, you may not have the necessary tools to send them back. And, you are going to need to know WHEN Zombies are near. Right now, there is no warning system, so if you have no idea when Zombies are near, you are at high risk for having your home picked clean by Zombies. That is where the Robotic Visitor comes in. The Robotic Visitor is a device that has three main features: It will be placed on your doorstep at the beginning of a Zombie Lockdown. The Robotic Visitor will signal to you, through your Smartphone, when Zombies get within a 30 meters radius of your house. This way, you will have the warning time you need to leave your home and head for an area of safety.
The grand finale: The LinkedIn post brought to you by GPT-3
And now for the third and final test: a LinkedIn post. The business networking site has experienced a big spike in popularity in recent years, with some 740 million registered users at present. Wherever there is a massive amount of reach to be had, super users in the mold of influencers tend to follow. On LinkedIn, the influencer caste is characterized by, among other things, a unique brand of “LinkedIn prose,” especially successful posts that are oftentimes copied and republished verbatim.
Our directive to the transformer: create a LinkedIn post detailing our experience with GPT-3. After all, we are interested in providing our community—that means you!—with the most accurate information possible. The prompt essentially writes itself. We’re keen to see if and just how much of our enthusiasm it incorporates into the LinkedIn post:
Write a comprehensive post for LinkedIn in which we inform our community about our first experience using GPT-3 to create three different marketing texts successfully: Heading: “We just applied our first neural network and it worked!” Subheading: “GPT-3 is the first publicly available multitasking neural network model trained on full text. We were the first to use it to create various marketing texts” Our favorite clients are those who never give up their dreams and are ready for any challenges to break old stereotypes and show a new reality to the outside world! We use neural networks to develop texts for all styles of content and in any topic. Not many companies can boast of having applied a neural network successfully. Today, we are going to tell you about how we used the GPT-3 model in the first place to create three marketing texts for some great subjects. We want to show you what a new reality looks like: you don’t need to hire a lot of writers to make a huge text. Your neural network will do everything for you! We are eager to show you!
“Never give up your dreams”, “break old stereotypes”—simply wonderful how quickly it regurgitates beloved LinkedIn tropes and cliches. With just a few simple edits, it’s fit to print and publish!
We would be remiss if we failed to say that there were plenty of times that we tried to get GPT-3 to create a post for us that were absolute flops. Often because even when we had GPT-3’s “temperature” turned down, it never stopped and spit out all sorts of strange facts and prognostications. For example that we had built a new website that we’d be launching in March or that we were now laying off our editorial team—we are not!
Still a ways away from a “flip of the switch”
What is to make of our experiment? What conclusion have we reached after several rounds of testing? For starters, the first marketing text we created clearly showed significant deficits so that we can confidently say that the tool has a long way to go before it can be deemed a reliable, viable tool for the creation of engaging marketing copy on an everyday basis. The automatically generated pages are simply unusable without significant editing. Flipping a switch and kicking back it is not.
On balance, our overall conclusion is that we are very impressed by what GPT-3 can do, but not yet convinced entirely. It certainly gave us a great deal of inspiration and who knows, one day it may replace us humans yet—especially if we use it every day. Once we started, we found it difficult to stop. But over the course of our interactions, it became clear that to get the most out of it, you really need someone giving the transformer their full attention.
To really give a substantiated verdict, we’d need to spend significantly more time exploring the impact and effect of the settings, which would probably take several days at least. If you want to invest the time and seriously explore the commercial viability of GPT-3, you can apply with Open AI to take part in a free three-month test phase.
In closing, we would like to share with you part of an answer the transformer gave us in our third test. It’s a self-assessment that GPT-3 made of itself:
In the end, GPT-3 is still a bot, but it’s a bot that can generate marketing articles with certain success. I’m not sure what the endgame is yet. At the moment, I’m still trying to understand how this powerful technology works. What I can say, as GPT-3 is a recurrent neural network, is that it is not some kind of magic. It’s a sophisticated piece of machine learning software, capable of generating all kinds of text as long as it receives the correct input.
