Finding the art in artificial intelligence

When freelance artist Derrick Schultz needed to create an illustration of Helen Rosner, a food writer at The New Yorker, he instantly thought of the work of 16th-century artist Giuseppe Arcimboldo, who painted portraits of monarchs with their faces comprised of fruit.

To create something similar for a Q&A with Rosner, published by typeface foundry Commercial Type, Schultz turned to an AI text-to-image generator to replicate Arcimboldo’s style in Rosner’s likeness. These tools make it simple for designers — or anyone with an internet connection — to generate images based on a few lines of text describing the visual they want to create. 

Schultz typed out a text prompt, ran the generator overnight, and checked the results the next morning. When he had 16 AI-generated pieces, he collaged them together in Photoshop. The final result — an abstract portrait of Rosner made of food items like fried chicken and noodles — is incredibly detailed, with a painting-like aesthetic complete with brushstrokes and intricate, interwoven textures.

AI-generated imagery has exploded onto the digital illustration, graphic design, and photography scene, exciting some artists and evoking existential dread in others. Earlier in 2022, controversy around using AI as a creative medium erupted when artist Jason Allen won an art competition with his AI-generated entry, Théåtre D’opéra Spatial. Some critics thought his use of AI constituted cheating, while others expressed concern about precedent: If a machine can win such a competition, where else might they supplant human talent? More recently, Instagram and Facebook feeds have been flooded with AI-generated selfies created by the app Lensa AI, created by Prisma Labs. For some, it's a cool new mode of self expression, but for others, it's raising concerns about privacy for users who upload images of themselves, the rights of original artists whose work is used to train the AI, and racial and gender representation.

The idea of technology “cheapening” human-made art isn’t new — it’s been a subject of debate since the advent of the camera. But some artists and designers still worry that the rise of AI-generated art may have a lasting impact on their livelihoods. As more people play around with and explore what’s possible with AI image generators, the buzzy new technology has raised a few convoluted questions: Is AI art … art? And what are its implications for human creators and their careers?

How is AI-generated art created?

Though AI art has been around in some form since the 1960s, previous tools like Generative Adversarial Networks (GANs) required users to have advanced coding skills to operate them. But with the rise of text-to-image generators like DALL-E, Midjourney, Artbreeder, and Stable Diffusion, the process is now accessible to anyone — no computer science background necessary. 

AI image generators don’t, however, equate to “instant art.” For the Rosner illustration, the creative process still took about a month. “When you see a great AI image, the artist isn’t telling you that they generated 50 images and 49 were terrible,” says Schultz. Illustrating this point, Allen’s award-winning entry to the Colorado State Fair took around 80 hours, the artist told Smithsonian Magazine.

RELATED: What is.. artificial intelligence?

Using text-to-image generators involves a process called “prompt engineering.” This refers to crafting the snippets of text that the engine uses to spit out related images. In the demo section of DALLE-2’s site, for example, you can choose segments of text to create prompts like, “teddy bears mixing chemicals as mad scientists in a steampunk style.” The engine will then show you a series of images it thinks meet those criteria.

Ahmed Elgammal, a professor of computer science and founder of Rutgers University’s Art and Artificial Intelligence Laboratory, has been studying the technology that underpins engines like DALL-E for over a decade. He notes that while the algorithms that fuel such platforms vary, most are predicated upon existing images from open-source databases, such as Google’s TensorFlow and Meta’s Torch. Text-to-image generators typically learn by looking at millions of pairs of text and images in such databases, assigning each a numerical value, and then using those numbers to produce corresponding imagery when it recognizes specific keywords. 

Kaloyan Chernev, chief technology officer at Deep Dream Generator, an AI-driven text-to-image platform, notes that the company’s Text 2 Dream tool is trained on a “diffusion model.” This means it learns by taking image data and adding “noise,” or small dots that degrade the image quality. It then reverses that process, de-pixelating the image until the original reappears. After the AI has learned this capability, it can create visuals whole-cloth from a “pure noise” image and a text prompt.

Regardless of the specifics of their training, the image results AI engines produce aren’t simply replicas of existing visuals, but brand new creations — though they may amalgamate elements from the material in their source datasets. 

But is AI art … actually art?

About a century ago, Marcel Duchamp unveiled “Fountain” — a porcelain urinal scribbled with graffiti. He dubbed the piece “ready-made art,” and the backlash was instant and vehement. Critics decried the piece a hoax and an insult. But a hundred years later, Fountain appears in nearly every Art History 101 syllabus.

Elgammal sees a parallel with the controversy over AI-generated images. He is the founder of Playform, a no-code, AI-based platform that allows users to create images. “In my mind, AI-generated images are a new kind of ‘ready-made art,’” he says. He says the main impetus for creating the tool was to enable artists who lack coding skills to use AI in their creative process, from inspiration and ideation to asset preparation and refinement. “The creator’s role is really to reverse engineer and make something else with it. That’s when it becomes valuable.” 

Additionally, there is still a good amount of human skill required to conquer the art of prompt engineering. Despite its strengths, AI doesn’t understand context. When you type in “date,” it doesn’t know if you’re referring to a romantic foray, a fruit, or a day of the year. In other words, striking a Goldilocks balance with your prompt is an artform in itself.  Image results can also be tweaked by adjusting the prompt keywords as many times as the artist wants — or recontextualized for a different “feel,” from whimsical and cartoonish to moody or baroque. 

Artist credit and copyrighting, however, is still a big question mark — creators who contribute to AI image generators’ training data aren’t always compensated if part of their image ends up in someone else’s work. An early sign of what may be to come, some marketplaces like DeviantArt are now building artist protections into their terms and conditions.

Benjamin Lowy, who has been experimenting with AI image generator Midjourney in addition to his work as a traditional photojournalist, says there’s also a fine line when it comes to using these tools ethically, particularly in journalism. In a digital media landscape where deep fakes are on the rise and disinformation is rampant, the idea that you can make hyper-realistic images from scratch in seconds is “tremendously dangerous,” Lowy says.

“For me, it's important to have a distinct set of ethics about the images I create with my camera for journalism versus what I'm trying to do with Midjourney,” he says. When he posts AI-generated work on his Instagram, for instance, he ensures the images aren’t “too realistic.” He wants it to be obvious that the work is art — not journalism. As for the threat to his job, he thinks that’s less of an immediate concern. “Just like painting wasn't destroyed by photography, I don't think photography is going to be destroyed by AI,” he says. “But we have to counterbalance it and think seriously about its repercussions.”

AI art as a tool for work and for art

There are valid concerns about how the technology may impact some artists’ careers, especially as they present cost savings for companies who no longer need to send an entire photography crew to a far-flung locale or hire a designer for marketing collateral. 

But according to Chernev, creatives who have used Text 2 Dream report it actually reduces their workload by more than 50%. Artists use the platform for everything from creating marketing materials to designing characters for video games, and Chernev notes it’s particularly useful in generating thousands of images that are conceptually similar but visually distinct. 

“From just one character, you can easily create all the characters for the entire game,” he explains.

That said, learning the intricacies of AI image generation is a time investment — something Schultz, who teaches the subject at New York University (NYU), tries to convey to his students. “It's about figuring out how to fit [AI] into your existing practice to make things more efficient, or to say something you're already saying in a different way,” he says.

As for how the technology will evolve, experts in the field think that as more platforms emerge, the technology will only continue to get easier to use. 

While it can take a while to nail the right look and feel of an image today, AI technology improves rapidly, thanks largely to machine learning algorithms that automatically adapt and improve as more people use them. Schultz, for example, is excited for how things like resolution may improve, as well as how artists might begin using AI generators for more complex multimedia like video. 

Chernev agrees that the technology will keep evolving — and his is a rosy view of the future of the medium. 

“I truly believe that AI-generated content will not only help us with our jobs as artists and designers, but it will give us the opportunity to create completely new forms of art,” he says.  

READ MORE: Welcome to the new creative economy