A.I. Restoration from B&W photogrpahs of destroyed Klimt paintings

  

From the Smithsonian Magazine 

A.I. Digitally Resurrects Trio of Lost Gustav Klimt Paintings

Viewers can explore the works, newly restored to lush greens, blues, pinks and golds, through a Google Arts and Culture hub

Nora McGreevy

Daily Correspondent

October 12, 2021

---

Two of the newly colorized paintings: Jurisprudence (left) and Medicine (right) Courtesy Google Arts and Culture

Austrian painter Gustav Klimt created some of his best-known masterpieces during his so-called Golden Phase. Spanning the first decade of the 20th century, this period saw the artist produce such works as The Kiss (1908), a sensual scene of two lovers embracing in a field of multicolored flowers, and Portrait of Adele Bloch-Bauer I (1907), in which an elegant woman in a strikingly geometric dress stands surrounded by a sea of gold.

Not all of Klimt’s vibrant paintings survive today. Nazi looting during World War II led to the destruction of many prized Klimt works, including the Faculty Paintings: three enormous allegorical scenes titled Philosophy, Medicine and Jurisprudence.

To study these paintings, all of which were likely destroyed in a 1945 fire, art historians have long had to make do with black-and-white photographs. Thanks to machine learning, however, researchers have now restored historical images of the Faculty trio to approximations of their original colors, offering viewers a sense of what Klimt’s works looked like before their destruction, reports Mexican newspaper El Universal.

Watch as a black-and-white historical photograph transitions to an A.I.-generated approximation of color in Klimt's Medicine, one of the three Faculty Paintings destroyed by a 1945 fire. Courtesy Google Arts and Culture

To create the images, Google Arts and Culture and the Belvedere Museum in Vienna developed a tool that culled information about Klimt’s use of color from disparate sources. As Shanti Escalante-De Mattei reports for ARTnews, the data set included contemporary journalistic descriptions of the Faculty Paintings, 1 million pictures of the real world and 80 full-color reproductions of Klimt paintings from the same period.

Google engineer Emil Wallner spent nearly six months coding the artificial intelligence (A.I.) algorithm to generate color predictions. He tells El Universal that some aspects of the final paintings surprised the researchers: For instance, one might expect the starry sky featured in Klimt’s Philosophy to carry a blue tint, but the A.I.-colored version is saturated with an emerald haze, based partly on journalistic accounts that describe the painting’s greenish hue.

“It really creates a kind of shocking effect, because you expect it to be something else,” says Wallner in a Google Arts and Culture video.

In a statement, Franz Smola, a curator at the Belvedere Museum who worked on the restoration with Wallner, adds, “The result for me was surprising because we were able to color [Klimt’s works] even in the places where we had no knowledge. With machine learning, we have good assumptions that Klimt used certain colors.”

Art lovers can explore these colorful recreations through a new online hub dedicated to Klimt, created by Google in collaboration with more than 30 partners. The restored paintings are paired with a virtual exhibition, “Klimt vs. Klimt: The Man of Contradictions,” that explores the painter’s idiosyncratic life and legacy.

Klimt portrayed Jurisprudence, pictured here, in an abstract scene of deep reds that features three naked women wrapped in snakes. Courtesy Google Arts and Culture

Simon Rein, a program manager at Google Arts and Culture, tells El Universal that the show’s title refers to the “ambivalence of an artist navigating between tradition and modernity, a womanizing artist but also a devoted lover, a private person and at the same time a revolutionary.”

Viewers can also zoom in on 63 high-resolution images of the artist’s masterworks, which Google has organized together under a single digital roof in an Augmented Reality Pocket Gallery.    

The lengthy history of Klimt’s Faculty Paintings brims with scandal, conflict and unresolved mystery. The tale begins in 1894, when the University of Vienna commissioned the rebellious Klimt and contemporary Franz Matsch to create allegorical paintings for the ceiling of a luxurious new assembly hall.

Klimt contributed three enormous panels of his own strikingly original design. Standing more than 14 feet tall, the compositions showed wild, unrestrained visions of an allegorical world “roiling with sex, death and chaos,” writes Sam Gaskin for Ocula magazine.

Gustav Klimt, The Kiss, 1908 Courtesy Google Arts and Culture

Naked women wrapped in snakes, pregnant women, children, skeletons and wraithlike beings with cascading hair populate the canvases, highlighted by the shimmer of Klimt’s trademark gold foil. According to the Belvedere Museum, the artist employed references to mythology and more: To represent the concept of philosophy, for example, Klimt painted a sphinx, “the keeper of unsolvable secrets and riddles ... whose shadowy figure emerges from a cosmic fog.”

Critics decried the works as pornography, with one anonymous observer deeming them “deeply offensive to the general public.” After being asked several times to revise his creations, Klimt refused to deliver the paintings and returned his fee of 30,000 crowns (about €162,000 today), notes the Gustav Klimt Foundation.

The artist eventually sold the canvases to Serena Lederer, a wealthy Jewish woman who lived in Vienna and was an avid collector of Klimt’s works, the Austrian National Library explains in an essay on the Google hub.

Nazi forces looted Lederer’s collection of Klimt paintings as part of a widespread campaign of cultural pillaging across Europe. As art historian Tina Marie Storkovich reported for Austrian newspaper Die Presse in 2015, Klimt’s paintings were stored with other precious artworks at Schloss Immendorf, a castle in remote northeast Austria. (Lederer, for her part, was forced to flee to Hungary, where she died in 1943.)

A newly restored version of Klimt's Philosophy Courtesy Google Arts and Culture

Tragedy struck on May 8, 1945, when the castle burned down in a fire—likely the result of arson. Some historians argue that the Nazis started the blaze as they fled the approaching Soviet Red Army.

Exactly how many Klimt works were destroyed in the fire is unclear. While the Klimt Foundation lists 16 lost works, including the Faculty Paintings, the National Library suggests a figure closer to “at least ten.”

Even if some Klimt works did escape the 1945 fire, it’s unlikely that the Faculty Paintings were among the survivors.

“Due to the enormous size of the Faculty Paintings … removing them at any speed during the events in the castle in 1945 would hardly have been possible,” the library essay notes. “This fact supports the general assumption that these three Klimt paintings were not saved.”

Nora McGreevy  is a daily correspondent for The Smithsonian.

She can be reached through her website, noramcgreevy.com.

Smithsonian