Novel AI system can draw caricatures

Scientists have developed an artificial intelligence (AI) system that can sketch caricatures from a given portrait.

Caricature portrait drawing is a distinct art form where artists draw a person's face in an exaggerated manner, most times to elicit humour.

Automating this technique poses challenges due to the amount of intricate details and shapes involved and level of professional skills it takes to transform a person artistically from their real-life selves to a creatively exaggerated one.

"Compared to traditional graphics-based methods which define hand-crafted rules, our novel approach leverages big data and machine learning to synthesize caricatures from thousands of examples drawn by professional artists," said Kaidi Cao, a graduate student at Stanford University in the US.

"While existing style transfer methods have focused mainly on appearance style, our technique achieves both geometric exaggeration and appearance stylisation involved in caricature drawing," said Cao, who conducted the work during his internship at Microsoft.

The method enables users to automate caricatures of portraits, and can be applied to tasks such as creating caricatured avatars for social media, and designing cartoon characters.

The technique also has potential applications in marketing, advertising and journalism.

The researchers turned to a well-known technique in machine learning, Generative Adversarial Network (GAN), for unpaired photo-to-caricature translation to generate caricatures that preserve the identity of the portrait.

Called "CariGANs", the computational framework precisely models geometric exaggeration in photos (shapes of faces, specific angles) and appearance stylization (look, feel, pencil strokes, shadowing) via two algorithms the researchers have labelled, CariGeoGAN and CariStyGAN.

CariGeoGAN only models the geometry-to-geometry mapping from face photos to caricatures and CariStyGAN transfers the style appearance from caricatures to face photos without any deformation to the geometry of the original image.

The two networks are separately trained for each task so that the learning procedure is more robust, notes the researchers.

The CariGANs framework enables users to control the exaggeration degree in geometric and appearance style by dragging slides or giving an example caricature.

Researchers conducted perceptual studies to evaluate their framework's ability to generate caricatures of portraits that are easily recognisable and not overly distorted in shape and appearance style.

They demonstrated, through several examples, that existing methods resulted in unrecognisable caricature translation.

Their method successfully generated clearer, more accurate caricature depictions of portrait photos, as if they were hand drawn by a professional artist.

Currently, the focus of this work has centred on caricatures of people, primarily headshots or portraits.

In future work, the researchers intend to explore beyond facial caricature generation into full body or more complex scenes.

They are also interested in designing improved human-computer interaction (HCI) systems that would give users more freedom and user control on the machine learning-generated results.