Draft:Computational aesthetics

Computational aesthetics is an interdisciplinary field at the intersection of computational science and aesthetics. It studies computational models and algorithms intended to quantify patterns of beauty and enable machines, including computers, robots and software agents, to perceive, interpret and generate aesthetically appealing works, thereby modelling aspects of how humans discover, appreciate and create beauty. Research in the field has addressed photography, painting, music, video, film, dance, design, sculpture, traditional opera, theatre, performance, food presentation and literature. Computational aesthetics uses methods from computer science and artificial intelligence to study aesthetic evaluation and art generation. In visual aesthetics research, it is also related to experimental aesthetics in psychology and computational neuroaesthetics in neuroscience. Computational methods have been used in photographic aesthetic assessment, computer-assisted photography, intelligent image management, image-capture quality assessment, outfit aesthetics, painting analysis and music aesthetics.

Definitions and perspectives

According to Encyclopædia Britannica, computational aesthetics is a subfield of artificial intelligence that aims to use computational methods to evaluate beauty in domains of human creative expression, including music, visual art, poetry and chess problems. It typically uses mathematical models that represent aesthetic features or rules, or statistical models combined with algorithms, to produce numerical aesthetic evaluations. Ideally, these evaluations correspond to those made by human experts. Computational evaluation can be useful when large numbers of works must be assessed and human evaluation is costly or limited; it seeks to provide consistent quantitative assessments despite variation among individual human judgments. Research in this area may also contribute to the study of human aesthetic perception.

Peking University professor Peng Feng has argued that computational aesthetics has become a reality, although its foundational questions remain unresolved. In his account, aesthetics commonly studies beauty, aesthetic experience and art; computational aesthetics, in a full sense, would require all three to be computationally tractable. He regards computational models of beauty as a central research topic, relates models of aesthetic experience to affective computing, and associates models of art with computable creativity.

Matjaž Perc of the University of Maribor has described two major tasks for computational approaches to aesthetics: quantifying beauty and modelling human aesthetic perception. The first includes quantitative analysis of artistic expression, such as cooking, painting, music and literature; the second concerns models of how people perceive beauty.

Related fields

Computational aesthetics differs from philosophical aesthetics, sociological aesthetics, experimental aesthetics and neuroaesthetics in its emphasis on building computational models of aesthetic phenomena and implementing algorithms.

History

In 1933, mathematician George David Birkhoff discussed the formula in his book Aesthetic Measure, where denotes order, complexity and an aesthetic measure. Although the book did not use the term computational aesthetics, later researchers have regarded it as an early attempt in the field.

In the mid-20th century, French engineer Abraham Moles and German philosopher Max Bense independently connected Claude Shannon's information theory with aesthetics, developing approaches to information-theoretic aesthetics. These approaches treat artworks as carriers of information and use models of communication to analyse the relationship between aesthetic objects and their observers, seeking to quantify aesthetic perception through methods drawn from the natural sciences.

In the 1970s, Elwood Shafer and colleagues developed a formula for estimating the aesthetic value of landscapes. Using 100 landscape photographs, they divided each image into regions such as sky, vegetation, non-vegetated land and bodies of water. They measured properties of these regions, including perimeter, area and vertical edges. After identifying six variables, including the perimeter of nearby vegetation and the area of water, and collecting public ratings of the photographs, they derived a formula for predicting aesthetic preference.

In 1993, Dutch scholars Remko Scha and Rens Bod published "Computationele Esthetica" (translated as "Computational Esthetics"), reviewing earlier computational work on aesthetics. The paper has been described as the earliest printed use of the term computational aesthetics.

In the early 1990s, Michael Leyton, a professor of psychology at Rutgers University, helped establish the International Society for Mathematical and Computational Aesthetics, intended to connect research in mathematics, computer science and aesthetics.

Modern developments

In 2004, researchers at Tsinghua University and Microsoft Research Asia proposed a method that used image features to distinguish photographs taken by professional photographers from those taken by home users, and extended it to overall image-quality scoring. The work was an early example of applying machine learning to image aesthetic assessment.

In 2005, Eurographics held its first Workshop on Computational Aesthetics in Graphics, Visualization and Imaging in Girona, Spain. The workshop addressed the use of computational methods in aesthetic judgment and provided a dedicated academic forum for the field.

In 2010, Jin Xin and colleagues proposed a content-based approach to image aesthetic assessment. Addressing the limited attention earlier methods paid to image content, they used portrait lighting as a case study and developed a model of its aesthetic qualities. In 2011, Luo Wei, Wang Xiaogang and Tang Xiaoou proposed selecting features according to photographic content, dividing photographs into seven categories: animals, plants, still life, architecture, landscapes, people and night scenes. These studies incorporated subject matter, such as portrait lighting or image category, into aesthetic assessment.

In 2012, Naila Murray and colleagues released AVA, a dataset for visual aesthetic analysis. It contains more than 250,000 images from a photography website, with user ratings, content labels and photographic-style annotations. The annotations provided data for training subsequent models. Studies including RAPID used AVA to evaluate model performance, and it became a commonly used benchmark for image aesthetic assessment.

In 2014, Lu Xin and colleagues introduced RAPID, a model that takes global and local views of images as inputs to a convolutional neural network and classifies their aesthetic quality using the AVA dataset. It was an early use of convolutional neural networks for image aesthetic assessment.

In 2018, Jin Xin and colleagues proposed a seven-level model of image aesthetic assessment tasks. Its aesthetic-perception levels comprise classification, scoring, score distributions and attributes; its aesthetic-appreciation levels comprise critique, question answering and guidance. The authors argued that higher-level tasks require more complex modelling of the subjective and dynamic aspects of aesthetic experience. This task framework has informed later work extending from deep learning models to multimodal large language models.

In 2023, the International Telecommunication Union approved Recommendation ITU-T F.740.4, specifying aesthetic attributes for image aesthetic assessment in mobile-terminal computational photography systems. It includes photographer, camera and user dimensions, providing a standardized basis for assessing mobile photography.

In 2024, Huang Yipo and colleagues introduced AesExpert, a multimodal model for image aesthetic perception, together with the AesMMIT instruction-tuning dataset. The model addresses tasks including aesthetic perception, rating and natural-language explanation. It illustrates the use of multimodal large language models for multiple aesthetic-assessment tasks.