果冻影院

Aims:

The module aims to equip students with knowledge of how to apply AI to problems from the creative industry; familiarity with basic ML-based algorithms and data structures to process digital media; ability of dealing with large scale data and training of machine intelligence; knowing rephrasing of existing concepts from digital media with tools from AI; and awareness of the difficulty of computed results and artistic freedom.

Intended learning outcomes:

On successful completion of the module, a student will be able to:

1. Formulate the diversity of image analysis and synthesis tasks as a machine learning process.
2. Understand theoretical and practical concepts allowing image processing and generation to become learnable and basic understanding of how to execute that learning.

Indicative content:

Creative industries such as print, feature films, music, fabrication or interactive media increasingly make use of multiple machine learning driven tools. This module enables students to contribute to a new shift of paradigm, where such tools become increasingly intelligent of the content being designed and the users designing them. This is enabled by machine learning, a new way of dealing with data and new forms of algorithms. We will cover an applied background of machine learning and focus on data structures particularly relevant for creative content such as images and video and focus on learnable algorithms that allow to machines to process them intelligently, such as convolutional neural networks.

The following are indicative of the topics the module will typically cover:

• Basic regression: Ability to explain basic (1D) data with a linear model and to make predictions. Knowledge of how what a linear model entails, also in higher dimensions. The resulting attitude could be both that a surprisingly simple thing can explain complex visual phenomena, yet others it cannot.
• Understanding of linearity and non-linearity: Participants will be able to judge if a linear or nonlinear model is adequate and how important non-linearity is for Visual Computing. They should have knowledge of how to represent such models. The key change in perspective is that almost all non-trivial processes are nonlinear.
• Classification: Ability to formalize a problem as classification code a simple classifier in. Knowledge that classification is regression over probabilities to be from a class.
• Neural networks: Students should be able to code up a simple perceptron from scratch and manipulate its parameters (e.g., number of units and layers), as well as control the non-linearities. They would understand that NN is just matrix multiplications followed by non-linearities.
• Audio/2D/3D images and pixel processing: Load and stored images/3D data, audio in the coding environment they use. Understand basic issues of sampling and representations like colour and luminance. The attitude should be changing such that not only the algorithm matters, but also the right representation is critical.
• Tunable image filters; convolutional neural networks: Participants will learn how to execute convolutions on images and how to set up their environment to optimize the parameters of the convolutions. The key knowledge is that learning convolutions allows to share learned parameters across the image.
• 3D meshes and point clouds: Until now data structures were regular, but now students will learn how to load irregular data (3D graphs, 3D meshes, 3D point clouds) into their environment and also how to process them as well as to learn tunable filters for them. They will deepen their knowledge of convolution by seeing how it extends to unstructured data, a generalization. The resulting attitude should be, that a well-defined convolution works on regular data as well as irregular ones.
• Ambiguity and style: Students will be explained, that under some conditions, multiple solutions are valid and that it can be adequate to report any of these. They will see how to change their code from working on paired data matching an output to a reference, it can be sufficient to match only statistics. The required skill is to change the loss and choose the right representation of these statistics.
• Generative modelling: We will first introduce the concept of a generative model, which takes simple parameters in low dimensions and maps them to complex objects in millions of dimensions. Using this requires the skill to load many exemplars and encode and decode them, which students can compose from components from before. The change of attitude should be, that a generative model is a very abstract way to 鈥渁ddress鈥� objects in a family of natural instances: images of faces, houses, cars, etc.
• Levels of supervision; Adversarial training: Students here will acquire the skill to replace the supervision in form of pairs that are mapped to each other by a paradigm where only random samples form a target distribution are given. To this end they have to understand that the loss is replaced by another network.

Requisites:

To be eligible to select this module an optional or elective, a student must be registered on a programme and year of study for which it is a formally available.