Technology

Auto Added by WPeMatico

What is Dual Base ISO and why is it important?

Almost all modern day video and electronic stills cameras have the ability to change the brightness of the images they record. The most common way to achieve this is through the addition of gain or through the amplification of the signal that comes from the sensor. 

On older video cameras this amplification was expressed as dB (decibels) of gain. A brightness change of 6dB is the same as one stop of exposure or a doubling of the ISO rating. But you must understand that adding gain to raise the ISO rating of a camera is very different to actually changing the sensitivity of a camera.

The problem with increasing the amplification or adding gain to the sensor output is that when you raise the gain you increase the level of the entire signal that comes from the sensor. So, as well as increasing the levels of the desirable parts of the image, making it brighter, the extra gain also increases the amplitude of the noise, making that brighter too.

Imagine you are listening to an FM radio. The signal starts to get a bit scratchy, so in order to hear the music better you turn up the volume (increasing the gain). The music will get louder, but so too will the scratchy noise, so you may still struggle to hear the music. Changing the ISO rating of an electronic camera by adding gain is little different. When you raise the gain the picture does get brighter but the increase in noise means that the darkest things that can be seen by the camera remain hidden in the noise which has also increased in amplitude.

Another issue with adding gain to make the image brighter is that you will also normally reduce the dynamic range that you can record.

Screenshot-2019-11-27-at-18.21.19-1024x576 What is Dual Base ISO and why is it important?

This is because amplification makes the entire signal bigger. So bright highlights that may be recordable within the recording range of the camera at 0dB or the native ISO may be exceed the upper range of the recording format when even only a small amount of gain is added, limiting the high end.

Screenshot-2019-11-27-at-18.22.59-1024x576 What is Dual Base ISO and why is it important?
Adding gain amplifies the brighter parts of the image so they can now exceed the cameras recording range.

 

At the same time the increased noise floor masks any additional shadow information so there is little if any increase in the shadow range.

Reducing the gain doesn’t really help either as now the brightest parts of the image from the sensor are not amplified sufficiently to reach the cameras full output. Very often the recordings from a camera with -3dB or -6dB  of gain will never reach 100%.

Screenshot-2019-11-27-at-18.23.08-1024x576 What is Dual Base ISO and why is it important?
Negative gain may also reduce the cameras dynamic range.

A camera with dual base ISO’s works differently.

Instead of adding gain to increase the sensitivity of the camera a camera with a dual base ISO sensor will operate the sensor in two different sensitivity modes. This will allow you to shoot at the low sensitivity mode when you have plenty of light, avoiding the need to add lots of ND filters when you want to obtain a shallow depth of field. Then when you are short of light you can switch the camera to it’s high sensitivity mode.

When done correctly, a dual ISO camera will have the same dynamic range and colour performance in both the high and low ISO modes and only a very small difference in noise between the two.

How dual sensitivity with no loss of dynamic range is achieved is often kept very secret by the camera and sensor manufacturers. Getting good, reliable and solid information is hard. Various patents describe different methods. Based on my own research this is a simplified description of how I believe Sony achieve two completely different sensitivity ranges on both the Venice and FX9 cameras.

The image below represents a single microscopic pixel from a CMOS video sensor. There will be millions of these on a modern sensor. Light from the camera lens passes first through a micro lens and colour filter at the top of the pixel structure. From there the light hits a part of the pixel called a photodiode. The photodiode converts the photons of light into electrons of electricity. 

Screenshot-2019-11-27-at-17.40.52-1024x605 What is Dual Base ISO and why is it important?
Layout of a sensor pixel including the image well.

In order to measure the pixel output we have to store the electrons for the duration of the shutter period. The part of the pixel used to store the electrons is called the “image well” (in an electrical circuit diagram the image well would be represented as a capacitor and is often simply the capacitance of the the photodiode itself).

Screenshot-2019-11-27-at-17.41.00-1024x605 What is Dual Base ISO and why is it important?
The pixels image well starts to fill up and the signal output level increases.

Then as more and more light hits the pixel, the photodiode produces more electrons. These pass into the image well and the signal increases. Once we reach the end of the shutter opening period the signal in the image well is read out, empty representing black and full representing very bright.

Screenshot-2019-11-27-at-17.41.09-1024x605 What is Dual Base ISO and why is it important?

Consider what would happen if the image well, instead of being a single charge storage area was actually two charge storage areas and there is a way to select whether we use the combined image well storage areas or just one part of the image well.

Screenshot-2019-11-27-at-18.10.02-1024x575 What is Dual Base ISO and why is it important?
Dual ISO pixel where the size of the image well can be altered.

When both areas are connected to the pixel the combined capacity is large. So it will take more electrons to fill it up, so more light is needed to produce the increased amount of electrons. This is the low sensitivity mode. 

If part of the charge storage area is disconnected and all of the photodiodes output is directed into the remaining, now smaller storage area then it will fill up faster, producing a bigger signal more quickly. This is the high sensitivity mode.

What about noise?

In the low sensitivity mode with the bigger storage area any unwanted noise generated by the photodiode will be more diluted by the greater volume of electrons, so noise will be low. When the size of the storage area or image well is reduced the noise from the photodiode will be less diluted so the noise will be a little bit higher. But overall the noise will be much less that that which would be seen if a large amount of extra gain was added.

Note for the more technical amongst you: Strictly speaking the image well starts full. Electrons have a negative charge so as more electrons are added the signal in the image well is reduced until maximum brightness output is achieved when the image well is empty!!

As well as what I have illustrated above there may be other things going on such as changes to the amplifiers that boost the pixels output before it is passed to the converters that convert the pixel output from an analog signal to a digital one. But hopefully this will help explain why dual base ISO is very different to the conventional gain changes used to give electronic cameras a wide range of different ISO rating.

On the Sony Venice and the PXW-FX9 there is only a very small difference between the noise levels when you switch from the low base ISO to the high one. This means that you can pick and choose between either base sensitivity level depending on the type of scene you are shooting without having to worry about the image becoming unusable due to noise.

NOTE: This article is my own work and was prepared without any input from Sony. I believe that the dual ISO process illustrated above is at the core of how Sony achieve two different base sensitivities on the Venice and FX9 cameras. However I can not categorically guarantee this to be correct.


What is Dual Base ISO and why is it important? was first posted on November 27, 2019 at 5:55 pm.
©2018 “XDCAM-USER.COM“. Use of this feed is for personal non-commercial use only. If you are not reading this article in your feed reader, then the site is guilty of copyright infringement. Please contact me at contact@xdcam-user.com

How we identified brain patterns of consciousness

How we identified brain patterns of consciousness

Brain connections have been linked to consciousness.
whitehoune/Shutterstock

Davinia Fernández-Espejo, University of Birmingham

Humans have learned to travel through space, eradicate diseases and understand nature at the breathtakingly tiny level of fundamental particles. Yet we have no idea how consciousness – our ability to experience and learn about the world in this way and report it to others – arises in the brain.

In fact, while scientists have been preoccupied with understanding consciousness for centuries, it remains one of the most important unanswered questions of modern neuroscience. Now our new study, published in Science Advances, sheds light on the mystery by uncovering networks in the brain that are at work when we are conscious.

It’s not just a philosophical question. Determining whether a patient is “aware” after suffering a severe brain injury is a huge challenge both for doctors and families who need to make decisions about care. Modern brain imaging techniques are starting to lift this uncertainty, giving us unprecedented insights into human consciousness.

For example, we know that complex brain areas including the prefrontal cortex or the precuneus, which are responsible for a range of higher cognitive functions, are typically involved in conscious thought. However, large brain areas do many things. We therefore wanted to find out how consciousness is represented in the brain on the level of specific networks.

The reason it is so difficult to study conscious experiences is that they are entirely internal and cannot be accessed by others. For example, we can both be looking at the same picture on our screens, but I have no way to tell whether my experience of seeing that picture is similar to yours, unless you tell me about it. Only conscious individuals can have subjective experiences and, therefore, the most direct way to assess whether somebody is conscious is to ask them to tell us about them.




Read more:
The way you see colour depends on what language you speak


But what would happen if you lose your ability to speak? In that case, I could still ask you some questions and you could perhaps sign your responses, for example by nodding your head or moving your hand. Of course, the information I would obtain this way would not be as rich, but it would still be enough for me to know that you do indeed have experiences. If you were not able to produce any responses though, I would not have a way to tell whether you’re conscious and would probably assume you’re not.

Scanning for networks

Our new study, the product of a collaboration across seven countries, has identified brain signatures that can indicate consciousness without relying on self-report or the need to ask patients to engage in a particular task, and can differentiate between conscious and unconscious patients after brain injury.

When the brain gets severely damaged, for example in a serious traffic accident, people can end up in a coma. This is a state in which you lose your ability to be awake and aware of your surrounding and need mechanical support to breathe. It typically doesn’t last more than a few days. After that, patients sometimes wake up but don’t show any evidence of having any awareness of themselves or the world around them – this is known as a “vegetative state”. Another possibility is that they show evidence only of a very minimal awareness – referred to as a minimally conscious state. For most patients, this means that their brain still perceives things but they don’t experience them. However, a small percentage of these patients are indeed conscious but simply unable to produce any behavioural responses.

fMRI scanner.
wikipedia

We used a technique known as functional magnetic resonance imaging (fMRI), which allows us to measure the activity of the brain and the way some regions “communicate” with others. Specifically, when a brain region is more active, it consumes more oxygen and needs higher blood supply to meet its demands. We can detect these changes even when the participants are at rest and measure how it varies across regions to create patterns of connectivity across the brain.

We used the method on 53 patients in a vegetative state, 59 people in a minimally conscious state and 47 healthy participants. They came from hospitals in Paris, Liège, New York, London, and Ontario. Patients from Paris, Liège, and New York were diagnosed through standardised behavioural assessments, such as being asked to move a hand or blink an eye. In contrast, patients from London were assessed with other advanced brain imaging techniques that required the patient to modulate their brain to produce neural responses instead of external physical ones – such as imagining moving one’s hand instead of actually moving it.

In consciousness and unconsciousness, our brains have different modes to self-organise as time goes by. When we are conscious, brain regions communicate with a rich temperament, showing both positive and negative connections.
Credit: E. Tagliazucchi & A. Demertzi

We found two main patterns of communication across regions. One simply reflected physical connections of the brain, such as communication only between pairs of regions that have a direct physical link between them. This was seen in patients with virtually no conscious experience. One represented very complex brain-wide dynamic interactions across a set of 42 brain regions that belong to six brain networks with important roles in cognition (see image above). This complex pattern was almost only present in people with some level of consciousness.

Importantly, this complex pattern disappeared when patients were under deep anaesthesia, confirming that our methods were indeed sensitive to the patients’ level of consciousness and not their general brain damage or external responsiveness.

Research like this has the potential to lead to an understanding of how objective biomarkers can play a crucial role in medical decision making. In the future it might be possible to develop ways to externally modulate these conscious signatures and restore some degree of awareness or responsiveness in patients who have lost them, for example by using non-invasive brain stimulation techniques such as transcranial electrical stimulation. Indeed, in my research group at the University of Birmingham, we are starting to explore this avenue.

Excitingly the research also takes us as step closer to understanding how consciousness arises in the brain. With more data on the neural signatures of consciousness in people experiencing various altered states of consciousness – ranging from taking psychedelics to experiencing lucid dreams – we may one day crack the puzzle.The Conversation

Davinia Fernández-Espejo, Senior Lecturer, School of Psychology and Centre for Human Brain Health, University of Birmingham

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post How we identified brain patterns of consciousness appeared first on Interalia Magazine.

We’ve discovered the world’s largest drum – and it’s in space

We’ve discovered the world’s largest drum – and it’s in space

The Earth’s magnetosphere bangs like a drum.
E. Masongsong/UCLA, M. Archer/QMUL, H. Hietala/UTU

Martin Archer, Queen Mary University of London

Universities in the US have long wrangled over who owns the world’s largest drum. Unsubstantiated claims to the title have included the “Purdue Big Bass Drum” and “Big Bertha”, which interestingly was named after the German World War I cannon and ended up becoming radioactive during the Manhattan Project.

Unfortunately for the Americans, however, the Guinness Book of World Records says a traditional Korean “CheonGo” drum holds the true title. This is over 5.5 metres in diameter, some six metres tall and weighs over seven tonnes. But my latest scientific results, just published in Nature Communications, have blown all of the contenders away. That’s because the world’s largest drum is actually several tens of times larger than our planet – and it exists in space.

You may think this is nonsense. But the magnetic field (magnetosphere) that surrounds the Earth, protecting us by diverting the solar wind around the planet, is a gigantic and complicated musical instrument. We’ve known for 50 years or so that weak magnetic types of sound waves can bounce around and resonate within this environment, forming well defined notes in exactly the same way wind and stringed instruments do. But these notes form at frequencies tens of thousands of times lower than we can hear with our ears. And this drum-like instrument within our magnetosphere has long eluded us – until now.

Massive magnetic membrane

The key feature of a drum is its surface – technically referred to as a membrane (drums are also known as membranophones). When you hit this surface, ripples can spread across it and get reflected back at the fixed edges. The original and reflected waves can interfere by reinforcing or cancelling each other out. This leads to “standing wave patterns”, in which specific points appear to be standing still while others vibrate back and forth. The specific patterns and their associated frequencies are determined entirely by the shape of the drum’s surface. In fact, the question “Can one hear the shape of a drum?” has intrigued mathematicians from the 1960s until today.

The outer boundary of Earth’s magnetosphere, known as the magnetopause, behaves very much like an elastic membrane. It grows or shrinks depending on the varying strength of the solar wind, and these changes often trigger ripples or surface waves to spread out across the boundary. While scientists have often focused on how these waves travel down the sides of the magnetosphere, they should also travel towards the magnetic poles.

Physicists often take complicated problems and simplify them considerably to gain insight. This approach helped theorists 45 years ago first demonstrate that these surface waves might indeed get reflected back, making the magnetosphere vibrate just like a drum. But it wasn’t clear whether removing some of the simplifications in the theory might stop the drum from being possible.

It also turned out to be very difficult to find compelling observational evidence for this theory from satellite data. In space physics, unlike say astronomy, we’re usually dealing with the completely invisible. We can’t just take a picture of what’s going on everywhere, we have to send satellites out and measure it. But that means we only know what’s happening in the locations where there are satellites. The conundrum is often whether the satellites are in the right place at the right time to find what you’re looking for.

Over the past few years, my colleagues and I have been further developing the theory of this magnetic drum to give us testable signatures to search for in our data. We were able to come up with some strict criteria that we thought could provide evidence for these oscillations. It basically meant that we needed at least four satellites all in a row near the magnetopause.

Thankfully, NASA’s THEMIS mission gave us not four but five satellites to play with. All we had to do was find the right driving event, equivalent to the drum stick hitting the drum, and measure how the surface moved in response and what sounds it created. The event in question was a jet of high speed particles impulsively slamming into the magnetopause. Once we had that, everything fell into place almost perfectly. We have even recreated what the drum actually sounds like (see the video above).

This research really goes to show how tricky science can be in reality. Something which sounds relatively straightforward has taken us 45 years to demonstrate. And this journey is far from over, there’s plenty more work to do in order to find out how often these drum-like vibrations occur (both here at Earth and potentially at other planets, too) and what their consequences on our space environment are.

This will ultimately help us unravel what kind of rhythm the magnetosphere produces over time. As a former DJ, I can’t wait – I love a good beat.The Conversation

Martin Archer, Space Plasma Physicist, Queen Mary University of London

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post We’ve discovered the world’s largest drum – and it’s in space appeared first on Interalia Magazine.

Altered States: 2D digital displays become 3D reality – Digital Technology Lets You Touch Great Art

It’s a natural impulse to reach out and touch an original artwork, perhaps to feel the strong brushstrokes in van Gogh’s Starry Night or to trace the shape of a compelling sculpture. You can’t though, and for good reason: a multitude of individual touches would soon damage the work, so museums sensibly use “Please don’t touch” signs, velvet ropes and alert guards to keep viewers at a distance. It helps that those same museums have put their collections online so you can appreciate great art right on your digital device. However, even at high resolution, images on flat screens do not clearly show surface texture or convey volumes in space. But now researchers in art and technology are finding ways for viewers to experience the texture of artworks in 2D and the solidity of those in 3D.

The missing third dimension is significant even for flat works, which typically show the texture of the background paper or canvas, or of the pigment. Some nominally two-dimensional works are inherently textured, such as Canadian artist Brian Jungen’s People’s Flag (2006), an immense vertical hanging piece made of red textiles. Helen Fielding of the University of Western Ontario has perceptively noted how vision and touch intertwine in this work:

As my eyes run across the texture of the flag, I can almost feel the textures of the materials I see; my hands know the softness of wool, the smoothness of vinyl. Though touching the work is prohibited…my hands are drawn to the fabrics, subtly reversing the priority of vision over touch…

Textural features like these are a material record of the artist’s effort that enhances a viewer’s interaction with the work. Such flat but textured works are art in “2.5D” because they extend only slightly into the third dimension. Now artworks shown in 2.5D and 3D on flat screens and as solid 3D models are giving new pleasures and insights to art lovers, curators, and scholars. As exact copies, these replicas can also help conserve fragile works while raising questions about the meaning of original art.

One approach, developed at the Swiss Federal Institute of Technology (EPFL) in Lausanne, creates a digital 2.5D image of an artwork by manipulating its lighting. Near sunset, when the sun’s rays enter a scene at an angle, small surface elevations cast long shadows that make them stand out. Similarly, the EPFL process shines a simulated light source onto a digital image. As the source is moved, it produces highlights and shadows that enhance surface details to produce a quasi-3D appearance.

This approach has links to CGI, computer-generated imagery, the technology that creates imaginary scenes and characters in science fiction and fantasy films. One powerful CGI tool is an algorithm called the bidirectional scattering distribution function (BSDF). For every point in an imagined scene, the BSDF shows how incoming light traveling in any direction would be reflected or transmitted to produce the outgoing ray seen by a viewer. The result fully describes the scene for any location of the light source.

In films, the BSDF is obtained from optical theory and the properties of the imaginary scene. The EPFL group, however, generated it from real art. In 2014, they illuminated a pane of stained glass with light from different directions and recorded the results with a high-resolution camera, creating a BSDF and showing that the method works for nearly planar items. This approach has been commercialized by Artmyn, a Swiss company co-founded by Luic Baboulaz who led the EPFL team. Artmyn makes 2.5D digital images of artworks by lighting them with LEDs at different visible wavelengths to provide color fidelity, and at infrared and ultraviolet wavelengths to further probe the surface. The result is a BSDF with up to a terabyte of data.

As an illustration, Artmyn has worked with Sotheby’s auction house to digitize two Marc Chagall works: Le Printemps (1975, oil on canvas), a village scene with a couple embracing, and Dans L’Atelier (1980, tempera on board), an artist’s studio. The Artmyn software lets a viewer zoom from the full artwork down to the fine scale of the weave of the canvas, while moving the lighting to display blobs, islands and layers of pigment. This reveals how Chagall achieves his effects and clearly illustrates the difference between oils and tempera as artistic media. Currently in process for similar digitization, Baboulaz told me, are a Leonardo da Vinci painting and a drawing, in recognition of the 500th anniversary of his death this year.

Artmyn has also digitized cultural artifacts such as a Sumerian clay tablet circa 2,000 BCE covered in cuneiform script; signatures and letters from important figures in the American Revolution; and a digital milestone, the original Apple-1 computer motherboard. These 2.5D images display signs of wear and of their creator’s presence that hugely enhance a viewer’s visceral appreciation of the real objects and their history.

For the next step, creating full 3D representations and physical replicas, the necessary data must be obtained without touching the original. One approach is LIDAR (light detection and ranging), where a laser beam is scanned over the object and reflected back to a sensor. The distance from the laser to each point on the object’s surface is found from the speed of light and its travel time, giving a map of the surface topography. LIDAR is most suitable for big artifacts such as a building façade at a coarse resolution of millimeters. Other approaches yield finer detail. In the triangulation method, for instance, a laser puts a dot of light on the object while a nearby camera records the dot’s location, giving data accurate to within 100 micrometers (0.1 millimeter). Copyists typically combine scanning methods to obtain the best surface replication and color rendition.

One big 3D copying effort is underway at the Smithsonian Institution, whose 19 museums preserve 155 million cultural and historic artifacts and artworks. Since 2013, the Smithsonian has put over 100 of these online as interactive 3D displays that can be viewed from different angles, and as data for 3D printers so people can make their own copies. The objects, chosen for popularity and diversity, include the original 1903 Wright Brothers flyer; a highly decorated 2nd century BCE Chinese incense burner; costume boots from the Broadway musical The Wiz from 1975; a mask of Abraham Lincoln’s face from shortly before his assassination in 1865; and for the 50th anniversary of the Apollo 11 moon landing, astronaut Neil Armstrong’s spacesuit. Recently added is a small 3D version of a full-sized dinosaur skeleton display at the National Museum of Natural History showing a T-rex attacking a triceratops, for which hundreds of bones were scanned by LIDAR and other methods.

A different goal animates the 3D art and technology studio Factum Arte in Madrid, Spain. Founded by British artist Adam Lowe in 2001, Factum Arte protects cultural artifacts by copying them, using its own high-resolution 3D scanning, printing and fabrication techniques.

Museums already use copies to preserve sensitive artworks on paper that need long recovery times in darkness and low humidity between showings. During these rests, the museum displays instead high-quality reproductions (and informs patrons that they are doing so). In a recent interview entitled “Datareality,” Adam Lowe expressed his similar belief that an artistically valid copy can provide a meaningful viewing experience while preserving a fragile original. One of his current projects is to replicate the tombs of the pharaohs Tutankhamun (King Tut) and Seti I, and queen Nefertari, in the Egyptian Valley of the Kings. The tombs were sealed by their builders, but once opened, they are deteriorating due to the throngs of visitors. As Lowe recently explained, “by going to see something that was designed to last for eternity, but never to be visited, you’re contributing to its destruction.”

The copies, approved by Egypt’s Supreme Council of Antiquities, will give visitors alternate sites to enter and view. At a resolution of 0.1 millimeter, the copies provide exact reproductions of the intricate colored images and text adorning thousands of square meters in the tombs. The first copy, King Tut’s burial chamber, was opened to the public in 2014, and in 2018, Factum Arte displayed its copied “Hall of Beauties” from the tomb of Seti I.

Earlier, Factum Arte had copied the huge Paolo Veronese oil on canvas The Wedding Feast at Cana (1563, 6.8 meters x 9.9 meters), which shows the biblical story where Jesus changes water into wine. The original was plundered from its church in Venice by Napoleon’s troops in 1797 and now hangs in the Louvre. The full-size copy, however, commissioned by the Louvre and an Italian foundation, was hung back at the original church site in 2007.

Factum Arte’s efforts highlight the questions that arise as exact physical copies of original art become available. Museums, after all, trade in authenticity. They offer viewers the chance to stand in the presence of a work that once felt the actual hands of its creator. But if the copy is indistinguishable from the work, does that dispel what the German cultural critic Walter Benjamin calls the “aura” of the original? In his influential 1935 essay The Work of Art in the Age of Mechanical Reproduction, he asserted that a copy lacks this aura:

In even the most perfect reproduction, one thing is lacking: the here and now of the work of art – its unique existence in a particular place. It is this unique existence – and nothing else – that bears the mark of the history to which the work has been subject.

The Factum Arte reproductions show that “original vs copy” is more nuanced than Benjamin indicates. The Egyptian authorities will charge a higher fee to enter the original tombs and a lower one for the copies, giving visitors the chance to feel the experience without causing damage. Surely this helps preserve a “unique existence in a particular place” for the original work. And for the repatriated Wedding at Cana, Lowe tellingly points out that a copy can bring its own authenticity of history and place:

Many people started to question about whether the experience of seeing [the copy] in its correct setting, with the correct light, in dialogue with this building that it was painted for, is actually more authentic than the experience of seeing the original in the Louvre.

We are only beginning to grasp what it means to have near-perfect copies of artworks, far beyond what Walter Benjamin could have imagined. One lesson is that such a copy can enhance an original rather than diminish it, by preserving it, and by recovering or extending its meaning.

Copying art by technical means has often been an unpopular idea. Two centuries ago, the English artist William Blake, known for his unique personal vision, expressed his dislike of mechanical reproduction such as imposing a grid to copy an artwork square by square. Current technology can also often stand rightfully accused of replacing the human and the intuitive with the robotic and the soulless. But properly used, today’s high-tech replications show that technology can also enlarge the power and beauty of an innately human impulse, the need to make art.

The post Altered States: 2D digital displays become 3D reality – Digital Technology Lets You Touch Great Art appeared first on Interalia Magazine.

Can You Shoot Anamorphic with the PXW-FX9?

The simple answer as to whether you can shoot anamorphic on the FX9 or not, is no, you can’t. The FX9 certainly to start with, will not have an anamorphic mode and it’s unknown whether it ever will. I certainly wouldn’t count on it ever getting one (but who knows, perhaps if we keep asking for it we will get it).

But just because a camera doesn’t have a dedicated anamorphic mode it doesn’t mean you can’t shoot anamorphic. The main thing you won’t have is de-squeeze. So the image will be distorted and stretched in the viewfinder. But most external monitors now have anamorphic de-squeeze so this is not a huge deal and easy enough to work around.

1.3x or 2x Anamorphic?

With a 16:9 or 17:9 camera you can use 1.3x anamorphic lenses to get a 2:39 final image. So the FX9, like most 16:9 cameras will be suitable for use with 1.3x anamorphic lenses out of the box.

But for the full anamorphic effect you really want to shoot with 2x  anamorphic lenses. A 2x anamorphic lens will give your footage a much more interesting look than a 1.3x anamorphic. But if you want to produce the classic 2:39 aspect ratio normally associated with anamorphic lenses you need a 4:3 sensor rather than a 16:9 one.

What about Full Frame 16:9?

But -that’s super 35mm 4:3 or s35mm open gate. The FX9 has a 6K full frame sensor and a full frame sensor is bigger, most importantly it’s taller than s35mm and tall enough for use with a 2x s35 anamorphic lens! The FX9 sensor is approx 34mm wide and 19mm tall in FF6K mode.

In comparison the Arri  35mm 4:3 open gate sensor is area is 28mm x 18mm and we know this works very well with 2x Anamorphic lenses. The important bit here is the height – 18mm with the Arri open gate and 18.8mm for the FX9 in Full Frame Scan Mode.

Crunching the numbers.

If you do the maths – Start with the FX9 in FF mode and use a s35mm 2x anamorphic lens. 

Because the image is 6K subsampled to 4K the resulting recording will have 4K resolution.

But you will need to crop the sides of the final recording by roughly 30% to remove the left/right vignette caused by using an anamorphic lens designed for s35 on a full frame sensor (the exact amount of crop will depend on the lens). This then results in a 2.8K ish resolution image depending on how much you need to crop.

4K Bayer doesn’t won’t give 4K resolution.

That doesn’t seem very good until you consider that a 4K 4:3 bayer sensor will only yield about 2.8K resolution anyway.

And Arri’s s35mm cameras are open gate 3.2K bayer sensors so will result in an even lower resolution image, perhaps around 2.2K? Do remember that the original Arri ALEV sensor was designed when 2K was the norm for the cinema and HD TV was still new. The Arri super 35 cameras were for a long time the gold standard for Anamorphic. But now cameras like Sony’s Venice that can shoot the equivalent of 6K open gate or 6K 4:3 and 6:5 are now taking over.

What about Netflix?

While Netflix normally insist on a minimum of a sensor with 4K pixels horizontally for capture, they are permitting sensors with lower horizontal pixel counts to be used for anamorphic capture because the increased sensor height needed for 2x anamorphic means that there are more pixels vertically. The total pixel count when using a camera such as the Arri LF with a super 35mm 2x anamorphic lens is 3148 x 2636 pixels. Thats a total of  8 megapixels which is similar to the 8 megapixel total pixel count of a 4K 16:9 sensor. The argument is that the total captured picture information is similar for both, so both should (and are) allowed.

So could the FX9 get Netflix approval for 2x Anamorphic?

The FX9’s sensor has is 3168 pixel tall when shooting FF 16:9  as it’s pixel pitch is finer than the Arri LF sensor.  When working with a 2x anamorphic super 35mm lens the image circle from the lens will cover around 4K x 3K of pixels, a total of 12 megapixels on the sensor when it’s operating in the 6K Full Frame scan mode. But then the FX9 will internally down scale this to that vignetted 4K recording that needs to be cropped.

6K down to 4K means that the 4K covered by the lens becomes roughly 2.7K. But then the 3.1K from the Arri when debayered will more than likely be even less than this, perhaps only 2.1K

But whether Netflix will accept the in camera down conversion is a very big question. The maths indicates that  the resolution of the final output of the FX9 would be greater than that of the LF, even taking the necessary crop into account. But this would need to be tested in practice. If the math is right, I see no reason why the FX9 won’t be able to meet Netflix’s minimum requirements for 2x anamorphic production. If this is a workflow you wish to pursue I would recommend taking the 10 bit 4:2:2 HDMI out to a ProRes recorder and record using the best codec you can until the FX9 gains the ability to output raw. Meeting the Netflix standard is speculation on my part, perhaps it never will get accepted for anamorphic, but to answer the original question –

 – Can you shoot anamorphic with the FX9 – Absolutely, yes you can and the end result should be pretty good. But you’ll have to put up with a distorted image with the supplied viewfinder (for now at least).


Can You Shoot Anamorphic with the PXW-FX9? was first posted on October 3, 2019 at 10:57 am.
©2018 “XDCAM-USER.COM“. Use of this feed is for personal non-commercial use only. If you are not reading this article in your feed reader, then the site is guilty of copyright infringement. Please contact me at contact@xdcam-user.com

From Computational Creativity to Creative AI and Back Again

Abstract

I compare and contrast the AI research field of Computational Creativity and the Creative AI technological movement, both of which are contributing to progress in the arts. I raise the sceptre of a looming crisis wherein public opinion moves on from the spectacle of software being creative to viewing the lack of authenticity in creative AI systems as being a major drawback. I propose a roadmap from Creative AI systems to Computationally Creative systems which address this lack of authenticity via the software expressing aspects of its computational life experiences in the art, music, games and literature that it produces. I posit that only by harnessing Creative AI technologies and Computational Creativity philosophies in the pursuit of truly creative software able to express the machine condition, will we gain maximum societal benefit in further understanding the human condition.

 

  1. Introduction

This year, we passed a milestone in my field, as the 10th annual International Conference on Computational Creativity (ICCC) was held in the USA. The conference brings together AI researchers who test the idea of software being independently creative, describing projects with goals ranging from enhancing human creativity to advancing our philosophical understanding of creativity and producing fully autonomous creative machines. The conference series was built on roughly ten years of preceding workshops [1], with interest in the idea of machine creativity going back to the birth of modern computing. For instance, in their 1958 paper [2], AI luminary Alan Newell and Nobel Prize winner Herbert Simon hypothesised that: “Within ten years, a digital computer will discover and prove an important mathematical theorem”. In [3], we proposed the following working definition of Computational Creativity research as:

“the philosophy, science and engineering of computational systems which, by taking on particular responsibilities, exhibit behaviours that unbiased observers would deem to be creative.”

In the last few years, we have seen unprecedented interest across society in generative AI systems able to create culturally interesting artefacts such as pictures, musical compositions, texts and games. Indeed, it’s difficult to read a newspaper or magazine these days without stumbling across a story about a new project to generate poems, or a symphony orchestra playing AI-generated music or an art exhibition in which AI systems are purported to be artists.

This wave of interest has been fuelled by a step change in the quality of computer-generated cultural artefacts, brought on largely by advances in machine learning technologies, and in particular the deep learning of artificial neural networks. Such techniques are able to generate new material by learning from data about the structure of existing material – such as a database of images, a corpus of texts or a collection of songs – and determining a way to create more of the same. An umbrella term for this groundswell of interest and activity in generative art/music/literature/games is “Creative AI”, and people from arts and sciences, within and outwith academia are actively engaged in producing art using AI techniques. We surveyed different communities engaged in generative arts – including Creative AI practitioners – in a recent ICCC paper [4].

While we might have expected the Creative AI community to have grown from the field of Computational Creativity, this is not the case. Indeed, somewhat of a schism has developed where the two communities have different aims and ambitions. Both communities have a main interest in the development of generative technologies for societal good. The Creative AI movement has an emphasis on quality of output and developing apps to commercial level for mass consumption. There is also a tendency to disavow the idea that software itself could/should be independently creative, in favour of a strong commitment to producing software purely for people to use to enhance their own creativity. In contrast, Computational Creativity researchers tend to be interested in the bigger picture of Artificial Intelligence, philosophical discourse around notions of human and machine creativity, novel ways to automate creative processes, and the idea that software, itself, could one day be deemed to be creative.

To highlight the schism: I personally find it difficult to think of any computational system as being “a Creative AI” if it cannot communicate details about a single decision it has taken, which is generally the case for approaches popular in Creative AI circles, such as Generative Adversarial Networks (GANs) [5]. I prefer therefore to describe Creative AI projects as “AI for creative people”, because the most literal reading of the phrase “Creative AI” is currently inaccurate for the majority of the projects under that banner. I often go further to point out that many Creative AI applications should be categorised as graphics (or audio, etc) projects which happen to employ techniques such as GANs that were originally developed by AI researchers.

As another example, I’ve argued in talks and papers many times that the end result of having more computer creativity in society is likely to be an increased understanding and celebration of human creativity, in much the same way that hand-made craft artefacts, like furniture or food, are usually preferred over machine-produced ones. I point out that I’ve met dozens of artists, musicians, poets and game designers, none of whom have expressed any concern about creative software, because they understand the value of humanity in creative practice. On the other hand, I’ve also spoken to Creative AI practitioners who remain convinced that truly creative software will lead to job losses, demoralisation and devaluation in the creative industries.

 

  1. Product versus Process

The Creative AI movement has helped to swing the global effort in engineering creative software systems firmly towards human-centric projects where AI techniques are used purely as tools for human use, with ease of use and quality of output disproportionately more important than any other considerations. I’ve been trying recently to put together arguments and thought experiments to help explain why I believe this is a retrograde step, and I’ve been trying to articulate ways in which the wealth of knowledge accrued through decades of Computational Creativity projects could be of use to Creative AI practitioners. Almost every project ever presented within Computational Creativity circles started with building a generative system with similar aims to Creative AI projects. Hence I feel we are well placed to consider the role that AI systems could have in creative practice, and to encourage Creative AI researchers and practitioners to consider some of the ideas we’ve developed over the years.

Imagine a generative music system created by a large technology company, which is able to generate 10,000 fully orchestrated symphonies in just 1 hour. Let’s say that each symphony would be lauded by experts as a beautiful work of genius had it been produced by a human composer like Beethoven; and each one sounds uniquely different to the others. If we accept the reality of an AI system (AlphaGo Zero) able to train itself from scratch to play Go, Chess and Shogi at superhuman levels [6], then we should entertain the idea that superhuman symphony writing is possible in our lifetimes. If we only concentrate on the quality of output and ease of which software can generate outputs as complex as a symphony, then the above scenario is presumably a suitable end point for generative music and would be a cause for celebration – it would certainly tick the box of huge technical achievement, as the AlphaGo project did. However, one has to wonder what the benefits of having these symphonies (and the ability to generate them so easily) are for society.

I would predict that the classical music world would find very few practical applications for a database of 10,000 high-quality symphonies, and it would likewise find little value in generating more such material. I would also predict that there would be little, if any, devaluation of symphonic music as a whole, and no devaluation of the work of gifted composers able to hand-produce symphonies. Superhuman chess playing by computers has been around since the time of Deep Blue, and has likely increased rather than decreased the popularity of the game. The chess world has responded to computer chess by being clearer about the human-centric struggle at the heart of every game of chess, and “[a]mong the chess elite, the idea of challenging a computer has fallen into the realm of farce and retort” [7]. It is clear that computer chess has made the game of chess more human. Part of the attraction of the music from composers such as Mozart and Beethoven is that these were mere mortals with superhuman creative abilities in composition. Society celebrates such creative people, often by lauding the works they produce, but also by applauding their motivations, exploring their backgrounds, expressing awe about their process, and by taking inspiration for a fresh wave of creative activity. Creativity in society serves various purposes, only one of which is to bring into being artefacts of value.

While board games have hugely driven forward AI research, chess isn’t some mathematical Drosophilia for AI problem solving (as some researchers would have you believe). It is actually a game and pastime played by two people, which can be elevated to highly competitive levels. Likewise, a symphony isn’t just a collection of notes to guide musicians to produce sound waves, but is created by human endeavour for human entertainment, often condensing into abstract form aspects of human life experience and expression. I would predict that – in an age of superhuman symphony generation – a huge premium would be placed on compositions borne of human blood, sweat and tears, with the generation of music via statistical manipulation of data by computer remaining a second class process.

 

  1. Computational Authenticity

To hit home with the points above, I usually turn to poetry, due to the highly human-centric nature of the medium: poems are condensed humanity, written by people, for people, usually about people. The following poem provides a useful focal point to illustrate the humanity gap [8] in Computational Creativity.

———————————————————————————————

Childbirth

by Maureen Q. Smith

The joy, the pain, the begin again. My boy.

Born of me, for me, through my tears, through my fears.

———————————————————————————————-

This short poem naturally invites interpretation, and we might think of the joy, pain, tears as fears as referring literally to the birth of a child, perhaps from the first-person perspective of the author, as possibly indicated by “My boy … Born of me”. We might also interpret the “begin again” as referring to the start of a baby’s life, but equally it might reflect a fresh start for the family.

Importantly, the poem was not actually written by Maureen Q. Smith. The author was in fact a man called Maurice Q. Smith. In this light, we might want to re-think our interpretation. The poem takes on a different flavour now, but we can still imagine the male author witnessing a childbirth, possibly with his own tears and fears, reflecting the joy and pain of a woman giving birth. However, I should reveal that Maurice Q. Smith was actually a convicted paedophile when he wrote this poem, and it was widely assumed to be about the act of grooming innocent children, which he referred to as “childbirth”. The poem now affords a rather sinister reading, with “tears” and “fears” perhaps reflecting the author’s concerns for his own freedom; and the phrases “Joy and pain” and “Born of me, for me” now taking on very dark tones.

Fortunately, as you may have guessed, the poem wasn’t written by a paedophile, but was instead generated by a computer program using a cut-up technique. Thankfully, we can now go back and project a different interpretation onto the poem. Looking at “Joy and pain”, perhaps the software was thinking about… Well, the part about “Born of me, for me” must have been written to convey… Hmmmm. We see fairly quickly that it is no longer possible to project feelings, background and experiences onto the author, and the poem has lost some of its value. If the words have been put together algorithmically with nothing resembling the human thought processes we might have expected, we may also think of the poem as having lost its authenticity and a lot, if not all, of its meaning. We could, of course, pretend that it was written by a person. In fact, it’s possible to imagine an entire anthology of computer generated poems that we are instructed to read as if written by various people. But then, why wouldn’t we prefer to read an anthology of poems written by actual people?

For full and final disclosure: I actually wrote the poem and found it remarkably easy to pen a piece for which a straightforward interpretation changes greatly as the nature of the author changes. I’ve been using this provocative poem to try to change the minds of researchers in Computational Creativity research for a few years, in particular to try and shift the focus away from an obsession with the quality of output judged as if it were produced by a person. I’ve argued that the nature of the generative processes [9], how software frames its creations [10], and where motivations for computational creativity come from [11] are more important for us to investigate than how to increase the quality or diversity of output. This led to a study of the notion of computational authenticity [12], which pays into the discussion below.

As with pretty much all things generative, the advent of deep learning has led to a step change in the quality of the output of poetry generators, which have a long history dating (at least) as far back as an anthology entitled: “The Policeman’s Beard is Half Constructed” [13]. On the whole, the scientists pushing forward these advances have barely thought of addressing the deficiencies with these poems, namely that they were made by an inauthentic process. It is not impossible to imagine a poem-shaped computer generated text that would have been classed as a masterpiece had it been written by a person, but is not accepted by anyone as even being a poem, because public opinion has swung against inauthentic generative processes. I have for many years advocated using the name “c-poem” for the poem-shaped texts produced by computers. Just as people know that they won’t be unwrapping a beautifully bound e-book for their birthday, they should know that their ability to project human beliefs, emotions and experiences onto the author of a c-poem will be very limited.

 

  1. Responses to the Rise of Creative AI

Returning to the observation that the quality of the artistic output of AI systems has much increased in recent years, we can consider some appropriate responses to this situation.

One response is to follow the lead from the Creative AI community, and disavow the idea that software should be developed to be fully creative, concentrating instead on using AI techniques to aid human creativity. This certainly simplifies the situation, with AI systems becoming just the latest tools for creative people. It is also a public-friendly response, as journalists, broadcasters and documentary makers (along with the occasional politician, member of the clergy, philosopher or royal) often publish missives about how AI software is going to take everyone’s job, strangle our cats and devalue our life. On the whole, I believe it would be very sad if this response dominates the discourse and drives the field, as it would certainly curtail the dream of Artificial General Intelligence, which brought many of us into AI, and it will limit the ways in which people interact with software, which has the potential to be much more than a mere muse or tool. Software systems we have developed in Computational Creativity projects can be seen as creative collaborators; motivating yet critical partners; and sometimes independent creative entities. We should not throw away the idea that software can itself be creative, as the world always needs more creativity, and truly creative AI systems could radically drive humanity forward.

A second response is to accept the point above that the processes and personality behind creative practice are indeed important in the cultural appreciation of output from generative AI systems. In this context, given that software won’t be particularly human-like anytime soon, we could say that it’s impossible to take an AI system seriously as an authentic creative voice. An extreme version of this argument is that machines will never be valuable in the arts because they are not human. I argue below that this is shortsighted and missing an opportunity to understand technology in-situ. A closely related opinion is that people should or could dislike computer generated material precisely because it has been made by computer. This point of view has certainly been simmering under the surface of many conversations I’ve had, leading people to talk of computers lacking a soul or a spark, and often employing other such obfuscating rhetoric. Perhaps surprisingly, I’ve argued on a number of occasions that such a view is not extreme, and is indeed perfectly natural: such a view would, in my opinion, be a suitable personal response to the childbirth poem above, if indeed it had been computer generated.

Well intentioned people would never dream of saying that they dislike something because it was produced by a particular minority (or majority) group of people. Hence it feels to those people that they are being prejudicial to say that a painting, poem or composition is inferior purely because it was computer generated. Moreover, the view that works such as paintings and novels should be evaluated in their own terms, i.e., independently from information about their author and the creative process, has been reinforced philosophically with movements such as the Death of the Author [14], and numerous artistic manifestos.

Software systems do not form a minority human group whose creative freedom has to be protected. Throughout the history of humanity, art has been celebrated as a particularly human endeavour, and the art world is utterly people-centric. Software is not human, but due to decades of anthropomorphic thinking on AI, it seems more acceptable to think of computers somehow as under-evolved or under-developed humans, perhaps like monkeys or toddlers, rather than non-humans with intelligence, albeit low. Disliking a work of art purely because of its computational origins is more akin to expressing a preference of one type of process over another, than it is to expressing preferences of one ethnicity, gender or religion over another. “I don’t like this painting because it is a pointillist piece” is not the same as: “I don’t like this painting because it was painted by a Brit”.

So, we could say that, while the output of the current/future wave of generative AI systems is remarkable, and could – under Turing-style conditions of anonymity – be taken for human works, there is a natural limiting factor in the non-humanity of computational systems which gives us a backstop against the devaluation of human artistic endeavour. This is a reasonable response and may lead to increased celebration of human creativity, which would be no bad thing. However, I believe that this response will also (eventually) be limiting and lead to missed opportunities, as I hope to explain below.

A third response, which I greatly favour, is to start from the truism that software is not human. In many research and industry circles, it often seems that creating human-like intelligence through nueroscience-inspired approaches such as deep learning, is the only goal and the only approach. Not every AI researcher wants to build a software version of the brain, but this fact is often lost, and helps to obfuscate the fact that software has different experiences to people. The Painting Fool is software that I’ve developed over nearly 20 years [15], and has met minor and major celebrities and painted their portraits in half a dozen different countries, often in front of large audiences in interesting venues ranging from science museums and art galleries to a pub in East London. I have, of course, anthropomorphised this experience and The Painting Fool didn’t experience it as I have portrayed. But it did have experiences, and those experiences were authentic in the sense that the software was present, did interact with people and created things independently of me which entertained and provoked people in equal measure.

We could therefore respond to the uptick in quality of output from Creative AI systems by agreeing to concentrate more on investigating plausible internal reasons for software to be creative, and developing ways in which it can impart its understanding of the world, through expressing aspects of its life experiences. Instead of challenging human creativity in terms of the quality of output, but failing due to lack of authenticity, Computational Creativity systems could be developed to explore aspects of creative independence such as intrinsic motivation, empowerment [10] and intentionality [8]. A side effect of this is that – if we get software to record and use its own experiences rather than pretending that it is a person having human experiences – we will gain a better understanding of computer processing, the impact of particular software systems and what it means for a machine to have a cultural existence in our human world. It may be that this communicative side effect actually becomes more important than having software be creative for the purpose of making things.

If software can express its experience of the world through artistic expression, surely this would add to our understanding of human culture in a digital age of tremendous, constant, technological change. While the non-human life experiences of software systems can seem other worldly, automation is very much a part of the human world, and our increasing interaction on a minute-by-minute basis with software means we should be constantly open to new ideas for understanding what it does. It’s not so strange to imagine building an automated painting system to add on to another piece of software so that it can express aspects of its experience. In fact, this would be a natural generalisation of projects such as DeepDream [16], where visualisations of deep-learned neural models were originally generated to enable people to better understand how the model processed image data. It turned out that the visualisations had artistic value as computational hallucinations, and were presented in artistic contexts, with this usage eventually dominating, fuelling a huge push in generative neural network research and development.

 

  1. A RoadMap from Creative AI to Computational Creativity

In a talk at a London Creative AI meetup event a while ago, I offered some advice for people in the Creative AI community who might be interested in pursuing the dream of making genuinely creative AI systems. At the time, there were already indications that Creative AI practitioners were beginning to see the limitations of mass generation of high-quality artefacts and were interested in handing over more creative responsibility to software. Some people were already testing the water using deep learning techniques in ways other than pastiche generation, for instance looking at style invention rather than just style transfer [17]. The advice I gave can be seen as a very rough roadmap, which reflects to some extent my own career arc in building creative AI systems, and provides one of many paths by which people can take their generative system into fascinating new territories.

While keeping much of the original, I will re-draw the roadmap below, from a fresh perspective of improving authenticity through expanding the recording and creative usage of life experiences that creative software might have. It is presented as a series of seven levels for Creative AI Systems to transition to via increased software engineering and cultural usage, with each level representing a different type of system that the software graduates to. Focused on generative visual art rather than poetry/music/games/etc., but intended to generalise over many domains, the roadmap offers direct advice to people who already have a generative system.

  • Generative Systems. So, you’ve designed a generative system and are having fun making pictures with it. You play around with input data and parameter settings, and realise that the output is not only high quality, but really varied. You write a little graphical user interface, which enables you to play around with the inputs/parameters, and this increases the fun and the variety. It becomes clear that the space of inputs/parameters is very You begin to suspect that the space of novel outputs is also vast. You’re at level one: you have an interesting generative system which is able to make stuff. 

 

  • Appreciative Systems. Generating images becomes addictive, and you gorge on the output. In your gluttony, you get a strong fear of missing out – what if I miss the parameters for a really interesting picture? You decide to systematically sample the space of outputs, but there are millions of images that can be produced. So, you encode your aesthetic preferences into a fitness function and get the software to rank/display its best results, according to the fitness function, perhaps tempered by a novelty measure to keep things fresh. You’re at level two: you have an appreciative system which is able to discern quality in output.

 

  • Artistic Systems. At some stage, some humility sinks in, and you begin to think that maybe… just maybe… your particular aesthetic preferences aren’t the only ones which could be used to mine images. You give the software the ability to invent its own aesthetic fitness functions and use them to filter and rank the images that it generates. You’re at level three, with an artistic system which has some potential to affect the world artistically.

 

  • Persuasive Systems. Some of the output is great – beautiful new images that you perhaps wouldn’t have found/made yourself. But some of the pictures are unpalatable and you can’t imagine why the software likes them. However, sometimes, an awful image grows in appeal to you, and you realise that your own aesthetic sensibilities are being changed by the software. This is weird, but fun. You want to give the software the ability to influence you more easily, so you add a module which produces a little essay as a commentary on the aesthetic generation, the artefact generation and the style that the software has invented. You’re at level four, with a persuasive system that can change your mind through explanations as well as high quality, surprising output.

 

  • Inventive Systems. You begin to realise that you enjoy the output partially because of what it looks like and partially because of the backstory to the generation of the output and the aesthetics being considered. You want to increase both aspects, by enabling the software to alter its own code, perhaps at process level, and by taking inspiration from outside sources like newspapers, twitter, art books, other artists, etc., so you have less control. And you add natural language generation to turn the commentary about the process/product into a little drama. You’re at level five, where what your inventive system does is as important, interesting and unpredictable as its output.

 

  • Authentic Systems. You’re loving the commentaries/essays/stories about how and why your software has made a particular picture/aesthetic/style/series or invented a new technique, and the software pretty much has an artistic persona. However, sometimes the persona doesn’t ring true and actually verges on being insulting, given how little the software knows about the world. You realise that you’re reading/viewing the output as if it were created by a person, which is a falsehood which has gotten very old and somewhat disturbing. You decide to give the software plausible and believable reasons to be creative, by implementing models of intrinsic motivation, reflection, self-improvement, self-determination, empowerment and maybe even consciousness. In particular, much of this depends on implementing techniques to record the life experiences that your software has, via: sensors detecting aspects of the environment the software operates in; improved in-situ and online HCI, wherein the software’s interactions with people are recorded and the software is able to probe people with questions; and methods which take life experiences and outside knowledge and operationalise them into opinions that can be reflected in generative processing and output. You then give the software the ability to use its recorded life experiences to influence its creative direction, in much the same way that twitter and newspaper sources were previously. You’re at level six, with an authentic system that is seen more as an autonomous AI individual than a pale reflection of a person.

 

  • Philosophical Systems. Ultimately, you find it thrilling to be in the presence of such an interesting creator as your software – it’s completely independent of you, and it teaches you new things, regularly inspiring you and others. You realise that for the software to be taken seriously as an artist, it needs to join the debate about what creativity means (as creativity is an essentially contested concept [18]) in practice and as a societal driving force. You implement methods for philosophical reasoning based on the software’s own creative endeavours, and you enable it to critique the thoughts of others. You add dialogue systems to propose, prove and disprove hypotheses about the nature of creativity, enabling your system to generally provoke discussion around the topic. You’re at level seven, where it’s difficult to argue that your philosophical system isn’t genuinely creative.

 

It is fair to say that no AI system gets close yet to levels 6 and 7 yet, but projects presented in Creative AI and Computational Creativity circles have tested the water up to and including level 5. If I were giving a talk about this roadmap, there would be much handwaving towards the end, as the road gets very blurry, with few signposts. This, of course, is the frontier of Computational Creativity research and reflects directions I will personally be taking software like The Painting Fool in. I’m particularly interested in exploring the notion of the machine condition and seeing how authentic we can make the processing and products from AI systems. That notwithstanding, I hope the roadmap offers some insight and inspiration to people from all backgrounds who are working with cool generative systems and want to take the project further.

 

  1. In Conclusion

More than a decade ago, I was dismayed to read in a graphics textbook the following statement:

“Simulating artistic techniques means also simulating human thinking and reasoning, especially creative thinking. This is impossible to do using algorithms or information processing systems. [19, p. 113]”

The topic of the textbook is Non-photorealistic Computer Graphics, part of which involves getting software to simulate paint/pencil/pastel strokes on-screen. Stating that computational creative thinking is impossible was short-sighted and presumably written to placate creative industry practitioners, who use software like the Adobe Creative Suite which employ such non-photorealistic graphics techniques. In the 17 years since the above statement was published, the argument seems to have moved on from whether software can be independently creative to whether it should be allowed to. It is my sincere hope that the argument will shift soon to the question of how best truly creative AI systems can enhance and inform the human world, and how we can use autonomous software creativity to help us understand how technology works.

Creative AI practitioners have emerged as much via scientists in the machine learning community embracing art practice as via tech-savvy artists picking up and applying tools such as Tensor Flow [20]. Speaking personally, and having witnessed numerous transitions, scientists tend to hold on too long to the idea that product is more important than process or personality in creative practice [21]. This is presumably due to scientific evaluation being objective, with scientific findings expected to be evaluated entirely independently of their origins.

It would be tempting to follow the lead of companies like DeepMind who often justify working on applications to the automated playing of board games and video games [22] by stating that this research pushes forward AI technologies in general, which ultimately leads to improvements in applications to other, more worthwhile, domains like protein structure prediction [23] and healthcare. Getting software to produce better poems, paintings, games, etc., will likely lead to improvements in AI techniques overall, so concentrating on improving quality of output is in some senses a good thing. However, this would serve to deflect from what I believe is a looming crisis in Creative AI, which is when the novelty of the computer generation gimmick wears off, and people begin to realise that authenticity of process, voice and life experience are more important than the so-called “quality” of computer generated artefacts.

The activities of playing games and predicting protein structures have the luxury of objective measures for success and thus progress (beating other players and nanoscale accuracy, respectively). This is not true in the arts, where there are only subjective – and highly debated – notions of the “best” painting, poem, game or musical composition. The humanity wrapped up in artefacts produced by creative people is absolutely critical in the evaluation of those artefacts, which is not true in scientific or (to a lesser extent) competitive scenarios.

It is similarly tempting to appeal to the creative outcomes of the AlphaGo match against Lee Sedol, which have been described beautifully by Cade Metz in [24]:

“In Game Two, the Google machine made a move that no human ever would. And it was beautiful. As the world looked on, the move so perfectly demonstrated the enormously powerful and rather mysterious talents of modern artificial intelligence.”

“But in Game Four, the human made a move that no machine would ever expect. And it was beautiful too. Indeed, it was just as beautiful as the move from the Google machine – no less and no more. It showed that although machines are now capable of moments of genius, humans have hardly lost the ability to generate their own transcendent moments. And it seems that in the years to come, as we humans work with these machines, our genius will only grow in tandem with our creations.”

In the thought experiment above, in the corpus of 10,000 new symphonies generated by computer, there would surely be many moments of inventive genius: a phrase, passage or flourish of orchestration found in the notes of the music produced. Humankind would learn from the software, and would in turn develop better generative approaches to music production. But would we necessarily learn anything about the human condition, as we generally hope to in the arts?

I posit that only if software is developed to record its life experiences and use them in the pursuit of creative practice will we learn anything about the human condition, through increased understanding of the machine condition. Developing better AI painters means engineering software with more interesting life experiences, not software with better technical abilities. While there might be advantages, there is no imperative for these life experiences to be particularly human-like, and society might be better served if we try and understand computational lives through art generation. We hear all the time that the workings of black box AI systems deep-learned over huge datasets are not understood even by the researchers in the project. While this difficulty is usually overstated, we are facing a situation of increased scenarios where AI-enhanced software makes decisions of real import for us, coupled with decreased understanding of how individual AI systems make those decisions.

Combining the best practices and understanding gained from both Computational Creativity as a research field and Creative AI as an artistic and technological movement, may be the best approach to bringing about a future enhanced by creative software expressing its life experiences artistically for our benefit. The diversity, enthusiasm and innovative thinking coming daily from the Creative AI community, guided by the philosophy of the Computational Creativity movement is a potent combination, and I’m optimistic that in my lifetime, we will reap the benefits of cross-discipline, cross-community collaborations. Creative AI practitioners may rail against interventions from people like myself: stuffy academic disciples of the Computational Creativity discipline. But it is worth mentioning that we were once the angry young men and women of a largely ostracised and ignored arm of AI, shouting into the void at an establishment who thought that notions of creativity in AI systems were too “wooly” to be taken seriously.

Who knows what history will record about the rise of creative machines in society. My sincere hope is that it will chart how Computational Creativity thinking evolved without the benefit of sophisticated technical implementations; this was massively influenced with a surge in the technical abilities of Creative AI Systems during the period of Deep Learning dominance; but then naturally turned back to the philosophical thinking of Computational Creativity in order to properly reap the benefits of truly creative technologies in society.

 

References

[1] Cardoso, A., Veale, T. and Wiggins, G. A. (2009). Converging on the divergent: The history (and future) of the international joint workshops in computational creativity. AI Magazine, 30(3), 15–22.

[2] Simon, H., and Newell, A. (1958). Heuristic problem solving: The next advance in operations research. Operations Research, 6(1), 1-10.

[3] Colton, S. and Wiggins, G. A. (2012). Computational Creativity: A Final Frontier? Proceedings of the European Conference on Artificial Intelligence, 2012.

[4] Cook, M. and Colton, S. (2018). Neighbouring Communities: Interaction, Lessons and Opportunities. Proceedings of the Ninth International Conference on Computational Creativity.

[5] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. (2014). Generative Adversarial Networks. Proceedings of the International Conference on Neural Information Processing Systems.

[6] Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., Chen, Y., Lillicrap, T., Hui, F., Sifre, L., van den Driessche, G., Graepel, T. and Hassabis, D. (2017). Mastering the game of Go without human knowledge. Nature 550, 354-359.

[7] Max, D. T. (2011) The Prince’s Gambit: A chess star emerges for the post-computer age. New Yorker, March 14th 2011 edition.

[8] Colton, S., Cook, M., Hepworth, R. and Pease, A. (2014). On Acid Drops and Teardrops: Observer Issues in Computational Creativity. Proceedings of the AISB’50 Symposium on AI and Philosophy.

[9] Colton, S. (2008). Creativity versus the Perception of Creativity in Computational Systems.

Proceedings of the AAAI Spring Symposium on Creative Systems.

[10] Charnley, J., Pease, A. and Colton, S. (2012). On the Notion of Framing in Computational Creativity. Proceedings of the Third International Conference on Computational Creativity.

[11] Guckelsberger, C., Salge, C. and Colton, S. (2017). Addressing the “Why?” in Computational Creativity: A Non-Anthropocentric, Minimal Model of Intentional Creative Agency. Proceedings of the Eighth International Conference on Computational Creativity.

[12] Colton, S., Pease, A. and Saunders, R. (2018). Issues of Authenticity in Autonomously Creative Systems. Proceedings of the Ninth International Conference on Computational Creativity.

[13] Chamberlain, W. and Etter, T. (1984). The Policeman’s Beard is Half-Constructed: Computer Prose and Poetry. Warner Books.

[14] Barthes, R. (1967). The death of the author. Aspen 5-6.

[15] Colton, S. (2012) The Painting Fool: Stories from building an automated painter. In McCormack, J. and d’Inverno, M., eds., Computers and Creativity, 3–38. Springer.

[16] Mordvintsev, A., Olah, C. and Tyka, M. (2015). DeepDream – a code example for visualizing Neural Networks. Google AI Blog, July 1st 2015.

[17] Elgammal, A., Liu, B., Elhoseiny, M. and Mazzone, M. (2017). CAN: Creative Adversarial Networks, Generating “Art” by Learning About Styles and Deviating from Style Norms. Proceedings of the Eighth International Conference on Computational Creativity.

[18] Gallie, W. (1956). Art as an essentially contested concept. The Philosophical Quarterly 6(23),97-114.

[19] Strothotte, H. and Schlechtweg, S. (2002). Non-Photorealistic Computer Graphics: Modelling, Rendering and Animation. Morgan Kaufmann.

[20] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A.,

Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jozefowicz, R.,  Jia, Y., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Schuster, M., Monga, R., Moore, S., Murray, D., Olah, F., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y. and Zheng, X. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org.

[21] Jordanous, A. (2016). Four PPPPerspectives on computational creativity in theory and in practice. Connection Science special issue on Computational Creativity, 28(2), 194-216.

[22] Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A., Veness, J., Bellemare, M., Graves, A., Riedmiller, M., Fidjeland, A., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S. and Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature 518, 529-533.

[23] Evans, R., Jumper, J., Kirkpatrick, J., Sifre, L., Green, T., Qin, C., Zidek, A., Nelson, A., Bridgland, A., Penedones, H., Petersen, S., Simonyan, K., Crossan, S., Jones, D., Silver, D., Kavukcuoglu, K., Hassabis, D. and Senior, A. (2018). De novo structure prediction with deep-learning based scoring. Proceedings of the Thirteenth Critical Assessment of Techniques for Protein Structure Prediction (Abstracts).

[24] Metz, C. (2016). In Two Moves, AlphaGo and Lee Sedol Redefined the Future. Wired, 16th March 2016 edition.

The post From Computational Creativity to Creative AI and Back Again appeared first on Interalia Magazine.

Neural Zoo

 

Sofia Crespo: Micro Beauty

 

Sofia Crespo: Bug

 

Sofia Crespo: Consistente

 

Sofia Crespo: Soft Sea of Awareness

 

Sofia Crespo: Self Acceptance

 

Sofia Crespo: Reward System

 

Sofia Crespo: Revivir

 

Sofia Crespo: Realization

 

Sofia Crespo: Morphing

 

Sofia Crespo: Merging

 

Sofia Crespo: Internet

 

Sofia Crespo: Free Will

 

Sofia Crespo: Courage

 

……………………..

https://sofiacrespo.com/

All images copyright and courtesy of Sofia Crespo

The post Neural Zoo appeared first on Interalia Magazine.

Art and Generative Systems

Gene Kogan: Neural Synthesis, 2017

Richard Bright: Can we begin by you saying something about your background?

Gene Kogan: I studied applied mathematics in university and became interested in machine learning through its application to music technology, especially the idea of music recommendation systems. That got me thinking more about creative and artistic uses of machine learning, which led me indirectly to discover media arts and art technology more broadly. Since then, I’ve become interested in computer science more generally in how it can be applied to emerging tech art.

RB: Have there been any particular influences to your art practice?

GK: A lot of the things that influence my art practice come from outside the art world. I don’t have a proper art background and don’t participate very much in the residency or gallery scene, with fairly rare exceptions, and I think this keeps my work a bit less influenced by art trends. I am very curiosity-driven and spend most of my time looking at scientific literature more so than artistic. That said, there have been many artists that have influenced me over the years, and I am especially grateful to arts-technology communities like OpenFrameworks and Processing, out of which I’ve made many friends, and gotten many ideas and help on projects. Community is very important in arts technology, otherwise we’re all just sitting alone in front of computers.

RB: What is the underlying focus of your work?

GK: I guess the underlying theme is emerging technology for creative practice and generative art. More recently, I’ve become interested in systems that facilitate mass collaboration among people, and creating generative systems built on collective intelligence.

RB: Can you say something about ml4a, the collection of free educational resources devoted to machine learning for artists?

GK: I started ml4a as a resource for a class I was teaching at NYU called “machine learning for artists” and slowly the scope of it grew to encompass most of the educational materials I was putting out, including outside of the university. I generally neither work as a creative technologist nor as a professional artist (selling my work) and so my educational output turns out to be the most stable part of my professional work, and it’s been really fun for me to keep a consistent workshop practice over the last few years, and ml4a has been a big part of that. These days, I’m thinking a lot about how to evolve ml4a, as some of the problems that it seeks to solve are becoming less relevant, now that there are so many more resources besides ml4a directed at artists and creatives. I’m thinking about how to make it more goal-driven and community-oriented.

Neural synthesis [2017]. Some recent experiments with neural channel synthesis. The video was created for the creativity exhibition at NIPS conference in 2017.

RB: A lot of the processes behind creative thinking are still unknown. Can AI-powered creativity and neural networks play a role in helping the understanding about our own creative methodology and imagination?

GK: Yeah, I think it can help us discover a lot of things by creating interesting interactions between us and our tools. When you work with systems that have flexible automation, it forces you to confront what the essence of creativity really is. Is it in the performance, or the composition, or the ideation? It’s all very subjective though, so I tend not to take the question too seriously.

RB: Can you say something about your project Abraham?

GK: I’ve been thinking about Abraham for a couple of years, and I’m pretty excited about it. It’s gradually becoming my main focus. In many ways, it’s a continuation of ml4a but a bit more goal-driven, with a tangible project as an end goal, and an expanded scope to include emerging topics besides machine learning, including decentralization technology, cryptography, economics, game theory, and even some philosophy. I’m trying to make the case now for why it’s an interesting construction, this idea of an autonomous artificial artist, and it’s been a learning process for me, trying to understand and articulate why I find it so meaningful. Hopefully, we start working towards a prototype later this year, and I expect 2020 for it to be my main focus.

Gene Kogan: Experiments with style transfer (2015). Mona Lisa restyled by Egyptian hieroglyphs, the Crab Nebula, and Google Maps.

RB: Can AI be taught how to create without guidance and develop its own sense of creativity?

GK: By definition, AI (or AGI) is all about creating agents that have all the same intellectual capabilities or even greater ones than human beings, and so in principle, if we believe a human can do that, then so could an AI eventually. How we achieve that in the future, and when or if that might ever actually happen is another question. I’m pretty optimistic in general and don’t see why we couldn’t accomplish this in principle, but it’s hard to predict.

RB: When does a neural network become an author of an artwork? And how can we form an understanding of the art that it makes?

GK: This question comes up a lot but I think it’s actually not well-defined. Authorship is a concept that very much predates AI and has not yet caught up. We are seeing now how limiting it is to try to assign one person or entity as the sole author to something, when AI brings in so many influences, and so many people, and so many data points. It very much fragments the notion of authorship, and certainly downstream ideas about intellectual property, copyright, and so on. No one ever asked “when does a paintbrush become an author of an artwork” even though it’s a tool just the same. But the more of the creative process AI takes on, much more so than the paintbrush itself, the more obsolete the authorship idea becomes, since AI is not really a singular being like a human. We may have to invent new words to really make this clear.

Gene Kogan: pix2pix webcam (meat puppet), 2017

RB: Emotions are essential for creativity and is a subject being explored in a relatively new area of AI, Affective Computing, which seeks to place a machine in the world such that it recognizes, interprets, processes, and simulates human affects. In order to be truly creative, will AI need to develop emotions and consciousness?

GK: Like with authorship, I think some of these terms are not well defined. Have to pass on the question, it’s a bit too abstract!

RB: Pushing the boundaries of the medium is a natural part of the art making process because, in some ways, the artist is exploring the medium itself. What boundaries do you wish to push with the medium that you use?

GK: I’ve been really inspired by the world of decentralization and peer-to-peer networks, in how they are trying to increase our ability to coordinate en masse with many people towards shared goals. Abraham is very much influenced by this trend. I think there’s a lot of room to innovate here and a lot of low-hanging fruits. In the context of art, I’d like to see how it’s possible to make creative artifacts — artworks, music, even novels — through this kind of mode of mass collaboration.

RB: As well as mI4a and Abraham, what other projects are you currently working on or planning?

GK: I am helping some friends to organize a free retreat to make art in something of an intentional community in the desert. The website is brahman.ai . It is closely tied with some of my arts projects, but it’s much more anarchical, and some of my friends will be initiating interesting art projects and group activities there. It’s an experiment in sustainable learning and living, and I’m pretty excited to spend a few months focused on it next year. I’m also just generally researching AI and decentralization, and working occasionally on art projects and installations. For example, I just finished a big installation at a brand new museum in Germany called The Futurium.

……………………

http://genekogan.com/

All images copyright and courtesy of Gene Kogan.

The post Art and Generative Systems appeared first on Interalia Magazine.

Missing Mass

Carey Young: Missing Mass, 2010 (installation view) 5,461 dark matter particles present in perspex container, on pedestal with silkscreened text container: 18 x 18 x 18 in. (45.7 x 45.7 x 45.7 cm) pedestal: 38 x 18 x 18 in. (96.5 x 45.7 x 45.7 cm) © Carey Young. Courtesy Paula Cooper Gallery, New York

Missing Mass (2010) is a sculptural work created with the scientific guidance of Prof. Malcolm Fairbairn, an astrophysicist based at King’s College London. The piece ‘presents’ a specific number of dark matter particles alongside a legal disclaimer which proposes the particles as the only truly free entities in existence. The work centres on the idea of artistic freedom, suggesting that if dark matter particles are the only free entities in existence, by implication, art, the artist, and any other societal or cultural element held to be symbolic of freedom, are merely constrained, whether by gravity, bureaucracy, institutional ties, etc. The work also proposes links between sculptural works associated with Minimalism and Conceptual Art (such as the early work of Hans Haacke) and contemporary developments in astrophysics.

The work was developed through a research process which involved regular meetings with Dr. Fairbairn, plus an astrophysics reading list, which necessitated five months of study. From this process I derived the idea for the work, as well as others including Terminal Velocity.

Carey Young Missing Mass, 2010 (detail) 5,461 dark matter particles present in perspex container, on pedestal with silkscreened text container: 18 x 18 x 18 in. (45.7 x 45.7 x 45.7 cm) pedestal: 38 x 18 x 18 in. (96.5 x 45.7 x 45.7 cm) Photo: Thierry Bal. © Carey Young. Courtesy Paula Cooper Gallery, New York

The text on the plinth says:

Carey Young
2010
5,461 dark matter particles present in perspex container, 18 x 18 x 18 inches.*

* Disclaimer

  1. i) Dark matter particles are governed by their own laws and may circulate freely.ii)  The figure of 5461 dark matter particles represents an average according to current scientific thinking. Actual amounts may vary from time to time.iii)  Dark matter is transparent and undetectable to the human eye.iv)  Since dark matter may at any time pass through any surrounding man-made or natural structures, including the walls of this container, your body, and the whole material structure of the planet, any collector of this work should not expect to own the same 5,461 dark matter particles at any one time.

Carey Young Missing Mass, 2010 (detail) 5,461 dark matter particles present in perspex container, on pedestal with silkscreened text container: 18 x 18 x 18 in. (45.7 x 45.7 x 45.7 cm) pedestal: 38 x 18 x 18 in. (96.5 x 45.7 x 45.7 cm) Photo: Steven Probert. © Carey Young. Courtesy Paula Cooper Gallery, New York

………………..

Artist Statement

Since 2003, visual artist Carey Young has developed a number of artworks that are also functional legal instruments, and which have conceptualised and explored law as an artistic medium. Young collaborates with legal advisors to make artworks in installation, video, performance, print, sculpture and photography, which have been exhibited internationally. These works have embodied such diverse forms as contracts, disclaimers, offers, licenses, cautionary statements and a will, and addressed disparate legal fields including human rights, inheritance law, intellectual property and law relating to outer space. Experimenting with ideas of time, space and physicality, Young’s body of artistic work explores law as a separate kind of ‘reality’, one with its own inherent subjectivities and points of breakdown.

……………………

www.careyyoung.com

 

The post Missing Mass appeared first on Interalia Magazine.

Voices

Aura Satz: The Trembling Line. Film and multi-channel sound installation, 2015

Richard Bright: Can we begin by you saying something about your background?

Aura Satz: I studied cultural studies and art history in Bologna (Italy) before coming to London to do a PhD by theory/practice at the Slade School of Fine Art. Initially I worked with sculpture and performance but over the last 20 years or so I have become more invested in film and sound. My works operate in constellations, I have a central theme which might manifest in multiple formats, as films, performances, sound works, and so on.

RB: Have there been any particular influences to your art practice?

AS: I was very much influenced by Lis Rhodes who taught me at the Slade, and whom I have since collaborated with. Intergenerational conversations are extremely important to me. I have been teaching for around 20 years now, and I get a lot of inspiration from my students. Teaching also keeps me attuned to practices outside of my own. In my undergraduate studies I was particularly fascinated by iconoclasm and theories of the image based on contact relics. I suppose this has carried through in my later works which attempt to look closely at technologies, prying the apparatus apart, as well as my interest in technologies of sound writing, such as the phonograph – where the groove is a trace and relic of the voice, so to speak. I have always been inspired by female voices, and there are a number of women composers who are key sources of inspiration. I often think of some of the more dialogic works I have made with people such as Lis Rhodes, Laurie Spiegel, Pauline Oliveros, as an opportunity to go deeper into the conversation, not just through the encounter, the film or the recording of a verbal exchange, but even later in the editing process, where I spend a lot of time listening and composing to the cadence of speech or a pause for breath.

Aura Satz: Her Marks a Measure

RB: What is the underlying focus of your work?

AS: I keep returning to the notion of a distributed, expanded and shared notion of voice. Works are made in conversation and use dialogue as both method and subject matter. In my works which draw on historical research I see myself in dialogic exchange with past voices, speaking backwards and forwards, being spoken through. When I have focussed on technologies of sound writing, recording and playback, it is precisely because I am interested in ways in which voices carry through, have been under-heard, and can be ‘listened into speech’. Many previous works focussed on minor histories, using archival research as a starting point, but in recent years I have shifted from the idea of notation of the past towards a logic that is more aligned with a visual or verbal score, an open invitation to think towards possible future manifestations. A score implies a non-hierarchical generosity, suggesting multiple future iterations and no singular privileged way of performing or enacting. Some scores simply suggest a shift in focus, such as Oliveros’ suggestion to listen with the soles of your feet. Many of my works could be read as an invitation to recalibrate attention, ways in which we give it, what is deemed worthy of it, how we might enact a different modality of attention, what we conceive of as foreground and what is background.

Aura Satz: Ventriloqua

RB: You began working with sound, with the piece Ventriloqua, in 2003 when you were pregnant. A number of your later works are to do with acoustic devices and vibration. Can you give some examples of these works?

AS: In Ventriloqua my pregnant belly became an instrument, a medium or antenna of sorts for a thereminist to play the electromagnetic waves. I wore a red outfit that covered all of my body, including my face, and the only visible part was the belly, which looked like an oracular eye or a breast of sorts. Through the trope of ventri-loquism (belly-speaking) I was able to explore the possibility of becoming a conduit for other voices. For me that performance was a powerful manifestation of speaking and being spoken through. In other works such as Automamusic (2008), Sound Seam (2010) Onomatopoeic Alphabet (2010), Vocal Flame (2011) and In and Out of Synch (2012) I focussed on devices such as orchestrions, mechanical music, phonographs, Chladni Plate, Ruben’s tube and optical sound on film as technologies of sound visualisation, some of which manifest sound patterns without quite constituting a notation system or code, and others which encrypt sound in order for it to be read back by a machine rather than a human. All of these enabled me to explore voices that align, interfere, interweave, synchronise, overlap, overwrite, hover between signal and noise, between decipherable meaning and the unfamiliar and as yet unencoded.

Aura Satz: Vocal Flame, 2012

 

Aura Satz: Sound Seam, 2010 (installation view)

In Sound Seam for example we worked with the surface noises of wax cylinders and vinyl glitch, as well as generating many layers of sounds by recording voices over each other.  At the same time there is something about seeing as informed by hearing, and vice versa, a listening that is in tension with the visible, that I find incredibly generative. This became central to In and Out of Synch, the 16mm film co-scripted and co-voiced with Lis Rhodes, where the optical sound on film patterns conveying our voices are ruptured by stroboscopic effects, due to a deliberate subtle misalignment of the monitoring eyepiece. You end up with a kind of Rorschach effect, certain sounds are punctuated or counteracted by the visual, and their respective rhythms generate a friction that is useful in unsettling standardized readings, making us hear or see differently.

Aura Satz: In and Out of Synch

RB: When did you begin to prioritise film-making and why?

AS: Initially I used film to document performances. When I made Automamusic in 2008 I realised that the only way to get inside these multiple mechanical music devices (which were housed in a museum in a small town in Switzerland), the best method of access to open them up and reconfigure them, was through the camera and the juxtaposition of sound patterns with visual rhythms. In other projects I found that there’s a kind of close-up looking and listening that can only be achieved through film. In my films I am keen to foreground sound, often it becomes the driving force, literally the engine driving the visuals or setting the rhythmic pace of the film. This is true of all of the sound visualisation films mentioned above, as well as a more recent project Preemptive Listening (2018), where the voice triggers an emergency rotating light. A film might feature moments of darkness or silence to allow for the senses to cross-pollinate, the eyes to take on the role of the ears or the other way around. I like the idea of an anagrammatic remapping of the senses, a disruption of hierarchies, a destabilizing of relations, of what is perceived, how, where, by who.

Aura Satz: Preemptive Listening (part 1 The Fork in the Road), 2018, installation view (photo Adam Reich)

RB: As well as exploring different techniques for visualizing sound, a number of your works focus on gender and women’s important contributions to technology. I’m thinking particularly of Oramics: Atlantis Anew (2011), Doorway for Natalie Kalmus (2013) and She Recalibrates (2018).  Can you say something about these works?

AS: Part of my commitment to the notion of a distributed voice is an unsettling of which voices are allowed, amplified within the range of the audible, who gets heard, who is written into the canon of history, and how can we destabilise these readings to allow for new voices to emerge. The film about Daphne Oram was central to my thinking on sound writing as a form of instantiating a new language or notation system, a new soundscape and in turn a new kind of listening.

 

Aura Satz: She Recalibrates (Pauline Oliveros), 2018 (photo Thierry Bal)

She Recalibrates follows on from this by focussing on women composers working with electronic music such as Laurie Spiegel, Pauline Oliveros, Eliane Radigue, Maryanne Amacher and others, who are portrayed with their hands on a dial, engaged in an experimental type of listening, modulating electricity, recalibrating what is considered noise or signal, what is worthy of being heard, and what can be understood as music. Their hands and ears are literally partaking in the circuit, tuning and recalibrating the signal. I made series of pencil drawings of hands on dials, framed inside a Fresnel lens which generates a diffractive pattern from the centre. The drawings only appear at a certain angle, due to the silver effect of graphite pencil on black paper, but also because the lens incorporates the interference of light reflections. It’s like looking at a lenticular print, or, more accurately, a CD or vinyl record with a diffractive centre, the image is continually changing according to the position of the viewer and the angle of light.  This is emblematic of what I try to do in all my works, allowing for an entangled space between voices, between signal and noise, for both to appear as method and subject matter.

Aura Satz: Tuning Interference on a Dark Matter Radio, 2019

RB: You are taking part in the Science Gallery, London exhibition ‘Dark Matter: 95% of the Universe is missing’, with a sound work Tuning Interference: Dark Matter Radio. Can you say something about your involvement in this?

AS: The curator Sandra Ross commissioned me to make a sound work responding to the theme of dark matter, under the guidance of the astrophysicist Prof. Malcolm Fairbairn, who invited Prof. David (Doddy) J.E. Marsh into the collaboration. I was really inspired by the way some of the experiments have been described as listening out for a signal that has not yet appeared. In particular I was drawn to the description of ADMX, one of many dark matter research initiatives (and a number of related experiments operating in Korea, Europe, and the USA), as “a radio that looks for a radio station, but we don’t know its frequency. We turn the knob slowly while listening. Ideally we will hear a tone when the frequency is right.[1] I wanted to work with this notion of experimental listening by making a sonic diagram of sorts, which would evoke a tuning experience. Together with Malcom and Doddy, as well as audio engineer/music AI specialist Dr. David Ronan who sonified the data, we made a 10 channel sound installation which renders a current hypothetical simulation of dark matter into sound. Essentially the sound patterns are a set of relations between the data, and we mapped it in such a way so as to generate intense psychoacoustic effects in the listeners, exploring sonic equivalents of interference and collision through beat frequencies and other diffractive qualities which shift according to the listener’s location. The listener becomes a radio dial of sorts, as the ears move through the soundscape, micro-tuning with each adjustment. It’s not dissimilar to the effect I described with the Fresnel lens framed drawings. There is no ideal vantage point or listening sweet spot, the listener is embedded within the sound, effectively generating the sound according to their orientation within the speaker ring.

Doddy showed me some visualisations of the simulation of dark matter in a hypothetical galaxy, and it looked like ripples of water or waves diffracting. This particular model of dark matter simulated contains waves[2], and we used speed and density to generate the shape of the harmonic structure. We chose the spacing of the speakers around the ring to be close to one wavelength, so that the coherence between speakers is audible, and yet varies in an interesting way around the ring. I wanted to create a soundscape that felt like a field of vibration and flux, with clusters of density, moments of tension and relief. Close frequency alignment and interference became a compositional principle, much like a kind of acoustic moiré. The arrangement of the speakers reflects the distribution of dark matter, so what you are hearing is not the sound of dark matter per se, but the hypothetical flux and motion of dark matter as rendered through sound. Each speaker is one point of data in the simulation, and if you listen close-up you will hear a singular slow-changing drone rather than all the beat frequencies that occur in the centre of the ring where the sounds interact with each other. Sometimes the wave shape of one point of data is extremely close to another, changing at a variable rate, and this alignment generates a sense of dense patterning, a pulse which gradually accelerates, intensifies, shifts focus and recedes. The sound is sculpted into a rippling flux which gathers and dissipates in such a way that is hard to hold onto or memorise. You can’t possibly internalise the rhythm of the piece, and each listening session will sound quite different from the previous one as your ears fabricate new acoustic illusions, adjust to the sounds, are de-sensitized or fatigued. I spent months tweaking the composition and by the end of a long session I wasn’t sure what I was hearing anymore, what was in between the speakers and what was between my ears.

RB: In terms of the viewer, what are you trying to communicate in this exhibition?

AS: I am interested in the ways in which scientific research activates or distorts a certain intuitive understanding of the world, and I try to find a way to make this come across on a very physical level, as visual or sonic experience. I wanted people to feel enmeshed within a dynamic rotational flow or current, something that can be sensed but which we don’t necessarily have the theoretical frameworks to account for. I find it fascinating that in current research on dark matter we are at a point of knowing unknowing, so to speak – we don’t know what exactly we are looking for and we haven’t yet identified what it is, all we know for certain is that its presence is somehow implied through the way it interacts or interferes with matter. Without dark matter many previously accepted theories are untenable, and as such it both disrupts and holds together different hypothetical theories.

I think the piece also conveys some of my previous concerns around modes of attention, a continuous retuning and recalibrating of what is heard and where one is positioned in relation to the signal or the noise (or what is understood as which). The idea of acoustic moiré – a morphing non-hierarchical, almost untethered grid – resonates with my interest in a multiplicity of voices which align and interfere with each other, activating the spaces in between. This is the reason I am drawn to work with sound – already at a very basic level it is doing the work as a vibratory in-between, as inherently relational, unsettling boundaries.

In addition to the speaker ring, I wanted a visual marker to provide a distinct sense of oscillation, that you are entering a vibratory sound field, so between each speaker there is a VU metre driven by the sound. The needle trembles to echo the volatile, dynamic and ever-changing frequency fields, though what exactly is being measured remains uncertain (the metres are blank and have no numbers).

Aura Satz, ‘Vera Rubin’s Irrefutable Evidence’, 2019

Nearby hangs a photo of Vera Rubin (1928-2016). Rubin was an American astronomer who discovered the galaxy rotation problem, providing evidence of the existence of dark matter. In the photo she is seen looking through a spectrograph mounted on the end of the telescope, recording an image of the spectrum (colours) of a small section of a galaxy. It’s quite an obscure image, in that she is wearing a hooded coat, so hardly any parts of her face or body are identifiable. Like the series She Recalibrates mentioned earlier, the image is framed inside a Fresnel lens, generating a diffractive pattern emanating from the centre, the point between her eye and the eyepiece of the telescope. The viewer has to somehow tune into the image for it to surface in amidst all the diffractive interferences and light play. It’s not central to the main piece, but I wanted to include Rubin as I think of the artwork as a space for naming, reconfiguring the canon, putting an underacknowledged female scientist into the conversation.

RB: In your view, what are the lines that connect art and science?

AS: I don’t think I could ever provide a definitive answer. The part that interests me from my recent experiences is the way in which both science and art ask questions and destabilise our current understanding of the world. Both are a response to curiosity and uncertainty, and can give us some orientation towards the future. Recalibration is key, an openness to change and a resistance to standardization.

Aura Satz: ‘Listen, Recalibrate’, solo show installation view, Fridman Gallery (photo Adam Reich)

RB: What other projects are you currently working on?

AS: For some years now I have been working on a project entitled Preemptive Listening, which looks at emergency signals and siren sounds. I read the siren as a specific kind of sound, one that requires attention, and demands an action or response. Citizens respond to its call, demonstrating obedience to its authority – it is a sound that commands submission, deflection, dispersion. It attracts in order to dispel, unsettles and resettles. It demands localized attention, and is the sonic architect of social order. It is a sonic marker that structures urban spaces in an emergency, a marker between future danger and dangers past, projecting a trajectory and expelling obstacles along the way. As the primary vocalization of the state, it articulates our relationship to power and civil order. All of which makes it fascinating, complex and in dire need of a re-wiring. My invitation is to pry it apart and recompose the siren, to think of it as a sound signal that requires recalibration. I am attempting to reimagine the siren sound: how can we open and destabilize this overly codified, prescriptive and stable semantic sound by taking a compositional approach, remapping new readings onto new sounds, how can we unlearn the existing code, find ways to listen differently, resist the hypervigilant, predetermined, automated call to obedience and set the intention to be curious, open, receptive, imaginative. If one remaps the sound, explores the possibility of different emotional registers, one can in turn generate distinct affective responses, more varied strategies for crisis management, and attend to a spectrum of different voices in need of our attention. And in our multiple modes of response, we might in turn enact an altered relationships to power.

[1] https://www.futurity.org/dark-matter-axions-detector-1726622/

[2]  Simulations performed at the University of Goettingen by Mr Jan Veltmaat, Dr Bodo Schwabe, and Prof Jens Niemeyer.

…………………….

https://www.iamanagram.com/

All images copyright and courtesy of Aura Satz

 

The post Voices appeared first on Interalia Magazine.

Cartoon Logic

Andy Holden: Laws of Motion in a Cartoon Landscape. Print No.4

Richard Bright: Can we begin by you saying something about your background?

Andy Holden: How far back should I go? I’ve just started reading Tristram Shandy and maybe I should go back to before I was born and start there. I wish either my father or my mother, or indeed both of them, as they were in duty both equally bound to it, had minded what they were about when they begot me; had they duly consider’d how much depended upon what they were then doing;–that not only the production of a rational Being was concerned in it, but that possibly the happy formation and temperature of his body, perhaps his genius and the very cast of his mind… I make art, in various forms, and have done since art-school, and did so before that, and made a few exhibitions in Public Houses. My Grandma collected ceramic cats and that was a big influence, my father is a bird-watcher by trade, which was initially not an influence, but now is crucial to what I do and resulted in us making an exhibition called Natural Selection together. I get my energy and the more social aspects of my personality from my mum, who did all manner of jobs. I also play music with my band the Grubby Mitts which we have been doing for a long term but struggle to get the music heard. I still live in Bedford where I grew up. I once started a failed art-movement called Maximum Irony! Maximum Sincerity, or MI!MS. Some of this sort of information filters into my art work, some of it gets used directly, some sincerely, some ironically, but non of it in itself is that interesting. I feel increasingly silly for having put as much of that information into my work as I have, but all of it informs how I see things and so I had to try and understand that filter that these biographical facts create, so I might know my own umwelt.

Andy Holden: Towards a Unified Thoery of MIMS, Zabludowicz Collection, installation view, 2013

RB: Have there been any particular influences to your art practice?

AH: There are a number of interaction with art works that in hindsight were pivotal. Some of these were seeing friends make things, particularly my group of friends that as teenagers tried with me to make MI!MS, especially the music they were writing. There were bolts of divine inspiration, the clouds parting and rays of light cutting through and the ground below gentle trembling from encounters with Andy Warhol paintings when I was about 14, as I saw a show at Tate and felt totally comfortable; like I understood it immediately, and felt for the first time legitimately like maybe I could be an artist too. I had similar encounters later with an epic Philip Guston painting; I had it peaking once into the microcosm of a Joseph Cornell box, I had it from following a trail of curious objects left behind by Marcel Broodthaers. I for a while idolised Robert Smithson. As a teenager I loved Silvia Plath and memorised some poems. I have complete reverence for Virginia Woolf. I for a time binged only on David Foster Wallace. I bought a Kurt Voneggut screen print that hangs over the Kitchen Table. I bought ever Super Furry Animals record. I had a Pavement phase. I love my friends Ed Atkins and Mark Leckey and Heather Phillipson’s work. The cultural black hole of Bedford keeps me routed as I can’t, as Alan Moore said of Northampton, get the velocity up to escape, and the influence of place can’t be under-estimated. My dad’s influence too, which I explore in Natural Selection. Cartoons maybe the single biggest influence, Wile E. Coyote, Bugs Bunny. Kids TV too. Reading Deleuze at college, reading Mark Fisher after college.  I’m a giant tangle of influences, for a time I feared I was just the total sum of all my influences, but now I have hopefully reached a point that I don’t really rely on them as much as I once clearly used to; or maybe there are now just so many that it’s hard to spot each of the ingredients in the murky brown mixture. Kanye West really kept me going for a while recently, the way he puts things together, but recently I had an allergic reaction and have weened myself off. I’m off to see Bob Dylan this evening at Hyde Park, I always promised myself I’d see him once, as of course at some stage he was an influence. In all honesty like most now my visual diet is weird clips on YouTube or Threads on Twitter and these are what get under my skin.

Andy Holden: Eyes in Space.

RB: What is the underlying focus of your work?

AH: The layer of strata at the bottom of all this, once all the sediment is scraped away? I’m still digging down and down, tunneling, trying to stop the inevitable synchronistic motion of things and enter into a more diachronic movement. However every time the spade strikes something blunt and hard I explain with glee, ‘the bottom, the bottom, the bed-rock’, I’ve arrived! – only to find it is just another rusty old chest or lump of slag from the anthropocene or worse still my memory; and the underlying matter is still deep below and I can still hear it rumbling. If one day I have exhausted the justifications, I have reached bedrock and my spade is turned. Then I am inclined to say: ‘This is simply what I do’. That last lines not mine, I just remembered it, after starting what I thought was an original metaphor, it’s a quote from Wittgenstein.

Andy Holden: Laws of Motion in a Cartoon Landscape, character study.

 

Andy Holden: Laws of Motion in a Cartoon Landscape, promo.

RB: You are taking part in the Science Gallery, London exhibition ‘Dark Matter: 95% of the Universe is missing’, with an immersive new installation ‘Laws of Motion in a Cartoon Landscape’. Can you say something about your involvement in this?

AH: I think in all honesty my involvement is a bit tenuous. Laws of Motion is as much about politics, economic and art as it is physics and really nothing to do with Dark Matter. It looks at Cartoon Logic, and re-writes the rules of the Cartoon World, based on O’Donnells laws of the Cartoon, but to try to explain that the world has now become a cartoon. This requires a quantum entanglement of physics with everything else, and makes claims that forces such as Gravity only take place when we are aware of them and so need to be linked to consciousness, and that all matter in the cartoon world is conscious and sentient.    It took six years to make it, and I finished it in 2016 just at the moment of Trump and Brexit and suddenly the notion the world was a cartoon has a more persuasive validity.

RB: You have stated that “an exploration of cartoon physics might help us understand the world we now inhabit”. Can you say more about this?

AH: If the world is now a cartoon, then the best way to understand it is to examine how physic and logic work in the very cartoons that first created this landscape, and how this new non-logical and physical space was created and able to be visualised. This is a diachronic movement, or Marxist premise; we look at how something was formed in order to understand how it now works. Cartoon physics was created by many things happening simultaneously; changes in theoretical physics – space-time changed, certainly were a major factor, but it wasn’t just a new understanding of the physical world that made this possible; simultaneously photography advanced, it became possible for images to move, Freud discovered the unconscious, Cinema created a new mass spectacle, modernism saw objects being split into artificial pieces as the whole was seemingly dismantled, and as speed increased understanding of the world shifted, and objects seemed to take on a life of their own. Law 1 is –  Any body suspended in space will remain in space until made aware of it situation. This for example, as the film shows, is a good way of explaining both the financial crash of 2008 and the method of the artist to make artwork in the world at the moment.  And in the last two years has been the go-to metaphor to explain almost every political moment from Brexit onwards, it’s an image entirely suited for our times; you won’t fall down until you look down. I wish I had collected every instance in which I had heard this analogy deployed on the news. Those that don’t look down are the only ones who can survive in the current moment. That’s why Bugs Bunny is who we need to aspire to, as he/she can navigate the landscape perfectly.

RB: In terms of the viewer, what are you trying to communicate with this work?

AH: At times the work feels close to the tone of a conspiracy theory video, and it should explain how a view of the world can be created and made plausible through the combination of otherwise unconnected elements, and make us aware of how easy this can be. And how in a space where it seems anything can happen not anything can, rules, or laws, are always being created. The work is 10,000 words spoken at the speed of a cartoon chase sequence, it is a cartoon of a lecture and a lecture on cartoons; it’s very hard to say something about the work that the work doesn’t already say.

Andy Holden: Cartharsis, ceramic cats, grandma collection, 2016

 

Andy Holden: Catharsis, ceramic cats, grandma collection, unboxing video, 2016

RB: In your view, what are the lines that connect art and science?

AH: Imagination and inquisitiveness are two major motorways between the two capital cities, but all lines connect to all other things, they are just more minor roads. We live inside an epic mesh, it’s just some lines become more dominant through more constant use, the ones we build service stations on. In the modern period we tried to make all disciplines appear separate and unconnected but this, even at the very first moment of the creation of the air-pump, as Bruno Latour shows us, was never really the case. Perhaps, to go back to the previous question, it’s to make this interconnectedness more visible that is part of what the work attempt to communicate.

Andy Holden: Pyramid Piece and Return of the Pyramid Piece, (knitted yarns, foam, steel, 3m x 4m x 5m) 2017, installation view Tate Britain, (photo credit: Andy Holden).

RB: What other projects are you currently working on?

AH: I’m actually a bit stuck right now. I’m running a small project space in Bedford, showing an exhibition I’ve curated called The Long Revolution, looking at change in the countryside since the enclosures act – from the poetry of John Clare as explored by Andrew Kotting to Mark Baumers death walking bare foot across America in 2016, however I have not had a single visitor to the show in three weeks. I have written a new pop album with the Grubby Mitts but we can find a record label after I folded the little label I used to run as it was suffering the same fate as the project space now is. My dad and I are collaborating on a project about bird migration routes for a performance in February, and our collaborative exhibition Natural Selection is currently on show at Bristol Museum until September and so that should give me some studio time to scratch around and feel out what the next project might be. I now unfortunately can no longer kid myself how long it takes me to make a large scale new project, four years is a quick one, although other things I can do quicker. So it is just little more digging until I’m able to just say, hopefully; this is simply what I do.

………………….

https://andyholdenartist.com/

All images copyright and courtesy of Andy Holden

The post Cartoon Logic appeared first on Interalia Magazine.

On the Surface

Richard Bright: Can we begin by you saying something about your background?

Rachel Pimm: I have a pretty straightforward background in Fine Art. I studied an undergraduate Fine Art degree at Central Saint Martins in a discipline called 4D which was developed from a blend of Critical Fine Art practice, film studies and then performative and video practice- all the extras to painting and sculpture. Then I started a project space called Auto Italia, which is still going, and went on to a postgraduate MFA programme at Goldsmiths. Those initial feelers into curating, which also included a curatorial internship at the ICA and a short stint after working in the department, have now morphed into more of a collaborative practice, where I invite people to do projects with me.

I wasn’t born in the UK though, my family are white Rhodesian and I came here from Harare, Zimbabwe in 1986- shortly after independence from Britain. This perhaps affects my interests in ways I haven’t yet unpacked, but It certainly gives me plenty to work on and address in terms of my own relationship to the world as a European settler thinking about land management and colonial histories.

Lori E Allen: My background is in social science with a Bachelors in Anthropology, Classical Studies, and Ancient Latin from New York University and a Masters in Archaeology from University College London. I never trained formally in fine art or music but when living in New York began a very low key experimental type of art practice around media archaeology. This began as a method of chopping up broad-casted media in real time to extract concurrent narratives across television networks, and took the form of a weekly public access television series. I then expanded that focus to include field recordings and began focusing more on sound and sound scape than image. My practice is still generally very low key in that I while I do make solo work it’s mostly for my own amusement, and I prefer a collaborative approach in creative work.

RP: when Lori and I work together we are good at thinking about larger scales than I do on my own- humans, animals, time, place. Lori is very technical and I do admin 😉 I’m only half joking. We have made performance together since 2015. You can listen to worming out of shit performed at the Chisenhale Gallery in 2015 here and Disintegration, performed at the Whitechapel Gallery earlier this year, here.

Lori E Allen and Rachel Pimm: Disintegration, Whitechapel Gallery, 2019

Lori E Allen and Rachel Pimm: worming out of shit, 2015, performed at CCA Glasgow, 2016

Lori E Allen and Rachel Pimm: Disintegration, Whitechapel Gallery, 2019

RB: Have there been any particular influences to your art practice?

RP: For a short while I did a full time office job at the Ideal Home Show, designing and specifying the model home village at the show – then building show houses inside Earl’s Court. For me, not only is this display format just as interesting historically as gallery exhibition, its lineage can be traced via British Colonial histories to the Crystal Palace, botanical gardens, raw materials, the industrial revolution and housing after the Garden City movement- really rich (and violent) contexts for the relationship between stuff we take from the ground and the systems of engineering and capitalism that surround its movement and processing.

I’m also influenced by Natural History. I find old books and go to libraries a lot – the Geological Society, Linnean Society, Bournemouth Natural Science Society library, the Teri Institute in New Delhi, and then gardens and greenhouses, physics evening classes, kids science kits, conversations and study days with friends, and strolling around, taking photos of infrastructure space, and ways in which ‘Nature’ is being put to work. I learn a lot from living with houseplants about care, and time and growing.

LEA: For me it’s probably studying archaeology that has influenced me most. I like thinking about and witnessing the ways people, myself included, interact with and build relationships with inanimate objects – of which I would also class mass media. There is so much story-telling in the scars, rips, wear and tear of a thing. Yet the thing is silent while retaining a record of events it’s undergone.  Conversely, mass media is not silent. I think the pull from the mute objects to the noise of constructed narratives led me to think more about noise and silence in internal dialogue, where it comes from, what records it holds  – and I suppose focusing on broad-casted media, especially the way it is told/digested/re-told/re-digested is a way of getting at/excavating the noise and silence of objects embedded in narrative rather than the earth.

RB: What is the underlying focus of your work?

RP: I suppose what things are made of, how things work, and how they change the environment are my primary focuses. I find the idea of the surface of something especially interesting because this is the place where the change visibly happens. Also because everything material comes from the surface of the earth. The theory conversations around materialism where they combine with climate activist, feminist, queer, crip or anti-racist work are part of what engages me in this and the desire to understand the environment through structures other than Cartesian western patriarchal power.

LEA: I’m really interested in mass behaviours, ritual, and taboo how they generate, what they come from, how enduring they are, and what influences rate of change. Rachel and my underlying interests are really different in this way, but her focus on material and the surface of the earth I have found to be a really good way round again to approach such questions. It is literally from the ground up and makes human society kind of less important – which is a relief.

RP: and Lori helps me think of ways to create narratives around images and sound. She’s a good storyteller and while I’ll Wikipedia something or buy something, she’ll just get on and make a test. Lori is very unafraid of trying something even if it sounds hard or stupid, because doing it mechanically almost always works something out.

Rachel Pimm photographing landscape at Dallol in the Afar Triangle, 2019

RB: In 2018 you were involved in the project ‘Experiments in Art & Science’, a collaboration between Kettle’s Yard and The Gurdon Institute in Cambridge. Can you say something about your work in this?

RP: I had a period of around 9 months where I was supported by a stipend and was given fantastic access to the genetics labs run by Eric Miska at the Gurdon Institute who had approached the art partners to be able to make a project with no fixed outcome, refreshingly. I was matched with some scientists whose research I could engage with and rather than being a project, it became more of a change of practice and an opportunity to learn and reflect. I read a lot, and used much of my production time and fee to collate a lovely library of rare and specific books on morphology, geology, biology, and also go on field trips to archives to see the lineage of the research into morphology- through fish and worms. I learnt some amazing things- like that transgenerational traumas are chemical in cells, that from suboptimal exposure to environments that suppress life (and those survived by ancestors) sit in a protein you can dye and actually see under the microscope in the RNi. That’s how life forms become resilient. That is mind blowing. I also now understand that ALL patterns of growth and shapes in nature are connected in a spectrum, and that this is due to chemistry and physics at a cellular level, also completely amazing and an overhaul of my thought process.

Rachel Pimm and Emilia Santos at the Gurdon Institute Cichlid Aquarium, production Morpho Chemical, 2018 (Experiments in Art and Science  residency, Cambridge University and Kettles Yard)

Rachel Pimm at St Andrews University Special Collections library, production for Morpho Chemical, 2018-19 (Experiments in Art and Science residency, Cambridge University and Kettles Yard)

Rachel Pimm at Giants Causeway, Northern Ireland, production for Morpho Chemical, 2018-19 (Experiments in Art and Science  residency, Cambridge University and Kettles Yard)

I’ve since also been lucky enough to do more labwork, including some photomicroscopy, with Radar, in Loughborough at the Chemical Engineering department.

SEM microscope samples from Afar triangle fieldwork, production for S, Lori E Allen and Rachel Pimm, 2019 (Radar residency, Loughborough University)

RB: You are taking part in the Science Gallery, London exhibition ‘Dark Matter: 95% of the Universe is missing’. Can you say something about your involvement in this?

RP: I was approached to work on a set of new elements in the Periodic Table- which is one of my catalogues of collaborations, and I approached Lori E Allen to come on a very amazing field trip to the Afar triangle triple rift junction in Ethiopia and make work together in response to Sulfur. This has since turned into a project about alchemy- we’ll be doing a performance at a late event, and then showing S and Hg at an event on August 7th. I recorded mainly images and Lori recorded mainly sound but we will fully collaborate on the works in the events.

LEA: I was invited by Rachel Pimm to take part in this project, which follows from a previous work we did with the material from Ethiopia, linked above :)!

S, Lori E Allen and Rachel Pimm, 2019 (Dallol)

S, Lori E Allen and Rachel Pimm, 2019 (Lava fields at Erta Ale crater)

RB: In terms of the viewer, what are you trying to communicate in this exhibition?

RP: Art doesn’t have to communicate. That’s one of the great privileges about making it.

LEA: I agree with Rachel’s comment above and would add that in my experience it’s very hard to control that aspect in the first place as well as limiting the opportunity of a work to communicate how it will or won’t with anyone seeking to engage with it.

RP: Maybe you can learn and control some of the tools at your disposal so you can take a didactic political position where needed or get people thinking on a particular track, like when we talk about the use of vocal samples.

LEA: Hmm. yeah. But even so what you want to communicate may not translate the way you expect it to, and that’s kind of the best part about a work becoming organic/non static

RP: But aside from this, we want to show people the images and sounds we experienced when we went off to investigate sulfur as an element important for life forms. We have looked at a lot of Alchemy practices and cultural references of burning hell holes (Sodom and Gomorroh! Volcano Deities!) and tried to incorporate those alongside harder sciences like  geology and chemistry. We think it will be messier than the show itself, perhaps more true to the way matter operates.

RB: In your view, what are the lines that connect art and science?

RP: Both seem to be about trial and error, failure, learning and experiments. Art can change form part way through though, methodologies can be really sloppy compared to the science. In both making and in publishing- peer review matters to both, but I find science a bit of a straight-jacket and would want to work on more things at once, not finish things, do things incorrectly or whatever. I have a lot of respect for the focus of scientists. Artists working with scientists have the pleasure of finding out facts and then ignoring them entirely if they want. I imagine scientists like the lateral thinking opportunity to flex their ideas. Sometimes scientists actually want an illustrator, which is a misunderstanding. Illustration is a whole other, also very interesting thing. But that’s not what artists do well.

LEA: Both are linked by inquiry of the observed or experienced and function in similar ways in the quest for understanding a thing, a relationship, a process, a being. While the scientific method is robust and disciplined this can have limitations. Similarly, so can art.

Lori E Allen making contact microphone and hydroponic sound recordings on a field trip in the Afar triangle, 2019

Rachel Pimm looking at Lava Rock samples, Erta Ale crater, Afar triangle, 2019

Geological samples and AV equipment on a field trip in the Afar triangle Lori E Allen and Rachel Pimm, 2019

RB: What other projects are you currently working on?

RP: I have a performance coming up making some music using cash crops and plants that are domesticated at the Serpentine Gallery and am working on a menu of earth-based food for an event in Lincoln at Mansions of the Future around the practice of Geophagy.

LEA: I have vinyl coming out soon with my band from a piece we did for the Tate Modern a couple of years ago. I am working with another artist producing soundscape compositions for a performance work investigating medical imaging, for which I will also produce the soundtrack for a short film version of the work. Perhaps what I’m currently most excited about is a children’s book.

RP: We have made a start working on a video project about Salt- its geology, history, production, trade and processing. That was an unexpectedly big part of what we saw in the Afar triangle and there are a lot of leads to follow. It was bigger than the elements so it has become its own project.

Lori E Allen making sound recordings on a field trip at Lake Asale in the Afar triangle, 2019

Rachel Pimm sampling lake salt crystals during a field trip at Lake Asale in the Afar triangle, 2019

…………………

All images copyright and courtesy of Rachel Pimm and Lori E Allen

 

The post On the Surface appeared first on Interalia Magazine.

95% of the Universe is missing

Richard Bright: Can we begin by you saying something about your background?

Malcom Fairbairn: I am originally from Wigan in the northwest of England, I did my UG and PhD at Birmingham, Cambridge and Sussex then I did postdocs in Brussels, Stockholm and at CERN before coming to King’s College London just over a decade ago. I am a theoretical physicist (so I do calculations rather than experiments), and I work at the intersection of Particle Physics, Astrophysics and Cosmology. I spend a lot of time thinking about the nature of dark matter and how we might learn more about it or even better, discover what it is precisely.

RB: You are involved in a season of events, ‘Dark Matter: 95% of the Universe is missing’, taking place at the Science Gallery, London. Can you say more about your involvement in this and what this season is hoping to achieve?

MF: I had two main roles.  The first was as curatorial advisor to the curators who were putting the exhibition together. The team has evolved a lot because of maternity leave and changes in personnel. When we started the core was myself, Jen Wong, and the main curator Sandra Ross. The old director of Science Gallery London, Daniel Glaser, was also involved in the motivation for this season. There was also a bigger advisory pool of artists but in the early days, Sandra and myself formed the core of putting the exhibition together under moderation by Jen. My main job, the guess the role my personality drove me towards, was to be open minded to the different kinds of artistic interpretations that Sandra would try to discuss and develop, the different threads and kinds of ideas she wanted to develop. However, when I felt she was moving too far away from anything even remotely related to the science that we do, I expressed myself quite clearly. Sometimes we had quite deep differences of opinion. It was not easy for me to work with Sandra and it was not easy for her to work with me, but I am proud of what emerged and I am proud to now call Sandra a friend.

RB: What is dark matter and how is its existence and properties inferred?

MF: We think Dark Matter is some mysterious particle which we think is zooming around, through the earth, through rooms, through the air, through whatever environment you are reading this. I say “zooming” because we think it is moving at several hundred kilometres per second. We think it is there because when we look at the motion of objects such as stars and galaxies, we can only explain how fast they are moving due to some extra gravitational acceleration created by some matter which must be there but which we cannot see.

From a physicist’s point of view, dark matter is rather simple, actually much simpler than normal matter. Normal matter is governed by several forces, gravitational force, electromagnetic forces and the nuclear forces, so it sticks together to form different elements and molecules and emits radiation to cool down and stuff like that.

The only force that dark matter appears to obey is gravity. In particular, it doesn’t emit any light and you can’t bounce light off it, so it’s transparent. Perhaps we should have called it invisible matter, but because we deduced its existence by looking into the night sky and because it doesn’t emit any light, we ended up calling it dark matter.

We can tell it doesn’t stick together or emit any invisible light or even collide with itself by mapping out its distribution in space due to its gravitational effects.

RB: Are there differing theories that aim to provide an explanation for dark matter?

MF: There are lots of different kinds of ideas as to what kind of particle the dark matter could be, and lots of different experiments trying to find the dark matter. Some of physicists favourite theories are the kinds of particles that explain some other complicated problems in physics, such as why there is more matter than anti-matter, but there isn’t really any reason why dark matter should do this. Some physicists think that there is no particle at all, but rather that we have misunderstood the force of gravity on huge scales, but most physicists, including myself, think that some particle which emits no light is the culprit.

RB: Dark Energy is pushing the universe to expand faster and faster. What is dark energy?

MF: Haha! What is dark energy? Well, this is the trillion dollar question. In some sense dark matter is quite boring in comparison, dark matter is just like normal matter that you can’t see, and as the Universe expands it gets spread out. Dark energy is an energy field which doesn’t get spread more thinly as the Universe expands, as if it is being constantly created from nothing. This sounds impossible but we think such exotic forms of matter and energy could exist in the Universe. It turns out we seem to require both dark matter and dark energy to explain the Universe, which is very unsatisfactory for science, but honestly, if you try to live without either it is very difficult to explain the observations.

RB: You collaborated with artists, Carey Young, Agnieszka Kurant and Aura Satz, who are taking part in the Science Gallery, London exhibition. Can you say something about your involvement in these collaborations?

MF: So, mainly I spent time with the artists, many happy hours, listening to their thoughts and explaining about dark matter and dark energy. They often generated analogies and asked me if this is a good fit to what I am describing. They wanted to describe things in terms of things they were familiar with. In some sense this is something we don’t need to do so much as physicists since we describe things in terms of mathematics. Sometimes there are analogies which are pretty exact, and sometimes this kind of exercise fails completely.

Agnieszka Kurant was extremely challenging since she tried to make a lot of connections between many different areas with a freedom I hadn’t experienced before working with artists. It was pretty vertiginous trying to keep up with her train of thought sometimes and the logical jumps that she took in using one issue to mirror another one were quite stretching for me, but in a fun way. She was trying to understand the link between invisible structure emerging from randomness both in society and in the Universe in dark matter. The link is less obvious than in other pieces and also contains elements which are based upon the transformation of energy from one form to another, which is very important in cosmology.

Aura Satz was perhaps more measured in her approach to the piece which she developed. I facilitated a collaboration between herself and some of my colleagues from Goettingen who had performed simulations of dark matter in galaxy. The piece is ten simultaneous pieces of sound which are representations of ten different positions in the galaxy and the interference between them as you move around gives rise to interesting effects. The idea was based upon the idea that you need to tune into the dark matter to detect it in certain models of dark matter referred to a axions.

RB: What questions do you want to address in these collaborations that could not be addressed before?

MF: Each artist had a different set of ideas they wanted to explore. The work with Carey Young, for example, was created nearly a decade ago, with me, and it explores ideas of ownership.  A vessel will have dark matter inside it but it isn’t trapped there – it is constantly flyting through it so if you were to sell the vessel you would be selling dark matter too, although not always the same dark matter.

Many of the pieces in the exhibition question the nature of scientific belief. I guess it takes a great deal of trust in the scientific method to convince oneself that dark matter should exist especially because you can’t see it. We try not to use the word “belief” as scientists but it regularly pops out of our mouths by accident. The word means something different for us though, I think the way physicists believe in things is somewhat different.

RB: What have you personally learnt from working in these collaborations and has this approach thrown up any surprises for you?

MF: It is surprisingly difficult to try to explain why you have come to the conclusions that you have about the Universe to someone who is coming from such a different perspective. I think some of the logical steps that you take about various things are shown to be slightly weaker than you expected.  Ultimately, I think explaining this stuff over and over again to different artists has at least explained to me what I am comfortable with and where my uncertainties lie.

RB: In terms of ‘ways of seeing’ what do regard as the main meeting points between artists and scientists? And what are the differences?

MF: I think artists and scientists are trying to represent things they observe around them in different ways, a scientist will take a thing and only record certain characteristics (position, mass, velocity etc.) and interpret it in a certain way. An artist will record an event using totally different information (feelings, colours) which may transmit equivalent data, with perhaps gross features explained less precisely but detailed features pointed out more clearly.

Obviously the biggest differences is the fact that scientists use mathematics to deduce things, and some things are discovered unexpectedly through calculations and simulations, which often we are not clever enough to predict until they fall out of the mathematical equations. So that is something that doesn’t follow over too well.

RB: Collaboration between the arts and sciences has the potential to create new knowledge, ideas and processes beneficial to both fields. Do you agree with this statement?

MF: I think that working deeply with artists is a healthy exercise for a scientist and will always help them to question their own ideas and practise. There are things that I learnt and new techniques that I picked up which are not immediately useful for my research, but I am a theoretical physicist, so it is quite difficult to come up with stuff that directly affects the day to day work I do. However, I do think it has led me to analyse the kind of ways I present arguments in papers. I think it has affected the way I write introductions and conclusions, and the kind of statistical analyses that I use to prove a point. I think that these changes are subtle, but the kind of self-questioning of practice that you can only get by literally spending hours with someone who is almost trying to understand an alien culture is deep and remains with you.

The post 95% of the Universe is missing appeared first on Interalia Magazine.