ML, VR and AI: How emerging technologies are reshaping creativity

It’s hardly news that emerging technologies, from artificial intelligence and machine learning to virtual and augmented reality, are beginning to impact our societies and shape our public discourse. But the conversations around how these tools are influencing, and will continue to mould, the creative fields of art and design are still really in their infancy.

In New York ten days ago, we teamed up with the website-building platform Wix to host an event in the company’s brand-new space in the city (just opposite the Whitney Museum). On the agenda were the ways in which pioneering designers and artists are using emerging technologies in their creative work. Four creatives took to the stage one by one to talk about how new technologies, whether AI or VR, feature in their practices. Running through the four talks were several common threads.

Each speaker explained how emerging technologies allow them to create effects and outputs that would otherwise be impossible. Multi-disciplinary creative Laura Juo-Hsin Chen took to the stage first and showed the audience a few of the projects she’s developed that involve virtual reality. Her work is incredibly playful – one project, for instance, allows people to eat a meal with peers worldwide around a shared virtual table, while another allows users to do a virtual poo in a virtual toilet bowl (no, really).

Yet underneath the jokiness, there’s often (although not always) a deeper message. A further project, Sleep Together, allows someone feeling lonely and unable to sleep to enter a peaceful virtual world, where other people around the world in a similar sleepless state (depicted as soft and cuddly clouds) can be joined. “One of the features of VR is creating the feeling of presence,” Laura explains. “It could be the presence of a human or an animal or nature or simply energy flow. And with good intentions and narratives, I believe it can make people feel the presence of a companion and a sense of belonging.”

The artist and creative coder Ezra Miller, whose work we’ve written about many times before, was up on stage second and also pointed to the fact that technology is a powerful enabler. His work is complex, combining his own animation style with the “morphing latent space animation” of a special type of machine-learning system called a generative adversarial network (or GAN). Still with me? Without going into the detail of it too much, this technique allows Ezra to create seemingly realistic images that are constantly shifting and changing, becoming by turns clear and immaterial. “I like the way they appear so uncanny yet realistic,” says Ezra.

His preferred tool is TouchDesigner, a node-based visual programming language used by many artists and creative coders. Using this, Ezra created the visuals for a live audio-visual performance by producer and DJ Objekt, which has been touring the world this year. Incredibly, these visuals are produced live on-stage by Ezra, something that wouldn’t really be possible without the technology. “TouchDesigner makes it extremely easy to work in real-time,” Ezra explains. “I find using it in combination with a MIDI controller gives me total control over creating a responsive live show. The visual interface also helps me organise ideas that would be hard to keep track of if I was using purely code.”

Yet, if such technologies are on the one hand a great enabler of creativity, they are also something that artists and designers need to treat with respect. Creatives have to figure out what their relationship to these technologies should be. One person who has thought about this more than most is the British artist Ben Cullen Williams, who divides his time between London and New York. Most of his work is fairly straightforward from a technology perspective, having previously spanned the more traditional mediums of sculpture, film and photography. But earlier this year, he began a project involving AI.

At the end of 2018, it was announced that world-renowned choreographer Wayne McGregor had collaborated with Google Arts & Culture Lab to create Living Archive, “an artificially intelligent choreographic tool, trained on hundreds of hours of video from the choreographer’s extensive back catalogue”. This summer, Ben was tasked with using the tech to create a video installation that accompanied a “live AI performance experiment”. The result was amazing.

During the process, Ben worked with the team at Google Arts & Culture to understand the AI. “I tried to see the AI as an equal collaborator rather than an output from a machine that is there to be manipulated to achieve a perfect vision,” Ben says. “As a result, the dialogue was closer to a collaboration with a person than a computer, respecting what the computer made and including it rather than discounting it.”

This approach led to an interesting revelation about the nature of beauty. “AI doesn’t have the same human bias as we do,” Ben explains. “As humans we are socially conditioned to recognise a certain type of beauty. The internet affirms this construct and the result is an echo chamber from which it’s hard to escape. As AI is not within this space, it has the power to disrupt the status quo and lead to unexpected results.” At our event in New York, Ben reflected on this and asked the audience if we would allow AI to help us interrogate our understanding of beauty. “It is,” he says, “the role of the artist to challenge the role of new technology within society.”

Questioning the role of technology within society is exactly what creative technologist and designer Ari Melenciano has spent the past few years doing. When she was in her second year at grad school in New York, she decided to set up Afrotectopia, a social institution fostering interdisciplinary innovation at the intersections of art, design, technology, black culture and activism. Having grown up in a predominantly black community in Washington, D.C., she arrived in New York and says she found herself entering a predominantly white and Asian space. “Beyond that, I was realising how easy building cool things with technology could be with the right access,” she says. “I became disheartened in realising how little access and exposure my community had to these technologies. I knew if more black people had access to these resources, we’d be making some amazing pieces of work.”

She also began to see “how deeply embedded technology is in our everyday lives” and that, more often than not, “it’s the replication of racist analogue practices.” Not being able to find a community of people who were thinking about race, black culture and technology in combination, she decided to begin setting that community up herself.

Ari shared several projects with the audience at the event, including Electro Negro Synesthesyo, which reimagines black cultural objects. “I was thinking about how beautiful I find black cultural artefacts, but also how negatively they are stigmatised when worn on black bodies,” Ari explains. The project consists of a series of sound and visual instruments, built with technology-infused black cultural artefacts. As you touch the objects – bamboo earrings or a durag, for example – sounds are triggered. As Ari says: “I was thinking about the future of black culture, and how we can control and design our own narratives, as black people.”

Today, Ari uses her platform to advocate for a better technology sector and more equitable tech products. Asked what needs to change, she says, “There absolutely needs to be some sort of manifesto that all public technology companies and digital services abide by, making sure that all applications are safe to each and every community. No community deserves to be exploited and further marginalised due to technological services.” For her, a major part of this is equitable access to technology education, leading to more diverse technology and design industries – something she’s doing with Afrotectopia. “The possibilities and humanness of the technology,” she says, “will only thrive with more racial and ethnic diversity.”

Our event at the Wix Playground in New York covered a huge amount of ground, from the nature of beauty to the need for equitable access to tech, via loneliness, sleep deprivation and GANs. Clearly, creatives are seeing new technologies and tools as ways to experiment and produce exciting and novel outcomes that would otherwise be difficult, if not impossible, to achieve. Yet, in the same way that the influence of certain technologies on society at large is currently being challenged, so designers and artists should also question their role in this and how they’re using such tools in their work. What’s clear, either way, is that the link between creativity and technology is only going to get stronger.