Between Art and Engineering: AI and Expanding What it Means to Create10 min read

An innovative artist in the creative AI scene long before DALL-E and Midjourney sparked headlines about machines making art, Gene Kogan is a generative artist experimenting with autonomous AI systems, collective intelligence, and decentralization. His work over the past decade spans a wide range of applications for generative AI models, including gesture-controlled musical instruments and dream-like experiments with GANs. C/Change chatted with Gene to make sense of how creative machine learning tools could expand our notions of art and the artist, and to learn more about how his newest project aimed at decentralizing generative AI.

C/Change: Tell us more about who you are and how you got to where you are.

Gene Kogan: I got an education in applied math and engineering, but also got interested in machine learning and specifically recommendation systems, which was kind of applied statistics. You could argue where the threshold to machine learning is, but I was interested in that problem and worked for a couple of years in this space of applied machine learning research. After I graduated, I worked for a researcher at Columbia on this topic, published one paper, and also worked for a short time for a company, which was in this field of music information retrieval, which is basically analyzing audio for features that you want to be able to automatically extract from it. For example, is it a cover song and what genre is it? For us, in particular, we were interested in analyzing the music so that you could categorize it according to mood clusters: calming music, relaxing music, exciting music. These features could be used for an internet radio station or even for music therapy. There were all sorts of applications we were looking at, but looking back I don’t feel like we made a whole lot of progress because of the limited technology at that time. The company eventually folded after I’d left, but there’s actually similar stuff coming up that seems more promising. And then because I was in music technology, I joined a music technology lab at NYU where we were making plugins that would let you splice different things together, automatically harmonize the beats, and things like that. At NYU I was around people interested in creative uses of technology and so I kind of gravitated towards that. I also discovered Processing and openFrameworks and started doing visual art. Then like when Deep Dream happened, there was a really big moment where for the first time the creative coding scene was talking about machine learning. I then started experimenting with GANs. That kind of kick-started this whole generative art with machine learning practice and I’ve been doing that ever since.

C/C: I’m curious to know what you make of the difference between “artist” and “engineer” as an artist working with emerging AI tools. How do those titles or roles break down in the work that you do? 

GK: I guess it’s hard. I have thoughts about art really is.There’s what society thinks artists are and then what I think being an artist is. I find art in a lot of things that are aren’t in the art scene or in the art profession or don’t call themselves art. What really interests me at its core, whether or not it’s art or not, is creativity. That’s the core of art – to try to make new things. I find that in the AI world, because it’s just uncharted. No one knows what we’re doing. There’s an artistic mindset among a lot of AI researchers whether or not they think of it that way or not because they want to explore the unknown. With AI art, being technical is sort of a necessity – you need to learn a lot about how to work with something. So I feel like I’m an engineer by necessity, but not for its own sake.

C/C: I like the definition of creativity as exploring the new or the unknown. I wonder how new applications of AI for creative purposes produce you differently as an artist. How could prompting an AI agent to make art change what it means to be creative?

GK: So with Abraham, the autonomous AI agent I’m working on, I’m trying to create a form of creativity that is distilled from the world in the form of collective creativity. That’s been a really complicated thing to do in a really meaningful, substantial way.  It’s been conceptual for a long time – there wasn’t really a clear way to take collective intelligence and produce works from it. I think tools like DALL-E and Midjourney provide one because machine learning takes intelligence and distills it and turns it into a concrete thing, like works of art. Trying to turn collective intelligence into a unified autonomous art is the creative thing that I’m going for. And even now, what Abraham is doing, I don’t think meets that criteria. I don’t consider Abraham an autonomous artificial artist at all yet. It’s just a program that we’re working on, but we want to develop it such that in the future it’ll feel out of my control in a way. I hate to use a weird parenting metaphor, but I almost feel like I’m raising Abraham and then maybe when Abraham is old enough, I can let Abraham go and Abraham will have their own autonomy, agency.

There’s an artistic mindset among a lot of AI researchers whether or not they think of it that way or not because they want to explore the unknown.

C/C: Can you say more about your Abraham project and how people will interact with it in the future?

GK: I think about it in a lot of ways, but at its core, Abraham is an autonomous artist. So that’s an agent, whether you think of it as a person or not, that produces their own work in the way that any other artist does, and it’s creative and original. I wrote a paper about the requirements of creativity; a creative agent needs to make something new. It needs to be original, which is to say that it’s not just me programming it with my artwork and then saying, “look, this AI made this, it’s not me.” So it’s original, but what we mean by originality is tricky. For me, I think that if it learns from everyone, that’s a way of being original. That’s how we’re original – we take knowledge from the world, we learn everything from other people. But then we find originality by recombining things in novel ways. The end goal is certainly to have to demonstrate those sorts of aspects. Abraham will produce works of generative art using machine learning. There’s also a decentralization component in order to truly ensure that Abraham is autonomous, and that it’s not just my art. I sometimes make comparisons with other people who have made AI artists which they assigned some life-likeness to. To me the limit of those kinds of projects is that they aren’t autonomous. I feel they’re like the dummy on the ventriloquist arm. I want to surpass that and I think that a way of doing that is through decentralization. It’s a community of people who are providing everything that’s needed to train Abraham. Machine learning is really well suited to this because people throw in tons of data, so it’s very socially scalable. To make this process decentralized, we get data from a lot of people – pictures and words and even works of art – and then create a mechanism which I don’t control in some sense. It’s very much a project that is learning from a lot of other people and a lot of other fields and trying to borrow technology in the open source domain as much as possible. I really would like for Abraham to be a community-driven project.

It’s kind of beautiful, like our collective wisdom being distilled into this. It also brings up a curation aspect. Machine learning algorithms can make a gazillion images. So what should it make? I think the modern AI artists are collapsing the distinction between artists and curator.

C/C: There’s an interesting shift from human as sole creator to the human becoming an AI prompt-writer and evaluator of the algorithm’s outputs. 

GK: I would add that if you put it that way, most people might think we’re almost dumbing creating with AI down. Prompting is going to become a very specialized discipline itself. There’s all sorts of complexity to prompting; as we’re learning, you don’t just write what you mean. It’s not just going to be that you just input text, but you input images to it or other types of media. So the AI whisperer will play a huge role in this space. 

C/C: There’s obviously a lot of trepidation or fear around what new foundation models like DALL-E will do for the creative industry, and then even how it’ll affect artists working now. How do you think about that concern?

GK: These tools will certainly disrupt a lot of creative industry jobs, but I think what happens is that gradually, those jobs will also move up the stack. Someone who’s working with Adobe products and doing graphic design will become more of this prompter and will be doing less of the low level technique. That doesn’t mean that it’s going to be easier. We humans have a way of going up the rungs, and so those jobs will evolve. To the extent that it’s difficult to adapt, there will be a human cost there. No doubt. But overall, those jobs will just move up the stack.

C/C: What is your approach to prompting tools like DALL-E? What are you trying to get out of it and what are some interesting things you’ve found?

GK: The funny thing is I don’t even do that much of it anymore myself. I would say the text-to-image stuff really started to take off in the early 2021 when CLIP was released. There was a relatively small number of people who started working with it. It was just a small scene and I was working with it a lot then, but at this point, I really want to create systems that facilitate prompting. As a prompter I just explore aesthetics. DALL-E is almost opposite to what I’m going for with Abraham. DALL-E is a very closed system. I’m incredibly blown away by the technology, it’s absolutely the best that we’ve seen. But I really want to go this other route, even if we have the worst or least realistic images.