What are your opinions about how AI will alter VSTs expression capabilities?

[ZeroZero]

I have been wondering how AI will change VSTs. IMO all teh current VSTs are very much like an aeroplane console, if I can put it that way, You have switches and dials, which is not at all like the experience of feeling a real instrument. I can't seem to get carnel with them, if you know what I mean. I cant express my emotion if I have to type in 36 rising to 89, and then alter a bezier curve. ETc Etc...

Will AI make expressiveness easier?

How will this happen? Will you type in give me a V7 1 and make it sound like Bach, then add seagulls? Seems to me if it is this its not personal expression.

Z

Click to read more...

 

[Axl]

this is a very interesting topic.
I hope that orchestral libraries will have let’s say a violin. You play the part you have written and it makes a performance of it. Then you have controls where you can tailor it more to your needs.
With synths you have the synplant. Very cool concept. Hopefully others will jump on it and expand on the idea.

 

[chapbot]

I'll tell you exactly how it will happen. Most sample developers are going to go out of business.

Because: I will create a MIDI line, pop it into an AI music generator and tell it to create a soaring solo violin in a baroque style. Or a pop style, or an avant-garde style.

It will spit out a wav file that will be so mind-bogglingly realistic your jaw will drop. You will pop it into your DAW, rinse and repeat.

I can hardly wait, we are in amazing times.

 

[lettucehat]

I'll tell you exactly how it will happen. Most sample developers are going to go out of business.

Because: I will create a MIDI line, pop it into an AI music generator and tell it to create a soaring solo violin in a baroque style. Or a pop style, or an avant-garde style.

It will spit out a wav file that will be so mind-bogglingly realistic your jaw will drop. You will pop it into your DAW, rinse and repeat.

I can hardly wait, we are in amazing times.

I think a major issue is whether individual instruments are going to be able to generate a high-fidelity part if the most successful models so far are trained on finished, full ensemble recordings. I know they can already generate pretty good stems, but it's going to have to stack up well with dozens of other parts without becoming a dusty cloud of artifacts if we're ever going to use AI in this way. That is, replacements for VIs on the instrument level that take detailed input from users. I'm sure it can be done but it might have to train on all new data sets, perhaps created specifically for the purpose..? Which in theory sample library developers could do, but anyone could. They might have an advantage in knowing how to go about it though.

 

[vitocorleone123]

FYI there’s a sub forum for AI threads now on VI-C.

 

[PancakesBear]

There is probably an AI that you could use that can start more AI threads for you

 

[GtrString]

I don’t know how much AI will make a difference here. If you consider Generative AI it will automate the process, so you end up losing (too much) control, and with Transformative AI, it won’t make much of a difference than what we have. So in this scenario, I think AI is too much of a consumer grade tech to really improve anything. As a producer, I still want to be able to control the material behaviour.

What I think will change are the expectations of perfection in production. When everybody can get good results this fast, just making things sound realistic won’t be the bar, there will be more time to excel and really push things over the top. We will be competing in increasingly nitpick’y details.. it could add up to increasingly working with classical sensibilities rather than nonchalant rock’n roll negliance..

 

[AudioLoco]

I'll tell you exactly how it will happen. Most sample developers are going to go out of business.

Because: I will create a MIDI line, pop it into an AI music generator and tell it to create a soaring solo violin in a baroque style. Or a pop style, or an avant-garde style.

It will spit out a wav file that will be so mind-bogglingly realistic your jaw will drop. You will pop it into your DAW, rinse and repeat.

I can hardly wait, we are in amazing times.

Yes that might be the future, but there are a few issues:
As it stands, you would be violating the copyright of thousands of other fellow musicians like you.
Because as we see happening with voices, they actually use real sources, real singers, real players.

At the moment you can have Peter Gabriel or anyone else to sing your song, I don't think Peter Gabriel and the likes would be happy to hear their own voices used against their will on probably crappy music they don't approve, and without remuneration.
Why should great violinists lend their tone and years of hard work without remuneration and without mention and without approval.

We are not entitled to pillage these recordings they are using for training.

This is with the current situation and even if they come up with the coveted "MIDI to AI" tech there are still legal and rights issues to be solved.
If a legal way brings to this then yes it is a possible future for tools for composers IMHO, and the good 'ol sample libraries already seem a bit of a relic after what we heard these days.

Maybe there will be licensed AI libraries where a single performer or ensemble give permission, under initial payment or/and a royalty agreement to use their talents to train the software.
Until then it would be probably illegal and most of all unethical.

 

[lychee]

AI will annoy a good number of developers but will make the work of those who are not opposed to it easier.
Me as a composer, I don't want the AI to play for me, but I want my flute to sound like that of a flutist and if the AI can help with that, I don't see any point in it objection, if it is to have realistic articulation or not to have to bother with 10 short note keyswitches.

In the idea I tested the VST DDSP Magenta, where by training an audio you can recreate an resynthesized instrument playable in midi or via an audio input.
The result is full of artifacts but it remains interesting, so we will see if it will be improved.

View attachment DDSP Test.mp3

 

[Casiquire]

Yes that might be the future, but there are a few issues:
As it stands, you would be violating the copyright of thousands of other fellow musicians like you.
Because as we see happening with voices, they actually use real sources, real singers, real players.

At the moment you can have Peter Gabriel or anyone else to sing your song, I don't think Peter Gabriel and the likes would be happy to hear their own voices used against their will on probably crappy music they don't approve, and without remuneration.
Why should great violinists lend their tone and years of hard work without remuneration and without mention and without approval.

We are not entitled to pillage these recordings they are using for training.

This is with the current situation and even if they come up with the coveted "MIDI to AI" tech there are still legal and rights issues to be solved.
If a legal way brings to this then yes it is a possible future for tools for composers IMHO, and the good 'ol sample libraries already seem a bit of a relic after what we heard these days.

Maybe there will be licensed AI libraries where a single performer or ensemble give permission, under initial payment or/and a royalty agreement to use their talents to train the software.
Until then it would be probably illegal and most of all unethical.

Unethical, maybe, but the legality angle is tough. That's why AI art is still thriving despite the exact same concerns and a couple years of media scrutiny, which isn't even there for music. And ethics alone has never, ever been a big roadblock in the pursuit of money lol

I think by the time "AI" (as we call it) gets to a place where it's profitable to generate convincing, emotional music to that level of control and quality, we'll probably already see multiple other bigger shake-ups to the sample library market. We're a NICHE market.

I'm hoping that by the time "AI" trickles to us, it's more like next-gen sample modeling, or it's such a benefit to developers in terms of code or editing that they can push the envelope across the board. Imagine if slicing and editing legato were as simple as the algorithm trimming and crossfading everything for you at the best phase-aligned point. Solo instruments become way easier. Or something else we can't even think of yet. Fingers crossed for that version of the future rather than "Alexa, play Cardi B songs in the style of Mozart" and it doesn't really nail either thing, or rise above the level of consumer novelty

 

[AudioLoco]

Unethical, maybe, but the legality angle is tough. That's why AI art is still thriving despite the exact same concerns and a couple years of media scrutiny, which isn't even there for music. And ethics alone has never, ever been a big roadblock in the pursuit of money lol

I think by the time "AI" (as we call it) gets to a place where it's profitable to generate convincing, emotional music to that level of control and quality, we'll probably already see multiple other bigger shake-ups to the sample library market. We're a NICHE market.

I'm hoping that by the time "AI" trickles to us, it's more like next-gen sample modeling, or it's such a benefit to developers in terms of code or editing that they can push the envelope across the board. Imagine if slicing and editing legato were as simple as the algorithm trimming and crossfading everything for you at the best phase-aligned point. Solo instruments become way easier. Or something else we can't even think of yet. Fingers crossed for that version of the future rather than "Alexa, play Cardi B songs in the style of Mozart" and it doesn't really nail either thing, or rise above the level of consumer novelty

Yes, hence my idea for an "AI SuperEvil Evil Dragon" coding expert embedded in Kontakt in order to create more easily and rapidly instruments.

I agree. Ethics won't touch any investor's heart but a musician's own dignity and pride in one's work should ideally show us a better path and using illegally trained AI should be a no no and a stigma around ourselves at least.

 

[Onemandown]

I have been wondering how AI will change VSTs. IMO all teh current VSTs are very much like an aeroplane console, if I can put it that way, You have switches and dials, which is not at all like the experience of feeling a real instrument. I can't seem to get carnel with them, if you know what I mean. I cant express my emotion if I have to type in 36 rising to 89, and then alter a bezier curve. ETc Etc...

Will AI make expressiveness easier?

How will this happen? Will you type in give me a V7 1 and make it sound like Bach, then add seagulls? Seems to me if it is this its not personal expression.

Z

Might wanna check the AI section for more...in any event AI can't add emotion any more than you can. Because the theory behind AI is that it's analyzed violin lines categorized as emotional and found that, for the sake of our example, the peak of the phrase must rise from 36 CC1 to 89 CC1. However, it can also adjust timbre, brightness, vibrato, and other attributes of the instrument, since it has created the entire sound. So really, we could accomplish everything AI could if we had the right library and the patience to use it correctly.

 

[jbuhler]

I want my flute to sound like that of a flutist and if the AI can help with that, I don't see any point in it object

Not even from flautists?

I mean I share the desire to have VIs that are less terrible. But I worry whenever VIs can replace real players. Because if they can they inevitably will. We know they already do. And if we are fine with VIs replacing real players then where do we have standing when they replace composers?

 

[tmpc]

I'll tell you exactly how it will happen. Most sample developers are going to go out of business.

Because: I will create a MIDI line, pop it into an AI music generator and tell it to create a soaring solo violin in a baroque style. Or a pop style, or an avant-garde style.

It will spit out a wav file that will be so mind-bogglingly realistic your jaw will drop. You will pop it into your DAW, rinse and repeat.

I can hardly wait, we are in amazing times.

It won't even need anyone to write anything. Won't that be great . . . wait a minute.

 

[Onemandown]

Not even from flautists?

I mean I share the desire to have VIs that are less terrible. But I worry whenever VIs can replace real players. Because if they can they inevitably will. We know they already do. And if we are fine with VIs replacing real players then where do we have standing when they replace composers?

Thank you! I'm assuming that you aren't all for using AI composition assistants then?..

As cool as AI is, I personally want the work I'm being paid to deliver to come from me. Maybe there could be an argument that using a MIDI keyboard or an external monitor is the same thing as using AI, but it feels different to me. Almost like cheating.

Remember when AI stood for Audio Imperia and X was a letter of the alphabet?

 

[jbuhler]

As cool as AI is, I personally want the work I'm being paid to deliver to come from me. Maybe there could be an argument that using a MIDI keyboard or an external monitor is the same thing as using AI, but it feels different to me. Almost like cheating.

Any time you are replacing labor with the machine there is an issue of ethics. But you can hardly complain about composing being replaced by AI if you are willing to replace other creative labor with AI. You’d rather not hire the flautist for your music? Fine. Maybe the director of the film would rather not hire a composer. In any case the fantasies that surround all of this, the desire to consolidate creative control to the self only to dispatch it to the machine, is also concerning.

I’m not saying I have any idea where the line should be drawn or whether you can even find a line. I’ve said before that I find the line very fuzzy. What’s the difference between using a commercial loop made by humans and a commercial loop made by AI for instance? Also what’s the difference between using old style deterministic or pseudo random generation and AI generation? There’s some space to draw lines of distinction between what I’ll call the various modes of automation (of which AI is one) but the situation is still fuzzy for me. And it also has to be measured against the total social good. Some industry and creative change is a response to better serving of larger social wants and needs.

 

[Onemandown]

Any time you are replacing labor with the machine there is an issue of ethics. But you can hardly complain about composing being replaced by AI if you are willing to replace other creative labor with AI. You’d rather not hire the flautist for your music? Fine. Maybe the director of the film would rather not hire a composer. In any case the fantasies that surround all of this, the desire to consolidate creative control to the self only to dispatch it to the machine, is also concerning.

I’m not saying I have any idea where the line should be drawn or whether you can even find a line. I’ve said before that I find the line very fuzzy. What’s the difference between using a commercial loop made by humans and a commercial loop made by AI for instance? Also what’s the difference between using old style deterministic or pseudo random generation and AI generation? There’s some space to draw lines of distinction between what I’ll call the various modes of automation (of which AI is one) but the situation is still fuzzy for me. And it also has to be measured against the total social good. Some industry and creative change is a response to better serving of larger social wants and needs.

Well, for a similar reason I don't use loops. Ever. Or ostinato engines. Or arps. This makes me sound like some sort of hardcore hippie but as someone who writes from the soul I try to keep my music as original as possible. Yes, I will sometimes use synth presets. Yes, I will use riser and strike samples. But there's some part of me that wants the music to be completely mine. I don't hire flautists because I like my hearing. (Kidding. Kinda.) But in all seriousness I agree with what you're saying. Lower-budget directors aren't going to pay for a composer when an AI can do a passably good job. To be fair, and this sounds strange, I exclusively use AI for my album art. I'll still touch it up, but the AI is making my album art. Now I'm not doing this because I think the AI is better: I'm doing it because my Spotify music doesn't make enough money to justify a graphic designer. So yes, it's hard. But my album art isn't part of my music.

Time to go watch The Creator.

 

[jbuhler]

Well, for a similar reason I don't use loops. Ever. Or ostinato engines. Or arps

You don’t. But many do. There’s a whole segment of the industry devoted to production in the area. And many who work with loops and presets produce very good music and they produce music that is not easily made otherwise. (Yes many use loops, presets and such to produce schlock, but that happens with composers who don’t use loops as well.)

I exclusively use AI for my album art. I'll still touch it up, but the AI is making my album art

I get where you are coming from and beyond that use I can see using AI art to prototype. (For that matter this is where I see VI orchestras generally fitting in the scheme, as generating detailed prototypes of the final music.) But again so long as you are willing to use AI to replace creative labor you have little standing when it comes to objecting when others choose AI to replace your creative labor.

I’ve also said that there’s a real market for AI music such as filling the needs of YouTubers who simply need music that can pass the content ID audit without eating too much into the little income (or no income) they are generating with their video. That use seems very analogous to your use of AI images for Spotify album art.

 

[Onemandown]

You don’t. But many do. There’s a whole segment of the industry devoted to production in the area. And many who work with loops and presets produce very good music and they produce music that is not easily made otherwise. (Yes many use loops, presets and such to produce schlock, but that happens with composers who don’t use loops as well.)



I get where you are coming from and beyond that use I can see using AI art to prototype. (For that matter this is where I see VI orchestras generally fitting in the scheme, as generating detailed prototypes of the final music.) But again so long as you are willing to use AI to replace creative labor you have little standing when it comes to objecting when others choose AI to replace your creative labor.

I’ve also said that there’s a real market for AI music such as filling the needs of YouTubers who simply need music that can pass the content ID audit without eating too much into the little income (or no income) they are generating with their video. That use seems very analogous to your use of AI images for Spotify album art.

Again, I'm not objecting. I'm predicting. In no way do I think it would be unreasonable to a director to choose AI over me if, to his ears, there is a somewhat comparable product with a vast difference in price. And I also think that there is a huge market for AI-assisted tracks when it comes to stock music. If I have to choose between stock art and AI art I'm going to take AI art. And I point out the part about loops and whatnot because I know that there are many very talented producers who use loops and samples like it's going out of style. That's why there's a market for that. My post has far less to do with what I think is right for the director than it does with what I think is going to happen to the composer. My point is that I, personally, with no judgement or recommendation for others, do not at this time wish to use AI to assist me with my music production. I'm not saying "if you use AI then you're not a creative." Because the argument could be made "if you don't build your synth sounds from the ground up on hardware synths then you're not a creative." I'm not even saying that a director using AI-composed music is making the wrong choice. If the ultimate goal of a composer is to serve the director so the director can serve the viewers, then if the director thinks AI music is the best way to do that then it's not my place to object.

This is a very long-winded way of saying that like it or not, lower-budget work might start disappearing here soon. And that's ok. It isn't morally wrong for OT to overprice their products. (cough cough...) It isn't morally wrong for a composer to use loops. It isn't morally wrong for a director to use Udio. It isn't morally wrong for me to subject you to a small novel of me ranting (debatable). But to me, I prefer not to use AI. You might prefer to only use SATB choirs.

I'm not judging anyone who uses AI or loops or samples. But I'm a purist what can I say.



Also sorry OT I really love y'all but my wallet hurts just talking about this.

 

[Henrik B. Jensen]

If something can be done, it will. It’s gonna be hard or impossible to prevent.