AI Mastering Survey
Added 2024-10-11 18:51:03 +0000 UTCI'm doing some double-blind tests with mastering (AI vs.AI-assisted vs. professional mastering engineers). 🙈🔊
Want to lend me your ear for a bit and participate in the survey?
Check it out! https://forms.gle/rqVrNoA1MoxLBphe6
I'll be letting it run for a day or 2 depending on how many responses it gets, which may delay the video a bit. (it was intended to drop Sunday, but good mastering engineers need a bit of time!)
Comments
I don't do shttt outside of dumping what I got from UDIO.. Is this good? https://youtu.be/SEPk2SY1YZ0?si=5_JS3nlUsNS8xJzA Same song, from my UDIO account https://www.udio.com/songs/nSjQqJ7hSztwXjG4QZuh3p HEY HAVE YOU EVER HEARD CHORDING PROGRESSIONS LIKE AT THE END OF THIS SONG IN ANY MODERN MUSIC ? I got this asking an AI to paraphrase Ashford/Simpsons SOLID.. Oh by the way , somewhere I have a song, where you can tell a sound is heading above about 15 feet, I kid you not, in stereo headphones.. Currently, today, I've been feeding the same 2 minute segment and remixing it with different variances, models, etc.. https://www.udio.com/songs/pR9C7KRdeXmVate7VSqw1c?utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing Actually for this one I borrowed the lyrics from ABBA, one of their songs in swedish, I converted it to english.. And just made it work somehow.. Seems people like it, I can't make heads or tails if these lyrics.. https://www.udio.com/songs/s3rwN2iyngtaHbtz7Kwoa3?utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing Oh, like you, I do stuff for free, all my tunes go into a creative commons public domain license. I personally hate asking for permissions to use stuff, it just gets in the way of stuff getting done..
Kiernan Holland
2024-12-18 23:04:21 +0000 UTCIs there any way to see the results on how I specifically did ?
Aron
2024-10-25 23:40:45 +0000 UTCGreat vid and results! What master was from what/who in the end?
kurt korthals
2024-10-23 19:16:26 +0000 UTCHey Benn, great video as usual. I'd really appreciate if you could release the different masters now "in the open" (so with the information which is which).
dengste
2024-10-23 15:33:21 +0000 UTCMastering? Whatever, everything was on different levels so hard to say anything. For me only one Mix/Master was ok and do not killed my ears. And the song is...not my taste. By the way, I still cant connect to the discord channel. Not sure if this is a fault on my side, but its not working for me. So I am out.
Ayak Krob
2024-10-15 19:13:45 +0000 UTCQUESTION for BENN: Can you please expound on what is - and isn't - included in the task of mastering these days? I haven't seen any mastering done since the 80s and I wonder what tasks are currently considered "in scope" for mastering. Is it just modulation and positioning within the sound stage? Does it include frequency analysis, application of filters? Don't need a class, but would love to know what today's mastering techs are expected to do.
David Haskins
2024-10-14 14:27:41 +0000 UTCI loved this so much. Thanks for including us in this! How did y'all listen? What was your methodology? I listened each each version all the way through, in the order that the survey presented them to me. As I listened I gave a + or - to each element that caught my ear in a good or bad way. When I was done, I added up all the + and then subtracted all the - for a preliminary score. 2 tracks stood out as extra bad in my listening. 3 tracks were ok, but 2 other tracks really stood out as much better. With this ordered list as a starting point, I then opened a separate player window for each track and played the same section (10-30 seconds) in each window, comparing my preference. If I liked one version over another, I gave that version another star. If I particularly disliked a particular version of a section, I took away a star. This helped me identify my preferences between tracks with close scores. The 2 at the bottom never even came close to rising above the other 5. The three in the middle swapped places a few times. The top 2 remained the top two, but the one I put 1st in the initial review went to second during my comparison round.
David Haskins
2024-10-14 14:18:26 +0000 UTCThat was a lot of fun. I don't love loudness wars, but i do like loud. I like fidelity, but I want to hear a pushed master also. I guess i want my cake and to eat it also. I found myself drawn to the masters with more low end saturation. I want the drums and the bass to hit, but i also want a more overall even sound. They all sounded great, they all didn't. Like there was stuff I liked and disliked about every version. HOW DOES ANYONE CHOOSE ANYTHING?! I can't imagine the nightmare that mixing or mastering this song was. The fact that they all sounded pretty good is a testament to the care you have to take when handling a composition this complex and interesting.
SacredHeartAttack
2024-10-13 19:47:20 +0000 UTCBefore I started listening I was thinking "what if I'm not gonna hear any difference" between the different masters. Oh my, was I proven wrong. The mastering process really does make a HUGE difference in how the end result sounds. I might have preferred a song that is slightly less complicated, although I understand why you picked such a song since it's of course really complicated to make a master on such a song as well. But very interesting study, looking forward to hear the results of the survey and to know who/what did which master. As always, Benn, your content is top tier and always interesting!
Björn Elwhagen
2024-10-13 14:15:39 +0000 UTCTwo comments: I wear hearing aids. I don't know that it affects my perception that much but it is a factor. Also, it seemed that the more I listened to the piece, the more I liked what I was hearing. I wonder if there wasn't some bias, at least on my part, resulting from the order I listened to the files in. Of course, I suspect the double blind part provided for that by offering the mixes up in a random order to each participant. I'm really excited to see the results from this study. Thanks.
Kellie Miller
2024-10-13 02:25:19 +0000 UTClooking forward to results 👀
M. Hatfield
2024-10-12 21:22:09 +0000 UTCThat was fascinating. I also pulled the files into a Logic session, and I think you can already tell what is what by looking at them. My listening setup goes way down to approx. 25Hz, and in some instances, I had to turn the volume down when the house started to shake and the cats began to act nervous :).
Oliver Christen-Drew (OC-D)
2024-10-12 20:05:25 +0000 UTCThat was fun! For me, one was clearly the best and some of them just... well.. rather weak. Great track! I tried to keep ears fresh and focused on just a couple of short passages for quick comparison.
Antti Tirilä
2024-10-12 19:14:54 +0000 UTCFor me there was one clear standout, a few with nothing too objectionable, and a couple which were either grating or just confusing. No experience evaluating this before so it’s cool to see what effect mastering has. I’m curious if the AI mastering tools are a “wav in wav out” thing, and if we can conclusively say that the AI is only changing things that a person hired to master a track would consider themselves “allowed” to change.
Taylor Thurlow
2024-10-12 16:36:58 +0000 UTCFun to partake in this! :)
koffiekop
2024-10-12 11:17:35 +0000 UTCWow, this is difficult. I think pretty much everyone here would agree that mastering is of course a creative choice, so I feel that this is more about "Do I agree with the creative choices that were made here" rather than "is this a good master or not". So in mastering, I think the most important thing is to TALK with the creator what they want, and this is where all these "automatic" mastering thingies fail. In this case, this track has the *potential* for a lot of dynamic range, IF the creator wants to have that. Knowing your music, I would *think* that you want this track to start quiet, but maybe not? Maybe you want this to be your breakout hit and everyone knows starting quiet is a surefire way to kill your streaming numbers? So while rating these masters, I would try to think how a Flashbulb mix would usually sound... (although you explicitly told us not to, but hey...)
dengste
2024-10-12 08:46:41 +0000 UTCOnly one did not master to maximze loudness and I preferred that one, simply because I heard less distortion of the synth lines. I think I could tell some of them that were AI enabled as they pushed some instruments into distortion, I think a human would have caught that (hopefully?). Many of the samples brought out the inner crunchiness of the synths more, making them sound dirtier. I bought the bandcamp version of the album a couple of months ago, so I was able to play it after I was done with the test. I can see now where some of the samples were going, and I was definitely in the dark! The synths still had that intrinsict distortion, but it sounded like part of the music, some of these samples pulled the distortion out like a sore thumb, making it unpleasant.
John Hughes
2024-10-12 07:36:33 +0000 UTCI appreciate being asked to do this because I'm someone who is both intimidated and kind of cynical about mastering. It's refreshing to realize that yes, I could hear very real differences and some of them sound legitimately terrible by comparison to the standouts. That bass part that kicks in around minute one is sick, btw.
leastbad
2024-10-12 07:18:54 +0000 UTCThis was fun to partake in. Thank you for doing this!
Brian Jared
2024-10-12 05:06:53 +0000 UTCLoved this. First tried to do it quickly just listening in the browser... then got into it and had to sync them all up in Ableton so that I could switch from track to track. Really looking forward to this video.
Gustaf Tadaa
2024-10-11 22:40:28 +0000 UTCI tried to get around listening fatigue by only judging in segments. Not commenting about what I heard as the survey is still ongoing, but there are some bad ones in there..
Aron
2024-10-11 21:50:49 +0000 UTCI mean, sure. Unless you do a lot of mixing and know your general threshold and can notice the signs of it (which I’d like to think is where I’m at with it) so I’m not sure how this is intended to relate to what I said. I was just sharing how I approached it in case it was helpful to others. ¯\_(ツ)_/¯
_truck
2024-10-11 21:07:21 +0000 UTCThe thing about listening fatigue is that you usually don't notice when it's happening to you. The only way to avoid it is to take breaks and to listen to something unrelated in a different environment.
Matt Chapman
2024-10-11 20:58:51 +0000 UTCI see why this would have been a tricky track to mix and to master. They all sound decent in the final section once it kicks in, but elsewhere the balances are all over the place. I do agree that one needs to account for listening fatigue with this many versions, especially since a few of them were fatiguing on their own in the midrange (at least on my monitors). My way of handling that was to listen in shorter segments at first to get a feel, and then do a full pass. Not sure if that really solves the problem, but it felt like it assisted.
_truck
2024-10-11 20:40:29 +0000 UTCI loved this. Just sat there, listening these tracks one by one, pausing them to analyze out loud to nobody about all the nuances i heard. 👌
huuru ukko
2024-10-11 20:34:42 +0000 UTCThey all sound the same to me lol, only thing I noticed was a bit different volume in some of them. Maybe I just don't have ear for this kind of listening. :D And I bet Benn has at least couple of them duplicates of others just to test US, quadruple-blind test! >:D
Pazi Gamer
2024-10-11 20:18:28 +0000 UTCExcellent song choice for this experiment! I hadn't heard it before, so I came in neutral. Didn't have an issue with listening to 7 versions. I find the overall selection of trade-offs very interesting :) And am curious how you briefed the mastering engineers.
rktic
2024-10-11 20:09:27 +0000 UTCFun experiment. Disagree with Matt on number and listening fatigue effects -- listened in my own random order, once through with notes@times. Mixed and blind comparison very valid in my view.
AArexx AAron Ruscetta
2024-10-11 20:07:34 +0000 UTCIt appears you didn't need to as the form rearranges every time anyway! Unless you mean you didn't listen to A-B-C in which case, ignore me haha
Alexander
2024-10-11 20:02:15 +0000 UTCFor what it's worth they're in a different order every time you open the form.
Alexander
2024-10-11 20:01:40 +0000 UTCI semi-cheated and normalized them all to the same integrated LKFS. It made it much easier to compare them.
Joachim Ekermann
2024-10-11 19:59:42 +0000 UTCI think we all know song _ is the best and song _ is clearly terrible
Leo
2024-10-11 19:46:14 +0000 UTCOh lordy lord, one of the 7 sounds so juicy good. Puts the other ones to shame really. 3 of them are terrible.
Aleksandar Janković
2024-10-11 19:46:01 +0000 UTCI chose to listen to the same 30 second segment midway through each song. Quick, no fatigue, and got very clear results for each song.
Leo
2024-10-11 19:45:23 +0000 UTCThanks Benn. That was fascinating. I didn't want the order you put them in to influence me so I randomized the order I listened to them in. What really surprised me is that my favorite by far was the last one I listened to. I of course won't tell you which one that was, but the difference was absolutely stark to my ears. I usually only listen this hard when it's baroque. That was fun!
Jeff Green
2024-10-11 19:33:15 +0000 UTCBTW, I do a version of this for my mastering clients-- I give them 2 different AI mastered versions and my "by hand" version, all with random labels for blind testing, and then they tell me which one they prefer. I've only lost to the AI once. There's no accounting for taste, I guess... 😉
Matt Chapman
2024-10-11 19:04:41 +0000 UTCThis is a flawed experiment. Listening fatigue is real, 7 versions is too many. A better experiment would be 3 or 4 songs with 2 versions each, one Human and one AI.
Matt Chapman
2024-10-11 18:59:43 +0000 UTC
