Dear Software Makers

OkayPhysicist

I strongly believe A/B testing users without their knowledge and enthusiastic consent is unethical. If you don't have enough confidence in your changes to make them carte blanche, then don't make them at all. Users don't want their shit changing all the time.

smarf

> The skill of being a “creator” is not about maximizing views, retention, etc. It’s about finding creative ways to share stories, teach concepts, explore ideas, etc. Views, retention, etc. are all downstream of that. this seems wildly naive

smugengineer69

I've seen this argument a few times and honestly I don't get it - how do people propose we otherwise measure relative success of particular interventions -- Vibes? I've found that people who complain about how everything needs to be measured and how measurements are imperfect are often just bad at devising measurable quantities as proxies of real-world benefit. Sure it can take a bit of time to devise such measurements, and they're not inherently valuable, but if I can prove that a given variant performs better, why not pursue the preferable alternative? It seems easy enough to say "keep the youtube we love" but how do you think this variant came about in the first place? I can assure you there have been numerous A/B tests that have led to the current feature set. And even if its a local rather than global maximum, at least there are measurable qualities by which it is preferable. Also - do you believe that everyone who says this is harkening back to the same historical reality? This is quickly approaching "make youtube great again" territory - when exactly was it great again? and why? This statement is easy to agree with and hard to prove. It also seems easy to handle links to different variants; just supply a URL parameter. This is a non-issue. If someone out there has a non-tea leaf divination style alternative to measuring things as a way to determine success, I'm all ears, but I don't believe in fortune tellers, and people who make software shouldn't either.

akst

In most cases the A and the B in an A/B test really should otherwise mostly identical, but one thing is different. If almost everything is different, it’s hard to learn for next time what exactly what led to a change in which ever dependent variable your observating.

fn-mote

Welcome to a world where commenting on a YouTube video is the same as writing a product review on Amazon. Now you need to include enough details that future readers will be able to tell that you are talking about a different product/video.

ChrisMarshallNY

I'm not a fan of the new approach, which is, basically "We don't need to be creative, up front. We'll just shove some crap out quickly, and work on the friction points." But it's not that simple. We definitely need to "pave the bare spots" (desire paths); It's just that we need to start off, at what we sincerely believe to be an optimal place, knowing that it isn't, in fact, optimal.

beloch

"There’s this sort of unspoken rule of community on YouTube, that like we’re all having this same experience together — we’re all watching the same video. That is what makes it a cultural phenomenon is that we all saw the same thing […] so letting people put up two or three different videos of the same thing at the same time kind of fundamentally breaks that, which doesn't feel right." ---------------- It's going to be awkward if you share a youtube link with somebody and what they see is significantly different from what you saw, perhaps even to the point of them replying, "Why on Earth did you send this to me? Are you on crack?" More importantly, this will almost inevitably lead to content creators being given the ability to not just randomly A/B test versions of a video, but produce different versions of the same video that are shown to users based on their data. e.g. Shania Twain used to produce different versions of her albums with different instrumentation based on which section of the music store they'd be sold in. There was a Country version for the Country section and a Rock version for the Rock section. She's still bootin' around today, so she could produce different versions of music videos targeting users based on whether Google thinks they like Rock or Country more. This would be relatively harmless, although one might be surprised by the version that appears on a friend's phone. Musical taste isn't what really divides people these days. What might content creators do if they could show different videos to people based on their political views? This might be good for their business, but it undermines objective reality. People would be shown different "facts" based on their beliefs. This is precisely the opposite of what needs to happen to reduce political polarization and bring people closer together. A common reality is necessary for society to function.

jaggederest

Could go even further and produce multiple sections and let the "algorithm" edit them together in whatever order. intro (2x 15 second clips pulled from random places in all the other sections) section 1 (a/b/c) section 2 (a/b/c/omit) section 3 (short/long) section 4 (a/b/c) It would be like a choose your own adventure video, without the choosing, or the adventure.

jordwest

As Seth Godin said: > Enough A/B testing will turn any website into a porn site

csnover

From [0]: > The funny thing about scoring systems is they are kind of little dictators. They tell you what you’re supposed to want and value. And that’s the weird thing. Scoring systems are little definitions of success and failure. I think one of the biggest differences is that, in games, those definitions are temporary and playful and under your control. And if you don’t like it, you can throw it away and you never have to play again. And in institutions, they’re authoritarian. […] After a period of time, [metrics] seem to drain what’s genuinely valuable from the system because they point people at something that’s very easily and mechanically checkable and measurable. [0] https://99percentinvisible.org/episode/673-the-score/transcr...

wewewedxfgdf

OT: YouTube has some big problems. I don't think I am the audience for YouTube any more. YouTube is converging towards where all the social media sites are: * shorts * photo/text posts * longer videos too - typically 12 minutes approx * AI videos - some of which are fine but I want to be able to filter them out * their algorithm/feed is very bad at letting me explore my interests, when I choose to - instead it feeds me stuff that leaves me unsatisified But I came here years ago to watch TV made by people NOT bound to 12 minutes. I watch pretty much nothing else at night on the couch except YouTube but I am coming to realise it's no longer what I want. That "original YouTube" seems to be gone. Nothing has replaced it.

skyberrys

What a strange choice from YouTube. They won't even let you create a draft of a video ( private not published anywhere ) and then replace the video before publishing. This seems much more confusing, it's like two videos yet they don't have different urls. Maybe they will control for a level of similarity?

neals

Anybody else seeing ghosty black lines on hn after reading that dark page? Like a negative version of some sort, lingering in your vision?

jamesforestwest

I’m not particularly biased against technology, but even to me, this sounds very dubious

j45

The comments here about A/B testing are one thing, the real wave that will be massive is unprecedented personalization compared to what exists today.

tonymet

recently i was suffering in a "no reviews" experiment on Amazon, and it took a few moments to overcome "you're crazy it works for me" . Despite all of us building AB tests, few have embraced the ramifications that we are all experiencing a different combination of experiences.

Barrin92

Completely horrific especially in the creative context. If you make something, you are a creator, and what you make should be an artifact of you. The idea to A/B test a video, as if you don't know what the best video is you can make, to me completely undermines the point of making anything at all. At that point you may as well outsource your 'content' to a marketing department. You evidently only care about clicks, not expressing anything. And even in the software world where there's a place to test say accessibility and whatnot this attitude is so prevalent that everything looks the same. Nobody's making a website like Larry Wall any more[1]. Can people please start making things out of their own volition again instead of following this brain dead attention economy [1] https://www.wall.org/~larry/