Pearls before Breakfast, the article I was talking about yesterday, got me thinking about the why and how. I’m convinced it’s bad experimental design (let alone any considerations about the playing style), and to have taken the single instance (a.k.a. “stunt”) as the representative result is a clear example of confirmation bias: they went in with a hypothesis, and in single stroke of brilliance, proved it! (asking the symphony orchestra director was merely a play to setting up the fall, I say – the article starts cynical and stays that way).
There’s two main flaws:
- He’s standing outside the station exit and inside the doors; it’s a commute area, not one which a group can easily gather
- It’s morning and turning up late at work is rarely an option
A more representative experiment would have included:
- Playing in the evening rush home (less time pressure)
- Playing at lunchtime (open wallets)
- Playing somewhere other than at the station (location influence neutered)
- Playing at a different station (comparative sample)
- Playing on the platform (captive audience of those waiting)
- A female expert playing (different to “standard” by visual)
There’s a hundred other variables to the experiment, naturally, but these are some obvious ones that wouldn’t be difficult or time consuming to test; they’re things which any ordinary busker would also need to take into account. Finally, the comparison needs to be done: if an “ordinary” violin player had been at the same spot at the same time of day, would the result have been the same, or less? or even more, as people appreciated the attempt? One factor to consider is that if the performance is exceptional, it may blend, in a curious way, with something the people have already heard before.
If you’ll indulge, what if the skill meant that people filed it away, unconsciously identifying it as pre-recorded and ignoring it much as they would a stereo system? :)
Crud, posted my comment in the wrong place…
http://pushingthesky.net/2007/04/09/suggested-to-kottke/#comment-10242
point taken :)
It shouldn’t matter, but I’m looking at it from the angle of experimental design – it would be interesting to see if that made a difference. I can say that I would have been more interested by a female; if not in the attraction sense, at the very least because 95% of buskers i’ve seen are male.
consider if you will: I don’t know who Josh is. Most people who aren’t classical music affectionados, i.e. a huge section of the population, wouldn’t know who he is. Specific skill does not translate to broad public recognition.
If it’s anything like Australia, no denying. But another question you’d have to ask is whether it was all that high in the first place; earlier access to such fine art would have been equally limited to the “upper” classes. The difference may have been that once it was more about lack of access as opposed to appreciation, but such is the mass market.
I’m not crticising the Post’s efforts – it’s certainly interesting. The experiment however was a bit more of a stunt than anything, with a lack of strong testing of possible cases.
All excellent points, karan. Nice blog and post! Glad I found you, bookmarked.
Ah, c’mon, the article is itself a work of art.
Also! Let’s not forget the etymology of “experiment.”
etymology of experiment?
but yeah, that’s why i’m saying what they did is not a rigorous experiment, it’s just at the level of “stunt”, and should probably be refered to as such, much like the guy himself says…
Experiment has the same root as experience, and until the enlightenment the terms were used interchangeably. There’s still a sense of experiment meaning “doing something to see what happens”, eg. “I’m experimenting with drugs.” To insist on scientific method, in this case, is to miss the point, I think.
However, if we must be scientific…
You have two main objections: inconvenient time, and inconvenient place. However, the experiment design explicitly asks, “In a banal setting at an inconvenient time, would beauty transcend?” Later, they go on to consider possible public safety issues. You are clearly begging the question. As for rigour, the sample size is 1097, which is certainly sufficient to draw some conclusions, and these can be cautiously extended from the population morning commuters to the population of people who are in a rush. If this were a survey, you would not be criticising the lack of rigour, but because they observed and not asked, making it an experiment, all of a sudden it’s a stunt?
It could have been made more rigourous still, granted, by playing at other places where people are in a rush, but this is the only of your alternative situations that apply to the question at hand. The rest are irrelevant. And this is not a strong criticism, because it can be reasonably argued that people rushing in one place are much the same as people rushing in another.
Hmm, some good points. However… :)
The sample size is sufficiently large, but without a control experiment to contrast, it proves little. I suppose this is my key argument: that’s a corner stone of the scientific method, isn’t it? Even if they just asked a “normal” busker with a violin how much they’d earn in the same time, it would be demonstrative.
I suggest the other scenarios because those are things that I observe are factors influencing me personally when ‘interacting’, in a sense, with buskers. I’ve never heard a busker in the morning, and even if I did I’d be more concerned with getting to work. However, at lunchtime and in the evening, I’m much more likely to prick my ears up, and maybe even drop some money in.
Yes, it would be violating the premise they go in with inconvenient-time-inconvenient-place, but if you’re a busker, you wouldn’t go in for such a time – therefore, i suggest it’s not an experiment, in the scientific sense, because it doesn’t operate in the right environment. It’s like… analysing the growth of bacteria in situation x at 10 degrees celsius, when the situation in the real world is far more likely to occur at room temperature (a vague term open to interpretation). bad analogy, yes, but you get the point right?
No! Randomised control trials are but one form of experiment, one that specifically allows you to investigate causality. This experiment was not designed to do this. It was designed to see how a particular group would respond to a particular stimuli. It clearly achieves this aim, and they do not attempt to draw causal conclusions from their data. They do discuss possible reasons, but they do not claim to have investigated these reasons. It’s surely scientifically valid to speculate in the discussion of an experiment. Eg: “Leithauser thinks he has some idea of what happened at that Metro station.” If they said, “Our test proves that television caused the result we saw,” then that would not be valid. What does the test demonstrate? That L’Enfant Metro users in the morning, by and large, ignore beauty. I would feel comfortable extending this to the broader population of busy people, although you might not. If you do not, can you please point out some theoretical reasons why busy people in other places may have paid more attention? In any case, their experiment is rigourous enough to back their article.
It is not an experiment about busking. The word does not even appear in the article. So I get your point, but it doesn’t apply. They’re not trying to work out if busking works or not: he is masquerading as a busker, but that is part of the method, not part of the aim. It may well be interesting to investigate what conditions effect the profitability of busking, but that would be a completely different experiment. The comparison in this case is not to what happens to a normal busker, but what we think will happen, and what we think should happen.
ahhh ok, i see your point…
(i’m not good at conceding, so you can take this victory, but I’ll get you next time, Gadget!)
*slaps Daniel*
NO! This is the internet! You are not allowed to use the phrase “begging the question” as it was originally intended. *slap*
Sorry Jack, I forgot where I was.
Now that that’s out of the way, the really important question: would you have stopped? I don’t think I would have.
In the morning, pretty unlikely… but then I don’t have much of an ear for classical music anyway, and if it’s outside the paltry number of peices that i recognise, I’m unlikely to appreciate it.
Kinda sad, innit?
I don’t think so; I’d stop for a rap-battle or something like that =) Jazz even. I’m sure back-in-the-day someone complained about kids these days only listening to that new fangled Bach guy rather than the hymns and folk songs of yesteryear. Music does have a timeless quality, but what’s popular today does dominate the landscape. (ok so it’s pretty unlikely someone will be dropping Eminem’s lines 300 years from now, but the mode of rap is more evolutionary)