Why Most Mic Shootouts Are Fundamentally Flawed

I never listen to sound samples, and would never evaluate microphones that way. Not even for clients who beg me. 

 1. Sound samples recorded with MP3 or other lossy formats obliterate the subtle characteristics we cherish in a good microphone - characteristics we reward by paying disproportionate financial premiums for that last bit of quality. 

 2. The intended audience is never present when these 'shootouts' are made. For all we know, totally different mics, preamps, cables and distances were used- any of these criteria may favor one mic over the other, regardless of a mic's true merit as recording tool in a real-world working environment.

 3. Here is the most overlooked and biggest flaw of shootouts - a flaw so big, it will render any comparison between two or more mics pointless and inconclusive: lining up several mics at the same exact distance from the sound source. 

We see pictures of totally different microphones with totally different sensitivities and timbres being meticulously positioned so that their capsules were aligned within millimeters, sometimes with lasers (!) - all in service to the illusion of scientific objectivity. 

Who would place an SM58 and a U47 at the same distance from the singer? Proximity effect, even among condenser microphones, is so pronounced that a 2" adjustment in distance to the sound source may double or half a mic's low end transmission. 

From my work I know that even two mics of the same type do not necessarily have the same sweetspot. 

There is science and then there is misapplication of the scientific method...

------------------------------------

Two years after starting this thread people continue to distribute "shootouts"on the internet without documenting methodologies or identifying vital sound-shaping microphone components. Results are then published, shared, consumed and commented on, again, with no inquiry into testing methodology or (sub)component details.

To sum up and expand on the fallacies of this approach:

*  Ignorance and incuriosity persists about the unscientific, untenable methodology applied to these comparison tests. Methods are never disclosed or detailed by the creators and never questioned by the consumers - an unscientific, wobbly foundation to forming a fact-based opinion or to making an informed purchase decision. 

* As prices for vintage originals (which are then often "shot out" against current-production copies of the same model) have skyrocketed in recent years, 'shootouts' also have skyrocketed, which can give a potential buyer with limited financial means the illusion of being able to achieve sonic excellence on the cheap.

* Test conditions are so far beyond any realistic setup and working experience encountered in professional recording scenarios that a microphone costing less than 1/10 of another one it is compared to sounds somehow indistinguishable from it. 

* Lively discussions ensue about a 60+ year-old Neumann M49 having a "slightly preferable" midrange, compared to a brand-new M49V reissue, without verifying capsule version and condition, tube health, power supply voltages, and other vital checks. This is akin to comparing performance of two otherwise similar cars - one veteran, one brand-new, one worth millions, the other a mere fraction - without as much as opening the hood.

As I pointed out in my previous post, disregarding the relevance and individual variation which proximity effect has on pressure gradient microphones does gross disservice when evaluating a mic's optimal distance from a sound source. 

Here is another example of the "sin of equal distance" (let alone, the ultimate sin of using mics with different capsule topology at the same, unrealistic distance from the sound source): https://www.youtube.com/watch?v=BTUcnWDMPJs

What is also striking to me: true professionals owning or working in successful commercial studios rarely participate in these amateur contests. People publishing these tests are usually thoae who have some kind of financial interest in the test results, or buyers who have succumbed to confirmation bias: I own the cheaper copy, and, see? it's just as good!

What about the argument: "let me do a preliminary check of a mic online, before I audition in person"?

The same fundamental flaw persists. If the online presentation is unreliable, how can it serve as a first round of elimination? A mic that may sound terrible in a video may sound great in reality, and vice versa.

So why then even bother bringing this up again? 

In the absence of rigorous discussion, and by continuing to ignore pleas to use a more scientific methodology when examining digital copies of sonic impressions of high-end microphones, the current trend of trusting online mic comparisons will eventually take hold and become the new working standard. It may bring headaches, heartaches and remorse to those who took these tests seriously and made purchasing decisions based on them.