As others have said, density is only one question and it’s not as simple as one way is good, the other way is bad. It’s like the question of yields, which is also often oversimplified to the point of being useless.
There are studies and theories, sometimes conflicting, and as always, they tend to be for other farm crops that had bigger companies interested - it’s relatively recently that wine grapes have become so important economically.
But seems like economics has a more to do with it than probably anything else.
In any event, grapevines can be fairly vigorous, which is one reason they do so well in poor soils that other plants would have difficulty with - their natural vigor lets them do OK. In some of the old vineyards in Europe, people found over the years that when vines are grown on soils that have little available water and that are rather low in nutrients, dense planting helps restrain the natural vigor of the vines and forces them to produce many little fine root hairs to take up whatever nourishment they can. It seems almost counter-intuitive, but the idea is that the vines go into survival mode and produce fruit rather than vegetal growth. It’s not that the vines send roots way down deep so that they take up more “minerals”.
But just as importantly, people knew that they were going to lose vines to pests, viruses, etc., and they also wanted to get the maximum economic return on their given plots, and the vines weren’t going to get very big on the poor soil anyway, so planting a lot of vines was smart.
In richer soils, particularly if you’re going to have widely spaced rows for tractors, the vines can be and are a lot more vigorous. Then your problem is different - you’ll have a lot of vegetative growth. So what to do with that? You can prune, but if you prune all of the growth away, you may get dormant buds to start growing. So you have to direct the growth some way and your canopy management and training systems become really important. In Medoc, the lesser producers traditionally had about half the density of the classified growths because they had better soil. In CA, there was also wide spacing because especially on the valley floors, soils weren’t so bad.
But again there’s economics. Andy Beckstoffer pointed out that everyone knew they could plant vines much more closely than they did, but in the end they still needed to sell the grapes. Today, with land costs in places like Napa thru the roof, it makes a lot more sense to plant much more densely. And in fact, in Oregon, California, and Washington, denser planting is the trend – sometimes as much as five times more densely.
Of course back to the site itself – if you have very similar soil and exposure, but less rainfall than another site, you might want to plant less densely to conserve some of the stored water. Ultimately, it comes down to the individual vine more than anything – is the vine producing optimally-ripened grapes, given whatever you’ve done to it? Up to some point, which varies based on all the other factors, you can increase both vine density and vineyard yields w/out sacrificing any quality. And you can sometimes increase vine density while retaining yield but with luck, you get better grapes so earn more money for the work.
All in all, before you can say anything about density, you have to take into account the natural condition of the soil - drainage, composition, nutrient potential, and then the slope of the site - flat, north facing, south facing, east, etc., and then the orientation of your rows depending on what kind of sunlight you want to capture/avoid, and then the seasonal climate - likelihood of frost early/late, and then the rootstocks you’re planning to put in - a very important consideration that can attenuate some of the other conditions, and then the scion clones you select, and then the training system you plan for the canopies.
Then in some places, training and densities are regulated, so you have to comply with those rules whether or not they make sense today or ever did. It’s nicely romantic to imagine that grandpa knew everything there was to know about grape growing, but a lot of things are just traditional or have become established rules only because of habit or incomplete understandings of what was really going on, and of course, because of political deals. I’d suspect the last is probably more important than any of the other factors. Grandpa may or may not have known what he was doing, but if he was a sharp political operator, he’d manage to swing things his way.
In the end, it’s often an economic decision as much as anything else. It’s more costly to plant a given vineyard densely because you’re buying more plants. Then you have more to take care of as well. But if that helps limit their growth and per-vine yields and you get better grapes, although your initial investment was higher, you earn more in the end.
As to whether current conditions are optimal - that’s a really interesting question and again depends on the specific vineyard. Could densities in many vineyards be changed? Could trellising be changed? Could rootstocks and scions be changed?
Probably in many, perhaps even most cases, the answer would be yes. You don’t know if you made the right decision until you have a few harvests in and at that point, who wants to start again? You’d have to have the money and the obsession of a Bill Gates to keep trying until you got it just right, and even he doesn’t have the lifetime to do it.