For any disease, usually we need to establish whether a treatment / intervention is effective (e.g. in saving lives, or improving quality of life) in randomized controlled trials where we control for all other factors and then randomly assign patients to one treatment or a control and then see if the intervention is effective and safe. And that the benefit exceeds harm.
In this case, of course this was impossible back in March and measure were generally being instituted at country levels where randomization is not possible. And as it was a new disease, we had no established treatment and so had to rely on so-called non-pharmaceutical interventions (NPIs). We didn’t even know which interventions were most likely to be effective – only really the experience of Wuhan – where a very strict lockdown did seem too have brought the disease under control – as would be expected given the virus needs human interaction to spread.
We therefore have to rely on lower quality observational evidence which are limited by bias and confounding. For example, we can look at the incidence of disease before and after the intervention – but then we don’t know for certain if the reduction in disease was due to the intervention or another factor. (e.g. did cases fall because of lockdown or because of the change in weather, or the virus mutating, etc.) The other problem is that nearly all interventions were instituted simultaneously and so we don’t know which of them had the biggest effect in reducing infection – hence the reliance on modelling.
Looking at the evidence from different countries and looking at the effect of lockdowns both on the coronavirus and the economy (so called ecological evidence) is also problematic because countries differ in so many important ways which will also affect the incidence of the disease (their population characteristics, health system, economies, etc.) and so are very prone to confounding and bias (it depends which countries you include in the analysis). Therefore, it is very difficult to prove that the interventions caused the effect.
This explains why there is so much conflicting evidence and differences of opinion among scientists and going forward priority should be given to doing randomised controlled trials for all interventions where this is possible.