"We never override the computer"
If you want models in drug discovery to work well, you need to test all of them, not just some of them.
People often say that the major problem with molecular modeling in drug discovery is that the models don't work well. But I think the bigger problem of which that problem is a corollary, from both a scientific and a culture standpoint, is that the models are not battle-tested the way they should be. I was reminded of the failure - and the value - of doing this when I read a recent memoir by Jim Simons. Simons has been called the most successful investor of all time. His Renaissance Technologies hedge fund has returned an average of 40% returns (after fees) over the last 20 years. The firm uses proprietary mathematical algorithms and models to exploit statistical asymmetries and fluctuations in stock prices to leverage price differentials and make money. Before starting Renaissance, Simons made important contributions to the development of both pure mathematics and string theory.
Simons's background enabled him to recruit top mathematicians, computer scientists, physicists and statisticians to Renaissance. In fact the company actively stays away from recruiting anyone with a financial or Wall Street background. The biography of Simons is titled "The Man Who Solved the Market", by Gregory Zuckerman, and it's worth reading for both the financial and the human stories. But there's an interesting video of a Simons talk at San Francisco State University from 2014 in which he says something very intriguing about the models that Renaissance builds, and that was what reminded me of models in drug discovery.
"The only rule is that we never override the computer. No one ever comes in any day and says the computer wants to do this and that’s crazy and we shouldn’t do it. You don’t do it because you can’t simulate that, you can’t study the past and wonder whether the boss was gonna come in and change his mind about something. So you just stick with it, and it’s worked."
It struck me that this is how molecular modeling should be done as well. As I mentioned in a previous post, a major problem with modeling is that it's mostly applied in a slapdash manner to drug discovery problems, with heavy human intervention that obscures the true successes and failures of the models. Computational chemists have all been in situations where either experimental chemists don't make all the molecules they have recommended, or where they make molecules that haven't been recommended. There's also no good way to keep track of all these modeled and unmodeled compounds. In both cases there's some method to the madness. In the former case, molecules may not be made because they are hard to make. But they may also not be made because of prejudice against modeling. In the latter case, the same reasons may apply: chemists make molecules using their own logic or because they don't trust the models enough.
But as Simons's quote indicates, while there are all kinds of reasons to make or not make molecules, the only way to truly improve the models would be to simply take their results at face value, without any human intervention, and test all the molecules that the models are recommending. This isn't important just for battle-testing the models but it's also critical for reproducibility and record-keeping. If you sometimes listen to the model and sometimes don't, it's not a record that can be accurately kept and tested. So the only way we'll know what works and what doesn't is if we trust the models and let them rip through.
Being able to test the models every single time needs a few pre-existing conditions, though. Jim Simons and Renaissance could do it because
They have the wisdom to realize that that's the only way in which they can get the models to work.
They have pockets that are deep enough so that even model failures can be tolerated.
If modeling is to be made effective in drug discovery organizations, these organizations need to mirror what Renaissance is doing, namely:
Have a "computation-first" culture, where every molecule that is predicted by a model is made and tested and its results fed back into the model, and modeling is on an equal footing as the other disciplines in the organization.
This paradigm is of course easier proclaimed than practiced. Most organizations don't have the resources to test every molecule the model recommends and can afford to waste money on only so many dead compounds. But many do, and they still don't do it because of an explicit or implicit mistrust of modeling. I always feel sad when I hear about organizations where there is so much mistrust of modeling that the discipline seems to be set up for failure. It then becomes a self-fulfilling prophecy: if you as a chemist constantly mistrust the models or expect them to be perfect, you will never give them a chance to get better and help you, and of course they are going to fail.
Fortunately I also know companies which very much have embodied this "computation-first" approach, and that approach often infuses the organization all the way from hiring up to the actual drug design process. Not only are the chemists and biologists in these organizations hyper-tuned to testing the predictions of modeling, but many of them do the modeling themselves. In these companies, the line between simulation and experiment is thin. Now this doesn't mean that the chemists never make compounds that aren't recommended - we all know the great value of pure intuition and experience in drug design - but it does mean that they do a good job of bookkeeping, of making sure which compounds do and not do come out of the model.
If modeling is to succeed in drug discovery, it's naturally important that it becomes more accurate, reliable and reproducible. But all that is actually a corollary of being able to test the models as comprehensively and completely as possible. And this testing is enabled both by resources and and by culture, both which are can be actively managed by organizations should they choose to do so.