HotpotBio is the data lab and research group of Hotpot.ai dedicated to biomedicine.
Our lab philosophy is to advance AI models without Internet-scale data. While supermodels have dominated since GPT-3, the smart model paradigm is finally starting to shift from fringe to credible. Labs like ours have long believed the future of AI will mirror the computing industry, where supercomputers tackle the most complex cases, but smartphones are what get used by billions of people.
Unable to publish commercial research, we established HotpotBio to advance science in other ways. We draw inspiration from open source, where ephemeral teams innovate by attracting talent across organizational boundaries. Since biomedicine is characterized by sparse data and evolving facts, the field presents a high-impact opportunity for validating hypotheses aligned with our lab vision. Furthermore, ML research resembles biomedical research more than most realize.
Just as findings in one patient may not generalize across patients due to genetic and lifestyle differences, ML findings -- even on core parameters like learning rate -- may not generalize due to architecture and training differences. In both, generalizability is far weaker than in physics, though reproducibility affords ML one key advantage.
Second, both cancer and ML research are constrained by data quality. Central to our effort is rethinking biomedical datasets and training approaches in clinical reasoning, oncology, neuroimmunology, drug development, and other specialty areas.
While some data errors are tolerable for general ML models, uncommon variants in biomedicine may drive pathology. Training on imprecise medical information may cause misdiagnosis, clinical errors, misfolded proteins, or pharmaceutical drugs with increased major adverse events (MAEs).
Complicating matters, evolving medical facts may invalidate training data and model knowledge. What was true last year may be false today. For instance, in April 2024 the U.S. Preventive Services Task Force reversed its longstanding advice and now urges biennial mammograms starting at age 40 -- down from the previous benchmark of 50 -- for average-risk women, citing rising breast-cancer incidence in younger patients.
Accurate annotation of medical data is challenging and demands expert verification based on the latest information. Even Google DeepMind's relabeled effort of MedQA from 2024 contained errors, which could produce subtle failures and model hallucinations if not addressed.
Cancer is the second leading cause of death worldwide, claiming the lives of roughly 10 million people per year and devastating the lives of millions more [1].
With 8 billion people and only 12.7 million doctors, personalized healthcare is impossible today. Human doctors alone cannot bridge this gap and provide the attentive care everyone deserves.
Quality datasets and benchmarks can unlock rapid progress in machine learning (ML), but most technologists lack medical expertise while most doctors lack technical expertise.
Without accurate and comprehensive datasets and evaluations, it is hard to train models and improve AI -- not unlike teaching students with poor textbooks and exams.
Our objective is to package medical knowledge into a format suitable for ML engineers and researchers to further AI biomedicine, regardless of medical background.
Concretely, this work involves investigating language model reasoning, developing datasets, and creating frameworks in collaboration with medical professionals from Stanford and other leading institutions.
Our research investigates the association between Epstein-Barr virus (EBV) and cancer, concentrating on the topics below.
Viruses cause cervical cancer, Burkitt lymphoma, nasopharyngeal cancer (NPC), and several other cancer types, but the data is inconclusive for more common cancer types like breast cancer and lung cancer [3-8].
We welcome contributors in the areas below.
Work can fit any schedule and take one of many forms:
Novel datasets for TNBC and non-smoking lung cancer could power tens to hundreds of studies and hopefully set a new precedent for tackling tumor subtypes. See here and here for details.
Clarence Hu
