Most people
Most people are expected to experience at least one diagnostic error in their lifetime.
Research
Researching diagnostic error — not just building around it
Sympify runs a research program alongside the product. This page includes the parts that have not worked, the evaluations that are not finished, and the claims we are not entitled to make.
How we decide what we are allowed to say
Sympify's research repository keeps two explicit lists: claims supported by a generated artifact, and claims that must never appear in Sympify-facing material regardless of how the results look. The prohibited list includes any statement of diagnostic accuracy for patients, any claim that Sympify "reduces misdiagnosis" or is "clinically validated", and any description of its internal score as a probability or confidence.
That rule governs this website too. If a claim is not backed by an artifact, it is not here.
The problem
Diagnosis goes wrong more often than most people realize
Three figures from the research literature. Each one links to its source — because a statistic without a citation is just a claim.
~12 million
About 12 million U.S. adults are estimated to experience an outpatient diagnostic error each year.
Approximately 5% of U.S. adults annually in the study's estimate.
~795,000
An estimated 795,000 U.S. patients die or experience permanent disability each year from diagnostic error.
A modeled national estimate, not a directly observed national count.
Publications
Papers
Each card states its actual status. A working draft is labelled as one.
Published2026
A Comprehensive, Equitable Diagnostic Error Correction System using Reverse-Dictionary Algorithms, Patient-AI Collaboration, and Sustainable AI Design
BDML 2026 · Computer Science & Information Technology (CS & IT) Conference Proceedings, vol. 16, no. 12
Describes the reverse-dictionary approach behind Sympify: starting from a condition a patient has already been given, surfacing the symptoms commonly reported for it, and using the resulting symptom profile to retrieve other conditions associated with the same symptoms.
Updated September 2026
Working draft2026
Sympify-KG: A Transparent Differential-Exploration Retrieval System over a Symptom Concept Graph
IJDKP (International Journal of Data Mining & Knowledge Management Process)
A journal extension evaluating the Sympify symptom graph and matcher: graph structure and sparsity, lexical and graph retrieval baselines, an inferred-ancestor weighting ablation, robustness to alias substitution and misspelling, and report-continuity safety checks.
This work is not yet peer reviewed and should not be cited as a finished result.
Not yet available online
Updated August 2026
Posters & presentations
Presented work
Sympify — Moving Targets 2024
Moving Targets 2024: Computational Data-Driven Drug Discovery & Development, USC Alfred E. Mann School of Pharmacy and Pharmaceutical Sciences — August 22, 2024
Research poster presenting the Sympify approach to diagnostic-error exploration, at the annual daylong symposium organized by the USC chapter of the American Association of Pharmaceutical Scientists (AAPS).
Jalen Cai
Updated September 2026
Sympify — BDML presentation
BDML 2026 — 2026
Conference presentation accompanying the BDML paper.
Jalen Cai
Updated August 2026
Current research
Work in progress
Separated from published work on purpose. Each project states what is established and what is not — the second list is the more important one.
Analysis underway
Evaluating the symptom graph and matcher
Does organizing symptoms into a concept graph improve how Sympify retrieves and ranks related conditions, compared with simpler text matching?
What we can say
- The symptom concept graph is currently small and shallow: 138 concepts, 61 directed relationships, a maximum depth of 2, and no cycles.
- Coverage is sparse — roughly 29% of condition–symptom links and about 2.8% of distinct symptom nodes are graph concepts.
- On an internal, catalog-derived development benchmark, simple lexical baselines score higher than the graph methods, because that benchmark rewards reproducing a condition name exactly.
- Among the graph methods, the full assembled matcher outranks the raw relationship graph.
- Changing the weight given to inferred broader symptoms had a negligible effect on ranking on this benchmark, which is attributable to graph sparsity.
What we cannot say yet
- How the system performs on external, held-out data. The planned DDXPlus evaluation has not been run because the dataset is not yet provisioned.
- How often the system is wrong in ways that would matter to a patient. The two-reviewer failure analysis has no collected labels yet.
- Anything about diagnostic accuracy, sensitivity or specificity for patients. Sympify has not been clinically validated and makes no such claim.
Updated August 2026
In progress
Robustness and report-consistency checks
Does the system behave predictably when a symptom is described differently, misspelled, or reached by a different path through the app?
What we can say
- The concept graph improves resilience to alias substitution (the same symptom described in different words) relative to a lexical baseline.
- Neither approach currently resists transposition-style misspellings.
- For an identical symptom profile, the full-report ranking does not change based on the order in which a user explored conditions — zero violations across 130 probed profiles.
What we cannot say yet
- How these results transfer to real user input rather than generated perturbations.
Updated August 2026
Technical approach
The system being studied
A symptom concept graph, not a keyword list
Symptoms are stored as canonical concepts with explicit alias wording, connected by directed relationships from a more specific symptom to a broader one. The structure is a directed acyclic graph rather than a strict tree, because a symptom can legitimately belong under more than one broader concept — an itchy rash sits under both itching and rash.
Inference only runs one direction
Selecting a specific symptom can infer the broader categories it belongs to. Selecting a broad symptom never infers a more specific one. Inferred categories carry less weight than something you selected yourself, and reports show them in their own section, labelled as inferred and not selected by the user.
Transparent retrieval, not a black box
Conditions are ranked by how much of your reported symptom profile corroborates them. Where a condition is linked to both a specific symptom and its broader parent, only the strongest match counts, so a single answer cannot be double-counted. The weights involved are engineering parameters chosen for sensible behaviour — they are not clinically calibrated constants.
Evaluation, and what it does not yet cover
Internal evaluation covers graph structure and coverage, retrieval baselines, weighting ablations, robustness to reworded and misspelled symptoms, and checks that an identical symptom profile produces an identical report regardless of the path taken to reach it. External held-out evaluation and a two-reviewer failure analysis are planned but not yet complete, so no accuracy claim is made.
Known limitations
- The symptom concept graph is small and sparse. Most condition–symptom links are not yet graph concepts, which limits how much the graph can contribute.
- On the current internal benchmark, simple text matching outperforms the graph methods. That benchmark rewards reproducing a condition name exactly, so it flatters lexical approaches — but we report the result rather than hiding it.
- The system does not resist transposition-style misspellings.
- No external held-out evaluation has been completed, and no clinician failure review has been collected. Until both exist, no accuracy claim is possible.
- The historical report data is a set of submissions, not a representative population, and is not treated as one.
Perspectives
Individual views from people who have seen Sympify. They describe personal impressions, not measured performance — Sympify has not been clinically validated and does not claim to improve diagnostic accuracy.
…supports doctors by providing precise, data-backed insights.
Dr. Amy ShaoResident Physician, Walter Reed National Military Medical Center
Do research with us.
Sympify takes on students who want to work on the symptom graph, retrieval evaluation, robustness testing, or the diagnostic-error literature — with mentorship from working researchers.