What We’ve Said

What we’ve said about Diagnostic validity

13 points from 1 episode, newest first

Also searched as validity, validators

In short

A diagnosis is valid to the degree it maps onto a real disease or condition, and that shows in convergence among independent validators: symptoms, genetics or family aggregation, longitudinal course, and biological markers or treatment response. We would put validity ahead of reliability and rank diagnoses by how well validated they are. Shared symptoms or genes do not erase real distinctions, and systems built on symptoms use only one validator.

What it is

  1. Validity is how well a diagnosis tracks, or maps onto, an actual disease or condition that exists in reality, biological or psychological.

    “when I speak about validity, I mean, What I'm basically referring to is how well does a diagnosis track or map onto an actual disease or condition existing in reality, in biological or psychological reality”
    5:46Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode
  2. A diagnosis is valid when multiple validators, independent lines of evidence, converge: symptoms, genetics or family aggregation, the longitudinal course of illness, and biological markers, with treatment response as a stand-in for markers. The greater the convergence, the more valid, or real, the condition.

    “It basically said that you needed to see convergence among multiple validators, multiple independent lines of evidence, including symptoms, genetics, or familial aggregation, longitudinal course of illness”
    6:23Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode
    Also said in 1 more episode
    “Biological markers. And as a substitute for biological markers, you could use treatment response. What happens when you give a particular treatment. And the greater the convergence amongst the validators, the more valid or real a condition or disease is”
    6:38Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode
  3. Diagnostic categories are social constructions, built to capture real aspects of the real world. A patient may meet criteria for several diagnoses and really have only one disease.

    “The categories are social constructions. We've developed these to try to capture real aspects of the real world. And sometimes a patient may meet the criteria for multiple diagnoses, and they really only have one disease”
    51:43Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode

How we use it

  1. Restore the focus on validity and put it ahead of reliability. Reliability is important, but you get it when you are studying real things.

    “we need to go back and we need to restore a focus on validity and prioritize validity over reliability. Reliability is important, but you get reliability when you're studying real things”
    9:58Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode
  2. Rate each diagnosis on its empirical validity and build a hierarchy: the more robustly validated disorders come first in diagnosis, and you move down the hierarchy if the patient does not meet criteria for them.

    “we ought to rate each diagnosis on its empirical validity. How valid is And we develop a sort of hierarchy where the more robustly validated disorders are prioritized in diagnosis, and you sort of move down the hierarchy if the patient doesn't meet the criteria for the more Robustly Validated Disorders”
    16:59Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode

Why it matters

  1. We think it is underrated how much our lack of diagnostic reliability, and so of treatment reliability across the field, undermines us as a field. With a wrong or heterogeneous set of diagnoses, the right treatment does not come in a timely way and people do not get better.

    “I think it's underrated the degree to which our lack of diagnostic reliability and therefore lack of treatment reliability across the field undermines us as a field”
    27:32Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026
    Also said in 1 more episode
    “And if you always have the wrong or heterogeneous set of diagnoses, you're not going to deliver the right treatment in a timely fashion and people don't get better”
    27:59Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026
  2. It is maybe a hundred times more important now to have a diagnostic system that teaches while it serves as a scientific and clinical tool, because people use it to learn rather than learning to use it.

    “It becomes extra important, maybe 100 times more important now, to have a diagnostic system that, while existing as a scientific and clinical tool, also teaches at the same time because people use it to learn rather than to learn to use it”
    25:19Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026

Pitfalls and myths

  1. Systems such as HiTOP (the Hierarchical Taxonomy of Psychopathology) are built around symptoms, but symptoms are only one of the classic validators. In general medicine, many diseases share symptoms like cough, fever and pain.

    “there's so much emphasis on symptoms and entire systems now being developed around symptoms like HITOP, the hierarchical taxonomy of psychopathology. But that's one, it's only one of the classic validators. Think about in general medicine, how many diseases share symptoms like cough and fever and pain”
    13:01Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode
  2. Shared symptoms or shared genes do not mean there are no distinctions in psychopathology. Two disorders can share a third of their associated genes and some symptoms, yet diverge on the other two thirds and have completely different longitudinal courses.

    “they share a third of their, a third of associated genes. They diverge on two thirds of their associated genes. They share some symptoms, but their courses are different. They have completely different longitudinal courses”
    29:38Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode
    Also said in 1 more episode
    “this idea that, oh, a lot of things share symptoms, there's a lot of overlap or a lot of things share genes, there's a lot of overlap does not necessarily imply that there are no You know, distinctions in the world of psychopathology”
    30:00Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode
  3. There is an emphasis on transdiagnostic markers and symptoms, and a real skepticism that there are natural kinds in psychopathology, actual different diseases or conditions.

    “There's an emphasis on transdiagnostic markers and symptoms, a real skepticism towards the idea that there are natural kinds in psychopathology, that there actually are different diseases or conditions”
    28:31Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode

How we got here

  1. While DSM-III was being developed, there was a real division between psychiatrists who wanted to prioritize validity and those who wanted to prioritize reliability.

    “there was a real division during the crafting or the development of DSM-3 between psychiatrists who wanted to prioritize validity and Versus psychiatrists who wanted to prioritize reliability”
    4:41Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode
  2. The Feighner paper listed 14 psychiatric disorders, and the Research Diagnostic Criteria of 1978 identified 25. DSM-III, published in 1980, just two years later, listed 265 distinct diagnoses.

    “the Feiner paper listed 14 psychiatric disorders. The RDC, the Research Diagnostic Criteria published in 1978, a group led by Robert Spitzer, identified 25 psychiatric conditions. When DSM-3 is published in 1980, just two years after the RDC is published, that paper, it lists 265 distinct psychiatric diagnoses”
    7:16Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode
  3. After DSM-III, the idea was to put the new categories through rigorous validity testing. That really did not happen, and the little research that was done was more or less ignored.

    “after DSM-3 is published, the idea is to subject these new categories to testing, rigorous validity testing. That really doesn't happen. The little research that is done is sort of ignored”
    9:04Diagnostic Hierarchy Explained: Why Patients Get 5 DiagnosesNov 2026Guest episode

It also comes up on

From these episodes

Every point above is something said on the Psychofarm podcast, with the words as spoken and a link to the moment in the episode. It is education, not medical advice, and an episode reflects what we thought when we recorded it.

Educational content only. Everything here is what was said on the Psychofarm podcast, for clinicians. It is not medical advice and not a substitute for clinical judgment, current labeling or guidelines. An episode reflects what we thought when we recorded it, so check anything before you rely on it.