11 ms·

Kinect can now be used to diagnose depression with 90% accuracy?

99 points by SilentStump 13y ago

anigbrowl 13y ago
I'm tempted to send this to my shrink to see if he'll take the hint and cut his prices. I'm not optimistic, though.
- SilentStump 13y ago
  This article actually reminded me a lot of iCouch.me, which is like virtual therapy.
tunesmith 13y ago
This is interesting to me because... is there even a scientific definition of depression yet? Such that, if you have a defined list of symptoms, you are definitely depressed, and if you don't, you are definitely not depressed? If not, we're at best talking about a magnitudinal thing or a probability thing. Like a Bayesian thing where each symptom's presence updates the probability of the person being depressed, which is turn based off of a database of previous cases of diagnosed depression and what symptoms they had. But even that may be circular, ultimately. It could just as well be that depression is a vague category of similar symptoms that each come from a wide variety of scientific causes. This is not to be mistaken for saying it doesn't exist or that it's all in someone's head - it's more just to say that if someone has depression, they have something that may deserve treatment and sensitivity, but that it's something that we still do not know exactly how to describe or diagnose, and so therefore we will just call it "depression" in the meantime. But what if we've sort of collectively forgotten the "in the meantime" part?
- EvanKelly 13y ago
  There's DSM. http://en.wikipedia.org/wiki/Diagnostic_and_Statistical_Manual_of_Mental_Disorders http://en.wikipedia.org/wiki/Diagnostic_and_Statistical_Manu... Worth a look if you're interested in how mental disorders are diagnosed.
  - primitur 13y ago
    .. in the US, according to rules imposed upon the society by the high priests of Industrialized Mental Health. Not everyone wants a pill for their un-normal quirks, yo.
- SilentStump 13y ago
  I studied psych in college a few years back and read about a variety of studies on how individuals with slight-moderate depression respond to certain stimuli. One of the most interesting studies (unrelated to the topic) had to deal with individuals looking into a mirror after failing a task. Individuals with >= slight depression would look themselves in the eyes, while happily ignorant individuals would avoid eye contact with themselves. They attributed this with a depressed person's desire to figure out what is wrong with themselves and internalize the failure, where as the happy people would avoid internalizing the failure.
  - stcredzero 13y ago
    > They attributed this with a depressed person's desire to figure out what is wrong with themselves and internalize the failure, where as the happy people would avoid internalizing the failure. I've also read that depressed people tend to have a more accurate picture of reality. Basically, happy people are walking Dunning-Kruger cases waiting to happen?
    - SilentStump 13y ago
      I believe I know the study you are referring to and you are sort of right. on a scale of accuracy, slightly depressed people are pretty accurate in self assessments, happy people tend to over exaggerate their abilities, and moderately-severely depressed individuals rate their abilities lower than their actual level. So if the person's average ability was a 5/10, happy people tended to rate themselves a 6/10 and moderately depressed people a 4/10 on average iirc. Edit: My favorite example of Dunning-Kruger happens to be found in religion. The people that know God the most tend to talk about how little they know God, where as the proselytizers that claim the most to know God, really have no clue about the teachings of christ, mohammed, etc.
- ggchappell 13y ago
  > This is interesting to me because... is there even a scientific definition of depression yet? Such that, if you have a defined list of symptoms, you are definitely depressed, and if you don't, you are definitely not depressed? It depends a bit on what you mean. In the U.S. at least, diagnosis of depression is generally based on the American Psychiatric Association's Diagnostic and Statistical Manual of Mental Disorders (DSM).[1] That lists a number of disorders that would be categorized under "depression", including "Major Depressive Disorder", "Dysthymia", and the catchily named "Depressive Disorder Not Otherwise Specified". In the DSM, these disorders are all treated as syndromes, meaning they are defined entirely by symptoms. They may have organic causes (troubles with neurotransmitters, etc.), but the definition of each disorder does not address that issue. This is in contrast to something like influenza, which refers to a particular organic cause (infection by a certain kind of virus). As for the actual definitions, they vary in how specific they are. Major Depressive Disorder is quite specific, requiring certain kinds of episodes to occur with specified frequencies and lengths. On the other end of the scale is DD-NOS, which is essentially defined as a depression-ish thing that doesn't fit into any of the other categories. (Okay, it's a little more precise than that, but, honestly, not much.) > This is not to be mistaken for saying it doesn't exist or that it's all in someone's head - it's more just to say that if someone has depression, they have something that may deserve treatment and sensitivity, but that it's something that we still do not know exactly how to describe or diagnose, and so therefore we will just call it "depression" in the meantime. I'd say that's pretty much on target. However, biology & biochemistry are proceeding forward at a breakneck pace these days. We do have some understanding of the causes for some kinds of depression, and this understanding seems to be improving significantly each year. [1] https://en.wikipedia.org/wiki/Diagnostic_and_Statistical_Manual_of_Mental_Disorders https://en.wikipedia.org/wiki/Diagnostic_and_Statistical_Man...
  - stcredzero 13y ago
    What about correlations with patterns in MRI scans?
    - fnordfnordfnord 13y ago
      MRI's are expensive (in the US). Specialist mental health care is not covered under most standard health insurance plans. How about you just fill out the questionnaire[1], and we'll match you with a nice (newly patented) pill? [1] http://www.depressedtest.com http://www.depressedtest.com See also: Questionable sponsorship of online depression tests. http://www.policymed.com/2010/02/letters-from-grassley-questions-webmd-advertising-model.html http://www.policymed.com/2010/02/letters-from-grassley-quest...
caycep 13y ago
Actually, I'm looking to do something like this in related applications. Just need to find time to go through the kinect capabilities....my own unit has been gathering dust on my shelf for a while. Anyone in southern california (LA area or within driving distance) want to talk about this?
moocow01 13y ago
"SimSensei can work out whether you’re exhibiting signs that indicate depression" Im not sure I totally buy it - Im sure it can pick up the signs with 90% accuracy but how often do the signs actually correspond to a clinically depressed person. Just because your in a down mood at one moment does not mean you are depressed - depression is just a lot more complex than that. Im not sure that this would be of much use in accurate diagnosis although it seems like pretty interesting tech.
- TillE 13y ago
  Depends on the kind of questions they're asking, really. As someone with relatively mild depression, you'd never know it in normal conversation. But if you ask directly about symptoms and I give honest answers, both the answers themselves and my body language will probably give you a pretty good indication.
gruseom 13y ago
Do psychiatrists' diagnoses of depression even agree with that level of accuracy amongst themselves?
danso 13y ago
First of all, this is blog spam, and the OP has apparently never tried viewing their own site on a tablet. Second, where's the methodology. Third, it's not the Kinect doing the diagnosing, it's the computer vision algorithm
- SilentStump 13y ago
  I'll address your second point first. I agree that it would be very interesting to see the methodology. You're right that it's the algorithm that determines it, but it's the Kinect that's powering the CVA. Also, sorry I opted for a shorter, less accurate title. (ps. I'm not affiliate to the blog or product. I just thought it was interesting)
- rck 13y ago
  It looks like the research paper this is based on is here: http://schererstefan.net/assets/files/scherer_etal_FG2013.pdf http://schererstefan.net/assets/files/scherer_etal_FG2013.pd... I've only skimmed it, but the vision work looks sound, and it looks like it makes pretty essential use of the depth map from the Kinect. But I don't think there's any speech recognition going on, so that part is just acting (from both the humans and the virtual platform). I'll bet an untrained user could get the system to break pretty quickly... Neat proof of concept, though.
  - a_bonobo 13y ago
    Another thing is that this needs a properly trained interviewer, and it's not based on the Kinect's measurements alone but also on the questions asked (and the reaction is then measured). The behaviour of the interviewer is a great confounding factor in this. Reminds me of a certain someone asking another certain someone why he flipped a tortoise in the desert...
    - rck 13y ago
      I think the problem isn't with the published paper - it looks like they used a trained human interviewer and recorded both humans to evaluate their descriptors. The real problem is with the PR video, which looks really impressive, but seriously oversells the capabilities of the system. Not that the problem is unique to this situation - I think a lot of the videos in AI oversell the research. It's a pity, because the research is usually very good, but by itself isn't "exciting enough for general consumption," so they add bells and whistles that have nothing to do with the science.
prsutherland 13y ago
90% isn't all that great of a result considering only about 6.7% of the US population suffers from depression each year[1] and the study was done in California. Just returning false would give you better accuracy. Still Eliza on steroids is pretty cool. Can't wait till they integrate it into emacs. [1] http://www.nimh.nih.gov/statistics/1MDD_ADULT.shtml http://www.nimh.nih.gov/statistics/1MDD_ADULT.shtml
- GhotiFish 13y ago
  hahahaha! I guess it's how you interpret that 90% figure. If you interpret it as a measure of the tests sensitivity, then you're right 90% is pretty easy to beat! If you interpret it as a measure of the tests specificity, then 90% accuracy is pretty darn good! A little more accuracy would be nice, but then the title wouldn't be click-bait. I like the idea though. Accurate depression test: Are you depressed? No. 93.3% accurate!
- aray 13y ago
  They give their hypothesis in the paper, and each is shown with a p value < 0.05, so is statistically significant (and better than just returning false :) http://schererstefan.net/assets/files/scherer_etal_FG2013.pdf http://schererstefan.net/assets/files/scherer_etal_FG2013.pd...
  - prsutherland 13y ago
    p values are a measure of the accuracy of the numbers. They are measuring how close the sample mean is to the population mean. In the case of the paper, the p values represent how close the sample mean of the head gaze, eye gaze, smile intensity and smile duration measurements are to the population means of the same values. http://en.wikipedia.org/wiki/P-value http://en.wikipedia.org/wiki/P-value
opminion 13y ago
As if depression were a Boolean label.
mistercow 13y ago
As usual, vague terms like "90% accuracy" are thrown around without specifying what exactly they mean. Is that a 10% false positive rate? A 10% false negative rate? 90% accuracy over a representative sample of the general population? Because the last one is what would seem to be implied, but I have a much simpler program, written in pure JavaScript, that is also 90% accurate at diagnosing Americans with depression: https://gist.github.com/osuushi/5297823 https://gist.github.com/osuushi/5297823 Since, according to the CDC[1], about 10% of Americans are depressed, my script will be accurate 90% of the time. [1] http://www.cdc.gov/features/dsdepression/ http://www.cdc.gov/features/dsdepression/
- barbs 13y ago
  > I have a much simpler program, written in pure JavaScript, that is also 90% accurate at Diagnosing Americans with depression ...well played.
- aray 13y ago
  This is a good point. The parent article doesn't link to the paper but in it they give all of their hypothesis with p values <0.05, so the can reject the null hypothesis and give statistical confidence to theirs: http://schererstefan.net/assets/files/scherer_etal_FG2013.pdf http://schererstefan.net/assets/files/scherer_etal_FG2013.pd...
- jimktrains2 13y ago
  I think the term you're looking for is "Lies, Damned Lies, and Statistics"
eric970 13y ago
If it's as accurate as the team says it is, this is pretty astounding IMO. They'd have to be capturing some sort of baseline in the beginning of the conversation, and recording relative shifts of variables (ie; smile, eye shift/location) to come up with how "depressed" the person is. Just writing some software that says "oh this person is looking down" probably wouldn't cut it-- because shy people look down too. I noticed in the demo it seemed as if it started monitoring these things after certain questions were asked, which would be the correct way to measure a shift from baseline. Elicit a predictable period of what could be change and record.
ck2 13y ago
How does it determine depression vs boredom?
deleted 13y ago
[deleted]
brunorsini 13y ago
Doctors tried something similar with paper forms in the 60s. It was pretty disastrous, given depression is such a complex, "3d" kind of issue. Adam Curtis explored this theme in perhaps his best series of documentaries, "The Trap" - http://www.youtube.com/watch?v=WbRApO3k_Jo http://www.youtube.com/watch?v=WbRApO3k_Jo
- jimktrains2 13y ago
  Well, the Kinect should give 3D data, so that should help, right? :)
us_and_them 13y ago
Maybe its just most Xbox360 users are depressed about spending $400 just so they can play Halo.
goggles99 13y ago
Wow - So has those 20 question things that have been in magazines and tabloids for 40 years.....

99 points by SilentStump 13y ago

anigbrowl 13y ago

I'm tempted to send this to my shrink to see if he'll take the hint and cut his prices. I'm not optimistic, though.

SilentStump 13y ago
This article actually reminded me a lot of iCouch.me, which is like virtual therapy.

tunesmith 13y ago

This is interesting to me because... is there even a scientific definition of depression yet? Such that, if you have a defined list of symptoms, you are definitely depressed, and if you don't, you are definitely not depressed? If not, we're at best talking about a magnitudinal thing or a probability thing. Like a Bayesian thing where each symptom's presence updates the probability of the person being depressed, which is turn based off of a database of previous cases of diagnosed depression and what symptoms they had. But even that may be circular, ultimately. It could just as well be that depression is a vague category of similar symptoms that each come from a wide variety of scientific causes. This is not to be mistaken for saying it doesn't exist or that it's all in someone's head - it's more just to say that if someone has depression, they have something that may deserve treatment and sensitivity, but that it's something that we still do not know exactly how to describe or diagnose, and so therefore we will just call it "depression" in the meantime. But what if we've sort of collectively forgotten the "in the meantime" part?

EvanKelly 13y ago
There's DSM. http://en.wikipedia.org/wiki/Diagnostic_and_Statistical_Manual_of_Mental_Disorders http://en.wikipedia.org/wiki/Diagnostic_and_Statistical_Manu... Worth a look if you're interested in how mental disorders are diagnosed.
- primitur 13y ago
  .. in the US, according to rules imposed upon the society by the high priests of Industrialized Mental Health. Not everyone wants a pill for their un-normal quirks, yo.
SilentStump 13y ago
I studied psych in college a few years back and read about a variety of studies on how individuals with slight-moderate depression respond to certain stimuli. One of the most interesting studies (unrelated to the topic) had to deal with individuals looking into a mirror after failing a task. Individuals with >= slight depression would look themselves in the eyes, while happily ignorant individuals would avoid eye contact with themselves. They attributed this with a depressed person's desire to figure out what is wrong with themselves and internalize the failure, where as the happy people would avoid internalizing the failure.
- stcredzero 13y ago
  > They attributed this with a depressed person's desire to figure out what is wrong with themselves and internalize the failure, where as the happy people would avoid internalizing the failure. I've also read that depressed people tend to have a more accurate picture of reality. Basically, happy people are walking Dunning-Kruger cases waiting to happen?
  - SilentStump 13y ago
    I believe I know the study you are referring to and you are sort of right. on a scale of accuracy, slightly depressed people are pretty accurate in self assessments, happy people tend to over exaggerate their abilities, and moderately-severely depressed individuals rate their abilities lower than their actual level. So if the person's average ability was a 5/10, happy people tended to rate themselves a 6/10 and moderately depressed people a 4/10 on average iirc. Edit: My favorite example of Dunning-Kruger happens to be found in religion. The people that know God the most tend to talk about how little they know God, where as the proselytizers that claim the most to know God, really have no clue about the teachings of christ, mohammed, etc.
ggchappell 13y ago
> This is interesting to me because... is there even a scientific definition of depression yet? Such that, if you have a defined list of symptoms, you are definitely depressed, and if you don't, you are definitely not depressed? It depends a bit on what you mean. In the U.S. at least, diagnosis of depression is generally based on the American Psychiatric Association's Diagnostic and Statistical Manual of Mental Disorders (DSM).[1] That lists a number of disorders that would be categorized under "depression", including "Major Depressive Disorder", "Dysthymia", and the catchily named "Depressive Disorder Not Otherwise Specified". In the DSM, these disorders are all treated as syndromes, meaning they are defined entirely by symptoms. They may have organic causes (troubles with neurotransmitters, etc.), but the definition of each disorder does not address that issue. This is in contrast to something like influenza, which refers to a particular organic cause (infection by a certain kind of virus). As for the actual definitions, they vary in how specific they are. Major Depressive Disorder is quite specific, requiring certain kinds of episodes to occur with specified frequencies and lengths. On the other end of the scale is DD-NOS, which is essentially defined as a depression-ish thing that doesn't fit into any of the other categories. (Okay, it's a little more precise than that, but, honestly, not much.) > This is not to be mistaken for saying it doesn't exist or that it's all in someone's head - it's more just to say that if someone has depression, they have something that may deserve treatment and sensitivity, but that it's something that we still do not know exactly how to describe or diagnose, and so therefore we will just call it "depression" in the meantime. I'd say that's pretty much on target. However, biology & biochemistry are proceeding forward at a breakneck pace these days. We do have some understanding of the causes for some kinds of depression, and this understanding seems to be improving significantly each year. [1] https://en.wikipedia.org/wiki/Diagnostic_and_Statistical_Manual_of_Mental_Disorders https://en.wikipedia.org/wiki/Diagnostic_and_Statistical_Man...
- stcredzero 13y ago
  What about correlations with patterns in MRI scans?
  - fnordfnordfnord 13y ago
    MRI's are expensive (in the US). Specialist mental health care is not covered under most standard health insurance plans. How about you just fill out the questionnaire[1], and we'll match you with a nice (newly patented) pill? [1] http://www.depressedtest.com http://www.depressedtest.com See also: Questionable sponsorship of online depression tests. http://www.policymed.com/2010/02/letters-from-grassley-questions-webmd-advertising-model.html http://www.policymed.com/2010/02/letters-from-grassley-quest...

caycep 13y ago

Actually, I'm looking to do something like this in related applications. Just need to find time to go through the kinect capabilities....my own unit has been gathering dust on my shelf for a while. Anyone in southern california (LA area or within driving distance) want to talk about this?

moocow01 13y ago

"SimSensei can work out whether you’re exhibiting signs that indicate depression" Im not sure I totally buy it - Im sure it can pick up the signs with 90% accuracy but how often do the signs actually correspond to a clinically depressed person. Just because your in a down mood at one moment does not mean you are depressed - depression is just a lot more complex than that. Im not sure that this would be of much use in accurate diagnosis although it seems like pretty interesting tech.

TillE 13y ago
Depends on the kind of questions they're asking, really. As someone with relatively mild depression, you'd never know it in normal conversation. But if you ask directly about symptoms and I give honest answers, both the answers themselves and my body language will probably give you a pretty good indication.

gruseom 13y ago

Do psychiatrists' diagnoses of depression even agree with that level of accuracy amongst themselves?

danso 13y ago

First of all, this is blog spam, and the OP has apparently never tried viewing their own site on a tablet. Second, where's the methodology. Third, it's not the Kinect doing the diagnosing, it's the computer vision algorithm

SilentStump 13y ago
I'll address your second point first. I agree that it would be very interesting to see the methodology. You're right that it's the algorithm that determines it, but it's the Kinect that's powering the CVA. Also, sorry I opted for a shorter, less accurate title. (ps. I'm not affiliate to the blog or product. I just thought it was interesting)
rck 13y ago
It looks like the research paper this is based on is here: http://schererstefan.net/assets/files/scherer_etal_FG2013.pdf http://schererstefan.net/assets/files/scherer_etal_FG2013.pd... I've only skimmed it, but the vision work looks sound, and it looks like it makes pretty essential use of the depth map from the Kinect. But I don't think there's any speech recognition going on, so that part is just acting (from both the humans and the virtual platform). I'll bet an untrained user could get the system to break pretty quickly... Neat proof of concept, though.
- a_bonobo 13y ago
  Another thing is that this needs a properly trained interviewer, and it's not based on the Kinect's measurements alone but also on the questions asked (and the reaction is then measured). The behaviour of the interviewer is a great confounding factor in this. Reminds me of a certain someone asking another certain someone why he flipped a tortoise in the desert...
  - rck 13y ago
    I think the problem isn't with the published paper - it looks like they used a trained human interviewer and recorded both humans to evaluate their descriptors. The real problem is with the PR video, which looks really impressive, but seriously oversells the capabilities of the system. Not that the problem is unique to this situation - I think a lot of the videos in AI oversell the research. It's a pity, because the research is usually very good, but by itself isn't "exciting enough for general consumption," so they add bells and whistles that have nothing to do with the science.

prsutherland 13y ago

90% isn't all that great of a result considering only about 6.7% of the US population suffers from depression each year[1] and the study was done in California. Just returning false would give you better accuracy. Still Eliza on steroids is pretty cool. Can't wait till they integrate it into emacs. [1] http://www.nimh.nih.gov/statistics/1MDD_ADULT.shtml http://www.nimh.nih.gov/statistics/1MDD_ADULT.shtml

GhotiFish 13y ago
hahahaha! I guess it's how you interpret that 90% figure. If you interpret it as a measure of the tests sensitivity, then you're right 90% is pretty easy to beat! If you interpret it as a measure of the tests specificity, then 90% accuracy is pretty darn good! A little more accuracy would be nice, but then the title wouldn't be click-bait. I like the idea though. Accurate depression test: Are you depressed? No. 93.3% accurate!
aray 13y ago
They give their hypothesis in the paper, and each is shown with a p value < 0.05, so is statistically significant (and better than just returning false :) http://schererstefan.net/assets/files/scherer_etal_FG2013.pdf http://schererstefan.net/assets/files/scherer_etal_FG2013.pd...
- prsutherland 13y ago
  p values are a measure of the accuracy of the numbers. They are measuring how close the sample mean is to the population mean. In the case of the paper, the p values represent how close the sample mean of the head gaze, eye gaze, smile intensity and smile duration measurements are to the population means of the same values. http://en.wikipedia.org/wiki/P-value http://en.wikipedia.org/wiki/P-value

opminion 13y ago

As if depression were a Boolean label.

mistercow 13y ago

As usual, vague terms like "90% accuracy" are thrown around without specifying what exactly they mean. Is that a 10% false positive rate? A 10% false negative rate? 90% accuracy over a representative sample of the general population? Because the last one is what would seem to be implied, but I have a much simpler program, written in pure JavaScript, that is also 90% accurate at diagnosing Americans with depression: https://gist.github.com/osuushi/5297823 https://gist.github.com/osuushi/5297823 Since, according to the CDC[1], about 10% of Americans are depressed, my script will be accurate 90% of the time. [1] http://www.cdc.gov/features/dsdepression/ http://www.cdc.gov/features/dsdepression/

barbs 13y ago
> I have a much simpler program, written in pure JavaScript, that is also 90% accurate at Diagnosing Americans with depression ...well played.
aray 13y ago
This is a good point. The parent article doesn't link to the paper but in it they give all of their hypothesis with p values <0.05, so the can reject the null hypothesis and give statistical confidence to theirs: http://schererstefan.net/assets/files/scherer_etal_FG2013.pdf http://schererstefan.net/assets/files/scherer_etal_FG2013.pd...
jimktrains2 13y ago
I think the term you're looking for is "Lies, Damned Lies, and Statistics"

eric970 13y ago

If it's as accurate as the team says it is, this is pretty astounding IMO. They'd have to be capturing some sort of baseline in the beginning of the conversation, and recording relative shifts of variables (ie; smile, eye shift/location) to come up with how "depressed" the person is. Just writing some software that says "oh this person is looking down" probably wouldn't cut it-- because shy people look down too. I noticed in the demo it seemed as if it started monitoring these things after certain questions were asked, which would be the correct way to measure a shift from baseline. Elicit a predictable period of what could be change and record.

ck2 13y ago

How does it determine depression vs boredom?

deleted 13y ago

[deleted]

brunorsini 13y ago

Doctors tried something similar with paper forms in the 60s. It was pretty disastrous, given depression is such a complex, "3d" kind of issue. Adam Curtis explored this theme in perhaps his best series of documentaries, "The Trap" - http://www.youtube.com/watch?v=WbRApO3k_Jo http://www.youtube.com/watch?v=WbRApO3k_Jo

jimktrains2 13y ago
Well, the Kinect should give 3D data, so that should help, right? :)

us_and_them 13y ago

Maybe its just most Xbox360 users are depressed about spending $400 just so they can play Halo.

goggles99 13y ago

Wow - So has those 20 question things that have been in magazines and tabloids for 40 years.....