This is interesting. One of the themes, or leitmotifs, of post-transition photography seems to be the use of photographs as data. I'm not really capable of evaluating the rigor of the methodology of "How an Algorithm Learned to Identify Depressed Individuals by Studying Their Instagram Photos," but the source, MIT Technology Review, is prestigious.
Bottom line? Researchers have trained a machine to diagnose depression based on the photos you post on Instagram.
An interesting idea. I've heard of discussions about how slow, sad music seems richer and more meaningful to depressed people, and that depressed teenagers tend to listen to music more than their non-depressed peers. In the old movie Three Days of the Condor, Robert Redford's character diagnoses the lonliness of Faye Dunaway's character, ostensibly a photographer, from her pictures on her wall. (The article says that pictures without people in them might be "sad selfies" of the photographer's mental state, but concedes that this "hypothesis is untested.")
I also don't know how the machine could separate authentic sad photos from poseur sad photos. But maybe there's a way.
The article concludes by saying that such algorithms "provide hope that mental illness can be accurately detected earlier, allowing for more effective intervention," which seems dubious. How's that supposed to work—you get an email from someone saying that their robot has detected that you're depressed, based on pictures you've posted online?
Seems to me if you were depressed, you might just find that...depressing.
But as I say, I'm not really qualified to evaluate.
Mike
(Thanks to Ned Bunnell, whose photos on Instagram make him seem like a happily retired businessman)
Original contents copyright 2016 by Michael C. Johnston and/or the bylined author. All Rights Reserved. Links in this post may be to our affiliates; sales through affiliate links may benefit this site.
Like what you read?
Join our support campaign or buy something
(To see all the comments, click on the "Comments" link below.)
Featured Comments from:
[A different] Mike: "As someone who has suffered with depression since childhood I can attest that it is a very real phenomena. It's a constant battle to keep moving forward through life, and when things happen that make people who are not depressed sad, it can devastate a depressed person.
"Depression is a complex disease that comes in many forms, and there are no simple answers as each person is unique. Some people can overcome depression, while others can only cope with it as best they can.
"People who don't have depression cannot possibly have a clue as to what it's all about. The glib comments from ignorant people can be offensive, but it's so commonplace, I just do my best to let it go.
"Yes, my depression comes through in my personal work. It's there that I find a voice for how I feel about what I see. I'm highly attuned to noticing things that most people don't see. Body language and facial expressions say more than one can imagine. And when I see someone that is expressing something that I can relate to, I make the photograph. Depression is the filter through which I see the world. I can't help that; it is what it is.
"Making photographs that express how I feel is very much a form of therapy for me. When I'm out shooting for myself I'm in the zone and everything else fades away. It's the most peaceful place I can be when I'm making photographs, there's no past or future, only the now.
"I keep my depression at bay in my professional work, and even find many expressions of exuberance and happiness to photograph for my employer and other clients. It does bring me a bit of happiness to find and photograph those moments, and it reminds me that there is much joy in the world.
"And, it's not that I'm morose all the time or lack a sense of humor. I tell the best/worst damn Dad jokes ever. I can interact with regular people and not seem any different from them. But I see how they are in the world, and I know that I'm different. In the end, I'm comfortable with who I am."
Tim (partial comment): "I seem to make some of my best photos when the mood is non-average, either a bit depressed or a bit sunny."
Possibly more helpful for depression are the reports that some/much of it is due to inflammation: http://www.bbc.co.uk/news/health-37166293
Posted by: Richard Parkin | Wednesday, 24 August 2016 at 04:04 PM
An interesting article regarding depression and the immune system, was on the BBC news today:
http://www.bbc.co.uk/news/health-37166293
Posted by: Dave Stewart | Wednesday, 24 August 2016 at 04:19 PM
More likely you get a pre-packaged reply from an automated email server telling you a robot decided, from looking at an algorithm, that you are depressed. And it's from a do not reply, unattended mailbox.
No that's depressing.
Gordon
Posted by: Gordon Cahill | Wednesday, 24 August 2016 at 04:43 PM
First Instagram, then every photo site. This is a trifle chilling to me.
Posted by: Tony Rowlett | Wednesday, 24 August 2016 at 05:04 PM
Well that just makes me sad.
Posted by: steven willard | Wednesday, 24 August 2016 at 05:28 PM
This sort of thing depresses me.
Posted by: David Brown | Wednesday, 24 August 2016 at 05:39 PM
From the report: "Depressed participants were less likely than healthy participants to use any filters at all." (p. 9)
Better start using those filters, folks. And no, B&W doesn't count: "When depressed participants did employ filters, they most disproportionately favored the “Inkwell” filter, which converts color photographs to black-and-white images."
Posted by: Jack | Wednesday, 24 August 2016 at 05:58 PM
From the abstract ...
Photos posted by depressed individuals were more likely to be bluer, grayer, and darker.
[ ... ]
Several different types of information were extracted from the collected Instagram data. We used total posts per user, per day, as a measure of user activity. We gauged community reaction by counting the number of comments and “likes” each posted photograph received. Face detection software was used to determine whether or not a photograph contained a human face, as well as count the total number of faces in each photo, as a proxy measure for participants’ social activity levels. Pixellevel averages were computed for Hue, Saturation, and Value ...
https://arxiv.org/ftp/arxiv/papers/1608/1608.03282.pdf
This worrisome note appeared at the end of the Significance Statement ...
These findings demonstrate how visual
social media may be harnessed to make accurate
inferences about mental health.
Posted by: Speed | Wednesday, 24 August 2016 at 06:02 PM
Get a blood test to check for thyroid activity first. Low thyroid activity leads to depression, among other things. Fortunately, a hormone replacement, oroxine, is readily available at a cost of pennies per day. one of the reasons that while I live in paradise, I hope there is not a compete collapse of the worldwide ponzi scheme before I lose my grip on the twig.
Cheers, Geoff
Posted by: Geoffrey Heard | Wednesday, 24 August 2016 at 06:35 PM
When I was leaving a partnership and serving out the six months leaving requirement, I used to slip out the back door to go to the sandwich place across the road, and I would think of Three Days of the Condor.
Posted by: David Bennett | Wednesday, 24 August 2016 at 07:04 PM
Like Microsoft's attempt a while back at determining a person's age from a photograph. Whereas in person most people will give my age up to 10 years younger than I am, when I tried the website for this software out of curiosity it put me 10 years over!
I really don't see the use in these things which just seem like a way for the researchers to keep their jobs.
Posted by: Kefyn Moss | Wednesday, 24 August 2016 at 08:01 PM
Mike,
I clicked the link to MIT Tech Review and got a blank page. Went to Review site and still didn't find the article. Do you know why??
Richard Newman
Posted by: Richard Newmsn | Wednesday, 24 August 2016 at 08:43 PM
You make 'sad' and 'depressed' equivalent when you talk about authentic versus poseur -- a very photographer kind of thing to say I think. Authenticity seems to be the prime directive in photography.
Obviously sadness and depression are not equivalent. Maybe there's an overlap.
There are enormous philosophical problems with the premise that (true) states of mind can be inferred (by anyone or anything) from a piece of art. Whether it's post-transition (to digital presumably) or not.
Of course, I haven't read the article so I could be blowing smoke. But on the face of it, it just seems fanciful and deluded to me. Perhaps because treating art as data is just kind of beside the point. Isn't it?
Posted by: Ed Nixon | Wednesday, 24 August 2016 at 09:07 PM
I liked it better when we talked about important stuff, like what kind of jeans Mike wears, and his lack of sponsorship......
Now if you switched to Speedo, there might be an opening there.....
Posted by: Michael Perini | Wednesday, 24 August 2016 at 09:17 PM
This algorithmic approach actually may be a useful adjunct diagnostic. It's worth recalling that the standard psychological evaluation tests, such as the MMPI, only tend to be about 60-65% accurate and that clinical correlation is always necessary for accurate diagnosis and treatment.
It's also worth noting that much of what the algorithm examines can be considered affirmative self-descriptives, such as the filters that are intentionally applied. That's a fair use of data that's likely more reliable than potentially self-conscious responses to a formal psychological diagnostic test.
Posted by: Joe Kashi | Wednesday, 24 August 2016 at 10:24 PM
Mike,
Regarding your comment on "I also don't know how the machine could separate authentic sad photos from poseur sad photos."
The basic answer to this is that the researchers probably used a supervised learning machine algorithm. Essentially the photos are tagged by real people, and this is done at a large scale as in machine learning the machine gets better at the task with more experience..
So it's ability to detect 'authentic' vs. 'poseur' sad photos ultimately depended the response of the people who tagged the photos. thus, the machine is a reflection of the biases of the people it learned from....
Posted by: Saiful Rizal MDRamli | Wednesday, 24 August 2016 at 10:48 PM
That's really interesting, and raises even more interesting questions (as some commuters have already noted).
Of course, all of this leaves aside the value of creative expression in mental health.
Posted by: Steve Caddy | Wednesday, 24 August 2016 at 11:41 PM
"How's that supposed to work—you get an email from someone saying that their robot has detected that you're depressed, based on pictures you've posted online?"
It depends on which robot finds you first. They won't actually tell you you're depressed, but Google's robot will send you uplifting links to click on. Amazon's robot will send you ads for hard liquor and razor blades.
More seriously, it might work through your health insurance. You could get a break on your health care payments if you let your health care provider monitor your activity on the Internet.
Insurance companies are starting to do this with car insurance. If you agree, a small device is installed in the car that sends speed, miles driven, and other data to the insurance company. In theory you pay less for your insurance that way.
Posted by: Bruce McL | Thursday, 25 August 2016 at 12:47 AM
There are plenty of psychometric tools based upon the premise that emotional states as reflected in the facial expressions are accessible to others instinctively and many psychiatric disorders are characterized by impairments in this ability. However this is a cross-sectional assessment. There is this online tool that will tell you the emotional state of a person in a photograph: https://www.microsoft.com/cognitive-services/en-us/emotion-api
The instagram tool probably uses a similar algorithm.
Depression on the other hand is a pervasive, emotional disorder and as has been pointed out above is not equivalent to sadness.
Posted by: SM Singh | Thursday, 25 August 2016 at 01:46 AM
This is a perfect example of bad science. There is so much assumption and subjective analysis in it that it cannot possibly have any value.
Posted by: Bob Johnston | Thursday, 25 August 2016 at 02:43 AM
So let's rename all landscape photography to 'sad selfie photography'. (By the way, some Indian gurus wouldn't protest calling every photo a selfie by definition - a lovely idea, but another matter entirely.)
Posted by: Hans Muus | Thursday, 25 August 2016 at 03:00 AM
Typical silly season news, but depression is a serious thing. Too serious for this kind of distraction, I'd say. People tend to use the term 'depression' as a blanket word that applies to all kinds of sad/blue moods, but it is actually a clinical condition. And one that can have very serious consequences.
It happened to me once, after a break-up and a rather draconian shortage of work: inactivity and loneliness got me so down I had to be medicated. It took me several months to pull through. I mightn't have made it if I hadn't been properly diagnosed by a doctor. Now some bunch of nerds is trying to tell us an algorythm can replace the knowledge of an MD. Some people get so infatuated with computer technology they lose the sense of reality.
As Carl Gustav Jung would have put it, "Oh well."
Posted by: Manuel | Thursday, 25 August 2016 at 05:14 AM
Lots of people say they find my photos depressing. Wonder what that says about me?
Voltz
Posted by: V.I. Voltz | Thursday, 25 August 2016 at 07:24 AM
A lot of interesting and sometimes contradictory stuff out there on inflammation and mental well being.
One study suggests that Statins reduce the risk of dementia. Another suggests that the same drugs interfere with memory. Go figure.
Makes a guy want to go cook up a steak, wash it down with a martini and pile some more dirty laundry on top of the treadmill but that would be bad.
Posted by: mike plews | Thursday, 25 August 2016 at 07:38 AM
Will the next version of Photoshop have a "sadness" or "depression" correction button similar to red-eye correction? Or maybe they should get to work on a radicalization algorithm, to find all the terrorists via their selfies.
I know it's too easy to make snide comments about something that I don't know much about. I'm weak. But history has shown that these things are usually badly reported on or oversold. Remember the story of the zen master and the little boy from the movie Charlie Wilson's war? We'll see.
I would not be surprised if there is something to this. But whatever it is, it's probably not what we think about it now.
Posted by: Robert Roaldi | Thursday, 25 August 2016 at 07:50 AM
I don't think it's about photos... I think it's about social media. There has been a wide range of research published on the links between depression and daily use of social media accounts.
One that I read (but can't find in 10 seconds of googling) indicated something like >20 minutes per day of 'online social media' = propensity to depression.
It probably stems from where one finds happiness; from our own interests and pursuits, or from trying to see what others think of us. (my $0.02 worth).
Posted by: Dalvorius | Thursday, 25 August 2016 at 08:19 AM
I would think that the truly depressed wouldn't post any pictures on Instagram because they felt they weren't worthy.
Posted by: Joseph Brunjes | Thursday, 25 August 2016 at 09:15 AM
Just think, that research is probably far behind that which analyzes blog posts...
I'd write more but probably should spend the time brightening up my Instagram!
Posted by: Ed Grossman | Thursday, 25 August 2016 at 09:34 AM
Cause or effect, I wonder?
I wish they'd make the data openly available. Calling the importance() function on a random forest model will give you a relative-quantified importance of each factor, for starters.
Back on topic - I seem to make some of my best photos when the mood is non-average, either a bit depressed or a bit sunny.
Posted by: Tim | Thursday, 25 August 2016 at 10:11 AM
Also, they should try running it on 500px users...
Posted by: Tim | Thursday, 25 August 2016 at 10:12 AM
Ok, what about those who don't have any idea of what instagram is?
Idle chatter?
Posted by: Herb Cunningham | Thursday, 25 August 2016 at 10:39 AM
Wouldn't it better to just go out and meet real people? While you can tell a lot from photographs, there's no substitute for the real thing, and a fleeting expression may give completely the wrong impression.
Posted by: Roger Bradbury | Thursday, 25 August 2016 at 10:50 AM
Is depressing a shutter button a form of depression?
[No, it's a form of release! --Mike]
Posted by: Herman | Thursday, 25 August 2016 at 11:36 AM
Well this is a novel use of Amazon's Mechanical Turk! I daresay it's not a representative sample though - if you look at the amount that is being paid for the tasks requested you'll see that the pay scale is very very (let me add another one...) very low and thus likely attracting people from a very thin socioeconomic sliver of society. Take a look - https://goo.gl/RBWjlL.
It will be interesting to see this theory tested by other researchers to ascertain it's real value. If it pans out, it will be even more interesting to point the machine at the works of Cartier-Bresson, Kertesz, Eisenstaedt, Lange, and others.
Posted by: JohnMFlores | Thursday, 25 August 2016 at 12:06 PM
Without some level of the so called depressed mood, where would art be?
Posted by: Animesh Ray | Thursday, 25 August 2016 at 12:27 PM
I wish I could be one of those "artists" making expressive images when my depression takes hold but I'm not. When it hits, the cameras sit. And that makes me sad.
Posted by: Wes | Thursday, 25 August 2016 at 01:35 PM
Reviewing the comments to this article makes me wonder how many commenters actually read the article in MIT Technology Review before posting dismissive or negative comments.
Posted by: Joe Kashi | Thursday, 25 August 2016 at 09:17 PM
I've battled depression and anxiety throughout my life. My passion for photography has been and still is a lifesaver--I enjoy taking pictures of gas stations at night, abstract images of the sea, and dogs.
I read the item in Technology Review. My knee jerk reaction: Sure, it's interesting to "draw" conclusions about people based on doodles, photos, music, prose, and poetry. I'm sure the MIT algorithm has an "AI" module embedded so that it will become more accurate over time. Will it reach or exceed six sigma?
I am rather skeptical this technology will be used to reach out to help people who suffer from mental illness. My guess is HR departments, health insurance providers, lending institution, and security agencies will be the beneficiaries.
I imagine a PI (principal investigator) at MIT suggested that analyzing billions of pictures from social media will serve a multitude of interests/end users. The PI applied for and received a grant (probably from the DoD). The methodology for codifying visual elements from photos taken by happy and sad people (developing a new pattern recognition algorithm) is labor intensive/costly.
The technology for acquiring and storing photos posted on social media has been around for years. Security agencies are clamoring for cheaper, better, faster ways of cataloging/codifying/profiling visual content and the people who post it.
I am sure more than a few grad students drank gallons of Red Bull while "mapping dots and vectors" onto happy and sad photos. (Were the photos culled from a control groups of normal and depressed people?)
Once the lab built up a sufficient library of "photo maps," it began the coding process. I am sure the task of writing the algorithm went through a long process of testing and debugging. Eventually it worked well enough to quantify qualitative data with a fair degree of predictability/accuracy.
After the lab published its paper and received favorable peer review, Technology Review published an article for the educated masses.
(I earned an MS in visual studies at MIT in 1984. I send annual tax deductible contributions to the McGovern Institute for Brain Research at MIT: http://mcgovern.mit.edu/)
----Mike, please take a look at this. It's way early in the morning and I'm sure there are some typos and other errors.
Posted by: Bob Rosinsky | Friday, 26 August 2016 at 01:55 AM