In Bereshit Rabbah 67:3, Rabbi Levi delineates six “organs” that humans use to engage with the world, dividing them into three that are under our “control” and three that are not. With the eyes, the human “sees what he does not wish to see.”
This is striking to many of us moderns, who would place the eye in the former category – because we can move our eyes and heads around, as well as close or cover our eyes to avoid seeing something. Research in neuroscience, however, has made it clearer what Rabbi Levi may have meant. The eye is basically an extension of the brain, and is also one of the most externally visible organs, making it simultaneously powerful and vulnerable. Tiny eye movements, only detectable with technologically advanced equipment, are increasingly opening up a window to activity in the brain that we previously thought fully concealed.
The tech industry has duly taken note, experimenting with using the eye as an interface by tracking all kinds of eye movement. But the eye is so much more than a vessel for information, as David Zakariaie found. His company, Senseye, is currently conducting large-scale clinical trials for an app that, with the help of artificial intelligence, could analyze eye movement – particularly, patterns of pupil dilation – and alert doctors to the possible presence of common psychological conditions like anxiety, depression and post-traumatic stress disorder (PTSD) that our society has struggled to address. What follows is an edited talk and interview about Senseye between Mr. Zakariaie and Rabbi Neil Blumofe, held at Congregation Agudas Achim in Austin, TX.
David Zakariaie: What originally got me into starting Senseye was kind of a couple of different things that came together at once. What really got me interested in the space of examining the eyes and looking at the eyes in general – for those of you that remember, in 2012, Google introduced a product called Google Glass. It was basically a full smartphone inside a pair of glasses. And just finding myself in the right place at the right time, I was one of the first few programmers that had been given a pair of them.
As cool as I thought it was, one thing that I thought was really stupid about it was that the best idea they had for how to interact with it was to slap a mousepad on the side of your head. But that got me really interested in the evolution of interface technology over time – so, how we went from a keyboard on a typewriter, to a monitor on a PC, to a touch screen on a cell phone, and so on and so forth. That kind of natural evolution got me thinking: if glasses or wearables are the next big thing, what is the interface for that?
And so that’s what got me into the eye originally. Google Glass was a fad and died and this and that, but just from the initial research I did, it became very clear that there was almost 100 years of scientific literature that talked about all these different cognitive, emotional and psychological states that you are able to detect based on changes in the eye.
There were really two main issues that were stopping us from being able to actually take advantage of it.
The first was the only thing you could really measure is the pupil getting bigger or smaller. You’d have a black circle changing size, and the size of that black circle would change for hundreds of thousands of reasons, including just a change in light.
And then, number two, you need a couple hundred thousand dollars worth of eye-tracking equipment to even measure that. The original idea that led to me starting Senseye was wanting to find a way to do that with just a normal video camera. And that if we could find different variables in the eye to measure beyond just the pupil getting bigger or smaller, we can probably find some sort of specificity about what types of activity in the brain are causing changes inside of the eye.
What’s really interesting is that from an anatomy and physiology standpoint, the eye is regarded as being effectively part of the brain, and it’s one of the only organs that you’re actually able to see. And so there’s a lot of information there, just about trying to find different ways to extrapolate that information from it. And so that’s what kind of got me down the journey of looking at the eyes.
As we got the technology to the point where it was working, we got some of the science published and peer-reviewed, we then had to do what every startup has to – just figure out what in the world the application for it really is. We had a lot of different ideas, but we settled on mental health for two main reasons.
One, it was a very untapped opportunity. I don’t think anyone here would be surprised to hear that the way we diagnose mental health today is not the best system in the world. I would argue that it’s probably the part of the science that is the most in need of innovation. And so it seemed like a very large, commercial, untapped opportunity that if we were able to solve it would create a lot of opportunities.
And then two, from a societal standpoint, if we could get Senseye to work, there would be very few lives in the world that it wouldn’t touch in a positive manner. The crossover of those two just made it kind of too large of an opportunity to not try.
How Senseye Works
Senseye is effectively an app on a mobile phone. The first time a patient interacts with it, it’s about a ten-and-a-half minute test prescribed by a doctor. You’re shown different types of stimuli on the screen of the phone while Senseye makes a video recording your eyes’ response to it.
Some of the stimuli we show is benign, and some of the images that we show are very annoying, like a picture of someone with their shoe half off the ground and a bunch of gum stuck underneath it, or a carton of milk in the fridge – but it’s empty.
In terms of what we’re actually measuring inside of the eyes, there are about a hundred different things, but they really fit into three main categories. The first is all the conventional ocular metrics that people normally think of. So things like pupil size, blinks, gaze, saccades, and eye movement.
The second category is a little bit more novel. We measure changes in the dilator and sphincter muscles of the iris. The dilator is innervated by the sympathetic nervous system and the sphincter by the parasympathetic. That’s a very fancy way of saying that we’re able to actually see what part of the nervous system the stimuli are activating. And third is to measure heart rate variability.
All of that information, if you were just to provide it raw to a clinician, wouldn’t really mean much to them. It’s a ten-minute test, and we’re measuring a lot of variables over those ten minutes. The core of what the product is: we’re taking all that information from the video of the eye being shown stimuli, and it’s all being fed into a series of machine-learning models, and that’s where the AI comes into play, taking all that information – and then spitting out a diagnosis and a severity score.
The Problems With Mental-Health Diagnosis
Along the lines of the relationship between the eyes and the mind, the problem that we’re effectively trying to solve at Senseye is that there’s really no objective way to measure anything within the behavioral-health space.
If you show up at the primary care doctor’s office today, most of them, as you’re doing your paperwork, will have you fill out an assessment called the PHQ-9, the Patient Health Questionnaire, which is very accurate if the patient is 100% honest about everything, but that’s not always the case. I think that’s become the standard way to assess mental health in the United States. And it’s wrong most of the time.
There are these very structured, long- form interviews that psychologists and psychiatrists are trained in using. An example of this is an assessment called the CAPS-5, which is used for diagnosing Post-Traumatic Stress Disorder (PTSD). And while those assessments, if the clinicians actually are trained properly, are actually pretty accurate. The problem is that we are working with a large population, and a lot of clinicians don’t have enough time for every patient.
As a result, what actually happens in the real world is that about 75% of all mental health problems are actually diagnosed and treated exclusively in primary care. You have a lot of clinicians who are doing the best that they can, but they’re not trained in diagnosing and treating mental health – and more importantly, they don’t really have any tools. They’re using these very simple questionnaires that are all kind of based on patient self-reporting. What often happens is that the condition ends up being labeled “anxiety” or “depression,” the physician prescribes an SSRI or Xanax, and that’s kind of the end of the road. And that’s unfortunate, and that’s what we’re hoping to fix.
What we’re really trying to do is not replace the clinician, but build a very powerful diagnostic that we’re able to put into their hands. If they’re able to much more quickly and accurately understand what a patient actually has, they can, in a much more rapid manner, figure out what the proper course of treatment is, whether it’s talk therapy or a drug, or some hybrid of the two. Or, if a condition is diagnosed in primary care, a rapid referral network can be set into motion, in the same way where if they find a heart issue in primary care, they’ll call a cardiologist.
As we get a lot better at diagnosing and like, not just giving a high-level diagnostic label, but like, understanding in a bit more granularity what’s going on with the patient, we can also triage a bit, so we’re able to know which patients can wait two weeks for an appointment and which ones need an appointment immediately, who needs to go to a psychiatrist versus a psychologist, and so on and so forth, and kind of help provide all that information to the primary-care physician.
In our ideal world, and where we hope to eventually get, would be Senseye basically being used as a general mental health screen for everyone in the United States when they’re going through a primary care visit. There are certainly applications for psychologists and psychiatrists as well. This is where we’re starting, but what we’ll see what the ideal application is down the line. You might think of it this way: if you’re being screened for so many other conditions, Senseye makes sense to do as well, because your eye can’t lie the same way that your brain can.
What if Isaac Used Senseye?
Neil Blumofe: Judaism speaks quite a bit about how eyes are really the gateway to the soul. We cover our eyes when we light the Shabbat candles; we maybe look at somebody when we’re blessing them. There’s a real sense of intimacy with all of that. So I just was thinking a little bit about what Judaism would have to say about something like Senseye. Let’s look at Isaac as if he were being assessed with Senseye. I think he would be an interesting customer.
“And it came to pass, that when Isaac was old and his eyes were dim so that he could not see…”
Whether the metrics of the eye can cited or not cited – that’s very interesting. But let’s assume that that might be a thing, so that you could see that there’s something going on with Isaac. It might be PTSD; it could be anxiety; it could be depression; it could be a lot of different things.
“Isaac’s eyes were dimmed through the smoke raised by those who are offering incense to idols.” (Midrash Tanchuma, Toldot 8).
Here, Isaac is in the wrong place at the wrong time. And the way that he can’t see so well, or the way that his eyes are being affected, is that he’s being affected by this incense smoke being offered not to God but to idols, which is interesting. Rashi gives us another idea:
“Another explanation is: When Isaac was bound upon the altar and Abraham, his father, was about to sacrifice him, at that very moment, the heavens opened and the ministering angels looked up at this and wept. Their tears flowed and fell into Isaac’s eyes, which thus became dim.” (Genesis Rabbah 65:5).
So, he’s got PTSD from what his dad did to him a couple of years before. The idea that he’s affected because he’s got the tears of angels in his eyes is very interesting. And then there’s maybe this other idea, which is an idea that it happens because of fate, or God’s plan. As Rashi continues:
“Another explanation: They became dim in order for Jacob to receive the blessings.” (Genesis Rabbah 65:8)
Because that’s how it’s supposed to be, because Esau (I don’t want to ruin it for you if you haven’t read it) was supposed to receive the blessing, and wasn’t able to.
Difficulties in Diagnosing PTSD
David Zakariaie: Building off the trauma Isaac experienced, let’s think about how there are different clusters of symptoms in a patient who has PTSD, and I think even a psychiatrist would tell you that no two patients present the same in the office.
In the opinion of a lot of the clinicians that we talked to, one of the biggest issues with diagnosing PTSD today is you have to have what’s called a “criterion,” a traumatic event. This means two things. One, you have to have a patient who is willing to actually discuss and disclose what that traumatic event is to be clinician. Research shows, generally, that it takes seven to eleven years for a patient to actually make that trauma disclosure from the point they were initially diagnosed.
And then B – and this is not always a problem – but the clinician does have to make a judgment call on whether they think what happened is traumatic enough for it to actually count as a criterion and actually be called PTSD. And there are obviously some things that are very clearly traumatic. Sexual assault, being in the military, someone dying, all those other things are very clear traumas. But there are also a lot of cases where it was just as traumatic as one of those things for the patient, but a clinician just doesn’t see it that way.
One of our board members, Steven Berkowitz, is a child psychiatrist. And he’s very good, so fortunately, but for him, unfortunately, he’s kind of become the “school shooting guy.” So when it’s school shootings happen, he’s one of those psychiatrists on the ground who is helping communities deal with the aftermath. And that goes back to kind of the Columbine days. He had a patient in Colorado, where he’s now at, that he was treating for 12 years. And only after 12 years of treating her did she disclose to him that she was a student at Columbine, and that she had gone through all of that. And so even in that case, when she sought out the psychiatrist who she had met back then. She took that long to actually disclose. He was not treating her for PTSD until then.
And so that’s kind of what we’re trying to take the subjectivity out of. The DSM is basically a book that has a list of all the mental health conditions and then a list of their symptoms, and you go through the symptoms and you kind of figure out if someone has a certain condition or not. It makes sense that the only way to tell the difference sometimes between PTSD and some of these other conditions is that you need that trauma disclosure. Rather than just measure the observable symptoms, we’re just trying to do is find a way to directly measure the underlying neurological reasons why those symptoms are presenting in the first place.
Privacy: Patient Information, And What Happens To It
There are other folks that are taking different biometric approaches. I would say there are about 40 companies that we’re aware of that are trying to, in some way, shape or form, measure mental health. Thirty-five of them are using voice as a biomarker. I’m not an expert on the voice stuff, but my understanding is that within a facial-expression space, there are general facial expressions, and there’s the concept of micro-expression, where you don’t have as much conscious control over it. And so the way they’re analyzing voice – I’m not an expert in sound waves, but there are similar aspects of your voice that you don’t have much conscious control over. And that’s what they’re looking at.
From our own testing, some of the voice stuff is actually really good, but some of it is not. But we view our approach as complementary to it: where we’re trying to deliver a diagnosis, while they can basically take a 30-second sample of your voice and give you a risk score. So for an application where you’re trying to screen a school district or any type of a large situation like that, a 30-second sample of someone’s voice is a much simpler deployment.
But our general approach for how we’re building this is that we just don’t want to touch the person. It’s very important to us that there’s no proprietary hardware, that the software is 100% stand-off. And so as long as you can measure it with a phone or a laptop, if it’s voice, or anything else, if it works, there’s no reason why we wouldn’t want to incorporate it as well.
So at the end of the day, although it’s a piece of software and it’s an app and all that, Senseye is still a regulated medical device, and it does have to be prescribed by a clinician. So the same privacy rules that apply to anything other type of record – whether it’s your x-rays or your MRI scans – also apply to Senseye. All of the information that we’re capturing still lives within those same kinds of cybersecurity standards and privacy standards that any other manufacturer of a medical device would have to comply with.
At this point, I think you can assume that every company is going to eventually have a data breach. It’s not a matter of if you get hacked, it’s when are you going to get hacked. We determined that the most sensitive thing that we’re capturing is people’s faces, so we extract the eyeballs from our footage. Then an algorithm translates this into tabular data. All of this is running 100% on the user’s own device. So we’re only transmitting a randomized, anonymized patient ID number, which doesn’t have any meaning to us at all. Plus, the tabular data too, the electronic health records and all that – there’s nothing identifying that is ever being saved on Senseye’s side.
Building Trust and Reducing Healthcare Costs
The healthcare system in the US is very weird for many reasons, one of which is that there are many stakeholders. We have all these drug companies advertising drugs on TV and online, and promoting them to doctors. At the end of the day, your customer is the patient, and you have to convince the doctor to actually prescribe the thing in the first place – but the insurance company is the one actually paying for it.
You have to build trust, and also make the sale to all three of those constituencies. Some of the insurance companies have come back to us: “Well, we know mental health is under-diagnosed. So you’re telling us that if you correctly diagnose mental health – all of a sudden we’re going to have all these extra patients, and that’s going to over-inflate our budget.” So, just as much as we’re going to have to prove to the insurance companies that the diagnostic actually works, we will also need to prove that if you are properly diagnosing mental health issues, their costs will actually go down.
For example, in PTSD, there’s a phenomenon called the insurance industry calls “frequent flyers,” which are basically patients who have undiagnosed PTSD who keep showing up at the doctor’s office or in the hospital room with heart issues, rashes, and all sort of other problems, but these are really just the PTSD manifesting and causing other issues. There was a study that the insurance company Cigna published with moderate-to-severe PTSD patients, that showed the cost difference for them between a PTSD patient who has been properly diagnosed and one who hasn’t. Those patients’ medical expenses cost Cigna $47,000 a year on average. By comparison, for a patient who’s been diagnosed correctly with PTSD, the cost is more like $5,000 a year. So, you have to basically just draw that picture for them, of “Well, yes, on one side of the equation, your cost will go up, but hopefully we can bring your costs down on the other side.”
You also have to convince the doctors that you’re not trying to replace them, and that it’s not just a diagnostic, but it’s a tool for them. It’s not something patients are diagnosing themselves with at home. You have to basically educate them, through the academic community, what you’re actually doing. And then you have to also market it to patients so that they see it as an option, know to ask for it, and understand the way it works.
The Trial Process
There do exist very accurate ways to diagnose mental health in the world right now. The problem is that they require a very, very specialized psychologist or psychiatrist who has been trained in how to deal with these particular batteries of tests. It’s just not what’s used in the real world because it just takes way too long. What we’re doing in Senseye’s clinical trials, though, is using those much higher, more accurate standards – in both the training phase and the validation phase.
Just to very quickly take you through it, using the basic design of our PTSD trial as an example: there’s a PTSD group, and that’s half the patients that are enrolled. The other half are evenly divided between four different control groups. There’s a depression group, an anxiety group, a clean control group, and then a “trauma-exposed, but it never developed into PTSD” group. This last one is meant to demonstrate the difference between mere trauma reactivity and clinical PTSD.
Before a patient is enrolled on any one of those trial arms, they have about a five- to six-hour workup with a psychiatrist. That process is videotaped and shown to an additional two psychiatrists, and all three of them, independently, have to come to the same diagnosis and a severity score within one standard deviation of one another. And only if they do that do we enroll the patient. Assuming they do, we then do a urine test, a blood biomarker test, and an endothelial dysfunction test, because one of the most common physical symptoms of having PTSD is low elasticity of the veins. It’s not a perfect correlation, but pretty prevalent. And then, specifically at the trial site at UT Austin, we also do functional MRI because they actually have one of the few non-magnetic cameras. You normally can’t put a camera inside of an fMRI because it’s magnetic. But these cameras can collect our ocular metrics, while the subject is also getting a functional MRI.
We’ve kind of tried to rope as many of these silver standards together to form a kind of ground truth that the FDA is comparing us to. It’s by no means perfect, but it’s the best we can get to.
The one thing that we do know is: you definitely have to take a multimodal approach. So if you look at what’s in the scientific literature, even around the kind of ocular metrics that we’re measuring, you’ll see maybe five or ten papers published talking about the link between pupil size and PTSD, and you’ll see another five papers talking about gaze and PTSD, and so on and so forth. And their predictions will get to about 60 or 65% accuracy, because there’s not one single metric that’s linked to a certain indication.
We are testing Senseye’s accuracy in a couple of different ways. If a patient takes a test with one set of stimuli, immediately after, we’ll re-test them a different set of stimuli, just to kind of ensure that we’re getting the same type of diagnosis. In this clinical trial, we’re also doing about six months of follow-up – so, we are testing the patients weekly for about six months. The patients are being enrolled in treatment after the initial diagnosis, however – so we are actually expecting changes in severity as we’re measuring them and ensuring that we’re still getting the right diagnosis on them.
One thing that we don’t 100% know the answer to yet is if what we’re measuring are state markers or trait markers. So, can we only see these reactions in patients who actively have PTSD? In people who are in remission, or whose symptoms have gone below below a threshold, will we no longer be able to see the same reactions? We don’t entirely know yet.
The reality of mental health is so messy and there are so many variables – what someone experiences in a given day can cause fluctuations in their mental health. Until we get to the point where the model has trained on millions, maybe tens of millions of patients’ worth of data, a 95%-accurate model is just not possible. There are just too many variables for a model training on 5,000 or 6,000 patients to be able to account for all of the edge cases. So we expect to the accuracy to land somewhere between 88 and 92% accuracy by the time we submit our trial results to the FDA.
The Sages on Suffering
Neil Blumofe: Rabbi Shlomo Wolbe, who’s a mid-20th century rabbi, said – maybe this is just obvious to people – but, “The very knowledge of the fact that it is natural to have good days and bad days has the power to assuage a lot of despair and sadness” (Alei Shur, vol. 1, p. 35).
Rabbi Nachman was an 18th-century great-grandson of the Baal Shem Tov. In the Hasidic literature, this is attributed to him:
In general, one must try with all one’s might to be joyful always. For it is human nature to be drawn to bitterness and sadness because of the wounds one has suffered – and every person is full of troubles. So one must force oneself, with a great effort, to be happy always… Now, it is also true that a broken heart is very good – but only at certain times. So it is wise to set an hour each day to break one’s heart and talk to God, as we do. But the rest of the day, one must be in joy.
We’re all suffering, but our task is to not have the suffering overpower us, to put it in a place where we can deal with it – let’s say, an hour a day – and to try to live our lives the best way that we can. Wouldn’t it be great to learn how to containerize that to one hour a day, as opposed to having it sort of inform everything?
So perhaps, you know, we can argue or think about what it means to talk to God. That might be the “talking” here that Rabbi Nachman is speaking about – that we have a time devoted to working this out, or thinking about it, or thinking about the trauma, or thinking about the ways that we’re affected. And our hearts are “broken,” to use his language, but not to have that overpower us to a point where we are immobile, and to work and to try to work to compartmentalize that – so, we’re working on it, but to find a place of joy. Now, Rabbi Nachman had a very tumultuous life. He died at the age of 37 of tuberculosis. But it would be interesting if he took the Senseye. Maybe Isaac was too old. But Rabbi Nachman – it might be interesting what you find.
Conclusion
David Zakariaie: So it’s my personal belief that, to an extent, we are all suffering from a mental health condition of some kind. And from my own experience and from that of folks that I’ve talked to, I think that it has a much more detrimental impact on our lives than we think. For some people, maybe it’s a day at a time; for some, it’s a couple years at a time.
What’s really exciting to me about the mental-health field is that, for the most part we have very powerful therapeutic options for all of these things, from talk therapy to medication. But we are not able to properly match patients with what condition they have, and then even from there, figure out what subset of that patient probability we map that onto and what type of treatment.
But with Senseye, we may be able to diagnose mental health accurately across a broad spectrum of indications, whether it’s a patient who has very complex and symptomatic bipolar disorder, all the way to someone who has anxiety, depression. From a medical standpoint, with treatments, or from a lifestyle standpoint, you can get their baseline. With Apple Watch and other wearables, we have a ton of activity monitoring available to us. We can monitor what we’re eating, what we’re drinking, how much exercise we’re doing. It’s not difficult to look at the data and figure out what variables to control in what way in your life, to get to maximum “peak performance.” And the same way you can do that from an athletic standpoint, you can do it from a mental-health standpoint as well.
So, that’s what excites me the most. Not so much the medical piece of this, of enabling doctors, which is where we have to start. And at some point, it becomes not just about managing people who are showing up as the most symptomatic, but you can also flip it and figure out how to enable people to kind of understand what they need to do to kind of get their mental health and their cognitive health in the best shape.
The mission of Senseye – even if you take all the mental health stuff out of it – is to enable everyone to build and live their best lives, be the best version of themselves, and kind of hit whatever their maximum potential is. If we’re able to help individuals treat and manage their mental health, that makes things so much easier for everyone.
(This post is part of Sinai and Synapses’ project Scientists in Synagogues, a grass-roots program to offer Jews opportunities to explore the most interesting and pressing questions surrounding Judaism and science. David Zakariaie, CEO of Senseye, kicked off a six-week series of post-Tikkun Leil Shavuot classes titled “Jewish Ethics, AI and the Future” at Congregation Agudas Achim in Austin, TX; this post is adapted from the transcript.)
0 Comments