=> Watson Spoken Healthcare QA

Video demo: Conversing With Watson - Natural Language Services

How to: IBM Watson, Cognitive Computing & Speech APIs

Hm,

I am not very excited.
I asked Watson:
“Is there a kind of malaria, which kills the patient?”
The right answer would have been:
Malaria tropica,
but Watson is talking about malaria
only in a subordinate clause in “his” first answer-attempt.
In 4 more attempts not even malaria is mentioned.

So let´s wait for the coming Sherlock: )

Andreas

Hmmm…they do state that the ANN is not trained yet, which makes me wonder why they opened the API unless they are depending on public opinion for weighting. And if that’s the case, is this really representative of an expert opinion. Anyway I tested the NLP to see how outside it would go;

WATSON
——————————————————————

ME: Hello
WATSON:Ring!’ Isabel’s husband Juan is at Dona Carla’s house. Juan whispers into the phone, ‘Hello, sweetheart. Is everything ready?’ Dona Carla stands in the background, putting her coat on. Isabel answers, ‘
ME: Jesus Christ I feel like shit
WATSON: I am a much calmer person. I feel like the stress level in my life has gone down tremendously. “I was mobilized in Desert Storm and came back with asthma and other respiratory issues. My breathing has really improved.
Was this answer relevant?
ME: I think I have a cold
WATSON: Nathan decided it was important to share his story. He wants to make everyone aware of the dangers of smoking and exposure to secondhand smoke. “Some people tell me they smoke, and I ask them if they have grandchildren. ‘If you smoke, you may be taking that smoke home to the grandchildren and their small lungs,’ I tell them. I never smoked; look what happened to me.
ME: THanks
WATSON: History of Polio Timeline.
ME: Goodbye
WATSON: 502 Bad Gateway: Registered endpoint failed to handle the request.

SAIL
——————————————————————
What’s your name: Hello
SAIL: Hi there!.
What’s your name: Jesus Christ I feel like shit
SAIL: What seems to be the problem?
What’s your name: I think I have a cold
SAIL: Try zicam. I hear this actually works
What’s your name: THanks
SAIL: No problem
What’s your name: Goodbye
SAIL: Of course. Goodbye.

Not that SAIL is specifically setup at this time to parse chunks of medical records but basic stuff you should be able to hit. Not so much an agent really, sort of reminds me of Ask Jeeves.  Anyway it was cool to be able to connect to WATSON although I confess I was expecting something a little more… JARVIS’ish.

Vince

If I didn’t know any better, I would think that this is NOT IBM’s Watson, but some sort of hoax. Or perhaps it’s simply a case of unrealistic expectations, borne of hyperbole and misguided hope? I dunno; but I can tell you that this wasn’t the ‘droid’ I was looking for.

Dave,

Its a free thirty day trial https://apps.admin.ibmcloud.com/manage/trial/bluemix.html so you can sign up and give it a whirl.  I haven’t looked at pricing to see what types of relationship might be maintained after that, but honestly with all the medical API’s out there, I would have to have a conversation or two directly with someone at IBM to see how much better this will be and why. Andreas makes a good point. If none of the subordinate answers contains anything close to the correct answer, then training is not going to have a net positive affect. And as I surmised, if they are relying on users for training there are 2 related problems, [1] How long, how many passes before the weighting produces an affect.
[2]  security problem. How do you prevent the results from being poisoned?

Vince

this sample is using the sample public healthcare data set, which is only trained to answer specific kinds of questions.

The impression I had from the video was that this application is a lot more like a keyword matcher (though I suppose the Jeopardy Watson was too), with a very limited database of answers. The first question in the video was a textbook question and thus found the perfect answer. The second question only got only a 48% certainty from Watson, which is also how relevant it sounded to me because the answer was plucked out of context or format and not custom generated (it starts off on a seemingly different subject, and seems to contain a title halfway). Notably, the candidate answers contained the words “taking aspirin daily” from the question literally and most frequently.

The original Watson was powerful because it crossreferenced results from many sources to increase its certainty. It’s as if it doesn’t do that here, and I can’t tell if it does much better than a normal FAQ search engine in this domain. It’s like a big ship in a small pond?
I am however impressed with the perfect accuracy of the voice recognition.

To be honest, It made me disappointed. There is no any context understanding in the demo of Watson.