Inside Watson’s challenges in an unexpected place

I tend to tune out at conference Q&A time, but there was a terrific little give-and-take about IBM’s Watson today at EmTech MIT, hosted by MIT Technology Review.

The session had been really quite good, a forthright discussion between Jason Pontin, TR’s editor-in-chief and publisher, and Mike Rhodin, who runs IBM’s Watson group. Then the audience got to ask a few questions. One of the people who spoke turned out to be Michael Bueche, who works in USAA’s innovation group. USAA, a financial services company for military personnel and their families, announced in July it would pilot Watson Engagement Advisor to help soldiers transitioning to civilian life.

Bueche said there had been some challenges in adopting Watson. One issue is that when you ask Watson a question, “You can’t necessarily clarify the question further,” Bueche said. “Watson tends to start from scratch every question,” he said. Bueche said that USAA also has had to work to figure out how to balance privacy issues around personal data. “When they ask questions you need to know things about the person to make the recommendation. Putting that type of information into Watson you get into issues of privacy and things like that.”Bueche also said that curating data to make it useful for Watson took “a lot of time,” and that Watson was designed to work with larger amounts of data than USAA was using for the pilot. He also recommended companies plan on extensive training time to get Watson up and running.

Rhodin took Bueche’s comments in stride, and said that IBM was working on them. He said IBM was working to change redefine the system so that the corpus, the data Watson uses to answer questions, is generally available information, and personal information “comes in as part of the question, not the corpus.”

For the issue of clarifying questions, Rhodin said that it is broader than just programming Watson to allow for follow-up questions. “We’re trying to teach Watson when to not answer the question,” Rhodin said. He said that some questions were ambiguous, which means Watson cannot give a correct answer to them. “We are starting down that journey,” Rhodin said, noting that in May, IBM bought a company called Cognea, which had used artificial intelligence techniques to help computers engage in dialogue with people. IBM is working to integrate Cognea with Watson.
Rhodin also acknowledged that Watson is designed to work with very large amounts of data.

Rhodin also said that IBM was also finding that corporate customers don’t necessarily come with questions. “Many times people hand us problems. Those have to be broken in to many questions,” he said. He mentioned that IBM was working with the Cleveland Clinic to develop a reasoning engine.

Rhodin said from the stage he looked forward to seeing Mr. Bueche at USAA on Thursday, before moving on to the next question.

This kind of open discussion about an ongoing technology project doesn’t happen often at conferences; Rhodin handled it well enough that I wondered if the discussion had been staged. I’d love to be at that meeting this Thursday.

Leave a Reply