Watson vs. Humans

IBM’s Watson computer, which recently made an impressive showing on Jeopardy!, will next be tasked with helping clinicians in the healthcare environment. In a deal with Nuance Communications, Inc., Watson’s Deep Question Answering, Natural Language Processing, and Machine Learning capabilities will be combined with Nuance’s speech recognition and Clinical Language Understanding to address the increasingly complex demands of healthcare diagnostics. The IBM press release has more details (and lots of capitalized words) on the deal.

One of the hotter topics surrounding Watson’s dominant performance over the course of the three day demo was its (his?) spectacular failure in Final Jeopardy!. The category: US Cities; the answer: “Its largest airport was named for a World War II hero; its second largest, for a World War II battle.” Watson answered “What is Toronto?” IBM’s explanation suggests that Watson relies on statistical analysis for its reasoning and that it downgrades the category description in importance because in this game, they can be vague or downright tricky. But you and I would immediately recognize “US Cities” as a constrained set in which the answer must reside. Watson was confused further, IBM explained, because there are several “Torontos” in the US.

Oh-Oh! If you look at the map they provide showing the various cities in the US named Toronto, it is clear poor Watson must depend on what others tell him is true. As of this writing, the map (click on “What is Toronto?”) labels “Toronto, MO” as being in Illinois, and “Toronto, IL” as being in Missouri. I thought Watson was impressive, and I think IBM has made great strides. As Watson demonstrated, machines can make mistakes. As the map at IBM’s site demonstrates, humans make mistakes too. We just have to remember that at least for now, the only knowledge computers can learn, is what we offer them…

[Update February 21, 2011: Alas, IBM appears to have removed the map above from its website. Cached versions don’t link to the map anymore either…]

© 2011 Rod Piechowski, Inc. Consulting

Leave a Comment

Information trumps technology

A friend of mine has worked at the same company for over 25 years. It’s rare nowadays indeed, but my friend is special to the organization. She is not the company’s CEO nor its CIO. She is not a partner and she is not in a highly visible position outside of the company. The company has been around for over 50 years. It is multinational and you’d recognize the name if I told you. The company has been through many changes. Its fortunes have risen and fallen in concert with the tides of the marketplace. This company has had its share of layoffs, but my friend has survived the cuts, for better or for worse. The company she works for has always tried to stay on top of the technology required to manage the incredible amounts of data that it collects, and as part of that effort, it has seen many iterations of application development, implementation and system retirement. Yet, through it all, my friend remains at this organization because her expertise lies not with the technology, but with the organization’s business model and its data. » Continue reading “Information trumps technology”

Leave a Comment

Technology as Gateway to Knowledge

(This entry is also available as pdf download: see end of article for link.)

“Meaningful Use” is upon us. Hospitals around the country are scrambling to assess its implications which, aside from the obvious financial investment that must be made, will affect healthcare providers in other ways for many years to come. Since change is inevitable, this is the perfect time to look beyond the short-term aspirations of meaningful use, and identify what healthcare should look like into the future, maybe even as far as 20 years. The government however, is not likely to create that vision, so the task will be left to those providers who are bold enough to shape new models for healthcare delivery. » Continue reading “Technology as Gateway to Knowledge”

Leave a Comment

In the Clouds

One of the underlying assumptions in this blog, The Art of Medicine and Technology, is that we will continue to gather more data, hopefully learn from it, and use it to provide continually improved care. Most of this data will be gathered using electronic medical record systems and other information technology systems that are increasingly finding their way into physician practices and hospitals. In our day-to-day lives, we use all kinds of tools that store messages, documents, websites and photos on servers outside of our physical control. We’re growing increasingly dependent on services in the “cloud” of connectivity to store, maintain and move our information. In the field of medicine, the growth of EHRs, and their eventual connectivity as part of the Nationwide Healthcare Information Network (NHIN) will add new dimensions to this concept. What they are talking about moving around however, is data about patient care, quality reporting and payment for services. What about the “knowledge” we will gain from research done on data? » Continue reading “In the Clouds”

Leave a Comment