This article provides insufficient context for those unfamiliar with the subject. (November 2017) |
Text Nailing (TN) is an information extraction method of semi-automatically extracting structured information from unstructured documents. The method allows a human to interactively review small blobs of text out of a large collection of documents, to identify potentially informative expressions. The identified expressions can be used then to enhance computational methods that rely on text (e.g., Regular expression) as well as advanced natural language processing (NLP) techniques. TN combines two concepts: 1) human-interaction with narrative text to identify highly prevalent non-negated expressions, and 2) conversion of all expressions and notes into non-negated alphabetical-only representations to create homogeneous representations. [1][2]
In traditional machine learning approaches for text classification, a human expert is required to label phrases or entire notes, and then a supervised learning algorithm attempts to generalize the associations and apply them to new data. In contrast, using non-negated distinct expressions eliminates the need for an additional computational method to achieve generalizability.[3][4][5]
History
editTN was developed at Massachusetts General Hospital and was tested in multiple scenarios including the extraction of smoking status, family history of coronary artery disease, identifying patients with sleep disorders,[6] improve the accuracy of the Framingham risk score for patients with non-alcoholic fatty liver disease, and classify non-adherence to type-2 diabetes. A comprehensive review regarding extracting information from textual documents in the electronic health record is available.[7][8]
The importance of using non-negated expressions to achieve an increased accuracy of text-based classifiers was emphasized in a letter published in Communications of the ACM in October 2018.[9]
Source code
editA sample code for extracting smoking status from narrative notes using "nailed expressions" is available in GitHub.[10]
TN as progressive cyber-human intelligence
editIn July 2018 researchers from Virginia Tech and University of Illinois at Urbana–Champaign referred TN as an example for progressive cyber-human intelligence (PCHI).[11]
Criticism of machine learning in health care
editChen & Asch 2017 wrote "With machine learning situated at the peak of inflated expectations, we can soften a subsequent crash into a “trough of disillusionment” by fostering a stronger appreciation of the technology’s capabilities and limitations."[12]
A letter published in Communications of the ACM, "Beyond brute force", emphasized that a brute force approach may perform better than traditional machine learning algorithms when applied to text. The letter stated "... machine learning algorithms, when applied to text, rely on the assumption that any language includes an infinite number of possible expressions. In contrast, across a variety of medical conditions, we observed that clinicians tend to use the same expressions to describe patients' conditions."[13]
In his viewpoint published in June 2018 concerning slow adoption of data-driven findings in medicine, Uri Kartoun, co-creator of Text Nailing states that " ...Text Nailing raised skepticism in reviewers of medical informatics journals who claimed that it relies on simple tricks to simplify the text, and leans heavily on human annotation. TN indeed may seem just like a trick of the light at first glance, but it is actually a fairly sophisticated method that finally caught the attention of more adventurous reviewers and editors who ultimately accepted it for publication."[14]
Criticism
editThe human in-the-loop process is a way to generate features using domain experts. Using domain experts to come up with features is not a novel concept. However, the specific interfaces and method which helps the domain experts create the features are most likely novel.
In this case the features the experts create are equivalent to regular expressions. Removing non-alphabetical characters and matching on "smokesppd" is equal to the regular expression /smokes[^a-zA-Z]*ppd/. Using regular expressions as features for text classification is not novel.
Given these features the classifier is a manually set threshold by the authors, decided by the performance on a set of documents. This is a classifier, it's just that the parameters of the classifier, in this case a threshold, is set manually. Given the same features and documents almost any machine learning algorithm should be able to find the same threshold or (more likely) a better one.
The authors note that using support vector machines (SVM) and hundreds of documents give inferior performance, but does not specify which features or documents the SVM was trained/tested on. A fair comparison would use the same features and document sets as those used by the manual threshold classifier.
References
edit- ^ Kartoun, Uri (2017). "Text nailing". Interactions. 24 (6): 44–9. doi:10.1145/3139488. S2CID 29010232.
- ^ Barbosa, Simone; Cockton, Gilbert (2017). "Avoiding agenda bias with design thoughtfulness". Interactions. 24 (6): 5. doi:10.1145/3151556. S2CID 657561.
- ^ Beam, Andrew L; Kartoun, Uri; Pai, Jennifer K; Chatterjee, Arnaub K; Fitzgerald, Timothy P; Shaw, Stanley Y; Kohane, Isaac S (2017). "Predictive Modeling of Physician-Patient Dynamics That Influence Sleep Medication Prescriptions and Clinical Decision-Making". Scientific Reports. 7: 42282. Bibcode:2017NatSR...742282B. doi:10.1038/srep42282. PMC 5299453. PMID 28181568.
- ^ Simon, Tracey G; Kartoun, Uri; Zheng, Hui; Chan, Andrew T; Chung, Raymond T; Shaw, Stanley; Corey, Kathleen E (2017). "Model for end-stage liver disease Na Score predicts incident major cardiovascular events in patients with nonalcoholic fatty liver disease". Hepatology Communications. 1 (5): 429–438. doi:10.1002/hep4.1051. PMC 5659323. PMID 29085919.
- ^ Corey, Kathleen E; Kartoun, Uri; Zheng, Hui; Chung, Raymond T; Shaw, Stanley Y (2016). "Using an Electronic Medical Records Database to Identify Non-Traditional Cardiovascular Risk Factors in Nonalcoholic Fatty Liver Disease". The American Journal of Gastroenterology. 111 (5): 671–6. doi:10.1038/ajg.2016.44. PMC 4864030. PMID 26925881.
- ^ Kartoun, Uri; et al. (2018). "Development of an Algorithm to Identify Patients with Physician-Documented Insomnia". Scientific Reports. 8 (1): 7862. Bibcode:2018NatSR...8.7862K. doi:10.1038/s41598-018-25312-z. PMC 5959894. PMID 29777125.
- ^ Meystre, S. M; Savova, G. K; Kipper-Schuler, K. C; Hurdle, J. F (2008). "Extracting information from textual documents in the electronic health record: A review of recent research". Yearbook of Medical Informatics: 128–44. PMID 18660887.
- ^ Wang, Yanshan; Wang, Liwei; Rastegar-Mojarad, Majid; Moon, Sungrim; Shen, Feichen; Afzal, Naveed; Liu, Sijia; Zeng, Yuqun; Mehrabi, Saeed; Sohn, Sunghwan; Liu, Hongfang (2018). "Clinical information extraction applications: A literature review". Journal of Biomedical Informatics. 77: 34–49. doi:10.1016/j.jbi.2017.11.011. PMC 5771858. PMID 29162496.
- ^ CACM Staff (2018). "More accurate text analysis for better patient outcomes". Communications of the ACM. 61 (10): 6–7. doi:10.1145/3273019. S2CID 52901757.
- ^ "GitHub - kartoun/text-nailing". GitHub. 2018-01-07.
- ^ Rikakis, Thanassis; Kelliher, Aisling; Huang, Jia-Bin; Sundaram, Hari (2018). "Progressive cyber-human intelligence for social good". Interactions. 25 (4): 52–56. doi:10.1145/3231559. S2CID 49563432.
- ^ Chen, Jonathan H; Asch, Steven M (2017). "Machine Learning and Prediction in Medicine — Beyond the Peak of Inflated Expectations". New England Journal of Medicine. 376 (26): 2507–9. doi:10.1056/NEJMp1702071. PMC 5953825. PMID 28657867.
- ^ CACM Staff (2017). "Beyond brute force". Communications of the ACM. 60 (10): 8–9. doi:10.1145/3135241.
- ^ Kartoun, Uri (2018). "Toward an accelerated adoption of data-driven findings in medicine". Medicine, Health Care and Philosophy. 22 (1): 153–157. doi:10.1007/s11019-018-9845-y. PMID 29882052. S2CID 46973857.