Reading comprehension is a really hot topic in deep learning for NLP right now. It’s a high-level NLP problem, and so is interesting just from a scientific perspective (how do we build statistical models that can reason about a passage of text?), and it has some potential product applications. There have been over 10 datasets released related to this task just this year, and dozens of papers publishing methods for solving this problem. The field is moving fast.

This is lightly edited from an email I sent to colleagues at AI2, so it’s somewhat AI2-specific in places. But, here’s what I thought about EMNLP.

There were two papers accepted at ACL 2016 that use sequence-to-sequence models for performing semantic parsing, where phrases in natural language are mapped to logical forms in some formal schema:

I had a good time at NAACL last week. I got to catch up with a lot of old friends, and made some new ones. I had two main takeaways from the actual content of the conference: (1) the science questions we’re focusing on at AI2 are similar to the cloze-style questions many reading comprehension datasets use, and (2) a lot of people are interested in learned execution models for semantic parsers.

I really enjoyed this paper, and it had a significant impact on my research direction. I am looking at combining compositional models of KB inference with models based on factorizing a KB tensor (or otherwise producing relation embeddings). That’s what this paper does too, though it does it in a different way than I was planning. The key insight of this paper is that some patterns in a KB tensor are very difficult to capture with a low-rank factorization, but are easy when a model has access to the original tensor. This paper solves that issue by making a factorization only capture those things that are not captured by features extracted from the original tensor.