RNNs and their more sophisticated cousins (GRUs, LSTMs) have proven to be “unreasonably effective” at learning from sequence data, although they can be tricky and expensive to train. But what if your data consists of unordered sets or bags of objects, or the data is ordered but the predictive value of that ordering is marginal? Deep Averaging Networks (Iyyer et al 2015) provide a cheap and effective way to learn task-oriented embeddings from this kind of data.
Andrew is a machine learning engineer with an academic background in NLP and IR, and a career that’s weaved its way through biomedical science, social media, online music, educational technology and e-commerce. These days he’s into recommendations, personalisation, search, cats, and food.