
Contents
-
-
-
-
-
-
-
-
-
-
-
-
-
15.1 Introduction 15.1 Introduction
-
15.2 Basic Setting 15.2 Basic Setting
-
15.3 Building Blocks 15.3 Building Blocks
-
15.3.1 Layers: Basic Building Blocks 15.3.1 Layers: Basic Building Blocks
-
15.3.2 Layers for Sequential Input 15.3.2 Layers for Sequential Input
-
15.3.2.1 Simple recurrent layer 15.3.2.1 Simple recurrent layer
-
Bidirectional Recurrent Layer Bidirectional Recurrent Layer
-
-
15.3.2.2 Gated recurrent layer 15.3.2.2 Gated recurrent layer
-
Long Short-Term Memory Long Short-Term Memory
-
Gated Recurrent Unit Gated Recurrent Unit
-
-
-
15.3.3 Extra Layers 15.3.3 Extra Layers
-
-
15.4 Deep Learning for Document Classification 15.4 Deep Learning for Document Classification
-
15.4.1 Document Representation 15.4.1 Document Representation
-
15.4.2 Document Classifiers 15.4.2 Document Classifiers
-
15.4.2.1 Fully connected network 15.4.2.1 Fully connected network
-
15.4.2.2 Convolutional network 15.4.2.2 Convolutional network
-
Convolution as n-Gram Detector Convolution as n-Gram Detector
-
Loss of Temporal Consistency Loss of Temporal Consistency
-
-
15.4.2.3 Recurrent network 15.4.2.3 Recurrent network
-
Convolutional–Recurrent Network: Hybrid Network Convolutional–Recurrent Network: Hybrid Network
-
-
-
15.4.3 Training and Evaluation 15.4.3 Training and Evaluation
-
-
15.5 Neural Language Modelling and Neural Machine Translation 15.5 Neural Language Modelling and Neural Machine Translation
-
15.5.1 Language Modelling 15.5.1 Language Modelling
-
15.5.1.1 n-gram language modelling 15.5.1.1 n-gram language modelling
-
15.5.1.2 Feedforward and recurrent language modelling 15.5.1.2 Feedforward and recurrent language modelling
-
Feedforward Language Model Feedforward Language Model
-
Recurrent Language Model Recurrent Language Model
-
-
15.5.1.3 Non-sequential language modelling: continuous bag-of-words 15.5.1.3 Non-sequential language modelling: continuous bag-of-words
-
-
15.5.2 Neural Machine Translation 15.5.2 Neural Machine Translation
-
15.5.2.1 Sentence generation using a recurrent language model 15.5.2.1 Sentence generation using a recurrent language model
-
15.5.2.2 Conditional recurrent language modelling 15.5.2.2 Conditional recurrent language modelling
-
15.5.2.3 Encoder–decoder model: sequence-to-sequence learning 15.5.2.3 Encoder–decoder model: sequence-to-sequence learning
-
15.5.2.4 Attention-based encoder–decoder model 15.5.2.4 Attention-based encoder–decoder model
-
Relationship to Word Alignment Relationship to Word Alignment
-
-
15.5.2.5 Approximate decoding 15.5.2.5 Approximate decoding
-
Greedy Search Greedy Search
-
Beam Search Beam Search
-
Diverse Decoding Strategies Diverse Decoding Strategies
-
-
-
-
15.6 Distributed Representation 15.6 Distributed Representation
-
15.6.1 What Is the Distributed Representation of a Linguistic Symbol? 15.6.1 What Is the Distributed Representation of a Linguistic Symbol?
-
15.6.2 Distributed Representations in Practice 15.6.2 Distributed Representations in Practice
-
15.6.2.1 Word vectors 15.6.2.1 Word vectors
-
A Word as a Sequence of Sub-Word Units A Word as a Sequence of Sub-Word Units
-
-
15.6.2.2 Sentence vectors 15.6.2.2 Sentence vectors
-
Bottom-Up Approach Bottom-Up Approach
-
Top-Down Approach Top-Down Approach
-
Hybrid Approach Hybrid Approach
-
-
-
-
15.7 New Opportunities with Deep Learning 15.7 New Opportunities with Deep Learning
-
15.7.1 Multilingual Modelling 15.7.1 Multilingual Modelling
-
15.7.2 Larger-Context and Multimodal Modelling 15.7.2 Larger-Context and Multimodal Modelling
-
-
Further Reading and Relevant Resources Further Reading and Relevant Resources
-
References References
-
Abbreviations Abbreviations
-
Glossary Glossary
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
15 Deep Learning
Get accessKyunghyun Cho is an Associate Professor of Computer Science and Data Science at New York University, and a Senior Director of Frontier Research at Prescient Design of Genentech Research and Early Development (gRED). He was a post-doctoral fellow at the University of Montreal until summer 2015 under the supervision of Professor Yoshua Bengio, and received PhD and MSc degrees from Aalto University in early 2014 under the supervision of Professor Juha Karhunen, Dr Tapani Raiko, and Dr Alexander Ilin. He tries his best to find a balance among machine learning, natural language processing, and life, but almost always fails to do so.
-
Published:07 March 2018
Cite
Abstract
Deep learning has rapidly gained huge popularity among researchers in natural-language processing and computational linguistics in recent years. This chapter gives a comprehensive and detailed overview of recent deep-learning-based approaches to challenging problems in natural-language processing, specifically focusing on document classification, language modelling, and machine translation. At the end of the chapter, new opportunities in natural-language processing made possible by deep learning are discussed, which are multilingual and larger-context modelling.
Sign in
Personal account
- Sign in with email/username & password
- Get email alerts
- Save searches
- Purchase content
- Activate your purchase/trial code
- Add your ORCID iD
Purchase
Our books are available by subscription or purchase to libraries and institutions.
Purchasing informationMonth: | Total Views: |
---|---|
October 2022 | 12 |
November 2022 | 12 |
December 2022 | 11 |
January 2023 | 16 |
February 2023 | 8 |
March 2023 | 9 |
April 2023 | 6 |
May 2023 | 4 |
June 2023 | 18 |
July 2023 | 10 |
August 2023 | 9 |
September 2023 | 3 |
October 2023 | 12 |
November 2023 | 12 |
December 2023 | 3 |
January 2024 | 6 |
February 2024 | 4 |
March 2024 | 18 |
April 2024 | 12 |
May 2024 | 12 |
June 2024 | 12 |
July 2024 | 7 |
August 2024 | 15 |
September 2024 | 10 |
October 2024 | 11 |
November 2024 | 3 |
December 2024 | 6 |
January 2025 | 8 |
February 2025 | 6 |
March 2025 | 18 |
April 2025 | 15 |
May 2025 | 5 |
Get help with access
Institutional access
Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:
IP based access
Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.
Sign in through your institution
Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.
If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.
Sign in with a library card
Enter your library card number to sign in. If you cannot sign in, please contact your librarian.
Society Members
Society member access to a journal is achieved in one of the following ways:
Sign in through society site
Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:
If you do not have a society account or have forgotten your username or password, please contact your society.
Sign in using a personal account
Some societies use Oxford Academic personal accounts to provide access to their members. See below.
Personal account
A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.
Some societies use Oxford Academic personal accounts to provide access to their members.
Viewing your signed in accounts
Click the account icon in the top right to:
Signed in but can't access content
Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.
Institutional account management
For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.