We present trellis networks, a new architecture for sequence modeling. On the one hand, a trellis network is a temporal convolutional network with special structure, characterized by weight tying across depth and direct injection of the input into deep layers. On the other hand, we show that truncated recurrent networks are equivalent to trellis networks with special sparsity structure in their weight matrices...
Related Content
Deep Layers as Stochastic Solvers
We provide a novel perspective on the forward pass through a block of layers in a deep network. In particular,...
Sparse Dictionary Learning by Dynamical Neural Networks
A dynamical neural network consists of a set of interconnected neurons that interact over time continuously. It can exhibit computational...
SPIGAN: Privileged Adversarial Learning from Simulation
Deep Learning for Computer Vision depends mainly on the source of supervision. Photo-realistic simulators can generate large-scale automatically labeled synthetic...
Graph-DQN: Fast Generalization To Novel Objects Using Prior...
Humans have a remarkable ability to both generalize known actions to novel objects, and reason about novel objects once their...
