{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n\nTransformer as a Graph Neural Network\n======================================\n\n**Author**: Zihao Ye, Jinjing Zhou, Qipeng Guo, Quan Gan, Zheng Zhang\n\n
The tutorial aims at gaining insights into the paper, with code as a mean\n of explanation. The implementation thus is NOT optimized for running\n efficiency. For recommended implementation, please refer to the `official\n examples
The sublayer connection part is little bit different from the\n original paper. However, this implementation is the same as `The Annotated\n Transformer
By calling ``update_graph`` function, you can create your own\n Transformer on any subgraphs with nearly the same code. This\n flexibility enables us to discover new, sparse structures (c.f. local attention\n mentioned `here
Training with wmt14 requires multi-GPU support and is not available. Contributions are welcome!
- :func:`~dgl.DGLGraph.filter_nodes` takes a predicate and a node\n ID list/tensor as input, then returns a tensor of node IDs that satisfy\n the given predicate. \n - :func:`~dgl.DGLGraph.filter_edges` takes a predicate\n and an edge ID list/tensor as input, then returns a tensor of edge IDs\n that satisfy the given predicate.
\n\nFor the full implementation, see the `GitHub\nrepoThe notebook itself is not executable due to many dependencies.\n Download `7_transformer.py