:orphan:

Paper Study with DGL
=========================================



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    </div>


Graph neural networks and its variants
--------------------------------------------

* **Graph convolutional network (GCN)** `[research paper] <https://arxiv.org/abs/1609.02907>`__ `[tutorial]
  <1_gnn/1_gcn.html>`__ `[Pytorch code]
  <https://github.com/dmlc/dgl/blob/master/examples/pytorch/gcn>`__
  `[MXNet code]
  <https://github.com/dmlc/dgl/tree/master/examples/mxnet/gcn>`__:

* **Graph attention network (GAT)** `[research paper] <https://arxiv.org/abs/1710.10903>`__ `[tutorial]
  <1_gnn/9_gat.html>`__ `[Pytorch code]
  <https://github.com/dmlc/dgl/blob/master/examples/pytorch/gat>`__
  `[MXNet code]
  <https://github.com/dmlc/dgl/tree/master/examples/mxnet/gat>`__:
  GAT extends the GCN functionality by deploying multi-head attention
  among neighborhood of a node. This greatly enhances the capacity and
  expressiveness of the model.

* **Relational-GCN** `[research paper] <https://arxiv.org/abs/1703.06103>`__ `[tutorial]
  <1_gnn/4_rgcn.html>`__ `[Pytorch code]
  <https://github.com/dmlc/dgl/tree/master/examples/pytorch/rgcn>`__
  `[MXNet code]
  <https://github.com/dmlc/dgl/tree/master/examples/mxnet/rgcn>`__:
  Relational-GCN allows multiple edges among two entities of a
  graph. Edges with distinct relationships are encoded differently. 

* **Line graph neural network (LGNN)** `[research paper] <https://openreview.net/pdf?id=H1g0Z3A9Fm>`__ `[tutorial]
  <1_gnn/6_line_graph.html>`__ `[Pytorch code]
  <https://github.com/dmlc/dgl/tree/master/examples/pytorch/line_graph>`__:
  This network focuses on community detection by inspecting graph structures. It
  uses representations of both the original graph and its line-graph
  companion. In addition to demonstrating how an algorithm can harness multiple
  graphs, this implementation shows how you can judiciously mix simple tensor
  operations and sparse-matrix tensor operations, along with message-passing with
  DGL.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Author: Qi Huang, Minjie Wang , Yu Gai, Quan Gan, Zheng Zhang">

.. only:: html

  .. image:: /tutorials/models/1_gnn/images/thumb/sphx_glr_1_gcn_thumb.png
    :alt:

  :ref:`sphx_glr_tutorials_models_1_gnn_1_gcn.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Graph Convolutional Network</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Author: Lingfan Yu, Mufei Li, Zheng Zhang">

.. only:: html

  .. image:: /tutorials/models/1_gnn/images/thumb/sphx_glr_4_rgcn_thumb.png
    :alt:

  :ref:`sphx_glr_tutorials_models_1_gnn_4_rgcn.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Relational Graph Convolutional Network</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Author: Qi Huang, Yu Gai, Minjie Wang, Zheng Zhang">

.. only:: html

  .. image:: /tutorials/models/1_gnn/images/thumb/sphx_glr_6_line_graph_thumb.png
    :alt:

  :ref:`sphx_glr_tutorials_models_1_gnn_6_line_graph.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Line Graph Neural Network</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Authors: Hao Zhang, Mufei Li, Minjie Wang  Zheng Zhang">

.. only:: html

  .. image:: /tutorials/models/1_gnn/images/thumb/sphx_glr_9_gat_thumb.png
    :alt:

  :ref:`sphx_glr_tutorials_models_1_gnn_9_gat.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Understand Graph Attention Network</div>
    </div>


.. raw:: html

    </div>


Batching many small graphs
-------------------------------

* **Tree-LSTM** `[paper] <https://arxiv.org/abs/1503.00075>`__ `[tutorial]
  <2_small_graph/3_tree-lstm.html>`__ `[PyTorch code]
  <https://github.com/dmlc/dgl/blob/master/examples/pytorch/tree_lstm>`__:
  Sentences have inherent structures that are thrown
  away by treating them simply as sequences. Tree-LSTM is a powerful model
  that learns the representation by using prior syntactic structures such as a parse-tree.
  The challenge in training is that simply by padding
  a sentence to the maximum length no longer works. Trees of different
  sentences have different sizes and topologies. DGL solves this problem by
  adding the trees to a bigger container graph, and then using message-passing
  to explore maximum parallelism. Batching is a key API for this.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Author: Zihao Ye, Qipeng Guo, Minjie Wang, Jake Zhao, Zheng Zhang">

.. only:: html

  .. image:: /tutorials/models/2_small_graph/images/thumb/sphx_glr_3_tree-lstm_thumb.png
    :alt:

  :ref:`sphx_glr_tutorials_models_2_small_graph_3_tree-lstm.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Tree-LSTM in DGL</div>
    </div>


.. raw:: html

    </div>


Generative models
--------------------

* **DGMG** `[paper] <https://arxiv.org/abs/1803.03324>`__ `[tutorial]
  <3_generative_model/5_dgmg.html>`__ `[PyTorch code]
  <https://github.com/dmlc/dgl/tree/master/examples/pytorch/dgmg>`__:
  This model belongs to the family that deals with structural
  generation. Deep generative models of graphs (DGMG) uses a state-machine approach. 
  It is also very challenging because, unlike Tree-LSTM, every
  sample has a dynamic, probability-driven structure that is not available
  before training. You can progressively leverage intra- and
  inter-graph parallelism to steadily improve the performance.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Author: Mufei Li, Lingfan Yu, Zheng Zhang">

.. only:: html

  .. image:: /tutorials/models/3_generative_model/images/thumb/sphx_glr_5_dgmg_thumb.png
    :alt:

  :ref:`sphx_glr_tutorials_models_3_generative_model_5_dgmg.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Generative Models of Graphs</div>
    </div>


.. raw:: html

    </div>



Revisit classic models from a graph perspective
-------------------------------------------------------

* **Capsule** `[paper] <https://arxiv.org/abs/1710.09829>`__ `[tutorial]
  <4_old_wines/2_capsule.html>`__ `[PyTorch code]
  <https://github.com/dmlc/dgl/tree/master/examples/pytorch/capsule>`__:
  This new computer vision model has two key ideas. First, enhancing the feature
  representation in a vector form (instead of a scalar) called *capsule*. Second,
  replacing max-pooling with dynamic routing. The idea of dynamic routing is to
  integrate a lower level capsule to one or several higher level capsules
  with non-parametric message-passing. A tutorial shows how the latter can be 
  implemented with DGL APIs.


* **Transformer** `[paper] <https://arxiv.org/abs/1706.03762>`__ `[tutorial] <4_old_wines/7_transformer.html>`__ 
  `[PyTorch code] <https://github.com/dmlc/dgl/tree/master/examples/pytorch/transformer>`__ and **Universal Transformer** 
  `[paper] <https://arxiv.org/abs/1807.03819>`__ `[tutorial] <4_old_wines/7_transformer.html>`__
  `[PyTorch code] <https://github.com/dmlc/dgl/tree/master/examples/pytorch/transformer/modules/act.py>`__:
  These two models replace recurrent neural networks (RNNs) with several layers of multi-head attention to
  encode and discover structures among tokens of a sentence. These attention
  mechanisms are similarly formulated as graph operations with message-passing.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Author: Jinjing Zhou, Jake Zhao, Zheng Zhang, Jinyang Li">

.. only:: html

  .. image:: /tutorials/models/4_old_wines/images/thumb/sphx_glr_2_capsule_thumb.png
    :alt:

  :ref:`sphx_glr_tutorials_models_4_old_wines_2_capsule.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Capsule Network</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Author: Zihao Ye, Jinjing Zhou, Qipeng Guo, Quan Gan, Zheng Zhang">

.. only:: html

  .. image:: /tutorials/models/4_old_wines/images/thumb/sphx_glr_7_transformer_thumb.png
    :alt:

  :ref:`sphx_glr_tutorials_models_4_old_wines_7_transformer.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Transformer as a Graph Neural Network</div>
    </div>


.. raw:: html

    </div>


.. toctree::
   :hidden:
   :includehidden:


   /tutorials/models//1_gnn/index.rst
   /tutorials/models//2_small_graph/index.rst
   /tutorials/models//3_generative_model/index.rst
   /tutorials/models//4_old_wines/index.rst



.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_