Graph Store – Graph for multi-processing and distributed training

class dgl.contrib.graph_store.SharedMemoryDGLGraph(graph_name, port)[source]

Shared-memory DGLGraph.

This is a client to access data in the shared-memory graph store that has loads the graph structure and node embeddings and edge embeddings to shared memory. It provides the DGLGraph interface.

Parameters:
  • graph_name (string) – Define the name of the graph.
  • port (int) – The port that the server listens to.

Querying the distributed setting

SharedMemoryDGLGraph.num_workers The number of workers using the graph store.
SharedMemoryDGLGraph.worker_id The id of the current worker using the graph store.
SharedMemoryDGLGraph.destroy() Destroy the graph store.

Using Node/edge features

SharedMemoryDGLGraph.init_ndata(ndata_name, …) Create node embedding.
SharedMemoryDGLGraph.init_edata(edata_name, …) Create edge embedding.

Computing with Graph store

SharedMemoryDGLGraph.apply_nodes([func, v, …]) Apply the function on the nodes to update their features.
SharedMemoryDGLGraph.apply_edges([func, …]) Apply the function on the edges to update their features.
SharedMemoryDGLGraph.group_apply_edges(…) Group the edges by nodes and apply the function on the grouped edges to
SharedMemoryDGLGraph.recv([v, reduce_func, …]) Receive and reduce incoming messages and update the features of node(s) \(v\).
SharedMemoryDGLGraph.send_and_recv(edges[, …]) Send messages along edges and let destinations receive them.
SharedMemoryDGLGraph.pull(v[, message_func, …]) Pull messages from the node(s)’ predecessors and then update their features.
SharedMemoryDGLGraph.push(u[, message_func, …]) Send message from the node(s) to their successors and update them.
SharedMemoryDGLGraph.update_all([…]) Distribute the computation in update_all among all pre-defined workers.

Construct a graph store

dgl.contrib.graph_store.create_graph_store_server(graph_data, graph_name, store_type, num_workers, multigraph=False, edge_dir='in', port=8000)[source]

Create the graph store server.

The server loads graph structure and node embeddings and edge embeddings.

Currently, only shared-memory graph store server is supported, so store_type can only be “shared_mem”.

After the server runs, the graph store clients can access the graph data with the specified graph name.

DGL graph accepts graph data of multiple formats:

  • NetworkX graph,
  • scipy matrix,
  • DGLGraph.

If the input graph data is DGLGraph, the constructed DGLGraph only contains its graph index.

Parameters:
  • graph_data (graph data) – Data to initialize graph.
  • graph_name (string) – Define the name of the graph.
  • store_type (string) – The type of the graph store. The current option is “shared_mem”.
  • num_workers (int) – The number of workers that will connect to the server.
  • multigraph (bool, optional) – Whether the graph would be a multigraph (default: False)
  • edge_dir (string) – the edge direction for the graph structure. The supported option is “in” and “out”.
  • port (int) – The port that the server listens to.
Returns:

The graph store server
Return type:

SharedMemoryStoreServer
dgl.contrib.graph_store.create_graph_from_store(graph_name, store_type, port=8000)[source]

Create a client from the graph store.

The client constructs the graph structure and node embeddings and edge embeddings that has been loaded by the graph store server.

Currently, only shared-memory graph store server is supported, so store_type can only be “shared_memory”.

Parameters:
  • graph_name (string) – Define the name of the graph.
  • store_type (string) – The type of the graph store. The current option is “shared_mem”.
  • port (int) – The port that the server listens to.
Returns:

The shared-memory DGLGraph
Return type:

SharedMemoryDGLGraph