About incremental update of graphs and Dynamic Distributed Partitioning

In some rigid scenarios such as social networks, as the dynamics of personnel change, the graph needs to rebuild the relationship in real time, and we wish some incremental update of graphs without performance loss. Noting that this doe not only relate to modify node/edge properties. Virtually, this also belongs to Dynamic Distributed Partitioning Of Graph.

Just as I have mentioned in the WeChat, this is the most difficult thing in dynamic graph computing field. We have found some academic researches about Graph Partitioning Algorithms [1][2]
[1] Hermes: Dynamic Partitioning for Distributed Social Network Graph Databases

[2] Incrementalization of Graph Partitioning Algorithms

This is not an easy thing landing industry-level graph db. But once it hits the ground or starts continuous improvement around the topic, I believe that Nebula will stand out in this market.


Re-balancing the data, especially an OLTP database’s storage requires heavy disk/network i/o’s, which, definitely, hurts the ad hoc query performance. And the OLTP performance is our main target right now.

So far, Nebula’s graph partitioning algorithm is simple: verter_id module partition_num. And you can find it here

Again, thank you for your interesting suggestions. I’ll dig into them later.

1 Like

OK,looking forward to deep discussion.