Performance Testing Comparison of Nebula Graph: v2.0.1 vs v1.2.0

Summary

The testing results are as follows:

  1. In a single-hop scenario, the throughput of Nebula Graph v2.0.1 is reduced by 10%-20%. For more information, see 5.3 Explanation.

  2. In a multi-hop scenario, Nebula Graph v2.0.1 outperforms Nebula Graph v1.2.0 in these aspects:

  3. In simple scenarios of querying destination vertices, the performance of both versions is basically the same.

  4. In other scenarios, the throughput of Nebula Graph v2.0.1 is increased by 35%-100% and its query latency is reduced by 10%-50%.

Let’s see how the testing is performed.

Testing Environment

Machine Specifications

Four physical machines, each with 48-Core CPU, 256 GB memory, 1.5 TB SSD, and 10 Gigabit network.

Deployment

Process Quantity Description
nebula-graphd 1 Single-node deployment
nebula-metad 3 Three-node hybrid deployment
nebula-storaged 3

Table 2-1 Deployment of Services

Graph Space Information

Graph Space Name Partition # Replica # VID Type
ldbc_snb_sf100_vid_int 24 1 INT64

Table 2-2 Graph Space for Nebula Graph v2.0.1

Graph Space Name Partition # Replica # VID Type
ldbc_snb_sf100 24 1 INT64

Table 2-3 Graph Space for Nebula Graph v1.2.0

Configuration Files

Storage:

  • --rocksdb_block_cache=81920
  • --max_handlers_per_req=1
  • --heartbeat_interval_secs=10

For other configurations, see the configuration files as follows for the production environment. They are under the conf directory.

  • nebula-graphd.conf.production
  • nebula-storaged.conf.production
  • nebula-metad.conf.production

Dataset for Testing

The LDBC 100G dataset is used for the testing.

Use Cases

Four use cases are used in this testing.

CASE 1

This use case aims to retrieve the IDs of the destination vertices at N hops. For example, GO {N} STEP FROM {vid} OVER knows .

CASE 2

This use case aims to retrieve the specified edge properties at N hops. For example, GO {N} STEP FROM {vid} OVER knows yield knows.time .

CASE 3

This use case aims to retrieve the specified properties of the destination vertices at N hops. For example, GO {N} STEP FROM {vid} OVER knows yield $.person.first_name .

CASE 4

This use case aims to retrieve the specified properties of edges and destination vertices and then return the first N number of ordered records. For example, GO {N} STEP FROM {vid} OVER knows YIELD DISTINCT knows.time as t, $.person.first_name, $.person.last_name, $.person.birthday as birth | order by $-.t,$-.birth | limit 10 .

Testing Results and Discussion

Testing Metrics

In the testing, throughput and query latency are the main testing metrics. The query latency on both the server side and the client side is evaluated by P99, P95, and P90.

Testing Results

CASE 1

The following figures show the results of GO {N} STEP FROM {vid} OVER knows when N is set to 1, 2, or 3.


Figure 5-1 Case 1: Throughput of Single-Hop Query


Figure 5-2 Case 1: Query Latency of Single-Hop Query on Server Side


Figure 5-3 Case 1: Query Latency of Single-Hop Query on Client Side


Figure 5-4 Case 1: Throughput of Two-Hop Query


Figure 5-5 Case 1: Query Latency of Two-Hop Query on Server Side


Figure 5-6 Case 1: Query Latency of Two-Hop Query on Client Side


Figure 5-7 Case 1: Throughput of Three-Hop Query


Figure 5-8 Case 1: Query Latency of Three-Hop Query on Server Side


Figure 5-9 Case 1: Query Latency of Three-Hop Query on Client Side

CASE 2

The following figures show the results of GO {N} STEP FROM {vid} OVER knows yield knows.time when N is set to 1, 2, or 3.


Figure 5-10 Case 2: Throughput of Single-Hop Query


Figure 5-11 Case 2: Query Latency of Single-Hop Query on Server Side


Figure 5-12 Case 2: Query Latency of Single-Hop Query on Client Side


Figure 5-13 Case 2: Throughput of Two-Hop Query


Figure 5-14 Case 2: Query Latency of Two-Hop Query on Server Side


Figure 5-15 Case 2: Query Latency of Two-Hop Query on Client Side


Figure 5-16 Case 2: Throughput of Three-Hop Query


Figure 5-17 Case 2: Query Latency of Three-Hop Query on Server Side


Figure 5-18 Case 2: Query Latency of Three-Hop Query on Client Side

CASE 3

The following figures show the results of GO {N} STEP FROM {vid} OVER knows yield $.person.first_name when N is set to 1, 2, or 3.


Figure 5-19 Case 3: Throughput of Single-Hop Query


Figure 5-20 Case 3: Query Latency of Single-Hop Query on Server Side


Figure 5-21 Case 3: Query Latency of Single-Hop Query on Client Side


Figure 5-22 Case 3: Throughput of Two-Hop Query


Figure 5-23 Case 3: Query Latency of Two-Hop Query on Server Side


Figure 5-24 Case 3: Query Latency of Two-Hop Query on Client Side


Figure 5-25 Case 3: Throughput of Three-Hop Query


Figure 5-26 Case 3: Query Latency of Three-Hop Query on Server Side


Figure 5-27 Case 3: Query Latency of Three-Hop Query on Client Side

CASE 4

The following figures show the results of GO {N} STEP FROM {vid} OVER knows YIELD DISTINCT knows.time as t, $.person.first_name, $.person.last_name, $.person.birthday as birth | order by $-.t,$-.birth | limit 10 when N is set to 1, 2, or 3.


Figure 5-28 Case 4: Throughput of Single-Hop Query


Figure 5-29 Case 4: Query Latency of Single-Hop Query on Server Side


Figure 5-30 Case 4: Query Latency of Single-Hop Query on Client Side


Figure 5-31 Case 4: Throughput of Two-Hop Query


Figure 5-32 Case 4: Query Latency of Two-Hop Query on Server Side


Figure 5-33 Case 4: Query Latency of Two-Hop Query on Client Side


Figure 5-34 Case 4: Throughput of Three-Hop Query


Figure 5-35 Case 4: Query Latency of Three-Hop Query on Server Side


Figure 5-36 Case 4: Query Latency of Three-Hop Query on Client Side

Explanation

Compared with v1.2.0 of Nebula Graph, its v2.0.1 version performs better on throughput in multi-hop scenarios and the query latency is reduced at the same time.

In single-hop scenarios, the throughput of Nebula Graph v2.0.1 is reduced by 10%-20% but the query latency is increased. It is mainly because in Nebula Graph v2.0.1, a query statement is divided into execution plans of multiple subtasks, which causes a sharp increase in the number of asynchronous tasks. Only small tasks are involved in single-hop scenarios, so in the case of massive throughput, tasks are assigned to a queue, which causes an increase of latency and a decrease of throughput compared with that of Nebula Graph v1.2.0.