Importing Incremental Data

Scenario

After you create a graph, you need to import graph data. If you need to add new graph data, you can import data to the graph.

• Currently, only graphs of version 1.1.8 and later support this function.
• To prevent failures in restoring the imported graph data during system restart, do not delete the data stored on OBS when the graph is in use.
• The default separator of data columns is comma (,). You cannot define a separator.
• The size of a single file in the import directory or the size of a single file to be imported cannot exceed 5 GB. Or the import will fail. You are advised to split the file into multiple files smaller than 5 GB before importing.
• The total size of files imported at once (including vertex and edge datasets) cannot exceed 1/5 of the available memory. For details about the available memory, check the Node Monitoring area on the O&M monitoring dashboard for the minimum value of available memory for nodes with the suffix ges-dn-1-1 and ges-dn-2-1 (hover over the memory usage rate).

Procedure

1. Log in to the GES management console. In the navigation pane on the left, choose Graph Management.
2. In the graph list, locate the target graph, click More in the Operation column, and select Import.
   Figure 1 Importing data
   

3. In the displayed Import dialog box, set the following parameters:
   • Graph Cluster (only available for database edition graphs): When a database edition graph is created, it is automatically upgraded to a multi-graph cluster. Such a cluster can contain multiple graph instances. For details, see Multi-Graph Management (Database Edition).
   • Metadata: Select an existing metadata file or create one. For details, see Creating a Metadata File.
   • Edge Data: Select the corresponding edge data set.
   • Vertex Data: Select the corresponding vertex data set. If you leave it blank, the vertices in the Edge Data set are used as the source of Vertex Data.
   • Log Storage Path: Stores vertex and edge data sets that do not comply with the metadata definition, as well as detailed logs generated during graph import. Storage on OBS may incur fees, so delete the data in time.
   • SortKey Included in Edge File (only available for database edition graphs): Different SortKey values can be configured to distinguish duplicate edges (edges with the same source vertex, end vertex, and label).
   • Edge Processing: Includes Allow repetitive edges, Ignore subsequent repetitive edges, Overwrite previous repetitive edges, and Ignore labels on repetitive edges.
     Edge Processing: Repetitive edges have the same source and target vertices. When labels are considered, repetitive edges must have the same source and target vertices and the same labels.

     • Allow repetitive edges: Multiple edges may exist between a source vertex and a target vertex.
     • Ignore subsequent repetitive edges: If there are multiple edges between a source vertex and a target vertex, only the first edge read is retained.
     • Overwrite previous repetitive edges: If there are multiple edges between a source vertex and a target vertex, only the last edge read is retained.
     • Ignore labels on repetitive edges: If labels are ignored, edges with the same source vertex and target vertex are repetitive edges.
   • Import Type: The value can be Online import or Offline import.
   

   • Graphs of the database edition support multi-graph management, and you need to select a graph name. Import Type is not supported.
   • The edge and vertex data sets can only be stored in English paths and folders.
   • Currently, you can import the edge and vertex data sets only from OBS. You need to store data files in an OBS bucket..
   • The sequence of the properties and labels in the selected edge or vertex data set must be the same as the sequence in the selected metadata file. Otherwise, The edge/vertex data file does not match the metadata file is displayed in the upper right corner and the graph fails to be created. For details about the graph data format, refer to Graph Data Formats.
   • You need to import the graph data (including the metadata file, and edge and vertex data sets) in the format specified in the template. The template contains a copy of movie information. You can click Download to download and import it.

4. Click OK.