Help Center> RES> API Reference> API> Data Source> Querying the Details of a Data Source

Querying the Details of a Data Source

Function

This API is used to query the details about a specified data source.

URI

GET /v2.0/{project_id}/workspaces/{workspace_id}/data-sources/{datasource_id}

Table 1 Path parameters

Parameter

Mandatory

Type

Description

datasource_id

Yes

String

Data source ID

project_id

Yes

String

Project ID. For details on how to obtain the project ID, see Obtaining a Project ID.

workspace_id

Yes

String

Workspace ID

Request Parameters

Table 2 Request header parameters

Parameter

Mandatory

Type

Description

Content-Type

Yes

String

Content type. The value is application/json.

X-Auth-Token

Yes

String

User token. For details on how to obtain the user token, see Obtaining a User Token Through Password Authentication.

Response Parameters

Status code: 200

Table 3 Response body parameters

Parameter

Type

Description

datasource

datasources object

Data source details

jobs

Array of jobs objects

Data source task details

is_success

Boolean

Whether the request is successful

message

String

Response message (This field is not returned when the request is successful.)

error_code

String

Error code (This parameter is not returned when the request is successful.)

Table 4 datasources

Parameter

Type

Description

datasource_id

String

Data source ID

datasource_name

String

Name

status

String

Status

structure

String

Structure

workspace_id

String

Workspace ID

data_config

DataConfig object

Data settings

specs_config

SpecsConfig object

Computing specifications settings

created_at

String

Creation time

update_at

Long

Update time

Table 5 DataConfig

Parameter

Type

Description

offline

Offline object

Offline computing specifications

nearline

Nearline object

Nearline data source

Table 6 Offline

Parameter

Type

Description

user_url

String

User data URL

item_url

String

Item data URL

behavior_url

String

Behavior data URL

Table 7 Nearline

Parameter

Type

Description

item_topic

Topic object

Item topic

user_topic

Topic object

User topic

behavior_topic

Topic object

Behavior topic

Table 8 Topic

Parameter

Type

Description

name

String

Name

id

String

ID

Table 9 SpecsConfig

Parameter

Type

Description

offline

String

Offline computing specifications

nearline

String

Real-time computing specifications

rank

String

Deep learning computing specifications

online_tps

Integer

Max. number of concurrent online services

Table 10 jobs

Parameter

Type

Description

category

String

Category

config_info

String

Configuration information

description

String

Description

job_id

String

Job ID

job_name

String

Job name

job_type

String

Job type

next_schedule_time

Integer

Next scheduling time

platform

String

Platform

resource_id

String

Resource ID

schedule

String

Scheduling parameter

status

String

Status

workspace_id

String

Workspace ID

job_config

jobConfig object

Job settings

Table 11 jobConfig

Parameter

Type

Description

nearline_recall_param

NearLineRecallParam object

Parameters of a nearline retrieval job (required for nearline retrieval jobs)

max_recommended_num

Integer

Max. number of candidate sets (required for retrieval jobs)

Minimum: 1

Maximum: 200

match_feature_pairs

Array of MatchFeaturePair objects

Matched feature pair (required for feature matching-based retrieval jobs)

striping

Striping object

Row-based strategy (required for feature matching-based retrieval job, itemCF retrieval job, and userCF retrieval job)

match_type

String

Matching type (required for feature matching-based retrieval jobs). The options are:

  • UI: recommending items based on users

  • UU: recommending users based on users

  • II: recommending items based on items

  • IU: recommending users based on items

matrix_factorization

MatrixFactorization object

Matrix decomposition parameter settings (required for the ALS-based MF jobs)

behavior_frequencys

Array of BehaviorFrequency objects

Behavior frequency information (required for historical behavior-based candidate set generation jobs)

file_path

String

File path (required for business rule - manual import retrieval jobs)

ucb_param

UcbParam object

UCB job parameter (required for UCB-based retrieval jobs)

behavior_gravity

BehaviorGravity object

Gravity decay factor (required for the comprehensive behavior popularity-based retrieval jobs)

category

Category object

Type (required for the comprehensive behavior popularity-based retrieval jobs)

behavior_logic

String

Behavior filter logic (required for historical behavior filter jobs). The options are:

  • AND: If both conditions are met, the data is filtered.

  • OR: If one condition is met, the data is filtered.

features_engineering

EtlBasicParameter object

Feature parameter (required for offline feature engineering jobs)

sample_param

SampleParam object

Sample parameter (required for offline feature engineering jobs)

deep_learning_parameters

DeepLearingParam object

General parameters of a ranking job (required by LR, DeepFM, and AutoGroup)

algorithm_specify_parameters

AlgorithmSpecifyParameters object

Specific parameter of a ranking algorithm (required for LR, DeepFM, and AutoGroup)

load_widetable

Boolean

Importing a wide table (required for offline data import jobs)

load_profile

Boolean

Importing a profile (required for offline data import jobs)

save_mode

String

Retaining a wide table (required for offline data import jobs). The options are:

  • append: Yes

  • new: No -Overwrite: Overwrite

indicators

Array of Indicator objects

Statistical indicator (required for effect evaluation jobs)

offline_rank_job_name

String

Name of an offline ranking job (required for online training jobs)

update_interval

Integer

Update interval (required for online training jobs)

optimizer

Optimizer object

Optimizer (required for online training jobs)

flows

Flow object

Online process flow (required for online training jobs)

Table 12 NearLineRecallParam

Parameter

Type

Description

time_limit

Boolean

Time filter

timeFeature

String

Time feature

retainDays

Integer

Retention period (days)

recall_fileds

Array of RecallFiled objects

Retrieved field

itemCF_job_name

String

Name of an itemCF job

Table 13 RecallFiled

Parameter

Type

Description

name

String

Field name

value

Integer

Number of used field values

Minimum: 1

Maximum: 10

Default: 1

Table 14 MatchFeaturePair

Parameter

Type

Description

user_feature_name

String

User feature

item_feature_name

String

Item feature

weight

Double

Weight

match_count

Boolean

Measurement of the number of matched tags

Table 15 Striping

Parameter

Type

Description

nearest_neighborhood

Integer

Nearest neighbors

band

Integer

Similarity degree

Minimum: 1

Maximum: 20

row

Integer

Similarity distance

Minimum: 1

Maximum: 10

Table 16 MatrixFactorization

Parameter

Type

Description

implicit_vector_rank

Integer

Embedding size

Minimum: 1

Maximum: 100

regular_param

Double

Optimization lambda

Minimum: 1.0E-8

Maximum: 1

max_iterator_num

Integer

Number of iterations

Minimum: 1

Maximum: 50

Table 17 BehaviorFrequency

Parameter

Type

Description

behavior_type

String

Behavior type. The options are:

  • view indicates that you browsed an item.

  • click indicates that you clicked an item.

  • collect indicates that you added an item to favorites.

  • uncollect indicates that you removed an item from favorites.

  • search_click indicates that you searched and clicked an item.

  • comment indicates that you made comments on an item.

  • share indicates that you shared an item with others.

  • like indicates that you gave an item a thumb-up.

  • dislike indicates that you gave an item a thumb-down.

  • grade indicates that you rated an item.

  • consume indicates that you bought an item (primarily refers to goods).

  • use indicates that you watched videos/listened to music/read books.

  • download indicates that you downloaded something. -tip indicates that you gave somebody or something a reward.

  • subscribe indicates that you followed somebody or something.

lower_limit

Integer

Min. times

Minimum: 1

upper_limit

Integer

Max. times

Minimum: 1

time_interval

Integer

Time range

Minimum: 1

Table 18 UcbParam

Parameter

Type

Description

alpha

Double

Tradeoff parameter

Minimum: 0

Maximum: 1

min_used_num

Integer

Min. number of behaviors

Minimum: 30

Maximum: 1000

Table 19 BehaviorGravity

Parameter

Type

Description

weaken_factor

Double

Decay factor

Minimum: 0.1

Maximum: 5

view_type

String

Behavior quantity counting mode. The options are:

  • pv: page views or clicks, measuring the number of web pages visited by website users.

  • uv: unique visitors, counting the number of individual users accessing a site within the reporting period.

algo_type

String

Algorithm type. The options are:

  • normal

  • time

Table 20 Category

Parameter

Type

Description

user_meta_list

Array of strings

User feature

item_meta_list

Array of strings

Item feature

Table 21 EtlBasicParameter

Parameter

Type

Description

user_features

Array of FeatureTransformation objects

User feature

item_features

Array of FeatureTransformation objects

Item feature

rank_etl_filter

RankETLFilter object

Filter parameter

Table 22 FeatureTransformation

Parameter

Type

Description

attr

Attribute object

Feature

discrete_method

String

Discrete method. The options are:

  • equal_distance_discrete indicates equal-distance discretization.

  • user_define_discrete indicates custom discrete distance. -normalize indicates normalization.

  • null indicates non-discretization.

params

Object

Specific processing parameter

Table 23 Attribute

Parameter

Type

Description

name

String

Name

data_type

String

Data type

other_uses

Array of strings

Other usage

Table 24 RankETLFilter

Parameter

Type

Description

filter_type

String

Behavior deduplication mode. The options are:

  • abs_weight indicates the absolute weight value.

  • date indicates the date.

time_type

String

Time type. The options are: Day Week Month

is_monday_first

Boolean

Whether Monday is the first day

Table 25 SampleParam

Parameter

Type

Description

divide_type

String

Division mode of training and test sets. The options are:

  • TIME indicates the time ratio.

  • RAMDOM indicates the number ratio.

train_rate

Double

Training data ratio

Minimum: 0.01

Maximum: 1

test_rate

Double

Test data ratio

Minimum: 0.01

Maximum: 1

Table 26 DeepLearingParam

Parameter

Type

Description

initial_parameters

Initial object

Initialization parameter

optimize_parameters

Optimizer object

Optimization parameter

regular_parameters

Regular object

Lambda parameter

max_iterations

Integer

Max. iterations

Minimum: 1

Maximum: 1000

early_stop_iterations

Integer

Iterations at early stopping

Minimum: 1

Maximum: 1000

batch_size

Integer

Batch size

Minimum: 1

dataset_split_parts

Integer

Number of training datasets to be split

Minimum: 1

Maximum: 10

restart_train

Boolean

Retraining

Table 27 Initial

Parameter

Type

Description

initial_method

String

Initialization method

Enumeration values:

  • normal

  • uniform

  • xavier

mean_value

Double

Mean

Minimum: -1

Maximum: 1

standard_deviation

Double

Standard deviation

Minimum: 0

Maximum: 1

min_value

Double

Min. value

Minimum: -1

Maximum: 0

max_value

Double

Max. value

Minimum: 0

Maximum: 1

Table 28 Regular

Parameter

Type

Description

l2_regularization

Double

Lambda 2

Minimum: 0

Maximum: 1

regular_loss_compute_mode

String

Regular loss calculation mode

Enumeration values:

  • full

  • batch

embed_l2_regularization

Double

Lambda 2 of embedding size

Minimum: 0

Maximum: 1

wide_l2_regularization

Double

Lambda 2 of the wide part

Minimum: 0

Maximum: 1

structure_l2_regularization

Double

Lambda 2 of the structured part

Minimum: 0

Maximum: 1

Table 29 AlgorithmSpecifyParameters

Parameter

Type

Description

latent_vector_length

Integer

Embedding size (required for DeepFM)

Minimum: 1

Maximum: 100

architecture

Array of integers

Neural network structure (required for DeepFM)

active_function

String

Activation function (required for DeepFM and AutoGroup)

Enumeration values:

  • relu

  • sigmoid

  • tanh

value_keep_probability

Double

Neuron retention probability (required for DeepFM and AutoGroup)

Minimum: 0

Maximum: 1

embed_size

Array of integers

Embedding size of each degree (required for AutoGroup)

mlp_architecture

Array of integers

Neural network structure (required for AutoGroup)

max_order

Integer

Max. interactions (required for AutoGroup)

hash_sizes

Array of integers

Hash length (required for AutoGroup)

hash_compensation

Array of numbers

Feature interaction penalty coefficient (required for AutoGroup)

use_wide_part

Boolean

Wide part required (required for AutoGroup)

structure_optimizer

Optimizer object

Optimizer parameter (required for AutoGroup)

merge_multi_hot

Boolean

Merge multi-value feature (required for AutoGroup)

fix_structure

Boolean

Fix hash structure (required for AutoGroup)

Table 30 Indicator

Parameter

Type

Description

indicator_name

String

Indicator. The options are: PV UV Custom

indicator_params

IndicatorParam object

Indicator parameter (required for custom metric)

Table 31 IndicatorParam

Parameter

Type

Description

customize_parameter

CustomizeParameter object

Custom parameter

customize_formula

CustomizeFormula object

Custom formula

Table 32 CustomizeParameter

Parameter

Type

Description

alias

String

Alias

behavior_type

String

Behavior type

threshold

Double

Threshold

Minimum: 0

Maximum: 1

deduplication

String

Deduplication

Table 33 CustomizeFormula

Parameter

Type

Description

alias

String

Alias

formula

String

Formula

Table 34 Optimizer

Parameter

Type

Description

type

String

Optimizer type

Enumeration values:

  • adam

  • adagrad

  • ftrl

learning_rate

Integer

Learning rate

Minimum: 0

Maximum: 1

initial_accumulator_value

Double

Initial gradient sum

Minimum: 0

Maximum: 1

lambda1

Double

Lambda 1

Minimum: 0

Maximum: 1

lambda2

Double

Lambda 2

Minimum: 0

Maximum: 1

epsilon

Double

Epsilon

Minimum: 0

Maximum: 1

decay_rate

Double

Decay factor

Minimum: 0

Maximum: 1

decay_steps

Double

Decay step

Minimum: 1

Table 35 Flow

Parameter

Type

Description

flow_id

String

Process flow ID

attr_pair_rules_filter

Array of AttrPairRules objects

Feature pair filter

attr_pair_rules_reserve

Array of AttrPairRules objects

Feature pair to be reserved

deduplication_list

Array of Deduplication objects

Feature deduplication

attribute_info

AttributeInfo object

Comprehensive ranking information

bloom_filter_conf

BloomFilterConf object

Bloom filter settings

group_attr

String

The scatter attribute for grouping

pre_deal

Boolean

Deduplication before ranking

rank_setting

String

Ranking configuration information

rules

Rule object

Candidate set merging

filter_sets

Array of strings

Filter configuration information

attr_value_rules_filter

Array of AttrValueRules objects

Feature filter

attr_value_rules_reserve

Array of AttrValueRules objects

Feature to be reserved

ctr_job

String

Ranking job (required when click-through rate is used)

ratio

Integer

Traffic proportion

Minimum: 1

Maximum: 100

toppings

Array of strings

List of candidate sets to be pinned on top

Table 36 AttrPairRules

Parameter

Type

Description

attr_pairs

Array of AttrPair objects

Feature pair

Table 37 AttrPair

Parameter

Type

Description

party_a

String

Feature name of the recommended item

party_b

String

Feature name of the recommended item

Table 38 Deduplication

Parameter

Type

Description

attributes

Array of strings

Feature

Table 39 AttributeInfo

Parameter

Type

Description

rank_feature_pairs

Array of RankFeaturePair objects

Matched feature pair

numerical_attrs

Array of NumericalAttr objects

Feature weight

num_statistics_type

String

Statistics mode. The options are:

  • ORDER indicates that weights are accumulated based on the feature value sequence.

  • ABS indicates that weights are accumulated based on the absolute value.

Enumeration values:

  • ORDER

  • ABS

Table 40 RankFeaturePair

Parameter

Type

Description

feature_name_a

String

Feature of the item to be recommended

feature_name_b

String

Feature of the recommended item

weight

Float

Weight

Minimum: 0.01

Maximum: 1

Table 41 NumericalAttr

Parameter

Type

Description

name

String

Feature name

weight

Float

Weight

Minimum: 0.001

Maximum: 1

Table 42 BloomFilterConf

Parameter

Type

Description

behaviors

Array of strings

Type of the behavior to be filtered

interval

Integer

Filter time

Minimum: 1

Maximum: 7

Table 43 Rule

Parameter

Type

Description

table_name

String

Table name of a candidate set

rule_ratio

Integer

Rule ratio

Minimum: 1

Maximum: 100

priority

Integer

Priority

Minimum: 1

Maximum: 10

Table 44 AttrValueRules

Parameter

Type

Description

attr_values_a

Array of AttrValue objects

Feature value settings of the recommended item

attr_values_b

Array of AttrValue objects

Feature value settings of the item to be recommended

Table 45 AttrValue

Parameter

Type

Description

name

String

Feature name

value

String

Feature value

Example Requests

This API is used to query the details about a specified data source.

/v2.0/testuuidxxxxxxxxxxxxxxxxxxxxxxxx/workspaces/testuuidxxxxxxxxxxxxxxxxxxxxxxxx/data-sources/testuuidxxxxxxxxxxxxxxxxxxxxxxxx

-

Example Responses

Status code: 200

OK

{
  "is_success" : true,
  "datasource" : {
    "specs_config" : {
      "offline" : "4CU"
    },
    "data_config" : {
      "offline" : {
        "user_url" : "<OBS path for storing user data>",
        "item_url" : "<OBS path for storing item data>",
        "behavior_url" : "<OBS path for storing behavior data>"
      },
      "nearline" : { }
    },
    "datasource_name" : "test-cyz",
    "workspace_id" : "06a7c49afc00d4972ff1c0113d8c49ba",
    "datasource_id" : "aa6d657026de42a2927a50400ae69a4b",
    "status" : "Succeed",
    "created_at" : 1587953962684,
    "update_at" : 1587953962684,
    "structure" : "{\"user_attrs\":[{\"name\":\"age\",\"data_type\":\"long\"},{\"name\":\"extend_float\",\"data_type\":\"float\"},{\"name\":\"extend_long\",\"data_type\":\"long\"},{\"name\":\"extend_strArray\",\"data_type\":\"strArray\"},{\"name\":\"extend_string\",\"data_type\":\"string\"},{\"name\":\"gender\",\"data_type\":\"string\"},{\"name\":\"location\",\"data_type\":\"location\"},{\"name\":\"tags\",\"data_type\":\"strArray\"},{\"name\":\"userId\",\"data_type\":\"string\"}],\"item_attrs\":[{\"name\":\"author\",\"data_type\":\"strArray\"},{\"name\":\"category\",\"data_type\":\"string\"},{\"name\":\"expireTime\",\"data_type\":\"long\"},{\"name\":\"extend_float\",\"data_type\":\"float\"},{\"name\":\"extend_long\",\"data_type\":\"long\"},{\"name\":\"extend_strArray\",\"data_type\":\"strArray\"},{\"name\":\"extend_string\",\"data_type\":\"string\"},{\"name\":\"itemId\",\"data_type\":\"string\"},{\"name\":\"itemType\",\"data_type\":\"string\"},{\"name\":\"location\",\"data_type\":\"location\"},{\"name\":\"publishTime\",\"data_type\":\"long\"},{\"name\":\"score\",\"data_type\":\"long\"},{\"name\":\"status\",\"data_type\":\"long\"},{\"name\":\"tags\",\"data_type\":\"strArray\"}],\"behaviors\":{\"positive_behaviors\":[{\"behavior_type\":\"click\",\"weight\":0.3},{\"behavior_type\":\"like\",\"weight\":0.8}],\"negative_behaviors\":[{\"behavior_type\":\"view\",\"weight\":0.1},{\"behavior_type\":\"dislike\",\"weight\":0.8}]}}"
  },
  "jobs" : [ {
    "category" : "DATASOURCE",
    "job_config" : {
      "save_mode" : "new",
      "load_profile" : true,
      "load_widetable" : true
    },
    "job_name" : "res-data-import",
    "job_id" : "3ccd65777a72477ca3c819adfd8db94e",
    "job_type" : "DataImport",
    "status" : "Succeed",
    "platform" : "OFFLINE",
    "workspace_id" : "06a7c49afc00d4972ff1c0113d8c49ba",
    "resource_id" : "aa6d657026de42a2927a50400ae69a4b"
  }, {
    "category" : "DATASOURCE",
    "job_config" : { },
    "job_name" : "res-data-exploration",
    "job_id" : "a02af2280fae4c2189e141679c6bc91f",
    "job_type" : "DataExploration",
    "status" : "Succeed",
    "platform" : "OFFLINE",
    "workspace_id" : "06a7c49afc00d4972ff1c0113d8c49ba",
    "resource_id" : "aa6d657026de42a2927a50400ae69a4b"
  }, {
    "category" : "DATASOURCE",
    "job_config" : { },
    "job_name" : "res-data-struct",
    "job_id" : "e5d9921aabea48c49164722f43b4ccb9",
    "job_type" : "DataStruct",
    "status" : "Succeed",
    "platform" : "OFFLINE",
    "workspace_id" : "06a7c49afc00d4972ff1c0113d8c49ba",
    "resource_id" : "aa6d657026de42a2927a50400ae69a4b"
  } ]
}

Status Codes

Status Code

Description

200

OK

Error Codes

See Error Codes.