Querying the Details of a Data Source
Function
This API is used to query the details about a specified data source.
URI
GET /v2.0/{project_id}/workspaces/{workspace_id}/data-sources/{datasource_id}
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| datasource_id | Yes | String | Data source ID |
| project_id | Yes | String | Project ID. For details on how to obtain the project ID, see Obtaining a Project ID. |
| workspace_id | Yes | String | Workspace ID |
Request Parameters
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| Content-Type | Yes | String | Content type. The value is application/json. |
| X-Auth-Token | Yes | String | User token. For details on how to obtain the user token, see Obtaining a User Token Through Password Authentication. |
Response Parameters
Status code: 200
| Parameter | Type | Description |
|---|---|---|
| datasource | datasources object | Data source details |
| jobs | Array of jobs objects | Data source task details |
| is_success | Boolean | Whether the request is successful |
| message | String | Response message (This field is not returned when the request is successful.) |
| error_code | String | Error code (This parameter is not returned when the request is successful.) |
| Parameter | Type | Description |
|---|---|---|
| datasource_id | String | Data source ID |
| datasource_name | String | Name |
| status | String | Status |
| structure | String | Structure |
| workspace_id | String | Workspace ID |
| data_config | DataConfig object | Data settings |
| specs_config | SpecsConfig object | Computing specifications settings |
| created_at | String | Creation time |
| update_at | Long | Update time |
| Parameter | Type | Description |
|---|---|---|
| offline | Offline object | Offline computing specifications |
| nearline | Nearline object | Nearline data source |
| Parameter | Type | Description |
|---|---|---|
| user_url | String | User data URL |
| item_url | String | Item data URL |
| behavior_url | String | Behavior data URL |
| Parameter | Type | Description |
|---|---|---|
| item_topic | Topic object | Item topic |
| user_topic | Topic object | User topic |
| behavior_topic | Topic object | Behavior topic |
| Parameter | Type | Description |
|---|---|---|
| offline | String | Offline computing specifications |
| nearline | String | Real-time computing specifications |
| rank | String | Deep learning computing specifications |
| online_tps | Integer | Max. number of concurrent online services |
| Parameter | Type | Description |
|---|---|---|
| category | String | Category |
| config_info | String | Configuration information |
| description | String | Description |
| job_id | String | Job ID |
| job_name | String | Job name |
| job_type | String | Job type |
| next_schedule_time | Integer | Next scheduling time |
| platform | String | Platform |
| resource_id | String | Resource ID |
| schedule | String | Scheduling parameter |
| status | String | Status |
| workspace_id | String | Workspace ID |
| job_config | jobConfig object | Job settings |
| Parameter | Type | Description |
|---|---|---|
| nearline_recall_param | NearLineRecallParam object | Parameters of a nearline retrieval job (required for nearline retrieval jobs) |
| max_recommended_num | Integer | Max. number of candidate sets (required for retrieval jobs) Minimum: 1 Maximum: 200 |
| match_feature_pairs | Array of MatchFeaturePair objects | Matched feature pair (required for feature matching-based retrieval jobs) |
| striping | Striping object | Row-based strategy (required for feature matching-based retrieval job, itemCF retrieval job, and userCF retrieval job) |
| match_type | String | Matching type (required for feature matching-based retrieval jobs). The options are:
|
| matrix_factorization | MatrixFactorization object | Matrix decomposition parameter settings (required for the ALS-based MF jobs) |
| behavior_frequencys | Array of BehaviorFrequency objects | Behavior frequency information (required for historical behavior-based candidate set generation jobs) |
| file_path | String | File path (required for business rule - manual import retrieval jobs) |
| ucb_param | UcbParam object | UCB job parameter (required for UCB-based retrieval jobs) |
| behavior_gravity | BehaviorGravity object | Gravity decay factor (required for the comprehensive behavior popularity-based retrieval jobs) |
| category | Category object | Type (required for the comprehensive behavior popularity-based retrieval jobs) |
| behavior_logic | String | Behavior filter logic (required for historical behavior filter jobs). The options are:
|
| features_engineering | EtlBasicParameter object | Feature parameter (required for offline feature engineering jobs) |
| sample_param | SampleParam object | Sample parameter (required for offline feature engineering jobs) |
| deep_learning_parameters | DeepLearingParam object | General parameters of a ranking job (required by LR, DeepFM, and AutoGroup) |
| algorithm_specify_parameters | AlgorithmSpecifyParameters object | Specific parameter of a ranking algorithm (required for LR, DeepFM, and AutoGroup) |
| load_widetable | Boolean | Importing a wide table (required for offline data import jobs) |
| load_profile | Boolean | Importing a profile (required for offline data import jobs) |
| save_mode | String | Retaining a wide table (required for offline data import jobs). The options are:
|
| indicators | Array of Indicator objects | Statistical indicator (required for effect evaluation jobs) |
| offline_rank_job_name | String | Name of an offline ranking job (required for online training jobs) |
| update_interval | Integer | Update interval (required for online training jobs) |
| optimizer | Optimizer object | Optimizer (required for online training jobs) |
| flows | Flow object | Online process flow (required for online training jobs) |
| Parameter | Type | Description |
|---|---|---|
| time_limit | Boolean | Time filter |
| timeFeature | String | Time feature |
| retainDays | Integer | Retention period (days) |
| recall_fileds | Array of RecallFiled objects | Retrieved field |
| itemCF_job_name | String | Name of an itemCF job |
| Parameter | Type | Description |
|---|---|---|
| name | String | Field name |
| value | Integer | Number of used field values Minimum: 1 Maximum: 10 Default: 1 |
| Parameter | Type | Description |
|---|---|---|
| user_feature_name | String | User feature |
| item_feature_name | String | Item feature |
| weight | Double | Weight |
| match_count | Boolean | Measurement of the number of matched tags |
| Parameter | Type | Description |
|---|---|---|
| nearest_neighborhood | Integer | Nearest neighbors |
| band | Integer | Similarity degree Minimum: 1 Maximum: 20 |
| row | Integer | Similarity distance Minimum: 1 Maximum: 10 |
| Parameter | Type | Description |
|---|---|---|
| implicit_vector_rank | Integer | Embedding size Minimum: 1 Maximum: 100 |
| regular_param | Double | Optimization lambda Minimum: 1.0E-8 Maximum: 1 |
| max_iterator_num | Integer | Number of iterations Minimum: 1 Maximum: 50 |
| Parameter | Type | Description |
|---|---|---|
| behavior_type | String | Behavior type. The options are:
|
| lower_limit | Integer | Min. times Minimum: 1 |
| upper_limit | Integer | Max. times Minimum: 1 |
| time_interval | Integer | Time range Minimum: 1 |
| Parameter | Type | Description |
|---|---|---|
| alpha | Double | Tradeoff parameter Minimum: 0 Maximum: 1 |
| min_used_num | Integer | Min. number of behaviors Minimum: 30 Maximum: 1000 |
| Parameter | Type | Description |
|---|---|---|
| weaken_factor | Double | Decay factor Minimum: 0.1 Maximum: 5 |
| view_type | String | Behavior quantity counting mode. The options are:
|
| algo_type | String | Algorithm type. The options are:
|
| Parameter | Type | Description |
|---|---|---|
| user_meta_list | Array of strings | User feature |
| item_meta_list | Array of strings | Item feature |
| Parameter | Type | Description |
|---|---|---|
| user_features | Array of FeatureTransformation objects | User feature |
| item_features | Array of FeatureTransformation objects | Item feature |
| rank_etl_filter | RankETLFilter object | Filter parameter |
| Parameter | Type | Description |
|---|---|---|
| attr | Attribute object | Feature |
| discrete_method | String | Discrete method. The options are:
|
| params | Object | Specific processing parameter |
| Parameter | Type | Description |
|---|---|---|
| name | String | Name |
| data_type | String | Data type |
| other_uses | Array of strings | Other usage |
| Parameter | Type | Description |
|---|---|---|
| filter_type | String | Behavior deduplication mode. The options are:
|
| time_type | String | Time type. The options are: Day Week Month |
| is_monday_first | Boolean | Whether Monday is the first day |
| Parameter | Type | Description |
|---|---|---|
| divide_type | String | Division mode of training and test sets. The options are:
|
| train_rate | Double | Training data ratio Minimum: 0.01 Maximum: 1 |
| test_rate | Double | Test data ratio Minimum: 0.01 Maximum: 1 |
| Parameter | Type | Description |
|---|---|---|
| initial_parameters | Initial object | Initialization parameter |
| optimize_parameters | Optimizer object | Optimization parameter |
| regular_parameters | Regular object | Lambda parameter |
| max_iterations | Integer | Max. iterations Minimum: 1 Maximum: 1000 |
| early_stop_iterations | Integer | Iterations at early stopping Minimum: 1 Maximum: 1000 |
| batch_size | Integer | Batch size Minimum: 1 |
| dataset_split_parts | Integer | Number of training datasets to be split Minimum: 1 Maximum: 10 |
| restart_train | Boolean | Retraining |
| Parameter | Type | Description |
|---|---|---|
| initial_method | String | Initialization method Enumeration values:
|
| mean_value | Double | Mean Minimum: -1 Maximum: 1 |
| standard_deviation | Double | Standard deviation Minimum: 0 Maximum: 1 |
| min_value | Double | Min. value Minimum: -1 Maximum: 0 |
| max_value | Double | Max. value Minimum: 0 Maximum: 1 |
| Parameter | Type | Description |
|---|---|---|
| l2_regularization | Double | Lambda 2 Minimum: 0 Maximum: 1 |
| regular_loss_compute_mode | String | Regular loss calculation mode Enumeration values:
|
| embed_l2_regularization | Double | Lambda 2 of embedding size Minimum: 0 Maximum: 1 |
| wide_l2_regularization | Double | Lambda 2 of the wide part Minimum: 0 Maximum: 1 |
| structure_l2_regularization | Double | Lambda 2 of the structured part Minimum: 0 Maximum: 1 |
| Parameter | Type | Description |
|---|---|---|
| latent_vector_length | Integer | Embedding size (required for DeepFM) Minimum: 1 Maximum: 100 |
| architecture | Array of integers | Neural network structure (required for DeepFM) |
| active_function | String | Activation function (required for DeepFM and AutoGroup) Enumeration values:
|
| value_keep_probability | Double | Neuron retention probability (required for DeepFM and AutoGroup) Minimum: 0 Maximum: 1 |
| embed_size | Array of integers | Embedding size of each degree (required for AutoGroup) |
| mlp_architecture | Array of integers | Neural network structure (required for AutoGroup) |
| max_order | Integer | Max. interactions (required for AutoGroup) |
| hash_sizes | Array of integers | Hash length (required for AutoGroup) |
| hash_compensation | Array of numbers | Feature interaction penalty coefficient (required for AutoGroup) |
| use_wide_part | Boolean | Wide part required (required for AutoGroup) |
| structure_optimizer | Optimizer object | Optimizer parameter (required for AutoGroup) |
| merge_multi_hot | Boolean | Merge multi-value feature (required for AutoGroup) |
| fix_structure | Boolean | Fix hash structure (required for AutoGroup) |
| Parameter | Type | Description |
|---|---|---|
| indicator_name | String | Indicator. The options are: PV UV Custom |
| indicator_params | IndicatorParam object | Indicator parameter (required for custom metric) |
| Parameter | Type | Description |
|---|---|---|
| customize_parameter | CustomizeParameter object | Custom parameter |
| customize_formula | CustomizeFormula object | Custom formula |
| Parameter | Type | Description |
|---|---|---|
| alias | String | Alias |
| behavior_type | String | Behavior type |
| threshold | Double | Threshold Minimum: 0 Maximum: 1 |
| deduplication | String | Deduplication |
| Parameter | Type | Description |
|---|---|---|
| type | String | Optimizer type Enumeration values:
|
| learning_rate | Integer | Learning rate Minimum: 0 Maximum: 1 |
| initial_accumulator_value | Double | Initial gradient sum Minimum: 0 Maximum: 1 |
| lambda1 | Double | Lambda 1 Minimum: 0 Maximum: 1 |
| lambda2 | Double | Lambda 2 Minimum: 0 Maximum: 1 |
| epsilon | Double | Epsilon Minimum: 0 Maximum: 1 |
| decay_rate | Double | Decay factor Minimum: 0 Maximum: 1 |
| decay_steps | Double | Decay step Minimum: 1 |
| Parameter | Type | Description |
|---|---|---|
| flow_id | String | Process flow ID |
| attr_pair_rules_filter | Array of AttrPairRules objects | Feature pair filter |
| attr_pair_rules_reserve | Array of AttrPairRules objects | Feature pair to be reserved |
| deduplication_list | Array of Deduplication objects | Feature deduplication |
| attribute_info | AttributeInfo object | Comprehensive ranking information |
| bloom_filter_conf | BloomFilterConf object | Bloom filter settings |
| group_attr | String | The scatter attribute for grouping |
| pre_deal | Boolean | Deduplication before ranking |
| rank_setting | String | Ranking configuration information |
| rules | Rule object | Candidate set merging |
| filter_sets | Array of strings | Filter configuration information |
| attr_value_rules_filter | Array of AttrValueRules objects | Feature filter |
| attr_value_rules_reserve | Array of AttrValueRules objects | Feature to be reserved |
| ctr_job | String | Ranking job (required when click-through rate is used) |
| ratio | Integer | Traffic proportion Minimum: 1 Maximum: 100 |
| toppings | Array of strings | List of candidate sets to be pinned on top |
| Parameter | Type | Description |
|---|---|---|
| party_a | String | Feature name of the recommended item |
| party_b | String | Feature name of the recommended item |
| Parameter | Type | Description |
|---|---|---|
| rank_feature_pairs | Array of RankFeaturePair objects | Matched feature pair |
| numerical_attrs | Array of NumericalAttr objects | Feature weight |
| num_statistics_type | String | Statistics mode. The options are:
Enumeration values:
|
| Parameter | Type | Description |
|---|---|---|
| feature_name_a | String | Feature of the item to be recommended |
| feature_name_b | String | Feature of the recommended item |
| weight | Float | Weight Minimum: 0.01 Maximum: 1 |
| Parameter | Type | Description |
|---|---|---|
| name | String | Feature name |
| weight | Float | Weight Minimum: 0.001 Maximum: 1 |
| Parameter | Type | Description |
|---|---|---|
| behaviors | Array of strings | Type of the behavior to be filtered |
| interval | Integer | Filter time Minimum: 1 Maximum: 7 |
| Parameter | Type | Description |
|---|---|---|
| table_name | String | Table name of a candidate set |
| rule_ratio | Integer | Rule ratio Minimum: 1 Maximum: 100 |
| priority | Integer | Priority Minimum: 1 Maximum: 10 |
Example Requests
This API is used to query the details about a specified data source.
/v2.0/testuuidxxxxxxxxxxxxxxxxxxxxxxxx/workspaces/testuuidxxxxxxxxxxxxxxxxxxxxxxxx/data-sources/testuuidxxxxxxxxxxxxxxxxxxxxxxxx -
Example Responses
Status code: 200
OK
{
"is_success" : true,
"datasource" : {
"specs_config" : {
"offline" : "4CU"
},
"data_config" : {
"offline" : {
"user_url" : "<OBS path for storing user data>",
"item_url" : "<OBS path for storing item data>",
"behavior_url" : "<OBS path for storing behavior data>"
},
"nearline" : { }
},
"datasource_name" : "test-cyz",
"workspace_id" : "06a7c49afc00d4972ff1c0113d8c49ba",
"datasource_id" : "aa6d657026de42a2927a50400ae69a4b",
"status" : "Succeed",
"created_at" : 1587953962684,
"update_at" : 1587953962684,
"structure" : "{\"user_attrs\":[{\"name\":\"age\",\"data_type\":\"long\"},{\"name\":\"extend_float\",\"data_type\":\"float\"},{\"name\":\"extend_long\",\"data_type\":\"long\"},{\"name\":\"extend_strArray\",\"data_type\":\"strArray\"},{\"name\":\"extend_string\",\"data_type\":\"string\"},{\"name\":\"gender\",\"data_type\":\"string\"},{\"name\":\"location\",\"data_type\":\"location\"},{\"name\":\"tags\",\"data_type\":\"strArray\"},{\"name\":\"userId\",\"data_type\":\"string\"}],\"item_attrs\":[{\"name\":\"author\",\"data_type\":\"strArray\"},{\"name\":\"category\",\"data_type\":\"string\"},{\"name\":\"expireTime\",\"data_type\":\"long\"},{\"name\":\"extend_float\",\"data_type\":\"float\"},{\"name\":\"extend_long\",\"data_type\":\"long\"},{\"name\":\"extend_strArray\",\"data_type\":\"strArray\"},{\"name\":\"extend_string\",\"data_type\":\"string\"},{\"name\":\"itemId\",\"data_type\":\"string\"},{\"name\":\"itemType\",\"data_type\":\"string\"},{\"name\":\"location\",\"data_type\":\"location\"},{\"name\":\"publishTime\",\"data_type\":\"long\"},{\"name\":\"score\",\"data_type\":\"long\"},{\"name\":\"status\",\"data_type\":\"long\"},{\"name\":\"tags\",\"data_type\":\"strArray\"}],\"behaviors\":{\"positive_behaviors\":[{\"behavior_type\":\"click\",\"weight\":0.3},{\"behavior_type\":\"like\",\"weight\":0.8}],\"negative_behaviors\":[{\"behavior_type\":\"view\",\"weight\":0.1},{\"behavior_type\":\"dislike\",\"weight\":0.8}]}}"
},
"jobs" : [ {
"category" : "DATASOURCE",
"job_config" : {
"save_mode" : "new",
"load_profile" : true,
"load_widetable" : true
},
"job_name" : "res-data-import",
"job_id" : "3ccd65777a72477ca3c819adfd8db94e",
"job_type" : "DataImport",
"status" : "Succeed",
"platform" : "OFFLINE",
"workspace_id" : "06a7c49afc00d4972ff1c0113d8c49ba",
"resource_id" : "aa6d657026de42a2927a50400ae69a4b"
}, {
"category" : "DATASOURCE",
"job_config" : { },
"job_name" : "res-data-exploration",
"job_id" : "a02af2280fae4c2189e141679c6bc91f",
"job_type" : "DataExploration",
"status" : "Succeed",
"platform" : "OFFLINE",
"workspace_id" : "06a7c49afc00d4972ff1c0113d8c49ba",
"resource_id" : "aa6d657026de42a2927a50400ae69a4b"
}, {
"category" : "DATASOURCE",
"job_config" : { },
"job_name" : "res-data-struct",
"job_id" : "e5d9921aabea48c49164722f43b4ccb9",
"job_type" : "DataStruct",
"status" : "Succeed",
"platform" : "OFFLINE",
"workspace_id" : "06a7c49afc00d4972ff1c0113d8c49ba",
"resource_id" : "aa6d657026de42a2927a50400ae69a4b"
} ]
} Status Codes
| Status Code | Description |
|---|---|
| 200 | OK |
Error Codes
See Error Codes.
Last Article: Querying the List of Data Sources
Next Article: Modifying a Data Source
Did this article solve your problem?
Thank you for your score!Your feedback would help us improve the website.