El contenido no se encuentra disponible en el idioma seleccionado. Estamos trabajando continuamente para agregar más idiomas. Gracias por su apoyo.

Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive
On this page

Using loader-backup to Back Up Job Data

Updated on 2024-10-25 GMT+08:00

Scenario

After a job is created using the Loader WebUI or loader-tool, use loader-backup to back up data.

NOTE:
  • Only Loader jobs of data export support data backup.
  • This tool is an internal Loader interface and is invoked by the upper-layer component HBase. Only the data backup from HDFS to SFTP is supported.

Prerequisites

The Loader client has been installed and configured. For details, see Running a Loader Job by Using Commands.

Procedure

  1. Log in to the node where the client is installed as the user who installs the client. For details, see Running a Loader Job by Using Commands.
  2. Run the following command to go to the directory where the backup.properties file is located. For example, if the Loader client installation directory is /opt/client/Loader/, run the following command:

    cd /opt/client/Loader/loader-tools-1.99.3/loader-backup/conf

  3. Run the following command to modify the configuration parameters of backup.properties. For details about the parameters, see Table 1.

    vi backup.properties

    server.url = 10.0.0.1:21351,10.0.0.2:12000
    authentication.type = kerberos
    authentication.user =
    authentication.password=
    job.jobId = 1
    use.keytab = true
    client.principal = loader/hadoop
    client.keytab = /opt/client/conf/loader.keytab
    Table 1 Configuration parameters

    Configuration parameters

    Description

    Example Value

    server.url

    Floating IP address and port (21351) for Loader.

    For compatibility, multiple IP addresses and ports can be configured and need to be separated by commas (,). The first IP address and port must be those of Loader (21351). The others can be configured based on service requirements.

    10.0.0.1:21351,10.0.0.2:12000

    authentication.type

    Login authentication mode.

    • kerberos indicates that the security mode is used and Kerberos authentication is performed. Kerberos authentication provides two authentication modes: the password mode and the keytab file mode.
    • simple indicates that the normal mode is used and Kerberos authentication is not performed.

    kerberos

    authentication.user

    User for login when the normal mode or password authentication is used.

    In the keytab login mode, this parameter does not need to be set.

    bar

    authentication.password

    User password for login when the password authentication mode is used.

    In the normal mode or keytab login mode, this parameter does not need to be set.

    The password needs to be encrypted. The encryption method is described as follows:

    1. Go to the directory where encrypt_tool is located. For example, if the Loader client installation directory is /opt/hadoopclient/Loader, run the following command:

      cd /opt/hadoopclient/Loader/loader-tools-1.99.3

    2. Run the following command to encrypt the non-encrypted password. Commands containing authentication passwords pose security risks. Disable the command recording function (history) before running such commands to prevent information leakage.

      ./encrypt_tool Unencrypted password

      The obtained encrypted password is used as the value of authentication.password.

      NOTE:

      If a non-encrypted password contains special characters, the special characters must be escaped. For example, the dollar sign ($) is a special character and can be escaped using single quotation marks ('). If a non-encrypted password contains single quotation marks, use double quotation marks to escape the single quotation marks. If a non-encrypted password contains double quotation marks, use backslashes (\) to escape the double quotation marks. For details, see the shell escape character rules.

    -

    job.jobId

    ID of the job whose data is to be backed up.

    Job IDs can be viewed under created jobs on the Loader web UI.

    1

    use.keytab

    Whether to use the keytab mode to log in.

    • true indicates using the keytab file to log in.
    • false indicates using the password to log in.

    true

    client.principal

    User principal for accessing the Loader service when the keytab authentication mode is used.

    In the normal mode or password login mode, this parameter does not need to be set.

    loader/hadoop

    client.keytab

    Directory where the used keytab file is located when the keytab authentication mode is used.

    In the normal mode or password login mode, this parameter does not need to be set.

    /opt/client/conf/loader.keytab

  4. Run the following command to go to the directory where the backup script run.sh is located. For example, if the Loader client installation directory is /opt/hadoopclient/Loader, run the following command:

    cd /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-backup

  5. Run the following command to run the backup script run.sh to back up Loader job data. The system backs up data to a directory at the same layer of the job output directory.

    ./run.sh Backup data input directory

    For example, the backup data input directory is /user/hbase/, and the job output directory is/opt/client/sftp/sftp1. sftp1 acts as a placeholder. Run the following command to back up data to the /opt/client/sftp/hbase directory:

    ./run.sh /user/hbase/

Utilizamos cookies para mejorar nuestro sitio y tu experiencia. Al continuar navegando en nuestro sitio, tú aceptas nuestra política de cookies. Descubre más

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback