Batch Migrating Maven/npm/PyPI Components to a Self-Hosted Repo

Background

Self-hosted repos allow you to manually upload and download components. They can also interconnect with your local development environment to upload and download components. For details, see Uploading/Downloading Components on the Self-Hosted Repo Page.

Uploading numerous packages one at a time can be cumbersome. You can use the migration tool provided by self-hosted repos to upload components in batches from Nexus or other repositories.

Preparations

• You have created a self-hosted repo of the corresponding type.
• You have the Python3 environment available.
• If Nexus is used, the migration tool and Nexus must run on the same Linux host and be connected to the CodeArts service network. Python3 must be installed on the host.

Migrating a Maven Component

1. Find the components to be migrated from the local Maven repository (for example, C:\Users\xxxxx\.m2\repository) and copy them to a specified directory.
2. Go to the self-hosted repo page and select the target Maven repository in the left pane.
3. Click the repository name. The Repository Path is displayed on the right. Click to copy it.
4. Click Tutorial in the upper right corner of the page. In the displayed dialog box, click Download Configuration File to download the settings.xml file to your local host.

   Open the file on the local host and find the username and password.

   

5. Click in the upper right corner of the page and click Download Migration Tool (uploadArtifact2.py script and artifact.conf configuration file) to the local host.
6. Configure artifact.conf.

   [artifact]
   packageType = Component type. Set it to Maven.
   userInfo = username:password (username and password obtained in step 4)
   repoRelease = Repository path (Release) (repository path obtained in step 3)
   repoSnapshot = Repository path (Snapshot) (repository path obtained in step 3)
   srcDir = Component directory (specified by users), for example, the target directory for storing the downloaded component in step 1.
   
   [nexus]
   nexusAddr = Nexus address
   nexusPort = Nexus port
   repoName = Name of the Nexus repository to be migrated
   userName = Nexus username
   passwd = Nexus password

7. Run the migration script python uploadArtifact2.py.
8. Go to the target self-hosted repo and check whether the component package is uploaded.

Migrating an npm Component

1. Go to the self-hosted repo page and select the target npm repository in the left pane.
2. Click the repository name. The Repository Path is displayed on the right. Click to copy it.
3. Click Tutorial in the upper right corner of the page. In the displayed dialog box, click Download Configuration File to download the npmrc configuration file to your local host.

   Open the configuration file on the local host, find the value of the _auth field, and decode the value using base64.

4. Click in the upper right corner of the page and click Download Migration Tool (uploadArtifact2.py script and artifact.conf configuration file) to the local host.
5. Configure artifact.conf.

   [artifact]
   packageType = Component type. Set it to npm.
   userInfo = Value of the _auth field decoded using base64 in the npmrc configuration file of the npm repository. (For details, see step 3.)
   repoRelease = Repository path (repository path obtained in step 2)
   repoSnapshot = Left empty
   srcDir = Component directory, for example, C:\Users\xxxxxx\repository. You can specify a directory.
   
   
   [nexus]
   nexusAddr = Nexus address
   nexusPort = Nexus port
   repoName = Name of the Nexus repository to be migrated
   userName = Nexus username
   passwd = Nexus password

6. Check whether the Include Patterns is configured for the target npm repository.

   Check whether the private binary packages in the package.json file are in the whitelist. Only packages on the whitelist can be uploaded successfully. If no whitelist is configured, all private binary packages in the package.json file can be uploaded successfully.

   

7. Run the migration script python uploadArtifact2.py.
8. Go to the target self-hosted repo and check whether the component package is uploaded.

Migrating a PyPI Component

1. Go to the self-hosted repo page and select the target PyPI repository in the left pane.
2. Click the repository name. The Repository Path is displayed on the right. Click to copy it.
3. Click Tutorial in the upper right corner of the page. In the displayed dialog box, click Download Configuration File to download the pypirc file to your local host.

   Open the file on the local host and find the username and password.

4. Click in the upper right corner of the page and click Download Migration Tool (uploadArtifact2.py script and artifact.conf configuration file) to the local host.
5. Configure artifact.conf.

   [artifact]
   packageType = Component type. Set it to PyPI.
   userInfo = username:password (username and password obtained in step 3)
   repoRelease = Repository path (repository path obtained in step 2)
   repoSnapshot = Left empty
   srcDir = Component directory, for example, C:\Users\xxxxxx\repository. You can specify a directory.
   
   [nexus]
   nexusAddr = Nexus address
   nexusPort = Nexus port
   repoName = Name of the Nexus repository to be migrated
   userName = Nexus username
   passwd = Nexus password

6. Run the migration script python uploadArtifact2.py.
7. Go to the self-hosted repo page and check whether the binary package is successfully uploaded.