Relationships Between MemArtsCC and Other Components

OBS

Hadoop-OBS provides a new InputStream: OBSMemArtsCCInputStream. This InputStream reads data from the MemArts cluster deployed on the compute side to reduce OBS server pressure and improve data read performance.

MemArtsCC persistently stores data to the storage (SSD) on the compute side. Hadoop-OBS interconnects with MemArtsCC to:

1. Reduce the pressure on the OBS server.

   MemArtsCC stores hotspot data in the compute cluster to reduce the bandwidth pressure of the OBS server.

2. Improve the data access performance of the storage decoupled from the compute.

   The local storage of MemArtsCC avoids the cross-network access of hotspot data. This accelerates the data reads of Hadoop-OBS upper-layer applications.

Spark

Spark reads data from OBS. OBSA reads data from MemArtsCC. If data is hit in the local cache, the data is read directly. Otherwise, the data is prefetched.