回传数据优化处理
当推理计算完成后,需要将推理结果或者推理结束信号发送给Host端,如果在推理Engine内部调用SendData回传数据到Host端,将会消耗推理Engine的时间。建议单独开一个专门负责回传数据的Engine(例如:DataOptEngine),当推理结束后,推理Engine将处理数据透传给DataOptEngine,由DataOptEngine负责将数据回传给Host侧,再由Host侧的Engine(例如:DstEngine)负责接收传过来的推理结果。
//Device侧的DataOptEngine负责将数据回传给Host侧 HIAI_IMPL_ENGINE_PROCESS("DataOptEngine", DataOptEngine, 1) { HIAI_StatusT hiaiRet = HIAI_OK; if (arg0 == nullptr) { HIAI_ENGINE_LOG(HIAI_INVALID_INPUT_MSG, "get inference result timeout"); return HIAI_INVALID_INPUT_MSG; } hiaiRet = SendData(0, "EngineTransNewT", arg0); } // Host侧的DstEngine负责接收从Device侧传过来的推理结果 HIAI_IMPL_ENGINE_PROCESS("DstEngine", DstEngine, 1) { HIAI_StatusT ret = HIAI_OK; if (nullptr != arg0) { printf("dest engine had receive data already\n"); std::shared_ptr<std::string> result = std::static_pointer_cast<std::string>(arg0); ret = SendData(0, "string", result); if (HIAI_OK != ret) { HIAI_ENGINE_LOG(ret, "DstEngine SendData to recv failed"); return ret; } } else { HIAI_ENGINE_LOG(HIAI_INVALID_INPUT_MSG, "DestEngine Fail to receive data"); printf("destengine do not receive data arg0 is null\n"); return HIAI_INVALID_INPUT_MSG; } return HIAI_OK; }