Inferring the Output Tensor Description of an Operator

Infer the output tensor description of the operator based on the input tensor description, operator logic, and operator attributes. The output tensor description includes the tensor shape, data type, and data layout format. In this way, all tensors can be statically allocated with memory during offline model conversion, thereby avoiding overhead caused by dynamic memory allocation.

Function Declaration

The function is declared as follows:

Status InferShapeAndTypexx(const ge::Operator& op, vector<ge::TensorDesc>& v_output_desc)

InferShapeAndTypexx: function name, which is user-defined and must be unique
op: compute node definition, which stores the input tensor description and operator attributes. For details about the ge::Operator type, see Class Operator in GE API Reference
v_output_desc: output tensor description of the compute node, including the shape, data layout format, and data type. For details about the TensorDesc type, see Class TensorDesc in GE API Reference

The following describes the implementation of the InferShapeAndTypexx function in different scenarios.

Operator with Same-Shape Output and Input Tensors

For an operator whose output and input tensors have the identical shape, you can directly insert the description of the input tensor into the vector space in which the output tensor description is located.

A code sample is provided as follows:

v_output_desc.push_back(op.GetInputDesc(0));

The GetInputDesc API is used to obtain the input tensor description based on the operator input name or input index in class Operator. For details about the API, see Class Operator in GE API Reference.

Operator with Reduced Dimensions

For common dimension reduction operations such as Reduction and Reduce, compute the shape of the output tensor (including the output tensor dimensions and element count of each dimension) according to information such as the operator input attribute axis, and then insert the shape of the output tensor into the v_output_desc vector.

A code sample is provided as follows:

Obtain the input tensor description and shape of the input tensor of the operator.

auto tensorDesc = op.GetInputDesc(0); // Obtain the input tensor description, including the shape, data layout format, and data type.
auto shape = tensorDesc.GetShape(); // Obtain the shape of the input tensor.

For details about the GetShape API, see Class TensorDesc in GE API Reference.

Obtain the attribute values of the operator according to the computation logic, and compute the shape of the output tensor of the operator.

For example, for the reduction operator in the MyLeNet network, because the upper layer of Reduction is Softmax and the output from Softmax is padded to 4-dimensional from 2-dimensional in the offline model, you need to adjust axis so that it points to a 2-dimensional position. After the reduce operation, and assign the adjusted Shape to the output tensor description.
- Obtain the key-value pair of the axis attribute from the operator object, obtain the axis attribute value from the key-value pair, convert the attribute from the INT type to the int64_t type, assign the attribute value to the variable axis, verify and adjust the value of axis, and point the value to axis 1, as follows:
```
int64_t axis = -1;
ge::AttrValue axisAttrValue;
    if ((ge::GRAPH_SUCCESS != op.GetAttr("axis", axisAttrValue)) || (ge::GRAPH_SUCCESS != axisAttrValue.GetValue<AttrValue::INT>(axis)))
    {
        printf("Get axis failed!\n");
    }
    // In the OM model, all shape are supplemented to 4d. In this case, axis needs to be repaired to point to the original 2d.
    if (axis < 0) axis -= 2;

    if (axis < 0) axis += shape.GetDimNum();

    if (axis < 0 || axis >= shape.GetDimNum())
    {
        printf("invalid axis:%d, dim_size:%d\n", (int32_t)axis, (int32_t)shape.GetDimNum());
        return PARAM_INVALID;
    }
```
- Adjust Shape and set the dimensions from axis 1 to 1. For example, if the input tensor is with shape (2, 3, 4, 5), adjust the shape to (2, 1, 1, 1).
```
 int32_t dimsize = (int32_t)shape.GetDimNum();
 int32_t idx = 0;
 for(idx=axis; idx<dimsize; idx++)
 {
     shape.SetDim(idx, 1);
 } 
```
- Set the adjusted shape to the tensorDesc object.
```
 tensorDesc.SetShape(shape); 
```
For details about APIs GetDimNum and SetDim, see Class Shape in GE API Reference.
Set the output tensor description of the operator.
```
v_output_desc.push_back(tensorDesc)
```
Assign tensorDesc to the description object v_output_desc of the output tensor.

Network with Multiple Operators of the Identical Type

A network can have multiple operator layers of the same type, such as the convolution operator. Sometimes, you need to customize the shape of an operator layer (redefine an existing operator). In this case, the output tensor description inference needs to be performed according to different situations of the network, determine the operator layers to be customized based on the attributes such as num_output, kernel, stride, and pad, and obtain the tensor information of the operators.

A code sample is provided as follows:

Assign the input tensor description to the output tensor description, and obtain the input tensor description and the shape of the input tensor of the operator.

v_output_desc.push_back(op.GetInputDesc(0)); // Assign the input tensor description to the output tensor description object. Alternatively, assign the shape obtained after inference to the tensorDesc object, and then assign the tensorDesc object to the output tensor description object.
auto tensorDesc = op.GetInputDesc(0); // Obtain the input tensor description, including the shape, data layout format, and data type.
auto shape = tensorDesc.GetShape(); // Obtain the shape of the input tensor.

For details about the GetShape API, see Class TensorDesc in GE API Reference.

Obtain the attribute values of the operator according to the computation logic, match the operator according to the attribute values of the operator and shape, and compute the shape of the output tensor of the operator.

For example, for operators of type convolution in a network, match the convolution operator whose num_outputs is 128, shape.GetDim(0) is 1, shape.GetDim(1) is 128, shape.GetDim(2) is 28, and shape.GetDim(3) is 28, and reassign the shape in the output tensor description of the operator.
- Obtains the value of num_outputs.
```
ge::AttrValue num_outputsAttrValue;
    if ((ge::GRAPH_SUCCESS != op.GetAttr("num_output", num_outputsAttrValue)) || 
        (ge::GRAPH_SUCCESS != num_outputsAttrValue.GetValue<AttrValue::INT>(num_outputs)))
    {
        printf("GetOpAttr num_outputs failed!\n");
    }
```
- Match the convolution operator whose num_outputs is 128, shape.GetDim(0) is 1, shape.GetDim(1) is 128, shape.GetDim(2) is 28, and shape.GetDim(3) is 28 , and reassign the shape in the output tensor description of the operator.
```
 if(shape.GetDim(0) == 1 && shape.GetDim(1) == 128 &&
        shape.GetDim(2) == 28 && shape.GetDim(3) == 28 && num_outputs == 128)
    {
        shape.SetDim(0, 1);
        shape.SetDim(1, 128);
        shape.SetDim(2, 28);
        shape.SetDim(3, 28);
        v_output_desc[0].SetShape(shape);
        return SUCCESS;
        return FAILED;
    }
```
For details about APIs GetDimNum and SetDim, see Class Shape in GE API Reference.
To use op_name for operator matching, obtain op_name as follows:
```
auto op_name = op.GetName();
```
The method for obtaining other attributes kernel_w, kernel_h, stride_w, stride_h, pad_w, and pad_h are similar. You only need to change the value of key in op.GetAttr.