Interface InferenceModel

Common interface for a machine learning model that can do inference.

interface InferenceModel {
    inputs: ModelTensorInfo[];
    outputs: ModelTensorInfo[];
    execute(inputs, outputs): Tensor<Rank> | Tensor<Rank>[];
    predict(inputs, config): Tensor<Rank> | Tensor<Rank>[] | NamedTensorMap;
}

Implemented by

Properties

inputs outputs

Methods

execute predict

Properties

Readonly inputs

inputs: ModelTensorInfo[]

Return the array of input tensor info.

Readonly outputs

outputs: ModelTensorInfo[]

Return the array of output tensor info.

Methods

execute

  • execute(inputs, outputs): Tensor<Rank> | Tensor<Rank>[]

  • Single Execute the inference for the input tensors and return activation values for specified output node names without batching.

    Parameters

    • inputs: Tensor<Rank> | Tensor<Rank>[] | NamedTensorMap
    • outputs: string | string[]

      string|string[]. List of output node names to retrieve activation from.

    Returns Tensor<Rank> | Tensor<Rank>[]

    Activation values for the output nodes result tensors. The return type matches specified parameter outputs type. The output would be single Tensor if single output is specified, otherwise Tensor[] for multiple outputs.

predict

  • predict(inputs, config): Tensor<Rank> | Tensor<Rank>[] | NamedTensorMap

  • Execute the inference for the input tensors.

    Parameters

    • inputs: Tensor<Rank> | Tensor<Rank>[] | NamedTensorMap
    • config: ModelPredictConfig

      Prediction configuration for specifying the batch size.

    Returns Tensor<Rank> | Tensor<Rank>[] | NamedTensorMap

    Inference result tensors. The output would be single Tensor if model has single output node, otherwise Tensor[] or NamedTensorMap[] will be returned for model with multiple outputs.

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc