Lightning Talk: High Performance Neural Network Inferencing Using WASI NN - Radu Matei, Microsoft

Описание: Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Lightning Talk: High Performance Neural Network Inferencing Using WASI NN - Radu Matei, Microsoft Azure

The sandbox environment and portability of WebAssembly provide an ideal deployment target for neural network inferencing. But while inferencing engines can be directly compiled to WebAssembly, the current limitations around parallelism and hardware acceleration present real limitations to performance. This is the main reason behind WASI-NN, a new WebAssembly System Interface (WASI) specification that enables near-native performance for neural network workloads with hardware acceleration on a variety of devices such as CPUs, GPUs (and in the future TPUs or FPGAs). In this session, we will present the WASI-NN interface proposal and detail the current state of two specialized implementations -- for the OpenVINO and ONNX runtimes. Finally, we will demonstrate an end-to-end process for machine learning, from building and training neural networks using PyTorch or TensorFlow, all the way to running the same neural network and WebAssembly module from different platform architectures and operating systems using the Wasmtime runtime.This project was developed in collaboration with Andrew Brown, Intel.