NVIDIA multi-process service

usage and internals

Posted by Peter Lau on October 20, 2023

Architecture

对于多个CPU process发起的cuda任务,scheduler会调配时间片分配给每个process;由上图可知,GPU在执行当前process任务一定时间后,会切换执行另一个process的任务,存在许多的context-switch。

MPI(Message Pass Interface)

constraints

practices

references

  1. Multi-Process Service