Release vGPU functionality for client grade GPUs.
This tool is terribly untested, use at your possess risk.
This tool enables the usage of Geforce and Quadro GPUs with the NVIDIA vGPU
instrument. NVIDIA vGPU typically easiest supports just a few Tesla GPUs but since some
Geforce and Quadro GPUs half the same physical chip as the Tesla this is easiest
a instrument limitation for these GPUs. This tool objectives to preserve this limitation.
- This tool requires Python3, essentially the most traditional model is urged.
- The python kit “frida” is required.
pip3 install frida.
- The tool requires the NVIDIA GRID vGPU driver.
- “dkms” is required as it simplifies the device of rebuilding the
driver alot. Set up DKMS with the kit supervisor in your OS.
Within the next instructions
must be replaced with
the path to this repository on the aim gadget and
replaced with the model of the NVIDIA GRID vGPU driver.
Set up the NVIDIA GRID vGPU driver, manufacture sure to put in it as a dkms module.
Adjust the line begining with
/lib/systemd/gadget/nvidia-vgpu-mgr.provider to use
executable and circulation the distinctive executable as essentially the predominant argument. Ex:
Reload the systemd daemons:
Adjust the file
/usr/src/nvidia- and add the
following line after the traces begining with
#encompass within the starting place of the
Adjust the file
/usr/src/nvidia- and add the
following line on the bottom of the file.
Make a selection away the nvidia kernel module the usage of dkms:
dkms preserve -m nvidia -v
Rebuild and reinstall the nvidia kernel module the usage of dkms:
dkms install -m nvidia -v
This script will easiest work if there exists a vGPU love minded Tesla GPU that
uses the same physical chip as the precise GPU being frail.
The device it works
In declare to rating out if a undeniable GPU supports the vGPU functionality the
driver looks on the PCI tool ID. This identifier along with the PCI vendor
ID is odd for every form of PCI tool. In declare to allow vGPU toughen we
must explain the driver that the PCI tool ID of the assign in GPU is one of
the tool IDs frail by a vGPU expedient GPU.
Userspace script: vgpu_unlock
The userspace companies nvidia-vgpud and nvidia-vgpu-mgr uses the ioctl syscall
to talk with the kernel module. Namely they read the PCI tool ID
and determines if the assign in GPU is vGPU expedient.
The python script vgpu_unlock intercepts all ioctl syscalls between the
executable specified as essentially the predominant argument and the kernel. The script then
modifies the kernel responses to direct a PCI tool ID with vGPU toughen
and a vGPU expedient GPU.
Kernel module hooks: vgpu_unlock_hooks.c
In declare to trade recordsdata with the GPU the kernel module maps the physical
tackle condominium of the PCI bus into its possess virtual tackle condominium. This is done
the usage of the ioremapkernel capabilities. The kernel module then reads and writes
recordsdata into that mapped tackle condominium. This is done the usage of the memcpy kernel
By including the vgpu_unlock_hooks.c file into the os-interface.c file we will have the option to
use C preprocessor macros to interchange and intercept calls to the iormeap and
memcpy capabilities. Doing this allows us to succor a gawk of what is mapped
the place and what recordsdata that is being accessed.
Kernel module linker script: kern.ld
This is a modified model of the default linker script equipped by gcc. The
script is modified to recount the .rodata fragment of nv-kernel.o into .recordsdata
fragment barely than .rodata, making it writable. The script also provide the
to allow us to know the place that fragment begins and ends.
The device it all comes together
After boot the nvidia-vgpud provider queries the kernel for all assign in GPUs
and assessments for vGPU capability. This call is intercepted by the vgpu_unlock
python script and the GPU is made vGPU expedient. If a vGPU expedient GPU is chanced on
then nvidia-vgpu creates an MDEV tool and the /sys/class/mdev_bus listing
is created by the gadget.
vGPU devices can now be created by echoing UUIDs into the
manufacture files within the
mdev bus illustration. This can manufacture extra constructions representing
the unique vGPU tool on the MDEV bus. These devices can then be assigned to VMs,
and when the VM begins it might perhaps per chance open the MDEV tool. This causes nvidia-vgpu-mgr
to initiating communicating with the kernel the usage of ioctl. All but again these calls are
intercepted by the vgpu_unlock python script and when nvidia-vgpu-mgr asks if
the GPU is vGPU expedient the acknowledge is modified to yes. After that check it
attempts to initialize the vGPU tool occasion.
Initialization of the vGPU tool is dealt with by the kernel module and it
performs its possess check for vGPU capability, this one is barely more sturdy.
The kernel module maps the physical PCI tackle fluctuate 0xf0000000-0xf1000000 into
its virtual tackle condominium, it then performs some magical operations which we
don’t in level of truth know what they stop. What we stop know is that after these operations
it accesses a 128 bit cost at physical tackle 0xf0029624, which we call the
magic cost. The kernel module also accessses a 128 bit cost at physical
tackle 0xf0029634, which we call essentially the predominant cost.
The kernel module then has just a few search for tables for the magic cost, one
for vGPU expedient GPUs and one for the others. So the kernel module looks for the
magic cost in each and every of these search for tables, and if it is chanced on that table entry
also comprises a place of AES-128 encrypted recordsdata blocks and a HMAC-SHA256
The signature is then validated by the usage of essentially the predominant cost talked about earlier to
calculate the HMAC-SHA256 signature over the encrypted recordsdata blocks. If the
signature is correct, then the blocks are decrypted the usage of AES-128 and the same
Inner of the decrypted recordsdata is as soon as extra the PCI tool ID.
So in declare for the kernel module to settle for the GPU as vGPU expedient the magic
cost will must be within the table of vGPU expedient magic values, essentially the predominant has
to generate a sound HMAC-SHA256 signature and the AES-128 decrypted recordsdata blocks
has to possess a vGPU expedient PCI tool ID. If any of these assessments fail, then
the error code 0x56 “Name not supported” is returned.
In declare to manufacture these assessments circulation the hooks in vgpu_unlock_hooks.c will peek
for a ioremap call that maps the physical tackle fluctuate that possess the magic
and key values, recalculate the addresses of these values into the virtual
tackle condominium of the kernel module, show screen memcpy operations reading at these
addresses, and if such an operation occurs, make a duplicate of the cost till each and every
are known, come across the search for tables within the .rodata fragment of nv-kernel.o, rating
the signature and recordsdata bocks, validate the signature, decrypt the blocks, edit
the PCI tool ID within the decrypted recordsdata, reencrypt the blocks, regenerate the
signature and insert the magic, blocks and signature into the table of vGPU
expedient magic values. And that’s what they stop.