Platform X: Access to AI

There are several options for using AI within Platform X. Platform X allows you to use AI on sensitive data (e.g. PHI) in a secure way.

If you are interested in having AI within Platform X for your project please reach out to Platform X support.

Serverless API

The most popular option is to have a small VM that connects to Foundry. This option is often the most economical as it allows a light-weight virtual machine (lower cost) to connect to AI Services.

To identify if this will work for you, review the models available in Foundry with either the "Serverless API" Deployment Option or if you open a model without a Deployment Option tag check if the billing is based off of a per token cost.

The main exceptions for when this would be a good fit is when there is very high use of the model or lots of sharing of work on the model between users.

Managed Compute

For some models there isn't an option to have an API connection. Managed compute allows a non-GPU enabled virtual machine to connect to Foundry and have Microsoft handle the computing. This option tends to be more expensive because Microsoft has high performance service level agreement. 

This option is best for groups who would utilize AI in a burst (high usage in short periods of time), or are doing fine-tuning. 

You have the option for tearing down the connection to help manage cost.

Deploy Virtual Machine with GPUs

An alternative to using managed compute is by deploying a virtual machine with a GPU. You can run the compute you need locally with the option of running it with less performance (and less cost) than the managed compute option. 

You also have the option of deploying two machines. One VM that doesn't have GPU used for non-AI work and one VM that is used for AI work. This option is best for groups who's primary work doesn't need AI. 

Using other AI Resources

If you are looking for options to keep your AI costs low, consider doing as much prompt refinement as possible using UW Madison campus AI resources or the National AI Research (NAIRR) pilot project. Ensure you understand the type of data that is approved for each resource. You can consider using synthetic data in options that are only safe for public and internal data.