Today we added NVIDIA L40S GPUs to our supported hardware types. These new GPUs are around 40% faster than A40 GPUs and usually better value.
We're also going to be removing support for A40 GPUs. We will begin migrating all existing models and deployments from A40 GPUs to L40S GPUs over the coming weeks.
You can now run L40S GPUs for any new models, existing models, or deployments. To learn how to change the hardware type for your models and deployments, check out the docs.
Starting today, you have the option to switch any of your existing models and deployments to L40S GPUs, but you are not required to do so. If you choose not to switch, your models and deployments will continue to run on A40 GPUs for another few weeks until we migrate them to L40S GPUs.
To give you better performance per dollar, we will begin migrating all existing models and deployments from A40 GPUs to L40S GPUs over the coming weeks:
The L40S GPUs are more expensive per second than the A40 GPUs, but they are also much faster.
We expect most models to see equal or better performance per dollar on the L40S GPUs. Our benchmarks across the highest-usage A40 models today show a ~40% median speed improvement when migrating from A40 GPUs to L40S GPUs. For more detail, see our Observable notebook which covers our benchmarking methodology, collection methods, and results.
To compare the performance and pricing of all our available hardware types, visit replicate.com/pricing.
If you have deployments that are using A40 GPUs, you will need to decrease their minimum instances to avoid being charged more.
For example, if your deployment is running 10 minimum instances on A40 GPUs, you should change your deployment configuration to 6 minimum instances when you switch to L40S GPUs, as they are approximately 40% faster.
You can edit your deployment configuration on the web or use the HTTP API.
If you're not sure how to best configure your deployments, email us at support@replicate.com.