Today we added NVIDIA L40S GPUs to our supported hardware types. These new GPUs are 40% faster and better value than A40s. We will also begin migrating all existing models and deployments from A40 GPUs to L40S GPUs over the coming weeks.
You can now run L40S GPUs for any new models, existing models, or deployments. To learn how to change the hardware type for your models and deployments, check out the docs.
Starting today, you have the option to switch any of your existing models and deployments to L40S GPUs, but you are not required to do so. If you choose not to switch, your models and deployments will continue to run on A40 GPUs for another few weeks until we migrate them to L40S GPUs.
To give you better performance per dollar, we will begin migrating all existing models and deployments from A40 GPUs to L40S GPUs over the coming weeks:
The L40S GPUs have a higher sticker price than the A40 GPUs, but they deliver better performance per dollar.
We expect most models to see equal or better performance per dollar on the L40S GPUs. Our benchmarks across the highest-usage A40 models today show a ~40% median speed improvement when migrating from A40 to L40S GPUs. For more detail, see our Observable notebook which covers our benchmarking methodology, collection methods, and results.
To compare the performance and pricing of all our available hardware types, visit replicate.com/pricing.
If you have any questions, email us at support@replicate.com.