AI running in serverless configurations on GPUs is coming to Cloudflare’s global network.
Content delivery network (CDN), security and web services company Cloudflare is opening its worldwide network to companies looking to build and deploy AI models with new serverless AI, database and observability features, working with several new tech partners to do so.
Part one of Cloudflare’s new AI-focused initiative, announced today, is the Workers AI framework, which offers access to GPUs in Cloudflare’s network for a serverless way to run AI models. For users trying to run AI systems that are heavily latency dependent, the framework should offer the option of running workloads much closer to the network edge, reducing round-trip time. The company said that Workers AI is also designed to separate inference from training data, ensuring that consumer information is not misused.
The second of Cloudflare’s new AI announcements is Vectorize, which is a vector database designed to allow developers to create AI-based applications entirely on Cloudflare’s own systems. Vectorize works in tandem with Cloudlflare’s underlying network, again allowing work to be done closer to the end user, and has integrations with Workers AI which should let users generate embeddings in Workers AI and index them in Vectorize.
On its part, AI Gateway is a performance-management and optimization system designed to offer observability into AI applications running on Cloudflare’s network. AI Gateway provides data like number and duration of requests, costs of running the app, and user counts for AI applications, according to the company, as well as cost-saving options like rate limiting and caching answers to common queries.
Cloudflare also announced collaborations with Microsoft, Databricks and AI startup Hugging Face. Microsoft brings its ONNX runtime for continuity of AI models across cloud, edge or on-device usage, while the Databricks partnership adds that company’s MLflow open source platform for machine learning cycle management. Finally, Cloudflare’s network will be the first venue for customers to deploy Hugging Face’s powerful generative AI models in a serverless, GPU-powered environment.
Matthew Prince, co-founder and CEO of Cloudflare, said that the new offerings represent a major extension of the company’s developer platform, and that the goal is to make “inference infrastructure” accessible for all potential customers.
The network, according to Prince, is the best place to run AI.
“We’ve already seen interest from companies who are trying to solve this exact challenge of providing powerful experiences without sacrificing battery life or latency,” he said. “That said, as LLMs and AI become an integral part of every application, we believe Cloudflare is well suited for powering those by making it easy and affordable for developers to get started.”
All features announced are available immediately. Pricing will be, essentially, based on usage, with different schema for Workers, Vectorize and AI Gateway. (Vectorize, Prince noted, will be free to use until 2024.)
