New Provider Families: A better cloud-native integration experience for Crossplane

Today we announce new “Provider Families” or groups of more narrowly scoped providers. This improves the integration experience for all users of Crossplane–the framework for building cloud native control planes. Starting today, the official Upbound providers will be more granularly organized to better align with how teams are using them. This change improves the performance of Crossplane and provides a path for future growth as the number of cloud resources grows.

We’ll explain the history of Crossplane providers, what's new and next steps. New users of Crossplane can get started with provider families today.

Upbound Marketplace and Upbound “Official” – Free for all Crossplane users

It’s hard to believe that just over six months ago Upbound released our first official providers for Crossplane. Providers bundle a set of Managed Resources and controllers to allow Crossplane to provision and manage the respective infrastructure resources. They get deployed at the Kubernetes cluster level and are ready whenever you need to deploy a set of resources that your application depends on. Providers use Custom Resource Definitions (CRDs) in Kubernetes to teach Crossplane about new kinds of Managed Resources. In addition to the official providers, there are community providers as well that organizations can use if they choose and it is great to see the community support Crossplane with all their contributions.

Since we released the official providers, our coverage of the most used resources for cloud-native deployments has quickly grown to 75%. The largest of the official providers (provider-aws) includes over 900 resources that are ready for organizations to use. These are validated, tested, and supported by Upbound. All providers are freely available for our customers and for the Crossplane community.

Introducing Provider Families

As the number of resources in each official provider has increased, it is clear that no organization needs ALL those resources in one single provider. The more CRDs you have on a cluster the more resources it will consume and this can result in a performance penalty, so reducing unused CRDs is definitely the way to go. In fact, on average most control planes actually use about 30 CRDs from a provider at any given time. With that in mind, we have decided to split up and organize the official providers into Provider Families or groups of smaller providers that more closely align to how platform teams actually use them. This reduces bloat, improves performance, and provides a path for future growth as the number of cloud resources grows on a seemingly daily basis.

Benefits and practical implications

What this means in practice is that instead of installing just provider-aws, which includes lots of things you don’t need, you will simply install smaller providers that are scoped to the services you require. For example, you might install provider-aws-ec2 and provider-aws-rds if all you need is an Amazon EC2 instance connecting to an RDS database. Any providers of your choosing can be easily grouped together into a “meta” provider to make installation easier for applications that have a distinct set of requirements.

In general, it’s always a best practice to limit risk by reducing the number of things installed on your systems. Even if the potential for problems is very small, this change helps teams improve their security posture by limiting their overall exposure.

We have provided early access to the new provider families to a small subset of users and we are pleased that the feedback has been overwhelmingly positive.

“Service-scoped providers have been a game-changer for us. The ability to selectively install CRDs based on Composition requirements has not only boosted the performance of our Crossplane implementation but it's also made our control plane more elastic by allowing us to better position Provider workloads across nodes,” said Adam House, Senior Staff Engineer, American Family Insurance.

Callum Loh, DevOps Engineer at Squiz, told us, “The new Provider Families from Upbound push Crossplane to the next level. It's given us a significant performance boost and made it easier to manage them, all whilst not impacting our end-user experience at all.”

A large global consumer brand noted the new Crossplane Provider Families delivered by Upbound have “already given us a huge improvement in provisioning time.”

How do I migrate?

Starting today, we recommend that new control planes being deployed make use of the new providers. To assist organizations in migrating to the new provider family packages we will soon release a tool to automate the process for users and will be following up this blog with one specifically addressing migrating from the legacy providers to the new Provider Families. If you want to get started right away, you can check out the migration documentation.

Upbound will continue to publish new versions of the monolithic version (i.e., the one containing all the resources in one package) of the official providers until 13 December, 2023, allowing organizations 6 months to transition to the new Provider Families. In addition, in case of security or other critical bugs we will continue to provide as-needed bug patches to the monolith providers for 1 year, until June 2024. Providers are available to all Crossplane users for free, and for customers, Upbound Support and Services can help in the transition. Contact us if you have questions.

Where can I find more information?

You can find more information and details about how to migrate to the new Provider Families on our provider marketplace page and in our documentation.