NEW
Fauna Schema: Flexible, Enforceable, and Driven by Code with Zero downtime Migration
Today we are excited to announce the alpha availability of three new features that round out Fauna’s native support for how developers create and manage their database schema across the development lifecycle.
Read on to learn more about this new functionality and if you're interested in exploring how these features can support your organization, please register for the Early Access Program, and schedule time with a Fauna expert if you have any questions.
Late last year we introduced Fauna Schema Language (FSL) which allows developers to define domain models, access controls, and user-defined functions in human-readable language that is managed as a set of files in a code repository. We followed that with support for Computed Fields and Check Constraints. Computed Fields allow values of document fields to be dynamically generated based on expressions defined by the developer, and Check Constraints ensure that documents when written to the database adhere to specific validation rules.
Today we are adding support for these powerful new capabilities:
- Schema enforcement via document types
- A zero-downtime migration system
- Declarative, schema-as-code migrations
Fauna is an operational database that brings together the best characteristics of document and relational databases. We provide comprehensive ACID transaction support and the strongest consistency guarantees. Our database language, FQL, combines the ability to express declarative relational queries and functional business logic in those transactions. And we are now completing the delivery of a set of functionality that supports working with structured and semi-structured data in one model, supports safe data migrations without requiring downtime, and supports the development and deployment of schema with the same tools as application code.
These are the foundational characteristics of Fauna's document-relational model.
Let's dive into these new capabilities in more detail.
Enforcing schema via document types
With introduction of document types, Fauna allows specifying a set of fields and types for documents in a collection as part of the collection's definition.. Fauna then enforces that all documents within the collection conform to this defined shape.
We have designed document types to naturally extend Fauna's type system to database schema: Similar to TypeScript, Fauna is gradually typed, meaning that it combines both static typing and dynamic typing into one system. As applied to document types, this means that Fauna blends the two extremes for data—fully structured (or static) and semi-structured (or dynamic)—into one coherent system.
collection Product {
name: String
price: Number
category: "Tools" | "Apparel" | "Accessories"
desc: String
index byCategory {
terms: [.category]
}
}Fauna's answer to schema in the form of document types is flexible and adaptable to a wide array of application data models where previously there was a choice: SQL databases, while having powerful schema capabilities, require it at all times even when it is a hindrance. For certain development styles, this is an area where document databases have an advantage, where their flexible schemaless nature is far less restrictive when iterating on a new application or feature. Conversely, once a project has started with a document database, the benefits of a relational database's schema capabilities are unavailable.
Fauna confers the advantages of both. Development can start with schemaless collections in support of iterating quickly, and gradually adopt more constraints in the form of precise document types as an application matures. Or, for applications which need to work with structured semi-structured data, Fauna can be used for both, eliminating the need for secondary systems.
In the end this leads to increased development speed as well as reduced infrastructure complexity and costs.
A Zero-downtime migration system
Along with document types, we built a migration system which is capable of affecting large schema migrations safely, without downtime. We considered this to be a key component in making document types—as part of Fauna's overall schema capabilities—practical to use and enhance developer productivity. To understand why, it is worth taking a look at the respective approaches usually taken by relational and document databases.
For relational databases the set of columns and types of a table are an integral part of its definition, including its physical layout on disk. In terms of modeling, the table's definition provides a guarantee that data will always conform to the shape the application expects. Therefore, when a table is altered in a traditional SQL database, existing rows must be migrated to conform to the new definition. If it cannot be (for example, adding a non-nullable column with no default), the alter command must be rejected.
Furthermore, due to architectural limitations of most mainstream relational systems, it becomes difficult or impossible to change a table's definition as your dataset grows. The alter table command will lock the table to perform the migration in most cases, incurring significant downtime at scale. Database schema becomes locked in as your application grows, resulting in reduced developer productivity and increased cost and complexity: The database no longer works for the application but against it.
Document databases, if they do support schema in some form, take a far more relaxed approach euphemistically called "schema validation", which is a form of write-time validation since document databases remain schemaless by nature under the hood. When a collection's schema definition is changed in a document database, existing documents are left untouched.
It is then up to the developer to manually migrate existing documents afterwards. The overall result is more or less the schemaless state where document databases started from, where the application itself must take on or duplicate schema management and data migrations.
With Fauna, we took the stricter approach where schema is considered to be a contract between the database and the application. Whenever a collection definition changes, Fauna will migrate every existing document in the collection to conform to the new shape. These migrations are natively zero-downtime, with no risk of locking up a production database for some indeterminate amount of time. From the perspective of the application, the migration is instantaneous: Physical document data is lazily migrated on demand or incrementally in the background.
Despite this strict approach, Fauna preserves development agility and reduces the risk of operational mistakes. Fauna's zero-downtime migration system means that your application never has to choose between "maintenance mode"—being unavailable—or being locked in to a database schema that has long since been outgrown and is no longer a source of truth for your application or business as a whole.
Declarative, schema-as-code migrations
We designed Fauna's schema definition language (FSL) from the ground up to support a style of database management that is better aligned with the modern software development and operational management practices we call schema-as-code. With FSL, your database schema definitions live alongside your application source code and can be managed with the same pipeline-based deployment tooling your application code is.
Fauna's migration system is integrated with FSL in order to allow you to encode explicit migration steps as part of a collection's definition. In doing so, Fauna preserves the ability to manage your application schema as code while allowing your schema to express more complex migration actions than are possible in a purely declarative approach, such as renaming a field or backfilling a new field.
collection Product {
name: String
price: Number
category: "Tools" | "Apparel" | "Accessories"
description: String
quantity: Number
index byCategory {
terms: [.category]
}
migration "2024-04-15: rename desc, backfill quantity" {
rename desc description
backfill quantity = 0
}
}Fauna checks each new FSL submission for internal consistency as well as migration compatibility with the database's current schema, and if all checks pass, applies the schema update. In development, this workflow provides a great degree of simplicity, since schema can be uploaded multiple times without worrying about prior database state.
In critical environments such as production, Fauna's migration system plugs into Fauna's existing "protected" mode for databases which enforces a higher degree of safety. Potentially dangerous actions must be explicitly annotated in FSL code. For example, uploading a new collection definition which has fewer fields than the current one will be rejected unless the FSL includes an explicit migration statement marking the field as dropped.
Ultimately Fauna's schema-as-code approach, along with database protection, increases the safety of production environments, and allows you to leverage existing tooling such as established CI/CD pipelines to increase the security and auditability of database changes.
Conclusion
Before Fauna, developers had to choose between relational and document databases, each with their pros and cons:
While document databases accelerate early development with a minimal, JSON-based, schemaless design, their ad hoc approach to schema enforcement and consistency results in hidden complexity and data correctness issues down the line.
Where relational databases start from a principled foundation, their rigid requirements for schema that force up-front design choices as well as architectural limitations leading to locked-in, sub-optimal schema designs hamper the development of new apps and features, and make change more costly as applications scale.
Fauna's document-relational model provides the best aspects of both approaches. Overall, Fauna provides a rich set of capabilities—including its TypeScript-inspired query language FQL and its consistency and multi-region resiliency provided by its Distributed Transaction Engine—which work together to maximize developer productivity while maintaining a high degree of application correctness.
The end result lets developers implement higher quality applications with sophisticated feature sets, brought to market more quickly, with best-in-class operational reliability. As always, we're excited to see what you build with Fauna!
If you enjoyed our blog, and want to work on systems and challenges related to globally distributed systems, and serverless databases, Fauna is hiring!
Subscribe to Fauna's newsletter
Get latest blog posts, development tips & tricks, and latest learning material delivered right to your inbox.