Want to deploy your JVM, Node.js and Go apps effortlessly to AWS? Try our service Boxfuse

Flyway 4.1.0 Released

Welcome to 2017! And do we have some great things in store for this year! But first let's see how last year went.

Looking back at 2016

Like every year since Flyway's initial launch in 2010, last year was Flyway's best year ever. We had both the fastest growth and the highest number of downloads since we started. In 2016 Flyway was downloaded 1.6 million times!

Flyway 2016 Downloads

Flyway yearly downloads

Introducing Flyway 4.1.0

So let's start the year with a brand new release! Flyway 4.1.0 comes with new features, better performance and higher reliability.

These are the highlights of this release:

  • EnterpriseDB support
  • PostgreSQL non-transactional statement support
  • Improved MySQL replication support
  • Performance improvements when dealing with lots of migrations
  • Improved parallel migration of empty databases
  • Overall polish

Let's dive right in!

EnterpriseDB support

Once again Flyway expands its ever growing list of supported relational databases to almost 20 (!) and now comes with support for EnterpriseDB.

EnterpriseDB

PostgreSQL non-transactional statement support

While PostgreSQL supports DDL transactions, there are unfortunately a number of statements like CREATE INDEX CONCURRENTLY, ALTER TYPE ... ADD VALUE and VACUUM among others that are not able to run within a transaction. In fact in the case of CREATE INDEX CONCURRENTLY there may not even be any other transaction active for the operation the be able to complete!

Flyway 4.1.0 comes with two major improvements to support these statements in a fully transparent fashion.

First Flyway now autodetects whether a non-transactional statement is present within a migration. If it is, then the whole migration is run without a transaction (and clearly labelled as such in the logs).

In the spirit of fail-fast and to avoid accidentally mixing transactional and non-transactional statements within the same migration, we have also introduced a new allowMixedMigrations parameter. When deactivated it will cause Flyway to throw an error in those situations.

This however still doesn't work with CREATE INDEX CONCURRENTLY due to Flyway's support for migrating databases from multiple applications nodes in parallel. To avoid things getting messy, Flyway uses a separate connection with a careful SELECT ... FOR UPDATE for locking the metadata table to ensure consistency.

To address this situation we have now introduced a new single connection mode in combination with PostgreSQL advisory locks. Now only a single connection is being used for both managing the metadata table and applying migrations while preserving the concurrent migration capabilities we already had, while at the same time enabling support for all non-transactional PostgreSQL statements, including CREATE INDEX CONCURRENTLY.

From an end-user perspective all of this is fully transparent. Migrating a PostgreSQL database has never been easier!

Improved MySQL replication support

When running in a cluster MySQL supports two types of replication: statement-based and row-based. Flyway has always worked great with row-based replication. However starting with MySQL 5.6, MySQL statement-based replication has started producing warnings due to Flyway's use of MySQL's USER() function to record the name of the current user in the metadata table. The reason for those warnings is that this could produce different results on separate nodes.

To prevent this, Flyway has now introduced a new installedBy setting allowing you to avoid the use of the USER() function by manually overriding the name of the database user being recorded.

Performance improvements when applying thousands of migrations

For projects that have been using Flyway for a long time and have accumulated thousands of migrations, migrating to new versions would take longer and longer due to Flyway re-reading the metadata table every time. This then meant more and more records had to be returned over time. While never an issue for smaller deployments, this certainly started to become a burden for larger projects.

Starting with Flyway 4.1.0 we have introduced a new careful metadata table caching mechanism that provides identical performance when applying thousands of migrations, all the way from the first to the last.

Improved parallel migration of empty databases

While Flyway has always worked great with multiple application nodes by using careful SELECT ... FOR UPDATE metadata table locking to ensure consistency, this didn't work as well for a completely empty database where there were no records to lock or where not even the metadata table (or the schema!) had already been created.

Flyway 4.1.0 now comes with a number of internal implementation and retry strategies to ensure this now also works cleanly in all cases.

Overall polish

The improvements outlined above are by far not the only ones part of this release. Here are a few more:

  • The DB2 parser now supports changing delimiters via the --#SET TERMINATOR
  • SQLite support is now also based on the new single connection mode, avoiding all kinds of potential locking issues
  • HsqlDB 2.3.4+ compatibility has been restored by also moving it on the new single connection mode
  • Classpath scanning has been improved for Spring Boot executable wars
  • A number of repeatable migrations issues have been ironed out
  • Repair is now able to recover from more situations
  • And much more!

Special thanks

For this release we would also like to extend special thanks to JustWatch, Porch and Startrack for sponsoring the PostgreSQL improvements as well as classpath scanning fixes

JustWatch Porch Startrack

Grab it while it's hot!

Enjoy Flyway 4.1.0 and grab it while it's hot!

It is the fastest, most complete and most reliable Flyway release so far. All users are encouraged to upgrade.

Flyway is brought to you with by Axel Fontaine, Boxfuse and the many contributors.

P.S.: Spread the word, follow @flywaydb on Twitter and subscribe to our monthly newsletter below to be notified about updates.

« Flyway 4.0.2 Released