Database migrations

Database migrations can be easily integrated into your deploy system, running as a decoupled process, so it can be replaced anytime by other tools if needed, and working with it without interfering with the project itself.

The entire process can be isolated into a Docker container or the tools can be all installed directly on your machine. Presented setup is for CentOS.

Let’s assume the following context:

– Machine to run the migrations from (with Docker installed)
– MySQL database with access from the machine mentioned above
– A secrets manager to keep the database access credentials safe
– Git repository holding the migration files included (there will be a directory with all the migration files in the proper format).
– Private SSH key to access the above mentioned repository

Every time you deploy your app, you could run all the migrations you committed to your repository. Your deploy system should trigger the migration tool at the proper moment.

The key in this setup is migrate, a flexible tool which I had no problems with.
As presented in this Dockerfile, there are different tools used to perform each required step:
– Get migration files from the repository
– Get a secret string with database credentials from the secrets manager
– Extract the database credentials from the secret string
– Execute the migrations

Take a look at the full setup on Github.

Migrating an infrastructure

At the end of last year I was given the task to move an infrastructure spread on various providers (cloud and physical servers) to a new cloud. My role was to discover the entire infrastructure and services communication, come up with a strategy of how things are going to take place, then manage and supervise the sysadmin in charge of effectively moving machines.  This was a production environment, so minimum down time was desired. Most of the work was during night time.

It was a challenge from day zero, being a legacy multi component system with no specific flow. I had high level knowledge of the project, but subtle info came up every day. And I learned and used some things that I’d like to mention.

Planning everything ahead

I took each component (or group of components where required) individually and wrote down on a whiteboard their role, technologies, firewalls, running processes, storage info, communication flow with the other components, and a step-by-step process of how the migration should go. Continue reading Migrating an infrastructure