Database CI/CD and Schema Migration with Amazon Aurora and GitHub
A series of articles about Database CI/CD and Schema Migration with Amazon Aurora, and is also applicable to Amazon Aurora Serverless.
- Database CI/CD and Schema Migration with Amazon Aurora
- Database CI/CD and Schema Migration with Amazon Aurora and GitHub (this one)
In the last article Database CI/CD and Schema Migration with Amazon Aurora, you have tried UI workflow in Bytebase.
This tutorial will bring your Amazon Aurora schema change to the next level by introducing the GitOps workflow, where you commit the schema change script to the GitHub repository, which will in turn trigger the schema deployment pipeline in Bytebase.
You can use Bytebase free version to finish the tutorial.
Features included
- GitOps Workflow
- Change History
Prerequisites
Before you start this tutorial, make sure you have the following ready:
- Followed our previous UI-based change tutorial Database CI/CD and Schema Migration with Amazon Aurora.
- An Amazon Aurora MySQL instance.
- A GitHub account.
- A public GitHub repository, e.g
aurora-test-bb-local
. - Docker installed locally.
- An ngrok account.
Step 1 - Run Bytebase in Docker and set the External URL generated by ngrok
ngrok is a reverse proxy tunnel, and in our case, we need it for a public network address in order to receive webhooks from VCS. ngrok we used here is for demonstration purposes. For production use, we recommend using Caddy.
-
Run Bytebase in Docker with the following command:
-
Bytebase is running successfully in Docker, and you can visit it via
localhost:8080
. Register an admin account and it will be granted theworkspace admin
role automatically. -
Login to ngrok Dashboard and complete the Getting Started steps to install and configure. If you want to use the same domain each time you launch ngrok, go to Cloud Edge > Domains, where you'll find the domain
<<YOURS>>.ngrok-free.app
linked to your account. -
Run the ngrok command
ngrok http --domain=<<YOURS>>.ngrok-free.app 8080
to start ngrok with your specific domain, and you will see the output displayed below: -
Log in Bytebase and click the gear icon (Settings) on the top right. Click General under Workspace. Paste
<<YOURS>>.ngrok-free.app
as External URL under Network section and click Update. -
Now you can access Bytebase via
<<YOURS>>.ngrok-free.app
.
Step 2 - Find your Amazon Aurora in Bytebase
-
Visit Bytebase Console through the browser via your ngrok URL. Log in using your account created from the previous tutorial.
-
If you followed the previous tutorial, you should see that the project and database created are still in your workspace.
Step 3 - Connect Bytebase with GitHub.com
- Click Settings on the top bar, and then click Workspace > GitOps. Choose GitHub.com and click Next.
-
Follow the instructions within STEP 2, and in this tutorial, we will use a personal account instead of an organization account. The configuration is similar.
-
Go to your GitHub account. Click your avatar on the top right, and then click Settings on the dropdown menu.
-
Click Developer Settings at the bottom of the left sidebar. Click OAuth Apps, and add a New OAuth App.
-
Fill Application name and then copy the Homepage and Authorization callback URL in Bytebase and fill them. Click Register application.
-
After the OAuth application is created, click Generate a new client secret. Copy the Client ID and the newly generated Client Secret, paste them back into Bytebase's Application ID and Secret.
-
Click Next. You will be redirected to the confirmation page. Click Confirm and add, and the Git provider is successfully added.
Step 4 - Enable GitOps Workflow with Amazon Aurora
-
Go to your project
TestAurora
, click GitOps, and choose GitOps Workflow. Click Configure GitOps. -
Choose
GitHub.com
- the provider you just added. It will display all the repositories you can manipulate. Chooseaurora-mysql-test-bb-local
. -
Keep the default setting, and click Finish.
Step 5 - Change schema for Amazon Aurora by pushing SQL schema change files to GitHub
-
In your GitHub repository
aurora-mysql-test-bb-local
, create a folderbytebase
, then create a subfolderTest
, and create a SQL file using the naming convention{{ENV_ID}}/{{DB_NAME}}##{{VERSION}}##{{TYPE}}##{{DESCRIPTION}}.sql
. It is the default configuration for the file path template setting under projectTestAurora
> GitOps.test/db_demo##202303100000##ddl##create_t2.sql
test
corresponds to{{ENV_ID}}
db_demo
corresponds to{{DB_NAME}}
202303100000
corresponds to{{VERSION}}
ddl
corresponds to{{TYPE}}
create_t2
corresponds to{{DESCRIPTION}}
Paste the sql script in it.
-
Commit and push this file.
-
Go to Bytebase, and go into project
TestAurora
. You’ll find there is a newPush Event
and a newissue 107
created. -
Click
issue/107
and go to the issue page. Click Resolve issue, and the issue will beDone
. You’ll see:- The issue is created via GitHub.com
- The issue is executed without approval because it’s on
Test
environment where manual approval is skipped by default. The Assignee isBytebase
, because the execution is automatic, and requires no manual approval. - The SQL is exactly the one we have committed to the GitHub repository.
- The Creator is
A
, because the GitHub user you use to commit the change has the same email address found in the Bytebase member list.
-
Click View change, you can view the schema diff.
-
Go to GitHub repository, and you will see besides your committed sql, there is a
.db_demo##LATEST.sql
file. Because you have configured Schema path template before, Bytebase will write back the latest schema to that specified path after completing the schema change. Thus you have access to an update-to-date full schema at any time.
Summary and What's Next
Now that you have tried the GitOps workflow, which stores your Amazon Aurora schema in GitHub and triggers the change upon committing change to the repository, to bring your Amazon Aurora change workflow to the next level of Database DevOps - Database as Code.
You can check out GitOps docs to learn more configuration details.
In the real world, you might have separated feature and main branches corresponding to your development and production environment, you can check out GitOps with Feature Branch Workflow to learn the setup. Have a try and look forward to your feedback!