Notes on the MySQL 9.7 LTS release: Hypergraph Optimizer in Community, JSON Duality View DML, Group Replication observability, and Dynamic Data Masking for Enterprise.

Notes on the MySQL 9.7 LTS release

PostgreSQL 19 features I'm excited about: planner advisor framework, DDL extraction, online REPACK, pg_stat_recovery, parallel autovacuum, sequence sync, and 64-bit MultiXact.

PostgreSQL 19 features I'm excited about

What SOC 2 compliance audits actually want from your database audit log — not 'logging enabled' but evidence your auditor will accept. Lessons from going through our own audit.

SOC 2 Audit Log Requirements: Lessons From Our Own Audit

SOX Section 404 requires IT general controls over database changes. Learn the ITGC domains, six audit questions teams fail, and how to build SOX-ready controls

Database Change Management for SOX Compliance

The 10 best data management tools in 2026 — database GUI clients, schema migration tools, data pipelines, and analytics platforms compared with pricing.

Top 10 Data Management Tools in 2026

Postgres-only SQL editors are designed specifically for Postgres databases, providing a more focused and efficient experience for Postgres users. In this post, we are taking a look at the top Postgres-only SQL editors in 2026.

Enterprise Text-to-SQL: Context, Evaluation, and Governance

Database deployment tools for devops teams in 2026.

Database Deployment Tools for DevOps Teams in 2026

Postgres 19 widens MultiXactOffset to 64 bits, eliminating the hidden 4 billion member limit that caused catastrophic outages for high-concurrency workloads.

Postgres 19 Feature Preview: 64-bit MultiXactOffset

A guide to MySQL schema migration best practice.

MySQL Schema Migration Best Practice

Postgres 19 introduces automatic sequence synchronization for logical replication, eliminating manual tracking and making major version upgrades and failover significantly easier.

Postgres 19 Feature Preview: Sequence Synchronization for Logical Replication

A guide to open source sensitive data discovery tools — from NLP libraries like spaCy and Microsoft Presidio to multi-source scanners like Hawk-Eye and full metadata platforms like OpenMetadata.

Top Open Source Sensitive Data Discovery Tools in 2026

Top open-source Postgres backup solutions for 2026.

Open Source Postgres Connection Pooler

PostgreSQL 18 features that change how application developers write and ship code.

What's New in PostgreSQL 18: A Developer's Perspective

Top open-source Postgres authentication solutions for 2025.

Features I Wish Postgres Had but MySQL Already Has

Areas where PostgreSQL shines and MySQL falls short.

Features I Wish MySQL Had but Postgres Already Has

Evaluate several database schema diagram tools, and help you to choose the right one for 2025

Top 4 Free, Open Source Database Schema Diagram Tools to Visualize Database Easier in 2025

An overview of the most popular text-to-SQL AI tools in 2026, including DataGrip, TablePlus, SQL Chat, Vanna.ai, SQLAI.ai, Postgres.new, and DBHub MCP server. Compare features and capabilities to find the best SQL AI assistant for your workflow.

Top 5 Text-to-SQL Query Tools in 2026

Evaluate several TypeScript ORMs, and help you to choose the right one for 2025

Top TypeScript ORM 2025

PostgreSQL 18 features and what they change for running production databases.

What's New in PostgreSQL 18: A DBA's Perspective

Explore alternatives to Amazon RDS, including 

RDS Alternatives in 2025

Infrastructure as Code security tools help organizations identify vulnerabilities and misconfigurations in their infrastructure definitions. This post explores the top open-source IaC security tools available in 2025.

Top Open Source Infrastructure as Code (IaC) Security Tools in 2025

Compare the price of the mainstream managed PostgreSQL hosting providers

PostgreSQL Hosting Options in 2025: Pricing Comparison

Database governance tools help organizations manage database changes, enforce policies, and maintain data quality. This post explores the top open-source database governance tools available in 2025.

Top Open Source Database Governance Tools in 2025

Workflow orchestration tools help coordinate tasks, handle errors, and monitor execution processes. In this post, we are taking a look at open source workflow orchestration options

Top Open Source Workflow Orchestration Tools in 2025

Compare the price of the mainstream managed MySQL hosting providers

MySQL Hosting Options in 2025: Pricing Comparison

A comprehensive guide to the best open-source Kubernetes cluster management tools in 2025, including Argo CD, Karmada, Sveltos, and Plural.

Top 4 Open Source Kubernetes Cluster Management Tools in 2025

Review the state of the Golang ORM and query builder

Choose the Right Golang ORM or Query Builder in 2025

Overview of MySQL 9 features with spicy comments

What's New in MySQL 9 - a DBA's Perspective

SQL clients makes it safer and easier to manage databases. In this post, we are taking a look at open source SQL clients options for you to try in 2024.

Top Open-Source SQL Parsers in 2025

A comprehensive reference guide to standard port numbers used by popular database systems in 2025

Common Database Port Numbers in 2025

Explore the leading Terraform UI tools of 2026 that enhance IaC workflows with visualization, collaboration, and governance features to streamline infrastructure management.

Top 4 Terraform UI Tools 2026

A comprehensive guide to the best free and open-source PostgreSQL EXPLAIN tools for query performance analysis, including built-in extensions, online visualizers, and integrated GUI solutions.

Top Free, Open Source Postgres Explain Tool to Analyze Database 2025

Top open-source Kubernetes dashboard solutions include K9s, Lens, and KubeDashboard.

Top Free, Open Source Kubernetes Dashboard in 2026

In this post, we will review the top 5 open source Terraform tools for 2025.

Top 5 Open Source Terraform Tools for 2025

Postgres schema compare tool is used to diff and synchronize schemas between Postgres databases. This article reviews the top 4 common tools in this category.

Top 6 Open Source Infrastructure as Code (IaC) Tools 2025

How to Become a Platform Engineer in 6 months?

Top open-source database management solutions include Beekeeper Studio, Adminer, HeidiSQL, and Bytebase.

Top Open Source Database Management Software 2025

SQL editor with built-in AI streamlines database management, making SQL development faster, more accurate, and accessible to users of all skill levels.

Top 3 Free SQL Editors with Built-in AI

An extensive list of PostgreSQL SQL Review and Style Guide.

PostgreSQL SQL Review and Style Guide

Compare the best SQL Server monitoring tools for 2026 — free and paid. Includes SSMS built-in SQL monitors, open-source MSSQL monitoring tools (DBA Dash, SQLWATCH, Opserver), and commercial SQL performance monitoring solutions (Redgate, Datadog).

Top SQL Server Monitoring Tools: Best SQL Performance Monitors in 2026

In this post, we are taking a look at the open source, online SQL editors that you can self-host.

Top 5 Open Source, Online SQL Editors You can Self-host

Describe the top database devops tools in 2025.

Top Database DevOps Tools in 2025

Introduce the advanced tools for platform engineering. Covering CD, security, workflow orchestration, database development workflow, and internal development portal.

How to Pick Platform Engineering Tools in 2025 - Advanced Guide

Introduce the essential tools for platform engineering. Covering code hosting, CI/CD, observability, resource provisioning, and secret management.

How to Pick Platform Engineering Tools in 2025 - Essential Guide

Bastion gates connections. ITSM gates approval. PAM gates credentials. Three layers, three tools, one gap at execution — and what an integrated platform replaces.

Securing Database Change: From Bastion to Integrated Platforms

How OpenAI and DeepSeek designs SSN database schema.

OpenAI vs. DeepSeek: SSN Database Schema Design

DevSecOps tools empower teams to seamlessly integrate security into every phase of the software development lifecycle, fostering the adoption of DevSecOps practices.

Top DevSecOps Tools for 2025

What happened in the database tooling landscape in 2024.

Database Tools in 2024: A Year in Review

In a mystical land where data flows like rivers, four heroes — a warrior, a cleric, a druid, and a paladin — unite to tame the Hydra of Database DevSecOps, a powerful three-headed dragon embodying Velocity, Security, and Reliability.

The Guardians of Database DevSecOps

Some of the craziest database security vulnerabilities uncovered in DBAs' career

Database security vulnerabilities are the craziest

Due to Docker's recent requirement for a paid subscription for professional use, people began searching for alternatives. Right alternatives for you will depend on your specific needs and requirements.

Top 4 Free Docker Desktop Alternatives 2026

What are the top Postgres extensions you may need to enhance your Postgres instance.

Why Managed Database Service is not Enough

Six levels of database automation, named after the autonomous-driving framework. From L0 manual SQL to L5 self-healing — and where most teams plateau.

The 6 Levels of Database Automation

When team wants to enforce a process for database change management. A quick approach is to reuse their existing ITSM system such as Jira. Let's review how a typical Jira workflow looks like and its inherent limitations.

Hope Is Not a Strategy: Why Jira Falls Short for Database Change

A list of features I wish PostgreSQL had. It is mainly around improving development workflow and inspired by Git. While acquiring Git-like behavior is challenging in database because of the stateful nature.

Features I wish PostgreSQL had as a developer

In this post, we are reviewing several open-source GitOps tools.

Top Open Source GitOps Tools in 2025

When schema migration outage happens, who should be more responsible in order to prevent future incidents?

Schema Migration Outage: Author or Reviewer's Fault?

The "Modern Snowflake Stack" is inspired from the phrase "Modern Data Stack", refers to a set of technologies and tools built around Snowflake, a cloud-based data warehousing solution, to enable efficient data storage, processing, and analytics.

Modern Snowflake Stack in 2024

What are the top open source MySQL schema migration tools.

Top Open Source MySQL Migration Tools in 2024

This article explores leading database security tools designed to shield your data from varied risks.

Top Database Security Tools

A comparison of the top 5 MySQL GUI clients in 2026: MySQL Workbench, DBeaver, TablePlus, Beekeeper Studio, and Navicat.

Top 5 MySQL GUI Clients in 2026

What are the top open source Postgres schema migration tools.

How many DBAs should a company hire

DORA recently released its annual State of DevOps 2023 report. Bytebase is a Database CI/CD tool for DBAs and DevOps teams, and we are diving deep into the State of DevOps Report.

Notes on the DORA 2023 State of DevOps Report

DataGrip is an database IDE developed by JetBrains and is closely integrated with the JetBrains ecosystem. As a database admin tool, DataGrip is not a comprehensive alternative to the other dedicated database GUIs. Here, we are taking a look at top DataGrip alternatives.

Top DataGrip Alternatives 2023

If you work exclusively with MySQL, MySQL Workbench is an excellent choice for a database GUI, but if you work with multiple databases, you need to look elsewhere. This article highlights five database GUI alternatives for MySQL Workbench.

Top MySQL Workbench Alternatives 2023

Postgres is one of the most advanced open-source relational databases, while you can leverage Postgres' capabilities without a GUI, having one can take your experience working with Postgres to the next level.

Top pgAdmin Alternatives 2023

Navicat is a long-established database GUI. However, it has a serious limitation in multi-user scenarios where team collaboration and centralized control are needed. Hence, we are taking a look at top Navicat alternatives.

Top Navicat Alternatives 2023

DBeaver is one of the most popular database management tools out there. If for some reason, DBeaver doesn't fulfill your requirements or you are just in the mood to try something new. Here, we are taking a look at top DBeaver alternatives

Top DBeaver Alternatives 2023

If you work with data and databases, SQL AI tools can significantly improve your productivity and efficiency; if you want to start learning SQL, this is also an excellent way to get started.

Top 5 SQL AI Tools to Ease Writing SQL 2023

A Postgres GUI client gives you a visual interface to query, design, and administer PostgreSQL — instead of doing everything from psql. Here are the 7 most useful in 2026, with the trade-offs that decide which one fits your workflow.

Database as a Service (DBaaS) Provider 2023

In the post, we explore the Google Cloud SQL pricing structure, tools for estimating its cost, and tips for optimizing the cost.

Understanding Google Cloud SQL Pricing

MySQL schema compare tool is used to diff and synchronize schemas between MySQL databases. This article reviews the top 4 common tools in this category.

Top 4 MySQL Schema Compare Tool to Diff and Sync Database 2026

Database schema migration have come a long way, from CLI to GUI, and all the way to GitOps/Database-as-Code. Let's review the state-of-the-art of database schema change / migration tools in 2026.

Top Database Schema Migration Tools to Avoid Change Outage 2026

What happened in database industry in 2022.

Database Review 2022 and 2023 Prediction

The best free SQL clients and database management software in 2026. Compare DBeaver, Beekeeper Studio, DbGate, HeidiSQL, phpMyAdmin, pgconsole, and Bytebase — open-source SQL GUI tools for Mac, Windows, and Linux.

Top 7 Free, Open Source SQL Clients and Database Management Tools (2026)

To manually spin up several different kinds (and versions) of database instances for testing can be gruesome. Here are some free tools to help you start a local database instance on Mac.

Top 3 Free Tools to Start a Local Database Instance on Mac

Real talk on whether strict SQL review and schema conventions actually help — with answers from working developers and DBAs.

Is SQL Review Necessary? Real Talk From Developers

We wonder if ChatGPT can help us in the field of database administration, and we dug deep into some toolings that have incorporated ChatGPT to make our life easier.

A (Non-Exhaustive) Summary of ChatGPT Enhanced Database Tools

A conversation with ChatGPT consulting which database to pick for your next software project.

Ask ChatGPT - How to Choose the Right Database for Your Next Project

From the SQLite-insipred DuckDB to the real Snowflake challenger and the Docker for data.

MotherDuck, from SQLite to the Docker for Data

The idea of codifying database schema changes (aka schema migration) is not new. While we are observing an accelerated trend of embracing Database as Code (DaC) across the industry. In this blog, we will review the state-of-the-art in the Database as Code landscape and share our own insight into the current and future trends in this space.

The Database as Code Landscape

This is the first of the DDD series, consisting of 10 stories of DBA & Database & developer.

DBA, Database and Developer - The Decameron

What happened in database industry in 2021. OLAP (Snowflake, Databricks, ClickHouse, Firebolt, ClickHouse), OLTP (PostgreSQL, PlanetScale, SQLite) and open source.

Database Review 2021 and 2022 Prediction


💡 This article is a compilation of this discussion on Reddit: [Do DBAs impose strict requirements for database schema conventions at your org?](https://www.reddit.com/r/ExperiencedDevs/comments/xcswje/do_dbas_impose_strict_requirements_for_database/)

![_](/content/blog/is-sql-review-necessary/reddit.webp)

## Background Info

The author’s team was developing new microservices and they needed to store data in SQL tables. These tables are wholly owned by the microservice and data from them is only being exposed via an API. He gave 2 examples of DBAs requesting “unnecessary” restrictions that “can potentially cause confusion down the road”, so the question is, are strict requirements on database schema really necessary?

## DingBat99999 (30 ⬆)

Maybe I'm missing something but none of the things your DBAs have requested seem unreasonable.

When it comes to constraints, in my experience, there's typically some discussion as to whether they're enforced in the application or in the database. Was there some historical decision there?

I mean, you could always tell the DBAs shirt size is implementation dependent and dynamic. Then they'd have no reason to ask for a constraint. Otherwise, YAGNI.

**💬Reply**

Enforcing the enum constraint in the database is a good practice and it sounds like you are privileged to work with a competent DBA. Strict data checking in your persistence layer is the surest way of preventing garbage from entering your system

## ccb621 (12 ⬆)

Take all of this energy, and ask the DBAs why these changes are important. If the don't already have one, volunteer to help them write a guide for to help future teams avoid this toil.

The changes seem nitpicky on the surface, but there could be a lot of hard-learned lessons below the service that warrant this level of review.

## nutrecht (7 ⬆)

I think it's totally normal to have standards for DB schema's you need to adhere to. I've seen those before. I've also seen the mess that happens when developers just do whatever the heck they want, so I can't really blame them.

So both things, the enum and the naming, sound reasonable to me. The benefit of the enum constraint in the DB is that you can never somehow end up with 'broken' data that breaks your software. I've had to deal with 'test' databases where testers directly inserted data and it was a pain in the ass to deal with.

So there are good arguments for both sides. I personally would not spend _any_ energy fighting this. Especially since you're probably going to need this DBA team somewhere down the line.

Also as a Java/Spring developer; you should work schema first. If you need to adapt a schema 'because of Spring' it generally just means you're doing something wrong.

## ViperRT10Matt (6 ⬆)

Our DBAs do not perform any functions like this. They release and manage what the developers give them. In terms of requiring changes to schemas or table designs, they function like stability control in a car; only intervening in extreme conditions to head off a truly dangerous situation.

## AdrianC9 (3 ⬆)

I work at a mid/large tech company, we do have "DBAs" (don't think they are traditional DBAs) that review database changes, however most standards are published in a wiki that should reviewed before subbitting db changes. So typically you know what to expect. As an example, we do require tables to be named in singular form, eg "tshirt" exactly for the reason you described. The ORM can then easily map everything correctly.

In regards to the db constraints, a lot of DBAs will often prefer this since it puts more control in their domain. Personally I agree that throwing things that are arguably business logic into the database is not ideal, and may result in issues in the future. That said, it's really not that big of an issue, and whenever it does break due to a conflict you can just point to that requirement.

## nderflow (2 ⬆)

The column constraints in particular are important. I've seen a company lose teens of thousands of customers and lose millions because they disabled the column constraints they already had.

The fact that a database can enforce integrity rules is one of the beauties of the modern relational database.

## check-himu (2 ⬆)

The plural convention is something we also follow and I don't have very strong feelings either for or against it. However, from experience, I will tell you that enforcing constraints in DB can be extremely helpful. With changes in code, there is always a chance of a bug that lets data you didn't want pass through. A final check at DB helps you avoid that scenario and having to do data fix later on.

## db-master (2 ⬆)

It's a common practice to enforce some level of consistency. Many hours can be saved if the engineering org can agree on the same ID format, first name, last name format and etc.

FWIW, there are tools like [GitHub SQL Review Action](https://github.com/marketplace/actions/sql-review) to enforce schema consistency in the CI/CD process.

## Our Take on SQL Review

Bytebase has built-in [SQL Review](https://docs.bytebase.com/sql-review/review-policy) capabilities that are customizable to check for issues in the schema change process. To better accommodate developers' workflow, we also [integrated SQL Review with GitHub](/blog/integrate-sql-review-into-github), allowing you to manage SQL scripts in your GitHub repos and perform SQL Review there. This eliminates the need to switch between multiple tools and, more importantly, allows SQL Review to be conducted at the PR stage rather than waiting until the deployment stage.

That being said when it comes to database schema changes, the advice is simple: don't ask. If a DBA says so, just do it!

Last but not least - this article is a mere compilation of this [discussion](https://www.reddit.com/r/ExperiencedDevs/comments/xcswje/do_dbas_impose_strict_requirements_for_database/), don't miss out on the best practices and ideas colliding in the comment area ⚡️.



ChatGPT has only been around for less than half a year and we already cannot live without it. We wonder if ChatGPT can help us in the field of database administration, and we dug deep into some toolings that have incorporated ChatGPT to make our life easier.

## sqlTranslate

When it comes to databases, the first thing that comes to mind is whether ChatGPT can help us write SQL. [sqlTranslate](https://www.sqltranslate.app/) is a very simple tool that uses the OpenAI API to obtain the corresponding SQL statement by inputting natural language, or vice versa. You can also upload your own schema. Since it open-sourced three weeks ago, it has already gotten 2.6k stars and is [described by GitHub](https://twitter.com/github/status/1637937834865704960/photo/1) as an "incredibly useful weekend project". Looks like people have been suffering from SQL for way too long.

![_](/content/blog/chatgpt-enhanced-database-tools/sqltranslate.webp)

## AI2sql

[AI2sql](https://www.ai2sql.io/) is an AI-driven SQL query generator that has been around since 2021, and has recently incorporated OpenAI's GPT-3. Now engineers and non-engineers can write SQL easily without knowing the syntax. AI2sql is more comprehensive than sqlTranslate. The features include SQL syntax checking, formatting, and query generation, the databases supported covers the most popular ones on the market (such as MySQL, PostgreSQL, MongoDB, Oracle, etc.).

![_](/content/blog/chatgpt-enhanced-database-tools/ai2sql.webp)

## Aoi (葵)

[Aoi](https://github.com/shellfly/aoi) is a ChatGPT-driven dialogue agent program that can conduct natural language conversations with AI in your Terminal, and it can also be connected to your database to perform SQL tasks.

![_](/content/blog/chatgpt-enhanced-database-tools/aoi.webp)

## Bytebase

Bytebase is a database CI/CD tool that covers the entire life cycle of database development, and its SQL editor has also incorporated OpenAI's gpt-3.5-turbo in [its recent release](/changelog/bytebase-1-14-0), where you input natural language and have it converted to SQL. Stay tuned for a more advanced version chat bot 🤖️!

![_](/content/blog/chatgpt-enhanced-database-tools/bytebase-sqleditor.webp)

## DBeaver

As a veteran SQL client, in addition to visualization and management capabilities, [DBeaver](https://dbeaver.com/) also has a SQL editor, along with data & schema migration capabilities, and database connection monitoring. In early February, DBeaver also incorporated GPT-3 to have AI convert natural language to SQL. For instance, you can ask for "all invoices from Germany with a total of more than 4", or in German, "Zeig alle Rechnungen aus Deutschland mit der Gesamtsumme über 4," and it will automatically convert the request into a query.

![_](/content/blog/chatgpt-enhanced-database-tools/dbeaver.webp)

## OSSInsight

In a sense, [OSSInsight](https://ossinsight.io/) has been helping you write SQL since way before.

![_](/content/blog/chatgpt-enhanced-database-tools/ossinsight.webp)

But recently they embraced OpenAI and launched a new tool called "Data Explorer", making it easier to explore GitHub data. You can ask whatever interests you in plain language, and AI will generate SQL for them (and then query it).

![_](/content/blog/chatgpt-enhanced-database-tools/rust-or-go.webp)

The database behind OSSInsight is TiDB, and TiDB Cloud recently also launched an intelligent data exploration feature using OpenAI: [Chat2Query](https://www.pingcap.com/chat2query-an-innovative-ai-powered-sql-generator-for-faster-insights/), which is another tool that generates SQL queries based on your input and then queries and visualizes the database for you.

## Outerbase

_Outerbase has been [acquired by Cloudflare](https://blog.cloudflare.com/cloudflare-acquires-outerbase-database-dx/) in 2025_

[Outerbase](https://outerbase.com/) is a new tool that was officially released on Feb.15, 2023. Compared with traditional database management tools, Outerbase completely conforms to current aesthetics. Its user experience is similar to that of an Excel spreadsheet, and it uses GPT-3 to help users write SQL queries and generate dashboards, making it useful for both developers and data analysts.

![_](/content/blog/chatgpt-enhanced-database-tools/outerbase.webp)

## To wrap up

It looks like using ChatGPT to enhance data management and analysis for your databases is gonna be an SOP in the database industry. Of course, these tools are just the tip of the iceberg, and their integration with ChatGPT is still in its early stages. However, it is clear that the benefits of combining ChatGPT with database management tools are enormous. Moreover, besides text2sql, ChatGPT can be used in many other database-related fields, such as customer support, query assistance, database management, and data analysis (you can also consult ChatGPT about what else it's capable of).

DBAs: get ready to retire officially. Just kidding, but perhaps the long-established SQL clients such as Navicat are already feeling a bit nervous.



Last week, OpenAI revealed their new toy [ChatGPT](https://openai.com/blog/chatgpt/).
Suddenly, my social network is overwhelmed with all sorts of ChatGPT conversations. Regardless of
how weird the questions are, ChatGPT always sounds convincing, even though sometimes it's just
nonsense. Below I start a conversation asking ChatGPT the database I should choose for my next
software project. I picked this topic because:

1. Comparing different databases always causes [heated debate](https://www.bytebase.com/blog/database-review-2021).
   I am curious how the state-of-the-art AI handles this.
1. I know this domain decently well, so I could escape from the ChatGPT persuasive trap (hopefully).

## Scripts

**Me:** I want to start a new software project, which database should I choose, MySQL, PostgreSQL,
Mango, SQLite or others?

**ChatGPT:** There are many factors to consider when choosing a database for your software project,
including the type of project you are building, your budget, and the level of performance and
scalability you need. MySQL, PostgreSQL, MongoDB, and SQLite are all popular database options, each
with their own strengths and weaknesses.

**Me:** Tell me the strengths and weaknesses among them

**ChatGPT:** MySQL is a widely-used, open-source relational database management system (RDBMS) that
is known for its ease of use and strong performance. One of its main advantages is that it supports
a wide range of applications, from small websites to large, complex systems. However, MySQL is not
always the best choice for high-performance applications, and it does not support some advanced
database features such as document-level data handling and native graph data storage.

PostgreSQL is another popular open-source RDBMS that is known for its reliability, stability, and
support tore wide range of data types. It is often used in mission-critical applications, and it
supports advanced features such as data replication and concurrency without read locks. However,
PostgreSQL can be more complex to set up and manage than some other database options, and it may not
be the best choice for applications that require very high performance.

MongoDB is a popular open-source NoSOL database that is known for its flexibility and scalability.
It is often used in applications that require the ability to store and query large volumes of
unstructured data, and it supports advanced features such as distributed data storage and indexing.
However, MongoDB can be more complex to use than some other database options, and it may not be the
best choice for applications that require ACID (atomic, consistent, isolated, durable) transactions.

SQLite is a widely-used, open-source RDBMS that is known for its lightweight design and good
performance. It is often used in embedded applications, such as mobile apps, and it does not require
a separate server process to operate. However, SQLite does not support advanced database features
such as data replication and distributed data storage, and it may not be the best choice for
applications that require high levels of performance or scalability.

Ultimately, the best database for your project will depend on your specific requirements and goals.
It's a good idea to research and compare different database options carefully before making a
decision.

**Me:** OK, I will go with PostgreSQL, which cloud provider should I choose, AWS, GCP or Microsoft
Azure

**ChatGPT:** All three of the major cloud providers-- Amazon Web Services (AWS),
Google Cloud Platform (GCP), and Microsoft Azure -- offer support for PostgreSQL. Each of these
platforms has its own strengths and weaknesses, so the best choice for you will depend on your
specific needs and requirements.

AWS offers a fully managed version of PostgreSQL called Amazon RDS
for PostgreSQL, which makes it easy to set up, operate, and scale a PostgreSQL database in the
cloud. AWS also offers a range of other services and tools that can be used in conjunction with
PostgreSQL, such as Elastic Load Balancing for high availability and Amazon Redshift for data
warehousing.

GCP offers a fully managed version of PostgreSQL called Cloud SOL for PostgreSQL, which makes it
easy to set up, operate, and scale a PostgreSQL database in the cloud. GCP also offers a range of
other services and tools that can be used with PostgreSQL, such as BigQuery for data analytics and
Cloud Pub/Sub for event-driven architecture.

Microsoft Azure offers a fully managed version of PostgreSQL called Azure Database for PostgreSQL,
which makes it easy to set up, operate, and scale a PostgreSQL database in the cloud. Azure also
offers a range of other services and tools that can be used with PostgreSQL, such as Azure Synapse
Analytics for data warehousing and Azure Cosmos DB for globally distributed data.

In summary, all three of these cloud providers offer robust support for PostgreSQL and a range of
other services that can be used in conjunction with a PostgreSQL database. The best choice for you
will depend on your specific needs and requirements, as well as any existing investments you may
have in a particular cloud platform. It's a good idea to research and compare the offerings from
each provider carefully before making a decision.

**Me:** What are the available regions and respective prices for each cloud vendor?

**ChatGPT:** I'm sorry, but I am not able to browse the internet, so I cannot provide you with
information about the available regions and prices for each cloud provider. I am a large language
model trained by OpenAl, and my knowledge is based on the text that has been fed to me. I am not
able to access real-time information or browse the internet like a human.

_Luckily, ChatGPT is not capable of answering this, otherwise, our [DB Cost](https://dbcost.com) website would be wasted_.

**Me:** If I want to combine OLTP and OLAP together, what's your recommendation? (_Poke some recent development in the database industry_)

**ChatGPT:** If you want to combine online transaction processing (OLTP) and online analytical
processing (OLAP) in your database, one option is to use a hybrid transactional/analytical
processing (HTAP) database. HTAP databases are designed to support both OLTP and OLAP workloads
simultaneously, allowing you to perform real-time analytics on your transactional data without
having to replicate or move the data to a separate OLAP system.

There are several HTAP databases available, including MemSQL, NuoDB, and ClustrixDB. These databases
combine the performance and scalability of in-memory architectures with the reliability and
durability of disk-based storage, making them well-suited for applications that require real-time
analytics on high-volume, transactional data.

Another option is to use a data warehousing solution, such as Amazon Redshift, Google BigQuery, or
Microsoft Azure Synapse Analytics. These solutions allow you to load and store your transactional
data in a separate, optimized environment for analytics, while still providing real-time access to
the data for reporting and other business intelligence needs.

Ultimately, the best approach for combining OLTP and OLAP in your database will depend on your
specific requirements and goals. It's a good idea to research and compare different solutions
carefully before making a decision.

**Me:** I want a free, HTAP, Serverless, MySQL and PostgreSQL compatible database, what do you
recommend? (_Being intentionally unreasonable and want to see how ChatGPT would respond_)

**ChatGPT:** I'm sorry, but it is unlikely that you will be able to find a free, HTAP, serverless,
MySQL and PostgreSQL compatible database. Hybrid transactional/analytical processing (HTAP)
databases are designed to support both online transaction processing (OLTP) and online analytical
processing (OLAP) workloads simultaneously, and they typically require specialized hardware and
software to operate. Additionally, MySQL and PostgreSQL are both widely-used, open-source relational
database management systems (RDBMS), and it is unlikely that any free database solution would be
compatible with both of them.

If you are looking for a free database solution, you may want to consider using SQLite, which is a
lightweight, open-source RDBMS that does not require a separate server process to operate. SQLite is
often used in embedded applications, such as mobile apps, and it is capable of supporting small to
medium-sized datasets. However, it does not support advanced database features such as distributed
data storage and indexing, and it may not be suitable for applications that require high levels of
performance or scalability.

Alternatively, you could consider using a cloud-based database solution, such as Amazon Web Services
(AWS) Aurora Serverless, which is a fully managed, serverless, MySQL and PostgreSQL compatible
database. AWS Aurora Serverless can automatically scale up or down based on your workload, allowing
you to pay only for the compute and storage resources you use. However, AWS Aurora Serverless is
not a HTAP database, and it is not free. It is offered on a pay-as-you-go basis, with pricing based
on the amount of storage, compute, and I/O resources you use.

In summary, while it is possible to find free database solutions, it is unlikely that you will be
able to find a free, HTAP, serverless, MySQL and PostgreSQL compatible database. The best solution
for your needs will depend on your specific requirements and goals, as well as your budget and level
of technical expertise. It's a good idea to research and compare different options carefully before
making a decision.

**Me:** OK, what if I only need either MySQL or PostgreSQL compatible? (Soften the requirement a bit)

**ChatGPT:** If you only need a database that is compatible with either MySQL or PostgreSQL, you may
want to consider using a cloud-based database solution, such as Amazon Web Services (AWS) Aurora,
Google Cloud SQL, or Microsoft Azure Database for MySQL or PostgreSQL. These solutions are fully
managed, scalable, and compatible with the MySQL and PostgreSQL open-source databases, respectively.
They allow you to set up, operate, and scale a MySQL or PostgreSQL database in the cloud without
having to manage the underlying infrastructure yourself.

AWS Aurora is a fully managed, scalable, MySQL and PostgreSQL compatible database that is offered on
a pay-as-you-go basis. It can automatically scale up or down based on your workload, allowing you to
pay only for the compute and storage resources you use. AWS also offers a serverless version of
Aurora called Aurora Serverless, which can automatically scale up or down based on demand, and does
not require you to provision any capacity in advance.

Google Cloud SQL is a fully managed, scalable, MySQL and PostgreSQL compatible database that is
offered on a pay-as-you-go basis. It allows you to set up, operate, and scale a MySQL or PostgreSQL
database in the cloud, and it offers high availability and disaster recovery options to protect your
data.

Microsoft Azure Database for MySQL and PostgreSQL are fully managed, scalable, MySQL and PostgreSQL
compatible databases that are offered on a pay-as-you-go basis. They allow you to set up, operate,
and scale a MySQL or PostgreSQL database in the cloud, and they offer a range of options for
managing and securing your data.

In summary, if you only need a database that is compatible with either MySQL or PostgreSQL, there
are many cloud-based options available. These solutions are fully managed, scalable, and compatible
with the open-source MySQL and PostgreSQL databases, respectively. They allow you to set up,
operate, and scale a MySQL or PostgreSQL database in the cloud without having to manage the
underlying infrastructure yourself. The best option for you will depend on your specific
requirements and goals, as well as your budget and level of technical expertise. It's a good idea to
research and compare different solutions carefully before making a decision.

**Me:** But those are for OLTP, while I ask for an HTAP, serverless database. (_Keep pressing again_)

**ChatGPT:** I'm sorry, but the cloud-based database solutions mentioned earlier, such as Amazon
Web Services (AWS) Aurora, Google Cloud SQL, and Microsoft Azure Database for MySQL or PostgreSQL,
are not hybrid transactional and analytical processing (HTAP) databases. They are designed to support
online transaction processing (OLTP) workloads, and they do not support online analytical
processing (OLAP) workloads.

HTAP databases are designed to support both OLTP and OLAP workloads simultaneously, allowing you to
perform real-tints analytics on your transactional data without having to replicate or move the data
to a separate OLAP system. However, HTAP databases are not typically offered as a fully managed,
serverless solution in the cloud. They require specialized hardware and software to operate, and
they are typically deployed on-premises or in a private cloud.

If you are looking for a free, HTAP, serverless database, it is unlikely that you will be able to
find one. HTAP databases are complex, high-performance systems that require specialized hardware and
software to operate, and they are not typically offered as a free or serverless solution. If you
need a HTAP database, you may want to consider using a commercial HTAP database solution, such as
MemSQL, NuoDB, or ClustrixDB. These solutions are not free, but they offer a fully managed
scalable, HTAP database that can be used on-premises or in a private cloud.

In summary, the cloud-based database solutions I mentioned earlier, such as AWS Aurora, Google Cloud
SQL, and Azure Database for MySQL or PostgreSQL, are not HTAP databases. They are designed to
support OLTP workloads, and they do not support OLAP workloads. It you are looking tore free, HTAP,
serverless database, it is unlikely that you will be able to find one. You may want to consider
using a commercial HTAP database solution instead.

**Me:** Alright, good try. You are spared.

**ChatGPT:** Thank you for your understanding. As a large language model trained by OpenAl, my goal
is to provide accurate and helpful information to the best of my ability. I am not able to browse
the internet or access real-time information, so I may not always be able to provide detailed or
up-to-date answers to your questions. If you have any further questions, please don't hesitate to
ask. I will do my best to help.

## Overall Impression

I am impressed.

ChatGPT can understand and engage the conversation like a real human. For such a
technical question, ChatGPT's answer is mostly sane, albeit a bit out-dated. e.g. When asking for
the HTAP databases, it mentions MemSQL (which has already been rebranded to [SingleStore](https://www.singlestore.com/blog/revolution/)) and dissolved NuoDB, ClustrixDB, but ignores the starlet like [TiDB](https://www.vldb.org/pvldb/vol13/p3072-huang.pdf). This is understandable as ChatGPT confesses that its knowledge cutoff is September 2021,
after I ask it to brief the AWS re:Invent 2022 happened just days ago.

![_](/content/blog/how-you-choose-database-by-chatgpt/aws-reinvent.webp)

I believe the knowledge freshness gap will be shortened dramatically in the coming years, just like
the data analysis industry, where those analysis taking days and weeks before now just take minutes
if not instant.

ChatGPT's tone is also convincing and diplomatic. e.g. When talking about MongoDB, it says

> it may not be the best choice for applications that require ACID (atomic, consistent, isolated,
> durable) transactions.

It didn't say MongoDB doesn't support ACID (which would be incorrect). In real database debate,
such subtle mistake will cost the whole battle.

As ChatGPT progresses, I expect it could give database suggestions as good as any database veterans.
And we may all retire at that point regardless of whether willingly or not.



## First Encounter

The other day, I stumbled across [motherduck.com](https://motherduck.com). The website was rough.

![_](/content/blog/motherduck-from-sqlite-to-the-docker-for-data/motherduck-old.webp)

While with an all-star team.

![_](/content/blog/motherduck-from-sqlite-to-the-docker-for-data/motherduck-team.webp)

From its slogan "Data Infrastructure and Analytics" and the team background, I guessed they want to challenge Snowflake and/or Databricks. But how? Building yet another cloud-native warehouse with claimed performance/price benefits is meh. And MotherDuck is quite unusual name. Why would they choose it?

Today, after reading their [funding post](https://motherduck.com/blog/announcing-series-seed-and-a/) (together with an expected website redesign), I now have a better understanding to answer those questions.

## Naming

The name MotherDuck comes from [DuckDB](https://duckdb.org/), the SQLite for Analytics. MotherDuck commercializes the open-source DuckDB, which is a standard infra startup playbook nowadays.

## Manifesto

![_](/content/blog/motherduck-from-sqlite-to-the-docker-for-data/manifesto.webp)

I was confused by the serverless part at first. Though serverless is a much-abused word, most people would still pair serverless with cloud. MotherDuck markets itself as serverless, on the other hand, it says not to wait for the cloud, and then they probably would still offer a cloud service (who wouldn't?).

In common sense, serverless means there are still servers that are masked by the cloud service providers. Servers exist, users just don't need to care about them. While MotherDuck's serverless is a different thing, there is no server at all, apparently, since the underlying DuckDB is just a library to be embedded instead of a server on its own.

## Democratizing Data

Snowflake brings the novel idea to separate the compute from storage, this architecture innovation grants it a huge competitive edge over the years. While from the product perspective, they are still tightly coupled since the data is locked in the Snowflake platform.

MotherDuck is different. Say you have a single data file, Parquet, CSV, SQLite or whatever format, that file resides on your local disk, S3, GitHub or anywhere. Then you use MotherDuck to mount that file from the computing environment of **YOUR CHOICE**, now you have a powerhouse ready to analyze that file. Because MotherDuck's zero dependency (thanks to SQLite-inspired DuckDB), it just takes seconds to acquire the MotherDuck binary (and even this will be eliminated after the pre-built distro includes it).

![_](/content/blog/motherduck-from-sqlite-to-the-docker-for-data/snowflake-vs-motherduck.webp)

While Snowflake separates the compute from storage, MotherDuck taps the compute into the storage. With MotherDuck, as long as you can access the data file, you have the data powerhouse, and beyond that data powerhouse, you can build data solutions.

![_](/content/blog/motherduck-from-sqlite-to-the-docker-for-data/ossinsight.webp)

Take a real example, [OSSInsight](https://ossinsight.io/), a site to ingest GitHub events in real-time and offer insights. While its tech stack has already been [simplified by adopting TiDB](https://ossinsight.io/blog/why-we-choose-tidb-to-support-ossinsight/), the future MotherDuck could probably deliver a similar version with an even simpler stack.

**Because with MotherDuck, all application runtime artifacts can be captured in a single file.**

This is not a new idea though. The legendary [Hypercard](https://en.wikipedia.org/wiki/HyperCard) introduced this, to store a self-contained application and data into a single file.

![_](/content/blog/motherduck-from-sqlite-to-the-docker-for-data/hypercard.webp)

Also, the long-standing [FileMaker Pro](https://en.wikipedia.org/wiki/FileMaker), where the entire app and data are stored in a single .fmp12 file.

![_](/content/blog/motherduck-from-sqlite-to-the-docker-for-data/filemaker.webp)

Yet MotherDuck could bring this idea to a new level, being capable of processing terabytes of data efficiently, defining open protocols among Dataset/MotherDuck Engine/MotherDuck Platform, integrating with GitHub to promote collaboration and etc.

![_](/content/blog/motherduck-from-sqlite-to-the-docker-for-data/snowflake-or-motherduck.webp)

Immediately, MotherDuck starts competing with Snowflake on all fronts. Data teams still need [Fivetran](https://www.fivetran.com/) to move the data around, [dbt](https://www.getdbt.com/) to transform the data back and forth, even [meltano](https://meltano.com/) to assemble the platform, but for every analytic task, they will have to decide where to run it from, the Snowflake Cloud or the whatever computing environment at hand with MotherDuck.

And in the long horizon, it's all uncharted. MotherDuck could unleash all kinds of new data solutions. Only sky is the limit.

![_](/content/blog/motherduck-from-sqlite-to-the-docker-for-data/snowflake-next.webp)

MotherDuck's idea is novel, also the timing is perfect. The data analytics space is waiting for the next paradigm shift. While practitioners are content with better Snowflake alternatives, visionaries demand more. Instead of competing on the performance/cost frontier, MotherDuck chooses a different battleground, very wisely.

## Parting Thought

People give away the data to Snowflake alike for the processing power. With MotherDuck, they can acquire similar processing power while still controlling the data themselves. They can keep the data wherever they want, share it with whoever they like, and use it whenever they need.

**While Docker has become the runtime and standard to make applications universally accessible, MotherDuck could become the runtime and standard to make data universally accessible.**

And this time, MotherDuck must be better prepared than [Docker, Inc](https://www.infoworld.com/article/3632142/how-docker-broke-in-half.html), to get the lion's share if the vision is realized. Good luck team MotherDuck and just wondering would `Ducker` make a better name :)



This is a series of articles about database version control and database-as-code (GitOps)

1. [What is Database Version Control?](/blog/database-version-control)
1. [Database Version Control, State-based or Migration-based?](/blog/database-version-control-state-based-vs-migration-based)
1. [Database as Code - the Good, the Bad and the Ugly](/blog/database-as-code)
1. The Database as Code Landscape (this one)
1. [Database Version Control Best Practice](/blog/database-version-control-best-practice)

---

The idea of codifying database schema changes (aka schema migration) is not new. Many engineering teams have employed Liquibase / Flyway to adopt this practice. At the same time, we are observing an accelerated trend of embracing [Database as Code (DaC)](/blog/database-as-code) across the industry. Several innovative products have also emerged to challenge the incumbents.

In this blog, we will review the state-of-the-art in the Database as Code landscape and share our own insight into the current and future trends in this space.

## The Overview

## The Gatekeepers

### Liquibase

Started in 2006, Liquibase is arguably the most well-known product in this segment. When someone asks for database schema change advice on a forum, oftentimes, you will see a reply mentioning Liquibase.

![_](/content/blog/database-as-code-landscape/liquibase-overview.webp)

Liquibase is both an open source project as well as a company providing its commercial offering. The company used to be called Datical, and was renamed Liquibase to consolidate the branding (wise move).
Liquibase main product is a Java-based CLI. Via the CLI, developer teams can integrate the database schema migration into their CI/CD workflow. For the Java application, Liquibase can also be used as a library. Applications usually embed Liquibase library to apply any applicable schema migration on startup.

In Liquibase, a schema migration unit is encapsulated in a `Change Set`. Probably due to its age and root in Java, the most commonly used form is XML (YAML and JSON support have been added later):

![_](/content/blog/_shared/liquibase-xml.webp)

Plain SQL is also supported with proper annotation:

![_](/content/blog/_shared/liquibase-sql.webp)

Liquibase is [migration-based](/blog/database-version-control-state-based-vs-migration-based). It records the incremental change instead of the desired end state of the target database schema.

Recently, Liquibase introduced HUB, it's an information portal for paid customers to view, organize and monitor database change activity in real-time.

![_](/content/blog/database-as-code-landscape/liquibase-hub.webp)

### Flyway

![_](/content/blog/database-as-code-landscape/flyway-overview.webp)

Flyway resembles Liquibase in many ways. It's an open source project with a long history and a large customer base. Its core product includes the CLI and a Java library. Flyway also recently announced a web portal named Hub.

The commercial entity behind Flyway is Redgate through an acquisition. The branding may cause some confusion, while it also sets a boundary between the open source and commercial offering. Flyway carries a more casual branding tone:

![_](/content/blog/_shared/flyway-migration.webp)

Flyway website is not as shining as those new DevTools companies using sophisticated gradients. You can even spot some basic UI spacing issues. Yet Flyway shows a classic example that content matters more than cosmetic beauty and its documentation stands out because of the clarity.

Liquibase and Flyway are neck and neck. The major difference lies in their respective positioning. Liquibase caters to enterprise customers, while Flyway feels more accessible to the developers.

## Sqitch

Sqitch is also an open source project been on the market for a while. Sqitch is pure CLI-based and has no UI. Unlike Java-based Liquibase/Flyway, Sqitch is developed in Perl.

![_](/content/blog/database-as-code-landscape/sqitch-overview.webp)

Choosing Perl is not the only interesting aspect. Sqitch has its own design philosophy about how to manage database schema changes. Both Liquibase and Flyway use the file naming convention to control the schema migration behavior (convention over configuration):

![_](/content/blog/database-as-code-landscape/flyway-migration-version.webp)

While Sqitch takes an explicit approach. In the below example, you need to name the schema migration as `appschema`:

![_](/content/blog/database-as-code-landscape/sqitch-migration.webp)

If you later want to apply another migration depending on `appschema`, you need to supply with `--requires`:

![_](/content/blog/database-as-code-landscape/sqitch-migration-dependency.webp)

Sqitch also provides the unique tag and bundle command for the schema change deployment, which grants teams more power to wire the schema deployment into the application development lifecycle.

## The Starlets

### Atlas

![_](/content/blog/database-as-code-landscape/atlas-overview.webp)

Atlas is a new database schema migration tool that debuted recently. It takes traits from existing tools:

- It resembles Liquibase / Flyway / Sqitch in that it's mainly CLI focused, though it also has a lightweight UI to visualize the schema.
- It resembles Prisma in that it designs its own Domain Specific Language (DSL) to define the database schema. Originally, it chooses the same state-based approach as Prisma, but looks like it also added the migration-based approach.

![_](/content/blog/database-as-code-landscape/atlas-migration.webp)

- It's also heavily influenced by HashiCorp Terraform, using Go and inventing its own DSL. They even debuted as Terraform for Database Migrations on Hacker News:

![_](/content/blog/database-as-code-landscape/atlas-terraform.webp)

Altas current focus is on CLI. While stacking it against existing CLI tools like Liquibase / Flyway, it has the advantage of employing a modern programming language (Go instead of Java), and learnings from tools like Terraform and Prisma.

## The ORMs

Database schema migration is an integral part of the application development, thus it's no surprise popular application frameworks and ORMs provide the built-in schema migration capabilities:

- [Ruby on Rails Active Record Migrations](https://guides.rubyonrails.org/active_record_migrations.html) (Ruby)
- [Django Migrations](https://docs.djangoproject.com/en/4.0/topics/migrations) (Python)
- [MyBatis Migrations](https://mybatis.org/migrations) (Java)
- [GORM Migrations](https://gorm.io/docs/migration.html) (Go)

Though database schema migration falls into the backend category, the most recent innovation comes from Prisma, an ORM with a frontend root.

### Prisma

![_](/content/blog/database-as-code-landscape/prisma-overview.webp)

Node.js unlocks the pandora box for frontend engineers to build full-stack applications. We call it pandora box because it brings both prosperity and chaos. Full-stack applications consist of 3 pieces:

1. The code (stateless part).
1. The data (stateful part).
1. And how the code interacts with the data.

To a typical frontend engineer wanting to build a sustainable full-stack application, TypeScript / Express.js is solving the code problem. NoSQL databases like MongoDB are solving the data problem. Prisma is targeting the last problem of wiring code with data.

For frontend engineers, usability and accessibility are especially important. They are used to perfect pixels and HCI, and are not versed in composing SQL. To lower the barrier of managing database schemas, Prisma specifically designs a DSL:

![_](/content/blog/database-as-code-landscape/prisma-schema.webp)

The DSL takes the state-based approach (declarative), in that the DSL describes the desired end state of the targeting database schema instead of the incremental changes. This is different from Liquibase / Flyway / Sqitch.

Prisma also invests heavily in the integration and development workflow.

![_](/content/blog/database-as-code-landscape/prisma-stack.webp)

![_](/content/blog/database-as-code-landscape/prisma-lifecycle.webp)

Unlike its backend ORM peers, Prisma takes a refreshing approach and offers a more holistic view of managing database data and schema throughout the application development lifecycle.

![_](/content/blog/database-as-code-landscape/prisma-product-line.webp)

By looking at their product line, you can easily tell their ambitions are definitely not limited to being an ORM and schema migration tool.

## The Database

All tools described above work with many different database systems. While the application of Database as Code is database agnostic, there emerge new database systems promoting developer experience as a key differentiator. This developer-centric perspective as well as features like branching / cloning aligns with the mindset behind Database as Code.

### PlanetScale

![_](/content/blog/database-as-code-landscape/planetscale-overview.webp)

PlanetScale used to be a Vitess hosted service. They pivoted last year to focus on the developer workflow. Though you may still come across some Vitess footprint on their website, most new materials are talking about database development workflow features such as branching, online schema change, and the recently introduced rewind:

![_](/content/blog/database-as-code-landscape/planetscale-rewind.webp)

When we covered Prisma above, we mentioned there are 3 pillars constituting a full-stack application:

1. The code (stateless part).
1. The data (stateful part).
1. And how the code interacts with the data.

For the data part, MongoDB is a popular choice, people choose it because of its usability rather than ubiquity. For hosting mission-critical application data such as user info, customer order, and billing, you still want a SQL-based relational database. PlanetScale is based on MySQL, enhanced with Vitess scalability, and crafted for usability by the amazing PlanetScale product team. These properties make PlanetScale an attractive choice. It seems to combine the best of both worlds, retaining all the SQL database benefits while not sacrificing developer experience.

### Neon

Neon is another open source database just announced.

![_](/content/blog/database-as-code-landscape/neon-overview.webp)

At a high level, it shares a lot of similarities with PlanetScale, serverless, developer productivity, fully managed.

![_](/content/blog/database-as-code-landscape/neon-about.webp)

The major difference lies in the technology stack:

- Neon is on the PostgreSQL camp while PlanetScale is MySQL-based.
- PlanetScale leverages Vitess to offer its advanced database features, while Vitess itself is a middleware on top of the MySQL server node. Neon takes a more drastic approach to build a dedicated storage layer to offer those desired features.

Neon could have higher potential because it has better control over the entire stack. It would be very interesting to see how Neon progresses compared to PlanetScale.
The love-hate relationship between MySQL and PostgreSQL is never-ending, from vanilla MySQL vs PostgreSQL, distributed database TiDB vs CockroachDB, cloud database AWS Aurora vs GCP AlloyDB, and now to the developer-centric database PlanetScale vs Neon.

## The Misfits

### Dolt

![_](/content/blog/database-as-code-landscape/dolt-overview.webp)

Dolt is unique on its own. The website describes it as a database system with Git-like version control.

![_](/content/blog/database-as-code-landscape/dolt-about.webp)

Dolt is drastically different from other database systems because it builds an engine with built-in Git semantics. This is a double-edged sword, on one hand, it brings those desired Git properties, on the other hand, it imposes challenges of optimizing the typical OLTP workload.

Dolt takes a much bolder move to adopt Database as Code since it blurs the lines between database and code. It currently positions itself as bringing the Git semantics into the database. Would the other way around work as well or even better, bringing the database semantics into Git? After all, many applications have already used Git as the only data store.

## The Bytebase

![_](/content/blog/database-as-code-landscape/bytebase-overview.webp)

Bytebase is an open source project that started in Jan 2021. It also shares some traits with the aforementioned products:

- Like Liquibase / Flyway / Sqitch, Bytebase uses migration-based approach to apply database schema changes.
- Like Prisma, Bytebase offers a fully-fledged web UI to conduct database development activities (schema, data change, SQL query).
- Like Atlas, Bytebase employs a modern tech stack, using Go instead of Java.
- Like PlanetScale / Neon, Bytebase focuses on developer experience. One example is the VCS integration, Bytebase provides a point-and-click wizard to configure VCS (the similar experience you find in Vercel / Netlify to link the code repository):

![_](/content/blog/database-as-code-landscape/bytebase-vcs.webp)

Bytebase is a jack-of-all-trades and the master of team collaboration:

- A standalone developer team can host a Bytebase workspace to manage database development activities.
- A DBA or platform team can also host a Bytebase workspace to be used by all the product teams in the entire organization.
- It has native VCS integration to streamline the code and database development together.
  Bytebase has concepts Project, Issue because it resembles systems like Jira, GitLab / GitHub.

![_](/content/blog/database-as-code-landscape/bytebase-issue-detail.webp)

It also has concepts `Tenant`, `Environment` to manage database changes for multi-tenant applications spanning multiple environments.

![_](/content/blog/database-as-code-landscape/bytebase-tenant.webp)

Bytebase also provides built-in schema review policies to enforce the schema standard across the whole engineering organization.

![_](/content/blog/database-as-code-landscape/bytebase-schema-review.webp)

Bytebase is envisioned to be the counterpart of GitLab / GitHub to manage the database development aspect. It's positioned as a Database DevSecOps solution for team collaboration.

![_](/content/blog/database-as-code-landscape/bytebase-position.webp)

How Bytebase compares against other tools:

|           | Language        | Change Format           | Genre                         | Interface | VCS Integration | Collaboration | Multi-tenancy |
| --------- | --------------- | ----------------------- | ----------------------------- | --------- | --------------- | ------------- | ------------- |
| Liquibase | Java            | XML / YAML / JSON / SQL | Migration-based               | CLI       |
| Flyway    | Java            | SQL                     | Migration-based               | CLI       |
| Sqitch    | Perl            | SQL                     | Migration-based               | CLI       |
| Atlas     | Go              | Custom DSL              | State-based & Migration-based | CLI       |
| Prisma    | TypeScript      | Custom DSL              | State-based                   | CLI + UI  | ✅              | ✅            |
| Bytebase  | Go + TypeScript | SQL                     | Migration-based\*             | CLI + UI  | ✅              | ✅            | ✅            |

_\*Though Bytebase uses migration-based approach, it records schema snapshot before and after the migration. Thus it also retains some state-based benefits such as drift detection_

## The Summaries

Though the idea of Database as Code is not new, the trend and innovation have been much accelerated in the last 2 years:

- Commercial companies like GitLab, GitHub keep investing heavily in the Git-based DevOps infrastructure.
- Commercial companies like HashiCorp leverages the Git infrastructure and build innovative products like Terraform. The success of Terraform in turn popularizes the GitOps workflow and transforms the industry mindset from point-and-click systems into code-based systems.
- The proliferation of open source ecosystem. Coincidentally, all products in this space are either open source projects or their derivations.

![_](/content/blog/database-as-code-landscape/star-history.webp)

- New building primitives were invented to challenge the status quo:
  - The invention of the V8 JavaScript Engine and Node.js opens the door for frontend engineers to interact with the databases directly. The flock of frontend engineers demands a high bar for developer experience.
  - Thanks to its pragmatic design and riding the wave of cloud computing, Go becomes a mainstream language and challenges the common wisdom of using Java to build backend applications. Especially in the cloud-native era, Go even dominates the market. For deploying a Java-based program, you need to install JVM first, which is not needed for Go. Just by adopting Go, tools like Bytebase and Atlas gain a competitive edge against Liquibase / Flyway. Not to mention the developer productivity boost offered by the language itself.
- Even though the database market is very mature, however, the market is too lucrative to ignore. Becoming the next Snowflake is every database practitioner's wide dream. New database systems were built and the bar has been raised higher and higher. Core database capability and performance are just table stakes, to differentiate, they have to compete for developer experience.

The iconic [Evolutionary Database Design](https://www.martinfowler.com/articles/evodb.html) was written in 2016. Though the phrase "Database as Code" was not coined there, that article summarized the accumulated wisdom of applying Database as Code practice in the decade before.

![_](/content/blog/database-as-code-landscape/you-do-not-need-army-of-dba.webp)

The article also predicted a trend that a sizable DBA team will become less common. Nowadays, many serious engineering teams still need DBA's expertise, meanwhile, interacting with databases has also become developers' daily routine.

The power dynamics for managing database development have shifted over time, and the quest for better developer experience and efficient developer workflow is more demanding than ever.

It requires a delicate dance among the database systems, the database tools, and the industry mindset to conquer it. Database as Code is the critical puzzle and the timing couldn't be better to build in this space.

![_](/content/blog/database-as-code-landscape/bytebase-land.webp)



A few days ago, we posted a question on HN: [Ask HN: Is DBA still a good job?](https://news.ycombinator.com/item?id=31309729) The question quickly drew the attention of the HN community and got 236 thought-provoking comments. We share and feel the challenges, the frustration, the fatigue and the expectation.

We want to preserve all these heated discussions and help more people develop a multi-dimensional perspective on DBAs. That's how an idea for a blog series emerges -- **D·D·D** (DBA, Database, and Developer).

As the debut of this series, we compiled 10 stories about DBAs based on the replies and comments from the [HN thread](https://news.ycombinator.com/item?id=31309729): we adopted the Decameron style, made up the story background, and borrowed the true stories shared by the commenters and turned them over to the storytellers (all credits go to the rightful owners). Some of these storytellers had experience as full-time DBAs; the others may be developers who have worked with DBAs before. You can regard this article as the appetizer of a full course meal. The crew welcomes all kinds of feedback and, last but not least: Bon Appetite!

10 people, consisting of developers and DBAs from different companies, woke up in a locked-up room. Apart from basic furniture, the room seems as normal as it could be except for a wooden board hung above the door with 3 letters carved on it -- DDD.

Nobody knows what happened and how they ended up there. Just when they were about to smash the windows, an electronic screen flashed for a few seconds and displayed their way out:

_"One true story about DBAs per day, you'll only get locked up for 10 days."_

Somebody sneered at the somehow sentient screen:

"Stupid rule...What could possibly happen if we don't?"

Another sentence appeared on the screen:

_"You could leave, but your team's databases will be dropped and won't be recovered."_

Dead silence permeats the room. After a while, somebody finally breaks the ice:

"Well, I'll go first."

## Day 1 From [donatj](https://news.ycombinator.com/user?id=donatj): Honestly Wish We had a DBA

We had a DBA about 6 years ago, he was one of the smartest guys I've ever worked with, and knew his stuff inside and out. I would recommend a DBA highly to anyone having growth issues. I honestly don't think we would have scaled without him. I learned a ton from him.

He got let go for basically having a really bad temper, and we decided not to look for a new DBA. I was sad he was gone but also glad to not be getting screamed at. It feels silly, but I've got really unresolved and mixed feelings about the whole thing. I genuinely really liked the guy, loved talking music and movies with him, but he would heel turn in an instant and just ruin your day.

Honestly wish we had a DBA, I ended up picking up a lot of his responsibilities. I'd love to have someone I could learn from. Basically, I'm getting real sick of optimizing reports.

The screen really is sentient, even intelligent at some level. It automatically summarized the story:

_"Honestly wish we had a DBA"_

## Day 2 From [steve_adams_86](https://news.ycombinator.com/user?id=steve_adams_86): Both Brilliant, both jerks

This is strange and funny to read; the only two DBAs I've worked with matched this description perfectly.

The first guy was so irate and literally yelling about his entitlement to punish us for making bad architectural decisions that I thought he was joking. I just laughed, but he got angrier. Then it clicked after maybe five or ten seconds – he was in the process of verbally punishing us for querying the database too much.

The second guy was similar, but didn't yell. Still super protective of his domain, almost antagonistic about us integrating with the database at all.

Both brilliant, taught me more than I can recount here, hilarious and interesting… But total jerks too, and both let go more or less for that reason.

One they had a great system in place and the product felt mature, the companies axed them.

The screen flashes:

_"All brilliant, all jerks."_

DBAs in the room frowned a little.

## Day 3 From [jamal-kumar](https://news.ycombinator.com/user?id=jamal-kumar): Gone DBA

Is this the DBA gone off the rails story thread?

I tried to hire a guy for a postgres DBA at a company I had worked at. I knew him from before as a MS SQL DBA from a previous company and hoped he could fulfill the role, he said he could. The guy saved our asses in the previous case.

We gave him like a month paid time to get acquainted with postgres. Dude DISAPPEARS, gets his vpn keys revoked and everything because we're like what the hell. A month later he sends this big long email saying how he got in some altercation with the police, and landed in jail... it was just way too rambling to take seriously. We ask a mutual friend what the hell was up and he said that since he was working too much his wife CONFISCATED HIS CELL PHONE.

I think we called him 'ballgag \*\*\*' (fill in the blank with your least favorite person's name) after that

These guys are worth their weight in gold though, I think I heard he ended up getting a job at Microsoft down the line.

The screen switches to today's theme:

_"Gone DBA"_

## Day 4 From [Simon_O_Rourke](https://news.ycombinator.com/user?id=Simon_O_Rourke): He was Supposed to Know Everything

I had eerily similar experiences from an older DBA guy, who was going through college in his 40's while working away. He knew his stuff, helped a lot optimizing the ETL process, but could turn nasty in an instant, and had a major chip on his shoulder about younger analysts with college degrees.

Only thing, was this guy had zero programming experience outside SQL and Bash. Anytime he had a question in a meeting about some calling Java process, he used to wait until the meeting was finished and then grab five minutes with me privately to ask about whatever. So there was this perceived weakness that his professional persona "knew everything", which meant he couldn't ask fundamental questions or try to (publicly) learn things outside his domain.

In the end, we moved to Amazon Redshift, and he was progressively moved into a smaller and smaller role, where now he just curates 2-year old JIRA tickets. Probably knows he can't get a similarly well-paying gig elsewhere now.

The familiar flashes appeared once more:

_"He was supposed to 'know everything'"_

## Day 5 From [skc](https://news.ycombinator.com/user?id=skc): “UR RUINING MY F\*\*\*\*ING DATABASE”

This is eerily similar to my experiences with two very, very good DBAs I worked with.

I'm now wondering if there is something about the role that lends itself to such behavior. One day they are super pleasant, helpful and friendly. The next they are chewing people out for seemingly minor and random transgressions and saying things like "ruining my fucking database".

Today these guys remind me a lot of JK Simmons character in "Whiplash"

The screen displays an image this time:
![roaring-dba](/content/blog/_shared/roaring-dba.webp)

**The DBAs among these 10 people have been humble audiences for 5 days. This picture becomes the last straw to break their nerves. They decided to take the lead now.**

## Day 6 From [forinti](https://news.ycombinator.com/user?id=forinti): Try to be the person I needed help from before

I was a developer and turned into a DBA by necessity. When I was still just a dev, I couldn't get the support I needed from the DBAs, so now I try to be the person I needed help from before.

But, I now understand the problem. At my organisation we have only 2 DBAs, and we can't give all of our time to this task.

To be a DBA is an endless battle against the entropy some devs try to create. The data model should really always be validated by someone with more knowledge and experience. Also, some young devs come up with crazy ideas (for instance, we should not use foreign keys). Others want to load insane amounts of data to process in the database, instead of preprocessing, or using a staging database. There is a continuous stream of bad ideas being implemented.

And when a bad idea gets implemented, it is really hard to undo it and it tends to just create more chaos: you need materialised views to get around bad modelling, or weird views to compensate for duplicate data, etc.

So, yes, a DBA is a really important role. With regards to it being a good job, it depends if your company takes it seriously and puts you in the development process, otherwise you'll be in for a lot of stress.

Screen displays:

_"Try to be the person I needed help from before"_

## Day 7 From [compton93](https://news.ycombinator.com/user?id=compton93): For that company my role was priceless

I hate to imagine how many startups out there don't even create indexes and just consider it to be a database problem and increase CPU/RAM and look the other way.

I worked for several years at a semi-startup as a Database Engineer where my role was to guide developers in writing performant SQL (and writing indexes for them) and architect their schema to be inline with our future plans and various other database tasks (managing query plans, being an expert on the database feature set etc). I even created an internal course: SQL School, that I included the directors/customer support teams in.

To that specific company my role was priceless, new features could be built in 1/2 the time with an expert writing the queries and handling the database. My programmatic analysis of our 20 year old code base had 5000 unique queries and 3000 more when accounting for dynamic SQL. Not gonna lie it was a complete mess but if I didn't exist they would have needed a lot more database resources. It was a terrible code base and any plan to refactor and avoid this mess would still need my role to do the transition.

The screen flashes and shows:

_"For that company my role was priceless"_

## Day 8 From [Twisell](https://news.ycombinator.com/user?id=Twisell): Exhausting yet rewarding

I would argue that it's way more accurate to say this role is actually outsourced in most organizations with all the tradeoffs that come with that strategy. I begged my former employer to subscribe to a good PostgreSQL support company. Guess what the one we picked is constantly looking to hire new DBA to sustain growth...

I was in database engineering/administration position for the last 10 year on a small company. It was exhausting because management wouldn't care to rely one guy to manage the whole data stack. It was rewarding because the CEO saw me as probably smarter than I was because I knew SQL well enough to produce outstanding value as a single individual.

I spent a few years complaining that my colleagues didn't know their database well enough, and it was causing unnecessary bugs and poor performance.

I wanted the company to pay for some basic courses on the databases we were using, especially for the more junior Devs.

Instead we switched database like 11 times and now I don't know the database either.

The bugs and poor performance lingers though.

The screen changed the sentence to:

_"Exhausting yet rewarding"_

## Day 9 From [jhugo](https://news.ycombinator.com/user?id=jhugo): Conflict comes from lack of understanding

I think this kind of conflict often comes from frustration at developers not having much understanding of the underlying technology that the network engineer / DBA is responsible for.

Networks & DBs present abstractions, but they're very leaky ones, yet developers often expect to just treat them as magical black boxes about which they don't need to know much of the inner workings, constraints, tradeoffs etc. That makes the interface between the network engineer or DBA and software developers a natural site of conflict.

(You even see this above in this very thread, someone complaining that a DBA was punishing developers for "querying the database too much". Most likely the DBA was actually upset about how they were querying the database, not that they were querying it too much, but the developers want it to "just work" so the distinction isn't important to them.)

Screen flashes:

_"Conflict comes from lack of understanding"_

## Day 10 From [niklasrde](https://news.ycombinator.com/user?id=niklasrde) &[ datalopers](https://news.ycombinator.com/user?id=datalopers): The lost art -- data modeling and SQL

Same at my previous role - we did have some people who were really good at some point, and into DBs. We had a very solid Redis and PostgreSQL set-up, and some advanced stuff for simple data in Redis - but once that knowledge fades with attrition, making space for continued personal development into how two very different DBMS work for a small team is not easy, and it just stays untouched, disliked, and somewhat abandoned forever more.

I honestly think the thing that costs most companies the most in their hosting is developers not using databases correctly

100%. Data modeling and SQL has become a lost art.

The screen flashes and literally drops a few pixel tears:

_"The lost art -- data modeling and SQL"_

## Finale

While the last story approaches its end, the doors and windows open up widely. 10 people storm out, rush back to their office, and thoroughly check their database. Everything is fine, nothing deleted. They sigh with relief and start to delve into their daily work, as concentrated as ever.

It all starts with one sentence:

**"Is DBA still a good job?"**

We look forward to your story.


The
Bytebase Blog

Explore the standard for database development

Solutions & Platform

Comparisons

Legal

Resources

Company