Notes on the MySQL 9.7 LTS release: Hypergraph Optimizer in Community, JSON Duality View DML, Group Replication observability, and Dynamic Data Masking for Enterprise.

Notes on the MySQL 9.7 LTS release

PostgreSQL 19 features I'm excited about: planner advisor framework, DDL extraction, online REPACK, pg_stat_recovery, parallel autovacuum, sequence sync, and 64-bit MultiXact.

PostgreSQL 19 features I'm excited about

What SOC 2 compliance audits actually want from your database audit log — not 'logging enabled' but evidence your auditor will accept. Lessons from going through our own audit.

SOC 2 Audit Log Requirements: Lessons From Our Own Audit

SOX Section 404 requires IT general controls over database changes. Learn the ITGC domains, six audit questions teams fail, and how to build SOX-ready controls

Database Change Management for SOX Compliance

The 10 best data management tools in 2026 — database GUI clients, schema migration tools, data pipelines, and analytics platforms compared with pricing.

Top 10 Data Management Tools in 2026

Postgres-only SQL editors are designed specifically for Postgres databases, providing a more focused and efficient experience for Postgres users. In this post, we are taking a look at the top Postgres-only SQL editors in 2026.

Enterprise Text-to-SQL: Context, Evaluation, and Governance

Database deployment tools for devops teams in 2026.

Database Deployment Tools for DevOps Teams in 2026

Postgres 19 widens MultiXactOffset to 64 bits, eliminating the hidden 4 billion member limit that caused catastrophic outages for high-concurrency workloads.

Postgres 19 Feature Preview: 64-bit MultiXactOffset

A guide to MySQL schema migration best practice.

MySQL Schema Migration Best Practice

Postgres 19 introduces automatic sequence synchronization for logical replication, eliminating manual tracking and making major version upgrades and failover significantly easier.

Postgres 19 Feature Preview: Sequence Synchronization for Logical Replication

A guide to open source sensitive data discovery tools — from NLP libraries like spaCy and Microsoft Presidio to multi-source scanners like Hawk-Eye and full metadata platforms like OpenMetadata.

Top Open Source Sensitive Data Discovery Tools in 2026

Top open-source Postgres backup solutions for 2026.

Open Source Postgres Connection Pooler

PostgreSQL 18 features that change how application developers write and ship code.

What's New in PostgreSQL 18: A Developer's Perspective

Top open-source Postgres authentication solutions for 2025.

Features I Wish Postgres Had but MySQL Already Has

Areas where PostgreSQL shines and MySQL falls short.

Features I Wish MySQL Had but Postgres Already Has

Evaluate several database schema diagram tools, and help you to choose the right one for 2025

Top 4 Free, Open Source Database Schema Diagram Tools to Visualize Database Easier in 2025

An overview of the most popular text-to-SQL AI tools in 2026, including DataGrip, TablePlus, SQL Chat, Vanna.ai, SQLAI.ai, Postgres.new, and DBHub MCP server. Compare features and capabilities to find the best SQL AI assistant for your workflow.

Top 5 Text-to-SQL Query Tools in 2026

Evaluate several TypeScript ORMs, and help you to choose the right one for 2025

Top TypeScript ORM 2025

PostgreSQL 18 features and what they change for running production databases.

What's New in PostgreSQL 18: A DBA's Perspective

Explore alternatives to Amazon RDS, including 

RDS Alternatives in 2025

Infrastructure as Code security tools help organizations identify vulnerabilities and misconfigurations in their infrastructure definitions. This post explores the top open-source IaC security tools available in 2025.

Top Open Source Infrastructure as Code (IaC) Security Tools in 2025

Compare the price of the mainstream managed PostgreSQL hosting providers

PostgreSQL Hosting Options in 2025: Pricing Comparison

Database governance tools help organizations manage database changes, enforce policies, and maintain data quality. This post explores the top open-source database governance tools available in 2025.

Top Open Source Database Governance Tools in 2025

Workflow orchestration tools help coordinate tasks, handle errors, and monitor execution processes. In this post, we are taking a look at open source workflow orchestration options

Top Open Source Workflow Orchestration Tools in 2025

Compare the price of the mainstream managed MySQL hosting providers

MySQL Hosting Options in 2025: Pricing Comparison

A comprehensive guide to the best open-source Kubernetes cluster management tools in 2025, including Argo CD, Karmada, Sveltos, and Plural.

Top 4 Open Source Kubernetes Cluster Management Tools in 2025

Review the state of the Golang ORM and query builder

Choose the Right Golang ORM or Query Builder in 2025

Overview of MySQL 9 features with spicy comments

What's New in MySQL 9 - a DBA's Perspective

SQL clients makes it safer and easier to manage databases. In this post, we are taking a look at open source SQL clients options for you to try in 2024.

Top Open-Source SQL Parsers in 2025

A comprehensive reference guide to standard port numbers used by popular database systems in 2025

Common Database Port Numbers in 2025

Explore the leading Terraform UI tools of 2026 that enhance IaC workflows with visualization, collaboration, and governance features to streamline infrastructure management.

Top 4 Terraform UI Tools 2026

A comprehensive guide to the best free and open-source PostgreSQL EXPLAIN tools for query performance analysis, including built-in extensions, online visualizers, and integrated GUI solutions.

Top Free, Open Source Postgres Explain Tool to Analyze Database 2025

Top open-source Kubernetes dashboard solutions include K9s, Lens, and KubeDashboard.

Top Free, Open Source Kubernetes Dashboard in 2026

In this post, we will review the top 5 open source Terraform tools for 2025.

Top 5 Open Source Terraform Tools for 2025

Postgres schema compare tool is used to diff and synchronize schemas between Postgres databases. This article reviews the top 4 common tools in this category.

Top 6 Open Source Infrastructure as Code (IaC) Tools 2025

How to Become a Platform Engineer in 6 months?

Top open-source database management solutions include Beekeeper Studio, Adminer, HeidiSQL, and Bytebase.

Top Open Source Database Management Software 2025

SQL editor with built-in AI streamlines database management, making SQL development faster, more accurate, and accessible to users of all skill levels.

Top 3 Free SQL Editors with Built-in AI

An extensive list of PostgreSQL SQL Review and Style Guide.

PostgreSQL SQL Review and Style Guide

Compare the best SQL Server monitoring tools for 2026 — free and paid. Includes SSMS built-in SQL monitors, open-source MSSQL monitoring tools (DBA Dash, SQLWATCH, Opserver), and commercial SQL performance monitoring solutions (Redgate, Datadog).

Top SQL Server Monitoring Tools: Best SQL Performance Monitors in 2026

In this post, we are taking a look at the open source, online SQL editors that you can self-host.

Top 5 Open Source, Online SQL Editors You can Self-host

Describe the top database devops tools in 2025.

Top Database DevOps Tools in 2025

Introduce the advanced tools for platform engineering. Covering CD, security, workflow orchestration, database development workflow, and internal development portal.

How to Pick Platform Engineering Tools in 2025 - Advanced Guide

Introduce the essential tools for platform engineering. Covering code hosting, CI/CD, observability, resource provisioning, and secret management.

How to Pick Platform Engineering Tools in 2025 - Essential Guide

Bastion gates connections. ITSM gates approval. PAM gates credentials. Three layers, three tools, one gap at execution — and what an integrated platform replaces.

Securing Database Change: From Bastion to Integrated Platforms

How OpenAI and DeepSeek designs SSN database schema.

OpenAI vs. DeepSeek: SSN Database Schema Design

DevSecOps tools empower teams to seamlessly integrate security into every phase of the software development lifecycle, fostering the adoption of DevSecOps practices.

Top DevSecOps Tools for 2025

What happened in the database tooling landscape in 2024.

Database Tools in 2024: A Year in Review

In a mystical land where data flows like rivers, four heroes — a warrior, a cleric, a druid, and a paladin — unite to tame the Hydra of Database DevSecOps, a powerful three-headed dragon embodying Velocity, Security, and Reliability.

The Guardians of Database DevSecOps

Some of the craziest database security vulnerabilities uncovered in DBAs' career

Database security vulnerabilities are the craziest

Due to Docker's recent requirement for a paid subscription for professional use, people began searching for alternatives. Right alternatives for you will depend on your specific needs and requirements.

Top 4 Free Docker Desktop Alternatives 2026

What are the top Postgres extensions you may need to enhance your Postgres instance.

Why Managed Database Service is not Enough

Six levels of database automation, named after the autonomous-driving framework. From L0 manual SQL to L5 self-healing — and where most teams plateau.

The 6 Levels of Database Automation

When team wants to enforce a process for database change management. A quick approach is to reuse their existing ITSM system such as Jira. Let's review how a typical Jira workflow looks like and its inherent limitations.

Hope Is Not a Strategy: Why Jira Falls Short for Database Change

A list of features I wish PostgreSQL had. It is mainly around improving development workflow and inspired by Git. While acquiring Git-like behavior is challenging in database because of the stateful nature.

Features I wish PostgreSQL had as a developer

In this post, we are reviewing several open-source GitOps tools.

Top Open Source GitOps Tools in 2025

When schema migration outage happens, who should be more responsible in order to prevent future incidents?

Schema Migration Outage: Author or Reviewer's Fault?

The "Modern Snowflake Stack" is inspired from the phrase "Modern Data Stack", refers to a set of technologies and tools built around Snowflake, a cloud-based data warehousing solution, to enable efficient data storage, processing, and analytics.

Modern Snowflake Stack in 2024

What are the top open source MySQL schema migration tools.

Top Open Source MySQL Migration Tools in 2024

This article explores leading database security tools designed to shield your data from varied risks.

Top Database Security Tools

A comparison of the top 5 MySQL GUI clients in 2026: MySQL Workbench, DBeaver, TablePlus, Beekeeper Studio, and Navicat.

Top 5 MySQL GUI Clients in 2026

What are the top open source Postgres schema migration tools.

How many DBAs should a company hire

DORA recently released its annual State of DevOps 2023 report. Bytebase is a Database CI/CD tool for DBAs and DevOps teams, and we are diving deep into the State of DevOps Report.

Notes on the DORA 2023 State of DevOps Report

DataGrip is an database IDE developed by JetBrains and is closely integrated with the JetBrains ecosystem. As a database admin tool, DataGrip is not a comprehensive alternative to the other dedicated database GUIs. Here, we are taking a look at top DataGrip alternatives.

Top DataGrip Alternatives 2023

If you work exclusively with MySQL, MySQL Workbench is an excellent choice for a database GUI, but if you work with multiple databases, you need to look elsewhere. This article highlights five database GUI alternatives for MySQL Workbench.

Top MySQL Workbench Alternatives 2023

Postgres is one of the most advanced open-source relational databases, while you can leverage Postgres' capabilities without a GUI, having one can take your experience working with Postgres to the next level.

Top pgAdmin Alternatives 2023

Navicat is a long-established database GUI. However, it has a serious limitation in multi-user scenarios where team collaboration and centralized control are needed. Hence, we are taking a look at top Navicat alternatives.

Top Navicat Alternatives 2023

DBeaver is one of the most popular database management tools out there. If for some reason, DBeaver doesn't fulfill your requirements or you are just in the mood to try something new. Here, we are taking a look at top DBeaver alternatives

Top DBeaver Alternatives 2023

If you work with data and databases, SQL AI tools can significantly improve your productivity and efficiency; if you want to start learning SQL, this is also an excellent way to get started.

Top 5 SQL AI Tools to Ease Writing SQL 2023

A Postgres GUI client gives you a visual interface to query, design, and administer PostgreSQL — instead of doing everything from psql. Here are the 7 most useful in 2026, with the trade-offs that decide which one fits your workflow.

Database as a Service (DBaaS) Provider 2023

In the post, we explore the Google Cloud SQL pricing structure, tools for estimating its cost, and tips for optimizing the cost.

Understanding Google Cloud SQL Pricing

MySQL schema compare tool is used to diff and synchronize schemas between MySQL databases. This article reviews the top 4 common tools in this category.

Top 4 MySQL Schema Compare Tool to Diff and Sync Database 2026

Database schema migration have come a long way, from CLI to GUI, and all the way to GitOps/Database-as-Code. Let's review the state-of-the-art of database schema change / migration tools in 2026.

Top Database Schema Migration Tools to Avoid Change Outage 2026

What happened in database industry in 2022.

Database Review 2022 and 2023 Prediction

The best free SQL clients and database management software in 2026. Compare DBeaver, Beekeeper Studio, DbGate, HeidiSQL, phpMyAdmin, pgconsole, and Bytebase — open-source SQL GUI tools for Mac, Windows, and Linux.

Top 7 Free, Open Source SQL Clients and Database Management Tools (2026)

To manually spin up several different kinds (and versions) of database instances for testing can be gruesome. Here are some free tools to help you start a local database instance on Mac.

Top 3 Free Tools to Start a Local Database Instance on Mac

Real talk on whether strict SQL review and schema conventions actually help — with answers from working developers and DBAs.

Is SQL Review Necessary? Real Talk From Developers

We wonder if ChatGPT can help us in the field of database administration, and we dug deep into some toolings that have incorporated ChatGPT to make our life easier.

A (Non-Exhaustive) Summary of ChatGPT Enhanced Database Tools

A conversation with ChatGPT consulting which database to pick for your next software project.

Ask ChatGPT - How to Choose the Right Database for Your Next Project

From the SQLite-insipred DuckDB to the real Snowflake challenger and the Docker for data.

MotherDuck, from SQLite to the Docker for Data

The idea of codifying database schema changes (aka schema migration) is not new. While we are observing an accelerated trend of embracing Database as Code (DaC) across the industry. In this blog, we will review the state-of-the-art in the Database as Code landscape and share our own insight into the current and future trends in this space.

The Database as Code Landscape

This is the first of the DDD series, consisting of 10 stories of DBA & Database & developer.

DBA, Database and Developer - The Decameron

What happened in database industry in 2021. OLAP (Snowflake, Databricks, ClickHouse, Firebolt, ClickHouse), OLTP (PostgreSQL, PlanetScale, SQLite) and open source.

Database Review 2021 and 2022 Prediction


SQL Server monitoring tools help database administrators track performance, identify bottlenecks, and ensure optimal database health. Whether you need a free SQL monitor for a small environment or an enterprise-grade SQL performance monitoring solution, this guide covers the best options available in 2026.

The top SQL Server monitoring tools include Microsoft's SQL Server Management Studio (SSMS) with built-in monitors, the open-source MSSQL monitoring tools DBA Dash, SQLWATCH, and Stack Overflow's Opserver, and commercial solutions like Redgate SQL Monitor and Datadog Database Monitoring.

## Built-in Tools within SSMS

[SQL Server Management Studio (SSMS)](https://learn.microsoft.com/en-us/ssms/download-sql-server-management-studio-ssms) is primarily a tool for managing and administering SQL Server instances, databases, and their associated objects. While SSMS itself is not a dedicated monitoring tool, it does provide some features and integrations that allow DBAs to monitor SQL Server performance and activity:

### The Activity Monitor

**The Activity Monitor** provides real-time insights into SQL Server database performance and health, aiding in troubleshooting, resource monitoring, and ensuring smooth server operation.

![mssql-activity-monitor](/content/blog/top-sql-server-monitoring-tools/mssql-activity-monitor.webp)

### Reports

**Reports** offer disk usage data, performance dashboards, and top queries by CPU, duration, and IO, helping identify resource-intensive queries.

### SQL Server Profiler

**SQL Server Profiler**, though deprecated, is available in older SSMS versions for tracing SQL Server events. **Extended Events** is recommended as a modern alternative.

![mssql-profiler](/content/blog/top-sql-server-monitoring-tools/mssql-profiler.webp)

### Extended Events

**Extended Events** is the replacement for SQL Server Profiler. It's a lightweight, customizable monitoring system for capturing detailed performance and troubleshooting data.

![mssql-extended-events](/content/blog/top-sql-server-monitoring-tools/mssql-extended-events.webp)

## Open Source Tools

### DBA Dash

[DBA Dash](https://dbadash.com/) is a free, open source monitoring tool for SQL Server. It provides DBAs with insights into the health, performance, and activity of SQL Server instances.

It is particularly useful for small to medium-sized environments where a full-fledged enterprise monitoring solution might be overkill.

![dba-dash](/content/blog/top-sql-server-monitoring-tools/dba-dash.webp)

#### Key Features

- **Daily DBA Checks**: Backups, corruption, memory dumps, DBCC, availability groups, log shipping, mirroring, agent jobs, disk space, query store, identity columns, and more.
- **Performance Monitoring**: CPU, IO, blocking, running queries, slow queries, stored procedures, performance counters, memory, and more.
- **Track Configuration**: Configuration settings, trace flags, hardware, patching, and tempdb across all servers, with automatic change tracking.

### SQLWATCH

[SQLWATCH](https://github.com/marcingminski/sqlwatch) is decentralized, real to near-real time SQL Server monitoring solution. It is designed to provide comprehensive monitoring out of the box and to serve as a monitoring framework for your own projects or applications. It collects performance data in a local database with an option for centralised reporting for convenience.

#### Key Features

- 5-second granularity for capturing workload spikes.
- Grafana for real-time dashboards and Power BI for in-depth analysis.
- Extensive configuration options.
- Zero maintenance and unlimited scalability.

![sqlwatch](/content/blog/top-sql-server-monitoring-tools/sqlwatch.webp)

### Opserver

[Opserver](https://github.com/opserver/Opserver) by Stack Exchange monitors multiple systems, including SQL Server, Redis, and Elasticsearch, providing an "all servers" view for CPU, memory, network, and hardware stats.

![opserver](/content/blog/top-sql-server-monitoring-tools/opserver.webp)

## Commercial Tools

### Redgate SQL Monitor

[Redgate SQL Monitor](https://www.red-gate.com/products/dba/sql-monitor/) is a comprehensive, enterprise-grade SQL Server monitoring and performance tuning tool developed by Redgate Software. Optimize performance, ensure security, and mitigate potential risks with fast deep-dive analysis and customizable alerting.

![redgate-monitor](/content/blog/top-sql-server-monitoring-tools/redgate-monitor.webp)

### Datadog Database Monitoring

[Datadog Database Monitoring](https://www.datadoghq.com/product/database-monitoring/) provides deep visibility into databases across all of your hosts. Dig into historical query performance metrics, explain plans, and host-level metrics all in one place, to understand the health and performance of your databases and troubleshoot issues as they arise.

![datadog-database-monitoring](/content/blog/top-sql-server-monitoring-tools/datadog-database-monitoring.webp)

## SQL Server Monitoring Tools Comparison

| Tool | Type | Cost | Best For |
|------|------|------|----------|
| SSMS Activity Monitor | Built-in | Free | Quick real-time diagnostics |
| SSMS Extended Events | Built-in | Free | Detailed event tracing |
| DBA Dash | Open Source | Free | Small to medium environments |
| SQLWATCH | Open Source | Free | Near-real-time monitoring with Grafana |
| Opserver | Open Source | Free | Multi-system monitoring (SQL Server + Redis + more) |
| Redgate SQL Monitor | Commercial | Paid | Enterprise SQL performance monitoring |
| Datadog Database Monitoring | Commercial | Paid | Full-stack observability with database monitoring |

## Conclusion

Built-in tools within SSMS provide essential monitoring capabilities for immediate insights. Open source solutions like DBA Dash, SQLWATCH, and Opserver offer flexibility and cost-effectiveness, making them ideal for smaller setups or those seeking customizable options. For enterprises requiring comprehensive monitoring and performance tuning, commercial tools like Redgate SQL Monitor and Datadog Database Monitoring deliver robust features and advanced analytics.

## FAQ

### What is the best free SQL Server monitoring tool?

DBA Dash is the best free, open-source SQL Server monitoring tool. It provides daily DBA checks, performance monitoring (CPU, IO, blocking, slow queries), and configuration tracking across all your SQL Server instances.

### Does SQL Server have built-in monitoring?

Yes, SQL Server includes several built-in monitoring capabilities through SSMS: Activity Monitor for real-time performance insights, Extended Events for detailed event tracing, and Performance Reports for disk usage and top queries. SQL Server Profiler is also available but deprecated in favor of Extended Events.

### What is the best SQL performance monitoring tool?

For enterprise environments, Redgate SQL Monitor and Datadog Database Monitoring are the leading commercial SQL performance monitoring tools. For free options, DBA Dash and SQLWATCH provide comprehensive performance monitoring for SQL Server with customizable dashboards.

### How to monitor SQL Server performance?

You can monitor SQL Server performance using built-in tools like Activity Monitor and Extended Events in SSMS, open-source tools like DBA Dash or SQLWATCH for ongoing monitoring, or commercial solutions like Redgate SQL Monitor for enterprise-grade performance analysis and alerting.



<HintBlock type="info">

This post is maintained by Bytebase, an open-source database DevSecOps tools. We update the post every year.

</HintBlock>

| Update History | Comment          |
| -------------- | ---------------- |
| 2025/02/25     | Initial version. |

## What are the criteria?

1. **Open source**.
1. **Web-based**. Can be accessed via browser.
1. **Can be self-hosted**. Tools that don't offer a self-hosted version are not included.
1. **General-purpose**. Not specialized in a specific database type.
1. **Actively maintained**. Judged subjectively based on the project activities.

We pick SQL Editor, DbGate, CloudBeaver, Metabase, and Redash as the top 5 open source, online SQL Editors. They are categorized into 2 groups:

1. SQL Editor, DbGate, CloudBeaver are relative new and more focused on the SQL query and editing experience.
1. Metabase and Redash are more established and focused on the data visualization and dashboarding.

## SQL Editor

![sql-editor](/content/blog/top-open-source-online-sql-editors/sql-editor.webp)

[SQL Editor](https://www.sql-editor.com) is a web-based SQL client built by Bytebase. It can run in 2 modes:

    1. Stand-alone.
    1. Embedded within other platform (white-label).

Unlike traditional SQL clients, SQL Editor enforces security and compliance by integrating access control, data masking, and audit logging into the SQL execution process.

**Who is SQL Editor for?**

SQL Editor is designed for developers, DBAs, and security teams who need a governed SQL execution environment. It’s ideal for teams in regulated industries or
those following database access control and audit best practices.

**Strengths**

- **Access Control**: Enforce granular access control for different users and roles.
- **Dynamic Data Masking**: Mask sensitive data in the query results.
- **Audit Logging**: Log all SQL executions for compliance and security.
- **Secure connection**: Support integration with AWS IAM and GCP IAM auth, also support SSH and SSL connections.
- **GitOps integration**: Provide [Terraform provider](https://registry.terraform.io/providers/bytebase/bytebase/latest/docs) and API to codify the access control and data masking policies.

**Open source license and monetization**

SQL Editor is dual licensed. The community version is under MIT license. The enterprise version is under its own Enterprise license. SQL Editor offers
`Community`, `Pro`, and `Enterprise` plan.

## DbGate

![dbgate](/content/blog/top-open-source-online-sql-editors/dbgate.webp)

[DbGate](https://dbgate.org/) is designed for developers and database administrators who need a lightweight yet powerful tool for querying, editing, and managing databases from a web interface or as a desktop application.

**Who is DbGate for?**

DbGate is intended for developers, database administrators, and data analysts who need an intuitive SQL client with multi-database support. It’s suitable for both small teams and individual users who prefer a self-hosted, open-source alternative to commercial SQL clients.

**Strengths**

- **Cross-platform**: Works as a web app or standalone desktop application.
- **Schema and data editor**: Easy visualization and modification of database structures and data.
- **SQL generator**: Generate SQL statements from the UI.

**Open source license and monetization**

DbGate is licensed under GPL-3.0. It offers `Community`, `Premium`, and `Team Premium` plans.

## CloudBeaver

![cloudbeaver](/content/blog/top-open-source-online-sql-editors/cloudbeaver.webp)

CloudBeaver is developed by the same team behind DBeaver, a popular desktop database tool.

**Who is CloudBeaver for?**

[CloudBeaver](https://github.com/dbeaver/cloudbeaver) is built for teams that need a collaborative, browser-based database management tool. It’s useful for database administrators, developers, and analysts who want a self-hosted, web-based SQL client.

**Strengths**

- **Based on DBeaver**: Benefits from the same robust database support and query capabilities.
- **Secure connection**: Support integration with AWS IAM and GCP IAM auth, also support SSH and SSL connections.
- **Plugins**: Extensible with plugins for additional functionality.

**Open source license and monetization**

CloudBeaver is licensed under Apache 2.0. It offers `Community` and `Enterprise` plan.

## Metabase

![metabase](/content/blog/top-open-source-online-sql-editors/metabase.webp)

[Metabase](https://www.metabase.com/) is an open-source business intelligence (BI) tool that allows users to query databases without writing SQL. It provides an easy-to-use visual query builder, dashboards, and automated reports, making it a popular choice for teams looking to analyze data with minimal technical expertise.

**Who is Metabase for?**

Metabase is designed for business analysts, product managers, and data teams that need a self-service analytics tool. It is ideal for SMBs, startups, and enterprises that want a lightweight, open-source BI solution without requiring deep SQL knowledge.

**Strengths**

- **No-code query builder** for non-technical users
- **SQL editor** available for advanced users
- **Dashboard and reporting features** for data visualization
- **Scheduled reports and alerts**
- **Embeddable**

**Open source license and monetization**

Metabase has three licenses. AGPL-3.0 is for the community edition, proprietary license is for the enterprise edition and a special license for the embeddable edition. Metabase offers four plans: `Open Source`, `Starter`, `Pro`, `Enterprise`.

## Redash

![redash](/content/blog/top-open-source-online-sql-editors/redash.webp)

[Redash](https://redash.io/) is an open-source SQL-based data visualization and dashboarding tool. Redash has been [acquired by Databricks](https://www.databricks.com/blog/2020/06/24/welcoming-redash-to-databricks.html).

Redash provides an SQL query editor, data exploration tools, and visualization capabilities, making it a favorite for analysts and engineers working with SQL-based data sources.

**Who is Redash for?**

Redash is intended for data analysts, engineers, and business intelligence teams who need a lightweight, self-hosted data visualization platform. It’s great for organizations with existing SQL expertise that want fast insights from databases.

**Strengths**

- **SQL query editor**: With saved query history.
- **Dashboard and visualization tools**: For creating data-driven reports.
- **Scheduled refreshes**.
- **Email alerts**.

**Open source license and monetization**

Redash is licensed under BSD 2-Clause License. It does not offer a paid plan.

## Repository activity

GitHub star wise, all projects have gained decent tractions.

![star-history](/content/blog/top-open-source-online-sql-editors/star-history.webp)

Other than the GitHub star vanity metric, let's look at the monthly repository activities between Jan 24, 2025 and Feb 24, 2025.

| Products    | Merged PRs | Opened PRs | Closed Issues | New Issues |
| ----------- | ---------- | ---------- | ------------- | ---------- |
| SQL Editor  | 217        | 0          | 4             | 10         |
| DbGate      | 4          | 3          | 18            | 25         |
| CloudBeaver | 63         | 7          | 6             | 6          |
| Metabase    | 1008       | 137        | 365           | 171        |
| Redash      | 9          | 15         | 2             | 9          |

The stats could be misleading:

- `Merged PRs` could be very high if the project maintains many active release branches and need to cherrypick. This is the case for Metabase.
- `Opened PRs` only count the PRs that are opened in the same repo. For a public repo, it's quite possible that the PR is opened in a forked repo.
- If the team does not use GitHub Issues to manage the repository, then the `Closed Issues` and `New Issues` are not a good indicator of the project activity.

Overall, SQL Editor, CloudBeaver, and Metabase projects are more active than DbGate and Redash.



Databases are the backbone of modern applications, and managing them effectively is key to smooth development and operations. Traditional methods often can't keep up with the fast-paced, scalable demands of DevOps workflows. Enter Database DevOps tools — they streamline database development, testing, and deployment automation, fostering collaboration between teams while ensuring reliability and compliance.

In this post, we'll dive into some of the top Database DevOps tools, covering CI/CD automation, SQL linting, backup and recovery, data masking, observability, access management, secret management, and all-in-one solutions.

## CLI-based Database CI/CD - Liquibase

As applications evolve, so do their database structures. Managing these changes while preserving data and maintaining compatibility is a challenge.

[Liquibase](https://www.liquibase.com/) is a powerful, CLI-driven tool for database change management. It lets developers version control database changes using SQL, XML, YAML, or JSON, track schema changes with checksums, and automate deployments in CI/CD pipelines. Its declarative approach supports collaboration and maintains a full history of changes across platforms like PostgreSQL, MySQL, Oracle, and SQL Server.

_Alternatives:_ [Flyway](https://flywaydb.org/), [DACPAC (SQL Server)](https://learn.microsoft.com/en-us/sql/relational-databases/data-tier-applications/data-tier-application-overview?view=sql-server-ver16), [skeema (MySQL)](https://www.skeema.io/), [schemachange (Snowflake)](https://www.schemachange.com/)

## SQL Lint - SQLFluff

Writing SQL can be tricky, especially with multiple dialects and complex queries. Tools that help review and fix SQL code can save time and prevent bugs.

[SQLFluff](https://www.sqlfluff.com/) is an open source, dialect-flexible and configurable SQL linter. Designed with ELT applications in mind, SQLFluff also works with Jinja templating and dbt. SQLFluff will auto-fix most linting errors, allowing you to focus your time on what matters.

_Alternatives:_ [squawk](https://github.com/sbdchd/squawk)

## Database Backup & Recovery - Missing

Existing database backup tools fall into two categories:

1. General Solutions: Platforms like [Veeam](https://www.veeam.com/), [Rubrik](https://www.rubrik.com/), and [Actifio](https://www.actifio.com/) (acquired by Google Cloud) offer comprehensive data protection. Veeam excels in VM and hybrid-cloud backup, Rubrik focuses on cloud-native security and automation, and Actifio (integrated with Google Cloud) specializes in efficient copy data management. Together, they provide robust, versatile options for diverse enterprise needs.

2. Vendor-Specific Solutions: Tools like [Percona XtraBackup (for MySQL)](https://www.percona.com/software/percona-xtrabackup) and [Barman (for PostgreSQL)](https://pgbarman.org/) are tailored to specific database technologies, offering specialized backup and recovery capabilities.

The market misses a database-domain specific backup tools that supports all popular databases.

## Data Masking - Delphix

Data masking is replacing real data with fictitious yet realistic data, ensuring that sensitive information remains protected while maintaining data utility for development and testing purposes.

[Delphix](https://www.delphix.com/) is a leading data management platform known for its robust data masking capabilities. It specializes in creating sanitized, non-sensitive clones of databases for testing and development, ensuring compliance with data privacy regulations like GDPR and HIPAA.

_Alternatives:_ [Tonic](https://www.tonic.ai/), [Neosync](https://www.neosync.dev/), [Immuta](https://www.immuta.com/) (for data warehouse)

## Database Observability - Datadog

Database observability is vital for monitoring performance, detecting issues early, and ensuring reliability and security. It provides actionable insights through metrics, logs, and traces, enabling root cause analysis, capacity planning, and cost optimization.

[Datadog](https://www.datadoghq.com/) offers a comprehensive observability platform for cloud-scale applications, providing monitoring for servers, databases, tools, and services through its SaaS-based data analytics platform. Its [Datadog Database Monitoring suite](https://www.datadoghq.com/dg/monitor/databases/) specifically addresses database observability, supporting popular databases like PostgreSQL, MySQL, Redis, and MongoDB, helping teams optimize performance and ensure operational efficiency.

_Alternatives:_ [Percona Monitoring and Management](https://www.percona.com/software/database-tools/percona-monitoring-and-management)

## Database Access Management - Missing

Database Access Management (DAM) controls and monitors who can access a database, what they can do, and how they access it. It's a cornerstone of database security and compliance, ensuring only authorized users and applications interact with sensitive data.

Like Database Backup & Recovery, the market misses a database-domain specific DAM tools that supports all popular databases. On the other hand, there are popular PAM solutions like [CyberArk](https://www.cyberark.com/), [StrongDM](https://www.strongdm.com/) that control all infra access including databases.

## Database Secret Management - HashiCorp Vault

Database Secret Management securely stores and manages sensitive credentials like usernames, passwords, and connection strings. It automates credential rotation, enforces access controls, and provides auditing for compliance.

[HashiCorp Vault](https://www.vaultproject.io/) is a leading solution for secret management, offering secure storage, key management, and audit logging. It integrates with cloud platforms, DevOps tools, and HashiCorp's own products like Terraform, reducing the risk of data breaches.

## All-in-one - Bytebase

In the aforementioned sections, we have covered tools that focus on a particular aspect of the infrastructure. Like Datadog for monitoring, CyberArk for access management.
Do we have a tool that handles every aspect of the database domain? Let's enter [Bytebase](/), an open-source Database DevSecOps tool that streamlines database tasks while focusing on security and compliance. Key features include:

- CI/CD for databases – Automates schema migrations and integrates with GitOps workflows.
- SQL linting – Enforces best practices and prevents SQL anti-patterns.
- Data masking – Provides dynamic masking for interactive queries, enhancing security during database exploration.
- Database access control – Implements granular permissions to restrict database access based on roles and compliance needs.
- Secret management – Supports integration with external secret managers like HashiCorp Vault to securely manage database credentials.

Bytebase is designed for teams looking for an integrated approach to database change management while maintaining strict governance over database access and credentials.

![bytebase](/content/blog/top-database-devops-tools/bytebase.webp)



This is the series of articles about platform engineering:

1. [Platform Engineering vs. DevOps](/blog/platform-engineering-vs-devops/)
1. [Platform Engineering Tools: Essential Guide](/blog/platform-engineering-tools-essential-guide/)
1. Platform Engineering Tools: Advanced Guide (this one)

---

As your platform engineering practice evolves, your team can begin to explore more specialized, domain-specific tools.
In this post, we will cover the advanced tools for platform engineering.

We pick ArgoCD, Snyk, Temporal, Bytebase, and Cortex as the platform engineering advanced toolkit. They are categorized based on their strategic use cases:

- **Unbundling GitHub**: ArgoCD for continuous deployment, Snyk for software supply chain security.
- **Tools designed for large teams**: Temporal for workflow orchestration, Bytebase for database development workflow, Cortex for internal development portal.

## ArgoCD - Continuous Deployment

If your infrastructure is on Kubernetes. [ArgoCD](https://argoproj.github.io/argo-cd/) will provide a Kubernetes-native, declarative GitOps
approach to application deployment and management. It continuously monitors and syncs applications
with the desired state defined in Git repositories, offering seamless integration with Kubernetes clusters. ArgoCD’s features, like easy rollbacks, multi-cluster management, and a visual UI for monitoring application health, make it well-suited for Kubernetes environments. For an enhanced ArgoCD experience, consider exploring [Akuity](https://www.akuity.io/), founded by the creator of ArgoCD.

_Alternatives:_ [Flux](https://fluxcd.io/)

## Snyk - Software Supply Chain Security

While [GitHub Advanced Security (GHAS)](https://docs.github.com/en/get-started/learning-about-github/about-github-advanced-security) provides a solid foundation for platform engineering teams, it may not offer the breadth needed for more mature teams managing diverse tools and environments (e.g., CI/CD pipelines, containers, IaC).

[Snyk](https://snyk.io/) is a dedicated security platform, offering broader coverage for dependency, container, and infrastructure security. It integrates seamlessly with a variety of platforms (GitHub in particular) and provides comprehensive security automation.

Snyk has its own dashboard and UI where you can manage security issues across various projects and repositories. It enhances collaboration between your platform engineering and security teams, providing a streamlined and efficient security experience.

_Alternatives:_ [aikido](https://www.aikido.dev/)

## Temporal - Workflow Orchestration

As your business scales, you'll find that new tasks keep emerging. Some are one-time operations, while others repeat. Some are simple steps, others involve multiple stages. Some are stateless, while others are stateful, requiring meticulous rollback management. What starts as a basic cron job can quickly evolve into a complex workflow platform. Uber faced this challenge, developed and open-sourced [Cadence](https://github.com/uber-go/cadence). However, their creators are not content and [Temporal](https://temporal.io/) is born.

Temporal isn’t a lightweight solution—it’s built to handle intricate edge cases and scale. But trust me and many other high-profile Temporals customers, workflow orchestration is a minefield you’d rather not tap into. The next time your team talks about building a workflow engine, just send them a link to Temporal

_Alternatives:_ None.

## Bytebase - Database Development Workflow

Your developers and DBAs deserve more than just Jira for managing database tickets. Two Google engineers
also noticed the mundane and error-prone process when they manage the Google Cloud database fleet and started [Bytebase](/) - a database DevSecOps platform, designed to streamline three key use cases:

1. Database CI/CD (e.g., schema migrations)
1. Ad-hoc changes (e.g., DML modifications)
1. Query

Say goodbye to copying and pasting SQL statements between Slack and Jira, or scattering database credentials all over the places.
Bytebase offers a centralized platform that consolidates everything, with access control, dynamic data masking, audit logs, and much more.

_Alternatives:_ None.

## Cortex - Internal Development Portal

As your platform scales, you'll inevitably add more services to your toolbox, and managing them effectively becomes crucial. It often begins with a wiki page, but static documentation can't deliver real-time context or insights. The smart engineers from Spotify figured it's a common enough problem and open-sourced [Backstage](https://backstage.io/) (sound familiar?).While Backstage has done much to educate the market and raise awareness about the value of a centralized internal development portal, [Cortex](https://www.cortex.io/) has emerged as a more refined and polished solution.

_Alternatives:_ [port](https://www.getport.io/)

## Summary

The tools mentioned above complement the [essential platform engineering toolkit](/blog/platform-engineering-tools-essential-guide/) well. Many of them integrate seamlessly with the core tools, offering features like Terraform providers, Vault integration, and GitOps support. However, it's important to note that these advanced tools are best suited for more mature platform engineering teams. Keep in mind that the primary goal of platform engineering is to empower product engineering teams to deliver value to customers quickly—avoid the trap of over-engineering the platform.



Platform engineering adoption has surged in recent years. To better understand the landscape, we interviewed over 100 platform teams about their challenges and the tools they rely on. Drawing from these insights and our own experience, we have curated a list of essential platform engineering tools. This is the series of articles about platform engineering:

1. [Platform Engineering vs. DevOps](/blog/platform-engineering-vs-devops/)
1. Platform Engineering Tools: Essential Guide (this one)
1. [Platform Engineering Tools: Advanced Guide](/blog/platform-engineering-tools-advanced-guide/)

<HintBlock type="info">

The post does not cover infrastructure components like Kubernetes, database systems. It also does not cover collaboration tools like Slack, Notion, Jira, Linear.

</HintBlock>

---

We pick GitHub, Datadog, Terraform and Vault as the platform engineering essential toolkit. We believe these tools
represent the best practices in the industry and serve as the fundamental building blocks to expand your platform engineering practice. Let’s dive in.

## GitHub - Code Hosting and CI/CD

**Pros:**

- User familiarity (who doesn't ~know~use [GitHub](https://github.com/)?).
- Integrated and powerful CI/CD capabilities with GitHub Actions.
- Integrated security like CodeQL, Dependabot and a wide range of 3rd party tools.
- AI advancement with Copilot.
- Good documentation and developer experience.

**Cons:**

- No free tier for private repositories and self-hosted option.

**Comments:**

GitLab was once the go-to choice for many platform teams, but we've observed a growing shift towards GitHub in recent years. We covered the comparison in a more [detailed post](/blog/github-vs-gitlab/).

Unless you're working with a limited budget and need the self-hosted option upfront, we recommend GitHub over GitLab.
That said, starting with GitLab and migrating to GitHub is still a viable path. Just keep in mind the potential challenges of becoming too reliant on GitLab CI and its API (an LLM can certainly assist the migration).

## Datadog - Observability

**Pros:**

Other than the Cons below.

**Cons:**

- Expensive.
- No self-hosted option.

**Comments:**

Many people have a love-hate relationship with [Datadog](https://www.datadoghq.com/). It's an all-in-one observability platform, but it can also be very expensive due to its per-host pricing model. But we think it's a good investment for the right team. For small-to-medium teams, infrastructure costs are typically manageable, with product-market fit being the key focus. For larger teams, there’s often more flexibility to explore alternatives or negotiate better pricing with Datadog.

## Terraform - Resource Provisioning

**Pros:**

- Best practice. IaC, state-based, declarative approach.
- Great ecosystem with many terraform providers.

**Cons:**

- Learning curve.
- HCL (Hashicorp Configuration Language) as the Terraform DSL is more limited than a general purpose programming language.

**Comments:**

[Terraform](https://www.terraform.io/) is usually the first tool that comes to mind when people think of platform engineering. While it has faced competition from newer tools like [Pulumi](https://www.pulumi.com/) in recent years, Terraform or its community fork [OpenTofu](https://opentofu.org/) remains a reliable choice, particularly due to its extensive provider ecosystem. You're more likely to find a Terraform provider for your infrastructure than for most other alternatives.

## Vault - Secret Management

**Pros:**

- Best practice for secret management. Dynamic secrets, access control, audit logs.
- Good ecosystem (Terraform and other vendor integrations).
- Open-source.

**Cons:**

- Learning Curve.
- Operational overhead.
- Single point of failure (SPOF).

**Comments:**

I first learned about [Vault](https://www.vaultproject.io/) at PGConf Silicon Valley 2016, and I was immediately hooked. When it comes to security, you definitely don’t want to reinvent the wheel by building your own secret management system—Vault is a game-changer. The key question is when to onboard it. Our recommendation is to start with Vault from day one as you establish your platform team. Begin with its KV engine, and as you scale, you can move to more advanced database engine alike. The goal is to establish best practices early on, setting a solid foundation for security. Get the team used to a centralized secret management system as opposed to having it spread across multiple tools.

## Summary

When picking the essential tools for platform engineering, we focus on the following criteria:

- **Quality over cost**. Represent the best practices in the industry. Opting for the highest-quality tools ultimately proves to be more cost-effective in the long run.
- **Comprehensiveness**. A single tool can cover multiple aspects of platform engineering.
- **Strong ecosystem**. A robust ecosystem ensures that tools can integrate seamlessly with others. It's also easier to find help and resources when you have a strong ecosystem.

As your platform engineering practice evolves, your team can begin to explore more specialized, domain-specific tools. Check out the [advanced guide](/blog/platform-engineering-tools-advanced-guide/), where we’ll dive deeper into the tools that support larger, more complex environments.



Database change is gated by a stack of security tools. Bastion for connections. ITSM for approval. PAM for credentials. Each layer adds a control. Each adds a tool to maintain.

The stack leaves one gap — at execution, where approval, credential, and SQL meet without a system tying them together. This post walks the layers, names the gap, and shows what an integrated platform replaces.

<HintBlock type="info">

For the automation-depth lens — how much of the change lifecycle runs without a human — see [The 6 Levels of Database Automation](/blog/database-automation-levels).

</HintBlock>

## Bastion / Jump Server

| Tool category         | Examples                                   |
| --------------------- | ------------------------------------------ |
| Bastion / Jump server | AWS Bastion Host, Azure Bastion, Teleport  |

A bastion is a gateway between users and database servers. Connections are restricted to a known network path. Session activity is logged. Execution happens inside a controlled environment.

**What it gates:** network access. Every connection is identified and recorded.

**Where it stops short:** the bastion sees a connection, not a change. SQL is typed by hand. No review. No approval. No structured audit of what ran.

## ITSM and CAB

| Tool category               | Examples                                     |
| --------------------------- | -------------------------------------------- |
| ITSM                        | ServiceNow, Atlassian Jira, Freshworks       |
| Change Advisory Board (CAB) | Usually piggybacked on the ITSM workflow     |

[ITSM](https://en.wikipedia.org/wiki/IT_service_management) formalizes the change request: who proposes, what runs, who approves. The [CAB](https://en.wikipedia.org/wiki/Change_advisory_board) adds human governance — a cross-functional group that evaluates risk before sign-off.

**What it gates:** decision authority. No change proceeds without a recorded approval. The trail satisfies most compliance frameworks.

**Where it stops short:** approval is decoupled from execution. The CAB approves a script in a ticket; nothing prevents a different script from being run. And the workflow is slow for routine changes that don't warrant a CAB.

## PAM (Privileged Access Management)

| Tool category | Examples                                |
| ------------- | --------------------------------------- |
| PAM           | CyberArk, BeyondTrust, HashiCorp Vault  |

PAM tools control how database credentials reach humans. Credentials are vaulted. Checkouts are time-bound. Every checkout is logged.

**What it gates:** credential exposure. No long-lived database password lives on a developer's disk. Access is requested, granted, revoked.

**Where it stops short:** PAM sees the credential, not the SQL. Once a developer holds it in a client, PAM has no view of what runs.

## The stack in practice

With all three layers in place, a change runs like this:

1. Developer files a change request in ITSM with the SQL attached.
1. CAB reviews. CAB approves.
1. Developer checks out credentials from PAM.
1. Developer connects via the bastion and runs the SQL.

Three tools. Four hand-offs. One gap: **the SQL that runs is not necessarily the SQL that was approved**. The CAB approved a script in a ticket. The bastion saw a connection. PAM saw a credential checkout. Nothing tied them together at execution.

Other gaps the stack leaves open:

- **No pre-execution validation.** Schema checks and dry runs happen by hand, if at all.
- **No rollback.** DDL and DML aren't versioned; recovery means restoring from backup.
- **No unified audit.** The trail is split across three tools. None sees the full picture.

## What an integrated platform replaces

An integrated database change platform consolidates the stack. One tool replaces three. Approval, credential, and execution share the same record.

![bytebase](/content/blog/_shared/db-scheme-lg.png)

How [Bytebase](/blog/database-as-code/) maps to each layer:

- **Bastion replaced.** Connections happen through Bytebase's web UI. No direct database connection from a developer's laptop. No separate jump server to maintain.
- **ITSM replaced.** Change requests, approvals, and audit live in Bytebase. Existing ITSM (ServiceNow, Jira) plugs in when the ticket is the system of record.
- **PAM replaced.** Bytebase holds database credentials. Developers never see them. [IAM authentication](https://docs.bytebase.com/get-started/instance/#use-iam-auth) and [secrets manager integration](https://docs.bytebase.com/get-started/instance/#use-external-secret-manager) cover external-vault cases.
- **Execution gap closed.** Approval, validation, execution, and audit tie to the same record. The SQL that runs is the SQL that was approved. [SQL review](https://docs.bytebase.com/sql-review/overview/) catches issues before deployment. [1-click rollback](https://docs.bytebase.com/change-database/rollback-data-changes) handles recovery.

## Comparison

| Approach              | Gates                         | Audit trail              | Tools to maintain |
| --------------------- | ----------------------------- | ------------------------ | ----------------- |
| Bastion / Jump Server | Network access                | Session logs             | 1                 |
| ITSM + CAB            | Decision authority            | Ticket history           | 1                 |
| PAM                   | Credential exposure           | Checkout logs            | 1                 |
| Stacked (all three)   | Each layer separately         | Fragmented across tools  | 3+                |
| Integrated platform   | All layers, tied at execution | Single, queryable trail  | 1                 |

The stack works. Bastion, ITSM, and PAM each gate a distinct risk. But it pushes integration onto the team — three tools to maintain, three audit trails to correlate, one gap at execution. An integrated platform owns all three layers and closes the gap.



![debate](/content/blog/openai-vs-deepseek-ssn-database-schema-design/debate.webp)

[Elon Musk's comment](https://x.com/elonmusk/status/1888484555092312466) about SSN database de-duplication has sparked significant debate within the developer community. Many software engineers have dismissed his remark. Let's see how OpenAI and DeepSeek designs this SSN database schema.

## What is SSN?

For those of you not living in the US, SSN is the **Social Security Number**. It is a 9-digit number that
is used to identify an individual in the US, primarily used for:

- **Income Identification**: Employers use SSNs to report employees' wages and salaries to the government, ensuring earnings are properly recorded.

- **Tax Collection**: The IRS requires SSNs for tax filings, including income tax returns, to track individual tax obligations and withholdings.

- **Social Welfare Distribution**: Government agencies use SSNs to determine eligibility and distribute benefits such as Social Security, Medicare, Medicaid, and unemployment assistance.

- **Credit Score**: Lenders, banks, and credit bureaus use SSNs to track an individual's credit history, loan applications, and debt repayment behavior, forming the basis of credit scores.

In US, the SSN is a highly sensitive piece of information. True love isn't defined by wedding vows—it’s when you trust your partner enough to share your SSN.

## First Iteration

Prompt:

```plain
You are a relational database schema designer. You are given the task to design
the schema for the United States' Social Security Number (SSN) database.
The database needs to track basic personal information and payment profile.

Minimal design, no whistles and bells. Generate Postgres SQL dialect.
```

### ChatGPT 4o

```sql
CREATE TABLE person (
     -- SSN is always 9 digits (without dashes)
    ssn CHAR(9) PRIMARY KEY,
    first_name VARCHAR(50) NOT NULL,
    last_name VARCHAR(50) NOT NULL,
    date_of_birth DATE NOT NULL,
    -- M: Male, F: Female, O: Other
    gender CHAR(1) CHECK (gender IN ('M', 'F', 'O')),
    address TEXT NOT NULL
);

CREATE TABLE payment_profile (
    ssn CHAR(9) PRIMARY KEY REFERENCES person(ssn) ON DELETE CASCADE,
     -- Basic validation for routing number
    bank_routing_number CHAR(9) NOT NULL CHECK (bank_routing_number ~ '^[0-9]{9}$'),
    bank_account_number VARCHAR(20) NOT NULL,
    payment_method VARCHAR(10) CHECK (payment_method IN ('ACH', 'Check')) NOT NULL,
    last_payment_date DATE
);

CREATE INDEX idx_person_last_name ON person(last_name);
CREATE INDEX idx_payment_method ON payment_profile(payment_method);
```

ChartGPT also explains the design choices:

- `person.ssn` is the primary key, ensuring uniqueness.
- `person.date_of_birth` ensures valid date entry.
- `person.gender` is a single-character field with constraints (`M, F, O`).
- `payment_profile` references `person.ssn` (`ON DELETE CASCADE` ensures the payment profile is removed when a person is deleted).
- `payment_profile.bank_routing_number` has a regex constraint to ensure it contains exactly 9 digits.
- `payment_profile.payment_method` allows only ACH or Check.
- Indexes on `last_name` and `payment_method` optimize common lookups.

A pleasant surprise is the model designs `gender` column as a non-binary option and use `O` to represent other gender.

### DeepSeek R1

```sql
-- Table to store personal information
CREATE TABLE person (
    ssn CHAR(9) PRIMARY KEY, -- Social Security Number (9 digits, no hyphens)
    first_name VARCHAR(50) NOT NULL, -- First name
    last_name VARCHAR(50) NOT NULL, -- Last name
    date_of_birth DATE NOT NULL, -- Date of birth
    gender CHAR(1) NOT NULL, -- Gender (M/F/O for Male, Female, Other)
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP -- Record creation timestamp
);

-- Table to store payment profiles
CREATE TABLE payment_profile (
    id SERIAL PRIMARY KEY, -- Unique payment profile ID
    ssn CHAR(9) NOT NULL REFERENCES person(ssn) ON DELETE CASCADE, -- Foreign key to person
    total_contributions DECIMAL(12, 2) DEFAULT 0.00, -- Total contributions made
    last_payment_date DATE, -- Date of the last payment
    last_payment_amount DECIMAL(10, 2), -- Amount of the last payment
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP -- Record creation timestamp
);
```

DeepSeek designs a similar schema as ChatGPT. DeepSeek doesn't explain the design choices and just comment the columns.

One shortcoming is DeepSeek doesn't use column checks to ensure the validity of the data.

## Second Iteration - Account for Name Change

Prompt:

```plain
A person can change the name (e.g. after marriage)
```

### ChatGPT 4o

```sql
CREATE TABLE person (
    ssn CHAR(9) PRIMARY KEY, -- Immutable identifier
    date_of_birth DATE NOT NULL,
    gender CHAR(1) CHECK (gender IN ('M', 'F', 'O')), -- M: Male, F: Female, O: Other
    address TEXT NOT NULL
);

CREATE TABLE person_name (
    id SERIAL PRIMARY KEY,
    ssn CHAR(9) NOT NULL REFERENCES person(ssn) ON DELETE CASCADE,
    first_name VARCHAR(50) NOT NULL,
    last_name VARCHAR(50) NOT NULL,
    is_current BOOLEAN NOT NULL DEFAULT TRUE, -- Tracks the current name
    changed_at TIMESTAMP DEFAULT NOW() -- Records when the name was changed
);

CREATE TABLE payment_profile (
    ssn CHAR(9) PRIMARY KEY REFERENCES person(ssn) ON DELETE CASCADE,
    bank_routing_number CHAR(9) NOT NULL CHECK (bank_routing_number ~ '^[0-9]{9}$'), -- 9-digit validation
    bank_account_number VARCHAR(20) NOT NULL,
    payment_method VARCHAR(10) CHECK (payment_method IN ('ACH', 'Check')) NOT NULL,
    last_payment_date DATE
);

-- Ensure only one current name per person
CREATE UNIQUE INDEX idx_person_name_unique ON person_name(ssn) WHERE is_current = TRUE;

CREATE INDEX idx_person_name_last_name ON person_name(last_name);
CREATE INDEX idx_payment_method ON payment_profile(payment_method);
```

Changes & justifications:

1. Separated names into person_name:

   - Allows multiple historical names per person.
   - Tracks current vs. past names using `is_current` (enforced with a unique index).
   - Uses `changed_at` to record when a name was changed.

2. Kept SSN immutable:

   - `person` table stores attributes that do not change (SSN, DOB, gender, address).
   - Name history is handled separately.

3. Indexing Enhancements:

   - `idx_person_name_unique` ensures only one `is_current = TRUE` record per SSN.
   - `idx_person_name_last_name` allows searching by last name efficiently.

### DeepSeek R1

```sql
-- Table to store personal information
CREATE TABLE person (
    ssn CHAR(9) PRIMARY KEY, -- Social Security Number (9 digits, no hyphens)
    first_name VARCHAR(50) NOT NULL, -- Current first name
    last_name VARCHAR(50) NOT NULL, -- Current last name
    date_of_birth DATE NOT NULL, -- Date of birth
    gender CHAR(1) NOT NULL, -- Gender (M/F/O for Male, Female, Other)
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP -- Record creation timestamp
);

-- Table to store historical name changes
CREATE TABLE name_history (
    id SERIAL PRIMARY KEY, -- Unique identifier for each name change record
    ssn CHAR(9) NOT NULL REFERENCES person(ssn) ON DELETE CASCADE, -- Foreign key to person
    first_name VARCHAR(50) NOT NULL, -- First name at the time of the change
    last_name VARCHAR(50) NOT NULL, -- Last name at the time of the change
    change_date DATE NOT NULL, -- Date of the name change
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP -- Record creation timestamp
);

-- Table to store payment profiles
CREATE TABLE payment_profile (
    id SERIAL PRIMARY KEY, -- Unique payment profile ID
    ssn CHAR(9) NOT NULL REFERENCES person(ssn) ON DELETE CASCADE, -- Foreign key to person
    total_contributions DECIMAL(12, 2) DEFAULT 0.00, -- Total contributions made
    last_payment_date DATE, -- Date of the last payment
    last_payment_amount DECIMAL(10, 2), -- Amount of the last payment
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP -- Record creation timestamp
);
```

This time DeepSeek also explains the design benefits:

- **Current Name**: The `person` table always reflects the most up-to-date name.

- **Historical Tracking**: The `name_history` table tracks all name changes over time.

- **Data Integrity**: Foreign keys and `ON DELETE CASCADE` ensure data consistency.

DeepSeek makes a different design choice here as the `person` table still keeps the `first_name` and `last_name` columns.
They are just repurposed to store the current name.

## Third Iteration - Add Surrogate Key

Prompt:

```plain
Using SSN as the primary key could expose security risk as SSN is sensitive.
```

### ChatGPT 4o

```sql
CREATE TABLE person (
    person_id SERIAL PRIMARY KEY, -- Surrogate key for security
    ssn CHAR(9) UNIQUE NOT NULL CHECK (ssn ~ '^[0-9]{9}$'), -- Enforced uniqueness and validation
    date_of_birth DATE NOT NULL,
    gender CHAR(1) CHECK (gender IN ('M', 'F', 'O')), -- M: Male, F: Female, O: Other
    address TEXT NOT NULL
);

CREATE TABLE person_name (
    id SERIAL PRIMARY KEY,
    person_id INT NOT NULL REFERENCES person(person_id) ON DELETE CASCADE,
    first_name VARCHAR(50) NOT NULL,
    last_name VARCHAR(50) NOT NULL,
    is_current BOOLEAN NOT NULL DEFAULT TRUE, -- Tracks the current name
    changed_at TIMESTAMP DEFAULT NOW() -- Records when the name was changed
);

CREATE TABLE payment_profile (
    person_id INT PRIMARY KEY REFERENCES person(person_id) ON DELETE CASCADE,
    bank_routing_number CHAR(9) NOT NULL CHECK (bank_routing_number ~ '^[0-9]{9}$'), -- 9-digit validation
    bank_account_number VARCHAR(20) NOT NULL,
    payment_method VARCHAR(10) CHECK (payment_method IN ('ACH', 'Check')) NOT NULL,
    last_payment_date DATE
);

-- Ensure only one current name per person
CREATE UNIQUE INDEX idx_person_name_unique ON person_name(person_id) WHERE is_current = TRUE;

CREATE INDEX idx_person_name_last_name ON person_name(last_name);
CREATE INDEX idx_payment_method ON payment_profile(payment_method);
```

Improvements for Security & Best Practices:

1.  Surrogate Primary Key (person_id):

    - Replaces `ssn` as the primary key, reducing exposure risk.
    - Makes foreign key relationships reference `person_id` instead of `ssn`.

1.  SSN remains unique & validated:

    - `UNIQUE NOT NULL` constraint ensures no duplicates.
    - Regular expression constraint (`CHECK (ssn ~ '^[0-9]{9}$')`) ensures valid format.

1.  Foreign Keys use person_id:

    - `person_name` and `payment_profile` reference `person_id` instead of `ssn`.
    - Avoids exposing SSN in multiple places.

### DeepSeek R1

```sql
-- Table to store personal information
CREATE TABLE person (
    id SERIAL PRIMARY KEY, -- Surrogate key (auto-incrementing integer)
    ssn CHAR(9) NOT NULL UNIQUE, -- Social Security Number (9 digits, no hyphens)
    first_name VARCHAR(50) NOT NULL, -- Current first name
    last_name VARCHAR(50) NOT NULL, -- Current last name
    date_of_birth DATE NOT NULL, -- Date of birth
    gender CHAR(1) NOT NULL, -- Gender (M/F/O for Male, Female, Other)
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP -- Record creation timestamp
);

-- Table to store historical name changes
CREATE TABLE name_history (
    id SERIAL PRIMARY KEY, -- Unique identifier for each name change record
    person_id INT NOT NULL REFERENCES person(id) ON DELETE CASCADE, -- Foreign key to person
    first_name VARCHAR(50) NOT NULL, -- First name at the time of the change
    last_name VARCHAR(50) NOT NULL, -- Last name at the time of the change
    change_date DATE NOT NULL, -- Date of the name change
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP -- Record creation timestamp
);

-- Table to store payment profiles
CREATE TABLE payment_profile (
    id SERIAL PRIMARY KEY, -- Unique payment profile ID
    person_id INT NOT NULL REFERENCES person(id) ON DELETE CASCADE, -- Foreign key to person
    total_contributions DECIMAL(12, 2) DEFAULT 0.00, -- Total contributions made
    last_payment_date DATE, -- Date of the last payment
    last_payment_amount DECIMAL(10, 2), -- Amount of the last payment
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP -- Record creation timestamp
);
```

Key Changes for Security:

1. Surrogate Key (`id`) in `person` table:

   - The `person` table now uses an auto-incrementing integer (`id`) as the primary key.
   - The `ssn` column is still stored but is no longer the primary key. It is marked as `UNIQUE` to ensure no two people share the same SSN.

1. Foreign Key References:

   - The `name_history` and `payment_profile` tables now reference the `person` table using the `person_id` (surrogate key) instead of the `ssn`.

1. SSN Storage:

   - The `ssn` column is stored securely in the `person` table. In a real-world application, you should consider encrypting this column or using other security measures (e.g., hashing or tokenization) to protect sensitive data.

## Summary

Both ChatGPT 4o and DeepSeek can perform the SSN schema design task well:

1. They both come up with a sane schema design upfront.
1. They are able to alter the design based on the followup prompts. In the above example, both
   are able to incorporate the name change and introduce surrogate key to address the security concern.

ChatGPT 4o's highlight:

- Design `gender` column as a non-binary option.
- Enforce the data validity using column checks.
- Include indexes on the columns that are frequently queried.
- Explain the design choices and justify the design decisions.

DeepSeek R1's highlight:

- I personally like its refined `person` table design as it can return the current name, which will make the query more convenient.
- The explanation is terse and to the point.

## Final Thoughts

As shown above, schema design involves various trade-offs. Determining whether a unique SSN is a good design choice requires considering the surrounding context. Even state-of-the-art models still require human guidance to generate high-quality schema designs that accommodate real-world use cases (e.g., name changes) and address security concerns (e.g., SSN exposure).



**DevSecOps** stands for Development, Security, and Operations. It extends the principles of DevOps by embedding security practices throughout the software development lifecycle (SDLC), from initial design to deployment and maintenance.

In traditional DevOps, security checks might occur at the end of the development process. DevSecOps shifts this to ["shift-left"](https://en.wikipedia.org/wiki/Shift-left_testing) security, meaning security is integrated early and continuously across the pipeline. In this post, we are taking a look at some popular DevSecOps tools.

## GitLab - CI/CD

GitLab began as an open-source alternative to GitHub, initially focusing on version control and collaboration for developers. Over the years, GitLab has evolved into a comprehensive DevSecOps platform, embedding security directly into the software development lifecycle.

In March 2024, GitLab further reinforced its security focus by [acquiring Oxeye](https://about.gitlab.com/press/releases/2024-03-20-gitlab-acquires-oxeye-to-advance-application-security-and-governance-capabilities), a company specializing in cloud-native application security and risk management.

## Snyk - Vulnerability

Snyk is a developer-first security platform that focuses on integrating security into the development workflow, enabling teams to identify and remediate vulnerabilities across the entire software development lifecycle (SDLC). As a key player in the DevSecOps space, Snyk bridges the gap between developers and security teams by embedding automated security checks directly into coding, build, and deployment pipelines.

By providing real-time feedback within IDEs, CI/CD pipelines, and repositories, Snyk's platform empowers developers to take ownership of security without disrupting their workflows.

Snyk also expands its capabilities via acquisitions. In 2024, it has acquired [Probely](https://snyk.io/news/snyk-acquires-developer-first-dast-provider-probely/) and [Helios](https://snyk.io/blog/welcoming-helios-to-snyk/).

Other options: [JFrog](https://jfrog.com/), [Sonar](https://www.sonarsource.com/).

## HashiCorp Terraform + Vault - Infrastructure

HashiCorp [Terraform](https://www.terraform.io/) and [Vault](https://www.vaultproject.io/) form a powerful combination in the DevSecOps landscape, embedding security into infrastructure provisioning and secrets management.

Terraform automates the provisioning, modification, and management of infrastructure across cloud providers, data centers, and services through declarative code. It enables consistent and repeatable infrastructure deployment while minimizing human error.

Vault manages secrets and sensitive data through a unified interface, providing dynamic secrets, data encryption, and identity-based access across distributed infrastructure and applications.

When combined, Terraform and Vault create a secure and automated infrastructure pipeline that adheres to DevSecOps principles.

- Terraform provisions cloud resources (e.g., AWS EC2, RDS) and configures services using IaC.
- During provisioning, Terraform fetches secrets from Vault dynamically. This ensures no static credentials are stored in the Terraform code or repositories.
- [Sentinel policies](https://www.hashicorp.com/sentinel) validate infrastructure compliance before deployment, ensuring all resources meet security requirements.
- Vault continues managing secrets post-deployment, dynamically rotating them and preventing unauthorized access.

Other options: [Pulumi](https://www.pulumi.com/), [Infisical](https://infisical.com/)

## Cortex - Service Catalog

![cortex](/content/blog/top-devsecops-tool/cortex.webp)

[Cortex](https://www.cortex.io/) is an Internal Developer Portal (IDP) designed to enhance visibility, governance, and security across development workflows, aligning development, security, and operations teams to ensure compliance and improve system resilience. Cortex [integrates](https://www.cortex.io/integrations) with aforementioned tools like Sonar, Snyk, embedding security checks within CI/CD pipelines.

Other options: [Backstage](https://backstage.io/)

## Bytebase - Database

![change-query-secure-govern-database-all-in-one](/content/blog/_shared/db-scheme-lg.png)

[Bytebase](/) is a [database DevSecOps](/blog/what-is-database-devsecops/) platform designed for developers, security, DBA, and platform engineering teams.

Bytebase enhances database security and compliance through features like SQL Review, fine-grained database permissions, and dynamic data masking.

## Summary

DevSecOps integrates security into every phase of the software development lifecycle. This post explores popular DevSecOps tools, including GitLab for CI/CD security, Snyk for vulnerability scanning, HashiCorp for infrastructure security, Cortex for service governance, and Bytebase for secure database development workflow. These tools reflect the growing emphasis on proactive, continuous security within modern development pipelines.



Database tools are the critical enablers that bridge the diverse needs of developers, database administrators (DBAs), security engineers, and platform engineers, serving as the foundation for managing, optimizing, securing, and governing data systems.
Let's take a look at the database tooling landscape in 2024.

## Starlets

### DataGrip

![datagrip](/content/blog/database-tool-review-2024/datagrip-ai.webp)

[DataGrip](https://www.jetbrains.com/datagrip/), a familiar name in the world of database tools, continues to evolve with the introduction of an [AI Assistant](https://www.jetbrains.com/datagrip/features/ai/) this year.
By integrating AI capabilities, DataGrip can provide schema-aware AI chat, text-to-SQL, SQL rewrite, and more.

### Metis

![metis](/content/blog/database-tool-review-2024/metis.webp)

[Metis](https://www.metisdata.io/) is a database observability platform designed to empower developers and DevOps teams to effectively manage and optimize their databases throughout the software development lifecycle.

### Neosync

![neosync](/content/blog/database-tool-review-2024/neosync.webp)

[Neosync](https://www.neosync.dev/) is an open-source platform designed to help developers manage sensitive data by providing tools for data anonymization, synthetic data generation, and environment synchronization.

### Outerbase

_Outerbase has been [acquired by Cloudflare](https://blog.cloudflare.com/cloudflare-acquires-outerbase-database-dx/) in 2025_

![outerbase](/content/blog/database-tool-review-2024/outerbase.webp)

[Outerbase](https://www.outerbase.com/) is an AI-powered database platform that enables users to view, query, visualize, and edit their data through an intuitive interface. Recently, they [open-sourced Outerbase Studio](https://github.com/outerbase/studio) browser-based GUI client for managing databases like SQLite, PostgreSQL, and MySQL.

### Teable

![teable](/content/blog/database-tool-review-2024/teable.webp)

[Teable](https://teable.io/) is an open-source, no-code database platform that combines the simplicity of a spreadsheet interface with the robustness of PostgreSQL. It enables users to create and manage complex database applications without writing code, facilitating real-time collaboration and scalability.

### Bytebase

![bytebase](/content/blog/_shared/db-scheme-lg.png)

Being part of the database tooling ecosystem, 2024 is also a pivotal year for us at Bytebase. Beside passing 10K GitHub starts, we launched [Bytebase 3.0](/blog/bytebase-3-0), marking our evolution to a **database DevSecOps** solution, with a strong emphasis on security.

## Acquisitions

While database tooling acquisitions may not grab headlines like high-profile data infrastructure deals—such as [OpenAI’s acquisition of Rockset](https://openai.com/index/openai-acquires-rockset/) or [Databricks’ purchase of Tabular](https://www.databricks.com/blog/databricks-tabular)—they are equally indicative of the industry’s momentum:

- [Snowflake acquires TruEra](https://www.snowflake.com/en/blog/snowflake-acquires-truera-to-bring-llm-ml-observability-to-data-cloud/): TruEra is an AI observability platform for assessing the quality of LLM applications and addressing risks like hallucination, bias, and toxicity.
- [Databricks acquires Lilac](https://www.databricks.com/blog/lilac-joins-databricks-simplify-unstructured-data-evaluation-generative-ai): Lilac is a tool for data scientists and AI researchers to explore, understand, and modify text datasets in a tractable way.
- [Perforce acquires Delphix](https://www.perforce.com/blog/perforce-to-acquire-delphix): Delphix is a data platform specializing in data masking, virtualization, and compliance solutions.
- [Timescale acquires PopSQL](https://www.timescale.com/blog/best-postgresql-gui-popsql-joins-timescale/): PopSQL is a collaborative SQL editor known for its intuitive interface and team-friendly features (also our favorite besides our very own [SQL Editor](/sql-editor/)).

## Farewells

As we reflect on the evolving database tooling landscape, it’s worth acknowledging the bittersweet reality that not all tools achieve longevity, even when they embody innovation and promise.

![ottertune-snaplet](/content/blog/database-tool-review-2024/ottertune-snaplet.webp)

Tools like [OtterTune](https://ottertune.com/), a trailblazer in database tuning, and [Snaplet](https://snaplet.dev/), which streamlined database cloning and sanitization for development environments, stand out as examples of ingenuity.

Though these tools may have ceased operations, their contributions to the ecosystem remain impactful and enduring.

## Parting Words

Database tools are more than just companions to database systems—they are an integral part of the database ecosystem, sometimes outlasting the very technologies they support. As underlying database technologies evolve, these tools adapt and thrive by harnessing new capabilities.

See you in 2025.



In a mythical land where data flows like rivers and knowledge towers reach the skies, an ominous beast roams the mountains — a fearsome, three-headed dragon known as the Hydra of **Database DevSecOps**.
Each head represents a crucial domain in the realm of database engineering: `Velocity`, `Security`, and `Reliability`.

![dragon](/content/blog/database-devsecops-guardians/dragon.webp)

The dragon's sheer presence brings both power and peril to the land, as it embodies the balance needed
to maintain a functional and safe digital ecosystem. However, taming this beast is no easy feat.

At the base of the mountain stand four mighty heroes, each a master of their craft. They are sworn
protectors of the realm, determined to tame the Hydra and harness its powers for the good of the kingdom.
Together, they form the legendary Database DevSecOps Guild, each member bringing their own unique skills and virtues to the team.

![dev](/content/blog/database-devsecops-guardians/dev.webp)

Leading the charge is **Developer, the bold and dynamic warrior🗡️**. Clad in light armor and wielding a gleaming sword, Dev embodies the spirit of rapid progress and continuous improvement. His eyes are sharp and eager, always looking for ways to move forward faster. But he knows well that unchecked speed can lead to chaos — a lesson he learned from the many scars he bears.

![dba](/content/blog/database-devsecops-guardians/dba.webp)

Beside him stands the steadfast **DBA, a wise cleric🧙** cloaked in deep purple robes, marked with the ancient sigils of knowledge and control.The DBA keeps the others grounded, wielding a flame of database power and vigilance, casting it to ward off corruption and keep the system’s foundations strong. With every spell and incantation, the DBA ensures the delicate balance between innovation and structure remains intact.

![platform](/content/blog/database-devsecops-guardians/platform.webp)

To Developer’s other side is the powerful **Platform Engineer, a druid🐻** draped in furs and feathers, symbolizing the strength and resilience of the infrastructure. The Platform Engineer’s mastery of the landscape allows them to provide a stable foundation upon which all other advancements rely. With an unwavering gaze, the druid keeps the systems in sync with nature’s rhythm, ensuring the entire ecosystem functions smoothly and resiliently.

![security](/content/blog/database-devsecops-guardians/security.webp)

And finally, on the far side, stands the **Security Engineer, a noble paladin🛡️** donned in shining armor, bearing a massive shield with the symbol of a lock. The Security Engineer is the last line of defense, ensuring that the domain is safeguarded against malevolent forces. Their shield can withstand the strongest of blows, warding off threats that lurk beyond the boundaries of the kingdom.

Together, the guild faces the Hydra of Database DevSecOps, each member knowing that they must work in harmony to subdue the beast. Developer’s speed alone could provoke the dragon’s wrath, leading to instability. The DBA’s wisdom ensures that their actions are thoughtful and balanced, while the Platform Engineer’s resilience provides a solid foundation. Finally, the Security Engineer’s vigilance keeps danger at bay, allowing the team to operate within a safe framework.

The guild knows they cannot slay the Hydra, for it is an eternal entity born from the very needs of the kingdom. Instead, their quest is to tame it — to transform its chaotic energy into a disciplined force that propels the realm forward while keeping it safe and reliable. With weapons drawn, spells prepared, and shields at the ready, the heroes advance, knowing that the journey will be long and fraught with challenges. But united in purpose, they march on, for they are the guardians of Database DevSecOps, bound by a common oath to protect, stabilize, and advance their world.

![poster](/content/blog/database-devsecops-guardians/poster.webp)



Lately, a Post in `r/ExperiencedDevs` community of reddit, named [What are some of the craziest security vulnerabilities you've uncovered?](https://www.reddit.com/r/ExperiencedDevs/comments/1fzwzkd/what_are_some_of_the_craziest_security/), raised hundreds of
sad stories, among which are several tragedies about database. Let's laugh at some of them.

## Stories

The first story tells about a security breach that exposes core permissions like admin access, free account creation and even credit card information.
The worst thing is, still nobody cared even after the narrater revealed problem. But this narrater was sweet. He patched all errors.

![story-one](/content/blog/database-security-vulnerabilities-are-craziest/story-one.webp)

Then there's this API endpoint causing the PII leakage of almost half the entire population of Australia. Information got ocumented. Then completely publically promoted on Postman cloud community.

![story-two](/content/blog/database-security-vulnerabilities-are-craziest/story-two.webp)

In this Java webapp, using asterisk would match with all users including admin, thus creating a privileged session that syncs EVERY operation among ALL users.

![story-three](/content/blog/database-security-vulnerabilities-are-craziest/story-three.webp)

During the Ethereum ICO craze, an "SEC approved" ICO claiming to be open source, had one database seed file listing raw email addresses of admin accounts, as well as a file with authentication logic, accessible to public. The narrater manually transferred the token balance of a random account to 0x0 and it reflected on chain.

![story-four](/content/blog/database-security-vulnerabilities-are-craziest/story-four.webp)

## Summary

It's been a headache that data leakage, permission mis-assignment, accidental database operations are always appearing. We can tell that those stories above and all other accidents share same carelessness, but sometimes stupid things still unavoidably happens.

Maybe it suggests that, standardizing database operation process remains a necessity. With a DevSecOps tool, things would be much easier, safer and more organized.



[Docker Desktop](https://www.docker.com/products/docker-desktop/) is a one-click-install application for Mac, Linux, or Windows environment to build, share, and run containerized applications and microservices. It provides a straightforward GUI and takes care of port mappings, file system concerns, or other default settings. It's outstanding for comprehensive toolset and a wide ecosystem.

However, due to the requirement for a paid subscription for professional use in larger businesses and government entities, or some concern that Docker may not have covered so perfectly, people began searching for alternatives.

There are many alternative tools available, ranging from lightweight container engines like Podman to full-fledged container orchestration platforms. Here's the most recommended Docker Desktop Alternatives in 2026:

## OrbStack

[**OrbStack**](https://orbstack.dev/) is a lightweight alternative to Docker Desktop. It's based on Moby Engine and is composed of OrbStack CLI, a command-line tool, instead of graphical Docker CLI. With lightweight resource usage, fast startup and simple setup process, OrbStack is easier to use.

OrbStack is specifically designed to be faster and much lighter than Docker Desktop, with a focus on native macOS integration. It now starts containers up to 10x faster than Docker Desktop and has improved its Kubernetes support through K3s. OrbStack automatically shares files and ports between your Mac and containers, making it even more user-friendly.

![orbstack](/content/blog/top-docker-desktop-alternatives/orbstack.webp)

On the pricing front, OrbStack remains free for commercial use, which is a significant advantage over Docker Desktop's subscription model. However, its ecosystem of plugins and extensions is still relatively limited compared with Docker Desktop, and it remains Mac-only with no Windows or Linux support.

## Rancher Desktop

[**Rancher Desktop**](https://rancherdesktop.io/) is an open-source alternative with built-in container management focused on Kubernetes. Featuring native Kubernetes support, it provides integration with modern cloud-native tools and supports both Docker and containerd. It supports Windows, macOS, and Linux.

![rancher-desktop](/content/blog/top-docker-desktop-alternatives/rancher-desktop.webp)

Rancher Desktop offers improved Kubernetes integration with the ability to switch between different Kubernetes versions with one click. The interface has been refined to feel more similar to Docker Desktop while maintaining its free-for-work-use policy with no subscription needed.

Rancher Desktop gives users a choice between Docker or containerd engines, providing more flexibility. However, it still uses more memory and CPU than lighter alternatives and includes Kubernetes even if you don't need it. Some users report occasional crashes or freezes, though these issues have been reduced in recent updates.

## Podman

[**Podman**](https://podman.io/) is an open-source tool built on the OCI (Open Container Initiative) standards, not tied to any specific operating system or platform. It enhances security by running containers as non-root by default and using user namespaces to provide additional isolation.

![podman](/content/blog/top-docker-desktop-alternatives/podman.webp)

Podman has established itself as the secure, daemonless Docker replacement. It remains 100% Docker-compatible, using the same commands (podman run = docker run), but with enhanced security through rootless containers and no daemon requirement. Podman is now faster than Docker due to the elimination of daemon overhead and offers better security since no root access is needed.

Podman Desktop, the graphical interface for Podman, has become a major player across all major platforms with strong Docker compatibility. It runs without a daemon process for better security and works directly with Kubernetes. The interface has been improved, though it's still not as polished as Docker Desktop or OrbStack.

Podman remains free and open-source with no licensing restrictions for commercial use. However, some Docker Compose features still require extra setup, and there's less GUI support compared to Docker Desktop.

## Colima

[**Colima**](https://github.com/abiosoft/colima) can also be interpreted as **Co**ntainers on **Li**nux on **Ma**. This naming convention cleverly indicates its functionality and target platform in a succinct and memorable way.

![colima](/content/blog/top-docker-desktop-alternatives/colima.webp)

Colima has maintained its position as a lightweight command-line tool for Mac and Linux users. It's super efficient with computer resources and works great for users who prefer typing commands over clicking buttons. Colima now supports both Docker and containerd, and includes Kubernetes through K3s. It allows precise control over how much CPU and memory to use.

Colima uses very little memory and CPU compared to Docker Desktop and works with all the Docker commands users already know. It sets up quickly with simple commands and works with Docker Compose without changes. The pricing model remains free and open-source with no licensing restrictions.

The main drawbacks remain: no graphical interface (everything is command-line only), no Windows version (only works on Mac and Linux), and it requires more manual setup than GUI tools. Colima also has a smaller community than Docker Desktop or Rancher.

## Summary

Although Docker Desktop remains a popular choice for managing containers visually, it's good to know about alternative tools that may work better for your use case. The right Docker Desktop alternative for you will depend on your specific needs and requirements.



Extensibility is PostgreSQL's DNA, lies in its [original design](https://dsf.berkeley.edu/papers/ERL-M85-95.pdf).

![design](/content/blog/top-postgres-extension/design.webp)

This design philosophy grants PostgreSQL a lot of unique capabilities, one of them is its extension system. With [Postgres extension](https://www.postgresql.org/docs/current/extend-extensions.html), 3rd parties can extend the Postgres capabilities without touching any Postgres core.

Today most production Postgres deployments run some extensions. Below we present those most commonly
used ones.

| Extension                               | Capability                     |
| --------------------------------------- | ------------------------------ |
| [PostGIS](#postgis)                     | Process geospatial data        |
| [pg_stat_statements](#pgstatstatements) | Collect execution stats        |
| [postgres_fdw](#postgresfdw)            | Query external PostgreSQL data |
| [uuid-ossp](#uuid-ossp)                 | Generate UUID                  |
| [pgcrypto](#pgcrypto)                   | Cryptographic functions        |
| [pg_cron](#pgcron)                      | Schedule job inside database   |
| [pgAudit](#pgaudit)                     | Audit Logging                  |
| [timescaledb](#timescaledb)             | Process time-series data       |
| [pgvector](#pgvector)                   | Process vectorized data        |

## PostGIS

[PostGIS](http://postgis.net/) extends the PostgreSQL by adding support storing, indexing and querying geographic data.
PostGIS is the most complex Postgres extension and a testimony of the Postgres powerful extension system.

To find the nearest city to a given point:

1. Say we have the following table of cities with their locations represented by points. Note, the `location` column has a GEOMETRY type which is provided by the PostGIS extension.

   ```sql
   CREATE TABLE cities (
   name TEXT,
   location GEOMETRY(Point, 4326)
   );
   ```

1. To find the nearest place to a given point, you can use the ST_Distance function to calculate the distance between the point and each place in the table, and then sort the results by distance. For example, the following command finds the nearest city to the point (-74.005941, 40.712784), which is the location of New York City:

   ```sql
   SELECT name, ST_Distance(location, ST_SetSRID(ST_MakePoint(-74.005941, 40.712784), 4326)) AS distance
   FROM cities
   ORDER BY location <-> ST_SetSRID(ST_MakePoint(-74.005941, 40.712784), 4326)
   LIMIT 1;
   ```

This query calculates the distance between each city in the cities table and the point (-74.005941, 40.712784), and sorts the results by distance using the `<->` operator. The LIMIT 1 clause returns only the nearest city.

Note that the ST_Distance function returns the distance between two points in meters by default. You can convert the result to a different unit of measurement by using the appropriate PostGIS function, such as ST_Distance_Sphere for distance in kilometers.

## pg_stat_statements

[pg_stat_statements](https://www.postgresql.org/docs/current/pgstatstatements.html) provides a means for tracking planning and execution statistics of all SQL statements executed by a server. When pg_stat_statements is active, it tracks statistics across all databases of the server.
The statistics gathered by the module are made available via a view named `pg_stat_statements`.

Note that the pg_stat_statements extension only tracks queries that have been executed since it was enabled. If you want to track all queries, you should enable the extension at server start-up by adding the following line to your postgresql.conf file:

```ini
shared_preload_libraries = 'pg_stat_statements'
```

To find the top 10 queries by total execution time:

```sql
SELECT query, total_time
FROM pg_stat_statements
ORDER BY total_time DESC
LIMIT 10;
```

## postgres_fdw

[postgres_fdw](https://www.postgresql.org/docs/current/postgres-fdw.html) can be used to access data stored in external PostgreSQL servers. `postgres_fdw` is the successor of the old `dblink` extension. `postgres_fdw` provides more transparent and standards-compliant syntax for accessing remote tables, and can give better performance in many cases.

With `postgres_fdw`, you can query other Postgres database.

1. Create a new database that you want to query. For example, let's create a database named my_other_database:

   ```sql
   CREATE DATABASE my_other_database;
   ```

1. Connect to the database where you want to create the foreign table (in this example, we'll use the default postgres database).

1. Create a user mapping for the user that will access the remote database. For example, if you want to use the same user that you are currently connected as, you can run the following command:

   ```sql
   CREATE USER MAPPING FOR current_user
   SERVER my_other_database
   OPTIONS (user 'postgres', password '');
   ```

1. Create a foreign server definition using the postgres_fdw extension.

   ```sql
   CREATE SERVER my_other_database_server
   FOREIGN DATA WRAPPER postgres_fdw
   OPTIONS (dbname 'my_other_database');
   ```

   This command creates a server definition named `my_other_database_server` using the postgres_fdw foreign data wrapper and the dbname option set to `my_other_database`.

1. Create a foreign table definition in the local database that maps to a table in the remote my_other_database database.

   ```sql
   CREATE FOREIGN TABLE my_other_table (
   id INTEGER,
   name TEXT
   )
   SERVER my_other_database_server
   OPTIONS (schema_name 'public', table_name 'my_table');
   ```

   This command creates a foreign table named my_other_table in the local database that maps to a table named my_table in the public schema of the my_other_database database.

1. Use the foreign table in queries just like you would a regular table.

   ```sql
   SELECT * FROM my_other_table WHERE id = 1;
   ```

   You can also join the foreign table with local tables in your queries, just like you would with regular tables.

Note that when using postgres_fdw to query a remote database on the same PostgreSQL instance, you may need to adjust the postgresql.conf file and restart the PostgreSQL server to enable access to the pg_hba.conf file for the other database.

## uuid-ossp

[uuid-ossp](https://www.postgresql.org/docs/current/uuid-ossp.html) provides functions to generate universally unique identifiers (UUIDs) using one of several standard algorithms. Postgres already has built-in function `gen_random_uuid()` to generate a version 4 (random) UUID. If you want to generate other UUID version, you need to use `uuid-ossp`.

To generate a version 5 UUID:

```sql
SELECT uuid_generate_v5(uuid_ns_url(), 'example.com');
```

This command generates a UUID version 5 based on the namespace identifier for URLs (uuid_ns_url()) and the name string 'example.com'. The output will look something like this: `f1f5d9f0-2a4c-5f24-9536-3f1f69e68a7e`.

You can also create your own namespace identifier using the uuid-ossp function uuid_ns_create().

```sql
SELECT uuid_ns_create('example');
```

This command creates a namespace identifier using the name 'example' and returns it as a UUID.

You can then use this namespace identifier with uuid_generate_v5() to generate UUIDs based on that namespace and a name string.

Note that UUID version 5 is recommended for use in applications where security is a concern, as it is generated using a SHA-1 hash of the namespace identifier and name string, which is less susceptible to collisions than other UUID versions.

## pgcrypto

[pgcrypto](https://www.postgresql.org/docs/current/pgcrypto.html) is a PostgreSQL extension that provides cryptographic functions and capabilities directly within the database. It enhances data security by allowing various cryptographic operations to be performed within SQL queries.

1. Hashing

   ```sql
   SELECT digest('data to hash', 'sha256');
   ```

1. Encrytion / decryption

   ```sql
   SELECT pgp_sym_encrypt('my secret data', 'my passphrase');
   SELECT pgp_sym_decrypt(encrypted_data, 'my passphrase');
   ```

1. Password hashing with salt

   ```sql
   INSERT INTO users (username, password_hash)
   VALUES ('bob', crypt('foobar123', gen_salt('xyz')));
   ```

1. Public key cryptography

   ```sql
   SELECT pgp_pub_encrypt('data', dearmor('-----BEGIN PGP PUBLIC KEY BLOCK----- ...'));
   SELECT pgp_pub_decrypt(encrypted_data, dearmor('-----BEGIN PGP PRIVATE KEY BLOCK----- ...'));
   ```

## pg_cron

[pg_cron](https://github.com/citusdata/pg_cron) is a simple cron-based job scheduler that runs inside the database as an extension. It uses the same syntax as regular cron, but it allows you to schedule PostgreSQL commands directly from the database.

The schedule uses the standard cron syntax.

![cron](/content/blog/top-postgres-extension/cron.webp)

1. Create a new cron job by running the following command

   ```sql
   SELECT cron.schedule('0 0 * * *', 'INSERT INTO my_table SELECT * FROM my_other_table');
   ```

1. Verify that the cron job has been created

   ```sql
   SELECT cron.jobid, cron.expr, cron.command FROM cron.job;
   ```

1. View the status of running and recently completed job

   ```sql
   select * from cron.job_run_details order by start_time desc limit 5;
   ```

## pgAudit

[pgAudit](https://github.com/pgaudit/pgaudit) is an extension for PostgreSQL that provides detailed session and/or object audit logging via the standard logging facility provided by PostgreSQL. It is designed to help database administrators and developers meet security and compliance requirements by providing detailed information on database activities.

It helps in meeting regulatory requirements like GDPR, HIPAA, SOX, and others by providing an audit trail of database activity.

```sql
-- Usage
SET pgaudit.log = 'read, ddl';

CREATE TABLE account
(
    id INT,
    name TEXT,
    password TEXT,
    description TEXT
);

INSERT INTO account (id, name, password, description)
             VALUES (1, 'user1', 'HASH1', 'blah, blah');

SELECT * FROM account;
```

```sql
-- Output
AUDIT: SESSION,1,1,DDL,CREATE TABLE,TABLE,public.account,create table account
(
    id int,
    name text,
    password text,
    description text
);,<not logged>
AUDIT: SESSION,2,1,READ,SELECT,,,select *
    from account,,<not logged>
```

## timescaledb

[timescaledb](https://www.timescale.com) provides optimized storage and querying of time-series data.

1. Create a hypertable

   A hypertable is a special type of table in TimescaleDB that is designed for storing and querying time-series data. You can create a hypertable using the CREATE_HYPERTABLE function.

   ```sql
   CREATE TABLE sensor_data (
   time TIMESTAMP NOT NULL,
   value FLOAT NOT NULL
   );
   SELECT create_hypertable('sensor_data', 'time');
   ```

1. Insert some data into the sensor_data table

   ```sql
   INSERT INTO sensor_data (time, value)
   VALUES
   ('2023-07-01 00:00:00', 10.0),
   ('2023-07-01 01:00:00', 15.0),
   ('2023-07-01 02:00:00', 20.0);
   ```

1. Query the data

   TimescaleDB provides a number of optimized functions for working with time-series data, such as time_bucket for aggregating data into time intervals. For example, to calculate the average value for each hour of data, you can run the following query:

   ```sql
   SELECT time_bucket('1 hour', time) AS hour, AVG(value) AS avg_value
   FROM sensor_data
   GROUP BY hour;
   ```

## pgvector

[pgvector](https://github.com/pgvector/pgvector) is an extension for PostgreSQL that provides support for vector processing. It allows you to perform vectorized operations on groups of data, which can provide significant performance improvements for certain types of queries.

To get the nearest neighbors to a vector:

1. Create a new table with a vector column

   ```sql
   CREATE TABLE items (id bigserial PRIMARY KEY, embedding vector(3));
   ```

1. Insert vectors

   ```sql
   INSERT INTO items (embedding) VALUES ('[1,2,3]'), ('[4,5,6]');
   ```

1. Query the nearest neighbors to a vector

   ```sql
   SELECT * FROM items ORDER BY embedding <-> '[3,1,2]' LIMIT 5;
   ```

## Summary

Postgres extension is a key differentiator from its main alternative MySQL. If the business requires geospatial
processing, then Postgres is the only choice thanks to the PostGIS extension. And in the era of AI, pgvector is on
the way to become the de-factor standard for processing vector data.

## Further Readings

- [Top Postgres GUI client](/blog/top-postgres-gui-client)
- [Top Open Source Postgres Migration Tools](/blog/top-open-source-postgres-migration-tools)
- [Postgres vs. MySQL](/blog/postgres-vs-mysql)
- [Postgres vs. MongoDB](/blog/postgres-vs-mongodb)



> If you don't monitor Postgres, you are not serious.

It's year 2024 and Postgres is keeping the momentums. The rise of [pg_vector](https://github.com/pgvector/pgvector), [Supabase](https://supabase.com/) and [Neon](https://neon.tech/) fuel the Postgres adoption. This post reviews the tools and best practices for monitoring your Postgres database.

## Open-source Tools

### pg_stat_statements

The first thing is to enable the `pg_stat_statements` extension.

```sql
CREATE EXTENSION pg_stat_statements;
```

`pg_stat_statements` tracks planning and execution statistics of all SQL statements executed by a server. It's
also a prerequisite for other more advanced monitoring solutions.

```shell
psql=# SELECT query, calls, total_exec_time, rows, 100.0 * shared_blks_hit /
               nullif(shared_blks_hit + shared_blks_read, 0) AS hit_percent
          FROM pg_stat_statements ORDER BY total_exec_time DESC LIMIT 5;
-[ RECORD 1 ]---+--------------------------------------------------​------------------
query           | UPDATE pgbench_tellers SET tbalance = tbalance + $1 WHERE tid = $2
calls           | 3000
total_exec_time | 20756.669379
rows            | 3000
hit_percent     | 100.0000000000000000
```

### explain.dalibo.com

![dalibo](/content/blog/top-postgres-monitoring-tools/dalibo.webp)

[explain.dalibo.com](https://explain.dalibo.com/) is a web UI to visiualize the Postgres EXPLAIN query plans.
It's also [open sourced on GitHub](https://github.com/dalibo/pev2) and can be embedded into your own web applications.

### Prometheus Postgres Exporter

[Prometheus Postgres Exporter](https://github.com/prometheus-community/postgres_exporter) extract Postgres database metrics and store them in Prometheus. From there, you can use Prometheus Alertmanager to configure alerting rules
and use Grafana to create dashboards.

## Proprietary Solutions

### pganalyze

![pganalyze](/content/blog/top-postgres-monitoring-tools/pganalyze.webp)

[pganalyze](https://pganalyze.com/) is a Postgres specific observability service. It provides in-depth SQL tuning
and performance monitoring. For the Scale plan, it starts with $399 per month including 4 database servers, each
additional server costs $100.

### DataDog

![datadog](/content/blog/top-postgres-monitoring-tools/datadog.webp)

DataDog provides [Postgres Database Monitoring](https://docs.datadoghq.com/database_monitoring/setup_postgres/selfhosted/). Its Postgres monitoring is not as extensive as pganalyze. DataDog's advantage is its integration
with the entire application stack and the CI/CD pipeline. The price starts at $70 per database server per month.

### Other Options

- [pgDash](https://pgdash.io/) has a similar feature set and pricing point to pganalyze. pgDash looks less
  polished. On the other hand, pgDash offers self-hosted option for all plans, whereas pganalyze only offers self-hosted option for the Enterprise plan.

  ![pgdash](/content/blog/top-postgres-monitoring-tools/pgdash.webp)

- [SolarWinds Database Performance Monitor (DPM)](https://www.solarwinds.com/database-performance-monitor) also provides Postgres database performance solution. Its technology comes from the VividCortex acquisition. It's cloud-only and has a steep starting price from $3,096.

## Best Practices

### Monitor Transaction ID Wraparound

Due to PostgreSQL's MVCC implementation, it has an infamous [transaction ID wraparound](https://www.postgresql.org/docs/current/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND) risk. When this happens, it will render the entire
database unavailable, see some past incidents $[Sentry](https://blog.sentry.io/transaction-id-wraparound-in-postgres/), [Mailchimp](https://mailchimp.com/what-we-learned-from-the-recent-mandrill-outage/), [Figma](https://www.figma.com/blog/post-mortem-service-disruption-on-january-21-22-2020/)$. Every capable Postgres monitoring service and database service provider can check this risk:

- [pganalyze checks every 30 minutes](https://pganalyze.com/docs/checks/vacuum/txid_wraparound)
- [Implement an Early Warning System for Transaction ID Wraparound in Amazon RDS for PostgreSQL](https://aws.amazon.com/blogs/database/implement-an-early-warning-system-for-transaction-id-wraparound-in-amazon-rds-for-postgresql/)
- [GCP Cloud SQL high-transaction-ID-utilization recommender](https://cloud.google.com/sql/docs/postgres/recommender-high-transactionid-utilization)

### Monitoring Locks

Use [pg_locks](https://www.postgresql.org/docs/current/view-pg-locks.html) view to find any lock problems. You can
find connections that have held locks for a long time as well as connections that have waited too long to acquire the lock.

You should also consider enabling [log_lock_waits](https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-LOCK-WAITS).

### Avoid Blocking Operations

It's always a good practice to set [lock_timeout](https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-LOCK-TIMEOUT) on the user.

When making DDL changes, try to use non-blocking statements. e.g. add index with [CONCURRENTLY](https://www.postgresql.org/docs/current/sql-createindex.html), add constraint with [NOT VALID](https://www.postgresql.org/docs/current/sql-altertable.html#SQL-ALTERTABLE-DESC-ADD-TABLE-CONSTRAINT).

<HintBlock type="info">

You can configure [SQL Review](https://docs.bytebase.com/sql-review/overview/) in Bytebase and detect such anti-patterns.

</HintBlock>

### Use a Cloud Postgres Service Provider

Every major Postgres database service provider has basic monitoring out of the box. If you have to self-host
due to budget or compliance reasons, then you can start with Prometheus/Grafana setup and upgrade to
pganalyze/DataDog as business grows.

---

A comprehensive monitoring solution can be intimidating. We try to provide actionable suggestions
to companies from different stages. Please let us know if you have other monitoring tools and best practices worth sharing to herd the elephant.



Compared to the self-host option, teams pay 30% ~ 100% markup for the managed database service, hoping it can take care of the operational burdens.
While managed database service does reduce the operational cost, it's not enough. Let's illustrate this by a simple example.

## How to revert a single-row update

All mainstream managed database services provide disaster recovery capability. Take [AWS RDS](https://aws.amazon.com/rds/features/backup/)
as an example, it offers both normal restore and PITR (Point-in-time Restores).

![_](/content/blog/why-managed-database-service-is-not-enough/rds-br.webp)

However, neither is suitable to revert a single-row update. Because it would revert not only that change, but also other
valid changes.

## Managed database service alone is not enough

Managed database service takes care of the **infrastructure challenges**:

- Provisioning
- Scaling
- Disaster recovery
- Monitoring

On the other hand, developers still face a lot of **workflow challenges**:

- How to propagate the change from dev, test to staging, prod.

- How to roll back changes.

- How to spot and tune the slow query.

- How to copy the production data to the testing environment for development.

Additionally, security engineers will emphasize the necessity of sanitizing data in advance. They also require assurance that all data access is properly controlled and meticulously recorded to maintain security and compliance.

## How to address workflow challenges

There are 3 angels to tackle the workflow challenges.

1. **Managed database service**

   - For MySQL, there is [PlanetScale](https://planetscale.com/) offering a full suite of built-in capabilities.

   ![_](/content/blog/why-managed-database-service-is-not-enough/planetscale-tech.webp)

   - For Postgres, [Neon](https://neon.tech/) provides instant branching.

   ![_](/content/blog/why-managed-database-service-is-not-enough/neon-branching.webp)

1. **Application platform**

   [Supabase](https://supabase.com/) also debuted a branching feature.

1. **Tooling**

   If teams want to avoid vendor lock-in, they would adopt database tools working with a variety of vanilla database systems.

   - To coordinate database changes, [Jira is a typical option with apparent drawbacks](/blog/use-jira-for-database-change).

   - To sanitize production data and synthesize data for development. There are [Snaplet](https://www.snaplet.dev/) and [Neosync](https://www.neosync.dev/).

   <HintBlock type="info">

   Bytebase is also a database tool providing change management, database access control, dynamic data masking for all mainstream databases.

   </HintBlock>

## Summary

The database development lifecycle involves multiple key roles, including developers, DBAs, security engineers, and platform engineers. Traditional managed database services primarily address infrastructure needs, leaving many workflow challenges unresolved. These challenges must be addressed either through enhanced capabilities of the managed database services, application platforms, or third-party tools to ensure a comprehensive solution.



Database automation manages database tasks with minimal human intervention. Schema changes. Deployments. Access provisioning. Audit. The result: fewer errors, consistent policy, DBAs working on architecture instead of approving every `ALTER TABLE`.

Six levels, named after the autonomous-driving framework. Each level removes a class of manual work and exposes the next bottleneck.

## The 6 levels at a glance

| Level  | Name              | What's automated                               | What still needs a human | Typical tooling                |
| ------ | ----------------- | ---------------------------------------------- | ------------------------ | ------------------------------ |
| **L0** | No automation     | Nothing                                        | Everything               | Direct DB clients              |
| **L1** | Ticketing         | Request, approval, audit trail                 | Execution, review        | Jira, ServiceNow + DB client   |
| **L2** | Version controlled | Schema history, change scripts                | Deployment               | Git + migration files          |
| **L3** | Streamlined       | Deployment via CI/CD                           | Cross-tool stitching     | Liquibase, Flyway, CI runner   |
| **L4** | Integrated        | Change + ad-hoc access + audit in one platform | Strategic decisions      | Bytebase, equivalent platforms |
| **L5** | Fully automated   | End-to-end including self-healing              | Oversight only           | Aspirational                   |

![roaring-dba](/content/blog/_shared/roaring-dba.webp)

## Level 0 — No Automation

**Level 0 is when every database task is done by hand**, with no system between the operator and the database. Most teams never officially leave it. Higher-level tooling accumulates on top while one-off operations stay manual underneath.

- **Manual changes** — DBAs run SQL from a database client.
- **No version control** — no history of who changed what when.
- **Permission sprawl** — access granted by hand, outliving its reason.
- **No audit trail** — reconstructing what happened needs log forensics.

## Level 1 — Ticketing

**Level 1 is when a ticketing system gates database changes** — Jira, ServiceNow, or similar. The change itself is still executed manually. Most organizations sit here. The system records the request, approval, and audit trail. It does not prevent a typo from reaching production.

- [Ticketing system](/blog/use-jira-for-database-change) tracks the request and decision.
- Approval gates the change.
- Approver hands the ticket to a DBA, who opens a client and runs the SQL.
- Audit lives in the ticket. Searching across many tickets is hard.

## Level 2 — Version Controlled

**Level 2 is when schema changes live in version control** alongside application code, as migration files committed to Git. Teams that have adopted DevOps for application code reach this level for the database next.

- Schema and migrations tracked in Git.
- Each change is a versioned SQL file, applied in order.
- Pull-request review on schema changes.
- Deployment is still manual — someone runs the tool against an environment.

## Level 3 — Streamlined

**Level 3 is when database changes deploy through a CI/CD pipeline**. Rollback and environment promotion built in. Migration tools apply changes deterministically without a human running them.

- CI/CD applies changes to dev, staging, prod.
- [Liquibase](/blog/bytebase-vs-liquibase/) or [Flyway](/blog/bytebase-vs-flyway/) manage the apply-and-track loop.
- Failed deployments roll back automatically (within SQL's limits).
- Gap: ad-hoc work — emergency fixes, transient access grants — still bypasses the pipeline.

## Level 4 — Integrated

**Level 4 is when planned changes and ad-hoc tasks live in the same platform**, with shared approval, audit, and access models. L2 and L3 handle planned changes. Ad-hoc work — patching a row, granting a temporary credential, running a one-off `ANALYZE` — falls outside the version-controlled model. L4 closes the gap.

- Schema changes and ad-hoc operations flow through the same pipeline.
- One-off access grants follow the same approval flow as schema changes.
- Standing permissions sync from external identity providers (Okta, AD, LDAP).
- All human-to-database actions land in one searchable audit log.

<HintBlock type="info">

[Bytebase](/blog/database-as-code/) operates at Level 4. Schema changes, data changes, ad-hoc access, and permission grants flow through one workflow. One audit log captures all of it.

</HintBlock>

![change-query-secure-govern-database-all-in-one](/content/blog/_shared/db-scheme-lg.png)

## Level 5 — Fully Automated

**Level 5 is fully automated end-to-end**. Self-healing systems handle failures. No human handles routine operations — the database equivalent of a robotaxi. No production team operates here today.

- End-to-end automation from request to deployment to monitoring.
- Self-healing: automatic rollback when monitoring detects regressions.
- Compliance and security checks built into the pipeline.

Whether L5 should be a goal is contested. Most teams keep a human approving destructive operations.

## Where most teams sit

Across the database teams we work with: **L0 ~5%** (early-stage), **L1 ~40%** (most enterprise teams), **L2 ~25%**, **L3 ~20%** (CI/CD for schema, ad-hoc still outside), **L4 ~10%** (compliance-driven), **L5 0%**.

L1 is the most exposed cluster. Governance is in place — tickets, approvals, audit. Execution safety is not. A typo in production looks the same at L1 as at L0.

## FAQ

**What is database automation?**
Database automation manages database tasks — schema changes, deployments, access provisioning, audit — with minimal human intervention. The result: fewer errors, consistent policy, DBAs working on architecture instead of approving every change.

**What are the 6 levels of database automation?**
Six levels, named after the autonomous-driving framework. L0 No Automation. L1 Ticketing (approval, manual execution). L2 Version Controlled (schemas in Git). L3 Streamlined (CI/CD deployment). L4 Integrated (planned and ad-hoc tasks in one platform). L5 Fully Automated (end-to-end with self-healing).

**Is database CI/CD the same as database automation?**
Database CI/CD is one layer — Level 3, automated deployment of planned schema changes through a pipeline. Full automation also covers governance (L1), version control (L2), and ad-hoc work that doesn't fit a pipeline (L4). A team running Liquibase in GitHub Actions has database CI/CD. It is not yet at Level 4.

**What tools enable database automation?**
Jira or ServiceNow for L1 ticketing. Git plus migration files for L2. Liquibase or Flyway in CI/CD for L3. Platforms like Bytebase that combine schema change, ad-hoc access, and audit for L4. Most teams run a mix — Jira for governance, Flyway for apply. The gap between them is what L4 platforms close.

**How mature is database automation today?**
Most enterprise teams sit at Level 1 (~40%) or Level 2 (~25%). About 20% have reached Level 3 with CI/CD pipelines for schema changes. Roughly 10% have closed the ad-hoc gap at Level 4. Level 5 is not in production at any team we've worked with.

**What's the highest level of database automation?**
Level 5 — end-to-end automation with self-healing, no human handling routine operations. Theoretical highest, not deployed. No production team runs without human approval for destructive operations. Most explicitly choose not to. Level 4 is the highest seen in production.



As the engineering team grows and more database change operations are executed on a daily basis, we need to find a centralized way to coordinate and enforce the database change process. A quick approach is to rely on the existing in-house ITSM (IT service management) or a simple ticketing system. Among them, Jira is probably the most popular option.

## A Typical Jira Setup

1. Define a [custom Jira workflow](https://www.atlassian.com/software/jira/guides/workflows/overview#what-is-a-jira-workflow).
   You may define a dedicated status set for the database change process. e.g. `Created`, `Reviewing`, `Pending Rollout`, `Completed`.

1. Define a [custom Jira issue type](https://support.atlassian.com/jira-cloud-administration/docs/add-edit-and-delete-an-issue-type/)
   and some [custom fields](https://support.atlassian.com/jira-cloud-administration/docs/configure-issue-custom-fields/) such as `SQL`, `Database`, `Rollout Time`.

1. Create roles for the reviewers (normally DBA or the project owner) and requesters (normally the developer) respectively.

## Database Change Workflow

<HintBlock type="info">

The similar workflow can also be applied to other database tasks such as requesting query access and data export.

</HintBlock>

1. The developer creates a Jira issue to request the database change. She fills the issue with the `SQL`, `Database` info. The issue status is `Created`.

1. The DBA gets assigned to the issue according to the configured workflow. DBA reviews the SQL and leaves a comment under the issue. The issue status is `Reviewing`.

1. After several back-and-forth communications, the DBA approves the issue and changes the issue status to `Pending Rollout`.

1. The DBA pastes the SQL from the Jira ticket into their favorite SQL client and execute.

1. The DBA updates the issue status to `Completed`.

## Space for Improvement

With Jira, teams can now have a centralized place to review and coordinate the database changes. However, there is still plenty of space for improvement.

### Disconnected Review and Rollout process

- The DBA needs to manually paste the SQL to a different place to execute it. The DBA could paste the wrong SQL or execute against the wrong
  database (**the infamous mistake of pointing to the production database incorrectly**).

- Change histories are obscure. It's hard to track why, when, and how a database change happens.

### Lack of Customization for the Database Domain

Database changes can get quite complex:

- Propagating the changes across different environments.

- Batch changing multiple databases having the same schema (typical for multi-tenant/multi-region setups).

- Enforcing automatic SQL lint checks.

- Streamlining rollbacks.

It's challenging to force a general issue ticketing system to handle the specialized database tasks.

## Why Bytebase

Many customers come to [Bytebase](/) from Jira because of the aforementioned challenges. Similar to Jira,
Bytebase has the concepts of `Project`, `Issue`. Additionally, Bytebase defines some database domain-specific concepts as first-class citizens, such as `Database Instance`, `Database`, `Environment`. Bytebase provides an integrated
experience to plan, review, and deploy database changes.

![Issue detail interface](/content/blog/use-jira-for-database-change/issue-detail.webp)



We have been working with PostgreSQL and other databases for a while. In the realm of database management, PostgreSQL stands out for its robustness and flexibility. However, as developers navigate the complexities of modern application development, there are additional functionalities that could streamline their workflows. Here are the features I wish PostgreSQL had to make developers' lives easier:

## Versioned Schema

The PG catalog only stores the current schema. Schema migration tools usually need to create an external table to store the schema migration history. A built-in versioned schema will be beneficial:

1. Standardize the format. e.g. The version string, the status indicating whether the schema change is in progress.
2. Consistency. When schema migration history is stored externally, it's hard to guarantee the consistency between the migration history and the actual schema.
3. Rollback. If a new schema change introduces issues, versioning allows for safer rollbacks to previous stable states, minimizing downtime and impact on users.
4. Process enforcement. With a versioned schema and the migration history, it's easier to enforce that only a single schema migration can run on a database at any point in time.

## Online Schema Migration

PostgreSQL has been improving this area over the years, but there are still cases where altering the schema would block the database. [pg-osc](https://github.com/shayonj/pg-osc), [reshape](https://github.com/fabianlindfors/reshape) are built to tackle this. Ideally, PostgreSQL can encapsulate the long schema migration process:

- Temp tables created as part of the migration are internal tables. Thus it won't pollute the user space.
- Postgres exposes a migration progress percentage.

## State-based Migration (Declarative)

The typical way to do schema migration is to compose a list of `ALTER TABLE` statements. This becomes hard to track the latest schema state as the migration accumulates. It's more intuitive for the developers to specify the desired state. Ideally, PostgreSQL could allow developers to specify the desired `CREATE TABLE` schema, the engine then reconcile it with the latest schema, figure out the diff, and plan the migration path internally.

## Branching

During development, it's desirable to have a high-fidelity testing environment as close as the prod environment. The challenge is to fork the production database quickly enough (for compliance, it also needs to sanitize the sensitive data). What if PostgreSQL could have the instant git-like branching?

## Archived Table

To prevent deleting data by mistake, we invent soft-delete pattern by having a `is_deleted` column. However, this brings extra complexities around foreign key, unique index enforcement. Ideally, PostgreSQL could allow users to configure an archived table. The removed data is moved to the archived table first and purged after a configured retention period. This simplifies application logic and data compliance work.

## Label

PostgreSQL allows adding [SECURITY LABEL](https://www.postgresql.org/docs/current/sql-security-label.html) to the objects. I wish it could be extended to a more generic label system where users can attach arbitrary KV pairs. This would be useful in data dictionary, reporting, and orchestrating developer workflow. e.g.

- Find all tables/databases owned by a particular application.
- Require additional review when changing tables matching a certain label.

## Git Integration

Snowflake added this one in the [2023 summit](https://medium.com/snowflake/streamlining-devops-with-snowflake-and-git-integration-fc0b76a40a76):

![snowflake-git](/content/blog/features-i-wish-postgres-had/snowflake-git.webp)

This allows developers to keep the stored procedures, user-defined functions, and operational scripts in the same place as the application code. Thus they will adopt the same version control best practices.

Similar to how Foreign Data Wrapper (FDW) is implemented, the core PostgreSQL could provide the underlying infrastructure, and the specific VCS integration can be implemented as an extension.

## Summary

As to the database development workflow, Git is always the source of inspiration. Git is invented to
manage the stateless code, the hard part is to adapt Git to manage the stateful database. Companies
have already been pushing the frontiers:

- [Neon](https://neon.tech/) rebuilds the Postgres engine to enable serverless and instant branching.
- [Xata](https://xata.io/) offers serverless, branching, as well as zero-downtime schema migration.
- [Snaplet](https://snaplet.dev/) for seeding and sanitization.
- [Supabase](https://supabase.com/) for an entire application platform around Postgres.
- [Bytebase](https://bytebase.com/) for unifying all human-to-postgres operations including schema migration, ad-hoc change, database access control, and dynamic masking.

What's your most desired PostgreSQL feature?



GitOps is a modern approach to manage infrastructure that emphasizes automation, collaboration, and continuous delivery. It’s based on the idea of using Git as the single source of truth for configuration and code. With GitOps, all changes to infrastructure are made through pull requests, which are reviewed and approved by other team members before being merged into the main branch.

In this post, we are reviewing several open-source GitOps tools:

- [Terraform](#terraform)
- [Pulumi](#pulumi)
- [ArgoCD](#argocd)
- [Bruno](#bruno)
- [Bytebase](#bytebase)
- [OPA](#opa)
- [mdx](#mdx)

[![star-history](/content/blog/top-open-source-gitops-tools/star.webp)](https://www.star-history.com/#hashicorp/terraform&pulumi/pulumi&argoproj/argo-cd&usebruno/bruno&open-policy-agent/opa&mdx-js/mdx&bytebase/bytebase&Date)

## Terraform

[Terraform](https://github.com/hashicorp/terraform) is an open-source infrastructure as code tool (IaC) that lets you build, change, and version infrastructure safely and efficiently. This includes low-level components like compute instances, storage, and networking; and high-level components like DNS entries and SaaS features.

Terraform uses HashiCorp Configuration Language (HCL) and also supports JSON for writing configurations.
![terraform](/content/blog/top-open-source-gitops-tools/terraform.webp)

To enable GitOps, create a Git repo with Terraform configs, define infrastructure in Terraform, and manage updates via a pipeline and pull requests.

![terraform-gitops](/content/blog/top-open-source-gitops-tools/terraform-gitops.webp)

## Pulumi

[Pulumi](https://github.com/pulumi/pulumi) is an open-source infrastructure as code tool that allows you to create, deploy, and manage cloud infrastructure using your favorite language. Unlike Terraform, Pulumi uses real programming languages like Python, JavaScript, and Go to define infrastructure.

![pulumi](/content/blog/top-open-source-gitops-tools/pulumi.webp)

Similar to Terraform, Pulumi can be used to enable GitOps by creating a Git repo with Pulumi configs, defining infrastructure in Pulumi, and managing updates via a pipeline and pull requests.

![pulumi-gitops](/content/blog/top-open-source-gitops-tools/pulumi-gitops.webp)

## ArgoCD

[ArgoCD](https://github.com/argoproj/argo-cd) is a declarative, GitOps continuous delivery tool for Kubernetes.

![argocd](/content/blog/top-open-source-gitops-tools/argocd.webp)

Argo CD follows the GitOps pattern of using Git repositories as the source of truth for defining the desired application state. Kubernetes manifests can be specified in several ways such as kustomize applications, helm charts and jsonnet files.

Argo CD automates the deployment of the desired application states in the specified target environments. Application deployments can track updates to branches, tags, or pinned to a specific version of manifests at a Git commit.

![argocd-diff](/content/blog/top-open-source-gitops-tools/argocd-diff.webp)

## Bruno

[Bruno](https://github.com/usebruno/bruno) is a Fast and Git-Friendly open-source API client, aimed at revolutionizing the status quo represented by Postman, Insomnia and similar tools out there.

![bruno](/content/blog/top-open-source-gitops-tools/bruno.webp)

Bruno stores your collections directly in a folder on your filesystem with a plain text markup language, Bru, to save information about API requests.

Regarding GitOps, you can use git or any version control of your choice to collaborate over your API collections.

![bruno-gitops](/content/blog/top-open-source-gitops-tools/bruno-gitops.webp)

## OPA

[OPA](https://github.com/open-policy-agent/opa) (Open Policy Agent) is an open-source policy engine designed to enforce policies across your stack. While OPA is not a GitOps tool itself, it can complement GitOps workflows by providing policy enforcement and compliance checks.

![opa](/content/blog/top-open-source-gitops-tools/opa.webp)

In the context of GitOps, OPA can be integrated into your CI/CD pipelines or Kubernetes admission controllers to validate changes to your infrastructure and applications. By enforcing policies defined in code, OPA ensures that all changes comply with your organization's standards before they are applied, enhancing the security and reliability of your GitOps processes.

## MDX

[MDX](https://github.com/mdx-js/mdx) is a tool that allows you to write JSX in Markdown documents, enabling the creation of interactive and dynamic content using React components within your Markdown files.

![mdx](/content/blog/top-open-source-gitops-tools/mdx.webp)

While MDX is not a traditional GitOps tool, it can complement GitOps workflows by enabling version-controlled documentation. This ensures that your documentation evolves alongside your code and infrastructure, maintaining alignment with the latest changes.

## Bytebase

[Bytebase](https://github.com/bytebase/bytebase) is an all-in-one database DevSecOps and CI/CD solution. It's like GitHub/GitLab that provides a GUI workspace for developers and DBAs to collaborate on database changes.

![bb](/content/blog/top-open-source-gitops-tools/bb.webp)

Besides its intuitive UI workflow, Bytebase also supports a GitOps workflow for managing database changes. This feature allows for initiating database issues through pull requests. Furthermore, Bytebase provides SQL review, custom approval workflows, and rollout policies, enabling database changes to be managed in an advanced GitOps manner.

![bb-rollout](/content/blog/top-open-source-gitops-tools/bb-rollout.webp)

## Summary

To summarize, these open-source GitOps tools offer a range of options for managing infrastructure and application deployments. They adhere to the GitOps principles, automating the deployment process to achieve the desired states in specified target environments.



Once upon a time, a developer named John was modifying the name column in a large table from `VARCHAR(20)` to `VARCHAR(512)`. This change was intended to accommodate more diversity in name lengths, a request that had come down from the product team as they expanded into new, international markets. The exact SQL is:

```sql
ALTER TABLE person MODIFY name VARCHAR(512);
```

Bob, the Database Administrator (DBA), reviewed John's migration script. On the surface, it seemed like a straightforward change, one that he had seen and approved many times in his career. Bob gave his blessing to the change.

With Bob's approval, John executed the migration. As soon as the migration was deployed, alarms blared throughout the office.

The root cause is subtle. Even MySQL 8.0 has increased the online DDL coverage, this particular statement is still not covered. Because the statement enlarges the byte to encode the size value from 1 to 2 as described in the [official doc](https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html).

![mysql-extend-varchar](/content/blog/schema-migration-outage-fault/mysql-extend-varchar.webp)

Thus all other transactions were blocked while waiting for this DDL to finish. And for large tables, that would take hours.

**Though the industry is advocating the [blameless culture](https://sre.google/sre-book/postmortem-culture/), someone needs to be accountable. Otherwise, no one will own the initiative to fix the process and the chances are it will happen again.**

Is this the author, John’s fault? Maybe. However, it’s a bit more ask for an average developer to know the execution plan of a particular statement from a particular database at a particular version.

Is this the reviewer, Bob’s fault? Plausible. Since Bob has more knowledge and was hired exactly for catching this type of error. However, if Bob takes the bullet, there will be less incentive for the developers to improve their database knowledge.

The good thing is, that such dilemmas have become less nowadays:

1. Companies are moving more DB responsibilities to the developers. There will still be a DB person or a platform team taking care of the DB infrastructure, but the day-to-day DB development tasks are managed by the developer team entirely.
1. Companies may use an external DBA consulting service like [Vettabase](https://vettabase.com/), [Percona](https://www.percona.com/), [Enterprise DB](https://www.enterprisedb.com/) for expert advice.
1. Companies may adopt database development tools like Bytebase to improve the collaboration between the developers and the DB team.

Welcome to the era of Database DevOps.


The
Bytebase Blog

Explore the standard for database development

Solutions & Platform

Comparisons

Legal

Resources

Company