go-mysql-org / go-mysql

go-mysql A pure Go library to handle MySQL network protocol and replication as used by MySQL and MariaDB. Platform Support As a pure Go library, this project follows Go's minimum requirements. This library has been tested or deployed on the following operating systems and architectures: | Operating System | Architecture | Runtime Supported | CI | Notes | |------------------|--------------|-------------------|----|--------------------------------------------------------------------------------------------------------------------------------| | Linux | amd64 | ✅ | ✅ | Check GitHub Actions of this project. | | Linux | s390x | ✅ | ✅ | A daily CI runs on an s390x VM, supported by the IBM Z and LinuxONE Community. | | Linux | arm64 | ✅ | ✅ | Deployed in a production environment of a user. | | Linux | arm | ✅ | ❌ | A test in CI to make sure builds for 32-bits platforms work. | | FreeBSD | amd64 | ✅ | ❌ | Sporadically tested by developers. | Other platforms supported by Go may also work, but they have not been verified. Feel free to report your test results. This library is not compatible with TinyGo. Changelog This library uses Changelog. --- Content • Replication - Process events from a binlog stream. • Incremental dumping - Sync from MySQL to Redis, Elasticsearch, etc. • Client - Simple MySQL client. • Fake server - server side of the MySQL protocol, as library. • database/sql like driver - An alternative driver for MySQL. • Logging - Custom logging options. • Migration - Information for how to migrate if you used the old location of this project. Examples The directory contains example applications that can be build by running in the root of the project. The resulting binaries will be places in . • : parses a binlog file at a given offset • : streams binlog events from a server to canal • : streams binlog events • : like , but in Go • : fake MySQL server Replication Replication package handles MySQL replication protocol like python-mysql-replication. You can use it as a MySQL replica to sync binlog from master then do something, like updating cache, etc... Example The output looks: MariaDB 11.4+ compatibility MariaDB 11.4+ introduced an optimization where events written through transaction or statement cache have so they can be copied directly to the binlog without computing the real end position. This optimization improves performance but makes position tracking unreliable for replication clients that need to track LogPos of events inside transactions. To address this, a configuration option is available: **Behavior:** • When is and flavor is , the library automatically: • Adds flag to binlog dump commands. This ensures correct position tracking by making the server send events which are needed for accurate position calculation. • Calculates LogPos dynamically for events with that are not artificial. • Only works with MariaDB flavor; has no effect with MySQL. • Should be set to if tracking of LogPos inside transactions is required. Canal Canal is a package that can sync your MySQL into everywhere, like Redis, Elasticsearch. First, canal will dump your MySQL data then sync changed data using binlog incrementally. You must use ROW format for binlog, full binlog row image is preferred, because we may meet some errors when primary key changed in update for minimal or noblob row image. A simple example: You can see go-mysql-elasticsearch for how to sync MySQL data into Elasticsearch. Client Client package supports a simple MySQL connection driver which you can use it to communicate with MySQL server. For an example see . You can run this testable example with . Tested MySQL versions for the client include: • 5.5.x • 5.6.x • 5.7.x • 8.0.x Example for SELECT streaming (v1.1.1) You can use also streaming for large SELECT responses. The callback function will be called for every result row without storing the whole resultset in memory. will be filled before the first callback call. Example for connection pool (v1.3.0) Server Server package supplies a framework to implement a simple MySQL server which can handle the packets from the MySQL client. You can use it to build your own MySQL proxy. The server connection is compatible with MySQL 5.5, 5.6, 5.7, and 8.0 versions, so that most MySQL clients should be able to connect to the Server without modifications. Example Minimalistic MySQL server implementation: Another shell > will use default server configurations: > 1. automatically generate default server certificates and enable TLS/SSL support. > 2. support three mainstream authentication methods **'mysql_native_password'**, **'caching_sha2_password'**, and **'sha256_password'** > and use **'mysql_native_password'** as default. > 3. use an in-memory user credential provider to store user and password. > > To customize server configurations, use and create connection via . Driver Driver is the package that you can use go-mysql with go database/sql like other drivers. A simple example: Structured Connector If you prefer a structured configuration over a DSN string, you can use with : Driver Options Configuration options can be provided by the standard DSN (Data Source Name). Set a collation during the Auth handshake. | Type | Default | Example | | --------- | --------------- | ----------------------------------------------------- | | string | utf8_general_ci | user:pass@localhost/mydb?collation=latin1_general_ci | Enable compression between the client and the server. Valid values are 'zstd','zlib','uncompressed'. | Type | Default | Example | | --------- | ------------- | --------------------------------------- | | string | uncompressed | user:pass@localhost/mydb?compress=zlib | I/O read timeout. The time unit is specified in the argument value using golang's ParseDuration format. 0 means no timeout. | Type | Default | Example | | --------- | --------- | ------------------------------------------- | | duration | 0 | user:pass@localhost/mydb?readTime…