pecl:mysqlnd_ms
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
pecl:mysqlnd_ms [2012/04/17 11:29] – [Development steps (release planning)] uw | pecl:mysqlnd_ms [2017/09/22 13:28] (current) – external edit 127.0.0.1 | ||
---|---|---|---|
Line 157: | Line 157: | ||
=== 1.4 - Tweaking based on user feedback | === 1.4 - Tweaking based on user feedback | ||
+ | |||
+ | == Coding/ | ||
+ | |||
+ | * [done] More fail over options | ||
+ | * [done] Automatic on-connect fail over, if activated, shall be done in a loop until a connection can be opened. Currently we stop after the first attempt. If automatic fail over is on, we try " | ||
+ | * [done] Remember failed hosts for the duration of a web request (the plugins' | ||
+ | |||
+ | * [done] Refine QoS session consistency server selection policy | ||
+ | * [done] Support "wait for GTID". Currently we loop over all servers until we find a matching one. MySQL 5.6 allows SQL users either to fetch the latest GTID or SQL users can ask for a GTID and their request will block until the GTID has been replicated on the server. We should support the latter logic as well. | ||
+ | |||
+ | * [done] Improve load balancing | ||
+ | * [done] Allow users to set a ratio/ | ||
+ | |||
+ | == QA == | ||
+ | |||
+ | * [done] 85-90% code coverage through tests | ||
+ | |||
+ | == Documentation, | ||
+ | |||
+ | * [done] Update documentation | ||
+ | * [done] Assorted blog postings | ||
+ | |||
+ | === 1.5 - Tweaking based on user feedback | ||
+ | |||
+ | Work has not started yet. | ||
== Coding/ | == Coding/ | ||
Line 163: | Line 188: | ||
* [open] Silent and automatic connection fail over if server returns configured error code | * [open] Silent and automatic connection fail over if server returns configured error code | ||
* NOTE: this may require refactoring of four filters. | * NOTE: this may require refactoring of four filters. | ||
- | * [open] | + | * [open] |
* [open] Support directing statements manually to a group of nodes for more efficient server cache usage | * [open] Support directing statements manually to a group of nodes for more efficient server cache usage | ||
- | * [open] Refine QoS session consistency server selection policy | ||
- | * [open] Support "wait for GTID". Currently we loop over all servers until we find a matching one. MySQL 5.6 allows SQL users either to fetch the latest GTID or SQL users can ask for a GTID and their request will block until the GTID has been replicated on the server. We should support the latter logic as well. | ||
* [open] Remember the most current server and test this one first when searching for a GTID (a synchronous server). Use of cached information is possible for the duration of a read-only request sequence. The cache must be flushed and refreshed for every write. | * [open] Remember the most current server and test this one first when searching for a GTID (a synchronous server). Use of cached information is possible for the duration of a read-only request sequence. The cache must be flushed and refreshed for every write. | ||
- | |||
- | * [open] Improve load balancing | ||
- | * [open] Allow users to set a ratio/ | ||
* [open] Work with php.net Windows team to provide PECL/ | * [open] Work with php.net Windows team to provide PECL/ | ||
+ | * Ensure the code builds | ||
+ | * Encourage Windows team to create builds | ||
== QA == | == QA == | ||
Line 184: | Line 206: | ||
* [open] Update documentation | * [open] Update documentation | ||
* [open] Assorted blog postings | * [open] Assorted blog postings | ||
- | |||
- | === Open and delayed tasks === | ||
- | |||
- | == Coding/ | ||
- | |||
- | * [incomplete - 1.2] make load balancer schemata aware | ||
- | * for each statement detect database/ | ||
- | * follow schemata based partitioning rules | ||
- | * basic idea: http:// | ||
- | * cancelled/ | ||
- | * [cancelled] improve trx_stickiness using MYSQL*/ | ||
- | * cancelled: http:// | ||
- | * [open] bail on unknown config settings. For example, bail if someone uses round_robin instead of roundrobin | ||
- | |||
- | == QA == | ||
- | |||
- | * [open] >= 90% code coverage through tests | ||
- | * [open] write report on parsing overhead, goal: <5% in worst case micro benchmarks | ||
- | |||
- | == Documentation, | ||
- | |||
- | * [done] impact of connection limits such as allow_persistent, | ||
Line 216: | Line 216: | ||
== Feature description == | == Feature description == | ||
- | === Eventual consistent: automatic chaining of PECL/ | + | === More fail over configuration options |
== Problem to solve / Idea == | == Problem to solve / Idea == | ||
- | MySQL replication is asnychronous. Slaves are //eventual consistent// | + | Whenever |
- | + | ||
- | MySQL replication used for read scale-out requires applications | + | |
- | + | ||
- | For certain queries, PECL/ | + | |
- | + | ||
- | (MySQL semi-synchronous replication does not change | + | |
- | **PECL/ | + | The fail over logic itself is basic. Smaller improvements will make it much more capable. |
== Feature description == | == Feature description == | ||
- | Applications shall be able to define a " | ||
- | * [RW_]CONSISTENT | + | Automatic fail over is basic. We shall: |
- | * [RO_]EVENTUAL_CONSISTENT | + | |
- | With CONSISTENT every statement shall be run on the master | + | - make it configurable whether fail over node search tries one or all possible alternatives |
+ | - link fail over to certain error codes | ||
+ | - remember failed hosts to skip them for the rest of the web request | ||
- | PECL/ | + | Up to version |
- | However, PECL/ | + | Whether this shall become a new default or become configurable is to be decided. |
- | /* Force use of master - consistent read although no SQL hint is used */ | + | Furthermore, |
- | mysqlnd_ms_set_service_level($link, CONSISTENT); | + | |
- | $link-> | + | |
- | $link-> | + | |
- | + | ||
- | /* Use master or slaves for reads - allow read stale, non-consistent data is acceptable */ | + | |
- | mysqlnd_ms_set_service_level($link, | + | |
- | $link-> | + | |
- | + | ||
- | The service level EVENTUAL_CONSISTENT | + | |
- | If PECL/ | + | Upon request |
- | + | ||
- | Use and integration of PECL/ | + | |
+ | === Support directing statements manually to a group of nodes for more efficient server cache usage === | ||
- | === Global Transaction ID === | ||
== Problem to solve / Idea == | == Problem to solve / Idea == | ||
- | In the scope of MySQL master - slave replication global transaction IDs can be understood as a measure | + | In large clusters users can improve performance by optimizing query distribution and server selection using criteria such as cache usage, distance or latency. Application developers shall be allowed |
- | The MySQL built-in replication does not feature global transaction IDs. MySQL replication uses transaction log files and log file positions | + | Server cache usage can be optimized by distributing queries |
- | If, for example, your setup is A -> B -> C (A is the master, B is a slave reading from A, C is a slave reading from B) and B fails you are faced with the challenge | + | Some nodes may be co-located |
- | + | ||
- | Today, it can become | + | |
- | + | ||
- | Fail-over and circular-replication is much easier | + | |
- | + | ||
- | The plugin | + | |
== Feature description == | == Feature description == | ||
- | Global transaction IDs can be used for multiple purposes: | + | For every node in the configuration users shall be able to set one or more group names. A SQL hint, for example, MS_GROUP=name |
- | * Simplify server fail-over (server-side, | ||
- | * Monitor slave replication status (client-side, | + | === Refine QoS session consistency server selection policy === |
- | * Identify " | + | |
- | * Identify slaves which are " | + | |
- | * Let the load balancer skip slaves which have fallen to far behind to reduce their load | + | |
- | The plugin shall support using a global transaction ID for: | ||
- | |||
- | - server fail-over (e.g. promoting new master after master failure) | ||
- | - selection of " | ||
- | |||
- | Gobal transaction ID generation shall be done by SQL injection. The plugin shall insert SQL at the end of each (write) transaction (see http:// | ||
- | |||
- | The SQL used for globals transaction ID generation can be different for the different purposes. It may be good enough to inject SET @_g_trx_id=UUID_SHORT() on the master before a SQL transaction commits to help with fail-over. Whereas the selection of slaves of a certain state (" | ||
- | |||
- | Thus, the user shall be able to configure: | ||
- | |||
- | * SQL used for ID generation | ||
- | * SQL used to retrieve ID | ||
- | |||
- | If no SQL is given to retrieve the ID, the plugin does not feature '2) selection of " | ||
- | |||
- | This is useful if, for example, an application does a write and needs to read the modified data " | ||
- | |||
- | /* write to master */ | ||
- | $link-> | ||
- | $write_trx_id = mysqlnd_ms_get_last_global_transaction_id(); | ||
- | | ||
- | /* Force use of master - consistent read although no SQL hint is used */ | ||
- | mysqlnd_ms_set_service_level($link, | ||
- | $link-> | ||
- | $link-> | ||
- | |||
- | /* Read from a slave which has replicated global transaction ID or, if no slave found, use master */ | ||
- | mysqlnd_ms_set_service_level($link, | ||
- | $link-> | ||
- | $link-> | ||
- | | ||
- | /* Use master or slaves for reads - allow read stale, non-consistent data is acceptable */ | ||
- | mysqlnd_ms_set_service_level($link, | ||
- | $link-> | ||
- | |||
- | The global transaction ID feature of the plugin is turned off by default. The feature is seen as an experts topic. | ||
- | |||
- | === mysqlnd_ms_change_user() === | ||
== Problem to solve / Idea == | == Problem to solve / Idea == | ||
- | Give users a way to overrule all automatic decisions, including | + | Users can request session consistency. Session consistency guarantees that a user will only be redirected |
== Feature description == | == Feature description == | ||
- | bool mysqlnd_ms_change_user(mixed $connection, | + | Two additional ways for finding candidates help lowering the overhead: |
- | + | ||
- | $where: MYSQLND_MS_ALL_CONNECTIONS, | + | |
- | + | ||
- | Consider: | + | |
- | | + | * Wait for GTID |
+ | | ||
- | === mysqlnd_ms_get_errors() === | + | There are two ways to check in MySQL 5.6 if a server has replicated |
- | == Problem | + | |
- | + | ||
- | Not all MS errors can be bubbled up through the standard APIs. For example, failing to parse parts of the MS config may result | + | |
- | + | ||
- | == Feature description == | + | |
- | + | ||
- | array mysqlnd_ms_get_errors([mixed $connection]) | + | |
- | + | ||
- | TDB. | + | |
- | + | ||
- | === More fail over configuration options === | + | |
- | == Problem to solve / Idea == | + | |
- | + | ||
- | Whenever the plugin tries to connect to a node it may fail to do so. Connection attempts | + | |
- | + | ||
- | + | ||
- | == Feature description == | + | |
- | + | ||
- | Up to version 1.3 the automatic fail over stops after trying one alternative. For example, if a connection to A fails we try B. If connecting to B fails we stop and bail out. In that case the user must handle the error although there may be node C and D, ... which could be used. | + | |
- | + | ||
- | In the future the search | + | |
- | + | ||
- | Whether this shall become a new default or become configurable | + | |
- | + | ||
- | Furthermore, | + | |
- | + | ||
- | === Support directing statements manually to a group of nodes for more efficient server cache usage === | + | |
- | + | ||
- | == Problem to solve / Idea == | + | |
- | + | ||
- | In large clusters users can improve performance by optimizing query distribution and server selection using criteria such as cache usage, distance or latency. Application developers shall be allowed to annotate a statement in a way that its executed on a certain group of nodes. | + | |
- | + | ||
- | Server cache usage can be optimized by distributing queries in a way that they hit hot caches. For example, clients may want to run all accesses to table A, B and C on the node group 1 and table accesses to D preferably on group 2. Because we do not support automatic table filtering/ | + | |
- | + | ||
- | Some nodes may be co-located to clients whereas others may be remote. This is often done in large systems when storing multiple copies, e.g. on the machine, on the rack, in the same data center, in a different data center. In case of MySQL Replication its unlikely to find such highly optimized setups, however, there may be nodes closer to the client than others. Nodes closer to a client may be given a certain alias or group name and application developers shall be allowed to hint routing to a group of such nodes. | + | |
- | + | ||
- | == Feature description == | + | |
- | + | ||
- | For every node in the configuration users shall be able to set one or more group names. A SQL hint, for example, MS_GROUP=name can be used to hint the load balancer to direct a request to a certain | + | |
+ | Furthermore we can persist GTID/ | ||
pecl/mysqlnd_ms.1334662154.txt.gz · Last modified: 2017/09/22 13:28 (external edit)