Companies like Facebook, Twitter, LinkedIn and Google have the task of operating and maintaining some of the biggest websites on the internet, and due to not only the amount of content they host, but the massive number of user accounts they must keep up and online at all times requires a massive database. These three companies have teamed up to create the perfect database designed to scale to massive proportions.
Facebook along host one of the largest MySQL database deployments ever built and as the service continues to grow, a vanilla distribution of Oracle's MySQL was just not cutting it, so Facebook teamed up with Google, LinkedIn and Twitter to develop a MySQL alternative, WebScaleSQL, that was both scalable and robust enough to keep the social networking empires running for the foreseeable future. WsSQL is built with collaboration in mind and speed in mind, and allows database engineers from different companies to work together to create the best database possible.
"Our goal in launching WebScaleSQL is to enable the scale-oriented members of the MySQL community to work more closely together in order to prioritize the aspects that are most important to us. We aim to create a more integrated system of knowledge-sharing to help companies leverage the great features already found in MySQL 5.6, while building and adding more features that are specific to deployments in large scale environments," Facebook said in the company's developers blog. "In the last few months, engineers from all four companies have contributed code and provided feedback to each other to develop a new, more unified, and more collaborative branch of MySQL."
Collaboration between database engineers from the four companies has already proved massively effective, and has allow WebScaleSQL to grow into a very powerful and scalable piece of infrastructure. The work done by WebScaleSQL partners has already aided in major changes and improvements to the new branch of MySQL, including the following:
- An automated framework that will, for each proposed change, run and publish the results of MySQL's built-in test system (mtr).
- A full new suite of stress tests and a prototype automated performance testing system.
- Several changes to the tests already found in MySQL, and to the structure of some existing code, to avoid problems where otherwise safe code changes had previously caused tests to fail or caused unnecessary conflicts. These changes make it easier to work on the code and helped us get started creating WebScaleSQL.
- Several changes to improve the performance of WebScaleSQL, including buffer pool flushing improvements (links here and here); optimizations to certain types of queries; support for NUMA interleave policy; and more.
- New features that make operating WebScaleSQL at true web scale easier, such as super_read_only, and the ability to specify sub-second client timeouts.
"We will keep all our WebScaleSQL work open, to create a useful branch for others within the MySQL community who are focused on scale deployments. We'll continue to follow the most up-to-date upstream version of MySQL. As long as the MySQL community releases continue, we are committed to remaining a branch - and not a fork - of MySQL," Facebook continued. "We're excited to expand our existing work on WebScaleSQL, and we think that this collaboration represents an opportunity for the scale-oriented members of the MySQL community to work together in a more efficient and transparent way that will benefit us all."