NXC Contributions to the Open Source Community
by Cindy Guignard | December 5, 2013 10:33 am
While NXC specializes in customizing open source solutions for our clients, we also work to enhance open source offerings available to the entire open source community. Recently, we have contributed a number of modules for use with Varnish.
Varnish Cache is an open source web/HTTP accelerator that is often deployed on websites in order to provide both a faster, better user experience and to conserve server resources. Varnish speeds up the opening of website pages by storing a copy of a page the first time it is opened by a customer. The next time a user requests the same page, Varnish serves the customer the copy page instead of returning to the web server to request the same page again.
For enterprises, this means that their servers need to handle less traffic, and website performance and scalability are both enhanced. For customers, this means that the page loads faster, and they have a better experience in interacting with the website. However, current Varnish open source software had some limitations which NXC needed to overcome in order to provide their clients with the best solutions.
In working with Varnish Cache, NXC System Administrator Sergey Poznyakoff has created a number of open source modules. All of these modules are released under the GNU General Public License. Three of them are currently accessible in the Varnish module list, and more are waiting for approval.
- Varnish has approved and released a Varnish Cache module created by NXC that implements basic HTTP authentication against password files created with the Apache htpasswd utility. This module allows the use of basic http-auth with Varnish, which is necessary for websites in development that are protected with http-auth, but also need to work with Varnish.
- NXC has also created a special vmod-binlog module that provides application programming interface (API) for writing binary log files from Varnish VCL scripts and the utilities for listing and searching in such files. A binary log file consists of an arbitrary number of equally sized records. Each record contains a UNIX timestamp in binary form and arbitrary user-defined data. These files are convenient for storing large amounts of information arriving at high rates.
- NXC created vmod-dbrw, a Varnish Cache module available from the Varnish site, implementing database-driven rewrite rules for use with web sites that need an exceedingly large number of redirects and/or rewrites. Many enterprises need their website to handle a large number of redirects. Unfortunately, having many redirects slows down every page load on the website because to load a page all redirect rules have to be checked one by one. This module speeds up the time for a user to be redirected to the desired page by making it possible to keep all rules in an SQL database of arbitrary structure, considerably speeding up their handling. Rewrite rules are stored in a database that is easier to maintain, .and the module supports both MySQL and PostgreSQL.
- Another module created by NXC is vmod-tbf, designed to implement token bucket filtering for Varnish Cache. This module provides a means of controlling request rates and can be configured to allow a limited number of queries from one IP address. It can also be configured to check other criterion to see if content is already available in cache to protect dynamic pages. When installed and used to shape search robots that shut down a website, this module reduced the peak load from 20-30 to 4-5, and the average load from 4-5 to 1-2. Only search robots were shaped, and no actual customers received temporary errors because of this protection. This module can be used as a DDOS protection from unprofessional hackers or offered as a part of a package.
For more information on these open source modules from NXC, visit Sergey’s web page. You can also visit the following links on Varnish main site: https://www.varnish-cache.org/vmod/basicauth, https://www.varnish-cache.org/vmod/tbf-token-bucket-filtering, https://www.varnish-cache.org/vmod/database-driven-rewrites and finally https://github.com/nxc/nxc_varnish, another contribution from one of our experts, Serhey Dolgushev.
Source URL: http://blog.nxcgroup.com/2013/nxc-contributions-to-the-open-source-community/