Web Application Firewalls (WAFs) are a critical layer in modern web security, providing a website’s first line of defense against vulnerabilities. WAFs can be used to defend against and notify on attempted exploits, allowing for mitigations faster than organizations can patch vulnerable software. For a global CDN, this functionality must be implemented in a way that is sensitive to performance, providing response times on the order of milliseconds. When we first introduced a WAF engine to the VDMS stack three years ago, we selected the ModSecurity Rules Engine, which we found to be first-rate for individual WAF use cases. Furthermore, ModSecurity’s support of the OWASP Core Rule Set (CRS), powerful rule language, and API access to the HTTP traffic stream in real time offered significant flexibility.
However, as the number of customers using the WAF increased, we
began to experience performance and resource bottlenecks. In particular,
ModSecurity’s dense ruleset propagated across every customer instance, driving
memory and CPU utilization up across our network, which increased operational
costs. Additionally, testing and deploying new rules was difficult: the rule
language was often unwieldy and difficult to write and parse. These issues,
along with development complexity with the existing ModSecurity library, led to
the development of
waflz, an open source WAF engine, published under the
Apache 2.0 license.
waflz is a significant improvement on ModSecurity: it consumes less
memory, offers better performance, and is API-driven.
waflz supports a subset of
ModSecurity capabilities, the OWASP Core rulesets 2.x and 3.x, and several
waflz is designed from the ground up with
considerations for high performance and multi-tenancy. Where necessary, the
design traded for performance over flexibility. Ultimately
a restricted subset
of ModSecurity capabilities. For example, some ModSecurity
inspectFile were deemed unsuitable for
running on the edge for security and performance reasons.
The engine can be configured with rules in either ModSecurity format or json.
Indeed, the entire WAF product was designed to be “API-first”. To this end,
waflz was designed to have first-class support for json as both inputs and
waflz uses Google Protocol Buffers
internally to represent both
configuration (including rules) and alert formats. Choosing Protocol Buffers
allows for interoperability with json inputs and outputs, as well as adding
strictly typed schemas for both.
Some of the principle engineering challenges in a CDN are dealing with the high concurrency, and multi-tenancy that comes from serving thousands of customers: Any edge server anywhere around the globe has to be able to process a request for any of our customers as fast as possible. Furthermore, edge server applications must provide real-time patching and processing for any given customer configuration.
Fig 1. customer config patching on an edge server
Having many WAF rulesets loaded into the memory of the running HTTP Application
Server process on the edge across all of our customers quickly presented
a scalability challenge.
waflz addressed this issue by creating the WAF
rulesets only once in memory and sharing read-only references between the
customer configurations and rule customizations. Additionally, several
potential performance optimizations
were identified, which improved request processing times, including space and
time savings to some critical internal data structures.
waflz is part of the VDMS CDN technology stack and can operate at
massive scale while enabling efficient granular rule testing and customization.
Despite the challenges for CDN application,
ModSecurity and the new
libmodsecurity are fantastic flexible libraries, ideal for many use-cases.
Indeed, in the process of developing
to ModSecurity development. After a successful trial period, the new WAF engine
has been running in production globally for over a year, concurrently supporting
thousands of different client configurations.
If you are interested in exploring
waflz functionality take a look at some of
the examples over on github. If you need fast and accurate security for your
website, see our web application firewall services that are delivered as part of
the VDMS Cloud Security Solution
and the VDMS Website Acceleration Solution.