Wordfence Launches Short-Circuit Scan Signatures – Up to 6X Performance Increase

In October 2016, the Wordfence team started chatting about a way to radically boost the speed of scans once we grow beyond a certain number of scan signatures. As a reminder, a scan signature is a pattern that recognizes a certain kind of malware.

Today Wordfence has 4,523 signatures available for the free community, and we have an additional 226 new signatures that are only available to Wordfence Premium users. These become free once they are 30 days old.

New Malware Constantly Emerging

Our team continuously adds from 30 to over 100 new scan signatures each week. The site cleaning team constantly discovers new kinds of malware as they clean hacked websites, and each malware sample is turned into a scan signature and released to Wordfence to help it detect that malware.

Wordfence is currently at a total of 4,749 scan signatures for our Premium customers (4,523 free + 226 Premium), and within one year, this will grow to somewhere between 6,000 to 10,000 signatures at the current rate we are discovering new malware.

The constant increase in the amount of malware targeting WordPress is clear, and Wordfence needs to continually grow our scan signatures to keep pace.

Radical Innovation to Address Growth in WordPress Malware

In October last year, Matt Rusnak, who heads up QA for Wordfence, and Ryan Britton, who is our senior core developer for Wordfence, started chatting about a new algorithm to radically speed up the Wordfence scan and make it able to handle a much larger number of signatures.

Matt suggested identifying common patterns across scan signatures, grouping those signatures together and then checking if a file contains the common pattern first before scanning with the signatures in each group.

In theory, if we had 10,000 signatures, and if we are able to identify groups of 100 scan signatures and create a pre-check for each one, we would need to match only 100 scan signatures for every item we scanned instead of 10,000. Ryan dubbed this “short-circuiting.”

Earlier this year, we started work on the project. We created the services to support short-circuiting, then tested and launched them on our back-end servers about a month ago. Support was released for short-circuiting within the Wordfence plugin in the past few weeks. And we have been working to create grouped scan signatures with common “short-circuit” patterns.

Matt Barry, our lead developer who created the Wordfence firewall, worked with Ryan and Matt Rusnak to make this project happen, and they received help from other members of the engineering team.

A 2X to 6X Speedup With Short-Circuit Scanning

When we released Wordfence 6.3.17 late last week, you may have noticed an entry in the changelog which said, “Improvement: Prepared code for upcoming scan improvement which will greatly increase scan performance by optimizing malware signatures.”

On Monday this week, we enabled short-circuit scanning on our servers. The speed improvement in Wordfence scans was breathtaking, to say the least.

We are seeing a 2-to-6-times performance increase across our test sites and customer sites. This is an incredible improvement.

On one major hosting provider, scans on one of our large test sites went from an average scan time of 8 minutes per scan to 1 minute and 20 seconds for a Wordfence scan to complete.

Continuous Engineering Innovation in WordPress Security

This is not the first time we have radically improved scan speed. Last year in September we increased scan speed by refactoring the way we perform many operations in the scan.

In July of this year, we further improved scan performance for hosting providers by monitoring scan distribution across hosting provider VPS instances and introducing a smoothing algorithm.

Short-circuiting scan signatures is a powerful new technique the team has created to provide a radical performance improvement on an already fast scan.

While our many of our competitors don’t even provide a firewall and malware scan in their security products, the Wordfence engineering team is at the forefront of engineering innovation, ensuring that you benefit from a powerful firewall and malware scan combination with lighting-fast performance.

Congratulations and thank you to Matt Rusnak, Ryan Britton, Matt Barry and Åsa Rosenberg, who all contributed to bringing short-circuit scanning to our customers.