Pipdig Update: Dishonest Denials, Erased Evidence, and Ongoing Offenses

In last week’s post, we reported on some concerning code identified in the Pipdig Power Pack (P3) plugin. The plugin, which is installed alongside WordPress themes sold by Pipdig, was found to contain a number of suspicious or malicious features. Among these features were a remote “killswitch” Pipdig could use to destroy sites, an obfuscated function used to change users’ passwords, and code which generated hourly requests with the apparent intent of DDoSing a competitor’s site.

In the days since we published that report, Pipdig has taken a series of increasingly questionable steps in their attempts to mitigate the fallout of their actions. Their team has issued baseless accusations that facts have been fabricated, collusion between their competitors had taken place, and that no wrongdoing of any sort had occurred.

These assertions stand in direct conflict with their actions. They’ve pulled down incriminating files from their sites, pushed undocumented updates to their plugins to remove additional malicious code, and have attempted to rewrite history by modifying dates of changelog entries. Then, perhaps most egregiously, Pipdig took down the Bitbucket repository containing a great deal of evidence of these actions. All of this had been done while an entire community of WordPress developers watched.

In today’s followup, we’ll use Pipdig’s official responses to recap their documented offenses. Then, we’ll discuss the timeline of events in the deployment and subsequent removal of Pipdig’s malicious code. After that, we’ll look at the evidence they’ve made efforts to destroy. Last, we’ll reveal new evidence that Pipdig’s suspected DDoS campaign against their competitor had still been active until April 1st, using Blogger themes.

Recap: Pipdig’s Words vs. Pipdig’s Code

In the uproar following the publication of their missteps, Pipdig released (and subsequently updated) a statement with the intent of dispelling the controversy. Unfortunately, instead of admitting fault and apologizing to their users, Pipdig leaned into a series of accusations and denials.

Let’s take a look at the individual points of these responses, and why each fails to help Pipdig’s cause. We’ll avoid using code samples in this section, though the Timeline section that follows will contain code as evidence.

Unauthenticated WordPress Database Deletion

In our report, we explained how a cron (scheduled process) built into the P3 plugin was effectively asking Pipdig’s server every hour for permission to destroy the database of the site it’s running on. Pipdig hasn’t denied that the code existed, but has attempted to “rebrand” the behavior in a number of ways.

In total, we’ve received three unique answers from Pipdig about this issue. We have yet to confirm which is actually authoritative. The answers, in the order Pipdig made them, are:

Answer 1: This was an anti-piracy feature but Pipdig never used it.
Source: Email response from Phil Clothier

On Thursday, March 28, we reached out to Pipdig’s team for comment on the first set of issues we had confirmed in their plugin. We quickly received a response from Phil Clothier, which among other statements contained the following:

Last year we had some serious problems after someone obtained a huge list of license keys and downloaded all of our products. The keys and files were then distributed on their file sharing site, which has since been taken down (not by us, ironically!). The drop tables function was put in place to try to try to stop this at the time. […] The function was never used.

Answer 2: It doesn’t destroy the database. It’s there to reset a site to defaults. We use it with permission from site owners.
Source: Pipdig’s public response, published March 29th.

Pipdig’s immediate public response told a different story. After claiming that the claims were “not based on reality”, Pipdig asserts that to “clear” database tables is similar behavior to what would be found in a backup plugin.

“contains a ‘kill switch’ which drops all database tables”

This is the most serious accusation of all, and not one which we take lightly. The portrayal of this feature is not based on reality. There is a function in the plugin which can be used to clear database tables, much like a backup or standard reset plugin. To confirm, we do not have the ability to “kill” a site, nor would we ever, ever want to do that! The function is in place to reset a site back to defaults, however it is only activated after being in touch with the site owner.

There are three problems in this short reply.

First a “default” WordPress site is no different than a destroyed one. The next visitor to the “defaulted” site would be prompted with a setup page and have the ability to install themselves as the administrator in the newly generated WordPress database.

Second, any honest plugin developer providing a legitimate means for a user to destroy their own database (whether that’s a good idea or not is a different story) would leave that choice strictly to the user. Instead of offering a user-facing button surrounded by “Are you sure you want to delete this entire database?” warnings, the P3 plugin silently asks their servers once every hour if the database should be vaporized.

Third, this answer conflicts with the previous one. Is this code for anti-piracy or for a “factory reset”? Has it ever been used or not?

Answer 3: It was an anti-piracy feature after all, we did use it, and it was effective.
Source: Pipdig’s amended response, updated March 31st.

To our great surprise, Pipdig followed up two days later by issuing a direct contradiction to an assertion made on the very same page.

Why did this function exist?

The function was available in the pipdig Power Pack in July last year, when a serious incident occurred:

A 3rd party was able to download all of our themes illegitimately and post them on a clone of our own site. This included previews of our themes and the ability to purchase them. We were first alerted to this by people which had purchased a pipdig theme from there, but were finding that certain features did not work correctly. After investigation, we found that the victim had purchased the theme from the 3rd party, thinking it was us. The 3rd party not only gained the financial benefit of the theme payment, but also used it as a way to inject malware and ads into the victim’s site. The reset function was put in place in order to remove the 3rd party’s ability to host preview sites with our themes. It worked, and they have since disappeared. The function was then removed in a later version of the plugin.

In their final response on the subject, Pipdig reverts to calling it an anti-piracy feature. They claim it was introduced in July 2018, following a case of stolen theme files and licenses, and that “It worked”. This claim, true or not, stands against Phil’s original statement in his email that the feature was never actually used. Furthermore, they end this statement with the line “The function was then removed in a later version of the plugin.”

The two statements regarding the timeline of these events stand out. First, their claim that the database deletion code was introduced in 2018 is false, which we’ll discuss in more detail in the “Timeline” section of this post. Next, they leave the statement “a later version of the plugin” ambiguous, likely to sidestep the fact that it was removed days prior, and only after they were contacted.

Overall though, despite the firestorm of controversy surrounding this database deletion feature, it remains one of the less-interesting chapters of this story. The code is dangerous, and Pipdig’s difficulty in explaining it is striking, but there are more serious concerns in play.

Unauthenticated Password Reset

Our first report additionally detailed code built to reset a user’s WordPress password. While their responses to this have been consistent, they haven’t exactly been satisfactory.

Do you have the ability to log in to any site using a pipdig theme without permission?

No. If we need to log in to provide assistance with an issue, we will ask for it in the support request. This is stored securely whilst the ticket is open and not shared with any 3rd parties.

A simple “No.” was the extent of their refutation to this claim. This is unfortunate, because the code recently deleted from their plugin strongly disagrees.

Suspected DDoS Against A Competitor

One of the larger points of contention in this story pertains to code generating suspicious requests which originally targeted a competitor’s site. Once per hour, this code fetched the contents of a file hosted by Pipdig, which contained a target address for a subsequent outgoing request from the affected site’s server. Until immediately after Pipdig was contacted, the file hosted by Pipdig contained the admin-ajax.php URL of one of their direct competitors’ WordPress sites.

“is using other blogger’s servers to perform a DDoS on a competitor”

This function is used to pass the theme’s license key to an external server, which then passes that data to our main site at pipdig.co. We use the Cloudflare CDN to make sure the data is sent securely and as quickly as possible regardless of where your site is hosted globally. This data includes what domain the theme is installed on, as well as a link to the “Author URL” from the theme’s readme.txt file. This is used to activate a theme’s license key.

In their response, Pipdig claims these requests were used to activate a theme’s license key. They reported that these requests contain data like the domain of the site and the Author URL field from their theme’s readme.txt file.

None of this statement aligns with the actual behavior of the code. First, neither of the issued requests (both to Pipdig’s server and second request to the target’s site) contain any meaningful data, much less a license key. Second, the claim that an Author URL is pulled from readme.txt for the second request is false.

We’re now looking into why this function is returning this url. However it seems to suggest that some of the “Author URLs” have been set to ‘[redacted].com’. We don’t currently know why this is the case, or whether the site owner has intentionally changed this. The response should hit our site’s wp-admin/admin-ajax.php file under normal circumstances. On the surface it could mean that some pipdig themes have been renamed to other authors. We will be looking further into this issue and provide more information as it comes up. We can confirm that it won’t cause any issues for sites using pipdig themes, even if the author name/URL has been changed.

Further, Pipdig goes on to speculate that their competitor’s URL had somehow ended up replacing their own Author URI, which is why requests were directed to this competitor. This claim is baseless by itself, and the logic doesn’t hold up to scrutiny.

Pipdig doesn’t acknowledge that the competitor’s URL was actually retrieved from their own servers as opposed to any local source. They also don’t address how a license key would be validated by this code when no license is sent.

Moreover, assuming that this code actually pertained to a license checking feature, one would have to assume that their license checking would be broken entirely by the sudden removal of this code following the attention drawn to it. At the risk of editorializing, this seems unlikely.

Do you DDOS competitors?

No.

Pipdig’s amended response to this claim was much simpler, and equally verifiable.

Further Miscellaneous Claims

Additional claims were offered in our post and those from other developers who picked up the story over time. For the sake of brevity, we won’t be detailing them all in this post. Suffice to say, the community has been left hungry for a more substantial response to their concerns.

Removal of Evidence

During the investigation it was revealed that Pipdig maintained a public Bitbucket repository. This repo contained the commit history of the P3 plugin going back to 2015. We cloned this repository to ensure we wouldn’t lose access as time went on, and saved snapshots of the relevant Bitbucket webpages for future citation.

These efforts proved useful when, on March 31st, Pipdig removed this repository and later replaced it with a “clean” one.

To many, the removal of this source of evidence had changed the story significantly. Twitter users responded with uniform disbelief when this had taken place.

Pipdig update: after hiding their public Git repository that contained all of the malicious code and commit history for their plugin, they have published a new public repository in its place with all commit history wiped and more suspect code removed. Covering tracks @pipdig? pic.twitter.com/Y0hVZhrsU6

— Nicky Bloor (@nickstadb) April 1, 2019

When compared to other examples of concealed evidence, like their lack of a disclosure in changelogs and improper versioning, this action was notably suspicious.

One other change stands out, and while it’s less directly nefarious, it still draws a lot of questions about Pipdig’s goals. The new, “clean” repository features a curious difference in its changelog.

This version of the changelog claims version 4.8.0 was released on March 24th. However, commits from the actual repository showed that 4.7.1 was released on that date. In reality, it wasn’t until the March 29th that Pipdig’s first release of 4.8.0 was deployed.

We can only speculate regarding Pipdig’s motives in changing this date. Either way, it should be assumed that someone investigating malicious activity would notice a change like this.

Timeline of Events

In this section, we’ll give a step-by-step rundown of when certain actions were taken before and after the discovery of Pipdig’s behavior. Where necessary, this will include code samples and archive links as supporting citations. This won’t be exhaustive, especially in cases where many edits were made to individual lines of code over time, but should offer an authoritative and verifiable history of how the story unfolded.

In light of the fact that Pipdig has been witnessed attempting to conceal evidence of these issues, we have backed up some of our sources through third-party means. These services are known to be trustworthy in their space, and can serve as documentation that these claims are true. When available, we will provide alternate links to these sources in addition to their original location. Additionally, instances of code added and removed from the P3 plugin are supported by the hash of the relevant commit to the plugin’s git repository.

November 7, 2017 – First instance of database drop code
Git Commit: edc47824200e15d64cab7270debc4a0526a8d323
Public Changeset: Original Link | Archived Page
Notes: The database deletion feature in P3 was originally added in this commit, not in July 2018 as Pipdig stated in their public response.

June 21, 2018 – First instance of suspected DDoS code
Git Commit: dbcf38b0168472b7620b2372edac02cd847db68a
Public Changeset: Taken down by Pipdig before discovery.
Notes: This first instance of code making potential DDoS requests was added here. With comments and a remote path suggesting licensing checks, the inclusion of a spoofed User-Agent string and a lack of any license data in the outbound requeststrongly suggests malicious intent.

September 18, 2018 – First instance of password reset code
Git Commit: 8e9cf87c57bff6beda963489846d14df09d04cae
Public Changeset: Original Link | Archived Page
Notes: Both inc/functions.php and inc/cron.php were modified as part of this addition. Interestingly, minor edits were also made to the suspected DDoS script during this commit as well.

The next several months following the first inclusion of the password reset code featured a number of tweaks and changes. However, for the purposes of this report we’re going to jump ahead to just before the Pipdig story broke publicly.

March 25, 2019 – Most recent code updates to password reset and database deletion features
Git Commits:
3896a43db805c74e9f95ed51d4299f21e479c46d
ac66fc19e4e771a1700f6cf814ca7c5e800b0a48
Public Changeset: Taken down by Pipdig before discovery
Archival Note: While we were unable to archive the changeset of this commit before Pipdig took down the repository, we did archive an annotation of inc/cron.php which corroborates this statement. You can use the Find feature of your browser to search for the shortened commit tags 3896a43 and ac66fc1 to see the exact lines modified in this file.
Notes: Much of Pipdig’s messaging regarding their malicious code has suggested it was all old and defunct. If there was no intent for these to be used, why would they be making code improvements to these features less than a week prior?

March 27, 2019 – Wordfence notified of issues in P3 plugin

It was right around here that things started picking up speed, so we’ll be adding timestamps moving forward. Times are in PDT (GMT -7).

March 28, 2019

12:29 PM – Reached out to Pipdig for comment via email

2:19 PM – Received response from Phil

2:30 PM – Most offending code removed from plugin
Git Commit: d1f803ec4b40c9163a73f9f08e534051a8a9dec4
Public Changeset: Original Link | Archived Page
Notes: This release was tagged as version 4.8.0 when it went live. It still contained the inc/functions.php code responsible for the password reset feature, though the call to it was removed, leaving it orphaned. We mentioned this orphaned function in our original post.

March 29, 2019

9:40 AM – Wordfence releases Pipdig report
Link: https://www.wordfence.com/blog/2019/03/peculiar-php-present-in-popular-pipdig-power-pack-plugin/

Yesterday, we were notified of obfuscated backdoors present in @pipdig‘s helper plugin, Pipdig Power Pack.

After some research, and contact with the Pipdig team for comment, we put together this post discussing our concerns.https://t.co/05uuOudNYm via @wordfence

— Mikey Veenstra (@heyitsmikeyv) March 29, 2019

11:09 AM – Orphaned password reset code removed from plugin
Git Commit: 394e9a3293a45eedc32e512ea6680dd4a94107c8
Public Changeset: Original Link | Archived Page
Notes: The orphaned password reset code was removed less than two hours after we reminded Pipdig it existed. This commit was also tagged version 4.8.0 on release, in an apparent effort to conceal the removal of this code.

12:04 PM – @jemjabella releases Pipdig report
Link: https://www.jemjabella.co.uk/2019/security-alert-pipdig-insecure-ddosing-competitors/
Notes: Jem’s investigation had been running in parallel to our own, which we learned upon release of her report. Jem confirmed with Pipdig’s competitor that a harmful amount of incoming traffic had been hitting various admin resources on their site, strengthening allegations of DDoS activity. Jem’s active social following spread this report very quickly.

Popular WordPress theme provider pipdig is using customer sites to DDoS competitors & harvesting user data (amongst other things): https://t.co/MjfH7izT9x

— Jem (@jemjabella) March 29, 2019

2:24 PM – Pipdig issues first response
Link: https://pipdig.co/blog/sad-times/

Hi Laura, we’ve just published a reply to this at https://t.co/HFQyFvEmdB. If you have any questions/concerns please don’t hesitate to contact us

— pipdig (@pipdig) March 29, 2019

March 31, 2019

4:05 AM – Pipdig issues updated response
Link: https://pipdig.co/blog/sad-times/

Sorry we’ve been a bit absent on Twitter. Our post at https://t.co/HFQyFvEmdB has been updated. This is our final word on all of this. Thank you for all the positive responses we have received from people! It is hugely appreciated ❤️

— pipdig (@pipdig) March 31, 2019

12:40-2:19 PM – Pipdig takes down public Bitbucket repository containing incriminating code
Notes: At some point in this timeframe, Pipdig removed the public Bitbucket repository used by investigators to find specific evidence of their activity. Fortunately for the investigation, snapshots of several incriminating pages had already been taken, and the git repository itself had been cloned.

New Evidence Suggests DDoS Behavior In Pipdig’s Blogger Themes

During the investigation of Pipdig’s WordPress plugin and themes, we also came across some curious code associated with their Blogger themes. This code is part of Pipdig’s suspected DDoS campaign against their competitor, and was active until April 1, four days after Pipdig’s denial of any such behavior.

Some of Pipdig’s Blogger themes have been confirmed to make external JavaScript calls to Pipdig’s server, specifically to the script hXXps://pipdigz[.]co[.]uk/js/zeplin1.js.

This file contained two lines of obfuscated JavaScript code. That is, code written so humans can’t read it.

When the second line of this code is deobfuscated and made readable, we get the following:

Here’s a quick breakdown of what this code is up to.

The code is executed in the browser of the users visiting sites using Pipdig’s themes. This takes place on every page load where zeplin1.js is sourced from Pipdig.

First, Pipdig creates an iframe element in the visitor’s browser. Then it generates some random numbers, and selects a random WordPress endpoint from the following list:

• /wp-json/oembed/1.0/embed
• /wp-json/oembed/
• /wp-json/oembed/1.0/
• /wp-json/
• /wp-json/{random number}/
• /xmlrpc.php
• /wp-admin/admin-ajax.php

Next, it sets the actual target address value: hXXps://nullrefer[.]com/?hXXps://www.{competitor's domain}[.]com/{random endpoint}. For those unfamiliar, NullRefer is a service used to strip the referrer data out of requests to the second link in the path. What this means, is Pipdig uses NullRefer to hide the actual source page of the requests to their competitor’s site. This is understandable, because all of these referrers would be sites running Pipdig’s themes.

Finally, the rest of the JavaScript is used to take the iframe all of this happens in, and makes it invisible before actually loading it.

To clarify, every time a visitor reaches any site running a Blogger theme from Pipdig using this script, their browser would fire an additional request to their competitor’s site. This request hides where it came from, hits a literally randomized file on the competitor’s server, and does nothing with the data. This behavior is hidden not only from the visitors to these sites, but to the owners of these sites as well.

Supporting Evidence

Pipdig has made additional efforts to hide evidence of this behavior from zeplin1.js . On April 1, Pipdig removed the second line from this script on their servers. Luckily, we had assembled a number of authoritative copies proving it was present. This was made easier due to the great number of Internet Archive snapshots available of the script. Because references to it appear on a great deal of sites running Pipdig’s code, the archive’s crawlers come across it fairly often.

The first known instance of suspected DDoS code appearing in zeplin1.js was archived on January 31, 2019. An archival of the file on this date can be found on the Internet Archive. Interestingly, the code wouldn’t actually work in this first incarnation of the script – they forgot the .com in their competitor’s domain name. This was fixed shortly afterward.

Edits to this script came and went in the following weeks. Sometimes, the code was removed. Other times, a different competitor’s domain showed up. However, the most recent time the originally reported competitor’s domain was added suggests that Pipdig was still undergoing suspicious behavior even after they had been called out on it.

• On March 13, the suspected DDoS code was present. Archive Link
  • Note on this snapshot, the target was a different site than the commonly reported competitor.
• On March 31, the code was NOT present. Archive Link
• On April 1, the code WAS present. Archive Link
• On a later snapshot from April 1, the code had been removed. Archive Link

In the event that Pipdig has this evidence taken down as well, we have made additional verifiable copies of this code through other trusted third parties.

It is unclear at this time whether Pipdig thought this behavior would go unnoticed, though their addition and subsequent removal of this incriminating code at a time when they were under close scrutiny is highly suspicious.

Update: April 2, 10:35 PST

It has come to our attention that, at the time of this update, Pipdig is still hosting JavaScript which abuses their customers’ visitors.

The file, hXXps://pipdigz[.]co[.]uk/js/jquery.menu.min.js is currently hosting similarly obfuscated JavaScript which is issuing suspected DDoS attacks against another competitor of theirs.

In case this too is removed by Pipdig, we’ve created an archived copy for your personal review.

Pipdig Avoids Acknowledging Wordfence

It’s worth noting that in all of Pipdig’s communications regarding this story, Pipdig has pointed the finger at @jemjabella’s article and the claims within. When making baseless legal threats and claims of collusion, it’s understandable why they wouldn’t want to draw attention to the security company who has a clear reputation in this space.

Have @pipdig responded to @wordfence‘s complaints yet or is just my character they’re attacking?

— Jem (@jemjabella) March 30, 2019

We’ve made further attempts to reach Pipdig regarding this case. We had additional questions following the first response that weren’t answered, and further questions over data that popped up later. Pipdig, unsurprisingly, has provided no response.

Our Recommendation

The single exception to Pipdig’s unwillingness to bring up Wordfence can be found in today’s post on The Register. In their response, they cite our article from last week to back up their claims that the new version of the plugin is safe. In light of Pipdig’s behavior in responding to these issues, and their demonstrated continued abuse of their users and their users’ visitors, we have amended our original recommendation.

Wordfence recommends removing all Pipdig content from your sites, both WordPress and Blogger. Pipdig has demonstrated a willingness to abuse users’ resources to execute unethical, and potentially illegal, activity. Furthermore their repeated denial of solid evidence, and subsequent attempts to conceal it, leave us unable to trust them in the future.

Conclusion

We’ve deployed a malware signature, Backdoor:PHP/pipdigPasswordReset, which has already alerted hundreds of Wordfence users to the danger present in the P3 plugin. P3 users will also be receiving Wordfence dashboard notifications alerting them to our reports on these issues, if they haven’t received them already.

Pipdig’s repeated, documented activity in creating this malicious code is a major problem they needed to answer for. However, the rest of the story would have unfolded much differently had they just admitted wrongdoing from the start. Instead, they’ve shattered the trust of their users by trying to pull the wool over their eyes, and drew the ire of the WordPress community in their attempts to discredit these claims.