Questionable Traffic and How to Block with CloudFlare WAF

Questionable Traffic and How to Block with CloudFlare WAF

ยท

3 min read

A bit of story ๐Ÿ“–

Lately, my websites have been dealing with massive amount of requests. There are literally hundreds of thousands of request each month for about three consecutive months. Well, trafic's a good thing but pardon me for saying that traffic seems.. "questionable", at best ๐Ÿค”.

Fortunately for me, I host my sites on CloudFlare Pages with does not charge for network egress. It just my curiosity that causes me to dig deeper on this topic.

At first I thought it was DDOS until I see the analytics. The requests were different in location and also consistent for three months. Three months. Who in their right minds would DDOS my sites for three months in a row?? โ‰๏ธ
I mean it's fine if you want to attack like NASA or whatever, but come on now..

It just so happens that I stumbled across this video by Matt KC that then I started thinking about it seriously:

So I dug a little deeper and found out how this "Bytespider" bots have been wrecking people's nerve on the internet. Just look at this post by @generosus on the Wordpress forum from 9 months ago: (link: https://wordpress.org/support/topic/psa-bytedance-and-bytespider-bots-recommend-blocking/)

Or this one in the phpBB forum from 2020: (https://www.phpbb.com/community/viewtopic.php?t=2550501)

Well, I might be a bit late to the party since the anomaly has only occured to me lately. Nevertheless, here's how to block these bots with CloudFlare WAF in case you experiencing the same issue or just want to be careful.


Here goes ๐Ÿ›ก๏ธ

Well first of all of course you need a CloudFlare account and I will assume you already have one.

Step 1: Go to your dashboard and click on your site

For this tutorial I will go with justforvan.com.

Step 2: On the sidebar, Go to "Security" then "WAF"

Pretty straightforward I guess. Just click on "Security" first then choose "WAF".

Step 3: Click "Create Rule"

On the middle pane that uncover after you done step 2, click the blue button "Create rule".

Step 4: Fill in the rule

Fill it like so:

SectionContentModifiable?
Rule nameByte botsYES
FieldUser AgentNO
OperatorcontainsNO
ValueBytedanceNO
Choose actionBlockNO

After that, press the OR button on the side:

Then in the new row, fill it like this:

So the entire page will be like:

Or in tabular form:

Rule name

SectionContentModifiable?
Rule namebyte botsYES
FieldUser AgentNO
OperatorcontainsNO
Value 1BytedanceNO
Value 2BytespiderNO
Choose actionBlockNO

Step 5: Deploying the rule

And yes finally just click the sweet blue button of Deploy โœ….


Closing ceremony

And yes that's how you do it. Pretty straightforward I suppose. Goes pretty well when you serve your site through Cloudflare, like your Angular site that is deployed on Pages.

Hope this helps and I will be seeing you next time! ๐Ÿ˜๐Ÿ‘‹


Special mentions

Big thanks for the amazing people behind these posts:

https://community.cloudflare.com/t/blocking-user-agents/523190

https://stackoverflow.com/questions/57908900/what-is-the-bytespider-user-agent

https://wordpress.org/support/topic/psa-bytedance-and-bytespider-bots-recommend-blocking/

https://www.phpbb.com/community/viewtopic.php?t=2550501

https://www.johnlarge.co.uk/blocking-aggressive-chinese-crawlers-scrapers-bots/#comment-11823

ย