[RESOLVED] Denial of service attack underway, server load is very high

gaekwad · 2020-07-13 09:43:01

Server processor load is very high, there’s a denial of service attack going on.

I’m investigating. I’ll update this thread as I find and fix.

Edit: That was fun. We are mostly back to normal.

Server load hit 16 at one point, typically it hovers around 0.3.

Source has been identified, blocked and reported.

Longer version: a host in Ukraine was hammering an article with opportunistic (script kiddy) URL parameters. Examples:

5qgQtq - - [13/Jul/2020:10:19:23 +0000] "GET /articles/welcome-to-your-site?color=%23cc0000&date=01/01/1967&datetime-local=01/01/1967&email=sample%40email.tst&file_upload=&month=7&number=1&password=g00dPa%24%24w0rD&range=1&search=&select_dd=1&select_dd2=2&select_multi=1&tel=555-666-0606&test_checkbox_1=on&test_checkbox_2=on&test_checkbox_3=
on&test_checkbox_4=on&test_radio_group=&text=1&text-data=01/01/1967&text_inline=1&text_inline2=1&textarea=/WEB-INF/web.xml HTTP/1.1" 403 661 46.101.158.114 "https://default-theme.textpattern.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21" "-" "1.27"
5qgQtq - - [13/Jul/2020:10:19:23 +0000] "GET /articles/welcome-to-your-site?color=%23cc0000&date=01/01/1967&datetime-local=01/01/1967&email=sample%40email.tst&file_upload=&month=7&number=1&password=g00dPa%24%24w0rD&range=1&search=&select_dd=1&select_dd2=2&select_multi=1&tel=555-666-0606&test_checkbox_1=on&test_checkbox_2=on&test_checkbox_3=
on&test_checkbox_4=on&test_radio_group=&text=1&text-data=01/01/1967&text_inline=1&text_inline2=1&textarea=../../../../../../../../../../windows/win.ini HTTP/1.1" 403 810 46.101.158.114 "https://default-theme.textpattern.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21" "-" "1
.28"
5qgQtq - - [13/Jul/2020:10:19:23 +0000] "GET /articles/welcome-to-your-site?color=%23cc0000&date=01/01/1967&datetime-local=01/01/1967&email=sample%40email.tst&file_upload=&month=7&number=1&password=g00dPa%24%24w0rD&range=1&search=&select_dd=1&select_dd2=2&select_multi=1&tel=555-666-0606&test_checkbox_1=on&test_checkbox_2=on&test_checkbox_3=
on&test_checkbox_4=on&test_radio_group=&text=1&text-data=01/01/1967&text_inline=1&text_inline2=1&textarea=C:%5CWINDOWS%5Csystem32%5Cdrivers%5Cetc%5Chosts HTTP/1.1" 403 594 46.101.158.114 "https://default-theme.textpattern.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21" "-"
"1.19"
5qgQtq - - [13/Jul/2020:10:19:23 +0000] "GET /articles/welcome-to-your-site?color=%23cc0000&date=01/01/1967&datetime-local=01/01/1967&email=sample%40email.tst&file_upload=&month=7&number=1&password=g00dPa%24%24w0rD&range=1&search=&select_dd=1&select_dd2=2&select_multi=1&tel=555-666-0606&test_checkbox_1=on&test_checkbox_2=on&test_checkbox_3=
on&test_checkbox_4=on&test_radio_group=&text=1&text-data=01/01/1967&text_inline=1&text_inline2=1&textarea=../../../../../../../../../../windows/win.ini%00.jpg HTTP/1.1" 403 499 46.101.158.114 "https://default-theme.textpattern.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21"
 "-" "1.24"
5qgQtq - - [13/Jul/2020:10:19:23 +0000] "GET /articles/welcome-to-your-site?color=%23cc0000&date=01/01/1967&datetime-local=01/01/1967&email=sample%40email.tst&file_upload=&month=7&number=1&password=g00dPa%24%24w0rD&range=1&search=&select_dd=1&select_dd2=2&select_multi=1&tel=555-666-0606&test_checkbox_1=on&test_checkbox_2=on&test_checkbox_3=
on&test_checkbox_4=on&test_radio_group=&text=1&text-data=01/01/1967&text_inline=1&text_inline2=1&textarea=%c0%ae%c0%ae%c0%af%c0%ae%c0%ae%c0%af%c0%ae%c0%ae%c0%af%c0%ae%c0%ae%c0%af%c0%ae%c0%ae%c0%af%c0%ae%c0%ae%c0%af%c0%ae%c0%ae%c0%af%c0%ae%c0%ae%c0%afwindows%c0%afwin.ini HTTP/1.1" 403 222 46.101.158.114 "https://default-theme.textpattern.com
/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21" "-" "-"
5qgQtq - - [13/Jul/2020:10:19:23 +0000] "GET /articles/welcome-to-your-site?color=%23cc0000&date=01/01/1967&datetime-local=01/01/1967&email=sample%40email.tst&file_upload=&month=7&number=1&password=g00dPa%24%24w0rD&range=1&search=&select_dd=1&select_dd2=2&select_multi=1&tel=555-666-0606&test_checkbox_1=on&test_checkbox_2=on&test_checkbox_3=
on&test_checkbox_4=on&test_radio_group=&text=1&text-data=01/01/1967&text_inline=1&text_inline2=1&textarea=................windowswin.ini HTTP/1.1" 403 862 46.101.158.114 "https://default-theme.textpattern.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21" "-" "1.28"
5qgQtq - - [13/Jul/2020:10:19:23 +0000] "GET /articles/welcome-to-your-site?color=%23cc0000&date=01/01/1967&datetime-local=01/01/1967&email=sample%40email.tst&file_upload=&month=7&number=1&password=g00dPa%24%24w0rD&range=1&search=&select_dd=1&select_dd2=2&select_multi=1&tel=555-666-0606&test_checkbox_1=on&test_checkbox_2=on&test_checkbox_3=
on&test_checkbox_4=on&test_radio_group=&text=1&text-data=01/01/1967&text_inline=1&text_inline2=1&textarea=..%5c..%5c..%5c..%5c..%5c..%5c..%5c..%5cwindows%5cwin.ini HTTP/1.1" 403 667 46.101.158.114 "https://default-theme.textpattern.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/53
7.21" "-" "1.27"
5qgQtq - - [13/Jul/2020:10:19:23 +0000] "GET /articles/welcome-to-your-site?color=%23cc0000&date=01/01/1967&datetime-local=01/01/1967&email=sample%40email.tst&file_upload=&month=7&number=1&password=g00dPa%24%24w0rD&range=1&search=&select_dd=1&select_dd2=2&select_multi=1&tel=555-666-0606&test_checkbox_1=on&test_checkbox_2=on&test_checkbox_3=
on&test_checkbox_4=on&test_radio_group=&text=1&text-data=01/01/1967&text_inline=1&text_inline2=1&textarea=/.%5C%5C./.%5C%5C./.%5C%5C./.%5C%5C./.%5C%5C./.%5C%5C./windows/win.ini HTTP/1.1" 403 325 46.101.158.114 "https://default-theme.textpattern.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.222
8.0 Safari/537.21" "-" "1.23"
5qgQtq - - [13/Jul/2020:10:19:23 +0000] "GET /articles/welcome-to-your-site?color=%23cc0000&date=01/01/1967&datetime-local=01/01/1967&email=sample%40email.tst&file_upload=&month=7&number=1&password=g00dPa%24%24w0rD&range=1&search=&select_dd=1&select_dd2=2&select_multi=1&tel=555-666-0606&test_checkbox_1=on&test_checkbox_2=on&test_checkbox_3=
on&test_checkbox_4=on&test_radio_group=&text=1&text-data=01/01/1967&text_inline=1&text_inline2=1&textarea=../..//../..//../..//../..//../..//../..//../..//../..//windows/win.ini HTTP/1.1" 403 703 46.101.158.114 "https://default-theme.textpattern.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.22
28.0 Safari/537.21" "-" "1.27"
5qgQtq - - [13/Jul/2020:10:19:23 +0000] "GET /articles/welcome-to-your-site?color=%23cc0000&date=01/01/1967&datetime-local=01/01/1967&email=sample%40email.tst&file_upload=&month=7&number=1&password=g00dPa%24%24w0rD&range=1&search=&select_dd=1&select_dd2=2&select_multi=1&tel=555-666-0606&test_checkbox_1=on&test_checkbox_2=on&test_checkbox_3=
on&test_checkbox_4=on&test_radio_group=&text=1&text-data=01/01/1967&text_inline=1&text_inline2=1&textarea=../.../.././../.../.././../.../.././../.../.././../.../.././../.../.././windows/win.ini HTTP/1.1" 403 530 46.101.158.114 "https://default-theme.textpattern.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko
) Chrome/41.0.2228.0 Safari/537.21" "-" "1.22"

Initial mitigations have been put into place, and some more will be implemented soon. I won’t go into detail right now, but I’ll tell you more when it’s done.

Last edited by gaekwad (2020-07-13 11:04:30)

colak · 2020-07-13 10:26:51

Hi Pete,

I can connect to all txp sites from here.

gaekwad · 2020-07-13 10:36:57

colak wrote #324473:

I can connect to all txp sites from here.

Thanks, Yiannis – the sites didn’t go offline, they were just veeeery sloooow for a short period. I’ve built the scaffolding around Nginx and toughened it up so we shouldn’t have any noticeable effects beyond things being slow when this thing happens, which is very rare thankfully.

My email inbox, however, is a bomb site with many alert emails being triggered!

Bloke · 2020-07-13 11:13:23

gaekwad wrote #324472:

GET /articles/welcome-to-your-site?color=%23cc0000&date=01/01/1967&datetime-local=01/01/1967&email=sample%40email.tst&file_upload=&month=7&number=1&password=g00dPa%24%24w0rD&range=1&search=&select_dd=1&select_dd2=2&select_multi=1&tel=555-666-0606&test_checkbox_1=on&test_checkbox_2=on&test_checkbox_3= on&test_checkbox_4=on&test_radio_group=&text=1&text-data=01/01/1967&text_inline=1&text_inline2=1&textarea=../..//../..//../..//../..//../..//../..//../..//../..//windows/win.ini

I can’t even begin to wonder what the intended destination system that exposes this exploit might be. Running on Windows, with a terrible (presumably default) password like that and trying to fill form fields with the contents of system files.

*shakes head* A complete waste of effort.

Thanks for being on this so fast, Pete.

gaekwad · 2020-07-13 11:17:40

Bloke wrote #324477:

I can’t even begin to wonder what the intended destination system that exposes this exploit might be.

I’ve another ~250MB of log files if you want some more bedtime reading!

etc · 2020-07-13 14:53:45

Actually, GitHub was 500-ing this morning. I thought of Pete..:-)

gaekwad · 2020-07-13 15:34:39

etc wrote #324484:

Actually, GitHub was 500-ing this morning. I thought of Pete..:-)

I’m almost 100% sure that particular GitHub site-wide 500 and completely unavailable API was probably not my fault. At least not directly, anyway. This time. Ahem.

etc · 2020-07-14 10:28:15

gaekwad wrote #324485:

I’m almost 100% sure that particular GitHub site-wide 500 and completely unavailable API was probably not my fault. At least not directly, anyway. This time. Ahem.

Must be NotPete virus then.

Algaris · 2020-07-14 15:11:20

What dashboard are you using there Pete?

gaekwad · 2020-07-14 15:36:29

Algaris wrote #324510:

What dashboard are you using there Pete?

Netdata – hands down the best I’ve used so far: github.com/netdata/netdata & www.netdata.cloud

Textpattern CMS

Textpattern CMS support forum

#1 2020-07-13 09:43:01