New firewall filters

Started by Toruk Makto, August 19, 2013, 11:10:32 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Toruk Makto

 Unfortunately owing to the total inability of the Chinese censorship webcrawlers (aka the "great firewall of China) to follow the robot rules set on the LearnNavi.org domain and their incessant slamming of our resources, I have had to add a deny filter for the IP range of 123.151.148.0/22. There is a distinct possibility that this may deny access to some of our Chinese tsmuk, although I am hoping this IP prefix is just for the crawlers.  If anyone hears of any LN soaia that have been denied, please let me know at [email protected] so I can add some specific allows for them.

Thanks!

Markì

Lì'fyari leNa'vi 'Rrtamì, vay set 'almong a fra'u zera'u ta ngrrpongu
Na'vi Dictionary: http://files.learnnavi.org/dicts/NaviDictionary.pdf

`Eylan Ayfalulukanä

I crossposted this on the Dothraki/Valyrian side as well.

Yawey ngahu!
pamrel si ro [email protected]

Toruk Makto


Lì'fyari leNa'vi 'Rrtamì, vay set 'almong a fra'u zera'u ta ngrrpongu
Na'vi Dictionary: http://files.learnnavi.org/dicts/NaviDictionary.pdf

Toruk Makto

kelutral# grep 123.151.148 httpd-access.log | wc -l
   35613

...in 14 hours. Mostly offloading images.


Lì'fyari leNa'vi 'Rrtamì, vay set 'almong a fra'u zera'u ta ngrrpongu
Na'vi Dictionary: http://files.learnnavi.org/dicts/NaviDictionary.pdf

`Eylan Ayfalulukanä

Quote from: Toruk Makto on August 19, 2013, 03:18:03 PM
kelutral# grep 123.151.148 httpd-access.log | wc -l
   35613

...in 14 hours. Mostly offloading images.


Images? Interesting.

Q: So what did one Unix sysadmin say to another Unix sysadmin who was stressing out over a search issue?
A: Calm down! Get a a grep on it!

;)

Yawey ngahu!
pamrel si ro [email protected]

Tìtstewan

Have this caused the "multiple-guest-looking-on-the-same-boad" on the /who (who is online) part?

-| Na'vi Vocab + Audio | Na'viteri as one HTML file | FAQ | Useful Links for Beginners |-
-| Kem si fu kem rä'ä si, ke lu tìfmi. |-

Taronyu Leleioae

They might be using the external IP to bounce off of too...  :(

Firewall rule chasing.  A never ending, thankless 24/7 battle. 

Karma for keeping up the good fight...

Toruk Makto

Not sure what you mean by the bouncing thing... ?

Lì'fyari leNa'vi 'Rrtamì, vay set 'almong a fra'u zera'u ta ngrrpongu
Na'vi Dictionary: http://files.learnnavi.org/dicts/NaviDictionary.pdf

Taronyu Leleioae

One trick sometimes hackers and other undesireables do to mask their source location, is to actually route their traffic and "bounce" literally off someone's external IP address.  Some firewalls are more effective than others in stopping this.  I had this problem with one of my sites, from a group located in Canada.  It would eat up my bandwidth.  So blocking the problem IP was a helpful step in reducing this issue.  But you had to sit there analyzing live traffic packets on the external nic to actually catch this.  At first I didn't think this was possible, but we finally figured it out, and blocked certain ranges at the Cisco router and also at our firewall behind it.  This way, not only could they not use our IP for masking their outbound, but any tricks they used, would not let them receive inbound to their IP address range either (from our systems).  IE... blocked the range both incoming and outgoing.  Outgoing, by default, tends to be wide open (all ports) on many firewalls.

Toruk Makto

Well, you can't really "bounce" off of an interface unless there is a comprised service or open proxy running on that interface. We're buttoned up pretty tight, so that is not a concern. :)

Lì'fyari leNa'vi 'Rrtamì, vay set 'almong a fra'u zera'u ta ngrrpongu
Na'vi Dictionary: http://files.learnnavi.org/dicts/NaviDictionary.pdf

Taronyu Leleioae

They managed it using some of our various ports as we had nics running multiple IP's.  I finally went and had WindStream (formerly Paetec) ISP block the range upsteam, in addition to our rule changing.  That ended it as far as I could tell.

Vawmataw

Quote from: Taronyu Leleioae on August 19, 2013, 04:29:55 PM
Karma for keeping up the good fight...
Of course.

Quote from: Toruk Makto on August 19, 2013, 11:10:32 AM
Unfortunately owing to the total inability of the Chinese censorship webcrawlers (aka the "great firewall of China) to follow the robot rules set on the LearnNavi.org domain and their incessant slamming of our resources, I have had to add a deny filter for the IP range of 123.151.148.0/22. T
Good admin lvl: Over 10100100
Fmawn Ta 'Rrta - News IN NA'VI ONLY (Discord)
Traducteur francophone de Kelutral.org, dict-navi et Reykunyu

Palulukan Maktoyu

Quote from: `Eylan Ayfalulukanä on August 19, 2013, 03:43:36 PM
Quote from: Toruk Makto on August 19, 2013, 03:18:03 PM
kelutral# grep 123.151.148 httpd-access.log | wc -l
   35613

...in 14 hours. Mostly offloading images.


Images? Interesting.

Q: So what did one Unix sysadmin say to another Unix sysadmin who was stressing out over a search issue?
A: Calm down! Get a a grep on it!

;)

*Keytsyok*
Fkol syaw oeru Palulukan Maktoyu Ta'lengean

Twitter: https://twitter.com/navi_wotd

Toruk Makto

Update:
I have moved the filtering to htaccess based on client type and narrowed the denied specific IP ranges. This should allow our Chinese soaia access while restricting the swarming, ill-behaved web spiders that the .cn ISPs and government seem to be addicted to.

Cheers!

Markì

Lì'fyari leNa'vi 'Rrtamì, vay set 'almong a fra'u zera'u ta ngrrpongu
Na'vi Dictionary: http://files.learnnavi.org/dicts/NaviDictionary.pdf

Kemaweyan

#14
I also can't access the forum, though my IP is 85.90.193.xxx. My ISP is using NAT, so I think this IP could be in spam lists. Now I'm using Tor to write this message.

Upd:

But learnnavi.org is available :-\ Only forum does not work.
Nìrangal frapo tsirvun pivlltxe nìNa'vi :D

Toruk Makto

Now that we have ample bandwidth, I have removed the agent filters from all sites. Nothing on this end is blocking your ip address.

Lì'fyari leNa'vi 'Rrtamì, vay set 'almong a fra'u zera'u ta ngrrpongu
Na'vi Dictionary: http://files.learnnavi.org/dicts/NaviDictionary.pdf

Kemaweyan

Hmm.. anyway I can't access the forum. Only via proxy, Tor, Opera turbo or anything else that changes IP.
Nìrangal frapo tsirvun pivlltxe nìNa'vi :D

Tìtstewan

Has your ISP forced disconnection? If yes, you could try to disable your router (unplug cable) for some seconds. Usually you will get a new IP.

-| Na'vi Vocab + Audio | Na'viteri as one HTML file | FAQ | Useful Links for Beginners |-
-| Kem si fu kem rä'ä si, ke lu tìfmi. |-

Toruk Makto

#18
Quote from: Kemaweyan on September 10, 2013, 10:20:21 AM
Hmm.. anyway I can't access the forum. Only via proxy, Tor, Opera turbo or anything else that changes IP.

Kemaweyan, when did this start?  I can't find anything that would be blocking your IP just on the forums. Are you getting an error message of some kind, or is the forum just completely unresponsive?

EDIT: I am not finding 85.90.193.* in any DNSBLs.

EDIT AGAIN: I looked at your info in mysql and there was an oddment in the IP info for your account. That may have been causing SMF to think you are trying to hack the forums. I have manually poked a likely-looking IP in the record to see if this fixes the problem.

Lì'fyari leNa'vi 'Rrtamì, vay set 'almong a fra'u zera'u ta ngrrpongu
Na'vi Dictionary: http://files.learnnavi.org/dicts/NaviDictionary.pdf

Irtaviš Ačankif

#19
Very unfortunately, based on my tests using a server I own in China, the entire Cloudflare IP block was null-routed from China starting Sep 7. All sites hosted by Cloudflare, including Cloudflare itself, are blocked by IP in China  :'(

You might look into DNSPod to turn off Cloudflare for Chinese users by serving different DNS replies based on the resolver address. I could be of help if you can't navigate the Chinese website. It is, in fact, possible to use Cloudflare without their DNS by manually coding Cloudflare A responses and this does work for my websites.

Edit: Apparently not those starting with 141.*.*.*, only the 108.*.*.* ones, so LearnNa'vi is fine, though half of the time it would take a logn time to load as 108.*.*.* times out first.

Also, I recommend enabling HTTPS. I see learnnavi already supports partial HTTPS for the dynamic content. I think a switch in SMF and one in Cloudflare will do the trick. Chinese firewall actually makes HTTPS much faster as the firewall gives up scanning each packet for keywords using slow regexes.
Previously Ithisa Kīranem, Uniltìrantokx te Skxawng.

Name from my Sakaš conlang, from Sakasul Ältäbisäl Acarankïp

"First name" is Ačankif, not Eltabiš! In Na'vi, Atsankip.