I am running a simple PERL web spider at home on a linux computer. When running 20 or more threads I tend to take down my router. This has been something i’ve looked into from time to time over the years.
I use dd-wrt and when the network starts to really slow down there are thousands of connections – with 3/4 of them being UDP to whatever DNS servers I try at the time …
Am I getting rate limited by DNS from doing too many lookups? Is there a way around this? I imagine hosting my own DNS server would be a nightmare but im ok with it - as the records im looking up are not super critical…
submitted by mxtszlpc
[link] [3 comments]
Powered by WPeMatico