Also... From both machines, I can scp to an external machine. These 2 boxes I'm having problems with are on opposite sides of an intracompany vpn. Could I be having problems with the vpn? The VPN is a big-expensive end to end hardware solution that I don't get to play with. If it were that, any ideas on how I could prove that to corporate IT? Please note... it's possible that the vpn has nothing to do with the problem. -Marc
Quoting Karl Hiramoto <karl@zoop.org>:
From both machines can you scp into them? are they having problems receiving large amounts of data? can you scp or ftp data into them?
Check for any open large files. Logs getting full? Possible you're reaching your open files limit. Recently i had a squid proxy machine stop working. There was plenty of free disk space and inodes. but the kernel had a limit on how many open files there could be. I deleted a 2G log file i hadn't setup with logrotate and the problem was fixed.
what does ulimit -a output? cat /proc/sys/fs/file-max try adding: "ulmit -n 4095" at the begining your rsync script.
try using the "find" command to find files larger than 1G see if they are causing problems. anything at the 2G limit could be an issue. depending on what filesystem you are using.
On Thu, 28 Aug 2003 mhughes@wpidalamar.com wrote:
Also... From both machines, I can scp to an external machine. These 2 boxes I'm having problems with are on opposite sides of an intracompany vpn. Could I be having problems with the vpn? The VPN is a big-expensive
end to end hardware solution that I don't get to play with. If it were that, any ideas on how I could prove that to corporate IT?
Please note... it's possible that the vpn has nothing to do with the problem.
-Marc
_______________________________________________ Wlug mailing list Wlug@mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug
--
¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø Karl Hiramoto <karl@hiramoto.org> Work: 978-425-2090 ext 25 Cell: 508-517-4819 http://karl.hiramoto.org/ AOL IM ID = KarlH420 Yahoo_IM = karl_hiramoto ¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø
QOTD: "I used to jog, but the ice kept bouncing out of my glass."
_______________________________________________ Wlug mailing list Wlug@mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug
The file-max param is for the max number of file descriptors not file size, Further more deleting large files would not solve either problem (Unless your fs was actually full in which case you would have another set of problems) FD's are taken then freed all the time (open/close files open/close sockets etc..) which is why fd problems are intermittent and hard to track.I suspect your squid problem would be better solved by increasing you file-max param or editing limits.h next time you build you kernel. On server systems file-max should be set to (RAM-SIZE/4)*256 256 fd per 4 megs of ram. This is not a hard number but a good rule of thumb. To set this every time you boot edit /etc/sysctl.conf (on most systems) or add it to rc.local. It is possible you are running out of file descriptors. Running out of fd's shows up in strange ways! Did you bounce ssh on both sides of the tunnel. More importantly on the receiving end ;). Try running ssh/scp with the -v flag set, it may give you some good info. Run tcpdump .. are you dropping a ton of packets? Do a simple ping to see if there is packet loss? Do a ifconfig are there a ton of errs on the interface you are leaving on/ receiving from? If you see you are dropping a lot of packets but there are no interface err's you can encourage the person that admins the VPN to check it out! Matt
On Thu, 28 Aug 2003, Matt Higgins wrote:
It is possible you are running out of file descriptors. Running out of fd's shows up in strange ways! Did you bounce ssh on both sides of the tunnel. More importantly on the receiving end ;). Try running ssh/scp with the -v flag set, it may give you some good info. Run tcpdump .. are you dropping a ton of packets? Do a simple ping to see if there is packet loss? Do a ifconfig are there a ton of errs on the interface you are leaving on/ receiving from?
If you see you are dropping a lot of packets but there are no interface err's you can encourage the person that admins the VPN to check it out!
Matt
_______________________________________________ Wlug mailing list Wlug@mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug
I tried bouncing ssh at both ends... even doing a kill of all the processes of both ssh and sshd at both ends. scp -v doesn't show any weirdness. Ping works well, there's not an abnormal amount of errors in ifconfig. Is the open file limit system wide, or user specific? Mine's set for 4096, I've tried killing off a bunch of rarely used services on both machines to get the file count down. Now... this morning I'm noticing I can scp from the remote machine to the local machine, but not the other way around. In other words, from the remote machine, this works: scp qaddonline hughesm@10.1.0.19: But copying it back doesn't. scp hughesm@10.1.0.19:qaddonline . (qaddonline is about a 2 meg file) Also... from the local machine, iI can pull things down scp hughesm@10.21.2.184:qaddonline . but not push things up scp qaddonline hughesm@10.21.2.184: This didn't work last night. I can also now send small files (I tried one less than 1k) fine. I'd also like to add there's 2 machines in the remote facility that both have identical behavior. So I'm thinking it's either the link or my local machine. I'm no tcpdump expert, and both these machines have a good deal of traffic. What's an easy way to see if packets are getting dropped? (yeah, I suck at tcpdump filters). Any other ideas on things to look for? -Marc
Is there anything showing up in /var/log/messages ? On Fri, 29 Aug 2003 mhughes@wpidalamar.com wrote:
On Thu, 28 Aug 2003, Matt Higgins wrote:
It is possible you are running out of file descriptors. Running out of fd's shows up in strange ways! Did you bounce ssh on both sides of the tunnel. More importantly on the receiving end ;). Try running ssh/scp with the -v flag set, it may give you some good info. Run tcpdump .. are you dropping a ton of packets? Do a simple ping to see if there is packet loss? Do a ifconfig are there a ton of errs on the interface you are leaving on/ receiving from?
If you see you are dropping a lot of packets but there are no interface err's you can encourage the person that admins the VPN to check it out!
Matt
_______________________________________________ Wlug mailing list Wlug@mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug
I tried bouncing ssh at both ends... even doing a kill of all the processes of both ssh and sshd at both ends.
scp -v doesn't show any weirdness.
Ping works well, there's not an abnormal amount of errors in ifconfig.
Is the open file limit system wide, or user specific? Mine's set for 4096, I've tried killing off a bunch of rarely used services on both machines to get the file count down.
Now... this morning I'm noticing I can scp from the remote machine to the local machine, but not the other way around. In other words, from the remote machine, this works: scp qaddonline hughesm@10.1.0.19: But copying it back doesn't. scp hughesm@10.1.0.19:qaddonline .
(qaddonline is about a 2 meg file)
Also... from the local machine, iI can pull things down scp hughesm@10.21.2.184:qaddonline . but not push things up scp qaddonline hughesm@10.21.2.184:
This didn't work last night.
I can also now send small files (I tried one less than 1k) fine.
I'd also like to add there's 2 machines in the remote facility that both have identical behavior. So I'm thinking it's either the link or my local machine.
I'm no tcpdump expert, and both these machines have a good deal of traffic. What's an easy way to see if packets are getting dropped? (yeah, I suck at tcpdump filters).
Any other ideas on things to look for? -Marc
_______________________________________________ Wlug mailing list Wlug@mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug
-- ¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø Karl Hiramoto <karl@hiramoto.org> Work: 978-425-2090 ext 25 Cell: 508-517-4819 http://karl.hiramoto.org/ AOL IM ID = KarlH420 Yahoo_IM = karl_hiramoto ¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø To iterate is human, to recurse, divine. -- Robert Heller
what is the output of: cat /proc/sys/fs/file-nr the format is documented in <linux-source>/Documentation/filesystems/proc.txt On Fri, 29 Aug 2003, Karl Hiramoto wrote:
Is there anything showing up in /var/log/messages ?
On Fri, 29 Aug 2003 mhughes@wpidalamar.com wrote:
On Thu, 28 Aug 2003, Matt Higgins wrote:
It is possible you are running out of file descriptors. Running out of fd's shows up in strange ways! Did you bounce ssh on both sides of the tunnel. More importantly on the receiving end ;). Try running ssh/scp with the -v flag set, it may give you some good info. Run tcpdump .. are you dropping a ton of packets? Do a simple ping to see if there is packet loss? Do a ifconfig are there a ton of errs on the interface you are leaving on/ receiving from?
If you see you are dropping a lot of packets but there are no interface err's you can encourage the person that admins the VPN to check it out!
Matt
_______________________________________________ Wlug mailing list Wlug@mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug
I tried bouncing ssh at both ends... even doing a kill of all the processes of both ssh and sshd at both ends.
scp -v doesn't show any weirdness.
Ping works well, there's not an abnormal amount of errors in ifconfig.
Is the open file limit system wide, or user specific? Mine's set for 4096, I've tried killing off a bunch of rarely used services on both machines to get the file count down.
Now... this morning I'm noticing I can scp from the remote machine to the local machine, but not the other way around. In other words, from the remote machine, this works: scp qaddonline hughesm@10.1.0.19: But copying it back doesn't. scp hughesm@10.1.0.19:qaddonline .
(qaddonline is about a 2 meg file)
Also... from the local machine, iI can pull things down scp hughesm@10.21.2.184:qaddonline . but not push things up scp qaddonline hughesm@10.21.2.184:
This didn't work last night.
I can also now send small files (I tried one less than 1k) fine.
I'd also like to add there's 2 machines in the remote facility that both have identical behavior. So I'm thinking it's either the link or my local machine.
I'm no tcpdump expert, and both these machines have a good deal of traffic. What's an easy way to see if packets are getting dropped? (yeah, I suck at tcpdump filters).
Any other ideas on things to look for? -Marc
_______________________________________________ Wlug mailing list Wlug@mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug
-- ¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø Karl Hiramoto <karl@hiramoto.org> Work: 978-425-2090 ext 25 Cell: 508-517-4819 http://karl.hiramoto.org/ AOL IM ID = KarlH420 Yahoo_IM = karl_hiramoto ¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø To iterate is human, to recurse, divine. -- Robert Heller
nothing out of the ordinary. I'll get a disconnect message when I kill the scp process... but if I don't kill it, it just sits around for at least 1/2 hour. On Fri, 29 Aug 2003, Karl Hiramoto wrote:
Is there anything showing up in /var/log/messages ?
On Fri, 29 Aug 2003 mhughes@wpidalamar.com wrote:
On Thu, 28 Aug 2003, Matt Higgins wrote:
It is possible you are running out of file descriptors. Running out of fd's shows up in strange ways! Did you bounce ssh on both sides of the tunnel. More importantly on the receiving end ;). Try running ssh/scp with the -v flag set, it may give you some good info. Run tcpdump .. are you dropping a ton of packets? Do a simple ping to see if there is packet loss? Do a ifconfig are there a ton of errs on the interface you are leaving on/ receiving from?
If you see you are dropping a lot of packets but there are no interface err's you can encourage the person that admins the VPN to check it out!
Matt
_______________________________________________ Wlug mailing list Wlug@mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug
I tried bouncing ssh at both ends... even doing a kill of all the processes of both ssh and sshd at both ends.
scp -v doesn't show any weirdness.
Ping works well, there's not an abnormal amount of errors in ifconfig.
Is the open file limit system wide, or user specific? Mine's set for 4096, I've tried killing off a bunch of rarely used services on both machines to get the file count down.
Now... this morning I'm noticing I can scp from the remote machine to the local machine, but not the other way around. In other words, from the remote machine, this works: scp qaddonline hughesm@10.1.0.19: But copying it back doesn't. scp hughesm@10.1.0.19:qaddonline .
(qaddonline is about a 2 meg file)
Also... from the local machine, iI can pull things down scp hughesm@10.21.2.184:qaddonline . but not push things up scp qaddonline hughesm@10.21.2.184:
This didn't work last night.
I can also now send small files (I tried one less than 1k) fine.
I'd also like to add there's 2 machines in the remote facility that both have identical behavior. So I'm thinking it's either the link or my local machine.
I'm no tcpdump expert, and both these machines have a good deal of traffic. What's an easy way to see if packets are getting dropped? (yeah, I suck at tcpdump filters).
Any other ideas on things to look for? -Marc
_______________________________________________ Wlug mailing list Wlug@mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug
--
¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø Karl Hiramoto <karl@hiramoto.org> Work: 978-425-2090 ext 25 Cell: 508-517-4819 http://karl.hiramoto.org/ AOL IM ID = KarlH420 Yahoo_IM = karl_hiramoto ¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø,¸¸,ø¤º°`°º¤ø
To iterate is human, to recurse, divine. -- Robert Heller
_______________________________________________ Wlug mailing list Wlug@mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug
participants (3)
-
Karl Hiramoto -
Matt Higgins -
mhughes＠wpidalamar.com