Broadcom EXTREMELY slow gigabit in one direction

Post questions regarding lower-level issues will here.

Broadcom EXTREMELY slow gigabit in one direction

Postby statikregimen » June 12th, 2015, 7:03 am

Hi all...Never registered here before, as normally I can sort myself out, yet not quite expert enough to be of much help to others. However, this one has be stumped. I've scoured the web, but most people that have a similar issue never seem to reach a solution. One of the NICs (eth1) is OK (or is at least twice as fast), the other (eth0) is only OK in one direction.

Here's the scenario for eth0:

Workstations confirmed 1000Mbps link to a Cicso M20 router (now running DD-WRT as of today, which changed nothing from stock firmware)
Cisco M20 router confirmed 1000mbps link to a Cisco 2924XL w/ 2x gigabit modules installed
Server (Dell Poweredge 2950 II, w/ 2xSAS 1tb drives in RAID 1 on the PERC) connected to Cisco 2924XL w/ confimed gigabit link, running the latest Debian Jessie.

All links confirmed as full duplex/1000Mbps.

I can PULL from the server to my workstations at a hefty 75 MegaBYTES/sec (Guess that boils down to 600 megaBITS/sec?) - howdydoody!! I'll take it, either way! Point being: I think I can rule out any form of hardware issue, but maybe not.

But as I mentioned, PUSHING (or pulling FROM other machines) to the server is a whole different ball game.

I am aware that WRITING to a RAID 1 is slower, but >1000% slower? I don't think so.

Workstations are running Windows 7 Pro, but I have the same issue when pushing from another Linux box (DNS320 NAS).

The best results I've had so far have been using a Windows 7 pro workstation, connecting via Filezilla, setting max simultaneous transfers to 10. Then I end up getting about ~100 megaBITS per second peak, but average still around 50-80Mbps.

Any help is GREATLY appreciated. I still have a crapload of files to transfer. The first round (600gigs) took >4 days because of interruptions. That was coming from the aforementioned NAS, trying tar copy, scp and rsync with all manner of options.

Hopefully that all makes sense, and I got my push/pulls straight. It's basically the server's downstream that is brutally slow, but it's upstream is good. This has cost me DAYS of transferring files that should have been done in a few hours...

Again: I appreciate ANY insight!!!

EDIT: I should also note that the issue also exists if I take the Cisco 2924 out of the picture (It actually didn't exist when I did the initial push). So my Cisco M20 (consumer grade) router could be the cause. I don't really have a sane way to test that right now. Anyway, I've just tested between two of the workstations - interesting. Got 45MB/s one way and 106MB/s the other (between two SSDs). Still, 45MB/s would be acceptable to the server, but >50% drop depending on which way the file is going?

EDIT2: i forgot to mention the server has 2x Broadcom BCM5708 NICs:

Code: Select all
lspci | grep -i ethernet
05:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet (rev 12)
09:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet (rev 12)


Code: Select all
lsmod | grep bnx
bnx2                   83625  0


Code: Select all
ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:19:b9:bd:5d:46
          inet addr:192.168.2.20  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::219:b9ff:febd:5d46/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:23242375 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5767710 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:33141034827 (30.8 GiB)  TX bytes:522486855 (498.2 MiB)


Code: Select all
ethtool eth0
Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: on
        Supports Wake-on: g
        Wake-on: d
        Link detected: yes


EDIT 3: I'm using a combination of iftop and the reports of various software doing the transfers to average my results... Average push to the server on the eth0 is 50megabits. I've thoroughly tested this with a number of transfers - both many small and single large files.
Last edited by statikregimen on June 13th, 2015, 7:04 am, edited 1 time in total.
statikregimen
 
Posts: 3
Joined: June 12th, 2015, 6:43 am

Re: Broadcom EXTREMELY slow gigabit in one direction

Postby statikregimen » June 13th, 2015, 6:49 am

In the words of Pink Floyd: "Is there anybody out there?"

I've done some more transfers with the 2nd NIC, and getting along OK... Upwards of 450Mb/s (megabits) for a single large file, and around 100-200Mb/s for large numbers (hundreds of thousands) of smaller files en masse. The later is using Filezilla SFTP set to 10 transfers at once w/ compression; the former was a straight samba xfer of a big (50GB) tar. Tomorrow I will try to compare configs between the two NICs and see if anything jumps out at me. At least now I can rule out any layer 1 issues....

Want to ensure there is hardware failure on the other NIC, though, before replacement.... money is tight :(
statikregimen
 
Posts: 3
Joined: June 12th, 2015, 6:43 am


Return to System Issues

Who is online

Users browsing this forum: No registered users and 1 guest

cron

x