1. OCAU Merchandise is available! Check out our 20th Anniversary Mugs, Classic Logo Shirts and much more! Discussion in this thread.
    Dismiss Notice

When is consumer 10GbE going to happen?

Discussion in 'Networking, Telephony & Internet' started by Smokin Whale, Sep 21, 2014.

  1. frankgobbo

    frankgobbo Member

    Joined:
    Sep 23, 2008
    Messages:
    148
    Yeah my Sabrent 5Gb USB adapters top out at approx 3.5, whether via my 10Gb switch (connected @5Gb) or point-to-point (USB adapter-USB adapter on different box). It's still around 750-1000mb higher than 2.5Gb, but you could argue that's not worth 3x the price of a 2.5Gb USB adapter. I think it is.

    They definitely get warm though. They've been ..... mostly reliable. I do, on a particular PC, have it basically strangle the bus and kill the box eventually, but I can also copy a few TB over the link before it happens so .. meh. It's just a download PC so a reboot isn't a big deal. The ones connected to a pair of Synologys have worked pretty well though after I compiled my own driver for them.
     
  2. Phido

    Phido Member

    Joined:
    Jun 20, 2003
    Messages:
    7,655
    Location:
    Dark City
    I played around with some 5GB USB adapters, I was a bit unimpressed. Its not really ready for it. You get basically 2.5Gb speeds, sort of. But with more headaches. If you have absolutely no other option, it may be useful say on a laptop for a short period of connection. Maybe its the usb 3.1 on the board that is limiting.

    There is a way to connect a 5 or 10GB ethernet card via NMVe, but its a lot of stuffing around, and you give up one of your NVME drives. Given the motherboards come with 2.5Gb built in, that will be sufficient for now. I would rather the 2400Mb/s from a local NVME. Plus when I do get a new board which will come with at least 5Gbe built in, you basically throw all that out.

    The NAS will have the full 10GbE so it will be able to easily handle multiple requests. If and when I get a new mobo with 5 or 10GbE, just slot it in and won't have to adjust any of the networking hardware. My current NAS isn't particularly fast, based around TrueNAS and probably tops out at single spinning disk speeds, so 300Mb/s will be fine. I might add some SSD storage, but it will be over SATA anyway.

    5 multispeed ports isn't a whole lot. But given everything is currently connected by Wifi, and its sucks, it should be a massive improvement. Currently over wifi I get ~30Mb/s.. So, it really does suck balls.
     
    wulfy23 likes this.
  3. Blinky

    Blinky Member

    Joined:
    Jul 4, 2001
    Messages:
    3,961
    Location:
    Brisbane
    It might have been the disk i/o unless you were on nvme.

    I have tried to get one of my Intel NUCs to do that^ but it failed on the adapter card I got (I think it was the card).
    https://jackharvest.com/index.php/2...-without-thunderbolt-3-ngff-to-pcie-x4-riser/
     
  4. Aetherone

    Aetherone Member

    Joined:
    Jan 15, 2002
    Messages:
    9,011
    Location:
    Adelaide, SA
    I've got 'em at both ends and know for small sequential transfers of around ~50gb they'll flood 10g. Plus iperf so i reckon I can do some damage :)
    I had a look at doing this with my older ITX rig with direct cabled M.2-> PCIex4 adapters. Didn't get out of the planning stage when I realised that by going for a slightly cheaper 2012era ITX board that the 4 spare PCIe lanes go absofreakinglutly nowhere. 16x VGA slut is it. :upset:
     
  5. Blinky

    Blinky Member

    Joined:
    Jul 4, 2001
    Messages:
    3,961
    Location:
    Brisbane
    Maybe turn it up to -v to get slightly more verbosity. Got nvme or your new 5Gb nics at both ends?
     
  6. Phido

    Phido Member

    Joined:
    Jun 20, 2003
    Messages:
    7,655
    Location:
    Dark City
    Could be lots of things. I originally wanted to use them on my nas but driver support was iffy. Could have set them up wrong. USB for NAS ethernet, feels, dodgy..

    For my ITX workstation I would rather 2Tb of fast NVME storage than 10GB over the existing 2.5GB. Maybe thats just me and my workflow. I am surprised people don't make ITX boards with 10GbE or at least 5 GbE.. ITX is now popular for workstations and NAS/Servers. I have 16 cores, 64Gb Ram, 3TB of NVME storage. why only 2.5GbE?

    While there are other 10GbE solutions I like the TP link because it can manage my mess of 2.5Gb connections and 10GB. Which solves a lot of my driver issues in NAS land trying to get 2.5GbE working there..
     
    wulfy23 likes this.
  7. Aetherone

    Aetherone Member

    Joined:
    Jan 15, 2002
    Messages:
    9,011
    Location:
    Adelaide, SA
    Ah ok, -v
    Big switch with FS.com 10GbE transceivers plus 10g optics. Server with 10g optics and NVMe, client with USB 3.1 and QNA-UC5G1T and another NVMe in there that I know will flood R or W if I use it with 10g optics.
    I did find a review where they were getting ~3.5 in iperf and 4-andabit on NVMe file copy. Keen to see if I can replicate.

    USB ethernet feels dodgy period. Still, needs must.

    When I last went looking, some top end ITX boards had faster than 1g ethernet. More common in the Supermicro server space.

    The bit that makes me feel really silly now is the NUC I've been eyeing off as the intended destination of the QNA-UC5G1T has been revised ... with Realtek 2.5g onboard now.
     
  8. Phido

    Phido Member

    Joined:
    Jun 20, 2003
    Messages:
    7,655
    Location:
    Dark City
    Specific server boards.. Typically, very expensive, also generally ill suited for workstation type stuff.

    2.5GbE is not great but still a step up from 1GbE.

    I am hoping the next generation of DDR5 Mitx board have at least 5 GbE.
     
  9. Aetherone

    Aetherone Member

    Joined:
    Jan 15, 2002
    Messages:
    9,011
    Location:
    Adelaide, SA
    Yeah, there's a fair old pants kicking compromise needed without thunderbolt and an expensive hot external box.

    1GbE is so utterly outmoded and archaic these days that everything - including spinning rust - has left it behind. 2.5GbE is entirely & fully usable without anything special. How much do you reckon a gamer would pay for 250% more FPS?
     
  10. Aetherone

    Aetherone Member

    Joined:
    Jan 15, 2002
    Messages:
    9,011
    Location:
    Adelaide, SA
    When is consumer 10GbE going to happen?

    Fucking never if Marvell/Aquantia keep on with this bullshit. Can't say I'm all that impressed with this QNAP QNA-UC5G1T. After the Asus (aquantia chipset) crap and now the QNAP (also aquantia chipset) being weird I'm royally sick of this shit. Intel and Mellanox were able to build this in a cave! With a box of scraps! 10 years ago!

     
    wulfy23 likes this.
  11. phreeky82

    phreeky82 Member

    Joined:
    Dec 10, 2002
    Messages:
    9,816
    Location:
    Qld
    For me (actual current systems, multiple of both):
    - AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion]
    - MT27500 Family [ConnectX-3]

    iperf between them (via an fs.com switch, but also works perfectly well directly between them, using an fs.com SFP):
    - [ 4] 0.0000-10.0155 sec 10.6 GBytes 9.13 Gbits/sec

    It's the pricing that gets me still, the quality of the products has been fine in my experience.

    Are you just having Windows driver issues?
     
  12. Aetherone

    Aetherone Member

    Joined:
    Jan 15, 2002
    Messages:
    9,011
    Location:
    Adelaide, SA
    Absolutely ... or firmware. Stuffed if I can tell.
    AQC100 SFP+ Asus card. Can slam 10g no worries - it actually slightly faster than the Mellanox - but stream that same UHD blurary file over the network results in hitches, hiccups, pauses, wtf. 10g no worries. 50mbps reliably, nope. It's a bloody drag car, useless on open roads.

    AQC111U 5g QNAP dongle. 400Mbps windows copy up, 186mbps down. Same on iperf. WTF?

    The hardest part of 10 year old Intel/MLX is the wait from China. Chuck 'em in, load newest driver, quick FW update (MLX wins this!) and boom ... just freaking works. Does everything, does it well.
     
    Last edited: Nov 23, 2021
  13. Aetherone

    Aetherone Member

    Joined:
    Jan 15, 2002
    Messages:
    9,011
    Location:
    Adelaide, SA
    Wednesday, 24th: more bloody fiddling and shenanigans

    Perhaps I've been a bit hasty in my judgement of the QNAP dongle. Testing an onboard 2.5g Realtek with my FS.com multi-gig transceivers and seeing the same phenomena - basically full rate upload and awful rate download.
    Have now tried two different switches & four firmwares - MKT CRS317 on SwOS and 2x versions of RouterOS and a QNAP QSW-M408S. Some combinations work better than others, but the problem persists amongst them all.
    However, if I plug the multi-gig cards into a 1g port, performance is bang on exactly 1g in both directions.

    This has me wondering - is the problem the switch itself not know what multigig is? The transceivers report themselves as 10g. So uploading from 5to10 has no bottleneck, but downloading from 10to5 does. Both switches have no troubles doing 10g-10g at full rate.

    Anyone else running a similar config that can try it out? I'm going to try and find the time tonight to test the transceivers with old 1g ports

    Edit: time found. 1g Intel port to FS multigig to MKT-CRS317 switch to 10g to server: 113MBps upload, 56MBps download.

    yeah, something aint right
     
    Last edited: Nov 24, 2021
  14. Blinky

    Blinky Member

    Joined:
    Jul 4, 2001
    Messages:
    3,961
    Location:
    Brisbane
    I’ll run it up for you when I get back to Brissy for you. There is a ASUS X- whatever aquantia 10Gb card someone.
     
  15. Aetherone

    Aetherone Member

    Joined:
    Jan 15, 2002
    Messages:
    9,011
    Location:
    Adelaide, SA
    Looks like the switches are having issues with the multigig adaptors (or the adapters? dunno). Got that particular combo? Or better yet, got a spare multigig sfp you'd lend me for a week? :D
     
  16. Blinky

    Blinky Member

    Joined:
    Jul 4, 2001
    Messages:
    3,961
    Location:
    Brisbane
  17. Aetherone

    Aetherone Member

    Joined:
    Jan 15, 2002
    Messages:
    9,011
    Location:
    Adelaide, SA
    It should be pretty simple to copy my setup - 10g link switch to server, multiG link switch to client via whatever, iperf both ways.
    Mine looks like this
    taskmanager.jpg
    Note the massive difference between send and receive. This piccy is IPERF with the 5G USB, 2.5G and 1G are smaller but pretty much the same - full bandwidth upload, half bandwidth download.
     
  18. phreeky82

    phreeky82 Member

    Joined:
    Dec 10, 2002
    Messages:
    9,816
    Location:
    Qld
    Boot both boxes to a live Linux distro and run iperf via that

    Slow only in one direction could obv mean some physical things but I assume you've swapped that stuff around to test. But the other thing to check is send/receive offload settings are typically separate. Also do the systems have both ends have enough grunt? Because if it can't offload, then the CPU can actually get worked quite hard.
     
  19. Blinky

    Blinky Member

    Joined:
    Jul 4, 2001
    Messages:
    3,961
    Location:
    Brisbane
    OK I’ll try but I’ve never had that personally. Linux or Windows?

    You’ll have to wait until I finish up the coast, that’s all.

    Edit: Booting to a live common Linux OS is a good idea.
     
    Last edited: Nov 24, 2021
  20. Aetherone

    Aetherone Member

    Joined:
    Jan 15, 2002
    Messages:
    9,011
    Location:
    Adelaide, SA
    Found this thread here on ServeTheHome. Quite a lot of it sounds very familiar - and they're making the same conclusion I am - the switch thinks the link is 10g when it not. 2 inch pipe and 3 inch arsehole and all that and the fs.com trannies only advertise 10g connection to the MKT switch.

    Apparently the MKT units will advertise the lower speeds so the switch has a clue. Anyone got a spare Mikrotik S+RJ10 they'd wanna lend me?
    Although for the price of a pair of them, the QNAP QSW-2104-2S switch starts to look like a solid option - 10g SFP+ to uplink and 4x 2.5g ports for clients.

    Well if I plug in the Mellanox 10g's, they'll blast files in both directions at 10g without breaking a sweat. Hell, I can iperf 18.8g full duplex (and more if I care enough to run sufficient IPERF process threads that Mhz stops mattering). So yeah, I have enough grunt for 2.5 and 5.

    It's also worth noting it's linked to the trannies - 1g, 2.5 and 5g all have the same issue in all my switches. Plugging into the 1g segments fixes it.

    EDIT: the plot thickens. If I plug the FS trannies into my Asus 8+2 switch so they're running at 10g like they say they are and client into the 1g segment, problem is mostly resolved. I say mostly, but I'll take 105MBps over 55 any day.

    EDIT No2: reading the STH review of the S+RJ10 and in the comments I noticed this
     
    Last edited: Nov 24, 2021

Share This Page

Advertisement: