Network Tests: Setting Our Expectations
Before we do anything else, we should test our hard drives without using the network to see what kind of bandwidth we can expect from them in an ideal scenario.
There are two PCs taking part in our real-world gigabit network. The first, which we’ll call the server, has two drives. Its primary drive is a 320 GB Seagate Barracuda ST3320620AS that’s a couple of years old. The server acts as a network-attached storage (NAS) device for a RAID array consisting of two 1 TB Hitachi Deskstar 0A-38016 drives, mirrored for redundancy.
The second PC on the network, which we’ll call the client, has only one drive: a 500 GB Western Digital Caviar 00AAJS-00YFA that is about six months old.
We first test the speed of both the server's and client computer‘s C: drives and see what kind of read performance we can expect to get from them. We’ll use SiSoftware Sandra 2009’s hard disk benchmark:
Right out of the gate, our hopes for achieving gigabit speed file transfers have disappeared. Both of our single hard disks have a maximum read speed of about 75 MB/s in ideal situations. Since this is a real-world test and the drives are about 60% full, we can expect read speeds closer to the 65 MB/s index that both of these drives share.
But have a look at the RAID 1 array performance–the great thing about this RAID 1 array is that the hardware RAID controller can increase read performance by pulling data from both drives simultaneously, similar to how a striped RAID 0 array can; note that this effect will probably only occur with hardware RAID controllers, not software RAID solutions. In our tests, the RAID array gives us much faster read performance than a single drive can, so our best chance for a quick file transfer over the network looks like it’s with our RAID 1 array. While the RAID array is delivering an impressive peak 108 MB/s, we should be seeing real-world performance close to its 88 MB/s index because this array is about 55% full.
So, we should be able to demonstrate about 88 MB/s over our gigabit network, right? It’s not that close to a gigabit network’s 125 MB/s ceiling, but it’s a lot faster than 100 megabit networks that have a 12.5 MB/s ceiling, so 88 MB/s would be great in practice.
Not so fast. Just because our drives can read this quickly doesn’t mean they can write that fast in a real-world situation. Let’s try a few disk write tests before even using the network to see what happens. We’ll start with our server and copy a 4.3 GB image file from the speedy RAID array to the 320 GB system drive, and then back again. Then we'll try copying a file from the client PC's D: drive to its C: drive.
Yuck! Copying from the quick RAID array to the C: drive results in a mere 41 MB/s average transfer speed. And when copying from the C: drive to our RAID 1 array, the transfer speed drops even more to about 25 MB/s. What is going on here?
Well, this is what happens in the real world: The C: drive is only a little over a year old, but it’s about 60% full, probably fragmented a bit, and it’s no speed demon when writing. There are other factors as well, such as how fast the system and memory are in general. As for the RAID 1 array, it’s made up of relatively new hardware, but because it’s a redundant array, it has to write data to two drives simultaneously and is going to take a huge write performance hit. While RAID 1 can offer fast read performance, write performance is sacrificed. Alternatively, we could use a RAID 0 array with striped drives that delivers fast write and read performance, but if one of the drives die, then all of the data is compromised. Realistically, RAID 1 is a better bet for folks who value their data enough to set up a NAS.
However, all is not lost yet, as we have a little light at the end of the tunnel. The newer 500 GB Western Digital Caviar is capable of writing this file at an average 70.3 MB/s over five runs, and even recorded a top-speed run of 73.2 MB/s performance.
With this in mind, we’re going to expect that our real-world gigabit LAN might demonstrate a maximum of about 73 MB/s on transfers from the NAS RAID 1 array to the client’s C: drive. We’ll also test transfer files from the client’s C: drive to the server’s C: drive to see if we can realistically expect about 40 MB/s in that direction.
- 1 / 2
- Next
-
It would have been interesting to see the network speeds with a decent SSD on either end.
The "limiting" factor at 111MB/s is the Ethernet overhead....... The actual data needs to be framed and packed and that requires additional bits to be transferred so an overhead of (125-111)/125 = 11% seems reasonable.
The "limiting" factor at 111MB/s is the Ethernet overhead....... The actual data needs to be framed and packed and that requires additional bits to be transferred so an overhead of (125-111)/125 = 11% seems reasonable.
I agree with Sir0 that the overheads will bring down the speed.
secondly if you have more than a point to point connection, (ie more than 2 devices) on the network your speeds will suffer.
Even if they are on seperate collision domains, you will still get broadcast trafic that will intrupt conversations.
And lastly the quility of you network cards! there is a reson that one network card will set you back £15 and another cost £150. I have found almost with out exception that a more expensive high quility card will sustain a higher through put than a cheap card.. you dont need to spend £150. but think twice before chosing a £10 card over a £25 one..
Ethernet is a sloppy standard. Typical case of just make everything 'big' and hope for the best.
Now if you had the efficiency of Token Ring with the bandwidth of ethernet....wow!
Funny, I just upgraded my home network last week and the results are phenomonal: I copied a 12GB file from the C: drive (OCZ SSD)of my htpc with Vista RTM to raid array on my server with W2008 R2 RC Datacenter and it started at 105MB/s and dropped slowly to a steady 98MB/s until the end!! Ofcourse, copying the file back in the other direction delivered a 37MB/s (high read but slow write speed on SSD...) In fact, it's so grand to have this speed that I now gathered all my data on the 6.5TB networkdrive and stream or use anything from there. I use a Sitecom 8-port switch and CAT6 cabling if you wanna copy this proven high-performance model
I didn't like the cable tests at all. Many buildings do not use ideal 25-28ft connections. In fact, the last Digital Overload event used cables over 100ft long (they throw them out afterwords, a friend took them). The real test between cable types is when using them BEYOND spec, because it is cheaper to buy a 1000ft. spool of cable rather than getting a repeater box every 20 ft.
My own computer runs off a 50ft. cable to the central hub.
I think the article is concluding something without considering how the networks works. The network does not work like the local PC buses and should probably not be compared with them at all.
First of all, the theoretical maximum transmit rate in one direction is not the same as how fast the file is copied. You have to consider, that a file copy will use TCP-IP protocol on the network. The Transmission Control Protocol splits the whole file to small pieces, usually of approx 1500 bytes long including the frame header, which needs to be acknowledged by the receiver. The sender and receiver operates with a window size, which is the number of unacknowledged packages. The sender sends a number of packages and then waits for an acknowledge signalling, that the receiver has received them all. If not, they need to be retransmitted. Then the sender can send the next portion of the file.
So the actual network card on both sides will be a very limitting factor. In theory you will be able to transfer (!) something between the half and 90% of the maximum rate in one direction.
But that is not the only limitting factor. The packages needs to be processed as well. The PC's will be interrupted for very package they receive and they need to get the data out of the network card in same speed as they arrive and put the data somewhere. For 125MB/sec and 1250-1400 bytes in one package, the PC needs to handle approx 100.000 requests/interrupts in a second from the network card and then probably the next 50.000 from the harddrive. It means 150.000 interrupts more, then required for normal functionality. It requires a lot of processing power from the hardware and from the operating system as well. Let me say, that windows will not be my choice of the operating system to handle this amount of data with very low latency.
The network speed measured in Gigabits is often more a sale trick then an actual information telling anything about the actual transfer speed. It is true, that when using higher speeds you will get the file over faster. However, you will be far from the maximum specification. If you want to utilize the full bandwidth, you need to invest into hardware, which is able to handle it. The majority of the systems, especially the notebooks, are able to send few packages with the right speed, but then they spend more time with waiting then with utilizing the full bandwith.
Another thing is the size of the package. The old ethernet uses 1500 bytes packages. It is a very limiting factor and I hope that in the future the ethernet specifications will be changed to support much larger packages. Today they are trying to use jumbo packages which are packages typ. up to 9kB. It is still to small for an effective data transfer on the network.
Would have been interesting to see if gigabit made any difference to latency/fps on a large LAN game of some kind. I imagine a 64 player BF2 server might be able to use more than 100 megabits/sec. Not an easy bench to set up though
Why not set the drives up as ram drives. This would remove almost all speed bariers from the equation.
Why not set the drives up as ram drives. This would remove almost all speed bariers from the equation.
hum..... he did that...
Why not set the drives up as ram drives. This would remove almost all speed bariers from the equation.
I doubt it, most game servers need very little bandwidth really since not a great deal of information is needed to be transfered.
You got your units wrong
"Each 1000BASE-T network segment can be a maximum length of 100 meters (328 feet), and must utilize "Category 5" cabling at a minimum".
http://en.wikipedia.org/wiki/1000BASE-T#1000BASE-T
amgsoft's post pretty much sums it up... The only thing I'd like to add is that network latency (or 'ping' - not the command) also makes a difference. They way TCP packets work is by the machines acknowledging packets received and though the ack is small and relatively simple to process, it will still take time to reach the sender/receiver and how fast that is will depend on ping.
Now calculate the time spent for transmission of each ack packet set back through the network , there's the rest of the time and bytes which constitute remainder of 125Mb/s, the rest is in Amgsoft's post.
Well written indeed...
It would be good to see the test using a cross over cable from computer to computer to see if the switch is the limiting factor. Ug!
Disappointed this didn't look at jumbo frames - something that is hard to set up for a typical home enthusiast.
Anyone know if a Gigabit switch connected to a Fast Ethernet router will slow down much? The router is the HDCP server for the network.
The one thing no-one seems to consider is that the purpose of a gigabit network is not to provide gigabit bandwidth for one PC. It is to provide a much higher bandwidth for multiple PC's. The article is incomplete in that it doesn't compare the throughput using multiple PC's.
This article is well below the usual standard of THG. Framesize, payload, concurrent transmission, collision detection, cut through vs store and forward in switches, where shall one begin? The "cable test" is silly beyond words (and so is bobucle's comment on going BEYOND spec). Schoolboy stuff. Amgsoft started to explain but clearly ran out of space/patience.
I wish you had added different switch models; I have noticed some gains with highend switchs in real world going from higher end netgears to Cisco 3950 series no other systems that was a missing layer; good article thanks!
You forgot one important item to try. If your switch supports it, you can send jumbo frames. You would also have to change the MTU and RWIN the corresponding workstations (if running windows). This will also increase the network transfer speed in a single file copy from workstation to workstation. Typical results are about 5% - 20% increase in speed network speed. Of course the disks are still the bottle neck. This is something that is commonly used on iSCSI connections with the MS iSCSI initiator and would work for the workstations as well as long as the swith supports it.