[rescue] Weird I/O behavior

[Chris Byrne]

Guys,

I'm seeing something strange here. I'm doing some I/O throughput testing on
large file transfers from local storage to a SAN.

I've got three boxes here all connected to a fiber channel storage array
with a proven sustained throughput to a single raid head with a single large
data stream of around 15MB/s and a peak of around 20-25MB/sec.

The boxes are as follows

E450, 2x400-4meg, 1g ram, 1g swap, 2 emulex 64 bit 66mhz (in the right
slots) FC hbas, DMP, default setup on sd and lpfc etc...
E450, 2x400-2meg, 1g ram, 1g swap, 2 emulex 64 bit 66mhz (in the right
slots) FC hbas, DMP, default setup on sd and lpfc etc...
SF-280R, 2x750-8mb 4g ram, 4g swap, 2 emulex 64 bit 66mhz (in the right
slots) FC hbas, DMP, default setup on sd and lpfc etc...

The SunFire is seeing sustained writes of between 15 and 18MB/sec with peaks
of around 20 and dips into the 10-12 range. This is what we would expect.
The 450's on the other hand are seeing sustained throughputs in the 6-9
MB/sec range with peaks at 12 or so and troughs into the 4 or 5 range. This
is clearly unacceptable write throughput from machines that should easily be
able to fill the 15MB/sec available.

Read performance for all of the systems is identical with sustained reads
around 20MB/sec with peaks of just under 30 and troughs of around 15

All are going to the same filesystem (not at the same time of course), a
14x72 gig 0+1 store, broken into several testing LUNs running VxFS and
volume manager with default install parameters and no optimisations. Similar
though slightly faster result occur for a raid 5 store. All raid is done in
the array none in software.

There are no application loads on any of the systems during the testing
procedures. Memory utilization doesn't change significantly and the cpus are
fairly quiet. Actually almost no system load is imposed by the testing
itself other than in the I/O subsystem.

The test itself does use large files. In particular files over 1GB in size,
significantly exceeding the physical memory of the 450's. The thing is there
isn't significant virtual memory activity on the 450's to indicate massive
swapping etc...

I've verified the OS configs on the systems are substantially identical
minus hardware specific differences. No real performance related parameters
are different.

Any ideas?