[rescue] advice on rescuing an e10k

[Mike Meredith]

On Mon, 23 Oct 2006 14:12:47 -0500 (CDT), Jonathan C. Patschke wrote:
> On Mon, 23 Oct 2006, Andrew Gaylard wrote:
> 
> > Is Solaris really that much faster on this hardware? And what do
> > you mean by "maximum use of the hardware" and "scalability"?
> 
> Well, first off, Linux doesn't support that hardware.

Which of course is a reason (for a certain kind of person) for getting
Linux onto such a system ... not only is it somewhat useful in itself,
but is quite often useful in throwing up new problems with Linux that
haven't been noticed on other large systems than Linux runs on.

> In general, open source support of large Sun hardware is a dicey
              ^^^^^^^^^^^

I don't think the problem is limited to open source. It took Sun a long
time to make Solaris as scalable as it is ... it is a pretty hard
problem.

> > And what in particular makes it faster? (A better kernel?  Better
> > compiler?  Something else?)  I'm not disputing this, I'm just
> > curious to know.
> 
> Well, Sun's compiler beats GCC in terms of both code size and code

Ignoring the compiler issue, you will probably find Linux is faster in
some areas and Solaris faster in others.

> SPARC systems).  Solaris can get more done at once than Linux above a
> certain hardware threshold (that used to be nearly any 4-way SMP
> system)

I'm pretty sure that Linux scales quite a bit better that 4-way ...
assuming it understands the underlying architecture. On the Fire series,
it is advantageous to keep processes within a CPU board when possible,
if Linux doesn't understand that it will run a fair bit slower.

> There are other options to consider, too.  Large systems (Sun, IBM,
> HP, etc.) tend to have embedded management hardware (thermal,
> partitioning, lights-out management, hot-swap, and component failover)
> that are very frequently only working with the vendor's operating

In principle that's right, however on the Fire systems most of that is
handled by the system controller(s). You'll loose hot-swap and some
component failover (power for instance would be handled by the SC).

> > I was planning to use Gentoo Linux; I've found that compiling for a
> > particular CPU does give a noticible speed improvement over vanilla
> > distributions.
> 
> I've found that any such benefits are largely outweighed by the
> inefficient code that runs on so many Unix and Linux systems today.

Indeed. Too many people are inclined to throw hardware at a performance
problem rather than fix the problem. 

You'll also find that Solaris does quite a bit of CPU optimisation by
stealth :-

% ldd /usr/bin/ls
...
	/platform/SUNW,Sun-Blade-1000/lib/libc_psr.so.1