[geeks] Nifty NAS board (Atom-based FlexATX from Supermicro)

[Nadine Miller]

On May 21, 2009, at 5:19 AM, Mark Benson wrote:
>>> Surely a later, lower power 9xx series chipset that supports 4GB  
>>> sticks of
>>> RAM and PCIe isn't  too much to ask, right? I'd pay more for the  
>>> board
> just
>>> to get that,
>>
>> See above, the 4 Gig RAM would be nice, but 2 Gigs is OK. The  
>> "mature"
>> chipset and on-board graphics ensures good compatibility with most  
>> OSes.
>
> Except Solaris 10 :P I guess it works okay in OpenSolaris though which
> would suffice for most applictions.
> Regardless, I'm sure there are better, more modern Intel chipsets they
> could use that work as well as a 945 with the OSs it's targeted at.

My big concern would be the on-board NIC.

I have two Gigabyte motherboards (EP45-UD3P, and G31M-ES2L), both with  
Realtek NICs (same chip, 8111C).  Both Solaris 10 (though one is u6  
patched up to May 15th, the other u7, patched up to May 15th).  Md5sum  
on the NIC driver is the *same*, so it's not a regression unless the  
bug is someplace else in the TCP stack.  One is a trooper, holds up  
fine under heavy network load.  The other falls off the network  
someplace between 40 and 60GB continuous traffic via ZFS send/recv and  
can only be brought back with unplumb/plumb (performs poorly  
thereafter) or via reboot (rinse, repeat).  It's not the hardware,  
because the same box held up for a 298GB transfer via rsync using a  
Linux live CD.

I'm going to purchase a PCI-E Intel card for box B so I can use it  
with Solaris.

>> OK, so your saying ZFS would benefit from additional RAM, that's  
>> what I was
>> curious about...
>
> Yeh, ZFS will basically use whatever is can find for Cache and 2GB is
> pretty much 40-60% used by the OS leaving only approx. 1GB for the
> Cache. On a system with 4GB you've got upwards of 75% of the RAM free
> to do that.
>
> The only other effective way I've seen mentioned of doing it, that
> might be practical on a 4-channel board, is use a SATA SSD drive for
> cache, it's something Sun have been getting excited about recently in
> arrays, and the facility is there in ZFS to use it, I don't know the
> full details though.

ARC is tuneable, but I would agree that 4GB is a better starting point  
unless you are not going to use the box for anything but NAS work.   
That said, I ran my big rsync (all large video files) copying to the  
NAS while hubby was streaming from the NAS over wireless.  Target ZFS  
fs is RAIDZ; source for outbound streaming was netatalk.  I have 8GB  
of RAM and have done no tuning.  Generally, tuning is not recommended  
unless you absolutely need it and understand why you need it.  ZFS is  
a moving target--and moving fast.  Nevada zpools are not compatible  
with Solaris 10, for example.

I advise reading through these two Solaris Internals wiki pages, just  
to get up to speed quickly:

http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide
http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide

The issue with ZFS is that it likes to allocate in big chunks of  
512MB.  By default, 7/8 of system RAM can be consumed by ZFS.  It will  
deallocate, too, so in practice, this is not as bad as it sounds  
unless you have a RAM hungry DB process or similar that doesn't like  
having to compete with other processes for memory.

You can use SSD for cache, you simply add it to the zpool with the  
monicker of "cache" instead of "mirror".   You can also move zil/slog  
to SSD or another, separate spindle.  Moving zil can help things, if  
you are bumping into space constraints, particularly, or if you have a  
heavily loaded zpool.  The zpool man page has a section on "Cache  
Devices" (really the ZFS man pages are pretty good references).

Aside from my hardware problems and a failed kernel patch install w/ 
the May 15th patches that required me to go into failsafe and re-apply  
it, everything has been pretty painless as far as Solaris goes.   
Getting OSS running on Solaris has been hit or miss, but I was  
expecting that.

=Nadine=