[geeks] RANT (ftp)

[Charles Shannon Hendrix]

Mon, 30 Oct 2006 @ 15:45 -0500, Bryan Fullerton said:

> On 10/25/06, Charles Shannon Hendrix <shannon at widomaker.com> wrote:
> > On Wed, 25 Oct 2006 10:32:30 -0400
> > "Bryan Fullerton" <fehwalker at gmail.com> wrote:
> >
> > > I would have given a kidney to switch to Urchin back when I had to
> > > support a WebTrends Enterprise install processing *log files* for a
> > > 1M+ view/day web site (there is no logic in this, don't try to find
> > > it).
> >
> > Just curious, what would you rather do than process log files to get
> > the information?
> 
> Page tagging is the recommended way to use current "enterprise" stats
> programs -- embed a reference to a 1x1 image in the page using
> javascript, and stuff a whole bunch of crap about the client browser
> into a cookie attached to that image. This is the way Google Analytics
> works, and all the other big ASP stats providers.

That should also make them more accurate too, since it is a higher level
view. Or less inaccurate anyway. It's less obtrusive, though it can
still slow page rendering at times.

Of course, a browser like Firefox can block that stuff pretty easily.

> [1] Admittedly related to the mandated log file retention policy -
> "keep them forever, and keep them available in case we want to do
> adhoc historical reporting". They liked their stats a lot.

I worked at a place like that. They wanted logs kept forever, and
eventually they moved to specialized log servers. One shop even hired a
third party data center to handle its logging.

I saved a lot of space on a couple of jobs by converting to my own
format which kept all the data but in a far smaller record size.
It was also a lot faster to process like this.

I just had to make sure I could always reverse the conversion so we
could run more standard log software now and then.

-- 
shannon "AT" widomaker.com -- ["That which is overdesigned, too highly
specific, anticipates outcome; the anticipation of outcome garantees, if
not failure, the absence of grace." -- William Gibson, All Tomorrow's
Parties]