[geeks] Upper Memory Limit - Java

[Joshua Boyd]

On Thu, Jan 31, 2008 at 07:44:45AM -0500, Francois Dion wrote:
> On Jan 30, 2008 7:34 PM, Brian Dunbar <brian.dunbar at liftport.com> wrote:
> > To be fair, after a round of phone calls last night and today this seems
> > to be a self-inflicted issue; other customers are putting far larger
> > amounts of data through without the problems we are having;  I suspect
> > the guys running EDI don't quite have a handle on 'how' to do what
> > they're asking the software to do.
> >
> > Which reminded me of a conversation I had with them two months ago ..
> >
> > Me: (Looking at the raw xml files that are hundreds and hundreds of mb
> > in size ) Those are some big files.
> 
> If it is not a trade secret, what kind of data are you transferring
> that might require such HUGE files? I've never seen XML files that
> big, short of a database dump we had to do for our corporate office.

I ran into a case once where there were huge XML files used as logs.
They were too large to fit in memory, so using DOM to anylyze them was
no good and SAX had to be used.  That was interesting to learn how to use.