Enhancement: Memory Management

Hi,

this issue has probably been discussed a lot of times.

The way java handles (or better wastes) the system memory is really horrible.

what about starting a cooperation with the ejabberd (http://ejabberd.jabber.ru/) team?

the erlang based systems runs like charm, has a really impressive memory management, is able to handle virtual hosts, is clusterable… etc.

the only issue with ejabberd is the somewhat painfull webinterface (at least painfull for “non-geeks”).

wouldn’'t it be a really nice thing to combine the best of both projects?

greetings!

Hi,

java does not waste any system memory. It manages its memory fairly good compared to other OO languages.

Does ejabbered run in an application server?

wouldn’'t it be a really nice thing to combine the best of both projects?[/i]

It’‘s usually very hard to convince one of the authors to stop working on its project. And yes it would be nice, so I wonder why no one did create a new project which combines the best of both projects. It’'s only a little bit of work to do …

LG

nume,

Ejabberd uses the very obscure Erlang platform. One thing that’‘s similar to Java about Erlang is that it uses a virtual machine. However, the Java virtual machines are much more advanced (as one might expect given Java’‘s popularity). So, I really doubt that Erlang has better memory management than Java. In fact, it’‘s quite easy to configure Wildfire to use as much or little memory as you’'d like. One community member runs Wildfire with something like 64 MB total RAM(!).

Anyway, I agree that ejabberd has some interesting features and we enjoy cooperating with them as part of the open XMPP protocol process. However, there’'s really no way to share code given our vastly different architectures.

Regards,

Matt

Dear Matt, Dear it2000,

thanks for your answers.

My posting wasn’'t ment as wildfire bashing.

Wildfire is a real great application!

i am no expert, i just repeated, what i read before.

and i tested both wildfire and ejabberd.

i really noticed much less memory consumption with using ejabberd.

and i thougth that erlang is THE swiss army knife for fault tolerant high capacity communication applications.

from what i tried myself and what i read i just expect, that wildfire is not adequate for sites with really a lot of users…?!?

i saw, that you are working on a clustering system, but this looks a bit like “okay - our car needs 40 litres per 100km; now we are developing a nice gasoline trailer”

greetings,

numE

i really noticed much less memory consumption with

using ejabberd.

Anything is possible. But, this would be due to Wildfire and not Java itself.

i saw, that you are working on a clustering system,

but this looks a bit like "okay - our car needs 40

litres per 100km; now we are developing a nice

gasoline trailer"

Heh, interesting analogy. There’'s actually two parts to the scalability work:

  1. Connection managers (what you mention). This will allow multiple servers to take on the burden of handling client connections.

  2. Switching to the NIO library. Instead of using a single thread for each connection, NIO handles all connections with a very small number of threads. This allow for much greater scalability.

Part 1 is done and stable for Wildfire 3.0. There has been some good work done on part 2, but it will be experimental for several months while in-depth testing is done.

Best Regards,

Matt

the only issue with ejabberd is the somewhat painfull

webinterface (at least painfull for “non-geeks”).

Read the second item in the feedback section of this page:

http://ejabberd.jabber.ru/online-demo-webadmin

wouldn’'t it be a really nice thing to combine the

best of both projects?

As Wildfire is GPL, you can port Wildfire’'s web interface to ejabberd

Ejabberd uses the very obscure Erlang platform.

Please do not spread FUD or do you like me recommending ejabberd in these forums? :-p :o)

Regarding your claim that Erlang is obscure:

  • Java’‘s development started in the early 1990s. ↔ Erlang’'s development started in the early 1980s.

  • Java is still not open-sourced. ↔ Erlang is already open-source since 1998.

  • There are several companies that can offer commercial support and consulting services regarding Erlang.

  • Erlang is also invented by a big company: Ericsson.

  • Erlang is also used for mission critical systems, like banking systems: http://www.erlang.se/euc/03/proceedings/1100Danie.ppt

An interesting article about Erlang written by and intended for people with a Java background can be found on http://www-128.ibm.com/developerworks/java/library/j-cb04186.html

However, the Java virtual machines are much more advanced

(as one might expect given Java’'s popularity). So, I really doubt

that Erlang has better memory management than Java.

I’‘m no engineer, but I think this might be caused by Erlang’'s lightweight processes concept. See also the example in my answer to next quote:

In fact, it’'s quite easy to configure Wildfire to use

as much or little memory as you’'d like. One community

member runs Wildfire with something like 64 MB total

RAM(!).

Mickaël Rémond tried Erlang on the Blackdog (also 64MB RAM): http://www.3pblog.net/index.php?entry=entry051104-214413

He also tested ejabberd on it (he just said me in an IM message) and he tells that it worked very responsive. Of course you shouldn’'t try to serve 100,000 concurrent users with such a device

Anyway, I agree that ejabberd has some interesting

features and we enjoy cooperating with them as part

of the open XMPP protocol process. However, there’'s

really no way to share code given our vastly

different architectures.

On the other hand, there is much cooperation possible in the non-code area (cfr. Jabberpedia and marketing a la LCS).

However, the Java virtual machines are much more

advanced

(as one might expect given Java’'s popularity). So,

I really doubt

that Erlang has better memory management than

Java.

Erlang has a better memory management than Java because it has to fullfill one more constraint: Soft real time. This means that the VM cannot block the execution flow and has to do some micro garbage collections. In Java, you often have to face one or two second lag when a heavy garbage collection is triggered. Erlang also offer the opportunity to tune the memory management in nearly every aspect. You can really fine tune the memory management of the system very deeply.

mredmond,

I definitely don’‘t want to start a Java vs. Erlang discussion. Modern Java VM’‘s generally all support soft-real time garbage collectors as well. For example, JRockit (BEA) has a new garbage collector that generally never pauses more than a few milliseconds. IBM and Sun have much the same in their latest VM’'s as well. Of course, you pay for soft-real time with lower total through-put. The cool thing is the ability to tune for different use cases which both Java and Erlang offer. There really is a lot of power by going with a virtual machine approach.

Anyway, I’‘m locking this thread now as it’'s getting off-topic.

Regards,

Matt