I’'ve written a perl script which is designed to take the XML output from the Jive Messenger IM audit log and put it into a MySQL database. The script was for real-time conversion and in order to do this was setup to read from a file pipe. I used the mkfifo command to create a pipe called jive.audit-0.log which is the filename used to log IM messages. Testing using a previously made log file worked perfectly.
The problem came in when I attempted to have Jive Messenger actually use this pipe to send the XML to my script. For some reason, the pipe was replaced with a normal log file. Is this typical behavior for for Jive Messenger?
I can’‘t use chattr to make the pipe immutable, so I kept the pipe write access for the user, but removed it for the directory so that it could not delete the file. However, Jive Messenger does not write to the pipe. I’'ve been able to cat the test log to the pipe as the jive user and know that the pipe works.
Unfortunately, I don’‘t know enough about unix pipes to say for sure what Jive Messenger might be doing wrong to break your setup. It might be that the “rotateFiles” method in the AuditorImpl class is causing the problem. It has logic to delete and rename the audit files, which might be messing up your pipes. One option would be to have a setting that would disable the file rotation logic – it would just always write to a single file name, which should work with your pipe setup. If this sounds like it might be the problem, I’'d be willing to add this code into CVS in the next day or two.
From what you’'ve said, it would seem that the rotatelog function is the cause of the problem. The ability to have it write to a single file without deleting it is exactly what I need. I was originally going to attempt to go through the code from the CVS and see if it I could put similar functionality into it, but I would appreciate it if you could do this as you know what to look for and to change.
When you make the changes, will I have to compile the source or will it already be compiled? Once I get this system operational, I’'ll post my script so that others can utilize it until you have the mysql code incorporated into a release.
How do you handle duplicate message packets? I.e when someone sends a message, it is logged with their stream ID, then it is logged again with the stream ID of the recipient.
Do you just log it anyway, or discard it based on the packet timestamp and message attributes which are identical for both entries?
When auditing nothing is discarded since the goal is to have an exact trace of what happened in the server (eg. things that were said). In any case, you can write your own packet interceptor with your own logic if you need.
When auditing nothing is discarded since the goal is
to have an exact trace of what happened in the server
(e.g… things that were said).
Completely understand.
In any case, you can
write your own packet interceptor with your own logic
if you need.
Possibly a bit beyond me, Java and I are not well acquainted.
I was just wondering if anyone discards the almost duplicate related packet elements when sifting through the audit data, and if so how they go about it. I.e. two packets are logged - one from the the sender and one to the recipient. For reviewing purposes, I would only need one as either have all the info I need (sender, recipient, time and message text). How likely would it be for unrelated packet elements to have exactly the same timestamp?
I’'m looking into knocking together a program to format the logs into an easy point and click, search and read format for our managers who might suspect not so nice conversations may be taking place. Unfortunately, this stuff can happen despite having a very clear network usage policy in place .
As for our managers, I’'m sure most of them have never even heard of XML let alone being able to pick their way through it . Of course, I could just wait until JD releases his Perl script, then I can just pull the stuff from the database.
I’'ve just thought, there is also the special case of using the broadcast plugin. I would only need the initial packet sent to all@mydomain.com, and not the 80 or so recipient packets. And, of course, due to the number, the timestamps vary. So that blows the duplicate timestamp idea out of the water
I guess it would be down to the sender and the message text and assuming that the first log entry with that text and sender would be the sender packet, and just retain that one for review. Hmm.
I see this thread started back in March and Matt was talking about changing the logging facility to allow writing a single file. Did that ever happen? Is this feature included in 2.1.3?
Also, JD, are you interested in releasing your Perl script? Perl is something I can do, unlike Java, so I’'d be interested in seeing this.
message text and assuming that the first log entry
with that text and sender would be the sender packet,
and just retain that one for review. Hmm.
I find it very strange that the XMPP protocol doesn’'t make any distinction in the XML between inbound and outbout packets!
It looks like something like you suggest would indeed need to be done. Probably just remembering the last message sent “from” each sender would be enough and cut down on the amount of data you needed to cache. Could someone send a second message before Messenger finished delivering the first one? I’'m not so concerned about broadcast messages, I guess.