Feature Request: OpenArchive and Monitoring Integration (convergance)

Openfire Plugins Community Plugins
1

Please excuse me if I made any incorrect assumptions about the nature of this discussion board, the nature of the OpenFire internals, or the nature of XEP-0136 and OpenArchive, I’ve only been working with the software (playing) for a week, however it’s really started a fire (pun intended) in me to help with this awesome project.

I noticed a few people suggest that OpenArchive should be integrated into the Monitoring and Conformance tool. I came to that same conclusion when I decided to dig into the database layout.

I’ve written up a conceptual implemention, however I know absolutey zero about programming in java, so I am unable to contribute any code or code examples.

My concept as follows:

We make one new table

OFARCHIVELINKS, which contains the fields SUBSCRIBERID, MESSAGEID, and ARCHIVE_BODY

Also, we modify one table

OFMESSAGEARCHIVE, to which we the fields, MESSAGEID (self explanitory), CONFORMANCE, and LINKCOUNT. Conformance field allows us to keep track of messages that are required regardless of the linkcount, NULL meaning unneeded, “state”, and “body” being valid values. The linkcount is an incremented/decremented field letting us know if a message may be purged.

We follow the criteria used by the existing OpenArchive plugin to decide if a message should be logged, OR’ing this behaviour with the CONFORMANCE logging behaviour.

For each conversation participant logging state that the message matches, we increment the OFARCHIVEMESSAGE LINKCOUNT field for said message (this should take place before the message is actually on the database). Also, an entry in OFARCHIVELINKS will be created.

When a user deletes something from their history, we don’t actually delete the message data directly. Instead, we only delete the reference entry in OFARCHIVELINKS and decrement the OFMESSAGEARCHIVE LINKCOUNT field.

When an admin removes entries from conformance logging (out of time frame or moved to other storage) we only update the OFMESSAGEARCHIVE CONFORMANCE field to NULL.

Periodically, the server will perform database cleanup and remove any messages from OFMESSAGEARCHIVE that have a LINKCOUNT of 0 and CONFORMANCE = NULL.

Also, somewhat less often, we can re-count the link counts by checking the OFARCHIVELINKS table for each messageid in the OFMESSAGEARCHIV, while at the same time allowing us to purge unneeded message bodies (where the state is still required).

Upside:

  • This method would require a minimum of rework in the system

  • Allows one database to provide user end logging aswell as conformance logging

  • Reduces overhead and storage requirements if running both types of archiving

  • Eliminates duplicate storage of messages for local to local messaging (as is the case for OpenArchive)

Downside:

  • Introduces one (or more) database transactions (may be batched) for each message (OFARCHIVELINKS entries)

  • OFARHIVELINKS table may have multiple entries per message (one for each archiving participant) and become unweildly

Disabling server side history while allowing conformance logging to take place would reduce/remove the impact of these changes while keeping the code available for future use in the installation.

Any thoughts or suggestions would be appreciated.

Like I said, I unfortunately cannot contribute code, however I would be mighty pleased to have a hand in this regardless.

2

I guess that would be possible though i would prefer to store the links at the conversation level instead of at the message level.

3

I had considered that aswell actually. At the conversation level it reduces some of the overhead, and reduces the entries in the link table in my model (substantially).

However, it also reduces the granularity for the user, forcing him/her to delete entire conversations instead of individual messages in the case one would want to remove something from their history.

I’m not sure that it’s a big deal, so conversation level might be a better method.

4

Afair XEP-0136 does only support operations at the conversation level (called collections in XEP-0136). If that’s the case having the link at the conversation level makes more sense, if not you are right that your original suggestion offers more flexibility.

5

You are absolutely correct.

XEP-0136 states: A collection is a set of messages to/from the same user that are received near each other in time or as part of the same conversation thread.

Furthermore, it says the only way to modify messages is to remove and recreate the collections. So, definitely working at conversation level is the proper way to do this.