Cache LDAP roster

Hello,

Is there any way to control when the cache for LDAP rosters gets cleared/refreshed

The issue I’m having is that the roster (shared groups) that gets pulled from our Active Directory server is pretty big (~200) users so it takes a long time to retrieve then push to client if not cached. On top of that, the client I’m using has a hardcoded timeout value (20sec) after which it disconnects from server if roster is not received.

If I retry login a few times, it eventually gets the roster down and logs in then works for the day. However, next day this happens again.

Is it possible to manually control the cache, so that for example, I can run a scheduled script at night that clears cache then reconstructs from LDAP?

This seems to be a client issue. Client roster changes are stored to some degree on the server as they will follow you from machine to machine. It sounds like your client needs to have the timeout changed to allow the roster to fully load.

Yes, it is a client issue, however I cannot change the timeout. I need to find a solution outside of the client that makes sure that the roster/groups are available faster. Even if I can increase the timeout, more than 20 seconds delay for connecting is a little too long for users.

I think this is at least partly related to http://www.igniterealtime.org/issues/browse/JM-379

I do want to see roster updates as I add/remove users from the LDAP side but I do not want the cache to be so small that clients have to wait for server to pull roster data from LDAP every time they log in. Has anyone found a workaround for this?

Or is there an optimization available for the LDAP integration so not so much data needs to be transmitted? Looking at a packet capture, there seems to be an awful lot of traffic for retrieving the roster (~ 2-3mb in 1min). Not sure if it pulls all the vcard data by default too as I do not use it so would be nice to be able to disable it.

you could eliminate the vcard data by editing your openfire.xml and removing or remarking out the vcard mapping. I would make a copy of the file first. Could you also share what client and LDAP server you are using?

I will try to remove the mappings and see if it speeds things up. I had removed the “provider” entry for vcard in the XML at some point but did not make a difference.

The LDAP server is Windows 2003 AD and the client is an in-house SIP softphone that has an XMPP implementation but is not very optimized such as the hardcoded timeout and the need to get the entire roster before considering a login as succesful.

I’ve removed the vcard sections from openfire.xml, but the delay is still there.

In summary, my problem is that it takes more than 30 seconds to retrieve the roster if it is not cached. I’m hoping I can somehow force a cache refresh nightly so updates are received and then does not expire during the day.

It may also be a problem with my LDAP searchFilter. Currently it is:

<searchFilter>(|(memberOf=CN=GROUP1,OU=Groups,OU=Corp,DC=test,DC=local)(m emberOf=CN=GROUP2,OU=Groups,OU=Corp ,DC=test,DC=local)(memberOf=CN=GROUP3,OU=Groups,OU=Corp,DC=test,DC=local)(membe rOf=CN=GROUP4,OU=Groups,OU=Corp,DC=test,DC=local))</searchFilter>

This same query takes about 2-3 seconds to retrieve all information from another application I use.

You could always restart your server nightly. Some caches clear on reboot. I do not think you are going to get beyond this though. Try manually clearing the roster cache in the server admin, then login. My guess is that it will have the same result. if not then use this link to see what other variables you can set the server to adjust:

Specifically the value cache.username2roster.maxLifetime

If I restart the server then I assume all cache is gone, which means next user that logs in gets the timeout. I’m looking for a way to clear the cache and then repopulate it automatically.

I’m not sure if anyone else has ~200 users in their roster through LDAP but seems like this takes up to 1 minute to send to client which is a lot. The LDAP server is about ~5ms away so I don’t know why it takes this long.

Maybe my LDAP settings are not optimized? Here they are:

<ldap>

<host>ldapserver</host>

<port>636</port>

<baseDN>ou=corp,DC=test,DC=local</baseDN>

<adminDN>TESTDOMAIN\ldap</adminDN>

<adminPassword>ldap</adminPassword>

<connectionPoolEnabled>true</connectionPoolEnabled>

<sslEnabled>true</sslEnabled>

<ldapDebugEnabled>false</ldapDebugEnabled>

<autoFollowReferrals>true</autoFollowReferrals>

<usernameField>sAMAccountName</usernameField>

<searchFilter>(|(memberOf=CN=GROUP1,OU=Groups,OU=Corp,DC=test,DC=local)(m emberOf=CN=GROUP2,OU=Groups,OU=corp ,DC=test,DC=local)(memberOf=CN=GROUP3,OU=Groups,OU=corp,DC=test,DC=local)(membe rOf=CN=GROUP4,OU=Groups,OU=corp,DC=test,DC=local))</searchFilter>

<nameField>cn</nameField>

<emailField>mail</emailField>

<groupNameField>cn</groupNameField>

<groupMemberField>member</groupMemberField>

<groupDescriptionField>description</groupDescriptionField>

<posixMode>false</posixMode>

<groupSearchFilter>(objectClass=group)</groupSearchFilter>

</ldap>

The roster is shared with all users and it’s made up of these 4 groups.

Also, is there any way to know what the default cache size is?

your SSL will slow you down some. I have 200+ users on a busy day and over 13 shared groups everyday. My sign on takes a little while with SSO but is almost instant without it.

Sure enough, disabling LDAP SSL speeds things up a bit.

With SSL, the time it takes to retrieve the roster and send to client is 2 minutes

Without SSL, it is around 40 seconds.

Unfortunately, I need to keep SSL as the connection goes over the public internet (through a port forward on the router) from Openfire server to the LDAP server inside my network.(Before you ask, the latency between them is less than 10ms so that is not the issue.)

I still don’t understand why it takes so long to retrieve a simple list of users based on an LDAP filter.I have added all the users into a single group so the filter now is much smaller and easier to manage. A simple PHP script I wrote that retrieves the same information as what is in the roster takes less than 5 seconds to retrieve all the data.

Anyways, I think the solution will be the following:

  1. Set cache to more than 24 hours

  2. Restart server nightly

  3. Run shell script that uses mcabber (console based Linux XMPP client) that logs in to the server nightly for 5 minutes so the LDAP roster is retrieved.

It is an ugly solution but I cannot think of anything else. I’m not sure about the cache settings though, I’m hoping these would do it.

cache.group.maxLifetime

86400000

cache.ldap.maxLifetime

86400000

cache.username2roster.maxLifetime

86400000

cache.username2roster.size

262144

The ldap filter is not used at the client level. That filter is only used at the server level to determine valid users and groups. The client instead authenticates to the server the server then looks at LDAP for information such as password then at the openfire database for user set changes to their profile and other things. then that data is handed off to the client that all takes time. I do not think anything you do will get your time down, other than removing ssl.