Too many database connections; openfire dies

Openfire Openfire Support
1

Openfire for some reason creates a lot of database connections. By default, openfire has 5 connections. I increased that number to 25, then 50, and now 250. The more i increase the number to, the more connections it makes and the worse it gets. here are the database properties (sorry it is so long)

what is this pool-monitoring and how can i stop it from spawning and not dying?

here are my settings for the database, i am really trying here!

House Keeping Interval:
30 seconds
Maximum Connection Lifetime:
1797 seconds
Minimum Connections:
5
Maximum Connections:
200

Has anyone seen this before, how can I fix? I have no idea how this started, 3.7.0 seemed to be pretty stable but about a month ago we saw this problem and no matter how many connections or how i adjust the settings this problem is getting worse. The only fix is to reboot the server. (i even set a scheduled task to do it nightly! and that still doesnt solve the problem).

105
06:52:11
09:21:29
client-13
104
06:51:36
09:21:36
Jetty-QTP-AdminConsole-31
103
06:51:04
06:51:14
pool-monitoring830
102
06:49:10
06:50:04
pool-monitoring829
101
06:49:04
06:49:04
pool-monitoring828
100
06:42:01
06:48:09
pool-monitoring827
99
06:05:13
06:40:04
pool-monitoring822
98
05:37:27
06:46:04
pool-monitoring824
97
02:08:39
05:37:26
pool-monitoring810
96
01:35:04
02:08:21
pool-monitoring767
95
01:27:04
01:34:09
pool-monitoring759
94
00:02:14
01:26:27
pool-monitoring756
93
23:30:55
00:02:04
pool-monitoring739
92
23:29:53
23:30:04
pool-monitoring731
91
23:29:04
23:29:04
pool-monitoring730
90
23:28:04
23:28:04
pool-monitoring728
89
22:07:20
23:27:22
pool-monitoring729
88
22:07:04
22:07:09
pool-monitoring713
87
22:06:04
22:06:04
pool-monitoring711
86
22:05:04
22:05:09
pool-monitoring710
85
22:03:15
22:04:04
pool-monitoring709
84
22:01:20
22:03:22
pool-monitoring708
83
22:01:04
22:01:04
pool-monitoring706
82
20:35:04
22:00:04
pool-monitoring705
81
20:34:02
20:34:15
pool-monitoring684
80
20:01:19
20:33:32
pool-monitoring683
79
20:01:04
20:01:04
pool-monitoring674
78
20:00:04
20:00:04
pool-monitoring673
77
19:19:18
19:59:37
pool-monitoring672
76
17:55:17
19:19:28
pool-monitoring663
75
17:55:04
17:55:04
pool-monitoring645
74
17:54:04
17:54:10
pool-monitoring644
73
17:53:04
17:53:10
pool-monitoring642
72
17:52:04
17:52:04
pool-monitoring643
71
17:51:04
17:51:10
pool-monitoring641
70
17:49:07
17:50:04
pool-monitoring640
69
17:08:04
17:08:04
pool-monitoring630
68
16:58:04
17:49:05
pool-monitoring639
67
16:13:16
16:57:29
pool-monitoring629
66
16:13:04
16:13:04
pool-monitoring618
65
15:16:04
16:12:28
pool-monitoring619
64
15:15:04
15:15:04
pool-monitoring606
63
15:13:14
15:14:04
pool-monitoring605
62
15:13:04
15:13:28
pool-monitoring603
61
15:12:04
15:12:10
pool-monitoring604
60
15:10:04
15:11:04
pool-monitoring602
59
15:07:14
15:09:21
pool-monitoring600
58
15:07:04
15:07:04
pool-monitoring601
57
15:06:04
15:06:10
pool-monitoring599
56
15:05:04
15:05:04
pool-monitoring598
55
13:48:52
15:03:16
pool-monitoring595
54
13:37:13
15:04:34
pool-monitoring597
53
13:37:04
13:37:04
pool-monitoring579
52
13:02:43
13:36:10
pool-monitoring577
51
12:05:05
13:02:09
pool-monitoring569
50
09:33:34
12:54:09
pool-monitoring565
49
08:07:09
08:13:04
pool-monitoring497
48
07:49:09
08:06:33
pool-monitoring494
47
07:43:09
11:20:04
pool-monitoring542
46
07:43:04
07:43:28
pool-monitoring486
45
07:42:04
07:42:09
pool-monitoring487
44
07:37:09
07:41:04
pool-monitoring484
43
07:31:04
07:37:20
pool-monitoring485
42
05:33:04
07:30:09
pool-monitoring482
41
05:31:04
05:32:04
pool-monitoring455
40
05:30:04
05:30:04
pool-monitoring453
39
05:23:04
05:29:27
pool-monitoring452
38
05:22:04
05:22:04
pool-monitoring451
37
05:19:07
05:21:09
pool-monitoring449
36
05:19:04
05:19:04
pool-monitoring448
35
05:18:04
05:18:15
pool-monitoring447
34
05:17:04
05:17:04
pool-monitoring446
33
05:16:04
05:16:10
pool-monitoring444
32
05:11:04
05:15:21
pool-monitoring443
31
05:01:07
05:10:22
pool-monitoring441
30
04:55:07
05:01:09
pool-monitoring437
29
04:43:07
04:48:16
pool-monitoring435
28
03:00:04
04:57:39
pool-monitoring438
27
02:59:04
02:59:05
pool-monitoring404
26
00:53:04
02:58:15
pool-monitoring403
25
00:52:04
00:52:04
pool-monitoring373
24
00:50:04
00:51:22
pool-monitoring371
23
00:01:02
00:49:38
pool-monitoring369
22
00:00:04
00:00:10
pool-monitoring353
21
23:47:04
23:59:04
pool-monitoring352
20
22:12:04
23:46:04
pool-monitoring348
19
22:07:00
22:11:04
pool-monitoring323
18
21:44:04
22:06:10
pool-monitoring321
17
21:43:00
21:43:09
pool-monitoring314
16
20:28:04
20:29:04
pool-monitoring299
15
19:26:04
21:42:34
pool-monitoring315
14
14:02:04
19:25:21
pool-monitoring283
13
08:55:54
10:11:04
pool-monitoring143
12
07:58:04
14:01:04
pool-monitoring202
11
04:10:04
07:57:26
pool-monitoring115
10
04:09:04
04:09:17
pool-monitoring62
9
04:07:58
04:08:04
pool-monitoring60
8
03:00:45
04:07:10
pool-monitoring61
7
02:48:45
03:00:11
pool-monitoring42
6
02:41:04
02:48:09
pool-monitoring39
5
00:11:56
01:38:04
pool-monitoring18
4
00:11:56
01:43:20
pool-monitoring20
3
00:11:56
02:29:09
pool-monitoring32
2
00:11:56
02:40:09
pool-monitoring34
1
00:11:56
00:17:38
pool-monitoring1

2

No payed support, but at least someone who want’s to listen. What’s your setup? Openfire Version? OS? Database? Installed Openfire Plugins? How long is the installation running? User base? Any recent changes? Was it working prior this issue? For how long?

More details will help correcting this issue.

3

Hi Walter. Thanks for your reply.

Openfire= 3.7.0

OS= Centos 5.4

Database= SQL Server 2005 (SP2) running on Windows 2003

Plugins=Broadcast (1.8.2); Client Control (1.1.0); Fastpath Service (4.2.0); Kraken IM Gateway (1.1.3beta3); Monitoring Service (1.2.0); Message Of the Day (1.0.4); Search (1.5.1)

About 50 users, no recent changes, it was working fine for months, and then it ran into the issue that needed a reboot, then nothing for 3 weeks, now frequency seems to be increasing. increasing the number of database connections does not seem to be decreasing the frequency.

any ideas? on the database (sql server) i check and there are not 100 connections, there are only a handful at most. What do you suggest to find the problem or do you have any clues here?

EDIT: frequency is such that i need to reboot every week via a cron job and still during the week i am having to reboot once or twice a week.

4

Some thoughts abut this setup:

50 users on CentOS and a seperate DB server on windows is unusual. I would run the DB on the OF server too. My setup 3.6.4+mysql/CentOS on VM Ware supports 12000 registerer and 3000 concurrent users with roughly 400 active sessions at peak. I have issues with the JVM of OF, but a weekly reboot takes care about that. I guess you are using SQL Server because you have backup procedures for it?

Some specutations about your setup:

Using the monitoring plugin increases the DB size quite a bit. You may wnat to review DB sizes. Take a look at this cleaning up document: http://community.igniterealtime.org/docs/DOC-2199

The other option is the Fastpath plugin that might cause this issue. There are not that many users of it around that take care about it. Is Fastpath in heavy use or became recently a popular service?

Other than that: IMHO nothing unusual in your setup. It’s a pretty small one actually.

5

We use SQL Server because we have good experience with it, so you are correct, easy to administer and so deployment was a snap and backups are easy.

I will look into that document because we have never cleaned up.

As for fastpath, I find it disconcerting that it is no longer really supported as well, but we do use it a lot for live chats and live support for our customers. Do you have any recommendation on what is a better “live chat” or “live support” module? All of our employees use spark because when we deployed it there was strong integration with spark/openfire/fastpath.

I will (a) follow the doc; (b) move DB to server itself; © investigate replacement to fastpath because i think it might go legacy

6

I want to share my findings to help others that might be experiencing this problem. I noticed that the openfire server had a hundred open sockets but the sql server had only two. So I started to think OS or driver issue.

I found in the jTDS (sql server driver) changelog documentation of a later version:

o Corrected bug [1755448], login failure leaves unclosed sockets.

o Added missing finalizer in connection class to ensure resources are released

if an application fails to close a connection.

So either i am experiencing some login failures (unlikely) or openfire simply has some bug where it does not release resources properly (likely and generally not troublesome or worth hunting down).

Given this, I upgraded from jTDS driver 1.2.2 to 1.2.5 and thus far this issue has gone away. Simply copied new jtds.jar to /opt/openfire/lib and deleted the old.

Openfire should package a newer jTDS driver with future releases.

7

The default maximum connection limit (in the absence of any configuration) is 10.

In the openfire.xml configuration file, in the section, add the following tags to set minimum and maximum connection limits.

5
15