I suspect that the source of the issue is similar to that described in Admin GUI not starting after migration to PBKDF2. If that’s the case, then rolling back a previous database backup (from before the pbkdf2 migration) needs to happen to recover the encrypted data that is now undecipherable.
Openfire 5.1.1 adds hardening, to prevent this kind of thing happening, but it cannot recover an already broken system.