Serious regression problems

Aman_Nanner · March 16, 2003, 7:17pm

Hi,

After using alpha5 for a while, I tried adding users to the contact list did not work anymore. If I tried to “set” a new user in my roster, the returned roster list that I got back did not contain this new user. I reverted back to alpha4, and this problem did not occur.

Then I discovered beta1 had been posted, so I downloaded that. I noticed that the “passwordHash” field in the JIVE_USER table was changed to “password” and now stores the plaintext password rather than a hash. Is there a particular reason for this? The reason I ask is that this seems to be a step back (although I’'m sure you have a good reason to have done it, and it could be part of some larger plan).

After creating a user, I am unable to authenticate to the server. I have noticed that the server now supports MD5 digest authentication. My client, which uses the Muse API, takes advantage of this. However, the server returns “incorrect username/password” so there is some problem. I’'m not sure if the problem lies with the Muse API or your server, but maybe you can try testing this to see if you experience the same problem. Perhaps it has to do with the fact that the plaintext password is stored in the database, and not the digest?

Also, perhaps you can verify if this roster problem I experienced in alpha5 also occurs in beta1. I’'d appreciate if you could look at these problems ASAP because of a demo deadline looming.

Thanks!

Iain_Shigeoka · March 17, 2003, 7:10am

I’‘ll look into the roster issue. A minor change was made between alpha 4 and 5 to bring the server behavior closer to the jabberd. I tested changes against Exodus and didn’'t see problems. In the meantime, if you have the chance, could you paste in a copy of the XML showing the roster set operations and notes where the behavior differs from what you expect.

Also, could you tell me in the forum or via email when your deadline is? I’'ll do my best to help resolve these for your deadline.

As for the password database field, it was changed to ‘‘password’’ because we are going to have an option to switch between using an MD5 hashed password (as previous) or a plain text password (it should make it into beta 2). The plain text password storage is needed for digest authentication (at least I think it is… it may be possible to support digest authentication using hashed passwords but haven’'t been able to find a obvious way to do it).

As for why digest auth is failing… I would guess it’‘s Muse. I tested against Exodus and that works fine. Once again, if you could send me the XML trace, I can verify if Muse is at fault or if it is doing something different but within spec. As a short term fix, you may want to force Muse to use plain-text just to get past your demo. I’‘m pretty sure digest is working though, so I’'d be looking at muse for issues.

-iain

Iain_Shigeoka · March 17, 2003, 4:14pm

Hi,

I’‘ve been looking at roster in alpha5 and beta1. Both appear to be functioning properly. If possible, could you run Exodus against the server and attempt the same roster subscription operation? I’‘d like to confirm that the server is malfunctioning using Exodus (I’'m using Exodus to confirm behavior in many cases).

As for digest authentication, if you can’'t force Muse to use plain-text auth, let me know. I can generate a version of the server that only uses plain-text as a workaround until we add the ability to switch the server behavior dynamically.

Thanks

-iain

Aman_Nanner · March 17, 2003, 5:42pm

Ok, I’'ve done some testing with the Exodus client and have narrowed down the problem:

Exchanging roster in Mxi client in Alpha5

3/17/03 9:19 AM <iq xmlns=“jabber:client” type=’‘set’’ id=’‘id_10040’’>

3/17/03 9:19 AM

Exchanging roster in Exodus in Alpha5

3/17/03 12:14 PM

3/17/03 12:14 PM <presence xmlns=“jabber:client” type=’‘subscribed’’ id=’‘id_10036’’ to=’‘ananner@clymene/Exodus’’>

3/17/03 12:14 PM

3/17/03 12:14 PM Available

3/17/03 12:14 PM <presence xmlns=“jabber:client” type=’‘subscribe’’ id=’‘id_10038’’ to=’‘ananner@clymene’’>

3/17/03 12:14 PM

3/17/03 12:14 PM Available

3/17/03 12:14 PM

3/17/03 12:14 PM available

Exchanging roster in Mxi client in Alpha4

3/17/03 12:27 PM <iq xmlns=“jabber:client” type=’‘set’’ id=’‘id_10106’’>

3/17/03 12:27 PM

3/17/03 12:27 PM <presence xmlns=“jabber:client” type=’‘subscribe’’ id=’‘id_10108’’ to=’‘cbartlett@clymene’’>

3/17/03 12:27 PM

3/17/03 12:27 PM <presence xmlns=“jabber:client” type=’‘subscribed’’ id=’‘id_10101’’ to=’‘ananner@clymene/Home’’>

3/17/03 12:27 PM

3/17/03 12:27 PM <presence xmlns=“jabber:client” type=’‘subscribe’’ id=’‘id_10103’’ to=’‘ananner@clymene’’>

3/17/03 12:27 PM

3/17/03 12:27 PM <presence xmlns=“jabber:client” type=’‘subscribed’’ id=’‘id_10115’’ to=’‘cbartlett@clymene/Home’’>

3/17/03 12:27 PM

As you can see, the difference is that the Alpha5 server no longer returns the inital roster with the added item and its subscription status set to “none”. This messes up my client, but does not mess up the Exodus client.

Another issued I have noticed exists since Alpha4. That is, registering the very first user (the user which receives a USER_ID of 0 in the DB) results in authentication problems for this user. The next user I register (USER_ID=1) is able to authenticate normally. If I manually change the first user’'s USER_ID to 2 from 0 in the database, and restart the server to clear the cache, I am able to authenticate the first user.

As for my MD5-digest authentication problems in Beta1, here is a trace of the XML messages (the plaintext is ‘‘password’’):

Authentication in Beta1

3/17/03 11:59 AM <iq xmlns=“jabber:client” type=’‘get’’ id=’‘id_10028’’>cbartlett

3/17/03 11:59 AM cbartlett

3/17/03 11:59 AM <iq xmlns=“jabber:client” type=’‘set’’ id=’‘id_10030’’>cbartlettBD7EDDB1782AE2DDB5 8EAE6A0DD9F4B87AC89C21Home

3/17/03 11:59 AM Unauthorized

The code freeze for the demo is supposed to be today, but don’'t worry about it because I have decided I will use the Alpha4 server for the demo, since I have tested that server for a long time and it seems pretty robust.

Iain_Shigeoka · March 17, 2003, 8:40pm

Ah. The changed roster behavior introduced in alpha 5 brings the server in-line with the behavior of Jabberd (the reference server). It also makes the server follow the subscription protocol description on the jabber.org site (which was written by examining jabberd’'s behavior).

Essentially, you don’‘t get a roster push until your subscription status (sub or ask) is not ‘‘none’’. I would suggest adjusting your client to react to the server’‘s new roster behavior otherwise it won’‘t work with other Jabber servers (jabberd, the open source server, and other servers based on it like Jabber Inc.’‘s server and the Tipic server). If this is not possible, please explain and we can work out something. I’‘d prefer to avoid making our server behave in a non-standard way (even if it’'s a configurable behavior). However, we want to make sure your solution works and will do what it takes to make that happen.

As for authentication, I see what the problem is. Jabber has standardized on using lower case hexadecimal numbers to represent binary data like digest’‘s result. I will modify the digest code to accept upper-cased hex results (mainly because I think it’‘s technically right (it does represent the correct binary hash) even if the Jabber standards say it isn’‘t (the strings don’‘t literally match)). However, if Muse uses uppercase hexadecimal strings for 0k authentication (does it even support 0k?), it will fail as the hashes won’'t work out correctly.

I don’‘t know if it’'s possible to get it fixed in Muse. If it is possible, you should try to get them to use lower case hexadecimal strings for auth to fall in line with the jabber standards.

Finally, there should never be a user created with id == 0. The database should always be initialized to contain counters starting at 1 for all sequences (see the jiveID table) except for the user id’‘s which start at 2 because an admin account (user id == 1) is always created. If you’'re deleting all records in all databases to clear your data for testing and development, you should run the two groups of inserts at the bottom of the *.sql file for your db. That will initialize the sequence table (jiveID) correctly and create the default admin account.

If you are using the provided scripts to create your database, then let me know. There is a bug somewhere as no id’'s in the Jive tables should ever be 0.

Thanks

-iain

Aman_Nanner · March 17, 2003, 8:59pm

I thought I had followed the protocol according to here:

http://www.jabber.org/protocol/rosters.html

There seems to be a roster push there with “subscription=none”. However, if the standard behaviour is jabberd’‘s behaviour, I will modify the client so that it works with this behaviour (not for the demo though, since I’'ll use Alpha4).

The database problem is my fault. I had manually deleted the records in the database, but I did not run the inserts in the scripts. Thanks for the tip.

Iain_Shigeoka · March 17, 2003, 11:07pm

My apologies. I was looking at the wrong part of the roster exchange when I answered previously. I see what you’‘re getting at now. When you do the initial roster set before the subscribe request, you’'re not seeing the correct return of the roster set information.

I’'ll file a bug and fix it.

-iain

Iain_Shigeoka · March 17, 2003, 11:11pm

Although it’'s essentially the same thing, I think the better document to follow is:

http://www.jabber.org/protocol/subscriptions.html

Chris_Chen · March 21, 2003, 12:09pm

Hi,

This is the author of Muse API. I just happened to chance upon this post during a search for a specific jabber issue. So I would like to take the opportunity and clear up some misunderstanding or misconceptions.

the digest is created and sent as all uppercase. This was done in conformance with Jabber’‘s own jabber client. I haven’‘t change the authentication since sometime last year so I am not positive when the authentication got changed to only accept lowercases. But I will take a look and see what’'s up.
Muse does indeed 0k authentication and it spits out the returned hash in lowercase. The 0k authentication works 100% with jabber.org and I have never had any problems doing testing. If possible, please follow up and let me know if this may be a problem on Muse side.

Thanks,

Chris

Iain_Shigeoka · March 21, 2003, 6:08pm

Hi,

Great to have you in the forums!

Since JIM, the JINC client, went completely proprietary, I’'ve been using Exodus and Jabberd as the definitive source of protocol information, with Psi as a backup to confirm behavior.

The behavior of Exodus and jabberd, follows the protocol description here:

http://www.jabber.org/protocol/authentication.html

(which is usually generated by Peter by looking at the behavior of jabberd and Exodus). At the bottom you’'ll see the sample digest exchange which does use lower case hex numbers. I believe jabberd still accepts upper case but it is non-standand.

I’‘d also like to point out that jabber auth is being deprepricated pretty hard. As you’‘ll see in the jabber.org doc for authentication, SASL is really highlighted and old iq:auth is sort of being presented in a historical light. iq:auth isn’'t even being included in the IETF xmpp standard so when people start using the xmpp standard as the source of information, iq:auth is going to be a real dinosaur.

SASL support is on the near term to-do list for Smack and Jive Messenger.

-iain

matt · March 21, 2003, 8:09pm

Iain,

Should we support upper-case hex chars as well, anyway? I’‘m not sure it would cause any harm. I’'m pretty sure it would just mean changing our hexToByte method to the following:

private static final byte hexCharToByte(char ch) {
    switch(ch) {
        case ''0'': return 0x00;
        case ''1'': return 0x01;
        case ''2'': return 0x02;
        case ''3'': return 0x03;
        case ''4'': return 0x04;
        case ''5'': return 0x05;
        case ''6'': return 0x06;
        case ''7'': return 0x07;
        case ''8'': return 0x08;
        case ''9'': return 0x09;
        case ''a'': return 0x0A;
        case ''b'': return 0x0B;
        case ''c'': return 0x0C;
        case ''d'': return 0x0D;
        case ''e'': return 0x0E;
        case ''f'': return 0x0F;
        case ''A'': return 0x0A;
        case ''B'': return 0x0B;
        case ''C'': return 0x0C;
        case ''D'': return 0x0D;
        case ''E'': return 0x0E;
        case ''F'': return 0x0F;
    }
    return 0x00;
}

Regards,

Matt

Iain_Shigeoka · March 21, 2003, 8:38pm

Ya, I’'ve filed an issue to accomodate uppercase hex values. Should make it in for beta2. I believe most SASL profiles assume hex values are case insensitive so this will be useful in future SASL support as well.

-iain