Anyone else having problems modifying groups for transports in Spark?

I’ve tried to rename groups, change memberships and delete groups, and it’s just not working very well.

Sometimes, when renaming a group, the group and all of it’s members will disappear from the roster completely until I close and reopen Spark.

Other times, when trying to delete a group, it will reappear upon next launch of Spark, resulting in all it’s membership being duplicated with other groups.

At other times, none of the transport buddies or groups show up in the roster when launching Spark. Signing out and back in to the tranport has no effect, only relaunching Spark will bring them back.

This may be specific to the AIM transport.

Yay! One that I’ve already caught and fixed! =) I have a pending fix for that in trunk. In fact I’ll even attach it so you can try it out if you’d like.
gateway.jar (1173701 Bytes)

Wow, that was quick and amazing. I’m an Openfire/Spark n00b, so where shall I drop and install this puppy?

M@

Do you manage your openfire server? (you copy it into the plugins directory in the install root) If you don’t manage it you may need to get someone who does to pop it into place. =/

Answered my own question! Just drop it into the /opt/openfire/plugins directory overwriting the existing jar file. Works like a champ!

BTW, Some new buddies showed up on my roster in a Transports group for each transport I use, are they safe to delete, part of debug code, or there to stay?

M@

That actually depends on how you register. =) Spark uses a special mechanism that involves not having the transports in your roster, but every other XMPP client uses the standard method where you have the transports in your roster. Did you register using the admin interface or directly from Spark? Glad to hear things are going smoothly now though!

I originally registered them thru Spark, but in my testing I removed them and readded them thru the Openfire Admin Console.

M@

ponder You can probably “remove login” and readd. I think that would probably do it the easiest. =)

i still have problems if move icq buddies to other groups or change their name. sometimes i got unsubscribed. the changes are lost after the next or 2. login. for msn buddies it’s working.

with icq we discover some more issues. i have to jabber account each have an icq contact it’s not possible to subscribe each other over icq. i always get unsubscribed. also it’s not working with some version of icq (i think icq lite). but it’s working to subscribe to pyICQt buddie.

also if i update account the other buddy sometimes get a subscription request.

we have setup on our office a fresh openfire and want to move all icq/msn clients to psi and a jabber account. but we still can’t move because their are too many issues with the icq transport.

Yeah I need to rework the way I’m doing OSCAR roster handling. It’s been hard to decide whether to go whole hog into the new API or not. In theory it’ll auto-solve a lot of issues, but by the same token it may introduce a lot more and do I really want that to hold up 1.1.1? ICQ is especially bad for this because of it’s authorization system. See, for ever group your buddy is in, you actually have an entirely separate buddy instance for them on the OSCAR SSI servers. (buddy a in group b, buddy a in group c, buddy a in group d) So to deal with changes and such, we run through and remove buddy instances that are in groups that we don’t want, and buddy instances for groups we do want. What’s wrong with this? Well, if you have an existing buddy A and you move it from group B to C, first off I think the end user might see a “buddy A just added you!” which is uncool. (because we’re “adding” an instance of A to C) More importantly though, in ICQ land, the group B instance has a tie to the fact that the target user (A) already accepted you. By simply removing it instead of modifying it, you lose that tie and may have to reauth. What can be done about this? Basically running through and trying to use modify as much as possible instead of deletes and adds. Just an extra step that in theory shouldn’t be hard but there’s a lot of logistics.

I’m also seeing some other odd behavior that I’m tracking. I can’t say for sure, but I’d swear AIM is arbitrarily adding a “Mobile Buddies” group to some of my contacts.

Anyway, one of my big goals is to “make sure this all works as smoothly as possible for Spark”. Why is this important? I’ve been using Adium X mostly and there’s a few things that it hides from me since it’s not a straight XMPP client. I sometimes, for example, find that Adium X is what was keeping contact A in group B, and that wasn’t really reflected on the server. That’s not a knock to Adium, but rather something that hadn’t occured to me. Spark lives in the server side world, so all changes REALLY occur server side. I think by making sure things work well with Spark, I’ll make sure things work well period. =)

ok, but why you don’t use the pseudo roster table for all transports?

so if i login i get the hole roster (jiveRoster + pseudoRoster). after you got the roster from icq/msn etc. merge it with the local pseudo roster and send only the diffs to the client. same thing roster updates from client.

so you have the option to disable group syncing (i.e. icq) with the real icq world. i think that all jabber user don’t care in which group the buddy is on icq side.

that, will be a much easier way to handle (i think the pyXxx transports have the same handling).

Ok, I was able to remove those Transport buddies by deleting my login information and adding it in thru Spark instead of registering in Openfire.

However, I have a user whose entire MSN contact list is not displaying when he logs into Spark. I’ve had him delete and reregister that transport from Spark, to no avail, and even restarted the openfire service on the server.

Also, I’m seeing issues now with my MSN contacts also, some of the groups I’d created are gone so I can’t move users out of the Unfiled group into them. When I restarted the openfire service, and Spark reconnected the groups all reappeared but my contacts weren’t showing, so I restarted Spark, and then the groups were gone again. They don’t even display when I choose to Show Empty Groups.

Lastly, none of my Yahoo users are showing in my roster, but if I sent a broadcast message, I see some of them listed there. Weird.

M@

notz wrote:

ok, but why you don’t use the pseudo roster table for all transports?

so if i login i get the hole roster (jiveRoster + pseudoRoster). after you got the roster from icq/msn etc. merge it with the local pseudo roster and send only the diffs to the client. same thing roster updates from client.

so you have the option to disable group syncing (i.e. icq) with the real icq world. i think that all jabber user don’t care in which group the buddy is on icq side.

that, will be a much easier way to handle (i think the pyXxx transports have the same handling).

The whole point of a lot of this support is to make it as smooth as possible. If you decide to fire up a real MSN client, I’d like to see you able to mirror your MSN contacts just as you used them with Spark. Pseudo roster is only for services that don’t support bits and pieces. Hell, I’m already having reports come in because using a single group with the Py’s causes a lot of problems. Lots of people are running into size limit caps for example. People often complain about their groups not being mirroed on the legacy service. There’s little reason for me not to use the services own mechanisms. Just needs to be fixed up apparently.

Maenxe wrote:

Ok, I was able to remove those Transport buddies by deleting my login information and adding it in thru Spark instead of registering in Openfire.

However, I have a user whose entire MSN contact list is not displaying when he logs into Spark. I’ve had him delete and reregister that transport from Spark, to no avail, and even restarted the openfire service on the server.

Any debug logs? What MSN client was he using before?

Also, I’m seeing issues now with my MSN contacts also, some of the groups I’d created are gone so I can’t move users out of the Unfiled group into them. When I restarted the openfire service, and Spark reconnected the groups all reappeared but my contacts weren’t showing, so I restarted Spark, and then the groups were gone again. They don’t even display when I choose to Show Empty Groups.

That’s actually something to be fixed in 1.1.1… I found out Openfire was doing something bizarre on me so I’ve countered it. I spoke with Gato about it and until it can be resolved in Openfire itself, I have a workaround. (of course, there’s little reason for me to ever remove the workaround)

Lastly, none of my Yahoo users are showing in my roster, but if I sent a broadcast message, I see some of them listed there. Weird.

Definitely odd. Can you send me some server side debug logs as you log into the Yahoo transport?

Daniel

M@

hmm,

i don’t understood the code completly here but is that line correct (in OSCARSession.java:262)?

if (buddy.getGroupId() == 0 && !grouplist.contains(DEFAULT_AIM_GROUP)) {

or should it be like this:

if (buddy.getGroupId() == 0 && !grouplist.contains(DEFAULT_AIM_GROUP) && !grouplist.contains(DEFAULT_ICQ_GROUP)) {

it seems that this change solves my icq buddy moving problem. i have to dig in further.

one other question:

i tried to change the option that all gateway roster items are stored in db in (BaseTransport.java:1193) and seems to work out of the box.

why i like this behaviour more?

because if want to store the roster offline it’s working also if the roster is not in cache and there is less traffic on login if the roster is not in cache, because i only the roster changes are transferred and not the hole roster, each in one stanza. (if have not enough cache for all users, there are too much of them)

for usage on a mobile client, it is very useful.

do you think, it is possible to add a option therefore?

notz

Hrm. Crap yes, thanks for the patch! (i’m tweaking it a tad but yes, you are correct in that it should be like that) GATE-319 (i am in the middle of trying things right now so I’m going to wait before posting this for real)

it seems that this change solves my icq buddy moving problem. i have to dig in further.

one other question:

i tried to change the option that all gateway roster items are stored in db in (BaseTransport.java:1193) and seems to work out of the box.

why i like this behaviour more?

because if want to store the roster offline it’s working also if the roster is not in cache and there is less traffic on login if the roster is not in cache, because i only the roster changes are transferred and not the hole roster, each in one stanza. (if have not enough cache for all users, there are too much of them)

for usage on a mobile client, it is very useful.

do you think, it is possible to add a option therefore?

GATE-320 =)

notz wrote:

notz

hmm,

i don’t understood the code completly here but is that line correct (in OSCARSession.java:262)?

if (buddy.getGroupId() == 0 && !grouplist.contains(DEFAULT_AIM_GROUP)) {

or should it be like this:

if (buddy.getGroupId() == 0 && !grouplist.contains(DEFAULT_AIM_GROUP) && !grouplist.contains(DEFAULT_ICQ_GROUP)) {

patch to make moving contacts bewteen group working without loose of information & authorization:

/**

  • Synchronizes the list of groups a contact is a member of, updating nicknames in

  • the process.

  • @param contact Screen name/UIN of the contact.

  • @param nickname Nickname of the contact (should not be null)

  • @param grouplist List of groups the contact should be a member of.

*/

public void syncContactGroupsAndNickname(String contact, String nickname, List();

}

if (grouplist.isEmpty()) {

if (getTransport().getType().equals(TransportType.icq)) {

grouplist.add(DEFAULT_ICQ_GROUP);

}

else {

grouplist.add(DEFAULT_AIM_GROUP);

}

}

// TODO: add another step here where we attempt to “move around” contacts instead of just adding and deleting

Log.debug("contact = “contact”, grouplist = "+grouplist);

OSCARBuddy oscarBuddy = null;

try {

oscarBuddy = (OSCARBuddy)getBuddyManager().getBuddy(getTransport().convertIDToJID(contact));

}

catch (NotFoundException e) {

}

Vector freeBuddyItems = new Vector();

if (oscarBuddy != null) {

// Now, lets clean up any groups this contact should no longer be a member of.

for (BuddyItem buddy : oscarBuddy.getBuddyItems()) {

if (buddy.getScreenname().equalsIgnoreCase(contact)) {

if (buddy.getGroupId() == 0 && !grouplist.contains(DEFAULT_AIM_GROUP)) {

// Ok this group is the “main group”, but contact isn’t in it.

Log.debug(“Removing “buddy” from main group”);

freeBuddyItems.add(b uddy);

}

else if (!groups.containsKey(buddy.getGroupId())) {

// Well this is odd, a group we don’t know about? Nuke it.

Log.debug(“Removing “buddy” because of unknown group”);

freeBuddyItems.add(b uddy);

}

else if (!grouplist.contains(groups.get(buddy.getGroupId()).getGroupName())) {

Log.debug(“Removing “buddy” because not in list of groups”);

freeBuddyItems.add(b uddy);

}

else {

if (buddy.getAlias() == null || !buddy.getAlias().equals(nickname)) {

Log.debug( "Updating alias for "+buddy);

buddy.setA lias(nickname);

request(ne w ModifyItemsCmd(buddy.toSsiItem()));

oscarBuddy .tieBuddyItem(buddy, true);

}

}

}

}

}

// First, lets take the known good list of groups and add whatever is missing on the server.

for (String group : grouplist) {

Integer groupId = getGroupIdOrCreateNew(group);

if (isMemberOfGroup(groupId, contact)) {

// Already a member, moving on

continue;

}

Integer newBuddyId = 1;

if (highestBuddyIdPerGroup.containsKey(groupId)) {

newBuddyId = highestBuddyIdPerGroup.get(groupId) + 1;

}

if (freeBuddyItems.size() > 0) { // moving a free buddy

BuddyItem buddy = (BuddyItem)freeBuddyItems.get(0);

BuddyItem newBuddy = new BuddyItem(contact, groupId, newBuddyId, nickname, buddy.getBuddyComment(), buddy.getAlertWhenMask(), buddy.getAlertActionMask(), buddy.getAlertSound(), false, buddy.getExtraTlvs());

request(new DeleteItemsCmd(buddy.toSsiItem()));

oscarBuddy.removeBuddyItem(buddy.getGrou pId(), false);

freeBuddyItems.remove(buddy);

oscarBuddy.tieBuddyItem(newBuddy, false);

request(new CreateItemsCmd(newBuddy.toSsiItem()));

} else { // creating a new buddy

BuddyItem newBuddy = new BuddyItem(contact, groupId, newBuddyId);

newBuddy.setAlias(nickname);

if (oscarBuddy == null) {

getBuddyManager().storeBuddy(o scarBuddy = new OSCARBuddy(getBuddyManager(), newBuddy));

} else {

oscarBuddy.tieBuddyItem(newBud dy, false);

}

request(new CreateItemsCmd(newBuddy.toSsiItem()));

// TODO: translate this

request(new BuddyAuthRequest(contact, “Automated add request on behalf of user.”));

}

}

for (int i=0; i<freeBuddyItems.size(); i++) {

BuddyItem buddy = (BuddyItem)freeBuddyItems.get(i);

request(new DeleteItemsCmd(buddy.toSsiItem()));

oscarBuddy.removeBuddyItem(buddy.getGroupId(), false);

}

}

/code

Wow notz! You’ve really been busy with this! Thank you! GATE-328

BTW, have you signed a contributor agreement with Jive? I don’t really know at what point I have to ask that. =/ I -think- that we’re still good, but wanted to go ahead and ask anyway.

jadestorm wrote:

BTW, have you signed a contributor agreement with Jive? I don’t really know at what point I have to ask that. =/ I -think- that we’re still good, but wanted to go ahead and ask anyway.

no, i never heard about that.

another question perhaps you can help me:

i want to add online status message for icq like pyicq:

that should to the trick in pyicq:

log.msg(“Setting extended status message to "%s"”%status)

self.backMessage = status

packet = struct.pack(

“!HHHbbH”,

0x001d, # H

len(status)+8, # H

0x0002, # H

0x04, # b

len(status)+4, # b

len(status) # H

) + str(status) + struct.pack(“H”,0x0000)

self.sendSNACnr(0x01, 0x1e, packet)

/code

if i implement this in the gateway plugin like that:

request(new SetExtraInfoCmd(new ExtraInfoBlock(ExtraInfoBlock.TYPE_AVAILMSG, ExtraInfoData.getAvailableMessageBlock(verboseStatus))));

/code

it doesn’t work…

2007.08.24 01:53:23 Sending SNAC command: SetExtraInfoCmd: blocks=[ExtraInfoBlock: type=0x2 (TYPE_AVAILMSG), extraData=<ExtraInfoData: flags=0x4 (FLAG_AVAILMSG_PRESENT), data=00 04 6b 68 6b 6a 00 00>]

2007.08.24 01:53:23 OSCAR bos snac packet received: SnacPacketEvent: snacProcessor=ClientSnacProcessor: lastreqid=16, requests: 16, paused=false, snacPacket=SnacPacket type 0x1/0x21, flag1=0x80: 20 bytes (id=3178446837), snacCommand=ExtraInfoAck: blocks=[ExtraInfoBlock: type=0x2 (TYPE_AVAILMSG), extraData=<ExtraInfoData: flags=0x4 (FLAG_AVAILMSG_PRESENT), data=00 04 6b 68 6b 6a 00 00>, ExtraInfoBlock: type=0xd (null), extraData=<ExtraInfoData: flags=0x4 (FLAG_AVAILMSG_PRESENT), data=46 ce 1d c0>] - FlapPacketEvent: flapProcessor=FlapProcessor: seqNum=SeqNum: min=0, max=65535, last(current)=17, flapCommand=SnacFlapCmd: packet=SnacPacket type 0x1/0x21, flag1=0x80: 28 bytes (id=3178446837), flapPacket=FlapPacket (channel=2, seq=8386)

2007.08.24 01:53:23 OSCAR snac packet received: SnacPacketEvent: snacProcessor=ClientSnacProcessor: lastreqid=16, requests: 16, paused=false, snacPacket=SnacPacket type 0x1/0x21, flag1=0x80: 20 bytes (id=3178446837), snacCommand=ExtraInfoAck: blocks=[ExtraInfoBlock: type=0x2 (TYPE_AVAILMSG), extraData=<ExtraInfoData: flags=0x4 (FLAG_AVAILMSG_PRESENT), data=00 04 6b 68 6b 6a 00 00>, ExtraInfoBlock: type=0xd (null), extraData=<ExtraInfoData: flags=0x4 (FLAG_AVAILMSG_PRESENT), data=46 ce 1d c0>] - FlapPacketEvent: flapProcessor=FlapProcessor: seqNum=SeqNum: min=0, max=65535, last(current)=17, flapCommand=SnacFlapCmd: packet=SnacPacket type 0x1/0x21, flag1=0x80: 28 bytes (id=3178446837), flapPacket=FlapPacket (channel=2, seq=8386)

/code

the command seems to go trough, but something have to be wrong. also have tried to set icqstatus and status message in one command, but that result is the same.

i don’t understand was the 0xd block mean.

have you an hint?

notz

Oh man, it’s been a long time since I had to think about that. If I recall correctly, AIM and ICQ use the same mechanism for that. The caveat being that you have to announce yourself as a proper version of ICQ. (which we may actually not be doing, which may be the whole problem)

This site: http://iserverd1.khstu.ru/oscar/ provides a wealth of useful information about the OSCAR protocol. I don’t know off the top of my head though.

The contributor agreement isn’t anything major. I had to sign it to start working on this. Basically it’s just a agreement with Jive that you won’t come back out of the blue and say “you owe me money for those contributions!”. I believe Matt Tucker is who you would have to get in touch with to sign it. (basically if you submit large patches and such I am required to ask you of that to protect Jive) I can faciliate if you are interested.

Regardless, I still think we’re fine at the moment.