forums.smartfoxserver.com

Is there a way to count how many messages on average is being sent per second with SFS? With version 1.5.5 PRO I get alot of dropped messages and with version 1.6.1 PRO I get alot of disconnections. In both cases, they seem to tell me I am sending too many messages per second. I need help/tips how to measure this. Can anyone help?

Regards,
darnpunk

It doesn't depend much on SFS, it depends on your average network lag.
If the client pings the server and finds an average time of, say, 100ms you can easily calculate it:
In this case it would be 10 ( 1000 / 100 )

Usually a lag of 100ms is acceptable and there can be users who have higher lag values, so I wouldn't recommend sending more than 10-20 messages on each second. If you need more you probably should rethink how your update system works and make the client side smarter about how to deal with missing/lagging data.

Hi Lapo,

Thanks for the reply. In your formula, what does the value 1000 represent? I assume its 1000 = 1 second?

By the way, does version 1.6.1 and 1.5.5 have different policies on disconnecting users? I seem to experience mass disconnections on version 1.6.1 quite often.

I observe that if a particular client has dropped messages more than the specified value in config file <MaxAllowedDroppedPackets>, the console will display a message that says something like "Disconnecting user dropped packets too high, username, packets dropped". I understand that when that message shows up in the console, it is an intended disconnection by SFS as the user dropped more packets than allowed.

However, I face often mass disconnections like around 50 users at one go. I checked for a similar message as above but nothing showed up. I also checked the dropped packets and it did not increase. It happens during off peak hours too. I checked that the bandwidth is not fully used up and there was no lag experienced by me or other users.

As soon as I switched back to 1.5.5 the mass disconnection stopped. I use the same codes for both client and server side (extensions, game client etc). So I am wondering if anything is causing it?

Regards,
Arsyad

Thanks for the reply. In your formula, what does the value 1000 represent? I assume its 1000 = 1 second?

Correct. 1 second == 1000 milliseconds
Network lag is usually calculated in ms.

By the way, does version 1.6.1 and 1.5.5 have different policies on disconnecting users? I seem to experience mass disconnections on version 1.6.1 quite often.

Yes, since 1.5.9 we introduced a more fine-grained system to adjust the tolerance of dropped messages. More infos are available in chapter 2.2 of the docs. If the default settings don't work well for you, you can customize them in order to meet your application requirements.

It usually involves doing a couple of tests in order to find the proper client queue size.

In the next releases we'll be adding even more fine grained tools, to control message priorities.

However, I face often mass disconnections like around 50 users at one go. I checked for a similar message as above but nothing showed up.

If there's no such message it's not depending on dropped messages. Maybe those are idle clients?

As soon as I switched back to 1.5.5 the mass disconnection stopped.

This seems to confirm what I said before, they definitely sound like idle users. Make sure your maxIdleTime setting is not too strict.
Additionally turn the console logging from INFO level to FINER level, this will show you more debug infos and you'll probably see what happens when you get multiple disconnections.

Hi Lapo,

My maxIdleTime is set to 600 seconds. The mass disconnection happens to active users too . I set my maxIdleTime to 600 seconds in version 1.5.5 but this never happened. I allowed 20 dropped packets for each client but it does not help. Maybe I am sending/receiving too many bursts of messages? I'm going to set the console logging to FINER and see what I can catch from there.

Hi again,

I am still facing mass disconnection. You can say a total disconnect of every user. The server doesnt crash however.

I am still using SFS 1.5.5 PRO as the mass disconnection occurs least for it. The network graph shows a healthy frequency and the bandwidth never reaches its peak. We have a 10mbps line and graph shows about approximately half is used up.

I'm suspecting there could be other issues. Is there anyway I can monitor this problem closely? Currently, I restarted the server and at 1 day, max connection of 69 users, the dropped out-msg shows 1.0K (from Admin Tool). This is not a good sign isn't it

Any help is appreciated.

Regards,
darnpunk

A cause of multiple disconnections could be too heavy messaging with wrong configuration of the user queues.
If clients receive lots of messages per second they might not always be able to keep up.
If this happens the server keeps data in the queue and waits the client to be able to receive more data.
You can configure the queue to allow more data to be kept server side. When the queue is full the server will disconnect the client to avoid "exploding" (a.k.a OutOfMemoryError)

Another note: SFS 1.6.0 has introduced finer configuration of the client queues, upgrade is highly recommended (and free )
If you can't do it right now, at least update to 1.5.9

Hi Lapo,

Thanks for your reply. It could be poor coding or poor planning on my side. But it would be really great if I can somehow have a way to find out if my rate of messaging is stressing the server. Is there any way to do this or it's just monitoring? I am not able to decide what's too much.

It seems to me sending a chunk of data once is better than sending series of small parts. Anything to higlight?

Regards,
darnpunk

Hi Lapo,

I have updated the server to version 1.5.9 and there are improvements. There are lesser dropped messages but mass disconnection still occurs. I have set logging to FINER level and saw nothing that could lead to the disconnection. Is it possible to email the log file to your side for brief checks?

Regards,
darnpunk

From 1.5.9+ we have modified how we handle the message queues for each user.

<ClientMessagQueue>
<QueueSize>120</QueueSize>
<MaxAllowedDroppedPackets>1</MaxAllowedDroppedPackets>
</ClientMessagQueue>

A few advices:
1. monitor your bandwidth and make sure you are not running short of network resources, this could be a primary cause for this problem
2. disable the user count updates if you don't need them. They can consume significant network resources especially if you have hundreds of active rooms
3. adjust the settings in <ClientMessageQueue> of your config.xml

Check chapter 2.2 of the docs for all the details (Client message queue settings)

Hi Lapo,

Thanks for your advice. Message queues settings on the server side are set as follows:
1. Our allowed bandwidth is 10mbps and on average 6mbps is used up. I will continue to monitor this.
2. This has been disabled - uCountUpdate="false"
3. As above.

The dropped messages are lesser but disconnections still happens except lesser. From the logs, everything seems to be running fine prior to the mass disconnections.

I tried pinging the server (using codes from documentation) on peak hours and the calculated value is average at 11ms. It seems as if something is periodically disconnecting users for some unknown reason.

I do have multiples and continuous rooms being destroyed and created but I don't think those are "heavy"?

Regards,
darnpunk

have you run tests on your side?
do you get disconnected?
how often?

Have you tried with the AdminTool, the tool is never disconnected. So you should be able to leave it open for hours without problems (unless your computer goes into sleep/standby mode)

Hi Lapo,

I have run tests on my side and I get disconnected as often as the users. My AdminTool is disconnected as well in the event of mass disconnection. Does this give any clue?

Regards,
darnpunk

Yes a network problem

Hi Lapo,

Thanks for your prompt reply. It almost feels like instant messaging. By network problem, you mean its the line providing bandwidth?

Regards,
Arsyad