forums.smartfoxserver.com

I'm sending webcam imagery as a sequence of byte arrays, each containing one jpg encoded frame. We had previously been doing this with Unity's built-in RPC functionality, so we know it works. Each frame is around 13kB.

When I tried to send 30 frames / second in two directions between two connected clients, I immediately got all sorts of exotic errors in the SmartFox server log and would be booted in under a second. When I slowed down the stream, from one client only at around 10-20 frames per second it would work for a while longer, but within a few seconds I would get the following error (it's slightly different every time):

10 May 2012 | 20:51:19,658 | WARN | SocketReader | protocol.binary.BinaryIoHandler | null
10 May 2012 | 20:51:19,897 | WARN | SocketReader | v2.protocol.SFSProtocolCodec | Error deserializing request: java.lang.IllegalStateException: Invalid SFSDataType. Expected: 18, found: -70
10 May 2012 | 20:51:19,898 | WARN | SocketReader | protocol.binary.BinaryIoHandler | Illegal request size: -1161512870 bytes, from: ( User Name: User 19, Id: 19, Priv: 0, Sess: 137.75.201.137:58163 )
10 May 2012 | 20:51:19,987 | WARN | SocketReader | protocol.binary.BinaryIoHandler | null
10 May 2012 | 20:51:19,988 | INFO | SocketReader | bitswarm.sessions.DefaultSessionManager | Session removed: { Id: 21, Type: DEFAULT, Logged: Yes, IP: 137.75.201.137:58163 }
10 May 2012 | 20:51:19,988 | INFO | SocketReader | v2.entities.SFSZone | User: User 19 was disconnected.

This was after a few dozen to a few hundred basically identical requests, where the only difference is the actual contents and a slight variation in size of the byte array. Any idea what's going on?

BTW, a bit more information:
I'm passing the calls through an extension which at the moment simply grabs the user list for the zone, strips off the sender, and sends the exact message to the remaining users (in this case there is one remaining user). The reason to use an extension is for when we go to more users. Each user will select a webcam to have maximized, and frames from that webcam will come through at full resolution. Other webcams will be downsampled on the server to 'preview' images and send along with the main image.

If anyone has better ideas about how to send webcam imagery, I'm open to any ideas. Unfortunately, I haven't found any good .NET streaming libraries that will work with Unity for video feeds, which is why we're sending jpg images instead.

Another issue I see (that may be related) when sending webcam frames at a much higher rate is:

10 May 2012 | 22:31:03,824 | WARN | SocketReader | protocol.binary.BinaryIoHandler | null
10 May 2012 | 22:31:03,824 | INFO | SocketReader | bitswarm.sessions.DefaultSessionManager | Session removed: { Id: 22, Type: DEFAULT, Logged: Yes, IP: 137.75.201.137:59125 }
10 May 2012 | 22:31:03,824 | INFO | SocketReader | v2.entities.SFSZone | User: User 20 was disconnected.
10 May 2012 | 22:31:03,825 | INFO | SocketReader | v2.api.SFSApi | User disconnected: ( User Name: User 20, Id: 20, Priv: 0, Sess: 137.75.201.137:59125 ) , SessionLen: 3030
10 May 2012 | 22:31:03,825 | WARN | SocketReader | protocol.binary.BinaryIoHandler | Incoming request size too large: 637415363, Current limit: 8000000, From: ( User Name: User 20, Id: 20, Priv: 0, Sess: 137.75.201.137:59125 )

The 'current limit' in that message is equal to "SmartFox.MaxMessageSize". The only documentation I could find on this is http://docs2x.smartfoxserver.com/api-do ... 916c08.htm, which isn't super helpful. I have no idea what this actually means - 80000000 what? In what amount of time or number of messages? Where is the server getting the '637415363' number? This is after sending maybe 30-300k worth of binary, so I can't imagine where 637M is coming from.

There is a discussion at http://forums.smartfoxserver.com/viewto ... 18&t=13566 that mentions issues with the socket server on larger binary sizes and suggests HTTP instead. I'll look into this, though I don't see why it would solve the IllegalStateException I'm getting.

Well... sending pictures at 30fps is not for everybody, you can rest assured
Webcams don't work that way, they use tiling and send only the portions that change, over UDP, not TCP.
Even for realtime games we don't recommend to send more than 12-15 packets/second to avoid overloading the client and server. The reason is simple in order to sustain 30fps over the network you need a lag < 33ms... which is not very common. You are lucky if you get a steady 50-60ms. Normally it's around 80-100 and when the networks slows down a bit it grows.

With this said you didn't specify which server version you are using, so I'd recommend to update both clients and server if you havent' done it recently. (We just release an update last week.)

Thanks
p.s. = I removed the double post, let's keep the conversation here so that Thomas (the Unity guy and moderator) can read too.

Thanks Lapo,

What we're working on isn't a game, it's a data visualization and collaboration tool. Beyond the webcam imagery, the other information that we're sending is tiny and not dependent on being realtime. We certainly recognize that there are better ways to send webcam imagery, but for testing and demoing our work we only need 'good enough.' We're working on very low-lag and high-speed networks, so the send rate isn't an issue for us.

We are using the very latest server version (2.1.0), downloaded a couple days ago.

-Julien

What we're working on isn't a game, it's a data visualization and collaboration tool.

I understand, however what I said about lag and best practices remains the same, regardless of the type of application.
Are you running these communications in a local network or over the internet? I'd be interested in knowing what happens in a LAN or what happens over the internet at a slower rate (e.g. 12fps)

Could you send us the zipped config/ folder and the logs with those errors?
You can use our contact details to send us the material.

Thanks

This seems usage / server related. But poke directly, via Marco or here if you need some attention on the client side.

/Thomas

Hello,

I'm resurrecting this thread because I'm having a similar problem. I'm using SFS2X 2.9.0, AS3 API and almost exclusively byte arrays to and from the server all via UDP.

My error is the classic invalid SFSDataType, happening sporadically (may not happen for an hour, may happen within 5 minutes of running):

[Fault] exception, information=Error: Invalid SFSDataType. Expected: 18, found: 120

I'm also using SFSArrays from the server to the client only, to just help me index the bytearrays.
Client and server send and receive packets at 20fps. I'd like to crank up the client to server sends to 30+ fps, but if the previous posts are any true, increasing that will only lead to more of these issues.
This happens on localhost as well as on the internet.

So here's my questions:
1. If I get rid of the SFSArrays altogether and just use one SFSObject containing the bytearray will this issue go away?
2. Is there a way to not have the client API throw the error or a way to catch it so I can allow my logic to deal with a bad transmission?
3. Could this issue be caused by a malformed byte array created by the server? The process is always the same:
Thank you!

It is possible for UDP packets to arrive fragmented on the other end. Typically they should be discarded altogether but your mileage may vary.
UDP is very sensitive to the size of packets. What is the average size of the data being sent?

What client platform are you using?

Thanks

Whoopsy, I just realized this is the C# forum. I'm using the AS3 client.

The client -> server payload varies between 15-25 bytes (that includes the bytearray int32/length).
The server->client payload wildly varies since it's always a delta. Could be as little as 30-50 bytes and as much as 400-500 bytes.