Best way to save chat logs?

isointeractive · Post by **isointeractive** » 26 Sep 2010, 21:56

Hello everyone.

We have built a number of projects with SmartFoxServer and like to ask others what they think the best way to save chat logs that do not cause major overhead with the servers or databases.

Mostly getting opinions on best practices.

Many clients want a stored history of the entire chat log. There are a number of ways to do it. The questions is what makes the best sense in a fairly large virtual world.

We believe that saving individual chat logs to the database per log-out is a very bad idea because you always want to limit the amount of read and write info. And it if is going to a database it could get full very fast. The amount of information is just to much.

So we think if there is a way to send the chat log to a txt or similar format from the actual server and creating a new file ever hour there would be a way to compact and store these files (and retrieve) via stored procedure or something. Also we can cross reference times and users to see who was in what room at what time so you can get a sense of what the chatter is all about.

Anyway. Any ideas or suggestions?

BigFIsh · Post by **BigFIsh** » 26 Sep 2010, 23:16

Well,

I haven't implemented the system to save chat logs, but I had a rough plan how to:

- Save the chat via zone extension, by intercepting the public messages - and only store related messages for that user in that room into a local array (via userId)
- Upon logging out or disconnection, get the last, let's say.. 200 lines, from the array and store it in h2database - overriding what's ever there (i.e. each user would only have one row to store chat history). Saving to h2database would be very fast as it's embedded to the server, it's almost like storing it locally in an array via zone extension. It's purely made in Java.
- When the user logs in again, only send the previous chat history if requested by the user. (there would be a button such as [Get previous chat history])

isointeractive · Post by **isointeractive** » 28 Sep 2010, 13:56

Interesting. have not really use the built in SFS database.

The issue is the history is limited. And every user will have to write to the database individually. That is a lot content.

I would think that it would be better to record the entire chat log in one place. Maybe dividing it up in to time segments. You can always do a request via id/username, room number and time range and server/zone.

The zone extension is a great idea. Just pull and push to something separate from the database. I guess I simply could be a separate database altogether although I really do not like that idea. Or maybe some sort of raw text format that can be accessed at some request?

Thanks
-Troy

Jade · Post by **Jade** » 09 Nov 2010, 18:05

Anyway. Any ideas or suggestions?

I would be interested to know what your solution to this has been. I've been talking it over with some of my friends and we've all come with various unique solutions. Have you decided on a way to do it yet and if so how is it working out for you?

Some ideas I had were:

1) Queue the messages then write them to the database after every x number of minutes

2) Use a third party XMPP software

3) Create a program running at the server level that captures all SFS message events and writes to log files every few hours/days

isointeractive · Post by **isointeractive** » 09 Nov 2010, 20:29

Well there are two ways to really do it.

The first is the easiest way but also takes up the most amount of space.

1. Every time there is a chat log for a user it can be saved on the client side. This is good because you can instantly follow the conversation, no parsing needed but not really secure and can be faked. Once the user leaves the chat the info is put in the database associated to there account. I would suggest to separate a foreign key to a table with that info or even a separate database. The bad thing is that is a lot of data and hits the server a bit. Have lots of users and not enough processor power you are in for a shock.

2. Another way that may be more efficient and made for larger systems but may take longer to setup is to create an extension that logs the entire logfile. That is easy enough. Save that every hour (or some smaller increment of time) via stored procedure. save as date time stamp for the name . txt

Store that in a txt file within a certain secure folder. Not a database.

Create a stored procedure that cleans out files (or moves them) with the date that is X old. Depends on COPPA laws.

To access them it is a request that can pull conversations based on date rage and room ID and proved within a certain time period.

For example a parent says - my son was talked to by this bad person during this day at this time EST. So you pull the logs from 30 mins before and after based on the time stamp and parse out the all the rooms the user was in within that time period. And do that via stored procedure.

Jade · Post by **Jade** » 09 Nov 2010, 20:44

I haven't been able to find anything in COPPA that dictates if you have to store logs and for how long. It would be easier for me to not bother with them completely. However my members are predominantly girls and I worry that I'll end up in a firestorm somewhere down the road if I don't have records of their conversations.

isointeractive · Post by **isointeractive** » 09 Nov 2010, 21:35

Well you can store them indefinitely with the larger more robust system.

You can see some of our games on the portfolio section at:

http://www.isointeractive.com

Thanks

-Troy