Hi Experts!
One of our smart fox based application is almost in a finishing stage and We expect very vast number of users concurrently in the virtual world application developed.In-order to do load/scalability test, we need to know which testing tool we should prefer for testing ? Is there any open source tools which would help us to test scalability testing effectively ? In the initial phase we would like to test with 100 concurrent users. In Virtual world project there are so many different activities like ‘Avatars walking, collecting objects ,sending messages, taking photos, playing games etc.. are included. it will be really nice if we are suggested with a good testing tool preferably open-source.
Kindly reply ASAP.
Regards,
Deepa
Scalability Testing Tool for Smartfox based application
-
deepamenon
- Posts: 3
- Joined: 07 Jun 2010, 11:09
This topic has been discussed several times in this board, so you might want to search for those previous threads.
Anyways this is what we usually recommend, in short:
Building a basic tool that simulates many connections and sends/receives messages isn't particularly complex, the difficult part is making it interact correctly with your application as a real user/player would do.
A possible high-level solution is to use our API for Actionscript 3.0 or Java. We've recently done a few tests using an Actionscript3 stress client and we were able to run around 4-500 concurrent clients in a single Flash Player instance, we didn't expect that much!
By using multiple Flash Player instances and multiple client machines you can easily simulate thousands of concurrent clients.
All you need to do is create a client that logs in, joins the right room (maybe with the help of the server side), and starts acting as if it was playing.
Once you have this test class, you can build a "Test Manager" class that manages the generation of hundreds of these clients.
Finally we suggest multiple strategies for testing an application: automated tests can be used to hammer spefic areas of the application that could be potentially problematic (i.e. database access, other server-side heavy jobs) and also a live public-beta for testing overall stability and efficiency of the service.
This allows to receive a lot of feedback from real users which in turn helps discovering other problems that usually slip to the developer's eyes
Hope it helps
Anyways this is what we usually recommend, in short:
Building a basic tool that simulates many connections and sends/receives messages isn't particularly complex, the difficult part is making it interact correctly with your application as a real user/player would do.
A possible high-level solution is to use our API for Actionscript 3.0 or Java. We've recently done a few tests using an Actionscript3 stress client and we were able to run around 4-500 concurrent clients in a single Flash Player instance, we didn't expect that much!
By using multiple Flash Player instances and multiple client machines you can easily simulate thousands of concurrent clients.
All you need to do is create a client that logs in, joins the right room (maybe with the help of the server side), and starts acting as if it was playing.
Once you have this test class, you can build a "Test Manager" class that manages the generation of hundreds of these clients.
Finally we suggest multiple strategies for testing an application: automated tests can be used to hammer spefic areas of the application that could be potentially problematic (i.e. database access, other server-side heavy jobs) and also a live public-beta for testing overall stability and efficiency of the service.
This allows to receive a lot of feedback from real users which in turn helps discovering other problems that usually slip to the developer's eyes
Hope it helps
-
deepamenon
- Posts: 3
- Joined: 07 Jun 2010, 11:09
Scalability Testing
Thanks for the your earliest reply.
Based on your reply I'm considering following 3 choices
1. One API developed by smart fox for Action script or Java
2. using external tools for testing specific areas of the application (Ex. Database access, other server side jobs)
3. Public beta testing.
or
1. we are thinking about an open source tool like Open STA .
When i discussed about the case 1 , developer reported that its the application they have developed and it cannot be considered as an automated tool.
I'm into testing and i would like to know the exact automated testing tool we can use. You had mentioned about using another API in your previous message . Can you please explain more about it.
Our application has the facility to login as avatars and access site features like walking , picking objects ,playing games etc.
Can we consider Open STA for testing or any other tools you can suggest ..
I'm new to the concept so kindly explain in detail.
Based on your reply I'm considering following 3 choices
1. One API developed by smart fox for Action script or Java
2. using external tools for testing specific areas of the application (Ex. Database access, other server side jobs)
3. Public beta testing.
or
1. we are thinking about an open source tool like Open STA .
When i discussed about the case 1 , developer reported that its the application they have developed and it cannot be considered as an automated tool.
I'm into testing and i would like to know the exact automated testing tool we can use. You had mentioned about using another API in your previous message . Can you please explain more about it.
Our application has the facility to login as avatars and access site features like walking , picking objects ,playing games etc.
Can we consider Open STA for testing or any other tools you can suggest ..
I'm new to the concept so kindly explain in detail.
Hi,
I don't really think OpenSTA can be used for this. It is a web load testing tool, it uses HTTP protocol while SmartFoxServer uses persistent connections and a custom protocol.
As regards using the API. That's the best "high level" solution.
You can create different client applications that simulate the interaction of a client with your application. Example: connect, login, send chat messages, send game requests etc...
All you need to do is creating a "TestManager" that is able to start many of these clients at the same time to perform the load testing.
The approach can extended with reporting from each simulated users, gathering statistics from the server side and a lot more.
Cheers
I don't really think OpenSTA can be used for this. It is a web load testing tool, it uses HTTP protocol while SmartFoxServer uses persistent connections and a custom protocol.
As regards using the API. That's the best "high level" solution.
You can create different client applications that simulate the interaction of a client with your application. Example: connect, login, send chat messages, send game requests etc...
All you need to do is creating a "TestManager" that is able to start many of these clients at the same time to perform the load testing.
The approach can extended with reporting from each simulated users, gathering statistics from the server side and a lot more.
Cheers
-
deepamenon
- Posts: 3
- Joined: 07 Jun 2010, 11:09
Automated Tool
Thanks Again
Here my problem is I have to test Avatars activities(like picking objects,walking,playing games) and Animal activities( Moving in the forest,eating,drinking etc) in the system not just login and send requests etc.
Is it possible to test movements of Avatars also using the API . If we use any external tool which one would you suggest .. Can we think about load runner for testing the scalability of our application? . I hope there wont be any open source tools we could make use for this application so can we use load runner ? Right now thinking of testing 100 users but in real time it will be much more . We expect around 25,000 plus users for future . We could use public beta testing but in-order to test the application for long run which automated tool should be considered.
Kindly reply.
Regards,
Deepa.
Here my problem is I have to test Avatars activities(like picking objects,walking,playing games) and Animal activities( Moving in the forest,eating,drinking etc) in the system not just login and send requests etc.
Is it possible to test movements of Avatars also using the API . If we use any external tool which one would you suggest .. Can we think about load runner for testing the scalability of our application? . I hope there wont be any open source tools we could make use for this application so can we use load runner ? Right now thinking of testing 100 users but in real time it will be much more . We expect around 25,000 plus users for future . We could use public beta testing but in-order to test the application for long run which automated tool should be considered.
Kindly reply.
Regards,
Deepa.
Walking, collecting objects, playing games are all server requests, aren't they? When a user walks from point A to point B it sends a request to the server in order to update all the other clients.Here my problem is I have to test Avatars activities(like picking objects,walking,playing games) and Animal activities( Moving in the forest,eating,drinking etc) in the system not just login and send requests etc.
You seem to be making some confusion. An "external tool" and the API work exactly the same way from the perspective of the server. They are external applications that speak the SFS protocol and that send meaningful requests.
We suggest to build a stress test application using the API because it simplifies the process by orders of magnitude. You already have the protocol, requests and events already at your hand. All you have to do is building the logic of the test simulating the various activities of a client (walking, chatting, picking objects, etc...)