I want to test the Elgg social network through the creation of a large number of users, relationships and contents. Thus, I developed and implemented a plugin with forms for executions vary according to the input parameters. When I executed the tests (obviously), they take more or less time depending on the input parameters.
When the tests are simple, the time required is around 2-4 minutes. With the increasing the parameters, the tests take hours. And with high values, the tests may take 1 or 2 days to finish (or so I think).
I executed the tests from the browser in my admin space of my Elgg social network, but only simple tests worked. Then I executed the tests as script and I could understand the problem (the ram memory). Increasing the memory with this lines on my code,
set_time_limit(0); ini_set('memory_limit', '-1');
I could run heavier but still insufficient evidence. By the way, I have 8Gb of RAM. I read that PHP is not a good programming language for heavy executions. How can solved my problem? Using another programming language that calls some pieces of my PHP code, maybe? I do not know.
I hope my problem can be solved. Thank.
info@elgg.org
Security issues should be reported to security@elgg.org!
©2014 the Elgg Foundation
Elgg is a registered trademark of Thematic Networks.
Cover image by RaĆ¼l Utrera is used under Creative Commons license.
Icons by Flaticon and FontAwesome.
- ihayredinov@ihayredinov
ihayredinov - 0 likes
- Jose Alemany Bordera@jalemany1
Jose Alemany Bordera - 0 likes
- Nikolai Shcherbin@rivervanrain
Nikolai Shcherbin - 0 likes
- Jose Alemany Bordera@jalemany1
Jose Alemany Bordera - 0 likes
- ihayredinov@ihayredinov
ihayredinov - 0 likes
- Jose Alemany Bordera@jalemany1
Jose Alemany Bordera - 0 likes
- ihayredinov@ihayredinov
ihayredinov - 0 likes
- Jose Alemany Bordera@jalemany1
Jose Alemany Bordera - 0 likes
- ihayredinov@ihayredinov
ihayredinov - 0 likes
- Jose Alemany Bordera@jalemany1
Jose Alemany Bordera - 0 likes
You must log in to post replies.What exactly are you testing? How long it will take to create x entities in a single thread? That doesn't tell you anything. A large batch iteration will take a long time no matter what programming language/server environment you use.
The main problem is that PHP language does not manage the memory correctly (or my programming capabilities are not good). And with a PHP execution where are deleted around of 400000 entities, created 400000 new entities and done some calculations; the memory is taken completely.
My solution was divide the PHP script in different scripts and call them from a shell script. Now the memory works fine and the execution time was reduced.
----------------------------------------------------------------------
My new question is...How can I delete entities more efficiently?
Currently the operation that I do is the following...
However, to delete around of 400000 entities my computer take approximately 20 hours (too much). Any solution?
Remove it by 'pieces':
It remains slow. There is no function that allows me to delete them all at once?
You can delete them with a single SQL query, but it will leave behind artifacts - metadata, files etc. How did you end up with 400000 fake entities to start with?
I have created. I'm running tests.
What's the purpose of these tests? How many entities you can delete in a minute on your local machine?
The purpose of the tests is to check the extent of the information in a social network using synthetic data to understand which factors influence. So we are experimenting with different types of networks and probabilities of interactions. We're running tests and store the information that interests us.
I can eliminate about 500 entities per minute on my local machine.
If you need a clean data set for each test, just wipe the database and reinstall.
Okay, I'll try.
Thank you very much.