The "optimizer" application will again leave testing status some day during next week, starting from monday, 15.September.

read below an old (but updated) post on what this application does, and how you can opt out of running it (by default, you should be getting workunits of this application, unless it is in testing state and you did not volunteer to run testing workunits)

Only windows hosts will get work.
____________
Michael

This post was last updated on 12. of Sept. 08.
, maintained only for general information about the optimizer science application.

Watch this thread for news, it will leave testing status during next week (from 15. September)

At first, it will be run as a test application, meaning that only users who have \"run test applications\" and \"run optimizer application\" checked in their account settings (under malariacontrol.net preferences) will get work.



In addition, only windows hosts will get work.


Work units will take from 1 to 2 hours depending on the model parameters. Checkpointing is now being done, and progress will be indicated, but not in a very reliable way (so don't worry if it says 100% and then continues for a long time - max 2 hours for now.)

Calculation is done by a java program, contained within the standard boinc-\"wrapper\" application. You don't need to have java installed.. a java runtime environment is included in the application.
Deadlines: Three days.

The name \"optimizer\" for this application was chosen because the server side components are essentially a \"general use\" optimization framework to be used by scientists in our group to work on more specific questions. E.g. to fit simpler models for which the \"big\" malaria model would just not be what you want. The insights from those calculations will help us to improve the main malariacontrol application in the future.


On the science of the project:


To make quantitative predictions of malaria transmission, it is very important to know how long an infection lasts in an infected human. Because the longer it lasts, the more mosquitoes can get infected, the more infected mosquitoes you have, the more humans are being infected etc, etc, etc..
It may at first seem very straightforward to measure this: you just look when somebody gets infected, and then you keep taking blood samples until that person is not infected anymore.
Unfortunately, you only have a chance of about 50% percent to detect an infection, given that it is there. So you already have a problem: you don\'t know when the infection started, and you don\'t know exactly when it ended.
In addition: In areas of high malaria transmission people are very often infected with up to ten or more infections simultaneously... so you never know if what you\'re seeing is still the same infection or a new one..
Recently some work at our institute has used new dna-based methods (which allow distinction of different infections), together with a mathematical approach, to estimate the average duration of an untreated p. falciparum infection.


see Sama etal. 2006
(sorry, only the abstract is freely available to the public)


So far so good, this was an important step forward. The problem that remains is: how are the durations distributed? In other words: do all of the infections last exactly 200 days and then all of them stop? Or does an infection have a constant probability to disappear, which remains constant no matter how old the infection is? Probably none of the two is true, but we need to describe the shape of that distribution of durations somehow, in order to make sensible predictions.


for more on that, see Sama etal.2006b


That\'s almost where we want to go, except for one thing: the above paper measures the distribution in people living in the US who had never experienced malaria before. They were infected on purpose, to cure their syphilis (the method of choice at that time..) We don\'t know what the picture looks like in people living in areas of high transmission, with multiple infections at a time and after decades of being constantly infected...

Attempts to find a mathematical solution to this problem did not work out.. the equations become unsolvable. But there is a way out: instead of using equations, we can use individual based simulations, that means we simulate every single infection in a computer program, and see what parameters can best produce the data we have. The big drawback there is, this just takes too long to calculate on a single computer.
That\'s what we need you guys and girls for, and thanks a lot for making this possible!!

P.S.: Something about the data collection mentioned above, to prevent misunderstandings: There are strict ethical guidelines on how one is allowed to obtain such data. Since most malaria infections in high transmission areas don\'t cause any symptoms, being infected with malaria doesn\'t mean you are sick (because of acquired immunity). People who did have symptoms were of course given treatment.
____________
Michael

Thanks to all those who helped testing! After running the new version in testing status over the weekend, everything seems to be fine,no major issues, so we are leaving testing status now! It's always a great moment to "unleash" the full power of a boinc project and see the results coming back! Why am I so motivated, it's monday morning 9.20am? I think I have an interesting job, not always, but right now this is very exciting...:)
We are confident to have a batch of sensible results from this application by end november latest. Then it will be written up and published, preferably in an open access journal, so we can post a link here.

cheers


____________
Michael

Michael

Where can I look up data/information on this optimiser client, as I run Win XP on both the rigs I use for Malaria?

Mabe I am a little thick, but I assume "optimiser" refers to a Malaria project client which uses the specialist Intel instruction sets (like MMX, SSE, SSE2, SSE3, SSSE3x or SSE4.1)?

I would love to volunteer to test if this is the correct view, or test when I understand what I can do to contribute, as well as using the stock client.

I did not look around too deeply before I posted this, so I have some reading to carry out.

Hopefully my questions will be answered during this reading, but if there is any short answers you might post I would apprecialte the understanding (both for me and from you).

good! thanks keith for helping..
____________
Michael

..does not sound good..

that it shows 100% and then continues is not bad by itself.. that indication of percentage is just approximate.. but not for 24 hours?
Maximum 2 hours (for the batch that went out now..)
please abort them by hand, and if possible, could you post a link to the workunits? that would help to find it.. this really shouldnt happen..

____________
Michael

I noticed the same issue and these WUs took longer than 2h to complete sitting at 100% for most of the time, for too long even with the CPU throttled at 17%:

• 
  
• 35406636
  
• 35408212
  

HTH

if you compare "cpu time" on the pages above, with the actual time it took, and take into account the trottling, it seems ok to me..

OK. I did have one system error out most WUs though...

TIA

I guess you're after a medal for the worst BOINC application?

Each application task opened a GUI window asking me to install JAVA - great thing on an unattended cruncher - CPUs stuck for hours.

After confirming the installation, they still crashed within no time, no reason given, just file transfer errors.

It should at least have a warning ("Java required") behind the OptIn selection.

update: optimizer is going back to testing status, at least for a few days.. it really looks as if certain client versions don't do well (some extremely not well) with this application). Trying to figure out if, and which ones.. stay tuned
____________
Michael

Looks like optimizer v1.55 has fixed the upper cannot equal lower problem with the opt_27_* tasks.

Just returned my first one with the new version, only to get a too many total results validation failure (WU 13134127). The 7th task was sent out 4 hours after the 6th was returned. If the 7th task was created when the 6th was returned it shouldn't have been. If it already existed its state should surely have been set to server state Unsent and outcome Didn't need when the 6th was returned.

version 1.59 is out.
It brings a change in that it avoids some redundancy in the calculations, i.e if two persons only differ by age, but are otherwise identical, we only have to calculate the older one of them, and write down the intermediate results... then look the result for the younger ones up in that table.. something along those lines.

This will bring higher precision of results (or shorter calculation times).. , hopefully.. we are still testing what the benefit is.
However it is clear, that this strategy can only be applied in some special cases of models that you want to fit - this doesn't bother us at the moment, because all we wanna do is fit exactly those special cases..


In addition, it collects debugging information about the "Jsmooth" launcher for the java app (not in the standard error though, it comes back to us as a second output file and does not appear in your web-based interface to the result-database). With this information we hope to track down error -161, which is still around, though not very common, but yet unsolved..

cheers
____________
Michael

Example WU that hung @ 100% and completed after restart:
wu_119_501_306221_0_1225446497_0

update: The optimizer workunits have now (with version 1.60) an average duration of 2 hours, and a maximum duration of 4hours (compared to avg 1h, max 2h before).

We are doing this to improve our fitting process.. that's why we are trying different settings. We can't really test this in the "testing state", because we wouldn't have enough hosts for that.. but even though one could call it "testing" we're not really testing if the software runs - it does run. We are now just fiddling with the parameters to fine-tune it.

you betcha?
____________
Michael

sorry for the late reply..
I think, this occured when old workunits with different configurations were still in circulation when the new app-version was already out.


But recently we are getting errors like this:

Exit status -177 (0xffffff4f) ERR_RSC_LIMIT_EXCEEDED

CPU time 593.1563

<message>
Maximum disk usage exceeded
</message>


even though maximum disk usage is at
<rsc_disk_bound>750000000</rsc_disk_bound>

750 megabite.. and that is way above what our app needs..

did anybody out there experience this error and has deeper insigths?

thanks

____________
Michael

did anybody out there experience this error and has deeper insigths?

Not personally, but the likely cause has been reported here.

ok, the bug is fixed, version 161 is out, and we're slowly starting to send out workunits again...
____________
Michael