Mappredictor 5.20


Advanced search

Message boards : Malaria Control : Mappredictor 5.20

AuthorMessage
Profile maire
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: Nov 7 05
Posts: 439
Credit: 118,258
RAC: 0
Message 3783 - Posted 13 Sep 2007 14:53:19 UTC

    Last modified: 13 Sep 2007 14:54:10 UTC

    We\'re getting ready to run another batch of mappredictor workunits. The first workunits will be sent out starting tomorrow Friday 14th. The new workunits are very short (around 1 min on an avarage PC), mainly because the prediction focus on a much smaller geographical range (see below). Otherwise the same as usual applies for the mappredictor: The application makes use of BOINC\'s wrapper approach for legacy applications, therefrore there is no checkpointing and no feedback from the application about progress. As usual you can opt out by changing your project preferences.
    Here\'s what the scientists say about the new version:

    The purpose of this application is to produce a smooth map of malaria risk in Mali taking into account two important factors:
    1) Malaria risk data are correlated in space since common environmental risk factors may influence the malaria risk similarly in locations within the same geographical area. In this application we modeled the spatial correlation as a function of the distance between locations and dependent on locations (non-stationarity).
    2) The relation between malaria risk and environmental factors may not be linear; therefore we use non-parametric regression models to quantify this relation.

    This application is similar to the previous one (West Africa) and it is used mainly to validate the previous results (compare the two maps of Mali).

    Nick

    ____________
    Nicolas Maire
    Swiss Tropical and Public Health Institute
    http://www.swisstph.ch

    Profile The Gas Giant
    Avatar
    Send message
    Joined: Mar 7 06
    Posts: 1214
    Credit: 3,719,032
    RAC: 1,397
    Message 3822 - Posted 17 Sep 2007 10:07:25 UTC

      Has anyone seen any of these yet?

      Marky-UK
      Send message
      Joined: Mar 7 06
      Posts: 4
      Credit: 78,672
      RAC: 0
      Message 3823 - Posted 17 Sep 2007 12:42:55 UTC - in response to Message 3822.

        Last modified: 17 Sep 2007 12:43:55 UTC

        Has anyone seen any of these yet?

        They seem to be available now. Run time on an Athlon 1500 is ~25 seconds (and there\'s a daily limit of 500, which that PC will reach in under 4 hours).

        Profile maire
        Volunteer moderator
        Project administrator
        Project developer
        Project scientist
        Send message
        Joined: Nov 7 05
        Posts: 439
        Credit: 118,258
        RAC: 0
        Message 3825 - Posted 17 Sep 2007 13:39:26 UTC

          Last modified: 17 Sep 2007 13:39:41 UTC

          We\'ve started sending them out this morning, phasing in slowly as usual. The daily quota has been increased, this should not be limiting any longer.
          There are a total of 95590 workunits. Visit the server status page and have a look at the batch progress monitor! The image is updated every 5 minutes.
          Nick

          ____________
          Nicolas Maire
          Swiss Tropical and Public Health Institute
          http://www.swisstph.ch

          Ageless
          Avatar
          Send message
          Joined: Jun 29 06
          Posts: 261
          Credit: 150,583
          RAC: 1
          Message 3826 - Posted 17 Sep 2007 17:18:44 UTC

            I had one. Quick little buggers. :)
            ____________
            Jord.

            BOINC FAQ Service

            Profile The Gas Giant
            Avatar
            Send message
            Joined: Mar 7 06
            Posts: 1214
            Credit: 3,719,032
            RAC: 1,397
            Message 3828 - Posted 17 Sep 2007 20:37:14 UTC

              YAY. I got some! And they are very quick little buggers! ;)

              Redbull
              Send message
              Joined: Mar 8 06
              Posts: 28
              Credit: 307,058
              RAC: 0
              Message 3833 - Posted 18 Sep 2007 9:21:53 UTC

                Im getting quite a few.

                I have taken to aborting workunits which are normal ones for now to give you guys some tests.

                Also I aborted some of the map predictor work units and it seemed to abort fine.
                ____________

                Profile The Gas Giant
                Avatar
                Send message
                Joined: Mar 7 06
                Posts: 1214
                Credit: 3,719,032
                RAC: 1,397
                Message 3836 - Posted 18 Sep 2007 10:49:30 UTC - in response to Message 3828.

                  Last modified: 18 Sep 2007 10:52:26 UTC

                  YAY. I got some! And they are very quick little buggers! ;)

                  So quick in fact that it clogs up the client_state.xml file as it needs to download so many to keep the wu cache at the desired level. Let\'s see
                  - 30 sec per wu
                  - 0.5 day cache = 21600 seconds
                  - 2 cores
                  - required work = 43200 seconds
                  = 1440 wu\'s!

                  I\'ve reduced my cache setting to 0.25 days to reduce the load on boinc.exe having to write 4MB of data (ie client_state.xml) every 15 seconds or so. In the last 14hrs boinc.exe has consumed 2hrs of cpu time on my 3.2GHz P4. Hmmmm, surely if the wu\'s only take 30sec then they could be run internally within reasonable time limits.

                  Live long and BOINC!
                  ____________
                  Paul
                  (S@H1 8888)

                  Profile The Gas Giant
                  Avatar
                  Send message
                  Joined: Mar 7 06
                  Posts: 1214
                  Credit: 3,719,032
                  RAC: 1,397
                  Message 3837 - Posted 18 Sep 2007 12:16:16 UTC

                    Gotta hate it when the power goes out in the middle of an edit....

                    0.5 day cache = 43200 secs
                    2 cores
                    required work = 86400 secs
                    = 2880 wu\'s

                    Which makes for a very large client_state.xml

                    Ageless
                    Avatar
                    Send message
                    Joined: Jun 29 06
                    Posts: 261
                    Credit: 150,583
                    RAC: 1
                    Message 3838 - Posted 18 Sep 2007 12:50:36 UTC - in response to Message 3836.

                      - 30 sec per wu

                      Throttle your CPU. ;-)

                      I\'ve got mine throttled at 65% and then they run in 1m 13s (average).
                      But run they do. Good thing there\'s no graphics included.
                      ____________
                      Jord.

                      BOINC FAQ Service

                      Tom Philippart
                      Send message
                      Joined: Jun 25 06
                      Posts: 29
                      Credit: 220,888
                      RAC: 0
                      Message 3841 - Posted 18 Sep 2007 16:48:50 UTC - in response to Message 3838.

                        These small WUs are getting a big problem for me, Boinc.exe takes up 100% CPU Time, because Boinc downloads and uploads so many WUs to get enough work for the 2 cores of my CPU. It takes around 11 seconds to crunch a single unit!!
                        ____________

                        Profile maire
                        Volunteer moderator
                        Project administrator
                        Project developer
                        Project scientist
                        Send message
                        Joined: Nov 7 05
                        Posts: 439
                        Credit: 118,258
                        RAC: 0
                        Message 3845 - Posted 19 Sep 2007 7:51:38 UTC - in response to Message 3841.

                          These small WUs are getting a big problem for me, Boinc.exe takes up 100% CPU Time, because Boinc downloads and uploads so many WUs to get enough work for the 2 cores of my CPU. It takes around 11 seconds to crunch a single unit!!


                          We still have about 18\'000 waiting to be sent for the first time, and expect the last one to go out later today.
                          If the short map workunits are a problem, please opt out using the project preferences.
                          Thanks
                          Nick
                          ____________
                          Nicolas Maire
                          Swiss Tropical and Public Health Institute
                          http://www.swisstph.ch

                          Ageless
                          Avatar
                          Send message
                          Joined: Jun 29 06
                          Posts: 261
                          Credit: 150,583
                          RAC: 1
                          Message 3848 - Posted 19 Sep 2007 10:35:02 UTC

                            Good thing I only work half days, as this morning, during the download of a lot of the small ones, my computer hung. Nothing spectacular about that, but worrisome in some sense anyway. No results that hung, though, so it was probably \'about time it hung again\'.

                            I\'ve reduced my connect to + additional days to 0.02 plus 0.02 and will continue testing.
                            ____________
                            Jord.

                            BOINC FAQ Service

                            Ageless
                            Avatar
                            Send message
                            Joined: Jun 29 06
                            Posts: 261
                            Credit: 150,583
                            RAC: 1
                            Message 3852 - Posted 19 Sep 2007 12:33:10 UTC

                              Last modified: 19 Sep 2007 12:39:46 UTC

                              Hmmm, let\'s not run this mappredictor anymore. It rebooted my system while I was away and since BOINC runs as a service, it continued to run on the login screen, then hung my system again.

                              I\'ll try to coerce/run the cache dry and have already changed my preferences to not run predictor again. Sorry, perhaps that it works better on a more stable system.

                              Edit: I managed to post the above, then my system hung again. If you don\'t mind, I\'ll be aborting all map predictors.
                              ____________
                              Jord.

                              BOINC FAQ Service

                              Profile maire
                              Volunteer moderator
                              Project administrator
                              Project developer
                              Project scientist
                              Send message
                              Joined: Nov 7 05
                              Posts: 439
                              Credit: 118,258
                              RAC: 0
                              Message 3916 - Posted 25 Sep 2007 14:23:23 UTC

                                We received the last result for this batch last night. I\'ve passed all the data on to the scientists of this project to analyze.
                                Many thanks for your help!
                                Nick
                                ____________
                                Nicolas Maire
                                Swiss Tropical and Public Health Institute
                                http://www.swisstph.ch

                                Profile The Gas Giant
                                Avatar
                                Send message
                                Joined: Mar 7 06
                                Posts: 1214
                                Credit: 3,719,032
                                RAC: 1,397
                                Message 3925 - Posted 26 Sep 2007 4:27:18 UTC - in response to Message 3916.

                                  We received the last result for this batch last night. I\'ve passed all the data on to the scientists of this project to analyze.
                                  Many thanks for your help!
                                  Nick

                                  Wow, that took a long time! I haven\'t seen one of those wu\'s for many days. To speed up the response in the future is it possible to target specific hosts known to be reliable to run any wu\'s that are stragglers?
                                  ____________
                                  Paul
                                  (S@H1 8888)

                                  Profile maire
                                  Volunteer moderator
                                  Project administrator
                                  Project developer
                                  Project scientist
                                  Send message
                                  Joined: Nov 7 05
                                  Posts: 439
                                  Credit: 118,258
                                  RAC: 0
                                  Message 3930 - Posted 26 Sep 2007 8:06:39 UTC - in response to Message 3925.


                                    Wow, that took a long time! I haven\'t seen one of those wu\'s for many days. To speed up the response in the future is it possible to target specific hosts known to be reliable to run any wu\'s that are stragglers?

                                    Yes, turnaround time is an issue in volunteer computing. We are now using a scheduler that resends error or no_reply results to reliable hosts only. This seems to make a big difference, but there is still the rare case where more than one resend is necessary.
                                    The last result to come back, for example, was for a workunit that was sent out at the very beginning (Sept 17th), and then resent on the 21th and 24th.
                                    Nick
                                    ____________
                                    Nicolas Maire
                                    Swiss Tropical and Public Health Institute
                                    http://www.swisstph.ch

                                    Post to thread

                                    Message boards : Malaria Control : Mappredictor 5.20


                                    Return to malariacontrol.net main page


                                    Copyright © 2013 africa@home