short deadlines and not-always-connected users


Advanced search

Message boards : Number crunching : short deadlines and not-always-connected users

AuthorMessage
Mikus
Send message
Joined: Jan 4 07
Posts: 13
Credit: 392,389
RAC: 641
Message 3630 - Posted 25 Aug 2007 21:58:27 UTC

    I run off-line, and connect only occasionally. By participating in the malaria project, I feel I\'m contributing to the sum of scientific knowledge. As long as malaria WUs have deadlines over three days, I\'m able to report their results on time.

    I just received a malaria WU that has a deadline of less than 28 hours from when it was downloaded to my system. The only way to __ensure__ that such short-deadline WUs will be reported on time would be for me to connect several times *every* day. I cannot guarantee that.
    .

    Profile The Gas Giant
    Avatar
    Send message
    Joined: Mar 7 06
    Posts: 1214
    Credit: 3,710,893
    RAC: 1,076
    Message 3633 - Posted 25 Aug 2007 23:37:22 UTC - in response to Message 3630.

      Last modified: 25 Aug 2007 23:38:10 UTC

      I run off-line, and connect only occasionally. By participating in the malaria project, I feel I\'m contributing to the sum of scientific knowledge. As long as malaria WUs have deadlines over three days, I\'m able to report their results on time.

      I just received a malaria WU that has a deadline of less than 28 hours from when it was downloaded to my system. The only way to __ensure__ that such short-deadline WUs will be reported on time would be for me to connect several times *every* day. I cannot guarantee that.
      .

      Me neither. On my \"work\" laptop, BOINC does not connect to the internet while at work and with the way the BOINC schedular works with these short deadlines, I will not be able to keep the requisite cache to ensure it doesn\'t run out of work, unless the current wu_42 run will be over within the next 20hrs.....

      I thought all this had been discussed before and short deadlines agreed to be utilised since it does play havoc with many volunteers crunching ability!

      Live long and BOINC!

      ____________
      Paul
      (S@H1 8888)

      Profile mikey
      Avatar
      Send message
      Joined: Mar 23 07
      Posts: 4682
      Credit: 5,419,033
      RAC: 529
      Message 3635 - Posted 26 Aug 2007 10:58:36 UTC - in response to Message 3633.

        I run off-line, and connect only occasionally. By participating in the malaria project, I feel I\'m contributing to the sum of scientific knowledge. As long as malaria WUs have deadlines over three days, I\'m able to report their results on time.

        I just received a malaria WU that has a deadline of less than 28 hours from when it was downloaded to my system. The only way to __ensure__ that such short-deadline WUs will be reported on time would be for me to connect several times *every* day. I cannot guarantee that.
        .

        Me neither. On my \"work\" laptop, BOINC does not connect to the internet while at work and with the way the BOINC schedular works with these short deadlines, I will not be able to keep the requisite cache to ensure it doesn\'t run out of work, unless the current wu_42 run will be over within the next 20hrs.....

        I thought all this had been discussed before and short deadlines agreed to be utilised since it does play havoc with many volunteers crunching ability!

        Live long and BOINC!


        I think you left out the word \'not\' in your statement, but anyway what I was going to suggest was in one of the up and coming versions of Boinc the program itself look at the frequency of user connections and not send a unit to a user that can\'t return it in time. I guess that would mean that those of us only crunching for Malria would get all the shorter units and you guys crunching for multiple projects and/or not always on connections would get the longer units. Either way that way everyone gets to crunch AND units don\'t get aborted as often.
        ____________

        Profile mikey
        Avatar
        Send message
        Joined: Mar 23 07
        Posts: 4682
        Credit: 5,419,033
        RAC: 529
        Message 3636 - Posted 26 Aug 2007 11:00:08 UTC - in response to Message 3630.

          I run off-line, and connect only occasionally. By participating in the malaria project, I feel I\'m contributing to the sum of scientific knowledge. As long as malaria WUs have deadlines over three days, I\'m able to report their results on time.

          I just received a malaria WU that has a deadline of less than 28 hours from when it was downloaded to my system. The only way to __ensure__ that such short-deadline WUs will be reported on time would be for me to connect several times *every* day. I cannot guarantee that.


          My personal suggestion would be to abort that unit as soon as you realize you can\'t return it in time. That way the server will know to send it out to someone else and you won\'t be crunching a unit you won\'t get any credit for. See my reply to The Gas Giant for my options for modifying the program.
          ____________

          Mikus
          Send message
          Joined: Jan 4 07
          Posts: 13
          Credit: 392,389
          RAC: 641
          Message 3642 - Posted 26 Aug 2007 20:09:21 UTC - in response to Message 3636.

            My personal suggestion would be to abort that unit as soon as you realize you can\'t return it in time. That way the server will know to send it out to someone else and you won\'t be crunching a unit you won\'t get any credit for. See my reply to The Gas Giant for my options for modifying the program.

            It\'s not that I can\'t return it in time. It\'s that I can\'t *guarantee* that I will return it in time. [What if I\'m gone overnight?]

            Because I spotted the one yesterday (are you suggesting that I perpetually __monitor__ all the work that boinc downloads ?) I was able to connect again when it had finished, and was able to report it in plenty of time. But if I hadn\'t spotted it -- who knows. [I happened to notice that today malaria again sent me one of those short-deadline workunits. Bah!]
            .

            Mikus
            Send message
            Joined: Jan 4 07
            Posts: 13
            Credit: 392,389
            RAC: 641
            Message 3643 - Posted 26 Aug 2007 20:30:51 UTC - in response to Message 3635.

              ... anyway what I was going to suggest was in one of the up and coming versions of Boinc the program itself look at the frequency of user connections and not send a unit to a user that can\'t return it in time. I guess that would mean that those of us only crunching for Malria would get all the shorter units and you guys crunching for multiple projects and/or not always on connections would get the longer units. Either way that way everyone gets to crunch AND units don\'t get aborted as often.

              FYI, my \'General preferences\' at the malaria project\'s webpage *already* specifies 38 hours as the interval at which my system gets connected to the Internet. That exceeds the wu_42 deadline by more than 10 hours. So the server __has__ the information needed to avoid sending me such short-deadline workunits.

              [By the way, thanks to the \"wonders of distributed computing\", 38 hours is *not* the actual \'connect interval\' value that my boinc client\'s work scheduler uses to calculate whether earliest-deadline-first scheduling is needed. I override the website\'s value with my own local value via a <global_prefs_override> file.]
              .

              Profile The Gas Giant
              Avatar
              Send message
              Joined: Mar 7 06
              Posts: 1214
              Credit: 3,710,893
              RAC: 1,076
              Message 3644 - Posted 26 Aug 2007 20:42:29 UTC - in response to Message 3635.

                I run off-line, and connect only occasionally. By participating in the malaria project, I feel I\'m contributing to the sum of scientific knowledge. As long as malaria WUs have deadlines over three days, I\'m able to report their results on time.

                I just received a malaria WU that has a deadline of less than 28 hours from when it was downloaded to my system. The only way to __ensure__ that such short-deadline WUs will be reported on time would be for me to connect several times *every* day. I cannot guarantee that.
                .

                Me neither. On my \"work\" laptop, BOINC does not connect to the internet while at work and with the way the BOINC schedular works with these short deadlines, I will not be able to keep the requisite cache to ensure it doesn\'t run out of work, unless the current wu_42 run will be over within the next 20hrs.....

                I thought all this had been discussed before and short deadlines agreed to be utilised since it does play havoc with many volunteers crunching ability!

                Live long and BOINC!


                I think you left out the word \'not\' in your statement, but anyway what I was going to suggest was in one of the up and coming versions of Boinc the program itself look at the frequency of user connections and not send a unit to a user that can\'t return it in time. I guess that would mean that those of us only crunching for Malria would get all the shorter units and you guys crunching for multiple projects and/or not always on connections would get the longer units. Either way that way everyone gets to crunch AND units don\'t get aborted as often.

                True...I did leave out not. It should have read,

                I thought all this had been discussed before and short deadlines agreed not to be utilised since it does play havoc with many volunteers crunching ability!

                FalconFly
                Avatar
                Send message
                Joined: Mar 7 06
                Posts: 92
                Credit: 5,517,713
                RAC: 0
                Message 3646 - Posted 26 Aug 2007 22:25:56 UTC - in response to Message 3644.

                  Last modified: 26 Aug 2007 22:28:41 UTC

                  Hm, I just returned from a weekend trip and fired my Network back up... Running on the short 0.25 Network cache, I assumed I would not run into deadline problems.

                  As it turns out, the deadlines for the \'normal\' workunits (not WU_45 which had theirs extended) are as short as ever.

                  Not a big loss, but I thought we\'d now have like 2-3 days on all WorkUnits instead of the mere 36h that I persistently missed during the weekend with the Network shutdown (?)

                  Anyway, being orderly I completed what I got, so far a few still made it back in time for crediting :)
                  ____________
                  Scientific Network : 44800 MHz - 77824 MB - 1970 GB

                  Profile alain studer
                  Send message
                  Joined: Jan 4 07
                  Posts: 34
                  Credit: 11,769
                  RAC: 0
                  Message 3650 - Posted 27 Aug 2007 13:50:36 UTC - in response to Message 3646.

                    Hm, I just returned from a weekend trip and fired my Network back up... Running on the short 0.25 Network cache, I assumed I would not run into deadline problems.

                    As it turns out, the deadlines for the \'normal\' workunits (not WU_45 which had theirs extended) are as short as ever.

                    Not a big loss, but I thought we\'d now have like 2-3 days on all WorkUnits instead of the mere 36h that I persistently missed during the weekend with the Network shutdown (?)

                    Anyway, being orderly I completed what I got, so far a few still made it back in time for crediting :)



                    The 36h deadline is used for the test workunits only (this is because they are more likely to crash). The test run is already finished so you should get the 83h deadlines again.



                    ____________
                    Alain Studer
                    Swiss Tropical Institute

                    FalconFly
                    Avatar
                    Send message
                    Joined: Mar 7 06
                    Posts: 92
                    Credit: 5,517,713
                    RAC: 0
                    Message 3651 - Posted 27 Aug 2007 20:02:45 UTC - in response to Message 3650.

                      Last modified: 27 Aug 2007 20:03:19 UTC

                      The 36h deadline is used for the test workunits only (this is because they are more likely to crash). The test run is already finished so you should get the 83h deadlines again.


                      Ah, thanks for the hint, I simply overlooked that.
                      In that case the Problem sat (as usual) in front of my monitor
                      ____________
                      Scientific Network : 44800 MHz - 77824 MB - 1970 GB

                      Mikus
                      Send message
                      Joined: Jan 4 07
                      Posts: 13
                      Credit: 392,389
                      RAC: 641
                      Message 3753 - Posted 9 Sep 2007 13:38:06 UTC

                        I run off-line. Just had a workunit downloaded <http://www.malariacontrol.net/workunit.php?wuid=3791047> that has a deadline of less than 28 hours. Let me say this again: I can __NOT__ guarantee that such workunits will be reported by their deadline.


                        [Let me suggest that the Malaria server examine a user\'s preferences before selecting the work to be downloaded to that user. My preferences (at the Malaria website) clearly state that the \"interval between connects\" to be expected from my system is some 38 hours. I fail to see how the Malaria project can expect my system to report on time work for which they have set a 28-hour deadline.]
                        .

                        Profile mikey
                        Avatar
                        Send message
                        Joined: Mar 23 07
                        Posts: 4682
                        Credit: 5,419,033
                        RAC: 529
                        Message 3754 - Posted 9 Sep 2007 14:06:37 UTC - in response to Message 3753.

                          I run off-line. Just had a workunit downloaded <http://www.malariacontrol.net/workunit.php?wuid=3791047> that has a deadline of less than 28 hours. Let me say this again: I can __NOT__ guarantee that such workunits will be reported by their deadline.


                          [Let me suggest that the Malaria server examine a user\'s preferences before selecting the work to be downloaded to that user. My preferences (at the Malaria website) clearly state that the \"interval between connects\" to be expected from my system is some 38 hours. I fail to see how the Malaria project can expect my system to report on time work for which they have set a 28-hour deadline.] .


                          If you don\'t return it in time it will just get resent to someone else to crunch. If you return it before they do you should still get credit for it. When I was at Seti they were trying a system where resent units were sent to people who had very short turn-around times though. I don\'t know if Malaria does that or not.
                          ____________

                          Chris Sutton
                          Send message
                          Joined: Nov 10 05
                          Posts: 297
                          Credit: 4,941,683
                          RAC: 0
                          Message 3755 - Posted 9 Sep 2007 17:16:46 UTC - in response to Message 3753.

                            Just had a workunit downloaded <http://www.malariacontrol.net/workunit.php?wuid=3791047> that has a deadline of less than 28 hours.

                            Mikus,

                            I also received a few units with short deadlines, but the application they are associated with is the test app: malariacontrol.net test version 5.51

                            In your malariacontrol.net preferences on the website, do you have Run test applications or Run malariacontrol test application set to yes?

                            If so, it may be necessary for you to set this to no, as the workunits for the test apps generally will have shorter deadlines to give the developers a quicker turnaround time on results.

                            Profile maire
                            Volunteer moderator
                            Project administrator
                            Project developer
                            Project scientist
                            Send message
                            Joined: Nov 7 05
                            Posts: 439
                            Credit: 118,258
                            RAC: 0
                            Message 3761 - Posted 10 Sep 2007 17:13:29 UTC

                              That\'s right, we really need the short turnaround to get feedback on the test workunits quickly. If this is a problem, please follow Chris\' instructions to opt out of testing.

                              @mikey: We have started to use the feature to send error or no reply results to reliable hosts only. We\'re still tuning the scheduler config. If we manage to significantly reduce the tail in turnaround times, this could help to make the parameter optimization process more efficient.
                              Nick
                              ____________
                              Nicolas Maire
                              Swiss Tropical and Public Health Institute
                              http://www.swisstph.ch

                              Profile mikey
                              Avatar
                              Send message
                              Joined: Mar 23 07
                              Posts: 4682
                              Credit: 5,419,033
                              RAC: 529
                              Message 3763 - Posted 10 Sep 2007 21:18:12 UTC - in response to Message 3761.

                                That\'s right, we really need the short turnaround to get feedback on the test workunits quickly. If this is a problem, please follow Chris\' instructions to opt out of testing.

                                @mikey: We have started to use the feature to send error or no reply results to reliable hosts only. We\'re still tuning the scheduler config. If we manage to significantly reduce the tail in turnaround times, this could help to make the parameter optimization process more efficient.
                                Nick


                                Yes I knew you would be tweaking it. If Seti has been and David Anderson is on the same email distribution list as you are, I am guessing, then he should be sharing.
                                ____________

                                Post to thread

                                Message boards : Number crunching : short deadlines and not-always-connected users


                                Return to malariacontrol.net main page


                                Copyright © 2013 africa@home