User talk:Daniel Mietchen/bot-recent

Add_Workgroup_to_Related_Articles_bot

Hi Daniel, I saw that you added a statement about an antidote to a test edit. We should make sure that works, too. D. Matt Innis 21:38, 10 January 2010 (UTC)

Done. --Daniel Mietchen 21:58, 10 January 2010 (UTC)

approval discussion

The following was a discussion held on the constable wiki concerning the development of the steps necessary for the Approval process for bots using this bot as the first attempt. As this is in email format, it runs in reverse. Further discussion should occur above the horizontal line preceding the code. D. Matt Innis 22:48, 10 January 2010 (UTC)

As for the name, I have no preference, as long as it is clear that this account is not bound to the one-script rule expressed in point 3 of the bot policy. The antidote is meant to be a general requirement (see point 5). I'm fine with you having access to this account, and with the proposed starting scheme. I will put a description and stopping instructions on the bot's user page once the account is up. I also think we should let all interested people with sysop rights have a try on stopping this, and improve the instructions accordingly. --Daniel Mietchen 23:20, 10 January 2010 (UTC)

Sounds good. The thing to do now is to create the account. We should use the regular account request page and use the bio section for the description and especially how to stop it. That way it will automatically show up when the account is created and we won't have to remember to put it in later. This should be done by the person requesting the bot account and it should include a link to the bot status template. Isn't there a way that you can transclude that template to this bot discussion page and then link to this page? If not, then the request should just link to that tempalte with everything filled out. The constable can then quickly check to see that all the tests have been completed and the approval is there and create the account by accepting the application. He will then have to change the account type to 'bot' so that it has the appropriate permissions and the edits won't show up in the recent changes (I think).

Please create the account using the usual Request account process with your email address as the confirmation email, then once you are finished confirming the request, I'll do my part...

Keep your fingers crossed :) If this works well, then the next one should be much easier. D. Matt Innis 00:08, 11 January 2010 (UTC)

---

Daniel Mietchen wrote:
> On Sun, Jan 10, 2010 at 10:36 PM, The Citizendium Constabulatory [131]
> Matt Innis wrote:
>  
>> Okay, if we create a general bot account,
>>
>> 1) what would we name it?
>>     
> Generalist Bot
>   
That sounds so... general :) ... Since this account is going to be used for various maintenance features, 
how about Maintenance bot?
>  
>> 2) who would have access to it?
>>     
> Let's start with me. Later on, this should be the Bot Manager and his deputies.
>   
Okay, let's put me on the list, too, for now, although I have no earthly idea why.
>  
>> 3) under what circumstances could we shut it down (or more importantly, does
>> anything bad happen IF we have to shut it down?)
>>     
> The easiest way for you to shut it down would be to block the account
> temporarily for some minutes - the script will then receive a server
> access error and stop executing (We can use one of the chunks of this
> one to test this).
>   
So any sysop can shut it down.  We should put instructions on the bot user page..
>  
>> Will it be able to start
>> where it left off?
>>     
> Yes. I will have to do this manually.
>   
okay.
>  
>> 4) how do we warn people about how to shut it down and who will be watching
>> it?
>>     
> The point of automation is not having to watch or interfere
> constantly. Given that over 100 test edits went smoothly, I am
> confident the script can be run without oversight, though I will
> certainly keep track of error messages that may come up.
>
> At the Wikipedias, they have a stop button for bots but I do not know
> how this is coded or operated. In the long run, I envision something
> like this, or having the script read a value from a page on the wiki
> which determines whether it will continue or not.
> However, with me currently being the only one doing automated edits, I
> do not see this as urgent, especially given that the antidote can undo
> any changes made by the script.
>   
That's what I thought...  that is the scary part (that no-one will be watching).  
I agree the antidote feature is a good idea for this purpose and we should likely 
require this in all bots especially until we can assure that it will stop itself.

I think I'm about ready, how about you?

As a test to see if we are going to cause any wiki problems, let's try running it 
for a two hour period starting when you want and wait a few hours then try running it for a 
little while during a peak time and see what happens... what do you think?

Matt


>  
>> Daniel Mietchen wrote:
>>    
>>> On Sun, Jan 10, 2010 at 4:51 PM, The Citizendium Constabulatory [131]
>>> <constables@citizendium.org> wrote:
>>>
>>>      
>>>> Daniel Mietchen wrote:
>>>>
>>>>        
>>>>> On Sun, Jan 10, 2010 at 3:03 AM, The Citizendium Constabulatory [131]
>>>>> <constables@citizendium.org> wrote:
>>>>>
>>>>>
>>>>>          
>>>>>> This is 7300 edits, by definition a bot.
>>>>>>
>>>>>>
>>>>>>             
>>>>> OK, a bot - I added the 500 edit limit to save others the potential
>>>>> annoyance of seeing the Related Changes cluttered with more than 500
>>>>> automated edits, and had forgotten about that aspect now.
>>>>>
>>>>>
>>>>>           
>>>> Agree about the annoyance, and that is part of the reason for the bot
>>>> policy
>>>> in the first place; to try and avoid the annoyance.
>>>>
>>>>         
>>> yes.
>>>
>>>
>>>      
>>>> It does slow down the
>>>> wiki as well.
>>>>
>>>>         
>>> I know that it does in theory, but have never heard the complaint at
>>> CZ, nor seen actual data on this.
>>> My impression was that the wiki is set up to handle much larger
>>> numbers of edits than we typically have in a given period of time, and
>>> that the my scripts have so far operated within this safety margin
>>> (when running scripts/ bots, I never received an error message
>>> indicative of server overload, while I often experience the wiki being
>>> slow when no automated edits are being done by anyone).
>>>
>>> Anyway, the standard way of dealing with server load is to change the
>>> -pt:1 in the command (pause time, which defines the number of seconds
>>> the script pauses between processing two consecutive edits). Most
>>> Wikipedias use values around 10 or, during peak hours, even 20 or 30.
>>> I would assume (I have no data on this) that setting it to 3 would be
>>> a sufficient reduction in server load for the present CZ. Of course,
>>> this means the total run time would change to about 21h.
>>>
>>>
>>>      
>>>> Can this be run several times for about an 30 minutes each
>>>> time during a time when the wiki is not as busy?
>>>>
>>>>         
>>> It can, of course, but there are times (around 7AM-3PM UCT) where I am
>>> often alone on the wiki, or with just one or two others. Then, doing
>>> it in larger chunks probably wouldn't annoy anyone, even with pt:1.
>>>
>>>
>>>      
>>>>>          
>>>>>> If we create a bot, we can flag it as a bot and run it at certain
>>>>>> times,
>>>>>> right?
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> As just explained in a previous mail, it would be run only once.
>>>>> Creating a _separate_ bot account in such cases is not a good idea.
>>>>>
>>>>>
>>>>>           
>>>> See question above.
>>>>
>>>>        
>>>>> I see the currently discussed code as a "well-defined minor job" in
>>>>> the sense of point 3 of
>>>>> http://en.citizendium.org/wiki/CZ:Bot_policy and think it could be
>>>>> performed equally well from the "maintenance bot" mentioned therein
>>>>> (which I would now rather call Generalist bot) that is dedicated to
>>>>> such minor jobs, or it could be performed as a one-time minor task (as
>>>>> opposed to the recurring major one) of the Related Articles Bot, to
>>>>> which it functionally belongs.
>>>>>
>>>>>
>>>>>           
>>>> This sounds like a good idea.
>>>>
>>>>        
>>>>> So I suggest to set up the Related Articles Bot now (do you still have
>>>>> my application for it?) and run it from there, since the idea of a
>>>>> maintenance bot or Generalist bot has not received much thought yet.
>>>>>
>>>>>
>>>>>           
>>>> We could also set up a Generalist Bot account now and run this same
>>>> script?
>>>>
>>>>         
>>> yes.
>>>
>>>
>>>      
>>>>  I realize that you are very interested in just running the darn script
>>>> and
>>>> getting this over with, but I am more interested in setting up the
>>>> process
>>>> so I don't have to have this discussion several more times.
>>>>
>>>>         
>>> fair enough.
>>>
>>>
>>>      
>>>>  What I need is
>>>> to make sure that any bot that you are intending to run does not result
>>>> in
>>>> users calling constables to shut it down like last time.
>>>>
>>>>         
>>> I hope that we will soon have procedures in place which do not call the
>>> constable in such cases, but where all automated actions are vetted by a
>>> bot manager before they are being performed. Here, also the test wiki may
>>> be an interesting tool, but I haven't explored this option yet.
>>>
>>>
>>>      
>>>>  The complaints
>>>> that we had before were related to 1) making changes that were not
>>>> wanted,
>>>> 2) clogging the related changes page with bot activity, and 3) slowing
>>>> the
>>>> wiki down.
>>>>
>>>>         
>>> You know that I didn't know the details, especially 3. And for any
>>> dispute in content, it has been time-honoured practice to leave a note
>>> on the talk page of articles or Citizens concerned - why not in case
>>> of the bot activities??
>>>
>>>
>>>      
>>>>  1) has passed,
>>>>
>>>>         
>>> dunno.
>>>
>>>
>>>      
>>>> 2) can be fixed with a bot that does not show up
>>>>
>>>>         
>>> yes.
>>>
>>>
>>>      
>>>> in related changes, and 3) since this will run for about 7 hours, we
>>>> could
>>>> run it in smaller pieces.
>>>>
>>>>         
>>> yes.
>>>

----
This is 7300 edits, by definition a bot.

If we create a bot, we can flag it as a bot and run it at certain times, right?

Matt

Daniel Mietchen wrote:
> both definitions linked from
> http://en.citizendium.org/wiki/CZ:Bot_policy -
> the difference is, in brief: script for once, bot for regular action.
>
> On Sun, Jan 10, 2010 at 2:18 AM, The Citizendium Constabulatory [131]
> <constables@citizendium.org> wrote:
>  
>> What's the difference between a bot and a script?
>>
>> Daniel Mietchen wrote:
>>    
>>> All bots yes, but not scripts - that's just too tedious.
>>> For definitions, see
>>> http://en.citizendium.org/wiki/CZ:Bot_policy .
>>>
>>> On Sun, Jan 10, 2010 at 2:04 AM, The Citizendium Constabulatory [131]
>>> <constables@citizendium.org> wrote:
>>>
>>>      
>>>> Are we not expecting that one day ALL bots will be separate accounts?
>>>>
>>>> Daniel Mietchen wrote:
>>>>
>>>>        
>>>>> No, it is a one-time script, not a regular one.
>>>>>
>>>>> On Sun, Jan 10, 2010 at 1:59 AM, The Citizendium Constabulatory [131]
>>>>> <constables@citizendium.org> wrote:
>>>>>
>>>>>
>>>>>          
>>>>>> Excellent, sorry I missed that.
>>>>>>
>>>>>> Do you think we should create a bot account for this one?
>>>>>>
>>>>>> Matt
>>>>>>
>>>>>> Daniel Mietchen wrote:
>>>>>>
>>>>>>
>>>>>>            
>>>>>>> http://en.citizendium.org/wiki/User:Daniel_Mietchen/bot-recent#Planned
>>>>>>> links to
>>>>>>> them:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> http://en.citizendium.org/wiki?title=
Special:Contributions&offset=2010-01-08+12%3A53%3A04%2B00&limit=193&target=Daniel+Mietchen&month=&year=
>>>>>>>
>>>>>>> On Sun, Jan 10, 2010 at 1:51 AM, The Citizendium Constabulatory [131]
>>>>>>> <constables@citizendium.org> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>              
>>>>>>>> Give me a link to the test articles.
>>>>>>>>
>>>>>>>> Matt
>>>>>>>>
>>>>>>>> Daniel Mietchen wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                
>>>>>>>>> It is documented in the edit summaries: I ran the command listed at
>>>>>>>>>
>>>>>>>>> http://en.citizendium.org/wiki/User:Daniel_Mietchen/bot-recent#Planned
>>>>>>>>> , and just added a link to the test edits.
>>>>>>>>>
>>>>>>>>> On Sun, Jan 10, 2010 at 1:31 AM, The Citizendium Constabulatory
>>>>>>>>> [131]
>>>>>>>>> <constables@citizendium.org> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                  
>>>>>>>>>> Have you (or can you) document the test somewhere?
>>>>>>>>>>
>>>>>>>>>> Matt
>>>>>>>>>>
>>>>>>>>>> Daniel Mietchen wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                    
>>>>>>>>>>> Hi Matt,
>>>>>>>>>>>
>>>>>>>>>>> no trouble - script worked as expected (though I tried, on top of
>>>>>>>>>>> it,
>>>>>>>>>>> to number the test edits in the edit summary, which did not work
>>>>>>>>>>> as
>>>>>>>>>>> intended, so I skipped the numbering).
>>>>>>>>>>> An example is at
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> http://en.citizendium.org/wiki?title=Adriatic_Sea/Related_Articles&diff=prev&oldid=100616933
>>>>>>>>>>> .
>>>>>>>>>>>
>>>>>>>>>>> Daniel
>>>>>>>>>>>
>>>>>>>>>>> On Sat, Jan 9, 2010 at 11:28 PM, The Citizendium Constabulatory
>>>>>>>>>>> [131]
>>>>>>>>>>> <constables@citizendium.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                      
>>>>>>>>>>>> Hi Danel,
>>>>>>>>>>>>
>>>>>>>>>>>> Any trouble with the test edits?
>>>>>>>>>>>>
>>>>>>>>>>>> Matt
>>>>>>>>>>>>
>>>>>>>>>>>> Daniel Mietchen wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>                        
>>>>>>>>>>>>> I also ran about 20 manually controlled and 100 automated test
>>>>>>>>>>>>> edits
>>>>>>>>>>>>> and will wait for your approval to run the script on the
>>>>>>>>>>>>> remaining
>>>>>>>>>>>>> ca.
>>>>>>>>>>>>> 7300 entries, which will take about 7h.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jan 8, 2010 at 11:31 AM, Daniel Mietchen
>>>>>>>>>>>>> <daniel.mietchen@googlemail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>                          
>>>>>>>>>>>>>> Update:
>>>>>>>>>>>>>> Latest and tested version of the code at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://en.citizendium.org/wiki/User:Daniel_Mietchen/bot-recent#Planned
>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> No edit made yet.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jan 8, 2010 at 12:03 AM, Daniel Mietchen
>>>>>>>>>>>>>> <daniel.mietchen@googlemail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                            
>>>>>>>>>>>>>>> Hi Cops,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I wish to run a script that does this change
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://en.citizendium.org/wiki?title=1_Timothy_%28Bible%29%2FRelated_Articles&diff=100616602&oldid=100570370
>>>>>>>>>>>>>>> for all pages listed at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://en.citizendium.org/wiki/Category:Bot-created_Related_Articles_subpages
>>>>>>>>>>>>>>> , so as to allow a listing of the bot-created pages by
>>>>>>>>>>>>>>> workgroup.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The command would be
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> python replace.py -cat:Bot-created_Related_Articles_subpages
>>>>>>>>>>>>>>> -regex
>>>>>>>>>>>>>>> "[[Category:Bot-created Related Articles subpages]]"
>>>>>>>>>>>>>>> "{{Bot-created_related_article_subpage}}" -summary:"Robot:
>>>>>>>>>>>>>>> Adding
>>>>>>>>>>>>>>> [[CZ:Workgroups]] to bot-created Related Articles subpage"
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and I will run ten pages first and then 100 pages (for
>>>>>>>>>>>>>>> testing)
>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>> going for all (which will take several hours).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Note that I consider this a script, not a bot, but to avoid
>>>>>>>>>>>>>>> confusion,
>>>>>>>>>>>>>>> I ask for your permission to run it anyway.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Daniel
>>>>>>>>>>>>>>>