Venus: My Homepage has suddently stopped updating

[Gordon Hodgson]

On 16 Apr 2009, at 18:23, Amit Chakradeo (अमित  
चक्रदेव) wrote:

> On Thu, Apr 16, 2009 at 10:13 AM, Gordon Hodgson <feeds at greenfeed.org.uk 
> > wrote:
> What I've done is cut my config.ini in half into two files called  
> config1.ini and config2.ini (the scripts were terminating before the  
> spidering was complete)
>
> Then I've run spider config1.ini
> And then 5 minutes later  spider config 2.ini
> And then 5 minutes later splice config.ini
>
> Does this make sense?
>
>
> It does. But it still depends on how many feeds does each config  
> file has and how much time the final splice takes. 60 seconds is  
> quite restrictive.

Yes, the host has limited scripts to 60 seconds, and says that the  
longest recorded time he could find for me running the planet.py  
script is 80 seconds... now I have to run three scripts. Surely that  
will take more than 60 seconds of CPU time...?


> Maybe you can just run the planet on your local machine and just scp/ 
> FTP the generated files to the web server ?

Not ideal. I don't like to leave my machines turned on more than I  
have to.


> I'm not completely sure about how much of the file info can be  
> removed (for instance, whether the avatar stuff and most of the  
> [planet] section needs to be in the spidered files (although I've  
> cut it out, and everything seems to work fine). Can anything be cut  
> out from the master config.ini (spliced) file? I haven;t been able  
> to work all this stuff out too easily from the Venus documentation  
> website (and I don't understand python).
>
> Take a look at the architecture diagram: http://intertwingly.net/code/venus/docs/venus.svg
> It looks like the splice.py would need the avatar stuff etc. BTW you  
> can leave all the stuff in the config file, parsing config file  
> takes minuscule amount of time compared with rest of the things the  
> scripts have to do...

Yeah, that sort of makes sense to me...

On slight tangent, is there any way the cron emails can report if a  
script finished early?

I can only tell now, because occassionally one of the lines of the  
email is truncated (but sometimes it isn't).

I've also reduced the time-out time to 10 seconds is this reasonable?  
I'm not even spidering a huge number of feeds... Something like 113...


> --Amit

Cheers for your help so far,

Gordon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /archives/devel/attachments/20090416/b10e8146/attachment-0001.htm