[Devel] Syndication schedules

Chris Dolan chris at clotho.com
Sun Jan 16 08:17:00 EST 2005


I've been looking into ways to update my Planet more often without 
swamping the upstream feeds.  Some feeds detail their syndication 
frequency via the RSS Syndication extension[1], for example the 
Slashdot feed[2] uses this to say that they update the feed hourly.  
Other feeds use out-of-band info (e.g. the CPAN Rating feed which 
announces on a website[3] that the update frequency is every 6h).

I propose that we make Planet smarter about scheduling.  Ideally, I'd 
like to update my Planet more often, say every 4 hours, and have it 
update only the feeds that may have new info.  For example, I know that 
Robert Cringely[4] only has new content on Fridays, so why check his 
feed multiple times per day?  I'm wondering if others may have interest 
in such an improvement.  Here the aspects of such a change:

  1) Make planet auto-detect syndication data in a feed
  2) Allow administrators to specify a feed update frequency in the 
config.ini (perhaps overriding an auto-detected data in the feed, if 
applicable)
  3) Keep track of when planet.py last ran
  4) Make the cache update code respect the update frequency and only 
query a feed on the appropriate schedule
  5) Provide syndication info of our own in our output XML files
  6) Provide a way to override the frequency data via the command line 
(an "update NOW" flag)

Thoughts?

Chris

[1] http://web.resource.org/rss/1.0/modules/syndication/
[2] http://slashdot.org/index.rss
[3] http://plasmasturm.org/feeds/
[4] http://www.pbs.org/cgi-registry/cringely/cringelyrdf.pl

--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703

Clotho Advanced Media, Inc. - Creators of MediaLandscape Software 
(http://www.media-landscape.com/) and partners in the revolutionary 
Croquet project (http://www.opencroquet.org/)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://lists.waugh.id.au/archives/devel/attachments/20050115/c038b4b6/PGP.pgp


More information about the Devel mailing list