Venus thoughts

[Sam Ruby]

Arvind Narayanan wrote:
> Great!
> 
> Well firstly, I would like for comments from individual blogs to
> automatically appear on the planet. The way I'm doing this now is to
> have a separate comment server that parses the atom file that is
> produced by venus, looks for the <link rel="replies"> tags and
> downloads the comments from those urls according to some freshness
> metrics. It produces JSON files of the comments which are fetched by
> the browser when the user clicks "show comments". It works pretty
> well. But I have to re-implement caching and everything so it would be
> good to have this functionality integrated into venus.

Yes, I could see an option to autofetch any replies feed if it changed, 
and to do the appropriate caching, etc.  While I'm a fan of JSON in 
general, I'm not sure I would bother with it here, as XMLHttpRequest can 
handle Atom entries just fine.  My search page does exactly that:

   http://planet.intertwingly.net/query.html

   http://planet.intertwingly.net/query.js

Unrelated: my biggest issue with 'show comments' is that initially a 
post tends to have few comments, but over time it tends to (1) accrue 
comments and (2) move down the page.  Any thoughts on how to highlight 
posts with new activity?

> Still on the subject of comments, I don't see any way in the template
> engine to control their appearance, so I have a filter that's invoked
> for every feed that changes the <entry> </entry> and inserts div's for
> comments into it. This is a kludge; aside from having to invoke a
> filter on every feed item, it means I have to regenerate all the
> entries by nuking the cache every time I need to make a change to the
> style or whatever in the comments box.

Something I haven't exploited as much as I can, but if you peek into the 
source of my planet intertwingly html page, you will see that each entry 
has a div which specifies the source as a class name, meaning that I can 
create custom CSS rules targeting individuals.

> Second, I'd like to be able to apply a filter to the feed _before_ it
> gets normalized. One of my subscriptions generates a broken-ass RSS
> feed that doesn't have a <published> or <updated> tag, and so these
> entries show up on top every time they're regenerated. I fix this by
> having a filter for this feed that downloads each article, parses the
> human-readable date off of the page and inserts a <published> tag.
> This still doesn't work because the mtime is computed before the
> filters are invoked. I made a one line change to spider.py to look for
> the mtime again after the filters are invoked, but it's still an ugly
> solution.

This I don't understand.  Entries without dates are assigned 'now' the 
first time they are seen.  When they are seen again (as determined by 
id/guid or computed from link or text), they should retain their 
original date.

> Third, it would be great to have a better category filter. The xpath
> sifter is ok but it's too limited. I put together a quick and dirty
> category filter that can handle arbitrary expressions; a typical
> use-case for this is something like "!personal || planet" (like, if I
> don't want my post about thanksgiving to appear on the planet, so I
> label it personal, but if I'm dying of cancer then I can override it
> by specifying the planet label  in addition.) I think it would be
> useful for venus to support this natively.

More example filters are always goodness.

> Finally, I want to be able to produce only the first 10 entries in the
> index.html file and a different file containing the contents of the
> rest of the entries so that the page can load quickly with the first
> 10 entries and the rest  of the entries can load in the background
> with javascript. Could you give me a rough description of what I need
> to change in the code in order to produce two separate files instead
> of one?

Producing two files can easily be done by having two templates.  Now, 
the question turns to how you can have a template display only the first 
10 entries, or display all but the first 10 entries.  That depends on 
the templating language.  Doable in XSLT, not doable in htmltmpl.

> Thanks!
> I do have some other minor issues but these are the major ones so I'd
> like to get your thoughts on these first.
> 
> --Arvind
> 
> On Nov 26, 2007 5:45 AM, Sam Ruby <rubys at intertwingly.net> wrote:
>> Arvind Narayanan wrote:
>>> Hi
>>>
>>> I'm using venus to run a bunch of feed aggregators example:
>>> http://feedworld.net/toc/
>>>
>>> Thank you for this nice piece of software. I have made several
>>> additions to the venus code.
>>> I have some suggestions, comments and feature requests and possibly
>>> contribute some code
>>> back. Would you be interested?
>> Certainly.
>>
>> http://intertwingly.net/code/venus/docs/contributing.html
>>
>> - Sam Ruby

- Sam Ruby