[Devel] Memory usage
mary-planet at puzzling.org
Sun Jan 15 22:46:35 EST 2006
I think I may have mentioned this here before, but just a reminder...
thanks to this code in planet/__init__.py:
329 info = feedparser.parse(self.url,
330 etag=self.url_etag, modified=self.url_modified,
(which causes the entire feed to be read into memory)
446 # Create the item if necessary and update
447 if self.has_item(entry_id):
448 item = self._items[entry_id]
450 item = NewsItem(self, entry_id)
451 self._items[entry_id] = item
(which is part of copying all seemingly recent entries into an attribute
of self -- ie something which unless removed will have the lifetime of
the channel object)
and these in planet.py:
137 for feed_url in config.sections():
141 # Create a channel, configure it and subscribe it
142 channel = planet.Channel(my_planet, feed_url)
planet uses up an enormous amount of memory -- essentially it builds up
all new items in memory and then dumps them as a lump to the template
manager. For some planets, I've seen it use 40MB of memory, which is
fine on a grunty server, less good for anyone (like me) running on a UML
with very restricted memory.
This is somewhere on my own to-hack list, but bringing it to folks'
attention just in case anyone feels inspired.
More information about the Devel