We should:

• propose a core API to get the logo
• propose a core API to strip HTML from a text (for the copyright)
• propose a core API to extract a sub part of a content and make sure it is still valid HTML
• propose a Feed API to get the feed from a HQL query (without having to run the search code in the velocity)
• add feed apis to set all the usual defaults for Wiki and Blog RSS feeds

This way we would get the RSS feed in 3 lines in the velocity, and customization is still possible

Ludovic

/xwiki/bin/feed/Main/WebRss

We could also propose some aliases for the standard feeds

/xwiki/bin/feed/BlogRss -> all the blogs
/xwiki/bin/feed/WebRss -> all the wiki
/xwiki/bin/feed/WebSearchRss -> search

/xwiki/bin/feed/SpaceName/BlogRss -> the blog in space RSS

This could all be based on reserved words.. This is just a suggestion.. We need to find a nice way..

• propose a core API to get the logo

Currently, getSkinFile method returns relative (internal) URLs. I couldn't find a way to get the external URL for a skin file. Is this something forbidden? I've noticed that we actually compute the full URL but we strip it with urlFactory.getURL(url, context). So what do you think of overloading the getSkinFile method by adding a absoluteURL flag?

getSkinFile(String filename, boolean forceSkinAction, boolean absoluteURL, XWikiContext context)

Then we could get the logo with:

#set($feedLogoAbsoluteURL = $xwiki.getSkinFile($xwiki.getSkinPreference('logo', 'logo.png'), false, true))

• propose a core API to strip HTML from a text (for the copyright)

Right now we use:

#set($feedRights = $xwiki.webCopyright.replaceAll('<', '<html:').replaceAll('<html:/', '</html:'))

Why do we need this? I've noticed that in the generated feed the HTML special characters from the copyright are escaped.

• propose a core API to extract a sub part of a content and make sure it is still valid HTML

I haven't found a library providing this feature. Do you know one? Of course, this is easy to implement using SAX. I could add a method in api.Util but want to be sure I don't reinvent the wheel.

• propose a Feed API to get the feed from a HQL query (without having to run the search code in the velocity)

Done.

• add feed apis to set all the usual defaults for Wiki and Blog RSS feeds

In progress.

1. 1. Velocity code for XWiki.WebRssCode
      $context.setCacheDuration(1800)
      #set($feedMetadata = { "title":"$!title", "description":"$!description", "url":"$!baseurl"})
      #if($list)
      {pre}$xwiki.feed.getWebFeedOutput($list, $feedMetadata, "rss_1.0"){/pre}
      #else{pre}$xwiki.feed.getWebFeedOutput($sql, 20, 0, $feedMetadata, "rss_1.0"){/pre}
      #end
      
      
      ## Velocity code for XWiki.BlogRssCode
      $context.setCacheDuration(1800)
      #set($cal = $xwiki.calendar.calendar)
      #set($ok = $cal.add(10, 1))
      #set($ok = $response.setDateHeader("Expires", $cal.time.time))
      #set($feedMetadata = { "title":"$!title", "description":"$!description", "url":"$!baseurl"})
      #if($list)
      {pre}$xwiki.feed.getBlogFeedOutput($list, $feedMetadata, "rss_1.0"){/pre}
      #else
      {pre}$xwiki.feed.getBlogFeedOutput($sql, 20, 0, $feedMetadata, "rss_1.0"){/pre}
      #end

Solution 1
Given the fact that I need only a part, a prefix actually, of an HTML fragment (usually plain text with some tags in it, like a, p, div etc.) there's no point in loading the whole content in memory with DOM but parsing only what I need with SAX. Unfortunately this is not possible in most of the cases because the HTML fragment is, with high probability, not well-formed causing fatal errors during the SAX parsing process. I haven't found a way to continue-after-fatal-error (even with http://apache.org/xml/features/continue-after-fatal-error on).

Solution 2
Sergiu suggested me to clean up the HTML fragment (make it well-formed) before giving it to the SAX parser, and pointed me to JTidy. JTidy surely does a great job, but it loads the entire document in memory, building a DOM-like tree. At first I thought of giving the output from JTidy to the SAX parser, but then I realized this is really stupid because having the HTML fragment already loaded into a DOM tree, I could easily extract the part I need from there.

Recap: the first solution has speed but works only if the input is well-formed (or at least well-formed until it reaches the length limit, i.e. when I want the first 200 characters) and the second solution is slower and takes more memory, but (almost) always provides an acceptable output.

Opinions?

This would lead to something like this for the blog :

Velocity :

## Velocity code for XWiki.BlogRssCode
#set($cal = $xwiki.calendar.calendar)
#set($ok = $cal.add(10, 1))
#set($ok = $response.setDateHeader("Expires", $cal.time.time))
#set($feedMetadata = { "title":"$!title", "description":"$!description", "url":"$!baseurl"})
#set($feedEntryMapping = { "entrytitle":"title", "content":"content"})
#if($list)
{pre}$xwiki.feed.getFeedOutput($list, $feedEntryMapping, $feedMetadata, "rss_1.0"){/pre}
#else
{pre}$xwiki.feed.getFeedOutput($sql, 20, 0, $feedEntryMapping, $feedMetadata, "rss_1.0"){/pre}
#end

SyndEntrySourceImpl.java

String titleProp = feedEntryMapping.get("title");
  if (titleProp != null && !"".equals(titleProp)) {
    String title = doc.display(titleProp);
  } else {
        String title = doc.getDisplayTitle(context);
  }

Writing I realize entry contents aren't managed in the last patch you provided, am I wrong ? Is it on purpose ?

#set($feedEntryMapping = { 
  "uri":'$doc.getExternalURL("view", "language=" + $doc.realLanguage())', 
  "link":'$doc.getExternalURL("view", "language=" + $doc.realLanguage())', 
  "title":'$doc.display("title", $doc.get("XWiki.ArticleClass"))', 
  "description":'$?.getHTMLPreview($doc.getRenderedContent($doc.display("content", $doc.get("XWiki.ArticleClass"))), 400)', 
  "categories":'$doc.display("category", $doc.get("XWiki.ArticleClass")).split(",")', 
  "publishedDate":'$doc.creationDate', 
  "updateDate":'$doc.date', 
  "author":'$xwiki.getUserName($doc.creator, null, false)', 
  "contributors":'[$xwiki.getUserName($doc.author, null, false)]'
})

which I don't even know if it will work, but for me it's ugly. Plus, because each Velocity string (what about collections?) must be parsed in Java for each feed entry I believe this is time consuming.

Also, I don't think that documents are the only feed source. What about a stats feed or an activity event feed? Now FeedPlugin calls searchDocumentsNames, but it can call search instead. I have to change it to do so.

I wrote SyndEntryDocumentSource and SyndEntryArticleSource classes to clone the behavior of WebRssCode and BlogRssCode respectively. So I didn't needed the contents field, mostly because I used the description field.

Another solution would be to implement the SyndEntrySource in a groovy page.

JV, I'm not sure about your solution too. I admit mine has little flexibility on the Velocity side, but it is flexible on the Java side, plus allowing me to write clean code. In you case, I think we'll end up writing something like:

#set($feedEntryMapping =

Unknown macro: { snip }

)
which I don't even know if it will work, but for me it's ugly. Plus, because each Velocity string (what about collections?) must be parsed in Java for each feed entry I believe this is time consuming.

Not if you write a default behavior for each field, allowing to override only some of them (or even no one) from velocity.

Also, I don't think that documents are the only feed source. What about a stats feed or an activity event feed? Now FeedPlugin calls searchDocumentsNames, but it can call search instead. I have to change it to do so.

If you call a search instead I think you open a security breach. About other sources like stats or activity stream (which are marginal today) sure we could have other implementations.

I wrote SyndEntryDocumentSource and SyndEntryArticleSource classes to clone the behavior of WebRssCode and BlogRssCode respectively. So I didn't needed the contents field, mostly because I used the description field.

Ok, could be a good improvement for XE 1.5 btw (having a diff in the case of a wiki page, the whole content in case of an article).

You say that we can write a default behavior for each field. Right. This is currently happening with SyndEntryDocumentSource and SyndEntryArticleSource. They both provide default values for link, title, description and all the rest of the fields. That's why I added methods like getWebFeedOutput and getBlogFeedOutput in FeedPluginApi. The question is how can the user overwrite these defaults? In my case, I can use the source parameters to complicate the current behavior of SyndEntryDocumentSource and SyndEntryArticleSource. Now they use only two parameters: ContentType and ContentLength. I can add more parameters like: ShowDiff, ContentOffset etc. if needed. This way the user can do some minor customization. The big problem is that he cannot change completely the feed output by only writing velocity code. With Java this is simple. The fix would be to implement a SyndEntrySource that maps each feed entry field to a velocity string which will be evaluated for each feed entry.

In your case, you want to map feed entry fields to xwiki class fields (If I understand correctly). For instance, considering XWiki.ArticleClass you say we can map feed entry title to article title and feed entry description to article content. I see this mapping as a particular way of generating feeds. I can write an implementation of SyndEntrySource, which we can call SyndEntryFieldMappingSource, for this particular behavior.

In the end the user will have to choose between:

• using defaults with little customization possible (SyndEntryDocumentSource, SyndEntryArticleSource etc.)
• using field mapping, also with little customization possible (SyndEntryFieldMappingSource)
• using a custom feed source (SyndEntryCustomSource, which evaluates velocity strings)

#set($feedMetadata = {
  "title":"My Feed Title", 
  "description":"My Feed Description", 
  "url":"http://www.xwiki.org"})
#set($sourceParams = {
  "ContentLength":10, 
  "title":"title", 
  "description":"XWiki.ArticleClass_content", 
  "publishedDate":"XWiki.ArticleClass_0_date", 
  "updatedDate":'{$doc.date.time}', 
  "categories":'{{pre}[$doc.space,$doc.name,$doc.isNew()]{/pre}}'
})
#set($sql = ", BaseObject as obj where obj.name=doc.fullName and obj.className='XWiki.ArticleClass' and obj.name<>'XWiki.ArticleClassTemplate' order by doc.creationDate desc")
{pre}$xwiki.feed.getDocumentFeedOutput($sql, 20, 0, $sourceParams, $feedMetadata, "atom_0.3"){/pre}

Of course, most of the time you wont care about $sourceParams or at most you'll need simple field mapping. The velocity code from my previous posts is still usable. This is for those who want more customization.

• by default there's no limit on the content/description/summary length of a feed entry;
• messages generated by JTidy are caught and logged

I wait to see if we add a action mapping for feeds before closing the issue. (see http://tinyurl.com/5ew7np)