DuoMesh Project Blog

After 4/5 months of development I have released a beta/testing version to the general public at http://beta.duomesh.com. There were approx. 25 people that had joined the mailing list to keep updated with latest developments, so I have dispatched a email to allow for user testing and an option for them to take part in a user experience questionnaire.

This version is a fully functional prototype of how the end website operate and function. I will use the feedback to assess user reaction and first responses to the website. It will also help to iron out any bugs or problems that I have missed.

It will also allow for the rest of the time for me to concentrate on documentation and marketing.

Allowing users to add different weather information to their profiles was one of the feature I suggested early on in the project.

This has now been implemented using Yahoo! Weather’s API service and PHP Class “weather” by Matt Brown. This allows developers to access Yahoo! Weather services and pull information back into their own applications and easily customise the layout and design.

The only problems encountered with the implementation where the transparent weather icons in PNG format didn’t render properly in Microsoft Internet Explorer 6 (the problem didn’t effect IE7 as Microsoft new version of IE has partial support for PNGs) as shown in the image below.

This was fixed by using a CSS hack, this involved putting each of the transparent PNG icons into a div and setting the icon as a background image for that div layer. Then using a special filter command to trick IE6 to render the images correctly.

After getting standard text based feeds functional, the next steps was to get other feeds types supported. As feeds are generated/written in XML this means that the tags used in the markup language are not restricted liked HTML and users can create their own tags such as:

This could have been used to detect/or describe which kind of feed the user was adding/viewing (e.g. text based, podcast, image feeds). But due to the nature of XML there is no standardised way of describing if the feed they have added, is entirely image based or contains some kind of video/audio information.

This means that I will have to let the user choose what type of feed they are trying to add.

When developing the Podcast module I encountered a wide verity of problems, as the XML based feed for a Podcast is not widely standardised. Apple has made an attempt at trying to solve this problem, as their media player "iTunes" supports Podcasts, but will only accept XML that has been written to conform to their technical specifications. After adding wide verity of feeds I found that some podcasts didn't include the <description> which caused the layout of the site to break. Also some Podcasts would link the user to a web page instead of linking them directly to the MP3 audio file or video file. These things all had to be taken into account while debugging the code.

One of the major problems with the site at the moment is the loading times. When a user visits the site and they are the only user to view it within 30 minutes, some of the feeds may have been updated, by default Magpie RSS parser will check the last time the feed was cached (i.e. downloaded before and stored in memory on the server). If the RSS feed has a newer creation date than the last time the feed was cached the RSS extension will attempt to automatically download the feed again.

This could adversely affect the loading speeds of the website. As a user may visit the site and the RSS module will find that all the feeds have expired and will have to be updated.

I have found that sometimes the site will take upto 20 to 40 seconds to load, and in some extreme cases this times out the browser. Once the user has been to the site the feeds will have been cached and the site will load in less than a second. One way of resolving the problem would be to change the way the RSS module caches feeds, possibly by forcing the system to keep caches for up to half an hour to an hour.

Another way of increasing the speed and most likely the best way, would be to setup a cron job on the server. A cron job is like a scheduled task and can be setup to automatically load up the website say every 30 minute. This would take the loading off the user and system would deal updating the feeds.

The core feeds types that the script will support are:

• Standard text based feeds e.g. BBC News
• Image based feeds from such image galleries as Flickr
• Audio/Video based feeds such as Podcast like Diggnation (weekly technology video Podcast)

Other feed types:

• Email
• Weather
• Access to non-RSS feed information by taking a website and only stripping out the text

So far I have implemented standard text based feeds and image based feeds see: http://beta.duomesh.com for the latest development build.

One of the features that I wanted to provide was access to websites that do not provide RSS feeds. So today I have been experimenting by making a custom demo/module that will take any website and read in the HTML into a string, then only pull out the content which the user is interested in.

To do this there were several steps that the system needs to do:

1. Pull the content (HTML code) from a website into a string or buffer
2. The user must identify two things on the website that doesn't normally change (e.g. text like Home or News, or a HTML tag like the start/end of a table etc.)
3. The system will then search for the first occurrence of that text/HTML and then remove anything before it
4. Then it will look for the last part which doesn't normally change and will delete everything after that
5. The next step was to remove all the HTML tags (except for hyperlinks or line breaks <br>) so that only the text is left

After a few hours of coding I managed to get it working by using various commands such as strpos, substr and strripos the only problem I encountered was that relative hyperlinks were pointing to a file that should exist on their website, but it was actually pointing to my own test server (my Mini Mac) which gave a 404 File not found message.After a bit of hunting online I found a command that converted a relative link into a absolute link from a coding discussion forum which provided very useful. An example can be seen in the following screenshot:

View Full Size

Today I have been looking into ways of processing and outputting RSS feeds. The best way of processing feeds is with a free PHP extension called Magpie RSS this will then allow you to call the function/program from within your own script and it will allow you to access all the various metadata within the RSS/XML feed.

The benefit of using Magpie RSS is that it will automatically cache the RSS feeds and will only update the feed when it there are updates. This greatly improves the overall speed of loading up a page with allot of RSS feeds on it, as the system will only need to request the feed online once and all subsequent requests for that feed will use a cached version stored in its cache folder
For example to load the BBC News RSS feed, the following command can be used:

This will then make a request and then cache that request. Each news item can be accessed by calling its specific tag, so to pull out the title of the article I can use $item['title'] and the same can be done to get a short description of that article $item['description']. So to display the complete feed the follow command can be used:

Today I have settled on my project idea, which is one of the ideas I have been thinking about over the summer.

"Social Networking Website for RSS feeds and Mobile content"

Really Simple Syndication (RSS for short) is what most website use to allow their viewers keep track of their latest publications without having to manually go to their website and see what's changed.

I hope to build a fully functioning website that will allow users to add/browse different RSS feeds and create a custom page where they can keep track of all their most important websites. Then I will allow users to access their custom page via a mobile device.

I have also chosen Paul Kyberd as personal project tutor as he has experience with web applications and databases so hopefully I can use his expertise to process with my ideas and any technical requirements.