Archive for August, 2009

The Issue of User Agent and Referrer Translation

August 21st, 2009

The Internet is built upon HTTP (Hyper Text Transfer Protocol). It is a relatively simple protocol that exchanges messages between a ‘client’ and a ‘server’. PHP provides access to these messages in the form of some global variables, e.g. $SERVER['USER_AGENT'] and $SERVER['HTTP_REFERER']. It is these variables that provide web analytics solutions some of the most important information they require.

For example think about the search phrases that your website’s visitors have used to find your website! Although you cannot really trust those variables (someone might lie saying that he found your website on Google by looking for “garbage guy”…), they provide the only way to get information about what browsers and operating systems your users are using and where they are coming from. All this knowledge is essential when it comes to optimizing your website and your advertising and linking activities.

The wopsta plug-in will naturally translate the different user agents (browsers) and referrers to provide meaningful statistics. Our twist is that instead of using an isolated client library that is only updated with every new release of the plug-in, we are going to implement a web service at wopsta.org. This web service allows clients to publish their recordings of unknown user agents, browsers, bots and spiders for the benefit of the wopsta community and themselves.

With this web service the plugin’s “rules engine” can be automatically updated by accessing the web service’s “update client library”-method. The new browser versions, operating systems, spiders and emerging search engines will be added automatically, enabling our plugin users to get the most out of the data.

As always we appreciate your comments and suggestions!

We are working on it…

Basic Thoughts about wopsta’s Architecture

August 17th, 2009

We think it is only appropriate that an open source project should share as much as possible with its community. With this in mind today’s post will be discussing some architecture issues in the design of the wopsta.org plug-in. Although our community is still small we hope that this post will inspire and inform; and as ever we encourage you all to make suggestion in the comments below.

Fundamental Components

We have decided to seperate the tracking of requests and the generation of statistics in two modules. Both will share an initialization within the plugin and a common data model / configuration that defines the various types of requests, visitors, visits, etc.

Beneath these two components the data model will hook into the wordpress API to gather contextual information, for example by adding the name a visitor has used while writing a comment to the visitor’s model in the wopsta.org plug-in.

Tracking

All requests made through HTTP are tracked. As expected, this leaves an abundant amount of information that requires processing with our rules engine. For example we’ll use of the “user agent” field to identify whether the request is a human or a bot. Other rules that our engine includes:

  • Which browser is used by a visitor?
  • Which searchengines and search phrases were used by a visitor?
  • Were there crawlers visiting the blog?

To identify returning visitors we intend to use cookies only.

Statistics

To provide flexibility the Ext JS component library extensive set of panels will feature heavily within a centralized customizable dashboard, further pages such as visitors,traffic sources and contents will be used as default homes for these components. The dashboard shall furthermore have a fixed general panel, displaying key statistics, Ext JS library also provides a chart library to further enhance the visitations.

Any comments are appreciated!

Typology of Weblogs

August 7th, 2009

One of our early research goals is to identify differences between a blog’s success metrics against metrics used by traditional websites. The identification of previously un-established metrics would give wopsta.org a deeper importance .

In part with the variety of blogs and owners on the world wide web there is no single set of shared desires or outcomes and therefore we believe at wopsta.org that they will not all share the same metrics needs. In order to understand the needs of owners and their motivations we will examine the topology of a cross section of blogs on the world wide web.

Our initial research points to a little book by Zerfaß and Boelter’s on blog topology which is briefly summarized below:

At first, private blogs and corporate blogs can be distinguished. The underlying premise is that owners of private blogs provide information of any kind but do not have the aim to earn money, the lines are blurring with a commoditization occurring leaving blogs that are owned by individuals setup for profit generation along side traditional blogs  and thus individuals being to behave like companies do.

Corporate blogs are owned by companies and non-governmental organizations. In accordance to Zerfaß and Boelter there are many different purposes to why corporations run blogs, e.g. campaign-blogs, topic-blogs, product- and brand-blogs, customer-relationship-blogs and crisis-blogs as well as internal knowledge- and collaboration-blogs.

Furthermore differences between blogs can be made by the type of content and the way of its delivery.

It is obvious that there cannot be a universal key performance indicator (KPI) able to represent success.

As mentioned earlier the next step of our research will be to take a closer look at the aims of differend kinds of blogs and their applied metrics.

Your opinions and ideas about our thoughts are highly welcome!