ParTech IT blog

It's been a long time since I posted here.
I'm not really doing anything with this blog at the moment, but I do regularly post new stuff on my company's Sitecore blog.
You can find it here: http://www.partechit.nl/nl/blog

/Ruud

HTML5 / JavaScript raycaster demo

Lately I have been playing around with raycasting, a technique used for rendering 3D worlds, just for the fun of it.
Once I had a working version in C#, I decided to port it to HTML5 (rendering on a canvas) as an exercise.
It supports texture mapping and uses a simple lighting and shadow effect to make it feel a little more realistic.

View the demo here: http://www.dottech.nl/raycaster/
It's best viewed in Chrome but it also works in Firefox and should work in IE9 (didn't test that).

Source code is available on GitHub: http://github.com/Stribe/HTML5-Raycaster

/Ruud

Stribe

It has been quiet on my blog for some time.
That's because i've been busy with my biggest and most important project so far: Stribe.
This article is meant to tell a little about project and the company behind it and moreover to shine some light on the technical details.

Stribe is a company i started together with a good friend under the wing of my former employer Colours.
In january i started the development on a platform for social shopping; a website that offers a huge catalog of products from lots of big webshops and allows users to create profiles showing what they like, sharing it with friends.
Make wishlists while shopping for goods, compare prices between shops and tons of more functionality... that we eventually did not use in the end result... :P

As it goes with these kind of projects, we have changed the concept along the way and eventually ended up with two very cool websites:

Stribe.nl
Offers discounts on online fashion for a huge amount of brands and webshops.
We also organize "collective discount" actions where users receive a higher discount if more people sign up for the action.
Users need to create a profile that reflect their brand and shop preference so that they will only receive updates for those discounts that are relevant for them.

Boetiek.nl
Is basicly an online catalog that hold all fashion products for over 25 webshops (and growing) which comes down to over a 100.000 products.
It's an easy tool for those who are looking to buy fashion online and don't want to search through all those websites individually.

Ok, so that's the functional (for us tech-geeks, boring) part of the story.
What is behind these sites that makes this all possible?

We use:
- .NET 4.0 Framework, developing in Visual Studio 2010
- C#, MVC 2, jQuery
- MS-SQL 2010
- Spark View Engine
- Xapian search engine
- NHibernate (2.something i think)
- Fluent NHibernate
- NHibernate Lambda Extensions
- NUnit
- Umbraco (for very basic content storage)
- GitHub for sourcecontrol with GitExtensions as the client
- FogBugz for issuetracking
- Yourzine's Footprint for outgoing mailings
- a lot more of small helper utilities

.NET 4.0 and MVC 2 were still in beta when we started.
That's one big advantage of running your own project: you may decide to use techniques that are so new they are not even final released and there is no one there to stop you! :)
Before the start i did not have any experience at all with MVC which made it even more exciting!

I got to develop some pretty cool components during the project.
First there is the whole custom site framework with connections to Umbraco to retrieve content (we did not use umbraco rendering, just for simple content storage) and of course connected to our SQL database (which at some point contained close to 100 tables).

Then there is the Importer Tool, a tool that imports all product feeds from the webshops we offer in our catalog and adds them to our database.
This also includes all related brands and product properties such as color, size, material, etc.
As most shops offer the feed in their own format, each one needs to be converted to our own standard, then added to our database and finally each product needs to be placed in a category which was a huge challenge on it's own.

Another interesting task was implementing the Xapian search engine that crashed when used in combination with 4.0 .NET framework.
This had to be solved using WCF to isolate the search service from the rest of the solution.

Last but not least, the automated build process.
With one push of the button i can build the solution and generate a deployable output that can be dragged and dropped to the production server.
It takes care of merging and minifying of css and javascripts and generating the correct configuration files for the production environment.

I can go on like this for hours, but you get the point: it was a project full of challenges and also with all freedom to solve them.
Every software engineer's wet dream 8-)

We're currently running the sites on 2 loadbalanced Windows 2008 webservers and one dedicated SQL Server.
There is 24h monitoring that alerts me immediately when any of the systems fail.
I'm happy with the result so far and believe that we now have a steady base for future development.

If i get more time and feel like writing, i would like to go more into the specifics of some of the sections and choices i've made.
One subject that deserves some more attention is the Spark View Engine we used, which has a lot of potential (although i suspect that Razor might replace Spark alltogether in the future...)
Also the Fluent NHibernate and Lambda Extension made things so much easier for us.
Hopefully more about that in the future!

For now, i'd like to invite you to check out the end result and see for yourself.
If you have any questions, suggestions, free drinks or marriage proposals, feel free to reply!

/BruuD

Tokenizer / tag extraction method

I was in need for a method to extract a list of tokens from a given text.
The list should not contain stopwords like "the, a, on, of, etc" or punctuation marks and needs to be sorted by most used token.
This is what i came up with:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

public static class Tokenizer
{
  private class Token
  {
    public string Value { get; set; }
    public int Count { get; set; }
  }

  /// <summary>
  /// Tokenize a string and return tokens as list sorted by most occurances
  /// </summary>
  /// <param name="input"></param>
  /// <returns></returns>
  public static List<string> TokenizeString(string input)
  {
    string cleanText = String.Empty;

    // Replace all non-alphanumeric characters with a space
    new List<Char>(input.ToCharArray()).ForEach(c => 
                                                cleanText += (Char.IsLetterOrDigit(c) || Char.IsWhiteSpace(c)) 
                                                ? c.ToString() 
                                                : " ");

    // We want all tokens in lowercase
    cleanText = cleanText.ToLower();

    // Split input string on whitespaces
    List<string> tokens = cleanText.Split(' ').ToList<string>();

    // Remove stop words, whitespace and tokens shorter than 3 chars from token list
    // NOTE: Configuration.StopWords is of type List<string> and contains the stopwords (loaded from a configuration file)
    Configuration.StopWords.ForEach(stopWord =>
                                    tokens.RemoveAll(token =>
                                                     token == stopWord || String.IsNullOrWhiteSpace(token) || token.Length < 3));

    // Copy the tokens to a new list
    // Count how many times each token occurs and add it only once, so we have no double entries
    List<Token> tokenList = new List<Token>();
    tokens.ForEach(delegate(string t)
    {
      // Continue only if token does not exist in tokenList
      if (!tokenList.Exists(e => e.Value == t))
      {
        // Add token to list, including the count of how many times it occurred
        tokenList.Add(new Token { Value = t, Count = tokens.Count<string>(c => c == t) });
      }
    });

    // Sort the list on occurrance count
    tokenList.Sort(delegate(Token a, Token b)
    {
      return b.Count.CompareTo(a.Count);
    });

    // At this point we have a list with unique tokens, sorted by most occurrances
    // We convert it back to a string list and return it
    tokens = new List<string>();
    tokenList.ForEach(t => tokens.Add(t.Value));

    return tokens;
  }
}

Here is an example of it's usage:

// Dummy input string
string input = "Google-oprichters Larry Page en Sergey Brin willen af van hun gedeelde meerderheid aan aandelen van het bedrijf. Uit een bericht van de Amerikaanse beurswaakhond SEC willen de twee de komende vijf jaar tien miljoen van hun Google aandelen verkopen. Brin en Page, nu nog voor 59 procent eigenaar, bezitten dan nog maar 48 procent van de aandelen. Met de huidige koers levert dat hun in totaal 5,5 miljard euro op. De Google oprichters kiezen bewust voor een geleidelijke afbouw om de aandelen op de beurs niet teveel onder de druk te zetten. Ondanks de aandelenverkoop zullen Brin en Page nog aardig hun stempel op het beleid kunnen drukken. Naast henzelf zijn voor besluiten slechts krap twee procent van de overige aandeelhouders nodig.";

List<string> tokens = Tokenizer.TokenizeString(input);

This will result in 59 tokens with the first 5 being:

aandelen
google
brin
nog
procent


The list of stopwords that was loaded in Configuration.StopWords is: aan,naar,dat,nu,de,om,den,onder,der,ons,des,onze,deze,ook,die,op,dit,over,door,een,te,enige,tegen,enkele,ten,enz,ter,etc,tot,haar,uit,het,hierin,hoe,vanaf,hun,ik,vol,inzake,voor,is,wat,je,wie,na,zijn,u,uw,met,naar,jij,hij,zij,zonder,en,van,amp

Because i already had a list with dutch stopwords loaded, i have used a dutch text in the example (sorry english readers :)

/Ruud

C# Coding Standards

I have always been very keen on coding standards and as believe that a strict standard is key to writing maintainable code.
Today i stumbled upon a document that summarizes all the C# coding standards that i usually apply.
Thanks to Lance Hunt for publishing this document!

Get the document here: http://weblogs.asp.net/lhunt/pages/CSharp-Coding-Standards-document.aspx

/Ruud

Learn ASP.NET MVC in 2 hours! An introduction to MVC.NET

There is a new sheriff in town and it's called ASP.NET MVC!
This (fairly) new framework from Microsoft provides the possiblity to write applications that truely follow the MVC design pattern which allows clear separation of data, logic and user interface.
That strict seperation results in better testibility and maintanability.
I was very excited about this framework from the start, but didn't get into the details until now and i found that - once again - a blog series from ScottGu is a very good start for those who want to learn to use ASP.NET MVC.

So here we go: learn MVC.NET in 2 hours (depending on how fast you read of course)!

Part 1: ASP.NET MVC Framework
http://weblogs.asp.net/scottgu/archive/2007/11/13/asp-net-mvc-framework-part-1.aspx

Part 2: URL Routing
http://weblogs.asp.net/scottgu/archive/2007/12/03/asp-net-mvc-framework-part-2-url-routing.aspx

Part 3: Passing ViewData from Controllers to Views
http://weblogs.asp.net/scottgu/archive/2007/12/06/asp-net-mvc-framework-part-3-passing-viewdata-from-controllers-to-views.aspx

Part 4: Handling Form Edit and Post Scenarios
http://weblogs.asp.net/scottgu/archive/2007/12/09/asp-net-mvc-framework-part-4-handling-form-edit-and-post-scenarios.aspx

Thank you Scott!

And for those who can't get enough, there is more!
The latest version - ASP.NET MVC 2.0 - is now available as preview and Brad Wilson wrote a blog series about one of the major new features in this release: Templates.

Part 1: Introduction
http://bradwilson.typepad.com/blog/2009/10/aspnet-mvc-2-templates-part-1-introduction.html

Part 2: ModelMetadata
http://bradwilson.typepad.com/blog/2009/10/aspnet-mvc-2-templates-part-2-modelmetadata.html

Part 3: Default Templates
http://bradwilson.typepad.com/blog/2009/10/aspnet-mvc-2-templates-part-3-default-templates.html

Part 4: Custom Object Templates
http://bradwilson.typepad.com/blog/2009/10/aspnet-mvc-2-templates-part-4-custom-object-templates.html

Part 5: Master Page Templates
http://bradwilson.typepad.com/blog/2009/10/aspnet-mvc-2-templates-part-5-master-page-templates.html

Thank you Brad!

/Ruud

.NET trace log implementation with EventLog and TextStream support

When you are developing an application you want to be able to log what is happening in your code.

In .NET you can accomplish that with the use of TraceListeners.

You use the Trace class to write messages during the execution of your application.

While in debug mode, these messages will be displayed in the Visual Studio output console and forwarded to the configured tracelisteners.

The TraceListener can then write the message to a destination like an EventLog, Xml file or a textfile.

One cool thing about TraceListeners is that you can configure them in your App.config or Web.config, so you can control their settings after you have built the application.

I think this is a great tool for implementing logging in your application, but the default implementation has a few setbacks:

1) TraceListeners intercept all trace messages and i would like to configure them so they intercept only those messages that have a specific priority level.

Eg. i want high priority messages to be logged in an eventlog and i want debug messages to be logged in a textfile.

So when i use a release build and something goes wrong, i can activate the TraceListener that writes debug info to a logfile and find out what is happening.

2) The EventLogTraceListener outputs to the Application log and i would like to have my events in a seperate log.

So to solve these problems i wrote my own tracelisteners.

It would be great if Microsoft was so kind to let us inherit from their default tracelistener implementations like EventLogTraceListener and TextStreamTraceListener, but for some reason they decided to seal these classes.

Therefore we have to inherit from the TraceListener class, do our custom stuff and pass the calls to an instance of, for example, EventLogTraceListener.
I wrote two custom tracelisteners: CustomEventLogTraceListener and CustomTextStreamTraceListener.

In my implementation i also wrote a CustomTrace class that acts as a wrapper for the Trace class and adds the possibility of specifying priority levels for trace messages.

The lowest priority is 1 and the highest is 9.

So instead of calling Trace.TraceError("This is wrong") you can call CustomTrace.TraceError("This is wrong", 9) which means this error is of priority level 9, so it is very important.

I created an example solution with the TraceListeners project and an example application.

It shows how to configure the listeners and how to send information (messages) to them.

You can download it here: http://www.dottechdev.nl/downloads/CustomTraceListeners.zip

I will be using this in all my .NET implementations from now on and i hope it comes in handy for you too!

/Ruud