How to efficiently break your code by using exceptions?

At first I wanted to call today’s post something along the line of “Exceptions should be exceptional” or “keep your exceptions exceptional”. However a quick search on Google made me realize I was not the first to think of this catchy title.

Instead, I decided to go with “How to efficiently break your code by using exceptions?” and this made me think of a different way to present the potential negative effects of overusing exceptions in your codebase. Instead of telling you why you should be careful when using exceptions, I will focus on showing you how to use them “efficiently” in order to break your application in no time. I will be using C# for the examples below, but those could be applied to other languages supporting run-time exceptions, like Java for instance.

A word of notice

most of the examples below are inspired by production code from real applications that I had the chance to work on over the years. I made up the stories around them though, to add a little dramatic touch to the whole. Do try this at home if you want, but please don’t try this at work!

Example 1: Pokemon exception catching (catch ’em all!)

The full code of this example is available in this public gist.

Richard is the sole developer of a small, local vindication company. He was tasked to make the integration between:

  • the in-house application that generates vindication reports for the company’s customer
    and
  • a third-party library, the famous (and imaginary) Vindigator.

One of the main functionalities of Vindigator is the IsCustomerAGoodCustomer method that, thanks to “advanced algorithms”, determines if a given customer is a good customer or a bad customer:

/*
 * 'Vindigator' is an imaginary 3rd-party library that contains
 * a lot of useful (if you're into the vindication business) APIs.
 */
public static class Vindigator
{
    /// <summary>
    /// Return true if the customer is a good customer, false otherwise.
    /// </summary>
    public static bool IsCustomerAGoodCustomer(Customer customer)
    {
        if (customer == null)
            throw new VindigatorInvalidCustomerException("Something is wrong with your customer!");

        if (customer.AmountOwed < 1000)
            return true;

        return false;
    }
}

Vindigator has its own set of exceptions, for instance VindigatorInvalidCustomerException that gets thrown when something is wrong with a given customer. Richard finally finishes the integration, as shown by the GenerateVindicationReport method below:

public static VindicationReport GenerateVindicationReport(Customer customer, ValuationRates rates)
{
    var report = new VindicationReport { Customer = customer };

    try
    {
        report.IsGoodCustomer = Vindigator.IsCustomerAGoodCustomer(report.Customer);

        if (report.IsGoodCustomer)
            report.MoneyOwed = report.Customer.AmountOwed;
        else
            report.MoneyOwed = report.Customer.AmountOwed + report.Customer.AmountOwed * rates.PercentageFee;
    }
    catch (Exception)
    {
        // [Richard] if we get an exception here, it means the customer is null!
        // Seems to happen when the main vindication job gets stuck (damn John!). Return a null report in that case.
        return null;
    }

    return report;
}

During tests, Richard noticed that the scheduled job that calls his GenerateVindicationReport method once a day for each customer is somehow broken. “It must be John’s fault, once again!” says Richard. John is the IT administrator of the company and he was the one who set up the scheduled job. John and Richard generally do not get along very well.

Anyway, John’s job sometimes passes null customers as the first argument of the method. This obviously causes the IsCustomerAGoodCustomer method to throw an exception. Richard knows his topic well and decides to wrap the call in a generic try / catch block, If an exception occurs, meaning the customer passed is null, he will simply return a null report. All is well and the reports are generated perfectly!

A few weeks later though, Richard gets called by his CEO. It appears that the company lost around 70 000 dollars because a lot of reports for indebted customers have been missing! Richard checks the logs of the application, but obviously nothing is there. It is only after 2 stressful days of debugging that Richard realizes that John’s job also occasionally sets the second argument of GenerateVindicationReport to null:

var report = GenerateVindicationReport(customer343, null);

This obviously causes the following line to throw a NullReference exception that gets swallowed by Richard’s generic catch block:

report.MoneyOwed = report.Customer.AmountOwed + report.Customer.AmountOwed * rates.PercentageFee;

This means that during all those weeks, perfectly valid reports for indebted customers were being ignored because of a generic Catch’em all block, leading to a significant loss of money!

This was a very costly and powerful lesson for Richard, who swore not to write such a try / catch block again.

Example 2: short-circuiting exception handling

The full code of this example is available in this public gist.

Months have passed and Richard is now much better at dealing with all types of exceptions, as the following snippet shows:

public static void SetupReportGeneration()
{
    try
    {
        VindicationUtils.CreateVindicationReportDirectory("Customers_Batch1");
        // The rest of the init goes here...
    }
    catch (FolderCouldNotBeCreatedException ex)
    {
        HandleFailureToCreateMainReportDirectory();
        Log(ex);
    }
    catch (Exception ex)
    {
        Log(ex);
    }
}

private static void HandleFailureToCreateMainReportDirectory()
{
    // this method creates a temporary directory to store the reports, in case the main directory couldn't be created
}

private static void Log(Exception ex)
{
    // Log the exception to a file and send an email alert to the admin...
}

All exceptions are logged so that Richard knows exactly what went wrong, and known exceptions are gracefully handled. For example, the custom FolderCouldNotBeCreatedException is being handled by calling the HandleFailureToCreateMainReportDirectory method.

FolderCouldNotBeCreatedException is sometimes thrown by the VIndicationUtils.CreateFolder method shown below:

public static class FileUtils
{
    public static void CreateFolder(string path, string foldername)
    {
        // check if the path is valid
        if (!IsPathValid(path))
            throw new FolderCouldNotBeCreatedException(path + "/" + foldername);

        // The rest of logic to create the folder goes here...
    }
}

The method itself is being used in VindicationUtils.CreateVindicationReportDirectory:

public static class VindicationUtils
{
    public static void CreateVindicationReportDirectory(string foldername)
    {
        FileUtils.CreateFolder(FileUtils.DefaultPath, foldername + "_" + DateTime.Today.ToShortDateString());
        // The rest of the logic to create the report directory goes here ...
    }
}

Everything is working well in production, until Alexandro, the new intern of the company, decides to refactor the code a bit. Alexandro’s current task is to create a web version of the report generator. One of the methods he wants to reuse to this purpose is VindicationUtils.CreateVindicationReportDirectory. However Alexandro quickly notices that it throws FolderCouldNotBeCreatedException from time to time, and remembers the advice of Mr Smith, his university teacher:

“always handle exceptions as close as possible to their source!”

With this precious knowledge in mind, Alexandro quickly refactors the method to this:

public static class VindicationUtils
{
    public static void CreateVindicationReportDirectory(string foldername)
    {
        try
        {
            FileUtils.CreateFolder(FileUtils.DefaultPath, foldername + "_" + DateTime.Today.ToShortDateString());
        }
        catch(FolderCouldNotBeCreatedException)
        {
            throw new InvalidOperationException("Oops! An error occurred during the creation of the report directory.");
        }

        // The rest of the logic to create the report directory goes here ...
    }
}

Alexandro is proud of his contribution! He followed the Boy Scout Rule and made the codebase a bit cleaner by handling the exception close to the source and by throwing a new exception that explicitly says there was an error during the creation of the directory.

By doing that, Alexandro also unknowingly broke another part of the application by short-circuiting Richard’s own exception handling one level above!

However Richard cannot really blame Alexandro for this. Alexandro didn’t introduce any breaking change in the interfaces that would cause the compilation to fail. All the methods have the exact same signature as before. And that’s the dangerous thing about exceptions:

Throwing exceptions will make your code lie about itself! You cannot trust the code to tell you what exception it will throw or handle.

Moral of today’s stories?

I have more examples in mind about how to use exceptions to break your application. Let’s keep them for another post though, as this one is getting pretty long already!

Even though the concept of throwing and catching exceptions is pretty simple to grasp, I consider that proper exception handling is one of the most difficult things to achieve in our profession. Don’t count on the compiler for helping you with it. Only thorough testing and proper team communication will save you here.

I hope that the two examples above are a good reminder that you should always keep on your toes when using exceptions anywhere (!) in your code base.

Cheers!

Creating a reactive Single-Page App in F# with WebSharper UI.Next and Bootstrap – Part 2

Hello again! This is the second post on our attempt to make a dynamically editable Bootstrap navbar using WebSharper’s UI.Next library. If you haven’t done so yet, I recommend you start by reading the first part here where I cover the basic of the application (reactive Single-Page App) and the different HTML files used. AS a reminder, here’s how the final application looks like:

(click on the .gif to enlarge)
Let’s jump right back where we left off and go over the F# logic.

Client.fs

I’m going to divide this file into smaller parts to make it easier to follow. Here’s the first part:

namespace UI.Next.Demo

open WebSharper
open WebSharper.UI.Next
open WebSharper.UI.Next.Client
open WebSharper.UI.Next.Html

[<JavaScript>]
module Client =
    
    type NavbarSampleTemplate = Templating.Template<"navbar_sample.html">
    type BsNavbarTemplate = Templating.Template<"bootstrap_navbar.html">

    type Tab = { Name: string; Url: string; IsActive: Var<bool> }

    let sampleTabs = [
        {Name = "First tab"; Url = "/#"; IsActive = Var.Create true }
        {Name = "Second tab"; Url = "/#"; IsActive = Var.Create false }
    ]

    let Main =
 // ... to be continued

Except for the bunch of necessary open statements and the creation of a new Client module that will transpile to JavaScript thanks to WebSharper’s [<JavaScript>] attribute, we create:

  • 2 types based on our HTML templates above, NavbarSampleTemplate and BsNavbarTemplate.
  • 1 type called Tab that will contain the Name and Url of new items we will be adding to our navbar, as well as a reactive flag IsActive to determine if the item is active or not.
  • Finally, we create a list of 2 tabs that will so we can populate our navbar when the page is launched.

Let’s continue with the Main function:

let Main =
        
	// Reactive variables
	let brand = Var.Create "Play With UI.Next!"
	let newTabName = Var.Create ""
	let newTabUrl = Var.Create ""
	let tabs = ListModel.Create (fun t -> t.Name) sampleTabs

We declare all the reactive variables that we will be using in our app. As you can see, a reactive variable of type String can be declared as follows:

let myReactiveVariable = Var.Create "Some string"

UI.Next also allows us to create reactive collections by using ListModel.Create. The line below instantiate a reactive collection that contains the 2 tabs that we declared in the previous snippet:

let tabs = ListModel.Create (fun t -> t.Name) sampleTabs

We will use this reactive collection to dynamically update the navbar when a new item is being added to it. In the next snippet, we declare a small function called mapTab that allows to map a given Tab instance to its HTML template defined by BsNavbarTemplate:

/// create a new tab (name and url) from the HTML template.
let mapTab tab = 
    BsNavbarTemplate.Tab.Doc(
        Name = View.Const tab.Name,
        Url = View.Const tab.Url,
        ActiveClass = Attr.DynamicClass "active" tab.IsActive.View id,
        SetActive = fun _ _ ->
            tabs.Iter (fun t -> if t.Name <> tab.Name then Var.Set t.IsActive false)
            Var.Set tab.IsActive true
    )

The function returns a Doc, meaning the result can be rendered as HTML. I used View.Const to map the name and URL of the tab to the $!{Name} and $!{Url} text holes respectively. Const means that the value will be static and won’t be observed over time (once the tab is added to the navbar, you cannot edit its properties anymore).

The very useful Attr.DynamicClass function allows us to dynamically add or remove the class=”active” css attribute to the HTML content generated. When tab.IsActive equals true, this results in:

<li class="active">...</li>

However when tab.IsActive equals false, the following HTML fragment is generated instead:

<li>...</li>

This is very convenient as we don’t have to explicitly update the HTML by ourselves when a different navbar item is clicked.

The HTML type provider in action!

There is one extra concept worth mentioning here: both NavbarSampleTemplate and BsNavbarTemplate are creating using something that we call a type provider in F#. This means that the properties Name, Url, ActiveClass and SetActive that you see in the snippet below are actually extracted from the HTML template at compile time, and therefore provide additional type-safety to our program. Let’s see it in action in the .gif below:

(click on the .gif to enlarge)

That’s right! After we change the HTML attribute from ActiveClass to ActiveClasssss, the compiler throws an error saying that it doesn’t know of the ActiveClass property in F#! No more typing issues that you might (will) experience in other languages. Type-safety for the win!

Putting everything together

We’re almost there! The last bit of F# code will cover:

  • how to instantiate the initial navbar.
  • how to dynamically add a new item to the navbar.
  • how  to wire up the whole page and run it under <div id=”main”/> in our index.html file.

Instantiating the navbar

I created a separate initNavbar function for that:

/// instantiate the navbar.
let initNavbar = 
    BsNavbarTemplate.Doc(
        Brand = brand.View,
        Tabs = [
            tabs.View |> Doc.BindSeqCached mapTab
        ]
    )

In this function, the reactive collection of navbar items (called tabs) is mapped to our Tabs data hole (data-hole=”Tabs” in bootstrap_navbar.html). To do that, we use the UI.Next’s Doc.BindSeqCached function that converts a collection to a Doc type. I am not quite sure why the function contains the Cached keyword though. This is something I need to look at.

Adding a new item to the navbar

This part is straightforward. We create a new item as inactive based on the name and URL text inputs shown below:

We then add the item to the tabs collection and reset both text inputs using the Var.Set function. This is shown in the snippet below:

/// add a new tab to the navbar and reset the input.
let addNewTab () =
    tabs.Add { Name = newTabName.Value; Url = newTabUrl.Value; IsActive = Var.Create false }
    Var.Set newTabName ""
    Var.Set newTabUrl ""

Running all components under the main div element

The last part is also easy to understand. We map all the elements we created to the template declared in navbar_sample.html. We then run the whole thing under the main div element:

// Wire up the whole page and run it under the 'main' div.
NavbarSampleTemplate.Doc(
    Brand = brand,
    NavBar = [initNavbar],
    NewTabName = newTabName,
    NewTabUrl = newTabUrl,
    CreateNewTab = fun _ _ -> addNewTab()
)
|> Doc.RunById "main"

Here again we can benefit from the HTML type provider that we described above. The last line allows us to inject the whole Doc under the main HTML element, thanks to the Doc.RunById function provided by WebSharper. Voila!

Wrapping up

That’s it for the second part of this mini-series. We managed to create a bootstrap navbar that we can dynamically update. The whole thing was wrapped into a Single-Page app with reactive elements.

Putting this together took me longer than I’d expected, mainly because the topic was new to me (I am more of a back-end developer myself, remember!) and I got a little lost in the documentation and the various APIs for manipulating ListModel reactive collections.

Also, most if not all examples featured on UI.Next’s samples page were creating HTML structures in F# directly, instead of reading it from real HTML templates, like I did here.

I feel however that the HTML template approach is cleaner when you understand it well, and this gives more freedom to your designers to tweak the visuals of the page independently from the F# logic.

And most importantly, this was a lot of fun and I didn’t have to write a single line of JavaScript! What else can I ask for?

Thanks for reading and until the next one!

Creating a reactive Single-Page App in F# with WebSharper UI.Next and Bootstrap – Part 1

Wow! That’s a lot of buzzwords we got here in that title! I’m not lying though, this is exactly what we are going to cover in today’s post. I hope you are familiar with all the terms above, but in case you’re not, let me start by explaining some of them.

Single-Page Application

Or SPA for short. A particular type of website that consists of a single HTML page that gets updated dynamically as the users interact with it. A good sign that a given website is a SPA is that it is not being refreshed every time you click on a link. This gives an experience similar to when using a desktop application. Gmail is a great example of a SPA.

Reactive Programming

Or RP for short. To keep things simple, it is a programming paradigm where you define data-sources that can be auto-magically observed. If the data-sources get updated (say, by a user or an external process), the elements that observe those data-sources will get notified and automatically react to the changes. The following pseudo-code illustrates a traditional, non-reactive approach:

var a <= 1
var b <= 2
var c <= a + b
echo c // c equals 3
a <= 2
echo c // c still equals 3

The next snippet illustrates the reactive approach however. Here c gets updated automatically every time a or b change, thus:

var a <= 1
var b <= 2
var c <= a + b
echo c // c equals 3
a <= 2
echo c // a changed, so c now equals 4!

This is obviously a very naive example and I encourage you to encourage you to do your own research about the topic and make your own opinion of it. There is a lot of information (and debates!) available online. I particularly like this presentation because it gives a global overview of different flavours of Reactive Programming.

Back to our application. We are going to use WebSharper UI.Next which is a library that enables both Reactive Programming and SPA within WebSharper. You can basically write entire reactive Single-Page apps from the comfort of your own F# editor! Isn’t it cool? Even Keanu Reeves approves!

Creating a dynamic Bootstrap navbar

That’s right. We will make a small application that allows users to dynamically customize a Bootstrap navbar:

Why Bootstrap? Because it is good-looking and easy enough so that even a back-end developer can use it! Before we go into the details of the implementation, let’s see how the app looks like:

(click on the .gif to enlarge)The application effectively offers the following functionalities:

  • Dynamically update the navbar-brand component by typing its name in a text input below.
  • Add a new item to the navbar by defining its name and URL.
  • The 2 text inputs along the Create new tab button get reset automatically upon adding a new item.
  • The new items are clickable and contain a working hyperlink. In the demo above, most hyperlinks are set to ‘#’, which will point to the current page. However I set the value of the last hyperlink to ‘test’ for the last item. As this URL doesn’t exist, a 404 error page is being shown instead.
  • Navbar items are set as active when clicked (different appearance than the inactive items), rendering all other items as inactive.

During the implementation, I used the term ‘Tab’ to refer to those navbar items. I came to realize that this is a bit misleading and I will probably update it in the future. My apologies for the potential confusion!

As usual, you will find the source code associated with this post on my GitHub repo.

The implementation

Let’s start by looking at the project’s structure. There are 4 files of interest:

  • index.html / 16 lines -> the HTML skeleton of the page.
  • bootstrap_navbar.html / 14 lines -> a file containing the default markup for the navbar. We will use it as a template for WebSharper later in this post.
  • navbar_sample.html / 23 lines -> a file containing the markup for the rest of the controls on the page (the 3 text inputs and the button below the navbar).
  • Client.fs / 63 lines -> the F# file containing the whole logic behind the application.

That’s 116 lines of code and markup I had to write for the entire application. The details of each file are presented below:

Index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <title>Dynamic BootStrap NavBar</title>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <!--<link rel="stylesheet" type="text/css" href="Content/UI.Next.Demo.css" />-->
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" />
    <script type="text/javascript" src="Content/UI.Next.Demo.head.js"></script>
</head>
<body>
    <div id="main"/>

    <script type="text/javascript" src="Content/UI.Next.Demo.min.js"></script>
</body>
</html>

Nothing out of the ordinary here. We use the hosted version of Boostrap here, and Content/UI.Next.Demo.min.js contains the minified JavaScript logic of our application (transpiled from the F# code in Client.fs). The <div id=”main” /> will serve as an injection point for our navbar and custom controls.

bootstrap_navbar.html

<div id="navbar-template">
    <nav class="navbar navbar-inverse">
        <div class="container-fluid">
            <div class="navbar-header">
                <a class="navbar-brand" href="#">$!{Brand}</a>
            </div>
            <ul class="nav navbar-nav" data-hole="Tabs">
                <li data-template="Tab" data-attr="ActiveClass">
                    <a data-event-click="SetActive" href="$!{Url}">$!{Name}</a>
                </li>
            </ul>
        </div>
    </nav>
</div>

This file contains the standard markup for a bootstrap navbar, decorated with a few WebSharper-specific attributes. All those elements can be directly accessed and manipulated from our F# code.

  • data-hole=”Tabs” defines a hole called Tabs that we can use as an injection point. If you want more information on the topic please check my other post here.
  • data-template=”Tab” defines an HTML template called Tab. The template corresponds to a single HTML <li> element with a nested <a href> .
  • data-attr=”ActiveClass” defines something similar to the data-hole above, but for injecting an HTML attribute instead. This is something new and very useful that I haven’t covered in my previous posts.
  • data-event-click=”SetActive” is another custom WebSharper attribute that allows us to define an action called SetActive. The action will get called when the user clicks on the associated <a href> link. Once again, we can access and setup this action directly from our F# code!
  • $!{Brand}, $!{Url}, and $!{Name} are the reactive equivalents of text holes that I covered in this previous post. If you don’t know what those are, they basically allow to display the values of their associated reactive variables. We will declare those reactive variables (In our case, Brand, Url, and Name respectively) in the Client.fs F# file.

navbar_sample.html

<div id="displayNavBar" data-replace="NavBar" />

<form class="form-horizontal">
    <div class="form-group">
        <div class="col-md-2">
            <input data-var="Brand" type="text" placeholder="Enter NavBar's brand" class="form-control" />
        </div>
    </div>

    <div class="form-group">
        <div class="col-md-2">
            <input data-var="NewTabName" type="text" placeholder="Name of the new tab" class="form-control" />
        </div>

        <div class="col-md-2">
            <input data-var="NewTabUrl" type="text" placeholder="Url of the new tab" class="form-control" />
        </div>

        <div class="col-md-2">
            <button type="button" data-event-click="CreateNewTab" class="btn btn-success">Create new tab</button>
        </div>
    </div>
</form>

This file contains the HTML markup for the controls available to customize the navbar. There are a few more WebSharper attributes defined in the file:

  • data-var=”Brand” binds whatever value entered in the associated text input to the same Brand reactive value described above. It means that any change made to the reactive variable will be reflected in real time everywhere where the $!{Brand} text hole is used.
  • The same logic applies to data-var=”NewTabName” and data-var=”NewTabUrl” except that those are mapped to $!{Name}, and $!{Url} respectively.
  • Finally, there is one more data-event-click attribute with data-event-click=”CreateNewTab”. We will use this action to define the logic of creating a new navbar item.

That’s it for the HTML part, I believe this has been straightforward so far.We’ll cover the F# logic in the next post as things will get a bit more complicated!

To be continued!

Load-testing the PizzaManager REST-like API with JMeter

Today we are going to do something different. Today is load testing day! Today is also the day when we will be borrowing something from the enemy Java World: Apache JMeter. What is JMeter? Let me shamelessly paste here the definition from their website:

The Apache JMeter™ application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.

But by the way, what is load testing? To keep it simple, it’s a specific domain of testing aimed at measuring how a given system behaves when being used by a relatively large number of simultaneous users. What large number means depends entirely of the goals you want to achieve. For instance, it might be written somewhere in the specifications of your project that “the application must handle up to 100 concurrent users with a maximum response time of 1s and an average response time of 200ms”. Load testing, and tools like JMeter, can help you prove that your application just does that. And if it doesn’t, then you might want to either review your specs or get to work on that performance!

My objective for today’s post is twofold:

  • Get more familiar with load testing in general. This is something I’ve never done outside of purely academic circles (read, at school) and I would like to introduce it to my day-to-day job. I think it’s bad not to test such things but let’s be honest, performance concerns are often pushed to the far end of the backlog until, well surprise surprise, things start to get ugly. Who would have thought that several users would use your app at the same time, right?

  • Get started with JMeter. I requested it at work in the hope to finally be able to use it in my current project, so I figured I might take the beast for a ride at home first. My Java colleagues said good things about it and it is free, so why not?

That being said, let’s load test!

Installing and running JMeter

First things first, let’s grab JMeter from their official website:

You can unzip it somewhere on your local machine and run the GUI client by using the ApacheJMeter.jar file from the bin folder, as shown below:


Setting up a new test plan

JMeter greats you with the following screen and an empty test plan:

Let’s create a test plan that does the following:

  • Create 100 users that will constantly call our PizzaManager REST-like API for 10 seconds. The API consists of 5 methods:
    • Get a pizza by name
    • Get all pizzas (names only)
    • Get all pizzas (full details)
    • Add a new pizza to the system
    • Delete a pizza by name
  • Measure how our system behaves, and more particularly the following elements:
    • Response time: the time elapsed from the moment a request is sent to the moment its response has been received. The lower the better.
    • Throughput: the number of requests the system is able to process per second. The higher the better.

Creating users

Let’s add the first JMeter component to our test plan: a Thread Group. This component will be responsible for creating the fake users who will test your application. Each user will be represented by a separate, independent thread, thus the name of the component.

(click on the .gif to enlarge)

We now need to configure the Thread group according to our requirements above:

The Number of Threads (users) parameter is explicit. Setting it to 100 will create 100 new users who will call our API in a concurrent fashion. I set the Loop Count parameter to Forever so the users keep calling the API in loop. We could have defined a predefined value instead, for instance 3. In that case each user would have made exactly 3 calls to the API. Finally, by enabling the scheduler, we can define the Duration (seconds) parameter. Here I set it to 10 so the users keep calling the PizzaManager API for 10 seconds.

It’s also worth mentioning the Ramp-up parameter. You can use it if you want to progressively reach the number of users over a given period of time. The graph below would represent the progressive creation of 100 users over a ramp-up period of 5 seconds:

By setting the parameter to 0, I declare that I want all my users to be created from the very start of the test.

Calling the API

In order to access our PizzaManager REST-like API, we need some kind of component to make create and send HTTP requests. Obviously JMeter is well equipped to do so. There are 2 components that we will be looking at: HTTP Request Defaults and HTTP Request.

the HTTP Request Defaults component is available under the Config Element category:

It allows to configure default HTTP settings for all the HTTP requests added to the current Thread Group. Here’s how ours looks like:

We just defined the protocol, server name and port number we want to use to access our API (http://localhost:9000/).

We can add our second component type, HTTP Request. It is available under the Sampler category.

It looks very similar to the previous component, but here we can define the exact path to our the methods we want to call as well as the appropriate HTTP verbs. Here’s the test plan hierarchy after I’ve added all 5 methods that we want to test:

And here are the details for each element, in the same order as above:

The POST method is a bit particular because it also features a JSON body, as shown below:

And that’s it for the configuration of our calls. Time to go over the last step: measuring!

Measuring performance

The last category of components of interest to us today is what JMeter calls listeners. Those components measure criteria like response times and throughput and display the results to the users. The full list of listeners is available here. We will be using the listener called Summary Report here. The screenshot below shows how to add it to our test plan:

Running the test plan

It’s time to run our test plan! In order to do so, just click on the play button in the ribbon:Here it is in action!
(click on the .gif to enlarge)

As you can see on the top right corner, the test runs for 10 seconds and create 100 users, as we defined in the Thread Group. Let’s go over the results of the execution (I excluded some other columns for clarity purposes):

In the 10 seconds the test ran, over 8100 HTTP requests were sent to the PizzaManager (roughly 1600 requests per method), resulting in an overall throughput of 804 operations per seconds. The average response time was around 120ms with peaks between 350 and 400ms. If we were to compare those results with our fake requirements from above (“the application must handle up to 100 concurrent users with a maximum response time of 1s and an average response time of 200ms”), we could say that our application runs fast enough to satisfy the specs.

Of course, this is just an introduction and we might want to run more tests with a different amount of users and various durations to make sure that everything works as expected.

Small bonus – functional testing

Aside from load testing, you can also do functional testing with JMeter. In order to do so, you can add the View Results Tree listener to your test plan. This will record all HTTP requests made along with their responses. You can use it to make sure the API works as expected from a business point of view. If the screenshot below, we can see that one of the DeletePizza requests failed with a 500 HTTP status code (Internal Server Error).

After investigation, it occurred that the following code was causing an error:

let private pizzas = new Dictionary<string, Pizza>()

let deleteCI (name:string) =
    let pizza = pizzas.Keys |> Seq.tryFind (fun p -> p.ToLower() = name.ToLower())
    match pizza with
    | None -> Failure(sprintf "Failed to delete '%s'. The pizza could not be found!" name)
    | Some p -> 
        pizzas.Remove(p) |> ignore
        Success(sprintf "'%s' was successfully deleted from the database." name)

This is because the pizza was found at the beginning of the function was being removed in the meantime by another user, before the following line was called:

pizzas.Remove(p) |> ignore

The View Results Tree component also allows use to visualize the JSON data sent as a response to a request:

This is also a useful feature for checking if your API returns the expected results.

That’s it for today’s post. I hope you found this introduction to JMeter useful. It seems to be a very powerful tool and I am looking forward to using it more regularly in the future! Please let me know if you have more experience using JMeter or other similar Software. What features do you use? How often do you use it? Do you use it in some automated way? I’d be very interested in your feedback.

Cheers

Lambda Days 2017 – My conference round-up

      No Comments on Lambda Days 2017 – My conference round-up

Almost 3 months ago, on February 9 and 10, the 2017 edition of Lambda Days, the conference dedicated to Functional Programming, took place in Kraków, Poland. I had the chance to attend the event with 3 of my friends and colleagues (thank you Infusion for covering all the expenses!).

The least I can say is that I don’t regret it! The 2 days were packed with awesome sessions, great speakers, and a very responsive audience. Not to mention the food (honestly, that probably was the best meringue pie I’ve ever had!). You can check the full program here.

Much water has flowed under the bridge since then, but I wanted to write a post about what were my favourite sessions and why. All the conferences were recorded so we will be able to watch (or re-watch?) them if you want to. Without further due, here are my top 3 talks. Enjoy!

***** Number 1 *****
Implementing Event-Driven Microservices Architecture using FP

Speaker: Nikhil Barthwal
Main topics: F#, Event Sourcing, Microservices, Cloud computing
Duration: 26 minutes

Why did I like the talk?
This was most probably the most pragmatic talk of all that I attended. Nikhil Barthwal from the startup Jet.com explains how F# and FP paradigms allowed them to scale from 0 to 8 million customers in around one year and a half! This shows that F# is often wrongly associated to scientific computing only, while in fact it can be successfully used in a large-scale production environment. A must-watch for sure.

*** Number 2 ***
Building the systems of tomorrow

Speaker: Txus Bach
Main topics: Idris, correctness of Software, type systems
Duration: 34 minutes

Why did I like the talk?
In this session Txus Bach talks about the importance of writing correct Software and the dangers of bugs and bad code. How to write correct programs? Txus Bach suggests we start by picking the right tools for the job: among others better type systems. Very nice presentation, check it out! Little bonus, the guy with the French accent asking the 2nd question at the end is me. I had completely forgot about it!

* Number 3 *
Building a security testing service with F#

Speaker: William Blum
Main topics: F#, Why use functional programming,
Duration: 45 minutes

Why did I like the talk?
Because William Blum is French shows well what are the strong points of F# and why you can and should use it if you want to bring value to your product very fast and in a safe way. The speaker presents Project Springfield, a security service built in F# and available from Microsoft’s cloud. The 2nd part of the session contains a lot of small samples of F# features that made me love the language when I first started. Accessible and well rounded talk.

And that’s it for my personal selection! All other videos are available from the program here.

Big thanks to the organizers of the conference, and hopefully see you next year!

Cheers

F# vs C#: a performance comparison with BenchmarkDotNet

[EDIT – 02/05/2017] I received quite a lot of comments regarding this post about how it does not fairly reflect the differences between F# and C# and also suggesting possible performance optimizations. I want to highlight from the get-go that this was done on purpose (this is explained in the post below but was maybe not explicit enough). My goal was to compare the traditional OOP approach of C# with the more FP’ish approach of F# on a real use-case. I left out any performance optimization from both versions because the aim was not to optimize code – but rather to show how both versions compare if we write them in a simple and naive way.


Ever since I became interested in functional programming, I’ve read and heard multiple times something along the lines of “F# (or functional programming in general) is slow and that’s why you should stay with OOP languages that guarantee good performance, like Java, C# or C++”.

Because I am curious by nature, I decided to see for myself if indeed I should sacrifice F# on the altar of performance. Specifically, there are two parts of the sentence above that deserve our attention:

“F# is slow”

What does it mean for a programming language to be slow? How can we measure the slowness of a given language? In what context? On what machine, OS, or architecture? Under what conditions? This is definitely not an easy question. Let’s try to put it in a more reasonable way: Is F# so much slower than other OOP languages like C# that we should automatically refrain from using it?

This is easier to answer already. We can definitely measure if a language is slower than another and we can also judge if the difference in performance is something we can live with. This is precisely what we’ll try to do in this post.

“OOP languages guarantee good performance”

Thankfully this is an easy one to debunk. I’ve seen enough systems written in C# that were riddled with performance issues to know this is not the exclusivity of FP languages. I trust in the C# programmers’ ability to create slow and unusable code, too. The same applies to Java and C++, of course.

Actually, there is one thing OOP languages are ultra fast at: introducing bugs in the codebase! I still remember a discussion from a few years ago with some of my former colleagues: “This module sure doesn’t work, but look at how fast it runs!”.

All jokes aside, I hope you get my point. Some languages may be faster than others, but they do not automatically guarantee you good performance. This is something we, programmers, must take care of. With that said, let’s benchmark!

BenchmarkDotNet to the rescue!

We are lucky! The fantastic BenchmarkDotNet tool supports both C# and F# (and also VB if it’s something you’re into). So we will be using it today. But first, we need to know what we will be measuring. I didn’t want to do yet another benchmark comparing the speed of strings concatenation, so I came up with a more realistic business case. We will be comparing the performance between F# and C# for the following tasks:

  • Generate a list of customers
  • Iterate through the list of customers and set as VIP those whose seniority is greater than 2 years.

To have a little more to discuss, we will conduct the experience twice, first with 100 000 customers, and finally with 1 000 000 customers.

Obviously, I wrote the C# and F# samples using different approaches. Therefore:

  • The C# sample uses good ol’ for loops, mutable classes and standard lists.
  • The F# sample uses recursive functions, immutable records and immutable lists.

I wrote both versions in a naive way, with no attempt at performance optimization! Also, before I started benchmarking, I knew that F# would most probably be slower than C# in all cases due to the additional allocations required to achieve immutability, but I had no idea how much slower it would be.

As usual, the full code is available on my GitHub repo here (C# version) and here (F# version).

The C# solution

Let’s have a look at the general skeleton for the C# solution first:

namespace BenchmarkDotNetCsharp
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var summary = BenchmarkRunner.Run<PerformanceTests>();
            Console.WriteLine(summary);
        }
    }

    [MemoryDiagnoser]
    public class PerformanceTests
    {
        private List<Customer> _customers;

        [Benchmark]
        public void CSharp_GenerateCustomers()
        {
            // TODO implement
        }

        [Benchmark]
        public void CSharp_ModifyCustomers()
        {
            // TODO implement
        }
    }

    public class Customer
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public int Age { get; set; }
        public bool IsVip { get; set; }
        public DateTime CustomerSince { get; set; }
    }
}

The structure is really simple. The Program class runs our benchmark using

BenchmarkRunner.Run<PerformanceTests>();

The PerformanceTests class contains the two methods that we want to benchmark, CSharp_GenerateCustomers and CSharp_ModifyCustomers. They are decorated by the Benchmark attribute from BenchmarkDotNet. And finally we have the straightforward Customer class that contains 5 properties.

The customers (respectively 100 000 and 1 000 000) will be held in a standard .NET list:

private List<Customer> _customers;

The F# solution

Now to the F# equivalent:

type Customer = {
    Id: int
    Name: string
    Age: int
    IsVip: bool
    CustomerSince: DateTime
}

[<MemoryDiagnoser>]
type PerformanceTests_ImmutableList() =
    
    let mutable _customers : Customer list = [] // Immutable F# linked-list
   
    [<Benchmark>]
    member this.FSharp_GenerateCustomers_Immutable() =
        // TODO implement

    [<Benchmark>]
    member this.FSharp_ModifyCustomers_Immutable() =
        // TODO implement
  
[<EntryPoint>]
let main argv = 
    BenchmarkRunner.Run<PerformanceTests_ImmutableList>() |> printfn "%A"
    Console.ReadKey() |> ignore
    0

Damn, I love the conciseness of the F# version! Here we have an immutable Customer record type containing the same fields as its C# counterpart, followed by PerformanceTests_ImmutableList. the class we want to benchmark. It contains both FSharp_GenerateCustomers_Immutable and FSharp_ModifyCustomers_Immutable, also decorated by the Benchmark attribute. The customers will be stored in a F# immutable list:

let mutable _customers : Customer list = [] // Immutable F# linked-list

The reference to the list itself is mutable, purely because we want to assign it from the first function and reuse it in the second one.

In case you were wondering, the MemoryDiagnoser attribute from BenchmarkDotNet allows to retrieves additional information about memory (Gen0, Gen1 and Gen2 allocations). This is the setup I used for the benchmarks:

But enough talking, let’s the fight begin!


Generating customers

This is the implementation of the C# method that generates customers:

[Benchmark]
public void CSharp_GenerateCustomers()
{
    var numberToGenerate = 1000000;
    var random = new Random(10);
    var customers = new List<Customer>();
    
    for (int index = 1; index <= numberToGenerate; index++)
    {
        var customer = new Customer()
        {
            Id = index,
            Age = random.Next(18, 100),
            Name = _names[random.Next(0, _names.Length)],
            IsVip = _booleans[random.Next(0, _booleans.Length)],
            CustomerSince = DateTime.Today.AddMonths(-random.Next(0, 121))
        };

        customers.Add(customer);
    }

    _customers = customers;
}

And here’s the F# equivalent:

[<Benchmark>]
member this.FSharp_GenerateCustomers_Immutable() =
    let numberToGenerate = 1000000
    let random = Random(10)
    
    let rec loop customers max current =
        if current > max then
            customers
        else
            let customer = {
                Id = current
                Age = random.Next(18, 100)
                Name = names.[random.Next(0, names.Length)]
                IsVip = booleans.[random.Next(0, booleans.Length)]
                CustomerSince = DateTime.Today.AddMonths(-random.Next(0, 121))
            }

            loop (customer::customers) max (current+1)
       
    // Assign the results to our F# immutable list of customers
    _customers <- loop [] numberToGenerate 1

In both cases, the code produces the exact same data (the Random instance uses the same seed).

Results for 100 000 customers

Below are the results for the generation of 100 000 customers:

And this is the legend for all columns:As you can see the performance is almost the same: 98.4ms for C# versus 101.6ms for F#. We can also notice that there’s a lot more GC collections on F#’s side, but fortunately no Gen2 collection in sight!

I call it a tie! Score: C# 1 / 1 F#

Results for 1 000 000 customers

Below are the results for the generation of 1 000 000 customers:

Here we obtain the following  timings: 1.01s for C# versus 1.12s for F#. F# is almost 11% slower than C# in that specific case. Nothing too bad but a difference still.

Let’s give the point to C# here. Score: C# 2 / 1 F#

Modifying customers

Below is the C# code that iterates through the list and updates customers if they have a seniority greater than 2 years:

[Benchmark]
public void CSharp_ModifyCustomers()
{
    var today = DateTime.Today;

    foreach (var customer in _customers)
    {
        // If customer for more than 2 years
        if ((today - customer.CustomerSince).TotalDays > 365*2)
        {
            customer.IsVip = true;
        }
    }
}

Straightforward. Here’s the F# version:

let setVip today customer =
    match (today - customer.CustomerSince).TotalDays > 365. * 2. with
    | true -> { customer with IsVip = true }
    | false -> customer

[<Benchmark>]
member this.FSharp_ModifyCustomers_Immutable() =
    let today = DateTime.Today

    _customers
    |> List.map (setVip today)
    |> ignore

As F# Customers are immutable, we need to return a new instance every time we need to update IsVip. This is done here:

// ...
| true -> { customer with IsVip = true }
// ...

Results for 100 000 customers

Let’s check the results for 100 000 customers first:

As I suspected, the difference starts to show here. At 4,61ms vs 1,40ms, F# is almost 3.3 times slower than C#. The most significant difference is on the memory side though: zero collections for C# (the modifications are made in-place) while F# has quite a lot of Gen0 and Gen1 collections! This is due to the immutability of both the F# list and the Customer type.

Another point for C#!
Score: C# 3 / 1 F#

Results for 1 000 000 customers

This time we go iterate through the list of 1 000 000 customers!

Now, that’s a huge difference! 14.21ms for C# versus 212.22ms for F#. This time F# is almost 15 times slower than C#, with again a significant amount of Gen0 and Gen1 collections. This is a clear win for C#!

Final Score: C# 4 / 1 F#

Analyzing the results

Now that we have a slightly better view on the topic, let’s go back to our original concern then: Is F# so much slower than other OOP languages like C# that we should automatically refrain from using it?

The first part of the answer is Yes, F# is slower than C# (at least in the scenarios that we saw above).

Now, would the figures above discourage me from using F#? My answer is No. Except for the last test where the difference was pretty significant, I wouldn’t sweat too much over the slight performance hit on the F# side.This is something I am willing to sacrifice in return for the better correctness and readability that F# brings to the table. Not to mention that immutability makes it much easier to introduce concurrency in your application, if you ever need to scale out.

Also, we are only talking about a few milliseconds difference for a simple and naive implementation. In a real business application, this would be completely drown in the total time dedicated to IO operations. You should spend your time on optimizing those instead!

Final words

Out of curiosity, I decided to run once again the last test with 1 000 000 customers in F#, but this time by swapping the immutable list for the .NET standard list (just like in the C# version):

let mutable _customers  = List<Customer>() // Mutable standard .NET list

[<Benchmark>]
member this.FSharp_ModifyCustomers_Mutable() =
    let today = DateTime.Today

    let rec loop index (customers:List<Customer>) =
        if index >= customers.Count
        then ignore
        else 
            customers.[index] <- setVip today customers.[index]
            loop (index+1) customers

    loop 0 _customers |> ignore

And here are the results:As you can see, the results are much better, almost twice as fast than before. This shows that there is place for improvement, but it comes with a trade-off.

That’s it for today. If you have anecdotes or data to share about performance in a functional code base, I’d be more than happy to hear about it!

Until next time!

Being REST-like in WebSharper – Part 2

      1 Comment on Being REST-like in WebSharper – Part 2

Today I will take up where I left off last time and flesh out my PizzaManager REST-like API a bit. But before I start, I promised last time that I would expand a little on the topic of RESTful APIs.

Is my API really RESTful?

For a few years now, we’ve seen an increasing number of web APIs boasting to be RESTful, providing nice and readable URLs for their resources, like the ones below:

  • GET api/employee/12
  • GET api/employees/all
  • POST api/employee { “firstName”: “John”, “lastName”: “Doe” }
  • DELETE api/documents/all

Are those APIs really RESTful though? We talked very quickly last time about the Richardson Maturity Model. Basically, The model says that they are 3 levels of RESTfulness:

  • Level 1: usage of resources. This is the part about how the URIs are constructed. Instead of querying an employee with a good ol’ query string (api/getEmployee/?id=12), we design a system where each resource has its own URI: api/employee/12
  • Level 2: usage of verbs. This is the part about using different Http Verbs like GET, POST, PUT or DELETE to express the action we want to apply on a given resource. So instead of having two queries like api/getEmployee/?id=12 and api/deleteEmployee/?id=12, we can simply define one and interchange the verbs as needed: GET api/employee/12 versus DELETE api/employee/12.
  • Level 3: usage of hypermedia controls. In short, this part is about how to make your API discoverable by communicating with hypermedia links rather than just sending back some data. For instance, if you were to create a new employee using the following request: POST api/employee { “firstName”: “John”, “lastName”: “Doe” }, the response would not only contain data about the newly created resource, but also other links to manipulate it, like DELETE api/employee/johnDoe or GET api/contactInfo/johnDoe/address. This allows to explore the API as you go along, from one link to another.

The problem is that according to the inventor of REST himself, Roy Fielding, an API cannot be deemed RESTful unless it achieves all 3 levels! However, the vast majority of APIs in the wild “only” achieve level 1 and level 2, like my own API below. In fact, achieving level 3 can be a very complex task on its own. Is it really worth making the distinction? Well, since truly RESTful APIs offer a rather different way to be navigated and explored, I’d say it is rather important to state the difference, for the sake of communication and to avoid any surprise when integrating with such APIs. I’ll let you make your own opinion on the topic.

Time to go go back to our PizzaManager!

Fixing the ugly Json generated

If you remember, I wasn’t really happy about how the generated Json looked like:

{
  "result": "success",
  "name": "Margherita",
  "price": 13,
  "ingredients": [
    {
      "name": "Tomato Sauce",
      "quantity": {
        "$": 2,
        "Item": 30
      }
    },
    {
      "name": "Mozzarella",
      "quantity": {
        "$": 1,
        "Item": 200
      }
    }
  ]
}

So decided to refactor it a bit. This is how our new pizza’s Json looks like now:

{
  "result": "success",
  "data": {
    "name": "Margherita",
    "price": 12,
    "ingredients": [
      {
        "unit": "ml",
        "name": "Tomato Sauce",
        "quantity": 30
      },
      {
        "unit": "g",
        "name": "Mozzarella",
        "quantity": 200
      }
    ]
  }
}

Much better! How did I achieve this? Let’s look at the F# types below:

[<NamedUnionCases"unit">]
type Ingredient =
  | [<Name "u">] Unit of name:string * quantity:int
  | [<Name "g">] Grams of name:string * quantity:int
  | [<Name "ml">] Milliliter of name:string * quantity:int

type Pizza =
  { [<Name"name">] Name : string
    [<Name"price">] Price : decimal
    [<Name"ingredients">][<OptionalField>] Ingredients : Ingredient array }

First, I refactored Ingredient so all data is contained in a single structure (name, unit and quantity). Each unit of measure (Unit, Grams and Milliliters) are now associated with a tuple containing both name and quantity of the given ingredient. A pizza can now be declared as follows:

{ Name = "Margherita"; Price = 12M; Ingredients = [|
    Milliliters("Tomato Sauce", 30)
    Grams("Mozzarella", 200) |]
}

Secondly, I took advantage of  the NamedUnionCases and Name attributes provided by WebSharper to further improve how the Json is generated. It was overall a very easy change to make.

I also added a few new actions to the API:

type PublicApi =
    /// GET /info
    | [<EndPoint"GET /">] ShowInfo
    /// GET /pizzas/{settings}
    | [<EndPoint"GET /pizzas">] GetPizzas of settings : string
    /// GET /pizza/margarita
    | [<EndPoint"GET /pizza">] GetPizza of name : string
    /// DELETE /pizza/margarita
    | [<EndPoint "DELETE /pizza">] DeletePizza of name : string
    /// POST /pizza/ + Json content
    | [<Method "POST"; CompiledName "pizza"; Json "pizzaData">] AddPizza of pizzaData : Pizza

Let’s go through the most noticeable changes quickly.

The ShowInfo action

The ShowInfo action is available under the following link: http://localhost:9000/pizzamanager/
It allows to see what are the different actions available in the API. Here’s how it looks in Postman:

The actual implementation is trivial and a bit naive, as it has to be maintained manually when adding new actions. It will do for now though. Here’s the code behind it:

/// Display info about the current API
type ApiInfo = { sampleUrl : string; description : string }

let showInfo =
    [|
        { sampleUrl = "[GET] /pizzamanager or /pizzamanager/"; description = "Display information about the current API." }
        { sampleUrl = "[GET] /pizzamanager/pizzas/names"; description = "List all pizzas available in the database." }
        { sampleUrl = "[GET] /pizzamanager/pizzas/ or /pizzamanager/pizzas/all"; description = "Get all pizzas in the database." }
        { sampleUrl = "[GET] /pizzamanager/pizza/{pizza_name}"; description = "Get the given pizza (case-insensitive)." }
        { sampleUrl = "[DELETE] /pizzamanager/pizza/{pizza_name}"; description = "Delete the given pizza from the database (case-insensitive)." }
    |]

// And the callsite from the requestHandler function below
| ShowInfo -> Content.Json (showInfo)

The updated GetPizzas action with settings

I decided to add a new settings parameter to the GetPizzas action, so it returns either the full details for all pizzas, or just the list of pizzas available in the database. You can check the final result in the .gif below:

(click on the .gif to enlarge)
The implementation was once again trivial:

/// Handle additional settings available for /pizzamanager/pizzas/
type ListSettings =
| All
| NamesOnly

let getSettings = function
    | "" | "all" -> Success All
    | "names" -> Success NamesOnly
    | notRecognized -> Failure (sprintf "'%s' is not a valid setting!" notRecognized)

// And the callsite from the requestHandler function below
| GetPizzas settings -> 
    match (getSettings settings) with
    | Success All -> Content.Json (PizzaManagerDb.getAll())
    | Success NamesOnly -> Content.Json (PizzaManagerDb.getNamesOnly())
    | failure -> Content.Json (failure)

The getSettings function has an interesting syntax here. It is what we call a pattern matching function and could also be written as below:

let getSettings2 settings =
    match settings with
    | "" | "all" -> Success All
    | "names" -> Success NamesOnly
    | notRecognized -> Failure (sprintf "'%s' is not a valid setting!" notRecognized)

However I prefer the conciseness of the first version!

The DeletePizza action

The URI of the DeletePizza endpoint looks almost like the one for GetPizza:

/// GET /pizza/margarita
| [<EndPoint"GET /pizza">] GetPizza of name : string
/// DELETE /pizza/margarita
| [<EndPoint "DELETE /pizza">] DeletePizza of name : string

The only difference is the latter uses the DELETE Http verb while the former uses GET. Making proper use of the verbs available is one of the features necessary to achieve RESTness. Your URLs designate the resources while the verbs indicate the operation:

GET /pizzamanager/pizza/rucola
vs
DELETE /pizzamanager/pizza/rucola

The AddPizza action

[EDIT: 02/05/2017] The issue described in the paragraph below is now solved! And it was indeed not related to WebSharper itself but to a mistake I made when configuring how the requests are being handled (more details here).

Unfortunately, I wasn’t able to properly make the AddPizza action work! I spent a few hours trying to figure out why, without success. I explained the problem here on WebSharper’s forums, hopefully someone will be able to help with that. Fingers crossed! In short, it seems that WebSharper is not able to probably route to the AddPizza action and a 404 Not Found status code is returned:


Wrapping up

Let’s see a short demo of all the working actions so far:

To sum up today’s post, I am really happy about how easy and quick it is to generate a decent REST-like API using WebSharper. However I still need to figure out why the AddPizza action is not working properly. I will update this post if and when I get more information about it.
As explained below, the issue was due to a configuration mistake in Owin and not to WebSharper itself. This also gave us the opportunity to check if WebSharper’s support works, and at least in our case it did!

Cheers

Putting together the Software Dream-Team

      4 Comments on Putting together the Software Dream-Team

If you read some of my previous posts on this blog, you will find that I am equally interested in both the technical and social aspects of Software development. As I explained in my post about project failures, statistics appear to show that most problems Software professionals encounter are not in fact technical but rather social. Based on my own experience, I am very inclined to believe those figures.

On my quest to understanding how we could make and deliver better Software, I often wonder what I would do if I was given the difficult mission to assemble the ideal development team from scratch – the Software Dream-Team, if you will. I guess it’s time to put my thoughts in order and lay them on paper!

First, we need to define what such a team will be tasked to achieve. For the sake of the exercise, let’s assume that the lucky guys will be working on a product aimed at Italian restaurant businesses: A web-based pizza manager with both cloud and on-premises data hosting options. As the product itself is available under a monthly subscription business model, the team will have to deliver new features very regularly to keep customers from going to the concurrence. All this while guaranteeing good performance and stability, obviously. Seems like there’s a lot on the agenda, so we’d better pick the right guys for the job! Since we’re at it, let’s add an additional challenge to the mix. Turning the pizza manager into a successful project will certainly be a long-term endeavour and therefore we also want our team members to be happy and proud of their achievements. Customer satisfaction starts with employee satisfaction.

Who will we need in our team to make our pizza manager the killer app it deserves to be? Let’s list the usual suspects we encounter in Software projects and try to pick the ones that’ll be useful to us.

  • Business analyst (BA)
  • Project manager (PM)
  • Release manager
  • Back-end developer
  • Front-end developer
  • Tools developer
  • QA specialist (QA)
  • Test automation engineer
  • Software architect
  • Graphic designer
  • UX designer
  • Database administrator (DBA)
  • Network specialist

That’s 13 roles already. So let’s hire 13 guys and rock’n’roll, right? RIGHT!? Not really. This would most probably end in a real communication mess and this is precisely what we want to avoid. Our goal is to deliver Software, not to manage a circus.

How many individuals should we hire then? In my opinion, the magical number is probably 5. Why 5? Because there were 5 Power Rangers (not counting the green / white one, this guy joined later as an external consultant!), and that was one hell of a team! Look at that!

More seriously though, I believe that having a team composed of 4 to 6 individuals is ideal because it doesn’t introduce too much communication overhead while still providing significant Software firepower. There is a problem though: we listed 13 positions above but we only have 5 team members to fill them in. Luckily, I might have an answer to that. I believe that people should ideally hold 2 roles within a given project. Not more, not less. Let me expand on that.

Why not more than 2?

  • Every team member must understand precisely what is expected from him and others within the project, and having too many roles introduces unnecessary confusion:
    “What is this guy doing? Is he our manager, our back-end developer or our DBA?”
    “Wait, am I supposed to look at that from the BA perspective or as a Tools programmer?”
  • Constantly switching between roles often consumes a non-trivial amount of time. This time could be better spent doing some actual work on the project. People usually try to compensate for that by introducing more planning into the project, but that’s just like solving a problem by introducing a new one.

Why not less than 2?

  • Having several roles can prevent someone from getting tired of doing the same things over and over again. Boredom and mental fatigue can be a real productivity killer.
  • Having several roles gives team members different paths to explore and more career options. It also allows people to have a wider perspective on the project. This can contribute to the overall happiness and sentiment of ownership of the individual.

“Very cool, Ricardo! 5 times 2 equals 10, so we can cover 10 roles out of 13 already!” Not so fast. There are a few more aspects that we need to take into account in our equation. What if our main developer decides he wants to take a vacation (crazy, I know!) or gets hit by a bus (the first cause of mortality among developers)? We can’t really afford to put the whole project on hold just because one individual is missing.  This is why we need some roles to overlap.

We already defined that each team member should hold 2 roles. Let’s go further and introduce the concept of major and minor roles. The idea is that each individual should spend around 70% of his time or effort on his major role and the remaining 30% on his minor role.

With this in mind, we need to make sure that each of the most important roles for the project is covered by 2 team members: as a major role for one and as a minor role for the other. Aside from partially solving the problem above, it brings two more things to the table:

  • A natural knowledge transfer within the team.
  • The possibility to work in pairs to tackle more complex problems.

However, not every combination will do just fine, as some roles are by nature incompatibles with others. During my career I’ve had the “chance” to experience this at several occasions. Here are a few examples of such bad combinations:

development + project management tasks

Those two aspects are fundamentally different. Project management requires to swiftly deal with the unexpected and remove the obstacles as they come so that the rest of the team remains focused and undisturbed. This requires from the manager to go back and forth between tasks and to shift his priorities. On the other hand, a developer needs long, uninterrupted sessions of focused work in order to be efficient. According to several surveys, it takes a developer between 15 and 30 minutes to fully go back to his programming bubble (also called the zone) after being interrupted by a phone call from the client or by the project manager asking “Do you think you can finish this by lunch?”. A few interruptions like those are synonym to a day lost for the developer.

Development + QA tasks

Writing Software and testing Software are two very different activities. Quality Assurance is a job on its own and should be handled by professionals. However this is often the case that developers are the only ones to test their own creation before releasing all hell on the poor end users. What are the issues with this approach? A developer will unconsciously be more indulgent toward his own work and half-ass the testing, or end up as below:

(click to enlarge)

Something that I do in my current project in order to mitigate this is that I try to leave a buffer of a few days between the moment I finish the implementation of a task and the moment I start testing. It allows me to somehow “forget” about the implementation details and have slightly more impartial testing sessions. This only works to an extent though, as I will naturally be tempted to test the happy paths only (at times I won’t even have enough knowledge to come up with negative test scenarios). As another downside, it will also take me longer to bring a given task to completion.

The Software Dream-Team?

Taking into account all the parameters explained above, this is how my Software Dream-Team would probably look like (for each member, the first item is his major role while the second is his minor role):

Member 1: Business analyst + QA specialist
Member 2: QA specialist + Test automation engineer
Member 3: Back-end developer + Front-end developer
Member 4: Front-end developer + Back-end developer
Member 5: Graphic designer + Front-end developer

Let me explain some of my choices. Member 1 and Member 2 can work together on gathering requirements (BA) and making sure those are properly implemented (QA). This is an idea that I already submitted in my previous post about writing better specifications. Having test automation skills will definitely help the team efficiently measure the overall quality of the product while new features are being added.

Member 3 and Member 4 are both full-stack developers, but each they specialize in different areas (back-end vs front-end). Member 5 is mainly a graphic designer but can also collaborate with Member 3 and Member 4 on implementing the front-end, if need be.

What about the remaining roles?

As you may have noticed, my Dream-Team contains only 6 roles out of the original 13. What happened to the remaining 7?

There are 2 roles that I long considered before finally deciding to exclude them:

  • Tools developer. I believe that using the proper tools is a great step toward success in an IT project. Having a dedicated Tools developer might be helpful, but nowadays there is a ton of great tools out there to help you solve almost any technical challenge you could think of (logging, caching, data persistence…). And chances are that those are probably better today than your home-brewed custom library will ever be.
  • UX designer. All other technical aspects equal, UX might be the most important criteria for consumers to choose between two applications. Does that mean you need a dedicated UX designer? I don’t know. Maybe you could treat UX as a separate cornerstone of your application and have all your members work on it. Or you could pay hourly an external UX designer to help you when necessary. This is a question that definitely deserves some thoughts.

The following 5 roles can be in my humble opinion ignored:

  • Project manager. I don’t think that a dedicated project manager would bring much to the table here. Our members should take all the global decisions regarding the project as a team, be it the planning, budget, or overall direction. This will also reinforce the general sentiment of ownership within the team.
  • Software architect. Once again, I believe that the all members of the team and especially Member 3, Member 4 and Member 5 should be knowledgeable enough to take all the technical decisions as a team.
  • Network specialist and Database administrator. With all the Cloud hosting options available today, maintaining your own infrastructure and servers might not be the safest and most cost-effective solution. Aside from the cost of the hardware alone, there are dozens of aspects that could potentially turn your business into a nightmare if done wrong: server backups, security updates, customer data confidentiality, network monitoring, load balancing or scalability… You name it. Your dollars will be better spent on platforms that do all that for you, out of the box.
  • Release manager. If you still do releases manually, then you might be doing something wrong. Your release process should be automated and include all sorts of tests to guarantee an acceptable level of quality to your customers. Releasing a new version shouldn’t be more difficult than pushing a button. No need to have a dedicated release manager for that.

That concludes this post on my Software Dream-Team. I hope this constitutes a good base for reflection on the topic. I’d like to thank Michał “MSI” and Piotrek, two of our awesome QA specialists at Infusion, for giving me precious input and helping me better understand their job. Also big thanks to Agnieszka for making those great illustrations for my posts on demand!

If you have your own idea on how the ideal Software team should look like or have suggestions about the above, please leave a comment. I’d be very interested in your opinion!

Until next time!

Short update on my DSP project – Being REST-like in WebSharper

Hello everyone, I am extremely busy lately with a few other side projects, so I just wanted to make a small update on my DSP2017 project. I managed to find a few hours today to implement the foundations of a REST-like service in WebSharper. Why am I saying REST-like and not RESTful? Well, because my service only reaches level 2 on the Richardson Maturity Model. I will expand on the subject in a future post, I promise!

The implementation is far from being finished but I finally managed to make a few simple cases work. You can check the code on my GitHub repo under PizzaManagerRestApi.fs. Obviously, the service is about managing pizzas (in case you were still wondering: yes, I like pizzas!). The root URL is http://localhost:9000/pizzamanager/ and for now I implemented two actions:

  • Get the full list of pizzas using /pizzamanager/pizzas.
  • Get a pizza by name by using /pizzamanager/pizza/pizza_name.

You can check out the work in progress in the .gif below:

(click on the .gif to enlarge)

Each call either returns with Success and a Json object or with Failure and an error message, following the now famous Railway Oriented Programming approach (ROP), enabled partly by the simple type Result shown below:

type Result<'T> =
   | Success of 'T
   | Failure of string

If you want to learn more about ROP, I highly recommend you check out this video from the awesome Scott Wlaschin.

As you might have noticed, the Json returned by service is pretty ugly right now:

{
  "result": "success",
  "name": "Margherita",
  "price": 13,
  "ingredients": [
    {
      "name": "Tomato Sauce",
      "quantity": {
        "$": 2,
        "Item": 30
      }
    },
    {
      "name": "Parma Ham",
      "quantity": {
        "$": 1,
        "Item": 100
      }
    },
    {
      "name": "Mozzarella",
      "quantity": {
        "$": 1,
        "Item": 200
      }
    }
  ]
}

I believe this is because of the default way WebSharper serializes to Json to enable communication between the server-side and the client-side. Hopefully we’ll be able to tweak it a little to make it more readable. But that’s the topic of another post.

Until next time!

Programmers want to have fun, too!

      2 Comments on Programmers want to have fun, too!

Being a programmer can be very stressful at times. Incomplete specifications, hidden bugs that hit on a Friday afternoon, project failures, Youtube being blocked at work… There’s a lot of factors that can negatively impact the psyche of even the strongest among us.

Luckily, there is also a lot of resources out there that help us stay sane on a daily basis. Yes, programmers want to have fun, too! In this post I will share with you what are my regular go-tos and hope that I’ll get a few new suggestions in return. Let’s get to it!

The all-time classic – Dilbert

Dilbert has been a source of fun and inspiration for thousands of programmers over the years.
The series began in April 1989 and the author, Scott Adams, has been publishing new comic strips daily since then! That’s over 10000 of them. I reckon I read a good half of them and still browse the site on a daily basis.

What I love about Dilbert? The fact that it is still fresh and relevant even after all those years and that the author managed to keep the original character of each of the protagonists. Here are some of my favourite strips:

Agile Methodology
The interview
The sales call

The modern and startup’ish – CommitStrip

CommitStrip is a “The blog relating the daily life of web agency developers” as the authors put it themselves. The authors, French guys working in web agencies, generally publish a few comic strips a week. They often take their inspiration from real news or trending technologies and excel at highlighting the paradoxes of our profession. The strips are available in French and English, and here are a few examples:

Let’s talk about code, shall we?

Why you should get back to your code, right now

The giffy one – The Coding Love

If you like gifs, The Coding Love is made for you! They publish a few gifs a day, so you probably won’t go short on fun. There’s also a French version available at Les Joies Du Code. Generally speaking, they are really harsh on interns, clients, and Internet Explorer. What more to ask for? Here are a few nice ones:

when the project manager wants all the transitions and effects in IE8 as well
when the client tell us he wants to create a social network to compete with facebook
Learning that the client wants to change everything the day before our demo

The CodeProject daily newsletter

You probably know CodeProject already. This site is a great source of information for programmers over the world. Well, they also have a daily newsletter which, aside from keeping you well informed, will also put a smile on your face. Why? Because the awesome guys who select the articles to appear in the newsletter don’t miss a single occasion to add witty comments to them. I generally read those in the morning while commuting to work and this is always a great way to start the day! Here are a few of those gems:

Wait, there’s more!

There’s quite a lot of other resources worth knowing. Here are a few of them:

Don’t Hit Save

XKCD

Do you also have a list of websites that you frequently visit to get your dose of fun?
if yes please share them in the comments below!

Cheers!