Being REST-like in WebSharper – Part 2

      No Comments on Being REST-like in WebSharper – Part 2

Today I will take up where I left off last time and flesh out my PizzaManager REST-like API a bit. But before I start, I promised last time that I would expand a little on the topic of RESTful APIs.

Is my API really RESTful?

For a few years now, we’ve seen an increasing number of web APIs boasting to be RESTful, providing nice and readable URLs for their resources, like the ones below:

  • GET api/employee/12
  • GET api/employees/all
  • POST api/employee { “firstName”: “John”, “lastName”: “Doe” }
  • DELETE api/documents/all

Are those APIs really RESTful though? We talked very quickly last time about the Richardson Maturity Model. Basically, The model says that they are 3 levels of RESTfulness:

  • Level 1: usage of resources. This is the part about how the URIs are constructed. Instead of querying an employee with a good ol’ query string (api/getEmployee/?id=12), we design a system where each resource has its own URI: api/employee/12
  • Level 2: usage of verbs. This is the part about using different Http Verbs like GET, POST, PUT or DELETE to express the action we want to apply on a given resource. So instead of having two queries like api/getEmployee/?id=12 and api/deleteEmployee/?id=12, we can simply define one and interchange the verbs as needed: GET api/employee/12 versus DELETE api/employee/12.
  • Level 3: usage of hypermedia controls. In short, this part is about how to make your API discoverable by communicating with hypermedia links rather than just sending back some data. For instance, if you were to create a new employee using the following request: POST api/employee { “firstName”: “John”, “lastName”: “Doe” }, the response would not only contain data about the newly created resource, but also other links to manipulate it, like DELETE api/employee/johnDoe or GET api/contactInfo/johnDoe/address. This allows to explore the API as you go along, from one link to another.

The problem is that according to the inventor of REST himself, Roy Fielding, an API cannot be deemed RESTful unless it achieves all 3 levels! However, the vast majority of APIs in the wild “only” achieve level 1 and level 2, like my own API below. In fact, achieving level 3 can be a very complex task on its own. Is it really worth making the distinction? Well, since truly RESTful APIs offer a rather different way to be navigated and explored, I’d say it is rather important to state the difference, for the sake of communication and to avoid any surprise when integrating with such APIs. I’ll let you make your own opinion on the topic.

Time to go go back to our PizzaManager!

Fixing the ugly Json generated

If you remember, I wasn’t really happy about how the generated Json looked like:

{
  "result": "success",
  "name": "Margherita",
  "price": 13,
  "ingredients": [
    {
      "name": "Tomato Sauce",
      "quantity": {
        "$": 2,
        "Item": 30
      }
    },
    {
      "name": "Mozzarella",
      "quantity": {
        "$": 1,
        "Item": 200
      }
    }
  ]
}

So decided to refactor it a bit. This is how our new pizza’s Json looks like now:

{
  "result": "success",
  "data": {
    "name": "Margherita",
    "price": 12,
    "ingredients": [
      {
        "unit": "ml",
        "name": "Tomato Sauce",
        "quantity": 30
      },
      {
        "unit": "g",
        "name": "Mozzarella",
        "quantity": 200
      }
    ]
  }
}

Much better! How did I achieve this? Let’s look at the F# types below:

[<NamedUnionCases"unit">]
type Ingredient =
  | [<Name "u">] Unit of name:string * quantity:int
  | [<Name "g">] Grams of name:string * quantity:int
  | [<Name "ml">] Milliliter of name:string * quantity:int

type Pizza =
  { [<Name"name">] Name : string
    [<Name"price">] Price : decimal
    [<Name"ingredients">][<OptionalField>] Ingredients : Ingredient array }

First, I refactored Ingredient so all data is contained in a single structure (name, unit and quantity). Each unit of measure (Unit, Grams and Milliliters) are now associated with a tuple containing both name and quantity of the given ingredient. A pizza can now be declared as follows:

{ Name = "Margherita"; Price = 12M; Ingredients = [|
    Milliliters("Tomato Sauce", 30)
    Grams("Mozzarella", 200) |]
}

Secondly, I took advantage of  the NamedUnionCases and Name attributes provided by WebSharper to further improve how the Json is generated. It was overall a very easy change to make.

I also added a few new actions to the API:

type PublicApi =
    /// GET /info
    | [<EndPoint"GET /">] ShowInfo
    /// GET /pizzas/{settings}
    | [<EndPoint"GET /pizzas">] GetPizzas of settings : string
    /// GET /pizza/margarita
    | [<EndPoint"GET /pizza">] GetPizza of name : string
    /// DELETE /pizza/margarita
    | [<EndPoint "DELETE /pizza">] DeletePizza of name : string
    /// POST /pizza/ + Json content
    | [<Method "POST"; CompiledName "pizza"; Json "pizzaData">] AddPizza of pizzaData : Pizza

Let’s go through the most noticeable changes quickly.

The ShowInfo action

The ShowInfo action is available under the following link: http://localhost:9000/pizzamanager/
It allows to see what are the different actions available in the API. Here’s how it looks in Postman:

The actual implementation is trivial and a bit naive, as it has to be maintained manually when adding new actions. It will do for now though. Here’s the code behind it:

/// Display info about the current API
type ApiInfo = { sampleUrl : string; description : string }

let showInfo =
    [|
        { sampleUrl = "[GET] /pizzamanager or /pizzamanager/"; description = "Display information about the current API." }
        { sampleUrl = "[GET] /pizzamanager/pizzas/names"; description = "List all pizzas available in the database." }
        { sampleUrl = "[GET] /pizzamanager/pizzas/ or /pizzamanager/pizzas/all"; description = "Get all pizzas in the database." }
        { sampleUrl = "[GET] /pizzamanager/pizza/{pizza_name}"; description = "Get the given pizza (case-insensitive)." }
        { sampleUrl = "[DELETE] /pizzamanager/pizza/{pizza_name}"; description = "Delete the given pizza from the database (case-insensitive)." }
    |]

// And the callsite from the requestHandler function below
| ShowInfo -> Content.Json (showInfo)

The updated GetPizzas action with settings

I decided to add a new settings parameter to the GetPizzas action, so it returns either the full details for all pizzas, or just the list of pizzas available in the database. You can check the final result in the .gif below:

(click on the .gif to enlarge)
The implementation was once again trivial:

/// Handle additional settings available for /pizzamanager/pizzas/
type ListSettings =
| All
| NamesOnly

let getSettings = function
    | "" | "all" -> Success All
    | "names" -> Success NamesOnly
    | notRecognized -> Failure (sprintf "'%s' is not a valid setting!" notRecognized)

// And the callsite from the requestHandler function below
| GetPizzas settings -> 
    match (getSettings settings) with
    | Success All -> Content.Json (PizzaManagerDb.getAll())
    | Success NamesOnly -> Content.Json (PizzaManagerDb.getNamesOnly())
    | failure -> Content.Json (failure)

The getSettings function has an interesting syntax here. It is what we call a pattern matching function and could also be written as below:

let getSettings2 settings =
    match settings with
    | "" | "all" -> Success All
    | "names" -> Success NamesOnly
    | notRecognized -> Failure (sprintf "'%s' is not a valid setting!" notRecognized)

However I prefer the conciseness of the first version!

The DeletePizza action

The URI of the DeletePizza endpoint looks almost like the one for GetPizza:

/// GET /pizza/margarita
| [<EndPoint"GET /pizza">] GetPizza of name : string
/// DELETE /pizza/margarita
| [<EndPoint "DELETE /pizza">] DeletePizza of name : string

The only difference is the latter uses the DELETE Http verb while the former uses GET. Making proper use of the verbs available is one of the features necessary to achieve RESTness. Your URLs designate the resources while the verbs indicate the operation:

GET /pizzamanager/pizza/rucola
vs
DELETE /pizzamanager/pizza/rucola

The AddPizza action

Unfortunately, I wasn’t able to properly make the AddPizza action work! I spent a few hours trying to figure out why, without success. I explained the problem here on WebSharper’s forums, hopefully someone will be able to help with that. Fingers crossed! In short, it seems that WebSharper is not able to probably route to the AddPizza action and a 404 Not Found status code is returned:


Wrapping up

Let’s see a short demo of all the working actions so far:

To sum up today’s post, I am really happy about how easy and quick it is to generate a decent REST-like API using WebSharper. However I still need to figure out why the AddPizza action is not working properly. I will update this post if and when I get more information about it.

Cheers

Putting together the Software Dream-Team

      4 Comments on Putting together the Software Dream-Team

If you read some of my previous posts on this blog, you will find that I am equally interested in both the technical and social aspects of Software development. As I explained in my post about project failures, statistics appear to show that most problems Software professionals encounter are not in fact technical but rather social. Based on my own experience, I am very inclined to believe those figures.

On my quest to understanding how we could make and deliver better Software, I often wonder what I would do if I was given the difficult mission to assemble the ideal development team from scratch – the Software Dream-Team, if you will. I guess it’s time to put my thoughts in order and lay them on paper!

First, we need to define what such a team will be tasked to achieve. For the sake of the exercise, let’s assume that the lucky guys will be working on a product aimed at Italian restaurant businesses: A web-based pizza manager with both cloud and on-premises data hosting options. As the product itself is available under a monthly subscription business model, the team will have to deliver new features very regularly to keep customers from going to the concurrence. All this while guaranteeing good performance and stability, obviously. Seems like there’s a lot on the agenda, so we’d better pick the right guys for the job! Since we’re at it, let’s add an additional challenge to the mix. Turning the pizza manager into a successful project will certainly be a long-term endeavour and therefore we also want our team members to be happy and proud of their achievements. Customer satisfaction starts with employee satisfaction.

Who will we need in our team to make our pizza manager the killer app it deserves to be? Let’s list the usual suspects we encounter in Software projects and try to pick the ones that’ll be useful to us.

  • Business analyst (BA)
  • Project manager (PM)
  • Release manager
  • Back-end developer
  • Front-end developer
  • Tools developer
  • QA specialist (QA)
  • Test automation engineer
  • Software architect
  • Graphic designer
  • UX designer
  • Database administrator (DBA)
  • Network specialist

That’s 13 roles already. So let’s hire 13 guys and rock’n’roll, right? RIGHT!? Not really. This would most probably end in a real communication mess and this is precisely what we want to avoid. Our goal is to deliver Software, not to manage a circus.

How many individuals should we hire then? In my opinion, the magical number is probably 5. Why 5? Because there were 5 Power Rangers (not counting the green / white one, this guy joined later as an external consultant!), and that was one hell of a team! Look at that!

More seriously though, I believe that having a team composed of 4 to 6 individuals is ideal because it doesn’t introduce too much communication overhead while still providing significant Software firepower. There is a problem though: we listed 13 positions above but we only have 5 team members to fill them in. Luckily, I might have an answer to that. I believe that people should ideally hold 2 roles within a given project. Not more, not less. Let me expand on that.

Why not more than 2?

  • Every team member must understand precisely what is expected from him and others within the project, and having too many roles introduces unnecessary confusion:
    “What is this guy doing? Is he our manager, our back-end developer or our DBA?”
    “Wait, am I supposed to look at that from the BA perspective or as a Tools programmer?”
  • Constantly switching between roles often consumes a non-trivial amount of time. This time could be better spent doing some actual work on the project. People usually try to compensate for that by introducing more planning into the project, but that’s just like solving a problem by introducing a new one.

Why not less than 2?

  • Having several roles can prevent someone from getting tired of doing the same things over and over again. Boredom and mental fatigue can be a real productivity killer.
  • Having several roles gives team members different paths to explore and more career options. It also allows people to have a wider perspective on the project. This can contribute to the overall happiness and sentiment of ownership of the individual.

“Very cool, Ricardo! 5 times 2 equals 10, so we can cover 10 roles out of 13 already!” Not so fast. There are a few more aspects that we need to take into account in our equation. What if our main developer decides he wants to take a vacation (crazy, I know!) or gets hit by a bus (the first cause of mortality among developers)? We can’t really afford to put the whole project on hold just because one individual is missing.  This is why we need some roles to overlap.

We already defined that each team member should hold 2 roles. Let’s go further and introduce the concept of major and minor roles. The idea is that each individual should spend around 70% of his time or effort on his major role and the remaining 30% on his minor role.

With this in mind, we need to make sure that each of the most important roles for the project is covered by 2 team members: as a major role for one and as a minor role for the other. Aside from partially solving the problem above, it brings two more things to the table:

  • A natural knowledge transfer within the team.
  • The possibility to work in pairs to tackle more complex problems.

However, not every combination will do just fine, as some roles are by nature incompatibles with others. During my career I’ve had the “chance” to experience this at several occasions. Here are a few examples of such bad combinations:

development + project management tasks

Those two aspects are fundamentally different. Project management requires to swiftly deal with the unexpected and remove the obstacles as they come so that the rest of the team remains focused and undisturbed. This requires from the manager to go back and forth between tasks and to shift his priorities. On the other hand, a developer needs long, uninterrupted sessions of focused work in order to be efficient. According to several surveys, it takes a developer between 15 and 30 minutes to fully go back to his programming bubble (also called the zone) after being interrupted by a phone call from the client or by the project manager asking “Do you think you can finish this by lunch?”. A few interruptions like those are synonym to a day lost for the developer.

Development + QA tasks

Writing Software and testing Software are two very different activities. Quality Assurance is a job on its own and should be handled by professionals. However this is often the case that developers are the only ones to test their own creation before releasing all hell on the poor end users. What are the issues with this approach? A developer will unconsciously be more indulgent toward his own work and half-ass the testing, or end up as below:

(click to enlarge)

Something that I do in my current project in order to mitigate this is that I try to leave a buffer of a few days between the moment I finish the implementation of a task and the moment I start testing. It allows me to somehow “forget” about the implementation details and have slightly more impartial testing sessions. This only works to an extent though, as I will naturally be tempted to test the happy paths only (at times I won’t even have enough knowledge to come up with negative test scenarios). As another downside, it will also take me longer to bring a given task to completion.

The Software Dream-Team?

Taking into account all the parameters explained above, this is how my Software Dream-Team would probably look like (for each member, the first item is his major role while the second is his minor role):

Member 1: Business analyst + QA specialist
Member 2: QA specialist + Test automation engineer
Member 3: Back-end developer + Front-end developer
Member 4: Front-end developer + Back-end developer
Member 5: Graphic designer + Front-end developer

Let me explain some of my choices. Member 1 and Member 2 can work together on gathering requirements (BA) and making sure those are properly implemented (QA). This is an idea that I already submitted in my previous post about writing better specifications. Having test automation skills will definitely help the team efficiently measure the overall quality of the product while new features are being added.

Member 3 and Member 4 are both full-stack developers, but each they specialize in different areas (back-end vs front-end). Member 5 is mainly a graphic designer but can also collaborate with Member 3 and Member 4 on implementing the front-end, if need be.

What about the remaining roles?

As you may have noticed, my Dream-Team contains only 6 roles out of the original 13. What happened to the remaining 7?

There are 2 roles that I long considered before finally deciding to exclude them:

  • Tools developer. I believe that using the proper tools is a great step toward success in an IT project. Having a dedicated Tools developer might be helpful, but nowadays there is a ton of great tools out there to help you solve almost any technical challenge you could think of (logging, caching, data persistence…). And chances are that those are probably better today than your home-brewed custom library will ever be.
  • UX designer. All other technical aspects equal, UX might be the most important criteria for consumers to choose between two applications. Does that mean you need a dedicated UX designer? I don’t know. Maybe you could treat UX as a separate cornerstone of your application and have all your members work on it. Or you could pay hourly an external UX designer to help you when necessary. This is a question that definitely deserves some thoughts.

The following 5 roles can be in my humble opinion ignored:

  • Project manager. I don’t think that a dedicated project manager would bring much to the table here. Our members should take all the global decisions regarding the project as a team, be it the planning, budget, or overall direction. This will also reinforce the general sentiment of ownership within the team.
  • Software architect. Once again, I believe that the all members of the team and especially Member 3, Member 4 and Member 5 should be knowledgeable enough to take all the technical decisions as a team.
  • Network specialist and Database administrator. With all the Cloud hosting options available today, maintaining your own infrastructure and servers might not be the safest and most cost-effective solution. Aside from the cost of the hardware alone, there are dozens of aspects that could potentially turn your business into a nightmare if done wrong: server backups, security updates, customer data confidentiality, network monitoring, load balancing or scalability… You name it. Your dollars will be better spent on platforms that do all that for you, out of the box.
  • Release manager. If you still do releases manually, then you might be doing something wrong. Your release process should be automated and include all sorts of tests to guarantee an acceptable level of quality to your customers. Releasing a new version shouldn’t be more difficult than pushing a button. No need to have a dedicated release manager for that.

That concludes this post on my Software Dream-Team. I hope this constitutes a good base for reflection on the topic. I’d like to thank Michał “MSI” and Piotrek, two of our awesome QA specialists at Infusion, for giving me precious input and helping me better understand their job. Also big thanks to Agnieszka for making those great illustrations for my posts on demand!

If you have your own idea on how the ideal Software team should look like or have suggestions about the above, please leave a comment. I’d be very interested in your opinion!

Until next time!

Short update on my DSP project – Being REST-like in WebSharper

Hello everyone, I am extremely busy lately with a few other side projects, so I just wanted to make a small update on my DSP2017 project. I managed to find a few hours today to implement the foundations of a REST-like service in WebSharper. Why am I saying REST-like and not RESTful? Well, because my service only reaches level 2 on the Richardson Maturity Model. I will expand on the subject in a future post, I promise!

The implementation is far from being finished but I finally managed to make a few simple cases work. You can check the code on my GitHub repo under PizzaManagerRestApi.fs. Obviously, the service is about managing pizzas (in case you were still wondering: yes, I like pizzas!). The root URL is http://localhost:9000/pizzamanager/ and for now I implemented two actions:

  • Get the full list of pizzas using /pizzamanager/pizzas.
  • Get a pizza by name by using /pizzamanager/pizza/pizza_name.

You can check out the work in progress in the .gif below:

(click on the .gif to enlarge)

Each call either returns with Success and a Json object or with Failure and an error message, following the now famous Railway Oriented Programming approach (ROP), enabled partly by the simple type Result shown below:

type Result<'T> =
   | Success of 'T
   | Failure of string

If you want to learn more about ROP, I highly recommend you check out this video from the awesome Scott Wlaschin.

As you might have noticed, the Json returned by service is pretty ugly right now:

{
  "result": "success",
  "name": "Margherita",
  "price": 13,
  "ingredients": [
    {
      "name": "Tomato Sauce",
      "quantity": {
        "$": 2,
        "Item": 30
      }
    },
    {
      "name": "Parma Ham",
      "quantity": {
        "$": 1,
        "Item": 100
      }
    },
    {
      "name": "Mozzarella",
      "quantity": {
        "$": 1,
        "Item": 200
      }
    }
  ]
}

I believe this is because of the default way WebSharper serializes to Json to enable communication between the server-side and the client-side. Hopefully we’ll be able to tweak it a little to make it more readable. But that’s the topic of another post.

Until next time!

Programmers want to have fun, too!

      2 Comments on Programmers want to have fun, too!

Being a programmer can be very stressful at times. Incomplete specifications, hidden bugs that hit on a Friday afternoon, project failures, Youtube being blocked at work… There’s a lot of factors that can negatively impact the psyche of even the strongest among us.

Luckily, there is also a lot of resources out there that help us stay sane on a daily basis. Yes, programmers want to have fun, too! In this post I will share with you what are my regular go-tos and hope that I’ll get a few new suggestions in return. Let’s get to it!

The all-time classic – Dilbert

Dilbert has been a source of fun and inspiration for thousands of programmers over the years.
The series began in April 1989 and the author, Scott Adams, has been publishing new comic strips daily since then! That’s over 10000 of them. I reckon I read a good half of them and still browse the site on a daily basis.

What I love about Dilbert? The fact that it is still fresh and relevant even after all those years and that the author managed to keep the original character of each of the protagonists. Here are some of my favourite strips:

Agile Methodology
The interview
The sales call

The modern and startup’ish – CommitStrip

CommitStrip is a “The blog relating the daily life of web agency developers” as the authors put it themselves. The authors, French guys working in web agencies, generally publish a few comic strips a week. They often take their inspiration from real news or trending technologies and excel at highlighting the paradoxes of our profession. The strips are available in French and English, and here are a few examples:

Let’s talk about code, shall we?

Why you should get back to your code, right now

The giffy one – The Coding Love

If you like gifs, The Coding Love is made for you! They publish a few gifs a day, so you probably won’t go short on fun. There’s also a French version available at Les Joies Du Code. Generally speaking, they are really harsh on interns, clients, and Internet Explorer. What more to ask for? Here are a few nice ones:

when the project manager wants all the transitions and effects in IE8 as well
when the client tell us he wants to create a social network to compete with facebook
Learning that the client wants to change everything the day before our demo

The CodeProject daily newsletter

You probably know CodeProject already. This site is a great source of information for programmers over the world. Well, they also have a daily newsletter which, aside from keeping you well informed, will also put a smile on your face. Why? Because the awesome guys who select the articles to appear in the newsletter don’t miss a single occasion to add witty comments to them. I generally read those in the morning while commuting to work and this is always a great way to start the day! Here are a few of those gems:

Wait, there’s more!

There’s quite a lot of other resources worth knowing. Here are a few of them:

Don’t Hit Save

XKCD

Do you also have a list of websites that you frequently visit to get your dose of fun?
if yes please share them in the comments below!

Cheers!

Overview of WebSharper’s templating engine – Part 2, Data holes

This is the second post of our series dedicated to WebSharper’s templating engine. In the first and previous post, we introduced the concept of text holes, a very simple but rather limited way to inject content in your HTML using F#. Today we are going to cover a more advanced mechanism that allows to inject whole chunks of HTML at once: data holes. There are two ways to declare data holes in your HTML template file:

  • using the data-hole attribute to decorate a given HTML element. Your content will be injected underneath the decorated element as a new child.
  • using the data-replace attribute to decorate a given HTML element. In this case, the whole element will be replaced by the injected content.

Let’s go over the data-hole attribute first.

Playing with data-hole

I modified the AdvancedTemplate.html file a bit and added:

  • a new data-hole called ‘somedatahole’
  • a new CSS class ‘.someStyle’ that we will be using in a moment.

As usual, you can also browse the full code on my GitHub repo.

<head>
    <style>
        .someStyle {
            color: darkcyan;
            font-style: oblique
        }
    </style>
</head>
<body>
    <div data-hole="somedatahole"></div>
</body>
</html>

We also need to make a few modifications to our F# code so we can inject new content into the hole. This is done in the Main.fs file. Here are the important bits:

type AdvancedPage = {
    Title: string
    SomeTextHole: string
    SomeDataHole: list<Element>
}

let Template =
    Content.Template<AdvancedPage>("~/AdvancedTemplate.html")
        .With("title", fun x -> x.Title)
        .With("sometexthole", fun x -> x.SomeTextHole)
        .With("somedatahole", fun x -> x.SomeDataHole)

let Main context endpoint title someTextHole someDataHole : Async<Content<EndPoint>> =
    Content.WithTemplate Template {
        Title = title
        SomeTextHole = someTextHole
        SomeDataHole = someDataHole
    }

As you can see, I added a new SomeDataHole field to our AdvancedPage type and mapped it to ‘somedatahole’. SomeDataHole is of type list<Element> where Element is a WebSharper type that can represent HTML content. We are ready to make our first test. Let’s extend our TemplatesPage function from the Site module to inject some HTML content into our page:

let TemplatesPage context =
    AdvancedTemplating.Main 
        context
        EndPoint.Templates
        "Templating in WebSharper" // Our title hole
        "Just a test page to play with the HTML templating engine in WebSharper!" // Our sometexthole hole
        // Our server-side data-hole content starts here
        [
            H1 [Text "This is inserted via a data-hole attribute (server-side)"]
            H3 [Text "Let's test a few HTML elements like H3..."]
            H4 [Text "... or unsorted lists:"]
            UL [
                LI [Text "one item..."]
                LI [Text "Another item!"]
                LI [Attr.Class "someStyle"]
                    -< [Text "and yet another "]
                    -< [B [Text "bold item"]]
                    -< [Text " with a style attached"]
            ]
        ]

Note that this will generate the HTML on the server-side. The gives us the following result (the content of the data-hole is represented by the red frame):

You may be wondering what is the strange -< operator doing in the following piece of code:

LI [Attr.Class "someStyle"]
	-< [Text "and yet another "]
	-< [B [Text "bold item"]]
	-< [Text " with a style attached"]

This operator simply allows to construct a new element based on a list of other elements. Here, the three text elements are merged into one and the CSS class ‘someStyle’ is applied to it. We can check it by opening the web console in the browser:


Next on the list, the data-replace attribute.

Playing with data-replace

I modified the code further to illustrate how the data-replace attribute works. Here is the gist of it:

<!-- HTML file -->
<body>
    <div data-replace="somedatareplacehole"></div>
</body>
// AdvancedTemplating.Template function
.With("somedatareplacehole", fun x -> x.ServerSideDataReplaceHole)

// AdvancedTemplating.Main function
let Main context endpoint title someTextHole someDataHole serverDataReplace : Async<Content<EndPoint>> =
    Content.WithTemplate Template {
        Title = title
        SomeTextHole = someTextHole
        SomeDataHole = someDataHole
        ServerSideDataReplaceHole = serverDataReplace
    }

// And finally, Site.TemplatesPage function
let TemplatesPage context =
    AdvancedTemplating.Main 
        
        // rest of the arguments...

        // Our server-side data-replace content starts here
        [
            H1 [Text "This is inserted via a data-replace attribute (server-side)"]
        ]

Here is the result when we run the program and load the page again:In the web console, we can effectively see that the whole div element was replaced by  h1:


Small bonus with data-replace

While going through the official documentation and various posts on the forum, I also discovered that data-replace can also be used to generate HTML that automatically includes all external JavaScript or CSS files required to run your page. This intrigued me so I decided to investigate. It took me quite some time to get it right, so the below will hopefully save you some time. First, here are the links to the related documentation:

In short, the documentation states the following that using data-replace=”scripts” in your HTML template will auto-load the required CSS / JavaScript dependencies to your page. I added a new, separate custom_styles.css file to the project and decided to check if WebSharper would be smart enough to load it automatically on the pages that were dependent on it. I added the following configuration to register the new css file:

<!-- HTML file -->
<body>
    <div data-replace="somedatareplacehole"></div>
    <script data-replace="scripts"></script>
</body>
<!-- CSS file -->
.styleFromSeparateCssFile {
    color: darkmagenta;
    font-size: large;
    background: lightblue
}
// HTML to be generated
Div [
    H1 [Attr.Class "styleFromSeparateCssFile"] -< [Text "This is inserted via a data-replace attribute (client-side)"]
    P [Attr.Class "styleFromSeparateCssFile"] -< [Text "This content is generated on client-side by JavaScript code!"]
]
module Resources =
    open WebSharper.Core.Resources

    // Declare resource files.
    [<assembly:System.Web.UI.WebResource("custom_styles.css", "text/css")>]
    do ()

    // Declare types for automatic dependency resolution
    type Styles() =
        inherit BaseResource("custom_styles.css")

And used the following Required attribute on my Site.TemplatesPage function to let WebSharper know of the dependency:

[<Require(typeof<Resources.Styles>)>]
let TemplatesPage context =
// rest of the code...

But it didn’t work! I tried to add the Require attribute to other functions, modules an even to the whole assembly, without success. After reading the documentation in details once again, I managed to find out what was wrong:

  • The CSS file needs to be added as an Embedded Resource to the Visual Studio project.
  • The mechanism only works for client-side generated HTML!

I changed the F# code so the HTML would be generated on the client-side instead:

[<JavaScript>]
module Client =
	[<Require(typeof<Resources.Styles>)>]
	let ReplaceDataExample () =
	    Div [
	        H1 [Attr.Class "styleFromSeparateCssFile"] -< [Text "This is inserted via a data-replace attribute (client-side)"]
	        P [Attr.Class "styleFromSeparateCssFile"] -< [Text "This content is generated on client-side by JavaScript code!"]
	    ]
module Site =
    let TemplatesPage context =
        AdvancedTemplating.Main 
        
            // rest of the arguments...
            // Our client-side data-replace content starts here
            [Div [ClientSide <@ Client.ReplaceDataExample() @>]]

And tadaaa, it worked!

As shown below in the web console, WebSharper automatically added our custom_styles.css file to the page:

However, it didn’t add it to our root page (the one under localhost:9000) because the CSS file was not needed there! Cool stuff.

Let’s wrap up!

That’s all for this post about data-hole and data-replace attributes in WebSharper, I hope this helped you understand better the basics of WebSharper’s templating engine. There’s more to talk about on the subject but that will be the topic of a future post. Once again, you can find the full code of the application on my GitHub repo.

Cheers!

Help yourself and your programmers by writing better specs!

Hi, as promised in my last post where I wrote about the scary world of project failures in IT, I am back today to help you avoid getting yourself in a similar situation. How so? By giving you a few tips on how to write better specs! Being able to capture requirements that are clear for both business and IT is a vital part of any Software project, yet I feel this part is often overlooked. As requirements are meant to convey the whole meaning of a project, this of course causes a lot of issues down the line.

In this post I will be freely using the term specifications, or simply specs, to refer to the document(s) handed over to the programmers so they can start implementing the Software. Now let’s try to answer the following questions:

  • What are the signs that you might have insufficient specs?
  • How should the specs look like?
  • Who should write specs?
  • What should the specs be about?

What are the signs that you might have insufficient specs?

Below is an non-exhaustive list of signs that may show that your project lack consistent and accurate specs:

  • high technical debt. Technical debt can be born from inaccurate specs that lead programmers to implement things they do not fully understand or are not necessary, thus introducing additional complexity.
  • Missing or Incomplete documentation. Good specifications often translate into good documentation as they should reflect precisely what the Software actually does.
  • Failure to meet deadlines. You cannot expect programmers to implement a program in a timely fashion if you are not able to explain in plain English what the said program should do or not do.
  • Angry users and angry programmers. The former complain because the Software does not do what it is supposed to (or does it in a very unintuitive and inefficient way). The latter complain because they were not given a fair chance to do good work.

If you experienced any of those in the past, you might find some valuable information down there.

How should the specs look like?

I believe most of us have played at least once in their life the popular game called Chinese Whispers. If not, I highly recommend you do so. It can be a lot of fun while teaching a very valuable lesson:

We humans often fail at communicating in an accurate and effective fashion.

This is even more true while using verbal communication. What’s the point of all this? Well, if you don’t want your application which ‘helps seniors share pictures of their pets in one click‘ to turn into a program that ‘takes photos of your dogs while they lick the pastor‘, then you’d better put those specifications on paper! Having written specifications is mandatory if you don’t want any important piece of information to be forgotten, lost, or even worse, misinterpreted.

So Alexandro, what’s the best format to write specs then?‘. This is my very personal opinion, but I believe that you should favor wiki-based specs over office-like documents, for the following reasons:

  • We all know the pain of sharing documents with 10 other people over email and exploding everyone’s mailbox quota.
  • There were always moments when Jessica made modifications to final_specifications_january_v4-5-31_final_bis_draft1.docx while Gregory was already working for a few days on specifications_january_v4-5-37_lastest_final_bis_bis_2.docx.
  • We all lost our sanity while editing a Word document in review mode, so full of red squiggles and comments that we needed a ultra-wide monitor to display it all.
  • We despise SharePoint from the bottom of our soul.

At work I am currently using (and abusing) Atlassian Confluence, which in addition to being super intuitive and ergonomic, allows you to:

  • easily share your specs by sending an URL.
  • quickly compare changes thanks to the built-in version control system.
  • Comment and collaborate on the same document in near-real time.

It is so easy to use that you might even start to like writing documentation! Writing specs is not going to be the sexiest thing you’ll do in your life, so make sure you have all the proper tools at hand to make your job easier. But whose job it is anyway? Let’s discuss it.

Who should write specs?

Let’s state this right away so it is clear for everyone:

No, programmers should NOT be the ones writing the specifications!

Why so? Well, the programmers are the consumers of the specs. Their job is to thoroughly review the specs and find potential anomalies or inconsistencies in them before heading to the implementation part. They cannot do that if them wrote them themselves. Also, as we will see below, a programmer’s mind is programmed (pun intended) in a certain way that makes it difficult for him to write specifications that are truthful to the business.

So we now know who shouldn’t write the specifications. But who should then? The clients or end users of the application? The project manager? The product owner? The business analyst? The answer is all of them, together, to avoid what is described in the famous drawing below:


Having end users take part in capturing requirements seems like a no-brainer. After all, those are the ones the Software will be written for and therefore should have a say in its making. In practice though, this is not always the case. The reasons are various: they sometimes don’t have time for it, or worse, no one actually saw fit to ask for their opinion.

The business analyst should also be a clear pick here, as he often is the bridge between business and IT, being able to understand both worlds. I’d argue that QA specialists, due to the very nature of their role, should also be involved in preparing specifications. This would bring the positive side effects:

  • Engagement of QA early in the project.
  • Avoiding situations where programmers have to explain to the QA peeps what the Software should do, thus defeating the point of doing QA in the first place.

Generally speaking, having a diversity of actors taking part in requirements gathering is a good thing and naturally contributes to having clear specifications. To sum up, having at least an end user, a business analyst and a QA specialist work together on the specs is a good idea.

What should the specs be about?

Good specifications should ideally contain enough information so none of the questions the programmers typically ask are left unanswered. What do programmers typically ask about? Let’s try to put ourselves in the shoes (and mind) of a programmer. Beware, sensitive souls refrain!


I wrote down a simple pitch of an application to serve as an example. Let’s give the application an original and catchy name: PizzaManager. Here it the pitch:

This new application will help our Italian restaurants manage our large choice of pizzas in an intuitive manner. The main screen should allow users to:

  • Display the list of pizzas available on the menu.
  • Add new pizzas to the menu.
  • Edit existing pizzas.
  • Remove pizzas from the menu.

Yipee! We might think that this is enough to get development starting, but this is very far from the reality. Below is a non-exhaustive list of questions your programmers will sooner than later have to answer when they go down the development road.

Hardware / platform requirements:

  • Does it have to be a desktop client? A website? a mobile application?
  • What OS should it run on? What is the hardware available in the restaurants?
  • Is touchscreen support needed?
  • Where should the data be stored? On a private centralized server? In the cloud? On the local machine? How will handle backups?
  • Will the DEV team have access to a realistic test environment?

Performance concerns:

  • How many users can be logged in at the same time?
  • How many pizzas should the application be able to handle?
  • How many actions per second/hour/day/month should the system process?
  • What are the expected response times from the system for each given action?

User Interface design:

  • How should the pizzas be displayed? In a grid or in a list? Maybe both?
  • How many attributes (size, price, name, ingredients, …) should be displayed?
  • How many pizzas should be displayed simultaneously on the screen?
  • Are advanced UI functionalities required (responsive design, drag&drop…)?
  • Should users be able to sort or filter the list of pizzas?
  • Can the user interface be customized?
  • Should the screen display pictures of the pizzas too?

Business requirements:

  • How can users log in to the system? Via password, smart card, badge, fingerprint…?
  • Can users be assigned roles and permissions, e.g. standard user versus administrator?
  • What are the validation rules to apply to a pizza?
  • Should the application be available in several languages?
  • Is an integration with the main website or any external system planned in the future?
  • Can users edit several pizzas at once?
  • Can users create a new pizza by copying an existing one?
  • Can users delete several pizzas at once?
  • Can there be two identical pizzas in the list under different names?
  • Should there be an additional prompt message displayed when deleting a pizza?
  • Is there any need for auditing capabilities (who created / edited / deleted pizzas…)? What is the expected granularity?

Migration:

(This one is often omitted during requirements gathering. I spent 18 months working exclusively on a data migration for a single medium-size project, so I can tell you this is not something you should take lightly!)

  • Who is going to prepare the data for the migration?
  • What are the differences in terms of data format between both systems?
  • Who is going to run the actual migration from the old system to the new one?
  • How will the migration be done? Database migration, manual insertion, custom interface and logic…?

We have around 30 questions already,each of those having a significant impact on the end product! The answers to all those should figure somewhere in the specs. Otherwise, the following can (and will) happen:

  • Programmers will realize that a big breaking change is required right in the middle of the implementation, because they did not foresee one specific case. This will introduce delays and technical debt in the project.
  • Programmers will make decisions driven by the technologies (the languages, databases or frameworks being used) to compensate for the lack of guidance that should have been provided by the specifications.

As you can see, programmers have many things to account for, and adding unclear business requirements to the balance might very well bring the project to his knees. Fortunately, you, as a specifications writer, can reuse the below collection of must-have items to ensure that your specs at least answer a minimum number of those concerns.

Explain the context

It is very important to set the context and scope of a project straight from the beginning. The initial pitch mentions Italian restaurants (plural). How many of them are those? 3 or 300? Will the application be used in each of them? Gather the information and put it at the beginning of the specifications.

Is the new PizzaManager application going to replace an existing system? If yes, what were the advantages and drawbacks of the old system? When is it going to be replaced? Why did they want to go with a new system? Answering all those questions on paper will help you identify a lot of hidden problems (read, hidden costs) and come up with appropriate solutions.

Similarly, you must define who your team will be working with: does the client have his own IT or DEV team? Are they responsible for some of the project deliverables? Will your programmers have to interact with them in any manner?Identifying all the actors within the project is crucial, so as usual, gather the answers and write them down in the specs.

Explain the domain

The application pitch is full of nouns that you must explain in great detail. For instance, the concept of menu seems pretty blurry and should be clearly defined: The menu refers to the whole list of pizzas currently available for sale within the application. It is not related with the physical menus handed out to clients of the restaurant.

The same applies to users. Define who will be using the application and what will be their role:

  • Managers can delete pizzas and change their prices.
  • Waiters can create and edit pizzas. They cannot change their prices.
  • Pizzaiolos can only browse the list of pizzas and see their main ingredients.

The pitch also contains several verbs, like manage, add, edit, remove, display… You must as well define what those verbs mean, for instance:

  • Removing a pizza will permanently delete the entry from the database.
    versus
  • Removing a pizza will only make it disappear from the main list of pizzas on sale. It can be added back later on by a manager.

Define a list of immutable business rules

In any application, there are cornerstones that prevent all hell from breaking loose on the poor users. The specifications should explicitly state those immutable business rules on which the Software relies. In the context of PizzaManager, it could be:

  • Only managers can remove pizzas from the system.
  • The system should not allow to remove the top 5 best-selling pizzas.
  • A pizza should never contain both pineapple and broccoli at the same time!

Those rules will allow your programmers to build the Software on stable foundations. Rules are usually easy to reason about and will bring sanity back in your DEV team.

Describe in details all business scenarios

All actions available to the users should be listed and described in details. The pitch mentions the following scenario: Add new pizzas to the menu. This is way too vague and your programmers will surely come to you and ask multiple questions about it. You can avoid that by explicitly stating what the action is meant to do:

  • Log in the application and navigate to the main screen.
  • Click on the Add button in the ribbon. The pizza creator wizard will appear as a popup.
  • Choose the list of ingredients the pizza will be composed of.
  • Click on the Create button at the bottom of the wizard.
  • The pizza you created will now appear on the main screen.

Writing down those scenarios will also help you verify that your immutable rules actually make sense! Additionally, there are a great starting point for your QA specialists.

But do not stop at the happy path!

The scenario we described below only goes through the happy path. However in real life users may encounter issues from different natures while using the application. Those issues also need to be addressed in the specifications. For instance:

  • After you clicked on the Create button, an error message will be displayed if the pizza already exists in the system: ‘Another pizza with similar ingredients already exists in the system. Duplicates are not allowed.’
  • If the connection to the main server was lost while creating the new pizza, the data should be stored locally until the connection is back.
  • If you are currently editing a pizza that was removed from the system in the meantime, a notification should be displayed at the top of the screen: ‘the pizza you are editing does not exist anymore’.

The general idea is to not let any element left to improvisation. Errors and failures will happen, so they should be taken into account in the specifications.

Define metrics up front

Last but not least, having an idea of how a system is expected to react is vital in order to properly choose the proper architecture and technologies for the job. Define those expectations in the specs as well:

  • Users are expected to create between 100 and 250 new pizzas per day.
  • Around 40 users will be simultaneously logged in in the system at all times.
  • Loading the the whole list of pizzas in  the user interface should take more than 2 seconds.
  • A pizza created in 1 restaurant should be available in the other 30+ restaurants after 1 minute at max.
  • Data should be backed up 4 times a day.

If you see metrics like those, you will know that what you are expected to build is not just another simple restaurant manager but in fact a complex piece of software with thousands of entities, near real-time synchronization between shops and a few dozen concurrent users logged in.

Let’s wrap up!

That’s it for today. I hope I helped you understand why gathering requirements is such a vital part of Software development. Writing clear, complete and consistent specifications is not an easy job but the success of your project largely depends on it. Hopefully the ideas and concepts above will help you get started. Happy writing!

Cheers

“Oh no, why is my software broken again?” A scary IT story

Today I am going to leave the technical stuff on the side and take a wider look at the Software industry. By reading the news you might get the impression that everything is nice and well in IT Land. Between success stories of startups worth millions or even billions of dollars and computer guys making a six-figures just by typing on a keyboard, it might seem like Software is the New Eldorado. Coding bootcamps and other programming schools are on the rise, promising that you will become a Web Developer in under 3 months! and live the millionaire life in just double that time.

Well, today I want to focus on the other side of the coin, and talk about a scary, yet very real aspect of our industry: project failures. If you type ‘software project failure rate‘ in your favourite search engine, you’ll find a lot of interesting statistics that seem to repeat over the last 10 to 15 years. Here are some of the crispy bits that I found:

  • only 39% of all projects succeed (delivered on time, on budget, and with required features and functions).
  • One in six IT projects have an average cost overrun of 200%.
  • 75% of IT executives believe their projects are ‘doomed from the start‘.
  • Only 2.5% of companies successfully complete 100% of their projects.
  • The United States economy loses $50-$150 billion per year due to failed IT projects.

That doesn’t look good at all. if the sight of those statistics makes your legs weak and palms sweaty, perfect. That was the desired effect. If not, you might want to read those once again. The message is pretty clear: Software development is hard. Anyone telling you otherwise is lying. It requires a lot to get it right. It takes very little to get it wrong.

Tell us why, Johnny! Why is this so hard to make software?!. Rest assured, for the past decades Mankind has spent a tremendous amount of time, energy, and money trying to answer this question. So what are the usual suspects? First, Let me give you my personal take on the matter (caution, touches of satire ahead).

Business vs IT

Those two main actors of almost any significant Software project seem to find an endless source of joy in making each other’s life miserable. The business will often report production issues at 5pm on Fridays and make constant and meaningless requests for things they won’t be using after all. Meanwhile, IT will plan a full upgrade of all servers right in the middle of the month-end closing and devise vague error messages because… Well because they can.

How IT views Business…
and how Business views IT!

Methodologies

– My Scrum is better than your Kanban!
Nah, my Waterfall is stronger and it will kick your Agile right in the n…!

Software development methodologies have been seen by many as Silver Bullets, the Holy Grail of IT that would solve all our problems. In practice though, they do not always prevent projects from failing.

Management

This is a tough one. Where to start?
– Office politics and people undermining the project to serve their own interests? Check
– Favoring reporting work over doing work (time-sheet hell)? Check
– Decision makers completely out of touch with the realities of Software development? Check
– Refusal to invest in tools and better processes? Check
– Fear of change? Check
And the list could go on…

But more seriously though, let’s have a look at some other figures I extracted from this source:

Most Common Causes of Project Failure:
Changing priorities within organization – 40%
Inaccurate requirements – 38%
Change in project objectives – 35%
Undefined risks/opportunities – 30%
Poor communication – 30%
Undefined project goals – 30%
Inadequate sponsor support – 29%
Inadequate cost estimates – 29%
Inaccurate task time estimate – 27%
Resource dependency – 25%
Poor change management – 25%
Inadequate resource forecasting – 23%
Inexperienced project manager – 20%
Limited resources – 20%
Procrastination within team – 13%
Task dependency – 11%
Other – 9%

There are a few interesting observations we can make based on that list.

  • Project failures seem very rarely caused by technical problems! Isn’t it surprising, considering the very nature of Software development? Those damn developers watching cat videos during work hours might not be totally responsible for that disaster, after all.
  • Project management issues are omnipresent. Failure to plan, adapt, or communicate is all over the place. According to this article from Joseph Gulla, a former IT specialist at IBM, ‘54 percent of IT project failures can be attributed to project management, whereas only 3 percent are attributed to technical challenges’.
  • The third point is curiously something that people very often tend to forget about: the lack of discipline in capturing requirements. Look again at those items:
    • Inaccurate requirements – 38%
    • Change in project objectives – 35%
    • Undefined project goals – 30%
    • Poor change management – 25%

This last point is the most striking to me for two reasons. First, requirements are the foundations upon which a project is built up. You probably know what would be the expected consequences of building a house foundation in a swamp. The results are not much different in the Software world.

Secondly, it is astonishing that such a fundamental part of the project is so frequently overlooked, and sometimes even simply ignored. People keep repeating the same mistakes again and again.

Now, it would be very presumptuous of me to criticize without at least suggesting a few ways to improve the current state of things. This is why I will be following up shortly with another post dedicated to capturing requirements and helping your programmers get their Software right.

Did you also encounter IT project failures during your career? Don’t hesitate to share them with us in the comments below!

Till next time!

Overview of WebSharper’s templating engine – Part 1, Text holes

In the previous Hello WebSharper post i intentionally left a few questions unanswered:

  • What is the role of the Main.html file in the project?
  • What solutions does WebSharper offer do interact with HTML content?

We will try to answer those questions in a series of posts dedicated to WebSharper’s capabilities.

General overview of the templating engine

WebSharper comes with a templating engine that helps programmers manipulate their HTML files from F# by using special attributes. Those attributes create holes in the HTML structure that can be filled with data using logic described in F#.

In our Main.fs file, we declared a few modules that we didn’t cover in the previous post. One of them is the Templating module that declares a new record type that represents the structure of a typical page from our website (check out the full code on the GitHub repo):

type Page = {
    Title : string
    Body : list<Element>
}

In our case, a page is basically made of a Title (some text) and a Body (a list of HTML elements such as div, h1, or p). We then define the MainTemplate function that will map our Page records to an existing HTML file:

let MainTemplate =
        Content.Template<Page>("~/Main.html")
            .With("title", fun x -> x.Title)
            .With("body", fun x -> x.Body)

Content.Template is a WebSharper type that returns an HTML template from a file passed as parameter, in our case Main.html. The two With functions allow to map data defined in F# with holes declared in the HTML markup. For instance, the following line maps the ‘title’ hole to the Title field from our Page record:

.With("title", fun x -> x.Title)

Time to see how those holes are defined in our HTML files. In today’s post, we are going to focus on the simplest type of holes available, text holes.

Warming up with text holes

Text holes, as you could guess, can only contain text. The Title field that we just saw above is mapped to a text hole. This is how the hole itself is defined in Main.html:

<title>${title}</title>

As you can see, a text hole is basically declared as follows: ${name_of_the_hole}

I added to the website a separate Templates page (accessible at localhost:9000/templates) and a new AdvancedTemplating module to help test those bindings. We will extend this module as we go along with this series. Here is the F# code that defines the template and bindings for this page:

module AdvancedTemplating =
    open System.Web

    type AdvancedPage = {
        Title: string
        SomeTextHole: string
    }

    let Template =
        Content.Template<AdvancedPage>("~/AdvancedTemplate.html")
            .With("title", fun x -> x.Title)
            .With("sometexthole", fun x -> x.SomeTextHole)

    let Main context endpoint title someTextHole : Async<Content<EndPoint>> =
        Content.WithTemplate Template {
            Title = title
            SomeTextHole = someTextHole
        }

And here is the associated advancedTemplate.html file, very simple for now:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8" />
    <title>${title}</title>
    <style></style>
</head>
<body>
    <H1>${sometexthole}</H1>
</body>
</html>

We can count two text holes there: ‘title’ and ‘sometexthole’. The last piece of the puzzle is the creation of the page itself in the Site module:

let TemplatesPage context =
    AdvancedTemplating.Main 
        context
        EndPoint.Templates
        "Templating in WebSharper" // Our title hole
        "Just a test page to play with the HTML templating engine in WebSharper!" // Our sometexthole hole

Finally, let’s check how the new page looks:

It worked! ‘Templating in WebSharper’ got mapped to the title HTML tag (the caption of the tab in the browser) while ‘Just a test page to play with the HTML templating engine in WebSharper!’ got mapped to the main H1 element on the page.

As a bonus, we can change the look of our page just by editing our HTML file, without having to recompile the whole F# project. This is useful as your designer can freely fiddle with the UI without bothering the programmers in the team.

That’s it for today. Stay tuned for the next post in the series where we will cover a slightly more complicated mechanism: data holes.

Cheers!

Integrating the Polish surname guessing into our WebSharper website

After spending a few posts on F# itself, It is high time we went back to our old friend WebSharper. I decided to integrate the Polish surname guessing functionality into the website we created in our Hello WebSharper post. This is how it looks like now:

(click on the GIF to enlarge)As you can see on the GIF above, the server can answer in 3 different ways:

  • ‘A stranger has no name?’ if the user didn’t type anything.
  • A prompt to the user to enter his surname too, if he only typed his first name.
  • The origin of the surname as determined by the server, if the user typed both his first name and surname.

You can read more about the Polish surname guessing algorithm here and here. Below is the list of notable changes I made to the website to make it work.

PolishSurnames.fs

I extracted all the functions from AreYouPolish.fsx and created a proper PolishSurnames module (source code here). The most notable refactor was the addition of a proper record type to store the results of the computations:

type SurnameOrigin = DefinitelyPolish | ProbablyPolish | NotPolish

type SurnameStatistics = {
    Surname: string
    Origin: SurnameOrigin
    PolishDensity: float
}

That way it will be easier to pass the data around, as we will see below. I also changed the decideIfPolish function accordingly:

/// Decide if a word is Polish based on its density
let private decideIfPolish (word, density) =
    if (density < 0.2) then {Surname = word; Origin = SurnameOrigin.NotPolish; PolishDensity = density }
    else if (density >= 0.2 && density <= 0.8) then {Surname = word; Origin = SurnameOrigin.ProbablyPolish; PolishDensity = density }
    else {Surname = word; Origin = SurnameOrigin.DefinitelyPolish; PolishDensity = density }

Remoting.fs

I added the new IsSurnamePolishAsync function to Remoting.fs so our JavaScript client can call it directly.

[<Remote>]
let IsSurnamePolishAsync username =
    async {
        let helloMessage = sayHello username       
        let surnameOnly = PolishSurnames.TryExtractSurname username
        
        match surnameOnly with
        | None -> return (username, helloMessage, None)
        | Some surname -> return (username, helloMessage, Some (PolishSurnames.IsSurnamePolish surname))
     }

The  function returns a tuple containing elements:

  • The original username passed as input
  • A custom Hello message based on username
  • Some SurnameStatistics record if a surname could be found, or None otherwise.

This naturally means I had to update the callsite in Client.fs too!

Client.fs

The previous version looked as follows:

async {
    inputValue.Text <- input.Value
    let! data = Server.SayHelloAsync input.Value
    output.Text <- data
}

While the new version is a bit more complex:

async {
    inputValue.Text <- input.Value
    let! (username, helloMessage, stats) = Server.IsSurnamePolishAsync input.Value
    helloOutput.Text <- helloMessage
    
    if username = "" then statsOutput.Text <- "No data, no stats! What were you thinking?"
    else               
        match stats with
        | None -> statsOutput.Text <- "That's a nice name. We could tell you interesting stuff if you provided us with a surname too!"
        | Some s -> statsOutput.Text <- DisplaySurnameStats s
}

This line shows nicely the tuple we saw above being deconstructed into 3 parameters username, helloMessage and stats:

let! (username, helloMessage, stats) = Server.IsSurnamePolishAsync input.Value

Now I have to confess that I didn’t unit-test all those new functions just yet. Nor did I write integration tests, for that matter… This is bad, I know!But fear not: I will write a whole new post dedicated to unit tests in F# soon. To conclude this post, let’s have a look at some JavaScript code generated by WebSharper. The client-side F# DisplaySurnameOrigin function:

let DisplaySurnameOrigin origin =
    match origin with
    | SurnameOrigin.NotPolish -> "not Polish"
    | SurnameOrigin.ProbablyPolish -> "probably Polish"
    | SurnameOrigin.DefinitelyPolish -> "definitely Polish"

is transpilated to the following JavaScript:

DisplaySurnameOrigin:function(o) {
    return o.$==1? "probably Polish" :
	   o.$==0?"definitely Polish" :
	   "not Polish";
}

If we look at the Json data returned from the server for Adam Młynarczyk, we can see that the value 0 in fact represents SurnameOrigin.DefinitelyPolish.

{
   "$DATA":[
      "Adam Młynarczyk",
      "Hello, Adam Młynarczyk!",
      {
         "$V":{
            "$":1,
            "$0":{
               "$V":{
                  "Surname":"Młynarczyk",
                  "Origin":{
                     "$V":{
                        "$":0
                     }
                  },
                  "PolishDensity":1
               }
            }
         }
      }
   ]
}

Wow, that’s a lot of brackets. Another thing I love about F#’s syntax is that I don’t have to deal with those anymore!

That’s it for today, thanks!

Make your pizzas taste better thanks to the F# Forward Pipe operator!

In today’s post we will take a closer look at the F# forward pipe operator represented by the following |> symbol. We already saw this operator in action in my two previous posts about Polish surnames that you can find here and here. In this session we will go over:

  • The definition and basic usage of the |> operator
  • An advanced example written in both C# and F#, illustrating the expressive power of |>

Back to Basics

If we search for the definition of the operator in the source code, we can find this:

/// Apply a function to a value, the value being on the left, the function on the right
/// arg: The argument.
/// func: The function.
val inline ( |> ) : arg:'T1 -> func:('T1 -> 'U) -> 'U

The function signature can be a bit scary if you’re not familiar with F#, but in reality it is pretty straightforward. Let’s look at an example:

let add a b = a + b
let res1 = add 2 3 // res1 = 5

We just defined a new add function that takes 2 operators a and b and sums them. Then we execute this function once and store the result in res1, 5 in this case. But what if we used the |> operator to achieve the same effect:

let res2 = 3 |> add 2 // res2 = 5

Or, with a bit of extra formatting:

let res3 =
    3
    |> add 2

What happens is that we simply take the value on the left of the operator, and we pass it (or pipe it) as the last parameter of the function on the right of the operator. And that’s all we need to know. Nothing more, nothing less.

Right now you’re probably thinking “But why, Roberto? Why!?”. That’s a very pertinent question. My answer is — and I feel this deserves the HTML <blockquote> treatment —

because it allows you to chain an arbitrary amount of function calls while keeping your code readable and elegant.

Here Be Pizzas

That was pretty simple. We need a more advanced example to illustrate the true power of the forward pipe operator: Pizzas


Let’s have a look at the following C# service that makes and delivers pizzas (I voluntarily left out the implementation details so we can focus on the topic at hand). The full code is available on this public gist.

public class PizzaRecipe {}
public class PizzaOrder {}
public class ColdPizza {}
public class HotPizza {}
public class PizzaDelivery {}

public static class PizzaService
{
    public static PizzaOrder OrderPizza(PizzaRecipe recipe, string size)
    {
        return new PizzaOrder();
    }
    
    public static ColdPizza PreparePizza(PizzaOrder order)
    {
        return new ColdPizza();
    }
    
    public static HotPizza CookPizza(ColdPizza pizza, int temperature)
    {
        return new HotPizza();
    }
    
    public static PizzaDelivery DeliverPizza(HotPizza pizza)
    {
        return new PizzaDelivery();
    }
}

Now, if we wanted to use this API and implement the full pizza delivery process, we’d certainly have a few design options. The first one would be:

// The ol' good way.
public PizzaDelivery BigPizzaProcess(PizzaRecipe recipe)
{	
    var order = OrderPizza(recipe, "big");
    var coldPizza = PreparePizza(order);
    var hotPizza = CookPizza(coldPizza, 180);
    var delivery = DeliverPizza(hotPizza);
	
    return delivery;
}

This is pretty standard, quite a lot of local variables but that does the job. There’s also this option:

// The Arabic way: you read from right to left!
public PizzaDelivery BigPizzaProcess(PizzaRecipe recipe)
{	
    return DeliverPizza(CookPizza(PreparePizza(OrderPizza(recipe, "big")), 180));
}

Although I might scream a bit if I saw something like this on production! As you can see, you’d have to read it from the right to the left to get the proper order of execution.Not very convenient. Also, adding new steps to the process would quickly turn the whole thing into a mess.

Finally, you might want to write a few extension methods:

public static class PizzaExtensions
{
    public static PizzaOrder Order(this PizzaRecipe recipe, string size)
    {
        return PizzaService.OrderPizza(recipe, size);
    }
	
    public static ColdPizza Prepare(this PizzaOrder order)
    {
        return PizzaService.PreparePizza(order);
    }
	
    public static HotPizza Cook(this ColdPizza pizza, int temperature)
    {
        return PizzaService.CookPizza(pizza, temperature);
    }
	
    public static PizzaDelivery Deliver(this HotPizza pizza)
    {
        return PizzaService.DeliverPizza(pizza);
    }
}

And end up with something like this:

// The nice way!
public static PizzaDelivery BigPizzaProcess(PizzaRecipe recipe)
{
    return recipe
        .Order("big")
        .Prepare()
        .Cook(180)
        .Deliver();
}

Now we’re talking! The code is clean and it almost reads like natural English. Even someone with little or no knowledge in programming would be able to quickly guess what the code is doing. I actually conducted the experiment with my girlfriend and she understood and explained the code in less that 30 seconds. Please try this at home and let me know of the results in the comments section!

It took some extra effort to get there though. The extensions methods that we created will need to be maintained and tested too. Nothing too complex so far, but it needs to be done anyway. Let’s have a look at the F# equivalent now:

open System

type PizzaRecipe() = class end
type PizzaOrder() = class end
type ColdPizza() = class end
type HotPizza() = class end
type PizzaDelivery() = class end

let order (size:string) (recipe:PizzaRecipe) =
    PizzaOrder()

let prepare (order:PizzaOrder) =
    ColdPizza()

let cook (temperature:int) (pizza:ColdPizza) =
    HotPizza()

let deliver (pizza:HotPizza) =
    PizzaDelivery()

let bigPizzaProcess (pizzaRecipe:PizzaRecipe) =
    pizzaRecipe
    |> order "big"
    |> prepare
    |> cook 180
    |> deliver

That’s it, really! Thanks to the Pipe operator in F#, we achieved the same nice and readable code as the C# version, but without the hassle of setting up all the additional extension methods. One could argue that the F# version falls even closer to natural English, due to the absence of the all the parentheses, brackets, dots and semicolons that are present in the C# sample.

As a result, the F# code is much more compact with only 26 lines of code versus 65 lines for the C# version. That’s 40% less code to test and maintain! What we have here is only a small example, but I let you imagine what would that mean for a large-scale application.

That’s all for today. As always, don’t hesitate to leave your comments or questions at the bottom of the page.

Cheers!