Chris James - Software developer and other things

Why you should deploy on Friday afternoon

28 May 2018

I originally posted this at Dev.to where lots of people agreed and disagreed.

Don't deploy on Friday afternoons!

This expression is taken as programmer wisdom but I hate it. I'm going to try and kill it, with words and experience.

The motivation behind it is sound. I don't want to spend my Friday nights debugging a production problem either.

To me the expressions smacks of unprofessionalism. Software development as an industry has a poor reputation and phrases like this do not help.

If you went to an ER on a Friday afternoon and was turned away because the doctors don't trust their own tools and know-how, how would you feel?

If we want people to take our craft seriously we need to own it and not give the impression that we dont understand the systems are we making enough to make changes at the end of the week.

Why a lot of people don't want to deploy on Fridays

Continuous Delivery (CD)

I have worked on teams that have deployed new versions of various services in a distributed system multiple times at 4:30pm and not broke a sweat.

Why? Because deployment is fully automated and is a non-event. Here is the groundbreaking process.

Not so long ago it was considered normal for there to be releases every 6 months, or even just one at the end of a project.

The forward thinkers in that age saw problems with this

So the industry as a whole worked on lots of tooling, techniques and best practices to allow us to release software far quicker.

Recognising that releasing often reduces risk is generally accepted nowadays but teams still often settle on weekly or fortnightly releases; often matching the cadence of their sprints.

What are the problems with weekly/fornightly releases?

With CD we recognise that we can go further, deploying new software to live every time the build is green. This has some amazing benefits,

But what if you break things?

Often people say with CD

Yeah it's nice but what if it breaks? We should have a QA check things over

Here's the thing, no process in the world prevents bugs. You will ship broken code. What's really important is how quickly you can detect and recover from it. Hoping manual testing will catch everything is wishful thinking.

How to CD on a new project

It is much easier to do CD on a new project since you can start small and evolve.

Generally your work should be centered on delivering the most valuable user journeys first, so this is an excellent chance to practice how to ensure that feature works without any humans checking anything.

For each subsequent story ask yourself

How to CD on an existing project

Peel away at the manual process

CD up to staging.

Some companies have many environments in their delivery pipeline. A good first start is to automatically ship all the way up to the environment before live. A better step is remove as many of them as you can. It's ok to have some kind of "dev" environment to maybe experiment with but ask yourself why cant just test these things locally in the first place.

Identify a sub-system you could work with as a starting point

If you're working with a distributed system you might be able to identify a system which is easier to CD than the rest. Start with that because it'll give your team some insights into the new way of working and can help you begin to break the cultural barriers.

CD is a cultural issue as much as a technical one

Roles and responsibility

Often a product owner or project manager wants to be the one who is in charge of releasing.

There are circumstances where exposing features to users should be controlled by a non-technical member of your team, but this can be managed with feature toggles.

But the copying of code from one computer to another is the responsibility of the developers on the team. After all we are the ones who are responsible for making sure the system works. It is a technical concern, not a business one.

What do QAs do now?

CD is actually liberating for QAs

Re-evaluate your tolerance for defects

Lots of companies think they cannot have any defects and will spend a lot of time and effort on complicated, time consuming (and therefore expensive) processes to try and stop them.

But think about the cost of all this? If you push a change to production that isn't covered by tests, perhaps a CSS change; consider if it's really catastrophic if there's a small visual fault for some browsers

Maybe it is, in which case there are techniques to test specifically for this too.

Recovery

Each release you do with CD will have the following qualities

So in my experience fixing anything that falls through the cracks is easy. It's much less complicated than trying to look through 2 week's worth of git history.

I would recommend in most cases not rolling back (unless it's really bad), but just fixing the issue and releasing it. Rollback is sometimes not an option anyway (e.g database migrations) so the fact that your system is geared to releasing quickly is actually a real strength of CD.

Other quick tips

Wrapping up

This has been a small introduction to CD, it's a huge topic with plenty of resources to investigate.

Continuous delivery is a big effort both technically and culturally but it pays off massively.

I have worked on distributed systems with over 30 separately deployable components with CD and it has been less stressful than another project I've worked on that had just a handful of systems but a ton of process and ceremony.

Being able to release software when it's written puts a higher emphasis on quality and reduces risk for the team. It also forces you into using agile best practices like testable, small, independently releasable user stories.

Maybe most importantly, it demands that you understand your system and that you're not deploying to production and crossing your fingers. That feels more professional to me.

Tis the season to write Clojure

02 December 2017

[:fa (take 9 (repeat :la))]

This post is aimed at beginners who want to take a look at Clojure, a super-interesting language which is probably quite different to what you’re used to.

You will learn some basic concepts and see how to write a Secret Santa assigner because IT’S CHRISTMAS!!!!! :D

Assumptions

What is Clojure?

Getting started

Here’s a link to install a tool called Leiningen which lets you easily create Clojure projects.

If you’re on a Mac and have homebrew installed you can just run brew install leiningen. If you dont have homebrew installed, you should install it as most useful coding tools are installed using it.

Check your installation is working by typing lein in your command line, you should see a load of blurb.

The first time you run leinit may take some download the stuff it needs. It won’t be that slow on subsequence runs.

If you see it complaining about JDK

Clojure is compiled using the Java Development Kit which you might not have installed on your computer. On Mac you can use brew to install it.

brew update && brew cask install java

Assuming it’s all working you can now type lein repl, this is a great way just to experiment with Clojure as you can just type code and try it out.

The last line in the terminal should read something like user =>. If you ever wish to quit the REPL just type exit and hit return.

Type (println "Hello, world") and hit return and you should see

Hello, world
nil

What’s with the nil?

Every time you execute some Clojure like (println "hello world") it will always return a value. They are called “expressions”. This is a strength of Clojure and adds to its simplicity via consistency.

Think about some functions you have called in other programming languages, sometimes they return a value and sometimes they don’t.

Given this Javascript

function myAdder(x, y) {
     x + y
}

Oops no result! You forgot to put return.

This approach results in Clojure being more terse, you don’t have to worry about explicitly returning, imagine how rubbish this would be (return + 20 (return - 50 20).

In the case of println it can’t return anything meaningful as it just prints to the screen so it return nil which means “nothing”. The REPL always prints the result of the expression you run, so in the last case you see our println being printed, and then the result of the expression being printed (nil).

If you try (+ 20 50) you will see it just prints => 70.

REPL tips

Clojure vs Javascript

Here’s some examples of some common operations you’d do while writing an application in both Clojure and Javascript. Try out the Clojure examples in the REPL and get in the habit of using doc

Note the variety of syntax for doing these tasks in Javascript compared to Clojure.

This isn’t picking on Javascript in particular, this is true of most mainstream programming languages.

Adding

Javascript: 1 + 1

Clojure (+ 1 1)

Calling a function

Javascript alert('Chris')

Clojure (println "Chris")

Flow control

Javascript

if(CHRISTMAS){
    return “PUT NUTMEG IN EVERYTHING”
} else {
    return “PUT SORROW IN EVERYTHING”`
}

Clojure

(if :CHRISTMAS "PUT NUTMEG IN EVERYTHING" "PUT SORROW IN EVERYTHING")

Doing stuff with lists

Javascript ["chris","ruth"].foreach(name => println('hello' + name)

Clojure (doseq [name ["chris" "ruth"]] (println name))

How is Clojure different?

You should notice that the syntax for all of this stuff is the same

(function argument1 argument2)

And you can build more complicated expressions, just like you do in maths by nesting.

(+ 20 (- 30 50))

The inner expression gets evaluated leaving us with

(+ 20 -20)

Which is 0.

Unlike in Javascript and most other mainstream languages where arguments and function names can appear in different orders and positions, you also need to know symbols like return and =>. It’s no wonder learning programming is so difficult.

What I find the most interesting thing about Clojure is that the syntax is completely uniform. The barrier to entry to learning how to write valid Clojure is very low. Writing Clojure that actually works is still a challenge just like any other language, but the syntax is very easy to pick up.

An interesting side-effect of this approach is that you will come to understand that the code in clojure is data. When you are constructing functions like (+ 2 2) that is actually a list of data, containing functions and arguments (just like a list of names like ("chris" "ruth")).

Accomplished clojure-ists use this to write code that parses and manipulates code called “macros”. Try not to worry too hard about this right now, but it is a very interesting property of Clojure.

Secret Santa time

Now you probably feel like a Clojure expert so lets ship some epic code.

Values

When you write code you generally need to store useful information in variables for later use. To do that in Clojure, there is no magic special syntax (as promised!) just call def.

In the REPL try (def my-santas ["Chris" "Ruth" "Turner" "Hooch"]).

The only new syntax you’ve seen here is [ ] which creates a data structure called a Vector which for now you can just think of as an Array like in Ruby or JS.

If you want to get to your variable, just type my-santas into the REPL.

How do you Secret Santa?

We need some kind of algorithm to pair up people for Secret Santa. When thinking about this stuff it’s often helpful to think about how would you solve the problem in real life.

How I have done this is just one of many ways of doing Secret Santa. I fully encourage you to try different ways!

When tackling a problem you’re not sure about it’s always best to break the problem down into smaller and simpler chunks.

Representing our data

For a starting point, we need a way of representing our pairing up of Santas. The map data type is perfect for this. If you’re unfamiliar, a map is a data structure like an Array but it has a set of keys with corresponding values.

They are called HashMaps in Ruby and Objects in Javascript. You can broadly think of them as dictionaries.

We can represent our result for Secret Santa in a map (syntax for maps are {key1 value1, key2 value2})

{"chris" "ruth", "ruth" "hooch", "hooch" "turner", "turner" "chris"}

i.e Chris gives to Ruth, Ruth gives to Hooch, etc..

Creating our first representation

zipmap is a function in Clojure that takes two vectors and “zips” them into a map (funnily enough).

So try in the REPL (zipmap my-santas my-santas)

You should get {"Chris" "Chris", "Ruth" "Ruth", "Turner" "Turner", "Hooch" "Hooch"}

OK, so everyone is just giving a gift to themselves but our basic data structure is there and we can see if we can just adjust the second argument to zip-map in such a way that everyone is moved along one, it could work!

Try experimenting with zipmap, what do you think happens with (zipmap my-santas [1 2]) ?

Rotate the list

We don’t want Ruth give a gift to Ruth, if we can take our second copy of my-santas and change it then everyone will give to someone different

Imagine our room of Santas standing in a line Chris Ruth Turner Hooch

Then you clone them (like our zipmap does)

Chris Ruth Turner Hooch

Chris Ruth Turner Hooch

If we just shifted the cloned row to the right, and put the person on the end back to the beginning…

Chris Ruth Turner Hooch

Hooch Chris Ruth Turner

Everyone could take their gift and give it to the person in front of them and we would have our Secret Santas.

How do we do that?

Again, try and simplify the problem

Clojure’s amazing standard lib again provides us with last to get the last element of a vector and butlast to get everything apart from the last item. Along with the function cons which lets us create new vectors let’s stick it all together.

In the spirit of breaking things down first try (cons (last my-santas) (butlast my-santas))

Then put it together

(zipmap my-santas (cons (last my-santas) (butlast my-santas)))

You’re allowed newlines to let the code breathe a bit

(zipmap my-santas 
         (cons (last my-santas) (butlast my-santas)))

Which results in {"Chris" "Hooch", "Ruth" "Chris", "Turner" "Ruth", "Hooch" "Turner"}

Awesome! Chris gives to Hooch, Ruth gives to Chris, etc.

Randomise

This is nice but it gives us the same result every time. We can make our algorithm even better by adding some randomisation.

Shuffle

Try typing (shuffle my-santas) in the REPL and you’ll see it takes a Vector and shuffles it. This could be useful for us to make our expression a bit more exciting.

Let

let is a function that allows you to declare values that can be used inside the function passed to it

(let [x 1, y 2] (+ x y)

(Remember this might look a bit weird but it still follows the convention of (fn arg1 arg2). It's just for let its 1st argument is a vector, and then an expression to run)

In Javascript it’s:

x = 1
y = 2
return x + y

The utility is the same, when writing non-trivial code you need to capture values and name them to improve readability. Coming back to our case we want to create a variable called random-santas that stores our shuffled Santas for the rest of the code to use.

Stick it all together

Let's use these functions to make our algorithm less predictable and store some values to let our code read easier.

(let [random-santas (shuffle my-santas)
   shuffled-santas (cons (last random-santas) (butlast random-santas))]
   (zipmap random-santas shuffled-santas))

The code now reads relatively ok!

Challenge for yourself

What if you want to re-use our amazing algorithm with different sets of Santas?

You probably want to define a function, so you can use it just like we’ve used functions like shuffle and zipmap.

The basic syntax for defining your own function is

(defn name-of-function [arg1, arg2, etc] function-body)

Example

(defn add-2 [number] (+ number 2))

And then you can call it just like any other function (add-2 10)

Give it a go! Write secret-santa.

We saved Christmas!

That was probably a lot to take in but I hope it has been at least interesting. Maybe try to re-implement it yourself and see how far you get. Can you think of different ways to write it?

What might be more interesting is to implement the Secret Santa in a language you’re more familiar with. Think about how different your implementation is to the Clojure one, especially in regards to potentially all the different kinds of syntax you faced vs (fun arg1 arg2).