Our Blog

design, code, business, & everything in-between

pebblecode.com says “Hello world”

The pebblecode.com website is now on github for those interested in taking a peak.

You’ll notice that it’s a bit of a polyglot. Sinatra is the base, but there is increasingly more and more JavaScript sprinkle, with technologies such as requirejs and backbonejs. We’ve also been testing with rspec, guard, and casper.

The development workflow has also been eased with the help of tools such as Rake and Gruntjs. They allow us to do some nifty 1 line commands such as

# start the server
bundle exec rake server

# deploy to staging (and generating optimized assets beforehand)
bundle exec rake shipit[staging]   

# run all tests
bundle exec rake test:all[http://localhost:7100/]

# build optimized assets
grunt build

If you are curious, you can have a look at the Rake tasks and the Gruntfile.js file.

The pebblecode.com site, apart from being our public face, is also a platform for us to play around and try new things. Feel free to do the same with the code, and send us any comments you have. Or if you find bugs, you can report them on github issues =)

The Chrome Pixel

Chrome Pixel on loan from the good people on the google chrome Team. Thanks guys! Currently scheming on some touch screen goodness.

REST Workshop

REST API workshop

We held our first internal workshop today on REST APIs.

The video and slides are below and a link to the apiary.io discussion project can be found here.

What is REST?

  • Roy Fielding
  • HTTP
  • Distributed data
  • Web APIs (GitHub, Twitter etc)

tl;dr

GET     /users              ->  index
GET     /users/new          ->  new
POST    /users              ->  create
GET     /users/:user        ->  show
PUT     /users/:user        ->  update
DELETE  /users/:user        ->  destroy

Convention Not Standard

  • Convention is an interpretation
  • Yoda
  • Missing patterns
  • You will be annoyed

Why this convention?

  • SOAP etc
  • Clients
  • Standardisation
  • Frameworks
  • pebble case study - TaxiApp

What to think about

  • Data
  • Documentation
  • Versioning
  • Authentication
  • Deprecation

What’s the best way to learn?

  • HTTP specification
  • Build a client
  • Resources
  • Work with clients
  • Be pragmatic

REST.next or Hypermedia

  • HATEOAS
  • Discoverability
  • Smart clients
  • Work in progress

REST doesn’t work for everything

  • Event driven-apis
  • WebSockets
  • Messaging

Conclusion

  • On the web? Use REST. Use JSON.
  • Prepare for frustration
  • Be a pragmatist
  • Be responsible
  • Love your clients
  • Free your data

Assessment Anywhere

I’m pleased to be able to reveal another of great results of our recent hack day on the theme of education, codenamed “Assessment Anywhere”!

The aim was to reduce some of the difficulties surrounding teachers’ time-consuming marking processes. Our idea was to automate the assignment of grade boundaries and calculate student statistics without spreadsheets. 

The team, consisting of me and Karen Williams, with UX help from John Mildinhall, managed to deliver a functional application meeting these requirements by the end of the hack day. 

After the hack day, I took this project on as a personal project in my own time, getting it ready to put in front of teachers to gather some real-world feedback. We have chosen to concentrate on the core features, and will be adding the statistics features at a later date. It is available for live demonstration at assessment-anywhere.apphb.com

We would like to be able to put even more development time behind Assessment Anywhere. This will allow us to take it in exciting new directions.  We recently applied for funding from the IC Tomorrow programme. Check out our two minute video pitch below!

Moving homebrew installs to a new Mac

You are a homebrew user and you have a new Mac. If you don’t want to use Apple’s migration tool to copy over everything you might be faced with running brew install [formula] for everything you have installed.

Unless you have a spare afternoon to do this you can achieve the same things with a few commands and be done in minutes, leaving you to get on with something more interesting.

On your old mac you can output a list of installed software from homebrew.

brew list

By piping this into a file you can create a text file containing all of your installed software.

brew list > homebrew.txt

Now you can copy this to your new Mac by Airdrop if you are on the same network, scp or using a cloud service.

Once you have the file on your new Mac and assuming you have homebrew setup you can install all of the software with a single command

cat homebrew.txt | xargs brew install

You might find that homebrew has removed some formulae since you installed it. If this is the case just edit the file and remove the formula that is no longer available.

If you have your dotfiles in git repository you’ve suddenly got a pretty portable setup.

Generating boilerplate sites for experimentation with Github and shell aliases.

I often want to create a small boilerplate site to experiment with as it’s great to try new things out in a sandboxed environment. You can completely focus on the problem at hand and not have to work around existing code. In this post I’m going to talk about how I generate my boilerplate and why I’ve chosen this workflow.

Why don’t you use…

There are a few online services that let you experiment with front end development such as js fiddle and Codepen. They do a great job and allow you to easily share or show off your code, however I still prefer to experiment locally for several reasons…

  • I want to use the tools I’m comfortable with. Working in a dedicated text editor is much more pleasant than writing code in a text box within a browser. I also have several command line and GUI tools I use regularly that I may want to experiment with.
  • If I decided to move the project past the experimentation stage I can do so easily, without having to copy code from an online service.
  • It’s easy to include images and other local media, I don’t need to upload assets to the web in order to experiment with them.

Or you could try…

There are also a couple of command line tools that let you generate boilerplate sites such as Yeoman. Tools such as Yeoman are amazing and can do much more than just generate boilerplate sites, unfortunately as with all software all these extra functions come at a price. Yeoman does far more than I need which makes it much harder for me to use and understand.

My approach

My experimentation workflow starts with a Github repo containing a bare-bones site that fits the way I work. My current experiments repo is a super simple static site that focuses on Sass. You can see my experiments base repo on Github here.

As my experiments base is stored as a simple Git repo, it’s easy for me to update it when I find new methods or tools that I’d like to include in future experiments.

I can then clone this boilerplate site into a directory of my choice with git clone.

$: git clone git@github.com:yourUserName/YourRepoName.git yourExperimentsDirectory

After this, I remove Git so that no changes I make, accidentally leak back to my original repo. That way any changes I make to the original repo don’t affect any of the experiments created from it.

$: rm -rf .git

To make this process quicker I have strung these commands together in a shell function. I’m using the excellent fishfish shell (which I would highly recommend) so my function looks something like this.

function experiment
  git clone git@github.com:yourUserName/YourRepoName.git $argv
  cd $argv
  rm -rf .git/
end

However if you’re using bash, you’ll want something more like this.

function experiment () {
  git clone git@github.com:yourUserName/YourRepoName.git $1
  cd $1
  rm -rf .git/
}

I’ve been using this system for a while now and it’s working really well for me. I especially like that there’s no ‘magic’ involved. I know what’s happening at every step in the process and there’s no framework or new technology to learn.

If you have any idea how this process could be improved or if you have an example of your personal boilerplate that you’d like to share please get in touch in the comments.

IoC containers: where you define the seams of applications

Cross posted from: http://nbevans.wordpress.com/2013/04/10/ioc-containers-where-you-define-the-seams-of-applications/

A few colleagues asked me to do a quick write up about the proper use of a IoC container. Particularly concerning what types you DO and DON’T register into the container.

So here we go: Things that you do and don’t wire up into an IoC container.

The big ones, the seams of the application

Components that are inherently cross-cutting concerns, and need to be “available everywhere” for possible injection. Things like:

  • Logging, tracing and instrumentation
  • Authentication and authorization
  • Configuration
  • Major application services (this includes things like the Controllers in a MVC web app)

Components that will be modularised as plug-ins / add-ins, things that get loaded dynamically. Consider using MEF as the discovery mechanism of these components.

Services with multiple implementations that can be “dynamically selected” through some means (app.config, differing registrations per DEBUG and RELEASE modes at compile-time, per-tenant configuration, etc.)

The little ones, the stylistic ones and where you “lean” on the power of your container to provide infrastructure services or as a development aid

Components that require lifetime scoping or management (transactions, sessions, units of work) and other IDisposable-like things that are longer lived than just a one-off use.

Components that are single instance. Never write “static” components.

Components that require testing / mocking out, etc. Note: I consider this to be a “development aid” and not at all mandatory.

When you want an “automatic factory” (Autofac isn’t called that for no reason!). A simple inline Func<ISomeService> expression is cleaner than a going down the stereotypical Java “Enterprise” route of manually rolling out a SomeServiceFactoryclass each time. Though that’s more as a result of the sad fact that they still don’t have lambdas.

And now the things that you leave out of the container

Anything that is never, and never has any need to be, referenced outside out of the module it is within.

Implementation details of a module. Your container registrations should be the facade that hides the complexities of how that module works.

Things that are essentially just DTOs, entities, POCOs, other dumb types, etc.

Little utility, helper functions.

Note: I refer to “module” a few times. This is in no way in direct reference to an assembly or package. It’s more in reference to a namespace, because components typically reside within a relatively self contained namespace with a container registration module.

Cardinal rules

Never ever call a “static” Kernel.Get / Resolve, or whatever equivalent your container might expose, anywhere. This is not dependency injection. It is service location. Which is a whole different pattern entirely. Autofac is quite neat in that it’s one of very few containers that actually does not, out of the box, provide any sort of “static” resolution/service location function. And that is good.

Only call Get / Resolve methods in your bootstrap code at the root of your object graph. And even then, there should only be less than a handful of such calls. If you can get it down to just one, then you’ve done well and you probably have an object graph that is very well expressed.

Always keep the object graph in the back of your mind. It’s a shame, in my opinion, that containers tend to keep this information hidden away in their internals. The only time you get a glimpse of it is in the exception message for when you’ve inadvertently introduced a circular dependency. Things could be so much better than this.

If you have a component that’s requiring injection of more than about five dependencies, then it should start coming onto your “radar of suspiciousness”. If it reaches about eight to nine dependencies you should almost certainly consider refactoring it and, probably, the wider namespace or module as a whole. I often see this happen on Controllers in MVC applications; the so called “fat controller” problem. Thankfully, because the dependencies are already “well expressed” (it’s just that there is too many of them) then normally refactoring such problem areas of the codebase is a relatively straight forward task.

Nowhere except your bootstrapper and container modules should reference the container, i.e. its namespaces. Arguably, your bootstrapper and container modules can be in a totally separate assembly by themselves and only that assembly holds references to your container’s assemblies. If you’re seeing namespace imports for your container all over your projects then something is very badly wrong.

Avoid the use of “service locator injections”, such as IComponentContext in Autofac. This is one of the very few ways that Autofac supports to allow you to shoot yourself in the foot. It’s not quite as bad as a “static” Kernel.Get style service locator, but it’s still pretty damn bad. As it implies you don’t actually know what possible dependencies your component has, which should be impossible. To avoid this, express your dependencies better. If there are multiple instances you wish to dynamically “select” from at runtime then you can roll your own resolution delegate function and lean on your container to implement it. Autofac makes this very easy using its IIndex relationship. For example: https://gist.github.com/nbevans/e7ab4dbc82880f92407b

Example of a Bootstrapper, Container Module and general structure of your Program Root

This is a little snippet of a relatively well structured IoC server application. I added some relevant comments to it.

https://gist.github.com/nbevans/b350d9addaeec5137b21

I’m open to feedback and discussion :)

Think Sprints

During the pebble {code} hack day Tom, Matt, and Mark created Think Sprints. Think Sprints is a visually engaging two player game based around simple maths questions.

The game automatically matches players into pairs, they then compete by taking turns answering questions to move their characters across the screen. Players can communicate whilst playing via in game chat or by video.

You can play think sprints here (You may need to open the game in two browser tabs if there are no other players online)

How it was made

Once the basic idea of the game was planed out the team briefly split in two. Mark started by designing and animating the characters and Matt & Tom started work on the back-end system.

Designing the characters

Mark started by sketching out some ideas for simple characters. The style of these characters was largely influenced by the excellent dumb ways to die.

After the initial sketches these ideas were taken into Illustrator the characters were created in vectors. Then each character was animated by creating 3 frames each for the running and jumping animations. These frames were scaled down and turned into sprites in Photoshop and animated using keyframe animations in CSS. You can see one of the running animations bellow.

As the back-end was built Mark worked on the layout and created styles for all the game elements.

Building the back-end

The back-end consists of:

  1. An ASP.NET MVC 4 application with a single Razor web page.
  2. jQuery was used for all the client side JavaScript.
  3. SignalR was used for two way communication for the player chat and game communication.
  4. SimpleWebRTC was used to displaying video in the web page.
  5. Microsoft Window Azure was used to host the game.

The basic game logic was worked out and then SignalR was plugged into the application along with some slightly hacked JavaScript (we were rusty with JavaScript and under a time limit)

The core piece of the application is SignalR which is used for the player chat, global chat, broadcasting the game state to both players and accepting answers from each player. A single hub was used for all communication between the server and the player. Whilst a hub group was used so messages could be sent out to a particular pair of players. Grouping connections allowed us to have multiple concurrent games but only send relevant messages to each game.

The video chat using SimpleWebRTC was added at the last minute since it was incredibly simple to get working with just a few lines of JavaScript.

Summing up

Think Sprints was a really fun project to work on, and the whole team was really pleased with the result. The team is hoping to spend some more time on Think Sprints soon to develop it into a even more useful teaching tool.

You can see the git repository here

And play the game here

Hack day: Education Pathways

Another project we did on the education-themed hack day was one that we coined “Education Pathways”.

The premise of this was to devise some user experience that allowed a syllabus of course material to be navigated in a logical order, by taking into account the various prerequisites for each particular subject.

The potential scope of this project was very large. We envisioned it as being some sort of central user interface on a next-generation education service. But in the confines of a 12 hour hack day we limited its scope to merely just getting the basics working.

We considered “the basics” to be a laid-out graph rendering of a syllabus, together with a simple “achievement” system to demonstrate one of the many ideas we had to introduce gamification to the concept.

We hacked this up using these tools: .NET MVC4, Windows Azure Service Bus and Springy.js and of course Twitter Bootstrap.

image

The achievements system was implemented by allowing the user to click one of the subjects and perform a simple test. They’d then submit their answer and if they got it correct they’d be awarded with an achievement. The idea was also extended to support “achievement paths”; so that if for example a user scores 3 answers correctly in a row they’d be given an additional achievement. Obviously this was all rather primitive stuff but the ideas have a much grander scope.

The reaction to Education Pathways in the office was very positive. Our colleagues loved the graph representation of a syllabus and considered this, or some form of this, to almost certainly be the user interface of the future in education.

Service bus, the most useful component you have (possibly) never heard of

Until a few months ago I had never heard of service bus let alone used it in the wild, now I consider it an essential part of the programmers toolkit. I don’t know why I never came across it before now, possibly because I did Physics rather than Comp Sci as a degree, possibly because I was just unlucky. Anyway, that is what prompted me to write my first blog post and hopefully bring its existence to the attention to a few more people. I will be looking specifically at Azure service bus since that’s the one I’m most comfortable with. Although there are lots of implementations of Enterprise Service Bus the underlining concepts should be universal.

So what is Service Bus?

A Service Bus is essentially a message queue, a little like a webservice but without any processing. It simply receives JSON/XML messages from one component and relays them on subscribers.

Why is Service Bus Useful?

  • Its reliable – Service bus (at least the good ones) guarantee the delivery of messages. Azure service bus has At-Least-Once and At-Most-Once delivery. This means that when you send a message it is first stored locally then sent to the service bus. It is guaranteed to get to the queue once and only once even if your network goes down, your laptop runs out of battery or someone unplugs your server.
  • It decouples components – When you implement a service bus both the producer and stop talking to each other and just talk to the bus. Loosely coupled components are easier to change and more robust, how can something you don’t know about going down affect you after all?
  • It makes adding consumers easy – Since the component producing messages is decoupled from the consumers you can add consumers with zero work and zero configuration to the producer. Simply set up a topic broadcasting messages when events happen. Any consumers that cares about those events can then subscribe and get a copy each of that information. If you build a new component that needs that information too then you just need to subscribe them to the topic as well, no extra work for the producer.
  • It allows easy multi platform integration – So you have a chat server in node, a website in .Net and some legacy back end process written in Java. Without service bus this would be a nightmare to tie together and maintain. Implementing service bus will allow these components to nativity consume the information from the bus and play nicely together. Azure Service Bus currently has an SDK in .Net Node, PHP, Java and Python as well as a REST Api. Several other Service Bus implimentations have an even wider array of support (ie http://www.rabbitmq.com/devtools.html)
  • It makes redundancy a piece of cake – Its a fact of life that Servers fail and occasionally need to be taken down for upgrades, maintenance and the like. In coupled applications if one component goes down it becomes very hard for other components to keep working correctly. There are several complex solutions out there to ensure redundancy but one of the simplest and best solutions is Service Bus. You simply set two or more servers to pull messages off the same queue. The service bus itself then ensures the messages are delivered to just one consumer. If one consumer goes down everything continues to work as normal, although messages are processed slower as only one consumer is pulling messages off. When everything is fixed the consumer just comes back on line and everything caries on as normal.
  • It flattens load – Any processing done by downstream consumers is all queued. It means that big spikes in load wont affect anything, the work will simply be queued up and dealt with as normal.

Why is it so essential for Azure?

What cloud services like Azure do really well is scaling out, having lots of small components rather than single large ones. It costs no more to have four small instances than one large one and in most cases four smaller servers can get a lot more done. This is especially true when you are using service bus since you don’t need to worry about load spikes for downstream consumers and you can keep load on each box high. Service bus is a great way of minimising your Azure expenditure and squeezing the most out of your CPU cycles.

How can I find out more?

Cloudcast has a great introductory guide to service bus and the underlying concepts. Its concise and very well written.

As usual MSDN has a wealth of information with some nice code examples.

You can also get the SDK and have a play.