For those unfamiliar, Unity is one of a handful of widely used game engines, authoring platforms designed for building games, however, especially with the advent of AR/VR they are gaining more general usage creating all kinds of experiences, normally revolving around 3D graphics and multimedia. From a web point of view I'd say the closest analogy is something like Adobe Flash (God rest it's soul) but that would be doing it a huge disservice. It's an expansive environment which has rich features for designers like Cinemachine for cinematic camera effects, animation tools, audio processing and video processing as well as a good set of APIs that do a great job of smoothing over all the tricky concepts and mathematics around 3D and physics (among many other things).

However, the most interesting aspect for me has been the development workflow that it enables. The process of building a project in Unity encourages you to work on application logic while, at the same time, work on code that extends Unity itself in order to make it a better tool for building that specific project. You essentially work on the problem from both ends: You build the tools you need to make the project at the same time and in the same environment that your are defining the logic of the app in and the programming experience of doing that is seamless.

One of the first things you encounter when starting with Unity are Serializable properties:

public class Grabbable : MonoBehaviour {
    [SerializeField]
    [Range(1, 10)]
    int weight = 1;
    
    // ... rest of the code here
}

What this actually does, as well as creating an instance variable, is expose the field within the inspector and provide some meta information about the allowable values etc. Because you've specified a range and it knows the type in question it provides an appropriate UI element.

There's a lot of extra behavior that goes with this. This value can be adjusted (and serialized per instance) during authoring and also live when the project is being executed. It's clear that's a powerful (and perhaps essential) tool for game developers where you're not working to specifications or business rules, you need to get the feel of the game right and to enable that you need to have your hands on the controls so you can tweak and adjust the experience easily. There's a whole slew of ways that you can extend and specialize Unity itself. You can add author time visual guides to visualize things like line of sight of a character or range that you need to be within to pick up and object, easy ways of writing code that runs either at author time, runtime or both to implement things like snapping object placement to a grid, for instance, and that's just the start. I've really enjoyed the workflow and just how seamless it is with to jump between application logic and extending Unity. You don't need to context switch from app development mode into writing an extension or plugin.

I've just touched on this huge subject and definitely not done it justice but I couldn't help but wonder how this way of working would translate into server-side application development and deployment. When running any none-trivial service its absolutely critical that you've got dynamic configuration (feature switching), real time monitoring and a way of running experiments but traditionally they are implemented as libraries. Because they require extra work to implement in an app they are typically retrofitted when a company matures to the point they can't go on any longer without these facilities. What if we built in an environment where dynamic configuration of the likes of [SerializeField] was baked in and altering any configuration at runtime, for instance, to rate limit an endpoint or disable some expensive feature under heavy traffic, was always available because it was part of the workflow of building any application?

The areas that I'd like to cover initially are roughly 3D graphics programming and the underlying principles and get familiar with game engines while, at the same time, I'm going to put a machine learning/computer vision curriculum together which I can interleave with the graphics work I'm doing to keep my brain fresh. I'm going to be careful to stick to the foundational stuff first rather than to head to quickly into VR/AR - I'll easily get trapped tinkering endlessly otherwise - time for that later.

To kick start my learning, I signed up to a whole load of Udemy courses. Udemy constantly have some kind of sale on but for the first week of the year they had a particularly good one where every course on the site was £10 so I just went in and bought the highest rated courses for every subject I could think of. At this point, over the last few months I've managed to get through a couple of those courses and I've been pretty impressed so far.

A particular highlight has been this Unity course. I'm in a bit of a weird spot whereby although I'm a pretty experienced programmer I know nothing about programming patterns around gaming/graphics, basic 3D concepts (apart from bits I've dredged from memories of my CS degree) or Unity as a piece of software. This has certainly been a bit frustrating but this course is one I'd highly recommend as it's very project based and has allowed me to learn the basics while diving a bit deeper into the programming side of things myself. There have been a good few points where I've grimaced at some of the programming decisions, decided to do it my own way and then discovered a bit further down the line why they'd done something a certain way which is a bit of a PITA but its not detracted too much. I must admit, working with Unity has been really exciting so far, it's quite an incredible piece of kit. Post forthcoming with my initial thoughts on Unity.

Aside from this I've been making my way through this short course on shader programming. This is pitched at a higher level from a programming point of view and I'm finding this as a great conduit through which I can properly apply and grasp some of the lower level mathematical concepts around graphics programming and 3D. Previously to this I'd made my way through this great set of videos that explain linear algebra in a brilliantly visual way - an amazing resource - there are also great videos on that channel explaining the maths behind things like blockchain and neural nets that are worth a watch too. It's already clear to me that getting good at all at both graphics and ML is going to involve really understanding linear algebra.

Finally, I've been breaking up work on the courses with working on this site (getting back into web dev briefly was fun - will post more about that in the future) and watching conference talks on various related subjects. I've found the library of Unite talks to be a great source as well as GDC's presentation library. A couple of highlights so far are this one on the design challenges of building the Rick and Morty VR experience and this one on Unity architecture using ScriptableObjects. Since the start of this I've been really curious as to how games/graphics programmers apply various patterns and this was an interesting glimpse.

If anyone out there has any tips, I'm all ears. Ping me on twitter with suggestions.

Fast forward to the late 90s when I first started to understand the possibilities of the web. The first encounter was actually at university where I was sitting around on some MUD in a computer lab and one of the graduate students came up to me and asked me if I knew how to make web pages. I was like "no, it looks a bit shit" because I thought I was going to be a famous MC at the time but they showed me anyway. I put together a few tags and put up a page that said something like "WOO! YEAH! INTERNET!" then promptly ignored it for at least a year. At the time browsers were very primitive, every page was under construction, e-commerce did not exist, dial up was slow and stopped my mum from getting phone calls to her land line. It was basically a pretty absurd proposition. However, once I started to understand it more, I became absolutely fascinated with it as a technology and a medium. In particular, I became fascinated by how we could capture it's interconnectedness, its universality and work around the unique technical constraints. The community around the web has been incredible, we all viewed each other's source and started to share our libraries and frameworks as well as knowledge and experiences at conferences. We all figured it out together.

As giant businesses and a huge industry built up around it so did my career. From working on Sainsbury's and Argos first online shopping sites, to making use of first generation EC2 to cope with the crazy peaks of traffic when we first launched the Supreme online store and then onto building things a giant scale at Twitter, I've thoroughly enjoyed making every single thing along the way even though it all disappears or gets superseded at a frightening pace. More recently, I moved into management, first at Twitter and then, to what's probably been the biggest and by far most rewarding challenge, building Deliveroo's engineering team but that's a whole other blog post.

On a bit of a whim, I went on a trip to GDC in 2016 with my brother who's part of the games industry. At the time, I was liking the idea of hanging with my brother, seeing some old friends and perhaps learning something interesting in some of the talks. However, GDC 2016 was the time that the first consumer grade VR headsets were about to be released so there was a huge buzz and accompanying giant lines for everything virtual reality. There were a lot of demos around the conference centre but they were pretty tricky to get a go on. As a result, we were just eager to try anything at all. We came across a sober looking conference booth with a Vive headset, a desk fan placed on a chair and a length of rope taped diagonally across the floor of the booth. There were only about five people in line so we dived in.

When it was my turn, I stood with my feet on the rope and put the headset and headphones on. All of a sudden I was standing on the edge of one of the twin towers about to walk the high wire across it. The combination of the wind audio in the headphones along with the now well known sense of scale that VR gives you combined with the feeling of that little desk fan to immerse me in a way I've never experienced before. It was The Walk VR experience which I don't think ever made it out to a public release on Vive but there are other similar experiences. After that I got to try Bullet Train, which let me pick bullets out of the sky and generally feel like a super hero [insert standard mind blown gif here]. The effects are physiological, not just audiovisual and intellectual - its a power I'd never thought I could wield as a programmer. However, the state of VR/AR so much reminds me of where the web was at when it first gave me that geek out moment. Heavy headsets, low resolutions, wires everywhere, motion sickness, awkward UIs - essentially, it's all pretty absurd but progress is blistering and the possibilities are incredible. This was my latest geek out moment, and just like when I discovered the possibilities of the web, my interest in it has been growing exponentially since this point.

So, it's been a tough decision for me but also, I think, an inevitability. I've decided to leave my role leading engineering at Deliveroo to pursue this full time. I've had a head start on it over some late nights over the past few weeks but the first item on the list is to learn everything I can about VR, AR and computer vision and start building things. I've quickly discovered that 3D graphics programming is a really different set of challenges and embodies very distinct ways of working from industrialised consumer internet practices and, while I'm hoping that some of my experience is transferable, I'm absolutely giddy at the prospect of learning such a lot of new stuff. After that, I really don't know yet. I'll see where it takes me.

I'm resurrecting this site, massiverobot.co.uk, which used to be the site of my old consulting company, as a place to journal my progress from becoming a web programmer to working with VR, AR and computer vision. To be honest, I'm doing it for my own benefit to keep myself on track and record useful things as I find them. But, many have moved from web to 3D graphics programming and a strongly suspect many more will so hopefully others following the same path my find it useful too. I'll be wrapping things up at Deliveroo until mid-April but will be posting regularly from that point onward.