Separation and Integration

Heads Up!

This article is several years old now, and much has happened since then, so please keep that in mind while reading it.

In the beginning, the web was simple, there was one job title for developers who built websites, the web developer. For many, the attraction of the job was the fusion between design and logic, a hybrid where one day you could be working on a database query and the next talking through page layouts with a designer. It was exciting and there was so much to learn. We were the original full-stack developers.

But this way of working didn’t last long, in 2006 web development started to split into backend and frontend developers. This split was inevitable as the job became too big for one person. Some people were better at working with designers to create the UI and some were better at making things work. There was an increasing complexity of HTML with HTML 5 and CSS with CSS2/3 and later responsiveness.

I call this the first stage of separation.

But the separation between frontend and backend developers caused problems in production. Frontend developers created mock-up layouts from the designs and sent the finished markup to the backend developer to integrate.

Integration was an inefficient process; UI code could be complicated and unforgiving, mistakes made during integration needed frontend developers to become involved. Later changes had to go through the same integration process, repeating the pain. A back and forth between frontend and backend developers became common, causing friction and tensions in development teams.

The solution to this problem came from the frontend, they decided to take control. In 2009 the first client-side frameworks appeared. The idea was to take over the rendering of templates and application logic on the client-side, reducing the friction between frontend and backend by relegating backend to supply data via an API.

I call this the second stage of separation.

Over the past ten years these client-side frameworks have developed and stabilised into large architectures. They have introduced new patterns and approaches that solve problems in UI interaction, making application state easier to work with while being more maintainable and less error-prone.  Whole communities of addons and plugins have grown up around these frameworks to complement and extend their functionality.

At the same time, Javascript has developed and improved beyond all expectations to become a powerful and mature programming language. But these advances have come at the cost of increased complexity. The job description for a frontend developer has grown during this time. It’s not unusual to see job specs that expect experience in HTML 5, CSS 3, responsiveness, modern Javascript, several client-side frameworks and Node Js.

Naturally, with all this excitement, client-side frameworks have also started to attract backend developers. Why not? It’s a fast-paced environment with lots of new and exciting technologies. Last year there was an article that used the phrase, the backendification of the frontend that went into more detail.

Throughout this time of rapid development on the client-side, the complexity of UI creation hadn’t stopped or gone away, if anything it has become more complex; flex-box, css-grids etc.

 

What has happened to UI specialists working in frontend?

Unfortunately, when we see a client-side framework used, the importance of the UI is often diminished; hiding it under UI frameworks, simple design patterns, HTML management solutions or just poorly created UI’s. Has the UI become less important? It hasn’t! But UI skills are now seen as less critical to the role, which is terrible for our industry.

Because much of web development has become compressed into the frontend, it feels to me like we have come full circle to pre-2006 where one developer does all of the production.

Are frontend developers now just web developers?

At the start of the year an article was published in CSS Tricks called The Great Divide, it highlights the problems with frontend development that I have talked about here, predicting that the frontend was about to split into Javascript developers and UI developers.

What has triggered this split to happen again?

I think that as the second step of separation took place frontend developers replicated backend processes on the client-side making a further split inevitable. UI definition and delivery had been recombined, removing separation entirely and moving everything onto the frontend. This was only a problem as client-side frameworks become more complex. We shouldn’t be surprised that history has repeated itself.

Unfortunately, a consequence of this new split is that we will need three developers to create a website; an API developer, a Javascript developer and a UI developer! That’s fine if you have the budget to support three different sets of specialists, but how often is that the case?

Also, I fear that the same problems of integration that affected the frontend and backend developers will affect the new UI and Javascript developers.

Are the new UI specialists going to retake control and repeat the whole cycle? Where will it end...

Client-side frameworks are not bad solutions, but they are not the next evolutionary step either. They are just a tool like anything else, useful for solving some problems. Unfortunately, they have been sold as the next step and we are now starting to see some of the longterm effects that these frameworks are causing.

How does this relate to CMS development in digital agencies?

Over the last year I’ve talked to numerous agencies (admittedly, this is no scientific study) that are using Umbraco and I have noticed a surprising trend; I’ve encountered more agencies that have stuck at stage one of separation than progressed to the second stage.

Why is that? I think that the reasons are complicated, but I’ll try and highlight a few of my theories, experiences and observations here.

Content led vs Data led vs Process led

The workload of most agencies that use Umbraco is content-led or a hybrid of content-led and data-led site. The value that customers gain from these sites is in the Content / UX, design fidelity and the content authoring experience.

Client-side frameworks add extra complexity to projects that the agency and the customer struggle to gain any value from. That is until the customer wants to use the content for multiple endpoints. But honestly, how many omnichannel projects are there out there?

How is not as important as what

Content-led sites are often tightly aligned to the brand, so the quality and implementation of the design are essential. It’s better to employ someone that is a UI specialist, they add more value to the project as their primary skill is converting designs into the UI. A full-stack frontend developer could be more expensive and probably wouldn’t be as current with the latest UI skills. No one can do everything at once.

Agencies have been afraid of committing to one framework and becoming locked in. Framework development has moved so quickly that it’s been hard to decide which one to use. Once that decision has been made it becomes hard to change later on. If production code becomes obsolete either because of breaking changes or a better framework being released then the agency is left with a codebase that is challenging to maintain and difficult to recruit for (the Umbraco admin area!?!?). Many agencies have wisely decided to keep it simple (and adaptable).

Technological locked in can ruin a good project through technical debt

Most content-led projects need little interaction. Much of the web is static content that only requires a sprinkling of Javascript. Sometimes adding a full client-side framework just adds too much weight to a project and their use of the shadow DOM makes it impossible to use in conjunction with anything else. It’s all or nothing.

Client-side frameworks have some fundamental problems when they are used to create content-led sites; SEO, client-side weight, time to first paint etc. Developers working with these frameworks have fixed these problems by creating aftermarket plugins creating complex chains of interdependencies to resolve issues that would have been trivial to solve using a more straightforward solution.

Often simpler is better

But… because these frameworks are the apparent new frontier, they have a lot of support from the community and the developer experience has advanced quicker than anywhere else. As a consequence complimentary plugins that add real value have been created such as StoryBook or Static Site Generation tools like Gatsby.

As the developer experiences and tooling around these frameworks become more advanced it is becoming harder for agencies to ignore them without any other alternative. Forcing agencies into making a decision that they have been avoiding, even though it will mean they give up control; of their interaction layer, their design fidelity and ultimately their costs.

What is the alternative?

I believe there needs to be a change of direction, looking for different production methods that try to resolve the issues caused by separation.

Let's accept that there is a difference between developers that want to program and those that want to build UI’s. Being able to do both is fine, but just wanting to work on the UI shouldn’t be seen as a bad thing either. I don’t think the labels frontend and backend have ever been correct, even more so today.

We prefer to use the words Definition and Delivery.

Definition developers concentrate on building and understanding the UI and its behaviour under different state changes. They work closely with designers to maximise the fidelity of the UI.

Delivery developers use the output from definition to deliver the UI using whatever delivery mechanism is required. They manipulate the data into a form that the UI is expecting and render templates in context at runtime.

The key to solving integration is to allow definition complete control of the UI and to fully separate the delivery process. One solution we have found is to ask definition to create an interface to describe the dynamicity and interdependency of each component. Then delivery only need to understand the interface to use definition components, they need not see any of the details or work with any markup.

Integration will become a trivial one-way release process from definition to delivery.

There’s no need for a third specialist developer,

The definition development environment can be separated and tailored to the needs of the definition developer, elevating markup and style to primary importance. Taking back control, improving design fidelity and showing how state impacts the UI under different scenarios.

By not being locked into a client-side framework the Javascript layer is freed. Developers can use one or more progressive framework, only when they are required, to implement interaction.

Instead of all or nothing, the solution becomes an artist palette where all the different elements can be swapped out with ease without affecting anything else, based on a core composed of simple ideas and technologies that are easy to change.

A fully separated solution can be understood as many different levels where any element can be replaced without affecting anything else.

For me, this is an alternative to using a client-side framework. Only by separating our environments and solving the problems of integration can we create sites that have a higher level of control. Websites that reach our customers’ expectations and will be maintainable for years to come.

And this is only the beginning, by better understanding separation we are going to discover new and unexplored paths. This rabbit is going to go much, much deeper.

We have a new framework in beta called Yuzu that implements the ideas above while also mimicking many of the features of a client-side framework. We're currently in beta and writing documentation, there are more details at balanced.dev and github.com/balanced-dev

 

Phillip Whittaker

Phillip is on Twitter as