Often we hear that someone has some new app on their smartphone that solves their daily problems or makes something easier. An app for ordering a cab, ordering food, buying movie tickets, navigation or whatever. But in fact mobile applications (that is, the ones installed on our smartphones) are quite often programs to remotely operate a web application that runs somewhere on the Internet. In this approach a mobile application is created mainly for the user’s convenience, so that the user can use the web application in a nice and easy way. Without a web application, a mobile application makes no sense. We have access to the Internet practically everywhere, so we don’t even always think about which of our mobile applications talks with the web application and which of them works independently. And it’s the web apps that actually do the most work and troubleshooting, then send the mobile ones the results, which can then be displayed nicely. Let’s talk about what web apps are and how they are developed.

Let’s start with the basics. A web application is a collection of server applications that work together for some purpose. A web application can consist of such server applications as a database, a business logic application, a payment application, and let’s say an application that allows you to display an interactive web page. Since a web application consists of several server applications, we can call these server applications components of the web application. So we have several components that run on one server or several servers. Some of them we can develop ourselves, and some of them we can simply run using ready-made solutions. The developers’ task is to provide appropriate ways of communication between these components, and the task of cloud administrators and engineers is to properly prepare the infrastructure and supervise it.

As a rule, applications are written. There are many programming languages and they change over time like almost everything in this industry. However, we can distinguish two groups of languages at this stage.

The first group are interpreted languages. When programmer writes code of such application, to run it he needs so called interpreter. Interpreter is such a program, which reads the source code in text form line by line and executes programmed instructions on this basis. So we can have the whole application source code, but if we don’t have an interpreter, it won’t give us anything, because we won’t be able to run it. In addition to the interpreter and the source code of the application written in such a language, we must also have libraries, which our code uses. The library is simply a set of functions and parameters, which is used by the application. If writing every program in the world involved having to redefine each time the standards for math, writing to files, reading, calendar, and hundreds of other things that our program does, the application development process would take many years. For this reason, libraries are created and they encapsulate a set of functionality that other programmers can later use in their projects. Libraries can therefore be developed independently of the application and used in many different, unrelated applications.

The second group of languages are compiled languages. In this case, a compiler is needed instead of an interpreter. A compiler is also a program that reads the source code of an application, but turns it into what is called machine code. This very process is called “compiling” the code. That is, in other words, from the source code we create a file that we can already run. After compilation, a compiler is no longer needed to run such a program. However, also in this case usually libraries are needed. Libraries of such languages are also compiled, but we can no longer run them directly, but through the program we write, we can use their functionality.

Now that we know that we have two types of programming languages for developing web applications, we can consider how these applications are developed. Let’s assume for the purpose of this episode that I am developing a very simple web application, consisting of so called “frontend” which is a component displaying graphical interface and receiving requests from my clients, “backend” which is a component with programmed entire business logic of my web application and the database which stores data about my users and all other data needed for the operation of the entire application. Web applications are of course much more complex, but let’s focus on a simple application consisting of these three components. Two of them clearly need to be developed by me, frontend and backend. It is these components that make up the core of my web application. The frontend displays the entire GUI, assembles client requests into coherent structures, and sends them to the backend. The backend receives the request, constructs a query to the database, sends it and then performs actions with the received results. If necessary, it sends several queries to the database. Once it has received all the responses, it re-packages a coherent response and sends it to the frontend. The frontend, understanding the responses received from the backend, displays the results. Let’s illustrate it with an example. Let this web application I am developing be used to book and rent ski equipment. We go to the website where my application can be found. A nice and colorful user interface appears to our eyes. This is the part of the frontend that shows us text or other fields that we can fill in specifying what we would like to rent. When we enter in the appropriate fields that we need skis with specific dimensions and a helmet in size S and green color, the frontend will collect all this information and send it to the backend. The backend knows how to deal with such a request. Based on the given preferences, it creates an appropriate query to the database to find out if such equipment is available in stock. The database answers that the skis will be available, but the helmet in size S and green color was not found. The backend, seeing that there is no helmet of this color, can send another query to the database, but without the color criterion. When it receives some results, it will reply to the frontend with a list of available skis (because we were mainly looking for skis) and the information that no green helmet was found, but helmets in other colors were found. It will also provide a list of helmets found. Frontend receiving this response knows how to gracefully apologize to the customer for the inconvenience and subtly suggest other helmets.

In this example I’m responsible for frontend and backend. I can run any database I want. Nowadays there are various databases available, so there is not much sense in creating them by yourself. So how do you develop the other two components?

Let’s assume that the frontend is developed by a team of Python programmers, one of the most popular interpreted languages. Let the backend be developed by a team of Go language developers. Both components are developed separately. Of course, it is not possible to make these teams fully independent of each other, but it can be done to some extent. Let’s say that a bug was found somewhere in the backend and one of the programmers just fixed it. So he sends his code changes to the code version control system. He declares that the changes he made are ready to be accepted and made permanent in the backend. Someone from the team looks at his changes and makes some comments if necessary. This time, however, there are no comments because the change is small and the bug, if not fixed quickly, could leak customers’ personal information. Our hero’s teammate therefore accepts the code changes and makes them permanent in the code of this component. At this step, all human interaction ends. All further fate of the code is already automated.

As soon as the source code version control system notices the change described above, it starts what in our nomenclature is called a “pipeline” which could literally translate to the word “pipeline”. Unfortunately, the Polish equivalents are not always accepted in the industry terminology. The pipeline in question is a process that is executed when the appropriate circumstances arise. Changing the code can be one of those circumstances. During this process, many things happen. Among others, the component is automatically compiled with the new code and tested. Then, the whole web application is placed in a specially prepared test environment to perform integration tests between individual components. Next, system tests are performed to perform some actions affecting the whole application in search of errors. Then more tests are performed and then even more tests. Generally speaking, the more tests are performed, the greater the chance that bugs will be discovered at an earlier stage, and not when the application is already available for customers. The order and scope of tests depends, of course, on a particular web application and on decisions of test engineers.

During this process, something happens beyond testing that today constitutes the core of modern web application delivery. It is containerization. Containerization is the process of encapsulating an application component in a specially crafted object called a “container”. The container imitates an operating system and allows the component to run in it. Of course, you can ask the question why to do it at all, when we can run a component of a web application without additional complications. Contrary to appearances, there are many advantages. First of all, with the help of proper container management you can get a significant resistance to failures. The specificity of a container is such, that if the program running in it turns off, the container management system will notice this event and immediately manages its restart. It will repeat this process until it happens. So, if the component works relatively well and rarely crashes, it should not significantly harm the stability of the entire web application. The second advantage of using containers is the ease of migrating applications. If every component of the application works as a container, moving it is very easy. At times, large companies with very large applications running in the cloud have decided to switch cloud providers. Moving containers was fairly straightforward in this case, which could not always be said for applications that were not containerized. A third advantage is potential scalability. A container is, in principle, a small piece of a web application that should be able to run both alone and as one of hundreds of the same containers. For a container management system, running one or hundreds of containers is just a matter of setting one number in the configuration. It is also possible to dynamically scale the number of containers depending on the application load. The fourth advantage is keeping the servers in order. In the old days, updating applications used to be quite breakneck. More so if programmers updated some libraries in their code or updated interpreter. Generally, for many reasons, any changes on the server were often a challenge, because you had to deal with a whole bunch of dependencies. Containers solve this problem already in advance. It is during the containerization process that both the application component, its libraries and all the dependencies, including the interpreter if needed, are placed inside the container. In other words, a properly built container should have absolutely everything needed to run the component, but absolutely nothing else. So updating an application in a container management system involves only one or more container version entries in the configuration, instead of hours of admin work. The system will simply note that it’s supposed to run a newer container from now on, and swap out the older containers for the newer ones.

Let’s go back to our “pipeline”, much more commonly referred to as “spidering”. During this process, once the changes made by the hero of our story have been accepted, a lot of testing and containerization takes place. If everything goes according to plan, the new version of the backend will become available to customers as soon as the process comes to an end. It is up to the creators of such automation whether to make the new version available to customers immediately or after they have explicitly agreed to the change.

Two things are worth noting here. The development team can focus almost entirely on code development. Programmers’ salaries are as high as they go and for this reason a great way to maximize the use of specialists’ time is to simply remove them from the problems related to service delivery. Programmers only deliver code and each of their individual tasks ends when the tests performed by the pipeline are successful. And since they don’t have to do the testing manually, they will be able to start another task, since at that time the automated testing will be happening somewhere on some server far away. The second thing worth mentioning is that all this automation seems to be doing activities that are inherently boring. I guess any employee, especially the more ambitious ones, if they had to copy code several times a day, compile it, run a dozen tests manually, containerize it, and so on, would pretty quickly say goodbye to such a job. Automation is supposed to take over the boring tasks from people so that they can just take care of the more creative activities.

Nowadays, when developing any code, we aim to fully automate virtually all processes. This is guided by a simple principle: “if you’ve had to do something more than once, it means it’s worth considering automation for that process. The cloud and various platforms have made it possible to automate a great many processes. From such simple tasks as notifying about failures, to setting up and modifying huge infrastructures in the cloud. Many companies have seen a significant acceleration in the development of their applications, thanks to automation. No doubt this trend will continue. The question is what awaits us in the near future, because technologies are already developing so quickly that it is difficult to predict even the next few years.