A high-level overview of how an application works: code to the user and everything in between!
Here is how I understood all the nitty gritty things that happen from coding an application to a user using it.
Step 1: Code
Use any of your favorite programming languages or an efficient programming language and get coding. Let’s say you make a website.
You are likely to do this on your laptop, which will then be called your development setup.
Step 1.5: Build
So a code cannot be executed as such, it has to be in an executable format. So converting a code into an executable format is called building. This can be done with tools such as Gradle, Maven, etc. A build script is used by these tools to build the final executable.
Step 2: Host
When you want a lot of users to access it, and the website must be up all the time, hosting it from your development system might not
be the most efficient choice. So you would likely need a server (physical, VM server, or cloud server) to host your website.
Now just copying your code onto this server will do. You’ll have to configure it to match your development setup. Includes runtimes, packages, etc. This would be your deployment/ production setup.
The executable produced in the build stage is run in this stage. Also called as ‘deployed’.
This is the basic setup. Now you can make it work with this, but there come additional factors like collaboration work, upscaling, and testing. For these, we have additional tools to aid us.
Step 1.2: Collaborate
When multiple users come together to make a piece of code, they work with the same code base. Might lead to conflicts. Hence a tool that helps resolve these is — GIT. Git is the underlying tool. The central code base is saved in a cloud-based repository called GitHub ( used for versioning of code, collaboration, etc.)
Step 1.6
Individually when you code, the code is in one single system and you can build it on the system itself. Now with multiple people collaborating, you need to build with everyone’s changes, so now we move the build operation to a dedicated server, let’s call it the build server.
Step 1.7
So all changes might not be compatible with each other. And we cannot deploy a faulty application. Therefore, each build is then further sent to the TESTING server.
Here the builds are tested. If it matches with the expectations they are then pushed onto for deployment, else pushed back for further development.
Now all of the above-mentioned are done manually, and they are very time-consuming. Hence came into existence CI/CD tools. Continuous Integration and Continuous Deployment Pipeline.
Step 0 (all along the way): CI/CD pipeline
These various stages are automatically put into place with the help of CI CD pipeline automation tools like JENKINS, GITHUB ACTIONS, and GITLAB CI/CD.
Here, the code from various users pushed to the central repository is automatically pushed to the build server, built, pushed to the testing server, tested, and then pushed to the production, when it is production-ready!
(More reasons you ask? faster and way less manual and more automatic)
Now imagine you are coming up with a change in the code that requires a change in the required dependencies. Now one has to manually install this in the development server, build server, test server, and deployment server. This is again tedious.
Step 1.4: Containerize
Containerization refers to adding a pseudo layer onto the application and packaging it with its required environment, dependencies, and other configurations necessary. This can be directly deployed onto any server without the worry of configuration errors.
So now is STEP 1.5 when we build, we build an image with the application and its respective dependencies.
The tool that lets us create images of applications is DOCKER. We create a docker file (similar to the build file that is used to build) which is used to build the image, which then could be easily run on any server/environment using a simple docker run command.
- The container is pro-isolation. Hence we can have multiple instances of containers running simultaneously. Each of these processes does not interfere with the other.
Moving on to the production side.
Step 3: Scale Up/ down (container orchestration)
When users increase, in order to take the incoming traffic, we need to increase the number of container instances. And when there is less traffic we need to shut a few off to be efficient. These can be on different servers or the same server. We need a container orchestration for this. KUBERNETES
- makes sure that the required number of containers is always up (that is if some container fails, automatically turns it back up)
- Manage the resources (the underlying servers effectively)
Step 4: Provisioning of servers
Imagine you bring in a new server or you want to repurpose an old server. The application deployment would not need to configure the server since it comes packaged as a container. But the server needs to have some configurations done pre. Like installing container run time, Kubernetes packages, etc.
This can be done by using an automation tool like TERRAFORM. The required state is mentioned in a terraform manifest file and the infrastructures are always present in this required state. It has the details of all servers and their required state.
- Automates provisioning and configuration of infrastructure.
- ensures they stay in the same state.
- Iac: Infrastructure as Code.
Step 4.5: Automation of configuration on infrastructure
After provisioning is done, the automation of configuration is done by tools like ANSIBLE. After tools like TERRAFORM provision the infrastructure, tools like ANSIBLE automate the configuration task. Ansible cookbooks are used to make this happen.
- Post-configuration automation
Step 5: Monitoring and Maintenance
PROMETHEUS collects all necessary info like CPU consumption etc. And tools like GRAPHANA view this data graphically as charts and graphs for ease of understanding.
THE DEVOPS INFINITY
Code — build — test — release — deploy — operate — Monitor — feedback and loop!
Hope you have a good understanding, lemme know in the comments if it was helpful!
