My blog for the DevOps Basics course === ## Introduction Hi! My name is **Lee C. Degeilh**. I enrolled H2C program at Kajaani AMK in 2020 autumn. Here's [a link to My Gitlab projects](https://gitlab.dclabra.fi/Lee-Lilly/). My programming experiences extend to web & mobile application development and data analysis. Please see [some old Github Repositories](https://github.com/Lee-Lilly). ## Module 1 - Development tools for teams and individuals This module is meant for setting up a project in https://gitlab.dclabra.fi, as well as start project blog on https://gitlab.dclabra.fi/wiki/ ### GitLab 1. Install git (normally github users already installed it) 2. Register to https://gitlab.dclabra.fi (with KAMK.fi credential) 3. Log in https://gitlab.dclabra.fi, create a new project name "devops-basics-course" (private). open gitbash or windows command line 4. Generate a SSH ED25519 key pair. By default is saved in local file ~/user/.ssh/id_ed25519 > _ssh-keygen -t ed25519_ 5. For new user, a SSH is required, copy the public SSH key to the required form of authentication. 6. Make a local directory named "DevOps" > _mkdir ~/DevOps_ 7. Configure the git settings > git config -global "firstName, lastName" 8. Go to local directory, clone the git repos to local > cd DevOps/ > git clone git@gitlab.dclabra.fi:Lee-Lilly/devops-basics-course.git 9. Go the cloned local repos , Created a new empty file (similar to "readME.md") > cd devops-basics-course/ > touch LeeDegeilh.md 10. Open the project in VS code (suppose it is installed), edit the file in VS code. > code . 11. Add file to git local > git add . 12. Commit the change (with a msg) and **push** to the remote git repos > git commit -am "added LeeDegeilh.md" > git push ### GitLab Wiki 1. Register to https://gitlab.dclabra.fi/wiki/ 2. Start a new note, title as project blog 3. Set statu "Locked" (owner can edit) 4. Publish ### WakaTime for VS code 1. Register to WakaTime 2. Open VS code 3. On project explore menu, click on "extenstion" 4. Search for "WakaTime", click on "install" 5. Get API key of own WakaTime account from WakaTime settings 6. Type "F1", search for "WakaTime API key", paste own API key to it. ## Module 2 - Introduction to Agile Distributed Software Development This module introduces **Scrum**, an Agile development framework, and tells how it differs from traditional plan-driven waterfall development. Scrum is an incremental and iterative process of product development, starting with **M**inimum **V**iable **F**eature (**MVF**), and regularly (each **Sprint**) releasing new features according to customer's requirement. The path is full of **inspection and adaptation**. A **Scrum Team** is composed of: 1. **P**roduction **O**wner(**PO**) A person who is in charge of Product Backlog (**PBL**), to prioritize the list of 'todo' features. 2. **Dev**elopment Team (**Dev**): A team who works on product development and releases required features during each Sprint. 3. **S**crum **M**aster (**SM**): A person support the process of Scrum, to help communication between Scrum member. We organise the work in Scrum **based on Sprint** - a **time box** (usually **2 - 4 weeks**), and each Sprint has four iterative phases: 1. **Sprint Planning** : _decide **a list of todos** from **PBL** for this Sprint, and **a road map** to incremently complish the tasks. This leads to a **Sprint Backlog** (**SBL**)._ 2. **Daily Sprint**: _**inspect** the work on previous day, e.g. **progress and obstacles**, then **adapt and plan** for current day._ 3. **Sprint Review**: _**PO** and **Dev** Team shows the **achieved** PBL items, as well as **yet achieved** ones, also discuss the **work progress** and **difficulties**. Get **feedback** from customers, and eventually **a new version** of PBL items. 4. **Sprint Retrospective**: _discuss the personnel, communication, tools, process of the ending Sprint, search ways of improvement._ Several important terminology: 1. **Sprint Backlog**: items from PBL as **tasks** for current Sprint, also the **implementation plans**. 2. **Product Backlog**: requirement, **user stories**, desired features etc. The list is **prioritized** by **importance** of features, with detailed **descriptions**. 3. **Scrum board** (Issue board): a board contains tasks in columns like "**Doing**" "**Reviewing**" "**Done**" to indicate the **work progress**. 4. **Iteration**: the same type of **events and procedure** happens **regularlly**. 5. **Increment**: the product **features are complished** within **Sprint** and are **built upon** **Sprint par Sprint**. **Requirement** and goal of project are **written in PBL**, the **PO** is in charge of negatiation and collecting the PBL items, therefore, he/she must **communicate** with Customers and Dev team. They must agree on "**Done**" notion, which is explained in PBL. Moreover, PBL items are in **continuous reviewing** and **increment** Sprint par Sprint. Through **Sprint Planning**, Scrum Team gets clear idea about the requirements, **selects** appropriate **items** for current Sprint, eventually sums up **a road map of increment**, which forms **Sprint Backlog**. ## Module 3 - Docker for DevOps **What is Docker?** To me, it would be equivalent to a **virtual machine**, but not the same mechanism. The differences made the revolution. - Various virtual machines are acquired from the central server by installing the required Operating System and targeted database. - With Docker, it only **pulls images from Docker Hub**. The image can be a specific version of ubuntu, Python, Node, or Postgres, etc. - Once the image is pulled to the host machine, the **docker engine (docker daemon)** will **create a container** of the image with proper volume. - The container has **a virtual copy** of the process table, network interface(s), and the file system mount point(s), which is inherited from the host machine. - The **kernel of the host’s operating system** is **shared across all the containers** that are running on it. - This allows **multiple containers** with different application requirements and dependencies to run on the same host. **No more need to switch between different virtual machines.** **How to Docker?** - Installation (Windows, Linux, Mac) - Testing > docker run hello-world - In your application folder, create a new file named exactly "Dockerfile". - Edit "Dockerfile", an example of NodeJS - # pull image from DockerHub - FROM node/apline - - # the directory in container for app files - WORKDIR /app - #copy all app files to contatiner ./app - COPY . . - # install required node package to container - RUN npm install - # build scrpit - CMD ["npm", "start"] - Build and run the Dockerfile with terminal (Docker Client is installed on host machine) 1. _build the app_ > docker build . --tag _appName:latest_ 2. _verify running containers_ > docker container ls 3. _run app with mapped port 3000 on hostmachine_ > docker run -p 3000:3000 _appName:latest_ 4. _log into container with shell terminal_ > docker run -it _app:latest_ sh **My Docker applications on Gitlab** **App 1. Test project** (https://gitlab.dclabra.fi/Lee-Lilly/devops_testproject) - _This application contains only Dockerfile and .gitlab-ci.yml._ - _Dockerfile_ pulls images of ubuntu and Python, and RUN python on the fly to printout text. - _.gitlab-ci.yml_ configures the gitlab CI/CD pipelines. It defines images, services, stages (buid, test, deploy). **APP 2. AgarioClone** (https://gitlab.dclabra.fi/Lee-Lilly/devops_agarioclone) - _This application is cloned from_ (https://gitlab.dclabra.fi/jaakkovan/agario-klooni) - Create and edite _Dockerfile_, _docker-compose.yml_. - _docker-compose.yml_ specify the services of application and dependent database, as well as the networks between them. - Once _docker-compose.yml_ is built, the **app container** and **database container** are created and **network** between them is created also. - _gitlab-ci.yml_ for testing login page and connection to registering/log platform. **APP 3. CustomerDatabse** (https://gitlab.dclabra.fi/Lee-Lilly/devops_customerpostgres) - This application was a **self-learning NodeJS project**, which intend to create a **REST API** connected with the **Postgres** database. - Create and edit _Dockerfile_ and _docker-compose.yml_. - Connect to database container by logging into the container with Postgres command line _psql_, which enables us to create database and table, then insert data. > docker container exec -it customer-database psql -U postgres > Postgres# - A test is created with mocha framework and chai library. I am not sure how to integrate the service of app and database with data into the gitlab-ci.yml for unit test. ## Module 4 - Agile Project Management User stories of Agario: 1. As a **player**, I want to **register my username**, because I want to have a unique identity/nickname in the game. 2. As a **player**, I want to **join the game**, with my username and password. 3. As a **player**, I want to **play the game**, able to **move bubbles**, or **eat bubbles**. 4. As a **player**, I want to be able to **quit the game**  ## Module 5 - Continuous Integration / Continuous Deployment (CI/CD) in Gitlab **Final Assignment - Part I: Create app and dev automation.** - Story 1: read temperature and humidity  - Implement unit test for webserver (with mocha framework)  - Automate build, test and deployment phases   **** **Final Assignment - Part II: Robot framework for testing** There are **two ways** of using Robotframework: **Way 1. Install and test locally** - Python _(already installed)_ - Robot framework and selenium Library _(pip install ...)_ - Chrome web driver _(copy driver .exe to path Python\Script)_ - Write a test script with Robotframework specifications - Run the script robot -d testRobot/results testRobot/chromeTest.robot testRobot/results/log.html  **Way 2. Define a Dockerfile to pull images from Python, robot framework, etc.** - create a Dockerfile in folder ./testRobot FROM python:3 WORKDIR /app COPY . . RUN python3 -m pip install robotframework RUN python3 -m pip install robotframework-seleniumlibrary RUN python3 -m pip install webdrivermanager RUN webdrivermanager chrome --linkpath /usr/local/bin # in case not run docker with bash CMD ["robot", "-d", "results", "chromeTest.robot"] - build and run the test in container > cd testRobot > docker build . -t weather/test-docker > docker run -it weather/test-docker bash # check the app directory in container > root@21ebd5234d69:/app# ls # run the test with robotframework, save the rpt in folder "results" > root@21ebd5234d69:/app# robot results chromeTest.robot