Views of DevOps
- DevOps during Dev (Development)
- DevOps during Ops (Operations), including deployment
- Example: Testing
- Require testing during deployment
- But it also requires testing during development
- Combined view: Extend development approach into deployment
Code Integration
- Once common practice to infrequently commit to the central repository
- The central repository became quite different than developers repository (or branch), so developer's dealt with complicated code integration
- merge hell or integration hell
- Integrating changes often took more time than developing a solution
- If code integration is painful, then developers wait, making it more painful
Continuous Integration (CI)
- Local Workflow (multiple times per day):
- Develop, test locally
- Commit
- Separate build server, building on each commit:
- Compile/build
- Run tests
- Deploy an artifact
Common CI Practices
- Version-control repository
- Automate the build to a single command, preferably without options
- Build runs tests
- Everyone commits to the baseline frequently, i.e., at least once per day
- Every commit to the baseline is built
- Every bug-fix commit comes with a test case
- Keep the build fast
- Test in a clone of the production environment
- Make it easy to get the latest deliverables
- Anyone can see the results of the build and the changes that made it
- Automate deployment
Continuous Deployment (CD)
- Automated deployments
- Once CI is done, then CD can take over
- E.g., changes to the website automatically appear
- Continuous Delivery
- Delivery of new version in a continuous fashion
- Allow for manual deployments
- E.g., post a new version of the installer, and the user chooses when to install
- Often Continuous Deployment is used when it really means Continuous Delivery
- Since CD is the end part of CI, the term CI/CD is used
Scenario
- Development
- Develop on a local machine
- Commit to version control, e.g., Git
- Automated testing triggered by commit
- Operations (among other things)
- Generate installers
- Test installers
- Post installers
- Note: What part is development and what part is deployment is not fixed
DevOps Systems
- Remote systems for deployment and operations
- Remote systems for development
- Use similar approaches and technologies to extend development into operations
- Create one long toolchain
Reality
- Much of development is automated (or should be automated)
- Parts of operation may be manual. Why?
- Requires manual "sign off"
- How operations were performed in the past
- More interactions with other parts of the organization
- Training
- Legal
- Administrative
Example: Bad Old Days
- A long-running Linux system requires a configuration change, e.g., fix a "Too many open files" error
- Change is made manually to the system
- Unexpected outage
- When the server is back up, the configuration change has disappeared
Solutions
- Manual
- Keep careful documentation, which operators religiously apply at every reboot
- Automated
- Configure that system so that it has the change
- mainframe (server)
- minicomputer (server)
- PC (server)
- OS: Linux and FreeBSD
- OS: Windows Server
- All servers are in a data center
Evolution of Servers
- Mainframes
- Expensive to purchase and operate
- Run all services on one large server
- Minicomputer, PC
- Not as expensive, could afford more than one
- Segment services by requirements: hardware, security issues, reliability
- Virtual Machine (VM)
- Easy to configure a new server
- One set of services per VM
- Containers
- Almost unlimited
- One service per container
O.S. & Servers
- Mainframes
- O.S. has to work for all potential services
- Minicomputer, PC
- Can tune one overall O.S. for each server
- Or use a different O.S.
- Virtual Machine (VM)
- Containers
- Pick O.S. for application
Operating Systems for DevOps
- Windows 10, Windows Server
- Develop on Windows 10, operate on Windows Server
- Linux distributions
- macOS
Linux
- A Unix-like operating system that uses the Linux kernel
- A premier example of “open source”
- Center of the largest collaborative software effort
- Typically used in O.S. work
Parts of “Linux”
- Linux kernel - manage hardware, run a program, provide security
- GNU Project – utilities: bash, gcc, emacs, etc.
- X.org – X Window system
- GNOME & KDE – desktop environments
- Many other sources
Forerunners to Linux
- 1970: Unix Operating System – Thompson & Ritchie
- 1978: BSD (Berkeley Standard Distribution) - kernel + distribution
- 1983: GNU Project – Richard Stallman, the goal of a free Unix-like operating system, GPL (General Public License), written in 1989
- 1987: MINIX – Unix-like operating system for education by Andrew Tannenbaum
Problems with Unix-like O.S.’s for x86
- GNU Project had many utilities but no kernel (GNU Hurd)
- BSD based on 6th edition of AT&T Unix, and lawsuits limited development
- MINIX development was lacking, and limited distribution
- No good solution for running a Unix-like system on a PC
Linus Torvalds
- Computer Science student at the University of Helsinki, Finland, in 1991
- Started to write a terminal emulator (independent of the operating system) for his PC
- Developed using MINIX and the GNU C Compiler
- Grew into the Linux kernel
- Announced what he had started in the newsgroup comp.os.minix in August 1991
What are some reasons that companies supported Linux?
- Make sure it runs on their hardware, e.g., Intel
- Distributors of Linux, e.g., RedHat, Novell
- Embedded devices, e.g., Sony
- Platform for technical development, e.g., Volkswagen
GNU Project
- “GNU’s Not Unix”
- Started by Richard Stallman (rms) in 1983
- Start of the Free Software Movement and the corresponding FSF (Free Software Foundation)
GNU Four Freedoms
- 0: The freedom to run the program for any purpose
- 1: The freedom to study how the program works, and adapt it to your needs
- 2: The freedom to redistribute copies so you can help your neighbor
- 3: The freedom to improve the program, and release your improvements to the public, so that the whole community benefits
GNU GPL
- General Public License
- “Copyleft”
- Copyright retained by the author
- GPL Software must be available for free
- Source code must be made available
- If other developers make changes, they must distribute it with the pre-change source code
GPL Issues
- Cannot directly use in proprietary code
- What about programs developed using GPL editors, compilers, o.s.?
– Not a problem
- What about programs that call a function library, e.g., GNU C library?
– LGPL (Lesser General Public License)
- Companies make money off of services, not off the IP (Intellectual Property) of their code. E.g., srcML
Open Source Licenses
- Variety including BSD license, Apache License, Mozilla Public License
- GPL Version 2 in 1991
- GPL Version 3 in 2008
- Linux (GPL v2) - individuals hold copyright
– Linux stuck with GPL
v3 v2
- GNU Project – individuals assign copyright to FSF
GNU/Linux Distributions (distros)
- Installation program
- General configuration
- Packages of compatible software
- Dependency handling
- Support (technical and legal)
- Updates
- Security fixes
Differences between Distributions
- Community-based versus commercial
- Community: Debian, Slackware, Gentoo, and Arch Linux
- Commercial: Fedora (Red Hat), openSUSE (SUSE), Ubuntu (Canonical Ltd.)
- Hardware and device support
- Number of packages (different software) available
- Use of proprietary packages
- Package management
- Size, software typically installed
- Security
- Frequency of updates/releases
- Support
Why is Linux Often Used for DevOps?
- Not the only platform used
- Easy to optimize for size
- Easy to only include needed packages
- Not necessarily used to develop but to operate
- Linux Desktop issues are not a concern
WSL for Windows 10
- Windows Subsystem for Linux
- WSL introduced Aug 2016 (Windows 10, Version 1607)
- Native Linux compatibility (aim)
- WSL2 introduced in May, 2019, real Linux kernel
State of Unix
- Linux on servers
- Android based on Linux
- macOS based on BSD
- iOS based on BSD
- WSL on Windows