CPSC 489/589-010 DevOps Spring 2024

Archive

• Week 16 Class 30 Wed May 08 Posted: May 08

  • Final Exam May 9, 2:30 - 4:30 pm
  • Project 5
  • Finals Week Schedule Includes my office hours
  • Exercises DOOD and DIND Due date was Monday, May 6 at 4:30 pm. I will accept through 2:30 pm on Thursday (start of exam time). See the last class meeting in the Archives for details.

  • Project 4

    In some cases, the failure that I got when running the tests may be due to where the build directory is located. When building in a docker container, the build directory is not a subdirectory. Some tests assumed that the input data was in the source directory, and the source directory was the parent directory of the build.

    However, your tests should run wherever your build directory is located, so some points will still be taken off. I updated the Report.md and the scores for those who had this issue.

• Week 15 Class 28 Thu May 02 Posted: May 02

  After Class

  • Exercises DOOD and DIND

    For some reason I did not put a due date. Let's make it Monday, May 6 at 4:30 pm.

  • Project 4

    In some cases, the failure that I got when running the tests may be due to where the build directory is located. When building in a docker container, the build directory is not a subdirectory. Some tests assumed that the input data was in the source directory, and the source directory was the parent directory of the build.

    I will take another look at those who had failing tests.

    However, your tests should run wherever your build directory is located, so some points will still be taken off.

    I will post here once I am through going back over these.

  Announcements

  • Project 5: Due date changed and requirements clarified.
  • Office Hours My Office Hours on Thursday, May 2 are from 10:30 - 11:00 am and 3:15 - 4:15 pm.
  • Project 4 Scores posted in Brightspace. Posted Report.md in your repository. Report shows what you missed and other notes.
  • Final Exam May 9, 2:30 - 4:30 pm
  • Project 5
  • Finals Week Schedule

  Agenda

  • GitHub Actions

    Triggering a Workflow

  • Docker - Where we build our executable?

    • Build in container
    • Build in image

  • DIND (Docker IN Docker)

    1. Start a container to run docker in:
    2. Install the necessary Docker packages
    3. Start a Docker daemon in the background

    At this point, you can start another container inside of this one, view what containers are running, and contrast with the Host docker.

  • Exercise 36: DIND

    Record an asciinema session of running DIND. Set it up, and then use the docker ps command to verify that DIND is in effect.

    Commands must be entered in character by character, no cut and paste.

  • DOOD (Docker Outside Of Docker)

    1. Start a container to run docker in:
    2. Install the necessary Docker packages

    At this point, you can start another container that is a sibling of this one, view what containers are running, and see how it is the same as the Host docker.

  • Exercise 37: DOOD

    Record an asciinema session of running DOOD. Set it up, and then use the docker ps command to verify that DOOD is in effect, both in the container and on the host

    Commands must be entered in character by character, no cut and paste.

  • Saving Docker Container State
  • End Notes

• Final Exam Posted: May 2

  The Final Exam is in our regular classroom Final Exam: May 9, 2:30 - 4:30 pm.

  You are responsible for the following. Questions marked 589 are optional for those in the 489 section.

  • CPack Purpose and use.
  • Docker Compose Read and explain. Add a service, including an image build with a simple command.
  • CircleCI Read and explain
  • GitHub Actions Read and explain
  • Difference between Ubuntu, Fedora, and OpenSUSE package management
  • Why documentation is important for DevOps
  • Why you should use the most recent version of DevOps tools
  • 589 Compare CircleCI and GitHub Actions. If you had to choose one, which one would you choose? For which situations?
  • 589 Explain how the technologies of CMake, Docker, CircleCI, and GitHub Actions combine to build, package, and test source code

  The following are specific "How to" that you may be required to show:

  • Install a package in Fedora
  • Build a multi-architecture image in docker on the command line
  • Change a step in a GitHub Action to only run on macOS
  • Trigger a GitHub Action both manually and on every push
  • Build a docker multi-platform image
  • Add a test case to ctest
  • Run ctest using 4 separate tests in parallel
  • Install an executable file in the component CLIENT
  • Add a new step to a GitHub Action
  • Add a new job to a GitHub Action
  • Add a new job to CircleCI

  Optional: DIND, DOOD

  The exam has various types of problems, including short answers, definitions, comparisons, drawing diagrams, and coding.

  The questions are based on what I covered in class, including the book, the exercises, and the projects. If I did not cover it in class, it is not on the exam.

• Week 15 Class 27 Tue Apr 30 Posted: Apr 30

  After Class

  • Office Hours My Office Hours on Thursday, May 2 are from 10:30 - 11:00 am and 3:15 - 4:15 pm.

  • Project 5: Due date changed and requirements clarified.

  Announcements

  • Project 3 Scores posted in Brightspace. Updated Report.md (if needed) in the repository.

  • CircleCI Committed a multi-job, with upload of artifacts .circleci/config.yml in both the UACPSC/DevOpsComplexityCMake and mlcollard/DevOpsComplexityCMake.

    One change from class: The path: for store_artifacts is a directory. You can specify specific files, but can't seem to use a wildcard for specific files.

  • .env-build, .env-package

  Agenda

  • Docker Builders

    • docker builder ls
    • Docker Desktop
    • docker buildx create --use
    • Deleting builders with errors

  • CPack

    DevOpsComplexityCMake

    • Fixes to Ubuntu packaging
    • Fedora packaging
    • Debian triggers
    • Fedora pre and post install scripts
    • Debian local install
    • Hide local components

  • GitHub Actions

    • Share artifacts between jobs
    • Ordering jobs
    • Paths vs files for upload-artifacts

  • Self-Hosted GitHub Action Runners

    • Use docker image srcml/ubuntu for runner environment

• Week 14 Class 26 Thu Apr 25 Posted: Apr 25

  After Class

  • Project 3 Scores posted in Brightspace. Updated Report.md (if needed) in the repository.

  • CircleCI Committed a multi-job, with upload of artifacts .circleci/config.yml in both the UACPSC/DevOpsComplexityCMake and mlcollard/DevOpsComplexityCMake.

    One change from class: The path: for store_artifacts is a directory. You can specify specific files, but can't seem to use a wildcard for specific files.

  Announcements

  • docker buildx create --use

    For multiplatform builds.

  • Experiencing some issues where images are built, but are not showing up in the image list (docker image ls). This may be due to a builder issue. Will post as I figure out more.

  • DevOps Engineer time efficiency

  Agenda

  • CircleCI

    DevOpsComplexityCMake

    CircleCI Configuration Reference

    • Improve run format
    • workflows between multiple jobs
    • Share data between jobs

  • CPack

    DevOpsComplexityCMake

    cmake_host_system_information()

    • Ubuntu package naming
    • Package names are in all lowercase
    • Ubuntu-specific package setup
    • Extract debian packaging
    • Detecting platform

  • Using the bash screen command.

    screen-command

    • screen -S upgrade
    • Enter some commands
    • detach [ctrl + a] + d
    • screen -list
    • Logout
    • Log back in
    • screen -list
    • screen -r upgrade

• Week 14 Class 25 Tue Apr 23 Posted: Apr 23

  After Class

  docker buildx create --use

  For multiplatform builds.

  Agenda

  • Project 5

  • Docker Compose: Build tags

    DevOpsComplexityCMake

    Ubuntu Docker Hub

    Build tags

  • CircleCI

    DevOpsComplexityCMake

    CircleCI Configuration Reference

    Setup:

    1. Log into your GitHub Account
    2. Fork your project from UACPSC to your GitHub account
    3. Clone your forked repo. You will probably have to use a different directory name.
    4. Create an account at CircleCI
    5. Tie your CircleCI into your GitHub account (not any of your organizations)
    6. Create a project selecting your forked repository
    7. Create the file config.yml in the directory .circleci (it will guide you). The contents are not important
    8. See your project run (and fail)

    Working:

    1. Develop your config.yml file in your own forked repo

    Periodically and at Completion:

    1. Create a pull request to pull it back into the UACPSC/…
    2. Accept the pull request
    3. Verify the config.yml has the proper contents

  • Docker Compose & Dockerfile Principle

    Base variations on attributes not specific versions

    DevOpsComplexityCMake

• Week 13 Class 24 Thu Apr 18 Posted: Apr 18

  Announcements

  • Office Hours My 10:30 - 11:30 office hours today are moved to 3:15 - 4:15 pm.

  • Project 3 Feedback is in the file Report.md in your repository.

    If your Report.md states "Fix the Dockerfile so that it builds", put your fixes in by 8 am Monday, Apr 22.

  Agenda

  • General Principles

    • Be paranoid that it won't work anywhere else
    • Keep in mind that others will work on the project
    • Be painfully aware of all dependencies
    • Verify, do not just hypothesize
    • Think from the future looking back
    • Document carefully all steps and changes
    • Lean towards CLI not GUI (where appropriate)

  • Project 5

  • ARG and ENV

    DockerExamples

  • Share secrets with a container: Local file

    DockerExamples

  • Share secrets with Docker Secret

    DockerExamples

  • Parameters to GitHub Actions

    DevOpsComplexityCMake

  • Run and manage workflows from the command line

    GitHub Client

• Project 5: CI/CD Posted: Apr 18

  The following counts as two projects and is due Friday, May 3 Thursday, May 9.

  Implement the following in the Makefile repository in the branch compose:

  • A CMake that will package the executable using cpack.
  • A docker subdirectory with Dockerfiles that works for 3 different versions of Ubuntu and 3 different versions of Fedora, where each sets up an environment to build the executable.
  • The builds, packaging, and tests must work for the architectures linux/amd64 and linux/arm64.
  • A Docker Compose file, compose.yml, in the root directory that will build and run the docker images (for the Dockerfiles in the docker directory. The images built via the Docker Compose file must be multi-architecture, but the execution is one architecture at a time.
  • A Docker Volume that is shared among the build, package, and test steps
  • A set of environment files with the names .env-build, .env-package, and .env-test. The default should be a build.
  • A multi-job GitHub Action that will build, package, and test the executable and provide downloadable packages (both RPM and Deb). This action will incorporate a secret. The generated packages must go through linters (i.e., lintian or rpmlint). The result should be saved. As many corrections as possible should be made to what the lint program finds.
  • A CircleCI configuration that will build, package, and test.

  Students in the 589 section must additionally do the following:

  • Builds for 3 different versions of OpenSUSE.
  • DEBs and RPMs must be signed
  • Improved linter results.

  For students in the 489 section, any part of the 589 requirements will count as extra credit.

  Items crossed out are not required. If you do complete them, they will count as extra credit.

  Some restrictions:

  • Image names must be part of your Docker Hub account and must be publicly available.
  • The images at Docker Hub must be linked to your GitHub repository so that when new Dockerfiles are pushed, the images at Docker Hub are rebuilt.
  • All parts must be in separate commits with good, clear commit messages. Commit messages must be in the proper form.
  • All good practices must be followed.

• Week 13 Class 23 Tue Apr 16 Posted: Apr 16

  After Class

  • Office Hours My 10:30 - 11:30 office hours today are moved to 3:15 - 4:15 pm.

  • Project 3 Feedback is in the file Report.md in your repository.

    If your Report.md states "Fix the Dockerfile so that it builds", put your fixes in by 8 am Monday, Apr 22.

  Agenda

  DevOpsComplexityCMake

  • CPack

    CMake Documentation

    CPack Gist

    CPack Naming

  • Docker Compose
  • Docker Volume
  • GitHub Actions - Custom containers

• Week 12 Class 22 Thu Apr 11 Posted: Apr 11

  Agenda

  • CPack

    DevOpsComplexityCMake

    CMake Documentation

  • Exercise 35: Commands

    Keep track of the commands entered in class. You may abbreviate long filenames

    With each command write the general purpose.

    Use the link of the title of the exercise to submit.

    Due by 4:30 pm on Monday, April 15.

• Week 12 Class 21 Tue Apr 09 Posted: Apr 09

  Agenda

  • xz vulnerability

  • Docker Compose

    DevOpsComplexityCMake

    • docker compose build
    • docker compose up
    • export PLATFORM="linux/amd64"
  • CPack
    • Installation is trickier than you think

• Week 11 Class 20 Thu Apr 04 Posted: Apr 04

  Announcements

  • Project 4

  Agenda

  • GitHub Actions

    • Conditional actions
    • Multi-stage Dockerfiles
    • DevOpsComplexityCMake
  • Docker Compose
    • Building on multiple platforms
    • Environment variables
    • Docker Compose
    • DockerCompose Example

  • CPack

    • DevOpsComplexityCMake

• Week 11 Class 19 Tue Apr 02 Posted: Apr 02

  Announcements

  • Project 4

  Agenda

  • GitHub Actions

    • DevOpsComplexityCMake Note: Moved from mlcollard to UACPSC
    • Workflow syntax for GitHub Actions
    • .github/workflows
    • Default branch
    • actions/checkout
    • curl -L "https://github.com/Kitware/CMake/releases/download/v3.28.3/cmake-3.28.3-linux-x86_64.tar.gz" | tar xz --strip-components=1 -C /usr/local
    • actions/upload-artifact
    • Matrix Strategy
    • GitHub Action Runners
    • Upload with Matrix
  • Docker with Fedora
    • Dockerfile
      • Branch: fedora
    • dnf
    • Package name differences
  • Docker Compose
    • Docker Compose
    • DockerCompose Example

• Week 10 Class 18 Thu Mar 21 Posted: Mar 21

  Announcements

  • Project 4

  Agenda

  • UNIX
    • hyperfine
  • ctest
    • DevOpsComplexityCMake
    • FlagsBase
    • --timeout n

  • GitHub Actions

    • DevOpsComplexityCMake Note: Moved from mlcollard to UACPSC
    • Workflow syntax for GitHub Actions
    • .github/workflows
    • Default branch
    • actions/checkout
    • curl -L "https://github.com/Kitware/CMake/releases/download/v3.28.3/cmake-3.28.3-linux-x86_64.tar.gz" | tar xz --strip-components=1 -C /usr/local
    • actions/upload-artifact

  • Docker Compose

    • Docker Compose
    • DockerCompose Example

• Project 4: Testing & Actions Posted: Mar 21

  Project 4 is due by Midnight on Friday Apr 5

  Implement the following for the Makefile repository in the new branch actions:

  Part A: Create a testsuite:

  • Must be in the test directory testsuite
  • Must have a separate CMakeLists.txt in the test directory (linked into the main CMakeLists.txt)
  • Must have at least 5 tests. They can be shell scripts, or executable programs.
  • Must be able to be run via ctest
  • All tests must pass

  Part B: Create a GitHub Action workflow to build and run tests:

  • Triggered manually at GitHub
  • Produces a downloaded test report and other artifacts
  • Works on both Ubuntu and macOS. Should be easily adaptable to Windows also.
  • Must use at least cmake 3.28

  As with all topics in this class, an iterative process of a small edit followed by build and test is core to DevOps (in general, not just this class). Failure to do so on this project will lead to a lower score.

• Midterm Class Meeting Posted: Mar 14

  • Midterm Exam Thursday, Mar 14
  • In our regular classroom
  • Paper is provided
  • No computer needed

• Week 9 Class 15 Tue Mar 12 Posted: Mar 12

  After Class

  • Project 2 Scores and feedback are all in Brightspace

  Announcements

  • Midterm
  • Project 3: Product

  Agenda

  • Environment variable

    • CMAKE_GENERATOR
    • sshpass -e
    • export UseMultiToolTask=true

  • srcml

    • srcml --text="int n;" -l C++
    • Installed on devops
    • Ubuntu package in /usr/share/srcml on devop

  • ctest

    • DevOpsComplexityCMake
    • FlagsBase

• Week 9 Class 15 Tue Mar 12 Posted: Mar 12

  Announcements

  • Office Hours 3:15 - 4:15 pm Office Hours are cancelled for today
  • Project 2 Scores and feedback are all in Brightspace

  Agenda

  • UNIX
    • df -h
    • du -hs
    • top
  • ctest
    • DevOpsComplexityCMake
    • FlagsBase
    • --timeout n

  • GitHub Actions

    • DevOpsComplexityCMake Note: Moved from mlcollard to UACPSC
    • Workflow syntax for GitHub Actions
    • .github/workflows
    • Default branch
    • actions/checkout
    • curl -L "https://github.com/Kitware/CMake/releases/download/v3.28.3/cmake-3.28.3-linux-x86_64.tar.gz" | tar xz --strip-components=1 -C /usr/local

• Week 8 Class 14 Thu Mar 07 Posted: Mar 07

  Announcements

  • Midterm
  • Project 3: Product
  • Project 4 Delayed for now

  Agenda

  • Shared vs. static builds
  • Docker: docker commit, docker image ls, docker system prune
  • Environment variable: CMAKE_GENERATOR
  • Docker: CMD & ENTRYPOINT, shell vs exec form
  • CMake: make install, make install/strip
  • CMake: ctest, shell scripts, testsuite

• Midterm Exam Posted: Mar 7

  The Midterm Exam is during class time in our regular classroom on Thursday, Mar 14.

  You are responsible for the following:

  • Development vs. operations - What they are, what they involve
  • Project Stakeholders
  • How does DevOps relate to development? How does DevOps relate to operations?
  • Operation Responsibilities: Know and be able to explain at least 5 with a short description
  • Operation Job Categories
  • Textbook definition of DevOps
  • What problems does DevOps try to solve?
  • Release Planning - steps
  • Agile - Relationships to DevOps
  • Continuous Integration
  • Continuous Deployment
  • GNU GPL Four Freedoms
  • ssh - command to generate keys, role of public vs. private key, basics of .ssh/config file, agent forwarding
  • Linux accounts - default directory for home directories, root account, /etc/passwd, /etc/group
  • Ubuntu package handling - apt-get, update, upgrade, install
  • Virtualization - emulator, simulator and their tradeoffs, VMM, hypervisor, Challenges of VMMs, Isolation, Consolidation, Migration
  • 589 Only Popek & Goldberg VMM Properties & Theorems
  • 589 Only Virtualization-specific challenges
  • Make - Given Makefile, improve: @PHONY, $<, $@, variables, separate compilation, standard targets, adding a target
  • GNU Autotools - purpose
  • CMake - Know the following CMake instructions: project(), cmake_minimum_required(), add_executable(), target_link_library(), , find_package(): libxml2 and libarchive. Be able to add a target.
  • CMake - Purpose, relationship to make and ninja, in-source vs. out-of-source builds
  • CMake - CMakeLists.txt, vs. toolchain files vs. presets
  • shared vs. static libraries
  • vcpkg
  • Docker commands and options: run, build, exec, history, --volume, --interactive, --tty
  • Dockerfile: FROM, RUN, COPY, ARG, ENV, CMD, ENTRYPOINT
  • Environment customization: aliases, bash functions, .bashrc

  The exam has assorted problem types, including short answers, definitions, comparisons, configuration, and coding.

  Questions are from what I covered in class, the exercises, and the projects. If I did not cover it in class, it is not on the exam.

• Week 8 Class 14 Tue Mar 05 Posted: Mar 05

  After Class

  • Project 4 Delayed for now

  Announcements

  vcpkg

  • The environment variable, VCPKG_ROOT, may not be set. You can replace $VCPKG_ROOT with the path to your vcpkg cloned repository, or set it with export VCPKG_ROOT=<INSERT PATH HERE>

  • The use of libxml2 has an issue with liblzma in Ubuntu (and therefore in WSL). To fix, install pkg-config.

  • Exercise 32: Docker Hub Accounts

    In order for me to view you Docker Hub account, you have to have an image. So push an image if you need to.

  Agenda

  • Project 3: Product
  • Filename case example
  • Docker: COPY
  • Docker: ARG & ENV
  • Docker: ARG ${TAG}
  • Customizing the environment

• Project 3: Product Posted: Mar 5

  Project 3 is due by Noon on Wed Mar 13.

  Instead of only providing an environment for building, e.g., srcml/codespaces, most Docker containers offer a service or produce a product. In our case, a product is an executable program or an installer for an executable program.

  Create a Dockerfile for an application. The image must:

  • Provide the correct environment for building an application
  • Build the application during the building of the image
  • Must use a library, libxml2 or libarchive, or another library of your choosing
  • Works for Ubuntu 22.04, 24.04, and 23.10 (single Dockerfile, multiple potential tags)
  • Use a very recent version of CMake
  • Be as small as possible
  • Uses git to download the files for the program, or add the files to the repository with the Dockerfile

  A container of the image must:

  • By default, run the application without any additional docker commands
  • Be able to handle parameters sent to the container and pass them on to the image

  The Dockerfile must:

  • End in a newline
  • Only have single blank lines
  • Document each major part
  • Document the parameters of the built-in application
  • Use LABEL
  • Must commit each part separately

• Week 7 Class 13 Thu Feb 29 Posted: Feb 29

  After Class

  vcpkg

  • The environment variable, VCPKG_ROOT, may not be set. You can replace $VCPKG_ROOT with the path to your vcpkg cloned repository, or set it with export VCPKG_ROOT=<INSERT PATH HERE>

  • The use of libxml2 has an issue with liblzma in Ubuntu (and therefore in WSL). To fix, install pkg-config.

  Agenda

  • Exercise 32: Docker Hub Accounts

    Create a Docker Hub account. Enter your Docker Hub username into the form.

  • Exercise 33: vcpkg

    DevOpsComplexityCMake

    Get started with vcpkg

  • Exercise 34: Dockerfile II

    Follow along.

    Dockerfile

    You can build an image from this Dockerfile, and create a container from this image, anywhere you have access to docker. You can even copy the Dockerfile to the devops server and build there:

      scp Dockerfile devops:.
    

  • GitHub Actions

  All exercises are due by 4:30 pm on Fri Mar 01

• Week 7 Class 12 Tue Feb 27 Posted: Feb 27

  Agenda

  • Exercise 30: DevOpsComplexity CMake IV

    Follow along and duplicate my changes to the CMake for the DevOps Complexity program.

    DevOpsComplexityCMake

    Get started with vcpkg

  • Exercise 31: Dockerfile

    Follow along and create a Dockerfile for an image that allows us to have containers that build programs.

    Dockerfile

    You can build an image from this Dockerfile, and create a container from this image, anywhere you have access to docker. You can even copy the Dockerfile to the devops server and build there:

      scp Dockerfile devops:.
    

  All exercises are due by 4:30 pm on Wed Feb 28

• Week 6 Class 11 Thu Feb 22 Posted: Feb 22

  Announcements

  • Exercise Scores I am caught up with the exercise scores, except for the optional Entropy.

  • Project 1

    I posted my current version of feedback for Project 1 in your GitHub repository. It is in the branch feedback in the file Report.md. Best to view it on GitHub.

    If you can use gh browse, then you can go directly to it:

      gh browse --branch feedback Report.md
    

    If you are using WSL and gh browse cannot open a browser, then you should try to fix that. You are in a DevOps course, and fixing stuff like that is common.

    I will post scores in Brightspace once I am done with them, and will update this page.

    The due date for Project 2 remains the same. However, I will allow changes to your Project 2 (based on feedback from Project 1) until Monday, Feb 26. The basics must be implemented by the original due date.

  • Overall Feedback

    • When you are given a list of things to implement, you must take care to do all of them. E.g., Project 1. In many cases, all items in a list may have the same importance.
    • Following conventions and examples is important. Keywords in ssh_config are case insensitive, e.g., host devops, and indentation is not required. I did not know this before grading your exercise (and had to look it up) However, not using the given form, e.g., Host devops, and not indenting will lead to unnecessary confusion, and with some configuration files, the wrong result.
    • Documentation is important. Build and configuration code is often developed and then left for long periods of time, until changes are needed.
    • At every possible point during development of build files, you must check that the build works. Incremental steps with as much full testing as possible.

  Agenda

  • Exercise 23: DevOpsComplexity CMake III

    Follow along and duplicate my changes to the CMake for the DevOps Complexity program.

    DevOpsComplexityCMake

  • Exercise 28: Docker: Build from Local Files

    In an asciinema screencast, show using Docker to build from a local file. This can be any code.

    Make sure to time how long the build takes (after a make clean).

  • Exercise 29: Docker: Build Using Another Architecture

    In an asciinema screencast, show using Docker to build a local file from another architecture. So if you are on amd64 (x86), build on arm64, and if you are on arm64 (Apple Silicon), build on amd64.

    Make sure to time how long the build takes (after a make clean).

    In your screencast, echo how much longer it takes as a ration (10x longer, etc.)

  All exercises are due by 4:30 pm on Monday, Feb 26

• Week 5 Class 10 Thu Feb 15 Posted: Feb 15

  After Class

  • Exercise Scores I am caught up with the exercise scores, except for the optional Entropy.

  • Project 1

    I posted my current version of feedback for Project 1 in your GitHub repository. It is in the branch feedback in the file Report.md. Best to view it on GitHub.

    If you can use gh browse, then you can go directly to it:

      gh browse --branch feedback Report.md
    

    If you are using WSL and gh browse cannot open a browser, then you should try to fix that. You are in a DevOps course, and fixing stuff like that is common.

    I will post scores in Brightspace once I am done with them, and will update this page.

    The due date for Project 2 remains the same. However, I will allow changes to your Project 2 (based on feedback from Project 1) until Monday, Feb 26. The basics must be implemented by the original due date.

  Announcements

  • After Class

  • Exercise 17: Entropy

    This exercise is optional for CPSC 489 students, but required for CPSC 589 students.

    Write a 1 page PDF that explains the relationship between the number of words in a Diceware password, the number of bits of entropy, and the estimated time it would take to break the password. For the estimated times, provide a reference or the calculation used.

    End the paper with an argument for what length of Diceware password to use.

    Upload the paper to Brightspace. This is due by 11:59:59 on Thu Feb 15.

  • Exercise 21: passwd /etc/passwd

  Agenda

  • Exercise 24: SSH Config

    Create your .ssh/config if you haven't already on your local machine.

    Make sure it has a Host of devops which handles the devops server IP address, your username, and agent forwarding. If on macOS, customize it to your O.S.

    Copy the .ssh/config from your local machine to the devops server in the same location:

      scp ~/.ssh/config devops:.ssh/.
    

    Verify on the devops server that the file copied and the contents are the same:

      ssh devops tree -a
      ssh devops cat .ssh/config
    

    Note that the tree command does not have to be installed on your local machine. You are running these commands on the devops server where they are installed.

  • Notes: Virtualization
  • Exercise 26: Virtualization Crossword
  • Guide: Docker Setup

  • Exercise 25: Docker rmcomment

    Create an asciinema screencast of applying the command rmcomment to some source code.

    However, in this case, the command will not be on the devops server, but in a Docker container.

    Do not log into the devops server, but use the docker command as part of a pipe.

    docker run -i mlcollard/srctools rmcomment

  • Exercise 23: DevOpsComplexity CMake III

    Follow along and duplicate my changes to the CMake for the DevOps Complexity program.

    DevOpsComplexityCMake

  All exercises are due by 4:30 pm on Wednesday, Feb 21

• Project 2: CMake Posted: Feb 15

  The application code is the same as for Project 1. The difference is that you will create a CMake build version.

  Make sure to include the following:

  • Complete build that works on Linux (GitHub Codespaces)
  • Demo file
  • all target
  • run target
  • check target that uses the demo file
  • Compiles with -O3 optimization
  • Checks for all warnings
  • …TBD

  You will work on the project in a branch called cmake. Create a branch of the current code:

  git switch -c cmake
  git push
  

  The git push will have an error message. Follow what it says.

  Then, delete the Makefile from the cmake branch (not from the main branch):

  git rm Makefile
  

  Commit and push as always.

  The project is due 11:59:59 pm on Thursday, Feb 22

• Week 5 Class 9 Tue Feb 13 Posted: Feb 13

  After Class

  • passwd The chfn now works.

  Announcements

  • Exercise 17: Entropy

    This exercise is optional for CPSC 489 students, but required for CPSC 589 students.

    Write a 1 page PDF that explains the relationship between the number of words in a Diceware password, the number of bits of entropy, and the estimated time it would take to break the password. For the estimated times, provide a reference or the calculation used.

    End the paper with an argument for what length of Diceware password to use.

    Upload the paper to Brightspace. This is due by 11:59:59 on Thu Feb 15.

  • Build Configurations

  Agenda

  • Exercise 21: passwd

    1. Shell into the devops server using your account
    2. Generate a password using mkpasswd
    3. Change your password using passwd
    4. Change your local (gecos) information using the command chfn
    5. Log out
  • Notes: More SSH

  • Exercise 22: rmcomment

    Create an asciinema screencast of applying the command rmcomment to some source code.

    Do not log into the devops server, but use the ssh command as part of a pipe.

  • Exercise 23: DevOpsComplexity CMake III

    Follow along and duplicate my changes to the CMake for the DevOps Complexity program.

    DevOpsComplexityCMake

  All exercises are due by 4:30 pm on Wed Feb 14

• Week 4 Class 8 Thu Feb 08 Posted: Feb 08

  After Class

  • Crossword answers that I did not specifically mention in class:

    • /usr/include Location of official include files
    • /usr/local/include Location of non-official include files
    • pipe The symbol "|" in a shell is the __ symbol

  Announcements

  • Project 1

  • Diceware

    • Important to remember that these are not any words. They are words randomly chosen from a specific dictionary.
    • Use a separator between the words. Do not run them all together, as that is much more difficult to remember, and is less secure.
    • If the 6-word diceware password is not secure enough for you, add another word to your password.
    • 12.9 bits of entropy per word

  • SSH Public Key

    • Your public key is the entire contents of the file id_ed25519.pub, including the prefix of ssh-ed25519 and the suffix with your username or comment. Make sure you entered the complete key in the form.
    • When directed to provide a specific type of SSH key, create that type. E.g., I did not ask for an RSA key.
    • Do not create the .ssh directory yourself. Let ssh-keygen do that. Why? You need to create things with the proper permissions. If you use mkdir to create the directory, you have to look up what permissions to make it. If you let ssh-keygen create it, then it uses the proper permissions.
    • Use defaults of commands such as ssh-keygen, especially when you are first using the command. Some of you did not use the default key filenames, and the ones you created are non-standard and not particularly good.

  Agenda

  • Exercise 17: Entropy

    This exercise is optional for CPSC 489 students, but required for CPSC 589 students.

    Write a 1 page PDF that explains the relationship between the number of words in a Diceware password, the number of bits of entropy, and the estimated time it would take to break the password. For the estimated times, provide a reference or the calculation used.

    End the paper with an argument for what length of Diceware password to use.

    Upload the paper to Brightspace. This is due by 11:59:59 on Thu Feb 15.

  • Notes: Linux Accounts
  • Exercise 18: Linux Accounts Crossword

  • Exercise 19: Login

    If you provided your SSH Public Key before the due date, your key is installed in your account on the devops server.

    Your username is the your UA id (email without the @uakron.edu, e.g., jdoe1@uakron.edu has the account name jdoe1

    The devops server IP address is on Brightspace.

    1. Shell into the devops server using your account
    2. Change your local (gecos) information using the command chfn Requires password, so wait on this step.
    3. Log out

  • Exercise 20: DevOpsComplexity CMake II

    Follow along and duplicate my changes to the CMake for the DevOps Complexity program.

    DevOpsComplexityCMake

  All exercises are due by 4:30 pm on Fri Feb 09

• Week 4 Class 7 Tue Feb 06 Posted: Feb 06

  After Class

  • SSH Public Key Your public key is the entire contents of the file id_ed25519.pub, including the prefix of ssh-ed25519 and the suffix with your username or comment. Make sure you entered the complete key in the form.

  Announcements

  • Project 1

  Agenda

  • Notes: Passwords & SSH

  • Exercise 14: Diceware

    Create a 6-word diceware password. For each word, show the random numbers generated, and the equivalent word from the word list. Then show the entire 6-word diceware password.

    Notes:

    • Use the EFF Long Word List
    • Preferably, use real dice. As an option (only for this assignment), use an online diceroll
    • Do not use a previous diceware password
    • Do not use this password for anything else

    Enter this data using the form in the link of the exercise.

  • Exercise 15: SSH Key Generation

    In a UNIX environment (Linux, macOS, GitHub Codespaces) perform the following steps:

    • Generate an ed25519 key
    • Show the files created in the directory .ssh
    • Show the contents of the public key

    Create an asciinema screencast, and upload the (proper) URL.

  • Exercise 16: SSH Public Key

    Fill in the form in the link of this exercise with the SSH public key you will use from your shell. This can be from WSL, Linux, or macOS.

    Make sure that this public key is installed in the machine/shell you will be using to access a remote server.

  • Exercise 13: DevOpsComplexity CMake

    Follow along and duplicate my changes to the CMake for the DevOps Complexity program.

    DevOpsComplexityCMake

    This is a different repo than the one we used for Make. Invitation link in GitHub.

  All exercises are due by 4:30 pm on Wed Feb 07

• Project 1: Makefile Posted: Feb 6

  The decompressor is a universal decompressor that uses libarchive to convert a compressed and archived file to the original content. If the compressed file is an archive with multiple entries, the output entries are separated by ASCII NULL ('\0'). The input formats are shown on the libarchive formats.

  Create a Makefile and associated files to build the Decompressor project. Make sure to include the following:

  • Complete build that works on Linux (GitHub Codespaces)
  • Demo file
  • all target
  • run target
  • check target that uses the demo file
  • Compiles with -O3 optimization
  • Checks for all warnings

  Notes:

  • The project is due 11:59:59 pm on Tuesday, Feb 13.
  • The GitHub Classroom Invitation Link is in Brightspace. It will disappear at the start of the next class meeting after this is assigned.
  • You must write the Makefile by hand and commit it in stages. The Makefile cannot be generated from another tool, i.e., CMake.
  • The Makefile cannot be generated by another tool. It must be committed to the repository in stages.
  • The full command to build on Linux that works on GitHub Codespaces:
  • The full command to build on macOS: While you can develop on macOS, you must finish the Makefile on Linux, e.g., GitHub Codespaces.
  • The decompressor can handle multiple compressions on the same file, e.g., text.bz2.gz.

• Week 3 Class 6 Thu Feb 01 Posted: Feb 01

  Announcements

  • After Class from previous class meeting
  • git issues from last class
  • Exercise scores, and when a feature is not a feature
  • The problem with dependencies

  Agenda

  • CMake

    • CMake Background
    • CMake Website
    • CMake Downloads
    • CMake Documentation
    • CMake Book
    • CMake Source
  • Exercise 10: CMake Background Crossword

  • Exercise 11: Dance

    Follow along and duplicate my changes to the CMake for the Dance program.

    mlcollard/Dance

  • Exercise 12: Dance Dance Dance

    An optional exercise to improve the dancing in the Dance program. This involves either increasing the number of frames, the dance moves, or the number of dancers.

    To implement this exercise, improve the dance program in your repository. Then tag the commit with the commands:

    As with all tags, verify on GitHub that the tag appears. This is a totally optional exercise, and credit will be given based on the quality of the result.

  All exercises are due by 4:30 pm on Mon, Feb 5

• Week 3 Class 5 Tue Jan 30 Posted: Jan 30

  After Class

  • There are some commits for the Git source code that cause compilation problems. To get around this, try just building for the executable git, not all the other stuff. To do so (with a parallel build):

  • The srcml/codespaces image (used in GitHub Codespaces) did not have autoconf nor autoheader installed. I updated the image so that they are installed. If you have a running codespace, delete it and start another one to get the newer image.

  • For WSL/Linux some packages need installing. From a base Ubuntu 22.04 image, install the following:

  • For macOS: brew install autoconf

  Announcements

  • Choices

    

  Agenda

  • Exercise 6: Makefile

    Follow along and duplicate my changes to the Makefile for the DevOpsComplexity program.

    mlcollard/DevOpsComplexity

  • GNU Autotools (GNU Build System)

    The make utility needs help

    • autoheader - Generates config file
    • autoconf - Generates configure script
    • configure - Generated by above, custom script for that build

    DevOpsHelloWorld

  • Exercise 8: HelloWorld Autotools

    Build and run the HelloWorld program. The steps are:

    • Clone the repository and move to the repo directory (Not required if you are using GitHub Codespaces)
    • Build the program using the GNU Autotools
    • Show the program running

    To demonstrate this, you will record a terminal session using asciinema. It records the commands you enter and the output the command produces. asciinema is already installed in the GitHub Codespaces image. You can also install asciinema on Linux/WSL/macOS.

    asciinema is relatively straightforward to use.

    You can play, pause, skip, etc. You can also copy the commands as text. Note that the replay even includes your pauses and any delay as the command is run.

    At the end of the terminal session, asciinema will show a URL. Anybody can use this URL to view the session. This is what you will submit for the exercise to the link in the exercise title. The URL should not contain "connect" or be a local file. Test your URL in a browser; an incorrect URL will receive a 0.

  • Exercise 9: Git Autotools

    Build and run the Git program. The steps are:

    • Clone the repository and move to the repo directory
    • Build the program using GNU Autotools
    • Show the git program running. Note that the executables are put in the same directory, so it is ./git

    As in Exercise 8, show this in an asciinema screencast and submit via the link of this exercise.

  All exercises are due by 4:30 pm on Wed Jan 31

• Week 2 Class 3 Tue Jan 23 Posted: Jan 23

  Agenda

  • Notes: Servers & OS
  • Exercise 4: Servers & OS Crossword

  All exercises are due by 4:30 pm on Wed Jan 24

• Week 2 Class 3 Tue Jan 23 Posted: Jan 23

  After Class

  • Exercise 6: Makefile - No changes to make yet, except for the removal of srcMLXPathCount_t.cpp

  Announcements

  • TA
  • Guide: GitHub Authentication

  Agenda

  • Notes: Agile
  • Notes: UNIX Shells
  • Exercise 5: UNIX Shells Crossword

  • Exercise 6: Makefile

    Follow along and duplicate my changes to the Makefile for the DevOpsComplexity program.

    mlcollard/DevOpsComplexity

    GitHub Codespaces is enabled for all repositories.

    Commit these changes to the your GitHub repository. Link to create your repository is available in Brightspace.

    Note If the file srcMLPathCount_t.cpp is in your repository, then it needs to be deleted. First, remove the file:

    Then commit with the following commit message (use copy/paste):

    Then push the commit.

    Note We did not get to this yet, so this exercise is delayed.

  • Exercise 7: Shell Commands

    Keep a list of the shell commands that I show in class. After class, submit this list using the link of this exercise.

    The list is in the form of one command per line, with the command name and a short description separated by a comma.

  All exercises are due by 4:30 pm on Monday, Jan 29

• Week 1 Class 2 Thu Jan 18 Posted: Jan 18

  Announcements

  • I posted grades and feedback for Exercises 1 and 2 in Brightspace
  • DevOps Survey

  • DevOps Tools Survey

    • Importance of format
    • Cleansed Tool Results
  • Development Environment: Tools

  Agenda

  • Notes: DevOps Introduction I
  • Notes: DevOps Introduction II
  • Exercise 3: DevOps Introduction Crossword
  • Notes: Servers & OS
  • Exercise 4: Servers & OS Crossword Since we did not complete the notes, this will be due the day after the next class

  All exercises are due by 4:30 pm on Fri Jan 19

• Week 1 Class 1 Tue Jan 16 Posted: Jan 16

  Announcements

  • Syllabus

  • GitHub We use GitHub and GitHub Classroom in the class to manage source code for projects, exercises, and examples. You are required to have a GitHub account.

    Optionally, you can apply for the Student Github Developer Pack and get a free GitHub Pro account. To apply, your account email can be a school account, or a personal account if you upload documents to prove your current enrollment status.

  • Development Environment: Tools Follow the directions in the guide to set up your development environment. Department servers have older versions of g++ and cmake, so do not rely on them.
  • GitHub Codespaces All exercises and projects have GitHub Codespaces as an option

  • O'Reilly Online Learning The course textbook is available in O'Reilly Online Learning.

    You do not have to be on the campus network (e.g., via VPN) to access O'Reilly Online Learning.

    To access content, click the "Institution not listed?" link below the dropdown and enter your UA email. You will then create an account and use that account from now on.

    For more information: Library LibGuide

  Exercises

  • Exercise 1: DevOps Survey
  • Notes: DevOps Motivation
  • Exercise 2: DevOps Tools Survey
  • Notes: DevOps Introduction I
  • Notes: DevOps Introduction II

  All exercises are due by 4:30 pm on Wed Jan 17

• Greetings and welcome to CPSC 489/589 DevOps for Spring 2024.

  • Syllabus The syllabus for the course is on the main course page. It is a draft that I will finalize before our first class meeting.
  • Survey The first exercise is to fill out the class survey. The survey is due after our first class meeting, but you can complete it before.
  • Development Environment Development Environment: Tools is a guide for setting up your programming development environment for the class. It is good to start early as you will need this early in the course.

  • GitHub We use GitHub and GitHub Classroom in the class to manage source code for projects, exercises, and examples. You are required to have a GitHub account.

    Optionally, you can apply for the Student Github Developer Pack and get a free GitHub Pro account. To apply, your account email can be a school account, or a personal account if you upload documents to prove your current enrollment status.

  See you on Tuesday, Jan 16 at 2:00 pm -   3:15 pm in (CAS) room 134