Vulnerability Assessment

Lecture Notes

This lecture will present an overview over vulnerability assessment processes and methodologies

Download here or here with additional generated notes.

Note: Notes are genereated by AI and edited by staff as way of providing additional comments. If you find issues, please report them.

Practical tasks

This exercise will be used to explore aspects of an auditing and or assessment, complementing the work developed in the last laboratory.

As the scope we consider all hosts and services available at the laboratory network. The assessment should not try to pentest the systems, but small explorations are allowed. No data should be corrupted, and no credentials should be modified. Normal operation of the VMs should not be disrupted.

Auditing Linux Systems

For this task we will setup local Ubuntu container inside the Virtual Machine and then audit them against reference bases.

In all containers you should download the compressed XML OVAL definitions from Canonical for that distribution. The definition files follow the following file naming convention:com.ubuntu.<release codename>.usn.oval.xml.bz2 Therefore, the OVAL definitions are specific for each distribution, as they relate with the actual software and configurations of that distribution.

The target will be Ubuntu 24.04. Setup a podman container as follows. The current folder will be mapped to /host inside the container. It will also persist across runs. Use it to store data you wish to save.

podman run -ti --rm -v .:/host ubuntu:24.04 /bin/bash
apt update
apt install openscap-scanner wget lsb-release bzip2 vim less bat

NOTE: The :Z suffix used on some systems is specific to SELinux (Fedora/RHEL) and is not required on Ubuntu, which uses AppArmor. If your VM runs Ubuntu, simply use -v .:/host as shown above.

Then download the OVAL definitions.

wget https://security-metadata.canonical.com/oval/com.ubuntu.$(lsb_release -cs).usn.oval.xml.bz2

Uncompress the data:

bunzip2 com.ubuntu.$(lsb_release -cs).usn.oval.xml.bz2

NOTE: There is a bug in recent versions of oscap and the XML must be patched. Look for the definition of ste:100 and replace it with:

<ind-def:family_state id="oval:com.ubuntu.noble:ste:100" version="1" comment="The singleton family state" >
  <ind-def:family>unix</ind-def:family>
</ind-def:family_state>

If you do not do this, you will see unknown states when evaluating the system.

Check the content of the XML file and analyze its structure. Take one vulnerability, such as USN-7457-1 (an OpenSSH vulnerability) and identify the test that is conducted to determine the existence of the vulnerability. Skip through the file and check it. First we have the OVAL definition for the USN, pointing to a test criterion named oval:com.ubuntu.noble:tst:745710000000

<definition id="oval:com.ubuntu.noble:def:745710000000" class="patch" version="1">
  <metadata>
    <title>USN-7457-1 -- OpenSSH vulnerability</title>
    <affected family="unix">
      <platform>Ubuntu 24.04 LTS</platform>
    </affected>
    ...
    <criterion test_ref="oval:com.ubuntu.noble:tst:745710000000" comment="openssh source package in noble, is affected and has been fixed (note: '1:9.6p1-3ubuntu13.11')." />
  </criteria>
</definition>

The criterion defines that at least one can be true for the test to be valid (USN exists)

<linux-def:dpkginfo_test id="oval:com.ubuntu.noble:tst:745710000000" version="1" check="at least one" check_existence="at_least_one_exists" comment="Does the 'openssh' package exist?">
  <linux-def:object object_ref="oval:com.ubuntu.noble:obj:685910000000" />
  <linux-def:state state_ref="oval:com.ubuntu.noble:ste:745710000000" />
</linux-def:dpkginfo_test>

The object refers to a state:

<linux-def:dpkginfo_object id="oval:com.ubuntu.noble:obj:685910000000" version="1" comment="The 'openssh' package binaries">
  <linux-def:name var_ref="oval:com.ubuntu.noble:var:685910000000" var_check="at least one" />
</linux-def:dpkginfo_object>

and finally the state (one of the states actually), conducts a test:

<linux-def:dpkginfo_state id="oval:com.ubuntu.noble:ste:685910000000" version="1" comment="The binary package version is less than '1:9.6p1-3ubuntu13.3'">
  <linux-def:evr datatype="debian_evr_string" operation="less than">1:9.6p1-3ubuntu13.3</linux-def:evr>
</linux-def:dpkginfo_state>

We are using standard OVAL definitions, that focus on vulnerable components, but it is possible to create several other definitions to audit a system against a desired state.

Using OpenSCAP you can evaluate the OVAL and generate an html report for all definitions:

oscap oval eval --report report1.html com.ubuntu.$(lsb_release -cs).usn.oval.xml

The output to stdout should be a series of lines for all the definitions being validated. A few seconds later, the tool should end. The output is generated in the file report1.html. You can check the file in the Host if you save it to the /host folder

cp report1.html /host/

TASKS 1 - An Updated Container:

Analyze the report produced. In your write-up, include: the total number of definitions evaluated, the counts per result (true / false / unknown / error), at least two definitions that evaluate to true (unpatched) with their USN/CVE references, and a brief explanation of what a false result means for a patch definition.

TASKS 2 - Updates Missing:

Run a second container of the same distribution (keep the first)
Remove the updates backports and security sources from the mirrors at /etc/apt/sources.list.d/ (on Ubuntu 24.04 this is the deb822 file ubuntu.sources)
Update the sources with apt update
Install software such as rclone, apache2, nginx.
Run the oscap oval eval (saving the report as report2.html to avoid clobbering the first one) and check the result.
What are the differences?

You can view the report using the browser.

As you can see, the results display the details of each vulnerability, as well as a link to the CVE for each vulnerability. If you see any result listed as true, you will want to address that vulnerability immediately. Do understand, there are quite a lot of vulnerabilities tested , so hopefully the Ubuntu server will come up false for every test. But this is commonly not the case. If updates are lacking, new OVAL definition files may uncover other known vulnerabilities.

Of course, you don’t really have to scroll through the entirety of the results. You can always do a quick glance at the report summary to see how many vulnerabilities are:

Unpatched (red)
Patched (green)
Errors (yellow)
Unknown (blue)
Other (white)

Developing custom controls

You can also create custom OVAL definitions to validate your own hardening controls. Popular lists of OVAL definitions to harden systems can be found in several places, either created by the community, or by specific communities and even standardization entities. Some examples are:

The DoD Cyber Exchange with Definitions under released as STIGs: https://www.cyber.mil/stigs/downloads/
The CIS Benchmark kits: https://learn.cisecurity.org/build-kits

For the purpose of this lab, we will consider a custom definition, which we plan to validate the compliance of a system. The definition is custom, but it could have been obtained from one of the previous sources. The definition aims to assert if root can login into the system.

<?xml version="1.0"?>
<ns0:oval_definitions
  xmlns:ns0="http://oval.mitre.org/XMLSchema/oval-definitions-5"
  xmlns:ns1="http://oval.mitre.org/XMLSchema/oval-common-5"
  xmlns:ns2="http://oval.mitre.org/XMLSchema/oval-definitions-5#independent"
  xmlns:ns3="http://oval.mitre.org/XMLSchema/oval-definitions-5#unix"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://oval.mitre.org/XMLSchema/oval-definitions-5 oval-definitions-schema.xsd
                      http://oval.mitre.org/XMLSchema/oval-common-5 oval-common-schema.xsd
                      http://oval.mitre.org/XMLSchema/oval-definitions-5#independent independent-definitions-schema.xsd">

  <ns0:generator>
    <ns1:schema_version>5.11</ns1:schema_version>
    <ns1:timestamp>2017-10-09T12:00:00</ns1:timestamp>
  </ns0:generator>

  <ns0:definitions>
    <ns0:definition id="oval:no_direct_root_logins:def:1" class="compliance" version="1">
      <ns0:metadata>
        <ns0:title>Direct root Logins Not Allowed</ns0:title>
        <ns0:description>Detailed description...</ns0:description>
        <ns0:reference ref_id="no_direct_root_logins"/>
      </ns0:metadata>
      <ns0:criteria operator="AND">
        <ns0:criterion comment="/etc/securetty file exists" test_ref="oval:etc_securetty_exists:tst:2"/>
        <ns0:criterion comment="/etc/securetty is empty" test_ref="oval:no_direct_root_logins:tst:1"/>
      </ns0:criteria>
    </ns0:definition>
  </ns0:definitions>

  <ns0:tests>
    <ns3:file_test id="oval:no_direct_root_logins:tst:1" check="all" check_existence="all_exist" 
        comment="/etc/securetty is empty" version="1">
      <ns3:object object_ref="oval:no_direct_root_logins:obj:1"/>
      <ns3:state state_ref="oval:no_direct_root_logins:ste:1"/>
    </ns3:file_test>
    <ns3:file_test id="oval:etc_securetty_exists:tst:2" check="all" check_existence="all_exist"
        comment="/etc/securetty file exists" version="1">
      <ns3:object object_ref="oval:etc_securetty_exists:obj:2"/>
    </ns3:file_test>
  </ns0:tests>

  <ns0:objects>
    <ns3:file_object id="oval:no_direct_root_logins:obj:1" version="1">
      <ns3:filepath>/etc/securetty</ns3:filepath>
    </ns3:file_object>
    <ns3:file_object id="oval:etc_securetty_exists:obj:2" version="1">
      <ns3:filepath>/etc/securetty</ns3:filepath>
    </ns3:file_object>
  </ns0:objects>

  <ns0:states>
    <ns3:file_state id="oval:no_direct_root_logins:ste:1" version="1">
      <ns3:size datatype="int" operation="equals">0</ns3:size>
    </ns3:file_state>
  </ns0:states>

</ns0:oval_definitions>

NOTE (corrected definition): the original version of this definition used textfilecontent54 tests with a pattern of ^$ to detect an empty file. That approach fails because an empty file contains zero lines (so the all_exist checks fail), making the definition report false even for a compliant system. The corrected definition above uses unix-def:file_test instead: one test asserts the file exists (check_existence="all_exist" with no state) and the other asserts the file size equals 0. The definition now evaluates to true only when /etc/securetty exists and is empty.

Background: What is `/etc/securetty`?

On many Linux systems, the /etc/securetty file acts as an access control list for the root user. It contains a list of “safe” TTY (teletypewriter) devices from which the root user is allowed to log in directly.

If a TTY is listed in this file, root can log in from it.
If a TTY is not listed, root login is denied.
If the file is empty, root cannot log in directly from any TTY device.
If the file does not exist, the behavior can vary, with some systems defaulting to allowing root login from anywhere—a significant security risk.

The most secure policy is for the file to exist but be completely empty, forcing all users (including administrators) to log in with their own account and use sudo to perform privileged tasks. This creates a clear audit trail.

NOTE: /etc/securetty only governs direct console/TTY logins; it does not restrict SSH root logins (those are controlled separately, e.g., PermitRootLogin in /etc/ssh/sshd_config).

Part 1: Auditing the `securetty` Configuration

We will use the provided OVAL definition, which checks for two conditions:

The /etc/securetty file must exist.
The file must be empty (size 0).

Step 1: Create a Non-Compliant State

Most default systems are already compliant. To test our OVAL definition, we will intentionally create an insecure configuration by allowing root to log in from a local TTY.

Open a terminal in your Linux VM.

Important: First, back up the original file so you can easily restore it later.


sudo cp /etc/securetty /etc/securetty.bak

Now, add tty1 to the file. This change will allow the root user to log in directly from the first virtual console. We use tee to write to the file with sudo.

echo "tty1" | sudo tee /etc/securetty

Verify that the file now contains content.

cat /etc/securetty

Expected output: tty1

Step 2: Run the OVAL Scan

With the system in a non-compliant state, let’s run the OpenSCAP scanner. Navigate to the directory where you saved securetty_check.xml.

Save the custom definition above to a file named securetty_check.xml before running the scan.

Execute the evaluation, generating an HTML report.

oscap oval eval --report report-fail.html securetty_check.xml

The scanner will immediately report the result as false in your terminal, indicating a failed check.

Definition oval:no_direct_root_logins:def:1: false

Step 3: Analyze the Results

Open the report-fail.html file in a web browser to inspect the detailed results. The report will show that the definition failed because the content of /etc/securetty did not match the expected state (an empty file).

Step 4: Remediate the Configuration and Verify the Fix

Now, let’s restore the secure configuration and confirm it with another scan. To remediate the system, we need to make the /etc/securetty file empty again. The truncate command is perfect for this.

sudo truncate -s 0 /etc/securetty

Verify that the file is now empty. The cat command should produce no output.

cat /etc/securetty

Run the OVAL scan again, saving the output to a new report file.

oscap oval eval --report report-pass.html securetty_check.xml

The terminal output should now be true, confirming compliance!

Definition oval:no_direct_root_logins:def:1: true

(Optional) When you are finished with the lab, restore the original file from your backup.

sudo mv /etc/securetty.bak /etc/securetty

Part 2: Your Turn! Author a New OVAL Definition

Now it’s your chance to create a new OVAL definition from scratch. Choose one of the following security controls, write the OVAL XML, and test it using the same methodology as in Part 1 (configure-fail, remediate-pass).

Choose a Security Control:

File Permissions: The file /etc/shadow, which contains hashed user passwords, must be owned by root and have permissions of 000 (no permissions for anyone). Hint: You will need a unix-def:file_test.
Prohibited Package: The rsh-server package (provides an insecure remote shell service) must not be installed. Hint: Use an rpminfo_test or dpkginfo_test and set check_existence="none_exist".
Service Disabled: The avahi-daemon service (for zero-configuration networking, often unneeded on servers) must be disabled. Hint: Use a unix-def:service_test.
Kernel Parameter: The kernel parameter net.ipv4.conf.default.accept_source_route (which can pose a security risk) must be set to 0. Hint: Use a unix-def:sysctl_test.

The skeleton for the first is:

<?xml version="1.0" encoding="UTF-8"?>
<oval_definitions
    xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5"
    xmlns:oval="http://oval.mitre.org/XMLSchema/oval-common-5"
    xmlns:unix-def="http://oval.mitre.org/XMLSchema/oval-definitions-5#unix"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

  <generator>
    <oval:product_name>TODO</oval:product_name>
    <oval:schema_version>5.11</oval:schema_version>
    <oval:timestamp>TODO</oval:timestamp>
  </generator>

  <definitions>
    <definition id="oval:etc.shadow.perms:def:1" version="1" class="compliance">
      <metadata>
        <title>TODO</title>
        <description>TODO</description>
      </metadata>
      <criteria>
        <criterion test_ref="oval:etc.shadow.perms:tst:1" comment="TODO"/>
      </criteria>
    </definition>
  </definitions>

  <tests>
    <unix-def:file_test id="oval:etc.shadow.perms:tst:1" version="1" check="all" check_existence="all_exist" comment="TODO">
      <unix-def:object object_ref="oval:etc.shadow.perms:obj:1"/>
      <unix-def:state state_ref="oval:etc.shadow.perms:ste:1"/>
    </unix-def:file_test>
  </tests>

  <objects>
    <unix-def:file_object id="oval:etc.shadow.perms:obj:1" version="1">
      <unix-def:filepath>/etc/shadow</unix-def:filepath>
    </unix-def:file_object>
  </objects>

  <states>
    <unix-def:file_state id="oval:etc.shadow.perms:ste:1" version="1">
      <unix-def:user_id datatype="int">0</unix-def:user_id>
      <unix-def:uread datatype="boolean">false</unix-def:uread>
      
      <!-- other: uwrite, uexec, gread, gwrite, gexec, oread, owrite, oexec -->

    </unix-def:file_state>
  </states>
</oval_definitions>

> NOTE: For a true `000` permission check, the state must set all permission bits (`uread`, `uwrite`, `uexec`, `gread`, `gwrite`, `gexec`, `oread`, `owrite`, `oexec`) to `false`. Also note that on a default Ubuntu system `/etc/shadow` is owned by `root:shadow` with permissions `0640`, so you will need to change it to `000` to create the non-compliant/compliant states for testing.


# References 

- The OVAL specification: https://github.com/OVAL-Community/OVAL
- OpenSCAP Project Website: The main portal for the OpenSCAP project, with documentation, tutorials, and download links. - https://www.open-scap.org/