IoT - Firmware Emulation

Firmware Reviewer Emulation Framework is an automated and scalable system for performing Dynamic Analysis of RTOS, QNX and Linux-based embedded Firmware, with No need of related physical Device.

 

Firmware Reviewer Emulation Framework

Firmware Reviewer Emulation Framework provides a collection of pre-built VMs, scripts, kernels and filesystem schemas to be used with our own patched version of QEMU to emulate IoT devices, aimed to facilitate IoT and Firmware Analysis by virtualising as much of the physical device as possible. It can emulate CPU different architectures and follows the IoT device boot-up process:

Device boot-up

The Emulation will cover such boot-up, building the following emulation architecture:

Firmware Reviewer Emulation Framework can do Partial (bootloader and/or Kernel+some drivers only) or Full Emulation (bootloader+kernel++some drivers+uncompressed filesystem+init+services) and includes the following components:

Bootloader

The bootloader is the first code that is executed after a system reset. Its goal is to bring the system to a state in which it can perform its main function. This requires hardware initialization and choosing the correct image to load from flash or other sources. Because of its key role, the bootloader is usually placed in a part of the Flash that is protected from accidental erasure or corruption. Hardware initialization may imply enabling access to RAM, setting up clocks and PLLs, and configuring other key peripherals. However, hardware initialization should be restricted to the essential at this level leaving the rest of the initializations for upper application code. The bootloader is a critical part of an embedded system because, if it is corrupted, there is no solution to start a program or to update a new bootloader. This is the reason why today, in all systems, there is a safe bootloader, stocked in ROM. The users or extrernal systems cannot access this memory and so cannot modify it. In case of problem, the system can use this bootloader to start. Generally, it allows to download a new bootloader to fix the older one which has been corrupted. Our Emulation Framework can simulate a bootloader corruption and modify the bootloader, inserting a Bootloader Agent.

Modified Kernels

MIPS, ARM, PPC, x86 and other libnvram-compatible architectures are supported, for instrumentation of firmware execution

This instrumented Kernels are built for the ARCH_VIRT virtual machine target, which uses VirtIO to perform hardware virtualization. In conjunction with newer versions of patched QEMU, this allows up to 32 VirtIO transports to be attached to the emulated machine, avoiding the PCI bus. This provides greater emulation flexibility by supporting more virtualized block and network devices.

  • A userspace NVRAM library to emulate a hardware NVRAM peripheral

Firmware Extractor

A recursive Firmware Extractor that aims to extract a kernel image and/or compressed-encrypted filesystem from a RTOS, QNX or Linux-based Firmware image. A number of heuristics are included to avoid extraction of certain blacklisted file types, and to avoid unproductive extraction beyond certain breadth and depth limitations. This is achieved through our own enhanced version of Refirm Labs' Binwalk (during the Static Analysis phase) as well as our own Dynamic Extractor complementing the static data found.

Encrypted Firmware

You may come up against a firmware file which is encrypted in some way. First, you’ll want to get a sense of whether the whole file is encrypted, or if it’s just a chunk. If it is encrypted, you may also want to get a quick sense of how well it’s done.

Entropy

Calculating entropy is a really good way of getting a sense of how compressed or encrypted any given series of bytes is. High entropy = probably encrypted (or compressed). Low entropy = probably not. But, even given the numbers, it’s not always immediately obvious. We calculate the entropy during firmware extraction, using our own modified version of Refirm Labs’s Binwalk.

When working with binary blobs such as Firmware images, you’ll eventually encounter unknown data. Particularly with regards to Firmware, unknown data is usually either compressed or encrypted. Analysis of these two types of data is typically approached in very different manners, so it is useful to be able to distinguish one from the other.

The entropy of data can tell us a lot about the data’s contents. Encrypted data is typically a flat line with no variation, while compressed data will often have at least some variation.

Entropy graph of an AES Encrypted Firmware:

Entropy graph of a compressed Firmware:

However, there are a few additional tests that can be performed to quantify the randomness of data. The two found most useful are Chi square distribution and Monte Carlo Pi approximation. These tests can be used to measure the randomness of data and are more sensitive to deviations in randomness than a visual entropy analysis.

For example, here is a comparison of the same 24MB file after being put through the AES, 3DES, gzip and lzma algorithms:

Algorithm

Chi Square Distribution

Monte Carlo Pi Approximation Error

Algorithm

Chi Square Distribution

Monte Carlo Pi Approximation Error

None

15048.60

.920%

AES

351.68

.022%

3DES

357.50

.029%

LZMA

253.54

.032%

Gzip

11814.28

.765%

As you can see, gzip has extreme differences between expected and observed data randomness, making it easy to identify. LZMA is much closer to the AES and 3DES encryption results, but still shows significant variations, particularly on the Chi Square distribution.

Using these tests, we can usually determine if an unknown block of data is encrypted or compressed and proceed with any further analysis accordingly.

The simplest method to decrypt the firmware is to look for the decryption routine within the firmware. If the router can decrypt the new firmware for updates, the decryption routine must be located in the old firmware image somewhere. If Firmware Reviewer meets an encrypted firmware, it goes to look for archived versions of the firmware, download all old versions and start poking around. A configuration panel provides the set of URL (can a FTP server), user/password (if any) for accessing the vendor website/ftp.

Below are three common firmware release scenarios:

Scenario 1

The device firmware was not encrypted nor did it contain any decryption routine when it was factory released. A decryption routine is shipped along with an unencrypted version of the firmware in a newer version (v1.1) for future encrypted firmware update. Subsequent firmware releases are encrypted.

In this scenario, we can obtain the decryption routine from firmware v1.1 and use it to decrypt the latest firmware version 1.2.

Scenario 2

The device firmware is encrypted in the original release. The vendor decided to change the encryption scheme and release an unencrypted transition version v1.2 which contains the new decryption routine.

Similar to scenario 1, we can obtain the decryption routine from v1.2 image and apply it to the latest encrypted firmware. Reading the release notes of the firmware releases could be helpful in identifying the unencrypted transition version. The release notes will usually direct the user to upgrade to an intermediate version before upgrading to the latest version. The intermediate version is very likely to be the unencrypted transition firmware.

Scenario 3

The device firmware is encrypted in the original release. However, the vendor decided to change the encryption scheme, and release an unencrypted transition version which contains the new decryption routine.

Consumer firmware are often limited in computing power. This limitation rules out slower, harder-to-crack asymmetrical encryption such as RSA. Also, vendors sometimes use the same encryption scheme for multiple routers. Look for routers that are in the same product line, with same processor architecture. Look around there and you might find the right decryption executable. In case of firmware related to Enteprise products, there is no easy method to obtain the decryption routine, other than using the Bootloader Agent.

Full system emulation

Sometimes, we’ll need to analyze the firmware more comprehensively and will benefit from full system emulation. There are many ways to fully emulate a device.

In the first part of the emulation process, Firmware Reviewer will use QEMU with a pre-built RTOS, QNX or Linux virtual machine simulating the target architecture with the same kernel level. Then transfer the firmware root file system into the VM and chroot into the root file system of the firmware, obtaining a working shell. 

With a working shell, Firmware Reviewer will automatically navigate to /etc/rc.d or /etc/init.d (or equivalent configurations files) and run the appropriate RC script to kick off the userland services. Closely analyze the rc.d folder (or equivalent) and inspect the scripts, tweak the startup scripts to account for missing network interfaces, failing of NVRAM library call. This part of the emulation process is very much like dealing with encrypted firmware; each firmware will be a case of its own which is the very definition of research. In most cases Firmware Reviewer automatically choose to tweak the rcS scripts just enough to get the target service to run properly. This part of the process, if done manually, could take up weeks of investigation and additional work. Firmware Reviewer will do the job itself, supporting both MIPS and ARM as well as x86 processors. It will extract the root file system, infer network interfaces, and create the QEMU disk image for emulation. It also will attempt to emulate the NVRAM.

Applied Tests

Anyway, using Dynamic Analysis, a lot of information can be extracted even in case of encrypted images or encrypted file systems, without having the related AES or 3DES keys, like:

Signature Information

This features uses an enhanced version of Refirm Labs' Binwalk, modified by our techinicians. The main feature of Binwalk is its signature scanning. Binwalk can scan a firmware image to search for different embedded file types and file systems.

You know the file command line utility, right? The file command will look at the header of the file and search for a signature (magic number) to identify the type of the file. For example, if the file starts with the sequence of bytes 0x89 0x50 0x4E 0x47 0x0D 0x0A 0x1A 0x0A, it knows it’s a PNG file. Check this Wikipedia page for a list of common file signatures.

Binwalk works the same way. But instead of looking for signatures just at the beginning of the file, Binwalk will scan the entire file. In addition, our own version of Binwalk is able to extract the files found in the image, in a bunch of binary formats, for example CISCO IOS XE custom compression format.

Both file and binwalk tools use the libmagic library to identify file signatures. But our modified Binwalk additionally supports a list of custom magic signatures to find compressed/archived files, firmware headers, Linux kernels, bootloaders, filesystems, and so on.

Exploit Mitigation

Firmware Reviewer detects CPU Architecture and elements with unprotected features., i.e. not hardened.

Linux, RTOS and QNX use the Executable and Linkable Format (ELF) as a common standard for executables (binaries), object code, shared libraries, and even core dump. ELF is flexible, extendable and cross-platform.

One mechanism to harden ELF binaries is to use Position Independent Executables (PIE) that are an output of the hardened package build process. A PIE binary and all of its dependencies are loaded into randomized locations within virtual memory each time the application is executed. This makes Return Oriented Programming (ROP) attacks much more difficult to execute reliably.

Another way is NX (no-execute) bit, a technology used in CPUs to segregate areas of memory for use by either storage of processor instructions (code) or for storage of data, a feature normally only found in Harvard architecture processors. However, the NX bit is being increasingly used in conventional von Neumann architecture processors for security reasons. An operating system with support for the NX bit may mark certain areas of memory as non-executable. The processor will then refuse to execute any code residing in these areas of memory. 

Further, there are ASLR and CANARY. ASLR: basically randomizes the base of the libraries (libc) so that we can’t know the memory address of functions of the libc. The ASLR avoids the technique Ret2libc and forces us to have to leak addresses of the same in order to calculate base. CANARY: Normally, a random value is generated at program initialization, and inserted at the end of the high risk area where the stack overflows, at the end of the function, it is checked whether the canary value has been modified.

Finally, a ELF file can be Stripped. The primary benefit, for the vast majority of cases, is just to save disk space. A secondary, and much more dubious benefit, is that it also makes the binary more difficult to disassemble or reverse engineer. It may also reduce the memory footprint a little, though in many cases it'll be a negligible savings. The biggest detriment is that it makes debugging significantly more difficult if you run into a problem. On a modern system with CPU speeds what they are, and memory/disk quantities what they are, stripping a binary will have very little practical impact on performance in any way. It's really about debugging, "cleanliness", and personal preferences. Some people prefer to always used stripped binaries for production or "shipped" software. Some people prefer to always leave the symbols included, "just in case".

Several techniques exist for hardening ELF binaries in RTOS, QNX and Linux. Firmware Reviewer is able to detect them all and measure their level of hardening.

Debugging Information

Discover debugging information, like: Left developer’s configuration files, SCM archives (GIT, SVN, CVS, etc.), Debuggable executables and Libraries. Debugging information in beta versions of IoT devices equips developers with internal systems knowledge of a device. Unfortunately, debugging systems are often left in production devices, giving hackers access to the same inside knowledge of a device.

Backdoors

Finds backdoors based in: Suspicious open TCP ports, suspicious connection to external IPs and URIs, presence of Non-standard services and Suspicious executables. When it comes to firmware, hidden backdoors are a favorite hacker exploit. Backdoors are intentional vulnerabilities that are planted into an embedded device to provide remote access to anyone with the "secret" authentication information. Although backdoors are potentially helpful for customer support, when they're discovered by malicious actors, they can have severe consequences. And hackers are great at finding them.

High-Risk executables

Firmware Reviewer detects exposed vulnerable standard library functions, like strcpy, sprintf, strcat, strncat, memcpy, memmove, gets, getw, scanf, system, etc, by analyzing all binaries existing in the firmware. They can expose it to Buffer Overflow, Heap Overflow and Stack Overflow, DoS, Comman Injection and Integer Overflow vulnerabilities, as described by CISA CERT coding practices and CERT C Secure Coding Standard. They can be included both in system components and application components. In case of system components it is suggested to update to a non-vulnerable version, while in case of vulnerable application components it is suggested to use safe functions, like: strcpy_s, sprintf_s, etc. The vulnerabilities are classified using CVE standard and CVSS scoring. Firmware Reviewer immediately identifies this binary as a High-risk executable, and viewing the details we see the code analysis has identified some insecure function calls inside of unsafe_functions as potential vulnerabilities. 

As their names imply, safe_functions plugin handles the user supplied data in a safe manner, while unsafe_function plugin handles user supplied data in an unsafe manner.

Although simplistic, these potential vulnerabilities are indicative of the bugs found in the vast majority of embedded devices.

Software Bill-Of-Materials (BOM)

Third-party components on which the analyzed Firmware depends on, with known vulnerabilities (Security Issues) and legal issues (License Analysis) are included in a BOM.

CVE Vulnerabilities

Firmware Reviewer detects components, libraries or executables with known vulnerabilities classified by CVE (Common Vulnerability and Exposure). CVE is a list of entries—each containing an identification number, a description, and at least one public reference—for publicly known cybersecurity vulnerabilities. Firmware Reviewer extracts every day such vulnerabilities regarding Firmware, to be used during Firmware Analysis.

Hardening Compliance

Other tools typically use the same data files to perform tests. Instead, Firmware Reviewer is not limited to a specific OS distribution, therefore it uses the knowledge of 10+ years from a wide range of sources. It may help you to automate or test against security best practices from sources like:

This is what happens during a typical Hardening Compliance scan:

  1. Initialization

  2. Perform basic checks, such as file ownership

  3. Determine operating system and tools

  4. Search for available software components

  5. Run enabled plugins

  6. Run security tests per category

  7. Report status of security scan

Other Tests

Test

Description

Malware Analysis

Dynamic analysis of ELF files:

  1. Starting and Termination: Time Stamps and Elapsed Time.

  2. Processes Information: clone, execve and exit etc.

  3. File I/O: open, read, write and delete etc.

  4. Network: TCP, UDP, HTTP and HTTPS etc.

  5. Typical Malicious Actions: self deletion, modification and lock.

  6. API Information: getpid, system, dup and other libc functions.

  7. syscall sequences.

Default Credentials

Finds default vendor’s and telco operators' credentials, using a monthly-updated list. The users rarely change those default credentials

Hardened Credentials

Finds hardened users and password stored in the firmware’s file system, as well as decrypting passwd and shadow files

Password Policy

Detects weak password policies, that increases the probability of an attacker having success using brute force and dictionary attacks against user accounts. An attacker who can determine user passwords can take over a user's account and potentially access sensitive data in the application. Weak passwords can be easily guessed and are an easy target for brute force attacks. This can lead to an authentication system failure and compromise system security

Password Expiration

There are different ways to detect the password expiration of a user. For example, one can easily check the user account password expiry information on Linux. The /etc/shadow files stores actual password in encrypted format for user’s account. You need to use the chage command. It can display password expiry information as well as changes the number of days between password changes and the date of the last password change. This information is used by the system to determine when a user must change his/her password

Password Recovery

It detects a wrong Password Recovery process inside the firmware

Account Lockout

It detects wrong Account Lockout policies

Role Separation

It checks the effectiveness of Role separation for different users

Role Hardening

It checks the effectiveness of Role hardeningfor different users

Check Authentication

Unauthenticated access: One of the most common vulnerabilities in firmware, unauthenticated access allows threat actors to gain access to an IoT device, which makes it easy to exploit device data and any controls provided by it. 

Weak authentication: Threat actors can easily gain access to devices when the firmware has a weak authentication mechanism. These mechanisms can range from single-factor and password-based authentication to systems based on weak cryptographic algorithms that can be broken into with brute-force attacks.

Upgrade Feature

It checks weaknesses during the firmware upgrade process

Exploitable Vulnerabilities

Tests for the presence of 60 known vulnerabilities using exploits from an embedded version of Metasploit. In addition, it also checks for a number of previously-unknown vulnerabilities discovered by our lab

Encrypted Upgrade Protocol

It checks a weak encryption algorithm during the firmware upgrade process

Upgrade Validation

It checks weaknesses during the firmware upgrade process, with focus on validity and integrity using a digital signature

Upgrade Rollback

It checks weaknesses during the firmware upgrade rollback process

Upgrade Security Alerting

It checks the presence of an effective security notification system during the firmware upgrade process

Sensitive Data

Dynamically checks the visibility of sensitive data

Sensitive Data Security

Dynamically checks the protection of sensitive data using ‘at rest’ cyphering or 'in transit'

Anonymized Personal Data

Dynamically checks the efficiency of anonimization of personal data (personal name, phone numbers, e-mails, etc.)

Data Collecting

Dynamically checks if Data Collecting is correctly notified to the user

Unencrypted Protocols

Dynamically checks the usage of Unencrypted Protocols

Traffic Anomalies

Dynamically checks anomalies in communications with unknown public IPs or URI, tracking their geo-location and the cipher level

Local Firewall

Dynamically checks the efficiency of the local firewall feature provided by the firmware

Security Event Logging

Dynamically checks the presence of separated Event log regarding security issues only

Security Event Alerting

Dynamically checks the presence of separated Event alerting system regarding security issues only

Binary Fuzzing Analysis

Dynamically executes a binary fuzzing analysis on executables/libraries exposing vulnerable functions

Network Fuzzing Analysis

Dynamically executes a network fuzzing analysis on exposed services

Directory Traversal and Discovery

Dynamically executes directory traversal tests and content discovery to Web pages, to identify debug or test functionalities

Wrong Input Validation

Dynamically executes web interface and API tests to find vulnerabilities like: weak authentication/authorization management, SQL Injection, XSS, XXE, CSRF, directory traversal, remote code execution

Wrong Error Handling

Dynamically executes a fuzzy analysis on executables parameters to observe the effectiveness of exception handling and stack trace

Bootloader Analysis

Dynamically checks if bootloader sequence modifications are permitted, as well as arbitrary kernel loading and boot from untrusted sources

Firmware Integrity Testing

Dynamically attempts to load a custom firmware or modified binaries to check if integrity testing is provided by the firmware upgrade process

Runtime Analysis

Dynamically attempts to switch in debugging mode for altering a process at runtime

Binary Exploitation

Dymanically exploit attempts in order to execute an arbitrary binary execution

Firmware Complexity

The Firmware of modern IoT devices is complex and made of multiple components. These components can take the form of either different binaries, packaged in an embedded QNX or Linux distribution, or different modules, compiled into a large, single-binary embedded OS (“blob firmware”).

The former type of firmware is, by far, the most ubiquitous: a large-scale experiment analyzed 19K firmware samples, and found that 86% of them were Linux-based. Similar to other Linux-based systems, Linux-based firmware includes a large number of interdependent binaries. The different binaries (or components) of the firmware on embedded devices share data to carry out the device’s tasks. Under our attacker model, this interaction is critical, as we focus on bugs that can be triggered by attacker input from “outside” of the device (i.e., over the network), but may affect binaries other than those directly facing the network.

Firmware Attacks

IoT devices exchange data over the network. This data can come directly from the user (e.g., through a web interface), or indirectly from a trusted remote service (e.g., cloud backends). Many devices, especially routers, smart meters, and a host of low-power devices, such as smart light bulbs and locks, use the former paradigm. Moreover, recent attacks have shown that such devices can be exploited by clever remote attackers, even when their communication is restricted to a closed local network. In this work, we consider builind or own network-based attack tasks who communicate directly with the device, either through a local network or the Internet. However, our Embedded Framework can be easily extended to other scenarios.

We first identify the set of binaries that export network services (i.e., network-facing binaries) in a, even encrypted, Firmware sample. We leverage the observation that network-facing binaries are the components of the Firmware sample that receive and parse user-provided data.

Therefore, we identify those binaries within the Firmware sample that parse data read from a network socket. We utilize three features to identify functions in embedded systems that implement parsers:

  • the number of basic blocks (#bb)

  • the number of branches (e.g., if-then-else, loops) (#br)

  • the number of conditional statements used in conjunction with memory comparisons (#cmp).

Since we want to specifically identify input-affected network parsers, we consider two additional features:

  • a metric we call network mark (#net)

  • a flag we call connection mark (#conn).

The network mark feature encodes the probability that a parsing function handles network messages, and it is calculated by identifying every memory comparison in the code of the function, and comparing the referenced memory locations against a preset list of network-encoding strings (e.g., soap or HTTP). We initialize #net to 0 and increment it for every comparison against network-encoding strings present in the code. The connection mark flag, instead, indicates if any data read from a network socket is used in a memory comparison. We initialize #conn to 0 and set it to 1 if there exists a data-flow between a socket read and a memory comparison operation. We combine the aforementioned five features to compute the parsing score psb of a binary b as follows:

(1) where each constant ki is set to maximize the parsing detection capabilities (kbb = 0.5, kbr = 0.4, kcmp = 0.7), whereas kn and kc promote functions that refer to network-encoding keywords and binaries that parse network data, respectively. The optimal values for the last two constants are found empirically. Finally, psj is the parsing score of the j-th function of b. Note that, we introduce our two features as multipliers in order to highlight input-affected network parsers. Since all binaries are likely to have a score greater than zero, we need to distinguish and separate the “most significant” scores. To this end, we leverage the DBSCAN density-based clustering algorithm, which groups binaries whose scores are closely packed together. Then, we select the cluster that contains the binary having the highest parsing score in the Firmware sample, and consider all the binaries belonging to the cluster as the initial set of network-facing binaries. Finally, the algorithm implemented by this module returns the unpacked Firmware sample, the set of identified network-facing binaries, and the program locations containing memory comparisons against network-encoding keywords. These memory comparisons represent the program locations where task attacker-controlled data is more likely to be referenced.

DISCLAIMER: Firmware Reviewer never operates on physical devices.

COPYRIGHT (C) 2014-2021 SECURITY REVIEWER SRL. ALL RIGHTS RESERVED.