Understanding and characterizing the Radio Frequency (RF) spectrum can be a challenging task, particularly with the projected ubiquity of wireless devices and protocols in the Internet of Things (IoT). Expert signal processing systems are hand-tuned to specific signal types, and therefore are not expected to scale to the number of devices expected in IoT populations. As the RF spectrum becomes a scarce resource, intelligent spectral monitoring will be a critical component for many RF applications. This domain has only begun to see the application of machine learning (ML) and deep learning (DL), partially due to the unique challenges specific to the RF modality. In this blog, we dive in to a number of different spectral monitoring use cases where ML/DL can play a key role. Along the way, we will highlight some of our technology in this arena.
A cognitive radio has the ability to analyze it's local RF environment and select a channel that minimizes interference and/or maximizes throughput. Intelligent spectral monitoring can simply be thought of as an extension of the spectrum sensing task in cognitive radio. Cognitive radios may also use spectrum sensing to change output parameters, such as power levels and even modulation schemes. The idea for cognitive radio was first developed by Joe Mitola at DARPA in 1998, and has since burgeoned into an active field of research and development in wireless communications. Implementing technology to intelligently sense the RF spectrum will be imperative to conduct successful operations in smart cities and the IoT.
Spectrum sensing can be accomplished in a number of different ways, ranging from simple to complex. A simple approach that is used by a number of RF devices is to perform energy detection by computing the power at a given frequency band. For example, if a device (e.g. WiFi transceiver) wanted to find an optimal (least amount of interference) channel for transmission, it might sense the spectrum at each channel periodically to find the one with the least amount of active energy present. This can be expanded to a more sophisticated approach by periodically estimating the power of the noise floor, and then using detection theory to make a more robust statistical judgment (e.g. N-sigma detection in the presence of additive white Gaussian noise) about the presence of a signal or interferer. Advanced spectrum sensing techniques may employ a number of algorithms to not only detect the presence of signals, but also identify them. Signals are typically associated to a specific class of RF devices by identifying the protocol and/or modulation scheme.
We can also use DL to tackle the spectrum sensing problem! Figure 1 above shows different bands of the RF spectrum being classified by a deep neural network (DNN) trained on spectrograms, dubbed SpecNet. This network is similar in architecture to KickView's Deep Spectral Detector, but was implemented in TensorFlow, and runs in real-time on the NVIDIA Jetson TX-2, an edge-computing platform. SpecNet was trained with time and frequency uncertainties purposefully introduced in order to generalize to real-world conditions. We also varied the gain settings on the radio during training so that SpecNet is robust to varying Signal-to-Noise Ratio (SNR). Our system has the ability to periodically collect new data and update the model in order to adapt to dynamic RF environments. It's important to note that this application of DL to the RF domain is both powerful and straightforward, and has a number of advantages over conventional spectrum sensing methods.
Imagine a state-of-the-art security system that is monitoring a sensitive facility, equipped with both cameras and RF sensors. The system needs to perform two basic tasks: 1) characterize normal behavior, and 2) flag anomalous activity. If one camera is pointed at the entrance roads and tracking cars, it might use DL algorithms to monitor driving patterns and flag cars that are driving erratically or significantly above the speed limit. Similarly, an RF sensor might perform spectral monitoring to characterize the signals in the environment and flag new signals. While this is a specific use-case, the concepts behind anomaly detection are important in many industries.
Spectral monitoring can be an essential tool for identifying anomalies within complex RF systems, particularly in the telecommunications and wireless industries. For example, cable providers scale their service to a massive number of users through a hybrid fiber coax (HFC) network consisting of core network components such as cable modem termination systems (called CMTS), hubs, nodes, optical-to-cable interfaces, and more. A cable network (also called a cable "plant") can experience an array of impairments that are dynamic in nature. Impairments caused by external signal ingress from motors, appliances, power lines, and LTE, can be difficult to track down. When quality of service degrades or an outage occurs, it can often become very difficult or even intractable to find the root cause of the issue because of the dynamic factors and limited measurements representing the total system. This problem can be viewed as a variant of anomaly detection: we want to detect when a problem in the system occurs, and perform some introspection to find out the root cause. While these systems have maintenance staff and operations teams to keep these systems up and running, they are often forced to rely on logs generated by various parts of the system, without having an understanding of the dynamic behavior and anomalies present at the physical layer. ML/DL based anomoly detection is a powerful tool to use for this use case since it provides both automation and the capability to learn highly complex configurations.
At KickView, we solve these types of problems by attacking the raw RF data directly. Figure 2 above shows some sample RF collections from a cable system, including a 256-QAM constellation generated by demodulating the MPEG payload from one of the 6 MHz carriers. Generally, anomalies can manifest in the RF data in a number of different ways, and are dependent on the medium of transmission (channel), the payload to be transmitted, and the specifics of the network. Examples of observable features in the RF data from anomalous behavior are shown below in Figure 3, and include:
- Frequency offset (and subsequently the phase shift) as a function of time
- Time epoch of each symbol (change in symbol rate)
- Amplitude of the waveform (properties of unique analog components)
- Non-linear effects (power amplifier characteristics)
Our RF deep learning solutions are directly informed by our signal processing domain knowledge, so we can robustly detect and identify these sorts of effects in the raw waveform. It becomes quite a challenging problem to solve in the real world because these effects often couple together in a non-linear fashion. In many cases, we often utilize various signal processing operations to pre-process the data so that the salient features of interest are maximized in the new representation.
Specific Emitter Identification
Lastly, we'll touch on how intelligent spectral monitoring ties into the specific emitter identification (SEI) problem. In spectrum sensing, the primary goal is to identify unoccupied frequency bands. In SEI, we wish to detect and identify a specific signal of interest, often in harsh channel conditions and low SNR environments. These challenging conditions motivate the need for sophisticated approaches which combine elements of signal processing and DL. Below, we highlight some ongoing work with a customer to detect 802.11g WiFi signals at low SNR using an innovative combination of signal processing and DL.
IEEE 802.11g, which is now a slightly outdated WiFi standard, uses Orthogonal Frequency Division Multiplexing (OFDM) as its modulation scheme at the physical layer. A deep dive into OFDM could be a lengthy blog in and of itself, so we won't dig into the specifics here. Suffice to say, OFDM has become a very popular form of wideband digital communication because it is very robust to severe channel conditions which commonly arise in wireless and mobile links. In fact, even the latest WiFi standards such as 802.11ac still utilize OFDM, but at even wider bandwidths. See Figure 4 below for more details as to how a digital communication system using OFDM is built.
For detection, we can focus on beacon frames, which are transmitted from every WiFi access point approximately every 100 msec. Beacon frames contain information about the network hotspot, including the SSID. Since nearly every hotspot broadcasts beacon frames at regular intervals, these types of frames are great candidates for building a detector against. Put another way, if we can develop a system that can robustly detect these beacon frames, we can be sure that WiFi is present in that area. To first created a very large training dataset utilizing our suite of ML tools developed specifically for generating labelled RF datasets. An example snippet from data collected is shown below in Figure 5.
One thing to understand about the RF modality is that the dimensionality of the data is much higher than an image. To think of it another way, there are many possible trajectories that a waveform can traverse through in the time-domain. Therefore, we came up with a pre-processing strategy to maximize the features in each beacon frame while also reducing the dimensionality presenst in the data. We exploited our knoweldge of the subcarrier structure in OFDM, and channelized each beacon frame snapshot to generate complex signal images with both time and frequency. In a sense, we're employing the channelization procedure to explicitly reveal structure inherently present within the OFDM signal.
During the channelization process, we also iterate over different combinations of sample phase in time and frequency. We do this because we want our system to generalize over these parameters, especially because we can't correct for these errors in a real-world collection. Finally, we also vary the noise levels of the snapshots. This entire process can be thought of as a form of data augmentation. We end up generating ~300,000 examples from every single beacon frame snapshot, which is a significant increase! An example OFDM complex signal image is shown above in Figure 6 with varying sample phase and Doppler. Notice how the structure of the sub-carriers changes as these parameters are modified.
We trained a custom deep network called RFNet on these complex signal images. Using this approach, we've been able to detect 802.11g WiFi beacon frames at negative SNRs, even to the point where the signal is no longer visible to the human eye. Pretty cool! We have extended this technique successfully to many other signal and modulation types.
Hopefully this blog gave you a better idea of how intelligent spectral monitoring can be applied to a number of different use-cases in the RF domain. We've shown that combining signal processing and machine learning in innovative ways can yield some truly remarkable results that have direct applications to real-world problems. We're always excited to share snippets of cool technology that we're creating, and would love to hear your feedback.