HackRF Shell FM Receiver: Real Audio Output!

It has been a few months since the last report, but I have still been working on my Scheme FM receiver project here and there. With toddlers about, and other responsibilities, it can be hard to get in more than 30 minutes of work on it a week. Also, they are fond of pressing the computer reset button in the moments of my most intense concentration.

Several weeks were spent trying to figure out how to improve program performance, enough that I could actuallt process the data in real time. The main key there was understanding how to properly arrange the processing and the decimation, to reduce calculations required. But I learned a lot in the process about efficient use of Guile Scheme and C bindings — mainly figuring out different ways to reduce function calls, type checks, and type conversions.

After quite a lot of work creating bindings, I got the bandpass filter and the fm demodulation into the processing chain. However, the data being spit out after the demodulation was all garbage, though with some patterns here and there. It took many weeks of bug hunting and studying signal theory, to root out the errors, most of which were mishandling of data pointers, though there was also some fuzzyiness in my understanding of convolution, FIR filters, and FM demodulation that I had to work through.

I got through that, and was finally able to get some data out of the receiver that sounded like real FM radio sounds. However, the desired sounds were buried deeply in distortion and noise. With some further study, I realized that I needed another filter after the FM demodulation. That seems to be what they do in the GNU Radio WBFM block. Also, I learned that in real FM broadcast radio signals, there is a stereo channel (or rather, the L-R channel) inserted as side-bands around the mono channel (or rather, the L+R channel) so you have to filter that out (or do the additional stereo processing). I inserted another filter, in the process learning a bit about C macro procedures.

After this, I was able to receive understandable FM broadcast audio, from a local country/western station. However, the audio still has a noticeable amount of static. Adjusting the filter values does affect that, so I am in the process of trying out different values to see what works best. I think I also need to do something to reverse the high-frequency “pre-emphasis” which is used in FM broadcasting.

HackRF Shell: Fun with filters

I had added bindings earlier for pulling data from the HackRF, and then more recently a binding to liquid-dsp frequency demodulation. But in between the two, filtering will be needed. So, I added a binding for the firdespm_lowpass function from liquid-dsp. This function gives you the coefficients for a Finite Impulse Response (FIR) filter, using the Parks-McClellan algorithm.

I wanted to see something interesting from that before going to bed, so I threw together a scheme procedure to generate and then print out the coefficients:

;; -*- geiser-scheme-implementation: guile -*-

;; Copyright 2020 Christopher Howard

;; This program is free software: you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <https://www.gnu.org/licenses/>.

(define-module (sdr filters)
  #:export (print-firdespm-lowpass-coefficients))

(use-modules (rnrs bytevectors)
             (sdr liquid-dsp))

(define (print-firdespm-lowpass-coefficients filter-len
                                             cutoff-freq
                                             stop-band-atten
                                             fract-samp-offset)
  (let ((output-bv (make-bytevector (* 4 filter-len))))
    (firdespm-lowpass filter-len
                      cutoff-freq
                      stop-band-atten
                      fract-samp-offset
                      output-bv)
    (let lp ((coeff 0))
      (when (< coeff filter-len)
        (display
         (number->string
          (bytevector-ieee-single-ref output-bv (* 4 coeff) (native-endianness))))
        (display "\n")
        (lp (1+ coeff))))))

Then I dumped some coefficients to a file:

(with-output-to-file "out.csv"
  (lambda () (print-firdespm-lowpass-coefficients 201 0.2 60.0 0)))

Plotting out the coefficients, I got this visual:

I don’t really know enough yet to be sure if those are sane results or not, but it looks pretty. I was hoping to pick up a DSP book from the library sometime this week and dive more into the details of FIR filters.

HackRF Shell Update, Explanations

What is the (current) goal of the HackRF Shell project?

I aim to provide a free software program which sets up a Guile Scheme shell, allowing lisp-style control of the HackRF Software Defined Radio (SDR). The shell will provide procedures for (1) controlling the HackRF, including pulling from and pushing data to the radio, (2) Digital Signal Processing (DSP) e.g., frequency modulation, and (3) some i/o interfaces, specifically to the file system and to PulseAudio.

Why does this project exist?

Partly to help me learn about SDR and DSP. Also, I want to be able to code applications for HackRF SDR in Scheme, rather than in Python (Gnu Radio).

Wouldn’t it be better to use Guile FFI than embedding Guile in a C program?

Erm, well, maybe. But the embedded approach seems easy to understand, develop, and troubleshoot.

Shouldn’t you make something modular to work with any SDR?

I wanted to focus on getting somewhere practical with the SDR I have and the code I already wrote. But I believe it would be possible to expand the shell by adding or swapping in osmocom bindings instead of just the libhackrf bindings.

What have you done so far?

  • Created the C program with libhackrf bindings
  • Set up autotools files for development
  • Created Guix package definitions for HackRF Shell as well as the hackrf and liquid-dsp libraries.

What are you working on now?

Adding liquid-dsp bindings.

Where is the code?

git clone git://git.librehacker.com/pub/git/hackrf-rkt.git

Why has it taken you so long to make progress on this project?

  • Toddlers
  • Moving to a new apartment.
  • A job with no PTO.

What makes Guix such an awesome development environment?

Once you’ve made package definitions, which isn’t very hard, you can set up your development environment (exposed packages and environment variables) quite easily. Right now, for HackRF I simple cd into my source code and run

guix environment hackrfshell --ad-hoc liquid-dsp

The I run ./configure, make, etc. In the next iteration of the hackrfshell package definition, I will add liquid-dsp as a dependency, so then it would just be guix environment hackrfshell.

Not all my package definitions have made it into the official Guix git repository yet, but I’ve added them to my local Guix channel for the same effect.

Also because of Guix’s functional package management approach, I can use any verions of libhackrf, etc. I want, and not have to worry about wrestling with base system dependencies.

HackRF Shell Guix Definition

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2019 Christopher Howard <christopher@librehacker.com>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (gnu packages hackrfshell)
  #:use-module (guix)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (guix packages)
  #:use-module (guix utils)
  #:use-module (guix download)
  #:use-module (guix git-download)
  #:use-module (guix build-system gnu)
  #:use-module (gnu packages)
  #:use-module (gnu packages algebra)
  #:use-module (gnu packages glib)
  #:use-module (gnu packages guile)
  #:use-module (gnu packages hackrf)
  #:use-module (gnu packages pkg-config))

(define-public hackrfshell
  (package
    (name "hackrfshell")
    (version "0.0.1")
    (source
     (origin (method url-fetch)
             (uri (string-append
                   "ftp://git.librehacker.com/hackrfshell/hackrfshell-"
                   version ".tar.gz"))
             (file-name (string-append "hackrfshell-" version "-tar.gz"))
             (sha256
              (base32 "0iqnd1jbhq5wjrp30d9i0yy9jqshyy00nzsmh78dnxp1yjm2wppa"))))
    (build-system gnu-build-system)
    (native-inputs
     `(("pkg-config" ,pkg-config)))
    (inputs
     `(("hackrf" ,hackrf)
       ("glib" ,glib)
       ("guile-2.2" ,guile-2.2)
       ("fftwf", fftwf)))
    (home-page "https://librehacker.com")
    (synopsis
     "Program which loads HackRF control procedures into a Scheme shell")
    (description
     "This is a C program loads up a Guile Scheme shell with primitive
HackRF control functions, as well as some other useful core functions
for Software Defined Radio (SDR) data processing.  This is something
that maybe could be done with a FFI interface, but this approach works
also and arguably has one or two advantages.")
    (license license:gpl3+)))

HackRF Shell: (Learning) Delays

I wanted to move on quickly from completing the FFT interface, on to FM demodulation. However, I realized that if the rest of the project was going to practical and modular, I would need some kind of block style I/O buffer management system, like GnuRadio has. And as I started to pursue that, I realized that I would need to properly modularize my code.

Up until now, all the functions were either being dumped into the default module by the C code, or loaded from one Scheme file in the source directory. I took all the external C code and organized it in sensible module divisions, and then took all the C-based Scheme primitives and put them in two modules ((sdr primitives) and (sdr hackrf primitives)). Those two modules are created and loaded when HackRF Shell is launched, while the other modules can be compiled independently.

That all required taking the time to actually learn how the module system works, as well as understanding how to create and fill-out modules through the C interfaces. So, not much amazing progress to the outside observer, but I learned a lot. The FFT demo function is still working:

One neat thing about programming in Guile is that you can jump into various modules at any time, to play around in them or see what variables are defined in them (private and public):

christopher@nightshade:~/Repos/hackrf-shell$ ./hackrf-shell 
HackRF Shell, copyright 2019 Christopher Howard

Initializing primitive functions.
Entering the Scheme shell.

GNU Guile 2.2.3
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (set! %load-path (cons "/home/christopher.qlfiles.net/Repos/hackrf-shell" %load-path))
scheme@(guile-user)> ,m (sdr hackrf util)
scheme@(sdr hackrf util)> ,b
hackrf-sensible-defaults #<variable 56314e6aaac0 value: #<procedure hackrf-sensible-defaults (d)>>
filename                #<variable 56314e6aa940 value: #<undefined>>
timed-read              #<variable 56314e6aa8a0 value: #<procedure timed-read (d file-path start-h start-m end-h end-m)>>
hackrf-cb-rx-to-file    #<variable 56314e6aa990 value: #<procedure hackrf-cb-rx-to-file (filename)>>
out                     #<variable 56314e6aa9e0 value: #<undefined>>
when-hour-min           #<variable 56314e6bd660 value: #<procedure when-hour-min (h m l)>>
hackrf-cb-rx-to-stream  #<variable 56314e6aaa30 value: #<procedure hackrf-cb-rx-to-stream (out)>>
cb-fft-to-gnuplot       #<variable 56314e6aa8f0 value: #<procedure cb-fft-to-gnuplot (center-freq samp-rate averaging-alpha)>>
scheme@(sdr hackrf util)> hackrf-
hackrf-cb-rx-to-file                  hackrf-open                           hackrf-set-sample-rate
hackrf-cb-rx-to-stream                hackrf-sensible-defaults              hackrf-set-vga-gain
hackrf-close                          hackrf-set-baseband-filter-bandwidth  hackrf-start-rx
hackrf-disable-amp                    hackrf-set-freq                       hackrf-stop-rx
hackrf-enable-amp                     hackrf-set-lna-gain                   
scheme@(sdr hackrf util)> ,m (sdr util)
scheme@(sdr util)> ,b
bv-complex-to-mag       #<variable 56314e6b4700 value: #<procedure bv-complex-to-mag (complex-float-bv n_complex_nums dest-bv)>>
freq-shifter            #<variable 56314e6b4670 value: #<procedure freq-shifter (freq samprate)>>

The next step in my project will be to work out my block-style I/O buffer management system, which will require me to re-learn GOOPS, Guile’s object oriented programming extension, so that will likely still take a while.

git clone git://git.librehacker.com/pub/git/hackrf-rkt.git