8sync Examples Part 1

I want to play around with the 8sync framework, but the project is unfortunately lacking an API reference. So, I am writing example code to help me learn each of the modules, and thought I might as well share. Here is some example code for the (8sync agenda) module.

(define-module (8sync agenda examples))

(use-modules (8sync agenda)
             (srfi srfi-26)
             (srfi srfi-31))

(export make-stuff)

(define (make-something thing amt)
  ((rec (l n)
        (when (positive? n)
          (display "Making ") (display thing) (display "\n")
          (8yield)
          (l (- n 1)))) amt))

(define (make-stuff)
  (define my-agenda
    (make-agenda
     #:queue (make-q* (cut make-something "widget" 10)
                      (cut make-something "thingamabob" 5)
                      (cut make-something "sprocket" 5))))
  (run-agenda my-agenda))

This illustrates how an Agenda can be used to interweave the execution of computations, by yielding to other computations during your loop:

scheme@(guile-user)> ,use (8sync agenda examples)
scheme@(guile-user)> (make-stuff)
Making widget
Making thingamabob
Making sprocket
Making widget
Making thingamabob
Making sprocket
Making widget
Making thingamabob
Making sprocket
Making widget
Making thingamabob
Making sprocket
Making widget
Making thingamabob
Making sprocket
Making widget
Making widget
Making widget
Making widget
Making widget
$2 = done

HackRF Shell: (Learning) Delays

I wanted to move on quickly from completing the FFT interface, on to FM demodulation. However, I realized that if the rest of the project was going to practical and modular, I would need some kind of block style I/O buffer management system, like GnuRadio has. And as I started to pursue that, I realized that I would need to properly modularize my code.

Up until now, all the functions were either being dumped into the default module by the C code, or loaded from one Scheme file in the source directory. I took all the external C code and organized it in sensible module divisions, and then took all the C-based Scheme primitives and put them in two modules ((sdr primitives) and (sdr hackrf primitives)). Those two modules are created and loaded when HackRF Shell is launched, while the other modules can be compiled independently.

That all required taking the time to actually learn how the module system works, as well as understanding how to create and fill-out modules through the C interfaces. So, not much amazing progress to the outside observer, but I learned a lot. The FFT demo function is still working:

One neat thing about programming in Guile is that you can jump into various modules at any time, to play around in them or see what variables are defined in them (private and public):

christopher@nightshade:~/Repos/hackrf-shell$ ./hackrf-shell 
HackRF Shell, copyright 2019 Christopher Howard

Initializing primitive functions.
Entering the Scheme shell.

GNU Guile 2.2.3
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (set! %load-path (cons "/home/christopher.qlfiles.net/Repos/hackrf-shell" %load-path))
scheme@(guile-user)> ,m (sdr hackrf util)
scheme@(sdr hackrf util)> ,b
hackrf-sensible-defaults #<variable 56314e6aaac0 value: #<procedure hackrf-sensible-defaults (d)>>
filename                #<variable 56314e6aa940 value: #<undefined>>
timed-read              #<variable 56314e6aa8a0 value: #<procedure timed-read (d file-path start-h start-m end-h end-m)>>
hackrf-cb-rx-to-file    #<variable 56314e6aa990 value: #<procedure hackrf-cb-rx-to-file (filename)>>
out                     #<variable 56314e6aa9e0 value: #<undefined>>
when-hour-min           #<variable 56314e6bd660 value: #<procedure when-hour-min (h m l)>>
hackrf-cb-rx-to-stream  #<variable 56314e6aaa30 value: #<procedure hackrf-cb-rx-to-stream (out)>>
cb-fft-to-gnuplot       #<variable 56314e6aa8f0 value: #<procedure cb-fft-to-gnuplot (center-freq samp-rate averaging-alpha)>>
scheme@(sdr hackrf util)> hackrf-
hackrf-cb-rx-to-file                  hackrf-open                           hackrf-set-sample-rate
hackrf-cb-rx-to-stream                hackrf-sensible-defaults              hackrf-set-vga-gain
hackrf-close                          hackrf-set-baseband-filter-bandwidth  hackrf-start-rx
hackrf-disable-amp                    hackrf-set-freq                       hackrf-stop-rx
hackrf-enable-amp                     hackrf-set-lna-gain                   
scheme@(sdr hackrf util)> ,m (sdr util)
scheme@(sdr util)> ,b
bv-complex-to-mag       #<variable 56314e6b4700 value: #<procedure bv-complex-to-mag (complex-float-bv n_complex_nums dest-bv)>>
freq-shifter            #<variable 56314e6b4670 value: #<procedure freq-shifter (freq samprate)>>

The next step in my project will be to work out my block-style I/O buffer management system, which will require me to re-learn GOOPS, Guile’s object oriented programming extension, so that will likely still take a while.

git clone git://git.librehacker.com/pub/git/hackrf-rkt.git

HackRF Shell: FFT Averaging and Display Progress

I’m not quite sure I have all the bugs worked out yet, but I implemented simple averaging, as well and the frequency display. The latter required implementing fft-shift to move center frequency to the center. The above plot shows FM stations at 101.1, 102.5, 103.9, and 104.7 Mhz, matching local stations I know about.

These simple things took quite a while, mainly because of the busyness of other life responsibilities; but also because the process of implementation led to fixing some bugs, which led to cleaning up some code, which led to learning new things about Guile scheme and C.

In Guile scheme, I learned how to implement a procedure to auto-close unreachable ports:

(define ports-guardian #f)

(define (close-guarded-ports)
  (let f ()
    (let ((p (ports-guardian)))
      (when p (begin (close-port p) (f))))))

(define (initialize-ports-guardian)
  (when (not ports-guardian)
    (set! ports-guardian (make-guardian))
    (add-hook! after-gc-hook (lambda () (close-guarded-ports)))))

(initialize-ports-guardian)

(define (register-autoclose p) (ports-guardian p) p)

register-autoclose can simply be wrapped around the port, e.g.:

(define (hackrf-cb-rx-to-file filename)
  (hackrf-cb-rx-to-stream
   (register-autoclose (open-output-file filename))))

This uses Guile scheme “guardian” functionality, which is an object that can return registered objects if the have become unreachable everywhere else.

git clone git://git.librehacker.com/pub/git/hackrf-rkt.git

Curse of War Tips

Curse of War, an Ncurses-based strategy game. I am the green population.

http://a-nikolaev.github.io/curseofwar/

Here are a few tips, some of which you might not learn unless you inspect the source code:

  • Population on a hex can only grow to a maximum of 499, at which point growth slows to zero. So, a full hex might look good but actually has stopped producing.
  • Subject to the caveat above, your hex population growth is a percentage of population already on the hex.
  • Migration from deep territory to the front is a slow process.
  • Because of the above three factors, you generally want to place new villages and towns at locations that are neither too close nor too far from the front of war. If they are too close, the bonus growth effect will be operating on hexes that have only a few troops that are passing through on their way to the front. But too far away, and the bonus will be applied to hexes that are saturated and cannot produce more population anyway.
  • Battle outcomes are basically just a function of which player has more troops on the hex. So, you generally don’t want to push forward your war front that part of the front is mostly full of saturated (499 troop) or nearly-saturated tiles. Pushing forward effectively generally means putting flags directly in front of your current line of flags and then removing the old ones. Then wait until those new hexes become saturated.
  • If you realized you have pushed your front forward too quickly and are getting overcome, you can usually save the line by just pulling the front back a little.
  • You may want to play with the -i 0 option to ensure basically fair starting balance between players.
  • It usually makes more sense, mathematically, to build a new village, than to upgrade to a town or castle. This is due to cost vs. growth percentage increase. Towns or castles don’t give any kind of actual defensive bonus, just the growth increase.
  • If you have a good grasp of the battle, migration, and population growth mechanics, you basically know what you need to know to win most games against the AI. However, a tricky part in some games is knowing how to manipulate the other players against each other, so that they are attacking each other more instead of you. If you have a strong, non-moving set of fronts, generally you AI will attack the other players who have weaker fronts. However, this can back-fire if you allow one opponent to become stronger than you, after slurping up weaker enemies. It is a careful balancing act that requires you to intelligently focus your resource development and front movement.
  • At first it is tempting to focus exclusively on capturing mines, but your overall front movement and population growth is generally the more important thing to focus on. Of course, a front-movement strategy that will eventually get you three more mines, certainly has some points going for it.

HackRF Shell: FFT Plot Progress

I threw together a scheme procedure which dumps FFT data to Gnuplot.

You see, however, that I have more work to do as properly labeling the graph, as well as implementing averaging for a more practical spectrum display. The 512 points on the x-axis correspond to the 512 complex numbers in the FFT output buffer, but center frequency in this case is actually 103Mhz, at 8 msps.

scheme@(guile-user)> (define d (hackrf-open))
scheme@(guile-user)> (hackrf-sensible-defaults d)
scheme@(guile-user)> (hackrf-set-freq d 103000000)
scheme@(guile-user)> (hackrf-set-baseband-filter-bandwidth d 2000000)
scheme@(guile-user)> (hackrf-enable-amp d)
scheme@(guile-user)> (define c (cb-fft-to-gnuplot))
scheme@(guile-user)> (hackrf-start-rx d c)
scheme@(guile-user)> (hackrf-stop-rx d)

It was mostly easy to write the scheme -> Gnuplot interface as that is just sending text commands to Gnuplot standard input. However, it took me a while to fix an elusive bug in my C FFT function.

HackRF Shell: RX Restart Bug

I spent quite a while troubleshooting a bug in which RX would mysteriously not restart, if you did a hackrf-stop-rx followed by another hackrf-start-rx. The problem actually was not in my code, but due to some old libhackrf bugs that had not been patched in the old Debian 9 version of libhackrf which I am using.

This is a serious enough annoyance that you won’t want to be using HackRF Shell with the unpatched version. So, I added instructions to my git repo (debian9-libhackrf-patch directory, see commit 346c50e) on how to get a patched version of the Debian 9 packages. I think that the Debian 10 library version is actually not new enough to avoid all the bugs, either, so the info I have provided might be of value to Debian 10 users as well.

I would like to switch my home system from Debian to Gnu Guix, and use Guix package management for development, but I’m not sure how soon that will happen.

git clone git://git.librehacker.com/pub/git/hackrf-rkt.git

HackRF Shell: Progress Update

I added in the baseband-filter-bandwidth control procedure, which is something I forgot to do earlier. This was critical for picking up the weaker stations, such as KJNP 100.3Mhz, which is around 20 or 30 miles away, I think. I coded some simple helper functions (in Scheme) to start and stop receiving data according to time parameters, which I will use to record my favorite radio program each morning. This example records data for one minute from 8:43pm to 8:44pm (code checkout a992f67).

scheme@(guile-user)> (define d (hackrf-open))
scheme@(guile-user)> (load "hackrf-shell-lib.scm")
scheme@(guile-user)> (hackrf-sensible-defaults d)
scheme@(guile-user)> (hackrf-set-baseband-filter-bandwidth d 2000000)
scheme@(guile-user)> (hackrf-enable-amp d)
scheme@(guile-user)> (timed-read d "out.bin" 20 43 20 44)

This still just dumps the floating point signal data to a file, rather that doing any demodulation, so the file size is very large, and I must feed it into GnuRadio. Yet, it is progress.

I need to go over the RX start/stop code again as I get an error if I try to start RX again after stopping it. I coded that part of the device management rather quickly so I am not surprised.

I started playing around with merging in FFT functionality. I added an fft-512 procedure which does FFT on a 512 byte buffer using libfftwf. I think it works, but I haven’t added any procedures yet to do anything useful with fft-512 so I don’t really know yet. I was going to code something which feeds data to GnuPlot for a spectrum analysis display, in the usual fashion like all the SDR software does:

I have been learning a lot lately about Fourier transform and DFT (Discrete Fourier Transform) and I think I have a mostly clear understanding of the basic math and concepts involved now. For fun, I did a DFT operation manually in Emacs Calc on a length 8 data sample, and the results came out making sense. This article is a nice introduction to the Fourier transform, though you need to have a good understanding of complex numbers to fully grasp the DFT equation:

An Interactive Guide to the Fourier Transform