Author: thequietlearner

Curse of War Tips

thequietlearner1 Comment
Curse of War, an Ncurses-based strategy game. I am the green population.

http://a-nikolaev.github.io/curseofwar/

Here are a few tips, some of which you might not learn unless you inspect the source code:

  • Population on a hex can only grow to a maximum of 499, at which point growth slows to zero. So, a full hex might look good but actually has stopped producing.
  • Subject to the caveat above, your hex population growth is a percentage of population already on the hex.
  • Migration from deep territory to the front is a slow process.
  • Because of the above three factors, you generally want to place new villages and towns at locations that are neither too close nor too far from the front of war. If they are too close, the bonus growth effect will be operating on hexes that have only a few troops that are passing through on their way to the front. But too far away, and the bonus will be applied to hexes that are saturated and cannot produce more population anyway.
  • Battle outcomes are basically just a function of which player has more troops on the hex. So, you generally don’t want to push forward your war front that part of the front is mostly full of saturated (499 troop) or nearly-saturated tiles. Pushing forward effectively generally means putting flags directly in front of your current line of flags and then removing the old ones. Then wait until those new hexes become saturated.
  • If you realized you have pushed your front forward too quickly and are getting overcome, you can usually save the line by just pulling the front back a little.
  • You may want to play with the -i 0 option to ensure basically fair starting balance between players.
  • It usually makes more sense, mathematically, to build a new village, than to upgrade to a town or castle. This is due to cost vs. growth percentage increase. Towns or castles don’t give any kind of actual defensive bonus, just the growth increase.
  • If you have a good grasp of the battle, migration, and population growth mechanics, you basically know what you need to know to win most games against the AI. However, a tricky part in some games is knowing how to manipulate the other players against each other, so that they are attacking each other more instead of you. If you have a strong, non-moving set of fronts, generally you AI will attack the other players who have weaker fronts. However, this can back-fire if you allow one opponent to become stronger than you, after slurping up weaker enemies. It is a careful balancing act that requires you to intelligently focus your resource development and front movement.
  • At first it is tempting to focus exclusively on capturing mines, but your overall front movement and population growth is generally the more important thing to focus on. Of course, a front-movement strategy that will eventually get you three more mines, certainly has some points going for it.

HackRF Shell: FFT Plot Progress

thequietlearnerLeave a comment

I threw together a scheme procedure which dumps FFT data to Gnuplot.

You see, however, that I have more work to do as properly labeling the graph, as well as implementing averaging for a more practical spectrum display. The 512 points on the x-axis correspond to the 512 complex numbers in the FFT output buffer, but center frequency in this case is actually 103Mhz, at 8 msps.

scheme@(guile-user)> (define d (hackrf-open))
scheme@(guile-user)> (hackrf-sensible-defaults d)
scheme@(guile-user)> (hackrf-set-freq d 103000000)
scheme@(guile-user)> (hackrf-set-baseband-filter-bandwidth d 2000000)
scheme@(guile-user)> (hackrf-enable-amp d)
scheme@(guile-user)> (define c (cb-fft-to-gnuplot))
scheme@(guile-user)> (hackrf-start-rx d c)
scheme@(guile-user)> (hackrf-stop-rx d)

It was mostly easy to write the scheme -> Gnuplot interface as that is just sending text commands to Gnuplot standard input. However, it took me a while to fix an elusive bug in my C FFT function.

HackRF Shell: RX Restart Bug

thequietlearner3 Comments

I spent quite a while troubleshooting a bug in which RX would mysteriously not restart, if you did a hackrf-stop-rx followed by another hackrf-start-rx. The problem actually was not in my code, but due to some old libhackrf bugs that had not been patched in the old Debian 9 version of libhackrf which I am using.

This is a serious enough annoyance that you won’t want to be using HackRF Shell with the unpatched version. So, I added instructions to my git repo (debian9-libhackrf-patch directory, see commit 346c50e) on how to get a patched version of the Debian 9 packages. I think that the Debian 10 library version is actually not new enough to avoid all the bugs, either, so the info I have provided might be of value to Debian 10 users as well.

I would like to switch my home system from Debian to Gnu Guix, and use Guix package management for development, but I’m not sure how soon that will happen.

git clone git://git.librehacker.com/pub/git/hackrf-rkt.git

HackRF Shell: Progress Update

thequietlearner3 Comments

I added in the baseband-filter-bandwidth control procedure, which is something I forgot to do earlier. This was critical for picking up the weaker stations, such as KJNP 100.3Mhz, which is around 20 or 30 miles away, I think. I coded some simple helper functions (in Scheme) to start and stop receiving data according to time parameters, which I will use to record my favorite radio program each morning. This example records data for one minute from 8:43pm to 8:44pm (code checkout a992f67).

scheme@(guile-user)> (define d (hackrf-open))
scheme@(guile-user)> (load "hackrf-shell-lib.scm")
scheme@(guile-user)> (hackrf-sensible-defaults d)
scheme@(guile-user)> (hackrf-set-baseband-filter-bandwidth d 2000000)
scheme@(guile-user)> (hackrf-enable-amp d)
scheme@(guile-user)> (timed-read d "out.bin" 20 43 20 44)

This still just dumps the floating point signal data to a file, rather that doing any demodulation, so the file size is very large, and I must feed it into GnuRadio. Yet, it is progress.

I need to go over the RX start/stop code again as I get an error if I try to start RX again after stopping it. I coded that part of the device management rather quickly so I am not surprised.

I started playing around with merging in FFT functionality. I added an fft-512 procedure which does FFT on a 512 byte buffer using libfftwf. I think it works, but I haven’t added any procedures yet to do anything useful with fft-512 so I don’t really know yet. I was going to code something which feeds data to GnuPlot for a spectrum analysis display, in the usual fashion like all the SDR software does:

I have been learning a lot lately about Fourier transform and DFT (Discrete Fourier Transform) and I think I have a mostly clear understanding of the basic math and concepts involved now. For fun, I did a DFT operation manually in Emacs Calc on a length 8 data sample, and the results came out making sense. This article is a nice introduction to the Fourier transform, though you need to have a good understanding of complex numbers to fully grasp the DFT equation:

An Interactive Guide to the Fourier Transform

HackRF Shell: Conversion to Floating Point

thequietlearner2 Comments

As expected, the bottle neck has disappeared at the floating point conversion. At least, I can say that I didn’t have trouble pulling 8 million samples per second (msps), which with 32 bit floating point (and 2 floating point numbers per sample) is 64 MB/sec. (I am short on time this evening, so I haven’t had a chance yet to try 20 msps.) As before, I fed the data into a Gnu Radio FM demodulator and got clean FM radio station audio out of it.

GCC 6.3 with -O3 appears to do some SSE optimization on the byte buffer to float buffer conversion:

christopher@nightshade:~/Repos/hackrf-shell$ objdump hackrf-shell -x -D | less
<snip>
    172c:       66 0f 6f c1             movdqa %xmm1,%xmm0
    1730:       66 0f 68 cc             punpckhbw %xmm4,%xmm1
    1734:       66 0f 60 c4             punpcklbw %xmm4,%xmm0
    1738:       66 0f 6f f1             movdqa %xmm1,%xmm6
    173c:       66 0f 65 e8             pcmpgtw %xmm0,%xmm5
    1740:       66 0f 6f f8             movdqa %xmm0,%xmm7
    1744:       66 0f 61 fd             punpcklwd %xmm5,%xmm7
    1748:       66 0f 69 c5             punpckhwd %xmm5,%xmm0
    174c:       66 0f 6f eb             movdqa %xmm3,%xmm5
    1750:       0f 5b ff                cvtdq2ps %xmm7,%xmm7
    1753:       66 0f 65 e9             pcmpgtw %xmm1,%xmm5
    1757:       0f 17 3c 24             movhps %xmm7,(%rsp)
<snip>

The next thing, perhaps, should be to create a little demo program where it captures the data at certain times of day. Or I could work on the next stages of an FM receiver, i.e., frequency multiplication, a low pass filter, and the FM demodulator.

HackRF Shell: It’s alive! Wha ha HA HA HA!

thequietlearner1 Comment

The shell is functioning, including RX functionality (not TX). Currently the process looks like so:

christopher@nightshade:~/Repos/hackrf-shell$ ./hackrf-shell 
GNU Guile 2.2.3
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (load "hackrf-shell-lib.scm")
scheme@(guile-user)> (define d (hackrf-open))
scheme@(guile-user)> (hackrf-sensible-defaults d)
scheme@(guile-user)> (define c (hackrf-cb-rx-to-file "out.bin"))
scheme@(guile-user)> (hackrf-start-rx d c)
scheme@(guile-user)> (hackrf-stop d c)

hackrf-sensible-defaults is simply an alias for

(hackrf-set-freq d 99500000)
(hackrf-set-sample-rate d 8000000)
(hackrf-disable-amp d)
(hackrf-set-lna-gain d 16)
(hackrf-set-vga-gain d 16))

The hackrf-start-rx procedure must receive a callback function which will handle the data each time a block of data is received through libhackrf. The callback function receives a pointer object to the data buffer, which can be converted to a bytevector with pointer->bytevector, and also the byte length of the buffer (an int). User can put together their own callback functions, though I was planning to add more for common use cases. Here are the ones included now:

(define (hackrf-cb-rx-to-stream out)
  (lambda (b bl)
    (let ([bv (pointer->bytevector b bl)])
      (put-bytevector out bv))))

(define (hackrf-cb-rx-to-file filename)
  (hackrf-cb-rx-to-stream
   (open-output-file filename)))

I could expand this by adding more such scheme functions for common use cases (e.g., FM demodulation or frequency analysis) with helper functions written in C to handle intensive mathematical operations (e.g., FFT). You can see here the power of Guile scheme, allowing user to load in any Guile scheme code they want, while I can provide helper functions coded in C to allow for maximum efficiency in critical mathematical operations.

The current code simply provides the raw 8-bit IQ data, but the user may want to receive the IQ data as 32-bit floating point numbers. This should be an easy function to add, which I will code in C to allow for SSE/AVX style compiler optimizations. It still remains to be seen if I will run into a performance bottle neck with the byte to float conversions, but since I do not need to do any IPC with hackrf-shell, I do not expect this to be a problem.

To confirm received data is not junk, I saved about 1 GB of IQ data to “out.bin”, converted the data to floating point values using Gnu Radio, and then ran the data through an FM demodulator Gnu Radio program. I was able to tune into several different radio stations recorded in the save data, such as a local country music station, and I did not hear any distortion or skips.

git clone git://git.librehacker.com/pub/git/hackrf-rkt.git

HackRF Shell Project

thequietlearner1 Comment

So, I got really far in the hackrf-stream project, with essentially everything working to control the HackRF and to manage the RX streams, and then ran into a fatal snag. Everything was based on the idea that I could use FIFO OS pipes to move the data from the C program to the processing program, but FIFO OS pipes turned out to be not fast enough for my needs. In the end, I was only able to move about 90% of the data live across the pipe, and profiling indicated the bottleneck was in system calls, rather than in number crunching. 90% is a lot of data, but I needed 100% going across.

So, I was faced with multiple options on how to proceed. On approach would be to switch instead to Shared Memory Objects for IPC. That I believe would have removed the i/o performance problem, but then all the control applications would need POSIX Shared Memory support, which rather went against the whole idea of having a simple, language agnostic interface.

Another idea was to abandon the idea of a simple, language agnostic interface, and just use POSIX Shared Memory Objects anyway. Unfortunately, Racket Scheme does not have a shared memory interface. Instead, it uses something called “places”, which is shared memory, but only allows sharing with other Racket programs. Chicken Scheme does have a POSIX shm interface, but I wasn’t sure if I really wanted to start using Chicken Scheme just for that reason.

Where I landed in the end was a different approach: embedding Guile Scheme inside my C program. This means when I start the C program, I’m dropped into a Guile Scheme shell, from which I can call whatever C functions I want (after writing the appropriate stubs). This takes me full circle back to what I originally had wanted to do with this project, but instead of using a Racket FFI, I am actually embedding Guile scheme inside the C application, with the C application providing all the core control and performance functionality. If it works, this should be the most fun approach.

I have renamed the project the “HackRF Shell” project.

git clone git://git.librehacker.com/pub/git/hackrf-rkt.git