Tag: hackrf

HackRF Shell: Conversion to Floating Point

thequietlearner2 Comments

As expected, the bottle neck has disappeared at the floating point conversion. At least, I can say that I didn’t have trouble pulling 8 million samples per second (msps), which with 32 bit floating point (and 2 floating point numbers per sample) is 64 MB/sec. (I am short on time this evening, so I haven’t had a chance yet to try 20 msps.) As before, I fed the data into a Gnu Radio FM demodulator and got clean FM radio station audio out of it.

GCC 6.3 with -O3 appears to do some SSE optimization on the byte buffer to float buffer conversion:

christopher@nightshade:~/Repos/hackrf-shell$ objdump hackrf-shell -x -D | less
<snip>
    172c:       66 0f 6f c1             movdqa %xmm1,%xmm0
    1730:       66 0f 68 cc             punpckhbw %xmm4,%xmm1
    1734:       66 0f 60 c4             punpcklbw %xmm4,%xmm0
    1738:       66 0f 6f f1             movdqa %xmm1,%xmm6
    173c:       66 0f 65 e8             pcmpgtw %xmm0,%xmm5
    1740:       66 0f 6f f8             movdqa %xmm0,%xmm7
    1744:       66 0f 61 fd             punpcklwd %xmm5,%xmm7
    1748:       66 0f 69 c5             punpckhwd %xmm5,%xmm0
    174c:       66 0f 6f eb             movdqa %xmm3,%xmm5
    1750:       0f 5b ff                cvtdq2ps %xmm7,%xmm7
    1753:       66 0f 65 e9             pcmpgtw %xmm1,%xmm5
    1757:       0f 17 3c 24             movhps %xmm7,(%rsp)
<snip>

The next thing, perhaps, should be to create a little demo program where it captures the data at certain times of day. Or I could work on the next stages of an FM receiver, i.e., frequency multiplication, a low pass filter, and the FM demodulator.

HackRF Shell: It’s alive! Wha ha HA HA HA!

thequietlearner1 Comment

The shell is functioning, including RX functionality (not TX). Currently the process looks like so:

christopher@nightshade:~/Repos/hackrf-shell$ ./hackrf-shell 
GNU Guile 2.2.3
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (load "hackrf-shell-lib.scm")
scheme@(guile-user)> (define d (hackrf-open))
scheme@(guile-user)> (hackrf-sensible-defaults d)
scheme@(guile-user)> (define c (hackrf-cb-rx-to-file "out.bin"))
scheme@(guile-user)> (hackrf-start-rx d c)
scheme@(guile-user)> (hackrf-stop d c)

hackrf-sensible-defaults is simply an alias for

(hackrf-set-freq d 99500000)
(hackrf-set-sample-rate d 8000000)
(hackrf-disable-amp d)
(hackrf-set-lna-gain d 16)
(hackrf-set-vga-gain d 16))

The hackrf-start-rx procedure must receive a callback function which will handle the data each time a block of data is received through libhackrf. The callback function receives a pointer object to the data buffer, which can be converted to a bytevector with pointer->bytevector, and also the byte length of the buffer (an int). User can put together their own callback functions, though I was planning to add more for common use cases. Here are the ones included now:

(define (hackrf-cb-rx-to-stream out)
  (lambda (b bl)
    (let ([bv (pointer->bytevector b bl)])
      (put-bytevector out bv))))

(define (hackrf-cb-rx-to-file filename)
  (hackrf-cb-rx-to-stream
   (open-output-file filename)))

I could expand this by adding more such scheme functions for common use cases (e.g., FM demodulation or frequency analysis) with helper functions written in C to handle intensive mathematical operations (e.g., FFT). You can see here the power of Guile scheme, allowing user to load in any Guile scheme code they want, while I can provide helper functions coded in C to allow for maximum efficiency in critical mathematical operations.

The current code simply provides the raw 8-bit IQ data, but the user may want to receive the IQ data as 32-bit floating point numbers. This should be an easy function to add, which I will code in C to allow for SSE/AVX style compiler optimizations. It still remains to be seen if I will run into a performance bottle neck with the byte to float conversions, but since I do not need to do any IPC with hackrf-shell, I do not expect this to be a problem.

To confirm received data is not junk, I saved about 1 GB of IQ data to “out.bin”, converted the data to floating point values using Gnu Radio, and then ran the data through an FM demodulator Gnu Radio program. I was able to tune into several different radio stations recorded in the save data, such as a local country music station, and I did not hear any distortion or skips.

git clone git://git.librehacker.com/pub/git/hackrf-rkt.git

HackRF Shell Project

thequietlearner1 Comment

So, I got really far in the hackrf-stream project, with essentially everything working to control the HackRF and to manage the RX streams, and then ran into a fatal snag. Everything was based on the idea that I could use FIFO OS pipes to move the data from the C program to the processing program, but FIFO OS pipes turned out to be not fast enough for my needs. In the end, I was only able to move about 90% of the data live across the pipe, and profiling indicated the bottleneck was in system calls, rather than in number crunching. 90% is a lot of data, but I needed 100% going across.

So, I was faced with multiple options on how to proceed. On approach would be to switch instead to Shared Memory Objects for IPC. That I believe would have removed the i/o performance problem, but then all the control applications would need POSIX Shared Memory support, which rather went against the whole idea of having a simple, language agnostic interface.

Another idea was to abandon the idea of a simple, language agnostic interface, and just use POSIX Shared Memory Objects anyway. Unfortunately, Racket Scheme does not have a shared memory interface. Instead, it uses something called “places”, which is shared memory, but only allows sharing with other Racket programs. Chicken Scheme does have a POSIX shm interface, but I wasn’t sure if I really wanted to start using Chicken Scheme just for that reason.

Where I landed in the end was a different approach: embedding Guile Scheme inside my C program. This means when I start the C program, I’m dropped into a Guile Scheme shell, from which I can call whatever C functions I want (after writing the appropriate stubs). This takes me full circle back to what I originally had wanted to do with this project, but instead of using a Racket FFI, I am actually embedding Guile scheme inside the C application, with the C application providing all the core control and performance functionality. If it works, this should be the most fun approach.

I have renamed the project the “HackRF Shell” project.

git clone git://git.librehacker.com/pub/git/hackrf-rkt.git

HackRF Racket Bindings: Update 5

thequietlearner1 Comment

The current approach is working out well so far. There is a “hackrf-kernel” C program which receives commands over a control input stream (pipe file) and sends responses back to a control output stream. Data from RX is sent to the data output stream, after calling the startrx command. (The starttx command is not yet implemented.) Since hackrf supports multiple devices streaming simultaneously, there was a question of how to handle the multiple data streams. Rather than having separate pipes for each device, I opted instead to stick with one output pipe, and just have hackrf-kernel send a notification to control output, explaining how many bytes where coming down the pipe, and which device it is for. I used semaphores to ensure that the order of the incoming RX data matches the order of the control notifications. So the external program must simply pull the correct number of bytes from the stream.

A benefit of this approach is that the stream interface is language agnostic, so that I could use it with a Racket program, and somebody else could use it with a C program, or Python program, or anything else that supports reading and writing from file streams. Since I have not written any of the Racket code, for testing I have just been connecting to the pipes with the cat command and redirection arrows. I successfully downloaded about 500MB of RX data this way in a few seconds before sending the stoprx command.

So, I can do RX now, but the hackrf-kernel is not quite useful yet: I need to implement the commands for set the frequency, sample rate, etc.

HackRF Racket Bindings: Update 4

thequietlearner1 Comment

I had too much trouble, unfortunately, trying to bind the libhackrf functions directly to Racket functions with the FFI interface. The two hold ups were (1) I discovered I wasn’t actually supposed to be doing I/O inside a callout function using async-apply, and was having trouble trying to set up some kind of thunk queuing system instead; (2) for some reason the hackrf_stop_rx function would always hang when called from within Racket. On the second problem, I didn’t really have any way to debug the issue, but it looked liked pthread_join was not finishing from inside hackrf_stop_rx, perhaps something to do with the async-apply mechanics.

Feeling like I didn’t have way to move forward, because I don’t know how to debug C functions called from Rcket, I instead decided to back up and change my approach. Instead, I’m just going to use a separate small C program “kernel” to call the libhackrf functions, and communicate with that over named pipes. C programming is a lot of work, but I’ve gotten as far as writing the code in the kernel that creates the pipes and pulls a line from the input control pipe. There are four pipes: control input, control output, data input, and data output.

git clone git://git.librehacker.com/pub/git/hackrf-rkt.git