Monday, March 12, 2018

Not merging stuff from FreeBSD-HEAD into production branches, or "hey FreeBSD-HEAD should just be production"

I get asked all the time why I don't backport my patches into stable FreeBSD release branches. It's a good question, so let me explain it here.

I don't get paid to do it.

Ok, so now you ask "but wait, surely the users matter?" Yes, of course they do! But, I also have other things going on in my life, and the stuff I do for fun is .. well, it's the stuff I do for fun. I'm not paid to do FreeBSD work, let alone open source wireless stuff in general.

So then I see posts like this:

https://www.anserinae.net/adventures-in-wifi-freebsd-edition.html

I understand his point of view, I really do. I'm also that user when it comes to a variety of other open source software and I ask why features aren't implemented that seem easy, or why they're not in a stable release. But then I remember that I'm also doing this for fun and it's totally up to me to spend my time however I want.

Now, why am I like this?

Well, the short-hand version is - I used to bend over backwards to try and get stuff in to stable releases of the open source software I once worked on. And that was taken advantage of by a lot of people and companies who turned around to incorporate that work into successful commercial software releases without any useful financial contribution to either myself or the project as a whole. After enough time of this, you realise that hey, maybe my spare time should just be my spare time.

My hope is that if people wish to backport my FreeBSD work to a stable release then they'll either pay me to do it, pay someone else to do it, or see if a company will sponsor that work for their own benefit. I don't want to get into the game of trying to backport things to one and potentially two stable releases and deal with all the ABI changes and support fallout that happens when you are porting things into a mostly ABI stable release. And yes, my spare time is my own.


Monday, December 25, 2017

More TS-440S hijinx, or "ok, what if you wanna homebrew a digital hookup?"

I've been homebrewing digital hookups between my amateur radios (HF, VHF, UHF) and a FreeBSD PC. It all ... mostly works. There's one or two FreeBSD hiccups though, which are summarised thusly:

The default package selection for audio paths is .. suboptimal. Some can be configured to use OSS and that is nice. Some provide ALSA but FreeBSD's "ALSA" implementation doesn't provide full ALSA device emulation so we don't get a list of ALSA devices by default. You have to put your devices into asound.conf with names for things. However ..

.. FreeBSD doesn't currently make it easy to hard-code say, USB device paths to serial port names or sound devices to something predictable. So every time I reboot or mess with the setup it goes pear shaped.

Then there's a bug where I can run three USB audio devices, but I can't do mic input on the last one. Output works fine. That's going to be amusing to diagnose.

So I do have my TS-440S, TS-711A and TS-811E all doing digital modes. They're just all .. subtly different.

The TS-711A and TS-811E have an accessory jack (ACC2) that has input and output. There's a PTT control line and a mic mute line. The line levels of those signals is a couple hundred millivolts, so it's good enough to build a little resistor divider with a potentiometer to get the computer output down to the right level. I'm using mic input on the USB audio devices, so that also works fine at a couple hundred millivolts.

The TS-440S also has a similar accessory jack, however the audio input in that jack seems to be quite a big higher than a couple hundred millivolts. It looks like it needs to be around 4-5v peak-to-peak for it to be at the right level internally on the IF board. The ACC2 path has a couple of resistor attenuators so it looks like this was intentional - after asking around it looks like it's expecting professional line audio output levels (~4v peak-to-peak) instead of consumer grade levels (~1.5v peak-to-peak.) I'll go dig into it some more. This path bypasses the microphone pre-amplifier entirely and goes straight into the Mic Gain control pot.

The TS-440S also has AFSK input/output RCA jacks on the back. The audio output is at the same level as the ACC2 jack, however the audio input side is routed via the microphone input side so it gets preamp'ed and processed appropriately. That's what I've been using for digital modes - I can divide down the input side to a couple hundred millivolts to keep it all kosher. However - and here's the really annoying part - the mic mute input on ACC2 also mutes the AFSK input line.

Then there's what happens if you leave the microphone connected. If you do leave it connected, even in a quiet room, it seems to present some load that requires a lot more signal on AFSK input to do its thing. If you tune it all up to the right signal levels and then disconnect the microphone, you'll be really overdriving the RF section. Ugh.

So - I don't have to do any of this for the TS-711 and TS-811 - their input values are a lot lower and grounding the mic line actually just quietens the mic input.

If I can score a slightly different radio - like a TS-680S for example - then I can do this stuff with a much lower level line input value. It'll be tricky to get it down to the TS-680S level (it wants it at 10mV!) but at least I can do that with passive, well shielded bits.

On the plus side - yes, this means I at least can do digital modes on my TS-440S. I just have to keep unplugging the microphone line for now. What I may end up doing for now though is adding another switch to the desktop microphone I have to turn /its/ microphone input off so it is fully disconnected. Hopefully that'll be enough to do digital modes without constantly screwing and unscrewing things.

If you're at all curious - https://www.pskreporter.info/pskmap?callsign=kk6vqk&search=Find

Tuesday, October 10, 2017

FreeBSD and APRS, or "hm what happens when none of this is well documented.."

Here's another point along my quest for amateur radio on FreeBSD - bring up basic APRS support. Yes, someone else has done the work, but in the normal open source way it was .. inconsistently documented.

First is figuring out the hardware platform. I chose the following:

  • A Baofeng UV5R2, since they're cheap, plentiful, and do both VHF and UHF;
  • A cable to do sound level conversion and isolation (and yes, I really should post a circuit diagram and picture..);
  • A USB sound device, primarily so I can whack it into FreeBSD/Linux devices to get a separate sound card for doing radio work;
  • FreeBSD laptop (it'll become a raspberry pi + GPS + sensor + LCD thingy later, but this'll do to start with.)
The Baofeng is easy - set it to the right frequency (VHF APRS sits on 144.390MHz), turn on VOX so I don't have to make up a PTT cable, done/done.

The PTT bit isn't that hard - one of the microphone jack pins is actually PTT (if you ground it, it engages PTT) so when you make the cable just ensure you expose a ground pin and PTT pin so you can upgrade it later.

The cable itself isn't that hard either - I had a baofeng handmic lying around (they're like $5) so I pulled it apart for the cable. I'll try to remember to take pictures of that.

Here's a picture I found on the internet that shows the pinout:


Now, I went a bit further. I bought a bunch of 600 ohm isolation transformers for audio work, so I wired it up as follows:

  • From the audio output of the USB sound card, I wired up a little attenuator - input is 2k to ground, then 10k to the input side of the transformer; then the output side of the transformer has a 0.01uF greencap capacitor to the microphone input of the baofeng;
  • From the baofeng I just wired it up to the transformer, then the output side of that went into a 0.01uF greencap capacitor in series to the microphone input of the sound card.
In both instances those capacitors are there as DC blockers.

(I'd draw up a circuit diagram but for some reason there's no easy tool here in blogger to do that in-line! Sigh.)

Ok, so that bit is easy.

Then on to the software side.

The normal way people do this stuff is "direwolf" on Linux. So, "pkg install direwolf" installed it. That was easy.

Configuring it up was a bit less easy. I found this guide to be helpful:

https://andrewmemory.wordpress.com/tag/direwolf/

FreeBSD has the example direwolf config in /usr/local/share/doc/direwolf/examples/direwolf.conf . Now, direwolf will run as a normal user (there's no rc.d script for it yet!) and by default runs out of the current directory. So:

$ cd ~
$ cp /usr/local/share/doc/direwolf/examples/direwolf.conf .
$ (edit it)
$ direwolf

Editing it isn't that hard - you need to change your callsign and the audio device.

OK, here is the main undocumented bit for FreeBSD - the sound device can just be /dev/dsp . It isn't an ALSA name! Don't waste time trying to use ALSA names. Instead, just find the device you want and reference it. For me the USB sound card shows up as /dev/dsp3 (which is very non specific as USB sound devices come and go, but that's a later problem!) but it's enough to bring it up.

So yes, following the above guide, using the right sound device name resulted in a working APRS modem.

Next up - something to talk to it. This is called 'xastir'. It's .. well, when you run it, you'll find exactly how old an X application it is. It's very nostalgically old. But, it is enough to get APRS positioning up and test both the TCP/IP side of APRS and the actual radio radio side.

Here's the guide I followed:

https://andrewmemory.wordpress.com/2015/03/22/setting-up-direwolfxastir-on-a-raspberry-pi/

So, that was it! So far so good. It actually works well enough to decode and watch APRS traffic around me. I managed to get out position information to the APRS network over both TCP/IP and relayed via VHF radio.

Monday, September 11, 2017

Fixing up TS-440s rigs

I've been teaching myself HF electronics and antennas lately. I thought I'd go write down some of the things I came across whilst tinkering with this hardware.

First up - I decided to choose the Kenwood TS-440S.


http://www.rigpix.com/kenwood/ts440s.htm

It's a mid 1980's solid state rig with only a few components hidden away in custom ICs. There are some parts that you just can't buy new anymore (mostly these custom parts and stuff in the final RF amplifier section) but by and large it's all a big set of interconnected single-sided PCBs covered in cables and discrete components. There's the occasional bit of 74LS logic too.

It has some pretty clean and sensitive RX paths and TX is supposed to be very good for hours and hours of work. However, these devices are pretty old, and 30+ year electronics can have a large amount of wear and tear.

There are some pretty well known crappy issues too. One of them is the SonyBond compound used in the VCO (voltage controlled oscillator) sections - it's hydroscopic and degrades over time. This ends up throwing the VCOs way off and you end up having to clean it off of the PLL/RF boards. This is pretty well documented and although time consuming, it's not impossible to fix yourself.

I'll try to do a follow-up post with some pointers on tuning the TX path behaviour because I found a lot of really inconsistent suggestions on tuning things. The service manual is also a bit confusing at times on what voltages you should see where.

There are also issues with broken connectors and a lot of dry joints. I found a lot of dry joints in my 440s, which required a lot of tidying up.

Next up in the TODO list for me is tuning the RX sensitivity to make sure it's optimal.

So, what I found with the first rig I fixed up! I wish I had taken more photos; I'll do that soon just for demonstration purposes.

  • There were lots and lots of dry joints. Everywhere.
  • Sonybond needed cleaning up - and when you do that, you have to tune VCO1 - which is the VCO generating the primary RF frequency for first stage mixing. This is used for both transmit and receive.
  • The IF board was putting out a nice, clean 8.83MHz carrier, so that was nice.
  • The transmit path hadn't been aligned in a long while, so the waveform on the output of the RF board was splatting overly large signals (> 3v peak to peak) into the finals, and this caused the finals to get very upset.
  • .. and then ALC wasn't adjusted, so the filter board was overly aggressively asserting ALC.
  • The transmit power control was very touchy - 100% carrier level at like 5% turning of the carrier level knob. Once the transmit path on the RF board was properly aligned, the RF drive output was properly putting out around 2.5v ptp -> 3.0v ptp and ALC was re-aligned on the filter board, the finals were behaving much better and the TX power control was much more linearly controllable.
  • Then you have to re-do the finals bias level controls. Careful though, the alignment process says "turn to minimum first" before you re-bias things, but the board is mounted upside down so you may be biasing them to max. :)
I'll go into some more detail later with pictures!


Monday, June 5, 2017

gqrx, direct sampling configuration, shortwave/AM reception, etc

This is one of those articles that I write specifically to, you know, not forget it the next time.

I picked up a v3 RTL-SDR dongle from http://www.rtl-sdr.com/rtl-sdr-blog-v-3-dongles-user-guide/ a few weeks ago. It's a solidly looking aluminium can dongle with a much more useful RF connector. But, it still only tunes down by default to the same RTL-SDR limits as .. well, the rest.

So I go digging. I bought this thing because on Windows you're supposed to be able to tune down to a couple hundred KHz. I found out some useful stuff after a bit of google abuse:

  • it's called "direct sampling mode";
  • it's supported by the rtl_sdr driver/library that you normally use;
  • you have to configure things up in a special way to get GQRX, etc to actually do the right thing;
  • There's a specific thing called "Q-Branch" that you have to care about.
Ok, so the actual details.

Firstly, the NIC needs to go into direct sampling mode. So instead of being mixed with a VCO, you're bypassing all of that and just acting as a really fast ADC. Which, yes, will alias signals everywhere.

Secondly, you have to tell it to only give you "Q" samples, rather than both I and Q. If it's direct sampling, mixing doesn't come into it, so the hardware is just giving you ADC samples.

Then, you just tell GQRX, etc to ignore enforcing tuning limits (tick "No limits") and you're golden.

The specific string to add to the device string when you set up which rtl_tcp instance to talk to is:

direct_samp=2

(0 = no direct sampling; 1 = direct sampling on I, 2 = direct sampling on Q.)

Ok, so does it work? I'm still testing it. I will need to acquire some RF filters to try and filter out bands of HF frequencies to avoid aliasing as there is limited to on actual passband filtering going on here. But it did tune to AM radio and I picked up some data transmissions in 3MHz.

Baby steps!


Sunday, May 7, 2017

gqrx on freebsd

I'm still tinkering with my own SDR code in C, but I decided I'd give gqrx a go. It sports a simple UI to visualise things as well as a simple control network protocol.

FreeBSD has packages for it, so it really was as simple as:

# pkg install gqrx

I already have the rtl-sdr bits installed, so that was pretty easy:

# rtl_tcp

.. to bring up the network service for it.

But running gqrx .. didn't work. It just stood there and did nothing when I ran it - no UI, no useful log output, nothing.

So out came truss. Step one in the "what is going on" showed no useful logging. Step 2 is "ok, what syscalls is it making." It turns out it was trying to connect to the IPv6 localhost address - but rtl_tcp defaults on FreeBSD to IPv4 localhost. gqrx doesn't handle connect() errors gracefully, so it was just stuck in an infinite connect() loop.

So I cleared the config, started gqrx again, and ensured this time it was set to '127.0.0.1' instead of 'localhost'. Voila! It worked!

Now - I need an SDR unit that can go down closer to DC so I can pick up shortwave. This RTL dongle I have doesn't tune below 27MHz.

Friday, April 28, 2017

Bringing up 802.11ac on FreeBSD

I've been chipping away at bringing up 802.11ac on FreeBSD. I've been meaning to write this post for a while, but you know, life got in the way.

net80211 has reasonably good 802.11n support, but no 802.11ac support. I decided a while ago to start adding basic 802.11ac support. It was a good exercise in figuring out what the minimum set of required features are and another excuse to go find some of the broken corner cases in net80211 that needed addressing. I'll talk about a few here.

802.11ac introduces a few new concepts that the stack needs to understand. I decided to use the QCA 802.11ac parts because (a) I know the firmware and general chip stuff from the first generation 11ac parts well, and (b) I know that it does a bunch of stuff (like rate control, packet scheduling, etc) so I don't have to do it. If I chose, say, the Intel 11ac parts then I'd have to implement a lot more of the fiddly stuff to get good behaviour.

So, the first step was to survey what I needed to at bring up a very small subset of functionality. The typical first target is "monitor" mode, since that requires channel configuration to work, but doesn't require the rest of the stack to know about anything (ie, no 11ac aware IE negotiation.)

Step one - adding VHT channels. I decided in the shorter term to cheat and just add VHT channels to the already very large ieee80211_channel map. The linux way of there being a channel context rather than hundreds of static channels to choose from is better in the long run, but I wanted to get things up and running. So, that's what I did first - I added VHT flags for 20, 40, 80, 80+80 and 160MHz operating modes and I did the bare work required to populate the channel lists with VHT channels as well.

This required adding support to lib80211 and ifconfig to support loading VHT channels from the regulatory domain (regdomain.xml.) And yes, this let me get VHT channels statically configured.

Then I needed to add VHT capabilities to the driver structure (ieee80211com) and the virtual interface structure (ieee80211vap.) This is so that drivers can be told that yes, we can do VHT things.

Then I needed to glue it into an 11ac driver. My ath10k port was far enough along to attempt this, so I added enough glue to say "I support VHT" to the ic_caps field and propagated it to the driver for monitor mode configuration. And yes, after a bit of dancing, I managed to get a VHT channel to show up in ath10k in monitor mode and could capture 80MHz wide packets. Success!

So at this point I took a big of a segue and decided to ensure that the receive packet flags made sense for VHT operation. I did a bunch of work to tidy this up so I could mark frames as 20/40/80/etc and had all of that mostly done in a few days.

Next up was station operation. So this required a few things to get right:
  • Channel promotion (ie, going from 11a -> 11n -> 11ac)
  • Knowing about VHT IEs, both for transmit and receive
  • Negotiating VHT capabilities to announce to the AP during association request/response and then telling the driver about what's going on.
By far the most fiddly was getting channel promotion to work. net80211 supports the concept of dumb NICs (like atheros 11abgn parts) very well, where you can have multiple virtual interfaces but the "driver" view of the right configuration is what's programmed into the hardware. For firmware NICs which do this themselves (like basically everything sold today) this isn't exactly all that helpful. So, for now, it's limited to a single VAP, and the VAP configuration is partially derived from the global state and partially derived from the negotiated state. It's annoying, but it is adding to the list of things I will have to fix later.

Sorry, I went off on a bit of a tangent. So one of the things net80211 does is it looks at the list of desired channels for each VAP and determines what the "NIC" channel should be in order to support it. So yes, ic_curchan is being used and it's what gets updated. the vap->iv_des_chan is the "I'd like to be this channel" but the active channel is actually vap->iv_bss->ni_chan (yes, the BSS node current channel.) That's what gets promoted. I need to uhm, "fix" the per-VAP state to have a "current channel" that gets updated with the BSS channel - so that's high on the list of things to fix.

Then the next part for channel promotion is how it's done. IT's not done via a call to "promote channel" - it instead was done by a call to net80211's 802.11n routines which just parse HT (11n) IEs. Yes, parsing an 11n IE also upgraded the channel. So I spent a bunch of time refactoring that, and now the bits that parse IEs are separate from channel promotion. Fun times.

So, once channel promotion worked, I associated fine as an open mode 11ac station. All of the negotiated pieces were wrong, so I spent a few days looking at packet captures to ensure I negotiated VHT capabilities correctly, but it seems to work fine.

Next was adding in support for things that stopped me doing encrypted data. The TL;DR here is - net80211's crypto key handling model needed overhauling.

Firstly the key management code assumes everything is synchronous and can't sleep. For firmware based devices this really isn't true. So, a lot of drivers defer key management into taskqueues to run things. This sounds fine, but then you realise that once the key addition succeeds, net80211 then will transmit frames expecting them to be encrypted. Userland too has this problem - it will do an ioctl() to set a key, then when it returns OK, it will queue further key negotiation frames. So, I had to do some uhm, clever things to try and work around this, and I'll cover how I need to fix it properly later.

Next up - the QCA chips/firmware do 802.11 crypto offload. They actually pretend that there's no key - you don't include the IV, you don't include padding, or anything. You send commands to set the crypto keys and then you send unencrypted 802.11 frames (or 802.3 frames if you want to do ethernet only.) This means that I had to teach net80211 a few things:

  • frames decrypted by the hardware needed to have a "I'm decrypted" bit set, because the 802.11 header field saying "I'm decrypted!" is cleared
  • frames encrypted don't have the "i'm encrypted" bit set
  • frames encrypted/decrypted have no padding, so I needed to teach the input path and crypto paths to not validate those if the hardware said "we offload it all."
So yeah, that's now done.

(Now, there's a whole separate discussion about hostap that I'll defer for later. Suffice to say - at this point I got hostap mostly working.)

Now comes the hard bit of fixing the shortcomings before I can commit the driver. There are .. lots.

The first one is the global state. The ath10k firmware allows what they call 'vdevs' (virtual devices) - for example, multiple SSID/BSSID support is implemented with multiple vdevs. STA+WDS is implemented with vdevs. STA+P2P is implemented with vdevs. So, technically speaking I should go and find all of the global state that should really be per-vdev and make it per-vdev. This is tricky though, because a lot of the state isn't kept per-VAP even though it should be.

So, I need to fix the following to be per-vdev rather than global:
  • Slot time configuration
  • ERP configuration (ie, 11g BSS overlap with 11b)
  • Frame protection configuration
  • QoS/WMM configuration
  • 20/40 channel width configuration - which needs extending to 11ac channel widths too
Then there are some things which are just plainly not implemented. A-MPDU offload for example is fine, but A-MSDU offload pretends that there are multiple 802.11 frames with the same sequence number / crypto information. This is problematic for the sequence number de-duplication code and also A-MPDU offload - in both cases, all but one A-MSDU frames are just tossed. This drastically kills performance. I'm almost done implementing A-MSDU in A-MPDU offload support, and when that's done, I'll implement A-MSDU offload de-duplication awareness and performance should drastically rise.

Another thing as mentioned is channel width, as well as other 802.11n things that need extending for 802.11ac. A bit one that I'd like to get right is SMPS (spatial multiplexing power save.) If the chip supports it in firmware then we just need to allow it to be configured and controlled, but that isn't even an option right now. I'd like to implement SMPS for Atheros 11n parts in ath(4), but that's a lot more work.

Anyway, so far so good. I need to do some of the above and land it in FreeBSD-HEAD so I can finish off the ath10k port and commit what I have to FreeBSD. There's a lot of stuff coming - including all of the wave-2 stuff (like multiuser MIMO / MU-MIMO) which I just plainly haven't talked about yet.

Viva la FreeBSD wireless!