diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25.md b/research/sonar-dsp-recursive-refinement-2026-05-25.md index d673cf9..be78db1 100644 --- a/research/sonar-dsp-recursive-refinement-2026-05-25.md +++ b/research/sonar-dsp-recursive-refinement-2026-05-25.md @@ -11,6 +11,11 @@ experiment rejected the naive objective function, not recursive refinement. ## Sources Checked +The durable source ledger is +`sonar-dsp-recursive-refinement-2026-05-25/sources.md`. Local mirrors of the +source pages and PDF live under +`sonar-dsp-recursive-refinement-2026-05-25/mirrors/`. + - liquid-dsp: open-source C DSP library for SDR-style filters, transforms, synchronizers, modulators, demodulators, and frame processing. diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/ahoi-hardware-overview.html b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/ahoi-hardware-overview.html new file mode 100644 index 0000000..5d19c79 --- /dev/null +++ b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/ahoi-hardware-overview.html @@ -0,0 +1,1537 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Overview - ahoi // Open-Source Acoustic Underwater Modem [beta] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+ +
+ + + + + +

Hardware Overview

+

System

+

+

The ahoi modem consists of 3 components (stacked boards): mainboard, receiver and transmitter. The boards are connected with a 31-pin connector, sharing an SPI and I2C Bus, GPIOs, control signals, and the analog hydrophone signals. +The hydrophone is connected to the transmitter board.

+
+

Hydrophone Connection

+

The hydrophone must be connected to the transmitter board; it must not be connected directly to the 31-pin connector lines.

+
+

Mainboard (MB)

+

+

The mainboard hosts the microcontroller (STM32F446RET), which performs the signal processing and controls the other boards. +All circuitry of the modem is powered by a single 12V supply, which must be connected to the mainboard by the corresponding connector. +This connector features a polarity protection.

+

The microcontroller and all logic use a 3.3V DC supply, which is generated on-board by a dedicated, low-noise DCDC converter. +This 3.3V rail is fed to the 31-pin connector and is used by the other boards.

+

A serial and control port is available via pin headers and on the 17-pin connector. These lines are isolated and require an external power supply. +We recommend a 3.3V supply and 3.3V logic levels; however, the isolator has a maximum rating of 5.5V. +The two control lines are reserved for future use. +The serial line enables communication with the modem for configuration purposes and packet sending and receiving. +The modem can also be programmed via the serial line.

+

The 17-pin connector exposes the lines for programming the modem via an STM programmer (ST-LINK/v2). These lines are protected by resistors only. +Please note that you must not connect an external power supply to this 17-pin connector.

+

Finally, six LEDs indicate the status of the modem.

+

Receiver (RX)

+

+

The receiver board acquires the analog signal from the corresponding lines of the 31-pin connector and feeds it through the following analog stages:

+
    +
  1. Biasing and light amplification based on a low-noise, low-power JFET. The input impedance must be adjusted to the hydrophone in use.
  2. +
  3. Pre-amplification with an optional circuitry for a software-switchable, reduced gain in noisy environments.
  4. +
  5. Bandpass filter to remove noise and lightly amplify the signal. This stage relies on precise components (1%) and requires adaption of the pass band based on the used frequency band.
  6. +
  7. Software-controlled variable gain stage with 18 gain levels in steps of 2 dB.
  8. +
  9. ADC
  10. +
+

The bandpass has a bandwidth of 50 kHz. Its center frequency can be adjusted via 8 capacitors (named CBP). +Per-stage and overall gain is summarized in the following table.

+ + + + + + + + + + + + + + + + + + + + + +
inputpre-ampbandpassamplifiertotal
gain (dB)1735 / (17)3dB3, 5, ..., 3940 - 94
+

The receiver features its own 5V analog supply to suppress digital circuit noise via a DCDC step-down converter followed by an LDO. +The former is supplied by the 12V line from the mainbaord. All logic levels on the receiver are 3.3V, powered from the digital/IO bus.

+

The receiver is enabled via line RX_EN by the microcontroller. Its typical power consumption is 100 mW in active mode. +Line PC0 is used to control the pre-amp gain reduction (low: full gain, high: -18 dB). The amplifier is controlled via I2C and the ADC is controlled by the microcontroller and connected via SPI (CS0).

+

Transmitter (TX/TXB)

+

The transmitter of the ahoi modem comes in two flavors: a low-power transmitter (TX) and a high-power transmitter (TXB). +The default setup is to use TXB.

+

+

The structure of both variants is equal, differences are in the power supply and the power amplifier topology and ICs.

+

The analog signal is generated by a DAC and fed through a 4th-order low-pass reconstruction filter. +Aftwerwards, the signal passes an attenuation stage, which is controlled by the microcontroller in 11 steps of -3 dB from 0 dB to -30 dB. +A power amplifier boosts the signal level and drives the hydrophone.

+

The TXB provides a +/-22 V supply and produces an output signal of +/-40 V amplitude via a bridge amplifier topology. +The maximum output current is 200 mA; we recommend a peak maximum of 100 mA to prevent damage from overheating. +The TX provides a +/-18 V supply via a low-power charge pump and produces an output signal of +/-17.5 V amplitude via a single op-amp. +The maximum peak output current is limited to ca. 40 mA.

+

Both boards host the hydrophone connector and a signal switch to connect the hydrophone either to the transmit output circuitry or the receiver (via the lines H+ and H- on the 31-pin connector). +The hydrophone is connected to the transmit output circuitry, if the transmitter is enabled (via line TX_EN) and line SW_TXRX is logic high. Otherwise, the hydrophone is connected to the receiver (via the 31-pin connector). The behavior of both lines is software-controlled. +The transmitter is only enabled for transmission and powered off otherwise with negligible consumption. Idle consumption, when the transmitter is on but not sending, is ca. 300 mW for TX and 800 mW for TXB.

+

The DAC interfaces with the microcontroller via SPI (CS1). Transmit output levels are selected by software via I2C.

+
+

Bug

+

Versions 5.0 to 7.0 of TXB have a hardware bug that needs to be fixed, please refer to Errata.

+
+

Programming board

+

The programming board attaches to both connectors at the mainboard and contains an SWD connector and pin headers for the signal pins. +The boards is mainly used for fast programming via an ST-LINK/v2 programmer. It can also be used for debugging, because all lines of the 31-pin connector are available via pin headers. +For programming, a separate 12V supply is required to power the mainboard. The available supply of the programmer is not connected. +Note that you should not connect a 12V power supply to the programming board: this will bypass the protection circuit; use the connector on the mainboard instead.

+

Debug board

+

Since the programming board blocks easy access to the top side of the connected PCB, we developed a debug board. +It has only one 31-pin connector and is mounted in a way that gives access to the entire connected PCB. +It exposes all lines of the 31-pin header, provides a standalone 3.3V supply, and several LEDs. +The 3.3V supply is required only when connecting an RX or TX(B) board for debugging or testing purposes.

+
+

Danger

+
    +
  • When you connect the debug board to an MB, you should not connect a 12V power supply to the programming board: this will bypass the protection circuit; use the connector on the mainboard instead! You must also disconnect the 3.3V supply of the debug board by removing its jumper!
  • +
  • When you connect the debug board to an RX or TX(B), you have to connect the 12V supply to the debug board 12V and GND pins. Be aware that the debug board has no polarity protection!
  • +
+
+ + + + + + + + + + + + + + + + + + + + +
+
+ + + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/gnuradio-correlate-access-code-tag-stream.html b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/gnuradio-correlate-access-code-tag-stream.html new file mode 100644 index 0000000..4cb26e6 --- /dev/null +++ b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/gnuradio-correlate-access-code-tag-stream.html @@ -0,0 +1,346 @@ + + + + +Correlate Access Code - Tag Stream - GNU Radio + + + + + + + + + + + + + +
+
+
+ +
+
+
+

Correlate Access Code - Tag Stream

+
+
From GNU Radio
+
+
+ +
+ Jump to navigation + Jump to search +

Examine input for specified access code, one bit at a time. +

input: stream of floats (generally, soft decisions) +

output: a tagged stream set of bits from the payload following the access code and header +

This block searches for the given access code by slicing the soft decision symbol inputs. Once found, it expects the following 32 samples to contain a header that includes the frame length (16 bits for the length, repeated). It decodes the header to get the frame length in order to set up the tagged stream key information. +

The output of this block is appropriate for use with tagged stream blocks. +

+

Parameters

+
Access Code
+
is represented with 1 byte per bit, e.g., 010101010111000100
+
It is important to choose an access code that is not "cyclical". A cyclical code contains a repetition of itself within the length of the code. This can cause false access code detections, and will cause byte boundaries (and hence valid packet length and payload recovery) to be recovered in error, with bizarre results!
+
For example, an access code of 11111111 is a poor choice because it is cyclical. Let's look at why this is.
+
The first eight 1 bits in a row from the block's input stream will correctly detect the first access code. The first packet (in standard form <access code> <length1> <length2> <payload>) will be recovered nominally.
+
However, when the correlate access code block's state machine goes back into scanning/detection mode after the first packet, the first bit of the NEXT access code (which forms the 11111111 preamble of the next packet) will cause an immediate access code detection. This is because the access code detection routine within the block always shifts the last detected access code one bit to the left, and ORs in the current input bit.
+
In our example, the valid bits 11111111 from the first access code detected will get shifted left one bit (to give 11111110), and then the next input bit (the first bit after the end of the first packet) will be ORed into the value (11111110 OR 1, giving 11111111) resulting in an erroneous detection of the next access code, because 11111111 will of course match the access code that the block is searching for.
+
This will falsely detect the start of the next packet (alignment off by 7 bits) and your flowgraph will explode with subsequent bad headers and invalid data. Not a pretty site.
+
So, when choosing an access code, select one that is not cyclical!
+
A possible (and historically interesting) non-cyclical 32-bit access code might be 0xe15ae893:
+
11100001010110101110100010010011
+
which was the "unique word" used on several early communications satellites.
+
Threshold
+
maximum number of bits that may be wrong
+
Tag Name
+
key of the tag inserted into the tag stream
+

Example Flowgraph

+

This flowgraph is taken from the Packet_Communications tutorial. +

+

+

Source Files

+
C++ files
+
[1]
+
Header files
+
[2]
+
Public header files
+
TODO
+
Block definition
+
[3]
+ + + + +
+
+ +
+
+ +
+

Navigation menu

+
+ + + +
+ + + + + + +
+
+ + + + + + + + + +
+
+ +
+ + + + + + + + + + + + +
+ +
+ + + + + + \ No newline at end of file diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/gnuradio-stream-tags.html b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/gnuradio-stream-tags.html new file mode 100644 index 0000000..c068ef0 --- /dev/null +++ b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/gnuradio-stream-tags.html @@ -0,0 +1,667 @@ + + + + +Stream Tags - GNU Radio + + + + + + + + + + + + + +
+
+
+ +
+
+
+

Stream Tags

+
+
From GNU Radio
+
+
+ +
+ Jump to navigation + Jump to search +


+This tutorial describes the use of stream tags. The next tutorial, Polymorphic Types (PMTs) covers Polymorphic_Types +

+ + +

Introduction

+

Stream tags are an isosynchronous data stream that runs parallel to +the main data stream. A stream tag is generated by a block's work +function and from there on flows downstream alongside a particular sample, +until it reaches a sink or is forced to stop propagating by another +block. +

Stream tags are defined for a specific item in the data stream and are +formed as a key:value pair. The key identifies what the value represents +while the value holds the data that the tag contains. Both key and +value are Polymorphic Types (PMTs) where the key is a PMT symbol while +the value is any type of PMT and can therefore handle any data we wish +to pass. An additional part of the tag is the srcid, which is a PMT +symbol and is used to identify the block that created the tag (which +is usually the block's alias). +

+

Background

+

GNU Radio was originally a streaming system with no other mechanism to +pass data between blocks. Streams of data are a model that work well +for samples, bits, etc., but can lack for control and metadata. +

Part of this is solved using the existing message passing interface, which +allows blocks to subscribe to messages published by any other block in +the flowgraph (see Message Passing). The main drawback to the +message passing system is that it works asynchronously, meaning that +there is no guarantee when a message may arrive relative to the data +stream. +

+

API Extensions to the gr::block

+

To enable the stream tags, we have extended the API of gr::block to +understand absolute item numbers. In the data stream model, each +block's work function is given a buffer in the data stream that is +referenced from 0 to N-1. This is a relative offset into the data +stream. The absolute reference starts from the beginning of the +flowgraph and continues to count up with every item. Each input stream +is associated with a concept of the 'number of items read' and each +output stream has a 'number of items written'. These are retrieved during +runtime using the two API calls: +

+
unsigned long int nitems_read(unsigned int which_input);
+unsigned long int nitems_written(unsigned int which_output);
+
+

Each tag is associated with some item in this absolute time scale that +is calculated using these functions. +

Like the rest of the data stream, the number of items read/written are +only updated once during the call to work. So in a work function, +nitems_read/written will refer to the state of the data stream at the +start of the work function. We must therefore add to this value the +current relative offset in the data stream. So if we are iterating i over all output items, we would write the stream tag to output ports +at nitems_written(0)+i for the 0th output port. +

+

Stream Tags API

+

The stream tags API is split into two parts: adding tags to a stream, +and getting tags from a stream. +Note that the functions described below are only meant to be accessed +within a call to general_work/work. While they can be called at other points +in time by a block, the behavior outside of work is undefined without +exact knowledge of the item counts in the buffers. +

+

Adding a Tag to a Stream

+

We add a tag to a particular output stream of the block using: +

+
  • gr::block::add_item_tag: Adds an item tag to a particular output port using a gr::tag_t data type or by specifying the tag values.
+

We can output them to multiple output streams if we want, but to do so +means calling this function once for each port. This function can be +provided with a gr::tag_t data type, or each value of the tag can be +explicitly given. +

Again, a tag is defined as: +

+
  1. offset: The offset, in absolute item time, of the tag in the data stream.
  2. +
  3. key: the PMT symbol identifying the type of tag.
  4. +
  5. value: the PMT holding the data of the tag.
  6. +
  7. srcid: (optional) the PMT symbol identifying the block which created the tag.
+

We can create a gr::tag_t structure to hold all of the above +information of a tag, which is probably the easiest/best way to do +it. The gr::tag_t struct is defined as having the same members as in +the above list. To add a gr::tag_t tag to a stream, use the function: +

+
 void add_item_tag(unsigned int which_output, const tag_t &tag);
+
+

The secondary API allows us to create a tag by explicitly listing all +of the tag information in the function call: +

+
 void add_item_tag(unsigned int which_output,
+                   uint64_t abs_offset,
+                   const pmt::pmt_t &key,
+                   const pmt::pmt_t &value,
+                   const pmt::pmt_t &srcid=pmt::PMT_F);
+
+

In Python, we can add a tag to a stream using one of the following: +

+
add_item_tag(which_output, abs_offset, key, value)
+add_item_tag(which_output, abs_offset, key, value, srcid)
+
+

Note that the key and value are both PMTs. To create a string type PMT you can use pmt.intern("example_key"). +

Consider the following flowgraph as an example. We will have an embedded python block insert stream tags at random intervals, and view these tags on the QT GUI Time sink. +

+

Add the following code into the embedded python block. This code outputs the same signal as in the input, except with tags on randomly selected input samples: +

+
 import numpy as np
+ from gnuradio import gr
+ import pmt
+ 
+ class blk(gr.sync_block):
+     def __init__(self):
+         gr.sync_block.__init__(
+             self,
+             name='Embedded Python Block',
+             in_sig=[np.float32],
+             out_sig=[np.float32]
+         )
+ 
+     def work(self, input_items, output_items):
+         for indx, sample in enumerate(input_items[0]): # Enumerate through the input samples in port in0
+             if np.random.rand() > 0.95: # 5% chance this sample is chosen
+                 key = pmt.intern("example_key")
+                 value = pmt.intern("example_value")
+                 self.add_item_tag(0, # Write to output port 0
+                          self.nitems_written(0) + indx, # Index of the tag in absolute terms
+                          key, # Key of the tag
+                          value # Value of the tag
+                 )
+                 # note: (self.nitems_written(0) + indx) is our current sample, in absolute time
+         output_items[0][:] = input_items[0] # copy input to output
+         return len(output_items[0])
+
+

You may expect an output similar to the screenshot of the time sink below. +

+

+

Getting tags from a Stream

+

To get tags from a particular input stream, we have two +functions we can use: +

+
  1. gr::block::get_tags_in_range: Gets all tags from a particular input port between a certain range of items (in absolute item time).
+
  1. gr::block::get_tags_in_window: Gets all tags from a particular input port between a certain range of items (in relative item time within the work function).
+

The difference between these functions is working in absolute item +time versus relative item time. Both of these pass back vectors of +gr::tag_t, and they both allow +specifying a particular key (as a PMT symbol) to filter against +(or the fifth argument can be left out to search for all keys). +Filtering for a certain key reduces the effort inside the work function +for getting the right tag's data. +

For example, this call just returns any tags between the given range of items: +

+
 void get_tags_in_range(std::vector<tag_t> &v,
+                        unsigned int which_input,
+                        uint64_t abs_start,
+                        uint64_t abs_end);
+
+

Adding a fifth argument to this function allows us to filter on the +key key. +

+
 void get_tags_in_range(std::vector<tag_t> &v,
+                        unsigned int which_input,
+                        uint64_t abs_start,
+                        uint64_t abs_end,
+                        const pmt::pmt_t &key);
+
+

In Python, the main difference from the C++ function is that instead of having the first +argument be a vector where the tags are stored, the Python version +just returns a list of tags. We would use it like this: +

+
 def work(self, input_items, output_items):
+     ....
+     tags = self.get_tags_in_window(which_input, rel_start, rel_end)
+     ....
+
+

If you want to grab all tags on the samples currently being processed by work(), on input port 0, here's a minimal example of doing that: +

+
 import numpy as np
+ from gnuradio import gr
+ import pmt
+ 
+ class blk(gr.sync_block):
+     def __init__(self):
+         gr.sync_block.__init__(self,name='Read Tags', in_sig=[np.float32], out_sig=None)
+ 
+     def work(self, input_items, output_items):
+         tags = self.get_tags_in_window(0, 0, len(input_items[0]))
+         for tag in tags:
+             key = pmt.to_python(tag.key) # convert from PMT to python string
+             value = pmt.to_python(tag.value) # Note that the type(value) can be several things, it depends what PMT type it was
+             print 'key:', key
+             print 'value:', value, type(value)
+             print ''
+         return len(input_items[0])
+
+

Tag Propagation

+

We now know how to add tags to streams, and how to read them. But what happens to tags after they were read? And what happens to tags that aren't used? After all, there are many blocks that don't care about tags at all. +

The answer is: It depends on the tag propagation policy of a block what happens to tags that enter it.
+There are three policies to choose from: +

+
  1. All-to-All: all tags from any input port are replicated to all output ports
  2. +
  3. One-to-One: tags from input port i are only copied to output port i (depends on num inputs = num outputs).
  4. +
  5. Dont: Does not propagate tags. Tags are either stopped here or the work function recreates them in some manner.
+

The default behavior of a block is the 'All-to-All' method of +propagation. +

We generally set the tag propagation policy in the block's constructor using @set_tag_propagation_policy +

+
 void set_tag_propagation_policy(tag_propagation_policy_t p);
+
+

See the gr::block::tag_propagation_policy_t documentation for details +on this enum type. +

When the tag propagation policy is set to TPP_ALL_TO_ALL or TPP_ONE_TO_ONE, the GNU Radio scheduler uses any information available to figure out which output item corresponds to which input item. The block may read them and add new tags, but existing tags are automatically moved downstream in a manner deemed appropriate. +

When a tag is propagated through a block that has a rate change, the +item's offset in the data stream will change. The scheduler uses the +block's gr::block::relative_rate concept to perform the update on the +tag's offset value. The relative rate of a block determines the +relationship between the input rate and output rate. Decimators that +decimate by a factor of D have a relative rate of 1/D. +

Synchronous blocks (gr::sync_block), decimators (gr::sync_decimator), +and interpolators (gr::sync_interpolator) all have pre-defined and +well-understood relative rates. A standard gr::block has a default +relative rate of 1.0, but this must be set if it does not work this +way. Often, we use a gr::block because we have no pre-conceived notion +of the number of input to output items. If it is important to pass +tags through these blocks that respect the change in item value, we +would have to use the TPP_DONT tag propagation policy and handle the +propagation internally. +

In no case is the value of the tag modified when propagating through a +block. This becomes relevant when using Tagged Stream Blocks. +

As an example, consider an interpolating block. See the following flow graph: +

tags_interp.grc.png +

tags_interp.png
+

As you can tell, we produce tags on every 10th sample, and then pass them through a block that repeats every sample 100 times. Tags do not get repeated with the standard tag propagation policies (after all, they're not tag manipulation policies), so the scheduler takes care that every tag is put on the output item that corresponds to the input item it was on before. Here, the scheduler makes an educated guess and puts the tag on the first of 100 items. +

Note: We can't use one QT GUI Time Sink for both signals here, because they are running at a different rate. Note the time difference on the x-axis! +

On decimating blocks, the behavior is similar. Consider this very simple flow graph, and the position of the samples: +

tags_decim.grc.png +

tags_decim.png
+


+We can see that no tags are destroyed, and tags are indeed spaced at one-tenth of the original spacing of 100 items. Of course, the actual item that was passed through the block might be destroyed, or modified (think of a decimating FIR filter). +

In fact, this works with any rate-changing block. Note that there are cases where the relation of tag positions of in- and output are ambiguous, the GNU Radio scheduler will then try and get as close as possible. +

Here's another interesting example: Consider this flow graph, which has a delay block, and the position of the tags after it: +

tags_ramp_delay.grc.png +

tags_ramp_delay.png
+


+Before the delay block, tags were positioned at the beginning of the ramp. After the delay, they're still in the same position! Would we inspect the source code of the delay block, we'd find that there is absolutely no tag handling code. Instead, the block declares a delay to the scheduler, which then propagates tags with this delay. +

Using these mechanisms, we can let GNU Radio handle tag propagation for a large set of cases. For specialized or corner cases, there is no option than to set the tag propagation policy to TPP_DONT and manually propagate tags (actually, there's another way: Say we want most tags to propagate normally, but a select few should be treated differently. We can use remove_item_tag() (DEPRECATED. Will be removed in 3.8.) to remove these tags from the input; they will then not be propagated any more even if the tag propagation policy is set to something other than TPP_DONT. But that's more advanced use and will not be elaborated on here). +

+

Notes on How to Use Tags

+

Tags can be very useful to an application, and their use is +spreading. USRP sources generate tag information on the time, sample +rate, and frequency of the board if anything changes. We have a metadata +file source/sink (see Metadata_Information) that use tags to store +information about the data stream. But there are things to think about +when using tags in a block. +

First, when tags are not being used, there is almost no effect on the +scheduler. However, when we use tags, we add overhead by getting and +extracting tags from a data stream. We also use overhead in +propagating the tags. For each tag, each block must copy a vector of +tags from the output port(s) of one block to the input port(s) of the +next block(s). These copy operations can add up. +

The key is to minimize the use of tags. Use them when and only when +necessary and try to provide some control over how tags are generated +to control their frequency. A good example is the USRP source, which +generates a time tag. If it generated a tag with every sample, we +would have thousands of tags per second, which would add a significant +amount of overhead. This is because if we started at time t0 at +sample rate sr, then after N samples, we know that +we are now at time t0 + N/sr. So continuously producing new +tags adds no information. +

The main issue we need to deal with in the above situation is when +there is a discontinuity in the packets received from the USRP. Since +we have no way of knowing in the flowgraph how many samples were +potentially lost, we have lost track of the timing information. The +USRP driver recognizes when packets have been dropped and uses this to +queue another tag, which allows us to resync. Likewise, any point the +sample rate or frequency changes, a new tag is issued. +

+

Example Flowgraph

+

Let's have a look at a simple example: +

tut5_tagstest_fg.png +

In this flow graph, we have two sources: A sinusoid and a tag strobe. A tag strobe is a block that will output a constant tag, in this case, on every 1000th item (the actual value of the items is always zero). Those sources get added up. The signal after the adder is identical to the sine wave we produced, because we are always adding zeros. However, the tags stay attached to the same position as they were coming from the tag strobe! This means every 1000th sample of the sinusoid now has a tag. The QT scope can display tags, and even trigger on them. +

tut5_tagstest_scope.png
+

We now have a mechanism to randomly attach any metadata to specific items. There are several blocks that use tags. One of them is the UHD Sink block, the driver used for transmitting with USRP devices. It will react to tags with certain keys, one of them being tx_freq, which can be used to set the transmit frequency of a USRP while streaming. +

+

Adding tags to the QPSK demodulator

+

Going back to our QPSK demodulation example, we might want to add a feature to tell downstream blocks that the demodulation is not going well. Remember the output of our block is always hard-decision, and we have to output something. So we could use tags to notify that the input is not well formed, and that the output is not reliable. +

As a failure criterion, we discuss the case where the input amplitude is too small, say smaller than 0.01. When the amplitude drops below this value, we output one tag. Another tag is only sent when the amplitude has recovered, and rises back above the threshold. We extend our work function like this: +

+
if (std::abs(in[i]) < 0.01 and not d_low_ampl_state) {
+    add_item_tag(0, // Port number
+                 nitems_written(0) + i, // Offset
+                 pmt::mp("amplitude_warning"), // Key
+                 pmt::from_double(std::abs(in[i])) // Value
+    );
+    d_low_ampl_state = true;
+}
+else if (std::abs(in[i]) >= 0.01 and d_low_ampl_state) {
+    add_item_tag(0, // Port number
+        nitems_written(0) + i, // Offset
+        pmt::mp("amplitude_recovered"), // Key
+        pmt::PMT_T // Value
+    );
+    d_low_ampl_state = false; // Reset state
+}
+

In Python, the code would look like this (assuming we have a member of our block class called d_low_ampl_state): +

+
# The vector 'in' is called 'in0' here because 'in' is a Python keyword
+if abs(in0[i]) < 0.01 and not d_low_ampl_state:
+    self.add_item_tag(0, # Port number
+                 self.nitems_written(0) + i, # Offset
+                 pmt.intern("amplitude_warning"), # Key
+                 pmt.from_double(numpy.double(abs(in0[i]))) # Value
+                 # Note: We need to explicitly create a 'double' here,
+                 # because in0[i] is an explicit 32-bit float here
+    )
+    self.d_low_ampl_state = True
+elif abs(in0[i]) >= 0.01 and d_low_ampl_state:
+    self.add_item_tag(0, # Port number
+                 self.nitems_written(0) + i, # Offset
+                 pmt.intern("amplitude_recovered"), # Key
+                 pmt.PMT_T # Value
+    )
+    self.d_low_ampl_state = False; // Reset state
+

We can also create a tag data type tag_t and directly pass this along: +

+
if (std::abs(in[i]) < 0.01 and not d_low_ampl_state) {
+    tag_t tag;
+    tag.offset = nitems_written(0) + i;
+    tag.key = pmt::mp("amplitude_warning");
+    tag.value = pmt::from_double(std::abs(in[i]));
+    add_item_tag(0, tag);
+    d_low_ampl_state = true;
+}
+

Here's a flow graph that uses the tagged version of the demodulator. We input 20 valid QPSK symbols, then 10 zeros. Since the output of this block is always either 0, 1, 2 or 3, we normally have no way to see if the input was not clearly one of these values. +

Demod_with_tags.grc.png +

Here's the output. You can see we have tags on those values which were not from a valid QPSK symbol, but from something unreliable. +

Demod_with_tags.png
+

+

Use case: FIR filters

+

Assume we have a block that is actually an FIR filter. We want to let GNU Radio handle the tag propagation. How do we configure the block? +

Now, an FIR filter has one input and one output. So, it doesn't matter if we set the propagation policy to TPP_ALL_TO_ALL or TPP_ONE_TO_ONE, and we can leave it as the default. The items going in and those coming out are different, so how do we match input to output? Since we want to preserve the timing of the tag position, we need to use the filter's group delay as a delay for tags (which for this symmetric FIR filter is (N-1)/2, where N is the number of filter taps). Finally, we might be interpolating, decimating or both (say, for sample rate changes) and we need to tell the scheduler about this as well. +

+ + + + +
+
+ +
+
+ +
+

Navigation menu

+
+ + + +
+ + + + + + +
+
+ + + + + + + + + +
+
+ +
+ + + + + + + + + + + + +
+ +
+ + + + + + \ No newline at end of file diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/liquid-dsp-readme.rst b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/liquid-dsp-readme.rst new file mode 100644 index 0000000..3817e1d --- /dev/null +++ b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/liquid-dsp-readme.rst @@ -0,0 +1,451 @@ + +liquid-dsp documentation +======================== + +Software-Defined Radio Digital Signal Processing Library - +`https://liquidsdr.org `_ + +liquid-dsp is a free and open-source digital signal processing (DSP) +library designed specifically for software-defined radios on embedded +platforms. The aim is to provide a lightweight DSP library that does not +rely on a myriad of external dependencies or proprietary and otherwise +cumbersome frameworks. All signal processing elements are designed to be +flexible, scalable, and dynamic, including filters, filter design, +oscillators, modems, synchronizers, complex mathematical operations, and +much more. + +.. code-block:: c + + // get in, process data, get out + #include + int main() { + unsigned int M = 4; // interpolation factor + unsigned int m = 12; // filter delay [symbols] + float As = 60.0f; // filter stop-band attenuation [dB] + + // create interpolator from prototype + firinterp_crcf interp = firinterp_crcf_create_kaiser(M,m,As); + float complex x = 1.0f; // input sample + float complex y[M]; // interpolated output buffer + + // repeat on input sample data as needed + { + firinterp_crcf_execute(interp, x, y); + } + + // destroy interpolator object + firinterp_crcf_destroy(interp); + return 0; + } + + +For more information, please refer to the +`documentation `_ online. + +Installation and Dependencies +============================= + +liquid-dsp only relies on ``libc`` and ``libm`` (standard C and math) +libraries to run; however liquid will take advantage of other libraries +(such as `FFTW `_) if they are available. +Starting with version 1.7.0, liquid-dsp has moved to the +`CMake `_ build system which can be installed with +``brew install cmake`` on macOS, +``sudo apt-get install cmake`` on Debian variants. + +Installation +------------ + +The recommended way to obtain the source code is to clone the entire +`repository `_ from +`GitHub `_: + +.. code-block:: bash + + git clone git://github.com/jgaeddert/liquid-dsp.git + +Building and installing the main library is a simple as + +.. code-block:: bash + + mkdir build + cd build + cmake .. + make + sudo make install + +If you are installing on Linux for the first time, you will also need +to rebind your dynamic libraries with ``sudo ldconfig`` to make the +shared object available. +This is not necessary on macOS. + + +Run all test scripts +-------------------- + +Source code validation is a critical step in any software library, +particularly for verifying the portability of code to different +processors and platforms. Packaged with liquid-dsp are a number of +automatic test scripts to validate the correctness of the source code. +The test scripts are located under each module's ``tests/`` directory and +take the form of a C source file. When configured with ``BUILD_AUTOTESTS`` +enabled, these tests are parsed, compiled, and linked into an executable +which will run the tests. + +.. code-block:: bash + + ./xautotest + # ... + # autotest seed: 1773004345 + # ================================== + # PASSED ALL 716651 CHECKS + # ================================== + +There are currently more than 700,000 checks across 1,316 tests to verify +functional correctness. Drop me a line if these aren't running on your platform. + +Testing Code Coverage +--------------------- + +In addition to the full test suite, you can configure ``gcc`` to export symbol +files to check for code coverage and then use ``gcovr`` to generate a full +report of precisely which lines are covered in the autotests. These symbol +files aren't generated by default and need to be enabled at compile-time +through a CMake option: + +.. code-block:: bash + + cmake -DBUILD_AUTOTESTS=ON -DCOVERAGE=ON .. + +A coverage report can be generated by running the autotests and running +`gcovr `_: + +.. code-block:: bash + + make -j4 xautotest + ./xautotest -q -o autotest.json + cd .. + gcovr --filter="src/.*/src/.*.c" --print-summary + # ... + # ------------------------------------------------------------------------------ + # TOTAL 20922 17396 83% + # ------------------------------------------------------------------------------ + # lines: 83.1% (17396 out of 20922) + # functions: 63.9% (1787 out of 2798) + # branches: 64.8% (5762 out of 8896) + +Examples +-------- + +Nearly all signal processing elements have a corresponding example in +the ``examples/`` directory. Most example scripts generate an output +``.m`` file for plotting with `GNU octave `_ +All examples are built as stand-alone programs and can be compiled with +the ``BUILD_EXAMPLES`` CMake flag: + +.. code-block:: bash + + cmake -DBUILD_EXAMPLES=ON .. + make + ./examples/modem_example -m qpsk + # + # 0 : 0.70710677 + j* 0.70710677 + # 1 : -0.70710677 + j* 0.70710677 + # 2 : 0.70710677 + j* -0.70710677 + # 3 : -0.70710677 + j* -0.70710677 + # num sym errors: 0 / 4 + # num bit errors: 0 / 8 + # results written to modem_example.m. + +Sometimes, however, it is useful to build one example individually. +This can be accomplished by directly targeting its binary +(e.g. ``make examples/modem_example``). The example then can be run at the +command line, viz. ``./examples/modem_example``. + +Benchmarking Tool +----------------- + +Packaged with liquid are benchmarks to determine the speed each signal +processing element can run on your machine. Initially the tool provides +an estimate of the processor's clock frequency and will then estimate +the number of trials so that each benchmark will take between 50 and +500 ms to run. You can build and run the benchmark program with the +following command: + +.. code-block:: bash + + make bench + +Linking from C++ +---------------- + +Compiling and linking to C++ programs is straightforward. +Just include ```` before ```` and use +``std::complex`` in favor of ``float complex``. +Here is the same example as the one above but in C++ instead of C: + +.. code-block:: c++ + + // get in, process data, get out + #include + #include + int main() { + unsigned int M = 4; // interpolation factor + unsigned int m = 12; // filter delay [symbols] + float As = 60.0f; // filter stop-band attenuation [dB] + + // create interpolator from prototype + firinterp_crcf interp = firinterp_crcf_create_kaiser(M,m,As); + std::complex x = 1.0f; // input sample + std::complex y[M]; // interpolated output buffer + + // repeat on input sample data as needed + { + firinterp_crcf_execute(interp, x, y); + } + + // destroy interpolator object + firinterp_crcf_destroy(interp); + return 0; + } + +Linking from External Project +----------------------------- + +Installing with `CMake `_ provides an +`exportable interface `_ +(``liquid::liquid``) that allows easy integration into external applications. +For example, say you have a simple application in ``main.c`` that requires +liquid-dsp as a dependency: + +.. code-block:: c + + // main.c - test linking to liquid + #include + #include + #include + + int main() + { + // create resampling object, print and return + printf("creating test object...\n"); + resamp_crcf q = resamp_crcf_create(0.12345f, 12, 0.25f, 60.0f, 256); + resamp_crcf_print(q); + resamp_crcf_destroy(q); + return 0; + } + +You now have two options within your project ``CMakeLists.txt`` file: + +1. clone, build, and install using CMake, then use + `find_package `_ +2. dynamically fetch liquid-dsp with + `FetchContent `_ + to pull down the source and build completely within your application + +Your ``CMakeLists.txt`` file might look something like this: + +.. code-block:: cmake + + # CMakeLists.txt - test finding package and linking against it + cmake_minimum_required(VERSION 3.10) + project(liquid_test C) + + # option 1: check for local installation + #find_package(liquid REQUIRED) + + # option 2: dynamically fetch content + include(FetchContent) + FetchContent_Declare( + liquid + GIT_REPOSITORY https://github.com/jgaeddert/liquid-dsp.git + GIT_TAG v1.7.0 + ) + set(BUILD_AUTOTESTS OFF CACHE INTERNAL "Disable building liquid tests") + set(BUILD_BENCHMARKS OFF CACHE INTERNAL "Disable building liquid benchmarks") + set(BUILD_EXAMPLES OFF CACHE INTERNAL "Disable building liquid examples") + FetchContent_MakeAvailable(liquid) + # FetchContent_Populate(liquid) <- older policy, see CMP0169 + + add_executable(main main.c) + target_link_libraries(main liquid) + +You can then compile and run your application the typical way: + +.. code-block:: bash + + mkdir build + cd build + cmake .. + make + ./main + # creating test object... + # + + +PlatformIO +---------- + +Cross-compling for embedded platforms is most easily achieved with +`platformio `_. +Just add ``liquid-dsp`` to your ``platform.io`` list of dependencies: + +.. code-block:: ini + + [env:native] + platform = native + lib_deps = https://github.com/jgaeddert/liquid-dsp.git + +To test this, compile the example program for a +`Raspberry Pi Pico microcontroller `_: + +.. code-block:: bash + + # create a virtual environment, install platformio, and compile an example + virtualenv pio + source pio/bin/activate + pip install platformio + pio ci --lib="." --board=pico examples/platformio_example.c + # ... + # Generating UF2 image + # elf2uf2 ".pio/build/pico/firmware.elf" ".pio/build/pico/firmware.uf2" + # Checking size .pio/build/pico/firmware.elf + # Advanced Memory Usage is available via "PlatformIO Home > Project Inspect" + # RAM: [== ] 15.5% (used 41820 bytes from 270336 bytes) + # Flash: [ ] 0.2% (used 5196 bytes from 2097152 bytes) + # Building .pio/build/pico/firmware.bin + # ===================== [SUCCESS] Took 23.63 seconds ===================== + +Build +----- + +Here is a table of CMake options available for configuring liquid: + ++------------------------+---------+--------------------------------------------------------------------+ +| Option | Default | Description | ++========================+=========+====================================================================+ +| ``BUILD_EXAMPLES`` | ON | Compile example programs | ++------------------------+---------+--------------------------------------------------------------------+ +| ``BUILD_AUTOTESTS`` | ON | Parse and compile autotests into executable binary | ++------------------------+---------+--------------------------------------------------------------------+ +| ``BUILD_BENCHMARKS`` | ON | Parse and compile benchmarks into executable binary | ++------------------------+---------+--------------------------------------------------------------------+ +| ``BUILD_SHARED_LIBS`` | ON | Build shared library | ++------------------------+---------+--------------------------------------------------------------------+ +| ``BUILD_STATIC_LIBS`` | OFF | Build static library | ++------------------------+---------+--------------------------------------------------------------------+ +| ``ENABLE_SIMD`` | ON | Enable use of single instruction, multiple data (SIMD) extensions | ++------------------------+---------+--------------------------------------------------------------------+ +| ``FIND_SIMD`` | ON | Try to find available SIMD instruction sets on host computer | ++------------------------+---------+--------------------------------------------------------------------+ +| ``FIND_THREADS`` | ON | Try to find available threading library for concurrency | ++------------------------+---------+--------------------------------------------------------------------+ +| ``FIND_FFTW`` | ON | Try to find `FFTW `_ if available | ++------------------------+---------+--------------------------------------------------------------------+ +| ``BUILD_SANDBOX`` | OFF | Compile sandbox (testing) programs | ++------------------------+---------+--------------------------------------------------------------------+ +| ``BUILD_DOC`` | OFF | Generate documentation | ++------------------------+---------+--------------------------------------------------------------------+ +| ``COVERAGE`` | OFF | Set flags to enable code coverage testing | ++------------------------+---------+--------------------------------------------------------------------+ +| ``ENABLE_LOGGING`` | ON | Enable global logging capabilities | ++------------------------+---------+--------------------------------------------------------------------+ +| ``LOGGING_LEVEL`` | trace | Set minimum logging level to compile; | +| | | select from "trace", "debug", "info", "warn", "error", or "fatal" | ++------------------------+---------+--------------------------------------------------------------------+ +| ``ENABLE_COLOR`` | ON | Enable use of color terminal output | ++------------------------+---------+--------------------------------------------------------------------+ + +For example, if you want to benchmark how fast a vector dot product +runs without SIMD extensions, you could run the following: + +.. code-block:: bash + + cmake -DENABLE_SIMD=OFF -DBUILD_BENCHMARKS=ON .. + make + ./benchmark -s dotprod_rrrf + +Available Modules +----------------- + +* **agc**: automatic gain control, received signal strength +* **audio**: source audio encoders/decoders: cvsd, filterbanks +* **buffer**: internal buffering, circular/static, ports (threaded) +* **channel**: additive noise, multi-path fading, carrier phase/frequency + offsets, timing phase/rate offsets +* **core**: core functionality (error handling) +* **dotprod**: inner dot products (real, complex), vector sum of squares +* **equalization**: adaptive equalizers: least mean-squares, recursive + least squares, semi-blind +* **fec**: basic forward error correction codes including several + Hamming codes, single error correction/double error detection, + Golay block code, as well as several checksums and cyclic + redundancy checks, interleaving, soft decoding +* **fft**: fast Fourier transforms (arbitrary length), discrete sin/cos + transforms +* **filter**: finite/infinite impulse response, polyphase, hilbert, + interpolation, decimation, filter design, resampling, symbol + timing recovery +* **framing**: flexible framing structures for amazingly easy packet + software radio; dynamically adjust modulation and coding on the + fly with single- and multi-carrier framing structures +* **math**: transcendental functions not in the C standard library + (gamma, besseli, etc.), polynomial operations (curve-fitting, + root-finding, etc.) +* **matrix**: basic math, LU/QR/Cholesky factorization, inversion, + Gauss elimination, Gram-Schmidt decomposition, linear solver, + sparse matrix representation +* **modem**: modulate, demodulate, PSK, differential PSK, QAM, optimal + QAM, as well as analog and non-linear digital modulations GMSK) +* **multichannel**: filterbank channelizers, OFDM +* **nco**: numerically-controlled oscillator: mixing, frequency + synthesis, phase-locked loops +* **optim**: (non-linear optimization) Newton-Raphson, evoluationary + algorithms, gradient descent, line search +* **quantization**: analog/digital converters, compression/expansion +* **random**: (random number generators) uniform, exponential, gamma, + Nakagami-m, Gauss, Rice-K, Weibull +* **sequence**: linear feedback shift registers, complementary codes, + maximal-length sequences +* **utility**: useful miscellany, mostly bit manipulation (shifting, + packing, and unpacking of arrays) +* **vector**: generic vector operations + +License +------- + +liquid projects are released under the X11/MIT license. +By default, this project will try to link to +`FFTW `_ +if it is available on your build platform. +Because FFTW starting with version 1.3 is +`licensed `_ +under the `GNU General Public License v2 `_ +this unfortunately means that (and I'm clearly not a lawyer, here) +you cannot distribute ``liquid-dsp`` without also distributing the source code +if you link to FFTW. +This is a similar situation with the classic +`libfec `_ +which uses the +`GNU Lesser GPL `_. +Finally, ``liquid-dsp`` makes extensive use of GNU +`autoconf `_, +`automake `_, +and related tools. +These are fantastic libraries with amazing functionality and their authors +should be lauded for their efforts. +In a similar vain, much the software I write for a living I give away for +free; +however I believe in more permissive licenses to allow individuals the +flexibility to use software with fewer limitations. +If these restrictions are not acceptable, ``liquid-dsp`` can be compiled and run +without use of these external libraries, albeit a bit slower and with limited +functionality. + +Short version: this code is copyrighted to me (Joseph D. Gaeddert), +I give you full permission to do whatever you want with it except remove my +name from the credits. +Seriously, go nuts! but take caution when linking to other libraries with +different licenses. +See the `license `_ for specific terms. + diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/liquid-dsp-releases.html b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/liquid-dsp-releases.html new file mode 100644 index 0000000..8db3429 --- /dev/null +++ b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/liquid-dsp-releases.html @@ -0,0 +1,3151 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Releases · jgaeddert/liquid-dsp · GitHub + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ Skip to content + + + + + + + + + + + +
+
+ + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + + + +
+ + + + + +
+ + + + + + + + + +
+
+
+ + + + + + + + + + + + + + + + + + +
+ + + + + + +

Releases: jgaeddert/liquid-dsp

+
+
+
+ + + +
+ +
+
+ + + +
+
+
+
+ +
+
+

Version 1.7.0

+ +
+
+
+ + 01 Feb 22:41 + +
+ +
+ @jgaeddert + jgaeddert +
+ + +
+
+ + + + + v1.7.0 + +
+ +
+ +
+ + + + + +
+
+
+

+ Choose a tag to compare +

+ +
+
+ +
+
+
+ + + +
+ + + + + + +
+ + +
+
+
+ + +
+
+ + +
+
+ + + + + +
+ +
+ +
+ +
+
+
+
+
+ + +
+
+ +
+
+
+ Version 1.7.0 + + Latest +
+ + +
+
+
+ +
+ Latest +
+
+ + +
+
+
+ + +
+
+ +

Version 1.7.0 includes support for the CMake build system. While the existing autotools environment will be supported for a time, this will be removed in a future release. This version also includes many changes to improve testing coverage across all interfaces, and make the interfaces across objects consistent.

+
    +
  • build +
      +
    • migrated to the CMake build system, including building examples, sandbox programs, benchmarks, autotests, and detecting SIMD instruction extensions.
    • +
    • increased testing across the board including automated memory validation with valgrind
    • +
    • detected and fixed memory leaks for numerous methods and testing harnesses including bessel_azpdf (thanks, @andreasbombe), ofdmflexframesync, qdsync, qs1dsearch (thanks, @nowls), framesync64, eqrls, and fskdem
    • +
    • added new LIQUID_ENOCONV error type to identify errors where algorithms did not converge
    • +
    • added new LIQUID_ENOIMP error type to identify methods that are not yet implemented
    • +
    • replaced warnings with internal error handling for many existing object methods
    • +
    • fixed SONAME not set in shared library (thanks again, @andreasbombe)
    • +
    • migrated README from markdown to re-structured text
    • +
    +
  • +
  • framing +
      +
    • added more description to method definitions such as qpacketmodem and qdetector
    • +
    • dsssframe64: extended functionality to use qdsync, added standard methods such as copy(), added interfaces for specifying thresholds, reduced default spreading gain
    • +
    +
  • +
  • filter +
      +
    • firinterp: added flush() method to run zeros through filter
    • +
    • rresamp: allow for default bandwidth with an input of -1
    • +
    +
  • +
  • nco +
      +
    • fixed issue where frequency was being set improperly, added more extensive testing
    • +
    • improved the NCO object with VCO precision (thanks, @ArtemPisarenko)
    • +
    +
  • +
  • random +
      +
    • added more extensive testing for various distributions to ensure values are generated properly
    • +
    +
  • +
+
+ +
+ +
+
+
+

Version 1.6.0

+ +
+
+
+ + 20 Jun 02:37 + +
+ +
+ @jgaeddert + jgaeddert +
+ + +
+
+ + + + + v1.6.0 + +
+ +
+
+ + + + 8bf87b6 + +
+
+ + + + + +
+
+
+

+ Choose a tag to compare +

+ +
+
+ +
+
+
+ + + +
+ + + + + + +
+ + +
+
+
+ + +
+
+ + +
+
+ + + + + +
+ +
+ +
+ +
+
+
+
+
+ + +
+
+ +
+
+
+ Version 1.6.0 + + +
+ + +
+
+
+ +
+ +
+
+ + +
+
+
+ + +
+
+ +

Version 1.6.0 includes a new qdsync object to greatly simplify the frame synchronization process in liquid, allowing for both detection and channel impairment correction with a simplified interface. Additionally, code coverage has been increased to 85% across the entire project, with numerous bug fixes, stability improvements, and massive testing enhancements. From an "architectural" standpoint, objects have been migrated to use standard methods for consistency.

+
    +
  • build +
      +
    • increased code coverage to 85% globally across entire project. This
      +is the single largest effort included in this version and touches
      +most modules in some way, most particularly the framing objects
    • +
    • cleaning build to remove compiler warnings (e.g. unused variables)
    • +
    • stripped version number off archive
    • +
    +
  • +
  • dotprod +
      +
    • added support for AVX512-F (thanks, @vankxr!)
    • +
    +
  • +
  • framing +
      +
    • added numerous tests to increase coverage to 84%
    • +
    • framesync64: using new qdsync object for simplified operation
    • +
    • qdsync: new frame detector and synchronizer to much more easily
      +support frame processing. The object not only detects the frame, but
      +also provides an initial carrier frequency, phase, and timign offset,
      +and also corrects for these impairments, passing the results to the
      +user in a clean callback function.
    • +
    +
  • +
  • modem +
      +
    • cpfskmod: increasing phase stability for long runs
    • +
    +
  • +
  • multichannel +
      +
    • added numerous tests to increase coverage to 88%
    • +
    +
  • +
  • optim +
      +
    • added numerous tests to increase coverage to 92%
    • +
    +
  • +
  • sequence +
      +
    • msequence: extended support for state variables up to m=31, reversed
      +order for generator polynomial and internal state definition to be
      +more consistent with literature and readily-available genpolys
    • +
    +
  • +
+
+ +
+ +
+
+
+

Version 1.5.0

+ +
+
+
+ + 20 Nov 21:07 + +
+ +
+ @jgaeddert + jgaeddert +
+ + +
+
+ + + + + v1.5.0 + +
+ +
+ +
+ + + + + +
+
+
+

+ Choose a tag to compare +

+ +
+
+ +
+
+
+ + + +
+ + + + + + +
+ + +
+
+
+ + +
+
+ + +
+
+ + + + + +
+ +
+ +
+ +
+
+
+
+
+ + +
+
+ +
+
+
+ Version 1.5.0 + + +
+ + +
+
+
+ +
+ +
+
+ + +
+
+
+ + +
+
+ +

This release includes substantially improved testing coverage, deep copy() methods for nearly all objects, improved speed, and resolves a number of issues and pull requests.

+
    +
  • build +
      +
    • added support for PlatformIO for embeedded
      +development (thanks, @jcw!)
    • +
    • incorporated recursive copy() methods to objects to facilitate c++ copy
      +constructors for bindings; now all objects can be deep copied to a new
      +object with their entire memory and state preserved
    • +
    • added convenience method to malloc and copy memory arrays
    • +
    • improved support for error codes across objects and methods
    • +
    • cleaned up spelling errors across project (thanks, @nowls!)
    • +
    • scrubbed function argument variable names to avoid underscore followed
      +by a capital letter, causing trouble with pre-compiler processing
    • +
    • added basic test to check linking to installed library, incorporating
      +into CI/CD pipelines
    • +
    • added more example programs
    • +
    +
  • +
  • autotest +
      +
    • increased coverage testing (81% across entire project)
    • +
    • added the ability to "hammer" a particular test by running repeatedly on
      +incremental seeds to assess specific edge cases (e.g. with random data)
    • +
    • added timer to show execution time for each test and identify areas for
      +speed improvements
    • +
    • added methods for testing spectral response of various fields
    • +
    • added special directory for storing output logs: autotest/logs/
    • +
    +
  • +
  • benchmark +
      +
    • replacing old C-based benchmark comparison with simpler python version
    • +
    +
  • +
  • dotprod +
      +
    • added support for AVX SIMD in vector dot products (thanks, @vankxr!)
    • +
    +
  • +
  • fft +
      +
    • adding macro to allow for aligned memory allocation if FFTW is used
      +(thanks, @nowls!)
    • +
    +
  • +
  • filter +
      +
    • added new halfband filter design using Parks-McClellan algorithm and
      +qs1dsearch method to provide as exact a specification as possible
    • +
    • added method to retrieve filter response from coefficients array
    • +
    • dds: adding methods to get/set scale
    • +
    • firhilb, iirhilb: added block processing method
    • +
    • msresamp, resamp: adding method to provide the exact number of output
      +samples with provided input size
    • +
    • msresamp2, resamp2: using better halfband filter design for exact user
      +specifications
    • +
    • resamp: adding methods to get/set scale, fixing filter bank resolution
      +(was hard-coded, now respects user configuration)
    • +
    +
  • +
  • framing +
      +
    • framesync64: added methods to set callback and userdata (context)
      +fields, adding support for exporting debugging files for post-analysis
      +as well as python script for processing, adding better estimate of
      +error vector magnitude
    • +
    • msource: added convenience method to recall number of samples generated
      +by each source
    • +
    • ofdmflexframesync: added methods to set callback and userdata (context)
      +fields
    • +
    • qpacketmodem: returning much better estimate of error vector magnitude
    • +
    • qsource: fixed issue with carrier frequency adjustment
    • +
    +
  • +
  • optim +
      +
    • added qs1dsearch object to perform quad-section 1-dimensional search:
      +similar to bisection search, but to find potentially non-continuous
      +minimum/maximum of function
    • +
    +
  • +
+
+ +
+ +
+
+
+

Version 1.4.0

+ +
+
+
+ + 03 Feb 11:41 + +
+ +
+ @jgaeddert + jgaeddert +
+ + +
+
+ + + + + v1.4.0 + +
+ +
+ +
+ + + + + +
+
+
+

+ Choose a tag to compare +

+ +
+
+ +
+
+
+ + + +
+ + + + + + +
+ + +
+
+
+ + +
+
+ + +
+
+ + + + + +
+ +
+ +
+ +
+
+
+
+
+ + +
+
+ +
+
+
+ Version 1.4.0 + + +
+ + +
+
+
+ +
+ +
+
+ + +
+
+
+ + +
+
+ +

This release includes a number of new objects, new methods for existing objects, better support for error handling, integrated GitHub workflows for testing & code coverage, and general stability enhancements:

+
    +
  • autotest +
      +
    • automated code coverage testing (72%)
    • +
    +
  • +
  • build +
      +
    • incorporated error handling in most methods with integer-based return value
    • +
    +
  • +
  • agc +
      +
    • added interface to check if AGC is locked or not
    • +
    +
  • +
  • buffer +
      +
    • cbuffer: added method to check if buffer is empty
    • +
    +
  • +
  • dotprod +
      +
    • included methods to create/recreate object in reverse order
    • +
    +
  • +
  • equalization +
      +
    • eqlms: added convenience methods for copying coefficients and running as
      +a decimator
    • +
    +
  • +
  • fft +
      +
    • spgram: adding convenience methods for getting/setting properties
    • +
    • spwaterfall: adding convenience methods for getting/setting properties
    • +
    +
  • +
  • filter +
      +
    • new direct digital synthesis (DDS) family of objects which allows
      +cascaded half-band interpolation/decimation with arbitrary frequency
      +translation
    • +
    • new fractional delay (fdelay) family of objects which allow for
      +adjustable large fractional delays
    • +
    • firdecim: adding convenience method to get decimation rate
    • +
    • firfilt: adding convenience methods to get/copy coefficients as well as
      +create object using firdespm algorithm
    • +
    • firinterp: adding convenience methods for new ways to create object as
      +well as getting object properties
    • +
    • firpfb: adding convenience methods to create default object, write
      +samples
    • +
    • resamp2: adding methods to get/set output scale
    • +
    • rresamp: adding methods to operate with block execution
    • +
    +
  • +
  • framing +
      +
    • new arbitrary rate symstreamr object (derived from symstream family)
    • +
    • framesync64: adding methods for getting/resetting frame data statistics
      +as well as getting/setting detection threshold
    • +
    • gmskframegen/gmskframesync: extending methods to support easier
      +operation, getting/resetting frame data statistics, more control over
      +parameters in create methods
    • +
    • ofdmflexframesync: adding methods to get/reset frame data statistics
    • +
    • qdetector: adding method to get threshold
    • +
    • qpilotsync: adding method to get error vector magnitude
    • +
    • symstream: adding convenience methods for getting properties
    • +
    • symtrack: adding convenience methods for getting/setting properties
    • +
    +
  • +
  • math +
      +
    • adding liquid_ prefix to winodowing functions while supporting backwards
      +compatibility with older methods
    • +
    • adding liquid_ prefix to internal polynomial functions, using
      +common double-precision method for finding polynomial roots
    • +
    +
  • +
  • multichannel +
      +
    • firpfbch2: adding convenience methods to get object properties
    • +
    +
  • +
  • modem +
      +
    • modem: adding type extension for more consistency: modem -> modemcf,
      +supporting backwards compatibility with API shim
    • +
    +
  • +
+
+ +
+ +
+
+ + +
+ +
+ +
+ +
+
+ +
+ + +
+ +
+ + +
+
+ +
+ +
+

Footer

+ + + + +
+
+ + + + + © 2026 GitHub, Inc. + +
+ + +
+
+ + + + + + + + + + + + + + + + + + + + +
+
+
+ + + diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/oceans2022-acoustic-modem-liquid-dsp.pdf b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/oceans2022-acoustic-modem-liquid-dsp.pdf new file mode 100644 index 0000000..5647aaa Binary files /dev/null and b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/oceans2022-acoustic-modem-liquid-dsp.pdf differ diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/open-echo-readme.md b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/open-echo-readme.md new file mode 100644 index 0000000..a49906e --- /dev/null +++ b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/open-echo-readme.md @@ -0,0 +1,170 @@ +Open Echo Cover + +## Universal Open-Source SONAR Controller and Development Stack + +An ongoing open-source hardware and software project for building sonar systems for testing, boating, bathymetry, and research. +The most commonly used hardware is the [TUSS4470 Arduino Shield](TUSS4470_shield_002/), which stacks on top of an Arduino Uno to drive the TUSS4470 ultrasonic driver. +The board can run the [RAW Data Firmware](TUSS4470_shield_002/getting_started_TUSS4470_firmware.md) to operate a wide variety of ultrasonic transducers, covering frequencies from 40 kHz up to 1000 kHz in different media such as air or water. + +The [NMEA Output Firmware](TUSS4470_shield_002/arduino/NMEA_DBT_OUT/NMEA_DBT_OUT.ino) can read depth data from commercially available in-water ultrasonic transducers (e.g., on boats) and output NMEA0183-compatible data to a computer or a UART-connected device such as a Pixhawk or other controllers. + +Open Echo has been tested on multiple ultrasonic transducers and is compatible with all of them—from car parking sensors to Lowrance Tripleshot side-scan transducers. + +The [Python Interface Software](TUSS4470_shield_002/getting_started_interface.md) connects to Open Echo boards running the [RAW Data Firmware](TUSS4470_shield_002/getting_started_TUSS4470_firmware.md). It can display raw echo data, change configurations, output a TCP depth data stream, and more. + +Check the [Getting Started Guide](TUSS4470_shield_002/README.md)! + +If something is unclear or you find a bug, please open an issue. + + +Raw Data Waterfall chart in the Python Desktop software: +Open Echo Interface Software + + +## Getting the Hardware + +If you need the hardware, you can order it using the [Hardware Files](TUSS4470_shield_002/TUSS4470_shield_hardware/TUSS4470_shield) from a board + SMT house ([JLC recommended](https://jlcpcb.com/?from=Neumi)). + +They can also be bought as a complete and tested set direclty from Elecrow: https://www.elecrow.com/open-echo-tuss4470-development-shield.html + +If they’re out of stock, or if you’d prefer to order them within Germany to reduce shipping costs, please send me an email at: openechoes@gmail.com + +All profits go directly toward supporting and advancing the Open Echo project! + +If you don't order the boards directly from me or Elecrow, please be aware that I can't provide support. + +[TUSS4470 Arduino Shield](TUSS4470_shield_002/): +PCB overview TUSS4470 + +### This project is currently in development. The [TUSS4470 Development Shield](TUSS4470_shield_002/) is ready for external use! +Development is ongoing! Check the documentation and Discord channel for the latest updates. + +Want to stay updated or participate? Join the [Discord](https://discord.com/invite/rerCyqAcrw)! + +Check the [Getting Started Guide](TUSS4470_shield_002/README.md). + +## Vision +An accessible Open Source SONAR stack for development, research and real use: +- Open Source SONAR technology for differernt use cases +- support for a variety of commercial or DIY transducers in water and air +- relatively simple hardware that can be ordered from a board house +- easy entry for less experienced users +- community for exchange and development +- development of more advanced SONAR applications + +-------- +## Current State +- Universal TUSS4470-based Arduino shield for testing ultrasonic transducers +- Python interface software for raw data visualization, configuration, and TCP data output +- Tested depth range of at least 50 m in water +- NMEA0183 compatible (DBT data output to other devices) +- New all-in-one boards with STM32, TUSS4470, and boost converter currently in development +- picoW implementation in development (already successfull) +- UDP network RAW data transfer from picoW to Python interface running + +-------- +## Progress + +The new (May 2025) [TUSS4470 Arduino Shield](TUSS4470_shield_002/) supports transducers from 40 kHz to 1000 kHz. It can drive them, receive echoes, filter signals, and send raw echo data to the Python backend. +Driver voltage can be supplied from Arduino VIN or via the external XT30 connector. With an MT3608 boost converter, USB can be used as a power supply and boosted to the desired drive voltage (manual soldering required). + +The [TUSS4470 Arduino Shield](TUSS4470_shield_002/) is and will remain the main development board for this project. It is an excellent platform for testing and development. +It can drive a wide range of transducers at different voltages, but it is limited by RAM size and sampling speed. You can capture 1800 samples at 12 microseconds per sample (~18 m range in water) with 8-bit resolution. For longer ranges, you can add a delay to capture echoes from more distant objects. +This makes it a perfect board for learning, testing, and prototyping. Most software development is done using this board. + +For special use cases, additional boards and software are under development. +Check the [development](development) folder for development work. + +-------- +## Open Echo Interface Software + +Firmware examples are available in each project folder. +The [**Open Echo Interface Software**](TUSS4470_shield_002/echo_interface.py) allows you to control Open Echo boards, view live data, and adjust board settings. + +Live echogram in water on Python software (6x speed): +Software running with live echo data + +## Test results Baltic Sea +Tested using a built-in 150 kHz SEAFARER transducer, powered at 15-20 V via an MT3608 boost converter from USB. In the plots, data capture begins at the bottom. + + + + + + + + + + + + + +
+ 38m Range
+ 38 m range test +
+ Kiel Channel
+ Kiel Channel sand and mud +
+ Stollergrund
+ Slope near Stollergrund +
+ Stones and Sand
+ Stones and sandy seabed +
+ Fish
+ Fish detection +
+ Multiple Reflections
+ Multiple reflections and seaweed +
+ +## Videos + +[![LINK TO LATEST VIDEO](https://img.youtube.com/vi/XF7rNGt6UYA/maxresdefault.jpg)](https://www.youtube.com/watch?v=XF7rNGt6UYA) + +https://www.youtube.com/watch?v=R3_NO2F7PsI +https://www.youtube.com/watch?v=msbLVsY8xhQ +https://www.youtube.com/watch?v=eJ8jVEQSx_Y +https://www.youtube.com/watch?v=Bxh3rWd5RZk +https://www.youtube.com/watch?v=UDYWQIizN7A + +## Useful Links +https://www.rapp-instruments.de/RemoteSensing/Roves/sidescan/sidescan.htm +https://www.youtube.com/watch?v=ZtUkt8Q4EJE + +## Transducers: + +| Transducer Name | Ranking | Description | Frequency | Range Air/Water |  Price | Link | +|-----------------|---------|-------------------------------------------------------------------------|-----------|-----------------|--------|------| +| NASA / Seafarer 150kHz Echo Sounder | ⭐ ⭐ ⭐ | NASA in-hull boat transducer for echo sounding and simple fish-finding | 150kHz | >2m/>50m | 50-100€ | https://www.nasamarine.com/product/depth-transducer-with-7-metre-cable/7 +| Raymarine CPT-S | ⭐ ⭐ ⭐ | High quality In-hull transducer with two frequencies | 50 + 200kHz | 2m/>50m | 200€ | https://www.raymarine.com/de-de/unsere-produkte/fischfinder-und-sonarmodule/fischfindergeber/cpt-s-durchbruchgeber | +| Lowrance Tripleshot Sidescan | ⭐ ⭐ ⭐ | Sidescan transducer with three FANS + temperature + down-scan | 200 + 455 + 600kHz | 0m/>20m | 200€ | https://www.echolotzentrum.de/shop/lowrance-tripleshot-heckgeber/ | +| Cheap Bathymetry 200kHz | ⭐ ⭐ ⭐ | good range, good price/performance, hard to order | 200kHz | 2m/>30m | 25€ | https://www.alibaba.com/product-detail/Range-customization-lakes-river-surveys-no_1600829423846.html | +| Cheap Car Parking Sensor | ⭐ ⭐ | Great for air, also works in water (not recommended) | 40kHz | >7m/>30m | 5€ | https://de.aliexpress.com/item/1005006546490802.html | +| Water flow sensor transducer | ⭐ ⭐ | Cheap and works in Air and Water. Very narrow beam in Water (about 5°)! | 1000kHz | 0.25m/>10m | 16€ for 5pcs | https://de.aliexpress.com/item/32818381566.html | +| Only for air 200 kHz | ⭐ | has a sinter glass matching layer, should only be used in air | 200kHz | 0.8m/(>8m) | 10€ | https://de.aliexpress.com/item/1005006007865920.html | +| Different Encapsualated Transducers | ⭐| Can be used for short range testing at higher frequencies | 200,300,400kHz | ?m/?m | 9-18€ | https://de.aliexpress.com/item/4000389134890.html | +| Very small PZT only | ⭐ | Only for testing / experimentation | 200kHz | 0.2m/?m | 10€ | https://de.aliexpress.com/item/1005007032482539.html | + +> [!Note] +> The range values were measured using the TUSS4470 shield and Arduino Uno with an MT3608 boost converter set to 20 V for vDRV, so consider them approximate. + +> [!Tip] +> As a general rule of thumb, lower frequencies propagate farther, while higher frequencies produce narrower beams and more detailed echoes. +> +> If you need a transducer for bathymetry, choose one of the first four options. Most commercial marine-grade (boating) transducers are also suitable, as long as they operate in the 40-1000 kHz range. + + +# Big Thanks for Your Support! +www.kogger.tech + +## to the awesome project contributors: +Check this Cheap Yellow Display implementation out: https://github.com/matztam/open_echo_cyd_display + +And all the great stuff JohnCHarrington contributed: https://github.com/JohnCHarrington/open_echo + + +## Star History + +[![Star History Chart](https://api.star-history.com/svg?repos=Neumi/open_echo&type=date&legend=top-left)](https://www.star-history.com/#Neumi/open_echo&type=date&legend=top-left) diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/opentds.html b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/opentds.html new file mode 100644 index 0000000..fc9a226 --- /dev/null +++ b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/opentds.html @@ -0,0 +1,136 @@ + + + + + + + + +OpenTDS | An Open Source Alternative to Generating Frequency Response and Time Response Graphs by Time Delay Spectrometry + + + + + + + + + + + + + + + +
+
+

OpenTDS

+ + + +

An Open Source Alternative to Generating Frequency Response and Time Response Graphs by Time Delay Spectrometry

+ + +

View the Project on GitHub syarusinsky/OpenTDS

+ + + + + +
+
+ +

OpenTDS

+

An Open Source Alternative to Generating Frequency Response and Time Response Graphs by Time Delay Spectrometry

+ +

This project was designed as an Open Source alternative to Gold Line’s TEF system and other similar (expensive) software and hardware bundles. These systems use Time Delay Spectrometry to achieve frequency response measurements of loudspeakers, rooms, audio circuits, and many other minimal-phase systems without the need of an anechoic chamber. This project benefits from being capable of taking measurements using any modern consumer-grade audio interface, while TEF and other systems require you to purchase their proprietray hardware. The goal of this project is to provide hobbyist and professional acousticians and audio engineers with a means to measure frequency response and time response without the need to purchase any additional hardware or software.

+ +

Current Status

+ +

Currently, the program is capable of producing reliable frequency response graphs comparable to Goldline’s TEF system and other TDS based systems. However, the accuracy of the results are currently dependant on the user’s ability to correctly measure the microphone’s distance from the loudspeaker and the temperature of the room (ie: correctly measure the time-of-flight from loudspeaker to microphone). In future versions of the program, calculation of the time-of-flight will instead be measured by the user using the Time Response Graphing Module. This module, along with the Phase Response Module, will be among the landmark goals of the project.

+ +

Landmark Goals

+ +
    +
  • Time Response Module
  • +
  • Phase Response Module
  • +
  • Improved TDS Functionality +
      +
    • Improved BandPass Filtering (A Custom IIR Filter Class)
    • +
    • Improved Amplitude Calculation (Algorithmically-Selected FFT-Bin-Based Magnitude Instead of Block-Based Magnitude Averages)
    • +
    • dB Referencing (Capability to Calibrate the System to Reference Pressure Levels. ie: 20uPa for dBSPL)
    • +
    +
  • +
  • Increased Driver Support (ASIO currently fully supported)
  • +
+ +

JUCE

+

This project was written using the JUCE framework which provides excellent cross-platform support.

+ +

License

+

This project is licensed under the GNU General Public License v3.0 - see the LICENSE.md file for details.

+ +

Authors

+

Stevey Yarusinsky - Initial Work

+ +

Acknowledgements

+

Richard C. Heyser - For his work developing the method of Time Delay Spectrometry. An anthology of his published works (“Time Delay Spectrometry: An Anthology of the Works of Richard C. Heyser on Measurement Analysis and Perception”) was an invaluable asset in developing this project.

+ +

Documentation

+ +

TDS Background Information and Concepts

+ +

In traditional swept signal analysis, a sinusoidal waveform is sent to a device-under-test (DUT) and the resulting magnitude and phase changes on the output of the DUT are plotted on a graph for each frequency under consideration. The plotted magnitudes are the Frequency Response of the graph. Often included are the plotted phases (or Phase Response) which are often plotted on the same graph usually in a different color.

+ +

Frequency response is critical to achieving the required performance of many systems such as loudspeakers. However, loudspeakers cannot be accurately measured with traditional swept signal analysis unless measurement is performed in an anechoic chamber. An anechoic chamber is a room designed to be completely acoustically absorptive which eliminates the possiblity of sound being reflected on any surfaces of the room. In this chamber a loudspeaker can be measured by inputting sinusoidal waves and measuring the magnitude and phase changes with a calibrated reference microphone, though most professionals and hobbyists do not have access to an anechoic chamber. In a normal room the waves generated by the speaker would reflect off of the surfaces of the room and eventually reach the microphone, skewing the measurement results considerably. Time Delay Spectrometry was designed to solve this problem.

+ +

Time Delay Spectrometry introduces a variable-bandwidth band-pass filter capable of both tracking the frequency of a paired oscillator and delaying that oscillator to compensate for the time-of-flight. This process filters out any reflections generated by the conditions of the space where measurement is taking place. The concepts are difficult to grasp initially so a procedural description is helpful.

+ +
    +
  • First, a sweeping oscillator generating a sinusoidal waveform is sent to the loudspeaker.
  • +
  • The sound waves generated by the loudspeaker propagate and reflect off of surfaces in the measurement space
  • +
  • The direct sound reaches the microphone first followed later by the reflected sound.
  • +
  • These sounds are recorded by the software then processed with the band pass filter, which is sweeping its cutoff frequency with a paired oscillator.
  • +
  • The reflections take more time to reach the microphone and therefore are lower frequency than the direct sound.
  • +
  • Since the reflections are lower frequency than the direct sound they are filtered out by the band pass filter whose cutoff frequency is equivalent to the paired oscillator, leaving only the direct sound in the signal.
  • +
  • This signal is then processed to determine its amplitude and phase information.
  • +
  • That information is then plotted as a frequency response graph.
  • +
+ +

It is also important to note that the paired oscillator is actually a slightly lower frequency than the oscillator being fed into the loudspeaker due to the delay placed on the paired oscillator. This is necessary to compensate for the amount of time required for the direct sound to reach the microphone. Figure 1a. and Figure 1b. should help visualize the TDS process.

+ +

Figure 1a. Block Diagram of TDS +Block Diagram of TDS

+ +

Figure 1b. TDS Geometry Example +TDS Geometry Example

+ +

Delay in TDS

+ +

Since the direct sound is the desired component, the accuracy of the frequency response measurements is determined by the delay placed on the sweeping of the tracking band pass filter. This is known as the time-of-flight. In an ideal analog system, this would simply be the time it takes for the sound to travel from the speaker to the microphone. However, this TDS program was designed to be used with any consumer grade audio interface and nearly all modern systems utilize digital audio concepts. Most notably, these systems use input and output audio buffers to lighten the CPU load. These buffers introduce a latency, or delay, into the signal path. While this delay is usually negligible for recording and playback purposes, it is catastrophic for the accuracy of TDS frequency response measurements. In addition, each audio interface will have its own unique latency. In order to be absolutely sure that any latency from any system is accounted for in our time-of-flight, OpenTDS allows the user to calibrate latency before taking measurements. The user must plug the output of their system into the input and click the calibrate latency button. After this is operation is performed the user can continue taking measurements until the program is terminated. The next time the program is opened, the user must calibrate the latency again before taking measurements.

+ +

Frequency Response Comparison

+ +

Speaker (QSC K8) Frequency Response at 1 ft away with TEF25 (Graph is Smoothed by the TEF Software) +Speaker (QSC K8) Frequency Response at 1 ft away with TEF25 +Speaker (QSC K8) Frequency Response at 1 ft away with OpenTDS and Akai EIE Pro audio interface +Speaker (QSC K8) Frequency Response at 1 ft away with OpenTDS and Akai EIE Pro audio interface

+ +

Video Tutorials coming soon . . .

+ + +
+ +
+ + + + diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/pyapril-readme.md b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/pyapril-readme.md new file mode 100644 index 0000000..bcb3bea --- /dev/null +++ b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/pyapril-readme.md @@ -0,0 +1,83 @@ +# py A P R i L + +# Advanced Passive Radar Library + +pyAPRiL is a python based DSP library which implements passive radar algorithms. The ultimate goal of the library is to make available the so far ellaborated passive radar algorithms to everyone including sceintific researchers, radar system designers and amateurs. All the implemented methods are tested and verified through real-life systems, field measurements and simulations. + +### Project guidelines: +- This project respects the authors of all the contributions, therfore references are always highlighted where applicable. +- Understanding the operation of the implemented algorithms is always of primary importance. The efficient execution and concise coding do not belong to the principles of the project, but encoured when it is not at the expense of comprehensibility and does not change the essential operation of the originally proposed algorithm. +### Contributions: +Contributions are welcome from anyone who shares the passion of passive radars and wants to help make opensource passive radar community even better. Whether you're a seasoned developer or just getting started, there are plenty of ways to get involved and contribute to pyAPRiL. By contributing to the project, you'll be helping to make a positive impact on passive radar developers around the globe and advancing the deeper sceintific understanding of this novel technology. Whether you can contribute code, documentation, or simply provide feedback, every contribution helps us move closer to our shared goal. +### Restrictions: +This projects strictly focuses on the support of scientific researches. The source code of the library is licensed under the GNU General Public License version 3 (GPLv3), which is a copyleft license. This means that anyone who uses, modifies, or distributes the project must also license their work under the same terms and conditions. Commercial use of this software is permitted, provided that any results obtained from such use are made available to the public under the same license. By adhering to these restrictions, this project gains comliance with the REGULATION (EU) 2021/821 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL. + +### The package is organized as follows: + +* pyAPRiL: Main package directory + * **channelPreparation**: This file contains wrapper functions for the different algorithms that aims to prepare the reference and the sureveillance signals for the detection stage. Such as reference signal regeneration and clutter cancellation techniques. + * **clutterCancellation** : It describes a huge variety of clutter cancellation techniques implemented in the space, time and the space-time domain. In the current version, the following algorithms are available: + * Timde-domain: + * Wiener-SMI: Wiener filter with Sample Matrix Inversion + * Wiener-SMI-MRE: Wiener filter with Sample Matrix Inversion and Minimum Redundancy Estimation + * ECA: Extensive Cancellation Algorithm + * ECA-B: Extensive Cancellation Algorithm - Batched + * ECA-S: Extensive Cancellation Algorithm - Sliding + * Space-domain (Beamforming): + * Max-SIR: Maximum Signal-to-Interference Ratio + * MVDR: Minimum Variance Distortionless Response + * Principal eigenvalue beaformer + * Beamstearing + * Explicit coefficient beamformer + * **hitProcessor**: Implements a number of hit and plot processing related functions such as the CFAR and the plot extractor. Implemented algorithms: + * CA-CFAR + * Target DoA estimator (from multichannel RD matrix using the pyArgus library) + * **detector** : In this file a number of implementation of the cross-correlation detector can be found. + * Cross-correlation detector - Time domain implementation + * Cross-correlation detector - Frequency domain implementation + * Cross-correlation detector - Batched implementation (Overlap and save) + * Doppler frequency windowing + * **metricExtract**: Implements various clutter cancellation and detection performance evaluation function. The supported metrics are the followings: + * CA: Clutter Attenuation + * Rnf: Noise floor reduction + * Mu : Peak-to-channel power ratio + * Delta: Estimated target SINR + * Alpha: Target peak to noise floor ratio + * L: Target peak power loss + * Rzdc: Zero Doppler clutter reduction + * P: Peak clutter reduction + * D: Dynamic range compression + * **targetParameterCalculator**: Calculates the expected parameters of an observed target (bistatic range, Doppler, ...) from geodetic data. + * **targetLocalization**: Calculates the location of the target + * **RDTools**: range-Doppler matrix image export and plotting tools + * **sim**: IoO simulator (FM) + * **docs**: Contains Ipython notebook files with demonstrations. + * **testing**: Contains demonstration functions. + +### Installing from Python Package Index: +```python +pip install pyapril +``` + +### Version history +* 1.0.0 Inital version +* 1.1.0 Automatic detection added (CA-CFAR) +* 1.2.0 Add temporal clutter filters (ECA, ECA-B, ECA-S) +* 1.3.0 Clutter filter and detection performance evaluation +* 1.4.0 Target DoA estimation +* 1.5.0 Surveillance channel beamforming +* 1.6.0 Target parameter calculator +* 1.7.0 FM IoO simulator +* 1.7.1 Extend RD Tools with Hit matrix plot +* 1.7.2 Adaptive RD matrix scaling to improve the visibility of the plots +* 1.7.3 Improved generality for metric extraction +* 1.7.4 Improve and fix metric extraction +* 1.7.5 Update project organization (2023 04) +* 1.7.6 Target localization +### Acknowledgements +This work was supported by the Microwave Remote Sensing Laboratory of BME ([Radarlab](http://radarlab.mht.bme.hu)). A special thanks of mine goes to the RTL-SDR site ([RTL-SDR](https://www.rtl-sdr.com/)) and to the KrakenRF (https://www.krakenrf.com/) who helped this project becoming mature with the development and testing the algorithms using Kerberos SDR (later KrakenSDR). + +For further information on passive radars check: [passiveradar.eu](https://passiveradar.eu) + + + diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/signalsmith-dsp.html b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/signalsmith-dsp.html new file mode 100644 index 0000000..7ea4d2e --- /dev/null +++ b/research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/signalsmith-dsp.html @@ -0,0 +1,101 @@ + + + + DSP Library  :  Open-source code  :  Signalsmith Audio + + + + + + + + + + + + + + + + + +
+ +
+
+
+

DSP Library

+
Signalsmith Audio
+
+ + + +

This is a set of C++11 header-only classes/templates to support certain DSP tasks (mostly audio).

+

It's still growing, but currently includes:

+
    +
  • Delay tools (circular buffers, single/multi-channel delay-lines)
  • +
  • Interpolators (Lagrange, polyphase, Kaiser-sinc)
  • +
  • Envelope tools (e.g. box-filter, peak-hold)
  • +
  • FFT and spectral processing (including multi-channel STFT)
  • +
+ +

How to use

+

Clone using Git:

+

+				git clone https://signalsmith-audio.co.uk/code/dsp.git
+			
There's also a GitHub mirror
+

Include the appropriate header file, and start using classes:

+
+

API docs

+

Check out the API documentation (Doxygen) for detailed information.

+ +

Testing

+

Tests (and scripts to plot graphs etc.) are in a separate repository, to keep the main repo neater:

+

+				git clone https://signalsmith-audio.co.uk/code/dsp-doc.git
+			
There's a GitHub mirror for this too
+

Goals

+

Where reasonable, the tests should measure actual output quality (not just basic correctness and smoke-tests).

+

For example, here's the aliasing/accuracy performance of a delay line with windowed-sinc interpolation:

+
+

The automated tests check this performance for various input bandwidths, using a table like this:

+

+				// A table of acceptable limits for the 20-point windowed-sinc
+				double bandwidth[] = {90, 80, 50, 25, 12.5};
+				double aliasing[] = {-20.5, -61, -70.5, -76, -77.5};
+				double ampLow[] = {-2, -0.02, -0.01, -0.01, -0.01};
+				double ampHigh[] = {0.02, 0.02, 0.01, 0.01, 0.01};
+				double delayError[] = {0.9, 0.03, 0.01, 0.01, 0.01};
+			
Each column specifies performance for a particular input bandwidth - e.g. 25% would correspond to 4x oversampled input.
+ +

Running the tests

+

To run the tests:

+
make test
+

To run just the tests in tests/delay/ (or similar):

+
make test-delay
+

There are a few other targets (e.g. for plotting graphs) - check the Makefile if you're curious.

+ +

License

+

The main library is MIT licensed (see LICENSE.txt in the main repo), but the tests, docs and support scripts are licensed differently (just for developing/testing the library).

+

We're pretty flexible though - if you need something else, get in touch.

+ +
+ + diff --git a/research/sonar-dsp-recursive-refinement-2026-05-25/sources.md b/research/sonar-dsp-recursive-refinement-2026-05-25/sources.md new file mode 100644 index 0000000..90fbeaf --- /dev/null +++ b/research/sonar-dsp-recursive-refinement-2026-05-25/sources.md @@ -0,0 +1,209 @@ +# Source Ledger: Recursive Sonar DSP Refinement + +This ledger preserves the source trail behind +`../sonar-dsp-recursive-refinement-2026-05-25.md`. It is not a literature review +for sport. Each source is here because it changes the next Mimir decoder cut. + +Local mirrors live under `mirrors/` so the next implementation pass has stable +context even if a page moves, throttles, or gets cute. + +## Source Index + +| ID | Source | Local mirror | Primary lesson for Mimir | +| --- | --- | --- | --- | +| S1 | liquid-dsp README | `mirrors/liquid-dsp-readme.rst` | Use lightweight complex DSP primitives, synchronizers, filters, NCOs, and modem-style framing as the reference shape for a portable hot path. | +| S2 | liquid-dsp release notes | `mirrors/liquid-dsp-releases.html` | `qdsync`-style synchronization estimates and corrects timing, carrier frequency, and phase as one acquisition/tracking problem. | +| S3 | Underwater acoustic modem using liquid-dsp | `mirrors/oceans2022-acoustic-modem-liquid-dsp.pdf` | Acoustic packet receivers use chirp matched filters for packet starts, then PN synchronization, equalization, PLL, AGC, and changing loop bandwidths for acquisition versus tracking. | +| S4 | pyAPRiL passive radar | `mirrors/pyapril-readme.md` | Reference/surveillance processing, clutter cancellation, overlap-save correlation, CFAR, and range-Doppler thinking map cleanly onto loopback/mic acoustic paths. | +| S5 | GNU Radio correlate access code | `mirrors/gnuradio-correlate-access-code-tag-stream.html` | Synchronization should emit stream-position facts/tags, not just side-channel estimates. | +| S6 | GNU Radio stream tags | `mirrors/gnuradio-stream-tags.html` | Time decisions must travel with sample streams as isosynchronous metadata. This matches Mimir's rolling-buffer anchor model. | +| S7 | OpenTDS / time-delay spectrometry | `mirrors/opentds.html` | Swept tracking filters can isolate direct sound from later reflections and then derive amplitude/phase response. | +| S8 | Signalsmith DSP | `mirrors/signalsmith-dsp.html` | Fractional delay quality is measurable; use Lagrange/polyphase/Kaiser-sinc references instead of linear interpolation for final fitting/actuation. | +| S9 | Open Echo | `mirrors/open-echo-readme.md` | Sonar systems treat transducer drive, receive filtering, raw echo capture, and visualization as one measurement stack. | +| S10 | ahoi acoustic modem hardware overview | `mirrors/ahoi-hardware-overview.html` | Front-end analog filtering and gain are part of the receiver; usable acoustic bands must be measured per mic/path before DSP pretends all bins are equal. | + +## Distilled Notes + +### S1: liquid-dsp README + +Upstream: + +liquid-dsp is a C library meant for SDR-style DSP on embedded platforms. It +provides filters, filter design, oscillators/NCOs, modems, synchronizers, FFTs, +complex math, and framing pieces without forcing a large framework. It also +ships extensive tests and benchmarks. + +Mimir implication: + +- The hot decoder should be structured as small streaming DSP objects, not as + whole-window analysis scripts. +- Complex baseband is the natural internal representation for phase-aware + contour matching. +- The decoder should expose benchmarkable primitives: filterbank, matched + filter, synchronizer, path tracker, actuator. + +### S2: liquid-dsp Release Notes + +Upstream: + +The relevant signal is `qdsync`: the release notes describe a synchronizer that +detects frames and estimates/corrects carrier frequency, carrier phase, and +timing offset before returning clean data to the user. That is the missing shape +in Mimir's failed recursion pass. + +Mimir implication: + +- Do not split detection, phase, delay, and drift into unrelated little patches. +- The recursive state should be a synchronizer object with acquisition and + tracking modes. +- Timing offset, SRO, phase intercept, and group delay are one coupled state + surface. + +### S3: Underwater Acoustic Modem With liquid-dsp + +Upstream: + + +The modem paper uses chirp preambles, matched filtering to find packet starts, +and later liquid-dsp flexible framing with PN synchronization, equalization, +PLL, AGC, and acquisition-mode loop bandwidths. It also explicitly notes that +chirps can support Doppler and channel impulse-response estimation. + +Mimir implication: + +- Canary/birdcall packets should expose a preamble-like matched-filter surface, + even if they sound musical. +- The contour word is not only an identifier; it is a channel probe. +- Recursive refinement should update channel impulse-response/state, not only + delay. + +### S4: pyAPRiL Passive Radar + +Upstream: + +pyAPRiL separates reference and surveillance channels, then provides clutter +cancellation, time-domain and frequency-domain cross-correlation, batched +overlap-save correlation, Doppler windowing, CFAR, beamforming, and performance +metrics. + +Mimir implication: + +- Scarlett loopback is the reference channel; mics are surveillance channels. +- Reflections and music leakage are clutter, not random noise. +- The path tracker should use clutter/residual modeling and CFAR-like detection + thresholds so strong reflection lobes stop impersonating direct-path timing. + +### S5: GNU Radio Correlate Access Code + +Upstream: + + +The block scans a stream for a chosen access code and emits a tagged stream when +the access code is found. Its documentation also warns that cyclic access codes +produce false detections and bad alignment. + +Mimir implication: + +- Song contours need low-autocorrelation / non-cyclic identity features. +- Decoder hits should become timestamped anchors in the stream, not global + mutable estimates. +- False-lock behavior is an architecture problem, not a threshold annoyance. + +### S6: GNU Radio Stream Tags + +Upstream: + +GNU Radio stream tags are metadata attached to exact sample positions and +propagated along the same stream as the samples. + +Mimir implication: + +- Mimir anchors should be sample-position metadata inside the rolling buffer. +- UI/debug state reads these tags; it should not recalculate timing. +- Networked Raven/phone decoders should emit the same typed anchor facts so mesh + state preserves timing authority. + +### S7: OpenTDS / Time-Delay Spectrometry + +Upstream: + +Time-delay spectrometry uses a swept source and tracking band-pass behavior to +separate direct sound from delayed reflections, then measures magnitude and +phase response. + +Mimir implication: + +- The room-path problem is directly analogous: direct arrival must be isolated + before response normalization is trusted. +- Frequency response normalization needs phase/group delay, not only magnitude. +- Our birdcall contours should be shaped so direct-path time/frequency evidence + survives before later reflections dominate. + +### S8: Signalsmith DSP + +Upstream: + +Signalsmith's open DSP library includes delay tools, Lagrange/polyphase/Kaiser- +sinc interpolators, envelopes, FFTs, and multi-channel STFT. Its docs discuss +measured delay-line error versus bandwidth. + +Mimir implication: + +- Linear interpolation is not credible for the last microseconds. +- Decoder refinement and Faust/native actuation need measured fractional-delay + quality gates. +- Oversampled or band-limited interpolation should be part of the timing kernel, + not an afterthought. + +### S9: Open Echo + +Upstream: + +Open Echo is an open sonar controller stack. It drives ultrasonic transducers, +captures raw echo data, filters signals, and surfaces raw waterfalls / TCP data +for tooling. + +Mimir implication: + +- Treat emission, analog/acoustic path, capture, raw receipts, and decoder as + one calibration system. +- Persist raw captures and derived receipts together. +- The UI should expose raw evidence and path state, not only final delay. + +### S10: ahoi Acoustic Modem Hardware Overview + +Upstream: + +The ahoi modem receiver chain includes hydrophone matching, pre-amplification, +software-switchable gain, and band-pass filtering adapted to the intended +frequency band. + +Mimir implication: + +- Per-path usable frequency bands are physical facts, not decoder preferences. +- Calibration must learn what each speaker/mic path can actually hear. +- The matched-filter bank should be weighted by measured response and confusion + surfaces. + +## Implementation Synthesis + +The next recursive decoder should be: + +```text +loopback reference + mic window +-> complex contour matched-filter bank +-> candidate access/word anchors with stream positions +-> direct-path channel tracker + - delay + - SRO + - phase intercept + - phase/group-delay curve + - per-band complex gain +-> sparse later reflection taps +-> bounded recursive update +-> typed rolling-buffer anchors + calibration receipt +``` + +The important correction is ownership: recursion refines the direct-path channel +state. It does not choose whichever delayed waveform lobe has the biggest +correlation this frame. diff --git a/state/evidence.jsonl b/state/evidence.jsonl index d19eb1d..01dd9ef 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -100,3 +100,4 @@ {"ts": "2026-05-25T00:42:12+01:00", "type": "experiment", "status": "partial", "note": "Added path-level loopback truth to the anchor-rich calibration: matched mic anchors are compared directly against loopback anchors, then event-local waveform correlation against the captured loopback packet chooses the path delay. Release calibration clears the realtime budget at 21.1x on the stored anchor-rich capture. Best stored-result path precision improved to cardioid 7.576 us and shotgun 6.578 us. A fresh room capture at calibration/bioacoustic/scarlett-canary-packet-anchor-rich-latest-192k.json keeps shotgun near 5.916 us but worsens cardioid to 18.930 us. The razor timing-chip emitter mutation was rejected because it reduced physical reliability. Current floor is single-digit microseconds for the shotgun, not yet the one-microsecond target; next required cut is phase/group-delay correction or a direct-path anchor family that survives both mics."} {"ts": "2026-05-25T01:05:00+01:00", "type": "experiment", "status": "rejected", "note": "Tested a recursive event-local waveform delay fitter: start from the current path hypothesis, resample candidate song windows against loopback, shrink the residual search radius, and weight events by phase-lock consistency. The naive time-domain version improved one stored cardioid path from 7.576 us to 5.141 us, but it also worsened the stored shotgun from 6.578 us to 10.236 us and locked the fresh cardioid onto a later reflection lobe at 65.666 us instead of the prior 18.930 us. Reverted the code and receipts. Lesson: recursive refinement is a valid shape only when constrained by a real phase/group-delay/path model; plain waveform resampling is not sufficient and nanosecond claims are physically dishonest in the current acoustic path."} {"ts": "2026-05-25T01:25:00+01:00", "type": "research", "status": "ok", "note": "Looked at open realtime DSP/sonar-adjacent systems after the recursive-fit rejection. liquid-dsp, an underwater acoustic modem using liquid-dsp, pyAPRiL passive radar, GNU Radio stream/tag synchronization, OpenTDS, Signalsmith DSP, Open Echo, and ahoi all point the same way: acquisition should use matched filters/ambiguity search; refinement should track complex phase, timing, SRO, and channel response; multipath should be modeled as later residual taps; and exact stream positions should be propagated as tags/state. Added research/sonar-dsp-recursive-refinement-2026-05-25.md. The next implementation cut should be a complex contour matched-filter bank plus direct-path acoustic tracker, not another raw waveform recursion pass."} +{"ts": "2026-05-25T01:41:00+01:00", "type": "research", "status": "ok", "note": "Persisted the sonar/DSP source trail behind recursive refinement. Added research/sonar-dsp-recursive-refinement-2026-05-25/sources.md with per-source summaries and Mimir implementation implications, plus local mirrors under research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/ for liquid-dsp, liquid-dsp releases, the acoustic modem PDF, pyAPRiL, GNU Radio correlate/tag docs, OpenTDS, Signalsmith DSP, Open Echo, and ahoi hardware overview."} diff --git a/state/map.yaml b/state/map.yaml index 3c456bb..4385f3c 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -216,7 +216,8 @@ accepted_design: `research/sonar-dsp-recursive-refinement-2026-05-25.md` points the next decoder cut at complex contour matched filters, phase-slope/group-delay fitting, acquisition/tracking loop bandwidths, and sparse multipath - residuals. + residuals. Its source ledger and local mirrors live under + `research/sonar-dsp-recursive-refinement-2026-05-25/`. Acoustic robustness remains open. The old arbitrary-chirplet matcher is a diagnostic/reference artifact only, not the runtime hot path.