Persist sonar DSP source ledger

This commit is contained in:
Meta Vens 2026-05-25 01:01:49 +01:00
commit 21815c1fa0
14 changed files with 6859 additions and 1 deletions

View file

@ -11,6 +11,11 @@ experiment rejected the naive objective function, not recursive refinement.
## Sources Checked
The durable source ledger is
`sonar-dsp-recursive-refinement-2026-05-25/sources.md`. Local mirrors of the
source pages and PDF live under
`sonar-dsp-recursive-refinement-2026-05-25/mirrors/`.
- liquid-dsp: open-source C DSP library for SDR-style filters, transforms,
synchronizers, modulators, demodulators, and frame processing.
<https://github.com/jgaeddert/liquid-dsp>

View file

@ -0,0 +1,346 @@
<!DOCTYPE html>
<html class="client-nojs" lang="en" dir="ltr">
<head>
<meta charset="UTF-8">
<title>Correlate Access Code - Tag Stream - GNU Radio</title>
<script>(function(){var className="client-js";var cookie=document.cookie.match(/(?:^|; )mediawikimwclientpreferences=([^;]+)/);if(cookie){cookie[1].split('%2C').forEach(function(pref){className=className.replace(new RegExp('(^| )'+pref.replace(/-clientpref-\w+$|[^\w-]+/g,'')+'-clientpref-\\w+( |$)'),'$1'+pref+'$2');});}document.documentElement.className=className;}());RLCONF={"wgBreakFrames":false,"wgSeparatorTransformTable":["",""],"wgDigitTransformTable":["",""],"wgDefaultDateFormat":"dmy","wgMonthNames":["","January","February","March","April","May","June","July","August","September","October","November","December"],"wgRequestId":"159adc96dad2d12f6950f0ef","wgCanonicalNamespace":"","wgCanonicalSpecialPageName":false,"wgNamespaceNumber":0,"wgPageName":"Correlate_Access_Code_-_Tag_Stream","wgTitle":"Correlate Access Code - Tag Stream","wgCurRevisionId":13769,"wgRevisionId":13769,"wgArticleId":3920,"wgIsArticle":true,"wgIsRedirect":false,"wgAction":"view","wgUserName":null,"wgUserGroups":["*"],"wgCategories":["Block Docs"],"wgPageViewLanguage":"en","wgPageContentLanguage":"en","wgPageContentModel":"wikitext","wgRelevantPageName":"Correlate_Access_Code_-_Tag_Stream","wgRelevantArticleId":3920,"wgIsProbablyEditable":false,"wgRelevantPageIsProbablyEditable":false,"wgRestrictionEdit":[],"wgRestrictionMove":[],"wgCheckUserClientHintsHeadersJsApi":["architecture","bitness","brands","fullVersionList","mobile","model","platform","platformVersion"]};
RLSTATE={"site.styles":"ready","user.styles":"ready","user":"ready","user.options":"loading","skins.vector.styles.legacy":"ready"};RLPAGEMODULES=["site","mediawiki.page.ready","skins.vector.legacy.js","ext.checkUser.clientHints","ext.moderation.notify","ext.moderation.notify.desktop"];</script>
<script>(RLQ=window.RLQ||[]).push(function(){mw.loader.impl(function(){return["user.options@12s5i",function($,jQuery,require,module){mw.user.tokens.set({"patrolToken":"+\\","watchToken":"+\\","csrfToken":"+\\"});
}];});});</script>
<link rel="stylesheet" href="/load.php?lang=en&amp;modules=skins.vector.styles.legacy&amp;only=styles&amp;skin=vector">
<script async="" src="/load.php?lang=en&amp;modules=startup&amp;only=scripts&amp;raw=1&amp;skin=vector"></script>
<meta name="generator" content="MediaWiki 1.43.6">
<meta name="robots" content="max-image-preview:standard">
<meta name="format-detection" content="telephone=no">
<meta name="viewport" content="width=1120">
<link rel="search" type="application/opensearchdescription+xml" href="/rest.php/v1/search" title="GNU Radio (en)">
<link rel="EditURI" type="application/rsd+xml" href="https://wiki.gnuradio.org/api.php?action=rsd">
<link rel="license" href="https://creativecommons.org/licenses/by-sa/3.0/">
<link rel="alternate" type="application/atom+xml" title="GNU Radio Atom feed" href="/index.php?title=Special:RecentChanges&amp;feed=atom">
</head>
<body class="skin-vector-legacy mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-Correlate_Access_Code_-_Tag_Stream rootpage-Correlate_Access_Code_-_Tag_Stream skin-vector action-view"><div id="mw-page-base" class="noprint"></div>
<div id="mw-head-base" class="noprint"></div>
<div id="content" class="mw-body" role="main">
<a id="top"></a>
<div id="siteNotice"></div>
<div class="mw-indicators">
</div>
<h1 id="firstHeading" class="firstHeading mw-first-heading"><span class="mw-page-title-main">Correlate Access Code - Tag Stream</span></h1>
<div id="bodyContent" class="vector-body">
<div id="siteSub" class="noprint">From GNU Radio</div>
<div id="contentSub"><div id="mw-content-subtitle"></div></div>
<div id="contentSub2"></div>
<div id="jump-to-nav"></div>
<a class="mw-jump-link" href="#mw-head">Jump to navigation</a>
<a class="mw-jump-link" href="#searchInput">Jump to search</a>
<div id="mw-content-text" class="mw-body-content"><div class="mw-content-ltr mw-parser-output" lang="en" dir="ltr"><p>Examine input for specified access code, one bit at a time.
</p><p>input: stream of floats (generally, soft decisions)
</p><p>output: a tagged stream set of bits from the payload following the access code and header
</p><p>This block searches for the given access code by slicing the soft decision symbol inputs. Once found, it expects the following 32 samples to contain a header that includes the frame length (16 bits for the length, repeated). It decodes the header to get the frame length in order to set up the tagged stream key information.
</p><p>The output of this block is appropriate for use with tagged stream blocks.
</p>
<h2><span class="mw-headline" id="Parameters">Parameters</span></h2>
<dl><dt>Access Code</dt>
<dd>is represented with 1 byte per bit, e.g., <code>010101010111000100</code></dd></dl>
<dl><dd>It is important to choose an access code that is not "cyclical". A cyclical code contains a repetition of itself within the length of the code. This can cause false access code detections, and will cause byte boundaries (and hence valid packet length and payload recovery) to be recovered in error, with bizarre results!</dd></dl>
<dl><dd>For example, an access code of <code>11111111</code> is a poor choice because it is cyclical. Let's look at why this is.</dd></dl>
<dl><dd>The first eight <code>1</code> bits in a row from the block's input stream will correctly detect the first access code. The first packet (in standard form <code>&lt;access code&gt; &lt;length1&gt; &lt;length2&gt; &lt;payload&gt;</code>) will be recovered nominally.</dd></dl>
<dl><dd>However, when the correlate access code block's state machine goes back into scanning/detection mode after the first packet, the first bit of the NEXT access code (which forms the <code>11111111</code> preamble of the next packet) will cause an immediate access code detection. This is because the access code detection routine within the block always shifts the last detected access code one bit to the left, and ORs in the current input bit.</dd></dl>
<dl><dd>In our example, the valid bits <code>11111111</code> from the first access code detected will get shifted left one bit (to give <code>11111110</code>), and then the next input bit (the first bit after the end of the first packet) will be ORed into the value (<code>11111110 OR 1</code>, giving <code>11111111</code>) resulting in an erroneous detection of the next access code, because <code>11111111</code> will of course match the access code that the block is searching for.</dd></dl>
<dl><dd>This will falsely detect the start of the next packet (alignment off by 7 bits) and your flowgraph will explode with subsequent bad headers and invalid data. Not a pretty site.</dd></dl>
<dl><dd>So, when choosing an access code, select one that is not cyclical!</dd></dl>
<dl><dd>A possible (and historically interesting) non-cyclical 32-bit access code might be 0xe15ae893:</dd></dl>
<dl><dd><code>11100001010110101110100010010011</code></dd></dl>
<dl><dd>which was the "unique word" used on several early communications satellites.</dd></dl>
<dl><dt>Threshold</dt>
<dd>maximum number of bits that may be wrong</dd></dl>
<dl><dt>Tag Name</dt>
<dd>key of the tag inserted into the tag stream</dd></dl>
<h2><span class="mw-headline" id="Example_Flowgraph">Example Flowgraph</span></h2>
<p>This flowgraph is taken from the <a href="/index.php?title=Packet_Communications" title="Packet Communications">Packet_Communications</a> tutorial.
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Pkt_rcv_fg.png" class="mw-file-description"><img src="/images/thumb/b/b4/Pkt_rcv_fg.png/800px-Pkt_rcv_fg.png" decoding="async" width="800" height="451" class="mw-file-element" srcset="/images/thumb/b/b4/Pkt_rcv_fg.png/1200px-Pkt_rcv_fg.png 1.5x, /images/b/b4/Pkt_rcv_fg.png 2x" /></a></span>
</p>
<h2><span class="mw-headline" id="Source_Files">Source Files</span></h2>
<dl><dt>C++ files</dt>
<dd><a rel="nofollow" class="external autonumber" href="https://github.com/gnuradio/gnuradio/blob/master/gr-digital/lib/correlate_access_code_bb_ts_impl.cc">[1]</a></dd></dl>
<dl><dt>Header files</dt>
<dd><a rel="nofollow" class="external autonumber" href="https://github.com/gnuradio/gnuradio/blob/master/gr-digital/lib/correlate_access_code_bb_ts_impl.cc">[2]</a></dd></dl>
<dl><dt>Public header files</dt>
<dd><a rel="nofollow" class="external text" href="https://github.com/gnuradio/gnuradio">TODO</a></dd></dl>
<dl><dt>Block definition</dt>
<dd><a rel="nofollow" class="external autonumber" href="https://github.com/gnuradio/gnuradio/blob/master/gr-digital/grc/digital_correlate_access_code_xx_ts.block.yml">[3]</a></dd></dl>
<!--
NewPP limit report
Cached time: 20260524135544
Cache expiry: 86400
Reduced expiry: false
Complications: []
CPU time usage: 0.008 seconds
Real time usage: 0.009 seconds
Preprocessor visited node count: 11/1000000
Post‐expand include size: 0/2097152 bytes
Template argument size: 0/2097152 bytes
Highest expansion depth: 2/100
Expensive parser function count: 0/100
Unstrip recursion depth: 0/20
Unstrip post‐expand size: 0/5000000 bytes
-->
<!--
Transclusion expansion time report (%,ms,calls,template)
100.00% 0.000 1 -total
-->
<!-- Saved in parser cache with key mediawiki:pcache:idhash:3920-0!canonical and timestamp 20260524135544 and revision id 13769. Rendering was triggered because: page-view
-->
</div>
<div class="printfooter" data-nosnippet="">Retrieved from "<a dir="ltr" href="https://wiki.gnuradio.org/index.php?title=Correlate_Access_Code_-_Tag_Stream&amp;oldid=13769">https://wiki.gnuradio.org/index.php?title=Correlate_Access_Code_-_Tag_Stream&amp;oldid=13769</a>"</div></div>
<div id="catlinks" class="catlinks" data-mw="interface"><div id="mw-normal-catlinks" class="mw-normal-catlinks"><a href="/index.php?title=Special:Categories" title="Special:Categories">Category</a>: <ul><li><a href="/index.php?title=Category:Block_Docs" title="Category:Block Docs">Block Docs</a></li></ul></div></div>
</div>
</div>
<div id="mw-navigation">
<h2>Navigation menu</h2>
<div id="mw-head">
<nav id="p-personal" class="mw-portlet mw-portlet-personal vector-user-menu-legacy vector-menu" aria-labelledby="p-personal-label" >
<h3
id="p-personal-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Personal tools</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="pt-login" class="mw-list-item"><a href="/index.php?title=Special:UserLogin&amp;returnto=Correlate+Access+Code+-+Tag+Stream" title="You are encouraged to log in; however, it is not mandatory [o]" accesskey="o"><span>Log in</span></a></li>
</ul>
</div>
</nav>
<div id="left-navigation">
<nav id="p-namespaces" class="mw-portlet mw-portlet-namespaces vector-menu-tabs vector-menu-tabs-legacy vector-menu" aria-labelledby="p-namespaces-label" >
<h3
id="p-namespaces-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Namespaces</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="ca-nstab-main" class="selected mw-list-item"><a href="/index.php?title=Correlate_Access_Code_-_Tag_Stream" title="View the content page [c]" accesskey="c"><span>Page</span></a></li><li id="ca-talk" class="new mw-list-item"><a href="/index.php?title=Talk:Correlate_Access_Code_-_Tag_Stream&amp;action=edit&amp;redlink=1" rel="discussion" class="new" title="Discussion about the content page (page does not exist) [t]" accesskey="t"><span>Discussion</span></a></li>
</ul>
</div>
</nav>
<nav id="p-variants" class="mw-portlet mw-portlet-variants emptyPortlet vector-menu-dropdown vector-menu" aria-labelledby="p-variants-label" >
<input type="checkbox"
id="p-variants-checkbox"
role="button"
aria-haspopup="true"
data-event-name="ui.dropdown-p-variants"
class="vector-menu-checkbox"
aria-labelledby="p-variants-label"
>
<label
id="p-variants-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">English</span>
</label>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
</ul>
</div>
</nav>
</div>
<div id="right-navigation">
<nav id="p-views" class="mw-portlet mw-portlet-views vector-menu-tabs vector-menu-tabs-legacy vector-menu" aria-labelledby="p-views-label" >
<h3
id="p-views-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Views</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="ca-view" class="selected mw-list-item"><a href="/index.php?title=Correlate_Access_Code_-_Tag_Stream"><span>Read</span></a></li><li id="ca-viewsource" class="mw-list-item"><a href="/index.php?title=Correlate_Access_Code_-_Tag_Stream&amp;action=edit" title="This page is protected.&#10;You can view its source [e]" accesskey="e"><span>View source</span></a></li><li id="ca-history" class="mw-list-item"><a href="/index.php?title=Correlate_Access_Code_-_Tag_Stream&amp;action=history" title="Past revisions of this page [h]" accesskey="h"><span>View history</span></a></li>
</ul>
</div>
</nav>
<nav id="p-cactions" class="mw-portlet mw-portlet-cactions emptyPortlet vector-menu-dropdown vector-menu" aria-labelledby="p-cactions-label" title="More options" >
<input type="checkbox"
id="p-cactions-checkbox"
role="button"
aria-haspopup="true"
data-event-name="ui.dropdown-p-cactions"
class="vector-menu-checkbox"
aria-labelledby="p-cactions-label"
>
<label
id="p-cactions-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">More</span>
</label>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
</ul>
</div>
</nav>
<div id="p-search" role="search" class="vector-search-box-vue vector-search-box-show-thumbnail vector-search-box-auto-expand-width vector-search-box">
<h3 >Search</h3>
<form action="/index.php" id="searchform" class="vector-search-box-form">
<div id="simpleSearch"
class="vector-search-box-inner"
data-search-loc="header-navigation">
<input class="vector-search-box-input"
type="search" name="search" placeholder="Search GNU Radio" aria-label="Search GNU Radio" autocapitalize="sentences" title="Search GNU Radio [f]" accesskey="f" id="searchInput"
>
<input type="hidden" name="title" value="Special:Search">
<input id="mw-searchButton"
class="searchButton mw-fallbackSearchButton" type="submit" name="fulltext" title="Search the pages for this text" value="Search">
<input id="searchButton"
class="searchButton" type="submit" name="go" title="Go to a page with this exact name if it exists" value="Go">
</div>
</form>
</div>
</div>
</div>
<div id="mw-panel" class="vector-legacy-sidebar">
<div id="p-logo" role="banner">
<a class="mw-wiki-logo" href="/index.php?title=Main_Page"
title="Visit the main page"></a>
</div>
<nav id="p-navigation" class="mw-portlet mw-portlet-navigation vector-menu-portal portal vector-menu" aria-labelledby="p-navigation-label" >
<h3
id="p-navigation-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Navigation</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="n-Wiki-Home" class="mw-list-item"><a href="/index.php?title=Main_Page"><span>Wiki Home</span></a></li><li id="n-GNU-Radio-Website" class="mw-list-item"><a href="https://gnuradio.org" rel="nofollow"><span>GNU Radio Website</span></a></li><li id="n-FAQ" class="mw-list-item"><a href="/index.php?title=FAQ"><span>FAQ</span></a></li><li id="n-Get-a-Wiki-Account" class="mw-list-item"><a href="/index.php?title=Wiki_account"><span>Get a Wiki Account</span></a></li>
</ul>
</div>
</nav>
<nav id="p-Guides" class="mw-portlet mw-portlet-Guides vector-menu-portal portal vector-menu" aria-labelledby="p-Guides-label" >
<h3
id="p-Guides-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Guides</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="n-Tutorials" class="mw-list-item"><a href="/index.php?title=Tutorials"><span>Tutorials</span></a></li><li id="n-Installing-GNU-Radio" class="mw-list-item"><a href="/index.php?title=InstallingGR"><span>Installing GNU Radio</span></a></li><li id="n-Contributing" class="mw-list-item"><a href="/index.php?title=Development"><span>Contributing</span></a></li>
</ul>
</div>
</nav>
<nav id="p-Wiki_Tools" class="mw-portlet mw-portlet-Wiki_Tools vector-menu-portal portal vector-menu" aria-labelledby="p-Wiki_Tools-label" >
<h3
id="p-Wiki_Tools-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Wiki Tools</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="n-recentchanges" class="mw-list-item"><a href="/index.php?title=Special:RecentChanges" title="A list of recent changes in the wiki [r]" accesskey="r"><span>Recent changes</span></a></li><li id="n-randompage" class="mw-list-item"><a href="/index.php?title=Special:Random" title="Load a random page [x]" accesskey="x"><span>Random page</span></a></li><li id="n-help" class="mw-list-item"><a href="https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents" title="The place to find out"><span>Help</span></a></li>
</ul>
</div>
</nav>
<nav id="p-tb" class="mw-portlet mw-portlet-tb vector-menu-portal portal vector-menu" aria-labelledby="p-tb-label" >
<h3
id="p-tb-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Tools</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="t-whatlinkshere" class="mw-list-item"><a href="/index.php?title=Special:WhatLinksHere/Correlate_Access_Code_-_Tag_Stream" title="A list of all wiki pages that link here [j]" accesskey="j"><span>What links here</span></a></li><li id="t-recentchangeslinked" class="mw-list-item"><a href="/index.php?title=Special:RecentChangesLinked/Correlate_Access_Code_-_Tag_Stream" rel="nofollow" title="Recent changes in pages linked from this page [k]" accesskey="k"><span>Related changes</span></a></li><li id="t-specialpages" class="mw-list-item"><a href="/index.php?title=Special:SpecialPages" title="A list of all special pages [q]" accesskey="q"><span>Special pages</span></a></li><li id="t-print" class="mw-list-item"><a href="javascript:print();" rel="alternate" title="Printable version of this page [p]" accesskey="p"><span>Printable version</span></a></li><li id="t-permalink" class="mw-list-item"><a href="/index.php?title=Correlate_Access_Code_-_Tag_Stream&amp;oldid=13769" title="Permanent link to this revision of this page"><span>Permanent link</span></a></li><li id="t-info" class="mw-list-item"><a href="/index.php?title=Correlate_Access_Code_-_Tag_Stream&amp;action=info" title="More information about this page"><span>Page information</span></a></li>
</ul>
</div>
</nav>
</div>
</div>
<footer id="footer" class="mw-footer" >
<ul id="footer-info">
<li id="footer-info-lastmod"> This page was last edited on 15 February 2024, at 16:29.</li>
<li id="footer-info-copyright">Content is available under <a class="external" rel="nofollow" href="http://creativecommons.org/licenses/by-sa/3.0/">Creative Commons Attribution-ShareAlike</a> unless otherwise noted.</li>
</ul>
<ul id="footer-places">
<li id="footer-places-privacy"><a href="/index.php?title=GNU_Radio:Privacy_policy">Privacy policy</a></li>
<li id="footer-places-about"><a href="/index.php?title=GNU_Radio:About">About GNU Radio</a></li>
<li id="footer-places-disclaimers"><a href="/index.php?title=GNU_Radio:General_disclaimer">Disclaimers</a></li>
</ul>
<ul id="footer-icons" class="noprint">
<li id="footer-copyrightico"><a href="http://creativecommons.org/licenses/by-sa/3.0/" class="cdx-button cdx-button--fake-button cdx-button--size-large cdx-button--fake-button--enabled"><img src="/resources/assets/licenses/cc-by-sa.png" alt="Creative Commons Attribution-ShareAlike" width="88" height="31" loading="lazy"></a></li>
<li id="footer-poweredbyico"><a href="https://www.mediawiki.org/" class="cdx-button cdx-button--fake-button cdx-button--size-large cdx-button--fake-button--enabled"><img src="/resources/assets/poweredby_mediawiki.svg" alt="Powered by MediaWiki" width="88" height="31" loading="lazy"></a></li>
</ul>
</footer>
<script>(RLQ=window.RLQ||[]).push(function(){mw.config.set({"wgBackendResponseTime":136,"wgPageParseReport":{"limitreport":{"cputime":"0.008","walltime":"0.009","ppvisitednodes":{"value":11,"limit":1000000},"postexpandincludesize":{"value":0,"limit":2097152},"templateargumentsize":{"value":0,"limit":2097152},"expansiondepth":{"value":2,"limit":100},"expensivefunctioncount":{"value":0,"limit":100},"unstrip-depth":{"value":0,"limit":20},"unstrip-size":{"value":0,"limit":5000000},"timingprofile":["100.00% 0.000 1 -total"]},"cachereport":{"timestamp":"20260524135544","ttl":86400,"transientcontent":false}}});});</script>
<script>(function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'a01041d9deb5b2d1',t:'MTc3OTY2NzE5OQ=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();</script></body>
</html>

View file

@ -0,0 +1,667 @@
<!DOCTYPE html>
<html class="client-nojs" lang="en" dir="ltr">
<head>
<meta charset="UTF-8">
<title>Stream Tags - GNU Radio</title>
<script>(function(){var className="client-js";var cookie=document.cookie.match(/(?:^|; )mediawikimwclientpreferences=([^;]+)/);if(cookie){cookie[1].split('%2C').forEach(function(pref){className=className.replace(new RegExp('(^| )'+pref.replace(/-clientpref-\w+$|[^\w-]+/g,'')+'-clientpref-\\w+( |$)'),'$1'+pref+'$2');});}document.documentElement.className=className;}());RLCONF={"wgBreakFrames":false,"wgSeparatorTransformTable":["",""],"wgDigitTransformTable":["",""],"wgDefaultDateFormat":"dmy","wgMonthNames":["","January","February","March","April","May","June","July","August","September","October","November","December"],"wgRequestId":"ae390450ba32e3a8e9e695ff","wgCanonicalNamespace":"","wgCanonicalSpecialPageName":false,"wgNamespaceNumber":0,"wgPageName":"Stream_Tags","wgTitle":"Stream Tags","wgCurRevisionId":13335,"wgRevisionId":13335,"wgArticleId":3482,"wgIsArticle":true,"wgIsRedirect":false,"wgAction":"view","wgUserName":null,"wgUserGroups":["*"],"wgCategories":["Usage Manual"],"wgPageViewLanguage":"en","wgPageContentLanguage":"en","wgPageContentModel":"wikitext","wgRelevantPageName":"Stream_Tags","wgRelevantArticleId":3482,"wgIsProbablyEditable":false,"wgRelevantPageIsProbablyEditable":false,"wgRestrictionEdit":[],"wgRestrictionMove":[],"wgCheckUserClientHintsHeadersJsApi":["architecture","bitness","brands","fullVersionList","mobile","model","platform","platformVersion"]};
RLSTATE={"site.styles":"ready","user.styles":"ready","user":"ready","user.options":"loading","ext.pygments":"ready","skins.vector.styles.legacy":"ready"};RLPAGEMODULES=["ext.pygments.view","site","mediawiki.page.ready","mediawiki.toc","skins.vector.legacy.js","ext.checkUser.clientHints","ext.moderation.notify","ext.moderation.notify.desktop"];</script>
<script>(RLQ=window.RLQ||[]).push(function(){mw.loader.impl(function(){return["user.options@12s5i",function($,jQuery,require,module){mw.user.tokens.set({"patrolToken":"+\\","watchToken":"+\\","csrfToken":"+\\"});
}];});});</script>
<link rel="stylesheet" href="/load.php?lang=en&amp;modules=ext.pygments%7Cskins.vector.styles.legacy&amp;only=styles&amp;skin=vector">
<script async="" src="/load.php?lang=en&amp;modules=startup&amp;only=scripts&amp;raw=1&amp;skin=vector"></script>
<meta name="generator" content="MediaWiki 1.43.6">
<meta name="robots" content="max-image-preview:standard">
<meta name="format-detection" content="telephone=no">
<meta name="viewport" content="width=1120">
<link rel="search" type="application/opensearchdescription+xml" href="/rest.php/v1/search" title="GNU Radio (en)">
<link rel="EditURI" type="application/rsd+xml" href="https://wiki.gnuradio.org/api.php?action=rsd">
<link rel="license" href="https://creativecommons.org/licenses/by-sa/3.0/">
<link rel="alternate" type="application/atom+xml" title="GNU Radio Atom feed" href="/index.php?title=Special:RecentChanges&amp;feed=atom">
</head>
<body class="skin-vector-legacy mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-Stream_Tags rootpage-Stream_Tags skin-vector action-view"><div id="mw-page-base" class="noprint"></div>
<div id="mw-head-base" class="noprint"></div>
<div id="content" class="mw-body" role="main">
<a id="top"></a>
<div id="siteNotice"></div>
<div class="mw-indicators">
</div>
<h1 id="firstHeading" class="firstHeading mw-first-heading"><span class="mw-page-title-main">Stream Tags</span></h1>
<div id="bodyContent" class="vector-body">
<div id="siteSub" class="noprint">From GNU Radio</div>
<div id="contentSub"><div id="mw-content-subtitle"></div></div>
<div id="contentSub2"></div>
<div id="jump-to-nav"></div>
<a class="mw-jump-link" href="#mw-head">Jump to navigation</a>
<a class="mw-jump-link" href="#searchInput">Jump to search</a>
<div id="mw-content-text" class="mw-body-content"><div class="mw-content-ltr mw-parser-output" lang="en" dir="ltr"><p><br />
This tutorial describes the use of stream tags. The next tutorial, <a href="/index.php?title=Polymorphic_Types_(PMTs)" title="Polymorphic Types (PMTs)">Polymorphic Types (PMTs)</a> covers Polymorphic_Types
</p>
<div id="toc" class="toc" role="navigation" aria-labelledby="mw-toc-heading"><input type="checkbox" role="button" id="toctogglecheckbox" class="toctogglecheckbox" style="display:none" /><div class="toctitle" lang="en" dir="ltr"><h2 id="mw-toc-heading">Contents</h2><span class="toctogglespan"><label class="toctogglelabel" for="toctogglecheckbox"></label></span></div>
<ul>
<li class="toclevel-1 tocsection-1"><a href="#Introduction"><span class="tocnumber">1</span> <span class="toctext">Introduction</span></a></li>
<li class="toclevel-1 tocsection-2"><a href="#Background"><span class="tocnumber">2</span> <span class="toctext">Background</span></a></li>
<li class="toclevel-1 tocsection-3"><a href="#API_Extensions_to_the_gr::block"><span class="tocnumber">3</span> <span class="toctext">API Extensions to the gr::block</span></a></li>
<li class="toclevel-1 tocsection-4"><a href="#Stream_Tags_API"><span class="tocnumber">4</span> <span class="toctext">Stream Tags API</span></a>
<ul>
<li class="toclevel-2 tocsection-5"><a href="#Adding_a_Tag_to_a_Stream"><span class="tocnumber">4.1</span> <span class="toctext">Adding a Tag to a Stream</span></a></li>
<li class="toclevel-2 tocsection-6"><a href="#Getting_tags_from_a_Stream"><span class="tocnumber">4.2</span> <span class="toctext">Getting tags from a Stream</span></a></li>
</ul>
</li>
<li class="toclevel-1 tocsection-7"><a href="#Tag_Propagation"><span class="tocnumber">5</span> <span class="toctext">Tag Propagation</span></a></li>
<li class="toclevel-1 tocsection-8"><a href="#Notes_on_How_to_Use_Tags"><span class="tocnumber">6</span> <span class="toctext">Notes on How to Use Tags</span></a></li>
<li class="toclevel-1 tocsection-9"><a href="#Example_Flowgraph"><span class="tocnumber">7</span> <span class="toctext">Example Flowgraph</span></a>
<ul>
<li class="toclevel-2 tocsection-10"><a href="#Adding_tags_to_the_QPSK_demodulator"><span class="tocnumber">7.1</span> <span class="toctext">Adding tags to the QPSK demodulator</span></a></li>
</ul>
</li>
<li class="toclevel-1 tocsection-11"><a href="#Use_case:_FIR_filters"><span class="tocnumber">8</span> <span class="toctext">Use case: FIR filters</span></a></li>
</ul>
</div>
<h2><span class="mw-headline" id="Introduction">Introduction</span></h2>
<p>Stream tags are an isosynchronous data stream that runs parallel to
the main data stream. A stream <i>tag</i> is generated by a block's work
function and from there on flows downstream alongside a particular sample,
until it reaches a sink or is forced to stop propagating by another
block.
</p><p>Stream tags are defined for a specific item in the data stream and are
formed as a key:value pair. The <i>key</i> identifies what the <i>value</i> represents
while the value holds the data that the tag contains. Both <i>key</i> and
<i>value</i> are <a href="/index.php?title=Polymorphic_Types_(PMTs)" title="Polymorphic Types (PMTs)">Polymorphic Types (PMTs)</a> where the <i>key</i> is a PMT symbol while
the <i>value</i> is any type of PMT and can therefore handle any data we wish
to pass. An additional part of the tag is the <i>srcid</i>, which is a PMT
symbol and is used to identify the block that created the tag (which
is usually the block's alias).
</p>
<h2><span class="mw-headline" id="Background">Background</span></h2>
<p>GNU Radio was originally a streaming system with no other mechanism to
pass data between blocks. Streams of data are a model that work well
for samples, bits, etc., but can lack for control and metadata.
</p><p>Part of this is solved using the existing message passing interface, which
allows blocks to subscribe to messages published by any other block in
the flowgraph (see <a href="/index.php?title=Message_Passing" title="Message Passing">Message Passing</a>). The main drawback to the
message passing system is that it works asynchronously, meaning that
there is no guarantee when a message may arrive relative to the data
stream.
</p>
<h2><span class="mw-headline" id="API_Extensions_to_the_gr::block">API Extensions to the gr::block</span></h2>
<p>To enable the stream tags, we have extended the API of gr::block to
understand <i>absolute</i> item numbers. In the data stream model, each
block's work function is given a buffer in the data stream that is
referenced from 0 to N-1. This is a <i>relative</i> offset into the data
stream. The absolute reference starts from the beginning of the
flowgraph and continues to count up with every item. Each input stream
is associated with a concept of the 'number of items read' and each
output stream has a 'number of items written'. These are retrieved during
runtime using the two API calls:
</p>
<pre>unsigned long int nitems_read(unsigned int which_input);
unsigned long int nitems_written(unsigned int which_output);
</pre>
<p>Each tag is associated with some item in this absolute time scale that
is calculated using these functions.
</p><p>Like the rest of the data stream, the number of items read/written are
only updated once during the call to work. So in a work function,
nitems_read/written will refer to the state of the data stream at the
start of the work function. We must therefore add to this value the
current relative offset in the data stream. So if we are iterating <i>i</i> over all output items, we would write the stream tag to output ports
at nitems_written(0)+i for the 0th output port.
</p>
<h2><span class="mw-headline" id="Stream_Tags_API">Stream Tags API</span></h2>
<p>The stream tags API is split into two parts: adding tags to a stream,
and getting tags from a stream.
Note that the functions described below are only meant to be accessed
within a call to general_work/work. While they can be called at other points
in time by a block, the behavior outside of work is undefined without
exact knowledge of the item counts in the buffers.
</p>
<h3><span class="mw-headline" id="Adding_a_Tag_to_a_Stream">Adding a Tag to a Stream</span></h3>
<p>We add a tag to a particular output stream of the block using:
</p>
<ul><li>gr::block::add_item_tag: Adds an item tag to a particular output port using a gr::tag_t data type or by specifying the tag values.</li></ul>
<p>We can output them to multiple output streams if we want, but to do so
means calling this function once for each port. This function can be
provided with a gr::tag_t data type, or each value of the tag can be
explicitly given.
</p><p>Again, a tag is defined as:
</p>
<ol><li>offset: The offset, in absolute item time, of the tag in the data stream.</li>
<li>key: the PMT symbol identifying the type of tag.</li>
<li>value: the PMT holding the data of the tag.</li>
<li>srcid: (optional) the PMT symbol identifying the block which created the tag.</li></ol>
<p>We can create a gr::tag_t structure to hold all of the above
information of a tag, which is probably the easiest/best way to do
it. The gr::tag_t struct is defined as having the same members as in
the above list. To add a gr::tag_t tag to a stream, use the function:
</p>
<pre> void add_item_tag(unsigned int which_output, const tag_t &amp;tag);
</pre>
<p>The secondary API allows us to create a tag by explicitly listing all
of the tag information in the function call:
</p>
<pre> void add_item_tag(unsigned int which_output,
uint64_t abs_offset,
const pmt::pmt_t &amp;key,
const pmt::pmt_t &amp;value,
const pmt::pmt_t &amp;srcid=pmt::PMT_F);
</pre>
<p>In Python, we can add a tag to a stream using one of the following:
</p>
<pre>add_item_tag(which_output, abs_offset, key, value)
add_item_tag(which_output, abs_offset, key, value, srcid)
</pre>
<p>Note that the key and value are both PMTs. To create a string type PMT you can use pmt.intern("example_key").
</p><p>Consider the following flowgraph as an example. We will have an embedded python block insert stream tags at random intervals, and view these tags on the QT GUI Time sink.
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Epy_stream_tags_example_flowchart.png" class="mw-file-description"><img src="/images/thumb/4/48/Epy_stream_tags_example_flowchart.png/800px-Epy_stream_tags_example_flowchart.png" decoding="async" width="800" height="126" class="mw-file-element" srcset="/images/4/48/Epy_stream_tags_example_flowchart.png 1.5x" /></a></span>
</p><p>Add the following code into the embedded python block. This code outputs the same signal as in the input, except with tags on randomly selected input samples:
</p>
<div class="mw-highlight mw-highlight-lang-python mw-content-ltr" dir="ltr"><pre><span></span> <span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
<span class="kn">from</span><span class="w"> </span><span class="nn">gnuradio</span><span class="w"> </span><span class="kn">import</span> <span class="n">gr</span>
<span class="kn">import</span><span class="w"> </span><span class="nn">pmt</span>
<span class="k">class</span><span class="w"> </span><span class="nc">blk</span><span class="p">(</span><span class="n">gr</span><span class="o">.</span><span class="n">sync_block</span><span class="p">):</span>
<span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">gr</span><span class="o">.</span><span class="n">sync_block</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span>
<span class="bp">self</span><span class="p">,</span>
<span class="n">name</span><span class="o">=</span><span class="s1">&#39;Embedded Python Block&#39;</span><span class="p">,</span>
<span class="n">in_sig</span><span class="o">=</span><span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">float32</span><span class="p">],</span>
<span class="n">out_sig</span><span class="o">=</span><span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">float32</span><span class="p">]</span>
<span class="p">)</span>
<span class="k">def</span><span class="w"> </span><span class="nf">work</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">input_items</span><span class="p">,</span> <span class="n">output_items</span><span class="p">):</span>
<span class="k">for</span> <span class="n">indx</span><span class="p">,</span> <span class="n">sample</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">input_items</span><span class="p">[</span><span class="mi">0</span><span class="p">]):</span> <span class="c1"># Enumerate through the input samples in port in0</span>
<span class="k">if</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">()</span> <span class="o">&gt;</span> <span class="mf">0.95</span><span class="p">:</span> <span class="c1"># 5% chance this sample is chosen</span>
<span class="n">key</span> <span class="o">=</span> <span class="n">pmt</span><span class="o">.</span><span class="n">intern</span><span class="p">(</span><span class="s2">&quot;example_key&quot;</span><span class="p">)</span>
<span class="n">value</span> <span class="o">=</span> <span class="n">pmt</span><span class="o">.</span><span class="n">intern</span><span class="p">(</span><span class="s2">&quot;example_value&quot;</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">add_item_tag</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="c1"># Write to output port 0</span>
<span class="bp">self</span><span class="o">.</span><span class="n">nitems_written</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="n">indx</span><span class="p">,</span> <span class="c1"># Index of the tag in absolute terms</span>
<span class="n">key</span><span class="p">,</span> <span class="c1"># Key of the tag</span>
<span class="n">value</span> <span class="c1"># Value of the tag</span>
<span class="p">)</span>
<span class="c1"># note: (self.nitems_written(0) + indx) is our current sample, in absolute time</span>
<span class="n">output_items</span><span class="p">[</span><span class="mi">0</span><span class="p">][:]</span> <span class="o">=</span> <span class="n">input_items</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="c1"># copy input to output</span>
<span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="n">output_items</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
</pre></div>
<p>You may expect an output similar to the screenshot of the time sink below.
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Epy_stream_tags_example_timesink.png" class="mw-file-description"><img src="/images/thumb/f/fb/Epy_stream_tags_example_timesink.png/800px-Epy_stream_tags_example_timesink.png" decoding="async" width="800" height="259" class="mw-file-element" srcset="/images/thumb/f/fb/Epy_stream_tags_example_timesink.png/1200px-Epy_stream_tags_example_timesink.png 1.5x, /images/thumb/f/fb/Epy_stream_tags_example_timesink.png/1600px-Epy_stream_tags_example_timesink.png 2x" /></a></span>
</p>
<h3><span class="mw-headline" id="Getting_tags_from_a_Stream">Getting tags from a Stream</span></h3>
<p>To get tags from a particular input stream, we have two
functions we can use:
</p>
<ol><li>gr::block::get_tags_in_range: Gets all tags from a particular input port between a certain range of items (in absolute item time).</li></ol>
<ol><li>gr::block::get_tags_in_window: Gets all tags from a particular input port between a certain range of items (in relative item time within the work function).</li></ol>
<p>The difference between these functions is working in absolute item
time versus relative item time. Both of these pass back vectors of
gr::tag_t, and they both allow
specifying a particular key (as a PMT symbol) to filter against
(or the fifth argument can be left out to search for all keys).
Filtering for a certain key reduces the effort inside the work function
for getting the right tag's data.
</p><p>For example, this call just returns any tags between the given range of items:
</p>
<pre> void get_tags_in_range(std::vector&lt;tag_t&gt; &amp;v,
unsigned int which_input,
uint64_t abs_start,
uint64_t abs_end);
</pre>
<p>Adding a fifth argument to this function allows us to filter on the
key <i>key</i>.
</p>
<pre> void get_tags_in_range(std::vector&lt;tag_t&gt; &amp;v,
unsigned int which_input,
uint64_t abs_start,
uint64_t abs_end,
const pmt::pmt_t &amp;key);
</pre>
<p>In Python, the main difference from the C++ function is that instead of having the first
argument be a vector where the tags are stored, the Python version
just returns a list of tags. We would use it like this:
</p>
<div class="mw-highlight mw-highlight-lang-python mw-content-ltr" dir="ltr"><pre><span></span> <span class="k">def</span><span class="w"> </span><span class="nf">work</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">input_items</span><span class="p">,</span> <span class="n">output_items</span><span class="p">):</span>
<span class="o">....</span>
<span class="n">tags</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_tags_in_window</span><span class="p">(</span><span class="n">which_input</span><span class="p">,</span> <span class="n">rel_start</span><span class="p">,</span> <span class="n">rel_end</span><span class="p">)</span>
<span class="o">....</span>
</pre></div>
<p>If you want to grab all tags on the samples currently being processed by work(), on input port 0, here's a minimal example of doing that:
</p>
<div class="mw-highlight mw-highlight-lang-python mw-content-ltr" dir="ltr"><pre><span></span> <span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
<span class="kn">from</span><span class="w"> </span><span class="nn">gnuradio</span><span class="w"> </span><span class="kn">import</span> <span class="n">gr</span>
<span class="kn">import</span><span class="w"> </span><span class="nn">pmt</span>
<span class="k">class</span><span class="w"> </span><span class="nc">blk</span><span class="p">(</span><span class="n">gr</span><span class="o">.</span><span class="n">sync_block</span><span class="p">):</span>
<span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">gr</span><span class="o">.</span><span class="n">sync_block</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span><span class="n">name</span><span class="o">=</span><span class="s1">&#39;Read Tags&#39;</span><span class="p">,</span> <span class="n">in_sig</span><span class="o">=</span><span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">float32</span><span class="p">],</span> <span class="n">out_sig</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
<span class="k">def</span><span class="w"> </span><span class="nf">work</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">input_items</span><span class="p">,</span> <span class="n">output_items</span><span class="p">):</span>
<span class="n">tags</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_tags_in_window</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">input_items</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
<span class="k">for</span> <span class="n">tag</span> <span class="ow">in</span> <span class="n">tags</span><span class="p">:</span>
<span class="n">key</span> <span class="o">=</span> <span class="n">pmt</span><span class="o">.</span><span class="n">to_python</span><span class="p">(</span><span class="n">tag</span><span class="o">.</span><span class="n">key</span><span class="p">)</span> <span class="c1"># convert from PMT to python string</span>
<span class="n">value</span> <span class="o">=</span> <span class="n">pmt</span><span class="o">.</span><span class="n">to_python</span><span class="p">(</span><span class="n">tag</span><span class="o">.</span><span class="n">value</span><span class="p">)</span> <span class="c1"># Note that the type(value) can be several things, it depends what PMT type it was</span>
<span class="nb">print</span> <span class="s1">&#39;key:&#39;</span><span class="p">,</span> <span class="n">key</span>
<span class="nb">print</span> <span class="s1">&#39;value:&#39;</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="nb">type</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="nb">print</span> <span class="s1">&#39;&#39;</span>
<span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="n">input_items</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
</pre></div>
<h2><span class="mw-headline" id="Tag_Propagation">Tag Propagation</span></h2>
<p>We now know how to add tags to streams, and how to read them. But what happens to tags after they were read? And what happens to tags that aren't used? After all, there are many blocks that don't care about tags at all.
</p><p>The answer is: It depends on the <i>tag propagation policy</i> of a block what happens to tags that enter it.<br />
There are <a rel="nofollow" class="external text" href="http://gnuradio.org/doc/doxygen/classgr_1_1block.html#abc40fd4d514724a5446a2b34b2352b4e">three policies</a> to choose from:
</p>
<ol><li>All-to-All: all tags from any input port are replicated to all output ports</li>
<li>One-to-One: tags from input port <i>i</i> are only copied to output port <i>i</i> (depends on num inputs = num outputs).</li>
<li>Dont: Does not propagate tags. Tags are either stopped here or the work function recreates them in some manner.</li></ol>
<p>The default behavior of a block is the 'All-to-All' method of
propagation.
</p><p>We generally set the tag propagation policy in the block's constructor using <a rel="nofollow" class="external text" href="http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a476e218927e426ac88c26431cbf086cd">@set_tag_propagation_policy</a>
</p>
<pre> void set_tag_propagation_policy(tag_propagation_policy_t p);
</pre>
<p>See the gr::block::tag_propagation_policy_t documentation for details
on this enum type.
</p><p>When the tag propagation policy is set to TPP_ALL_TO_ALL or TPP_ONE_TO_ONE, the GNU Radio scheduler uses any information available to figure out which output item corresponds to which input item. The block may read them and add new tags, but existing tags are automatically moved downstream in a manner deemed appropriate.
</p><p>When a tag is propagated through a block that has a rate change, the
item's offset in the data stream will change. The scheduler uses the
block's gr::block::relative_rate concept to perform the update on the
tag's offset value. The relative rate of a block determines the
relationship between the input rate and output rate. Decimators that
decimate by a factor of D have a relative rate of 1/D.
</p><p>Synchronous blocks (gr::sync_block), decimators (gr::sync_decimator),
and interpolators (gr::sync_interpolator) all have pre-defined and
well-understood relative rates. A standard gr::block has a default
relative rate of 1.0, but this must be set if it does not work this
way. Often, we use a gr::block because we have no pre-conceived notion
of the number of input to output items. If it is important to pass
tags through these blocks that respect the change in item value, we
would have to use the TPP_DONT tag propagation policy and handle the
propagation internally.
</p><p>In no case is the value of the tag modified when propagating through a
block. This becomes relevant when using <a href="/index.php?title=Tagged_Stream_Blocks" title="Tagged Stream Blocks">Tagged Stream Blocks</a>.
</p><p>As an example, consider an interpolating block. See the following flow graph:
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Tags_interp.grc.png" class="mw-file-description" title="tags_interp.grc.png"><img alt="tags_interp.grc.png" src="/images/thumb/0/02/Tags_interp.grc.png/700px-Tags_interp.grc.png" decoding="async" width="700" height="215" class="mw-file-element" srcset="/images/0/02/Tags_interp.grc.png 1.5x" /></a></span>
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Tags_interp.png" class="mw-file-description" title="tags_interp.png"><img alt="tags_interp.png" src="/images/thumb/a/a6/Tags_interp.png/500px-Tags_interp.png" decoding="async" width="500" height="487" class="mw-file-element" srcset="/images/thumb/a/a6/Tags_interp.png/750px-Tags_interp.png 1.5x, /images/a/a6/Tags_interp.png 2x" /></a></span><br />
</p><p>As you can tell, we produce tags on every 10th sample, and then pass them through a block that repeats every sample 100 times. Tags do not get repeated with the standard tag propagation policies (after all, they're not tag <i>manipulation</i> policies), so the scheduler takes care that every tag is put on the output item that corresponds to the input item it was on before. Here, the scheduler makes an educated guess and puts the tag on the <b>first</b> of 100 items.
</p><p>Note: We can't use one QT GUI Time Sink for both signals here, because they are running at a different rate. Note the time difference on the x-axis!
</p><p>On decimating blocks, the behavior is similar. Consider this very simple flow graph, and the position of the samples:
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Tags_decim.grc.png" class="mw-file-description" title="tags_decim.grc.png"><img alt="tags_decim.grc.png" src="/images/thumb/a/a3/Tags_decim.grc.png/400px-Tags_decim.grc.png" decoding="async" width="400" height="79" class="mw-file-element" srcset="/images/a/a3/Tags_decim.grc.png 1.5x" /></a></span>
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Tags_decim.png" class="mw-file-description" title="tags_decim.png"><img alt="tags_decim.png" src="/images/thumb/a/a3/Tags_decim.png/500px-Tags_decim.png" decoding="async" width="500" height="319" class="mw-file-element" srcset="/images/a/a3/Tags_decim.png 1.5x" /></a></span><br />
</p><p><br />
We can see that no tags are destroyed, and tags are indeed spaced at one-tenth of the original spacing of 100 items. Of course, the actual item that was passed through the block might be destroyed, or modified (think of a decimating FIR filter).
</p><p>In fact, this works with any rate-changing block. Note that there are cases where the relation of tag positions of in- and output are ambiguous, the GNU Radio scheduler will then try and get as close as possible.
</p><p>Here's another interesting example: Consider this flow graph, which has a delay block, and the position of the tags after it:
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Tags_ramp_delay.grc.png" class="mw-file-description" title="tags_ramp_delay.grc.png"><img alt="tags_ramp_delay.grc.png" src="/images/thumb/0/0c/Tags_ramp_delay.grc.png/500px-Tags_ramp_delay.grc.png" decoding="async" width="500" height="148" class="mw-file-element" srcset="/images/0/0c/Tags_ramp_delay.grc.png 1.5x" /></a></span>
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Tags_ramp_delay.png" class="mw-file-description" title="tags_ramp_delay.png"><img alt="tags_ramp_delay.png" src="/images/thumb/c/cd/Tags_ramp_delay.png/500px-Tags_ramp_delay.png" decoding="async" width="500" height="263" class="mw-file-element" srcset="/images/thumb/c/cd/Tags_ramp_delay.png/750px-Tags_ramp_delay.png 1.5x, /images/c/cd/Tags_ramp_delay.png 2x" /></a></span><br />
</p><p><br />
Before the delay block, tags were positioned at the beginning of the ramp. After the delay, they're still in the same position! Would we inspect the source code of the delay block, we'd find that there is absolutely no tag handling code. Instead, the block <a rel="nofollow" class="external text" href="http://gnuradio.org/doc/doxygen/classgr_1_1block.html#acad5d6e62ea885cb77d19f72451581c2">declares a delay</a> to the scheduler, which then propagates tags with this delay.
</p><p>Using these mechanisms, we can let GNU Radio handle tag propagation for a large set of cases. For specialized or corner cases, there is no option than to set the tag propagation policy to <code>TPP_DONT</code> and manually propagate tags (actually, there's another way: Say we want most tags to propagate normally, but a select few should be treated differently. We can use <a rel="nofollow" class="external text" href="http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a461f6cc92174e83b10c3ec5336036768"><code>remove_item_tag()</code></a> (DEPRECATED. Will be removed in 3.8.) to remove these tags from the input; they will then not be propagated any more even if the tag propagation policy is set to something other than <code>TPP_DONT</code>. But that's more advanced use and will not be elaborated on here).
</p>
<h2><span class="mw-headline" id="Notes_on_How_to_Use_Tags">Notes on How to Use Tags</span></h2>
<p>Tags can be very useful to an application, and their use is
spreading. USRP sources generate tag information on the time, sample
rate, and frequency of the board if anything changes. We have a metadata
file source/sink (see <a href="/index.php?title=Metadata_Information" title="Metadata Information">Metadata_Information</a>) that use tags to store
information about the data stream. But there are things to think about
when using tags in a block.
</p><p>First, when tags are not being used, there is almost no effect on the
scheduler. However, when we use tags, we add overhead by getting and
extracting tags from a data stream. We also use overhead in
propagating the tags. For each tag, each block must copy a vector of
tags from the output port(s) of one block to the input port(s) of the
next block(s). These copy operations can add up.
</p><p>The key is to minimize the use of tags. Use them when and only when
necessary and try to provide some control over how tags are generated
to control their frequency. A good example is the USRP source, which
generates a time tag. If it generated a tag with every sample, we
would have thousands of tags per second, which would add a significant
amount of overhead. This is because if we started at time t0 at
sample rate sr, then after N samples, we know that
we are now at time t0 + N/sr. So continuously producing new
tags adds no information.
</p><p>The main issue we need to deal with in the above situation is when
there is a discontinuity in the packets received from the USRP. Since
we have no way of knowing in the flowgraph how many samples were
potentially lost, we have lost track of the timing information. The
USRP driver recognizes when packets have been dropped and uses this to
queue another tag, which allows us to resync. Likewise, any point the
sample rate or frequency changes, a new tag is issued.
</p>
<h2><span class="mw-headline" id="Example_Flowgraph">Example Flowgraph</span></h2>
<p>Let's have a look at a simple example:
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Tut5_tagstest_fg.png" class="mw-file-description" title="tut5_tagstest_fg.png"><img alt="tut5_tagstest_fg.png" src="/images/thumb/7/76/Tut5_tagstest_fg.png/600px-Tut5_tagstest_fg.png" decoding="async" width="600" height="183" class="mw-file-element" srcset="/images/7/76/Tut5_tagstest_fg.png 1.5x" /></a></span>
</p><p>In this flow graph, we have two sources: A sinusoid and a tag strobe. A tag strobe is a block that will output a constant tag, in this case, on every 1000th item (the actual value of the items is always zero). Those sources get added up. The signal after the adder is identical to the sine wave we produced, because we are always adding zeros. However, the tags stay attached to the same position as they were coming from the tag strobe! This means every 1000th sample of the sinusoid now has a tag. The QT scope can display tags, and even trigger on them.
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Tut5_tagstest_scope.png" class="mw-file-description" title="tut5_tagstest_scope.png"><img alt="tut5_tagstest_scope.png" src="/images/thumb/d/da/Tut5_tagstest_scope.png/500px-Tut5_tagstest_scope.png" decoding="async" width="500" height="307" class="mw-file-element" srcset="/images/thumb/d/da/Tut5_tagstest_scope.png/750px-Tut5_tagstest_scope.png 1.5x, /images/d/da/Tut5_tagstest_scope.png 2x" /></a></span><br />
</p><p>We now have a mechanism to randomly attach any metadata to specific items. There are several blocks that use tags. One of them is the UHD Sink block, the driver used for transmitting with USRP devices. It will react to tags with certain keys, one of them being <code>tx_freq</code>, which can be used to set the transmit frequency of a USRP while streaming.
</p>
<h3><span class="mw-headline" id="Adding_tags_to_the_QPSK_demodulator">Adding tags to the QPSK demodulator</span></h3>
<p>Going back to our <a href="/index.php?title=Guided_Tutorial_PSK_Demodulation" class="mw-redirect" title="Guided Tutorial PSK Demodulation">QPSK demodulation example</a>, we might want to add a feature to tell downstream blocks that the demodulation is not going well. Remember the output of our block is always hard-decision, and we have to output something. So we could use tags to notify that the input is not well formed, and that the output is not reliable.
</p><p>As a failure criterion, we discuss the case where the input amplitude is too small, say smaller than 0.01. When the amplitude drops below this value, we output one tag. Another tag is only sent when the amplitude has recovered, and rises back above the threshold. We extend our work function like this:
</p>
<pre>if (std::abs(in[i]) &lt; 0.01 and not d_low_ampl_state) {
add_item_tag(0, // Port number
nitems_written(0) + i, // Offset
pmt::mp(&quot;amplitude_warning&quot;), // Key
pmt::from_double(std::abs(in[i])) // Value
);
d_low_ampl_state = true;
}
else if (std::abs(in[i]) &gt;= 0.01 and d_low_ampl_state) {
add_item_tag(0, // Port number
nitems_written(0) + i, // Offset
pmt::mp(&quot;amplitude_recovered&quot;), // Key
pmt::PMT_T // Value
);
d_low_ampl_state = false; // Reset state
}</pre>
<p>In Python, the code would look like this (assuming we have a member of our block class called <code>d_low_ampl_state</code>):
</p>
<pre># The vector 'in' is called 'in0' here because 'in' is a Python keyword
if abs(in0[i]) &lt; 0.01 and not d_low_ampl_state:
self.add_item_tag(0, # Port number
self.nitems_written(0) + i, # Offset
pmt.intern(&quot;amplitude_warning&quot;), # Key
pmt.from_double(numpy.double(abs(in0[i]))) # Value
# Note: We need to explicitly create a 'double' here,
# because in0[i] is an explicit 32-bit float here
)
self.d_low_ampl_state = True
elif abs(in0[i]) &gt;= 0.01 and d_low_ampl_state:
self.add_item_tag(0, # Port number
self.nitems_written(0) + i, # Offset
pmt.intern(&quot;amplitude_recovered&quot;), # Key
pmt.PMT_T # Value
)
self.d_low_ampl_state = False; // Reset state</pre>
<p>We can also create a tag data type <a rel="nofollow" class="external text" href="http://gnuradio.org/doc/doxygen/structgr_1_1tag__t.html">tag_t</a> and directly pass this along:
</p>
<pre>if (std::abs(in[i]) &lt; 0.01 and not d_low_ampl_state) {
tag_t tag;
tag.offset = nitems_written(0) + i;
tag.key = pmt::mp(&quot;amplitude_warning&quot;);
tag.value = pmt::from_double(std::abs(in[i]));
add_item_tag(0, tag);
d_low_ampl_state = true;
}</pre>
<p>Here's a flow graph that uses the tagged version of the demodulator. We input 20 valid QPSK symbols, then 10 zeros. Since the output of this block is always either 0, 1, 2 or 3, we normally have no way to see if the input was not clearly one of these values.
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Demod_with_tags.grc.png" class="mw-file-description" title="Demod_with_tags.grc.png"><img alt="Demod_with_tags.grc.png" src="/images/thumb/5/5d/Demod_with_tags.grc.png/750px-Demod_with_tags.grc.png" decoding="async" width="750" height="97" class="mw-file-element" srcset="/images/thumb/5/5d/Demod_with_tags.grc.png/1125px-Demod_with_tags.grc.png 1.5x, /images/5/5d/Demod_with_tags.grc.png 2x" /></a></span>
</p><p>Here's the output. You can see we have tags on those values which were not from a valid QPSK symbol, but from something unreliable.
</p><p><span typeof="mw:File"><a href="/index.php?title=File:Demod_with_tags.png" class="mw-file-description" title="Demod_with_tags.png"><img alt="Demod_with_tags.png" src="/images/thumb/7/7c/Demod_with_tags.png/500px-Demod_with_tags.png" decoding="async" width="500" height="430" class="mw-file-element" srcset="/images/7/7c/Demod_with_tags.png 1.5x" /></a></span><br />
</p>
<h2><span class="mw-headline" id="Use_case:_FIR_filters">Use case: FIR filters</span></h2>
<p>Assume we have a block that is actually an FIR filter. We want to let GNU Radio handle the tag propagation. How do we configure the block?
</p><p>Now, an FIR filter has one input and one output. So, it doesn't matter if we set the propagation policy to <code>TPP_ALL_TO_ALL</code> or <code>TPP_ONE_TO_ONE</code>, and we can leave it as the default. The items going in and those coming out are different, so how do we match input to output? Since we want to preserve the timing of the tag position, we need to use the filter's <b>group delay</b> as a delay for tags (which for this symmetric FIR filter is <code>(N-1)/2</code>, where <code>N</code> is the number of filter taps). Finally, we might be interpolating, decimating or both (say, for sample rate changes) and we need to tell the scheduler about this as well.
</p>
<!--
NewPP limit report
Cached time: 20260524115448
Cache expiry: 86400
Reduced expiry: false
Complications: [show‐toc]
CPU time usage: 0.030 seconds
Real time usage: 0.035 seconds
Preprocessor visited node count: 87/1000000
Post‐expand include size: 0/2097152 bytes
Template argument size: 0/2097152 bytes
Highest expansion depth: 2/100
Expensive parser function count: 3/100
Unstrip recursion depth: 0/20
Unstrip post‐expand size: 11107/5000000 bytes
-->
<!--
Transclusion expansion time report (%,ms,calls,template)
100.00% 0.000 1 -total
-->
<!-- Saved in parser cache with key mediawiki:pcache:idhash:3482-0!canonical and timestamp 20260524115448 and revision id 13335. Rendering was triggered because: diff-page
-->
</div>
<div class="printfooter" data-nosnippet="">Retrieved from "<a dir="ltr" href="https://wiki.gnuradio.org/index.php?title=Stream_Tags&amp;oldid=13335">https://wiki.gnuradio.org/index.php?title=Stream_Tags&amp;oldid=13335</a>"</div></div>
<div id="catlinks" class="catlinks" data-mw="interface"><div id="mw-normal-catlinks" class="mw-normal-catlinks"><a href="/index.php?title=Special:Categories" title="Special:Categories">Category</a>: <ul><li><a href="/index.php?title=Category:Usage_Manual" title="Category:Usage Manual">Usage Manual</a></li></ul></div></div>
</div>
</div>
<div id="mw-navigation">
<h2>Navigation menu</h2>
<div id="mw-head">
<nav id="p-personal" class="mw-portlet mw-portlet-personal vector-user-menu-legacy vector-menu" aria-labelledby="p-personal-label" >
<h3
id="p-personal-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Personal tools</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="pt-login" class="mw-list-item"><a href="/index.php?title=Special:UserLogin&amp;returnto=Stream+Tags" title="You are encouraged to log in; however, it is not mandatory [o]" accesskey="o"><span>Log in</span></a></li>
</ul>
</div>
</nav>
<div id="left-navigation">
<nav id="p-namespaces" class="mw-portlet mw-portlet-namespaces vector-menu-tabs vector-menu-tabs-legacy vector-menu" aria-labelledby="p-namespaces-label" >
<h3
id="p-namespaces-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Namespaces</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="ca-nstab-main" class="selected mw-list-item"><a href="/index.php?title=Stream_Tags" title="View the content page [c]" accesskey="c"><span>Page</span></a></li><li id="ca-talk" class="new mw-list-item"><a href="/index.php?title=Talk:Stream_Tags&amp;action=edit&amp;redlink=1" rel="discussion" class="new" title="Discussion about the content page (page does not exist) [t]" accesskey="t"><span>Discussion</span></a></li>
</ul>
</div>
</nav>
<nav id="p-variants" class="mw-portlet mw-portlet-variants emptyPortlet vector-menu-dropdown vector-menu" aria-labelledby="p-variants-label" >
<input type="checkbox"
id="p-variants-checkbox"
role="button"
aria-haspopup="true"
data-event-name="ui.dropdown-p-variants"
class="vector-menu-checkbox"
aria-labelledby="p-variants-label"
>
<label
id="p-variants-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">English</span>
</label>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
</ul>
</div>
</nav>
</div>
<div id="right-navigation">
<nav id="p-views" class="mw-portlet mw-portlet-views vector-menu-tabs vector-menu-tabs-legacy vector-menu" aria-labelledby="p-views-label" >
<h3
id="p-views-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Views</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="ca-view" class="selected mw-list-item"><a href="/index.php?title=Stream_Tags"><span>Read</span></a></li><li id="ca-viewsource" class="mw-list-item"><a href="/index.php?title=Stream_Tags&amp;action=edit" title="This page is protected.&#10;You can view its source [e]" accesskey="e"><span>View source</span></a></li><li id="ca-history" class="mw-list-item"><a href="/index.php?title=Stream_Tags&amp;action=history" title="Past revisions of this page [h]" accesskey="h"><span>View history</span></a></li>
</ul>
</div>
</nav>
<nav id="p-cactions" class="mw-portlet mw-portlet-cactions emptyPortlet vector-menu-dropdown vector-menu" aria-labelledby="p-cactions-label" title="More options" >
<input type="checkbox"
id="p-cactions-checkbox"
role="button"
aria-haspopup="true"
data-event-name="ui.dropdown-p-cactions"
class="vector-menu-checkbox"
aria-labelledby="p-cactions-label"
>
<label
id="p-cactions-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">More</span>
</label>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
</ul>
</div>
</nav>
<div id="p-search" role="search" class="vector-search-box-vue vector-search-box-show-thumbnail vector-search-box-auto-expand-width vector-search-box">
<h3 >Search</h3>
<form action="/index.php" id="searchform" class="vector-search-box-form">
<div id="simpleSearch"
class="vector-search-box-inner"
data-search-loc="header-navigation">
<input class="vector-search-box-input"
type="search" name="search" placeholder="Search GNU Radio" aria-label="Search GNU Radio" autocapitalize="sentences" title="Search GNU Radio [f]" accesskey="f" id="searchInput"
>
<input type="hidden" name="title" value="Special:Search">
<input id="mw-searchButton"
class="searchButton mw-fallbackSearchButton" type="submit" name="fulltext" title="Search the pages for this text" value="Search">
<input id="searchButton"
class="searchButton" type="submit" name="go" title="Go to a page with this exact name if it exists" value="Go">
</div>
</form>
</div>
</div>
</div>
<div id="mw-panel" class="vector-legacy-sidebar">
<div id="p-logo" role="banner">
<a class="mw-wiki-logo" href="/index.php?title=Main_Page"
title="Visit the main page"></a>
</div>
<nav id="p-navigation" class="mw-portlet mw-portlet-navigation vector-menu-portal portal vector-menu" aria-labelledby="p-navigation-label" >
<h3
id="p-navigation-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Navigation</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="n-Wiki-Home" class="mw-list-item"><a href="/index.php?title=Main_Page"><span>Wiki Home</span></a></li><li id="n-GNU-Radio-Website" class="mw-list-item"><a href="https://gnuradio.org" rel="nofollow"><span>GNU Radio Website</span></a></li><li id="n-FAQ" class="mw-list-item"><a href="/index.php?title=FAQ"><span>FAQ</span></a></li><li id="n-Get-a-Wiki-Account" class="mw-list-item"><a href="/index.php?title=Wiki_account"><span>Get a Wiki Account</span></a></li>
</ul>
</div>
</nav>
<nav id="p-Guides" class="mw-portlet mw-portlet-Guides vector-menu-portal portal vector-menu" aria-labelledby="p-Guides-label" >
<h3
id="p-Guides-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Guides</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="n-Tutorials" class="mw-list-item"><a href="/index.php?title=Tutorials"><span>Tutorials</span></a></li><li id="n-Installing-GNU-Radio" class="mw-list-item"><a href="/index.php?title=InstallingGR"><span>Installing GNU Radio</span></a></li><li id="n-Contributing" class="mw-list-item"><a href="/index.php?title=Development"><span>Contributing</span></a></li>
</ul>
</div>
</nav>
<nav id="p-Wiki_Tools" class="mw-portlet mw-portlet-Wiki_Tools vector-menu-portal portal vector-menu" aria-labelledby="p-Wiki_Tools-label" >
<h3
id="p-Wiki_Tools-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Wiki Tools</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="n-recentchanges" class="mw-list-item"><a href="/index.php?title=Special:RecentChanges" title="A list of recent changes in the wiki [r]" accesskey="r"><span>Recent changes</span></a></li><li id="n-randompage" class="mw-list-item"><a href="/index.php?title=Special:Random" title="Load a random page [x]" accesskey="x"><span>Random page</span></a></li><li id="n-help" class="mw-list-item"><a href="https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents" title="The place to find out"><span>Help</span></a></li>
</ul>
</div>
</nav>
<nav id="p-tb" class="mw-portlet mw-portlet-tb vector-menu-portal portal vector-menu" aria-labelledby="p-tb-label" >
<h3
id="p-tb-label"
class="vector-menu-heading "
>
<span class="vector-menu-heading-label">Tools</span>
</h3>
<div class="vector-menu-content">
<ul class="vector-menu-content-list">
<li id="t-whatlinkshere" class="mw-list-item"><a href="/index.php?title=Special:WhatLinksHere/Stream_Tags" title="A list of all wiki pages that link here [j]" accesskey="j"><span>What links here</span></a></li><li id="t-recentchangeslinked" class="mw-list-item"><a href="/index.php?title=Special:RecentChangesLinked/Stream_Tags" rel="nofollow" title="Recent changes in pages linked from this page [k]" accesskey="k"><span>Related changes</span></a></li><li id="t-specialpages" class="mw-list-item"><a href="/index.php?title=Special:SpecialPages" title="A list of all special pages [q]" accesskey="q"><span>Special pages</span></a></li><li id="t-print" class="mw-list-item"><a href="javascript:print();" rel="alternate" title="Printable version of this page [p]" accesskey="p"><span>Printable version</span></a></li><li id="t-permalink" class="mw-list-item"><a href="/index.php?title=Stream_Tags&amp;oldid=13335" title="Permanent link to this revision of this page"><span>Permanent link</span></a></li><li id="t-info" class="mw-list-item"><a href="/index.php?title=Stream_Tags&amp;action=info" title="More information about this page"><span>Page information</span></a></li>
</ul>
</div>
</nav>
</div>
</div>
<footer id="footer" class="mw-footer" >
<ul id="footer-info">
<li id="footer-info-lastmod"> This page was last edited on 17 August 2023, at 18:46.</li>
<li id="footer-info-copyright">Content is available under <a class="external" rel="nofollow" href="http://creativecommons.org/licenses/by-sa/3.0/">Creative Commons Attribution-ShareAlike</a> unless otherwise noted.</li>
</ul>
<ul id="footer-places">
<li id="footer-places-privacy"><a href="/index.php?title=GNU_Radio:Privacy_policy">Privacy policy</a></li>
<li id="footer-places-about"><a href="/index.php?title=GNU_Radio:About">About GNU Radio</a></li>
<li id="footer-places-disclaimers"><a href="/index.php?title=GNU_Radio:General_disclaimer">Disclaimers</a></li>
</ul>
<ul id="footer-icons" class="noprint">
<li id="footer-copyrightico"><a href="http://creativecommons.org/licenses/by-sa/3.0/" class="cdx-button cdx-button--fake-button cdx-button--size-large cdx-button--fake-button--enabled"><img src="/resources/assets/licenses/cc-by-sa.png" alt="Creative Commons Attribution-ShareAlike" width="88" height="31" loading="lazy"></a></li>
<li id="footer-poweredbyico"><a href="https://www.mediawiki.org/" class="cdx-button cdx-button--fake-button cdx-button--size-large cdx-button--fake-button--enabled"><img src="/resources/assets/poweredby_mediawiki.svg" alt="Powered by MediaWiki" width="88" height="31" loading="lazy"></a></li>
</ul>
</footer>
<script>(RLQ=window.RLQ||[]).push(function(){mw.config.set({"wgBackendResponseTime":149,"wgPageParseReport":{"limitreport":{"cputime":"0.030","walltime":"0.035","ppvisitednodes":{"value":87,"limit":1000000},"postexpandincludesize":{"value":0,"limit":2097152},"templateargumentsize":{"value":0,"limit":2097152},"expansiondepth":{"value":2,"limit":100},"expensivefunctioncount":{"value":3,"limit":100},"unstrip-depth":{"value":0,"limit":20},"unstrip-size":{"value":11107,"limit":5000000},"timingprofile":["100.00% 0.000 1 -total"]},"cachereport":{"timestamp":"20260524115448","ttl":86400,"transientcontent":false}}});});</script>
<script>(function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'a01041dd6b486930',t:'MTc3OTY2NzE5OQ=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();</script></body>
</html>

View file

@ -0,0 +1,451 @@
liquid-dsp documentation
========================
Software-Defined Radio Digital Signal Processing Library -
`https://liquidsdr.org <https://liquidsdr.org>`_
liquid-dsp is a free and open-source digital signal processing (DSP)
library designed specifically for software-defined radios on embedded
platforms. The aim is to provide a lightweight DSP library that does not
rely on a myriad of external dependencies or proprietary and otherwise
cumbersome frameworks. All signal processing elements are designed to be
flexible, scalable, and dynamic, including filters, filter design,
oscillators, modems, synchronizers, complex mathematical operations, and
much more.
.. code-block:: c
// get in, process data, get out
#include <liquid/liquid.h>
int main() {
unsigned int M = 4; // interpolation factor
unsigned int m = 12; // filter delay [symbols]
float As = 60.0f; // filter stop-band attenuation [dB]
// create interpolator from prototype
firinterp_crcf interp = firinterp_crcf_create_kaiser(M,m,As);
float complex x = 1.0f; // input sample
float complex y[M]; // interpolated output buffer
// repeat on input sample data as needed
{
firinterp_crcf_execute(interp, x, y);
}
// destroy interpolator object
firinterp_crcf_destroy(interp);
return 0;
}
For more information, please refer to the
`documentation <https://liquidsdr.org/doc>`_ online.
Installation and Dependencies
=============================
liquid-dsp only relies on ``libc`` and ``libm`` (standard C and math)
libraries to run; however liquid will take advantage of other libraries
(such as `FFTW <http://www.fftw.org>`_) if they are available.
Starting with version 1.7.0, liquid-dsp has moved to the
`CMake <https://cmake.org>`_ build system which can be installed with
``brew install cmake`` on macOS,
``sudo apt-get install cmake`` on Debian variants.
Installation
------------
The recommended way to obtain the source code is to clone the entire
`repository <https://github.com/jgaeddert/liquid-dsp>`_ from
`GitHub <https://github.com>`_:
.. code-block:: bash
git clone git://github.com/jgaeddert/liquid-dsp.git
Building and installing the main library is a simple as
.. code-block:: bash
mkdir build
cd build
cmake ..
make
sudo make install
If you are installing on Linux for the first time, you will also need
to rebind your dynamic libraries with ``sudo ldconfig`` to make the
shared object available.
This is not necessary on macOS.
Run all test scripts
--------------------
Source code validation is a critical step in any software library,
particularly for verifying the portability of code to different
processors and platforms. Packaged with liquid-dsp are a number of
automatic test scripts to validate the correctness of the source code.
The test scripts are located under each module's ``tests/`` directory and
take the form of a C source file. When configured with ``BUILD_AUTOTESTS``
enabled, these tests are parsed, compiled, and linked into an executable
which will run the tests.
.. code-block:: bash
./xautotest
# ...
# autotest seed: 1773004345
# ==================================
# PASSED ALL 716651 CHECKS
# ==================================
There are currently more than 700,000 checks across 1,316 tests to verify
functional correctness. Drop me a line if these aren't running on your platform.
Testing Code Coverage
---------------------
In addition to the full test suite, you can configure ``gcc`` to export symbol
files to check for code coverage and then use ``gcovr`` to generate a full
report of precisely which lines are covered in the autotests. These symbol
files aren't generated by default and need to be enabled at compile-time
through a CMake option:
.. code-block:: bash
cmake -DBUILD_AUTOTESTS=ON -DCOVERAGE=ON ..
A coverage report can be generated by running the autotests and running
`gcovr <https://gcovr.com/en/stable/>`_:
.. code-block:: bash
make -j4 xautotest
./xautotest -q -o autotest.json
cd ..
gcovr --filter="src/.*/src/.*.c" --print-summary
# ...
# ------------------------------------------------------------------------------
# TOTAL 20922 17396 83%
# ------------------------------------------------------------------------------
# lines: 83.1% (17396 out of 20922)
# functions: 63.9% (1787 out of 2798)
# branches: 64.8% (5762 out of 8896)
Examples
--------
Nearly all signal processing elements have a corresponding example in
the ``examples/`` directory. Most example scripts generate an output
``.m`` file for plotting with `GNU octave <https://www.gnu.org/software/octave/>`_
All examples are built as stand-alone programs and can be compiled with
the ``BUILD_EXAMPLES`` CMake flag:
.. code-block:: bash
cmake -DBUILD_EXAMPLES=ON ..
make
./examples/modem_example -m qpsk
# <liquid.modemcf, scheme="qpsk", order=4>
# 0 : 0.70710677 + j* 0.70710677
# 1 : -0.70710677 + j* 0.70710677
# 2 : 0.70710677 + j* -0.70710677
# 3 : -0.70710677 + j* -0.70710677
# num sym errors: 0 / 4
# num bit errors: 0 / 8
# results written to modem_example.m.
Sometimes, however, it is useful to build one example individually.
This can be accomplished by directly targeting its binary
(e.g. ``make examples/modem_example``). The example then can be run at the
command line, viz. ``./examples/modem_example``.
Benchmarking Tool
-----------------
Packaged with liquid are benchmarks to determine the speed each signal
processing element can run on your machine. Initially the tool provides
an estimate of the processor's clock frequency and will then estimate
the number of trials so that each benchmark will take between 50 and
500 ms to run. You can build and run the benchmark program with the
following command:
.. code-block:: bash
make bench
Linking from C++
----------------
Compiling and linking to C++ programs is straightforward.
Just include ``<complex>`` before ``<liquid/liquid.h>`` and use
``std::complex<float>`` in favor of ``float complex``.
Here is the same example as the one above but in C++ instead of C:
.. code-block:: c++
// get in, process data, get out
#include <complex>
#include <liquid/liquid.h>
int main() {
unsigned int M = 4; // interpolation factor
unsigned int m = 12; // filter delay [symbols]
float As = 60.0f; // filter stop-band attenuation [dB]
// create interpolator from prototype
firinterp_crcf interp = firinterp_crcf_create_kaiser(M,m,As);
std::complex<float> x = 1.0f; // input sample
std::complex<float> y[M]; // interpolated output buffer
// repeat on input sample data as needed
{
firinterp_crcf_execute(interp, x, y);
}
// destroy interpolator object
firinterp_crcf_destroy(interp);
return 0;
}
Linking from External Project
-----------------------------
Installing with `CMake <https://cmake.org>`_ provides an
`exportable interface <https://cmake.org/cmake/help/latest/manual/cmake-packages.7.html>`_
(``liquid::liquid``) that allows easy integration into external applications.
For example, say you have a simple application in ``main.c`` that requires
liquid-dsp as a dependency:
.. code-block:: c
// main.c - test linking to liquid
#include <stdio.h>
#include <stdlib.h>
#include <liquid/liquid.h>
int main()
{
// create resampling object, print and return
printf("creating test object...\n");
resamp_crcf q = resamp_crcf_create(0.12345f, 12, 0.25f, 60.0f, 256);
resamp_crcf_print(q);
resamp_crcf_destroy(q);
return 0;
}
You now have two options within your project ``CMakeLists.txt`` file:
1. clone, build, and install using CMake, then use
`find_package <https://cmake.org/cmake/help/latest/command/find_package.html>`_
2. dynamically fetch liquid-dsp with
`FetchContent <https://cmake.org/cmake/help/latest/module/FetchContent.html>`_
to pull down the source and build completely within your application
Your ``CMakeLists.txt`` file might look something like this:
.. code-block:: cmake
# CMakeLists.txt - test finding package and linking against it
cmake_minimum_required(VERSION 3.10)
project(liquid_test C)
# option 1: check for local installation
#find_package(liquid REQUIRED)
# option 2: dynamically fetch content
include(FetchContent)
FetchContent_Declare(
liquid
GIT_REPOSITORY https://github.com/jgaeddert/liquid-dsp.git
GIT_TAG v1.7.0
)
set(BUILD_AUTOTESTS OFF CACHE INTERNAL "Disable building liquid tests")
set(BUILD_BENCHMARKS OFF CACHE INTERNAL "Disable building liquid benchmarks")
set(BUILD_EXAMPLES OFF CACHE INTERNAL "Disable building liquid examples")
FetchContent_MakeAvailable(liquid)
# FetchContent_Populate(liquid) <- older policy, see CMP0169
add_executable(main main.c)
target_link_libraries(main liquid)
You can then compile and run your application the typical way:
.. code-block:: bash
mkdir build
cd build
cmake ..
make
./main
# creating test object...
# <liquid.resamp_crcf, rate=0.12345, m=12, as=60.000, fc=0.25, npfb=256>
PlatformIO
----------
Cross-compling for embedded platforms is most easily achieved with
`platformio <https://platformio.org>`_.
Just add ``liquid-dsp`` to your ``platform.io`` list of dependencies:
.. code-block:: ini
[env:native]
platform = native
lib_deps = https://github.com/jgaeddert/liquid-dsp.git
To test this, compile the example program for a
`Raspberry Pi Pico microcontroller <https://www.raspberrypi.com/documentation/microcontrollers/pico-series.html>`_:
.. code-block:: bash
# create a virtual environment, install platformio, and compile an example
virtualenv pio
source pio/bin/activate
pip install platformio
pio ci --lib="." --board=pico examples/platformio_example.c
# ...
# Generating UF2 image
# elf2uf2 ".pio/build/pico/firmware.elf" ".pio/build/pico/firmware.uf2"
# Checking size .pio/build/pico/firmware.elf
# Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
# RAM: [== ] 15.5% (used 41820 bytes from 270336 bytes)
# Flash: [ ] 0.2% (used 5196 bytes from 2097152 bytes)
# Building .pio/build/pico/firmware.bin
# ===================== [SUCCESS] Took 23.63 seconds =====================
Build
-----
Here is a table of CMake options available for configuring liquid:
+------------------------+---------+--------------------------------------------------------------------+
| Option | Default | Description |
+========================+=========+====================================================================+
| ``BUILD_EXAMPLES`` | ON | Compile example programs |
+------------------------+---------+--------------------------------------------------------------------+
| ``BUILD_AUTOTESTS`` | ON | Parse and compile autotests into executable binary |
+------------------------+---------+--------------------------------------------------------------------+
| ``BUILD_BENCHMARKS`` | ON | Parse and compile benchmarks into executable binary |
+------------------------+---------+--------------------------------------------------------------------+
| ``BUILD_SHARED_LIBS`` | ON | Build shared library |
+------------------------+---------+--------------------------------------------------------------------+
| ``BUILD_STATIC_LIBS`` | OFF | Build static library |
+------------------------+---------+--------------------------------------------------------------------+
| ``ENABLE_SIMD`` | ON | Enable use of single instruction, multiple data (SIMD) extensions |
+------------------------+---------+--------------------------------------------------------------------+
| ``FIND_SIMD`` | ON | Try to find available SIMD instruction sets on host computer |
+------------------------+---------+--------------------------------------------------------------------+
| ``FIND_THREADS`` | ON | Try to find available threading library for concurrency |
+------------------------+---------+--------------------------------------------------------------------+
| ``FIND_FFTW`` | ON | Try to find `FFTW <http://www.fftw.org)>`_ if available |
+------------------------+---------+--------------------------------------------------------------------+
| ``BUILD_SANDBOX`` | OFF | Compile sandbox (testing) programs |
+------------------------+---------+--------------------------------------------------------------------+
| ``BUILD_DOC`` | OFF | Generate documentation |
+------------------------+---------+--------------------------------------------------------------------+
| ``COVERAGE`` | OFF | Set flags to enable code coverage testing |
+------------------------+---------+--------------------------------------------------------------------+
| ``ENABLE_LOGGING`` | ON | Enable global logging capabilities |
+------------------------+---------+--------------------------------------------------------------------+
| ``LOGGING_LEVEL`` | trace | Set minimum logging level to compile; |
| | | select from "trace", "debug", "info", "warn", "error", or "fatal" |
+------------------------+---------+--------------------------------------------------------------------+
| ``ENABLE_COLOR`` | ON | Enable use of color terminal output |
+------------------------+---------+--------------------------------------------------------------------+
For example, if you want to benchmark how fast a vector dot product
runs without SIMD extensions, you could run the following:
.. code-block:: bash
cmake -DENABLE_SIMD=OFF -DBUILD_BENCHMARKS=ON ..
make
./benchmark -s dotprod_rrrf
Available Modules
-----------------
* **agc**: automatic gain control, received signal strength
* **audio**: source audio encoders/decoders: cvsd, filterbanks
* **buffer**: internal buffering, circular/static, ports (threaded)
* **channel**: additive noise, multi-path fading, carrier phase/frequency
offsets, timing phase/rate offsets
* **core**: core functionality (error handling)
* **dotprod**: inner dot products (real, complex), vector sum of squares
* **equalization**: adaptive equalizers: least mean-squares, recursive
least squares, semi-blind
* **fec**: basic forward error correction codes including several
Hamming codes, single error correction/double error detection,
Golay block code, as well as several checksums and cyclic
redundancy checks, interleaving, soft decoding
* **fft**: fast Fourier transforms (arbitrary length), discrete sin/cos
transforms
* **filter**: finite/infinite impulse response, polyphase, hilbert,
interpolation, decimation, filter design, resampling, symbol
timing recovery
* **framing**: flexible framing structures for amazingly easy packet
software radio; dynamically adjust modulation and coding on the
fly with single- and multi-carrier framing structures
* **math**: transcendental functions not in the C standard library
(gamma, besseli, etc.), polynomial operations (curve-fitting,
root-finding, etc.)
* **matrix**: basic math, LU/QR/Cholesky factorization, inversion,
Gauss elimination, Gram-Schmidt decomposition, linear solver,
sparse matrix representation
* **modem**: modulate, demodulate, PSK, differential PSK, QAM, optimal
QAM, as well as analog and non-linear digital modulations GMSK)
* **multichannel**: filterbank channelizers, OFDM
* **nco**: numerically-controlled oscillator: mixing, frequency
synthesis, phase-locked loops
* **optim**: (non-linear optimization) Newton-Raphson, evoluationary
algorithms, gradient descent, line search
* **quantization**: analog/digital converters, compression/expansion
* **random**: (random number generators) uniform, exponential, gamma,
Nakagami-m, Gauss, Rice-K, Weibull
* **sequence**: linear feedback shift registers, complementary codes,
maximal-length sequences
* **utility**: useful miscellany, mostly bit manipulation (shifting,
packing, and unpacking of arrays)
* **vector**: generic vector operations
License
-------
liquid projects are released under the X11/MIT license.
By default, this project will try to link to
`FFTW <http://www.fftw.org>`_
if it is available on your build platform.
Because FFTW starting with version 1.3 is
`licensed <http://www.fftw.org/faq/section1.html>`_
under the `GNU General Public License v2 <http://www.fftw.org/doc/License-and-Copyright.html>`_
this unfortunately means that (and I'm clearly not a lawyer, here)
you cannot distribute ``liquid-dsp`` without also distributing the source code
if you link to FFTW.
This is a similar situation with the classic
`libfec <https://github.com/quiet/libfec>`_
which uses the
`GNU Lesser GPL <https://www.gnu.org/licenses/licenses.html#LGPL>`_.
Finally, ``liquid-dsp`` makes extensive use of GNU
`autoconf <https://www.gnu.org/software/autoconf/>`_,
`automake <https://www.gnu.org/software/automake/>`_,
and related tools.
These are fantastic libraries with amazing functionality and their authors
should be lauded for their efforts.
In a similar vain, much the software I write for a living I give away for
free;
however I believe in more permissive licenses to allow individuals the
flexibility to use software with fewer limitations.
If these restrictions are not acceptable, ``liquid-dsp`` can be compiled and run
without use of these external libraries, albeit a bit slower and with limited
functionality.
Short version: this code is copyrighted to me (Joseph D. Gaeddert),
I give you full permission to do whatever you want with it except remove my
name from the credits.
Seriously, go nuts! but take caution when linking to other libraries with
different licenses.
See the `license <https://opensource.org/licenses/MIT>`_ for specific terms.

File diff suppressed because one or more lines are too long

View file

@ -0,0 +1,170 @@
<img alt="Open Echo Cover" src="documentation/images/open_echo_logo.svg">
## Universal Open-Source SONAR Controller and Development Stack
An ongoing open-source hardware and software project for building sonar systems for testing, boating, bathymetry, and research.
The most commonly used hardware is the [TUSS4470 Arduino Shield](TUSS4470_shield_002/), which stacks on top of an Arduino Uno to drive the TUSS4470 ultrasonic driver.
The board can run the [RAW Data Firmware](TUSS4470_shield_002/getting_started_TUSS4470_firmware.md) to operate a wide variety of ultrasonic transducers, covering frequencies from 40 kHz up to 1000 kHz in different media such as air or water.
The [NMEA Output Firmware](TUSS4470_shield_002/arduino/NMEA_DBT_OUT/NMEA_DBT_OUT.ino) can read depth data from commercially available in-water ultrasonic transducers (e.g., on boats) and output NMEA0183-compatible data to a computer or a UART-connected device such as a Pixhawk or other controllers.
Open Echo has been tested on multiple ultrasonic transducers and is compatible with all of them—from car parking sensors to Lowrance Tripleshot side-scan transducers.
The [Python Interface Software](TUSS4470_shield_002/getting_started_interface.md) connects to Open Echo boards running the [RAW Data Firmware](TUSS4470_shield_002/getting_started_TUSS4470_firmware.md). It can display raw echo data, change configurations, output a TCP depth data stream, and more.
Check the [Getting Started Guide](TUSS4470_shield_002/README.md)!
If something is unclear or you find a bug, please open an issue.
Raw Data Waterfall chart in the Python Desktop software:
<img alt="Open Echo Interface Software" src="/documentation/images/echo_software_screenshot.jpg">
## Getting the Hardware
If you need the hardware, you can order it using the [Hardware Files](TUSS4470_shield_002/TUSS4470_shield_hardware/TUSS4470_shield) from a board + SMT house ([JLC recommended](https://jlcpcb.com/?from=Neumi)).
They can also be bought as a complete and tested set direclty from Elecrow: https://www.elecrow.com/open-echo-tuss4470-development-shield.html
If they’re out of stock, or if you’d prefer to order them within Germany to reduce shipping costs, please send me an email at: openechoes@gmail.com
All profits go directly toward supporting and advancing the Open Echo project!
<b>If you don't order the boards directly from me or Elecrow, please be aware that I can't provide support.</b>
[TUSS4470 Arduino Shield](TUSS4470_shield_002/):
<img alt="PCB overview TUSS4470" src="/documentation/images/TUSS4470_shield002.jpg">
### This project is currently in development. The [TUSS4470 Development Shield](TUSS4470_shield_002/) is ready for external use!
Development is ongoing! Check the documentation and Discord channel for the latest updates.
Want to stay updated or participate? Join the [Discord](https://discord.com/invite/rerCyqAcrw)!
Check the [Getting Started Guide](TUSS4470_shield_002/README.md).
## Vision
An accessible Open Source SONAR stack for development, research and real use:
- Open Source SONAR technology for differernt use cases
- support for a variety of commercial or DIY transducers in water and air
- relatively simple hardware that can be ordered from a board house
- easy entry for less experienced users
- community for exchange and development
- development of more advanced SONAR applications
--------
## Current State
- Universal TUSS4470-based Arduino shield for testing ultrasonic transducers
- Python interface software for raw data visualization, configuration, and TCP data output
- Tested depth range of at least 50 m in water
- NMEA0183 compatible (DBT data output to other devices)
- New all-in-one boards with STM32, TUSS4470, and boost converter currently in development
- picoW implementation in development (already successfull)
- UDP network RAW data transfer from picoW to Python interface running
--------
## Progress
The new (May 2025) [TUSS4470 Arduino Shield](TUSS4470_shield_002/) supports transducers from 40 kHz to 1000 kHz. It can drive them, receive echoes, filter signals, and send raw echo data to the Python backend.
Driver voltage can be supplied from Arduino VIN or via the external XT30 connector. With an MT3608 boost converter, USB can be used as a power supply and boosted to the desired drive voltage (manual soldering required).
The [TUSS4470 Arduino Shield](TUSS4470_shield_002/) is and will remain the main development board for this project. It is an excellent platform for testing and development.
It can drive a wide range of transducers at different voltages, but it is limited by RAM size and sampling speed. You can capture 1800 samples at 12 microseconds per sample (~18 m range in water) with 8-bit resolution. For longer ranges, you can add a delay to capture echoes from more distant objects.
This makes it a perfect board for learning, testing, and prototyping. Most software development is done using this board.
For special use cases, additional boards and software are under development.
Check the [development](development) folder for development work.
--------
## Open Echo Interface Software
Firmware examples are available in each project folder.
The [**Open Echo Interface Software**](TUSS4470_shield_002/echo_interface.py) allows you to control Open Echo boards, view live data, and adjust board settings.
Live echogram in water on Python software (6x speed):
<img alt="Software running with live echo data" src="documentation/images/echogram_fast.gif">
## Test results Baltic Sea
Tested using a built-in 150 kHz SEAFARER transducer, powered at 15-20 V via an MT3608 boost converter from USB. In the plots, data capture begins at the bottom.
<table>
<tr>
<td align="center">
<img src="documentation/images/baltic_sea_tests/38m_range.png" alt="38m Range" width="300"/><br/>
<sub><b>38 m range test</b></sub>
</td>
<td align="center">
<img src="documentation/images/baltic_sea_tests/kiel_channel.png" alt="Kiel Channel" width="300"/><br/>
<sub><b>Kiel Channel sand and mud</b></sub>
</td>
<td align="center">
<img src="documentation/images/baltic_sea_tests/stollergrund.png" alt="Stollergrund" width="300"/><br/>
<sub><b>Slope near Stollergrund</b></sub>
</td>
</tr>
<tr>
<td align="center">
<img src="documentation/images/baltic_sea_tests/stones_sand.png" alt="Stones and Sand" width="300"/><br/>
<sub><b>Stones and sandy seabed</b></sub>
</td>
<td align="center">
<img src="documentation/images/baltic_sea_tests/fish.png" alt="Fish" width="300"/><br/>
<sub><b>Fish detection</b></sub>
</td>
<td align="center">
<img src="documentation/images/baltic_sea_tests/multi_reflections_seaweed.png" alt="Multiple Reflections" width="300"/><br/>
<sub><b>Multiple reflections and seaweed</b></sub>
</td>
</tr>
</table>
## Videos
[![LINK TO LATEST VIDEO](https://img.youtube.com/vi/XF7rNGt6UYA/maxresdefault.jpg)](https://www.youtube.com/watch?v=XF7rNGt6UYA)
https://www.youtube.com/watch?v=R3_NO2F7PsI
https://www.youtube.com/watch?v=msbLVsY8xhQ
https://www.youtube.com/watch?v=eJ8jVEQSx_Y
https://www.youtube.com/watch?v=Bxh3rWd5RZk
https://www.youtube.com/watch?v=UDYWQIizN7A
## Useful Links
https://www.rapp-instruments.de/RemoteSensing/Roves/sidescan/sidescan.htm
https://www.youtube.com/watch?v=ZtUkt8Q4EJE
## Transducers:
| Transducer Name | Ranking | Description | Frequency | Range Air/Water |  Price | Link |
|-----------------|---------|-------------------------------------------------------------------------|-----------|-----------------|--------|------|
| NASA / Seafarer 150kHz Echo Sounder | ⭐ ⭐ ⭐ | NASA in-hull boat transducer for echo sounding and simple fish-finding | 150kHz | >2m/>50m | 50-100€ | https://www.nasamarine.com/product/depth-transducer-with-7-metre-cable/7
| Raymarine CPT-S | ⭐ ⭐ ⭐ | High quality In-hull transducer with two frequencies | 50 + 200kHz | 2m/>50m | 200€ | https://www.raymarine.com/de-de/unsere-produkte/fischfinder-und-sonarmodule/fischfindergeber/cpt-s-durchbruchgeber |
| Lowrance Tripleshot Sidescan | ⭐ ⭐ ⭐ | Sidescan transducer with three FANS + temperature + down-scan | 200 + 455 + 600kHz | 0m/>20m | 200€ | https://www.echolotzentrum.de/shop/lowrance-tripleshot-heckgeber/ |
| Cheap Bathymetry 200kHz | ⭐ ⭐ ⭐ | good range, good price/performance, hard to order | 200kHz | 2m/>30m | 25€ | https://www.alibaba.com/product-detail/Range-customization-lakes-river-surveys-no_1600829423846.html |
| Cheap Car Parking Sensor | ⭐ ⭐ | Great for air, also works in water (not recommended) | 40kHz | >7m/>30m | 5€ | https://de.aliexpress.com/item/1005006546490802.html |
| Water flow sensor transducer | ⭐ ⭐ | Cheap and works in Air and Water. Very narrow beam in Water (about 5°)! | 1000kHz | 0.25m/>10m | 16€ for 5pcs | https://de.aliexpress.com/item/32818381566.html |
| Only for air 200 kHz | ⭐ | has a sinter glass matching layer, should only be used in air | 200kHz | 0.8m/(>8m) | 10€ | https://de.aliexpress.com/item/1005006007865920.html |
| Different Encapsualated Transducers | ⭐| Can be used for short range testing at higher frequencies | 200,300,400kHz | ?m/?m | 9-18€ | https://de.aliexpress.com/item/4000389134890.html |
| Very small PZT only | ⭐ | Only for testing / experimentation | 200kHz | 0.2m/?m | 10€ | https://de.aliexpress.com/item/1005007032482539.html |
> [!Note]
> The range values were measured using the TUSS4470 shield and Arduino Uno with an MT3608 boost converter set to 20 V for vDRV, so consider them approximate.
> [!Tip]
> As a general rule of thumb, lower frequencies propagate farther, while higher frequencies produce narrower beams and more detailed echoes.
>
> If you need a transducer for bathymetry, choose one of the first four options. Most commercial marine-grade (boating) transducers are also suitable, as long as they operate in the 40-1000 kHz range.
# Big Thanks for Your Support!
www.kogger.tech
## to the awesome project contributors:
Check this Cheap Yellow Display implementation out: https://github.com/matztam/open_echo_cyd_display
And all the great stuff JohnCHarrington contributed: https://github.com/JohnCHarrington/open_echo
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=Neumi/open_echo&type=date&legend=top-left)](https://www.star-history.com/#Neumi/open_echo&type=date&legend=top-left)

View file

@ -0,0 +1,136 @@
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<title>OpenTDS | An Open Source Alternative to Generating Frequency Response and Time Response Graphs by Time Delay Spectrometry</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta property="og:title" content="OpenTDS" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="An Open Source Alternative to Generating Frequency Response and Time Response Graphs by Time Delay Spectrometry" />
<meta property="og:description" content="An Open Source Alternative to Generating Frequency Response and Time Response Graphs by Time Delay Spectrometry" />
<link rel="canonical" href="https://syarusinsky.github.io/OpenTDS/" />
<meta property="og:url" content="https://syarusinsky.github.io/OpenTDS/" />
<meta property="og:site_name" content="OpenTDS" />
<script type="application/ld+json">
{"@type":"WebSite","url":"https://syarusinsky.github.io/OpenTDS/","headline":"OpenTDS","name":"OpenTDS","description":"An Open Source Alternative to Generating Frequency Response and Time Response Graphs by Time Delay Spectrometry","@context":"http://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/OpenTDS/assets/css/style.css?v=3afa7e4fc89c8f25df666c58abed0e7ddcf5b243">
<!--[if lt IE 9]>
<script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
</head>
<body>
<div class="wrapper">
<header>
<h1><a href="https://syarusinsky.github.io/OpenTDS/">OpenTDS</a></h1>
<p>An Open Source Alternative to Generating Frequency Response and Time Response Graphs by Time Delay Spectrometry</p>
<p class="view"><a href="https://github.com/syarusinsky/OpenTDS">View the Project on GitHub <small>syarusinsky/OpenTDS</small></a></p>
</header>
<section>
<h1 id="opentds">OpenTDS</h1>
<p>An Open Source Alternative to Generating Frequency Response and Time Response Graphs by Time Delay Spectrometry</p>
<p>This project was designed as an Open Source alternative to Gold Line’s TEF system and other similar (expensive) software and hardware bundles. These systems use Time Delay Spectrometry to achieve frequency response measurements of loudspeakers, rooms, audio circuits, and many other minimal-phase systems without the need of an anechoic chamber. This project benefits from being capable of taking measurements using any modern consumer-grade audio interface, while TEF and other systems require you to purchase their proprietray hardware. The goal of this project is to provide hobbyist and professional acousticians and audio engineers with a means to measure frequency response and time response without the need to purchase any additional hardware or software.</p>
<h2 id="current-status">Current Status</h2>
<p>Currently, the program is capable of producing reliable frequency response graphs comparable to Goldline’s TEF system and other TDS based systems. However, the accuracy of the results are currently dependant on the user’s ability to correctly measure the microphone’s distance from the loudspeaker and the temperature of the room (ie: correctly measure the time-of-flight from loudspeaker to microphone). In future versions of the program, calculation of the time-of-flight will instead be measured by the user using the Time Response Graphing Module. This module, along with the Phase Response Module, will be among the landmark goals of the project.</p>
<h3 id="landmark-goals">Landmark Goals</h3>
<ul>
<li>Time Response Module</li>
<li>Phase Response Module</li>
<li>Improved TDS Functionality
<ul>
<li>Improved BandPass Filtering (A Custom IIR Filter Class)</li>
<li>Improved Amplitude Calculation (Algorithmically-Selected FFT-Bin-Based Magnitude Instead of Block-Based Magnitude Averages)</li>
<li>dB Referencing (Capability to Calibrate the System to Reference Pressure Levels. ie: 20uPa for dBSPL)</li>
</ul>
</li>
<li>Increased Driver Support (ASIO currently fully supported)</li>
</ul>
<h2 id="juce">JUCE</h2>
<p>This project was written using the <a href="https://juce.com/">JUCE framework</a> which provides excellent cross-platform support.</p>
<h2 id="license">License</h2>
<p>This project is licensed under the GNU General Public License v3.0 - see the <a href="LICENSE.md">LICENSE.md</a> file for details.</p>
<h2 id="authors">Authors</h2>
<p>Stevey Yarusinsky - Initial Work</p>
<h2 id="acknowledgements">Acknowledgements</h2>
<p>Richard C. Heyser - For his work developing the method of Time Delay Spectrometry. An anthology of his published works (“Time Delay Spectrometry: An Anthology of the Works of Richard C. Heyser on Measurement Analysis and Perception”) was an invaluable asset in developing this project.</p>
<h2 id="documentation">Documentation</h2>
<h3 id="tds-background-information-and-concepts">TDS Background Information and Concepts</h3>
<p>In traditional swept signal analysis, a sinusoidal waveform is sent to a device-under-test (DUT) and the resulting magnitude and phase changes on the output of the DUT are plotted on a graph for each frequency under consideration. The plotted magnitudes are the <em>Frequency Response</em> of the graph. Often included are the plotted phases (or <em>Phase Response</em>) which are often plotted on the same graph usually in a different color.</p>
<p>Frequency response is critical to achieving the required performance of many systems such as loudspeakers. However, loudspeakers cannot be accurately measured with traditional swept signal analysis unless measurement is performed in an anechoic chamber. An anechoic chamber is a room designed to be completely acoustically absorptive which eliminates the possiblity of sound being reflected on any surfaces of the room. In this chamber a loudspeaker can be measured by inputting sinusoidal waves and measuring the magnitude and phase changes with a calibrated reference microphone, though most professionals and hobbyists do not have access to an anechoic chamber. In a normal room the waves generated by the speaker would reflect off of the surfaces of the room and eventually reach the microphone, skewing the measurement results considerably. <em>Time Delay Spectrometry</em> was designed to solve this problem.</p>
<p>Time Delay Spectrometry introduces a variable-bandwidth band-pass filter capable of both tracking the frequency of a paired oscillator and delaying that oscillator to compensate for the time-of-flight. This process filters out any reflections generated by the conditions of the space where measurement is taking place. The concepts are difficult to grasp initially so a procedural description is helpful.</p>
<ul>
<li>First, a sweeping oscillator generating a sinusoidal waveform is sent to the loudspeaker.</li>
<li>The sound waves generated by the loudspeaker propagate and reflect off of surfaces in the measurement space</li>
<li>The direct sound reaches the microphone first followed later by the reflected sound.</li>
<li>These sounds are recorded by the software then processed with the band pass filter, which is sweeping its cutoff frequency with a paired oscillator.</li>
<li>The reflections take more time to reach the microphone and therefore are lower frequency than the direct sound.</li>
<li>Since the reflections are lower frequency than the direct sound they are filtered out by the band pass filter whose cutoff frequency is equivalent to the paired oscillator, leaving only the direct sound in the signal.</li>
<li>This signal is then processed to determine its amplitude and phase information.</li>
<li>That information is then plotted as a frequency response graph.</li>
</ul>
<p>It is also important to note that the paired oscillator is actually a slightly lower frequency than the oscillator being fed into the loudspeaker due to the delay placed on the paired oscillator. This is necessary to compensate for the amount of time required for the direct sound to reach the microphone. Figure 1a. and Figure 1b. should help visualize the TDS process.</p>
<p>Figure 1a. Block Diagram of TDS
<img src="TDS_Block_Diagram.png?raw=true" alt="Block Diagram of TDS" title="TDS Block Diagram" /></p>
<p>Figure 1b. TDS Geometry Example
<img src="TDS_Geometry_Example.png?raw=true" alt="TDS Geometry Example" title="TDS Geometry Example" /></p>
<h3 id="delay-in-tds">Delay in TDS</h3>
<p>Since the direct sound is the desired component, the accuracy of the frequency response measurements is determined by the delay placed on the sweeping of the tracking band pass filter. This is known as the <em>time-of-flight</em>. In an ideal analog system, this would simply be the time it takes for the sound to travel from the speaker to the microphone. However, this TDS program was designed to be used with any consumer grade audio interface and nearly all modern systems utilize digital audio concepts. Most notably, these systems use input and output <em>audio buffers</em> to lighten the CPU load. These buffers introduce a <em>latency</em>, or delay, into the signal path. While this delay is usually negligible for recording and playback purposes, it is catastrophic for the accuracy of TDS frequency response measurements. In addition, each audio interface will have its own unique latency. In order to be absolutely sure that any latency from any system is accounted for in our <em>time-of-flight</em>, OpenTDS allows the user to calibrate latency before taking measurements. The user must plug the output of their system into the input and click the calibrate latency button. After this is operation is performed the user can continue taking measurements until the program is terminated. The next time the program is opened, the user must calibrate the latency again before taking measurements.</p>
<h3 id="frequency-response-comparison">Frequency Response Comparison</h3>
<p>Speaker (QSC K8) Frequency Response at 1 ft away with TEF25 (Graph is Smoothed by the TEF Software)
<img src="TEFComparisonSpeakerTrial1_3.PNG?raw=true" alt="Speaker (QSC K8) Frequency Response at 1 ft away with TEF25" title="Speaker (QSC K8) Frequency Response at 1 ft away with TEF25" />
Speaker (QSC K8) Frequency Response at 1 ft away with OpenTDS and Akai EIE Pro audio interface
<img src="TEFComparisonSpeakerTrial1_4.png?raw=true" alt="Speaker (QSC K8) Frequency Response at 1 ft away with OpenTDS and Akai EIE Pro audio interface" title="Speaker (QSC K8) Frequency Response at 1 ft away with OpenTDS and Akai EIE Pro audio interface" /></p>
<p>Video Tutorials coming soon . . .</p>
</section>
<footer>
<p>This project is maintained by <a href="https://github.com/syarusinsky">syarusinsky</a></p>
<p><small>Hosted on GitHub Pages &mdash; Theme by <a href="https://github.com/orderedlist">orderedlist</a></small></p>
</footer>
</div>
<script src="/OpenTDS/assets/js/scale.fix.js"></script>
</body>
</html>

View file

@ -0,0 +1,83 @@
# py A P R i L
# Advanced Passive Radar Library
pyAPRiL is a python based DSP library which implements passive radar algorithms. The ultimate goal of the library is to make available the so far ellaborated passive radar algorithms to everyone including sceintific researchers, radar system designers and amateurs. All the implemented methods are tested and verified through real-life systems, field measurements and simulations.
### Project guidelines:
- This project respects the authors of all the contributions, therfore references are always highlighted where applicable.
- Understanding the operation of the implemented algorithms is always of primary importance. The efficient execution and concise coding do not belong to the principles of the project, but encoured when it is not at the expense of comprehensibility and does not change the essential operation of the originally proposed algorithm.
### Contributions:
Contributions are welcome from anyone who shares the passion of passive radars and wants to help make opensource passive radar community even better. Whether you're a seasoned developer or just getting started, there are plenty of ways to get involved and contribute to pyAPRiL. By contributing to the project, you'll be helping to make a positive impact on passive radar developers around the globe and advancing the deeper sceintific understanding of this novel technology. Whether you can contribute code, documentation, or simply provide feedback, every contribution helps us move closer to our shared goal.
### Restrictions:
This projects strictly focuses on the support of scientific researches. The source code of the library is licensed under the GNU General Public License version 3 (GPLv3), which is a copyleft license. This means that anyone who uses, modifies, or distributes the project must also license their work under the same terms and conditions. Commercial use of this software is permitted, provided that any results obtained from such use are made available to the public under the same license. By adhering to these restrictions, this project gains comliance with the REGULATION (EU) 2021/821 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL.
### The package is organized as follows:
* pyAPRiL: Main package directory
* **channelPreparation**: This file contains wrapper functions for the different algorithms that aims to prepare the reference and the sureveillance signals for the detection stage. Such as reference signal regeneration and clutter cancellation techniques.
* **clutterCancellation** : It describes a huge variety of clutter cancellation techniques implemented in the space, time and the space-time domain. In the current version, the following algorithms are available:
* Timde-domain:
* Wiener-SMI: Wiener filter with Sample Matrix Inversion
* Wiener-SMI-MRE: Wiener filter with Sample Matrix Inversion and Minimum Redundancy Estimation
* ECA: Extensive Cancellation Algorithm
* ECA-B: Extensive Cancellation Algorithm - Batched
* ECA-S: Extensive Cancellation Algorithm - Sliding
* Space-domain (Beamforming):
* Max-SIR: Maximum Signal-to-Interference Ratio
* MVDR: Minimum Variance Distortionless Response
* Principal eigenvalue beaformer
* Beamstearing
* Explicit coefficient beamformer
* **hitProcessor**: Implements a number of hit and plot processing related functions such as the CFAR and the plot extractor. Implemented algorithms:
* CA-CFAR
* Target DoA estimator (from multichannel RD matrix using the pyArgus library)
* **detector** : In this file a number of implementation of the cross-correlation detector can be found.
* Cross-correlation detector - Time domain implementation
* Cross-correlation detector - Frequency domain implementation
* Cross-correlation detector - Batched implementation (Overlap and save)
* Doppler frequency windowing
* **metricExtract**: Implements various clutter cancellation and detection performance evaluation function. The supported metrics are the followings:
* CA: Clutter Attenuation
* Rnf: Noise floor reduction
* Mu : Peak-to-channel power ratio
* Delta: Estimated target SINR
* Alpha: Target peak to noise floor ratio
* L: Target peak power loss
* Rzdc: Zero Doppler clutter reduction
* P: Peak clutter reduction
* D: Dynamic range compression
* **targetParameterCalculator**: Calculates the expected parameters of an observed target (bistatic range, Doppler, ...) from geodetic data.
* **targetLocalization**: Calculates the location of the target
* **RDTools**: range-Doppler matrix image export and plotting tools
* **sim**: IoO simulator (FM)
* **docs**: Contains Ipython notebook files with demonstrations.
* **testing**: Contains demonstration functions.
### Installing from Python Package Index:
```python
pip install pyapril
```
### Version history
* 1.0.0 Inital version
* 1.1.0 Automatic detection added (CA-CFAR)
* 1.2.0 Add temporal clutter filters (ECA, ECA-B, ECA-S)
* 1.3.0 Clutter filter and detection performance evaluation
* 1.4.0 Target DoA estimation
* 1.5.0 Surveillance channel beamforming
* 1.6.0 Target parameter calculator
* 1.7.0 FM IoO simulator
* 1.7.1 Extend RD Tools with Hit matrix plot
* 1.7.2 Adaptive RD matrix scaling to improve the visibility of the plots
* 1.7.3 Improved generality for metric extraction
* 1.7.4 Improve and fix metric extraction
* 1.7.5 Update project organization (2023 04)
* 1.7.6 Target localization
### Acknowledgements
This work was supported by the Microwave Remote Sensing Laboratory of BME ([Radarlab](http://radarlab.mht.bme.hu)). A special thanks of mine goes to the RTL-SDR site ([RTL-SDR](https://www.rtl-sdr.com/)) and to the KrakenRF (https://www.krakenrf.com/) who helped this project becoming mature with the development and testing the algorithms using Kerberos SDR (later KrakenSDR).
For further information on passive radars check: [passiveradar.eu](https://passiveradar.eu)

View file

@ -0,0 +1,101 @@
<!DOCTYPE html>
<html lang="en-gb">
<head>
<title>DSP Library &nbsp;:&nbsp; Open-source code &nbsp;:&nbsp; Signalsmith Audio</title>
<meta charset="utf-8">
<meta content="width=device-width,initial-scale=1" name="viewport">
<link href="/style/blog/dist.css" rel="stylesheet" id="dist-css">
<script src="/style/blog/dist.js" defer class="style-main" onerror="
let p='https://signalsmith-audio.co.uk/style/article/dist.',
s=document.createElement('script');
s.className='style-main';this.parentNode.removeChild(this);
s.src=p+'js';document.body.appendChild(s);
document.getElementById('dist-css').href=p+'css';"></script>
<link rel="shortcut icon" type="image/png" sizes="128x128" href="/style/images/logo-v3/square.png">
<meta name="twitter:title" content="DSP Library">
<meta property="og:title" content="DSP Library">
<meta name="author" content="Geraint Luff">
<meta name="twitter:description" content="">
<meta property="og:site_name" content="Signalsmith Audio">
<meta property="og:type" content="article">
<meta name="og:twitter:card" content="article">
<meta name="twitter:site" content="@SignalsmithDSP">
<meta name="twitter:creator" content="@SignalsmithDSP">
</head>
<body class="no-footer">
<header class="blog">
<nav>
<a href="/"><img src="/style/images/logo-v3/square.png" height="48" alt=""></a>
<a href="../../">Signalsmith Audio</a>
<span class="link"></span>
<a href="../">Open-source code</a>
<span class="link"></span>
<a href="#">DSP Library</a>
</nav>
</header>
<article class="main full">
<header>
<h1>DSP Library</h1>
<address><a href="https://signalsmith-audio.co.uk">Signalsmith Audio</a></address>
</header>
<nav class="auto"></nav>
<p>This is a set of <strong>C++11</strong> header-only classes/templates to support certain DSP tasks (mostly audio).</p>
<p>It's still growing, but currently includes:</p>
<ul>
<li>Delay tools (circular buffers, single/multi-channel delay-lines)</li>
<li>Interpolators (Lagrange, polyphase, Kaiser-sinc)</li>
<li>Envelope tools (e.g. box-filter, peak-hold)</li>
<li>FFT and spectral processing (including multi-channel STFT)</li>
</ul>
<h2>How to use</h2>
<p>Clone using Git:</p>
<figure><pre><code class="full clipboard-copy">
git clone https://signalsmith-audio.co.uk/code/dsp.git
</code></pre><figcaption>There's also a <a href="https://github.com/signalsmith-audio/dsp" target="_blank">GitHub mirror</a></figcaption></figure>
<p>Include the appropriate header file, and start using classes:</p>
<figure><pre class="language-cpp"><script type="code">
#include "dsp/delay.h";
using Delay = signalsmith::delay::Delay<float>;
Delay delayLine(1024);
</script></pre></figure>
<h3>API docs</h3>
<p>Check out the <a href="html/modules.html">API documentation (Doxygen)</a> for detailed information.</p>
<h2>Testing</h2>
<p>Tests (and scripts to plot graphs etc.) are in a separate repository, to keep the main repo neater:</p>
<figure><pre><code class="full clipboard-copy">
git clone https://signalsmith-audio.co.uk/code/dsp-doc.git
</code></pre><figcaption>There's a <a href="https://github.com/signalsmith-audio/dsp-doc" target="_blank">GitHub mirror</a> for this too</figcaption></figure>
<h3>Goals</h3>
<p>Where reasonable, the tests should measure actual output quality (not just basic correctness and smoke-tests).</p>
<p>For example, here's the aliasing/accuracy performance of a delay line with windowed-sinc interpolation:</p>
<figure><img src="html/interpolator-KaiserSincN.svg"></figure>
<p>The automated tests check this performance for various input bandwidths, using a table like this:</p>
<figure><pre><code class="language-cpp">
// A table of acceptable limits for the 20-point windowed-sinc
double bandwidth[] = {90, 80, 50, 25, 12.5};
double aliasing[] = {-20.5, -61, -70.5, -76, -77.5};
double ampLow[] = {-2, -0.02, -0.01, -0.01, -0.01};
double ampHigh[] = {0.02, 0.02, 0.01, 0.01, 0.01};
double delayError[] = {0.9, 0.03, 0.01, 0.01, 0.01};
</code></pre><figcaption>Each column specifies performance for a particular input bandwidth - e.g. 25% would correspond to 4x oversampled input.</figcaption></figure>
<h3>Running the tests</h3>
<p>To run the tests:</p>
<figure><pre><code>make test</code></figure>
<p>To run just the tests in <code>tests/delay/</code> (or similar):</p>
<figure><pre><code>make test-delay</code></pre></figure>
<p>There are a few other targets (e.g. for plotting graphs) - check the <code>Makefile</code> if you're curious.</p>
<h2>License</h2>
<p>The main library is <a href="https://choosealicense.com/licenses/mit/">MIT licensed</a> (see <code>LICENSE.txt</code> in the main repo), but the tests, docs and support scripts are licensed differently (just for developing/testing the library).</p>
<p>We're pretty flexible though - if you need something else, <a href="https://signalsmith-audio.co.uk/contact">get in touch</a>.</p>
</article>
</body>
</html>

View file

@ -0,0 +1,209 @@
# Source Ledger: Recursive Sonar DSP Refinement
This ledger preserves the source trail behind
`../sonar-dsp-recursive-refinement-2026-05-25.md`. It is not a literature review
for sport. Each source is here because it changes the next Mimir decoder cut.
Local mirrors live under `mirrors/` so the next implementation pass has stable
context even if a page moves, throttles, or gets cute.
## Source Index
| ID | Source | Local mirror | Primary lesson for Mimir |
| --- | --- | --- | --- |
| S1 | liquid-dsp README | `mirrors/liquid-dsp-readme.rst` | Use lightweight complex DSP primitives, synchronizers, filters, NCOs, and modem-style framing as the reference shape for a portable hot path. |
| S2 | liquid-dsp release notes | `mirrors/liquid-dsp-releases.html` | `qdsync`-style synchronization estimates and corrects timing, carrier frequency, and phase as one acquisition/tracking problem. |
| S3 | Underwater acoustic modem using liquid-dsp | `mirrors/oceans2022-acoustic-modem-liquid-dsp.pdf` | Acoustic packet receivers use chirp matched filters for packet starts, then PN synchronization, equalization, PLL, AGC, and changing loop bandwidths for acquisition versus tracking. |
| S4 | pyAPRiL passive radar | `mirrors/pyapril-readme.md` | Reference/surveillance processing, clutter cancellation, overlap-save correlation, CFAR, and range-Doppler thinking map cleanly onto loopback/mic acoustic paths. |
| S5 | GNU Radio correlate access code | `mirrors/gnuradio-correlate-access-code-tag-stream.html` | Synchronization should emit stream-position facts/tags, not just side-channel estimates. |
| S6 | GNU Radio stream tags | `mirrors/gnuradio-stream-tags.html` | Time decisions must travel with sample streams as isosynchronous metadata. This matches Mimir's rolling-buffer anchor model. |
| S7 | OpenTDS / time-delay spectrometry | `mirrors/opentds.html` | Swept tracking filters can isolate direct sound from later reflections and then derive amplitude/phase response. |
| S8 | Signalsmith DSP | `mirrors/signalsmith-dsp.html` | Fractional delay quality is measurable; use Lagrange/polyphase/Kaiser-sinc references instead of linear interpolation for final fitting/actuation. |
| S9 | Open Echo | `mirrors/open-echo-readme.md` | Sonar systems treat transducer drive, receive filtering, raw echo capture, and visualization as one measurement stack. |
| S10 | ahoi acoustic modem hardware overview | `mirrors/ahoi-hardware-overview.html` | Front-end analog filtering and gain are part of the receiver; usable acoustic bands must be measured per mic/path before DSP pretends all bins are equal. |
## Distilled Notes
### S1: liquid-dsp README
Upstream: <https://github.com/jgaeddert/liquid-dsp>
liquid-dsp is a C library meant for SDR-style DSP on embedded platforms. It
provides filters, filter design, oscillators/NCOs, modems, synchronizers, FFTs,
complex math, and framing pieces without forcing a large framework. It also
ships extensive tests and benchmarks.
Mimir implication:
- The hot decoder should be structured as small streaming DSP objects, not as
whole-window analysis scripts.
- Complex baseband is the natural internal representation for phase-aware
contour matching.
- The decoder should expose benchmarkable primitives: filterbank, matched
filter, synchronizer, path tracker, actuator.
### S2: liquid-dsp Release Notes
Upstream: <https://github.com/jgaeddert/liquid-dsp/releases>
The relevant signal is `qdsync`: the release notes describe a synchronizer that
detects frames and estimates/corrects carrier frequency, carrier phase, and
timing offset before returning clean data to the user. That is the missing shape
in Mimir's failed recursion pass.
Mimir implication:
- Do not split detection, phase, delay, and drift into unrelated little patches.
- The recursive state should be a synchronizer object with acquisition and
tracking modes.
- Timing offset, SRO, phase intercept, and group delay are one coupled state
surface.
### S3: Underwater Acoustic Modem With liquid-dsp
Upstream:
<https://signet.dei.unipd.it/wp-content/uploads/2022/02/Oceans2022_ac_modem-2.pdf>
The modem paper uses chirp preambles, matched filtering to find packet starts,
and later liquid-dsp flexible framing with PN synchronization, equalization,
PLL, AGC, and acquisition-mode loop bandwidths. It also explicitly notes that
chirps can support Doppler and channel impulse-response estimation.
Mimir implication:
- Canary/birdcall packets should expose a preamble-like matched-filter surface,
even if they sound musical.
- The contour word is not only an identifier; it is a channel probe.
- Recursive refinement should update channel impulse-response/state, not only
delay.
### S4: pyAPRiL Passive Radar
Upstream: <https://github.com/pyapril/pyapril>
pyAPRiL separates reference and surveillance channels, then provides clutter
cancellation, time-domain and frequency-domain cross-correlation, batched
overlap-save correlation, Doppler windowing, CFAR, beamforming, and performance
metrics.
Mimir implication:
- Scarlett loopback is the reference channel; mics are surveillance channels.
- Reflections and music leakage are clutter, not random noise.
- The path tracker should use clutter/residual modeling and CFAR-like detection
thresholds so strong reflection lobes stop impersonating direct-path timing.
### S5: GNU Radio Correlate Access Code
Upstream:
<https://wiki.gnuradio.org/index.php/Correlate_Access_Code_-_Tag_Stream>
The block scans a stream for a chosen access code and emits a tagged stream when
the access code is found. Its documentation also warns that cyclic access codes
produce false detections and bad alignment.
Mimir implication:
- Song contours need low-autocorrelation / non-cyclic identity features.
- Decoder hits should become timestamped anchors in the stream, not global
mutable estimates.
- False-lock behavior is an architecture problem, not a threshold annoyance.
### S6: GNU Radio Stream Tags
Upstream: <https://wiki.gnuradio.org/index.php/Stream_Tags>
GNU Radio stream tags are metadata attached to exact sample positions and
propagated along the same stream as the samples.
Mimir implication:
- Mimir anchors should be sample-position metadata inside the rolling buffer.
- UI/debug state reads these tags; it should not recalculate timing.
- Networked Raven/phone decoders should emit the same typed anchor facts so mesh
state preserves timing authority.
### S7: OpenTDS / Time-Delay Spectrometry
Upstream: <https://syarusinsky.github.io/OpenTDS/>
Time-delay spectrometry uses a swept source and tracking band-pass behavior to
separate direct sound from delayed reflections, then measures magnitude and
phase response.
Mimir implication:
- The room-path problem is directly analogous: direct arrival must be isolated
before response normalization is trusted.
- Frequency response normalization needs phase/group delay, not only magnitude.
- Our birdcall contours should be shaped so direct-path time/frequency evidence
survives before later reflections dominate.
### S8: Signalsmith DSP
Upstream: <https://signalsmith-audio.co.uk/code/dsp/>
Signalsmith's open DSP library includes delay tools, Lagrange/polyphase/Kaiser-
sinc interpolators, envelopes, FFTs, and multi-channel STFT. Its docs discuss
measured delay-line error versus bandwidth.
Mimir implication:
- Linear interpolation is not credible for the last microseconds.
- Decoder refinement and Faust/native actuation need measured fractional-delay
quality gates.
- Oversampled or band-limited interpolation should be part of the timing kernel,
not an afterthought.
### S9: Open Echo
Upstream: <https://github.com/Neumi/open_echo>
Open Echo is an open sonar controller stack. It drives ultrasonic transducers,
captures raw echo data, filters signals, and surfaces raw waterfalls / TCP data
for tooling.
Mimir implication:
- Treat emission, analog/acoustic path, capture, raw receipts, and decoder as
one calibration system.
- Persist raw captures and derived receipts together.
- The UI should expose raw evidence and path state, not only final delay.
### S10: ahoi Acoustic Modem Hardware Overview
Upstream: <https://www3.tuhh.de/acps/projects/ahoi/hardware_overview/>
The ahoi modem receiver chain includes hydrophone matching, pre-amplification,
software-switchable gain, and band-pass filtering adapted to the intended
frequency band.
Mimir implication:
- Per-path usable frequency bands are physical facts, not decoder preferences.
- Calibration must learn what each speaker/mic path can actually hear.
- The matched-filter bank should be weighted by measured response and confusion
surfaces.
## Implementation Synthesis
The next recursive decoder should be:
```text
loopback reference + mic window
-> complex contour matched-filter bank
-> candidate access/word anchors with stream positions
-> direct-path channel tracker
- delay
- SRO
- phase intercept
- phase/group-delay curve
- per-band complex gain
-> sparse later reflection taps
-> bounded recursive update
-> typed rolling-buffer anchors + calibration receipt
```
The important correction is ownership: recursion refines the direct-path channel
state. It does not choose whichever delayed waveform lobe has the biggest
correlation this frame.

View file

@ -100,3 +100,4 @@
{"ts": "2026-05-25T00:42:12+01:00", "type": "experiment", "status": "partial", "note": "Added path-level loopback truth to the anchor-rich calibration: matched mic anchors are compared directly against loopback anchors, then event-local waveform correlation against the captured loopback packet chooses the path delay. Release calibration clears the realtime budget at 21.1x on the stored anchor-rich capture. Best stored-result path precision improved to cardioid 7.576 us and shotgun 6.578 us. A fresh room capture at calibration/bioacoustic/scarlett-canary-packet-anchor-rich-latest-192k.json keeps shotgun near 5.916 us but worsens cardioid to 18.930 us. The razor timing-chip emitter mutation was rejected because it reduced physical reliability. Current floor is single-digit microseconds for the shotgun, not yet the one-microsecond target; next required cut is phase/group-delay correction or a direct-path anchor family that survives both mics."}
{"ts": "2026-05-25T01:05:00+01:00", "type": "experiment", "status": "rejected", "note": "Tested a recursive event-local waveform delay fitter: start from the current path hypothesis, resample candidate song windows against loopback, shrink the residual search radius, and weight events by phase-lock consistency. The naive time-domain version improved one stored cardioid path from 7.576 us to 5.141 us, but it also worsened the stored shotgun from 6.578 us to 10.236 us and locked the fresh cardioid onto a later reflection lobe at 65.666 us instead of the prior 18.930 us. Reverted the code and receipts. Lesson: recursive refinement is a valid shape only when constrained by a real phase/group-delay/path model; plain waveform resampling is not sufficient and nanosecond claims are physically dishonest in the current acoustic path."}
{"ts": "2026-05-25T01:25:00+01:00", "type": "research", "status": "ok", "note": "Looked at open realtime DSP/sonar-adjacent systems after the recursive-fit rejection. liquid-dsp, an underwater acoustic modem using liquid-dsp, pyAPRiL passive radar, GNU Radio stream/tag synchronization, OpenTDS, Signalsmith DSP, Open Echo, and ahoi all point the same way: acquisition should use matched filters/ambiguity search; refinement should track complex phase, timing, SRO, and channel response; multipath should be modeled as later residual taps; and exact stream positions should be propagated as tags/state. Added research/sonar-dsp-recursive-refinement-2026-05-25.md. The next implementation cut should be a complex contour matched-filter bank plus direct-path acoustic tracker, not another raw waveform recursion pass."}
{"ts": "2026-05-25T01:41:00+01:00", "type": "research", "status": "ok", "note": "Persisted the sonar/DSP source trail behind recursive refinement. Added research/sonar-dsp-recursive-refinement-2026-05-25/sources.md with per-source summaries and Mimir implementation implications, plus local mirrors under research/sonar-dsp-recursive-refinement-2026-05-25/mirrors/ for liquid-dsp, liquid-dsp releases, the acoustic modem PDF, pyAPRiL, GNU Radio correlate/tag docs, OpenTDS, Signalsmith DSP, Open Echo, and ahoi hardware overview."}

View file

@ -216,7 +216,8 @@ accepted_design:
`research/sonar-dsp-recursive-refinement-2026-05-25.md` points the next
decoder cut at complex contour matched filters, phase-slope/group-delay
fitting, acquisition/tracking loop bandwidths, and sparse multipath
residuals.
residuals. Its source ledger and local mirrors live under
`research/sonar-dsp-recursive-refinement-2026-05-25/`.
Acoustic robustness remains open.
The old arbitrary-chirplet matcher is a diagnostic/reference artifact only,
not the runtime hot path.