Restore research archive as reference memory
This commit is contained in:
Meta Vens
parent
6ca7915f2b
commit
21fb3f6636
95 changed files with 109121 additions and 3 deletions
22
research/README.md
Normal file
22
research/README.md
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
# Research Archive
|
||||
|
||||
This directory preserves implementation research and hard-won calibration
|
||||
lessons. It is allowed to mention external tools, papers, vendor docs, and
|
||||
failed paths because forgetting those days would be an expensive little act of
|
||||
self-sabotage.
|
||||
|
||||
Research is not runtime authority.
|
||||
|
||||
Use it to shape native implementation decisions:
|
||||
|
||||
- direct camera and audio drivers;
|
||||
- pinned/native buffers;
|
||||
- GPU feature extraction, flow, matching, and fusion;
|
||||
- Aquarium D3D12 compute/render paths;
|
||||
- Faust/native DSP;
|
||||
- OBS-facing program outputs.
|
||||
|
||||
Do not promote mirrored examples, old probes, or external library demos into the
|
||||
Mimir hot path just because they exist here. If research points at a concept,
|
||||
rebuild the live version in the Mimir/Aquarium/native stack with a clear owner
|
||||
and invariant.
|
||||
43
research/ambisonic-aquasynth/bibliography.md
Normal file
43
research/ambisonic-aquasynth/bibliography.md
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
# Bibliography: Ambisonic AquaSynth Audio Field
|
||||
|
||||
Mirrored copies live under `research/ambisonic-aquasynth/mirrors/`.
|
||||
|
||||
## Core Ambisonics
|
||||
|
||||
- [ICST Ambisonics, "Ambisonics Formats Explained: A-Format, B-Format, FuMa, AmbiX, FOA and HOA"](https://ambisonics.ch/learn/ambisonics-formats/)
|
||||
Mirror: `mirrors/icst-ambisonics-formats.html`
|
||||
|
||||
- [RingBuffer, "Understanding Ambisonics"](https://ringbuffer.org/spatial_audio/ambisonics/understanding-ambisonics/)
|
||||
Mirror: `mirrors/ringbuffer-understanding-ambisonics.html`
|
||||
|
||||
- Leo McCormack, Archontis Politis, Raimundo Gonzalez, Tapio Lokki, Ville Pulkki, ["Parametric Ambisonic Encoding of Arbitrary Microphone Arrays"](https://doi.org/10.1109/TASLP.2022.3182857), IEEE/ACM TASLP, 2022.
|
||||
Open access landing page: [Aaltodoc](https://aaltodoc.aalto.fi/items/cb74eb6f-dd96-4bde-826c-f77f837a42d8)
|
||||
Mirrored PDF: `mirrors/mccormack-2022-parametric-ambisonic-encoding.pdf`
|
||||
|
||||
## Faust / Production Tooling
|
||||
|
||||
- [Faust Libraries, `hoa.lib`](https://faustlibraries.grame.fr/libs/hoa/)
|
||||
Mirror: `mirrors/faust-hoa-lib.html`
|
||||
|
||||
- [IEM Plug-in Suite](https://plugins.iem.at/)
|
||||
Mirror: `mirrors/iem-plugin-suite.html`
|
||||
|
||||
- [IEM Plug-in Suite, "Compatibility with other Suites"](https://plugins.iem.at/docs/compatibility/)
|
||||
Mirror: `mirrors/iem-compatibility.html`
|
||||
|
||||
- [SPARTA, "Overview"](https://leomccormack.github.io/sparta-site/docs/plugins/overview/)
|
||||
Mirror: `mirrors/sparta-overview.html`
|
||||
|
||||
## Capture, Clocking, Transport
|
||||
|
||||
- [FFmpeg Devices Documentation](https://www.ffmpeg.org/ffmpeg-devices.html)
|
||||
Mirror: `mirrors/ffmpeg-devices.html`
|
||||
|
||||
- [JACK Audio Connection Kit API Overview](https://jackaudio.org/api/index.html)
|
||||
Mirror: `mirrors/jack-api-overview.html`
|
||||
|
||||
- [JACK Audio Connection Kit, "Using JACK on Windows"](https://jackaudio.org/faq/jack_on_windows.html)
|
||||
Mirror: `mirrors/jack-on-windows.html`
|
||||
|
||||
- [RME Manuals, "Using multiple HDSPe AoX cards"](https://docs.rme-audio.com/aoxd/705-1c_multiple_cards_win/)
|
||||
Mirror: `mirrors/rme-multiple-cards-sync.html`
|
||||
1130
research/ambisonic-aquasynth/mirrors/faust-hoa-lib.html
Normal file
1130
research/ambisonic-aquasynth/mirrors/faust-hoa-lib.html
Normal file
File diff suppressed because it is too large
Load diff
2434
research/ambisonic-aquasynth/mirrors/ffmpeg-devices.html
Normal file
2434
research/ambisonic-aquasynth/mirrors/ffmpeg-devices.html
Normal file
File diff suppressed because it is too large
Load diff
File diff suppressed because one or more lines are too long
188
research/ambisonic-aquasynth/mirrors/iem-compatibility.html
Normal file
188
research/ambisonic-aquasynth/mirrors/iem-compatibility.html
Normal file
|
|
@ -0,0 +1,188 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en-us">
|
||||
|
||||
<head>
|
||||
<meta name="generator" content="Hugo 0.119.0">
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title> Compatibility with other Suites </title>
|
||||
|
||||
|
||||
<meta name="description" content="What Ambisonic format is used? Which plug-ins can I combine?">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta name="author" content="Daniel Rudrich">
|
||||
|
||||
|
||||
<meta property="og:title" content="Compatibility with other Suites" />
|
||||
<meta property="og:description" content="What Ambisonic format is used? Which plug-ins can I combine?" />
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://plugins.iem.at/docs/compatibility/" /><meta property="og:image" content="https://plugins.iem.at/images/IEMPluginSuiteLogo.png"/><meta property="article:section" content="docs" />
|
||||
<meta property="article:published_time" content="2020-11-05T09:26:04+01:00" />
|
||||
<meta property="article:modified_time" content="2020-11-05T09:26:04+01:00" />
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta property="og:image" content="images/IEMPluginSuiteLogo.png">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="canonical" href="https://plugins.iem.at/docs/compatibility/">
|
||||
|
||||
<link rel="shortcut icon" type="image/png" href="images/favicon_96x96.png">
|
||||
|
||||
|
||||
<link href="https://plugins.iem.at/css/font.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.min.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.legenda.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/highlight.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/master.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.demo.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<link href="https://plugins.iem.at/css/custom.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<script src="https://plugins.iem.at/js/jquery-3.5.1.min.js" type="text/javascript">
|
||||
</script>
|
||||
|
||||
<script type="text/javascript" src="https://plugins.iem.at/js/tocbot.min.js"></script>
|
||||
</head>
|
||||
|
||||
|
||||
<body class="page-kube">
|
||||
<header> <div class="show-sm">
|
||||
<div id="nav-toggle-box">
|
||||
<div id="nav-toggle-brand">
|
||||
<a href="https://plugins.iem.at/">IEM Plug-in Suite</a>
|
||||
</div><a data-component="toggleme" data-target="#top" href="#" id="nav-toggle"><i class="kube-menu"></i></a>
|
||||
</div>
|
||||
</div>
|
||||
<div class="hide-sm" id="top">
|
||||
<div id="top-brand">
|
||||
<a href="https://plugins.iem.at/" title="home">IEM Plug-in Suite</a>
|
||||
</div>
|
||||
|
||||
<nav id="top-nav-main">
|
||||
<ul>
|
||||
|
||||
|
||||
<li><a href="https://plugins.iem.at/download/" >Download</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/docs/" >Documentation</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/faq/" >FAQ</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/contact/" >Contact</a></li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
<div id="nav-toggle" style="text-align: center;">
|
||||
<a href="https://atmoky.com"><img src="https://plugins.iem.at/images/atmokyfriends.png" width="200"></a>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
<main>
|
||||
<div id="main">
|
||||
<div id="hero">
|
||||
<h1> Compatibility with other Suites </h1>
|
||||
<p class="hero-lead">
|
||||
This guide tells you information about the used Ambisonic format and the compatibility with other plug-ins suites.
|
||||
</p>
|
||||
|
||||
</div>
|
||||
<div id="kube-component" class="content">
|
||||
|
||||
<nav id="contents">
|
||||
<ol class="js-toc">
|
||||
</ol>
|
||||
</nav>
|
||||
<script type="text/javascript">
|
||||
document.addEventListener("DOMContentLoaded",
|
||||
function(){
|
||||
tocbot.init({
|
||||
|
||||
tocSelector: '.js-toc',
|
||||
|
||||
contentSelector: '.content',
|
||||
|
||||
headingSelector: 'h3'
|
||||
})
|
||||
}
|
||||
);
|
||||
</script>
|
||||
|
||||
|
||||
|
||||
<h3 id="the-short-facts">The short facts</h3>
|
||||
<p>The IEM Plug-in Suite uses the AmbiX convention, that is <strong>ACN channel ordering</strong> and <strong>SN3D normalization</strong>. That is the current standard, which most plug-in suites use. Some older suites use the Furse-Malham (<strong>FuMa</strong>) format, in order to use them with AmbiX, you’ll need to convert their signals to AmbiX and vice versa.</p>
|
||||
<p>Here’s a short and non-complete list of plug-in suites which also use the AmbiX convetion and can be used directly with the IEM Plug-in Suite:</p>
|
||||
<ul>
|
||||
<li>Matthias Kronlachner’s ambix plugin suite</li>
|
||||
<li>SPARTA / Compass plug-in suites</li>
|
||||
<li>SSA-Plugins</li>
|
||||
<li>BlueRippleSound plug-ins</li>
|
||||
<li>…</li>
|
||||
</ul>
|
||||
<h3 id="ambix-format">AmbiX format</h3>
|
||||
<p>The AmbiX format is the current standard for Ambisonics. It handles two important things: the <strong>channel ordering</strong> and the <strong>normalization</strong>. Both are related to the spherical harmonics, the math behind Ambisonics. Let’s take a look!</p>
|
||||
<h5 id="channel-ordering-acn">Channel ordering: ACN</h5>
|
||||
<p>The channel ordering used in the Ambix Format is called <code>ACN</code> which stands for <code>Ambisonic Channel Order</code>. What a greate name! The difference to the <code>FuMa</code> odering is that the channels aren’t ordered alphabetically any more. Here the first oder components for both FuMa and ACN:
|
||||
<code>FuMa</code>: WXYZ
|
||||
<code>ACN</code>: WYZX</p>
|
||||
<p>(<code>W</code> is the omni-directional signal; <code>X</code>, <code>Y</code>, and <code>Z</code> the figure-of-eights in x, y, and z direction)</p>
|
||||
<p>So why that non-sense? Well, let’s say it’s math… However, in combination with the SN3D normalization it has a neat property, which will we come to in a minute (or second for the fast readers)!</p>
|
||||
<h5 id="sn3d-normalization">SN3D Normalization</h5>
|
||||
<p>Semi-normalization is used in the AmbiX format. This one makes sure that when you encode a source, the levels of all channels won’t exceed the one in the first (omni, W) channel. In general, this is quite handy, so that clipping doesn’t occur in your DAW.</p>
|
||||
<p><strong>Note</strong>: in a mix, the levels <strong>can</strong> exceed the one of the omni-channel!</p>
|
||||
<p>N3D normalization on the other side will boost the signals in each order, which will lead to very high levels let’s say in the 7th order components in channels 50-64. So why would N3D be any good? Well, the math only works in N3D. For a matter of fact, things like beamforming or decoding (which is the same actually), will happen in N3D. But you as a user don’t have to take care of that, the plug-ins will internally apply a conversion if necessary, no worries :-)</p>
|
||||
<h5 id="encoding-a-signal-using-the-iem-plug-in-suite">Encoding a signal using the IEM Plug-in Suite</h5>
|
||||
<p>Per default, signals will be encoded with <strong>SN3D</strong> and <strong>ACN</strong>. You <em>can</em> set the normalization to <strong>N3D</strong> in most plug-ins, but you shouldn’t, as then the signals won’t be AmbiX compatible any more. Please do that only, if you know what you are doing :-)</p>
|
||||
<p>When using spherical harmonics, the coefficient (factor/weight) for the omni-channel is usualy <code>1/sqrt(4*pi)</code> which is something like <code>0.282094792</code>. In all encoders, the encoding coefficients will be adjusted so that the omni channel will have <strong>unity gain</strong>.</p>
|
||||
<p>On the other side, when decoding, e.g. with the <code>**Probe**Decoder</code> the weights are adjusted, so that encoding into one direction and decoding with the same direction will lead to <strong>unity</strong> a<strong>gain</strong>.</p>
|
||||
<h5 id="the-neat-property">The neat property!</h5>
|
||||
<p>Using <strong>SN3D</strong> and <strong>ACN</strong>, in other words <strong>AmbiX</strong>, the first two channels of your Ambisonic signals will hold mid/side signals, which you can simply decode with a mid/side decoder, like the one which comes with Reaper. So a very very simple StereoDecoder would be exactlyt that. Neat, right?</p>
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
<footer>
|
||||
<footer id="footer">
|
||||
<nav>
|
||||
<ul>
|
||||
<li><span>Plug-in Suite by IEM - Institute of Electronic Music and Acoustics</span></li>
|
||||
<li>
|
||||
<a href="https://iem.at/">IEM.at</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="https://plugins.iem.at/docs/">Docs</a>
|
||||
</li>
|
||||
</ul>
|
||||
</nav>
|
||||
<p>Website created with <a href="https://gohugo.io">Hugo</a> using the <a href="https://github.com/jeblister/kube">kube</a> theme.</p>
|
||||
</footer>
|
||||
|
||||
</footer>
|
||||
|
||||
|
||||
<script src="https://plugins.iem.at/js/kube.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/kube.legenda.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/master.js" type="text/javascript">
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
258
research/ambisonic-aquasynth/mirrors/iem-plugin-suite.html
Normal file
258
research/ambisonic-aquasynth/mirrors/iem-plugin-suite.html
Normal file
|
|
@ -0,0 +1,258 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en-us">
|
||||
|
||||
<head>
|
||||
<meta name="generator" content="Hugo 0.119.0">
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title> IEM Plug-in Suite </title>
|
||||
|
||||
|
||||
<meta name="description" content="The IEM Plug-in Suite is a free and Open-Source audio plugin suite including Ambisonic plug-ins up to 7th order created by staff and students of the <br/> Institute of Electronic Music and Acoustics.">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta name="author" content="Daniel Rudrich">
|
||||
|
||||
|
||||
<meta property="og:title" content="IEM Plug-in Suite" />
|
||||
<meta property="og:description" content="The IEM Plug-in Suite is a free and Open-Source audio plugin suite including Ambisonic plug-ins up to 7th order created by staff and students of the <br/> Institute of Electronic Music and Acoustics." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://plugins.iem.at/" /><meta property="og:image" content="https://plugins.iem.at/images/IEMPluginSuiteLogo.png"/>
|
||||
|
||||
|
||||
|
||||
<meta property="og:image" content="images/IEMPluginSuiteLogo.png">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="canonical" href="https://plugins.iem.at/">
|
||||
|
||||
<link rel="shortcut icon" type="image/png" href="images/favicon_96x96.png">
|
||||
|
||||
|
||||
<link href="https://plugins.iem.at/css/font.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.min.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.legenda.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/highlight.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/master.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.demo.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<link href="https://plugins.iem.at/css/custom.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<script src="https://plugins.iem.at/js/jquery-3.5.1.min.js" type="text/javascript">
|
||||
</script>
|
||||
|
||||
<script type="text/javascript" src="https://plugins.iem.at/js/tocbot.min.js"></script>
|
||||
</head>
|
||||
|
||||
|
||||
<body class="page-kube">
|
||||
<header> <div class="show-sm">
|
||||
<div id="nav-toggle-box">
|
||||
<div id="nav-toggle-brand">
|
||||
<a href="https://plugins.iem.at/">IEM Plug-in Suite</a>
|
||||
</div><a data-component="toggleme" data-target="#top" href="#" id="nav-toggle"><i class="kube-menu"></i></a>
|
||||
</div>
|
||||
</div>
|
||||
<div class="hide-sm" id="top">
|
||||
<div id="top-brand">
|
||||
<a href="https://plugins.iem.at/" title="home">IEM Plug-in Suite</a>
|
||||
</div>
|
||||
|
||||
<nav id="top-nav-main">
|
||||
<ul>
|
||||
|
||||
|
||||
<li><a href="https://plugins.iem.at/download/" >Download</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/docs/" >Documentation</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/faq/" >FAQ</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/contact/" >Contact</a></li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
<div id="nav-toggle" style="text-align: center;">
|
||||
<a href="https://atmoky.com"><img src="https://plugins.iem.at/images/atmokyfriends.png" width="200"></a>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
<main>
|
||||
|
||||
<div id="main">
|
||||
<div id="hero">
|
||||
<img alt="IEM Plug-in Suite" src="images/IEMPluginSuiteLogo.png" width="450">
|
||||
<h1>IEM Plug-in Suite</h1>
|
||||
<p>The IEM Plug-in Suite is a free and Open-Source audio plugin suite including Ambisonic plug-ins up to 7th order created by staff and students of the <br/> Institute of Electronic Music and Acoustics.</p>
|
||||
</div>
|
||||
|
||||
<div id="action-buttons">
|
||||
<a class="button primary big" href="download/">Download</a>
|
||||
<a class="button outline big" href="https://git.iem.at/audioplugins/IEMPluginSuite/tags/v1.15.0">Release Page</a>
|
||||
<a class="button outline big" href="https://git.iem.at/audioplugins/IEMPluginSuite" onclick="_gaq.push(['_trackEvent', 'kube', 'github', '1.0.0']);">Source Code</a>
|
||||
<p>Latest Version v1.15.0, 10th April 2025</p>
|
||||
</div>
|
||||
|
||||
<div class="message focus open" data-component="message" data-loaded="true">
|
||||
<h5>NOTE: Breaking changes!</h5> Version v1.15.0 and v1.3.0 have introduced some breaking changes! Projects using one of the previous versions might not work as expected. For more information see the <a href="docs/breakingchanges/">Breaking changes article</a> and/or the <a href="https://git.iem.at/audioplugins/IEMPluginSuite/blob/master/ChangeLog.md">ChangeLog</a>.
|
||||
</div>
|
||||
|
||||
<div id="main">
|
||||
<h3 class="section" id="h-featured">Featured Plug-ins</h3>
|
||||
<nav data-component="tabs" data-live=".tab-live" id="livetabs"></nav>
|
||||
<div class="tab-live" data-title="DualDelay" id="tab-DualDelay">
|
||||
<div class="row gutters">
|
||||
<div class="col-6"><img src="images/featured/DualDelay.png"></div>
|
||||
<div class="col-6">
|
||||
<h5>DualDelay</h5>
|
||||
<p>The <b>Dual</b>Delay has been one of the first plug-ins in the IEM Suite. Therefore it was time for a big makeover! It has been optimized to suite needs of composers and musicians by adding a sync-to-DAW function, a tap delay button as well as moving from milliseconds to BPM with multiplies for setting times. As cherry on top, it doesn't only support a rotation of the signal, but also warping!</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="tab-live" data-title="GranularEncoder" id="tab-GranularEncoder">
|
||||
<div class="row gutters">
|
||||
<div class="col-6"><img src="images/featured/GranularEncoder.png"></div>
|
||||
<div class="col-6">
|
||||
<h5>GranularEncoder</h5>
|
||||
<p>The <b>Granular</b>Encoder is an Ambisonic granular synthesis plug-in. That's right, it is the first Ambisonic synthesizer plug-in released in the suite! The best thing about it: you can use any mono or stereo audio signal, and turn it into an enveloping sound scene, or crush the signal into microscopic sonic particles!</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="tab-live" data-title="MultiBandCompressor" id="tab-MultiBandCompressor">
|
||||
<div class="row gutters">
|
||||
<div class="col-6"><img src="images/featured/MultiBandCompressor.png"></div>
|
||||
<div class="col-6">
|
||||
<h5>MultiBandCompressor</h5>
|
||||
<p>The <b>MultiBand</b>Compressor is a very handy tool in your project's mastering stage! It works like a stereo multi-band compressor, however, it will make sure the spatial image is not altered! The Ambisonic input signal is split up into four bands, which can be compressed individually.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="tab-live" data-title="StereoEncoder" id="tab-StereoEncoder">
|
||||
<div class="row gutters">
|
||||
<div class="col-6"><img src="images/featured/StereoEncoder.png"></div>
|
||||
<div class="col-6">
|
||||
<h5>StereoEncoder</h5>
|
||||
<p>Use the <b>Stereo</b>Encoder plug-in to encode mono or stereo audio signals into the Ambisonic domain - up to 7th order. The interface lets you easily set the orientation, width and rotation of the stereo pair. With a simple double-click onto the SpherePanner you can change the display to a linear elevation, which gives you a higher accuracy towards the horizon.</p>
|
||||
<p>With help of the Quaternion input, you can use use data from orientation trackers (e.g. headtrackers) to steer your source into the right direction.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="tab-live" data-title="RoomEncoder" id="tab-RoomEncoder">
|
||||
<div class="row gutters">
|
||||
<div class="col-6"><img src="images/featured/RoomEncoder.png"></div>
|
||||
<div class="col-6">
|
||||
<h5>RoomEncoder</h5>
|
||||
<p>The <b>Room</b>Encoder is so far the most complex and computation-intensive plug-in within the IEM plug-in suite. It allows you to put a source and a listener into a virtual shoebox-shaped room and render over 200 wall reflections. Perfect to enhance the externalization of binaural playback.</p>
|
||||
<p>You can move the source and the listener freely within the room, which will automatically generate Doppler-shift effects. Use the overall reflection coefficient and the low-shelve and high-shelve filters to change the character of the reflections.</p>
|
||||
<p><b>But thats not all:</b> The RoomEncoder can make use of DirectivitySignals as an input signal. This means that your source can have a frequency-dependent and time-variable directivity. The <b>Directivity</b>Shaper plugin is a good tool to create various directivities.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="tab-live" data-title="AllRADecoder" id="tab-AllRADecoder">
|
||||
<div class="row gutters">
|
||||
<div class="col-6"><img src="images/featured/AllRADecoder.png"></div>
|
||||
<div class="col-6">
|
||||
<h5>AllRADecoder</h5>
|
||||
<p>With the AllRADecoder you can design an Ambisonic decoder for an any desired loudspeaker layout using the AllRAD approach.</p>
|
||||
<p>The resulting decoder is automatically applied to the incoming Ambisonic audio stream, so it can be directly inserted into a decoder track of your audio project. You might also want to use it as a standalone application to design a decoder and write it into a JSON configuration file for the use in other plug-ins (e.g. <b>Simple</b>Decoder).</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="tab-live" data-title="EnergyVisualizer" id="tab-EnergyVisualizer">
|
||||
<div class="row gutters">
|
||||
<div class="col-6"><img src="images/featured/EnergyVisualizer.png"></div>
|
||||
<div class="col-6">
|
||||
<h5>EnergyVisualizer</h5>
|
||||
<p>This plug-in is perfect for metering the spatial image. The <b>Energy</b>Visualizer visualizes the energy distribution on the sphere of the Ambisonic input signal. The energy levels are color-coded according to the color map. The Ambisonic Signal is decoded using 480 virtual loudspeakers with a subsequent RMS calculation and OpenGL visualization.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
<div id="kube-features">
|
||||
<div class="row gutters">
|
||||
<div class="col col-4 item">
|
||||
<figure>
|
||||
<img alt="Higher Order Ambisonics" height="60" src="https://plugins.iem.at/images/ambisonics_60px.png" width="60">
|
||||
</figure>
|
||||
<h3>One Plug-In - Seven Orders</h3>
|
||||
<p>Support for up to <b>7th</b> order Ambisonics, selectable <b>within</b> each plug-in.</p>
|
||||
|
||||
</div>
|
||||
<div class="col col-4 item">
|
||||
<figure>
|
||||
<img alt="Open-Source" height="30" src="images/openSource.png">
|
||||
</figure>
|
||||
<h3>Open-Source</h3>
|
||||
<p>Free to use, free to change. Contribute!</p>
|
||||
</div>
|
||||
<div class="col col-4 item">
|
||||
<figure>
|
||||
<img alt="Cross-platform" height="60" src="https://plugins.iem.at/images/crossPlatform.png">
|
||||
</figure>
|
||||
<h3>Cross-platform</h3>
|
||||
<p>Compiled Plug-ins available for macOS, Windows and Linux.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="row gutters">
|
||||
<div class="col col-4 item">
|
||||
<h4>Made with JUCE</h4>
|
||||
<p>Created with the <a href="https://juce.com">JUCE</a> framework.</p>
|
||||
</div>
|
||||
<div class="col col-4 item">
|
||||
<h4>Created by staff and students</h4>
|
||||
<p>The plug-ins result from the institute's current research and student projects.</p>
|
||||
</div>
|
||||
<div class="col col-4 item">
|
||||
<h4>License</h4>
|
||||
<p>The IEM Plug-in Suite is licensed under the <a href="https://www.gnu.org/licenses/">GPLv3</a> License.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
<footer> <footer id="footer">
|
||||
<nav>
|
||||
<ul>
|
||||
<li><span>Plug-in Suite by IEM - Institute of Electronic Music and Acoustics</span></li>
|
||||
<li>
|
||||
<a href="https://iem.at/">IEM.at</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="https://plugins.iem.at/docs/">Docs</a>
|
||||
</li>
|
||||
</ul>
|
||||
</nav>
|
||||
<p>Website created with <a href="https://gohugo.io">Hugo</a> using the <a href="https://github.com/jeblister/kube">kube</a> theme.</p>
|
||||
</footer>
|
||||
</footer>
|
||||
|
||||
|
||||
<script src="https://plugins.iem.at/js/kube.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/kube.legenda.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/master.js" type="text/javascript">
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
176
research/ambisonic-aquasynth/mirrors/jack-api-overview.html
Normal file
176
research/ambisonic-aquasynth/mirrors/jack-api-overview.html
Normal file
|
|
@ -0,0 +1,176 @@
|
|||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||||
<head>
|
||||
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
|
||||
<meta name="generator" content="Doxygen 1.9.1"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1"/>
|
||||
<title>JACK-AUDIO-CONNECTION-KIT: JACK Audio Connection Kit</title>
|
||||
<link href="tabs.css" rel="stylesheet" type="text/css"/>
|
||||
<script type="text/javascript" src="jquery.js"></script>
|
||||
<script type="text/javascript" src="dynsections.js"></script>
|
||||
<link href="search/search.css" rel="stylesheet" type="text/css"/>
|
||||
<script type="text/javascript" src="search/searchdata.js"></script>
|
||||
<script type="text/javascript" src="search/search.js"></script>
|
||||
<script type="text/javascript">
|
||||
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
|
||||
$(document).ready(function() { init_search(); });
|
||||
/* @license-end */
|
||||
</script>
|
||||
<link href="doxygen.css" rel="stylesheet" type="text/css" />
|
||||
</head>
|
||||
<body>
|
||||
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
|
||||
<div id="titlearea">
|
||||
<table cellspacing="0" cellpadding="0">
|
||||
<tbody>
|
||||
<tr style="height: 56px;">
|
||||
<td id="projectalign" style="padding-left: 0.5em;">
|
||||
<div id="projectname">JACK-AUDIO-CONNECTION-KIT
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<!-- end header part -->
|
||||
<!-- Generated by Doxygen 1.9.1 -->
|
||||
<script type="text/javascript">
|
||||
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
|
||||
var searchBox = new SearchBox("searchBox", "search",false,'Search','.html');
|
||||
/* @license-end */
|
||||
</script>
|
||||
<div id="navrow1" class="tabs">
|
||||
<ul class="tablist">
|
||||
<li class="current"><a href="index.html"><span>Main Page</span></a></li>
|
||||
<li><a href="pages.html"><span>Related Pages</span></a></li>
|
||||
<li><a href="modules.html"><span>Modules</span></a></li>
|
||||
<li><a href="annotated.html"><span>Data Structures</span></a></li>
|
||||
<li><a href="files.html"><span>Files</span></a></li>
|
||||
<li>
|
||||
<div id="MSearchBox" class="MSearchBoxInactive">
|
||||
<span class="left">
|
||||
<img id="MSearchSelect" src="search/mag_sel.svg"
|
||||
onmouseover="return searchBox.OnSearchSelectShow()"
|
||||
onmouseout="return searchBox.OnSearchSelectHide()"
|
||||
alt=""/>
|
||||
<input type="text" id="MSearchField" value="Search" accesskey="S"
|
||||
onfocus="searchBox.OnSearchFieldFocus(true)"
|
||||
onblur="searchBox.OnSearchFieldFocus(false)"
|
||||
onkeyup="searchBox.OnSearchFieldChange(event)"/>
|
||||
</span><span class="right">
|
||||
<a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.svg" alt=""/></a>
|
||||
</span>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div><!-- top -->
|
||||
<!-- window showing the filter options -->
|
||||
<div id="MSearchSelectWindow"
|
||||
onmouseover="return searchBox.OnSearchSelectShow()"
|
||||
onmouseout="return searchBox.OnSearchSelectHide()"
|
||||
onkeydown="return searchBox.OnSearchSelectKey(event)">
|
||||
</div>
|
||||
|
||||
<!-- iframe showing the search results (closed by default) -->
|
||||
<div id="MSearchResultsWindow">
|
||||
<iframe src="javascript:void(0)" frameborder="0"
|
||||
name="MSearchResults" id="MSearchResults">
|
||||
</iframe>
|
||||
</div>
|
||||
|
||||
<div class="PageDoc"><div class="header">
|
||||
<div class="headertitle">
|
||||
<div class="title">JACK Audio Connection Kit </div> </div>
|
||||
</div><!--header-->
|
||||
<div class="contents">
|
||||
<div class="textblock"><h1><a class="anchor" id="intro"></a>
|
||||
Introduction</h1>
|
||||
<p>JACK is a low-latency audio server, written for any operating system that is reasonably POSIX compliant. It currently exists for Linux, OS X, Solaris, FreeBSD and Windows. It can connect several client applications to an audio device, and allow them to share audio with each other. Clients can run as separate processes like normal applications, or within the JACK server as "plugins".</p>
|
||||
<p>JACK was designed from the ground up for professional audio work, and its design focuses on two key areas: synchronous execution of all clients, and low latency operation.</p>
|
||||
<dl class="section see"><dt>See also</dt><dd><a href="http://jackaudio.org">http://jackaudio.org</a></dd></dl>
|
||||
<h1><a class="anchor" id="jack_overview"></a>
|
||||
JACK Overview</h1>
|
||||
<p>Traditionally it has been hard if not impossible to write audio applications that can share data with each other. In addition, configuring and managing audio interface hardware has often been one of the most complex aspect of writing audio software.</p>
|
||||
<p>JACK changes all this by providing an API that does several things: </p><pre class="fragment">1. provides a high level abstraction for programmers that
|
||||
removes the audio interface hardware from the picture and
|
||||
allows them to concentrate on the core functionality of
|
||||
their software.
|
||||
|
||||
2. allows applications to send and receive audio data to/from
|
||||
each other as well as the audio interface. There is no
|
||||
difference in how an application sends or receives
|
||||
data regardless of whether it comes from/goes to another application
|
||||
or an audio interface.
|
||||
</pre><p> For programmers with experience of several other audio APIs such as PortAudio, Apple's CoreAudio, Steinberg's VST and ASIO as well as many others, JACK presents a familiar model: your program provides a "callback" function that will be executed at the right time. Your callback can send and receive data as well as do other signal processing tasks. You are not responsible for managing audio interfaces or threading, and there is no "format negotiation": all audio data within JACK is represented as 32 bit floating point values.</p>
|
||||
<p>For those with experiences rooted in the Unix world, JACK presents a somewhat unfamiliar API. Most Unix APIs are based on the read/write model spawned by the "everything is a file" abstraction that Unix is rightly famous for. The problem with this design is that it fails to take the realtime nature of audio interfaces into account, or more precisely, it fails to force application developers to pay sufficient attention to this aspect of their task. In addition, it becomes rather difficult to facilitate inter-application audio routing when different programs are not all running synchronously.</p>
|
||||
<p>Using JACK within your program is very simple, and typically consists of just:</p>
|
||||
<ul>
|
||||
<li>calling <a class="el" href="group__ClientFunctions.html#gabbd2041bca191943b6ef29a991a131c5">jack_client_open()</a> to connect to the JACK server.</li>
|
||||
<li>registering "ports" to enable data to be moved to and from your application.</li>
|
||||
<li>registering a "process callback" which will be called at the right time by the JACK server.</li>
|
||||
<li>telling JACK that your application is ready to start processing data.</li>
|
||||
</ul>
|
||||
<p>There is a lot more that you can do with JACK's interfaces, but for many applications, this is all that is needed. The simple_client.c example demonstrates a complete (simple!) JACK application that just copies the signal arriving at its input port to its output port. Similarly, inprocess.c shows how to write an internal client "plugin" that runs within the JACK server process.</p>
|
||||
<h1><a class="anchor" id="reference"></a>
|
||||
Reference</h1>
|
||||
<p>The JACK programming interfaces are described in several header files. We present them here broken into useful categories to make the API a little clearer:</p>
|
||||
<ul>
|
||||
<li><a class="el" href="group__ClientFunctions.html">Creating & manipulating clients</a></li>
|
||||
<li><a class="el" href="group__ClientCallbacks.html">Setting Client Callbacks</a></li>
|
||||
<li><a class="el" href="group__ClientThreads.html">Creating and managing client threads</a></li>
|
||||
<li><a class="el" href="group__ServerControl.html">Controlling & querying JACK server operation</a></li>
|
||||
<li><a class="el" href="group__PortFunctions.html">Creating & manipulating ports</a></li>
|
||||
<li><a class="el" href="group__PortSearching.html">Looking up ports</a></li>
|
||||
<li><a class="el" href="group__LatencyFunctions.html">Managing and determining latency</a></li>
|
||||
<li><a class="el" href="group__TimeFunctions.html">Handling time</a></li>
|
||||
<li><a class="el" href="group__TransportControl.html">Transport and Timebase control</a></li>
|
||||
<li><a class="el" href="group__ErrorOutput.html">Controlling error/information output</a></li>
|
||||
<li><a class="el" href="group__NonCallbackAPI.html">The non-callback API</a></li>
|
||||
<li><a class="el" href="group__MIDIAPI.html">Reading and writing MIDI data</a></li>
|
||||
<li><a class="el" href="group__SessionClientFunctions.html">Session API for clients.</a></li>
|
||||
<li><a class="el" href="group__WeakLinkage.html">managing support for newer/older versions of JACK</a></li>
|
||||
<li><a class="el" href="group__ControlAPI.html">the API for starting and controlling a JACK server</a></li>
|
||||
<li><a class="el" href="group__Metadata.html">Metadata API.</a></li>
|
||||
</ul>
|
||||
<p>The full API is described in:</p>
|
||||
<ul>
|
||||
<li><a class="el" href="jack_8h.html"><jack/jack.h></a> is the main JACK interface.</li>
|
||||
<li><a class="el" href="statistics_8h.html"><jack/statistics.h></a> provides interfaces for monitoring the performance of a running JACK server.</li>
|
||||
<li><a class="el" href="intclient_8h.html"><jack/intclient.h></a> allows loading and unloading JACK internal clients.</li>
|
||||
<li><a class="el" href="ringbuffer_8h.html"><jack/ringbuffer.h></a> defines a simple API for using lock-free ringbuffers. These are a good way to pass data between threads, when streaming realtime data to slower media, like audio file playback or recording.</li>
|
||||
<li><a class="el" href="transport_8h.html"><jack/transport.h></a> defines a simple transport control mechanism for starting, stopping and repositioning clients. This is described in the <a class="el" href="transport-design.html">JACK Transport Design</a> document.</li>
|
||||
<li><a class="el" href="types_8h.html"><jack/types.h></a> defines the main JACK data types.</li>
|
||||
<li><a class="el" href="thread_8h.html"><jack/thread.h></a> functions standardize thread creation for JACK and its clients.</li>
|
||||
<li><a class="el" href="midiport_8h.html"><jack/midiport.h></a> functions to handle reading and writing of MIDI data to a port</li>
|
||||
<li><a class="el" href="session_8h.html"><jack/session.h></a> functions that form the JACK session API</li>
|
||||
<li><a class="el" href="control_8h.html"><jack/control.h></a> the API for starting and controlling a JACK server</li>
|
||||
<li><a class="el" href="metadata_8h.html"><jack/metadata.h></a> the API for managing metadata about objects within JACK (clients and ports)</li>
|
||||
</ul>
|
||||
<p>In addition, the tools directory provides numerous examples of simple JACK clients that nevertheless use the API to do something useful. It includes</p>
|
||||
<ul>
|
||||
<li>a metronome.</li>
|
||||
<li>a recording client that can capture any number of channels from any JACK sources and store them as an audio file.</li>
|
||||
<li>command line clients to control the transport mechanism, change the buffer size and more. <br />
|
||||
</li>
|
||||
<li>commands to load and unload JACK internal clients.</li>
|
||||
<li>tools for checking the status of a running JACK system.</li>
|
||||
</ul>
|
||||
<p>and many more.</p>
|
||||
<h1><a class="anchor" id="porting"></a>
|
||||
Porting</h1>
|
||||
<p>JACK is designed to be portable to any system supporting the relevant POSIX and ANSI C standards. It currently runs under GNU/Linux, Mac OS X and Berkeley Unix on several different processor architectures. If you want to port JACK to another platform, please read the <a class="el" href="porting-guide.html">Porting JACK</a> document.</p>
|
||||
<h1><a class="anchor" id="license"></a>
|
||||
License</h1>
|
||||
<p>Copyright (C) 2001-2019 by Paul Davis, Stephane Letz, Jack O'Quinn, Torben Hohn, Filipe Coelho and others.</p>
|
||||
<p>JACK is free software; you can redistribute it and/or modify it under the terms of the GNU GPL and LGPL licenses as published by the Free Software Foundation, <a href="http://www.gnu.org">http://www.gnu.org</a>. The JACK server uses the GPL, as noted in the source file headers. However, the JACK library is licensed under the LGPL, allowing proprietary programs to link with it and use JACK services. You should have received a copy of these Licenses along with the program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.</p>
|
||||
<p>This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. </p>
|
||||
</div></div><!-- PageDoc -->
|
||||
</div><!-- contents -->
|
||||
<!-- start footer part -->
|
||||
<hr class="footer"/><address class="footer"><small>
|
||||
Generated by <a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.9.1
|
||||
</small></address>
|
||||
</body>
|
||||
</html>
|
||||
183
research/ambisonic-aquasynth/mirrors/jack-on-windows.html
Normal file
183
research/ambisonic-aquasynth/mirrors/jack-on-windows.html
Normal file
|
|
@ -0,0 +1,183 @@
|
|||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<meta charset='utf-8' />
|
||||
<meta name="description" content="JACK Audio Connection Kit | Using JACK on Windows" />
|
||||
<link rel="stylesheet" type="text/css" media="screen" href="/assets/css/style.css">
|
||||
<link rel="alternate" type="application/atom+xml" href="/atom.xml" title="JACK Audio News">
|
||||
<title>Using JACK on Windows | JACK Audio Connection Kit</title>
|
||||
<!-- Favicon Generator Icons -->
|
||||
<link rel="apple-touch-icon" sizes="180x180" href="/assets/icons/apple-touch-icon.png">
|
||||
<link rel="icon" type="image/png" sizes="32x32" href="/assets/icons/favicon-32x32.png">
|
||||
<link rel="icon" type="image/png" sizes="16x16" href="/assets/icons/favicon-16x16.png">
|
||||
<link rel="manifest" href="/assets/icons/site.webmanifest">
|
||||
<link rel="shortcut icon" href="/assets/icons/favicon.ico">
|
||||
<meta name="msapplication-TileColor" content="#212121">
|
||||
<meta name="msapplication-config" content="/assets/icons/browserconfig.xml">
|
||||
<meta name="theme-color" content="#212121">
|
||||
</head>
|
||||
<body>
|
||||
<!-- HEADER -->
|
||||
<div id="header_wrap" class="outer">
|
||||
<header class="inner">
|
||||
<a id="forkme_banner" href="https://github.com/jackaudio">View on GitHub</a>
|
||||
<a href="https://jackaudio.org">
|
||||
<img src="/assets/images/logo.png"
|
||||
alt="JACK Logo - Home" border="0">
|
||||
</a>
|
||||
<nav>
|
||||
<a href="/"> Home </a> |
|
||||
<a href="/news/"> News </a> |
|
||||
<a href="/applications/"> Applications </a> |
|
||||
<a href="/faq/"> FAQ </a> |
|
||||
<a href="https://github.com/jackaudio/jackaudio.github.com/wiki"> WIKI </a> |
|
||||
<a href="/api/"> API </a> |
|
||||
<a href="/downloads/"> Downloads </a> |
|
||||
<a href="/community.html"> Community Network </a>
|
||||
</nav>
|
||||
</header>
|
||||
</div>
|
||||
<!-- MAIN CONTENT -->
|
||||
<div id="main_content_wrap" class="outer">
|
||||
<section id="main_content" class="inner">
|
||||
<h1 id="using-jack-on-windows">Using JACK on Windows</h1>
|
||||
|
||||
<h2 id="introduction">Introduction</h2>
|
||||
|
||||
<p>As on other platforms, JACK on Windows is a multi-purpose audio server that
|
||||
connects audio and MIDI streams using the Windows system drivers.
|
||||
It connects (OUTPUT) to those audio drivers using the PortAudio project
|
||||
(so JACK can connect using dsound, wmme, ASIO, etc) depending on what
|
||||
the soundcard’s drivers support.</p>
|
||||
|
||||
<p>JACK provides an Audio, MIDI and transport API (Application Programming Interface)
|
||||
that can be used natively by audio application developers.
|
||||
There are already some existing <em>native</em> JACK Windows applications
|
||||
(that is, audio applications that use the JACK API directly).
|
||||
Through the API, JACK on Windows offers native JACK connections (INPUT),
|
||||
and the JACK-Router ASIO driver (INPUT) that offers an ASIO linker to the JACK Server.</p>
|
||||
|
||||
<p>So any ASIO application will be able to use the JACK-Router ASIO driver to
|
||||
connect to JACK, and JACK can then be used to route that audio to other
|
||||
applications connected to the JACK Server, or out to the system sound device.</p>
|
||||
|
||||
<p>Several applications have native JACK support:</p>
|
||||
|
||||
<ul>
|
||||
<li><a href="https://ardour.org/">Ardour</a></li>
|
||||
<li><a href="https://www.bitwig.com/">Bitwig Studio</a></li>
|
||||
<li><a href="https://kx.studio/Applications:Carla">Carla Plugin Host</a></li>
|
||||
<li><a href="https://github.com/sampov2/foo-yc20">Foo YC20</a></li>
|
||||
<li><a href="https://harrisonconsoles.com/product/mixbus/">Harisson MixBus</a></li>
|
||||
<li><a href="https://jamulus.io">Jamulus</a> (starting from Version 3.8.1 as separate build)</li>
|
||||
<li><a href="https://lmms.io/">LMMS</a> (starting from v1.3)</li>
|
||||
<li><a href="http://musescore.org/">MuseScore</a></li>
|
||||
<li><a href="http://users.notam02.no/~kjetism/radium/">Radium</a></li>
|
||||
<li><a href="https://www.zrythm.org/en/">Zrythm</a></li>
|
||||
</ul>
|
||||
|
||||
<p>Native JACK applications can make use of the full power of the JACK audio and
|
||||
MIDI connection system (for instance, they can open or close JACK connections
|
||||
from within the application, rather than having to set an ASIO driver to use
|
||||
use JACK-Router).
|
||||
Also, the use of natively provided JACK ports provides the ability to sync
|
||||
audio streams between applications.</p>
|
||||
|
||||
<p>The JACK-Router ASIO driver is “only” there so that users are able to use ASIO
|
||||
applications in the Windows context of JACK, but this connection obviously has
|
||||
some limitations that native applications won’t have.</p>
|
||||
|
||||
<p>JACK uses a ‘Server <-> Client’ architecture to supply the audio connection
|
||||
between your sound device and the ASIO application(s).
|
||||
When the JACK Server is running, the JACK-Router driver is then able to create
|
||||
connections between clients (ASIO applications) and the JACK Server.
|
||||
These connections can be seen in the QjackCtl -> Graph screen.</p>
|
||||
|
||||
<p>If the JACK Server is not running, then JACK-Router will not be available to
|
||||
your ASIO application(s), and an error report may be shown.</p>
|
||||
|
||||
<p>By default, JACK-Router is set to provide 4 Input and 4 Output connections.
|
||||
This setting can be customized to suite your needs by editing the
|
||||
JackRouter.ini file within your
|
||||
<code class="language-plaintext highlighter-rouge">C:\Program Files\JACK\jack-router\win32</code> (or <code class="language-plaintext highlighter-rouge">C:\Program Files\JACK\jack-router\win64</code>
|
||||
folder depending of the JACK 32 or 64-bit version).
|
||||
In Windows Explorer, right click on the file and choose the ‘Edit’ option.
|
||||
Change the settings in the [IO] portion of the file.</p>
|
||||
|
||||
<h2 id="installation-and-configuration">Installation and Configuration</h2>
|
||||
|
||||
<ol>
|
||||
<li>
|
||||
<p>Download the JACK2 installer for Windows from the <a href="/downloads/">download page.</a></p>
|
||||
</li>
|
||||
<li>
|
||||
<p>Use the installer. Either the 32-bit or the 64-bit one (which still allows 32-bit clients).</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>Pick “Full installation (with JACK-Router)” during installation.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>From the Windows Start menu, locate the QjackCtl application and run it.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>On the QjackCtl application, click “Setup” and adjust “Interface”, “Sample Rate” and “Frames/Period” as needed for your device/sound-card.
|
||||
Do not change settings you do not understand.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>Click on OK to close the Setup dialog.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>Click on “Start” to start the JACK Server with the parameters you just set up.</p>
|
||||
</li>
|
||||
</ol>
|
||||
|
||||
<p>When the JACK Server is running the JACK-Router ASIO interface will be
|
||||
available to all ASIO capable applications.</p>
|
||||
|
||||
<h2 id="uninstalling-jack">Uninstalling JACK</h2>
|
||||
|
||||
<p>Just open the “Programs and Features” in the Windows Control Panel and
|
||||
remove “JACK2 version …” from there.
|
||||
This will uninstall the JACK Server, JACK-Router and QjackCtl.</p>
|
||||
|
||||
<h2 id="connecting-audio-applications">Connecting Audio Applications</h2>
|
||||
|
||||
<ol>
|
||||
<li>
|
||||
<p>Start the JACK Server using QjackCtl</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>With the Jack Audio Server started, the JackRouter ASIO driver
|
||||
will be available for your ASIO supported applications.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>Within your audio application, select JACK-Router as the audio device.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>Within QjackCtl, click the “Graph” button.</p>
|
||||
</li>
|
||||
</ol>
|
||||
|
||||
<p>You can route the output of ASIO applications into the inputs of other ASIO
|
||||
applications (like a VST host application), and then connect the audio output
|
||||
of that application to your system output (or a combination of both).</p>
|
||||
|
||||
<p>Multiple channel audio is also supported, but may require more configuration
|
||||
than the simple setup described here.</p>
|
||||
|
||||
</section>
|
||||
</div>
|
||||
<!-- FOOTER -->
|
||||
<div id="footer_wrap" class="outer">
|
||||
<footer class="inner">
|
||||
<p class="copyright" style="text-align:left">Slate theme maintained by
|
||||
<a href="https://github.com/pages-themes">pages-themes</a>
|
||||
<span style="float:right">This site is open source.
|
||||
<a href="https://github.com/jackaudio/jackaudio.github.com/edit/master/faq/jack_on_windows.md">
|
||||
Improve this page</a>
|
||||
</span>
|
||||
</p>
|
||||
</footer>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
Binary file not shown.
|
|
@ -0,0 +1,478 @@
|
|||
<!DOCTYPE html>
|
||||
<html prefix="og: http://ogp.me/ns# article: http://ogp.me/ns/article# " lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Understanding Ambisonics | RingBuffer</title>
|
||||
<link href="../../../assets/css/all-nocdn.css" rel="stylesheet" type="text/css">
|
||||
<link href="../../../assets/css/ipython.min.css" rel="stylesheet" type="text/css">
|
||||
<link href="../../../assets/css/nikola_ipython.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://fonts.googleapis.com/css?family=Playfair+Display:700,900" rel="stylesheet">
|
||||
<meta name="theme-color" content="#5670d4">
|
||||
<meta name="generator" content="Nikola (getnikola.com)">
|
||||
<link rel="alternate" type="application/rss+xml" title="RSS" href="../../../rss.xml">
|
||||
<link rel="canonical" href="http://ringbuffer.org/spatial_audio/ambisonics/understanding-ambisonics/">
|
||||
<script type="text/x-mathjax-config">
|
||||
MathJax.Hub.Config({
|
||||
tex2jax: {
|
||||
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
|
||||
displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
|
||||
processEscapes: true
|
||||
},
|
||||
displayAlign: 'center', // Change this to 'left' if you want left-aligned equations.
|
||||
"HTML-CSS": {
|
||||
styles: {'.MathJax_Display': {"margin": 0}}
|
||||
}
|
||||
});
|
||||
</script><!--[if lt IE 9]><script src="../../../assets/js/html5.js"></script><![endif]--><link rel="stylesheet" href="../../../assets/hvc/pkgindex.css">
|
||||
<meta name="author" content="Henrik von Coler">
|
||||
<link rel="prev" href="../../history/williams-mix/" title="John Cage's Williams Mix" type="text/html">
|
||||
<link rel="next" href="../../../computer_music_basics/physical_computing/puredata_ldr/" title="Pure Data: Light Dependent Resistor" type="text/html">
|
||||
<meta property="og:site_name" content="RingBuffer">
|
||||
<meta property="og:title" content="Understanding Ambisonics">
|
||||
<meta property="og:url" content="http://ringbuffer.org/spatial_audio/ambisonics/understanding-ambisonics/">
|
||||
<meta property="og:description" content="Mid–Side Stereo: 1D Ambisonics
|
||||
Mid–Side (MS) stereo can be understood as a one-dimensional decomposition
|
||||
of a sound field along a single horizontal axis and works with the same principle as Ambisonics">
|
||||
<meta property="og:type" content="article">
|
||||
<!-- <meta property="article:published_time" content="2022-04-28T14:00:00Z"> -->
|
||||
</head>
|
||||
<body>
|
||||
<a href="#content" class="sr-only sr-only-focusable">Skip to main content</a>
|
||||
|
||||
<!-- Header and menu bar -->
|
||||
<div class="container">
|
||||
<header class="blog-header py-3"><div class="row nbb-header align-items-center">
|
||||
<div class="col-md-3 col-xs-2 col-sm-2" style="width: auto;">
|
||||
<button class="navbar-toggler navbar-light bg-light nbb-navbar-toggler" type="button" data-toggle="collapse" data-target=".bs-nav-collapsible" aria-controls="bs-navbar" aria-expanded="false" aria-label="Toggle navigation">
|
||||
<span class="navbar-toggler-icon"></span>
|
||||
</button>
|
||||
<div class="collapse bs-nav-collapsible bootblog4-search-form-holder">
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-6 col-xs-10 col-sm-10 bootblog4-brand" style="width: auto;">
|
||||
<a class="navbar-brand blog-header-logo text-dark" href="http://ringbuffer.org/">
|
||||
|
||||
<!-- <span id="blog-title">RingBuffer</span> -->
|
||||
<img src="../../../images/ringbuffer_BANNER.png" alt="LOGO" class="center"></a>
|
||||
</div>
|
||||
<div class="col-md-3 justify-content-end align-items-center bs-nav-collapsible collapse flex-collapse bootblog4-right-nav">
|
||||
<nav class="navbar navbar-light bg-white"><ul class="navbar-nav bootblog4-right-nav"></ul></nav>
|
||||
</div>
|
||||
</div>
|
||||
</header><nav class="navbar navbar-expand-md navbar-light bg-white static-top"><div class="collapse navbar-collapse bs-nav-collapsible" id="bs-navbar">
|
||||
<ul class="navbar-nav nav-fill d-flex w-100">
|
||||
<li class="nav-item">
|
||||
<a href="../../../computer-music-basics" class="nav-link">Computer Music Basics</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a href="../../../dsp" class="nav-link">DSP</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a href="../../../sound-synthesis-introduction" class="nav-link">Sound Synthesis Introduction</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a href="../../../spatial-audio" class="nav-link">Spatial Audio</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a href="../../../music_interaction_systems" class="nav-link">Music Interaction Systems</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a href="../../../building-instruments-in-faust" class="nav-link">Faust</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a href="../../../sound-synthesis-cpp" class="nav-link">C++</a>
|
||||
|
||||
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<!-- /.navbar-collapse -->
|
||||
</nav>
|
||||
</div>
|
||||
|
||||
<div class="container" id="content" role="main">
|
||||
<div class="body-content">
|
||||
<!--Body content-->
|
||||
|
||||
|
||||
<article class="post-text h-entry hentry postpage" itemscope="itemscope" itemtype="http://schema.org/Article"><header><h1 class="p-name entry-title" itemprop="headline name"><a href="." class="u-url">Understanding Ambisonics</a></h1>
|
||||
|
||||
<div class="metadata">
|
||||
<p class="byline author vcard p-author h-card">
|
||||
<span class="byline-name fn p-name" itemprop="author">
|
||||
<a class="u-url" href="../../../authors/henrik-von-coler/">Henrik von Coler</a>
|
||||
</span>
|
||||
</p>
|
||||
|
||||
|
||||
<p class="dateline">
|
||||
<a href="." rel="bookmark">
|
||||
<time class="published dt-published" datetime="2022-04-28" itemprop="datePublished" title="2022-04-28">
|
||||
2022-04-28
|
||||
</time></a>
|
||||
</p>
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
</header><div class="e-content entry-content" itemprop="articleBody text">
|
||||
<section id="midside-stereo-1d-ambisonics"><h2>Mid–Side Stereo: 1D Ambisonics</h2>
|
||||
<p>Mid–Side (MS) stereo can be understood as a one-dimensional decomposition
|
||||
of a sound field along a single horizontal axis and works with the same principle as Ambisonics.
|
||||
MS represents the sound field using:</p>
|
||||
<ul class="simple">
|
||||
<li><p><strong>M (Mid)</strong> — an <strong>cardioid (or omnidirectional) component</strong></p></li>
|
||||
<li><p><strong>S (Side)</strong> — a <strong>bidirectional (figure-of-eight) component</strong> along the left–right axis</p></li>
|
||||
</ul>
|
||||
<figure class="align-center" style="width: 100%"><img alt="/images/spatial/mid_side.png" src="../../../images/spatial/mid_side.png" style="width: 40%;"><figcaption><p><em>MS configuration with omni (blue) and figure of 8 (greed/red).</em></p>
|
||||
</figcaption></figure><p>This structure already mirrors the first-order Ambisonic idea of
|
||||
decomposing the sound field into <strong>orthogonal basis functions</strong>.</p>
|
||||
<hr class="docutils">
|
||||
<section id="ms-encoding"><h3>MS Encoding</h3>
|
||||
<p>Given conventional stereo signals <span class="math">\(L\)</span> and <span class="math">\(R\)</span>:</p>
|
||||
<div class="math">
|
||||
\begin{equation*}
|
||||
M = \frac{L + R}{2}
|
||||
\end{equation*}
|
||||
</div>
|
||||
<div class="math">
|
||||
\begin{equation*}
|
||||
S = \frac{L - R}{2}
|
||||
\end{equation*}
|
||||
</div>
|
||||
<p>The inverse decoding is:</p>
|
||||
<div class="math">
|
||||
\begin{equation*}
|
||||
L = M + S
|
||||
\end{equation*}
|
||||
</div>
|
||||
<div class="math">
|
||||
\begin{equation*}
|
||||
R = M - S
|
||||
\end{equation*}
|
||||
</div>
|
||||
</section><hr class="docutils">
|
||||
<section id="interpretation"><h3>Interpretation</h3>
|
||||
<ul class="simple">
|
||||
<li><p><span class="math">\(M\)</span> corresponds to an <strong>omnidirectional pressure term</strong></p></li>
|
||||
<li><p><span class="math">\(S\)</span> corresponds to a <strong>dipole aligned left–right</strong></p></li>
|
||||
</ul>
|
||||
<p><strong>The phases of the dipole encode the spatial information.</strong></p>
|
||||
<p>Mathematically, this is equivalent to a <strong>1D first-order harmonic expansion</strong>:</p>
|
||||
<ul class="simple">
|
||||
<li><p><span class="math">\(M \leftrightarrow Y_0^0\)</span> (monopole)</p></li>
|
||||
<li><p><span class="math">\(S \leftrightarrow Y_1^{\pm1}\)</span> (horizontal dipole)</p></li>
|
||||
</ul>
|
||||
<p>In first-order Ambisonics (ACN/SN3D), the related components are:</p>
|
||||
<ul class="simple">
|
||||
<li><p><span class="math">\(W\)</span> — omnidirectional</p></li>
|
||||
<li><p><span class="math">\(Y\)</span> — left/right dipole</p></li>
|
||||
</ul></section><hr class="docutils">
|
||||
<section id="matrix-form"><h3>Matrix Form</h3>
|
||||
<p>MS encoding can also be written as a matrix transform:</p>
|
||||
<div class="math">
|
||||
\begin{equation*}
|
||||
\begin{bmatrix}
|
||||
M \\
|
||||
S
|
||||
\end{bmatrix}
|
||||
=
|
||||
\frac{1}{2}
|
||||
\begin{bmatrix}
|
||||
1 & 1 \\
|
||||
1 & -1
|
||||
\end{bmatrix}
|
||||
\begin{bmatrix}
|
||||
L \\
|
||||
R
|
||||
\end{bmatrix}
|
||||
\end{equation*}
|
||||
</div>
|
||||
<p>and decoding:</p>
|
||||
<div class="math">
|
||||
\begin{equation*}
|
||||
\begin{bmatrix}
|
||||
L \\
|
||||
R
|
||||
\end{bmatrix}
|
||||
=
|
||||
\begin{bmatrix}
|
||||
1 & 1 \\
|
||||
1 & -1
|
||||
\end{bmatrix}
|
||||
\begin{bmatrix}
|
||||
M \\
|
||||
S
|
||||
\end{bmatrix}
|
||||
\end{equation*}
|
||||
</div>
|
||||
</section></section><hr class="docutils">
|
||||
<section id="a-vs-b-format"><h2>A vs B Format</h2>
|
||||
<p><strong>A-format</strong> refers to the raw microphone capsule signals (e.g., tetrahedral mic outputs) when capturing 3D sound fields.
|
||||
This format is mic-specific and not interchangeable. To further process it, it usually needs to be converted to B-format.
|
||||
A→B conversion matrices are microphone-model-specific, considering capsule geometry and calibration.
|
||||
Microphone vendors need to supply the decoder.</p>
|
||||
<figure class="align-center" style="width: 100%"><img alt="/images/spatial/ambisonics/ambisonics_capsule.png" src="../../../images/spatial/ambisonics/ambisonics_capsule.png" style="width: 30%;"></figure><p><strong>B-format</strong> is made up of the spherical-harmonic components of the sound field.
|
||||
This is the standard Ambisonics format with defined channel order and normalization.
|
||||
B-format is portable and can be reproduced on any rendering system (loudspeaker setups, binaural), following standardized decoding algorithms.</p>
|
||||
</section><hr class="docutils">
|
||||
<section id="spherical-harmonics"><h2>Spherical Harmonics</h2>
|
||||
<p>Basic Ambisonics does not define a sound filed through positions, but through angles of incedence.
|
||||
Ambisonics is based on a decomposition of a sound field into <em>spherical harmonics</em> and dates back to Gerzon's theory of Peryphony (Gerzon, 1973).
|
||||
These spherical harmonics encode a sound field into to different axes,
|
||||
The number of Ambisonics channels $N$ is equal to the number of spherical harmonics.
|
||||
It can be calculated for a given order $M$ with the following formula:</p>
|
||||
<div class="math">
|
||||
\begin{equation*}
|
||||
N = (M+1)^2
|
||||
\end{equation*}
|
||||
</div>
|
||||
<p>Figure 1 shows the first 16 spherical harmonics. The first row ($N=1$) is the omnidirectional sound pressure
|
||||
for the order $M=0$.</p>
|
||||
<ul class="simple">
|
||||
<li><p>Rows 1-2 together represent the $N=4$ spherical harmonics of the first order Ambisonics signal.</p></li>
|
||||
<li><p>Rows 1-3 correspond to $M=2$, respectively $N=9$.</p></li>
|
||||
<li><p>Rows 1-4 to the third order Ambisonics signal with $N=16$ spherical harmonics.</p></li>
|
||||
</ul>
|
||||
<p>First order ambisonics is sufficient to encode a threedimensional sound field.
|
||||
The higher the Ambisonics order, the more precise the directional encoding and the better the localization of virtual sound sources.</p>
|
||||
<figure class="align-center" style="width: 100%"><img alt="/images/spatial/ambisonics/third-order-ambisonics.png" src="../../../images/spatial/ambisonics/third-order-ambisonics.png" style="width: 600px;"><figcaption><p>Fig. 1: Spherical harmonics up to order 3 <a class="brackets" href="#footnote-1" id="footnote-reference-1" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>.</p>
|
||||
</figcaption></figure><aside class="footnote-list brackets"><aside class="footnote brackets" id="footnote-1" role="doc-footnote"><span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#footnote-reference-1">1</a><span class="fn-bracket">]</span></span>
|
||||
<p><a class="reference external" href="https://commons.wikimedia.org/wiki/Category:Spherical_harmonics#/media/File:Spherical_Harmonics_deg3.png">https://commons.wikimedia.org/wiki/Category:Spherical_harmonics#/media/File:Spherical_Harmonics_deg3.png</a></p>
|
||||
</aside></aside></section><hr class="docutils">
|
||||
<section id="common-b-format-conventions"><h2>Common B-format Conventions</h2>
|
||||
<p>B-format conventions define the relation between Ambisonic channel order and spherical harmonics.
|
||||
There are different conventions for the sequence of the individual signals,
|
||||
as well as for the normalization.</p>
|
||||
<table>
|
||||
<caption>Ambisonics conventions (Common B-format variants)</caption>
|
||||
<thead><tr>
|
||||
<th>Convention</th>
|
||||
<th>Type</th>
|
||||
<th>Channel order (1st order)</th>
|
||||
<th>Normalization</th>
|
||||
<th>Notes / Where used</th>
|
||||
</tr></thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>
|
||||
<strong>FuMa</strong> (Furse–Malham)</td>
|
||||
<td>B-format (FOA)</td>
|
||||
<td><code>W, X, Y, Z</code></td>
|
||||
<td>FuMa (“maxN” style; <code>W</code> is scaled by <code>1/√2</code>)</td>
|
||||
<td>Legacy 1st-order B-format used in older DAWs and toolchains; awkward for higher orders (≥2).</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>AmbiX</strong></td>
|
||||
<td>B-format (FOA/HOA)</td>
|
||||
<td>
|
||||
<code>ACN</code> order → <code>[0:W, 1:Y, 2:Z, 3:X]</code>
|
||||
</td>
|
||||
<td><strong>SN3D</strong></td>
|
||||
<td>De-facto modern production standard (Reaper+AmbiX, many VR/AR SDKs, YouTube VR). Portable and HOA-friendly.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>ACN/N3D</strong></td>
|
||||
<td>B-format (FOA/HOA)</td>
|
||||
<td>
|
||||
<code>ACN</code> order → <code>[0:W, 1:Y, 2:Z, 3:X]</code>
|
||||
</td>
|
||||
<td>
|
||||
<strong>N3D</strong> (orthonormal)</td>
|
||||
<td>Common in research and HOA libraries; convenient for math/analysis and per-order processing.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<!-- Handy mappings (not part of the table, keep if useful):
|
||||
- FuMa ↔ AmbiX (FOA):
|
||||
• Gain: W_AmbiX = √2 · W_FuMa; X,Y,Z identical.
|
||||
• Order: FuMa [W,X,Y,Z] vs. AmbiX [W,Y,Z,X].
|
||||
- SN3D ↔ N3D (any order ℓ): Yℓm|N3D = √(2ℓ+1) · Yℓm|SN3D.
|
||||
--></section><hr class="docutils">
|
||||
<section id="ambisonic-formats"><h2>Ambisonic Formats</h2>
|
||||
<p>An Ambisonics B Format file or signal carries all $N$ spherical harmonics.
|
||||
Figure 2 shows a first order B Format signal.</p>
|
||||
<figure class="align-center" style="width: 100%"><img alt="/images/spatial/ambisonics/first-order-signal.png" src="../../../images/spatial/ambisonics/first-order-signal.png" style="width: 100%;"><figcaption><p>Fig. 2: Four channels of a first order Ambisonics signal.</p>
|
||||
</figcaption></figure><figure class="align-center" style="width: 100%"><img alt="/images/spatial/ambisonics/first-order-harmonics.png" src="../../../images/spatial/ambisonics/first-order-harmonics.png" style="width: 50%;"><figcaption><p>Fig. 2: Spherical harmonics of a first order Ambisonics signal.</p>
|
||||
</figcaption></figure></section><hr class="docutils">
|
||||
<section id="acn-normalizations-and-1st-order-mappings"><h2>ACN, Normalizations, and 1st-Order Mappings</h2>
|
||||
<p><strong>ACN (Ambisonic Channel Numbering)</strong></p>
|
||||
<p>The channel index <span class="math">\(n\)</span> is</p>
|
||||
<div class="math">
|
||||
\begin{equation*}
|
||||
n = \ell(\ell+1) + m, \qquad \ell = 0..L,\ \ m = -\ell..\ell.
|
||||
\end{equation*}
|
||||
</div>
|
||||
<p>For <strong>1st order</strong> (<span class="math">\(\ell = 0,1\)</span>), the ACN indices map to channels as</p>
|
||||
<div class="math">
|
||||
\begin{equation*}
|
||||
[\,0:\ Y_0^0 = W,\quad 1:\ Y_1^{-1} = Y,\quad 2:\ Y_1^{0} = Z,\quad 3:\ Y_1^{1} = X\,].
|
||||
\end{equation*}
|
||||
</div>
|
||||
<section id="normalizations"><h3>Normalizations</h3>
|
||||
<ul>
|
||||
<li><p><strong>SN3D</strong> (“semi-normalized”): <span class="math">\(Y_0^0 = 1\)</span>. Widely used in production (AmbiX).</p></li>
|
||||
<li>
|
||||
<p><strong>N3D</strong> (fully normalized): spherical harmonics are orthonormal over the unit sphere, i.e.,</p>
|
||||
<div class="math">
|
||||
\begin{equation*}
|
||||
\int_{S^2} Y_n^m(\Omega)\, Y_{n'}^{m'}(\Omega)\, d\Omega
|
||||
= \delta_{nn'}\,\delta_{mm'}
|
||||
\end{equation*}
|
||||
</div>
|
||||
<p>This normalization yields consistent energy per order and simplifies theoretical work, analysis, and algorithm design in higher-order Ambisonics; however, the order-dependent scaling can increase peak amplitudes in higher orders, requiring additional headroom or gain compensation to avoid clipping in practical implementations.</p>
|
||||
</li>
|
||||
<li><p><strong>FuMa</strong> (legacy): distinct scaling; notably <span class="math">\(W_{\text{FuMa}} = \tfrac{1}{\sqrt{2}}\,W_{\text{SN3D}}\)</span>.</p></li>
|
||||
</ul></section><section id="st-order-mappings-fuma-ambix-acnsn3d"><h3>1st-Order Mappings (FuMa ↔ AmbiX/ACN–SN3D)</h3>
|
||||
<p><strong>Channel order</strong></p>
|
||||
<ul class="simple">
|
||||
<li><p>FuMa order: <code>[W, X, Y, Z]</code></p></li>
|
||||
<li><p>AmbiX (ACN/SN3D) order: <code>[W, Y, Z, X]</code> (i.e., ACN indices <code>[0,1,2,3] -> [W,Y,Z,X]</code>)</p></li>
|
||||
</ul></section><hr class="docutils">
|
||||
<section id="references"><h3>References</h3>
|
||||
<div class="publication-list">
|
||||
<h4>2019</h4>
|
||||
<ul><li class="publication" style="padding-bottom: 1em;">Franz Zotter and Matthias Frank.
|
||||
<em><span class="bibtex-protected">Ambisonics: A Practical 3D Audio Theory for Recording, Studio Production, Sound Reinforcement, and Virtual Reality</span></em>.
|
||||
Springer, 2019.<br>[<a href="http://ringbuffer.org/papers/zotter2019ambisonics.html">details</a>]
|
||||
[<a href="javascript:void(0)" onclick="
|
||||
(function(target, id) {
|
||||
if ($('#' + $.escapeSelector(id)).css('display') == 'block')
|
||||
{
|
||||
$('#' + $.escapeSelector(id)).hide('fast');
|
||||
$(target).text('BibTeX▼')
|
||||
}
|
||||
else
|
||||
{
|
||||
$('#' + $.escapeSelector(id)).show('fast');
|
||||
$(target).text('BibTeX▲')
|
||||
}
|
||||
})(this, 'bibtex-zotter2019ambisonics');">BibTeX▼</a>]
|
||||
<div id="bibtex-zotter2019ambisonics" style="display:none"><pre>@book{zotter2019ambisonics,
|
||||
author = "Zotter, Franz and Frank, Matthias",
|
||||
publisher = "Springer",
|
||||
title = "{Ambisonics: A Practical 3D Audio Theory for Recording, Studio Production, Sound Reinforcement, and Virtual Reality}",
|
||||
year = "2019"
|
||||
}
|
||||
</pre></div>
|
||||
</li></ul>
|
||||
<h4>2015</h4>
|
||||
<ul><li class="publication" style="padding-bottom: 1em;">Matthias Frank, Franz Zotter, and Alois Sontacchi.
|
||||
<a href="http://ringbuffer.org/papers/frank2015producing.html"><strong>Producing 3d audio in ambisonics.</strong></a>
|
||||
In <em>Audio Engineering Society Conference: 57th International Conference: The Future of Audio Entertainment Technology–Cinema, Television and the Internet</em>. Audio Engineering Society, 2015.<br>[<a href="http://ringbuffer.org/papers/frank2015producing.html">details</a>]
|
||||
[<a href="javascript:void(0)" onclick="
|
||||
(function(target, id) {
|
||||
if ($('#' + $.escapeSelector(id)).css('display') == 'block')
|
||||
{
|
||||
$('#' + $.escapeSelector(id)).hide('fast');
|
||||
$(target).text('BibTeX▼')
|
||||
}
|
||||
else
|
||||
{
|
||||
$('#' + $.escapeSelector(id)).show('fast');
|
||||
$(target).text('BibTeX▲')
|
||||
}
|
||||
})(this, 'bibtex-frank2015producing');">BibTeX▼</a>]
|
||||
<div id="bibtex-frank2015producing" style="display:none"><pre>@inproceedings{frank2015producing,
|
||||
author = "Frank, Matthias and Zotter, Franz and Sontacchi, Alois",
|
||||
title = "Producing 3D audio in ambisonics",
|
||||
booktitle = "Audio Engineering Society Conference: 57th International Conference: The Future of Audio Entertainment Technology--Cinema, Television and the Internet",
|
||||
year = "2015",
|
||||
organization = "Audio Engineering Society"
|
||||
}
|
||||
</pre></div>
|
||||
</li></ul>
|
||||
<h4>2009</h4>
|
||||
<ul><li class="publication" style="padding-bottom: 1em;">Frank Melchior, Andreas Gr<span class="bibtex-protected">ä</span>fe, and Andreas Partzsch.
|
||||
<a href="http://ringbuffer.org/papers/melchior2009spatial.html"><strong>Spatial audio authoring for ambisonics reproduction.</strong></a>
|
||||
In <em>Proc. of the Ambisonics Symposium</em>. 2009.<br>[<a href="http://ringbuffer.org/papers/melchior2009spatial.html">details</a>]
|
||||
[<a href="javascript:void(0)" onclick="
|
||||
(function(target, id) {
|
||||
if ($('#' + $.escapeSelector(id)).css('display') == 'block')
|
||||
{
|
||||
$('#' + $.escapeSelector(id)).hide('fast');
|
||||
$(target).text('BibTeX▼')
|
||||
}
|
||||
else
|
||||
{
|
||||
$('#' + $.escapeSelector(id)).show('fast');
|
||||
$(target).text('BibTeX▲')
|
||||
}
|
||||
})(this, 'bibtex-melchior2009spatial');">BibTeX▼</a>]
|
||||
<div id="bibtex-melchior2009spatial" style="display:none"><pre>@inproceedings{melchior2009spatial,
|
||||
author = {Melchior, Frank and Gr{\"a}fe, Andreas and Partzsch, Andreas},
|
||||
title = "Spatial audio authoring for Ambisonics reproduction",
|
||||
booktitle = "Proc. of the Ambisonics Symposium",
|
||||
year = "2009"
|
||||
}
|
||||
</pre></div>
|
||||
</li></ul>
|
||||
<h4>1973</h4>
|
||||
<ul><li class="publication" style="padding-bottom: 1em;">Michael A. Gerzon.
|
||||
<a href="http://ringbuffer.org/papers/gerzon1973periphony.html"><strong><span class="bibtex-protected">Periphony: With-Height Sound Reproduction</span>.</strong></a>
|
||||
<em>Journal of the Audio Engineering Society</em>, 21(1):2–10, 1973.<br>[<a href="http://ringbuffer.org/papers/gerzon1973periphony.html">details</a>]
|
||||
[<a href="javascript:void(0)" onclick="
|
||||
(function(target, id) {
|
||||
if ($('#' + $.escapeSelector(id)).css('display') == 'block')
|
||||
{
|
||||
$('#' + $.escapeSelector(id)).hide('fast');
|
||||
$(target).text('BibTeX▼')
|
||||
}
|
||||
else
|
||||
{
|
||||
$('#' + $.escapeSelector(id)).show('fast');
|
||||
$(target).text('BibTeX▲')
|
||||
}
|
||||
})(this, 'bibtex-gerzon1973periphony');">BibTeX▼</a>]
|
||||
<div id="bibtex-gerzon1973periphony" style="display:none"><pre>@article{gerzon1973periphony,
|
||||
author = "Gerzon, Michael A.",
|
||||
journal = "Journal of the Audio Engineering Society",
|
||||
number = "1",
|
||||
pages = "2--10",
|
||||
title = "{Periphony: With-Height Sound Reproduction}",
|
||||
volume = "21",
|
||||
year = "1973"
|
||||
}
|
||||
</pre></div>
|
||||
</li></ul>
|
||||
</div></section></section>
|
||||
</div>
|
||||
<aside class="postpromonav"><nav><ul class="pager hidden-print">
|
||||
<!-- <li class="previous"> --><!-- <a href="/spatial_audio/history/williams-mix/" rel="prev" title="John Cage's Williams Mix">Previous post</a> --><!-- </li> --><!-- <li class="next">
|
||||
<a href="/computer_music_basics/physical_computing/puredata_ldr/" rel="next" title="Pure Data: Light Dependent Resistor">Next post</a>
|
||||
</li> -->
|
||||
</ul></nav></aside><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_HTMLorMML" integrity="sha384-3lJUsx1TJHt7BA4udB5KPnDrlkO8T6J6v/op7ui0BbCjvZ9WqV4Xm6DTP6kQ/iBH" crossorigin="anonymous"></script><script type="text/x-mathjax-config">
|
||||
MathJax.Hub.Config({
|
||||
tex2jax: {
|
||||
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
|
||||
displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
|
||||
processEscapes: true
|
||||
},
|
||||
displayAlign: 'center', // Change this to 'left' if you want left-aligned equations.
|
||||
"HTML-CSS": {
|
||||
styles: {'.MathJax_Display': {"margin": 0}}
|
||||
}
|
||||
});
|
||||
</script></article><!--End of body content--><footer id="footer"><center>
|
||||
<hr>
|
||||
<br>
|
||||
</center>
|
||||
<br></footer>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<script src="../../../assets/js/all-nocdn.js"></script><script>
|
||||
baguetteBox.run('div#content', {
|
||||
ignoreClass: 'islink',
|
||||
captions: function(element) {
|
||||
return element.getElementsByTagName('img')[0].alt;
|
||||
}});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -0,0 +1,331 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<title>Using multiple HDSPe AoX cards :: RME Manuals</title>
|
||||
<link rel="canonical" href="https://docs.rme-audio.com/aoxd/705-1c_multiple_cards_win/">
|
||||
<link rel="prev" href="../704-1c_operation_asio/">
|
||||
<link rel="next" href="../706-1c_digicheck_win/">
|
||||
<meta name="description" content=""Requirements when using more than one HDSPe AoX-D"">
|
||||
<meta name="keywords" content="multiple, several, aggregate, combine, asio, windows">
|
||||
<meta name="generator" content="Antora 3.1.14">
|
||||
<link rel="stylesheet" href="../../_/css/site.css">
|
||||
<script>(function (w, d, s, l, i) {
|
||||
w[l] = w[l] || []; w[l].push({
|
||||
'gtm.start':
|
||||
new Date().getTime(), event: 'gtm.js'
|
||||
}); var f = d.getElementsByTagName(s)[0],
|
||||
j = d.createElement(s), dl = l != 'dataLayer' ? '&l=' + l : ''; j.async = true; j.src =
|
||||
'https://www.googletagmanager.com/gtm.js?id=' + i + dl; f.parentNode.insertBefore(j, f);
|
||||
})(window, document, 'script', 'dataLayer', 'GTM-K7Q74RF');</script>
|
||||
</head>
|
||||
<body class="article">
|
||||
<!-- Google Tag Manager (noscript) -->
|
||||
<noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-K7Q74RF"
|
||||
height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>
|
||||
<!-- End Google Tag Manager (noscript) -->
|
||||
<script async id="search-script" src="../../_/js/vendor/rsearch.js"
|
||||
data-app-id="6I7K1NDPUD" data-api-key="a9fd16b8374ad54610afd2baa40b9ab9"
|
||||
data-index-name="dev_aoxd" data-stylesheet="../../_/css/vendor/docsearch.css"
|
||||
data-product-home="/aoxd/" data-product-title="RME HDSPe AoX-D User's Guide">
|
||||
</script>
|
||||
<header class="header">
|
||||
<nav class="navbar">
|
||||
<div id="navbar-brand" class="navbar-brand">
|
||||
<div class="navbar-item navbar-brand-logo hide-on-search"></div>
|
||||
<a class="navbar-item navbar-title hide-on-search" href="https://docs.rme-audio.com" id="navbar-title">RME Manuals</a>
|
||||
</div>
|
||||
<div id="search-fullwidth" class="navbar-search">
|
||||
<form class="navbar-item search">
|
||||
<input type="text" placeholder="Search" class="query" id="search-input">
|
||||
<button type="button" class="search"><span>Search [S]</span></button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
</nav>
|
||||
</header>
|
||||
<div class="body">
|
||||
<div class="nav-container" data-component="aoxd" data-version="">
|
||||
<aside class="nav">
|
||||
<div class="panels">
|
||||
<div class="nav-panel-menu is-active" data-panel="menu">
|
||||
<nav class="nav-menu">
|
||||
<h3 class="title"><a href="../">RME HDSPe AoX-D User's Guide</a></h3>
|
||||
<ul class="nav-list">
|
||||
<li class="nav-item" data-depth="0">
|
||||
<ul class="nav-list">
|
||||
<li class="nav-item" data-depth="1">
|
||||
<button class="nav-item-toggle"></button>
|
||||
<a class="nav-link" href="../015-1c_introduction/">Introduction</a>
|
||||
<ul class="nav-list">
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../015-2r_package_contents/">Package Contents</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../015-3r_system_requirements/">System Requirements</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../015-4c_brief_description/">Brief Description and Characteristics</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../015-5t_hardware_installation/">Hardware Installation</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../015-6c_hardware_overview/">Hardware Overview</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../015-7r_blockdiagram/">Signal Flow Diagram</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="1">
|
||||
<a class="nav-link" href="../095-1r_accessories/">Accessories</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="1">
|
||||
<button class="nav-item-toggle"></button>
|
||||
<a class="nav-link" href="../070-1c_support/">Warranty and Support</a>
|
||||
<ul class="nav-list">
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../070-2r_support_contacts/">Support Contacts</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="1">
|
||||
<button class="nav-item-toggle"></button>
|
||||
<a class="nav-link" href="../700-1c_operation_usage_windows/">Usage under Windows</a>
|
||||
<ul class="nav-list">
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../701-1c_drivers/">Driver and Firmware (Windows)</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../702-1c_configuration_win/">Configuring the HDSPe AoX-D</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../703-1c_operation_usage_win/">Operation and Usage</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../704-1c_operation_asio/">Operation under ASIO</a>
|
||||
</li>
|
||||
<li class="nav-item is-current-page" data-depth="2">
|
||||
<a class="nav-link" href="./">Using multiple HDSPe AoX cards</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../706-1c_digicheck_win/">DIGICheck Software Overview</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="1">
|
||||
<button class="nav-item-toggle"></button>
|
||||
<a class="nav-link" href="../710-1c_operation_usage_macos/">Usage under macOS</a>
|
||||
<ul class="nav-list">
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../711-1c_drivers_mac/">Driver and Firmware (macOS)</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../712-1c_configuration_mac/">Configuring the HDSPe AoX-D</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="1">
|
||||
<button class="nav-item-toggle"></button>
|
||||
<a class="nav-link" href="../720-1c_connectors/">Connections</a>
|
||||
<ul class="nav-list">
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../720-2c_network/">Network Ports</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../720-3c_headphones/">Line - Headphones</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../720-4r_phones_specifications/">Phones Specifications</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../720-5c_aesebu/">AES/EBU</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../720-6c_midi/">MIDI</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../720-7r_dsub_pinout/">D-Sub 9 Connector Pinout</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="1">
|
||||
<button class="nav-item-toggle"></button>
|
||||
<a class="nav-link" href="../800-1c_totalmix/">Routing and Monitoring with TotalMix</a>
|
||||
<ul class="nav-list">
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../810-1c_totalmix_user_interface/">The User Interface</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../811-1c_channel_strip/">The Channel Strip</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../812-1c_control_room/">Control Room Strip</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../813-1c_control/">Control Strip</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../815-1c_preferences/">Preferences</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../816-1c_settings/">Settings</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../817-1c_hotkeys_usage/">Hotkeys and Usage</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../818-1c_menu_options/">Menu 'Options'</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../819-1c_menu_window/">Menu Window</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../830-1c_matrix/">The Matrix</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../840-1c_asio_direct_monitoring/">ASIO Direct Monitoring (Windows)</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../850-1c_midi_remote_tmfx/">MIDI Remote Overview</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="1">
|
||||
<button class="nav-item-toggle"></button>
|
||||
<a class="nav-link" href="../500-1c_annex/">Annex</a>
|
||||
<ul class="nav-list">
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../520-2r_ce/">CE Compliance</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../520-3r_fcc/">FCC Compliance</a>
|
||||
</li>
|
||||
<li class="nav-item" data-depth="2">
|
||||
<a class="nav-link" href="../520-4r_weee/">Note on Disposal</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</aside>
|
||||
</div>
|
||||
<main class="article">
|
||||
<div class="toolbar" role="navigation">
|
||||
<button class="nav-toggle"></button>
|
||||
<a href="../../shared/" class="home-link"></a>
|
||||
<nav class="breadcrumbs" aria-label="breadcrumbs">
|
||||
<ul>
|
||||
<li><a href="../">RME HDSPe AoX-D User's Guide</a></li>
|
||||
<li><a href="../700-1c_operation_usage_windows/">Usage under Windows</a></li>
|
||||
<li><a href="./">Using multiple HDSPe AoX cards</a></li>
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
<div class="content">
|
||||
<article class="doc">
|
||||
<h1 class="page">Using multiple HDSPe AoX cards</h1>
|
||||
<div class="paragraph">
|
||||
<p>The current driver supports the operation of up to three HDSPe AoX cards.
|
||||
It is possible to use both HDSPe AoX-M and HDSPe AoX-D at the same time.
|
||||
To ensure proper functionality, all cards must remain synchronized, either through a valid sync source via word clock or by using <strong>Auto-Sync</strong> with synchronized signals.</p>
|
||||
</div>
|
||||
<h2 id="_synchronization_setup" class="discrete">Synchronization Setup</h2>
|
||||
<div class="paragraph">
|
||||
<p>To maintain precise synchronization between multiple cards, internal connectors are available.
|
||||
The cards should be synchronized to the leader card by connecting the corresponding <strong>SYNC</strong> headers on the card with the supplied cable.
|
||||
Ensure that the clock modes for all cards are configured correctly in their respective <strong>Settings</strong> dialogs.</p>
|
||||
</div>
|
||||
<div class="ulist">
|
||||
<ul>
|
||||
<li>
|
||||
<p>When all units are supplied with a synchronous clock signal (i.e., all units display <strong>Sync</strong> in their <strong>Settings</strong> dialogs), all channels across the cards can be utilized simultaneously.
|
||||
This setup is particularly streamlined under ASIO, as the ASIO driver combines all cards into a single interface.</p>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<h2 id="_totalmix_with_multiple_cards" class="discrete">TotalMix with multiple cards</h2>
|
||||
<div class="paragraph">
|
||||
<p><strong>TotalMix</strong> is integrated into the hardware of each HDSPe AoX-D, allowing for up to three independent mixers when using three cards.
|
||||
However, these mixers are separate and do not support data exchange between cards.
|
||||
As a result, a global mixer for all units is not available.</p>
|
||||
</div>
|
||||
<div class="admonitionblock tip">
|
||||
<table>
|
||||
<tr>
|
||||
<td class="icon">
|
||||
<i class="fa icon-tip" title="Tip"></i>
|
||||
</td>
|
||||
<td class="content">
|
||||
If required, submixes could be aggregated from different cards using the optional MADI ports.
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
</div>
|
||||
<form action="https://submit-form.com/zq8cJcer0V5_Gt-P_RyS5" class="feedback" id="FeedbackForm">
|
||||
<div class="form-wrapper.show">
|
||||
<div class="question">
|
||||
<p>Was this topic helpful?</p>
|
||||
<div class="buttons">
|
||||
<button aria-expanded="false" type="button" id="feedback_yes"
|
||||
class="button button--secondary button--compact feedback--yes">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" height="24" viewBox="0 0 24 24" width="24">
|
||||
<path d="M0 0h24v24H0V0z" fill="none" />
|
||||
<path
|
||||
d="M1 21h4V9H1v12zm22-11c0-1.1-.9-2-2-2h-6.31l.95-4.57.03-.32c0-.41-.17-.79-.44-1.06L14.17 1 7.59 7.59C7.22 7.95 7 8.45 7 9v10c0 1.1.9 2 2 2h9c.83 0 1.54-.5 1.84-1.22l3.02-7.05c.09-.23.14-.47.14-.73v-2z" />
|
||||
</svg>
|
||||
</button>
|
||||
<button aria-expanded="false" type="button" id="feedback_no"
|
||||
class="button button--secondary button--compact feedback--no">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" height="24" viewBox="0 0 24 24" width="24">
|
||||
<path d="M0 0h24v24H0z" fill="none" />
|
||||
<path
|
||||
d="M15 3H6c-.83 0-1.54.5-1.84 1.22l-3.02 7.05c-.09.23-.14.47-.14.73v2c0 1.1.9 2 2 2h6.31l-.95 4.57-.03.32c0 .41.17.79.44 1.06L9.83 23l6.59-6.59c.36-.36.58-.86.58-1.41V5c0-1.1-.9-2-2-2zm4 0v12h4V3h-4z" />
|
||||
</svg>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div id="form-success" class="form-success">Thank you!</div>
|
||||
<div id="form-message" class="form-message">
|
||||
<p>
|
||||
Thank you for letting us know.
|
||||
</p>
|
||||
<p>
|
||||
Please tell us what you were looking for:
|
||||
</p>
|
||||
<input type="text" name="URL" value="/aoxd/705-1c_multiple_cards_win/" hidden="true" />
|
||||
<input type="text" name="honig_topf_48xz8" class="honig_topf_48xz8" />
|
||||
<input type="text" name="sweetFormField" class="honig_topf_48xz8" />
|
||||
<textarea name="message" class="form-input full" placeholder="Your message" maxlength="5000" required></textarea>
|
||||
<p> Your E-Mail (in case you would like us to contact you):</p>
|
||||
<input type="email" name="email" class="form-input form-input--wide" placeholder="your.email@rme-audio.de">
|
||||
<button type="submit" class="submitForm" id="submitForm">Submit your feedback!</button>
|
||||
</div>
|
||||
<p id="js-form-response"></p>
|
||||
</div>
|
||||
</form>
|
||||
<nav class="pagination">
|
||||
<span class="prev"><a href="../704-1c_operation_asio/">Operation under ASIO</a></span>
|
||||
<span class="next"><a href="../706-1c_digicheck_win/">DIGICheck Software Overview</a></span>
|
||||
</nav>
|
||||
</article>
|
||||
<aside class="toc sidebar" data-title="Contents" data-levels="2">
|
||||
<div class="toc-menu"></div>
|
||||
</aside>
|
||||
</div>
|
||||
</main>
|
||||
</div>
|
||||
<footer class="footer">
|
||||
<p>©2025 RME / m2lab Ltd.</p>
|
||||
</footer>
|
||||
<script src="../../_/js/site.js"></script>
|
||||
<script async src="../../_/js/vendor/highlight.js"></script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
10
research/ambisonic-aquasynth/mirrors/sparta-overview.html
Normal file
10
research/ambisonic-aquasynth/mirrors/sparta-overview.html
Normal file
File diff suppressed because one or more lines are too long
83
research/ambisonic-aquasynth/summary.md
Normal file
83
research/ambisonic-aquasynth/summary.md
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
# Ambisonic AquaSynth Field: Distilled Research
|
||||
|
||||
## Objective
|
||||
|
||||
Consolidate four live microphones -- shielded cardioid, Deru shotgun, and two camera mics -- into one synchronized spatial audio scene that can feed an AquaSynth Faust DSP graph, while also placing dynamic synth patch outputs in 3D.
|
||||
|
||||
## Current Mechanism
|
||||
|
||||
Mimir currently treats audio as independent OBS-ingestable endpoints: one FFmpeg process per audio source, one SRT port per source, and OBS owns the mix. That is coherent for manual OBS mixing, but it is the wrong ownership model for a single spatial field. An ambisonic bus needs clocked multichannel audio before spatial encoding, then effects, then decode/render.
|
||||
|
||||
## Invariants
|
||||
|
||||
- Capture sync must be owned before spatial encoding. Drift correction after the fact can make playback tolerable, but it does not create a physically coherent sound field.
|
||||
- The ambisonic representation must declare order, channel count, channel order, normalization, sample rate, and render target.
|
||||
- The Faust graph should receive one declared spatial bus, not four unsynchronized mono stories wearing a coat.
|
||||
- Microphone geometry and directivity must be explicit if microphone feeds are treated as a captured field.
|
||||
- Synth outputs are easier: each synth voice is a mono or stereo source with explicit azimuth/elevation/radius metadata, then encoded into the same ambisonic bus.
|
||||
|
||||
## Main Finding
|
||||
|
||||
Four arbitrary microphones are not automatically a first-order ambisonic microphone. ICST distinguishes A-format as microphone-specific raw capsule output and B-format as the production/exchange sound-field representation; A-format must be converted before it is a usable ambisonic master [ICST]. A tetrahedral ambisonic mic works because capsule geometry and calibration are part of the device model. Your mic set is heterogeneous: cardioid, shotgun, and camera mics have different positions, directivities, gains, noise floors, latency, and likely clocks.
|
||||
|
||||
McCormack et al. show that arbitrary microphone arrays can be encoded into Ambisonics, but even conventional linear encoding is limited by geometry, sensor placement, regularization, spatial aliasing, and bandwidth; their proposed parametric method uses spatial analysis, filtering, source/ambient decomposition, and measured or modeled array behavior [McCormack 2022]. Translation: this is research-grade if we want "real recovered sound field," not a cute four-channel matrix.
|
||||
|
||||
## Recommended Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["4 physical mic feeds"] --> B["Clocked multichannel capture"]
|
||||
B --> C["Calibration: gain, delay, polarity, noise, geometry"]
|
||||
C --> D["Scene encoder"]
|
||||
E["Dynamic synth patch voices"] --> F["Position metadata: azimuth, elevation, radius"]
|
||||
F --> G["Faust HOA/FOA source encoders"]
|
||||
D --> H["Ambisonic bus"]
|
||||
G --> H
|
||||
H --> I["AquaSynth Faust spatial DSP"]
|
||||
I --> J["Decoder / renderer: binaural, stereo, OBS, speakers"]
|
||||
```
|
||||
|
||||
## Practical Path
|
||||
|
||||
1. Capture all four microphones through one clock domain.
|
||||
Prefer one multichannel audio interface or a driver/server that exposes one synchronized device. RME's documentation for multiple cards is blunt that cards need a shared sync source or sync headers to remain synchronized [RME]. JACK's design centers synchronous client execution and low-latency inter-application routing, and on Windows it can bridge ASIO clients through JACK-Router [JACK API; JACK Windows]. FFmpeg DirectShow can open multiple devices and says opening them on the same input "should improve synchronism," but that is not the same as proving sample-accurate shared-clock capture [FFmpeg Devices].
|
||||
|
||||
2. Pick the honest spatial model.
|
||||
For v1, do source-based ambisonic scene building: treat each mic as a directional source at a measured position, with calibration delays and gains. This produces a usable spatial mix, not a physically exact reconstruction. For v2, investigate measured arbitrary-array encoding if the room-capture realism matters enough to justify calibration sweeps and signal analysis.
|
||||
|
||||
3. Use ambiX unless a tool forces otherwise.
|
||||
ICST and IEM both identify ambiX as ACN channel ordering with SN3D normalization, and IEM explicitly uses that convention [ICST; IEM Compatibility]. This should be the default interchange format for the bus.
|
||||
|
||||
4. Start with FOA, design the graph for HOA.
|
||||
First-order Ambisonics is 4 channels. Higher order uses `(order + 1)^2` channels and gives sharper spatial resolution at higher routing and CPU cost [ICST]. Faust `hoa.lib` provides encoders, decoders, rotators, optimizers, and 3D encoders; `encoder3D(N, x, a, e)` directly fits dynamic synth voices [Faust hoa.lib]. The graph should parameterize order so FOA can ship first without boxing out HOA.
|
||||
|
||||
5. Keep OBS as render target, not spatial authority.
|
||||
If this path becomes live, OBS should ingest the rendered output or selected post-DSP stems. OBS should not be asked to preserve the ambisonic field while separately mixing the pre-field microphones. That splits authority and invites drift.
|
||||
|
||||
## Cut Line
|
||||
|
||||
Cut or suspend the old one-port-per-audio-source model for any microphone that participates in the ambisonic field. It can remain for plain OBS monitor sources, but the spatial bus must be one synchronized capture path.
|
||||
|
||||
Do not build a bespoke arbitrary-array encoder first. Use Faust `hoa.lib` for source encoding and established tooling like IEM/SPARTA for verification. Only consider a custom array encoder after we have measured mic geometry, clock behavior, impulse responses, and proven that source-based encoding is not enough.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Are the two camera mics available as embedded channels from one capture device, or as separate USB/device clocks?
|
||||
- Do we have a multichannel interface that can ingest all four mics into one ASIO device?
|
||||
- Is AquaSynth currently a Faust host, a generated plugin target, or a separate graph compiler/runtime?
|
||||
- Is the desired output binaural headphone monitoring, stereo OBS program audio, surround speakers, or a stored ambisonic master?
|
||||
- Do we need the room itself reconstructed, or do we need a convincing controllable spatial mix for performance?
|
||||
|
||||
## Citations
|
||||
|
||||
- [ICST] ICST Ambisonics, "Ambisonics Formats Explained: A-Format, B-Format, FuMa, AmbiX, FOA and HOA." Mirror: `mirrors/icst-ambisonics-formats.html`.
|
||||
- [RingBuffer] RingBuffer, "Understanding Ambisonics." Mirror: `mirrors/ringbuffer-understanding-ambisonics.html`.
|
||||
- [McCormack 2022] McCormack, Politis, Gonzalez, Lokki, and Pulkki, "Parametric Ambisonic Encoding of Arbitrary Microphone Arrays," IEEE/ACM TASLP, 2022. Mirror: `mirrors/mccormack-2022-parametric-ambisonic-encoding.pdf`.
|
||||
- [Faust hoa.lib] Faust Libraries, `hoa.lib`. Mirror: `mirrors/faust-hoa-lib.html`.
|
||||
- [IEM] IEM Plug-in Suite. Mirror: `mirrors/iem-plugin-suite.html`.
|
||||
- [IEM Compatibility] IEM Plug-in Suite, "Compatibility with other Suites." Mirror: `mirrors/iem-compatibility.html`.
|
||||
- [SPARTA] SPARTA, "Overview." Mirror: `mirrors/sparta-overview.html`.
|
||||
- [FFmpeg Devices] FFmpeg Devices Documentation. Mirror: `mirrors/ffmpeg-devices.html`.
|
||||
- [JACK API] JACK Audio Connection Kit API Overview. Mirror: `mirrors/jack-api-overview.html`.
|
||||
- [JACK Windows] JACK Audio Connection Kit, "Using JACK on Windows." Mirror: `mirrors/jack-on-windows.html`.
|
||||
- [RME] RME Manuals, "Using multiple HDSPe AoX cards." Mirror: `mirrors/rme-multiple-cards-sync.html`.
|
||||
32
research/feedback-calibration/bibliography.md
Normal file
32
research/feedback-calibration/bibliography.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# Bibliography: Feedback As Continuous Calibration
|
||||
|
||||
Mirrored copies live under `research/feedback-calibration/mirrors/`.
|
||||
|
||||
## Echo / Feedback System Identification
|
||||
|
||||
- S. Goetze, M. Kallinger, A. Mertins, and K. Kammeyer, "System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller," HSCMA 2008.
|
||||
Mirror: `mirrors/goetze-2008-aec-listening-room-compensation.html`
|
||||
|
||||
- F. Lindstrom, C. Schueldt, and I. Claesson, "Efficient Multichannel NLMS Implementation for Acoustic Echo Cancellation," EURASIP Journal on Audio, Speech, and Music Processing, 2007.
|
||||
Mirror: `mirrors/lindstrom-2007-efficient-multichannel-nlms.html`
|
||||
|
||||
- J.-M. Valin, "A New Robust Frequency Domain Echo Canceller With Closed-Loop Learning Rate Adaptation," arXiv:1602.08609, 2016.
|
||||
Mirror: `mirrors/valin-2016-frequency-domain-echo-canceller-learning-rate.pdf`
|
||||
|
||||
- "Double-Talk Robust Acoustic Echo Cancellation," EUSIPCO 2021.
|
||||
Mirror: `mirrors/eusipco-2021-double-talk-robust-aec.pdf`
|
||||
|
||||
- Apple Machine Learning Research, "Double-talk Robust Multichannel Acoustic Echo Cancellation Using Least Squares MIMO Adaptive Filtering," 2020.
|
||||
Mirror: `mirrors/apple-mimo-adaptive-echo-cancellation.html`
|
||||
|
||||
## Online Room / Acoustic Impulse Response Estimation
|
||||
|
||||
- J. Nikunen and T. Virtanen, "Estimation of Time-Varying Room Impulse Responses of Multiple Sound Sources from Observed Mixture and Isolated Source Signals," ICASSP 2018.
|
||||
Mirror: `mirrors/nikunen-virtanen-2018-time-varying-rir.pdf`
|
||||
|
||||
- T. Haubner, A. Brendel, and W. Kellermann, "Online Acoustic System Identification Exploiting Kalman Filtering and an Adaptive Impulse Response Subspace Model," arXiv:2105.03337, 2021.
|
||||
Mirror: `mirrors/haubner-2021-online-acoustic-system-identification.pdf`
|
||||
|
||||
## Notes
|
||||
|
||||
MathWorks AEC and Filtered-X LMS examples were consulted via browser snippets/opened page content, but direct mirroring was blocked by access controls from PowerShell. They are not relied on as mirrored primary artifacts in the summary.
|
||||
File diff suppressed because one or more lines are too long
Binary file not shown.
|
|
@ -0,0 +1,881 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<!DOCTYPE html>
|
||||
<html lang="de" class="nexus-font-support">
|
||||
|
||||
|
||||
<head>
|
||||
<meta charset="utf-8"/>
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1"/>
|
||||
<meta name="google" content="nositelinkssearchbox"/>
|
||||
<meta name="tdm-reservation" content="0">
|
||||
<title>
|
||||
System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller
|
||||
- Universität zu Lübeck</title>
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel='alternate' hreflang='en-gb' href='https://research.uni-luebeck.de/en/publications/system-identification-for-multi-channel-listening-room-compensati/' /><link rel='alternate' hreflang='de-de' href='https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/' />
|
||||
|
||||
<meta property="og:site_name" content="Universität zu Lübeck">
|
||||
|
||||
|
||||
<meta property="og:title" content="System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller"/>
|
||||
|
||||
|
||||
<meta property="og:url" content="https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/">
|
||||
<meta property="og:locale" content="de_DE">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta property="og:image" content="https://research.uni-luebeck.de/skin/headerImage/">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta property="og:type" content="article">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="/assets/application-119d61ec31b02051205711822747d1ca.css" />
|
||||
<!-- OneTrust Cookies Consent Notice start -->
|
||||
<script async src="https://cdn.cookielaw.org/scripttemplates/otSDKStub.js" type="text/javascript" charset="UTF-8" data-domain-script="d0404472-8daf-4764-aeb6-b4242ae76a56" data-document-language="true"></script>
|
||||
<script type="text/javascript">
|
||||
function OptanonWrapper() { }
|
||||
</script>
|
||||
<!-- OneTrust Cookies Consent Notice end -->
|
||||
|
||||
<link rel="shortcut icon" href="/skin/favIcon/"/>
|
||||
|
||||
<link href="/skin/css/" rel="stylesheet"/>
|
||||
|
||||
<script src="/skin/script/" type="text/javascript"></script>
|
||||
<script type="text/javascript" src="/assets/application-34cc2221eabdb23f03948cb449f4d1ca.js" ></script>
|
||||
<!--[if lt IE 9]>
|
||||
<script type="text/javascript" src="/assets/html5shiv.min-ca5300beead9a4a9d2c9c584a9016410.js" ></script>
|
||||
<script type="text/javascript" src="/assets/respond.min-678d372b349e434d9923e1b591f14a1e.js" ></script>
|
||||
<![endif]-->
|
||||
|
||||
|
||||
|
||||
<meta name="citation_title" content="System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller"></meta><meta name="citation_inbook_title" content="2008 Hands-Free Speech Communication and Microphone Arrays"></meta><meta name="citation_author" content="S. Goetze"></meta><meta name="citation_author" content="M. Kallinger"></meta><meta name="citation_author" content="A. Mertins"></meta><meta name="citation_author_institution" content="Institut für Signalverarbeitung"></meta><meta name="citation_author_email" content="alfred.mertins@uni-luebeck.de"></meta><meta name="citation_author" content="K. Kammeyer"></meta><meta name="citation_publication_date" content="2008/05/01"></meta><meta name="citation_doi" content="10.1109/HSCMA.2008.4538727"></meta><meta name="citation_language" content="Englisch"></meta><meta name="citation_publisher" content="IEEE"></meta><meta name="citation_firstpage" content="224"></meta><meta name="citation_lastpage" content="227"></meta><meta name="citation_conference_title" content="2008 Hands-free Speech Communication and Microphone Arrays"></meta>
|
||||
<script src="/assets/Chart.min-27bff060c3e539ceb22e763c8f2e9e10.js"></script>
|
||||
<script src="/assets/jquery.content-chart-9c09bb79ba3fc3829b6478d0e114f737.js"></script>
|
||||
|
||||
|
||||
|
||||
<!-- Google Analytics: No account ID (G) found -->
|
||||
<!-- SiteCatalyst not enabled -->
|
||||
|
||||
|
||||
</head>
|
||||
|
||||
<body class="" data-page="publications">
|
||||
|
||||
<div id="aria-polite" aria-live="polite"></div>
|
||||
<div id="aria-assertive" aria-live="assertive"></div>
|
||||
|
||||
<a href="#main-navigation" class="sr-only sr-only-focusable">Zur Hauptnavigation wechseln</a>
|
||||
|
||||
<a href="#main-search" class="sr-only sr-only-focusable">Zur Suche wechseln</a>
|
||||
|
||||
<a href="#main-content" class="sr-only sr-only-focusable">Zum Hauptinhalt wechseln</a>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<header class="header" id="page-header">
|
||||
<div class="primary ">
|
||||
<div class="container">
|
||||
|
||||
<a href="/de/" class="logo">
|
||||
|
||||
<span class="sr-only">Universität zu Lübeck Home</span>
|
||||
<img src="/skin/headerImage/" alt="Universität zu Lübeck Logo"/>
|
||||
|
||||
</a>
|
||||
|
||||
<ul class="header-tools ">
|
||||
|
||||
|
||||
<li class="language-dropdown dropdown">
|
||||
<button data-toggle="dropdown" class="language_btn" aria-label="Sprache auswählen" aria-expanded="false">
|
||||
<i class="icon icon-globe" aria-hidden="true"></i></button>
|
||||
|
||||
<div class="dropdown-container">
|
||||
<ul class='language-selector dropdown-menu'><li><a href="/en/publications/system-identification-for-multi-channel-listening-room-compensati/" lang="en">English</a></li><li class='selected'><a href="/de/publications/system-identification-for-multi-channel-listening-room-compensati/" aria-current="true" lang="de">Deutsch</a></li></ul>
|
||||
</div>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<div class="header-search">
|
||||
|
||||
<form action="/de/searchAll/index/system-identification-for-multi-channel-listening-room-compensati/" method="get"
|
||||
class="search-form" id="search-everything">
|
||||
|
||||
<div class="global-search-wrapper">
|
||||
<div class="global-search mini">
|
||||
<div class="field-button-wrapper">
|
||||
<div class="search-cell search-field">
|
||||
<label for="main-search" class="sr-only">Inhalt suchen unter Universität zu Lübeck</label>
|
||||
<input id="main-search" class="search-field" value="" name="search"
|
||||
placeholder="Suchen …"/>
|
||||
</div>
|
||||
|
||||
<div class="search-submit">
|
||||
<button type="submit" aria-label="Suchen">
|
||||
<i class="icon icon-magnifying-glass" aria-hidden="true"></i>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</form>
|
||||
<script>
|
||||
$(function () {
|
||||
var searchForm = $("#search-everything");
|
||||
var searchField = $(".search-field", searchForm);
|
||||
searchForm.on("submit", function () {
|
||||
var field = searchField.find("input");
|
||||
var searchString = field.val();
|
||||
|
||||
// No enter key submit nothing :-)
|
||||
if (!searchString) return false;
|
||||
|
||||
if (searchString.length >= 200) {
|
||||
searchForm.attr("method", "post");
|
||||
searchForm.attr("action", "/de/concepts/copypaste/");
|
||||
field.attr("name", "text");
|
||||
}
|
||||
});
|
||||
});
|
||||
</script>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<div class="subMenu">
|
||||
<div class="secondary">
|
||||
<div class="container">
|
||||
<nav id="main-navigation" aria-label="Hauptnavigation" class="content-navigation three-third">
|
||||
<ul data-value="Mehr" role="menubar" aria-label="Hauptnavigation">
|
||||
<li class="" role="none">
|
||||
<a href="/de/" role="menuitem" tabindex="-1">Home</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="" role="none">
|
||||
<a class="portal_link" tabindex="-1" role="menuitem" href="/de/organisations/"><span>Einrichtungen</span></a>
|
||||
</li>
|
||||
|
||||
<li class="" role="none">
|
||||
<a class="portal_link" tabindex="-1" role="menuitem" href="/de/persons/"><span>Personen</span></a>
|
||||
</li>
|
||||
|
||||
<li class="selected" role="none">
|
||||
<a class="portal_link" tabindex="0" role="menuitem" href="/de/publications/"><span>Publikationen</span></a>
|
||||
</li>
|
||||
|
||||
<li class="" role="none">
|
||||
<a class="portal_link" tabindex="-1" role="menuitem" href="/de/projects/"><span>Projekte</span></a>
|
||||
</li>
|
||||
|
||||
<li class="" role="none">
|
||||
<a class="portal_link" tabindex="-1" role="menuitem" href="/de/prizes/"><span>Preise</span></a>
|
||||
</li>
|
||||
|
||||
<li class="" role="none">
|
||||
<a class="portal_link" tabindex="-1" role="menuitem" href="/de/impacts/"><span>Spin-Offs</span></a>
|
||||
</li>
|
||||
|
||||
<li class="" role="none">
|
||||
<a class="portal_link" tabindex="-1" role="menuitem" href="/de/clippings/"><span>Medien</span></a>
|
||||
</li>
|
||||
|
||||
<li class="" role="none">
|
||||
<a class="portal_link" tabindex="-1" role="menuitem" href="/de/activities/"><span>Aktivitäten</span></a>
|
||||
</li>
|
||||
|
||||
<li class="" role="none">
|
||||
<a class="portal_link" tabindex="-1" role="menuitem" href="/de/datasets/"><span>Forschungsdatensätze</span></a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</header>
|
||||
|
||||
<main id="page-content">
|
||||
|
||||
<div class="page-section page-section-publications page-section-publications-view ">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section
|
||||
class="page-section-header page-section-header-publications page-section-header-publications-view
|
||||
no-backdrop-image"
|
||||
data-stickyheader=""
|
||||
style="">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<div class="container">
|
||||
<div class="row">
|
||||
<div class="introduction no-metrics">
|
||||
<div class="rendering"><h1><span>System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller</span></h1></div>
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_associatespersonsclassifiedportal rendering_contributiontobookanthology rendering_associatespersonsclassifiedportal rendering_contributiontobookanthology_associatespersonsclassifiedportal"><p class="relations persons">S. Goetze, M. Kallinger, <a rel="Person" href="https://research.uni-luebeck.de/de/persons/alfred-mertins/" class="link person"><span>A. Mertins</span></a>, K. Kammeyer</div>
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_associatesorganisationsportal rendering_contributiontobookanthology rendering_associatesorganisationsportal rendering_contributiontobookanthology_associatesorganisationsportal"><ul class="relations organisations"><li class="department"><a rel="Organisation" href="https://research.uni-luebeck.de/de/organisations/institute-of-signal-processing/" class="link"><span>Institut für Signalverarbeitung</span></a></li></ul></div>
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<div class="publication-metrics" role="region" aria-label="Publikationsmetrik">
|
||||
|
||||
<div class="metrics-container">
|
||||
|
||||
<div class="metrics-wrap metrics-top one">
|
||||
|
||||
<div class="metric scopus-citations">
|
||||
|
||||
<a target="_blank" href="https://www.scopus.com/inward/citedby.url?scp=50449102952&partnerID=8YFLogxK" aria-label="Quelle: Scopus: 7">
|
||||
|
||||
<span class="count">7</span>
|
||||
|
||||
<span class="sr-only"> Link öffnet neuen Tab</span> </a>
|
||||
|
||||
<span class="label">Zitate (Scopus)</span>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
<div class="metrics-wrap metrics-bottom ">
|
||||
|
||||
<div class="metric metric-plum-x">
|
||||
<a href="https://plu.mx/plum/a/?doi=10.1109/HSCMA.2008.4538727" class="plumx-plum-print-popup" data-on-empty="plumXError" data-on-fail="plumXError" data-on-success="plumXSuccess" data-popup="left" data-hide-when-empty="true" data-site="plum"
|
||||
data-badge="false"></a>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
<div class="metric metric-altmetric">
|
||||
|
||||
<div data-badge-popover="top" data-badge-type="donut" data-doi="10.1109/HSCMA.2008.4538727" data-hide-no-mentions="true" data-link-target='_blank'
|
||||
class="altmetric-embed"></div>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
<div class="subMenu">
|
||||
<div class="content-profile-navigation">
|
||||
<div class="container">
|
||||
<nav role="navigation">
|
||||
<ul role="menubar">
|
||||
<li class="selected" role="none">
|
||||
<a class="portal_link" role="menuitem" href="/de/publications/system-identification-for-multi-channel-listening-room-compensati/"><span>
|
||||
<i class="icon icon-publication" aria-hidden="true"></i>
|
||||
<span class="label">Übersicht </span>
|
||||
</span></a>
|
||||
</li>
|
||||
|
||||
<li class="" role="none">
|
||||
<a class="portal_link" role="menuitem" href="/de/publications/system-identification-for-multi-channel-listening-room-compensati/fingerprints/"><span>
|
||||
<i class="icon icon-fingerprint" aria-hidden="true"></i>
|
||||
<span class="label">Fingerprint</span>
|
||||
</span></a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
</section>
|
||||
|
||||
<div id="main-content"
|
||||
class="page-section-content page-section-content-publications page-section-content-publications-view ">
|
||||
|
||||
<section class="page-section content-relation-section">
|
||||
<div class="container">
|
||||
<div class="clearfix">
|
||||
<div class="content-content publication-content">
|
||||
|
||||
|
||||
<h2 class="subheader">Abstract</h2>
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_abstractportal rendering_contributiontobookanthology rendering_abstractportal rendering_contributiontobookanthology_abstractportal"><div class="textblock">Modern hands-free telecommunication devices jointly apply several subsystems, e.g. for noise reduction (NR), acoustic echo cancellation (AEC) and listening-room compensation (LRC). In this contribution the combination of an equalizer for listening room compensation and an acoustic echo canceller is analyzed. Inverse filtering of room impulse responses (RIRs) is a challenging task since they are, in general, mixed phase systems having hundreds of zeros inside and outside near the unit circle in the z-domain. Furthermore, a reliable estimate of the RIR which shall be inverted is important. Since RIRs are time-variant due to possible changes of the acoustic environment, they have to be identified adaptively. If an AEC (or any other adaptive method) is used to identify the time variant room impulse responses the estimate's distance to the real RIRs may be too high for a satisfying equalization, especially in periods of initial convergence of the AEC or after RIR changes. Therefore, we propose to estimate the convergence state of the AEC and to incorporate this knowledge into the equalizer design.</div></div>
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_detailsportal rendering_contributiontobookanthology rendering_detailsportal rendering_contributiontobookanthology_detailsportal"><div class="conference"><table class="properties"><tbody><tr class="language"><th scope="row">Originalsprache</th><td>Englisch</td></tr><tr><th scope="row">Titel</th><td>2008 Hands-Free Speech Communication and Microphone Arrays</td></tr><tr><th scope="row">Seitenumfang</th><td>4</td></tr><tr><th scope="row">Herausgeber (Verlag)</th><td><a rel="Publisher" href="#" class="link"><span>IEEE</span></a></td></tr><tr><th scope="row">Erscheinungsdatum</th><td><span class="date">01.05.2008</span></td></tr><tr><th scope="row">Seiten</th><td>224-227</td></tr><tr><th scope="row">Aufsatznummer</th><td>4538727</td></tr><tr class="isbn-print"><th scope="row">ISBN (Print)</th><td>978-1-4244-2337-8</td></tr><tr class="isbn-elektronisch"><th scope="row">ISBN (elektronisch)</th><td>978-1-4244-2338-5</td></tr><tr class="digital_object_identifiers"><th scope="row">DOIs</th><td><ul class="relations digital_object_identifiers"><li><a onclick="window.open(this.href, '_blank','noopener,noreferrer'); return false;" href="https://doi.org/10.1109/HSCMA.2008.4538727" class="link"><span>https://doi.org/10.1109/HSCMA.2008.4538727</span></a> </li></ul></td></tr><tr class="status"><th scope="row">Publikationsstatus</th><td><span class="prefix">Veröffentlicht - </span><span class="date">01.05.2008</span></td></tr><tr class="event"><th scope="row">Veranstaltung</th><td><a rel="Event" href="#" class="link"><span><span class="prefix">2008 Hands-free Speech Communication and Microphone Arrays</span></a> - Trento, Italien<br />Dauer: <span class="date">06.05.2008</span> → <span class="date">08.05.2008</span><br />Konferenznummer: 73345</span></td></tr></tbody></table></div></div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<div class="sdgs-content-page">
|
||||
<h2 id="sdg-title" class="subheader">UN SDGs</h2>
|
||||
<p class="sdg-text">Dieser Output leistet einen Beitrag zu folgendem(n) Ziel(en) für nachhaltige Entwicklung</p>
|
||||
<ol class="sdg_icons" aria-labelledby="sdg-title">
|
||||
|
||||
<li class="content-page">
|
||||
|
||||
<img alt="SDG 9 – Industrie, Innovation und Infrastruktur" title="SDG 9 – Industrie, Innovation und Infrastruktur" class="industry_innovation_and_infrastructure"
|
||||
src="/assets/sdg_icons/industry_innovation_and_infrastructure-38d336019fd4b8ef110b4e64f4c8d344.svg"
|
||||
role="presentation"/>
|
||||
<div class="sdg-name">
|
||||
<span>SDG 9 – Industrie, Innovation und Infrastruktur</span>
|
||||
<span></span>
|
||||
</div>
|
||||
</li>
|
||||
|
||||
</ol>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
<div class="content-sidebar publication-sidebar">
|
||||
|
||||
<h2 class="subheader">Dokumente</h2>
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_publicationaccessrenderer rendering_contributiontobookanthology rendering_publicationaccessrenderer rendering_contributiontobookanthology_publicationaccessrenderer"><ul class="dois"><li><div class="doi"><a onclick="window.open(this.href, '_blank','noopener,noreferrer'); return false;" href="https://doi.org/10.1109/HSCMA.2008.4538727" class="link"><span>10.1109/HSCMA.2008.4538727</span></a></div></li></ul></div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<h2 class="subheader">Weitere Links</h2>
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_publicationotherfilesandlinksrenderer rendering_contributiontobookanthology rendering_publicationotherfilesandlinksrenderer rendering_contributiontobookanthology_publicationotherfilesandlinksrenderer"><ul class="links"><li><div class="link"><a onclick="window.open(this.href, '_blank','noopener,noreferrer'); return false;" href="https://www.scopus.com/pages/publications/50449102952" class="link"><span>https://www.scopus.com/pages/publications/50449102952</span></a></div></li></ul></div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="page-section content-relation-section publication-fingerprint">
|
||||
<div class="container">
|
||||
<div class="header">
|
||||
<h2 class="section-title">
|
||||
<i class="icon icon-fingerprint" aria-hidden="true"></i>
|
||||
<span class="title">Fingerprint</span>
|
||||
</h2>
|
||||
<span class="sub-title">
|
||||
Untersuchen Sie die Forschungsthemen von „System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller“. Zusammen bilden sie einen einzigartigen Fingerprint.
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<ul class="publication-top-concepts" role="menubar" aria-label="Fingerprints">
|
||||
|
||||
<li class="concept-badge-large-container dropdown-overflow" role="none">
|
||||
|
||||
<button tabindex="0"
|
||||
class="concept-badge-large dropdown-toggle"
|
||||
type="button"
|
||||
role="menuitem"
|
||||
aria-haspopup="true"
|
||||
aria-expanded="false"
|
||||
aria-controls="4eafe263-ae6b-42d5-b0ce-d6da7277e755"
|
||||
data-concept-uuid="4eafe263-ae6b-42d5-b0ce-d6da7277e755"
|
||||
data-rank-size="32"
|
||||
data-rank-thickness="8"
|
||||
data-rank-value="1.0"
|
||||
data-summary="true"
|
||||
data-summary-url="/de/concepts/summary/">
|
||||
<span class="concept-wrapper" title="100%">
|
||||
<span class="rank"></span>
|
||||
<span class="concept">Multichannel</span>
|
||||
<span class="thesauri">Engineering</span>
|
||||
<span class="value sr-only">100%</span>
|
||||
</span>
|
||||
</button>
|
||||
|
||||
<div class="dropdown-container"></div>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="concept-badge-large-container dropdown-overflow" role="none">
|
||||
|
||||
<button tabindex="0"
|
||||
class="concept-badge-large dropdown-toggle"
|
||||
type="button"
|
||||
role="menuitem"
|
||||
aria-haspopup="true"
|
||||
aria-expanded="false"
|
||||
aria-controls="86470407-3128-4455-8684-e3e947f97f7f"
|
||||
data-concept-uuid="86470407-3128-4455-8684-e3e947f97f7f"
|
||||
data-rank-size="32"
|
||||
data-rank-thickness="8"
|
||||
data-rank-value="1.0"
|
||||
data-summary="true"
|
||||
data-summary-url="/de/concepts/summary/">
|
||||
<span class="concept-wrapper" title="100%">
|
||||
<span class="rank"></span>
|
||||
<span class="concept">Impulse Response</span>
|
||||
<span class="thesauri">Engineering</span>
|
||||
<span class="value sr-only">100%</span>
|
||||
</span>
|
||||
</button>
|
||||
|
||||
<div class="dropdown-container"></div>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="concept-badge-large-container dropdown-overflow" role="none">
|
||||
|
||||
<button tabindex="0"
|
||||
class="concept-badge-large dropdown-toggle"
|
||||
type="button"
|
||||
role="menuitem"
|
||||
aria-haspopup="true"
|
||||
aria-expanded="false"
|
||||
aria-controls="4741b98b-483a-4788-88a9-4952bd6bb3cd"
|
||||
data-concept-uuid="4741b98b-483a-4788-88a9-4952bd6bb3cd"
|
||||
data-rank-size="32"
|
||||
data-rank-thickness="8"
|
||||
data-rank-value="1.0"
|
||||
data-summary="true"
|
||||
data-summary-url="/de/concepts/summary/">
|
||||
<span class="concept-wrapper" title="100%">
|
||||
<span class="rank"></span>
|
||||
<span class="concept">Listening Room</span>
|
||||
<span class="thesauri">Engineering</span>
|
||||
<span class="value sr-only">100%</span>
|
||||
</span>
|
||||
</button>
|
||||
|
||||
<div class="dropdown-container"></div>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="concept-badge-large-container dropdown-overflow" role="none">
|
||||
|
||||
<button tabindex="0"
|
||||
class="concept-badge-large dropdown-toggle"
|
||||
type="button"
|
||||
role="menuitem"
|
||||
aria-haspopup="true"
|
||||
aria-expanded="false"
|
||||
aria-controls="ae39eb57-add6-4a50-9674-b1de7b887840"
|
||||
data-concept-uuid="ae39eb57-add6-4a50-9674-b1de7b887840"
|
||||
data-rank-size="32"
|
||||
data-rank-thickness="8"
|
||||
data-rank-value="1.0"
|
||||
data-summary="true"
|
||||
data-summary-url="/de/concepts/summary/">
|
||||
<span class="concept-wrapper" title="100%">
|
||||
<span class="rank"></span>
|
||||
<span class="concept">Noise Reduction</span>
|
||||
<span class="thesauri">Earth and Planetary Sciences</span>
|
||||
<span class="value sr-only">100%</span>
|
||||
</span>
|
||||
</button>
|
||||
|
||||
<div class="dropdown-container"></div>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="concept-badge-large-container dropdown-overflow" role="none">
|
||||
|
||||
<button tabindex="0"
|
||||
class="concept-badge-large dropdown-toggle"
|
||||
type="button"
|
||||
role="menuitem"
|
||||
aria-haspopup="true"
|
||||
aria-expanded="false"
|
||||
aria-controls="31592306-d0d0-4620-8ba6-638c18aeb985"
|
||||
data-concept-uuid="31592306-d0d0-4620-8ba6-638c18aeb985"
|
||||
data-rank-size="32"
|
||||
data-rank-thickness="8"
|
||||
data-rank-value="0.3333333"
|
||||
data-summary="true"
|
||||
data-summary-url="/de/concepts/summary/">
|
||||
<span class="concept-wrapper" title="33%">
|
||||
<span class="rank"></span>
|
||||
<span class="concept">Equalizer</span>
|
||||
<span class="thesauri">Engineering</span>
|
||||
<span class="value sr-only">33%</span>
|
||||
</span>
|
||||
</button>
|
||||
|
||||
<div class="dropdown-container"></div>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="concept-badge-large-container dropdown-overflow" role="none">
|
||||
|
||||
<button tabindex="0"
|
||||
class="concept-badge-large dropdown-toggle"
|
||||
type="button"
|
||||
role="menuitem"
|
||||
aria-haspopup="true"
|
||||
aria-expanded="false"
|
||||
aria-controls="a773d085-8c98-44f2-b25a-bb2a9ed59a30"
|
||||
data-concept-uuid="a773d085-8c98-44f2-b25a-bb2a9ed59a30"
|
||||
data-rank-size="32"
|
||||
data-rank-thickness="8"
|
||||
data-rank-value="0.1666667"
|
||||
data-summary="true"
|
||||
data-summary-url="/de/concepts/summary/">
|
||||
<span class="concept-wrapper" title="16%">
|
||||
<span class="rank"></span>
|
||||
<span class="concept">Filtration</span>
|
||||
<span class="thesauri">Engineering</span>
|
||||
<span class="value sr-only">16%</span>
|
||||
</span>
|
||||
</button>
|
||||
|
||||
<div class="dropdown-container"></div>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="concept-badge-large-container dropdown-overflow" role="none">
|
||||
|
||||
<button tabindex="0"
|
||||
class="concept-badge-large dropdown-toggle"
|
||||
type="button"
|
||||
role="menuitem"
|
||||
aria-haspopup="true"
|
||||
aria-expanded="false"
|
||||
aria-controls="d14b1a5b-d90e-4f0a-bc73-bb45c75b9f3b"
|
||||
data-concept-uuid="d14b1a5b-d90e-4f0a-bc73-bb45c75b9f3b"
|
||||
data-rank-size="32"
|
||||
data-rank-thickness="8"
|
||||
data-rank-value="0.1666667"
|
||||
data-summary="true"
|
||||
data-summary-url="/de/concepts/summary/">
|
||||
<span class="concept-wrapper" title="16%">
|
||||
<span class="rank"></span>
|
||||
<span class="concept">Adaptive Method</span>
|
||||
<span class="thesauri">Engineering</span>
|
||||
<span class="value sr-only">16%</span>
|
||||
</span>
|
||||
</button>
|
||||
|
||||
<div class="dropdown-container"></div>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
<div class="footer">
|
||||
<div class="view-all align-center">
|
||||
<a class="portal_link btn-primary btn-large btn-content-link" href="/de/publications/system-identification-for-multi-channel-listening-room-compensati/fingerprints/"><span>
|
||||
Vollständigen Fingerprint anzeigen
|
||||
<i class="icon icon-right-open-mini" aria-hidden="true"></i>
|
||||
</span></a>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section class="page-section cite">
|
||||
<div class="container">
|
||||
<section class="publication-cite" aria-labelledby="publications-cite-title">
|
||||
<h2 class="subheader" id="publications-cite-title">Zitieren</h2>
|
||||
<ul class="tabs" role="tablist" aria-label="Zitieren">
|
||||
|
||||
<li class="selected" role="tab" id="tab-0" aria-controls="cite-apa" tabindex="-1" aria-selected="false" >
|
||||
APA
|
||||
</li>
|
||||
|
||||
<li class="" role="tab" id="tab-1" aria-controls="cite-author" tabindex="-1" aria-selected="false" >
|
||||
Author
|
||||
</li>
|
||||
|
||||
<li class="" role="tab" id="tab-2" aria-controls="cite-BIBTEX" tabindex="-1" aria-selected="false" >
|
||||
BIBTEX
|
||||
</li>
|
||||
|
||||
<li class="" role="tab" id="tab-3" aria-controls="cite-harvard" tabindex="-1" aria-selected="false" >
|
||||
Harvard
|
||||
</li>
|
||||
|
||||
<li class="" role="tab" id="tab-4" aria-controls="cite-standard" tabindex="-1" aria-selected="false" >
|
||||
Standard
|
||||
</li>
|
||||
|
||||
<li class="" role="tab" id="tab-5" aria-controls="cite-RIS" tabindex="-1" aria-selected="false" >
|
||||
RIS
|
||||
</li>
|
||||
|
||||
<li class="" role="tab" id="tab-6" aria-controls="cite-vancouver" tabindex="-1" aria-selected="false" >
|
||||
Vancouver
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
<div class="tab-container">
|
||||
|
||||
<div id="cite-apa" class="tab selected" role="tabpanel" aria-labelledby="tab-0" tabindex="-1" aria-hidden="true">
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_apa rendering_contributiontobookanthology rendering_apa rendering_contributiontobookanthology_apa">Goetze, S., Kallinger, M.<a rel="Person" href="https://research.uni-luebeck.de/de/persons/alfred-mertins/" class="link"><span>, Mertins, A.</span></a>, & Kammeyer, K. (2008). <a rel="ContributionToBookAnthology" href="https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/" class="link"><span>System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller</span></a>. in <em>2008 Hands-Free Speech Communication and Microphone Arrays </em>(S. 224-227). Artikel 4538727 (Hands-free Speech Communication and Microphone Arrays, Proceedings). IEEE. <a onclick="window.open(this.href, '_blank','noopener,noreferrer'); return false;" href="https://doi.org/10.1109/HSCMA.2008.4538727" class="link"><span>https://doi.org/10.1109/HSCMA.2008.4538727</span></a></div>
|
||||
</div>
|
||||
|
||||
<div id="cite-author" class="tab " role="tabpanel" aria-labelledby="tab-1" tabindex="-1" aria-hidden="true">
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_author rendering_contributiontobookanthology rendering_author rendering_contributiontobookanthology_author">Goetze, S. ; Kallinger, M. <a rel="Person" href="https://research.uni-luebeck.de/de/persons/alfred-mertins/" class="link"><span> ; Mertins, A.</span></a> et al. / <a rel="ContributionToBookAnthology" href="https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/" class="link"><span><strong>System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller</strong></span></a>. 2008 Hands-Free Speech Communication and Microphone Arrays. IEEE, 2008. S. 224-227 (Hands-free Speech Communication and Microphone Arrays, Proceedings).</div>
|
||||
</div>
|
||||
|
||||
<div id="cite-BIBTEX" class="tab " role="tabpanel" aria-labelledby="tab-2" tabindex="-1" aria-hidden="true">
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_bibtex rendering_contributiontobookanthology rendering_bibtex rendering_contributiontobookanthology_bibtex"><div>@inproceedings{0c5486f2317d4415922969e27b2a9485,</div><div> title = "System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller",</div><div> abstract = "Modern hands-free telecommunication devices jointly apply several subsystems, e.g. for noise reduction (NR), acoustic echo cancellation (AEC) and listening-room compensation (LRC). In this contribution the combination of an equalizer for listening room compensation and an acoustic echo canceller is analyzed. Inverse filtering of room impulse responses (RIRs) is a challenging task since they are, in general, mixed phase systems having hundreds of zeros inside and outside near the unit circle in the z-domain. Furthermore, a reliable estimate of the RIR which shall be inverted is important. Since RIRs are time-variant due to possible changes of the acoustic environment, they have to be identified adaptively. If an AEC (or any other adaptive method) is used to identify the time variant room impulse responses the estimate's distance to the real RIRs may be too high for a satisfying equalization, especially in periods of initial convergence of the AEC or after RIR changes. Therefore, we propose to estimate the convergence state of the AEC and to incorporate this knowledge into the equalizer design.",</div><div> author = "S. Goetze and M. Kallinger and A. Mertins and K. Kammeyer",</div><div> year = "2008",</div><div> month = may,</div><div> day = "1",</div><div> doi = "10.1109/HSCMA.2008.4538727",</div><div> language = "English",</div><div> isbn = "978-1-4244-2337-8",</div><div> series = "Hands-free Speech Communication and Microphone Arrays, Proceedings",</div><div> publisher = "IEEE",</div><div> pages = "224--227",</div><div> booktitle = "2008 Hands-Free Speech Communication and Microphone Arrays",</div><div> note = "2008 Hands-free Speech Communication and Microphone Arrays, HSCMA 2008 ; Conference date: 06-05-2008 Through 08-05-2008",</div><p>}</p></div>
|
||||
</div>
|
||||
|
||||
<div id="cite-harvard" class="tab " role="tabpanel" aria-labelledby="tab-3" tabindex="-1" aria-hidden="true">
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_harvard rendering_contributiontobookanthology rendering_harvard rendering_contributiontobookanthology_harvard">Goetze, S, Kallinger, M<a rel="Person" href="https://research.uni-luebeck.de/de/persons/alfred-mertins/" class="link"><span>, Mertins, A</span></a> & Kammeyer, K 2008, <a rel="ContributionToBookAnthology" href="https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/" class="link harvard_title"><span>System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller</span></a>. in <em>2008 Hands-Free Speech Communication and Microphone Arrays.</em>, 4538727, Hands-free Speech Communication and Microphone Arrays, Proceedings, IEEE, S. 224-227, 2008 Hands-free Speech Communication and Microphone Arrays, Trento, Italien, <span class="date">06.05.08</span>. <a onclick="window.open(this.href, '_blank','noopener,noreferrer'); return false;" href="https://doi.org/10.1109/HSCMA.2008.4538727" class="link"><span>https://doi.org/10.1109/HSCMA.2008.4538727</span></a></div>
|
||||
</div>
|
||||
|
||||
<div id="cite-standard" class="tab " role="tabpanel" aria-labelledby="tab-4" tabindex="-1" aria-hidden="true">
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_standard rendering_contributiontobookanthology rendering_standard rendering_contributiontobookanthology_standard"><a rel="ContributionToBookAnthology" href="https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/" class="link"><span><strong>System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller.</strong></span></a> / Goetze, S.; Kallinger, M.<a rel="Person" href="https://research.uni-luebeck.de/de/persons/alfred-mertins/" class="link"><span>; Mertins, A.</span></a> et al. <br />2008 Hands-Free Speech Communication and Microphone Arrays. IEEE, 2008. S. 224-227 4538727 (Hands-free Speech Communication and Microphone Arrays, Proceedings).<p class="type"><span class="type_family">Publikation<span class="type_family_sep">: </span></span><span class="type_classification_parent">Kapitel in Büchern/Berichten/Konferenzbänden<span class="type_parent_sep"> › </span></span><span class="type_classification">Konferenzbeitrag</span><span class="type_classification"><span class="type_parent_sep"> › </span>Begutachtung</span></p></div>
|
||||
</div>
|
||||
|
||||
<div id="cite-RIS" class="tab " role="tabpanel" aria-labelledby="tab-5" tabindex="-1" aria-hidden="true">
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_ris rendering_contributiontobookanthology rendering_ris rendering_contributiontobookanthology_ris"><p>TY - GEN</p><p>T1 - System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller</p><p>AU - Goetze, S.</p><p>AU - Kallinger, M.</p><p>AU - Mertins, A.</p><p>AU - Kammeyer, K.</p><p>N1 - Conference code: 73345</p><p>PY - 2008/5/1</p><p>Y1 - 2008/5/1</p><p>N2 - Modern hands-free telecommunication devices jointly apply several subsystems, e.g. for noise reduction (NR), acoustic echo cancellation (AEC) and listening-room compensation (LRC). In this contribution the combination of an equalizer for listening room compensation and an acoustic echo canceller is analyzed. Inverse filtering of room impulse responses (RIRs) is a challenging task since they are, in general, mixed phase systems having hundreds of zeros inside and outside near the unit circle in the z-domain. Furthermore, a reliable estimate of the RIR which shall be inverted is important. Since RIRs are time-variant due to possible changes of the acoustic environment, they have to be identified adaptively. If an AEC (or any other adaptive method) is used to identify the time variant room impulse responses the estimate's distance to the real RIRs may be too high for a satisfying equalization, especially in periods of initial convergence of the AEC or after RIR changes. Therefore, we propose to estimate the convergence state of the AEC and to incorporate this knowledge into the equalizer design.</p><p>AB - Modern hands-free telecommunication devices jointly apply several subsystems, e.g. for noise reduction (NR), acoustic echo cancellation (AEC) and listening-room compensation (LRC). In this contribution the combination of an equalizer for listening room compensation and an acoustic echo canceller is analyzed. Inverse filtering of room impulse responses (RIRs) is a challenging task since they are, in general, mixed phase systems having hundreds of zeros inside and outside near the unit circle in the z-domain. Furthermore, a reliable estimate of the RIR which shall be inverted is important. Since RIRs are time-variant due to possible changes of the acoustic environment, they have to be identified adaptively. If an AEC (or any other adaptive method) is used to identify the time variant room impulse responses the estimate's distance to the real RIRs may be too high for a satisfying equalization, especially in periods of initial convergence of the AEC or after RIR changes. Therefore, we propose to estimate the convergence state of the AEC and to incorporate this knowledge into the equalizer design.</p><p>UR - https://www.scopus.com/pages/publications/50449102952</p><p>U2 - 10.1109/HSCMA.2008.4538727</p><p>DO - 10.1109/HSCMA.2008.4538727</p><p>M3 - Conference contribution</p><p>SN - 978-1-4244-2337-8</p><p>T3 - Hands-free Speech Communication and Microphone Arrays, Proceedings</p><p>SP - 224</p><p>EP - 227</p><p>BT - 2008 Hands-Free Speech Communication and Microphone Arrays</p><p>PB - IEEE</p><p>T2 - 2008 Hands-free Speech Communication and Microphone Arrays</p><p>Y2 - 6 May 2008 through 8 May 2008</p><p>ER - </p></div>
|
||||
</div>
|
||||
|
||||
<div id="cite-vancouver" class="tab " role="tabpanel" aria-labelledby="tab-6" tabindex="-1" aria-hidden="true">
|
||||
<div class="rendering rendering_researchoutput rendering_researchoutput_vancouver rendering_contributiontobookanthology rendering_vancouver rendering_contributiontobookanthology_vancouver">Goetze S, Kallinger M<a rel="Person" href="https://research.uni-luebeck.de/de/persons/alfred-mertins/" class="link"><span>, Mertins A</span></a>, Kammeyer K. <span class="title"><a rel="ContributionToBookAnthology" href="https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/" class="link"><span>System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller</span></a></span>. in 2008 Hands-Free Speech Communication and Microphone Arrays. IEEE. 2008. S. 224-227. 4538727. (Hands-free Speech Communication and Microphone Arrays, Proceedings). doi: 10.1109/HSCMA.2008.4538727</div>
|
||||
</div>
|
||||
|
||||
|
||||
<script>
|
||||
$(function() {
|
||||
$(".tab-container .rendering a:not([href^='http://dx.doi.org'])").contents().unwrap();
|
||||
});
|
||||
</script>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<script>
|
||||
$(function () {
|
||||
if($('.publication-sidebar').length > 0){
|
||||
if($('a.document-link').length === 0){
|
||||
$(".metrics-top").addClass("one");
|
||||
$('div.metrics-downloads').remove();
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
function plumXError(e) {
|
||||
$(e).closest('.metric-plum-x').closest('.metrics-bottom').addClass('one');
|
||||
$(e).closest('.metric-plum-x').remove();
|
||||
}
|
||||
$('div.altmetric-embed').on('altmetric:hide', function () {
|
||||
$(this).closest('.metric-altmetric').closest('.metrics-bottom').addClass('one');
|
||||
$(this).closest('.metric-altmetric').remove();
|
||||
});
|
||||
</script>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<div class="share-box">
|
||||
<ul>
|
||||
<li class="icon-share"></li>
|
||||
<li><a aria-label="Auf Facebook teilen"
|
||||
href="https://www.facebook.com/sharer.php?u=https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/%3Futm_source%3Dfacebook%26utm_medium%3Dsocial%26utm_campaign%3Dsharelink%26doi%3D10.1109%2FHSCMA.2008.4538727&p[summary]=Schauen+Sie+sich+diese+Publikationen+an+unter+Universit%C3%A4t+zu+L%C3%BCbeck%3A+System+Identification+for+Multi-Channel+Listening-Room+Compensation+Using+an+Acoustic+Echo+Canceller"
|
||||
class="icon-facebook"
|
||||
onclick="window.location.origin('https://www.facebook.com/sharer.php?u=https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/%3Futm_source%3Dfacebook%26utm_medium%3Dsocial%26utm_campaign%3Dsharelink%26doi%3D10.1109%2FHSCMA.2008.4538727&p[summary]=Schauen+Sie+sich+diese+Publikationen+an+unter+Universit%C3%A4t+zu+L%C3%BCbeck%3A+System+Identification+for+Multi-Channel+Listening-Room+Compensation+Using+an+Acoustic+Echo+Canceller', 'Facebook', 'width=600,height=600,scrollbars=no,resizable=no');
|
||||
return false;" target="popup"></a></li>
|
||||
<li><a aria-label="Auf Twitter teilen"
|
||||
href="https://twitter.com/intent/tweet?url=https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/%3Futm_source%3Dtwitter%26utm_medium%3Dsocial%26utm_campaign%3Dsharelink%26doi%3D10.1109%2FHSCMA.2008.4538727&text=Schauen+Sie+sich+diese+Publikationen+an+unter+Universit%C3%A4t+zu+L%C3%BCbeck%3A+System+Identification+for+Multi-Channel+Listening-Room+Compensation+Using+an+Acoustic+Echo+Canceller"
|
||||
onclick="window.location.origin('https://twitter.com/intent/tweet?url=https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/%3Futm_source%3Dtwitter%26utm_medium%3Dsocial%26utm_campaign%3Dsharelink%26doi%3D10.1109%2FHSCMA.2008.4538727&text=Schauen+Sie+sich+diese+Publikationen+an+unter+Universit%C3%A4t+zu+L%C3%BCbeck%3A+System+Identification+for+Multi-Channel+Listening-Room+Compensation+Using+an+Acoustic+Echo+Canceller doi%3A10.1109%2FHSCMA.2008.4538727', 'Twitter', 'width=600,height=600,scrollbars=no,resizable=no');
|
||||
return false;" class="twitter" target="popup"><svg xmlns="http://www.w3.org/2000/svg" width="22" height="22" viewBox="0 0 300 271">
|
||||
|
||||
<path d="m236 0h46l-101 115 118 156h-92.6l-72.5-94.8-83 94.8h-46l107-123-113-148h94.9l65.5 86.6zm-16.1 244h25.5l-165-218h-27.4z"></path>
|
||||
</svg></a></li>
|
||||
<li><a aria-label="Auf LinkedIn teilen"
|
||||
href="https://www.linkedin.com/shareArticle?mini=true&url=https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/%3Futm_source%3Dlinkedin%26utm_medium%3Dsocial%26utm_campaign%3Dsharelink%26doi%3D10.1109%2FHSCMA.2008.4538727&summary=Schauen+Sie+sich+diese+Publikationen+an+unter+Universit%C3%A4t+zu+L%C3%BCbeck%3A+System+Identification+for+Multi-Channel+Listening-Room+Compensation+Using+an+Acoustic+Echo+Canceller"
|
||||
onclick="window.location.origin('https://www.linkedin.com/shareArticle?mini=true&url=https://research.uni-luebeck.de/de/publications/system-identification-for-multi-channel-listening-room-compensati/%3Futm_source%3Dlinkedin%26utm_medium%3Dsocial%26utm_campaign%3Dsharelink%26doi%3D10.1109%2FHSCMA.2008.4538727&summary=Schauen+Sie+sich+diese+Publikationen+an+unter+Universit%C3%A4t+zu+L%C3%BCbeck%3A+System+Identification+for+Multi-Channel+Listening-Room+Compensation+Using+an+Acoustic+Echo+Canceller doi%3A10.1109%2FHSCMA.2008.4538727', 'LinkedIn', 'width=600,height=600,scrollbars=no,resizable=no');
|
||||
return false;" class="icon-linkedin" target="popup"></a></li>
|
||||
<li><a aria-label="Per E-Mail teilen" href="/cdn-cgi/l/email-protection#bb84c8ced9d1ded8cf86e8c2c8cfded69e898bf2dfded5cfd2ddd2d8dacfd2d4d59e898bddd4c99e898bf6ced7cfd296f8d3dad5d5ded79e898bf7d2c8cfded5d2d5dc96e9d4d4d69e898bf8d4d6cbded5c8dacfd2d4d59e898beec8d2d5dc9e898bdad59e898bfad8d4cec8cfd2d89e898bfed8d3d49e898bf8dad5d8ded7d7dec99dd9d4dfc286e8d8d3daceded59e898be8d2de9e898bc8d2d8d39e898bdfd2dec8de9e898bebced9d7d2d0dacfd2d4d5ded59e898bdad59e898bced5cfdec99e898beed5d2cddec9c8d2cf9ef8889efa8fcf9e898bc1ce9e898bf79ef8889ef9f8d9ded8d09e88fa9e898be8c2c8cfded69e898bf2dfded5cfd2ddd2d8dacfd2d4d59e898bddd4c99e898bf6ced7cfd296f8d3dad5d5ded79e898bf7d2c8cfded5d2d5dc96e9d4d4d69e898bf8d4d6cbded5c8dacfd2d4d59e898beec8d2d5dc9e898bdad59e898bfad8d4cec8cfd2d89e898bfed8d3d49e898bf8dad5d8ded7d7dec99bc79bd3cfcfcbc8819494c9dec8dedac9d8d395ced5d296d7ceded9ded8d095dfde94dfde94cbced9d7d2d8dacfd2d4d5c894c8c2c8cfded696d2dfded5cfd2ddd2d8dacfd2d4d596ddd4c996d6ced7cfd296d8d3dad5d5ded796d7d2c8cfded5d2d5dc96c9d4d4d696d8d4d6cbded5c8dacfd29484cecfd6e4c8d4cec9d8de86ded6dad2d79ddad6cb80cecfd6e4d6dedfd2ced686ded6dad2d79ddad6cb80cecfd6e4d8dad6cbdad2dcd586c8d3dac9ded7d2d5d0" class="icon-email"></a></li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="new-modal-overlay"></div>
|
||||
|
||||
</main>
|
||||
|
||||
<footer id="page-footer" class="main-footer">
|
||||
<div class="container">
|
||||
<div class="footer-wrapper">
|
||||
<div class="content-navigation three-third">
|
||||
<div dir="ltr" class="footer-columns">
|
||||
|
||||
<div class="footer-column">
|
||||
<p>
|
||||
Angetrieben von <a href="http://www.elsevier.com/online-tools/research-intelligence/products-and-services/pure">Pure</a>, <a href="http://www.scopus.com/">Scopus</a> & <a href="https://www.elsevier.com/products/elsevier-fingerprint-engine">Elsevier Fingerprint Engine™</a>
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Alle Inhalte auf dieser Website: Copyright © 2026 Universität zu Lübeck, zugehörige Lizenzgebende und Mitwirkende. Alle Rechte vorbehalten, einschließlich der Rechte für Text- und Data-Mining, KI-Training und ähnliche Technologien. Für alle Open Access-Inhalte gelten die entsprechenden Lizenzbedingungen.
|
||||
</p>
|
||||
|
||||
<p>Wir verwenden Cookies, um unseren Service bereitzustellen und zu verbessern und um Inhalte anzupassen. Wenn Sie fortfahren, stimmen Sie der <a href="/de/about/cookies/">Verwendung von Cookies</a> zu.</p>
|
||||
<p><button id="ot-sdk-btn" class="ot-sdk-show-settings"></button></p>
|
||||
|
||||
<p></p>
|
||||
</div>
|
||||
|
||||
<div class="footer-column">
|
||||
|
||||
|
||||
|
||||
<p><a href="/de/web-accessibility/" rel="nofollow" target="_blank">Über die Web-Zugänglichkeit <span class="sr-only"> Link öffnet neuen Tab</span></a></p>
|
||||
|
||||
|
||||
<p>
|
||||
|
||||
</p>
|
||||
|
||||
<p>
|
||||
|
||||
</p>
|
||||
|
||||
<p>
|
||||
<a href="https://elsevier.responsibledisclosure.com/hc/en-us/requests/new" target="_blank" rel="nofollow">Sicherheitslücke melden <span class="sr-only"> Link öffnet neuen Tab</span></a>
|
||||
</p>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div id="back-to-top">
|
||||
<a href="#" class="back-to-top" aria-label="Zum Anfang"><i class="icon icon-up-open"></i></a>
|
||||
</div>
|
||||
</footer>
|
||||
|
||||
|
||||
<script data-cfasync="false" src="/cdn-cgi/scripts/5c5dd728/cloudflare-static/email-decode.min.js"></script><script src="//d1bxh8uas1mnw7.cloudfront.net/assets/embed.js"></script>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
||||
Binary file not shown.
|
|
@ -0,0 +1,88 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta
|
||||
http-equiv="Content-Security-Policy"
|
||||
content="default-src 'self'; img-src 'self' data:; media-src 'self' data:; object-src 'none'; style-src 'self' 'sha256-o4vzfmmUENEg4chMjjRP9EuW9ucGnGIGVdbl8d0SHQQ='; script-src 'self' 'sha256-a9bHdQGvRzDwDVzx8m+Rzw+0FHZad8L0zjtBwkxOIz4=';"
|
||||
/>
|
||||
<link
|
||||
href="/_fs-ch-1T1wmsGaOgGaSxcX/assets/inter-var.woff2"
|
||||
rel="preload"
|
||||
as="font"
|
||||
type="font/woff2"
|
||||
crossorigin
|
||||
/>
|
||||
<link href="/_fs-ch-1T1wmsGaOgGaSxcX/assets/styles.css" rel="stylesheet" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>Client Challenge</title>
|
||||
<style>
|
||||
#loading-error {
|
||||
font-size: 16px;
|
||||
font-family: 'Inter', sans-serif;
|
||||
margin-top: 10px;
|
||||
margin-left: 10px;
|
||||
display: none;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<noscript>
|
||||
<div class="noscript-container">
|
||||
<div class="noscript-content">
|
||||
<img
|
||||
src="/_fs-ch-1T1wmsGaOgGaSxcX/assets/errorIcon.svg"
|
||||
alt=""
|
||||
role="presentation"
|
||||
class="error-icon"
|
||||
/>
|
||||
<span class="noscript-span"
|
||||
>JavaScript is disabled in your browser.</span
|
||||
>
|
||||
<p>Please enable JavaScript to proceed.</p>
|
||||
</div>
|
||||
</div>
|
||||
</noscript>
|
||||
<div id="loading-error" role="alert" aria-live="polite">
|
||||
A required part of this site couldn’t load. This may be due to a browser
|
||||
extension, network issues, or browser settings. Please check your
|
||||
connection, disable any ad blockers, or try using a different browser.
|
||||
</div>
|
||||
<script>
|
||||
function loadScript(src) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const script = document.createElement('script');
|
||||
script.onload = resolve;
|
||||
script.onerror = (event) => {
|
||||
console.error('Script load error event:', event);
|
||||
document.getElementById('loading-error').style.display = 'block';
|
||||
reject(
|
||||
new Error(
|
||||
`Failed to load script: ${src}, Please contact the service administrator.`
|
||||
)
|
||||
);
|
||||
};
|
||||
script.src = src;
|
||||
document.body.appendChild(script);
|
||||
});
|
||||
}
|
||||
|
||||
loadScript('/_fs-ch-1T1wmsGaOgGaSxcX/errors.js')
|
||||
.then(() => {
|
||||
const script = document.createElement('script');
|
||||
script.src = '/_fs-ch-1T1wmsGaOgGaSxcX/script.js?reload=true';
|
||||
script.onerror = (event) => {
|
||||
console.error('Script load error event:', event);
|
||||
const errorMsg = new Error(
|
||||
`Failed to load script: ${script.src}. Please contact the service administrator.`
|
||||
);
|
||||
console.error(errorMsg);
|
||||
handleScriptError();
|
||||
};
|
||||
document.body.appendChild(script);
|
||||
})
|
||||
.catch((error) => {
|
||||
console.error(error);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
Binary file not shown.
Binary file not shown.
96
research/feedback-calibration/summary.md
Normal file
96
research/feedback-calibration/summary.md
Normal file
|
|
@ -0,0 +1,96 @@
|
|||
# Feedback As Continuous Calibration
|
||||
|
||||
## Objective
|
||||
|
||||
Treat speaker-to-microphone feedback as a useful measurement stream, not merely noise to suppress, so the system can continuously learn the playback-room-microphone transfer paths while running.
|
||||
|
||||
## Current Mechanism
|
||||
|
||||
The playback calibration note treats speaker measurements as an explicit calibration pass. The live adaptive sync note warns that speaker bleed can corrupt mic sync/localization. Both are true, but incomplete: because the emitted speaker signal is known, the bleed is also an online system-identification signal.
|
||||
|
||||
## Main Finding
|
||||
|
||||
This is the territory of acoustic echo cancellation, adaptive feedback cancellation, and online acoustic system identification. The useful model is:
|
||||
|
||||
```text
|
||||
mic_signal = live_sources + speaker_output convolved with room/speaker/mic paths + noise
|
||||
```
|
||||
|
||||
If the system knows `speaker_output`, an adaptive filter can estimate the transfer path from each speaker output channel to each microphone. Goetze et al. explicitly connect acoustic echo cancellation to listening-room compensation and note that room impulse responses are time-varying and must be identified adaptively, while also warning that equalizer design should account for the echo canceller's convergence state [Goetze 2008]. That warning matters: a half-converged estimator is not truth; it is a drunk ruler.
|
||||
|
||||
Nikunen and Virtanen describe online estimation of time-varying room impulse responses when isolated source signals are available, including playback material output through loudspeakers in a live performance recorded for 3D spatial audio [Nikunen/Virtanen 2018]. That is nearly our shape: known synth/playback streams, observed far-field mic mixtures, and spatial reconstruction.
|
||||
|
||||
Haubner et al. frame online acoustic impulse response estimation as a Kalman/system-identification problem and target faster convergence under correlated excitation and interfering noise [Haubner 2021]. Multichannel AEC work, including Apple ML's MIMO least-squares framing, addresses the harder version where multiple speakers and multiple microphones are coupled and "double-talk" or near-end sound is present [Apple 2020].
|
||||
|
||||
## Required Ownership
|
||||
|
||||
Feedback mining should be its own subsystem:
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["Known speaker/render output"] --> B["Echo path estimator"]
|
||||
C["Microphone returns"] --> B
|
||||
B --> D["Speaker->mic transfer matrix H_sm"]
|
||||
D --> E["Echo prediction"]
|
||||
C --> F["Live source estimator"]
|
||||
E --> F
|
||||
D --> G["Playback calibration updates"]
|
||||
D --> H["Sync/localization confidence gates"]
|
||||
F --> I["Aligned capture/spatial scene"]
|
||||
```
|
||||
|
||||
It should not be hidden inside the drift loop or the ambisonic encoder. The transfer matrix `H_sm` owns "what our speakers currently sound like at our microphones." The clock loop owns sampling-rate mismatch. The localization loop owns external source position. The decoder owns speaker rendering. Four authorities. No soup.
|
||||
|
||||
## Design Implications
|
||||
|
||||
- Continuous feedback is a calibration signal when the outgoing audio is known and time-aligned.
|
||||
- It is also an interference signal for voice capture, so the system should predict and optionally subtract or downweight it.
|
||||
- Adaptation must be gated during strong unknown near-field speech, heavy clipping, nonlinear speaker behavior, or low coherence.
|
||||
- Each speaker-to-mic path is a filter, not just one delay. Early taps are useful for direct-path timing and geometry; later taps describe room reflections.
|
||||
- In multichannel playback, the estimator is MIMO: every speaker can leak into every mic.
|
||||
- The estimator should publish confidence, convergence, residual error, and last-change time. Do not let downstream systems treat a stale or diverging filter as a fact.
|
||||
|
||||
## Candidate V1
|
||||
|
||||
1. Start offline/controlled:
|
||||
- play one speaker at a time with chirp/noise
|
||||
- estimate fixed speaker-to-reference-mic impulse responses
|
||||
- store speaker delay, gain, polarity, and early reflection sketch
|
||||
|
||||
2. Add online passive tracking:
|
||||
- feed known speaker output and mic input into an adaptive filter
|
||||
- start with two paths: left speaker -> reference mic, right speaker -> reference mic
|
||||
- estimate residual echo and convergence
|
||||
|
||||
3. Add live gating:
|
||||
- freeze or slow adaptation during strong unknown speech
|
||||
- adapt faster during known probe/noise windows or synth-only output
|
||||
- reject updates when residual/error spikes indicate nonlinearity or movement
|
||||
|
||||
4. Scale to MIMO:
|
||||
- speaker channels x microphone channels
|
||||
- use the resulting transfer matrix to improve playback correction, echo prediction, and confidence in mic localization
|
||||
|
||||
## How It Helps AquaSynth
|
||||
|
||||
- The system can tell whether a synth voice rendered to "front left" actually arrives front-left-ish at the mic array.
|
||||
- Speaker leakage can be subtracted or modeled before estimating external voice position.
|
||||
- The room response can update slowly as people move, doors open, or speaker volume changes.
|
||||
- Calibration pings become optional accelerators rather than the only source of truth.
|
||||
|
||||
## Risks
|
||||
|
||||
- Adaptive echo filters can diverge under double-talk unless gated or made robust.
|
||||
- Loudspeakers are nonlinear at high levels; a linear impulse response will not explain distortion.
|
||||
- Correlated stereo/ambisonic speaker feeds make MIMO echo identification harder because the system has trouble deciding which speaker caused which mic energy.
|
||||
- If we use the estimator to aggressively cancel feedback, we may remove useful spatial ambience. The goal is not sterile cancellation; it is intelligible ownership of the signal.
|
||||
|
||||
## Citations
|
||||
|
||||
- [Goetze 2008] S. Goetze, M. Kallinger, A. Mertins, and K. Kammeyer, "System Identification for Multi-Channel Listening-Room Compensation Using an Acoustic Echo Canceller." Mirror: `mirrors/goetze-2008-aec-listening-room-compensation.html`.
|
||||
- [Lindstrom 2007] F. Lindstrom, C. Schueldt, and I. Claesson, "Efficient Multichannel NLMS Implementation for Acoustic Echo Cancellation." Mirror: `mirrors/lindstrom-2007-efficient-multichannel-nlms.html`.
|
||||
- [Valin 2016] J.-M. Valin, "A New Robust Frequency Domain Echo Canceller With Closed-Loop Learning Rate Adaptation." Mirror: `mirrors/valin-2016-frequency-domain-echo-canceller-learning-rate.pdf`.
|
||||
- [EUSIPCO 2021] "Double-Talk Robust Acoustic Echo Cancellation." Mirror: `mirrors/eusipco-2021-double-talk-robust-aec.pdf`.
|
||||
- [Apple 2020] Apple Machine Learning Research, "Double-talk Robust Multichannel Acoustic Echo Cancellation Using Least Squares MIMO Adaptive Filtering." Mirror: `mirrors/apple-mimo-adaptive-echo-cancellation.html`.
|
||||
- [Nikunen/Virtanen 2018] J. Nikunen and T. Virtanen, "Estimation of Time-Varying Room Impulse Responses of Multiple Sound Sources from Observed Mixture and Isolated Source Signals." Mirror: `mirrors/nikunen-virtanen-2018-time-varying-rir.pdf`.
|
||||
- [Haubner 2021] T. Haubner, A. Brendel, and W. Kellermann, "Online Acoustic System Identification Exploiting Kalman Filtering and an Adaptive Impulse Response Subspace Model." Mirror: `mirrors/haubner-2021-online-acoustic-system-identification.pdf`.
|
||||
51
research/gpu-fusion-splatting/bibliography.md
Normal file
51
research/gpu-fusion-splatting/bibliography.md
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
# Bibliography: GPU Fusion And Live Splat Rendering
|
||||
|
||||
Mirrored copies live under `research/gpu-fusion-splatting/mirrors/`.
|
||||
|
||||
## CUDA / GPU Runtime
|
||||
|
||||
- [NVIDIA CUDA C++ Programming Guide](https://docs.nvidia.com/cuda/cuda-programming-guide/pdf/cuda-programming-guide.pdf)
|
||||
Mirror: `mirrors/nvidia-cuda-programming-guide.pdf`
|
||||
|
||||
- [NVIDIA CUDA C++ Programming Guide, asynchronous/concurrent execution](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#asynchronous-concurrent-execution)
|
||||
Mirror: `mirrors/nvidia-cuda-async-concurrent-execution.html`
|
||||
|
||||
- [NVIDIA Vision Programming Interface](https://docs.nvidia.com/vpi/)
|
||||
Mirror: `mirrors/nvidia-vpi-main.html`
|
||||
|
||||
- [NVIDIA Optical Flow SDK Programming Guide](https://docs.nvidia.com/video-technologies/optical-flow-sdk/pdf/NVOFA_Programming_Guide.pdf)
|
||||
Mirror: `mirrors/nvidia-optical-flow-programming-guide.pdf`
|
||||
|
||||
- [OpenCV CUDA-accelerated Computer Vision module](https://docs.opencv.org/3.4/d1/d1e/group__cuda.html)
|
||||
Mirror: `mirrors/opencv-cuda-module.html`
|
||||
|
||||
## Gaussian Splatting Implementations
|
||||
|
||||
- [GraphDeco/Inria, `gaussian-splatting`](https://github.com/graphdeco-inria/gaussian-splatting)
|
||||
Mirrors: `mirrors/graphdeco-gaussian-splatting-github.html`, `mirrors/graphdeco-gaussian-splatting-readme.md`
|
||||
|
||||
- [NVIDIA Developer Blog, "Real-Time GPU-Accelerated Gaussian Splatting with NVIDIA DesignWorks Sample vk_gaussian_splatting"](https://developer.nvidia.com/blog/real-time-gpu-accelerated-gaussian-splatting-with-nvidia-designworks-sample-vk_gaussian_splatting/)
|
||||
Mirror: `mirrors/nvidia-vk-gaussian-splatting-blog.html`
|
||||
|
||||
- [gsplat documentation](https://docs.gsplat.studio/main/)
|
||||
Mirror: `mirrors/gsplat-docs.html`
|
||||
|
||||
- [Nerfstudio Project, `gsplat`](https://github.com/nerfstudio-project/gsplat)
|
||||
Mirror: `mirrors/gsplat-github.html`
|
||||
|
||||
- M. Ye et al., ["gsplat: An Open-Source Library for Gaussian Splatting"](https://jmlr.org/papers/volume26/24-1476/24-1476.pdf), JMLR, 2025.
|
||||
Mirror: `mirrors/ye-2025-gsplat.pdf`
|
||||
|
||||
- [Nerfstudio, "Splatfacto"](https://docs.nerf.studio/nerfology/methods/splat.html)
|
||||
Mirror: `mirrors/nerfstudio-splatfacto.html`
|
||||
|
||||
## Gaussian Splatting Papers
|
||||
|
||||
- Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, and George Drettakis, ["3D Gaussian Splatting for Real-Time Radiance Field Rendering"](https://arxiv.org/abs/2308.04079), SIGGRAPH 2023.
|
||||
Mirror: `mirrors/kerbl-2023-3d-gaussian-splatting.pdf`
|
||||
|
||||
- Guanjun Wu et al., ["4D Gaussian Splatting for Real-Time Dynamic Scene Rendering"](https://openaccess.thecvf.com/content/CVPR2024/papers/Wu_4D_Gaussian_Splatting_for_Real-Time_Dynamic_Scene_Rendering_CVPR_2024_paper.pdf), CVPR 2024.
|
||||
Mirror: `mirrors/wu-2024-4d-gaussian-splatting.pdf`
|
||||
|
||||
- Zhexi Peng et al., ["RTG-SLAM: Real-time 3D Reconstruction at Scale using Gaussian Splatting"](https://arxiv.org/abs/2404.19706), SIGGRAPH 2024.
|
||||
Mirror: `mirrors/peng-2024-rtg-slam.pdf`
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,615 @@
|
|||
# 3D Gaussian Splatting for Real-Time Radiance Field Rendering
|
||||
Bernhard Kerbl*, Georgios Kopanas*, Thomas Leimkühler, George Drettakis (* indicates equal contribution)<br>
|
||||
| [Webpage](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) | [Full Paper](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/3d_gaussian_splatting_high.pdf) | [Video](https://youtu.be/T_kXY43VZnk) | [Other GRAPHDECO Publications](http://www-sop.inria.fr/reves/publis/gdindex.php) | [FUNGRAPH project page](https://fungraph.inria.fr) |<br>
|
||||
| [T&T+DB COLMAP (650MB)](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/datasets/input/tandt_db.zip) | [Pre-trained Models (14 GB)](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/datasets/pretrained/models.zip) | [Viewers for Windows (60MB)](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/binaries/viewers.zip) | [Evaluation Images (7 GB)](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/evaluation/images.zip) |<br>
|
||||
|
||||
|
||||
This repository contains the official authors implementation associated with the paper "3D Gaussian Splatting for Real-Time Radiance Field Rendering", which can be found [here](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/). We further provide the reference images used to create the error metrics reported in the paper, as well as recently created, pre-trained models.
|
||||
|
||||
<a href="https://www.inria.fr/"><img height="100" src="assets/logo_inria.png"> </a>
|
||||
<a href="https://univ-cotedazur.eu/"><img height="100" src="assets/logo_uca.png"> </a>
|
||||
<a href="https://www.mpi-inf.mpg.de"><img height="100" src="assets/logo_mpi.png"> </a>
|
||||
<a href="https://team.inria.fr/graphdeco/"> <img style="width:100%;" src="assets/logo_graphdeco.png"></a>
|
||||
|
||||
Abstract: *Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (≥ 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.*
|
||||
|
||||
<section class="section" id="BibTeX">
|
||||
<div class="container is-max-desktop content">
|
||||
<h2 class="title">BibTeX</h2>
|
||||
<pre><code>@Article{kerbl3Dgaussians,
|
||||
author = {Kerbl, Bernhard and Kopanas, Georgios and Leimk{\"u}hler, Thomas and Drettakis, George},
|
||||
title = {3D Gaussian Splatting for Real-Time Radiance Field Rendering},
|
||||
journal = {ACM Transactions on Graphics},
|
||||
number = {4},
|
||||
volume = {42},
|
||||
month = {July},
|
||||
year = {2023},
|
||||
url = {https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/}
|
||||
}</code></pre>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
## Funding and Acknowledgments
|
||||
|
||||
This research was funded by the ERC Advanced grant FUNGRAPH No 788065. The authors are grateful to Adobe for generous donations, the OPAL infrastructure from Université Côte d’Azur and for the HPC resources from GENCI–IDRIS (Grant 2022-AD011013409). The authors thank the anonymous reviewers for their valuable feedback, P. Hedman and A. Tewari for proofreading earlier drafts also T. Müller, A. Yu and S. Fridovich-Keil for helping with the comparisons.
|
||||
|
||||
## NEW FEATURES !
|
||||
|
||||
We have limited resources for maintaining and updating the code. However, we have added a few new features since the original release that are inspired by some of the excellent work many other researchers have been doing on 3DGS. We will be adding other features within the ability of our resources.
|
||||
|
||||
**Update of October 2024**: We integrated [training speed acceleration](#training-speed-acceleration) and made it compatible with [depth regularization](#depth-regularization), [anti-aliasing](#anti-aliasing) and [exposure compensation](#exposure-compensation). We have enhanced the SIBR real time viewer by correcting bugs and adding features in the [Top View](#sibr-top-view) that allows visualization of input and user cameras.
|
||||
|
||||
**Update of Spring 2024**:
|
||||
Orange Labs has kindly added [OpenXR support](#openxr-support) for VR viewing.
|
||||
|
||||
## Step-by-step Tutorial
|
||||
|
||||
Jonathan Stephens made a fantastic step-by-step tutorial for setting up Gaussian Splatting on your machine, along with instructions for creating usable datasets from videos. If the instructions below are too dry for you, go ahead and check it out [here](https://www.youtube.com/watch?v=UXtuigy_wYc).
|
||||
|
||||
## Colab
|
||||
|
||||
User [camenduru](https://github.com/camenduru) was kind enough to provide a Colab template that uses this repo's source (status: August 2023!) for quick and easy access to the method. Please check it out [here](https://github.com/camenduru/gaussian-splatting-colab).
|
||||
|
||||
## Cloning the Repository
|
||||
|
||||
The repository contains submodules, thus please check it out with
|
||||
```shell
|
||||
# SSH
|
||||
git clone git@github.com:graphdeco-inria/gaussian-splatting.git --recursive
|
||||
```
|
||||
or
|
||||
```shell
|
||||
# HTTPS
|
||||
git clone https://github.com/graphdeco-inria/gaussian-splatting --recursive
|
||||
```
|
||||
|
||||
## Overview
|
||||
|
||||
The codebase has 4 main components:
|
||||
- A PyTorch-based optimizer to produce a 3D Gaussian model from SfM inputs
|
||||
- A network viewer that allows to connect to and visualize the optimization process
|
||||
- An OpenGL-based real-time viewer to render trained models in real-time.
|
||||
- A script to help you turn your own images into optimization-ready SfM data sets
|
||||
|
||||
The components have different requirements w.r.t. both hardware and software. They have been tested on Windows 10 and Ubuntu Linux 22.04. Instructions for setting up and running each of them are found in the sections below.
|
||||
|
||||
|
||||
|
||||
|
||||
## Optimizer
|
||||
|
||||
The optimizer uses PyTorch and CUDA extensions in a Python environment to produce trained models.
|
||||
|
||||
### Hardware Requirements
|
||||
|
||||
- CUDA-ready GPU with Compute Capability 7.0+
|
||||
- 24 GB VRAM (to train to paper evaluation quality)
|
||||
- Please see FAQ for smaller VRAM configurations
|
||||
|
||||
### Software Requirements
|
||||
- Conda (recommended for easy setup)
|
||||
- C++ Compiler for PyTorch extensions (we used Visual Studio 2019 for Windows)
|
||||
- CUDA SDK 11 for PyTorch extensions, install *after* Visual Studio (we used 11.8, **known issues with 11.6**)
|
||||
- C++ Compiler and CUDA SDK must be compatible
|
||||
|
||||
### Setup
|
||||
|
||||
#### Local Setup
|
||||
|
||||
Our default, provided install method is based on Conda package and environment management:
|
||||
```shell
|
||||
SET DISTUTILS_USE_SDK=1 # Windows only
|
||||
conda env create --file environment.yml
|
||||
conda activate gaussian_splatting
|
||||
```
|
||||
Please note that this process assumes that you have CUDA SDK **11** installed, not **12**. For modifications, see below.
|
||||
|
||||
Tip: Downloading packages and creating a new environment with Conda can require a significant amount of disk space. By default, Conda will use the main system hard drive. You can avoid this by specifying a different package download location and an environment on a different drive:
|
||||
|
||||
```shell
|
||||
conda config --add pkgs_dirs <Drive>/<pkg_path>
|
||||
conda env create --file environment.yml --prefix <Drive>/<env_path>/gaussian_splatting
|
||||
conda activate <Drive>/<env_path>/gaussian_splatting
|
||||
```
|
||||
|
||||
#### Modifications
|
||||
|
||||
If you can afford the disk space, we recommend using our environment files for setting up a training environment identical to ours. If you want to make modifications, please note that major version changes might affect the results of our method. However, our (limited) experiments suggest that the codebase works just fine inside a more up-to-date environment (Python 3.8, PyTorch 2.0.0, CUDA 12). Make sure to create an environment where PyTorch and its CUDA runtime version match and the installed CUDA SDK has no major version difference with PyTorch's CUDA version.
|
||||
|
||||
#### Known Issues
|
||||
|
||||
Some users experience problems building the submodules on Windows (```cl.exe: File not found``` or similar). Please consider the workaround for this problem from the FAQ.
|
||||
|
||||
### Running
|
||||
|
||||
To run the optimizer, simply use
|
||||
|
||||
```shell
|
||||
python train.py -s <path to COLMAP or NeRF Synthetic dataset>
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary><span style="font-weight: bold;">Command Line Arguments for train.py</span></summary>
|
||||
|
||||
#### --source_path / -s
|
||||
Path to the source directory containing a COLMAP or Synthetic NeRF data set.
|
||||
#### --model_path / -m
|
||||
Path where the trained model should be stored (```output/<random>``` by default).
|
||||
#### --images / -i
|
||||
Alternative subdirectory for COLMAP images (```images``` by default).
|
||||
#### --eval
|
||||
Add this flag to use a MipNeRF360-style training/test split for evaluation.
|
||||
#### --resolution / -r
|
||||
Specifies resolution of the loaded images before training. If provided ```1, 2, 4``` or ```8```, uses original, 1/2, 1/4 or 1/8 resolution, respectively. For all other values, rescales the width to the given number while maintaining image aspect. **If not set and input image width exceeds 1.6K pixels, inputs are automatically rescaled to this target.**
|
||||
#### --data_device
|
||||
Specifies where to put the source image data, ```cuda``` by default, recommended to use ```cpu``` if training on large/high-resolution dataset, will reduce VRAM consumption, but slightly slow down training. Thanks to [HrsPythonix](https://github.com/HrsPythonix).
|
||||
#### --white_background / -w
|
||||
Add this flag to use white background instead of black (default), e.g., for evaluation of NeRF Synthetic dataset.
|
||||
#### --sh_degree
|
||||
Order of spherical harmonics to be used (no larger than 3). ```3``` by default.
|
||||
#### --convert_SHs_python
|
||||
Flag to make pipeline compute forward and backward of SHs with PyTorch instead of ours.
|
||||
#### --convert_cov3D_python
|
||||
Flag to make pipeline compute forward and backward of the 3D covariance with PyTorch instead of ours.
|
||||
#### --debug
|
||||
Enables debug mode if you experience erros. If the rasterizer fails, a ```dump``` file is created that you may forward to us in an issue so we can take a look.
|
||||
#### --debug_from
|
||||
Debugging is **slow**. You may specify an iteration (starting from 0) after which the above debugging becomes active.
|
||||
#### --iterations
|
||||
Number of total iterations to train for, ```30_000``` by default.
|
||||
#### --ip
|
||||
IP to start GUI server on, ```127.0.0.1``` by default.
|
||||
#### --port
|
||||
Port to use for GUI server, ```6009``` by default.
|
||||
#### --test_iterations
|
||||
Space-separated iterations at which the training script computes L1 and PSNR over test set, ```7000 30000``` by default.
|
||||
#### --save_iterations
|
||||
Space-separated iterations at which the training script saves the Gaussian model, ```7000 30000 <iterations>``` by default.
|
||||
#### --checkpoint_iterations
|
||||
Space-separated iterations at which to store a checkpoint for continuing later, saved in the model directory.
|
||||
#### --start_checkpoint
|
||||
Path to a saved checkpoint to continue training from.
|
||||
#### --quiet
|
||||
Flag to omit any text written to standard out pipe.
|
||||
#### --feature_lr
|
||||
Spherical harmonics features learning rate, ```0.0025``` by default.
|
||||
#### --opacity_lr
|
||||
Opacity learning rate, ```0.05``` by default.
|
||||
#### --scaling_lr
|
||||
Scaling learning rate, ```0.005``` by default.
|
||||
#### --rotation_lr
|
||||
Rotation learning rate, ```0.001``` by default.
|
||||
#### --position_lr_max_steps
|
||||
Number of steps (from 0) where position learning rate goes from ```initial``` to ```final```. ```30_000``` by default.
|
||||
#### --position_lr_init
|
||||
Initial 3D position learning rate, ```0.00016``` by default.
|
||||
#### --position_lr_final
|
||||
Final 3D position learning rate, ```0.0000016``` by default.
|
||||
#### --position_lr_delay_mult
|
||||
Position learning rate multiplier (cf. Plenoxels), ```0.01``` by default.
|
||||
#### --densify_from_iter
|
||||
Iteration where densification starts, ```500``` by default.
|
||||
#### --densify_until_iter
|
||||
Iteration where densification stops, ```15_000``` by default.
|
||||
#### --densify_grad_threshold
|
||||
Limit that decides if points should be densified based on 2D position gradient, ```0.0002``` by default.
|
||||
#### --densification_interval
|
||||
How frequently to densify, ```100``` (every 100 iterations) by default.
|
||||
#### --opacity_reset_interval
|
||||
How frequently to reset opacity, ```3_000``` by default.
|
||||
#### --lambda_dssim
|
||||
Influence of SSIM on total loss from 0 to 1, ```0.2``` by default.
|
||||
#### --percent_dense
|
||||
Percentage of scene extent (0--1) a point must exceed to be forcibly densified, ```0.01``` by default.
|
||||
|
||||
</details>
|
||||
<br>
|
||||
|
||||
Note that similar to MipNeRF360, we target images at resolutions in the 1-1.6K pixel range. For convenience, arbitrary-size inputs can be passed and will be automatically resized if their width exceeds 1600 pixels. We recommend to keep this behavior, but you may force training to use your higher-resolution images by setting ```-r 1```.
|
||||
|
||||
The MipNeRF360 scenes are hosted by the paper authors [here](https://jonbarron.info/mipnerf360/). You can find our SfM data sets for Tanks&Temples and Deep Blending [here](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/datasets/input/tandt_db.zip). If you do not provide an output model directory (```-m```), trained models are written to folders with randomized unique names inside the ```output``` directory. At this point, the trained models may be viewed with the real-time viewer (see further below).
|
||||
|
||||
### Evaluation
|
||||
By default, the trained models use all available images in the dataset. To train them while withholding a test set for evaluation, use the ```--eval``` flag. This way, you can render training/test sets and produce error metrics as follows:
|
||||
```shell
|
||||
python train.py -s <path to COLMAP or NeRF Synthetic dataset> --eval # Train with train/test split
|
||||
python render.py -m <path to trained model> # Generate renderings
|
||||
python metrics.py -m <path to trained model> # Compute error metrics on renderings
|
||||
```
|
||||
|
||||
If you want to evaluate our [pre-trained models](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/datasets/pretrained/models.zip), you will have to download the corresponding source data sets and indicate their location to ```render.py``` with an additional ```--source_path/-s``` flag. Note: The pre-trained models were created with the release codebase. This code base has been cleaned up and includes bugfixes, hence the metrics you get from evaluating them will differ from those in the paper.
|
||||
```shell
|
||||
python render.py -m <path to pre-trained model> -s <path to COLMAP dataset>
|
||||
python metrics.py -m <path to pre-trained model>
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary><span style="font-weight: bold;">Command Line Arguments for render.py</span></summary>
|
||||
|
||||
#### --model_path / -m
|
||||
Path to the trained model directory you want to create renderings for.
|
||||
#### --skip_train
|
||||
Flag to skip rendering the training set.
|
||||
#### --skip_test
|
||||
Flag to skip rendering the test set.
|
||||
#### --quiet
|
||||
Flag to omit any text written to standard out pipe.
|
||||
|
||||
**The below parameters will be read automatically from the model path, based on what was used for training. However, you may override them by providing them explicitly on the command line.**
|
||||
|
||||
#### --source_path / -s
|
||||
Path to the source directory containing a COLMAP or Synthetic NeRF data set.
|
||||
#### --images / -i
|
||||
Alternative subdirectory for COLMAP images (```images``` by default).
|
||||
#### --eval
|
||||
Add this flag to use a MipNeRF360-style training/test split for evaluation.
|
||||
#### --resolution / -r
|
||||
Changes the resolution of the loaded images before training. If provided ```1, 2, 4``` or ```8```, uses original, 1/2, 1/4 or 1/8 resolution, respectively. For all other values, rescales the width to the given number while maintaining image aspect. ```1``` by default.
|
||||
#### --white_background / -w
|
||||
Add this flag to use white background instead of black (default), e.g., for evaluation of NeRF Synthetic dataset.
|
||||
#### --convert_SHs_python
|
||||
Flag to make pipeline render with computed SHs from PyTorch instead of ours.
|
||||
#### --convert_cov3D_python
|
||||
Flag to make pipeline render with computed 3D covariance from PyTorch instead of ours.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><span style="font-weight: bold;">Command Line Arguments for metrics.py</span></summary>
|
||||
|
||||
#### --model_paths / -m
|
||||
Space-separated list of model paths for which metrics should be computed.
|
||||
</details>
|
||||
<br>
|
||||
|
||||
We further provide the ```full_eval.py``` script. This script specifies the routine used in our evaluation and demonstrates the use of some additional parameters, e.g., ```--images (-i)``` to define alternative image directories within COLMAP data sets. If you have downloaded and extracted all the training data, you can run it like this:
|
||||
```shell
|
||||
python full_eval.py -m360 <mipnerf360 folder> -tat <tanks and temples folder> -db <deep blending folder>
|
||||
```
|
||||
In the current version, this process takes about 7h on our reference machine containing an A6000. If you want to do the full evaluation on our pre-trained models, you can specify their download location and skip training.
|
||||
```shell
|
||||
python full_eval.py -o <directory with pretrained models> --skip_training -m360 <mipnerf360 folder> -tat <tanks and temples folder> -db <deep blending folder>
|
||||
```
|
||||
|
||||
If you want to compute the metrics on our paper's [evaluation images](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/evaluation/images.zip), you can also skip rendering. In this case it is not necessary to provide the source datasets. You can compute metrics for multiple image sets at a time.
|
||||
```shell
|
||||
python full_eval.py -m <directory with evaluation images>/garden ... --skip_training --skip_rendering
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary><span style="font-weight: bold;">Command Line Arguments for full_eval.py</span></summary>
|
||||
|
||||
#### --skip_training
|
||||
Flag to skip training stage.
|
||||
#### --skip_rendering
|
||||
Flag to skip rendering stage.
|
||||
#### --skip_metrics
|
||||
Flag to skip metrics calculation stage.
|
||||
#### --output_path
|
||||
Directory to put renderings and results in, ```./eval``` by default, set to pre-trained model location if evaluating them.
|
||||
#### --mipnerf360 / -m360
|
||||
Path to MipNeRF360 source datasets, required if training or rendering.
|
||||
#### --tanksandtemples / -tat
|
||||
Path to Tanks&Temples source datasets, required if training or rendering.
|
||||
#### --deepblending / -db
|
||||
Path to Deep Blending source datasets, required if training or rendering.
|
||||
</details>
|
||||
<br>
|
||||
|
||||
## Interactive Viewers
|
||||
We provide two interactive viewers for our method: remote and real-time. Our viewing solutions are based on the [SIBR](https://sibr.gitlabpages.inria.fr/) framework, developed by the GRAPHDECO group for several novel-view synthesis projects.
|
||||
|
||||
### Hardware Requirements
|
||||
- OpenGL 4.5-ready GPU and drivers (or latest MESA software)
|
||||
- 4 GB VRAM recommended
|
||||
- CUDA-ready GPU with Compute Capability 7.0+ (only for Real-Time Viewer)
|
||||
|
||||
### Software Requirements
|
||||
- Visual Studio or g++, **not Clang** (we used Visual Studio 2019 for Windows)
|
||||
- CUDA SDK 11, install *after* Visual Studio (we used 11.8)
|
||||
- CMake (recent version, we used 3.24)
|
||||
- 7zip (only on Windows)
|
||||
|
||||
### Pre-built Windows Binaries
|
||||
We provide pre-built binaries for Windows [here](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/binaries/viewers.zip). We recommend using them on Windows for an efficient setup, since the building of SIBR involves several external dependencies that must be downloaded and compiled on-the-fly.
|
||||
|
||||
### Installation from Source
|
||||
If you cloned with submodules (e.g., using ```--recursive```), the source code for the viewers is found in ```SIBR_viewers```. The network viewer runs within the SIBR framework for Image-based Rendering applications.
|
||||
|
||||
#### Windows
|
||||
CMake should take care of your dependencies.
|
||||
```shell
|
||||
cd SIBR_viewers
|
||||
cmake -Bbuild .
|
||||
cmake --build build --target install --config RelWithDebInfo
|
||||
```
|
||||
You may specify a different configuration, e.g. ```Debug``` if you need more control during development.
|
||||
|
||||
#### Ubuntu 22.04
|
||||
You will need to install a few dependencies before running the project setup.
|
||||
```shell
|
||||
# Dependencies
|
||||
sudo apt install -y libglew-dev libassimp-dev libboost-all-dev libgtk-3-dev libopencv-dev libglfw3-dev libavdevice-dev libavcodec-dev libeigen3-dev libxxf86vm-dev libembree-dev
|
||||
# Project setup
|
||||
cd SIBR_viewers
|
||||
cmake -Bbuild . -DCMAKE_BUILD_TYPE=Release # add -G Ninja to build faster
|
||||
cmake --build build -j24 --target install
|
||||
```
|
||||
|
||||
#### Ubuntu 20.04
|
||||
Backwards compatibility with Focal Fossa is not fully tested, but building SIBR with CMake should still work after invoking
|
||||
```shell
|
||||
git checkout fossa_compatibility
|
||||
```
|
||||
|
||||
### Navigation in SIBR Viewers
|
||||
The SIBR interface provides several methods of navigating the scene. By default, you will be started with an FPS navigator, which you can control with ```W, A, S, D, Q, E``` for camera translation and ```I, K, J, L, U, O``` for rotation. Alternatively, you may want to use a Trackball-style navigator (select from the floating menu). You can also snap to a camera from the data set with the ```Snap to``` button or find the closest camera with ```Snap to closest```. The floating menues also allow you to change the navigation speed. You can use the ```Scaling Modifier``` to control the size of the displayed Gaussians, or show the initial point cloud.
|
||||
|
||||
### Running the Network Viewer
|
||||
|
||||
|
||||
|
||||
https://github.com/graphdeco-inria/gaussian-splatting/assets/40643808/90a2e4d3-cf2e-4633-b35f-bfe284e28ff7
|
||||
|
||||
|
||||
|
||||
After extracting or installing the viewers, you may run the compiled ```SIBR_remoteGaussian_app[_config]``` app in ```<SIBR install dir>/bin```, e.g.:
|
||||
```shell
|
||||
./<SIBR install dir>/bin/SIBR_remoteGaussian_app
|
||||
```
|
||||
The network viewer allows you to connect to a running training process on the same or a different machine. If you are training on the same machine and OS, no command line parameters should be required: the optimizer communicates the location of the training data to the network viewer. By default, optimizer and network viewer will try to establish a connection on **localhost** on port **6009**. You can change this behavior by providing matching ```--ip``` and ```--port``` parameters to both the optimizer and the network viewer. If for some reason the path used by the optimizer to find the training data is not reachable by the network viewer (e.g., due to them running on different (virtual) machines), you may specify an override location to the viewer by using ```-s <source path>```.
|
||||
|
||||
<details>
|
||||
<summary><span style="font-weight: bold;">Primary Command Line Arguments for Network Viewer</span></summary>
|
||||
|
||||
#### --path / -s
|
||||
Argument to override model's path to source dataset.
|
||||
#### --ip
|
||||
IP to use for connection to a running training script.
|
||||
#### --port
|
||||
Port to use for connection to a running training script.
|
||||
#### --rendering-size
|
||||
Takes two space separated numbers to define the resolution at which network rendering occurs, ```1200``` width by default.
|
||||
Note that to enforce an aspect that differs from the input images, you need ```--force-aspect-ratio``` too.
|
||||
#### --load_images
|
||||
Flag to load source dataset images to be displayed in the top view for each camera.
|
||||
</details>
|
||||
<br>
|
||||
|
||||
### Running the Real-Time Viewer
|
||||
|
||||
|
||||
|
||||
|
||||
https://github.com/graphdeco-inria/gaussian-splatting/assets/40643808/0940547f-1d82-4c2f-a616-44eabbf0f816
|
||||
|
||||
|
||||
|
||||
|
||||
After extracting or installing the viewers, you may run the compiled ```SIBR_gaussianViewer_app[_config]``` app in ```<SIBR install dir>/bin```, e.g.:
|
||||
```shell
|
||||
./<SIBR install dir>/bin/SIBR_gaussianViewer_app -m <path to trained model>
|
||||
```
|
||||
|
||||
It should suffice to provide the ```-m``` parameter pointing to a trained model directory. Alternatively, you can specify an override location for training input data using ```-s```. To use a specific resolution other than the auto-chosen one, specify ```--rendering-size <width> <height>```. Combine it with ```--force-aspect-ratio``` if you want the exact resolution and don't mind image distortion.
|
||||
|
||||
**To unlock the full frame rate, please disable V-Sync on your machine and also in the application (Menu → Display). In a multi-GPU system (e.g., laptop) your OpenGL/Display GPU should be the same as your CUDA GPU (e.g., by setting the application's GPU preference on Windows, see below) for maximum performance.**
|
||||
|
||||
|
||||
|
||||
In addition to the initial point cloud and the splats, you also have the option to visualize the Gaussians by rendering them as ellipsoids from the floating menu.
|
||||
SIBR has many other functionalities, please see the [documentation](https://sibr.gitlabpages.inria.fr/) for more details on the viewer, navigation options etc. There is also a Top View (available from the menu) that shows the placement of the input cameras and the original SfM point cloud; please note that Top View slows rendering when enabled. The real-time viewer also uses slightly more aggressive, fast culling, which can be toggled in the floating menu. If you ever encounter an issue that can be solved by turning fast culling off, please let us know.
|
||||
|
||||
<details>
|
||||
<summary><span style="font-weight: bold;">Primary Command Line Arguments for Real-Time Viewer</span></summary>
|
||||
|
||||
#### --model-path / -m
|
||||
Path to trained model.
|
||||
#### --iteration
|
||||
Specifies which of state to load if multiple are available. Defaults to latest available iteration.
|
||||
#### --path / -s
|
||||
Argument to override model's path to source dataset.
|
||||
#### --rendering-size
|
||||
Takes two space separated numbers to define the resolution at which real-time rendering occurs, ```1200``` width by default. Note that to enforce an aspect that differs from the input images, you need ```--force-aspect-ratio``` too.
|
||||
#### --load_images
|
||||
Flag to load source dataset images to be displayed in the top view for each camera.
|
||||
#### --device
|
||||
Index of CUDA device to use for rasterization if multiple are available, ```0``` by default.
|
||||
#### --no_interop
|
||||
Disables CUDA/GL interop forcibly. Use on systems that may not behave according to spec (e.g., WSL2 with MESA GL 4.5 software rendering).
|
||||
</details>
|
||||
<br>
|
||||
|
||||
## Processing your own Scenes
|
||||
|
||||
Our COLMAP loaders expect the following dataset structure in the source path location:
|
||||
|
||||
```
|
||||
<location>
|
||||
|---images
|
||||
| |---<image 0>
|
||||
| |---<image 1>
|
||||
| |---...
|
||||
|---sparse
|
||||
|---0
|
||||
|---cameras.bin
|
||||
|---images.bin
|
||||
|---points3D.bin
|
||||
```
|
||||
|
||||
For rasterization, the camera models must be either a SIMPLE_PINHOLE or PINHOLE camera. We provide a converter script ```convert.py```, to extract undistorted images and SfM information from input images. Optionally, you can use ImageMagick to resize the undistorted images. This rescaling is similar to MipNeRF360, i.e., it creates images with 1/2, 1/4 and 1/8 the original resolution in corresponding folders. To use them, please first install a recent version of COLMAP (ideally CUDA-powered) and ImageMagick. Put the images you want to use in a directory ```<location>/input```.
|
||||
```
|
||||
<location>
|
||||
|---input
|
||||
|---<image 0>
|
||||
|---<image 1>
|
||||
|---...
|
||||
```
|
||||
If you have COLMAP and ImageMagick on your system path, you can simply run
|
||||
```shell
|
||||
python convert.py -s <location> [--resize] #If not resizing, ImageMagick is not needed
|
||||
```
|
||||
Alternatively, you can use the optional parameters ```--colmap_executable``` and ```--magick_executable``` to point to the respective paths. Please note that on Windows, the executable should point to the COLMAP ```.bat``` file that takes care of setting the execution environment. Once done, ```<location>``` will contain the expected COLMAP data set structure with undistorted, resized input images, in addition to your original images and some temporary (distorted) data in the directory ```distorted```.
|
||||
|
||||
If you have your own COLMAP dataset without undistortion (e.g., using ```OPENCV``` camera), you can try to just run the last part of the script: Put the images in ```input``` and the COLMAP info in a subdirectory ```distorted```:
|
||||
```
|
||||
<location>
|
||||
|---input
|
||||
| |---<image 0>
|
||||
| |---<image 1>
|
||||
| |---...
|
||||
|---distorted
|
||||
|---database.db
|
||||
|---sparse
|
||||
|---0
|
||||
|---...
|
||||
```
|
||||
Then run
|
||||
```shell
|
||||
python convert.py -s <location> --skip_matching [--resize] #If not resizing, ImageMagick is not needed
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary><span style="font-weight: bold;">Command Line Arguments for convert.py</span></summary>
|
||||
|
||||
#### --no_gpu
|
||||
Flag to avoid using GPU in COLMAP.
|
||||
#### --skip_matching
|
||||
Flag to indicate that COLMAP info is available for images.
|
||||
#### --source_path / -s
|
||||
Location of the inputs.
|
||||
#### --camera
|
||||
Which camera model to use for the early matching steps, ```OPENCV``` by default.
|
||||
#### --resize
|
||||
Flag for creating resized versions of input images.
|
||||
#### --colmap_executable
|
||||
Path to the COLMAP executable (```.bat``` on Windows).
|
||||
#### --magick_executable
|
||||
Path to the ImageMagick executable.
|
||||
</details>
|
||||
<br>
|
||||
|
||||
### Training speed acceleration
|
||||
|
||||
We integrated the drop-in replacements from [Taming-3dgs](https://humansensinglab.github.io/taming-3dgs/)<sup>1</sup> with [fused ssim](https://github.com/rahul-goel/fused-ssim/tree/main) into the original codebase to speed up training times. Once installed, the accelerated rasterizer delivers a **$\times$ 1.6 training time speedup** using `--optimizer_type default` and a **$\times$ 2.7 training time speedup** using `--optimizer_type sparse_adam`.
|
||||
|
||||
To get faster training times you must first install the accelerated rasterizer to your environment:
|
||||
|
||||
```bash
|
||||
pip uninstall diff-gaussian-rasterization -y
|
||||
cd submodules/diff-gaussian-rasterization
|
||||
rm -r build
|
||||
git checkout 3dgs_accel
|
||||
pip install .
|
||||
```
|
||||
|
||||
Then you can add the following parameter to use the sparse adam optimizer when running `train.py`:
|
||||
|
||||
```bash
|
||||
--optimizer_type sparse_adam
|
||||
```
|
||||
|
||||
*Note that this custom rasterizer has a different behaviour than the original version, for more details on training times please see [stats for training times](results.md/#training-times-comparisons)*.
|
||||
|
||||
*1. Mallick and Goel, et al. ‘Taming 3DGS: High-Quality Radiance Fields with Limited Resources’. SIGGRAPH Asia 2024 Conference Papers, 2024, https://doi.org/10.1145/3680528.3687694, [github](https://github.com/humansensinglab/taming-3dgs)*
|
||||
|
||||
|
||||
### Depth regularization
|
||||
|
||||
To have better reconstructed scenes we use depth maps as priors during optimization with each input images. It works best on untextured parts ex: roads and can remove floaters. Several papers have used similar ideas to improve various aspects of 3DGS; (e.g. [DepthRegularizedGS](https://robot0321.github.io/DepthRegGS/index.html), [SparseGS](https://formycat.github.io/SparseGS-Real-Time-360-Sparse-View-Synthesis-using-Gaussian-Splatting/), [DNGaussian](https://fictionarry.github.io/DNGaussian/)). The depth regularization we integrated is that used in our [Hierarchical 3DGS](https://repo-sam.inria.fr/fungraph/hierarchical-3d-gaussians/) paper, but applied to the original 3DGS; for some scenes (e.g., the DeepBlending scenes) it improves quality significantly; for others it either makes a small difference or can even be worse. For example results showing the potential benefit and statistics on quality please see here: [Stats for depth regularization](results.md).
|
||||
|
||||
When training on a synthetic dataset, depth maps can be produced and they do not require further processing to be used in our method.
|
||||
|
||||
For real world datasets depth maps should be generated for each input images, to generate them please do the following:
|
||||
1. Clone [Depth Anything v2](https://github.com/DepthAnything/Depth-Anything-V2?tab=readme-ov-file#usage):
|
||||
```
|
||||
git clone https://github.com/DepthAnything/Depth-Anything-V2.git
|
||||
```
|
||||
2. Download weights from [Depth-Anything-V2-Large](https://huggingface.co/depth-anything/Depth-Anything-V2-Large/resolve/main/depth_anything_v2_vitl.pth?download=true) and place it under `Depth-Anything-V2/checkpoints/`
|
||||
3. Generate depth maps:
|
||||
```
|
||||
python Depth-Anything-V2/run.py --encoder vitl --pred-only --grayscale --img-path <path to input images> --outdir <output path>
|
||||
```
|
||||
5. Generate a `depth_params.json` file using:
|
||||
```
|
||||
python utils/make_depth_scale.py --base_dir <path to colmap> --depths_dir <path to generated depths>
|
||||
```
|
||||
|
||||
A new parameter should be set when training if you want to use depth regularization `-d <path to depth maps>`.
|
||||
|
||||
### Exposure compensation
|
||||
To compensate for exposure changes in the different input images we optimize an affine transformation for each image just as in [Hierarchical 3dgs](https://repo-sam.inria.fr/fungraph/hierarchical-3d-gaussians/).
|
||||
|
||||
This can greatly improve reconstruction results for "in the wild" captures, e.g., with a smartphone when the exposure setting of the camera is not fixed. For example results showing the potential benefit and statistics on quality please see here: [Stats for exposure compensation](results.md).
|
||||
|
||||
Add the following parameters to enable it:
|
||||
```
|
||||
--exposure_lr_init 0.001 --exposure_lr_final 0.0001 --exposure_lr_delay_steps 5000 --exposure_lr_delay_mult 0.001 --train_test_exp
|
||||
```
|
||||
Again, other excellent papers have used similar ideas e.g. [NeRF-W](https://nerf-w.github.io/), [URF](https://urban-radiance-fields.github.io/).
|
||||
|
||||
### Anti-aliasing
|
||||
We added the EWA Filter from [Mip Splatting](https://niujinshuchong.github.io/mip-splatting/) in our codebase to remove aliasing. It is disabled by default but you can enable it by adding `--antialiasing` when training on a scene using `train.py` or rendering using `render.py`. Antialiasing can be toggled in the SIBR viewer, it is disabled by default but you should enable it when viewing a scene trained using `--antialiasing`.
|
||||
|
||||
*this scene was trained using `--antialiasing`*.
|
||||
|
||||
### SIBR: Top view
|
||||
> `Views > Top view`
|
||||
|
||||
The `Top view` renders the SfM point cloud in another view with the corresponding input cameras and the `Point view` user camera. This allows visualization of how far the viewer is from the input cameras for example.
|
||||
|
||||
It is a 3D view so the user can navigate through it just as in the `Point view` (modes available: FPS, trackball, orbit).
|
||||
<!-- _gif showing the top view, showing it is realtime_ -->
|
||||
<!--  -->
|
||||
|
||||
|
||||
Options are available to customize this view, meshes can be disabled/enabled and their scales can be modified.
|
||||
<!-- _gif showing different options_ -->
|
||||
<!--  -->
|
||||
|
||||
A useful additional functionality is to move to the position of an input image, and progressively fade out to the SfM point view in that position (e.g., to verify camera alignment). Views from input cameras can be displayed in the `Top view` (*note that `--images-path` must be set in the command line*). One can snap the `Top view` camera to the closest input camera from the user camera in the `Point view` by clicking `Top view settings > Cameras > Snap to closest`.
|
||||
<!-- _gif showing for a snapped camera the ground truth image with alpha_ -->
|
||||
<!--  -->
|
||||
|
||||
|
||||
### OpenXR support
|
||||
|
||||
OpenXR is supported in the branch `gaussian_code_release_openxr`
|
||||
Within that branch, you can find documentation for VR support [here](https://gitlab.inria.fr/sibr/sibr_core/-/tree/gaussian_code_release_openxr?ref_type=heads).
|
||||
|
||||
|
||||
## FAQ
|
||||
- *Where do I get data sets, e.g., those referenced in ```full_eval.py```?* The MipNeRF360 data set is provided by the authors of the original paper on the project site. Note that two of the data sets cannot be openly shared and require you to consult the authors directly. For Tanks&Temples and Deep Blending, please use the download links provided at the top of the page. Alternatively, you may access the cloned data (status: August 2023!) from [HuggingFace](https://huggingface.co/camenduru/gaussian-splatting)
|
||||
|
||||
|
||||
- *How can I use this for a much larger dataset, like a city district?* The current method was not designed for these, but given enough memory, it should work out. However, the approach can struggle in multi-scale detail scenes (extreme close-ups, mixed with far-away shots). This is usually the case in, e.g., driving data sets (cars close up, buildings far away). For such scenes, you can lower the ```--position_lr_init```, ```--position_lr_final``` and ```--scaling_lr``` (x0.3, x0.1, ...). The more extensive the scene, the lower these values should be. Below, we use default learning rates (left) and ```--position_lr_init 0.000016 --scaling_lr 0.001"``` (right).
|
||||
|
||||
|  <!-- --> | <!-- -->  |
|
||||
| --- | --- |
|
||||
|
||||
- *I'm on Windows and I can't manage to build the submodules, what do I do?* Consider following the steps in the excellent video tutorial [here](https://www.youtube.com/watch?v=UXtuigy_wYc), hopefully they should help. The order in which the steps are done is important! Alternatively, consider using the linked Colab template.
|
||||
|
||||
- *It still doesn't work. It says something about ```cl.exe```. What do I do?* User Henry Pearce found a workaround. You can you try adding the visual studio path to your environment variables (your version number might differ);
|
||||
```C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\Hostx64\x64```
|
||||
Then make sure you start a new conda prompt and cd to your repo location and try this;
|
||||
```
|
||||
conda activate gaussian_splatting
|
||||
cd <dir_to_repo>/gaussian-splatting
|
||||
pip install submodules\diff-gaussian-rasterization
|
||||
pip install submodules\simple-knn
|
||||
```
|
||||
|
||||
- *I'm on macOS/Puppy Linux/Greenhat and I can't manage to build, what do I do?* Sorry, we can't provide support for platforms outside of the ones we list in this README. Consider using the linked Colab template.
|
||||
|
||||
- *I don't have 24 GB of VRAM for training, what do I do?* The VRAM consumption is determined by the number of points that are being optimized, which increases over time. If you only want to train to 7k iterations, you will need significantly less. To do the full training routine and avoid running out of memory, you can increase the ```--densify_grad_threshold```, ```--densification_interval``` or reduce the value of ```--densify_until_iter```. Note however that this will affect the quality of the result. Also try setting ```--test_iterations``` to ```-1``` to avoid memory spikes during testing. If ```--densify_grad_threshold``` is very high, no densification should occur and training should complete if the scene itself loads successfully.
|
||||
|
||||
- *24 GB of VRAM for reference quality training is still a lot! Can't we do it with less?* Yes, most likely. By our calculations it should be possible with **way** less memory (~8GB). If we can find the time we will try to achieve this. If some PyTorch veteran out there wants to tackle this, we look forward to your pull request!
|
||||
|
||||
|
||||
- *How can I use the differentiable Gaussian rasterizer for my own project?* Easy, it is included in this repo as a submodule ```diff-gaussian-rasterization```. Feel free to check out and install the package. It's not really documented, but using it from the Python side is very straightforward (cf. ```gaussian_renderer/__init__.py```).
|
||||
|
||||
- *Wait, but ```<insert feature>``` isn't optimized and could be much better?* There are several parts we didn't even have time to think about improving (yet). The performance you get with this prototype is probably a rather slow baseline for what is physically possible.
|
||||
|
||||
- *Something is broken, how did this happen?* We tried hard to provide a solid and comprehensible basis to make use of the paper's method. We have refactored the code quite a bit, but we have limited capacity to test all possible usage scenarios. Thus, if part of the website, the code or the performance is lacking, please create an issue. If we find the time, we will do our best to address it.
|
||||
493
research/gpu-fusion-splatting/mirrors/gsplat-docs.html
Normal file
493
research/gpu-fusion-splatting/mirrors/gsplat-docs.html
Normal file
|
|
@ -0,0 +1,493 @@
|
|||
<!doctype html>
|
||||
<html class="no-js" lang="en" data-content_root="./">
|
||||
<head><meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<meta name="color-scheme" content="light dark"><meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<link rel="index" title="Index" href="genindex.html"><link rel="search" title="Search" href="search.html"><link rel="next" title="Fit a COLMAP Capture" href="examples/colmap.html">
|
||||
|
||||
<!-- Generated with Sphinx 8.1.3 and Furo 2025.12.19 -->
|
||||
<title>gsplat documentation</title>
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=d111a655" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/furo.css?v=7bdb33bb" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?v=8dab3a3b" />
|
||||
|
||||
|
||||
|
||||
|
||||
<style>
|
||||
body {
|
||||
--color-code-background: #f2f2f2;
|
||||
--color-code-foreground: #1e1e1e;
|
||||
|
||||
}
|
||||
@media not print {
|
||||
body[data-theme="dark"] {
|
||||
--color-code-background: #202020;
|
||||
--color-code-foreground: #d0d0d0;
|
||||
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
body:not([data-theme="light"]) {
|
||||
--color-code-background: #202020;
|
||||
--color-code-foreground: #d0d0d0;
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
</style></head>
|
||||
<body>
|
||||
|
||||
<script>
|
||||
document.body.dataset.theme = localStorage.getItem("theme") || "auto";
|
||||
</script>
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
|
||||
<symbol id="svg-toc" viewBox="0 0 24 24">
|
||||
<title>Contents</title>
|
||||
<svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 1024 1024">
|
||||
<path d="M408 442h480c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8H408c-4.4 0-8 3.6-8 8v56c0 4.4 3.6 8 8 8zm-8 204c0 4.4 3.6 8 8 8h480c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8H408c-4.4 0-8 3.6-8 8v56zm504-486H120c-4.4 0-8 3.6-8 8v56c0 4.4 3.6 8 8 8h784c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8zm0 632H120c-4.4 0-8 3.6-8 8v56c0 4.4 3.6 8 8 8h784c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8zM115.4 518.9L271.7 642c5.8 4.6 14.4.5 14.4-6.9V388.9c0-7.4-8.5-11.5-14.4-6.9L115.4 505.1a8.74 8.74 0 0 0 0 13.8z"/>
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-menu" viewBox="0 0 24 24">
|
||||
<title>Menu</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather-menu">
|
||||
<line x1="3" y1="12" x2="21" y2="12"></line>
|
||||
<line x1="3" y1="6" x2="21" y2="6"></line>
|
||||
<line x1="3" y1="18" x2="21" y2="18"></line>
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-arrow-right" viewBox="0 0 24 24">
|
||||
<title>Expand</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather-chevron-right">
|
||||
<polyline points="9 18 15 12 9 6"></polyline>
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-sun" viewBox="0 0 24 24">
|
||||
<title>Light mode</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="1" stroke-linecap="round" stroke-linejoin="round" class="feather-sun">
|
||||
<circle cx="12" cy="12" r="5"></circle>
|
||||
<line x1="12" y1="1" x2="12" y2="3"></line>
|
||||
<line x1="12" y1="21" x2="12" y2="23"></line>
|
||||
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
|
||||
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
|
||||
<line x1="1" y1="12" x2="3" y2="12"></line>
|
||||
<line x1="21" y1="12" x2="23" y2="12"></line>
|
||||
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
|
||||
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-moon" viewBox="0 0 24 24">
|
||||
<title>Dark mode</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="1" stroke-linecap="round" stroke-linejoin="round" class="icon-tabler-moon">
|
||||
<path stroke="none" d="M0 0h24v24H0z" fill="none" />
|
||||
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z" />
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-sun-with-moon" viewBox="0 0 24 24">
|
||||
<title>Auto light/dark, in light mode</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="1" stroke-linecap="round" stroke-linejoin="round"
|
||||
class="icon-custom-derived-from-feather-sun-and-tabler-moon">
|
||||
<path style="opacity: 50%" d="M 5.411 14.504 C 5.471 14.504 5.532 14.504 5.591 14.504 C 3.639 16.319 4.383 19.569 6.931 20.352 C 7.693 20.586 8.512 20.551 9.25 20.252 C 8.023 23.207 4.056 23.725 2.11 21.184 C 0.166 18.642 1.702 14.949 4.874 14.536 C 5.051 14.512 5.231 14.5 5.411 14.5 L 5.411 14.504 Z"/>
|
||||
<line x1="14.5" y1="3.25" x2="14.5" y2="1.25"/>
|
||||
<line x1="14.5" y1="15.85" x2="14.5" y2="17.85"/>
|
||||
<line x1="10.044" y1="5.094" x2="8.63" y2="3.68"/>
|
||||
<line x1="19" y1="14.05" x2="20.414" y2="15.464"/>
|
||||
<line x1="8.2" y1="9.55" x2="6.2" y2="9.55"/>
|
||||
<line x1="20.8" y1="9.55" x2="22.8" y2="9.55"/>
|
||||
<line x1="10.044" y1="14.006" x2="8.63" y2="15.42"/>
|
||||
<line x1="19" y1="5.05" x2="20.414" y2="3.636"/>
|
||||
<circle cx="14.5" cy="9.55" r="3.6"/>
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-moon-with-sun" viewBox="0 0 24 24">
|
||||
<title>Auto light/dark, in dark mode</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="1" stroke-linecap="round" stroke-linejoin="round"
|
||||
class="icon-custom-derived-from-feather-sun-and-tabler-moon">
|
||||
<path d="M 8.282 7.007 C 8.385 7.007 8.494 7.007 8.595 7.007 C 5.18 10.184 6.481 15.869 10.942 17.24 C 12.275 17.648 13.706 17.589 15 17.066 C 12.851 22.236 5.91 23.143 2.505 18.696 C -0.897 14.249 1.791 7.786 7.342 7.063 C 7.652 7.021 7.965 7 8.282 7 L 8.282 7.007 Z"/>
|
||||
<line style="opacity: 50%" x1="18" y1="3.705" x2="18" y2="2.5"/>
|
||||
<line style="opacity: 50%" x1="18" y1="11.295" x2="18" y2="12.5"/>
|
||||
<line style="opacity: 50%" x1="15.316" y1="4.816" x2="14.464" y2="3.964"/>
|
||||
<line style="opacity: 50%" x1="20.711" y1="10.212" x2="21.563" y2="11.063"/>
|
||||
<line style="opacity: 50%" x1="14.205" y1="7.5" x2="13.001" y2="7.5"/>
|
||||
<line style="opacity: 50%" x1="21.795" y1="7.5" x2="23" y2="7.5"/>
|
||||
<line style="opacity: 50%" x1="15.316" y1="10.184" x2="14.464" y2="11.036"/>
|
||||
<line style="opacity: 50%" x1="20.711" y1="4.789" x2="21.563" y2="3.937"/>
|
||||
<circle style="opacity: 50%" cx="18" cy="7.5" r="2.169"/>
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-pencil" viewBox="0 0 24 24">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="1" stroke-linecap="round" stroke-linejoin="round" class="icon-tabler-pencil-code">
|
||||
<path d="M4 20h4l10.5 -10.5a2.828 2.828 0 1 0 -4 -4l-10.5 10.5v4" />
|
||||
<path d="M13.5 6.5l4 4" />
|
||||
<path d="M20 21l2 -2l-2 -2" />
|
||||
<path d="M17 17l-2 2l2 2" />
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-eye" viewBox="0 0 24 24">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="1" stroke-linecap="round" stroke-linejoin="round" class="icon-tabler-eye-code">
|
||||
<path stroke="none" d="M0 0h24v24H0z" fill="none" />
|
||||
<path d="M10 12a2 2 0 1 0 4 0a2 2 0 0 0 -4 0" />
|
||||
<path
|
||||
d="M11.11 17.958c-3.209 -.307 -5.91 -2.293 -8.11 -5.958c2.4 -4 5.4 -6 9 -6c3.6 0 6.6 2 9 6c-.21 .352 -.427 .688 -.647 1.008" />
|
||||
<path d="M20 21l2 -2l-2 -2" />
|
||||
<path d="M17 17l-2 2l2 2" />
|
||||
</svg>
|
||||
</symbol>
|
||||
</svg>
|
||||
|
||||
<input type="checkbox" class="sidebar-toggle" name="__navigation" id="__navigation" aria-label="Toggle site navigation sidebar">
|
||||
<input type="checkbox" class="sidebar-toggle" name="__toc" id="__toc" aria-label="Toggle table of contents sidebar">
|
||||
<label class="overlay sidebar-overlay" for="__navigation"></label>
|
||||
<label class="overlay toc-overlay" for="__toc"></label>
|
||||
|
||||
<a class="skip-to-content muted-link" href="#furo-main-content">Skip to content</a>
|
||||
|
||||
|
||||
|
||||
<div class="page">
|
||||
<header class="mobile-header">
|
||||
<div class="header-left">
|
||||
<label class="nav-overlay-icon" for="__navigation">
|
||||
<span class="icon"><svg><use href="#svg-menu"></use></svg></span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="header-center">
|
||||
<a href="#"><div class="brand">gsplat documentation</div></a>
|
||||
</div>
|
||||
<div class="header-right">
|
||||
<div class="theme-toggle-container theme-toggle-header">
|
||||
<button class="theme-toggle" aria-label="Toggle Light / Dark / Auto color theme">
|
||||
<svg class="theme-icon-when-auto-light"><use href="#svg-sun-with-moon"></use></svg>
|
||||
<svg class="theme-icon-when-auto-dark"><use href="#svg-moon-with-sun"></use></svg>
|
||||
<svg class="theme-icon-when-dark"><use href="#svg-moon"></use></svg>
|
||||
<svg class="theme-icon-when-light"><use href="#svg-sun"></use></svg>
|
||||
</button>
|
||||
</div>
|
||||
<label class="toc-overlay-icon toc-header-icon" for="__toc">
|
||||
<span class="icon"><svg><use href="#svg-toc"></use></svg></span>
|
||||
</label>
|
||||
</div>
|
||||
</header>
|
||||
<aside class="sidebar-drawer">
|
||||
<div class="sidebar-container">
|
||||
|
||||
<div class="sidebar-sticky"><a class="sidebar-brand" href="#">
|
||||
|
||||
<span class="sidebar-brand-text">gsplat documentation</span>
|
||||
</a>
|
||||
|
||||
<!-- Dropdown for different versions of the docs. Slightly hacky. -->
|
||||
<div style="padding: 0 1em;">
|
||||
<script>
|
||||
var DocsVersionsPopulated = false;
|
||||
|
||||
async function getVersionList() {
|
||||
// This index.txt file is written by the docs.yml GitHub action.
|
||||
// https://github.com/nerfstudio-project/gsplat/blob/main/.github/workflows/doc.yml
|
||||
const response = await fetch(
|
||||
"https://docs.gsplat.studio/versions/index.txt",
|
||||
{ cache: "no-cache" }
|
||||
);
|
||||
return await response.text();
|
||||
}
|
||||
async function DocsPopulateVersionDropDown () {
|
||||
// Load the version list lazily...
|
||||
if (DocsVersionsPopulated) {
|
||||
return;
|
||||
}
|
||||
DocsVersionsPopulated = true;
|
||||
|
||||
console.log("Populating docs version list!")
|
||||
const versions = (await getVersionList()).trim().split("\n").reverse();
|
||||
console.log(versions);
|
||||
let htmlString = "<ul style='margin: 0.5rem 0 0 0'>";
|
||||
htmlString += `<li><a href="https://docs.gsplat.studio/main">main</a></li>`;
|
||||
for (let version of versions) {
|
||||
htmlString += `<li><a href="https://docs.gsplat.studio/versions/${version}">${version}</a></li>`;
|
||||
}
|
||||
|
||||
htmlString += "</ul>";
|
||||
document.getElementById("version-dropdown").innerHTML = htmlString;
|
||||
}
|
||||
</script>
|
||||
<details
|
||||
style="padding: 0.5rem; background: var(--color-background-primary); border-radius: 0.5rem; border: 1px solid var(--color-sidebar-background-border);"
|
||||
ontoggle="DocsPopulateVersionDropDown()"
|
||||
>
|
||||
<summary style="cursor: pointer;"><strong>Version:</strong> <em>main</em></summary>
|
||||
<div id="version-dropdown"></div>
|
||||
</details>
|
||||
<!-- End dropdown -->
|
||||
</div>
|
||||
|
||||
<div style="text-align: left; padding: 1em">
|
||||
<script async defer src="https://buttons.github.io/buttons.js"></script>
|
||||
<a class="github-button" href="https://github.com/nerfstudio-project/gsplat"
|
||||
data-color-scheme="no-preference: light; light: light; dark: light;" data-size="large" data-show-count="true"
|
||||
aria-label="Download buttons/github-buttons on GitHub">
|
||||
Github
|
||||
</a>
|
||||
</div><form class="sidebar-search-container" method="get" action="search.html" role="search">
|
||||
<input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
|
||||
<input type="hidden" name="check_keywords" value="yes">
|
||||
<input type="hidden" name="area" value="default">
|
||||
</form>
|
||||
<div id="searchbox"></div><div class="sidebar-scroll"><div class="sidebar-tree">
|
||||
<p class="caption" role="heading"><span class="caption-text">Examples</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="examples/colmap.html">Fit a COLMAP Capture</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="examples/image.html">Fit a Single Image</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="examples/large_scale.html">Render a Large Scene</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="examples/ncore.html">Train on an NCore v4 Capture</a></li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">Conventions</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="conventions/data_conventions.html">Data Conventions</a></li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">Python API</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="apis/compression.html">Compression</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="apis/rasterization.html">Rasterization</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="apis/strategy.html">Densification</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="apis/utils.html">Utils</a></li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">Tests</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="tests/eval.html">Evaluation</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="tests/profile.html">Profiling</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="tests/tests.html">Tests</a></li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">Migration</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="migration/migration_inria.html">Migrate from diff-gaussian-rasterization</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="migration/migration_legacy.html">Migrate from gsplat v0.1.11</a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</aside>
|
||||
<div class="main">
|
||||
<div class="content">
|
||||
<div class="article-container">
|
||||
<a href="#" class="back-to-top muted-link">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24">
|
||||
<path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8v12z"></path>
|
||||
</svg>
|
||||
<span>Back to top</span>
|
||||
</a>
|
||||
<div class="content-icon-container">
|
||||
<div class="view-this-page">
|
||||
<a class="muted-link" href="_sources/index.rst.txt" title="View this page">
|
||||
<svg><use href="#svg-eye"></use></svg>
|
||||
<span class="visually-hidden">View this page</span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="theme-toggle-container theme-toggle-content">
|
||||
<button class="theme-toggle" aria-label="Toggle Light / Dark / Auto color theme">
|
||||
<svg class="theme-icon-when-auto-light"><use href="#svg-sun-with-moon"></use></svg>
|
||||
<svg class="theme-icon-when-auto-dark"><use href="#svg-moon-with-sun"></use></svg>
|
||||
<svg class="theme-icon-when-dark"><use href="#svg-moon"></use></svg>
|
||||
<svg class="theme-icon-when-light"><use href="#svg-sun"></use></svg>
|
||||
</button>
|
||||
</div>
|
||||
<label class="toc-overlay-icon toc-content-icon" for="__toc">
|
||||
<span class="icon"><svg><use href="#svg-toc"></use></svg></span>
|
||||
</label>
|
||||
</div>
|
||||
<article role="main" id="furo-main-content">
|
||||
<section id="gsplat">
|
||||
<h1>gsplat<a class="headerlink" href="#gsplat" title="Link to this heading">¶</a></h1>
|
||||
<a class="reference internal image-reference" href="_images/training.gif"><img alt="Example training image" src="_images/training.gif" style="width: 800px;" />
|
||||
</a>
|
||||
<section id="overview">
|
||||
<h2>Overview<a class="headerlink" href="#overview" title="Link to this heading">¶</a></h2>
|
||||
<p><em>gsplat</em> is an open-source library for CUDA-accelerated differentiable rasterization of
|
||||
3D gaussians with Python bindings. It is inspired by the SIGGRAPH paper “3D Gaussian Splatting for
|
||||
Real-Time Rendering of Radiance Fields” <span class="bibtex-citation" id="id1">[<a class="reference internal" href="#id3" title="Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, July 2023. URL: https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/.">1</a>]</span>, but we’ve made <em>gsplat</em> even
|
||||
faster, more memory efficient, and with a growing list of new features!</p>
|
||||
<ul class="simple">
|
||||
<li><p><em>gsplat</em> is developed with efficiency in mind. Comparing to the <a class="reference external" href="https://github.com/graphdeco-inria/gaussian-splatting">official implementation</a>,
|
||||
<em>gsplat</em> enables up to <strong>4x less training memory footprint</strong>, and up to <strong>15% less training time</strong> on Mip-NeRF 360 captures, and potential more on larger scenes. See <a class="reference internal" href="tests/eval.html"><span class="doc">Evaluation</span></a> for details.</p></li>
|
||||
<li><p><em>gsplat</em> is designed to <strong>support extremely large scene rendering</strong>, which is magnitudes
|
||||
faster than the official CUDA backend <a class="reference external" href="https://github.com/graphdeco-inria/diff-gaussian-rasterization">diff-gaussian-rasterization</a>. See <a class="reference internal" href="examples/large_scale.html"><span class="doc">Render a Large Scene</span></a> for an example.</p></li>
|
||||
<li><p><em>gsplat</em> offers many extra features, including <strong>batch rasterization</strong>,
|
||||
<strong>N-D feature rendering</strong>, <strong>depth rendering</strong>, <strong>sparse gradient</strong>,
|
||||
<strong>multi-GPU distributed rasterization</strong>
|
||||
etc. See <a class="reference internal" href="apis/rasterization.html"><span class="doc">Rasterization</span></a> for details.</p></li>
|
||||
<li><p><em>gsplat</em> is equipped with the <strong>latest and greatest</strong> 3D Gaussian Splatting techniques,
|
||||
including <a class="reference external" href="https://ty424.github.io/AbsGS.github.io/">absgrad</a>,
|
||||
<a class="reference external" href="https://niujinshuchong.github.io/mip-splatting/">anti-aliasing</a>,
|
||||
<a class="reference external" href="https://ubc-vision.github.io/3dgs-mcmc/">3DGS-MCMC</a> etc. And more to come!</p></li>
|
||||
</ul>
|
||||
<div style="position:relative; padding-bottom:56.25%; height:0; width:100%">
|
||||
<iframe style="position:absolute; top:0; left:0; width:100%; height:100%" src="https://www.youtube.com/embed/d5vOUYm5k34?si=fQBdwPgoi0FZOSyD" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
|
||||
</div></section>
|
||||
<section id="installation">
|
||||
<h2>Installation<a class="headerlink" href="#installation" title="Link to this heading">¶</a></h2>
|
||||
<p><em>gsplat</em> is available on PyPI and can be installed with pip:</p>
|
||||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>pip<span class="w"> </span>install<span class="w"> </span>gsplat
|
||||
</pre></div>
|
||||
</div>
|
||||
<p>To get the latest features, it can also be installed from source:</p>
|
||||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>pip<span class="w"> </span>install<span class="w"> </span>git+https://github.com/nerfstudio-project/gsplat
|
||||
</pre></div>
|
||||
</div>
|
||||
</section>
|
||||
<section id="contributing">
|
||||
<h2>Contributing<a class="headerlink" href="#contributing" title="Link to this heading">¶</a></h2>
|
||||
<p>This repository was born from the curiosity of people on the Nerfstudio team trying to
|
||||
understand a new rendering technique. We welcome contributions of any kind and are open
|
||||
to feedback, bug-reports, and improvements to help expand the capabilities of this software.</p>
|
||||
<p>This project is developed by the contributors coming from following institutes (unordered):</p>
|
||||
<ul class="simple">
|
||||
<li><p>UC Berkeley</p></li>
|
||||
<li><p>NVIDIA</p></li>
|
||||
<li><p>ShanghaiTech University</p></li>
|
||||
<li><p>Amazon</p></li>
|
||||
<li><p>Meta</p></li>
|
||||
<li><p>IIIT</p></li>
|
||||
<li><p>LumaAI</p></li>
|
||||
<li><p>SpectacularAI</p></li>
|
||||
<li><p>Aalto University</p></li>
|
||||
<li><p>CMU</p></li>
|
||||
</ul>
|
||||
</section>
|
||||
<section id="links">
|
||||
<h2>Links<a class="headerlink" href="#links" title="Link to this heading">¶</a></h2>
|
||||
<div class="toctree-wrapper compound">
|
||||
<p class="caption" role="heading"><span class="caption-text">Examples</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="examples/colmap.html">Fit a COLMAP Capture</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="examples/image.html">Fit a Single Image</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="examples/large_scale.html">Render a Large Scene</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="examples/ncore.html">Train on an NCore v4 Capture</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="toctree-wrapper compound">
|
||||
<p class="caption" role="heading"><span class="caption-text">Conventions</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="conventions/data_conventions.html">Data Conventions</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="toctree-wrapper compound">
|
||||
<p class="caption" role="heading"><span class="caption-text">Python API</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="apis/compression.html">Compression</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="apis/rasterization.html">Rasterization</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="apis/strategy.html">Densification</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="apis/utils.html">Utils</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="toctree-wrapper compound">
|
||||
<p class="caption" role="heading"><span class="caption-text">Tests</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="tests/eval.html">Evaluation</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="tests/profile.html">Profiling</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="tests/tests.html">Tests</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="toctree-wrapper compound">
|
||||
<p class="caption" role="heading"><span class="caption-text">Migration</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="migration/migration_inria.html">Migrate from diff-gaussian-rasterization</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="migration/migration_legacy.html">Migrate from gsplat v0.1.11</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
</section>
|
||||
<section id="citations">
|
||||
<h2>Citations<a class="headerlink" href="#citations" title="Link to this heading">¶</a></h2>
|
||||
<div class="docutils container" id="id2">
|
||||
<div role="list" class="citation-list">
|
||||
<div class="citation" id="id3" role="doc-biblioentry">
|
||||
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id1">1</a><span class="fn-bracket">]</span></span>
|
||||
<p>Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. <em>ACM Transactions on Graphics</em>, July 2023. URL: <a class="reference external" href="https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/">https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/</a>.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
|
||||
</article>
|
||||
</div>
|
||||
<footer>
|
||||
|
||||
<div class="related-pages">
|
||||
<a class="next-page" href="examples/colmap.html">
|
||||
<div class="page-info">
|
||||
<div class="context">
|
||||
<span>Next</span>
|
||||
</div>
|
||||
<div class="title">Fit a COLMAP Capture</div>
|
||||
</div>
|
||||
<svg class="furo-related-icon"><use href="#svg-arrow-right"></use></svg>
|
||||
</a>
|
||||
|
||||
</div>
|
||||
<div class="bottom-of-page">
|
||||
<div class="left-details">
|
||||
<div class="copyright">
|
||||
Copyright © 2023, nerfstudio team
|
||||
</div>
|
||||
Made with <a href="https://www.sphinx-doc.org/">Sphinx</a> and <a class="muted-link" href="https://pradyunsg.me">@pradyunsg</a>'s
|
||||
|
||||
<a href="https://github.com/pradyunsg/furo">Furo</a>
|
||||
|
||||
</div>
|
||||
<div class="right-details">
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</footer>
|
||||
</div>
|
||||
<aside class="toc-drawer">
|
||||
|
||||
|
||||
<div class="toc-sticky toc-scroll">
|
||||
<div class="toc-title-container">
|
||||
<span class="toc-title">
|
||||
On this page
|
||||
</span>
|
||||
</div>
|
||||
<div class="toc-tree-container">
|
||||
<div class="toc-tree">
|
||||
<ul>
|
||||
<li><a class="reference internal" href="#">gsplat</a><ul>
|
||||
<li><a class="reference internal" href="#overview">Overview</a></li>
|
||||
<li><a class="reference internal" href="#installation">Installation</a></li>
|
||||
<li><a class="reference internal" href="#contributing">Contributing</a></li>
|
||||
<li><a class="reference internal" href="#links">Links</a><ul>
|
||||
</ul>
|
||||
</li>
|
||||
<li><a class="reference internal" href="#citations">Citations</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
</aside>
|
||||
</div>
|
||||
</div><script src="_static/documentation_options.js?v=5929fcd5"></script>
|
||||
<script src="_static/doctools.js?v=9bcbadda"></script>
|
||||
<script src="_static/sphinx_highlight.js?v=dc90522c"></script>
|
||||
<script src="_static/scripts/furo.js?v=46bd48cc"></script>
|
||||
<script async="async" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
1886
research/gpu-fusion-splatting/mirrors/gsplat-github.html
Normal file
1886
research/gpu-fusion-splatting/mirrors/gsplat-github.html
Normal file
File diff suppressed because one or more lines are too long
Binary file not shown.
610
research/gpu-fusion-splatting/mirrors/nerfstudio-splatfacto.html
Normal file
610
research/gpu-fusion-splatting/mirrors/nerfstudio-splatfacto.html
Normal file
|
|
@ -0,0 +1,610 @@
|
|||
<!doctype html>
|
||||
<html class="no-js" lang="en" >
|
||||
|
||||
<head><meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1" />
|
||||
<meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
|
||||
|
||||
<meta property="og:title" content="Splatfacto" />
|
||||
|
||||
<meta property="og:type" content="website" />
|
||||
|
||||
<meta property="og:url" content="http://docs.nerf.studio/nerfology/methods/splat.html" />
|
||||
|
||||
<meta property="og:description" content="Paper Website 3D Gaussian Splatting was proposed in SIGGRAPH 2023 from INRIA, and is a completely different method of representing radiance fields by explicitly storing a collection of 3D volumetri..." />
|
||||
|
||||
<meta property="og:image" content="https://assets.nerf.studio/opg.png" />
|
||||
|
||||
<meta property="og:image:alt" content="Splatfacto" />
|
||||
<link rel="index" title="Index" href="../../genindex.html" /><link rel="search" title="Search" href="../../search.html" /><link rel="next" title="Splatfacto in the Wild" href="splatw.html" /><link rel="prev" title="Instant-NGP" href="instant_ngp.html" />
|
||||
|
||||
<meta name="generator" content="sphinx-5.2.1, furo 2022.09.29" />
|
||||
<title>Splatfacto - nerfstudio</title>
|
||||
<link rel="stylesheet" type="text/css" href="../../_static/pygments.css" />
|
||||
<link rel="stylesheet" type="text/css" href="../../_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
|
||||
<link rel="stylesheet" type="text/css" href="../../_static/mystnb.css" />
|
||||
<link rel="stylesheet" type="text/css" href="../../_static/togglebutton.css" />
|
||||
<link rel="stylesheet" type="text/css" href="../../_static/copybutton.css" />
|
||||
<link rel="stylesheet" type="text/css" href="../../_static/design-style.b7bb847fb20b106c3d81b95245e65545.min.css" />
|
||||
<link rel="stylesheet" type="text/css" href="../../_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
|
||||
|
||||
|
||||
|
||||
|
||||
<style>
|
||||
body {
|
||||
--color-code-background: #f8f9fb;
|
||||
--color-code-foreground: black;
|
||||
--color-brand-primary: #d34600;
|
||||
--color-brand-content: #ff6f00;
|
||||
|
||||
}
|
||||
@media not print {
|
||||
body[data-theme="dark"] {
|
||||
--color-code-background: #282C34;
|
||||
--color-code-foreground: #ABB2BF;
|
||||
--color-brand-primary: #fdd06c;
|
||||
--color-brand-content: ##fea96a;
|
||||
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
body:not([data-theme="light"]) {
|
||||
--color-code-background: #282C34;
|
||||
--color-code-foreground: #ABB2BF;
|
||||
--color-brand-primary: #fdd06c;
|
||||
--color-brand-content: ##fea96a;
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
</style><script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
|
||||
<script src="../../_static/jquery.js"></script>
|
||||
<script src="../../_static/underscore.js"></script>
|
||||
<script src="../../_static/_sphinx_javascript_frameworks_compat.js"></script>
|
||||
<script src="../../_static/doctools.js"></script>
|
||||
<script src="../../_static/sphinx_highlight.js"></script>
|
||||
<script src="../../_static/scripts/furo.js"></script>
|
||||
<script src="../../_static/clipboard.min.js"></script>
|
||||
<script src="../../_static/copybutton.js"></script>
|
||||
<script>let toggleHintShow = 'Click to show';</script>
|
||||
<script>let toggleHintHide = 'Click to hide';</script>
|
||||
<script>let toggleOpenOnPrint = 'true';</script>
|
||||
<script src="../../_static/togglebutton.js"></script>
|
||||
<script>var togglebuttonSelector = '.toggle, .admonition.dropdown, .tag_hide_input div.cell_input, .tag_hide-input div.cell_input, .tag_hide_output div.cell_output, .tag_hide-output div.cell_output, .tag_hide_cell.cell, .tag_hide-cell.cell';</script>
|
||||
<script src="../../_static/design-tabs.js"></script>
|
||||
<script src="../../_static/require.min.js"></script>
|
||||
<script src="../../_static/custom.js"></script>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
|
||||
<script>
|
||||
document.body.dataset.theme = localStorage.getItem("theme") || "auto";
|
||||
</script>
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
|
||||
<symbol id="svg-toc" viewBox="0 0 24 24">
|
||||
<title>Contents</title>
|
||||
<svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 1024 1024">
|
||||
<path d="M408 442h480c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8H408c-4.4 0-8 3.6-8 8v56c0 4.4 3.6 8 8 8zm-8 204c0 4.4 3.6 8 8 8h480c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8H408c-4.4 0-8 3.6-8 8v56zm504-486H120c-4.4 0-8 3.6-8 8v56c0 4.4 3.6 8 8 8h784c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8zm0 632H120c-4.4 0-8 3.6-8 8v56c0 4.4 3.6 8 8 8h784c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8zM115.4 518.9L271.7 642c5.8 4.6 14.4.5 14.4-6.9V388.9c0-7.4-8.5-11.5-14.4-6.9L115.4 505.1a8.74 8.74 0 0 0 0 13.8z"/>
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-menu" viewBox="0 0 24 24">
|
||||
<title>Menu</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather-menu">
|
||||
<line x1="3" y1="12" x2="21" y2="12"></line>
|
||||
<line x1="3" y1="6" x2="21" y2="6"></line>
|
||||
<line x1="3" y1="18" x2="21" y2="18"></line>
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-arrow-right" viewBox="0 0 24 24">
|
||||
<title>Expand</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather-chevron-right">
|
||||
<polyline points="9 18 15 12 9 6"></polyline>
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-sun" viewBox="0 0 24 24">
|
||||
<title>Light mode</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round" class="feather-sun">
|
||||
<circle cx="12" cy="12" r="5"></circle>
|
||||
<line x1="12" y1="1" x2="12" y2="3"></line>
|
||||
<line x1="12" y1="21" x2="12" y2="23"></line>
|
||||
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
|
||||
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
|
||||
<line x1="1" y1="12" x2="3" y2="12"></line>
|
||||
<line x1="21" y1="12" x2="23" y2="12"></line>
|
||||
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
|
||||
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-moon" viewBox="0 0 24 24">
|
||||
<title>Dark mode</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round" class="icon-tabler-moon">
|
||||
<path stroke="none" d="M0 0h24v24H0z" fill="none" />
|
||||
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z" />
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-sun-half" viewBox="0 0 24 24">
|
||||
<title>Auto light/dark mode</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round" class="icon-tabler-shadow">
|
||||
<path stroke="none" d="M0 0h24v24H0z" fill="none"/>
|
||||
<circle cx="12" cy="12" r="9" />
|
||||
<path d="M13 12h5" />
|
||||
<path d="M13 15h4" />
|
||||
<path d="M13 18h1" />
|
||||
<path d="M13 9h4" />
|
||||
<path d="M13 6h1" />
|
||||
</svg>
|
||||
</symbol>
|
||||
</svg>
|
||||
|
||||
<input type="checkbox" class="sidebar-toggle" name="__navigation" id="__navigation">
|
||||
<input type="checkbox" class="sidebar-toggle" name="__toc" id="__toc">
|
||||
<label class="overlay sidebar-overlay" for="__navigation">
|
||||
<div class="visually-hidden">Hide navigation sidebar</div>
|
||||
</label>
|
||||
<label class="overlay toc-overlay" for="__toc">
|
||||
<div class="visually-hidden">Hide table of contents sidebar</div>
|
||||
</label>
|
||||
|
||||
|
||||
|
||||
<div class="page">
|
||||
<header class="mobile-header">
|
||||
<div class="header-left">
|
||||
<label class="nav-overlay-icon" for="__navigation">
|
||||
<div class="visually-hidden">Toggle site navigation sidebar</div>
|
||||
<i class="icon"><svg><use href="#svg-menu"></use></svg></i>
|
||||
</label>
|
||||
</div>
|
||||
<div class="header-center">
|
||||
<a href="../../index.html"><div class="brand">nerfstudio</div></a>
|
||||
</div>
|
||||
<div class="header-right">
|
||||
<div class="theme-toggle-container theme-toggle-header">
|
||||
<button class="theme-toggle">
|
||||
<div class="visually-hidden">Toggle Light / Dark / Auto color theme</div>
|
||||
<svg class="theme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
|
||||
<svg class="theme-icon-when-dark"><use href="#svg-moon"></use></svg>
|
||||
<svg class="theme-icon-when-light"><use href="#svg-sun"></use></svg>
|
||||
</button>
|
||||
</div>
|
||||
<label class="toc-overlay-icon toc-header-icon" for="__toc">
|
||||
<div class="visually-hidden">Toggle table of contents sidebar</div>
|
||||
<i class="icon"><svg><use href="#svg-toc"></use></svg></i>
|
||||
</label>
|
||||
</div>
|
||||
</header>
|
||||
<aside class="sidebar-drawer">
|
||||
<div class="sidebar-container">
|
||||
|
||||
<div class="sidebar-sticky"><a
|
||||
class="sidebar-brand"
|
||||
href="../../index.html"
|
||||
>
|
||||
|
||||
<div class="sidebar-logo-container">
|
||||
<img
|
||||
class="sidebar-logo only-light"
|
||||
src="../../_static/imgs/logo.png"
|
||||
alt="Light Logo"
|
||||
/>
|
||||
<img
|
||||
class="sidebar-logo only-dark"
|
||||
src="../../_static/imgs/logo-dark.png"
|
||||
alt="Dark Logo"
|
||||
/>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
<div style="text-align: center">
|
||||
<script async defer src="https://buttons.github.io/buttons.js"></script>
|
||||
<!-- Place this tag where you want the button to render. -->
|
||||
<a
|
||||
class="github-button"
|
||||
href="https://github.com/nerfstudio-project/nerfstudio/"
|
||||
data-color-scheme="no-preference: light; light: light; dark: light;"
|
||||
data-size="large"
|
||||
data-show-count="true"
|
||||
aria-label="Download buttons/github-buttons on GitHub"
|
||||
>
|
||||
Github
|
||||
</a>
|
||||
<br />
|
||||
<a
|
||||
href="https://colab.research.google.com/github/nerfstudio-project/nerfstudio/blob/main/colab/demo.ipynb"
|
||||
style="
|
||||
background-color: #ebf0f4;
|
||||
background-image: linear-gradient(180deg, #f6f8fa, #ebf0f4 90%);
|
||||
border-radius: 3px;
|
||||
border: 1px solid #1b1f2426;
|
||||
display: inline-flex;
|
||||
cursor: pointer;
|
||||
color: #0d0d0d;
|
||||
font-family: Arial;
|
||||
font-size: 12px;
|
||||
font-weight: bold;
|
||||
padding: 5px 41px;
|
||||
line-height: 16px;
|
||||
text-decoration: none;
|
||||
"
|
||||
>
|
||||
<svg
|
||||
viewBox="0 0 24 24"
|
||||
width="16"
|
||||
height="16"
|
||||
class="octicon octicon-mark-github"
|
||||
>
|
||||
<g>
|
||||
<path
|
||||
d="M4.54,9.46,2.19,7.1a6.93,6.93,0,0,0,0,9.79l2.36-2.36A3.59,3.59,0,0,1,4.54,9.46Z"
|
||||
style=""
|
||||
fill="#E8710A"
|
||||
></path>
|
||||
<path
|
||||
d="M2.19,7.1,4.54,9.46a3.59,3.59,0,0,1,5.08,0l1.71-2.93h0l-.1-.08h0A6.93,6.93,0,0,0,2.19,7.1Z"
|
||||
style=""
|
||||
fill="#F9AB00"
|
||||
></path>
|
||||
<path
|
||||
d="M11.34,17.46h0L9.62,14.54a3.59,3.59,0,0,1-5.08,0L2.19,16.9a6.93,6.93,0,0,0,9,.65l.11-.09"
|
||||
style=""
|
||||
fill="#F9AB00"
|
||||
></path>
|
||||
<path
|
||||
d="M12,7.1a6.93,6.93,0,0,0,0,9.79l2.36-2.36a3.59,3.59,0,1,1,5.08-5.08L21.81,7.1A6.93,6.93,0,0,0,12,7.1Z"
|
||||
style=""
|
||||
fill="#F9AB00"
|
||||
></path>
|
||||
<path
|
||||
d="M21.81,7.1,19.46,9.46a3.59,3.59,0,0,1-5.08,5.08L12,16.9A6.93,6.93,0,0,0,21.81,7.1Z"
|
||||
style=""
|
||||
fill="#E8710A"
|
||||
></path>
|
||||
</g>
|
||||
</svg>
|
||||
<span> colab</span></a
|
||||
>
|
||||
</div><form class="sidebar-search-container" method="get" action="../../search.html" role="search">
|
||||
<input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
|
||||
<input type="hidden" name="check_keywords" value="yes">
|
||||
<input type="hidden" name="area" value="default">
|
||||
</form>
|
||||
<div id="searchbox"></div><div class="sidebar-scroll"><div class="sidebar-tree">
|
||||
<p class="caption" role="heading"><span class="caption-text">Getting Started</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../quickstart/installation.html">Installation</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../quickstart/first_nerf.html">Training your first model</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../quickstart/existing_dataset.html">Using existing data</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../quickstart/custom_dataset.html">Using custom data</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../quickstart/viewer_quickstart.html">Using the viewer</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../quickstart/export_geometry.html">Export geometry</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../quickstart/data_conventions.html">Data conventions</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../reference/contributing.html">Contributing</a></li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">Extensions</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../extensions/blender_addon.html">Blender VFX add-on</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../extensions/maya_plugin.html">Autodesk Maya Plug-in</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../extensions/unreal_engine.html">Exporting to Unreal Engine</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../extensions/sdfstudio.html">SDFStudio</a></li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">NeRFology</span></p>
|
||||
<ul class="current">
|
||||
<li class="toctree-l1 current has-children"><a class="reference internal" href="index.html">Methods</a><input checked="" class="toctree-checkbox" id="toctree-checkbox-1" name="toctree-checkbox-1" role="switch" type="checkbox"/><label for="toctree-checkbox-1"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul class="current">
|
||||
<li class="toctree-l2"><a class="reference internal" href="instant_ngp.html"> Instant-NGP</a></li>
|
||||
<li class="toctree-l2 current current-page"><a class="current reference internal" href="#"> Splatfacto</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="splatw.html"> Splatfacto-W</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="in2n.html"> Instruct-NeRF2NeRF</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="igs2gs.html"> Instruct-GS2GS</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="signerf.html"> SIGNeRF</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="kplanes.html"> K-Planes</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="lerf.html"> LERF</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="livescene.html"> LiveScene</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="feature_splatting.html"> Feature-Splatting</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="mipnerf.html"> Mip-NeRF</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="nerf.html"> NeRF</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="nerfacto.html"> Nerfacto</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="nerfbusters.html"> Nerfbusters</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="nerfplayer.html"> NeRFPlayer</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="tetranerf.html"> Tetra-NeRF</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="tensorf.html"> TensoRF</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="generfacto.html"> Generfacto</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="pynerf.html"> PyNeRF</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="seathru_nerf.html"> SeaThru-NeRF</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="zipnerf.html"> Zip-NeRF</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="bionerf.html"> BioNeRF</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="nerf2gs2nerf.html"> NeRFtoGSandBack</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="opennerf.html"> OpenNeRF</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1 has-children"><a class="reference internal" href="../model_components/index.html">Model components</a><input class="toctree-checkbox" id="toctree-checkbox-2" name="toctree-checkbox-2" role="switch" type="checkbox"/><label for="toctree-checkbox-2"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../model_components/visualize_cameras.html"> Cameras models</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../model_components/visualize_samples.html"> Sample representation</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../model_components/visualize_samplers.html"> Ray samplers</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../model_components/visualize_spatial_distortions.html"> Spatial distortions</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../model_components/visualize_encoders.html"> Encoders</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">Developer Guides</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../developer_guides/new_methods.html">Adding a New Method</a></li>
|
||||
<li class="toctree-l1 has-children"><a class="reference internal" href="../../developer_guides/pipelines/index.html">Pipelines overview</a><input class="toctree-checkbox" id="toctree-checkbox-3" name="toctree-checkbox-3" role="switch" type="checkbox"/><label for="toctree-checkbox-3"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l2 has-children"><a class="reference internal" href="../../developer_guides/pipelines/dataparsers.html">DataParsers</a><input class="toctree-checkbox" id="toctree-checkbox-4" name="toctree-checkbox-4" role="switch" type="checkbox"/><label for="toctree-checkbox-4"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/data/dataparsers.html">Data Parsers</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../developer_guides/pipelines/datamanagers.html">DataManagers</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../developer_guides/pipelines/models.html">Models</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../developer_guides/pipelines/fields.html">Fields</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../developer_guides/pipelines/pipelines.html">Pipelines</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1 has-children"><a class="reference internal" href="../../developer_guides/viewer/index.html">Viewer</a><input class="toctree-checkbox" id="toctree-checkbox-5" name="toctree-checkbox-5" role="switch" type="checkbox"/><label for="toctree-checkbox-5"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../developer_guides/viewer/custom_gui.html">Custom GUI</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../developer_guides/viewer/viewer_control.html">Python Viewer Control</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../developer_guides/viewer/local_viewer.html">(Legacy Viewer) Local Server</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../../developer_guides/config.html">Customizable configs</a></li>
|
||||
<li class="toctree-l1 has-children"><a class="reference internal" href="../../developer_guides/debugging_tools/index.html">Debugging tools</a><input class="toctree-checkbox" id="toctree-checkbox-6" name="toctree-checkbox-6" role="switch" type="checkbox"/><label for="toctree-checkbox-6"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../developer_guides/debugging_tools/local_logger.html">Local writer</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../developer_guides/debugging_tools/profiling.html">Code profiling support</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../developer_guides/debugging_tools/benchmarking.html">Benchmarking workflow</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">Reference</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1 has-children"><a class="reference internal" href="../../reference/cli/index.html">CLI</a><input class="toctree-checkbox" id="toctree-checkbox-7" name="toctree-checkbox-7" role="switch" type="checkbox"/><label for="toctree-checkbox-7"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/cli/ns_install_cli.html">ns-install-cli</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/cli/ns_process_data.html">ns-process-data</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/cli/ns_download_data.html">ns-download-data</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/cli/ns_train.html">ns-train</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/cli/ns_render.html">ns-render</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/cli/ns_viewer.html">ns-viewer</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/cli/ns_export.html">ns-export</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/cli/ns_eval.html">ns-eval</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1 has-children"><a class="reference internal" href="../../reference/api/index.html">API</a><input class="toctree-checkbox" id="toctree-checkbox-8" name="toctree-checkbox-8" role="switch" type="checkbox"/><label for="toctree-checkbox-8"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/api/cameras.html">Cameras</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/api/config.html">Configs</a></li>
|
||||
<li class="toctree-l2 has-children"><a class="reference internal" href="../../reference/api/data/index.html">Data</a><input class="toctree-checkbox" id="toctree-checkbox-9" name="toctree-checkbox-9" role="switch" type="checkbox"/><label for="toctree-checkbox-9"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/data/dataparsers.html">Data Parsers</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/data/datamanagers.html">Datamanagers</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/data/datasets.html">Datasets</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/data/utils.html">Utils</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/api/fields.html">Fields</a></li>
|
||||
<li class="toctree-l2 has-children"><a class="reference internal" href="../../reference/api/field_components/index.html">Field Modules</a><input class="toctree-checkbox" id="toctree-checkbox-10" name="toctree-checkbox-10" role="switch" type="checkbox"/><label for="toctree-checkbox-10"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/field_components/encodings.html">Encodings</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/field_components/embeddings.html">Embeddings</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/field_components/field_heads.html">Field Heads</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/field_components/mlp.html">MLP</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/field_components/spatial_distortions.html">Spatial Distortions</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/api/models.html">Models</a></li>
|
||||
<li class="toctree-l2 has-children"><a class="reference internal" href="../../reference/api/model_components/index.html">Model components</a><input class="toctree-checkbox" id="toctree-checkbox-11" name="toctree-checkbox-11" role="switch" type="checkbox"/><label for="toctree-checkbox-11"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/model_components/ray_sampler.html">Ray Sampler</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/model_components/losses.html">Losses</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/model_components/renderers.html">Renderers</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/api/optimizers.html">Engine</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/api/plugins.html">Plugins</a></li>
|
||||
<li class="toctree-l2 has-children"><a class="reference internal" href="../../reference/api/utils/index.html">Utils</a><input class="toctree-checkbox" id="toctree-checkbox-12" name="toctree-checkbox-12" role="switch" type="checkbox"/><label for="toctree-checkbox-12"><div class="visually-hidden">Toggle child pages in navigation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/utils/colors.html">Colors</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/utils/math.html">Math</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/utils/colormaps.html">Colormaps</a></li>
|
||||
<li class="toctree-l3"><a class="reference internal" href="../../reference/api/utils/tensor_dataclass.html">TensorDataclass</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../../reference/api/viewer.html">Viewer</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</aside>
|
||||
<div class="main">
|
||||
<div class="content">
|
||||
<div class="article-container">
|
||||
<a href="#" class="back-to-top muted-link">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24">
|
||||
<path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8v12z"></path>
|
||||
</svg>
|
||||
<span>Back to top</span>
|
||||
</a>
|
||||
<div class="content-icon-container">
|
||||
|
||||
<div class="theme-toggle-container theme-toggle-content">
|
||||
<button class="theme-toggle">
|
||||
<div class="visually-hidden">Toggle Light / Dark / Auto color theme</div>
|
||||
<svg class="theme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
|
||||
<svg class="theme-icon-when-dark"><use href="#svg-moon"></use></svg>
|
||||
<svg class="theme-icon-when-light"><use href="#svg-sun"></use></svg>
|
||||
</button>
|
||||
</div>
|
||||
<label class="toc-overlay-icon toc-content-icon" for="__toc">
|
||||
<div class="visually-hidden">Toggle table of contents sidebar</div>
|
||||
<i class="icon"><svg><use href="#svg-toc"></use></svg></i>
|
||||
</label>
|
||||
</div>
|
||||
<article role="main">
|
||||
<section class="tex2jax_ignore mathjax_ignore" id="splatfacto">
|
||||
<h1>Splatfacto<a class="headerlink" href="#splatfacto" title="Permalink to this heading">#</a></h1>
|
||||
<h4>Nerfstudio's Gaussian Splatting Implementation</h4>
|
||||
<iframe width="560" height="315" src="https://www.youtube.com/embed/0yueTFx-MdQ?si=GxiYnFAeYVVl-soJ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
|
||||
<p><a class="sd-sphinx-override sd-btn sd-text-wrap sd-btn-outline-primary reference external" href="https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/"><span>Paper Website</span></a></p>
|
||||
<p><a class="reference external" href="https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/">3D Gaussian Splatting</a> was proposed in SIGGRAPH 2023 from INRIA, and is a completely different method of representing radiance fields by explicitly storing a collection of 3D volumetric gaussians. These can be “splatted”, or projected, onto a 2D image provided a camera pose, and rasterized to obtain per-pixel colors. Because rasterization is very fast on GPUs, this method can render much faster than neural representations of radiance fields.</p>
|
||||
<p>To avoid confusion with the original paper, we refer to nerfstudio’s implementation as “Splatfacto”, which will drift away from the original as more features are added. Just as Nerfacto is a blend of various different methods, Splatfacto will be a blend of different gaussian splatting methodologies.</p>
|
||||
<section id="installation">
|
||||
<h2>Installation<a class="headerlink" href="#installation" title="Permalink to this heading">#</a></h2>
|
||||
<p><a class="sd-sphinx-override sd-btn sd-text-wrap sd-btn-outline-primary reference external" href="https://docs.gsplat.studio/"><span>GSplat </span></a></p>
|
||||
<p>Nerfstudio uses <a class="reference external" href="https://github.com/nerfstudio-project/gsplat">gsplat</a> as its gaussian rasterization backend, an in-house re-implementation which is designed to be more developer friendly. This can be installed with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">gsplat</span></code>. The associated CUDA code will be compiled the first time gsplat is executed. Some users with PyTorch 2.0 have experienced issues with this, which can be resolved by either installing gsplat from source, or upgrading torch to 2.1.</p>
|
||||
</section>
|
||||
<section id="data">
|
||||
<h2>Data<a class="headerlink" href="#data" title="Permalink to this heading">#</a></h2>
|
||||
<p>Gaussian splatting works much better if you initialize it from pre-existing geometry, such as SfM points from COLMAP. COLMAP datasets or datasets from <code class="docutils literal notranslate"><span class="pre">ns-process-data</span></code> will automatically save these points and initialize gaussians on them. Other datasets currently do not support initialization, and will initialize gaussians randomly. Initializing from other data inputs (i.e. depth from phone app scanners) may be supported in the future.</p>
|
||||
<p>Because the method trains on <em>full images</em> instead of bundles of rays, there is a new datamanager in <code class="docutils literal notranslate"><span class="pre">full_images_datamanager.py</span></code> which undistorts input images, caches them, and provides single images at each train step.</p>
|
||||
</section>
|
||||
<section id="running-the-method">
|
||||
<h2>Running the Method<a class="headerlink" href="#running-the-method" title="Permalink to this heading">#</a></h2>
|
||||
<p>To run splatfacto, run <code class="docutils literal notranslate"><span class="pre">ns-train</span> <span class="pre">splatfacto</span> <span class="pre">--data</span> <span class="pre"><data></span></code>. Just like NeRF methods, the splat can be interactively viewed in the web-viewer, loaded from a checkpoint, rendered, and exported.</p>
|
||||
<p>We provide a few additional variants:</p>
|
||||
<div class="table-wrapper colwidths-auto docutils container">
|
||||
<table class="docutils align-default">
|
||||
<thead>
|
||||
<tr class="row-odd"><th class="head"><p>Method</p></th>
|
||||
<th class="head"><p>Description</p></th>
|
||||
<th class="head"><p>Memory</p></th>
|
||||
<th class="head"><p>Speed</p></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">splatfacto</span></code></p></td>
|
||||
<td><p>Default Model</p></td>
|
||||
<td><p>~6GB</p></td>
|
||||
<td><p>Fast</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">splatfacto-big</span></code></p></td>
|
||||
<td><p>More Gaussians, Higher Quality</p></td>
|
||||
<td><p>~12GB</p></td>
|
||||
<td><p>Slower</p></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<p>A full evalaution of Nerfstudio’s implementation of Gaussian Splatting against the original Inria method can be found <a class="reference external" href="https://docs.gsplat.studio/main/tests/eval.html">here</a>.</p>
|
||||
<section id="quality-and-regularization">
|
||||
<h3>Quality and Regularization<a class="headerlink" href="#quality-and-regularization" title="Permalink to this heading">#</a></h3>
|
||||
<p>The default settings provided maintain a balance between speed, quality, and splat file size, but if you care more about quality than training speed or size, you can decrease the alpha cull threshold
|
||||
(threshold to delete translucent gaussians) and disable culling after 15k steps like so: <code class="docutils literal notranslate"><span class="pre">ns-train</span> <span class="pre">splatfacto</span> <span class="pre">--pipeline.model.cull_alpha_thresh=0.005</span> <span class="pre">--pipeline.model.continue_cull_post_densification=False</span> <span class="pre">--data</span> <span class="pre"><data></span></code></p>
|
||||
<p>A common artifact in splatting is long, spikey gaussians. <a class="reference external" href="https://xpandora.github.io/PhysGaussian/">PhysGaussian</a> proposes a scale regularizer that encourages gaussians to be more evenly shaped. To enable this, set the <code class="docutils literal notranslate"><span class="pre">pipeline.model.use_scale_regularization</span></code> flag to <code class="docutils literal notranslate"><span class="pre">True</span></code>.</p>
|
||||
</section>
|
||||
</section>
|
||||
<section id="details">
|
||||
<h2>Details<a class="headerlink" href="#details" title="Permalink to this heading">#</a></h2>
|
||||
<p>For more details on the method, see the <a class="reference external" href="https://arxiv.org/abs/2308.04079">original paper</a>. Additionally, for a detailed derivation of the gradients used in the gsplat library, see <a class="reference external" href="https://arxiv.org/abs/2312.02121">here</a>.</p>
|
||||
</section>
|
||||
<section id="exporting-splats">
|
||||
<h2>Exporting splats<a class="headerlink" href="#exporting-splats" title="Permalink to this heading">#</a></h2>
|
||||
<p>Gaussian splats can be exported as a <code class="docutils literal notranslate"><span class="pre">.ply</span></code> file which are ingestable by a variety of online web viewers. You can do this via the viewer, or <code class="docutils literal notranslate"><span class="pre">ns-export</span> <span class="pre">gaussian-splat</span> <span class="pre">--load-config</span> <span class="pre"><config></span> <span class="pre">--output-dir</span> <span class="pre">exports/splat</span></code>. Currently splats can only be exported from trained splats, not from nerfacto.</p>
|
||||
<p>Nerfstudio’s splat export currently supports multiple third-party splat viewers:</p>
|
||||
<ul class="simple">
|
||||
<li><p><a class="reference external" href="https://poly.cam/tools/gaussian-splatting">Polycam Viewer</a></p></li>
|
||||
<li><p><a class="reference external" href="https://playcanvas.com/super-splat">Playcanvas SuperSplat</a></p></li>
|
||||
<li><p><a class="reference external" href="https://antimatter15.com/splat/">WebGL Viewer by antimatter15</a></p></li>
|
||||
<li><p><a class="reference external" href="https://spline.design/">Spline</a></p></li>
|
||||
<li><p><a class="reference external" href="https://github.com/mkkellogg/GaussianSplats3D">Three.js Viewer by mkkellogg</a></p></li>
|
||||
</ul>
|
||||
</section>
|
||||
<section id="faq">
|
||||
<h2>FAQ<a class="headerlink" href="#faq" title="Permalink to this heading">#</a></h2>
|
||||
<ul class="simple">
|
||||
<li><p>Can I export a mesh or pointcloud?</p></li>
|
||||
</ul>
|
||||
<p>Currently these export options are not supported, but may be in the future. Contributions are always welcome!</p>
|
||||
<ul class="simple">
|
||||
<li><p>Can I render fisheye, equirectangular, orthographic images?</p></li>
|
||||
</ul>
|
||||
<p>Currently, no. Gaussian rasterization assumes a perspective camera for its rasterization pipeline. Implementing other camera models is of interest but not currently planned.</p>
|
||||
</section>
|
||||
</section>
|
||||
|
||||
</article>
|
||||
</div>
|
||||
<footer>
|
||||
|
||||
<div class="related-pages">
|
||||
<a class="next-page" href="splatw.html">
|
||||
<div class="page-info">
|
||||
<div class="context">
|
||||
<span>Next</span>
|
||||
</div>
|
||||
<div class="title">Splatfacto in the Wild</div>
|
||||
</div>
|
||||
<svg class="furo-related-icon"><use href="#svg-arrow-right"></use></svg>
|
||||
</a>
|
||||
<a class="prev-page" href="instant_ngp.html">
|
||||
<svg class="furo-related-icon"><use href="#svg-arrow-right"></use></svg>
|
||||
<div class="page-info">
|
||||
<div class="context">
|
||||
<span>Previous</span>
|
||||
</div>
|
||||
|
||||
<div class="title">Instant-NGP</div>
|
||||
|
||||
</div>
|
||||
</a>
|
||||
</div>
|
||||
<div class="bottom-of-page">
|
||||
<div class="left-details">
|
||||
<div class="copyright">
|
||||
Copyright © 2022, nerfstudio Team
|
||||
</div>
|
||||
Made with <a href="https://www.sphinx-doc.org/">Sphinx</a> and <a class="muted-link" href="https://pradyunsg.me">@pradyunsg</a>'s
|
||||
|
||||
<a href="https://github.com/pradyunsg/furo">Furo</a>
|
||||
|
||||
</div>
|
||||
<div class="right-details">
|
||||
<div class="icons">
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</footer>
|
||||
</div>
|
||||
<aside class="toc-drawer">
|
||||
|
||||
|
||||
<div class="toc-sticky toc-scroll">
|
||||
<div class="toc-title-container">
|
||||
<span class="toc-title">
|
||||
On this page
|
||||
</span>
|
||||
</div>
|
||||
<div class="toc-tree-container">
|
||||
<div class="toc-tree">
|
||||
<ul>
|
||||
<li><a class="reference internal" href="#">Splatfacto</a><ul>
|
||||
<li><a class="reference internal" href="#installation">Installation</a></li>
|
||||
<li><a class="reference internal" href="#data">Data</a></li>
|
||||
<li><a class="reference internal" href="#running-the-method">Running the Method</a><ul>
|
||||
<li><a class="reference internal" href="#quality-and-regularization">Quality and Regularization</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><a class="reference internal" href="#details">Details</a></li>
|
||||
<li><a class="reference internal" href="#exporting-splats">Exporting splats</a></li>
|
||||
<li><a class="reference internal" href="#faq">FAQ</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
</aside>
|
||||
</div>
|
||||
</div></body>
|
||||
|
||||
</html>
|
||||
File diff suppressed because it is too large
Load diff
Binary file not shown.
Binary file not shown.
File diff suppressed because one or more lines are too long
228
research/gpu-fusion-splatting/mirrors/nvidia-vpi-main.html
Normal file
228
research/gpu-fusion-splatting/mirrors/nvidia-vpi-main.html
Normal file
|
|
@ -0,0 +1,228 @@
|
|||
<!-- HTML header for doxygen 1.8.7-->
|
||||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||||
<head>
|
||||
<!--
|
||||
* Copyright (c) 2009-2024 NVIDIA CORPORATION. All rights reserved.
|
||||
*
|
||||
* NVIDIA Corporation and its licensors retain all intellectual property
|
||||
* and proprietary rights in and to this software, related documentation
|
||||
* and any modifications thereto. Any use, reproduction, disclosure or
|
||||
* distribution of this software and related documentation without an express
|
||||
* license agreement from NVIDIA Corporation is strictly prohibited.
|
||||
-->
|
||||
<!-- OneTrust Cookies Consent Notice start for nvidia.com -->
|
||||
<script src="https://cdn.cookielaw.org/scripttemplates/otSDKStub.js" data-document-language="true" type="text/javascript" charset="UTF-8" data-domain-script="3e2b62ff-7ae7-4ac5-87c8-d5949ecafff5" ></script>
|
||||
<script type="text/javascript">
|
||||
function OptanonWrapper()
|
||||
{
|
||||
var event = new Event('bannerLoaded');
|
||||
window.dispatchEvent(event);
|
||||
}
|
||||
</script>
|
||||
<!-- OneTrust Cookies Consent Notice end for nvidia.com -->
|
||||
<script type="text/javascript" src="https://images.nvidia.com/aem-dam/Solutions/ot-js/ot-custom.js"></script>
|
||||
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
|
||||
<meta name="generator" content="Doxygen 1.9.1"/>
|
||||
<title>VPI - Vision Programming Interface: Main Page</title>
|
||||
<link href="tabs.css" rel="stylesheet" type="text/css"/>
|
||||
<link rel="icon" href="Nvidia.ico" type="image/x-icon" />
|
||||
<script type="text/javascript" src="jquery.js"></script>
|
||||
<script type="text/javascript" src="dynsections.js"></script>
|
||||
<script src="//assets.adobedtm.com/5d4962a43b79/c1061d2c5e7b/launch-191c2462b890.min.js"></script>
|
||||
<link href="navtree.css" rel="stylesheet" type="text/css"/>
|
||||
<script type="text/javascript" src="resize.js"></script>
|
||||
<script type="text/javascript" src="navtreedata.js"></script>
|
||||
<script type="text/javascript" src="navtree.js"></script>
|
||||
<link href="search/search.css" rel="stylesheet" type="text/css"/>
|
||||
<script type="text/javascript" src="search/searchdata.js"></script>
|
||||
<script type="text/javascript" src="search/search.js"></script>
|
||||
<script type="text/x-mathjax-config">
|
||||
MathJax.Hub.Config({
|
||||
extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
|
||||
jax: ["input/TeX","output/HTML-CSS"],
|
||||
});
|
||||
</script>
|
||||
<script type="text/javascript" async="async" src="https://cdn.jsdelivr.net/npm/mathjax@2/MathJax.js"></script>
|
||||
<link href="doxygen.css" rel="stylesheet" type="text/css" />
|
||||
<link href="nv_theme_dark.css" rel="stylesheet" type="text/css"/>
|
||||
<link href="vpi.css" rel="stylesheet" type="text/css"/>
|
||||
<style>
|
||||
<!--
|
||||
#body {
|
||||
#background-position: center center;
|
||||
#background-image: url(confidential.png);
|
||||
#background-repeat: no-repeat;
|
||||
#background-attachment: fixed;
|
||||
#}
|
||||
-->
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
|
||||
<div id="titlearea">
|
||||
<table width="99%" border="0" cellspacing="1" cellpadding="1">
|
||||
<tbody>
|
||||
<tr valign="middle">
|
||||
<td rowspan="2" height="44" width="19%">
|
||||
<a id="nv-logo" href="https://www.nvidia.com/"></a>
|
||||
<td width="81%" height="44">
|
||||
<h2 style="text-align:right">VPI - Vision Programming Interface</h2>
|
||||
<h4 style="text-align:right"> 4.0 Release</h4>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<!-- end header part -->
|
||||
<!-- Generated by Doxygen 1.9.1 -->
|
||||
<script type="text/javascript">
|
||||
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
|
||||
var searchBox = new SearchBox("searchBox", "search",false,'Search','.html');
|
||||
/* @license-end */
|
||||
</script>
|
||||
<script type="text/javascript" src="menudata.js"></script>
|
||||
<script type="text/javascript" src="menu.js"></script>
|
||||
<script type="text/javascript">
|
||||
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
|
||||
$(function() {
|
||||
initMenu('',true,false,'search.php','Search');
|
||||
$(document).ready(function() { init_search(); });
|
||||
});
|
||||
/* @license-end */</script>
|
||||
<div id="main-nav"></div>
|
||||
</div><!-- top -->
|
||||
<div id="side-nav" class="ui-resizable side-nav-resizable">
|
||||
<div id="nav-tree">
|
||||
<div id="nav-tree-contents">
|
||||
<div id="nav-sync" class="sync"></div>
|
||||
</div>
|
||||
</div>
|
||||
<div id="splitbar" style="-moz-user-select:none;"
|
||||
class="ui-resizable-handle">
|
||||
</div>
|
||||
</div>
|
||||
<script type="text/javascript">
|
||||
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
|
||||
$(document).ready(function(){initNavTree('index.html',''); initResizable(); });
|
||||
/* @license-end */
|
||||
</script>
|
||||
<div id="doc-content">
|
||||
<!-- window showing the filter options -->
|
||||
<div id="MSearchSelectWindow"
|
||||
onmouseover="return searchBox.OnSearchSelectShow()"
|
||||
onmouseout="return searchBox.OnSearchSelectHide()"
|
||||
onkeydown="return searchBox.OnSearchSelectKey(event)">
|
||||
</div>
|
||||
|
||||
<!-- iframe showing the search results (closed by default) -->
|
||||
<div id="MSearchResultsWindow">
|
||||
<iframe src="javascript:void(0)" frameborder="0"
|
||||
name="MSearchResults" id="MSearchResults">
|
||||
</iframe>
|
||||
</div>
|
||||
|
||||
<div class="PageDoc"><div class="header">
|
||||
<div class="headertitle">
|
||||
<div class="title">VPI - Vision Programming Interface Documentation</div> </div>
|
||||
</div><!--header-->
|
||||
<div class="contents">
|
||||
<div class="textblock"><p><b>NVIDIA® Vision Programming Interface (VPI)</b> is a software library that implements computer vision (CV) and image processing (IP) algorithms on several computing hardware platforms available in NVIDIA embedded and discrete devices. Previous releases are available: <a href="https://docs.nvidia.com/vpi/3.2/index.html" target="_blank">3.2 Release, </a><a href="https://docs.nvidia.com/vpi/3.1/index.html" target="_blank">3.1 Release, or the </a><a href="https://docs.nvidia.com/vpi/3.0/index.html" target="_blank">3.0 Release.</a></p>
|
||||
<p>VPI provides seamless access to computing hardware that must be accessed through different and sometimes incompatible APIs, such as <a href="http://opencv.org">OpenCV</a> and <a href="https://developer.nvidia.com/cuda-toolkit">NVIDIA® CUDA® SDK</a>, or for which public APIs do not exist, such as PVA (Programmable Vision Accelerator), VIC (Video and Image Compositor) and OFA (Optical Flow Accelerator).</p>
|
||||
<p>The API is designed to be easy to use, without sacrificing performance. This allows for rapid prototyping and fine tuning, significantly reducing time-to-market. It can used directly in systems based in C/C++ and Python.</p>
|
||||
<p>Within VPI, the same algorithm is implemented in different backends, such as CPU, GPU, PVA, VIC and OFA. The processing pipeline can be set up to utilize the full installed computing capacity of the target device. For example, the GPU can perform inference on one frame while the PVA and VIC preprocess the image in a subsequent frame. The CPU can perform housekeeping tasks such as updating the GUI and postprocessing images without affecting the performance of other tasks.</p>
|
||||
<p>VPI provides seamless zero-copy memory mapping among the supported device backends, depending on certain memory characteristics. For platforms that support it, zero-copy memory mapping yields a substantial increase in throughput.</p>
|
||||
<p>VPI supports easy interoperation with existing projects that make use of OpenCV and NVIDIA® CUDA® SDK libraries, among others. This allows for gradual replacement of existing computing tasks with faster VPI equivalents.</p>
|
||||
<p>Here are some examples of algorithms provided by VPI:</p>
|
||||
<table class="gallery">
|
||||
<tr>
|
||||
<td><p class="starttd"><a class="el" href="algo_stereo_disparity.html"><img src="disparity_cuda.png" alt="" width="320px" class="inline"/> </a> </p>
|
||||
<p class="intertd"><a class="el" href="algo_stereo_disparity.html">Stereo Disparity Estimator</a></p>
|
||||
<p class="endtd"></p>
|
||||
</td><td><p class="starttd"><a class="el" href="algo_klt_tracker.html"><img src="klt.jpg" alt="" width="320px" class="inline"/> </a> </p>
|
||||
<p class="intertd"><a class="el" href="algo_klt_tracker.html">KLT Feature Tracker</a></p>
|
||||
<p class="endtd"></p>
|
||||
</td><td><p class="starttd"><a class="el" href="algo_persp_warp.html"><img src="persp_corrected.jpg" alt="" width="320px" class="inline"/> </a> </p>
|
||||
<p class="intertd"><a class="el" href="algo_persp_warp.html">Perspective Warp</a></p>
|
||||
<p class="endtd"></p>
|
||||
</td></tr>
|
||||
<tr>
|
||||
<td><p class="starttd"><a class="el" href="algo_harris_corners.html"><img src="kodim08_harris_corners.png" alt="" width="320px" class="inline"/> </a> </p>
|
||||
<p class="intertd"><a class="el" href="algo_harris_corners.html">Harris Corner Detector</a></p>
|
||||
<p class="endtd"></p>
|
||||
</td><td><p class="starttd"><a class="el" href="algo_remap.html"><img src="little_planet.jpg" alt="" width="320px" class="inline"/> </a> </p>
|
||||
<p class="intertd"><a class="el" href="algo_remap.html">Remap</a></p>
|
||||
<p class="endtd"></p>
|
||||
</td><td><p class="starttd"><a class="el" href="algo_optflow_lk.html"><img src="lk_frame_0009.jpg" alt="" width="320px" class="inline"/> </a> </p>
|
||||
<p class="intertd"><a class="el" href="algo_optflow_lk.html">Pyramidal LK Optical Flow</a></p>
|
||||
<p class="endtd"></p>
|
||||
</td></tr>
|
||||
<tr>
|
||||
<td><p class="starttd"><a class="el" href="algo_ldc.html"><img src="ldc_corrected.jpg" alt="" width="320px" class="inline"/> </a> </p>
|
||||
<p class="intertd"><a class="el" href="algo_ldc.html">Lens Distortion Correction</a></p>
|
||||
<p class="endtd"></p>
|
||||
</td><td><p class="starttd"><a class="el" href="algo_optflow_dense.html">
|
||||
<img src="pedestrians_93.jpg" width=320px><br>
|
||||
<img src="denseoptflow_mv_ofa_93.jpg" width=320px>
|
||||
</a> </p>
|
||||
<p class="intertd"><a class="el" href="algo_optflow_dense.html">Dense Optical Flow</a></p>
|
||||
<p class="endtd"></p>
|
||||
</td><td><p class="starttd"><a class="el" href="algo_background_subtractor.html">
|
||||
<img src="pedestrians_93.jpg" width=320px><br>
|
||||
<img src="background_subtractor_fgmask_93.jpg" width=320px>
|
||||
</a> </p>
|
||||
<p class="intertd"><a class="el" href="algo_background_subtractor.html">Background Subtractor</a></p>
|
||||
<p class="endtd"></p>
|
||||
</td></tr>
|
||||
</table>
|
||||
</div></div><!-- PageDoc -->
|
||||
</div><!-- contents -->
|
||||
</div><!-- doc-content -->
|
||||
|
||||
<div id="nav-path" class="navpath">
|
||||
<ul>
|
||||
<li class="footer">
|
||||
Generated by NVIDIA | Wed Mar 18 2026 02:14:17 | d79ae8c4aa4cf06467feccb062f7f1efa66ae088
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="nvidia_footer">
|
||||
<div class="pane">
|
||||
<p>Corporate Info</p>
|
||||
<ul>
|
||||
<li><a href="https://www.nvidia.com/en-us/" target="_blank">NVIDIA.com HOME</a></li>
|
||||
<li><a href="https://www.nvidia.com/en-us/about-nvidia/" target="_blank">About NVIDIA</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="pane">
|
||||
<p>NVIDIA Developer</p>
|
||||
<ul>
|
||||
<li><a href="https://developer.nvidia.com/" target="_blank">Developer Home</a></li>
|
||||
<li><a href="https://blogs.nvidia.com/" target="_blank">Blog</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="pane">
|
||||
<p>Resources</p>
|
||||
<ul>
|
||||
<li><a href="https://www.nvidia.com/en-us/contact/" target="_blank">Contact Us</a></li>
|
||||
<li><a href="https://developer.nvidia.com/developer-program" target="_blank">Developer Program</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="bottom">
|
||||
<p>
|
||||
<a href="https://www.nvidia.com/en-us/about-nvidia/privacy-policy/" target="_blank" rel="noopener">Privacy Policy </a>
|
||||
| <a href="https://www.nvidia.com/en-us/about-nvidia/privacy-center/" target="_blank" rel="noopener">Manage My Privacy </a>
|
||||
| <a href="https://www.nvidia.com/en-us/preferences/start/" target="_blank" rel="noopener">Do Not Sell or Share My Data</a>
|
||||
| <a href="https://www.nvidia.com/en-us/about-nvidia/terms-of-service/" target="_blank" rel="noopener">Terms of Service</a>
|
||||
| <a href="https://www.nvidia.com/en-us/about-nvidia/accessibility/" target="_blank" rel="noopener">Accessibility</a>
|
||||
| <a href="https://www.nvidia.com/en-us/about-nvidia/company-policies/" target="_blank" rel="noopener">Corporate Policies</a>
|
||||
| <a href="https://www.nvidia.com/en-us/product-security/" target="_blank" rel="noopener">Product Security</a>
|
||||
| <a href="https://www.nvidia.com/en-us/contact/" target="_blank" rel="noopener">Contact</a>
|
||||
</p>
|
||||
<p>Copyright © 2026 NVIDIA Corporation</p>
|
||||
</div>
|
||||
</div>
|
||||
<script type="text/javascript">_satellite.pageBottom();</script>
|
||||
</body>
|
||||
</html>
|
||||
161
research/gpu-fusion-splatting/mirrors/opencv-cuda-module.html
Normal file
161
research/gpu-fusion-splatting/mirrors/opencv-cuda-module.html
Normal file
|
|
@ -0,0 +1,161 @@
|
|||
<!-- HTML header for doxygen 1.8.6-->
|
||||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||||
<head>
|
||||
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
|
||||
<meta name="generator" content="Doxygen 1.8.13"/>
|
||||
<title>OpenCV: CUDA-accelerated Computer Vision</title>
|
||||
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
|
||||
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
|
||||
<script type="text/javascript" src="../../jquery.js"></script>
|
||||
<script type="text/javascript" src="../../dynsections.js"></script>
|
||||
<script type="text/javascript" src="../../tutorial-utils.js"></script>
|
||||
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
|
||||
<script type="text/javascript" src="../../search/searchdata.js"></script>
|
||||
<script type="text/javascript" src="../../search/search.js"></script>
|
||||
<script type="text/x-mathjax-config">
|
||||
MathJax.Hub.Config({
|
||||
extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
|
||||
jax: ["input/TeX","output/HTML-CSS"],
|
||||
});
|
||||
//<![CDATA[
|
||||
MathJax.Hub.Config(
|
||||
{
|
||||
TeX: {
|
||||
Macros: {
|
||||
matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
|
||||
fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
|
||||
forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
|
||||
forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
|
||||
vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
|
||||
vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
|
||||
cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
|
||||
distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
|
||||
distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
|
||||
hdotsfor: ["\\dots", 1],
|
||||
mathbbm: ["\\mathbb{#1}", 1],
|
||||
bordermatrix: ["\\matrix{#1}", 1]
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
//]]>
|
||||
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
|
||||
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
|
||||
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
|
||||
</head>
|
||||
<body><a href="https://docs.opencv.org/cdn-cgi/content?id=oFKfAIHR0W6Mbw7NszErWCnbIg37jbP9BmlR1Co_AdA-1779107439.5246308-1.0.1.1-SMtIuXwfmjOs9kWZGlSqjq2aBiqgfqRUAS06_qnNeNA" aria-hidden="true" rel="nofollow noopener" style="display: none !important; visibility: hidden !important"></a>
|
||||
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
|
||||
<div id="titlearea">
|
||||
<script>
|
||||
(function() {
|
||||
var cx = '002541620211387084530:kaexgxg7oxu';
|
||||
var gcse = document.createElement('script');
|
||||
gcse.type = 'text/javascript';
|
||||
gcse.async = true;
|
||||
gcse.src = 'https://cse.google.com/cse.js?cx=' + cx;
|
||||
var s = document.getElementsByTagName('script')[0];
|
||||
s.parentNode.insertBefore(gcse, s);
|
||||
})();
|
||||
</script>
|
||||
<gcse:search></gcse:search>
|
||||
<script type="text/javascript" src="/version.js"></script>
|
||||
|
||||
<table cellspacing="0" cellpadding="0">
|
||||
<tbody>
|
||||
<tr style="height: 56px;">
|
||||
<td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
|
||||
<td style="padding-left: 0.5em;">
|
||||
<div id="projectname">OpenCV
|
||||
 <span id="projectnumber">3.4.20-dev</span>
|
||||
</div>
|
||||
<div id="projectbrief">Open Source Computer Vision</div>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<!-- end header part -->
|
||||
<!-- Generated by Doxygen 1.8.13 -->
|
||||
<script type="text/javascript">
|
||||
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
|
||||
</script>
|
||||
<script type="text/javascript" src="../../menudata.js"></script>
|
||||
<script type="text/javascript" src="../../menu.js"></script>
|
||||
<script type="text/javascript">
|
||||
$(function() {
|
||||
initMenu('../../',true,false,'search.php','Search');
|
||||
$(document).ready(function() { init_search(); });
|
||||
});
|
||||
</script>
|
||||
<div id="main-nav"></div>
|
||||
</div><!-- top -->
|
||||
<!-- window showing the filter options -->
|
||||
<div id="MSearchSelectWindow"
|
||||
onmouseover="return searchBox.OnSearchSelectShow()"
|
||||
onmouseout="return searchBox.OnSearchSelectHide()"
|
||||
onkeydown="return searchBox.OnSearchSelectKey(event)">
|
||||
</div>
|
||||
|
||||
<!-- iframe showing the search results (closed by default) -->
|
||||
<div id="MSearchResultsWindow">
|
||||
<iframe src="javascript:void(0)" frameborder="0"
|
||||
name="MSearchResults" id="MSearchResults">
|
||||
</iframe>
|
||||
</div>
|
||||
|
||||
<div class="header">
|
||||
<div class="summary">
|
||||
<a href="#groups">Modules</a> </div>
|
||||
<div class="headertitle">
|
||||
<div class="title">CUDA-accelerated Computer Vision</div> </div>
|
||||
</div><!--header-->
|
||||
<div class="contents">
|
||||
<table class="memberdecls">
|
||||
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a name="groups"></a>
|
||||
Modules</h2></td></tr>
|
||||
<tr class="memitem:d7/d05/group__cudacore"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../d7/d05/group__cudacore.html">Core part</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:d5/d8e/group__cudaarithm"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../d5/d8e/group__cudaarithm.html">Operations on Matrices</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:d6/d17/group__cudabgsegm"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../d6/d17/group__cudabgsegm.html">Background Segmentation</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:d0/d61/group__cudacodec"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../d0/d61/group__cudacodec.html">Video Encoding/Decoding</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:d6/d1d/group__cudafeatures2d"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../d6/d1d/group__cudafeatures2d.html">Feature Detection and Description</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:dc/d66/group__cudafilters"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../dc/d66/group__cudafilters.html">Image Filtering</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:d0/d05/group__cudaimgproc"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../d0/d05/group__cudaimgproc.html">Image Processing</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:d5/dc3/group__cudalegacy"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../d5/dc3/group__cudalegacy.html">Legacy support</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:d9/d3f/group__cudaobjdetect"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../d9/d3f/group__cudaobjdetect.html">Object Detection</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:d7/d3f/group__cudaoptflow"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../d7/d3f/group__cudaoptflow.html">Optical Flow</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:dd/d47/group__cudastereo"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../dd/d47/group__cudastereo.html">Stereo Correspondence</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:db/d29/group__cudawarping"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../db/d29/group__cudawarping.html">Image Warping</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
<tr class="memitem:df/dfc/group__cudev"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="../../df/dfc/group__cudev.html">Device layer</a></td></tr>
|
||||
<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
|
||||
</table>
|
||||
<a name="details" id="details"></a><h2 class="groupheader">Detailed Description</h2>
|
||||
</div><!-- contents -->
|
||||
<!-- HTML footer for doxygen 1.8.6-->
|
||||
<!-- start footer part -->
|
||||
<hr class="footer"/><address class="footer"><small>
|
||||
Generated on Tue Jun 17 2025 23:15:48 for OpenCV by  <a href="http://www.doxygen.org/index.html">
|
||||
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
|
||||
</a> 1.8.13
|
||||
</small></address>
|
||||
<script type="text/javascript">
|
||||
//<![CDATA[
|
||||
addTutorialsButtons();
|
||||
//]]>
|
||||
</script>
|
||||
<script defer src="https://static.cloudflareinsights.com/beacon.min.js/v833ccba57c9e4d2798f2e76cebdd09a11778172276447" integrity="sha512-57MDmcccJXYtNnH+ZiBwzC4jb2rvgVCEokYN+L/nLlmO8rfYT/gIpW2A569iJ/3b+0UEasghjuZH/ma3wIs/EQ==" data-cf-beacon='{"version":"2024.11.0","token":"55698296dd8c4381b9a17117972ffe0c","server_timing":{"name":{"cfCacheStatus":true,"cfEdge":true,"cfExtPri":true,"cfL4":true,"cfOrigin":true,"cfSpeedBrain":true},"location_startswith":null}}' crossorigin="anonymous"></script>
|
||||
</body>
|
||||
</html>
|
||||
BIN
research/gpu-fusion-splatting/mirrors/peng-2024-rtg-slam.pdf
Normal file
BIN
research/gpu-fusion-splatting/mirrors/peng-2024-rtg-slam.pdf
Normal file
Binary file not shown.
Binary file not shown.
BIN
research/gpu-fusion-splatting/mirrors/ye-2025-gsplat.pdf
Normal file
BIN
research/gpu-fusion-splatting/mirrors/ye-2025-gsplat.pdf
Normal file
Binary file not shown.
165
research/gpu-fusion-splatting/summary.md
Normal file
165
research/gpu-fusion-splatting/summary.md
Normal file
|
|
@ -0,0 +1,165 @@
|
|||
# GPU Fusion And Live Splat Rendering
|
||||
|
||||
## Objective
|
||||
|
||||
Design the implementation path for fusing live camera/Leap/audio observations and rendering a Dreams-like volumetric brushstroke or Gaussian-splat field with the hot path on GPU compute.
|
||||
|
||||
## Current Mechanism
|
||||
|
||||
The visual-spatial map now says the world model owns truth, while brushstroke/splat packets are render lowerings. This note adds the GPU implementation constraint:
|
||||
|
||||
```text
|
||||
capture ingress
|
||||
-> pinned host staging
|
||||
-> async GPU upload / CUDA streams
|
||||
-> GPU feature/marker/depth work
|
||||
-> GPU-friendly track/claim buffers
|
||||
-> splat/brush packet generation
|
||||
-> CUDA/Vulkan/OpenGL renderer interop
|
||||
```
|
||||
|
||||
The CPU remains the conductor: device enumeration, calibration files, graph scheduling, small control decisions, logging, and fallback. It should not be the pixel furnace.
|
||||
|
||||
## Key Findings
|
||||
|
||||
The original 3D Gaussian Splatting implementation is CUDA-backed and includes a real-time viewer requiring CUDA-capable hardware for its real-time path [GraphDeco]. Kerbl et al. use anisotropic Gaussians, adaptive density control, and a fast visibility-aware renderer to avoid wasting work on empty space [Kerbl 2023].
|
||||
|
||||
NVIDIA's `vk_gaussian_splatting` sample is useful because it treats 3DGS as a real-time visualization testbed and compares graphics-pipeline approaches for splat rendering, including mesh-shader and vertex-shader paths [NVIDIA vkGS]. That suggests we should not blindly force every render step through CUDA if Vulkan/D3D mesh/compute stages do the final raster work better. GPU compute is the goal; CUDA is one implementation authority, not a religion with a tiny hat.
|
||||
|
||||
`gsplat` is the strongest practical CUDA library reference. Its documentation describes CUDA-accelerated Gaussian rasterization, and the JMLR paper frames it as a PyTorch front-end with highly optimized CUDA kernels, intended to be faster, more memory-efficient, and easier to modify than the original backend [gsplat Docs; Ye 2025]. Nerfstudio's Splatfacto uses gsplat as its rasterization backend [Nerfstudio].
|
||||
|
||||
For sensor processing, NVIDIA VPI matters because it exposes computer vision operations across hardware backends, including CUDA, and includes building blocks such as stereo disparity, KLT feature tracking, and perspective warp [NVIDIA VPI]. OpenCV's CUDA module provides GPU-accelerated computer vision primitives [OpenCV CUDA]. NVIDIA's Optical Flow SDK exposes hardware acceleration for optical flow and stereo disparity on supported GPUs [NVIDIA OFA].
|
||||
|
||||
CUDA streams and asynchronous execution are the runtime spine. NVIDIA's CUDA guide defines streams as ordered work queues for memory copies and kernels; separate streams can express overlap between capture upload, feature extraction, track update, and render prep when the hardware supports it [CUDA Async].
|
||||
|
||||
## Proposed GPU Ownership Model
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["Camera/Leap frame capture"] --> B["Pinned host frame rings"]
|
||||
B --> C["Async H2D upload streams"]
|
||||
C --> D["GPU image preprocessing"]
|
||||
D --> E["GPU marker/features/stereo/depth"]
|
||||
E --> F["GPU observation buffers"]
|
||||
G["Audio estimators"] --> H["CPU/GPU world claims"]
|
||||
F --> I["Track update / fusion buffers"]
|
||||
H --> I
|
||||
I --> J["Brush/splat packet builder"]
|
||||
J --> K["GPU splat renderer"]
|
||||
K --> L["OBS / preview / virtual camera output"]
|
||||
```
|
||||
|
||||
## Data Model
|
||||
|
||||
Prefer structure-of-arrays buffers for CUDA kernels:
|
||||
|
||||
```text
|
||||
SensorState:
|
||||
intrinsics[]
|
||||
extrinsics_world_from_sensor[]
|
||||
latency_model[]
|
||||
clock_model[]
|
||||
|
||||
ObservationBuffer:
|
||||
sensor_id[]
|
||||
timestamp_ns[]
|
||||
uv_or_xyz[]
|
||||
confidence[]
|
||||
marker_id[]
|
||||
|
||||
TrackBuffer:
|
||||
id[]
|
||||
position[]
|
||||
velocity[]
|
||||
covariance[]
|
||||
confidence[]
|
||||
last_update_time[]
|
||||
|
||||
BrushPacketBuffer:
|
||||
stable_key[]
|
||||
position[]
|
||||
orientation_or_quat[]
|
||||
scale[]
|
||||
color[]
|
||||
opacity[]
|
||||
confidence[]
|
||||
source_track_id[]
|
||||
```
|
||||
|
||||
Do not start with a generic object graph on the GPU. That path becomes pointer soup with a profiler attached.
|
||||
|
||||
## Pipeline Proposal
|
||||
|
||||
1. Capture ingress.
|
||||
Each camera writes into a small ring of pinned host buffers. Use one capture thread per physical device if the driver requires it, but keep frame metadata uniform: device id, sequence, host timestamp, exposure/gain, and intended world transform version.
|
||||
|
||||
2. Async upload.
|
||||
Copy frames into GPU memory on per-device or per-class CUDA streams. Use CUDA events to timestamp stage completion. CPU blocks only on bounded ring pressure, not on per-frame processing.
|
||||
|
||||
3. GPU image work.
|
||||
PS3 Eyes: threshold/blob/marker extraction.
|
||||
LeapUVC: stereo rectification, close-range disparity or marker extraction.
|
||||
Kiyos: downsampled feature/color projection path; keep full-res RGB for texture/projection only when budget allows.
|
||||
|
||||
4. Fusion.
|
||||
First implementation can run track updates on CPU if observation counts are tiny, but keep buffers GPU-shaped from day one. Move Kalman/EKF batch updates to CUDA once there are enough tracks or per-pixel/depth observations to justify it.
|
||||
|
||||
5. Packet generation.
|
||||
Convert tracks and local surfaces into brush/splat packets on GPU or with a CPU fallback writing the same SoA buffer. This packet buffer is the render ABI.
|
||||
|
||||
6. Render.
|
||||
Use an existing CUDA 3DGS/gsplat path for experiments, but the production live renderer may be Vulkan/D3D compute/mesh shader with CUDA interop depending on OBS/preview integration. The invariant is GPU-resident packet data, not a specific API logo.
|
||||
|
||||
7. Readback discipline.
|
||||
Read back metrics and small debug summaries only: counts, residuals, confidence, dropped frames, GPU timings. Do not read back full frames or point clouds during live operation unless recording/debugging.
|
||||
|
||||
## Gaussian vs Brushstroke
|
||||
|
||||
Classic 3DGS wants optimized radiance-field Gaussians from posed images. Our live scene wants expressive, sensor-driven volumetric strokes. Use 3DGS machinery where it helps:
|
||||
|
||||
- anisotropic splat parameterization
|
||||
- tile/bin/sort/raster kernels
|
||||
- packed GPU buffers
|
||||
- visibility-aware culling
|
||||
- LOD/density control ideas
|
||||
|
||||
But do not require full offline radiance-field training before the live rig can show anything. V1 brush packets can be directly emitted from tracks and RGB projections. Later, local regions can be optimized into Gaussian fields when the data is stable enough.
|
||||
|
||||
## First GPU Slice
|
||||
|
||||
1. GPU-upload one PS3 Eye frame and one Kiyo frame using pinned host rings.
|
||||
2. Run a CUDA threshold/blob extraction kernel on the PS3 Eye frame.
|
||||
3. Emit one `ObservationBuffer` entry for a bright marker.
|
||||
4. Triangulate on CPU or GPU from two PS3 Eye observations.
|
||||
5. Write one `BrushPacketBuffer` entry.
|
||||
6. Render one translucent anisotropic splat/brush from that GPU buffer.
|
||||
7. Expose timings: capture, upload, detect, fuse, packetize, render.
|
||||
|
||||
That proves the GPU memory path before the larger system starts wearing a cape.
|
||||
|
||||
## Cut Line
|
||||
|
||||
Cut any plan that:
|
||||
|
||||
- captures frames, processes pixels on CPU, then uploads final pretty geometry only
|
||||
- requires CPU readback of dense point clouds every frame
|
||||
- starts with full 4D Gaussian training before sparse live tracking works
|
||||
- makes CUDA kernels own calibration semantics
|
||||
- ties the world model to one renderer backend
|
||||
- ignores USB/capture bandwidth and upload overlap
|
||||
|
||||
## Citations
|
||||
|
||||
- [CUDA Guide] NVIDIA CUDA C++ Programming Guide. Mirror: `mirrors/nvidia-cuda-programming-guide.pdf`.
|
||||
- [CUDA Async] NVIDIA CUDA C++ Programming Guide, asynchronous/concurrent execution. Mirror: `mirrors/nvidia-cuda-async-concurrent-execution.html`.
|
||||
- [NVIDIA VPI] NVIDIA Vision Programming Interface. Mirror: `mirrors/nvidia-vpi-main.html`.
|
||||
- [NVIDIA OFA] NVIDIA Optical Flow SDK Programming Guide. Mirror: `mirrors/nvidia-optical-flow-programming-guide.pdf`.
|
||||
- [OpenCV CUDA] OpenCV CUDA-accelerated Computer Vision module. Mirror: `mirrors/opencv-cuda-module.html`.
|
||||
- [GraphDeco] GraphDeco/Inria `gaussian-splatting`. Mirrors: `mirrors/graphdeco-gaussian-splatting-github.html`, `mirrors/graphdeco-gaussian-splatting-readme.md`.
|
||||
- [NVIDIA vkGS] NVIDIA Developer Blog, `vk_gaussian_splatting`. Mirror: `mirrors/nvidia-vk-gaussian-splatting-blog.html`.
|
||||
- [gsplat Docs] gsplat documentation. Mirror: `mirrors/gsplat-docs.html`.
|
||||
- [Ye 2025] M. Ye et al., "gsplat: An Open-Source Library for Gaussian Splatting." Mirror: `mirrors/ye-2025-gsplat.pdf`.
|
||||
- [Nerfstudio] Nerfstudio, "Splatfacto." Mirror: `mirrors/nerfstudio-splatfacto.html`.
|
||||
- [Kerbl 2023] Bernhard Kerbl et al., "3D Gaussian Splatting for Real-Time Radiance Field Rendering." Mirror: `mirrors/kerbl-2023-3d-gaussian-splatting.pdf`.
|
||||
- [Wu 2024] Guanjun Wu et al., "4D Gaussian Splatting for Real-Time Dynamic Scene Rendering." Mirror: `mirrors/wu-2024-4d-gaussian-splatting.pdf`.
|
||||
- [Peng 2024] Zhexi Peng et al., "RTG-SLAM." Mirror: `mirrors/peng-2024-rtg-slam.pdf`.
|
||||
36
research/live-adaptive-sync/bibliography.md
Normal file
36
research/live-adaptive-sync/bibliography.md
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
# Bibliography: Live Adaptive Sync
|
||||
|
||||
Mirrored copies live under `research/live-adaptive-sync/mirrors/`.
|
||||
|
||||
## Sampling-Rate Offset / Clock Drift
|
||||
|
||||
- L. Wang and S. Doclo, "Correlation Maximization-Based Sampling Rate Offset Estimation for Distributed Microphone Arrays," IEEE/ACM TASLP, 2016.
|
||||
Mirror: `mirrors/wang-doclo-2016-correlation-maximization-sro.pdf`
|
||||
|
||||
- J. Schmalenstroeer and R. Haeb-Umbach, "Sampling Rate Synchronisation in Acoustic Sensor Networks with a Pre-Trained Clock Skew Error Model," 2013.
|
||||
Mirror: `mirrors/schmalenstroeer-haeb-umbach-2013-pretrained-clock-skew.pdf`
|
||||
|
||||
- P. Didier, T. van Waterschoot, S. Doclo, and M. Moonen, "Sampling Rate Offset Estimation and Compensation for Distributed Adaptive Node-Specific Signal Estimation in Wireless Acoustic Sensor Networks," 2023.
|
||||
Mirror: `mirrors/didier-2023-sro-estimation-compensation-wasn.pdf`
|
||||
|
||||
- "On Synchronization of Wireless Acoustic Sensor Networks in the Presence of Time-varying Sampling Rate Offsets and Speaker Changes," arXiv:2110.12820.
|
||||
Mirror: `mirrors/synchronization-time-varying-sro-speaker-changes.pdf`
|
||||
|
||||
- "Online Estimation of Sampling Rate Offsets," EUSIPCO 2021.
|
||||
Mirror: `mirrors/online-estimation-sampling-rate-offsets-2021.pdf`
|
||||
|
||||
## Delay / Temporal Alignment / Localization
|
||||
|
||||
- A. Shrestha et al., "Temporal Synchronization of Multiple Audio Signals," Google Research.
|
||||
Mirror: `mirrors/google-temporal-synchronization-multiple-audio.html`
|
||||
|
||||
- F. Grondin and J. Glass, "A Study of the Complexity and Accuracy of Direction of Arrival Estimation Methods Based on GCC-PHAT for a Pair of Close Microphones," arXiv:1811.11787, 2018.
|
||||
Mirror: `mirrors/grondin-glass-2018-gcc-phat-close-mics.pdf`
|
||||
|
||||
## Implementation References
|
||||
|
||||
- FFmpeg Filters Documentation, especially `aresample` / timestamp-driven audio compensation.
|
||||
Mirror: `mirrors/ffmpeg-filters.html`
|
||||
|
||||
- SoX Resampler library / libsoxr documentation, especially variable-rate resampling (`SOXR_VR`).
|
||||
Mirror: `mirrors/soxr-wiki-home.html`
|
||||
Binary file not shown.
40068
research/live-adaptive-sync/mirrors/ffmpeg-filters.html
Normal file
40068
research/live-adaptive-sync/mirrors/ffmpeg-filters.html
Normal file
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
Binary file not shown.
Binary file not shown.
Binary file not shown.
1618
research/live-adaptive-sync/mirrors/soxr-wiki-home.html
Normal file
1618
research/live-adaptive-sync/mirrors/soxr-wiki-home.html
Normal file
File diff suppressed because one or more lines are too long
Binary file not shown.
Binary file not shown.
123
research/live-adaptive-sync/summary.md
Normal file
123
research/live-adaptive-sync/summary.md
Normal file
|
|
@ -0,0 +1,123 @@
|
|||
# Live Adaptive Sync: Distilled Research
|
||||
|
||||
## Objective
|
||||
|
||||
Find prior work that supports synchronizing independent live microphone feeds well enough to build a coherent spatial bus for AquaSynth/Ambisonics.
|
||||
|
||||
## Current Mechanism
|
||||
|
||||
Mimir currently launches separate FFmpeg capture processes and SRT endpoints. That preserves OBS mix control but does not give us one clocked multichannel stream. For Ambisonics, independent device clocks create two separate problems:
|
||||
|
||||
- initial offset: one stream starts late or has more device/buffer latency
|
||||
- sampling-rate offset: one device runs slightly faster/slower, so offset grows over time
|
||||
|
||||
The research literature calls the second problem **sampling-rate offset** or **SRO**, often measured in parts per million.
|
||||
|
||||
## Main Finding
|
||||
|
||||
Live adaptive sync is a known problem in wireless/distributed acoustic sensor networks. The common model is:
|
||||
|
||||
1. estimate clock drift/SRO between independent nodes
|
||||
2. compensate by resampling, STFT-domain phase correction, or clock control when hardware permits
|
||||
3. estimate current TDOA/delay separately for localization or beamforming
|
||||
|
||||
Wang and Doclo model SRO as linear phase drift in the STFT domain and note that independent microphone nodes have their own ADC/clock sources; because oscillator tolerances differ, sampling-rate mismatch is inevitable [Wang/Doclo 2016]. They also spell out the practical compensation options: time-domain resampling, interpolation filters, sinc/Lagrange-style correction, or STFT-domain compensation [Wang/Doclo 2016].
|
||||
|
||||
Didier et al. report SROs up to hundreds of ppm in common heterogeneous devices and emphasize that SRO causes increasing time drift, breaking coherent signal processing [Didier 2023]. Their method estimates SRO from coherence drift and compensates with frequency-domain phase shifts, while also detecting full-sample drifts in a WOLA/STFT processing chain [Didier 2023].
|
||||
|
||||
Schmalenstroeer and Haeb-Umbach show the control-loop version: timestamp exchange gives a first clock-frequency estimate, a Kalman filter smooths it, and then one node's sampling frequency is adjusted; their long-term tests kept oscillator deviation under half a sample, enabling cross-device TDOA estimation [Schmalenstroeer 2013]. That is closest to "lock them into sync," but it requires either controllable clocks or an equivalent software resampling actuator.
|
||||
|
||||
## Architecture We Can Steal Without Shame
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["Raw mic streams"] --> B["Ring buffers with monotonic local timestamps"]
|
||||
C["Calibration ping / chirp"] --> D["Initial offset, polarity, gain, impulse response"]
|
||||
B --> E["Passive coherence / cross-correlation tracker"]
|
||||
D --> E
|
||||
E --> F["SRO estimator: ppm drift"]
|
||||
E --> G["Delay/TDOA estimator: current sample offset"]
|
||||
F --> H["Adaptive resampler per non-reference device"]
|
||||
G --> I["Delay line / fractional delay correction"]
|
||||
H --> I
|
||||
I --> J["Aligned multichannel frame bus"]
|
||||
J --> K["Spatial estimator / source encoder"]
|
||||
K --> L["Ambisonic bus into AquaSynth Faust DSP"]
|
||||
```
|
||||
|
||||
## Design Implications
|
||||
|
||||
- Pick one reference stream.
|
||||
One mic/interface becomes the local time authority. Every other stream is resampled toward it. This is not philosophical. Somebody has to be the clock or the whole room starts freelancing.
|
||||
|
||||
- Separate the slow and fast loops.
|
||||
SRO is a slow ppm-level estimate. Current TDOA/delay is a frame-level estimate. If one loop owns both, it will overreact to speech movement and reverberation.
|
||||
|
||||
- Use pings for initialization and health checks, not as the only sync.
|
||||
A chirp/MLS/pulse can estimate fixed latency, polarity, gain, and rough impulse response. It does not prevent future drift. Periodic pings can re-anchor the system, but they are intrusive unless hidden in a controlled band and supported by the microphones/signal path.
|
||||
|
||||
- Prefer passive SRO tracking during speech.
|
||||
Coherence-drift and STFT phase-drift methods are designed for exactly this: estimate rate mismatch from live audio without requiring dedicated sync tones [Wang/Doclo 2016; Didier 2023].
|
||||
|
||||
- Use GCC-PHAT for TDOA/localization, not clock locking.
|
||||
GCC-PHAT is a standard TDOA/DOA tool and has real-time variants studied for close microphone pairs [Grondin/Glass 2018]. It estimates arrival-time differences; it does not by itself distinguish source movement from clock drift over long windows.
|
||||
|
||||
- Resampling is the actuator.
|
||||
If hardware clocks cannot be controlled, the software actuator is adaptive sample-rate conversion. FFmpeg's `aresample` can stretch/squeeze audio to match timestamps, and libsoxr supports variable-rate resampling via `SOXR_VR` [FFmpeg Filters; SoXR]. For this project, a custom live engine will probably need an explicit variable-ratio resampler rather than hoping FFmpeg guesses the right timestamps.
|
||||
|
||||
## Candidate V1 Algorithm
|
||||
|
||||
1. Capture every mic into its own ring buffer at nominal 48 kHz.
|
||||
2. Select the shielded cardioid or a stable interface channel as reference.
|
||||
3. Run startup calibration:
|
||||
- play/capture broadband chirp or sharp pulse
|
||||
- estimate fixed delay per mic by cross-correlation
|
||||
- estimate polarity and gain
|
||||
- store confidence and impulse-response sketch
|
||||
4. Process live in frames, probably 10-20 ms with overlap for analysis.
|
||||
5. For each non-reference stream:
|
||||
- estimate short-window delay with GCC-PHAT/cross-correlation
|
||||
- estimate longer-window SRO from phase/coherence drift
|
||||
- smooth SRO with a Kalman/PLL-style filter
|
||||
- drive a variable-rate resampler plus fractional delay line
|
||||
6. Emit aligned multichannel frames into the spatial encoder.
|
||||
7. Track health:
|
||||
- estimated ppm
|
||||
- residual delay error
|
||||
- correlation/coherence confidence
|
||||
- dropouts and buffer pressure
|
||||
- "sync lost" state when confidence collapses
|
||||
|
||||
## Failure Modes
|
||||
|
||||
- Moving speakers can look like drift if the estimator uses the wrong window.
|
||||
- Reverberation and camera mic processing can corrupt phase/TDOA estimates.
|
||||
- USB/camera mics may apply AGC, noise suppression, compression, or hidden resampling.
|
||||
- Bluetooth paths are probably disqualified for coherent spatial work.
|
||||
- Ultrasonic pings may be eaten by mic frequency response, codecs, driver filtering, or camera audio processing.
|
||||
- Separate FFmpeg processes may not expose enough timing authority for tight adaptive control.
|
||||
|
||||
## Recommended Next Cut
|
||||
|
||||
Do not start with a full arbitrary-array ambisonic encoder. First build a sync harness:
|
||||
|
||||
- two live mic inputs
|
||||
- startup chirp calibration
|
||||
- live SRO estimate in ppm
|
||||
- adaptive resampler correction
|
||||
- residual delay meter
|
||||
- recorded before/after WAV evidence
|
||||
|
||||
Once two mics can stay aligned for 20-30 minutes, expand to four. Only then should the output feed the Ambisonic/AquaSynth graph. Spatial DSP built on drifting channels is just a confident hallucination with XLR cables.
|
||||
|
||||
## Citations
|
||||
|
||||
- [Wang/Doclo 2016] L. Wang and S. Doclo, "Correlation Maximization-Based Sampling Rate Offset Estimation for Distributed Microphone Arrays." Mirror: `mirrors/wang-doclo-2016-correlation-maximization-sro.pdf`.
|
||||
- [Schmalenstroeer 2013] J. Schmalenstroeer and R. Haeb-Umbach, "Sampling Rate Synchronisation in Acoustic Sensor Networks with a Pre-Trained Clock Skew Error Model." Mirror: `mirrors/schmalenstroeer-haeb-umbach-2013-pretrained-clock-skew.pdf`.
|
||||
- [Didier 2023] P. Didier, T. van Waterschoot, S. Doclo, and M. Moonen, "Sampling Rate Offset Estimation and Compensation for Distributed Adaptive Node-Specific Signal Estimation in Wireless Acoustic Sensor Networks." Mirror: `mirrors/didier-2023-sro-estimation-compensation-wasn.pdf`.
|
||||
- [Time-varying SRO] "On Synchronization of Wireless Acoustic Sensor Networks in the Presence of Time-varying Sampling Rate Offsets and Speaker Changes." Mirror: `mirrors/synchronization-time-varying-sro-speaker-changes.pdf`.
|
||||
- [Online SRO 2021] "Online Estimation of Sampling Rate Offsets." Mirror: `mirrors/online-estimation-sampling-rate-offsets-2021.pdf`.
|
||||
- [Google Sync] A. Shrestha et al., "Temporal Synchronization of Multiple Audio Signals." Mirror: `mirrors/google-temporal-synchronization-multiple-audio.html`.
|
||||
- [Grondin/Glass 2018] F. Grondin and J. Glass, "A Study of the Complexity and Accuracy of Direction of Arrival Estimation Methods Based on GCC-PHAT for a Pair of Close Microphones." Mirror: `mirrors/grondin-glass-2018-gcc-phat-close-mics.pdf`.
|
||||
- [FFmpeg Filters] FFmpeg Filters Documentation. Mirror: `mirrors/ffmpeg-filters.html`.
|
||||
- [SoXR] SoX Resampler library / libsoxr wiki. Mirror: `mirrors/soxr-wiki-home.html`.
|
||||
28
research/playback-calibration/bibliography.md
Normal file
28
research/playback-calibration/bibliography.md
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
# Bibliography: Playback Calibration And Loudspeaker Rendering
|
||||
|
||||
Mirrored copies live under `research/playback-calibration/mirrors/`.
|
||||
|
||||
## Ambisonic Loudspeaker Decoding
|
||||
|
||||
- [IEM Plug-in Suite, "AllRADecoder Guide"](https://plugins.iem.at/docs/allradecoder/)
|
||||
Mirror: `mirrors/iem-allradecoder-guide.html`
|
||||
|
||||
- [IEM Plug-in Suite, "Configuration Files"](https://plugins.iem.at/docs/configurationfiles/)
|
||||
Mirror: `mirrors/iem-configuration-files.html`
|
||||
|
||||
- [IEM Plug-in Suite, "Plug-in Descriptions"](https://plugins.iem.at/docs/plugindescriptions/)
|
||||
Mirror: `mirrors/iem-plugin-descriptions.html`
|
||||
|
||||
- [RingBuffer, "Decoding to Loudspeaker Setups"](https://ringbuffer.org/spatial_audio/iem-reaper/decoding-loudspeaker-iem/)
|
||||
Mirror: `mirrors/ringbuffer-decoding-loudspeaker-iem.html`
|
||||
|
||||
- [SPARTA, "Overview"](https://leomccormack.github.io/sparta-site/docs/plugins/overview/)
|
||||
Mirror: `mirrors/sparta-overview.html`
|
||||
|
||||
- [ICST Ambisonics, "How it Works"](https://ambisonics.ch/icst-ambisonics-plugins/03_how_it_works/)
|
||||
Mirror: `mirrors/icst-how-it-works.html`
|
||||
|
||||
## Measurement
|
||||
|
||||
- [FLUX:: MiRA, "Impulse response measurement"](https://doc.flux.audio/mira/Impulse_response_measurement.html)
|
||||
Mirror: `mirrors/flux-mira-impulse-response-measurement.html`
|
||||
File diff suppressed because it is too large
Load diff
31
research/playback-calibration/mirrors/icst-how-it-works.html
Normal file
31
research/playback-calibration/mirrors/icst-how-it-works.html
Normal file
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,185 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en-us">
|
||||
|
||||
<head>
|
||||
<meta name="generator" content="Hugo 0.119.0">
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title> AllRADecoder Guide </title>
|
||||
|
||||
|
||||
<meta name="description" content="How to create a decoder using the AllRADecoder plug-in ">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta name="author" content="Daniel Rudrich">
|
||||
|
||||
|
||||
<meta property="og:title" content="AllRADecoder Guide" />
|
||||
<meta property="og:description" content="How to create a decoder using the AllRADecoder plug-in " />
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://plugins.iem.at/docs/allradecoder/" /><meta property="og:image" content="https://plugins.iem.at/images/IEMPluginSuiteLogo.png"/><meta property="article:section" content="docs" />
|
||||
<meta property="article:published_time" content="2017-12-18T11:26:04+01:00" />
|
||||
<meta property="article:modified_time" content="2017-12-18T11:26:04+01:00" />
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta property="og:image" content="images/IEMPluginSuiteLogo.png">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="canonical" href="https://plugins.iem.at/docs/allradecoder/">
|
||||
|
||||
<link rel="shortcut icon" type="image/png" href="images/favicon_96x96.png">
|
||||
|
||||
|
||||
<link href="https://plugins.iem.at/css/font.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.min.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.legenda.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/highlight.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/master.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.demo.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<link href="https://plugins.iem.at/css/custom.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<script src="https://plugins.iem.at/js/jquery-3.5.1.min.js" type="text/javascript">
|
||||
</script>
|
||||
|
||||
<script type="text/javascript" src="https://plugins.iem.at/js/tocbot.min.js"></script>
|
||||
</head>
|
||||
|
||||
|
||||
<body class="page-kube">
|
||||
<header> <div class="show-sm">
|
||||
<div id="nav-toggle-box">
|
||||
<div id="nav-toggle-brand">
|
||||
<a href="https://plugins.iem.at/">IEM Plug-in Suite</a>
|
||||
</div><a data-component="toggleme" data-target="#top" href="#" id="nav-toggle"><i class="kube-menu"></i></a>
|
||||
</div>
|
||||
</div>
|
||||
<div class="hide-sm" id="top">
|
||||
<div id="top-brand">
|
||||
<a href="https://plugins.iem.at/" title="home">IEM Plug-in Suite</a>
|
||||
</div>
|
||||
|
||||
<nav id="top-nav-main">
|
||||
<ul>
|
||||
|
||||
|
||||
<li><a href="https://plugins.iem.at/download/" >Download</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/docs/" >Documentation</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/faq/" >FAQ</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/contact/" >Contact</a></li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
<div id="nav-toggle" style="text-align: center;">
|
||||
<a href="https://atmoky.com"><img src="https://plugins.iem.at/images/atmokyfriends.png" width="200"></a>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
<main>
|
||||
<div id="main">
|
||||
<div id="hero">
|
||||
<h1> AllRADecoder Guide </h1>
|
||||
<p class="hero-lead">
|
||||
This guide shows how to create a decoder for any desired loudspeaker layout using the AllRAD approach.
|
||||
</p>
|
||||
|
||||
</div>
|
||||
<div id="kube-component" class="content">
|
||||
|
||||
<nav id="contents">
|
||||
<ol class="js-toc">
|
||||
</ol>
|
||||
</nav>
|
||||
<script type="text/javascript">
|
||||
document.addEventListener("DOMContentLoaded",
|
||||
function(){
|
||||
tocbot.init({
|
||||
|
||||
tocSelector: '.js-toc',
|
||||
|
||||
contentSelector: '.content',
|
||||
|
||||
headingSelector: 'h3'
|
||||
})
|
||||
}
|
||||
);
|
||||
</script>
|
||||
|
||||
|
||||
|
||||
<h3 id="general-information">General Information</h3>
|
||||
<p>This plug-in let’s you create an Ambisonic decoder for any desired loudspeaker layout and Ambisonic orders up to 7th. The resulting decoder is automatically applied to the incoming Ambisonic audio stream, so it can be directly inserted into a decoder track of your audio project. You might also want to use it as a standalone application to design a decoder and write it into a JSON configuration file for the use in other plug-ins (e.g. <strong>Simple</strong>Decoder).</p>
|
||||
<p>A theoretical description of the AllRAD approach can be found <a href="http://www.aes.org/e-lib/browse.cfm?elib=16554">here</a>.</p>
|
||||
<h3 id="how-to-start">How to start</h3>
|
||||
<p>The first thing you want to do is inserting the loudspeaker coordinates (azimuth and elevation) and the channel numbers by clicking ‘add loudspeaker’. The fields ‘radius’ and ‘gain’ will only affect imaginary loudspeakers (see below). Loudspeaker at the same level as the listener’s head should by specified by an elevation angle of 0°.</p>
|
||||
<p>You can click the <code>Noise</code>-button which will send a short noise-signal to the corresponding channel. That way you can verify whether your routing is correct.</p>
|
||||
<p>The loudspeakers will appear both in the 3D visualization (top left) and in the loudness map (bottom left). The selected loudspeaker is visualized in green and imaginary ones in orange, while the others are blue. You can rotate the 3D view by dragging, and zoom in/out using the mouse wheel. As opposed to imaginary loudspeakers, physical loudspeaker directions are always visualized with a radius of one.</p>
|
||||
<p>You can rotate the whole layout around the z-axis by clicking the rotate button. This will add a constant value to all azimuth angles.</p>
|
||||
<p>After inserting enough loudspeakers, the layout is going to be triangulated immediately and it is also displayed in the 3D visualization. Any problems with your layout will be shown in the message area below the loudspeaker list.</p>
|
||||
<h3 id="adding-imaginary-loudspeakers">Adding Imaginary Loudspeakers</h3>
|
||||
<p>Sometimes, a loudspeaker layout cannot be triangulated in a non-ambiguous way. With the AllRAD approach, imaginary loudspeakers can be added to circumvent these cases.</p>
|
||||
<p>For example, almost every layout without loudspeakers in the lower hemisphere has an ambiguous ground plane. Inserting an imaginary loudspeaker just below the listener will prevent that case. The following layout depicts that example. You also might want to set the gain of that specific imaginary loudspeaker to zero, otherwise sources from below will be reproduced by all of the floor level loudspeakers. Download the configuration file for this layout here: <a href="../../allradecoder/layoutWithOneImaginaryLoudspeaker.json">layoutWithOneImaginaryLoudspeaker.json</a></p>
|
||||
<p><img src="../../images/allradecoder/withOneImaginary.png" alt="Without imaginary loudspeakers"></p>
|
||||
<p>However, in that layout there are several other planes, which could be divided into triangles in a different way: the top plane, left and right side, and also the plane right behind the listener. Additionally, there is no way the top plane can be triangulated symmetrically. It’s advisable to add imaginary loudspeakers here, too. In the following layout four additional imaginary loudspeakers were added. The configuration can be downloaded here: <a href="../../allradecoder/layoutWithAdditionalImaginaryLoudspeakers.json">layoutWithAdditionalImaginaryLoudspeakers.json</a></p>
|
||||
<p><img src="../../images/allradecoder/withAllImaginary.png" alt="With imaginary loudspeakers"></p>
|
||||
<p>The gain is left unchanged, as one does not want silent spots in those areas. However, for loudspeaker #14 the radius had to be altered. Otherwise, there would have been a connection between two imaginary loudspeakers, which is not supported by the AllRAD approach.</p>
|
||||
<h3 id="choosing-a-suitable-ambisonic-order">Choosing a suitable Ambisonic order</h3>
|
||||
<p>The choice of the Ambisonic order is a trade-off between constant source width and smooth panning (low order) and precise localization and bumpier panning getting more similar to VBAP (high order). It’s typical to choose 2nd order for 60° spaced loudspeaker layouts, 3rd for 45°, and 5th order for 30°, respectively. 7th order is the highest you can choose and is suitable for 15° spaced layouts.</p>
|
||||
<h3 id="energy-distribution-and-re-source-width-map">Energy distribution and rE source width map</h3>
|
||||
<p>When a decoder was calculated successfully, it’s energy distribution is shown in the loudness map in the bottom left (red). Peak values are visualized in red (up to +3dB) while areas of low energy (down to -3dB) are transparent. The choice of the Ambisonic order will influence the energy distribution.</p>
|
||||
<p>When double-clicking on the map, it will display the acos-rE source width (green). The brighter the color, the wider the image of a source which is panned to that direction. In general, sources at a loudspeaker position will be smaller (less green) than sources between loudspeakers, as several loudspeaker will try to reproduce the signal.</p>
|
||||
<h3 id="export-layouts-and-decoders">Export Layouts and Decoders</h3>
|
||||
<p>You can export your layout to a JSON file by clicking the ’export’ button. Additionally, you can check the ‘Export Decoder’ checkbox to also write the calculated (don’t forget to calculate it first) decoder matrix to the configuration file, which e.g. can be used with the <strong>Simple</strong>Decoder plug-in.</p>
|
||||
<h3 id="future-features">Future Features</h3>
|
||||
<p>In future implementations there will be more analysis than the loudness (energy) map, with maps of the rE-vector-based analysis of directional mapping and perceived width. Also the AllRAD<strong>2</strong> approach will be implemented soon. It brings improvements to a panning-invariant loudness on typically irregular layout as the 4+5+0 layout from ITU BS.2051 with dense loudspeaker spacing only in front.</p>
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
<footer>
|
||||
<footer id="footer">
|
||||
<nav>
|
||||
<ul>
|
||||
<li><span>Plug-in Suite by IEM - Institute of Electronic Music and Acoustics</span></li>
|
||||
<li>
|
||||
<a href="https://iem.at/">IEM.at</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="https://plugins.iem.at/docs/">Docs</a>
|
||||
</li>
|
||||
</ul>
|
||||
</nav>
|
||||
<p>Website created with <a href="https://gohugo.io">Hugo</a> using the <a href="https://github.com/jeblister/kube">kube</a> theme.</p>
|
||||
</footer>
|
||||
|
||||
</footer>
|
||||
|
||||
|
||||
<script src="https://plugins.iem.at/js/kube.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/kube.legenda.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/master.js" type="text/javascript">
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
|
@ -0,0 +1,350 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en-us">
|
||||
|
||||
<head>
|
||||
<meta name="generator" content="Hugo 0.119.0">
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title> Configuration Files </title>
|
||||
|
||||
|
||||
<meta name="description" content="Information about the structure of the configuration files">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta name="author" content="Daniel Rudrich">
|
||||
|
||||
|
||||
<meta property="og:title" content="Configuration Files" />
|
||||
<meta property="og:description" content="Information about the structure of the configuration files" />
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://plugins.iem.at/docs/configurationfiles/" /><meta property="og:image" content="https://plugins.iem.at/images/IEMPluginSuiteLogo.png"/><meta property="article:section" content="docs" />
|
||||
<meta property="article:published_time" content="2017-12-18T11:26:04+01:00" />
|
||||
<meta property="article:modified_time" content="2017-12-18T11:26:04+01:00" />
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta property="og:image" content="images/IEMPluginSuiteLogo.png">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="canonical" href="https://plugins.iem.at/docs/configurationfiles/">
|
||||
|
||||
<link rel="shortcut icon" type="image/png" href="images/favicon_96x96.png">
|
||||
|
||||
|
||||
<link href="https://plugins.iem.at/css/font.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.min.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.legenda.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/highlight.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/master.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.demo.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<link href="https://plugins.iem.at/css/custom.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<script src="https://plugins.iem.at/js/jquery-3.5.1.min.js" type="text/javascript">
|
||||
</script>
|
||||
|
||||
<script type="text/javascript" src="https://plugins.iem.at/js/tocbot.min.js"></script>
|
||||
</head>
|
||||
|
||||
|
||||
<body class="page-kube">
|
||||
<header> <div class="show-sm">
|
||||
<div id="nav-toggle-box">
|
||||
<div id="nav-toggle-brand">
|
||||
<a href="https://plugins.iem.at/">IEM Plug-in Suite</a>
|
||||
</div><a data-component="toggleme" data-target="#top" href="#" id="nav-toggle"><i class="kube-menu"></i></a>
|
||||
</div>
|
||||
</div>
|
||||
<div class="hide-sm" id="top">
|
||||
<div id="top-brand">
|
||||
<a href="https://plugins.iem.at/" title="home">IEM Plug-in Suite</a>
|
||||
</div>
|
||||
|
||||
<nav id="top-nav-main">
|
||||
<ul>
|
||||
|
||||
|
||||
<li><a href="https://plugins.iem.at/download/" >Download</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/docs/" >Documentation</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/faq/" >FAQ</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/contact/" >Contact</a></li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
<div id="nav-toggle" style="text-align: center;">
|
||||
<a href="https://atmoky.com"><img src="https://plugins.iem.at/images/atmokyfriends.png" width="200"></a>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
<main>
|
||||
<div id="main">
|
||||
<div id="hero">
|
||||
<h1> Configuration Files </h1>
|
||||
<p class="hero-lead">
|
||||
This guide shows how the configuration files are structured.
|
||||
</p>
|
||||
|
||||
</div>
|
||||
<div id="kube-component" class="content">
|
||||
|
||||
<nav id="contents">
|
||||
<ol class="js-toc">
|
||||
</ol>
|
||||
</nav>
|
||||
<script type="text/javascript">
|
||||
document.addEventListener("DOMContentLoaded",
|
||||
function(){
|
||||
tocbot.init({
|
||||
|
||||
tocSelector: '.js-toc',
|
||||
|
||||
contentSelector: '.content',
|
||||
|
||||
headingSelector: 'h3'
|
||||
})
|
||||
}
|
||||
);
|
||||
</script>
|
||||
|
||||
|
||||
|
||||
<h3 id="general-information">General Information</h3>
|
||||
<p>All the configuration files for the <strong>Simple</strong>Decoder and <strong>Matrix</strong>Multiplier are using the JSON format. See <a href="https://json.org">https://json.org</a> for detailed information about this format. We are aware that we introduce a breaking change by switching from Kronlachner’s configuration files to the JSON format. However, we believe that the JSON format is more future proof and more expandable, especially for the features we’ve planned to introduce in the future (e.g. MultibandDecoder). Additionally, it is easier to en- and decode, is human-readable and decent text editors will alert you, if the data is not JSON conform.</p>
|
||||
<p>The idea is to have one configuration file for your whole project, including information about decoding, routing, positions of loudspeakers and additional matrix operations.</p>
|
||||
<h3 id="structure-of-configuration-files">Structure of Configuration Files</h3>
|
||||
<p>The basic structure only holds name and description attributes of the configuration and looks like this:</p>
|
||||
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-JSON" data-lang="JSON"><span style="display:flex;"><span>{
|
||||
</span></span><span style="display:flex;"><span><span style="color:#f92672">"Name"</span> : <span style="color:#e6db74">"name of this configuration"</span>,
|
||||
</span></span><span style="display:flex;"><span><span style="color:#f92672">"Description"</span> : <span style="color:#e6db74">"description text"</span>
|
||||
</span></span><span style="display:flex;"><span>}
|
||||
</span></span></code></pre></div><p>It can be extended by adding objects to the configuration. For now, only the following objects are defined:</p>
|
||||
<ul>
|
||||
<li><a href="#the-transformationmatrix-object">TransformationMatrix</a></li>
|
||||
<li><a href="#the-decoder-object">Decoder</a></li>
|
||||
<li><a href="#the-loudspeakerlayout-object">LoudspeakerLayout</a></li>
|
||||
</ul>
|
||||
<p>Each of them has some more sub-objects, described below.</p>
|
||||
<p>So a decoder configuration could look like this (note, that you have to add the necessary attributes to the decoder object):</p>
|
||||
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-JSON" data-lang="JSON"><span style="display:flex;"><span>{
|
||||
</span></span><span style="display:flex;"><span><span style="color:#f92672">"Name"</span> : <span style="color:#e6db74">"name of this configuration"</span>,
|
||||
</span></span><span style="display:flex;"><span><span style="color:#f92672">"Description"</span> : <span style="color:#e6db74">"description text"</span>,
|
||||
</span></span><span style="display:flex;"><span><span style="color:#f92672">"Decoder"</span> : {
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#960050;background-color:#1e0010"><put</span> <span style="color:#960050;background-color:#1e0010">decoder</span> <span style="color:#960050;background-color:#1e0010">information</span> <span style="color:#960050;background-color:#1e0010">here></span>
|
||||
</span></span><span style="display:flex;"><span> }
|
||||
</span></span><span style="display:flex;"><span>}
|
||||
</span></span></code></pre></div><p>There will be more objects in the future. For example definitions of loudspeakers to visualize their position; or a MultibandDecoder consisting of an array of Decoders with additional information about frequency bands.</p>
|
||||
<h3 id="the-transformationmatrix-object">The <code>TransformationMatrix</code> Object</h3>
|
||||
<p>With the <code>TransformationMatrix</code> object you can matrix transformation functionality to your configuration. You can define the following attributes:</p>
|
||||
<ul>
|
||||
<li><code>Name</code>: The name of your matrix as a string. <strong>(optional)</strong></li>
|
||||
<li><code>Description</code>: The description of your matrix as a string. <strong>(optional)</strong></li>
|
||||
<li><code>TransformationMatrix</code>: The transformation matrix given as a two-dimensional array of floats. See example below.</li>
|
||||
</ul>
|
||||
<h3 id="the-decoder-object">The <code>Decoder</code> Object</h3>
|
||||
<p>With the <code>Decoder</code> object you can add a decoder functionality to your configuration. You can define the following attributes:</p>
|
||||
<ul>
|
||||
<li><code>Name</code>: The name of your decoder as a string. <strong>(optional)</strong></li>
|
||||
<li><code>Description</code>: The description of your decoder as a string. <strong>(optional)</strong></li>
|
||||
<li><code>ExpectedInputNormalization</code>: Define the expected normalization of the Ambisonic input signal. Can be <code>"n3d"</code> or <code>"sn3d"</code>.</li>
|
||||
<li><code>Weights</code>: Define what kind of weighting your decoder uses. Can be <code>none</code>, <code>inPhase</code> or <code>maxrE</code>.</li>
|
||||
<li><code>WeightsAlreadyApplied</code>: Define, whether the weights are already applied to the matrix (bool value: true or false).</li>
|
||||
<li><code>SubwooferChannel</code> : Define the subwoofer channel as an integer. If defined, the <strong>Simple</strong>Decoder will use the discrete subwoofer mode and route the filtered input signal to that channel. <strong>(optional)</strong></li>
|
||||
<li><code>Matrix</code>: The decoder matrix given as a two-dimensional array of floats. See <a href="#how-to-create-a-decoder-matrix">How-to</a> and <a href="#example-decoder-configuration">example</a> below.</li>
|
||||
<li><code>Routing</code>: Define an output routing as an array of integers. Each loudspeaker signal will be routed to the declared channel number. See example below. <strong>(optional)</strong></li>
|
||||
</ul>
|
||||
<h5 id="notes">Notes</h5>
|
||||
<p>Specifying the weights-attribute is necessary for the variable order functionality. The decoder plug-in will automatically apply the correct weights according to the selected Ambisonic order of the input signal. If the weights are already applied to the matrix, the decoder will extract the Ambisonic order from the matrix’ input size and internally create a matrix without the weights.</p>
|
||||
<h3 id="the-loudspeakerlayout-object">The <code>LoudspeakerLayout</code> Object</h3>
|
||||
<p>The <code>LoudspeakerLayout</code> object holds information about a loudspeaker layout, which can be loaded into the <strong>AllRA</strong>Decoder plug-in, e.g. to create a decoder. You can define the following attributes:</p>
|
||||
<ul>
|
||||
<li><code>Name</code>: The name of your loudspeaker layout as a string. <strong>(optional)</strong></li>
|
||||
<li><code>Description</code>: The description of your loudspeaker layout as a string. <strong>(optional)</strong></li>
|
||||
<li><code>Loudspeakers</code>: This is an array holding the list of loudspeakers.</li>
|
||||
</ul>
|
||||
<p>Each loudspeaker within the <code>Loudspeakers</code> object (array) holds information about its coordinates in the spherical representation, a flag whether it is an imaginary loudspeaker (important for the AllRAD approach), a channel number for routing a signal to, and a gain value.</p>
|
||||
<p>Here’s a list of these attributes:</p>
|
||||
<ul>
|
||||
<li><code>Azimuth</code>: The azimuth angle of the loudspeaker in degrees as a float value. Front would be 0°, right -90°, left 90° (right-hand-rule).</li>
|
||||
<li><code>Elevation</code>: The elevation angle in degrees as a float value. Horizontal plane is at 0°, zenith at 90° and nadir at -90°. Using values higher than 90° or lower than -90° is not recommended.</li>
|
||||
<li><code>Radius</code>: Distance from the listening position in meters as a float value.</li>
|
||||
<li><code>IsImaginary</code>: Flag whether or not the loudspeaker is imaginary (bool value: true or false).</li>
|
||||
<li><code>Channel</code>: The channel number the signals will be routed to as an integer value.</li>
|
||||
<li><code>Gain</code>: A linear gain as a float value.</li>
|
||||
</ul>
|
||||
<h5 id="notes-1">Notes</h5>
|
||||
<p>The <strong>AllRA</strong>Decoder plug-in uses the <code>Gain</code> and <code>Radius</code> only for imaginary loudspeakers, so the gain won’t affect real loudspeakers. See the <a href="../allradecoder/">AllRADecoder Guide</a> for more information. You can also use the <strong>AllRA</strong>Decoder to export the loudspeaker layout to a configuration file.</p>
|
||||
<h3 id="how-to-create-a-decoder-matrix">How to create a Decoder Matrix</h3>
|
||||
<p>Next to a well designed loudspeaker layout, a decent Ambisonic decoder is essential for playing back an Ambisonic audio signal. Creating a decoder is not a trivial and there are several strategies to do so. At the IEM we use the <a href="http://www.aes.org/e-lib/browse.cfm?elib=16554">AllRAD</a> strategy to create our decoders. We created the <strong>AllRA</strong>Decoder plug-in for designing an Ambisonic Decoder which suits your loudspeaker layout. As the name implies, it also uses the AllRAD approach. See the <a href="../allradecoder/">AllRADecoder Guide</a> for a how-to.</p>
|
||||
<p>As an alternative, the <a href="https://bitbucket.org/ambidecodertoolbox/adt.git">Ambisonic Decoder ToolBox</a> is a handy tool for Matlab and Octave, which also supports that <strong>All</strong>-<strong>R</strong>ound <strong>A</strong>mbisonic <strong>D</strong>ecoding. You can use it to create a decoder matrix and export it with Matlab’s <code>jsonencode</code>.</p>
|
||||
<h3 id="example-decoder-configuration">Example Decoder Configuration</h3>
|
||||
<p>This configuration holds a Decoder object, whose matrix is of size 12x36 (output: 12 loudspeakers; input: 5th order Ambisonics). The output channel 4 is skipped by the routing, as that channel is the one the subwoofer is connected to.</p>
|
||||
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-JSON" data-lang="JSON"><span style="display:flex;"><span>{
|
||||
</span></span><span style="display:flex;"><span><span style="color:#f92672">"Name"</span> : <span style="color:#e6db74">"Produktionsstudio decoder"</span>,
|
||||
</span></span><span style="display:flex;"><span><span style="color:#f92672">"Description"</span> : <span style="color:#e6db74">""</span>,
|
||||
</span></span><span style="display:flex;"><span><span style="color:#f92672">"Decoder"</span> : {
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"Name"</span> : <span style="color:#e6db74">"Produktionsstudio decoder o5"</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"Description"</span> : <span style="color:#e6db74">"This is a decoder for the IEM Produktionsstudio."</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"ExpectedInputNormalization"</span> : <span style="color:#e6db74">"n3d"</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"Weights"</span> : <span style="color:#e6db74">"maxrE"</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"WeightsAlreadyApplied"</span> : <span style="color:#66d9ef">true</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"SubwooferChannel"</span> : <span style="color:#ae81ff">4</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"Matrix"</span> :
|
||||
</span></span><span style="display:flex;"><span> [[<span style="color:#ae81ff">0.524667</span>,<span style="color:#ae81ff">0.469572</span>,<span style="color:#ae81ff">-0.209473</span>,<span style="color:#ae81ff">0.540143</span>,<span style="color:#ae81ff">0.535468</span>,<span style="color:#ae81ff">-0.176045</span>,<span style="color:#ae81ff">-0.20627</span>,<span style="color:#ae81ff">-0.207037</span>,<span style="color:#ae81ff">0.07898</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.278071</span>,<span style="color:#ae81ff">-0.171502</span>,<span style="color:#ae81ff">-0.07617</span>,<span style="color:#ae81ff">0.080337</span>,<span style="color:#ae81ff">-0.087312</span>,<span style="color:#ae81ff">-0.031127</span>,<span style="color:#ae81ff">-0.168645</span>,<span style="color:#ae81ff">0.052887</span>,<span style="color:#ae81ff">-0.068575</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.033469</span>,<span style="color:#ae81ff">0.013674</span>,<span style="color:#ae81ff">0.034073</span>,<span style="color:#ae81ff">0.018059</span>,<span style="color:#ae81ff">-0.004703</span>,<span style="color:#ae81ff">0.03349</span>,<span style="color:#ae81ff">-0.135629</span>,<span style="color:#ae81ff">-0.00736</span>,<span style="color:#ae81ff">-0.01102</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.003696</span>,<span style="color:#ae81ff">-0.006332</span>,<span style="color:#ae81ff">0.01286</span>,<span style="color:#ae81ff">0.000698</span>,<span style="color:#ae81ff">0.015034</span>,<span style="color:#ae81ff">0.000453</span>,<span style="color:#ae81ff">0.00238</span>,<span style="color:#ae81ff">0.013449</span>,<span style="color:#ae81ff">-0.038748</span>],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0.52467</span>,<span style="color:#ae81ff">-0.469604</span>,<span style="color:#ae81ff">-0.20946</span>,<span style="color:#ae81ff">0.540139</span>,<span style="color:#ae81ff">-0.535474</span>,<span style="color:#ae81ff">0.17603</span>,<span style="color:#ae81ff">-0.2063</span>,<span style="color:#ae81ff">-0.207068</span>,<span style="color:#ae81ff">0.078934</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.278033</span>,<span style="color:#ae81ff">0.171505</span>,<span style="color:#ae81ff">0.076174</span>,<span style="color:#ae81ff">0.080367</span>,<span style="color:#ae81ff">-0.087303</span>,<span style="color:#ae81ff">-0.031177</span>,<span style="color:#ae81ff">-0.168678</span>,<span style="color:#ae81ff">-0.052849</span>,<span style="color:#ae81ff">0.068601</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.033452</span>,<span style="color:#ae81ff">-0.013671</span>,<span style="color:#ae81ff">0.034052</span>,<span style="color:#ae81ff">0.018066</span>,<span style="color:#ae81ff">-0.004701</span>,<span style="color:#ae81ff">0.033452</span>,<span style="color:#ae81ff">-0.135628</span>,<span style="color:#ae81ff">0.007376</span>,<span style="color:#ae81ff">0.011046</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.003689</span>,<span style="color:#ae81ff">0.006338</span>,<span style="color:#ae81ff">-0.012852</span>,<span style="color:#ae81ff">0.000708</span>,<span style="color:#ae81ff">0.015035</span>,<span style="color:#ae81ff">0.000457</span>,<span style="color:#ae81ff">0.002368</span>,<span style="color:#ae81ff">0.013438</span>,<span style="color:#ae81ff">-0.038739</span>],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0.48232</span>,<span style="color:#ae81ff">9E-6</span>,<span style="color:#ae81ff">-0.046894</span>,<span style="color:#ae81ff">0.673631</span>,<span style="color:#ae81ff">5E-6</span>,<span style="color:#ae81ff">-2E-6</span>,<span style="color:#ae81ff">-0.19333</span>,<span style="color:#ae81ff">-0.042387</span>,<span style="color:#ae81ff">0.538062</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-6E-6</span>,<span style="color:#ae81ff">-1E-6</span>,<span style="color:#ae81ff">2E-5</span>,<span style="color:#ae81ff">-0.030678</span>,<span style="color:#ae81ff">-0.09633</span>,<span style="color:#ae81ff">-0.027138</span>,<span style="color:#ae81ff">0.356096</span>,<span style="color:#ae81ff">-1.2E-5</span>,<span style="color:#ae81ff">-1E-6</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">2.9E-5</span>,<span style="color:#ae81ff">-6E-6</span>,<span style="color:#ae81ff">0.006201</span>,<span style="color:#ae81ff">-0.048245</span>,<span style="color:#ae81ff">-0.016251</span>,<span style="color:#ae81ff">-0.020807</span>,<span style="color:#ae81ff">0.186287</span>,<span style="color:#ae81ff">-9E-6</span>,<span style="color:#ae81ff">-2E-6</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">1.7E-5</span>,<span style="color:#ae81ff">-6E-6</span>,<span style="color:#ae81ff">-3E-6</span>,<span style="color:#ae81ff">0.014683</span>,<span style="color:#ae81ff">-0.0051</span>,<span style="color:#ae81ff">-0.023478</span>,<span style="color:#ae81ff">0.004609</span>,<span style="color:#ae81ff">-0.013344</span>,<span style="color:#ae81ff">0.06439</span>],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0.85076</span>,<span style="color:#ae81ff">0.503559</span>,<span style="color:#ae81ff">-0.266537</span>,<span style="color:#ae81ff">-0.952247</span>,<span style="color:#ae81ff">-0.541567</span>,<span style="color:#ae81ff">-0.191062</span>,<span style="color:#ae81ff">-0.337222</span>,<span style="color:#ae81ff">0.256713</span>,<span style="color:#ae81ff">0.330954</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.232285</span>,<span style="color:#ae81ff">0.174682</span>,<span style="color:#ae81ff">-0.082557</span>,<span style="color:#ae81ff">0.078565</span>,<span style="color:#ae81ff">0.147907</span>,<span style="color:#ae81ff">-0.038838</span>,<span style="color:#ae81ff">0.043984</span>,<span style="color:#ae81ff">-8E-6</span>,<span style="color:#ae81ff">-0.05472</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.034604</span>,<span style="color:#ae81ff">0.016569</span>,<span style="color:#ae81ff">0.042964</span>,<span style="color:#ae81ff">0.003534</span>,<span style="color:#ae81ff">-0.010906</span>,<span style="color:#ae81ff">-0.036433</span>,<span style="color:#ae81ff">-0.073705</span>,<span style="color:#ae81ff">-0.028779</span>,<span style="color:#ae81ff">-6E-6</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.003308</span>,<span style="color:#ae81ff">0.00572</span>,<span style="color:#ae81ff">0.014052</span>,<span style="color:#ae81ff">0.007977</span>,<span style="color:#ae81ff">-0.014555</span>,<span style="color:#ae81ff">-0.011586</span>,<span style="color:#ae81ff">-0.006084</span>,<span style="color:#ae81ff">0.011077</span>,<span style="color:#ae81ff">0.01356</span>],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0.850906</span>,<span style="color:#ae81ff">-0.503624</span>,<span style="color:#ae81ff">-0.26653</span>,<span style="color:#ae81ff">-0.952374</span>,<span style="color:#ae81ff">0.54162</span>,<span style="color:#ae81ff">0.191019</span>,<span style="color:#ae81ff">-0.337233</span>,<span style="color:#ae81ff">0.256664</span>,<span style="color:#ae81ff">0.330977</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.232295</span>,<span style="color:#ae81ff">-0.174643</span>,<span style="color:#ae81ff">0.08258</span>,<span style="color:#ae81ff">0.078468</span>,<span style="color:#ae81ff">0.147914</span>,<span style="color:#ae81ff">-0.038837</span>,<span style="color:#ae81ff">0.043988</span>,<span style="color:#ae81ff">1.8E-5</span>,<span style="color:#ae81ff">0.05471</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.034603</span>,<span style="color:#ae81ff">-0.016557</span>,<span style="color:#ae81ff">0.042992</span>,<span style="color:#ae81ff">0.003586</span>,<span style="color:#ae81ff">-0.010914</span>,<span style="color:#ae81ff">-0.03641</span>,<span style="color:#ae81ff">-0.073692</span>,<span style="color:#ae81ff">0.028764</span>,<span style="color:#ae81ff">2E-6</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.003306</span>,<span style="color:#ae81ff">-0.005728</span>,<span style="color:#ae81ff">-0.014047</span>,<span style="color:#ae81ff">0.007974</span>,<span style="color:#ae81ff">-0.014559</span>,<span style="color:#ae81ff">-0.011597</span>,<span style="color:#ae81ff">-0.006063</span>,<span style="color:#ae81ff">0.011066</span>,<span style="color:#ae81ff">0.013551</span>],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0.75186</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">-0.156013</span>,<span style="color:#ae81ff">0.078043</span>,<span style="color:#ae81ff">0.104081</span>,<span style="color:#ae81ff">-0.172615</span>,<span style="color:#ae81ff">-0.296441</span>,<span style="color:#ae81ff">-0.029433</span>,<span style="color:#ae81ff">-0.689955</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.346121</span>,<span style="color:#ae81ff">-0.035164</span>,<span style="color:#ae81ff">-0.144883</span>,<span style="color:#ae81ff">0.009708</span>,<span style="color:#ae81ff">-0.012092</span>,<span style="color:#ae81ff">0.089756</span>,<span style="color:#ae81ff">-0.071726</span>,<span style="color:#ae81ff">-0.022223</span>,<span style="color:#ae81ff">0.030376</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.005582</span>,<span style="color:#ae81ff">-0.03736</span>,<span style="color:#ae81ff">0.022544</span>,<span style="color:#ae81ff">0.002024</span>,<span style="color:#ae81ff">0.022081</span>,<span style="color:#ae81ff">0.020618</span>,<span style="color:#ae81ff">0.109559</span>,<span style="color:#ae81ff">0.011133</span>,<span style="color:#ae81ff">0.0051</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.006416</span>,<span style="color:#ae81ff">-0.001135</span>,<span style="color:#ae81ff">0.003645</span>,<span style="color:#ae81ff">0.015434</span>,<span style="color:#ae81ff">0.002047</span>,<span style="color:#ae81ff">0.023454</span>,<span style="color:#ae81ff">0.000338</span>,<span style="color:#ae81ff">-0.005592</span>,<span style="color:#ae81ff">-0.003701</span>],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0.751767</span>,<span style="color:#ae81ff">-0.999922</span>,<span style="color:#ae81ff">-0.156027</span>,<span style="color:#ae81ff">0.07806</span>,<span style="color:#ae81ff">-0.104104</span>,<span style="color:#ae81ff">0.172666</span>,<span style="color:#ae81ff">-0.296466</span>,<span style="color:#ae81ff">-0.029437</span>,<span style="color:#ae81ff">-0.689946</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.346149</span>,<span style="color:#ae81ff">0.035161</span>,<span style="color:#ae81ff">0.144916</span>,<span style="color:#ae81ff">0.009764</span>,<span style="color:#ae81ff">-0.012099</span>,<span style="color:#ae81ff">0.08978</span>,<span style="color:#ae81ff">-0.071742</span>,<span style="color:#ae81ff">0.022225</span>,<span style="color:#ae81ff">-0.030366</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.005591</span>,<span style="color:#ae81ff">0.037336</span>,<span style="color:#ae81ff">0.02256</span>,<span style="color:#ae81ff">0.002004</span>,<span style="color:#ae81ff">0.022082</span>,<span style="color:#ae81ff">0.020616</span>,<span style="color:#ae81ff">0.109586</span>,<span style="color:#ae81ff">-0.011143</span>,<span style="color:#ae81ff">-0.0051</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.006428</span>,<span style="color:#ae81ff">0.001146</span>,<span style="color:#ae81ff">-0.003654</span>,<span style="color:#ae81ff">0.015432</span>,<span style="color:#ae81ff">0.002053</span>,<span style="color:#ae81ff">0.023463</span>,<span style="color:#ae81ff">0.000337</span>,<span style="color:#ae81ff">-0.005577</span>,<span style="color:#ae81ff">-0.003705</span>],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0.637174</span>,<span style="color:#ae81ff">0.451681</span>,<span style="color:#ae81ff">0.620473</span>,<span style="color:#ae81ff">0.452977</span>,<span style="color:#ae81ff">0.287188</span>,<span style="color:#ae81ff">0.468269</span>,<span style="color:#ae81ff">0.139043</span>,<span style="color:#ae81ff">0.468783</span>,<span style="color:#ae81ff">0.001525</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.051188</span>,<span style="color:#ae81ff">0.271546</span>,<span style="color:#ae81ff">0.194653</span>,<span style="color:#ae81ff">-0.14563</span>,<span style="color:#ae81ff">0.194029</span>,<span style="color:#ae81ff">0.000703</span>,<span style="color:#ae81ff">-0.04915</span>,<span style="color:#ae81ff">0.000835</span>,<span style="color:#ae81ff">0.034811</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.09865</span>,<span style="color:#ae81ff">0.001314</span>,<span style="color:#ae81ff">-0.108523</span>,<span style="color:#ae81ff">0.000868</span>,<span style="color:#ae81ff">-0.000392</span>,<span style="color:#ae81ff">-0.033847</span>,<span style="color:#ae81ff">0.00361</span>,<span style="color:#ae81ff">0.006586</span>,<span style="color:#ae81ff">0.000343</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.002819</span>,<span style="color:#ae81ff">0.001608</span>,<span style="color:#ae81ff">-0.01643</span>,<span style="color:#ae81ff">-0.017683</span>,<span style="color:#ae81ff">-0.016347</span>,<span style="color:#ae81ff">-0.000279</span>,<span style="color:#ae81ff">-0.00303</span>,<span style="color:#ae81ff">0.005699</span>,<span style="color:#ae81ff">0.006292</span>],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0.637184</span>,<span style="color:#ae81ff">-0.451677</span>,<span style="color:#ae81ff">0.62047</span>,<span style="color:#ae81ff">0.452974</span>,<span style="color:#ae81ff">-0.287146</span>,<span style="color:#ae81ff">-0.46826</span>,<span style="color:#ae81ff">0.13902</span>,<span style="color:#ae81ff">0.468777</span>,<span style="color:#ae81ff">0.001515</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.051165</span>,<span style="color:#ae81ff">-0.2715</span>,<span style="color:#ae81ff">-0.194646</span>,<span style="color:#ae81ff">-0.145654</span>,<span style="color:#ae81ff">0.194027</span>,<span style="color:#ae81ff">0.000686</span>,<span style="color:#ae81ff">-0.049135</span>,<span style="color:#ae81ff">-0.00084</span>,<span style="color:#ae81ff">-0.034777</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.09864</span>,<span style="color:#ae81ff">-0.001314</span>,<span style="color:#ae81ff">-0.108533</span>,<span style="color:#ae81ff">0.000869</span>,<span style="color:#ae81ff">-0.000403</span>,<span style="color:#ae81ff">-0.033837</span>,<span style="color:#ae81ff">0.003608</span>,<span style="color:#ae81ff">-0.006587</span>,<span style="color:#ae81ff">-0.000337</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.002805</span>,<span style="color:#ae81ff">-0.001621</span>,<span style="color:#ae81ff">0.016428</span>,<span style="color:#ae81ff">-0.017681</span>,<span style="color:#ae81ff">-0.016347</span>,<span style="color:#ae81ff">-0.000283</span>,<span style="color:#ae81ff">-0.003032</span>,<span style="color:#ae81ff">0.005699</span>,<span style="color:#ae81ff">0.006286</span>],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0.638428</span>,<span style="color:#ae81ff">0.453622</span>,<span style="color:#ae81ff">0.620934</span>,<span style="color:#ae81ff">-0.453725</span>,<span style="color:#ae81ff">-0.288777</span>,<span style="color:#ae81ff">0.469057</span>,<span style="color:#ae81ff">0.138133</span>,<span style="color:#ae81ff">-0.469196</span>,<span style="color:#ae81ff">3E-6</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.050871</span>,<span style="color:#ae81ff">-0.272325</span>,<span style="color:#ae81ff">0.193751</span>,<span style="color:#ae81ff">-0.146243</span>,<span style="color:#ae81ff">-0.193823</span>,<span style="color:#ae81ff">2.8E-5</span>,<span style="color:#ae81ff">0.050953</span>,<span style="color:#ae81ff">3.7E-5</span>,<span style="color:#ae81ff">0.034674</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.098291</span>,<span style="color:#ae81ff">0.000659</span>,<span style="color:#ae81ff">-0.108277</span>,<span style="color:#ae81ff">-0.000645</span>,<span style="color:#ae81ff">1.8E-5</span>,<span style="color:#ae81ff">0.034723</span>,<span style="color:#ae81ff">0.003108</span>,<span style="color:#ae81ff">0.006353</span>,<span style="color:#ae81ff">1.2E-5</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.002849</span>,<span style="color:#ae81ff">-0.001321</span>,<span style="color:#ae81ff">-0.016311</span>,<span style="color:#ae81ff">-0.017459</span>,<span style="color:#ae81ff">0.016335</span>,<span style="color:#ae81ff">-6E-6</span>,<span style="color:#ae81ff">0.002865</span>,<span style="color:#ae81ff">0.005478</span>,<span style="color:#ae81ff">-0.006362</span>],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0.638375</span>,<span style="color:#ae81ff">-0.453609</span>,<span style="color:#ae81ff">0.620938</span>,<span style="color:#ae81ff">-0.453603</span>,<span style="color:#ae81ff">0.288705</span>,<span style="color:#ae81ff">-0.469075</span>,<span style="color:#ae81ff">0.138194</span>,<span style="color:#ae81ff">-0.469142</span>,<span style="color:#ae81ff">-6.4E-5</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.05085</span>,<span style="color:#ae81ff">0.272288</span>,<span style="color:#ae81ff">-0.193774</span>,<span style="color:#ae81ff">-0.146229</span>,<span style="color:#ae81ff">-0.19386</span>,<span style="color:#ae81ff">-5E-6</span>,<span style="color:#ae81ff">0.050949</span>,<span style="color:#ae81ff">-2.4E-5</span>,<span style="color:#ae81ff">-0.03467</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">0.098293</span>,<span style="color:#ae81ff">-0.000651</span>,<span style="color:#ae81ff">-0.108301</span>,<span style="color:#ae81ff">-0.000667</span>,<span style="color:#ae81ff">3E-5</span>,<span style="color:#ae81ff">0.034709</span>,<span style="color:#ae81ff">0.003096</span>,<span style="color:#ae81ff">-0.006357</span>,<span style="color:#ae81ff">-1E-6</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">-0.002853</span>,<span style="color:#ae81ff">0.001324</span>,<span style="color:#ae81ff">0.016324</span>,<span style="color:#ae81ff">-0.017463</span>,<span style="color:#ae81ff">0.016343</span>,<span style="color:#ae81ff">4E-6</span>,<span style="color:#ae81ff">0.002853</span>,<span style="color:#ae81ff">0.005472</span>,<span style="color:#ae81ff">-0.006355</span>],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0.413602</span>,<span style="color:#ae81ff">1.9E-5</span>,<span style="color:#ae81ff">0.610689</span>,<span style="color:#ae81ff">5.3E-5</span>,<span style="color:#ae81ff">1.9E-5</span>,<span style="color:#ae81ff">3E-5</span>,<span style="color:#ae81ff">0.565073</span>,<span style="color:#ae81ff">7.3E-5</span>,<span style="color:#ae81ff">1E-5</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">1E-6</span>,<span style="color:#ae81ff">2.7E-5</span>,<span style="color:#ae81ff">2.6E-5</span>,<span style="color:#ae81ff">0.38914</span>,<span style="color:#ae81ff">5.4E-5</span>,<span style="color:#ae81ff">1.7E-5</span>,<span style="color:#ae81ff">-2.1E-5</span>,<span style="color:#ae81ff">2E-6</span>,<span style="color:#ae81ff">1E-6</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">1.9E-5</span>,<span style="color:#ae81ff">1.3E-5</span>,<span style="color:#ae81ff">0.192325</span>,<span style="color:#ae81ff">1.8E-5</span>,<span style="color:#ae81ff">1.6E-5</span>,<span style="color:#ae81ff">-2.8E-5</span>,<span style="color:#ae81ff">-0.0034</span>,<span style="color:#ae81ff">3E-6</span>,<span style="color:#ae81ff">3E-6</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">1E-6</span>,<span style="color:#ae81ff">5E-6</span>,<span style="color:#ae81ff">1E-6</span>,<span style="color:#ae81ff">0.054697</span>,<span style="color:#ae81ff">-3E-6</span>,<span style="color:#ae81ff">8E-6</span>,<span style="color:#ae81ff">-1.7E-5</span>,<span style="color:#ae81ff">-0.004302</span>,<span style="color:#ae81ff">-3E-6</span>]],
|
||||
</span></span><span style="display:flex;"><span>
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"Routing"</span> : [<span style="color:#ae81ff">1</span>, <span style="color:#ae81ff">2</span>, <span style="color:#ae81ff">3</span>, <span style="color:#ae81ff">5</span>, <span style="color:#ae81ff">6</span>, <span style="color:#ae81ff">7</span>, <span style="color:#ae81ff">8</span>, <span style="color:#ae81ff">9</span>, <span style="color:#ae81ff">10</span>, <span style="color:#ae81ff">11</span>, <span style="color:#ae81ff">12</span>, <span style="color:#ae81ff">13</span>]
|
||||
</span></span><span style="display:flex;"><span> }
|
||||
</span></span><span style="display:flex;"><span>}
|
||||
</span></span></code></pre></div><h3 id="example-transformationmatrix-configuration">Example TransformationMatrix Configuration</h3>
|
||||
<p>This example shows a simple configuration which uses a TransformationMatrix.</p>
|
||||
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-JSON" data-lang="JSON"><span style="display:flex;"><span>{
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"Name"</span> : <span style="color:#e6db74">"Left/right flip"</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"Description"</span> : <span style="color:#e6db74">"This configuration contains a simple left/right flip matrix."</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"TransformationMatrix"</span> : {
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"Name"</span> : <span style="color:#e6db74">"Flip: left/right o5"</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"Description"</span> : <span style="color:#e6db74">"This matrix mirrors a 5th order Ambisonic signal across the xy-plane (left right flip)"</span>,
|
||||
</span></span><span style="display:flex;"><span> <span style="color:#f92672">"Matrix"</span> : [[<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">-1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>,<span style="color:#ae81ff">0</span>],
|
||||
</span></span><span style="display:flex;"><span> [<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">0</span>,<span style="color:#ae81ff">1</span>]]
|
||||
</span></span><span style="display:flex;"><span> }
|
||||
</span></span><span style="display:flex;"><span>}
|
||||
</span></span></code></pre></div>
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
<footer>
|
||||
<footer id="footer">
|
||||
<nav>
|
||||
<ul>
|
||||
<li><span>Plug-in Suite by IEM - Institute of Electronic Music and Acoustics</span></li>
|
||||
<li>
|
||||
<a href="https://iem.at/">IEM.at</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="https://plugins.iem.at/docs/">Docs</a>
|
||||
</li>
|
||||
</ul>
|
||||
</nav>
|
||||
<p>Website created with <a href="https://gohugo.io">Hugo</a> using the <a href="https://github.com/jeblister/kube">kube</a> theme.</p>
|
||||
</footer>
|
||||
|
||||
</footer>
|
||||
|
||||
|
||||
<script src="https://plugins.iem.at/js/kube.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/kube.legenda.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/master.js" type="text/javascript">
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
|
@ -0,0 +1,247 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en-us">
|
||||
|
||||
<head>
|
||||
<meta name="generator" content="Hugo 0.119.0">
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title> Plug-in Descriptions </title>
|
||||
|
||||
|
||||
<meta name="description" content="List of all plug-ins and their descriptions">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta name="author" content="Daniel Rudrich">
|
||||
|
||||
|
||||
<meta property="og:title" content="Plug-in Descriptions" />
|
||||
<meta property="og:description" content="List of all plug-ins and their descriptions" />
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://plugins.iem.at/docs/plugindescriptions/" /><meta property="og:image" content="https://plugins.iem.at/images/IEMPluginSuiteLogo.png"/><meta property="article:section" content="docs" />
|
||||
<meta property="article:published_time" content="2018-06-07T13:29:04+01:00" />
|
||||
<meta property="article:modified_time" content="2018-06-07T13:29:04+01:00" />
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta property="og:image" content="images/IEMPluginSuiteLogo.png">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="canonical" href="https://plugins.iem.at/docs/plugindescriptions/">
|
||||
|
||||
<link rel="shortcut icon" type="image/png" href="images/favicon_96x96.png">
|
||||
|
||||
|
||||
<link href="https://plugins.iem.at/css/font.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.min.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.legenda.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/highlight.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/master.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.demo.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<link href="https://plugins.iem.at/css/custom.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<script src="https://plugins.iem.at/js/jquery-3.5.1.min.js" type="text/javascript">
|
||||
</script>
|
||||
|
||||
<script type="text/javascript" src="https://plugins.iem.at/js/tocbot.min.js"></script>
|
||||
</head>
|
||||
|
||||
|
||||
<body class="page-kube">
|
||||
<header> <div class="show-sm">
|
||||
<div id="nav-toggle-box">
|
||||
<div id="nav-toggle-brand">
|
||||
<a href="https://plugins.iem.at/">IEM Plug-in Suite</a>
|
||||
</div><a data-component="toggleme" data-target="#top" href="#" id="nav-toggle"><i class="kube-menu"></i></a>
|
||||
</div>
|
||||
</div>
|
||||
<div class="hide-sm" id="top">
|
||||
<div id="top-brand">
|
||||
<a href="https://plugins.iem.at/" title="home">IEM Plug-in Suite</a>
|
||||
</div>
|
||||
|
||||
<nav id="top-nav-main">
|
||||
<ul>
|
||||
|
||||
|
||||
<li><a href="https://plugins.iem.at/download/" >Download</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/docs/" >Documentation</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/faq/" >FAQ</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/contact/" >Contact</a></li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
<div id="nav-toggle" style="text-align: center;">
|
||||
<a href="https://atmoky.com"><img src="https://plugins.iem.at/images/atmokyfriends.png" width="200"></a>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
<main>
|
||||
<div id="main">
|
||||
<div id="hero">
|
||||
<h1> Plug-in Descriptions </h1>
|
||||
<p class="hero-lead">
|
||||
Here you find a list of available plug-ins in alphabetical order and their description.
|
||||
</p>
|
||||
|
||||
</div>
|
||||
<div id="kube-component" class="content">
|
||||
|
||||
<nav id="contents">
|
||||
<ol class="js-toc">
|
||||
</ol>
|
||||
</nav>
|
||||
<script type="text/javascript">
|
||||
document.addEventListener("DOMContentLoaded",
|
||||
function(){
|
||||
tocbot.init({
|
||||
|
||||
tocSelector: '.js-toc',
|
||||
|
||||
contentSelector: '.content',
|
||||
|
||||
headingSelector: 'h3'
|
||||
})
|
||||
}
|
||||
);
|
||||
</script>
|
||||
|
||||
|
||||
|
||||
<h3 id="allradecoder">AllRADecoder</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> and <strong>Franz Zotter</strong> <mark>Type</mark> <strong>Decoder</strong> <mark>Input</mark> <strong>Ambisonic signals</strong> <mark>Output</mark> <strong>Audio signals</strong></small></p>
|
||||
<p>With the <strong>AllRA</strong>Decoder you can design an Ambisonic decoder for an arbitrary loudspeaker layout using the AllRAD approach. More information can be found in the <a href="../allradecoder/">AllRADecoder Guide</a>.</p>
|
||||
<h3 id="binauraldecoder">BinauralDecoder</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Binaural Decoder</strong> <mark>Input</mark> <strong>Ambisonic signals</strong> <mark>Output</mark> <strong>Binaural headphone signals</strong></small></p>
|
||||
<p>The <strong>Binaural</strong>Decoder renders the Ambisonic input signal to a binaural headphone signal using the <em>MagLS</em> approach proposed in [1]. The used HRTFs come from the Neumann KU 100 dummy head. Additionally, you can apply headphone equalizations from measurements by <a href="http://audiogroup.web.th-koeln.de/ku100hrir.html">Benjamin Bernschütz et. al.</a>.</p>
|
||||
<p>In contrast to conventional binaural decoders, the <strong>Binaural</strong>Decoder does not use virtual loudspeakers, but converts the Ambisonic signals directly to binaural headphone signals, with help of pre-processed HRTFs. They are processed in way, that the original frequency response of the HRTFs is maintained as best as possible, when used with Ambisonics.</p>
|
||||
<p>[1] Schoerkhuber, Christian; Zaunschirm, Markus; Hoeldrich, Robert. “Binaural Rendering of Ambisonic Signals via Magnitude Least Squares”, Fortschritte der Akustik, DAGA, 2018</p>
|
||||
<h3 id="coordinateconverter">CoordinateConverter</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Tool</strong> <mark>Input</mark> <strong>N/A</strong> <mark>Output</mark> <strong>N/A</strong></small></p>
|
||||
<p>The <strong>Coordinate</strong>Converter converts VST parameters from a spherical representation to a cartesian, and vice versa. There’s no audio processing, just the conversion of parameters. A typical application would be the conversion of xyz-position data (e.g. in an automation) to a spherical representation (azimuth, elevation, radius) for the use of Ambisonic encoders.</p>
|
||||
<p>Another example would be the usage with the <strong>Room</strong>Encoder: With using the sphere panner (spherical coordinates) and mapping the calculated cartesian coordinates from the <strong>Coordinate</strong>Converter to the source position of the <strong>Room</strong>Encoder, you can make a source rotate around the listener in the virtual room, maintaining the radius. For that you will need to set the radius normalization to any desired maximum value, and set x- and y-range to 15.0m, z-range to 10.0m, as these values are the parameter limits of the source’s x-, y- and z-position. Adjusting the ranges is necessary, as VST parameters are normlized to a range from 0 to 1.</p>
|
||||
<h3 id="directionalcompressor">DirectionalCompressor</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Dynamics</strong> <mark>Input</mark> <strong>Ambisonic signals</strong> <mark>Output</mark> <strong>Ambisonic signals</strong></small></p>
|
||||
<p>The <strong>Directional</strong>Compressor is a more sophisticated Ambisonic compressor/limiter which lets you control the dynamics for different spatial regions. Example: a singer in the front could be compressed more than the rest. You can realize a ducking effect, with which the singer’s level can control the level of everything else.
|
||||
Use the sphere panner and width settings to create a spatial mask. You can listen to the masked and unmasked signals by selecting them in the combobox. There are two identical compressors available. In each you can choose between three input signals (full = omni, masked signal, and unmasked signal = the rest). You can also select the target to which the compressor gains are applied to.</p>
|
||||
<h3 id="directivityshaper">DirectivityShaper</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Directivity encoder</strong> <mark>Input</mark> <strong>Mono audio signal</strong> <mark>Output</mark> <strong>Directivity signals</strong></small></p>
|
||||
<p>This plugin let’s you filter your input signal into four independent bands, to which different directivity patterns can be applied to. Also the bands can be spatially panned.</p>
|
||||
<p>For each band, you can choose from four types of filters (all-pass, low-pass, band-pass, high-pass) blend continuously between different directivity orders, as well as different shapes: basic, maxRE and in-phase. The latter makes sure that there will be no side-lobes, at cost of a wider main-lobe. Use the sphere in the upper left to pan the individual bands. The black dot represents the probe, with which you can measure the frequency-response for that direction (white line in the filter visualizer). You can also lock the relative orientations of all bands to the probe, by enabling the “lock orientations”-button. Now every band will rotate according to the probe’s orientation.
|
||||
See the <a href="https://plugins.iem.at/docs/directivityshaper/"><strong>Directivity</strong>Shaper Guide</a> for a detailed manual of the plug-in.</p>
|
||||
<h3 id="distancecompensator">DistanceCompensator</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Loudspeaker compensation</strong> <mark>Input</mark> <strong>Audio signals</strong> <mark>Output</mark> <strong>Audio signals</strong></small></p>
|
||||
<p>With the <strong>Distance</strong>Compensator you can insert the distances from the listening position to each loudspeaker and the plug-in will calculate the needed delays and gains in order to compensate for distance differences. So this plug-in is best place at the end of your signal chain (e.g. right after the decoders).</p>
|
||||
<p>Gain and delay compensation can be enabled/disabled separately and their calculation parameters can also be set. Regarding delays, the speed of sound can be specified, which is 343.2m/s (at 20°C) per default. The gains are calculated using a distance-ratio-exponent, which has a value of 1.0 assuming free field conditions. In rooms, smaller values might be more adequate due to wall reflections. Additionally, you can specify whether there should be only attenuations or using the zero-mean setting, which also will have positive gains applied to loudspeakers further away from the listener.</p>
|
||||
<p>It’s also possible to load the distance information via configuration files holding a LoudspeakerLayout object (see the <a href="https://plugins.iem.at/docs/configurationfiles/">Configuration Files Guide</a> for more information). Once loaded a LoudspeakerLayout, you can define the reference position to which the loudspeakers are compensated for.</p>
|
||||
<h3 id="dualdelay">DualDelay</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Delay Effect</strong> <mark>Input</mark> <strong>Ambisonic signals</strong> <mark>Output</mark> <strong>Ambisonic signals</strong></small></p>
|
||||
<p>The <strong>Dual</strong>Delay is the plug-in-version of our <a href="https://www.researchgate.net/publication/315025916_Efficient_Spatial_Ambisonic_Effects_for_Live_Audio">RotationalDelay</a> we first used for the Al Di Meola concert. There are two delay-lines, which can be configured independently. The most exciting feature is the rotation or warping of the whole sound field every single time it runs through the delay-line. The delay’s timbre can be shaped with the high- and low-pass filters. The output of each delay-line can be fed back to itself or/and to the other one and mixed to the output signal. Delay times can be synced to the host BPM.</p>
|
||||
<h3 id="energyvisualizer">EnergyVisualizer</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Visualizer</strong> <mark>Input</mark> <strong>Ambisonic signals</strong> <mark>Output</mark> <strong>Ambisonic signals</strong></small></p>
|
||||
<p>The <strong>Energy</strong>Visualizer visualizes the energy distribution on the sphere of the Ambisonic input signal using a <a href="https://en.wikipedia.org/wiki/Hammer_projection">Hammer-Aitoff projection</a> (a spherical projection which is area-preserving). The visualized dynamic range can be adjusted between 10 and 60dB, the range can be moved up or down using the slider next to the projection. The energy levels are color-coded with a perceptually motivated colormap. The used colormap is depicted next to the projection. As the Ambisonic Signal is decoded using 480 virtual loudspeakers with a subsequent RMS calculation and OpenGL visualization, the EnergyVisualizer is quite computation-intensive. Make sure you use only a few of them. The processing will be automatically deactivated when the GUI window is closed.</p>
|
||||
<h3 id="fdnreverb">FdnReverb</h3>
|
||||
<p><small><mark>Author</mark> <strong>Sebastian Grill</strong> <mark>Type</mark> <strong>Reverb</strong> <mark>Input</mark> <strong>Audio/Ambisonic signals</strong> <mark>Output</mark> <strong>Audio/Ambisonic signals</strong></small></p>
|
||||
<p>The <strong>Fdn</strong>Reverb is a Feedback-Delay-Network reverberation. You can use it with single audio signals but also with Ambisonic signals, leading to a spatial and temporal diffuse reverberation. You can set the overall reverberation time and refine it frequency-dependently with the help of two shelving-filters. With the fade-in control you can make the effect even more diffuse by emplyoing a second network in parallel. Make sure the fade-in time is below the overall reverberation time.</p>
|
||||
<h3 id="granularencoder">GranularEncoder</h3>
|
||||
<p><small><mark>Author</mark> <strong>Stefan Riedel</strong> <mark>Type</mark> <strong>Encoder/Synthesizer</strong> <mark>Input</mark> <strong>Mono or stereo audio signals</strong> <mark>Output</mark> <strong>Ambisonic signals</strong></small></p>
|
||||
<p>The <strong>Granular</strong>Encoder takes a mono or stereo input signal, and encodes short ‘grains’ of audio into the Ambisonic domain. The grains are distributed around a controllable center direction, and you are able to specify the size and shape of the distribution. The ‘Mix’ parameter allows to blend between standard encoding (dry) and granular encoding (wet). This allows to use GranularEncoder either as an Ambisonic granulator (wet setting) or as an Ambisonic grain delay (mixed setting). The plug-in comes with integrated modulators for the grain parameters (cf. small sliders), which are useful to vary e.g. the grain seed ‘Position’ in the internal audio buffer.</p>
|
||||
<p>To get a visual impression of the grain distribution it is recommended to use the <strong>Energy</strong>Visualizer plug-in, placed e.g. on your Ambisonic bus before decoding. A detailed description can be found in the <a href="../granularencoder/"><strong>Granular</strong>Encoder Guide</a>.</p>
|
||||
<h3 id="matrixmultiplier">MatrixMultiplier</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Matrix Transformation</strong> <mark>Input</mark> <strong>Audio/Ambisonic signals</strong> <mark>Output</mark> <strong>Audio/Ambisonic signals</strong></small></p>
|
||||
<p>With the <strong>Matrix</strong>Multiplier you can load a configuration which contains a TransformationMatrix object (see the <a href="https://plugins.iem.at/docs/configurationfiles/">Configuration Files Guide</a>) which will be applied to the input signal.</p>
|
||||
<h3 id="multibandcompressor">MultiBandCompressor</h3>
|
||||
<p><small><mark>Author</mark> <strong>Markus Huber</strong> <mark>Type</mark> <strong>Dynamics</strong> <mark>Input</mark> <strong>Ambisonic signals</strong> <mark>Output</mark> <strong>Ambisonic signals</strong></small></p>
|
||||
<p>The <strong>MultiBand</strong>Compressor splits your Ambisonic signal into four bands and compresses them individually. The filters’ crossover frequencies can be adjusted to taste, and the individual compressors are fully configurable as well, making use of the parameters threshold, knee, ratio, attac, release, and make-up gain. With the master controls you can adjust the parameters of all compressors simultaneously.</p>
|
||||
<p>The used filters are Linkwitz-Rilely crossovers, so the magnitude response of your mix will be retained, without introducing any phase issues. The plug-in will also retain the spatial image of your mix, by analyzing the omni-channel (W) of each band and applying the same gain reduction to all channels.</p>
|
||||
<h3 id="multiencoder">MultiEncoder</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Encoder</strong> <mark>Input</mark> <strong>Audio signals</strong> <mark>Output</mark> <strong>Ambisonic signals</strong></small></p>
|
||||
<p>With the <strong>Multi</strong>Encoder you can encode multiple sources with just one plug-in. Select your desired number of input sources in the upper left corner (up to 64). Every source can be panned, muted and soloed individually. Also the gain and label color can be adjusted. Enable the „Lock Directions“-button to let the sources stick to the Master-Panner, which can be controlled separately.</p>
|
||||
<p>Double-click on the sphere, to switch to a linear elevation representation, to get a higher resolution at the horizon. Every ring represents 15° of elevation.</p>
|
||||
<h3 id="multieq">MultiEQ</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Filter</strong> <mark>Input</mark> <strong>Audio signals</strong> <mark>Output</mark> <strong>Audio signals</strong></small></p>
|
||||
<p>The <strong>Multi</strong>EQ is a simple multi-channel equalizer, filtering up to 64 audio channels. Each of the six filters can be switched on and of, and different filter types can be selected. The first and last filter offer additional filter types: low-/high-pass of different filter orders. The 4th order HP / LP can be used to create a Linkwitz-Riley crossover, good for splitting audio into two bands, with a constant magnitude response when added together.</p>
|
||||
<h3 id="omnicompressor">OmniCompressor</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Dynamics</strong> <mark>Input</mark> <strong>Ambisonic signals</strong> <mark>Output</mark> <strong>Ambisonic signals</strong></small></p>
|
||||
<p>The <strong>Omni</strong>Compressor is an Ambisonic compressor which can also be used as a limiter. It works like a regular audio compressor. It uses the first input channel (W-channel, hence the name ‘Omni’) as the driving signal and applies the calculated gains to all the whole Ambisonic signal. When the ratio is set to infinity and the attack time is set to 0.0 ms, the compressor acts as a brick-wall limiter. With that setting you might want to enable the lookahead feature to avoid unwanted distortions. This introduces a delay of 5ms to your signal. There is a hidden parameter (when disabling the plug-ins GUI in your DAW) with which you can deactivate latency reporting to your DAW to compensate for it. This improves the playback performance of your project, however the delay won’t be compensated for with that setting.</p>
|
||||
<h3 id="probedecoder">ProbeDecoder</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Analysis tool/Decoder</strong> <mark>Input</mark> <strong>Ambisonic signals</strong> <mark>Output</mark> <strong>Mono audio signal</strong></small></p>
|
||||
<p>The <strong>Probe</strong>Decoder is a little tool to sample/decode the Ambisonic input signal for one specific direction and listen to the output, mostly used for testing purposes.</p>
|
||||
<h3 id="roomencoder">RoomEncoder</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Encoder+Reverb</strong> <mark>Input</mark> <strong>Mono audio signal</strong> or <strong>Directivity signals</strong> <mark>Output</mark> <strong>Ambisonic signals</strong></small></p>
|
||||
<p>The <strong>Room</strong>Encoder is so far the most complex and computation-intensive plug-in within the IEM Plug-in Suite. With it, you can put a source and a listener into a virtual shoebox-shaped room and render over 200 wall reflections. You can move the source and the listener freely within the room, which will automatically generate the Doppler-shift effect. Use the overall reflection coefficient and the low-shelve and high-shelve filters to change the character of the reflections. With the <em>additional attenuation</em> controls, you can add an additional attenuation for each of the six walls. E.g. you can get rid of the upper wall by completly turning down the ceiling attenuation.</p>
|
||||
<p>But thats not all: The RoomEncoder takes DirectivitySignals (or when set to 0th order also monophonic signals) as an input signal. This means that your source can have a frequency-dependent and time-variable (e.g. changing the direction of the source) directivity. The DirectivityShaper plugin is a good tool to create various directivity patterns.</p>
|
||||
<p>You can sync several RoomEncoder instances to render the same room using one of the four synchronization channels. You can also select, which parameters to sync (listener position, room size, and reflection properties).</p>
|
||||
<h3 id="scenerotator">SceneRotator</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Rotator</strong> <mark>Input</mark> <strong>Ambisonic signals</strong> <mark>Output</mark> <strong>Ambisonic signals</strong></small></p>
|
||||
<p>The <strong>Scene</strong>Rotator is an Ambisonic rotator plug-in, which rotates the Ambisonic scene, hence the name! It can be used with Yaw-Pitch-Roll rotation data, or alternativeley with Quaternions. Each of the three rotation axes can be flipped, and aso the Quaternion data can be inverted. Rotations then happen in reversed direction. You can also select the order of rotations. Ambisonic rotation is very useful in combination with binaural playback, so head-movements can be compensated for.</p>
|
||||
<h3 id="simpledecoder">SimpleDecoder</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Decoder</strong> <mark>Input</mark> <strong>Ambisonic signals</strong> <mark>Output</mark> <strong>Audio signals</strong></small></p>
|
||||
<p>The <strong>Simple</strong>Decoder reads JSON configuration files and decodes the Ambisonic input signal to loudspeaker signals. You can create these configuration files with the <strong>AllRA</strong>Decoder. For more information about the structure of the configuration files read the <a href="https://plugins.iem.at/docs/configurationfiles/">Configuration Files Guide</a>.</p>
|
||||
<p>When setting the subwoofer mode to <strong>discrete</strong> or <strong>virtual</strong> the Ambisonic input signal is filtered into two bands (using 4th order high-/low-pass). The upper band is decoded according to the loaded configuration file. If set to <strong>discrete</strong>, the lower is sent to the defined subwoofer channel. You will get a warning, if you choose a channel, which is already used by the decoder. If set to <strong>virtual</strong>, the low-pass signal is passed to all loudspeakers, creating a <em>virtual</em> subwoofer.</p>
|
||||
<h3 id="stereoencoder">StereoEncoder</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Encoder</strong> <mark>Input</mark> <strong>Mono or stereo audio signals</strong> <mark>Output</mark> <strong>Ambisonic signals</strong></small></p>
|
||||
<p>Use the <strong>Stereo</strong>Encoder plug-in to encode mono or stereo audio signals into the Ambisonic domain. In the upper right corner you can set your desired Ambisonic order and normalization. Make sure your bus size is big enough. Set to Auto-mode and the plug-in adapts to changes in bus size by using the highest possible order.</p>
|
||||
<p>Use the <em>Azimuth</em>, <em>Elevation</em> and <em>Roll</em> sliders to pan your source. With the <em>Width</em> slider you can separate the two input channels. You can, of course, use the <strong>Stereo</strong>Encoder also for mono sources. Additionally, there is a quaternion input you can use as an alternative to <em>Azimuth</em> and <em>Elevation</em>, e.g. for orientation data of (head-)trackers.</p>
|
||||
<p>Use the following modifiers and the mouse-wheel to pan with the sphere panner:</p>
|
||||
<ul>
|
||||
<li><kbd>⌘</kbd>/<kbd>CTRL</kbd> : <strong>Azimuth</strong></li>
|
||||
<li><kbd>ALT</kbd> : <strong>Elevation</strong></li>
|
||||
<li><kbd>CMD</kbd> / <kbd>CTRL</kbd> + <kbd>ALT</kbd> : <strong>Roll</strong></li>
|
||||
<li><kbd>SHIFT</kbd> : <strong>Width</strong></li>
|
||||
</ul>
|
||||
<p>Double-click on the sphere, to switch to a linear elevation representation, to get a higher resolution at the horizon. Every ring represents 15° of elevation.</p>
|
||||
<p>There are some key-combinations to directly pan to a specific direction: <kbd>SHIFT</kbd> + <kbd>F</kbd> -> <strong>F</strong>ront, <kbd>SHIFT</kbd> + <kbd>U</kbd> -> <strong>U</strong>p, <kbd>SHIFT</kbd> + <kbd>L</kbd> -> <strong>L</strong>eft, and so on… you can figure out the rest :-)</p>
|
||||
<h3 id="toolbox">ToolBox</h3>
|
||||
<p><small><mark>Author</mark> <strong>Daniel Rudrich</strong> <mark>Type</mark> <strong>Ambisonic tool</strong> <mark>Input</mark> <strong>Ambisonic signals</strong> <mark>Output</mark> <strong>Ambisonic signals</strong></small></p>
|
||||
<p>This is a small plug-in with a growing set of handy features. For now, you can flip the Ambisonic input signal along the x, y and/or z axis. You can also use it as an adapter between plug-ins with an automatic order settings and plug-ins with fixed order settings. The LOA (lower order Ambisonics) Weighting feature lets you mix Ambisonic signals with different orders. Example: use the ToolBox with the setting 1st (input), 5th (output) and maxrE weights for a FOA microphone signal (which is LOA) to include it in your 5th order mix. Note, that the weights have to match you decoder’s weights. The gain control boosts or attenuates the overall audio level.</p>
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
<footer>
|
||||
<footer id="footer">
|
||||
<nav>
|
||||
<ul>
|
||||
<li><span>Plug-in Suite by IEM - Institute of Electronic Music and Acoustics</span></li>
|
||||
<li>
|
||||
<a href="https://iem.at/">IEM.at</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="https://plugins.iem.at/docs/">Docs</a>
|
||||
</li>
|
||||
</ul>
|
||||
</nav>
|
||||
<p>Website created with <a href="https://gohugo.io">Hugo</a> using the <a href="https://github.com/jeblister/kube">kube</a> theme.</p>
|
||||
</footer>
|
||||
|
||||
</footer>
|
||||
|
||||
|
||||
<script src="https://plugins.iem.at/js/kube.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/kube.legenda.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/master.js" type="text/javascript">
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
|
@ -0,0 +1,162 @@
|
|||
<!DOCTYPE html>
|
||||
<html prefix="og: http://ogp.me/ns# article: http://ogp.me/ns/article# " lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Decoding to Loudspeaker Setups | RingBuffer</title>
|
||||
<link href="../../../assets/css/all-nocdn.css" rel="stylesheet" type="text/css">
|
||||
<link href="../../../assets/css/ipython.min.css" rel="stylesheet" type="text/css">
|
||||
<link href="../../../assets/css/nikola_ipython.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://fonts.googleapis.com/css?family=Playfair+Display:700,900" rel="stylesheet">
|
||||
<meta name="theme-color" content="#5670d4">
|
||||
<meta name="generator" content="Nikola (getnikola.com)">
|
||||
<link rel="alternate" type="application/rss+xml" title="RSS" href="../../../rss.xml">
|
||||
<link rel="canonical" href="http://ringbuffer.org/spatial_audio/iem-reaper/decoding-loudspeaker-iem/">
|
||||
<script type="text/x-mathjax-config">
|
||||
MathJax.Hub.Config({
|
||||
tex2jax: {
|
||||
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
|
||||
displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
|
||||
processEscapes: true
|
||||
},
|
||||
displayAlign: 'center', // Change this to 'left' if you want left-aligned equations.
|
||||
"HTML-CSS": {
|
||||
styles: {'.MathJax_Display': {"margin": 0}}
|
||||
}
|
||||
});
|
||||
</script><!--[if lt IE 9]><script src="../../../assets/js/html5.js"></script><![endif]--><link rel="stylesheet" href="../../../assets/hvc/pkgindex.css">
|
||||
<meta name="author" content="Paul Schuladen">
|
||||
<link rel="prev" href="../../../faust/physical_modeling/faust-waveguide-input/" title="Waveguide with Excitation Input" type="text/html">
|
||||
<link rel="next" href="../../../sound_synthesis_introduction/Additive_Spectral/additive-spectral-parabolic-interpolation/" title="Additive & Spectral: Parabolic Interpolation" type="text/html">
|
||||
<meta property="og:site_name" content="RingBuffer">
|
||||
<meta property="og:title" content="Decoding to Loudspeaker Setups">
|
||||
<meta property="og:url" content="http://ringbuffer.org/spatial_audio/iem-reaper/decoding-loudspeaker-iem/">
|
||||
<meta property="og:description" content='Creating a Decoder
|
||||
Decoder can be created using the allrad decoder. In there the coordinats for each speaker can be set. The decoder requires a "body" meaning that a purely two dimensional speaker set'>
|
||||
<meta property="og:type" content="article">
|
||||
<!-- <meta property="article:published_time" content="2022-05-06T14:00:00Z"> -->
|
||||
</head>
|
||||
<body>
|
||||
<a href="#content" class="sr-only sr-only-focusable">Skip to main content</a>
|
||||
|
||||
<!-- Header and menu bar -->
|
||||
<div class="container">
|
||||
<header class="blog-header py-3"><div class="row nbb-header align-items-center">
|
||||
<div class="col-md-3 col-xs-2 col-sm-2" style="width: auto;">
|
||||
<button class="navbar-toggler navbar-light bg-light nbb-navbar-toggler" type="button" data-toggle="collapse" data-target=".bs-nav-collapsible" aria-controls="bs-navbar" aria-expanded="false" aria-label="Toggle navigation">
|
||||
<span class="navbar-toggler-icon"></span>
|
||||
</button>
|
||||
<div class="collapse bs-nav-collapsible bootblog4-search-form-holder">
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-6 col-xs-10 col-sm-10 bootblog4-brand" style="width: auto;">
|
||||
<a class="navbar-brand blog-header-logo text-dark" href="http://ringbuffer.org/">
|
||||
|
||||
<!-- <span id="blog-title">RingBuffer</span> -->
|
||||
<img src="../../../images/ringbuffer_BANNER.png" alt="LOGO" class="center"></a>
|
||||
</div>
|
||||
<div class="col-md-3 justify-content-end align-items-center bs-nav-collapsible collapse flex-collapse bootblog4-right-nav">
|
||||
<nav class="navbar navbar-light bg-white"><ul class="navbar-nav bootblog4-right-nav"></ul></nav>
|
||||
</div>
|
||||
</div>
|
||||
</header><nav class="navbar navbar-expand-md navbar-light bg-white static-top"><div class="collapse navbar-collapse bs-nav-collapsible" id="bs-navbar">
|
||||
<ul class="navbar-nav nav-fill d-flex w-100">
|
||||
<li class="nav-item">
|
||||
<a href="../../../teaching/computer-music-basics" class="nav-link">Computer Music Basics</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a href="../../../teaching/sound-synthesis-introduction" class="nav-link">Sound Synthesis Introduction</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a href="../../../teaching/spatial-audio" class="nav-link">Spatial Audio</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a href="../../../teaching/building-instruments-in-faust" class="nav-link">Faust</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a href="../../../teaching/sound-synthesis-cpp" class="nav-link">C++</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a href="../../../teaching/misc-tools" class="nav-link">Misc Tools</a>
|
||||
|
||||
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<!-- /.navbar-collapse -->
|
||||
</nav>
|
||||
</div>
|
||||
|
||||
<div class="container" id="content" role="main">
|
||||
<div class="body-content">
|
||||
<!--Body content-->
|
||||
|
||||
|
||||
<article class="post-text h-entry hentry postpage" itemscope="itemscope" itemtype="http://schema.org/Article"><header><h1 class="p-name entry-title" itemprop="headline name"><a href="." class="u-url">Decoding to Loudspeaker Setups</a></h1>
|
||||
|
||||
<div class="metadata">
|
||||
<p class="byline author vcard p-author h-card"><span class="byline-name fn p-name" itemprop="author">
|
||||
<a class="u-url" href="../../../authors/paul-schuladen/">Paul Schuladen</a>
|
||||
</span></p>
|
||||
<!-- <p class="dateline">
|
||||
<a href="/spatial_audio/iem-reaper/decoding-loudspeaker-iem/" rel="bookmark">
|
||||
<time class="published dt-published" datetime="2022-05-06T14:00:00Z" itemprop="datePublished" title="2022-05-06 14:00">2022-05-06 14:00</time>
|
||||
</a>
|
||||
</p> -->
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
</header><div class="e-content entry-content" itemprop="articleBody text">
|
||||
<div class="section" id="creating-a-decoder">
|
||||
<h2>Creating a Decoder</h2>
|
||||
<p>Decoder can be created using the allrad decoder. In there the coordinats for each speaker can be set. The decoder requires a "body" meaning that a purely two dimensional speaker setup will not work. For that Imaginary speaker can be placed. Audio that is rendered on it is send to all connected speakers. To avoid this a gain factor can be applied.</p>
|
||||
</div>
|
||||
<div class="section" id="using-a-decoder">
|
||||
<h2>Using a Decoder</h2>
|
||||
<p>Route an ambisonics signal into the decoder channel. Make sure to send all channels (and not just 1/2).</p>
|
||||
<p>Allrad Decoder itself can already process and decode ambisonics. But you can also export the created decoder and speakerlayouts for use in other IEM-Plugins.</p>
|
||||
<p>The "SimpleDecoder" can import decoder files. With this plugin you are able to use a discrete subwoofer without extra signal splitting and processing and also a so called virtual subwoofer which sends the low frequency audio to all speakers.</p>
|
||||
</div>
|
||||
<div class="section" id="distance-compensator">
|
||||
<h2>Distance Compensator</h2>
|
||||
<p>A normal ambisonics decoder does only decode the direction of the incoming soundfield. So this would only work cleanly on a perfect sphere or circle. In realworld scenario this is barely the case, the distance from the speaker to the listener might diverge leading to different level and delays of the signal at the listeners position. To compensate for that effect the "DistanceCompensator" can be used. You can load the speaker layout file into it or adjust the distance manually.</p>
|
||||
</div>
|
||||
</div>
|
||||
<aside class="postpromonav"><nav><ul class="pager hidden-print">
|
||||
<!-- <li class="previous"> --><!-- <a href="/faust/physical_modeling/faust-waveguide-input/" rel="prev" title="Waveguide with Excitation Input">Previous post</a> --><!-- </li> --><!-- <li class="next">
|
||||
<a href="/sound_synthesis_introduction/Additive_Spectral/additive-spectral-parabolic-interpolation/" rel="next" title="Additive & Spectral: Parabolic Interpolation">Next post</a>
|
||||
</li> -->
|
||||
</ul></nav></aside><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_HTMLorMML" integrity="sha384-3lJUsx1TJHt7BA4udB5KPnDrlkO8T6J6v/op7ui0BbCjvZ9WqV4Xm6DTP6kQ/iBH" crossorigin="anonymous"></script><script type="text/x-mathjax-config">
|
||||
MathJax.Hub.Config({
|
||||
tex2jax: {
|
||||
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
|
||||
displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
|
||||
processEscapes: true
|
||||
},
|
||||
displayAlign: 'center', // Change this to 'left' if you want left-aligned equations.
|
||||
"HTML-CSS": {
|
||||
styles: {'.MathJax_Display': {"margin": 0}}
|
||||
}
|
||||
});
|
||||
</script></article><!--End of body content--><footer id="footer"><center>
|
||||
<hr>
|
||||
<br>
|
||||
Contents © Henrik von Coler 2022 - <a href="../../../contact"> Contact</a>
|
||||
</center>
|
||||
<br></footer>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<script src="../../../assets/js/all-nocdn.js"></script><script>
|
||||
baguetteBox.run('div#content', {
|
||||
ignoreClass: 'islink',
|
||||
captions: function(element) {
|
||||
return element.getElementsByTagName('img')[0].alt;
|
||||
}});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
10
research/playback-calibration/mirrors/sparta-overview.html
Normal file
10
research/playback-calibration/mirrors/sparta-overview.html
Normal file
File diff suppressed because one or more lines are too long
83
research/playback-calibration/summary.md
Normal file
83
research/playback-calibration/summary.md
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
# Playback Calibration And Loudspeaker Rendering
|
||||
|
||||
## Objective
|
||||
|
||||
Account for the room's actual speaker output path when rendering the AquaSynth/Ambisonic scene, so the spatial field is not merely correct inside the DSP graph but wrong in the air.
|
||||
|
||||
## Current Mechanism
|
||||
|
||||
The previous Ambisonic research path ends at "decoder / renderer: binaural, stereo, OBS, speakers." That is too vague. For a speaker playback rig, the renderer has to own the actual speaker layout, output-device latency, per-speaker delay/gain, and room/loudspeaker response at the listening position.
|
||||
|
||||
## Main Finding
|
||||
|
||||
Ambisonics is scene-based: the B-format/HOA bus is not a speaker feed. A decoder converts that scene into a specific playback layout. ICST describes the workflow as source -> encoder -> B-format master -> decoder -> speakers; the same scene can feed different decoders without changing the scene itself [ICST].
|
||||
|
||||
IEM's AllRADecoder is explicitly for arbitrary loudspeaker layouts, and IEM configuration files can store loudspeaker directions and decoder matrices [IEM AllRAD; IEM Config]. SPARTA also includes `sparta_ambiDEC`, a frequency-dependent loudspeaker ambisonic decoder with user-specified loudspeaker directions and JSON import [SPARTA]. So the playback side should use an explicit decoder configuration, not a hard-coded stereo-ish downmix.
|
||||
|
||||
Impulse response measurement matters because the physical speaker chain contributes delay, level, frequency response, and room reflections. FLUX:: MiRA summarizes impulse response measurement as a way to characterize a room/venue and amplifier/loudspeaker behavior [MiRA]. For this project, we need enough measurement to align arrivals and preserve spatial imaging, not a maximal audiophile pilgrimage with incense and regret.
|
||||
|
||||
## Required Playback Model
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["AquaSynth Ambisonic bus"] --> B["Layout-specific ambisonic decoder"]
|
||||
C["Speaker geometry: x/y/z or az/el/r"] --> B
|
||||
D["Output device latency"] --> E["Render alignment"]
|
||||
F["Per-speaker IR measurements"] --> G["Delay/gain/EQ correction"]
|
||||
B --> G
|
||||
G --> E
|
||||
E --> H["Physical speakers in room"]
|
||||
H --> I["Listening position / performance zone"]
|
||||
I --> J["Measurement feedback"]
|
||||
J --> F
|
||||
```
|
||||
|
||||
## Invariants
|
||||
|
||||
- The ambisonic bus remains speaker-independent until the decoder stage.
|
||||
- The decoder is specific to the active speaker layout.
|
||||
- Speaker positions, output channel mapping, delays, trims, and correction filters are configuration, not folklore.
|
||||
- Capture-side sync and playback-side alignment are separate responsibilities.
|
||||
- If speakers are also used for calibration pings, acoustic echo from playback into microphones must be modeled or gated; otherwise the sync estimator may happily synchronize to its own tail.
|
||||
|
||||
## Practical Design
|
||||
|
||||
1. Declare one playback profile.
|
||||
Store speaker count, channel order, device name, sample rate, nominal buffer size, and speaker coordinates.
|
||||
|
||||
2. Measure output latency and per-speaker impulse responses.
|
||||
Send a chirp/pulse through each speaker one at a time and capture it at the reference mic/listening position. Extract direct-arrival delay, level, polarity, and a short correction response.
|
||||
|
||||
3. Generate or load a decoder.
|
||||
Use AllRAD/IEM or SPARTA-style layout config first. Export the decoder matrix/profile if possible. Do not invent a decoder until existing tools fail a real invariant.
|
||||
|
||||
4. Apply per-speaker correction after decoding.
|
||||
The Ambisonic decoder outputs ideal speaker feeds. Delay/gain/EQ correction belongs after that, because it corrects the physical playback channels.
|
||||
|
||||
5. Align live monitoring latency.
|
||||
The live mic field, synth outputs, Faust DSP latency, decoder latency, audio driver buffer, and speaker propagation delay all contribute to when sound arrives. Keep a latency ledger so visual/OBS output and room playback can be aligned deliberately.
|
||||
|
||||
6. Prevent speaker output from corrupting mic sync.
|
||||
During calibration, mute/route intentionally. During live operation, the sync estimator should distinguish external voices from playback bleed when possible. At minimum, tag known outgoing calibration signals and avoid using those windows for passive drift estimates unless deliberately measuring echo.
|
||||
|
||||
## Fit With Live Adaptive Sync
|
||||
|
||||
The live adaptive sync system aligns microphone streams into one capture-time bus. The playback calibration system aligns rendered speaker feeds into one room-time presentation. These two machines meet at the AquaSynth graph, but they should not share a compensator. Shared compensators are where clear systems go to become soup.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- How many speakers are wired in, and through what output device/interface?
|
||||
- Are they stereo, quad, 5.1-ish, asymmetrical room speakers, or something stranger?
|
||||
- What is the intended sweet spot: the operator chair, camera zone, whole room, or a performer's position?
|
||||
- Will the speakers be audible to the microphones during live use?
|
||||
- Does AquaSynth host the decoder/correction itself, or should the speaker renderer live outside AquaSynth after the Faust graph?
|
||||
|
||||
## Citations
|
||||
|
||||
- [IEM AllRAD] IEM Plug-in Suite, "AllRADecoder Guide." Mirror: `mirrors/iem-allradecoder-guide.html`.
|
||||
- [IEM Config] IEM Plug-in Suite, "Configuration Files." Mirror: `mirrors/iem-configuration-files.html`.
|
||||
- [IEM Plugins] IEM Plug-in Suite, "Plug-in Descriptions." Mirror: `mirrors/iem-plugin-descriptions.html`.
|
||||
- [RingBuffer] RingBuffer, "Decoding to Loudspeaker Setups." Mirror: `mirrors/ringbuffer-decoding-loudspeaker-iem.html`.
|
||||
- [SPARTA] SPARTA, "Overview." Mirror: `mirrors/sparta-overview.html`.
|
||||
- [ICST] ICST Ambisonics, "How it Works." Mirror: `mirrors/icst-how-it-works.html`.
|
||||
- [MiRA] FLUX:: MiRA, "Impulse response measurement." Mirror: `mirrors/flux-mira-impulse-response-measurement.html`.
|
||||
5
research/spout-obs-bridge/bibliography.md
Normal file
5
research/spout-obs-bridge/bibliography.md
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
# Spout OBS Bridge Bibliography
|
||||
|
||||
- Off World Live, `obs-spout2-plugin`, OBS plugin implementing Spout2 input/output. Mirror: `mirrors/obs-spout2-plugin-github.html`.
|
||||
- leadedge, `Spout2`, Windows video frame sharing system and SDK. Mirror: `mirrors/leadedge-spout2-github.html`.
|
||||
- leadedge, Spout project home. Mirror: `mirrors/spout-home.html`.
|
||||
1822
research/spout-obs-bridge/mirrors/leadedge-spout2-github.html
Normal file
1822
research/spout-obs-bridge/mirrors/leadedge-spout2-github.html
Normal file
File diff suppressed because one or more lines are too long
1821
research/spout-obs-bridge/mirrors/obs-spout2-plugin-github.html
Normal file
1821
research/spout-obs-bridge/mirrors/obs-spout2-plugin-github.html
Normal file
File diff suppressed because one or more lines are too long
102
research/spout-obs-bridge/mirrors/spout-home.html
Normal file
102
research/spout-obs-bridge/mirrors/spout-home.html
Normal file
|
|
@ -0,0 +1,102 @@
|
|||
<!DOCTYPE html>
|
||||
<html>
|
||||
|
||||
<head>
|
||||
<title>Spout</title>
|
||||
<link rel="stylesheet" type="text/css" href="styles\style2.css">
|
||||
<meta name="viewport" content="width=device-width">
|
||||
</head>
|
||||
|
||||
<body>
|
||||
|
||||
<div class="layout">
|
||||
|
||||
<script>
|
||||
function goBack() {
|
||||
window.history.back();
|
||||
}
|
||||
</script>
|
||||
|
||||
<center>
|
||||
|
||||
<div class="subheading">
|
||||
<h1>Spout</h1>
|
||||
<p>
|
||||
<a href="https://spout.zeal.co/" target="_blank">Spout</a>
|
||||
is a system for Microsoft Windows for GPU accelerated sharing<br>
|
||||
of graphics between applications in a similar way to <a href="http://syphon.v002.info/" target="_blank">Syphon</a> for OSX.<br>
|
||||
Spout supports DirectX 9, DirectX 11 and OpenGL textures.<br>
|
||||
A comprehensive SDK can be used for creating applications.<br>
|
||||
Explore the other <a href="https://github.com/leadedge/" target="_blank">Spout repositories</a> for more detail.</a><br>
|
||||
Spout setup or the Spout SDK can be downloaded below.
|
||||
</p>
|
||||
|
||||
<h2><a href="spout-download.html">Download</a></h2>
|
||||
|
||||
</p><br>
|
||||
|
||||
<h1>Open source projects</h1>
|
||||
<p>
|
||||
As a convenient start, either for development or for practical use<br>
|
||||
explore a <a href="spout-projects.html">sorted list</a></h2> of open source projects that use Spout.
|
||||
</p><br>
|
||||
|
||||
<h1>SpoutCam</h1>
|
||||
|
||||
<p>"SpoutCam" is a virtual webcam that is also a Spout receiver. A setup program provides options for frame rate and resolution and can be used to register for both 32 bit and 64 bit host programs.
|
||||
<br><br>
|
||||
Download the latest release from the repository Releases page below.<br>
|
||||
Refer to the instructions and "readme.txt" in the archive for further details.
|
||||
</p>
|
||||
|
||||
<h2><a href="https://github.com/leadedge/SpoutCam/releases">Download</a></h2>
|
||||
<br>
|
||||
|
||||
<h1>Spout To NDI</h1>
|
||||
<p>
|
||||
<a href="https://raw.githubusercontent.com/leadedge/leadedge.github.io/downloads/SPOUTtoNDI_2030.pdf" target="_blank">Spout to NDI</a> is a set of programs that allow Spout senders and receivers to share video and audio by way of a network using the <a href="https://ndi.video/" target="_blank">NDI</a> (“Network Device Interface”) protocol, a video connectivity standard that enables multimedia systems to communicate with one another over IP. Options are available for Spout to NDI to capture system audio and send it together with image data and for NDI to Spout to receive audio from the NDI source as well as the graphics and play it through the speakers. Conversion from GPU textures is done using Openframeworks shaders by way of the <a href="https://github.com/leadedge/ofxNDI" target="_blank">ofxNDI</a> addon. Unzip the distribution file to any convenient folder.<br>
|
||||
|
||||
<h2><a href="https://github.com/leadedge/leadedge.github.io/raw/downloads/SPOUTtoNDI_v2030.zip">Download</a></h2>
|
||||
<br>
|
||||
|
||||
<h1>Spout To OMT</h1>
|
||||
<p>
|
||||
<a href="https://raw.githubusercontent.com/leadedge/leadedge.github.io/downloads/SPOUT to OMT.pdf" target="_blank">Spout to OMT</a> is a set of programs that allow Spout senders and receivers to share video by way of a network using the <a href="https://www.openmediatransport.org/" target="_blank">OMT</a> (“Open Media Transport”) protocol. This is an open-source network protocol for high performance, low latency video over a local area network. Conversion from GPU textures is done using high speed compute shaders for minimum overhead. Unzip the distribution file to any convenient folder.<br>
|
||||
|
||||
<h2><a href="https://github.com/leadedge/leadedge.github.io/raw/downloads/Spout to OMT_1000.zip">Download</a></h2>
|
||||
|
||||
</p><br>
|
||||
|
||||
<h1>Contact and support</h1>
|
||||
<p>
|
||||
Please visit the <a href="https://spout.discourse.group/" target="_blank">Spout Discourse group</a>.
|
||||
</p><br>
|
||||
|
||||
<h1>Donations</h1>
|
||||
<p>Spout is open source software and there is no company or income to assist ongoing development, maintenance and support. So please consider becoming a supporter to help keep the project going. All contributions are most welcome.</p>
|
||||
|
||||
Show your support and become a <a href="https://github.com/leadedge/" target="_blank"><font size="4">Sponsor</font></b></a> through GitHub<br><br>
|
||||
or<br><br>
|
||||
|
||||
Make a donation through PayPal<br>
|
||||
|
||||
|
||||
<form action="https://www.paypal.com/cgi-bin/webscr" method="post" target="_top">
|
||||
<input type="hidden" name="cmd" value="_s-xclick">
|
||||
<input type="hidden" name="hosted_button_id" value="P4P4QJZBT87PJ">
|
||||
<input type="image" src="https://www.paypalobjects.com/en_AU/i/btn/btn_donate_SM.gif" border="0" name="submit" alt="PayPal – The safer, easier way to pay online!">
|
||||
<img alt="" border="0" src="https://www.paypalobjects.com/en_AU/i/scr/pixel.gif" width="1" height="1">
|
||||
</form>
|
||||
|
||||
</p><br>
|
||||
|
||||
<button
|
||||
style="font-family: Arial, Helvetica, sans-serif;"
|
||||
onclick="goBack()">Back</button></br></br></br></br>
|
||||
|
||||
</center>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
7
research/spout-obs-bridge/summary.md
Normal file
7
research/spout-obs-bridge/summary.md
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
# Spout OBS Bridge Summary
|
||||
|
||||
Spout is the Windows-local GPU texture sharing bridge for this rig. The OBS path is the Off World Live OBS Spout2 plugin, which creates an OBS source from a Spout shared texture and can also send the OBS canvas back to Spout. The Spout2 SDK is the application-side sender/receiver surface; its modern path centers on DirectX shared textures and has native DirectX/OpenGL integration options.
|
||||
|
||||
Decision: Mimir should not render final pixels itself. It should publish typed render-frame packets from the fusion cache. Aquarium Engine should consume those packets, render them into a GPU render target, and publish that texture as a named Spout sender. OBS then receives that sender through the OBS Spout2 plugin.
|
||||
|
||||
Latency decision: the render packet carries both source timestamps and intended presentation time. The stream compositor can buffer visual frames until they align with the ambisonic audio field's presentation timeline. Latency is allowed; unbounded drift is not.
|
||||
62
research/visual-spatial-map/bibliography.md
Normal file
62
research/visual-spatial-map/bibliography.md
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
# Bibliography: Visual Spatial Map And Splat Scene
|
||||
|
||||
Mirrored copies live under `research/visual-spatial-map/mirrors/`.
|
||||
|
||||
## Camera Hardware / Capture
|
||||
|
||||
- [PS3 Developer Wiki, "PlayStation Eye"](https://www.psdevwiki.com/ps3/PlayStation_Eye)
|
||||
Mirror: `mirrors/ps3-playstation-eye-devwiki.html`
|
||||
|
||||
- [PS2 Developer Wiki, "EyeToy"](https://www.psdevwiki.com/ps2/EyeToy)
|
||||
Mirror: `mirrors/ps2-eyetoy-devwiki.html`
|
||||
|
||||
- [iPiSoft Wiki, "User Guide for Multiple PlayStation Eye Cameras Configuration"](https://wiki.ipisoft.com/index.php?title=User_Guide_for_Multiple_PlayStation_Eye_Cameras_Configuration_%28printable_version%29&useskin=cleanmonobook)
|
||||
Mirror: `mirrors/ipisoft-multiple-playstation-eye-cameras.html`
|
||||
|
||||
- [opentrack Wiki, "PS3 Eye open driver instructions"](https://github.com/opentrack/opentrack/wiki/PS3-Eye-open-driver-instructions/54df68ead70a4069d24bc96f987feb1f57f1b25a)
|
||||
Mirror: `mirrors/opentrack-ps3-eye-open-driver-instructions.html`
|
||||
|
||||
- [Zadig](https://zadig.akeo.ie/)
|
||||
Mirror: `mirrors/zadig-home.html`
|
||||
|
||||
- [Code Laboratories, "CL Eye Platform Driver"](https://codelaboratories.com/products/eye/driver/)
|
||||
Mirror: `mirrors/code-laboratories-cl-eye-driver.html`
|
||||
|
||||
- [DelanClip, "PS3 Eye Camera Head Tracking - OpenTrack - Open Driver"](https://delanclip.com/manuals/ps3-eye-camera-head-tracking/)
|
||||
Mirror: `mirrors/delanclip-ps3-eye-open-driver.html`
|
||||
|
||||
- [Leap Motion, `leapuvc`](https://github.com/leapmotion/leapuvc)
|
||||
Mirrors: `mirrors/leapuvc-github.html`, `mirrors/leapuvc-readme.md`, `mirrors/leapuvc-manual.pdf`
|
||||
|
||||
- [Ultraleap, "Images"](https://docs.ultraleap.com/api-reference/tracking-api/leapc-guide/images.html)
|
||||
Mirror: `mirrors/ultraleap-leapc-images.html`
|
||||
|
||||
- [Ultraleap, "Images Example"](https://docs.ultraleap.com/api-reference/tracking-api/examples/images-example.html)
|
||||
Mirror: `mirrors/ultraleap-images-example.html`
|
||||
|
||||
## Multi-Camera Calibration / Tracking Geometry
|
||||
|
||||
- [OpenCV, "Multi-camera Calibration"](https://docs.opencv.org/3.4/d2/d1c/tutorial_multi_camera_main.html)
|
||||
Mirror: `mirrors/opencv-multi-camera-calibration.html`
|
||||
|
||||
- [Multi-camera Calibration documentation](https://multi-camera-calibration.readthedocs.io/en/latest/)
|
||||
Mirror: `mirrors/multi-camera-calibration-docs.html`
|
||||
|
||||
- Florian Jacob and Reinhold Haeb-Umbach, ["Absolute Geometry Calibration of Distributed Microphone Arrays in an Audio-Visual Sensor Network"](https://arxiv.org/abs/1504.03128), arXiv:1504.03128, 2015.
|
||||
Mirror: `mirrors/krekovic-2015-audio-visual-geometry-calibration.pdf`
|
||||
|
||||
## Gaussian Splatting / Dynamic Scene Rendering
|
||||
|
||||
- Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, and George Drettakis, ["3D Gaussian Splatting for Real-Time Radiance Field Rendering"](https://arxiv.org/abs/2308.04079), SIGGRAPH 2023.
|
||||
Mirror: `mirrors/kerbl-2023-3d-gaussian-splatting.pdf`
|
||||
|
||||
- Guanjun Wu et al., ["4D Gaussian Splatting for Real-Time Dynamic Scene Rendering"](https://openaccess.thecvf.com/content/CVPR2024/papers/Wu_4D_Gaussian_Splatting_for_Real-Time_Dynamic_Scene_Rendering_CVPR_2024_paper.pdf), CVPR 2024.
|
||||
Mirror: `mirrors/wu-2024-4d-gaussian-splatting.pdf`
|
||||
|
||||
- Zhexi Peng et al., ["RTG-SLAM: Real-time 3D Reconstruction at Scale using Gaussian Splatting"](https://arxiv.org/abs/2404.19706), SIGGRAPH 2024.
|
||||
Mirror: `mirrors/peng-2024-rtg-slam.pdf`
|
||||
|
||||
## Ambisonic View / Pose Coupling
|
||||
|
||||
- [IEM Plug-in Suite, "SceneRotator Guide"](https://plugins.iem.at/docs/scenerotator/)
|
||||
Mirror: `mirrors/iem-scenerotator-guide.html`
|
||||
202
research/visual-spatial-map/brushstroke-fusion-rendering.md
Normal file
202
research/visual-spatial-map/brushstroke-fusion-rendering.md
Normal file
|
|
@ -0,0 +1,202 @@
|
|||
# Brushstroke Fusion Rendering Approach
|
||||
|
||||
## Objective
|
||||
|
||||
Fuse two PS3 Eyes, two Razer Kiyo-class RGB/mic cameras, one Leap Motion/LeapUVC stereo IR sensor, microphones, and speakers into one live spatial state, then render it as a volumetric brushstroke scene with a virtual camera/listener pose that also drives the ambisonic response.
|
||||
|
||||
## Current Mechanism
|
||||
|
||||
The visual-spatial map research established the required world authority: every sensor, emitter, prop, and virtual camera/listener must publish into one canonical coordinate system. The EpiphanyAquarium `fractal-domains-cache-that-bites.md` note adds the render architecture lesson:
|
||||
|
||||
```text
|
||||
semantic intent
|
||||
-> spatial domain
|
||||
-> field grammar
|
||||
-> ownership tree
|
||||
-> conservative summaries
|
||||
-> contribution cache
|
||||
-> backend packets
|
||||
-> renderer
|
||||
```
|
||||
|
||||
For Mimir, the equivalent is:
|
||||
|
||||
```text
|
||||
sensor observations
|
||||
-> world-space tracks and calibration domains
|
||||
-> semantic scene claims
|
||||
-> confidence/ownership tree
|
||||
-> conservative summaries
|
||||
-> contribution cache
|
||||
-> brushstroke / splat packets
|
||||
-> live renderer + ambisonic pose
|
||||
```
|
||||
|
||||
## Invariants
|
||||
|
||||
- The world model owns truth, not the brushstroke renderer.
|
||||
- Every observation is timestamped and carries sensor identity, transform, confidence, and latency.
|
||||
- Camera, microphone, speaker, prop, and listener coordinates live in one frame.
|
||||
- Render packets are disposable cached lowerings of scene claims.
|
||||
- Audio response follows the same virtual listener/camera pose as the visual render.
|
||||
- Sparse tracking must work before dense beauty rendering earns its keep.
|
||||
|
||||
## Fusion Ownership
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["PS3 Eyes"] --> B["high-rate marker detections"]
|
||||
C["LeapUVC stereo IR"] --> D["near-field hands / markers / depth hints"]
|
||||
E["Kiyo / Kiyo Pro RGB"] --> F["RGB frames + feature tracks"]
|
||||
G["Microphones"] --> H["audio source / sync observations"]
|
||||
I["Speakers"] --> J["known playback + echo paths"]
|
||||
B --> K["World Model Authority"]
|
||||
D --> K
|
||||
F --> K
|
||||
H --> K
|
||||
J --> K
|
||||
K --> L["Semantic scene claims"]
|
||||
L --> M["Brushstroke packet cache"]
|
||||
L --> N["Ambisonic scene / listener pose"]
|
||||
M --> O["Volumetric brushstroke renderer"]
|
||||
N --> P["AquaSynth spatial output"]
|
||||
```
|
||||
|
||||
The fusion stack should use a state-estimation model with explicit tracks:
|
||||
|
||||
- `Sensor`: calibrated device, intrinsics/extrinsics, clock model, latency model.
|
||||
- `Observation`: timestamped 2D/3D evidence from a sensor.
|
||||
- `Track`: world-space entity estimate with covariance/confidence.
|
||||
- `Claim`: semantic scene contribution, such as marker, prop, person silhouette, surface patch, speaker, microphone, or audio source.
|
||||
- `Domain`: local coordinate frame that owns recurring detail or brush behavior.
|
||||
- `RenderPacket`: cached brush/splat/impostor representation of claims.
|
||||
|
||||
The tracker can start simple: per-marker Kalman filters or an EKF/UKF once camera geometry and nonlinear projection need it. Factor-graph/SLAM machinery can come later if the rig starts moving or the calibration is solved jointly.
|
||||
|
||||
## Sensor Roles
|
||||
|
||||
- PS3 Eyes: high-rate low-resolution marker observations. Best for bright balls, retroreflective dots, LEDs, wand tracking, and fast prop motion.
|
||||
- LeapUVC: near-field stereo IR observations. Best for hands/control surface, close markers, and calibration events in its view. LeapUVC mode is generic UVC and should not be assumed to coexist with Ultraleap hand tracking.
|
||||
- Kiyos: RGB texture, object appearance, human-readable frames, slower feature tracking, and microphone attachment transforms.
|
||||
- Microphones: audio observations, source localization evidence, and sync residuals.
|
||||
- Speakers: known emitted signals for transfer-path estimation and room calibration.
|
||||
|
||||
## Brushstroke Render Model
|
||||
|
||||
Dreams-style volumetric brushstrokes fit better as a **semantic splat/brush backend** than as the canonical map.
|
||||
|
||||
Each visible scene claim lowers into brush packets:
|
||||
|
||||
```text
|
||||
BrushPacket {
|
||||
stableKey
|
||||
sourceClaim
|
||||
worldFrame
|
||||
envelope
|
||||
strokeAxis / tangent frame
|
||||
radius / anisotropy
|
||||
opacity
|
||||
color source
|
||||
temporal confidence
|
||||
motion blur / trail decay
|
||||
material tags
|
||||
lodRange
|
||||
costTier
|
||||
}
|
||||
```
|
||||
|
||||
Packet sources:
|
||||
|
||||
- tracked marker trails become luminous volumetric strokes
|
||||
- sparse point tracks become soft surfels or wisps
|
||||
- Kiyo RGB projections color nearby strokes/points
|
||||
- Leap near-field depth/IR becomes dense local hand/prop strokes
|
||||
- audio source estimates become optional visible aura/field strokes
|
||||
- speaker/mic calibration paths can render as debug-only transfer wisps
|
||||
|
||||
This is where the Aquarium cache discipline applies. The renderer should get a selected cut of brush packets, not the whole history of every sensor trace. Parent summaries render when children are stale or absent.
|
||||
|
||||
## Contribution Cache
|
||||
|
||||
Each track/claim carries conservative summaries:
|
||||
|
||||
```text
|
||||
stableKey
|
||||
world bounds
|
||||
last observation time
|
||||
confidence
|
||||
velocity bound
|
||||
projected footprint
|
||||
color availability
|
||||
audio relevance
|
||||
estimated render cost
|
||||
fallback brush packet
|
||||
```
|
||||
|
||||
Contribution score should include both visual and audio importance:
|
||||
|
||||
```text
|
||||
visualScore = projectedCoveragePx * confidence * freshness
|
||||
audioScore = listenerRelevance * sourceEnergy * localizationConfidence
|
||||
motionScore = velocity / trackingUncertainty
|
||||
score = max(visualScore, audioScore, motionScore) / estimatedCost
|
||||
```
|
||||
|
||||
That gives the renderer permission to spend detail where the virtual camera sees it, where the listener hears it, or where uncertainty/motion needs more frequent updates.
|
||||
|
||||
## Volumetric Brushstroke Pipeline
|
||||
|
||||
1. Calibrate all sensors into the world frame.
|
||||
2. Ingest timestamped observations into ring buffers.
|
||||
3. Estimate tracks and confidence.
|
||||
4. Convert tracks/surfaces/sources into semantic claims.
|
||||
5. Lower claims into brush packets under budget.
|
||||
6. Render brush packets as anisotropic camera-facing or view-stable volumetric splats.
|
||||
7. Project Kiyo RGB onto brush packets using camera poses.
|
||||
8. Feed virtual camera/listener pose into the ambisonic renderer.
|
||||
|
||||
The first renderer can be points/surfels with fat translucent strokes. A proper Dreams-like look can then add:
|
||||
|
||||
- anisotropic ellipsoid splats
|
||||
- temporal stroke trails
|
||||
- stroke clustering by domain
|
||||
- RGB projection and confidence blending
|
||||
- procedural texture/noise in stroke-local coordinates
|
||||
- LOD summaries for distant or stale tracks
|
||||
|
||||
## Audio Coupling
|
||||
|
||||
The visual camera pose should define or link to the audio listener pose:
|
||||
|
||||
- yaw/pitch/roll rotates the ambisonic scene
|
||||
- translation changes source distances, occlusion candidates, and room response estimates
|
||||
- tracked props can drive synth patch positions
|
||||
- visible speaker/mic locations constrain playback and capture calibration
|
||||
|
||||
Do not make the ambisonic renderer read brush packets. It should read world claims: sources, listener pose, speaker/mic transforms, and confidence. Brush packets are how those claims are seen, not what they are.
|
||||
|
||||
## First Slice
|
||||
|
||||
Build the minimum machine that proves the spine:
|
||||
|
||||
1. Enumerate the two PS3 Eyes, two Kiyos, and LeapUVC mode.
|
||||
2. Calibrate camera intrinsics/extrinsics.
|
||||
3. Track one visible marker in at least two cameras.
|
||||
4. Triangulate a world-space point with confidence.
|
||||
5. Render that point as a volumetric brushstroke with a stable key.
|
||||
6. Move the virtual camera and verify visual parallax.
|
||||
7. Bind that same point to a mono synth source and verify ambisonic position changes with listener pose.
|
||||
|
||||
Only after that should RGB projection, dense splats, and dynamic point-cloud beauty work begin.
|
||||
|
||||
## Cut Line
|
||||
|
||||
Cut any design where:
|
||||
|
||||
- Gaussian/brush packets become the source of truth.
|
||||
- Leap, PS3 Eyes, and Kiyos each maintain separate private coordinate systems.
|
||||
- camera-attached microphones are treated as synchronized because they are physically attached.
|
||||
- the renderer owns calibration.
|
||||
- ambisonics reads rendered geometry instead of semantic world claims.
|
||||
|
||||
Pretty volumetric paint is allowed. Pretty volumetric paint with no ownership model is just smoke with a GPU budget.
|
||||
|
|
@ -0,0 +1,476 @@
|
|||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
||||
<html xmlns="//www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
||||
<head profile="//www.w3.org/2005/10/profile">
|
||||
<title>CL - About CL Eye Platform - Code Laboratories</title>
|
||||
<meta name="Description" content="Code Laboratories is an agile research & development firm which provides consulting solutions for government, commercial & communal sectors.">
|
||||
<meta name="keywords" content="ps3eye, ps3, playstation eye, windows driver, code labs, alexp, krism, code laboratories, codelabs, the code labs, cl-eye, cl eye, cleye, nuigroup, computer vision, webcamera, security">
|
||||
<meta http-equiv="Content-type" content="text/html;charset=UTF-8" />
|
||||
<link rel="shortcut icon" type="image/ico" href="//codelaboratories.com/favicon.png" />
|
||||
<link href="//codelaboratories.com/template/css/layout.css?v1" rel="stylesheet" type="text/css" />
|
||||
<!--[if IE 6]><link rel="stylesheet" type="text/css" href="//codelaboratories.com/template/css/ie6.css" /><![endif]-->
|
||||
<!--[if IE 7]><link rel="stylesheet" type="text/css" href="//codelaboratories.com/template/css/ie7.css" /><![endif]-->
|
||||
<link rel="stylesheet" type="text/css" href="//codelaboratories.com/template/css/panels.css" />
|
||||
<link rel="stylesheet" type="text/css" href="//codelaboratories.com/template/css/letterbox.css" />
|
||||
<link rel="stylesheet" type="text/css" href="//codelaboratories.com/template/css/local_panels.css" />
|
||||
<!-- <link rel="stylesheet" type="text/css" href="//codelaboratories.com/template/css/local_letterbox.css" /> -->
|
||||
<link rel="stylesheet" href="//codelaboratories.com/css-buttons.css" type="text/css" title="default" />
|
||||
|
||||
<!--<script type='text/javascript' src='//codelaboratories.com/template//js/jquery-1.3.2.js'></script> -->
|
||||
<!-- --><script src="//codelaboratories.com/plugins/note/js/jquery-1.9.0.min.js"></script>
|
||||
<script type='text/javascript' src='//codelaboratories.com/template//js/js2.js'></script>
|
||||
|
||||
<!--
|
||||
<link href="//codelaboratories.com/plugins/note/css/MetroBanner.css?v1" rel="stylesheet" type="text/css" media="screen">
|
||||
<script type='text/javascript' src="//codelaboratories.com/plugins/note/js/jquery.cookie.js"></script>
|
||||
<script type='text/javascript' src="//codelaboratories.com/plugins/note/js/MetroBanner.js?v1.2"></script>
|
||||
|
||||
<link href="//codelaboratories.com/js/facebox/facebox.css" media="screen" rel="stylesheet" type="text/css"/>
|
||||
<script src="//codelaboratories.com/js/facebox/facebox.js" type="text/javascript"></script>
|
||||
-->
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</head>
|
||||
<body id="top">
|
||||
|
||||
|
||||
|
||||
<style>
|
||||
.bg {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
position: absolute;
|
||||
top: 0;
|
||||
left: 0;
|
||||
z-index: -1000;
|
||||
background:#151515;
|
||||
}
|
||||
</style>
|
||||
|
||||
<div id="background-img" class="bg"></div>
|
||||
|
||||
<style>
|
||||
#open_bar ul {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
#open_bar li {
|
||||
list-style-type: none;
|
||||
line-height: normal;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
#open_bar ul.menus {
|
||||
padding: 0 35px;
|
||||
}
|
||||
|
||||
#open_bar li.menu {
|
||||
float: left;
|
||||
position: relative;
|
||||
margin-right: 4px;
|
||||
}
|
||||
|
||||
#open_bar li.menu a {
|
||||
position: relative;
|
||||
display: block;
|
||||
padding: 4px 5px;
|
||||
font-size: 10px;
|
||||
color: #999;
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
#open_bar li.menu a:hover {
|
||||
xbackground: #000;color:#FFF
|
||||
}
|
||||
|
||||
#open_bar li.menu a.on {
|
||||
color: #ddd;
|
||||
}
|
||||
|
||||
#open_bar li.menu a.on:hover {
|
||||
color: #ddd;
|
||||
}
|
||||
|
||||
#open_bar li.menu a.account:hover {
|
||||
color: #fff;
|
||||
}
|
||||
#open_bar li.menu a.current_account:hover {
|
||||
color: #999;
|
||||
}
|
||||
|
||||
|
||||
#menux {
|
||||
position:absolute;xmargin-top:10px;z-index:1000;xcursor:pointer; font-size: 9px; xpadding: 4px 6px;
|
||||
color: #999;
|
||||
text-decoration: none;
|
||||
font-family: Verdana, sans-serif;}
|
||||
#menux ul .item{display:none;xmargin:5px;}
|
||||
#menux ul:hover .item{display:block;background:#000;
|
||||
xpadding:0px 15px 5px 5px ;
|
||||
xpadding-top:3px;
|
||||
xpadding-bottom:3px;
|
||||
xpadding-left:12px;
|
||||
xpadding-right:17px;
|
||||
margin:0px;
|
||||
}
|
||||
|
||||
#menux ul:hover .item a{color:#fff;text-decoration:none;}
|
||||
#menux ul:hover .item a:hover{color:#999;}
|
||||
|
||||
#menux ul{xwidth:100px;float:left;margin:0px;xpadding:2px;xbackground:#222;list-style:none;}
|
||||
|
||||
#menuy {
|
||||
position:absolute;xmargin-top:10px;z-index:1000;xcursor:pointer; font-size: 9px; xpadding: 4px 6px;
|
||||
color: #999;
|
||||
text-decoration: none;
|
||||
font-family: Verdana, sans-serif;}
|
||||
|
||||
#menuy ul .item{display:none;xmargin:5px;}
|
||||
|
||||
#menuy ul:hover .item{
|
||||
display:block;background:#000;
|
||||
xpadding:0px 15px 5px 5px ;
|
||||
xpadding-top:3px;
|
||||
xpadding-bottom:3px;
|
||||
xpadding-left:12px;
|
||||
xpadding-right:17px;
|
||||
margin:0px;
|
||||
}
|
||||
|
||||
#menuy ul:hover .item a{color:#fff;text-decoration:none;}
|
||||
#menuy ul:hover .item a:hover{color:#999;}
|
||||
|
||||
#menuy ul{xwidth:100px;float:left;margin:0px;xpadding:2px;xbackground:#222;list-style:none;}
|
||||
</style>
|
||||
|
||||
|
||||
<div id="open_bar" style="position:relative; width : 988px;
|
||||
margin : 0 auto;height:22px; background-color:#000;xdisplay:none">
|
||||
|
||||
|
||||
<ul class="menus" style="xfloat:right; margin-left:-15px;">
|
||||
|
||||
<!--
|
||||
<li class="menu hover_container"><a href="http://codelaboratories.com/home/" style="xcolor:#5CCE61">CL</a> </li>
|
||||
<li class="menu hover_container" style="color:#ccc; padding-top:4px">·</li>
|
||||
<li class="menu hover_container"><a href="http://codelaboratories.com/studio/" style="xcolor:#5CCE61">Studio</a> </li>
|
||||
<li class="menu hover_container" style="color:#ccc; padding-top:4px">·</li>
|
||||
<li class="menu hover_container"><a href="http://codelaboratories.com/live/">Live</a> </li>
|
||||
-->
|
||||
<li class="menu hover_container"><a href="http://codelaboratories.com/home/" style="xcolor:#5CCE61">CL</a> </li>
|
||||
<li class="menu hover_container" style="color:#ccc; padding-top:4px">·</li>
|
||||
<li class="menu hover_container"><a href="http://codelaboratories.com/studio/" style="xcolor:#5CCE61">Studio</a> </li>
|
||||
<li class="menu hover_container" style="color:#ccc; padding-top:4px">·</li>
|
||||
<li class="menu hover_container"><a href="http://duo3d.com/">Duo</a> </li>
|
||||
|
||||
</ul>
|
||||
<!--<li class="menu hover_container"><a href="http://codelaboratories.com/nui/">Kinect</a> </li>
|
||||
|
||||
|
||||
<ul class="menus" style="xfloat:right; margin-left:-20px;">
|
||||
<li class="menu hover_container"><a href="http://clstudiolive.com/">CL Studio Live - 10/10/10</a>
|
||||
</ul>
|
||||
|
||||
|
||||
-->
|
||||
|
||||
|
||||
<ul class="menus" style="float:right">
|
||||
|
||||
<li class="menu hover_container"><a href="http://codelaboratories.com/login/create">New Account</a> </li>
|
||||
<li class="menu hover_container" style="color:#ccc; padding-top:4px"> · </li>
|
||||
<li class="menu hover_container"><a href="http://codelaboratories.com/login/go">Login</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<div id="header" >
|
||||
<a href="#pageContent" class="hidden">Skip to content</a>
|
||||
<div class="column3" style="width:800px">
|
||||
<a href="https://codelaboratories.com/home/"><img src="//codelaboratories.com/template/images/logo.png" alt="Code Laboratories" style="margin-left:34px;margin-top:5px" width=32 height=40 /></a>
|
||||
<ul id="navMain" class="inlineUL" xstyle="display:none" >
|
||||
<li id="About" ><a href="https://codelaboratories.com/about/">Code Laboratories</a></li>
|
||||
<!--<li id="Community" ><a href="https://codelaboratories.com/community/">Community</a></li>-->
|
||||
<li id="Downloads" ><a href="https://codelaboratories.com/downloads/">Downloads</a></li>
|
||||
<li id="Forums" ><a href="https://codelaboratories.com/forums/">Forums</a></li>
|
||||
<li id="Products" class="on"><a href="https://codelaboratories.com/products/">Products</a></li>
|
||||
<li id="Research" ><a href="https://codelaboratories.com/research/">Research</a></li>
|
||||
<!--<li id="Showcase" ><a href="https://codelaboratories.com/showcase/">Showcase</a></li>-->
|
||||
<li id="Support" ><a href="https://codelaboratories.com/support/">Support</a></li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<div class="container">
|
||||
<div class="container">
|
||||
<style>
|
||||
#container{
|
||||
border:0px;
|
||||
width:100%;
|
||||
background:#FFF;
|
||||
}
|
||||
.myFade{opacity:0;}
|
||||
</style>
|
||||
<div class="containerWrap" style="background:#FFF">
|
||||
<div id="test" style="width:100%;height:300px;background:#FFF url('//codelaboratories.com/images/eye-driver.png') center 0px no-repeat; color:#FFF" class="myFade">
|
||||
<a href="https://codelaboratories.com/products/" style="position:absolute; border:0px solid #FF0000; width:250px; height:300px;margin-left:-750px;"></a>
|
||||
</div>
|
||||
</div>
|
||||
<div class="content">
|
||||
|
||||
|
||||
|
||||
|
||||
<ul class="breadcrumbs inlineUL column3" xstyle="margin-bottom:6px; padding-top:3px;">
|
||||
<li><a href="https://codelaboratories.com/home/" style="text-decoration:none">Home</a></li>
|
||||
<li><a href="https://codelaboratories.com/products/">Products</a></li>
|
||||
<li><a href="https://codelaboratories.com/products/eye/">CL Eye Platform</a></li>
|
||||
<li><span class="currentcrumb">Driver</span></li>
|
||||
|
||||
|
||||
</ul>
|
||||
<hr id="pageContent" class="headerHR"/>
|
||||
<div class="letterboxHome" id="letterbox" style="height:5px">
|
||||
</div>
|
||||
<ul class="inlineUL clear">
|
||||
<li style="margin-top:-7px">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li>
|
||||
<div class="column1 panelNoHeightHP" style="height:424px;background:#111 url('//codelaboratories.com/images/green_bg_split.png') center -2px;">
|
||||
<span class="header">CL Eye Platform Driver </span>
|
||||
<hr class="myHR">
|
||||
<ul style="height:170px">
|
||||
The <b style="color:#FFF">CL Eye Platform Driver</b> provides users a signed hardware driver which exposes supported cameras to third party applications such as Adobe Flash, Skype, MSN or Yahoo for video chat or conferencing. Two modes allow basic compatibility mode and a advanced mode for high frame rate development use.
|
||||
|
||||
|
||||
</ul>
|
||||
|
||||
<hr class="myHR">
|
||||
<br>
|
||||
|
||||
<div class="column1 panelNoHeightHP;" style=" background: url('https://codelaboratories.com/uploads/buttons/grey_button2.png') no-repeat; padding:10px; width:190px; xheight:20px; padding-left:1px; text-align:center">
|
||||
<a href="https://codelaboratories.com/products/eye/" style="color:#FFF; xfont-size:12px; xfont-weight:bold">CL Eye Platform Overview</a>
|
||||
</div>
|
||||
|
||||
<div class="column1 panelNoHeightHP;" style=" background: url('https://codelaboratories.com/uploads/buttons/green_button.png') no-repeat; padding:10px; width:190px; xheight:20px; padding-left:0px; text-align:center">
|
||||
<a href="https://codelaboratories.com/products/eye/driver/" style="color:#FFF;xfont-size:12px;font-weight:bold">CL Eye Platform Driver</a>
|
||||
</div>
|
||||
<div class="column1 panelNoHeightHP;" style=" background: url('https://codelaboratories.com/uploads/buttons/blue_button.png') no-repeat; padding:10px; width:190px; xheight:20px; padding-left:1px; text-align:center">
|
||||
<a href="https://codelaboratories.com/products/eye/sdk/" style="color:#FFF; xfont-size:12px; xfont-weight:bold">CL Eye Platform SDK</a>
|
||||
</div>
|
||||
<div class="column1 panelNoHeightHP;" style=" background: url('https://codelaboratories.com/uploads/buttons/red_button.png') no-repeat; padding:10px; width:190px; xheight:20px; padding-left:1px; text-align:center">
|
||||
<a href="https://codelaboratories.com/products/eye/packs/" style="color:#FFF; xfont-size:12px; xfont-weight:bold">CL Eye Platform Packs</a>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</li>
|
||||
<li style="margin-top:-7px">
|
||||
|
||||
|
||||
|
||||
<li>
|
||||
<div href="https://codelaboratories.com/camera/driver/" class="column1 panelNoHeightHP" style="height:424px;background:#111 url('//codelaboratories.com/images/green_bg_long.png') center -2px no-repeat">
|
||||
<span class="header">Features</span>
|
||||
<hr class="myHR">
|
||||
<ul>
|
||||
<li style="display:list-item; list-style:square;color:#ccc; padding-left:2px;margin-left:14px"><strong style="color:#FFF">Driver License</strong> <br> <span style="color:#999">Single camera support for personal non-commercial use.</span><br><br></li>
|
||||
<li style="display:list-item; list-style:square;color:#ccc; padding-left:2px;margin-left:14px"><strong style="color:#FFF">Camera Controls</strong> <br> <span style="color:#999">Fully configurable settings such as resolution, exposure and gain.</span><br><br></li>
|
||||
<li style="display:list-item; list-style:square;color:#ccc; padding-left:2px;margin-left:14px"><strong style="color:#FFF">Chat/Communication Usage</strong> <br> <span style="color:#999">Rich video chat applications (Adobe FlashAIM/Skype/MSN)</span><br><br></li>
|
||||
|
||||
<li style="display:list-item; list-style:square;color:#ccc; padding-left:2px;margin-left:14px"><strong style="color:#FFF">Dual Operation Modes</strong> <br> <span style="color:#999">Offers two modes for basic and advanced usage.</span><br><br></li>
|
||||
|
||||
<li style="display:list-item; list-style:square;color:#ccc; padding-left:2px;margin-left:14px"><strong style="color:#FFF">Testing Application</strong> <br> <span style="color:#999">A platform utility for running baseline audio and video tests.</span><br><br></li>
|
||||
<li style="display:list-item; list-style:square;color:#ccc; padding-left:2px;margin-left:14px"><strong style="color:#FFF">Web Video Streaming</strong> <br> <span style="color:#999">Supports for web based video chat applications.</span><br><br></li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
|
||||
<br>
|
||||
</div >
|
||||
</li>
|
||||
|
||||
|
||||
</li>
|
||||
<li style="margin-top:-7px">
|
||||
|
||||
|
||||
<li>
|
||||
<div href="https://codelaboratories.com/camera/driver/" class="column1 panelNoHeightHP" style="height:424px;background:#111 url('//codelaboratories.com/images/green_bg_long.png') center -2px no-repeat">
|
||||
<span class="header">Hardware Supported<a href="#" style="color:#FFF"></a></span>
|
||||
<hr class="myHR">
|
||||
<ul>
|
||||
|
||||
<li style="display:list-item; list-style:square;color:#ccc; padding-left:2px;margin-left:14px"><a href="https://codelaboratories.com/wiki/Sony Playstation 3 Eye/">Sony Playstation™ 3 Eye</a></li>
|
||||
</ul>
|
||||
<span class="header" style="margin-top:40px;">Supported Applications<a href="#" style="color:#FFF"></a></span>
|
||||
<hr class="myHR">
|
||||
<ul>
|
||||
<li style="display:list-item; list-style:square;color:#999; padding-left:2px;margin-left:14px">AOL Aim</li><br>
|
||||
<li style="display:list-item; list-style:square;color:#999; padding-left:2px;margin-left:14px">Adobe Flash™</li><br>
|
||||
<li style="display:list-item; xlist-style:square;color:#999; padding-left:2px;margin-left:14px"> - www.cameroid.com</li><br>
|
||||
<li style="display:list-item; xlist-style:square;color:#999; padding-left:2px;margin-left:14px"> - www.ustream.com</li><br>
|
||||
<li style="display:list-item; xlist-style:square;color:#999; padding-left:2px;margin-left:14px"> - www.youtube.com</li><br>
|
||||
<li style="display:list-item; list-style:square;color:#999; padding-left:2px;margin-left:14px">MSN Messenger</li><br>
|
||||
<li style="display:list-item; list-style:square;color:#999; padding-left:2px;margin-left:14px">Skype Video</li><br>
|
||||
<li style="display:list-item; list-style:square;color:#999; padding-left:2px;margin-left:14px"><a href="https://codelaboratories.com/eye/apps/" style="color:#FFF">View more...</a></li>
|
||||
|
||||
</ul>
|
||||
<span class="header" style="margin-top:30px;">Software Modes</span>
|
||||
<hr class="myHR">
|
||||
<ul>
|
||||
<li style="display:list-item; list-style:square;color:#ccc; padding-left:2px;margin-left:14px"><strong style="color:#FFF">Basic Mode</strong> <br> <span style="color:#999">Featuring frame rates optimized for compatibility with chat usage.</span><br><br></li>
|
||||
<li style="display:list-item; list-style:square;color:#ccc; padding-left:2px;margin-left:14px"><strong style="color:#FFF">Advanced Mode</strong> <br> <span style="color:#999"> Featuring full range of supported frame rates for research usage. </span><br><br></li>
|
||||
|
||||
|
||||
</ul>
|
||||
</div>
|
||||
</li>
|
||||
|
||||
|
||||
</li>
|
||||
<li style="margin-top:-7px">
|
||||
|
||||
|
||||
<li>
|
||||
<div class="column1 panelNoHeightHP" style="height:424px;background:#111 url('//codelaboratories.com/images/green_bg.png') center -2px;">
|
||||
|
||||
|
||||
<span class="header" xstyle="margin-top:30px;">Technical Specifications</span>
|
||||
<hr class="myHR">
|
||||
<ul style="xmargin-top:30px;height:170px">
|
||||
<li style="display:list-item; list-style:square;xcolor:#ccc; padding-left:2px;margin-left:14px">Microsoft Windows XP, Vista, 7 </li>
|
||||
<li style="display:list-item; list-style:square;xcolor:#ccc; padding-left:2px;margin-left:14px">Support for 32/64 bit Windows</li>
|
||||
<li style="display:list-item; list-style:square;xcolor:#ccc; padding-left:2px;margin-left:14px">15-125 FPS Capture Frame Rate</li>
|
||||
<li style="display:list-item; list-style:square;xcolor:#ccc; padding-left:2px;margin-left:14px">320x240/640x480 Resolution</li>
|
||||
<li style="display:list-item; list-style:square;xcolor:#ccc; padding-left:2px;margin-left:14px">High Quality Uncompressed Image</li>
|
||||
<li style="display:list-item; list-style:square;xcolor:#ccc; padding-left:2px;margin-left:14px">Microsoft DirectShow™ Support</li>
|
||||
<li style="display:list-item; list-style:square;xcolor:#ccc; padding-left:2px;margin-left:14px">Signed USB 2.0 Driver</li><br><br>
|
||||
<li style="display:list-item; list-style:square;xcolor:#ccc; padding-left:2px;margin-left:14px">Windows Installer Application</li><br><br>
|
||||
<li style="display:list-item; list-style:square;xcolor:#ccc; padding-left:2px;margin-left:14px">Basic and Advanced Modes</li><br><br>
|
||||
</ul>
|
||||
|
||||
<hr class="myHR">
|
||||
|
||||
<br>
|
||||
<div class="column1 panelNoHeightHP;" style=" background: url('https://codelaboratories.com/uploads/buttons/green_button.png') no-repeat; padding:10px; width:190px; xheight:20px; padding-left:0px; text-align:center">
|
||||
<a href="https://codelaboratories.com/get/cl-eye-driver/" style="color:#FFF; xfont-size:12px; font-weight:bold">Download CL Eye Driver</a>
|
||||
</div>
|
||||
<div class="column1 panelNoHeightHP;" style=" background: url('https://codelaboratories.com/uploads/buttons/grey_button2.png') no-repeat; padding:10px; width:190px; xheight:20px; padding-left:1px; text-align:center">
|
||||
<a href="https://codelaboratories.com/screencast/cl-eye-driver/" style="color:#FFF; xfont-size:12px; xfont-weight:bold">View Installation Video</a>
|
||||
</div>
|
||||
<div class="column1 panelNoHeightHP;" style=" background: url('https://codelaboratories.com/uploads/buttons/grey_button2.png') no-repeat; padding:10px; width:190px; xheight:20px; padding-left:1px; text-align:center">
|
||||
<a href="https://codelaboratories.com/eye/apps/" style="color:#FFF; xfont-size:12px; xfont-weight:bold">View Supported Applications</a>
|
||||
</div>
|
||||
<div class="column1 panelNoHeightHP;" style=" background: url('https://codelaboratories.com/uploads/buttons/grey_button2.png') no-repeat; padding:10px; width:190px; xheight:20px; padding-left:1px; text-align:center">
|
||||
<a href="https://codelaboratories.com/test/eye/" style="color:#FFF; xfont-size:12px; xfont-weight:bold">View Web Camera Test</a>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</li>
|
||||
|
||||
|
||||
</li>
|
||||
</ul>
|
||||
<hr class="footerHR" />
|
||||
<ul class="breadcrumbs inlineUL column3" style="margin-bottom:5px; padding-top:3px;">
|
||||
<li><a href="https://codelaboratories.com/home/" style="text-decoration:none">Home</a></li>
|
||||
<li><a href="https://codelaboratories.com/products/">Products</a></li>
|
||||
<li><a href="https://codelaboratories.com/products/eye/">CL Eye Platform</a></li>
|
||||
<li><span class="currentcrumb">Driver</span></li>
|
||||
|
||||
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
<div id="footer_wrap" style="background:#151515 url('../../../uploads/mainfootbg.png') center 0 no-repeat;height:125px; width:100%;z-index:0">
|
||||
<div id="footer" style="padding-top:14px;background:#151515;background-image:url('../../../uploads/mainfooterbg.png');z-index:500; height:75px">
|
||||
|
||||
|
||||
<a href="https://codelaboratories.com/home/" class="noBorder"><img src="//codelaboratories.com/template/images/logo.png" alt="CL" class="fLeft" width=21 style="margin-top:17px; margin-left:35px"/> </a>
|
||||
<div class="column3" style="width:800px">
|
||||
<ul class="sitemap inlineUL" xstyle="font-size:9px">
|
||||
<li><a href="https://codelaboratories.com/about/" class="noBorder" >Code Laboratories</a></li>
|
||||
<!-- <li><a href="https://codelaboratories.com/community/" >Community</a></li> -->
|
||||
<li><a href="https://codelaboratories.com/downloads/" >Downloads</a></li>
|
||||
<li><a href="https://codelaboratories.com/forums/" >Forums</a></li>
|
||||
<li><a href="https://codelaboratories.com/products/" style="color:#FFF">Products</a></li>
|
||||
<li><a href="https://codelaboratories.com/research/" >Research</a></li>
|
||||
<!-- <li><a href="https://codelaboratories.com/showcase/" >Showcase</a></li> -->
|
||||
<li><a href="https://codelaboratories.com/support/" >Support</a></li>
|
||||
</ul>
|
||||
<ul class="sitemap inlineUL cBoth" xstyle="xfont-size:9px; float:right">
|
||||
<li><a href="https://codelaboratories.com/contact/" class="noBorder">Contact us</a></li>
|
||||
<li><a href="https://codelaboratories.com/kb/" xclass="noBorder">Knowledge Base</a></li>
|
||||
<li><a href="https://codelaboratories.com/terms/">Terms & Trademarks</a></li>
|
||||
|
||||
<li><a href="https://codelaboratories.com/privacy/">Privacy</a></li> <li><a href="https://codelaboratories.com/press/">Press</a></li>
|
||||
|
||||
<!--
|
||||
<li><a href="https://codelaboratories.com/wiki/">Wiki</a></li>
|
||||
<li><a href="https://codelaboratories.com/wiki/">Test</a></li>
|
||||
-->
|
||||
<li><a href="https://codelaboratories.com/sitemap/">Sitemap</a></li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<script type="text/javascript">
|
||||
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
|
||||
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
|
||||
</script>
|
||||
<script type="text/javascript">
|
||||
try {
|
||||
var pageTracker = _gat._getTracker("UA-12104152-1");
|
||||
pageTracker._trackPageview();
|
||||
} catch(err) {}</script>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<script type="text/javascript">
|
||||
/*
|
||||
$(document).ready(function() {
|
||||
|
||||
$.MetroBanner(
|
||||
{
|
||||
title: "",
|
||||
content: '<a href="http://duo3d.com/"><img src="https://codelaboratories.com/plugins/note/img/KS_B2.png" border="0/"></a>',
|
||||
//position: "top",
|
||||
position: "botton",
|
||||
color: "#141414"
|
||||
//color: "#0f1010"
|
||||
});
|
||||
|
||||
});
|
||||
*/
|
||||
</script>
|
||||
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
||||
File diff suppressed because one or more lines are too long
167
research/visual-spatial-map/mirrors/iem-scenerotator-guide.html
Normal file
167
research/visual-spatial-map/mirrors/iem-scenerotator-guide.html
Normal file
|
|
@ -0,0 +1,167 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en-us">
|
||||
|
||||
<head>
|
||||
<meta name="generator" content="Hugo 0.119.0">
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title> SceneRotator Guide </title>
|
||||
|
||||
|
||||
<meta name="description" content="How to use the SceneRotator plug-in">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta name="author" content="Daniel Rudrich">
|
||||
|
||||
|
||||
<meta property="og:title" content="SceneRotator Guide" />
|
||||
<meta property="og:description" content="How to use the SceneRotator plug-in" />
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://plugins.iem.at/docs/scenerotator/" /><meta property="og:image" content="https://plugins.iem.at/images/IEMPluginSuiteLogo.png"/><meta property="article:section" content="docs" />
|
||||
<meta property="article:published_time" content="2019-01-07T11:26:04+01:00" />
|
||||
<meta property="article:modified_time" content="2019-01-07T11:26:04+01:00" />
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<meta property="og:image" content="images/IEMPluginSuiteLogo.png">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="canonical" href="https://plugins.iem.at/docs/scenerotator/">
|
||||
|
||||
<link rel="shortcut icon" type="image/png" href="images/favicon_96x96.png">
|
||||
|
||||
|
||||
<link href="https://plugins.iem.at/css/font.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.min.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.legenda.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/highlight.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/master.css" rel="stylesheet" type="text/css">
|
||||
<link href="https://plugins.iem.at/css/kube.demo.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<link href="https://plugins.iem.at/css/custom.css" rel="stylesheet" type="text/css">
|
||||
|
||||
<script src="https://plugins.iem.at/js/jquery-3.5.1.min.js" type="text/javascript">
|
||||
</script>
|
||||
|
||||
<script type="text/javascript" src="https://plugins.iem.at/js/tocbot.min.js"></script>
|
||||
</head>
|
||||
|
||||
|
||||
<body class="page-kube">
|
||||
<header> <div class="show-sm">
|
||||
<div id="nav-toggle-box">
|
||||
<div id="nav-toggle-brand">
|
||||
<a href="https://plugins.iem.at/">IEM Plug-in Suite</a>
|
||||
</div><a data-component="toggleme" data-target="#top" href="#" id="nav-toggle"><i class="kube-menu"></i></a>
|
||||
</div>
|
||||
</div>
|
||||
<div class="hide-sm" id="top">
|
||||
<div id="top-brand">
|
||||
<a href="https://plugins.iem.at/" title="home">IEM Plug-in Suite</a>
|
||||
</div>
|
||||
|
||||
<nav id="top-nav-main">
|
||||
<ul>
|
||||
|
||||
|
||||
<li><a href="https://plugins.iem.at/download/" >Download</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/docs/" >Documentation</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/faq/" >FAQ</a></li>
|
||||
|
||||
<li><a href="https://plugins.iem.at/contact/" >Contact</a></li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
<div id="nav-toggle" style="text-align: center;">
|
||||
<a href="https://atmoky.com"><img src="https://plugins.iem.at/images/atmokyfriends.png" width="200"></a>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
<main>
|
||||
<div id="main">
|
||||
<div id="hero">
|
||||
<h1> SceneRotator Guide </h1>
|
||||
<p class="hero-lead">
|
||||
How to use the SceneRotator plug-in.
|
||||
</p>
|
||||
|
||||
</div>
|
||||
<div id="kube-component" class="content">
|
||||
|
||||
<nav id="contents">
|
||||
<ol class="js-toc">
|
||||
</ol>
|
||||
</nav>
|
||||
<script type="text/javascript">
|
||||
document.addEventListener("DOMContentLoaded",
|
||||
function(){
|
||||
tocbot.init({
|
||||
|
||||
tocSelector: '.js-toc',
|
||||
|
||||
contentSelector: '.content',
|
||||
|
||||
headingSelector: 'h3'
|
||||
})
|
||||
}
|
||||
);
|
||||
</script>
|
||||
|
||||
|
||||
|
||||
<h3 id="introduction">Introduction</h3>
|
||||
<p>The <strong>Scene</strong>Rotator plug-in can be used to rotate an Ambisonic scene around the three axes <em>Yaw</em>, <em>Pitch</em>, and <em>Roll</em>. Alternatively, you can use <em>Quaternion</em> data to controll the rotations. In case you use the Euler angles (<em>Yaw</em>, <em>Pitch</em>, and <em>Roll</em>) you need to define the order of rotations.</p>
|
||||
<h3 id="midi-connection">MIDI Connection</h3>
|
||||
<p>With the MIDI Connection feature, you can directly select a MIDI device, which sends rotation data. With the scheme selector, you select the type of MIDI device, so the plug-in correctly interprets the rotation data.</p>
|
||||
<p>For now, our <a href="https://git.iem.at/DIY/MrHeadTracker">MrHeadtracker</a> device as well as the <a href="https://supperware.co.uk/headtracker-overview">Supperware Head Tracker 1</a> is supported. If you’d like to have other headtracking devices supported, please let us know! We are happy to add them to the list.</p>
|
||||
<h3 id="troubleshooting">Troubleshooting</h3>
|
||||
<p>We are aware of an issue on Windows: In case the MIDI device has already been opened in a process (e.g. DAW), the plug-in won’t be able to use it. E.g. when Reaper has enabled the device in the Midi Device options, it can’t be loaded again. This is different on macOS or linux, it works just fine. A workaround is to disable the device in the Reaper options. Sometimes also a restart of Reaper helps.</p>
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
<footer>
|
||||
<footer id="footer">
|
||||
<nav>
|
||||
<ul>
|
||||
<li><span>Plug-in Suite by IEM - Institute of Electronic Music and Acoustics</span></li>
|
||||
<li>
|
||||
<a href="https://iem.at/">IEM.at</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="https://plugins.iem.at/docs/">Docs</a>
|
||||
</li>
|
||||
</ul>
|
||||
</nav>
|
||||
<p>Website created with <a href="https://gohugo.io">Hugo</a> using the <a href="https://github.com/jeblister/kube">kube</a> theme.</p>
|
||||
</footer>
|
||||
|
||||
</footer>
|
||||
|
||||
|
||||
<script src="https://plugins.iem.at/js/kube.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/kube.legenda.js" type="text/javascript">
|
||||
</script>
|
||||
<script src="https://plugins.iem.at/js/master.js" type="text/javascript">
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,15 @@
|
|||
PS3EYE Driver
|
||||
========
|
||||
|
||||
Some driver like routines to connect and work with Sony PS3 EYE Camera.
|
||||
|
||||
The driver is using [libusb](https://github.com/libusbx/libusbx) to handle USB communications.
|
||||
|
||||
Most of the code was ported from Linux kernel source.
|
||||
|
||||
You can find example code for [Cinder](https://github.com/cinder/Cinder) and [openFrameworks](https://github.com/openframeworks/openFrameworks) in corresponding folders.
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load diff
Binary file not shown.
Binary file not shown.
1737
research/visual-spatial-map/mirrors/leapuvc-github.html
Normal file
1737
research/visual-spatial-map/mirrors/leapuvc-github.html
Normal file
File diff suppressed because one or more lines are too long
BIN
research/visual-spatial-map/mirrors/leapuvc-manual.pdf
Normal file
BIN
research/visual-spatial-map/mirrors/leapuvc-manual.pdf
Normal file
Binary file not shown.
15
research/visual-spatial-map/mirrors/leapuvc-readme.md
Normal file
15
research/visual-spatial-map/mirrors/leapuvc-readme.md
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
# leapuvc
|
||||
|
||||
LeapUVC gives you access to the Leap Motion Controller image data through the industry standard UVC (Universal Video Class) interface. This gives you low level controls such as LED brightness, gamma, exposure, gain, resolution, and more.
|
||||
|
||||
The LeapUVC release features examples in C, Python, and Matlab, as well as OpenCV bindings that show how to stream from multiple devices, track ArUco markers, change camera settings, grab lens distortion parameters, and compute stereo depth maps from the images. Use the Leap Motion Controller to track physical objects, capture high-speed infrared footage, or see the world in new ways.
|
||||
|
||||
The bundled firmware updating binaries, which are required to unlock LeapUVC access, must be executed on a Windows machine. The included Python Examples have been tested on Windows and Linux; Mac is not currently supported. Since this is experimental software, use it at your own risk.
|
||||
|
||||
With this experimental release we hope to open up entirely new use cases for the Leap Motion Controller in education, robotics, art, academic research, and more.
|
||||
|
||||
## License
|
||||
|
||||
These examples are licensed for use under the terms of the [Leap Motion Developer SDK agreement][sdkagreement].
|
||||
|
||||
[sdkagreement]: https://developer.leapmotion.com/sdk_agreement "Leap Motion Developer SDK Agreement"
|
||||
|
|
@ -0,0 +1,219 @@
|
|||
|
||||
|
||||
<!DOCTYPE html>
|
||||
<html class="writer-html5" lang="en" data-content_root="./">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="readthedocs-addons-api-version" content="1"><meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Multi-Camera Calibration — Multi-camera Calibration documentation</title>
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=b86133f3" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=e59714d7" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/mystnb.4510f1fc1dee50b3e5859aac5469c37c29e427902b24a333a5f9fcb2f0b3ac41.css" />
|
||||
|
||||
|
||||
<script src="_static/jquery.js?v=5d32c60e"></script>
|
||||
<script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
|
||||
<script src="_static/documentation_options.js?v=5929fcd5"></script>
|
||||
<script src="_static/doctools.js?v=9a2dae69"></script>
|
||||
<script src="_static/sphinx_highlight.js?v=dc90522c"></script>
|
||||
<script src="_static/js/theme.js"></script>
|
||||
<script src="_static/js/versions.js"></script>
|
||||
<link rel="index" title="Index" href="genindex.html" />
|
||||
<link rel="search" title="Search" href="search.html" />
|
||||
<link rel="next" title="Installation" href="setup.html" />
|
||||
<script async type="text/javascript" src="/_/static/javascript/readthedocs-addons.js"></script><meta name="readthedocs-project-slug" content="multi-camera-calibration" /><meta name="readthedocs-version-slug" content="latest" /><meta name="readthedocs-resolver-filename" content="/" /><meta name="readthedocs-http-status" content="200" /></head>
|
||||
|
||||
<body class="wy-body-for-nav">
|
||||
<div class="wy-grid-for-nav">
|
||||
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
|
||||
<div class="wy-side-scroll">
|
||||
<div class="wy-side-nav-search" >
|
||||
|
||||
|
||||
|
||||
<a href="#" class="icon icon-home">
|
||||
Multi-camera Calibration
|
||||
</a>
|
||||
<div class="switch-menus">
|
||||
<div class="version-switch"></div>
|
||||
<div class="language-switch"></div>
|
||||
</div>
|
||||
<div role="search">
|
||||
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
|
||||
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
|
||||
<input type="hidden" name="check_keywords" value="yes" />
|
||||
<input type="hidden" name="area" value="default" />
|
||||
</form>
|
||||
</div>
|
||||
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
|
||||
<p class="caption" role="heading"><span class="caption-text">Setup</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="setup.html">Installation</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="setup.html#camera-setup">Camera setup</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="setup.html#building-the-calibration-object">Building the calibration object</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="setup.html#acquiring-calibration-videos">Acquiring calibration videos</a></li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">Tutorial</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="calibration_tutorial.html">Detect chessboard</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="calibration_tutorial.html#calibrate">Calibrate</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="calibration_tutorial.html#save-results">Save results</a></li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">Under the hood</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="math.html">Background</a></li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">Troubleshooting</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="troubleshooting.html">High reprojection error</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="troubleshooting.html#detection-errors">Detection errors</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="troubleshooting.html#poor-detection-of-calibration-object">Poor detection of calibration object</a></li>
|
||||
</ul>
|
||||
<p class="caption" role="heading"><span class="caption-text">API Documentation</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="geometry.html">General geometry utilities</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="calibration.html">Core calibration functions</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="detection.html">Calibration object detection</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="io.html">Input / Output</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="viz.html">Visualization</a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
|
||||
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
|
||||
<a href="#">Multi-camera Calibration</a>
|
||||
</nav>
|
||||
|
||||
<div class="wy-nav-content">
|
||||
<div class="rst-content">
|
||||
<div role="navigation" aria-label="Page navigation">
|
||||
<ul class="wy-breadcrumbs">
|
||||
<li><a href="#" class="icon icon-home" aria-label="Home"></a></li>
|
||||
<li class="breadcrumb-item active">Multi-Camera Calibration</li>
|
||||
<li class="wy-breadcrumbs-aside">
|
||||
<a href="https://github.com/calebweinreb/multicam-calibration/blob/main/docs/source/index.rst" class="fa fa-github"> Edit on GitHub</a>
|
||||
</li>
|
||||
</ul>
|
||||
<hr/>
|
||||
</div>
|
||||
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
|
||||
<div itemprop="articleBody">
|
||||
|
||||
<section id="multi-camera-calibration">
|
||||
<h1>Multi-Camera Calibration<a class="headerlink" href="#multi-camera-calibration" title="Link to this heading"></a></h1>
|
||||
<p>This package provides a pipeline for multi-camera calibration, as well as general geometry utilities like keypoint triangulation and reprojection. There are several other packages include their own calibration pipelines, including <a class="reference external" href="https://github.com/lambdaloop/anipose">anipose</a>, <a class="reference external" href="https://github.com/lmb-freiburg/FreiCalib">FreiPose</a> and <a class="reference external" href="https://jarvis-mocap.github.io/jarvis-docs/">JARVIS</a>. We created a new library for multicamera calibration because we wanted:</p>
|
||||
<ul class="simple">
|
||||
<li><p>a standalone calibration solution that can be mixed and matched with other tools</p></li>
|
||||
<li><p>a modular, extendable and well-documented code base</p></li>
|
||||
<li><p>a pedagogical pipeline that exposes each step of the process</p></li>
|
||||
<li><p>the ability to register camera that point in opposite directions</p></li>
|
||||
</ul>
|
||||
<section id="what-is-calibration">
|
||||
<h2>What is calibration?<a class="headerlink" href="#what-is-calibration" title="Link to this heading"></a></h2>
|
||||
<p>Calibration is the process of measuring the <em>intrinsic</em> and <em>extrinsic</em> parameters of two or more cameras that are being used to track things in 3D. <strong>Intrinsics</strong> are private to each camera and include parameters like focal length and radial distortion that determine the relationship between pixels in the camera image and lines in 3D space. <strong>Extrinsics</strong> capture the relative translations and rotations between cameras (see diagram). For more details, see <a class="reference external" href="math.html">Under the hood</a>.</p>
|
||||
<img alt="_images/setup_intrinsics_extrinsics.jpg" class="align-center" src="_images/setup_intrinsics_extrinsics.jpg" />
|
||||
<div class="line-block">
|
||||
<div class="line"><br /></div>
|
||||
</div>
|
||||
</section>
|
||||
<section id="who-should-use-this-tool">
|
||||
<h2>Who should use this tool?<a class="headerlink" href="#who-should-use-this-tool" title="Link to this heading"></a></h2>
|
||||
<p>Anyone using multiple cameras (or a single camera with mirrors) to track stuff in 3D can use this package. The main use case we had in mind is 3D pose tracking for animal behavior research. We include import/export functions for a variety of 3D tracking packages, including JARVIS and GIMBAL. In fact, our pipeline can function as a drop-in replacement for the calibration steps in JARVIS or anipose. If would like to integrate <code class="docutils literal notranslate"><span class="pre">multicam_calibration</span></code> into your pipeline, feel free to open a <a class="reference external" href="https://github.com/dattalab-6-cam/multicam-calibration/issues">github issue</a>!</p>
|
||||
<div class="toctree-wrapper compound">
|
||||
<p class="caption" role="heading"><span class="caption-text">Setup</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="setup.html">Installation</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="setup.html#camera-setup">Camera setup</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="setup.html#building-the-calibration-object">Building the calibration object</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="setup.html#acquiring-calibration-videos">Acquiring calibration videos</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="toctree-wrapper compound">
|
||||
<p class="caption" role="heading"><span class="caption-text">Tutorial</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="calibration_tutorial.html">Detect chessboard</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="calibration_tutorial.html#calibrate">Calibrate</a><ul>
|
||||
<li class="toctree-l2"><a class="reference internal" href="calibration_tutorial.html#initialization">Initialization</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="calibration_tutorial.html#bundle-adjustment">Bundle adjustment</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="calibration_tutorial.html#save-results">Save results</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="toctree-wrapper compound">
|
||||
<p class="caption" role="heading"><span class="caption-text">Under the hood</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="math.html">Background</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="toctree-wrapper compound">
|
||||
<p class="caption" role="heading"><span class="caption-text">Troubleshooting</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="troubleshooting.html">High reprojection error</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="troubleshooting.html#detection-errors">Detection errors</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="troubleshooting.html#poor-detection-of-calibration-object">Poor detection of calibration object</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="toctree-wrapper compound">
|
||||
<p class="caption" role="heading"><span class="caption-text">API Documentation</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="geometry.html">General geometry utilities</a><ul>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.rodrigues"><code class="docutils literal notranslate"><span class="pre">rodrigues()</span></code></a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.rodrigues_inv"><code class="docutils literal notranslate"><span class="pre">rodrigues_inv()</span></code></a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.rigid_transform_from_correspondences"><code class="docutils literal notranslate"><span class="pre">rigid_transform_from_correspondences()</span></code></a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.apply_rigid_transform"><code class="docutils literal notranslate"><span class="pre">apply_rigid_transform()</span></code></a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.get_transformation_matrix"><code class="docutils literal notranslate"><span class="pre">get_transformation_matrix()</span></code></a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.get_transformation_vector"><code class="docutils literal notranslate"><span class="pre">get_transformation_vector()</span></code></a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.get_projection_matrix"><code class="docutils literal notranslate"><span class="pre">get_projection_matrix()</span></code></a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.euclidean_to_homogenous"><code class="docutils literal notranslate"><span class="pre">euclidean_to_homogenous()</span></code></a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.homogeneous_to_euclidean"><code class="docutils literal notranslate"><span class="pre">homogeneous_to_euclidean()</span></code></a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.project_points"><code class="docutils literal notranslate"><span class="pre">project_points()</span></code></a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.undistort_points"><code class="docutils literal notranslate"><span class="pre">undistort_points()</span></code></a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="geometry.html#multicam_calibration.geometry.triangulate"><code class="docutils literal notranslate"><span class="pre">triangulate()</span></code></a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="calibration.html">Core calibration functions</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="detection.html">Calibration object detection</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="io.html">Input / Output</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="viz.html">Visualization</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
|
||||
<a href="setup.html" class="btn btn-neutral float-right" title="Installation" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
|
||||
</div>
|
||||
|
||||
<hr/>
|
||||
|
||||
<div role="contentinfo">
|
||||
<p>© Copyright .</p>
|
||||
</div>
|
||||
|
||||
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
|
||||
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
|
||||
provided by <a href="https://readthedocs.org">Read the Docs</a>.
|
||||
|
||||
|
||||
</footer>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
<script>
|
||||
jQuery(function () {
|
||||
SphinxRtdTheme.Navigation.enable(true);
|
||||
});
|
||||
</script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -0,0 +1,150 @@
|
|||
<!-- HTML header for doxygen 1.8.6-->
|
||||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||||
<head>
|
||||
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
|
||||
<meta name="generator" content="Doxygen 1.8.13"/>
|
||||
<title>OpenCV: Multi-camera Calibration</title>
|
||||
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
|
||||
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
|
||||
<script type="text/javascript" src="../../jquery.js"></script>
|
||||
<script type="text/javascript" src="../../dynsections.js"></script>
|
||||
<script type="text/javascript" src="../../tutorial-utils.js"></script>
|
||||
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
|
||||
<script type="text/javascript" src="../../search/searchdata.js"></script>
|
||||
<script type="text/javascript" src="../../search/search.js"></script>
|
||||
<script type="text/x-mathjax-config">
|
||||
MathJax.Hub.Config({
|
||||
extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
|
||||
jax: ["input/TeX","output/HTML-CSS"],
|
||||
});
|
||||
//<![CDATA[
|
||||
MathJax.Hub.Config(
|
||||
{
|
||||
TeX: {
|
||||
Macros: {
|
||||
matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
|
||||
fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
|
||||
forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
|
||||
forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
|
||||
vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
|
||||
vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
|
||||
cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
|
||||
distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
|
||||
distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
|
||||
hdotsfor: ["\\dots", 1],
|
||||
mathbbm: ["\\mathbb{#1}", 1],
|
||||
bordermatrix: ["\\matrix{#1}", 1]
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
//]]>
|
||||
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
|
||||
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
|
||||
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
|
||||
</head>
|
||||
<body><a href="https://docs.opencv.org/cdn-cgi/content?id=zYxjvxYdVNKlxLrMBTJL3G2dwGEDBkx8ORovAM7..M4-1779105529.0450983-1.0.1.1-xgi3WyNX2pz9kL_YHlOrPwN9m7IJLZYFahLuKJID6kI" aria-hidden="true" rel="nofollow noopener" style="display: none !important; visibility: hidden !important"></a>
|
||||
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
|
||||
<div id="titlearea">
|
||||
<script>
|
||||
(function() {
|
||||
var cx = '002541620211387084530:kaexgxg7oxu';
|
||||
var gcse = document.createElement('script');
|
||||
gcse.type = 'text/javascript';
|
||||
gcse.async = true;
|
||||
gcse.src = 'https://cse.google.com/cse.js?cx=' + cx;
|
||||
var s = document.getElementsByTagName('script')[0];
|
||||
s.parentNode.insertBefore(gcse, s);
|
||||
})();
|
||||
</script>
|
||||
<gcse:search></gcse:search>
|
||||
<script type="text/javascript" src="/version.js"></script>
|
||||
|
||||
<table cellspacing="0" cellpadding="0">
|
||||
<tbody>
|
||||
<tr style="height: 56px;">
|
||||
<td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
|
||||
<td style="padding-left: 0.5em;">
|
||||
<div id="projectname">OpenCV
|
||||
 <span id="projectnumber">3.4.20-dev</span>
|
||||
</div>
|
||||
<div id="projectbrief">Open Source Computer Vision</div>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<!-- end header part -->
|
||||
<!-- Generated by Doxygen 1.8.13 -->
|
||||
<script type="text/javascript">
|
||||
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
|
||||
</script>
|
||||
<script type="text/javascript" src="../../menudata.js"></script>
|
||||
<script type="text/javascript" src="../../menu.js"></script>
|
||||
<script type="text/javascript">
|
||||
$(function() {
|
||||
initMenu('../../',true,false,'search.php','Search');
|
||||
$(document).ready(function() { init_search(); });
|
||||
});
|
||||
</script>
|
||||
<div id="main-nav"></div>
|
||||
<!-- window showing the filter options -->
|
||||
<div id="MSearchSelectWindow"
|
||||
onmouseover="return searchBox.OnSearchSelectShow()"
|
||||
onmouseout="return searchBox.OnSearchSelectHide()"
|
||||
onkeydown="return searchBox.OnSearchSelectKey(event)">
|
||||
</div>
|
||||
|
||||
<!-- iframe showing the search results (closed by default) -->
|
||||
<div id="MSearchResultsWindow">
|
||||
<iframe src="javascript:void(0)" frameborder="0"
|
||||
name="MSearchResults" id="MSearchResults">
|
||||
</iframe>
|
||||
</div>
|
||||
|
||||
<div id="nav-path" class="navpath">
|
||||
<ul>
|
||||
<li class="navelem"><a class="el" href="../../d3/d81/tutorial_contrib_root.html">Tutorials for contrib modules</a></li> </ul>
|
||||
</div>
|
||||
</div><!-- top -->
|
||||
<div class="header">
|
||||
<div class="headertitle">
|
||||
<div class="title">Multi-camera Calibration </div> </div>
|
||||
</div><!--header-->
|
||||
<div class="contents">
|
||||
<div class="textblock"><p>This tutorial will show how to use the multiple camera calibration toolbox. This toolbox is based on the usage of "random" pattern calibration object, so the tutorial is mainly two parts: an introduction to "random" pattern and multiple camera calibration.</p>
|
||||
<h2>Random Pattern Calibration Object </h2>
|
||||
<p>The random pattern is an image that is randomly generated. It is "random" so that it has many feature points. After generating it, one print it out and use it as a calibration object. The following two images are random pattern and a photo taken for it.</p>
|
||||
<div class="image">
|
||||
<img src="../../random_pattern.jpg" alt="random_pattern.jpg"/>
|
||||
<div class="caption">
|
||||
image</div></div>
|
||||
<div class="image">
|
||||
<img src="../../pattern_img.jpg" alt="pattern_img.jpg"/>
|
||||
<div class="caption">
|
||||
image</div></div>
|
||||
<p> To generate a random pattern, use the class <code><a class="el" href="../../d7/d43/classcv_1_1randpattern_1_1RandomPatternGenerator.html">cv::randpattern::RandomPatternGenerator</a></code> in <code>ccalib</code> module. Run it as </p><div class="fragment"><div class="line">cv::randpattern::RandomPatternGenerator generator(width, height);</div><div class="line">generator.generatePattern();</div><div class="line">pattern = generator.getPattern();</div></div><!-- fragment --><p> Here <code>width</code> and <code>height</code> are width and height of pattern image. After getting the pattern, print it out and take some photos of it.</p>
|
||||
<p>Now we can use these images to calibrate camera. First, <code>objectPoints</code> and <code>imagePoints</code> need to be detected. Use class <code><a class="el" href="../../df/d8d/classcv_1_1randpattern_1_1RandomPatternCornerFinder.html" title="Class for finding features points and corresponding 3D in world coordinate of a "random" pattern...">cv::randpattern::RandomPatternCornerFinder</a></code> to detect them. A sample code can be </p><div class="fragment"><div class="line">cv::randpattern::RandomPatternCornerFinder finder(patternWidth, patternHeight, nMiniMatches);</div><div class="line">finder.loadPattern(pattern);</div><div class="line">finder.computeObjectImagePoints(vecImg);</div><div class="line">vector<Mat> objectPoints = finder.getObjectPoints();</div><div class="line">vector<Mat> imagePoints = finder.getImagePoints();</div></div><!-- fragment --><p> Here variable <code>patternWidth</code> and <code>patternHeight</code> are physical pattern width and height with some user defined unit. <code>vecImg</code> is a vector of images that stores calibration images.</p>
|
||||
<p>Second, use calibration functions like <code><a class="el" href="../../d9/d0c/group__calib3d.html#ga3207604e4b1a1758aa66acb6ed5aa65d" title="Finds the camera intrinsic and extrinsic parameters from several views of a calibration pattern...">cv::calibrateCamera</a></code> or <code><a class="el" href="../../d3/ddc/group__ccalib.html#gaf285e757a4091bfbc2bb742ada3ccba7" title="Perform omnidirectional camera calibration, the default depth of outputs is CV_64F. ">cv::omnidir::calibrate</a></code> to calibrate camera.</p>
|
||||
<h2>Multiple Cameras Calibration </h2>
|
||||
<p>Now we move to multiple camera calibration, so far this toolbox must use random pattern object.</p>
|
||||
<p>To calibrate multiple cameras, we first need to take some photos of random pattern. Of cause, to calibrate the extrinsic parameters, one pattern need to be viewed by multiple cameras (at least two) at the same time. Another thing is that to help the program know which camera and which pattern the photo is taken, the image file should be named as "cameraIdx-timestamp.*". Photos with same timestamp means that they are the same object taken by several cameras. In addition, cameraIdx should start from 0. Some examples of files names are "0-129.png", "0-187.png", "1-187", "2-129".</p>
|
||||
<p>Then, we can run multiple cameras calibration as </p><div class="fragment"><div class="line">cv::multicalib::MultiCameraCalibration multiCalib(cameraType, nCamera, inputFilename,patternWidth, patternHeight, showFeatureExtraction, nMiniMatches);</div><div class="line">multiCalib.run();</div><div class="line">multiCalib.writeParameters(outputFilename);</div></div><!-- fragment --><p> Here <code>cameraType</code> indicates the camera type, <code>multicalib::MultiCameraCalibration::PINHOLE</code> and <code>multicalib::MultiCameraCalibration::OMNIDIRECTIONAL</code> are supported. For omnidirectional camera, you can refer to <code><a class="el" href="../../db/dd2/namespacecv_1_1omnidir.html">cv::omnidir</a></code> module for detail. <code>nCamera</code> is the number of camers. <code>inputFilename</code> is the name of a file generated by <code>imagelist_creator</code> from <code>opencv/sample</code>. It stores names of random pattern and calibration images, the first file name is the name of random pattern. <code>patternWidth</code> and <code>patternHeight</code> are physical width and height of pattern. <code>showFeatureExtraction</code> is a flags to indicate whether show feature extraction process. <code>nMiniMatches</code> is a minimal points that should be detected in each frame, otherwise this frame will be abandoned. <code>outputFilename</code> is a xml file name to store parameters. </p>
|
||||
</div></div><!-- contents -->
|
||||
<!-- HTML footer for doxygen 1.8.6-->
|
||||
<!-- start footer part -->
|
||||
<hr class="footer"/><address class="footer"><small>
|
||||
Generated on Tue Jun 17 2025 23:15:47 for OpenCV by  <a href="http://www.doxygen.org/index.html">
|
||||
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
|
||||
</a> 1.8.13
|
||||
</small></address>
|
||||
<script type="text/javascript">
|
||||
//<![CDATA[
|
||||
addTutorialsButtons();
|
||||
//]]>
|
||||
</script>
|
||||
<script defer src="https://static.cloudflareinsights.com/beacon.min.js/v833ccba57c9e4d2798f2e76cebdd09a11778172276447" integrity="sha512-57MDmcccJXYtNnH+ZiBwzC4jb2rvgVCEokYN+L/nLlmO8rfYT/gIpW2A569iJ/3b+0UEasghjuZH/ma3wIs/EQ==" data-cf-beacon='{"version":"2024.11.0","token":"55698296dd8c4381b9a17117972ffe0c","server_timing":{"name":{"cfCacheStatus":true,"cfEdge":true,"cfExtPri":true,"cfL4":true,"cfOrigin":true,"cfSpeedBrain":true},"location_startswith":null}}' crossorigin="anonymous"></script>
|
||||
</body>
|
||||
</html>
|
||||
File diff suppressed because one or more lines are too long
BIN
research/visual-spatial-map/mirrors/peng-2024-rtg-slam.pdf
Normal file
BIN
research/visual-spatial-map/mirrors/peng-2024-rtg-slam.pdf
Normal file
Binary file not shown.
378
research/visual-spatial-map/mirrors/ps2-eyetoy-devwiki.html
Normal file
378
research/visual-spatial-map/mirrors/ps2-eyetoy-devwiki.html
Normal file
|
|
@ -0,0 +1,378 @@
|
|||
|
||||
<!DOCTYPE html>
|
||||
<html class="client-nojs" lang="en" dir="ltr">
|
||||
<head>
|
||||
<meta charset="UTF-8"/>
|
||||
<title>EyeToy - PS2 Developer wiki</title>
|
||||
<script>document.documentElement.className="client-js";RLCONF={"wgBreakFrames":false,"wgSeparatorTransformTable":["",""],"wgDigitTransformTable":["",""],"wgDefaultDateFormat":"dmy","wgMonthNames":["","January","February","March","April","May","June","July","August","September","October","November","December"],"wgRequestId":"0b283846cccb0aa5dce7fc8c","wgCSPNonce":false,"wgCanonicalNamespace":"","wgCanonicalSpecialPageName":false,"wgNamespaceNumber":0,"wgPageName":"EyeToy","wgTitle":"EyeToy","wgCurRevisionId":2605,"wgRevisionId":2605,"wgArticleId":648,"wgIsArticle":true,"wgIsRedirect":false,"wgAction":"view","wgUserName":null,"wgUserGroups":["*"],"wgCategories":[],"wgPageContentLanguage":"en","wgPageContentModel":"wikitext","wgRelevantPageName":"EyeToy","wgRelevantArticleId":648,"wgIsProbablyEditable":true,"wgRelevantPageIsProbablyEditable":true,"wgRestrictionEdit":[],"wgRestrictionMove":[],"wgVisualEditor":{"pageLanguageCode":"en","pageLanguageDir":"ltr","pageVariantFallbacks":"en"},
|
||||
"simpleBatchUploadMaxFilesPerBatch":{"*":1000},"wgVector2022PreviewPages":[],"wgEditSubmitButtonLabelPublish":false};RLSTATE={"site.styles":"ready","user.styles":"ready","user":"ready","user.options":"loading","skins.vector.styles.legacy":"ready","ext.visualEditor.desktopArticleTarget.noscript":"ready","ext.CookieWarning.styles":"ready","oojs-ui-core.styles":"ready","oojs-ui.styles.indicators":"ready","mediawiki.widgets.styles":"ready","oojs-ui-core.icons":"ready","ext.DarkMode.styles":"ready"};RLPAGEMODULES=["site","mediawiki.page.ready","mediawiki.toc","skins.vector.legacy.js","ext.visualEditor.desktopArticleTarget.init","ext.visualEditor.targetLoader","ext.CookieWarning","ext.DarkMode","ext.moderation.notify","ext.moderation.notify.desktop","ext.moderation.ve","ext.moderation.ajaxhook"];</script>
|
||||
<script>(RLQ=window.RLQ||[]).push(function(){mw.loader.implement("user.options@12s5i",function($,jQuery,require,module){mw.user.tokens.set({"patrolToken":"+\\","watchToken":"+\\","csrfToken":"+\\"});});});</script>
|
||||
<link rel="stylesheet" href="/ps2/load.php?lang=en&modules=ext.CookieWarning.styles%7Cext.DarkMode.styles%7Cext.visualEditor.desktopArticleTarget.noscript%7Cmediawiki.widgets.styles%7Coojs-ui-core.icons%2Cstyles%7Coojs-ui.styles.indicators%7Cskins.vector.styles.legacy&only=styles&skin=vector"/>
|
||||
<script async="" src="/ps2/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector"></script>
|
||||
<style>:root { --sdm-accent: gold; }</style>
|
||||
<meta name="ResourceLoaderDynamicStyles" content=""/>
|
||||
<link rel="stylesheet" href="/ps2/load.php?lang=en&modules=site.styles&only=styles&skin=vector"/>
|
||||
<meta name="generator" content="MediaWiki 1.39.6"/>
|
||||
<meta name="format-detection" content="telephone=no"/>
|
||||
<meta name="description" content="Playstation Development Wiki - PS5, PS4, PS3, PS2, PS1, PSP, Vita Information"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=0.5"/>
|
||||
<meta name="viewport" content="width=1000"/>
|
||||
<link rel="alternate" type="application/x-wiki" title="Edit" href="/ps2/edit/EyeToy"/>
|
||||
<link rel="icon" href="/favicon.ico"/>
|
||||
<link rel="search" type="application/opensearchdescription+xml" href="/ps2/opensearch_desc.php" title="PS2 Developer wiki (en)"/>
|
||||
<link rel="EditURI" type="application/rsd+xml" href="https://www.psdevwiki.com/ps2/api.php?action=rsd"/>
|
||||
<link rel="license" href="https://www.gnu.org/licenses/old-licenses/fdl-1.2.txt" />
|
||||
<link rel="alternate" type="application/atom+xml" title="PS2 Developer wiki Atom feed" href="/ps2/index.php?title=Special:RecentChanges&feed=atom"/>
|
||||
</head>
|
||||
<body class="mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject mw-editable page-EyeToy rootpage-EyeToy skin-vector action-view skin-vector-legacy vector-feature-language-in-header-enabled vector-feature-language-in-main-page-header-disabled vector-feature-language-alert-in-sidebar-disabled vector-feature-sticky-header-disabled vector-feature-sticky-header-edit-disabled vector-feature-table-of-contents-disabled vector-feature-visual-enhancement-next-disabled"><a href="https://www.psdevwiki.com/cdn-cgi/content?id=qhF0jhCCgQ8ahbFqp65AdGceEfc6nSegHJ0R.KGGYTs-1779105525.4895601-1.0.1.1-WJ5ICF8Sy7C.15hbaKE2E0uMM5Jc3G2fGqhYkZs2gUU" aria-hidden="true" rel="nofollow noopener" style="display: none !important; visibility: hidden !important"></a><div id="mw-page-base" class="noprint"></div>
|
||||
<div id="mw-head-base" class="noprint"></div>
|
||||
<div id="content" class="mw-body" role="main">
|
||||
<a id="top"></a>
|
||||
<div id="siteNotice"></div>
|
||||
<div class="mw-indicators">
|
||||
</div>
|
||||
<h1 id="firstHeading" class="firstHeading mw-first-heading"><span class="mw-page-title-main">EyeToy</span></h1>
|
||||
<div id="bodyContent" class="vector-body">
|
||||
<div id="siteSub" class="noprint">From PS2 Developer wiki</div>
|
||||
<div id="contentSub"></div>
|
||||
<div id="contentSub2"></div>
|
||||
|
||||
<div id="jump-to-nav"></div>
|
||||
<a class="mw-jump-link" href="#mw-head">Jump to navigation</a>
|
||||
<a class="mw-jump-link" href="#searchInput">Jump to search</a>
|
||||
<div id="mw-content-text" class="mw-body-content mw-content-ltr" lang="en" dir="ltr"><div class="mw-parser-output"><div id="toc" class="toc" role="navigation" aria-labelledby="mw-toc-heading"><input type="checkbox" role="button" id="toctogglecheckbox" class="toctogglecheckbox" style="display:none" /><div class="toctitle" lang="en" dir="ltr"><h2 id="mw-toc-heading">Contents</h2><span class="toctogglespan"><label class="toctogglelabel" for="toctogglecheckbox"></label></span></div>
|
||||
<ul>
|
||||
<li class="toclevel-1 tocsection-1"><a href="#Overview"><span class="tocnumber">1</span> <span class="toctext">Overview</span></a></li>
|
||||
<li class="toclevel-1 tocsection-2"><a href="#Variants"><span class="tocnumber">2</span> <span class="toctext">Variants</span></a>
|
||||
<ul>
|
||||
<li class="toclevel-2 tocsection-3"><a href="#SLEH-00030"><span class="tocnumber">2.1</span> <span class="toctext">SLEH-00030</span></a></li>
|
||||
<li class="toclevel-2 tocsection-4"><a href="#SLEH-00031"><span class="tocnumber">2.2</span> <span class="toctext">SLEH-00031</span></a></li>
|
||||
<li class="toclevel-2 tocsection-5"><a href="#SCEH-0004"><span class="tocnumber">2.3</span> <span class="toctext">SCEH-0004</span></a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<h1><span class="mw-headline" id="Overview">Overview</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps2/index.php?title=EyeToy&veaction=edit&section=1" class="mw-editsection-visualeditor" title="Edit section: Overview">edit</a><span class="mw-editsection-divider"> | </span><a href="/ps2/edit/EyeToy?section=1" title="Edit section: Overview">edit source</a><span class="mw-editsection-bracket">]</span></span></h1>
|
||||
<p>The EyeToy is essentially just a normal USB 1.1 camera in a case that matches the PS2's design language. It was quite popular and many games either optionally support or even require it. Several variants have been made by different OEMs.
|
||||
</p>
|
||||
<h1><span class="mw-headline" id="Variants">Variants</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps2/index.php?title=EyeToy&veaction=edit&section=2" class="mw-editsection-visualeditor" title="Edit section: Variants">edit</a><span class="mw-editsection-divider"> | </span><a href="/ps2/edit/EyeToy?section=2" title="Edit section: Variants">edit source</a><span class="mw-editsection-bracket">]</span></span></h1>
|
||||
<p>SLEH-00030 and SLEH-00031 follow the fat PS2's design language and were only available in black. SCEH-0004 follows the slim PS2s design language and was only available in silver.
|
||||
</p>
|
||||
<h2><span class="mw-headline" id="SLEH-00030">SLEH-00030</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps2/index.php?title=EyeToy&veaction=edit&section=3" class="mw-editsection-visualeditor" title="Edit section: SLEH-00030">edit</a><span class="mw-editsection-divider"> | </span><a href="/ps2/edit/EyeToy?section=3" title="Edit section: SLEH-00030">edit source</a><span class="mw-editsection-bracket">]</span></span></h2>
|
||||
<p>Manufactured by Logitech.
|
||||
</p>
|
||||
<h2><span class="mw-headline" id="SLEH-00031">SLEH-00031</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps2/index.php?title=EyeToy&veaction=edit&section=4" class="mw-editsection-visualeditor" title="Edit section: SLEH-00031">edit</a><span class="mw-editsection-divider"> | </span><a href="/ps2/edit/EyeToy?section=4" title="Edit section: SLEH-00031">edit source</a><span class="mw-editsection-bracket">]</span></span></h2>
|
||||
<p>Manufactured by Namtai.
|
||||
</p>
|
||||
<h2><span class="mw-headline" id="SCEH-0004">SCEH-0004</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps2/index.php?title=EyeToy&veaction=edit&section=5" class="mw-editsection-visualeditor" title="Edit section: SCEH-0004">edit</a><span class="mw-editsection-divider"> | </span><a href="/ps2/edit/EyeToy?section=5" title="Edit section: SCEH-0004">edit source</a><span class="mw-editsection-bracket">]</span></span></h2>
|
||||
<p>Redesigned. Smaller, silver case. Manufactured by Namtai or Chicony (same model number for both manufacturers).
|
||||
</p>
|
||||
<!--
|
||||
NewPP limit report
|
||||
Cached time: 20250412025124
|
||||
Cache expiry: 2592000
|
||||
Reduced expiry: false
|
||||
Complications: [show‐toc]
|
||||
CPU time usage: 0.003 seconds
|
||||
Real time usage: 0.003 seconds
|
||||
Preprocessor visited node count: 14/1000000
|
||||
Post‐expand include size: 0/2097152 bytes
|
||||
Template argument size: 0/2097152 bytes
|
||||
Highest expansion depth: 2/100
|
||||
Expensive parser function count: 0/100
|
||||
Unstrip recursion depth: 0/20
|
||||
Unstrip post‐expand size: 0/5000000 bytes
|
||||
-->
|
||||
<!--
|
||||
Transclusion expansion time report (%,ms,calls,template)
|
||||
100.00% 0.000 1 -total
|
||||
-->
|
||||
|
||||
<!-- Saved in parser cache with key psdev_ps2devw:pcache:idhash:648-0!canonical and timestamp 20250412025124 and revision id 2605.
|
||||
-->
|
||||
</div>
|
||||
<div class="printfooter" data-nosnippet="">Retrieved from "<a dir="ltr" href="https://www.psdevwiki.com/ps2/index.php?title=EyeToy&oldid=2605">https://www.psdevwiki.com/ps2/index.php?title=EyeToy&oldid=2605</a>"</div></div>
|
||||
<div id="catlinks" class="catlinks catlinks-allhidden" data-mw="interface"></div>
|
||||
</div>
|
||||
</div>
|
||||
<div id='mw-data-after-content'>
|
||||
<div class="gas-bottom"><script async src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script><ins class="adsbygoogle gas-bottom-first"
|
||||
style="display:table; width:728px; height:90px; margin:1em auto; clear:both;"
|
||||
data-ad-client="ca-pub-9015332520988458"
|
||||
data-ad-slot="7897517726"></ins><script>
|
||||
(adsbygoogle = window.adsbygoogle || []).push({});
|
||||
</script></div><div class="gas-bottom"><script async src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script><ins class="adsbygoogle gas-bottom-first"
|
||||
style="display:table; width:728px; height:90px; margin:1em auto; clear:both;"
|
||||
data-ad-client="ca-pub-9015332520988458"
|
||||
data-ad-slot="1931323254"></ins><script>
|
||||
(adsbygoogle = window.adsbygoogle || []).push({});
|
||||
</script></div><div class="mw-cookiewarning-container"><div class="mw-cookiewarning-text"><span>Cookies help us deliver our services. By using our services, you agree to our use of cookies.</span></div><form method="POST"><div class='oo-ui-layout oo-ui-horizontalLayout'><span class='oo-ui-widget oo-ui-widget-enabled oo-ui-buttonElement oo-ui-buttonElement-framed oo-ui-labelElement oo-ui-flaggedElement-progressive oo-ui-buttonWidget'><a role='button' tabindex='0' href='https://www.psdevwiki.com/ps3/PS3_Developer_wiki:Privacy_policy' rel='nofollow' class='oo-ui-buttonElement-button'><span class='oo-ui-iconElement-icon oo-ui-iconElement-noIcon oo-ui-image-progressive'></span><span class='oo-ui-labelElement-label'>More information</span><span class='oo-ui-indicatorElement-indicator oo-ui-indicatorElement-noIndicator oo-ui-image-progressive'></span></a></span><span class='oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-buttonElement oo-ui-buttonElement-framed oo-ui-labelElement oo-ui-flaggedElement-primary oo-ui-flaggedElement-progressive oo-ui-buttonInputWidget'><button type='submit' tabindex='0' name='disablecookiewarning' value='OK' class='oo-ui-inputWidget-input oo-ui-buttonElement-button'><span class='oo-ui-iconElement-icon oo-ui-iconElement-noIcon oo-ui-image-invert'></span><span class='oo-ui-labelElement-label'>OK</span><span class='oo-ui-indicatorElement-indicator oo-ui-indicatorElement-noIndicator oo-ui-image-invert'></span></button></span></div></form></div>
|
||||
</div>
|
||||
|
||||
<div id="mw-navigation">
|
||||
<h2>Navigation menu</h2>
|
||||
<div id="mw-head">
|
||||
|
||||
|
||||
<nav id="p-personal" class="vector-menu mw-portlet mw-portlet-personal vector-user-menu-legacy" aria-labelledby="p-personal-label" role="navigation" >
|
||||
<h3
|
||||
id="p-personal-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Personal tools</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list"><li id="pt-anonuserpage" class="mw-list-item"><span title="The user page for the IP address you are editing as">Not logged in</span></li><li id="pt-anontalk" class="mw-list-item"><a href="/ps2/Special:MyTalk" title="Discussion about edits from this IP address [n]" accesskey="n"><span>Talk</span></a></li><li id="pt-darkmode" class="mw-list-item"><a href="#" class="ext-darkmode-link"><span>Dark mode (beta)</span></a></li><li id="pt-anoncontribs" class="mw-list-item"><a href="/ps2/Special:MyContributions" title="A list of edits made from this IP address [y]" accesskey="y"><span>Contributions</span></a></li><li id="pt-createaccount" class="mw-list-item"><a href="/ps2/index.php?title=Special:CreateAccount&returnto=EyeToy" title="You are encouraged to create an account and log in; however, it is not mandatory"><span>Create account</span></a></li><li id="pt-login" class="mw-list-item"><a href="/ps2/index.php?title=Special:UserLogin&returnto=EyeToy" title="You are encouraged to log in; however, it is not mandatory [o]" accesskey="o"><span>Log in</span></a></li></ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<div id="left-navigation">
|
||||
|
||||
|
||||
<nav id="p-namespaces" class="vector-menu mw-portlet mw-portlet-namespaces vector-menu-tabs vector-menu-tabs-legacy" aria-labelledby="p-namespaces-label" role="navigation" >
|
||||
<h3
|
||||
id="p-namespaces-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Namespaces</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list"><li id="ca-nstab-main" class="selected mw-list-item"><a href="/ps2/EyeToy" title="View the content page [c]" accesskey="c"><span>Page</span></a></li><li id="ca-talk" class="new mw-list-item"><a href="/ps2/edit/Talk:EyeToy?redlink=1" rel="discussion" title="Discussion about the content page (page does not exist) [t]" accesskey="t"><span>Discussion</span></a></li></ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
|
||||
<nav id="p-variants" class="vector-menu mw-portlet mw-portlet-variants emptyPortlet vector-menu-dropdown" aria-labelledby="p-variants-label" role="navigation" >
|
||||
<input type="checkbox"
|
||||
id="p-variants-checkbox"
|
||||
role="button"
|
||||
aria-haspopup="true"
|
||||
data-event-name="ui.dropdown-p-variants"
|
||||
class="vector-menu-checkbox"
|
||||
aria-labelledby="p-variants-label"
|
||||
/>
|
||||
<label
|
||||
id="p-variants-label"
|
||||
aria-label="Change language variant"
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">English</span>
|
||||
</label>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list"></ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
</div>
|
||||
<div id="right-navigation">
|
||||
|
||||
|
||||
<nav id="p-views" class="vector-menu mw-portlet mw-portlet-views vector-menu-tabs vector-menu-tabs-legacy" aria-labelledby="p-views-label" role="navigation" >
|
||||
<h3
|
||||
id="p-views-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Views</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list"><li id="ca-view" class="selected mw-list-item"><a href="/ps2/EyeToy"><span>Read</span></a></li><li id="ca-ve-edit" class="mw-list-item"><a href="/ps2/index.php?title=EyeToy&veaction=edit" title="Edit this page [v]" accesskey="v"><span>Edit</span></a></li><li id="ca-edit" class="collapsible mw-list-item"><a href="/ps2/edit/EyeToy" title="Edit this page [e]" accesskey="e"><span>Edit source</span></a></li><li id="ca-history" class="mw-list-item"><a href="/ps2/history/EyeToy" title="Past revisions of this page [h]" accesskey="h"><span>View history</span></a></li></ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
|
||||
<nav id="p-cactions" class="vector-menu mw-portlet mw-portlet-cactions emptyPortlet vector-menu-dropdown" aria-labelledby="p-cactions-label" role="navigation" title="More options" >
|
||||
<input type="checkbox"
|
||||
id="p-cactions-checkbox"
|
||||
role="button"
|
||||
aria-haspopup="true"
|
||||
data-event-name="ui.dropdown-p-cactions"
|
||||
class="vector-menu-checkbox"
|
||||
aria-labelledby="p-cactions-label"
|
||||
/>
|
||||
<label
|
||||
id="p-cactions-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">More</span>
|
||||
</label>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list"></ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
<div id="p-search" role="search" class="vector-search-box-vue vector-search-box-show-thumbnail vector-search-box-auto-expand-width vector-search-box">
|
||||
<div>
|
||||
<h3 >
|
||||
<label for="searchInput">Search</label>
|
||||
</h3>
|
||||
<form action="/ps2/index.php" id="searchform"
|
||||
class="vector-search-box-form">
|
||||
<div id="simpleSearch"
|
||||
class="vector-search-box-inner"
|
||||
data-search-loc="header-navigation">
|
||||
<input class="vector-search-box-input"
|
||||
type="search" name="search" placeholder="Search PS2 Developer wiki" aria-label="Search PS2 Developer wiki" autocapitalize="sentences" title="Search PS2 Developer wiki [f]" accesskey="f" id="searchInput"
|
||||
>
|
||||
<input type="hidden" name="title" value="Special:Search">
|
||||
<input id="mw-searchButton"
|
||||
class="searchButton mw-fallbackSearchButton" type="submit" name="fulltext" title="Search the pages for this text" value="Search">
|
||||
<input id="searchButton"
|
||||
class="searchButton" type="submit" name="go" title="Go to a page with this exact name if it exists" value="Go">
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<div id="mw-panel">
|
||||
<div id="p-logo" role="banner">
|
||||
<a class="mw-wiki-logo" href="/ps2/Main_Page"
|
||||
title="Visit the main page"></a>
|
||||
</div>
|
||||
|
||||
|
||||
<nav id="p-PlayStation_Devwikis" class="vector-menu mw-portlet mw-portlet-PlayStation_Devwikis vector-menu-portal portal" aria-labelledby="p-PlayStation_Devwikis-label" role="navigation" >
|
||||
<h3
|
||||
id="p-PlayStation_Devwikis-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">PlayStation Devwikis</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list"><li id="n-Home" class="mw-list-item"><a href="https://www.psdevwiki.com" rel="nofollow" target="_blank"><span>Home</span></a></li><li id="n-PS1-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/ps1/" rel="nofollow" target="_blank"><span>PS1 Dev Wiki</span></a></li><li id="n-PS2-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/ps2/" rel="nofollow" target="_blank"><span>PS2 Dev Wiki</span></a></li><li id="n-PS3-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/ps3/" rel="nofollow" target="_blank"><span>PS3 Dev Wiki</span></a></li><li id="n-PS4-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/ps4/" rel="nofollow" target="_blank"><span>PS4 Dev Wiki</span></a></li><li id="n-PS5-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/ps5/" rel="nofollow" target="_blank"><span>PS5 Dev Wiki</span></a></li><li id="n-PSP-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/psp/" rel="nofollow" target="_blank"><span>PSP Dev Wiki</span></a></li><li id="n-Vita-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/vita/" rel="nofollow" target="_blank"><span>Vita Dev Wiki</span></a></li><li id="n-PS-Classic-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/psclassic" rel="nofollow" target="_blank"><span>PS Classic Dev Wiki</span></a></li></ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
|
||||
<nav id="p-Other_Wikis" class="vector-menu mw-portlet mw-portlet-Other_Wikis vector-menu-portal portal" aria-labelledby="p-Other_Wikis-label" role="navigation" >
|
||||
<h3
|
||||
id="p-Other_Wikis-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Other Wikis</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list"><li id="n-HENkaku-Dev-Wiki" class="mw-list-item"><a href="http://wiki.henkaku.xyz/" rel="nofollow" target="_blank"><span>HENkaku Dev Wiki</span></a></li><li id="n-ConsoleMods.org" class="mw-list-item"><a href="https://www.consolemods.org/wiki/Main_Page" rel="nofollow" target="_blank"><span>ConsoleMods.org</span></a></li><li id="n-TeamXLink-Wiki" class="mw-list-item"><a href="https://www.teamxlink.co.uk/wiki/Main_Page" rel="nofollow" target="_blank"><span>TeamXLink Wiki</span></a></li></ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
<nav id="p-Discord" class="vector-menu mw-portlet mw-portlet-Discord vector-menu-portal portal" aria-labelledby="p-Discord-label" role="navigation" >
|
||||
<h3
|
||||
id="p-Discord-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Discord</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list"><li id="n-PlayStation-Devwiki" class="mw-list-item"><a href="https://discord.gg/K7fjFt3" rel="nofollow" target="_blank"><span>PlayStation Devwiki</span></a></li><li id="n-ConsoleMods.org" class="mw-list-item"><a href="https://discord.gg/x5vEnkR4C8" rel="nofollow" target="_blank"><span>ConsoleMods.org</span></a></li></ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
<nav id="p-navigation" class="vector-menu mw-portlet mw-portlet-navigation vector-menu-portal portal" aria-labelledby="p-navigation-label" role="navigation" >
|
||||
<h3
|
||||
id="p-navigation-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Navigation</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list"><li id="n-mainpage-description" class="mw-list-item"><a href="/ps2/Main_Page" title="Visit the main page [z]" accesskey="z"><span>Main page</span></a></li><li id="n-portal" class="mw-list-item"><a href="/ps2/PS2_Developer_wiki:Community_portal" title="About the project, what you can do, where to find things"><span>Community portal</span></a></li><li id="n-currentevents" class="mw-list-item"><a href="/ps2/PS2_Developer_wiki:Current_events" title="Find background information on current events"><span>Current events</span></a></li><li id="n-recentchanges" class="mw-list-item"><a href="/ps2/Special:RecentChanges" title="A list of recent changes in the wiki [r]" accesskey="r"><span>Recent changes</span></a></li><li id="n-contributionscores" class="mw-list-item"><a href="/ps2/Special:ContributionScores"><span>Contribution scores</span></a></li><li id="n-randompage" class="mw-list-item"><a href="/ps2/Special:Random" title="Load a random page [x]" accesskey="x"><span>Random page</span></a></li><li id="n-All-Pages" class="mw-list-item"><a href="/ps2/Special:AllPages"><span>All Pages</span></a></li><li id="n-Staff-contacts" class="mw-list-item"><a href="https://www.psdevwiki.com/ps3/Staff_Contacts" rel="nofollow" target="_blank"><span>Staff contacts</span></a></li><li id="n-help" class="mw-list-item"><a href="https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents" target="_blank" title="The place to find out"><span>Help</span></a></li></ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
<nav id="p-tb" class="vector-menu mw-portlet mw-portlet-tb vector-menu-portal portal" aria-labelledby="p-tb-label" role="navigation" >
|
||||
<h3
|
||||
id="p-tb-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Tools</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list"><li id="t-whatlinkshere" class="mw-list-item"><a href="/ps2/Special:WhatLinksHere/EyeToy" title="A list of all wiki pages that link here [j]" accesskey="j"><span>What links here</span></a></li><li id="t-recentchangeslinked" class="mw-list-item"><a href="/ps2/Special:RecentChangesLinked/EyeToy" rel="nofollow" title="Recent changes in pages linked from this page [k]" accesskey="k"><span>Related changes</span></a></li><li id="t-specialpages" class="mw-list-item"><a href="/ps2/Special:SpecialPages" title="A list of all special pages [q]" accesskey="q"><span>Special pages</span></a></li><li id="t-print" class="mw-list-item"><a href="javascript:print();" rel="alternate" title="Printable version of this page [p]" accesskey="p"><span>Printable version</span></a></li><li id="t-permalink" class="mw-list-item"><a href="/ps2/index.php?title=EyeToy&oldid=2605" title="Permanent link to this revision of this page"><span>Permanent link</span></a></li><li id="t-info" class="mw-list-item"><a href="/ps2/index.php?title=EyeToy&action=info" title="More information about this page"><span>Page information</span></a></li></ul>
|
||||
<div class="after-portlet after-portlet-tb"><script async src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
|
||||
<ins class="adsbygoogle"
|
||||
style="display:inline-block;width:160px;height:1000px;margin:20px 0 0 -20px;"
|
||||
data-ad-client="ca-pub-9015332520988458"
|
||||
data-ad-slot="1352290675"
|
||||
</ins>
|
||||
<script>
|
||||
(adsbygoogle = window.adsbygoogle || []).push({});
|
||||
</script></div>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
<footer id="footer" class="mw-footer" role="contentinfo" >
|
||||
<ul id="footer-info">
|
||||
<li id="footer-info-lastmod"> This page was last edited on 7 May 2023, at 16:25.</li>
|
||||
<li id="footer-info-copyright">Content is available under <a class="external" rel="nofollow" href="http://www.gnu.org/licenses/old-licenses/fdl-1.2.txt">GNU Free Documentation License 1.2</a> unless otherwise noted.</li>
|
||||
</ul>
|
||||
|
||||
<ul id="footer-places">
|
||||
<li id="footer-places-privacy"><a href="/ps2/PS2_Developer_wiki:Privacy_policy">Privacy policy</a></li>
|
||||
<li id="footer-places-about"><a href="/ps2/PS2_Developer_wiki:About">About PS2 Developer wiki</a></li>
|
||||
<li id="footer-places-disclaimer"><a href="/ps2/PS2_Developer_wiki:General_disclaimer">Disclaimers</a></li>
|
||||
</ul>
|
||||
|
||||
<ul id="footer-icons" class="noprint">
|
||||
<li id="footer-copyrightico"><a href="http://www.gnu.org/licenses/old-licenses/fdl-1.2.txt" target="_blank"><img src="/ps2/resources/assets/licenses/gnu-fdl.png" alt="GNU Free Documentation License 1.2" width="88" height="31" loading="lazy"/></a></li>
|
||||
<li id="footer-poweredbyico"><a href="https://www.mediawiki.org/" target="_blank"><img src="/ps2/resources/assets/poweredby_mediawiki_88x31.png" alt="Powered by MediaWiki" srcset="/ps2/resources/assets/poweredby_mediawiki_132x47.png 1.5x, /ps2/resources/assets/poweredby_mediawiki_176x62.png 2x" width="88" height="31" loading="lazy"/></a></li>
|
||||
</ul>
|
||||
|
||||
</footer>
|
||||
|
||||
<script src="https://www.googletagmanager.com/gtag/js?id=G-0DGNXPL208" async=""></script><script>
|
||||
window.dataLayer = window.dataLayer || [];
|
||||
|
||||
function gtag(){dataLayer.push(arguments);}
|
||||
gtag('js', new Date());
|
||||
gtag('config', 'G-0DGNXPL208', {"anonymize_ip":true});
|
||||
</script>
|
||||
|
||||
<script>(RLQ=window.RLQ||[]).push(function(){mw.config.set({"wgPageParseReport":{"limitreport":{"cputime":"0.003","walltime":"0.003","ppvisitednodes":{"value":14,"limit":1000000},"postexpandincludesize":{"value":0,"limit":2097152},"templateargumentsize":{"value":0,"limit":2097152},"expansiondepth":{"value":2,"limit":100},"expensivefunctioncount":{"value":0,"limit":100},"unstrip-depth":{"value":0,"limit":20},"unstrip-size":{"value":0,"limit":5000000},"timingprofile":["100.00% 0.000 1 -total"]},"cachereport":{"timestamp":"20250412025124","ttl":2592000,"transientcontent":false}}});mw.config.set({"wgBackendResponseTime":224});});</script>
|
||||
<script>(function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9fdab11e48b36918',t:'MTc3OTEwNTUyNQ=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();</script></body>
|
||||
<!-- Cached 20250412073914 -->
|
||||
</html>
|
||||
|
|
@ -0,0 +1,482 @@
|
|||
|
||||
<!DOCTYPE html>
|
||||
<html class="client-nojs" lang="en" dir="ltr">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<title>PlayStation Eye - PS3 Developer wiki</title>
|
||||
<script>(function(){var className="client-js";var cookie=document.cookie.match(/(?:^|; )psdevwiki_ps3_wiki_mwclientpreferences=([^;]+)/);if(cookie){cookie[1].split('%2C').forEach(function(pref){className=className.replace(new RegExp('(^| )'+pref.replace(/-clientpref-\w+$|[^\w-]+/g,'')+'-clientpref-\\w+( |$)'),'$1'+pref+'$2');});}document.documentElement.className=className;}());RLCONF={"wgBreakFrames":false,"wgSeparatorTransformTable":["",""],"wgDigitTransformTable":["",""],"wgDefaultDateFormat":"dmy","wgMonthNames":["","January","February","March","April","May","June","July","August","September","October","November","December"],"wgRequestId":"9dd65bff71596c0cdfa164d1","wgCanonicalNamespace":"","wgCanonicalSpecialPageName":false,"wgNamespaceNumber":0,"wgPageName":"PlayStation_Eye","wgTitle":"PlayStation Eye","wgCurRevisionId":74326,"wgRevisionId":74326,"wgArticleId":1522,"wgIsArticle":true,"wgIsRedirect":false,"wgAction":"view","wgUserName":null,"wgUserGroups":["*"],"wgCategories":["Peripherals","Main"],"wgPageViewLanguage":"en","wgPageContentLanguage":"en","wgPageContentModel":"wikitext","wgRelevantPageName":"PlayStation_Eye","wgRelevantArticleId":1522,"wgIsProbablyEditable":true,"wgRelevantPageIsProbablyEditable":true,"wgRestrictionEdit":[],"wgRestrictionMove":[],"simpleBatchUploadMaxFilesPerBatch":{"*":1000},"wgVisualEditor":{"pageLanguageCode":"en","pageLanguageDir":"ltr","pageVariantFallbacks":"en"},"wgCiteReferencePreviewsActive":true,"wgEditSubmitButtonLabelPublish":false,"wgCheckUserClientHintsHeadersJsApi":["architecture","bitness","brands","fullVersionList","mobile","model","platform","platformVersion"]};
|
||||
RLSTATE={"site.styles":"ready","user.styles":"ready","user":"ready","user.options":"loading","mediawiki.page.gallery.styles":"ready","skins.vector.styles.legacy":"ready","jquery.tablesorter.styles":"ready","ext.CookieWarning.styles":"ready","oojs-ui-core.styles":"ready","oojs-ui.styles.indicators":"ready","mediawiki.widgets.styles":"ready","oojs-ui-core.icons":"ready","ext.visualEditor.desktopArticleTarget.noscript":"ready","ext.embedVideo.styles":"ready","ext.DarkMode.styles":"ready"};RLPAGEMODULES=["mediawiki.page.media","site","mediawiki.page.ready","jquery.tablesorter","mediawiki.toc","skins.vector.legacy.js","ext.CookieWarning","ext.visualEditor.desktopArticleTarget.init","ext.visualEditor.targetLoader","ext.embedVideo.overlay","ext.checkUser.clientHints","ext.DarkMode","ext.moderation.notify","ext.moderation.ve","ext.moderation.ajaxhook","ext.moderation.notify.desktop","ext.purge"];</script>
|
||||
<script>(RLQ=window.RLQ||[]).push(function(){mw.loader.impl(function(){return["user.options@12s5i",function($,jQuery,require,module){mw.user.tokens.set({"patrolToken":"+\\","watchToken":"+\\","csrfToken":"+\\"});
|
||||
}];});});</script>
|
||||
<link rel="stylesheet" href="/ps3/load.php?lang=en&modules=ext.CookieWarning.styles%7Cext.DarkMode.styles%7Cext.embedVideo.styles%7Cext.visualEditor.desktopArticleTarget.noscript%7Cjquery.tablesorter.styles%7Cmediawiki.page.gallery.styles%7Cmediawiki.widgets.styles%7Coojs-ui-core.icons%2Cstyles%7Coojs-ui.styles.indicators%7Cskins.vector.styles.legacy&only=styles&skin=vector">
|
||||
<script async="" src="/ps3/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector"></script>
|
||||
<style>:root { --sdm-accent: gold; }</style>
|
||||
<meta name="ResourceLoaderDynamicStyles" content="">
|
||||
<link rel="stylesheet" href="/ps3/load.php?lang=en&modules=site.styles&only=styles&skin=vector">
|
||||
<meta name="generator" content="MediaWiki 1.43.3">
|
||||
<meta name="robots" content="max-image-preview:standard">
|
||||
<meta name="format-detection" content="telephone=no">
|
||||
<meta name="description" content="Playstation Development Wiki - PS5, PS4, PS3, PS2, PS1, PSP, Vita Information">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=0.5">
|
||||
<meta name="viewport" content="width=1120">
|
||||
<link rel="alternate" type="application/x-wiki" title="Edit" href="/ps3/edit/PlayStation_Eye">
|
||||
<link rel="search" type="application/opensearchdescription+xml" href="/ps3/rest.php/v1/search" title="PS3 Developer wiki (en)">
|
||||
<link rel="EditURI" type="application/rsd+xml" href="https://www.psdevwiki.com/ps3/api.php?action=rsd">
|
||||
<link rel="license" href="https://www.gnu.org/licenses/old-licenses/fdl-1.2.txt">
|
||||
<link rel="alternate" type="application/atom+xml" title="PS3 Developer wiki Atom feed" href="/ps3/index.php?title=Special:RecentChanges&feed=atom">
|
||||
</head>
|
||||
<body class="skin-vector-legacy mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject mw-editable page-PlayStation_Eye rootpage-PlayStation_Eye skin-vector action-view"><a href="https://www.psdevwiki.com/cdn-cgi/content?id=vJpkKLVvZiyzkold2W6YfhT3Ju.6SdC2WoMYWsGCGgo-1779105525.012252-1.0.1.1-4bI2Hz7RrzkbrkLDMwAOPbi3hQnr8rW6Omw0sxMFB7A" aria-hidden="true" rel="nofollow noopener" style="display: none !important; visibility: hidden !important"></a><div id="mw-page-base" class="noprint"></div>
|
||||
<div id="mw-head-base" class="noprint"></div>
|
||||
<div id="content" class="mw-body" role="main">
|
||||
<a id="top"></a>
|
||||
<div id="siteNotice"></div>
|
||||
<div class="mw-indicators">
|
||||
</div>
|
||||
<h1 id="firstHeading" class="firstHeading mw-first-heading"><span class="mw-page-title-main">PlayStation Eye</span></h1>
|
||||
<div id="bodyContent" class="vector-body">
|
||||
<div id="siteSub" class="noprint">From PS3 Developer wiki</div>
|
||||
<div id="contentSub"><div id="mw-content-subtitle"></div></div>
|
||||
<div id="contentSub2"></div>
|
||||
|
||||
<div id="jump-to-nav"></div>
|
||||
<a class="mw-jump-link" href="#mw-head">Jump to navigation</a>
|
||||
<a class="mw-jump-link" href="#searchInput">Jump to search</a>
|
||||
<div id="mw-content-text" class="mw-body-content"><div class="mw-content-ltr mw-parser-output" lang="en" dir="ltr"><div style="float:right"><figure class="mw-halign-right" typeof="mw:File/Thumb"><a href="/ps3/File:PlayStation_Eye.png" class="mw-file-description"><img src="/ps3/images/thumb/5/52/PlayStation_Eye.png/200px-PlayStation_Eye.png" decoding="async" width="200" height="207" class="mw-file-element" srcset="/ps3/images/thumb/5/52/PlayStation_Eye.png/300px-PlayStation_Eye.png 1.5x, /ps3/images/thumb/5/52/PlayStation_Eye.png/400px-PlayStation_Eye.png 2x" /></a><figcaption>PlayStation Eye</figcaption></figure></div>
|
||||
<div id="toc" class="toc" role="navigation" aria-labelledby="mw-toc-heading"><input type="checkbox" role="button" id="toctogglecheckbox" class="toctogglecheckbox" style="display:none" /><div class="toctitle" lang="en" dir="ltr"><h2 id="mw-toc-heading">Contents</h2><span class="toctogglespan"><label class="toctogglelabel" for="toctogglecheckbox"></label></span></div>
|
||||
<ul>
|
||||
<li class="toclevel-1 tocsection-1"><a href="#PlayStation_Eye"><span class="tocnumber">1</span> <span class="toctext">PlayStation Eye</span></a>
|
||||
<ul>
|
||||
<li class="toclevel-2 tocsection-2"><a href="#Video_and_Audio"><span class="tocnumber">1.1</span> <span class="toctext">Video and Audio</span></a>
|
||||
<ul>
|
||||
<li class="toclevel-3 tocsection-3"><a href="#Video"><span class="tocnumber">1.1.1</span> <span class="toctext">Video</span></a></li>
|
||||
<li class="toclevel-3 tocsection-4"><a href="#Audio"><span class="tocnumber">1.1.2</span> <span class="toctext">Audio</span></a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toclevel-1 tocsection-5"><a href="#Gallery"><span class="tocnumber">2</span> <span class="toctext">Gallery</span></a></li>
|
||||
<li class="toclevel-1 tocsection-6"><a href="#PS3Eye_multicam_driver"><span class="tocnumber">3</span> <span class="toctext">PS3Eye multicam driver</span></a></li>
|
||||
<li class="toclevel-1 tocsection-7"><a href="#PS3_Eye_Streaming_with_Raspberry"><span class="tocnumber">4</span> <span class="toctext">PS3 Eye Streaming with Raspberry</span></a></li>
|
||||
<li class="toclevel-1 tocsection-8"><a href="#Firmware_related_files"><span class="tocnumber">5</span> <span class="toctext">Firmware related files</span></a></li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<h2><span class="mw-headline" id="PlayStation_Eye">PlayStation Eye</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps3/index.php?title=PlayStation_Eye&veaction=edit&section=1" title="Edit section: PlayStation Eye" class="mw-editsection-visualeditor"><span>edit</span></a><span class="mw-editsection-divider"> | </span><a href="/ps3/edit/PlayStation_Eye?section=1" title="Edit section's source code: PlayStation Eye"><span>edit source</span></a><span class="mw-editsection-bracket">]</span></span></h2>
|
||||
<p>many disassembly pictures @ <a target="_blank" rel="nofollow noreferrer noopener" class="external free" href="http://codelaboratories.com/research/view/ps3-eye-disassembly">http://codelaboratories.com/research/view/ps3-eye-disassembly</a>
|
||||
</p><p><br />
|
||||
CEJH-15001 (SCPH-98047, SLEH-00448, SLEH-00201, SLEH-00203, ) <br />
|
||||
DC 5V, 500mA <br />
|
||||
</p>
|
||||
<h3><span class="mw-headline" id="Video_and_Audio">Video and Audio</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps3/index.php?title=PlayStation_Eye&veaction=edit&section=2" title="Edit section: Video and Audio" class="mw-editsection-visualeditor"><span>edit</span></a><span class="mw-editsection-divider"> | </span><a href="/ps3/edit/PlayStation_Eye?section=2" title="Edit section's source code: Video and Audio"><span>edit source</span></a><span class="mw-editsection-bracket">]</span></span></h3>
|
||||
<h4><span class="mw-headline" id="Video">Video</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps3/index.php?title=PlayStation_Eye&veaction=edit&section=3" title="Edit section: Video" class="mw-editsection-visualeditor"><span>edit</span></a><span class="mw-editsection-divider"> | </span><a href="/ps3/edit/PlayStation_Eye?section=3" title="Edit section's source code: Video"><span>edit source</span></a><span class="mw-editsection-bracket">]</span></span></h4>
|
||||
<ul><li>640×480 pixels @ 60 Hz</li>
|
||||
<li>320×240 pixels @ 120 Hz</li></ul>
|
||||
<h4><span class="mw-headline" id="Audio">Audio</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps3/index.php?title=PlayStation_Eye&veaction=edit&section=4" title="Edit section: Audio" class="mw-editsection-visualeditor"><span>edit</span></a><span class="mw-editsection-divider"> | </span><a href="/ps3/edit/PlayStation_Eye?section=4" title="Edit section's source code: Audio"><span>edit source</span></a><span class="mw-editsection-bracket">]</span></span></h4>
|
||||
<ul><li>16-bit audio @ 48 kHz</li>
|
||||
<li>SNR (sound to noise ratio) @ 90 decibles</li></ul>
|
||||
<h2><span class="mw-headline" id="Gallery">Gallery</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps3/index.php?title=PlayStation_Eye&veaction=edit&section=5" title="Edit section: Gallery" class="mw-editsection-visualeditor"><span>edit</span></a><span class="mw-editsection-divider"> | </span><a href="/ps3/edit/PlayStation_Eye?section=5" title="Edit section's source code: Gallery"><span>edit source</span></a><span class="mw-editsection-bracket">]</span></span></h2>
|
||||
<ul class="gallery mw-gallery-traditional">
|
||||
<li class="gallerybox" style="width: 155px">
|
||||
<div class="thumb" style="width: 150px; height: 150px;"><span typeof="mw:File"><a href="/ps3/File:IMG_20110516_174500.jpg" class="mw-file-description" title="Inside, back"><img alt="Inside, back" src="/ps3/images/thumb/9/99/IMG_20110516_174500.jpg/120px-IMG_20110516_174500.jpg" decoding="async" width="120" height="90" class="mw-file-element" srcset="/ps3/images/thumb/9/99/IMG_20110516_174500.jpg/180px-IMG_20110516_174500.jpg 1.5x, /ps3/images/thumb/9/99/IMG_20110516_174500.jpg/240px-IMG_20110516_174500.jpg 2x" /></a></span></div>
|
||||
<div class="gallerytext">Inside, back</div>
|
||||
</li>
|
||||
<li class="gallerybox" style="width: 155px">
|
||||
<div class="thumb" style="width: 150px; height: 150px;"><span typeof="mw:File"><a href="/ps3/File:IMG_20110516_175011.jpg" class="mw-file-description" title="Inside, front"><img alt="Inside, front" src="/ps3/images/thumb/6/69/IMG_20110516_175011.jpg/90px-IMG_20110516_175011.jpg" decoding="async" width="90" height="120" class="mw-file-element" srcset="/ps3/images/thumb/6/69/IMG_20110516_175011.jpg/135px-IMG_20110516_175011.jpg 1.5x, /ps3/images/thumb/6/69/IMG_20110516_175011.jpg/180px-IMG_20110516_175011.jpg 2x" /></a></span></div>
|
||||
<div class="gallerytext">Inside, front</div>
|
||||
</li>
|
||||
<li class="gallerybox" style="width: 155px">
|
||||
<div class="thumb" style="width: 150px; height: 150px;"><span typeof="mw:File"><a href="/ps3/File:PS3Eye_-_kylemcdonald.jpg" class="mw-file-description" title="Board detail"><img alt="Board detail" src="/ps3/images/thumb/a/a6/PS3Eye_-_kylemcdonald.jpg/120px-PS3Eye_-_kylemcdonald.jpg" decoding="async" width="120" height="90" class="mw-file-element" srcset="/ps3/images/thumb/a/a6/PS3Eye_-_kylemcdonald.jpg/180px-PS3Eye_-_kylemcdonald.jpg 1.5x, /ps3/images/thumb/a/a6/PS3Eye_-_kylemcdonald.jpg/240px-PS3Eye_-_kylemcdonald.jpg 2x" /></a></span></div>
|
||||
<div class="gallerytext">Board detail</div>
|
||||
</li>
|
||||
<li class="gallerybox" style="width: 155px">
|
||||
<div class="thumb" style="width: 150px; height: 150px;"><span typeof="mw:File"><a href="/ps3/File:Playstation_Eye_-_Front_PCB.jpg" class="mw-file-description" title="Front"><img alt="Front" src="/ps3/images/thumb/5/53/Playstation_Eye_-_Front_PCB.jpg/120px-Playstation_Eye_-_Front_PCB.jpg" decoding="async" width="120" height="90" class="mw-file-element" srcset="/ps3/images/thumb/5/53/Playstation_Eye_-_Front_PCB.jpg/180px-Playstation_Eye_-_Front_PCB.jpg 1.5x, /ps3/images/thumb/5/53/Playstation_Eye_-_Front_PCB.jpg/240px-Playstation_Eye_-_Front_PCB.jpg 2x" /></a></span></div>
|
||||
<div class="gallerytext">Front</div>
|
||||
</li>
|
||||
<li class="gallerybox" style="width: 155px">
|
||||
<div class="thumb" style="width: 150px; height: 150px;"><span typeof="mw:File"><a href="/ps3/File:Playstation_Eye_-_circuitboard.jpg" class="mw-file-description" title="Circuit Board"><img alt="Circuit Board" src="/ps3/images/thumb/7/7f/Playstation_Eye_-_circuitboard.jpg/120px-Playstation_Eye_-_circuitboard.jpg" decoding="async" width="120" height="90" class="mw-file-element" srcset="/ps3/images/thumb/7/7f/Playstation_Eye_-_circuitboard.jpg/180px-Playstation_Eye_-_circuitboard.jpg 1.5x, /ps3/images/thumb/7/7f/Playstation_Eye_-_circuitboard.jpg/240px-Playstation_Eye_-_circuitboard.jpg 2x" /></a></span></div>
|
||||
<div class="gallerytext">Circuit Board</div>
|
||||
</li>
|
||||
<li class="gallerybox" style="width: 155px">
|
||||
<div class="thumb" style="width: 150px; height: 150px;"><span typeof="mw:File"><a href="/ps3/File:Playstation_Eye_-_dismantled.jpg" class="mw-file-description" title="Dismantled"><img alt="Dismantled" src="/ps3/images/thumb/e/e7/Playstation_Eye_-_dismantled.jpg/120px-Playstation_Eye_-_dismantled.jpg" decoding="async" width="120" height="90" class="mw-file-element" srcset="/ps3/images/thumb/e/e7/Playstation_Eye_-_dismantled.jpg/180px-Playstation_Eye_-_dismantled.jpg 1.5x, /ps3/images/thumb/e/e7/Playstation_Eye_-_dismantled.jpg/240px-Playstation_Eye_-_dismantled.jpg 2x" /></a></span></div>
|
||||
<div class="gallerytext">Dismantled</div>
|
||||
</li>
|
||||
</ul>
|
||||
<h2><span class="mw-headline" id="PS3Eye_multicam_driver">PS3Eye multicam driver</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps3/index.php?title=PlayStation_Eye&veaction=edit&section=6" title="Edit section: PS3Eye multicam driver" class="mw-editsection-visualeditor"><span>edit</span></a><span class="mw-editsection-divider"> | </span><a href="/ps3/edit/PlayStation_Eye?section=6" title="Edit section's source code: PS3Eye multicam driver"><span>edit source</span></a><span class="mw-editsection-bracket">]</span></span></h2>
|
||||
<p><a target="_blank" rel="nofollow noreferrer noopener" class="external free" href="http://nuigroup.com/forums/viewthread/6903/">http://nuigroup.com/forums/viewthread/6903/</a>
|
||||
</p>
|
||||
<h2><span class="mw-headline" id="PS3_Eye_Streaming_with_Raspberry">PS3 Eye Streaming with Raspberry</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps3/index.php?title=PlayStation_Eye&veaction=edit&section=7" title="Edit section: PS3 Eye Streaming with Raspberry" class="mw-editsection-visualeditor"><span>edit</span></a><span class="mw-editsection-divider"> | </span><a href="/ps3/edit/PlayStation_Eye?section=7" title="Edit section's source code: PS3 Eye Streaming with Raspberry"><span>edit source</span></a><span class="mw-editsection-bracket">]</span></span></h2>
|
||||
<p><a target="_blank" rel="nofollow noreferrer noopener" class="external free" href="http://www.raspberrypi.org/phpBB3/viewtopic.php?f=26&t=7397">http://www.raspberrypi.org/phpBB3/viewtopic.php?f=26&t=7397</a>
|
||||
</p>
|
||||
<h1><span class="mw-headline" id="Firmware_related_files">Firmware related files</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/ps3/index.php?title=PlayStation_Eye&veaction=edit&section=8" title="Edit section: Firmware related files" class="mw-editsection-visualeditor"><span>edit</span></a><span class="mw-editsection-divider"> | </span><a href="/ps3/edit/PlayStation_Eye?section=8" title="Edit section's source code: Firmware related files"><span>edit source</span></a><span class="mw-editsection-bracket">]</span></span></h1>
|
||||
<p>The camera is supported using the AVC library
|
||||
</p>
|
||||
<table class="wikitable sortable" style="font-size:small;">
|
||||
<caption>dev_flash\vsh\resource\avc\ec\cfg <div class="editsection noprint editlink plainlinks" align="right" style="float: right; margin-left: 5px;"><small>[</small><small><a target="_blank" rel="nofollow noreferrer noopener" class="external text" href="https://www.psdevwiki.com/ps3/edit/Template:Dev_flash%5Cvsh%5Cresource%5Cavc%5Cec%5Ccfg">edit</a></small><small>]</small></div>
|
||||
</caption>
|
||||
<tbody><tr>
|
||||
<th>Filename</th>
|
||||
<th>Version</th>
|
||||
<th>Size</th>
|
||||
<th>MD5</th>
|
||||
<th>Remarks
|
||||
</th></tr>
|
||||
<tr>
|
||||
<td>ps3eyetoy.cfg</td>
|
||||
<td style="text-align:center; font-size:small;">1.00-1.02</td>
|
||||
<td style="text-align:right; font-size:small;">5440</td>
|
||||
<td style="text-align:center; font-size:x-small;">e79f611e83108028a2f6b5a40cf83939</td>
|
||||
<td>content: <a target="_blank" rel="nofollow noreferrer noopener" class="external free" href="http://pastie.org/10227111">http://pastie.org/10227111</a>
|
||||
</td></tr>
|
||||
<tr>
|
||||
<td>micarray.cfg</td>
|
||||
<td style="text-align:center; font-size:small;">1.00-1.02</td>
|
||||
<td style="text-align:right; font-size:small;">5594</td>
|
||||
<td style="text-align:center; font-size:x-small;">299e8e01a3c50961bf79e8948ee51310</td>
|
||||
<td>content: <a target="_blank" rel="nofollow noreferrer noopener" class="external free" href="http://pastie.org/10227109">http://pastie.org/10227109</a>
|
||||
</td></tr>
|
||||
<tr>
|
||||
<td>array_es7.mod</td>
|
||||
<td style="text-align:center; font-size:small;">1.00-1.02</td>
|
||||
<td style="text-align:right; font-size:small;">26648</td>
|
||||
<td style="text-align:center; font-size:x-small;">bf6096054995f5c6d799883d7dfdf933</td>
|
||||
<td>
|
||||
</td></tr>
|
||||
</tbody></table><p><b>Note:</b> no variants and no longer present in versions above 1.02
|
||||
</p><p><br />
|
||||
</p>
|
||||
<table class="navbox" cellspacing="0" style=";"><tbody><tr><td style="padding:2px;"><table cellspacing="0" class="nowraplinks collapsible uncollapsed" style="width:100%;background:transparent;color:inherit;;"><tbody><tr><th scope="col" style=";" class="navbox-title" colspan="3"><div class="noprint plainlinks hlist navbar" style=""><ul><li class="nv-view"><a href="/ps3/Template:Peripherals" title="Template:Peripherals"><span style=";;background:none transparent;border:none;" title="View this template">v</span></a></li><li class="nv-edit"><a target="_blank" rel="nofollow noreferrer noopener" class="external text" href="https://www.psdevwiki.com/ps3/edit/Template:Peripherals"><span style=";;background:none transparent;border:none;" title="Edit this template">e</span></a></li></ul></div><div class="" style="font-size:110%;">
|
||||
<a href="/ps3/Peripherals" title="Peripherals">Peripherals</a></div></th></tr><tr style="height:2px;"><td></td></tr><tr><th scope="row" class="navbox-group" style=";padding:0em 0.3em;">Controllers</th><td style="text-align:left;border-left-width:2px;border-left-style:solid;width:100%;padding:0px;;;" class="navbox-list navbox-odd hlist"><div style="padding:0em 0.25em">
|
||||
<a href="/ps3/BD_Remote_Control" title="BD Remote Control">BD Remote Control</a> <span style="font-weight:bold;">·</span> <a href="/ps3/DualShock_3" title="DualShock 3">DualShock 3</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Keyboard" title="Keyboard">Keyboard</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Move_Motion_Controller" title="Move Motion Controller">Move Motion Controller</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Move_Navigation_Controller" title="Move Navigation Controller">Move Navigation Controller</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Move_Racing_Wheel" title="Move Racing Wheel">Move Racing Wheel</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Move_Sharp_Shooter" title="Move Sharp Shooter">Move Sharp Shooter</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Sixaxis" title="Sixaxis">Sixaxis</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Wireless_Keypad" title="Wireless Keypad">Wireless Keypad</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Charging_Station" title="Charging Station">Charging Station</a></div></td><td class="navbox-image" style="width:0%;padding:0px 0px 0px 2px;" rowspan="5"><div>
|
||||
<span typeof="mw:File"><span><img alt="" src="/ps3/images/thumb/b/b2/Peripherals.png/50px-Peripherals.png" decoding="async" width="50" height="41" class="mw-file-element" srcset="/ps3/images/thumb/b/b2/Peripherals.png/75px-Peripherals.png 1.5x, /ps3/images/thumb/b/b2/Peripherals.png/100px-Peripherals.png 2x" /></span></span></div></td></tr><tr style="height:2px"><td></td></tr><tr><th scope="row" class="navbox-group" style=";padding:0em 0.3em;">Cables</th><td style="text-align:left;border-left-width:2px;border-left-style:solid;width:100%;padding:0px;;;" class="navbox-list navbox-odd hlist"><div style="padding:0em 0.25em">
|
||||
<a href="/ps3/D-terminal_Cable" title="D-terminal Cable">D-terminal Cable</a> <span style="font-weight:bold;">·</span> <a href="/ps3/GbLAN_Cable" title="GbLAN Cable">GbLAN Cable</a> <span style="font-weight:bold;">·</span> <a href="/ps3/HDMI_Cable" title="HDMI Cable">HDMI Cable</a> <span style="font-weight:bold;">·</span> <a href="/ps3/TOSLINK_Cable" title="TOSLINK Cable">TOSLINK Cable</a> <span style="font-weight:bold;">·</span> <a href="/ps3/USB_2.0_Cable" title="USB 2.0 Cable">USB 2.0 Cable</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Component_AV_Cable" title="Component AV Cable">Component AV Cable</a> <span style="font-weight:bold;">·</span> <a href="/ps3/S_Video_Cable" title="S Video Cable">S Video Cable</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Power_Cord" title="Power Cord">Power Cord</a></div></td></tr><tr style="height:2px"><td></td></tr><tr><th scope="row" class="navbox-group" style=";padding:0em 0.3em;">Others</th><td style="text-align:left;border-left-width:2px;border-left-style:solid;width:100%;padding:0px;;;" class="navbox-list navbox-odd hlist"><div style="padding:0em 0.25em">
|
||||
<a href="/ps3/AC_Adaptor_USB" title="AC Adaptor USB">AC Adaptor USB</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Card_Adapter" title="Card Adapter">Card Adapter</a> <span style="font-weight:bold;">·</span> <a href="/ps3/HPRM" title="HPRM">HPRM</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Move_Charger" title="Move Charger">Move Charger</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Nasne" title="Nasne">Nasne</a> <span style="font-weight:bold;">·</span> <a class="mw-selflink selflink">PlayStation Eye</a> <span style="font-weight:bold;">·</span> <a href="/ps3/PlayTV" title="PlayTV">PlayTV</a> <span style="font-weight:bold;">·</span> <a href="/ps3/PocketStation" title="PocketStation">PocketStation</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Skateboard_Peripheral" title="Skateboard Peripheral">Skateboard Peripheral</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Surround_Sound_System" title="Surround Sound System">Surround Sound System</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Torne" class="mw-redirect" title="Torne">Torne</a> <span style="font-weight:bold;">·</span> <a href="/ps3/USB_NFC_Reader" title="USB NFC Reader">USB NFC Reader</a> <span style="font-weight:bold;">·</span> <a href="/ps3/VitaTV" title="VitaTV">VitaTV</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Wireless_Headset" title="Wireless Headset">Wireless Headset</a> <span style="font-weight:bold;">·</span> <a href="/ps3/Vertical_Stand" title="Vertical Stand">Vertical Stand</a></div></td></tr></tbody></table></td></tr></tbody></table>
|
||||
|
||||
<!--
|
||||
NewPP limit report
|
||||
Cached time: 20250813182613
|
||||
Cache expiry: 2592000
|
||||
Reduced expiry: false
|
||||
Complications: [show‐toc]
|
||||
CPU time usage: 0.081 seconds
|
||||
Real time usage: 0.092 seconds
|
||||
Preprocessor visited node count: 506/1000000
|
||||
Post‐expand include size: 18040/2097152 bytes
|
||||
Template argument size: 8946/2097152 bytes
|
||||
Highest expansion depth: 14/100
|
||||
Expensive parser function count: 0/100
|
||||
Unstrip recursion depth: 0/20
|
||||
Unstrip post‐expand size: 4083/5000000 bytes
|
||||
-->
|
||||
<!--
|
||||
Transclusion expansion time report (%,ms,calls,template)
|
||||
100.00% 32.549 1 -total
|
||||
71.57% 23.296 1 Template:Peripherals
|
||||
52.15% 16.976 1 Template:Navbox
|
||||
28.03% 9.124 1 Template:Dev_flash\vsh\resource\avc\ec\cfg
|
||||
10.30% 3.352 1 Template:Navbar
|
||||
9.27% 3.017 1 Template:Ed_right
|
||||
7.61% 2.476 3 Template:Transclude
|
||||
7.38% 2.403 30 Template:Dot
|
||||
-->
|
||||
|
||||
<!-- Saved in parser cache with key psdevwiki_ps3-wiki_:pcache:idhash:1522-0!canonical and timestamp 20250813182613 and revision id 74326. Rendering was triggered because: page-view
|
||||
-->
|
||||
</div>
|
||||
<div class="printfooter" data-nosnippet="">Retrieved from "<a dir="ltr" href="https://www.psdevwiki.com/ps3/index.php?title=PlayStation_Eye&oldid=74326">https://www.psdevwiki.com/ps3/index.php?title=PlayStation_Eye&oldid=74326</a>"</div></div>
|
||||
<div id="catlinks" class="catlinks" data-mw="interface"><div id="mw-normal-catlinks" class="mw-normal-catlinks"><a href="/ps3/Special:Categories" title="Special:Categories">Categories</a>: <ul><li><a href="/ps3/Category:Peripherals" title="Category:Peripherals">Peripherals</a></li><li><a href="/ps3/Category:Main" title="Category:Main">Main</a></li></ul></div></div>
|
||||
</div>
|
||||
</div>
|
||||
<div id='mw-data-after-content'>
|
||||
<div class="gas-bottom"><script async src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script><ins class="adsbygoogle gas-bottom-first"
|
||||
style="display:table; width:728px; height:90px; margin:1em auto; clear:both;"
|
||||
data-ad-client="ca-pub-9015332520988458"
|
||||
data-ad-slot="7897517726"></ins><script>
|
||||
(adsbygoogle = window.adsbygoogle || []).push({});
|
||||
</script></div><div class="gas-bottom"><script async src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script><ins class="adsbygoogle gas-bottom-first"
|
||||
style="display:table; width:728px; height:90px; margin:1em auto; clear:both;"
|
||||
data-ad-client="ca-pub-9015332520988458"
|
||||
data-ad-slot="1931323254"></ins><script>
|
||||
(adsbygoogle = window.adsbygoogle || []).push({});
|
||||
</script></div><div class="mw-cookiewarning-container"><div class="mw-cookiewarning-text"><span>Cookies help us deliver our services. By using our services, you agree to our use of cookies.</span></div><form method="POST"><div class='oo-ui-layout oo-ui-horizontalLayout'><span class='oo-ui-widget oo-ui-widget-enabled oo-ui-buttonElement oo-ui-buttonElement-framed oo-ui-labelElement oo-ui-flaggedElement-progressive oo-ui-buttonWidget'><a role='button' tabindex='0' href='https://www.psdevwiki.com/ps3/PS3_Developer_wiki:Privacy_policy' rel='nofollow' class='oo-ui-buttonElement-button'><span class='oo-ui-iconElement-icon oo-ui-iconElement-noIcon oo-ui-image-progressive'></span><span class='oo-ui-labelElement-label'>More information</span><span class='oo-ui-indicatorElement-indicator oo-ui-indicatorElement-noIndicator oo-ui-image-progressive'></span></a></span><span class='oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-buttonElement oo-ui-buttonElement-framed oo-ui-labelElement oo-ui-flaggedElement-primary oo-ui-flaggedElement-progressive oo-ui-buttonInputWidget'><button type='submit' tabindex='0' name='disablecookiewarning' value='OK' class='oo-ui-inputWidget-input oo-ui-buttonElement-button'><span class='oo-ui-iconElement-icon oo-ui-iconElement-noIcon oo-ui-image-invert'></span><span class='oo-ui-labelElement-label'>OK</span><span class='oo-ui-indicatorElement-indicator oo-ui-indicatorElement-noIndicator oo-ui-image-invert'></span></button></span></div></form></div>
|
||||
</div>
|
||||
|
||||
<div id="mw-navigation">
|
||||
<h2>Navigation menu</h2>
|
||||
<div id="mw-head">
|
||||
|
||||
<nav id="p-personal" class="mw-portlet mw-portlet-personal vector-user-menu-legacy vector-menu" aria-labelledby="p-personal-label" >
|
||||
<h3
|
||||
id="p-personal-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Personal tools</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list">
|
||||
|
||||
<li id="pt-anonuserpage" class="mw-list-item"><span title="The user page for the IP address you are editing as">Not logged in</span></li><li id="pt-anontalk" class="mw-list-item"><a href="/ps3/Special:MyTalk" title="Discussion about edits from this IP address [n]" accesskey="n"><span>Talk</span></a></li><li id="pt-darkmode" class="mw-list-item"><a href="#" class="ext-darkmode-link"><span>Dark mode (beta)</span></a></li><li id="pt-anoncontribs" class="mw-list-item"><a href="/ps3/Special:MyContributions" title="A list of edits made from this IP address [y]" accesskey="y"><span>Contributions</span></a></li><li id="pt-createaccount" class="mw-list-item"><a href="/ps3/index.php?title=Special:CreateAccount&returnto=PlayStation+Eye" title="You are encouraged to create an account and log in; however, it is not mandatory"><span>Create account</span></a></li><li id="pt-login" class="mw-list-item"><a href="/ps3/index.php?title=Special:UserLogin&returnto=PlayStation+Eye" title="You are encouraged to log in; however, it is not mandatory [o]" accesskey="o"><span>Log in</span></a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<div id="left-navigation">
|
||||
|
||||
<nav id="p-namespaces" class="mw-portlet mw-portlet-namespaces vector-menu-tabs vector-menu-tabs-legacy vector-menu" aria-labelledby="p-namespaces-label" >
|
||||
<h3
|
||||
id="p-namespaces-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Namespaces</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list">
|
||||
|
||||
<li id="ca-nstab-main" class="selected mw-list-item"><a href="/ps3/PlayStation_Eye" title="View the content page [c]" accesskey="c"><span>Page</span></a></li><li id="ca-talk" class="new mw-list-item"><a href="/ps3/edit/Talk:PlayStation_Eye?redlink=1" rel="discussion" class="new" title="Discussion about the content page (page does not exist) [t]" accesskey="t"><span>Discussion</span></a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
<nav id="p-variants" class="mw-portlet mw-portlet-variants emptyPortlet vector-menu-dropdown vector-menu" aria-labelledby="p-variants-label" >
|
||||
<input type="checkbox"
|
||||
id="p-variants-checkbox"
|
||||
role="button"
|
||||
aria-haspopup="true"
|
||||
data-event-name="ui.dropdown-p-variants"
|
||||
class="vector-menu-checkbox"
|
||||
aria-labelledby="p-variants-label"
|
||||
>
|
||||
<label
|
||||
id="p-variants-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">English</span>
|
||||
</label>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list">
|
||||
|
||||
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
</div>
|
||||
<div id="right-navigation">
|
||||
|
||||
<nav id="p-views" class="mw-portlet mw-portlet-views vector-menu-tabs vector-menu-tabs-legacy vector-menu" aria-labelledby="p-views-label" >
|
||||
<h3
|
||||
id="p-views-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Views</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list">
|
||||
|
||||
<li id="ca-view" class="selected mw-list-item"><a href="/ps3/PlayStation_Eye"><span>Read</span></a></li><li id="ca-ve-edit" class="mw-list-item"><a href="/ps3/index.php?title=PlayStation_Eye&veaction=edit" title="Edit this page [v]" accesskey="v"><span>Edit</span></a></li><li id="ca-edit" class="collapsible mw-list-item"><a href="/ps3/edit/PlayStation_Eye" title="Edit the source code of this page [e]" accesskey="e"><span>Edit source</span></a></li><li id="ca-history" class="mw-list-item"><a href="/ps3/history/PlayStation_Eye" title="Past revisions of this page [h]" accesskey="h"><span>View history</span></a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
<nav id="p-cactions" class="mw-portlet mw-portlet-cactions vector-menu-dropdown vector-menu" aria-labelledby="p-cactions-label" title="More options" >
|
||||
<input type="checkbox"
|
||||
id="p-cactions-checkbox"
|
||||
role="button"
|
||||
aria-haspopup="true"
|
||||
data-event-name="ui.dropdown-p-cactions"
|
||||
class="vector-menu-checkbox"
|
||||
aria-labelledby="p-cactions-label"
|
||||
>
|
||||
<label
|
||||
id="p-cactions-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">More</span>
|
||||
</label>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list">
|
||||
|
||||
<li id="ca-purge" class="mw-list-item"><a href="#"><span>Purge</span></a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
<div id="p-search" role="search" class="vector-search-box-vue vector-search-box-show-thumbnail vector-search-box-auto-expand-width vector-search-box">
|
||||
<h3 >Search</h3>
|
||||
<form action="/ps3/index.php" id="searchform" class="vector-search-box-form">
|
||||
<div id="simpleSearch"
|
||||
class="vector-search-box-inner"
|
||||
data-search-loc="header-navigation">
|
||||
<input class="vector-search-box-input"
|
||||
type="search" name="search" placeholder="Search PS3 Developer wiki" aria-label="Search PS3 Developer wiki" autocapitalize="sentences" title="Search PS3 Developer wiki [f]" accesskey="f" id="searchInput"
|
||||
>
|
||||
<input type="hidden" name="title" value="Special:Search">
|
||||
<input id="mw-searchButton"
|
||||
class="searchButton mw-fallbackSearchButton" type="submit" name="fulltext" title="Search the pages for this text" value="Search">
|
||||
<input id="searchButton"
|
||||
class="searchButton" type="submit" name="go" title="Go to a page with this exact name if it exists" value="Go">
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="mw-panel" class="vector-legacy-sidebar">
|
||||
<div id="p-logo" role="banner">
|
||||
<a class="mw-wiki-logo" href="/ps3/Main_Page"
|
||||
title="Visit the main page"></a>
|
||||
</div>
|
||||
|
||||
<nav id="p-PlayStation_Devwikis" class="mw-portlet mw-portlet-PlayStation_Devwikis vector-menu-portal portal vector-menu" aria-labelledby="p-PlayStation_Devwikis-label" >
|
||||
<h3
|
||||
id="p-PlayStation_Devwikis-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">PlayStation Devwikis</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list">
|
||||
|
||||
<li id="n-Home" class="mw-list-item"><a href="https://www.psdevwiki.com" rel="nofollow" target="_blank"><span>Home</span></a></li><li id="n-PS1-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/ps1/" rel="nofollow" target="_blank"><span>PS1 Dev Wiki</span></a></li><li id="n-PS2-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/ps2/" rel="nofollow" target="_blank"><span>PS2 Dev Wiki</span></a></li><li id="n-PS3-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/ps3/" rel="nofollow" target="_blank"><span>PS3 Dev Wiki</span></a></li><li id="n-PS4-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/ps4/" rel="nofollow" target="_blank"><span>PS4 Dev Wiki</span></a></li><li id="n-PS5-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/ps5/" rel="nofollow" target="_blank"><span>PS5 Dev Wiki</span></a></li><li id="n-PSP-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/psp/" rel="nofollow" target="_blank"><span>PSP Dev Wiki</span></a></li><li id="n-Vita-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/vita/" rel="nofollow" target="_blank"><span>Vita Dev Wiki</span></a></li><li id="n-PS-Classic-Dev-Wiki" class="mw-list-item"><a href="https://www.psdevwiki.com/psclassic" rel="nofollow" target="_blank"><span>PS Classic Dev Wiki</span></a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
<nav id="p-Other_Wikis" class="mw-portlet mw-portlet-Other_Wikis vector-menu-portal portal vector-menu" aria-labelledby="p-Other_Wikis-label" >
|
||||
<h3
|
||||
id="p-Other_Wikis-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Other Wikis</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list">
|
||||
|
||||
<li id="n-HENkaku-Dev-Wiki" class="mw-list-item"><a href="http://wiki.henkaku.xyz/" rel="nofollow" target="_blank"><span>HENkaku Dev Wiki</span></a></li><li id="n-ConsoleMods.org" class="mw-list-item"><a href="https://www.consolemods.org/wiki/Main_Page" rel="nofollow" target="_blank"><span>ConsoleMods.org</span></a></li><li id="n-TeamXLink-Wiki" class="mw-list-item"><a href="https://www.teamxlink.co.uk/wiki/Main_Page" rel="nofollow" target="_blank"><span>TeamXLink Wiki</span></a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<nav id="p-Discord" class="mw-portlet mw-portlet-Discord vector-menu-portal portal vector-menu" aria-labelledby="p-Discord-label" >
|
||||
<h3
|
||||
id="p-Discord-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Discord</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list">
|
||||
|
||||
<li id="n-PlayStation-Devwiki" class="mw-list-item"><a href="https://discord.gg/K7fjFt3" rel="nofollow" target="_blank"><span>PlayStation Devwiki</span></a></li><li id="n-ConsoleMods.org" class="mw-list-item"><a href="https://discord.gg/x5vEnkR4C8" rel="nofollow" target="_blank"><span>ConsoleMods.org</span></a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<nav id="p-navigation" class="mw-portlet mw-portlet-navigation vector-menu-portal portal vector-menu" aria-labelledby="p-navigation-label" >
|
||||
<h3
|
||||
id="p-navigation-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Navigation</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list">
|
||||
|
||||
<li id="n-mainpage-description" class="mw-list-item"><a href="/ps3/Main_Page" title="Visit the main page [z]" accesskey="z"><span>Main page</span></a></li><li id="n-portal" class="mw-list-item"><a href="/ps3/PS3_Developer_wiki:Community_portal" title="About the project, what you can do, where to find things"><span>Community portal</span></a></li><li id="n-currentevents" class="mw-list-item"><a href="/ps3/PS3_Developer_wiki:Current_events" title="Find background information on current events"><span>Current events</span></a></li><li id="n-recentchanges" class="mw-list-item"><a href="/ps3/Special:RecentChanges" title="A list of recent changes in the wiki [r]" accesskey="r"><span>Recent changes</span></a></li><li id="n-contributionscores" class="mw-list-item"><a href="/ps3/Special:ContributionScores"><span>Contribution scores</span></a></li><li id="n-randompage" class="mw-list-item"><a href="/ps3/Special:Random" title="Load a random page [x]" accesskey="x"><span>Random page</span></a></li><li id="n-All-Pages" class="mw-list-item"><a href="/ps3/Special:AllPages"><span>All Pages</span></a></li><li id="n-Staff-contacts" class="mw-list-item"><a href="https://www.psdevwiki.com/ps3/Staff_Contacts" rel="nofollow" target="_blank"><span>Staff contacts</span></a></li><li id="n-help" class="mw-list-item"><a href="https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents" target="_blank" title="The place to find out"><span>Help</span></a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<nav id="p-tb" class="mw-portlet mw-portlet-tb vector-menu-portal portal vector-menu" aria-labelledby="p-tb-label" >
|
||||
<h3
|
||||
id="p-tb-label"
|
||||
|
||||
class="vector-menu-heading "
|
||||
>
|
||||
<span class="vector-menu-heading-label">Tools</span>
|
||||
</h3>
|
||||
<div class="vector-menu-content">
|
||||
|
||||
<ul class="vector-menu-content-list">
|
||||
|
||||
<li id="t-whatlinkshere" class="mw-list-item"><a href="/ps3/Special:WhatLinksHere/PlayStation_Eye" title="A list of all wiki pages that link here [j]" accesskey="j"><span>What links here</span></a></li><li id="t-recentchangeslinked" class="mw-list-item"><a href="/ps3/Special:RecentChangesLinked/PlayStation_Eye" rel="nofollow" title="Recent changes in pages linked from this page [k]" accesskey="k"><span>Related changes</span></a></li><li id="t-specialpages" class="mw-list-item"><a href="/ps3/Special:SpecialPages" title="A list of all special pages [q]" accesskey="q"><span>Special pages</span></a></li><li id="t-print" class="mw-list-item"><a href="javascript:print();" rel="alternate" title="Printable version of this page [p]" accesskey="p"><span>Printable version</span></a></li><li id="t-permalink" class="mw-list-item"><a href="/ps3/index.php?title=PlayStation_Eye&oldid=74326" title="Permanent link to this revision of this page"><span>Permanent link</span></a></li><li id="t-info" class="mw-list-item"><a href="/ps3/index.php?title=PlayStation_Eye&action=info" title="More information about this page"><span>Page information</span></a></li>
|
||||
</ul>
|
||||
<div class="after-portlet after-portlet-tb"><script async src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
|
||||
<ins class="adsbygoogle"
|
||||
style="display:inline-block;width:160px;height:1000px;margin:20px 0 0 -20px;"
|
||||
data-ad-client="ca-pub-9015332520988458"
|
||||
data-ad-slot="1352290675"
|
||||
</ins>
|
||||
<script>
|
||||
(adsbygoogle = window.adsbygoogle || []).push({});
|
||||
</script></div>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
<footer id="footer" class="mw-footer" >
|
||||
<ul id="footer-info">
|
||||
<li id="footer-info-lastmod"> This page was last edited on 4 September 2024, at 13:21.</li>
|
||||
<li id="footer-info-copyright">Content is available under <a class="external" rel="nofollow" href="http://www.gnu.org/licenses/old-licenses/fdl-1.2.txt">GNU Free Documentation License 1.2</a> unless otherwise noted.</li>
|
||||
</ul>
|
||||
|
||||
<ul id="footer-places">
|
||||
<li id="footer-places-privacy"><a href="/ps3/PS3_Developer_wiki:Privacy_policy">Privacy policy</a></li>
|
||||
<li id="footer-places-about"><a href="/ps3/PS3_Developer_wiki:About">About PS3 Developer wiki</a></li>
|
||||
<li id="footer-places-disclaimers"><a href="/ps3/PS3_Developer_wiki:General_disclaimer">Disclaimers</a></li>
|
||||
</ul>
|
||||
|
||||
<ul id="footer-icons" class="noprint">
|
||||
<li id="footer-copyrightico"><a href="http://www.gnu.org/licenses/old-licenses/fdl-1.2.txt" class="cdx-button cdx-button--fake-button cdx-button--size-large cdx-button--fake-button--enabled" target="_blank"><img src="/ps3/resources/assets/licenses/gnu-fdl.png" alt="GNU Free Documentation License 1.2" width="88" height="31" loading="lazy"></a></li>
|
||||
<li id="footer-poweredbyico"><a href="https://www.mediawiki.org/" class="cdx-button cdx-button--fake-button cdx-button--size-large cdx-button--fake-button--enabled" target="_blank"><img src="/ps3/resources/assets/poweredby_mediawiki.svg" alt="Powered by MediaWiki" width="88" height="31" loading="lazy"></a></li>
|
||||
</ul>
|
||||
|
||||
</footer>
|
||||
|
||||
<script src="https://www.googletagmanager.com/gtag/js?id=G-0DGNXPL208" async=""></script><script>
|
||||
window.dataLayer = window.dataLayer || [];
|
||||
|
||||
function gtag(){dataLayer.push(arguments);}
|
||||
gtag('js', new Date());
|
||||
gtag('config', 'G-0DGNXPL208', {"anonymize_ip":true});
|
||||
</script>
|
||||
|
||||
<script>(RLQ=window.RLQ||[]).push(function(){mw.config.set({"wgBackendResponseTime":184,"wgPageParseReport":{"limitreport":{"cputime":"0.081","walltime":"0.092","ppvisitednodes":{"value":506,"limit":1000000},"postexpandincludesize":{"value":18040,"limit":2097152},"templateargumentsize":{"value":8946,"limit":2097152},"expansiondepth":{"value":14,"limit":100},"expensivefunctioncount":{"value":0,"limit":100},"unstrip-depth":{"value":0,"limit":20},"unstrip-size":{"value":4083,"limit":5000000},"timingprofile":["100.00% 32.549 1 -total"," 71.57% 23.296 1 Template:Peripherals"," 52.15% 16.976 1 Template:Navbox"," 28.03% 9.124 1 Template:Dev_flash\\vsh\\resource\\avc\\ec\\cfg"," 10.30% 3.352 1 Template:Navbar"," 9.27% 3.017 1 Template:Ed_right"," 7.61% 2.476 3 Template:Transclude"," 7.38% 2.403 30 Template:Dot"]},"cachereport":{"timestamp":"20250813182613","ttl":2592000,"transientcontent":false}}});});</script>
|
||||
<script>(function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9fdab11b5c366918',t:'MTc3OTEwNTUyNQ=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();</script></body>
|
||||
<!-- Cached 20250813182613 -->
|
||||
</html>
|
||||
194
research/visual-spatial-map/mirrors/psmoveapi-camera-docs.html
Normal file
194
research/visual-spatial-map/mirrors/psmoveapi-camera-docs.html
Normal file
|
|
@ -0,0 +1,194 @@
|
|||
<!DOCTYPE html>
|
||||
|
||||
<html lang="en" data-content_root="./">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
|
||||
<title>Tracking and Camera — PS Move API 4.0.12 documentation</title>
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/classic.css?v=2bf1fcf8" />
|
||||
|
||||
<script src="_static/documentation_options.js?v=2b25b4f2"></script>
|
||||
<script src="_static/doctools.js?v=fd6eb6e6"></script>
|
||||
<script src="_static/sphinx_highlight.js?v=6ffebe34"></script>
|
||||
|
||||
<link rel="index" title="Index" href="genindex.html" />
|
||||
<link rel="search" title="Search" href="search.html" />
|
||||
<link rel="next" title="Networking (Move Daemon)" href="moved.html" />
|
||||
<link rel="prev" title="Pairing the Controller to your PC" href="pairing.html" />
|
||||
<script async type="text/javascript" src="/_/static/javascript/readthedocs-addons.js"></script><meta name="readthedocs-project-slug" content="psmoveapi" /><meta name="readthedocs-version-slug" content="latest" /><meta name="readthedocs-resolver-filename" content="/camera.html" /><meta name="readthedocs-http-status" content="200" /></head><body>
|
||||
<div class="related" role="navigation" aria-label="Related">
|
||||
<h3>Navigation</h3>
|
||||
<ul>
|
||||
<li class="right" style="margin-right: 10px">
|
||||
<a href="genindex.html" title="General Index"
|
||||
accesskey="I">index</a></li>
|
||||
<li class="right" >
|
||||
<a href="moved.html" title="Networking (Move Daemon)"
|
||||
accesskey="N">next</a> |</li>
|
||||
<li class="right" >
|
||||
<a href="pairing.html" title="Pairing the Controller to your PC"
|
||||
accesskey="P">previous</a> |</li>
|
||||
<li class="nav-item nav-item-0"><a href="index.html">PS Move API 4.0.12 documentation</a> »</li>
|
||||
<li class="nav-item nav-item-this"><a href="">Tracking and Camera</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<div class="document">
|
||||
<div class="documentwrapper">
|
||||
<div class="bodywrapper">
|
||||
<div class="body" role="main">
|
||||
|
||||
<section id="tracking-and-camera">
|
||||
<h1>Tracking and Camera<a class="headerlink" href="#tracking-and-camera" title="Link to this heading">¶</a></h1>
|
||||
<p>This page contains information about the camera integration, especially
|
||||
as it relates to tracking and the different variants for different OSes.</p>
|
||||
<section id="ps3-camera">
|
||||
<h2>PS3 Camera<a class="headerlink" href="#ps3-camera" title="Link to this heading">¶</a></h2>
|
||||
<p>This camera is always supported on Linux, and supported on macOS and Windows
|
||||
when using the PS3EYEDriver camera control driver.</p>
|
||||
<p>On Linux, the kernel includes built-in support for the camera using the
|
||||
<code class="docutils literal notranslate"><span class="pre">ov534</span></code> driver, so the V4L2 camera control driver also support the PS3 Eye.</p>
|
||||
<p>The camera needs a USB 2.0 port.</p>
|
||||
<p>On Windows, you need to install <a class="reference external" href="https://zadig.akeo.ie/">Zadig</a> (WinUSB driver) for the
|
||||
“USBCamera-B4.09.24.1” (or similar) device so that the PS3EYEDriver
|
||||
can access the camera using <code class="docutils literal notranslate"><span class="pre">libusb</span></code>. This has been tested with the
|
||||
“WinUSB (v6.1.7600.16385)” driver and Zadig 2.7.</p>
|
||||
<ul class="simple">
|
||||
<li><p>Tested models: SLEH-00448</p></li>
|
||||
<li><p>No firmware upload necessary</p></li>
|
||||
</ul>
|
||||
<p>Supported resolutions:</p>
|
||||
<ul class="simple">
|
||||
<li><p><strong>640x480 @ 60 Hz</strong> (default)</p></li>
|
||||
<li><p>320x240 @ 187 Hz</p></li>
|
||||
</ul>
|
||||
</section>
|
||||
<section id="ps4-camera">
|
||||
<h2>PS4 Camera<a class="headerlink" href="#ps4-camera" title="Link to this heading">¶</a></h2>
|
||||
<p>This camera is currently supported on Linux when using the V4L2 camera control
|
||||
driver, and macOS when using the AVFoundation camera control driver.</p>
|
||||
<p>You need to upload a firmware binary blob using the <code class="docutils literal notranslate"><span class="pre">psmove</span> <span class="pre">camera-firmware</span></code> command.</p>
|
||||
<p>The camera needs a USB 3.0 port.</p>
|
||||
<p>If you have a PSVR kit, you can order the <a class="reference external" href="https://camera-adaptor.support.playstation.com/">Camera Adaptor</a> (to convert
|
||||
PS4 AUX to USB 3.0) from Sony for free. You only need to enter the serial
|
||||
number of your PSVR kit during the ordering process, no need for a PS5.
|
||||
If you only have the camera but no PSVR, you can look to buy the
|
||||
adaptor online. The official Sony one is CFI-ZAA1.</p>
|
||||
<p>The official PSVR camera adaptor for PS5 (CFI-ZAA1) will be detected
|
||||
automatically. If you are using a homemade PS4 AUX-to-USB adapter, use
|
||||
<code class="docutils literal notranslate"><span class="pre">psmove</span> <span class="pre">camera-firmware</span> <span class="pre">--force-ps4</span></code> to detect any OV580 in boot mode
|
||||
as PS4 (otherwise it’s assumed that the camera is a PS5 camera, as there
|
||||
is no way yet to distinguish the PS4 and PS5 cameras from the USB
|
||||
descriptor in boot mode only, which is probably why the CFI-ZAA1 appears
|
||||
as USB parent device during enumeration).</p>
|
||||
<p>After the firmware upload the camera is supported using
|
||||
the <code class="docutils literal notranslate"><span class="pre">uvcvideo</span></code> kernel driver in OpenCV/V4L2 (Linux)
|
||||
and <code class="docutils literal notranslate"><span class="pre">UVCService</span></code> on OpenCV/AVFoundation (macOS).</p>
|
||||
<ul class="simple">
|
||||
<li><p>Tested models: CUH-ZEY2 (“round model v2”)</p></li>
|
||||
<li><p>Tested firmware SHA-1: <code class="docutils literal notranslate"><span class="pre">fe86162309518a0ffe267075a2fcf728c5856b3e</span></code></p></li>
|
||||
<li><p>Firmware search path: <code class="docutils literal notranslate"><span class="pre">/etc/psmoveapi/camera-firmware-ps4.bin</span></code>, <code class="docutils literal notranslate"><span class="pre">$HOME/.psmoveapi/camera-firmware-ps4.bin</span></code></p></li>
|
||||
</ul>
|
||||
<p>Supported resolutions:</p>
|
||||
<ul class="simple">
|
||||
<li><p><strong>1280x800 @ 60 Hz</strong> (default)</p></li>
|
||||
<li><p>640x400 @ 120 Hz</p></li>
|
||||
<li><p>320x192 @ 240 Hz</p></li>
|
||||
</ul>
|
||||
</section>
|
||||
<section id="ps5-camera">
|
||||
<h2>PS5 Camera<a class="headerlink" href="#ps5-camera" title="Link to this heading">¶</a></h2>
|
||||
<p>This camera is currently supported on Linux when using the V4L2 camera control
|
||||
driver, and macOS when using the AVFoundation camera control driver.</p>
|
||||
<p>You need to upload a firmware binary blob using the <code class="docutils literal notranslate"><span class="pre">psmove</span> <span class="pre">camera-firmware</span></code> command.</p>
|
||||
<p>The camera needs a USB 3.0 port.</p>
|
||||
<p>After the firmware upload the camera is supported using
|
||||
the <code class="docutils literal notranslate"><span class="pre">uvcvideo</span></code> kernel driver in OpenCV/V4L2 (Linux)
|
||||
and <code class="docutils literal notranslate"><span class="pre">UVCService</span></code> on OpenCV/AVFoundation (macOS).</p>
|
||||
<ul class="simple">
|
||||
<li><p>Tested models: CFI-ZEY1</p></li>
|
||||
<li><p>Tested firmware SHA-1: <code class="docutils literal notranslate"><span class="pre">0fa4da31a12b662a9a80abc8b84932770df8f7e1</span></code></p></li>
|
||||
<li><p>Firmware search path: <code class="docutils literal notranslate"><span class="pre">/etc/psmoveapi/camera-firmware-ps5.bin</span></code>, <code class="docutils literal notranslate"><span class="pre">$HOME/.psmoveapi/camera-firmware-ps5.bin</span></code></p></li>
|
||||
</ul>
|
||||
<p>Supported resolutions:</p>
|
||||
<ul class="simple">
|
||||
<li><p>1920x1080 @ 30 Hz</p></li>
|
||||
<li><p><strong>1280x800 @ 60 Hz</strong> (default)</p></li>
|
||||
<li><p>960x520 @ 60 Hz</p></li>
|
||||
<li><p>640x376 @ 120 Hz</p></li>
|
||||
<li><p>448x256 @ 120 Hz</p></li>
|
||||
<li><p>320x184 @ 240 Hz</p></li>
|
||||
</ul>
|
||||
</section>
|
||||
</section>
|
||||
|
||||
|
||||
<div class="clearer"></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="sphinxsidebar" role="navigation" aria-label="Main">
|
||||
<div class="sphinxsidebarwrapper">
|
||||
<h3><a href="index.html">Table of Contents</a></h3>
|
||||
<ul class="current">
|
||||
<li class="toctree-l1"><a class="reference internal" href="build.html">Building from Source</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="pairing.html">Pairing</a></li>
|
||||
<li class="toctree-l1 current"><a class="current reference internal" href="#">Camera</a><ul>
|
||||
<li class="toctree-l2"><a class="reference internal" href="#ps3-camera">PS3 Camera</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="#ps4-camera">PS4 Camera</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="#ps5-camera">PS5 Camera</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="moved.html">Network (Move Daemon)</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="config.html">Configuration</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="navcon.html">Navigation Controller</a></li>
|
||||
</ul>
|
||||
|
||||
<div>
|
||||
<h4>Previous topic</h4>
|
||||
<p class="topless"><a href="pairing.html"
|
||||
title="previous chapter">Pairing the Controller to your PC</a></p>
|
||||
</div>
|
||||
<div>
|
||||
<h4>Next topic</h4>
|
||||
<p class="topless"><a href="moved.html"
|
||||
title="next chapter">Networking (Move Daemon)</a></p>
|
||||
</div>
|
||||
<search id="searchbox" style="display: none" role="search">
|
||||
<h3 id="searchlabel">Quick search</h3>
|
||||
<div class="searchformwrapper">
|
||||
<form class="search" action="search.html" method="get">
|
||||
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
|
||||
<input type="submit" value="Go" />
|
||||
</form>
|
||||
</div>
|
||||
</search>
|
||||
<script>document.getElementById('searchbox').style.display = "block"</script>
|
||||
</div>
|
||||
</div>
|
||||
<div class="clearer"></div>
|
||||
</div>
|
||||
<div class="related" role="navigation" aria-label="Related">
|
||||
<h3>Navigation</h3>
|
||||
<ul>
|
||||
<li class="right" style="margin-right: 10px">
|
||||
<a href="genindex.html" title="General Index"
|
||||
>index</a></li>
|
||||
<li class="right" >
|
||||
<a href="moved.html" title="Networking (Move Daemon)"
|
||||
>next</a> |</li>
|
||||
<li class="right" >
|
||||
<a href="pairing.html" title="Pairing the Controller to your PC"
|
||||
>previous</a> |</li>
|
||||
<li class="nav-item nav-item-0"><a href="index.html">PS Move API 4.0.12 documentation</a> »</li>
|
||||
<li class="nav-item nav-item-this"><a href="">Tracking and Camera</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="footer" role="contentinfo">
|
||||
© Copyright 2020, Thomas Perl.
|
||||
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 9.1.0.
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,325 @@
|
|||
<html>
|
||||
|
||||
<head>
|
||||
<title> PSEye Software Setup (Windows) - psmoveservice/PSMoveService GitHub Wiki </title>
|
||||
<style>
|
||||
#main {
|
||||
max-width: 38rem;
|
||||
margin: auto;
|
||||
overflow-wrap: break-word;
|
||||
min-width: 0;
|
||||
background-color: white;
|
||||
}
|
||||
|
||||
body {
|
||||
background-image: url('/callToAction.svg');
|
||||
}
|
||||
|
||||
img {
|
||||
max-width: 100%;
|
||||
height: auto;
|
||||
}
|
||||
|
||||
#header_button {
|
||||
position: sticky;
|
||||
top: 0;
|
||||
padding: 10px;
|
||||
border-style: ridge;
|
||||
border-color: lightgrey;
|
||||
border-width: thin;
|
||||
color: red;
|
||||
font-weight: bold;
|
||||
background-color: honeydew;
|
||||
font-size: large;
|
||||
}
|
||||
|
||||
#header_button p {
|
||||
margin-bottom: 0;
|
||||
margin-top: 10px;
|
||||
}
|
||||
|
||||
#header_info {
|
||||
padding: 10px;
|
||||
border-style: ridge;
|
||||
border-color: lightgrey;
|
||||
border-width: thin;
|
||||
color: red;
|
||||
font-weight: bold;
|
||||
background-color: honeydew;
|
||||
font-size: large;
|
||||
}
|
||||
|
||||
#content,
|
||||
#index {
|
||||
padding-left: 10px;
|
||||
padding-right: 10px;
|
||||
}
|
||||
|
||||
#footer_info {
|
||||
position: sticky;
|
||||
bottom: 0;
|
||||
padding: 10px;
|
||||
border-style: ridge;
|
||||
border-color: lightgrey;
|
||||
border-width: thin;
|
||||
color: red;
|
||||
font-weight: bold;
|
||||
background-color: honeydew;
|
||||
}
|
||||
|
||||
#footer_info .blocks {
|
||||
background: black;
|
||||
color: white;
|
||||
/* outline: 1px solid blue; */
|
||||
padding: .1em .2em;
|
||||
}
|
||||
|
||||
/* Hide useless-parts of transcluded HTML */
|
||||
.width-full.input-group {
|
||||
display: none;
|
||||
}
|
||||
|
||||
h5:nth-last-child(2) {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.Box-row.wiki-more-pages-link {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.filter-bar {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.Box-title>.octicon {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.anchor {
|
||||
display: none;
|
||||
}
|
||||
|
||||
pre {
|
||||
overflow: auto;
|
||||
}
|
||||
|
||||
#main {
|
||||
word-wrap: break-word;
|
||||
}
|
||||
|
||||
.Counter {
|
||||
background-color: lightgray;
|
||||
border-radius: 20px;
|
||||
padding: 0 10px;
|
||||
}
|
||||
|
||||
table {
|
||||
max-width: 100%;
|
||||
overflow: auto;
|
||||
width: 100%;
|
||||
display: block;
|
||||
}
|
||||
|
||||
.js-wiki-sidebar-toc-toggle-chevron-button {
|
||||
display: none;
|
||||
}
|
||||
|
||||
a.visit_url_original {
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
line-height: 1.2em; /* adjust as needed */
|
||||
max-height: 1.2em; /* should match line-height */
|
||||
display: block;
|
||||
}
|
||||
|
||||
/* White background for span */
|
||||
/* Rounded corners */
|
||||
/* Black border */
|
||||
.visit_url_pillbar {
|
||||
background-color: white;
|
||||
border-radius: 10px;
|
||||
padding: 5px;
|
||||
margin: 5px;
|
||||
display: block;
|
||||
border: 1px solid black;
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
|
||||
/* Broken Images */
|
||||
img:before {
|
||||
content: "⚠️ MISSING OR BROKEN IMAGE RENDERING ERROR: PLEASE VIEW ORIGINAL PAGE ON GITHUB.COM ⚠️";
|
||||
display: block;
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: "color-emoji";
|
||||
src: local("Apple Color Emoji"),
|
||||
local("Segoe UI Emoji"),
|
||||
local("Segoe UI Symbol"),
|
||||
local("Noto Color Emoji");
|
||||
}
|
||||
|
||||
.emoji {
|
||||
font-family: "color-emoji";
|
||||
}
|
||||
|
||||
table,
|
||||
th,
|
||||
td {
|
||||
border: 1px solid black;
|
||||
}
|
||||
|
||||
.visit_url_link {
|
||||
font-size: xx-large;
|
||||
}
|
||||
|
||||
.why_exist {
|
||||
position: relative;
|
||||
text-align: center;
|
||||
/* italics */
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
</style>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<!-- Not used, did not want to risk non-indexing. -->
|
||||
<!-- <meta property="og:url" content="https://github.com/psmoveservice/PSMoveService/wiki/PSEye-Software-Setup-%28Windows%29"> -->
|
||||
<!-- Did not want to risk contaminating content to be indexed. -->
|
||||
<!-- <meta property="og:description" content="not used"> -->
|
||||
<meta property="og:type" content="website">
|
||||
<meta property="og:title" content="PSEye Software Setup (Windows) - psmoveservice/PSMoveService GitHub Wiki">
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<div id="main">
|
||||
<nav id="header_button">
|
||||
<p>Content mirrored for search engine indexing from:</p>
|
||||
<div>
|
||||
<span class="visit_url_pillbar">
|
||||
<a href="https://github.com/psmoveservice/PSMoveService/wiki/PSEye-Software-Setup-%28Windows%29" class="visit_url_original">https://github.com/psmoveservice/PSMoveService/wiki/PSEye-Software-Setup-%28Windows%29</a>
|
||||
</span>
|
||||
|
||||
</div>
|
||||
<div class="why_exist">
|
||||
<a href="/">Why does this service exist?</a>
|
||||
</div>
|
||||
</nav>
|
||||
<aside id="header_info">
|
||||
<!-- Last Modified Date added around here from redirect checker-->
|
||||
<p>📅 Last Modified: Thu, 08 Aug 2019 14:42:55 GMT</p></aside>
|
||||
<article id="content">
|
||||
<h1>PSEye Software Setup (Windows) - psmoveservice/PSMoveService GitHub Wiki</h1>
|
||||
<div id="mirrored"> <h1><a href="#overview" aria-hidden="true" class="anchor" id="overview"></a>Overview</h1>
|
||||
<p>There are many different ways to get the PS Eye working on a Windows PC. This is pretty confusing, so expect to be confused.</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Driver</th>
|
||||
<th>Cost</th>
|
||||
<th>Notes</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><strong>IPISoft PS3EYEDriver</strong></td>
|
||||
<td>Free</td>
|
||||
<td>Supports multiple cameras. Applications like ex. skype cant see the cameras. The most easy PSEYE driver setup !</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>PS3EYEDriver</strong></td>
|
||||
<td>Free</td>
|
||||
<td>Wraps libusb. Open source and continues to be worked on so sometimes can be flaky. Supports as many cameras as there is bandwidth for. Instructions are below.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>CL Eye Platform Driver</strong></td>
|
||||
<td>$2.99 USD</td>
|
||||
<td>Exposes the camera as a DirectShow object so can be used easily by this and other software (e.g. Skype, I think, no guarantees). Only supports one camera.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>CL Eye Platform SDK</strong></td>
|
||||
<td>$4.99 USD</td>
|
||||
<td>Supports multiple cameras. Only works if you use your own (non-redistributable, developer version) CLEyeMulticam.dll. Delete the dll that comes with PSMoveService to use the DLL that gets installed with the SDK.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>CL Eye redistributable + registered camera</strong></td>
|
||||
<td>$?? USD * n_cameras</td>
|
||||
<td>Apparently, after you register your cameras, the redistributable CLEyeMulticam.dll that we supply works. I don't think I ever correctly registered my camera so I don't know if it actually works.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>PS3EyeDirectShow</strong></td>
|
||||
<td>Free</td>
|
||||
<td>Exposes the camera as a DirectShow object so can be used easily by this and other software (e.g. Skype, I think, no guarantees). <a href="https://github.com/jkevin/PS3EyeDirectShow" rel="nofollow ugc">https://github.com/jkevin/PS3EyeDirectShow</a></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<h1><a href="#setup-the-camera-drivers" aria-hidden="true" class="anchor" id="setup-the-camera-drivers"></a>Setup the camera drivers</h1>
|
||||
<h2><a href="#option-a-ipisoft-pseye-driver-installation" aria-hidden="true" class="anchor" id="option-a-ipisoft-pseye-driver-installation"></a>Option A) IPIsoft PSEYE Driver Installation</h2>
|
||||
<p><strong>NOTE: If you have already install libusb based drivers, please uninstall those drivers first in Windows device manager</strong></p>
|
||||
<p>This installation option is very simple. Just download the driver for your Windows OS architecture and all PSEYE Cameras should be recognized. IPISoft is software for Motion Capture Full Body system. This PS3Eye driver was not originally intended to be used with PSMoveService (opensource software), though it does appear to work. Therefore, please do not spam the support of IPISoft with questions about this driver. Use at your own risk.</p>
|
||||
<p>Downloads:</p>
|
||||
<p>64-bit OS:</p>
|
||||
<p><a href="http://files.ipisoft.com/drivers/PlayStation3_Eye_iPi-x64.msi" rel="nofollow ugc">http://files.ipisoft.com/drivers/PlayStation3_Eye_iPi-x64.msi</a></p>
|
||||
<p>32-bit OS:</p>
|
||||
<p><a href="http://files.ipisoft.com/drivers/PlayStation3_Eye_iPi-x86.msi" rel="nofollow ugc">http://files.ipisoft.com/drivers/PlayStation3_Eye_iPi-x86.msi</a></p>
|
||||
<p>Result in 'Device Manager', tested successfully in Windows 10 x64:</p>
|
||||
<p><img src="http://i.imgur.com/4Kdyp96.jpg" alt="" /></p>
|
||||
<p>Source info:
|
||||
<a href="http://docs.ipisoft.com/User_Guide_for_Multiple_PS_Eye_Cameras_Configuration" rel="nofollow ugc">http://docs.ipisoft.com/User_Guide_for_Multiple_PS_Eye_Cameras_Configuration</a></p>
|
||||
<h2><a href="#option-b-libusb-pseye-driver-installation" aria-hidden="true" class="anchor" id="option-b-libusb-pseye-driver-installation"></a>Option B) Libusb PSEYE Driver Installation</h2>
|
||||
<p>The instructions below apply to our customized version of the <a href="https://github.com/HipsterSloth/PS3EYEDriver/tree/psmoveservice" rel="nofollow ugc">PS3EYEDriver wrapper</a> around <a href="http://libusb.info/" rel="nofollow ugc">libusb</a>.</p>
|
||||
<h3><a href="#libusb-driver-pre-install" aria-hidden="true" class="anchor" id="libusb-driver-pre-install"></a>Libusb Driver Pre-Install</h3>
|
||||
<p>Download UsbDeview from <a href="http://www.nirsoft.net/utils/usbdeview-x64.zip" rel="nofollow ugc">Nirsoft.net</a>. This is a handy utility for showing all connected usb devices attached to your system and what drivers they are running. Once downloaded, launch <code>usbdeview.exe</code>.</p>
|
||||
<ol>
|
||||
<li>After you have plugged in the PS Eye camera you should see entries for the camera and the microphone of the ps3 eye.
|
||||
<ul>
|
||||
<li>The cameras take a lot of USB bandwidth. This is especially a concern if using more than one camera. We have some tips <a href="https://github.com/cboulay/PSMoveService/wiki/Troubleshooting-(Windows)#general-weird-camera-behavior-clean-camera-driver-reinstall" rel="nofollow ugc">here</a> on how to manage your camera-connections. Also see <a href="https://www.oculus.com/blog/oculus-roomscale-balancing-bandwidth-on-usb/" rel="nofollow ugc">what Oculus has to say</a> on the matter.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>Under the "service name" column for the camera you should see "usbccgp" for the camera device and "usbaudio" for the audio device.
|
||||
a. If you don't see "usbccgp" for the camera, right-click on the entry with the purple dot and select "uninstall selected devices", after which, unplug the ps3eye camera, and then plug it back in.</li>
|
||||
<li>Close usbdeview</li>
|
||||
</ol>
|
||||
<p><a href="/m/psmoveservice/PSMoveService/wiki/images/USBCameraPreInstall.jpg">images/USBCameraPreInstall.jpg</a></p>
|
||||
<h3><a href="#libusb-driver-install" aria-hidden="true" class="anchor" id="libusb-driver-install"></a>Libusb Driver Install</h3>
|
||||
<p>Download the Zadig usb utility from <a href="http://zadig.akeo.ie/" rel="nofollow ugc">here</a>. This is another handy usb utility for installing custom usb drivers. In our case we're going to use it to install the libusb driver over the top of the default Windows driver for the PS Eye camera. Once downloaded, Launch <code>zadig_2.2.exe</code>.</p>
|
||||
<ol>
|
||||
<li>From the options menu launch select "List all devices"</li>
|
||||
<li>From the device list drop down select "USB Camera-B4.09.24.1 (Interface 0)"</li>
|
||||
<li>From the driver drop down select "libusb-win32 (v 1.2.6.0)"</li>
|
||||
<li>Click the Install Driver option</li>
|
||||
<li>There should be a few second delay and then it should say "driver installed successfully"</li>
|
||||
</ol>
|
||||
<p>Note: Computer reboot might be require to complete installation.
|
||||
Note 2: Having an error message from the timeout expiration might not mean the driver was not install properly.</p>
|
||||
<p><a href="/m/psmoveservice/PSMoveService/wiki/images/ZadigInstall.jpg">images/ZadigInstall.jpg</a></p>
|
||||
<h3><a href="#libusb-driver-post-install" aria-hidden="true" class="anchor" id="libusb-driver-post-install"></a>Libusb Driver Post-Install</h3>
|
||||
<ol>
|
||||
<li>Launch <code>usbdeview.exe</code> again.</li>
|
||||
<li>You should now see an entry in the "service name" column for the PS3Eye camera that says "libusb0"
|
||||
a. If you don't see "libusb0" for the camera, right-click on the entry with the purple dot and select "uninstall selected devices", then unplug the ps3eye camera, then plug it back in, and try reinstalling the lib usb driver.</li>
|
||||
<li>Close usbdeview</li>
|
||||
</ol>
|
||||
<p><a href="/m/psmoveservice/PSMoveService/wiki/images/USBCameraPostInstall.jpg">images/USBCameraPostInstall.jpg</a></p>
|
||||
<h2><a href="#testing" aria-hidden="true" class="anchor" id="testing"></a>Testing</h2>
|
||||
<p>Now that the camera drivers are installed you should be able to run <code>test_camera.exe</code> and see a video feed (640x480/60Hz) for each camera you have plugged in (up to 3 cameras max).</p>
|
||||
<p>If <code>test_camera.exe</code> closes immediately, or the video feed is stuck on a solid color, don't panic. This is sadly a common occurrence, especially on a initial setup. It's quite likely that you have to <a href="https://github.com/cboulay/PSMoveService/wiki/Troubleshooting-%28Windows%29#cameras-arent-showing-up-using-libusb-driver" rel="nofollow ugc">do some driver shenanigans in zadig</a> to make this work.</p>
|
||||
<p>If that worked the first time, please take a moment to celebrate your good fortune. Then launch <code>PSMoveService.exe</code>. You should see messages about any attached cameras getting opened in the log window.</p>
|
||||
<p>Then launch the <code>PSMoveConfigTool.exe</code> and open the "Tracker Settings" and "Test Video Feed" sub menu. If your camera is set up properly then this should show you the images from the camera at 640x480/60Hz. While you have the video feed up, it's also a good time to grab the move controller and see if the frame encompasses enough of your intended interaction area with the controller.</p>
|
||||
<p>Also when the PS Eye camera is recording you should see the red LED turn on. Sometimes this light will get stuck on in the event of an application crash. Most of the time it won't matter, but you can reset the camera simply by unplugging it and then plugging it back in again.</p>
|
||||
<p><a href="/m/psmoveservice/PSMoveService/wiki/images/Recording.jpg">images/Recording.jpg</a></p>
|
||||
<h1><a href="#proceed-to-the-next-step" aria-hidden="true" class="anchor" id="proceed-to-the-next-step"></a>Proceed to the next step</h1>
|
||||
<p><a href="/m/psmoveservice/PSMoveService/wiki/PSMove-Bluetooth-Pairing">PSMove-Bluetooth-Pairing</a></p>
|
||||
</div>
|
||||
</article>
|
||||
<nav id="index">
|
||||
<a href="/m/psmoveservice/PSMoveService/wiki_index">🗂️ Page Index for this GitHub Wiki</a>
|
||||
</nav>
|
||||
</div>
|
||||
<script defer src="https://static.cloudflareinsights.com/beacon.min.js/v833ccba57c9e4d2798f2e76cebdd09a11778172276447" integrity="sha512-57MDmcccJXYtNnH+ZiBwzC4jb2rvgVCEokYN+L/nLlmO8rfYT/gIpW2A569iJ/3b+0UEasghjuZH/ma3wIs/EQ==" data-cf-beacon='{"version":"2024.11.0","token":"4645764ca8384bfc86034b71f1dafd76","r":1,"server_timing":{"name":{"cfCacheStatus":true,"cfEdge":true,"cfExtPri":true,"cfL4":true,"cfOrigin":true,"cfSpeedBrain":true},"location_startswith":null}}' crossorigin="anonymous"></script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
|
@ -0,0 +1,279 @@
|
|||
<html>
|
||||
|
||||
<head>
|
||||
<title> Setting up the PlayStation 3 Eye Camera - Timocop/PSMoveServiceEx GitHub Wiki </title>
|
||||
<style>
|
||||
#main {
|
||||
max-width: 38rem;
|
||||
margin: auto;
|
||||
overflow-wrap: break-word;
|
||||
min-width: 0;
|
||||
background-color: white;
|
||||
}
|
||||
|
||||
body {
|
||||
background-image: url('/callToAction.svg');
|
||||
}
|
||||
|
||||
img {
|
||||
max-width: 100%;
|
||||
height: auto;
|
||||
}
|
||||
|
||||
#header_button {
|
||||
position: sticky;
|
||||
top: 0;
|
||||
padding: 10px;
|
||||
border-style: ridge;
|
||||
border-color: lightgrey;
|
||||
border-width: thin;
|
||||
color: red;
|
||||
font-weight: bold;
|
||||
background-color: honeydew;
|
||||
font-size: large;
|
||||
}
|
||||
|
||||
#header_button p {
|
||||
margin-bottom: 0;
|
||||
margin-top: 10px;
|
||||
}
|
||||
|
||||
#header_info {
|
||||
padding: 10px;
|
||||
border-style: ridge;
|
||||
border-color: lightgrey;
|
||||
border-width: thin;
|
||||
color: red;
|
||||
font-weight: bold;
|
||||
background-color: honeydew;
|
||||
font-size: large;
|
||||
}
|
||||
|
||||
#content,
|
||||
#index {
|
||||
padding-left: 10px;
|
||||
padding-right: 10px;
|
||||
}
|
||||
|
||||
#footer_info {
|
||||
position: sticky;
|
||||
bottom: 0;
|
||||
padding: 10px;
|
||||
border-style: ridge;
|
||||
border-color: lightgrey;
|
||||
border-width: thin;
|
||||
color: red;
|
||||
font-weight: bold;
|
||||
background-color: honeydew;
|
||||
}
|
||||
|
||||
#footer_info .blocks {
|
||||
background: black;
|
||||
color: white;
|
||||
/* outline: 1px solid blue; */
|
||||
padding: .1em .2em;
|
||||
}
|
||||
|
||||
/* Hide useless-parts of transcluded HTML */
|
||||
.width-full.input-group {
|
||||
display: none;
|
||||
}
|
||||
|
||||
h5:nth-last-child(2) {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.Box-row.wiki-more-pages-link {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.filter-bar {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.Box-title>.octicon {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.anchor {
|
||||
display: none;
|
||||
}
|
||||
|
||||
pre {
|
||||
overflow: auto;
|
||||
}
|
||||
|
||||
#main {
|
||||
word-wrap: break-word;
|
||||
}
|
||||
|
||||
.Counter {
|
||||
background-color: lightgray;
|
||||
border-radius: 20px;
|
||||
padding: 0 10px;
|
||||
}
|
||||
|
||||
table {
|
||||
max-width: 100%;
|
||||
overflow: auto;
|
||||
width: 100%;
|
||||
display: block;
|
||||
}
|
||||
|
||||
.js-wiki-sidebar-toc-toggle-chevron-button {
|
||||
display: none;
|
||||
}
|
||||
|
||||
a.visit_url_original {
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
line-height: 1.2em; /* adjust as needed */
|
||||
max-height: 1.2em; /* should match line-height */
|
||||
display: block;
|
||||
}
|
||||
|
||||
/* White background for span */
|
||||
/* Rounded corners */
|
||||
/* Black border */
|
||||
.visit_url_pillbar {
|
||||
background-color: white;
|
||||
border-radius: 10px;
|
||||
padding: 5px;
|
||||
margin: 5px;
|
||||
display: block;
|
||||
border: 1px solid black;
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
|
||||
/* Broken Images */
|
||||
img:before {
|
||||
content: "⚠️ MISSING OR BROKEN IMAGE RENDERING ERROR: PLEASE VIEW ORIGINAL PAGE ON GITHUB.COM ⚠️";
|
||||
display: block;
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: "color-emoji";
|
||||
src: local("Apple Color Emoji"),
|
||||
local("Segoe UI Emoji"),
|
||||
local("Segoe UI Symbol"),
|
||||
local("Noto Color Emoji");
|
||||
}
|
||||
|
||||
.emoji {
|
||||
font-family: "color-emoji";
|
||||
}
|
||||
|
||||
table,
|
||||
th,
|
||||
td {
|
||||
border: 1px solid black;
|
||||
}
|
||||
|
||||
.visit_url_link {
|
||||
font-size: xx-large;
|
||||
}
|
||||
|
||||
.why_exist {
|
||||
position: relative;
|
||||
text-align: center;
|
||||
/* italics */
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
</style>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<!-- Not used, did not want to risk non-indexing. -->
|
||||
<!-- <meta property="og:url" content="https://github.com/Timocop/PSMoveServiceEx/wiki/Setting-up-the-PlayStation-3-Eye-Camera"> -->
|
||||
<!-- Did not want to risk contaminating content to be indexed. -->
|
||||
<!-- <meta property="og:description" content="not used"> -->
|
||||
<meta property="og:type" content="website">
|
||||
<meta property="og:title" content="Setting up the PlayStation 3 Eye Camera - Timocop/PSMoveServiceEx GitHub Wiki">
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<div id="main">
|
||||
<nav id="header_button">
|
||||
<p>Content mirrored for search engine indexing from:</p>
|
||||
<div>
|
||||
<span class="visit_url_pillbar">
|
||||
<a href="https://github.com/Timocop/PSMoveServiceEx/wiki/Setting-up-the-PlayStation-3-Eye-Camera" class="visit_url_original">https://github.com/Timocop/PSMoveServiceEx/wiki/Setting-up-the-PlayStation-3-Eye-Camera</a>
|
||||
</span>
|
||||
|
||||
</div>
|
||||
<div class="why_exist">
|
||||
<a href="/">Why does this service exist?</a>
|
||||
</div>
|
||||
</nav>
|
||||
<aside id="header_info">
|
||||
<!-- Last Modified Date added around here from redirect checker-->
|
||||
<p>📅 Last Modified: Thu, 07 May 2026 22:16:43 GMT</p></aside>
|
||||
<article id="content">
|
||||
<h1>Setting up the PlayStation 3 Eye Camera - Timocop/PSMoveServiceEx GitHub Wiki</h1>
|
||||
<div id="mirrored"> <blockquote>
|
||||
<p>[!WARNING]
|
||||
Before connecting or acquiring PlayStation 3 Eye cameras, first take a look at <a href="https://github.com/Timocop/PSMoveServiceEx/wiki/Compatible-Devices" rel="nofollow ugc">Compatible Devices</a> and <a href="https://github.com/Timocop/PSMoveServiceEx/wiki/Hardware-Requirements" rel="nofollow ugc">Hardware Requirements</a>!</p>
|
||||
</blockquote>
|
||||
<h1><a href="#-playstation-3-eye" aria-hidden="true" class="anchor" id="-playstation-3-eye"></a>📷 PlayStation 3 Eye</h1>
|
||||
<p><img src="https://github.com/user-attachments/assets/ec9a8cef-30d4-4b9e-9ec8-48bf7d70a98e" alt="ps3eye" /></p>
|
||||
<h1><a href="#-connection" aria-hidden="true" class="anchor" id="-connection"></a>🔌 Connection</h1>
|
||||
<p>Simply connect the PlayStation 3 Eye with any USB port on your Computer.</p>
|
||||
<p>Do not exceed 5 meters of total cable length (including the 1 meter USB cable of the PlayStation 3 Eye camera), if you are using passive USB extensions cables. If you need to lay out cables over longer distances, its required to get active repeater USB cables or place USB Hub between them to repeat the data signals to avoid signal loss. Externally powered active USB repeater cables are not required as PlayStation 3 Eye cameras can still run at a very low USB voltage (<code>~3.3v</code>).</p>
|
||||
<h1><a href="#-installation" aria-hidden="true" class="anchor" id="-installation"></a>💿 Installation</h1>
|
||||
<p>To use the PlayStation 3 Eye camera, you will need LibUSB drivers in order for <strong>PSMoveServiceEx</strong> to receive image streams and change settings from the camera. You can use <strong><a href="https://github.com/Timocop/PSMoveServiceEx-Virtual-Device-Manager" rel="nofollow ugc">PSMoveServiceEx - Virtual Device Manager</a></strong> to install all necessary camera drivers.</p>
|
||||
<p>Navigate to <code>VDM - Service Management</code> and click <code>Install PlayStation Eye Drivers</code> to install the drivers for the PlayStation 3 Eye camera.</p>
|
||||
<blockquote>
|
||||
<p>[!NOTE]
|
||||
Those drivers are not compatible with DirectShow and PlayStation 3 Eye cameras will not show as Webcams in Microsoft Windows. Decreasing the risk of other applications accessing them.</p>
|
||||
</blockquote>
|
||||
<p><img src="https://github.com/Timocop/PSMoveServiceEx/assets/22834512/831ca78a-8a2a-4b0a-89c1-4355d9b2ea81" alt="" /></p>
|
||||
<h1><a href="#️-configuration" aria-hidden="true" class="anchor" id="️-configuration"></a>⚙️ Configuration</h1>
|
||||
<p>Use <strong>PSMoveServiceEx Config Tool</strong> to configure the PlayStation 3 Eye camera. You can find all camera settings when calibrating controller or head-mounted display colors in <code>Config Tool > Tracker Settings > Color Calibration</code></p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Steps</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><img src="https://github.com/Timocop/PSMoveServiceEx/assets/22834512/6d18eff6-aa38-467c-bed5-14deceb7a36a" alt="" /></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><img src="https://github.com/Timocop/PSMoveServiceEx/assets/22834512/3b1f11e0-36ee-493f-befa-1077382cd775" alt="" /></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><img src="https://github.com/Timocop/PSMoveServiceEx/assets/22834512/478f5ad9-393a-4ef6-923b-0938e7a98897" alt="" /></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<h1><a href="#-troubleshooting" aria-hidden="true" class="anchor" id="-troubleshooting"></a>🧰 Troubleshooting</h1>
|
||||
<h3><a href="#-psmoveserviceex-config-tool-stream-shows-tracker-timed-out-or-has-a-unstable-framerate" aria-hidden="true" class="anchor" id="-psmoveserviceex-config-tool-stream-shows-tracker-timed-out-or-has-a-unstable-framerate"></a>📌 PSMoveServiceEx Config Tool stream shows 'Tracker timed out.' or has a unstable framerate.</h3>
|
||||
<p>This means PSMoveServiceEx did not receive any image from the camera in a while or the image could not be received in time from the camera.
|
||||
This can be caused by the following reasons:</p>
|
||||
<ul>
|
||||
<li>USB cable too long (signal loss).</li>
|
||||
<li>Too many heavy-duty devices connected (such as cameras) to one USB Host Controller causing insufficient bandwidth.</li>
|
||||
<li>Device disconnected.</li>
|
||||
<li>Improper configuration (such as too high framerate).</li>
|
||||
<li>Bad or broken cable.</li>
|
||||
</ul>
|
||||
<blockquote>
|
||||
<p>[!NOTE]
|
||||
Please check <a href="https://github.com/Timocop/PSMoveServiceEx/wiki/Compatible-Devices" rel="nofollow ugc">Compatible Devices</a> and <a href="https://github.com/Timocop/PSMoveServiceEx/wiki/Hardware-Requirements" rel="nofollow ugc">Hardware Requirements</a> to properly connect your PlayStation 3 Eye camera.</p>
|
||||
</blockquote>
|
||||
<h3><a href="#-playstation-eye-3-cameras-do-not-show-in-psmoveserviceex-config-tool" aria-hidden="true" class="anchor" id="-playstation-eye-3-cameras-do-not-show-in-psmoveserviceex-config-tool"></a>📌 PlayStation Eye 3 Cameras do not show in PSMoveServiceEx Config Tool.</h3>
|
||||
<p>Check if all camera drivers are installed properly as seen in <a href="https://github.com/Timocop/PSMoveServiceEx/wiki/Setting-up-the-PlayStation-3-Eye-Camera#-installation" rel="nofollow ugc"><strong>💿 Installation</strong></a> section of this page or restart PSMoveServiceEx. If they are installed properly and they still wont show up, check troubleshooting section <a href="https://github.com/Timocop/PSMoveServiceEx/wiki/Setting-up-the-PlayStation-3-Eye-Camera#-psmoveserviceex-config-tool-stream-shows-tracker-timed-out-or-has-a-unstable-framerate" rel="nofollow ugc"><strong>📌 PlayStation Eye 3 Cameras do not show in PSMoveServiceEx Config Tool.</strong></a>.</p>
|
||||
</div>
|
||||
</article>
|
||||
<nav id="index">
|
||||
<a href="/m/Timocop/PSMoveServiceEx/wiki_index">🗂️ Page Index for this GitHub Wiki</a>
|
||||
</nav>
|
||||
</div>
|
||||
<script defer src="https://static.cloudflareinsights.com/beacon.min.js/v833ccba57c9e4d2798f2e76cebdd09a11778172276447" integrity="sha512-57MDmcccJXYtNnH+ZiBwzC4jb2rvgVCEokYN+L/nLlmO8rfYT/gIpW2A569iJ/3b+0UEasghjuZH/ma3wIs/EQ==" data-cf-beacon='{"version":"2024.11.0","token":"4645764ca8384bfc86034b71f1dafd76","r":1,"server_timing":{"name":{"cfCacheStatus":true,"cfEdge":true,"cfExtPri":true,"cfL4":true,"cfOrigin":true,"cfSpeedBrain":true},"location_startswith":null}}' crossorigin="anonymous"></script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
1056
research/visual-spatial-map/mirrors/ultraleap-images-example.html
Normal file
1056
research/visual-spatial-map/mirrors/ultraleap-images-example.html
Normal file
File diff suppressed because it is too large
Load diff
1062
research/visual-spatial-map/mirrors/ultraleap-leapc-images.html
Normal file
1062
research/visual-spatial-map/mirrors/ultraleap-leapc-images.html
Normal file
File diff suppressed because it is too large
Load diff
Binary file not shown.
205
research/visual-spatial-map/mirrors/zadig-home.html
Normal file
205
research/visual-spatial-map/mirrors/zadig-home.html
Normal file
|
|
@ -0,0 +1,205 @@
|
|||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head profile="http://www.w3.org/2005/10/profile">
|
||||
<meta charset='utf-8'>
|
||||
<title>Zadig - USB driver installation made easy</title>
|
||||
<script>
|
||||
function setCookie(name, value, days) {
|
||||
var expires = "";
|
||||
if (days) {
|
||||
var date = new Date();
|
||||
date.setTime(date.getTime() + (days * 24 * 60 * 60 * 1000));
|
||||
expires = "; expires=" + date.toUTCString();
|
||||
}
|
||||
document.cookie = name + "=" + (value || "") + expires + "; path=/";
|
||||
}
|
||||
function getCookie(name) {
|
||||
var nameEQ = name + "=";
|
||||
var ca = document.cookie.split(';');
|
||||
for (var i = 0; i < ca.length; i++) {
|
||||
var c = ca[i];
|
||||
while (c.charAt(0) == ' ') c = c.substring(1, c.length);
|
||||
if (c.indexOf(nameEQ) == 0) return c.substring(nameEQ.length, c.length);
|
||||
}
|
||||
return "";
|
||||
}
|
||||
document.addEventListener("DOMContentLoaded", function(event)
|
||||
{
|
||||
if (getCookie('display_cookie_notice') != 'no')
|
||||
document.getElementById('cookie-statement').style.display = 'block';
|
||||
});
|
||||
</script>
|
||||
<script async src="https://www.googletagmanager.com/gtag/js?id=G-6PCK0CM7G9"></script>
|
||||
<script>
|
||||
window.dataLayer = window.dataLayer || [];
|
||||
function gtag(){dataLayer.push(arguments);}
|
||||
gtag('js', new Date());
|
||||
gtag('config', 'G-6PCK0CM7G9');
|
||||
</script>
|
||||
<style type="text/css">
|
||||
body {
|
||||
background-color: #3f4555;
|
||||
font-family: Helvetica, Arial, FreeSans, san-serif;
|
||||
color: #ffffff;
|
||||
}
|
||||
#cookie-statement {
|
||||
background-color: #5a5a5a;
|
||||
border: none;
|
||||
border-radius: 0;
|
||||
color: #fff;
|
||||
-moz-border-radius: 0;
|
||||
-webkit-border-radius: 0;
|
||||
margin: -8px -8px;
|
||||
padding: 5px 0px 5px 15px;
|
||||
position: relative;
|
||||
width: 100%;
|
||||
z-index: 999;
|
||||
}
|
||||
#cookie-statement a {
|
||||
background-color: #303030;
|
||||
border: 1px solid rgba(0,0,0,.1);
|
||||
-moz-border-radius: 2px;
|
||||
-webkit-border-radius: 2px;
|
||||
border-radius: 2px;
|
||||
color: #fff;
|
||||
cursor: pointer;
|
||||
line-height: 200%;
|
||||
padding: 4px 8px;
|
||||
text-decoration: none;
|
||||
white-space: nowrap;
|
||||
}
|
||||
#right_column {
|
||||
float: right;
|
||||
margin: 0 auto;
|
||||
margin-top: 10px;
|
||||
margin-right: 10px;
|
||||
width: 230px;
|
||||
font-size: 0.8em;
|
||||
}
|
||||
#container {
|
||||
margin: 0 auto;
|
||||
width: 730px;
|
||||
}
|
||||
table.reference
|
||||
{
|
||||
color: #000030;
|
||||
background-color:#ffffff;
|
||||
border:1px solid #c3c3c3;
|
||||
border-collapse:collapse;
|
||||
width:100%;
|
||||
}
|
||||
table.reference th
|
||||
{
|
||||
background-color:#e5eecc;
|
||||
border:1px solid #c3c3c3;
|
||||
padding:3px;
|
||||
vertical-align:top;
|
||||
}
|
||||
table.reference td
|
||||
{
|
||||
border:1px solid #c3c3c3;
|
||||
padding:3px;
|
||||
vertical-align:top;
|
||||
}
|
||||
h1 { font-size: 3.8em; color: #c0baaa; }
|
||||
h1 .small { font-size: 0.4em; }
|
||||
h1 a { text-decoration: none }
|
||||
h2 { padding: 12px; font-size: 1.5em; color: #c0baaa; line-height: 1.3em; border: 2px solid #706a6a;}
|
||||
h3 { text-align: center; color: #c0baaa; }
|
||||
a { color: #c0baaa; }
|
||||
span.centerImage { text-align: center; }
|
||||
.cookie-notification { background: #fffbe4; border-color: #f8f6e6; overflow: hidden; }
|
||||
.tagline { font-size: 1.6em; margin-bottom: 20px; margin-top: -30px; font-style: italic;}
|
||||
.download { float: right; }
|
||||
pre { background: #000; color: #fff; padding: 15px;}
|
||||
hr { border: 0; width: 80%; border-bottom: 1px solid #aaa}
|
||||
.footer { font-size: 0.9em; text-align:center; padding-top:30px; font-style: italic; }
|
||||
.treeView{ -moz-user-select:none; position:relative; }
|
||||
.treeView ul{ margin:0 0 0 -1.5em; padding:0 0 0 1.5em; }
|
||||
.treeView li{ margin:0; padding:0; list-style-position:inside; list-style-image:none; cursor:auto; }
|
||||
.treeView li.collapsibleListOpen{ list-style-image:url('pics/button-open.png'); cursor:pointer; }
|
||||
.treeView li.collapsibleListClosed{ list-style-image:url('pics/button-closed.png'); cursor:pointer; }
|
||||
.treeView li.collapsibleListItem{ padding-left:1.5em; }
|
||||
.treeView li li{ padding-left:1.5em; }
|
||||
</style>
|
||||
<meta name="google-translate-customization" content="9515a7216137b185-44ae73c3870830a7-g8260146ad1003132-d"></meta>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<div id="cookie-statement" style="display: none;" class="cookie-notification">
|
||||
<div class="text">
|
||||
<span>This page uses Google services, which may serve <u>non-personalised</u> cookies for advertising, analytics and security.</span>
|
||||
<a href="https://policies.google.com/technologies/cookies" target="_blank">See details</a>
|
||||
<a href="#" onclick="document.getElementById('cookie-statement').style.display = 'none'; setCookie('display_cookie_notice', 'no', 365);">OK</a>
|
||||
</div>
|
||||
</div>
|
||||
<div id="right_column">
|
||||
<script async src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
|
||||
<!-- Zadig -->
|
||||
<ins class="adsbygoogle"
|
||||
style="display:inline-block;width:160px;height:600px"
|
||||
data-ad-client="ca-pub-8924382055379825"
|
||||
data-ad-slot="5382671908"></ins>
|
||||
<script>
|
||||
(adsbygoogle = window.adsbygoogle || []).requestNonPersonalizedAds=1;
|
||||
(adsbygoogle = window.adsbygoogle || []).push({});
|
||||
</script>
|
||||
</div>
|
||||
|
||||
<div id="container">
|
||||
<h1><img border="0" src="pics/zadig-128.png" alt="[Zadig icon]" style="width:128px;height:128px;vertical-align:middle; margin-bottom: 10px;"/> Zadig</h1>
|
||||
<div class="tagline">USB driver installation made easy</div>
|
||||
<img border="0" src="pics/zadig.png" alt="[Zadig screenshot]"/>
|
||||
<p><b><i>Zadig</i></b> is a Windows application that installs generic USB drivers, such as <a href="https://learn.microsoft.com/en-us/windows-hardware/drivers/usbcon/introduction-to-winusb-for-developers">WinUSB</a>, <a href="https://sourceforge.net/p/libusb-win32/wiki/Home/">libusb-win32/libusb0.sys</a> or <a href="https://libusbk.sourceforge.net/UsbK3/">libusbK</a>, to help you access USB devices.</p>
|
||||
<p>It can be especially useful for cases where:
|
||||
<ul>
|
||||
<li>you want to access a device using a libusb-based application</li>
|
||||
<li>you want to upgrade a generic USB driver</li>
|
||||
<li>you want to access a device using WinUSB</li>
|
||||
</ul>
|
||||
</p>
|
||||
<p><b>Note:</b> <i>"libusb-based"</i> above means an application that uses either <a href="https://libusb.info">libusb</a>, <a href="https://sourceforge.net/p/libusb-win32/wiki/Home/">libusb-win32</a> or <a href="https://libusbk.sourceforge.net/UsbK3/">libusbK</a>.</p>
|
||||
<h2 style="border: 4px solid #a09a8a;"><span style="font-size: 133%">Download</span></h2>
|
||||
<p><b>Updated 2024.06.13:</b></p>
|
||||
<p><ul><li><span style="font-size: 133%"><b><a href="https://github.com/pbatard/libwdi/releases/download/v1.5.1/zadig-2.9.exe">Zadig 2.9</a></b></span> (5.1 MB)</li>
|
||||
<li><a href="/downloads/">Other versions</a></li>
|
||||
</ul></p>
|
||||
<h4>System Requirements:</h4>
|
||||
Windows 7 or later.<br/>
|
||||
Windows XP and Windows Vista are <b>NO LONGER SUPPORTED</b>.</p>
|
||||
<h2>Usage</h2>
|
||||
<p>Download the executable and run it — no installation is necessary.<br/>If elevation is required, you will be prompted for it.</p>
|
||||
<p>An usage guide for Zadig is available <a href="https://github.com/pbatard/libwdi/wiki/Zadig">HERE</a>.</p>
|
||||
<p>The executable is digitally signed and the signature should state: <i>"Akeo Consulting"</i></p>
|
||||
<a name="FAQ"></a>
|
||||
<h2>Frequently Asked Questions (FAQ)</h2>
|
||||
<p>A Zadig FAQ is available <b><a href="https://github.com/pbatard/libwdi/wiki/FAQ#wiki-Zadig">HERE</a></b>.</p>
|
||||
<p>To provide feedback, report a bug or request an enhancement please use the github <a href="https://github.com/pbatard/libwdi/issues">issue tracker</a>. Or you can <a href="mailto:pete@akeo.ie?subject=Zadig">send an e-mail</a>.</p>
|
||||
<h2>License</h2>
|
||||
<p><a href="https://www.gnu.org/licenses/gpl.html">GNU General Public License (GPL) version 3</a> or later.<br />You are free to distribute, modify or even sell the software, insofar as you respect the GPLv3 license.</p>
|
||||
<p>Zadig is based on <a href="https://github.com/pbatard/libwdi">libwdi</a> which uses an <a href="https://www.gnu.org/licenses/lgpl.html">LGPL version 3</a> or later license</a>.
|
||||
<p>The executable is produced in a 100% transparent manner, from its <a href="https://github.com/pbatard/libwdi/tree/master/examples">public source</a>, using a <a href="https://visualstudio.microsoft.com/">Visual Studio</a> environment.</p>
|
||||
<a name="versions"></a>
|
||||
<h2>Changelog</h2>
|
||||
<ul>
|
||||
<li><b>Version 2.9</b> (2024.06.13)<ul>
|
||||
<li>Add libusb-win32 ARM64 driver installation support (courtesy of <b>Peter Dons Tychsen</b>)<br/>
|
||||
<b>Note:</b> Requires Windows 10 1511 or later with x86 (32-bit) emulation support.</li>
|
||||
<li>Fix UAC not being properly triggered during driver installation (courtesy of <b>Andrew Meyer</b>)</li>
|
||||
<li>Embedded drivers: WinUSB v6.1.7600.16385, libusb-win32 v1.4.0.0, libusbK v3.1.0.0 & USBSer (native)</li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<h2>Source Code</h2>
|
||||
<p><ul><li><a href="https://github.com/pbatard/libwdi/archive/v1.5.1.zip">libwdi 1.5.1 source</a> (330 KB). Includes Zadig in the examples directory.</li>
|
||||
<li>Alternatively, you can clone the <a href="https://git-scm.com">git</a> repository using:
|
||||
<pre>$ git clone git://github.com/pbatard/libwdi</pre></li>
|
||||
<li>For more information, see the <a href="https://github.com/pbatard/libwdi">GitHub project</a>.</li></ul>
|
||||
If you are a developer, you are very much encouraged to tinker with Zadig/libwdi and submit patches.</p>
|
||||
</div>
|
||||
<div class="footer">
|
||||
Copyright © 2013-2024 <a href="https://pete.akeo.ie">Pete Batard</a><br/>
|
||||
Hosting by <a href="https://github.com">GitHub</a>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
153
research/visual-spatial-map/summary.md
Normal file
153
research/visual-spatial-map/summary.md
Normal file
|
|
@ -0,0 +1,153 @@
|
|||
# Visual Spatial Map And Splat Scene
|
||||
|
||||
## Objective
|
||||
|
||||
Extend the AquaSynth/Mimir concept from "one camera view plus spatial audio" into a shared live spatial scene:
|
||||
|
||||
- two Razer Kiyo-class camera/mic rigs provide RGB and audio; one is Kiyo Pro
|
||||
- two PlayStation Eye / PS3 Eye cameras provide cheap high-rate low-resolution tracking views
|
||||
- one Leap Motion provides close-range stereo IR/hand/marker sensing
|
||||
- tracked objects and speakers/mics live in one world coordinate frame
|
||||
- nice-camera RGB is projected onto reconstructed/tracked geometry
|
||||
- renderer exposes adjustable virtual camera position
|
||||
- ambisonic render follows the same listener/camera pose
|
||||
|
||||
## Hardware Sanity
|
||||
|
||||
The high-rate cheap cameras in this rig are **two PlayStation Eye / PS3 Eye** units, not PS2 EyeToys. The PS3 developer wiki lists PlayStation Eye video modes at 640x480 @ 60 Hz and 320x240 @ 120 Hz [PS3 Eye]. Community tracking stacks sometimes expose higher practical modes/settings, but that needs verification against the exact driver and USB topology. The PS2 EyeToy is described as a normal USB 1.1 camera [PS2 EyeToy], so it remains only a rejected naming confusion path.
|
||||
|
||||
Windows driver state matters. The opentrack wiki documents a current open-driver route for PS3 Eye using Zadig to install `libusb-win32` on interface 0, with `libusbK` as a fallback if needed [opentrack PS3 Eye; Zadig]. The old Code Laboratories CL Eye driver exists and advertised DirectShow/high-frame-rate support, but its own page targets Windows XP/Vista/7 era systems [CL Eye]. For this rig, the open-driver path is the default candidate; CL Eye is historical reference material, not a dependency to pirate or preserve.
|
||||
|
||||
The beauty/RGB cameras are **two Razer Kiyos, one of them Kiyo Pro**. Their microphones can participate in the audio system, but camera attachment does not imply audio clock sync. They need the same live adaptive sync treatment as other independent USB audio devices.
|
||||
|
||||
The **Leap Motion** is a separate stereo IR sensor. LeapUVC gives low-level UVC access to Leap Motion image data and controls such as LED brightness, gamma, exposure, gain, resolution, and camera calibration/depth examples [LeapUVC]. Ultraleap's LeapC image API also exposes stereo infrared image buffers and distortion/calibration maps, with functions for rectilinear/pixel mapping [Ultraleap Images]. In this rig, installing the LeapUVC driver means the device shows up as a generic UVC device and is no longer addressable through Ultraleap software in that mode. Treat LeapUVC and Ultraleap tracking as mutually exclusive operating modes unless proven otherwise on the actual machine.
|
||||
|
||||
Cut line: enumerate the actual five visual devices and their capture modes before designing the capture layer. Nostalgia is not a transport protocol, and USB bandwidth is not moved by vibes.
|
||||
|
||||
## Current Mechanism
|
||||
|
||||
The audio research now has four separate authorities:
|
||||
|
||||
- live adaptive mic sync
|
||||
- feedback/system identification
|
||||
- playback calibration
|
||||
- ambisonic scene/rendering
|
||||
|
||||
The visual system needs the same discipline. It should not be bolted onto OBS as "some extra cameras." It needs its own map authority.
|
||||
|
||||
## Core Invariant
|
||||
|
||||
There must be one canonical world coordinate system.
|
||||
|
||||
Every sensor and emitter needs a transform into that world:
|
||||
|
||||
- two PS3 Eye tracking cameras
|
||||
- two Razer Kiyo-class RGB cameras
|
||||
- Leap Motion stereo IR sensor
|
||||
- camera-attached microphones
|
||||
- standalone microphones
|
||||
- speakers
|
||||
- tracked props/markers
|
||||
- virtual render camera/listener
|
||||
|
||||
Audio and visual pipelines can run at different rates, but they must publish timestamped observations into the same spatial frame.
|
||||
|
||||
## Research Findings
|
||||
|
||||
OpenCV and related tools already cover the geometry spine: multi-camera calibration estimates camera intrinsics/extrinsics, while triangulation and reprojection operate over calibrated camera sets [OpenCV; Multi-camera Calibration]. The `multi-camera-calibration` project is explicitly built for multi-camera 3D tracking workflows and exposes keypoint triangulation and reprojection utilities [Multi-camera Calibration].
|
||||
|
||||
Jacob and Haeb-Umbach address the exact cross-modal trap: audio and visual self-localization can produce separate coordinate systems with unknown rotation, scale, and translation. Their paper proposes using audio-visual events localized by both microphones and cameras to align acoustic sensors into the visual coordinate system [Jacob/Haeb-Umbach 2015]. This is directly relevant for camera-attached mics and speaker/mic calibration pings that are also visually tracked.
|
||||
|
||||
Gaussian splatting is plausible as a render representation, but it is not the first authority. Kerbl et al. start from sparse points produced during camera calibration and optimize 3D Gaussians for real-time novel-view rendering [Kerbl 2023]. Wu et al. extend Gaussian splatting into dynamic scenes with 4D-GS [Wu 2024]. RTG-SLAM shows real-time reconstruction using Gaussian splatting, but its published shape assumes RGB-D input [Peng 2024]. The PS3 Eyes are not RGB-D; they are marker/pose sensors. Leap can provide close-range stereo IR evidence. The Kiyos provide RGB texture/evidence.
|
||||
|
||||
For audio coupling, IEM SceneRotator confirms the ordinary ambisonic operation: rotate an ambisonic scene by yaw/pitch/roll or quaternion data [IEM SceneRotator]. That covers orientation. Translation/listener movement is more involved and should be represented in the world model before pretending one rotator solves full six-degree navigation.
|
||||
|
||||
## Recommended Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["2 PS3 Eyes: high-rate low-res"] --> B["2D marker detections"]
|
||||
C["2 Razer Kiyos: RGB + mics"] --> D["RGB frames + camera poses"]
|
||||
P["Leap Motion: stereo IR"] --> Q["near-field hands/markers/depth cues"]
|
||||
E["Mic/speaker/audio estimators"] --> F["Audio landmarks and transfer paths"]
|
||||
G["Calibration observations"] --> H["World model authority"]
|
||||
B --> H
|
||||
D --> H
|
||||
Q --> H
|
||||
F --> H
|
||||
H --> I["Tracked 3D points / prop poses"]
|
||||
H --> J["Sensor and emitter transforms"]
|
||||
I --> K["Geometry/point cloud/splat update"]
|
||||
D --> K
|
||||
K --> L["Free-view renderer"]
|
||||
J --> M["Ambisonic listener/render pose"]
|
||||
L --> N["Virtual camera output"]
|
||||
M --> O["AquaSynth spatial audio output"]
|
||||
```
|
||||
|
||||
## Practical Path
|
||||
|
||||
1. Build a calibration rig.
|
||||
Use a ChArUco/checkerboard or wand for camera intrinsics/extrinsics, then a visible-and-audible marker event for audio-visual alignment. Ping pong balls are good tracking props if they are visually distinctive; retroreflective or lit markers are better if the cameras can be filtered/exposed for them.
|
||||
|
||||
2. Separate camera roles.
|
||||
PS3 Eyes own low-resolution marker tracking and temporal precision. Kiyos own RGB projection and human-readable video. Leap owns close-range stereo IR/hand/marker evidence. Do not ask the Kiyos to be high-rate trackers, do not ask PS Eyes to be beautiful, and do not ask Leap to see the whole room.
|
||||
|
||||
3. Start with sparse 3D tracking.
|
||||
Triangulate marker positions from calibrated camera detections. Publish timestamped 3D points/rigid-body poses. This proves the coordinate system before the splat renderer enters wearing sunglasses.
|
||||
|
||||
4. Register audio devices into the same world.
|
||||
Camera-attached mics inherit an initial transform from their camera rig plus measured mic offset. Standalone mics/speakers need calibration observations. Audio-visual events can align modality-specific maps [Jacob/Haeb-Umbach 2015].
|
||||
|
||||
5. Add RGB projection.
|
||||
Project nice-camera RGB onto tracked geometry/point cloud using known camera intrinsics/extrinsics. Begin with colored points or surfels before live Gaussian optimization.
|
||||
|
||||
6. Add splatting as a render layer, not the state owner.
|
||||
A Gaussian scene can render the current world beautifully, but the canonical state should remain calibrated transforms, timestamps, tracked points, and confidence. Gaussian parameters are a view/render representation.
|
||||
|
||||
7. Couple virtual camera/listener pose to ambisonics.
|
||||
The adjustable visual camera pose should also define the audio listener pose. Orientation can rotate the ambisonic scene; translation requires source/listener geometry and distance/room modeling.
|
||||
|
||||
## Failure Modes
|
||||
|
||||
- USB bandwidth can dominate before CPU/GPU does, especially with multiple PS Eye cameras.
|
||||
- LeapUVC driver mode can make the Leap unavailable to Ultraleap software; do not design a runtime that requires both raw LeapUVC frames and Ultraleap hand tracking at the same time.
|
||||
- Unsynchronized cameras produce triangulation jitter unless timestamps/exposure timing are handled.
|
||||
- Rolling shutter or auto-exposure can corrupt high-speed marker tracking.
|
||||
- Ping pong balls are visually cheap but ambiguous if multiple identical balls cross paths.
|
||||
- Gaussian splatting from sparse/noisy live markers alone will not create a rich scene; it needs RGB observations and/or depth/geometric priors.
|
||||
- Dynamic 4D splatting is research-grade. A reliable v1 should render sparse points/surfels first.
|
||||
- Camera-attached microphones still need audio clock/drift handling; physical attachment does not synchronize sample clocks.
|
||||
|
||||
## Cut Line
|
||||
|
||||
Do not begin by building a live Gaussian splat engine. Begin with the world model:
|
||||
|
||||
- enumerate the two PS3 Eyes, two Razer Kiyos, and Leap Motion
|
||||
- choose Leap operating mode: LeapUVC raw stereo IR or Ultraleap tracking service
|
||||
- verify frame rates and USB topology
|
||||
- calibrate intrinsics/extrinsics
|
||||
- triangulate one marker
|
||||
- attach one audio source to the same coordinate frame
|
||||
- drive ambisonic scene orientation from virtual camera/listener pose
|
||||
|
||||
After that spine holds, add RGB projection, then splats.
|
||||
|
||||
## Citations
|
||||
|
||||
- [PS3 Eye] PS3 Developer Wiki, "PlayStation Eye." Mirror: `mirrors/ps3-playstation-eye-devwiki.html`.
|
||||
- [PS2 EyeToy] PS2 Developer Wiki, "EyeToy." Mirror: `mirrors/ps2-eyetoy-devwiki.html`.
|
||||
- [iPiSoft] iPiSoft Wiki, "User Guide for Multiple PlayStation Eye Cameras Configuration." Mirror: `mirrors/ipisoft-multiple-playstation-eye-cameras.html`.
|
||||
- [opentrack PS3 Eye] opentrack Wiki, "PS3 Eye open driver instructions." Mirror: `mirrors/opentrack-ps3-eye-open-driver-instructions.html`.
|
||||
- [Zadig] Zadig. Mirror: `mirrors/zadig-home.html`.
|
||||
- [CL Eye] Code Laboratories, "CL Eye Platform Driver." Mirror: `mirrors/code-laboratories-cl-eye-driver.html`.
|
||||
- [DelanClip] DelanClip, "PS3 Eye Camera Head Tracking - OpenTrack - Open Driver." Mirror: `mirrors/delanclip-ps3-eye-open-driver.html`.
|
||||
- [LeapUVC] Leap Motion, `leapuvc` repository and manual. Mirrors: `mirrors/leapuvc-github.html`, `mirrors/leapuvc-readme.md`, `mirrors/leapuvc-manual.pdf`.
|
||||
- [Ultraleap Images] Ultraleap, "Images." Mirror: `mirrors/ultraleap-leapc-images.html`.
|
||||
- [OpenCV] OpenCV, "Multi-camera Calibration." Mirror: `mirrors/opencv-multi-camera-calibration.html`.
|
||||
- [Multi-camera Calibration] Multi-camera Calibration documentation. Mirror: `mirrors/multi-camera-calibration-docs.html`.
|
||||
- [Jacob/Haeb-Umbach 2015] Florian Jacob and Reinhold Haeb-Umbach, "Absolute Geometry Calibration of Distributed Microphone Arrays in an Audio-Visual Sensor Network." Mirror: `mirrors/krekovic-2015-audio-visual-geometry-calibration.pdf`.
|
||||
- [Kerbl 2023] Bernhard Kerbl et al., "3D Gaussian Splatting for Real-Time Radiance Field Rendering." Mirror: `mirrors/kerbl-2023-3d-gaussian-splatting.pdf`.
|
||||
- [Wu 2024] Guanjun Wu et al., "4D Gaussian Splatting for Real-Time Dynamic Scene Rendering." Mirror: `mirrors/wu-2024-4d-gaussian-splatting.pdf`.
|
||||
- [Peng 2024] Zhexi Peng et al., "RTG-SLAM: Real-time 3D Reconstruction at Scale using Gaussian Splatting." Mirror: `mirrors/peng-2024-rtg-slam.pdf`.
|
||||
- [IEM SceneRotator] IEM Plug-in Suite, "SceneRotator Guide." Mirror: `mirrors/iem-scenerotator-guide.html`.
|
||||
|
|
@ -1,3 +1,4 @@
|
|||
{"ts":"2026-05-21T17:31:13+00:00","type":"implementation","status":"ok","note":"MimirVideoFrameDescriptor carries dimensions, pixel format, stride, device timestamp, and optional native/GPU handle; MimirNativeIngestStreamSource can push LeapStereoIr or other camera frames directly into rolling buffers while process-backed ingest remains a network/diagnostic edge."}
|
||||
{"ts":"2026-05-21T17:40:33+00:00","type":"decision","status":"ok","note":"Mimir live camera ingest should talk directly to drivers through IMimirVideoCaptureDriver: LeapUVC/libusb, LeapC images, Media Foundation, DirectShow, or GPU shared textures."}
|
||||
{"ts":"2026-05-21T18:55:00+00:00","type":"implementation","status":"ok","note":"Deleted script-era packages, tests, helper tools, dependency manifests, archived research mirrors, and stale docs from the working tree. The repo's live surface is C# Mimir.Runtime, Mimir.App, native reservoir, PowerShell bridge utilities, Aquarium, and Faust/native DSP."}
|
||||
{"ts":"2026-05-21T19:05:00+00:00","type":"correction","status":"ok","note":"Restored the research archive as reference-only memory. Research may preserve external examples and failed-path lessons, but live implementation must be rebuilt through direct drivers, native buffers, Aquarium GPU compute, and Faust/native DSP."}
|
||||
|
|
|
|||
|
|
@ -76,9 +76,10 @@ current_status:
|
|||
summary: >
|
||||
The repo owns a C# Mimir app/runtime scaffold, a native reservoir crate, OBS
|
||||
bridge scripts, and persistence surfaces. Script-era packages, tests,
|
||||
helper tools, dependency manifests, and archived mirrors have been removed from the
|
||||
working tree. The next useful cut is direct driver ingest, not reviving old
|
||||
diagnostic machinery.
|
||||
helper tools, dependency manifests, and stale docs have been removed from
|
||||
the working tree. Research archives remain as reference-only memory for raw
|
||||
native/GPU implementation shape. The next useful cut is direct driver
|
||||
ingest, not reviving old diagnostic machinery.
|
||||
next_action: >
|
||||
Feed Leap stereo IR frames through `IMimirVideoCaptureDriver` into
|
||||
`MimirSynchronizationHub`, measure sustained cadence over the default
|
||||
|
|
|
|||
Loading…
Reference in a new issue