Archive CultNet distributed database research

This commit is contained in:
Meta Vens 2026-05-19 17:14:28 +01:00
commit 5c4195677d
10 changed files with 19086 additions and 0 deletions

View file

@ -0,0 +1,709 @@
<!DOCTYPE html>
<html lang="en">
<head> <title>[1805.06358] Conflict-free Replicated Data Types (CRDTs)</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="apple-touch-icon" sizes="180x180" href="/static/browse/0.3.4/images/icons/apple-touch-icon.png">
<link rel="icon" type="image/png" sizes="32x32" href="/static/browse/0.3.4/images/icons/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="16x16" href="/static/browse/0.3.4/images/icons/favicon-16x16.png">
<link rel="manifest" href="/static/browse/0.3.4/images/icons/site.webmanifest">
<link rel="mask-icon" href="/static/browse/0.3.4/images/icons/safari-pinned-tab.svg" color="#5bbad5">
<meta name="msapplication-TileColor" content="#da532c">
<meta name="theme-color" content="#ffffff">
<link rel="stylesheet" type="text/css" media="screen" href="/static/browse/0.3.4/css/arXiv.css?v=20260318" />
<link rel="stylesheet" type="text/css" media="print" href="/static/browse/0.3.4/css/arXiv-print.css?v=20200611" />
<link rel="stylesheet" type="text/css" media="screen" href="/static/browse/0.3.4/css/browse_search.css" />
<script language="javascript" src="/static/browse/0.3.4/js/accordion.js" ></script>
<script language="javascript" src="/static/browse/0.3.4/js/optin-modal.js?v=20250819"></script>
<link rel="canonical" href="https://arxiv.org/abs/1805.06358"/>
<meta name="description" content="Abstract page for arXiv paper 1805.06358: Conflict-free Replicated Data Types (CRDTs)"><meta property="og:type" content="website" />
<meta property="og:site_name" content="arXiv.org" />
<meta property="og:title" content="Conflict-free Replicated Data Types (CRDTs)" />
<meta property="og:url" content="https://arxiv.org/abs/1805.06358v1" />
<meta property="og:image" content="/static/browse/0.3.4/images/arxiv-logo-fb.png" />
<meta property="og:image:secure_url" content="/static/browse/0.3.4/images/arxiv-logo-fb.png" />
<meta property="og:image:width" content="1200" />
<meta property="og:image:height" content="700" />
<meta property="og:image:alt" content="arXiv logo"/>
<meta property="og:description" content="A conflict-free replicated data type (CRDT) is an abstract data type, with a well defined interface, designed to be replicated at multiple processes and exhibiting the following properties: (1) any replica can be modified without coordinating with another replicas; (2) when any two replicas have received the same set of updates, they reach the same state, deterministically, by adopting mathematically sound rules to guarantee state convergence."/>
<meta name="twitter:site" content="@arxiv"/>
<meta name="twitter:card" content="summary"/>
<meta name="twitter:title" content="Conflict-free Replicated Data Types (CRDTs)"/>
<meta name="twitter:description" content="A conflict-free replicated data type (CRDT) is an abstract data type, with a well defined interface, designed to be replicated at multiple processes and exhibiting the following properties: (1)..."/>
<meta name="twitter:image" content="https://static.arxiv.org/icons/twitter/arxiv-logo-twitter-square.png"/>
<meta name="twitter:image:alt" content="arXiv logo"/>
<link rel="stylesheet" media="screen" type="text/css" href="/static/browse/0.3.4/css/tooltip.css"/><link rel="stylesheet" media="screen" type="text/css" href="https://static.arxiv.org/js/bibex-dev/bibex.css?20200709"/> <script src="/static/browse/0.3.4/js/mathjaxToggle.min.js" type="text/javascript"></script> <script src="//code.jquery.com/jquery-latest.min.js" type="text/javascript"></script>
<script src="//cdn.jsdelivr.net/npm/js-cookie@2/src/js.cookie.min.js" type="text/javascript"></script>
<script src="//cdn.jsdelivr.net/npm/dompurify@2.3.5/dist/purify.min.js"></script>
<script src="/static/browse/0.3.4/js/toggle-labs.js?20241022" type="text/javascript"></script>
<script src="/static/browse/0.3.4/js/cite.js" type="text/javascript"></script><meta name="citation_title" content="Conflict-free Replicated Data Types (CRDTs)" /><meta name="citation_author" content="Preguiça, Nuno" /><meta name="citation_author" content="Baquero, Carlos" /><meta name="citation_author" content="Shapiro, Marc" /><meta name="citation_doi" content="10.1007/978-3-319-63962-8\_185-1" /><meta name="citation_date" content="2018/05/16" /><meta name="citation_online_date" content="2018/05/16" /><meta name="citation_pdf_url" content="https://arxiv.org/pdf/1805.06358" /><meta name="citation_arxiv_id" content="1805.06358" /><meta name="citation_abstract" content="A conflict-free replicated data type (CRDT) is an abstract data type, with a well defined interface, designed to be replicated at multiple processes and exhibiting the following properties: (1) any replica can be modified without coordinating with another replicas; (2) when any two replicas have received the same set of updates, they reach the same state, deterministically, by adopting mathematically sound rules to guarantee state convergence." />
</head>
<body class="with-cu-identity">
<div class="flex-wrap-footer">
<header>
<a href="#content" class="is-sr-only">Skip to main content</a>
<!-- start desktop header -->
<div class="columns is-vcentered is-hidden-mobile" id="cu-identity">
<div class="column" id="cu-logo">
<a href="https://www.cornell.edu/"><img src="/static/browse/0.3.4/images/icons/cu/cornell-reduced-white-SMALL.svg" alt="Cornell University" /></a>
</div><!-- /from April 7 at 1:00 AM to May 29 at 21:40 --><!-- /from May 2 at 1:00 AM to May 5 at 9:45 AM --><div class="column banner-minimal">
<a href="https://tech.cornell.edu/arxiv/" target="_blank">Learn about arXiv becoming an independent nonprofit.</a>
</div><div class="column" id="support-ack">
<span id="support-ack-url">We gratefully acknowledge support from the Simons Foundation, <a href="https://info.arxiv.org/about/ourmembers.html">member institutions</a>, and all contributors.</span>
<a href="https://info.arxiv.org/about/donate.html" class="btn-header-donate">Donate</a>
</div>
</div>
<div id="header" class="is-hidden-mobile">
<a aria-hidden="true" tabindex="-1" href="/IgnoreMe"></a>
<div class="header-breadcrumbs is-hidden-mobile">
<a href="/"><img src="/static/browse/0.3.4/images/arxiv-logo-one-color-white.svg" alt="arxiv logo" style="height:40px;"/></a> <span>&gt;</span> <a href="/list/cs/recent">cs</a> <span>&gt;</span> arXiv:1805.06358
</div>
<div class="columns is-vcentered is-mobile" style="justify-content: flex-end;">
</div>
<div class="search-block level-right">
<form class="level-item mini-search" method="GET" action="https://arxiv.org/search">
<div class="field has-addons">
<div class="control">
<input class="input is-small" type="text" name="query" placeholder="Search..." aria-label="Search term or terms" />
<p class="help"><a href="https://info.arxiv.org/help">Help</a> | <a href="https://arxiv.org/search/advanced">Advanced Search</a></p>
</div>
<div class="control">
<div class="select is-small">
<select name="searchtype" aria-label="Field to search">
<option value="all" selected="selected">All fields</option>
<option value="title">Title</option>
<option value="author">Author</option>
<option value="abstract">Abstract</option>
<option value="comments">Comments</option>
<option value="journal_ref">Journal reference</option>
<option value="acm_class">ACM classification</option>
<option value="msc_class">MSC classification</option>
<option value="report_num">Report number</option>
<option value="paper_id">arXiv identifier</option>
<option value="doi">DOI</option>
<option value="orcid">ORCID</option>
<option value="author_id">arXiv author ID</option>
<option value="help">Help pages</option>
<option value="full_text">Full text</option>
</select>
</div>
</div>
<input type="hidden" name="source" value="header">
<button class="button is-small is-cul-darker">Search</button>
</div>
</form>
</div>
</div><!-- /end desktop header -->
<div class="mobile-header">
<div class="columns is-mobile">
<div class="column logo-arxiv"><a href="https://arxiv.org/"><img src="/static/browse/0.3.4/images/arxiv-logomark-small-white.svg" alt="arXiv logo" style="height:60px;" /></a></div>
<div class="column logo-cornell"><a href="https://www.cornell.edu/">
<picture>
<source media="(min-width: 501px)"
srcset="/static/browse/0.3.4/images/icons/cu/cornell-reduced-white-SMALL.svg 400w"
sizes="400w" />
<source srcset="/static/browse/0.3.4/images/icons/cu/cornell_seal_simple_black.svg 2x" />
<img src="/static/browse/0.3.4/images/icons/cu/cornell-reduced-white-SMALL.svg" alt="Cornell University Logo" />
</picture>
</a></div>
<div class="column nav" id="toggle-container">
<button class="toggle-control" aria-label="Open search"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" class="icon filter-white" aria-hidden="true"><path d="M505 442.7L405.3 343c-4.5-4.5-10.6-7-17-7H372c27.6-35.3 44-79.7 44-128C416 93.1 322.9 0 208 0S0 93.1 0 208s93.1 208 208 208c48.3 0 92.7-16.4 128-44v16.3c0 6.4 2.5 12.5 7 17l99.7 99.7c9.4 9.4 24.6 9.4 33.9 0l28.3-28.3c9.4-9.4 9.4-24.6.1-34zM208 336c-70.7 0-128-57.2-128-128 0-70.7 57.2-128 128-128 70.7 0 128 57.2 128 128 0 70.7-57.2 128-128 128z"/></svg></button>
<div class="mobile-toggle-block toggle-target">
<form class="mobile-search-form" method="GET" action="https://arxiv.org/search">
<div class="field has-addons">
<input class="input" type="text" name="query" placeholder="Search..." aria-label="Search term or terms" />
<input type="hidden" name="source" value="header">
<input type="hidden" name="searchtype" value="all">
<button class="button">GO</button>
</div>
</form>
</div>
<button class="toggle-control" aria-label="Open navigation menu"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512" class="icon filter-white" aria-hidden="true"><path d="M16 132h416c8.837 0 16-7.163 16-16V76c0-8.837-7.163-16-16-16H16C7.163 60 0 67.163 0 76v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16z"/></svg></button>
<div class="mobile-toggle-block toggle-target">
<nav class="mobile-menu" aria-labelledby="mobilemenulabel">
<h2 id="mobilemenulabel">quick links</h2>
<ul>
<li><a href="https://arxiv.org/login">Login</a></li>
<li><a href="https://info.arxiv.org/help">Help Pages</a></li>
<li><a href="https://info.arxiv.org/about">About</a></li>
</ul>
</nav>
</div>
</div>
</div>
</div><!-- /end mobile-header -->
</header>
<main>
<div id="content">
<!--
rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:trackback="http://madskills.com/public/xml/rss/module/trackback/">
<rdf:Description
rdf:about="/abs/1805.06358"
dc:identifier="/abs/1805.06358"
dc:title="Conflict-free Replicated Data Types (CRDTs)"
trackback:ping="/trackback/1805.06358" />
</rdf:RDF>
--><div id="abs-outer">
<div class="leftcolumn">
<div class="subheader">
<h1>Computer Science > Distributed, Parallel, and Cluster Computing</h1>
</div>
<div class="header-breadcrumbs-mobile">
<strong>arXiv:1805.06358</strong> (cs)
</div>
<link rel="stylesheet" type="text/css" href="/static/base/1.0.1/css/abs.css">
<div id="content-inner">
<div id="abs">
<div class="dateline">
[Submitted on 16 May 2018]</div>
<h1 class="title mathjax"><span class="descriptor">Title:</span>Conflict-free Replicated Data Types (CRDTs)</h1>
<div class="authors"><span class="descriptor">Authors:</span><a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Pregui%C3%A7a,+N" rel="nofollow">Nuno Preguiça</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Baquero,+C" rel="nofollow">Carlos Baquero</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Shapiro,+M" rel="nofollow">Marc Shapiro</a> (Regal)</div> <div id="download-button-info" hidden>View a PDF of the paper titled Conflict-free Replicated Data Types (CRDTs), by Nuno Pregui\c{c}a and 2 other authors</div>
<a class="mobile-submission-download" href="/pdf/1805.06358">View PDF</a>
<blockquote class="abstract mathjax">
<span class="descriptor">Abstract:</span>A conflict-free replicated data type (CRDT) is an abstract data type, with a well defined interface, designed to be replicated at multiple processes and exhibiting the following properties: (1) any replica can be modified without coordinating with another replicas; (2) when any two replicas have received the same set of updates, they reach the same state, deterministically, by adopting mathematically sound rules to guarantee state convergence.
</blockquote>
<!--CONTEXT-->
<div class="metatable">
<table summary="Additional metadata"><tr>
<td class="tablecell label">Subjects:</td>
<td class="tablecell subjects">
<span class="primary-subject">Distributed, Parallel, and Cluster Computing (cs.DC)</span></td>
</tr><tr>
<td class="tablecell label">Cite as:</td>
<td class="tablecell arxivid"><span class="arxivid"><a href="https://arxiv.org/abs/1805.06358">arXiv:1805.06358</a> [cs.DC]</span></td>
</tr>
<tr>
<td class="tablecell label">&nbsp;</td>
<td class="tablecell arxividv">(or <span class="arxivid">
<a href="https://arxiv.org/abs/1805.06358v1">arXiv:1805.06358v1</a> [cs.DC]</span> for this version)
</td>
</tr>
<tr>
<td class="tablecell label">&nbsp;</td>
<td class="tablecell arxivdoi"> <a href="https://doi.org/10.48550/arXiv.1805.06358" id="arxiv-doi-link">https://doi.org/10.48550/arXiv.1805.06358</a><div class="button-and-tooltip">
<button class="more-info" aria-describedby="more-info-desc-1">
<svg height="15" role="presentation" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path fill="currentColor" d="M256 8C119.043 8 8 119.083 8 256c0 136.997 111.043 248 248 248s248-111.003 248-248C504 119.083 392.957 8 256 8zm0 110c23.196 0 42 18.804 42 42s-18.804 42-42 42-42-18.804-42-42 18.804-42 42-42zm56 254c0 6.627-5.373 12-12 12h-88c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h12v-64h-12c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h64c6.627 0 12 5.373 12 12v100h12c6.627 0 12 5.373 12 12v24z" class=""></path></svg>
<span class="visually-hidden">Focus to learn more</span>
</button>
<!-- tooltip description -->
<div role="tooltip" id="more-info-desc-1">
<span class="left-corner"></span> arXiv-issued DOI via DataCite</div>
</div>
</td>
</tr> <tr>
<td class="tablecell label">Journal&nbsp;reference:</td>
<td class="tablecell jref">Sakr, Sherif and Zomaya, Albert. Encyclopedia of Big Data Technologies, Springer International Publishing, 2018, Encyclopedia of Big Data Technologies, 978-3-319-63962-8</td>
</tr>
<tr>
<td class="tablecell label">
<abbr title="Digital Object Identifier">Related DOI</abbr>:
</td>
<td class="tablecell doi"><a href="https://doi.org/10.1007/978-3-319-63962-8%5C_185-1" data-doi="10.1007/978-3-319-63962-8\_185-1" class="link-https link-external" rel="external noopener nofollow">https://doi.org/10.1007/978-3-319-63962-8%5C_185-1</a>
<!-- accessible tooltip example -->
<div class="button-and-tooltip">
<button class="more-info" aria-describedby="more-info-desc-1">
<svg height="15" role="presentation" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path fill="currentColor" d="M256 8C119.043 8 8 119.083 8 256c0 136.997 111.043 248 248 248s248-111.003 248-248C504 119.083 392.957 8 256 8zm0 110c23.196 0 42 18.804 42 42s-18.804 42-42 42-42-18.804-42-42 18.804-42 42-42zm56 254c0 6.627-5.373 12-12 12h-88c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h12v-64h-12c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h64c6.627 0 12 5.373 12 12v100h12c6.627 0 12 5.373 12 12v24z" class=""></path></svg>
<span class="visually-hidden">Focus to learn more</span>
</button>
<!-- tooltip description -->
<div role="tooltip" id="more-info-desc-1">
<span class="left-corner"></span>
DOI(s) linking to related resources
</div>
</div>
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="submission-history">
<h2>Submission history</h2> From: Marc Shapiro [<a href="/show-email/386ae749/1805.06358" rel="nofollow">view email</a>]&nbsp;[via CCSD proxy] <br/> <strong>[v1]</strong>
Wed, 16 May 2018 15:00:36 UTC (43 KB)<br/>
</div>
</div>
<!--end leftcolumn-->
<div class="extra-services"> <div class="full-text">
<a name="other"></a>
<span class="descriptor">Full-text links:</span>
<h2>Access Paper:</h2>
<ul>
<div id="download-button-info" hidden>
View a PDF of the paper titled Conflict-free Replicated Data Types (CRDTs), by Nuno Pregui\c{c}a and 2 other authors</div><li><a href="/pdf/1805.06358" aria-describedby="download-button-info" accesskey="f" class="abs-button download-pdf">View PDF</a></li><li><a href="/src/1805.06358" class="abs-button download-eprint">TeX Source
</a></li></ul>
<div class="abs-license"><a href="http://arxiv.org/licenses/nonexclusive-distrib/1.0/" title="Rights to this article">view license</a></div>
</div>
<!--end full-text--> <div class="browse">
<h3 class="browse-context-heading">Current browse context:</h3>
<div class="current">cs.DC</div>
<div class="prevnext">
<span class="arrow">
<a class="abs-button prev-url" href="/prevnext?id=1805.06358&amp;function=prev&amp;context=cs.DC"
accesskey="p" title="previous in cs.DC (accesskey p)" rel="nofollow">&lt;&nbsp;prev</a>
</span>
<span class="is-hidden-mobile">&nbsp; | &nbsp;</span> <span class="arrow">
<a class="abs-button next-url" href="/prevnext?id=1805.06358&amp;function=next&amp;context=cs.DC" accesskey="n"
title="next in cs.DC (accesskey n)" rel="nofollow">next&nbsp;&gt;</a>
</span><br/>
</div><div class="list">
<a class="abs-button abs-button-grey abs-button-small context-new" href="/list/cs.DC/new" rel="nofollow">new</a>
<span class="is-hidden-mobile"> | </span>
<a class="abs-button abs-button-grey abs-button-small context-recent" href="/list/cs.DC/recent" rel="nofollow">recent</a>
<span class="is-hidden-mobile"> | </span><a class="abs-button abs-button-grey abs-button-small context-id" href="/list/cs.DC/2018-05" rel="nofollow">2018-05</a>
</div><div class="abs-switch-cat">
Change to browse by:
<div class="switch context-change">
<a href="/abs/1805.06358?context=cs" rel="nofollow">cs</a><br class="is-hidden-mobile">
</div>
</div>
</div>
<div class="extra-ref-cite">
<h3>References &amp; Citations</h3>
<ul>
<li><a class="abs-button abs-button-small cite-ads" href="https://ui.adsabs.harvard.edu/abs/arXiv:1805.06358">NASA ADS</a></li><li><a class="abs-button abs-button-small cite-google-scholar" href="https://scholar.google.com/scholar_lookup?arxiv_id=1805.06358" target="_blank" rel="noopener">Google Scholar</a></li>
<li><a class="abs-button abs-button-small cite-semantic-scholar" href="https://api.semanticscholar.org/arXiv:1805.06358" target="_blank" rel="noopener">Semantic Scholar</a></li>
</ul>
<div style="clear:both;"></div>
</div>
<div class="dblp">
<h3><a href="https://dblp.uni-trier.de">DBLP</a> - CS Bibliography</h3>
<div class="list">
<a href="https://dblp.uni-trier.de/db/journals/corr/corr1805.html#abs-1805-06358" title="listing on DBLP">listing</a> | <a href="https://dblp.uni-trier.de/rec/bibtex/journals/corr/abs-1805-06358" title="DBLP bibtex record">bibtex</a> </div>
<div class="list">
<a href="https://dblp.uni-trier.de/search/author?author=Nuno%20M.%20Pregui%C3%A7a" title="DBLP author search">Nuno M. Preguiça</a><br/><a href="https://dblp.uni-trier.de/search/author?author=Carlos%20Baquero" title="DBLP author search">Carlos Baquero</a><br/><a href="https://dblp.uni-trier.de/search/author?author=Marc%20Shapiro" title="DBLP author search">Marc Shapiro</a> </div>
</div><div class='extra-ref-cite'>
<button type="button" id='bib-cite-trigger' class="bib-cite-button abs-button">export BibTeX citation</button>
<span id='bib-cite-loading' hidden='true'>Loading...</span>
</div>
<div id='bib-cite-modal' class='bib-modal' hidden='true'>
<div class='bib-modal-content'>
<div class='bib-modal-title'>
<h2>BibTeX formatted citation</h2>
<button type="button" class='bib-modal-close' aria-label="Close">&times;</button>
</div>
<div>
<textarea id='bib-cite-target' class="bib-citation-content" aria-label="loading the citation">loading...</textarea>
</div>
<div>
<span>Data provided by: </span>
<a id='bib-cite-source-api'></a>
</div>
</div>
</div><div class="bookmarks">
<div><h3>Bookmark</h3></div><a class="abs-button abs-button-grey abs-button-small" href="http://www.bibsonomy.org/BibtexHandler?requTask=upload&amp;url=https://arxiv.org/abs/1805.06358&amp;description=Conflict-free Replicated Data Types (CRDTs)"
title="Bookmark on BibSonomy">
<img src="/static/browse/0.3.4/images/icons/social/bibsonomy.png"
alt="BibSonomy"/>
</a>
<a class="abs-button abs-button-grey abs-button-small" href="https://reddit.com/submit?url=https://arxiv.org/abs/1805.06358&amp;title=Conflict-free Replicated Data Types (CRDTs)"
title="Bookmark on Reddit">
<img src="/static/browse/0.3.4/images/icons/social/reddit.png"
alt="Reddit"/>
</a>
</div> </div>
<!--end extra-services-->
<!-- LABS AREA -->
<div id="labstabs">
<div class="labstabs" role="tablist" aria-label="arXivLabs tools"><input type="radio" name="tabs" id="tabone"checked="checked"class="labs-tab-input">
<label for="tabone" role="tab" aria-selected="true" aria-controls="tabpanel-one" id="tab-label-one">Bibliographic Tools</label>
<div class="tab labs-display-bib" role="tabpanel" id="tabpanel-one" aria-labelledby="tab-label-one">
<h1>Bibliographic and Citation Tools</h1>
<div class="toggle">
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input id="bibex-toggle" type="checkbox" class="lab-toggle"
data-script-url="/static/browse/0.3.4/bibex/bibex.js?20241202">
<span class="slider"></span>
<span class="is-sr-only">Bibliographic Explorer Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-bibex">Bibliographic Explorer</span> <em>(<a href="https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer">What is the Explorer?</a>)</em>
</div>
</div>
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="connectedpapers-toggle"
type="checkbox"
class="lab-toggle"
data-script-url="/static/browse/0.3.4/js/connectedpapers.js"
aria-labelledby="label-for-connected-papers">
<span class="slider"></span>
<span class="is-sr-only">Connected Papers Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-connected-papers">Connected Papers</span> <em>(<a href="https://www.connectedpapers.com/about" target="_blank">What is Connected Papers?</a>)</em>
</div>
</div><div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="litmaps-toggle"
type="checkbox"
class="lab-toggle"
data-script-url="/static/browse/0.3.4/js/litmaps.js?20210617"
aria-labelledby="label-for-litmaps">
<span class="slider"></span>
<span class="is-sr-only">Litmaps Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-litmaps">Litmaps</span> <em>(<a href="https://www.litmaps.co/" target="_blank">What is Litmaps?</a>)</em>
</div>
</div>
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="scite-toggle"
type="checkbox"
class="lab-toggle"
data-script-url="/static/browse/0.3.4/js/scite.js?20210617"
aria-labelledby="label-for-scite">
<span class="slider"></span>
<span class="is-sr-only">scite.ai Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-scite">scite Smart Citations</span> <em>(<a href="https://www.scite.ai/" target="_blank">What are Smart Citations?</a>)</em>
</div>
</div>
</div>
<div class="labs-content-placeholder labs-display" style="display: none;"></div>
<div style="min-height: 15px" id="connectedpapers-output"></div>
<div style="min-height: 15px" id="litmaps-open-in"></div>
<div style="min-height: 15px" id="scite-open-in"></div>
</div>
<input type="radio" name="tabs" id="tabtwo" class="labs-tab-input">
<label for="tabtwo" role="tab" aria-selected="false" aria-controls="tabpanel-two" id="tab-label-two">Code, Data, Media</label>
<div class="tab" role="tabpanel" id="tabpanel-two" aria-labelledby="tab-label-two">
<h1>Code, Data and Media Associated with this Article</h1>
<div class="toggle">
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="alphaxiv-toggle"
data-script-url="/static/browse/0.3.4/js/alphaxiv.js"
type="checkbox" class="lab-toggle" aria-labelledby="label-for-alphaxiv">
<span class="slider"></span>
<span class="is-sr-only">alphaXiv Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-alphaxiv">alphaXiv</span> <em>(<a href="https://alphaxiv.org/" target="_blank">What is alphaXiv?</a>)</em>
</div>
</div>
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="catalyzex-toggle"
data-script-url="/static/browse/0.3.4/js/catalyzex.js"
type="checkbox" class="lab-toggle" aria-labelledby="label-for-cx">
<span class="slider"></span>
<span class="is-sr-only">Links to Code Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-cx">CatalyzeX Code Finder for Papers</span> <em>(<a href="https://www.catalyzex.com" target="_blank">What is CatalyzeX?</a>)</em>
</div>
</div>
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="dagshub-toggle"
data-script-url="/static/browse/0.3.4/js/dagshub.js"
type="checkbox" class="lab-toggle" aria-labelledby="label-for-dagshub">
<span class="slider"></span>
<span class="is-sr-only">DagsHub Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-dagshub">DagsHub</span> <em>(<a href="https://dagshub.com/" target="_blank">What is DagsHub?</a>)</em>
</div>
</div>
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="gotitpub-toggle"
data-script-url="/static/browse/0.3.4/js/gotitpub.js"
type="checkbox" class="lab-toggle" aria-labelledby="label-for-gotitpub">
<span class="slider"></span>
<span class="is-sr-only">GotitPub Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-gotitpub">Gotit.pub</span> <em>(<a href="http://gotit.pub/faq" target="_blank">What is GotitPub?</a>)</em>
</div>
</div>
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="huggingface-toggle"
data-script-url="/static/browse/0.3.4/js/huggingface.js"
type="checkbox" class="lab-toggle" aria-labelledby="label-for-huggingface">
<span class="slider"></span>
<span class="is-sr-only">Huggingface Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-huggingface">Hugging Face</span> <em>(<a href="https://huggingface.co/huggingface" target="_blank">What is Huggingface?</a>)</em>
</div>
</div>
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="sciencecast-toggle"
data-script-url="/static/browse/0.3.4/js/sciencecast.js"
type="checkbox" class="lab-toggle" aria-labelledby="label-for-sciencecast">
<span class="slider"></span>
<span class="is-sr-only">ScienceCast Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-sciencecast">ScienceCast</span> <em>(<a href="https://sciencecast.org/welcome" target="_blank">What is ScienceCast?</a>)</em>
</div>
</div>
</div>
<div id="alphaxiv-output" style="display:none"></div>
<div id="catalyzex-output" style="display:none"></div>
<div id="dagshub-output" style="display:none"></div>
<div id="gotitpub-output" style="display:none"></div>
<div id="sciencecast-output" style="display:none"></div>
<div id="huggingface-output" style="display:none"></div>
</div>
<input type="radio" name="tabs" id="labstabs-demos-input" class="labs-tab-input">
<label for="labstabs-demos-input" id="labstabs-demos-label" role="tab" aria-selected="false" aria-controls="tabpanel-demos">Demos</label>
<div class="tab" role="tabpanel" id="tabpanel-demos" aria-labelledby="labstabs-demos-label">
<h1>Demos</h1>
<div class="toggle">
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="replicate-toggle"
data-script-url="/static/browse/0.3.4/js/replicate.js"
type="checkbox" class="lab-toggle" aria-labelledby="label-for-replicate">
<span class="slider"></span>
<span class="is-sr-only">Replicate Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-replicate">Replicate</span> <em>(<a href="https://replicate.com/docs/arxiv/about" target="_blank">What is Replicate?</a>)</em>
</div>
</div>
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="spaces-toggle"
data-script-url="/static/browse/0.3.4/js/spaces.js"
type="checkbox" class="lab-toggle" aria-labelledby="label-for-spaces">
<span class="slider"></span>
<span class="is-sr-only">Spaces Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-spaces">Hugging Face Spaces</span> <em>(<a href="https://huggingface.co/docs/hub/spaces" target="_blank">What is Spaces?</a>)</em>
</div>
</div>
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input
id="txyz-toggle"
data-script-url="/static/browse/0.3.4/js/txyz.js"
type="checkbox" class="lab-toggle" aria-labelledby="label-for-txyz">
<span class="slider"></span>
<span class="is-sr-only">Spaces Toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-txyz">TXYZ.AI</span> <em>(<a href="https://txyz.ai" target="_blank">What is TXYZ.AI?</a>)</em>
</div>
</div>
</div>
<div id="replicate-output"></div>
<div id="spaces-output"></div>
<div id="txyz-output"></div>
</div>
<input type="radio" name="tabs" id="tabfour" class="labs-tab-input">
<label for="tabfour" role="tab" aria-selected="false" aria-controls="tabpanel-four" id="tab-label-four">Related Papers</label>
<div class="tab" role="tabpanel" id="tabpanel-four" aria-labelledby="tab-label-four">
<h1>Recommenders and Search Tools</h1>
<div class="toggle">
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input id="influenceflower-toggle"
data-script-url="/static/browse/0.3.4/js/influenceflower.js"
type="checkbox" class="lab-toggle" aria-labelledby="label-for-influenceflower">
<span class="slider"></span>
<span class="is-sr-only">Link to Influence Flower</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-influenceflower">Influence Flower</span> <em>(<a href="https://influencemap.cmlab.dev/" target="_blank">What are Influence Flowers?</a>)</em>
</div>
</div>
<div class="columns is-mobile lab-row">
<div class="column lab-switch">
<label class="switch">
<input id="core-recommender-toggle" type="checkbox" class="lab-toggle" aria-labelledby="label-for-core">
<span class="slider"></span>
<span class="is-sr-only">Core recommender toggle</span>
</label>
</div>
<div class="column lab-name">
<span id="label-for-core">CORE Recommender</span> <em>(<a href="https://core.ac.uk/services/recommender">What is CORE?</a>)</em>
</div>
</div></div>
<div id="influenceflower-output"></div>
<div id="influenceflower-output-graph" style="display:none">
<ul class="flower-tabs">
<li class="active"><a class="btn tab-btn" onclick="openTab(event, 'tab-author')">Author</a></li>
<li><a class="btn tab-btn" onclick="openTab(event, 'tab-venue')">Venue</a></li>
<li><a class="btn tab-btn" onclick="openTab(event, 'tab-inst')">Institution</a></li>
<li><a class="btn tab-btn" onclick="openTab(event, 'tab-topic')">Topic</a></li>
</ul>
<div class="flower-tab-content">
<div class="tab-flower active" id="tab-author"><svg id="flower-graph-author"></svg></div>
<div class="tab-flower" id="tab-venue"><svg id="flower-graph-venue"></svg></div>
<div class="tab-flower" id="tab-inst"><svg id="flower-graph-inst"></svg></div>
<div class="tab-flower" id="tab-topic"><svg id="flower-graph-topic"></svg></div>
</div>
</div>
<div id="coreRecommenderOutput"></div>
<div id="iarxivOutput"></div>
</div>
<input type="radio" name="tabs" id="tabfive" class="labs-tab-input">
<label for="tabfive" role="tab" aria-selected="false" aria-controls="tabpanel-five" id="tab-label-five">
About arXivLabs
</label>
<div class="tab" role="tabpanel" id="tabpanel-five" aria-labelledby="tab-label-five">
<div class="columns">
<div class="column">
<h1>arXivLabs: experimental projects with community collaborators</h1>
<p>arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.</p>
<p>Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.</p>
<p>Have an idea for a project that will add value for arXiv's community? <a href="https://info.arxiv.org/labs/index.html"><strong>Learn more about arXivLabs</strong></a>.</p>
</div>
<div class="column is-narrow is-full-mobile">
<p class="icon-labs"><svg xmlns="http://www.w3.org/2000/svg" role="presentation" viewBox="0 0 635.572 811"><path d="M175.6 676v27h-27v-27zm-54 27v27h27v-27zm-27 27v27h27v-27zm396-54v27h-27v-27zm0 27v27h27v-27zm27 27v27h27v-27zm-27-414h27v27h-27zm27 0h27v-27h-27zm27-27h27v-27h-27zm-396 45h-27v-27h27zm-27-54h-27v27h27zm-27-27h-27v27h27z"/><path d="M94.6 730v27h-27v-27zm477 0v27h-27v-27zm-27-495h27v27h-27zm-450 18h-27v-27h27zm477 9h27v27h-27zm-54 495h27v27h-27zm-423 0h27v27h-27zm-54-504h27v27h-27z" fill="#666"/><path d="M67.6 730v27h-27v-27zm54 54v27h-27v-27zm0-108v27h27v-27zm-27 27v27h27v-27zm-81 0v27h27v-27zm585 27v27h-27v-27zm-108-54v27h27v-27zm27 27v27h27v-27zm81 0v27h27v-27zm-54-495h27v27h-27zm-54 108h27v-27h-27zm27-27h27v-27h-27zm0-81h27v-27h-27zm-423 18h-27v-27h27zm54 54h-27v27h27zm-27-27h-27v27h27zm0-81h-27v27h27zm423 612v27h-27v-27zm81-522v27h-27v-27zm-585-9v27h-27v-27z" fill="#999"/><path d="M94.6 784v27h-27v-27zm-27-27v27h27v-27zm-27-54v27h27v-27zm27 0v27h27v-27zm0-27v27h27v-27zm27 0v27h27v-27zm0-27v27h27v-27zm27 0v27h27v-27zm-108 81v27h27v-27zm558 54v27h-27v-27zm-27-27v27h27v-27zm27-54v27h27v-27zm-27 0v27h27v-27zm0-27v27h27v-27zm-27 0v27h27v-27zm0-27v27h27v-27zm-27 0v27h27v-27zm108 81v27h27v-27zm0-495h27v27h-27zm-27 27h27v-27h-27zm-54-27h27v-27h-27zm0 27h27v-27h-27zm-27 0h27v-27h-27zm0 27h27v-27h-27zm-27 0h27v-27h-27zm0 27h27v-27h-27zm81-108h27v-27h-27zm-504 45h-27v-27h27zm27-27h-27v27h27zm54-27h-27v27h27zm0 27h-27v27h27zm27 0h-27v27h27zm0 27h-27v27h27zm27 0h-27v27h27zm0 27h-27v27h27zm-81-108h-27v27h27z" fill="#ccc"/><path d="M598.6 665.1H41.5C-76.5 667 176 280.2 176 280.2h53a46.5 46.5 0 0162.8-56.3 29.2 29.2 0 1128.5 35.9h-1a46.5 46.5 0 01-1.5 20.3l142.5-.1s255.3 387 138.3 385.1zM291 181a29.3 29.3 0 10-29.2-29.3A29.3 29.3 0 00291 181zm65.4-66.8a22.4 22.4 0 10-22.5-22.4 22.4 22.4 0 0022.5 22.4z" fill="#fc0"/><path d="M245.5 172V10h153v162s324 495 198 495h-558c-126 0 207-495 207-495zm126 54h56m-13 72h56m-9 72h56m-20 72h56m-22 72h56m-29 72h56m-457-45c20.8 41.7 87.3 81 160.7 81 72.1 0 142.1-38.2 163.4-81" fill="none" stroke="#000" stroke-miterlimit="10" stroke-width="20"/><path d="M273.3 421.7c0 31-9.8 56.3-21.9 56.3s-21.8-25.2-21.8-56.3 9.8-56.3 21.8-56.3 21.9 25.2 21.9 56.3zm114.4-56.3c-12 0-21.8 25.2-21.8 56.3s9.7 56.3 21.8 56.3 21.9-25.2 21.9-56.3-9.8-56.3-21.9-56.3zM150.1 526.6c-18.2 6.7-27.5 22.9-23.2 30.2s14.8-5.5 33-12.2 37.4-4.9 33-12.2-24.5-12.6-42.8-5.8zm296 5.8c-4.2 7.3 14.9 5.5 33.1 12.2s28.7 19.5 33 12.2-5-23.5-23.2-30.2-38.5-1.5-42.8 5.8z"/></svg></p>
</div>
</div>
</div>
</div>
</div>
<!-- END LABS AREA -->
<div class="endorsers">
<a href="/auth/show-endorsers/1805.06358" class="endorser-who" rel="nofollow">Which authors of this paper are endorsers?</a> |
<a id="mathjax_toggle" href="javascript:setMathjaxCookie()">Disable MathJax</a> (<a href="https://info.arxiv.org/help/mathjax.html">What is MathJax?</a>)
<span class="help" style="font-style: normal; float: right; margin-top: 0; margin-right: 1em;"></span>
</div>
<script type="text/javascript" language="javascript">mathjaxToggle();</script>
</div>
</div>
</main>
<footer style="clear: both;">
<div class="columns is-desktop" role="navigation" aria-label="Secondary" style="margin: -0.75em -0.75em 0.75em -0.75em">
<!-- Macro-Column 1 -->
<div class="column" style="padding: 0;">
<div class="columns">
<div class="column">
<ul style="list-style: none; line-height: 2;">
<li><a href="https://info.arxiv.org/about">About</a></li>
<li><a href="https://info.arxiv.org/help">Help</a></li>
</ul>
</div>
<div class="column">
<ul style="list-style: none; line-height: 2;">
<li>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" class="icon filter-black" role="presentation"><title>contact arXiv</title><desc>Click here to contact arXiv</desc><path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"/></svg>
<a href="https://info.arxiv.org/help/contact.html"> Contact</a>
</li>
<li>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" class="icon filter-black" role="presentation"><title>subscribe to arXiv mailings</title><desc>Click here to subscribe</desc><path d="M476 3.2L12.5 270.6c-18.1 10.4-15.8 35.6 2.2 43.2L121 358.4l287.3-253.2c5.5-4.9 13.3 2.6 8.6 8.3L176 407v80.5c0 23.6 28.5 32.9 42.5 15.8L282 426l124.6 52.2c14.2 6 30.4-2.9 33-18.2l72-432C515 7.8 493.3-6.8 476 3.2z"/></svg>
<a href="https://info.arxiv.org/help/subscribe"> Subscribe</a>
</li>
</ul>
</div>
</div>
</div>
<!-- End Macro-Column 1 -->
<!-- Macro-Column 2 -->
<div class="column" style="padding: 0;">
<div class="columns">
<div class="column">
<ul style="list-style: none; line-height: 2;">
<li><a href="https://info.arxiv.org/help/license/index.html">Copyright</a></li>
<li><a href="https://info.arxiv.org/help/policies/privacy_policy.html">Privacy Policy</a></li>
</ul>
</div>
<div class="column sorry-app-links">
<ul style="list-style: none; line-height: 2;">
<li><a href="https://info.arxiv.org/help/web_accessibility.html">Web Accessibility Assistance</a></li>
<li>
<p class="help">
<a class="a11y-main-link" href="https://status.arxiv.org" target="_blank">arXiv Operational Status <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 256 512" class="icon filter-dark_grey" role="presentation"><path d="M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z"/></svg></a><br>
</p>
</li>
</ul>
</div>
</div>
</div> <!-- end MetaColumn 2 -->
<!-- End Macro-Column 2 -->
</div>
</footer>
</div>
<script src="/static/base/1.0.1/js/member_acknowledgement.js"></script>
</body>
</html>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View file

@ -0,0 +1,676 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<meta charset="utf-8" />
<title>In Search of an Understandable Consensus Algorithm (Extended Version)</title>
<meta name="generator" content="Org mode" />
<meta name="author" content="Diego Ongaro and John Ousterhout, Stanford University" />
<link rel="stylesheet" type="text/css" href="/themes/readtheorg/styles/readtheorg/css/htmlize.css"/>
<link rel="stylesheet" type="text/css" href="/themes/readtheorg/styles/readtheorg/css/readtheorg.css"/>
<script src="/themes/jquery/2.1.3/jquery.min.js"></script>
<script src="/themes/bootstrap/3.3.4/js/bootstrap.min.js"></script>
<script type="text/javascript" src="/themes/readtheorg/styles/lib/js/jquery.stickytableheaders.min.js"></script>
<script type="text/javascript" src="/themes/readtheorg/styles/readtheorg/js/readtheorg.js"></script>
</head>
<body>
<div id="org-div-home-and-up">
<a accesskey="h" href="../index.html"> UP </a>
|
<a accesskey="H" href="paperlist.html"> HOME </a>
</div><div id="content">
<h1 class="title">In Search of an Understandable Consensus Algorithm (Extended Version)</h1>
<div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#orgf960eb7">DECLARATION</a></li>
<li><a href="#orgd05d3d0">Abstract</a></li>
<li><a href="#orge9839bb">1. Introduction</a></li>
<li><a href="#orgae8bb96">2. Replicated state machines</a></li>
<li><a href="#org1fe8017">3. What's wrong with Paxos?</a></li>
<li><a href="#orgfc9c6b4">4. Designing for understandability</a></li>
<li><a href="#orge3d8f10">5. The Raft consensus algorithm</a>
<ul>
<li><a href="#org752f98e">5.1. Raft basics</a></li>
<li><a href="#org291797f">5.2. Leader election</a></li>
<li><a href="#org44bb0a6">5.3. Log replication</a></li>
</ul>
</li>
</ul>
</div>
</div>
<div id="outline-container-orgf960eb7" class="outline-2">
<h2 id="orgf960eb7">DECLARATION</h2>
<div class="outline-text-2" id="text-orgf960eb7">
<p>
This page is the In Search of an Understandable Consensus Algorithm
(Extended Version) paper. Original Paper Link
is: <a href="https://raft.github.io/raft.pdf">https://raft.github.io/raft.pdf</a>
</p>
<p>
In Search of an Understandable Consensus Algorithm<br />
(Extended Version)<br />
Diego Ongaro and John Ousterhout<br />
Stanford University
</p>
</div>
</div>
<div id="outline-container-orgd05d3d0" class="outline-2">
<h2 id="orgd05d3d0">Abstract</h2>
<div class="outline-text-2" id="text-orgd05d3d0">
<p>
Raft is a consensus algorithm for managing a replicated log. It produces a
result equivalent to (multi-)Paxos, and it is as efficient as Paxos, but its
structure is different from Paxos; this makes Raft more understandable than
Paxos and also provides a better foundation for building practical systems.
In order to enhance understandability, Raft separates the key elements of
consensus, such as leader election, log replication, and safety, and it enforces
a stronger degree of coherency to reduce the number of states that must be
considered. Results from a user study demonstrate that Raft is easier for
students to learn than Paxos. Raft also includes a new mechanism for changing
the cluster membership, which uses overlapping majorities to guarantee safety.
</p>
</div>
</div>
<div id="outline-container-orge9839bb" class="outline-2">
<h2 id="orge9839bb"><span class="section-number-2">1</span> Introduction</h2>
<div class="outline-text-2" id="text-1">
<p>
Consensus algorithms allow a collection of machines to work as a coherent group
that can survive the failures of some of its members. Because of this, they play
a key role in building reliable large-scale software systems. Paxos [15, 16] has
dominated the discussion of consensus algorithms over the last decade: most
implementations of consensus are based on Paxos or influenced by it, and Paxos
has become the primary vehicle used to teach students about consensus.
</p>
<p>
Unfortunately, Paxos is quite difficult to understand, in spite of numerous
attempts to make it more approachable. Furthermore, its architecture requires
complex changes to support practical systems. As a result, both system builders
and students struggle with Paxos.
</p>
<p>
After struggling with Paxos ourselves, we set out to find a new consensus
algorithm that could provide a better foundation for system building and
education. Our approach was unusual in that our primary goal was
understandability: could we define a consensus algorithm for practical systems
and describe it in a way that is significantly easier to learn than Paxos?
Furthermore, we wanted the algorithm to facilitate the development of intuitions
that are essential for system builders. It was important not just for the
algorithm to work, but for it to be obvious why it works.
</p>
<p>
The result of this work is a consensus algorithm called Raft. In designing Raft
we applied specific techniques to improve understandability,including
decomposition (Raft separates leader election, log replication, and safety) and
This tech report is an extended version of [32]; additional material is noted
with a gray bar in the margin. Published May 20, 2014. state space reduction
(relative to Paxos, Raft reduces the degree of nondeterminism and the ways
servers can be inconsistent with each other). A user study with 43 students
at two universities shows that Raft is significantly easier to understand than
Paxos: after learning both algorithms, 33 of these students were able to answer
questions about Raft better than questions about Paxos.
</p>
<p>
Raft is similar in many ways to existing consensus algorithms (most notably, Oki
and Liskov's Viewstamped Replication [29, 22]), but it has several novel
features:
</p>
<p>
1/. Strong leader: Raft uses a stronger form of leadership than other consensus
algorithms. For example, log entries only flow from the leader to other servers.
This simplifies the management of the replicated log and makes Raft easier to
understand.
</p>
<p>
2/. Leader election: Raft uses randomized timers to elect leaders. This adds
only a small amount of mechanism to the heartbeats already required for any
consensus algorithm, while resolving conflicts simply and rapidly.
</p>
<p>
3/. Membership changes: Raft's mechanism for changing the set of servers in the
cluster uses a new joint consensus approach where the majorities of two
different configurations overlap during transitions. This allows the cluster to
continue operating normally during configuration changes.
</p>
<p>
We believe that Raft is superior to Paxos and other consensus algorithms, both
for educational purposes and as a foundation for implementation. It is simpler
and more understandable than other algorithms; it is described completely enough
to meet the needs of a practical system; it has several open-source
implementations and is used by several companies; its safety properties have
been formally specified and proven; and its efficiency is comparable to other
algorithms.
</p>
<p>
The remainder of the paper introduces the replicated state machine problem
(Section 2), discusses the strengths and weaknesses of Paxos (Section 3),
describes our general approach to understandability (Section 4), presents the
Raft consensus algorithm (Sections 5–8), evaluates Raft (Section 9), and
discusses related work (Section 10).
</p>
</div>
</div>
<div id="outline-container-orgae8bb96" class="outline-2">
<h2 id="orgae8bb96"><span class="section-number-2">2</span> Replicated state machines</h2>
<div class="outline-text-2" id="text-2">
<p>
Consensus algorithms typically arise in the context of replicated state machines
[37]. In this approach, state machines on a collection of servers compute
identical copies of the same state and can continue operating even if some of
the servers are down. Replicated state machines are used to solve a variety of
fault tolerance problems in distributed systems. For example, large-scale
systems that have a single cluster leader, such as GFS [8], HDFS [38], and
RAMCloud [33], typically use a separate replicated state machine to manage
leader election and store configuration information that must survive leader
crashes. Examples of replicated state machines include Chubby [2] and
ZooKeeper [11].
</p>
<div id="org1bdf0c5" class="figure">
<p><img src="img/raft-figure1.jpg" alt="raft-figure1.jpg" />
</p>
<p><span class="figure-number">Figure 1: </span>Replicated state machine architecture. The consensus algorithm manages a replicated log containing state machine commands from clients. The state machines process identical sequences of commands from the logs, so they produce the same outputs.</p>
</div>
<p>
Replicated state machines are typically implemented using a replicated log, as
shown in Figure 1. Each server stores a log containing a series of commands,
which its state machine executes in order. Each log contains the same commands
in the same order, so each state machine processes the same sequence of
commands. Since the state machines are deterministic, each computes the same
state and the same sequence of outputs.
</p>
<p>
Keeping the replicated log consistent is the job of the consensus algorithm.
The consensus module on a server receives commands from clients and adds them to
its log. It communicates with the consensus modules on other servers to ensure
that every log eventually contains the same requests in the same order, even if
some servers fail. Once commands are properly replicated, each server's state
machine processes them in log order, and the outputs are returned to clients. As
a result, the servers appear to form a single, highly reliable state machine.
</p>
<p>
Consensus algorithms for practical systems typically have the following
properties:
</p>
<p>
• They ensure <i>safety</i> (never returning an incorrect result) under all
non-Byzantine conditions, including network delays, partitions, and packet loss,
duplication, and reordering.
</p>
<p>
• They are fully functional (available) as long as any majority of the servers
are operational and can communicate with each other and with clients. Thus, a
typical cluster of five servers can tolerate the failure of any two servers.
Servers are assumed to fail by stopping; they may later recover from state on
stable storage and rejoin the cluster.
</p>
<p>
• They do not depend on timing to ensure the consistency of the logs: faulty
clocks and extreme message delays can, at worst, cause availability problems.
</p>
<p>
• In the common case, a command can complete as soon as a majority of the
cluster has responded to a single round of remote procedure calls; a minority of
slow servers need not impact overall system performance.
</p>
</div>
</div>
<div id="outline-container-org1fe8017" class="outline-2">
<h2 id="org1fe8017"><span class="section-number-2">3</span> What's wrong with Paxos?</h2>
<div class="outline-text-2" id="text-3">
<p>
Over the last ten years, Leslie Lamport's Paxos protocol [15] has become almost
synonymous with consensus: it is the protocol most commonly taught in courses,
and most implementations of consensus use it as a starting point. Paxos first
defines a protocol capable of reaching agreement on a single decision, such as
a single replicated log entry. We refer to this subset as single-decree Paxos.
Paxos then combines multiple instances of this protocol to facilitate a series
of decisions such as a log (multi-Paxos). Paxos ensures both safety and
liveness, and it supports changes in cluster membership. Its correctness has
been proven, and it is efficient in the normal case.
</p>
<p>
Unfortunately, Paxos has two significant drawbacks. The first drawback is that
Paxos is exceptionally difficult to understand. The full explanation [15] is
notoriously opaque; few people succeed in understanding it, and only with great
effort. As a result, there have been several attempts to explain Paxos in
simpler terms [16, 20, 21]. These explanations focus on the single-decree
subset, yet they are still challenging. In an informal survey of attendees at
NSDI 2012, we found few people who were comfortable with Paxos, even among
seasoned researchers. We struggled with Paxos ourselves; we were not able to
understand the complete protocol until after reading several simplified
explanations and designing our own alternative protocol, a process that took
almost a year.
</p>
<p>
We hypothesize that Paxos' opaqueness derives from its choice of the
single-decree subset as its foundation. Single-decree Paxos is dense and subtle:
it is divided into two stages that do not have simple intuitive explanations and
cannot be understood independently. Because of this, it is difficult to develop
intuitions about why the single-decree protocol works. The composition rules for
multi-Paxos add significant additional complexity and subtlety. We believe that
the overall problem of reaching consensus on multiple decisions (i.e., a log
instead of a single entry) can be decomposed in other ways that are more direct
and obvious.
</p>
<p>
The second problem with Paxos is that it does not provide a good foundation for
building practical implementations. One reason is that there is no widely
agreed-upon algorithm for multi-Paxos. Lamport's descriptions are mostly about
single-decree Paxos; he sketched possible approaches to multi-Paxos, but many
details are missing. There have been several attempts to flesh out and optimize
Paxos, such as [26], [39], and [13], but these differ from each other and from
Lamport's sketches. Systems such as Chubby [4] have implemented Paxos-like
algorithms, but in most cases their details have not been published.
</p>
<p>
Furthermore, the Paxos architecture is a poor one for building practical
systems; this is another consequence of the single-decree decomposition. For
example, there is little benefit to choosing a collection of log entries
independently and then melding them into a sequential log; this just adds
complexity. It is simpler and more efficient to design a system around a log,
where new entries are appended sequentially in a constrained order. Another
problem is that Paxos uses a symmetric peer-to-peer approach at its core (though
it eventually suggests a weak form of leadership as a performance optimization).
This makes sense in a simplified world where only one decision will be made, but
few practical systems use this approach. If a series of decisions must be made,
it is simpler and faster to first elect a leader, then have the leader
coordinate the decisions.
</p>
<p>
As a result, practical systems bear little resemblance to Paxos. Each
implementation begins with Paxos, discovers the difficulties in implementing it,
and then develops a significantly different architecture. This is timeconsuming
and error-prone, and the difficulties of understanding Paxos exacerbate the
problem. Paxos' formulation may be a good one for proving theorems about its
correctness, but real implementations are so different from Paxos that the
proofs have little value. The following comment from the Chubby implementers is
typical:
</p>
<blockquote>
<p>
There are significant gaps between the description of the Paxos algorithm and
the needs of a real-world system. . . . the final system will be based on an
unproven protocol [4].
</p>
</blockquote>
<p>
Because of these problems, we concluded that Paxos does not provide a good
foundation either for system building or for education. Given the importance of
consensus in large-scale software systems, we decided to see if we could design
an alternative consensus algorithm with better properties than Paxos. Raft is
the result of that experiment.
</p>
</div>
</div>
<div id="outline-container-orgfc9c6b4" class="outline-2">
<h2 id="orgfc9c6b4"><span class="section-number-2">4</span> Designing for understandability</h2>
<div class="outline-text-2" id="text-4">
<p>
We had several goals in designing Raft: it must provide a complete and practical
foundation for system building, so that it significantly reduces the amount of
design work required of developers; it must be safe under all conditions and
available under typical operating conditions; and it must be efficient for
common operations. But our most important goal—and most difficult challenge
was understandability. It must be possible for a large audience to understand
the algorithm comfortably. In addition, it must be possible to develop
intuitions about the algorithm, so that system builders can make the extensions
that are inevitable in real-world implementations.
</p>
<p>
There were numerous points in the design of Raft where we had to choose among
alternative approaches. In these situations we evaluated the alternatives
based on understandability: how hard is it to explain each alternative (for
example, how complex is its state space, and does it have subtle implications?),
and how easy will it be for a reader to completely understand the approach and
its implications?
</p>
<p>
We recognize that there is a high degree of subjectivity in such analysis;
nonetheless, we used two techniques that are generally applicable. The first
technique is the well-known approach of problem decomposition: wherever
possible, we divided problems into separate pieces that could be solved,
explained, and understood relatively independently. For example, in Raft we
separated leader election, log replication, safety, and membership changes.
</p>
<p>
Our second approach was to simplify the state space by reducing the number of
states to consider, making the system more coherent and eliminating
nondeterminism where possible. Specifically, logs are not allowed to have holes,
and Raft limits the ways in which logs can become inconsistent with each other.
Although in most cases we tried to eliminate nondeterminism, there are some
situations where nondeterminism actually improves understandability.
In particular, randomized approaches introduce nondeterminism, but they tend to
reduce the state space by handling all possible choices in a similar fashion
("choose any; it doesn't matter"). We used randomization to simplify the Raft
leader election algorithm.
</p>
</div>
</div>
<div id="outline-container-orge3d8f10" class="outline-2">
<h2 id="orge3d8f10"><span class="section-number-2">5</span> The Raft consensus algorithm</h2>
<div class="outline-text-2" id="text-5">
<p>
Raft is an algorithm for managing a replicated log of the form described in
Section 2. Figure 2 summarizes the algorithm in condensed form for reference,
and Figure 3 lists key properties of the algorithm; the elements of these
figures are discussed piecewise over the rest of this section.
</p>
<p>
<img src="img/raft-figure2-1.jpg" alt="raft-figure2-1.jpg" />
<img src="img/raft-figure2-2.jpg" alt="raft-figure2-2.jpg" />
<img src="img/raft-figure2-3.jpg" alt="raft-figure2-3.jpg" />
</p>
<div id="orgf3f186a" class="figure">
<p><img src="img/raft-figure2-4.jpg" alt="raft-figure2-4.jpg" />
</p>
<p><span class="figure-number">Figure 2: </span>A condensed summary of the Raft consensus algorithm (excluding membership changes and log compaction). The server behavior in the upper-left box is described as a set of rules that trigger independently and repeatedly. Section numbers such as §5.2 indicate where particular features are discussed.A formal specification [31] describes the algorithm more precisely.</p>
</div>
<p>
Raft implements consensus by first electing a distinguished leader, then giving
the leader complete responsibility for managing the replicated log. The leader
accepts log entries from clients, replicates them on other servers, and tells
servers when it is safe to apply log entries to their state machines. Having a
leader simplifies the management of the replicated log. For example, the leader
can decide where to place new entries in the log without consulting other
servers, and data flows in a simple fashion from the leader to other servers.
A leader can fail or become disconnected from the other servers, in which case
a new leader is elected.
</p>
<p>
Given the leader approach, Raft decomposes the consensus problem into three
relatively independent subproblems, which are discussed in the subsections that
follow:
</p>
<p>
• Leader election: a new leader must be chosen when an existing leader fails
(Section 5.2).
</p>
<p>
• Log replication: the leader must accept log entries from clients and replicate
them across the cluster, forcing the other logs to agree with its own (Section
5.3).
• Safety: the key safety property for Raft is the State Machine Safety Property
in Figure 3: if any server has applied a particular log entry to its state
machine, then no other server may apply a different command for the same log
index. Section 5.4 describes how Raft ensures this property; the solution
involves an additional restriction on the election mechanism described in
Section 5.2.
</p>
<p>
After presenting the consensus algorithm, this section discusses the issue of
availability and the role of timing in the system.
</p>
<div id="org159660c" class="figure">
<p><img src="img/raft-figure3.jpg" alt="raft-figure3.jpg" />
</p>
<p><span class="figure-number">Figure 3: </span>Raft guarantees that each of these properties is true at all times. The section numbers indicate where each property is discussed.</p>
</div>
</div>
<div id="outline-container-org752f98e" class="outline-3">
<h3 id="org752f98e"><span class="section-number-3">5.1</span> Raft basics</h3>
<div class="outline-text-3" id="text-5-1">
<p>
A Raft cluster contains several servers; five is a typical number, which allows
the system to tolerate two failures. At any given time each server is in one of
three states: leader, follower, or candidate. In normal operation there is
exactly one leader and all of the other servers are followers. Followers are
passive: they issue no requests on their own but simply respond to requests from
leaders and candidates. The leader handles all client requests (if a client
contacts a follower, the follower redirects it to the leader). The third state,
candidate, is used to elect a new leader as described in Section 5.2. Figure 4
shows the states and their transitions; the transitions are discussed below.
</p>
<div id="org3f316c7" class="figure">
<p><img src="img/raft-figure4.jpg" alt="raft-figure4.jpg" />
</p>
<p><span class="figure-number">Figure 4: </span>Server states. Followers only respond to requests from other servers. If a follower receives no communication, it becomes a candidate and initiates an election. A candidate that receives votes from a majority of the full cluster becomes the new leader. Leaders typically operate until they fail.</p>
</div>
<p>
Raft divides time into terms of arbitrary length, as shown in Figure 5. Terms
are numbered with consecutive integers. Each term begins with an election, in
which one or more candidates attempt to become leader as described in Section
5.2. If a candidate wins the election, then it serves as leader for the rest of
the term. In some situations an election will result in a split vote. In this
case the term will end with no leader; a new term (with a new election) will
begin shortly. Raft ensures that there is at most one leader in a given term.
</p>
<div id="orgc7ff282" class="figure">
<p><img src="img/raft-figure5.jpg" alt="raft-figure5.jpg" />
</p>
<p><span class="figure-number">Figure 5: </span>Time is divided into terms, and each term begins with an election. After a successful election, a single leader manages the cluster until the end of the term. Some elections fail, in which case the term ends without choosing a leader. The transitions between terms may be observed at different times on different servers.</p>
</div>
<p>
Different servers may observe the transitions between terms at different times,
and in some situations a server may not observe an election or even entire
terms. Terms act as a logical clock [14] in Raft, and they allow servers to
detect obsolete information such as stale leaders. Each server stores a current
term number, which increases monotonically over time. Current terms are
exchanged whenever servers communicate; if one server's current term is smaller
than the other's, then it updates its current term to the larger value. If a
candidate or leader discovers that its term is out of date, it immediately
reverts to follower state. If a server receives a request with a stale term
number, it rejects the request.
</p>
<p>
Raft servers communicate using remote procedure calls (RPCs), and the basic
consensus algorithm requires only two types of RPCs. RequestVote RPCs are
initiated by candidates during elections (Section 5.2), and AppendEntries RPCs
are initiated by leaders to replicate log entries and to provide a form of
heartbeat (Section 5.3). Section 7 adds a third RPC for transferring snapshots
between servers. Servers retry RPCs if they do not receive a response in a
timely manner, and they issue RPCs in parallel for best performance.
</p>
</div>
</div>
<div id="outline-container-org291797f" class="outline-3">
<h3 id="org291797f"><span class="section-number-3">5.2</span> Leader election</h3>
<div class="outline-text-3" id="text-5-2">
<p>
Raft uses a heartbeat mechanism to trigger leader election. When servers start
up, they begin as followers. A server remains in follower state as long as it
receives valid RPCs from a leader or candidate. Leaders send periodic heartbeats
(AppendEntries RPCs that carry no log entries) to all followers in order to
maintain their authority. If a follower receives no communication over a period
of time called the election timeout, then it assumes there is no viable leader
and begins an election to choose a new leader.
</p>
<p>
To begin an election, a follower increments its current term and transitions to
candidate state. It then votes for itself and issues RequestVote RPCs in
parallel to each of the other servers in the cluster. A candidate continues in
this state until one of three things happens: (a) it wins the election, (b)
another server establishes itself as leader, or (c) a period of time goes by
with no winner. These outcomes are discussed separately in the paragraphs below.
</p>
<p>
A candidate wins an election if it receives votes from a majority of the servers
in the full cluster for the same term. Each server will vote for at most one
candidate in a given term, on a first-come-first-served basis (note: Section 5.4
adds an additional restriction on votes). The majority rule ensures that at most
one candidate can win the election for a particular term (the Election Safety
Property in Figure 3). Once a candidate wins an election, it becomes leader. It
then sends heartbeat messages to all of the other servers to establish its
authority and prevent new elections.
</p>
<p>
While waiting for votes, a candidate may receive an AppendEntries RPC from
another server claiming to be leader. If the leader's term (included in its RPC)
is at least as large as the candidate's current term, then the candidate
recognizes the leader as legitimate and returns to follower state. If the term
in the RPC is smaller than the candidate's current term, then the candidate
rejects the RPC and continues in candidate state.
</p>
<p>
The third possible outcome is that a candidate neither wins nor loses the
election: if many followers become candidates at the same time, votes could be
split so that no candidate obtains a majority. When this happens, each candidate
will time out and start a new election by incrementing its term and initiating
another round of RequestVote RPCs. However, without extra measures split votes
could repeat indefinitely.
</p>
<p>
Raft uses randomized election timeouts to ensure that split votes are rare and
that they are resolved quickly. To prevent split votes in the first place,
election timeouts are chosen randomly from a fixed interval (e.g., 150–300ms).
This spreads out the servers so that in most cases only a single server will
time out; it wins the election and sends heartbeats before any other servers
time out. The same mechanism is used to handle split votes. Each candidate
restarts its randomized election timeout at the start of an election, and it
waits for that timeout to elapse before starting the next election; this reduces
the likelihood of another split vote in the new election. Section 9.3 shows
that this approach elects a leader rapidly.
</p>
<p>
Elections are an example of how understandability guided our choice between
design alternatives. Initially we planned to use a ranking system: each
candidate was assigned a unique rank, which was used to select between
competing candidates. If a candidate discovered another candidate with higher
rank, it would return to follower state so that the higher ranking candidate
could more easily win the next election. We found that this approach created
subtle issues around availability (a lower-ranked server might need to time out
and become a candidate again if a higher-ranked server fails, but if it does so
too soon, it can reset progress towards electing a leader). We made adjustments
to the algorithm several times, but after each adjustment new corner cases
appeared. Eventually we concluded that the randomized retry approach is more
obvious and understandable.
</p>
</div>
</div>
<div id="outline-container-org44bb0a6" class="outline-3">
<h3 id="org44bb0a6"><span class="section-number-3">5.3</span> Log replication</h3>
<div class="outline-text-3" id="text-5-3">
<p>
Once a leader has been elected, it begins servicing client requests. Each client
request contains a command to be executed by the replicated state machines. The
leader appends the command to its log as a new entry, then issues AppendEntries
RPCs in parallel to each of the other servers to replicate the entry. When the
entry has been safely replicated (as described below), the leader applies the
entry to its state machine and returns the result of that execution to the
client. If followers crash or run slowly, or if network packets are lost, the
leader retries AppendEntries RPCs indefinitely (even after it has responded to
the client) until all followers eventually store all log entries.
</p>
<div id="org879421f" class="figure">
<p><img src="img/raft-figure6.jpg" alt="raft-figure6.jpg" />
</p>
<p><span class="figure-number">Figure 6: </span>Logs are composed of entries, which are numbered sequentially. Each entry contains the term in which it was created (the number in each box) and a command for the state machine. An entry is considered committed if it is safe for that entry to be applied to state machines.</p>
</div>
<p>
Logs are organized as shown in Figure 6. Each log entry stores a state machine
command along with the term number when the entry was received by the leader.
The term numbers in log entries are used to detect inconsistencies between logs
and to ensure some of the properties in Figure 3. Each log entry also has an
integer index identifying its position in the log.
</p>
<p>
The leader decides when it is safe to applyy a log en-
try to the state machines; such an entry is called commit-
ted. Raft guarantees that committed entries are durable
and will eventually be executed by all of the available
state machines. A log entry is committed once the leader
that created the entry has replicated it on a majority of
the servers (e.g., entry 7 in Figure 6). This also commits
all preceding entries in the leader's log, including entries
created by previous leaders. Section 5.4 discusses some
subtleties when applying this rule after leader changes,
and it also shows that this definition of commitment is
safe. The leader keeps track of the highest index it knows
to be committed, and it includes that index in future
AppendEntries RPCs (including heartbeats) so that the
other servers eventually find out. Once a follower learns
that a log entry is committed, it applies the entry to its
local state machine (in log order).
We designed the Raft log mechanism to maintain a high
level of coherency between the logs on different servers.
Not only does this simplify the system's behavior and
make it more predictable, but it is an important component
of ensuring safety. Raft maintains the following proper-
ties, which together constitute the Log Matching Property
in Figure 3:
</p>
</div>
</div>
</div>
<script src="https://utteranc.es/client.js"
repo="yygcode/yygcode.github.io"
issue-term="pathname"
label=" Utterances"
theme="github-light"
crossorigin="anonymous"
async>
</script>
<noscript>Please enable JavaScript to view the <a href="https://github.com/utterance">comments powered by utterances.</a></noscript>
</div>
<div id="postamble" class="status">
<div class="copyright">
2012-2020 Copyright&copy; <i> YANYG<br/>Powered by Emacs Orgmode</i>
</div>
</div>
</body>
</html>

View file

@ -0,0 +1,845 @@
<!doctype html>
<html>
<head>
<title>Changefeeds in RethinkDB - RethinkDB</title>
<meta charset="utf-8">
<meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta property="twitter:account_id" content="52351557" />
<link rel="alternate" type="application/rss+xml" title="RethinkDB: the open-source database for the realtime web" href="/feed.xml">
<link rel="stylesheet" media="screen" href="/assets/css/styles-02.css"/>
<link rel="stylesheet" media="screen" href="/assets/css/jekyll-github.css"/>
<link rel="icon" type="image/png" href="/favicon.png">
<!-- typekit -->
<script type="text/javascript" src="//use.typekit.net/lmb6cah.js"></script>
<script type="text/javascript">try{Typekit.load();}catch(e){}</script>
<!-- jquery -->
<script type="text/javascript" src="/assets/js/jquery-2.1.3.min.js"></script>
<!-- js-cookie -->
<script type="text/javascript" src="/assets/js/js.cookie.js"></script>
<!-- waypoints -->
<script type="text/javascript" src="/assets/js/waypoints/jquery.waypoints.min.js"></script>
<script type="text/javascript" src="/assets/js/waypoints/shortcuts/sticky.min.js"></script>
<script type="text/javascript" src="/assets/js/site.js"></script>
<!-- algolia -->
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/docsearch.js@2/dist/cdn/docsearch.min.css" />
<script>
(function(h,o,t,j,a,r){
h.hj=h.hj||function(){(h.hj.q=h.hj.q||[]).push(arguments)};
h._hjSettings={hjid:1540062,hjsv:6};
a=o.getElementsByTagName('head')[0];
r=o.createElement('script');r.async=1;
r.src=t+h._hjSettings.hjid+j+h._hjSettings.hjsv;
a.appendChild(r);
})(window,document,'https://static.hotjar.com/c/hotjar-','.js?sv=');
</script>
</head>
<body class="docs">
<div id="wrapper">
<!-- site nav -->
<nav class="mobile-menu push-menu-right">
<ul>
<li class="menu-trigger"><a href="/">Close</a></li>
<li><a href="/faq">FAQ</a></li>
<li><a href="/docs">Docs</a></li>
<li><a href="/api">API</a></li>
<li><a href="/community">Community</a></li>
<li><a href="/blog">Blog</a></li>
</ul>
</nav>
<nav class="site-nav">
<div class="site-container">
<a class="logo" href="/">RethinkDB</a>
<ul>
<li class="menu-trigger"><a href="/"><img class="menu-trigger-icon" src="/assets/images/icons/icon-menu-white.png"></a></li>
<li><a href="/faq">FAQ</a></li>
<li><a href="/docs">Docs</a></li>
<li><a href="/api">API</a></li>
<li><a href="/community">Community</a></li>
<li><a href="/blog">Blog</a></li>
</ul>
</div>
</nav>
<section class="documentation">
<div class="site-container">
<section class="docs-sidebar-left">
<div class="search">
<input class="small" type="text" placeholder="Search documentation...">
</div>
<nav class="docs-nav">
<section>
<h1><a href="#">Getting started</a></h1>
<ul>
<li class="">
<a href="/docs/quickstart/">Thirty-second quickstart</a>
</li>
<li class="">
<a href="/docs/install/">Install the server</a>
</li>
<li class="">
<a href="/docs/install-drivers/">Install client drivers</a>
</li>
<li class="">
<a href="/docs/start-a-server/">Start a RethinkDB server</a>
</li>
</ul>
</section>
<section>
<h1><a href="#">The ReQL query language</a></h1>
<ul class="expanded">
<li class="">
<a href="/docs/guide/">Ten-minute guide</a>
</li>
<li class="">
<a href="/docs/introduction-to-reql/">Introduction to ReQL</a>
</li>
<li class="">
<a href="/docs/secondary-indexes/">Using secondary indexes</a>
</li>
<li class="">
<a href="/docs/nested-fields/">Accessing nested fields</a>
</li>
<li class="">
<a href="/docs/table-joins/">Table joins in RethinkDB</a>
</li>
<li class="">
<a href="/docs/map-reduce/">Map-reduce in RethinkDB</a>
</li>
<li class="active ">
<a href="/docs/changefeeds/">Changefeeds</a>
</li>
<li class="">
<a href="/docs/async-connections/">Asynchronous connections</a>
</li>
<li class="">
<a href="/docs/external-api-access/">Accessing HTTP APIs</a>
</li>
<li class="">
<a href="/docs/error-types/">ReQL error types</a>
</li>
<li class="">
<a href="/api/">API command reference</a>
</li>
</ul>
</section>
<section>
<h1><a href="#">The RethinkDB data model</a></h1>
<ul>
<li class="">
<a href="/docs/data-types/">ReQL data types</a>
</li>
<li class="">
<a href="/docs/dates-and-times/">Dates and times</a>
</li>
<li class="">
<a href="/docs/storing-binary/">Binary objects</a>
</li>
<li class="">
<a href="/docs/geo-support/">Geospatial queries</a>
</li>
<li class="">
<a href="/docs/data-modeling/">Modeling relationships</a>
</li>
</ul>
</section>
<section>
<h1><a href="#">ReQL in practice</a></h1>
<ul>
<li class="">
<a href="/docs/optimization/">Optimizing query performance</a>
</li>
<li class="">
<a href="/docs/reql-data-exploration/">Data exploration</a>
</li>
<li class="">
<a href="/docs/sql-to-reql/">SQL to ReQL cheat sheet</a>
</li>
<li class="">
<a href="/docs/cookbook/">Cookbook</a>
</li>
<li class="">
<a href="/docs/examples/">Example projects</a>
</li>
</ul>
</section>
<section>
<h1><a href="#">Administration</a></h1>
<ul>
<li class="">
<a href="/docs/administration-tools/">Administration tools</a>
</li>
<li class="">
<a href="/docs/sharding-and-replication/">Scaling, sharding and replication</a>
</li>
<li class="">
<a href="/docs/permissions-and-accounts/">Permissions and user accounts</a>
</li>
<li class="">
<a href="/docs/failover/">Failover</a>
</li>
<li class="">
<a href="/docs/backup/">Backing up your data</a>
</li>
<li class="">
<a href="/docs/system-tables/">System tables</a>
</li>
<li class="nested ">
<a href="/docs/system-issues/">Current issues</a>
</li>
<li class="nested ">
<a href="/docs/system-stats/">Statistics</a>
</li>
<li class="nested ">
<a href="/docs/system-jobs/">Jobs</a>
</li>
<li class="">
<a href="/docs/migration/">Version migration</a>
</li>
<li class="">
<a href="/docs/importing/">Importing data</a>
</li>
<li class="">
<a href="/docs/third-party-admin-tools/">Third-party admin tools</a>
</li>
</ul>
</section>
<section>
<h1><a href="#">Deployment</a></h1>
<ul>
<li class="">
<a href="/docs/start-on-startup/">Start on system startup</a>
</li>
<li class="">
<a href="/docs/security/">Securing your cluster</a>
</li>
<li class="">
<a href="/docs/config-file/">Configuration file settings</a>
</li>
<li class="">
<a href="/docs/cli-options/">Command line options</a>
</li>
<li class="">
<a href="/docs/paas/">Deploying with PaaS</a>
</li>
<li class="">
<a href="/docs/deployment-tools/">Third-party deployment tools</a>
</li>
</ul>
</section>
<section>
<h1><a href="#">Troubleshooting</a></h1>
<ul>
<li class="">
<a href="/docs/crashes/">Crash recovery</a>
</li>
<li class="">
<a href="/docs/troubleshooting/">Troubleshooting FAQ</a>
</li>
<li class="">
<a href="/docs/memory-usage/">Memory requirements</a>
</li>
</ul>
</section>
<section>
<h1><a href="#">Integration</a></h1>
<ul>
<li class="">
<a href="/docs/publish-subscribe/">Publish-subscribe</a>
</li>
<li class="">
<a href="/docs/elasticsearch/">Integrating Elasticsearch</a>
</li>
<li class="">
<a href="/docs/rabbitmq/">Integrating RabbitMQ</a>
</li>
<li class="">
<a href="/docs/rails">RethinkDB and Rails</a>
</li>
<li class="">
<a href="/docs/frameworks-and-libraries/">Third-party libraries</a>
</li>
<li class="">
<a href="/docs/writing-drivers/">Writing drivers</a>
</li>
</ul>
</section>
<section>
<h1><a href="#">RethinkDB architecture</a></h1>
<ul>
<li class="">
<a href="/docs/architecture/">Architecture details</a>
</li>
<li class="">
<a href="/docs/consistency/">Consistency guarantees</a>
</li>
<li class="">
<a href="/docs/limitations/">Limitations in RethinkDB</a>
</li>
<li class="">
<a href="/docs/stability/">Stability report</a>
</li>
<li class="">
<a href="/docs/faq/">Frequently asked questions</a>
</li>
<li class="">
<a href="/docs/2-1-5-performance-report/">Performance report</a>
</li>
</ul>
</section>
<section>
<h1><a href="#">Security</a></h1>
<ul>
<li class="">
<a href="/docs/vulnerabilities/">Vulnerabilities</a>
</li>
<li class="">
<a href="/docs/mobile-connections/">Connections from mobile devices</a>
</li>
</ul>
</section>
</nav>
</section>
<section class="docs-article">
<h1 class='title'>Changefeeds in RethinkDB</h1>
<section class="byline">
<nav class="lang-selector">
<p class="caption">Language: </p>
<section class="languages">
<a class="lang javascript"
href="
/docs/changefeeds/javascript/
">
<i class="javascript"></i>
<span >JavaScript</span>
</a><span class="divider">|</span><!--
--><a class="lang python"
href="
/docs/changefeeds/python/
">
<i class="python"></i>
<span >Python</span>
</a><span class="divider">|</span><!--
--><a class="lang ruby "
href="
/docs/changefeeds/ruby/
">
<i class="ruby"></i>
<span >Ruby</span>
</a><span class="divider">|</span><!--
--><a class="lang java active"
href="
/docs/changefeeds/java/
">
<i class="java"></i>
<span >Java</span>
</a>
</section>
</nav>
</section>
<div class="alg-content"><p><strong>Changefeeds</strong> lie at the heart of RethinkDB’s real-time functionality.</p>
<div class="toc">
<h1 class="no_toc" id="in-this-article">In this article</h1>
<ul id="markdown-toc">
<li><a href="#basic-usage" id="markdown-toc-basic-usage">Basic usage</a></li>
<li><a href="#point-single-document-changefeeds" id="markdown-toc-point-single-document-changefeeds">Point (single document) changefeeds</a></li>
<li><a href="#changefeeds-with-filtering-and-aggregation-queries" id="markdown-toc-changefeeds-with-filtering-and-aggregation-queries">Changefeeds with filtering and aggregation queries</a></li>
<li><a href="#including-state-changes" id="markdown-toc-including-state-changes">Including state changes</a></li>
<li><a href="#including-initial-values" id="markdown-toc-including-initial-values">Including initial values</a></li>
<li><a href="#including-result-types" id="markdown-toc-including-result-types">Including result types</a></li>
<li><a href="#handling-latency" id="markdown-toc-handling-latency">Handling latency</a></li>
<li><a href="#scaling-considerations" id="markdown-toc-scaling-considerations">Scaling considerations</a></li>
<li><a href="#read-more" id="markdown-toc-read-more">Read more</a></li>
</ul>
</div>
<p>They allow clients to receive changes on a table, a single document, or even the results from a specific query as they happen. Nearly any ReQL query can be turned into a changefeed.</p>
<p><img alt="Data Modeling Illustration" class="api_command_illustration" src="/assets/images/docs/api_illustrations/change-feeds.png" /></p>
<h1 id="basic-usage">Basic usage</h1>
<p>Subscribe to a feed by calling <a href="/api/java/changes">changes</a> on a table:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">Cursor</span> <span class="n">changeCursor</span> <span class="o">=</span> <span class="n">r</span><span class="o">.</span><span class="na">table</span><span class="o">(</span><span class="s">"users"</span><span class="o">).</span><span class="na">changes</span><span class="o">().</span><span class="na">run</span><span class="o">(</span><span class="n">conn</span><span class="o">);</span>
<span class="k">for</span> <span class="o">(</span><span class="n">Object</span> <span class="n">change</span> <span class="o">:</span> <span class="n">changeCursor</span><span class="o">)</span> <span class="o">{</span>
<span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">change</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">changes</code> command returns a cursor (like the <code class="language-plaintext highlighter-rouge">table</code> or <code class="language-plaintext highlighter-rouge">filter</code> commands do). You can iterate through its contents using ReQL. Unlike other cursors, the output of <code class="language-plaintext highlighter-rouge">changes</code> is infinite: the cursor will block until more elements are available. Every time you make a change to the table or document the <code class="language-plaintext highlighter-rouge">changes</code> feed is monitoring, a new object will be returned to the cursor. For example, if you insert a user <code class="language-plaintext highlighter-rouge">{"id": 1, "name": "Slava", "age": 31}</code> into the <code class="language-plaintext highlighter-rouge">users</code> table, RethinkDB will post this document to changefeeds subscribed to <code class="language-plaintext highlighter-rouge">users</code>:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="s2">"old_val"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="s2">"new_val"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s2">"id"</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="s2">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Slava"</span><span class="p">,</span><span class="w"> </span><span class="s2">"age"</span><span class="p">:</span><span class="w"> </span><span class="mi">31</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>Here <code class="language-plaintext highlighter-rouge">old_val</code> is the old version of the document, and <code class="language-plaintext highlighter-rouge">new_val</code> is a new version of the document. On an <code class="language-plaintext highlighter-rouge">insert</code>, <code class="language-plaintext highlighter-rouge">old_val</code> will be <code class="language-plaintext highlighter-rouge">null</code>; on a <code class="language-plaintext highlighter-rouge">delete</code>, <code class="language-plaintext highlighter-rouge">new_val</code> will be <code class="language-plaintext highlighter-rouge">null</code>. On an <code class="language-plaintext highlighter-rouge">update</code>, both <code class="language-plaintext highlighter-rouge">old_val</code> and <code class="language-plaintext highlighter-rouge">new_val</code> are present.</p>
<h1 id="point-single-document-changefeeds">Point (single document) changefeeds</h1>
<p>A “point” changefeed returns changes to a single document within a table rather than the table as a whole.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">r</span><span class="o">.</span><span class="na">table</span><span class="o">(</span><span class="s">"users"</span><span class="o">).</span><span class="na">get</span><span class="o">(</span><span class="mi">100</span><span class="o">).</span><span class="na">changes</span><span class="o">().</span><span class="na">run</span><span class="o">(</span><span class="n">conn</span><span class="o">);</span>
</code></pre></div></div>
<p>The output format of a point changefeed is identical to a table changefeed.</p>
<h1 id="changefeeds-with-filtering-and-aggregation-queries">Changefeeds with filtering and aggregation queries</h1>
<p>Like any ReQL command, <code class="language-plaintext highlighter-rouge">changes</code> integrates with the rest of the query language. You can call <code class="language-plaintext highlighter-rouge">changes</code> after most commands that transform or select data:</p>
<ul>
<li><a href="/api/java/filter">filter</a></li>
<li><a href="/api/java/get_all">getAll</a></li>
<li><a href="/api/java/map">map</a></li>
<li><a href="/api/java/pluck">pluck</a></li>
<li><a href="/api/java/between">between</a></li>
<li><a href="/api/java/union">union</a></li>
<li><a href="/api/java/min">min</a></li>
<li><a href="/api/java/max">max</a></li>
<li><a href="/api/java/order_by">orderBy</a>.<a href="/api/java/limit">limit</a></li>
</ul>
<p>You can also chain <code class="language-plaintext highlighter-rouge">changes</code> before any command that operates on a sequence of documents, as long as that command doesn’t consume the entire sequence. (For instance, <code class="language-plaintext highlighter-rouge">count</code> and <code class="language-plaintext highlighter-rouge">orderBy</code> cannot come after the <code class="language-plaintext highlighter-rouge">changes</code> command.)</p>
<p>Suppose you have a chat application with multiple clients posting messages to different chat rooms. You can create feeds that subscribe to messages posted to a specific room:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">r</span><span class="o">.</span><span class="na">table</span><span class="o">(</span><span class="s">"messages"</span><span class="o">).</span><span class="na">filter</span><span class="o">(</span>
<span class="n">row</span> <span class="o">-&gt;</span> <span class="n">row</span><span class="o">.</span><span class="na">g</span><span class="o">(</span><span class="s">"room_id"</span><span class="o">).</span><span class="na">eq</span><span class="o">(</span><span class="n">ROOM_ID</span><span class="o">)</span>
<span class="o">).</span><span class="na">changes</span><span class="o">().</span><span class="na">run</span><span class="o">(</span><span class="n">conn</span><span class="o">);</span>
</code></pre></div></div>
<p>You can also use more complicated expressions. Let’s say you have a table <code class="language-plaintext highlighter-rouge">scores</code> that contains the latest game score for every user of your game. You can create a feed of all games where a user beats their previous score, and get only the new value:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">r</span><span class="o">.</span><span class="na">table</span><span class="o">(</span><span class="s">"scores"</span><span class="o">).</span><span class="na">changes</span><span class="o">().</span><span class="na">filter</span><span class="o">(</span>
<span class="n">change</span> <span class="o">-&gt;</span> <span class="n">change</span><span class="o">.</span><span class="na">g</span><span class="o">(</span><span class="s">"new_val"</span><span class="o">).</span><span class="na">g</span><span class="o">(</span><span class="s">"score"</span><span class="o">).</span><span class="na">gt</span><span class="o">(</span><span class="n">change</span><span class="o">.</span><span class="na">g</span><span class="o">(</span><span class="s">"old_val"</span><span class="o">).</span><span class="na">g</span><span class="o">(</span><span class="s">"score"</span><span class="o">))</span>
<span class="o">).</span><span class="na">g</span><span class="o">(</span><span class="s">"new_val"</span><span class="o">).</span><span class="na">run</span><span class="o">(</span><span class="n">conn</span><span class="o">);</span>
</code></pre></div></div>
<p>There are some limitations and caveats on chaining with changefeeds.</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">min</code>, <code class="language-plaintext highlighter-rouge">max</code> and <code class="language-plaintext highlighter-rouge">orderBy</code> must be used with indexes.</li>
<li><code class="language-plaintext highlighter-rouge">orderBy</code> requires <code class="language-plaintext highlighter-rouge">limit</code>; neither command works by itself.</li>
<li><code class="language-plaintext highlighter-rouge">orderBy</code> must be used with a <a href="/docs/secondary-indexes/java">secondary index</a> or the primary index; it cannot be used with an unindexed field.</li>
<li>You cannot use changefeeds after <a href="/api/java/concat_map">concatMap</a> or other transformations whose results cannot be pushed to the shards.</li>
<li>You cannot apply a <code class="language-plaintext highlighter-rouge">filter</code> after <code class="language-plaintext highlighter-rouge">orderBy.limit</code> in a changefeed.</li>
<li>Transformations are applied before changes are calculated.</li>
</ul>
<h1 id="including-state-changes">Including state changes</h1>
<p>The <code class="language-plaintext highlighter-rouge">include_states</code> optional argument to <code class="language-plaintext highlighter-rouge">changes</code> allows you to receive extra “status” documents in changefeed streams. These can allow your application to distinguish between initial values returned at the start of a stream and subsequent changes. Read the <a href="/api/java/changes">changes</a> API documentation for a full explanation and example.</p>
<h1 id="including-initial-values">Including initial values</h1>
<p>By specifying <code class="language-plaintext highlighter-rouge">true</code> to the <code class="language-plaintext highlighter-rouge">include_initial</code> optional argument, the changefeed stream will start with the current contents of the table or selection being monitored. The initial results will have <code class="language-plaintext highlighter-rouge">new_val</code> fields, but no <code class="language-plaintext highlighter-rouge">old_val</code> fields, so it’s easy to distinguish them from change events.</p>
<p>If an initial result for a document has been sent and a change is made to that document that would move it to the unsent part of the result set (for instance, a changefeed monitors the top 100 posters, the first 50 have been sent, and poster 48 has become poster 52), an “uninitial” notification will be sent, with an <code class="language-plaintext highlighter-rouge">old_val</code> field but no <code class="language-plaintext highlighter-rouge">new_val</code> field. This is distinct from a delete change event, which would have a <code class="language-plaintext highlighter-rouge">new_val</code> of <code class="language-plaintext highlighter-rouge">null</code>. (In the top 100 posters example, that could indicate the poster has been deleted, or has dropped out of the top 100.)</p>
<p>If you specify <code class="language-plaintext highlighter-rouge">true</code> for both <code class="language-plaintext highlighter-rouge">include_states</code> and <code class="language-plaintext highlighter-rouge">include_initial</code>, the changefeed stream will start with a <code class="language-plaintext highlighter-rouge">{"state": "initializing"}</code> status document, followed by initial values. A <code class="language-plaintext highlighter-rouge">{"state": "ready"}</code> status document will be sent when all the initial values have been sent.</p>
<h1 id="including-result-types">Including result types</h1>
<p>The <code class="language-plaintext highlighter-rouge">include_types</code> optional argument adds a third field, <code class="language-plaintext highlighter-rouge">type</code>, to each result sent. The string values for <code class="language-plaintext highlighter-rouge">type</code> are largely self-explanatory:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">add</code>: a new value added to the result set.</li>
<li><code class="language-plaintext highlighter-rouge">remove</code>: an old value removed from the result set.</li>
<li><code class="language-plaintext highlighter-rouge">change</code>: an existing value changed in the result set.</li>
<li><code class="language-plaintext highlighter-rouge">initial</code>: an initial value notification.</li>
<li><code class="language-plaintext highlighter-rouge">uninitial</code>: an uninitial value notification.</li>
<li><code class="language-plaintext highlighter-rouge">state</code>: a status document from <code class="language-plaintext highlighter-rouge">include_states</code>.</li>
</ul>
<p>Including the <code class="language-plaintext highlighter-rouge">type</code> field can simplify code that handles different cases for changefeed results.</p>
<h1 id="handling-latency">Handling latency</h1>
<p>Depending on how fast your application makes changes to monitored data and how fast it processes change notifications, it’s possible that more than one change will happen between calls to the <code class="language-plaintext highlighter-rouge">changes</code> command. You can control what happens in that case with the <code class="language-plaintext highlighter-rouge">squash</code> optional argument.</p>
<p>By default, if more than one change occurs between invocations of <code class="language-plaintext highlighter-rouge">changes</code>, your application will receive a single change object whose <code class="language-plaintext highlighter-rouge">new_val</code> will incorporate <em>all</em> the changes to the data. Suppose three updates occurred to a monitored document between <code class="language-plaintext highlighter-rouge">change</code> reads:</p>
<table>
<thead>
<tr>
<th>Change</th>
<th>Data</th>
</tr>
</thead>
<tbody>
<tr>
<td>Initial state (<code class="language-plaintext highlighter-rouge">old_val</code>)</td>
<td>{ name: “Fred”, admin: true }</td>
</tr>
<tr>
<td>update({name: “George”})</td>
<td>{ name: “George”, admin: true }</td>
</tr>
<tr>
<td>update({admin: false})</td>
<td>{ name: “George”, admin: false }</td>
</tr>
<tr>
<td>update({name: “Jay”})</td>
<td>{ name: “Jay”, admin: false }</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">new_val</code></td>
<td>{ name: “Jay”, admin: false }</td>
</tr>
</tbody>
</table>
<p>Your application would by default receive the object as it existed in the database after the <em>most recent</em> change. The previous two updates would be “squashed” into the third.</p>
<p>If you wanted to receive <em>all</em> the changes, including the interim states, you could do so by passing <code class="language-plaintext highlighter-rouge">squash: false</code>. The server will buffer up to 100,000 changes. (This number can be changed with the <code class="language-plaintext highlighter-rouge">changefeed_queue_size</code> optional argument.)</p>
<p>A third option is to specify how many seconds to wait between squashes. Passing <code class="language-plaintext highlighter-rouge">squash: 5</code> to the <code class="language-plaintext highlighter-rouge">changes</code> command tells RethinkDB to squash changes together every five seconds. Depending on your application’s use case, this might reduce the load on the server. A number passed to <code class="language-plaintext highlighter-rouge">squash</code> may be a float. Note that the requested interval is not guaranteed, but is rather a best effort.</p>
<p><strong>Note:</strong> Changefeeds ignore the <code class="language-plaintext highlighter-rouge">read_mode</code> flag to <code class="language-plaintext highlighter-rouge">run</code>, and always behave as if it is set to <code class="language-plaintext highlighter-rouge">single</code> (i.e., the values they return are in memory on the primary replica, but have not necessarily been written to disk yet). For more details read <a href="/docs/consistency">Consistency guarantees</a>.</p>
<h1 id="scaling-considerations">Scaling considerations</h1>
<p>Changefeeds perform well as they scale, although they create extra intracluster messages in proportion to the number of servers with open feed connections on each write. This can be mitigated by running a RethinkDB proxy server (the <code class="language-plaintext highlighter-rouge">rethinkdb proxy</code> startup option); read <a href="/docs/sharding-and-replication/#running-a-proxy-node">Running a proxy node</a> for details.</p>
<p>Since changefeeds are unidirectional with no acknowledgement returned from clients, they cannot guarantee delivery. If you need real-time updating with delivery guarantees, consider using a model that distributes to the clients through a message broker such as <a href="https://www.rabbitmq.com/">RabbitMQ</a>.</p>
<h1 id="read-more">Read more</h1>
<ul>
<li>The <a href="/api/java/changes">changes</a> command API reference</li>
<li><a href="/docs/introduction-to-reql/">Introduction to ReQL</a></li>
<li><a href="/docs/data-types/">ReQL data types</a></li>
</ul>
</div>
</section>
</div>
</section>
<!-- site footer -->
<footer class="site-footer">
<div class="site-container">
<section class="get-started">
<h1>Join 100,000 developers</h1>
<p>Install in seconds on Linux or OS X.</p>
<a class="button small install" href="/install">Install</a>
<a class="button small outlined learn-more" href="/faq">Learn more</a>
<hr>
<section class="contact-info">
<p class="irc">#rethinkdb on freenode</p>
</section>
</section>
<ul class="footer-links">
<li>
<header>
<span>Getting started</span>
</header>
<ul>
<li><a href="/faq">FAQ</a></li>
<li><a href="/install">Install</a></li>
<li><a href="/docs/quickstart">Quickstart</a></li>
<li><a href="/docs/guide">Tutorial</a></li>
</ul>
</li>
<li>
<header>
<span>Documentation</span>
</header>
<ul>
<li><a href="/api">API</a></li>
<li><a href="/docs/introduction-to-reql/">Intro to ReQL</a></li>
<li><a href="/docs/cookbook">Cookbook</a></li>
<li><a href="/docs/architecture">Architecture</a></li>
</ul>
</li>
<li>
<header>
<span>Community</span>
</header>
<ul>
<li><a href="/community">Get Involved</a></li>
<li><a href="https://github.com/rethinkdb/rethinkdb">GitHub</a></li>
<li><a href="https://twitter.com/rethinkdb">Twitter</a></li>
<li><a href="https://join.slack.com/t/rethinkdb/shared_invite/enQtNzAxOTUzNTk1NzMzLWY5ZTA0OTNmMWJiOWFmOGVhNTUxZjQzODQyZjIzNjgzZjdjZDFjNDg1NDY3MjFhYmNhOTY1MDVkNDgzMWZiZWM">Slack</a></li>
</ul>
</li>
<li>
<header>
<span>RethinkDB</span>
</header>
<ul>
<li><a href="/blog/">Blog</a></li>
</ul>
</li>
</ul>
<p class="github-footer"><i class="icon-github-footer"></i><a href="https://github.com/rethinkdb/rethinkdb">Star this project</a> on GitHub.</p>
</div>
</footer>
</div>
<!-- algolia -->
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/docsearch.js@2/dist/cdn/docsearch.min.css" />
<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/docsearch.js@2/dist/cdn/docsearch.min.js"></script>
<script type="text/javascript"> docsearch({
apiKey: 'a33e586ddcc6f914b6c3eb48760d6235',
indexName: 'rethinkdb',
inputSelector: '#wrapper > section > div > section.docs-sidebar-left > div > input',
debug: false
});
</script>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-9922584-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</body>
</html>

View file

@ -0,0 +1,166 @@
# CultNet Distributed Database Research Summary
Purpose: ground CultNet's distributed realtime database design in prior art
before adding more machinery. The goal is a coherent local-first mesh database
over CultCache, not a hand-rolled distributed-systems costume party.
## Sources Stored Here
- `hashgraph-swirlds-tr-2016-01.pdf`
- Source: <https://www.swirlds.com/downloads/SWIRLDS-TR-2016-01.pdf>
- Topic: gossip-about-gossip, virtual voting, fair ordering, asynchronous BFT.
- `raft-extended.html`
- Source: <https://yygcode.com/papers/consensus-raft-extended-version.html>
- Topic: understandable consensus, leader election, log replication, safety.
- `raft-extended.pdf`
- Source: <https://web.stanford.edu/~ouster/cgi-bin/papers/raft-extended.pdf>
- Topic: canonical Raft paper copy.
- `dynamo-amazon-science.html`
- Source: <https://www.amazon.science/publications/dynamo-amazons-highly-available-key-value-store>
- Topic: Dynamo publication page and abstract.
- `dynamo-sosp2007.pdf`
- Source: <https://web.stanford.edu/class/cs244/papers/amazon-dynamo-sosp2007.pdf>
- Topic: highly available key-value store, consistent hashing, versioning,
quorums, hinted handoff, anti-entropy.
- `swim.pdf`
- Source: <https://www.cs.cornell.edu/projects/quicksilver/public_pdfs/SWIM.pdf>
- Topic: scalable weakly consistent process-group membership.
- `rethinkdb-changefeeds.html`
- Source: <https://rethinkdb.com/docs/changefeeds/java/>
- Topic: realtime query/changefeed ergonomics.
- `firebase-realtime-offline.html`
- Source: <https://firebase.google.com/docs/database/web/offline-capabilities>
- Topic: offline behavior, presence, server-side disconnect operations.
- `crdt-arxiv-1805.06358.html`
- Source: <https://arxiv.org/abs/1805.06358>
- Topic: conflict-free replicated data types and deterministic convergence.
## Design Takeaways
### RethinkDB
Keep: changefeeds as the product feel. Subscribers should receive document
changes continuously, with enough old/new context to render, reconcile, or
debug. Point subscriptions and filtered subscriptions are both first-class.
CultNet implication: database subscriptions should be explicit schema-v0
messages. The server should stream raw document changes that clients can apply
through the same CultCache reconciliation path as snapshots.
### Firebase Realtime Database
Keep: realtime sync and local/offline ergonomics. Presence and disconnect
behavior are database features, not application afterthoughts.
Defer: full offline writes. CultNet should not pretend arbitrary offline
multi-writer changes merge safely. Offline/local-first behavior needs declared
per-document conflict policy.
CultNet implication: add presence/disconnect records later as ordinary
CultCache documents with server authority, not as hidden transport state.
### Raft
Keep: the understandable authority model. One leader/primary owns ordered
writes for a shard. Decompose the problem into ownership, log/mutation
replication, and safety.
Defer: full automatic leader election and replicated logs until shard catalogs
and explicit epochs exist.
CultNet implication: the current primary-shard policy is the correct first
foundation. Every write should either hit the primary, be forwarded to the
primary, or fail with routing information.
### Dynamo
Keep: partitioning, replication metadata, vector-ish causality, hinted recovery,
anti-entropy, and application-visible conflicts.
Reject for now: "always writeable" semantics. That is attractive, expensive,
and easy to lie about.
CultNet implication: shard descriptors should grow into a shard catalog with
owner runtime id, epoch, schema/key ranges, and later replica/preference-list
metadata. Conflicts must surface as data, not disappear into last-writer-wins
unless a document explicitly chose that policy.
### SWIM
Keep: gossip-shaped membership once the mesh grows beyond a small static
cluster. SWIM separates failure detection from membership dissemination and
keeps per-node message load stable.
Defer: membership implementation until there is a shard catalog to disseminate.
CultNet implication: do not build peer discovery as a side-channel registry.
When it arrives, it should update membership and shard-catalog state together.
### CRDTs
Keep: CRDTs for documents whose merge law is explicit and deterministic.
Reject: generic automatic merge for arbitrary domain objects. That is a
language cop with a nicer hat.
CultNet implication: CRDT support belongs in schema metadata or document
contract metadata. A document type can opt into a known merge strategy; the
default distributed write policy remains primary authority.
### Hashgraph
Keep: gossip-about-gossip as an idea for compact event provenance and possible
fair ordering research. Virtual voting is interesting when every member sees
the same gossip history.
Reject for now: adopting hashgraph consensus as CultNet's core. The public
ledger/crypto smell is not the main issue; the issue is that CultNet does not
currently need asynchronous Byzantine total ordering to become a good mesh
database. Dragging that in now would make the machine harder to explain.
CultNet implication: if we later need decentralized event ordering, use a
dedicated design pass. Do not smuggle hashgraph metadata into ordinary document
replication because it sounds powerful.
## Current CultNet Direction
The coherent path is:
1. Primary-shard authority.
2. Explicit schema-v0 snapshot, put, delete, subscribe, unsubscribe, and change
messages.
3. Shard catalog exchange with owner runtime id and epoch.
4. Optional forwarding from non-owner nodes to owners.
5. Membership/failure detection, likely SWIM-shaped.
6. Replication and failover, likely Raft-shaped per shard.
7. Optional CRDT policies for document types that deserve offline multi-writer
semantics.
8. Optional gossip-history research if fair decentralized ordering becomes a
real requirement.
## Live Invariants
- CultCache owns document identity, schema compatibility, local indexes, and
reconciliation.
- CultNet owns transport, shard authority, subscriptions, and remote mutation
delivery.
- Raw wire records must pass through `CultNetDatabase` before mutating local
cache state.
- A write without authority is rejected or explicitly forwarded. It is never
silently applied.
- Realtime change streams publish domain changes, not storage implementation
details.
- Conflict policy must be declared before conflicting writes are accepted.
## Next Cut
Build shard catalog exchange:
- `cultnet.shard_catalog_request.v0`
- `cultnet.shard_catalog_response.v0`
- per-shard id, owner runtime id, epoch, schema ids, key prefix/range
- stale epoch rejection on writes
- routing error that tells clients where the primary lives when known
That is the next foundation needed before forwarding, membership, Raft-style
replication, or CRDT policy work can be honest.

Binary file not shown.