Archive CultNet distributed database research
This commit is contained in:
parent
b54ed59135
commit
5c4195677d
10 changed files with 19086 additions and 0 deletions
709
research/cultnet-distributed-database/crdt-arxiv-1805.06358.html
Normal file
709
research/cultnet-distributed-database/crdt-arxiv-1805.06358.html
Normal file
|
|
@ -0,0 +1,709 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
|
||||
<head> <title>[1805.06358] Conflict-free Replicated Data Types (CRDTs)</title>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<link rel="apple-touch-icon" sizes="180x180" href="/static/browse/0.3.4/images/icons/apple-touch-icon.png">
|
||||
<link rel="icon" type="image/png" sizes="32x32" href="/static/browse/0.3.4/images/icons/favicon-32x32.png">
|
||||
<link rel="icon" type="image/png" sizes="16x16" href="/static/browse/0.3.4/images/icons/favicon-16x16.png">
|
||||
<link rel="manifest" href="/static/browse/0.3.4/images/icons/site.webmanifest">
|
||||
<link rel="mask-icon" href="/static/browse/0.3.4/images/icons/safari-pinned-tab.svg" color="#5bbad5">
|
||||
<meta name="msapplication-TileColor" content="#da532c">
|
||||
<meta name="theme-color" content="#ffffff">
|
||||
<link rel="stylesheet" type="text/css" media="screen" href="/static/browse/0.3.4/css/arXiv.css?v=20260318" />
|
||||
<link rel="stylesheet" type="text/css" media="print" href="/static/browse/0.3.4/css/arXiv-print.css?v=20200611" />
|
||||
<link rel="stylesheet" type="text/css" media="screen" href="/static/browse/0.3.4/css/browse_search.css" />
|
||||
<script language="javascript" src="/static/browse/0.3.4/js/accordion.js" ></script>
|
||||
<script language="javascript" src="/static/browse/0.3.4/js/optin-modal.js?v=20250819"></script>
|
||||
|
||||
<link rel="canonical" href="https://arxiv.org/abs/1805.06358"/>
|
||||
<meta name="description" content="Abstract page for arXiv paper 1805.06358: Conflict-free Replicated Data Types (CRDTs)"><meta property="og:type" content="website" />
|
||||
<meta property="og:site_name" content="arXiv.org" />
|
||||
<meta property="og:title" content="Conflict-free Replicated Data Types (CRDTs)" />
|
||||
<meta property="og:url" content="https://arxiv.org/abs/1805.06358v1" />
|
||||
<meta property="og:image" content="/static/browse/0.3.4/images/arxiv-logo-fb.png" />
|
||||
<meta property="og:image:secure_url" content="/static/browse/0.3.4/images/arxiv-logo-fb.png" />
|
||||
<meta property="og:image:width" content="1200" />
|
||||
<meta property="og:image:height" content="700" />
|
||||
<meta property="og:image:alt" content="arXiv logo"/>
|
||||
<meta property="og:description" content="A conflict-free replicated data type (CRDT) is an abstract data type, with a well defined interface, designed to be replicated at multiple processes and exhibiting the following properties: (1) any replica can be modified without coordinating with another replicas; (2) when any two replicas have received the same set of updates, they reach the same state, deterministically, by adopting mathematically sound rules to guarantee state convergence."/>
|
||||
<meta name="twitter:site" content="@arxiv"/>
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Conflict-free Replicated Data Types (CRDTs)"/>
|
||||
<meta name="twitter:description" content="A conflict-free replicated data type (CRDT) is an abstract data type, with a well defined interface, designed to be replicated at multiple processes and exhibiting the following properties: (1)..."/>
|
||||
<meta name="twitter:image" content="https://static.arxiv.org/icons/twitter/arxiv-logo-twitter-square.png"/>
|
||||
<meta name="twitter:image:alt" content="arXiv logo"/>
|
||||
<link rel="stylesheet" media="screen" type="text/css" href="/static/browse/0.3.4/css/tooltip.css"/><link rel="stylesheet" media="screen" type="text/css" href="https://static.arxiv.org/js/bibex-dev/bibex.css?20200709"/> <script src="/static/browse/0.3.4/js/mathjaxToggle.min.js" type="text/javascript"></script> <script src="//code.jquery.com/jquery-latest.min.js" type="text/javascript"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/js-cookie@2/src/js.cookie.min.js" type="text/javascript"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/dompurify@2.3.5/dist/purify.min.js"></script>
|
||||
<script src="/static/browse/0.3.4/js/toggle-labs.js?20241022" type="text/javascript"></script>
|
||||
<script src="/static/browse/0.3.4/js/cite.js" type="text/javascript"></script><meta name="citation_title" content="Conflict-free Replicated Data Types (CRDTs)" /><meta name="citation_author" content="Preguiça, Nuno" /><meta name="citation_author" content="Baquero, Carlos" /><meta name="citation_author" content="Shapiro, Marc" /><meta name="citation_doi" content="10.1007/978-3-319-63962-8\_185-1" /><meta name="citation_date" content="2018/05/16" /><meta name="citation_online_date" content="2018/05/16" /><meta name="citation_pdf_url" content="https://arxiv.org/pdf/1805.06358" /><meta name="citation_arxiv_id" content="1805.06358" /><meta name="citation_abstract" content="A conflict-free replicated data type (CRDT) is an abstract data type, with a well defined interface, designed to be replicated at multiple processes and exhibiting the following properties: (1) any replica can be modified without coordinating with another replicas; (2) when any two replicas have received the same set of updates, they reach the same state, deterministically, by adopting mathematically sound rules to guarantee state convergence." />
|
||||
</head>
|
||||
|
||||
<body class="with-cu-identity">
|
||||
|
||||
|
||||
<div class="flex-wrap-footer">
|
||||
<header>
|
||||
<a href="#content" class="is-sr-only">Skip to main content</a>
|
||||
<!-- start desktop header -->
|
||||
<div class="columns is-vcentered is-hidden-mobile" id="cu-identity">
|
||||
<div class="column" id="cu-logo">
|
||||
<a href="https://www.cornell.edu/"><img src="/static/browse/0.3.4/images/icons/cu/cornell-reduced-white-SMALL.svg" alt="Cornell University" /></a>
|
||||
</div><!-- /from April 7 at 1:00 AM to May 29 at 21:40 --><!-- /from May 2 at 1:00 AM to May 5 at 9:45 AM --><div class="column banner-minimal">
|
||||
<a href="https://tech.cornell.edu/arxiv/" target="_blank">Learn about arXiv becoming an independent nonprofit.</a>
|
||||
</div><div class="column" id="support-ack">
|
||||
<span id="support-ack-url">We gratefully acknowledge support from the Simons Foundation, <a href="https://info.arxiv.org/about/ourmembers.html">member institutions</a>, and all contributors.</span>
|
||||
<a href="https://info.arxiv.org/about/donate.html" class="btn-header-donate">Donate</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="header" class="is-hidden-mobile">
|
||||
<a aria-hidden="true" tabindex="-1" href="/IgnoreMe"></a>
|
||||
<div class="header-breadcrumbs is-hidden-mobile">
|
||||
<a href="/"><img src="/static/browse/0.3.4/images/arxiv-logo-one-color-white.svg" alt="arxiv logo" style="height:40px;"/></a> <span>></span> <a href="/list/cs/recent">cs</a> <span>></span> arXiv:1805.06358
|
||||
</div>
|
||||
|
||||
<div class="columns is-vcentered is-mobile" style="justify-content: flex-end;">
|
||||
</div>
|
||||
|
||||
<div class="search-block level-right">
|
||||
<form class="level-item mini-search" method="GET" action="https://arxiv.org/search">
|
||||
<div class="field has-addons">
|
||||
<div class="control">
|
||||
<input class="input is-small" type="text" name="query" placeholder="Search..." aria-label="Search term or terms" />
|
||||
<p class="help"><a href="https://info.arxiv.org/help">Help</a> | <a href="https://arxiv.org/search/advanced">Advanced Search</a></p>
|
||||
</div>
|
||||
<div class="control">
|
||||
<div class="select is-small">
|
||||
<select name="searchtype" aria-label="Field to search">
|
||||
<option value="all" selected="selected">All fields</option>
|
||||
<option value="title">Title</option>
|
||||
<option value="author">Author</option>
|
||||
<option value="abstract">Abstract</option>
|
||||
<option value="comments">Comments</option>
|
||||
<option value="journal_ref">Journal reference</option>
|
||||
<option value="acm_class">ACM classification</option>
|
||||
<option value="msc_class">MSC classification</option>
|
||||
<option value="report_num">Report number</option>
|
||||
<option value="paper_id">arXiv identifier</option>
|
||||
<option value="doi">DOI</option>
|
||||
<option value="orcid">ORCID</option>
|
||||
<option value="author_id">arXiv author ID</option>
|
||||
<option value="help">Help pages</option>
|
||||
<option value="full_text">Full text</option>
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
<input type="hidden" name="source" value="header">
|
||||
<button class="button is-small is-cul-darker">Search</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
</div><!-- /end desktop header -->
|
||||
|
||||
<div class="mobile-header">
|
||||
<div class="columns is-mobile">
|
||||
<div class="column logo-arxiv"><a href="https://arxiv.org/"><img src="/static/browse/0.3.4/images/arxiv-logomark-small-white.svg" alt="arXiv logo" style="height:60px;" /></a></div>
|
||||
<div class="column logo-cornell"><a href="https://www.cornell.edu/">
|
||||
<picture>
|
||||
<source media="(min-width: 501px)"
|
||||
srcset="/static/browse/0.3.4/images/icons/cu/cornell-reduced-white-SMALL.svg 400w"
|
||||
sizes="400w" />
|
||||
<source srcset="/static/browse/0.3.4/images/icons/cu/cornell_seal_simple_black.svg 2x" />
|
||||
<img src="/static/browse/0.3.4/images/icons/cu/cornell-reduced-white-SMALL.svg" alt="Cornell University Logo" />
|
||||
</picture>
|
||||
</a></div>
|
||||
<div class="column nav" id="toggle-container">
|
||||
<button class="toggle-control" aria-label="Open search"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" class="icon filter-white" aria-hidden="true"><path d="M505 442.7L405.3 343c-4.5-4.5-10.6-7-17-7H372c27.6-35.3 44-79.7 44-128C416 93.1 322.9 0 208 0S0 93.1 0 208s93.1 208 208 208c48.3 0 92.7-16.4 128-44v16.3c0 6.4 2.5 12.5 7 17l99.7 99.7c9.4 9.4 24.6 9.4 33.9 0l28.3-28.3c9.4-9.4 9.4-24.6.1-34zM208 336c-70.7 0-128-57.2-128-128 0-70.7 57.2-128 128-128 70.7 0 128 57.2 128 128 0 70.7-57.2 128-128 128z"/></svg></button>
|
||||
<div class="mobile-toggle-block toggle-target">
|
||||
<form class="mobile-search-form" method="GET" action="https://arxiv.org/search">
|
||||
<div class="field has-addons">
|
||||
<input class="input" type="text" name="query" placeholder="Search..." aria-label="Search term or terms" />
|
||||
<input type="hidden" name="source" value="header">
|
||||
<input type="hidden" name="searchtype" value="all">
|
||||
<button class="button">GO</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<button class="toggle-control" aria-label="Open navigation menu"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512" class="icon filter-white" aria-hidden="true"><path d="M16 132h416c8.837 0 16-7.163 16-16V76c0-8.837-7.163-16-16-16H16C7.163 60 0 67.163 0 76v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16z"/></svg></button>
|
||||
<div class="mobile-toggle-block toggle-target">
|
||||
<nav class="mobile-menu" aria-labelledby="mobilemenulabel">
|
||||
<h2 id="mobilemenulabel">quick links</h2>
|
||||
<ul>
|
||||
<li><a href="https://arxiv.org/login">Login</a></li>
|
||||
<li><a href="https://info.arxiv.org/help">Help Pages</a></li>
|
||||
<li><a href="https://info.arxiv.org/about">About</a></li>
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div><!-- /end mobile-header -->
|
||||
</header>
|
||||
|
||||
<main>
|
||||
<div id="content">
|
||||
<!--
|
||||
rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
|
||||
xmlns:dc="http://purl.org/dc/elements/1.1/"
|
||||
xmlns:trackback="http://madskills.com/public/xml/rss/module/trackback/">
|
||||
<rdf:Description
|
||||
rdf:about="/abs/1805.06358"
|
||||
dc:identifier="/abs/1805.06358"
|
||||
dc:title="Conflict-free Replicated Data Types (CRDTs)"
|
||||
trackback:ping="/trackback/1805.06358" />
|
||||
</rdf:RDF>
|
||||
--><div id="abs-outer">
|
||||
|
||||
<div class="leftcolumn">
|
||||
<div class="subheader">
|
||||
<h1>Computer Science > Distributed, Parallel, and Cluster Computing</h1>
|
||||
</div>
|
||||
|
||||
<div class="header-breadcrumbs-mobile">
|
||||
<strong>arXiv:1805.06358</strong> (cs)
|
||||
</div>
|
||||
<link rel="stylesheet" type="text/css" href="/static/base/1.0.1/css/abs.css">
|
||||
<div id="content-inner">
|
||||
<div id="abs">
|
||||
<div class="dateline">
|
||||
[Submitted on 16 May 2018]</div>
|
||||
<h1 class="title mathjax"><span class="descriptor">Title:</span>Conflict-free Replicated Data Types (CRDTs)</h1>
|
||||
<div class="authors"><span class="descriptor">Authors:</span><a href="https://arxiv.org/search/cs?searchtype=author&query=Pregui%C3%A7a,+N" rel="nofollow">Nuno Preguiça</a>, <a href="https://arxiv.org/search/cs?searchtype=author&query=Baquero,+C" rel="nofollow">Carlos Baquero</a>, <a href="https://arxiv.org/search/cs?searchtype=author&query=Shapiro,+M" rel="nofollow">Marc Shapiro</a> (Regal)</div> <div id="download-button-info" hidden>View a PDF of the paper titled Conflict-free Replicated Data Types (CRDTs), by Nuno Pregui\c{c}a and 2 other authors</div>
|
||||
<a class="mobile-submission-download" href="/pdf/1805.06358">View PDF</a>
|
||||
|
||||
|
||||
|
||||
<blockquote class="abstract mathjax">
|
||||
<span class="descriptor">Abstract:</span>A conflict-free replicated data type (CRDT) is an abstract data type, with a well defined interface, designed to be replicated at multiple processes and exhibiting the following properties: (1) any replica can be modified without coordinating with another replicas; (2) when any two replicas have received the same set of updates, they reach the same state, deterministically, by adopting mathematically sound rules to guarantee state convergence.
|
||||
</blockquote>
|
||||
|
||||
<!--CONTEXT-->
|
||||
<div class="metatable">
|
||||
<table summary="Additional metadata"><tr>
|
||||
<td class="tablecell label">Subjects:</td>
|
||||
<td class="tablecell subjects">
|
||||
<span class="primary-subject">Distributed, Parallel, and Cluster Computing (cs.DC)</span></td>
|
||||
</tr><tr>
|
||||
<td class="tablecell label">Cite as:</td>
|
||||
<td class="tablecell arxivid"><span class="arxivid"><a href="https://arxiv.org/abs/1805.06358">arXiv:1805.06358</a> [cs.DC]</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td class="tablecell label"> </td>
|
||||
<td class="tablecell arxividv">(or <span class="arxivid">
|
||||
<a href="https://arxiv.org/abs/1805.06358v1">arXiv:1805.06358v1</a> [cs.DC]</span> for this version)
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td class="tablecell label"> </td>
|
||||
<td class="tablecell arxivdoi"> <a href="https://doi.org/10.48550/arXiv.1805.06358" id="arxiv-doi-link">https://doi.org/10.48550/arXiv.1805.06358</a><div class="button-and-tooltip">
|
||||
<button class="more-info" aria-describedby="more-info-desc-1">
|
||||
<svg height="15" role="presentation" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path fill="currentColor" d="M256 8C119.043 8 8 119.083 8 256c0 136.997 111.043 248 248 248s248-111.003 248-248C504 119.083 392.957 8 256 8zm0 110c23.196 0 42 18.804 42 42s-18.804 42-42 42-42-18.804-42-42 18.804-42 42-42zm56 254c0 6.627-5.373 12-12 12h-88c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h12v-64h-12c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h64c6.627 0 12 5.373 12 12v100h12c6.627 0 12 5.373 12 12v24z" class=""></path></svg>
|
||||
<span class="visually-hidden">Focus to learn more</span>
|
||||
</button>
|
||||
<!-- tooltip description -->
|
||||
<div role="tooltip" id="more-info-desc-1">
|
||||
<span class="left-corner"></span> arXiv-issued DOI via DataCite</div>
|
||||
</div>
|
||||
</td>
|
||||
</tr> <tr>
|
||||
<td class="tablecell label">Journal reference:</td>
|
||||
<td class="tablecell jref">Sakr, Sherif and Zomaya, Albert. Encyclopedia of Big Data Technologies, Springer International Publishing, 2018, Encyclopedia of Big Data Technologies, 978-3-319-63962-8</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td class="tablecell label">
|
||||
<abbr title="Digital Object Identifier">Related DOI</abbr>:
|
||||
</td>
|
||||
<td class="tablecell doi"><a href="https://doi.org/10.1007/978-3-319-63962-8%5C_185-1" data-doi="10.1007/978-3-319-63962-8\_185-1" class="link-https link-external" rel="external noopener nofollow">https://doi.org/10.1007/978-3-319-63962-8%5C_185-1</a>
|
||||
|
||||
<!-- accessible tooltip example -->
|
||||
<div class="button-and-tooltip">
|
||||
<button class="more-info" aria-describedby="more-info-desc-1">
|
||||
<svg height="15" role="presentation" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path fill="currentColor" d="M256 8C119.043 8 8 119.083 8 256c0 136.997 111.043 248 248 248s248-111.003 248-248C504 119.083 392.957 8 256 8zm0 110c23.196 0 42 18.804 42 42s-18.804 42-42 42-42-18.804-42-42 18.804-42 42-42zm56 254c0 6.627-5.373 12-12 12h-88c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h12v-64h-12c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h64c6.627 0 12 5.373 12 12v100h12c6.627 0 12 5.373 12 12v24z" class=""></path></svg>
|
||||
<span class="visually-hidden">Focus to learn more</span>
|
||||
</button>
|
||||
<!-- tooltip description -->
|
||||
<div role="tooltip" id="more-info-desc-1">
|
||||
<span class="left-corner"></span>
|
||||
DOI(s) linking to related resources
|
||||
</div>
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="submission-history">
|
||||
<h2>Submission history</h2> From: Marc Shapiro [<a href="/show-email/386ae749/1805.06358" rel="nofollow">view email</a>] [via CCSD proxy] <br/> <strong>[v1]</strong>
|
||||
Wed, 16 May 2018 15:00:36 UTC (43 KB)<br/>
|
||||
</div>
|
||||
</div>
|
||||
<!--end leftcolumn-->
|
||||
<div class="extra-services"> <div class="full-text">
|
||||
<a name="other"></a>
|
||||
<span class="descriptor">Full-text links:</span>
|
||||
<h2>Access Paper:</h2>
|
||||
<ul>
|
||||
<div id="download-button-info" hidden>
|
||||
View a PDF of the paper titled Conflict-free Replicated Data Types (CRDTs), by Nuno Pregui\c{c}a and 2 other authors</div><li><a href="/pdf/1805.06358" aria-describedby="download-button-info" accesskey="f" class="abs-button download-pdf">View PDF</a></li><li><a href="/src/1805.06358" class="abs-button download-eprint">TeX Source
|
||||
</a></li></ul>
|
||||
<div class="abs-license"><a href="http://arxiv.org/licenses/nonexclusive-distrib/1.0/" title="Rights to this article">view license</a></div>
|
||||
</div>
|
||||
<!--end full-text--> <div class="browse">
|
||||
<h3 class="browse-context-heading">Current browse context:</h3>
|
||||
<div class="current">cs.DC</div>
|
||||
|
||||
<div class="prevnext">
|
||||
<span class="arrow">
|
||||
<a class="abs-button prev-url" href="/prevnext?id=1805.06358&function=prev&context=cs.DC"
|
||||
accesskey="p" title="previous in cs.DC (accesskey p)" rel="nofollow">< prev</a>
|
||||
</span>
|
||||
<span class="is-hidden-mobile"> | </span> <span class="arrow">
|
||||
<a class="abs-button next-url" href="/prevnext?id=1805.06358&function=next&context=cs.DC" accesskey="n"
|
||||
title="next in cs.DC (accesskey n)" rel="nofollow">next ></a>
|
||||
</span><br/>
|
||||
</div><div class="list">
|
||||
<a class="abs-button abs-button-grey abs-button-small context-new" href="/list/cs.DC/new" rel="nofollow">new</a>
|
||||
<span class="is-hidden-mobile"> | </span>
|
||||
<a class="abs-button abs-button-grey abs-button-small context-recent" href="/list/cs.DC/recent" rel="nofollow">recent</a>
|
||||
<span class="is-hidden-mobile"> | </span><a class="abs-button abs-button-grey abs-button-small context-id" href="/list/cs.DC/2018-05" rel="nofollow">2018-05</a>
|
||||
</div><div class="abs-switch-cat">
|
||||
Change to browse by:
|
||||
<div class="switch context-change">
|
||||
<a href="/abs/1805.06358?context=cs" rel="nofollow">cs</a><br class="is-hidden-mobile">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
<div class="extra-ref-cite">
|
||||
<h3>References & Citations</h3>
|
||||
<ul>
|
||||
<li><a class="abs-button abs-button-small cite-ads" href="https://ui.adsabs.harvard.edu/abs/arXiv:1805.06358">NASA ADS</a></li><li><a class="abs-button abs-button-small cite-google-scholar" href="https://scholar.google.com/scholar_lookup?arxiv_id=1805.06358" target="_blank" rel="noopener">Google Scholar</a></li>
|
||||
<li><a class="abs-button abs-button-small cite-semantic-scholar" href="https://api.semanticscholar.org/arXiv:1805.06358" target="_blank" rel="noopener">Semantic Scholar</a></li>
|
||||
</ul>
|
||||
<div style="clear:both;"></div>
|
||||
</div>
|
||||
|
||||
<div class="dblp">
|
||||
<h3><a href="https://dblp.uni-trier.de">DBLP</a> - CS Bibliography</h3>
|
||||
<div class="list">
|
||||
<a href="https://dblp.uni-trier.de/db/journals/corr/corr1805.html#abs-1805-06358" title="listing on DBLP">listing</a> | <a href="https://dblp.uni-trier.de/rec/bibtex/journals/corr/abs-1805-06358" title="DBLP bibtex record">bibtex</a> </div>
|
||||
<div class="list">
|
||||
<a href="https://dblp.uni-trier.de/search/author?author=Nuno%20M.%20Pregui%C3%A7a" title="DBLP author search">Nuno M. Preguiça</a><br/><a href="https://dblp.uni-trier.de/search/author?author=Carlos%20Baquero" title="DBLP author search">Carlos Baquero</a><br/><a href="https://dblp.uni-trier.de/search/author?author=Marc%20Shapiro" title="DBLP author search">Marc Shapiro</a> </div>
|
||||
</div><div class='extra-ref-cite'>
|
||||
<button type="button" id='bib-cite-trigger' class="bib-cite-button abs-button">export BibTeX citation</button>
|
||||
<span id='bib-cite-loading' hidden='true'>Loading...</span>
|
||||
</div>
|
||||
|
||||
<div id='bib-cite-modal' class='bib-modal' hidden='true'>
|
||||
<div class='bib-modal-content'>
|
||||
<div class='bib-modal-title'>
|
||||
<h2>BibTeX formatted citation</h2>
|
||||
<button type="button" class='bib-modal-close' aria-label="Close">×</button>
|
||||
</div>
|
||||
<div>
|
||||
<textarea id='bib-cite-target' class="bib-citation-content" aria-label="loading the citation">loading...</textarea>
|
||||
</div>
|
||||
<div>
|
||||
<span>Data provided by: </span>
|
||||
<a id='bib-cite-source-api'></a>
|
||||
</div>
|
||||
</div>
|
||||
</div><div class="bookmarks">
|
||||
<div><h3>Bookmark</h3></div><a class="abs-button abs-button-grey abs-button-small" href="http://www.bibsonomy.org/BibtexHandler?requTask=upload&url=https://arxiv.org/abs/1805.06358&description=Conflict-free Replicated Data Types (CRDTs)"
|
||||
title="Bookmark on BibSonomy">
|
||||
<img src="/static/browse/0.3.4/images/icons/social/bibsonomy.png"
|
||||
alt="BibSonomy"/>
|
||||
</a>
|
||||
<a class="abs-button abs-button-grey abs-button-small" href="https://reddit.com/submit?url=https://arxiv.org/abs/1805.06358&title=Conflict-free Replicated Data Types (CRDTs)"
|
||||
title="Bookmark on Reddit">
|
||||
<img src="/static/browse/0.3.4/images/icons/social/reddit.png"
|
||||
alt="Reddit"/>
|
||||
</a>
|
||||
</div> </div>
|
||||
<!--end extra-services-->
|
||||
<!-- LABS AREA -->
|
||||
<div id="labstabs">
|
||||
<div class="labstabs" role="tablist" aria-label="arXivLabs tools"><input type="radio" name="tabs" id="tabone"checked="checked"class="labs-tab-input">
|
||||
<label for="tabone" role="tab" aria-selected="true" aria-controls="tabpanel-one" id="tab-label-one">Bibliographic Tools</label>
|
||||
<div class="tab labs-display-bib" role="tabpanel" id="tabpanel-one" aria-labelledby="tab-label-one">
|
||||
<h1>Bibliographic and Citation Tools</h1>
|
||||
<div class="toggle">
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input id="bibex-toggle" type="checkbox" class="lab-toggle"
|
||||
data-script-url="/static/browse/0.3.4/bibex/bibex.js?20241202">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">Bibliographic Explorer Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-bibex">Bibliographic Explorer</span> <em>(<a href="https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer">What is the Explorer?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="connectedpapers-toggle"
|
||||
type="checkbox"
|
||||
class="lab-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/connectedpapers.js"
|
||||
aria-labelledby="label-for-connected-papers">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">Connected Papers Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-connected-papers">Connected Papers</span> <em>(<a href="https://www.connectedpapers.com/about" target="_blank">What is Connected Papers?</a>)</em>
|
||||
</div>
|
||||
</div><div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="litmaps-toggle"
|
||||
type="checkbox"
|
||||
class="lab-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/litmaps.js?20210617"
|
||||
aria-labelledby="label-for-litmaps">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">Litmaps Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-litmaps">Litmaps</span> <em>(<a href="https://www.litmaps.co/" target="_blank">What is Litmaps?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="scite-toggle"
|
||||
type="checkbox"
|
||||
class="lab-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/scite.js?20210617"
|
||||
aria-labelledby="label-for-scite">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">scite.ai Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-scite">scite Smart Citations</span> <em>(<a href="https://www.scite.ai/" target="_blank">What are Smart Citations?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="labs-content-placeholder labs-display" style="display: none;"></div>
|
||||
<div style="min-height: 15px" id="connectedpapers-output"></div>
|
||||
<div style="min-height: 15px" id="litmaps-open-in"></div>
|
||||
<div style="min-height: 15px" id="scite-open-in"></div>
|
||||
</div>
|
||||
|
||||
|
||||
<input type="radio" name="tabs" id="tabtwo" class="labs-tab-input">
|
||||
<label for="tabtwo" role="tab" aria-selected="false" aria-controls="tabpanel-two" id="tab-label-two">Code, Data, Media</label>
|
||||
<div class="tab" role="tabpanel" id="tabpanel-two" aria-labelledby="tab-label-two">
|
||||
<h1>Code, Data and Media Associated with this Article</h1>
|
||||
<div class="toggle">
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="alphaxiv-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/alphaxiv.js"
|
||||
type="checkbox" class="lab-toggle" aria-labelledby="label-for-alphaxiv">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">alphaXiv Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-alphaxiv">alphaXiv</span> <em>(<a href="https://alphaxiv.org/" target="_blank">What is alphaXiv?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="catalyzex-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/catalyzex.js"
|
||||
type="checkbox" class="lab-toggle" aria-labelledby="label-for-cx">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">Links to Code Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-cx">CatalyzeX Code Finder for Papers</span> <em>(<a href="https://www.catalyzex.com" target="_blank">What is CatalyzeX?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="dagshub-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/dagshub.js"
|
||||
type="checkbox" class="lab-toggle" aria-labelledby="label-for-dagshub">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">DagsHub Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-dagshub">DagsHub</span> <em>(<a href="https://dagshub.com/" target="_blank">What is DagsHub?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="gotitpub-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/gotitpub.js"
|
||||
type="checkbox" class="lab-toggle" aria-labelledby="label-for-gotitpub">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">GotitPub Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-gotitpub">Gotit.pub</span> <em>(<a href="http://gotit.pub/faq" target="_blank">What is GotitPub?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="huggingface-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/huggingface.js"
|
||||
type="checkbox" class="lab-toggle" aria-labelledby="label-for-huggingface">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">Huggingface Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-huggingface">Hugging Face</span> <em>(<a href="https://huggingface.co/huggingface" target="_blank">What is Huggingface?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="sciencecast-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/sciencecast.js"
|
||||
type="checkbox" class="lab-toggle" aria-labelledby="label-for-sciencecast">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">ScienceCast Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-sciencecast">ScienceCast</span> <em>(<a href="https://sciencecast.org/welcome" target="_blank">What is ScienceCast?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="alphaxiv-output" style="display:none"></div>
|
||||
<div id="catalyzex-output" style="display:none"></div>
|
||||
<div id="dagshub-output" style="display:none"></div>
|
||||
<div id="gotitpub-output" style="display:none"></div>
|
||||
<div id="sciencecast-output" style="display:none"></div>
|
||||
<div id="huggingface-output" style="display:none"></div>
|
||||
</div>
|
||||
|
||||
|
||||
<input type="radio" name="tabs" id="labstabs-demos-input" class="labs-tab-input">
|
||||
<label for="labstabs-demos-input" id="labstabs-demos-label" role="tab" aria-selected="false" aria-controls="tabpanel-demos">Demos</label>
|
||||
<div class="tab" role="tabpanel" id="tabpanel-demos" aria-labelledby="labstabs-demos-label">
|
||||
<h1>Demos</h1>
|
||||
<div class="toggle">
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="replicate-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/replicate.js"
|
||||
type="checkbox" class="lab-toggle" aria-labelledby="label-for-replicate">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">Replicate Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-replicate">Replicate</span> <em>(<a href="https://replicate.com/docs/arxiv/about" target="_blank">What is Replicate?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="spaces-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/spaces.js"
|
||||
type="checkbox" class="lab-toggle" aria-labelledby="label-for-spaces">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">Spaces Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-spaces">Hugging Face Spaces</span> <em>(<a href="https://huggingface.co/docs/hub/spaces" target="_blank">What is Spaces?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input
|
||||
id="txyz-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/txyz.js"
|
||||
type="checkbox" class="lab-toggle" aria-labelledby="label-for-txyz">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">Spaces Toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-txyz">TXYZ.AI</span> <em>(<a href="https://txyz.ai" target="_blank">What is TXYZ.AI?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div id="replicate-output"></div>
|
||||
<div id="spaces-output"></div>
|
||||
<div id="txyz-output"></div>
|
||||
</div>
|
||||
<input type="radio" name="tabs" id="tabfour" class="labs-tab-input">
|
||||
<label for="tabfour" role="tab" aria-selected="false" aria-controls="tabpanel-four" id="tab-label-four">Related Papers</label>
|
||||
<div class="tab" role="tabpanel" id="tabpanel-four" aria-labelledby="tab-label-four">
|
||||
<h1>Recommenders and Search Tools</h1>
|
||||
<div class="toggle">
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input id="influenceflower-toggle"
|
||||
data-script-url="/static/browse/0.3.4/js/influenceflower.js"
|
||||
type="checkbox" class="lab-toggle" aria-labelledby="label-for-influenceflower">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">Link to Influence Flower</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-influenceflower">Influence Flower</span> <em>(<a href="https://influencemap.cmlab.dev/" target="_blank">What are Influence Flowers?</a>)</em>
|
||||
</div>
|
||||
</div>
|
||||
<div class="columns is-mobile lab-row">
|
||||
<div class="column lab-switch">
|
||||
<label class="switch">
|
||||
<input id="core-recommender-toggle" type="checkbox" class="lab-toggle" aria-labelledby="label-for-core">
|
||||
<span class="slider"></span>
|
||||
<span class="is-sr-only">Core recommender toggle</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="column lab-name">
|
||||
<span id="label-for-core">CORE Recommender</span> <em>(<a href="https://core.ac.uk/services/recommender">What is CORE?</a>)</em>
|
||||
</div>
|
||||
</div></div>
|
||||
<div id="influenceflower-output"></div>
|
||||
<div id="influenceflower-output-graph" style="display:none">
|
||||
<ul class="flower-tabs">
|
||||
<li class="active"><a class="btn tab-btn" onclick="openTab(event, 'tab-author')">Author</a></li>
|
||||
<li><a class="btn tab-btn" onclick="openTab(event, 'tab-venue')">Venue</a></li>
|
||||
<li><a class="btn tab-btn" onclick="openTab(event, 'tab-inst')">Institution</a></li>
|
||||
<li><a class="btn tab-btn" onclick="openTab(event, 'tab-topic')">Topic</a></li>
|
||||
</ul>
|
||||
<div class="flower-tab-content">
|
||||
<div class="tab-flower active" id="tab-author"><svg id="flower-graph-author"></svg></div>
|
||||
<div class="tab-flower" id="tab-venue"><svg id="flower-graph-venue"></svg></div>
|
||||
<div class="tab-flower" id="tab-inst"><svg id="flower-graph-inst"></svg></div>
|
||||
<div class="tab-flower" id="tab-topic"><svg id="flower-graph-topic"></svg></div>
|
||||
</div>
|
||||
</div>
|
||||
<div id="coreRecommenderOutput"></div>
|
||||
<div id="iarxivOutput"></div>
|
||||
</div>
|
||||
|
||||
<input type="radio" name="tabs" id="tabfive" class="labs-tab-input">
|
||||
<label for="tabfive" role="tab" aria-selected="false" aria-controls="tabpanel-five" id="tab-label-five">
|
||||
About arXivLabs
|
||||
</label>
|
||||
<div class="tab" role="tabpanel" id="tabpanel-five" aria-labelledby="tab-label-five">
|
||||
<div class="columns">
|
||||
<div class="column">
|
||||
<h1>arXivLabs: experimental projects with community collaborators</h1>
|
||||
<p>arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.</p>
|
||||
<p>Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.</p>
|
||||
<p>Have an idea for a project that will add value for arXiv's community? <a href="https://info.arxiv.org/labs/index.html"><strong>Learn more about arXivLabs</strong></a>.</p>
|
||||
</div>
|
||||
<div class="column is-narrow is-full-mobile">
|
||||
<p class="icon-labs"><svg xmlns="http://www.w3.org/2000/svg" role="presentation" viewBox="0 0 635.572 811"><path d="M175.6 676v27h-27v-27zm-54 27v27h27v-27zm-27 27v27h27v-27zm396-54v27h-27v-27zm0 27v27h27v-27zm27 27v27h27v-27zm-27-414h27v27h-27zm27 0h27v-27h-27zm27-27h27v-27h-27zm-396 45h-27v-27h27zm-27-54h-27v27h27zm-27-27h-27v27h27z"/><path d="M94.6 730v27h-27v-27zm477 0v27h-27v-27zm-27-495h27v27h-27zm-450 18h-27v-27h27zm477 9h27v27h-27zm-54 495h27v27h-27zm-423 0h27v27h-27zm-54-504h27v27h-27z" fill="#666"/><path d="M67.6 730v27h-27v-27zm54 54v27h-27v-27zm0-108v27h27v-27zm-27 27v27h27v-27zm-81 0v27h27v-27zm585 27v27h-27v-27zm-108-54v27h27v-27zm27 27v27h27v-27zm81 0v27h27v-27zm-54-495h27v27h-27zm-54 108h27v-27h-27zm27-27h27v-27h-27zm0-81h27v-27h-27zm-423 18h-27v-27h27zm54 54h-27v27h27zm-27-27h-27v27h27zm0-81h-27v27h27zm423 612v27h-27v-27zm81-522v27h-27v-27zm-585-9v27h-27v-27z" fill="#999"/><path d="M94.6 784v27h-27v-27zm-27-27v27h27v-27zm-27-54v27h27v-27zm27 0v27h27v-27zm0-27v27h27v-27zm27 0v27h27v-27zm0-27v27h27v-27zm27 0v27h27v-27zm-108 81v27h27v-27zm558 54v27h-27v-27zm-27-27v27h27v-27zm27-54v27h27v-27zm-27 0v27h27v-27zm0-27v27h27v-27zm-27 0v27h27v-27zm0-27v27h27v-27zm-27 0v27h27v-27zm108 81v27h27v-27zm0-495h27v27h-27zm-27 27h27v-27h-27zm-54-27h27v-27h-27zm0 27h27v-27h-27zm-27 0h27v-27h-27zm0 27h27v-27h-27zm-27 0h27v-27h-27zm0 27h27v-27h-27zm81-108h27v-27h-27zm-504 45h-27v-27h27zm27-27h-27v27h27zm54-27h-27v27h27zm0 27h-27v27h27zm27 0h-27v27h27zm0 27h-27v27h27zm27 0h-27v27h27zm0 27h-27v27h27zm-81-108h-27v27h27z" fill="#ccc"/><path d="M598.6 665.1H41.5C-76.5 667 176 280.2 176 280.2h53a46.5 46.5 0 0162.8-56.3 29.2 29.2 0 1128.5 35.9h-1a46.5 46.5 0 01-1.5 20.3l142.5-.1s255.3 387 138.3 385.1zM291 181a29.3 29.3 0 10-29.2-29.3A29.3 29.3 0 00291 181zm65.4-66.8a22.4 22.4 0 10-22.5-22.4 22.4 22.4 0 0022.5 22.4z" fill="#fc0"/><path d="M245.5 172V10h153v162s324 495 198 495h-558c-126 0 207-495 207-495zm126 54h56m-13 72h56m-9 72h56m-20 72h56m-22 72h56m-29 72h56m-457-45c20.8 41.7 87.3 81 160.7 81 72.1 0 142.1-38.2 163.4-81" fill="none" stroke="#000" stroke-miterlimit="10" stroke-width="20"/><path d="M273.3 421.7c0 31-9.8 56.3-21.9 56.3s-21.8-25.2-21.8-56.3 9.8-56.3 21.8-56.3 21.9 25.2 21.9 56.3zm114.4-56.3c-12 0-21.8 25.2-21.8 56.3s9.7 56.3 21.8 56.3 21.9-25.2 21.9-56.3-9.8-56.3-21.9-56.3zM150.1 526.6c-18.2 6.7-27.5 22.9-23.2 30.2s14.8-5.5 33-12.2 37.4-4.9 33-12.2-24.5-12.6-42.8-5.8zm296 5.8c-4.2 7.3 14.9 5.5 33.1 12.2s28.7 19.5 33 12.2-5-23.5-23.2-30.2-38.5-1.5-42.8 5.8z"/></svg></p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<!-- END LABS AREA -->
|
||||
<div class="endorsers">
|
||||
<a href="/auth/show-endorsers/1805.06358" class="endorser-who" rel="nofollow">Which authors of this paper are endorsers?</a> |
|
||||
<a id="mathjax_toggle" href="javascript:setMathjaxCookie()">Disable MathJax</a> (<a href="https://info.arxiv.org/help/mathjax.html">What is MathJax?</a>)
|
||||
<span class="help" style="font-style: normal; float: right; margin-top: 0; margin-right: 1em;"></span>
|
||||
</div>
|
||||
<script type="text/javascript" language="javascript">mathjaxToggle();</script>
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
|
||||
<footer style="clear: both;">
|
||||
<div class="columns is-desktop" role="navigation" aria-label="Secondary" style="margin: -0.75em -0.75em 0.75em -0.75em">
|
||||
<!-- Macro-Column 1 -->
|
||||
<div class="column" style="padding: 0;">
|
||||
<div class="columns">
|
||||
<div class="column">
|
||||
<ul style="list-style: none; line-height: 2;">
|
||||
<li><a href="https://info.arxiv.org/about">About</a></li>
|
||||
<li><a href="https://info.arxiv.org/help">Help</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="column">
|
||||
<ul style="list-style: none; line-height: 2;">
|
||||
<li>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" class="icon filter-black" role="presentation"><title>contact arXiv</title><desc>Click here to contact arXiv</desc><path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"/></svg>
|
||||
<a href="https://info.arxiv.org/help/contact.html"> Contact</a>
|
||||
</li>
|
||||
<li>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" class="icon filter-black" role="presentation"><title>subscribe to arXiv mailings</title><desc>Click here to subscribe</desc><path d="M476 3.2L12.5 270.6c-18.1 10.4-15.8 35.6 2.2 43.2L121 358.4l287.3-253.2c5.5-4.9 13.3 2.6 8.6 8.3L176 407v80.5c0 23.6 28.5 32.9 42.5 15.8L282 426l124.6 52.2c14.2 6 30.4-2.9 33-18.2l72-432C515 7.8 493.3-6.8 476 3.2z"/></svg>
|
||||
<a href="https://info.arxiv.org/help/subscribe"> Subscribe</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- End Macro-Column 1 -->
|
||||
<!-- Macro-Column 2 -->
|
||||
<div class="column" style="padding: 0;">
|
||||
<div class="columns">
|
||||
<div class="column">
|
||||
<ul style="list-style: none; line-height: 2;">
|
||||
<li><a href="https://info.arxiv.org/help/license/index.html">Copyright</a></li>
|
||||
<li><a href="https://info.arxiv.org/help/policies/privacy_policy.html">Privacy Policy</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="column sorry-app-links">
|
||||
<ul style="list-style: none; line-height: 2;">
|
||||
<li><a href="https://info.arxiv.org/help/web_accessibility.html">Web Accessibility Assistance</a></li>
|
||||
<li>
|
||||
<p class="help">
|
||||
<a class="a11y-main-link" href="https://status.arxiv.org" target="_blank">arXiv Operational Status <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 256 512" class="icon filter-dark_grey" role="presentation"><path d="M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z"/></svg></a><br>
|
||||
</p>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div> <!-- end MetaColumn 2 -->
|
||||
<!-- End Macro-Column 2 -->
|
||||
</div>
|
||||
</footer>
|
||||
</div>
|
||||
|
||||
<script src="/static/base/1.0.1/js/member_acknowledgement.js"></script>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
4897
research/cultnet-distributed-database/dynamo-amazon-science.html
Normal file
4897
research/cultnet-distributed-database/dynamo-amazon-science.html
Normal file
File diff suppressed because one or more lines are too long
BIN
research/cultnet-distributed-database/dynamo-sosp2007.pdf
Normal file
BIN
research/cultnet-distributed-database/dynamo-sosp2007.pdf
Normal file
Binary file not shown.
11793
research/cultnet-distributed-database/firebase-realtime-offline.html
Normal file
11793
research/cultnet-distributed-database/firebase-realtime-offline.html
Normal file
File diff suppressed because one or more lines are too long
Binary file not shown.
676
research/cultnet-distributed-database/raft-extended.html
Normal file
676
research/cultnet-distributed-database/raft-extended.html
Normal file
|
|
@ -0,0 +1,676 @@
|
|||
<!DOCTYPE html>
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<title>In Search of an Understandable Consensus Algorithm (Extended Version)</title>
|
||||
<meta name="generator" content="Org mode" />
|
||||
<meta name="author" content="Diego Ongaro and John Ousterhout, Stanford University" />
|
||||
<link rel="stylesheet" type="text/css" href="/themes/readtheorg/styles/readtheorg/css/htmlize.css"/>
|
||||
<link rel="stylesheet" type="text/css" href="/themes/readtheorg/styles/readtheorg/css/readtheorg.css"/>
|
||||
<script src="/themes/jquery/2.1.3/jquery.min.js"></script>
|
||||
<script src="/themes/bootstrap/3.3.4/js/bootstrap.min.js"></script>
|
||||
<script type="text/javascript" src="/themes/readtheorg/styles/lib/js/jquery.stickytableheaders.min.js"></script>
|
||||
<script type="text/javascript" src="/themes/readtheorg/styles/readtheorg/js/readtheorg.js"></script>
|
||||
</head>
|
||||
<body>
|
||||
<div id="org-div-home-and-up">
|
||||
<a accesskey="h" href="../index.html"> UP </a>
|
||||
|
|
||||
<a accesskey="H" href="paperlist.html"> HOME </a>
|
||||
</div><div id="content">
|
||||
<h1 class="title">In Search of an Understandable Consensus Algorithm (Extended Version)</h1>
|
||||
<div id="table-of-contents">
|
||||
<h2>Table of Contents</h2>
|
||||
<div id="text-table-of-contents">
|
||||
<ul>
|
||||
<li><a href="#orgf960eb7">DECLARATION</a></li>
|
||||
<li><a href="#orgd05d3d0">Abstract</a></li>
|
||||
<li><a href="#orge9839bb">1. Introduction</a></li>
|
||||
<li><a href="#orgae8bb96">2. Replicated state machines</a></li>
|
||||
<li><a href="#org1fe8017">3. What's wrong with Paxos?</a></li>
|
||||
<li><a href="#orgfc9c6b4">4. Designing for understandability</a></li>
|
||||
<li><a href="#orge3d8f10">5. The Raft consensus algorithm</a>
|
||||
<ul>
|
||||
<li><a href="#org752f98e">5.1. Raft basics</a></li>
|
||||
<li><a href="#org291797f">5.2. Leader election</a></li>
|
||||
<li><a href="#org44bb0a6">5.3. Log replication</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="outline-container-orgf960eb7" class="outline-2">
|
||||
<h2 id="orgf960eb7">DECLARATION</h2>
|
||||
<div class="outline-text-2" id="text-orgf960eb7">
|
||||
<p>
|
||||
This page is the In Search of an Understandable Consensus Algorithm
|
||||
(Extended Version) paper. Original Paper Link
|
||||
is: <a href="https://raft.github.io/raft.pdf">https://raft.github.io/raft.pdf</a>
|
||||
</p>
|
||||
|
||||
<p>
|
||||
In Search of an Understandable Consensus Algorithm<br />
|
||||
(Extended Version)<br />
|
||||
Diego Ongaro and John Ousterhout<br />
|
||||
Stanford University
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="outline-container-orgd05d3d0" class="outline-2">
|
||||
<h2 id="orgd05d3d0">Abstract</h2>
|
||||
<div class="outline-text-2" id="text-orgd05d3d0">
|
||||
<p>
|
||||
Raft is a consensus algorithm for managing a replicated log. It produces a
|
||||
result equivalent to (multi-)Paxos, and it is as efficient as Paxos, but its
|
||||
structure is different from Paxos; this makes Raft more understandable than
|
||||
Paxos and also provides a better foundation for building practical systems.
|
||||
In order to enhance understandability, Raft separates the key elements of
|
||||
consensus, such as leader election, log replication, and safety, and it enforces
|
||||
a stronger degree of coherency to reduce the number of states that must be
|
||||
considered. Results from a user study demonstrate that Raft is easier for
|
||||
students to learn than Paxos. Raft also includes a new mechanism for changing
|
||||
the cluster membership, which uses overlapping majorities to guarantee safety.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="outline-container-orge9839bb" class="outline-2">
|
||||
<h2 id="orge9839bb"><span class="section-number-2">1</span> Introduction</h2>
|
||||
<div class="outline-text-2" id="text-1">
|
||||
<p>
|
||||
Consensus algorithms allow a collection of machines to work as a coherent group
|
||||
that can survive the failures of some of its members. Because of this, they play
|
||||
a key role in building reliable large-scale software systems. Paxos [15, 16] has
|
||||
dominated the discussion of consensus algorithms over the last decade: most
|
||||
implementations of consensus are based on Paxos or influenced by it, and Paxos
|
||||
has become the primary vehicle used to teach students about consensus.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Unfortunately, Paxos is quite difficult to understand, in spite of numerous
|
||||
attempts to make it more approachable. Furthermore, its architecture requires
|
||||
complex changes to support practical systems. As a result, both system builders
|
||||
and students struggle with Paxos.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
After struggling with Paxos ourselves, we set out to find a new consensus
|
||||
algorithm that could provide a better foundation for system building and
|
||||
education. Our approach was unusual in that our primary goal was
|
||||
understandability: could we define a consensus algorithm for practical systems
|
||||
and describe it in a way that is significantly easier to learn than Paxos?
|
||||
Furthermore, we wanted the algorithm to facilitate the development of intuitions
|
||||
that are essential for system builders. It was important not just for the
|
||||
algorithm to work, but for it to be obvious why it works.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
The result of this work is a consensus algorithm called Raft. In designing Raft
|
||||
we applied specific techniques to improve understandability,including
|
||||
decomposition (Raft separates leader election, log replication, and safety) and
|
||||
This tech report is an extended version of [32]; additional material is noted
|
||||
with a gray bar in the margin. Published May 20, 2014. state space reduction
|
||||
(relative to Paxos, Raft reduces the degree of nondeterminism and the ways
|
||||
servers can be inconsistent with each other). A user study with 43 students
|
||||
at two universities shows that Raft is significantly easier to understand than
|
||||
Paxos: after learning both algorithms, 33 of these students were able to answer
|
||||
questions about Raft better than questions about Paxos.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Raft is similar in many ways to existing consensus algorithms (most notably, Oki
|
||||
and Liskov's Viewstamped Replication [29, 22]), but it has several novel
|
||||
features:
|
||||
</p>
|
||||
|
||||
<p>
|
||||
1/. Strong leader: Raft uses a stronger form of leadership than other consensus
|
||||
algorithms. For example, log entries only flow from the leader to other servers.
|
||||
This simplifies the management of the replicated log and makes Raft easier to
|
||||
understand.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
2/. Leader election: Raft uses randomized timers to elect leaders. This adds
|
||||
only a small amount of mechanism to the heartbeats already required for any
|
||||
consensus algorithm, while resolving conflicts simply and rapidly.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
3/. Membership changes: Raft's mechanism for changing the set of servers in the
|
||||
cluster uses a new joint consensus approach where the majorities of two
|
||||
different configurations overlap during transitions. This allows the cluster to
|
||||
continue operating normally during configuration changes.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
We believe that Raft is superior to Paxos and other consensus algorithms, both
|
||||
for educational purposes and as a foundation for implementation. It is simpler
|
||||
and more understandable than other algorithms; it is described completely enough
|
||||
to meet the needs of a practical system; it has several open-source
|
||||
implementations and is used by several companies; its safety properties have
|
||||
been formally specified and proven; and its efficiency is comparable to other
|
||||
algorithms.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
The remainder of the paper introduces the replicated state machine problem
|
||||
(Section 2), discusses the strengths and weaknesses of Paxos (Section 3),
|
||||
describes our general approach to understandability (Section 4), presents the
|
||||
Raft consensus algorithm (Sections 5–8), evaluates Raft (Section 9), and
|
||||
discusses related work (Section 10).
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="outline-container-orgae8bb96" class="outline-2">
|
||||
<h2 id="orgae8bb96"><span class="section-number-2">2</span> Replicated state machines</h2>
|
||||
<div class="outline-text-2" id="text-2">
|
||||
<p>
|
||||
Consensus algorithms typically arise in the context of replicated state machines
|
||||
[37]. In this approach, state machines on a collection of servers compute
|
||||
identical copies of the same state and can continue operating even if some of
|
||||
the servers are down. Replicated state machines are used to solve a variety of
|
||||
fault tolerance problems in distributed systems. For example, large-scale
|
||||
systems that have a single cluster leader, such as GFS [8], HDFS [38], and
|
||||
RAMCloud [33], typically use a separate replicated state machine to manage
|
||||
leader election and store configuration information that must survive leader
|
||||
crashes. Examples of replicated state machines include Chubby [2] and
|
||||
ZooKeeper [11].
|
||||
</p>
|
||||
|
||||
|
||||
<div id="org1bdf0c5" class="figure">
|
||||
<p><img src="img/raft-figure1.jpg" alt="raft-figure1.jpg" />
|
||||
</p>
|
||||
<p><span class="figure-number">Figure 1: </span>Replicated state machine architecture. The consensus algorithm manages a replicated log containing state machine commands from clients. The state machines process identical sequences of commands from the logs, so they produce the same outputs.</p>
|
||||
</div>
|
||||
|
||||
<p>
|
||||
Replicated state machines are typically implemented using a replicated log, as
|
||||
shown in Figure 1. Each server stores a log containing a series of commands,
|
||||
which its state machine executes in order. Each log contains the same commands
|
||||
in the same order, so each state machine processes the same sequence of
|
||||
commands. Since the state machines are deterministic, each computes the same
|
||||
state and the same sequence of outputs.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Keeping the replicated log consistent is the job of the consensus algorithm.
|
||||
The consensus module on a server receives commands from clients and adds them to
|
||||
its log. It communicates with the consensus modules on other servers to ensure
|
||||
that every log eventually contains the same requests in the same order, even if
|
||||
some servers fail. Once commands are properly replicated, each server's state
|
||||
machine processes them in log order, and the outputs are returned to clients. As
|
||||
a result, the servers appear to form a single, highly reliable state machine.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Consensus algorithms for practical systems typically have the following
|
||||
properties:
|
||||
</p>
|
||||
|
||||
<p>
|
||||
• They ensure <i>safety</i> (never returning an incorrect result) under all
|
||||
non-Byzantine conditions, including network delays, partitions, and packet loss,
|
||||
duplication, and reordering.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
• They are fully functional (available) as long as any majority of the servers
|
||||
are operational and can communicate with each other and with clients. Thus, a
|
||||
typical cluster of five servers can tolerate the failure of any two servers.
|
||||
Servers are assumed to fail by stopping; they may later recover from state on
|
||||
stable storage and rejoin the cluster.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
• They do not depend on timing to ensure the consistency of the logs: faulty
|
||||
clocks and extreme message delays can, at worst, cause availability problems.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
• In the common case, a command can complete as soon as a majority of the
|
||||
cluster has responded to a single round of remote procedure calls; a minority of
|
||||
slow servers need not impact overall system performance.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="outline-container-org1fe8017" class="outline-2">
|
||||
<h2 id="org1fe8017"><span class="section-number-2">3</span> What's wrong with Paxos?</h2>
|
||||
<div class="outline-text-2" id="text-3">
|
||||
<p>
|
||||
Over the last ten years, Leslie Lamport's Paxos protocol [15] has become almost
|
||||
synonymous with consensus: it is the protocol most commonly taught in courses,
|
||||
and most implementations of consensus use it as a starting point. Paxos first
|
||||
defines a protocol capable of reaching agreement on a single decision, such as
|
||||
a single replicated log entry. We refer to this subset as single-decree Paxos.
|
||||
Paxos then combines multiple instances of this protocol to facilitate a series
|
||||
of decisions such as a log (multi-Paxos). Paxos ensures both safety and
|
||||
liveness, and it supports changes in cluster membership. Its correctness has
|
||||
been proven, and it is efficient in the normal case.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Unfortunately, Paxos has two significant drawbacks. The first drawback is that
|
||||
Paxos is exceptionally difficult to understand. The full explanation [15] is
|
||||
notoriously opaque; few people succeed in understanding it, and only with great
|
||||
effort. As a result, there have been several attempts to explain Paxos in
|
||||
simpler terms [16, 20, 21]. These explanations focus on the single-decree
|
||||
subset, yet they are still challenging. In an informal survey of attendees at
|
||||
NSDI 2012, we found few people who were comfortable with Paxos, even among
|
||||
seasoned researchers. We struggled with Paxos ourselves; we were not able to
|
||||
understand the complete protocol until after reading several simplified
|
||||
explanations and designing our own alternative protocol, a process that took
|
||||
almost a year.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
We hypothesize that Paxos' opaqueness derives from its choice of the
|
||||
single-decree subset as its foundation. Single-decree Paxos is dense and subtle:
|
||||
it is divided into two stages that do not have simple intuitive explanations and
|
||||
cannot be understood independently. Because of this, it is difficult to develop
|
||||
intuitions about why the single-decree protocol works. The composition rules for
|
||||
multi-Paxos add significant additional complexity and subtlety. We believe that
|
||||
the overall problem of reaching consensus on multiple decisions (i.e., a log
|
||||
instead of a single entry) can be decomposed in other ways that are more direct
|
||||
and obvious.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
The second problem with Paxos is that it does not provide a good foundation for
|
||||
building practical implementations. One reason is that there is no widely
|
||||
agreed-upon algorithm for multi-Paxos. Lamport's descriptions are mostly about
|
||||
single-decree Paxos; he sketched possible approaches to multi-Paxos, but many
|
||||
details are missing. There have been several attempts to flesh out and optimize
|
||||
Paxos, such as [26], [39], and [13], but these differ from each other and from
|
||||
Lamport's sketches. Systems such as Chubby [4] have implemented Paxos-like
|
||||
algorithms, but in most cases their details have not been published.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Furthermore, the Paxos architecture is a poor one for building practical
|
||||
systems; this is another consequence of the single-decree decomposition. For
|
||||
example, there is little benefit to choosing a collection of log entries
|
||||
independently and then melding them into a sequential log; this just adds
|
||||
complexity. It is simpler and more efficient to design a system around a log,
|
||||
where new entries are appended sequentially in a constrained order. Another
|
||||
problem is that Paxos uses a symmetric peer-to-peer approach at its core (though
|
||||
it eventually suggests a weak form of leadership as a performance optimization).
|
||||
This makes sense in a simplified world where only one decision will be made, but
|
||||
few practical systems use this approach. If a series of decisions must be made,
|
||||
it is simpler and faster to first elect a leader, then have the leader
|
||||
coordinate the decisions.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
As a result, practical systems bear little resemblance to Paxos. Each
|
||||
implementation begins with Paxos, discovers the difficulties in implementing it,
|
||||
and then develops a significantly different architecture. This is timeconsuming
|
||||
and error-prone, and the difficulties of understanding Paxos exacerbate the
|
||||
problem. Paxos' formulation may be a good one for proving theorems about its
|
||||
correctness, but real implementations are so different from Paxos that the
|
||||
proofs have little value. The following comment from the Chubby implementers is
|
||||
typical:
|
||||
</p>
|
||||
|
||||
<blockquote>
|
||||
<p>
|
||||
There are significant gaps between the description of the Paxos algorithm and
|
||||
the needs of a real-world system. . . . the final system will be based on an
|
||||
unproven protocol [4].
|
||||
</p>
|
||||
</blockquote>
|
||||
|
||||
<p>
|
||||
Because of these problems, we concluded that Paxos does not provide a good
|
||||
foundation either for system building or for education. Given the importance of
|
||||
consensus in large-scale software systems, we decided to see if we could design
|
||||
an alternative consensus algorithm with better properties than Paxos. Raft is
|
||||
the result of that experiment.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="outline-container-orgfc9c6b4" class="outline-2">
|
||||
<h2 id="orgfc9c6b4"><span class="section-number-2">4</span> Designing for understandability</h2>
|
||||
<div class="outline-text-2" id="text-4">
|
||||
<p>
|
||||
We had several goals in designing Raft: it must provide a complete and practical
|
||||
foundation for system building, so that it significantly reduces the amount of
|
||||
design work required of developers; it must be safe under all conditions and
|
||||
available under typical operating conditions; and it must be efficient for
|
||||
common operations. But our most important goal—and most difficult challenge
|
||||
was understandability. It must be possible for a large audience to understand
|
||||
the algorithm comfortably. In addition, it must be possible to develop
|
||||
intuitions about the algorithm, so that system builders can make the extensions
|
||||
that are inevitable in real-world implementations.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
There were numerous points in the design of Raft where we had to choose among
|
||||
alternative approaches. In these situations we evaluated the alternatives
|
||||
based on understandability: how hard is it to explain each alternative (for
|
||||
example, how complex is its state space, and does it have subtle implications?),
|
||||
and how easy will it be for a reader to completely understand the approach and
|
||||
its implications?
|
||||
</p>
|
||||
|
||||
<p>
|
||||
We recognize that there is a high degree of subjectivity in such analysis;
|
||||
nonetheless, we used two techniques that are generally applicable. The first
|
||||
technique is the well-known approach of problem decomposition: wherever
|
||||
possible, we divided problems into separate pieces that could be solved,
|
||||
explained, and understood relatively independently. For example, in Raft we
|
||||
separated leader election, log replication, safety, and membership changes.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Our second approach was to simplify the state space by reducing the number of
|
||||
states to consider, making the system more coherent and eliminating
|
||||
nondeterminism where possible. Specifically, logs are not allowed to have holes,
|
||||
and Raft limits the ways in which logs can become inconsistent with each other.
|
||||
Although in most cases we tried to eliminate nondeterminism, there are some
|
||||
situations where nondeterminism actually improves understandability.
|
||||
In particular, randomized approaches introduce nondeterminism, but they tend to
|
||||
reduce the state space by handling all possible choices in a similar fashion
|
||||
("choose any; it doesn't matter"). We used randomization to simplify the Raft
|
||||
leader election algorithm.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="outline-container-orge3d8f10" class="outline-2">
|
||||
<h2 id="orge3d8f10"><span class="section-number-2">5</span> The Raft consensus algorithm</h2>
|
||||
<div class="outline-text-2" id="text-5">
|
||||
<p>
|
||||
Raft is an algorithm for managing a replicated log of the form described in
|
||||
Section 2. Figure 2 summarizes the algorithm in condensed form for reference,
|
||||
and Figure 3 lists key properties of the algorithm; the elements of these
|
||||
figures are discussed piecewise over the rest of this section.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
<img src="img/raft-figure2-1.jpg" alt="raft-figure2-1.jpg" />
|
||||
<img src="img/raft-figure2-2.jpg" alt="raft-figure2-2.jpg" />
|
||||
<img src="img/raft-figure2-3.jpg" alt="raft-figure2-3.jpg" />
|
||||
</p>
|
||||
|
||||
<div id="orgf3f186a" class="figure">
|
||||
<p><img src="img/raft-figure2-4.jpg" alt="raft-figure2-4.jpg" />
|
||||
</p>
|
||||
<p><span class="figure-number">Figure 2: </span>A condensed summary of the Raft consensus algorithm (excluding membership changes and log compaction). The server behavior in the upper-left box is described as a set of rules that trigger independently and repeatedly. Section numbers such as §5.2 indicate where particular features are discussed.A formal specification [31] describes the algorithm more precisely.</p>
|
||||
</div>
|
||||
|
||||
<p>
|
||||
Raft implements consensus by first electing a distinguished leader, then giving
|
||||
the leader complete responsibility for managing the replicated log. The leader
|
||||
accepts log entries from clients, replicates them on other servers, and tells
|
||||
servers when it is safe to apply log entries to their state machines. Having a
|
||||
leader simplifies the management of the replicated log. For example, the leader
|
||||
can decide where to place new entries in the log without consulting other
|
||||
servers, and data flows in a simple fashion from the leader to other servers.
|
||||
A leader can fail or become disconnected from the other servers, in which case
|
||||
a new leader is elected.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Given the leader approach, Raft decomposes the consensus problem into three
|
||||
relatively independent subproblems, which are discussed in the subsections that
|
||||
follow:
|
||||
</p>
|
||||
|
||||
<p>
|
||||
• Leader election: a new leader must be chosen when an existing leader fails
|
||||
(Section 5.2).
|
||||
</p>
|
||||
|
||||
<p>
|
||||
• Log replication: the leader must accept log entries from clients and replicate
|
||||
them across the cluster, forcing the other logs to agree with its own (Section
|
||||
5.3).
|
||||
• Safety: the key safety property for Raft is the State Machine Safety Property
|
||||
in Figure 3: if any server has applied a particular log entry to its state
|
||||
machine, then no other server may apply a different command for the same log
|
||||
index. Section 5.4 describes how Raft ensures this property; the solution
|
||||
involves an additional restriction on the election mechanism described in
|
||||
Section 5.2.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
After presenting the consensus algorithm, this section discusses the issue of
|
||||
availability and the role of timing in the system.
|
||||
</p>
|
||||
|
||||
|
||||
<div id="org159660c" class="figure">
|
||||
<p><img src="img/raft-figure3.jpg" alt="raft-figure3.jpg" />
|
||||
</p>
|
||||
<p><span class="figure-number">Figure 3: </span>Raft guarantees that each of these properties is true at all times. The section numbers indicate where each property is discussed.</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="outline-container-org752f98e" class="outline-3">
|
||||
<h3 id="org752f98e"><span class="section-number-3">5.1</span> Raft basics</h3>
|
||||
<div class="outline-text-3" id="text-5-1">
|
||||
<p>
|
||||
A Raft cluster contains several servers; five is a typical number, which allows
|
||||
the system to tolerate two failures. At any given time each server is in one of
|
||||
three states: leader, follower, or candidate. In normal operation there is
|
||||
exactly one leader and all of the other servers are followers. Followers are
|
||||
passive: they issue no requests on their own but simply respond to requests from
|
||||
leaders and candidates. The leader handles all client requests (if a client
|
||||
contacts a follower, the follower redirects it to the leader). The third state,
|
||||
candidate, is used to elect a new leader as described in Section 5.2. Figure 4
|
||||
shows the states and their transitions; the transitions are discussed below.
|
||||
</p>
|
||||
|
||||
<div id="org3f316c7" class="figure">
|
||||
<p><img src="img/raft-figure4.jpg" alt="raft-figure4.jpg" />
|
||||
</p>
|
||||
<p><span class="figure-number">Figure 4: </span>Server states. Followers only respond to requests from other servers. If a follower receives no communication, it becomes a candidate and initiates an election. A candidate that receives votes from a majority of the full cluster becomes the new leader. Leaders typically operate until they fail.</p>
|
||||
</div>
|
||||
|
||||
<p>
|
||||
Raft divides time into terms of arbitrary length, as shown in Figure 5. Terms
|
||||
are numbered with consecutive integers. Each term begins with an election, in
|
||||
which one or more candidates attempt to become leader as described in Section
|
||||
5.2. If a candidate wins the election, then it serves as leader for the rest of
|
||||
the term. In some situations an election will result in a split vote. In this
|
||||
case the term will end with no leader; a new term (with a new election) will
|
||||
begin shortly. Raft ensures that there is at most one leader in a given term.
|
||||
</p>
|
||||
|
||||
|
||||
<div id="orgc7ff282" class="figure">
|
||||
<p><img src="img/raft-figure5.jpg" alt="raft-figure5.jpg" />
|
||||
</p>
|
||||
<p><span class="figure-number">Figure 5: </span>Time is divided into terms, and each term begins with an election. After a successful election, a single leader manages the cluster until the end of the term. Some elections fail, in which case the term ends without choosing a leader. The transitions between terms may be observed at different times on different servers.</p>
|
||||
</div>
|
||||
|
||||
<p>
|
||||
Different servers may observe the transitions between terms at different times,
|
||||
and in some situations a server may not observe an election or even entire
|
||||
terms. Terms act as a logical clock [14] in Raft, and they allow servers to
|
||||
detect obsolete information such as stale leaders. Each server stores a current
|
||||
term number, which increases monotonically over time. Current terms are
|
||||
exchanged whenever servers communicate; if one server's current term is smaller
|
||||
than the other's, then it updates its current term to the larger value. If a
|
||||
candidate or leader discovers that its term is out of date, it immediately
|
||||
reverts to follower state. If a server receives a request with a stale term
|
||||
number, it rejects the request.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Raft servers communicate using remote procedure calls (RPCs), and the basic
|
||||
consensus algorithm requires only two types of RPCs. RequestVote RPCs are
|
||||
initiated by candidates during elections (Section 5.2), and AppendEntries RPCs
|
||||
are initiated by leaders to replicate log entries and to provide a form of
|
||||
heartbeat (Section 5.3). Section 7 adds a third RPC for transferring snapshots
|
||||
between servers. Servers retry RPCs if they do not receive a response in a
|
||||
timely manner, and they issue RPCs in parallel for best performance.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="outline-container-org291797f" class="outline-3">
|
||||
<h3 id="org291797f"><span class="section-number-3">5.2</span> Leader election</h3>
|
||||
<div class="outline-text-3" id="text-5-2">
|
||||
<p>
|
||||
Raft uses a heartbeat mechanism to trigger leader election. When servers start
|
||||
up, they begin as followers. A server remains in follower state as long as it
|
||||
receives valid RPCs from a leader or candidate. Leaders send periodic heartbeats
|
||||
(AppendEntries RPCs that carry no log entries) to all followers in order to
|
||||
maintain their authority. If a follower receives no communication over a period
|
||||
of time called the election timeout, then it assumes there is no viable leader
|
||||
and begins an election to choose a new leader.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
To begin an election, a follower increments its current term and transitions to
|
||||
candidate state. It then votes for itself and issues RequestVote RPCs in
|
||||
parallel to each of the other servers in the cluster. A candidate continues in
|
||||
this state until one of three things happens: (a) it wins the election, (b)
|
||||
another server establishes itself as leader, or (c) a period of time goes by
|
||||
with no winner. These outcomes are discussed separately in the paragraphs below.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
A candidate wins an election if it receives votes from a majority of the servers
|
||||
in the full cluster for the same term. Each server will vote for at most one
|
||||
candidate in a given term, on a first-come-first-served basis (note: Section 5.4
|
||||
adds an additional restriction on votes). The majority rule ensures that at most
|
||||
one candidate can win the election for a particular term (the Election Safety
|
||||
Property in Figure 3). Once a candidate wins an election, it becomes leader. It
|
||||
then sends heartbeat messages to all of the other servers to establish its
|
||||
authority and prevent new elections.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
While waiting for votes, a candidate may receive an AppendEntries RPC from
|
||||
another server claiming to be leader. If the leader's term (included in its RPC)
|
||||
is at least as large as the candidate's current term, then the candidate
|
||||
recognizes the leader as legitimate and returns to follower state. If the term
|
||||
in the RPC is smaller than the candidate's current term, then the candidate
|
||||
rejects the RPC and continues in candidate state.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
The third possible outcome is that a candidate neither wins nor loses the
|
||||
election: if many followers become candidates at the same time, votes could be
|
||||
split so that no candidate obtains a majority. When this happens, each candidate
|
||||
will time out and start a new election by incrementing its term and initiating
|
||||
another round of RequestVote RPCs. However, without extra measures split votes
|
||||
could repeat indefinitely.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Raft uses randomized election timeouts to ensure that split votes are rare and
|
||||
that they are resolved quickly. To prevent split votes in the first place,
|
||||
election timeouts are chosen randomly from a fixed interval (e.g., 150–300ms).
|
||||
This spreads out the servers so that in most cases only a single server will
|
||||
time out; it wins the election and sends heartbeats before any other servers
|
||||
time out. The same mechanism is used to handle split votes. Each candidate
|
||||
restarts its randomized election timeout at the start of an election, and it
|
||||
waits for that timeout to elapse before starting the next election; this reduces
|
||||
the likelihood of another split vote in the new election. Section 9.3 shows
|
||||
that this approach elects a leader rapidly.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Elections are an example of how understandability guided our choice between
|
||||
design alternatives. Initially we planned to use a ranking system: each
|
||||
candidate was assigned a unique rank, which was used to select between
|
||||
competing candidates. If a candidate discovered another candidate with higher
|
||||
rank, it would return to follower state so that the higher ranking candidate
|
||||
could more easily win the next election. We found that this approach created
|
||||
subtle issues around availability (a lower-ranked server might need to time out
|
||||
and become a candidate again if a higher-ranked server fails, but if it does so
|
||||
too soon, it can reset progress towards electing a leader). We made adjustments
|
||||
to the algorithm several times, but after each adjustment new corner cases
|
||||
appeared. Eventually we concluded that the randomized retry approach is more
|
||||
obvious and understandable.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="outline-container-org44bb0a6" class="outline-3">
|
||||
<h3 id="org44bb0a6"><span class="section-number-3">5.3</span> Log replication</h3>
|
||||
<div class="outline-text-3" id="text-5-3">
|
||||
<p>
|
||||
Once a leader has been elected, it begins servicing client requests. Each client
|
||||
request contains a command to be executed by the replicated state machines. The
|
||||
leader appends the command to its log as a new entry, then issues AppendEntries
|
||||
RPCs in parallel to each of the other servers to replicate the entry. When the
|
||||
entry has been safely replicated (as described below), the leader applies the
|
||||
entry to its state machine and returns the result of that execution to the
|
||||
client. If followers crash or run slowly, or if network packets are lost, the
|
||||
leader retries AppendEntries RPCs indefinitely (even after it has responded to
|
||||
the client) until all followers eventually store all log entries.
|
||||
</p>
|
||||
|
||||
|
||||
<div id="org879421f" class="figure">
|
||||
<p><img src="img/raft-figure6.jpg" alt="raft-figure6.jpg" />
|
||||
</p>
|
||||
<p><span class="figure-number">Figure 6: </span>Logs are composed of entries, which are numbered sequentially. Each entry contains the term in which it was created (the number in each box) and a command for the state machine. An entry is considered committed if it is safe for that entry to be applied to state machines.</p>
|
||||
</div>
|
||||
|
||||
<p>
|
||||
Logs are organized as shown in Figure 6. Each log entry stores a state machine
|
||||
command along with the term number when the entry was received by the leader.
|
||||
The term numbers in log entries are used to detect inconsistencies between logs
|
||||
and to ensure some of the properties in Figure 3. Each log entry also has an
|
||||
integer index identifying its position in the log.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
The leader decides when it is safe to applyy a log en-
|
||||
try to the state machines; such an entry is called commit-
|
||||
ted. Raft guarantees that committed entries are durable
|
||||
and will eventually be executed by all of the available
|
||||
state machines. A log entry is committed once the leader
|
||||
that created the entry has replicated it on a majority of
|
||||
the servers (e.g., entry 7 in Figure 6). This also commits
|
||||
all preceding entries in the leader's log, including entries
|
||||
created by previous leaders. Section 5.4 discusses some
|
||||
subtleties when applying this rule after leader changes,
|
||||
and it also shows that this definition of commitment is
|
||||
safe. The leader keeps track of the highest index it knows
|
||||
to be committed, and it includes that index in future
|
||||
AppendEntries RPCs (including heartbeats) so that the
|
||||
other servers eventually find out. Once a follower learns
|
||||
that a log entry is committed, it applies the entry to its
|
||||
local state machine (in log order).
|
||||
We designed the Raft log mechanism to maintain a high
|
||||
level of coherency between the logs on different servers.
|
||||
Not only does this simplify the system's behavior and
|
||||
make it more predictable, but it is an important component
|
||||
of ensuring safety. Raft maintains the following proper-
|
||||
ties, which together constitute the Log Matching Property
|
||||
in Figure 3:
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="https://utteranc.es/client.js"
|
||||
repo="yygcode/yygcode.github.io"
|
||||
issue-term="pathname"
|
||||
label=" Utterances"
|
||||
theme="github-light"
|
||||
crossorigin="anonymous"
|
||||
async>
|
||||
</script>
|
||||
<noscript>Please enable JavaScript to view the <a href="https://github.com/utterance">comments powered by utterances.</a></noscript>
|
||||
</div>
|
||||
<div id="postamble" class="status">
|
||||
<div class="copyright">
|
||||
2012-2020 Copyright© <i> YANYG<br/>Powered by Emacs Orgmode</i>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
BIN
research/cultnet-distributed-database/raft-extended.pdf
Normal file
BIN
research/cultnet-distributed-database/raft-extended.pdf
Normal file
Binary file not shown.
845
research/cultnet-distributed-database/rethinkdb-changefeeds.html
Normal file
845
research/cultnet-distributed-database/rethinkdb-changefeeds.html
Normal file
|
|
@ -0,0 +1,845 @@
|
|||
<!doctype html>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<html>
|
||||
<head>
|
||||
|
||||
<title>Changefeeds in RethinkDB - RethinkDB</title>
|
||||
|
||||
<meta charset="utf-8">
|
||||
<meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<meta property="twitter:account_id" content="52351557" />
|
||||
|
||||
<link rel="alternate" type="application/rss+xml" title="RethinkDB: the open-source database for the realtime web" href="/feed.xml">
|
||||
<link rel="stylesheet" media="screen" href="/assets/css/styles-02.css"/>
|
||||
<link rel="stylesheet" media="screen" href="/assets/css/jekyll-github.css"/>
|
||||
<link rel="icon" type="image/png" href="/favicon.png">
|
||||
|
||||
<!-- typekit -->
|
||||
<script type="text/javascript" src="//use.typekit.net/lmb6cah.js"></script>
|
||||
<script type="text/javascript">try{Typekit.load();}catch(e){}</script>
|
||||
<!-- jquery -->
|
||||
<script type="text/javascript" src="/assets/js/jquery-2.1.3.min.js"></script>
|
||||
<!-- js-cookie -->
|
||||
<script type="text/javascript" src="/assets/js/js.cookie.js"></script>
|
||||
<!-- waypoints -->
|
||||
<script type="text/javascript" src="/assets/js/waypoints/jquery.waypoints.min.js"></script>
|
||||
<script type="text/javascript" src="/assets/js/waypoints/shortcuts/sticky.min.js"></script>
|
||||
|
||||
<script type="text/javascript" src="/assets/js/site.js"></script>
|
||||
|
||||
|
||||
<!-- algolia -->
|
||||
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/docsearch.js@2/dist/cdn/docsearch.min.css" />
|
||||
|
||||
<script>
|
||||
(function(h,o,t,j,a,r){
|
||||
h.hj=h.hj||function(){(h.hj.q=h.hj.q||[]).push(arguments)};
|
||||
h._hjSettings={hjid:1540062,hjsv:6};
|
||||
a=o.getElementsByTagName('head')[0];
|
||||
r=o.createElement('script');r.async=1;
|
||||
r.src=t+h._hjSettings.hjid+j+h._hjSettings.hjsv;
|
||||
a.appendChild(r);
|
||||
})(window,document,'https://static.hotjar.com/c/hotjar-','.js?sv=');
|
||||
</script>
|
||||
</head>
|
||||
<body class="docs">
|
||||
<div id="wrapper">
|
||||
|
||||
<!-- site nav -->
|
||||
<nav class="mobile-menu push-menu-right">
|
||||
<ul>
|
||||
<li class="menu-trigger"><a href="/">Close</a></li>
|
||||
<li><a href="/faq">FAQ</a></li>
|
||||
<li><a href="/docs">Docs</a></li>
|
||||
<li><a href="/api">API</a></li>
|
||||
<li><a href="/community">Community</a></li>
|
||||
<li><a href="/blog">Blog</a></li>
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
<nav class="site-nav">
|
||||
<div class="site-container">
|
||||
<a class="logo" href="/">RethinkDB</a>
|
||||
<ul>
|
||||
<li class="menu-trigger"><a href="/"><img class="menu-trigger-icon" src="/assets/images/icons/icon-menu-white.png"></a></li>
|
||||
<li><a href="/faq">FAQ</a></li>
|
||||
<li><a href="/docs">Docs</a></li>
|
||||
<li><a href="/api">API</a></li>
|
||||
<li><a href="/community">Community</a></li>
|
||||
<li><a href="/blog">Blog</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<section class="documentation">
|
||||
<div class="site-container">
|
||||
<section class="docs-sidebar-left">
|
||||
|
||||
<div class="search">
|
||||
<input class="small" type="text" placeholder="Search documentation...">
|
||||
</div>
|
||||
|
||||
|
||||
<nav class="docs-nav">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section>
|
||||
<h1><a href="#">Getting started</a></h1>
|
||||
<ul>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/quickstart/">Thirty-second quickstart</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/install/">Install the server</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/install-drivers/">Install client drivers</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/start-a-server/">Start a RethinkDB server</a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section>
|
||||
<h1><a href="#">The ReQL query language</a></h1>
|
||||
<ul class="expanded">
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/guide/">Ten-minute guide</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/introduction-to-reql/">Introduction to ReQL</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/secondary-indexes/">Using secondary indexes</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/nested-fields/">Accessing nested fields</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/table-joins/">Table joins in RethinkDB</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/map-reduce/">Map-reduce in RethinkDB</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="active ">
|
||||
<a href="/docs/changefeeds/">Changefeeds</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/async-connections/">Asynchronous connections</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/external-api-access/">Accessing HTTP APIs</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/error-types/">ReQL error types</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/api/">API command reference</a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section>
|
||||
<h1><a href="#">The RethinkDB data model</a></h1>
|
||||
<ul>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/data-types/">ReQL data types</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/dates-and-times/">Dates and times</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/storing-binary/">Binary objects</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/geo-support/">Geospatial queries</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/data-modeling/">Modeling relationships</a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section>
|
||||
<h1><a href="#">ReQL in practice</a></h1>
|
||||
<ul>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/optimization/">Optimizing query performance</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/reql-data-exploration/">Data exploration</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/sql-to-reql/">SQL to ReQL cheat sheet</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/cookbook/">Cookbook</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/examples/">Example projects</a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section>
|
||||
<h1><a href="#">Administration</a></h1>
|
||||
<ul>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/administration-tools/">Administration tools</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/sharding-and-replication/">Scaling, sharding and replication</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/permissions-and-accounts/">Permissions and user accounts</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/failover/">Failover</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/backup/">Backing up your data</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/system-tables/">System tables</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="nested ">
|
||||
<a href="/docs/system-issues/">Current issues</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="nested ">
|
||||
<a href="/docs/system-stats/">Statistics</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="nested ">
|
||||
<a href="/docs/system-jobs/">Jobs</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/migration/">Version migration</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/importing/">Importing data</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/third-party-admin-tools/">Third-party admin tools</a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section>
|
||||
<h1><a href="#">Deployment</a></h1>
|
||||
<ul>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/start-on-startup/">Start on system startup</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/security/">Securing your cluster</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/config-file/">Configuration file settings</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/cli-options/">Command line options</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/paas/">Deploying with PaaS</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/deployment-tools/">Third-party deployment tools</a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section>
|
||||
<h1><a href="#">Troubleshooting</a></h1>
|
||||
<ul>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/crashes/">Crash recovery</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/troubleshooting/">Troubleshooting FAQ</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/memory-usage/">Memory requirements</a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section>
|
||||
<h1><a href="#">Integration</a></h1>
|
||||
<ul>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/publish-subscribe/">Publish-subscribe</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/elasticsearch/">Integrating Elasticsearch</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/rabbitmq/">Integrating RabbitMQ</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/rails">RethinkDB and Rails</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/frameworks-and-libraries/">Third-party libraries</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/writing-drivers/">Writing drivers</a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section>
|
||||
<h1><a href="#">RethinkDB architecture</a></h1>
|
||||
<ul>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/architecture/">Architecture details</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/consistency/">Consistency guarantees</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/limitations/">Limitations in RethinkDB</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/stability/">Stability report</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/faq/">Frequently asked questions</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/2-1-5-performance-report/">Performance report</a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<section>
|
||||
<h1><a href="#">Security</a></h1>
|
||||
<ul>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/vulnerabilities/">Vulnerabilities</a>
|
||||
</li>
|
||||
|
||||
|
||||
<li class="">
|
||||
<a href="/docs/mobile-connections/">Connections from mobile devices</a>
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
</nav>
|
||||
|
||||
</section>
|
||||
<section class="docs-article">
|
||||
|
||||
<h1 class='title'>Changefeeds in RethinkDB</h1>
|
||||
|
||||
<section class="byline">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="lang-selector">
|
||||
<p class="caption">Language: </p>
|
||||
<section class="languages">
|
||||
<a class="lang javascript"
|
||||
href="
|
||||
/docs/changefeeds/javascript/
|
||||
">
|
||||
<i class="javascript"></i>
|
||||
<span >JavaScript</span>
|
||||
</a><span class="divider">|</span><!--
|
||||
--><a class="lang python"
|
||||
href="
|
||||
/docs/changefeeds/python/
|
||||
">
|
||||
<i class="python"></i>
|
||||
<span >Python</span>
|
||||
</a><span class="divider">|</span><!--
|
||||
--><a class="lang ruby "
|
||||
href="
|
||||
/docs/changefeeds/ruby/
|
||||
">
|
||||
<i class="ruby"></i>
|
||||
<span >Ruby</span>
|
||||
</a><span class="divider">|</span><!--
|
||||
--><a class="lang java active"
|
||||
href="
|
||||
/docs/changefeeds/java/
|
||||
">
|
||||
<i class="java"></i>
|
||||
<span >Java</span>
|
||||
</a>
|
||||
</section>
|
||||
</nav>
|
||||
|
||||
</section>
|
||||
|
||||
<div class="alg-content"><p><strong>Changefeeds</strong> lie at the heart of RethinkDB’s real-time functionality.</p>
|
||||
|
||||
<div class="toc">
|
||||
<h1 class="no_toc" id="in-this-article">In this article</h1>
|
||||
<ul id="markdown-toc">
|
||||
<li><a href="#basic-usage" id="markdown-toc-basic-usage">Basic usage</a></li>
|
||||
<li><a href="#point-single-document-changefeeds" id="markdown-toc-point-single-document-changefeeds">Point (single document) changefeeds</a></li>
|
||||
<li><a href="#changefeeds-with-filtering-and-aggregation-queries" id="markdown-toc-changefeeds-with-filtering-and-aggregation-queries">Changefeeds with filtering and aggregation queries</a></li>
|
||||
<li><a href="#including-state-changes" id="markdown-toc-including-state-changes">Including state changes</a></li>
|
||||
<li><a href="#including-initial-values" id="markdown-toc-including-initial-values">Including initial values</a></li>
|
||||
<li><a href="#including-result-types" id="markdown-toc-including-result-types">Including result types</a></li>
|
||||
<li><a href="#handling-latency" id="markdown-toc-handling-latency">Handling latency</a></li>
|
||||
<li><a href="#scaling-considerations" id="markdown-toc-scaling-considerations">Scaling considerations</a></li>
|
||||
<li><a href="#read-more" id="markdown-toc-read-more">Read more</a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
<p>They allow clients to receive changes on a table, a single document, or even the results from a specific query as they happen. Nearly any ReQL query can be turned into a changefeed.</p>
|
||||
|
||||
<p><img alt="Data Modeling Illustration" class="api_command_illustration" src="/assets/images/docs/api_illustrations/change-feeds.png" /></p>
|
||||
|
||||
<h1 id="basic-usage">Basic usage</h1>
|
||||
|
||||
<p>Subscribe to a feed by calling <a href="/api/java/changes">changes</a> on a table:</p>
|
||||
|
||||
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">Cursor</span> <span class="n">changeCursor</span> <span class="o">=</span> <span class="n">r</span><span class="o">.</span><span class="na">table</span><span class="o">(</span><span class="s">"users"</span><span class="o">).</span><span class="na">changes</span><span class="o">().</span><span class="na">run</span><span class="o">(</span><span class="n">conn</span><span class="o">);</span>
|
||||
<span class="k">for</span> <span class="o">(</span><span class="n">Object</span> <span class="n">change</span> <span class="o">:</span> <span class="n">changeCursor</span><span class="o">)</span> <span class="o">{</span>
|
||||
<span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">change</span><span class="o">);</span>
|
||||
<span class="o">}</span>
|
||||
</code></pre></div></div>
|
||||
|
||||
<p>The <code class="language-plaintext highlighter-rouge">changes</code> command returns a cursor (like the <code class="language-plaintext highlighter-rouge">table</code> or <code class="language-plaintext highlighter-rouge">filter</code> commands do). You can iterate through its contents using ReQL. Unlike other cursors, the output of <code class="language-plaintext highlighter-rouge">changes</code> is infinite: the cursor will block until more elements are available. Every time you make a change to the table or document the <code class="language-plaintext highlighter-rouge">changes</code> feed is monitoring, a new object will be returned to the cursor. For example, if you insert a user <code class="language-plaintext highlighter-rouge">{"id": 1, "name": "Slava", "age": 31}</code> into the <code class="language-plaintext highlighter-rouge">users</code> table, RethinkDB will post this document to changefeeds subscribed to <code class="language-plaintext highlighter-rouge">users</code>:</p>
|
||||
|
||||
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
|
||||
</span><span class="s2">"old_val"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
|
||||
</span><span class="s2">"new_val"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s2">"id"</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="s2">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Slava"</span><span class="p">,</span><span class="w"> </span><span class="s2">"age"</span><span class="p">:</span><span class="w"> </span><span class="mi">31</span><span class="w"> </span><span class="p">}</span><span class="w">
|
||||
</span><span class="p">}</span><span class="w">
|
||||
</span></code></pre></div></div>
|
||||
|
||||
<p>Here <code class="language-plaintext highlighter-rouge">old_val</code> is the old version of the document, and <code class="language-plaintext highlighter-rouge">new_val</code> is a new version of the document. On an <code class="language-plaintext highlighter-rouge">insert</code>, <code class="language-plaintext highlighter-rouge">old_val</code> will be <code class="language-plaintext highlighter-rouge">null</code>; on a <code class="language-plaintext highlighter-rouge">delete</code>, <code class="language-plaintext highlighter-rouge">new_val</code> will be <code class="language-plaintext highlighter-rouge">null</code>. On an <code class="language-plaintext highlighter-rouge">update</code>, both <code class="language-plaintext highlighter-rouge">old_val</code> and <code class="language-plaintext highlighter-rouge">new_val</code> are present.</p>
|
||||
|
||||
<h1 id="point-single-document-changefeeds">Point (single document) changefeeds</h1>
|
||||
|
||||
<p>A “point” changefeed returns changes to a single document within a table rather than the table as a whole.</p>
|
||||
|
||||
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">r</span><span class="o">.</span><span class="na">table</span><span class="o">(</span><span class="s">"users"</span><span class="o">).</span><span class="na">get</span><span class="o">(</span><span class="mi">100</span><span class="o">).</span><span class="na">changes</span><span class="o">().</span><span class="na">run</span><span class="o">(</span><span class="n">conn</span><span class="o">);</span>
|
||||
</code></pre></div></div>
|
||||
|
||||
<p>The output format of a point changefeed is identical to a table changefeed.</p>
|
||||
|
||||
<h1 id="changefeeds-with-filtering-and-aggregation-queries">Changefeeds with filtering and aggregation queries</h1>
|
||||
|
||||
<p>Like any ReQL command, <code class="language-plaintext highlighter-rouge">changes</code> integrates with the rest of the query language. You can call <code class="language-plaintext highlighter-rouge">changes</code> after most commands that transform or select data:</p>
|
||||
|
||||
<ul>
|
||||
<li><a href="/api/java/filter">filter</a></li>
|
||||
<li><a href="/api/java/get_all">getAll</a></li>
|
||||
<li><a href="/api/java/map">map</a></li>
|
||||
<li><a href="/api/java/pluck">pluck</a></li>
|
||||
<li><a href="/api/java/between">between</a></li>
|
||||
<li><a href="/api/java/union">union</a></li>
|
||||
<li><a href="/api/java/min">min</a></li>
|
||||
<li><a href="/api/java/max">max</a></li>
|
||||
<li><a href="/api/java/order_by">orderBy</a>.<a href="/api/java/limit">limit</a></li>
|
||||
</ul>
|
||||
|
||||
<p>You can also chain <code class="language-plaintext highlighter-rouge">changes</code> before any command that operates on a sequence of documents, as long as that command doesn’t consume the entire sequence. (For instance, <code class="language-plaintext highlighter-rouge">count</code> and <code class="language-plaintext highlighter-rouge">orderBy</code> cannot come after the <code class="language-plaintext highlighter-rouge">changes</code> command.)</p>
|
||||
|
||||
<p>Suppose you have a chat application with multiple clients posting messages to different chat rooms. You can create feeds that subscribe to messages posted to a specific room:</p>
|
||||
|
||||
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">r</span><span class="o">.</span><span class="na">table</span><span class="o">(</span><span class="s">"messages"</span><span class="o">).</span><span class="na">filter</span><span class="o">(</span>
|
||||
<span class="n">row</span> <span class="o">-></span> <span class="n">row</span><span class="o">.</span><span class="na">g</span><span class="o">(</span><span class="s">"room_id"</span><span class="o">).</span><span class="na">eq</span><span class="o">(</span><span class="n">ROOM_ID</span><span class="o">)</span>
|
||||
<span class="o">).</span><span class="na">changes</span><span class="o">().</span><span class="na">run</span><span class="o">(</span><span class="n">conn</span><span class="o">);</span>
|
||||
</code></pre></div></div>
|
||||
|
||||
<p>You can also use more complicated expressions. Let’s say you have a table <code class="language-plaintext highlighter-rouge">scores</code> that contains the latest game score for every user of your game. You can create a feed of all games where a user beats their previous score, and get only the new value:</p>
|
||||
|
||||
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">r</span><span class="o">.</span><span class="na">table</span><span class="o">(</span><span class="s">"scores"</span><span class="o">).</span><span class="na">changes</span><span class="o">().</span><span class="na">filter</span><span class="o">(</span>
|
||||
<span class="n">change</span> <span class="o">-></span> <span class="n">change</span><span class="o">.</span><span class="na">g</span><span class="o">(</span><span class="s">"new_val"</span><span class="o">).</span><span class="na">g</span><span class="o">(</span><span class="s">"score"</span><span class="o">).</span><span class="na">gt</span><span class="o">(</span><span class="n">change</span><span class="o">.</span><span class="na">g</span><span class="o">(</span><span class="s">"old_val"</span><span class="o">).</span><span class="na">g</span><span class="o">(</span><span class="s">"score"</span><span class="o">))</span>
|
||||
<span class="o">).</span><span class="na">g</span><span class="o">(</span><span class="s">"new_val"</span><span class="o">).</span><span class="na">run</span><span class="o">(</span><span class="n">conn</span><span class="o">);</span>
|
||||
</code></pre></div></div>
|
||||
|
||||
<p>There are some limitations and caveats on chaining with changefeeds.</p>
|
||||
|
||||
<ul>
|
||||
<li><code class="language-plaintext highlighter-rouge">min</code>, <code class="language-plaintext highlighter-rouge">max</code> and <code class="language-plaintext highlighter-rouge">orderBy</code> must be used with indexes.</li>
|
||||
<li><code class="language-plaintext highlighter-rouge">orderBy</code> requires <code class="language-plaintext highlighter-rouge">limit</code>; neither command works by itself.</li>
|
||||
<li><code class="language-plaintext highlighter-rouge">orderBy</code> must be used with a <a href="/docs/secondary-indexes/java">secondary index</a> or the primary index; it cannot be used with an unindexed field.</li>
|
||||
<li>You cannot use changefeeds after <a href="/api/java/concat_map">concatMap</a> or other transformations whose results cannot be pushed to the shards.</li>
|
||||
<li>You cannot apply a <code class="language-plaintext highlighter-rouge">filter</code> after <code class="language-plaintext highlighter-rouge">orderBy.limit</code> in a changefeed.</li>
|
||||
<li>Transformations are applied before changes are calculated.</li>
|
||||
</ul>
|
||||
|
||||
<h1 id="including-state-changes">Including state changes</h1>
|
||||
|
||||
<p>The <code class="language-plaintext highlighter-rouge">include_states</code> optional argument to <code class="language-plaintext highlighter-rouge">changes</code> allows you to receive extra “status” documents in changefeed streams. These can allow your application to distinguish between initial values returned at the start of a stream and subsequent changes. Read the <a href="/api/java/changes">changes</a> API documentation for a full explanation and example.</p>
|
||||
|
||||
<h1 id="including-initial-values">Including initial values</h1>
|
||||
|
||||
<p>By specifying <code class="language-plaintext highlighter-rouge">true</code> to the <code class="language-plaintext highlighter-rouge">include_initial</code> optional argument, the changefeed stream will start with the current contents of the table or selection being monitored. The initial results will have <code class="language-plaintext highlighter-rouge">new_val</code> fields, but no <code class="language-plaintext highlighter-rouge">old_val</code> fields, so it’s easy to distinguish them from change events.</p>
|
||||
|
||||
<p>If an initial result for a document has been sent and a change is made to that document that would move it to the unsent part of the result set (for instance, a changefeed monitors the top 100 posters, the first 50 have been sent, and poster 48 has become poster 52), an “uninitial” notification will be sent, with an <code class="language-plaintext highlighter-rouge">old_val</code> field but no <code class="language-plaintext highlighter-rouge">new_val</code> field. This is distinct from a delete change event, which would have a <code class="language-plaintext highlighter-rouge">new_val</code> of <code class="language-plaintext highlighter-rouge">null</code>. (In the top 100 posters example, that could indicate the poster has been deleted, or has dropped out of the top 100.)</p>
|
||||
|
||||
<p>If you specify <code class="language-plaintext highlighter-rouge">true</code> for both <code class="language-plaintext highlighter-rouge">include_states</code> and <code class="language-plaintext highlighter-rouge">include_initial</code>, the changefeed stream will start with a <code class="language-plaintext highlighter-rouge">{"state": "initializing"}</code> status document, followed by initial values. A <code class="language-plaintext highlighter-rouge">{"state": "ready"}</code> status document will be sent when all the initial values have been sent.</p>
|
||||
|
||||
<h1 id="including-result-types">Including result types</h1>
|
||||
|
||||
<p>The <code class="language-plaintext highlighter-rouge">include_types</code> optional argument adds a third field, <code class="language-plaintext highlighter-rouge">type</code>, to each result sent. The string values for <code class="language-plaintext highlighter-rouge">type</code> are largely self-explanatory:</p>
|
||||
|
||||
<ul>
|
||||
<li><code class="language-plaintext highlighter-rouge">add</code>: a new value added to the result set.</li>
|
||||
<li><code class="language-plaintext highlighter-rouge">remove</code>: an old value removed from the result set.</li>
|
||||
<li><code class="language-plaintext highlighter-rouge">change</code>: an existing value changed in the result set.</li>
|
||||
<li><code class="language-plaintext highlighter-rouge">initial</code>: an initial value notification.</li>
|
||||
<li><code class="language-plaintext highlighter-rouge">uninitial</code>: an uninitial value notification.</li>
|
||||
<li><code class="language-plaintext highlighter-rouge">state</code>: a status document from <code class="language-plaintext highlighter-rouge">include_states</code>.</li>
|
||||
</ul>
|
||||
|
||||
<p>Including the <code class="language-plaintext highlighter-rouge">type</code> field can simplify code that handles different cases for changefeed results.</p>
|
||||
|
||||
<h1 id="handling-latency">Handling latency</h1>
|
||||
|
||||
<p>Depending on how fast your application makes changes to monitored data and how fast it processes change notifications, it’s possible that more than one change will happen between calls to the <code class="language-plaintext highlighter-rouge">changes</code> command. You can control what happens in that case with the <code class="language-plaintext highlighter-rouge">squash</code> optional argument.</p>
|
||||
|
||||
<p>By default, if more than one change occurs between invocations of <code class="language-plaintext highlighter-rouge">changes</code>, your application will receive a single change object whose <code class="language-plaintext highlighter-rouge">new_val</code> will incorporate <em>all</em> the changes to the data. Suppose three updates occurred to a monitored document between <code class="language-plaintext highlighter-rouge">change</code> reads:</p>
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Change</th>
|
||||
<th>Data</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Initial state (<code class="language-plaintext highlighter-rouge">old_val</code>)</td>
|
||||
<td>{ name: “Fred”, admin: true }</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>update({name: “George”})</td>
|
||||
<td>{ name: “George”, admin: true }</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>update({admin: false})</td>
|
||||
<td>{ name: “George”, admin: false }</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>update({name: “Jay”})</td>
|
||||
<td>{ name: “Jay”, admin: false }</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code class="language-plaintext highlighter-rouge">new_val</code></td>
|
||||
<td>{ name: “Jay”, admin: false }</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p>Your application would by default receive the object as it existed in the database after the <em>most recent</em> change. The previous two updates would be “squashed” into the third.</p>
|
||||
|
||||
<p>If you wanted to receive <em>all</em> the changes, including the interim states, you could do so by passing <code class="language-plaintext highlighter-rouge">squash: false</code>. The server will buffer up to 100,000 changes. (This number can be changed with the <code class="language-plaintext highlighter-rouge">changefeed_queue_size</code> optional argument.)</p>
|
||||
|
||||
<p>A third option is to specify how many seconds to wait between squashes. Passing <code class="language-plaintext highlighter-rouge">squash: 5</code> to the <code class="language-plaintext highlighter-rouge">changes</code> command tells RethinkDB to squash changes together every five seconds. Depending on your application’s use case, this might reduce the load on the server. A number passed to <code class="language-plaintext highlighter-rouge">squash</code> may be a float. Note that the requested interval is not guaranteed, but is rather a best effort.</p>
|
||||
|
||||
<p><strong>Note:</strong> Changefeeds ignore the <code class="language-plaintext highlighter-rouge">read_mode</code> flag to <code class="language-plaintext highlighter-rouge">run</code>, and always behave as if it is set to <code class="language-plaintext highlighter-rouge">single</code> (i.e., the values they return are in memory on the primary replica, but have not necessarily been written to disk yet). For more details read <a href="/docs/consistency">Consistency guarantees</a>.</p>
|
||||
|
||||
<h1 id="scaling-considerations">Scaling considerations</h1>
|
||||
|
||||
<p>Changefeeds perform well as they scale, although they create extra intracluster messages in proportion to the number of servers with open feed connections on each write. This can be mitigated by running a RethinkDB proxy server (the <code class="language-plaintext highlighter-rouge">rethinkdb proxy</code> startup option); read <a href="/docs/sharding-and-replication/#running-a-proxy-node">Running a proxy node</a> for details.</p>
|
||||
|
||||
<p>Since changefeeds are unidirectional with no acknowledgement returned from clients, they cannot guarantee delivery. If you need real-time updating with delivery guarantees, consider using a model that distributes to the clients through a message broker such as <a href="https://www.rabbitmq.com/">RabbitMQ</a>.</p>
|
||||
|
||||
<h1 id="read-more">Read more</h1>
|
||||
|
||||
<ul>
|
||||
<li>The <a href="/api/java/changes">changes</a> command API reference</li>
|
||||
<li><a href="/docs/introduction-to-reql/">Introduction to ReQL</a></li>
|
||||
<li><a href="/docs/data-types/">ReQL data types</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
|
||||
<!-- site footer -->
|
||||
<footer class="site-footer">
|
||||
<div class="site-container">
|
||||
<section class="get-started">
|
||||
<h1>Join 100,000 developers</h1>
|
||||
<p>Install in seconds on Linux or OS X.</p>
|
||||
<a class="button small install" href="/install">Install</a>
|
||||
<a class="button small outlined learn-more" href="/faq">Learn more</a>
|
||||
|
||||
<hr>
|
||||
<section class="contact-info">
|
||||
<p class="irc">#rethinkdb on freenode</p>
|
||||
</section>
|
||||
</section>
|
||||
<ul class="footer-links">
|
||||
<li>
|
||||
<header>
|
||||
<span>Getting started</span>
|
||||
</header>
|
||||
<ul>
|
||||
<li><a href="/faq">FAQ</a></li>
|
||||
<li><a href="/install">Install</a></li>
|
||||
<li><a href="/docs/quickstart">Quickstart</a></li>
|
||||
<li><a href="/docs/guide">Tutorial</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<header>
|
||||
<span>Documentation</span>
|
||||
</header>
|
||||
<ul>
|
||||
<li><a href="/api">API</a></li>
|
||||
<li><a href="/docs/introduction-to-reql/">Intro to ReQL</a></li>
|
||||
<li><a href="/docs/cookbook">Cookbook</a></li>
|
||||
<li><a href="/docs/architecture">Architecture</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<header>
|
||||
<span>Community</span>
|
||||
</header>
|
||||
<ul>
|
||||
<li><a href="/community">Get Involved</a></li>
|
||||
<li><a href="https://github.com/rethinkdb/rethinkdb">GitHub</a></li>
|
||||
<li><a href="https://twitter.com/rethinkdb">Twitter</a></li>
|
||||
<li><a href="https://join.slack.com/t/rethinkdb/shared_invite/enQtNzAxOTUzNTk1NzMzLWY5ZTA0OTNmMWJiOWFmOGVhNTUxZjQzODQyZjIzNjgzZjdjZDFjNDg1NDY3MjFhYmNhOTY1MDVkNDgzMWZiZWM">Slack</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<header>
|
||||
<span>RethinkDB</span>
|
||||
</header>
|
||||
<ul>
|
||||
<li><a href="/blog/">Blog</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<p class="github-footer"><i class="icon-github-footer"></i><a href="https://github.com/rethinkdb/rethinkdb">Star this project</a> on GitHub.</p>
|
||||
</div>
|
||||
</footer>
|
||||
</div>
|
||||
|
||||
<!-- algolia -->
|
||||
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/docsearch.js@2/dist/cdn/docsearch.min.css" />
|
||||
<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/docsearch.js@2/dist/cdn/docsearch.min.js"></script>
|
||||
<script type="text/javascript"> docsearch({
|
||||
apiKey: 'a33e586ddcc6f914b6c3eb48760d6235',
|
||||
indexName: 'rethinkdb',
|
||||
inputSelector: '#wrapper > section > div > section.docs-sidebar-left > div > input',
|
||||
debug: false
|
||||
});
|
||||
</script>
|
||||
|
||||
<script type="text/javascript">
|
||||
|
||||
var _gaq = _gaq || [];
|
||||
_gaq.push(['_setAccount', 'UA-9922584-1']);
|
||||
_gaq.push(['_trackPageview']);
|
||||
|
||||
(function() {
|
||||
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
|
||||
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
|
||||
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
166
research/cultnet-distributed-database/summary.md
Normal file
166
research/cultnet-distributed-database/summary.md
Normal file
|
|
@ -0,0 +1,166 @@
|
|||
# CultNet Distributed Database Research Summary
|
||||
|
||||
Purpose: ground CultNet's distributed realtime database design in prior art
|
||||
before adding more machinery. The goal is a coherent local-first mesh database
|
||||
over CultCache, not a hand-rolled distributed-systems costume party.
|
||||
|
||||
## Sources Stored Here
|
||||
|
||||
- `hashgraph-swirlds-tr-2016-01.pdf`
|
||||
- Source: <https://www.swirlds.com/downloads/SWIRLDS-TR-2016-01.pdf>
|
||||
- Topic: gossip-about-gossip, virtual voting, fair ordering, asynchronous BFT.
|
||||
- `raft-extended.html`
|
||||
- Source: <https://yygcode.com/papers/consensus-raft-extended-version.html>
|
||||
- Topic: understandable consensus, leader election, log replication, safety.
|
||||
- `raft-extended.pdf`
|
||||
- Source: <https://web.stanford.edu/~ouster/cgi-bin/papers/raft-extended.pdf>
|
||||
- Topic: canonical Raft paper copy.
|
||||
- `dynamo-amazon-science.html`
|
||||
- Source: <https://www.amazon.science/publications/dynamo-amazons-highly-available-key-value-store>
|
||||
- Topic: Dynamo publication page and abstract.
|
||||
- `dynamo-sosp2007.pdf`
|
||||
- Source: <https://web.stanford.edu/class/cs244/papers/amazon-dynamo-sosp2007.pdf>
|
||||
- Topic: highly available key-value store, consistent hashing, versioning,
|
||||
quorums, hinted handoff, anti-entropy.
|
||||
- `swim.pdf`
|
||||
- Source: <https://www.cs.cornell.edu/projects/quicksilver/public_pdfs/SWIM.pdf>
|
||||
- Topic: scalable weakly consistent process-group membership.
|
||||
- `rethinkdb-changefeeds.html`
|
||||
- Source: <https://rethinkdb.com/docs/changefeeds/java/>
|
||||
- Topic: realtime query/changefeed ergonomics.
|
||||
- `firebase-realtime-offline.html`
|
||||
- Source: <https://firebase.google.com/docs/database/web/offline-capabilities>
|
||||
- Topic: offline behavior, presence, server-side disconnect operations.
|
||||
- `crdt-arxiv-1805.06358.html`
|
||||
- Source: <https://arxiv.org/abs/1805.06358>
|
||||
- Topic: conflict-free replicated data types and deterministic convergence.
|
||||
|
||||
## Design Takeaways
|
||||
|
||||
### RethinkDB
|
||||
|
||||
Keep: changefeeds as the product feel. Subscribers should receive document
|
||||
changes continuously, with enough old/new context to render, reconcile, or
|
||||
debug. Point subscriptions and filtered subscriptions are both first-class.
|
||||
|
||||
CultNet implication: database subscriptions should be explicit schema-v0
|
||||
messages. The server should stream raw document changes that clients can apply
|
||||
through the same CultCache reconciliation path as snapshots.
|
||||
|
||||
### Firebase Realtime Database
|
||||
|
||||
Keep: realtime sync and local/offline ergonomics. Presence and disconnect
|
||||
behavior are database features, not application afterthoughts.
|
||||
|
||||
Defer: full offline writes. CultNet should not pretend arbitrary offline
|
||||
multi-writer changes merge safely. Offline/local-first behavior needs declared
|
||||
per-document conflict policy.
|
||||
|
||||
CultNet implication: add presence/disconnect records later as ordinary
|
||||
CultCache documents with server authority, not as hidden transport state.
|
||||
|
||||
### Raft
|
||||
|
||||
Keep: the understandable authority model. One leader/primary owns ordered
|
||||
writes for a shard. Decompose the problem into ownership, log/mutation
|
||||
replication, and safety.
|
||||
|
||||
Defer: full automatic leader election and replicated logs until shard catalogs
|
||||
and explicit epochs exist.
|
||||
|
||||
CultNet implication: the current primary-shard policy is the correct first
|
||||
foundation. Every write should either hit the primary, be forwarded to the
|
||||
primary, or fail with routing information.
|
||||
|
||||
### Dynamo
|
||||
|
||||
Keep: partitioning, replication metadata, vector-ish causality, hinted recovery,
|
||||
anti-entropy, and application-visible conflicts.
|
||||
|
||||
Reject for now: "always writeable" semantics. That is attractive, expensive,
|
||||
and easy to lie about.
|
||||
|
||||
CultNet implication: shard descriptors should grow into a shard catalog with
|
||||
owner runtime id, epoch, schema/key ranges, and later replica/preference-list
|
||||
metadata. Conflicts must surface as data, not disappear into last-writer-wins
|
||||
unless a document explicitly chose that policy.
|
||||
|
||||
### SWIM
|
||||
|
||||
Keep: gossip-shaped membership once the mesh grows beyond a small static
|
||||
cluster. SWIM separates failure detection from membership dissemination and
|
||||
keeps per-node message load stable.
|
||||
|
||||
Defer: membership implementation until there is a shard catalog to disseminate.
|
||||
|
||||
CultNet implication: do not build peer discovery as a side-channel registry.
|
||||
When it arrives, it should update membership and shard-catalog state together.
|
||||
|
||||
### CRDTs
|
||||
|
||||
Keep: CRDTs for documents whose merge law is explicit and deterministic.
|
||||
|
||||
Reject: generic automatic merge for arbitrary domain objects. That is a
|
||||
language cop with a nicer hat.
|
||||
|
||||
CultNet implication: CRDT support belongs in schema metadata or document
|
||||
contract metadata. A document type can opt into a known merge strategy; the
|
||||
default distributed write policy remains primary authority.
|
||||
|
||||
### Hashgraph
|
||||
|
||||
Keep: gossip-about-gossip as an idea for compact event provenance and possible
|
||||
fair ordering research. Virtual voting is interesting when every member sees
|
||||
the same gossip history.
|
||||
|
||||
Reject for now: adopting hashgraph consensus as CultNet's core. The public
|
||||
ledger/crypto smell is not the main issue; the issue is that CultNet does not
|
||||
currently need asynchronous Byzantine total ordering to become a good mesh
|
||||
database. Dragging that in now would make the machine harder to explain.
|
||||
|
||||
CultNet implication: if we later need decentralized event ordering, use a
|
||||
dedicated design pass. Do not smuggle hashgraph metadata into ordinary document
|
||||
replication because it sounds powerful.
|
||||
|
||||
## Current CultNet Direction
|
||||
|
||||
The coherent path is:
|
||||
|
||||
1. Primary-shard authority.
|
||||
2. Explicit schema-v0 snapshot, put, delete, subscribe, unsubscribe, and change
|
||||
messages.
|
||||
3. Shard catalog exchange with owner runtime id and epoch.
|
||||
4. Optional forwarding from non-owner nodes to owners.
|
||||
5. Membership/failure detection, likely SWIM-shaped.
|
||||
6. Replication and failover, likely Raft-shaped per shard.
|
||||
7. Optional CRDT policies for document types that deserve offline multi-writer
|
||||
semantics.
|
||||
8. Optional gossip-history research if fair decentralized ordering becomes a
|
||||
real requirement.
|
||||
|
||||
## Live Invariants
|
||||
|
||||
- CultCache owns document identity, schema compatibility, local indexes, and
|
||||
reconciliation.
|
||||
- CultNet owns transport, shard authority, subscriptions, and remote mutation
|
||||
delivery.
|
||||
- Raw wire records must pass through `CultNetDatabase` before mutating local
|
||||
cache state.
|
||||
- A write without authority is rejected or explicitly forwarded. It is never
|
||||
silently applied.
|
||||
- Realtime change streams publish domain changes, not storage implementation
|
||||
details.
|
||||
- Conflict policy must be declared before conflicting writes are accepted.
|
||||
|
||||
## Next Cut
|
||||
|
||||
Build shard catalog exchange:
|
||||
|
||||
- `cultnet.shard_catalog_request.v0`
|
||||
- `cultnet.shard_catalog_response.v0`
|
||||
- per-shard id, owner runtime id, epoch, schema ids, key prefix/range
|
||||
- stale epoch rejection on writes
|
||||
- routing error that tells clients where the primary lives when known
|
||||
|
||||
That is the next foundation needed before forwarding, membership, Raft-style
|
||||
replication, or CRDT policy work can be honest.
|
||||
BIN
research/cultnet-distributed-database/swim.pdf
Normal file
BIN
research/cultnet-distributed-database/swim.pdf
Normal file
Binary file not shown.
Loading…
Reference in a new issue