RCDS_GO/docs/index.html at master · String-Reconciliation-Ditributed-System/RCDS_GO · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>RCDS - Recursive Content-Dependent Shingling</title>
    <meta
      name="description"
      content="RCDS_GO is a Go implementation of Recursive Content-Dependent Shingling for efficient set and file reconciliation across distributed systems."
    >
    <meta property="og:title" content="RCDS - Recursive Content-Dependent Shingling">
    <meta
      property="og:description"
      content="A Go toolkit and CLI for difference-aware set and file reconciliation."
    >
    <meta property="og:url" content="https://string-reconciliation-ditributed-system.github.io/RCDS_GO/">
    <meta property="og:type" content="website">
    <link rel="canonical" href="https://string-reconciliation-ditributed-system.github.io/RCDS_GO/">
    <link rel="icon" href="./favicon.svg" type="image/svg+xml">
    <link rel="stylesheet" href="./assets/site.css">
  </head>
  <body>
    <a class="skip-link" href="#content">Skip to content</a>

    <header class="site-header" aria-label="Primary navigation">
      <a class="brand" href="./index.html" aria-label="RCDS home">
        <span class="brand-mark" aria-hidden="true">R</span>
        <span>RCDS</span>
      </a>
      <nav class="nav-links" aria-label="Documentation">
        <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO/blob/master/docs/CLI.md">CLI</a>
        <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO/blob/master/docs/PAPER.md">Paper</a>
        <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO/blob/master/docs/ARCHITECTURE.md">Architecture</a>
        <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO/blob/master/docs/DEPLOYMENT.md">Deploy</a>
        <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO">GitHub</a>
      </nav>
    </header>

    <main id="content">
      <section class="hero" aria-labelledby="hero-title">
        <div class="sync-scene" aria-hidden="true">
          <div class="node node-a">
            <span>node A</span>
            <i></i><i></i><i></i><i></i>
          </div>
          <div class="node node-b">
            <span>node B</span>
            <i></i><i></i><i></i><i></i>
          </div>
          <div class="packet packet-one"></div>
          <div class="packet packet-two"></div>
          <div class="packet packet-three"></div>
          <div class="shingle-track">
            <b></b><b></b><b></b><b></b><b></b><b></b>
          </div>
        </div>
        <div class="hero-content">
          <p class="eyebrow">Go toolkit and CLI for reconciliation</p>
          <h1 id="hero-title">RCDS</h1>
          <p class="hero-copy">
            Based on the Allerton 2019 RCDS paper, this Go project explores content-dependent
            shingling, set reconciliation, and exact file sync workflows for distributed data.
          </p>
          <div class="hero-actions" aria-label="Primary actions">
            <a class="button primary" href="#quickstart">Quick Start</a>
            <a class="button secondary" href="https://ieeexplore.ieee.org/document/8919901">Read Paper</a>
          </div>
        </div>
      </section>

      <section class="quick-strip" aria-label="Project status">
        <div>
          <strong>3 algorithms</strong>
          <span>RCDS, IBLT, and full sync</span>
        </div>
        <div>
          <strong>2 modes</strong>
          <span>Set reconciliation and file pull</span>
        </div>
        <div>
          <strong>Verified</strong>
          <span>Unit, integration, e2e, and vet</span>
        </div>
      </section>

      <section id="quickstart" class="section two-column" aria-labelledby="quickstart-title">
        <div class="section-copy">
          <p class="eyebrow">Quick start</p>
          <h2 id="quickstart-title">Run a real sync locally.</h2>
          <p>
            Start a one-shot server with a local set, then connect a client with a different set.
            Both sides finish with the same reconciled values and byte counters.
          </p>
        </div>
        <div class="terminal" role="img" aria-label="Terminal commands for building and running RCDS">
          <div class="terminal-bar"><span></span><span></span><span></span></div>
          <pre><code>make build
./bin/rcds server --algorithm full \
  --items server-only,shared \
  --output server.out

./bin/rcds client --algorithm full \
  --items client-only,shared \
  --output client.out</code></pre>
        </div>
      </section>

      <section class="section" aria-labelledby="capabilities-title">
        <div class="section-heading">
          <p class="eyebrow">Capabilities</p>
          <h2 id="capabilities-title">Built for research code that needs to behave like software.</h2>
        </div>
        <div class="feature-grid">
          <article class="feature">
            <h3>Set reconciliation</h3>
            <p>Use `rcds`, `iblt`, or `full` from the same CLI and GenSync-compatible API.</p>
          </article>
          <article class="feature">
            <h3>Chunked file pull</h3>
            <p>Reuses chunks the client already has and verifies the final SHA-256 checksum.</p>
          </article>
          <article class="feature">
            <h3>TCP transport</h3>
            <p>Length-prefixed payloads, bounded reads, full writes, and one-shot server workflows.</p>
          </article>
          <article class="feature">
            <h3>Tested paths</h3>
            <p>Unit, integration, and e2e tests exercise the CLI, algorithms, and file workflow.</p>
          </article>
        </div>
      </section>

      <section class="section protocol" aria-labelledby="protocol-title">
        <div>
          <p class="eyebrow">How it flows</p>
          <h2 id="protocol-title">Metadata first. Payloads only when needed.</h2>
          <p>
            RCDS_GO separates the reconciliation decision from payload movement. The set algorithms
            discover differences; the file path exchanges manifests and missing chunks before writing
            the verified result.
          </p>
        </div>
        <div class="protocol-visual" role="img" aria-label="Protocol flow from local data to digest, difference, transfer, and verified result">
          <div>local data</div>
          <div>digest</div>
          <div>diff</div>
          <div>transfer</div>
          <div>verify</div>
        </div>
      </section>

      <section class="section docs-list" aria-labelledby="docs-title">
        <div class="section-heading">
          <p class="eyebrow">Documentation</p>
          <h2 id="docs-title">Pick the entry point that matches your job.</h2>
        </div>
        <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO/blob/master/docs/CLI.md">
          <span>CLI Reference</span>
          <small>Commands, flags, set examples, and file examples.</small>
        </a>
        <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO/blob/master/docs/PAPER.md">
          <span>Paper Background</span>
          <small>How the Allerton 2019 RCDS paper maps to this Go implementation.</small>
        </a>
        <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO/blob/master/docs/ARCHITECTURE.md">
          <span>Architecture</span>
          <small>Package layout, algorithm adapters, protocol flow, and design notes.</small>
        </a>
        <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO/blob/master/docs/DEPLOYMENT.md">
          <span>Deployment</span>
          <small>Local binaries, containers, Kubernetes manifests, and production cautions.</small>
        </a>
      </section>

      <section class="section research" aria-labelledby="research-title">
        <p class="eyebrow">Research lineage</p>
        <h2 id="research-title">Based on the RCDS paper from Allerton 2019.</h2>
        <p>
          Song and Trachtenberg's paper reduces string reconciliation to set reconciliation by
          recursively partitioning strings into content-dependent shingles. This repository makes
          that lineage explicit while separating research-grade RCDS metadata from the production
          readiness of the current CLI and transport.
        </p>
      </section>
    </main>

    <footer class="site-footer">
      <span>RCDS_GO</span>
      <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO">GitHub</a>
      <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO/blob/master/README.md">README</a>
      <a href="https://github.com/String-Reconciliation-Ditributed-System/RCDS_GO/blob/master/LICENSE">GPL-3.0</a>
    </footer>
  </body>
</html>