MINOR: Extends RocksDB docs (apache#10046)

We recommend users to switch to jemalloc for RocksDB.

Co-authored-by: Jim Galasyn <[email protected]>

Reviewers: Jim Galasyn <[email protected]>, Rohan Desai <[email protected]>, A. Sophie Blee-Goldman <[email protected]>

docs/streams/developer-guide/memory-mgmt.html

@@ -168,7 +168,15 @@
       <h2><a class="toc-backref" href="#id3">RocksDB</a><a class="headerlink" href="#rocksdb" title="Permalink to this headline"></a></h2>
       <p> Each instance of RocksDB allocates off-heap memory for a block cache, index and filter blocks, and memtable (write buffer). Critical configs (for RocksDB version 4.1.0) include
         <code class="docutils literal"><span class="pre">block_cache_size</span></code>, <code class="docutils literal"><span class="pre">write_buffer_size</span></code> and <code class="docutils literal"><span class="pre">max_write_buffer_number</span></code>.  These can be specified through the
-        <code class="docutils literal"><span class="pre">rocksdb.config.setter</span></code> configuration.</li>
+        <code class="docutils literal"><span class="pre">rocksdb.config.setter</span></code> configuration.</p>
+      <p>Also, we recommend changing RocksDB's default memory allocator, because the default allocator may lead to increased memory consumption.
+        To change the memory allocator to <code>jemalloc</code>, you need to set the environment variable <code>LD_PRELOAD</code>before you start your Kafka Streams application:</p>
+      <pre>
+# example: install jemalloc (on Debian)
+$ apt install -y libjemalloc-dev
+# set LD_PRELOAD before you start your Kafka Streams application
+$ export LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libjemalloc.so”
+      </pre>
       <p> As of 2.3.0 the memory usage across all instances can be bounded, limiting the total off-heap memory of your Kafka Streams application. To do so you must configure RocksDB to cache the index and filter blocks in the block cache, limit the memtable memory through a shared <a class="reference external" href="https://github.com/facebook/rocksdb/wiki/Write-Buffer-Manager">WriteBufferManager</a> and count its memory against the block cache, and then pass the same Cache object to each instance. See <a class="reference external" href="https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB">RocksDB Memory Usage</a> for details. An example RocksDBConfigSetter implementing this is shown below:</p>
 
       <div class="highlight-java"><div class="highlight"><pre><span></span>    <span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">BoundedMemoryRocksDBConfig</span> <span class="kd">implements</span> <span class="n">RocksDBConfigSetter</span> <span class="o">{</span>
@@ -201,7 +209,7 @@ <h2><a class="toc-backref" href="#id3">RocksDB</a><a class="headerlink" href="#r
          <span class="c1">// Cache and WriteBufferManager should not be closed here, as the same objects are shared by every store instance.</span>
        <span class="o">}</span>
     <span class="o">}</span>
-      </div>
+      <div>
         <sup id="fn1">1. INDEX_FILTER_BLOCK_RATIO can be used to set a fraction of the block cache to set aside for "high priority" (aka index and filter) blocks, preventing them from being evicted by data blocks. See the full signature of the <a class="reference external" href="https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/LRUCache.java#L72">LRUCache constructor</a>.
           NOTE: the boolean parameter in the cache constructor lets you control whether the cache should enforce a strict memory limit by failing the read or iteration in the rare cases where it might go larger than its capacity. Due to a
           <a class="reference external" href="https://github.com/facebook/rocksdb/issues/6247">bug in RocksDB</a>, this option cannot be used
@@ -215,8 +223,8 @@ <h2><a class="toc-backref" href="#id3">RocksDB</a><a class="headerlink" href="#r
             <dt>Note:</dt>
             While we recommend setting at least the above configs, the specific options that yield the best performance are workload dependent and you should consider experimenting with these to determine the best choices for your specific use case. Keep in mind that the optimal configs for one app may not apply to one with a different topology or input topic.
             In addition to the recommended configs above, you may want to consider using partitioned index filters as described by the <a class="reference external" href="https://github.com/facebook/rocksdb/wiki/Partitioned-Index-Filters">RocksDB docs</a>.
-
           </dl>
+        </sup>
       </div>
       <div class="section" id="other-memory-usage">
       <h2><a class="toc-backref" href="#id4">Other memory usage</a><a class="headerlink" href="#other-memory-usage" title="Permalink to this headline"></a></h2>
@@ -231,16 +239,13 @@ <h2><a class="toc-backref" href="#id4">Other memory usage</a><a class="headerlin
           timestamp and buffered in the streams space. Currently this is only indirectly controlled by
           <code class="docutils literal"><span class="pre">buffered.records.per.partition</span></code>.</li>
       </ul>
+      </div>
       <div class="admonition tip">
         <p><b>Tip</b></p>
         <p><strong>Iterators should be closed explicitly to release resources:</strong> Store iterators (e.g., <code class="docutils literal"><span class="pre">KeyValueIterator</span></code> and <code class="docutils literal"><span class="pre">WindowStoreIterator</span></code>) must be closed explicitly upon completeness to release resources such as open file handlers and in-memory read buffers, or use try-with-resources statement (available since JDK7) for this Closeable class.</p>
         <p class="last">Otherwise, stream application&#8217;s memory usage keeps increasing when running until it hits an OOM.</p>
-</div>
-</div>
-
+      </div>
 
-               </div>
-              </div>
   <div class="pagination">
     <a href="/{{version}}/documentation/streams/developer-guide/interactive-queries" class="pagination__btn pagination__btn__prev">Previous</a>
     <a href="/{{version}}/documentation/streams/developer-guide/running-app" class="pagination__btn pagination__btn__next">Next</a>