<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title><![CDATA[Paconte's blog]]></title>
<description><![CDATA[Paconte's blog]]></description>
<link>https://paconte.com/</link>
<lastBuildDate>Wed, 03 Apr 2024 07:39:44 +0200</lastBuildDate>
<item>
  <title><![CDATA[Differences of the builder pattern in Java and Python]]></title>
  <description><![CDATA[
<nav id="table-of-contents" role="doc-toc">
<h2>Table of Contents</h2>
<div id="text-table-of-contents" role="doc-toc">
<ul>
<li><a href="#orgb37e09c">1. Introduction</a></li>
<li><a href="#orgd81a2f4">2. Language features in Java and Python</a></li>
<li><a href="#org034fc7f">3. The builder pattern in Java </a></li>
<li><a href="#org2a12896">4. The builder pattern in Python</a></li>
<li><a href="#orgaab50aa">5. Summary</a></li>
</ul>
</div>
</nav>
<p>
In this post, we explore the differences in implementing the builder
pattern in Java and Python.
</p>

<div id="outline-container-orgb37e09c" class="outline-2">
<h2 id="orgb37e09c"><span class="section-number-2">1.</span> Introduction</h2>
<div class="outline-text-2" id="text-1">
<p>
The builder design pattern <sup><a id="fnr.1" class="footref" href="#fn.1" role="doc-backlink">1</a></sup>, first introduced in the "Design
Patterns: Elements of Reusable Object-Oriented Software" (1994) <sup><a id="fnr.2" class="footref" href="#fn.2" role="doc-backlink">2</a></sup>,
popularly known as Gang of Four (GoF), is classified as a creational
design pattern. This post focuses on its application in Java and Python.
</p>

<p>
At my current job we are using different programming languages. We
decided to use the buildern pattern for one of our python microservices.
While writing the buildern pattern in python, the team brought their
"java style" to the python code. However, there is a pythonic way
to write it and save some lines of code. This is the story behind
this post.
</p>
</div>
</div>

<div id="outline-container-orgd81a2f4" class="outline-2">
<h2 id="orgd81a2f4"><span class="section-number-2">2.</span> Language features in Java and Python</h2>
<div class="outline-text-2" id="text-2">
<p>
<a id="org2897cb6"></a>
Let's first examine the features of each language that we will later
use to implement the design patterns in Python and Java. Python
supports function default arguments <sup><a id="fnr.3" class="footref" href="#fn.3" role="doc-backlink">3</a></sup> and named parameters <sup><a id="fnr.4" class="footref" href="#fn.4" role="doc-backlink">4</a></sup>,
while Java supports function overloading <sup><a id="fnr.5" class="footref" href="#fn.5" role="doc-backlink">5</a></sup>.
</p>

<p>
The following examples demonstrates default arguments:
</p>

<div class="org-src-container">
<pre class="src src-python"><span style="color: #b22222;"># </span><span style="color: #b22222;">Function with default arguments</span>
<span style="color: #a020f0;">def</span> <span style="color: #0000ff;">calculate_area</span>(length, width=5):
    <span style="color: #a0522d;">area</span> = length * width
    <span style="color: #483d8b;">print</span>(f<span style="color: #8b2252;">"Area: </span>{area}<span style="color: #8b2252;">"</span>)

<span style="color: #b22222;"># </span><span style="color: #b22222;">Using the function with both parameters</span>
calculate_area(8, 4)

<span style="color: #b22222;"># </span><span style="color: #b22222;">Using the function with the default value for 'width'</span>
calculate_area(10)  <span style="color: #b22222;"># </span><span style="color: #b22222;">Width defaults to 5</span>

<span style="color: #b22222;"># </span><span style="color: #b22222;">You can still explicitly provide a value for 'width'</span>
calculate_area(6, 3)
</pre>
</div>

<p>
The following Python example demonstrates both default arguments
and named parameters:
</p>

<div class="org-src-container">
<pre class="src src-python"><span style="color: #b22222;"># </span><span style="color: #b22222;">Function with named parameters</span>
<span style="color: #a020f0;">def</span> <span style="color: #0000ff;">print_user_info</span>(name, age, city=<span style="color: #8b2252;">"Unknown"</span>, country=<span style="color: #8b2252;">"Unknown"</span>):
    <span style="color: #483d8b;">print</span>(f<span style="color: #8b2252;">"Name: </span>{name}<span style="color: #8b2252;">, Age: </span>{age}<span style="color: #8b2252;">, City: </span>{city}<span style="color: #8b2252;">, Country: </span>{country}<span style="color: #8b2252;">"</span>)

<span style="color: #b22222;"># </span><span style="color: #b22222;">Using named parameters</span>
print_user_info(name=<span style="color: #8b2252;">"John"</span>, age=25, city=<span style="color: #8b2252;">"New York"</span>, country=<span style="color: #8b2252;">"USA"</span>)

<span style="color: #b22222;"># </span><span style="color: #b22222;">Omitting some named parameters (using defaults)</span>
print_user_info(name=<span style="color: #8b2252;">"Alice"</span>, age=30)

<span style="color: #b22222;"># </span><span style="color: #b22222;">Mixing ordered and named parameters</span>
print_user_info(<span style="color: #8b2252;">"Bob"</span>, 28, country=<span style="color: #8b2252;">"Canada"</span>)
</pre>
</div>


<p>
The following Java example shows how function overloading works:
</p>

<div class="org-src-container">
<pre class="src src-java">
<span style="color: #b22222;">// </span><span style="color: #b22222;">Java program to demonstrate working of method overloading in Java</span>

<span style="color: #a020f0;">public</span> <span style="color: #a020f0;">class</span> <span style="color: #228b22;">Person</span> {
    <span style="color: #a020f0;">private</span> <span style="color: #228b22;">String</span> <span style="color: #a0522d;">firstName</span>;
    <span style="color: #a020f0;">private</span> <span style="color: #228b22;">String</span> <span style="color: #a0522d;">lastName</span>;

    <span style="color: #b22222;">// </span><span style="color: #b22222;">Person constructor with two arguments, the first and last name.</span>
    <span style="color: #a020f0;">public</span> Person(<span style="color: #228b22;">String</span> <span style="color: #a0522d;">firstName</span>, String lastName) {
        <span style="color: #a020f0;">this</span>.firstName = firstName;
        <span style="color: #a020f0;">this</span>.lastName = lastName;
    }

    <span style="color: #b22222;">// </span><span style="color: #b22222;">Another constructor, this time with a single argument, the first name.</span>
    <span style="color: #a020f0;">public</span> Person(String firstName) {
        <span style="color: #a020f0;">this</span>.firstName = firstName;
    }

    <span style="color: #a020f0;">public</span> <span style="color: #a020f0;">static</span> <span style="color: #228b22;">void</span> <span style="color: #0000ff;">main</span>(<span style="color: #228b22;">String</span>[] <span style="color: #a0522d;">args</span>) {
      <span style="color: #b22222;">// </span><span style="color: #b22222;">Using constructor with two parameters</span>
      <span style="color: #228b22;">Person</span> <span style="color: #a0522d;">person1</span> = <span style="color: #a020f0;">new</span> <span style="color: #228b22;">Person</span>(<span style="color: #8b2252;">"John"</span>, <span style="color: #8b2252;">"Doe"</span>);
      <span style="color: #b22222;">// </span><span style="color: #b22222;">Using constructor with one parameter</span>
      <span style="color: #228b22;">Person</span> <span style="color: #a0522d;">person2</span> = <span style="color: #a020f0;">new</span> <span style="color: #228b22;">Person</span>(<span style="color: #8b2252;">"John"</span>);
  }
}
</pre>
</div>
</div>
</div>


<div id="outline-container-org034fc7f" class="outline-2">
<h2 id="org034fc7f"><span class="section-number-2">3.</span> The builder pattern in Java <sup><a id="fnr.6" class="footref" href="#fn.6" role="doc-backlink">6</a></sup></h2>
<div class="outline-text-2" id="text-3">
<p>
Factories and constructors share a limitation: they do not scale well
to large numbers of optional parameters. Consider the case of a class
representing the ingredients of a Pizza. Most ingredients have nonzero
values for only a few of these optional fields.
</p>

<p>
What sort of constructors or static factories should you write for
such a class? Traditionally, programmers have used the telescoping
constructor pattern, in which you provide a constructor with only the
required parameters, another with a single optional parameter, a
third with two optional parameters, and so on, culminating in a
constructor with all the optional parameters. Here’s how it looks
in practice. For brevity’s sake, only three optional cheese fields
are shown:
</p>

<div class="org-src-container">
<pre class="src src-java"><span class="linenr"> 1: </span><span style="color: #a020f0;">public</span> <span style="color: #a020f0;">class</span> <span style="color: #228b22;">Pizza</span> {
<span class="linenr"> 2: </span>    <span style="color: #a020f0;">private</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">dough</span>;      <span style="color: #b22222;">// </span><span style="color: #b22222;">100g, 200g, 300g  required</span>
<span class="linenr"> 3: </span>    <span style="color: #a020f0;">private</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">mozarella</span>;  <span style="color: #b22222;">// </span><span style="color: #b22222;">100g, 200g, 300g  optional</span>
<span class="linenr"> 4: </span>    <span style="color: #a020f0;">private</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">parmesan</span>;   <span style="color: #b22222;">// </span><span style="color: #b22222;">50g, 100g, 150g   optional</span>
<span class="linenr"> 5: </span>    <span style="color: #a020f0;">private</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">gorgonzola</span>; <span style="color: #b22222;">// </span><span style="color: #b22222;">50g, 100g, 150g   optional</span>
<span class="linenr"> 6: </span>
<span class="linenr"> 7: </span>    <span style="color: #a020f0;">public</span> Pizza(<span style="color: #228b22;">int</span> <span style="color: #a0522d;">dough</span>) {
<span class="linenr"> 8: </span>        <span style="color: #a020f0;">this</span>(dough, 0, 0, 0);
<span class="linenr"> 9: </span>    }
<span class="linenr">10: </span>
<span class="linenr">11: </span>    <span style="color: #a020f0;">public</span> Pizza(<span style="color: #228b22;">int</span> <span style="color: #a0522d;">dough</span>, <span style="color: #228b22;">int</span> <span style="color: #a0522d;">mozarella</span>) {
<span class="linenr">12: </span>        <span style="color: #a020f0;">this</span>(dough, mozarella, 0, 0);
<span class="linenr">13: </span>    }
<span class="linenr">14: </span>
<span class="linenr">15: </span>    <span style="color: #a020f0;">public</span> Pizza(<span style="color: #228b22;">int</span> <span style="color: #a0522d;">dough</span>, <span style="color: #228b22;">int</span> <span style="color: #a0522d;">mozarella</span>, <span style="color: #228b22;">int</span> <span style="color: #a0522d;">parmesan</span>) {
<span class="linenr">16: </span>        <span style="color: #a020f0;">this</span>(dough, mozarella, parmesan, 0);
<span class="linenr">17: </span>    }
<span class="linenr">18: </span>
<span class="linenr">19: </span>    <span style="color: #a020f0;">public</span> Pizza(<span style="color: #228b22;">int</span> <span style="color: #a0522d;">dough</span>, <span style="color: #228b22;">int</span> <span style="color: #a0522d;">mozarella</span>, <span style="color: #228b22;">int</span> <span style="color: #a0522d;">parmesan</span>, <span style="color: #228b22;">int</span> <span style="color: #a0522d;">gorgonzola</span>) {
<span class="linenr">20: </span>        <span style="color: #a020f0;">this</span>.dough = dough;
<span class="linenr">21: </span>        <span style="color: #a020f0;">this</span>.mozarella = mozarella;
<span class="linenr">22: </span>        <span style="color: #a020f0;">this</span>.parmesan = parmesan;
<span class="linenr">23: </span>        <span style="color: #a020f0;">this</span>.gorgonzola = gorgonzola;
<span class="linenr">24: </span>    }
<span class="linenr">25: </span>}
</pre>
</div>

<p>
When you want to create an instance, you use the constructor with the
shortest parameter list containing all the parameters you want to set:
</p>

<div class="org-src-container">
<pre class="src src-java"><span style="color: #228b22;">Pizza</span> <span style="color: #a0522d;">margarita</span> = <span style="color: #a020f0;">new</span> <span style="color: #228b22;">Pizza</span>(200, 200);
</pre>
</div>


<p>
In short, the telescoping constructor pattern works, but it is hard to
write client code when there are many parameters, and harder still to read it.
</p>

<p>
Luckily, the builder pattern helps us with the readability and
tediousness of the code. Instead of making the desired object directly, the
client calls a constructor  with all of the required parameters and gets a
builder object. Then the client calls setter-like methods on the builder
object to set each optional parameter of interest. Finally, the client calls
a parameterless build method to generate the object, which is typically
immutable. The builder is typically a static member class  of the class
it builds. Here’s how it looks in practice:
</p>

<div class="org-src-container">
<pre class="src src-java"><span style="color: #a020f0;">public</span> <span style="color: #a020f0;">class</span> <span style="color: #228b22;">Pizza</span> {
    <span style="color: #a020f0;">private</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">dough</span>;      <span style="color: #b22222;">// </span><span style="color: #b22222;">100g, 200g, 300g  required</span>
    <span style="color: #a020f0;">private</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">mozarella</span>;  <span style="color: #b22222;">// </span><span style="color: #b22222;">100g, 200g, 300g  optional</span>
    <span style="color: #a020f0;">private</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">parmesan</span>;   <span style="color: #b22222;">// </span><span style="color: #b22222;">50g, 100g, 150g   optional</span>
    <span style="color: #a020f0;">private</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">gorgonzola</span>; <span style="color: #b22222;">// </span><span style="color: #b22222;">50g, 100g, 150g   optional</span>

    <span style="color: #a020f0;">public</span> <span style="color: #a020f0;">static</span> <span style="color: #a020f0;">class</span> <span style="color: #228b22;">Builder</span> {
        <span style="color: #b22222;">// </span><span style="color: #b22222;">Required parameters</span>
        <span style="color: #a020f0;">private</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">dough</span>;

        <span style="color: #b22222;">// </span><span style="color: #b22222;">Optional parameters - initialized to default values</span>
        <span style="color: #a020f0;">private</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">mozarella</span> = 0;
        <span style="color: #a020f0;">private</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">parmesan</span> = 0;
        <span style="color: #a020f0;">private</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">gorgonzola</span> = 0;

        <span style="color: #a020f0;">public</span> Builder(<span style="color: #228b22;">int</span> <span style="color: #a0522d;">dough</span>) {
            <span style="color: #a020f0;">this</span>.dough = dough;
        }

        <span style="color: #a020f0;">public</span> <span style="color: #228b22;">Builder</span> <span style="color: #0000ff;">setMozarella</span>(<span style="color: #228b22;">int</span> <span style="color: #a0522d;">val</span>) {
            mozarella = val;
            <span style="color: #a020f0;">return</span> <span style="color: #a020f0;">this</span>;
        }

        <span style="color: #a020f0;">public</span> <span style="color: #228b22;">Pizza</span> <span style="color: #0000ff;">build</span>() {
            <span style="color: #a020f0;">return</span> <span style="color: #a020f0;">new</span> <span style="color: #228b22;">Pizza</span>(<span style="color: #a020f0;">this</span>);
        }
    }

    <span style="color: #a020f0;">private</span> Pizza(Builder builder) {
        dough = builder.dough;
        mozarella = builder.mozarella;
        parmesan = builder.parmesan;
        gorgonzola = builder.gorgonzola;
    }
}
</pre>
</div>

<p>
The Pizza class is immutable, and all parameter default values
are in one place. The builder’s setter methods return the builder
itself so that invocations can be chained.  Here’s how the
client code looks:
</p>

<div class="org-src-container">
<pre class="src src-java"><span style="color: #228b22;">Pizza</span> <span style="color: #a0522d;">pizza</span> = <span style="color: #a020f0;">new</span> <span style="color: #008b8b;">Pizza</span>.<span style="color: #228b22;">Builder</span>(200).setMozarella(200).setGorgonzola(50).build();
</pre>
</div>

<p>
The Builder pattern simulates default arguments and named parameters as
found in Python and eludes the telescoping pattern avoiding function
overloading.
</p>
</div>
</div>

<div id="outline-container-org2a12896" class="outline-2">
<h2 id="org2a12896"><span class="section-number-2">4.</span> The builder pattern in Python</h2>
<div class="outline-text-2" id="text-4">
<p>
In python, we just simply leverage the language support for named
parameters and default values as explained in <a href="#org2897cb6">2</a> to write
pythonic code for the builder pattern.
</p>

<div class="org-src-container">
<pre class="src src-python"><span style="color: #a020f0;">class</span> <span style="color: #228b22;">Pizza</span>:
    <span style="color: #8b2252;">"""</span>
<span style="color: #8b2252;">    Pizza class to represent a pizza with its ingredients.</span>
<span style="color: #8b2252;">    To set the ingredients the builder pattern is used.</span>
<span style="color: #8b2252;">    """</span>

    <span style="color: #a020f0;">def</span> <span style="color: #0000ff;">__init__</span>(
        <span style="color: #a020f0;">self</span>,
        dough: <span style="color: #483d8b;">int</span>,
        mozarella: <span style="color: #483d8b;">int</span> = 0,
        parmesan: <span style="color: #483d8b;">int</span> = 0,
        gorgonzola: <span style="color: #483d8b;">int</span> = 0,
    ) -&gt; <span style="color: #008b8b;">None</span>:
        <span style="color: #a020f0;">self</span>.<span style="color: #a0522d;">dough</span> = dough
        <span style="color: #a020f0;">self</span>.<span style="color: #a0522d;">mozarella</span> = mozarella
        <span style="color: #a020f0;">self</span>.<span style="color: #a0522d;">parmesan</span> = parmesan
        <span style="color: #a020f0;">self</span>.<span style="color: #a0522d;">gorgonzola</span> = gorgonzola
</pre>
</div>

<p>
This time we do not need to concatenate calls, nor call a build
method to instantiate a pizza object.
</p>

<div class="org-src-container">
<pre class="src src-python"><span style="color: #a0522d;">pizza</span> = Pizza(200, mozarella=200, gorgonzola=50)                                                                                                                                        
</pre>
</div>
</div>
</div>

<div id="outline-container-orgaab50aa" class="outline-2">
<h2 id="orgaab50aa"><span class="section-number-2">5.</span> Summary</h2>
<div class="outline-text-2" id="text-5">
<p>
Exploring the Builder Pattern in Java and Python, we uncovered
language-specific nuances. While Java employs an inner builder
class to simulate features like named parameters and default
arguments found natively in Python, the latter provides a more
concise and idiomatic approach. The post contrasts these
implementations, offering insights into the divergent paths each
language takes when applying the Builder Pattern.
</p>
</div>
</div>
<div id="footnotes">
<h2 class="footnotes">Footnotes: </h2>
<div id="text-footnotes">

<div class="footdef"><sup><a id="fn.1" class="footnum" href="#fnr.1" role="doc-backlink">1</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://en.wikipedia.org/wiki/Builder_pattern">https://en.wikipedia.org/wiki/Builder_pattern</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.2" class="footnum" href="#fnr.2" role="doc-backlink">2</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://en.wikipedia.org/wiki/Design_Patterns">https://en.wikipedia.org/wiki/Design_Patterns</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.3" class="footnum" href="#fnr.3" role="doc-backlink">3</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://en.wikipedia.org/wiki/Default_argument">https://en.wikipedia.org/wiki/Default_argument</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.4" class="footnum" href="#fnr.4" role="doc-backlink">4</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://en.wikipedia.org/wiki/Named_parameter">https://en.wikipedia.org/wiki/Named_parameter</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.5" class="footnum" href="#fnr.5" role="doc-backlink">5</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://en.wikipedia.org/wiki/Function_overloading">https://en.wikipedia.org/wiki/Function_overloading</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.6" class="footnum" href="#fnr.6" role="doc-backlink">6</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
Based on the excellent book "Effective Java: Programming
Language Guide" (Third edition 2017) from Joshua Bloch. Item 2:
Consider a builder when faced with many constructor paramters.
</p></div></div>


</div>
</div><div class="taglist"><a href="https://paconte.com/tags.html">Tags</a>: <a href="https://paconte.com/tag-java.html">java</a> <a href="https://paconte.com/tag-python.html">python</a> <a href="https://paconte.com/tag-design-patterns.html">design-patterns</a> </div>]]></description>
  <category><![CDATA[java]]></category>
  <category><![CDATA[python]]></category>
  <category><![CDATA[design-patterns]]></category>
  <link>https://paconte.com/2024-03-03-builder-pattern-python-vs-java.html</link>
  <guid>https://paconte.com/2024-03-03-builder-pattern-python-vs-java.html</guid>
  <pubDate>Sun, 03 Mar 2024 15:07:00 +0100</pubDate>
</item>
<item>
  <title><![CDATA[Java singleton]]></title>
  <description><![CDATA[
<nav id="table-of-contents" role="doc-toc">
<h2>Table of Contents</h2>
<div id="text-table-of-contents" role="doc-toc">
<ul>
<li><a href="#orgec9ed45">1. Introduction</a></li>
<li><a href="#org2ae9f0e">2. Java features to consider when creating a singleton</a>
<ul>
<li><a href="#orgdc77709">2.1. Concurrency</a></li>
<li><a href="#org292399c">2.2. Reflection</a></li>
<li><a href="#orgaee47b0">2.3. Serialization</a></li>
</ul>
</li>
<li><a href="#org80fe133">3. Singleton implementation</a></li>
<li><a href="#org87ad9df">4. Summary</a></li>
</ul>
</div>
</nav>
<p>
Not long time ago, I discovered in Java it is possible to
implement a Singleton with an Enum. And not only that, it is the
recommended way to implement them!
</p>

<p>
In this post, we explore the best practices when implementing the
singleton pattern in Java.
</p>

<div id="outline-container-orgec9ed45" class="outline-2">
<h2 id="orgec9ed45"><span class="section-number-2">1.</span> Introduction</h2>
<div class="outline-text-2" id="text-1">
<p>
The following text is extracted from the wikipedia <sup><a id="fnr.1" class="footref" href="#fn.1" role="doc-backlink">1</a></sup>:
</p>

<p>
In software engineering, the singleton pattern is a software
design pattern that restricts the instantiation of a class to
a singular instance. One of the well-known "Gang of Four" design
patterns <sup><a id="fnr.2" class="footref" href="#fn.2" role="doc-backlink">2</a></sup>, which describes how to solve recurring problems
in object-oriented software, the pattern is useful when exactly
one object is needed to coordinate actions across a system.
</p>

<p>
More specifically, the singleton pattern allows objects to:
</p>

<ul class="org-ul">
<li>Ensure they only have one instance</li>
<li>Provide easy access to that instance</li>
<li>Control their instantiation (for example, hiding the
constructors of a class)</li>
</ul>

<p>
The term comes from the mathematical concept of a singleton <sup><a id="fnr.3" class="footref" href="#fn.3" role="doc-backlink">3</a></sup>.
</p>

<p>
In this post, we'll dive into the best practices for implementing
the singleton pattern in Java, considering key features such as
concurrency, reflection, and serialization.
</p>
</div>
</div>

<div id="outline-container-org2ae9f0e" class="outline-2">
<h2 id="org2ae9f0e"><span class="section-number-2">2.</span> Java features to consider when creating a singleton</h2>
<div class="outline-text-2" id="text-2">
</div>
<div id="outline-container-orgdc77709" class="outline-3">
<h3 id="orgdc77709"><span class="section-number-3">2.1.</span> Concurrency</h3>
<div class="outline-text-3" id="text-2-1">
<p>
<a id="orga2bf656"></a>
When creating a singleton in Java we need to make sure it is
thread safe<sup><a id="fnr.4" class="footref" href="#fn.4" role="doc-backlink">4</a></sup>. That is, when two or more threads instantiate a
singleton only a single instance is created.
</p>

<p>
For this purpose, we will leverage the static keywork. The keyword
static means that the particular member belongs to a type itself,
rather than to an instance of that type.
</p>

<p>
From the java docs<sup><a id="fnr.5" class="footref" href="#fn.5" role="doc-backlink">5</a></sup>: "If a field is declared static, there
exists exactly one incarnation of the field, no matter how many
instances (possibly zero) of the class may eventually be created.
</p>
</div>
</div>

<div id="outline-container-org292399c" class="outline-3">
<h3 id="org292399c"><span class="section-number-3">2.2.</span> Reflection</h3>
<div class="outline-text-3" id="text-2-2">
<p>
<a id="org9c3deba"></a>
In the later examples, we will make use of a private contructor to
implement the Singleton. With the help of Reflection<sup><a id="fnr.6" class="footref" href="#fn.6" role="doc-backlink">6</a></sup>, an
attacker could modify the runtime behaviour of the application
modifying the accessibility of the private constructor to public.
In other words, with the help of Reflection it would be possible
to create a second instance of the class.
</p>
</div>
</div>

<div id="outline-container-orgaee47b0" class="outline-3">
<h3 id="orgaee47b0"><span class="section-number-3">2.3.</span> Serialization</h3>
<div class="outline-text-3" id="text-2-3">
<p>
<a id="orgba07b11"></a>
Serialization is the process of translating an object into a format
that can be stored or transmitted and reconstructed later. See
figure 1.
</p>


<figure id="orge3baf7b">
<img src="./images/java-singleton/Serialization.jpg" alt="Serialization.jpg" width="600px" height="268px">

<figcaption><span class="figure-number">Figure 1: </span>Serialization [WnbKrumov, CC BY-SA 3.0, via Wikimedia Commons].</figcaption>
</figure>

<p>
In Java, serializability of a class is enabled by the class
implementing the java.io.Serializable interface.
</p>

<p>
Now, imagine your application serializes your singleton instance, so far
so good, a stream of bytes has been created. However, when your application
deserialize the same stream of bytes, it might end creating a new instance,
leading, in the case of our example, to have two different instances!
</p>

<p>
Therefore we need to be carefull with Serialization when implementing our
Singleton.
</p>
</div>
</div>
</div>

<div id="outline-container-org80fe133" class="outline-2">
<h2 id="org80fe133"><span class="section-number-2">3.</span> Singleton implementation</h2>
<div class="outline-text-2" id="text-3">
<p>
The two common ways of implementing singletons are based on keeping the constructor
private and exporting a public static member to provide access to the instance.
</p>

<p>
In this first approach, the member INSTANCE is a final static field. The private
constructor is called only once, to initialize the public static final field
Spyderman.INSTANCE.
</p>

<div class="org-src-container">
<pre class="src src-java"><span style="color: #b22222;">// </span><span style="color: #b22222;">Singleton with public final field</span>
<span style="color: #a020f0;">public</span> <span style="color: #a020f0;">class</span> <span style="color: #228b22;">Spyderman</span> {
    <span style="color: #a020f0;">public</span> <span style="color: #a020f0;">static</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">Spyderman</span> <span style="color: #a0522d;">INSTANCE</span> = <span style="color: #a020f0;">new</span> <span style="color: #228b22;">Spyderman</span>();
    <span style="color: #a020f0;">private</span> Spyderman() { ... }
    <span style="color: #a020f0;">public</span> <span style="color: #228b22;">void</span> <span style="color: #0000ff;">withGreatPowerComesGreatResponsibility</span>() { ... }
}
</pre>
</div>

<p>
Thanks to the static final INSTANCE, we make sure the singleton is thread
safe <a href="#orga2bf656">2.1</a>. However, an attacker could change thanks to reflection <a href="#org9c3deba">2.2</a>
the accessibility to the constructor, making it public or private and invoking it
twice or more. We can defend ourselves against this attack making the constructor
to throw an exception if it is invoked to create a second instance.
</p>

<p>
In the second approach to implement sigletons, the public member is a static
method:
</p>

<div class="org-src-container">
<pre class="src src-java"><span style="color: #b22222;">// </span><span style="color: #b22222;">Singleton with static factory</span>
<span style="color: #a020f0;">public</span> <span style="color: #a020f0;">class</span> <span style="color: #228b22;">Spyderman</span> {
    <span style="color: #a020f0;">private</span> <span style="color: #a020f0;">static</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">Spyderman</span> <span style="color: #a0522d;">INSTANCE</span> = <span style="color: #a020f0;">new</span> <span style="color: #228b22;">Spyderman</span>();
    <span style="color: #a020f0;">private</span> Spyderman() { ... }
    <span style="color: #a020f0;">public</span> <span style="color: #a020f0;">static</span> <span style="color: #228b22;">Spyderman</span> <span style="color: #0000ff;">getInstance</span>() { <span style="color: #a020f0;">return</span> INSTANCE; }
    <span style="color: #a020f0;">public</span> <span style="color: #228b22;">void</span> <span style="color: #0000ff;">withGreatPowerComesGreatResponsibility</span>() { ... }
}
</pre>
</div>

<p>
All calls to <i>Spyderman.getInstance()</i> return the same object reference. Same as the
first example, it is thread safe thanks to the statics fields and it has the same
caveat with reflection.
</p>

<p>
This second implementation is the prefered one, as it is more flexible, allowing
the developer to remove the singleton property without changing the API as well as
implement one singleton per thread or incorparating generics.
</p>

<p>
Now, if we need to make one of the above approaches Serializable adding
<i>implements Serializable</i>, we need to modify our code to keep the singleton
guarantee. Otherwise, each time a serialized instance is deserialized, as explained
before <a href="#orgba07b11">2.3</a>, a new instance will be created. The solution would be to
declare all instance fields <i>transient</i> and implement the <i>readResolve</i> method. We
can find examples of this in the <i>java.util.Collections</i><sup><a id="fnr.7" class="footref" href="#fn.7" role="doc-backlink">7</a></sup> file.
</p>

<p>
There is a third approach to the Singleton pattern, where we do not have to deal with
thread safety, reflection nor serialization. It is the enum approach:
</p>

<div class="org-src-container">
<pre class="src src-java"><span style="color: #b22222;">// </span><span style="color: #b22222;">Enum singleton - the preferred approach</span>
<span style="color: #a020f0;">public</span> <span style="color: #a020f0;">enum</span> <span style="color: #228b22;">Spyderman</span> {
    <span style="color: #a0522d;">INSTANCE</span>;
    <span style="color: #a020f0;">public</span> <span style="color: #228b22;">void</span> <span style="color: #0000ff;">withGreatPowerComesGreatResponsibility</span>() { ... }
}
</pre>
</div>

<p>
This approach look at first a bit inuntuituve. But lets check it properties. Every
enum constant is always implicitly public static final which make it thread safe.
There is no constructor that can be called by the developer so it makes it safe
againts reflection attacks and as per documentation<sup><a id="fnr.8" class="footref" href="#fn.8" role="doc-backlink">8</a></sup> Enums are <i>Serializable</i>
and special methods like <i>readResolve</i> are ignored.
</p>

<p>
The main disadvante is that Enums can not extend a superclass other than <i>Enum</i>,
luckily, we can implement other interfaces.
</p>
</div>
</div>


<div id="outline-container-org87ad9df" class="outline-2">
<h2 id="org87ad9df"><span class="section-number-2">4.</span> Summary</h2>
<div class="outline-text-2" id="text-4">
<p>
We have seen why the Enum approach is often the best way to implement a singleton.
The jvm provides the serialization and the protection against multiple instantiation
and reflection attacks for free.
</p>
</div>
</div>
<div id="footnotes">
<h2 class="footnotes">Footnotes: </h2>
<div id="text-footnotes">

<div class="footdef"><sup><a id="fn.1" class="footnum" href="#fnr.1" role="doc-backlink">1</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://en.wikipedia.org/wiki/Singleton_pattern">https://en.wikipedia.org/wiki/Singleton_pattern</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.2" class="footnum" href="#fnr.2" role="doc-backlink">2</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://en.wikipedia.org/wiki/Design_Patterns">https://en.wikipedia.org/wiki/Design_Patterns</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.3" class="footnum" href="#fnr.3" role="doc-backlink">3</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://en.wikipedia.org/wiki/Singleton_(mathematics)">https://en.wikipedia.org/wiki/Singleton_(mathematics)</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.4" class="footnum" href="#fnr.4" role="doc-backlink">4</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://en.wikipedia.org/wiki/Thread_safety">https://en.wikipedia.org/wiki/Thread_safety</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.5" class="footnum" href="#fnr.5" role="doc-backlink">5</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://docs.oracle.com/javase/specs/jls/se21/html/jls-8.html#jls-8.3.1.1">https://docs.oracle.com/javase/specs/jls/se21/html/jls-8.html#jls-8.3.1.1</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.6" class="footnum" href="#fnr.6" role="doc-backlink">6</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://www.oracle.com/technical-resources/articles/java/javareflection.html">https://www.oracle.com/technical-resources/articles/java/javareflection.html</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.7" class="footnum" href="#fnr.7" role="doc-backlink">7</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://github.com/openjdk/jdk/blob/jdk-17%2B35/src/java.base/share/classes/java/util/Collections.java">https://github.com/openjdk/jdk/blob/jdk-17%2B35/src/java.base/share/classes/java/util/Collections.java</a>
</p></div></div>

<div class="footdef"><sup><a id="fn.8" class="footnum" href="#fnr.8" role="doc-backlink">8</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">
<a href="https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/io/Serializable.html">https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/io/Serializable.html</a>
</p></div></div>


</div>
</div><div class="taglist"><a href="https://paconte.com/tags.html">Tags</a>: <a href="https://paconte.com/tag-java.html">java</a> <a href="https://paconte.com/tag-design-patterns.html">design-patterns</a> </div>]]></description>
  <category><![CDATA[java]]></category>
  <category><![CDATA[design-patterns]]></category>
  <link>https://paconte.com/2023-03-17-java-singleton.html</link>
  <guid>https://paconte.com/2023-03-17-java-singleton.html</guid>
  <pubDate>Fri, 17 Mar 2023 08:37:00 +0100</pubDate>
</item>
<item>
  <title><![CDATA[Comparing Java map and Python dict]]></title>
  <description><![CDATA[
<nav id="table-of-contents" role="doc-toc">
<h2>Table of Contents</h2>
<div id="text-table-of-contents" role="doc-toc">
<ul>
<li><a href="#org70df295">1. Hash collisions</a>
<ul>
<li><a href="#org1ce7519">1.1. Separate chaining</a></li>
<li><a href="#org8ba4c78">1.2. Open addressing</a></li>
</ul>
</li>
<li><a href="#orgf114c71">2. Java map</a></li>
<li><a href="#org7e0e4db">3. Python dictionary</a></li>
<li><a href="#org5642fd6">4. Performance</a>
<ul>
<li><a href="#orgb3240c8">4.1. Load factor</a></li>
</ul>
</li>
<li><a href="#org80655b3">5. What about sets</a></li>
<li><a href="#orgb374ad8">6. Summary</a></li>
<li><a href="#org1a98dad">7. A note on source code</a></li>
</ul>
</div>
</nav>
<p>
Recently, I have been asked how python solves hash collisions in
dictionaries. At that moment, I knew the answer for java maps, but
not for python dictionaries. That was the starting point of this entry.<br>
</p>

<p>
Java maps and Python dictionaries are both implemented using hash tables,
which are highly efficient data structures for storing and retrieving
key-value pairs. Hash tables have an average search complexity of O(1),
making them more efficient than search trees or other table lookup
structures. For this reason, they are widely used in computer science.<br>
</p>

<p>
Figure 1 is a refresher of how hash tables works. Looking after a phone
number in an agenda is a repetitive task, and we want it to be as fast
as possible.
</p>


<figure id="org73f6ec6">
<img src="./images/comparing-java-map-python-dict/hash_table_example_630x460px.svg.png" alt="hash_table_example_630x460px.svg.png" width="567px" height="414px">

<figcaption><span class="figure-number">Figure 1: </span>A small phone book as a hash table [Jorge Stolfi, CC BY-SA 3.0, via Wikimedia Commons].</figcaption>
</figure>


<div id="outline-container-org70df295" class="outline-2">
<h2 id="org70df295"><span class="section-number-2">1.</span> Hash collisions</h2>
<div class="outline-text-2" id="text-1">
<p>
Java maps and python dictionaries implementations differs from each
other in how they solve hash collisions. A collision is when two keys
share the same hash value. Hash collisions are inevitable, and two of
the most common strategies of collision resolution are <b>open addressing</b>
and <b>separate chaining</b>.
</p>
</div>

<div id="outline-container-org1ce7519" class="outline-3">
<h3 id="org1ce7519"><span class="section-number-3">1.1.</span> Separate chaining</h3>
<div class="outline-text-3" id="text-1-1">

<figure id="org4cdca4a">
<img src="./images/comparing-java-map-python-dict/hash_table_chaining_450x310px.svg.png" alt="hash_table_chaining_450x310px.svg.png" width="495px" height="341px">

<figcaption><span class="figure-number">Figure 2: </span>Hash collision resolved by separate chaining [Jorge Stolfi, CC BY-SA 3.0, via Wikimedia Commons].</figcaption>
</figure>

<p>
As shown in figure two, when two different keys point to the same cell value,
the cell value contains a linked list with all the collisions. In this case
the search average time is O(n) where n is the number of collisions. The
worst case scenario is when the table has only one cell, then n is the length
of your whole collection.
</p>
</div>
</div>

<div id="outline-container-org8ba4c78" class="outline-3">
<h3 id="org8ba4c78"><span class="section-number-3">1.2.</span> Open addressing</h3>
<div class="outline-text-3" id="text-1-2">
<p>
This technique is not as simple as separate chaining, but it should have a
better performance. If a hash collision occurs, the table will be probed
to move the record to a different cell that is empty. There are different
probing techniques, for example in figure three the next cell position is used.
</p>


<figure id="org0f0d31f">
<img src="./images/comparing-java-map-python-dict/hash_table_open_addressing_380x330px.svg.png" alt="hash_table_open_addressing_380x330px.svg.png" width="418px" height="363px">

<figcaption><span class="figure-number">Figure 3: </span>Hash collusion resolved by open addressing [Jorge Stolfi, CC BY-SA 3.0, via Wikimedia Commons].</figcaption>
</figure>
</div>
</div>
</div>


<div id="outline-container-orgf114c71" class="outline-2">
<h2 id="orgf114c71"><span class="section-number-2">2.</span> Java map</h2>
<div class="outline-text-2" id="text-2">
<p>
If we inspect the Hashmap class from the <code>java.util</code> package, we will see
that java uses separate chaining for solving hash clashes. Java also adds
a performance improvement, instead of using linked list to chain the
collisions, when there are too many collisions the linked list is
converted into a binary tree reducing the search average time from <code>O(n)</code>
to <code>O(log2 n)</code>.
</p>

<p>
The code below correspond to the <code>putVal</code> method from the <code>Hashmap</code>
class which is in
charge of writing into the hash table. <code>Line 6</code> writes into an empty
cell, <code>line 17</code> inserts a new collision into a linked list and <code>line 13</code>
into a binary tree. Finally, <code>line 19</code> converts the linked list into a
binary tree.
</p>

<div class="org-src-container">
<pre class="src src-java"><span class="linenr"> 1: </span><span style="color: #a020f0;">final</span> <span style="color: #228b22;">V</span> <span style="color: #0000ff;">putVal</span>(<span style="color: #228b22;">int</span> <span style="color: #a0522d;">hash</span>, <span style="color: #228b22;">K</span> <span style="color: #a0522d;">key</span>, <span style="color: #228b22;">V</span> <span style="color: #a0522d;">value</span>, <span style="color: #228b22;">boolean</span> <span style="color: #a0522d;">onlyIfAbsent</span>, <span style="color: #228b22;">boolean</span> <span style="color: #a0522d;">evict</span>) {
<span class="linenr"> 2: </span>    <span style="color: #228b22;">Node</span>&lt;<span style="color: #228b22;">K</span>,<span style="color: #228b22;">V</span>&gt;[] <span style="color: #a0522d;">tab</span>; <span style="color: #228b22;">Node</span>&lt;<span style="color: #228b22;">K</span>,<span style="color: #228b22;">V</span>&gt; <span style="color: #a0522d;">p</span>; <span style="color: #228b22;">int</span> <span style="color: #a0522d;">n</span>, <span style="color: #a0522d;">i</span>;
<span class="linenr"> 3: </span>    <span style="color: #a020f0;">if</span> ((tab = table) == <span style="color: #008b8b;">null</span> || (n = tab.length) == 0)
<span class="linenr"> 4: </span>        n = (tab = resize()).length;
<span class="linenr"> 5: </span>    <span style="color: #a020f0;">if</span> ((p = tab[i = (n - 1) &amp; hash]) == <span style="color: #008b8b;">null</span>)
<span class="linenr"> 6: </span>        tab[i] = newNode(hash, key, value, <span style="color: #008b8b;">null</span>);
<span class="linenr"> 7: </span>    <span style="color: #a020f0;">else</span> {
<span class="linenr"> 8: </span>        <span style="color: #228b22;">Node</span>&lt;<span style="color: #228b22;">K</span>,<span style="color: #228b22;">V</span>&gt; <span style="color: #a0522d;">e</span>; <span style="color: #228b22;">K</span> <span style="color: #a0522d;">k</span>;
<span class="linenr"> 9: </span>        <span style="color: #a020f0;">if</span> (p.hash == hash &amp;&amp;
<span class="linenr">10: </span>            ((k = p.key) == key || (key != <span style="color: #008b8b;">null</span> &amp;&amp; key.equals(k))))
<span class="linenr">11: </span>            e = p;
<span class="linenr">12: </span>        <span style="color: #a020f0;">else</span> <span style="color: #a020f0;">if</span> (p <span style="color: #a020f0;">instanceof</span> TreeNode)
<span class="linenr">13: </span>            e = ((<span style="color: #228b22;">TreeNode</span>&lt;<span style="color: #228b22;">K</span>,<span style="color: #228b22;">V</span>&gt;)p).putTreeVal(<span style="color: #a020f0;">this</span>, tab, hash, key, value);
<span class="linenr">14: </span>        <span style="color: #a020f0;">else</span> {
<span class="linenr">15: </span>            <span style="color: #a020f0;">for</span> (<span style="color: #228b22;">int</span> <span style="color: #a0522d;">binCount</span> = 0; ; ++binCount) {
<span class="linenr">16: </span>                <span style="color: #a020f0;">if</span> ((e = p.next) == <span style="color: #008b8b;">null</span>) {
<span class="linenr">17: </span>                    p.next = newNode(hash, key, value, <span style="color: #008b8b;">null</span>);
<span class="linenr">18: </span>                    <span style="color: #a020f0;">if</span> (binCount &gt;= TREEIFY_THRESHOLD - 1) <span style="color: #b22222;">// </span><span style="color: #b22222;">-1 for 1st</span>
<span class="linenr">19: </span>                        treeifyBin(tab, hash);
<span class="linenr">20: </span>                    <span style="color: #a020f0;">break</span>;
<span class="linenr">21: </span>                }
<span class="linenr">22: </span>                <span style="color: #a020f0;">if</span> (e.hash == hash &amp;&amp;
<span class="linenr">23: </span>                    ((k = e.key) == key || (key != <span style="color: #008b8b;">null</span> &amp;&amp; key.equals(k))))
<span class="linenr">24: </span>                    <span style="color: #a020f0;">break</span>;
<span class="linenr">25: </span>                p = e;
<span class="linenr">26: </span>            }
<span class="linenr">27: </span>        }
<span class="linenr">28: </span>    }
<span class="linenr">29: </span>}
</pre>
</div>

<p>
The above code is extracted from the Java SE 17 (LTS) version, the binary
tree improvement has been introduced in Java 8, in older versions the
chaining was done only with linked lists.
</p>
</div>
</div>


<div id="outline-container-org7e0e4db" class="outline-2">
<h2 id="org7e0e4db"><span class="section-number-2">3.</span> Python dictionary</h2>
<div class="outline-text-2" id="text-3">
<p>
Python in contrast to java uses open addressing to resolve hash collisions.
I recommend you to read the documentation from the source code: 
<a href="https://github.com/python/cpython/blob/3.11/Objects/dictobject.c">https://github.com/python/cpython/blob/3.11/Objects/dictobject.c</a>
Where the collision resolution is nicely explained at the beginning of the
file.
</p>

<p>
Python resolves hash table collisions applying the formula
<code>idx = (5 * idx + 1) % size</code>, where <code>idx</code> is the table position. 
Let's see an example:<br>
</p>


<figure id="orgf20b96d">
<img src="./images/comparing-java-map-python-dict/python_open_chaining_formula_720x228px.png" alt="python_open_chaining_formula_720x228px.png" width="648px" height="205px">

</figure>

<ol class="org-ol">
<li>Given a table of size <code>8</code></li>
<li>We want to insert an element in position <code>0</code>, which is not empty.</li>
<li>Applying the formula, the next cell to check is position <code>1</code>  <code>[(5*0 + 1) % 8 = 1]</code></li>
<li>The cell is not empty, next try is position <code>6</code> <code>[(5*1 + 1) % 8 = 6]</code></li>
<li>The following try is position <code>7 [(5*6 + 1) % 8 = 7]</code></li>
<li>Next try is position  <code>4 [(5*7 + 1) % 8 = 5]</code></li>
<li>Etc&#x2026;</li>
</ol>

<p>
Can you spot the pattern?
</p>

<p>
Python adds some randomness to the process adding a perturb value which is
calculated with the low bits of the hash, the final formula is
<code>idx = ((5 * idx) + 1 + perturb) % size</code>.
</p>


<p>
Unfortunately, the C source code of CPython dictionaries is not as straight
forward as Java code is. This is due to some optimizations that we will
see later on, when we will talk about performance. We can see the formula
<code>idx = ((5 * idx) + 1 + perturb) % size</code>
in action in the method <code>lookdict_index</code>, where <code>line 9</code> is an infinite
loop to find out the index, <code>line 17</code> recalculates the perturb value and
<code>line 18</code> applies the formula.
</p>


<div class="org-src-container">
<pre class="src src-c"><span class="linenr"> 1: </span><span style="color: #b22222;">/* </span><span style="color: #b22222;">Search index of hash table from offset of entry table</span><span style="color: #b22222;"> */</span>
<span class="linenr"> 2: </span><span style="color: #a020f0;">static</span> <span style="color: #228b22;">Py_ssize_t</span>
<span class="linenr"> 3: </span><span style="color: #0000ff;">lookdict_index</span>(<span style="color: #228b22;">PyDictKeysObject</span> *<span style="color: #a0522d;">k</span>, <span style="color: #228b22;">Py_hash_t</span> <span style="color: #a0522d;">hash</span>, <span style="color: #228b22;">Py_ssize_t</span> <span style="color: #a0522d;">index</span>)
<span class="linenr"> 4: </span>{
<span class="linenr"> 5: </span>    <span style="color: #228b22;">size_t</span> <span style="color: #a0522d;">mask</span> = DK_MASK(k);
<span class="linenr"> 6: </span>    <span style="color: #228b22;">size_t</span> <span style="color: #a0522d;">perturb</span> = (<span style="color: #228b22;">size_t</span>)hash;
<span class="linenr"> 7: </span>    <span style="color: #228b22;">size_t</span> <span style="color: #a0522d;">i</span> = (<span style="color: #228b22;">size_t</span>)hash &amp; mask;
<span class="linenr"> 8: </span>
<span class="linenr"> 9: </span>    <span style="color: #a020f0;">for</span> (;;) {
<span class="linenr">10: </span>        <span style="color: #228b22;">Py_ssize_t</span> <span style="color: #a0522d;">ix</span> = dictkeys_get_index(k, i);
<span class="linenr">11: </span>        <span style="color: #a020f0;">if</span> (ix == index) {
<span class="linenr">12: </span>            <span style="color: #a020f0;">return</span> i;
<span class="linenr">13: </span>        }
<span class="linenr">14: </span>        <span style="color: #a020f0;">if</span> (ix == DKIX_EMPTY) {
<span class="linenr">15: </span>            <span style="color: #a020f0;">return</span> DKIX_EMPTY;
<span class="linenr">16: </span>        }
<span class="linenr">17: </span>        perturb &gt;&gt;= PERTURB_SHIFT;
<span class="linenr">18: </span>        i = mask &amp; (i*5 + perturb + 1);
<span class="linenr">19: </span>    }
<span class="linenr">20: </span>    Py_UNREACHABLE();
<span class="linenr">21: </span>}
</pre>
</div>
</div>
</div>

<div id="outline-container-org5642fd6" class="outline-2">
<h2 id="org5642fd6"><span class="section-number-2">4.</span> Performance</h2>
<div class="outline-text-2" id="text-4">

<figure id="org3791d3c">
<img src="./images/comparing-java-map-python-dict/hash_table_average_insertion_time_362x235px.png" alt="hash_table_average_insertion_time_362x235px.png" width="362px" height="235px">

<figcaption><span class="figure-number">Figure 4: </span>Derrick Coetzee, Public domain, via Wikimedia Commons.</figcaption>
</figure>


<p>
The above graph compares the average number of CPU cache
misses required to look up elements in large hash tables with chaining
and linear probing. Linear probing performs better due to better locality
of reference, though as the table gets full, its performance degrades
drastically.
</p>

<p>
Python uses dicts internally when it creates objects, functions,
import modules, etc&#x2026; Therefore, the performance of dictionaries is
critical and linear probing is the way to go for them. The code below
shows how python uses dictionaries internally when creating classes.
</p>

<div class="org-src-container">
<pre class="src src-python"><span style="color: #a020f0;">class</span> <span style="color: #228b22;">Foo</span>():
    <span style="color: #a020f0;">def</span> <span style="color: #0000ff;">bar</span>(x):
        <span style="color: #a020f0;">return</span> x+1

&gt;&gt;&gt; <span style="color: #483d8b;">print</span>(Foo.<span style="color: #483d8b;">__dict__</span>)
{
  <span style="color: #8b2252;">'__module__'</span>: <span style="color: #8b2252;">'__main__'</span>,
  <span style="color: #8b2252;">'bar'</span>: &lt;function Foo.bar at 0x100d6b370&gt;,
  <span style="color: #8b2252;">'__dict__'</span>: &lt;attribute <span style="color: #8b2252;">'__dict__'</span> of <span style="color: #8b2252;">'Foo'</span> objects&gt;,
  <span style="color: #8b2252;">'__weakref__'</span>: &lt;attribute <span style="color: #8b2252;">'__weakref__'</span> of <span style="color: #8b2252;">'Foo'</span> objects&gt;,
  <span style="color: #8b2252;">'__doc__'</span>: <span style="color: #008b8b;">None</span>
}
</pre>
</div>
</div>


<div id="outline-container-orgb3240c8" class="outline-3">
<h3 id="orgb3240c8"><span class="section-number-3">4.1.</span> Load factor</h3>
<div class="outline-text-3" id="text-4-1">
<p>
As we have seen the load factor is crucial for the performance of hash table,
python has a load factor of 2/3 and java of 0.75. This makes sense, as linear
probing performance is very bad when there are no empty hash spaces. On the
other hand, java uses a threshold of 8 elements to switch from a linked list
to a binary tree, as we can see in the code below.
</p>

<div class="org-src-container">
<pre class="src src-java"><span style="color: #a020f0;">static</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">float</span> <span style="color: #a0522d;">DEFAULT_LOAD_FACTOR</span> = 0.75f;

<span style="color: #8b2252;">/**</span>
<span style="color: #8b2252;"> * The bin count threshold for using a tree rather than list for a</span>
<span style="color: #8b2252;"> * bin. Bins are converted to trees when adding an element to a</span>
<span style="color: #8b2252;"> * bin with at least this many nodes. The value must be greater</span>
<span style="color: #8b2252;"> * than 2 and should be at least 8 to mesh with assumptions in</span>
<span style="color: #8b2252;"> * tree removal about conversion back to plain bins upon</span>
<span style="color: #8b2252;"> * shrinkage.</span>
<span style="color: #8b2252;">*/</span>
<span style="color: #a020f0;">static</span> <span style="color: #a020f0;">final</span> <span style="color: #228b22;">int</span> <span style="color: #a0522d;">TREEIFY_THRESHOLD</span> = 8;
</pre>
</div>
</div>
</div>
</div>


<div id="outline-container-org80655b3" class="outline-2">
<h2 id="org80655b3"><span class="section-number-2">5.</span> What about sets</h2>
<div class="outline-text-2" id="text-5">
<p>
Dictionaries and maps are closely related to sets. In fact, sets are just
dictionaries/maps without values. Indeed, Java uses this approach to
implement sets, lets look at the source code: 
</p>

<div class="org-src-container">
<pre class="src src-java"><span style="color: #8b2252;">/**</span>
<span style="color: #8b2252;"> * Constructs a new, empty set; the backing </span><span style="color: #008b8b;">{@code HashMap}</span><span style="color: #8b2252;"> instance has</span>
<span style="color: #8b2252;"> * default initial capacity (16) and load factor (0.75).</span>
<span style="color: #8b2252;"> */</span>
<span style="color: #a020f0;">public</span> HashSet() {
    map = <span style="color: #a020f0;">new</span> <span style="color: #228b22;">HashMap</span>&lt;&gt;();
}
</pre>
</div>

<p>
There are two different reasons why python does not reuse Objects/dictobject.c for
implementing sets,
the first one is that CPython does not use sets internally and the requirements
are different. Looking after performance, CPython optimize the sets for the use
case of membership. It is well documented in the source code to be found in
Objects/setobject.c.:
</p>

<div class="org-src-container">
<pre class="src src-c"><span style="color: #b22222;">/*</span>
<span style="color: #b22222;">   Use cases for sets differ considerably from dictionaries where looked-up</span>
<span style="color: #b22222;">   keys are more likely to be present.  In contrast, sets are primarily</span>
<span style="color: #b22222;">   about membership testing where the presence of an element is not known in</span>
<span style="color: #b22222;">   advance.  Accordingly, the set implementation needs to optimize for both</span>
<span style="color: #b22222;">   the found and not-found case.</span>
<span style="color: #b22222;">*/</span>
</pre>
</div>

<p>
A set is a different object and the hash table works a bit different,
the set load factor is 60% instead of 66.6%, every time the table grows
it uses a factor of 4 instead of 2, and the main difference is in the
linear probing algorithm, where it inspects more than one cell for every
probe. 
</p>
</div>
</div>

<div id="outline-container-orgb374ad8" class="outline-2">
<h2 id="orgb374ad8"><span class="section-number-2">6.</span> Summary</h2>
<div class="outline-text-2" id="text-6">
<p>
CPython and Java use different approach to resolve hash collisions,
while Java uses separate chaining, CPython uses linear probing. Java
implements sets reusing hash tables but with dummy values, while python
using also linear probing, optimizes the sets for different use cases,
implementing a new linear probing algorithm. The reason for that is because
CPython uses dictionaries internally and a high performance is critical
for a proper performance of Python.
</p>
</div>
</div>

<div id="outline-container-org1a98dad" class="outline-2">
<h2 id="org1a98dad"><span class="section-number-2">7.</span> A note on source code</h2>
<div class="outline-text-2" id="text-7">
<p>
The source code examples are extracted respectively from the Java SE 17
(LTS) and CPython 3.11 versions.
</p>
</div>
</div>
<div class="taglist"><a href="https://paconte.com/tags.html">Tags</a>: <a href="https://paconte.com/tag-java.html">java</a> <a href="https://paconte.com/tag-python.html">python</a> <a href="https://paconte.com/tag-hash-table.html">hash-table</a> </div>]]></description>
  <category><![CDATA[java]]></category>
  <category><![CDATA[python]]></category>
  <category><![CDATA[hash-table]]></category>
  <link>https://paconte.com/2023-02-02-comparing-java-map-and-python-dictionary.html</link>
  <guid>https://paconte.com/2023-02-02-comparing-java-map-and-python-dictionary.html</guid>
  <pubDate>Thu, 02 Feb 2023 21:02:00 +0100</pubDate>
</item>
</channel>
</rss>
