deepmerge: deep merge dictionaries, lists and more in Python
Introducing deepmerge. It’s a library designed to provide simple controls around a merging system for basic Python data structures like dicts and lists.
It provides a few common cases for merging (like always merge + override, or raise an exception):
<span></span><span class="kn">from</span> <span class="nn">deepmerge</span> <span class="kn">import</span> <span class="n">always_merger</span><span class="p">,</span> <span class="n">merge_or_raise</span>
<span class="n">base</span> <span class="o">=</span> <span class="p">{</span>
<span class="s2">"a"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"b"</span><span class="p">],</span>
<span class="s2">"c"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s2">"nested"</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">"nested_dict"</span><span class="p">:</span> <span class="s2">"value"</span><span class="p">,</span>
<span class="s2">"nested_list"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"a"</span><span class="p">]</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">nxt</span> <span class="o">=</span> <span class="p">{</span>
<span class="s2">"new_key"</span><span class="p">:</span> <span class="s2">"new_value"</span><span class="p">,</span>
<span class="s2">"nested"</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">"nested_dict"</span><span class="p">:</span> <span class="s2">"new_value"</span><span class="p">,</span>
<span class="s2">"nested_list"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"b"</span><span class="p">],</span>
<span class="s2">"new_nested_key"</span><span class="p">:</span> <span class="s2">"value"</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">always_merge</span><span class="p">(</span><span class="n">base</span><span class="p">,</span> <span class="n">nxt</span><span class="p">)</span>
<span class="k">assert</span> <span class="n">base</span> <span class="o">==</span> <span class="p">{</span>
<span class="s2">"a"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"b"</span><span class="p">],</span>
<span class="s2">"c"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s2">"new_key"</span><span class="p">:</span> <span class="s2">"new_value"</span>
<span class="s2">"nested"</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">"nested_dict"</span><span class="p">:</span> <span class="s2">"new_value"</span><span class="p">,</span>
<span class="s2">"nested_list"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"a"</span><span class="p">,</span> <span class="s2">"b"</span><span class="p">],</span>
<span class="s2">"new_nested_key"</span><span class="p">:</span> <span class="s2">"value"</span>
<span class="p">}</span>
<span class="p">}</span>
deepmerge allows customization as well, for when you want to specify the merging strategy:
<span></span><span class="kn">from</span> <span class="nn">deepmerge</span> <span class="kn">import</span> <span class="n">Merger</span>
<span class="n">my_merger</span> <span class="o">=</span> <span class="n">Merger</span><span class="p">(</span>
<span class="c1"># pass in a list of tuples,with the</span>
<span class="c1"># strategies you are looking to apply</span>
<span class="c1"># to each type.</span>
<span class="p">[</span>
<span class="p">(</span><span class="nb">list</span><span class="p">,</span> <span class="p">[</span><span class="s2">"prepend"</span><span class="p">]),</span>
<span class="p">(</span><span class="nb">dict</span><span class="p">,</span> <span class="p">[</span><span class="s2">"merge"</span><span class="p">])</span>
<span class="p">],</span>
<span class="c1"># next, choose the fallback strategies,</span>
<span class="c1"># applied to all other types:</span>
<span class="p">[</span><span class="s2">"override"</span><span class="p">],</span>
<span class="c1"># finally, choose the strategies in</span>
<span class="c1"># the case where the types conflict:</span>
<span class="p">[</span><span class="s2">"override"</span><span class="p">]</span>
<span class="p">)</span>
<span class="n">base</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"foo"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"bar"</span><span class="p">]}</span>
<span class="nb">next</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"bar"</span><span class="p">:</span> <span class="s2">"baz"</span><span class="p">}</span>
<span class="n">my_merger</span><span class="o">.</span><span class="n">merge</span><span class="p">(</span><span class="n">base</span><span class="p">,</span> <span class="nb">next</span><span class="p">)</span>
<span class="k">assert</span> <span class="n">base</span> <span class="o">==</span> <span class="p">{</span><span class="s2">"foo"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"bar"</span><span class="p">],</span> <span class="s2">"bar"</span><span class="p">:</span> <span class="s2">"baz"</span><span class="p">}</span>
For each strategy choice, pass in a list of strings specifying built in strategies, or a function defining your own:
<span></span><span class="k">def</span> <span class="nf">merge_sets</span><span class="p">(</span><span class="n">merger</span><span class="p">,</span> <span class="n">path</span><span class="p">,</span> <span class="n">base</span><span class="p">,</span> <span class="n">nxt</span><span class="p">):</span>
<span class="n">base</span> <span class="o">|=</span> <span class="n">nxt</span>
<span class="k">return</span> <span class="n">base</span>
<span class="k">def</span> <span class="nf">merge_list</span><span class="p">(</span><span class="n">merger</span><span class="p">,</span> <span class="n">path</span><span class="p">,</span> <span class="n">base</span><span class="p">,</span> <span class="n">nxt</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">nxt</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
<span class="n">base</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">nxt</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
<span class="k">return</span> <span class="n">base</span>
<span class="k">return</span> <span class="n">Merger</span><span class="p">(</span>
<span class="p">[</span>
<span class="p">(</span><span class="nb">list</span><span class="p">,</span> <span class="n">merge_list</span><span class="p">),</span>
<span class="p">(</span><span class="nb">dict</span><span class="p">,</span> <span class="s2">"merge"</span><span class="p">),</span>
<span class="p">(</span><span class="nb">set</span><span class="p">,</span> <span class="n">merge_sets</span><span class="p">)</span>
<span class="p">],</span>
<span class="p">[],</span>
<span class="p">[],</span>
<span class="p">)</span>
That’s it! Give and try, and Pull Requests are always encouraged.