Aiohttp vs Multithreaded Flask for High I/O Applications

Date Tags python

Over the past year, my team has been making the transition from Flask to aiohttp. We're making this transition because of a lot of the situations where non-blocking I/O theoretically scales better:

  • large numbers of simultaneous connections
  • remote http requests with long response times

There is agreement that ...

more ...

MongoDB Streaming Pattern, Allowing for Batching

An interesting problem arose at work today, regarding how to build an aggregate of changes to a MongoDB collection.

A more general version of the problem is:

  1. you have a document which has multiple buckets it could belong to. Say, an animal which an arbitrary set of tags, such as ...

more ...

deepmerge: deep merge dictionaries, lists and more in Python

Date Tags python

Introducing deepmerge. It's a library designed to provide simple controls around a merging system for basic Python data structures like dicts and lists.

It provides a few common cases for merging (like always merge + override, or raise an exception):

from deepmerge import always_merger, merge_or_raise

base = {
    "a": ["b"],
    "c": 1 ...
more ...

The CID Pattern: a strategy to keep your web service code clean

Date Tags design

The Problem

Long term maintenance of a web application, will, at some point, require changes. Code grows with the functionality it serves, and an increase in functionality is inevitable.

It is impossible to foresee what sort of changes are required, but there are changes that are common and are commonly ...

more ...

KeyError in self._handlers: a journey deep into Tornado's internals

Date Tags python

If you've worked with tornado, you may have encountered a traceback of a somewhat bewildering error:

Traceback (most recent call last):
    File "/usr/local/lib/python2.7/site-packages/tornado/ioloop.py", line 832, in start
fd_obj, handler_func = self._handlers[fd]
KeyError: 16

A few other people have been confused ...

more ...


Global logging with flask

As of December 2016, Flask has a built-in logger that it instantiates for you. Unfortunately, this misses the errors and other log messages in other libraries that may also be valuable.

It would be nice to have a single logger, one that captures BOTH library AND app logs. For those ...

more ...

Hierarchal Naming

Date Tags design

One of the most interesting artifacts of most programming languages using English conventions is variable naming. Today I contend that:

English Grammar is a Terrible Programming Default

Consider how you would specify that a room is for guests in English, or a car is designed to be sporty. In both ...

more ...

Test Classes Don't Work

Date Tags testing

Test Classes don't work as a test structure.

It's worth clarifying what I mean by the test class. I'm speaking specifically about the following structure of an test:

  • having a test class, that contains the setup and teardown method for test fixtures
  • putting multiple tests in that ...
more ...

How I Design Test Suites

Date Tags testing

At Zillow, I've done a lot of work on the design and development of the test infrastructure we use for full-stack tests. It's always fun to watch your tool become popular, but even more interesting is the discussions around test suite design that come with it.

Many discussions ...

more ...

About Yusuke Tsutsumi
I work at Zillow. I focus on tools and services for developer productivity, including build and testing.

My other interests include programming language design, game development, and learning languages (the non-programming ones).