Python – how to refactor many singletons

Having written and then refactored a python "God object", I sympathize. What I did is break the original object down into sub sections based upon methods. For instance, the original looked like this pseudo code:

method A():
    self.bla += 1

method B():
    self.bla += 1

do stuff():
    self.bla = 1
    method A()
    method B()
    print self.bla

The stuff method is a self contained "unit" of work. I migrated it out to a new class which the original instantiates. This pulled out the necessary properties as well. Some were used only by the sub class and could move straight across. Others were shared, and got moved into a shared class.

The "God object" creates a new copy of the shared class at start up, and each of the new sub classes accepts a pointer as part of their init method. For example, here's a stripped version of the mailer:

#!/usr/bin/env python
# -*- coding: ascii -*-
'''Functions for emailing with dirMon.'''

from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email.Utils import COMMASPACE, formatdate
from email import Encoders
import os
import smtplib
import datetime
import logging

class mailer:
    def __init__(self,SERVER="mail.server.com",FROM="support@server.com"):
        self.server = SERVER
        self.send_from = FROM
        self.logger = logging.getLogger('dirMon.mailer')

    def send_mail(self, send_to, subject, text, files=[]):
        assert type(send_to)==list
        assert type(files)==list
        if self.logger.isEnabledFor(logging.DEBUG):
            self.logger.debug(' '.join(("Sending email to:",' '.join(send_to))))
            self.logger.debug(' '.join(("Subject:",subject)))
            self.logger.debug(' '.join(("Text:",text)))
            self.logger.debug(' '.join(("Files:",' '.join(files))))
        msg = MIMEMultipart()
        msg['From'] = self.send_from
        msg['To'] = COMMASPACE.join(send_to)
        msg['Date'] = formatdate(localtime=True)
        msg['Subject'] = subject
        msg.attach( MIMEText(text) )
        for f in files:
            part = MIMEBase('application', "octet-stream")
            part.set_payload( open(f,"rb").read() )
            Encoders.encode_base64(part)
            part.add_header('Content-Disposition', 'attachment; filename="%s"' % os.path.basename(f))
            msg.attach(part)
        smtp = smtplib.SMTP(self.server)
        mydict = smtp.sendmail(self.send_from, send_to, msg.as_string())
        if self.logger.isEnabledFor(logging.DEBUG):
            self.logger.debug("Email Successfully Sent!")
        smtp.close()
        return mydict

It is created once and shared between the different classes that need mailing capabilities.

So for you, create a class larry with the properties and methods you need. Everywhere the client says larry = blah replace it with larryObj.larry = blah. This migrates things to sub projects without breaking the current interface.

The only other thing to do is look for "units of work". If you were going to turn part of the "God Object" into it's own method, do so. But, put the method outside it. This forces you to create an interface between the components.

Laying that groundwork allows everything else to follow it. For example, a piece of the helper object demonstrating how it interfaces with the mailer:

#!/usr/bin/env python
'''This module holds a class to spawn various subprocesses'''
import logging, os, subprocess, time, dateAdditionLib, datetime, re

class spawner:
    def __init__(self, mailer):
        self.logger = logging.getLogger('dirMon.spawner')
        self.myMailer = mailer

Concentrate on the smallest individual unit of work possible, and move it out. This is easier to do, and lets you play with the setup quickly. Don't look at properties for moving stuff, they are ancillary to the tasks being done with them in most cases. Whatever is left over after you have dealt with the methods should probably stay in the original object, since it is part of the shared state.

But, the new objects should now accept the properties they need as init variables, not touching the caller objects property. They then return any necessary values, which the caller can use to update the shared properties as necessary. This helps to decouple the objects and makes for a more robust system.

In order to define what a service's responsibilities are, you first need to define what a service is.

Service is not a canonical or generic software term. In fact, the suffix Service on a class name is a lot like the much-maligned Manager: It tells you almost nothing about what the object actually does.

In reality, what a service ought to do is highly architecture-specific:

1. In a traditional layered architecture, service is literally synonymous with business logic layer. It's the layer between UI and Data. Therefore, all business rules go into services. The data layer should only understand basic CRUD operations, and the UI layer should deal only with the mapping of presentation DTOs to and from the business objects.

2. In an RPC-style distributed architecture (SOAP, UDDI, BPEL, etc.), the service is the logical version of a physical endpoint. It is essentially a collection of operations that the maintainer wishes to provide as a public API. Various best practices guides explain that a service operation should in fact be a business-level operation and not CRUD, and I tend to agree.

   However, because routing everything through an actual remote service can seriously hurt performance, it's normally best not to have these services actually implement the business logic themselves; instead, they should wrap an "internal" set of business objects. A single service might involve one or several business objects.

3. In an MVP/MVC/MVVM/MV* architecture, services don't exist at all. Or if they do, the term is used to refer to any generic object that can be injected into a controller or view model. The business logic is in your model. If you want to create "service objects" to orchestrate complicated operations, that's seen as an implementation detail. A lot of people, sadly, implement MVC like this, but it's considered an anti-pattern (Anemic Domain Model) because the model itself does nothing, it's just a bunch of properties for the UI.

   Some people mistakenly think that taking a 100-line controller method and shoving it all into a service somehow makes for a better architecture. It really doesn't; all it does is add another, probably unnecessary layer of indirection. Practically speaking, the controller is still doing the work, it's just doing so through a poorly named "helper" object. I highly recommend Jimmy Bogard's Wicked Domain Models presentation for a clear example of how to turn an anemic domain model into a useful one. It involves careful examination of the models you're exposing and which operations are actually valid in a business context.

   For example, if your database contains Orders, and you have a column for Total Amount, your application probably shouldn't be allowed to actually change that field to an arbitrary value, because (a) it's history and (b) it's supposed to be determined by what's in the order as well as perhaps some other time-sensitive data/rules. Creating a service to manage Orders does not necessarily solve this problem, because user code can still grab the actual Order object and change the amount on it. Instead, the order itself should be responsible for ensuring that it can only be altered in safe and consistent ways.

4. In DDD, services are meant specifically for the situation when you have an operation that doesn't properly belong to any aggregate root. You have to be careful here, because often the need for a service can imply that you didn't use the correct roots. But assuming you did, a service is used to coordinate operations across multiple roots, or sometimes to handle concerns that don't involve the domain model at all (such as, perhaps, writing information to a BI/OLAP database).

   One notable aspect of the DDD service is that it is allowed to use transaction scripts. When working on large applications, you're very likely to eventually run into instances where it's just way easier to accomplish something with a T-SQL or PL/SQL procedure than it is to fuss with the domain model. This is OK, and it belongs in a Service.

   This is a radical departure from the layered-architecture definition of services. A service layer encapsulates domain objects; a DDD service encapsulates whatever isn't in the domain objects and doesn't make sense to be.

5. In a Service-Oriented Architecture, a service is considered to be the technical authority for a business capability. That means that it is the exclusive owner of a certain subset of the business data and nothing else is allowed to touch that data - not even to just read it.

   By necessity, services are actually an end-to-end proposition in an SOA. Meaning, a service isn't so much a specific component as an entire stack, and your entire application (or your entire business) is a set of these services running side-by-side with no intersection except at the messaging and UI layers. Each service has its own data, its own business rules, and its own UI. They don't need to orchestrate with each other because they are supposed to be business-aligned - and, like the business itself, each service has its own set of responsibilities and operates more or less independently of the others.

   So, by the SOA definition, every piece of business logic anywhere is contained within the service, but then again, so is the entire system. Services in an SOA can have components, and they can have endpoints, but it's fairly dangerous to call any piece of code a service because it conflicts with what the original "S" is supposed to mean.

   Since SOA is generally pretty keen on messaging, the operations that you might have packaged in a service before are generally encapsulated in handlers, but the multiplicity is different. Each handler handles one message type, one operation. It's a strict interpretation of the Single Responsibility Principle, but makes for great maintainability because every possible operation is in its own class. So you don't really need centralized business logic, because commands represents business operations rather than technical ones.

Ultimately, in any architecture you choose, there is going to be some component or layer that has most of the business logic. After all, if business logic is scattered all over the place then you just have spaghetti code. But whether or not you call that component a service, and how it's designed in terms of things like number or size of operations, depends on your architectural goals.

There's no right or wrong answer, only what applies to your situation.