That Rotting Design | JavalobbyWe’ve all experienced that sinking feeling when maintaining a piece of crappy software. Has my change broken the system in some unintended way? What is the ramification of my change on other parts of the system? If you’re lucky, and the system has a robust suite of unit tests, they can offer some support in proving your work. In practice, however, few systems have thorough automated test coverage. Mostly we’re alone, left to verify our changes as best as possible. We might privately criticize the original developers for creating such garbage. It certainly lends a plausble excuse in explaining why the maintenance effort is so costly or time-consuming. Or it might serve as the basis upon which we recommend a re-write. But mostly, we should wonder how it happened.
For sure, most software doesn’t start out this way. Most software starts out clean, with a clear design strategy. But as the system grows over time, strange things begin to happen. Business rules change. Deadline pressures mount. Test coverage slips. Refactoring is a forgotten luxury. And the inherent flaws present in every initial design begin to surface. Reality has proven that few enterprise development teams have the time or resources to fix a broken design. More often, we are left to work within the constraints of the original design. As change continues, our compromises exacerbate the problem. The consequence of rotting design is seen throughout the enterprise on a daily basis. Most apparent is the affect on software maintenance. But rotting design leads to buggy software and performance degradation, as well. Over time, at least a portion of every enterprise software system experiences the problem of rotting design. A quote from Brook’s sums it well:
"All repairs tend to destroy the structure, to increase the entropy and disorder of the system. Less and less effort is spent on fixing the original design flaws; more and more is spent on fixing flaws introduced by earlier fixes. As time passes, the system becomes less and less well-ordered. Sooner or later the fixing ceases to gain any ground. Each forward step is matched by a backward one. Although in principle usable forever, the system has worn out as a base for progress."
The most obvious question is, “How do we prevent rotting design?” Unfortunately, rotting design is not preventable, only reducable. Of the past ten years, the design patterns movement has provided insight to the qualities of good design. Dissecting design patterns reveals many important design principles that contribute to more resilient software design. Favor object composition over class inheritance, and program to an interface, not an implementation, are two examples. Alone however, design patterns that emphasize class structure are not enough to help reduce rotting design.
Reducing Rot
Most patterns emphasize class design, and present techniques that can be used in specific contexts to minimize dependencies between classes. Teasing apart the underlying goal of most patterns shows us that each aim to manage the dependencies between classes through abstract coupling. Conceptually, classes with the fewest dependencies are highly reusable, extensible, and testable. The greatest influence in reducing design rot is minimizing unnecessary dependencies. Yet enterprise development involves creating many more entities beyond only classes. Teams must define the package structure in which those classes live, and the module structure in which they are deployed. Increasing the survivability of your design involves managing dependencies between all software entities - classes, packages, and modules.
But if minimal dependencies were the only traits of great design, developers would lean towards creating very heavy, self-contained software entities with a rich API. While these entities might have minimal dependencies, extreme attempts to minimize dependencies results in excessive redundancy across entities with each providing its own built-in implementation of common behavior. Ironically, avoiding redundant implementations, thereby maximizing reuse, requires that we delegate to external entities, increasing dependencies. Attempts to maximize reuse results in excessive dependencies and attempts to minimize dependencies results in excessive redundancy. Neither is ideal, and a gentle balance must be sought when defining the behavior, or granularity, of all software entities - classes, packages, and modules. For more on the use/reuse paradox, see Reuse: Is the Dream Dead?
Software design is in a constant quandary. Any single element key to crafting great designs, if taken to its individual extreme, results in directly the opposite - a brittle design. The essential complexity surrounding design is different for every software development effort. The ideal design for a software system is always the product of it’s current set of behavioral specifications. As behavior changes, so too must the granularity of the software entities and the dependencies between them. The most successful designs are not characterized by their initial brilliance, but instead through their ability to withstand the test, and evolve over the course, of time. As the complexity of software design is an essential complexity surrounding software development, our hopes lie with technologies and principles that help increase the ability of your design to survive. Such is the reason why agile architecture is so important.
A Promising Future
I’m hopeful that all software developers have experienced the pleasure of a design that, through the course of time, has withstood the test of time. Unfortunately, many enterprise development teams have too few of these experiences. Likewise, few enterprise development teams devote adequate effort to package and module design. It’s unreasonable to believe that even the most flexible class structure can survive should the higher level software entities containing those classes not exhibit similarily flexible qualities. The problems are rampant. Increased dependencies between packages and modules inhibit reusability, hinder maintenance, prevent extensibility, restrict testability, and limit a developer’s ability to understand the ramification of change.
Services offer some promise to remedy our failures with object-oriented development. Yet, while services may offer tangible business value, within each awaits a rotting design. There exists a world between class design and web services that deserves more exploration, and as an industry, we are beginning to notice. OSGi is a proven module system for the Java platform, while Jigsaw aims to modularize the JDK. JSR-294 aims to improve modularity on the Java platform. While some friction might exist between the constituencies involved, it’s only because they too recognize that something has been missing, and are passionate about fixing the problem. Of course, it doesn’t stop there. A plethora of application servers and tools are also including support for modularity using OSGi, which has grown into the defacto standard module system on the Java platform.
All aim to help manage the complexity, from design through deployment, of enterprise software development. With each, new practices, heuristics, and patterns will emerge that increase the ability of a design to grow and adapt.
Coding: An Outside in Observation | Javalobbyit's important to drive the design of an API based on the way that it will be used by its clients:
"A great way to get usable APIs is to let the customer (namely, the caller) write the function signature, and to give that signature to a programmer to implement. This step alone eliminates at least half of poor APIs: too often, the implementers of APIs never use their own creations, with disastrous consequences for usability"
This is similar to Michael Feathers' Golden Rule of API Design:
"It's not enough to write tests for an API you develop, you have to write unit tests for code that uses your API.
When you do, you learn first-hand the hurdles that your users will have to overcome when they try to test their code independently"
When we don't do this we start guessing how we think things might be used and we end up with generic solutions which solve many potential use cases in an average to poor way and none of them in a good way.
Why Is Agile So Hard To Sell? | Agile ZoneNow we have pockets of agile within large organizations... sometimes we might even have agile across entire large enterprises. We are exploring agile methods and trying to see if they can deliver on the small team promise... but in the large. The main difference with these larger organizations is that value isn't the same as it is in a small team or a small company. There is not a direct correlation between team performance and business outcomes... there is not a direct connection between what the team delivers and what we can sell to our customers. It takes the output of too many small teams coming together to deliver anything of value.
Coding: Naming | Javalobby...the importance of creating a shared understanding of the different types/objects in the systems that we build.
Names are there to make it easier for us to talk and reason about
code – if they're not doing that then we need to change them until they
do.
"Agile is treating the symptoms, not the disease" | Agile Zonehow does an agile process reduce the complexity load? And the answer, of course, is that it doesn't—it simply tries to muddle through as best it can, by doing all of the things that developers need to be doing: gathering as much feedback from every corner of their world as they can, through tests, customer interaction, and frequent releases. All of which is good. I'm not here to suggest that we should all give up agile and immediately go back to waterfall and Big Design Up Front. Anybody who uses Billy's quote as a sound bite to suggest that is a subversive and a terrorist and should have their arguments refuted with extreme prejudice.
But agile is not going to reduce the technology complexity load, which is the root cause of the problem.
Tips November 2009http://www.hpts.ws/session9/shoup.pdf
eBay?s Challenges and Lessons (Page last updated October 2009, Added 2009-11-30, Author Randy Shoup, Publisher eBay). Tips:
* Partition everything so that you can scale it as necessary.
* Use asynchrony everywhere, connected through queues.
* Use recovery and reconciliation rather than synchronous transactions.
* Automate everything - components should automatically adjust and the system should learn and improve itself.
* Monitor everything, providing service failover for parts that fail.
* Avoid distributed transactions.
* Design for extensibility, deploy changes incrementally.
* Minimize and control dependencies, use abstract interfaces and virtualization, components should have an SLA.
* Save all (monitoring) data - this provides optimization opportunities, predictions, recommendations.
* Maximize the utilization of every resource.
http://java.dzone.com/news/performance-considerations
Performance Considerations in Distributed Applications (Page last updated September 2009, Added 2009-11-30, Author Alois Reitbauer, Publisher JavaLobby). Tips:
* Many distributed application performance issues stem inefficient serialization of objects (to send across the wire).
* Connection pooling can be an important performance improvement for many types of distributed communications (not just database communications).
* Asynchronous communication is more difficult to design and implement, but is usually more efficient for distributed communications.
* Making too many service calls from the client is a frequent source of performance problems.
* Using a more efficient communication protocol can make a huge difference to overall performance (Wrong Protocol Anti-pattern) - the overhead of SOAP compared to RMI-JRMP is significant. An inefficient protocol can cause a performance degradation by a factor of ten and significantly higher CPU and memory consumption.
* When building distributed applications, make as few remote calls as possible (Chatty Application Anti-pattern).
* Keep message size as small as possible, for example the overhead of serialisation is proportional to the size of the transferred objects (Big Messages Anti-pattern).
* With a high number of deployment units, the interaction of services become more and more difficult to understand. This can lead to a situation where two heavily-interacting services are deployed on different hardware resulting in a high number of remote calls. You need to analyze the frequency of interactions as well as data volume to structure deployment accordingly and avoid inefficent deployment configurations (Distributed Deployment Anti-pattern).
How To Be A Successful Developer | Javalobby * Always strive to improve yourself and learn more.
* Share information freely with others — be generous.
* Focus on developing good working relationships with your coworkers, both technical staff and others.
* Effective communication, both written and spoken is crucial.
* Get involved in open source.
* Be precise.
* Deliver on commitments, or if you need to renegotiate your commitments.
* In everything that you do, do it with integrity.
Almost none of these have anything to do with knowledge of technology. I believe that social aspects have far more impact on success than anything else. Of course being knowledgeable helps too, however what's more important than knowing a specific technology is being able to pick up the knowledge that you need, when you need it.
A few things that I missed in my response because I take them for granted:
* Have passion for what you do.
* Strive for excellence.
* Avoid being self-righteous.
DavMail POP/IMAP/SMTP/CalDav/LDAP Exchange GatewayEver wanted to get rid of Outlook ? DavMail is a POP/IMAP/SMTP/Caldav/LDAP exchange gateway allowing users to use any mail/calendar client (e.g. Thunderbird with Lightning or Apple iCal) with an Exchange server, even from the internet or behind a firewall through Outlook Web Access. DavMail now includes an LDAP gateway to Exchange global address book to allow recipient address completion in mail compoze window and full calendar support with attendees free/busy display.
DavMail Architecture
The main goal of DavMail is to provide standard compliant protocols in front of proprietary Exchange. This means LDAP for address book, SMTP to send messages, IMAP to browse messages on the server in any folder, POP to retrieve inbox messages only and Caldav for calendar support. Thus any standard compliant client can be used with Microsoft Exchange.
DavMail gateway is implemented in java and should run on any platform. Releases are tested on Windows, Linux (Ubuntu) and Mac OSX. Tray does not work on MacOS and is replaced with a full frame. Tested successfully with the Iphone (gateway running on a server)
