September 11, 2014

Caveat Emptor - Why Paying Someone To Build Your AMS Might Cost You Your Job

Caveat Emptor - Why Paying Someone To Build Your AMS Might Cost You Your Job

It’s tempting. I know it is.

You’ve been out scouring the marketplace for software to meet the needs of your organization. None seem to be a perfect fit. Some are too expensive, carrying the right features but at too high a cost. Some are affordable, but are missing the one thing you can’t live without. Still others seem like they’d work, but you don’t have a good feeling about all the things you need that “are coming”.

But then there’s Jim. Jim’s been working with you for years now - you know Jim, you trust him. Jim seems to know everything about technology, or at least, it feels that way, since you’re not exactly a “tech guy/gal” yourself. And Jim is telling you that you don’t have to settle, you don’t have to compromise; he can build a software package for you cheaper, faster and better than any of the solutions you’ve looked at. What’s more, because he knows you so well, it’ll work just the way you need it. Oh, Jim.

It’s just so tempting.

In this post, I’m going to describe how and why trying to build your own AMS might just cost you your job - even on a road well-paved with best intentions. I’d like to tell you why by the end of this adventure you and Jim might not be on speaking terms. And I’d like to tell you from my own experience. You see, I was Jim.

Before MemberSuite was the enterprise class SaaS product it is, I started a company in 2003 while I was consulting for an association. Fresh out of MIT at 23 years old, the executive director trusted me with several hundred thousand dollars to build a web based AMS. Having cleaned up 100% of their IT problems in five weeks, I had built up a lot trust. After two painful years, I was finally able to build them a workable system and maintain my relationship with the executive director. But I also learned some critical lessons I’d like to share with you.

Building your own AMS, or paying a company to build it, is a bad idea. It will always cost you more that buying it, regardless of how inexpensive it seems. To support this assertion, I’ll go through each part in detail, using my experience to paint a picture you need to see.

The reasons why building your own AMS is a bad idea:

  1. Building a software solution is a lot more than just writing code
  2. You don’t know what you don’t know
  3. What about five years from now?
  4. Security can’t be an after thought

Building A Software Solution Is More Than Just Writing Code

Jim’s an engineer, and Jim is looking to build a tool; you’re an executive, and you’re looking to build a solution.

The difference between the two is the world. A tool is just a piece of software - lines of code, servers and web connections. But the solution is so much more. In order to solve your problems someone must sit down with your staff and understand what they need. Someone needs to unearth implicit requirements - processes that exist only out of necessity. Someone has to push back on exceptions that cost more to build than the problems they solve. All of these take intense people skills, finely honed listening skills and the ability to ask the right questions and discover implicit requirements. He has to account for the requirements that are unknown - the processes from which new requirements might emerge as the software is put into use. As someone who wrote his first line of code when he was 6 ½, who has a computer science and engineering degree from MIT, I’m going to share with you an absolute truth: most engineers don’t possess these skills.

We just don’t. It’s sad, but true. They don’t teach you requirements gathering at MIT, they teach you algorithms. There are no classes on listening either. It’s not in the curriculum. And when you’re a hammer, everything looks like a nail. That’s why teams build software, not engineers. Software design is far too important to leave to engineers.

I like to use an analogy of building a house. It’s a mystery to many of us how something so elegantly complex gets built. You get a mortgage, make a down payment, hire a builder and eight months later, you have a house. Now what if in the middle of this process, the bricklayer told you he could build the house quicker and cheaper for you if you cut out the middlemen architects and developers?

Well, let’s evaluate that for a second. Clearly, this person has built a lot of houses. And paying him, without having to go through the developer, would be cheaper. And maybe you know him - you’ve worked with him, he’s built your deck and done some landscaping for you and he’s done a good job. Maybe he even was the lead builder on your house. Perhaps this is a reasonable proposition.

But there are a lot of implicit assumptions here. For one, building a house is more than just laying bricks. You’ve got to get the necessary permits, which require relationships, knowledge and the patience to navigate municipal bureaucracy. Does the bricklayer have the people skills to forge and nurture these relationships? Someone has to architect the house and make adjustments for the type of soil in the plot of land you bought. Does the bricklayer know how to do this? Does he have an architecture background? And what about subcontractors for HVAC and electrical? Does the bricklayer know how to manage the personalities of contractors and ensure there are no cost overruns? And speaking of cost, does the bricklayer have any experience with the financial components of building a house? Materials? Security?

The dangerous assumption here, the one that may get you fired, is that the bricklayer, having been successful on your deck and your landscaping, will be successful in building your next house. Probably not so much.

Let’s move this metaphor along. Jim’s a brick layer. He can write really good code. He’s hooked up a bunch of different IT projects for you and he’s done them flawlessly. But now you’re asking him to build mission critical software that can’t go down. Are those two things really the same thing? Does building your internet event registration app make him qualified to build a sophisticated order processing and A/R application that’s mission-critical? Building a software solution is more than just laying down code.

In order to build a commercial solution that works, many different skills sets and activities must be brought to bear, just like in building a house. People that are good at bricklaying may not be good at managing budget creep for HVAC software contracts, which is why in that example you have a team of people. Software development is no different. Rarely is one person good at every aspect of developing a solution. Here’s just a few things that need to happen, correctly:

  • Requirements Gathering - Jim has to figure out what needs to be built and navigate requirements for automating manual processes that your organization has never even had to think through. This is hard.
  • User Experience - Jim has to build something people can use. How many rock star IT engineers do you think are good at design? Please, you probably wouldn’t even let Jim pick out your holiday cards. You’re going to let him design the user interface for your mission critical AMS?
  • Architecture - Someone has to think through a high level architecture that doesn’t just meet your needs now, but will in five years, when everything is changed. How does Jim know what those will be? Do you know what those will be? If you built an AMS four years ago, would you be thinking about engagement? Eight years ago, would you be thinking about social collaboration?
  • Quality Assurance - Whose testing this thing anyway? You? Okay, what about six months after it’s finished? Who is regression testing after the latest bug patch to make sure nothing that was broken isn’t anymore? Are test scripts documented? Is there automation?
  • Maintenance - Who gets called at 3 a.m. when the SQL server is crashed and Jim’s in France on vacation?
  • Technical Writing - Someone’s got to document what’s been built, right? Who does that? Who documents changes as they get made? Who documents bugs as they get fixed?

This stuff is not easy, man.

You Don’t Know What You Don’t Know

“There are known knowns, there are known unknowns, and then there are unknown unknowns.”
-Donald Rumsfeld, Former Secretary of Defense

As scary as the stuff you don’t know is, even more scary is the stuff you haven’t figured out that you don’t know. Do you know exactly how your auditors will review a new system to which everyone has access? With years of manual processes gone, your auditors will place different demands on your new custom AMS than they did on your janky paper process. Does Jim know? Do you? Do you really want to find that out on the day they show up?

Put another way, how will you explain to the board that you don’t have accurate financials because no one explained to Jim that you had to defer membership revenue until after the AMS got built? You want to explain to the board, after falling on the sword for the money to pay Jim, why your shiny new AMS failed an audit because Jim had no idea how to build a role-based security system? How no one did penetration testing on your web app when the “secure” encryption algorithm got hacked, and so 10,000 member addresses got stolen?

What else might you not encounter? How will renewals work when the system is completely online? How do you handle refunds? It was easy when things were manual, you just had accounting do it through the back end credit card processor. But what about when there’s a central system that needs to be updated?

You see, AMS vendors collectively have these problems licked (or at least in their sights), because when all of these obstacles cropped up with their first few customers, someone called them and yelled at them and demanded a solution (ask me how I know). Over time, the collective requirements of an entire industry became part of the minimum viable product (MVP) that AMS vendors could sell. AMS vendors have the ultimate imperative to fix these problems - do so or die. The inherent advantage of buying and not building is that you get the combined experience and best practices of hundreds of customers who’ve had the same challenges you have - all done for you already, for free. With Jim, you’ve got learn all these painful lessons as you go.

The crux of this issue is that you and Jim won’t see it this way. Remember hammers and nails? When these obstacles arise, you’ll feel like these problems should have been obvious and that Jim should have seen them. He’s the technology contractor, after all. He should figure these things out. But Jim will feel like he built what you told him. No one told him about these problems, and now you’re moving the goal posts on him, changing requirements. From Jim’s perspective, he’s working his butt off and it’s never enough. It never ends. He’s underappreciated. Jim might not be coming to any more family get togethers. And why? Because you wanted a solution, and Jim’s an engineer - and engineers build tools, not solutions.

What About Five Years From Now?

Jim loves you, and you love him. And when he’s on, it’s a beautiful thing, man. But Jim just had a baby. Or he just fell in love and wants to get married. Or he wants to go back to school. Or any one of the things that make professional love not enough.

Sooner or later, Jim will move on.

When Jim built your AMS, he did a ton of different jobs at once, many that you weren’t aware of. He collected requirements, he challenged assumptions, he built an object model and an architecture. He coded the product, he debugged it and he supported it.

But did he ever write any documentation? Did he keep that documentation updated as new tweaks were made? Did he document the database? The object model? The design methodology or architecture? Do you know any of those things?


When Jim’s gone, how do you replace him? Okay, you hire an engineer. But how does that engineer learn about all those assumptions? How does he learn what and why things were built? And, by the way, are you even qualified to ascertain whether or not someone is a good engineer?

Wow. Let’s just stop there. More likely than not, you’ll use False Indicators of Fit (FIT) to make a hire - James Q. went to a great school, Jane G. worked at Microsoft, Jeff L. worked at an association - and chances are, you’ll make a bad engineering hire. Trust me, I’ve personally hired dozens of engineers and there are more bad engineers out there than good ones. But they all look the same in jeans and Converse sneakers. The worst part is, when you’re wrong, you won’t find out until months later. Until things are so bad the staff and/or your members are in revolt. And who knows what permanent damage has been caused. And even when you fire them, you’ll still have to figure out how to hire another, better engineer. Perhaps one that went to a better school, worked at a bigger software company, or came from a bigger association. Setting up Epic Fail #2. Scary stuff.

When you have someone build your AMS, you need to build in a transition plan. You need to budget for follow-on help and you need to risk adjusting your plan in the likelihood that you make a hiring mistake (even software firms have bad engineering hires). If you’re thinking these costs are starting to add up, you’re catching on.

Security Can’t Be An Afterthought

One of the craziest things I heard from the association market circa 2004-2007 was that they didn’t want some other vendor hosting their AMS. In their minds, their data was more secure in their closet than in a qualified, audited software company’s data center.

Nevermind that it was hooked up to the public Internet, all ports open, prancing around like a baby seal in Shark’s Circle.

Security is hard. When you see billion dollar companies getting virtual wedgies, you have to realize this isn’t something to be taken lightly. You can’t do it by yourself. No one person is going to be able to keep up with all of the security updates, perform routine intrusion detection, detect and block DDOS attacks, build and iterate a security network and do the other 100+ tasks it takes to be high security. And that’s just infrastructure security - we haven’t even talked about the application itself. You can have fifteen different alarms at your house, but they’re useless if the ADT password gets leaked on Twitter.

So now you’re asking, “Don’t software companies have the same problem?” Yes… yes we do. But there are two important differences for us:

  1. We benefit from economies of scale
  2. If you have a security breach, you have an uncomfortable conversation with your board. If we have one, we’re all looking for jobs

Economies Of Scale

We can spend a few bucks on security consultants, penetration testing and full time staff dedicated to staying secure. We have to secure one architecture, ours, but many, many customers pay into that. So if we spend $1 million to ensure our product is secure, that’s just fine for us.

Can you spend that?

No, you just have Jim, and Jim just has the weekends. But security vulnerabilities emerge every day. Secure protocols become insecure overnight. It’s too much to ask of one person, especially when it’s not their area of expertise.

The Ultimate Imperative

They say when a man is to be hanged in a fortnight, it concentrates the mind wonderfully. Let’s face it - if we have a big enough security breach, we’ll die. Trust me when I tell you that those kind of arguments become pretty persuasive in budgeting meetings. A security breach would cause a massive blow to customer trust and confidence that we may never recover from. I tell my staff that two things will kill us: An end of days, four horsemen of the apocalypse, world’s-coming-to-an-end type corporate death. or the loss of a customer’s database through a massive security breach. With that kind of fear, money flows towards to the problem, not away from it.

Jim just can’t do this. And it’s unfair to expect him to.


You may be dead set on building this AMS yourself. And if so, then godspeed. But at least be on the lookout for the things I talked about in this article, and have a plan to correct for it. Try to utilize consultants to work with Jim to manage requirements and prevent scope creep. Take undue responsibilities off of Jim’s shoulder. And realize, regardless of whatever math you’ve done, that it will always cost you more to build than to buy. You might just be paying the piper in a different way.