On Architecting: The Qualities

The first and most important step of software construction is the identification of the qualities that the software must possess. It is a step that can sometimes be overlooked or missed in the requirements process because we are often too eager to get into the details so that we can start building. We tell ourselves that "correctness," which is the quality that dictates software must correctly do what it is designed to do, is the end-all be-all upon which everything else hangs. And while this probably the most important quality to strive for in software construction, it is only one of many qualities that may be important to your client or customer.

As an architect, I have to understand the qualities that are most important to my client so that I know what my solution constraints are. There are a million ways to design a solution to any given software problem. It is my job as an architect to pick from that universe of solutions the one solution that most appropriately produces the qualities that matter the most. This task is a lot easier to accomplish when I know what those qualities are from the onset of a project.

Here are the top ten qualities that my clients generally desire when building solutions:

Adaptability	How easy should it be to change the software in the future? If you work in an environment where change is a normal part of the business process, then adaptability is probably important to you. Having worked on a software-as-a-service eCommerce platform before, I know a thing or two about how essential adaptable software can be for a clothing manufacturer trying to keep an edge on the competition. However, I've also worked in many enterprises where adaptability isn't very important at all because of time or budgetary constraints. Whether an application should be adaptable or not may drive whether you use the Adapter or Dependency Injection (DI) design patterns to build your system.
Availability	For how long does your application need to be available for your  users? Knowing the answer to this question will determine whether you will need to built in additional redundancy to ensure that the systems doesn't fail. You will also need to think about how to perform patching and feature upgrades without taking the system down for extended periods of time.
Correctness	Does the software do what it is designed to do? This is one of the more obvious qualities that most people would identify as being a very important. Hard to argue with the fact that, if software is being built for a specific set of jobs, it should do those jobs correctly.
Maintainability	Can other engineers easily support the system and perform debugging to resolve production issues? This quality makes it pretty high on the list for most engineers. With many shops being understaffed and strained for resources, it can be very important to customers to keep any learning curve associated with supporting the system at a very low level.
Robustness	How does the system respond to bad input or unexpected errors? An application designed to be used as a quick and dirty tool for an internal engineering team might not need to be very robust. However, an educational game designed to be used by hundreds of thousands of elementary school children might need to be very, very robust. Building robust software is not easy and will require very rigorous testing and software construction standards to ensure that a system can hold up when things take a turn for the worst.
Portability	Will the system need to execute across multiple platforms and devices? With the increase of phones, tablets, and other mobile devices, we live in a world where applications must often support multiple platforms simultaneously. Making a system portable requires a great deal of planning and consideration in order to determine what versions of each platform should be supported. There are a number of tools and frameworks that can assist with making applications more portable such as Java, Mono, and Silverlight.
Reusability	Will components implemented or used by the system need to be reused in other systems? When building a suite of applications that share the same features, reusability is an extremely valuable quality since it can save both time and money. Highly reusable components also ensure that your users will have a consistent experience from one application to another.
Scalability	Can the system perform well under increased load? Some systems do not experience the same load consistently from day to day or week to week. Systems and applications supported by a professional sports league will experience peak utilization while in season, demonstrating loads that are dramatically higher than they would be during the offseason.
Security	Is the system susceptible to attacks and intrusions? System security can be very challenging to design correctly. You have to think like a potential attacker, assess threats, and design the appropriate solution. There may be standards and regulations that must be adhered to in order to maintain certification. Whatever the case, security constraints should be examined up front since doing so too late will almost always be significantly more costly.
Usability	How well can users execute the various functions of the system? Users won't care what or how well your application works if they don't like using it. Applications that suffer from poorly designed user interfaces can double or triple the amount of work users have to do in order to use the system correctly, leading to big headaches for them and higher support costs for you.

On Architecting: The Books

Times have changed since I started my career back in 2001 as a lowly Programmer Analyst intern. Back in the day, you could tell what skills a developer had by looking at the small library of books on their desk. I can still remember having to flip through pages of a 400-500 page book to figure out how to write a database connection string or how to read text from a file. It always seemed to me like I would spend half of my time thumbing through books to figure out how to do a particular task.

Nowadays, I just about Google everything (sorry Bing). Mastering the art and science of building the right search terms is just as important as knowing how to code. I still have books, of course, but rarely do I ever need to use them since everything I need is usually a web search away. Blogs, articles, and forum posts consist of the majority of the content that I use in my daily research. The kinds of resources are much easier for me to parse through instead of an exhaustive book. Plus, I can get access to most relevant content for free online, whereas you have to shell out at least $40 bucks for a decent tech book.

Yet there are a few books that I believe every architect (or developer) should own. Whether you've been coding for three months or three decades, these books serve as a solid foundation regardless of your programming language of choice (though these books mostly cater to the Microsoft programming languages).

Programming Pearls (2nd Edition)
This book has nothing to do with .NET and has been around for decades, but it is an incredible read. What I love about this book is that it was written during a time when computers were slow and were so meagerly resourced that developers had to pay attention to good algorithmic design (nowadays, developers usually get away with writing sloppy, resource-greedy code on beefy machines). Most problems that we will encounter in code will be solved by the building block solutions in this book. I highly recommend it.

Code Complete: A Practical Handbook of Software Construction, Second Edition
This is a great resource for understanding the fundamentals of constructing software applications. It covers topics like how to build high quality functions and classes, code documentation, and a garden variety of software testing strategies. It's required reading at Microsoft, though I think it should be required at any company.

Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable .NET Libraries
Whether designing a public facing API or a common framework library that can be reused across applications, this essential book will help developers understand how to properly design classes and interfaces for better usability and maintainability. This book provides great insights into the design of the .NET framework namespaces and libraries and provides extremely valuable information on the .NET data structures and interfaces.

Microsoft® Application Architecture Guide, 2nd Edition (Patterns & Practices)
Of course, I have to have a basic go-to resource for basic fundamentals. Microsoft developers will take special interest in this resource of course, but it is generic enough to satisfy the needs of architects from any language background. The full book is available online at MSDN, but you can also buy it in paperback form wherever tech books are sold.

On Architecting: The Soft Skills

As much as some of us would rather avoid this, all architects will have to employ the soft skills from time to time. Soft skills are those abilities that are employed in interpersonal relationships such as communication, leadership, conflict resolution, et cetera. Real architects will have to demonstrate mastery in one or more soft skills in order to be effective at their jobs.

I think the most important soft skill an architect can have is the ability to communicate effectively. An architect will use this skill to gather requirements from business users, relay the technical vision to developers, and provide valuable high level information on the progress of design and implementation. Good communication will involve knowing how to speak clearly and concisely to both technical and non-technical audiences. It will also require an architect to be a great listener so that they can understand problems that need to be solved and mediate between parties during conflicts. An architect that can't communicate effectively is as useful as an owner's manual written in a foreign language.

Another necessary soft skill that a good architect must master is persuasiveness. Because architects are often responsible for driving the technical vision of the team, architects must be able to convince other people to understand and follow that vision. I believe that persuasive leadership begins with humility and servitude. A good architect will demonstrate the ability to empathize with developers who are more junior and may require mentoring. They will also know how to provide good examples for the team to follow and will take time to include individuals in major decisions so that everyone feels like valued contributors.

As an architect, not everyone will agree with your point of view (regardless of how objective your analysis may be). When arguments get heated and opinions strong, you must be able understand all points of view and build consensus amongst all parties whenever possible. Facts and research should be used to argue positions effectively, but make sure to be empathetic with your audience. The trick is to avoid compromising what's right for your users while also navigating diverse motivations and political situations. An architect might even have to negotiate in order to achieve a peaceful resolution to an issue. Whatever the case, you must be able to resolve conflicts amicably in order to move the needle forward for your client or customer.

On Architecting: Introduction

Today I am starting a new series that I'd like to call "On Architecting". The goal is to share my insights as an architect with the world in hopes of helping others discover the path to becoming a good architect. I'm also hopeful that I myself will also be able to learn some new things through the process of sharing my knowledge with you.

Over the next couple of months, I plan on releasing a new article every couple of weeks that will highlight various topics regarding the practice of architecting software. The following is a list of topics we'll cover:

• The Soft Skills: The abilities architects need to lead projects effectively
• The Books: A list of the most useful and recommended books in my library
• The Qualities: Breaking down the qualities of a software system
• The Framework: How to create and use frameworks to control quality and simplify execution
• The Process: What to do during the design and implementation phase

Tick, Tick: A Software Engineer's Guide To Hustlin'

Without a doubt, I love being a Software Engineer. According to this year's U.S. News Best Jobs report (link), Software Developement is #2 on the list of the top 25 careers. It really doesn't come as any suprise to me - I've enjoyed quite a bit of success in my career and see unbounded opportunity for the future. Demand is high for guys who can do the kind of work that I do, and the pay is really good. Sure, the work can be stressful and the hours are long sometimes, but on the whole software developers have it pretty good.

While I am feeling very confident about my future in this field of discipline, I have started to become aware of the sentiment expressed in a recent Bloomberg article that proclaims software engineering is a career dead-end (link). I already feel the clock ticking, and it's definitely not ticking in my favor. I am constantly struggling to keep my skillset fresh and up-to-date while also trying to build up on my soft skills and leadership qualities. One of my greatest fears is that my skills and experience will one day prove irrelevant as the younger, faster crowd will try to swoop in and take the same opportunities that I now thrive upon. I might very well have been that kind of guy myself, though I can't imagine myself that way.

I used to think that I would be playing the catch-up game for a few years before I could settle into some sufficient level of life-long technical mastery. I'm eleven years wiser now and it turns out that I have to be just as, if not more, tenacious as I've ever been as I grow older and more experienced. There's no such thing as a tenured engineer or a made-developer. I don't even want to know what it would be like to be just a step behind the competition. And while this is proving to be quite a tiring exercise, I know that I can't settle for anything less than greatness. That is just the legacy I want to leave for my children.

So, to my fellow colleagues, I have a few points of career advice. First, be aware of the copious resources at your disposal. I can't think of any other career where it is as easy (or as cheap) to grow and learn as it is in our profession. There is an endless of free resources spread all across the web that can help you build your technical brand. Don't ever stop seeking out opportunities to be hands on with new ideas and technologies. And try visiting sites like Google Code or CodePlex to explore other peoples code to build your own programming literacy. Every little bit can be the difference in getting that promotion, higher pay, or more employment opportunites.

Secondly, know your worth. We software developers build useful stuff out of practically nothing all the time. That's a gift that not many people in this world possess, so figure out how to harness that knowledge to your advantage when negotiating your career path. We are professional problem solvers, adept thinkers, and can be just as creative as any other artist, actor, or musician out there. Don't be afraid to challenge yourself and test your limits, you'll be suprised what you can achieve.

Lastly, make sure to write down a list of your goals. Writing them down is important because it forces you to think about what you want and make it clear. It also helps you to seek out and identify the right opportunities that will help you meet those goals. Written goals will help motivate you to work for what you want, and will provide you a measure with which to determine how well you're doing as you progress in your career.

Think like a hustla, and you'll be fine. Move with urgency and always think a step ahead. Slack off, and you might find yourself out of a job real quick. It's how the game is played.

Gettin' Loopy Wtih Table Variables

It's midnight on a Wednesday and I can't seem to get to sleep. So what do I want to do? Code.

So let us engage in a little SQL exercise. Suppose that we need to copy data from one table to another. A task like this, on its own, can be tackled with a single INSERT...SELECT query. For example:

insert into Employee(firstName, lastName,...,postalCode)
select firstName, lastName,...,postalCode
from EmployeeImport

For the sake of discussion, we'll refer to EmployeesImport as the table we need to copy rows from and Employees as their destination. Imagine that, instead of just copying the rows from EmployeeImport into Employees, we also need to generate a special hash value generated from the content of each row that's inserted into the Employees table. Top that off with the fact that this special index value can only be generated via a stored procedure and, voilà, you've now got an interesting situation on your hands. We can't use the simple INSERT...SELECT query like we did previously because. as we all well know, you can't execute stored procedures inline with SELECT queries. That means that we are going to need to iterate through each row of the EmployeesImport table and insert them one at a time into the Employees table. Here's one way I've seen this task accomplished:

declare @firstName varchar(50),
	@lastName varchar(50),
	@birthDt datetime,
	@addressLine1 varchar(100),
	@city varchar(80),
	@province varchar(80),
	@country varchar(80),
	@postalCode varchar(10),
	@hashKey char(10)


declare @err bit
set @err=0

begin transaction importEmployees
	-- Setup the cursor
	declare cur_employee cursor for
		select * from EmployeesImport
	open cur_employee
	fetch next from cur_employee
		into @firstName,@lastName,@birthDt,@addressLine1,@city,@province,@country,@postalCode
	-- Main cursor loop
	while(@@FETCH_STATUS=0) begin
		-- Generate the hash
		exec GetEmployeeHash
			@hashKey output
		,	@lastName
		,	@firstName
		,	@postalCode
		,	@birthDt
		-- Insert record into Employees table
		insert into Employees(EmployeeID,FirstName,LastName,BirthDt,AddressLine1,City,Province,Country,PostalCode)
			select @hashKey,@firstName,@lastName,@birthDt,@addressLine1,@city,@province,@country,@postalCode
		if(@@ROWCOUNT=0) begin set @err=1 goto RollBackTransaction end
		-- Retrieve the next record using the cursor
		fetch next from cur_employee
			into @firstName,@lastName,@birthDt,@addressLine1,@city,@province,@country,@postalCode	
	end
	-- Cleanup
	close cur_employee
	deallocate cur_employee
	
if(@err=0) commit transaction importEmployees
RollBackTransaction:
if(@err>1) rollback transaction importEmployees

Now there are a number of issues that I have with this code. For one, lines 21-22 show that the cur_employee cursor is fetching rows directly from the EmployeesImport table and not from a temporary table. This means that it will create locks against the table that can cause blocking issues for other queries that might be working against the same table. This can lead to performance degradation in high-volume environments. Not only that, but developers using cursors must always be mindful to de-allocate and close  any they've declare to avoid memory leaks. Personally, I prefer to avoid cursors at all costs as I've found them to be slow, tedious, and high on overhead.

So how would I solve the problem? Like this:

declare @err bit
set @err=0

declare @EmployeesTemp as table (
	RowID int identity(1,1),
	FirstName varchar(50),
	LastName varchar(50),
	BirthDt datetime,
	AddressLine1 varchar(100),
	City varchar(80),
	province varchar(80),
	country varchar(80),
	postalCode varchar(10)
)

insert into @EmployeesTemp(FirstName,LastName,BirthDt,AddressLine1,City,Province,Country,PostalCode)
select * from EmployeesImport

declare @firstName varchar(50),
	@lastName varchar(50),
	@birthDt datetime,
	@addressLine1 varchar(100),
	@city varchar(80),
	@province varchar(80),
	@country varchar(80),
	@postalCode varchar(10),
	@hashKey char(10)
	
declare @total int
declare @rowID int
select @total=count(*) from @EmployeesTemp
set @rowID=1 

begin transaction importEmployees

	while (@rowID<=@total) begin
		select
			@firstName=FirstName
		,	@lastName=LastName
		,	@birthDt=birthDt
		,	@addressLine1=AddressLine1
		,	@city=City
		,	@province=Province
		,	@country=Country
		,	@postalCode=PostalCode
		from @EmployeesTemp
		where RowID=@rowID
		-- Generate the hash
		exec GetEmployeeHash
			@hashKey output
		,	@lastName
		,	@firstName
		,	@birthDt
		,	@postalCode
		-- Insert record into Employee table
		insert into Employees(EmployeeID,FirstName,LastName,BirthDt,AddressLine1,City,Province,Country,PostalCode)
			select @hashKey,@firstName,@lastName,@birthDt,@addressLine1,@city,@province,@country,@postalCode
		if(@@ROWCOUNT=0) begin set @err=1 goto RollBackTransaction end
		-- Increment the loop counter
		set @rowID=@rowID+1
	end

if(@err=0) commit transaction importEmployees
RollBackTransaction:
if(@err>1) rollback transaction importEmployees

My solution presents a number of improvements over the previous. Instead of repeatedly querying against the source table, the needed data is cached in a table variable. I love table variables because, unlike temp tables, they are scoped to the execution of the stored procedure and are disposed automatically once execution is complete. While I'm using the table variable, I also don't have to worry about locks against the source table. Temp tables also work for the same purpose (for more on the difference between temp tables and table variables, see this excellent Stack Overflow post)

Another cool feature of my solution is that I don't need to maintain cursors in order to move sequentially though my data set. Because I used an identity column in the temp table, I can just increment my counter so that I know the next rowID I need to query. Overall, I find it much simpler to use table variables effectively than to remember the complexities of managing cursors correctly.

On the whole, 95% of the work that you'll ever need to do in SQL can and should be done using set-based queries. That's the whole point of using relational databases in the first place. But for the small minority of tasks that require row-by-row sequential processing, table variables are definitely the better alternative to cursors.

For a great article on cursor performance, check out the article "Performance Tuning SQL Server Cursors" by Brad McGee.

Introducing the "Spittin' Bits" Blog

So what is this blog all about? As they say where I come from, “lemme break it down fo ya”.

» The Name

The name of this blog, Spittin’ Bits, is what you get when you mix a slang word with software engineering terminology. On the one hand, there’s spittin’. In hip hop vernacular, spittin’ is what a rapper does when spontaneously rhyming on top of a melody or beat. If you have ever seen the movie “8 MILE” featuring rap artist Eminem, then you know how this works. A good rapper can spit’ a verse to a random beat in a matter of seconds. He will create artfully crafted phrases and clever puns out of thin air, drawing upon stories of past life experiences. Sometimes making you laugh and other times making you think, a good rap verse can educate the mind and move the soul with amazing efficiency.

On the other hand, bits refers to the sequences of binary numbers, represented by 1’s and 0’s, that are used by computers to do whatever humans tell them to do. Every line of code in any piece of software I write boils down to bits. It is literally the only language that computers really “understand”. I often hear it said that there are only 10 types of people in the world: those who understand binary, and those who don’t. Anybody who wants to be a good programmer has to know how to spin those 1's and 0's!

So when I talk high-tech, I'm spittin' bits. When I'm writing code, I'm spittin' bits. I'm steppin' up to the metaphorical mic and makin' magic happen. That's how I get it done.

» The Purpose

Spittin’ Bits is about telling my story as a software engineer on my own terms and in my own words. While it’s true that I want this to develop into a solid programming blog, it’s not just about programming. Sure, this should be a place where you can find great commentary on patterns, practices, and good software design principles. And yes, I want to share the lessons learned over my career so that I can benefit someone else. My greatest desire is to show you how I have used all aspects of my life to transform from just another kid with a rough upbringing, to an accomplished software engineer. And I'm going to do it in a way that makes my story accessible to as many people as can be reached, whether geek or ghetto.

As an engineer, I will talk about writing efficient and robust code. As an ear-trained musician, I will show you how to recognize design patterns and teach you about how I use improvisation in my work. As a church boy, I’ll preach on developing good character and use examples from my upbringing to demonstrate the importance of ethical decision making. As a guy that grew up in a rough neighborhood, I’ll talk about how to be self-motivated, find positive influences, and about how to work with difficult people. And as a father and husband, I will share with you how I attempt to balance all the things that are important in my life beyond what I do between 9 and 5.


» The Audience

If you’re a tech head like me, then this blog is definitely for you. If you’re not, that’s OK. There will be plenty for you to enjoy as well. If you’re a teenager or a college student thinking about a career in software engineering, I am especially talking to you. In fact, anyone who doesn’t mind thought provoking conversation can find something on this blog that will resonate with them.

My name is Anthony Mays, and I hope you enjoy the Spittin’ Bits blog. And don’t worry, imma keep it real and represent ALL the way.

So wassup ya’ll, let’s get it!