Data and System Architecture – It Really is the Small Things

It really is the small things. The small things make or break a project, a relationship, or a Mazda6. I joined a project at work that is extremely challenging requiring the migration of a massive database from MSSQL 2000 to MSSQL 2008, conversion of classic asp.net pages written in 2000 to c#.net and the migration of reports to SSRS. The project had been going for about 6 months before they brought 4 of us on to take over the technical duties. First thing out of my mouth was “What is the system architecture? which was promptly met by an incredibly awkward silence over the phone. My follow up question of “Do you have a data model?” fared much worse. Turns out they do not have a documented architecture or a data model. The database is over 500 tables, 1,000 stored procedures, and the web components in the old system number in the hundreds and the transition required an overhaul to data architecture to segregate and protect data based on its sensitivity but during that process no one noted where the new data came from and where the old data went to.

There are nine modules for this project with the goal of moving a module at least every other month. The schedule is incredibly aggressive, and already three months behind before they brought on the new team. As the people coding reports, ETL, and .net web application we absolutely must know the system architecture. How each module works together, how many servers we have, and how many environments there are is crucial to efficient and secure development. Existing modules were not communicated to the new team or documented by the old team leaving gaps in what needs to be completed and what data objects were currently in use. Many system architects often forget that the data architecture mirrors the system architecture which leads to further delays and communication issues. In this case it actually gets worse, not only is there no system or data architecture, but the data model is perpetually changing. For example, I built the .net application only to find out that the security architecture was not communicated accurately. After a rework based on a very heated meeting, another email came with more changes to the design, leading me to pointedly ask the program manager if she “was absolutely sure that this was actually how security is supposed to be or if I should expect another email in 5 hours?” The next morning the technical lead sent me a message that he was going to update the security tables to use best practices, which I had complained about two weeks ago when I spent 30 hours straight coding, and now has resulted in the destruction of the data foundation for the security architecture.

The small things. A system architecture design, even something as simple as a PowerPoint, would have set the ground work for all modules, no matter which team was performing the effort. A current and well documented data model, while most certainly not as easy as a PowerPoint would have brought the failure to implement best practices and corporate standards in thousands of columns to light sooner. Having a clear goal, architectures, and leadership that can effectively communicate that vision are small but powerful tools that lead to project success.

This entry was posted in Coding and tagged , , , , , . Bookmark the permalink.