Collaborative Consulting Blog

Oatmeal in the Gas Tank

2009-08-06T00:00:00-05:00

In the old days, I used to tune my car to get the most performance and efficiency by adjusting the spark plug gap, adjusting the distributor point gaps, playing around with the ignition timing, and a slew of other such things. Alas, those days are gone. Now we have more sophisticated engines equipped with ECU’s, or Engine Control Units. ECU’s make most of the automotive tools in my garage obsolete. Oh, don’t get me wrong – there is still plenty you can do to tune up modern cars to increase performance and efficiency. However, I don’t happen to own a lambda probe or a dynamometer, nor do I have the skill required to program an ECU. Yup, you heard me right – ECU’s are programmable, allowing you to map such things as the ignition, water temperature correction valves, transient fueling (used to adjust the amount of fuel to use when smashing down on the gas petal), and the closed loop lambda settings for modifying the combustion rates.

Fortunately, software performance tuning hasn’t gotten that complex – in most cases the same old tricks still work. Over the past year I have found myself doing a lot of performance-related gigs. Most of the time I try to attack performance-related issues through configuration changes, database settings, and external tuning parameters. While this sometimes produces the desired result, in most cases it simply isn’t enough. Like automobiles, you can program and tune the ECU for days, and still the car runs sluggish (or not at all). Why? Oh that’s easy – because there is oatmeal in the gas tank!

I find myself using that phrase more and more these days. You can only go so far with configuration and parameter tuning. Most of the time, however, I find myself opening the hood and tearing the application apart, and sure enough, I find oatmeal.

Configuration and tuning parameters are easy to change – they generally do not require a re-deployment of the application, and they can sometimes make all the difference in the world – big bang, small bucks. Once you start changing code, however, now you are talking about application deployment, unit testing, functional testing, regression testing, etc., etc., etc. This equates to longer and more expensive gigs. However, this is reality. If you have oatmeal in the gas tank, tuning won’t make a bit of difference. Bad code is bad code, regardless of tuning parameters, and no amount of configuration changes is going to fix it.

What I am talking about poorly written SQL statements, bad transaction management, poor object use, inefficient coding practices, poor architecture or design, and the worse one, something I like to call “single user” code – code that works functionally, but only for a single user. In most cases, these are the real culprits, not configuration settings or tuning parameters.

The bottom line is all about expectations - expectations that turning a few dials and changing a few settings will solve all of the performance and scalability issues of an application. Sure, I will always try to do those first, because it sometimes does work, and it’s fast and cheap. But I always walk into an engagement with the foresight that I will be reviewing, profiling, and modifying code – and in my experiences I have seldom walked away from a performance gig without having to modify at least some code.

So, the next time you have some performance-related issues, don’t forget to take into account the oatmeal in the gas tank.

Do you see that road over there? Well, don’t take that … it’ll do you no good.

2009-07-01T00:00:00-05:00

With that quotable movie line (try it with an Irish brogue!), Molouney, the train guard, gives Sean Thornton (John Wayne) a taste of what awaits him in Innisfree, Ireland. The Quiet Man (1952) is arguably director John Ford’s greatest movie triumph, although this is decidedly a US-centric point of view as the film is marred by a patronizing view of a “nostalgic” Ireland while unfairly depicting some unflattering Irish stereotypes. The movie unfolds with Sean (Wayne) trying to reclaim his ancestral home, escape his past and ultimately settle down, get married, and make the rolling hills of Innisfree his home. This blog entry isn't about the movie; rather it's about the signals (spoken and unspoken) that Sean (Wayne) must decipher.

Signals. Dictionary.com defines “signal (n.)” as anything that serves to indicate, warn, direct or command". Sean’s first step in Innisfree is met with a curious signal that possibly means: you’re not welcomed there, that road is a dead-end, welcome to Innisfree (viz. unlikely!), or something else open to interpretation. Lately, it seems, the ‘net is a-buzz with a singular application: Twitter—well, “lately” is what it seems; it’s been around since 2006. Twitter’s aim is simple, answer the question: “What are you doing?” How you answer (tweet) depends on time, location, situation, mood, or any particular bias (personal, athletic, regional, organizational, etc.) that you wish to inject into the response. The response is a signal. It indicates to your followers (folks listening to your signals) that you are communicating something; what that “something” is, and if it’s of interest, is relevant only to those who follow you, right? Yes and no.

Hmm, what? Twitter is the latest in a line of online collaboration and communication tools whose aim is to keep contacts informed. On a personal level, it can help organize your day and inform those around you what is going on with you. On an organizational level Twitter can be used in various powerful and creative ways, such as: a “real-time” pulse used to manage a customer base, keeping track of service activity, managing a project, marketing a product, or anything else that allows an organization to stay in touch with its constituents throughout the day. At first Twitter does look like other web applications used to socialize and fraternize with friends, certainly not a tool for work or client engagement. Yes, Twitter allows you to do that.

Fundamentally, however, Twitter is a communication platform that facilitates collaboration based on relationships. The velocity of today’s business environment makes collaboration an imperative and a top strategic objective for all enterprises. Each business function in an enterprise must have access to the information needed to enhance customer service, speed problem resolution time, engage clients in an open fashion and generally communicate in the most expedient fashion possible. Twitter allows you to do this. Quickly. Openly. Succinctly—Twitter only allows 140-character messages.

I recently attended the Enterprise 2.0 conference in Boston (http://www.e2conf.com); I came away with a renewed sense that forward-thinking enterprises wishing to accelerate their responsiveness should be paying attention to what is taking place with social networking (SN) and interaction tools (e.g., Facebook, Twitter.) To change direction rapidly and respond to shifting business opportunities (signals), enterprises will require new levels of business interaction and technical agility.

As enterprise architects, we are always on the lookout for emerging patterns and trends that can help accelerate enterprise responsiveness. Today, the most common form of interaction is through a combination of customer support centers and software built along the lines of consumer/provider patterns. These have provided the foundation for all modern enterprises; however, there are new patterns emerging – how can enterprises use social interaction architectures and harness the necessary contextual usage patterns critical in getting closer to the customer and protecting the brand (listening to the signals)?

Enterprise 2.0 is all about facilitating communication. The enterprise that can make the switch from communicating in a hierarchical fashion (IT-driven, no user control) into managing their communications in a collaborative, distributed, transparent, and socially driven (people-driven and controlled) fashion will reap the benefits of Enterprise 2.0—namely, getting the right user/customer feedback to the appropriate enterprise owner, react to user/customer demands quickly, and create efficiencies in information management that lead to better decisions.

So? Do you see that road over there? Well, it’ll do you no good if you don’t take it…

Resource Calendars – A simple tool to aid in realistic project schedules and timeline

2009-06-22T00:00:00-05:00

Since I am a handy man and I enjoy fixing things around the house, I am all about adding tools to my toolbox. The same holds true when it comes to managing projects. I am constantly adding or renewing my PM tools.

I find that one of the easiest and most effective tools in my Project Management toolbox is the Resource calendar. All too often, we PMs find ourselves in the midst of a project when team members are not available due to out of office events, vacation and holidays which could impact a project schedule. One way to reduce surprises is to create a resource calendar. While there are many manual and automated tools in the market, the simplest way to do this is to create a calendar template (read-only), typically three months out, at a minimum, using an MS word document. Included in the template should be a legend denoting planned time off such as V=vacation, O=out of office etc. Once created, request the team members to submit their planned time out and list each individual by name in the appropriate day with the legend value. This can then be distributed to the group giving everyone a view of when and who is available. The PM can then incorporate this information into the different Project Management tools reflecting a more realistic schedule and timeline.

How many toolboxes do you carry?

2009-06-15T00:00:00-05:00

One could argue that project management is a bizarre way to make a living. Where else does someone have the ultimate responsibility of delivering a product, yet their true power so limited and their success or failure is so dependent upon the performance of others. Most companies have Project Management Offices (PMOs) whose dictates and procedures must be adhered to across the enterprise and Project Managers (PMs) must work within the confines of its directives. The manner in which projects are staffed differs from company to company and the ability of PMs to ‘pencil in their starting lineups’ varies greatly from company to company. If lucky, a PM can request that certain individuals be assigned to their team, but in many companies teams are assigned with little or no input from the project manager.

I find that successful project managers understand that their success is truly dependent on the success of those assigned to their team. Their goal is to have everyone on their team perform to the best of their capabilities. While most professionals are self-motivated, there are those whose performance is impacted by the manner in which they are managed. Some resources need a more hands-on approach while others are more self-sufficient and independent. A management style that maximizes the performance of one individual will, in all likelihood, not have the same effect on another. Maximizing the performance of these resources can mean the difference between a triumphant endeavor and one that is not.

I believe that while most project managers bring a toolbox of assorted “hard” project management tools to the job - such as work breakdown structures, project planning and scheduling software packages, and risk assessment procedures, just to name a few; only the really successful PMs have a second toolbox. This second toolbox which is packed with different “soft” personal management styles, tools and techniques that help them navigate through “the people” dimension.

These successful managers are adept at using this second “soft” toolbox. They can quickly assess the needs of each team member and when necessary, implement a style that motivates an individual to perform at their peak. The focus is as much on the team as it is on the individual team member, and adapting their management style to achieve hard results.

Given the benefits associated with applying different styles of management to different people, it is time for all of us PMs to care for and build up toolboxes especially our “soft toolbox”.

How integrated is BI with Excel and could the Kilimanjaro release of SQL Server fill a gap in the marketplace?

2009-06-15T00:00:00-05:00

Recently I was at a client, who was very decentralized from a report creation perspective - a series of power users resided in all functions, creating content and distributing it out across the organization. This client had an existing BI tool - SPSS Showcase, running on an I-Series IBM environment, which the power users liked. Based on cost, the client had already made a decision to go with the complete Microsoft 2008 solution, including its BI toolset.

The existing Showcase Query tool allowed these power users to query the existing data warehouse via a wizard. Key features that the Showcase Query tool had were:

The ability to add tables - with the joins being detected automatically. These default joins could be changed so that outer joins could also be made.
The ability to create derived fields using case logic through an expression builder. A key feature of this expression builder is that it allowed users the ability to 'see' a list of existing values that could be selected for any given field - especially useful if the values in the data warehouse were not cleansed (as was the case) and had leading or trailing spaces in them.
The ability to create prompts.
The ability to embed these queries within existing Excel spreadsheets. Excel was the primary method of reporting

The client knew that there was a gap in the MS BI tool stack to offer these capabilities and needed some help. Apart from budget, which was a big constraint, and the fact that they needed a desktop tool (they were thin on the ground as far as IT resources go), when I started to look out in the marketplace at potential vendors to fit the bill, I was surprised by what I found.

Most vendors (regardless of budget), either did not allow a complex derived field to be created (that used case logic) during query creation, or did not have an ability for users to select from a list of existing values.
Most of the vendors had the ability to create Excel as an output format from the reporting tool, but not to be embedded within an existing Excel Spreadsheet - a feature that was very important to this client as several power users had created 'applications' using VBA.

Microsoft offered the following capabilities:

Power users could query the data warehouse via SQL Management studio. The Query Editor gave them a GUI capability that allowed them to add tables, detect and change joins and create 'group by' queries, but it lacked any assistance in creating expressions of showing lists of values.
As far as Excel integration capabilities were concerned, these power users could copy and paste query results from SQL Management studio or could use Excel's ability to connect to a data source. This latter method would only work against a star schema if either MS Query was used (very unfriendly) or a 'view' was created which did all the logic and joins for them.

Seeing this dilemma made me realize that a big gap exists out there in the BI space. For most BI tools, reports have to be created in another tool first, and then either embedded into Excel or saved in Excel format. No tool really offers the capability from within Excel to create a query. The nearest out-of-box tool I came across was free Excel add-in tool from www.sqldrill.com. This gave the ability from within excel to access a user interface (that was like the query editor in SQL Managment Studio), create a query and have the results delivered into Excel. Of course you can use code something yourself and use Microsoft ADO through VBA.

In my research, I came across the Microsoft webcast that gave a peak into some of the potential functionality that the new Kilimanjaro release of SQL Server would deliver around the Q2 timeframe in 2010. This webcast talked about having the potential ability to query a data warehouse directly from within Excel and deliver back up to 100 million rows! I think that if Microsoft gets this capability right, they could have a tool that would be way ahead of the competition. Like it or not, Excel is one of the most widely used reporting tools in all organizations. Having a capability to create relational queries from within Excel itself, and thus not having to use or be trained on another tool would be nirvana to many users.

When to and when not to use Web Click and Script

2009-06-12T00:00:00-05:00

In this blog I will walk you through LoadRunner Web Click and Script protocol and answer some of the questions that are always left open.

I have seen situations where a tester is very reluctant to try this protocol. He/She does not have an exact answer for why they don’t want to use this.

Well there are always pros and cons to anything in this world as I mentioned before in my previous blog. But unless you explore you don’t see the bright side of it.

Let’s get started:

What is Web Click and Script Protocol?

It is sequel to “Web HTTP/HTML protocol
It is a GUI based scripting similar to a Quick Test Professional
It records both application level and network level user activity
Uses QTP engine with LoadRunner engine

When not to use Web Click and Script protocol

When using huge load with weaker load Generators (this protocol consumes lots of system resources as it is using QTP engine)
If your application is using ActiveX UI or Applets

What are the BENEFITS?

Easy to use
Easy to maintain
Easy to learn
Lower technical skills needed to script
Goodbye to Correlation
Saves 80% of scripting time
Less time less money spent
Scripts are self-explanatory

Here is the script generated with Web HTTP/HTML protocol.

This script was generated with Web Click and Script protocol. As you can see web click and script records every user action on application and network level.

Prototyping…. the technique that keeps on giving

2009-06-08T00:00:00-05:00

As a frequent manager of software development projects, one of my favorite techniques to gather requirements or vet designs early with the system users is “Prototyping,” where prototyping is defined as the simulation of some or most of the system features to provide stakeholders early on with a tangible “look and feel”.

One prototype example I have used in the past was an interactive power point presentation which defined the functional flow of an electronic payment device. This prototype was used to define the navigation of using a debt/credit card on a device with cash back features to be used at Point of Sale. The interactive prototype was distributed to project stakeholders, evaluated and after multiple assessments, used to define and approve the functional flow before any code was developed for the payment device.

The prototyping had several benefits: the software designer and implementer were able to obtain feedback from the users early in the project, and the client and the contractor were able to compare if the software made matched the software specification, according to which the software program was built. Additionally, the project team got some insight into the accuracy of initial project estimates and whether the deadlines and milestones proposed could be successfully met.
In general, I find that prototyping helps prevent requirements misunderstandings and miscommunications among developers and users; it also reduces the number of changes that need to be implemented beyond the design phase which saves time and money.

So….. Have you tried Prototyping lately?!!!

A Bridge to Nowhere

2009-06-01T00:00:00-05:00

A few weekends ago, I took the train to New York to visit my brother. As I was standing in the café car, I overheard this interesting conversation about the gaps between Business and IT and whether the gaps are being closed or not.

Person A believed that gaps are indeed closing as the industry has established not only methodologies, sophisticated business processes and supporting tools, but a common modeling language that Business and IT can rally around to communicate, define requirements, and bound scope, etc. This structured modeling language is Business Process Modeling Notation (BPMN).

Person B thought that while progress had been made, the ball will not move much further until both Business and IT “truly adopt” BPMN, IT is able to build flexible applications and think more like business people; and the Business is able to consume structured change.
Person B explained that Business is used to constant change and improvement but for the most part, it is all ad-hoc. There is no real structure.

On my way to my seat, I thought… “I agree with both. We have made great strides in our efforts to bridge the gaps between business and technology but there is still a long way to go. Of the three dimensions in any change, People, Process & Technology, we’ve paid little attention to the People dimension, in terms of communications, competencies and cultural barriers associated with building better solutions. Until we fully address the “People dimension,” we may have built a bridge that few will walk or worse yet, “A bridge to nowhere”.

MDmM - Master Data micro-Management (Part 2 of 2)

2009-05-27T00:00:00-05:00

Now that I have characterized Master Data Management on a small scale as Master Data micro-Management, and having placed some guidelines for attributes that are candidates for MDmM, let’s explore some real-world examples that should bring the discussion down to earth:
Example 1 – Account-holder Status
I was involved with a large MDM initiative, motivated by SOX-compliance, which was focused on account-holder information. The overall project was scoped to support <20 attributes across 6 systems, but one attribute was so significant that we had to define a mini-release to deliver a consistent view of the attribute across all systems to support new functionality for the web. This attribute, the account holder’s status, fits the criteria above. It is tightly-bound in that each system had 2-5 values when defining the status. These values could easily be derived from the domain values into the enterprise standard. The definition of each attribute was clear and unambiguous. Finally, the account holder status was crucial to each operational status – from card-generation, to benefits management – every business process must be status-aware to function properly.
Example 2 – Product Description
I deliberately choose this attribute to generate discussion and debate – it also crosses into a different subject area (product). Product description is not as clear-cut as account-holder status. It is not tightly-bound (it’s free-form), which is actually part of the issue. Thus, finding a consistent product description across systems is a challenge. A product description may not always be well-defined. Is the description used for labeling, if so on a package or a shelf? However, the product description is a business-essential attribute – it is used in searches, sorting, user and customer display (in-store, web), which makes me ask the question could it be easily derived (i.e. having a “description-builder” that concatenates the product name, manufacturer, and product dimensions?) This is when the debate escalates – how can you have a derived description without standardizing the product names or manufacturer? Can you rely on product dimensions – are these the dimensions from the manufacturer or from the system? Those who have implemented a PIM recognize these complexities. I believe that this derived product description could hold value to the business and could be easily derived and therefore is a candidate for MDmM. However, my confidence level is higher when it comes to account-holder status based on theory (since it meets all of the MDmM guidelines), and I have seen this in practice on a project. Alas, I never saw the product description work in practice.

The concept of MDmM is deliberately limited to attributes that meet the criteria and would not be suited for more complex MDM domains, such as hierarchies. It also requires a pragmatic approach for choosing the enabling technologies during implementation. However, MDmM can deliver value to the business in less time then it takes to launch a full-blown MDM effort, and therefore help build a strong case for the investment required of larger-scale MDM initiatives.

MDmM - Master Data micro-Management (Part 1 of 2)

2009-05-20T00:00:00-05:00

The tip of the MDM iceberg can intimidate the most experienced captain. One way to break the proverbial MDM ice is to start with a single attribute so that the business problem can be solved incrementally. Addressing a single attribute (or attribute group) through MDM principles and best practices can be thought of as Master Data micro-Management (MDmM).

A typical master data management initiative requires significant investment before an implementation that delivers business value is realized. A MDM initiative requires a serious commitment so that data standards can be defined and systematically enforced at an enterprise level for an entire subject area, such as customer or product. Although the scope of MDM is constrained by information shared by systems across the enterprise and therefore is a subset of the complete universe of attributes for customer, product, et al., the overall effort is increased by the number of participating systems and the complexity of the integration.

Facing the daunting aspects of full-scale MDM, you probably have implemented a MDmM solution and you know that a critical success factor to MDmM is determining what attribute is suited for a small-scale MDM implementation. To help guide the discussion, let’s start with establishing criteria for identifying MDmM attributes:

• The attribute must be tightly-bound. An attribute is tightly-bound when its domain values are limited (ideally <=5).
• The attribute must be easily-derived. An attribute is easily derived if it can be transformed with minimal complexity into a defined standard.
• The attribute must be well-defined. An attribute is well-defined when you can look at the column name and immediately recognize what the information represents. This is an exaggeration of course, but you shouldn’t have to dig for the dead-sea scrolls to determine the true meaning of the attribute.
• The attribute must be business-essential. Saving the best for last, an attribute is business-essential when the absence of the attribute within the scope of a process causes failures, unhandled events, or non-compliance.

Like all MDM standards, MDmM guidelines are open to amendment and interpretation, but in the next installation of this topic, we will explore two attributes (account-holder status and product description) that should illustrate the concept.