Benchmarking the Software Development Life Cycle Process - Assignment Example

Service Management Executive Summary The report is an attempt to understand the concept of software development life cycle process, benchmarking process and feasibility of the strategy to outsource the SDLC process as against using in-house IS resources, from the perspectives of an IS project manager. The report includes introduction to the topics of SDLC, benchmarking and proposed approaches that can be used in the given scenario. It further discusses the ten key metrics or measures that can help to track a project and compare the results with the option of using outsourced services. It stresses upon the strengths and weaknesses of each of these metrics and suggests a method to quantitatively record them. Finally it highlights the obstacles that can be faced in the decision to benchmark and also outsource. Benchmarking the Software Development Life Cycle (SDLC) Process This report titled “Bench Marking SDLC” is an attempt to understand, assimilate and convey the bench marking exercise by taking example of the system life cycle development process. The aim of the report is to delve into understanding of the areas of concern for several projects which have run over budget and time, as a Program Manager, and decide on the areas of improvement visa-a-via the option of outsourcing. Introduction The aim of this part of the report is to get an introduction to the topics that will be covered in the report. This will not only help to connect and relate each of these topics but will also help to understand and appreciate their significance. The SDLC or the software development life cycle is a process that includes several logical and formal activities combined together to build a software product. The logical and formal activities that make up the SDLC include the steps of project conceptualization; analysis of requirements; analysis of cost versus benefits; comprehensive software requirement specifications; software designing and programming; testing the product; revising the product to correct faults, deficiencies or incorporate changes, new additions or alterations; product implementation; providing technical and user training and lastly continuous maintenance of the developed product. According to James Taylor (2003), the aim of the process of software development life cycle (SDLC) is to provide the customer with a superior quality system, completed within the pre estimated cost and time budgets, which can meet or surpass his expectations by performing efficiently and effectively within the present and planned Information Technology setup of the customer and thereby being economical to sustain, easy to replace if required and cost-effective to augment upon if future need be.1 Though there are several definitions of ‘Benchmarking’. The process is commonly defined as the practice of measurement and understanding of an organisation’s internal business processes in terms of cost, productivity, quality of output, cycle time, etc. in comparison to a similar business process from another organisation or from within the organisation which is considered ‘best-in class’ and then adapting it and tracking it constantly to measure the success of the adaptation.2 There are several types of benchmarking processes that involves differing techniques and their success or failure are assessed by tracking various measures and matrices. Collaborative and Metric Benchmarking processes are the most popular approaches to benchmarking. Both these approaches can be used in different types of benchmarking initiatives. For the SDLC, the use of performance benchmarking seems the best approach to achieve the target of understanding and defining the failure causes, compare the key performance indicators or matrices and also consider the option of outsourcing. Collaborative benchmarking is the process of benchmarking that involves a highly cooperative method of data or information collection from organizations within the same industry. Thus, this method involves collection of information about a ‘best practice’ even from one’s competitor. This approach may be either done by a single business that wants to improve or turn around its business process, or may be done on a collaborative basis, as the name suggests, by a group of companies. According to Boxwell (1994), collaborative benchmarking is one of the most commonly used benchmarking approach since it is fairly easy to observe, follow and put into practice.3 Metric benchmarking is the approach commonly used by the Information Systems professionals to benchmark their performances in specific business processes such as the software development life cycle process. According to Charles Greene (1993), it is the quantification of performance as compared to other performances over a period of time based on the scale of Key Performance Indicators (KPIs). This approach is useful in determining the regions of perceptible gaps in performance and thereby tries fixing those gaps. Metric benchmarking uses the concept of measuring the reference points quantitatively to compare against best-in-class businesses, previous experiences and / or industry standards.4 One of the most important shortcoming of this approach is the failure to make out the reasons of and likely resolutions to a below standard performance. Performance benchmarking is the assessment of an organization’s service or product performance as against those of the target competitors. It is the most difficult benchmarking method since a business is trying to better or meet its competitor’s best-in-class standards of product or service and hence sharing of information between direct or indirect competitors within an industry is difficult as it might result in losing the competitive advantage.5 This method of benchmarking uses the competitive method of reverse engineering followed by benchmarking. It involves taking into account the every facet of the competitor’s strategy from in-depth assessment of the product or the service being benchmarked to marketing, pricing, placement, positioning, and brand and customer strategy used by the competitor to market the product or service to its customers. Performance benchmarking using competitor information is said to be most difficult since the information sought by an organisation to undertake performance benchmarking is not easily available and entails elaborate and comprehensive research. For adopting the performance benchmarking initiative, it is critical to bear in mind that the aim of the exercise is to focus on competition in particular and not the industry in totality. Key Measures and Metrics of Benchmarking This section is an attempt to understand the ten key measures and metrics which can be used to undertake the software development life cycle process’s benchmarking efforts and can help an organization to determine the option of outsourcing. It is important to note that the list of ten key measure and metrics are not exhaustive in itself and there are several other ways in which this list could have been prepared. But, with the understanding of SDLC and the benchmarking method being considered, these ten measures and metrics seem to hold the key to understanding and assimilating the benchmarking of the process and the option to outsource the process6. The ten key measures and metrics for the software development life cycle process’s benchmarking effort are: Table 1: Table representing the Metrics/Measures for benchmarking of different stages of the SDLC process.7 Goal Characteristics Metric/Measure Quality of requirements Understandability Readability Index and structure of documentation Volatility Number of requirements and changes Traceability Position of change Quality of Product Maintainability and reusability Size and complexity correlation Documentation Comment percentage Effectiveness of Implementation Rate of Completion Planned task completion percentage (based on cost and time guidelines) Resources utilization Manpower hours spent on tasks Effectiveness of Testing Correctness Error criticality Time to find errors Time to fix errors Readability Index and Structure of Documentation Metric: This matrix will help a project manager to not only measure the quality of the project requirements through the goal of achieving understandability but will also make it easy for the customer or the end user of the SDLC product to be on the same page as that of the project team during the entire SDLC process and afterwards. Number of Requirements and Changes Metric: This metric will not only help to reduce the volatility of the project but will also enhance the requirement gathering stage and consequent changes in the requirements during SDLC, thereby limiting on vagaries on requirement collection and change adjustments. This will help to reduce cost and man power requirements of the project and help to get a more accurate time plan. Position of Change: This metric represents the stage in SDLC when change is made in the software and where the change is documented for everyone to easily track the change, thereby greatly improving on the traceability of a change. Size and Complexity Correlation: This metric will ensure the ability of the project team to maintain and reuse the codes and programs used in the software. It will help to reduce time and costs associated with development of similar products in the future. This metric will help to greatly improve the overall quality of the product. Comment Percentage: This step in documentation of a SDLC project is critical to track every step of the project with comments from staff in charge of that particular step. This metric helps to act as a dictionary for anyone who wishes to understand the project, the timeline, the difficulties and the capabilities associated with a project and its product. Planned Task Completion Percentage: This metric helps to track the progress of the project based on pre estimated cost and time schedules. It helps to track the project cost and timeliness. The use of Gnatt charts, PERT or CPM, etc. methods can be used to obtain this measure. It helps to find a team’s performance based on rate of completion. Manpower Hours Spent: This measure helps to track the man power resource being used for various tasks and track their efficiency. This helps to measure the effectiveness of the entire team, ability of a team to meet deadlines, ability of a team to handle and manage projects etc. It is an important measure of whether a project should be outsourced or handled internally. Error Criticality: The metric that measures the criticality of the errors that occur in a coding, testing and deployment environment, help to assess the knowledge, capability and intellect of the team handling the project. It helps to determine whether a critical project can be handled by in- house software development team or will the organization have to spend on outsourcing. Time to Find Errors: The metric of ‘time to find errors’ is a measure of a team’s ability to locate the sources and causes of dysfunction of a coding in a SDLC project. It also demonstrates the team’s ability to handle a project within time and budget constraints. Time to Fix Errors: This measure if used in conjunction with the metrics of ‘error criticality’ and ‘time to find errors’ will help to comprehensively determine the correctness of a product developed by a SDLC team and the effectiveness of the testing and coding phases of project. Strengths and Weaknesses of the Metrics/Measures This section is an attempt to understand the relative strengths and weaknesses associated with each of the metrics or measures that have been identified. The table below lists the strengths and weaknesses: Metric / Measure Strengths Weakness Readability Index and structure of documentation Helps an IS project manager to measure the quality of the project requirements Makes it easy for the customer or the end user of the SDLC product to be on the same page of the project. Difficult to measure. If the same team or same human resources are not involved, there might be differences and hence confusion in documentation. Number of requirements and changes Helps to reduce the volatility of the project and enhance requirement gathering process. Limits vagaries of requirement collection and change adjustments. This will help to reduce cost and man power requirements of the project and forecast a more accurate time plan. If incorrectly documented, will result in inappropriate product and documentation. If done for a project with large amounts of changes, will prove to be costly and time consuming, thereby having a potential to delay the project. Position of change Helps to easily track the change, thereby greatly improving on the traceability of a change in the entire life cycle of software. Inconsistent method of recording and tracking change will cause confusion and impair traceability. Needs dedicated efforts from manpower to record changes. Size and complexity correlation Ensures the ability of the project team to maintain and reuse the codes and programs used in the software. Helps to reduce time and costs associated with development of similar products in the future. Improves overall product quality. Reusability of same codes makes a new project prone to similar errors and hence time wasted on fixing those errors as well. Comment percentage The documentation of a SDLC project helps to track every step of the project with comments from staff in charge of that particular step. Helps to act as a directory for anyone who wishes to understand the project, the timeline, the difficulties and the capabilities associated with a project and its product. High percentage of comment might lead to ambiguity of work or confusion. Wastage of time trying to document every single step. If only major steps documented will result in deficient documentation. Planned task completion percentage It can track the progress of the project based on pre-estimated cost and time schedules and track the project cost and timeliness. It helps to measure the IS team’s performance based on rate of completion. Difficult to measure accurately. Errors in methods of measurement reflects incorrectly on the IS team. Might not always be a best way to measure the IS team’s performance. Manpower hours spent on tasks Tracks the man power resource being used for various tasks and track their efficiency. Helps to measure the effectiveness of the entire team, ability of a team to meet deadlines, ability of a team to handle and manage projects. It is an important measure of whether a project should be outsourced or handled internally. Difficult to track. Might not give individual performances of each team member. Error criticality It measures the criticality of the errors that occur in a coding, testing and deployment environment. Helps to assess the knowledge, capability and intellect of the team handling the project. Helps to determine whether a critical project can be handled by in- house software development team or will it need to be outsourced. Mistake in determining criticality of error might result in improper representation of the error or the time taken to locate and fix it. The degree of criticality of an error might be influenced by human fallacy of favorability for a particular team member. Difficult to ascertain. Time to find errors A measure of the IS team’s ability to locate the sources and causes of dysfunction of a coding in a SDLC project. It also demonstrates the team’s ability to handle a project within time and budget constraints. Dependent on criticality of error and hence suffer from same weaknesses. Dependent on type of coding and type of project. Time to fix errors Determines the effectiveness of the IS team during the testing and coding phases of a project. Demonstrates the team’s ability to handle a project within time and budget constraints. Dependent on type of error and the criticality assigned to it. Dependent on type of coding and type of project. Incorporates errors from other metrics. Difficult to measure if the type of error is new. Relevance of the Method and Metrics Chosen The section of ‘relevance of the method chosen’ tries to highlight the conditions, the reasons and the situations that help to understand why an organization decides to benchmark to begin with. The rationale behind the strenuous and difficult process of benchmarking may range from justification of the organization’s IT investments, to the evaluation of the IT teams and its management’s performance, to improvement of internal IT functions and processes to decisions like ‘build or buy’ to decisions about outsourcing of a specific IT process or to undertake the initiative of total quality management (TQM) as a strategic initiative or to finally justify cost reduction or cost assessment endeavors as a part of company’s budgetary policy. 8 The suggested benchmarking method and the metrics or measures have the following relevance: Setting of performance goals by an organization or the IS manager of an organization for the IS team. Assess job performance of the team, thereby identifying top performers and also to validate in-house investment in IS team as against option of outsourcing. Not fall into the trap of setting easy goals or performance standards and be competitive with the best-in-class. Objective and measurable parameters or metrics make the comparison between team members’ performances and comparison with best-in-class competition more authentic, easy to achieve and measure, and less controversial. Helps to provide concrete evidence for budgetary and strategic considerations of in-house IS team versus outsourcing. All of these metrics can be found even for an outsourced project and thus provides a relative measurement condition. These metrics when used quantitatively might help to provide justification for use of in-house IS team or outsourced services and compare the expenditure in IT by an organisation with those of others in the industry. Such metrics and benchmarking process can act as a source of constant improvement and development of higher performance standards, increase efficiency, cost effectiveness and quality of product. Steps to Obtain Data This section helps to obtain data to substantiate the metrics identified that can help to distinguish between the relevance of use of internal services or external outsourced services for a software project9. The steps to obtain data of the various metrics for performing the benchmarking exercise can be based on the following steps: Analysis of the numbering structure of a document to assess the details of the project. Analysis of the percentage of changed requirements for a specific time period by finding the ratio between the counts of requirement changes made in a document and the count of initial requirements in a given time period. Collection of traceable changes to the documents and the codes and tests of a software program. Measurement of lines of codes, non-comment non-blank lines, executable statements, etc. to measure the size and complexity correlation. For ‘comment percentage’ find the ratio between the comments and the actual code lines. Completion rates found from data based on pre-estimated cost and time schedules as against actual values10. Tracking of manpower hours spent on each step of the life cycle to compute resource utilization. Use of inspections, unit and system testing, and peer reviews to measure error criticality, and time to find and fix errors. A cumulative error curve also helps to measure the rate of finding errors in the testing phase. Weaknesses of the Benchmarking Process Data Collection The process of performance benchmarking to improve upon the internal IS team performance or to decide about obtaining external outsourcing also has certain weaknesses in terms of availability of data and information to compare the decisions11. The following are the weaknesses12: Difficulty in obtaining competitor data for process benchmarking. Quantity of information available about best-in-class competitor. Incorrect, inappropriate, falsified and obsolete data provided by outsourcing service provider and the competitor source making comparison inaccurate. Internal inertia and opposition to the benchmarking process making the decision and transformation difficult. Conclusion Thus to conclude, it can be said that whatever the method or approach of benchmarking selected, the decision to outsources a software development project or use internal IS team is complex and through investigation needs to be put into collecting data to determine the strategy. Collection of data for the various metrics that can help to benchmark are often complex and cumbersome and come with their respective strengths and weaknesses. The list mentioned in this report is not a comprehensive one and has the limitation of time, understanding and research. Further studies into the aspects discussed in this report will enhance our understanding of the concept of benchmarking and help to relate it to software development project and the decision about use of in-house IS team or external outsourcing service. References 8 Bogan, C. (1994) Benchmarking Best Practice. McGraw Hill. 3 Boxwell, R. (1994) Benchmarking for a Competitive Advantage. McGraw Hill. 2 Camp, R. C. (1989) Benchmarking: The Search for Industry Best Prectises that Lead to Superior Performance. Productivity Press. 12 Graham, A. (1997) “Association Publications: Benchmarking Common Problems” The Magazine for Magazine Management. vol. 25. 4 Greene, C. B. (1993) Benchmarking the Information Technology Function. New York. NY: The Conference Board. 5 John, L. K. & Eeckhout, L. (eds) (2005) Performance Evaluation and Benchmarking. CRC: Taylor & Francis Group. 6 Kitchenham, B. & Walker, J. (January, 1989) “A Quantitative Approach to Monitoring Software Development” Software Engineering Journal. 11 Spendolini, M. J. (1994) Benchmarking Book. AMACON. 9 Stevenson, W. (1996) Productions / Operations Management. (5Th edition). Irwin Publishing Company. 1 Taylor, J. (2003) Managing Information Technology Projects: Applying Project Management Strategies to Software, Hardware, and Integration Initiatives. AMACON. 7 Hyatt, L. & Rosenberg, L. (April, 1996) SATC Software Quality Model ‘in’ A Software Quality Model and Metrics for Identifying Project Risks and Assessing Software Quality [online]. 8th Annual Software Technology Conference, Utah. Available from: http://satc.gsfc.nasa.gov/support/STC_APR96/qualtiy/stc_qual.html [Accessed 23 February 2009]. 10 Skogstad, E. (January, 2003) “Using Benchmarking to Uncover Best Practices” IT Management News. [online] Available from: http://www.itmanagementnews.com/itmanagementnews-54-20030701UsingBenchmarkingMetricstoUncoverBestPractices.html [Accessed 23 February 2009]. Read More