Domostroi Wallpaper

Steve has made hands-on practical use of T4 from its earliest beginnings. He actively employs T4, TLViz & CSVPNG as essential ingredients of his pe rformance work for TrendsThatMatter. Steve's audio segments on T4 technical tips are a regular feature of the and Steve wrote the first in-depth article on T4 that appeared in Volume 3 of the OpenVMS Technical Journal . The easy reader version ) is a great way to begin learning OpenVMS Boot Camp and at the popular OpenVMS Technical Updates in Europe and in the United States. He has delive red seminars and hands-on workshops on practical performance management at more than two dozen locations in the U.S, Canada, Europe, and the Pacific Rim. Steve also participated on the Transact ion Processing Council in its formative years while it was developing and defining the TPC-C be nchmark. You can learn more about Steve at TrendsThatMatter.com It is hard to believe that the T4 approach has now been around for almost seven years, and that year by year it has achieved wider acceptance while in crementally improving its underlying capabilities. This is easily seen in the long list of enhancem ents added to the T4 collector, the large number of productivity changes for TLViz and CSVPNG, and the step-by-step increase in the number of T4 compatible collectors, now totaling more than 20. We have seen time and again that the T4 approach encourages collaboration while doing performance work. This article briefly looks back at T4's origins and where it has been, and then projects the likely trajectory it will take in the years to come as it bu ilds even further improvements onto an already solid foundation. This article also provides you with refe rences to all the latest T4-related resources and to where you can turn for help and advice so that your OpenVMS performance work can achieve the maximum benefit from this approach. T4 began its life seven years ago during a large- scale benchmark just before the release of the AlphaServer GS160 system. Tom Cafarella (OpenV MS Engineering) and I were assisting an important customer compare the GS160 performance to that of the largest AS8400 system available. The benchmark was especially complex because of its use of captured log data from a live production environment to drive the load. Each test showed highly variable behavior during its brief, 30-minute run – a far cry from steady-state industry-standard be nchmarks, which prove much easier to analyze. We were using the OpenVMS Monitor utility (MONITOR) to watch OpenVMS performance while the customer was using their own software logging program to measure and report on application response time and throughput. The customer wanted to test many different variations of CPU count, memory size, disk configuration, T4 approach was born into this cauldron of ac tivity. Tom Cafarella wrote a DCL script that started with a file and ended up producing a comm a-separated value (CSV) text file. The text file contained what Tom and I consid ered the 30 most important OpenVMS performance factors. These included CPU idle, kernel mode time , interrupt mode time, mp synch time, buffered I/O, paging I/O, and direct I/O. Tom’s code replaced a manual process we had been using that took us hours or even days to finish and that was not nearly The CSV file that we created later became known as T4-style format. The file contained a table that had a few header rows, including one where each pe rformance factor was listed by name, followed This time savings achieved with Tom’s DCL script made a huge difference in our ability to analyze these complex benchmark test runs in a reasonable amount of time. Whereas it had been taking days per run, we were now able to discover importan t lessons about a run within an hour or two of the end of the test. We could then use that information for feedback into the ongoing process, thereby enabling more focused selection for the next round of benchmark tests. In honor of Tom’s yeoman and timely efforts to cr eate the DCL conversion script, the original name Terrific Timeline Tool. Since then, as T4 evolved and a new compiled version of the extractor wa s constructed, the T4 name came to mean Total Timeline Tracking Tool. Given how much time this approach has saved us over the years, we might think of renaming it Within a very short time after its first use, the OpenVMS Engineering organization began to apply T4 to real-life production performance problems. Just like the original benchmark, T4 was perfect in these situations because production systems rarely or. To deal with the ongoing variability of each important factor, it is essential to first grab a time -series view of how that factor changes over time, and then to be able to examine it visually. The T4 approach evolved rapidly in response to a spike in performance issues related to the introduction of the GS160 system. The extraction of MONITOR data was rewritten by Ian Megarity (OpenVMS Engineering) as compiled code, and many new performance factors we New collectors that generated output in the T4-s tyle CSV format were then added that captured important performance factors not available with MONITOR. These included a TCP/IP collector called TPCMON (developed by Matt Muggeridge ) and a dedicated lock manager collector (developed by Ian Megarity). The collectors were bundled into a new DCL script (that we called T4) so that MONITOR and all the associated collector s started and stopped at the same time, and all used the same sampling rate. Another new tool called Append Re cord (APRC) was created by Ian files generated by the different co llectors into a single, larger CS V file (known as the *COMP.CSV file) containing all the factors from different collectors. Another magic piece of puzzle was completed when Ian Megarity created a new timeline visualization tool ca lled TLViz (TimeLine VIsualiZer). TLViz made it incredibly easy to visualize the trend data of each in dividual factor. And TLViz made it just as easy to view any interesting combination of factors. Our experience with this tool proved that we could ster than when we read the same CSV file into Microsoft Excel. Kevin Jenkins (from OpenVMS Engineering) suggested the idea of opening CSV files and having TLViz overlay the data. Ian Megarity implemented version 1 of that idea literally overnight, changing forever and Because of its visual nature, TLVi ood for doing collaborative work with a group of analysts, as well as being a most effective tool for demonstrating the results of analysis to both technical and no n-technical audiences. If you ca re about performance but you have To give you a flavor of TLViz and its powerful Before and After feature, Figure 1 shows a visual example from the more than two dozen charts contained in the simplified version of the original T4 article in the OpenVMS Technical Journal. While th ese kinds of graphs in a static format are useful, we have found that direct use of TLViz can be even more powerful. We have also discovered that interactive use of TLViz to present results turns out to be much more effective than creating static improvements . The first T4 involved manually running MONITOR and then manually running the T4 DCL extractor code to pr oduce a CSV file that could then be read into Microsoft Excel. Very early on, Ian Megarity combined these two steps and added an automatic way to run the T4 collector once and instruct it to turn on what might be called History Mode. In this mode, the collector continued measuring and creati ng T4-style CSV files, once per day, 365 days a year. With History Mode turned on, we were as sured that we would always have the baseline data we needed for comparison whenever anything changed or whenever a new problem arose. The T4 DCL script continued to add new collectors and to integrate them as part of a standard T4 Optional collectors for spinlock data and for TDC data are also now available with the prerelease Thanks to the continued efforts of Ian Megarity, TLViz experienced a steady progress of new features that made it easie r to use and that allowed the analyst to dig deeper and deeper in reasonably short periods of time. Some of the most useful features added include moving averages, calculation of the ratio of two factors, scatter plots and correlations Recently, Pat McConnell (OpenVMS Engineering) has added some terrifi c new features in version 2.0- 1. There are now more “radio” buttons on the botto m left of the screen that simplify and speed up some common and important operations that ar ise frequently during CSVPNG improvements. In the capable hands of Pat Mo ran, CSVPNG has continued to evolve, improve, and add new functional capability. CSVPNG now includes more than 30 separate command-line directives that work individually or together to manipulate, automate, visualize, and report on T4-style CSV files. CSVPNG is particularly powerful now for trimming down a huge data file by selecting exactly the performance factors you are most concerned with and further selecting the precise time periods that interest you. CSVPNG contains multiple options for combining T4- style CSV files in different and useful ways. These include assembling a week or month of data into a single file and combining data from multiple nodes in a cluster. CSVPNG can also input and apply its magic to an in creasing number of files of slightly different formats, including tab-separated files from the Linu x COLLECT-L utility, and outputs from the Microsoft Windows® PerfMon utility. T4-compatible collectors and extractors A growing number of T4-compatible collectors and ex tractors continue to appe ar. Recently, we have seen the introduction of the VEVAMON collector that Other T4-style collectors have been built for Rdb, Oracle®, additional OpenVMS memory factors, the Collect utility for Tru64 UNIX®, and NFS. With its latest releases for the EVA, HP Storage now includes instructions for how to convert EvaPerf data into T4-style, TLViz-compatible format. These developments are part of a wave of growing appreciation for the idea that the T4 approach is not just for OpenVMS. The conclusion: When dat a is organized in T4-sty le files, time-saving, enhanced productivity, improved depth-of-analysis benefits, strengthened co llaboration, and greater New versions of TLViz and CSVPNG are now available. For more information, see the links at the end of this article. A number of developments are underway to build add itional analysis tools that will accept T4-style data as input and provide additional analysis benefits not currently av ailable. We will be reporting about these on the TrendsThatMatter blog as soon as they become available. The T4 approach has proved that it works and that it saves huge amounts of time for the analyst. It has proved that it benefits OpenVMS performance wo rk. It has also proved that any trend data can be assembled into a T4-style format and thereby tap into T4’s time-saving benefits. A solid foundation is now in place. A substantia l percentage of mission-critical OpenVMS systems now in production operation have been collecting T4 data and building perfor mance histories that go back one or more years. New collectors, when they became available, have proved to be easily integrated into the picture because the basic T4 approach is inhe rently extendable. This is a huge area of opportunity because we have found that while “out of the box” benefits are good, the benefits of local customization are Even after 7 years of T4 development and the existe nce of more than 20 collec Extending the T4 Approach to New Territory In addition, the users of TLViz and CSVPNG continue to come up with good ideas for how to make these two important tools even more useful and powerf For those who want to extend the life of their current AlphaServer systems, the T4 approach can provide you with the data and analysis tools that help you push your systems as far as they are capable of going. For those moving toward using the new powerful, Du al-Core Intel® Itanium® 2-based Integrity server For those who are thinking of consolidating many current OpenVMS systems into a new, larger system and of taking advantage of the growing virt ualization capabilities, yo ur T4-based performance history on your existing systems will prove to be essential input data. When dealing with consolidation, CSVPNG’s ability to combine data from separate CSV fi les will surely prove to be an invaluable time saver that can help you achieve the best possible results. The following links show you where to find the mo st important T4-related information and where you can turn for help and advice along the way. If you are interested in The T4 Universe , these resources If you have T4 issues or questions: T4@TrendsThatMatter.com Want an advance copy of T4 V4.2 with the option for Spinlock Tracing and TDC collection? Send your request to Struggling with a particularly complex performance situation where the T4 data just doesn’t add up and you are looking for a quick second opinion? You can send email to T4 Mini-Evaluation Would you like to know more about the TLViz Formatter that you can use with EVAPerf data? You HP StorageWorks Command View EVA user guide Looking for the current version of the T4 collect or for OpenVMS and other T4 approach tools? You can download these from the HP T4 Home Page HP T4 FAQ The latest T4 & Friends news ca n be found by checking on the TrendsThatMatter blog You will find some useful downloads of new versions of T4-style tools at the TrendsThatMatter download New to T4? The article from the OpenVMS Technical Journal, Volume 3 is a good A streamlined version of the Ti meLine Collaboration article ( ) is a great place to start learning about T4. This work is inherently visual and is most understandable when you look at the actual performance trend data with tools like TLVi z and CSVPNG. This quick introductory version includes many easy-to-understand examples. Interested in using TLViz or CSVPNG to examine the rich range of T4-style data available from the many different collectors? Please send mail to: T4-style data samples Rather learn more about T4 by audio? You can T4 Audio Segments