by Thomas Murphy | January 6, 2015 | Submit a Comment
We still receive a fair number of questions about browser compatibility testing. There are a good number of ways to complete this depending on when you are testing and what you want to learn. Most browser compatibility shouldn’t be something you learn in the testing tools but at design buy following appropriate design guides and the tools built into page development environments, ie developers should deliver pages that render in the right way in every browser.
Every browser increasingly means mobile and many sites have not been created with mobile in mind. There are many full feature tools to assist with the such as Perfecto Mobile, Gomez, and CrossBrowserTesting.com. However Google also provides a free basic service and guidance oriented at determining is my current site mobile friendly and guidance on how to make it so.
Will provide a quick report and rendering of any accessible page you point it at.
Gartner’s site for instance is not mobile-friendly. (see screen captures) This is different than does it work. The site works but the text is too small, links are too close to each other, content is wider than the screen leading to wasted space and the mobile viewport isn’t set. Google provides a series of educational material and additional tools aimed at making the site more effective.
If you haven’t already begun an assessment of your mobile friendliness, you are already behind.
Category: Mobile Quality Testing Usability Tags: compatability, Mobile, Testing
by Thomas Murphy | December 10, 2014 | Submit a Comment
Mobile continues to be growing as the majority of calls we take about testing and many other areas of development and integration. To that we have just released several new research notes including a Marketguide for Mobile Testing and a special report on Mobility and Digial Business. You can find both of these on Gartner’s website by looking at my recent research. The first note covers testing tools/frameworks for functional automation on devices as well as vendors/providers of device management solutions.
The latter provides pointers to documents produced recently on mobile platform, development and integration. As we plan for next year, the topics of digital business and bimodal IT strategy will be key elements of the AD agenda. I encourage you to dialog with us about the research you want to see.
Category: Uncategorized Tags: AD, integration, Mobile, special report, Testing
by Thomas Murphy | October 11, 2014 | Submit a Comment
We have recently published a new market guide for requirements definition and management tools. This has always been a challenging market with most IT groups identifying challenges with missed, incorrect, and changing requirements but mixed success in tools adoption.
Our market guide outlines considerations and a number of the available products. We believe a key element currently is the balance in needs between definition and requirements. Strong management tools often bring more complexity and process rigor that may derail adoption especially where definition is the largest challenge.
We are interested in you feedback. Matt Light is also working on a note titled “Just Enough Requirements” that will help with defining needs. Note that one size fits all will generally not be the case. Like overall IT strategy organizations need to think in terms of bi-modal approaches.
Category: Uncategorized Tags: ADLM, Agile, bi-modal, Requirements
by Thomas Murphy | December 17, 2013 | Submit a Comment
The Lean Startup conference is underway and presentations are available on YouTube. As companies make shifts to agile and/or lean practices, they often run into barriers of culture because we often live in politicized work environments. Clusters of fiefdoms built to survive often warring against each other. The bureaucracy supports the status quo.
Public education is often a great case study for failure. We develop new mandates and policies but we dint really change the process. Everyone is frustrated, yet we can’t seem to move forward. Thus it would appear to be an organization as resistant to change as any. Thus this talk https://www.youtube.com/embed/LBxJNFAILw8 about the MVP team and its ability to drive substantial change and improvement is not only inspiring on a level of if they can do it, so can we, but also because it is improving the value of our most precious assets. Find ways to inspire and you will find ways to succeed.
Category: Uncategorized Tags: Lean
by Thomas Murphy | December 11, 2013 | Submit a Comment
While recuperating from knee surgery, one of my ways to kill time is watching #ESPN. We are entering the NFL silly season where we try to figure out why some teams with great expectations failed miserably and whose fault it is.
A common thread today is the challenge of having the right personnel for the “system” IE the method being employed. Different offensive and defensive systems require different types of athletes. While a great athlete may be able to adapt, they generally will fall short of the need because the scheme is looking for attributes they don’t have: size, speed, strength, etc.
Development teams are the same. The are teams, those teams need the right mix of skills and training to achieve in an optimal fashion. It is more than simply knowing the playbook.
Embarking on a shift to agile for instance may place great developers, testers, BAs, and managers in unfamiliar roles to which they are not as well suited. Good managers will recognize where personalities don’t fit thus causing communication silos, or where patience and training will be required to gain the necessary skills. In some cases the personnel wont fit and should be used in other project types where they can excel.
Matt Hotle has produced some excellent research on organizational structures and transitions. Our pace layering research also deals with the needs for different skills and techniques for different layers and the desired outcomes. Take time as we head to the end of the year and make sure you have the proper coach/team/approach alignment.
Category: Uncategorized Tags:
by Thomas Murphy | December 5, 2013 | 3 Comments
In a quick poll we ran at my ADLM session at #gartneraadi I asked a simple question: what is your company’s primary requirements management tool. I only provided two options: MS Office or An RM Tool. The result: 55% answered that Office was the tool in use. Does your organization also rely on Office? If so why? What holds back the adoption of directed RM tools? I have several thoughts about this but would love to hear from you.
Category: ADLM Requirements Tags: Gartneraadi
by Thomas Murphy | December 4, 2013 | 1 Comment
As we ramp up to update the MQ for Integrated Software Quality Suites, we are kicking off a survey about tool recognition and use. The survey is located at https://t.co/9NXS0lJVPO and asks a few demographic questions, asks about various categories of tools e.g. load testing, test management, etc. And a couple quick questions about results. You should find it fast to take and we do not collect any company or personal data. If you are involved in testing software, I hope you will consider spending 10 minutes taking the survey. I will share findings here.
Category: Uncategorized Tags:
by Thomas Murphy | November 30, 2013 | Submit a Comment
As much as I dislike Las Vegas, I am excited for the Gartner’s annual Dev conference and look forward to meeting with our clients. I am going to be trying out some new data collection techniques and will also spin up a new survey on testing tool use. This will include in session polling using a technique introduced to me by my HS freshman daughter (the best place to learn trends).
We are working to firm up our research agenda and are working on the annual agenda overview notes. The testing tools MQ is underway, we’ve added additional QA and ADLM coverage in Hong Kong and I look forward to helping IT orgs make challenging transitions.
When I can I will be tweeting @metamurph from the event. Hope to see you there and have you all engaged in dialog. [Read more →]
Category: Agile ALM IT Governance MQ Testing Tags: AADI, agenda
by Thomas Murphy | November 13, 2013 | 1 Comment
Microsoft launched its 2013 suite of development tools today and it made me think over the evolution of development environments, from tools focused on making individuals productive to tools that enable teams to work together effectively. Now as we move into a world focused on devices and services, Microsoft has advanced the art of the IDE by playing a common theme: blend the development world with the target environment. This means creating tools that not only make it easy to target a wide variety of clients but also that pull the cloud directly into the development experience.
The cloud has been part of the experience since VS2012 with the ability to utilize TFS running on Azure and including Azure credits in MSDN. Microsoft has continued to expand this offering and continues to add services to the Visual Studio Azure experience. Microsoft announce preview support for a new service called AppInsights which provides a variety of correlative data on how your services and applications are being accessed, performance, etc. They also announced project Monaco which is a light weight IDE running in a browser primarily targeted at editing Azure web apps. They have also integrated SignalR into Azure and leverage in the development tools. This enables a great edit/update experience while debugging and prototyping applications and aids in the ability to test applications across a variety of browsers.
As you run through the scenarios that VS and TFS enable you can see continued steps toward developers being able to work as effective teams not in a traditional world of isolation and a compile/thrash/debug loop but a continuous collaborative experience. This also includes support for improved management level views of projects with “roll-up” reports across the portfolio as well as integration of the recently acquired InRelease release management assets.
Lab management continues to be a common question in my calls, not everything is here, but the 2013 release is making a nice step for those who are targetting Azure cloud apps. Microsoft has always been about making it easy to deliver applications to their platform and while they are certainly battling hard to drive adoption of Windows 8.1 and Windows Phone, you can see that maybe the biggest prize is not the client but the services.
If you have investments in Microsoft development, this is a solid upgrade and Microsoft continues to add good value to MSDN. However, users need to make sure they make use of these benefits.
[Read more →]
Category: ALM IDE Tags: Azure, Microsoft, TFS, visual studio, vs2013
by Thomas Murphy | November 6, 2013 | 5 Comments
Recently we have seen a spate of highly visible application roll out failures. Because of the exposure it seems like these may be “isolated” incidents but I feel they are more normal in large system rollouts then we want to admit. The core challenge is that generally the business does’t value QA or understand the “ROI” — “testing just increases the time and cost of delivery”. This manifests as a constant drive to reduce the cost of testing and if possible avoid certain elements all together. This persists until somethign blows up or regulatory compliance creates an outside driver. In general the focus of a company with respect to Quality the focus is on Risk Management (thus risk-based testing methods) rather than Quality as defined by ISO 9126 : Functionality (which includes Security), Reliability, Usability, Efficiency, Maintainability, Portability.
Quality is important not just from the perspective of “does the application work” but also from the perspective of how will the application impact our maintenance costs? However, it is very difficult to turn this into an ROI and driving quality can’t just mean triple the testing budget. A key element is driving automation and getting consistent and continuous.
Even if you aren’t “agile” use continuous integration, it drives consistency. Your CI system should run a consistent set of tests including unit, static analysis, performance, and functional automation. Static analysis tools provide the ability to detect errors but also to help drive a view of ISO 9126 “conformance”. Optimyth’s Kiuwan is a great example and you can see on their site what 230 open source projects look like. I like the way they not only measure quality (5 different components – no one really does 9126) but also provide an estimate of the cost to hit target levels. Sonar (open source) and the associated commercial offering from SonarSource are becoming increasingly popular due to broad language support and integration to popular CI systems. There are many compentent static analysis tools to choose from — the key is to make sure you get one, put it to use, build a dashboard and make it visible.
Functional automation has been a constant challenge for companies. However pace of deliver, the need to support multiple browsers, devices, geographies will either blow testing costs or be met by automation. I suggest looking at Google’s Test Blog and their TechTalk’s youtube channel. Search for GTAC (Google Test Automation Conference) to find a solid set of presentation like this year’s day one keynote: How Facebook Tests Facebook on Android. If your automation tools and skill set don’t enable you to drive a high degree of automation, you must invest.
Load and performance testing can’t just happen in a box at the end of the life-cycle. There should be a level of performance tests that run every day. Performance testing must also go beyond meeting some pre-defined user load. Unless you are absolutely sure of the load (and guess what it is a new function, release…people will do things differently than in the “last version”) you have to test to learn. Test beyond expectation, run failure scenarios (ps, Netflix is another great example to model). Use Cloud-based load testing that enables you to scale tests to web-scale loads, utilizes real browsers, and enables multiple points of presence. In other words, make your testing as real as possible.
Build A Production-like Lab
Automate the lab and build one that as closely as possible mimics production. This may mean the use of virtualization (service, database, network in addition to the lab itself) it also requires, yes, automation. Use the same ARA tools to release into the test lab that are used in production (ie test release everyday too). Automate the production of test date either via data generation (ex. Grid-tools VTF) or automated subsetting and masking such as IBM Optim, Informatica Data Subset/Mask, or Grid-Tools.
Don’t become the next headline, quality matters and if your company wants to move to an agile, continuous delivery world, it is going to need to invest in quality and automation.
Category: Agile DevOps Quality Virtualization Tags: