by Thomas Murphy | April 2, 2013 | Submit a Comment
The Nexus Forces, in particular devices and the browser explosion, combined with the “money saving” BYOD movement are putting a tremendous squeeze on software testing organizations. No longer is it good enough to test on a couple browsers or a device or two. Yet at the same time, every combination can not be tested.
Mobile e-commerce only complicates the issue with a diverse set of offers and customer behaviors. Raising questions such as:
- Should I test ‘checkout’ with 5 items in the basket or 1?
- Should I test ‘checkout’ with credit card payments or using gift cards or a combination?
- Can I get insight into genuine customer journeys on our website so I can mimic those across a variety of browsers and devices?
- Do I know the demographics of my customers and understand if the features are useful or confusing for them?
This goes beyond traditional test development where requirements are analyzed, risks assessed an a test plan is developed in a relatively deterministic fashion.
I believe that this is another realm where User Monitoring and Software Testing need to deliver blended solutions (see: Leverage Your Application Performance Monitoring Through the Application Life Cycle ID: 00227165). Yet, tools vendors and platforms aren’t there yet. Most testing providers do not have Real User Monitoring abilities and there are few strong partnerships. Beyond this there is little to no guidance about how to utilize what we know about user patterns to better inform our decisions about technologies to be tested and user behavior patterns. I expect this will change during the next 2 years but for now, you will in general be on your own. I suggest using Gartner Peer Connect and other forums discuss experiences and build a foundation for Analytic Driven Testing, testing must get smarter or it will kill us.
Category: Agile Mobile Quality Tags:
by Thomas Murphy | January 30, 2013 | Submit a Comment
The old adage says you can’t teach an old dog new tricks. Microsoft, in the software world is certainly an old dog, but it seems to be learning “new tricks” in changing the way it defines and ships software. This may not work yet for all of Microsoft, my view is primarily at the Developer Division and the Visual Studio/Team Foundation System set of tools. In particular this behavior is best seen with the Team Foundation group which has transformed itself from a traditional boxed product vendor with a two year release cycle to a full agile, continuous delivery style organization similar to Rally, VersionOne, Atlassian, Collabnet and other agile focused tools.
This hasn’t been a speedy transformation. It is an old dog after all with a strong history in traditional product management, large teams and a broad customer base that often has competing needs. The company as a whole has had pockets of “agile” for at least the last 10 years. So what is different now? The cloud. The move to deliver TFS on Azure has been a transformation point enabling new styles of delivery and requiring a new approach to the definition and delivery of customer value. With TFS Azure, Microsoft can now build minimal viable products, turn the features on selectively, and gain feedback from a broad set of users.
Microsoft certainly has brought in some “new dogs” that have agile experience but the real transformation has been driven by Cloud and the drive that social driven open source development has provided. The key isn’t the new dogs but that the old ones have been able to adapt, to learn new tricks and adapt to new conditions. Remember, survivors are those that can adapt to changing conditions and if Microsoft can evolve, your organization can too. The Nexus of Forces represents an inflection point for survival either you will adapt and thrive or remain stagnant.
Category: Agile Tags:
by Thomas Murphy | January 29, 2013 | 5 Comments
Or why your organization needs to get Agile.
An increasing number of the calls I take are dealing with the challenges of BYOD, the growth in browser diversity and the loss of control over the target environment. The business realizes that there may be an opportunity for lower device costs and happier employees through BYOD (not thinking about the cost to development and test, etc).
So how do you deal with this in test? By getting on top of it as soon as possible. If your application is going to support multiple browsers, multiple devices, multiple user constituencies, you can’t batch all that test work up in a classic waterfall, iterative fashion. Developers and testers need access to resources that will enable them to test continuously on any browser/OS/device. I expect to see a long list of announcements in the mobile testing and cloud space during the next two years as vendors seek to fill-in gaps and respond to the very different nature of continuous delivery required to compete effectively and drive engaged customers and employees. Examples include MobileLabs Inc. release late last year of DeviceConnect making it easy to provision a private device cloud or last week’s announcement by SOASTA and Appcelerator of integration between SOASTA’s mobile testing facilities with Appcelerator’s mobile platform such that apps under development can make use of continuous testing.
If devices and the Cloud are going to truly transform your business, your business must transform its culture and the practices that support it. It won’t be pain free but like a sunny day in Seattle, it can be spectacular.
Category: Uncategorized Tags:
by Thomas Murphy | December 18, 2012 | 1 Comment
I have been using a Windows 8 tablet for a little over a week now (not for this post though). It is an Acer Iconia W510 and generally it has been a pleasurable experience. This is a full Windows 8 machine with Intel Atom processor, ability to doc to a keyboard/external battery combo to use as a laptop (I don’t have this, I plan to use it with a bluetooth Microsoft Wedge keyboard in the future). I have Office 2013 and VS 2012 loaded on the machine (this is not a giant developer desktop) and they run well. I mainly have used VS to connect to my TFS Azure instance to enter and update tasks. But I like that I can fire up the debugger and just run direct on the device rather than over a connection, etc.
Office 2013 comes with support for tablet (finger) and keyboard/mouse support (changes the spacing of items and sensitivity) and it works great. I like running full Outlook vs. the Windows 8 mail client (though I have run the mail client on my home desktop for a little more than a year). Neither in some ways are as fast for certain things as my Windows Phone (selecting and deleting lots of mail). Utilizing the combo of Office and SkyDrive is clean and simple making it easy to move between my desktop and the tablet keeping work in synch or doing shared edits with others. Note, I am looking at this purely as a personal productivity situation and in comparison to the iPad that I have used in the past. We are a MSFT Office shop so there are a lot of documents, presentation, spreadsheets that get shared and passed around for review and right now this is a “challenge” on an ipad.
A downside is that Windows 8 devices are not “standard” sizes…ie you can’t walk into the local office supply store etc. and find the wall of covers/carry cases, etc that occur with products that have a thinner variety of options. However other elements of the system are generally standard: micro usb, hdmi, microSD slots are all nice. The anomaly here is that the power connector (because it is designed for their docking keyboard) is a non-standard connection.
Stability – still needs work. I would say I “reboot” daily. That means I am essentially cycling power. MSFT is dropping regular updates to Windows update. I can say from an Apps perspective that so far they all work on the tablet where I can’t say the same on my desktop. App vendors need to do a better job of defining support requirements or raising a flag when they fail rather than start and tombstone.
Apps, for me, it is fine but I am not always worried about the latest cool thing and unless the App is really awesome it doesn’t make a lot of sense vs. just using the web site. I think this is more an issue for Phones where real-estate is constrained.
The user experience, I have been using Windows 8 for over a year on my home PC but while I like the Windows 8 experience, there are non-intuitive things and little to guide you to short cuts, gestures, etc. and I think as the number of Apps grows there will need to be work on how you organize and manage them.
Last note, since this is a fun toy, I love the integration of Windows 8 through Xbox Smartglass to my Xbox. It acts as a remote, provides side by side enhanced show information and connects to all my xBox live information.
Category: Windows 8 Tags:
by Thomas Murphy | October 24, 2012 | Submit a Comment
The market has gained a many new products to aid with capturing software requirements through wireframes and prototypes. While this is great, a conversation I had today with Vernalis Systems about their approach to drive business improvement via technology. We got into a discussion about wire framing and the importance in the beginning to keep it simple because you are iterating through concepts to understand business needs and to create effective design. Later in the day I rand across an interesting article in Fast Company by Matthew May discussing the design of the FedEx logo. This iconic logo was created by Lindon Leader and here is the key point: “I strive for two things in design: simplicity and clarity”.
Too often UIs, like “requirements” docs in general get crammed with everything including the kitchen sink. While your organization may not have great human factors specialists or designers, putting this concept of simplicity and clarity at the heart of your design will lead to great foundation for project success and improving the usability of our software. This concepts maps well against the concepts of Pretotyping and Minimal Viable Product. This is not to discount the value of a good designer and it is key that you end up with Viable not “barely good enough”. Take time to iterate through ideas and keep them simple without getting hung up on polish, that will come later.
Category: Agile Requirements Usability Tags:
by Thomas Murphy | October 11, 2012 | Submit a Comment
Susan Cain, a former wall street lawyer shifted her career to match her original goal of being an author deliver a book this year on the power of introverts and delivers a challenging TED talk about how to work better with Introverts and changes we need to consider in education and workspaces that challenges the current push to highly collaborative working. This would include the move to “agile spaces” and the need to balance this with space for solitude. She states that “Solitude is a critical component of creativity.” These concepts are discussed in greater detail in her book which is titled: Quiet: the Power of Introverts in a World That Can’t Stop Talking.
This is a challenging conundrum. My experience would find that many in the technical fields tend more naturally toward introversion rather than extroversion. The current stream of thought around decision making is based on concepts of the wisdom of crowds and collaborative teams. “Way back” in 1995, Jim McCarthy captured a set of rules for managing teams and development that included the rule: Don’t Go Dark. IE don’t let developers go into a cave and wait for software to magically appear after you’ve shoved enough pizza under the door (here is a link to the latest version of The Dynamics of Software Development). I certainly don’t have “the answer” here and as an analyst have the perfect job that mixes working from a home office with occasional forays into public speaking but we need to find ways to balance and it this is especially important for organizations transitioning from a more isolated worker floor plan to open spaces. How do you help introverts feel comfortable in daily stand-ups, pair programming and other out of the comfort zone experiences so that you truly get the wisdom of the crowd, rather than the march of the lemmings.
And introverts, maybe you should take a look at “power posing” and the work of Amy Cuddy from Harvard School of Business. Steel yourselves before the daily battle and let your awesomeness through.
Category: Agile IT Governance Space Planning Tags: @amyjccuddy, @susancain
by Thomas Murphy | October 10, 2012 | 1 Comment
As part of our research for the Integrated Quality Suites Magic Quadrant we have been looking at how people spend time in testing, the types of development processes and team structures used and I thought I would share one slice of that here. We ask what types of tools are being used for capturing various types of requirements.
As can be seen, the survey participants generally rely on generic tools more than application specific requirements tools. While there has been solid growth in the variety of tools and vendors in the market adoption still lags behind other tool categories in the SDLC. This while we still see in Capers Jones’ latest software quality report that Requirements are the largest source of Delivered Defects in an average organization with a rate of 0.23 per function point. The real question is what holds us back? Cost of tools? Usability of tools? We think somehow we are doing better? Maybe it is time for a change.
Category: ALM Quality Requirements Tags:
by Thomas Murphy | September 25, 2012 | 1 Comment
The Harvard Business Review Blog Network published a post from Anne Kreamer about The Rise of Coworking Office Spaces on the 19th that describes a trend that combines a combination of traditional office leasing provided by companies like Regus together with the Open Floor plans that many Agile development teams use (if you are a client see my note on Agile Spaces or look at my earlier posts on the topic). Kreamer notes that Coworking spaces provide the following benefits
- collaborative networks and an ecosystem
- fostering innovation
- simplify starting a business
This last bullet emphasizes the current prime use for such working arrangements a collaborative think tank that provide not only the infrastructure but also the people to bounce ideas off, validate legal issues, etc. While there will be no dramatic shift I do expect to see a continued evolution of our working spaces and Kreamer notes;
It’s just one of the many ways that companies are capitalizing on flexible workspaces for a flexible workforce — a trend we can all expect to see more of in the future.
Gone are the days of the pension and the battle for the corner office. Going away are also the tyranny of a business hierarchy. Which brings to recollection a product naming email thread from many years ago during my product management days in the Smalltalk market. Richard Gabriel railed against the marketing organization for the product name VisualWorks as a name that connoted business buyers wanted to purchase a product that told their developers they were there to work and that we may as well call it Visual Gulag. While you might not think of work in such terms, there is no doubt that the way we work, and the things we value as well as what our companies value in us will continue to evolve. This means a continued move to enhance cross-discipline collaboration and open spaces with loosely defined project team boundaries in a drive to find innovation and exploit business opportunities.
Category: Agile Space Planning Tags:
by Thomas Murphy | September 21, 2012 | 2 Comments
I started a small survey a few weeks back to look at gaining a bit more quantitative information about testing organizations. I have suspended the survey for now (and folded it into the survey we are running with the Integrated Quality Suites MQ reference customers). While we need to do more digging here and strengthen the data set with the input from the reference customers, I thought I would share a couple response charts as food for thought.
First the Tester to Developer ratio. This falls about where we expected though I am sure in a broader market there are more people that would fall in the malpractice areas of 1:10 or 1:20.
And organization style
Category: Metrics Quality Testing Tags:
by Thomas Murphy | September 18, 2012 | 3 Comments
As we work through vendor conversation for the update to the Integrated Quality Suites MQ, we are seeing a tremendous number of interesting developments from the market though often they are fragmented. At the same time it seems that we are being pushed forward to continue to do more and more to ensure that we are building software right and resulting in a pressure for more automation, more tests, more metrics in the: they didn’t work before but if we just do more then it will all be good mindset.
While there is value in this, there are limits to the ability to do more and we find that often we end up with conflicting directives. I believe that as we continue to shift toward a production-centered mentality, away from projects, and continue to progress to Chief Marketing Officer involvement in the development process (oh no!) responsiveness will be critical and in a good way we may turn our metric focus away from a pile of function point-based measures to Customer Satisfaction and ROI. This will change the way we approach testing (and development as a whole because quality will continue to not be some abstract at the end process but a fundamental goal) to meet new kinds of needs from the business. Pushing this envelope is the Keynote from GTAC 2011: Test Is Dead. It is purposefully pushing a point of view that only “those wild internet companies can do” that I believe will settle in over the next 10 years. Rather than the focus on Build it Right, the focus is Build the Right It.
Harvard Business Review’s “Management Tip of the Day” for Sept 18th reinforces this idea
When asked, consumers almost always say they want more options. But their purchasing behavior often indicates otherwise. Consumers are often overwhelmed by the flood of product information and choices available to them. Many report unnecessarily agonizing over trivial purchases. This cognitive overload causes them to make poor decisions, repeatedly change their minds, give up on purchases altogether, or regret the purchases they do make — none of which is good for your brand. Help your customers simplify their decisions. You can reduce choice by getting rid of less popular products. Or you can simplify their choices by helping them navigate their options and giving them trustworthy information they can use to weigh the alternatives.
To often we see to focus on more, more choice, more options, more features—meaning more WIP, more tests to write and run, more, just give me more metrics and best practices and life will be great. Instead we need less, the challenge is which less.
Category: Agile Quality Tags: