Fumi Matsumoto, CTO of uTest, spoke. Since this talk was one of the workshops following the XBox key note, he began with a humorous apology for his lack of videos of cities going up in flames. uTest provides software testing services using crowd sourcing. They use a community of 40k testers around the workd.
Software testing is dynamic. The methodology is changing to Agile, the development cycle is shortening, and automation is becoming a critical part of testing. Changes include enterprise consumerization: technologies like mobile devices, social networking, and the cloud, that started out as consumer technology, and now becoming enterprise technology. This is making corporate IT harder to control. The past is Windows and IE6. The present is hundreds of smart phones. This is the year of mobile IT.
#1 Mobile Apps: Mobile devices have taken over the boardroom. They’re used not just for email, but for business apps, productivity apps like documents and spreadsheets, medical apps, etc.
Mobile’s impact on testing: Privacy and security is one of the big challenges. Coverage and interoperability are also issues. You have to support different versions of iPhone, Droid, and Blackberry.
The Big Decision: Mobile Web vs. Native Apps. Mobile Web involves a single platform, only one app to maintain, but a less controlled user experience. Native apps involve rich media functionality, but you must build, maintain, and test multiple apps.
#2 OS wars are hearing up in the mobile market. Mobile device operating systems keep changing. How does testing keep up?
#3 Tablets: Goldman Sachs predicts shipments to grow by about 500% in 2011. Purchases by businesses are also increasing. For example, Mercedes Benz uses iPads to present and execute financing options to customers. There is pressure to make sure apps are tablet optimized. There are much higher conversion rates for people shopping on tablets than on PCs. Tablets, like mobile phones, have issues with fragmentation and different operating system versions.
#4 4G: The impact of 4G networks is really about speed. Wide adoption won’t happen in the next couple of years, but it is starting. Technology drives obsolescence. We will need 4G for movies. The testing challenge here is multiple devices and network speeds. The White House plans to increase 4G coverage to 98% of Americans. The impact on testing will be shorter shelf lives and new devices.
#5 HTML5: Geolocation and animation on web sites are not technically part of HTML5, but are bundled in. HTML5 is an alternative to Flash, an “app-like” browser. Test impact: It’s not standardized yet. But many developers have ditched Flash for HTML5, not waiting for 2014. Mobile OSes don’t support Flash. Code will need to be rewritten for cross-functionality, and staff retrained, making sure the new code works.
#6 IPv6: This network standard for IP addresses is replacing IPv4. 4 billion addresses are no longer enough. IANA is running out of IP addresses. [LG: And I remember the early days, when IP addresses were so abundant that both Stanford and MIT got top level IP address blocks.] Also, the way IP addresses have been given out did not make the best use of the address space. IPv6 provides a huge address space, ready to go, which just needs to be rolled out to consumers. Verizon Wireless will require 4G support for IPv4 and IPv6. Facebook and Google already support IPv6. Initially, IPv6 will work in parallel with IPv4, with increasing usage during the next couple of years. It has been issued on devices for over 4 years, and could be used now, but the Internet for IPv6 is not mature yet. Test Impact: IPv6 is incompatible with existing infrastructure and code. The transition will be painful and require testing.
#7 Cloud: Juniper Research predicts that 130 million enterprise customers will use mobile, cloud-based apps by 2014. Backend server testing is harder to manage, resources are scattered, and there is more variability and edge cases. Front end VI testing reduces barriers to testing, and allows more people to test, with more test on demand.
#8 Geolocating: Several companies work to use location for more than marketing. Apps, especially social apps, take advantage. But this needs to be tested in the wild. One app tested in the wild worked everything except Germany and Japan, because of how they configured their 4G network.
#9 NoSQL: A datastore that doesn’t use SQL, for example Netflix, Amazon, Facebook, Twitter, eBay, and similar large scale applications. It scales horizontally by adding servers. NoSQL is emerging technology that can handle a large number of connections, with a shallow learning curve, but a shortage of tools to diagnose problems. Complex queries with no joins are used to suggest friends, connections, etc. Test Impact: Enables scalability. Use cases: Managing large schemes of non-transactional data, syncing online and offline data, avoiding heavy and complex joins that load relational databases. This is backend technology.
#10 Social: Real-time communication is moving to business. Companies use external social networks, like LinkedIn, or social network applications like SalesForce. Test Impact: Security hurdles, must protect sensitive data when talking on Facebook and Twitter, integration with third party apps.
#11 BONUS Connected TV, e.g. XBox: Test impact: Another multiplier to the test matrix. This is still an emerging technology, mostly applicable to the consumer market.
Q) What are the positives?
A) A very vibrant software testing industry and more automated testing. Automation tools are really happening right now. Ability to work with testers remotely, outsource, crowd source, and do high volume testing at the end of the development cycle. People register on Amazon with prices to test. You can rent browsers in the cloud.