When developing mobile applications especially iOS applications and especially for OPD models, one of the major pain point is Beta Testing or User Acceptance Testing. Well no offense here on the testing rather the pain point is over the distribution of test builds to multiple parties involved.
The ultimatum behind this article is something very close to what I have experienced as a Business Manager, as we complete developing applications, we proceed on to distribute the build to client or third party testing services or whatever the case is with the A/B Testing or UAT but what gets struck up is a need for publishing them in ad-hoc basis or per se as test build with cluttered steps involved like syncing and re-syncing on every test steps.
This article will feature the prominent determining factors in choosing tools for iOS and Android as main picture.
Before picking out a tool for A/B Testing or UAT testing for mobile application or “mobile first” web application, consider the following points twice!
- I have an iPhone, can I run APK file?
- Is OTA enough?
- Crash Reports, Logs is okay, but touch data and in-app analytics is simply awesome!
- I have uploaded a build two days back, didn’t you get a notification?
As a very first step to filter out a best tool for you, consider what platform your app is and to what platform does the testing service offers support. Well, the answer for the above question is absolutely NO, run APK in Android and IPA in iOS!
It is quite common to look out for OTA i.e. Over the Air deployment of beta apps, but what more important is “Analytics engine” a service offers.
Yes, that sums the point! It is fine to get logs, crash reports, installed status and base but it is more important to analyse about touch data and in-app analytics such as in which page a user is spending more time and less in which page enabling us to come to an approximation about the app usage.
My bad! Getting to know about when a test build is shared is a primary and a simple thing to expect! Few test services fail to push notifications when an update activity occurs, beware that you don’t fall for them like the heading of this point!
Embarking onwards for tools that offer most or all of the said points is here,
- The Beta Family
- Beta Glide
- Launchpad App
- Utest Apphance
- App Blade
AppSendr is a Mac application that provides tools for OTA installs. This service provides a Web and command line app that lets developers share ad hoc builds with testers - and it's free absolutely and is written on Ruby.
A service called TestFlight Beta also allows developers to send out OTA installs to beta testers. TestFlight also helps to collect UDIDs and can send out push notifications when updates are available tracking all the associated analytics.
BetaBuilder is a Mac OS X tool designed to simplify creating and distributing beta releases of iOS apps. It's available here on github. It's similar to TestFlight, but not as comprehensive as TF. However, it is free and the source is available.
Hockeykit is an iOS developer framework from buzzworks.de that provides support for ad hoc OTA installations. It consists of a server and client framework and provides a Web interface for the beta testers. Hockey is also available here on github.
Supports: iOS and Android
The Beta Family is a crowdsourcing platform for beta testing iOS as well as Android applications. Developers can test apps on real people and get an honest opinion on the user experience. More about Beta Family is here.
Supports: iOS and Android
BetaGuide provides a semi complete iOS distribution OTA along with user Touch data, CPU Usage and Session recording. The integration of BetaGlide is simple with minimal or no coding requirement. More about Beta Glide is here.
Supports: iOS and Android
Harnessing the power of desktop browser, Launchpad App comforts the users by downloading and installing test applications on their device simply by browsing to a URL.
Launchpad App presents them with a mobile optimized webpage and complete installation instructions as per the tester’s operating system.
Supports: iOS and Beta support for Android
uTest Apphance is a set of tools that allows mobile app developers to distribute builds over-the-air, get in app bug reporting, collect extensive crash reports and obtain user feedback from within the app. More info can be found here
Supports: iOS, Android, Windows and Unity
Appaloosa-Store allows you to create your own store and give access to the members of your organisation and beta testers, only authenticated users will be able to see, browse and install any of the apps available in your store.
Your store is completely secured over the cloud with all logs and reports created on demand. Another factor is that it also powered by Jenkins plugin and supports both Android and iOS.
Supports: iOS, Android and Windows
AppBlade automates and simplify various steps across iOS, Android, and BlackBerry to ensure that users are able to install applications just like TestFlight and other OTA services.
This includes functions such as automatically collecting the UDID from an iPhone to validate that applications can be installed without issue.
Automating an A/B testing or UAT is one of the fail-safe methods for testing mobile applications and web applications alike. However, when I wrote this article one conclusion that prickled my thoughts was,
It is wise to automate than regret later!
Have an appie time! J