Table of Contents
Some Bugs are Hard to Diagnose
When you start to diagnose software problems reported by your app’s users, you may find yourself at a loss. What is causing such odd behavior? Is it specific to the user’s phone, or an outdated version of the app, or something else? It’s hard to know.
However, there are plenty of data points that can lead you down the right path. In addition to device logs, the following 8 data points will help you identify the root cause of software problems.
Don’t worry if all of these data points seem overwhelming! We have an easy way for you to collect all of these data points and review them alongside your users’ bug reports.
Diagnose Software Problems with These Data Points
Device Model
When developers write software, they typically have a hit list of target devices that they must support. Certainly, you want your app to work on every mobile device, but each manufacturer and model can have its own quirks. From different hardware capabilities to screen size and resolution differences, to peculiarities with specific hardware sensors, the phone itself has a big impact on how your app performs.
You may find that a user experiencing a bug is using a device model you’ve never tested. Once you discover the device model, you’ve still got some homework to do. How many of your users have that device model? Is it even worth supporting? If it is worth supporting, how can you get your hands on one? Older device models may not be available in stores, meaning you’ll have to search Amazon resellers or eBay listings to find a second-hand phone. If you can’t find the device anywhere, you can see if a device emulator exists for the relevant model. Alternatively, device testing services like AWS Device Farm offer remote access to real devices. The device in question may be available on AWS Device Farm.
Operating System Version
Similarly, the operating system (OS) version installed on your user’s device can have an impact on app behavior. Singling out the device model alone will not necessarily tell you the OS version being used on the device. Plenty of manufacturers now push out regular OS updates, which users may or may not install. Therefore it’s important to determine what OS version the user’s device is running.
The Android SDK Manager and Apple Xcode both provide emulators capable of running previous OS versions. For web apps, Microsoft offers virtual machines for testing older versions of Internet Explorer and Edge, or you can use a cloud-based service like BrowserStack to access any web browsers you need to test. Once you have a way to test the specific OS version, fire up your app and see if you can reproduce and diagnose the bug your user reported.
Application Version
While you’re checking software versions, ensure the user has the latest version of your app installed. Just like they can ignore OS updates, users may ignore mobile app updates. If you introduced new API calls or breaking changes in the latest release of your app, a user hanging on to an old app install may be the source of the problem.
Thankfully, the fix for these types of bugs is the simplest. Inform your user of the app update, and walk them through downloading and installing the latest version of your app. Once your latest and greatest product is on their phone, they can tell you if they still see the problem.
User Account Info
Reported software problems can also be user-specific. For example, perhaps something corrupted data related specifically to the user’s account. A bad database record, unacceptable user input, or other unexpected circumstance can cause strange problems that are difficult to reproduce. If the user gives you their account username, ID, or email address, you can review any data you have for their account. While not a common cause of problems, an account review is a good sanity check when you see data-related issues or unexplainable weirdness that only happens with a certain user account.
Diagnose software problems like this by manually reviewing database data, exporting data for review from the user’s device, or—with their permission—log in to the user’s account to see if you can replicate the issue. If the problem appears to be with the user’s device copy of data, simply having them log out and log back in to your app may fix things. Unfortunately, if the problem is with your system’s data, you will likely need to manually fix the data. Additionally, depending on the nature of the problem, you may need to track down the root cause and implement a software fix even after resolving the issue for the reporting user. That is, you will want to prevent this from possibly happening to other users in the future.
Battery Life
Battery life of the user’s device is also worth examining. If the device is plugged in and charging, is running low on power, or is in a low power mode or energy-conserving state, you may experience problems in your app. Knowing the user’s battery level when they reported a bug helps you diagnose software problems related to how your app performs in a low-power state. This is especially important for resource-intensive applications that perform a lot of processing, generate a lot of network activity, or keep the screen on for long periods of time.
Replicating issues related to battery life is as simple as draining your battery. Playing a game or watching a video for a while is a nice leisurely way of draining a mobile device battery. Once you have a similar battery level as the user reporting a problem, see if you encounter the same problem they reported. If so, you have a few options for handling power-related issues. You can display an alert in your app that encourages users to charge their phone before continuing. Alternatively, you can detect a low battery level, say less than 20% charge, and disable or postpone power-hungry tasks in your app. Turning down graphical effects like animations can also help make the most of a dying battery.
Disk Usage
There’s also a few types of storage on mobile devices to be aware of. Permanent storage is referred to as “disk” (hard disk) storage. This is where apps, files, documents, and media you have on your phone are kept. If your device runs out of disk space, apps will begin failing in interesting ways. Random crashes or apps that seem to do nothing can be attributed to a full disk. If there’s no free space on the file system, your app will not be able to save data to the device.
To make the most of disk usage, alert the user when they are running out. Also try to clean up old files you create whenever they are no longer needed. A great example of both of these behaviors is Google Photos. Since Photos backs up media to a cloud-based account, it will often prompt users to delete their safely backed up photos to regain some disk space on their phones.
Memory Usage
The other type of storage on mobile devices is referred to as “memory” or RAM. This storage is used by the operating system and apps that are currently running in the foreground or background. If you run out of memory on a device, you’ll likely have to close some running applications or restart the device to regain space. When your device runs out of memory, apps may become unresponsive, crash, or be incapable of displaying data. If you’re completely out of memory, your app may not even start.
Similar to disk usage, you should notify the user if their device is low on memory. For some lower-end devices, this may be unavoidable. Cheaper phones have less memory to decrease the cost of manufacturing. However, regardless of specs, you can make the best of the memory you have. Close file handles (file references) once you’re done with them, use buffers and pagination or chunking of data to keep as little data sitting in memory as possible. Reviewing how you handle large files, images, videos, or audio files is a good place to start if your app appears to be a memory hog.
Network Connectivity
Finally, it’s important to ensure that your user has appropriate network connectivity. Without this, your application may fail to sync data with your web server, display web pages, send messages, or receive push notifications.
If there is no cellular or WiFi connectivity available, it’s best to inform the user of the problem. Telling them they have no Internet connectivity will prompt them to resolve that issue instead of blaming your application for poor performance. However, it’s still important to understand how your app behaves without network connectivity. You can implement request queues to store API calls until the device regains an Internet connection. You can also implement retry logic if a web request fails. For example, you may have a request retry up to three times to overcome temporary network issues. You can also have request retries progressively “back off” (take longer to retry). Emulators can simulate some network connectivity problem scenarios to help you diagnose software problems.
Getting the Data to Diagnose Software Problems
Collecting all of the aforementioned information can be a chore. It requires either automating data collection from your users, or having them answer a number of questions. Interviewing users is also prone to error. They may not remember their battery life at the time they encountered a problem. They definitely won’t know what their device’s memory usage looked like. The only way to ensure you get accurate data points to diagnose software problems is to automate data collection as part of customer feedback. Bug reports should always contain the data points you need to perform root cause analysis on reported defects.
Find What You Need with Critic
This is exactly why we made Critic, an automated way to gather actionable customer feedback. Critic supports customer feedback and bug reporting for mobile, desktop, and web applications. Every report sent by your users will include the data points mentioned in this article, as well as application logs. Additionally, you can configure Critic to send file attachments of your choosing—screenshots, database dumps, or anything else useful—and product-specific data like account IDs or user session info.
Try Critic free for 30 days. If you like it, we offer monthly pricing that scales with the number of products you support. If you don’t see a plan that works for you, let us know and we will come up with something that fits your needs.