The following is a true story. It’s nothing I am proud of, but I want to share the story of our launch anyway. Maybe someone reads it and avoids failure in the future. Or maybe someone knows how to prevent such a situation and shares that knowledge.
We had been building and testing our app treffn for almost a year. More than a hundred early adopters were using it on a regular basis. The user feedback was generally positive, early suggestions for improvements had been addressed. Our backend had not shown a single problem for months – neither bugs nor problems under load. We felt we were ready to launch.
So we did what most careful app publishers do – we did a Soft Launch. Soft launches are a good idea for several reasons, but the main reason is this: If there is a problem somewhere in your system, it’s more likely to be found with a larger user base than during a beta like ours. And then you can fix those problems so they won’t affect your main launch – the one you want to succeed and the many, many new users you want to bring on board.
We decided to launch in the German-speaking region, in Germany, Austria and Switzerland (we are based in Berlin, Germany). That’s a big area to launch in with a population of almost 100 million. But since our app is iOS-only and available only in English, we effectively targeted only a small fraction of that.
And everything worked just fine. With the soft launch, we added a significant number of users and our backend coped with them without problems. We got some constructive user feedback, squashed a few bugs in the app – everything was lining up as planned for the main event.
Between the soft launch and our main launch, we introduced four new versions. Each version had a few more features and a few less bugs. We really felt ready to launch.
About a week before the launch date, we uploaded the version we wanted to launch with to the App Store. We had thoroughly tested it with no problems found. It passed Apple’s automated tests without a hitch. We tested that exact version again using Apple’s Testflight. No problems, we were ready to go. So we submitted our version to Apple for review, expecting no problems.
Before you can publish an app in the Apple App Store, it needs to be reviewed by Apple. These reviews can take between a day and a week but are usually done within 2-3 days. No one knows exactly what Apple does in those reviews. But Apple are usually pretty good at spotting problems and will reject an app if any problems were found. If an app is rejected, Apple usually sends back note with the reasons for the rejection. It’s not very detailed, but enough to get an idea of the problem and how to fix it.
But our app sailed right through the review process – Apple approved it in three days. No problems, just as we expected. We really were ready to launch, we only had to flip the switch and go live!
After you flip the switch, it can take up to two hours until your app is actually available in the App Store. So we flipped in time to be ready for the official launch event. We pressed refresh in the App Store until finally, we found our app and could download it from the official App Store.
That’s when we noticed a big problem: On our test devices with iOS 10 – the exact same ones which we used before going live – the app crashed during the user onboarding. And every time it was started afterward: Launch – 1,2,3 – crash. Our users could not even type their names – the app just launched and closed immediately afterward.
Our help desk got the same feedback: Any user with iOS10 had the same problem. Users who had not yet upgraded from iOS9 were fine, but that was only about 25% of our installs.
Panic kicked in. What did we miss? Of course, we had tested on devices with iOS10. Our app treffn had been running on iOS10 without any specific problems since the first beta of iOS10. And every version of iOS since.
Fortunately, it did not take long to find the error: It turned out to be not a bug, just some missing text string regarding the permission to access the user’s address book. How on earth did we miss that?
Turns out, we did not. Weeks earlier, we had a version rejected by Apple for a similar problem (“Missing usage description”). So we were well aware of that requirement, and we added those text strings everywhere. I am confident to say that because any version without these strings would have crashed immediately after start – just like the version in the App Store. No testing would have been possible without these text strings in the app.
So we added that text string to the app, uploaded it to Apple again and submitted it for review. And then the wait began. 1 day, 2 days, 3 days – finally, approval! Three days after the launch, we had a version in the App Store that actually worked for all our users.
We pretty much lost all of our new iOS10 users of those first three days. Horrible for all of us, but I cannot blame them – why would you stick with an app that does not work? That pain is augmented by the fact that we still do not know how that happened. And if you don’t know what the problem is, you cannot fix it.
After checking and double-checking everything, we are still not sure what we did wrong. We can only speculate that Apple changed the behavior of iOS with some update but did not update their automated checks. So when we uploaded our version, it was not checked for that particular text string. But once installed on a device, that device did check and terminated the app if that string was missing. If that’s indeed true, that was really bad luck. If not, we remain puzzled.
Can anyone help? Have you ever run into a similar problem? Did you find the cause? We would love to exchange experiences and lay this to rest, once and for all. Just send me an email at [email protected]