MarketJS
  • MarketJS
  • Sign in
  • Register
  • Cart
  •   Contact  
  •   Case Studies  
  •   Blog  

Product launch: Triumphs and Tribulations

published 01-24-2017

Alt text

Recently we launched a consumer based web-app on a high-traffic channel. It was a 4-day, staying up late till 3-am nerve-wracking experience, with highs and lows.

More so that try to analyze what went wrong, and what went well, I would like to focus more on the journey, the emotions and try to share with everyone, what it feels like launching a product.

We were tasked to build the web-app two months ago, on a "moving target" basis. That means, there was an end goal, but it wasn't clear what steps had to be taken to get there, due to ever-changing requirements - the goals change every day, sometimes twice a day.

The high level architecture looks like this - there are three separate systems:

  • System A, a 3rd party Salesforce app
  • System B, a cloud platform that provides login systems, event loggings and analytics
  • System C, a consumer facing web app

We were tasked to build C, and connect them to A and B.

So we started out the development, breezing through the first two months with moderate stress (which includes the daily milestone changes). Everyone thought the product looked good, but it wasn't tested against real users yet.

3 days before launch, we started getting concerns about the stability of all 3 systems. If any one system goes down, it means a big service disruption to end users. We started thinking about full-on stress tests and how to simulate a real user load on the front page.

I cracked up a simple load test, hammering our web-app with 5000 requests per second for 36 minutes. It was the most expensive load test I ever had to make.

Alt text

(Whew), the results looked good. We had a failure rate of about 0.02%, a negligable amount. But the loading time still wasn't good enough. We had to start embarking on using an ajax-based loading system, which promised to cut the loading time by 50%. The thing is, there was no time to complete the ajax loading system by launch day, because we had other issues to handle: more requirements we just piling up on launch day itself.

Come launch day, we went ahead without ajax loading, and it turned out ok. However, a bigger issue occured: there was some miscommunication about a certain mechanic for System C. It caused an overflow of free items given out to end users. About 900 were able to acquire some real items for free within the first 10 minutes of launch. We ended up taking some heat and monetary loss. Ouch.

Day 2 was all disaster management, the first 8 hours. We had to build custom patch, and root out the end users who were deemed worthy of the real items. A lot of customer support work :(

On the 2nd half of Day 2, we scrambled to fix the overflow issue. First test looked promising, but we couldn't plug the patch into the live version yet, until the next evening. In the meantime, we had to fix other logic issues in the end-consumer templates.

Day 3 of launch, the overflow still managed to happen within the first 10 seconds of a mini-product going live. We figured out that simultaneous writes to the database caused contentions. The server couldn't take writes of more than 5 per second. We spent the next 8 hours building a sharding counter for our systems, to ensure more efficient writes. 3 am woes. Plus there was more logic errors in the end-user templates (causing some users to view unwanted UI elements). Mini victory: we finally managed to complete the ajax loading system in time for a midnight patch.

Day 4 looked more promising. We had a item come up at 6pm, with 100 people being able to acquire it. Within 2 minutes, it was gone: with zero overflow. Yay! But wait, yesterday's UI fixes didn't show. Help! Luckily we found a simple fix on the backend, and the issue was solved within 5 minutes. What a scare!

At the end of Day 4, everyone was weary from the product launch. Because everything finally stabilized (quasi), the team finally tasted some success, and it was fulfilling.

I let everyone leave by 8pm. We'll see what happens tomorrow (fingers crossed).


Tags:  Life
Tweet this post

About the blog


MarketJS is a company that leverages HTML5 to provide cross-platform solutions to clients. We have been serving clients since 2012.

Here's a summary of what we do:

  1. We operate a B2B marketplace for other companies and brands to license high quality games.
  2. We help companies reskin or rebrand existing games from our portfolio.
  3. We launch white label game portals for clients.
  4. We design and develop bespoke games for brands, which include advergames, lead generation games, newsgames, and social impact games
  5. We develop game solutions for taxis, airlines, hospitals, schools and more.
  6. We build playable ads for app and gaming companies

This blog is dedicated to HTML5, game development, product design, startups, and life in general.

If you're a large organization interested in using HTML5, we highly recommend reading our case studies.

How to reach us



Subscribe

Contact

Follow

Recent blog posts

4 Common Missed Opportunities in Reskinning Games for Brands and Agencies
Split Screen Chromecast HTML5 Games
Funny Faces: Spatial Memory Game for Kids
HTML5 Game for Cognitive Training
Classic Game of Tic Tac Toe with New Fun Features
4 in a Row: Two-Player HTML5 Game with a Chat System
Smart Soccer: Physics-based Game with a Smart AI
New 3D Game: Jumpee Land
Escape from Aztec: New and Exciting Endless Runner Game
Social Blackjack: Card Game with Friends
Wasteland Warriors: a real time multiplayer game
We built a Google Chromecast game
Rising Cost Per Install In The App Economy
New HTML5 Game: Monsters and Cake
New HTML5 Game: Street Fight
New HTML5 Game: Taxi Pickup
Merchandising done right
Superbowl Ad Recap : Clash of Clans
The Old Game Flow Design
New HTML5 Game: Guess The Celebrity
New HTML5 Game: Leave Me Alone
HTML5 Endless Runner: History and Design Thinking
New HTML5 Game: Snowball Office Fight
Thinking HTML5 with castle walls and moats
Thoughts on Supercell videos
New HTML5 Game: Hidden Objects Pirate Adventures
Would a brand spend $20,000 to acquire 10,000 users?
New HTML5 Game: Banana Jungle
New HTML5 Game: Snowball Fight
New HTML5 Game: Totem Volcano
New HTML5 Game: Shark Attack
Good old times with OpenRA
New HTML5 game: Pop Star Dentist 2
Indie Reality Check: Guidelines
New HTML5 game: Slots Beach
HTML5 Game Development Workshop
How we can all learn from Jay-Z
Product launch: Triumphs and Tribulations
3 AM days and ways to avoid them
When the errors keep coming, what do you do?
What a great producer does
Mini-Detachment Experiment 1: Stop reading tech news
New game: Monster Mash
Gmail is winning on mobile
Timely donation
Why software matters
Serendipity as a game mechanic
A blast from the past: Buenos Aires
Upcoming event worth attending
New game: Sector 49
Make meaning
New game: Vampire Dress Up
New game: Kitten Bounce
New game: Ships and Monsters
iOS7 Mobile Safari: Big Branding Opportunity
New game: Mouse and Cheese
New game: Whats My Brand?
New game: Royal Princess Dress Up
New game: Feed the Grandma
New game: Pop Star Dentist
Mobile first mentality
How to focus
On raising money
98 percent of everything is crap
Common Fallacy #1
Offline advertising done right
Blogging from the phone
Re-write
Avoid gatekeepers, and focus
Refresher - Mobile HTML5 browsers
A dataset that I enjoy
On the Paul Graham accent debacle

All tags (39)


Acquisition Advertising Artists Artwork Blackberry Card Casino Causes Christmas Chromecast Cost Per Install Design Entrepreneurship Firefox Focus Games HTML5 Hidden Object Insane Licensing Life Marketing Merchandise Multiplayer NSFW Opera Pirates Project Management Real World Self-Development Server Soccer Strategy Supermarket Tizen Travel Venture Capital Workshop iOS7
Company
  • About us
  • Jobs
  • Contact us
  • Terms of Service
  • Privacy
  • Blog
  • Case Studies
  • 12 Ways To Work With MarketJS
  • Testimonials
  • Free HTML5 Games
  • Press
Services
  • Game Reskin
  • HTML5 Playable Ads
  • Messenger HTML5 Games
  • Custom Development
  • Lead Generation Games
  • Game Leaderboard System
  • Game Login System
  • Contactless Digital Signage Games
  • Digital Signage Games
  • Private Label Games
  • Telco & mobile operators
  • Airlines and IFE providers
  • Bespoke Games
  • Google Play Instant App
  • Game Feed API
Categories
  • Web Games
  • Mobile Games
  • Browser Games
  • HTML5 Mobile Games
  • Kids Games
  • Educational Games
  • Multiplayer Games
  • Puzzle Games
  • Marketing Games
  • Branded Games
  • B2B Games
  • Mini HTML5 Games
  • Browser Games For Sale
  • Game Rights For Sale
  • Offline HTML5 Games
Native app licensing
  • Android Games
  • iOS Games
  • iPhone Games
  • Android APK
  • iPad HTML5 Games
  • iPhone HTML5 Games
  • Android HTML5 Games
  • HTML5 Games
  • Mobile HTML Game
  • H5 Games
  • OEM Games
White Label Solutions
  • Fortumo Games Portal
  • Credit Card Games Portal
  • WAP Games Portal
  • Java Games Portal
  • SMS Billing For Games
  • HTML5 Games Portal
  • Mobile Games Portal
  • Subscription Games Portal
  • SMS Games Portal
  • Games For Online Events, Virtual Conferences, Exhibitions and Webinars
More white label games
  • Spin The Wheel
  • Trivia
  • Board Games
  • Slots
  • Endless Runner
  • Grab and Win (Claw Machine)
  • Select a Picture and Win
  • Memory Match and Win
  • Scratch and Win
  • Gamification With Coupon Codes, Vouchers and QR Codes
Mobile Analytics
All content on this website, excluding any 3rd party logos is copyright MarketJS.com, 2011 - 2021