MarketJS
  • MarketJS
  •   Contact  
  •   Case Studies  
  •   Blog  

Product launch: Triumphs and Tribulations

Jan 24, 2017
7 minute read

Alt text

Recently we launched a consumer based web-app on a high-traffic channel. It was a 4-day, staying up late till 3-am nerve-wracking experience, with highs and lows.

More so that try to analyze what went wrong, and what went well, I would like to focus more on the journey, the emotions and try to share with everyone, what it feels like launching a product.

We were tasked to build the web-app two months ago, on a "moving target" basis. That means, there was an end goal, but it wasn't clear what steps had to be taken to get there, due to ever-changing requirements - the goals change every day, sometimes twice a day.

The high level architecture looks like this - there are three separate systems:

  • System A, a 3rd party Salesforce app
  • System B, a cloud platform that provides login systems, event loggings and analytics
  • System C, a consumer facing web app

We were tasked to build C, and connect them to A and B.

So we started out the development, breezing through the first two months with moderate stress (which includes the daily milestone changes). Everyone thought the product looked good, but it wasn't tested against real users yet.

3 days before launch, we started getting concerns about the stability of all 3 systems. If any one system goes down, it means a big service disruption to end users. We started thinking about full-on stress tests and how to simulate a real user load on the front page.

I cracked up a simple load test, hammering our web-app with 5000 requests per second for 36 minutes. It was the most expensive load test I ever had to make.

Alt text

(Whew), the results looked good. We had a failure rate of about 0.02%, a negligable amount. But the loading time still wasn't good enough. We had to start embarking on using an ajax-based loading system, which promised to cut the loading time by 50%. The thing is, there was no time to complete the ajax loading system by launch day, because we had other issues to handle: more requirements we just piling up on launch day itself.

Come launch day, we went ahead without ajax loading, and it turned out ok. However, a bigger issue occured: there was some miscommunication about a certain mechanic for System C. It caused an overflow of free items given out to end users. About 900 were able to acquire some real items for free within the first 10 minutes of launch. We ended up taking some heat and monetary loss. Ouch.

Day 2 was all disaster management, the first 8 hours. We had to build custom patch, and root out the end users who were deemed worthy of the real items. A lot of customer support work :(

On the 2nd half of Day 2, we scrambled to fix the overflow issue. First test looked promising, but we couldn't plug the patch into the live version yet, until the next evening. In the meantime, we had to fix other logic issues in the end-consumer templates.

Day 3 of launch, the overflow still managed to happen within the first 10 seconds of a mini-product going live. We figured out that simultaneous writes to the database caused contentions. The server couldn't take writes of more than 5 per second. We spent the next 8 hours building a sharding counter for our systems, to ensure more efficient writes. 3 am woes. Plus there was more logic errors in the end-user templates (causing some users to view unwanted UI elements). Mini victory: we finally managed to complete the ajax loading system in time for a midnight patch.

Day 4 looked more promising. We had a item come up at 6pm, with 100 people being able to acquire it. Within 2 minutes, it was gone: with zero overflow. Yay! But wait, yesterday's UI fixes didn't show. Help! Luckily we found a simple fix on the backend, and the issue was solved within 5 minutes. What a scare!

At the end of Day 4, everyone was weary from the product launch. Because everything finally stabilized (quasi), the team finally tasted some success, and it was fulfilling.

I let everyone leave by 8pm. We'll see what happens tomorrow (fingers crossed).

Life 

About the blog

MarketJS is a company that leverages HTML5 to provide cross-platform solutions to clients. We have been serving clients since 2012.

Here's a summary of what we do:

  1. We operate a B2B marketplace for other companies and brands to license high quality games.
  2. We help companies reskin or rebrand existing games from our portfolio.
  3. We launch white label game portals for clients.
  4. We design and develop bespoke games for brands, which include advergames, lead generation games, newsgames, and social impact games
  5. We develop game solutions for taxis, airlines, hospitals, schools and more.
  6. We build playable ads for app and gaming companies

This blog is dedicated to HTML5, game development, product design, startups, and life in general.

If you're a large organization interested in using HTML5, we highly recommend reading our case studies.

How to reach us

Subscribe

Contact

Follow

Recent blog posts

5 Design Tips For Real Money HTML5 Games
4 Common Missed Opportunities in Reskinning Games for Brands and Agencies
Split Screen Chromecast HTML5 Games
Funny Faces: Spatial Memory Game for Kids
HTML5 Game for Cognitive Training
Classic Game of Tic Tac Toe with New Fun Features
4 in a Row: Two-Player HTML5 Game with a Chat System
Smart Soccer: Physics-based Game with a Smart AI
New 3D Game: Jumpee Land
Escape from Aztec: New and Exciting Endless Runner Game
Social Blackjack: Card Game with Friends
Wasteland Warriors: a real time multiplayer game
We built a Google Chromecast game
Rising Cost Per Install In The App Economy
New HTML5 Game: Monsters and Cake
New HTML5 Game: Street Fight
New HTML5 Game: Taxi Pickup
Merchandising done right
Superbowl Ad Recap : Clash of Clans
The Old Game Flow Design
New HTML5 Game: Guess The Celebrity
New HTML5 Game: Leave Me Alone
HTML5 Endless Runner: History and Design Thinking
New HTML5 Game: Snowball Office Fight
Thinking HTML5 with castle walls and moats
Thoughts on Supercell videos
New HTML5 Game: Hidden Objects Pirate Adventures
Would a brand spend $20,000 to acquire 10,000 users?
New HTML5 Game: Banana Jungle
New HTML5 Game: Snowball Fight
New HTML5 Game: Totem Volcano
New HTML5 Game: Shark Attack
Good old times with OpenRA
New HTML5 game: Pop Star Dentist 2
Indie Reality Check: Guidelines
New HTML5 game: Slots Beach
HTML5 Game Development Workshop
How we can all learn from Jay-Z
Product launch: Triumphs and Tribulations
3 AM days and ways to avoid them
When the errors keep coming, what do you do?
What a great producer does
Mini-Detachment Experiment 1: Stop reading tech news
New game: Monster Mash
Gmail is winning on mobile
Timely donation
Why software matters
Serendipity as a game mechanic
A blast from the past: Buenos Aires
Upcoming event worth attending
New game: Sector 49
Make meaning
New game: Vampire Dress Up
New game: Kitten Bounce
New game: Ships and Monsters
iOS7 Mobile Safari: Big Branding Opportunity
New game: Mouse and Cheese
New game: Whats My Brand?
New game: Royal Princess Dress Up
New game: Feed the Grandma
New game: Pop Star Dentist
Mobile first mentality
How to focus
On raising money
98 percent of everything is crap
Common Fallacy #1
Offline advertising done right
Blogging from the phone
Re-write
Avoid gatekeepers, and focus
Refresher - Mobile HTML5 browsers
A dataset that I enjoy
On the Paul Graham accent debacle

All tags (39)

Acquisition  Advertising  Artists  Artwork  Blackberry  Card  Casino  Causes  Christmas  Chromecast  Cost Per Install  Design  Entrepreneurship  Firefox  Focus  Games  HTML5  Hidden Object  Insane  Licensing  Life  Marketing  Merchandise  Multiplayer  NSFW  Opera  Pirates  Project Management  Real World  Self-Development  Server  Soccer  Strategy  Supermarket  Tizen  Travel  Venture Capital  Workshop  iOS7 
Company
  • About us
  • Jobs
  • Contact us
  • Terms of Service
  • Privacy
  • Blog
  • Case Studies
  • 12 Ways To Work With MarketJS
  • Testimonials
  • Free HTML5 Games
  • Press
  • Podcast
Services
  • Game Reskin Services
  • Marketing Games
  • Microsite Games
  • HTML5 Playable Ads
  • Messenger HTML5 Games
  • Custom Development
  • Lead Generation Games
  • Game Leaderboard System
  • Game Login System
  • Contactless Digital Signage Games
  • Digital Signage Games
  • Private Label Games
  • Telco & mobile operators
  • Airlines and IFE providers
  • Bespoke Games
  • Google Play Instant App
  • Game Feed API
  • Trade Show Games
  • Carnival Games
  • Exhibition Games
  • Corporate Event Games
Categories
  • Web Games
  • Mobile Games
  • Browser Games
  • Retro HTML5 Games
  • Classic HTML5 Games
  • HTML5 Mobile Games
  • Kids Games
  • Educational Games
  • Multiplayer Games
  • Puzzle Games
  • Branded Games
  • Mini HTML5 Games
  • Browser Games For Sale
  • Offline HTML5 Games
New solutions
  • Skill Based Games
  • NFT Game Development
  • Skill Based Game Development
  • Real Money Games
  • Skill Based Games Vendor
  • Skill Based Games Supplier
Other solutions
  • Android Games
  • iOS Games
  • iPhone Games
  • Android APK
  • iPad HTML5 Games
  • iPhone HTML5 Games
  • Android HTML5 Games
  • HTML5 Games
  • Mobile HTML Game
  • H5 Games
  • OEM Games
White Label Solutions
  • White Label Games
  • HTML5 Game Library
  • HTML5 Games Portal
  • Mobile Games Portal
  • Subscription Games Portal
  • SMS Games Portal
  • Games For Online Events, Virtual Conferences, Exhibitions and Webinars
More white label games
  • Spin The Wheel
  • Crack The Egg
  • Roll The Dice
  • Trivia
  • Board Games
  • Slots
  • Endless Runner
  • Grab and Win (Claw Machine)
  • Select a Picture and Win
  • Memory Match and Win
  • Scratch and Win
  • Gamification With Coupon Codes, Vouchers and QR Codes
Mobile Analytics
All content on this website, excluding any 3rd party logos is copyright MarketJS.com, 2011 - 2022