Evo.Do - Good Idea, 10 Years Too Early.

As I'm writing this, Nataly Bendersky Shalem and I are in the process of shutting down our company, Evo.Do.

We started Evo ~15 months ago (it was called "Autoplay AI" back then) after a few months of brainstorming and talking to smart people. Nataly and I have known each other for almost a decade and worked on a bunch of passion projects over the years - we knew we wanted to build something together even before we knew what problem we'd be tackling. I quit my role as CEO of Aperio Systems and we hit the ground running.

Evo was based on a simple idea:

In 2017 Reinforcement Learning-based AI started beating top human players in complex video games. The AI trained on its own, through self-play, with almost no human help. And yet game testing (making sure there are no crashes, every level can be completed, you can't run through walls etc.) is still done manually, by disengaged outsourced workers. 

This can't continue. It's obvious that 10-20 years from now every game, app and website will be tested mostly by intelligent and adaptive bots. It just doesn't make sense to use humans for such simple tasks.

That's what our bot could do. Most of the time ;)

And we wanted to make it happen. Starting with game testing, then mobile apps, then VR applications, then websites, then the world :).

It fit us well as founders, with Nataly's deep background in game development and my experience with AI and data algorithms - and of course with both of us being avid gamers. 

We had an interesting idea, utilizing cutting-edge research and tech, and brought to the table industry expertise, experience leading a startup, zero-to-product know-how and a good track record of working together. Sounds promising, right?

The Tel Aviv University Ventures Fund thought so and accepted us to the inaugural batch of the Xcelerator program (which included 7 deep tech startups) - less than 2 months after starting the company. We got our first pilot. Expanded the team. 

In October we got into Y Combinator (batch of winter ‘19) - the birthplace of giants like AirBNB, Dropbox, Twitch and many others. I heard about Y Combinator 8 years ago, via Paul Graham's (YC's co-founder) amazing blog, and participating was on my "life bucket list" - achievement unlocked :). We moved for a few months to the Silicon Valley, went through the YC program, got some more pilots, met tons of amazing people, heard a lot of (sometimes contradictory) advice. Met dozens of investors during the several weeks of fundraising (including the fabled Sequoia Capital), raised some money - less than we had hoped but more than enough to stay alive.

Our YC group

"Wait a second," you might say. “Why is this a postmortem? Didn't you just tell me about all the amazing progress you had?"

Well, things are never as perfect as CEOs make them sound :). We did get some good results and recognition - and in certain areas we were very strong. But other areas were moving too slowly.
Some of the bigger challenges were:

 - Initially I assumed that training bots to perform simple tests would be 100 times easier than training bots to play the game better than humans. In certain ways that's definitely true (amount of data/time of exploration needed, size and complexity of the neural network that contains the bot brain etc.), but in other aspects the challenges we faced were similar or as hard as OpenAI's (real-time integration with the game graphics engine, strong non-linearity of the action-response relation etc.). Don't worry if you don't know these terms - it's just crap that Reinforcement Learning professionals have to deal with to make anything work. The tech is still very rough and not mature - so unsurprisingly we had to find creative solutions or hacks to get the initial version of the AI working.

 - Since we really understood the technical details and challenges of our customers, once we got to talk to the right person in the game studio we always found common language and they usually ended up being impressed. Unfortunately, that's not at all the same as closing a paid contract. We got people excited - but weren’t good enough at converting that excitement into revenue. In addition, the initial product was technical and industry specific - so bringing a sales expert with no game development understanding didn't work.

 - Fundraising is really hard when you have the soul of a tech guy :). I can talk for hours about the tech and the vision and the future and how self-learning AI will revolutionize the world - because I have a pretty clear, basic-principle-based understanding why the future has to be this way - it’s inevitable. But when I need to convince a potential investor that Evo is going to have a 100M yearly revenue I am acutely aware of the uncertainty and variance of events leading to this specific future (despite the effort we put into planning and data collection). Although I still believed we were a good deal (despite the risk) - the crazy 4-investor-meetings-per-day weeks took a toll on me.

 - Evo's promise was a revolution in the way human-facing software is tested. That's both a blessing and a curse. It's hard to convince potential customers to radically change the way they define tests, how they think about what's considered "tested" and many other product issues.

So was it that bad?

It's hard to say - every startup is unique - but I believe we were somewhere in the middle of the pack in terms of both success and hardship compared to companies at our stage. Not amazing, not bad. 

In any case - that’s not why we're closing.

When we got back from the valley, disaster struck. Due to a personal tragedy Nataly could no longer stay in Evo. She offered to give up her stake - so I can continue working on the company on my own - but I decided the right choice was to close the company altogether.

You see, unlike some “advice” you might hear in the startup ecosystem about how one should treat “other people’s money”, I feel very very (very) responsible - even a little guilty - for every investor dollar I spend (especially with much of Evo’s money coming from angels). To put Evo on track for (potential) massive success the company had to undergo a complete overhaul:

 - At least 1 (preferably 2) additional co-founders had to join - both with a background in the gaming industry (to replace Nataly - as it’s a very bad idea to build a product for an industry you don’t have experience with).

 - I would step down and become CTO (letting one of the new team members take the lead).

 - The product and value proposition had to be shifted and focused.

 - Some of the tech shelved - awaiting for RL algorithms and implementations to mature.

 - The marketing approach had to change from top-down to bottom-up etc.

Each part would have been hard - but possible. Getting all of them “right enough” to achieve meaningful traction and/or additional investment before running out of resources would have been very unlikely - but still possible. The thing is - I felt strongly that this is not what our investors signed up for.

It would have been a different team and leadership, a different product, different go-to-market strategy - and most importantly a much greater risk. I couldn’t tell them with a straight face “don’t worry, everything will still be ok”. I felt that the honest thing to do is give their money back.

Some random final thoughts:

 - Complete trust and compassion between co-founders are soooo important. That was (and still is) our biggest strength as founders.

 - Don't know how they do it in a 10 minute interview, but YC seem to choose way-above-average founders. Most batch-mates we connected with were super smart, driven and nice. I was proud to be in the room :).

 - I was surprisingly unafraid of the lack of stability, salary or clarity that came with being both your own boss and your own employee. I love planning for myself, managing myself and being mad at myself when I’m not up to my own standards :)

 - I don't regret the ~15 months I've "wasted" working hard on Evo.Do. I've learned a ton about startups, about the Valley, about sales and adoption hurdles, about family and partnership. Not to mention teaching myself enough Reinforcement Learning to actually apply it in a real product.

 - Despite the many challenges and imperfections I encountered while trying to build a real product based on Reinforcement Learning algorithms (and there are many such challenges), I still believe RL will have a huge impact on the world in the next 20 years. Hence I'll probably keep doing AI in the near future - because it's the single highest impact technology at the moment.

This short post cannot describe the full 15-month adventure that was Evo, but it does paint a rough picture of the ups and downs that we felt on a weekly basis. I still don't know if it was the RL technology that needed to be more mature, the market more open to autonomous software, or maybe the founders needed to be more experienced... Probably all 3 could use another 10 years of progress :)

I'd like to say thank you to all the friends and colleagues who listened to ideas, sometimes supporting and encouraging, sometimes telling me that I'm full of shit - I appreciate both. Thank you to the many fellow entrepreneurs - both in Israel and abroad - who helped us with advice, feedback and occasionally a useful connection. 

And last, I'm very grateful for the investors who believed in us at such an early stage. Though they are getting most of their money back, I'm still very sorry we didn’t make it work. I appreciate each and every one of them.

Until next time, may you take the leap. Even if you end up falling back to the ground - for a little while you'll be flying.

Michael.

{If you find my ideas or analysis interesting - consider subscribing (box on the right). You'll never miss a post and I'll know I'm not talking only to myself :) }