Sunday, August 27, 2017

Path of Exile Economy: Currency Trading

Path of Exile (POE) is quietly becoming a world-wide phenomena for the past several years. The action-RPG (think Diablo) is created by an indie game studio based in New Zealand - Grinding Gear Games - which was established in 2006 specifically to create POE - "the only game we ever wanted to make". Today the game has millions of active players and consistently rises in popularity from one expansion to another, not to mention being quite profitable (despite being technically free-to-play).

POE is considered to be geared towards a more hardcore-gamer audience - with it's famously intimidating passive skill tree, highly customizable and optimizable skills and abilities - and most importantly for this article, an incredibly rich player-based economy.

A good indication of the size and dedication of the POE audience is the wealth of independently developed tools and calculators for the game (which is also a proof of the complexity of the game - as some of these tools are practically a must for efficient gameplay).

My personal experience with POE started about 3 years ago. A friend recommended, and it's free to play, so I thought "why not?". It took me almost a week to complete the game on normal difficulty (for comparison, it takes me ~6-8 hours today), while fighting through the many complexities of the character progression and skill mechanics - and making a million mistakes along the way. I quit then, but after a while the infinite possibilities for customization of skills and character builds tempted me back. I've been playing on and off ever since. 

Any good game that has depth and complexity lets players discover more and more levels of gameplay - that naturally come with both character and player progression. Being part of the POE economy - namely trading items with other players - is an additional level of gameplay that is not directly related to the game itself, but is surprisingly rich and requires above average understanding of game mechanics to participate fully. 

I've traded more and more in the past several months and recently (at the end of patch 2.6) started "flipping" game currency. Being part of the economy transformed the way I experience the game as a whole - this article summarizes some of my observations and calculations on the subject.

POE Currency

One of the most interesting decisions Grinding Gear Games (GGG) made when designing the core mechanics of POE was removing the gold. It is almost universally accepted and expected that any RPG has some form of basic currency that everything can be traded for - the proverbial "gold coins". The characteristic that defines "gold" is that it's the fundamental trading currency, but also that it has no intrinsic functionality - thus its value is defined solely by the other valuable resources/items a player can buy with it. 
Any game where monsters occasionally drop loot and/or gold when defeated has an inherent inflation mechanism: the amount of total gold in the world is constantly increasing simply through players playing the game. Hence, the value of each gold coin diminishes - unless there are "money sinks" built in the game world economy that naturally remove some of the influx of gold from monster loot (equipment occasionally requiring costly repairs by an NPC is a popular example of such a sink - though perhaps not the best game design).
Once gold loses its value, players often turn to other currency that they use for P2P trading (like the classic Stone of Jordan in Diablo II).

GGG solved most of these problems by canceling gold all together. POE has no currency that is used only for trading - every single item in the game has some intrinsic functionality and value. Thus, there are many items that are considered "currency items" in POE - each does something different when consumed (providing a natural money sink mechanism):
My currency item storage tab (end of patch 2.6 "Legacy League"). And even more types of currency were introduced in patch 3.0.
Despite the multitude of currency items, historically one POE currency always rose as the "gold standard" - completely through natural evolution. This is a fascinating free market model that has many similarities to real world barter based economies that evolved in situations where money was meaningless - for example due to inflation or banking collapse. Today the dominant currency is "chaos orbs", with a few other types used for smaller trades, but there were times when for example "gemcutter's prisms" were the "gold standard" - so the market shifts as the game evolves.
"That's interesting, but how does it effect the player experience?" you might ask...

Trading in POE and the "Price of Exchange"

Let's say I find a good item while progressing in the game, but I don't need it at the moment (perhaps it doesn't fit my current character or maybe I already have a better one). Often I would try to "sell" it - list it publicly hoping to trade it to another player in return for some currency. The idea is that, like in any true free market, the item might not hold value for me - but does hold value for someone else, so we can make both of us better off via an exchange (assuming he gives something of value for me in return).
In principal I could try trading it for a different piece of equipment that I do need (instead of some currency items) but, just like in the real world, this type of barter has numerous disadvantages and is almost never used.

As in most RPGs, in order to trade the two players must be in the same game zone. This seemingly trivial requirement actually creates a natural friction in the trading process. Usually while I'm in the game I'm, well, playing the game - so when a potential buyer sends me a message that he is interested in something I have for sale, I must stop what I'm doing and go meet the buyer at some location (one of the towns or player "hideouts"). The process usually takes less than a minute if both players are used to trading in POE, but it's somewhat annoying and partially breaks the game flow.

So a few months back I've decided to stop selling equipment for cheaper than a certain price (1 "chaos orb"), figuring that smaller profit isn't worth my time - I'd rather keep playing the game itself. If the equipment doesn't sell for at least 1 "chaos" after some time, I'd just get rid of it to free up space in my "shop" storage.
Basically I've defined that an exchange must net at least 1 "chaos" of profit or shouldn't happen at all - that is my "price of exchange" (which varies between players of course).
In a sense this effect can be compared to the transaction cost in a real world stock market - where the exchange itself takes a small percentage of every deal as a brokerage fee, creating friction.

Currency Flipping

Currency trading in general refers to the act of trading one type of currency items for a different type - currency<->currency instead of equipment<->currency trading.
The first question someone who has never played POE might ask is "why would you trade currency in the first place? Isn't it like trading gold for silver or vise verse in a normal RPG?". Since POE currency items are consumables which have an effect in their own right - sometimes a player wants the functionality of the currency but doesn't have it (or not enough of it). That player can either keep playing/grinding for this specific currency type, but a more practical and much faster method would be to buy it.
In that sense, players who make currency<->currency trades are like money changers in the real world. You could in principle earn some Euro directly before you go on your vacation - but no one actually does that. Everyone prefers convenience - even if it means paying someone a little extra.

Currency "flipping" is the act of trading the same 2 currencies back and forth, repeatedly, between different players. Let's say we're talking about alchemy orbs (which upgrade a normal item into a random rare one - alc's for short) and chaos orbs (which transform a rare item into a different rare item randomly - often denominated simply as c). 2 weeks before patch 3.0 the rates of trade I was offering were:
1C -> 3.125alc
1alc -> 0.23C

So a player could buy 3.125alc for each C he gave me, or buy 0.23C for 1alc.

The first thing you might notice is the fact that the rates aren't equal - the transaction is not reversible (you couldn't buy back 1C with the 3.125alc you would get by selling that 1C). This is similar to the gap between the buying and selling price of real world currency at money changers: they provide value in the form of convenience so they monetize some of it by giving slightly "unfair" exchange rates.
And just like real world changes, I prefer to buy and sell the same amount of each currency - on average, otherwise I'll run out of alc's (in this example) or stockpile too much of them. More on that later.

The second thing you've perhaps noticed is that the prices are not round numbers, and it is impossible to split currency items into parts - so what does "0.23C" mean?
The trick is simple - most trades involve volumes larger than 1 of a currency item. In reality the rates are:
8C -> 25alc
70alc -> 16C

The C->alc side is clear - 8 is the lowest number you can multiply both sides by and get round numbers, but the other direction could be written as 35alc -> 8C to get the same result - so why? This is where the "price of exchange" plays a role.

I have 2 goals while trading currency in the long run:
 - Maintain an approximately even rate of inflow and outflow for each currency.
 - Make sure I earn at least 1C of profit per trade (on average) - otherwise the trade doesn't overcome the friction of exchange.

And I have 4 "dials" to fulfill them:
 - Exchange rate for C -> alc trades.
 - Exchange volume for C -> alc trades.
 - Exchange rate for alc -> C trades.
 - Exchange volume for alc -> C trades.

Note that the second goal is not trivial. If I sell 25alc's and receive 8C in return - I did not make an 8C profit. Unlike the case of selling equipment - where the gear itself held 0 value for me in practical terms (it was just laying around in my stash), so getting paid 1C meant 1C clean profit - when trading currency I'm giving away something that does hold value for me. I use alc's all the time as part of normal gameplay - and the only reason I'm able to trade them away is because I don't need more than ~20 each day, so if I have 100alc's - 80 of them can be treated as "just laying" at the moment, but might become useful tomorrow. So how can we calculate profit per trade?

Assuming the first requirement (balanced trade per currency) is fulfilled (if it's not then this business model is not sustainable anyway), we can look at a single cycle of a chunk of currency I receive:
 - Pay 16C to get 70alc's. That takes 1 transaction (our balance is now C: -16, alc: +70, transactions: 1)
 - Pay 25alc to get 8C (C: -8, alc: +45, T: 2).
 - Pay 25alc to get 8C (C: 0, alc: +20, T: 3).
 - Pay 25alc to get 8C (C: +8, alc: -5, T: 4).
Since we are looking at long term averages we can simply say it takes 2.8 alc-selling transactions to "get rid" of the chunk of alc's from 1 alc-buying transaction, and nets a profit of 6.4C per cycle:
Profit per Exchange = Profit from Cycle / Number of Transactions = 
6.4 / 3.8 = 1.68C

If we do the same calculation for an exchange volume of 35alc -> 8C, we'll get Profit per Exchange = 1.33C - despite the identical rates of exchange - because it would require 1 additional transaction to complete the same currency cycle. 
Alternatively we could ramp up the transaction volumes to get more profit per trade. So why don't we?

Free Market Forces

When a player wants to buy currency, usually he would go to the most prominent POE trading website (appropriately called and run the relevant search:

Which is ordered from best deal to worst deal by exchange rate. Obviously he would then try to buy the chaos orbs for the cheapest price - basically going from the top to the bottom (and there are much much more results than what you see here) messaging each seller until 1 responds and the transaction takes place.
Note that both of the cheapest price offers require very large trade volumes. If the buying player only needs 5C, or simply doesn't have so many alc's, these trades are irrelevant for him - so he would go on and message all the 1:3 sellers.

In order to get business, a currency trader must first offer a competitive price - preferably be in the first page of search results (just like a real product on Amazon would) - but also offer small enough trade volumes to be comfortable for most players.
So you can't increase the profit per transaction by setting a huge exchange volume - even if you were rich enough to be able to make the trades - because the vast majority of players don't want/have such large sums. 
On the flip side, the sellers that offer the lowest prices can only do that by insisting on large trade volumes to maintain the desired profit-per-transaction value - as we calculated.

Personally, I usually set a competitive exchange price for both directions of the currency flip, then tweak (usually increase) the exchange volume until I reach the 1C profit per transaction benchmark. 
Then I wait and see how many trade offers I get in each direction. Remember that the balance of currency "import" and "export" must be maintained, so I try to increase the exchange volume of the direction that's going "too well" - the larger the minimum trade amount is - the less players take it. Sometimes the exchange rates must be updated - either because I couldn't achieve balance just by volume manipulation or because the market price is shifting. 

"But Michael", you might ask, "how are the exchange prices determined to begin with? Why does 1C cost 3 alc's and not 10? or 0.5?"

Exchange Rates

Let's imagine a grossly oversimplified situation: there are only 2 POE players in the world and only 2 types of currency (chaos and alcs for example). Also let's assume both players play on similar levels of content - and have the same drop rate of currency: 1C and 4 alc's per hour.
In principal, if the 2 players wanted to trade - a reasonably fair trade might be at an 1C:4alc exchange rate - since that represents the same amount of "labor" invested in collecting this currency. A more realistic capitalist/free market approach would be to understand that the price is actually determined by the relative abundance of resources - and not by the amount of effort invested in collecting them. In our case the second approach gives us the same result - since both players have the same "earning" rate so the abundance of currency items is also 1:4. Or is it?

During higher character level gameplay, alcs are often constantly used - which means they are consumed and no longer exist in the POE economy. If, for example, the 2 players consume a quarter of the alc's they find (and approximately none of the C), the relative abundance in this POE economy would be 1:3. Which brings us back to the real case. 
The visible prices are set by players - and every single player chooses the price mostly by conforming with the average market prices (like we discussed before - when I'm choosing my exchange rates I'm only looking at what other players are offering - not trying to calculate relative abundance etc.). But, in the long run, prices seem to be closely correlated to the availability of each single resource on the market - just as they do in real life free markets (basically, if a resource is rare then the number of people willing to sell it cheaply quickly runs out). For any 2 currencies this translates approximately to:

(Price of A) /   ~   (Drop rate A - Consumption rate A) /
(Price of B)            (Drop rate B - Consumption rate B)

What are the "true" drop rates from Path of Exile monsters? That's not something GGG talk about publicly... But the trading prices are a good place to start. 

This simple calculation can also tell us certain things about the changes in player behavior patterns. For example, if the price of a certain currency suddenly raises substantially (assuming no major changes were made to the drop rates), that means they are being consumed more frequently. What might be the reason? Perhaps a portion of the players figured out a use for it in a certain situation that was not known before? 
On the flip side, if the price seems to be surprisingly low - perhaps a portion of the players figured out a method to farm that currency more efficiently than before?

Money usually tells a story.

Currency Trading Tool

I've made a spreadsheet tool to help me do the currency trading calculations we've discussed. I hope you find it useful if you want to become a currency trader yourself. It looks like this (and contains all currency types that existed in Path of Exile patch 2.6 - feel free to add the new ones of course):

The only thing that you need to update for your own case are the numbers in the white cells - the rest is calculated. In case you're wondering, these prices are from a couple of weeks before the end of Legacy league.


There are many more fascinating economic effects that I haven't touched on at all - like the cyclical dynamics of the fresh league economies (from creation all the way to integration back into Standard), the difference in valuations of items between Standard gameplay and Hardcore mode (in which characters die permanently), the visible shifts in pricing following announcements by prominent streamers or Youtubers (the so called "Mathil effect" - named after a popular streamer - that was analyzed in this video for example), "price fixing" done by rich and knowledgeable players to manipulate the market - and many others.

Path of Exile's naturally emerging P2P economy is an amazing case study for a "virtual economist" like me. It's a living, breathing free market - where every player can participate - as a buyer, a seller, a flipper, an investor, and of course value producer (finding equipment and currency by playing the game). Unlike POE, in the real world there are high barriers of entry to any of these roles (initial capital and legal approval are often needed) - but in the virtual world anyone can be anything: a true free market.
This ability to both gather data for macro calculations and simultaneously "feel the market" by being an active participant and stakeholder allows for a first hand experience of economic theories - quite a valuable lesson for a game I play to relax at night :)

Until next time, may you know the value of your time.


{If you find my ideas or analysis interesting - consider subscribing (box on the right). You'll never miss a post and I'll know I'm not talking only to myself :) }

Friday, November 11, 2016

"The Boy Who Cried Wolf" and Bayesian Probability

One of the most famous and well known fables in the world is "The Boy Who Cried Wolf". It was written by Aesop (6th century BCE) as a cautionary tale against false alarms:

"The tale concerns a shepherd boy who repeatedly tricks nearby villagers into thinking wolves are attacking his flock. When one actually does appear and the boy again calls for help, the villagers believe that it is another false alarm and the sheep are eaten by the wolf. In later English-language poetic versions of the fable, the wolf also eats the boy" (from Wikipedia)

It's a simple message, really. If you lie and get caught - people stop believing you. 

Think of your own surroundings. Do you know any "false-alarm boys"?
Maybe it's the co-worker who keeps committing to deadlines but never completes the task in time. Maybe it's the boss who summons you to a 1-on-1 meeting but postpones it time and time again. Maybe it's that friend who has an existential life crisis - twice a week, every week...

What should we do? At which point should we stop trusting them and start assuming every alarm is a false alarm? Or maybe we should be flexible and give them a second (or third) chance?
Bayesian probability might hold the answers.

As humans we must make decisions all the time. When should I leave to get to work in time? Should I go to that meetup? What career should I choose? These decisions force us to (partially) predict the future - based on our beliefs about the world. In order to decide when should you leave for work, you must have some idea of how long is the drive to the office - some knowledge about the surrounding world (often from past experience) that allows you to make a prediction: "the drive usually takes me 30 minutes so I should leave at 8:30". 
Unfortunately, we are unable to predict the future exactly. A more accurate saying would be "on average, the drive takes 30 minutes, but it depends on traffic. Most often it's between 25 and 35 minutes". So there is some probability distribution of drive length - with a mean of 30 and most of it is concentrated between 25 and 35. If we leave at 8:30 we'll often be late due to this uncertainty. Perhaps then we should leave for work at 8:25 so we have a buffer against the common fluctuations?

Let's rephrase these considerations in Bayesian notation. Our goal is to get to work in time - to minimize the chance of being late. We could leave 3 hours ahead of time - thus practically ensuring that we're at the office at 9:00, but that's somewhat extreme and inefficient. Let's say being late once a month is an acceptable rate, so we need to make the probability of being late on each day about 5%, written as: P [ late ] = 5%.
The knowledge about the world that we've accumulated so far - in our example it's our past experience of driving to work many times before - is called the prior probability distribution - or just prior. Let's say that according to our experience the drive takes longer than 35 only 5% of the days, so leaving at 8:25 will satisfy our goal.

Now imagine there's a politician visiting town. Some of the roads are blocked for security purposes, there are many people attending her rally etc. The visit is creating traffic jams around the city - making the drive to the office about 45 to 55 minutes. If we use our normal strategy and leave at 8:25 we actually have a very large probability to be late - the distribution changed because the world changed. This is written as:
P [ late | politician, leave at 8:25] > 5%
Everything after the dividing line is information about the world - in addition to our prior beliefs - which change our predictions about the future. In our case - we should probably leave for work at 8:05 on politician days to preserve the regular probability for being late ( P [ late | politician, leave at 8:05] = P [ late | no politician, leave at 8:25] ).

But what if we don't know whether there's a politician in town or is it a normal day? A political visit is a rare event so on any given day it's reasonable to assume there isn't one. Imagine you leave the house at 8:25, as always, but the met with huge traffic jams - making the drive 50 minute long. You'd probably say "what happened? It never takes so long. Is there a politician in town or something?" - because under your baseline assumptions about the world ("no politician") a 50 minute drive to work is extremely unlikely. You thus deduce that your hypothesis about the state of the world might be wrong. 

Congratulations! You now know the basic ideas behind Bayesian probability: you try to figure out what's more and less likely to happen, according to your existing beliefs about the world - both these probabilities and your beliefs change as new information is discovered. 
Now let's apply these ideas, in a slightly more formal way, to Aesop's fable.

Imagine the villagers whose herd of sheep is guarded by "false-alarm boy". When they hear him "cry wolf", should they believe him? How likely is it that there actually is a wolf?
It depends on what they think about the boy - do they trust him? Is he a trustworthy person? And how should their opinion about him change as new information becomes available?

Humans often think of people as belonging to one of several categories. Let's assume for simplicity 3 archetypical types: "completely honest and trustworthy" (with a small percentage of "no-wolf" because he might make an innocent mistake), "not trustworthy" (his word basically has no meaning - there might be a wolf and might not be, they'll believe it when they see it) and "malicious" (he's probably bullshitting us again - but it might be the one time he's telling the truth).
The villagers are good and honest people so they naturally assume other people are probably honest as well. So before they know anything about the boy they assume there's a 90% probability he's "completely honest", 5% he's "not trustworthy" and 5% he's "malicious".

Probability that the wolf actually existsProbability that there is no wolf
"Completely Honest"99%1%
"Not Trustworthy"50%50%

Under these beliefs about the boy, what are the odds there's actually a wolf when the boy raises the alarm?

P [wolf] = P [wolf | boy is honest] * P [boy is honest] +  
                                        P [wolf | boy is untrustworthy] * P [boy is untrustworthy] +  
                              P [wolf | boy is malicious] * P [boy is malicious] = 92% 

So it is highly probable there actually is a wolf - and the villagers should run to check it out.
But there is no wolf. This is new information - an alarm was raised and turned out to be false. Now it seems less likely that the boy is "honest", since a false alarm is a very rare event if he's "honest". How much less likely? 
Bayesian math gives us the answer:  

P [boy is honest | there was no wolf] = 
P [there was no wolf | boy is honest] * P [boy is honest] /
 P [there was no wolf]

I won't go into the math behind it - you can find it here - but the idea is simple: the new probability that the boy is honest (now that we know there was no wolf) is the base probability that the boy is honest (what we used to believe) multiplied by the relation between how likely the new evidence is according to the hypothesis that the boy is honest vs. its general likelihood. So, if some belief about the world predicted an occurrence to be more likely than other beliefs - and that occurrence turned out to be what actually happened - we should now have more faith in that belief about the world.

Using the numbers from above we get:
P [boy is honest | there was no wolf] = 11.4%
Wow. The villagers' trust in the boy is shattered after only a single false alarm. It is now more likely he is malicious or at least untrustworthy:

P [boy is untrustworthy | there was no wolf] = 31.6%
P [boy is malicious | there was no wolf] = 57%

Under these new views about the boy, when he now raises the alarm a second time - how likely is it to be true?
P [wolf] = 33%
Already it's more likely that there is no wolf. But maybe the damage that an undefeated wolf can cause (X) is greater than the pain of dropping everything and running to the fields (Y) - let's say 5 times greater (X = 5Y). The rational thing would still be to go check it out, for the 1/3 chance that the wolf is actually there:

Damage [responding to alarm] < Damage [ignoring alarm]
Y < P [wolf] * X + P [no wolf] * 0
Y < 0.33*5Y

Let's say the villagers indeed follow this logic and believe the alarm for the second time - and it turns out to be false again. The new opinions about the boy are:

P [boy is honest | twice there was no wolf] = 0.17%
P [boy is untrustworthy | twice there was no wolf] = 23.6%
P [boy is malicious | twice there was no wolf] = 76.6%

After 2 false alarms the villagers should be pretty much convinced that the boy is maliciously lying to them. If he raises the alarm a third time, the probability that there really is a wolf would be: P [wolf | twice there was no wolf] = 19.6%.
Even the threat of a rampaging wolf is not enough to convince the villagers to respond, since Damage [responding to alarm] > Damage [ignoring alarm].

Note that true alarms would gradually restore the trust in the boy - but, unfortunately, trust is rebuilt much slower than it is destroyed. Let's say the 3rd alarm - which the villagers probably already ignored - was true. The new opinions about the boy's personality would be:

P [boy is honest | 2 false alarms, 1 true] = 0.86%
P [boy is untrustworthy | 2 false alarms, 1 true] = 60.2%
P [boy is malicious | 2 false alarms, 1 true] = 39%

The boy looks less malicious and more simply untrustworthy - but still highly unlikely to be honest.
Once broken, real trust is hard to rebuild. It's not enough to do the right thing more often than the wrong thing. Every mistake has a substantially higher price than what's gained through honest actions. 
This is something we've all felt ourselves, the "1/2 strikes you're out" approach, and the math seems to support it...

Or does it?

Imagine the boy was actually a different kind of person, from a 4th category that wasn't even in the villagers' minds - we'll call it "good intentioned yet fallible human" - or "human" in short.

Probability that the wolf actually existsProbability that there is no wolf

Unfortunately, the villagers don't have a mental model for such a person. How would their belief change with each "wolf" cry? Using the same calculations as above we can extrapolate the villagers attitude to the "human" boy:

New Evidence"Completely Honest""Not Trustworthy""Malicious"P [False Alarm]
Initial Belief90.00%5.00%5.00%7.90%

Note how the villagers (should) gradually be convinced of the boy's honesty with successive true alarms, but their faith in him drops by 20-25% every time he makes a mistake: 

Thankfully, with every instance of correct behaviour the villagers gradually "forgive" the boy and after 6-7 true alarms he regains their full trust.

This is what the math suggests you should think about the boy in our story - as a rational person. 
Well, do you? 
Can you forgive someone who did the wrong thing once or twice - but made up for it by doing the right thing 10 times? What about 100 times? 

Until next time, may you be cautious enough to be safe from "malicious" friends, but generous enough not to lose the "human" ones.


{If you find my ideas or analysis interesting - consider subscribing (box on the right). You'll never miss a post and I'll know I'm not talking only to myself :) }