This post is designed to be a follow up to our ‘trusted volume’ framework post (link) earlier this summer. We have made a few adjustments to take into account changes in the industry as well as reader feedback.
In the original post, we used a three pronged approach to measure the reporting quality of an exchange’s volume. This included volume correlation between the exchange and a group of ‘benchmark’ exchanges, an analysis of key ratios including web traffic and trading volume, and a blended score to quantify more qualitative features of an exchange such as developer tools, trading rules, and KYC thresholds.
One aspect that we would like to clarify is that the purpose of this framework is to lay a foundation for more dependable asset level metrics based on volume. It is not to discredit any exchanges. For example, this more conservative definition of trading volume can help institutions considering ETFs to more confidently gauge an asset’s daily spot trading volume for market sizing.
In this post we’ll cover a number of updates to our original framework, including:
- Removing potential Western bias in correlation and qualitative measures
- Adding BitMEX to the benchmark set
- Including volume of perpetual futures in correlation tests
- Changing the weightings of web traffic data
Removing Western Bias
We’re constantly working to make our frameworks as objective as possible, and remove any unforeseen biases that may crop up. With that in mind, we made a few changes to make our framework less biased towards Western countries and less location sensitive.
One of the initial changes that we made was to the correlated volume metrics. To reduce some of the seasonality differences in hourly volume that may present itself when comparing Eastern vs. Western exchanges, we have now used daily volume. This should create a more holistic image of daily trading as opposed to hourly trends they may negatively impact by exchanges located in timezone outside of the control group.
Above is a look at the correlations between the volume from exchanges and the volume from our ‘trusted’ control group for a few of the more well known assets, sorted by the correlation in the Bitcoin markets.
Additionally, we felt that the qualitative parameter regarding a U.S. headquarter was unnecessarily bringing down the scores of exchanges that are reputable. Other parameters regarding regulatory oversight, trading rules, KYC and other compliance based features were being taken into account independently of an exchange’s legal residence. The country on an exchange’s legal documents may give some general idea of “trading fairness” but it is not an efficient or precise measure. This change in methodology evened the qualitative scores between the U.S. and non-U.S. exchanges we reviewed.
Adding BitMEX to the Benchmark Set
Regardless of recent news (see last week’s SOTN), BitMEX has served as one of the longest running and has historically been considered the most active crypto currency derivatives exchange. Even though this analysis focuses on spot volume we believe that it is important to consider BitMEX’s volume in the benchmark group of exchanges. This addition will help to reduce bias in the analysis. Primarily it will add another data point that is not considered to be Western or U.S. centric. Secondly, it will add trading data related to liquidations and other related futures activities that will reduce any bias against exchanges that see primary activity related to futures products.
Similar to our first version, above is a series of distributions with each row representing the dispersion of the correlation between an exchange’s markets and the trusted markets. The further right the distribution, the more closely correlated it is with the trusted market’s volume. The further to the left, the less correlated. These markets are sorted by the median correlation of all of the exchange’s qualifying markets. The ‘trusted volume’ benchmark set also includes BitMEX’s Perpetual contracts. For our test on correlation we used data from the entire month of September.
Related to the addition of BitMEX’s perpetual volume was the addition of the trading volume for perpetual contracts listed on our sample of exchanges. This alteration in the methodology was put into place to better represent the activity on those exchanges with respect to the broader set. For example, an exchange that supports both spot and futures trading may see a majority of trading volume occur in their futures market and only a fraction within their spot markets. Comparing just the spot volume to the benchmark set that includes perpetual contract volume would not be fairly representing the venue.
Again, in order to make this measurement better represent the exchanges’ overall volumes we create an aggregated volume weighted correlation based on the relative volumes of the exchanges’ markets (in simpler terms: a volume weighted correlation). This was accomplished by taking the volumes for the month of September 2020 and calculating the percentage that each base asset made up of the exchange’s total qualified volume. The result is the table above.
Web Traffic
We have continued to use web traffic data in our analysis. Similar to last time, we used third party data on web traffic from Alexa and SimilarWeb. Page visit metrics from these providers will be our proxy for exchange customers.
A small change that we have made is to rely less on the page view data. This data was reducing the scores of exchanges that offered margin trading, where traders would be able to trade greater amounts than usual due to the use of leverage.
We have modified this test to be more forgiving on exchanges that may fall outside of the “normal” ratios due to margin access. An exchange now only needs to have either less than $50 of volume per pageview or less than $250 volume per visit.
Room for Improvement
A known gap in our methodology that we are working to address is the addition of a liquidity aspect. We are currently collecting a wide range of orderbook data from hundreds of markets across the exchanges that we cover. Our initial assumption is that volume is largely dependent on liquidity and that this dataset will allow us a much clearer understanding of the relationship between the two. We may also consider the correlation between market liquidity across exchanges in addition to volume correlation.
Conclusion
With the updated framework we are able to add five new exchanges: FTX, LBank, Liquid, The Rock Trading, and Upbit. This does not mean that those that did not pass have fake volume, just that they did not meet our parameters to be included.
The entire list as of Q3 2020 is as follows:
- Binance and Binance.us
- Bitbank
- Bitfinex
- Bitflyer
- Bitstamp
- Bittrex
- CEX.io
- Coinbase
- FTX (New)
- Gate.io
- Gemini
- itBit
- Kraken
- Lbank (New)
- Liquid (New)
- Poloniex
- The Rock Trading (New)
- UpBit (New)
