Coin Metrics has conducted substantial research and backtesting to develop the methodology used in the CM Real-Time Reference Rates.

Like all Coin Metrics products, we took a systematic approach to our research for the CM Real-Time Reference Rates. First, we asked a series of core methodology questions. Using the recommendation to those questions, we evaluated various methodology options during times of reasonable stability. Finally, we extensively stress tested and evaluated the most promising methodologies during volatile markets. The best-performing methodology at the end of this process was selected. Below, we have provided a summary of our research.

The CM Real-Time Reference Rates and CM Reference Rates were extensively backtested against certain extraordinary periods of market stress. These include incidents in which prices were volatile, large spreads were observed between major exchanges, suspected price manipulation occurred, failures in internet and exchange-specific infrastructure, or periods during scheduled exchange maintenance.

A sample of six recent events are shown below. In each case, the CM Real-Time Reference Rates and CM Reference Rates were robust to errors, outliers, and manipulation while being close to price where the majority of trades occurred.

Widespread BTC-USD Rally on April 2, 2019

On April 2, 2019, Bitcoin’s price increased from roughly $4,200 to $5,000 in a span of one hour. There was no impactful news released during this time. Large trades were observed on HitBTC’s Bitcoin-Tether market immediately prior to the price movement (not shown below), although HitBTC’s pattern of trades are suspicious and wash trading cannot be ruled out. Large trades were then observed on Coinbase’s Bitcoin-US Dollar market followed by Bitfinex’s Bitcoin-US Dollar market.

Both the CM Real-Time Reference Rates and CM Reference Rates were robust to the large spreads observed between exchanges. The CM Reference Rates were robust to the momentary spike above $5,000.

Bitstamp BTC-USD Incident on May 17, 2019

On May 17, 2019, the crypto market experienced a sharp decline without any news-related catalyst. Bitcoin’s price decreased from $7,700 to $6,600 in the span of one hour. Focused selling was observed on Bitstamp’s BTC-US Dollar market resulting in a sharp divergence in Bitstamp’s price relative to other major exchanges. Bitstamp was likely targeted because, at the time, it was one of two constituent markets used in the calculation of Bitmex’s bitcoin index. The selling on Bitstamp was executed in a way to maximize price impact and a $300 discount was observed on Bitstamp compared to other major exchanges. As the bid side of the order book was wiped completely clean on Bitstamp, a short and volatile period of price discovery followed.

Both the CM Real-Time Reference Rates and CM Reference Rates were robust to the suspected manipulation attempt on Bitstamp’s BTC-USD market.

Coinbase BTC-USD Incident on June 26, 2019

On June 26, 2019 at approximately 20:47 Coinbase reported degraded performance on their website. This coincided with a sharp sell-off in major markets and meaningful spreads were observed in US-based and US-regulated exchanges.

Both the CM Real-Time Reference Rates and CM Reference Rates were robust to the large spreads observed between exchanges. The CM Reference Rates were robust to the two momentary dip below $12,000 at 20:45 and 21:15.

Gemini Maintenance on July 2, 2019

On the night of July 1, 2019, Gemini initiated a scheduled maintenance for a period of four hours. This maintenance was extended for another four hours and concluded on the morning of June 2, 2016. Upon returning from maintenance, a large spread was observed between Gemini and other US-based and US-regulated markets for a short period of time.

Both the CM Real-Time Reference Rates and CM Reference Rates were robust to the anomalous trades that occurred when Gemini finished its scheduled maintenance and began trading.

Cloudflare Incident on July 2, 2019

On July 2, 2019, a major outage impacted all of Cloudflare’s services globally. Many cryptocurrency exchanges were affected and unavailable to many customers for a span of about one hour, roughly from 13:50 to 14:50. During this hour, decreased trading activity was observed on major exchanges.

Despite the global interruption caused by Cloudflare, Coin Metrics’ market input data layer was unaffected and continued to ingest trades and produce reliable Real-Time Reference Rates and Reference Rates during this time.

Bitstamp ETH-USD Incident on July 14, 2019

Similar to the incident in BTC-USD markets on May 17, 2019, focused selling was observed on Bitstamp’s ETH-USD market over the course of several minutes on July 14, 2019. Ethereum’s price decreased from $270 to $245 during this time. A $60 spread was observed on Bitstamp’s market and other major markets. Bitstamp was once again targeted likely because it is one of three constituents in Bitmex’s ethereum index.

Similar to the event that occurred on Bitstamp’s BTC-USD market on May 17, 2019, both the CM Real-Time Reference Rates and CM Reference Rates were robust to the suspected manipulation attempt on Bitstamp’s market.

Please reach out to Coin Metrics ([email protected]) for more information on the CM Real-Time Reference Rates or the CM Reference Rates.

CM Reference Rates Historical Prices and Methodology Updates reference-rate-historical-prices-and-methodology-updates 2019-05-31 21:05:11

Overview

‘This post contains the following items:

1. Launch of Historical Prices for CM Reference Rates
2. Release of CM Reference Rates Methodology v1.1
3. Insights and Background on the changes included in the new methodology
4. Kraken duplicate trades issue
5. Publication of the CM Reference Rate Methodology online
6. Roadmap

‘

Historical Prices for CM Reference Rates

• As promised, Coin Metrics has launched historical reference rates for all 102 assets covered in CM Reference Rates
• Historical reference rates can be found in the /referencerates endpoint of our API

Release of CM Reference Rates Methodology v1.1

Background

• Coin Metrics launched the CM Reference Rates on May 15, 2019 using Methodology v1.0
• The expectation was that Coin Metrics would gain insights into the functioning of the market upon launch and adjust our methodology and quality control processes accordingly
  • Insights were primarily gained from the Quality Control Report, created by Coin Metrics as part of the CM Reference Rates Quality Control Framework
  • The CM Reference Rates Quality Control Framework acts as the governing structure for all machine and human review and expert judgement
  • A sample of the Quality Control Report is included at the end of this post
  • Please reach out to Coin Metrics ([email protected]) for more information on the CM Reference Rates Quality Control Framework
• Consistent with that expectation, Coin Metrics has made the following modifications associated with the release of the CM Reference Rates Methodology v1.1

Insights and Background on the Changes

Data Contingency Rule Changes

The following changes were made to section 4.5 Data Contingency Rules of the Methodology:

• Update to the second Data Contingency Rule: instead of using the price of the last observable transaction of the selected market before the start of the first time interval, use the next available time interval’s volume-weighted median price.
  • This update was implemented to give greater weight to transactions that occur closer to the Fixing Time and address low-volume markets in which the last observable transaction before the start of the first time interval can be stale.
• Update to the third Data Contingency Rule: instead of applying to any 1-minute time interval, only apply to any of the non-first or non-last 1-minute time intervals.
  • The desired outcome was to select the 1-minute time interval closest to the Fixing Time. Thus, the first and last 1-minute time intervals will have dedicated logic (second and fourth Data Contingency Rules) and all other 1-minute time intervals would follow this rule.
  • As part of the CM Reference Rates Quality Control Framework, Coin Metrics generates a Test Reference Rate for each of the three Fixing Times before the official Fixing Time
    • Thus, there is a Test Reference Rate generated for 23:00 UTC, 22:00 UTC and 21:00 UTC
  • Coin Metrics felt that leveraging one of these rates, subject to the oversight and review requirements, would better represent the Reference Rate given the relative proximity to the Fixing Time as compared to the most recent published reference rate.
• Update to the fourth Data Contingency Rule: instead of using the last published rate, use the last hourly reference rate in which there were trades during that hour’s Observation Window.
  • As part of the CM Reference Rates Quality Control Framework, Coin Metrics generates a Test Reference Rate for each of the three Fixing Times before the official Fixing Time
    • Thus, there is a Test Reference Rate generated for 23:00 UTC, 22:00 UTC and 21:00 UTC
  • Coin Metrics felt that leveraging one of these rates, subject to the oversight and review requirements, would better represent the Reference Rate given the relative proximity to the Fixing Time as compared to the most recent published reference rate.

Data Source Changes

1. The Quality Control Report noted instances of low liquidity and identified assets that are in danger of having insufficient trades in the 60-minute calculation window.
2. Low trades were observed for several reasons:
   1. Trading activity during weekends is lower than on weekdays which affects a small number of infrequently-traded assets
   2. Some stablecoins such as Dai and Gemini Dollar are susceptible to low trades because the CM Reference Rate Methodology selects markets quoted in USD, Bitcoin, or Ether. Special consideration is given to stablecoins where selected markets include markets where the stablecoin itself can be the quote currency and the base currency can be USD, Bitcoin, or Ether to match market convention. The methodology does not select stablecoin:stablecoin markets.
3. To address the low liquidity, several changes to the market whitelist were made — expanding the whitelist for some assets and contracting it for others. The net impact of these changes eliminated the issue of low trades in the calculation window for several assets.
   1. In the case of additions, ratings produced by the Market Selection Framework were used to select the next best markets to add.
   2. In the case of removals, expert judgment was applied to remove extremely illiquid markets with prices that differed materially from the other markets.
4. The changes to the whitelist were
   1. WAX (WAX)
      1. added “upbit-wax-eth-spot”, “bitfinex-wax-eth-spot”, “huobi-wax-btc-spot”, “huobi-wax-eth-spot”
   2. Odyssey (OCN)
      1. added “gate.io-ocn-eth-spot”, “hitbtc-ocn-eth-spot”, “gate.io-ocn-btc-spot”, “huobi-ocn-eth-spot”, “huobi-ocn-btc-spot”
   3. FunFair (FUN)
      1. added “hitbtc-fun-btc-spot”, “hitbtc-fun-eth-spot”
   4. Maker (MKR)
      1. added “hitbtc-mkr-eth-spot”, “ethfinex-mkr-btc-spot”, “ethfinex-mkr-eth-spot”, “ethfinex-mkr-usd-spot”, “bibox-mkr-btc-spot”, “bibox-mkr-eth-spot”
   5. QASH (QASH)
      1. added “cex.io-qash-btc-spot”, “gate.io-qash-eth-spot”, “liquid-qash-eth-spot”, “gate.io-qash-btc-spot”, “huobi-qash-btc-spot”, “huobi-qash-eth-spot”
   6. TenX (PAY)
      1. added “huobi-pay-eth-spot”, “huobi-pay-btc-spot”
   7. Gemini Dollar (GUSD)
      1. added “bibox-btc-gusd-spot”, “bibox-eth-gusd-spot”
   8. Bitcoin Diamond (BCD)
      1. added “hitbtc-bcd-btc-spot”
      2. removed “huobi-bcd-btc-spot”
   9. Bytom (BTM)
      1. added “huobi-btm-btc-spot”
      2. removed “hitbtc-btm-btc-spot”, “hitbtc-btm-eth-spot”

Quality Control Report Sample

The below sample is for illustrative purposes only

Kraken Duplicate Trades

Issue Background

• Coin Metrics collects trades from Kraken exchange via both WebSocket and HTTP
• Coin Metrics noticed that Kraken trades received via WebSocket often had a different timestamp from the same trades received from HTTP
• That, in turn, resulted in some trades received from Kraken being duplicated in the Observation Window

Impact and Resolution

• Coin Metrics noticed and resolved this issue within 48 hours
• Coin Metric’s robust Reference Rate methodology meant that the net impact of this issue was minimal
  • Most assets were largely unaffected. For Bitcoin, resolving this issue resulted in a median absolute difference of 0.001 percent and the largest single-day difference is 0.005 percent. For Ethereum, the median absolute difference is 0.001 percent and the largest single-day difference is 0.001 percent.
  • The largest median absolute difference was observed in Project Pai at 1 percent. The largest single-day difference for this asset is 2.5 percent.
• However, out of an abundance of caution and transparency, Coin Metrics has recalculated and republished all Reference Rates for the impacted time period (May 15th to May 26th)

Publication of the CM Reference Rates Methodology online

• One of Coin Metrics’ core values is to be open:
  • We believe open data will empower the public to better understand, value, use, and ultimately steward public crypto networks
• Consistent with that principle, we have added the CM Reference Rate Methodology to our website: https://coinmetrics.io/reference-rates/
• Check it out and let us know what you think

Roadmap

• Coin Metrics expects to make consistent updates to the CM Reference Rates product
• In the short term, those updates may include
  • Inclusion of more assets
  • More publication times (New York, London, Tokyo close)
• We will keep you updated on our progress

If there is something you would like us to add to our roadmap, let us know at [email protected]