Skip to content

Oracle Performace

The data that can be collected on an oracle paints a picture as to the quality of an oracle’s performance. In order to determine the metrics that are favourable to attain, discussed below are the qualities COR believes constitutes the ideal oracle.

The Ideal Oracle

Easy to Measure

  • Lightning fast response time
  • Sets a high gas price
  • 100% completion rate of jobs
  • 100% correctness of response
  • Guarantees service with collateral if required
  • Zero penalty payments incurred
  • 100% uptime and readily available for new work, has been live for a long time

Difficult to Measure

In achieving uptime and response time, should any critical component of the oracle fail, there should be an oracle failover protocol that kicks in to ensure that the oracle remains online and responsive. For network robustness and decentralisation, the computational components and other hardware requirements would ideally be hosted in a local, private environment. This not only promotes decentralisation of oracles, but also means there is no reliance on third party cloud providers that have absolute power over infrastructure.

This oracle should be honest in its answers by subscribing to API providers or owning a licence to access a primary data source - it should be transparent in its methods of sourcing. Ideally the oracle or controlling entity should have primary access to the change of state in reality, whether that be through an IoT device, or through licencing rights to host a database that receives primary data regarding a particular data feed.

The oracle is a trustless third party which prioritises the correct and intended execution of a smart contract that has been agreed upon by the parties involved; it’s impartial to bribes or rent-seeking that may alter the data provided and contract execution. If one entity commands more than one oracle, it provides this information so that the job requester is aware of the chances of a sybil attack undermining their service agreement.

Dissecting the Observable Metrics

Response Time

It can be assumed that the oracle that responds first to a job should be considered the most performant oracle in that instance. With this logic response time is measured and recorded by ordering oracles gas price and determining how many blocks it takes them to respond. As well as this however, a one block response is more desirable than a ten block response.

Consider the example where two oracles may submit their response on-chain at the exact same time. If both oracle’s answers are placed in the same block of transactions it will be the oracle that has used the highest gas price that will have earned the most reputation for that transaction.

Using gas price as a determination factor may allude to the idea that response time reputation can be ‘bought’. The most competitive, highest reputation oracle is most likely to be one that has the required resources to successfully manage their oracle. For the Chainlink network, fast response times are demanded for effective network volume scaling, if by increasing the gas price decreases response time then this is a desirable outcome.

Considering a future where there may be extremely computationally difficult or time consuming job runs, where the block time may tend upwards of 15+ blocks it would be unfair for block response time to negatively affect an oracle’s reputation. To address this, percentile of response could be implemented in the future to determine reputation for certain job types. In the instance where an oracle submits an answer 20 blocks after a request but is the first out of many oracles to respond, placing them in the top percentile of respondents, this oracle would be considered the most performant. For such a process however, minimum required respondents would have to be taken into consideration.

Mempool transactions can be viewed and within them an oracle’s answer can be deduced unless there are privacy measures taken that obfuscate the transaction’s contained data. It would be unfair to justify response time based on when an oracle’s response is seen in a node’s mempool due to differing network latencies resulting in transactions being seen by different nodes at different times. As well as this, there may be some transactions that appear in the mempool but do not ever get placed in a block due to gas or functional issues.

Measurables

  • Gas Price Used: 20 Gwei
  • Duration between request and response: 1 Block, 13 Seconds
  • Average percentile of response: Top 30%

Jobs Completed

When measuring the job completion metric and considering how it affects the reputation of an oracle it is assumed the most important consideration is the job completion ratio. An oracle that has accepted many jobs, but has barely completed any of them, should not be able to maintain a high reputation. The successful completion of a job is the critical factor that job requesters are looking and paying for.

On the COR platform, oracles are considered to have failed a job if they do not respond within 10 blocks. This will likely vary in the future as there is expected to be more functionally dynamic smart contracts that use more than simple price data feeds.

In terms of deciphering the value of each job completion, data such as LINK earned per job would be a worthwhile metric for job requesters to analyse. An operator may have completed 1000 jobs worth 1 LINK each but have only earned as much LINK as an operator that has completed 1 job worth 1000 LINK. However, LINK earned and LINK earned per request is an extremely easy metric to artificially game and should therefore not be taken in as a consideration of the quality of an oracle.

Measurables

  • Jobs Assigned, accepted by oracle: 100
  • Job Completed: 100
  • Completion Ratio: 100%