RAI I-term Tuning Update
This post will go over the results of the first attempt at tuning the integral (I) term as part of RAI’s onchain PI controller implementation. For an introduction to what RAI is and how it works, please read the whitepaper or my RAI launch post.
Background
- RAI is a USD-denominated, floating exchange rate stablecoin that can only be minted using ETH collateral
- RAI uses an onchain PI controller to autonomously sets the redemption rate to oppose market imbalances and drive market price → redemption price convergence
- Tuning PID controllers for dynamic real world systems takes time, we ran the controller in P-only mode for 1 year despite the full code for PI being done
- After we settled on what seems to be a reasonable parameter for the P-term, we added the I-term in Feb 2022 as part of a 6-month planned experiment to evaluate its effects in prod
- In this post we go over what we learned and what comes next!
PI Controller Design
What separates RAI from other $1 pegged stablecoins is that RAI doesn’t have a fixed $1 peg, instead it actively updates its own peg — called the redemption price — to oppose deviations of the market price from the redemption price. RAI uses an onchain PI controller to set the rate of change of its redemption price — called the redemption rate — which is expressed as an annual interest rate. The sensitivity of the redemption rate to market price deviations is determined by the controller PI parameters, which we aim to tune such that RAI is able to autonomously maintain price stability in the face of a wide variety of scenarios and potential shocks.
A PI controller is composed of two terms, the Proportional (P) and the Integral (I). The proportional term is based on the current error only, and so will immediately reset the redemption rate to 0% as the error goes away. The error is defined as the difference between the RAI redemption price (also called the “target price”) and the RAI market price. The integral term accumulates error over time, increasing the redemption rate with the duration of error, and not just its magnitude (unlike the P-term, which has no time component). The final redemption rate set by the controller is calculated by adding the rates determined by the P-term and I-term together, as shown below:
The reason the controller has two terms is because the P-term and I-term operate at different timescales, allowing the controller to detect and response to both sudden and prolonged disturbances. The P-term is more important for responding to sudden shocks, as it can quickly ramp up as RAI detects a price shock and reset just as quickly as the price shock fades. The I-term is more important for correcting prolonged price deviations — called steady state error — as it will ramp up slowly, getting stronger so long as the error persists, and then only resetting slowly over time. The idea is for the I term hold the redemption rate at whatever value brings the error to zero. In effect, the I term is searching for the average long term redemption rate that will keep the error at zero.
Before we move on to parameter tuning, let’s look at how the two rates work together in practice. In the last few months of RAI, pictured above, we have two distinct shocks that RAI responded to. The first shock was during the Luna/UST collapse from 5/12–5/20, about 1 week. During that time the ETH price stayed relatively stable around $2,000 (down from ~$3,000 1 week earlier), but the RAI market price spiked up 2% above the redemption price. The controller immediately responded by setting a P-rate of -15%/y, and RAI market participants began to respond to the negative rates by selling RAI, driving the price down. As prices converged over the week, the P-rate was proportionally reduced back to around -3%/y. In the same time period, the I-rate gradually grew from -1%/y to -3%/y.
The second shock was during the 3AC liquidation & contagion starting 6/15 and was mostly over by 7/15, lasting about a month. The ETH price remained around $1,200 during the shock, down from $1,800 the previous week. The price of RAI spiked up to $3.05, about 2% up from the redemption price, and then over the next two weeks climbed to $3.09 while the redemption price dropped to $2.96 (due to negative rates), so the peak error was around 4.4%, resulting in a peak P-rate of -27%/y and an I-rate of -6%/y, for a total redemption rate of -33%/y. After two weeks the RAI market price started dropping back towards the redemption price, quickly resetting the P-rate back to single digits even while the I-rate grew from -6%/y to -11%/y. The inflection point of the I-rate corresponds to the peak P-rate, because once the P-rate stops growing, the I-rate growth starts decelerating, and then eventually starts shrinking.
For the purposes of tuning the controller parameters, we are concerned with questions like:
- Is the controller working?
- Are the controller parameters optimal?
- How well can the controller handle extreme cases?
Is the controller working?
On the surface, it looks to be that way. The purpose of the controller is to set rates to correct market price imbalances, and in both cases above the controller set rates were sufficient to do so. One might naturally wonder about the counterfactual scenario: “would the market price error have been corrected without the controller set rates?” We are extremely fortunate to be able to compare RAI with LUSD, a $1 pegged stablecoin also backed only by ETH, as a sort of control group.
These charts look different. Compared to the LUSD chart, the RAI chart looks like it has gravity, with the redemption price eventually pulling the market price back to equilibrium.
LUSD, by virtue of its $1 peg, can’t impose negative interest rates, and so it has no additional incentives to offer market participants besides the price arbitrage opportunity to help correct the LUSD price back to $1. In the 6 weeks after the mid-June shock, LUSD has steadily climbed up to $1.04 (4% error).
RAI on the other hand devalued its peg down around 3% over the six week period (since mid-June), and corrected the market price gap down to just 0.5%. So we can conclude that the controller is fact working. 🗿
Of course, this stability comes with a cost, in this case realized as a 3% net value transfer from RAI holders to RAI borrowers. For a longer comparison between LUSD and RAI, see this thread I wrote titled: “How I learned to stop worrying and love negative interest rates”.
Are the controller parameters optimal?
There are 3 parameters to tune in RAI: Kp, Ki, and integral decay.
Kp and Ki are constants that determine the sensitivity of the P-term and I-term to error. The Ki controls the rate at which the I-rate accumulates error. Integral decay determines the rate at which the accumulated I-rate resets back to 0%. Right now, the integral decay half-life is set to 30 days.
Tuning the Proportional Term: Kp
The Kp is currently tuned to offset any price error in about 1–2 months, depending on the market response. For example, given a constant -10% price error, the P-rate will reduce the redemption price by 10% after 50 days, and likewise for the reverse case with a constant +10% price error. The charts below simulate example scenarios with the controller in P-only mode, with Kp set to its mainnet value, and with market price and redemption price both starting at $3.00.
If the market is unwilling to accommodate the controller rates and the market price stays constant after a 10% price shock, the P-rate increases the error, which in turn increases the P-rate, and so on exponentially. In this scenario (shown below), the controller updates the redemption price to offset the market price error in 37 days. The exponential growth of the P-rate is due to the RAI control system being somewhat unique. The RAI controller output results in changes to the setpoint, which in our case is the redemption price, which gives a P-only controller (in this scenario) integral-like qualities.
The 10% number for the initial error was arbitrary, the P-rate will takes roughly the same amount of time to offset a 10% error as a 1% error (as seen below).
Note — the error % is calculated from redemption price — market price, so when the market price is +1% above the redemption price, the error is -1%, and when the market price is 1% below the redemption price, the error is +1%.
In practical terms, the scenarios above represent supply shock and demand shock.
In the supply shock scenario, RAI is scarce and the market price is up +10%. RAI borrowers who want to buy RAI to repay their debts to avoid liquidation while RAI is trading at a premium are faced with what amounts to an exit tax, imposed on them by the other RAI holders who are unwilling to sell, even at a premium. With the Kp tuned as it is, RAI borrowers can have a predictable 1–2 month timeframe over which the redemption price will drop 10% offset the market price premium. In the constant error scenario, the RAI market price will have also dropped 10% after 50 days, allowing RAI borrowers to exit at the initial market price before the supply shock. In the constant price scenario, it would only take 37 days for the redemption price to offset the initial market price shock, which would in turn double the price error, doubling the P-rate, growing exponentially until price convergence.
The demand shock scenario is the same as above but reversed. RAI is overly abundant and the market price is down 10%. RAI holders who want to sell can only do so at a discount, or wait the 1–2 months for the redemption price to offset the market price shock and drive the RAI market price back up.
In practice, this timing of 1–2 months seems like a reasonable pace for the controller to oppose market shocks. A year would be too slow, and a week would be too quick. And as we mentioned above, RAI handled its most recent supply shock gracefully, reducing the redemption price by roughly 3–4% over 6 weeks to offset a 3–4% market price error over that timeframe, thus driving price convergence.
Tuning the Integral Term: Ki and Integral Decay
Just like deciding how fast & far to turn the knob of a new shower to get to the ideal temperature, the RAI controller parameters are optimal when they achieve system stability in a reasonable timeframe and without significant overshoot.
An important factor to consider as you adjust your shower is the time it takes before you start to feel any change in temperature at all — in control systems this is called the latency (also called “dead time”). If you turn the knob too quickly before you feel anything, you are likely to overshoot. Likewise for RAI, if there are structural limitations that prevent the market participants from reacting to the controller set rates, they are important to pay attention to.
For example, it took dxDAO, the largest single RAI holder (holding ~20% of the RAI supply), about two weeks of elevated rates to submit their proposal to sell 70% of their 1M RAI (June 24th), and due to a voting period and execution delays, it was another 11 days before their first RAI sale (July 4th). In their proposal they cited a RAI rate of -17.9%/y, which was enough of an impetus for them to sell. The rate peaked at -33%/y on June 30th, a few days before they sold.
From this, we could estimate that the latency for RAI market participants responding to rate updates is currently about 2–3 weeks. Some RAI holders sold sooner, and some RAI shorts went in earlier as well, but RAI selling by holders and arbitrageurs only happened in earnest after a 2–3 week delay.
What that tells me is that the integral term is probably not optimal (for the current RAI market dynamics) and probably too sensitive. This is because the integral rate was ramping up in the 2–3 week latency window. At the time that dxDAO decided to sell, the redemption rate was -17.9%/y, and only ~25% (-4%/y) of that was I-rate, with the remaining 75% being the P-rate. By the time they sold a few weeks later, the I-rate had already ramped up to -10%/y. This might not seem like a problem given the peak P-rate was -27%/y, but unlike the P-rate, the I-rate is sticky, so the accumulated -10%/y doesn’t go away instantly once price equilibrium is reached, and continues to impact the redemption price for some time. This means that while the I-rate probably only played a small part in the dxDAO decision to sell, it had a large impact in the net change in redemption price.
As a refresher, changes in the redemption price are felt by RAI borrowers and RAI holders as a net value transfer between the two parties. When the redemption rate is positive, it means RAI borrowers are paying RAI holders. When the redemption rate is negative, it means RAI holders are paying RAI borrowers. Thus the RAI system stabilizes by imposing net value transfers between RAI borrowers and holders to oppose market price movements. In this system “overshoot” can be thought of as “excess” net value transfer between RAI borrowers and holders, where the redemption price moved more than it needed to in order to achieve price convergence.
Above is an updated chart of the RAI price/rate history. After mid-July when the redemption and market prices converged, the P-rate went to the low single digits (2–3%/y) and the I-rate started to dominate. For the next two months, the I-rate was the main contributor to the falling redemption price, which fell from ~$2.93 on July 15th to ~$2.89 by August 15th. This continued drop in the redemption price for months, even after price equilibrium was mostly achieved is what makes me believe that the integral response was too strong and resulted in overshoot.
Since we know that the net change in the redemption price is equal to the integral of the redemption rate over time, we can estimate it by looking at the area under the curve of the P-rate and I-rate in the chart above. The blue area (integral of P-rate) and the orange area (integral of I-rate) represent the respective contribution of each to the net change in redemption price. In other words, the summary of my argument for updating the I-rate params is: “the blue area seems about right, the orange area grows too big too fast”.
Now, it could be argued that in a counterfactual scenario without the I-rate as strong as it was, RAI holders and arbitrageurs may have taken longer to sell and short RAI back to equilibrium, but given the structural limitations around the dxDAO sell timing and their decision to do so at a redemption rate of 17.9%/y (with the I-rate being only ~4%/y of that), I don’t think the I-rate played a significant role in correcting the most recent shock.
Integral Windup
The tendency in PI controllers for the integral response to accumulate too much and contribute to overshoot is called integral windup. In RAI, we proactively introduced the integral decay half-life of 30 days to mitigate the effects of integral windup.
As the chart above shows, should the RAI price reach immediate convergence (market price = redemption price), the I-rate will decay back to 0%/y with a 30 day half life. So if on the day of convergence RAI is -7%/y (as above), 30 days later the I-rate will be -3.5%/y, and so on.
Of course, this projection only holds if the market price also drops in unison with the redemption price as the integral rate continues to decrease the redemption price, maintaining price convergence. So far however, we observe (in the part of the chart before the dotted line) that while RAI came close to convergence in late July and early August, the market price has not decreased with the redemption price, leading to an uptick in error and the P-rate, and a slowdown in the rate at which the I-rate is reseting to 0%/y (because the I-rate is now accumulating increased error).
Interplay Between the P and I rates
We should be afraid of the I-rate winding up. As the I-rate unwinds, if the market price doesn’t follow the redemption price, the increased price error will increase the P-rate, and could lead to a feedback loop that exacerbates the error, prevents the integral from ever completely unwinding, and gradually drives the RAI redemption price to 0 or infinity.
Vitalik was ultimately correct to fear the complexity introduced by the integral term. The interaction between the I-rate unwinding and the P-rate means that an I-rate of some value is much more powerful than the equivalent P-rate in terms of how far it can drive the redemption price. For example, given the current prod Ki and integral decay half life of 30 days, an I-rate of -10%/y will drop the redemption price by ~1% over 2 months as it decays from -10%/y to -2.5%/y. If the market price doesn’t follow the redemption price down, this new 1% price error will also induce a P-rate of -7%/y. This 1% error will also get accumulated by the I-rate, which all else equal reaches -4%/y after 1 month of 1% error. Thus the total redemption rate will linger around -10%/y, and the I-rate won’t actually decay to 0%/y.
The takeaway is this: For the I-rate to properly reset, the market price either needs to:
- maintain price convergence until I-rate resets (see “immediate convergence” chart above)
- overshoot the redemption price (see the two charts below)
So far, however, we have yet to observe the overshoot required to reset the I-rate. Instead we observe the I-rate attempting to unwind, but not quite succeeding as the market price isn’t maintaining convergence with the redemption price as it drops, let alone overshooting.
If RAI only used an I-term, this effect would be mitigated as the unwinding I-rate wouldn’t trigger the P-rate feedback loop. If RAI didn’t use an I-term at all, this interaction obviously wouldn’t be a problem. But currently with RAI using both the I-term and P-term, we see the I-rate contributing to induced error and increased P-rate, preventing the I-rate from decaying as intended.
Parameter suggestions for Ki and integral decay
The RAI controller P-term performs well at opposing sudden shocks, as the last few months have demonstrated once again. The addition of the I-term, in my opinion, didn’t seem to significantly enhance the shock response, but it did contribute (and is continuing to contribute) to redemption price overshoot. And we haven’t yet observed the I-rate responding gradually to steady state error, as intended.
As such, we want to mitigate the potential for integral windup in response to short term shocks, but maintain the ability for RAI to have directional interest rates at equilibrium, which the I-rate allows. To do this, I propose weakening the Ki by roughly a factor of ~5x, and updating the integral decay to a 90 day half life.
- Ki: 2.4E-14 → 5.55E-15
- integral decay: 30 day half-life → 90 day half-life
The way to read the above sheet is as follows:
- the top sheet is for our proposed PI parameters
- the middle sheet is for our current production PI params
- the bottom sheet shows the ratio of the rates under the proposed PI params to the rates under the current PI params
- the left column is error %, showing both positive and negative for 1,3,5,10 error %
- the P-rate % column is the redemption rate contribution from the P-term, constant over time
- the 30d/60d/etc PI-rate % column represents the redemption rate after 30d of 1,3,5,10 error %
- the 30d/60d/etc I-rate % column represents the APY of the I-rate by itself (e.g. the redemption rate if equilibrium was reached the next day)
- so in prod today, if the system encountered a +1% error, after 30d the redemption rate would be 11.9%/y, with the I-rate contributing 4.23%/y
- the same +1% error after 30d with the proposed params would only result in a 8.6%/y rate, with the I-rate contributing 1.2%/y
As we can observe, the P-rate is still doing most of the work in terms of the controller response, with the I-rate only kicking in if errors persist for long periods of time. The relevant box for the last few months is the -3% error for 30d, which results in a -28.6%/y redemption rate with our current prod params and only a -22%/y redemption rate with the proposed params. Putting aside the -19.2%/y P-rate, the I-rate with the current prod params would have been -11.7%/y, but with the proposed params would be only -3.5%/y. Given what we know about the danger of integral windup, these I-rates seem less potentially destabilizing than what we have with the current prod params.
With a 90d half life, the I-rate will decay 3x slower, decaying 50% over 3 months instead of each month. This will allow the newly weaker I-rate to still gradually accumulate in response to persisted error while better skipping over short-term shocks. Specifically, if we observe the ratio of the redemption rates set by the proposed params to the rates set by the current params, the proposed I-rates are about ~3x lower in the short term (30–60 days), ~2x lower in the medium term (90–180 days), and only ~30% lower in the long term (360 days). This is consistent with our objective of pushing out the timescale of the integral response from 1–2 months to 3–6 months. We also observe that the overall reduction the combined PI rate with the proposed I-rate params is only around 30%.
Similar to how the P-rate is tuned to offset the redemption price by the error in 1–2 months, we can measure how long it takes for the I-rate to first accumulate and then offset a given error. With a 90 day integral half life for the integral decay, the net redemption price offset after 1 year for an accumulated I-rate is roughly 33% of the I-rate.
For example, as the chart above shows, a -10%/y I-rate will shift an initial redemption price of $3.00 to $2.90 over a 1 year, a 3.3% drop.
The proposed I-rate accumulates the error as a redemption rate roughly linearly for the first 90 days, then slowing down as the 90 day half-life integral decay kicks in. For example, an error of 1% results in an I-rate of 1.2%/y after 30 days, 2.16%/y after 60 days, 2.92%/y after 90 days, 4.34%/y after 180 days, and 5.34%/y after 360 days.
Combining the accumulation and offset relationships above, we can see that the I-rate would only accumulate enough to offset a 1% error after roughly 90 days. This is because it would take 90 days for the I-rate to accumulate up to 2.92%/y, and over the next year the accumulated I-rate would shift the redemption price by 33% of 2.92%, or ~1%. By playing this out we can more clearly understand the effects of the I-rate parameters. The I-rate accumulation rate is primarily driven by the Ki parameter, while the net redemption price offset is primarily driven by the integral decay parameter.
By weakening the Ki by ~5x, we slow down the accumulation of the I-rate, but by increasing the half-life from 30 days to 90 days, we increase the resulting offset for a given I-rate. For comparison, with the current integral decay half-life of 30 days, a -10%/y I-rate would only result in a redemption price offset of 1.1%, 3x lower than with a half-life of 90 days.
From this, we can also infer that in a certain sense, even though we are proposing to weaken the accumulation rate of the I-rate (Ki), because we are also proposing to slow down the integral decay as well, we would still be making the integral response stronger. For any I-rate under the proposed 90 day half-life decay, the net redemption price offset will be 3x what it would be under the current I-rate 30 day half-life decay.
In English, the integral decay half-life of 90 days means that of the accumulated I-rate, 50% of it reflects the previous quarter, 75% reflects the previous half-year, 87.5% represents the previous 9-months, and 93.75% of the I-rate reflects the previous year. A 1-year lookback window with 50% of the weight placed on the most recent quarter seems like a reasonable timeframe to consider historical price error.
Note that while this parameter change should serve to mitigate the potential for integral windup, it doesn’t prevent it. The integral can still wind up to -10%/y as it did recently, it would just take 30d at -10% error or 60d at -5% error to get there, instead of 30d of -3% error as we just saw. This feels more sensible, because the main problem with the current I-rate was that the integral was ramping up within the 2–3 week latency window. That said, we should be aware that if the I-rate does reach -10%/y with the new decay, it will decay 3x more slowly, and will move the redemption price further.
I-term Migration Complications
One annoying gotcha about migrating to a new Ki and integral decay is that unless the I-rate is 0%/y at the time of the migration, the existing I-rate will inevitably be effected. For example, we want to weaken the Ki and increase the integral decay half-life from 30d to 90d, but if we did this tomorrow, the existing I-rate will start to decay slower and thus actually be somewhat stronger. The actual I-rate value wouldn’t increase, but the I-rate would take longer to decay, so the existing I-rate it would have a greater net impact on the redemption rate. If we don’t want to wait for the I-rate to hit 0%/y, but also don’t to slow down the decay of our existing I-rate, we could reset the I-rate to 0%/y ourselves as part of the migration. This has the unfortunate effect of being a “violation” of the Money God and our aspirations of controller ungovernance.
To mitigate the possibility for sacrilege, one option is to first update the Ki to its new weaker value, and only later update the integral decay once the existing I-rate has come down more from its current value of -7.5%/y. Should the I-rate still take too long to unwind, however, I would suggest time-boxing this parameter update to take place within 3–4 months, or sometime before the end of 2022, with the exact date broadcasted well in advance.
I will note here that should we decide in the future to further update the I-term parameters, the same migration complications will apply. RAI users who made financial projections around the I-rate will necessarily need to update their expectations based on the new I-term parameters, so any parameter updates would have to be broadcast in advance.
Our goal, of course, is to identify controller parameters such that we can ungovern RAI and allow the controller to autonomously drive RAI stability without human interference. Ideally, the proposed Ki and integral decay parameters are final and never need to be changed again.
However, if RAI grows such that the major market participants are mega-DAOs and other larger, slower-moving capital allocators, the latency may increase from 2–3 weeks to 2–3 months, and we might find ourselves once again wanting to update the controller parameters, and we will attempt to do so such that those parameters are final.
Summary of I-rate Update Proposal
For our first attempt at tuning the Ki and integral decay params, we were running an experiment with a 6-month timeline, and so intentionally made the I-rate stronger and also decay more quickly so we could observe the lifecycle of the I-rate as it wound up and wound down, and its effect on the market. The experiment concluded and we observed that the I-rate was ramping up too much and too quickly, within the system latency window, and resulting in overshoot. We also observed unexpected interaction with the P-rate, amplifying the effect of the I-rate.
For our second attempt at tuning the Ki and integral decay params, we are optimistic that the proposed parameters are final and close to optimal. We propose to weaken the Ki by 5x and slow down the integral decay rate by 3x, such that the I-rate has a 1 year lookback window. We expect this will diminish the I-rate response to short term shocks, while maintaining most of the I-rate response to long-term disturbances.
How well can the controller handle extreme cases?
When tuning the RAI controller parameters, it’s also important to evaluate their performance at the limits. Note — for the following simulations we use the proposed Ki and integral decay parameters.
10% Demand Shock — Constant Price
Suppose RAI is being dumped en masse (demand shock) and the price remains -10% from some initial redemption price. The RAI redemption rate starts at 100%/y from the P-rate and increases exponentially until price convergence. In the scenario above, after 60 days the redemption price is $3.66, the P-rate is 500%/y, but the I-rate is only about 13%/y, so the integral windup is relatively low. The main risk a rising redemption poses is to RAI borrowers, who see their RAI debt become more valuable in USD terms and may get liquidated as a result. The risk to the protocol during a demand shock as described is minimal.
10% Supply Shock — Constant Price
On the other hand, suppose RAI is being hoarded (supply shock) and the price remains +10% from some initial redemption price. The RAI redemption rate starts at -50%/y from the P-rate and decreases exponentially until convergence. After 60 days the redemption price is $2.35, the P-rate is -93.8%/y, and the I-rate is -14%/y, so again the integral windup is relatively low. In this scenario individual RAI holders and borrowers aren’t at risk of liquidation, but there is a real protocol risk present.
If the RAI market price is ever 10% above the RAI redemption price, it is no longer profitable to liquidate RAI borrowers whose debt exceeds their minimum collateral ratio (135%). This is because the maximum collateral discount offered to liquidators in the RAI system is 10%. If the price of RAI is +10% above the redemption price as ETH crashes, it could lead to bad debt accumulating in the RAI protocol.
To somewhat mitigate this risk, RAI increased the duration of liquidation collateral auctions from 3 days to 28 days, which provides additional time for the RAI market price to come down to a value less than 10% higher than the redemption price, such that liquidations can still be profitably processed.
10% Supply Shock — Constant Error
Given the same 10% supply shock, but this time with constant -10% error, we observe that the P-rate is -50%/y and the I-rate only grows to ~5%/y after 30 days. In other words, if the market price of RAI is ever slightly more than +10% above the redemption price, we are counting primarily the -50%/y redemption rate to sufficiently incentivize RAI selling until the price gap is less than 10% and liquidations are once again profitable.
Maximum Rates
As we explained in Redemption Rate Bounds, the maximum RAI redemption rates are -88%/y and 695%/y. For the P-rate by itself to reach these values, the error % must reach +/-30%. When introducing the I-rate, there is an additional complication with the max rates. The numbers above represent the maximum redemption rate, but there is no maximum P-rate or I-rate. This means that the I-rate can continue to accumulate even if the system hits maximum rates, and if it does, will likely result in excess integral windup and make it difficult to recover stability.
As currently implemented, if the max rates are ever hit and the I-rate is allowed to accumulate while at max rates, it’s probably time to trigger emergency shutdown. Possible solutions to avoid shutdown include:
- raising the redemption rate limits (e.g. by 3–5x)
- adding an additional anti-windup term to the I-rate to make it stop accumulating once the max redemption rate is reached
- weakening the Ki generally (as proposed)
Weakening the Ki and raising the redemption rate limits are easy to do because they don’t require code changes, and still help mitigate the total potential risk of integral windup at max rates. Adding an additional anti-windup term to the I-rate would be a long term solution, but would also require a code change to the controller.
Alternative Proposals
There are several other paths we could take for tuning the PI parameters.
Weakening the P-rate
The P-rate was tuned in isolation, with the I-rate added on after it was already tuned. As we’ve seen, the I-rate not only combines with the P-rate to set a higher combined PI-rate, but the interplay between the P-rate and I-rate also amplifies the effects of the I-rate. As such, it may be worth considering reducing the Kp now that we’ve added the I-rate. Weakening the Kp would increase the time-to-offset from its current 1–2 months to something higher, say 3–6 months for a 3x reduction in Kp, which could also be more friendly for slower moving capital allocators.
However, in the interest of tuning one control parameter at a time, I presently propose only adjusting the I-rate, and leaving the P-rate as is. The P-rate has been proven to work in practice, and so we can bootstrap our understanding of the I-rate dynamics while mostly relying on the P-rate to drive stability. In the future, as we build confidence in the I-rate dynamics, we could feasibly rely more on the I-rate to drive stability, and weaken the P-rate.
Integral Anti-Windup: Clamping
A common anti-windup technique is called “clamping”, and it involves capping the I-rate accumulation above some error % threshold. In RAI for example, this would look like picking some error % threshold, say 3%, above which the I-rate only accumulates the error as if it was 3%, to limit the maximum growth rate of the I-rate. Given that the purpose of I-rate in the PI model is to resolve steady state error (small errors persisted over long time periods), clamping helps prevent integral windup in response to shocks.
We originally decided against implementing clamping, primarily because it would have required us to pick the bounds in advance, and we felt somewhat unprepared doing so without first observing the need for doing so in practice. We also felt that the integral decay would help with anti-windup in place of clamping.
Today I would be more open to clamping the I-rate accumulation in the future, although choosing the clamping bounds would still require additional deliberation. Had we implemented I-rate clamping with a 3% error threshold for example, over the last few months it would have had only a small impact since the error was only around 3–4%.
Integral Anti-Windup: Tick Integral (e.g. Vox, e.g Nikolai’s Vision)
Nikolai’s original design for the Vox component of DAI is a controller where rates tick up and down by a constant amount depending on only the sign of the error, ignoring its magnitude. While this controller would no longer be an “integral” since it isn’t integrating anything, it is conceptually similar to a clamped I-rate (as described above), except that the I-rate could be considered always clamped.
One motivation for this controller, as Nikolai points out, is that PI control may simply be a bad model for stabilizing a currency against a reference asset using by controlling the floating redemption price of the currency. As we discussed earlier, RAI is a unique control system in that the output of the controller — the redemption rate — changes the redemption price setpoint, which gives even a P-only controller integral-like qualities. We also discussed how the net change in the redemption price is integral of the redemption rate. It follows that if the I-rate is the integral of the error, then the I-rate contributions to the redemption rate are a double integral of the error. This is a big part of why we have learned to fear integral windup with the current PI controller.
The advantage of the Vox controller is that it is “scale invariant”, the only factor determining its rate is the duration of the error, not its magnitude. This makes the Vox effectively ignore all price shocks, and only response to long-term price imbalances. Vox is effectively designed to work like a central bank, slowly ticking rates up and down, with the goal of being friendly for the largest, slowest moving capital allocators. This also makes Vox somewhat unfriendly for smaller, faster moving capital allocators.
Vox was designed to work without a P-rate. In my view this is risky, because without a P-rate there is no controller response to short term price shocks. Market participants caught offsides during a price shock might be waiting a long time for Vox to ramp up enough to drive price convergence.
If Vox was run with a P-rate, the Vox + P-rate combo could potentially help address short term shocks, though at the cost of long-term predictability of the redemption price. Vox + P-rate would also likely be subject to the same interactions we observed our P-rate and I-rate, so I would expect the addition of the P-rate to amplify the effect of Vox.
Vox was also designed to work without an integral decay anti-windup term. This is more reasonable for Vox because Vox itself is already essentially an integral anti-windup term, but still seems risky in that Vox isn’t immune to overshoot, which the integral decay mitigates.
The main reason we decided not to go with Vox and use a more traditional I-rate was the potential for asymmetry. It didn’t make sense to us that Vox would set the same rate in response to a +1% error as a +10% error, so long as both lasted 1 month. It also didn’t make sense to us that a month of +10% error could be reset by a month of -1% error.
Upon reflection, we can see that the I-rate + integral decay results in a similar asymmetry. Whereas Vox can reset the rates accumulated from a +10% error with a -1% error so long as they last for the same amount of time, the I-rate + integral decay will also serve to gradually reset the accumulated I-rate, even in the absence of equivalent overshoot. In both cases, the asymmetry is actually helpful, as it mitigates the overshoot required to reset the accumulated rate.
Moving forward, we may consider migrating RAI from the I-rate to a Vox controller, but for now I think it’s OK to continue the I-rate experiment with the weaker Ki and slower decay. With the integral decay, it isn’t as bad that the I-rate accumulates a 10% error at a rate 10x that of a 1% error, because the decay will still asymptotically reduce the accumulated I-rate to 0 over time, regardless of its starting value. If the integral windup continues to be a concern however, the introduction of clamping or a migration to the Vox may be prudent.
Conclusion
As the last 18 months of RAI in production have demonstrated, RAI mostly works. The P-rate is simple way to balance RAI supply and demand, and with the addition of the I-rate, RAI can express directional rates even while at price equilibrium. After the last 6 months trial run of the I-rate, we propose to 1) weaken the Ki by a factor of 5x, such that the I-rate accumulates more slowly and 2) to increase the integral delay half-life from 30 days to 90 days, to give market participants more time to respond to the now slower I-rate.
Should we adopt the new I-rate parameters and run the experiment for another year, my predictions are as follows:
- Most likely — we like the params, finalize it as is or with minor tweaks (if I had to guess right now: we reduce both Kp & Ki by 50%, one at a time, and call it a day)
- Somewhat likely — the I-rate integral windup is still annoying, we weaken Ki even more, or add additional anti-windup like clamping or switch to Vox
- Less likely — we decide that RAI works fine with P-rate by itself
Either way, I don’t think we’ll regret running the experiment in prod. We will share our findings with our compatriots in the hope that together, we can help humanity transcend the dark ages of central banking. We are already making progress on our mission!
Thank you for joining us on this RAI-search adventure!
Join the RAI-volution 🗿
- RAI website
- RAI Dune Dashboard (price charts)
- RAI Forward Guidance (price & rate charts)
- RAI Discord
- RAI Twitter
- RAI Docs
