Originally written in February 2024 by BintuParis and Femi Olah

What if all Solana validators, tired of shrinking rewards, stop producing blocks? The foundation has two options: Wait until the network comes to a halt and face immense pressure. Or redesign the incentive structure.

The thought experiment is rather fanciful. There is a low likelihood that all the 1634 validators would agree to halt block production.

However, it showcases the importance of validators in sustaining the network.

This research study attempts to understand validators and their unique challenges, the current incentive structures, and provide suggestions on how to tackle the validator challenges and improve the current incentive structure.

Before diving deep into this long report, here are the key insights.

Key Takeaways

The primary validator incentive structure requires significant effort for an average validator to break even.
Economic challenges contribute to validators finding innovative ways to earn, including commission cheating.
The ecosystem should “stop supporting 0% commission validators!”.
Solana should document common error messages validators encounter and how to resolve them.

Now to the basics 👇

Or would you rather jump right into the findings and analysis – Can Validators Break Even?

How does Blockchain work?

The blockchain is like a busy city. A city where everyone needs to agree on the truth – what happened, who owns what, and how transactions take place. It is a distributed ledger technology that stores information across multiple computers, ensuring transparency and immutability.

How then does the network ensures the accuracy and security of these transactions?

How do blockchains achieve consensus? Consensus Mechanisms

A consensus mechanism is a set of rules and procedures used in blockchains to achieve agreement on the state of the system. It ensures that all participants (validators) in the network agree on the same truth in a decentralised manner.

Different consensus mechanisms exist, such as Proof-of-work, Proof-of-Stake, and Byzantine Fault Tolerance.

Blockchains like Bitcoin use the Proof-of-work mechanism.

The proof-of-work mechanism is akin to a shouting contest. Everyone competes to solve puzzles required to secure the network. Participants with loud voices, i.e., those with vast computing power, have a higher chance of having their transactions confirmed and recorded in the city’s ledger (the blockchain). While effective in maintaining security and decentralisation, this method requires immense energy and favours participants with the “loudest voice,” hindering scalability and inclusivity.

Proof-of-stake(PoS) offers a quieter, fairer solution. Instead of shouting, individuals(validators) “raise their hand” based on their stake in the city’s economy (cryptocurrency holdings), leading to a more sustainable and efficient way to reach a consensus.

Solana is a proof-of-history and Proof-of-stake Blockchain.

How does PoS work on Solana?

Validators are at the core of Solana’s PoS mechanism. They are responsible for checking incoming new transactions on the blockchain while voting and appending new blocks.

Solana Validators

On Solana, a validator is a term used for any person or computer operating a custom software that allows independent verification of new blocks and changes to the network. This software is called a validator client. Now, while anyone can run a validator software, they are run for different purposes.

It leads us to the two broad categories of validators:

Consensus validators
RPC nodes

Consensus validators

These constitute the main engine of the Solana blockchain. They decide on the transactions that are processed. They also vote on which blocks of transactions are valid and can be added to the blockchain.

RPC Nodes

RPC is short for Remote Procedure Call, a communication protocol that is useful for interaction across remote systems. RPC nodes are set up to allow users who do not have their nodes to interact with blockchains, e.g. to send transactions or query data.

As already hinted, RPC nodes run the same software as consensus validators, but they do not participate in voting for performance reasons. However, these nodes allow applications to be built on the blockchain.

More on Consensus (voting) validators

A Solana validator (called the cluster leader) usually arranges transactions into blocks. Other Solana validators then verify that these blocks are good and put their stamp of approval by stake-weighted voting.

Also, the cluster leader is rotated periodically.

It is critical to ensure that the quality and quantity of validators are high because these determine the integrity of the blockchain data. Imagine if a group of validators maliciously decide not to process transactions from a particular dApp. They would censor that application.

Thankfully, as per the most recent health report by the Solana Foundation, Solana is the most resilient to this attack.

Now that we have discussed the role of validators for the network let us consider how they are being incentivised.

Solana Validator Incentives

There are two core incentive structures for validators:

Protocol-based rewards
Transaction fees

Protocol-based rewards (Inflationary Rewards)

According to Solana, “protocol-based rewards are issuances from a global, protocol-defined, inflation rate.” It implies that new tokens are continuously created and distributed.

Let’s break it down.

$SOL was launched with an initial 8% yearly inflation rate. But, the yearly inflation rate was designed to decrease over time (by about 15%), ultimately reaching a stable long-term inflation rate of 1.5%.

Here’s a visual representation:

Source: Solana

Distribution of Newly Created $SOL

Protocol-based rewards are distributed per epoch to wallets containing actively staked tokens. The active validator set, responsible for validating transactions, receives and distributes these rewards.

Oh yes, it goes to the stake accounts.

So, validators only get incentivised when they charge a commission on the distributed rewards.

As you would think, validators can only generate substantial income when they accumulate a fat amount of staked $SOL or/and charge a large commission.

Little wonder, a good number of validators resort to commission cheating.

Remember that the total amount of inflationary rewards is fixed. So, as more tokens are staked (locked up), the individual reward for each staked token decreases. It means the yield for staked tokens primarily depends on the ratio of staked tokens to the total supply.

“As you would think, validators can only generate substantial income when they accumulate a fat amount of staked $SOL or/and charge a large commission.”

In simpler terms, the more people participate, the smaller the individual reward, but the more secure the network becomes.

How to Calculate Staking Rewards?

Rewards Distribution

The total distributed rewards are allocated proportionately based on stake-weighted credits. In other words, validators with greater stakes and more successful votes command a larger share of the reward pool.

For example: a validator with 1% of the total stake and average performance can expect roughly 1% of the rewards.

Validator Commissions

Individual validators can set commission rates between 0% and 100%, determining a portion of the staking rewards they deduct from their delegators. It serves as a service fee for securing the network.

However, it is crucial to note that commission rates can be modified instantly within an epoch, impacting that epoch’s rewards. Malicious actors might exploit this flexibility through “commission cheating,” abruptly raising rates before an epoch ends to maximise their earnings at the expense of delegators.

Transaction fees

Solana’s documentation describes transaction fees as “the small fees paid to process instructions on the Solana blockchain.”

Users using the blockchain for network interactions such as sending and receiving tokens, swapping, etc., and must pay transaction fees. Each fee consists of a base fee per signature, a computational fee based on resource usage, and an optional prioritisation fee for faster processing.

Fees are a cost barrier for submitting transactions to the network. They discourage participants from sending low-value or malicious transactions. All-in-all, reducing spam and network congestion.

50% of each fee is burned, reducing SOL supply and increasing value. The remaining 50% goes to the lead validation client. Portions of fees sent to validators compensate them for using computational resources (CPU, GPU, etc.),

How to Calculate Transaction Fees Earned

Precisely estimating the transaction cost on Solana requires understanding two components: signatures and the fee per signature.

1. Signatures

Each unique public key involved in a transaction requires a signature, acting as a digital authorisation. It includes the sender, any involved programs, and any other parties needing to sign.

2. Fee per Signature

The current cost per signature fluctuates but can be retrieved using the Solana fees command. When writing this article, the current cost was 5,000 Lamports per signature.

It can also be checked using this Solana CLI:

Source: https://solana.com/docs/core/transactions/fees

5000 Lamports is 0.000005 on https://www.solconverter.com/

Total Fee

To calculate the total transaction fee, simply multiply the number of signatures by the current fee per signature:

Total Fee = Number of Signatures * Fee per Signature

Do note that validators vote on blocks (containing lots of transactions) instead of individual transactions.

For instance:

If a block contains 1500 transactions, and each transaction has an average of 7 signatures, the total fees would be:

1500 * 7 * 5000 Lamports = 52,500,000 Lamports = 0.0525 SOL

Remember that 50% of fees generated goes to the validation leader, while the remaining 50% gets burned.

In the above instance, the validator receives only 0.02625 SOL per block.

Do note that this calculation excludes prioritisation fees.

Prioritisation fee

Solana offers an “optional express lane” for transactions: the prioritisation fee. By paying this additional fee, users can boost the transaction’s priority compared to others, potentially leading to faster execution times.

Solana validators are excited that “priority fees will soon be fully paid to validators and not burned. This, in turn, will give flexibility for validators to share with stakers. The more avenues there are to easily share economics with stakers the better.”

Commission from stake rewards and transaction fees are the major ways that validators are incentivized for securing the network.

“priority fees are soon going to be fully paid to validators and not burned. This, in turn, will give flexibility for validators to share with stakers. ”

Can Solana Validators Break Even with Staking Rewards and Transaction Fees?

Let us paint potential scenarios using the Cogent Crypto Validator Profit Calculator.

Case 1: A validator has a delegated stake of 200,000 SOL and charges a 10% commission. They make a net income of $103,651 per annum. Looks good, yes?

Case 2: The same validator in case 1 reduces their commission rate to 0%. Guess what? They generate a net loss of $15,475.64

And best believe, securing a delegated stake of 200,000 SOL requires a great level of marketing.

Case 3: What if the validator can only get 20,000 SOL?

See what happened, they ran at a loss of $27,841.27, even with a 10% commission, think of what happens when they charge 0% commission in a bid to attract more stakers. They generate even more loss.

Little wonder, Solana Compass hinted that many validators were running at a loss.

So are there reasons for struggling validators not to resolve to commission cheating?

Also, every epoch year, the creation of new tokens will decrease by 15%. Let’s decrease the current inflation rate by 15%, we get 4.6376.

The first validator with 200,000 SOL with a 10% commission rate will witness his annual income decrease to $85,782

The validator who charges a 10% commission with a $20,000 SOL delegated stake incurs even more loss.

Here’s a Barchart summarising the difference in income for the above four cases.

Case 1: $200,000 SOL with 10% commission

Case 2: $200,000 SOL with 0% commission

Case 3: $20,000 SOL with 10% commission

Case 4: $20,000 SOL with 0% commission

Year 1: Inflation rate of 5.456%

Year 2: Inflation rate of 4.6376%

Case study Infographic of Solana Validators' Income

At this point, wouldn’t it be safe to say 0% commission fees, high voting costs, and the deflationary inflation of tokens do not in any way support validators?

Validators do not think so. Many validators worry less about the ever-visible incurred losses in hopes that fee prices will increase.

In fact, the above hypotheticals do not consider that $SOL value may increase and that fee prices will increase. No doubt, in the past few years, transaction fees have indeed gone up which is evident in this chart, created by Mawutor:

Source: Flipside Dashboard by Mawutor

But, have validators’ rewards spiked alongside fee volume?

Sadly, the chart says otherwise. There have been dwindling rewards in the last 1 year, as shown in the chart above. Also, the analyst only calculates the inflation rewards earned per epoch per year and not the transaction fees. Hence, providing a nuanced view of the total income of validators over the years.

As highlighted earlier, validators are excited that priority fees may soon be distributed 100% to validators, thus increasing their income. It is unsurprising as the average priority fees charged have increased tremendously over the last four years.

Yearly Average Priority Fees for Solana Validators

Regardless, validators receiving all priority fees have not yet been implemented. Thus, we can not help but posit that the existing validator incentive structure requires significant effort for an average validator to achieve break-even.

So, attracting more delegators may be the only strategy, albeit necessitating robust investment and the application of commission fees, which adds a marketing burden.

If existing validators are struggling, how then do new validators enter into the market and compete with existing ones?

Solana Foundation Delegation program (SFDP)

The delegation program allows new validators to join the network. As we know, more validators equal more network decentralisation. Before joining the network, participants must have run a testnet validator for the last 5 epochs. Meeting the specified performance requirements is essential for delegation on the mainnet.

There are two queues to join the program. There is a standard priority queue for onboarding, with 25 new validators joining each week. Developers and ecosystem contributors get a special experimental queue thanks to their unique insights into the network’s needs.

How does the SFDP work?

New validators get a helping hand with voting costs for their first year, starting at 100% and gradually tapering down over time (3 months at 75%, three months at 50%, and three months at 25%) to encourage self-sufficiency.

The SFDP also offers stake matching. They help validators double their stakes (up to a 100,000 SOL cap), attracting external trust and delegation. Doubling stakes encourages validators to engage with the broader community and optimise their performance, strengthening the network.

Any remaining Foundation stake goes to eligible new validators to help them get started and meet the requirements for stake matching.

Despite the support given to validators, they still experience significant challenges.

Economic and Technical Challenges

Economic Challenges

Some of the biggest challenges validators face are economic. Setting up a validator can be cost-prohibitive, especially in some parts of the world. One validator highlighted “. . . in Africa where it is 4x more costly than the US”. The costs include payment for equipment, voting costs and other operational needs.

According to Helius, a leading infrastructure provider on the Solana blockchain, many validators pay a minimum of $350/month on basic hardware. This expense does not include the additional cost of sending and receiving data. Also, validators pay about one sol per day in voting costs, which usually accumulates significantly, especially when the price of $Sol is high. These costs exclude people of lower means from participating in providing consensus, ultimately making the blockchain less decentralised.

Another challenge, as one of the validators we interviewed put it, is “compressing commissions due to the nature of increasing competition in stake pools”.

Essentially, this means validators now receive less and less from the rewards distributed to stakers because there are an increasing number of stake pools, and these pools are trying to stay competitive.

A major challenge that validators face is

“compressing commissions due to the nature of increasing competition in stake pools”

Some validators engage in commission cheating whereby they discreetly manipulate the percentage of commission in order to gain more. This could result from them trying to mitigate the effect of the economic challenges.

Solutions

Our interviewee’s proposed solution to the economic challenges of validators is that the ecosystem should “stop supporting 0% commission validators!”. They explained that “stakers must change their mindset seeking 0% commission validators and pay to support the advancement of Solana. Supporting validators with economic incentives ensures Solana can continue to be highly performant and scale globally.”

Some initiatives by the Solana community have proven very helpful. In addition to the Solana Foundation Delegation Program, there are also the stake pools. These pools help validators acquire stakes. The validator we interviewed alluded to this by saying, “The stake pools are great to assist in setting up the validator.”

But could this be all of the validators’ challenges? It goes beyond economic challenges, validators also encounter technical problems.

Technical Challenges of Solana Validators

Validators face technical challenges that can impact performance and stability. The Validator Support and the Validator Beginner channel messages (on Solana Tech Discord Server) were analysed to understand common technical challenges validators experienced.

High Hardware Requirements

The hardware requirements make securing the network difficult for an individual without deep pockets. Here are the requirements on Solana docs:

Analysing discord chats revealed that several new validators reported issues with their validators not catching up or performing poorly due to hardware not meeting the minimum requirements. This includes CPU clock speed, RAM, and storage. Looking at the validator-support channel reveals that validators experienced crashes due to insufficient disk space for the ledger or accounts.

Network Connectivity Issues

Another common challenge that validators face is connectivity, such as timeouts and unreachable ports. Validators need a stable and high-speed internet connection to function correctly. It is not surprising, as Solana suggests, that “Internet service should be at least 1GBbit/s symmetric, commercial. 10GBit/s preferred.”

Software configuration errors

Some validators encountered issues due to incorrect configuration of their validator software, such as missing flags or incorrect settings. It is unsurprising as setting up and configuring the validator software can be challenging for beginners, especially those unfamiliar with Linux or command-line tools. Other validators have experienced startup failures due to incorrect permissions or ownership of directories and files. Also, downloading or managing snapshots (full and incremental) can be complex and prone to errors.

Error messages

Validators often encounter errors they do not understand and need help interpreting and troubleshooting. Interpreting error messages in validator logs requires some technical knowledge.

Other interesting findings

The minimum stake required to earn leader slots is a common point of discussion, as not being included in the leader schedule can limit earning potential.
The Delegation Program and its requirements for participation are not always well understood by new validators.
There is confusion about the role of Edgevana and TDS ’22 in the validator onboarding process.

Suggestions

These technical challenges may sometimes translate into economic challenges for validators. For instance, validators falling behind the network (due to hardware requirements and connectivity) will lead to missed rewards and penalties.

The analysis also revealed that the community is generally helpful and supportive, but moderators often express frustration with repetitive beginner questions. It reveals the need for documentation that caters to new validators. These documentations should not only guide validators through setting up and running a validator but should also touch on common error messages and how to resolve them.

Should the Incentive Structure be redesigned?

The existing system provides crucial rewards, but it also presents hurdles for validators. These challenges threaten the long-term sustainability and decentralisation of the network.

One possible solution is redesigning the system so that more transaction fees are distributed to validators. Thankfully, there’s a report of an ongoing proposal for all the priority fees to go to validators. Our analysis shows that a 0% commission disincentivises validators, so a minimum commission cap should become a standard across all validators and possibly be enforced in code. The commission caps should be set so validators can cover operational costs without discouraging stakers.

Also, providing validators with direct rewards not dependent on commission fees could encourage broader participation.

While all these suggestions are good, introducing significant changes to the incentive structure could introduce unforeseen consequences and potentially destabilise the network. Careful evaluation and community involvement are crucial to ensure a smooth transition.

Methodology

This study employed a mixed-methods approach to investigate the challenges experienced by Solana validators. Data collection and analysis included:

Primary Research

Semi-structured Interview: We conducted a semi-structured interview with an active Solana validator to gain in-depth insights into their first-hand experiences, strategies, and challenges.

Social Media Data: We scraped and analysed discussions from “Solana validators” and “Validator beginner” channels (Solana Tech Discord). Topic extraction techniques were applied to identify recurrent themes and challenges expressed by community members. Also, the findings were anonymised

Case Studies: We developed case studies of Solana validators under different scenarios using insights from Cogent Crypto Calculator. These case studies highlighted the net income validators made with different commission levels, inflation rates, and delegated stakes. Also, we created a bar chart to summarise and visualise key findings from the case study comprehensibly. This presented an overview of Solana validators’ gains/losses incurred under different circumstances.

Secondary Research

Solana documentation: Examined official Solana documentation to understand technical requirements, network protocols, and best practices for validators.

Key opinion leaders: Analysed insights and blog posts from experts and leaders in the Solana validator community to understand validator economics.

Additionally, we comprehensively reviewed existing data sources and information, including validator dashboards (extracted relevant on-chain metrics and performance data) to analyse economic challenges.

Rationale

This multi-faceted methodology ensured robust data collection. By combining quantitative data from validator dashboards with qualitative insights from interviews, social media analysis, and case studies, we developed a nuanced understanding of the challenges faced by Solana validators.

Limitations

It is important to acknowledge that our sample size for interviews was limited. Additionally, the dynamic nature of the Solana ecosystem means that technical and economic challenges may evolve.