{"id":551,"date":"2023-08-30T16:43:54","date_gmt":"2023-08-30T14:43:54","guid":{"rendered":"https:\/\/ackeeblockchain.com\/blog\/?p=551"},"modified":"2024-07-04T14:26:27","modified_gmt":"2024-07-04T12:26:27","slug":"introducing-trdelnik-fuzz-testing-framework-for-solana-and-anchor","status":"publish","type":"post","link":"https:\/\/ackee.xyz\/blog\/introducing-trdelnik-fuzz-testing-framework-for-solana-and-anchor\/","title":{"rendered":"Introducing Trdelnik: Fuzz Testing Framework for Solana and Anchor"},"content":{"rendered":"

Developers need to test the reliability and security of their programs before deployment. Traditional unit tests often fail to reveal the edge-case vulnerabilities. Trdelnik addresses this by introducing fuzz testing for Solana programs, which generates a large set of random inputs and call orders to probe for unexpected weaknesses.<\/span><\/p>\n

Why we created Trdelnik\u00a0<\/span><\/h2>\n

Programs for the Solana blockchain are mostly written in Rust programming language. There is a good reason to use Rust because it guarantees memory safety without compromising performance as it is the case with other programming languages that use garbage collection. Rust on the other hand uses a new unique concept of ownership. The downside is that Rust might become very complex and the learning curve is steep.\u00a0<\/span><\/p>\n

But wait, have you also heard that \u201conly the best programmers can write in Rust and the best make fewer mistakes\u201d? Well even if it was partially true, there are better ways to prevent bugs and one of them is testing. Extensive and systematic testing is an integral part of any software development and is a great way to discover bugs early during development.\u00a0<\/span><\/p>\n

How is it possible that most projects have only basic tests or no tests at all? Believe it or not, writing good tests is not as easy as it might seem and in some cases can take even longer than the development of the product alone. In the fast-paced world of crypto, the timelines are often very short and the pressure to launch new projects is extremely high.<\/span><\/p>\n

That is why we have developed <\/span>Trdelnik<\/span><\/a>, our Rust-based testing framework providing several convenient developer tools for testing Solana programs written in<\/span> Anchor<\/span><\/a>. The main goal of Trdelnik is to simplify the setup of the testing environment, provide an automatically generated API to send instructions to custom programs and accelerate the testing process.<\/span><\/p>\n

Fuzz Testing Solana Programs<\/span><\/h2>\n

A new feature recently introduced to Trdelnik is fuzz testing. It is an automated software testing technique that provides generated random, invalid, or unexpected input data to your program. This helps to discover unknown bugs and vulnerabilities and may prevent zero-day exploits of your program.\u00a0<\/span><\/p>\n

There are several fuzzers used for Rust programs, however, googling for Solana fuzz tests does not show any results and that is why we decided to integrate this feature into our framework. Under the hood, Trdelnik uses a well known fuzzer <\/span>honggfuzz<\/span><\/a> developed by Google.\u00a0<\/span><\/p>\n

In the following text we will describe step by step how to use Trdelnik for fuzz testing.<\/span><\/p>\n

TL;DR Solana Fuzzing<\/span><\/h2>\n
# In the root of your Anchor project, execute these commands:\ncargo install trdelnik-cli\ncargo install honggfuzz\ntrdelnik init\n# now to go .\/trdelnik-tests\/src\/bin\/fuzz_target.rs and edit the fuzz test template\n# to run the fuzz test replace <TARGET_NAME> with fuzz_target\ntrdelnik fuzz run <TARGET_NAME>\n# to debug a crash pass in a crash *.fuzz file with following path  trdelnik-tests\/hfuzz_workspace\/<TARGET_NAME>\/<CRASH_FILE>.fuzz\ntrdelnik fuzz run-debug <TARGET_NAME> <CRASH_FILE_PATH><\/code><\/pre>\n
Fuzzing with Trdelnik Step by Step<\/span><\/h3>\n
In this tutorial we will go through the complete setup involving these steps:<\/span><\/p>\n
    \n
  1. Setting up a new Anchor project<\/span><\/li>\n
  2. Creating a program that contains bugs to detect<\/span><\/li>\n
  3. Initializing Trdelnik test framework<\/span><\/li>\n
  4. Writing a simple fuzz test<\/span><\/li>\n
  5. Running the fuzz test<\/span><\/li>\n
  6. Debugging our program using fuzz test crash files<\/span><\/li>\n
  7. Setting up a new Anchor project<\/span><\/li>\n<\/ol>\n
    For the purpose of this tutorial we will create a new Anchor project. If you do not have Anchor framework yet, then <\/span>install<\/span><\/a> the version 0.28.0.\u00a0<\/span><\/p>\n
    Open a terminal and go to your project folder where we will create the new project and verify if the project can be built:<\/span><\/p>\n
    anchor init my-trdelnik-fuzz-test\ncd my-trdelnik-fuzz-test\nanchor build<\/code><\/pre>\n
    Verify that the Anchor project is initialized and built successfully.<\/span><\/p>\n
    Creating a program that contains bugs to detect<\/span><\/h4>\n
    Next we will create a simple program where we will intentionally introduce bugs that we will try to find with our fuzzer. Open the source file of your program <\/span>programs <\/span>\/my-trdelnik-fuzz-test\/src\/lib.rs<\/code><\/span> and replace everything after the <\/span>declare_id!()<\/code> <\/span>macro with the following code:<\/span><\/p>\n
    const MAGIC_NUMBER: u8 = 254;\n\n#[program]\npub mod my_trdelnik_fuzz_test {\n   use super::*;\n\n   pub fn initialize(ctx: Context<Initialize>) -> Result<()> {\n       let counter = &mut ctx.accounts.counter;\n\n       counter.count = 0;\n       counter.authority = ctx.accounts.user.key();\n\n       Ok(())\n   }\n\n   pub fn update(ctx: Context<Update>, input1: u8, input2: u8) -> Result<()> {\n       let counter = &mut ctx.accounts.counter;\n\n       msg!("input1 = {}, input2 = {}", input1, input2);\n\n       \/\/ comment this to fix the black magic panic\n       if input1 == MAGIC_NUMBER {\n           panic!("Black magic not supported!");\n       }\n       counter.count = buggy_math_function(input1, input2).into();\n       Ok(())\n   }\n}\npub fn buggy_math_function(input1: u8, input2: u8) -> u8 {\n   \/\/ comment the if statement to cause div-by-zero or subtract with overflow panic\n   if input2 >= MAGIC_NUMBER {\n       return 0;\n   }\n   let divisor = MAGIC_NUMBER - input2;\n   input1 \/ divisor\n}\n\n#[derive(Accounts)]\npub struct Initialize<'info> {\n   #[account(init, payer = user, space = 8 + 40)]\n   pub counter: Account<'info, Counter>,\n\n   #[account(mut)]\n   pub user: Signer<'info>,\n   pub system_program: Program<'info, System>,\n}\n\n#[derive(Accounts)]\npub struct Update<'info> {\n   #[account(mut, has_one = authority)]\n   pub counter: Account<'info, Counter>,\n   pub authority: Signer<'info>,\n}\n\n#[account]\npub struct Counter {\n   pub authority: Pubkey,\n   pub count: u64,\n}<\/code><\/pre>\n
    It is a simple program with two instructions: <\/span>initialize<\/code><\/span> and <\/span>update<\/code><\/span>. The <\/span>initialize<\/code><\/span> instruction will create the necessary data account and the <\/span>update<\/code><\/span> instruction will update the on-chain data. It also contains an intentional <\/span>panic!<\/code><\/span> macro that will immediately terminate the program simulating a crash if the first input to the program equals the constant <\/span>MAGIC_NUMBER<\/code><\/span><\/span>.\u00a0<\/span><\/p>\n
    Now you can again verify if your program builds successfully using <\/span>anchor build<\/code><\/span>.<\/span><\/p>\n
    Initialize Trdelnik test framework<\/span><\/h4>\n
    In order to use Trdelnik and the fuzzer, we have to install them:<\/span><\/p>\n
    cargo install trdelnik-cli\ncargo install honggfuzz<\/code><\/pre>\n
    If you had Trdelnik already installed then you need to upgrade to version 0.5.0.<\/span><\/p>\n
    After that, we must initialize Trdelnik framework in our project using the command:<\/span><\/p>\n
    trdelnik init\n<\/code><\/pre>\n
    This command will automatically generate the .program_client folder with an API to our program, add necessary dependencies to the configuration files and generate trdelnik tests templates.\u00a0<\/span><\/p>\n
    Write a simple fuzz test<\/span><\/h4>\n
    The fuzz test target template is located in the <\/span>trdelnik-tests\/src\/bin\/fuzz_target.rs<\/code><\/span> file. This file can be modified according to your needs.<\/span><\/p>\n
    Now, you can only replace its content with the following code:<\/span><\/p>\n
    use my_trdelnik_fuzz_test::entry;\nuse program_client::my_trdelnik_fuzz_test_instruction::*;\nconst PROGRAM_NAME: &str  = "my_trdelnik_fuzz_test";\nuse assert_matches::*;\nuse trdelnik_client::fuzzing::*;\n\n#[derive(Arbitrary)]\npub struct FuzzData {\n   param1: u8,\n   param2: u8,\n}\n\nfn main() {\n   loop {\n       fuzz!(|fuzz_data: FuzzData| {\n           Runtime::new().unwrap().block_on(async {\n               let program_test = ProgramTest::new(PROGRAM_NAME, PROGRAM_ID, processor!(entry));\n\n               let mut ctx = program_test.start_with_context().await;\n\n               let counter = Keypair::new();\n\n               let init_ix =\n                   initialize_ix(counter.pubkey(), ctx.payer.pubkey(), SYSTEM_PROGRAM_ID);\n               let mut transaction =\n                   Transaction::new_with_payer(&[init_ix], Some(&ctx.payer.pubkey().clone()));\n\n               transaction.sign(&[&ctx.payer, &counter], ctx.last_blockhash);\n               let res = ctx.banks_client.process_transaction(transaction).await;\n               assert_matches!(res, Ok(()));\n\n               let res = fuzz_update_ix(\n                   &fuzz_data,\n                   &mut ctx.banks_client,\n                   &ctx.payer,\n                   &counter,\n                   ctx.last_blockhash,\n               )\n               .await;\n               assert_matches!(res, Ok(()));\n           });\n       });\n   }\n}\n\nasync fn fuzz_update_ix(\n   fuzz_data: &FuzzData,\n   banks_client: &mut BanksClient,\n   payer: &Keypair,\n   counter: &Keypair,\n   blockhash: Hash,\n) -> core::result::Result<(), BanksClientError> {\n   let update_ix = update_ix(\n       fuzz_data.param1,\n       fuzz_data.param2,\n       counter.pubkey(),\n       payer.pubkey(),\n   );\n\n   let mut transaction = Transaction::new_with_payer(&[update_ix], Some(&payer.pubkey()));\n   transaction.sign(&[payer], blockhash);\n\n   banks_client.process_transaction(transaction).await<\/code><\/pre>\n
    For the purpose of this tutorial and for simplicity we are fuzzing only the instruction parameters. Namely the two parameters <\/span>input1<\/code><\/span> and <\/span>input2<\/code><\/span> of the <\/span>update<\/code><\/span> instruction.\u00a0<\/span><\/p>\n
    If you look at the fuzz target code, you can see that the <\/span>main<\/code><\/span> function contains an infinite loop with the <\/span>fuzz!<\/code><\/span> macro that takes a closure where we pass the <\/span>FuzzData<\/code><\/span> structure.\u00a0<\/span><\/p>\n
    Let\u2019s dissect the code a little. The entry point of the fuzz target is the <\/span>main<\/code><\/span> function, where we call the <\/span>fuzz!<\/code><\/span> macro over and over again. This macro executes the code in the closure, and in each loop, the passed <\/span>fuzz_data<\/code><\/span> are different.\u00a0<\/span><\/p>\n
    We are using the <\/span>arbitrary<\/code><\/span> crate that enables us to easily transform unstructured random data to structured data as we defined it in the <\/span>FuzzData<\/code><\/span> structure. The closure code gets executed, and if the program does not crash, the fuzz test continues with a new loop iteration, and the <\/span>fuzz_data<\/code><\/span> passed to the closure is modified. If the <\/span>fuzz_data<\/code><\/span> variable contains data that produces a crash in our program, the fuzzer automatically stores the data in a separate fuzz crash file <\/span>.\/trdelnik-tests\/hfuzz_workspace\/<TARGET_NAME>\/<CRASH_FILE_NAME>.fuzz<\/code><\/span>. This is useful especially for subsequent debugging.\u00a0<\/span><\/p>\n
    The code executed in the closure in our fuzz target first creates a new <\/span>TestProgram<\/code><\/span> and adds our program to the test environment. Then the test client is started that enables us to send transactions to our program. Having that, we have to first initialize our program. We create the initialize instruction with the help of Trdelnik that automatically generated the <\/span>initialize_ix<\/code><\/span> function for us. Finally we create a new transaction, that we sign and send via the client to the test environment. Great, our program is initialized!<\/span><\/p>\n
    Now we call the <\/span>fuzz_update_ix<\/code><\/span> function in order to supply random data to the update instruction that we want to fuzz and where we want to find bugs. Again, we construct the update instruction using the automatically generated <\/span>update_ix<\/code><\/span> function and we supply the randomly generated parameters from the <\/span>FuzzData<\/code><\/span> structure. Finally, we create the transaction, sign and send it. And that\u2019s it, our fuzz target is ready to go!<\/span><\/p>\n
    Run the fuzz test<\/span><\/h4>\n
    Trdelnik provides a convenient way to run the fuzz tests. Anywhere in your Anchor project, you can execute the command:<\/span><\/p>\n
    trdelnik fuzz run <TARGET_NAME>\n<\/code><\/pre>\n
    So in our case if we replace the <\/span><TARGET_NAME><\/code><\/span> with the actual name it gets the following:<\/span><\/p>\n
    trdelnik fuzz run fuzz_target\n<\/code><\/pre>\n
    Once you execute this command, the whole project has to be built with instrumentation for fuzzing so it takes some time. After the build is finished, the fuzzer starts automatically and looks like this:<\/span><\/p>\n
    \"\"<\/p>\n
    At the top, you can see fuzzing statistics. Especially how many times your program crashed, how many of these crashes were unique, how many iterations were done and so on.\u00a0<\/span><\/p>\n
    To stop fuzzing, you can simply hit CTRL+C. As Trdelnik uses Honggfuzz under the hood, you can also pass parameters directly to the fuzzer using environment variables.\u00a0<\/span><\/p>\n
    For example:<\/span><\/p>\n
    # Time-out: 10 secs\n# Number of concurrent fuzzing threads: 1\n# Number of fuzzing iterations: 10000\n# Display Solana logs in the terminal\n# Exit upon crash\nHFUZZ_RUN_ARGS="-t 10 -n 1 -N 10000 -Q --exit_upon_crash" trdelnik fuzz run fuzz_target<\/code><\/pre>\n
    If you run the command above, the fuzzer will stop after the first encountered crash and passing the <\/span>-Q<\/code><\/span> flag allows us to see the Solana logs. In our Solana example program, we use the <\/span>msg!<\/code><\/span> macro to display the value of the update instruction parameters <\/span>input1<\/code><\/span> and <\/span>input2<\/code><\/span> that you can see from the log that have values 254 and 255 respectively.<\/span><\/p>\n
    \"\"<\/strong><\/p>\n
    Debug our program using fuzz test crash files<\/span><\/h4>\n
    As you can see from the log above, the data that caused the crash of our program was stored in the fuzz crash file in <\/span>.\/trdelnik-tests\/hfuzz_workspace\/fuzz_target<\/code><\/span> folder.\u00a0<\/span><\/p>\n
    Now it is possible to use this crash file and \u201creplay\u201d the crash in the debugger to inspect the bug. The corresponding command is:<\/span><\/p>\n
    trdelnik fuzz run-debug <TARGET_NAME> <CRASH_FILE_PATH>\n<\/code><\/pre>\n
    So in our case if we replace the <\/span><TARGET_NAME><\/code><\/span> and <\/span><CRASH_FILE_PATH><\/code><\/span> with the actual values. The name of the crash file might differ so you will have to modify it accordingly.\u00a0<\/span><\/p>\n
    In our case the resulting command is as follows:<\/span><\/p>\n
    trdelnik fuzz run-debug fuzz_target .\/trdelnik-tests\/hfuzz_workspace\/fuzz_target\/SIGABRT.PC.7ffff7c8e83c.STACK.1b3a7a7882.CODE.-6.ADDR.0.INSTR.mov____\\%eax,\\%ebx.fuzz\n<\/code><\/pre>\n
    Once you execute this command, the whole project has to be built for debugging. After that, the fuzzer runs your program with supplied parameters from the crash file and simulates the crash again for inspection.\u00a0<\/span><\/p>\n
    Now you can see in the debugger, that the thread panicked at ‘Black magic not supported!’, programs\/my-trdelnik-fuzz-test\/src\/lib.rs:27:13<\/span><\/p>\n
    \"\"<\/p>\n
    This is the expected output because we have intentionally introduced a panic in our program if the <\/span>input1<\/code><\/span> variable is equal to our <\/span>MAGIC_NUMBER<\/code><\/span> which is 254.<\/span><\/p>\n
    if input1 == MAGIC_NUMBER {\n    panic!("Black magic not supported!");\n}<\/code><\/pre>\n
    While in the debugger, you can either execute a <\/span>help<\/code><\/span> command for further actions or quit by executing the <\/span>q<\/code><\/span> command.<\/span><\/p>\n
    If you want to try to find another bug in the program, you can uncomment the if statement<\/span><\/p>\n
    if input2 >= MAGIC_NUMBER {\n       return 0;\n   }<\/code><\/pre>\n
    in the <\/span>buggy_math_function<\/code><\/span> function in the Solana program in <\/span>programs\/my-trdelnik-fuzz-test\/src\/lib.rs<\/code><\/span> and run the fuzzer again. After some time, a new unique crash should be found that you can again analyze using the debugger as shown before.<\/span><\/p>\n
    Conclusion<\/span><\/h2>\n
    We have shown how to use the Trdelnik framework to write fuzz tests for Solana programs written in Anchor. You can find the whole example project in Trdelnik\u2019s GitHub repository<\/a>.<\/span><\/p>\n
    The goal of Trdelnik is not to implement a new fuzzer but rather to provide a convenient way to use the existing honggfuzz fuzzer without the hassle of setting up the testing environment.\u00a0<\/span><\/p>\n
    It is as simple as initializing Trdelnik in your project and you are ready to fuzz!\u00a0<\/span><\/p>\n
    The next steps will be the accounts and instructions flow fuzzing.<\/span><\/p>\n
    Stay tuned for more tutorials in the future!<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"
    Developers need to test the reliability and security of their programs before deployment. Traditional unit tests often fail to reveal the edge-case vulnerabilities. Trdelnik addresses this by introducing fuzz testing for Solana programs, which generates a large set of random inputs and call orders to probe for unexpected weaknesses. Why we created Trdelnik\u00a0 Programs for the Solana blockchain are mostly written in…<\/p>\n","protected":false},"author":20,"featured_media":563,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[65,5,113,63],"tags":[6,114],"class_list":["post-551","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-announcements","category-solana","category-trident","category-tutorial","tag-solana","tag-trident"],"aioseo_notices":[],"featured_image_src":"https:\/\/ackee.xyz\/blog\/wp-content\/uploads\/2023\/08\/Trdelnik-600x400.png","featured_image_src_square":"https:\/\/ackee.xyz\/blog\/wp-content\/uploads\/2023\/08\/Trdelnik-600x600.png","author_info":{"display_name":"Adam Hrazdira","author_link":"https:\/\/ackee.xyz\/blog\/author\/adam-hrazdira\/"},"_links":{"self":[{"href":"https:\/\/ackee.xyz\/blog\/wp-json\/wp\/v2\/posts\/551","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ackee.xyz\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ackee.xyz\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ackee.xyz\/blog\/wp-json\/wp\/v2\/users\/20"}],"replies":[{"embeddable":true,"href":"https:\/\/ackee.xyz\/blog\/wp-json\/wp\/v2\/comments?post=551"}],"version-history":[{"count":0,"href":"https:\/\/ackee.xyz\/blog\/wp-json\/wp\/v2\/posts\/551\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ackee.xyz\/blog\/wp-json\/wp\/v2\/media\/563"}],"wp:attachment":[{"href":"https:\/\/ackee.xyz\/blog\/wp-json\/wp\/v2\/media?parent=551"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ackee.xyz\/blog\/wp-json\/wp\/v2\/categories?post=551"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ackee.xyz\/blog\/wp-json\/wp\/v2\/tags?post=551"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}