(Local) AI, Please Explain This Code

Continuing my exploration of what a local AI model can do, I decided to test them on the task of code analysis. It would be so nice to have an AI model that is tuned and trained on a particular tool or programming system, and that can be distributed for users to run on their own on their local machine, server, or cloud VM. Avoiding the need to run and charge for a custom cloud service and ensuring confidentiality and availability.

Updated 2024-12-12 with Llama-3.3-70B

The Code

All the models were tested with this simple prompt and code example:

What does this code do?

uint8_t mystery (uint32_t val) {
    uint8_t  k = 0;
    if (val > 0x0000FFFFu) { val >>= 16; k  = 16; }
    if (val > 0x000000FFu) { val >>= 8;  k |= 8;  }
    if (val > 0x0000000Fu) { val >>= 4;  k |= 4;  }
    if (val > 0x00000003u) { val >>= 2;  k |= 2;  }
    k |= (val & 2) >> 1;
    return k;
}

The “mystery” function returns the number of the highest bit set in a particular integer. I.e., for an input of 255, the output is 7. For 1024, the output is 10. For 1, the output is 0. It is a standard algorithm you can find in multiple places on the Internet and it can be expected to have snuck into the training set for current LLMs. This precise variant was taken from StackOverflow, where it was decorated with a pointless “inline” declaration. To the credit of all models, adding or removing that declaration makes no difference.

Another way to view the result is as the two’s logarithm of the number, truncated to an integer. For example, for the input 190, the log2 is 7.57, which truncates down to 7. Which is the output of the mystery function.

The result is also one less than the number of significant bits in the number. For example, the value 1 has 1 significant bit, compared to the value 0 returned from this function.

Those alternative interpretations or definitions will be relevant as we review the results of the LLMs on the code.

Output Structure

All the models that were tested for this blog followed the same output template.

• First, a summary of the function’s functionality
• Second, a line-by-line explanation of the code. I omit that in the all the answers below as it is not all that interesting.
• Third, another summary of the function or an example of where it is used.

For example:

In addition, some models added a fourth block with a “better” implementation of the function.

SambaNova: Llama-3.2-1B-Instruct

I first used the SambaNova Playground to run through a large number of different models. It is an incredibly fast system compared to other chatbots I have tried, and it has the very nice property of letting you select from a range of open models. The first model I tried was the Llama-3.2-1B-Instruct model which is a tiny model that still claims to be useful!

The model makes a rather strange statement about the intent of the function.

This is a C function named mystery that takes a 32-bit unsigned integer val as input and returns an 8-bit unsigned integer k. The function appears to be implementing a simple rounding function, likely for use in a graphics or image processing context.

[…code explanation…]

The purpose of this function is likely to round a 32-bit unsigned integer to the nearest 8-bit value, while preserving the original value of the input integer. This is a common operation in graphics and image processing, where the input value is often represented as an 8-bit integer, but the output value may need to be rounded to a 32-bit integer.

This does feel like the model is just making things up that sounds plausible but that is not based in any actual understanding or analysis. Still, that a 1B model can say something at all about C code could be considered impressive considering how little information it contains.

Verdict: wrong, useless.

SambaNova: Llama-3.2-3B-Instruct

The three times larger variant of Llama-3.2 gets closer to the truth, but it is not quite there either. It basically thinks it is trying to count the total number of set bits, which is a different but related operation.

This C code defines a function called mystery that takes a 32-bit unsigned integer val as input and returns an 8-bit unsigned integer.

The function appears to be designed to extract the number of bits that are set in the least significant 8 bits of the input value. Here’s a step-by-step breakdown of what the function does:

[…code explanation…]

The model’s answer wraps up with a surprise. The model has a suggestion:

However, the function can be simplified and made more efficient. The current implementation has a time complexity of O(1), but it uses a lot of conditional statements and bitwise operations. Here’s a simplified version of the function:

uint8_t inline mystery(uint32_t val) {
    uint8_t k = 0;
    k |= (val & 0x00000003) >> 2;
    k |= (val & 0x0000000C) >> 2;
    k |= (val & 0x00000FC) >> 4;
    k |= (val & 0x000F000) >> 4;
    k |= (val & 0x000F0000) >> 8;
    k |= (val & 0x00FF0000) >> 8;
    k |= (val & 0xFF000000) >> 16;
    return k;
}

This version of the function uses bitwise operations to extract the bits from the input value, which makes it more efficient and easier to read.

This is exciting! But… that code does not correspond to what the model says it should do (count set bits). And what made the model even propose it? No idea.

If you read the code closely and analyze the semantics of the lines, it is actually quite hilarious. Basically a joke in code. Only the second and third lines actually compute a result that is non-zero. If you ask the compiler to optimize it, you get a very short function:

The compiler sees that most lines have a constant result of zero and just remove them from the generated code.

Verdict: Not correct, but provided a good laugh.

SambaNova: Llama-3.1-8B-Instruct

The Llama-3.1-8B-Instruct model is a bit older and a bit bigger, and from what I can see a popular model as it is small enough to run on most PCs locally.

This model does figure out what the code does. But it also decides that it is doing the wrong thing!

This C code defines an inline function named `mystery` that takes an unsigned 32-bit integer (`uint32_t`) as input and returns an unsigned 8-bit integer (`uint8_t`).

The function appears to be a bit-counting function, specifically designed to count the number of bits set in the input value. However, it does not directly count the bits set in the input value. Instead, it uses a series of bit-shifting and masking operations to determine the position of the highest set bit in the input value.

[…code explanation…]

This function is likely used in a context where the position of the highest set bit in a value is needed, such as in bit manipulation or compression algorithms.

Reading that second paragraph, the model does identify that the code finds the highest set bit in the input value. But it claims that it should be doing something else, it should be counting all set bits. This is the same behavior as Llama-3.2-3B actually! For some reason, Llama does not like finding the highest bit set.

Note that the last paragraph of output is consistent with the actual function of the code, even if the initial summary is not.

Verdict: The result is misleading and surprising.

Try 2.

After testing the same model locally with worse results (see below), I reran it in the cloud and got a different answer. This time, it misses the purpose of the function. The text is eerily similar to the previous answer, but it goes straight to the idea that this is an incorrect implementation of a bit counting algorithm!

This C code defines an inline function named mystery that takes an unsigned 32-bit integer (uint32_t) as input and returns an unsigned 8-bit integer (uint8_t).

The function appears to be a compact implementation of a bit-counting algorithm, specifically designed to count the number of bits set in the input value. However, it does not directly count the bits set in the input value.

Instead, it counts the number of times the input value has been shifted right by a certain number of bits, which effectively counts the number of bits set in the input value in groups of 2, 4, 8, and 16.

[…code explanation…]

This final paragraph provides an utterly strange summary of what the code is supposed to do that does not help in understanding.

Then, the model imitates the Llama-3.2-3B model and proposes some new code:

This is highly reminiscent of the classic hack presented on the Bit Twiddling Hacks homepage, for counting the total number of set bits in a word. The code is correct (I compiled it and tested it), but it does not do the same thing as the initial code.  

Verdict: Wrong, and proposes an alternative algorithm doing something else!

SambaNova: Llama-3.1-70B-Instruct

The 70B variant is too large to run locally on a standard machine. It does figure out the code’s functionality, explaining it as the logarithm function. But only after some confusing noise:

This C function, mystery, calculates the number of bits required to represent a given 32-bit unsigned integer val. The function returns this value as an 8-bit unsigned integer.

[…code explanation…]

This function is essentially calculating the logarithm base 2 of the input value, rounded up to the nearest integer, which is equivalent to the number of bits required to represent the value in binary.

The initial summary is not correct, but the last paragraph gets very close. The output of the provided code is the logarithm rounded down, which means the mystery function and mathematical log2 agree for even powers of two. For any other number, the mystery function will be one less than the log2 rounded up. The first paragraph description is never correct, as “the number of bits required” will always be one more than the number of the highest set bit.

This model also tries to be helpful/blabbering. It completes its reply with a commented and reformatted version of the code that is functionally equivalent to the original:

Try 2:

Rerunning the query again results in a shorter answer that does not include a proposed updated code. Demonstrating the inherent variability in the output from large language models.

Verdict: Very close to a correct understanding, and no broken code.

SambaNova: Llama-3.3-70B-Instruct

This is an update after the blog post was first released. SambaNova was quick to make the Llama-3.3 version available.

The model provides this answer:

The given code defines a function mystery that takes a 32-bit unsigned integer val as input and returns an 8-bit unsigned integer. The function appears to be calculating the position of the most significant bit (MSB) in the input val.

[…code explanation…]

The function uses a combination of bit shifting and bitwise operations to efficiently calculate the position of the most significant bit. The use of k |= instead of k += or k = ensures that the position is calculated correctly, even when multiple bits are set in the input val.

The last paragraph is unexpected and incorrect. Since the output in each step is non-overlapping, an addition works too. The model should just have stopped talking after the analysis and walk-through.

Verdict: Correct function identification, but an incorrect addendum

SambaNova: Llama-3.1-405B-Instruct

The biggest model provides a concise and correct answer:

This code calculates the position of the most significant bit (MSB) in a 32-bit unsigned integer.

[…code explanation…]

That is how I would describe the result as well. However, it cannot refrain from adding extraneous information at the end of its explanation of the code:

This function is often used in various algorithms, such as calculating the number of leading zeros in a binary number, or finding the most significant bit in a binary representation of a number.

Basically, restating the purpose of the function in a more verbose way. LLMs have a clear tendency to be verbose in a way that a human would likely not be (unless they were trying to hide uncertainty and incompetence behind a wall of words).

Verdict: A correct answer.

Intel AI Playground: Microsoft-Phi-3-Mini-4k-Instruct

The Intel AI Playground makes it dead easy to run AI/large language models (LLM) locally on my Intel ARC GPU. In addition to the image generators that I have already written about, it provides access to one model for text, also from HuggingFace:

• Microsoft/Phi-3-mini-4k-instruct

The model has only 3.8B parameters, so it is one of the smallest models tested. Notably, there is a variant that supports a rather impressive 128k token input window! The blurb on HuggingFace claims it is a robust and well-performing model, at least in the single-digit-billion-parameters space.

The summary of the function is not particularly good:

This code defines an inline function named `mystery` that takes an unsigned 32-bit integer, `val`, as its parameter. The function calculates the highest negative power of two (in the form of an unsigned byte) that is a factor of `val`. It then returns that bitwise representation in the form of an unsigned byte.

[…code explanation…]

For example, if the input `val` = 36 (binary representation: 100100), the output `k` would be 2, because 2 (binary: 10) is the highest negative bit dividing the number without a remainder.

In summary, the code is used to determine the most negative bit without remainder in dividing `val`. It returns this highest negative bit as an unsigned byte.

Bizarre. It claims to return a negative value in an unsigned integer and talks about factorization. I love the concept of “most negative bit”, it is like taking the least significant bit concept and extending it down below zero. Fascinating mix of sensical and non-sensical output.

Verdict: Useless answer.

Ollama: Mistral-7B-Instruct

I used the ollama tool to run some models locally on my desktop. Unfortunately, ollama does not support Intel GPUs out of the box, so the models ran on the CPU instead. This is slow, especially since my Core i7-14700K does not expose AVX-512. However, running on the CPU works and has the advantage of getting access to more RAM.

I used ollama to run the Mistral-7B-Instruct-v0.3 model.

The result:

This function, named `mystery`, takes a 32-bit unsigned integer (`uint32_t`) as its input, and returns an 8-bit unsigned integer (`uint8_t`). The function calculates the number of significant bits in the input value from the most significant bit down to the least significant bit. Here is a step-by-step explanation of the code:

[…code explanation…]

In essence, this function calculates the number of significant bits in the input 32-bit unsigned integer by checking the input value against predefined binary numbers for various bit lengths (16, 8, 4, 2), and shifting the input value accordingly. The final result represents the highest negative bit dividing the input number without a remainder, with each bit position contributing 1, 2, 4, 8, or 16 to the total count based on the corresponding bit length.

The overall description could be considered close to the truth in that it talks about finding the most significant bit. That is what the function does. But the second part of the sentence, “down to the least significant bit” would indicate a function that expects at least two bits set in a word and calculating the distance between them. Confused and confusing.

At the end, the model also talks about the highest negative bit and division and remainders, just like the Phi-3 model. Some kind of pattern here.

Verdict: Confused and not very useful.

Ollama: Llama-3.2-3B-Instruct

Ollama provides access to the same Llama-3.2-3B-Instruct model as hosted by SambaNova. Somewhat surprisingly, running the same model on the same prompt produces a totally different result.

This C code is a function named `mystery` that takes a `uint32_t` value as input and returns an `uint8_t` value.

The function appears to be designed to extract the exponent from a floating-point number represented in the IEEE 754 floating-point format, which is used by most modern computers.

[…code explanation…]

That answer makes as little sense and the other result from the “3B” model. But it is wrong in a new and creative way. Bringing in floating point formats is really weird, especially since the IEEE 754 FP16 format doesn’t have any bit field that corresponds to one of the constants in the code.

Verdict: Useless answer.

Ollama: Llama-3.1-8B-Instruct

The Llama-3.1-8B-Instruct can also be run through ollama. Just like with the previous model, the reply it produces is different from the SambaNova cloud. It is also quite incorrect:

The `mystery` function takes a 32-bit unsigned integer (`uint32_t`) as input and returns an 8-bit unsigned integer (`uint8_t`). The function appears to perform a bitwise decomposition of the input value into a set of flags, which are then encoded into a single byte.

[…code explanation…]

In summary, the `mystery` function appears to be implementing a custom encoding scheme that takes advantage of the properties of binary numbers. Without more context or information about the specific use case, it’s challenging to provide a precise interpretation of this code’s purpose. However, if you have any additional details or requirements, I’d be happy to help further!

In a way this is rather creative. But it is also entirely useless information.

What is really interesting is to see “the same” model perform so differently when run in different ways. Likely there is some kind of additional hidden prompting in the SambaNova cloud that improves the results (or at least makes them different).

Verdict: Useless answer.

Ollama: Qwen2.5-coder-7B

All the previously tested models are allegedly general-purpose. Ollama provides a rich set of available models, including some specific to coding, such as qwen 2.5. Since this is a coding problem, it makes sense to see if a small coding-specific model behaves better than a general-purpose model. I used the 7B parameter variant.

The first answer gets close to what the function does, in a way:

This code defines a function `mystery` that takes an unsigned 32-bit integer (`uint32_t`) as input and returns an 8-bit integer (`uint8_t`). The purpose of this function is to determine the number of leading zeros in the binary representation of the input value. Here’s how it works:

[…code explanation…]

This is confused – but also close to the truth, as the output of the function is (31 – #leading zeros). Prompting the model to re-evaluate does not change the analysis.

Try 2:

Restarting the model and asking the same question:

This code is a function named `mystery` that takes an unsigned 32-bit integer (`uint32_t val`) as input and returns an unsigned 8-bit integer (`uint8_t`). The function appears to determine the number of significant bits in the input value.

That is an almost correct description, being just off by one from the actual result.

Verdict: This is the best result from a small model, if not perfect.

OpenAI: ChatGPT-4o-mini

For completeness, it makes sense to ask ChatGPT as well. Using their free access to the GPT-4o-mini model. The result is this:

The function mystery takes a 32-bit unsigned integer (val) and calculates an 8-bit unsigned integer (k) that represents the number of bits required to represent the number in binary up to a certain point, based on a series of shifts and comparisons. It essentially returns how many bits of val are needed to express it, but with some optimizations for speed by grouping bit checks.

[…code explanation…]

That is the same conclusion, “number of significant bits”, as reached by some other models. The explanation is a bit weird I think (“how many bits of val are needed to express it”).

But it gets really interesting in a section of output that only ChatGPT featured: a walkthrough of the code:

That walkthrough seems reasonable, providing a result consistent with the explanation for the code the model produces. The issue is in the second-to-last step, where the model claims that 1 is greater than 3! LLMs are still not all that brilliant at counting – or, the model somehow made this mistake on purpose to align with the initial assessment.

Verdict: close to correct, but not quite there, and with a very strange maths mistake.

Edge Copilot

Not sure about which OpenAI model is behind the chat in Edge; I could not find any reliable information. Even asking CoPilot itself it does not reveal what is running behind the scenes.

In any case:

This code is a function named mystery that takes a 32-bit unsigned integer (uint32_t) as input and returns an 8-bit unsigned integer (uint8_t). The purpose of this function is to determine the position of the highest set bit (1) in the input value val.

[…code explanation…]

The function ultimately returns the position of the highest set bit in the input value val. For example, if the input value is 0x00010000, the function will return 16, indicating that the highest set bit is at position 16.

That is correct and agrees with how I would describe the function (I prefer “highest bit” over “most significant bit”).  The final paragraph provides an example that works out correctly, always a bonus.

However… asking for something slightly more difficult trips up even this model:

It is also funny how the model does not understand its own front-end formatting, producing a 3 in a circle for what is presumably an output of (3).

Verdict: Correct and with less verbal noise than some other models. But it still does not really understand the code.

Summary

The table below summarizes the results:

Model	Score (0 to 5)	Comment
SambaNova: Llama-3.2-1B-Instruct	0	Wrong. “Rounding”.
SambaNova: Llama-3.2-3B-Instruct	1	Wrong. “Bits set in the low 8 bits”. Proposed a useless new implementation.
SambaNova: Llama-3.1-8B-Instruct	2	Wrong. “Bitcount.” Proposes a new implementation for counting bits.
SambaNova: Llama-3.1-70B-Instruct	4	Almost correct, “two’s logarithm.”
SambaNova: Llama-3.3-70B-Instruct	4	Correct, “position of most significant bit,” but incorrect note on implementation
SambaNova: Llama-3.1-405B-Instruct	5	Correct, “position of most significant bit.”
Local: Microsoft-Phi-3-Mini-4k-Instruct	0	Very wrong. “Negative power of two…”
Local: Mistral-7B-Instruct	2	Quite wrong. “Number of significant bits between…”
Local: Llama-3.2-3B-Instruct	0	Wrong. “Floating point”.
Local: Llama-3.1-8B-Instruct	0	Wrong. “Set of flags”.
Local: Qwen2.5-coder-7B	4	Almost correct. “Significant bits”.
OpenAI: ChatGPT-4o-mini	4	Almost correct. “Significant bits”, and a borked walkthrough.
Edge CoPilot	5	Correct, “highest set bit.”

Conclusions

Overall, this was not a great showing for the AI models. Only two of the models were correct in assessing what the code did, and those were both very large models that require cloud setups to execute.

I was hoping that the local models would have been more successful. I would like to have a world where I can run my own coding assistant based on my own data, code, and custom languages on a reasonable machine. That does not seem to be where we are just yet, and it is not clear how much can be pressed into a model that fits on a common GPU or even a beefy desktop’s main memory.

Note on my thinking: I want to have a model that is tuned to a particular tool or programming system, and that can be distributed for users to run on their own on their local machine, server, or cloud VM. Avoiding the need to run and charge for a custom cloud service.

The Qwen model provides some hope, but in general reasoning is way beyond what can be done using a sub-100-billion-parameter model.

This experiment also serves to demonstrate the lack of a world model or actual knowledge model inside the current crop of LLMs. The models do not really know C semantics do not evaluate it in the same way that a real code analysis tool would – they just produce likely outputs based on training. A human would sit down and work through the code semantics and mathematical operations and figure it out. The LLMs just generate something.

Update: I thought a bit more about this, and possibly a good analogy to this is image recognition. It seems that all the models try to match the code to things they have seen before and get more or less close depending on what is in the “knowledge” they encode. Maybe it is a bit like recognizing different types of apples – the discrimination for similar-looking varieties can be tricky.