Spinsel

Debugging Ghidra and its decompiler

Fri, 02 Apr 2021 00:00:00 +0000

Recently, I came across an oddity in the Ghidra decompiler that I wanted to debug. The decompiler is a native C++ application that runs separate from Ghidra itself (the Java application). Whenever Ghidra needs to decompile a function, display a function graph, etc., Ghidra launches the decompiler and delegates that task to it.

Debugging the decompiler when it’s running standalone is as simple as running gdb path_to_repo/Ghidra/Features/Decompiler/build/os/linux64/decompile. In my case, however, I wanted to debug the decompiler when it’s instantiated by Ghidra. It’s not so straightforward to just attach a debugger to a running decompiler process, since it would require you to launch the debugger at the same time as the decompiler is launched. If you write a script that waits until a decompiler process is created and then attaches to it, you’ll likely run into timing issues.

In this post, I’ll explain how to properly debug Ghidra’s decompiler. Since it’s easy to debug Ghidra itself using the development environment we’ll have to set up for the decompiler, I’ll cover that too. The commands and code in this post should work for a Debian-based distro, but the same can be achieved on any (not-too-exotic) operating system.

NB: there’s already some basic functionality built into Ghidra that allows for inspecting the global state of the decompiler, but it’s not (intended to be) a full-fledged debugger.

Setting things up

The Developer’s Guide that comes with Ghidra explains how to set up a Ghidra development environment. A short summary is given here, but be sure to read the official guide if you want to customize the installation.

First, install some basic prerequisites:

sudo apt update && sudo apt upgrade -y
sudo apt install -y build-essential git wget unzip
sudo apt install -y default-jre default-jdk

Install Gradle, if it’s not already on your system:

wget -P /tmp/ https://services.gradle.org/distributions/gradle-6.8.3-bin.zip
unzip -d /opt/gradle /tmp/gradle-6.8.3-bin.zip

Add Gradle to PATH by appending the following code to ~/.bashrc:

export GRADLE_HOME=/opt/gradle/gradle-6.8.3
export PATH=$GRADLE_HOME/bin:$PATH

Run source ~/.bashrc to apply these changes.

Build Ghidra and set up the development environment (this may take some time, so relax and grab a cup of coffee in the meantime):

git clone https://github.com/NationalSecurityAgency/ghidra.git
gradle --init-script gradle/support/fetchDependencies.gradle init
gradle buildGhidra
gradle prepDev
gradle eclipse

You’ll also probably want to install Eclipse to run and debug Ghidra. Eclipse can be downloaded here.

Debugging Ghidra

If you’ve followed the steps above or the Developer’s Guide, debugging Ghidra will be a breeze. First, we’ll open the Ghidra project in Eclipse:

In Eclipse, click File → Import....
Select General → Existing Projects into Workspace.
Choose the Ghidra repository as root directory.
Mark Search for nested projects and click Select all.
Click Finish.

To debug Ghidra for the first time, go to Debug → Debug configurations... in the menu bar, select Java Application → Ghidra and hit Debug.

The next time you want to debug Ghidra, you can just press the debug button in the toolbar.

Eclipse now allows you to set breakpoints, step through code, inspect and modify variables etc. for the Ghidra application.

Debugging the decompiler

The approach to debug the decompiler is as follows:

We’ll add a boolean debugger_present to the decompiler, which is initially false.
Before executing the rest of the program, the decompiler will loop until debugger_present is true.
The debugger attaches to the running decompiler process, sets debugger_present to true and optionally configures some breakpoints. After that, the normal execution flow continues.

We add the following code to Ghidra/Features/Decompiler/src/decompile/cpp/ghidra_process.cc¹:

#include <thread>
#include <chrono>

volatile bool debugger_present = false;
void wait_for_debugger() {
    while (!debugger_present)
        std::this_thread::sleep_for(std::chrono::milliseconds(500));
}

In this same file, insert a call to wait_for_debugger in the main function, which is the entry point of the decompiler:

int main(int argc,char **argv)

{
  wait_for_debugger();
  ...
}

We also need to build the decompiler with debug flags and no optimization flags. This can be done by commenting/uncommenting the appropriate lines for your compiler in Ghidra/Features/Decompiler/build.gradle#L537-L568. In my case (gcc) I had to change lines 537-538 from

b.cppCompiler.args "-O2"    // for DEBUG, comment this line out
//b.cppCompiler.args "-g"   // for DEBUG, uncomment this line

//b.cppCompiler.args "-O2"  // for DEBUG, comment this line out
b.cppCompiler.args "-g"     // for DEBUG, uncomment this line

Now we need to recompile the project. Since we’ve only modified the decompiler code, only this part of the project will be rebuilt.

gradle buildGhidra

To test this approach, open an arbitrary executable in Ghidra and try to decompile any function. The decompiler should now hang forever because it’s waiting for debugger_present to be true!

We use pgrep to find the decompiler process and attach to it using gdb:

pgrep -fn decomp
4367

sudo gdb -q path_to_repo/Ghidra/Features/Decompiler/build/os/linux64/decompile 4367
Reading symbols from path_to_repo/Ghidra/Features/Decompiler/build/os/linux64/decompile...
Attaching to program: path_to_repo/Ghidra/Features/Decompiler/build/os/linux64/decompile, process 4367
...
(gdb) set debugger_present = 1

Even after setting debugger_present to true, gdb keeps halting the program until we enter c or continue. Before doing that, we’ll set a breakpoint at GhidraDecompCapability::initialize:

(gdb) b GhidraDecompCapability::initialize
Breakpoint 1 at 0x55b6d48e5f56: file path_to_repo/Ghidra/Features/Decompiler/src/decompile/cpp/ghidra_process.cc, line 499.

If we type c to continue the program, it’ll hit the breakpoint we’ve just set. We can now inspect and modify memory and then resume the program:

(gdb) c
Continuing.

Breakpoint 1, GhidraDecompCapability::initialize (this=0x558714894618 <std::vector<PrintLanguageCapability*, std::allocator<PrintLanguageCapability*> >::push_back(PrintLanguageCapability*&&)+50>)
    at path_to_repo/Ghidra/Features/Decompiler/src/decompile/cpp/ghidra_process.cc:499
(gdb) p commandmap
$1 = std::map with 0 elements
(gdb) c
Continuing.

The decompile window in Ghidra should now show the decompilation results.

We can automate this process using two scripts that we’ll call gdb_init and debug.sh.

gdb_init is a gdb command file we’ll pass to gdb using the -x flag. It contains the commands to be run after attaching to the decompiler.

gdb_init:

set debugger_present = 1

b GhidraDecompCapability::initialize

# uncomment the next line if you want to automatically continue the program
#c

debug.sh looks for the decompiler process and (if found) attaches to it. It passes the gdb_init command file to gdb.

debug.sh:

#!/bin/bash

decompiler_path=path_to_repo/Ghidra/Features/Decompiler/build/os/linux64/decompile

pid=$(pgrep -fn "$decompiler_path")
if [ -z $pid ]
then
    echo "No decompiler process found"
    exit 1
fi

echo "Attaching to decompiler process with pid $pid"
gdb -q -x gdb_init "$decompiler_path" $pid

This script can be invoked using sudo ./debug.sh.

Pro-tip: if you want to debug the decompiler, it’s a good idea to read the decompiler documentation. This documentation isn’t built by default, but you can build it by installing doxygen and running make doc in Ghidra/Features/Decompiler/src/decompile/cpp. The documentation will be written to Ghidra/Features/Decompiler/src/decompile/doc.

Conclusion

In this post we’ve looked at one possible way of debugging a decompiler process created by Ghidra. I’m sure there are many other ways to do this, but – after modifying the source code – this method is easy and works reliably. A possible improvement to this method would be to only wait for debugger_present to be set to true if an environment variable such as GHIDRA_DECOMPILER_DEBUG is set to true. That way you can conveniently disable the debugging functionality altogether using one simple command. If you’ve got an alternative method, let me know!

Happy debugging!

Footnotes

In the original version of this post, I forgot to mark debugger_present as volatile. While it seems to work fine with gcc, it’s better to mark it as volatile to ensure the compiler doesn’t optimize debugger_present away. Thanks to pelrun for pointing this out! ↩

Implementing a brainfuck CPU in Ghidra - part 4: Renaming the analyzer and adding a manual

Mon, 27 Jul 2020 00:00:00 +0000

This blog post is part of the series Implementing a brainfuck CPU in Ghidra:

Setup and disassembly
Decompilation of pointer and arithmetic instructions
Decompilation of I/O and control flow instructions
Renaming the analyzer and adding a manual (current post)
Recognizing common patterns (future post)

More posts may be added to this series in the future.

Now that our processor module can finally disassemble and decompile brainfuck binaries, we can make some slight improvements to the module. We’ll do two things in this blogpost: first we’ll rename the analyzer and then we add a processor manual

Renaming the analyzer

Right now, the analyzer that resolves branch destinations is called BrainfuckAnalyzer.java. This name doesn’t really make it clear what it does. Maybe BranchDestinationResolver.java would be a better name, as it’s more descriptive.

This sounds easy, right? Just right click BrainfuckAnalyzer.java and rename it to BrainfuckAnalyzer.java. Don’t forget to also rename the class and constructor to BranchDestinationResolver. But, it’s not that easy or I wouldn’t have devoted half a blog post to it :)

If we now restart Ghidra, the analyzer has disappeared from analysis window. There’s no BranchDestinationResolver or BrainfuckAnalyzer in the list. What happened?

As it turns out, Ghidra only shows analyzers that are valid extension points. An extension point is a class that extends the functionality of Ghidra. There are two requirements for a class to be an extension point:

It must (directly or indirectly) derive from the ExtensionPoint class.
The containing file must have a valid extenstion point suffix.

The first requirement is met. The BranchDestinationResolver class indirectly extends ExtensionPoint¹. The problem is with the filename. By default only certain suffixes (there are about fifty) are recognized as extension point suffixes, the Analyzer suffix among them. Resolver is not a valid extension point suffix, so Ghidra doesn’t recognize BranchDestinationResolver as an analyzer, while BrainfuckAnalyzer is fine because it ends with Analyzer.

We could append a valid suffix to the analyzer name to ensure Ghidra recognizes it, but it would result in an akward name (e.g. BranchDestinationResolverAnalyzer or BranchDestinationAddressCorrelator). We could also drop the Resolver suffix and use the Analyzer suffix instead (BranchDestinationAnalyzer sounds better). Instead of doing this, we can also register a new extension point suffix.

To register a new extension point suffix, we create a file in the data directory of the project called ExtensionPoint.manifest. It contains only one line:

Resolver

That’s all. Ghidra recognizes the ExtensionPoint.manifest file and registers all suffixes in the manifest (separated by newlines). This is also how the default extension points suffixes are registered. For example, the Analyzer suffix is registered in Ghidra/Features/Base/data/ExtensionPoint.manifest.

If we now start Ghidra and open the analysis window, the analyzer shows up again:

This was a very brief look at extension points. The whole extension point mechanism deserves a post of its own. If you’re interested in how it works, I suggest looking at the ClassSearcher, and ClassFinder classes.

Adding a manual

Now something different: adding a manual to the processor module. As said in the first post, a language in the .ldefs file can have a manualindexfile attribute pointing to a processor manual index file. A processor manual consists of one or more PDF files that contain documentation for the instruction set. A processor manual index maps instruction mnemonics to their corresponding page in the manual.

The manual index is documented at GhidraDocs/languages/manual_index.txt, but the documentation is not fully complete. The code that loads the manual index file can be found here.

Suppose we’ve got a dummy manual, bfman.pdf. It’s manual index file would look like this:

data/manuals/brainfuck.idx:

@ bfman.pdf [Brainfuck Manual]

>, 1
# the '<' instruction is intentionally omitted

+, 2
-, 2

,, 3
., 3

[, 4
], 4

The first line starting with @ is called a file switch. It sets the current manual file. The [Brainfuck Manual] part provides a description for this manual. It’s optional and is shown when Ghidra can’t find the manual file, so the user can locate the manual elsewhere. There can be multiple file switches in a manual index. This is useful when the processor manual spans over several volumes.

As stated in the index file, the < instruction is omitted. This is because the < character has a special (undocumented) meaning. It can be used to import another index file. There’s no way to escape this character, which makes it impossible to create an entry for the < instruction. The only way to get around this is to rename the < instruction. For now, we’ll just omit the < instruction.

Also good to know (and again undocumented): a # indicates a comment.

The only thing left to do now is to add the manual index file to the language definition in brainfuck.ldefs. This is done by adding the following attribute to the language tag: manualindexfile="../manuals/brainfuck.idx".

That’s all there is to manual index files. If you now right-click an instruction and click Processor Manual..., Ghidra will show the documentation for that instruction!

In this post, we’ve improved the usability of the processor module. Meanwhile, the module still produces poor decompilation output. Next time, we’ll look at improving the decompilation of our module. Hopefully, we’ll manage to produce better decompilations.

Footnotes

BranchDestinationResolver ⊂ AbstractAnalyzer ⊂ Analyzer ⊂ ExtensionPoint ↩

Implementing a brainfuck CPU in Ghidra - part 3: Decompilation of I/O and control flow instructions

Wed, 17 Jun 2020 00:00:00 +0000

This blog post is part of the series Implementing a brainfuck CPU in Ghidra:

Setup and disassembly
Decompilation of pointer and arithmetic instructions
Decompilation of I/O and control flow instructions (current post)
Renaming the analyzer and adding a manual
Recognizing common patterns (future post)

More posts may be added to this series in the future.

This is already the third post in this series. This time we’ll complete the decompilation part of the processor module by adding semantics for the I/O (. and ,) and control flow ([ and ]) instructions.

I/O instructions

The . (output) and , (input) instructions can’t be described by regular p-code operations, as regular p-code operations operate only on varnodes. They don’t allow for modelling external input. Luckily, we can define custom p-code operations like this:

define pcodeop in;
define pcodeop out;

This code declares two custom p-code operations: in and out. User-defined p-code operations can have any number of input varnodes and an additional output varnode. Ghidra will not (and cannot) make any assumptions about these operations; they are treated as black boxes. Because Ghidra can’t analyze these black boxes, you should always try to keep the number of user-defined p-code operations to a minimum.

in and out can be used in SLEIGH semantic statements using using functional syntax (calling them like a function, e.g. f(x)). The following statements are all valid (assuming x, a, b and c are defined):

x = in();
x = in(a);
in(a, b, c);
x = out()
x = out(a, b)

It is even valid to call an operation with one operand first and later with two:

out(a);
...
out(a, b);

Most of these don’t make sense. Only x = in() and out(x) make sense. in should not take any input and outputs one value. out is the opposite: it takes exactly one input and does not return anything. There is no way to enforce this using the specification. It is the task of the language creator to do this.

Something like this is not allowed:

x, y = in()

This is because p-code operations can have at most one output varnode.

Let’s see how we can use our newly defined p-code operations to define the semantics of the . instruction and later of the , instruction. The . instruction does the following: it takes the value of the current cell and outputs this value.

:. is op=0x4 {
    local c:2 = *[ram]ptr;
    out(c);
}

You can also write out(*[ram]:2 ptr);, if you’re feeling brave. In my opinion, this reduces readability.

The , instruction takes input and stores it in the current cell:

:, is op=0x5
{
    local c:2 = in();
    *[ram]ptr = c;
}

(Or write *[ram]:2 ptr = in();.)

We can test these instructions using a simple program called add3.bf. It takes user input, adds three to it and outputs the result:

,      store user input
+++    add three
.      output

Compile it (./bfc.py add3.bf add3.bin) and open it in Ghidra. The disassembly will look like this:

Here it can be seen that user-defined p-code operations in SLEIGH are converted to CALLOTHER operations in the p-code. These CALLOTHER operations take the user-defined operation name as their first argument, followed by the arguments of that operation. More concretely, an operation like out(c) gets converted to:

CALLOTHER "out", $U90

Where $U90 is the varnode in the unique address space corresponding to the symbol c.

In the decompilation, these user-defined p-code operations are represented by functions:

An interesting thing about this decompilation is that the order of execution is not preserved. In the decompilation, out is called before the value is written to memory. An order-preserving decompilation may look like this:

sRam0000 = in();
sRam0000 += 3;
out(sRam0000);

The decompiler is free to move statements around as long as it can prove the outcome is the same. In this case, the output is preserved: in both cases the result of in() + 3 is written out using out and written to ram at address 0x0000.

Control flow instructions

Implementing the I/O instructions was the easy part. The implementation of the control flow instructions is a bit more complex, because of the unusual branch implementation of brainfuck. Common architectures have branch instructions that encode an absolute or relative address for the CPU to branch to. Brainfuck doesn’t encode the branch destination in the instruction itself. Instead if it encounters [, it may jump to the corresponding ], depending on the current cell value. If it encounters a ], it jumps to the corresponding [. Compare the following brainfuck code to its (unoptimized) ARM (ARMv6-M Thumb) equivalent (r0 is ptr and r1 is a temporary register to hold the current cell value):

control_flow_test.bf:

+-[-]+

control_flow_test_arm.s:

; +
ldr r1, [r0]
add r1, #1
str r1, [r0]

; -
ldr r1, [r0]
sub r1, #1
str r1, [r0]

loop_start:
; [
ldr r1, [r0]
cmp r1, #0
beq loop_end

; -
ldr r1, [r0]
sub r1, #1
str r1, [r0]

; ]
b loop_start
loop_end:

; +
ldr r1, [r0]
add r1, #1
str r1, [r0]

The ARM assembly may look complicated, but it produces a nice decompilation¹:

void FUN_00010000(void){
    int *ptr;

    (*ptr)++;
    (*ptr)--;
    while (*ptr != 0)
        (*ptr)--;

    *ptr = 1;

    halt_baddata();
}

The b instruction looks like this in p-code:

BRANCH *[ram]0x10006:4

The beq instruction looks as follows:

$Uefa0:1 = INT_NOTEQUAL ZR, 0:1
CBRANCH *[ram]0x10012:4, $Uefa0

The p-code for these ARM instructions is very short because they have an almost one-to-one correspondence with their p-code operations. The control flow graph of the program looks like this:

Ghidra can easily calculate the branch destinations and then recover the loop-structure of this program. It is of great importance that Ghidra can compute the branch destination. If not, it can’t recover the structure of the program.

The brainfuck variant of the program looks a lot simpler, but its p-code is more complicated. Let’s model the semantics of the [ instruction. We’ll restrict ourselves to the context of this program, which means that (a) there are just two control flow instructions in the entire program (one [ and one ]) and (b) the cell value at the [ instruction is zero, so the cpu should always branch to the corresponding ] instruction.

The following code searches for the next ] (opcode 0x7) and branches to it:

:[ is op=0x6 {
    local addr:2 = inst_start + 1;

    <check>
    local instr:2 = *addr;
    if(instr == 0x7) goto <end>;
    addr = addr + 1;
    goto <check>;

    <end>
    goto [addr];
}

This produces the following decompilation for control_flow_test.bf:

void start(void){
    short *UNRECOVERED_JUMPTABLE;

    /* WARNING: Could not recover jumptable at 0x0002. Too many branches */
    /* WARNING: Treating indirect jump as call */
    UNRECOVERED_JUMPTABLE = (short *)0x3;
    while (*UNRECOVERED_JUMPTABLE != 7)
        UNRECOVERED_JUMPTABLE = (short *)((short)UNRECOVERED_JUMPTABLE + 1);

    (*(code *)UNRECOVERED_JUMPTABLE)();
    return;
}

The control flow graph of this program consists of just one block, ending with a BRANCHIND to an unknown destination:

Clearly, Ghidra is unable to statically calculate the branch destination of the [ instruction, even though it is statically computable². Simper calculations in the semantics (with control flow constructs , but without loops) are computed correctly by Ghidra. My guess is Ghidra doesn’t bother computing loops in p-code, although I have not really looked into this (yet). (If there is a way to compute this statically in SLEIGH semantics, please contact me!)

It’s no use continuing to implement the semantics of [ and ] this way. We need another way to statically compute branch destinations that Ghidra does understand. I came up with the following:

Compute the branch destinations using an analyzer, written in Java
Write these destinations to memory that’s marked as read-only
The instruction semantics for the control flow instructions will consist of a check whether the cpu should branch, a load from memory and a branch to the loaded destination.

We’ll call this memory that holds branch destinations bdest. This memory can be created by adding the following to the <default_memory_blocks> tag in the .pspec:

<memory_block name="bdest" start_address="bdest:0x0000" length="0x1000" mode="r" initialized="true"/>

Expose bdest to SLEIGH by extending the address space definitions in the .slaspec:

define space bdest type=ram_space size=2 wordsize=2;

The last thing to do is to mark bdest as read-only in the .cspec. Add this to the <compiler_spec> tag:

<readonly>
    <range space="bdest" first="0x0000" last="0x0fff"/>
</readonly>

Without this, the decompiler doesn’t assume that bdest is read-only, even though it is marked as such in the .pspec (mode="r").

Eclipse automatically created an analyzer for us when we selected analyzer in the project wizard during the project setup. It is called BrainfuckAnalyzer.java. The functions getDefaultEnablement and registerOptions can be removed, because this analyzer won’t use them. Change the constructor to this:

private final static String NAME = "Brainfuck Branch Destination Resolver";
private final static String DESCRIPTION = "Computes branch destinations for control flow instructions ([ and ]) and writes them to bdest memory";
...
public BrainfuckAnalyzer() {
    super(NAME, DESCRIPTION, AnalyzerType.BYTE_ANALYZER);
    setDefaultEnablement(true);
    setSupportsOneTimeAnalysis(true);
    setPriority(AnalysisPriority.DISASSEMBLY.before());
}

The analyzer is marked as byte analyzer instead of instruction analyzer, because it operates on raw bytes before they are converted to instructions so that Ghidra can correctly analyze branches. Read more about analyzer types in the source code.

The next statements notify Ghidra that the analyzer should be enabled by default and that it supports one time (one-shot) analysis. One time analysis does not mean it can only be called once, but that it can be called at any time and not just during auto analysis. It is obvious that the analyzer should be called before disassembly. Hence the priority DISASSEMBLY.before().

There are two functions left: canAnalyze and added. We’ll get to canAnalyze first. This function tells Ghidra whether the analyzer can analyze the program. This analyzer should only analyze Brainfuck programs:

private final static String PROCESSOR_NAME = "brainfuck";
...
@Override
public boolean canAnalyze(Program program) {
    String procName = program.getLanguage().getProcessor().toString();
    return procName.equals(PROCESSOR_NAME);
}

added is where the real magic happens. This function is called when bytes are added to the program or during auto or one-shot analysis. It’s responsible for the analysis. We’ll put the main logic in a function called resolveBranchDests and call that function from added, which catches and reports exceptions.

@Override
public boolean added(Program program, AddressSetView set, TaskMonitor monitor, MessageLog log)
        throws CancelledException {
    try {
        return resolveBranchDests(program, monitor);
    } catch(CancelledException e) {
        throw e;
    } catch(Exception e) {
        log.appendException(e);
        return false;
    }
}

This constructions throws CancelledException (raised by monitor.checkCanceled() when the user cancels the operation) and logs any other exceptions.

Let’s write resolveBranchDests. This function receives a Program and a TaskMonitor and it can throw a CancelledException and a MemoryAccessException. The latter will be raised if the memory we’re trying to write (bdest) is uninitialized. It will not be thrown because we set initialized="true" in the .pspec. It is a checked exception, however, so we must catch it.

private boolean resolveLoops(Program program, AddressSetView set, TaskMonitor monitor, MessageLog log)
        throws CancelledException, MemoryAccessException {
    Memory mem = program.getMemory();
    MemoryBlock rom = mem.getBlock("rom");
    MemoryBlock branchDests = mem.getBlock("bdest");

These lines get references to memory blocks we’ll be using.

Make sure the bdest memory is big enough:

if(bdest.getSize() < rom.getSize())
    throw new AddressOutOfBoundsException("The bdest memory can't be smaller than the rom memory.");

Initialize the task monitor. We set the maximum progress equal to the size of rom, as we’ll be iterating over the rom memory. It isn’t necessary to use the TaskMonitor, but it provides a more user-friendly interface.

monitor.initialize(rom.getSize());

Create a stack to keep track of addresses of [ and ] and iterate over the rom. Each iteration we update the monitor and get the current instruction byte. If it is an [, we push the current address on the stack. If an ] is encountered. The address of the corresponding [ is popped from the stack. The branch destination of the ] is set to the address of the [. The branch destination of the [ is set to the address of the instruction following the ]. (It should jump over, but not to, the ].)

Stack<Long> addrStack = new Stack<Long>();
for(long addr = 0; addr < rom.getSize(); addr++){
    monitor.checkCanceled();
    monitor.incrementProgress(1);

    byte instr = readInstruction(rom, addr);
    switch(instr){
        case OPEN_LOOP:
            addrStack.push(addr);
            break;
        
        case CLOSE_LOOP:
            if(addrStack.empty())
                throw new UnmatchedBranchException(addr);

            long openInstr = addrStack.pop();
            writeBranchDest(bdest, addr, openInstr);
            writeBranchDest(bdest, openInstr, addr + 1);
            break;
    }
}

if(!addrStack.empty())
    throw new UnmatchedBranchException();

If a branch is unmatched (has no corresponding [ or ]), a custom exception called UnmatchedBranchException is raised.

UnbalancedLoopException.java:

package brainfuck;

public class UnmatchedBranchException extends RuntimeException {
    public UnmatchedBranchException() {
        super();
    }

    public UnmatchedBranchException(long address){
        super("Unmatched branch at address 0x" + Long.toHexString(address));
    }
}

The rest of the source code can be found here.

Once this is implemented, writing semantics for the control flow instructions is very easy. The [ checks the current cell value. If it is non-zero, execution should continue at the next instruction and this instruction is essentially a NOP. If the cell is zero, a branch should be performed. The address is retrieved from bdest.

:[ is op=0x6 {
    local c:2 = *[ram]ptr;

    if(c != 0x0) goto <end>;
    local idx:2 = inst_start;
    local dest:2 = *[bdest]idx;
    goto [dest];
    
    <end>
}

The semantics for ] are even easier. It just branches to the corresponding [:

:] is op=0x7 {
    local idx:2 = inst_start;
    local dest:2 = *[bdest]idx;
    goto [dest];
}

We could have also implemented semantics for ] to check the current cell value and perform a branch accordingly. The current implementation is functionally equivalent to that, is shorter and follows the construction that is commonly used in binaries emitted by compilers (arrows represent branch destinations):

┏━━━> compare
┃ ┌─< conditional branch
┃ │   code (condition false)
┗━│━< unconditional branch to compare
  └─> code (condition true)

The produced flow control graph for the brainfuck binary after running the Brainfuck Branch Destination Resolver analyzer is now identical to the graph produced for the ARM binary:

It also produces a reasonable decompilation (c0 is cell zero):

void start(void){
    while (c0 != 0)
        c0--;

    c0 = 1;

    halt_baddata();
}

The decompiler recognizes the redundancy of +- and it does not show up in the decompilation.

That’s it! The decompiler is now able to decompile any compiled brainfuck binary. Call bfc.py on your favorite brainfuck program and see what the decompilation looks like in Ghidra. One thing that will massively clean up the decompilation is recognizing common patterns. Right now [-] is decompiled to while(c!=0) c--;, while it could theoretically also be decompiled to the more concise c=0;. This is something we’ll take a look at in a future post.

Next time, we’ll make some small improvements to our processor module. See you then!

You can find the final code for this post here.

Footnotes

To reproduce this, assemble it using arm-none-eabi-as -march=armv6-m -mthumb control_flow_test_arm.s -o control_flow_test_arm.bin (change ; to //) and open control_flow_test_arm.bin it in Ghidra. ↩
Even marking rom as read-only using the <readonly> tag in the .cspec didn’t work. ↩

Implementing a brainfuck CPU in Ghidra - part 2: Decompilation of pointer and arithmetic instructions

Wed, 17 Jun 2020 00:00:00 +0000

This blog post is part of the series Implementing a brainfuck CPU in Ghidra:

Setup and disassembly
Decompilation of pointer and arithmetic instructions (current post)
Decompilation of I/O and control flow instructions
Renaming the analyzer and adding a manual
Recognizing common patterns (future post)

More posts may be added to this series in the future.

In the previous post we created a processor module for a compiled version of brainfuck. Right now, it can only disassemble brainfuck binaries, but not decompile them. In this post we’ll take a look at the decompilation part of the processor module. More specifically, we’ll implement the pointer (> and <) and arithmetic (+ and -) instructions. We’ll handle the other instructions later.

Auto analysis

Before we get to writing semantics for the instructions, let’s first address a problem from the previous post: the auto analysis does not disassemble the binary.

When a binary is opened with the code browser, Ghidra asks if it should perform auto analysis on the binary. Upon clicking yes the analysis options menu opens, showing all the available analyzers for this binary. Among these is the Disassemble Entry Points analyzer:

The purpose of this analyzer is disassembling code at the entry points of the binary. If we mark address 0x0000 as an entry point, this analyzer should automatically disassemble it when we perform the auto analysis. Marking entry points is very simple, though there is no documentation on it. Add the following code to the <processor_spec> tag in brainfuck.pspec:

<default_symbols>
    <symbol name="start" address="rom:0x0000" type="code" entry="true"/>
</default_symbols>

This creates a symbol start (you can rename it) at address 0x0000 and marks it as entry point. If we now delete mul2.bin from Ghidra, reimport it¹ and run the auto analysis, the binary will be decompiled automatically. The stack analyzer throws an error, because there is no stack. This is safe to ignore. (Optionally, this analyzer can be disabled in the analysis options menu.)

Pointer instructions

The semantics of the pointer instructions (> and <) are the simplest to implement. Let’s implement the semantics of > first. The semantics of < can be implemented very similarly.

The > instruction adds 1 to the ptr register. This can be represented using the SLEIGH operator +²:

:> is op=0x0 {
    ptr = ptr + 1;
}

The semantics for < are almost identical, except that the + is now a -:

:< is op=0x1 {
    ptr = ptr - 1;
}

Arithmetic instructions

The arithmetic instructions (+ and -) are slightly more complex than the pointer instructions. They do not operate on a register, but on memory (ram) pointed to by the ptr register. The arithmetic instructions can be broken up into three operations:

Load the value of the current cell from ram into a temporary variable (let’s name it c for cell). In SLEIGH this looks like: local c:2 = *[ram]ptr;
Increment or decrement the cell value by one: c = c + 1 or c = c - 1
Store the new value in the current cell: *[ram]ptr = c;

Thus, we have:

:+ is op=0x2 {
    local c:2 = *[ram]ptr;
    c = c + 1;
    *[ram]ptr = c;
}

:- is op=0x3 {
    local c:2 = *[ram]ptr;
    c = c - 1;
    *[ram]ptr = c;
}

P-code and varnodes

This section covers the internal representation of instruction semantics in Ghidra. It isn’t strictly necessary to understand this in order to create a processor module, so you may skip it. I do advise you to read it though, as it can be very useful to have some knowledge of the underlying representation that Ghidra uses.

As described in the previous post, Ghidra doesn’t use the .slaspec directly. Instead it uses the .sla file, which is compiled from the .slaspec file by the SLEIGH compiler³. The .slaspec is readable and human-friendly, while the .sla is very verbose and more machine-friendly. Similarly, Ghidra doesn’t use SLEIGH operators and symbols directly to describe instructions, but p-code operations and varnodes.

While the SLEIGH operators have a C-like syntax, p-code operations have an assembly-like syntax. Consider the following SLEIGH semantics, for example:

local x:2;
x = 36;
x = x + 44;

This may be translated to the following p-code⁴:

$U10:2 = COPY 36:2
$U10:2 = INT_ADD $U10, 44:2

The SLEIGH compiler has created a varnode in the unique address space for the temporary symbol x in the SLEIGH semantics. This varnode starts at address 0x10, has a size of 2 bytes and is denoted by $U10 or $U10:2.

The first operation, COPY, copies the value 36 into $U10. The second operation, INT_ADD, adds 44 to $U10 and stores the result in $U10.

Now that we know more about the translation from SLEIGH semantics to p-code, let’s view the generated p-code for our implemented SLEIGH semantics.

If we open mul2.bin in the code browser, we can view the p-code of the >, <, + and - instructions by enabling the p-code field in the listing. First, click the Edit the Listing fields button in the top right corner of the listing. The icon looks like a wall above an arrow:

A menu should open now. Click the Instruction/Data tab:

Right-click the PCode field and enable it. The listing now shows the p-code for every instruction that has p-code associated with it:

You may also enable the more verbose raw p-code. To do this go to Edit → Tool Options..., then Listing Fields → Pcode Field and enable the Display Raw Pcode option. The raw p-code for the + looks like this:

You’ll rarely need to look at (raw) p-code during disassembly and decompilation of real-world programs. Still, it can be useful to have this field enabled during the development of a processor module, when emulating code or developing abstract analyzers. In most other cases you needn’t worry about these low-level details.

Decompilation

Ghidra should now be able to decompile any brainfuck programs that use (only) the >, <, + and - instructions. Here is such a program, mark_uneven.bf:

>+>
>+>
>+>

This program increments the first three cells that have an uneven address. It’s not particularly useful, but it only uses instructions that we have described the semantics of.

We can compile it (./bfc.py mark_uneven.bf mark_uneven.bin) and load it into Ghidra. Thanks to the entry point definition in the .pspec, Ghidra creates a function called start at 0x0000 and disassembles it. When we click the function, the decompile window should show a decompilation. Unfortunately this is not what happens. What happens is that the decompiler shows an error:

This is because of the following bug: the decompiler refers to the stack⁵, but we haven’t specified one, because there is no stack in brainfuck⁶. This causes the decompiler to crash. To work around this, we can create a dummy register and address space in the .slaspec:

define space dummy_space type=ram_space size=2 wordsize=2;
...
define register offset=0x00 size=2 [pc ptr dummy_reg];

We’ll use dummy_space and dummy_reg anytime a register or address space is required that doesn’t exist in the brainfuck architecture. The stack pointer can be defined by adding the following code to the <compiler_spec> tag in the .cspec:

<stackpointer register="dummy_reg" space="dummy_space" growth="negative"/>

We’re not done yet (but almost, so don’t worry!). If we’d run the decompiler now, it would - again - show an error:

This error is more descriptive than the previous one: the decompiler won’t decompile instructions at address 0x0000, because the memory hasn’t been marked as global. Any memory not marked as global is assumed to be temporary. Hence, it is important that we mark rom and ram as global. Add the following code to the <compiler_spec> tag in the .cspec:

<global>
    <range space="rom"/>
    <range space="ram"/>
</global>

While we’re at it, we can also add ram memory to the .pspec so that it shows up in the program tree. (rom is already shown in the program tree, because it is the default address space.) Add this to the <processor_spec> tag in the .pspec:

<default_memory_blocks>
    <memory_block name="ram" start_address="ram:0x0000" length="0x1000" mode="rw"/>
</default_memory_blocks>

Now, the decompiler is able to decompile the binary:

It shows two warnings, because the function unexpectedly terminates, instead of returning, endlessly looping or calling to a non-returning function (e.g. exit). These warnings can be safely ignored.

Cleaning up decompilation

The current decompilation doesn’t look very good, but we can improve it. The increments are represented in the decompilation by:

*(short *)(in_ptr + x) = *(short *)(in_ptr + x) + 1

This can be drastically improved by assigning an initial value to the ptr register. This value is called an assumption. Press ctrl + R on the start of the function and set the value of ptr (16) to 0. The decompilation looks a lot better now:

Instead of setting the ptr value to 0 for every brainfuck binary we want to disassemble, we can also add some code to <processor_spec> in the <processor_spec> tag of the .pspec (or .cspec, which also accepts the <context_data> tag):

<context_data>
    <tracked_set space="rom" first="0x0000" last="0x0000">
        <set name="ptr" val="0"/>
    </tracked_set>
</context_data>

This code lets Ghidra assume that the value of ptr equals 0 at the function starting at address 0x0000, also known as the entry point in this case.

If we now reimport the binary, ptr is automatically assumed to be 0 at address 0x0000.

The final step in cleaning up the decompilation is marking the first three uneven addresses as data (uint16_t or ushort) and naming them. This results in the following readable decompilation:

An even better decompilation would be the following:

void start(void){
    uneven_1 = 1;
    uneven_3 = 1;
    uneven_5 = 1;
}

This would require Ghidra to assume that ram is zero at address 0x0000. I don’t think this is possible. (I hope Cunningham’s Law holds in this case and someone proves me wrong.)

That’s it for this post. We’ve implemented decompilation of the pointer and arithmetic instructions and cleaned up the decompilation. We’ve also taken a quick look at Ghidra’s internal representation of instruction semantics (p-code). In the next post, we’ll complete the semantics for the brainfuck instruction set by writing code for the I/O (. and ,) and control flow ([ and ]) instructions. See you then!

You can find the final code for this post here.

Footnotes

From now on it is assumed that you reimport binaries everytime a change is made to the .ldefs, .pspec, .cspec and in some cases even the .slaspec. ↩
Refer to SLEIGH - 7.7. The Semantic Section for information on the semantics section of constructors. ↩
This compilation is done automatically by Ghidra when it encounters a .slaspec, so there is no need for us to compile it. ↩
See the P-Code Reference Manual and SLEIGH - 9. P-code Tables, both part of the Ghidra Language Specification. ↩
The decompiler dereferences pointers to the stack space in a lot of places. For example, coreaction.cc:1644 calls a function gather with a pointer to the stack space. Since there is no stack space defined, this pointer is NULL. gather dereferences the pointer at varmap.cc:542. The dereference of this NULL pointer will result in a segmentation fault: The decompiler dies. ↩
It could be argued whether this is really a bug, as virtually every architecture uses a stack. ↩

Implementing a brainfuck CPU in Ghidra - part 1: Setup and disassembly

Wed, 17 Jun 2020 00:00:00 +0000

This blog post is part of the series Implementing a brainfuck CPU in Ghidra:

Setup and disassembly (current post)
Decompilation of pointer and arithmetic instructions
Decompilation of I/O and control flow instructions
Renaming the analyzer and adding a manual
Recognizing common patterns (future post)

More posts may be added to this series in the future.

Ghidra supports quite some processors out of the box, but if you come across a somewhat obscure architecture, chances are you’ll have to write the processor module yourself (if someone else hasn’t published it on the internet already). At first glance, it may seem daunting: it’s not clear where to start and the documentation is often lacking. That’s why I decided to write this blog post series. The goal is to create a Ghidra processor module for a compiled version of the esoteric programming language brainfuck, from scratch. The sources I used are the Ghidra Language Specification¹, the Compiler Specification² and the source code³.

Brainfuck

Brainfuck is a very minimal language consisting of only eight instructions. See the wikipedia page for a full description of the language. My brainfuck variant deviates slightly from the original:

The memory array is called ram memory and consists of 0x10000 16-bit cells. The pointer ptr (16-bit) points to the current cell.

Instructions are compiled to a binary format, using the following translation table:

Instruction	Opcode
`>`	`0x0`
`<`	`0x1`
`+`	`0x2`
`-`	`0x3`
`.`	`0x4`
`,`	`0x5`
`[`	`0x6`
`]`	`0x7`

Compiled instructions are 8-bit wide. The four least significant bits make up the opcode, the other bits are unspecified and should be set to zero.

Compiled instructions live in rom memory. The program counter pc (16-bit) points to the current instruction.

I created a simple brainfuck compiler (bfc.py) that takes a brainfuck file and outputs a binary. Non-instruction characters (anything but ><+-.,[]) are interpreted as comments and will not be compiled. I didn’t bother to write a VM that executes compiled brainfuck, although it wouldn’t be hard to write yourself. (If you do, I’d love to see it.)

Let’s compile a simple program to see the compiler in action. mul2.bf takes some user input, multiplies it by two and prints the result. The source code is as follows:

,          store user input in #0
[>++<-]    #1 = #0 * 2
>.         move to #1 and print its content

Compile it to mul2.bin and show the generated binary:

$ ./bfc.py mul2.bf mul2.bin
$ hexdump -C mul2.bin
00000000  05 06 00 02 02 01 03 07  00 04                    |..........|
0000000a

The rest of this blog post series will be dedicated to reversing and analyzing compiled brainfuck binaries like mul2.bin.

Preparation

We’ll be using Eclipse to create the processor module. After installing Eclipse, we’re ready to add the GhidraDev extension to Eclipse. The extension can be found in Extensions/Eclipse/GhidraDev/ in the Ghidra installation directory, along with some documentation on how to install this extension. The extension can be added by opening the Help → Install New Software... menu in Eclipse and selecting the GhidraDev zip. The documentation covers this in more detail.

Although we’re using Eclipse for development, you can use any other IDE of your preference for editing the files. I recommend VS Code with the SleigHighlight extension for syntax highlighting of .slaspec files. XML syntax highlighting of .ldefs, .pspec and .cspec files is done in VS Code by marking them as XML in the bottom right corner. This is not required, it just eases the development of processor modules.

Creating a project

Create a new project by going to File → New → Project and select the Ghidra Module Project wizard 🧙.

Specify a name and directory for this project and hit next. The next screen lets us choose the components we want the wizard to add to our project. We only need the analyzer and the processor. Disable the others.

If you haven’t used Eclipse with Ghidra before, you’ll need to specify the root directory of your Ghidra installation and hit next. The last step lets us enable Python support. We don’t need this, so leave the checkbox unchecked.

Then hit finish. We’ve now created our Ghidra project!

Creating the language definitions

The processor lives in data/languages. The wizard has already populated this folder with definitions for the (hypothetical) skel processor, which are of no use to us. Delete these files.

In Ghidra you don’t define processors, but languages instead. A language specifies a variant of a processor family. For example, x86.ldefs contains specifications for 16-bit, 32-bit and 64-bit variants of the x86 processor architecture. Ghidra interprets all files with an .ldefs extension as a language definitions file⁴. The format of this file is not really documented, but there is an XML schema file (written in RELAX NG) that describes the structure of .ldefs files: language_definitions.rxg. Using this schema we can compose brainfuck.ldefs (create a new file using Right click on languages → New → File):

<?xml version="1.0" encoding="UTF-8"?>

<language_definitions>
    <language processor="brainfuck"
              endian="little"
              size="16"
              variant="default"
              version="1.0"
              slafile="brainfuck.sla"
              processorspec="brainfuck.pspec"
              id="brainfuck:2:default">
        <description>brainfuck</description>
        <compiler name="default" spec="brainfuck.cspec" id="default"/>
    </language>
</language_definitions>

There is only one variant of the brainfuck processor, so we only specify one language. The <language> tag has a few required attributes that define the most important properties of the processor variant. One thing to note is the processorspec attribute. This attribute points to a .pspec file, which specifies the processor. The file specified in the .sla file is responsible for providing Ghidra information about the disassembly and decompilation of machine code. Optionally, you can add a processor manual using the manualindexfile attribute. This will be covered in a future post.

A <language> tag must contain a <description> subtag and at least one <compiler> subtag (but you may specify more). The <compiler> tag points to brainfuck.cspec, which describes specific information about the compiler. There are some other tags you can include, but they are not interesting for now.

The processor specification

The structure of the .pspec file is described by processor_spec.rxg. There is no official documentation. The file can contain a bunch of tags describing the memory and registers of the processor, all of which are optional. We’ll leave it empty for now:

<?xml version="1.0" encoding="UTF-8"?>

<processor_spec>
</processor_spec>

The compiler specification

The layout and purpose of the .cspec file is pretty well⁵ documented in the Compiler Specification:

The compiler specification is a required part of a Ghidra language module for supporting disassembly and analysis of a particular processor. Its purpose is to encode information about a target binary which is specific to the compiler that generated that binary. Within Ghidra, the SLEIGH specification allows the decoding of machine instructions for a particular processor, like Intel x86, but more than one compiler can produce those instructions. For a particular target binary, understanding details about the specific compiler used to build it is important to the reverse engineering process. The compiler specification fills this need, allowing concepts like parameter passing conventions and stack mechanisms to be formally described.

Only the <default_proto> tag is required. This tag describes the default calling convention (prototype) for this compiler. Brainfuck has branches, but it doesn’t have calls, so there is not really a prototype. We create a prototype as minimal as possible:

<?xml version="1.0" encoding="UTF-8"?>

<compiler_spec>
    <default_proto>
        <prototype name="default" extrapop="unknown" stackshift="0">
            <input></input>
            <output></output>
        </prototype>
    </default_proto>
</compiler_spec>

Be sure to read the Compiler Specification and compiler_spec.rxg for more detailed information.

The language specification

The .sla file describes the instruction set and makes it possible for Ghidra to disassemble and decompile binaries. The file contains information for both disassembly and decompilation. The file describes the translation from machine code to a textual representation of each instruction (mnemonic, operands, etc.) for the disassembly. It also describes how these instructions affect memory and registers using an intermediate language called p-code. P-code operations operate on varnodes, which are generalizations of memory. A CPU register is a varnode, as well as a word or halfword in memory. The information in the .sla is used by the decompiler to construct the decompiled code.

Writing .sla files manually is very tedious. (Just look at 6502.sla if you don’t believe me.) Luckily, you don’t have to write .sla files by hand. You can write .slaspec files instead, which are much more pleasant to write and maintain. (Compare 6502.slaspec to 6502.sla in terms of readability.) These .slaspec files are automatically compiled to .sla files by Ghidra.

The .slaspec is very well documented by the Ghidra Language Specification, so I will not go too deeply into the syntax of the .slaspec. Refer to the official specification for that.

The very first step is to create brainfuck.slaspec. Ghidra will compile this to brainfuck.sla, which is referred to by the .ldefs.

.slaspec files always start by defining the endianness of the processor, usually followed by an instruction alignment definition. The brainfuck processor is little endian and instructions are aligned to a 1-byte boundary (which is equivalent to no alignment, but let’s define it anyway):

define endian=little;
define alignment=1;

The next step is to define the address spaces:

define space register  type=register_space  size=2  wordsize=2;
define space ram       type=ram_space       size=2  wordsize=2;
define space rom       type=ram_space       size=2  wordsize=1  default;

This is rather straightforward, except for one thing: the Ghidra Language Specification mentions the space type rom_space, but this type does not exist⁶. Using this space type will result in a crash. That’s why rom is defined as ram_space, instead of rom_space.

The rom space is marked as default. This causes the binary to be loaded into the rom space. Note that the registers get their own address space.

Then we define registers in the register space:

define register offset=0x00 size=2 [pc ptr];

We can refer to these defined address spaces and registers in the instructions, but before we can define those instructions, we have to define a token. A token specifies the layout of an instruction or a part of it (in variable length instructions). Instructions in the brainfuck instruction set are very simple. The four least significant bits ([0,3]) define the opcode of the instruction. The other bits are undefined. Our token looks like this:

define token instr(8)
    op = (0, 3)
;

We can now finally define the instructions themselves. For now we’ll only define the translation from machine code to the textual representation (disassembly) and not their semantics (decompilation). This is very simple. The > instruction looks like this:

:> is op=0x0 {}

Let’s break this down. The : indicates that this ‘translation’, more formally called a constructor, should be added to the root instruction table. All instructions live in this table. Subtables can be used for constructing more complex instructions, which you can read about in the Ghidra Language Specification.

After the : comes the textual representation, formally called the display section, of the instruction. For this instruction, this is simply >. If an instruction has operands they can also be specified here (add op1, op2, for example).

The display section is followed by the is keyword, which itself is followed by the bit pattern section. The bit pattern section is used to match the bits of the machine code to constructors. The bit pattern section for this instruction (and all other brainfuck instructions) consists of one constrain: the opcode is 0x0. No more is needed to identify the > instruction.

Constructors end with a semantic section, surrounded by curly braces ({}). This section describes how instructions manipulate memory and registers. We leave it empty for now.

All constructors for compiled brainfuck instructions can be defined like this:

:> is op=0x0 {}
:< is op=0x1 {}

:+ is op=0x2 {}
:- is op=0x3 {}

:. is op=0x4 {}
:, is op=0x5 {}

:[ is op=0x6 {}
:] is op=0x7 {}

Testing

This is all that’s needed to describe the language, processor, compiler and instruction set. We can now test it. Hit the debug button in the toolbar to start a debugged instance of Ghidra with our module automatically loaded in.

Create a new project in Ghidra and give it a name. Now we need a brainfuck file to disassemble. I’ll be using the previously generated mul2.bin. Drag the binary into Ghidra and open it with the code browser. Ghidra asks if it should analyze the file. It doesn’t matter whether you choose yes or no, because we haven’t told Ghidra how to auto-analyze brainfuck binaries. Nothing will happen if you click yes.

The listing view shows the bytes of our binary. To disassemble them press D at the first address. The listing will now show the disassembly of our binary, which should look similar to this:

Hooray! We’ve now created an overcomplicated brainfuck disassembler that could be replaced by a few lines of python code. This seems like a huge overkill, but in the next post we’ll look at getting the decompiler to work so we can do some real reversing on brainfuck binaries. See you then!

You can find the final code for this post here.

Footnotes

See docs/languages/ in your Ghidra installation. Online (compiled) version available here. ↩
See Ghidra/Features/Decompiler/src/main/doc/ in the source code. You’ll have to compile these yourself (see issue #472). Online (compiled) version available here. ↩
NationalSecurityAgency/ghidra on GitHub, commit eaf6ab2. It may not be the latest commit by the time you’re reading this. ↩
The SleighLanguageProvider is responsible for finding and parsing these .ldefs files. ↩
AFAIK, the following valid tags are not documented in the Compiler Specification: spacebase, deadcodedelay, segmentop, resolveprototype, eval_current_prototype and eval_called_prototype. Some of them are documented elsewhere. There might be other subtle differences between the docs and the code. ↩
See space_class.java. Only register_space and ram_space are defined. ↩