JISA: Jallenator Instruction Set Assembler

JISA: Jallenator Instruction Set Assembler - Printable Version

+- Forums - Open Redstone Engineers (https://forum.openredstone.org)
+-- Forum: ORE General (https://forum.openredstone.org/forum-39.html)
+--- Forum: Projects & Inventions (https://forum.openredstone.org/forum-19.html)
+---- Forum: In Progress (https://forum.openredstone.org/forum-20.html)
+---- Thread: JISA: Jallenator Instruction Set Assembler (/thread-5756.html)

JISA: Jallenator Instruction Set Assembler - Jallen - 02-09-2015

JISA: An easy to use custom instruction set assembler
*UPDATED*
Repo link
*UPDATED*
Prepare for incoming update on the tutorial.

Disclaimer: This post is subject to changes as is the script itself. I will try my best to keep the documentation as up to date as possible but it may not always be right on time.

Overview

JISA is a python script that takes a .yaml file (default name IS.yaml) that describes your instruction set and will then be able to assemble any code written in your assembly language into either a hex or a binary format. Both the source and the output file must be in a plain text format.

Requirements:

Python 3.* Installed
PyYaml package (see requirements.txt in the repo for link)

Constructing your .yaml file

The first thing you must do is have an instruction set. I'll be creating various instructions to demonstrate the way you lay it out.
Within the .yaml file it is important to label everything with bit widths so that the assembler knows how much to fill. It is important to start the file and separate sections with '---' so the script knows where to start.

The .yaml file is constructed of three sections:

Formats
Instructions
Custom arguments

Formats
In this section you define the bit layout of various different types of instruction.
For example: Say I wanted to layout the format for a simple two operand logic type instruction. I am going to have a 4 bit op code for this IS so I must state that the op code will take up 4 bits explicitly:

Code:
---

RR;16:

    - op;4

As you can see, I have also defined the line length in the declaration. RR(Register, Register) will be the format for any instruction that takes two operands and has an op code. However we haven't finished defining the entire format. We have to ensure that the sum of the bit widths of the constituent parts is equal to the total line length.

Code:
---

RR;16:

    - op;4

    - r;3

    - r;3

    - 0;6

We have now completed our first basic format. As you can see I have used 3 different types of predefined argument types (op, r, 0) there are 6 total predefined argument types:

op: op code
r: register argument
i: immediate (can be given in binary or hex(prefix with '0x' for hex))
var: forks the format based on what type of argument is provided
0: ...it's a 0
1: you guessed it!

Now the var argument probably needs a bit of explaining. It's probably easier to explain with an example. So let's change how the RR instruction works. Say I wanted to select between using either a second register or an immediate. I would do something like this:

Code:
---

RRI;16:

    - op;4

    - r;3

    - var;9

    - var:

        r:

            - r;3

            - 0;6

        i:

            - i;8

            - 1;1

I've used the final bit in the line as a signifier bit that decides whether or not our ISA uses an immediate or a register argument.
When you use the var argument type, you have to define the different types of arguments in the final element of the list. It must always be at the end of the format so that the assembler knows where to look for the forks. There are two different types of argument that have valid forks: r and i.
If one of these types of argument is found, the line format follows the corresponding fork.

Custom Arguments

'But Jallen!' I hear you cry 'What if I want to use an argument other than the ones you've already said?!' Well we can do that!
You need to use a custom argument. There are two stages to making a custom argument:

Define
Declare

To define a custom argument, you need to go to the third section of the .yaml file and define what values you can give. Let's use the example of a simple jump instruction. First we have to define it's format:

Code:
---

JMP;16:

    - op;4

    - ~cond;4

    - i;8

---

(see later for defining the instruction)

---

cond:

    'if0': '0001'

    '!if0': '0010'

    'Cout': '0011'

    '!Cout': '0100'

    'gt': '0101'

    'lt': '0110'

    'eq': '0111'

    '!eq': '1000'

...

In this I've both defined the custom argument and declared it. To declare it you must use the '~' symbol in the format followed by the tag and the bit width. In the definition you define the values the argument can take, you then define the binary value each one represents. It is important to use quotation marks so that the assembler treats it as a string rather than a number. It also allows for special character usage in the values your custom can take.

What this .yaml will allow for is a conditional jump instruction to an address specified by the 8-bit immediate and the condition can be any from the list.

Multi-line:

If you want to have a multi-line instruction, it's pretty simple to add it that in:

Code:
---

JMP;16,16:

    - op;4

    - ~cond;4

    - 0;8

    - i;16

---

===

---

cond:

    'if0': '0001'

    '!if0': '0010'

    'Cout': '0011'

    '!Cout': '0100'

    'gt': '0101'

    'lt': '0110'

    'eq': '0111'

    '!eq': '1000'

All you have to do is separate the line lengths for your two or more lines with a comma.

Defining Instructions

It's very simple to define an instruction. All you have to do is declare a keyword with a list of it's format tag and it's opcode. For instance:

Code:
[code]---

JMP;16,16:

    - op;4

    - ~cond;4

    - 0;8

    - i;16

---

jump:

    - JMP

    - '1000'

---

cond:

    'if0': '0001'

    '!if0': '0010'

    'Cout': '0011'

    '!Cout': '0100'

    'gt': '0101'

    'lt': '0110'

    'eq': '0111'

    '!eq': '1000'

Each instruction takes 2 values, the format and the opcode. Again it is important to use quotes for the opcode so the object is kept as a string.
This .yaml file will fully allow us to write a line like this:

Code:
jump !Cout 0xabc1

N.B. you must write the arguments in the order they are delcared in the format. I may add the ability to mix them up in the future but for now, that's how it is.

To use the script enter:

Code:
python <script_name>.py -h

This will show you what arguments you need to use and some other options too

If you have any suggestions or find a bug please leave a comment or find me ingame.

RE: JISA: Jallenator Instruction Set Assembler - Apocryphan - 02-09-2015

And suddenly I felt a big *whoosh* overhead, not my forte but the more i stare at it the more it sorta makes sense.

RE: JISA: Jallenator Instruction Set Assembler - Magic :^) - 02-09-2015

I read through this and it's really helpful Big Grin

I can't wait to implement my IS in this!

RE: JISA: Jallenator Instruction Set Assembler - LordDecapo - 02-09-2015

YES YES YES! A VERSION OF YOUR ASSEMBLER THAT I DONT HAVE TO GIVE YOU MY IS FOR !!! I CAN CONFIG MYSELZZZZZZZZ!!! Big Grin

RE: JISA: Jallenator Instruction Set Assembler - Jallen - 02-09-2015

(02-09-2015, 09:08 PM)LordDecapo Wrote: YES YES YES! A VERSION OF YOUR ASSEMBLER THAT I DONT HAVE TO GIVE YOU MY IS FOR !!! I CAN CONFIG MYSELZZZZZZZZ!!!

That's the idea Tongue

RE: JISA: Jallenator Instruction Set Assembler - greatgamer34 - 02-09-2015

Awesome work man!

RE: JISA: Jallenator Instruction Set Assembler - Magic :^) - 02-10-2015

after this, I just have to parse my binary into ./autoprogram format and I'm all set! Big Grin

RE: JISA: Jallenator Instruction Set Assembler - Jallen - 02-10-2015

depending on the format of your PROM, it's pretty simple. I made one for IizR's prom. Also when you do make your .yaml file, can you send me a copy so I can put it in an examples folder in the repo (same goes for anyone making a .yaml file)

RE: JISA: Jallenator Instruction Set Assembler - LordDecapo - 02-10-2015

Sure,, but u need to look at the new IizR13 PROM... it's kinda diagnal. Lol so the coding for "next line" will be a tad more annoying. If you /warp IizR, then fly in the direction of the old IS board ull see in front Of you,... the new PROM and CPU Are out there... can't miss it, it's 256 lines of diagnal PROM

RE: JISA: Jallenator Instruction Set Assembler - Jallen - 02-10-2015

(02-10-2015, 12:34 PM)LordDecapo Wrote: Sure,, but u need to look at the new IizR13 PROM... it's kinda diagnal. Lol so the coding for "next line" will be a tad more annoying. If you /warp IizR, then fly in the direction of the old IS board ull see in front Of you,... the new PROM and CPU Are out there... can't miss it, it's 256 lines of diagnal PROM

Does it start at the top or somewhere else?

RE: JISA: Jallenator Instruction Set Assembler - Magic :^) - 02-10-2015

(02-10-2015, 12:34 PM)LordDecapo Wrote: Lol so the coding for "next line" will be a tad more annoying.

lol tell me about it, my prom uses torches OR repeaters at different bits, and which one is used is staggered awkwardly aswell xD

RE: JISA: Jallenator Instruction Set Assembler - tokumei - 02-11-2015

It would be nice to have native ./autoprogram support Smile

RE: JISA: Jallenator Instruction Set Assembler - Jallen - 02-11-2015

(02-11-2015, 03:55 PM)DJ8X Wrote: It would be nice to have native ./autoprogram support

The only problem is that everyone (as you can see from this thread) has different ROM configurations so adding native support would mean a different section of code for everyone's CPU. I'd be willing to help out anyone who was going to make a parser but adding it into the script would be impractical.

RE: JISA: Jallenator Instruction Set Assembler - LordDecapo - 02-12-2015

It would be easier just to make a filter program to turn the binary assembly into autoprogram
hell, have the program prompt for the variables it needs to generate the autoprogram, then have it just do a final pass to convert it the last bit. But I would still want the plane binary output as well

RE: JISA: Jallenator Instruction Set Assembler - Jallen - 02-12-2015

I'd rather just focus on getting the assembler itself going with as much functionality as possible before adding another stage. At the moment I'm looking for the ability to split variables across the instruction (e.g. if you had your op code in two halves at either end of your bit layout or something). Also I want to add the ability to place the instruction keyword anywhere in the line rather than always at the start. I'd like to actually have a chat with a few people about what features they would find important for their instruction set and also the actual layout of the .yaml file. There is a lot that can be improved on and a lot will change over time. For now, it should work for simply laid out ISs but I don't want that to be the only case it will work in.

RE: JISA: Jallenator Instruction Set Assembler - Magic :^) - 02-12-2015

Could I also suggest supporting optional arguments in an instruction?

For example, I have an instruction where 1 bit decides wether or not to invert a register value.
At the moment I can have an argument to specify wether or not to invert, but It's kinda awkward how I have to say when I DON'T want it inverted too.

e.g. I'd like to be able to define an instruction that could take such combinations:

ld r1 r2
ld r1 ! r2
ld r1 r2 !
ld r1 ! r2 !

Another suggestion would be to allow the counting of address lines and a little bit of macro, so you could do this:

ld r1 r2

#loopstart

add r1

jmp $loopstart

RE: JISA: Jallenator Instruction Set Assembler - LordDecapo - 02-12-2015

Address line counting as mentioned by mag... would be amazing

RE: JISA: Jallenator Instruction Set Assembler - Jallen - 02-12-2015

(02-12-2015, 11:16 AM)The Magical Gentleman Wrote: ld r1 r2
ld r1 ! r2
ld r1 r2 !
ld r1 ! r2 !

you could make custom arguments for something line '!r1' instead of having a separate thing then use a var and have one possibility being a normal register (like 'r1') and that puts a 0 in the signifier bit and the reg value in the next bits and the other possibility being the custom argument with a 1 in the signifier bit and then the value you assigned in your definition.

I agree it's a bit of an ugly solution but it's the best I've got at the moment Tongue

As for the addressing and referencing, I love that. It's definitely top of the TODO list.

RE: JISA: Jallenator Instruction Set Assembler - Jallen - 02-23-2015

*UPDATE* See original post

RE: JISA: Jallenator Instruction Set Assembler - fuirippu - 02-23-2015

Hi Jallen,

If I understand: I write an IS.yaml file, provide an input file (a program in my custom assembly-language), and the python program will output the assembled machine code.

A custom assembler would be a very handy tool to have. This is a nice service to the community, props to you.

I have a question/feature request:
Can the tool include line numbers, and the assembly instructions in its output?

With this, the output would be in 3 columns, and might look something like...

00 # 00101101 # LDX $01,$11
01 # 00111000 # LDY $00,$10
.... and so on....

<excuse the bad formatting, I can't work html>
<arbitrary spacing/separator, hopefully you get the idea>

Cheers,

RE: JISA: Jallenator Instruction Set Assembler - Magic :^) - 02-23-2015

You can use this tag for that kinda formatting:

Code:
[code]

(look up BBCode)

RE: JISA: Jallenator Instruction Set Assembler - Jallen - 02-25-2015

(02-23-2015, 03:54 PM)fuirippu Wrote: I have a question/feature request:
Can the tool include line numbers, and the assembly instructions in its output?

With this, the output would be in 3 columns, and might look something like...

Code:
00 # 00101101 # LDX $01,$11 01 # 00111000 # LDY $00,$10 .... and so on....

It would be possible yes, but my question would be why would you want this? The main purpose of the assembler is to provide code that can be easily parsed into something like an /autoprogram format. Also, you can already reference lines by inserting a tag (I haven't added this to the tutorial yet) such as '@address_name' on it's own line which means you can use that line as an address in other places.

RE: JISA: Jallenator Instruction Set Assembler - fuirippu - 02-25-2015

Fair point, if I want a line-numbered-file which contains my program in both assembly and binary formats, it should be easy enough to write a script which combines 1) the output from the JISA and 2) the assembly language input used. If only I had an operating system with 1/2 decent support for scripts... d'oh. (maybe time to dust off MinGW/MSYS).

RE: JISA: Jallenator Instruction Set Assembler - Jallen - 02-25-2015

The only issue is multiline instructions. Would the value at the start reference the PROM address, or the code line?

RE: JISA: Jallenator Instruction Set Assembler - fuirippu - 03-09-2015

I envision using the combined assembly and machine code file as a debugging aide while in-game placing and removing torches, so I think numbering the lines according to their PROM address would be the more useful option.

RE: JISA: Jallenator Instruction Set Assembler - Magic :^) - 03-09-2015

Hopefully we will be able to compile directly to mc in the near future. I'm developing a scripter that will translate your binary to the format read by the autoprogram tool. The only extra step on your part would be defining your rom's physical properties.

RE: JISA: Jallenator Instruction Set Assembler - LordDecapo - 03-11-2015

So can't wait for that Mag. Big Grin

I can't wait to be able to go from a programming Lang (C *like* made by Dylan John and I, most likely others too lol) compile to asm. Use Jall's asm to binary then Mag'so binary to auto program...
I kinda want to combine all 3 steps into 1 single script... may work on that when they are all made

RE: JISA: Jallenator Instruction Set Assembler - Magic :^) - 03-11-2015

The program has been up for a while: http://forum.openredstone.org/showthread.php?tid=5843

I'm not changing the syntax anymore, and any extras will only expand on the scripting lang instead of change it.

RE: JISA: Jallenator Instruction Set Assembler - LordDecapo - 03-11-2015

Omg.. y don't I ever check the programming subforum XD

RE: JISA: Jallenator Instruction Set Assembler - Magic :^) - 03-11-2015

Oh and it won't be too difficult to link the different stages together, as afaik all the programs are used via command line.

so it would be like:
python34 assemble.py <sourcefile>
python34 convert.py <assemble_output> <templatefile>

... don't type those lines in, they are about as pseudocodey as you can get xD