amiga-bootcamp/04_linking_and_libraries/compiler_stubs.md

← Home · Linking & Libraries

Compiler Library Call Stubs

Overview

This document explains how each major AmigaOS compiler generates library calls and what the resulting machine code looks like — essential for recognising compiler signatures during reverse engineering.

---

The AmigaOS Calling Convention

All library calls use the same register-based convention:

• A6 = library base pointer
• Arguments in A0–A3, D0–D7 as specified by the .fd file
• Return value in D0 (or D0+D1 for 64-bit returns)
• Callee preserves: D2–D7, A2–A6
• Caller may destroy: D0, D1, A0, A1

The code generated by all compilers follows this — the difference is how they set up A6 before the JSR.

---

SAS/C 6.x

SAS/C uses global library base variables (_DOSBase, _SysBase, etc.) and generates direct JSR LVO(A6) calls. The standard pattern:

; Calling Write(file, buffer, length):
;   D1=file, D2=buffer, D3=length, A6=_DOSBase

MOVE.L  _DOSBase, A6        ; load library base from global
MOVE.L  file, D1            ; arg 1
MOVE.L  buffer, D2          ; arg 2
MOVE.L  length, D3          ; arg 3
JSR     -48(A6)             ; Write LVO = -48

SAS/C Inline Pragmas

SAS/C uses #pragma libcall to define inline stubs:

#pragma libcall DOSBase Write 30 32103
/* 30 = bias/6 = 8th function (LVO -48)
   32103 = register encoding: D3,D2,D1 → args 3,2,1 */

The pragma-generated stub wraps the JSR into a C-callable inline function.

SAS/C Library Base Globals

SAS/C's clib/dos_protos.h + startup code declares:

extern struct DosLibrary *DOSBase;
extern struct ExecBase    *SysBase;

These are initialized in c.o (the startup stub):

_c_start:
    MOVE.L  4.W, A6             ; SysBase from exception vector
    MOVE.L  A6, _SysBase        ; store in global
    LEA     _dosname, A1        ; "dos.library"
    MOVEQ   #0, D0              ; version 0
    JSR     -552(A6)            ; OpenLibrary
    MOVE.L  D0, _DOSBase

---

GCC (m68k-amigaos / bebbo GCC)

GCC uses a different stub mechanism — inline functions from inline/ headers or the libnix library:

/* inline/exec.h (generated from exec_lib.fd) */
static __inline APTR
AllocMem(ULONG byteSize, ULONG requirements)
{
    register APTR _res __asm("d0");
    register struct ExecBase *const _SysBase __asm("a6") = SysBase;
    register ULONG _byteSize __asm("d0") = byteSize;
    register ULONG _requirements __asm("d1") = requirements;
    __asm volatile ("jsr a6@(-198:W)"
        : "=r"(_res)
        : "r"(_SysBase), "r"(_byteSize), "r"(_requirements)
        : "a0", "a1", "fp0", "fp1", "cc", "memory");
    return _res;
}

The jsr a6@(-198:W) emits a 4-byte instruction: JSR -198(A6) using 16-bit word displacement.

GCC Code Pattern

; GCC calling AllocMem(1024, MEMF_CHIP):
MOVEA.L _SysBase, A6        ; GCC uses same global SysBase
MOVEQ   #$01, D1             ; MEMF_CHIP = 2... actually:
MOVE.L  #$00000400, D0       ; byteSize = 1024
MOVE.L  #$00000002, D1       ; MEMF_CHIP
JSR     -$C6(A6)             ; -198 decimal = AllocMem
MOVE.L  D0, _mybuf           ; save return value

PC-Relative Addressing (GCC Default)

GCC with -fpic or -fbaserel generates PC-relative accesses:

LEA     _SysBase(PC), A0    ; load SysBase address PC-relative
MOVEA.L (A0), A6            ; dereference to get SysBase value
JSR     -$228(A6)           ; OpenLibrary LVO = -552

This reduces relocation entries and is the default for GCC on AmigaOS.

---

VBCC (vbcc m68k-amigaos)

VBCC uses inline.h style stubs similar to GCC. The calling pattern is identical at the assembly level:

; VBCC compiled library call — indistinguishable from GCC at asm level
MOVEA.L (_SysBase), A6
MOVE.L  D2, -(SP)           ; VBCC may push/pop D2 as scratch
MOVE.L  arg, D0
JSR     -$C6(A6)
MOVE.L  (SP)+, D2

VBCC is slightly more aggressive about not clobbering D2-D7, matching the calling convention exactly.

---

Recognising Compiler Signatures During RE

SAS/C Signature

; Entry prologue:
LINK    A5, #-N              ; frame setup (A5 = frame pointer)
MOVEM.L D2-D7/A2-A4, -(SP)  ; save callee-saved regs
; ...
MOVEM.L (SP)+, D2-D7/A2-A4  ; restore
UNLK    A5                   ; teardown
RTS

SAS/C uses A5 as frame pointer and LINK/UNLK for stack frames.

GCC Signature

; Entry prologue (GCC without frame pointer):
MOVEM.L D2/A2, -(SP)        ; only saves what it uses
; ...
MOVEM.L (SP)+, D2/A2
RTS

GCC typically does not use LINK/UNLK unless required. It uses A5 as an additional scratch register and often generates JSR through A0 for indirect calls.

VBCC Signature

VBCC generates very clean, minimal code — similar to GCC but with more consistent callee-save patterns and no GCC-specific idioms (e.g., no __builtin_ calls).

---

Library Call Pattern Summary

Pattern	Compiler	Key Signature
LINK A5, #-N + MOVEM D2-D7/A2-A4	SAS/C	Frame pointer A5, full reglist
JSR LVO(A6) with global _LibBase	SAS/C / GCC	Both use global
jsr a6@(-LVO:W) (GAS syntax)	GCC inline	16-bit short displacement
PC-relative LEA _SysBase(PC)	GCC -fpic	No absolute refs
No LINK, minimal MOVEM	GCC / VBCC	No frame pointer

---

References

• NDK39: fd/, proto/, inline/, clib/ directories
• SAS/C 6.x Programmer's Guide — pragma libcall chapter
• GCC m68k-amigaos port documentation (bebbo/m68k-amigaos-toolchain)
• VBCC documentation — register calling convention
• Amiga ROM Kernel Reference Manual: Libraries — calling conventions appendix