amiga-bootcamp/04_linking_and_libraries/compiler_stubs.md
2026-04-26 14:46:18 -04:00

188 lines
5.6 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

[← Home](../README.md) · [Linking & Libraries](README.md)
# Compiler Library Call Stubs
## Overview
This document explains how each major AmigaOS compiler generates library calls and what the resulting machine code looks like — essential for recognising compiler signatures during reverse engineering.
---
## The AmigaOS Calling Convention
All library calls use the same register-based convention:
- **A6** = library base pointer
- Arguments in **A0A3**, **D0D7** as specified by the `.fd` file
- Return value in **D0** (or D0+D1 for 64-bit returns)
- **Callee** preserves: D2D7, A2A6
- **Caller** may destroy: D0, D1, A0, A1
The code generated by all compilers follows this — the difference is how they **set up A6** before the `JSR`.
---
## SAS/C 6.x
SAS/C uses **global library base variables** (`_DOSBase`, `_SysBase`, etc.) and generates direct `JSR LVO(A6)` calls. The standard pattern:
```asm
; Calling Write(file, buffer, length):
; D1=file, D2=buffer, D3=length, A6=_DOSBase
MOVE.L _DOSBase, A6 ; load library base from global
MOVE.L file, D1 ; arg 1
MOVE.L buffer, D2 ; arg 2
MOVE.L length, D3 ; arg 3
JSR -48(A6) ; Write LVO = -48
```
### SAS/C Inline Pragmas
SAS/C uses `#pragma libcall` to define inline stubs:
```c
#pragma libcall DOSBase Write 30 32103
/* 30 = bias/6 = 8th function (LVO -48)
32103 = register encoding: D3,D2,D1 → args 3,2,1 */
```
The pragma-generated stub wraps the JSR into a C-callable inline function.
### SAS/C Library Base Globals
SAS/C's `clib/dos_protos.h` + startup code declares:
```c
extern struct DosLibrary *DOSBase;
extern struct ExecBase *SysBase;
```
These are initialized in `c.o` (the startup stub):
```asm
_c_start:
MOVE.L 4.W, A6 ; SysBase from exception vector
MOVE.L A6, _SysBase ; store in global
LEA _dosname, A1 ; "dos.library"
MOVEQ #0, D0 ; version 0
JSR -552(A6) ; OpenLibrary
MOVE.L D0, _DOSBase
```
---
## GCC (m68k-amigaos / bebbo GCC)
GCC uses a different stub mechanism — **inline functions** from `inline/` headers or the `libnix` library:
```c
/* inline/exec.h (generated from exec_lib.fd) */
static __inline APTR
AllocMem(ULONG byteSize, ULONG requirements)
{
register APTR _res __asm("d0");
register struct ExecBase *const _SysBase __asm("a6") = SysBase;
register ULONG _byteSize __asm("d0") = byteSize;
register ULONG _requirements __asm("d1") = requirements;
__asm volatile ("jsr a6@(-198:W)"
: "=r"(_res)
: "r"(_SysBase), "r"(_byteSize), "r"(_requirements)
: "a0", "a1", "fp0", "fp1", "cc", "memory");
return _res;
}
```
The `jsr a6@(-198:W)` emits a 4-byte instruction: `JSR -198(A6)` using 16-bit word displacement.
### GCC Code Pattern
```asm
; GCC calling AllocMem(1024, MEMF_CHIP):
MOVEA.L _SysBase, A6 ; GCC uses same global SysBase
MOVEQ #$01, D1 ; MEMF_CHIP = 2... actually:
MOVE.L #$00000400, D0 ; byteSize = 1024
MOVE.L #$00000002, D1 ; MEMF_CHIP
JSR -$C6(A6) ; -198 decimal = AllocMem
MOVE.L D0, _mybuf ; save return value
```
### PC-Relative Addressing (GCC Default)
GCC with `-fpic` or `-fbaserel` generates **PC-relative** accesses:
```asm
LEA _SysBase(PC), A0 ; load SysBase address PC-relative
MOVEA.L (A0), A6 ; dereference to get SysBase value
JSR -$228(A6) ; OpenLibrary LVO = -552
```
This reduces relocation entries and is the default for GCC on AmigaOS.
---
## VBCC (vbcc m68k-amigaos)
VBCC uses `inline.h` style stubs similar to GCC. The calling pattern is identical at the assembly level:
```asm
; VBCC compiled library call — indistinguishable from GCC at asm level
MOVEA.L (_SysBase), A6
MOVE.L D2, -(SP) ; VBCC may push/pop D2 as scratch
MOVE.L arg, D0
JSR -$C6(A6)
MOVE.L (SP)+, D2
```
VBCC is slightly more aggressive about not clobbering D2-D7, matching the calling convention exactly.
---
## Recognising Compiler Signatures During RE
### SAS/C Signature
```asm
; Entry prologue:
LINK A5, #-N ; frame setup (A5 = frame pointer)
MOVEM.L D2-D7/A2-A4, -(SP) ; save callee-saved regs
; ...
MOVEM.L (SP)+, D2-D7/A2-A4 ; restore
UNLK A5 ; teardown
RTS
```
SAS/C uses `A5` as frame pointer and `LINK/UNLK` for stack frames.
### GCC Signature
```asm
; Entry prologue (GCC without frame pointer):
MOVEM.L D2/A2, -(SP) ; only saves what it uses
; ...
MOVEM.L (SP)+, D2/A2
RTS
```
GCC typically does **not** use LINK/UNLK unless required. It uses `A5` as an additional scratch register and often generates `JSR` through `A0` for indirect calls.
### VBCC Signature
VBCC generates very clean, minimal code — similar to GCC but with more consistent callee-save patterns and no GCC-specific idioms (e.g., no `__builtin_` calls).
---
## Library Call Pattern Summary
| Pattern | Compiler | Key Signature |
|---|---|---|
| `LINK A5, #-N` + `MOVEM D2-D7/A2-A4` | SAS/C | Frame pointer A5, full reglist |
| `JSR LVO(A6)` with global `_LibBase` | SAS/C / GCC | Both use global |
| `jsr a6@(-LVO:W)` (GAS syntax) | GCC inline | 16-bit short displacement |
| PC-relative `LEA _SysBase(PC)` | GCC -fpic | No absolute refs |
| No LINK, minimal MOVEM | GCC / VBCC | No frame pointer |
---
## References
- NDK39: `fd/`, `proto/`, `inline/`, `clib/` directories
- SAS/C 6.x Programmer's Guide — pragma libcall chapter
- GCC m68k-amigaos port documentation (bebbo/m68k-amigaos-toolchain)
- VBCC documentation — register calling convention
- *Amiga ROM Kernel Reference Manual: Libraries* — calling conventions appendix