---
description: >-
  Calling conventions refer to the standardised rules used for function calls.
  They define how function parameters are passed, and how return values are
  handled and how the stack is managed.
---

# Calling Conventions

Windows primarily uses the `__stdcall` calling convention when calling Win32 APIs from 32-bit code. 64-bit differs; it uses the x64 Application Binary Interface (ABI). It is similar, but not to be confused with, the `__fastcall` calling convention.

## Parameters

When invoking a Win32 API, the initial four arguments are typically passed via registers, commonly `rcx`, `rdx`, `r8`, and `r9` (except for floating-point arguments). If the API call requires additional arguments beyond the initial four, they are typically passed on the stack. Consider the following example:

{% code title="MessageBoxA Syntax" lineNumbers="true" %}
```c
int MessageBoxA(
  [in, optional] HWND   hWnd,
  [in, optional] LPCSTR lpText,
  [in, optional] LPCSTR lpCaption,
  [in]           UINT   uType
);
```
{% endcode %}

When calling the `MessageBoxA` function:

1. The `hWnd` parameter is passed in the `rcx` register.
2. The `lpText` parameter is passed in the `rdx` register.
3. The `lpCaption` parameter is passed in the `r8` register.
4. The `uType` parameter is passed in the `r9` register.

**Note**: any parameters that do not fit into an 64 bit register must be passed by reference. In the example given `lpText` and `lpCaption` must be passed as pointers to the string.

The assembly given below shows how this call would be made:

{% code title="Call MessageBox Snippet" lineNumbers="true" %}
```nasm
; this code assumes that the address of the MessageBox function is in r10
call_messageboxa:
    xor  rax, rax                     ; RAX = 0
    mov  rcx, rax                     ; RCX = hWnd = NULL
    mov  r9,  rax                     ; R9 = uType = NULL (default)
    mov  rax, 0x646c72                ; push HelloWorld\0 on to the stack
    push rax                          ; .
    mov  rax, 0x6f57206f6c6c6548      ; .
    push rax                          ; ---
    mov  rdx, rsp                     ; RDX = lpText = *HelloWorld\0
    mov  r8,  rdx                     ; R8 = lpCaption = *HelloWorld\0
    call r10                          ; CALL R10 (MessageBoxA)
```
{% endcode %}

**Note:** we have not discussed how to find the address of functions yet. It is also worth noting that in the example the parameters are not moved in to the registers in any specific order. The `rax` register was nullified on line `1` so it makes sense to assign this value to `hWnd` and `uType` in this order (lines `4` and `5`).

The assembly instructions given above do not take into account another x68 ABI convention relating to stack space and stack alignment, that will be discusssed shortly.

## Return Values

Return values that fit into 64-bits are returned in to the `rax` register. Vaues that are larger than 64-bits, such as `__m128`, `__m128i`, and `__m128d` are returned in the `xmm0` register. These will not be discussed now.

Depending on the type being returned this could be a value or a pointer to a value. An `int` will be stored directly as a value in `rax` but a pointer will contain the memory address of the 'returned' value.&#x20;

In the `MessageBox` example the return value is an `int` type that refers to a user action. If the **Cancel** button is pressed the function returns an `IDCANCEL` value represented by a `2`. It is always  good practice to refer to the Microsoft Win32 API documentation.

## Shadow Space and Stack Alignment

When making a function call in the x64 calling convention, it is necessary to allocate 32 bytes (0x20) on the stack for the called function to utilise, even if there are no additional arguments beyond the four registers. This allocation is done to ensure stack alignment and is referred to as **shadow space**.

Additionally, the stack space should be aligned to a 16-byte boundary. Allocating 32 bytes on the stack ensures this alignment. However, if you push values to the stack, such as strings, it may be necessary to re-align the stack before the function call to maintain the required alignment.

In the `MessageBox` example above we push 24 bytes on to the stack, this misaligns the stack by 8 bytes. Before we make the call to `MessageBox` we need to push 40  bytes instead of 32 bytes:

{% code title="Call MessageBox Snippet 2" lineNumbers="true" %}
```nasm
sub  rsp, 0x28                    ; Allocate 40 bytes for call 
                                  ; and stack alignment   
call r10                          ; CALL R10 (MessageBox)
add  rsp, 0x20                    ; cleanup allocated stack space
```
{% endcode %}

By adhering to proper stack alignment and allocating the necessary space, you can ensure consistent behavior and compatibility when making function calls in the x64 calling convention.

In x64 ABI it is also the callers responsibility to clean up the stack when the function returns. This can be seen on line `4`.

The full assembly listing is shown below:

{% code title="Call MessageBox Snippet 3" lineNumbers="true" %}
```nasm
; this code assumes that the address of the MessageBox function is in r10
call_messagebox:
    xor  rax, rax                     ; RAX = 0
    mov  rcx, rax                     ; RCX = hWnd = NULL
    mov  r9,  rax                     ; R9 = uType = NULL (default)
    mov  rax, 0x646c72                ; push HelloWorld\0 on to the stack
    push rax                          ; .
    mov  rax, 0x6f57206f6c6c6548      ; .
    push rax                          ; ---
    mov  rdx, rsp                     ; RDX = lpText = *HelloWorld\0
    mov  r8,  rdx                     ; R8 = lpCaption = *HelloWorld\0
    sub  rsp, 0x28                    ; Allocate 40 bytes for call 
                                      ; and stack alignment   
    call r10                          ; CALL R10 (MessageBox)
    add  rsp, 0x20                    ; cleanup allocated stack space
```
{% endcode %}

In some instances we can simply align the stack using the `and rsp, byte -0x10` instruction.

## Checking Stack Alignment

We can check stack alignment by placing an `int3` instruction just before the `call` instruction. When the breakpoint is hit, we can check that the current value in `rsp` is correctly aligned by issuing the `?rsp%10` command. If the result is `0` then the stack is correctly aligned.

## Advanced Win32 Calls and the Stack

When making calls to functions that require more than 4 arguments it is necessary to push the remaining arguments on to the stack. This can be quite confusing at first but is quite simple if we follow some basic rules.

Let's take the `WinHttpSendRequest` call as an example, the `C` code is shown below:

{% code overflow="wrap" lineNumbers="true" %}
```c
BOOL result = WinHttpSendRequest(hRequest, WINHTTP_NO_ADDITIONAL_HEADERS, 0, WINHTTP_NO_REQUEST_DATA, 0, 0, 0);
```
{% endcode %}

And here is the syntax:

```c
WINHTTPAPI BOOL WinHttpSendRequest(
  [in]           HINTERNET hRequest,
  [in, optional] LPCWSTR   lpszHeaders,
  [in]           DWORD     dwHeadersLength,
  [in, optional] LPVOID    lpOptional,
  [in]           DWORD     dwOptionalLength,
  [in]           DWORD     dwTotalLength,
  [in]           DWORD_PTR dwContext
);
```

This call is used as an example because it is a simple call; we can pass `NULL` or `0` to all of the arguments except `hRequest`. Before the call is made, this is how the stack and the registers should look:

<figure><img src="https://1133556953-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6fSnuRpvF32xyL2X75m5%2Fuploads%2FYCK2xyXTgQ0xAZ70gjcC%2Fpre-call-alignment.png?alt=media&#x26;token=2227fb98-1128-4797-be47-23f0fcccfaa1" alt=""><figcaption><p>Arguments pre-WinHttpOpenRequest call</p></figcaption></figure>

Here is an example of the assembly that would be used to make the call:

{% code overflow="wrap" lineNumbers="true" %}
```nasm
call_WinHttpSendRequest:
    mov rcx, [rbp-0x40]             ; RCX = hRequest (returned from 
                                    ; WinHttpOpenRequest)
    xor rdx, rdx                    ; RDX = lpszHeaders = NULL
    xor r8, r8                      ; R8 = dwHeadersLength = 0
    xor r9, r9                      ; R9 = lpOptional = 0
    sub rsp, 0x10                   ; Align stack BEFORE the pushes (this value
                                    ; may differ for others depending on stack
                                    ; alignment)       
    xor r11, r11                    ;
    push r11                        ; dwContext= 0
    push r11                        ; dwTotalLength = 0
    push r11                        ; dwOptionalLength = 0
    sub rsp, 0x20                   ; Allocate stack space for the function call
                                    ; AFTER the pushes
    call r10                        ; make the CALL here, assuming the address of
                                    ; WinHttpOpenRequest is in r10
```
{% endcode %}

Remember, when a value is pushed on to the stack, `rsp` is adjusted by 8 bytes to point at a **lower** memory address. So, we push the 5th, 6th, and 7th argument on to the stack then create the shadow space for the callee.

We will discuss writing, running and debugging shellcode in the next section.
