1. Objectives

The objective of this tutorial is to provide the necessary elements to begin in vulnerability research, reverse engineering and exploit development.
After doing some research on the internet, I found that the world of 64 bits is not necessarily very well represented. Indeed, there are many tutorials/resources on 32 bits but not much on 64 bits. That is why I have decided to make my contribution.
Through these series of tutorials, you will first learn the basics for developing assembler code and developing shellcodes. In a second step, we will approach the areas of exploitation on Linux and Windows systems by trying to show the differences between these two universes. Finally, I will finish with the kernel exploitation.

2. Why learn assembler ?

Whether you are a high-level language developer like python or you are a pentester, learning assembler is an essential step in my opinion.
Take the case of a penetration test: You come across an application developed in-house, which perform administration tasks. On a Unix system, this binary has suid root rights. With assembler basics, you can do a little analysis of this program and why not find a vulnerability to exploit to elevate your privileges.
As a developer, I think it's harder to imagine the utility of this language. However, a good knowledge allows you to better understand what your program is doing and to find ways to optimize it. By developing your programs directly in assembler, you will be able to understand how the STACK works, how to optimize the treatments, how to pass its parameters, ect.

3. What assembler is ?

Processors are physical components on which program instructions are executed. However they do not understand human language or even programming languages. The only thing they understand is binary language, which is a sequence of 0 and 1.
For example, "Hello" means "01001000 01100101 01101100 01101100 01101100 01101111". That's why we make compilations.
The assembler is the programming language closest to the machine language. On the other hand, scripting languages are considered high level because they are interpreted.

You must know that the assembly language is specific to each processor family. It will therefore learn another language for MIPS or ARM architectures.

4. How the assembler is translated into machine code? ?

When we have finished developing our program, whether in assembler or C, we must transform it into machine code so that it is understood and executed by our processor. This is where Assembler and Linker come in. Let's look at the picture below to better understand:


It can be seen that the compiler makes it possible to convert the code "C" into assembler language, then the assembler makes it possible to create an object file ".o", which contains all the information concerning the program. The linker when it allows you to create the executable by retrieving the information from the object files and adding all the necessary information such as libraries, relocation addresses ect.

5. Prerequisites to follow the tutorial.

Before proceeding in the tutorial, it is necessary to install various tools:

You will need nasm to compile assembly code and gdb to debug it, understand how it works.
You're gonna need the "binutils" package.

5. References

  1. Screenshot linkage:
  2. Book on how compilation works
  3. GDB Extension