Building a Swift executable without a build system
With the recent release of Swift support in CMake, I became interested in understanding a bit more about how Swift's build process actually works, and tasked myself with building some Swift projects the old fashioned way: step-by-step with just the compiler CLI, and without the benefit of a build system.
As I suspect a vast majority of all swift projects are built using either XCode or the Swift Package Manager, I found there to be something of a dearth of documentation and examples of how to use the swift compiler directly. As it did take some trial-and-error to demystify the process, I want to use this article to document and share my learnings.
Mental model
Before we actually get to building some code, it helps to have a mental model of what we're trying to do. The scope of this article is quite narrow, so the astute reader might notice that there are quite a few details left out for the sake of simplicity and clarity.
What exactly is "building"
Starting at a very high level, the problem we want to solve is how to convert our Swift source code, which can be read and written by humans, to machine code which can actually be run on hardware.
As with any compiled language, this build process with Swift consists of two steps: compilation and linking:
compilation takes source files (.swift) and converts them to object files (.o). When you compile a source file or a set of source files, the resulting object file is the machine code representation of that source, but it's not yet complete. For instance, if your source files depend on a system library, or another Swift module defined in other source files, the object file has to be linked to that other compiled code.
linking takes a group of object files and packages them together into an executable. An executable is a program which can actually run on your hardware.
In Swift, the basic unit of code for compilation and linking is the module. In the .swift world, a module corresponds to everything which comes with a single import statement: a set of type, variable and function definitions which exist within a common scope. Modules can talk to each other, but the smallest unit of Swift which can be compiled at one time is a complete module.
So, to sum that up, the basic problem which is solved in the following examples is this:
We have inputs in the form of .swift source files. We want to convert those inputs to output in the form of a native executable. This process is conducted in two steps:
First we compile the inputs to produce
.o(object files).Next we link the object files to produce our executable.
Simplest possible example
So now that the problem has been stated, let's try to actually do it. We'll start with the simplest possible example: to produce an executable from a single-file swift module.
So first let's create our source file: Hello.swift:
// Hello.swift
print("Hello world!")
Now the simplest way to build our source file is with a single command:
$ swiftc Hello.swift
This will invoke the swift compiler to build and link the contents of Hello.swift into an executable named Hello in the same directory. If we run it we get the expected output:
$ ./Hello
Hello world!
Now this is simple, but it's not very informative about what's actually going on. Let's dig a little deeper and do things step-by-step.
First let's generate our object file. We can do this by passing a -c flag to the compiler. This informs the compiler to do compilation only and to skip linking:
$ swiftc -c Hello.swift
This will generate the file Hello.o in the same directory where we run the command. So now to link our executable, we use the -emit-executable option, with the Hello.o we just generated as input:
$ swiftc -emit-executable Hello.o
This will produce the Hello executable, which again produces the correct output:
$ ./Hello
Hello world!
Multiple source files, one module
Now that we've successfully created an executable from a single source file, let's try something a tiny bit more complex: let's build an executable spread across multiple source files.
We can start by creating our sources. First let's create a file called World.swift:
// World.swift
let world: String = "world"
This file contains a single constant declaration which will be used in our other source, main.swift:
// main.swift
print("Hello \(world)")
It's worth noting that it's very important that our print statement is in a file called main.swift. The swift compiler treats the source file named "main.swift" differently than all other source files. main.swift is the designated entry point for swift executables. Similar to the int main() function in C programs, the contents of main.swift will be executed when we run the compiled program.
As a result, main.swift has some special properties. For one thing, this is the only place where we are allowed to put executable expressions at the top level. If we put that print statement in World.swift instead, we would be greeted by this error message on compilation:
error: expressions are not allowed at the top level
In the previous example, we were allowed to put a print statement in Hello.swift because in the special case that a single source file is compiled, it is treated as the entry-point automatically.
So to compile our sources, we can use the following command:
$ swiftc -c main.swift World.swift -module-name Hello
You'll notice this differs from the previous example in a couple of ways. First of all, after the -c flag we have both of our input sources listed. Any number of source files can be included in the build in this way.
Also we now have the option -module-name specified, with a value of Hello. This is telling the compiler that the sources we are building make up a module called "Hello". The module name is not very important to us at this stage, since our executable only has one module, but the compiler requires it to be specified anyway. We got away with not specifying it in the previous example, because when there's only one source file the module name is implicitly set to match the name of the source file. The module name will become important to us when we're working with multiple modules.
So that command will create two outputs: main.o and World.o - one object file per input source - and our directory will now look like this:
.
├── main.o
├── main.swift
├── World.o
└── World.swift
Finally we need to complete the linking step, which is almost exactly like the previous example:
$ swiftc -emit-executable main.o World.o -o Hello
Here, as with compilation, the main difference is that we're passing the two object files to the linker instead of one.
Also the option -o Hello is added here to specify that the executable will be output with the filename "Hello". This can be omitted, and the resulting executable would be called "main".
Finally, we can run our executable, and see the usual result:
$ ./Hello
Hello world!
Including multiple modules
To round out this basic tutorial, let's take a look at how we build an executable which is made up of two modules. In this case we will build the same executable, but instead of including main.swift and World.swift in the same module, each source file will get it's own module.
Before we get to the code, we have to build our intuition a bit around modules. To repeat: in swift a module is a self-contained set of type definitions, variables and functions with a shared scope. A module also represents a unit of compilation. This means in Swift, a module is always compiled with all the source files included in that module and only the source files included in that module.
Let's think about how that's going to affect our example, since we are moving main and World into separate modules. The main point this is going to affect is the world variable, since it's referenced in both World.swift and main.swift. Previously, we passed both source files to the compiler together, so when the compiler saw world accessed in main.swift, it already knew about the declaration in World.swift. Now that we will be compiling main.swift and World.swift in separate compilation steps, we need another way of telling the compiler about the existence of world when we're compiling main.swift.
That's where .swiftmodule files come in. A .swiftmodule file is like a map of a module which tells the compiler where to find things in an object file. This will add an additional step we haven't seen before when bilding World.swift
So first we need to make some changes to our source files. Let's edit World.swift to look like this:
// World.swift
public let world: String = "world"
Here the public access modifier has been added to world to make it visible to our main module. In Swift everything has an internal access level by default, meaning it can be freely accessed from within the same module, but public must be added to specifically white-list a variable, type or function to be accessible from other modules.
Next we modify main.swift to look like this:
// main.swift
import World
print("Hello \(world)")
So here the line import World has been added to tell the compiler that this source file depends on the module named "World". This is where the -module-name argument we saw in the previous example becomes important.
Now that our source files are ready, the fist step is to build the World module. The process will be a bit different than before, because rather than building World as an executable, we are building it as a library.
So we can start by compiling the object file:
$ swiftc -c World.swift -parse-as-library
Here we have a new argument: -parse-as-library. This option is required because the compiler treats libraries and executables differently. For instance, without this argument, the compiler might strip out the variable world from World.o, since that variable is never accessed within the World module.
Next we have to produce our .swiftmodule file. This will be required when building main.swift, so the compiler knows the contents of the World module when it is imported. We can create it with the following command:
$ swiftc World.swift -emit-module
You'll notice that this command produces two files: World.swiftdoc is a generated documentation file for this module, which can be ignored for our purposes. World.swiftmodule is required for the next step.
So now that we have our object file and our swift module file, the next step is to build our executable. As before the first step is to produce an object file for main.swift:
$ swiftc -c main.swift -I.
Here we have a new argument: -I.. The option -I adds an "include path" where the compiler will look for module definitions. In this case, we pass it "." to tell it to look in the current working directory. If we put our .swiftmodule flags somewhere else, we could pass it a path to that location, i.e: -I path/to/some/build/directory.
The -I option is important, because without it, the compiler doesn't know anything about the World module, and we would get this error:
$ swiftc -c main.swift
main.swift:1:8: error: no such module 'World'
import World
^
So now that we have compiled our main.o object file, the final step is to link our object files into an executable. This can be achieved in exactly the same way as with the previous example:
$ swiftc -emit-executable main.o World.o -o Hello
And if we run our executable, we get the same output:
$ ./Hello
Hello world!
The rest of the owl
So obviously this is only a very small subset of what's possible with the Swift compiler. My goal with this post has been to compose a few minimal examples of the necessary-and-sufficient steps required to build and compose Swift code with the swiftc cli only. If there is interest I may post follow-ups on more advanced topics.
All the examples outlined here are available on github.