ccache & distcc w/ clang

This is a summary of several related blogs. Since we plan to move back from Swift to Objective-C for our project, I did some research on cache today and grabbed some blogs.

ccache & clang

ccache basically sits on top of clang and checks if a compilation is needed against its cache, and only call into the compiler when a real compilation is granted.

How to install?

brew install ccache (didn’t work for me because of some weird xls issues. I switched to port install cache and that worked.

How to use?

Replace clang -> ccache clang. In Xcode, add flag

CC=ccache /usr/bin/clang

to the project settings. To be safer, check for the existence of ccache first:

#!/bin/sh
if type -p ccache >/dev/null 2>&1; then
  export CCACHE_MAXSIZE=10G
  export CCACHE_CPP2=true
  export CCACHE_HARDLINK=true
  export CCACHE_SLOPPINESS=file_macro,time_macros,include_file_mtime,include_file_ctime,file_stat_matches
  exec ccache /usr/bin/clang "$@"
else
  exec clang "$@"
fi

Configurable flags: https://ccache.samba.org/manual.html#_configuration

Caveats?

Clang modules are not supported. Swift is definitely not supported.

How to validate?

ccache -s

How to configure ccache?

~/.ccache/ccache.conf

ccache and clang

ccache and clang, part 2

ccache and clang, part 3

Using ccache for Fun and Profit | Inside PSPDFKit

Distcc

Now we are able to build locally fast, how are we able to leverage cloud to make build even faster? For example, is that possible to distribute the compiling process to a pool of machines and only perform the linking locally? Here we have distcc.

How to install?

brew install distcc

How to set it up?

1. Create a file ~/.distcc/hosts and put the addresses of the builder servers there.
2. To start a distcc daemon with the following command:

distccd --no-detach --daemon --allow 192.168.0.0/16 --allow 127.0.0.1 --log-stderr --verbose

3. In client side:

export DISTCC_FALLBACK=0
distcc -c test.c

Daemon response:

distccd[97259] (dcc_check_client) connection from 127.0.0.1:62993
distccd[97259] (check_address_inet) match client 0x100007f, value 0x100007f, mask 0xffffffff
distccd[97259] (dcc_r_token_int) got DIST00000001
distccd[97259] (dcc_r_token_int) got ARGC00000005
distccd[97259] (dcc_r_argv) reading 5 arguments from job submission
distccd[97259] (dcc_r_token_int) got ARGV00000002
distccd[97259] (dcc_r_token_string) got 'cc'
distccd[97259] (dcc_r_argv) argv[0] = "cc"
distccd[97259] (dcc_r_token_int) got ARGV00000002
distccd[97259] (dcc_r_token_string) got '-c'
distccd[97259] (dcc_r_argv) argv[1] = "-c"
distccd[97259] (dcc_r_token_int) got ARGV00000006
distccd[97259] (dcc_r_token_string) got 'test.c'
distccd[97259] (dcc_r_argv) argv[2] = "test.c"
distccd[97259] (dcc_r_token_int) got ARGV00000002
distccd[97259] (dcc_r_token_string) got '-o'
distccd[97259] (dcc_r_argv) argv[3] = "-o"
distccd[97259] (dcc_r_token_int) got ARGV00000006
distccd[97259] (dcc_r_token_string) got 'test.o'
distccd[97259] (dcc_r_argv) argv[4] = "test.o"
distccd[97259] (dcc_r_argv) got arguments: cc -c test.c -o test.o
distccd[97259] (dcc_scan_args) scanning arguments: cc -c test.c -o test.o
distccd[97259] (dcc_scan_args) found input file "test.c"
distccd[97259] (dcc_scan_args) found object/output file "test.o"
distccd[97259] compile from test.c to test.o
distccd[97259] (dcc_run_job) output file test.o
distccd[97259] (dcc_input_tmpnam) input file test.c
distccd[97259] (dcc_r_token_int) got DOTI000036fc
distccd[97259] (dcc_r_file) received 14076 bytes to file /var/folders/_g/r6syzgbx6ylctp7m45l_shlw000b3h/T//distccd_2ea9329e.i
distccd[97259] (dcc_r_file_timed) 14076 bytes received in 0.000176s, rate 78103kB/s
distccd[97259] (dcc_set_input) changed input from "test.c" to "/var/folders/_g/r6syzgbx6ylctp7m45l_shlw000b3h/T//distccd_2ea9329e.i"
distccd[97259] (dcc_set_input) command after: cc -c /var/folders/_g/r6syzgbx6ylctp7m45l_shlw000b3h/T//distccd_2ea9329e.i -o test.o
distccd[97259] (dcc_set_output) changed output from "test.o" to "/var/folders/_g/r6syzgbx6ylctp7m45l_shlw000b3h/T//distccd_2e47329e.o"
distccd[97259] (dcc_set_output) command after: cc -c /var/folders/_g/r6syzgbx6ylctp7m45l_shlw000b3h/T//distccd_2ea9329e.i -o /var/folders/_g/r6syzgbx6ylctp7m45l_shlw000b3h/T//distccd_2e47329e.o
distccd[97259] (dcc_check_compiler_masq) /usr/bin/cc is a safe symlink to clang
distccd[97259] (dcc_spawn_child) forking to execute: cc -c /var/folders/_g/r6syzgbx6ylctp7m45l_shlw000b3h/T//distccd_2ea9329e.i -o /var/folders/_g/r6syzgbx6ylctp7m45l_shlw000b3h/T//distccd_2e47329e.o
distccd[97259] (dcc_spawn_child) child started as pid97406
distccd[97406] (dcc_new_pgrp) entered process group
distccd[97406] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distccd[97259] (dcc_collect_child) cc child 97406 terminated with status 0
distccd[97259] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[97259] (dcc_x_token_int) send DONE00000001
distccd[97259] (dcc_x_token_int) send STAT00000000
distccd[97259] (dcc_x_file) send 0 byte file /var/folders/_g/r6syzgbx6ylctp7m45l_shlw000b3h/T//distcc_2b99329e.stderr with token SERR and compression 69
distccd[97259] (dcc_x_token_int) send SERR00000000
distccd[97259] (dcc_x_file) send 0 byte file /var/folders/_g/r6syzgbx6ylctp7m45l_shlw000b3h/T//distcc_2bc6329e.stdout with token SOUT and compression 69
distccd[97259] (dcc_x_token_int) send SOUT00000000
distccd[97259] (dcc_x_file) send 752 byte file /var/folders/_g/r6syzgbx6ylctp7m45l_shlw000b3h/T//distccd_2e47329e.o with token DOTO and compression 69
distccd[97259] (dcc_x_token_int) send DOTO000002f0
distccd[97259] (dcc_pump_sendfile) decided to use read/write rather than sendfile
distccd[97259] cc test.c on localhost completed ok
distccd[97259] job complete
distccd[97259] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[97259] (dcc_job_summary) client: 127.0.0.1:62993 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:51ms cc test.c

To integrate with previous Xcode script for ccache, use this script instead:

# Does the hosts file exist and is distcc in the path?
if test -f ~/.distcc/hosts && type -p distcc >/dev/null 2>&1
then
    # Tell ccache to prefix calls to the compiler with 'distcc'
    export CCACHE_PREFIX="distcc"
fi

Make Xcode more powerful now:

defaults write com.apple.dt.Xcode IDEBuildOperationMaxNumberOfConcurrentCompileTasks 24

Crazy Fast Builds Using distcc | Inside PSPDFKit