Since we use k8s more and more at work and some of my small micro-services are using ConfigMaps I needed a simple way to reload the config on-the-fly without the need to restart the whole service. At the time of solving the problem there was nothing my Google-Fu skills could find about it so I thought I would share my solution with the world.
The way ConfigMaps work in k8s is that files are created inside the running container using a couple of symlinks. Example of running ls inside the container with the added configmap:
/etc/healthmonit # ls -al
total 32
drwxrwxrwx 3 root root 4096 Jul 11 13:24 .
drwxr-xr-x 1 root root 4096 Jul 11 12:52 ..
drwxr-xr-x 2 root root 4096 Jul 11 13:24 ..2018_07_11_11_24_39.108325508
lrwxrwxrwx 1 root root 31 Jul 11 13:24 ..data -> ..2018_07_11_11_24_39.108325508
lrwxrwxrwx 1 root root 23 Jul 11 12:52 healthmonit.conf -> ..data/healthmonit.conf
So what we need is to use the fsnotify library for Go and to create a watcher for our config-file. Since its actually a symlink we are watching its not Write events we need to react on but Remove. I personally did the mistake to only watch Writes, but later found out about the whole symlink part. In the snippet below I have simple working example that I personally use:
func configWatcher() {
watcher, err := fsnotify.NewWatcher()
if err != nil {
//handle err
}
defer watcher.Close()
err = watcher.Add(configfile)
if err != nil {
//handle err
}
for {
select {
case event := <-watcher.Events:
// k8s configmaps uses symlinks, we need this workaround.
// original configmap file is removed
if event.Op == fsnotify.Remove {
// remove the watcher since the file is removed
watcher.Remove(event.Name)
// add a new watcher pointing to the new symlink/file
watcher.Add(configfile)
reloadConfig()
}
// also allow normal files to be modified and reloaded.
if event.Op&fsnotify.Write == fsnotify.Write {
reloadConfig()
}
case err := <-watcher.Errors:
// handle error
}
}
}