text:
LXD assigns wrong IP to container

The template below is mostly useful for bug reports and support questions.
Feel free to remove anything which doesn't apply to you and add more information where it makes sense.
Required information

Distribution: Ubuntu
Distribution version: Xenial
The output of "lxc info" or if that fails:

config:
  core.https_address: '[::]:8443'
  core.trust_password: true
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
api_status: stable
api_version: "1.0"
auth: trusted
public: false
environment:
  addresses:
  - 192.168.100.2:8443
  - 10.0.3.1:8443
  - 10.0.4.1:8443
  architectures:
  - x86_64
  - i686
  certificate: |
    -----BEGIN CERTIFICATE-----

    -----END CERTIFICATE-----
  certificate_fingerprint: df7f2254e113336dba1df531f202e7f4948da7b8044212cf8b5326330310456d
  driver: lxc
  driver_version: 2.0.8
  kernel: Linux
  kernel_architecture: x86_64
  kernel_version: 4.4.0-92-generic
  server: lxd
  server_pid: 1765
  server_version: "2.16"
  storage: btrfs
  storage_version: "4.4"

Issue description
When creating multiple containers with static dhcp entry it fails to assign the correct IP for one container
Steps to reproduce
for i in {5..10}; do lxc init ubuntu:16.04 cont$i; lxc config device add cont$i eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.$i;lxc start cont$i; done

lxc list
+--------------------+---------+-------------------+------+------------+-----------+
|        NAME        |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------------------+---------+-------------------+------+------------+-----------+
| cont10             | RUNNING | 10.0.4.146 (eth0) |      | PERSISTENT | 0         |
+--------------------+---------+-------------------+------+------------+-----------+
| cont5              | RUNNING | 10.0.4.5 (eth0)   |      | PERSISTENT | 0         |
+--------------------+---------+-------------------+------+------------+-----------+
| cont6              | RUNNING | 10.0.4.6 (eth0)   |      | PERSISTENT | 0         |
+--------------------+---------+-------------------+------+------------+-----------+
| cont7              | RUNNING | 10.0.4.7 (eth0)   |      | PERSISTENT | 0         |
+--------------------+---------+-------------------+------+------------+-----------+
| cont8              | RUNNING | 10.0.4.8 (eth0)   |      | PERSISTENT | 0         |
+--------------------+---------+-------------------+------+------------+-----------+
| cont9              | RUNNING | 10.0.4.9 (eth0)   |      | PERSISTENT | 0         |
+--------------------+---------+-------------------+------+------------+-----------+

lxc config show cont10
architecture: x86_64
config:
  image.architecture: amd64
  image.description: ubuntu 16.04 LTS amd64 (release) (20170815.1)
  image.label: release
  image.os: ubuntu
  image.release: xenial
  image.serial: "20170815.1"
  image.version: "16.04"
  volatile.base_image: 58f90cbf68927c3fc43e6ee1386446a04f3d8068c1a75a291339cb2be01dec08
  volatile.eth0.hwaddr: 00:16:3e:67:43:5e
  volatile.eth0.name: eth0
  volatile.idmap.base: "0"
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":165536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":165536,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":165536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":165536,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    ipv4.address: 10.0.4.10
    nictype: bridged
    parent: lxdbr0
    type: nic
ephemeral: false
profiles:
- default
stateful: false
description: ""

lxc network show lxdbr0
config:
  dns.mode: dynamic
  ipv4.address: 10.0.4.1/24
  ipv4.dhcp.ranges: 10.0.4.100-10.0.4.254
  ipv4.nat: "true"
  ipv6.address: none
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/containers/cont10
- /1.0/containers/cont5
- /1.0/containers/cont6
- /1.0/containers/cont7
- /1.0/containers/cont8
- /1.0/containers/cont9
managed: true

cat /storage/lxd/networks/lxdbr0/dnsmasq.hosts
00:16:3e:52:a0:07,id:*,10.0.4.5
00:16:3e:2b:79:c0,id:*,10.0.4.6
00:16:3e:20:64:cb,id:*,10.0.4.7
00:16:3e:4c:24:f0,id:*,10.0.4.8
00:16:3e:fe:28:c5,id:*,10.0.4.9

Does restarting it solve the problem?

Unfortunately not:
lxc stop cont10
lxc start cont10
lxc list
+--------------------+---------+-------------------+------+------------+-----------+
|        NAME        |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------------------+---------+-------------------+------+------------+-----------+
| cont10             | RUNNING | 10.0.4.175 (eth0) |      | PERSISTENT | 0         |
+--------------------+---------+-------------------+------+------------+-----------+

Can you tail /var/log/syslog  while restarting the container to see what dnsmasq does?

Sure:
Aug 20 19:52:53 prod-server01 kernel: [ 4143.943324] audit: type=1400 audit(1503251573.801:570): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-cont10_</var/lib/lxd>" pid=12973 comm="apparmor_parser"
Aug 20 19:52:54 prod-server01 kernel: [ 4144.232091] device veth55HJFA entered promiscuous mode
Aug 20 19:52:54 prod-server01 kernel: [ 4144.232160] IPv6: ADDRCONF(NETDEV_UP): veth55HJFA: link is not ready
Aug 20 19:52:54 prod-server01 kernel: [ 4144.287090] eth0: renamed from vethGTY888
Aug 20 19:52:54 prod-server01 kernel: [ 4144.304678] IPv6: ADDRCONF(NETDEV_CHANGE): veth55HJFA: link becomes ready
Aug 20 19:52:54 prod-server01 kernel: [ 4144.304798] lxdbr0: port 8(veth55HJFA) entered forwarding state
Aug 20 19:52:54 prod-server01 kernel: [ 4144.304828] lxdbr0: port 8(veth55HJFA) entered forwarding state
Aug 20 19:52:55 prod-server01 kernel: [ 4145.489285] audit: type=1400 audit(1503251575.345:571): apparmor="STATUS" operation="profile_load" label="lxd-cont10_</var/lib/lxd>//&:lxd-cont10_<var-lib-lxd>://unconfined" name="/sbin/dhclient" pid=13215 comm="apparmor_parser"
Aug 20 19:52:55 prod-server01 kernel: [ 4145.489307] audit: type=1400 audit(1503251575.345:572): apparmor="STATUS" operation="profile_load" label="lxd-cont10_</var/lib/lxd>//&:lxd-cont10_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=13215 comm="apparmor_parser"
Aug 20 19:52:55 prod-server01 kernel: [ 4145.489321] audit: type=1400 audit(1503251575.345:573): apparmor="STATUS" operation="profile_load" label="lxd-cont10_</var/lib/lxd>//&:lxd-cont10_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=13215 comm="apparmor_parser"
Aug 20 19:52:55 prod-server01 kernel: [ 4145.489335] audit: type=1400 audit(1503251575.345:574): apparmor="STATUS" operation="profile_load" label="lxd-cont10_</var/lib/lxd>//&:lxd-cont10_<var-lib-lxd>://unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=13215 comm="apparmor_parser"
Aug 20 19:52:55 prod-server01 kernel: [ 4145.492378] audit: type=1400 audit(1503251575.349:575): apparmor="STATUS" operation="profile_load" label="lxd-cont10_</var/lib/lxd>//&:lxd-cont10_<var-lib-lxd>://unconfined" name="/usr/bin/lxc-start" pid=13216 comm="apparmor_parser"
Aug 20 19:52:55 prod-server01 kernel: [ 4145.493579] audit: type=1400 audit(1503251575.349:576): apparmor="STATUS" operation="profile_load" label="lxd-cont10_</var/lib/lxd>//&:lxd-cont10_<var-lib-lxd>://unconfined" name="lxc-container-default" pid=13214 comm="apparmor_parser"
Aug 20 19:52:55 prod-server01 kernel: [ 4145.493604] audit: type=1400 audit(1503251575.349:577): apparmor="STATUS" operation="profile_load" label="lxd-cont10_</var/lib/lxd>//&:lxd-cont10_<var-lib-lxd>://unconfined" name="lxc-container-default-cgns" pid=13214 comm="apparmor_parser"
Aug 20 19:52:55 prod-server01 kernel: [ 4145.493621] audit: type=1400 audit(1503251575.349:578): apparmor="STATUS" operation="profile_load" label="lxd-cont10_</var/lib/lxd>//&:lxd-cont10_<var-lib-lxd>://unconfined" name="lxc-container-default-with-mounting" pid=13214 comm="apparmor_parser"
Aug 20 19:52:55 prod-server01 kernel: [ 4145.493637] audit: type=1400 audit(1503251575.349:579): apparmor="STATUS" operation="profile_load" label="lxd-cont10_</var/lib/lxd>//&:lxd-cont10_<var-lib-lxd>://unconfined" name="lxc-container-default-with-nesting" pid=13214 comm="apparmor_parser"
Aug 20 19:52:55 prod-server01 avahi-daemon[1475]: Joining mDNS multicast group on interface veth55HJFA.IPv6 with address fe80::fc54:c0ff:fed4:5728.
Aug 20 19:52:55 prod-server01 avahi-daemon[1475]: New relevant interface veth55HJFA.IPv6 for mDNS.
Aug 20 19:52:55 prod-server01 avahi-daemon[1475]: Registering new address record for fe80::fc54:c0ff:fed4:5728 on veth55HJFA.*.
Aug 20 19:52:55 prod-server01 dnsmasq-dhcp[6503]: DHCPDISCOVER(lxdbr0) 10.0.4.175 00:16:3e:50:df:55
Aug 20 19:52:55 prod-server01 dnsmasq-dhcp[6503]: DHCPOFFER(lxdbr0) 10.0.4.175 00:16:3e:50:df:55
Aug 20 19:52:55 prod-server01 dnsmasq-dhcp[6503]: DHCPREQUEST(lxdbr0) 10.0.4.175 00:16:3e:50:df:55
Aug 20 19:52:55 prod-server01 dnsmasq-dhcp[6503]: DHCPACK(lxdbr0) 10.0.4.175 00:16:3e:50:df:55 cont10
Aug 20 19:53:09 prod-server01 kernel: [ 4159.314628] lxdbr0: port 8(veth55HJFA) entered forwarding state

echo "00:16:3e:50:df:55,id:*,10.0.4.10" >> /storage/lxd/networks/lxdbr0/dnsmasq.hosts
lxc start cont10
lxc list
+--------------------+---------+-------------------+------+------------+-----------+
|        NAME        |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------------------+---------+-------------------+------+------------+-----------+
| cont10             | RUNNING | 10.0.4.175 (eth0) |      | PERSISTENT | 0         |
+--------------------+---------+-------------------+------+------------+-----------+

doesn´t work either. Weird.

I´ve find also this lines in the logs:
Aug 20 20:32:46 prod-server01 kernel: [ 6536.902937] audit: type=1400 audit(1503253966.644:586): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont10_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=18922 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Aug 20 20:32:46 prod-server01 kernel: [ 6536.902959] audit: type=1400 audit(1503253966.644:587): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont10_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=18922 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536

Hi @stgraber ,
can you confirm this behavior? Can you explain the issue and is there a workaround till you can fix it in 2.17? Many thanks
Cheers

I've not had time to look into this yet. I'll take a brief look before tagging 2.17.

Take your time, thank you.

First attempt to reproduce on current LXD didn't work (they all got their expected IPs).

So I managed to reproduce a case where the container didn't get its expected address, but that one looked like a dnsmasq bug as the hosts file contained the expected MAC and a restart of the container got it to release its current address and get the right one from dnsmasq.

Ran your reproducer 10 times (so 50 containers total). Out of those 10 runs, I had 3 cases where one of the containers didn't get the expected IP.
In all 3 cases, dnsmasq.hosts was correct, syslog showed dnsmasq was properly respawned after dnsmasq.hosts got updated, the request was coming from the correct MAC, yet dnsmasq returned the wrong IP. In all cases, bouncing the dhcp client in the container fixed the issue (ifdown eth0 && ifup eth0), this would indicate a dnsmasq bug of some sort.
I've never had a case where a container had a valid mac address in volatile, also had a valid ipv4 address in ipv4.address and didn't get the expected entry in dnsmasq.hosts.

Moving this issue back to un-milestoned and incomplete as I can't reproduce the case described in this issue.
Can you reproduce this behavior somewhat reliably on your side?

For the occasional failure I'm seeing, I think it can be blamed on dnsmasq being a bit slow at reloading its configuration. Sadly the only interface we have for that is SIGHUP, so that's not telling us when it's actually done processing it...

I wonder if using --dhcp-hostsdir would be more reliable.

Hi @stgraber , thank you for looking into this.
ok, so here is what I did:

deleted the container which got the wrong IP (contX)
re-created the container with:

lxc init ubuntu:16.04 contX
cat /storage/lxd/networks/lxdbr0/dnsmasq.hosts
00:16:3e:7a:e8:37,id:*,10.0.4.3
00:16:3e:03:44:e1,id:*,10.0.4.2

10.0.4.3 and 10.0.4.2 are production container
lxc config device add contX eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.4
lxc start contX && lxc list contX
+--------------------+---------+-------------------+------+------------+-----------+
|        NAME        |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------------------+---------+-------------------+------+------------+-----------+
| contX              | RUNNING | 10.0.4.46 (eth0)  |      | PERSISTENT | 0         |
+--------------------+---------+-------------------+------+------------+-----------+

The dnsmasq.host file doesn´t get updated for contX:
 cat /storage/lxd/networks/lxdbr0/dnsmasq.hosts
00:16:3e:7a:e8:37,id:*,10.0.4.3
00:16:3e:03:44:e1,id:*,10.0.4.2

Trying to reload the interface:
lxc exec contX -- ifdown eth0 && ifup eth0
Killed old client process
Internet Systems Consortium DHCP Client 4.3.3
Copyright 2004-2015 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/eth0/00:16:3e:e4:b7:06
Sending on   LPF/eth0/00:16:3e:e4:b7:06
Sending on   Socket/fallback
DHCPRELEASE on eth0 to 10.0.4.1 port 67 (xid=0x4e8dbb57)
Unknown interface eth0

Unknown interface eth0 is kind of weird
lxc exec contX -- cat /etc/network/interfaces.d/50-cloud-init.cfg
# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

lxc list contX
+--------------------+---------+-------------------+------+------------+-----------+
|        NAME        |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------------------+---------+-------------------+------+------------+-----------+
| contX              | RUNNING |                   |      | PERSISTENT | 0         |
+--------------------+---------+-------------------+------+------------+-----------+

Now, the container didn´t even get an IP address.
After a while I can see the MAC of contX in dnsmasq.hosts
cat /storage/lxd/networks/lxdbr0/dnsmasq.hosts
00:16:3e:e4:b7:06,id:*,10.0.4.4
00:16:3e:7a:e8:37,id:*,10.0.4.3
00:16:3e:03:44:e1,id:*,10.0.4.2

but the container doesn´t get the IP:
lxc list contX
+-------+---------+------+------+------------+-----------+
| NAME  |  STATE  | IPV4 | IPV6 |    TYPE    | SNAPSHOTS |
+-------+---------+------+------+------------+-----------+
| contX | RUNNING |      |      | PERSISTENT | 0         |
+-------+---------+------+------+------------+-----------+

restarting the container and voila:
lxc restart contX && lxc list contX
+-------+---------+-----------------+------+------------+-----------+
| NAME  |  STATE  |      IPV4       | IPV6 |    TYPE    | SNAPSHOTS |
+-------+---------+-----------------+------+------------+-----------+
| contX | RUNNING | 10.0.4.4 (eth0) |      | PERSISTENT | 0         |
+-------+---------+-----------------+------+------------+-----------+

If tried creating a different container/IP which never was previously used:
lxc config device add foobar eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.55
Device eth0 added to foobar
lxc start foobar
lxc list foobar
+--------+---------+-------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+------+------------+-----------+
| foobar | RUNNING | 10.0.4.188 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+

lxc exec foobar -- ifdown eth0 && ifup eth0
Killed old client process
Internet Systems Consortium DHCP Client 4.3.3
Copyright 2004-2015 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/eth0/00:16:3e:b9:89:cd
Sending on   LPF/eth0/00:16:3e:b9:89:cd
Sending on   Socket/fallback
DHCPRELEASE on eth0 to 10.0.4.1 port 67 (xid=0x3d28e70e)
Unknown interface eth0

lxc list foobar
+--------+---------+------+------+------------+-----------+
|  NAME  |  STATE  | IPV4 | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+------+------+------------+-----------+
| foobar | RUNNING |      |      | PERSISTENT | 0         |
+--------+---------+------+------+------------+-----------+

restarting the container:
lxc restart foobar && lxc list foobar
+--------+---------+-------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+------+------------+-----------+
| foobar | RUNNING | 10.0.4.188 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+

cat /storage/lxd/networks/lxdbr0/dnsmasq.hosts
00:16:3e:e4:b7:06,id:*,10.0.4.4
00:16:3e:7a:e8:37,id:*,10.0.4.3
00:16:3e:03:44:e1,id:*,10.0.4.2

lxc info foobar
Name: foobar
Remote: unix://
Architecture: x86_64
Created: 2017/08/23 07:18 UTC
Status: Running
Type: persistent
Profiles: default
Pid: 30540
Ips:
  eth0:	inet	10.0.4.188	veth0M6KO8
  eth0:	inet6	fe80::216:3eff:feb9:89cd	veth0M6KO8
  lo:	inet	127.0.0.1
  lo:	inet6	::1
Resources:
  Processes: 26
  CPU usage:
    CPU usage (in seconds): 4
  Memory usage:
    Memory (current): 22.78MB
    Memory (peak): 45.12MB
  Network usage:
    lo:
      Bytes received: 0B
      Bytes sent: 0B
      Packets received: 0
      Packets sent: 0
    eth0:
      Bytes received: 4.02kB
      Bytes sent: 1.45kB
      Packets received: 24
      Packets sent: 13

lxc config show foobar
architecture: x86_64
config:
  image.architecture: amd64
  image.description: ubuntu 16.04 LTS amd64 (release) (20170815.1)
  image.label: release
  image.os: ubuntu
  image.release: xenial
  image.serial: "20170815.1"
  image.version: "16.04"
  volatile.base_image: 58f90cbf68927c3fc43e6ee1386446a04f3d8068c1a75a291339cb2be01dec08
  volatile.eth0.hwaddr: 00:16:3e:b9:89:cd
  volatile.eth0.name: eth0
  volatile.idmap.base: "0"
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":165536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":165536,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":165536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":165536,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    ipv4.address: 10.0.4.55
    nictype: bridged
    parent: lxdbr0
    type: nic
ephemeral: false
profiles:
- default
stateful: false
description: ""

and here are the logs:
cat /var/log/syslog | grep foobar

Aug 23 09:19:15 prod-server01 kernel: [225336.811147] audit: type=1400 audit(1503472755.565:1137): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-foobar_</var/lib/lxd>" pid=25324 comm="apparmor_parser"
Aug 23 09:19:20 prod-server01 kernel: [225341.710293] audit: type=1400 audit(1503472760.461:1138): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/sbin/dhclient" pid=25557 comm="apparmor_parser"
Aug 23 09:19:20 prod-server01 kernel: [225341.718996] audit: type=1400 audit(1503472760.473:1139): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=25557 comm="apparmor_parser"
Aug 23 09:19:20 prod-server01 kernel: [225341.719564] audit: type=1400 audit(1503472760.473:1140): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=25557 comm="apparmor_parser"
Aug 23 09:19:20 prod-server01 kernel: [225341.720056] audit: type=1400 audit(1503472760.473:1141): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=25557 comm="apparmor_parser"
Aug 23 09:19:20 prod-server01 kernel: [225341.753717] audit: type=1400 audit(1503472760.505:1142): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/bin/lxc-start" pid=25578 comm="apparmor_parser"
Aug 23 09:19:20 prod-server01 kernel: [225341.977461] audit: type=1400 audit(1503472760.729:1143): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/lxd/lxd-bridge-proxy" pid=25580 comm="apparmor_parser"
Aug 23 09:19:20 prod-server01 kernel: [225342.093111] audit: type=1400 audit(1503472760.845:1144): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default" pid=25558 comm="apparmor_parser"
Aug 23 09:19:20 prod-server01 kernel: [225342.093861] audit: type=1400 audit(1503472760.845:1145): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default-cgns" pid=25558 comm="apparmor_parser"
Aug 23 09:19:20 prod-server01 kernel: [225342.094752] audit: type=1400 audit(1503472760.849:1146): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default-with-mounting" pid=25558 comm="apparmor_parser"
Aug 23 09:19:26 prod-server01 dnsmasq-dhcp[18002]: DHCPACK(lxdbr0) 10.0.4.188 00:16:3e:b9:89:cd foobar
Aug 23 09:25:53 prod-server01 kernel: [225734.543443] audit: type=1400 audit(1503473153.273:1170): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="lxd-foobar_</var/lib/lxd>" pid=28273 comm="apparmor_parser"
Aug 23 09:25:54 prod-server01 kernel: [225735.411987] audit: type=1400 audit(1503473154.145:1171): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-foobar_</var/lib/lxd>" pid=28353 comm="apparmor_parser"
Aug 23 09:25:55 prod-server01 kernel: [225736.613614] audit: type=1400 audit(1503473155.345:1172): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/sbin/dhclient" pid=28589 comm="apparmor_parser"
Aug 23 09:25:55 prod-server01 kernel: [225736.613639] audit: type=1400 audit(1503473155.345:1173): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=28589 comm="apparmor_parser"
Aug 23 09:25:55 prod-server01 kernel: [225736.613656] audit: type=1400 audit(1503473155.345:1174): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=28589 comm="apparmor_parser"
Aug 23 09:25:55 prod-server01 kernel: [225736.613674] audit: type=1400 audit(1503473155.345:1175): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=28589 comm="apparmor_parser"
Aug 23 09:25:55 prod-server01 kernel: [225736.618379] audit: type=1400 audit(1503473155.349:1176): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default" pid=28588 comm="apparmor_parser"
Aug 23 09:25:55 prod-server01 kernel: [225736.618397] audit: type=1400 audit(1503473155.349:1177): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default-cgns" pid=28588 comm="apparmor_parser"
Aug 23 09:25:55 prod-server01 kernel: [225736.618407] audit: type=1400 audit(1503473155.349:1178): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default-with-mounting" pid=28588 comm="apparmor_parser"
Aug 23 09:25:55 prod-server01 kernel: [225736.618417] audit: type=1400 audit(1503473155.349:1179): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default-with-nesting" pid=28588 comm="apparmor_parser"
Aug 23 09:25:55 prod-server01 dnsmasq-dhcp[18002]: DHCPACK(lxdbr0) 10.0.4.188 00:16:3e:b9:89:cd foobar
Aug 23 09:26:15 prod-server01 kernel: [225756.383191] audit: type=1400 audit(1503473175.112:1187): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-foobar_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=29207 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Aug 23 09:26:15 prod-server01 kernel: [225756.383211] audit: type=1400 audit(1503473175.112:1188): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-foobar_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=29207 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Aug 23 09:26:17 prod-server01 kernel: [225759.208483] audit: type=1400 audit(1503473177.936:1189): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="lxd-foobar_</var/lib/lxd>" pid=29365 comm="apparmor_parser"
Aug 23 09:26:18 prod-server01 kernel: [225760.073462] audit: type=1400 audit(1503473178.804:1190): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-foobar_</var/lib/lxd>" pid=29453 comm="apparmor_parser"
Aug 23 09:26:19 prod-server01 kernel: [225761.119536] audit: type=1400 audit(1503473179.848:1191): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/sbin/dhclient" pid=29696 comm="apparmor_parser"
Aug 23 09:26:19 prod-server01 kernel: [225761.119555] audit: type=1400 audit(1503473179.848:1192): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=29696 comm="apparmor_parser"
Aug 23 09:26:19 prod-server01 kernel: [225761.119565] audit: type=1400 audit(1503473179.848:1193): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=29696 comm="apparmor_parser"
Aug 23 09:26:19 prod-server01 kernel: [225761.119576] audit: type=1400 audit(1503473179.848:1194): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=29696 comm="apparmor_parser"
Aug 23 09:26:19 prod-server01 kernel: [225761.126097] audit: type=1400 audit(1503473179.856:1195): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default" pid=29695 comm="apparmor_parser"
Aug 23 09:26:19 prod-server01 kernel: [225761.126116] audit: type=1400 audit(1503473179.856:1196): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default-cgns" pid=29695 comm="apparmor_parser"
Aug 23 09:26:20 prod-server01 kernel: [225761.797829] audit: type=1400 audit(1503473180.528:1204): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-foobar_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=29785 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Aug 23 09:26:20 prod-server01 kernel: [225761.797853] audit: type=1400 audit(1503473180.528:1205): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-foobar_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=29785 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Aug 23 09:26:20 prod-server01 dnsmasq-dhcp[18002]: DHCPACK(lxdbr0) 10.0.4.188 00:16:3e:b9:89:cd foobar
Aug 23 09:26:37 prod-server01 kernel: [225778.370293] audit: type=1400 audit(1503473197.096:1206): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-foobar_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=30261 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Aug 23 09:26:37 prod-server01 kernel: [225778.370355] audit: type=1400 audit(1503473197.100:1207): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-foobar_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=30261 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Aug 23 09:26:39 prod-server01 kernel: [225780.632356] audit: type=1400 audit(1503473199.360:1208): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="lxd-foobar_</var/lib/lxd>" pid=30430 comm="apparmor_parser"
Aug 23 09:26:45 prod-server01 kernel: [225787.197833] audit: type=1400 audit(1503473205.923:1209): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-foobar_</var/lib/lxd>" pid=30533 comm="apparmor_parser"
Aug 23 09:26:47 prod-server01 kernel: [225788.301576] audit: type=1400 audit(1503473207.027:1210): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/sbin/dhclient" pid=30777 comm="apparmor_parser"
Aug 23 09:26:47 prod-server01 kernel: [225788.301594] audit: type=1400 audit(1503473207.027:1211): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=30777 comm="apparmor_parser"
Aug 23 09:26:47 prod-server01 kernel: [225788.301605] audit: type=1400 audit(1503473207.027:1212): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=30777 comm="apparmor_parser"
Aug 23 09:26:47 prod-server01 kernel: [225788.301615] audit: type=1400 audit(1503473207.027:1213): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=30777 comm="apparmor_parser"
Aug 23 09:26:47 prod-server01 kernel: [225788.307859] audit: type=1400 audit(1503473207.035:1214): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default" pid=30776 comm="apparmor_parser"
Aug 23 09:26:47 prod-server01 kernel: [225788.307876] audit: type=1400 audit(1503473207.035:1215): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default-cgns" pid=30776 comm="apparmor_parser"
Aug 23 09:26:47 prod-server01 kernel: [225788.307887] audit: type=1400 audit(1503473207.035:1216): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default-with-mounting" pid=30776 comm="apparmor_parser"
Aug 23 09:26:47 prod-server01 kernel: [225788.307896] audit: type=1400 audit(1503473207.035:1217): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="lxc-container-default-with-nesting" pid=30776 comm="apparmor_parser"
Aug 23 09:26:47 prod-server01 kernel: [225788.309619] audit: type=1400 audit(1503473207.035:1218): apparmor="STATUS" operation="profile_load" label="lxd-foobar_</var/lib/lxd>//&:lxd-foobar_<var-lib-lxd>://unconfined" name="/usr/bin/lxc-start" pid=30778 comm="apparmor_parser"
Aug 23 09:26:47 prod-server01 dnsmasq-dhcp[18002]: DHCPACK(lxdbr0) 10.0.4.188 00:16:3e:b9:89:cd foobar

Another try:
lxc init ubuntu:16.04 bar && lxc init ubuntu:16.04 foo
lxc config device add bar eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.57
lxc config device add foo eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.56
lxc start foo && lxc start bar
lxc list foo

+--------+---------+-------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+------+------------+-----------+
| foo    | RUNNING | 10.0.4.56 (eth0)  |      | PERSISTENT | 0         |

lxc list bar
+------+---------+-------------------+------+------------+-----------+
| NAME |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+------+---------+-------------------+------+------------+-----------+
| bar  | RUNNING | 10.0.4.215 (eth0) |      | PERSISTENT | 0         |
+------+---------+-------------------+------+------------+-----------+

cat /storage/lxd/networks/lxdbr0/dnsmasq.hosts
00:16:3e:e4:b7:06,id:*,10.0.4.4
00:16:3e:b9:89:cd,id:*,10.0.4.55
00:16:3e:7a:e8:37,id:*,10.0.4.3
00:16:3e:03:44:e1,id:*,10.0.4.2

As you can see, it isn´t always reproducible but it happens often that the containers get the wrong IP.
If you need anything else for troubleshooting let me know, thank you.

The "unknown interface" and missing IP problems you've been running into is because you ran "ifdown eth0" inside the container but "ifup eth0" outside of it.
lxc exec contX -- ifdown eth0 && ifup eth0

Should have been:
lxc exec contX -- ifdown eth0 && lxc exec contX -- ifup eth0

Or:
lxc exec contX -- sh -c "ifdown eth0 && ifup eth0"

The missing entry in the hosts file is still troubling. Can you paste your /var/log/lxd/lxd.log to see if there's anything in there that could explain it?

I'm hoping that switch to per-container hosts file will solve this issue or at least make it easier to debug, will try to set that up later today.

Thank you for merging. I´m looking forward to test the fix.

Updated to lxc 2.17 now this is getting weird, none of the containers got the actual IP address:
for i in {5..10}; do lxc init ubuntu:16.04 cont$i; lxc config device add cont$i eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.$i;lxc start cont$i; done
 lxc list cont
+--------+---------+-------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+------+------------+-----------+
| cont10 | RUNNING | 10.0.4.246 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont5  | RUNNING | 10.0.4.25 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont6  | RUNNING | 10.0.4.58 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont7  | RUNNING | 10.0.4.120 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont8  | RUNNING | 10.0.4.29 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont9  | RUNNING | 10.0.4.169 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+

cat /storage/lxd/networks/lxdbr0/dnsmasq.leases
1504428520 00:16:3e:5f:be:4d 10.0.4.246 cont10 *
1504428490 00:16:3e:04:01:27 10.0.4.169 cont9 *
1504428452 00:16:3e:ba:d7:ec 10.0.4.29 cont8 *
1504428426 00:16:3e:ac:14:f3 10.0.4.120 cont7 *
1504428391 00:16:3e:2b:f2:32 10.0.4.58 cont6 *
1504428351 00:16:3e:d6:fb:b7 10.0.4.25 cont5 *

cat /var/log/syslog | grep DENIED

Sep  3 09:45:47 nas-10-media kernel: [  586.579504] audit: type=1400 audit(1504424747.820:74): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont5_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=5882 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 09:45:47 nas-10-media kernel: [  586.579542] audit: type=1400 audit(1504424747.820:75): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont5_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=5882 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 09:46:28 nas-10-media kernel: [  626.842624] audit: type=1400 audit(1504424788.083:90): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont6_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=6516 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 09:46:28 nas-10-media kernel: [  626.842644] audit: type=1400 audit(1504424788.083:91): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont6_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=6516 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 09:46:59 nas-10-media kernel: [  658.615438] audit: type=1400 audit(1504424819.854:105): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont7_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=7380 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 09:46:59 nas-10-media kernel: [  658.615467] audit: type=1400 audit(1504424819.854:106): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont7_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=7380 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 09:47:29 nas-10-media kernel: [  688.007096] audit: type=1400 audit(1504424849.242:121): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont8_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=8164 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 09:47:29 nas-10-media kernel: [  688.007289] audit: type=1400 audit(1504424849.242:122): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont8_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=8164 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 09:48:07 nas-10-media kernel: [  726.008675] audit: type=1400 audit(1504424887.241:137): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont9_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=8987 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 09:48:07 nas-10-media kernel: [  726.008695] audit: type=1400 audit(1504424887.241:138): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont9_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=8987 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 09:48:37 nas-10-media kernel: [  755.867659] audit: type=1400 audit(1504424917.096:153): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont10_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=9775 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 09:48:37 nas-10-media kernel: [  755.867707] audit: type=1400 audit(1504424917.096:154): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont10_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=9775 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536

Another steps which apparently worked:
for i in {5..10}; do lxc stop cont$i;done
Deleting wrong leases:
cat /storage/lxd/networks/lxdbr0/dnsmasq.leases | grep -v "cont" > /storage/lxd/networks/lxdbr0/dnsmasq.leases
killall dnsmasq -SIGHUP
Extracting MAC addreses of containers:
for i in {5..10};do mac=lxc config show cont$i | grep "volatile.eth0.hwaddr:"| awk '{print $2}'; echo "$mac,10.0.4.$i"> /storage/lxd/networks/lxdbr0/dnsmasq.hosts/cont$i;done
for i in {5..10}; do lxc start cont$i;done
lxc list cont
+--------+---------+------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4       | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+------------------+------+------------+-----------+
| cont10 | RUNNING | 10.0.4.10 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont5  | RUNNING | 10.0.4.5 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont6  | RUNNING | 10.0.4.6 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont7  | RUNNING | 10.0.4.7 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont8  | RUNNING | 10.0.4.8 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont9  | RUNNING | 10.0.4.9 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+

This is getting frustrating:
for i in {21..25}; do lxc init ubuntu:16.04 cont$i; lxc config device add cont$i eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.$i;mac=`lxc config show cont$i | grep "volatile.eth0.hwaddr:"| awk '{print $2}'`; echo "$mac,10.0.4.$i"> /storage/lxd/networks/lxdbr0/dnsmasq.hosts/cont$i; lxc start cont$i; done

lxc list cont
+--------+---------+------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4       | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+------------------+------+------------+-----------+
| cont21 | RUNNING | 10.0.4.21 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont22 | RUNNING | 10.0.4.22 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont23 | RUNNING | 10.0.4.23 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont24 | RUNNING | 10.0.4.24 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont25 | RUNNING | 10.0.4.66 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+

ls -la dnsmasq.hosts/
total 28
drwxr-xr-x 1 root root 114 Sep  3 12:33 .
drwx--x--x 1 root root  98 Sep  3 12:28 ..
-rw-r--r-- 1 root root  28 Sep  3 12:32 cont21
-rw-r--r-- 1 root root  28 Sep  3 12:32 cont22
-rw-r--r-- 1 root root  28 Sep  3 12:32 cont23
-rw-r--r-- 1 root root  28 Sep  3 12:33 cont24
-rw-r--r-- 1 root root  28 Sep  3 12:33 cont25

cat dnsmasq.hosts/cont25
00:16:3e:45:8f:78,10.0.4.25

--------+
/storage/lxd/networks/lxdbr0/dnsmasq.hosts# lxc config show cont25
architecture: x86_64
config:
  image.architecture: amd64
  image.description: ubuntu 16.04 LTS amd64 (release) (20170815.1)
  image.label: release
  image.os: ubuntu
  image.release: xenial
  image.serial: "20170815.1"
  image.version: "16.04"
  volatile.base_image: 58f90cbf68927c3fc43e6ee1386446a04f3d8068c1a75a291339cb2be01dec08
  volatile.eth0.hwaddr: 00:16:3e:45:8f:78
  volatile.eth0.name: eth0
  volatile.idmap.base: "0"
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":165536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":165536,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":165536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":165536,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    ipv4.address: 10.0.4.25
    nictype: bridged
    parent: lxdbr0
    type: nic
ephemeral: false
profiles:
- default
stateful: false
description: ""

No cont* leases in dnsmasq.leases
cat dnsmasq.leases
1504436692 00:16:3e:03:44:e1 10.0.4.2 nas-10-rundeck *

cat /var/log/syslog | grep cont25
Sep  3 12:25:36 prod-server01 kernel: [10176.196002] audit: type=1400 audit(1504434336.825:1504): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-cont25_</var/lib/lxd>" pid=6976 comm="apparmor_parser"
Sep  3 12:25:43 prod-server01 kernel: [10183.065681] audit: type=1400 audit(1504434343.692:1505): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="lxc-container-default" pid=7398 comm="apparmor_parser"
Sep  3 12:25:43 prod-server01 kernel: [10183.066468] audit: type=1400 audit(1504434343.692:1506): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="lxc-container-default-cgns" pid=7398 comm="apparmor_parser"
Sep  3 12:25:43 prod-server01 kernel: [10183.067056] audit: type=1400 audit(1504434343.692:1507): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="lxc-container-default-with-mounting" pid=7398 comm="apparmor_parser"
Sep  3 12:25:43 prod-server01 kernel: [10183.068195] audit: type=1400 audit(1504434343.696:1508): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="lxc-container-default-with-nesting" pid=7398 comm="apparmor_parser"
Sep  3 12:25:43 prod-server01 kernel: [10183.087643] audit: type=1400 audit(1504434343.716:1509): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="/usr/bin/lxc-start" pid=7413 comm="apparmor_parser"
Sep  3 12:25:44 prod-server01 kernel: [10183.430450] audit: type=1400 audit(1504434344.056:1510): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="/usr/lib/lxd/lxd-bridge-proxy" pid=7415 comm="apparmor_parser"
Sep  3 12:25:44 prod-server01 kernel: [10183.542807] audit: type=1400 audit(1504434344.168:1511): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="/sbin/dhclient" pid=7404 comm="apparmor_parser"
Sep  3 12:25:44 prod-server01 kernel: [10183.543484] audit: type=1400 audit(1504434344.168:1512): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=7404 comm="apparmor_parser"
Sep  3 12:25:44 prod-server01 kernel: [10183.544304] audit: type=1400 audit(1504434344.172:1513): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=7404 comm="apparmor_parser"
Sep  3 12:27:11 prod-server01 kernel: [10270.965484] audit: type=1400 audit(1504434431.590:1532): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont25_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=9579 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 12:27:11 prod-server01 kernel: [10270.965507] audit: type=1400 audit(1504434431.590:1533): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont25_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=9579 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 12:27:13 prod-server01 kernel: [10273.253370] audit: type=1400 audit(1504434433.878:1534): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="lxd-cont25_</var/lib/lxd>" pid=9760 comm="apparmor_parser"
Sep  3 12:33:48 prod-server01 kernel: [10667.894565] audit: type=1400 audit(1504434828.495:1599): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-cont25_</var/lib/lxd>" pid=13584 comm="apparmor_parser"
Sep  3 12:33:55 prod-server01 kernel: [10674.668746] audit: type=1400 audit(1504434835.267:1600): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="/sbin/dhclient" pid=13981 comm="apparmor_parser"
Sep  3 12:33:55 prod-server01 kernel: [10674.669430] audit: type=1400 audit(1504434835.267:1601): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=13981 comm="apparmor_parser"
Sep  3 12:33:55 prod-server01 kernel: [10674.669937] audit: type=1400 audit(1504434835.267:1602): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=13981 comm="apparmor_parser"
Sep  3 12:33:55 prod-server01 kernel: [10674.670429] audit: type=1400 audit(1504434835.267:1603): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=13981 comm="apparmor_parser"
Sep  3 12:33:55 prod-server01 kernel: [10674.688827] audit: type=1400 audit(1504434835.287:1604): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="/usr/bin/lxc-start" pid=13995 comm="apparmor_parser"
Sep  3 12:33:55 prod-server01 kernel: [10675.110618] audit: type=1400 audit(1504434835.707:1605): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="/usr/lib/lxd/lxd-bridge-proxy" pid=13997 comm="apparmor_parser"
Sep  3 12:33:55 prod-server01 kernel: [10675.328322] audit: type=1400 audit(1504434835.927:1606): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="lxc-container-default" pid=13988 comm="apparmor_parser"
Sep  3 12:33:55 prod-server01 kernel: [10675.328922] audit: type=1400 audit(1504434835.927:1607): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="lxc-container-default-cgns" pid=13988 comm="apparmor_parser"
Sep  3 12:33:55 prod-server01 kernel: [10675.329456] audit: type=1400 audit(1504434835.927:1608): apparmor="STATUS" operation="profile_load" label="lxd-cont25_</var/lib/lxd>//&:lxd-cont25_<var-lib-lxd>://unconfined" name="lxc-container-default-with-mounting" pid=13988 comm="apparmor_parser"
Sep  3 12:34:00 prod-server01 kernel: [10679.759825] audit: type=1400 audit(1504434840.359:1613): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont25_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=14209 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536
Sep  3 12:34:00 prod-server01 kernel: [10679.759859] audit: type=1400 audit(1504434840.359:1614): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-cont25_<var-lib-lxd>" profile="/sbin/dhclient" name="/run/systemd/journal/stdout" pid=14209 comm="dhclient" requested_mask="wr" denied_mask="wr" fsuid=165536 ouid=165536

This issue cannot be reproduced and I´m not sure if this in fact a LXD or dnsmasq issue

That really feels like dnsmasq being racy... Would have been interesting to try an "ifdown eth0 && ifup eth0" inside that container without restarting it and without reloading dnsmasq. If it then gets the right IP, that'd show that dnsmasq didn't process the reload signal in time and so the container got assigned an IP before dnsmasq actually refreshed its cache...

OK here we go:
for i in {21..25}; do lxc stop cont$i && lxc delete cont$i;done
Creating containers:
for i in {21..25}; do lxc init ubuntu:16.04 cont$i; lxc config device add cont$i eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.$i;mac=`lxc config show cont$i | grep "volatile.eth0.hwaddr:"| awk '{print $2}'`; echo "$mac,10.0.4.$i"> /storage/lxd/networks/lxdbr0/dnsmasq.hosts/cont$i; lxc start cont$i; done

lxc list cont
+--------+---------+-------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+------+------------+-----------+
| cont21 | RUNNING | 10.0.4.21 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont22 | RUNNING | 10.0.4.22 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont23 | RUNNING | 10.0.4.23 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont24 | RUNNING | 10.0.4.24 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont25 | RUNNING | 10.0.4.164 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+

cont25:~# ifdown eth0 && ifup eth0
Killed old client process
Internet Systems Consortium DHCP Client 4.3.3
Copyright 2004-2015 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/eth0/00:16:3e:15:43:74
Sending on   LPF/eth0/00:16:3e:15:43:74
Sending on   Socket/fallback
DHCPRELEASE on eth0 to 10.0.4.1 port 67 (xid=0x2a5ec24b)
Internet Systems Consortium DHCP Client 4.3.3
Copyright 2004-2015 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/eth0/00:16:3e:15:43:74
Sending on   LPF/eth0/00:16:3e:15:43:74
Sending on   Socket/fallback
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3 (xid=0x1da55c0b)
DHCPREQUEST of 10.0.4.164 on eth0 to 255.255.255.255 port 67 (xid=0xb5ca51d)
DHCPOFFER of 10.0.4.164 from 10.0.4.1
DHCPACK of 10.0.4.164 from 10.0.4.1
bound to 10.0.4.164 -- renewal in 1744 seconds.

lxc list cont
+--------+---------+-------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+------+------------+-----------+
| cont21 | RUNNING | 10.0.4.21 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont22 | RUNNING | 10.0.4.22 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont23 | RUNNING | 10.0.4.23 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont24 | RUNNING | 10.0.4.24 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont25 | RUNNING | 10.0.4.164 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+

cat dnsmasq.leases | grep cont
1504472050 00:16:3e:15:43:74 10.0.4.164 cont25 *
1504471952 00:16:3e:6f:89:04 10.0.4.24 cont24 *
1504471917 00:16:3e:9e:e0:e4 10.0.4.23 cont23 *
1504471881 00:16:3e:e2:6b:66 10.0.4.22 cont22 *
1504471847 00:16:3e:61:67:55 10.0.4.21 cont21 *

And dnsmasq.hosts/cont25 contains 10.0.4.25?

If so, can you send SIGHUP to dnsmasq and then try the ifdown&ifup again (so without ever stopping/restarting the container)?
That way only dnsmasq gets poked and LXD never gets involved. If that fixes it, then that pretty much narrows it down to dnsmasq's reload code being racy and occasionally dropping events on the floor...

I really wish dnsmasq had a proper API to configure and query it... text files and POSIX signals doesn't really make a reliable API...

Hi @stgraber , thank you for you patience.
cat dnsmasq.hosts/cont25
00:16:3e:15:43:74,10.0.4.25

Yes, you´re right:
killall dnsmasq -SIGHUP
prod-servoer01:/storage/lxd/networks/lxdbr0# lxc exec cont25 -- ifdown eth0 && lxc exec cont25 -- ifup eth0
Killed old client process
Internet Systems Consortium DHCP Client 4.3.3
Copyright 2004-2015 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/eth0/00:16:3e:15:43:74
Sending on   LPF/eth0/00:16:3e:15:43:74
Sending on   Socket/fallback
DHCPRELEASE on eth0 to 10.0.4.1 port 67 (xid=0x2b39402e)
Internet Systems Consortium DHCP Client 4.3.3
Copyright 2004-2015 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/eth0/00:16:3e:15:43:74
Sending on   LPF/eth0/00:16:3e:15:43:74
Sending on   Socket/fallback
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3 (xid=0xc341d53a)
DHCPREQUEST of 10.0.4.25 on eth0 to 255.255.255.255 port 67 (xid=0x3ad541c3)
DHCPOFFER of 10.0.4.25 from 10.0.4.1
DHCPACK of 10.0.4.25 from 10.0.4.1

lxc list cont
+--------+---------+------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4       | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+------------------+------+------------+-----------+
| cont21 | RUNNING | 10.0.4.21 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont22 | RUNNING | 10.0.4.22 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont23 | RUNNING | 10.0.4.23 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont24 | RUNNING | 10.0.4.24 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+
| cont25 | RUNNING | 10.0.4.25 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------+------+------------+-----------+

However, what I do not understand: shouldn´t LXD create the static host entry if the ipv4.address property is set on the container's network device?
If I create the container with the command:
for i in {21..25}; do lxc init ubuntu:16.04 cont$i; lxc config device add cont$i eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.$i; lxc start cont$i; done

All container receive bogus IP addresses:
lxc list cont
+--------+---------+-------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+------+------------+-----------+
| cont21 | RUNNING | 10.0.4.145 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont22 | RUNNING | 10.0.4.93 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont23 | RUNNING | 10.0.4.32 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont24 | RUNNING | 10.0.4.113 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont25 | RUNNING | 10.0.4.81 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+

and no host files are created:
ls -la dnsmasq.hosts/
total 8
drwxr-xr-x 1 root root 54 Sep  4 08:22 .
drwx--x--x 1 root root 98 Sep  4 08:22 ..

If I create the container with their corresponding host files:
for i in {21..25}; do lxc init ubuntu:16.04 cont$i; lxc config device add cont$i eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.$i;mac=`lxc config show cont$i | grep "volatile.eth0.hwaddr:"| awk '{print $2}'`; echo "$mac,10.0.4.$i"> /storage/lxd/networks/lxdbr0/dnsmasq.hosts/cont$i; lxc start cont$i; done

it works as expected, well except the dnsmasq SIGHUP issue.
Now, if I create the container without setting the ipv4.address but only the host files:
for i in {21..25}; do lxc init ubuntu:16.04 cont$i;mac=`lxc config show cont$i | grep "volatile.eth0.hwaddr:"| awk '{print $2}'`; echo "$mac,10.0.4.$i"> /storage/lxd/networks/lxdbr0/dnsmasq.hosts/cont$i; lxc start cont$i; done

it works also:
So, what´s the point in setting the ipv4.address if one have to create the host files as well? Am I messing something?
Thank you.

The files certainly do get created here...

stgraber@castiana:~/data/code/lxc/lxd (stgraber/master)$ sudo ls -lh /var/lib/lxd/networks/lxdbr0/dnsmasq.hosts/
total 2.0K
-rw-r--r-- 1 root root 26 Sep  4 02:52 keybase
-rw-r--r-- 1 root root 28 Sep  4 02:52 snapcraft
-rw-r--r-- 1 root root 30 Sep  4 02:52 ups-monitor
-rw-r--r-- 1 root root 27 Sep  4 02:52 zerotier

stgraber@castiana:~/data/code/lxc/lxd (stgraber/master)$ for i in {21..25}; do lxc init ubuntu:16.04 cont$i; lxc config device add cont$i eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.204.119.$i; lxc start cont$i; done
Creating cont21
Device eth0 added to cont21
Creating cont22
Device eth0 added to cont22
Creating cont23
Device eth0 added to cont23
Creating cont24
Device eth0 added to cont24
Creating cont25
Device eth0 added to cont25

stgraber@castiana:~/data/code/lxc/lxd (stgraber/master)$ lxc list cont
+--------+---------+----------------------+----------------------------------------------+------------+-----------+
|  NAME  |  STATE  |         IPV4         |                     IPV6                     |    TYPE    | SNAPSHOTS |
+--------+---------+----------------------+----------------------------------------------+------------+-----------+
| cont21 | RUNNING | 10.204.119.21 (eth0) | 2001:470:b368:4242:216:3eff:fe47:e84e (eth0) | PERSISTENT | 0         |
+--------+---------+----------------------+----------------------------------------------+------------+-----------+
| cont22 | RUNNING | 10.204.119.22 (eth0) | 2001:470:b368:4242:216:3eff:fe8e:fe08 (eth0) | PERSISTENT | 0         |
+--------+---------+----------------------+----------------------------------------------+------------+-----------+
| cont23 | RUNNING | 10.204.119.23 (eth0) | 2001:470:b368:4242:216:3eff:fe76:c39f (eth0) | PERSISTENT | 0         |
+--------+---------+----------------------+----------------------------------------------+------------+-----------+
| cont24 | RUNNING | 10.204.119.24 (eth0) | 2001:470:b368:4242:216:3eff:fe04:9731 (eth0) | PERSISTENT | 0         |
+--------+---------+----------------------+----------------------------------------------+------------+-----------+
| cont25 | RUNNING | 10.204.119.25 (eth0) | 2001:470:b368:4242:216:3eff:fedd:d090 (eth0) | PERSISTENT | 0         |
+--------+---------+----------------------+----------------------------------------------+------------+-----------+

stgraber@castiana:~/data/code/lxc/lxd (stgraber/master)$ sudo ls -lh /var/lib/lxd/networks/lxdbr0/dnsmasq.hosts/total 4.5K
-rw-r--r-- 1 root root 39 Sep  4 02:57 cont21
-rw-r--r-- 1 root root 39 Sep  4 02:57 cont22
-rw-r--r-- 1 root root 39 Sep  4 02:57 cont23
-rw-r--r-- 1 root root 39 Sep  4 02:57 cont24
-rw-r--r-- 1 root root 39 Sep  4 02:57 cont25
-rw-r--r-- 1 root root 26 Sep  4 02:57 keybase
-rw-r--r-- 1 root root 28 Sep  4 02:57 snapcraft
-rw-r--r-- 1 root root 30 Sep  4 02:57 ups-monitor
-rw-r--r-- 1 root root 27 Sep  4 02:57 zerotier

Well, as you can see I can reproduce this every time. No host files are created:
root@prod-server01:/storage/lxd/networks/lxdbr0# for i in {51..55}; do lxc init ubuntu:16.04 cont$i; lxc config device add cont$i eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.$i; lxc start cont$i; done
Creating cont51
Device eth0 added to cont51
Creating cont52
Device eth0 added to cont52
Creating cont53
Device eth0 added to cont53
Creating cont54
Device eth0 added to cont54
Creating cont55
Device eth0 added to cont55

s -la dnsmasq.hosts/
total 28
drwxr-xr-x 1 root root 114 Sep  4 08:41 .
drwx--x--x 1 root root  98 Sep  4 08:35 ..
-rw-r--r-- 1 root root  28 Sep  4 08:38 cont21
-rw-r--r-- 1 root root  28 Sep  4 08:39 cont22
-rw-r--r-- 1 root root  28 Sep  4 08:39 cont23
-rw-r--r-- 1 root root  28 Sep  4 08:40 cont24
-rw-r--r-- 1 root root  28 Sep  4 08:41 cont25

lxc list cont
+--------+---------+-------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+------+------------+-----------+
| cont21 | RUNNING | 10.0.4.21 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont22 | RUNNING | 10.0.4.22 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont23 | RUNNING | 10.0.4.23 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont24 | RUNNING | 10.0.4.24 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont25 | RUNNING | 10.0.4.25 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont51 | RUNNING | 10.0.4.10 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont52 | RUNNING | 10.0.4.9 (eth0)   |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont53 | RUNNING | 10.0.4.107 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont54 | RUNNING | 10.0.4.89 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont55 | RUNNING | 10.0.4.57 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| contX  | RUNNING | 10.0.4.150 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+

cat dnsmasq.leases
1504512350 00:16:3e:61:82:61 10.0.4.57 cont55 *
1504512323 00:16:3e:e4:3b:f6 10.0.4.89 cont54 *
1504512291 00:16:3e:7b:6d:51 10.0.4.107 cont53 *
1504512255 00:16:3e:dd:c8:2c 10.0.4.9 cont52 *
1504512216 00:16:3e:c9:00:bd 10.0.4.10 cont51 *
1504512064 00:16:3e:98:f3:97 10.0.4.150 contX *
1504511889 00:16:3e:99:0b:e5 10.0.4.25 cont25 *
1504510870 00:16:3e:46:0e:a6 10.0.4.24 cont24 *
1504512547 00:16:3e:e7:b0:54 10.0.4.23 cont23 *
1504512235 00:16:3e:89:86:df 10.0.4.22 cont22 *
1504512458 00:16:3e:d4:56:02 10.0.4.21 cont21 *

lxc network show lxdbr0
ls -lh /var/lib/lxd/networks/lxdbr0/

lxc network show lxdbr0
config:
  dns.mode: dynamic
  ipv4.address: 10.0.4.1/24
  ipv4.dhcp.ranges: 10.0.4.2-10.0.4.254
  ipv4.nat: "true"
  ipv6.address: none
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/containers/cont21
- /1.0/containers/cont22
- /1.0/containers/cont23
- /1.0/containers/cont24
- /1.0/containers/cont25
- /1.0/containers/cont51
- /1.0/containers/cont52
- /1.0/containers/cont53
- /1.0/containers/cont54
- /1.0/containers/cont55
- /1.0/containers/contX
managed: true

ls -lh /storage/lxd/networks/lxdbr0/
total 12K
drwxr-xr-x 1 root root 114 Sep  4 08:41 dnsmasq.hosts
-rw-r--r-- 1 root root 583 Sep  4 09:37 dnsmasq.leases
-rw-r--r-- 1 root root   6 Sep  4 08:35 dnsmasq.pid
---------- 1 root root   1 Sep  4 08:35 dnsmasq.raw

What happens if you restart the LXD daemon? do they appear then?

Also, what exactly are you running? Are you running current LXD from git master?

lxc info
config:
  core.https_address: '[::]:8443'
  core.trust_password: true
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
api_status: stable
api_version: "1.0"
auth: trusted
public: false
environment:
  addresses:
  - 192.168.100.2:8443
  - 10.0.3.1:8443
  - 10.0.4.1:8443
  architectures:
  - x86_64
  - i686
  certificate: |
    -----BEGIN CERTIFICATE-----

    -----END CERTIFICATE-----
  certificate_fingerprint: df7f2254e113336dba1df531f202e7f4948da7b8044212cf8b5326330310456d
  driver: lxc
  driver_version: 2.0.8
  kernel: Linux
  kernel_architecture: x86_64
  kernel_version: 4.4.0-93-generic
  server: lxd
  server_pid: 24328
  server_version: "2.17"
  storage: btrfs
  storage_version: "4.4"

Package: lxd
Status: install ok installed
Priority: optional
Section: admin
Installed-Size: 13751
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: amd64
Version: 2.17-0ubuntu2~ubuntu16.04.1
Depends: acl, adduser, dnsmasq-base, ebtables, iptables, liblxc1 (>= 2.0.0~), lsb-base (>= 3.0-6), lxd-client (= 2.17-0ubuntu2~ubuntu16.04.1), lxcfs, passwd (>= 1:4.1.5.1-1ubuntu5~), rsync, squashfs-tools, uidmap (>= 1:4.1.5.1-1ubuntu5~), xdelta3, xz-utils, init-system-helpers (>= 1.18~), libacl1 (>= 2.2.51-8), libc6 (>= 2.14), libsqlite3-0 (>= 3.7.15)
Suggests: criu, lxd-tools
Conffiles:
 /etc/dnsmasq.d-available/lxd c72be7e51163b9fa73e2eb8986958986
 /etc/init.d/lxd 9d2a101c9d9a5484f16607f4d7e10c4d
 /etc/logrotate.d/lxd 312901c6c80bd878756257952fd24068
 /etc/sysctl.d/10-lxd-inotify.conf 278c217e18c2aae80d847838869e4ffd
Description: Container hypervisor based on LXC - daemon
 LXD offers a REST API to remotely manage containers over the network,
 using an image based workflow and with support for live migration.
 .
 This package contains the LXD daemon.
Built-Using: golang-1.6 (= 1.6.2-0ubuntu5~16.04.3), golang-defaults (= 2:1.6-1ubuntu4)
Homepage: https://linuxcontainers.org/

service lxd restart
ls -la /storage/lxd/networks/lxdbr0/dnsmasq.hosts/
total 32
drwxr-xr-x 1 root root 124 Sep  4 10:27 .
drwx--x--x 1 root root  98 Sep  4 10:27 ..
-rw-r--r-- 1 root root  28 Sep  4 10:27 cont51
-rw-r--r-- 1 root root  28 Sep  4 10:27 cont52
-rw-r--r-- 1 root root  28 Sep  4 10:27 cont53
-rw-r--r-- 1 root root  28 Sep  4 10:27 cont54
-rw-r--r-- 1 root root  28 Sep  4 10:27 cont55
-rw-r--r-- 1 root root  28 Sep  4 10:27 contX

lxc list cont
+--------+---------+-------------------+------+------------+-----------+
|  NAME  |  STATE  |       IPV4        | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+------+------------+-----------+
| cont21 | RUNNING | 10.0.4.21 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont22 | RUNNING | 10.0.4.22 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont23 | RUNNING | 10.0.4.23 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont24 | RUNNING | 10.0.4.24 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont25 | RUNNING | 10.0.4.25 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont51 | RUNNING | 10.0.4.10 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont52 | RUNNING | 10.0.4.9 (eth0)   |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont53 | RUNNING | 10.0.4.107 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont54 | RUNNING | 10.0.4.89 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| cont55 | RUNNING | 10.0.4.57 (eth0)  |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+
| contX  | RUNNING | 10.0.4.150 (eth0) |      | PERSISTENT | 0         |
+--------+---------+-------------------+------+------------+-----------+

The manually created hosts (cont2x) are removed
cat dnsmasq.leases
1504516835 00:16:3e:61:82:61 10.0.4.57 cont55 *
1504517018 00:16:3e:e4:3b:f6 10.0.4.89 cont54 *
1504516205 00:16:3e:7b:6d:51 10.0.4.107 cont53 *
1504516759 00:16:3e:dd:c8:2c 10.0.4.9 cont52 *
1504516767 00:16:3e:c9:00:bd 10.0.4.10 cont51 *
1504516670 00:16:3e:98:f3:97 10.0.4.150 contX *
1504516360 00:16:3e:99:0b:e5 10.0.4.25 cont25 *
1504516722 00:16:3e:46:0e:a6 10.0.4.24 cont24 *
1504517050 00:16:3e:e7:b0:54 10.0.4.23 cont23 *
1504516689 00:16:3e:89:86:df 10.0.4.22 cont22 *
1504516761 00:16:3e:d4:56:02 10.0.4.21 cont21 *

I´m almost tempted do backup all containers, remove lxc/lxd and re-install again :/

I´ve deinstalled all lxd packages and removed orphaned configuration files.
Then:
add-apt-repository ppa:ubuntu-lxc/lxd-stable
apt update && apt install lxd lxd-tools
for i in {51..55}; do lxc init ubuntu:16.04 cont$i; lxc config device add cont$i eth0 nic nictype=bridged parent=lxdbr0 ipv4.address=10.0.4.$i; lxc start cont$i; done

+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
|  NAME  |  STATE  |       IPV4        |                     IPV6                      |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
| cont51 | RUNNING | 10.0.4.134 (eth0) | fd8b:551a:ca29:8dd3:216:3eff:fec9:47d6 (eth0) | PERSISTENT | 0         |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
| cont52 | RUNNING | 10.0.4.120 (eth0) | fd8b:551a:ca29:8dd3:216:3eff:fe7c:41ca (eth0) | PERSISTENT | 0         |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
| cont53 | RUNNING | 10.0.4.147 (eth0) | fd8b:551a:ca29:8dd3:216:3eff:fef7:46f7 (eth0) | PERSISTENT | 0         |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
| cont54 | RUNNING | 10.0.4.166 (eth0) | fd8b:551a:ca29:8dd3:216:3eff:fe8c:c11c (eth0) | PERSISTENT | 0         |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
| cont55 | RUNNING | 10.0.4.197 (eth0) | fd8b:551a:ca29:8dd3:216:3eff:fe98:6a6f (eth0) | PERSISTENT | 0         |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+

for i in {51..55};do lxc exec cont$i -- ifdown eth0 && lxc exec cont$i -- ifup eth0;done
lxd doesnt care about setting the ip manually:
lxc list

+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
|  NAME  |  STATE  |       IPV4        |                     IPV6                      |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
| cont51 | RUNNING | 10.0.4.134 (eth0) | fd8b:551a:ca29:8dd3:216:3eff:fec9:47d6 (eth0) | PERSISTENT | 0         |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
| cont52 | RUNNING | 10.0.4.120 (eth0) | fd8b:551a:ca29:8dd3:216:3eff:fe7c:41ca (eth0) | PERSISTENT | 0         |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
| cont53 | RUNNING | 10.0.4.147 (eth0) | fd8b:551a:ca29:8dd3:216:3eff:fef7:46f7 (eth0) | PERSISTENT | 0         |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
| cont54 | RUNNING | 10.0.4.166 (eth0) | fd8b:551a:ca29:8dd3:216:3eff:fe8c:c11c (eth0) | PERSISTENT | 0         |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
| cont55 | RUNNING | 10.0.4.197 (eth0) | fd8b:551a:ca29:8dd3:216:3eff:fe98:6a6f (eth0) | PERSISTENT | 0         |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+

ls -la /storage/lxd/networks/lxdbr0/dnsmasq.hosts/
total 0
drwxr-xr-x 1 root root  0 Sep  9 18:38 .
drwx--x--x 1 root root 98 Sep  9 18:38 ..

cat /storage/lxd/networks/lxdbr0/dnsmasq.leases
1504978880 00:16:3e:98:6a:6f 10.0.4.197 cont55 *
1504978878 00:16:3e:8c:c1:1c 10.0.4.166 cont54 *
1504978876 00:16:3e:f7:46:f7 10.0.4.147 cont53 *
1504978874 00:16:3e:7c:41:ca 10.0.4.120 cont52 *
1504978873 00:16:3e:c9:47:d6 10.0.4.134 cont51 *
duid 00:01:00:01:21:46:d5:80:c8:cb:b8:c8:e9:c4

 driver: lxc
  driver_version: 2.0.8
  kernel: Linux
  kernel_architecture: x86_64
  kernel_version: 4.4.0-93-generic
  server: lxd
  server_pid: 31604
  server_version: "2.17"
  storage: btrfs
  storage_version: "4.4"

I´m out of ideas.

I'll re-check with exactly the same packages and environment you have to see if I can get it to finally happen here too.

Ok i figured this out. Actually this post was helpful: https://ubuntuforums.org/showthread.php?t=1890662&p=12390176#post12390176
My DHCP range started from .101 on wards but the assigned IP addresses where not from this range (.21-25 and .51-55)
I´ve edited the lxdbr0 and changed the range. Now the container get the proper IP address assigned:
+--------+---------+------------------+-----------------------------------------------+------------+-----------+
|  NAME  |  STATE  |       IPV4       |                     IPV6                      |    TYPE    | SNAPSHOTS |
+--------+---------+------------------+-----------------------------------------------+------------+-----------+
| cont21 | RUNNING | 10.0.4.21 (eth0) | fd42:779b:f72b:be73:216:3eff:fefc:2bd3 (eth0) | PERSISTENT | 0         |
+--------+---------+------------------+-----------------------------------------------+------------+-----------+
| cont22 | RUNNING | 10.0.4.22 (eth0) | fd42:779b:f72b:be73:216:3eff:fee4:5794 (eth0) | PERSISTENT | 0         |
+--------+---------+------------------+-----------------------------------------------+------------+-----------+
| cont23 | RUNNING | 10.0.4.23 (eth0) | fd42:779b:f72b:be73:216:3eff:fea8:37d0 (eth0) | PERSISTENT | 0         |
+--------+---------+------------------+-----------------------------------------------+------------+-----------+
| cont24 | RUNNING | 10.0.4.24 (eth0) | fd42:779b:f72b:be73:216:3eff:fef5:df3a (eth0) | PERSISTENT | 0         |
+--------+---------+------------------+-----------------------------------------------+------------+-----------+
| cont25 | RUNNING | 10.0.4.25 (eth0) | fd42:779b:f72b:be73:216:3eff:fe19:ff4b (eth0) | PERSISTENT | 0         |
+--------+---------+------------------+-----------------------------------------------+------------+-----------+
| cont51 | RUNNING | 10.0.4.51 (eth0) | fd42:779b:f72b:be73:216:3eff:fe4d:76d3 (eth0) | PERSISTENT | 0         |
+--------+---------+------------------+-----------------------------------------------+------------+-----------+
| cont52 | RUNNING | 10.0.4.52 (eth0) | fd42:779b:f72b:be73:216:3eff:feeb:3091 (eth0) | PERSISTENT | 0         |
+--------+---------+------------------+-----------------------------------------------+------------+-----------+
| cont53 | RUNNING | 10.0.4.53 (eth0) | fd42:779b:f72b:be73:216:3eff:fe70:2ae2 (eth0) | PERSISTENT | 0         |
+--------+---------+------------------+-----------------------------------------------+------------+-----------+
| cont54 | RUNNING | 10.0.4.54 (eth0) | fd42:779b:f72b:be73:216:3eff:fe1e:ecde (eth0) | PERSISTENT | 0         |
+--------+---------+------------------+-----------------------------------------------+------------+-----------+
| cont55 | RUNNING | 10.0.4.55 (eth0) | fd42:779b:f72b:be73:216:3eff:fecd:c8ff (eth0) | PERSISTENT | 0         |
+--------+---------+------------------+-----------------------------------------------+------------+-----------+

Sorry for the noise.

For me actually only these steps works:

delete profile
run sudo lxd init --auto
recreate containers

authors: huegelc rostenkowski stgraber


text:
doc: move Sphinx extensions to a separate repo

Just checking if this works, for now.
The extension is here: https://github.com/canonical/lxd-sphinx-extensions/pull/1
The extensions can be reused by other projects and should not
be part of the LXD code base.
Signed-off-by: Ruth Fuchss ruth.fuchss@canonical.com

Documentation preview available at: https://linuxcontainers.org/lxd/docs/pr.10150/

source: gharchive/pull-request

authors: lxc-jenkins ru-fu

repo: lxc/lxd

license_type: permissive

license_source: bigquery


text:
update func instanceLogDelete()

Updated the instanceLogDelete() function to allow deleting of logs from instances in projects other than default.

This pull request didn't trigger Jenkins as its author isn't in the whitelist.
An organization member must perform one of the following:

To have this branch tested by Jenkins, use the "ok to test" command.
To have a one time test done, use the "test this please" command.

Those commands are simple Github comments of the format: "jenkins: COMMAND"

Nice catch!
Can you rebase your branch to get rid of the merge commit and also edit your commit message to include the required:
Signed-off-by: Firstname Lastname <email>

I would expect something like:

git rebase origin
git commit --amend -s
git push --force

Should do it.

Sorry first time creating a pull request for someone else's code. Updated the Signed-off with my name but still failing.

I am going to close this one and try this again, sorry learning how to do this

authors: lxc-jenkins matthewalanpenning stgraber


text:
Instance: Reduce calls to getLxcState to reduce crashes and improve efficiency

There were several unnecessary calls to getLxcState which meant that doing:
while :; do lxc ls; done

With 2 containers would quickly crash LXD.
This PR reduces the calls to getLxcState (which is the function that calls liblxc that triggers the crash mentioned in https://github.com/lxc/lxd/issues/8327), so it is more efficient and crashes less often, although it is still not a full fix. I suspect there is an issue in liblxc or Go but I've not been able to confirm that yet.
Also applies similar optimisation techniques to Qemu driver to reduce calls to QMP and align the two driver's behaviours.

@stgraber fixed


text:
support lxd clustering

i.e. no way to specify the target host in container.create.
in the rest api this is done by adding ?target=target_id
api extension are document at:
https://github.com/lxc/lxd/blob/master/doc/api-extensions.md#clustering

Yes, this is a 3.x feature added via the 'clustering' API extension.
The 'clustering' extension affects a number of areas: containers, storage-pools, and networks.
(Note for implementation: remember to check that the 'clustering' API extension exists as a check as to whether to accept this parameter.)

Any timeline on when clustering will be supported?

@ghoshanjega there is now an api endpoint for getting cluster and member info. What else would you need?
@bit there is a target parameter for containers now.

Hi guys,
I have started using pylxd very recently and the first thing I noticed missing in the cluster supports is creation of networks. Where in the standalone LXD it's enough to do
lxc network create ...

in a clustered environment the process is a bit more complex
lxc network create ... --target node1
...
lxc network create ... --target nodeN
lxc network create ...

Any idea if we could get this supported?

The addition of a target argument for containers was very lightweight.
To support networks in cluster mode, I need the following information:

@stgraber does the lxd api support an undocumented ?target= query parameter for networks like the instance endpoint?
@ajkavanagh what is the architecture decision on the abstraction of this library: Make one complex call which manages the network creation on all nodes or leave it up to the user to perform the necessary calls and add a simple target argument?

@felix-engelmann ?target for networks and storage-pools is supported and lets you create/set the host-specific config ahead of adding a network or storage-pool globally.
That's why you usually need to create a network like:
lxc network create blahbr0 --target a
lxc network create blahbr0 --target b
lxc network create blahbr0 --target c
lxc network create blahbr0

The targeted ones can set keys that are only relevant to a particular member, in the network case, that's pretty much just bridge.external_interfaces.

source: gharchive/issue

authors: ajkavanagh bit felix-engelmann ghoshanjega mkowalski stgraber

repo: lxc/pylxd


text:
${{ steps.lychee.outputs.exit_code }} is not set correctly

According to https://github.com/lycheeverse/lychee-action/blob/5047c2a4052946424ce139fe111135f6d7c0fe0b/action.yml#L40-L43 ${{ steps.lychee.outputs.exit_code }} should be set, but instead the exit_code is solely saved to an env lychee_exit_code.
IMO the quickest fix is to change it to:
echo "lychee_exit_code=$exit_code" >> $GITHUB_ENV
echo "exit_code=$exit_code" >> $GITHUB_OUTPUT

Oh, that's bad. Would you mind opening a pull request to apply your suggested fix?

authors: mre sebastiaanspeck

repo: lycheeverse/lychee-action

license_source: github-api


text:
Have the output summary be optional, and just generate a list of the errors as the default

Assume we just want to see errors and not the summary. Ties into #271.
OPTIONS:
    -s, --summary                    Show a small list detailing the query results.

So, the summary would not be shown by default?

Probably it can depend on whether STDOUT is a terminal or not, so show the summary by default, when it is a terminal, hence most likely an interactive call, and hide it otherwise, when it is most likely a CI call or similar, where indeed often errors are all that is needed. But then -S, --no-summary could be required as well, consequently 🤔.
-s, --scheme <scheme>... btw reserves that short form already. Probably -q, --quite serves as a common alternative?

Alternatively we could tie that into the verbose option: -v would give you the full status output, while by default it wouldn't. Not sure if I like that solution. 😅

Jep, through about that, then moving output for each checked link to next verbosity level -vv, but somehow a summary behind a verbose flag feels wrong. Especially on interactive calls usually one expects some output to know that the command did something 🙂.

So, the summary would not be shown by default?

The final summary table would not be shown. Valid links and what not would still be in the output.

There is a new, compact output format, which only shows a single summary line; see https://github.com/lycheeverse/lychee/pull/375.
Does that work for you? If not, we can add a --format=quiet or --format=raw option, which would get rid of the final status line as well. It would be an easy thing to do, but it depends on your use-case.

Ah, I re-read the original issue at #271. You mention machine-readable output. I guess --format=raw could work in this case and there is also --format=json.

Having both options available would be nice.

Summary for future reference:

[ ]  Add format=raw which skips the summary output.

authors: MichaIng T145 mre

repo: lycheeverse/lychee


text:
grpc: support chaining emitters

We should support chaining with gRPC stream emitters the same way we do for HTTP stream emitters.
I.e.,
let emitter = ...
  .sendMessage(...)
  .sendMessage(...)

This PR also updates tests to use chaining syntax for gRPC.
Signed-off-by: Michael Rebello me@michaelrebello.com

Actually, do you mind updating tests for the grpc emitters for this functionality?

Done

repo: lyft/envoy-mobile


text:
core: enable sending date header with requests

Depends on https://github.com/envoyproxy/envoy/pull/11915.
Signed-off-by: Michael Rebello me@michaelrebello.com

Closing for now, will reopen if upstream lands


text:
Bazel support for //test/common/{common,event}/... (#415).

In addition to the handful of additional tests, this patch introduces additional rules covering ~30%
of the build targets we will need for Envoy build/test, which are transitiviely depended on by the
tests. External deps added for boringssl, c-ares, libevent and tclap.

@lizan @PiotrSikora for review.

Can you update this PR align with #602 too? Thanks.

@lizan @PiotrSikora This is now rebased on #602 and other recent changes. PTAL.

authors: htuch lizan

repo: lyft/envoy


text:
Add platform metadata to execution metadata

TL;DR
Add platform metadata to execution metadata
Type

[ ] Bug Fix
[x] Feature
[ ] Plugin

Are all requirements met?

[ ] Code completed
[ ] Smoke tested
[ ] Unit tests added
[ ] Code documentation added
[ ] Any pending items have an associated Issue

Complete description
How did you fix the bug, make the feature etc. Link to any design docs etc
Tracking Issue
N/A
Follow-up issue
N/A

@kumare3

Actually lets open an issue before we do this, we will close the issue too, once it is completely implemented and add it here in the description

sure created https://github.com/lyft/flyte/issues/205

repo: lyft/flyteidl


text:
Cannot run the test shell script

Hello. Thanks for your work. However, when I try to run the test script of coseg-alien, it gives me an error message like this:
name: coseg-alien
0:   0%|                    | 0/37 [00:00<?, ?it/s]/home/xxx/.cache/jittor/default/g++/jit/_opkey0:array_T:int32__JIT:1__JIT_cuda:1__index_t:int32___opkey1:broadcast_to_Tx:int32__DI...hash:bc2f95c82b48131a_op.cc(40): error: calling a constexpr __host__ function("floor") from a __global__ function("func_bc2f95c82b48131a_0") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

1 error detected in the compilation of "/home/xxx/.cache/jittor/default/g++/jit/_opkey0:array_T:int32__JIT:1__JIT_cuda:1__index_t:int32___opkey1:broadcast_to_Tx:int32__DI...hash:bc2f95c82b48131a_op.cc".
[e 0721 14:21:37.874137 48:C15 parallel_compiler.cc:261] [Error] source file location: /home/xxx/.cache/jittor/default/g++/jit/_opkey0:array_T:int32__JIT:1__JIT_cuda:1__index_t:int32___opkey1:broadcast_to_Tx:int32__DI...hash:bc2f95c82b48131a_op.cc
[e 0721 14:21:37.874176 48:C15 parallel_compiler.cc:264] Compile fused operator(18/56) failed: [Op(0x5561911a3de0:0:0:1:i0:o1:s0,array->0x5561911a3e80),Op(0x5561911a2360:0:0:1:i1:o1:s0,broadcast_to->0x5561911a2410),Op(0x5561911a37f0:0:0:1:i2:o1:s0,binary.mod->0x5561911a30a0),]

Reason: [f 0721 14:21:37.873882 48:C15 log.cc:387] Check failed ret(256) == 0(0) Run cmd failed: cd /home/xxx/.cache/jittor/default/g++ && /home/xxx/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc '/home/xxx/.cache/jittor/default/g++/jit/_opkey0:array_T:int32__JIT:1__JIT_cuda:1__index_t:int32___opkey1:broadcast_to_Tx:int32__DI...hash:bc2f95c82b48131a_op.cc'     -std=c++14 -Xcompiler -fPIC  -Xcompiler -march=native  -Xcompiler -fdiagnostics-color=always  -I/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/src -I/home/xxx/anaconda3/envs/subdivnet/include/python3.7m -I/home/xxx/anaconda3/envs/subdivnet/include/python3.7m -DHAS_CUDA -I'/home/xxx/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/include' -I'/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/extern/cuda/inc'  -lstdc++ -ldl -shared  -x cu --cudart=shared -ccbin='/usr/bin/g++' --use_fast_math  -w  -I'/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/extern/cuda/inc'  -arch=compute_61  -code=sm_61  -o '/home/xxx/.cache/jittor/default/g++/jit/_opkey0:array_T:int32__JIT:1__JIT_cuda:1__index_t:int32___opkey1:broadcast_to_Tx:int32__DI...hash:bc2f95c82b48131a_op.so'
0:   0%|                    | 0/37 [00:02<?, ?it/s]
Traceback (most recent call last):
  File "train_seg.py", line 162, in <module>
    test(net, test_dataset, writer, 0, args)
  File "/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/__init__.py", line 257, in inner
    ret = func(*args, **kw)
  File "train_seg.py", line 64, in test
    preds = np.argmax(outputs.data, axis=1)
RuntimeError: Wrong inputs arguments, Please refer to examples(help(jt.data)).

Types of your inputs are:
 self   = Var,

The function declarations are:
 inline DataView data()

Failed reason:[f 0721 14:21:38.068584 96 parallel_compiler.cc:316] Error happend during compilation, see error above.

Any ideas on why this happens? I have downloaded the data of coseg-alien via the shell script provided.

Hi @unw9527 ,
Could you please upgrade jittor and clean the cache by rm -r ~/.cache/jittor ?
If there are still problems, please let me know.

Could you please specify which version of Jittor should I upgrade to? Mine is 1.2.3.71 currently.

Hi @lzhengning ,
Thanks for your reply. I did what you said above and now jittor's version is 1.2.3.73 (originally 1.2.3.71). And I clean the cache as well.
But it gives me another error message now, as follows.
nvcc fatal   : Value 'c++14' is not defined for option 'std'
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor_utils/__init__.py", line 152, in do_compile
    return cc.cache_compile(cmd, cache_path, jittor_path)
RuntimeError: [f 0721 21:17:00.171152 12 log.cc:387] Check failed ret(256) == 0(0) Run cmd failed: cd /home/xxx/.cache/jittor/default/g++ && /usr/local/cuda/bin/nvcc /home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/src/misc/nan_checker.cu      -std=c++14 -Xcompiler -fPIC  -Xcompiler -march=native  -Xcompiler -fdiagnostics-color=always  -I/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/src -I/home/xxx/anaconda3/envs/subdivnet/include/python3.7m -I/home/xxx/anaconda3/envs/subdivnet/include/python3.7m -DHAS_CUDA -I'/usr/local/cuda/include' -I'/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/extern/cuda/inc'  -I/home/xxx/.cache/jittor/default/g++  -O2  -x cu --cudart=shared -ccbin='/usr/bin/g++'   -w  -I'/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/extern/cuda/inc'  -c  -o /home/xxx/.cache/jittor/default/g++/obj_files/nan_checker.cu.o
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "train_seg.py", line 10, in <module>
    import jittor as jt
  File "/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/__init__.py", line 18, in <module>
    from . import compiler
  File "/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/compiler.py", line 1106, in <module>
    compile(cc_path, cc_flags+opt_flags, files, 'jittor_core'+extension_suffix)
  File "/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/compiler.py", line 93, in compile
    jit_utils.run_cmds(cmds, cache_path, jittor_path, "Compiling "+base_output)
  File "/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor_utils/__init__.py", line 193, in run_cmds
    for i,_ in enumerate(p.imap_unordered(do_compile, cmds)):
  File "/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/multiprocessing/pool.py", line 748, in next
    raise value
RuntimeError: [f 0721 21:17:00.171152 12 log.cc:387] Check failed ret(256) == 0(0) Run cmd failed: cd /home/xxx/.cache/jittor/default/g++ && /usr/local/cuda/bin/nvcc /home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/src/misc/nan_checker.cu      -std=c++14 -Xcompiler -fPIC  -Xcompiler -march=native  -Xcompiler -fdiagnostics-color=always  -I/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/src -I/home/xxx/anaconda3/envs/subdivnet/include/python3.7m -I/home/xxx/anaconda3/envs/subdivnet/include/python3.7m -DHAS_CUDA -I'/usr/local/cuda/include' -I'/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/extern/cuda/inc'  -I/home/xxx/.cache/jittor/default/g++  -O2  -x cu --cudart=shared -ccbin='/usr/bin/g++'   -w  -I'/home/xxx/anaconda3/envs/subdivnet/lib/python3.7/site-packages/jittor/extern/cuda/inc'  -c  -o /home/xxx/.cache/jittor/default/g++/obj_files/nan_checker.cu.o

I reproduced this error in the latest jittor. This seems to be a bug that was introduced recently, and will be fixed soon.
Can you try to install jittor by python3.7 -m pip install jittor==1.2.3.48? I have tested this version and it works.

It works. Thanks.

Closed because the latest jittor has fixed the bugs.

no! they are not fix this bug

repo: lzhengning/SubdivNet

license: MIT