Sometimes you want to test if changes you are about to make to a CRUSH map will cause data to move or not.

In this case I wanted to change a rule in CRUSH where it would use device classes, but I didn’t want any of the ~1PB of data in that cluster to move.

By swapping IDs I could prevent data to move:

root default {
     id -50      # do not change unnecessarily
     id -53 class hdd        # do not change unnecessarily
     id -122 class ssd       # do not change unnecessarily

root default {
     id -53      # do not change unnecessarily
     id -50 class hdd        # do not change unnecessarily
     id -122 class ssd       # do not change unnecessarily

Notice how I swapped the IDs. After this I updated the rule:

rule rgw {
         id 6
         type replicated
         min_size 1
         max_size 10
         step take ams02-objects class hdd
         step chooseleaf firstn 0 type host
         step emit
 }

I then compiled the CRUSHMap and ran crushtool to see if there were any differences:

root@mon01:~# crushtool -i crushmap --compare crushmap.new 
 rule 0 had 0/10240 mismatched mappings (0)
 rule 1 had 0/10240 mismatched mappings (0)
 rule 2 had 0/10240 mismatched mappings (0)
 rule 3 had 0/10240 mismatched mappings (0)
 rule 4 had 0/10240 mismatched mappings (0)
 rule 5 had 0/3072 mismatched mappings (0)
 rule 6 had 0/10240 mismatched mappings (0)
 maps appear equivalent
 root@mon01:~#

No changes! So it was safe to inject this map:

root@mon01:~# ceph osd setcrushmap -i crushmap.new

42on helps you with all kinds of Ceph stuff like: Ceph latency, Ceph benchmark, Ceph optimization, Ceph iops, Ceph troubleshooting, Ceph architecture, Ceph consistency, Ceph performance, Ceph stability, Ceph redundancy, Ceph RBD, Ceph deploy, Ceph management, Ceph ipv6, rados gateway, Ceph rgw, Ceph monitor, Ceph OSD, Ceph libv