ó
oBś]c @ sö d Z d d l m Z d d l m Z d d l Z d d l Z d d l Z e Z e j d Z
e j d Z e j d Z e j d Z
e j d Z e j d
Z e j d Z i d d
6Z e j e Z d Z d Z d Z d d Z e d Z d Z e d Z d Z d Z d d d e d d Z! d Z" d Z# d Z$ d Z% d Z& d Z' d Z( d Z) d Z* d! Z+ d" Z, d# Z- d$ Z. d% Z/ d& Z0 d' Z1 d( Z2 d) Z3 d* Z4 d S(+ sć
Disk Setup
----------
**Summary:** configure partitions and filesystems
This module is able to configure simple partition tables and filesystems.
.. note::
for more detail about configuration options for disk setup, see the disk
setup example
For convenience, aliases can be specified for disks using the
``device_aliases`` config key, which takes a dictionary of alias: path
mappings. There are automatic aliases for ``swap`` and ``ephemeral<X>``, where
``swap`` will always refer to the active swap partition and ``ephemeral<X>``
will refer to the block device of the ephemeral image.
Disk partitioning is done using the ``disk_setup`` directive. This config
directive accepts a dictionary where each key is either a path to a block
device or an alias specified in ``device_aliases``, and each value is the
configuration options for the device. The ``table_type`` option specifies the
partition table type, either ``mbr`` or ``gpt``. The ``layout`` option
specifies how partitions on the device are to be arranged. If ``layout`` is set
to ``true``, a single partition using all the space on the device will be
created. If set to ``false``, no partitions will be created. Partitions can be
specified by providing a list to ``layout``, where each entry in the list is
either a size or a list containing a size and the numerical value for a
partition type. The size for partitions is specified in **percentage** of disk
space, not in bytes (e.g. a size of 33 would take up 1/3 of the disk space).
The ``overwrite`` option controls whether this module tries to be safe about
writing partition talbes or not. If ``overwrite: false`` is set, the device
will be checked for a partition table and for a file system and if either is
found, the operation will be skipped. If ``overwrite: true`` is set, no checks
will be performed.
.. note::
Using ``overwrite: true`` is dangerous and can lead to data loss, so double
check that the correct device has been specified if using this option.
File system configuration is done using the ``fs_setup`` directive. This config
directive accepts a list of filesystem configs. The device to create the
filesystem on may be specified either as a path or as an alias in the format
``<alias name>.<y>`` where ``<y>`` denotes the partition number on the device.
The partition can also be specified by setting ``partition`` to the desired
partition number. The ``partition`` option may also be set to ``auto``, in
which this module will search for the existance of a filesystem matching the
``label``, ``type`` and ``device`` of the ``fs_setup`` entry and will skip
creating the filesystem if one is found. The ``partition`` option may also be
set to ``any``, in which case any file system that matches ``type`` and
``device`` will cause this module to skip filesystem creation for the
``fs_setup`` entry, regardless of ``label`` matching or not. To write a
filesystem directly to a device, use ``partition: none``. A label can be
specified for the filesystem using ``label``, and the filesystem type can be
specified using ``filesystem``.
.. note::
If specifying device using the ``<device name>.<partition number>`` format,
the value of ``partition`` will be overwritten.
.. note::
Using ``overwrite: true`` for filesystems is dangerous and can lead to data
loss, so double check the entry in ``fs_setup``.
.. note::
``replace_fs`` is ignored unless ``partition`` is ``auto`` or ``any``.
**Internal name:** ``cc_disk_setup``
**Module frequency:** per instance
**Supported distros:** all
**Config keys**::
device_aliases:
<alias name>: <device path>
disk_setup:
<alias name/path>:
table_type: <'mbr'/'gpt'>
layout:
- [33,82]
- 66
overwrite: <true/false>
fs_setup:
- label: <label>
filesystem: <filesystem type>
device: <device>
partition: <"auto"/"any"/"none"/<partition number>>
overwrite: <true/false>
replace_fs: <filesystem type>
i’’’’( t PER_INSTANCE( t utilNt udevadmt sfdiskt sgdiskt lsblkt blkidt blockdevt wipefst Ct LANGc C sģ | j d } t | t rō t | | j | j d t | x | j D] \ } } t | t s | j d | qQ n y@ | j d t j
d t j d d | d t d | | f WqQ t
k
rģ } t j t d
| qQ XqQ Wn | j d } t | t rč| j d t | t | | j x | D]¢ } t | t sk| j d
| q?n yL | j d | j d }
t j
d t j d d |
d t d | f Wq?t
k
rą} t j t d | q?Xq?Wn d S( s[
See doc/examples/cloud-config-disk-setup.txt for documentation on the
format.
t
disk_setups Partitioning disks: %ss Invalid disk definition for %ss! Creating new partition table/diskt logfunct msgs Creating partition on %st funct argss Failed partitioning operation
%st fs_setups setting up filesystems: %ss" Invalid file system definition: %ss Creating new filesystem.t devices Creating fs for %ss% Failed during filesystem operation
%sN( t gett
isinstancet dictt update_disk_setup_devicest device_name_to_devicet debugt strt itemst warningR t log_timet LOGt mkpartt Exceptiont logexct listt update_fs_setup_devicest mkfs( t _namet cfgt cloudt logt _argsR t diskt
definitiont eR R ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt handlez s>
c C s¤ x | j D] } | | } | d k s
| | k r= q
n | | k rf t j d | | | | =n | | | | <| | | d <| | =t j d | | q
Wd S( Ns0 Replacing %s in disk_setup for translation of %st _orignames, updated disk_setup device entry '%s' to '%s'( t keyst NoneR t infoR ( R t tformert orignamet transformed( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyR £ s
c C s xś | D]ņ } t | t s2 t j d | q n | j d } | d k rS q n t j | \ } } | | } | d k r³ | } t j d | | | | | d <| | d <n | r d | k rģ t j d | | | | d | d <n | | d <q q Wd S( Ns" entry in disk_setup not a dict: %sR s %s is mapped to disk=%s part=%sR, t partitionsK Partition '%s' from dotted device name '%s' overrides 'partition' key in %st
_partition( R R R R R R. R t expand_dotted_devnameR ( R R0 R) R1 t devt partt tformed( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyR! ¶ s,
c c sd t j | } | r" | | } n x; g | D] } | j d ^ q, D] \ } } | | f VqE Wd S( sd
Returns the key/value pairs of output sent as string
like: FOO='BAR' HOME='127.0.0.1'
t =N( t shlext split( t valuest startt _valuest xt keyt value( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt value_splitter× s
/c c s t d d d | g } | r+ | j d n d } y t j | \ } } Wn) t k
ru } t d | | f n Xg | j j D]$ } t | j d k r | ^ q } xb | D]Z } i d d 6d d 6d d 6d d
6} x* t
| D] \ }
} | | |
j <qļ W| Vqŗ Wd S( s
Enumerate the elements of a child device.
Parameters:
device: the kernel device name
nodeps <BOOL>: don't enumerate children devices
Return a dict describing the disk:
type: the entry type, i.e disk or part
fstype: the filesystem type, if it exists
label: file system label, if it exists
name: the device name, i.e. sda
s --pairss --outputs NAME,TYPE,FSTYPE,LABELs --nodepss" Failed during disk check for %s
%si t namet typet fstypet labelN( t LSBLK_CMDt appendR. R t subpR t stript
splitlinest lenR; RB t lower( R t nodepst lsblk_cmdR/ t _errR* R? t partsR7 t dR@ RA ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt enumerate_diskä s& =
c C s; x4 t | d t D] } d | k r | d j Sq Wd S( s@
Return the device type of the device by calling lsblk.
RN RD N( RS t TrueRM R. ( R RR ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt device_type s c C sl d } y t | } Wn" t k
r: t j d | t SX| rQ | d k rQ t S| rh | d k rh t St S( s0
Check if the device is a valid device.
t s Query against device %s failedR7 R( ( RU R R R t FalseRT ( RC R3 t d_type( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt is_device_valid s
c
C s d
\ } } } } t d d | g } y% t j | d d d g \ } } Wn) t k
rt } t d | | f n X| r
t | j d k r
xt t | d d D]] \ } } | j d k rĶ | } q¦ | j d
k rč | } q¦ | j d k r¦ | } q¦ q¦ Wq
n | | | f S( sĻ
Check if the device has a filesystem on it
Output of blkid is generally something like:
/dev/sda: LABEL="Backup500G" UUID="..." TYPE="ext4"
Return values are device, label, type, uuid
s -cs /dev/nullt rcsi i s" Failed during disk check for %s
%si R= RF RD t uuidN( NNNN( R. t BLKID_CMDR RI R RL RK RB RM (
R t outRF t fs_typeR[ t blkid_cmdRP R* R@ RA ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt check_fs, s % c C s t | \ } } } | S( s7
Returns true if the device has a file system.
( R` ( R t _R^ ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt
is_filesystemJ s c C s@ | d
k r d } n | s* d d g } n t } xę t | D]Ų } | d | k rq | t k rq d | d t f S| d | k r° | r | d | k s | r° d | d t f S| d | k r= | d d k sŚ | d rć t } n | d d k rö q| d sd | d t f Sq= q= W| s)| t f St j d d
t f S( s
Find a device that is either matches the spec, or the first
The return is value is (<device>, <bool>) where the device is the
device to use and the bool is whether the device matches the
fs_type and label.
Note: This works with GPT partition tables!
RV R( R7 RE s /dev/%sRC RF RD s5 Failed to find device during available device search.N( R. RW RS RT R R ( R R^ RF t
valid_targetst label_matcht
replace_fst raw_device_usedRR ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt find_device_nodeR s,
c C sE t t t | d k r" t St | \ } } } | rA t St S( s„
Check if the device is currently used. Returns true if the device
has either a file system or a partition entry
is no filesystem found on the disk.
i ( RL R RS RT R` RW ( R Ra t check_fstype( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt is_disk_used s c G s t | d k r! t d n | d | d } | d } y& | rW t | | St | SWn! t k
r t d | n Xd S( s-
Call the appropriate function.
The first value is the template for function name
The second value is the template replacement
The remain values are passed to the function
For example: get_dyn_func("foo_%s", 'bar', 1, 2, 3,)
would call "foo_bar" with args of 1, 2, 3
i s* Unable to determine dynamic funcation namei i s No such function %s to call!N( RL R t globalst KeyError( R t func_namet func_args( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt get_dyn_func s
c C s y@ t j t d | g \ } } t j t d | g \ } } Wn) t k
rk } t d | | f n Xt | t | S( Ns --getsize64s --getsss Failed to get %s size
%s( R RI t
BLKDEV_CMDR t int( R t
size_in_bytesRa t sector_sizeR* ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt get_hdd_size² s "c C sI t | t d | g } y# t j | d d | \ } } Wn) t k
rg } t d | | f n Xg } xŌ | j D]Ę } | j } t | d k r„ q{ n | | d k r{ | d j d k rŃ q{ n d
} xW t
t d t | d
t D]4 }
| |
j
rł | |
d k rł | |
} Pqł qł W| j | q{ q{ W| S( sŅ
Returns true if the partition layout matches the one on the disk
Layout should be a list of values. At this time, this only
verifies that the number of partitions and their labels is correct.
s -lt datas %s
s( Error running partition command on %s
%si i’’’’t extendedt emptyi t reverset /( Ru Rv N( t read_parttblt
SFDISK_CMDR RI R RK R; RL RM R. t sortedt rangeRT t isdigitRH ( R t layoutt prt_cmdR] RP R* t found_layoutt linet _linet
type_labelR? ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt check_partition_mbr_layout¼ s,
# (
c C s t d | g } y t j | d t \ } } Wn) t k
rY } t d | | f n Xt | j } x' | D] } | j j d rs Pqs qs Wg | D] } | j j d ^ q } g } xL | D]D }
t
|
d k r|
j d r|
d d !}
n | j |
qĢ W| S(
Ns -pt
update_envs( Error running partition command on %s
%st Numberi i t 00i i (
t
SGDISK_CMDR RI t
LANG_C_ENVR t iterRK RJ t
startswithR; RL t endswithRH ( R R~ R R] RP R* t out_linesR t codest cleanedt code( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt check_partition_gpt_layoutā s"
)
!c C s t d | | | } t j d | | | | t | t r] | rY t | d k rY t St St | t | k rg | D]1 } t | t t f r§ t
| d n d ^ q| } t j d | | xE t | | D]4 \ } } | d k rÖ t
| t
| k rÖ t SqÖ Wt St S( sĪ
See if the partition lay out matches.
This is future a future proofing function. In order
to add support for other disk layout schemes, add a
function called check_partition_%s_layout
s check_partition_%s_layouts6 called check_partition_%s_layout(%s, %s), returned: %si s Layout types=%s. Found types=%sN(
Rn R R R t boolRL RT RW t tupleR R R. t zip( t
table_typeR R~ R R? t layout_typest itypet ftype( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt check_partition_layout
s$ ;
$c
C s t | t r# t | t r# d St | d k rD t | t sT t | t rc t d n t | } | d k r t d n g } d } x¾ | D]¶ } d } | } | d 7} t | t rü t | d k rķ t d | n | \ } } n t t | t | d
} | | k r<| j d | q | j d | | f q Wd
j | } t | d k rt d | n | S( s@
Calculate the layout of the partition table. Partition sizes
are defined as percentage values or a tuple of percentage and
partition type.
For example:
[ 33, [66: 82] ]
Defines the first partition to be a size of 1/3 the disk,
while the remaining 2/3's will be of type Linux Swap.
s 0,i s Partition layout is invalidi s$ Only simply partitioning is allowed.iS i i s% Partition was incorrectly defined: %sid s ,,%ss ,%s,%ss
s- Calculated partition definition is too big
%s( R R R RL R Rp t floatRH t join(
t sizeR~ t
last_part_numt part_definitiont part_numR7 t part_typet percentt part_sizet sfdisk_definition( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt get_partition_mbr_layout- s6
!
c C są t | t r" d d d g f g Sg } x | D] } t | t rx t | d k ri t d | n | \ } } n | } d } t t | t | d } | j | d d j | g f q/ Wd | d d d <| S( Ni i s% Partition was incorrectly defined: %sid s +{}i’’’’(
R R R. R RL R Rp R RH t format( R R~ t partition_specsR3 R” t partition_typeR¢ ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt get_partition_gpt_layouta s
&c C sx d } d } d } t | d G } | j | | | j | t j | j | | | j Wd QXt | d S( Nt i s rb+i i ( t opent writet seekt ost SEEK_ENDt flushRy ( R t nullt start_lent end_lent fp( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt purge_disk_ptablex s c C s x t | D]} } | d d
k r
t d d | d g } y% t j d | d t j | Wq t k
r t d | d q Xq
q
Wt | d S( s'
Remove parition table entries
RD R( t crypts --alls /dev/%sRC s Purging filesystem on /dev/%ss Failed FS purge of /dev/%sN( s diskRµ ( RS t
WIPEFS_CMDR R/ R RI R R“ ( R RR t
wipefs_cmd( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt
purge_disk s
c C s t d | | | S( sļ
Call the appropriate function for creating the table
definition. Returns the table definition
This is a future proofing function. To add support for
other layouts, simply add a "get_partition_%s_layout"
function.
s get_partition_%s_layout( Rn ( R R R~ ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt get_partition_layout s c C sb t d | g } t j y t j | Wn' t k
rS } t j t d | n Xt j d S( sr
Use partprobe instead of 'udevadm'. Partprobe is the only
reliable way to probe the partition table.
s
--rereadpts% Failed reading the partition table %sN( Ro R t udevadm_settleRI R R R ( R t
blkdev_cmdR* ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyRy „ s
c C sj t d d d | g } y t j | d d | Wn) t k
r[ } t d | | f n Xt | d S( sV
Break out of mbr partition to allow for future partition
types, i.e. gpt
s --Linuxs --unit=Ss --forceRt s %s
s Failed to partition device %s
%sN( Rz R RI R Ry ( R R~ R R* ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt exec_mkpart_mbr“ s c C sö yĮ t j t d | g x¤ t | D] \ } \ } \ } } | d 7} t j t d d j | | | | g | d k r& t | j d d } t j t d d j | | | g q& q& WWn$ t k
rē t j
d | n Xt | d S(
Ns -Zi s -ns {}:{}:{}i t 0s -ts {}:{}s Failed to partition device %s( R RI R t enumerateR„ R. R t ljustR R R Ry ( R R~ t indexR§ R= t endt pinput( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt exec_mkpart_gptĆ s %
*
c C s t d | | | S( s
Fetches the function for creating the table type.
This allows to dynamically find which function to call.
Paramaters:
table_type: type of partition table to use
device: the device to work on
layout: layout definition specific to partition table
s exec_mkpart_%s( Rn ( R R R~ ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt exec_mkpart× s
c C sR t j j | sD t j t j j | sD t d | qD n t j d S( s? Assert that device exists and settle so it is fully recognized.sB Device %s did not exist and was not created with a udevadm settle.N( R t patht existsR Rŗ t RuntimeError( R ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt assert_and_settle_deviceä s
c C s’ t | t j j | } t j d | | j d t } | j d t } | j d d } t j d t | t r | s | r t j d d St j d | t
| sŌ t d
j d | n t | t
r| j d k rt j d
t | d St j d t | | | r@t j d t St j d | rt | slt | rt j d | d St j d | t | } t j d t | | | } t j d | t j d | t | | | t j d | d S( sń
Creates the partition table.
Parameters:
definition: dictionary describing how to create the partition.
The following are supported values in the dict:
overwrite: Should the partition table be created regardless
of any pre-exisiting data?
layout: the layout of the partition table
table_type: Which partition table to use, defaults to MBR
device: the device to work on.
s! Checking values for %s definitiont overwriteR~ R t mbrs Checking against default devicess) Device is not to be partitioned, skippingNs' Checking if device %s is a valid devices% Device {device} is not a disk device!R t removes, Instructed to remove partition table entriess! Checking if device layout matchess" Device partitioning layout matchess' Checking if device is safe to partitions- Skipping partitioning on configured device %ss Checking for device size of %ss Calculating partition layouts Layout is: %ss Creating partition table on %ss Partition table created for %s( RČ R RÅ t realpathR R R RW R R RY R R„ R RM Rø R RT Ri Rb Rs R¹ RÄ ( R R) RÉ R~ R t device_sizeR ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyR ņ sD
!
c C sk i d d 6d d 6d d 6d d 6} d | j k r= d } n | j | k rW | | St j d | d S( s9
A force flag might be -F or -F, this look it up
s -Ft exts -ft btrfst xfst reiserfss Force flag for %s is unknown.RV ( RM R R ( t fst flags( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt lookup_force_flag3 s
c C s¹ | j d } | j d } t | j d d } | j d } | j d g } | j d g } | j d t } | j d t } t | t j j | } t j d
| | sĢ | j r”| j rž d | | f } t j d | | n t j d
| t
| \ } }
} t j d | | |
| | k r|
| k rt j d | | s{t j d | d St j d | qŠt j d | n/| rt | j d- k r| } t j d | t
}
| j d k röt }
n t | d | d | d |
d | \ } } t j d | | | rJt j d d S| rp| rp| rpt j d | n | sŠt j d | d SnC | s¬t | j d k ræt j d | n t j d d St j d | | | | s t j d | d S| p | s't d! j d | n t } | r| d i | d 6| d 6| d 6} t
} | rut j d" | n | rIt j d# | qInø t j d$ | } | sĄt j d% | } n | sŻt j d& | | d S| | g } | r| j d' | g n | s t | d( k r3| j t | n | rI| j | n t j d) | | t j d* t | y t j | d+ | Wn) t k
r“} t d, | | f n Xd S(. s¦
Create a file system on the device.
label: defines the label to use on the device
fs_cfg: defines how the filesystem is to look
The following values are required generally:
device: which device or cloud defined default_device
filesystem: which file system type
overwrite: indiscriminately create the file system
partition: when device does not define a partition,
setting this to a number will mean
device + partition. When set to 'auto', the
first free device or the first device which
matches both label and type will be used.
'any' means the first filesystem that matches
on the device.
When 'cmd' is provided then no other parameter is required.
RF R R3 t anyt
filesystemt cmdt
extra_optsRe RÉ s# Checking %s against default devicess %s%ss% Manual request of partition %s for %ss Checking device %ss0 Device '%s' has check_label='%s' check_fstype=%ss Existing file system found at %ss" Device %s has required file systemNs Destroying filesystem on %ss" Device %s is cleared for formatingt autos, Identifying device to create %s filesytem onR^ Rd s( Automatic device for %s identified as %ss+ Found filesystem match, skipping formating.s* Replacing file system on %s as instructed.sC No device aviable that matches request. Skipping fs creation for %st nones. Using the raw device to place filesystem %s ons( Error in device identification handling.s; File system type '%s' with label '%s' will be created on %ss Device is not known: %ssE No way to create filesystem '{label}'. fs_type or fs_cmd must be set.s8 fs_setup:overwrite ignored because cmd was specified: %ss9 fs_setup:extra_opts ignored because cmd was specified: %ss mkfs.%ss mk%ss. Cannot create fstype '%s'. No mkfs.%s commands -LR( s Creating file system %s on %ss Using cmd: %st shells Failed to exec of '%s':
%s( RŁ s any( R R RW RČ R RÅ RĢ R R R} R` R RM RT Rg R R„ R t whicht extendRU RH RŌ RI ( t fs_cfgRF R R3 R^ t fs_cmdt fs_optst
fs_replaceRÉ t check_labelRh Ra t odeviceRd t reuseRŪ t mkfs_cmdR* ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyR" H s¼
(5 t __doc__t cloudinit.settingsR t cloudinitR t loggingR R: t frequencyRÜ t UDEVADM_CMDRz R RG R\ Ro R¶ R t getLoggert __name__R R+ R R! R. RB RW RS RU RY R` Rb RT Rg Ri Rn Rs R R R R¤ RØ R“ Rø R¹ Ry R¼ RĆ RÄ RČ R RŌ R" ( ( ( sB /usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.pyt <module>b sZ
) !
+ 0
& ( # 4
A