Information Technology Grimoire

Version .0.0.1

IT Notes from various projects because I forget, and hopefully they help you too.

Storage

Storage Type Notes

I use ZFS for snapshots. Refer to table:

https://pve.proxmox.com/pve-docs/chapter-pvesm.html#_storage_types

DescriptionPlugin typeLevelSharedSnapshotsStable
ZFS (local)zfspoolbothnoyesyes
Directorydirfilenonoyes
BTRFSbtrfsfilenoyestechnology preview
NFSnfsfileyesnoyes
CIFScifsfileyesnoyes
Proxmox Backuppbsbothyesn/ayes
GlusterFSglusterfsfileyesnoyes
CephFScephfsfileyesyesyes
LVMlvmblocknonoyes
LVM-thinlvmthinblocknoyesyes
iSCSI/kerneliscsiblockyesnoyes
iSCSI/libiscsiiscsidirectblockyesnoyes
Ceph/RBDrbdblockyesyesyes
ZFS over iSCSIzfsblockyesyesyes

Several ways

When you first setup and install proxmox it will not automatically setup all of your disks

In my example, /dev/nvme0n1 is a nvme drive that I use as a boot, that is where proxmox was installed

The other devices are as follows:

/dev/sda/	SSD/EXT4
/dev/sdb/	SPINDLE/Unused
/dev/sdc/	SPINDLE/ZFS

the partitions are numbered: /dev/sdc1 is the main ZFS partition /dev/sdc9 is the ZFS overhead partition

Partitions are created and mounted in the fstab file.    We’ll go over mounting later.

ZFS RAID

ZFS is a great file structure system. To fully utilize it, you should have drives of the same size. I only have 1 drive in my example, and if I combine the 2TB with the 8TB, I will waste 6TB.

ZFS is a good file system, but might cause extra boot complexities, that is why my nvme does not have ZFS as it’s boot partition.

LVM

EXT4

GPT3

lsblk

lblk is a tool to viewing partitions from the CLI

lsblk 

root@pve128:~# lsblk
NAME               MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                  8:0    0 465.8G  0 disk
├─sda1               8:1    0   100M  0 part
└─sda2               8:2    0 465.7G  0 part
sdb                  8:16   0   1.8T  0 disk
└─sdb1               8:17   0   1.8T  0 part
sdc                  8:32   0   5.5T  0 disk
└─sdc4               8:36   0   5.5T  0 part
sr0                 11:0    1  1024M  0 rom
nvme0n1            259:0    0 465.8G  0 disk
├─nvme0n1p1        259:1    0  1007K  0 part
├─nvme0n1p2        259:2    0   512M  0 part /boot/efi
└─nvme0n1p3        259:3    0 465.3G  0 part
  ├─pve-swap       253:0    0     8G  0 lvm  [SWAP]
  ├─pve-root       253:1    0    96G  0 lvm  /
  ├─pve-data_tmeta 253:2    0   3.5G  0 lvm
  │ └─pve-data     253:4    0 338.4G  0 lvm
  └─pve-data_tdata 253:3    0 338.4G  0 lvm
    └─pve-data     253:4    0 338.4G  0 lvm

parted

parted is a tool for making various partition sizes

apt install parted
apt policy parted
parted /dev/sdc mklabel gpt

install parted

root@pve128:~# apt install parted
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libparted2
Suggested packages:
  libparted-dev libparted-i18n parted-doc
The following NEW packages will be installed:
  libparted2 parted
0 upgraded, 2 newly installed, 0 to remove and 6 not upgraded.
Need to get 554 kB of archives.
After this operation, 935 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ftp.us.debian.org/debian bullseye/main amd64 libparted2 amd64 3.4-1 [335 kB]
Get:2 http://ftp.us.debian.org/debian bullseye/main amd64 parted amd64 3.4-1 [219 kB]
Fetched 554 kB in 0s (1,574 kB/s)
Selecting previously unselected package libparted2:amd64.
(Reading database ... 43876 files and directories currently installed.)
Preparing to unpack .../libparted2_3.4-1_amd64.deb ...
Unpacking libparted2:amd64 (3.4-1) ...
Selecting previously unselected package parted.
Preparing to unpack .../parted_3.4-1_amd64.deb ...
Unpacking parted (3.4-1) ...
Setting up libparted2:amd64 (3.4-1) ...
Setting up parted (3.4-1) ...
Processing triggers for libc-bin (2.31-13+deb11u5) ...
Processing triggers for man-db (2.9.4-2) ...

View Policy/Version

root@pve128:~# apt policy parted
parted:
  Installed: (none)
  Candidate: 3.4-1
  Version table:
     3.4-1 500
        500 http://ftp.us.debian.org/debian bullseye/main amd64 Packages

make partition with parted

There are many ways, but here is an example of making a gpt partition on /dev/sdc. I decided later to delete this partition structure and use ZFS, but here is how to make a gpt structure

root@pve128:~# parted /dev/sdc mklabel gpt
Warning: The existing disk label on /dev/sdc will be destroyed and all data on this disk will be lost. Do you want to continue?
Yes/No? yes
Information: You may need to update /etc/fstab.

list drive ID

lsblk 

root@pve128:~# lsblk
NAME               MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                  8:0    0 465.8G  0 disk
├─sda1               8:1    0   100M  0 part
└─sda2               8:2    0 465.7G  0 part
sdb                  8:16   0   1.8T  0 disk
└─sdb1               8:17   0   1.8T  0 part
sdc                  8:32   0   5.5T  0 disk
└─sdc4               8:36   0   5.5T  0 part
sr0                 11:0    1  1024M  0 rom
nvme0n1            259:0    0 465.8G  0 disk
├─nvme0n1p1        259:1    0  1007K  0 part
├─nvme0n1p2        259:2    0   512M  0 part /boot/efi
└─nvme0n1p3        259:3    0 465.3G  0 part
  ├─pve-swap       253:0    0     8G  0 lvm  [SWAP]
  ├─pve-root       253:1    0    96G  0 lvm  /
  ├─pve-data_tmeta 253:2    0   3.5G  0 lvm
  │ └─pve-data     253:4    0 338.4G  0 lvm
  └─pve-data_tdata 253:3    0 338.4G  0 lvm
    └─pve-data     253:4    0 338.4G  0 lvm

Make Partition and Format EXT4

Make primary partition with ext4 and use 100%

parted -a opt /dev/sdc mkpart primary ext4 0% 100%

create filesystem

Make ext4 filesystem on the drive

mkfs.ext4 -L storage6T /dev/sdc1 

root@pve128:~# mkfs.ext4 -L storage6T /dev/sdc1
mke2fs 1.46.5 (30-Dec-2021)
/dev/sdc1 contains a VMFS_volume_member file system
Proceed anyway? (y,N) y
Creating filesystem with 1465130240 4k blocks and 183144448 inodes
Filesystem UUID: 3602de26-ec39-421c-be98-d54dd92f2945
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done

mount

mkdir -p /mnt/storage6T

Mount on boot

vim /etc/fstab

add this line to automount

LABEL=storage6T /mnt/storage6T ext4 defaults 0 2

test mount now

mount -a

lsblk to verify

lsblk
NAME               MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                  8:0    0 465.8G  0 disk
├─sda1               8:1    0   100M  0 part
└─sda2               8:2    0 465.7G  0 part
sdb                  8:16   0   1.8T  0 disk
└─sdb1               8:17   0   1.8T  0 part
sdc                  8:32   0   5.5T  0 disk
└─sdc1               8:33   0   5.5T  0 part /mnt/storage6T
sr0                 11:0    1  1024M  0 rom
nvme0n1            259:0    0 465.8G  0 disk
├─nvme0n1p1        259:1    0  1007K  0 part
├─nvme0n1p2        259:2    0   512M  0 part /boot/efi
└─nvme0n1p3        259:3    0 465.3G  0 part
  ├─pve-swap       253:0    0     8G  0 lvm  [SWAP]
  ├─pve-root       253:1    0    96G  0 lvm  /
  ├─pve-data_tmeta 253:2    0   3.5G  0 lvm
  │ └─pve-data     253:4    0 338.4G  0 lvm
  └─pve-data_tdata 253:3    0 338.4G  0 lvm
    └─pve-data     253:4    0 338.4G  0 lvm

make a backup folder

mkdir /mnt/storage2/vm_backup