Grid’5000 Admin Tutorial¶

Note

WARNING: This document is deprecated.

The tutorial will focus on the three following activities:

Create and customize G5k appliance as a simple end-user
Manage a recipe repository
Share recipes with users

Introduction: What is Kameleon?¶

Kameleon Architecture¶

Kameleon can be seen as a shell sequencer which will boost your shell scripts. It is based on the execution of shell scripts but it also provides some syntax sugar that makes working with shell scripts less painful.

Kameleon recipe hierarchy¶

There are two ways you can start working with Kameleon. First, you can manually write recipes from scratch. Secondly, you can build recipes from existing models. We are going to focus on the latter in this tutorial. We are going to see how to create new recipes and share them.

For the purposes of this tutorial, I have already created a Grid5000 environment, which you can download on the Grenoble site.

Note

$ oarsub -I -t deploy
$ kadeploy3 -a /home/sharrache/public/kameleon-demo-env_img.yaml -f $OAR_NODEFILE

Building Debian based appliance as a simple end-user¶

Kameleon is delivered without any template by default:

$ kameleon template list

To begin, a recipe repository has to be added:

$ kameleon repository add tutorial https://github.com/oar-team/kameleon-g5k-tutorial.git
$ kameleon template list

Now, you should see the template list prefixed by the repository name, called “tutorial”.

To build a Debian 7 image, it is possible to choose from several virtualization motors: chroot, qemu, virtualbox

In this tutorial, we are going to choose chroot, but you can also choose virtualbox or qemu. Let’s import the debian7 template in our workspace:

$ mkdir my_recipes && cd my_recipes  ## create a workspace
$ kameleon new my_debian7 tutorial/debian7/chroot

This will generate the following files in the current directory:

create  tutorial/debian7/chroot.yaml
create  tutorial/debian7/base.yaml
create  tutorial/steps/aliases/defaults.yaml
create  tutorial/steps/checkpoints/qcow2.yaml
create  tutorial/steps/bootstrap/initialize_disk_chroot.yaml
create  tutorial/steps/enable_checkpoint.yaml
create  tutorial/steps/bootstrap/debian/debootstrap.yaml
create  tutorial/steps/bootstrap/start_chroot.yaml
create  tutorial/steps/setup/debian/configure_apt.yaml
create  tutorial/steps/setup/debian/upgrade_system.yaml
create  tutorial/steps/setup/debian/install_software.yaml
create  tutorial/steps/setup/debian/configure_kernel.yaml
create  tutorial/steps/setup/debian/install_bootloader.yaml
create  tutorial/steps/setup/debian/configure_system.yaml
create  tutorial/steps/setup/debian/configure_keyboard.yaml
create  tutorial/steps/setup/debian/configure_network.yaml
create  tutorial/steps/setup/create_group.yaml
create  tutorial/steps/setup/create_user.yaml
create  tutorial/steps/disable_checkpoint.yaml
create  tutorial/steps/export/qemu_save_appliance.yaml
create  my_debian7.yaml

We have thus the following recipes in our workspace:

$ kameleon list
NAME                    | DESCRIPTION
------------------------|------------------------------------------------------------
my_debian7              | <MY RECIPE DESCRIPTION>
tutorial/debian7/base   | Base template for Debian appliance.
tutorial/debian7/chroot | Debian 7 (Wheezy) appliance built with chroot and qemu-nbd.

The new recipe my_debian7.yaml inherits the base recipe tutorial/debian7/chroot.yaml as we can see in the my_debian7.yaml file with the keyword extend

 #==============================================================================
 # vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
 #==============================================================================
 #
 # DESCRIPTION: <MY RECIPE DESCRIPTION>
 #
 #==============================================================================

 ---
 extend: tutorial/debian7/chroot.yaml

 global:
     # You can see the base template `tutorial/debian7/chroot.yaml` to know the
     # variables that you can override

 bootstrap:
   - "@base"

 setup:
   - "@base"

 export:
   - "@base"

This recipe acts exactly as the parent recipe thanks to the keyword “@base”. (see Inheritance)

Let’s start building the appliance:

$ kameleon build my_debian7.yaml --enable-cache

We enable caching for all network data that will be used to build the appliance. Thanks to this, the recipe reconstructability is ensured.

The process will start and take some time…

While waiting, it is possible to analyze all steps and variables contained the parent recipe tutorial/debian7/chroot.yaml.

 export:
   - disable_checkpoint
   - qemu_save_appliance:
     - input: $(readlink $${image_disk})
     - output: $${kameleon_cwd}/$${kameleon_recipe_name}
     - save_as_qcow2
     - save_as_tar_gz
     # - save_as_qed
     # - save_as_tar_xz
     # - save_as_tar_bz2
     # - save_as_raw
     # - save_as_vmdk
     # - save_as_vdi

When Kameleon ends, a directory called build will be generated in the current directory. You will have a debian wheezy appliance in a different format, that you can try out by executing:

$ qemu-system-x86_64 -enable-kvm -m 512 -curses build/my_debian7/my_debian7.qcow2

Creating a Grid’5000 environment¶

Now, let’s customize our recipe to create a Grid’5000 environment. With this step, we will see how the code can be re-used with Kameleon. Therefore, we can extend the recipe created earlier.

 export:
   - disable_checkpoint
   - qemu_save_appliance:
     - input: $(readlink $${image_disk})
     - output: $${kameleon_cwd}/$${kameleon_recipe_name}
     - save_as_tar_gz

   - g5k_custom:
     - kadeploy_file:
       - write_local:
         - $${kameleon_cwd}/$${kameleon_recipe_name}.yaml
         - |
           #
           # Kameleon generated based on kadeploy description file
           #
           ---
           name: $${kameleon_recipe_name}

           version: 1

           os: linux

           image:
             file: $${kameleon_recipe_name}.tar.gz
             kind: tar
             compression: gzip

           postinstalls:
             - archive: server:///grid5000/postinstalls/debian-x64-base-2.5-post.tgz
               compression: gzip
               script: traitement.ash /rambin

           boot:
             kernel: /vmlinuz
             initrd: /initrd.img

           filesystem: $${filesystem_type}

This recipe will generate in the build directory a tar.gz image and a configuration file for Kadeploy. We have to copy them in a Grid’5000 site for instance (Grenoble) by doing:

$ scp my_debian7.tar.gz my_debian7.yaml grenoble.g5k:~/

Therefore, if we log in the respective site, we can then submit a deploy job and deploy the image using kadeploy:

user@fgrenoble:~$ oarsub -I t deploy
[ADMISSION RULE] Set default walltime to 3600.
[ADMISSION RULE] Modify resource description with type constraints
Generate a job key...
OAR_JOB_ID=1663465
Interactive mode : waiting...
Starting...

Connect to OAR job 1663465 via the node fgrenoble.grenoble.grid5000.fr

user@fgrenoble:~$ kadeploy -a my_debian7.yaml -f $OAR_NODEFILE

The image will be deployed on baremetal after a few minutes…

Exercice¶

In the previous recipe my_debian.yaml, find the change to make in order to build a 32-bit appliance (and not a 64-bit one). Do no hesitate to look at the parent recipes.
Add a new grid5000 user and add it to the sudo group.
Introduce an error in the recipe execution and resolve it in Kameleon interactive mode

e.g.

exec_local: cat /non_existing_file

Try to build a debian jessie
Create a git repository and try to share it with your neighbour.