Page_c76a3fcf

captain — Sat, 06 Jul 2019 23:42:35 +0000

pyenv — утилита, которая позволяет легко переключаться между несколькими версиями Python которые установлены на хосте. Это очень полезно при разработке ПО, когда имеется несколько разных верий и их нужно поддерживать.

Установка pyenv в Unix/Linux

pyenv — замечательный инструмент для управления несколькими версиями Python. Даже если у вас уже установлен Python в вашей системе, стоит установить pyenv, чтобы вы могли легко опробовать новые языковые функции или помочь внести свой вклад в проект, который находится на другой версии Python.

Рекомендуемое чтиво:

Установка virtualenv + virtualenvwrapper в Unix/Linux

И так, начнем установку…

Установка pyenv в CentOS/Fedora/RedHat

Подключаем EPEL репозиторий:

Установка EPEL

Далее, стоит установить следующие пакеты:

# yum install zlib-devel bzip2 bzip2-devel readline-devel sqlite \
sqlite-devel openssl-devel xz xz-devel libffi-devel

Имеется простой скрипт для установки данного ПО, можно использовать:

$ cd /usr/local/src && curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash

После установки, стоит настроить и потом, можно будет юзать данную тулу.

Установка pyenv в Debian/Ubuntu

Стоит установить следующие пакеты:

# apt-get install -y make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev \
libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python-openssl

Имеется простой скрипт для установки данного ПО, можно использовать:

$ cd /usr/local/src && curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash

После установки, стоит настроить и потом, можно будет юзать данную тулу.

Установка pyenv в MacOS

Ставим homebrew, статью можно взять тут:

Установка homebrew на Mac OS X

Выполним поиск пакета:

$ brew search pyenv

Чтобы поставить данное ПО, выполняем:

$ brew install pyenv

Перейдем к настройке.

Настройка pyenv в Unix/Linux

Открываем файл:

$ vim  ~/.bashrc

Прописываем:

export PYENV_ROOT="${HOME}/.pyenv"

if [ -d "${PYENV_ROOT}" ]; then
    export PATH="${PYENV_ROOT}/bin:${PATH}"
    eval "$(pyenv init -)"
    eval "$(pyenv virtualenv-init -)"
fi

Просто добавьте этот код в конец вашего ~/ .bashrc и затем загрузите ваш профиль, чтобы загрузить эти дополнения:

Для RPM’s ОС:

$ source ~/.bash_profile

Или:

$ . ~/.bash_profile

Для DEB’s ОС:

$ source ~/.profile

Для MacOS:

$ source ~/.bash_profile

Можно уже юзать!

Использование pyenv в Unix/Linux

Для того, чтобы посмотреть какие версии python используется на данный момент, имеется команда:

$ pyenv install -l

Для установки какой-либо версии, используем:

$ pyenv install 2.7.16 && \
  pyenv install 3.5.7

Смотрим что имеется в системе:

$ pyenv versions
* system
  2.7.16
  3.5.7

Чтобы переключится на нужную версию, используем:

$ pyenv local 2.7.16

Так же, можно будет использовать:

$ pyenv local some_project

Я пока что буду использовать:

$ pyenv local 2.7.16 3.5.7

Для глобального использования, можно выставить тоже нужную(ые) версию(и):

$ pyenv global 3.5.7 2.7.16

Можно выставить конкретную версию питона для shell оболочки, например:

$ pyenv shell 3.5.7

Как это выглядит на картинке:

Создадим песочницу:

$ pyenv virtualenv -p python2.7 2.7.16 my_project-with-2.7.16

Или:

$ pyenv virtualenv 2.7.16 my_project-with-2.7.16

Смотрим песочницы:

$ pyenv virtualenvs

Сейчас, я выставлю нужные мне версии:

$ pyenv local my_project-with-2.7.16 3.5.7

Смотрим версию питона:

$ python -V
Python 2.7.16

Смотрим где находится питон:

$ which python
/Users/captain/.pyenv/shims/python

И:

$ pyenv which python
/Users/captain/.pyenv/versions/my_project-with-2.7.16/bin/python

Для получения списка команд, множно использовать:

$ pyenv commands
--version
activate
commands
completions
deactivate
exec
global
help
hooks
init
install
local
prefix
realpath.dylib
rehash
root
shell
shims
uninstall
version
version-file
version-file-read
version-file-write
version-name
version-origin
versions
virtualenv
virtualenv-delete
virtualenv-init
virtualenv-prefix
virtualenvs
virtualenvwrapper
virtualenvwrapper_lazy
whence
which

Получить помощь можно так:

$ pyenv --help
Usage: pyenv  []

Some useful pyenv commands are:
   commands    List all available pyenv commands
   local       Set or show the local application-specific Python version
   global      Set or show the global Python version
   shell       Set or show the shell-specific Python version
   install     Install a Python version using python-build
   uninstall   Uninstall a specific Python version
   rehash      Rehash pyenv shims (run this after installing executables)
   version     Show the current Python version and its origin
   versions    List all Python versions available to pyenv
   which       Display the full path to an executable
   whence      List all Python versions that contain the given executable

See `pyenv help ' for information on a specific command.
For full documentation, see: https://github.com/pyenv/pyenv#readme

Или, для какой-то конкретной команды:

$ pyenv activate --help
Usage: pyenv activate 
       pyenv activate --unset

Activate a Python virtualenv environment in current shell.
This acts almost as same as `pyenv shell`, but this invokes the `activate`
script in your shell.

 should be a string matching a Python version known to pyenv.

Т.е для активации созданной среды, используем:

$ pyenv activate my_project-with-2.7.16

Для диактивации, следующая команда:

$ pyenv deactivate

Для удаления версии питона с ОС, используем:

$ pyenv uninstall 2.7.16

Для обновления pyenv, имеется конманда:

$ pyenv update

Для удаления, используйте:

$ rm -rf $PYENV_ROOT

ЗАМЕЧАНИЕ: По умолчанию, $PYENV_ROOT — ~/.pyenv.

Так же, стоит удалить следующие строки с ~/.bashrc:

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

Вот и все, статья «Установка pyenv в Unix/Linux» завершена.

The post Установка pyenv в Unix/Linux first appeared on linux-notes.org.

Page_c76a3fcf

captain — Sun, 21 Apr 2019 13:27:54 +0000

New Relic — это веб-приложение, нацеленное в первую очередь на разработчиков. Его основная цель — отслеживание путей взаимодействия пользователя с разработанными продуктами. Это средство аналитики ПО.

Ключевые особенности New Relic:

Мониторинг доступности.
Предупреждения и уведомления.
Профили производительности.
Плагины.
Гистограммы и процентили.
Доступ API.
Время отклика в режиме реального времени.
Анализатор производительности виртуальной машины Java.
Обнаружение ошибок, анализ.
Вызов базы данных с временем отклика и пропускной способностью.
Диагностика, трассировка транзакций, трассировка стека.
Анализ времени отклика, пропускной способности и разбивка по компонентам.
Подробности по производительности SQL.
Анализ пользователя в режиме реального времени при просмотре веб-страниц.
Кастомные дэшборды.
Отслеживание отдельных бизнес-операций.
X-Ray сессии для деловых операций.
Кросс-трассировка приложений для распределенных приложений.
Отчёты по доступности, масштабируемости, развёртыванию.

Установка terraform в Unix/Linux

Установка крайне примитивная и я описал как это можно сделать тут:

Установка terraform в Unix/Linux

Так же, в данной статье, я создал скрипт для автоматической установки данного ПО. Он был протестирован на CentOS 6/7, Debian 8 и на Mac OS X. Все работает должным образом!

Чтобы получить помощь по использованию команд, выполните:

$ terraform --help
Usage: terraform [-version] [-help]  [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
    destroy            Destroy Terraform-managed infrastructure
    env                Workspace management
    fmt                Rewrites config files to canonical format
    get                Download and install modules for the configuration
    graph              Create a visual graph of Terraform resources
    import             Import existing infrastructure into Terraform
    init               Initialize a Terraform working directory
    output             Read an output from a state file
    plan               Generate and show an execution plan
    providers          Prints a tree of the providers used in the configuration
    push               Upload this Terraform module to Atlas to run
    refresh            Update local state file against real resources
    show               Inspect Terraform state or plan
    taint              Manually mark a resource for recreation
    untaint            Manually unmark a resource as tainted
    validate           Validates the Terraform files
    version            Prints the Terraform version
    workspace          Workspace management

All other commands:
    debug              Debug output management (experimental)
    force-unlock       Manually unlock the terraform state
    state              Advanced state management

Приступим к использованию!

Работа с New Relic (newrelic nrql) и Terraform в Unix/Linux —

У меня есть папка terraform, в ней у меня будут лежать провайдеры с которыми я буду работать. Т.к в этом примере я буду использовать newrelic, то создам данную папку и перейду в нее. Далее, в этой папке, стоит создать:

$ mkdir examples modules

В папке examples, я буду хранить так званые «плейбуки» для разварачивания различных служб, например — zabbix-server, grafana, web-серверы и так далее. В modules директории, я буду хранить все необходимые модули.

Начнем писать модуль, но для этой задачи, я создам папку:

$  mkdir modules/newrelic_nrql

Переходим в нее:

$ cd modules/newrelic_nrql

Открываем файл:

$ vim newrelic_nrql_alert_condition.tf

В данный файл, вставляем:

#---------------------------------------------------
# Add newrelic nrql alert condition
#---------------------------------------------------
resource "newrelic_nrql_alert_condition" "nrql_alert_condition" {
    count               = "${var.nrql_alert_condition ? 1 : 0}"
            
    name                = "${var.nrql_alert_condition_name !="" ? var.nrql_alert_condition_name : "${lower(var.name)}-nr-nrql-alert-condition-${lower(var.environment)}" }"
    policy_id           = "${var.nrql_alert_condition_policy_id}"

    runbook_url         = "${var.nrql_alert_condition_runbook_url}"
    enabled             = "${var.nrql_alert_condition_enabled}"
    value_function      = "${var.nrql_alert_condition_value_function}"

    term {
        duration      = "${var.nrql_alert_condition_term_duration}"
        operator      = "${var.nrql_alert_condition_term_operator}"
        priority      = "${var.nrql_alert_condition_term_priority}"
        threshold     = "${var.nrql_alert_condition_term_threshold}"
        time_function = "${var.nrql_alert_condition_term_time_function}"
    }

   # term {
   #     duration      = 5
   #     operator      = "below"
   #     priority      = "warning"
   #     threshold     = "1"
   #     time_function = "all"
   # }
    
    nrql {
        query       = "${var.nrql_alert_condition_nrql_query}"
        since_value = "${var.nrql_alert_condition_nrql_since_value}"
    }

    lifecycle = {
        create_before_destroy   = true,
        ignore_changes          = []
    }

    depends_on  = []
    
}

Открываем файл:

$ vim variables.tf

И прописываем:

#-----------------------------------------------------------
# Global 
#-----------------------------------------------------------
variable "name" {
    description = "The name for newrelic_alert resources"
    default     = "test"
}   

variable "environment" {
    description = "environment"
    default     = "prod"
}

#-----------------------------------------------------------
# newrelic_nrql_alert_condition
#-----------------------------------------------------------
variable "nrql_alert_condition" {
    description = "Enable newrelic_nrql_alert_condition"
    default     = "false"
}

variable "nrql_alert_condition_name" {
    description = "(Required) The title of the condition"
    default     = ""
}

variable "nrql_alert_condition_policy_id" {
    description = "(Required) The ID of the policy where this condition should be used."
    default     = ""
}

variable "nrql_alert_condition_runbook_url" {
    description = "(Optional) Runbook URL to display in notifications."
    default     = ""
}

variable "nrql_alert_condition_enabled" {
    description = "(Optional) Set whether to enable the alert condition. Defaults to true."
    default     = "true"
}

variable "nrql_alert_condition_value_function" {
    description = "(Optional) Possible values are single_value, sum."
    default     = "single_value"
}

variable "nrql_alert_condition_term_duration" {
    description = "(Required) In minutes, must be: 1, 2, 3, 4, 5, 10, 15, 30, 60, or 120."
    default     = 5
}

variable "nrql_alert_condition_term_operator" {
    description = "(Optional) above, below, or equal. Defaults to equal."
    default     = "equal"
}

variable "nrql_alert_condition_term_priority" {
    description = "(Optional) critical or warning. Defaults to critical."
    default     = "critical"
}

variable "nrql_alert_condition_term_threshold" {
    description = "(Required) Must be 0 or greater."
    default     = 0
}

variable "nrql_alert_condition_term_time_function" {
    description = "(Required) all or any."
    default     = "all"
}


variable "nrql_alert_condition_nrql_query" {
    description = "(Required) The NRQL query to execute for the condition."
    default     = ""
}

variable "nrql_alert_condition_nrql_since_value" {
    description = "(Required) The value to be used in the SINCE  MINUTES AGO clause for the NRQL query. Must be: 1, 2, 3, 4, or 5."
    default     = 1
}

Собственно в этом файле храняться все переменные. Спасибо кэп!

Открываем последний файл:

$ vim outputs.tf

И в него вставить нужно следующие строки:

#-----------------------------------------------------------
# newrelic_nrql_alert_condition
#-----------------------------------------------------------
output "nrql_alert_condition_id" {
    description = "ID for newrelic_nrql_alert_condition"
    value       = "${newrelic_nrql_alert_condition.nrql_alert_condition.*.id}"
}

Переходим теперь в папку aws/examples и создадим еще одну папку для проверки написанного чуда:

$ mkdir newrelic_nrql && cd $_

Внутри созданной папки открываем файл:

$ vim main.tf

И вставим в него следующий код:

#
# MAINTAINER Vitaliy Natarov "vitaliy.natarov@yahoo.com"
#
terraform {
  required_version = "~> 0.11.11"
}

provider "newrelic" {
    api_key = "75e6741e6326cce1666ecfb94c3c0b8fdf"
}

module "newrelic_alert" {
    source                                              = "../../modules/newrelic_alert"

    # vars for newrelic_alert_policy
    alert_policy                                        = "true"
    #alert_policy_name                                  = "new-relic-policy-PER_CONDITION"
    alert_policy_incident_preference                    = "PER_CONDITION"
    alert_policy_simple_default                         = "true"

    alert_channel                                       = "true"
    #
    alert_channel_slack                                 = "true"
    alert_channel_slack_configuration_channel           = "new-relic"
    alert_channel_slack_configuration_url               = "https://hooks.slack.com/services/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666"
    #
    alert_condition                                     = "true"
    alert_condition_policy_id                           = "${element(module.newrelic_alert.alert_policy_id, 0)}"
    #alert_condition_policy_id                           = "${element(module.newrelic_alert.simple_default_alert_policy_id, 0)}"
    alert_condition_type                                = "apm_app_metric"
    alert_condition_entities                            = ["PHP Application"]

    alert_policy_channel                                = "false"
}

module "newrelic_nrql" {
    source                                              = "../../modules/newrelic_nrql"

    nrql_alert_condition                                = "true"
    nrql_alert_condition_policy_id                      = "${element(module.newrelic_alert.alert_policy_id, 0)}"

    nrql_alert_condition_nrql_query                     = "SELECT count(*) FROM SyntheticCheck WHERE monitorId = '1'"
    
}

В файле стоит прописать все необходимое, но самое главное:

api_key — API ключ, который можно сгенерировать в «‘Account settings’->’API keys'».

Все уже написано и готово к использованию. Ну что, начнем тестирование. В папке с вашим плейбуком, выполняем:

$ terraform init

Этим действием я инициализирую проект. Затем, подтягиваю модуль:

$ terraform get

PS: Для обновление изменений в самом модуле, можно выполнять:

$ terraform get -update

Проверим валидацию:

$ terraform validate

Запускем прогон:

$ terraform plan

Мне вывело что все у меня хорошо и можно запускать деплой:

$ terraform apply

Как видно с вывода, — все прошло гладко! Чтобы удалить созданное творение, можно выполнить:

$ terraform destroy

Еще полезности:

Работа с New Relic (newrelic alert) и Terraform в Unix/Linux

Работа с New Relic (newrelic nrql) и Terraform в Unix/Linux

Работа с New Relic (newrelic infra) и Terraform в Unix/Linux

Работа с New Relic (newrelic synthetics) и Terraform в Unix/Linux

Работа с New Relic (newrelic dashboard) и Terraform в Unix/Linux

Весь материал аплоаджу в github аккаунт для удобства использования:

$ git clone https://github.com/SebastianUA/terraform.git

Вот и все на этом. Данная статья «Работа с New Relic (newrelic nrql) и Terraform в Unix/Linux» завершена.

The post Работа с New Relic (newrelic nrql) и Terraform в Unix/Linux first appeared on linux-notes.org.

Page_c76a3fcf

captain — Sat, 20 Apr 2019 23:57:58 +0000

Ключевые особенности New Relic:

Мониторинг доступности.
Предупреждения и уведомления.
Профили производительности.
Плагины.
Гистограммы и процентили.
Доступ API.
Время отклика в режиме реального времени.
Анализатор производительности виртуальной машины Java.
Обнаружение ошибок, анализ.
Вызов базы данных с временем отклика и пропускной способностью.
Диагностика, трассировка транзакций, трассировка стека.
Анализ времени отклика, пропускной способности и разбивка по компонентам.
Подробности по производительности SQL.
Анализ пользователя в режиме реального времени при просмотре веб-страниц.
Кастомные дэшборды.
Отслеживание отдельных бизнес-операций.
X-Ray сессии для деловых операций.
Кросс-трассировка приложений для распределенных приложений.
Отчёты по доступности, масштабируемости, развёртыванию.

Установка terraform в Unix/Linux

Установка крайне примитивная и я описал как это можно сделать тут:

Установка terraform в Unix/Linux

Чтобы получить помощь по использованию команд, выполните:

$ terraform --help
Usage: terraform [-version] [-help]  [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
    destroy            Destroy Terraform-managed infrastructure
    env                Workspace management
    fmt                Rewrites config files to canonical format
    get                Download and install modules for the configuration
    graph              Create a visual graph of Terraform resources
    import             Import existing infrastructure into Terraform
    init               Initialize a Terraform working directory
    output             Read an output from a state file
    plan               Generate and show an execution plan
    providers          Prints a tree of the providers used in the configuration
    push               Upload this Terraform module to Atlas to run
    refresh            Update local state file against real resources
    show               Inspect Terraform state or plan
    taint              Manually mark a resource for recreation
    untaint            Manually unmark a resource as tainted
    validate           Validates the Terraform files
    version            Prints the Terraform version
    workspace          Workspace management

All other commands:
    debug              Debug output management (experimental)
    force-unlock       Manually unlock the terraform state
    state              Advanced state management

Приступим к использованию!

Работа с New Relic (newrelic alert) и Terraform в Unix/Linux —

$ mkdir examples modules

Начнем писать модуль, но для этой задачи, я создам папку:

$  mkdir modules/newrelic_alert

Переходим в нее:

$ cd modules/newrelic_alert

Открываем файл:

$ vim newrelic_alert_policy.tf

В данный файл, вставляем:

resource "newrelic_alert_policy" "alert_policy" {
    count   = "${var.alert_policy ? 1 : 0}"

    name    = "${var.alert_policy_name !="" ? "${lower(var.alert_policy_name)}" : "${lower(var.name)}-nr-policy-${lower(var.alert_policy_incident_preference)}-${lower(var.environment)}" }"
    incident_preference = "${var.alert_policy_incident_preference}"

    lifecycle = {
        create_before_destroy   = true,
        ignore_changes          = []
    }

    depends_on  = []
}

resource "newrelic_alert_policy" "simple_default" {
    count   = "${var.alert_policy && var.alert_policy_simple_default ? 1 : 0}"
                        
    name    = "${var.alert_policy_simple_default_name !="" ? "${lower(var.alert_policy_simple_default_name)}" : "simple-default-nr-policy-${lower(var.alert_policy_incident_preference)}-${lower(var.environment)}" }"
    lifecycle = {
        create_before_destroy   = true,
        ignore_changes          = []
    }

    depends_on  = []
}

Открываем файл:

$ vim newrelic_alert_channel.tf

В данный файл, вставляем:

#---------------------------------------------------
# Add newrelic alert channel
#---------------------------------------------------
# campfire
resource "newrelic_alert_channel" "alert_channel_campfire" {
    count   = "${var.alert_channel_campfire && var.alert_channel ? 1 : 0}"

    name    = "${var.alert_channel_campfire_name !="" ? var.alert_channel_campfire_name : "${lower(var.name)}-nr-channel-campfire-${lower(var.environment)}" }"
    type    = "campfire"
    
    configuration = {
        room        = "${var.alert_channel_campfire_configuration_room}"
        subdomain   = "${var.alert_channel_campfire_configuration_subdomain}"
        token       = "${var.alert_channel_campfire_configuration_token}"
    }

    lifecycle = {
        create_before_destroy   = true,
        ignore_changes          = []
    }

    depends_on  = []
    
}

#email
resource "newrelic_alert_channel" "alert_channel_email" {
    count   = "${var.alert_channel_email && var.alert_channel ? 1 : 0}"

    name    = "${var.alert_channel_email_name !="" ? var.alert_channel_email_name : "${lower(var.name)}-nr-channel-email-${lower(var.environment)}" }"
    type    = "email"

    configuration = {
        recipients              = "${var.alert_channel_email_configuration_recipients}"
        include_json_attachment = "${var.alert_channel_email_configuration_include_json_attachment}"
    }

    lifecycle = {
        create_before_destroy   = true,
        ignore_changes          = []
    }

    depends_on  = []

}

#opsgenie
resource "newrelic_alert_channel" "alert_channel_opsgenie" {
    count   = "${var.alert_channel_opsgenie && var.alert_channel ? 1 : 0}"

    name    = "${var.alert_channel_opsgenie_name !="" ? var.alert_channel_opsgenie_name : "${lower(var.name)}-nr-channel-opsgenie-${lower(var.environment)}" }"
    type    = "opsgenie"

    configuration = {
        api_key                 = "${var.alert_channel_opsgenie_configuration_api_key}"
        recipients              = "${var.alert_channel_opsgenie_configuration_recipients}"
        tags                    = "${var.alert_channel_opsgenie_configuration_tags}"
        teams                   = "${var.alert_channel_opsgenie_configuration_teams}"
    }

    lifecycle = {
        create_before_destroy   = true,
        ignore_changes          = []
    }

    depends_on  = []

}

#pagerduty
resource "newrelic_alert_channel" "alert_channel_pagerduty" {
    count   = "${var.alert_channel_pagerduty && var.alert_channel ? 1 : 0}"

    name    = "${var.alert_channel_pagerduty_name !="" ? var.alert_channel_pagerduty_name : "${lower(var.name)}-nr-channel-pagerduty-${lower(var.environment)}" }"
    type    = "pagerduty"

    configuration = {
        service_key = "${var.alert_channel_pagerduty_configuration_service_key}"
    }

    lifecycle = {
        create_before_destroy   = true,
        ignore_changes          = []
    }

    depends_on  = []

}

#slack
resource "newrelic_alert_channel" "alert_channel_slack" {
    count   = "${var.alert_channel_slack && var.alert_channel ? 1 : 0}"
        
    name    = "${var.alert_channel_slack_name !="" ? var.alert_channel_slack_name : "${lower(var.name)}-nr-channel-slack-${lower(var.environment)}" }"
    type    = "slack"

    configuration = {
        channel     = "${var.alert_channel_slack_configuration_channel}"
        url         = "${var.alert_channel_slack_configuration_url}"
    }

    lifecycle = {
        create_before_destroy   = true,
        ignore_changes          = ["configuration"]
    }

    depends_on  = []

}

#user
resource "newrelic_alert_channel" "alert_channel_user" {
    count   = "${var.alert_channel_user && var.alert_channel ? 1 : 0}"

    name    = "${var.alert_channel_user_name !="" ? var.alert_channel_user_name : "${lower(var.name)}-nr-channel-user-${lower(var.environment)}" }"
    type    = "user"

    configuration = {
        user_id     = "${var.alert_channel_user_configuration_user_id}"
    }

    lifecycle = {
        create_before_destroy   = true,
        ignore_changes          = []
    }

    depends_on  = []
}

#victorops
resource "newrelic_alert_channel" "alert_channel_victorops" {
    count   = "${var.alert_channel_victorops && var.alert_channel ? 1 : 0}"

    name    = "${var.alert_channel_victorops_name !="" ? var.alert_channel_victorops_name : "${lower(var.name)}-nr-channel-victorops-${lower(var.environment)}" }"
    type    = "victorops"

    configuration = {
        key         = "${var.alert_channel_victorops_configuration_key}"
        route_key   = "${var.alert_channel_victorops_configuration_route_key}"
    }

    lifecycle = {
        create_before_destroy   = true,
        ignore_changes          = []
    }

    depends_on  = []

}

#webhook
resource "newrelic_alert_channel" "alert_channel_webhook" {
    count   = "${var.alert_channel_webhook && var.alert_channel ? 1 : 0}"
       
    name    = "${var.alert_channel_webhook_name !="" ? var.alert_channel_webhook_name : "${lower(var.name)}-nr-channel-webhook-${lower(var.environment)}" }"
    type    = "webhook"

    configuration = {                           
        auth_password           = "${var.alert_channel_webhook_configuration_auth_password}"
        auth_type               = "${var.alert_channel_webhook_configuration_auth_type}"
        auth_username           = "${var.alert_channel_webhook_configuration_auth_username}"
        base_url                = "${var.alert_channel_webhook_configuration_base_url}"
        headers                 = "${var.alert_channel_webhook_configuration_headers}"
        payload_type            = "${var.alert_channel_webhook_configuration_payload_type}"
        payload                 = "${var.alert_channel_webhook_configuration_payload}"
    }

    lifecycle = {
        create_before_destroy   = true,
        ignore_changes          = []
    }

    depends_on  = []

}

Открываем файл:

$ vim newrelic_alert_condition.tf

Вставляем:

resource "newrelic_alert_condition" "alert_condition" {
    count                       = "${var.alert_condition? 1 : 0}"
                            
    name                        = "${var.alert_condition_name !="" ? "${lower(var.alert_condition_name)}" : "${lower(var.name)}-nr-alert-condition-${var.alert_condition_type}-${lower(var.environment)}" }"
    policy_id                   = "${var.alert_condition_policy_id}"
    type                        = "${var.alert_condition_type}"
    entities                    = ["${data.newrelic_application.application.id}"]
    metric                      = "${var.alert_condition_metric}"

    gc_metric                   = "${var.alert_condition_gc_metric}"
    violation_close_timer       = "${var.alert_condition_violation_close_timer}"
    runbook_url                 = "${var.alert_condition_runbook_url}"
    condition_scope             = "${var.alert_condition_condition_scope}"
    user_defined_metric         = "${var.alert_condition_user_defined_metric}"
    user_defined_value_function = "${var.alert_condition_user_defined_value_function}"

    term {
        duration      = "${var.alert_condition_term_duration}"
        operator      = "${var.alert_condition_term_operator}"
        priority      = "${var.alert_condition_term_priority}"
        threshold     = "${var.alert_condition_term_threshold}"
        time_function = "${var.alert_condition_term_time_function}"
    }

    lifecycle = {
        create_before_destroy   = true,
        # I think, newrelic provider has a bug. To fix it, I have added the next ignoring changes.
        # Note: If you're changing those values, please use `terraform destroy` and `terraform plan && terraform apply` to update all your values.
        ignore_changes          = ["gc_metric", "user_defined_value_function", "violation_close_timer"]
    }

    depends_on  = ["data.newrelic_application.application"]
    
}

data "newrelic_application" "application" {
    count   = "${length(var.alert_condition_entities) >0 ? 1 : 0}"

    name    = "${var.alert_condition_entities[count.index]}"
}

Открываем файл:

$ vim newrelic_alert_policy_channel.tf

resource "newrelic_alert_policy_channel" "alert_policy_channel" {
    count       = "${var.alert_policy_channel ? 1 : 0}"
            
    policy_id   = "${var.alert_policy_policy_id}"
    channel_id  = "${var.alert_policy_channel_id}"
}

Открываем файл:

$ vim variables.tf

И прописываем:

#-----------------------------------------------------------
# Global 
#-----------------------------------------------------------
variable "name" {
    description = "The name for newrelic_alert resources"
    default     = "test"
}   

variable "environment" {
    description = "environment"
    default     = "prod"
}

#-----------------------------------------------------------
# newrelic_alert_channel
#-----------------------------------------------------------
variable "alert_channel" {
    description = "Enable newrelic alert channel at general"
    default     = "false"
}

#-----------------------------------------------------------
# newrelic_alert_channel_campfire
#-----------------------------------------------------------
variable "alert_channel_campfire" {
    description = "Enable newrelic alert channel campfire usage"
    default     = "false"
}

variable "alert_channel_campfire_name" {
    description = "Set custom name for newrelic_alert_channel"
    default     = ""
}

variable "alert_channel_campfire_configuration_room" {
    description = "Set room"
    default     = ""
}

variable "alert_channel_campfire_configuration_subdomain" {
    description = "Set subdomain"
    default     = ""
}

variable "alert_channel_campfire_configuration_token" {
    description = "Set token"
    default     = ""
}

#-----------------------------------------------------------
# newrelic_alert_channel_email
#-----------------------------------------------------------
variable "alert_channel_email" {
    description = "Enable newrelic alert channel email usage"
    default     = "false"
}

variable "alert_channel_email_name" {
    description = "Set custom name for newrelic_alert_channel"
    default     = ""
}

variable "alert_channel_email_configuration_recipients" {
    description = "Set reemail. In this case, its - email address"
    default     = ""
}

variable "alert_channel_email_configuration_include_json_attachment" {
    description = "Set include_json_attachment"
    default     = 1
}

#-----------------------------------------------------------
# newrelic_alert_channel_opsgenie
#-----------------------------------------------------------
variable "alert_channel_opsgenie" {
    description = "Enable newrelic alert channel opsgenie"
    default     = "false"
}

variable "alert_channel_opsgenie_name" {
    description = "Set custom name for newrelic_alert_channel"
    default     = ""
}

variable "alert_channel_opsgenie_configuration_api_key" {
    description = "Set api_key"
    default     = ""
}

variable "alert_channel_opsgenie_configuration_recipients" {
    description = "Set reemail. In this case, its - email address"
    default     = ""
}

variable "alert_channel_opsgenie_configuration_tags" {
    description = "Set tags"
    type        = "list"
    default     = []
}


variable "alert_channel_opsgenie_configuration_teams" {
    description = "Set teams"
    type        = "list"
    default     = []
}

#-----------------------------------------------------------
# newrelic_alert_channel_pagerduty
#-----------------------------------------------------------
variable "alert_channel_pagerduty" {
    description = "Enable newrelic alert channel pagerduty usage"
    default     = "false"
}

variable "alert_channel_pagerduty_name" {
    description = "Set custom name for newrelic_alert_channel"
    default     = ""
}

variable "alert_channel_pagerduty_configuration_service_key" {
    description = "Set service_key"
    default     = ""
}

#-----------------------------------------------------------
# newrelic_alert_channel_slack
#-----------------------------------------------------------
variable "alert_channel_slack" {
    description = "Enable newrelic alert channel slack usage"
    default     = "false"
}

variable "alert_channel_slack_name" {
    description = "Set custom name for newrelic_alert_channel"
    default     = ""
}

variable "alert_channel_slack_configuration_channel" {
    description = "Set channel"
    default     = ""
}

variable "alert_channel_slack_configuration_url" {
    description = "Set url"
    default     = 1
}

#-----------------------------------------------------------
# newrelic_alert_channel_user
#-----------------------------------------------------------
variable "alert_channel_user" {
    description = "Enable newrelic alert channel user usage"
    default     = "false"
}

variable "alert_channel_user_name" {
    description = "Set custom name for newrelic_alert_channel"
    default     = ""
}

variable "alert_channel_user_configuration_user_id" {
    description = "Set user_id"
    default     = ""
}

#-----------------------------------------------------------
# newrelic_alert_channel_victorops
#-----------------------------------------------------------
variable "alert_channel_victorops" {
    description = "Enable newrelic alert channel victorops usage"
    default     = "false"
}

variable "alert_channel_victorops_name" {
    description = "Set custom name for newrelic_alert_channel"
    default     = ""
}

variable "alert_channel_victorops_configuration_key" {
    description = "Set key"
    default     = ""
}

variable "alert_channel_victorops_configuration_route_key" {
    description = "Set route_key"
    default     = ""
}

#-----------------------------------------------------------
# newrelic_alert_channel_webhook
#-----------------------------------------------------------
variable "alert_channel_webhook" {
    description = "Enable newrelic alert channel webhook usage"
    default     = "false"
}

variable "alert_channel_webhook_name" {
    description = "Set custom name for newrelic_alert_channel"
    default     = ""
}

variable "alert_channel_webhook_configuration_auth_password" {
    description = "Set auth_password"
    default     = ""
}

variable "alert_channel_webhook_configuration_auth_type" {
    description = "Set auth_type"
    default     = ""
}

variable "alert_channel_webhook_configuration_auth_username" {
    description = "Set auth_username"
    default     = ""
}

variable "alert_channel_webhook_configuration_base_url" {
    description = "Set base_url"
    default     = ""
}

variable "alert_channel_webhook_configuration_headers" {
    description = "Set headers"
    type        = "list" 
    default     = []
}

variable "alert_channel_webhook_configuration_payload_type" {
    description = "Set type"
    default     = ""
}

variable "alert_channel_webhook_configuration_payload" {
    type        = "list"
    default     = []
}

#-----------------------------------------------------------
# newrelic_alert_policy
#-----------------------------------------------------------

variable "alert_policy" {
    description = "Enable newrelic_alert_policy usage"
    default     = "false"
}

variable "alert_policy_name" {
    description = "Set custom name for newrelic_alert_policy"
    default     = ""
}

variable "alert_policy_incident_preference" {
    description = "(Optional) The rollup strategy for the policy. Options include: PER_POLICY, PER_CONDITION, or PER_CONDITION_AND_TARGET. The default is PER_POLICY."
    default     = "PER_POLICY"
}
 
variable "alert_policy_simple_default" {
    description = "Enable newrelic_alert_policy_simple_default"
    default     = "false"
}

variable "alert_policy_simple_default_name" {
    description = "Name"
    default     = ""
}

#-----------------------------------------------------------
# newrelic_alert_condition
#-----------------------------------------------------------

variable "alert_condition" {
    description = "Enable newrelic_alert_condition"
    default     = "false"
}

variable "alert_condition_policy_id" {
    description = "(Required) The ID of the policy where this condition should be used."
    default     = ""
}

variable "alert_condition_name" {
    description = "(Required) The title of the condition"
    default     = ""
}

variable "alert_condition_type" {
    description = "(Required) The type of condition. One of: apm_app_metric, apm_jvm_metric, apm_kt_metric, servers_metric, browser_metric, mobile_metric"
    default     = "apm_app_metric"
}

variable "alert_condition_entities" {
    description = "(Required) The instance IDS associated with this condition."
    type        = "list"
    default     = []
}

variable "alert_condition_metric" {
    description = "(Required) The metric field accepts parameters based on the `type` set."
    default     = "apdex"
}

variable "alert_condition_gc_metric" {
    description = "(Optional) A valid Garbage Collection metric e.g. GC/G1 Young Generation. This is required if you are using apm_jvm_metric with gc_cpu_time condition type."
    default     = "GC/G1 Young Generation"
}

variable "alert_condition_violation_close_timer" {
    description = "(Optional) Automatically close instance-based violations, including JVM health metric violations, after the number of hours specified. Must be: 1, 2, 4, 8, 12 or 24."
    default     = 1
}

variable "alert_condition_runbook_url" {
    description = "(Optional) Runbook URL to display in notifications."
    default     = ""
}

variable "alert_condition_condition_scope" {
    description = "(Optional) instance or application. This is required if you are using the JVM plugin in New Relic."
    default     = "application"
}

variable "alert_condition_user_defined_metric" {
    description = "(Optional) A custom metric to be evaluated."
    default     = ""
}

variable "alert_condition_user_defined_value_function" {
    description = "(Optional) One of: average, min, max, total, or sample_size."
    default     = "average"
}

variable "alert_condition_term_duration" {
    description = "(Required) In minutes, must be: 5, 10, 15, 30, 60, or 120."
    default     = 5
}

variable "alert_condition_term_operator" {
    description = "(Optional) above, below, or equal. Defaults to equal."
    default     = "equal"
}

variable "alert_condition_term_priority" {
    description = "(Optional) critical or warning. Defaults to critical."
    default     = "critical"
}

variable "alert_condition_term_threshold" {
    description = "(Required) Must be 0 or greater."
    default     = 0
}

variable "alert_condition_term_time_function" {
    description = "(Required) all or any."
    default     = "all"
}

#-----------------------------------------------------------
# newrelic_alert_policy_channel
#-----------------------------------------------------------
variable "alert_policy_channel" {
    description = "Enable newrelic_alert_policy_channel"
    default     = "false"
}

variable "alert_policy_policy_id" {
    description = "(Required) The ID of the policy."
    default     = ""
}

variable "alert_policy_channel_id" {
    description = "(Required) The ID of the channel."
    default     = ""
}

Собственно в этом файле храняться все переменные. Спасибо кэп!

Открываем последний файл:

$ vim outputs.tf

И в него вставить нужно следующие строки:

#-----------------------------------------------------------
# newrelic_alert_channel
#-----------------------------------------------------------
output "alert_channel_campfire_id" {
    description = "ID for newrelic_alert_channel_campfire"
    value       = "${newrelic_alert_channel.alert_channel_campfire.*.id}"
}

output "alert_channel_email_id" {
    description = "ID for newrelic_alert_channel_email"
    value       = "${newrelic_alert_channel.alert_channel_email.*.id}"
}

output "alert_channel_opsgenie_id" {
    description = "ID for newrelic_alert_channel_opsgenie"
    value       = "${newrelic_alert_channel.alert_channel_opsgenie.*.id}"
}

output "alert_channel_pagerduty_id" {
    description = "ID for newrelic_alert_channel_pagerduty"
    value       = "${newrelic_alert_channel.alert_channel_pagerduty.*.id}"
}

output "alert_channel_slack_id" {
    description = "ID for newrelic_alert_channel_slack"
    value       = "${newrelic_alert_channel.alert_channel_slack.*.id}"
}

output "alert_channel_user_id" {
    description = "ID for newrelic_alert_channel_user"
    value       = "${newrelic_alert_channel.alert_channel_user.*.id}"
}

output "alert_channel_victorops_id" {
    description = "ID for newrelic_alert_channel_victorops"
    value       = "${newrelic_alert_channel.alert_channel_victorops.*.id}"
}

output "alert_channel_webhook_id" {
    description = "ID for newrelic_alert_channel_webhook"
    value       = "${newrelic_alert_channel.alert_channel_webhook.*.id}"
}

#-----------------------------------------------------------
# newrelic_alert_policy
#-----------------------------------------------------------
output "alert_policy_id" {
    description = "ID of the policy."
    value       = "${newrelic_alert_policy.alert_policy.*.id}"
}

output "simple_default_alert_policy_id" {
    description = "ID of the policy for simple_default"
    value       = "${newrelic_alert_policy.simple_default.*.id}"
}

#-----------------------------------------------------------
# newrelic_alert_condition
#-----------------------------------------------------------
output "alert_condition_id" {
    description = "The ID of the alert condition."
    value       = "${newrelic_alert_condition.alert_condition.*.id}"
}

#-----------------------------------------------------------
# newrelic_alert_policy_channel
#-----------------------------------------------------------
output "alert_policy_channel_id" {
    description = "The ID of the alert policy channel"
    value       = "${newrelic_alert_policy_channel.alert_policy_channel.*.id}"
}

Переходим теперь в папку aws/examples и создадим еще одну папку для проверки написанного чуда:

$ mkdir newrelic_alert && cd $_

Внутри созданной папки открываем файл:

$ vim main.tf

И вставим в него следующий код:

#
# MAINTAINER Vitaliy Natarov "vitaliy.natarov@yahoo.com"
#
terraform {
  required_version = "~> 0.11.11"
}

provider "newrelic" {
    api_key = "75e6741e6326cce1666ecfb94c3c0b8fdf"
}

module "newrelic_alert" {
    source                                              = "../../modules/newrelic_alert"
                                                        
    # vars for newrelic_alert_policy
    alert_policy                                        = "true"
    #alert_policy_name                                  = "new-relic-policy-PER_CONDITION"
    alert_policy_incident_preference                    = "PER_CONDITION"
    alert_policy_simple_default                         = "true"
    
    alert_channel                                       = "true"
    #
    alert_channel_campfire                              = "false"
    #
    alert_channel_email                                 = "false"
    #
    alert_channel_opsgenie                              = "false"
    #
    alert_channel_pagerduty                             = "false"
    # 
    alert_channel_slack                                 = "true"
    alert_channel_slack_configuration_channel           = "new-relic"
    alert_channel_slack_configuration_url               = "https://hooks.slack.com/services/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666"
    # 
    alert_channel_user                                  = "false"
    # 
    alert_channel_victorops                             = "false"
    #
    alert_channel_webhook                               = "false"
    #
    alert_condition                                     = "true"     
    alert_condition_policy_id                           = "${element(module.newrelic_alert.simple_default_alert_policy_id, 0)}"
    alert_condition_type                                = "apm_app_metric"
    alert_condition_entities                            = ["PHP Application"]

    alert_policy_channel                                = "false"
}

В файле стоит прописать все необходимое, но самое главное:

api_key — API ключ, который можно сгенерировать в «‘Account settings’->’API keys'».

Можно так же еще создать:

$ vim output.tf

И в него вставить что-то типа:

#-----------------------------------------------------------
#newrelic_alert_channel 
#-----------------------------------------------------------
output "nr_alert_channel_slack" {
    description = "ID for newrelic_alert_channel_slack"
    value       = "${module.newrelic_alert.alert_channel_slack_id}"
}

#-----------------------------------------------------------
# newrelic_alert_policy
#-----------------------------------------------------------
output "nr_alert_policy_id" {
    description = "ID of the policy."
    value       = "${module.newrelic_alert.alert_policy_id}"
}

output "nr_simple_default_alert_policy_id" {
    description = "ID of the policy for simple_default"
    value       = "${module.newrelic_alert.simple_default_alert_policy_id}"
}

Все уже написано и готово к использованию. Ну что, начнем тестирование. В папке с вашим плейбуком, выполняем:

$ terraform init

Этим действием я инициализирую проект. Затем, подтягиваю модуль:

$ terraform get

PS: Для обновление изменений в самом модуле, можно выполнять:

$ terraform get -update

Проверим валидацию:

$ terraform validate

Запускем прогон:

$ terraform plan

Мне вывело что все у меня хорошо и можно запускать деплой:

$ terraform apply

Как видно с вывода, — все прошло гладко! Чтобы удалить созданное творение, можно выполнить:

$ terraform destroy

Еще полезности:

Работа с New Relic (newrelic alert) и Terraform в Unix/Linux

Работа с New Relic (newrelic nrql) и Terraform в Unix/Linux

Работа с New Relic (newrelic infra) и Terraform в Unix/Linux

Работа с New Relic (newrelic synthetics) и Terraform в Unix/Linux

Работа с New Relic (newrelic dashboard) и Terraform в Unix/Linux

Весь материал аплоаджу в github аккаунт для удобства использования:

$ git clone https://github.com/SebastianUA/terraform.git

Вот и все на этом. Данная статья «Работа с New Relic (newrelic alert) и Terraform в Unix/Linux» завершена.

The post Работа с New Relic (newrelic alert) и Terraform в Unix/Linux first appeared on linux-notes.org.

Page_c76a3fcf

captain — Thu, 19 Jul 2018 21:29:51 +0000

Работа с Google Cloud Platform (google bigquery) и Terraform в Unix/Linux

Google Cloud Platrorm — это платформа вида «инфраструктура как сервис» (IaaS), позволяющая клиентам создавать, тестировать и развертывать собственные приложения на инфраструктуре Google, в высокопроизводительных виртуальных машинах.

Google Compute Engine предоставляет виртуальные машины, работающие в инновационных центрах обработки данных Google и всемирной сети.

Google BigQuery – это инструмент, обеспечивающий высокую скорость обработки запросов в больших наборах данных.

Установка terraform в Unix/Linux

Установка крайне примитивная и я описал как это можно сделать тут:

Установка terraform в Unix/Linux

Вот еще полезные статьи по GCP + Terrafrom:

Работа с Google Cloud Platform (compute instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute health check) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute target pool) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute forwarding rule) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute firewall) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute disk) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute image) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance template) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance group manager) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute autoscaler) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google kms) и Terraform в Unix/Linux

Работа с Google Cloud Platform (storage bucket) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google pubsub) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google dns) и Terraform в Unix/Linux

Работа с Google Cloud Platform (cloudbuild_trigger) и Terraform в Unix/Linux

Работа с Google Cloud Platform (redis instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (sql database instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (sourcerepo repository) и Terraform в Unix/Linux

Работа с Google Cloud Platform (dataflow job) и Terraform в Unix/Linux

Работа с Google Cloud Platform (endpoints service) и Terraform в Unix/Linux

Работа с Google Cloud Platform (spanner instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google runtimeconfig) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google bigtable) и Terraform в Unix/Linux

Генерация документации для Terraform с Python в Unix/Linux

Чтобы получить помощь по использованию команд, выполните:

$ terraform --help
Usage: terraform [--version] [--help]  [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
    destroy            Destroy Terraform-managed infrastructure
    env                Workspace management
    fmt                Rewrites config files to canonical format
    get                Download and install modules for the configuration
    graph              Create a visual graph of Terraform resources
    import             Import existing infrastructure into Terraform
    init               Initialize a Terraform working directory
    output             Read an output from a state file
    plan               Generate and show an execution plan
    providers          Prints a tree of the providers used in the configuration
    push               Upload this Terraform module to Atlas to run
    refresh            Update local state file against real resources
    show               Inspect Terraform state or plan
    taint              Manually mark a resource for recreation
    untaint            Manually unmark a resource as tainted
    validate           Validates the Terraform files
    version            Prints the Terraform version
    workspace          Workspace management

All other commands:
    debug              Debug output management (experimental)
    force-unlock       Manually unlock the terraform state
    state              Advanced state management

Приступим к использованию!

Работа с Google Cloud Platform (google bigquery) и Terraform в Unix/Linux

Первое что нужно сделать — это настроить «Cloud Identity». С помощью сервиса Google Cloud Identity вы сможете предоставлять доменам, пользователям и аккаунтам в организации доступ к ресурсам Cloud, а также централизованно управлять пользователями и группами через консоль администратора Google.

Полезное чтиво:

Установка Google Cloud SDK/gcloud в Unix/Linux

У меня есть папка terraform, в ней у меня будут лежать провайдеры с которыми я буду работать. Т.к в этом примере я буду использовать google_cloud_platform, то создам данную папку и перейду в нее. Далее, в этой папке, стоит создать:

$ mkdir examples modules

Начнем писать модуль, но для этой задачи, я создам папку:

$  mkdir modules/google_bigquery

Переходим в нее:

$ cd modules/google_bigquery

Открываем файл:

$ vim google_bigquery.tf

В данный файл, вставляем:

#---------------------------------------------------
# Create google bigquery dataset
#---------------------------------------------------
resource "random_integer" "dataset" {
    count   = "${var.enable_bigquery_dataset ? 1 : 0}" 
                    
    min     = 10
    max     = 1024
    keepers = {
        dataset_id = "${var.dataset_id}"
    }
}

resource "google_bigquery_dataset" "bigquery_dataset" {
    count                       = "${var.enable_bigquery_dataset ? 1 : 0}"
    
    dataset_id                  = "${length(var.dataset_id) >0 ? var.dataset_id : "${random_integer.dataset.result}" }"
    friendly_name               = "${length(var.friendly_name) >0 ? var.friendly_name : "${lower(var.name)}-bq-dataset-${lower(var.environment)}" }"
    description                 = "${var.description}"
    project                     = "${var.project}"
    location                    = "${var.location}"
    default_table_expiration_ms = "${var.default_table_expiration_ms}"
    
    labels {
        name            = "${lower(var.name)}-bq-dataset-${lower(var.environment)}"
        dataset_id      = "${length(var.dataset_id) >0 ? var.dataset_id : "${random_integer.dataset.result}" }"
        environment     = "${lower(var.environment)}"
        orchestration   = "${lower(var.orchestration)}"
    }

    lifecycle {
        ignore_changes = ["dataset_id"]
        create_before_destroy = true
    }

    depends_on  = ["random_integer.dataset"] 
}

#---------------------------------------------------
# Create google bigquery table
#---------------------------------------------------
resource "random_integer" "table" {
    count   = "${var.enable_bigquery_table ? 1 : 0}"

    min     = 10
    max     = 1024
    keepers = {
        table_id = "${var.table_id}"
    }
}

resource "google_bigquery_table" "bigquery_table" {
    count           = "${var.enable_bigquery_table ? 1 : 0}"
    
    project         = "${var.project}"
    dataset_id      = "${var.dataset_id}"
    table_id        = "${length(var.table_id) >0 ? var.table_id : "${random_integer.table.result}" }"
    description     = "${var.table_description}"    
    
    expiration_time = "${var.expiration_time}"
    friendly_name   = "${length(var.table_friendly_name) >0 ? var.table_friendly_name : "${lower(var.name)}-bq-table-${lower(var.environment)}" }"
    
    time_partitioning {
        type            = "${var.time_partitioning_type}"
        expiration_ms   = "${var.time_partitioning_expiration_ms}"
        field           = "${var.time_partitioning_field}"
    } 

    #view {
    #    query           = "${var.view_query}"
    #    use_legacy_sql  = "${var.view_use_legacy_sql}"
    #}

    labels {
        name            = "${length(var.table_friendly_name) >0 ? var.table_friendly_name : "${lower(var.name)}-bq-table-${lower(var.environment)}" }"
        table_id        = "${length(var.table_id) >0 ? var.table_id : "${random_integer.table.result}" }"
        environment     = "${lower(var.environment)}"
        orchestration   = "${lower(var.orchestration)}"
    }

    schema = "${file("${path.module}/${var.schema_file}")}"

    lifecycle {
        ignore_changes = ["dataset_id"]
        create_before_destroy = true
    }

    depends_on  = ["random_integer.table"]
}

resource "random_integer" "table_view" {
    count   = "${var.enable_bigquery_table_view ? 1 : 0}"

    min     = 10
    max     = 1024
    keepers = {
        table_id = "${var.table_id}"
    }
}

resource "google_bigquery_table" "bigquery_table_view" {
    count           = "${var.enable_bigquery_table_view ? 1 : 0}"

    project         = "${var.project}"
    dataset_id      = "${var.dataset_id}"
    table_id        = "${length(var.table_id) >0 ? var.table_id : "${random_integer.table_view.result}" }"
    description     = "${var.table_description}"

    expiration_time = "${var.expiration_time}"
    friendly_name   = "${length(var.table_friendly_name) >0 ? var.table_friendly_name : "${lower(var.name)}-bq-table-${lower(var.environment)}" }"

    time_partitioning {
        type            = "${var.time_partitioning_type}"
        expiration_ms   = "${var.time_partitioning_expiration_ms}"
        field           = "${var.time_partitioning_field}"
    }

    view {
        query           = "${var.view_query}"
        use_legacy_sql  = "${var.view_use_legacy_sql}"
    }

    labels {
        name            = "${length(var.table_friendly_name) >0 ? var.table_friendly_name : "${lower(var.name)}-bq-table-${lower(var.environment)}" }"
        table_id        = "${length(var.table_id) >0 ? var.table_id : "${random_integer.table_view.result}" }"
        environment     = "${lower(var.environment)}"
        orchestration   = "${lower(var.orchestration)}"
    }

    schema = "${file("${path.module}/${var.schema_file}")}"

    lifecycle {
        ignore_changes = ["dataset_id"]
        create_before_destroy = true
    }

    depends_on  = ["random_integer.table_view"]
}

Открываем файл:

$ vim variables.tf

И прописываем:

variable "name" {
    description = "A unique name for the resource, required by GCE. Changing this forces a new resource to be created."
    default     = "TEST"
}

variable "zone" {
  description = "The zone that the machine should be created in"
  default     = "us-east1-b"    
} 

variable "environment" {
    description = "Environment for service"
    default     = "STAGE"
}

variable "orchestration" {
    description = "Type of orchestration"
    default     = "Terraform"
}

variable "enable_bigquery_dataset" {
    description = "Enable bigquery dataset usage"
    default     = "true"
}

variable "dataset_id" {
    description = "(Required) A unique ID for the resource. Changing this forces a new resource to be created."
    default     = ""
}

variable "friendly_name" {
    description = "(Optional) A descriptive name for the dataset."
    default     = ""
}

variable "description" {
    description = "(Optional) A user-friendly description of the dataset."
    default     = ""
}

variable "project" {
    description = "(Optional) The ID of the project in which the resource belongs. If it is not provided, the provider project is used."
    default     = ""
}

variable "location" {
    description = "(Optional) The geographic location where the dataset should reside. See official docs. There are two types of locations, regional or multi-regional. A regional location is a specific geographic place, such as Tokyo, and a multi-regional location is a large geographic area, such as the United States, that contains at least two geographic places. Possible regional values include: asia-northeast1 Possible multi-regional values:EU and US. The default value is multi-regional location US. Changing this forces a new resource to be created."
    default     = "US"
}

variable "default_table_expiration_ms" {
    description = "(Optional) The default lifetime of all tables in the dataset, in milliseconds. The minimum value is 3600000 milliseconds (one hour). Once this property is set, all newly-created tables in the dataset will have an expirationTime property set to the creation time plus the value in this property, and changing the value will only affect new tables, not existing ones. When the expirationTime for a given table is reached, that table will be deleted automatically. If a table's expirationTime is modified or removed before the table expires, or if you provide an explicit expirationTime when creating a table, that value takes precedence over the default expiration time indicated by this property."
    default     = "3600000"
}

variable "enable_bigquery_table" {
    description = "Enable bigquery table usage"
    default     = "false"
}

variable "table_id" {
    description = "(Required) A unique ID for the resource. Changing this forces a new resource to be created."
    default     = ""
}

variable "table_description" {
    description = "(Optional) The field description."
    default     = ""
}

variable "expiration_time" {
    description = "(Optional) The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed."
    default     = "0"
}

variable "table_friendly_name" {
    description = "(Optional) A descriptive name for the table."
    default     = ""
}

variable "time_partitioning_type" {
    description = "(Required) The only type supported is DAY, which will generate one partition per day based on data loading time."
    default     = "DAY"
}
    
variable "time_partitioning_expiration_ms" {
    description = "(Optional) Number of milliseconds for which to keep the storage for a partition."
    default     = "0"
}

variable "time_partitioning_field" {
    description = "(Optional) The field used to determine how to create a time-based partition. If time-based partitioning is enabled without this value, the table is partitioned based on the load time."
    default     = ""
}

variable "view_query" {
    description = "(Required) A query that BigQuery executes when the view is referenced."
    default     = ""
}

variable "view_use_legacy_sql" {
    description = "(Optional) Specifies whether to use BigQuery's legacy SQL for this view. The default value is true. If set to false, the view will use BigQuery's standard SQL."
    default     = ""
}

variable "enable_bigquery_table_view" {
    description = "Enable bigquery table view usage"
    default     = "false"
}

variable "schema_file" {
    description = "(Optional) A JSON schema for the table."
    default     = "files/schema.json"
}

Собственно в этом файле храняться все переменные. Спасибо кэп!

Открываем последний файл:

$ vim outputs.tf

И в него вставить нужно следующие строки:

#---------------------------------------------------------------------------
# Google bigquery dataset
#---------------------------------------------------------------------------
output "google_bigquery_dataset_dataset_id" {
    description = "dataset_id of google_bigquery_dataset"
    value       = "${google_bigquery_dataset.bigquery_dataset.*.dataset_id}"
}

output "google_bigquery_dataset_id" {
    description = "ID"
    value       = "${google_bigquery_dataset.bigquery_dataset.*.id}"
}

output "google_bigquery_dataset_etag" {
    description = "etag"
    value       = "${google_bigquery_dataset.bigquery_dataset.*.etag}"
}

output "google_bigquery_dataset_friendly_name" {
    description = "friendly_name"
    value       = "${google_bigquery_dataset.bigquery_dataset.*.friendly_name}"
}

output "google_bigquery_dataset_self_link" {
    description = "self_link"
    value       = "${google_bigquery_dataset.bigquery_dataset.*.self_link}"
}

output "google_bigquery_dataset_creation_time" {
    description = "creation_time"
    value       = "${google_bigquery_dataset.bigquery_dataset.*.creation_time}"
}

#---------------------------------------------------------------------------
# Google bigquery table
#---------------------------------------------------------------------------
output "google_bigquery_table_etag" {
    description = "etag"
    value       = "${google_bigquery_table.bigquery_table.*.etag}"
}

output "google_bigquery_table_self_link" {
    description = "self_link"
    value       = "${google_bigquery_table.bigquery_table.*.self_link}"
}

output "google_bigquery_table_table_id" {
    description = "table_id"
    value       = "${google_bigquery_table.bigquery_table.*.table_id}"
}

output "google_bigquery_table_view_etag" {
    description = "etag"
    value       = "${google_bigquery_table.bigquery_table_view.*.etag}"
}

output "google_bigquery_table_view_self_link" {
    description = "self_link"
    value       = "${google_bigquery_table.bigquery_table_view.*.self_link}"
}

output "google_bigquery_table_view_table_id" {
    description = "table_id"
    value       = "${google_bigquery_table.bigquery_table_view.*.table_id}"
}

Переходим теперь в папку google_cloud_platform/examples и создадим еще одну папку для проверки написанного чуда:

$ mkdir google_bigquery && cd $_

Внутри созданной папки открываем файл:

$ vim main.tf

Вставляем:

#
# MAINTAINER Vitaliy Natarov "vitaliy.natarov@yahoo.com"
#
terraform {
  required_version = "> 0.9.0"
}
provider "google" {
    credentials = "${file("/Users/captain/.config/gcloud/creds/terraform_creds.json")}"
    project     = "terraform-2018"
    region      = "us-east1"
}   
module "google_bigquery" {
    source                              = "../../modules/google_bigquery"
    name                                = "TEST"

    enable_bigquery_dataset             = true
    #
    #enable_bigquery_table               = true
    #dataset_id                          = "668"
    
}

Все уже написано и готово к использованию. Ну что, начнем тестирование. В папке с вашим плейбуком, выполняем:

$ terraform init

Этим действием я инициализирую проект. Затем, подтягиваю модуль:

$ terraform get

PS: Для обновление изменений в самом модуле, можно выполнять:

$ terraform get -update

Проверим валидацию:

$ terraform validate

Запускем прогон:

$ terraform plan

Мне вывело что все у меня хорошо и можно запускать деплой:

$ terraform apply

Как видно с вывода, — все прошло гладко! Чтобы удалить созданное творение, можно выполнить:

$ terraform destroy

Весь материал аплоаджу в github аккаунт для удобства использования:

$ git clone https://github.com/SebastianUA/terraform.git

Вот и все на этом. Данная статья «Работа с Google Cloud Platform (google bigquery) и Terraform в Unix/Linux» завершена.

The post Работа с Google Cloud Platform (google bigquery) и Terraform в Unix/Linux first appeared on linux-notes.org.

Page_c76a3fcf

captain — Wed, 18 Jul 2018 21:55:08 +0000

Работа с Google Cloud Platform (google bigtable) и Terraform в Unix/Linux

BigTable — проприетарная высокопроизводительная база данных, построенная на основе Google File System (GFS), Chubby Lock Service и некоторых других продуктах Google. В настоящий момент не распространяется и не используется за пределами Google, хотя Google предлагает использовать её как часть Google App Engine.

Установка terraform в Unix/Linux

Установка крайне примитивная и я описал как это можно сделать тут:

Установка terraform в Unix/Linux

Вот еще полезные статьи по GCP + Terrafrom:

Работа с Google Cloud Platform (compute instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute health check) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute target pool) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute forwarding rule) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute firewall) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute disk) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute image) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance template) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance group manager) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute autoscaler) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google kms) и Terraform в Unix/Linux

Работа с Google Cloud Platform (storage bucket) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google pubsub) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google dns) и Terraform в Unix/Linux

Работа с Google Cloud Platform (cloudbuild_trigger) и Terraform в Unix/Linux

Работа с Google Cloud Platform (redis instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (sql database instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (sourcerepo repository) и Terraform в Unix/Linux

Работа с Google Cloud Platform (dataflow job) и Terraform в Unix/Linux

Работа с Google Cloud Platform (endpoints service) и Terraform в Unix/Linux

Работа с Google Cloud Platform (spanner instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google runtimeconfig) и Terraform в Unix/Linux

Генерация документации для Terraform с Python в Unix/Linux

Чтобы получить помощь по использованию команд, выполните:

$ terraform --help
Usage: terraform [--version] [--help]  [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
    destroy            Destroy Terraform-managed infrastructure
    env                Workspace management
    fmt                Rewrites config files to canonical format
    get                Download and install modules for the configuration
    graph              Create a visual graph of Terraform resources
    import             Import existing infrastructure into Terraform
    init               Initialize a Terraform working directory
    output             Read an output from a state file
    plan               Generate and show an execution plan
    providers          Prints a tree of the providers used in the configuration
    push               Upload this Terraform module to Atlas to run
    refresh            Update local state file against real resources
    show               Inspect Terraform state or plan
    taint              Manually mark a resource for recreation
    untaint            Manually unmark a resource as tainted
    validate           Validates the Terraform files
    version            Prints the Terraform version
    workspace          Workspace management

All other commands:
    debug              Debug output management (experimental)
    force-unlock       Manually unlock the terraform state
    state              Advanced state management

Приступим к использованию!

Работа с Google Cloud Platform (google bigtable) и Terraform в Unix/Linux

Полезное чтиво:

Установка Google Cloud SDK/gcloud в Unix/Linux

$ mkdir examples modules

Начнем писать модуль, но для этой задачи, я создам папку:

$  mkdir modules/google_bigtable

Переходим в нее:

$ cd modules/google_bigtable

Открываем файл:

$ vim google_bigtable.tf

В данный файл, вставляем:

#---------------------------------------------------
# Create bigtable instance
#---------------------------------------------------
resource "google_bigtable_instance" "bigtable_instance" {
    count           = "${var.enable_bigtable_instance ? 1 :0}"

    name            = "${lower(var.name)}-bt-instance-${lower(var.environment)}"
    display_name    = "${var.display_name}"
    cluster_id      = "${length(var.cluster_id) >0 ? var.cluster_id : "${lower(var.name)}-bt-instance-${lower(var.environment)}" }"
    project         = "${var.project}"
    zone            = "${var.zone}"
    num_nodes       = "${var.num_nodes}"
    instance_type   = "${var.instance_type}"
    storage_type    = "${var.storage_type}"
    
    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

#---------------------------------------------------
#Create bigtable table
#---------------------------------------------------
resource "google_bigtable_table" "bigtable_table" {
    count           = "${var.enable_bigtable_table ? 1 : 0}"
    
    name            = "${length(var.table_name) >0 ? var.table_name : "${lower(var.name)}-bt-table-${lower(var.environment)}" }"
    project         = "${var.project}"
    instance_name   = "${var.instance_name}"
    split_keys      = ["${var.split_keys}"]

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

Открываем файл:

$ vim variables.tf

И прописываем:

variable "name" {
    description = "A unique name for the resource, required by GCE. Changing this forces a new resource to be created."
    default     = "TEST"
}

variable "enable_bigtable_instance" {
  description = "Enable bigtable instance creating"
  default     = "true"    
} 

variable "environment" {
    description = "Environment for service"
    default     = "STAGE"
}

variable "cluster_id" {
    description = "(Required) The ID of the Cloud Bigtable cluster."
    default     = ""
}

variable "zone" {
    description = "(Required) The zone to create the Cloud Bigtable cluster in. Zones that support Bigtable instances are noted on the Cloud Bigtable locations page."
    default     = "us-east1-c"
}

variable "num_nodes" {
    description = "(Optional) The number of nodes in your Cloud Bigtable cluster. Minimum of 3 for a PRODUCTION instance. Cannot be set for a DEVELOPMENT instance."
    default     = "3"
}

variable "instance_type" {
    description = "(Optional) The instance type to create. One of 'DEVELOPMENT' or 'PRODUCTION'. Defaults to 'PRODUCTION'."
    default     = "PRODUCTION"
}

variable "storage_type" {
    description = "(Optional) The storage type to use. One of 'SSD' or 'HDD'. Defaults to 'SSD'."
    default     = "SSD"
}

variable "project" {
    description = "(Optional) The ID of the project in which the resource belongs. If it is not provided, the provider project is used."
    default     = ""
}

variable "display_name" {
    description = "(Optional) The human-readable display name of the Bigtable instance. Defaults to the instance name."
    default     = ""
}

variable "enable_bigtable_table" {
    description = "Enable bigtable table usage"
    default     = "false"
}

variable "table_name" {
    description = "The name of the table."
    default     = ""
}

variable "instance_name" {
    description = "(Required) The name of the Bigtable instance."
    default     = ""
}

variable "split_keys" {
    description = "(Optional) A list of predefined keys to split the table on."
    default     = []
}

Собственно в этом файле храняться все переменные. Спасибо кэп!

Открываем последний файл:

$ vim outputs.tf

И в него вставить нужно следующие строки:

output "google_bigtable_instance_name" {
    description = "Name of google bigtable instance"
    value       = "${google_bigtable_instance.bigtable_instance.*.name}"
}

output "google_bigtable_instance_id" {
    description = "ID"
    value       = "${google_bigtable_instance.bigtable_instance.*.id}"
}

output "google_bigtable_table_name" {
    description = "Name of google_bigtable_table_name"
    value       = "${google_bigtable_table.bigtable_table.*.name}"
}

output "google_bigtable_table_id" {
    description = "ID"
    value       = "${google_bigtable_table.bigtable_table.*.id}"
}

Переходим теперь в папку google_cloud_platform/examples и создадим еще одну папку для проверки написанного чуда:

$ mkdir google_bigtable && cd $_

Внутри созданной папки открываем файл:

$ vim main.tf

Вставляем:

#
# MAINTAINER Vitaliy Natarov "vitaliy.natarov@yahoo.com"
#
terraform {
  required_version = "> 0.9.0"
}
provider "google" {
    credentials = "${file("/Users/captain/.config/gcloud/creds/terraform_creds.json")}"
    project     = "terraform-2018"
    region      = "us-east1"
}   
module "google_bigtable" {
    source                              = "../../modules/google_bigtable"
    name                                = "TEST"

    enable_bigtable_instance            = true
    # table
    #enable_bigtable_table               = true
    #instance_name                       = "test-bt-instance-stage"
    #split_keys                          = ["a", "b", "c"]
}           

module "google_bigtable_table" {
    source                              = "../../modules/google_bigtable"
    name                                = "TEST"

    enable_bigtable_instance            = false
    enable_bigtable_table               = true
    instance_name                       = "${element(module.google_bigtable.google_bigtable_instance_name, 0)}"
    split_keys                          = ["a", "b", "c"]

}

Все уже написано и готово к использованию. Ну что, начнем тестирование. В папке с вашим плейбуком, выполняем:

$ terraform init

Этим действием я инициализирую проект. Затем, подтягиваю модуль:

$ terraform get

PS: Для обновление изменений в самом модуле, можно выполнять:

$ terraform get -update

Проверим валидацию:

$ terraform validate

Запускем прогон:

$ terraform plan

Мне вывело что все у меня хорошо и можно запускать деплой:

$ terraform apply

Как видно с вывода, — все прошло гладко! Чтобы удалить созданное творение, можно выполнить:

$ terraform destroy

Весь материал аплоаджу в github аккаунт для удобства использования:

$ git clone https://github.com/SebastianUA/terraform.git

Вот и все на этом. Данная статья «Работа с Google Cloud Platform (google bigtable) и Terraform в Unix/Linux» завершена.

The post Работа с Google Cloud Platform (google bigtable) и Terraform в Unix/Linux first appeared on linux-notes.org.

Page_c76a3fcf

captain — Wed, 18 Jul 2018 17:46:49 +0000

Работа с Google Cloud Platform (google runtimeconfig) и Terraform в Unix/Linux

Google runtimeconfig — сервис который хранит в себе ключ\значение. Собственно вот такой k/v сторедж. Я пока еще не понял для чего его можно использовать. Возможно дальше станет более понятен резон при использовании других сервисов.

Установка terraform в Unix/Linux

Установка крайне примитивная и я описал как это можно сделать тут:

Установка terraform в Unix/Linux

Вот еще полезные статьи по GCP + Terrafrom:

Работа с Google Cloud Platform (compute instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute health check) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute target pool) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute forwarding rule) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute firewall) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute disk) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute image) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance template) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance group manager) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute autoscaler) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google kms) и Terraform в Unix/Linux

Работа с Google Cloud Platform (storage bucket) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google pubsub) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google dns) и Terraform в Unix/Linux

Работа с Google Cloud Platform (cloudbuild_trigger) и Terraform в Unix/Linux

Работа с Google Cloud Platform (redis instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (sql database instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (sourcerepo repository) и Terraform в Unix/Linux

Работа с Google Cloud Platform (dataflow job) и Terraform в Unix/Linux

Работа с Google Cloud Platform (endpoints service) и Terraform в Unix/Linux

Работа с Google Cloud Platform (spanner instance) и Terraform в Unix/Linux

Генерация документации для Terraform с Python в Unix/Linux

Чтобы получить помощь по использованию команд, выполните:

$ terraform --help
Usage: terraform [--version] [--help]  [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
    destroy            Destroy Terraform-managed infrastructure
    env                Workspace management
    fmt                Rewrites config files to canonical format
    get                Download and install modules for the configuration
    graph              Create a visual graph of Terraform resources
    import             Import existing infrastructure into Terraform
    init               Initialize a Terraform working directory
    output             Read an output from a state file
    plan               Generate and show an execution plan
    providers          Prints a tree of the providers used in the configuration
    push               Upload this Terraform module to Atlas to run
    refresh            Update local state file against real resources
    show               Inspect Terraform state or plan
    taint              Manually mark a resource for recreation
    untaint            Manually unmark a resource as tainted
    validate           Validates the Terraform files
    version            Prints the Terraform version
    workspace          Workspace management

All other commands:
    debug              Debug output management (experimental)
    force-unlock       Manually unlock the terraform state
    state              Advanced state management

Приступим к использованию!

Работа с Google Cloud Platform (google runtimeconfig) и Terraform в Unix/Linux

Полезное чтиво:

Установка Google Cloud SDK/gcloud в Unix/Linux

$ mkdir examples modules

Начнем писать модуль, но для этой задачи, я создам папку:

$  mkdir modules/google_runtimeconfig

Переходим в нее:

$ cd modules/google_runtimeconfig

Открываем файл:

$ vim google_runtimeconfig.tf

В данный файл, вставляем:

#---------------------------------------------------
# Create runtimeconfig config
#---------------------------------------------------
resource "google_runtimeconfig_config" "runtimeconfig_config" {
    
    name        = "${lower(var.name)}-rc-config-${lower(var.environment)}"
    description = "${var.description}"
    project     = "${var.project}"

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}
#---------------------------------------------------
# Create runtimeconfig variable 
#---------------------------------------------------
resource "google_runtimeconfig_variable" "runtimeconfig_variable_text" {
    count       = "${var.enable_runtimeconfig_variable_text_usage ? 1 : 0}"
    
    parent      = "${var.runtimeconfig_config_name}"
    name        = "${lower(var.name)}-rc-variable-${lower(var.environment)}"
    project     = "${var.project}"
    text        = "${var.text}"

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

resource "google_runtimeconfig_variable" "runtimeconfig_variable_value" {
    count       = "${var.enable_runtimeconfig_variable_value_usage ? 1 : 0}"

    parent      = "${var.runtimeconfig_config_name}"
    name        = "${lower(var.name)}-rc-variable-${lower(var.environment)}"
    project     = "${var.project}"
    value       = "${length(var.value) >0 ? "${base64encode(var.value)}" : "${base64encode(file("${path.module}/${var.value_file}"))}" }"
    
    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

Открываем файл:

$ vim variables.tf

И прописываем:

variable "name" {
    description = "A unique name for the resource, required by GCE. Changing this forces a new resource to be created."
    default     = "TEST"
}

variable "environment" {
    description = "Environment for service"
    default     = "STAGE"
}

variable "description" {
    description = "(Optional) The description to associate with the runtime config."
    default     = ""
}

variable "project" {
    description = "(Optional) The ID of the project in which the resource belongs. If it is not provided, the provider project is used."
    default     = ""
}

variable "enable_runtimeconfig_variable_text_usage" {
    description = "Enable runtimeconfig variable with text usage"
    default     = "false"
}

variable "runtimeconfig_config_name" {
    description = "(Required) The name of the RuntimeConfig resource containing this variable."
    default     = ""
}

variable "text" {
    description = "(Required) The content to associate with the variable. Exactly one of text or variable must be specified. If text is specified, it must be a valid UTF-8 string and less than 4096 bytes in length. If value is specified, it must be base64 encoded and less than 4096 bytes in length."
    default     = ""
}

variable "value" {
    description = "(Required) The content to associate with the variable. Exactly one of text or variable must be specified. If text is specified, it must be a valid UTF-8 string and less than 4096 bytes in length. If value is specified, it must be base64 encoded and less than 4096 bytes in length."
    default     = ""
}

variable "enable_runtimeconfig_variable_value_usage" {
    description = "Enable runtimeconfig variable with value usage"
    default     = "false"
}

variable "value_file" {
    description = "Path to file with value data"
    default     = "files/file_name.dat"
}

Собственно в этом файле храняться все переменные. Спасибо кэп!

Открываем последний файл:

$ vim outputs.tf

И в него вставить нужно следующие строки:

output "google_runtimeconfig_config_name" {
    description = "Name of google_runtimeconfig_config"
    value       = "${google_runtimeconfig_config.runtimeconfig_config.name}"
}

output "google_runtimeconfig_config_id" {
    description = "ID"
    value       = "${google_runtimeconfig_config.runtimeconfig_config.id}"
}

output "google_runtimeconfig_variable_text_update_time" {
    description = "update_time"
    value       = "${google_runtimeconfig_variable.runtimeconfig_variable_text.*.update_time}"
}

output "google_runtimeconfig_variable_value_update_time" {
    description = "update_time"
    value       = "${google_runtimeconfig_variable.runtimeconfig_variable_value.*.update_time}"
}

Переходим теперь в папку google_cloud_platform/examples и создадим еще одну папку для проверки написанного чуда:

$ mkdir google_runtimeconfig && cd $_

Внутри созданной папки открываем файл:

$ vim main.tf

Вставляем:

#
# MAINTAINER Vitaliy Natarov "vitaliy.natarov@yahoo.com"
#
terraform {
  required_version = "> 0.9.0"
}
provider "google" {
    credentials = "${file("/Users/captain/.config/gcloud/creds/terraform_creds.json")}"
    project     = "terraform-2018"
    region      = "us-east1"
}   
module "google_runtimeconfig" {
    source                                      = "../../modules/google_runtimeconfig"
    name                                        = "TEST"
                                        
    # variable with text
    #enable_runtimeconfig_variable_text_usage    = "true"
    #runtimeconfig_config_name                   = "test-rc-config-stage"
    #text                                        = "linux-notes.org"

    # variable with value
    #enable_runtimeconfig_variable_value_usage   = true
    #runtimeconfig_config_name                   = "test-rc-config-stage"
    #value                                       = "test2"
    #value_file                                  = "files/file_name.dat"
    
}

Все уже написано и готово к использованию. Ну что, начнем тестирование. В папке с вашим плейбуком, выполняем:

$ terraform init

Этим действием я инициализирую проект. Затем, подтягиваю модуль:

$ terraform get

PS: Для обновление изменений в самом модуле, можно выполнять:

$ terraform get -update

Проверим валидацию:

$ terraform validate

Запускем прогон:

$ terraform plan

Мне вывело что все у меня хорошо и можно запускать деплой:

$ terraform apply

Как видно с вывода, — все прошло гладко! Чтобы удалить созданное творение, можно выполнить:

$ terraform destroy

Весь материал аплоаджу в github аккаунт для удобства использования:

$ git clone https://github.com/SebastianUA/terraform.git

Вот и все на этом. Данная статья «Работа с Google Cloud Platform (google runtimeconfig) и Terraform в Unix/Linux» завершена.

The post Работа с Google Cloud Platform (google runtimeconfig) и Terraform в Unix/Linux first appeared on linux-notes.org.

Page_c76a3fcf

captain — Wed, 18 Jul 2018 11:10:14 +0000

Работа с Google Cloud Platform (spanner instance) и Terraform в Unix/Linux

Cloud Spanner – полностью управляемый сервис реляционных баз данных, который обеспечивает согласованность критически важных транзакций на глобальном уровне. Он поддерживает стандартную реляционную семантику (схемы, ACID-транзакции, SQL) и автоматическую синхронную репликацию для обеспечения высокой доступности.

Установка terraform в Unix/Linux

Установка крайне примитивная и я описал как это можно сделать тут:

Установка terraform в Unix/Linux

Вот еще полезные статьи по GCP + Terrafrom:

Работа с Google Cloud Platform (compute instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute health check) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute target pool) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute forwarding rule) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute firewall) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute disk) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute image) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance template) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance group manager) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute autoscaler) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google kms) и Terraform в Unix/Linux

Работа с Google Cloud Platform (storage bucket) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google pubsub) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google dns) и Terraform в Unix/Linux

Работа с Google Cloud Platform (cloudbuild_trigger) и Terraform в Unix/Linux

Работа с Google Cloud Platform (redis instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (sql database instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (sourcerepo repository) и Terraform в Unix/Linux

Работа с Google Cloud Platform (dataflow job) и Terraform в Unix/Linux

Работа с Google Cloud Platform (endpoints service) и Terraform в Unix/Linux

Генерация документации для Terraform с Python в Unix/Linux

Чтобы получить помощь по использованию команд, выполните:

$ terraform --help
Usage: terraform [--version] [--help]  [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
    destroy            Destroy Terraform-managed infrastructure
    env                Workspace management
    fmt                Rewrites config files to canonical format
    get                Download and install modules for the configuration
    graph              Create a visual graph of Terraform resources
    import             Import existing infrastructure into Terraform
    init               Initialize a Terraform working directory
    output             Read an output from a state file
    plan               Generate and show an execution plan
    providers          Prints a tree of the providers used in the configuration
    push               Upload this Terraform module to Atlas to run
    refresh            Update local state file against real resources
    show               Inspect Terraform state or plan
    taint              Manually mark a resource for recreation
    untaint            Manually unmark a resource as tainted
    validate           Validates the Terraform files
    version            Prints the Terraform version
    workspace          Workspace management

All other commands:
    debug              Debug output management (experimental)
    force-unlock       Manually unlock the terraform state
    state              Advanced state management

Приступим к использованию!

Работа с Google Cloud Platform (spanner instance) и Terraform в Unix/Linux

Полезное чтиво:

Установка Google Cloud SDK/gcloud в Unix/Linux

$ mkdir examples modules

Начнем писать модуль, но для этой задачи, я создам папку:

$  mkdir modules/spanner_instance

Переходим в нее:

$ cd modules/spanner_instance

Открываем файл:

$ vim spanner_instance.tf

В данный файл, вставляем:

#---------------------------------------------------
# Create google spanner instance
#---------------------------------------------------
resource "google_spanner_instance" "spanner_instance" {
  
    config          = "${var.config}"
    display_name    = "${length(var.display_name) >0 ? var.display_name : "${lower(var.name)}-si-${lower(var.environment)}" }"
    name            = "${lower(var.name)}-si-${lower(var.environment)}"
    num_nodes       = "${var.num_nodes}"
    project         = "${var.project}"

    labels {
        name            = "${lower(var.name)}-si-${lower(var.environment)}"
        environment     = "${lower(var.environment)}"
        orchestration   = "${lower(var.orchestration)}"
    }

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

#---------------------------------------------------
# Create google spanner iam
#---------------------------------------------------
data "google_iam_policy" "iam_policy" {
  
    binding {
        role = "${var.role}"
        members = ["${var.members}"]
    }
}

resource "google_spanner_instance_iam_policy" "spanner_instance_iam_policy" {
    count       = "${var.enable_spanner_instance_iam_policy ? 1 : 0}"  

    instance    = "${var.instance}"
    policy_data = "${data.google_iam_policy.iam_policy.policy_data}"
                    
    depends_on  = ["data.google_iam_policy.iam_policy"]

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }

}

resource "google_spanner_instance_iam_binding" "spanner_instance_iam_binding" {
    count       = "${var.enable_spanner_instance_iam_binding ? 1 : 0}"  

    instance    = "${var.instance}"
    role        = "${var.role}"

    members     = ["${var.members}"]
    
    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }

}

resource "google_spanner_instance_iam_member" "spanner_instance_iam_member" {
    count       = "${var.enable_spanner_instance_iam_member ? 1 : 0}"

    instance    = "${var.instance}"
    role        = "${var.role}"
    member      = "${element(var.members, 0)}"

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

#---------------------------------------------------
# Create spanner database 
#---------------------------------------------------
resource "google_spanner_database" "spanner_database" {
    count       = "${var.enable_spanner_database ? 1 : 0}"    

    instance    = "${var.instance}"
    name        = "${var.db_name}"
    project     = "${var.project}"
   
    ddl       =  ["${var.ddl}"]

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }

}

#---------------------------------------------------
# Create spanner database iam
#---------------------------------------------------
data "google_iam_policy" "database_iam_policy" {
    binding {
        role = "${var.role}"
    
        members = ["${var.members}"]
    }
}

resource "google_spanner_database_iam_policy" "spanner_database_iam_policy" {
    count           = "${var.enable_spanner_database_iam_policy ? 1 : 0}"

    instance        = "${var.instance}"
    database        = "${var.db_name}"
    policy_data     = "${data.google_iam_policy.database_iam_policy.policy_data}"

    depends_on      = ["data.google_iam_policy.database_iam_policy"]

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

resource "google_spanner_database_iam_binding" "spanner_database_iam_binding" {
    count       = "${var.enable_spanner_database_iam_binding ? 1 : 0}"  

    instance    = "${var.instance}"
    database    = "${var.db_name}"
    role        = "${var.role}"
    
    members     = ["${var.members}"]
    
    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

resource "google_spanner_database_iam_member" "spanner_database_iam_member" {
    count       = "${var.enable_spanner_database_iam_member ? 1 :0}"

    instance    = "${var.instance}"
    database    = "${var.db_name}"
    role        = "${var.role}"
    member      = "${element(var.members, 0)}"

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

Открываем файл:

$ vim variables.tf

И прописываем:

variable "name" {
    description = "A unique name for the resource, required by GCE. Changing this forces a new resource to be created."
    default     = "TEST"
}

variable "environment" {
    description = "Environment for service"
    default     = "STAGE"
}

variable "project" {
    description = "(Optional) The ID of the project in which to look for the instance specified. If it is not provided, the provider project is used."
    default     = ""
}

variable "orchestration" {
    description = "Type of orchestration"
    default     = "Terraform"
}

variable "config" {
    description = "(Required) The name of the instance's configuration (similar but not quite the same as a region) which defines defines the geographic placement and replication of your databases in this instance. It determines where your data is stored. Values are typically of the form regional-europe-west1 , us-central etc. In order to obtain a valid list please consult the Configuration section of the docs."
    default     = "nam3"
}

variable "display_name" {
    description = "(Required) The descriptive name for this instance as it appears in UIs. Can be updated, however should be kept globally unique to avoid confusion."
    default     = ""
}

variable "num_nodes" {
    description = "(Optional, Computed) The number of nodes allocated to this instance. Defaults to 1. This can be updated after creation."
    default     = "1"
}

variable "enable_spanner_instance_iam_policy" {
    description = "Enable spanner instance iam policy"
    default     = "false"
}

variable "role" {
    description = "(Required) The role that should be applied. Only one google_spanner_instance_iam_binding can be used per role. Note that custom roles must be of the format [projects|organizations]/{parent-name}/roles/{role-name}."
    default     = "roles/editor"
}

variable "members" {
    description = "(Required) Identities that will be granted the privilege in role."
    default     = []
}

variable "instance" {
    description = "(Required) The name of the instance."
    default     = ""
}

variable "enable_spanner_instance_iam_binding" {
    description = "Enable spanner instance iam binding"
    default     = "false"
}

variable "enable_spanner_instance_iam_member" {
    description = "Enable spanner instance iam member"
    default     = "false"
}

variable "enable_spanner_database" {
    description = "Enable spanner DB"
    default     = "false"
}

variable "db_name" {
    description = "(Required) The name of the database."
    default     = "db-test"
}

variable "ddl" {
    description = "(Optional) An optional list of DDL statements to run inside the newly created database. Statements can create tables, indexes, etc. These statements execute atomically with the creation of the database: if there is an error in any statement, the database is not created."
    default     = []
}

variable "enable_spanner_database_iam_policy" {
    description = "Enable spanner database iam policy"
    default     = "false"
}

variable "enable_spanner_database_iam_binding" {
    description = "Enable spanner database iam binding"
    default     = "false"
}

variable "enable_spanner_database_iam_member" {
    description = "Enable spanner database iam member"
    default     = "false"
}

Собственно в этом файле храняться все переменные. Спасибо кэп!

Открываем последний файл:

$ vim outputs.tf

И в него вставить нужно следующие строки:

#-------------------------------------------------------------------
# instance
#-------------------------------------------------------------------
output "google_spanner_instance_name" {
    description = "Name of spanner_instance"
    value       = "${google_spanner_instance.spanner_instance.name}"
}

output "google_spanner_instance_state" {
    description = "The current state of the instance."
    value       = "${google_spanner_instance.spanner_instance.state}"
}

output "google_spanner_instance_id" {
    description = "ID"
    value       = "${google_spanner_instance.spanner_instance.id}"
}
#-------------------------------------------------------------------
# IAM
#-------------------------------------------------------------------
output "google_spanner_instance_iam_policy_etag" {
    description = "etag"
    value       = "${google_spanner_instance_iam_policy.spanner_instance_iam_policy.*.etag}"
}

output "google_spanner_instance_iam_binding_etag" {
    description = "etag"
    value       = "${google_spanner_instance_iam_binding.spanner_instance_iam_binding.*.etag}"
}

output "google_spanner_instance_iam_member_etag" {
    description = "etag"
    value       = "${google_spanner_instance_iam_member.spanner_instance_iam_member.*.etag}"
}
#-------------------------------------------------------------------
# DB
#-------------------------------------------------------------------
output "google_spanner_database_name" {
    description = "Name"
    value       = "${google_spanner_database.spanner_database.*.name}"
}

output "google_spanner_database_state" {
    description = "state" 
    value       = "${google_spanner_database.spanner_database.*.state}"
}
#-------------------------------------------------------------------
# DM IAM
#-------------------------------------------------------------------
output "google_spanner_database_iam_policy_etag" {
    description = "Etag"
    value       = "${google_spanner_database_iam_policy.spanner_database_iam_policy.*.etag}"
}

output "google_spanner_database_iam_binding_etag" {
    description = "Etag"
    value       = "${google_spanner_database_iam_binding.spanner_database_iam_binding.*.etag}"
}

output "google_spanner_database_iam_member_etag" {
    description = "Etag"
    value       = "${google_spanner_database_iam_member.spanner_database_iam_member.*.etag}"
}

Переходим теперь в папку google_cloud_platform/examples и создадим еще одну папку для проверки написанного чуда:

$ mkdir spanner_instance && cd $_

Внутри созданной папки открываем файл:

$ vim main.tf

Вставляем:

#
# MAINTAINER Vitaliy Natarov "vitaliy.natarov@yahoo.com"
#
terraform {
  required_version = "> 0.9.0"
}
provider "google" {
    credentials = "${file("/Users/captain/.config/gcloud/creds/terraform_creds.json")}"
    project     = "terraform-2018"
    region      = "us-east1"
}   
module "spanner_instance" {
    source                              = "../../modules/spanner_instance"
    name                                = "TEST"

    num_nodes                           = 1
    
    # spanner instance iam policy
    #enable_spanner_instance_iam_policy  = true
    #instance                            = "test-si-stage"
    #members                             = ["user:solo.metaliSebastian@gmail.com",]
    #role                                = "roles/editor"    
    #
    # spanner instance iam binding
    #enable_spanner_instance_iam_binding = true
    #instance                            = "test-si-stage"
    #members                             = [
    #    "user:solo.metaliSebastian@gmail.com",
    #]
    #role                                = "roles/editor"    
    #
    # spanner instance iam member
    #enable_spanner_instance_iam_member  = true
    #instance                            = "test-si-stage"
    #members                             = [
    #    "user:solo.metaliSebastian@gmail.com",
    #]
    #role                                = "roles/editor"
    #
    # spanner_database
    #enable_spanner_database             = true
    #instance                            = "test-si-stage"
    #db_name                             = "db-test"
    #ddl                                 = [
    #]
    #
    # Create spanner database iam
    #enable_spanner_database_iam_policy  = true
    #instance                            = "test-si-stage"
    #members                             = [
    #    "user:solo.metaliSebastian@gmail.com",
    #]
    #role                                = "roles/editor"
    #
    # spanner_database_iam_binding
    #enable_spanner_database_iam_binding = true
    #instance                            = "test-si-stage"
    #members                             = [
    #    "user:solo.metaliSebastian@gmail.com",
    #]
    #role                                = "roles/editor"
    #
    # spanner_database_iam_member
    #enable_spanner_database_iam_member  = true
    #instance                            = "test-si-stage"
    #members                             = [
    #    "user:solo.metaliSebastian@gmail.com",
    #]
    #role                                = "roles/editor"
}

Все уже написано и готово к использованию. Ну что, начнем тестирование. В папке с вашим плейбуком, выполняем:

$ terraform init

Этим действием я инициализирую проект. Затем, подтягиваю модуль:

$ terraform get

PS: Для обновление изменений в самом модуле, можно выполнять:

$ terraform get -update

Проверим валидацию:

$ terraform validate

Запускем прогон:

$ terraform plan

Мне вывело что все у меня хорошо и можно запускать деплой:

$ terraform apply

Как видно с вывода, — все прошло гладко! Чтобы удалить созданное творение, можно выполнить:

$ terraform destroy

Весь материал аплоаджу в github аккаунт для удобства использования:

$ git clone https://github.com/SebastianUA/terraform.git

Вот и все на этом. Данная статья «Работа с Google Cloud Platform (spanner instance) и Terraform в Unix/Linux» завершена.

The post Работа с Google Cloud Platform (spanner instance) и Terraform в Unix/Linux first appeared on linux-notes.org.

Page_c76a3fcf

captain — Tue, 17 Jul 2018 07:29:21 +0000

Работа с Google Cloud Platform (dataflow job) и Terraform в Unix/Linux

Dataflow job — Создает задание в Dataflow, которое представляет собой реализацию Apache Beam, запущенную в Google Compute Engine. Для получения дополнительной информации смотрите официальную документацию для Beam и Dataflow.

Установка terraform в Unix/Linux

Установка крайне примитивная и я описал как это можно сделать тут:

Установка terraform в Unix/Linux

Вот еще полезные статьи по GCP + Terrafrom:

Работа с Google Cloud Platform (compute instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute health check) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute target pool) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute forwarding rule) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute firewall) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute disk) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute image) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance template) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance group manager) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute autoscaler) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google kms) и Terraform в Unix/Linux

Работа с Google Cloud Platform (storage bucket) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google pubsub) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google dns) и Terraform в Unix/Linux

Работа с Google Cloud Platform (cloudbuild_trigger) и Terraform в Unix/Linux

Работа с Google Cloud Platform (redis instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (sql database instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (sourcerepo repository) и Terraform в Unix/Linux

Генерация документации для Terraform с Python в Unix/Linux

Чтобы получить помощь по использованию команд, выполните:

$ terraform --help
Usage: terraform [--version] [--help]  [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
    destroy            Destroy Terraform-managed infrastructure
    env                Workspace management
    fmt                Rewrites config files to canonical format
    get                Download and install modules for the configuration
    graph              Create a visual graph of Terraform resources
    import             Import existing infrastructure into Terraform
    init               Initialize a Terraform working directory
    output             Read an output from a state file
    plan               Generate and show an execution plan
    providers          Prints a tree of the providers used in the configuration
    push               Upload this Terraform module to Atlas to run
    refresh            Update local state file against real resources
    show               Inspect Terraform state or plan
    taint              Manually mark a resource for recreation
    untaint            Manually unmark a resource as tainted
    validate           Validates the Terraform files
    version            Prints the Terraform version
    workspace          Workspace management

All other commands:
    debug              Debug output management (experimental)
    force-unlock       Manually unlock the terraform state
    state              Advanced state management

Приступим к использованию!

Работа с Google Cloud Platform (dataflow job) и Terraform в Unix/Linux

Полезное чтиво:

Установка Google Cloud SDK/gcloud в Unix/Linux

$ mkdir examples modules

Начнем писать модуль, но для этой задачи, я создам папку:

$  mkdir modules/dataflow_job

Переходим в нее:

$ cd modules/dataflow_job

Открываем файл:

$ vim dataflow_job.tf

В данный файл, вставляем:

#---------------------------------------------------
# Create google dataflow job
#---------------------------------------------------
resource "google_dataflow_job" "dataflow_job" {

    name                = "${lower(var.name)}-df-job-${lower(var.environment)}"    
    project             = "${var.project}"
    zone                = "${var.zone}"

    template_gcs_path   = "${var.template_gcs_path}"
    temp_gcs_location   = "${var.temp_gcs_location}"
    
    max_workers         = "${var.max_workers}"
    on_delete           = "${var.on_delete}"
    
    parameters {
        foo = "bar"
        baz = "qux"
    }
    
    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

Открываем файл:

$ vim variables.tf

И прописываем:

variable "name" {
    description = "A unique name for the resource, required by GCE. Changing this forces a new resource to be created."
    default     = "TEST"
}

variable "environment" {
    description = "Environment for service"
    default     = "STAGE"
}

variable "project" {
    description = "(Optional) The project in which the resource belongs. If it is not provided, the provider project is used."
    default     = "terraform-2018"
}

variable "template_gcs_path" {
    description = "(Required) The GCS path to the Dataflow job template. Expect 'gs:///'. Ex: 'gs://my-bucket/templates/template_file'"
    default     = ""
}

variable "temp_gcs_location" {
    description = "(Required) A writeable location on GCS for the Dataflow job to dump its temporary data. Expect 'gs:///'. Ex: 'gs://my-bucket/tmp_dir'"
    default     = ""
}

variable "zone" {
    description = "(Optional) The zone in which the created job should run. If it is not provided, the provider zone is used."
    default     = "us-east1"
}

variable "max_workers" {
    description = "(Optional) The number of workers permitted to work on the job. More workers may improve processing speed at additional cost."
    default     = "1"
}

variable "on_delete" {
    description = "(Optional) One of 'drain' or 'cancel'. Specifies behavior of deletion during terraform destroy. See above note."
    default     = "cancel"
}

variable "parameters" {
    description = "(Optional) Key/Value pairs to be passed to the Dataflow job (as used in the template)."
    default     = []
}

Собственно в этом файле храняться все переменные. Спасибо кэп!

Открываем последний файл:

$ vim outputs.tf

И в него вставить нужно следующие строки:

output "google_dataflow_job_name" {
    description = "Name of google dataflow job"
    value       = "${google_dataflow_job.dataflow_job.name}"
}

output "google_dataflow_job_state" {
    description = "State"
    value       = "${google_dataflow_job.dataflow_job.state}"
}

output "google_dataflow_job_id" {
    description = "ID"
    value       = "${google_dataflow_job.dataflow_job.id}"
}

Переходим теперь в папку google_cloud_platform/examples и создадим еще одну папку для проверки написанного чуда:

$ mkdir dataflow_job && cd $_

Внутри созданной папки открываем файл:

$ vim main.tf

Вставляем:

#
# MAINTAINER Vitaliy Natarov "vitaliy.natarov@yahoo.com"
#
terraform {
  required_version = "> 0.9.0"
}
provider "google" {
    credentials = "${file("/Users/captain/.config/gcloud/creds/terraform_creds.json")}"
    project     = "terraform-2018"
    region      = "us-east1"
}   
module "dataflow_job" {
    source                              = "../../modules/dataflow_job"
    name                                = "TEST"

    template_gcs_path                   = "gs://terraform-2018/templates/template_file"
    temp_gcs_location                   = "gs://terraform-2018/tmp_dir"
}

Все уже написано и готово к использованию. Ну что, начнем тестирование. В папке с вашим плейбуком, выполняем:

$ terraform init

Этим действием я инициализирую проект. Затем, подтягиваю модуль:

$ terraform get

PS: Для обновление изменений в самом модуле, можно выполнять:

$ terraform get -update

Проверим валидацию:

$ terraform validate

Запускем прогон:

$ terraform plan

Мне вывело что все у меня хорошо и можно запускать деплой:

$ terraform apply

Как видно с вывода, — все прошло гладко! Чтобы удалить созданное творение, можно выполнить:

$ terraform destroy

Весь материал аплоаджу в github аккаунт для удобства использования:

$ git clone https://github.com/SebastianUA/terraform.git

Вот и все на этом. Данная статья «Работа с Google Cloud Platform (dataflow job) и Terraform в Unix/Linux» завершена.

The post Работа с Google Cloud Platform (dataflow job) и Terraform в Unix/Linux first appeared on linux-notes.org.

Page_c76a3fcf

captain — Mon, 16 Jul 2018 20:59:30 +0000

Работа с Google Cloud Platform (sourcerepo repository) и Terraform в Unix/Linux

Google Cloud Source Repositories — это сервис для управления версиями ПО. Больше можно почитать на офичиальном сайте. А сейчас я приведу пример создание терраформ модуля.

Установка terraform в Unix/Linux

Установка крайне примитивная и я описал как это можно сделать тут:

Установка terraform в Unix/Linux

Вот еще полезные статьи по GCP + Terrafrom:

Работа с Google Cloud Platform (compute instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute health check) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute target pool) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute forwarding rule) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute firewall) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute disk) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute image) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance template) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance group manager) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute autoscaler) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google kms) и Terraform в Unix/Linux

Работа с Google Cloud Platform (storage bucket) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google pubsub) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google dns) и Terraform в Unix/Linux

Работа с Google Cloud Platform (cloudbuild_trigger) и Terraform в Unix/Linux

Работа с Google Cloud Platform (redis instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (sql database instance) и Terraform в Unix/Linux

Генерация документации для Terraform с Python в Unix/Linux

Чтобы получить помощь по использованию команд, выполните:

$ terraform --help
Usage: terraform [--version] [--help]  [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
    destroy            Destroy Terraform-managed infrastructure
    env                Workspace management
    fmt                Rewrites config files to canonical format
    get                Download and install modules for the configuration
    graph              Create a visual graph of Terraform resources
    import             Import existing infrastructure into Terraform
    init               Initialize a Terraform working directory
    output             Read an output from a state file
    plan               Generate and show an execution plan
    providers          Prints a tree of the providers used in the configuration
    push               Upload this Terraform module to Atlas to run
    refresh            Update local state file against real resources
    show               Inspect Terraform state or plan
    taint              Manually mark a resource for recreation
    untaint            Manually unmark a resource as tainted
    validate           Validates the Terraform files
    version            Prints the Terraform version
    workspace          Workspace management

All other commands:
    debug              Debug output management (experimental)
    force-unlock       Manually unlock the terraform state
    state              Advanced state management

Приступим к использованию!

Работа с Google Cloud Platform (sourcerepo repository) и Terraform в Unix/Linux

Полезное чтиво:

Установка Google Cloud SDK/gcloud в Unix/Linux

$ mkdir examples modules

Начнем писать модуль, но для этой задачи, я создам папку:

$  mkdir modules/sourcerepo_repository

Переходим в нее:

$ cd modules/sourcerepo_repository

Открываем файл:

$ vim sourcerepo_repository.tf

В данный файл, вставляем:

#---------------------------------------------------
# Create sourcerepo repository
#---------------------------------------------------
resource "google_sourcerepo_repository" "sourcerepo_repository" {

    name    = "${lower(var.name)}-sr-repository-${lower(var.environment)}"
    project = "${var.project}"

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

Открываем файл:

$ vim variables.tf

И прописываем:

variable "name" {
    description = "(Required) The name of the repository that will be created."
    default     = "TEST"
}

variable "project" {
  description = "(Optional) The ID of the project in which the resource belongs. If it is not provided, the provider project is used."
  default     = ""    
} 

variable "environment" {
    description = "Environment for service"
    default     = "STAGE"
}

Собственно в этом файле храняться все переменные. Спасибо кэп!

Открываем последний файл:

$ vim outputs.tf

И в него вставить нужно следующие строки:

output "google_sourcerepo_repository_name" {
    description = "Name of google sourcerepo repository"
    value       = "${google_sourcerepo_repository.sourcerepo_repository.name}"
}

output "google_sourcerepo_repository_url" {
    description = "The url to clone the repository."
    value       = "${google_sourcerepo_repository.sourcerepo_repository.url}"
}

output "google_sourcerepo_repository_size" {
    description = "The size of the repository."
    value       = "${google_sourcerepo_repository.sourcerepo_repository.size}"
}

output "google_sourcerepo_repository_id" {
    description = "ID"
    value       = "${google_sourcerepo_repository.sourcerepo_repository.id}"
}

Переходим теперь в папку google_cloud_platform/examples и создадим еще одну папку для проверки написанного чуда:

$ mkdir sourcerepo_repository && cd $_

Внутри созданной папки открываем файл:

$ vim main.tf

Вставляем:

#
# MAINTAINER Vitaliy Natarov "vitaliy.natarov@yahoo.com"
#
terraform {
  required_version = "> 0.9.0"
}
provider "google" {
    credentials = "${file("/Users/captain/.config/gcloud/creds/terraform_creds.json")}"
    project     = "terraform-2018"
    region      = "us-east1"
}   
module "sourcerepo_repository" {
    source                              = "../../modules/sourcerepo_repository"
    name                                = "TEST"
    
}

Все уже написано и готово к использованию. Ну что, начнем тестирование. В папке с вашим плейбуком, выполняем:

$ terraform init

Этим действием я инициализирую проект. Затем, подтягиваю модуль:

$ terraform get

PS: Для обновление изменений в самом модуле, можно выполнять:

$ terraform get -update

Проверим валидацию:

$ terraform validate

Запускем прогон:

$ terraform plan

Мне вывело что все у меня хорошо и можно запускать деплой:

$ terraform apply

Как видно с вывода, — все прошло гладко! Чтобы удалить созданное творение, можно выполнить:

$ terraform destroy

Весь материал аплоаджу в github аккаунт для удобства использования:

$ git clone https://github.com/SebastianUA/terraform.git

Вот и все на этом. Данная статья «Работа с Google Cloud Platform (sourcerepo repository) и Terraform в Unix/Linux» завершена.

The post Работа с Google Cloud Platform (sourcerepo repository) и Terraform в Unix/Linux first appeared on linux-notes.org.

Page_c76a3fcf

captain — Mon, 16 Jul 2018 18:59:18 +0000

Работа с Google Cloud Platform (sql database instance) и Terraform в Unix/Linux

Sql database instance — сервис от гугл для создания кластера (MySQL или Postgres). Так же поддерживает репликацию.

Установка terraform в Unix/Linux

Установка крайне примитивная и я описал как это можно сделать тут:

Установка terraform в Unix/Linux

Вот еще полезные статьи по GCP + Terrafrom:

Работа с Google Cloud Platform (compute instance) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute health check) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute target pool) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute forwarding rule) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute firewall) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute disk) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute image) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance template) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute instance group manager) и Terraform в Unix/Linux

Работа с Google Cloud Platform (compute autoscaler) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google kms) и Terraform в Unix/Linux

Работа с Google Cloud Platform (storage bucket) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google pubsub) и Terraform в Unix/Linux

Работа с Google Cloud Platform (google dns) и Terraform в Unix/Linux

Работа с Google Cloud Platform (cloudbuild_trigger) и Terraform в Unix/Linux

Работа с Google Cloud Platform (redis instance) и Terraform в Unix/Linux

Генерация документации для Terraform с Python в Unix/Linux

Чтобы получить помощь по использованию команд, выполните:

$ terraform --help
Usage: terraform [--version] [--help]  [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
    destroy            Destroy Terraform-managed infrastructure
    env                Workspace management
    fmt                Rewrites config files to canonical format
    get                Download and install modules for the configuration
    graph              Create a visual graph of Terraform resources
    import             Import existing infrastructure into Terraform
    init               Initialize a Terraform working directory
    output             Read an output from a state file
    plan               Generate and show an execution plan
    providers          Prints a tree of the providers used in the configuration
    push               Upload this Terraform module to Atlas to run
    refresh            Update local state file against real resources
    show               Inspect Terraform state or plan
    taint              Manually mark a resource for recreation
    untaint            Manually unmark a resource as tainted
    validate           Validates the Terraform files
    version            Prints the Terraform version
    workspace          Workspace management

All other commands:
    debug              Debug output management (experimental)
    force-unlock       Manually unlock the terraform state
    state              Advanced state management

Приступим к использованию!

Работа с Google Cloud Platform (sql database instance) и Terraform в Unix/Linux

Полезное чтиво:

Установка Google Cloud SDK/gcloud в Unix/Linux

$ mkdir examples modules

Начнем писать модуль, но для этой задачи, я создам папку:

$  mkdir modules/sql_database

Переходим в нее:

$ cd modules/sql_database

Открываем файл:

$ vim sql_database.tf

В данный файл, вставляем:

#---------------------------------------------------
# Create google sql database instance
#---------------------------------------------------
resource "google_sql_database_instance" "sql_database_instance_mysql" {
    count                            = "${var.database_version !="POSTGRES_9_6" && !var.enable_replication ? var.count_sql_database_instance : 0}"
                                                                                                                                             
    name                             = "${lower(var.name)}-sd-instance-${lower(var.environment)}-${count.index+1}"
    project                          = "${var.project}"                
    database_version                 = "${var.database_version}"
    region                           = "${lookup(var.region, var.database_version)}"
                                                 
    master_instance_name             = "${var.master_instance_name}"
    
    settings {
        tier                        = "${lookup(var.settings_tier, var.database_version)}"
        activation_policy           = "${var.settings_activation_policy}"
        authorized_gae_applications = ["${var.settings_authorized_gae_applications}"]
        availability_type           = "${var.settings_availability_type}"
        crash_safe_replication      = "${var.settings_crash_safe_replication}"
        replication_type            = "${var.settings_replication_type}"
        pricing_plan                = "${var.settings_pricing_plan}"        
                
        user_labels {
            name            = "${lower(var.name)}-sd-instance-${lower(var.environment)}-${count.index+1}"
            environment     = "${lower(var.environment)}"
            orchestration   = "${lower(var.orchestration)}"
        }
        
        database_flags  = ["${var.settings_database_flags}"]
        
        backup_configuration {      
            binary_log_enabled  = "${var.settings_backup_configuration_binary_log_enabled}"
            enabled             = "${var.settings_backup_configuration_enabled}"
            start_time          = "${var.settings_backup_configuration_start_time}"
        }
        
        ip_configuration {
            ipv4_enabled    = "${var.settings_ip_configuration_ipv4_enabled}"
            require_ssl     = "${var.settings_ip_configuration_require_ssl}"
             
            authorized_networks {
                expiration_time = "${var.settings_authorized_networks_expiration_time}"
                name            = "${var.settings_authorized_networks_name}"
                value           = "${var.settings_authorized_networks_value}" 
            }
        }
        
        location_preference {           
            follow_gae_application  = "${var.settings_location_preference_follow_gae_application}"
            zone                    = "${var.settings_location_preference_zone}"
        }
    
        maintenance_window = []
         
    }
    
    timeouts {
        create  = "${var.timeouts_create}"
        update  = "${var.timeouts_update}"
        delete  = "${var.timeouts_delete}"
    }

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

resource "google_sql_database_instance" "sql_database_instance_postgres" {
    count                            = "${var.database_version =="POSTGRES_9_6" && !var.enable_replication ? var.count_sql_database_instance : 0}"

    name                             = "${lower(var.name)}-sd-instance15-${lower(var.environment)}-${count.index+1}"
    project                          = "${var.project}"         
    database_version                 = "${var.database_version}"
    region                           = "${lookup(var.region, var.database_version)}"

    master_instance_name             = "${var.master_instance_name}"

    settings {
        tier                        = "${lookup(var.settings_tier, var.database_version)}"
        activation_policy           = "${var.settings_activation_policy}"
        authorized_gae_applications = ["${var.settings_authorized_gae_applications}"]
        availability_type           = "${var.settings_availability_type}"
        crash_safe_replication      = "${var.settings_crash_safe_replication}"
        replication_type            = "${var.settings_replication_type}"
        
        disk_autoresize             = "${var.settings_disk_autoresize}"
        disk_size                   = "${var.database_version =="POSTGRES_9_6" ? var.settings_disk_size : 0}"
        disk_type                   = "${var.settings_disk_type}"
        
        user_labels {
            name            = "${lower(var.name)}-sd-instance-${lower(var.environment)}-${count.index+1}"
            environment     = "${lower(var.environment)}"
            orchestration   = "${lower(var.orchestration)}"
        }
        
        database_flags  = ["${var.settings_database_flags}"]
        
        backup_configuration {
            binary_log_enabled  = "${var.settings_backup_configuration_binary_log_enabled}"
            enabled             = "${var.settings_backup_configuration_enabled}"
            start_time          = "${var.settings_backup_configuration_start_time}"
        }
        
        ip_configuration {
            ipv4_enabled    = "${var.settings_ip_configuration_ipv4_enabled}"
            require_ssl     = "${var.settings_ip_configuration_require_ssl}"
        
            authorized_networks {
                expiration_time = "${var.settings_authorized_networks_expiration_time}"
                name            = "${var.settings_authorized_networks_name}"
                value           = "${var.settings_authorized_networks_value}"
            }
        }
        
        location_preference {
            follow_gae_application  = "${var.settings_location_preference_follow_gae_application}"
            zone                    = "${var.settings_location_preference_zone}"
        }
        
        maintenance_window = []

    }

    timeouts {
        create  = "${var.timeouts_create}"
        update  = "${var.timeouts_update}"
        delete  = "${var.timeouts_delete}"
    }

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}
#---------------------------------------------------
# Create google sql database instance replication
#---------------------------------------------------
resource "google_sql_database_instance" "sql_database_instance_mysql_replication" {
    count                            = "${var.database_version !="POSTGRES_9_6" && var.enable_replication ? var.count_sql_database_instance : 0}"

    name                             = "${lower(var.name)}-sd-instance-${lower(var.environment)}-${count.index+1}"
    project                          = "${var.project}"             
    database_version                 = "${var.database_version}"
    region                           = "${lookup(var.region, var.database_version)}"

    master_instance_name             = "${var.master_instance_name}"

    replica_configuration {
        ca_certificate              = "${var.replica_configuration_ca_certificate}"
        client_certificate          = "${var.replica_configuration_client_certificate}"
        client_key                  = "${var.replica_configuration_client_key}"
        connect_retry_interval      = "${var.replica_configuration_connect_retry_interval}"
        dump_file_path              = "${var.replica_configuration_dump_file_path}"
        failover_target             = "${var.replica_configuration_failover_target}"
        master_heartbeat_period     = "${var.replica_configuration_master_heartbeat_period}"
        username                    = "${var.replica_configuration_username}"
        password                    = "${var.replica_configuration_password}"
        verify_server_certificate   = "${var.replica_configuration_verify_server_certificate}"
    }

    settings {
        tier                        = "${lookup(var.settings_tier, var.database_version)}"
        activation_policy           = "${var.settings_activation_policy}"
        authorized_gae_applications = ["${var.settings_authorized_gae_applications}"]
        availability_type           = "${var.settings_availability_type}"
        crash_safe_replication      = "${var.settings_crash_safe_replication}"
        replication_type            = "${var.settings_replication_type}"
        pricing_plan                = "${var.settings_pricing_plan}"

        user_labels {
            name            = "${lower(var.name)}-sd-instance-${lower(var.environment)}-${count.index+1}"
            environment     = "${lower(var.environment)}"
            orchestration   = "${lower(var.orchestration)}"
        }

        database_flags  = ["${var.settings_database_flags}"]

        backup_configuration {
            binary_log_enabled  = "${var.settings_backup_configuration_binary_log_enabled}"
            enabled             = "${var.settings_backup_configuration_enabled}"
            start_time          = "${var.settings_backup_configuration_start_time}"
        }

        ip_configuration {
            ipv4_enabled    = "${var.settings_ip_configuration_ipv4_enabled}"
            require_ssl     = "${var.settings_ip_configuration_require_ssl}"

            authorized_networks {
                expiration_time = "${var.settings_authorized_networks_expiration_time}"
                name            = "${var.settings_authorized_networks_name}"
                value           = "${var.settings_authorized_networks_value}"
            }
        }

        location_preference {
            follow_gae_application  = "${var.settings_location_preference_follow_gae_application}"
            zone                    = "${var.settings_location_preference_zone}"
        }

        maintenance_window = []

    }

    timeouts {
        create  = "${var.timeouts_create}"
        update  = "${var.timeouts_update}"
        delete  = "${var.timeouts_delete}"
    }

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

resource "google_sql_database_instance" "sql_database_instance_postgres_replication" {
    count                            = "${var.database_version =="POSTGRES_9_6" && var.enable_replication ? var.count_sql_database_instance : 0}"

    name                             = "${lower(var.name)}-sd-instance-${lower(var.environment)}-${count.index+1}"
    project                          = "${var.project}"
    database_version                 = "${var.database_version}"
    region                           = "${lookup(var.region, var.database_version)}"

    master_instance_name             = "${var.master_instance_name}"

    replica_configuration {
        ca_certificate              = "${var.replica_configuration_ca_certificate}"
        client_certificate          = "${var.replica_configuration_client_certificate}"
        client_key                  = "${var.replica_configuration_client_key}"
        connect_retry_interval      = "${var.replica_configuration_connect_retry_interval}"
        dump_file_path              = "${var.replica_configuration_dump_file_path}"
        failover_target             = "${var.replica_configuration_failover_target}"
        master_heartbeat_period     = "${var.replica_configuration_master_heartbeat_period}"
        username                    = "${var.replica_configuration_username}"
        password                    = "${var.replica_configuration_password}"
        verify_server_certificate   = "${var.replica_configuration_verify_server_certificate}"
    }

    settings {
        tier                        = "${lookup(var.settings_tier, var.database_version)}"
        activation_policy           = "${var.settings_activation_policy}"
        authorized_gae_applications = ["${var.settings_authorized_gae_applications}"]
        availability_type           = "${var.settings_availability_type}"
        crash_safe_replication      = "${var.settings_crash_safe_replication}"
        replication_type            = "${var.settings_replication_type}"

        disk_autoresize             = "${var.settings_disk_autoresize}"
        disk_size                   = "${var.database_version =="POSTGRES_9_6" ? var.settings_disk_size : 0}"
        disk_type                   = "${var.settings_disk_type}"

        user_labels {
            name            = "${lower(var.name)}-sd-instance-${lower(var.environment)}-${count.index+1}"
            environment     = "${lower(var.environment)}"
            orchestration   = "${lower(var.orchestration)}"
        }

        database_flags  = ["${var.settings_database_flags}"]

        backup_configuration {
            binary_log_enabled  = "${var.settings_backup_configuration_binary_log_enabled}"
            enabled             = "${var.settings_backup_configuration_enabled}"
            start_time          = "${var.settings_backup_configuration_start_time}"
        }

        ip_configuration {
            ipv4_enabled    = "${var.settings_ip_configuration_ipv4_enabled}"
            require_ssl     = "${var.settings_ip_configuration_require_ssl}"

            authorized_networks {
                expiration_time = "${var.settings_authorized_networks_expiration_time}"
                name            = "${var.settings_authorized_networks_name}"
                value           = "${var.settings_authorized_networks_value}"
            }
        }

        location_preference {
            follow_gae_application  = "${var.settings_location_preference_follow_gae_application}"
            zone                    = "${var.settings_location_preference_zone}"
        }

        maintenance_window = []

    }

    timeouts {
        create  = "${var.timeouts_create}"
        update  = "${var.timeouts_update}"
        delete  = "${var.timeouts_delete}"
    }

    lifecycle {
        ignore_changes = []
        create_before_destroy = true
    }
}

#---------------------------------------------------
# Create google sql database
#---------------------------------------------------
resource "google_sql_database" "sql_database" {
    count       = "${var.enable_sql_database_creating && length(var.sql_database_instance_name) >0 ? 1 : 0}"  

    name        = "${var.sql_database_name}"
    project     = "${var.project}"
    instance    = "${var.sql_database_instance_name}"
    charset     = "${var.sql_database_charset}"
    collation   = "${var.sql_database_collation}"
}

#---------------------------------------------------
# Create google sql user
#---------------------------------------------------
resource "google_sql_user" "sql_user" {
    count       = "${var.enable_sql_user_creating && length(var.sql_database_instance_name) >0 ? 1 : 0}"

    name        = "${var.sql_user_name}"
    project     = "${var.project}"
    instance    = "${var.sql_database_instance_name}"
    host        = "${var.sql_user_host}"
    password    = "${var.sql_user_password}"
}

Открываем файл:

$ vim variables.tf

И прописываем:

variable "name" {
    description = "A unique name for the resource, required by GCE. Changing this forces a new resource to be created."
    default     = "TEST"
}

variable "environment" {
    description = "Environment for service"
    default     = "STAGE"
}

variable "orchestration" {
    description = "Type of orchestration"
    default     = "Terraform"
}

variable "count_sql_database_instance" {
    description = "Counter, which count how many nodes which will be created"
    default     = "1"
}

variable "database_version" {
    description = "(Optional, Default: MYSQL_5_6) The MySQL version to use. Can be MYSQL_5_6, MYSQL_5_7 or POSTGRES_9_6 for second-generation instances, or MYSQL_5_5 or MYSQL_5_6 for first-generation instances. See Second Generation Capabilities for more information. POSTGRES_9_6 support is in Beta."
    default     = "MYSQL_5_6"
}

variable "region" {
    description = "(Required) The region the instance will sit in. Note, first-generation Cloud SQL instance regions do not line up with the Google Compute Engine (GCE) regions, and Cloud SQL is not available in all regions - choose from one of the options listed here. A valid region must be provided to use this resource. If a region is not provided in the resource definition, the provider region will be used instead, but this will be an apply-time error for all first-generation instances and for second-generation instances if the provider region is not supported with Cloud SQL. If you choose not to provide the region argument for this resource, make sure you understand this."
    type        = "map"
    default     = {
        MYSQL_5_6       = "us-east1"
        MYSQL_5_7       = "us-east1"
        POSTGRES_9_6    = "us-east1"
    }
}

variable "timeouts_create" {
    description = "Time to create redis node. Default is 6 minutes. Valid units of time are s, m, h."
    default     = "6m"
}

variable "timeouts_update" {
    description = "Time to update redis node. Default is 4 minutes. Valid units of time are s, m, h."
    default     = "4m"
}

variable "timeouts_delete" {
    description = "Time to delete redis node. Default is 4 minutes. Valid units of time are s, m, h."
    default     = "4m"
}

variable "master_instance_name" { 
    description = "(Optional) The name of the instance that will act as the master in the replication setup. Note, this requires the master to have binary_log_enabled set, as well as existing backups."
    default     = ""
}

variable "project" {
    description = "(Optional) The ID of the project in which the resource belongs. If it is not provided, the provider project is used."
    default     = "terraform-2018"
}

variable "replica_configuration_ca_certificate" {
    description = "(Optional) PEM representation of the trusted CA's x509 certificate."
    default     = ""
}

variable "replica_configuration_client_certificate" {
    description = "(Optional) PEM representation of the slave's x509 certificate."
    default     = ""
}

variable "replica_configuration_client_key" {
    description = "(Optional) PEM representation of the slave's private key. The corresponding public key in encoded in the client_certificate."
    default     = ""
}

variable "replica_configuration_connect_retry_interval" {
    description = "(Optional, Default: 60) The number of seconds between connect retries."
    default     = "60"
}

variable "replica_configuration_dump_file_path" {
    description = "(Optional) Path to a SQL file in GCS from which slave instances are created. Format is gs://bucket/filename."
    default     = ""
}

variable "replica_configuration_failover_target" {
    description = "(Optional) Specifies if the replica is the failover target. If the field is set to true the replica will be designated as a failover replica. If the master instance fails, the replica instance will be promoted as the new master instance."
    default     = ""
}

variable "replica_configuration_master_heartbeat_period" {
    description = "(Optional) Time in ms between replication heartbeats."
    default     = "60000"
}

variable "replica_configuration_password" {
    description = "(Optional) Password for the replication connection."
    default     = ""
}

variable "replica_configuration_username" {
    description = "(Optional) Username for replication connection."
    default     = ""
}

variable "replica_configuration_verify_server_certificate" {
    description = "(Optional) True if the master's common name value is checked during the SSL handshake."
    default     = ""
}

variable "settings_tier" {
    description = "(Required) The machine tier (First Generation) or type (Second Generation) to use. See tiers for more details and supported versions. Postgres supports only shared-core machine types such as db-f1-micro, and custom machine types such as db-custom-2-13312. See the Custom Machine Type Documentation to learn about specifying custom machine types."
    type        = "map"
    default     = {
        MYSQL_5_6       = "D0"
        MYSQL_5_7       = "D0"
        POSTGRES_9_6    = "db-f1-micro"
    }
}

variable "settings_activation_policy" {
    description = "(Optional) This specifies when the instance should be active. Can be either ALWAYS, NEVER or ON_DEMAND."
    default     = "ALWAYS"
}

variable "settings_authorized_gae_applications" {
    description = "(Optional) A list of Google App Engine (GAE) project names that are allowed to access this instance."
    default     = []
}

variable "settings_availability_type" {
    description = "(Optional) This specifies whether a PostgreSQL instance should be set up for high availability (REGIONAL) or single zone (ZONAL)."
    default     = "REGIONAL"
}

variable "settings_crash_safe_replication" {
    description = "(Optional) Specific to read instances, indicates when crash-safe replication flags are enabled."
    default     = ""
}

variable "settings_disk_autoresize" {
    description = "(Optional, Second Generation, Default: true) Configuration to increase storage size automatically."
    default     = "true"
}

variable "settings_disk_size" {
    description = "(Optional, Second Generation, Default: 10) The size of data disk, in GB. Size of a running instance cannot be reduced but can be increased."
    default     = "10"
}

variable "settings_disk_type" {
    description = "(Optional, Second Generation, Default: PD_SSD) The type of data disk: PD_SSD or PD_HDD."
    default     = "PD_SSD"
}

variable "settings_pricing_plan" {
    description = "(Optional, First Generation) Pricing plan for this instance, can be one of PER_USE or PACKAGE."
    default     = "PER_USE"
}

variable "settings_replication_type" {
    description = "(Optional) Replication type for this instance, can be one of ASYNCHRONOUS or SYNCHRONOUS."
    default     = "SYNCHRONOUS"
}

variable "settings_backup_configuration_binary_log_enabled" {
    description = "(Optional) True if binary logging is enabled. If logging is false, this must be as well."
    default     = "false"
}

variable "settings_backup_configuration_enabled" {
    description = "(Optional) True if backup configuration is enabled."
    default     = "true"
}

variable "settings_backup_configuration_start_time" {
    description = "(Optional) HH:MM format time indicating when backup configuration starts."
    default     = "23:00"
}

variable "settings_ip_configuration_ipv4_enabled" {
    description = "(Optional) True if the instance should be assigned an IP address. The IPv4 address cannot be disabled for Second Generation instances."
    default     = "true"
}

variable "settings_ip_configuration_require_ssl" {
    description = "(Optional) True if mysqld should default to REQUIRE X509 for users connecting over IP."
    default     = ""
}

variable "settings_authorized_networks_expiration_time" {
    description = "(Optional) The RFC 3339 formatted date time string indicating when this whitelist expires."
    default     = ""
}

variable "settings_authorized_networks_name" {
    description = "(Optional) A name for this whitelist entry."
    default     = "allow_from_all"
}

variable "settings_authorized_networks_value" {
    description = "(Optional) A CIDR notation IPv4 or IPv6 address that is allowed to access this instance. Must be set even if other two attributes are not for the whitelist to become active."
    default     = "0.0.0.0/0"
}

variable "settings_location_preference_follow_gae_application" {
    description = "(Optional) A GAE application whose zone to remain in. Must be in the same region as this instance."
    default     = ""
}

variable "settings_location_preference_zone" {
    description = "(Optional) The preferred compute engine zone."
    default     = ""
}

variable "settings_maintenance_window" {
    description = "Set maintenance window. Need to set flags (day (1-7), start from Monday; hour (0-23); update_track ((Optional) Receive updates earlier (canary) or later (stable)))."
    default     = []
}

variable "settings_database_flags" {
    description = "Set database flags"
    default     = []
}

variable "enable_replication" {
    description = "Enable replication"
    default     = "false"
}

variable "enable_sql_database_creating" {
    description = "Enable sql DB creating"
    default     = "true"
}

variable "sql_database_name" {
    description = "(Required) The name of the database."
    default     = "test_db"
}

variable "sql_database_instance_name" {
    description = "(Required) The name of containing instance."
    default     = ""
}

variable "sql_database_charset" {
    description = "(Optional) The charset value. See MySQL's Supported Character Sets and Collations and Postgres' Character Set Support for more details and supported values. Postgres databases are in Beta, and have limited charset support; they only support a value of UTF8 at creation time."
    default     = ""
}

variable "sql_database_collation" {
    description = "(Optional) The collation value. See MySQL's Supported Character Sets and Collations and Postgres' Collation Support for more details and supported values. Postgres databases are in Beta, and have limited collation support; they only support a value of en_US.UTF8 at creation time."
    default     = ""
}

variable "enable_sql_user_creating" {
    description = "Enable sql user"
    default     = "true"
}

variable "sql_user_name" {
    description = "(Required) The name of the user. Changing this forces a new resource to be created."
    default     = "test_user"
}

variable "sql_user_host" {
    description = "(Optional) The host the user can connect from. This is only supported for MySQL instances. Don't set this field for PostgreSQL instances. Can be an IP address. Changing this forces a new resource to be created."
    default     = ""
}

variable "sql_user_password" {
    description = "(Optional) The password for the user. Can be updated."
    default     = ""
}

Собственно в этом файле храняться все переменные. Спасибо кэп!

Открываем последний файл:

$ vim outputs.tf

И в него вставить нужно следующие строки:

#--------------------------------------------------------------------------------
# MySQL
#--------------------------------------------------------------------------------
output "sql_database_instance_mysql_name" {
    description = "Name of sql database instance for MYSQL"
    value       = "${google_sql_database_instance.sql_database_instance_mysql.*.name}"
}

output "sql_database_instance_mysql_replication_name" {
    description = "Name of sql database instance for MYSQL replica"
    value       = "${google_sql_database_instance.sql_database_instance_mysql_replication.*.name}"
}

output "sql_database_instance_mysql_self_link" {
    description = "self_link of sql database instance for MYSQL"
    value       = "${google_sql_database_instance.sql_database_instance_mysql.*.self_link}"
}

output "sql_database_instance_mysql_replication_self_link" {
    description = "self_link of sql database instance for mysql replica"
    value       = "${google_sql_database_instance.sql_database_instance_mysql_replication.*.self_link}"
}

output "sql_database_instance_mysql_connection_name" {
    description = "connection_name of sql database instance for MYSQL"
    value       = "${google_sql_database_instance.sql_database_instance_mysql.*.connection_name}"
}

output "sql_database_instance_mysql_replication_connection_name" {
    description = "connection_name of sql database instance for mysql replica"
    value       = "${google_sql_database_instance.sql_database_instance_mysql_replication.*.connection_name}"
}
#--------------------------------------------------------------------------------
# PostGres
#--------------------------------------------------------------------------------

output "sql_database_instance_postgres_name" {
    description = "Name of sql database instance for postgres"
    value       = "${google_sql_database_instance.sql_database_instance_postgres.*.name}"
}

output "sql_database_instance_postgres_replication_name" {
    description = "Name of sql database instance for postgres replica"
    value       = "${google_sql_database_instance.sql_database_instance_postgres_replication.*.name}"
}

output "sql_database_instance_postgres_self_link" {
    description = "self_link of sql database instance for postgres"
    value       = "${google_sql_database_instance.sql_database_instance_postgres.*.self_link}"
}

output "sql_database_instance_postgres_replication_self_link" {
    description = "self_link of sql database instance for postgres replica"
    value       = "${google_sql_database_instance.sql_database_instance_postgres_replication.*.self_link}"
}

output "sql_database_instance_postgres_connection_name" {
    description = "connection_name of sql database instance for postgres"
    value       = "${google_sql_database_instance.sql_database_instance_postgres.*.connection_name}"
}       

output "sql_database_instance_postgres_replication_connection_name" {
    description = "connection_nameof sql database instance for postgres replica"
    value       = "${google_sql_database_instance.sql_database_instance_postgres_replication.*.connection_name}"
}

#--------------------------------------------------------------------------------
# DB
#--------------------------------------------------------------------------------
output "sql_database_name" {
    description = "Name of sql database"
    value       = "${google_sql_database.sql_database.*.name}"
}

output "sql_database_self_link" {
    description = "self_link for DB"
    value       = "${google_sql_database.sql_database.*.self_link}"
}

#--------------------------------------------------------------------------------
# User
#--------------------------------------------------------------------------------
output "sql_user_name" {
    description = "User"
    value       = "${google_sql_user.sql_user.*.name}"
}

Переходим теперь в папку google_cloud_platform/examples и создадим еще одну папку для проверки написанного чуда:

$ mkdir sql_database && cd $_

Внутри созданной папки открываем файл:

$ vim main.tf

Вставляем:

#
# MAINTAINER Vitaliy Natarov "vitaliy.natarov@yahoo.com"
#
terraform {
  required_version = "> 0.9.0"
}
provider "google" {
    credentials = "${file("/Users/captain/.config/gcloud/creds/terraform_creds.json")}"
    project     = "terraform-2018"
    region      = "us-east1"
}   
module "sql_database" {
    source                              = "../../modules/sql_database"
    name                                = "TEST"

    #MySQL without replication
    enable_replication                  = false
    database_version                    = "MYSQL_5_6"
    settings_database_flags             = [
        {    
            name    = "skip_show_database"
            value   = "on"
        }
    ]
    #settings_maintenance_window         = []
    #
    #MySQL with replication
    #enable_replication                  = true
    #database_version                    = "MYSQL_5_6"    
    #settings_database_flags             = [
    #    {
    #        name    = "skip_show_database"
    #        value   = "on"
    #    }
    #]
    #settings_maintenance_window         = []
    #
    # Postgres without replication
    #enable_replication                  = false
    #database_version                    = "POSTGRES_9_6"
    #settings_database_flags             = []
    #settings_maintenance_window         = [
    #    {
    #        day             = "6"
    #        hour            = "23"
    #        update_track    = ""
    #    }
    #]
    # Postgres with replication
    #enable_replication                  = true
    #database_version                    = "POSTGRES_9_6"
    #settings_database_flags             = []
    #settings_maintenance_window         = [
    #    {
    #        day             = "6"
    #        hour            = "23"
    #        update_track    = ""
    #    }
    #]

    # DB
    sql_database_instance_name           = "test-sd-instance-stage-1"
    enable_sql_database_creating         = true
    enable_sql_user_creating             = true
    
}

Все уже написано и готово к использованию. Ну что, начнем тестирование. В папке с вашим плейбуком, выполняем:

$ terraform init

Этим действием я инициализирую проект. Затем, подтягиваю модуль:

$ terraform get

PS: Для обновление изменений в самом модуле, можно выполнять:

$ terraform get -update

Проверим валидацию:

$ terraform validate

Запускем прогон:

$ terraform plan

Мне вывело что все у меня хорошо и можно запускать деплой:

$ terraform apply

Как видно с вывода, — все прошло гладко! Чтобы удалить созданное творение, можно выполнить:

$ terraform destroy

Весь материал аплоаджу в github аккаунт для удобства использования:

$ git clone https://github.com/SebastianUA/terraform.git

Вот и все на этом. Данная статья «Работа с Google Cloud Platform (redis instance) и Terraform в Unix/Linux» завершена.

The post Работа с Google Cloud Platform (sql database instance) и Terraform в Unix/Linux first appeared on linux-notes.org.